KVM: x86: hyper-v: XMM fast hypercalls fixes
Subsystems (4)
Vitaly Kuznetsov
maintainercover KVM: x86: hyper-v: XMM fast hypercalls fixes
While working on some Hyper-V TLB flush improvements and Direct TLB flush feature for Hyper-V on KVM I experienced Windows Server 2019 crashes on boot when XMM fast hypercall input feature is advertised. Turns out, HVCALL_SEND_IPI_EX is also an XMM fast hypercall and returning an error kills the guest. This is fixed in PATCH4. PATCH3 fixes erroneous capping of sparse CPU banks for XMM fast TLB flush hypercalls. The problem should be reproducible with >360 vCPUs.
Vitaly Kuznetsov (4):
KVM: x86: hyper-v: Drop redundant 'ex' parameter from
kvm_hv_send_ipi()
KVM: x86: hyper-v: Drop redundant 'ex' parameter from
kvm_hv_flush_tlb() KVM: x86: hyper-v: Fix the maximum number of sparse banks for XMM fast
TLB flush hypercalls
KVM: x86: hyper-v: HVCALL_SEND_IPI_EX is an XMM fast hypercall
arch/x86/kvm/hyperv.c | 84 +++++++++++++++++++++++-------------------- 1 file changed, 45 insertions(+), 39 deletions(-)
Signature · 1 line
2.35.1
Vitaly Kuznetsov
maintainerPATCH 1/4 KVM: x86: hyper-v: Drop redundant 'ex' parameter from kvm_hv_send_ipi()
'struct kvm_hv_hcall' has all the required information already, there's no need to pass 'ex' additionally.
No functional change intended.
--- arch/x86/kvm/hyperv.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
arch/x86/kvm/hyperv.c +4 -4
@@ -1875,7 +1875,7 @@ static void kvm_send_ipi_to_many(struct kvm *kvm, u32 vector, } } -static u64 kvm_hv_send_ipi (struct kvm_vcpu *vcpu,struct kvm_hv_hcall *hc,bool ex) +static u64 kvm_hv_send_ipi (struct kvm_vcpu *vcpu,struct kvm_hv_hcall *hc) {struct kvm *kvm = vcpu->kvm;struct hv_send_ipi_ex send_ipi_ex;
@@ -1889,7 +1889,7 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, boolu32 vector;bool all_cpus; -if (!ex) { +if (hc->code ==HVCALL_SEND_IPI ) {if (!hc->fast) {if (unlikely (kvm_read_guest (kvm, hc->ingpa, &send_ipi,sizeof (send_ipi))))
@@ -2279,14 +2279,14 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu) ret =HV_STATUS_INVALID_HYPERCALL_INPUT ;break ; } - ret =kvm_hv_send_ipi (vcpu, &hc, false); + ret =kvm_hv_send_ipi (vcpu, &hc);break ;case HVCALL_SEND_IPI_EX :if (unlikely (hc.fast || hc.rep)) { ret =HV_STATUS_INVALID_HYPERCALL_INPUT ;break ; } - ret =kvm_hv_send_ipi (vcpu, &hc, true); + ret =kvm_hv_send_ipi (vcpu, &hc);break ;case HVCALL_POST_DEBUG_DATA :case HVCALL_RETRIEVE_DEBUG_DATA :
Signature · 1 line
2.35.1
Re: [PATCH 1/4] KVM: x86: hyper-v: Drop redundant 'ex' parameter from kvm_hv_send_ipi()
Quoted 24 lines · Vitaly Kuznetsov
'struct
kvm_hv_hcall' has all the required information already, there's no need to pass 'ex' additionally.No functional change intended.
Signed-off-by Vitaly Kuznetsov <[email protected]>--- arch/x86/kvm/hyperv.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)arch/x86/kvm/hyperv.c +1
-1@@ -1875,7 +1875,7 @@ static void kvm_send_ipi_to_many(struct kvm *kvm, u32 vector, } } -static u64 kvm_hv_send_ipi (struct kvm_vcpu *vcpu,struct kvm_hv_hcall *hc,bool ex) +static u64 kvm_hv_send_ipi (struct kvm_vcpu *vcpu,struct kvm_hv_hcall *hc) {struct kvm *kvm = vcpu->kvm;struct hv_send_ipi_ex send_ipi_ex;
arch/x86/kvm/hyperv.c static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, bool · line 1892
Quoted 5 lines
arch/x86/kvm/hyperv.c +1
-1@@ -1889,7 +1889,7 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, boolu32 vector;bool all_cpus; -if (!ex) { +if (hc->code ==HVCALL_SEND_IPI ) {
I am thinking, if we already touch this code,
why not to use switch here instead on the hc->code,
so that we can catch this function being called with something else than HVCALL_SEND_IPI_EX
Quoted 10 lines
arch/x86/kvm/hyperv.c +1
-1if (!hc->fast) {if (unlikely (kvm_read_guest (kvm, hc->ingpa, &send_ipi,sizeof (send_ipi))))@@ -2279,14 +2279,14 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu) ret =HV_STATUS_INVALID_HYPERCALL_INPUT ;break ; } - ret =kvm_hv_send_ipi (vcpu, &hc, false); + ret =kvm_hv_send_ipi (vcpu, &hc);break ;
arch/x86/kvm/hyperv.c int kvm_hv_hypercall(struct kvm_vcpu *vcpu) · line 2292
Quoted 10 lines
arch/x86/kvm/hyperv.c +1
-1case HVCALL_SEND_IPI_EX :if (unlikely (hc.fast || hc.rep)) { ret =HV_STATUS_INVALID_HYPERCALL_INPUT ;break ; } - ret =kvm_hv_send_ipi (vcpu, &hc, true); + ret =kvm_hv_send_ipi (vcpu, &hc);break ;case HVCALL_POST_DEBUG_DATA :case HVCALL_RETRIEVE_DEBUG_DATA :
Other than this minor nitpick:
Best regards,
Maxim Levitsky
Vitaly Kuznetsov
maintainerRe: [PATCH 1/4] KVM: x86: hyper-v: Drop redundant 'ex' parameter from kvm_hv_send_ipi()
Quoted 28 lines · Maxim Levitsky
Quoted 27 lines · Vitaly Kuznetsov
'struct
kvm_hv_hcall' has all the required information already, there's no need to pass 'ex' additionally.No functional change intended.
Signed-off-by Vitaly Kuznetsov <[email protected]>--- arch/x86/kvm/hyperv.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)arch/x86/kvm/hyperv.c +1
-1@@ -1875,7 +1875,7 @@ static void kvm_send_ipi_to_many(struct kvm *kvm, u32 vector, } } -static u64 kvm_hv_send_ipi (struct kvm_vcpu *vcpu,struct kvm_hv_hcall *hc,bool ex) +static u64 kvm_hv_send_ipi (struct kvm_vcpu *vcpu,struct kvm_hv_hcall *hc) {struct kvm *kvm = vcpu->kvm;struct hv_send_ipi_ex send_ipi_ex;@@ -1889,7 +1889,7 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, boolu32 vector;bool all_cpus;
Quoted 7 lines
arch/x86/kvm/hyperv.c static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, bool · line 1892
Quoted 2 lines
arch/x86/kvm/hyperv.c +1
-1-if (!ex) { +if (hc->code ==HVCALL_SEND_IPI ) {I am thinking, if we already touch this code,
why not to use switch here instead on thehc->code,
so that we can catch this function being called with something else thanHVCALL_SEND_IPI_EX
I'm not against this second line of defense but kvm_hv_send_ipi() is only called explicitly from kvm_hv_hypercall()'s switch so something is really screwed up if we end up seeing something different from HVCALL_SEND_IPI_EX/HVCALL_SEND_IPI here.
I'm now working on a bigger series for TLB flush improvements, will use your suggestion there, thanks!
Quoted 32 lines
Quoted 11 lines
arch/x86/kvm/hyperv.c +1
-1if (!hc->fast) {if (unlikely (kvm_read_guest (kvm, hc->ingpa, &send_ipi,sizeof (send_ipi))))@@ -2279,14 +2279,14 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu) ret =HV_STATUS_INVALID_HYPERCALL_INPUT ;break ; } - ret =kvm_hv_send_ipi (vcpu, &hc, false); + ret =kvm_hv_send_ipi (vcpu, &hc);break ;arch/x86/kvm/hyperv.c int kvm_hv_hypercall(struct kvm_vcpu *vcpu) · line 2292
Quoted 10 lines
arch/x86/kvm/hyperv.c +1
-1case HVCALL_SEND_IPI_EX :if (unlikely (hc.fast || hc.rep)) { ret =HV_STATUS_INVALID_HYPERCALL_INPUT ;break ; } - ret =kvm_hv_send_ipi (vcpu, &hc, true); + ret =kvm_hv_send_ipi (vcpu, &hc);break ;case HVCALL_POST_DEBUG_DATA :case HVCALL_RETRIEVE_DEBUG_DATA :Other than this minor nitpick:
Reviewed-by Maxim Levitsky <[email protected]>Best regards,
Maxim Levitsky
Signature · 1 line
Vitaly
Re: [PATCH 1/4] KVM: x86: hyper-v: Drop redundant 'ex' parameter from kvm_hv_send_ipi()
In reply to Vitaly KuznetsovQuoted 10 lines · Vitaly Kuznetsov
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
'struct
kvm_hv_hcall' has all the required information already, there's no need to pass 'ex' additionally.No functional change intended.
Signed-off-by Vitaly Kuznetsov <[email protected]>
Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879
Vitaly Kuznetsov
maintainerPATCH 2/4 KVM: x86: hyper-v: Drop redundant 'ex' parameter from kvm_hv_flush_tlb()
In reply to Vitaly Kuznetsov'struct kvm_hv_hcall' has all the required information already, there's no need to pass 'ex' additionally.
No functional change intended.
--- arch/x86/kvm/hyperv.c | 23 ++++++----------------- 1 file changed, 6 insertions(+), 17 deletions(-)
arch/x86/kvm/hyperv.c +6 -17
@@ -1750,7 +1750,7 @@ struct kvm_hv_hcall {sse128_t xmm[HV_HYPERCALL_MAX_XMM_REGISTERS ]; }; -static u64 kvm_hv_flush_tlb (struct kvm_vcpu *vcpu,struct kvm_hv_hcall *hc,bool ex) +static u64 kvm_hv_flush_tlb (struct kvm_vcpu *vcpu,struct kvm_hv_hcall *hc) {int i;gpa_t gpa;
@@ -1765,7 +1765,8 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, boolint sparse_banks_len;bool all_cpus; -if (!ex) { +if (hc->code ==HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST || + hc->code ==HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE ) {if (hc->fast) { flush.address_space = hc->ingpa; flush.flags = hc->outgpa;
@@ -2247,32 +2248,20 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu) kvm_hv_hypercall_complete_userspace;return 0 ;case HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST : -if (unlikely (!hc.rep_cnt || hc.rep_idx)) { - ret =HV_STATUS_INVALID_HYPERCALL_INPUT ; -break ; - } - ret =kvm_hv_flush_tlb (vcpu, &hc, false); -break ; -case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE : -if (unlikely (hc.rep)) { - ret =HV_STATUS_INVALID_HYPERCALL_INPUT ; -break ; - } - ret =kvm_hv_flush_tlb (vcpu, &hc, false); -break ;case HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX :if (unlikely (!hc.rep_cnt || hc.rep_idx)) { ret =HV_STATUS_INVALID_HYPERCALL_INPUT ;break ; } - ret =kvm_hv_flush_tlb (vcpu, &hc, true); + ret =kvm_hv_flush_tlb (vcpu, &hc);break ; +case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE :case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX :if (unlikely (hc.rep)) { ret =HV_STATUS_INVALID_HYPERCALL_INPUT ;break ; } - ret =kvm_hv_flush_tlb (vcpu, &hc, true); + ret =kvm_hv_flush_tlb (vcpu, &hc);break ;case HVCALL_SEND_IPI :if (unlikely (hc.rep)) {
Signature · 1 line
2.35.1
Re: [PATCH 2/4] KVM: x86: hyper-v: Drop redundant 'ex' parameter from kvm_hv_flush_tlb()
Quoted 59 lines · Vitaly Kuznetsov
'struct
kvm_hv_hcall' has all the required information already, there's no need to pass 'ex' additionally.No functional change intended.
Signed-off-by Vitaly Kuznetsov <[email protected]>--- arch/x86/kvm/hyperv.c | 23 ++++++----------------- 1 file changed, 6 insertions(+), 17 deletions(-)arch/x86/kvm/hyperv.c +5
-16@@ -1750,7 +1750,7 @@ struct kvm_hv_hcall {sse128_t xmm[HV_HYPERCALL_MAX_XMM_REGISTERS ]; }; -static u64 kvm_hv_flush_tlb (struct kvm_vcpu *vcpu,struct kvm_hv_hcall *hc,bool ex) +static u64 kvm_hv_flush_tlb (struct kvm_vcpu *vcpu,struct kvm_hv_hcall *hc) {int i;gpa_t gpa;@@ -1765,7 +1765,8 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, boolint sparse_banks_len;bool all_cpus; -if (!ex) { +if (hc->code ==HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST || + hc->code ==HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE ) {if (hc->fast) { flush.address_space = hc->ingpa; flush.flags = hc->outgpa;@@ -2247,32 +2248,20 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu) kvm_hv_hypercall_complete_userspace;return 0 ;case HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST : -if (unlikely (!hc.rep_cnt || hc.rep_idx)) { - ret =HV_STATUS_INVALID_HYPERCALL_INPUT ; -break ; - } - ret =kvm_hv_flush_tlb (vcpu, &hc, false); -break ; -case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE : -if (unlikely (hc.rep)) { - ret =HV_STATUS_INVALID_HYPERCALL_INPUT ; -break ; - } - ret =kvm_hv_flush_tlb (vcpu, &hc, false); -break ;case HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX :if (unlikely (!hc.rep_cnt || hc.rep_idx)) { ret =HV_STATUS_INVALID_HYPERCALL_INPUT ;break ; } - ret =kvm_hv_flush_tlb (vcpu, &hc, true); + ret =kvm_hv_flush_tlb (vcpu, &hc);break ; +case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE :
arch/x86/kvm/hyperv.c int kvm_hv_hypercall(struct kvm_vcpu *vcpu) · line 2267
Quoted 10 lines
arch/x86/kvm/hyperv.c +1
-1case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX :if (unlikely (hc.rep)) { ret =HV_STATUS_INVALID_HYPERCALL_INPUT ;break ; } - ret =kvm_hv_flush_tlb (vcpu, &hc, true); + ret =kvm_hv_flush_tlb (vcpu, &hc);break ;case HVCALL_SEND_IPI :if (unlikely (hc.rep)) {
Best regards,
Maxim Levitsky
Re: [PATCH 2/4] KVM: x86: hyper-v: Drop redundant 'ex' parameter from kvm_hv_flush_tlb()
In reply to Vitaly KuznetsovQuoted 10 lines · Vitaly Kuznetsov
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
'struct
kvm_hv_hcall' has all the required information already, there's no need to pass 'ex' additionally.No functional change intended.
Signed-off-by Vitaly Kuznetsov <[email protected]>
Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879
Vitaly Kuznetsov
maintainerPATCH 3/4 KVM: x86: hyper-v: Fix the maximum number of sparse banks for XMM fast TLB flush hypercalls
In reply to Vitaly KuznetsovWhen TLB flush hypercalls (HVCALL_FLUSH_VIRTUAL_ADDRESS_{LIST,SPACE}_EX are issued in 'XMM fast' mode, the maximum number of allowed sparse_banks is not 'HV_HYPERCALL_MAX_XMM_REGISTERS - 1' (5) but twice as many (10) as each XMM register is 128 bit long and can hold two 64 bit long banks.
--- arch/x86/kvm/hyperv.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
arch/x86/kvm/hyperv.c +2 -1
@@ -1820,7 +1820,8 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)if (!all_cpus) {if (hc->fast) { -if (sparse_banks_len >HV_HYPERCALL_MAX_XMM_REGISTERS -1 ) +/* XMM0 is already consumed, each XMM holds two sparse banks. */ +if (sparse_banks_len >2 * (HV_HYPERCALL_MAX_XMM_REGISTERS -1 ))return HV_STATUS_INVALID_HYPERCALL_INPUT ;for (i =0 ; i < sparse_banks_len; i +=2 ) { sparse_banks[i] =sse128_lo (hc->xmm[i /2 +1 ]);
Signature · 1 line
2.35.1
Re: [PATCH 3/4] KVM: x86: hyper-v: Fix the maximum number of sparse banks for XMM fast TLB flush hypercalls
Quoted 11 lines · Vitaly Kuznetsov
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
When TLB flush hypercalls (HVCALL_FLUSH_VIRTUAL_ADDRESS_{LIST,SPACE}_EX are issued in 'XMM fast' mode, the maximum number of allowed
sparse_banksis not 'HV_HYPERCALL_MAX_XMM_REGISTERS- 1' (5) but twice as many (10) as each XMM register is 128 bit long and can hold two 64 bit long banks.Fixes 5974565bc26d ("KVM: x86: kvm_hv_flush_tlb use inputs from XMM registers")Signed-off-by Vitaly Kuznetsov <[email protected]>
Indeed :)
Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879
Vitaly Kuznetsov
maintainerPATCH 4/4 KVM: x86: hyper-v: HVCALL_SEND_IPI_EX is an XMM fast hypercall
In reply to Vitaly KuznetsovIt has been proven on practice that at least Windows Server 2019 tries using HVCALL_SEND_IPI_EX in 'XMM fast' mode when it has more than 64 vCPUs and it needs to send an IPI to a vCPU > 63. Similarly to other XMM Fast hypercalls (HVCALL_FLUSH_VIRTUAL_ADDRESS_{LIST,SPACE}{,_EX}), this information is missing in TLFS as of 6.0b. Currently, KVM returns an error (HV_STATUS_INVALID_HYPERCALL_INPUT) and Windows crashes.
Note, HVCALL_SEND_IPI is a 'standard' fast hypercall (not 'XMM fast') as all its parameters fit into RDX:R8 and this is handled by KVM correctly.
--- arch/x86/kvm/hyperv.c | 52 ++++++++++++++++++++++++++++--------------- 1 file changed, 34 insertions(+), 18 deletions(-)
arch/x86/kvm/hyperv.c +34 -18
@@ -1890,6 +1890,7 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)int sparse_banks_len;u32 vector;bool all_cpus; +int i;if (hc->code ==HVCALL_SEND_IPI ) {if (!hc->fast) {
@@ -1910,9 +1911,15 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)trace_kvm_hv_send_ipi (vector, sparse_banks[0 ]); }else { -if (unlikely (kvm_read_guest (kvm, hc->ingpa, &send_ipi_ex, -sizeof (send_ipi_ex)))) -return HV_STATUS_INVALID_HYPERCALL_INPUT ; +if (!hc->fast) { +if (unlikely (kvm_read_guest (kvm, hc->ingpa, &send_ipi_ex, +sizeof (send_ipi_ex)))) +return HV_STATUS_INVALID_HYPERCALL_INPUT ; + }else { + send_ipi_ex.vector = (u32 )hc->ingpa; + send_ipi_ex.vp_set.format = hc->outgpa; + send_ipi_ex.vp_set.valid_bank_mask =sse128_lo (hc->xmm[0 ]); + } trace_kvm_hv_send_ipi_ex(send_ipi_ex.vector, send_ipi_ex.vp_set.format,
@@ -1920,8 +1927,7 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc) vector = send_ipi_ex.vector; valid_bank_mask = send_ipi_ex.vp_set.valid_bank_mask; - sparse_banks_len =bitmap_weight (&valid_bank_mask,64 ) * -sizeof (sparse_banks[0 ]); + sparse_banks_len =bitmap_weight (&valid_bank_mask,64 ); all_cpus = send_ipi_ex.vp_set.format ==HV_GENERIC_SET_ALL ;
@@ -1931,12 +1937,27 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)if (!sparse_banks_len) goto ret_success; -if (kvm_read_guest (kvm, - hc->ingpa + offsetof(struct hv_send_ipi_ex , - vp_set.bank_contents), - sparse_banks, - sparse_banks_len)) -return HV_STATUS_INVALID_HYPERCALL_INPUT ; +if (!hc->fast) { +if (kvm_read_guest (kvm, + hc->ingpa + offsetof(struct hv_send_ipi_ex , + vp_set.bank_contents), + sparse_banks, + sparse_banks_len * sizeof(sparse_banks[0]))) +return HV_STATUS_INVALID_HYPERCALL_INPUT ; + }else { +/* +* The lower half of XMM0 is already consumed, each XMM holds +* two sparse banks. +*/ +if (sparse_banks_len > (2 *HV_HYPERCALL_MAX_XMM_REGISTERS -1 )) +return HV_STATUS_INVALID_HYPERCALL_INPUT ; +for (i =0 ; i < sparse_banks_len; i++) { +if (i %2 ) + sparse_banks[i] =sse128_lo (hc->xmm[(i +1 ) /2 ]); +else + sparse_banks[i] =sse128_hi (hc->xmm[i /2 ]); + } + } } check_and_send_ipi:
@@ -2098,6 +2119,7 @@ static bool is_xmm_fast_hypercall(struct kvm_hv_hcall *hc)case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE :case HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX :case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX : +case HVCALL_SEND_IPI_EX :return true; }
@@ -2265,14 +2287,8 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu) ret =kvm_hv_flush_tlb (vcpu, &hc);break ;case HVCALL_SEND_IPI : -if (unlikely (hc.rep)) { - ret =HV_STATUS_INVALID_HYPERCALL_INPUT ; -break ; - } - ret =kvm_hv_send_ipi (vcpu, &hc); -break ;case HVCALL_SEND_IPI_EX : -if (unlikely (hc.fast || hc.rep)) { +if (unlikely (hc.rep)) { ret =HV_STATUS_INVALID_HYPERCALL_INPUT ;break ; }
Signature · 1 line
2.35.1
Re: [PATCH 4/4] KVM: x86: hyper-v: HVCALL_SEND_IPI_EX is an XMM fast hypercall
Quoted 48 lines · Vitaly Kuznetsov
It has been proven on practice that at least Windows Server 2019 tries using
HVCALL_SEND_IPI_EXin 'XMM fast' mode when it has more than 64 vCPUs and it needs to send an IPI to a vCPU > 63. Similarly to other XMM Fast hypercalls (HVCALL_FLUSH_VIRTUAL_ADDRESS_{LIST,SPACE}{,_EX}), this information is missing in TLFS as of 6.0b. Currently, KVM returns an error (HV_STATUS_INVALID_HYPERCALL_INPUT) and Windows crashes.Note,
HVCALL_SEND_IPIis a 'standard' fast hypercall (not 'XMM fast') as all its parameters fit into RDX:R8 and this is handled by KVM correctly.Fixes d8f5537a8816 ("KVM: hyper-v: Advertise support for fast XMM hypercalls")Signed-off-by Vitaly Kuznetsov <[email protected]>--- arch/x86/kvm/hyperv.c | 52 ++++++++++++++++++++++++++++--------------- 1 file changed, 34 insertions(+), 18 deletions(-)arch/x86/kvm/hyperv.c +10
-3@@ -1890,6 +1890,7 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)int sparse_banks_len;u32 vector;bool all_cpus; +int i;if (hc->code ==HVCALL_SEND_IPI ) {if (!hc->fast) {@@ -1910,9 +1911,15 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)trace_kvm_hv_send_ipi (vector, sparse_banks[0 ]); }else { -if (unlikely (kvm_read_guest (kvm, hc->ingpa, &send_ipi_ex, -sizeof (send_ipi_ex)))) -return HV_STATUS_INVALID_HYPERCALL_INPUT ; +if (!hc->fast) { +if (unlikely (kvm_read_guest (kvm, hc->ingpa, &send_ipi_ex, +sizeof (send_ipi_ex)))) +return HV_STATUS_INVALID_HYPERCALL_INPUT ; + }else { + send_ipi_ex.vector = (u32 )hc->ingpa; + send_ipi_ex.vp_set.format = hc->outgpa; + send_ipi_ex.vp_set.valid_bank_mask =sse128_lo (hc->xmm[0 ]); + } trace_kvm_hv_send_ipi_ex(send_ipi_ex.vector, send_ipi_ex.vp_set.format,
arch/x86/kvm/hyperv.c static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc) · line 1930
Quoted 6 lines
arch/x86/kvm/hyperv.c +1
-2@@ -1920,8 +1927,7 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc) vector = send_ipi_ex.vector; valid_bank_mask = send_ipi_ex.vp_set.valid_bank_mask; - sparse_banks_len =bitmap_weight (&valid_bank_mask,64 ) * -sizeof (sparse_banks[0 ]); + sparse_banks_len =bitmap_weight (&valid_bank_mask,64 );
Is this change intentional?
I haven't fully reviewed this, because kvm/queue seem to have a bit different version of this, and I didn't fully follow on all of this.
Best regards,
Maxim Levitsky
Quoted 61 lines
arch/x86/kvm/hyperv.c +23
-13all_cpus = send_ipi_ex.vp_set.format ==HV_GENERIC_SET_ALL ;@@ -1931,12 +1937,27 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)if (!sparse_banks_len) goto ret_success; -if (kvm_read_guest (kvm, - hc->ingpa + offsetof(struct hv_send_ipi_ex , - vp_set.bank_contents), - sparse_banks, - sparse_banks_len)) -return HV_STATUS_INVALID_HYPERCALL_INPUT ; +if (!hc->fast) { +if (kvm_read_guest (kvm, + hc->ingpa + offsetof(struct hv_send_ipi_ex , + vp_set.bank_contents), + sparse_banks, + sparse_banks_len * sizeof(sparse_banks[0]))) +return HV_STATUS_INVALID_HYPERCALL_INPUT ; + }else { +/* +* The lower half of XMM0 is already consumed, each XMM holds +* two sparse banks. +*/ +if (sparse_banks_len > (2 *HV_HYPERCALL_MAX_XMM_REGISTERS -1 )) +return HV_STATUS_INVALID_HYPERCALL_INPUT ; +for (i =0 ; i < sparse_banks_len; i++) { +if (i %2 ) + sparse_banks[i] =sse128_lo (hc->xmm[(i +1 ) /2 ]); +else + sparse_banks[i] =sse128_hi (hc->xmm[i /2 ]); + } + } } check_and_send_ipi:@@ -2098,6 +2119,7 @@ static bool is_xmm_fast_hypercall(struct kvm_hv_hcall *hc)case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE :case HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX :case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX : +case HVCALL_SEND_IPI_EX :return true; }@@ -2265,14 +2287,8 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu) ret =kvm_hv_flush_tlb (vcpu, &hc);break ;case HVCALL_SEND_IPI : -if (unlikely (hc.rep)) { - ret =HV_STATUS_INVALID_HYPERCALL_INPUT ; -break ; - } - ret =kvm_hv_send_ipi (vcpu, &hc); -break ;case HVCALL_SEND_IPI_EX : -if (unlikely (hc.fast || hc.rep)) { +if (unlikely (hc.rep)) { ret =HV_STATUS_INVALID_HYPERCALL_INPUT ;break ; }
Vitaly Kuznetsov
maintainerRe: [PATCH 4/4] KVM: x86: hyper-v: HVCALL_SEND_IPI_EX is an XMM fast hypercall
Quoted 50 lines · Maxim Levitsky
Quoted 49 lines · Vitaly Kuznetsov
It has been proven on practice that at least Windows Server 2019 tries using
HVCALL_SEND_IPI_EXin 'XMM fast' mode when it has more than 64 vCPUs and it needs to send an IPI to a vCPU > 63. Similarly to other XMM Fast hypercalls (HVCALL_FLUSH_VIRTUAL_ADDRESS_{LIST,SPACE}{,_EX}), this information is missing in TLFS as of 6.0b. Currently, KVM returns an error (HV_STATUS_INVALID_HYPERCALL_INPUT) and Windows crashes.Note,
HVCALL_SEND_IPIis a 'standard' fast hypercall (not 'XMM fast') as all its parameters fit into RDX:R8 and this is handled by KVM correctly.Fixes d8f5537a8816 ("KVM: hyper-v: Advertise support for fast XMM hypercalls")Signed-off-by Vitaly Kuznetsov <[email protected]>--- arch/x86/kvm/hyperv.c | 52 ++++++++++++++++++++++++++++--------------- 1 file changed, 34 insertions(+), 18 deletions(-)arch/x86/kvm/hyperv.c +10
-3@@ -1890,6 +1890,7 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)int sparse_banks_len;u32 vector;bool all_cpus; +int i;if (hc->code ==HVCALL_SEND_IPI ) {if (!hc->fast) {@@ -1910,9 +1911,15 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)trace_kvm_hv_send_ipi (vector, sparse_banks[0 ]); }else { -if (unlikely (kvm_read_guest (kvm, hc->ingpa, &send_ipi_ex, -sizeof (send_ipi_ex)))) -return HV_STATUS_INVALID_HYPERCALL_INPUT ; +if (!hc->fast) { +if (unlikely (kvm_read_guest (kvm, hc->ingpa, &send_ipi_ex, +sizeof (send_ipi_ex)))) +return HV_STATUS_INVALID_HYPERCALL_INPUT ; + }else { + send_ipi_ex.vector = (u32 )hc->ingpa; + send_ipi_ex.vp_set.format = hc->outgpa; + send_ipi_ex.vp_set.valid_bank_mask =sse128_lo (hc->xmm[0 ]); + } trace_kvm_hv_send_ipi_ex(send_ipi_ex.vector, send_ipi_ex.vp_set.format,@@ -1920,8 +1927,7 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)
Quoted 7 lines
arch/x86/kvm/hyperv.c static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc) · line 1930
Quoted 5 lines
arch/x86/kvm/hyperv.c +1
-2vector = send_ipi_ex.vector; valid_bank_mask = send_ipi_ex.vp_set.valid_bank_mask; - sparse_banks_len =bitmap_weight (&valid_bank_mask,64 ) * -sizeof (sparse_banks[0 ]); + sparse_banks_len =bitmap_weight (&valid_bank_mask,64 );Is this change intentional?
Yes it is. Previously, 'sparse_banks_len' was the number of bytes to read, now it's in u64-s.
(see below)
Quoted 10 lines
I haven't fully reviewed this, because kvm/queue seem to have a bit different version of this, and I didn't fully follow on all of this.
Quoted 7 lines
arch/x86/kvm/hyperv.c +0
-0all_cpus = send_ipi_ex.vp_set.format ==HV_GENERIC_SET_ALL ;@@ -1931,12 +1937,27 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)if (!sparse_banks_len) goto ret_success;
Quoted 12 lines
arch/x86/kvm/hyperv.c static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc) · line 1945
Quoted 12 lines
arch/x86/kvm/hyperv.c +6
-6-if (kvm_read_guest (kvm, - hc->ingpa + offsetof(struct hv_send_ipi_ex , - vp_set.bank_contents), - sparse_banks, - sparse_banks_len)) -return HV_STATUS_INVALID_HYPERCALL_INPUT ; +if (!hc->fast) { +if (kvm_read_guest (kvm, + hc->ingpa + offsetof(struct hv_send_ipi_ex , + vp_set.bank_contents), + sparse_banks, + sparse_banks_len * sizeof(sparse_banks[0])))
^^^ here ^^^
Quoted 8 lines
arch/x86/kvm/hyperv.c static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc) · line 1953
Quoted 8 lines
arch/x86/kvm/hyperv.c +8
-0+return HV_STATUS_INVALID_HYPERCALL_INPUT ; + }else { +/* +* The lower half of XMM0 is already consumed, each XMM holds +* two sparse banks. +*/ +if (sparse_banks_len > (2 *HV_HYPERCALL_MAX_XMM_REGISTERS -1 )) +return HV_STATUS_INVALID_HYPERCALL_INPUT ;
And here. This is the reason for change: it's more convenient to count it 'xmm halves' than in bytes.
Quoted 36 lines
Quoted 34 lines
arch/x86/kvm/hyperv.c +9
-7+for (i =0 ; i < sparse_banks_len; i++) { +if (i %2 ) + sparse_banks[i] =sse128_lo (hc->xmm[(i +1 ) /2 ]); +else + sparse_banks[i] =sse128_hi (hc->xmm[i /2 ]); + } + } } check_and_send_ipi:@@ -2098,6 +2119,7 @@ static bool is_xmm_fast_hypercall(struct kvm_hv_hcall *hc)case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE :case HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX :case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX : +case HVCALL_SEND_IPI_EX :return true; }@@ -2265,14 +2287,8 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu) ret =kvm_hv_flush_tlb (vcpu, &hc);break ;case HVCALL_SEND_IPI : -if (unlikely (hc.rep)) { - ret =HV_STATUS_INVALID_HYPERCALL_INPUT ; -break ; - } - ret =kvm_hv_send_ipi (vcpu, &hc); -break ;case HVCALL_SEND_IPI_EX : -if (unlikely (hc.fast || hc.rep)) { +if (unlikely (hc.rep)) { ret =HV_STATUS_INVALID_HYPERCALL_INPUT ;break ; }
Signature · 1 line
Vitaly
Re: [PATCH 4/4] KVM: x86: hyper-v: HVCALL_SEND_IPI_EX is an XMM fast hypercall
In reply to Vitaly KuznetsovQuoted 11 lines · Vitaly Kuznetsov
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
It has been proven on practice that at least Windows Server 2019 tries using
HVCALL_SEND_IPI_EXin 'XMM fast' mode when it has more than 64 vCPUs and it needs to send an IPI to a vCPU > 63. Similarly to other XMM Fast hypercalls (HVCALL_FLUSH_VIRTUAL_ADDRESS_{LIST,SPACE}{,_EX}), this information is missing in TLFS as of 6.0b. Currently, KVM returns an error (HV_STATUS_INVALID_HYPERCALL_INPUT) and Windows crashes.
Quoted 5 lines
Note,
HVCALL_SEND_IPIis a 'standard' fast hypercall (not 'XMM fast') as all its parameters fit into RDX:R8 and this is handled by KVM correctly.Fixes d8f5537a8816 ("KVM: hyper-v: Advertise support for fast XMM hypercalls")Signed-off-by Vitaly Kuznetsov <[email protected]>
Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879
Paolo Bonzini
maintainerQuoted 13 lines · Vitaly Kuznetsov
While working on some Hyper-V TLB flush improvements and Direct TLB flush feature for Hyper-V on KVM I experienced Windows Server 2019 crashes on boot when XMM fast hypercall input feature is advertised. Turns out,
HVCALL_SEND_IPI_EXis also an XMM fast hypercall and returning an error kills the guest. This is fixed in PATCH4. PATCH3 fixes erroneous capping of sparse CPU banks for XMM fast TLB flush hypercalls. The problem should be reproducible with >360 vCPUs.Vitaly Kuznetsov (4):
KVM: x86: hyper-v: Drop redundant 'ex' parameter fromkvm_hv_send_ipi() KVM: x86: hyper-v: Drop redundant 'ex' parameter from kvm_hv_flush_tlb()
Quoted 7 lines
KVM: x86: hyper-v: Fix the maximum number of sparse banks for XMM fast
TLB flush hypercalls
KVM: x86: hyper-v:HVCALL_SEND_IPI_EXis an XMM fast hypercallarch/x86/kvm/hyperv.c | 84 +++++++++++++++++++++++-------------------- 1 file changed, 45 insertions(+), 39 deletions(-)
Merging this in 5.18 is a bit messy. Please check that the below patch against kvm/next makes sense:
arch/x86/kvm/hyperv.c +50 -37
@@ -1770,9 +1770,11 @@ struct kvm_hv_hcall { };static u64 kvm_get_sparse_vp_set (struct kvm *kvm,struct kvm_hv_hcall *hc, +int consumed_xmm_halves,u64 *sparse_banks,gpa_t offset) {u16 var_cnt; +int i;if (hc->var_cnt >64 )return -EINVAL ;
@@ -1780,13 +1782,29 @@ static u64 kvm_get_sparse_vp_set(struct kvm *kvm, struct kvm_hv_hcall *hc,/* Ignore banks that cannot possibly contain a legal VP index. */ var_cnt =min_t (u16, hc->var_cnt,KVM_HV_MAX_SPARSE_VCPU_SET_BITS ); +if (hc->fast) { +/* +* Each XMM holds two sparse banks, but do not count halves that +* have already been consumed for hypercall parameters. +*/ +if (hc->var_cnt >2 *HV_HYPERCALL_MAX_XMM_REGISTERS - consumed_xmm_halves) +return HV_STATUS_INVALID_HYPERCALL_INPUT ; +for (i =0 ; i < var_cnt; i++) { +int j = i + consumed_xmm_halves; +if (j %2 ) + sparse_banks[i] =sse128_lo (hc->xmm[j /2 ]); +else + sparse_banks[i] =sse128_hi (hc->xmm[j /2 ]); + } +return 0 ; + } +return kvm_read_guest (kvm, hc->ingpa + offset, sparse_banks, var_cnt *sizeof (*sparse_banks)); } -static u64 kvm_hv_flush_tlb (struct kvm_vcpu *vcpu,struct kvm_hv_hcall *hc,bool ex) +static u64 kvm_hv_flush_tlb (struct kvm_vcpu *vcpu,struct kvm_hv_hcall *hc) { -int i;struct kvm *kvm = vcpu->kvm;struct hv_tlb_flush_ex flush_ex;struct hv_tlb_flush flush;
@@ -1803,7 +1821,8 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, bool*/ BUILD_BUG_ON (KVM_HV_MAX_SPARSE_VCPU_SET_BITS >64 ); -if (!ex) { +if (hc->code ==HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST || + hc->code ==HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE ) {if (hc->fast) { flush.address_space = hc->ingpa; flush.flags = hc->outgpa;
@@ -1859,17 +1878,7 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, boolif (!hc->var_cnt) goto ret_success; -if (hc->fast) { -if (hc->var_cnt >HV_HYPERCALL_MAX_XMM_REGISTERS -1 ) -return HV_STATUS_INVALID_HYPERCALL_INPUT ; -for (i =0 ; i < hc->var_cnt; i +=2 ) { - sparse_banks[i] =sse128_lo (hc->xmm[i /2 +1 ]); - sparse_banks[i +1 ] =sse128_hi (hc->xmm[i /2 +1 ]); - } - goto do_flush; - } - -if (kvm_get_sparse_vp_set (kvm, hc, sparse_banks, +if (kvm_get_sparse_vp_set (kvm, hc,2 , sparse_banks, offsetof(struct hv_tlb_flush_ex , hv_vp_set.bank_contents)))return HV_STATUS_INVALID_HYPERCALL_INPUT ;
@@ -1913,7 +1922,7 @@ static void kvm_send_ipi_to_many(struct kvm *kvm, u32 vector, } } -static u64 kvm_hv_send_ipi (struct kvm_vcpu *vcpu,struct kvm_hv_hcall *hc,bool ex) +static u64 kvm_hv_send_ipi (struct kvm_vcpu *vcpu,struct kvm_hv_hcall *hc) {struct kvm *kvm = vcpu->kvm;struct hv_send_ipi_ex send_ipi_ex;
@@ -1924,7 +1933,7 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, boolu32 vector;bool all_cpus; -if (!ex) { +if (hc->code ==HVCALL_SEND_IPI ) {if (!hc->fast) {if (unlikely (kvm_read_guest (kvm, hc->ingpa, &send_ipi,sizeof (send_ipi))))
@@ -1943,9 +1952,15 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, booltrace_kvm_hv_send_ipi (vector, sparse_banks[0 ]); }else { -if (unlikely (kvm_read_guest (kvm, hc->ingpa, &send_ipi_ex, -sizeof (send_ipi_ex)))) -return HV_STATUS_INVALID_HYPERCALL_INPUT ; +if (!hc->fast) { +if (unlikely (kvm_read_guest (kvm, hc->ingpa, &send_ipi_ex, +sizeof (send_ipi_ex)))) +return HV_STATUS_INVALID_HYPERCALL_INPUT ; + }else { + send_ipi_ex.vector = (u32 )hc->ingpa; + send_ipi_ex.vp_set.format = hc->outgpa; + send_ipi_ex.vp_set.valid_bank_mask =sse128_lo (hc->xmm[0 ]); + } trace_kvm_hv_send_ipi_ex(send_ipi_ex.vector, send_ipi_ex.vp_set.format,
@@ -1964,7 +1979,7 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, boolif (!hc->var_cnt) goto ret_success; -if (kvm_get_sparse_vp_set (kvm, hc, sparse_banks, +if (kvm_get_sparse_vp_set (kvm, hc,1 , sparse_banks, offsetof(struct hv_send_ipi_ex , vp_set.bank_contents)))return HV_STATUS_INVALID_HYPERCALL_INPUT ;
@@ -2126,6 +2141,7 @@ static bool is_xmm_fast_hypercall(struct kvm_hv_hcall *hc)case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE :case HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX :case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX : +case HVCALL_SEND_IPI_EX :return true; }
@@ -2283,46 +2299,43 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu) kvm_hv_hypercall_complete_userspace;return 0 ;case HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST : -if (unlikely (!hc.rep_cnt || hc.rep_idx || hc.var_cnt)) { +if (unlikely (hc.var_cnt)) { ret =HV_STATUS_INVALID_HYPERCALL_INPUT ;break ; } - ret =kvm_hv_flush_tlb (vcpu, &hc, false); -break ; -case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE : -if (unlikely (hc.rep || hc.var_cnt)) { - ret =HV_STATUS_INVALID_HYPERCALL_INPUT ; -break ; - } - ret =kvm_hv_flush_tlb (vcpu, &hc, false); -break ; + fallthrough;case HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX :if (unlikely (!hc.rep_cnt || hc.rep_idx)) { ret =HV_STATUS_INVALID_HYPERCALL_INPUT ;break ; } - ret =kvm_hv_flush_tlb (vcpu, &hc, true); + ret =kvm_hv_flush_tlb (vcpu, &hc);break ; +case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE : +if (unlikely (hc.var_cnt)) { + ret =HV_STATUS_INVALID_HYPERCALL_INPUT ; +break ; + } + fallthrough;case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX :if (unlikely (hc.rep)) { ret =HV_STATUS_INVALID_HYPERCALL_INPUT ;break ; } - ret =kvm_hv_flush_tlb (vcpu, &hc, true); + ret =kvm_hv_flush_tlb (vcpu, &hc);break ;case HVCALL_SEND_IPI : -if (unlikely (hc.rep || hc.var_cnt)) { +if (unlikely (hc.var_cnt)) { ret =HV_STATUS_INVALID_HYPERCALL_INPUT ;break ; } - ret =kvm_hv_send_ipi (vcpu, &hc, false); -break ; + fallthrough;case HVCALL_SEND_IPI_EX : -if (unlikely (hc.fast || hc.rep)) { +if (unlikely (hc.rep)) { ret =HV_STATUS_INVALID_HYPERCALL_INPUT ;break ; } - ret =kvm_hv_send_ipi (vcpu, &hc, true); + ret =kvm_hv_send_ipi (vcpu, &hc);break ;case HVCALL_POST_DEBUG_DATA :case HVCALL_RETRIEVE_DEBUG_DATA :
The resulting merge commit is already in kvm/queue shortly (which should become the next kvm/next as soon as tests complete).
Paolo
Vitaly Kuznetsov
maintainerQuoted 18 lines · Paolo Bonzini
Quoted 17 lines · Vitaly Kuznetsov
While working on some Hyper-V TLB flush improvements and Direct TLB flush feature for Hyper-V on KVM I experienced Windows Server 2019 crashes on boot when XMM fast hypercall input feature is advertised. Turns out,
HVCALL_SEND_IPI_EXis also an XMM fast hypercall and returning an error kills the guest. This is fixed in PATCH4. PATCH3 fixes erroneous capping of sparse CPU banks for XMM fast TLB flush hypercalls. The problem should be reproducible with >360 vCPUs.Vitaly Kuznetsov (4):
KVM: x86: hyper-v: Drop redundant 'ex' parameter fromkvm_hv_send_ipi() KVM: x86: hyper-v: Drop redundant 'ex' parameter from kvm_hv_flush_tlb()KVM: x86: hyper-v: Fix the maximum number of sparse banks for XMM fast
TLB flush hypercalls
KVM: x86: hyper-v:HVCALL_SEND_IPI_EXis an XMM fast hypercall
Quoted 6 lines
Quoted 3 lines
arch/x86/kvm/hyperv.c | 84 +++++++++++++++++++++++-------------------- 1 file changed, 45 insertions(+), 39 deletions(-)Merging this in 5.18 is a bit messy. Please check that the below patch against kvm/next makes sense:
Something is wrong with the diff as it doesn't apply :-(
Quoted 24 lines
arch/x86/kvm/hyperv.c +5
-0@@ -1770,9 +1770,11 @@ struct kvm_hv_hcall { };static u64 kvm_get_sparse_vp_set (struct kvm *kvm,struct kvm_hv_hcall *hc, +int consumed_xmm_halves,u64 *sparse_banks,gpa_t offset) {u16 var_cnt; +int i;if (hc->var_cnt >64 )return -EINVAL ;@@ -1780,13 +1782,29 @@ static u64 kvm_get_sparse_vp_set(struct kvm *kvm, struct kvm_hv_hcall *hc,/* Ignore banks that cannot possibly contain a legal VP index. */ var_cnt =min_t (u16, hc->var_cnt,KVM_HV_MAX_SPARSE_VCPU_SET_BITS ); +if (hc->fast) { +/* +* Each XMM holds two sparse banks, but do not count halves that
arch/x86/kvm/hyperv.c static u64 kvm_get_sparse_vp_set(struct kvm *kvm, struct kvm_hv_hcall *hc, · line 1797
Quoted 10 lines
arch/x86/kvm/hyperv.c +10
-0+* have already been consumed for hypercall parameters. +*/ +if (hc->var_cnt >2 *HV_HYPERCALL_MAX_XMM_REGISTERS - consumed_xmm_halves) +return HV_STATUS_INVALID_HYPERCALL_INPUT ; +for (i =0 ; i < var_cnt; i++) { +int j = i + consumed_xmm_halves; +if (j %2 ) + sparse_banks[i] =sse128_lo (hc->xmm[j /2 ]); +else + sparse_banks[i] =sse128_hi (hc->xmm[j /2 ]);
Let's say we have 1 half of XMM0 consumed. Now:
i = 0;
j = 1;
if (1)
sparse_banks[0] = sse128_lo(hc->xmm[0]);
This doesn't look right as we need to get the upper half of XMM0.
I guess it should be reversed,
if (j % 2)
sparse_banks[i] = sse128_hi(hc->xmm[j / 2]);
else
sparse_banks[i] = sse128_lo(hc->xmm[j / 2]);Quoted 16 lines
arch/x86/kvm/hyperv.c +5
-2+ } +return 0 ; + } +return kvm_read_guest (kvm, hc->ingpa + offset, sparse_banks, var_cnt *sizeof (*sparse_banks)); } -static u64 kvm_hv_flush_tlb (struct kvm_vcpu *vcpu,struct kvm_hv_hcall *hc,bool ex) +static u64 kvm_hv_flush_tlb (struct kvm_vcpu *vcpu,struct kvm_hv_hcall *hc) { -int i;struct kvm *kvm = vcpu->kvm;struct hv_tlb_flush_ex flush_ex;struct hv_tlb_flush flush;
arch/x86/kvm/hyperv.c static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, bool · line 1825
Quoted 6 lines
arch/x86/kvm/hyperv.c +2
-1@@ -1803,7 +1821,8 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, bool*/ BUILD_BUG_ON (KVM_HV_MAX_SPARSE_VCPU_SET_BITS >64 ); -if (!ex) { +if (hc->code ==HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST || + hc->code ==HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE ) {
In case you're trying to come up with a smaller patch for 5.18, we can certainly drop these 'ex'/'non-ex' changes as these are merely cosmetic.
Quoted 11 lines
arch/x86/kvm/hyperv.c +0
-4if (hc->fast) { flush.address_space = hc->ingpa; flush.flags = hc->outgpa;@@ -1859,17 +1878,7 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, boolif (!hc->var_cnt) goto ret_success; -if (hc->fast) { -if (hc->var_cnt >HV_HYPERCALL_MAX_XMM_REGISTERS -1 ) -return HV_STATUS_INVALID_HYPERCALL_INPUT ; -for (i =0 ; i < hc->var_cnt; i +=2 ) {
arch/x86/kvm/hyperv.c static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, bool · line 1883
Quoted 10 lines
arch/x86/kvm/hyperv.c +1
-7- sparse_banks[i] =sse128_lo (hc->xmm[i /2 +1 ]); - sparse_banks[i +1 ] =sse128_hi (hc->xmm[i /2 +1 ]); - } - goto do_flush; - } - -if (kvm_get_sparse_vp_set (kvm, hc, sparse_banks, +if (kvm_get_sparse_vp_set (kvm, hc,2 , sparse_banks, offsetof(struct hv_tlb_flush_ex , hv_vp_set.bank_contents)))
I like your idea to put 'consumed_xmm_halves' into kvm_get_sparse_vp_set() as kvm_hv_flush_tlb is getting too big.
Quoted 106 lines
arch/x86/kvm/hyperv.c +25
-21return HV_STATUS_INVALID_HYPERCALL_INPUT ;@@ -1913,7 +1922,7 @@ static void kvm_send_ipi_to_many(struct kvm *kvm, u32 vector, } } -static u64 kvm_hv_send_ipi (struct kvm_vcpu *vcpu,struct kvm_hv_hcall *hc,bool ex) +static u64 kvm_hv_send_ipi (struct kvm_vcpu *vcpu,struct kvm_hv_hcall *hc) {struct kvm *kvm = vcpu->kvm;struct hv_send_ipi_ex send_ipi_ex;@@ -1924,7 +1933,7 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, boolu32 vector;bool all_cpus; -if (!ex) { +if (hc->code ==HVCALL_SEND_IPI ) {if (!hc->fast) {if (unlikely (kvm_read_guest (kvm, hc->ingpa, &send_ipi,sizeof (send_ipi))))@@ -1943,9 +1952,15 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, booltrace_kvm_hv_send_ipi (vector, sparse_banks[0 ]); }else { -if (unlikely (kvm_read_guest (kvm, hc->ingpa, &send_ipi_ex, -sizeof (send_ipi_ex)))) -return HV_STATUS_INVALID_HYPERCALL_INPUT ; +if (!hc->fast) { +if (unlikely (kvm_read_guest (kvm, hc->ingpa, &send_ipi_ex, +sizeof (send_ipi_ex)))) +return HV_STATUS_INVALID_HYPERCALL_INPUT ; + }else { + send_ipi_ex.vector = (u32 )hc->ingpa; + send_ipi_ex.vp_set.format = hc->outgpa; + send_ipi_ex.vp_set.valid_bank_mask =sse128_lo (hc->xmm[0 ]); + } trace_kvm_hv_send_ipi_ex(send_ipi_ex.vector, send_ipi_ex.vp_set.format,@@ -1964,7 +1979,7 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, boolif (!hc->var_cnt) goto ret_success; -if (kvm_get_sparse_vp_set (kvm, hc, sparse_banks, +if (kvm_get_sparse_vp_set (kvm, hc,1 , sparse_banks, offsetof(struct hv_send_ipi_ex , vp_set.bank_contents)))return HV_STATUS_INVALID_HYPERCALL_INPUT ;@@ -2126,6 +2141,7 @@ static bool is_xmm_fast_hypercall(struct kvm_hv_hcall *hc)case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE :case HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX :case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX : +case HVCALL_SEND_IPI_EX :return true; }@@ -2283,46 +2299,43 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu) kvm_hv_hypercall_complete_userspace;return 0 ;case HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST : -if (unlikely (!hc.rep_cnt || hc.rep_idx || hc.var_cnt)) { +if (unlikely (hc.var_cnt)) { ret =HV_STATUS_INVALID_HYPERCALL_INPUT ;break ; } - ret =kvm_hv_flush_tlb (vcpu, &hc, false); -break ; -case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE : -if (unlikely (hc.rep || hc.var_cnt)) { - ret =HV_STATUS_INVALID_HYPERCALL_INPUT ; -break ; - } - ret =kvm_hv_flush_tlb (vcpu, &hc, false); -break ; + fallthrough;case HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX :if (unlikely (!hc.rep_cnt || hc.rep_idx)) { ret =HV_STATUS_INVALID_HYPERCALL_INPUT ;break ; } - ret =kvm_hv_flush_tlb (vcpu, &hc, true); + ret =kvm_hv_flush_tlb (vcpu, &hc);break ; +case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE : +if (unlikely (hc.var_cnt)) { + ret =HV_STATUS_INVALID_HYPERCALL_INPUT ; +break ; + } + fallthrough;case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX :if (unlikely (hc.rep)) { ret =HV_STATUS_INVALID_HYPERCALL_INPUT ;break ; } - ret =kvm_hv_flush_tlb (vcpu, &hc, true); + ret =kvm_hv_flush_tlb (vcpu, &hc);break ;case HVCALL_SEND_IPI : -if (unlikely (hc.rep || hc.var_cnt)) { +if (unlikely (hc.var_cnt)) { ret =HV_STATUS_INVALID_HYPERCALL_INPUT ;break ; } - ret =kvm_hv_send_ipi (vcpu, &hc, false); -break ; + fallthrough;case HVCALL_SEND_IPI_EX :
arch/x86/kvm/hyperv.c int kvm_hv_hypercall(struct kvm_vcpu *vcpu) · line 2341
Quoted 10 lines
arch/x86/kvm/hyperv.c +2
-2-if (unlikely (hc.fast || hc.rep)) { +if (unlikely (hc.rep)) { ret =HV_STATUS_INVALID_HYPERCALL_INPUT ;break ; } - ret =kvm_hv_send_ipi (vcpu, &hc, true); + ret =kvm_hv_send_ipi (vcpu, &hc);break ;case HVCALL_POST_DEBUG_DATA :case HVCALL_RETRIEVE_DEBUG_DATA :
I've smoke tested this (with the change I've mentioned above) and WS2019 booted with 65 vCPUs. This is a good sign)
Quoted 5 lines
The resulting merge commit is already in kvm/queue shortly (which should become the next kvm/next as soon as tests complete).
I see, please swap sse128_lo()/sse128_hi() there too :-)
Signature · 1 line
Vitaly
Paolo Bonzini
maintainerQuoted 7 lines · Vitaly Kuznetsov
Let's say we have 1 half of XMM0 consumed. Now:
i = 0; j = 1; if (1) sparse_banks[0] = sse128_lo(hc->xmm[0]);
Quoted 8 lines
This doesn't look right as we need to get the upper half of XMM0.
I guess it should be reversed,
if (j % 2) sparse_banks[i] = sse128_hi(hc->xmm[j / 2]); else sparse_banks[i] = sse128_lo(hc->xmm[j / 2]);
Yeah, of course.
Thanks!
Paolo
Quoted 8 lines · Paolo Bonzini
Quoted 7 lines · Vitaly Kuznetsov
Let's say we have 1 half of XMM0 consumed. Now:
i = 0; j = 1; if (1) sparse_banks[0] = sse128_lo(hc->xmm[0]);
Quoted 8 lines
Quoted 8 lines
This doesn't look right as we need to get the upper half of XMM0.
I guess it should be reversed,
if (j % 2) sparse_banks[i] = sse128_hi(hc->xmm[j / 2]); else sparse_banks[i] = sse128_lo(hc->xmm[j / 2]);
Maybe I am missing parts of this series.. I dont see this change in any of the 4 patches Vitaly sent. Yes, they look swapped to me too.
~ Sid.
Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879
Vitaly Kuznetsov
maintainerQuoted 13 lines · Siddharth Chandrasekaran
Quoted 12 lines · Paolo Bonzini
Quoted 11 lines · Vitaly Kuznetsov
Let's say we have 1 half of XMM0 consumed. Now:
i = 0; j = 1; if (1) sparse_banks[0] = sse128_lo(hc->xmm[0]);This doesn't look right as we need to get the upper half of XMM0.
I guess it should be reversed,
Quoted 8 lines
Quoted 4 lines
Quoted 4 lines
if (j % 2) sparse_banks[i] = sse128_hi(hc->xmm[j / 2]); else sparse_banks[i] = sse128_lo(hc->xmm[j / 2]);Maybe I am missing parts of this series.. I dont see this change in any of the 4 patches Vitaly sent. Yes, they look swapped to me too.
There was a conflict with a patch series from Sean:
https://lore.kernel.org/kvm/[email protected]/
and this is a part of the resolution:
commit c0f1eaeb9e628bf86bf50f11cb4a2b671528391e
Merge: 4dfc4ec2b7f5 47d3e5cdfe60
Author: Paolo Bonzini <[email protected]>
Date: Fri Feb 25 06:28:10 2022 -0500
Merge branch 'kvm-hv-xmm-hypercall-fixes' into HEAD
Signature · 1 line
Vitaly
Quoted 26 lines · Vitaly Kuznetsov
Quoted 13 lines · Siddharth Chandrasekaran
Quoted 12 lines · Paolo Bonzini
Quoted 11 lines · Vitaly Kuznetsov
Let's say we have 1 half of XMM0 consumed. Now:
i = 0; j = 1; if (1) sparse_banks[0] = sse128_lo(hc->xmm[0]);This doesn't look right as we need to get the upper half of XMM0.
I guess it should be reversed,
Quoted 8 lines
Quoted 4 lines
Quoted 4 lines
if (j % 2) sparse_banks[i] = sse128_hi(hc->xmm[j / 2]); else sparse_banks[i] = sse128_lo(hc->xmm[j / 2]);Maybe I am missing parts of this series.. I dont see this change in any of the 4 patches Vitaly sent. Yes, they look swapped to me too.
There was a conflict with a patch series from Sean:
https://lore.kernel.org/kvm/[email protected]/
Quoted 8 lines
and this is a part of the resolution:
commit
c0f1eaeb9e628bf86bf50f11cb4a2b671528391e
Merge:4dfc4ec2b7f547d3e5cdfe60
Author: Paolo Bonzini <[email protected]>
Date: Fri Feb 25 06:28:10 2022 -0500Merge branch 'kvm-hv-xmm-hypercall-fixes' into HEAD
Got it, thank you!
~ Sid.
Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879