x86/hyper-v: micro-optimize send_ipi_one case
Subsystems (3)
PATCH v2 1/1 x86/hyper-v: micro-optimize send_ipi_one case
When sending an IPI to a single CPU there is no need to deal with cpumasks. With 2 CPU guest on WS2019 I'm seeing a minor (like 3%, 8043 -> 7761 CPU cycles) improvement with smp_call_function_single() loop benchmark. The optimization, however, is tiny and straitforward. Also, send_ipi_one() is important for PV spinlock kick.
I was also wondering if it would make sense to switch to using regular APIC IPI send for CPU > 64 case but no, it is twice as expesive (12650 CPU cycles for __send_ipi_mask_ex() call, 26000 for orig_apic.send_IPI(cpu, vector)).
---
Changes since v1:
- Style changes [Roman, Joe]
--- arch/x86/hyperv/hv_apic.c | 13 ++++++++++--- arch/x86/include/asm/trace/hyperv.h | 15 +++++++++++++++ 2 files changed, 25 insertions(+), 3 deletions(-)
arch/x86/hyperv/hv_apic.c +10 -3
@@ -194,10 +194,17 @@ static bool __send_ipi_mask(const struct cpumask *mask, int vector)static bool __send_ipi_one (int cpu,int vector) { -struct cpumask mask =CPU_MASK_NONE ; +trace_hyperv_send_ipi_one (cpu, vector); -cpumask_set_cpu (cpu, &mask); -return __send_ipi_mask (&mask, vector); +if (!hv_hypercall_pg || (vector <HV_IPI_LOW_VECTOR ) || + (vector >HV_IPI_HIGH_VECTOR )) +return false; + +if (cpu >=64 ) +return __send_ipi_mask_ex (cpumask_of (cpu), vector); + +return !hv_do_fast_hypercall16 (HVCALL_SEND_IPI , vector, +BIT_ULL (hv_cpu_number_to_vp_number (cpu))); }static void hv_send_ipi (int cpu,int vector)
arch/x86/include/asm/trace/hyperv.h +15 -0
@@ -71,6 +71,21 @@ TRACE_EVENT(hyperv_send_ipi_mask, __entry->ncpus, __entry->vector) ); +TRACE_EVENT (hyperv_send_ipi_one, + TP_PROTO(int cpu, + int vector), +TP_ARGS (cpu, vector), +TP_STRUCT__entry ( + __field(int, cpu) +__field (int, vector) + ), + TP_fast_assign(__entry->cpu = cpu; + __entry->vector = vector; + ), +TP_printk ("cpu %d vector %x" , + __entry->cpu, __entry->vector) + ); +#endif /* CONFIG_HYPERV */ #undef TRACE_INCLUDE_PATH
Signature · 1 line
2.20.1
Quoted 11 lines · Vitaly Kuznetsov
When sending an IPI to a single CPU there is no need to deal with cpumasks. With 2 CPU guest on WS2019 I'm seeing a minor (like 3%, 8043 -> 7761 CPU cycles) improvement with
smp_call_function_single()loop benchmark. The optimization, however, is tiny and straitforward. Also,send_ipi_one()is important for PV spinlock kick.I was also wondering if it would make sense to switch to using regular APIC IPI send for CPU > 64 case but no, it is twice as expesive (12650 CPU cycles for
__send_ipi_mask_ex()call, 26000 fororig_apic.send_IPI(cpu, vector)).
Quoted 8 lines
Signed-off-by Vitaly Kuznetsov <[email protected]>---
Changes since v1:
- Style changes [Roman, Joe]--- arch/x86/hyperv/hv_apic.c | 13 ++++++++++--- arch/x86/include/asm/trace/hyperv.h | 15 +++++++++++++++ 2 files changed, 25 insertions(+), 3 deletions(-)
From: Vitaly Kuznetsov <[email protected]>
Quoted 31 lines
When sending an IPI to a single CPU there is no need to deal with cpumasks. With 2 CPU guest on WS2019 I'm seeing a minor (like 3%, 8043 -> 7761 CPU cycles) improvement with
smp_call_function_single()loop benchmark. The optimization, however, is tiny and straitforward. Also,send_ipi_one()is important for PV spinlock kick.I was also wondering if it would make sense to switch to using regular APIC IPI send for CPU > 64 case but no, it is twice as expesive (12650 CPU cycles for
__send_ipi_mask_ex()call, 26000 fororig_apic.send_IPI(cpu, vector)).Signed-off-by Vitaly Kuznetsov <[email protected]>---
Changes since v1:
- Style changes [Roman, Joe]--- arch/x86/hyperv/hv_apic.c | 13 ++++++++++--- arch/x86/include/asm/trace/hyperv.h | 15 +++++++++++++++ 2 files changed, 25 insertions(+), 3 deletions(-)arch/x86/hyperv/hv_apic.c +0
-1@@ -194,10 +194,17 @@ static bool __send_ipi_mask(const struct cpumask *mask, int vector) static bool __send_ipi_one(int cpu, int vector) { -struct cpumask mask =CPU_MASK_NONE ;
arch/x86/hyperv/hv_apic.c static bool __send_ipi_mask(const struct cpumask *mask, int · line 205
Quoted 10 lines
arch/x86/hyperv/hv_apic.c +7
-2+trace_hyperv_send_ipi_one (cpu, vector); -cpumask_set_cpu (cpu, &mask); -return __send_ipi_mask (&mask, vector); +if (!hv_hypercall_pg || (vector <HV_IPI_LOW_VECTOR ) || + (vector >HV_IPI_HIGH_VECTOR )) +return false; + +if (cpu >=64 ) +return __send_ipi_mask_ex (cpumask_of (cpu), vector);
The above test should be checking the VP number, not the CPU number, since the VP number is used to form the bitmap argument to the hypercall. In all current implementations of Hyper-V, the CPU number and VP number are the same as far as I am aware, but that's not guaranteed in the future.
Michael
Quoted 5 lines
arch/x86/hyperv/hv_apic.c +3
-0+ +return !hv_do_fast_hypercall16 (HVCALL_SEND_IPI , vector, +BIT_ULL (hv_cpu_number_to_vp_number (cpu))); }
Quoted 34 lines · Michael Kelley
From: Vitaly Kuznetsov <[email protected]>
Quoted 33 lines
When sending an IPI to a single CPU there is no need to deal with cpumasks. With 2 CPU guest on WS2019 I'm seeing a minor (like 3%, 8043 -> 7761 CPU cycles) improvement with
smp_call_function_single()loop benchmark. The optimization, however, is tiny and straitforward. Also,send_ipi_one()is important for PV spinlock kick.I was also wondering if it would make sense to switch to using regular APIC IPI send for CPU > 64 case but no, it is twice as expesive (12650 CPU cycles for
__send_ipi_mask_ex()call, 26000 fororig_apic.send_IPI(cpu, vector)).Signed-off-by Vitaly Kuznetsov <[email protected]>---
Changes since v1:
- Style changes [Roman, Joe]--- arch/x86/hyperv/hv_apic.c | 13 ++++++++++--- arch/x86/include/asm/trace/hyperv.h | 15 +++++++++++++++ 2 files changed, 25 insertions(+), 3 deletions(-)arch/x86/hyperv/hv_apic.c +1
-1@@ -194,10 +194,17 @@ static bool __send_ipi_mask(const struct cpumask *mask, int vector) static bool __send_ipi_one(int cpu, int vector) { -struct cpumask mask =CPU_MASK_NONE ; +trace_hyperv_send_ipi_one (cpu, vector);
Quoted 11 lines
arch/x86/hyperv/hv_apic.c static bool __send_ipi_mask(const struct cpumask *mask, int vector) · line 204
Quoted 8 lines
arch/x86/hyperv/hv_apic.c +6
-2-cpumask_set_cpu (cpu, &mask); -return __send_ipi_mask (&mask, vector); +if (!hv_hypercall_pg || (vector <HV_IPI_LOW_VECTOR ) || + (vector >HV_IPI_HIGH_VECTOR )) +return false; + +if (cpu >=64 ) +return __send_ipi_mask_ex (cpumask_of (cpu), vector);The above test should be checking the VP number, not the CPU number,
Oops, of course, thanks for catching this! v3 is coming!
Quoted 12 lines
since the VP number is used to form the bitmap argument
to the hypercall. In all current implementations of Hyper-V, the CPU number and VP number are the same as far as I am aware, but that's not guaranteed in the future.Michael
Quoted 5 lines
arch/x86/hyperv/hv_apic.c +3
-0+ +return !hv_do_fast_hypercall16 (HVCALL_SEND_IPI , vector, +BIT_ULL (hv_cpu_number_to_vp_number (cpu))); }
Signature · 1 line
Vitaly