cpuidle-haltpoll driver (v2)
Subsystems (6)
cover cpuidle-haltpoll driver (v2)
The cpuidle-haltpoll driver allows the guest vcpus to poll for a specified amount of time before halting. This provides the following benefits to host side polling:
1) The POLL flag is set while polling is performed, which allows a remote vCPU to avoid sending an IPI (and the associated cost of handling the IPI) when performing a wakeup.
2) The HLT VM-exit cost can be avoided.
The downside of guest side polling is that polling is performed even with other runnable tasks in the host.
Results comparing halt_poll_ns and server/client application where a small packet is ping-ponged:
host --> 31.33 halt_poll_ns=300000 / no guest busy spin --> 33.40 (93.8%) halt_poll_ns=0 / guest_halt_poll_ns=300000 --> 32.73 (95.7%)
For the SAP HANA benchmarks (where idle_spin is a parameter of the previous version of the patch, results should be the same):
hpns == halt_poll_ns
idle_spin=0/ idle_spin=800/ idle_spin=0/
hpns=200000 hpns=0 hpns=800000
DeleteC06T03 (100 thread) 1.76 1.71 (-3%) 1.78 (+1%)
InsertC16T02 (100 thread) 2.14 2.07 (-3%) 2.18 (+1.8%)
DeleteC00T01 (1 thread) 1.34 1.28 (-4.5%) 1.29 (-3.7%)
UpdateC00T03 (1 thread) 4.72 4.18 (-12%) 4.53 (-5%)V2:
- Move from x86 to generic code (Paolo/Christian).
- Add auto-tuning logic (Paolo).
- Add MSR to disable host side polling (Paolo).
PATCH 1/3 drivers/cpuidle: add cpuidle-haltpoll driver
The cpuidle_kvm driver allows the guest vcpus to poll for a specified amount of time before halting. This provides the following benefits to host side polling:
1) The POLL flag is set while polling is performed, which allows a remote vCPU to avoid sending an IPI (and the associated cost of handling the IPI) when performing a wakeup.
2) The HLT VM-exit cost can be avoided.
The downside of guest side polling is that polling is performed even with other runnable tasks in the host.
Results comparing halt_poll_ns and server/client application where a small packet is ping-ponged:
host --> 31.33 halt_poll_ns=300000 / no guest busy spin --> 33.40 (93.8%) halt_poll_ns=0 / guest_halt_poll_ns=300000 --> 32.73 (95.7%)
For the SAP HANA benchmarks (where idle_spin is a parameter of the previous version of the patch, results should be the same):
hpns == halt_poll_ns
idle_spin=0/ idle_spin=800/ idle_spin=0/
hpns=200000 hpns=0 hpns=800000
DeleteC06T03 (100 thread) 1.76 1.71 (-3%) 1.78 (+1%)
InsertC16T02 (100 thread) 2.14 2.07 (-3%) 2.18 (+1.8%)
DeleteC00T01 (1 thread) 1.34 1.28 (-4.5%) 1.29 (-3.7%)
UpdateC00T03 (1 thread) 4.72 4.18 (-12%) 4.53 (-5%)--- Documentation/virtual/guest-halt-polling.txt | 78 ++++++++++++ arch/x86/kernel/process.c | 2 drivers/cpuidle/Kconfig | 10 + drivers/cpuidle/Makefile | 1 drivers/cpuidle/cpuidle-haltpoll-trace.h | 65 ++++++++++ drivers/cpuidle/cpuidle-haltpoll.c | 172 +++++++++++++++++++++++++++ 6 files changed, 327 insertions(+), 1 deletion(-)
Documentation/virtual/guest-halt-polling.txt new file +78 -0
@@ -0,0 +1,78 @@ +Guest halt polling +================== + +The cpuidle_haltpoll driver allows the guest vcpus to poll for a specified +amount of time before halting. This provides the following benefits +to host side polling: + + 1) The POLL flag is set while polling is performed, which allows + a remote vCPU to avoid sending an IPI (and the associated + cost of handling the IPI) when performing a wakeup. + + 2) The HLT VM-exit cost can be avoided. + +The downside of guest side polling is that polling is performed +even with other runnable tasks in the host. + +The basic logic as follows: A global value, guest_halt_poll_ns, +is configured by the user, indicating the maximum amount of +time polling is allowed. This value is fixed. + +Each vcpu has an adjustable guest_halt_poll_ns +("per-cpu guest_halt_poll_ns"), which is adjusted by the algorithm +in response to events (explained below). + +Module Parameters +================= + +The cpuidle_haltpoll module has 5 tunable module parameters: + +1) guest_halt_poll_ns: +Maximum amount of time, in nanoseconds, that polling is +performed before halting. + +Default: 200000 + +2) guest_halt_poll_shrink: +Division factor used to shrink per-cpu guest_halt_poll_ns when +wakeup event occurs after the global guest_halt_poll_ns. + +Default: 2 + +3) guest_halt_poll_grow: +Multiplication factor used to grow per-cpu guest_halt_poll_ns +when event occurs after per-cpu guest_halt_poll_ns +but before global guest_halt_poll_ns. + +Default: 2 + +4) guest_halt_poll_grow_start: +The per-cpu guest_halt_poll_ns eventually reaches zero +in case of an idle system. This value sets the initial +per-cpu guest_halt_poll_ns when growing. This can +be increased from 10000, to avoid misses during the initial +growth stage: + +10000, 20000, 40000, ... (example assumes guest_halt_poll_grow=2). + +Default: 10000 + +5) guest_halt_poll_allow_shrink: + +Bool parameter which allows shrinking. Set to N +to avoid it (per-cpu guest_halt_poll_ns will remain +high once achieves global guest_halt_poll_ns value). + +Default: Y + +The module parameters can be set from the debugfs files in: + + /sys/module/cpuidle_haltpoll/parameters/ + +Further Notes +============= + +- Care should be taken when setting the guest_halt_poll_ns parameter as a +large value has the potential to drive the cpu usage to 100% on a machine which +would be almost entirely idle otherwise. +
arch/x86/kernel/process.c +1 -1
@@ -580,7 +580,7 @@safe_halt ();trace_cpu_idle_rcuidle (PWR_EVENT_EXIT ,smp_processor_id ()); } -#ifdef CONFIG_APM_MODULE +#if defined(CONFIG_APM_MODULE ) || defined(CONFIG_HALTPOLL_CPUIDLE_MODULE )EXPORT_SYMBOL (default_idle);#endif
drivers/cpuidle/Kconfig +10 -0
@@ -51,6 +51,16 @@ source "drivers/cpuidle/Kconfig.powerpc" endmenu +config HALTPOLL_CPUIDLE + tristate "Halt poll cpuidle driver" + depends on X86 + default y + help + This option enables halt poll cpuidle driver, which allows to poll + before halting in the guest (more efficient than polling in the + host via halt_poll_ns for some scenarios). + + endif config ARCH_NEEDS_CPU_IDLE_COUPLED
drivers/cpuidle/Makefile +1 -0
@@ -7,6 +7,7 @@ obj-$(CONFIG_ARCH_NEEDS_CPU_IDLE_COUPLED) += coupled.o obj-$(CONFIG_DT_IDLE_STATES) += dt_idle_states.o obj-$(CONFIG_ARCH_HAS_CPU_RELAX) += poll_state.o +obj-$(CONFIG_HALTPOLL_CPUIDLE) += cpuidle-haltpoll.o ################################################################################## # ARM SoC drivers
drivers/cpuidle/cpuidle-haltpoll-trace.h new file +65 -0
65 lines not on this page · raw message
drivers/cpuidle/cpuidle-haltpoll.c new file +172 -0
172 lines not on this page · raw message
Re: [patch 1/3] drivers/cpuidle: add cpuidle-haltpoll driver
Quoted 49 lines · Marcelo Tosatti
The
cpuidle_kvmdriver allows the guest vcpus to poll for a specified amount of time before halting. This provides the following benefits to host side polling:1) The POLL flag is set while polling is performed, which allows a remote vCPU to avoid sending an IPI (and the associated cost of handling the IPI) when performing a wakeup.
2) The HLT VM-exit cost can be avoided.
The downside of guest side polling is that polling is performed even with other runnable tasks in the host.
Results comparing
halt_poll_nsand server/client application where a small packet is ping-ponged:host --> 31.33 halt_poll_ns=300000 / no guest busy spin --> 33.40 (93.8%) halt_poll_ns=0 / guest_halt_poll_ns=300000 --> 32.73 (95.7%)For the SAP HANA benchmarks (where
idle_spinis a parameter of the previous version of the patch, results should be the same):hpns ==
halt_poll_nsidle_spin=0/ idle_spin=800/ idle_spin=0/ hpns=200000 hpns=0 hpns=800000 DeleteC06T03 (100 thread) 1.76 1.71 (-3%) 1.78 (+1%) InsertC16T02 (100 thread) 2.14 2.07 (-3%) 2.18 (+1.8%) DeleteC00T01 (1 thread) 1.34 1.28 (-4.5%) 1.29 (-3.7%) UpdateC00T03 (1 thread) 4.72 4.18 (-12%) 4.53 (-5%)--- Documentation/virtual/guest-halt-polling.txt | 78 ++++++++++++ arch/x86/kernel/process.c | 2 drivers/cpuidle/Kconfig | 10 + drivers/cpuidle/Makefile | 1 drivers/cpuidle/cpuidle-haltpoll-trace.h | 65 ++++++++++ drivers/cpuidle/cpuidle-haltpoll.c | 172 +++++++++++++++++++++++++++ 6 files changed, 327 insertions(+), 1 deletion(-)Index:
linux-2.6.git/Documentation/virtual/guest-halt-polling.txt
===================================================================Documentation/virtual/guest-halt-polling.txt new file +2
-0@@ -0,0 +1,78 @@ +Guest halt polling +==================
Documentation/virtual/guest-halt-polling.txt line 12
Quoted 10 lines
Documentation/virtual/guest-halt-polling.txt new file +10
-0+ +The cpuidle_haltpoll driver allows the guest vcpus to poll for a specified +amount of time before halting. This provides the following benefits +to host side polling: + + 1) The POLL flag is set while polling is performed, which allows + a remote vCPU to avoid sending an IPI (and the associated + cost of handling the IPI) when performing a wakeup. + + 2) The HLT VM-exit cost can be avoided.
Nit: s/HLT// (also in the commit message). We might be using MWAIT passthrough via default_idle_call()
Documentation/virtual/guest-halt-polling.txt line 22
Quoted 10 lines
Documentation/virtual/guest-halt-polling.txt new file +10
-0+ +The downside of guest side polling is that polling is performed +even with other runnable tasks in the host. + +The basic logic as follows: A global value, guest_halt_poll_ns, +is configured by the user, indicating the maximum amount of +time polling is allowed. This value is fixed. + +Each vcpu has an adjustable guest_halt_poll_ns +("per-cpu guest_halt_poll_ns"), which is adjusted by the algorithm
I believe the per-cpu variable is halt_poll_ns (below as well.)
Quoted 235 lines
Documentation/virtual/guest-halt-polling.txt new file +56
-0+in response to events (explained below). + +Module Parameters +================= + +The cpuidle_haltpoll module has 5 tunable module parameters: + +1) guest_halt_poll_ns: +Maximum amount of time, in nanoseconds, that polling is +performed before halting. + +Default: 200000 + +2) guest_halt_poll_shrink: +Division factor used to shrink per-cpu guest_halt_poll_ns when +wakeup event occurs after the global guest_halt_poll_ns. + +Default: 2 + +3) guest_halt_poll_grow: +Multiplication factor used to grow per-cpu guest_halt_poll_ns +when event occurs after per-cpu guest_halt_poll_ns +but before global guest_halt_poll_ns. + +Default: 2 + +4) guest_halt_poll_grow_start: +The per-cpu guest_halt_poll_ns eventually reaches zero +in case of an idle system. This value sets the initial +per-cpu guest_halt_poll_ns when growing. This can +be increased from 10000, to avoid misses during the initial +growth stage: + +10000, 20000, 40000, ... (example assumes guest_halt_poll_grow=2). + +Default: 10000 + +5) guest_halt_poll_allow_shrink: + +Bool parameter which allows shrinking. Set to N +to avoid it (per-cpu guest_halt_poll_ns will remain +high once achieves global guest_halt_poll_ns value). + +Default: Y + +The module parameters can be set from the debugfs files in: + + /sys/module/cpuidle_haltpoll/parameters/ + +Further Notes +============= + +- Care should be taken when setting the guest_halt_poll_ns parameter as a +large value has the potential to drive the cpu usage to 100% on a machine which +would be almost entirely idle otherwise. +Index:
linux-2.6.git/arch/x86/kernel/process.c
===================================================================arch/x86/kernel/process.c +1
-1@@ -580,7 +580,7 @@safe_halt ();trace_cpu_idle_rcuidle (PWR_EVENT_EXIT ,smp_processor_id ()); } -#ifdef CONFIG_APM_MODULE +#if defined(CONFIG_APM_MODULE ) || defined(CONFIG_HALTPOLL_CPUIDLE_MODULE )EXPORT_SYMBOL (default_idle);#endif Index: linux-2.6.git/drivers/cpuidle/Kconfig
===================================================================drivers/cpuidle/Kconfig +10
-0@@ -51,6 +51,16 @@ source "drivers/cpuidle/Kconfig.powerpc" endmenu +config HALTPOLL_CPUIDLE + tristate "Halt poll cpuidle driver" + depends on X86 + default y + help + This option enables halt poll cpuidle driver, which allows to poll + before halting in the guest (more efficient than polling in the + host via halt_poll_ns for some scenarios). + + endif config ARCH_NEEDS_CPU_IDLE_COUPLEDIndex: linux-2.6.git/drivers/cpuidle/Makefile
===================================================================drivers/cpuidle/Makefile +1
-0@@ -7,6 +7,7 @@ obj-$(CONFIG_ARCH_NEEDS_CPU_IDLE_COUPLED) += coupled.o obj-$(CONFIG_DT_IDLE_STATES) += dt_idle_states.o obj-$(CONFIG_ARCH_HAS_CPU_RELAX) += poll_state.o +obj-$(CONFIG_HALTPOLL_CPUIDLE) += cpuidle-haltpoll.o ################################################################################## # ARM SoC driversIndex:
linux-2.6.git/drivers/cpuidle/cpuidle-haltpoll-trace.h
===================================================================drivers/cpuidle/cpuidle-haltpoll-trace.h new file +65
-0@@ -0,0 +1,65 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#if !defined(_HALTPOLL_TRACE_H_) || defined(TRACE_HEADER_MULTI_READ ) +#define _HALTPOLL_TRACE_H_ + +#include <linux/stringify.h> +#include <linux/types.h> +#include <linux/tracepoint.h> + +#undef TRACE_SYSTEM +#define TRACE_SYSTEM cpuidle_haltpoll + +TRACE_EVENT (cpuidle_haltpoll_success, + TP_PROTO(unsigned int cpu_halt_poll_ns, u64 block_ns), +TP_ARGS (cpu_halt_poll_ns, block_ns), + +TP_STRUCT__entry ( + __field(unsigned int, cpu_halt_poll_ns) +__field (u64, block_ns) + ), + +TP_fast_assign ( + __entry->cpu_halt_poll_ns = cpu_halt_poll_ns; + __entry->block_ns = block_ns; + ), + +TP_printk ("cpu_halt_poll_ns %u block_ns %lld" , + __entry->cpu_halt_poll_ns, + __entry->block_ns) +); + +TRACE_EVENT (cpuidle_haltpoll_fail, + TP_PROTO(unsigned int prev_cpu_halt_poll_ns, + unsigned int cpu_halt_poll_ns, + u64 block_ns), +TP_ARGS (prev_cpu_halt_poll_ns, cpu_halt_poll_ns, block_ns), + +TP_STRUCT__entry ( + __field(unsigned int, prev_cpu_halt_poll_ns) + __field(unsigned int, cpu_halt_poll_ns) +__field (u64, block_ns) + ), + +TP_fast_assign ( + __entry->prev_cpu_halt_poll_ns = + prev_cpu_halt_poll_ns; + __entry->cpu_halt_poll_ns = cpu_halt_poll_ns; + __entry->block_ns = block_ns; + ), + +TP_printk ("prev_cpu_halt_poll_ns %u cpu_halt_poll_ns %u block_ns %lld" , + __entry->prev_cpu_halt_poll_ns, + __entry->cpu_halt_poll_ns, + __entry->block_ns) +); + +#endif /* _HALTPOLL_TRACE_H_ */ + +/* This part must be outside protection */ +#undef TRACE_INCLUDE_PATH +#define TRACE_INCLUDE_PATH ../../drivers/cpuidle/ +#undef TRACE_INCLUDE_FILE +#define TRACE_INCLUDE_FILE cpuidle-haltpoll-trace +#include <trace/define_trace.h> + +Index:
linux-2.6.git/drivers/cpuidle/cpuidle-haltpoll.c
===================================================================drivers/cpuidle/cpuidle-haltpoll.c new file +58
-0@@ -0,0 +1,172 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* cpuidle driver for halt polling. +* +* Copyright 2019 Red Hat, Inc. and/or its affiliates. +* +* This work is licensed under the terms of the GNU GPL, version 2. See +* the COPYING file in the top-level directory. +* +* Authors: Marcelo Tosatti <[email protected]> +*/ + +#include <linux/init.h> +#include <linux/cpuidle.h> +#include <linux/module.h> +#include <linux/timekeeping.h> +#include <linux/sched/idle.h> +#define CREATE_TRACE_POINTS +#include "cpuidle-haltpoll-trace.h" + +unsigned int guest_halt_poll_ns =200000 ; +module_param (guest_halt_poll_ns, uint,0644 ); + +/* division factor to shrink halt_poll_ns */ +unsigned int guest_halt_poll_shrink =2 ; +module_param (guest_halt_poll_shrink, uint,0644 ); + +/* multiplication factor to grow per-cpu halt_poll_ns */ +unsigned int guest_halt_poll_grow =2 ; +module_param (guest_halt_poll_grow, uint,0644 ); + +/* value in ns to start growing per-cpu halt_poll_ns */ +unsigned int guest_halt_poll_grow_start =10000 ; +module_param (guest_halt_poll_grow_start, uint,0644 ); + +/* value in ns to start growing per-cpu halt_poll_ns */ +bool guest_halt_poll_allow_shrink = true; +module_param (guest_halt_poll_allow_shrink, bool,0644 ); + +static DEFINE_PER_CPU(unsigned int, halt_poll_ns); + +static void adjust_haltpoll_ns (unsigned int block_ns, +unsigned int *cpu_halt_poll_ns) +{ +unsigned int val; +unsigned int prev_halt_poll_ns = *cpu_halt_poll_ns; + +/* Grow cpu_halt_poll_ns if +* cpu_halt_poll_ns < block_ns < guest_halt_poll_ns +*/ +if (block_ns > *cpu_halt_poll_ns && block_ns <= guest_halt_poll_ns) { + val = *cpu_halt_poll_ns * guest_halt_poll_grow; + +if (val < guest_halt_poll_grow_start) + val = guest_halt_poll_grow_start; +if (val > guest_halt_poll_ns) + val = guest_halt_poll_ns; +
drivers/cpuidle/cpuidle-haltpoll.c line 68
Quoted 10 lines
drivers/cpuidle/cpuidle-haltpoll.c new file +10
-0+ *cpu_halt_poll_ns = val; + }else if (block_ns > guest_halt_poll_ns && + guest_halt_poll_allow_shrink) { +unsigned int shrink = guest_halt_poll_shrink; + + val = *cpu_halt_poll_ns; +if (shrink ==0 ) + val =0 ; +else + val /= shrink;
Nit, feel free to ignore: we could check for guest_halt_poll_shrink along with guest_halt_poll_allow_shrink and get rid of this whole conditional.
Ankur
Quoted 106 lines
drivers/cpuidle/cpuidle-haltpoll.c new file +104
-0+ *cpu_halt_poll_ns = val; + } + +trace_cpuidle_haltpoll_fail (prev_halt_poll_ns, *cpu_halt_poll_ns, + block_ns); +} + +static int haltpoll_enter_idle (struct cpuidle_device *dev, +struct cpuidle_driver *drv,int index) +{ +int do_halt =0 ; +unsigned int *cpu_halt_poll_ns; +ktime_t start, now; +int cpu =smp_processor_id (); + + cpu_halt_poll_ns =per_cpu_ptr (&halt_poll_ns, cpu); + +/* No polling */ +if (guest_halt_poll_ns ==0 ) { +if (current_clr_polling_and_test ()) { +local_irq_enable (); +return index; + } +default_idle (); +return index; + } + +local_irq_enable (); + + now = start =ktime_get (); +if (!current_set_polling_and_test ()) { +ktime_t end_spin; + + end_spin =ktime_add_ns (now, *cpu_halt_poll_ns); + +while (!need_resched ()) { +cpu_relax (); + now =ktime_get (); + +if (!ktime_before (now, end_spin)) { + do_halt =1 ; +break ; + } + } + } + +if (do_halt) { +u64 block_ns; + +/* +* No events while busy spin window passed, +* halt. +*/ +local_irq_disable (); +if (current_clr_polling_and_test ()) { +local_irq_enable (); +return index; + } +default_idle (); + block_ns =ktime_to_ns (ktime_sub (ktime_get (), start)); +adjust_haltpoll_ns (block_ns, cpu_halt_poll_ns); + }else { +u64 block_ns =ktime_to_ns (ktime_sub (now, start)); + +trace_cpuidle_haltpoll_success (*cpu_halt_poll_ns, block_ns); +current_clr_polling (); + } + +return index; +} + +static struct cpuidle_driver haltpoll_driver = { + .name ="haltpoll_idle" , + .owner =THIS_MODULE , + .states = { + {/* entry 0 is for polling */ }, + { + .enter = haltpoll_enter_idle, + .exit_latency =0 , + .target_residency =0 , + .power_usage =-1 , + .name ="Halt poll" , + .desc ="Halt poll idle" , + }, + }, + .safe_state_index =0 , + .state_count =2 , +}; + +static int __init haltpoll_init (void ) +{ +return cpuidle_register (&haltpoll_driver,NULL ); +} + +static void __exit haltpoll_exit(void) +{ +cpuidle_unregister (&haltpoll_driver); +} + +module_init (haltpoll_init); +module_exit (haltpoll_exit); +MODULE_LICENSE ("GPL" ); +MODULE_AUTHOR ("Marcelo Tosatti <[email protected]>" ); +
Re: [patch 1/3] drivers/cpuidle: add cpuidle-haltpoll driver
In reply to Marcelo TosattiHello,
Quoted 8 lines · Marcelo Tosatti
drivers/cpuidle/cpuidle-haltpoll.c new file +8
-0+unsigned int guest_halt_poll_ns =200000 ; +module_param (guest_halt_poll_ns, uint,0644 ); + +/* division factor to shrink halt_poll_ns */ +unsigned int guest_halt_poll_shrink =2 ; +module_param (guest_halt_poll_shrink, uint,0644 ); + +/* multiplication factor to grow per-cpu halt_poll_ns */
drivers/cpuidle/cpuidle-haltpoll.c line 38
Quoted 10 lines
drivers/cpuidle/cpuidle-haltpoll.c new file +10
-0+unsigned int guest_halt_poll_grow =2 ; +module_param (guest_halt_poll_grow, uint,0644 ); + +/* value in ns to start growing per-cpu halt_poll_ns */ +unsigned int guest_halt_poll_grow_start =10000 ; +module_param (guest_halt_poll_grow_start, uint,0644 ); + +/* value in ns to start growing per-cpu halt_poll_ns */ +bool guest_halt_poll_allow_shrink = true; +module_param (guest_halt_poll_allow_shrink, bool,0644 );
These variables can all be static. They also should be __read_mostly to be sure not to unnecessarily hit false sharing while going idle.
drivers/cpuidle/cpuidle-haltpoll.c line 112
Quoted 9 lines
drivers/cpuidle/cpuidle-haltpoll.c new file +9
-0+while (!need_resched ()) { +cpu_relax (); + now =ktime_get (); + +if (!ktime_before (now, end_spin)) { + do_halt =1 ; +break ; + } + }
On skylake pause takes ~75 cycles with ple_gap=0 (and Marcelo found it takes 6 cycles with pause loop exiting enabled but that shall be fixed in the CPU and we can ignore it).
So we could call ktime_get() only once every 100 times or more and we'd be still accurate down to at least 1usec.
Ideally we'd like a ktime_try_get() that will break the seqcount loop if read_seqcount_retry fails. Something like below pseudocode:
#define KTIME_ERR ((ktime_t) { .tv64 = 0 })
ktime_t ktime_try_get(void)
{
[..]
seq = read_seqcount_begin(&timekeeper_seq);
secs = tk->xtime_sec + tk->wall_to_monotonic.tv_sec;
nsecs = timekeeping_get_ns(&tk->tkr_mono) +
tk->wall_to_monotonic.tv_nsec;
if (unlikely(read_seqcount_retry(&timekeeper_seq, seq)))
return KTIME_ERR;
[..]
}If it ktime_try_get() fails we keep calling it at every iteration of the loop, when finally it succeeds we call it again only after 100 pause instructions or more. So we continue polling need_resched() while we wait timerkeeper_seq to be released (and hopefully by looping 100 times or more we'll reduce the frequency when we find timekeeper_seq locked).
All we care is to react to need_resched ASAP and to have a resolution of the order of 1usec for the spin time.
If 100 is too wired a new module parameter in __read_mostly section configured to 100 or more by default, sounds preferable than hitting every 75nsec on the timekeeper_seq seqcount cacheline.
I doubt it'd make any measurable difference with a few vcpus, but with hundred of host CPUs and vcpus perhaps it's worth it.
This of course can be done later once the patch is merged and if it's confirmed the above makes sense in practice and not just in theory. I wouldn't want to delay the merging for a possible micro optimization.
Thanks,
Andrea
Re: [patch 1/3] drivers/cpuidle: add cpuidle-haltpoll driver
Hi Andrea,
Quoted 16 lines · Andrea Arcangeli
Hello,
Quoted 13 lines · Marcelo Tosatti
drivers/cpuidle/cpuidle-haltpoll.c new file +13
-0+unsigned int guest_halt_poll_ns =200000 ; +module_param (guest_halt_poll_ns, uint,0644 ); + +/* division factor to shrink halt_poll_ns */ +unsigned int guest_halt_poll_shrink =2 ; +module_param (guest_halt_poll_shrink, uint,0644 ); + +/* multiplication factor to grow per-cpu halt_poll_ns */ +unsigned int guest_halt_poll_grow =2 ; +module_param (guest_halt_poll_grow, uint,0644 ); + +/* value in ns to start growing per-cpu halt_poll_ns */ +unsigned int guest_halt_poll_grow_start =10000 ;
Quoted 8 lines
drivers/cpuidle/cpuidle-haltpoll.c line 38
Quoted 5 lines
drivers/cpuidle/cpuidle-haltpoll.c new file +5
-0+module_param (guest_halt_poll_grow_start, uint,0644 ); + +/* value in ns to start growing per-cpu halt_poll_ns */ +bool guest_halt_poll_allow_shrink = true; +module_param (guest_halt_poll_allow_shrink, bool,0644 );These variables can all be static. They also should be
__read_mostlyto be sure not to unnecessarily hit false sharing while going idle.
Fixed.
Quoted 14 lines
drivers/cpuidle/cpuidle-haltpoll.c line 112
Quoted 10 lines
drivers/cpuidle/cpuidle-haltpoll.c new file +9
-0+while (!need_resched ()) { +cpu_relax (); + now =ktime_get (); + +if (!ktime_before (now, end_spin)) { + do_halt =1 ; +break ; + } + }On skylake pause takes ~75 cycles with ple_gap=0 (and Marcelo found it takes 6 cycles with pause loop exiting enabled but that shall be fixed in the CPU and we can ignore it).
Right, that is a generic problem.
Quoted 27 lines
So we could call
ktime_get()only once every 100 times or more and we'd be still accurate down to at least 1usec.Ideally we'd like a
ktime_try_get()that will break the seqcount loop ifread_seqcount_retryfails. Something like below pseudocode:#define
KTIME_ERR((ktime_t) { .tv64 = 0 })
ktime_tktime_try_get(void){ [..] seq = read_seqcount_begin(&timekeeper_seq); secs = tk->xtime_sec + tk->wall_to_monotonic.tv_sec; nsecs = timekeeping_get_ns(&tk->tkr_mono) + tk->wall_to_monotonic.tv_nsec; if (unlikely(read_seqcount_retry(&timekeeper_seq, seq))) return KTIME_ERR; [..] }If it
ktime_try_get()fails we keep calling it at every iteration of the loop, when finally it succeeds we call it again only after 100 pause instructions or more. So we continue pollingneed_resched()while we waittimerkeeper_seqto be released (and hopefully by looping 100 times or more we'll reduce the frequency when we findtimekeeper_seqlocked).
Quoted 6 lines
All we care is to react to
need_reschedASAP and to have a resolution of the order of 1usec for the spin time.If 100 is too wired a new module parameter in
__read_mostlysection configured to 100 or more by default, sounds preferable than hitting every 75nsec on thetimekeeper_seqseqcount cacheline.
I'll switch to the cheaper sched_clock as suggested by Peter.
Thanks!
Quoted 12 lines
I doubt it'd make any measurable difference with a few vcpus, but with hundred of host CPUs and vcpus perhaps it's worth it.
This of course can be done later once the patch is merged and if it's confirmed the above makes sense in practice and not just in theory. I wouldn't want to delay the merging for a possible micro optimization.
Reviewed-by Andrea Arcangeli <[email protected]>Thanks,
Andrea
Re: [patch 1/3] drivers/cpuidle: add cpuidle-haltpoll driver
In reply to Marcelo TosattiQuoted 53 lines · Marcelo Tosatti
drivers/cpuidle/cpuidle-haltpoll.c new file +53
-0+static int haltpoll_enter_idle (struct cpuidle_device *dev, +struct cpuidle_driver *drv,int index) +{ +int do_halt =0 ; +unsigned int *cpu_halt_poll_ns; +ktime_t start, now; +int cpu =smp_processor_id (); + + cpu_halt_poll_ns =per_cpu_ptr (&halt_poll_ns, cpu); + +/* No polling */ +if (guest_halt_poll_ns ==0 ) { +if (current_clr_polling_and_test ()) { +local_irq_enable (); +return index; + } +default_idle (); +return index; + } + +local_irq_enable (); + + now = start =ktime_get (); +if (!current_set_polling_and_test ()) { +ktime_t end_spin; + + end_spin =ktime_add_ns (now, *cpu_halt_poll_ns); + +while (!need_resched ()) { +cpu_relax (); + now =ktime_get (); + +if (!ktime_before (now, end_spin)) { + do_halt =1 ; +break ; + } + } + } + +if (do_halt) { +u64 block_ns; + +/* +* No events while busy spin window passed, +* halt. +*/ +local_irq_disable (); +if (current_clr_polling_and_test ()) { +local_irq_enable (); +return index; + } +default_idle (); + block_ns =ktime_to_ns (ktime_sub (ktime_get (), start));
drivers/cpuidle/cpuidle-haltpoll.c line 138
Quoted 10 lines
drivers/cpuidle/cpuidle-haltpoll.c new file +10
-0+adjust_haltpoll_ns (block_ns, cpu_halt_poll_ns); + }else { +u64 block_ns =ktime_to_ns (ktime_sub (now, start)); + +trace_cpuidle_haltpoll_success (*cpu_halt_poll_ns, block_ns); +current_clr_polling (); + } + +return index; +}
You might want to look at using sched_clock() here instead of ktime_get(). ktime_get() can get _very_ expensive when it drops back to HPET or things like that, where sched_clock() will always keep using TSC, even when it is not globally synchronized.
(and since this code runs with preemption disabled, we don't care about the clock being globally sane)
So something like this:
start = sched_clock();
if (current_set_polling_and_test()) {
local_irq_enable();
goto out;
}
local_irq_enable();
for (;;) {
if (need_resched()) {
current_clr_polling();
trace_..();
goto out;
}
now = sched_clock();
if (now - start > cpu_halt_poll_ns)
break;
cpu_relax();
}
local_irq_disable();
if (current_clr_polling_and_test()) {
local_irq_enable();
goto out;
}
default_idle();
block_ns = sched_clock() - start;
adjust_haltpoll_ns(block_ns, cpu_halt_poll_ns);out:
return index;
Re: [patch 1/3] drivers/cpuidle: add cpuidle-haltpoll driver
In reply to Marcelo TosattiOn 6/4/2019 12:52 AM, Marcelo Tosatti wrote:
It is rather inconvenient to comment patches posted as attachments.
Anyway, IMO adding trace events at this point is premature, please move adding them to a separate patch at the end of the series.
PATCH 2/3 kvm: x86: add host poll control msrs
In reply to Marcelo TosattiAdd an MSRs which allows the guest to disable
host polling (specifically the cpuidle-haltpoll,
when performing polling in the guest, disables
host side polling).
--- Documentation/virtual/kvm/msr.txt | 9 +++++++++ arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/include/uapi/asm/kvm_para.h | 2 ++ arch/x86/kvm/Kconfig | 1 + arch/x86/kvm/cpuid.c | 3 ++- arch/x86/kvm/x86.c | 23 +++++++++++++++++++++++ 6 files changed, 39 insertions(+), 1 deletion(-)
Documentation/virtual/kvm/msr.txt +9 -0
@@ -273,3 +273,12 @@ guest must both read the least significant bit in the memory area and clear it using a single CPU instruction, such as test and clear, or compare and exchange. + +MSR_KVM_POLL_CONTROL: 0x4b564d05 + Control host side polling. + + data: Bit 0 enables (1) or disables (0) host halt poll + logic. + KVM guests can disable host halt polling when performing + polling themselves. +
arch/x86/include/asm/kvm_host.h +2 -0
@@ -755,6 +755,8 @@struct gfn_to_hva_cache data; } pv_eoi; +u64 msr_kvm_poll_control; +/* * Indicate whether the access faults on its page table in guest * which is set when fix page fault and used to detect unhandeable
arch/x86/include/uapi/asm/kvm_para.h +2 -0
@@ -29,6 +29,7 @@#define KVM_FEATURE_PV_TLB_FLUSH 9#define KVM_FEATURE_ASYNC_PF_VMEXIT 10#define KVM_FEATURE_PV_SEND_IPI 11 +#define KVM_FEATURE_POLL_CONTROL 12#define KVM_HINTS_REALTIME 0
@@ -47,6 +48,7 @@#define MSR_KVM_ASYNC_PF_EN 0x4b564d02#define MSR_KVM_STEAL_TIME 0x4b564d03#define MSR_KVM_PV_EOI_EN 0x4b564d04 +#define MSR_KVM_POLL_CONTROL 0x4b564d05struct kvm_steal_time {__u64 steal;
arch/x86/kvm/Kconfig +1 -0
@@ -41,6 +41,7 @@ select PERF_EVENTS select HAVE_KVM_MSI select HAVE_KVM_CPU_RELAX_INTERCEPT + select HAVE_KVM_NO_POLL select KVM_GENERIC_DIRTYLOG_READ_PROTECT select KVM_VFIO select SRCU
arch/x86/kvm/cpuid.c +2 -1
@@ -643,7 +643,8 @@ (1 <<KVM_FEATURE_PV_UNHALT ) | (1 <<KVM_FEATURE_PV_TLB_FLUSH ) | (1 <<KVM_FEATURE_ASYNC_PF_VMEXIT ) | - (1 <<KVM_FEATURE_PV_SEND_IPI ); + (1 <<KVM_FEATURE_PV_SEND_IPI ) | + (1 <<KVM_FEATURE_POLL_CONTROL );if (sched_info_on ()) entry->eax |= (1 <<KVM_FEATURE_STEAL_TIME );
arch/x86/kvm/x86.c +23 -0
@@ -1177,6 +1177,7 @@MSR_IA32_POWER_CTL ,MSR_K7_HWCR , + MSR_KVM_POLL_CONTROL, };static unsigned num_emulated_msrs;
@@ -2628,6 +2629,14 @@return 1 ;break ; +case MSR_KVM_POLL_CONTROL : +/* only enable bit supported */ +if (data & (-1ULL <<1 )) +return 1 ; + + vcpu->arch.msr_kvm_poll_control = data; +break ; +case MSR_IA32_MCG_CTL :case MSR_IA32_MCG_STATUS : case MSR_IA32_MC0_CTL ... MSR_IA32_MCx_CTL(KVM_MAX_MCE_BANKS) - 1:
@@ -2877,6 +2886,9 @@case MSR_KVM_PV_EOI_EN : msr_info->data = vcpu->arch.pv_eoi.msr_val;break ; +case MSR_KVM_POLL_CONTROL : + msr_info->data = vcpu->arch.msr_kvm_poll_control; +break ;case MSR_IA32_P5_MC_ADDR :case MSR_IA32_P5_MC_TYPE :case MSR_IA32_MCG_CAP :
@@ -8874,6 +8886,10 @@ msr.host_initiated = true;kvm_write_tsc (vcpu, &msr);vcpu_put (vcpu); + +/* poll control enabled by default */ + vcpu->arch.msr_kvm_poll_control =1 ; +mutex_unlock (&vcpu->mutex);if (!kvmclock_periodic_sync)
@@ -9948,6 +9964,13 @@ }EXPORT_SYMBOL_GPL (kvm_vector_hashing_enabled); +bool kvm_arch_no_poll (struct kvm_vcpu *vcpu) +{ +return (vcpu->arch.msr_kvm_poll_control &1 ) ==0 ; +} +EXPORT_SYMBOL_GPL (kvm_arch_no_poll); + +EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_exit);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_fast_mmio);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_inj_virq);
Paolo Bonzini
maintainerRe: [patch 2/3] kvm: x86: add host poll control msrs
Quoted 6 lines
Add an MSRs which allows the guest to disable
host polling (specifically the cpuidle-haltpoll,
when performing polling in the guest, disables
host side polling).Signed-off-by Marcelo Tosatti <[email protected]>
Queued this patch while waiting for Rafal's review.
Paolo
Quoted 149 lines
--- Documentation/virtual/kvm/msr.txt | 9 +++++++++ arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/include/uapi/asm/kvm_para.h | 2 ++ arch/x86/kvm/Kconfig | 1 + arch/x86/kvm/cpuid.c | 3 ++- arch/x86/kvm/x86.c | 23 +++++++++++++++++++++++ 6 files changed, 39 insertions(+), 1 deletion(-)Index:
linux-2.6.git/Documentation/virtual/kvm/msr.txt
===================================================================Documentation/virtual/kvm/msr.txt +9
-0@@ -273,3 +273,12 @@ guest must both read the least significant bit in the memory area and clear it using a single CPU instruction, such as test and clear, or compare and exchange. + +MSR_KVM_POLL_CONTROL: 0x4b564d05 + Control host side polling. + + data: Bit 0 enables (1) or disables (0) host halt poll + logic. + KVM guests can disable host halt polling when performing + polling themselves. +Index:
linux-2.6.git/arch/x86/include/asm/kvm_host.h
===================================================================arch/x86/include/asm/kvm_host.h +2
-0@@ -755,6 +755,8 @@struct gfn_to_hva_cache data; } pv_eoi; +u64 msr_kvm_poll_control; +/* * Indicate whether the access faults on its page table in guest * which is set when fix page fault and used to detect unhandeable Index:
linux-2.6.git/arch/x86/include/uapi/asm/kvm_para.h
===================================================================arch/x86/include/uapi/asm/kvm_para.h +2
-0@@ -29,6 +29,7 @@#define KVM_FEATURE_PV_TLB_FLUSH 9#define KVM_FEATURE_ASYNC_PF_VMEXIT 10#define KVM_FEATURE_PV_SEND_IPI 11 +#define KVM_FEATURE_POLL_CONTROL 12#define KVM_HINTS_REALTIME 0@@ -47,6 +48,7 @@#define MSR_KVM_ASYNC_PF_EN 0x4b564d02#define MSR_KVM_STEAL_TIME 0x4b564d03#define MSR_KVM_PV_EOI_EN 0x4b564d04 +#define MSR_KVM_POLL_CONTROL 0x4b564d05struct kvm_steal_time {__u64 steal;Index: linux-2.6.git/arch/x86/kvm/Kconfig
===================================================================arch/x86/kvm/Kconfig +1
-0@@ -41,6 +41,7 @@ select PERF_EVENTS select HAVE_KVM_MSI select HAVE_KVM_CPU_RELAX_INTERCEPT + select HAVE_KVM_NO_POLL select KVM_GENERIC_DIRTYLOG_READ_PROTECT select KVM_VFIO select SRCUIndex:
linux-2.6.git/arch/x86/kvm/cpuid.c
===================================================================arch/x86/kvm/cpuid.c +2
-1@@ -643,7 +643,8 @@ (1 <<KVM_FEATURE_PV_UNHALT ) | (1 <<KVM_FEATURE_PV_TLB_FLUSH ) | (1 <<KVM_FEATURE_ASYNC_PF_VMEXIT ) | - (1 <<KVM_FEATURE_PV_SEND_IPI ); + (1 <<KVM_FEATURE_PV_SEND_IPI ) | + (1 <<KVM_FEATURE_POLL_CONTROL );if (sched_info_on ()) entry->eax |= (1 <<KVM_FEATURE_STEAL_TIME );Index:
linux-2.6.git/arch/x86/kvm/x86.c
===================================================================arch/x86/kvm/x86.c +23
-0@@ -1177,6 +1177,7 @@MSR_IA32_POWER_CTL ,MSR_K7_HWCR , + MSR_KVM_POLL_CONTROL, };static unsigned num_emulated_msrs;@@ -2628,6 +2629,14 @@return 1 ;break ; +case MSR_KVM_POLL_CONTROL : +/* only enable bit supported */ +if (data & (-1ULL <<1 )) +return 1 ; + + vcpu->arch.msr_kvm_poll_control = data; +break ; +case MSR_IA32_MCG_CTL :case MSR_IA32_MCG_STATUS : case MSR_IA32_MC0_CTL ... MSR_IA32_MCx_CTL(KVM_MAX_MCE_BANKS) - 1:@@ -2877,6 +2886,9 @@case MSR_KVM_PV_EOI_EN : msr_info->data = vcpu->arch.pv_eoi.msr_val;break ; +case MSR_KVM_POLL_CONTROL : + msr_info->data = vcpu->arch.msr_kvm_poll_control; +break ;case MSR_IA32_P5_MC_ADDR :case MSR_IA32_P5_MC_TYPE :case MSR_IA32_MCG_CAP :@@ -8874,6 +8886,10 @@ msr.host_initiated = true;kvm_write_tsc (vcpu, &msr);vcpu_put (vcpu); + +/* poll control enabled by default */ + vcpu->arch.msr_kvm_poll_control =1 ; +mutex_unlock (&vcpu->mutex);if (!kvmclock_periodic_sync)@@ -9948,6 +9964,13 @@ }EXPORT_SYMBOL_GPL (kvm_vector_hashing_enabled); +bool kvm_arch_no_poll (struct kvm_vcpu *vcpu) +{ +return (vcpu->arch.msr_kvm_poll_control &1 ) ==0 ; +} +EXPORT_SYMBOL_GPL (kvm_arch_no_poll); + +EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_exit);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_fast_mmio);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_inj_virq);
PATCH 3/3 cpuidle-haltpoll: disable host side polling when kvm virtualized
In reply to Marcelo TosattiWhen performing guest side polling, it is not necessary to also perform host side polling.
So disable host side polling, via the new MSR interface, when loading cpuidle-haltpoll driver.
--- arch/x86/Kconfig | 7 +++++ arch/x86/include/asm/cpuidle_haltpoll.h | 8 ++++++ arch/x86/kernel/kvm.c | 40 ++++++++++++++++++++++++++++++++ drivers/cpuidle/cpuidle-haltpoll.c | 9 ++++++- include/linux/cpuidle_haltpoll.h | 16 ++++++++++++ 5 files changed, 79 insertions(+), 1 deletion(-)
arch/x86/include/asm/cpuidle_haltpoll.h new file +8 -0
@@ -0,0 +1,8 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ARCH_HALTPOLL_H +#define _ARCH_HALTPOLL_H + +void arch_haltpoll_enable (void ); +void arch_haltpoll_disable (void ); + +#endif
drivers/cpuidle/cpuidle-haltpoll.c +8 -1
@@ -15,6 +15,7 @@#include <linux/module.h> #include <linux/timekeeping.h> #include <linux/sched/idle.h> +#include <linux/cpuidle_haltpoll.h> #define CREATE_TRACE_POINTS #include "cpuidle-haltpoll-trace.h"
@@ -157,11 +158,17 @@static int __init haltpoll_init (void ) { -return cpuidle_register (&haltpoll_driver,NULL ); +int ret =cpuidle_register (&haltpoll_driver,NULL ); + +if (ret ==0 ) +arch_haltpoll_enable (); + +return ret; } static void __exit haltpoll_exit(void) { +arch_haltpoll_disable ();cpuidle_unregister (&haltpoll_driver); }
include/linux/cpuidle_haltpoll.h new file +16 -0
@@ -0,0 +1,16 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _CPUIDLE_HALTPOLL_H +#define _CPUIDLE_HALTPOLL_H + +#ifdef CONFIG_ARCH_CPUIDLE_HALTPOLL +#include <asm/cpuidle_haltpoll.h> +#else +static inline void arch_haltpoll_enable (void ) +{ +} + +static inline void arch_haltpoll_disable (void ) +{ +} +#endif +#endif
arch/x86/Kconfig +7 -0
@@ -787,6 +787,7 @@ bool "KVM Guest support (including kvmclock)" depends on PARAVIRT select PARAVIRT_CLOCK + select ARCH_CPUIDLE_HALTPOLL default y ---help--- This option enables various optimizations for running under the KVM
@@ -795,6 +796,12 @@ underlying device model, the host provides the guest with timing infrastructure such as time of day, and system time +config ARCH_CPUIDLE_HALTPOLL + def_bool n + prompt "Disable host haltpoll when loading haltpoll driver" + help + If virtualized under KVM, disable host haltpoll. + config PVH bool "Support for running PVH guests" ---help---
arch/x86/kernel/kvm.c +40 -0
@@ -853,3 +853,43 @@ }#endif /* CONFIG_PARAVIRT_SPINLOCKS */ + +#ifdef CONFIG_ARCH_CPUIDLE_HALTPOLL + +void kvm_disable_host_haltpoll (void *i) +{ +wrmsrl (MSR_KVM_POLL_CONTROL ,0 ); +} + +void kvm_enable_host_haltpoll (void *i) +{ +wrmsrl (MSR_KVM_POLL_CONTROL ,1 ); +} + +void arch_haltpoll_enable (void ) +{ +if (!kvm_para_has_feature (KVM_FEATURE_POLL_CONTROL )) +return ; + +preempt_disable (); +/* Enabling guest halt poll disables host halt poll */ +kvm_disable_host_haltpoll (NULL ); +smp_call_function (kvm_disable_host_haltpoll,NULL ,1 ); +preempt_enable (); +} +EXPORT_SYMBOL_GPL (arch_haltpoll_enable); + +void arch_haltpoll_disable (void ) +{ +if (!kvm_para_has_feature (KVM_FEATURE_POLL_CONTROL )) +return ; + +preempt_disable (); +/* Enabling guest halt poll disables host halt poll */ +kvm_enable_host_haltpoll (NULL ); +smp_call_function (kvm_enable_host_haltpoll,NULL ,1 ); +preempt_enable (); +} +} +EXPORT_SYMBOL_GPL (arch_haltpoll_disable); +#endif
kbuild test robot
build botRe: [patch 3/3] cpuidle-haltpoll: disable host side polling when kvm virtualized
Thank you for the patch! Yet something to improve:
Hi Marcelo,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on kvm/linux-next]
[also build test ERROR on v5.2-rc3 next-20190603]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
url: https://github.com/0day-ci/linux/commits/Marcelo-Tosatti/cpuidle-haltpoll-driver-v2/20190604-083002 base: https://git.kernel.org/pub/scm/virt/kvm/kvm.git linux-next config: x86_64-allyesconfig (attached as .config) compiler: gcc-7 (Debian 7.3.0-1) 7.3.0 reproduce: # save the attached .config to linux build tree make ARCH=x86_64
If you fix the issue, kindly add following tag
All errors (new ones prefixed by >>):
>> arch/x86//kernel/kvm.c:906:1: error: expected identifier or '(' before '}' token
}
^vim +906 arch/x86//kernel/kvm.c
894
895 void arch_haltpoll_disable(void)
896 {
897 if (!kvm_para_has_feature(KVM_FEATURE_POLL_CONTROL))
898 return;
899
900 preempt_disable();
901 /* Enabling guest halt poll disables host halt poll */
902 kvm_enable_host_haltpoll(NULL);
903 smp_call_function(kvm_enable_host_haltpoll, NULL, 1);
904 preempt_enable();
905 }
> 906 }
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
PATCH v2 3/3 cpuidle-haltpoll: disable host side polling when kvm virtualized
In reply to Marcelo TosattiWhen performing guest side polling, it is not necessary to also perform host side polling.
So disable host side polling, via the new MSR interface, when loading cpuidle-haltpoll driver.
---
v2: remove extra "}"
arch/x86/Kconfig | 7 +++++ arch/x86/include/asm/cpuidle_haltpoll.h | 8 ++++++ arch/x86/kernel/kvm.c | 40 ++++++++++++++++++++++++++++++++ drivers/cpuidle/cpuidle-haltpoll.c | 9 ++++++- include/linux/cpuidle_haltpoll.h | 16 ++++++++++++ 5 files changed, 79 insertions(+), 1 deletion(-)
arch/x86/include/asm/cpuidle_haltpoll.h new file +8 -0
@@ -0,0 +1,8 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ARCH_HALTPOLL_H +#define _ARCH_HALTPOLL_H + +void arch_haltpoll_enable (void ); +void arch_haltpoll_disable (void ); + +#endif
drivers/cpuidle/cpuidle-haltpoll.c +8 -1
@@ -15,6 +15,7 @@#include <linux/module.h> #include <linux/timekeeping.h> #include <linux/sched/idle.h> +#include <linux/cpuidle_haltpoll.h> #define CREATE_TRACE_POINTS #include "cpuidle-haltpoll-trace.h"
@@ -157,11 +158,17 @@static int __init haltpoll_init (void ) { -return cpuidle_register (&haltpoll_driver,NULL ); +int ret =cpuidle_register (&haltpoll_driver,NULL ); + +if (ret ==0 ) +arch_haltpoll_enable (); + +return ret; } static void __exit haltpoll_exit(void) { +arch_haltpoll_disable ();cpuidle_unregister (&haltpoll_driver); }
include/linux/cpuidle_haltpoll.h new file +16 -0
@@ -0,0 +1,16 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _CPUIDLE_HALTPOLL_H +#define _CPUIDLE_HALTPOLL_H + +#ifdef CONFIG_ARCH_CPUIDLE_HALTPOLL +#include <asm/cpuidle_haltpoll.h> +#else +static inline void arch_haltpoll_enable (void ) +{ +} + +static inline void arch_haltpoll_disable (void ) +{ +} +#endif +#endif
arch/x86/Kconfig +7 -0
@@ -787,6 +787,7 @@ bool "KVM Guest support (including kvmclock)" depends on PARAVIRT select PARAVIRT_CLOCK + select ARCH_CPUIDLE_HALTPOLL default y ---help--- This option enables various optimizations for running under the KVM
@@ -795,6 +796,12 @@ underlying device model, the host provides the guest with timing infrastructure such as time of day, and system time +config ARCH_CPUIDLE_HALTPOLL + def_bool n + prompt "Disable host haltpoll when loading haltpoll driver" + help + If virtualized under KVM, disable host haltpoll. + config PVH bool "Support for running PVH guests" ---help---
arch/x86/kernel/kvm.c +40 -0
@@ -853,3 +853,43 @@ }#endif /* CONFIG_PARAVIRT_SPINLOCKS */ + +#ifdef CONFIG_ARCH_CPUIDLE_HALTPOLL + +void kvm_disable_host_haltpoll (void *i) +{ +wrmsrl (MSR_KVM_POLL_CONTROL ,0 ); +} + +void kvm_enable_host_haltpoll (void *i) +{ +wrmsrl (MSR_KVM_POLL_CONTROL ,1 ); +} + +void arch_haltpoll_enable (void ) +{ +if (!kvm_para_has_feature (KVM_FEATURE_POLL_CONTROL )) +return ; + +preempt_disable (); +/* Enabling guest halt poll disables host halt poll */ +kvm_disable_host_haltpoll (NULL ); +smp_call_function (kvm_disable_host_haltpoll,NULL ,1 ); +preempt_enable (); +} +EXPORT_SYMBOL_GPL (arch_haltpoll_enable); + +void arch_haltpoll_disable (void ) +{ +if (!kvm_para_has_feature (KVM_FEATURE_POLL_CONTROL )) +return ; + +preempt_disable (); +/* Enabling guest halt poll disables host halt poll */ +kvm_enable_host_haltpoll (NULL ); +smp_call_function (kvm_enable_host_haltpoll,NULL ,1 ); +preempt_enable (); +} + +EXPORT_SYMBOL_GPL (arch_haltpoll_disable); +#endif
Re: [patch v2 3/3] cpuidle-haltpoll: disable host side polling when kvm virtualized
Quoted 126 lines · Marcelo Tosatti
When performing guest side polling, it is not necessary to also perform host side polling.
So disable host side polling, via the new MSR interface, when loading cpuidle-haltpoll driver.
Signed-off-by Marcelo Tosatti <[email protected]>---
v2: remove extra "}"
arch/x86/Kconfig | 7 +++++ arch/x86/include/asm/cpuidle_haltpoll.h | 8 ++++++ arch/x86/kernel/kvm.c | 40 ++++++++++++++++++++++++++++++++ drivers/cpuidle/cpuidle-haltpoll.c | 9 ++++++- include/linux/cpuidle_haltpoll.h | 16 ++++++++++++ 5 files changed, 79 insertions(+), 1 deletion(-)Index:
linux-2.6.git/arch/x86/include/asm/cpuidle_haltpoll.h
===================================================================arch/x86/include/asm/cpuidle_haltpoll.h new file +8
-0@@ -0,0 +1,8 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ARCH_HALTPOLL_H +#define _ARCH_HALTPOLL_H + +void arch_haltpoll_enable (void ); +void arch_haltpoll_disable (void ); + +#endif Index:
linux-2.6.git/drivers/cpuidle/cpuidle-haltpoll.c
===================================================================drivers/cpuidle/cpuidle-haltpoll.c +8
-1@@ -15,6 +15,7 @@#include <linux/module.h> #include <linux/timekeeping.h> #include <linux/sched/idle.h> +#include <linux/cpuidle_haltpoll.h> #define CREATE_TRACE_POINTS #include "cpuidle-haltpoll-trace.h" @@ -157,11 +158,17 @@static int __init haltpoll_init (void ) { -return cpuidle_register (&haltpoll_driver,NULL ); +int ret =cpuidle_register (&haltpoll_driver,NULL ); + +if (ret ==0 ) +arch_haltpoll_enable (); + +return ret; } static void __exit haltpoll_exit(void) { +arch_haltpoll_disable ();cpuidle_unregister (&haltpoll_driver); }Index:
linux-2.6.git/include/linux/cpuidle_haltpoll.h
===================================================================include/linux/cpuidle_haltpoll.h new file +16
-0@@ -0,0 +1,16 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _CPUIDLE_HALTPOLL_H +#define _CPUIDLE_HALTPOLL_H + +#ifdef CONFIG_ARCH_CPUIDLE_HALTPOLL +#include <asm/cpuidle_haltpoll.h> +#else +static inline void arch_haltpoll_enable (void ) +{ +} + +static inline void arch_haltpoll_disable (void ) +{ +} +#endif +#endif Index: linux-2.6.git/arch/x86/Kconfig
===================================================================arch/x86/Kconfig +7
-0@@ -787,6 +787,7 @@ bool "KVM Guest support (including kvmclock)" depends on PARAVIRT select PARAVIRT_CLOCK + select ARCH_CPUIDLE_HALTPOLL default y ---help--- This option enables various optimizations for running under the KVM@@ -795,6 +796,12 @@ underlying device model, the host provides the guest with timing infrastructure such as time of day, and system time +config ARCH_CPUIDLE_HALTPOLL + def_bool n + proUmpt "Disable host haltpoll when loading haltpoll driver" + help + If virtualized under KVM, disable host haltpoll. + config PVH bool "Support for running PVH guests" ---help---Index:
linux-2.6.git/arch/x86/kernel/kvm.c
===================================================================arch/x86/kernel/kvm.c +8
-0@@ -853,3 +853,43 @@ }#endif /* CONFIG_PARAVIRT_SPINLOCKS */ + +#ifdef CONFIG_ARCH_CPUIDLE_HALTPOLL + +void kvm_disable_host_haltpoll (void *i) +{ +wrmsrl (MSR_KVM_POLL_CONTROL ,0 ); +} +
arch/x86/kernel/kvm.c line 873
Quoted 10 lines
arch/x86/kernel/kvm.c +10
-0+void kvm_enable_host_haltpoll (void *i) +{ +wrmsrl (MSR_KVM_POLL_CONTROL ,1 ); +} + +void arch_haltpoll_enable (void ) +{ +if (!kvm_para_has_feature (KVM_FEATURE_POLL_CONTROL )) +return ; +
Perhaps warn the user when failing to disable host poll e.g.:
if (!kvm_para_has_feature(KVM_FEATURE_POLL_CONTROL)) {
pr_warn_once("haltpoll: Failed to disable host halt polling\n");
return;
}But I wonder whether we should fail to load cpuidle-haltpoll when host halt polling can't be disabled[*]? That is to avoid polling in both host and guest and *possibly* avoid chances for performance regressions when running on older hypervisors?
[*] with guest still able load with lack of host polling control via modparam
Quoted 23 lines
arch/x86/kernel/kvm.c +22
-0+preempt_disable (); +/* Enabling guest halt poll disables host halt poll */ +kvm_disable_host_haltpoll (NULL ); +smp_call_function (kvm_disable_host_haltpoll,NULL ,1 ); + preempt_enable(); +} +EXPORT_SYMBOL_GPL (arch_haltpoll_enable); + +void arch_haltpoll_disable (void ) +{ +if (!kvm_para_has_feature (KVM_FEATURE_POLL_CONTROL )) +return ; + +preempt_disable (); +/* Enabling guest halt poll disables host halt poll */ +kvm_enable_host_haltpoll (NULL ); +smp_call_function (kvm_enable_host_haltpoll,NULL ,1 ); +preempt_enable (); +} + +EXPORT_SYMBOL_GPL (arch_haltpoll_disable); +#endif
Re: [patch v2 3/3] cpuidle-haltpoll: disable host side polling when kvm virtualized
Hello,
Quoted 4 lines · Joao Martins
But I wonder whether we should fail to load cpuidle-haltpoll when host halt polling can't be disabled[*]? That is to avoid polling in both host and guest and *possibly* avoid chances for performance regressions when running on older hypervisors?
I don't think it's necessary: that would force an upgrade of the host KVM version in order to use the guest haltpoll feature with an upgraded guest kernel that can use the guest haltpoll.
The guest haltpoll is self contained in the guest, so there's no reason to prevent that by design or to force upgrade of the KVM host version. It'd be more than enough to reload kvm.ko in the host with the host haltpoll set to zero with the module parameter already available, to achieve the same runtime without requiring a forced host upgrade.
The warning however sounds sensible.
Thanks,
Andrea
Re: [patch v2 3/3] cpuidle-haltpoll: disable host side polling when kvm virtualized
Quoted 12 lines · Andrea Arcangeli
Hello,
Quoted 4 lines · Joao Martins
But I wonder whether we should fail to load cpuidle-haltpoll when host halt polling can't be disabled[*]? That is to avoid polling in both host and guest and *possibly* avoid chances for performance regressions when running on older hypervisors?
I don't think it's necessary: that would force an upgrade of the host KVM version in order to use the guest haltpoll feature with an upgraded guest kernel that can use the guest haltpoll.
Hence why I was suggesting a *guest* cpuidle-haltpoll module parameter to still allow it to load or otherwise (or allow guest to pick).
Quoted 7 lines
The guest haltpoll is self contained in the guest, so there's no reason to prevent that by design or to force upgrade of the KVM host version. It'd be more than enough to reload kvm.ko in the host with the host haltpoll set to zero with the module parameter already available, to achieve the same runtime without requiring a forced host upgrade.
It's just with the new driver we unilaterally poll on both sides, just felt I would point it out should this raise unattended performance side effects ;)
Quoted 2 lines
The warning however sounds sensible.
Cool.
Joao
Re: [patch v2 3/3] cpuidle-haltpoll: disable host side polling when kvm virtualized
Quoted 9 lines · Joao Martins
Quoted 8 lines · Andrea Arcangeli
Hello,
Quoted 4 lines · Joao Martins
But I wonder whether we should fail to load cpuidle-haltpoll when host halt polling can't be disabled[*]? That is to avoid polling in both host and guest and *possibly* avoid chances for performance regressions when running on older hypervisors?
Quoted 7 lines
Quoted 4 lines
I don't think it's necessary: that would force an upgrade of the host KVM version in order to use the guest haltpoll feature with an upgraded guest kernel that can use the guest haltpoll.
Hence why I was suggesting a *guest* cpuidle-haltpoll module parameter to still allow it to load or otherwise (or allow guest to pick).
By 'still allow it to load', I meant specifically to handle the case when host polling control is not supported and what to do in that case.
Quoted 10 lines
Quoted 7 lines
The guest haltpoll is self contained in the guest, so there's no reason to prevent that by design or to force upgrade of the KVM host version. It'd be more than enough to reload kvm.ko in the host with the host haltpoll set to zero with the module parameter already available, to achieve the same runtime without requiring a forced host upgrade.
It's just with the new driver we unilaterally poll on both sides, just felt I would point it out should this raise unattended performance side effects ;)
To be clear: by 'unilaterally' I was trying to refer to hosts KVM without polling control (which is safe to say that it is the majority atm?).
Alternatively, there's always the option that if guest sees any issues on that case (with polling on both sides=, that it can always blacklist cpuidle-haltpoll. But may this is not an issue and perhaps majority of users still observes benefit when polling is enabled on guest and host.
Quoted 7 lines
Quoted 2 lines
The warning however sounds sensible.
Cool.
Joao
Re: [patch v2 3/3] cpuidle-haltpoll: disable host side polling when kvm virtualized
Quoted 14 lines · Joao Martins
Quoted 13 lines · Joao Martins
Quoted 12 lines · Andrea Arcangeli
Hello,
Quoted 4 lines · Joao Martins
But I wonder whether we should fail to load cpuidle-haltpoll when host halt polling can't be disabled[*]? That is to avoid polling in both host and guest and *possibly* avoid chances for performance regressions when running on older hypervisors?
I don't think it's necessary: that would force an upgrade of the host KVM version in order to use the guest haltpoll feature with an upgraded guest kernel that can use the guest haltpoll.
Quoted 5 lines
Quoted 3 lines
Hence why I was suggesting a *guest* cpuidle-haltpoll module parameter to still allow it to load or otherwise (or allow guest to pick).
By 'still allow it to load', I meant specifically to handle the case when host polling control is not supported and what to do in that case.
All right, we could add a force=1 parameter to force loading as an opt-in (and fail load by default with force=0).
Quoted 10 lines
Quoted 10 lines
Quoted 7 lines
The guest haltpoll is self contained in the guest, so there's no reason to prevent that by design or to force upgrade of the KVM host version. It'd be more than enough to reload kvm.ko in the host with the host haltpoll set to zero with the module parameter already available, to achieve the same runtime without requiring a forced host upgrade.
It's just with the new driver we unilaterally poll on both sides, just felt I would point it out should this raise unattended performance side effects ;)
Quoted 7 lines
To be clear: by 'unilaterally' I was trying to refer to hosts KVM without polling control (which is safe to say that it is the majority atm?).
Alternatively, there's always the option that if guest sees any issues on that case (with polling on both sides=, that it can always blacklist cpuidle-haltpoll. But may this is not an issue and perhaps majority of users still observes benefit when polling is enabled on guest and host.
It should be workload dependent if it increases performance to haltpoll both in both host kernel and guest, the only sure cons is that it'd burn some more energy..
By default the cpuidle-haltpoll driver shouldn't get loaded if it's built as a module (the expectation is the default will be =m), and it can still easily disabled with rmmod if it hurts performance.
So the policy if to activate guest haltpoll or not will still reside in the guest userland: the guest userland code or guest admin, without a force parameter has to decide if to load or not to load the module, with the force=1 parameter it'll have to decide if to load it with =0 or =1 (or load it first with =0 and then decide if to try to load it again with =1 which would be the benefit the force parameter provides). To decide if to load or not to load it, the guest userland could check if there's no support for disabling the host haltpoll, which can be verified with rdmsr too.
# rdmsr 0x4b564d05 rdmsr: CPU 0 cannot read MSR 0x4b564d05
Adding a force=1 parameter to force loading will just add a few lines more of kernel code, I'm neutral on that but it looks fine either ways.
Thanks,
Andrea
Re: [patch v2 3/3] cpuidle-haltpoll: disable host side polling when kvm virtualized
In reply to Andrea ArcangeliQuoted 12 lines · Andrea Arcangeli
Hello,
Quoted 4 lines · Joao Martins
But I wonder whether we should fail to load cpuidle-haltpoll when host halt polling can't be disabled[*]? That is to avoid polling in both host and guest and *possibly* avoid chances for performance regressions when running on older hypervisors?
I don't think it's necessary: that would force an upgrade of the host KVM version in order to use the guest haltpoll feature with an upgraded guest kernel that can use the guest haltpoll.
Quoted 8 lines
The guest haltpoll is self contained in the guest, so there's no reason to prevent that by design or to force upgrade of the KVM host version. It'd be more than enough to reload kvm.ko in the host with the host haltpoll set to zero with the module parameter already available, to achieve the same runtime without requiring a forced host upgrade.
The warning however sounds sensible.
Alright, added a warning:
void arch_haltpoll_enable(void)
{
if (!kvm_para_has_feature(KVM_FEATURE_POLL_CONTROL)) {
WARN_ONCE(1, "kvm: host does not support poll control, can't "
"disable host polling (host upgrade "
" recommended).\n"
return;
}Thanks
Re: [patch v2 3/3] cpuidle-haltpoll: disable host side polling when kvm virtualized
In reply to Joao MartinsHi Joao,
Quoted 145 lines · Joao Martins
Quoted 126 lines · Marcelo Tosatti
When performing guest side polling, it is not necessary to also perform host side polling.
So disable host side polling, via the new MSR interface, when loading cpuidle-haltpoll driver.
Signed-off-by Marcelo Tosatti <[email protected]>---
v2: remove extra "}"
arch/x86/Kconfig | 7 +++++ arch/x86/include/asm/cpuidle_haltpoll.h | 8 ++++++ arch/x86/kernel/kvm.c | 40 ++++++++++++++++++++++++++++++++ drivers/cpuidle/cpuidle-haltpoll.c | 9 ++++++- include/linux/cpuidle_haltpoll.h | 16 ++++++++++++ 5 files changed, 79 insertions(+), 1 deletion(-)Index:
linux-2.6.git/arch/x86/include/asm/cpuidle_haltpoll.h
===================================================================arch/x86/include/asm/cpuidle_haltpoll.h new file +8
-0@@ -0,0 +1,8 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ARCH_HALTPOLL_H +#define _ARCH_HALTPOLL_H + +void arch_haltpoll_enable (void ); +void arch_haltpoll_disable (void ); + +#endif Index:
linux-2.6.git/drivers/cpuidle/cpuidle-haltpoll.c
===================================================================drivers/cpuidle/cpuidle-haltpoll.c +8
-1@@ -15,6 +15,7 @@#include <linux/module.h> #include <linux/timekeeping.h> #include <linux/sched/idle.h> +#include <linux/cpuidle_haltpoll.h> #define CREATE_TRACE_POINTS #include "cpuidle-haltpoll-trace.h" @@ -157,11 +158,17 @@static int __init haltpoll_init (void ) { -return cpuidle_register (&haltpoll_driver,NULL ); +int ret =cpuidle_register (&haltpoll_driver,NULL ); + +if (ret ==0 ) +arch_haltpoll_enable (); + +return ret; } static void __exit haltpoll_exit(void) { +arch_haltpoll_disable ();cpuidle_unregister (&haltpoll_driver); }Index:
linux-2.6.git/include/linux/cpuidle_haltpoll.h
===================================================================include/linux/cpuidle_haltpoll.h new file +16
-0@@ -0,0 +1,16 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _CPUIDLE_HALTPOLL_H +#define _CPUIDLE_HALTPOLL_H + +#ifdef CONFIG_ARCH_CPUIDLE_HALTPOLL +#include <asm/cpuidle_haltpoll.h> +#else +static inline void arch_haltpoll_enable (void ) +{ +} + +static inline void arch_haltpoll_disable (void ) +{ +} +#endif +#endif Index: linux-2.6.git/arch/x86/Kconfig
===================================================================arch/x86/Kconfig +7
-0@@ -787,6 +787,7 @@ bool "KVM Guest support (including kvmclock)" depends on PARAVIRT select PARAVIRT_CLOCK + select ARCH_CPUIDLE_HALTPOLL default y ---help--- This option enables various optimizations for running under the KVM@@ -795,6 +796,12 @@ underlying device model, the host provides the guest with timing infrastructure such as time of day, and system time +config ARCH_CPUIDLE_HALTPOLL + def_bool n + proUmpt "Disable host haltpoll when loading haltpoll driver" + help + If virtualized under KVM, disable host haltpoll. + config PVH bool "Support for running PVH guests" ---help---Index:
linux-2.6.git/arch/x86/kernel/kvm.c
===================================================================arch/x86/kernel/kvm.c +8
-0@@ -853,3 +853,43 @@ }#endif /* CONFIG_PARAVIRT_SPINLOCKS */ + +#ifdef CONFIG_ARCH_CPUIDLE_HALTPOLL + +void kvm_disable_host_haltpoll (void *i) +{ +wrmsrl (MSR_KVM_POLL_CONTROL ,0 ); +} +arch/x86/kernel/kvm.c line 873
Quoted 10 lines
arch/x86/kernel/kvm.c +10
-0+void kvm_enable_host_haltpoll (void *i) +{ +wrmsrl (MSR_KVM_POLL_CONTROL ,1 ); +} + +void arch_haltpoll_enable (void ) +{ +if (!kvm_para_has_feature (KVM_FEATURE_POLL_CONTROL )) +return ; +Perhaps warn the user when failing to disable host poll e.g.:
if (!kvm_para_has_feature(KVM_FEATURE_POLL_CONTROL)) { pr_warn_once("haltpoll: Failed to disable host halt polling\n"); return; }
Quoted 6 lines
But I wonder whether we should fail to load cpuidle-haltpoll when host halt polling can't be disabled[*]? That is to avoid polling in both host and guest and *possibly* avoid chances for performance regressions when running on older hypervisors?
[*] with guest still able load with lack of host polling control via modparam
I don't think so. First, the driver reduces the halt poll interval when the system is idle, so overhead is reduced to zero in such cases.
Moreover, when both host and guest side polling are performed, power might be wasted, but i haven't seen a significant performance regression caused by it.
Thanks.
Quoted 24 lines
Quoted 24 lines
arch/x86/kernel/kvm.c +22
-0+preempt_disable (); +/* Enabling guest halt poll disables host halt poll */ +kvm_disable_host_haltpoll (NULL ); +smp_call_function (kvm_disable_host_haltpoll,NULL ,1 ); + preempt_enable(); +} +EXPORT_SYMBOL_GPL (arch_haltpoll_enable); + +void arch_haltpoll_disable (void ) +{ +if (!kvm_para_has_feature (KVM_FEATURE_POLL_CONTROL )) +return ; + +preempt_disable (); +/* Enabling guest halt poll disables host halt poll */ +kvm_enable_host_haltpoll (NULL ); +smp_call_function (kvm_enable_host_haltpoll,NULL ,1 ); +preempt_enable (); +} + +EXPORT_SYMBOL_GPL (arch_haltpoll_disable); +#endif
Quoted 33 lines · Marcelo Tosatti
The cpuidle-haltpoll driver allows the guest vcpus to poll for a specified amount of time before halting. This provides the following benefits to host side polling:
1) The POLL flag is set while polling is performed, which allows a remote vCPU to avoid sending an IPI (and the associated cost of handling the IPI) when performing a wakeup.
2) The HLT VM-exit cost can be avoided.
The downside of guest side polling is that polling is performed even with other runnable tasks in the host.
Results comparing
halt_poll_nsand server/client application where a small packet is ping-ponged:host --> 31.33 halt_poll_ns=300000 / no guest busy spin --> 33.40 (93.8%) halt_poll_ns=0 / guest_halt_poll_ns=300000 --> 32.73 (95.7%)For the SAP HANA benchmarks (where
idle_spinis a parameter of the previous version of the patch, results should be the same):hpns ==
halt_poll_nsidle_spin=0/ idle_spin=800/ idle_spin=0/ hpns=200000 hpns=0 hpns=800000 DeleteC06T03 (100 thread) 1.76 1.71 (-3%) 1.78 (+1%) InsertC16T02 (100 thread) 2.14 2.07 (-3%) 2.18 (+1.8%) DeleteC00T01 (1 thread) 1.34 1.28 (-4.5%) 1.29 (-3.7%) UpdateC00T03 (1 thread) 4.72 4.18 (-12%) 4.53 (-5%)
Quoted 8 lines
V2:
- Move from x86 to generic code (Paolo/Christian).
- Add auto-tuning logic (Paolo).
- Add MSR to disable host side polling (Paolo).
First of all, please CC power management patches (including cpuidle, cpufreq etc) to [email protected] (there are people on that list who may want to see your changes before they go in) and CC cpuidle material (in particular) to Peter Zijlstra.
Second, I'm not a big fan of this approach to be honest, as it kind of is a driver trying to play the role of a governor.
We have a "polling state" already that could be used here in principle so I wonder what would be wrong with that. Also note that there seems to be at least some code duplication between your code and the "polling state" implementation, so maybe it would be possible to do some things in a common way?
Quoted 42 lines · Rafael J. Wysocki
Quoted 33 lines · Marcelo Tosatti
The cpuidle-haltpoll driver allows the guest vcpus to poll for a specified amount of time before halting. This provides the following benefits to host side polling:
1) The POLL flag is set while polling is performed, which allows a remote vCPU to avoid sending an IPI (and the associated cost of handling the IPI) when performing a wakeup.
2) The HLT VM-exit cost can be avoided.
The downside of guest side polling is that polling is performed even with other runnable tasks in the host.
Results comparing
halt_poll_nsand server/client application where a small packet is ping-ponged:host --> 31.33 halt_poll_ns=300000 / no guest busy spin --> 33.40 (93.8%) halt_poll_ns=0 / guest_halt_poll_ns=300000 --> 32.73 (95.7%)For the SAP HANA benchmarks (where
idle_spinis a parameter of the previous version of the patch, results should be the same):hpns ==
halt_poll_nsidle_spin=0/ idle_spin=800/ idle_spin=0/ hpns=200000 hpns=0 hpns=800000 DeleteC06T03 (100 thread) 1.76 1.71 (-3%) 1.78 (+1%) InsertC16T02 (100 thread) 2.14 2.07 (-3%) 2.18 (+1.8%) DeleteC00T01 (1 thread) 1.34 1.28 (-4.5%) 1.29 (-3.7%) UpdateC00T03 (1 thread) 4.72 4.18 (-12%) 4.53 (-5%)Quoted 8 lines
V2:
- Move from x86 to generic code (Paolo/Christian).
- Add auto-tuning logic (Paolo).
- Add MSR to disable host side polling (Paolo).
Quoted 4 lines
First of all, please CC power management patches (including cpuidle, cpufreq etc) to [email protected] (there are people on that list who may want to see your changes before they go in) and CC cpuidle material (in particular) to Peter Zijlstra.
Ok, Peter is CC'ed, will include linux-pm@vger in the next patches.
Quoted 2 lines
Second, I'm not a big fan of this approach to be honest, as it kind of is a driver trying to play the role of a governor.
Well, its not trying to choose which idle state to enter, because there is only one idle state to enter when virtualized (HLT).
Quoted 2 lines
We have a "polling state" already that could be used here in principle so I wonder what would be wrong with that.
There is no "target residency" concept in the virtualized use-case (which is what poll_state.c uses to calculate the poll time).
Moreover the cpuidle-haltpoll driver uses an adaptive logic to tune poll time (which appparently does not make sense for poll_state).
The only thing they share is the main loop structure:
"while (!need_resched()) {
cpu_relax();
now = ktime_get();
}"Quoted 4 lines
Also note that
there seems to be at least some code duplication between your code and the "polling state" implementation, so maybe it would be possible to do some things in a common way?
Again, its just the main loop structure that is shared:
there is no target residency in the virtualized case, and we want an adaptive scheme.
Lets think about deduplication: you would have a cpuidle driver, with a fake "target residency".
Now, it makes no sense to use a governor for the virtualized case (again, there is only one idle state: HLT, the host governor is used for the actual idle state decision in the host).
So i fail to see how i would go about integrating these two and what are the advantages of doing so ?
Paolo Bonzini
maintainerQuoted 2 lines · Marcelo Tosatti
There is no "target residency" concept in the virtualized use-case (which is what
poll_state.cuses to calculate the poll time).
Actually there is: it is the cost of a vmexit, and it be calibrated with a very short CPUID loop (e.g. run 100 CPUID instructions and take the smallest TSC interval---it should take less than 50 microseconds, and less than a millisecond even on nested virt).
I think it would make sense to improve poll_state.c to use an adaptive algorithm similar to the one you implemented, which includes optionally allowing to poll for an interval larger than the target residency.
Paolo
Quoted 8 lines · Paolo Bonzini
Quoted 2 lines · Marcelo Tosatti
There is no "target residency" concept in the virtualized use-case (which is what
poll_state.cuses to calculate the poll time).Actually there is: it is the cost of a vmexit, and it be calibrated with a very short CPUID loop (e.g. run 100 CPUID instructions and take the smallest TSC interval---it should take less than 50 microseconds, and less than a millisecond even on nested virt).
For a given application, you want to configure the poll time to the maximum time an event happen after starting the idle procedure. For SAP HANA, that value is between 200us - 800us (most tests require less than 800us, but some require 800us, to significantly avoid the IPIs).
"The target residency is the minimum time the hardware must spend in the given state, including the time needed to enter it (which may be substantial), in order to save more energy than it would save by entering one of the shallower idle states instead."
Clearly these are two different things...
Quoted 5 lines
I think it would make sense to improve
poll_state.cto use an adaptive algorithm similar to the one you implemented, which includes optionally allowing to poll for an interval larger than the target residency.Paolo
Ok, so i'll move the adaptive code to poll_state.c, where the driver selects whether to use target_residency or the adaptive value (based on module parameters).
Not sure if its an adaptible value is desirable for the non virtualized case.
Quoted 47 lines · Rafael J. Wysocki
Quoted 33 lines · Marcelo Tosatti
The cpuidle-haltpoll driver allows the guest vcpus to poll for a specified amount of time before halting. This provides the following benefits to host side polling:
1) The POLL flag is set while polling is performed, which allows a remote vCPU to avoid sending an IPI (and the associated cost of handling the IPI) when performing a wakeup.
2) The HLT VM-exit cost can be avoided.
The downside of guest side polling is that polling is performed even with other runnable tasks in the host.
Results comparing
halt_poll_nsand server/client application where a small packet is ping-ponged:host --> 31.33 halt_poll_ns=300000 / no guest busy spin --> 33.40 (93.8%) halt_poll_ns=0 / guest_halt_poll_ns=300000 --> 32.73 (95.7%)For the SAP HANA benchmarks (where
idle_spinis a parameter of the previous version of the patch, results should be the same):hpns ==
halt_poll_nsidle_spin=0/ idle_spin=800/ idle_spin=0/ hpns=200000 hpns=0 hpns=800000 DeleteC06T03 (100 thread) 1.76 1.71 (-3%) 1.78 (+1%) InsertC16T02 (100 thread) 2.14 2.07 (-3%) 2.18 (+1.8%) DeleteC00T01 (1 thread) 1.34 1.28 (-4.5%) 1.29 (-3.7%) UpdateC00T03 (1 thread) 4.72 4.18 (-12%) 4.53 (-5%)Quoted 8 lines
V2:
- Move from x86 to generic code (Paolo/Christian).
- Add auto-tuning logic (Paolo).
- Add MSR to disable host side polling (Paolo).First of all, please CC power management patches (including cpuidle, cpufreq etc) to [email protected] (there are people on that list who may want to see your changes before they go in) and CC cpuidle material (in particular) to Peter Zijlstra.
Quoted 8 lines
Second, I'm not a big fan of this approach to be honest, as it kind of is a driver trying to play the role of a governor.
We have a "polling state" already that could be used here in principle so I wonder what would be wrong with that. Also note that there seems to be at least some code duplication between your code and the "polling state" implementation, so maybe it would be possible to do some things in a common way?
Hi Rafael,
After modifying poll_state.c to use a generic "poll time" driver callback [1] (since using a variable "target_residency" for that looks really ugly), would need a governor which does:
haltpoll_governor_select_next_state()
if (prev_state was poll and evt happened on prev poll window) -> POLL. if (prev_state == HLT) -> POLL otherwise -> HLT
And a "default_idle" cpuidle driver that:
defaultidle_idle()
if (current_clr_polling_and_test()) {
local_irq_enable();
return index;
}
default_idle();
returnUsing such governor with any other cpuidle driver would be pointless (since it would enter the first state only and therefore not save power).
Not certain about using the default_idle driver with other governors: one would rather use a driver that supports all states on a given machine.
This combination of governor/driver pair, for the sake of sharing the idle loop, seems awkward to me. And fails the governor/driver separation: one will use the pair in practice.
But i have no problem with it, so i'll proceed with that.
Let me know otherwise.
Thanks.
Rafael J. Wysocki
maintainerQuoted 87 lines · Marcelo Tosatti
Quoted 47 lines · Rafael J. Wysocki
Quoted 33 lines · Marcelo Tosatti
The cpuidle-haltpoll driver allows the guest vcpus to poll for a specified amount of time before halting. This provides the following benefits to host side polling:
1) The POLL flag is set while polling is performed, which allows a remote vCPU to avoid sending an IPI (and the associated cost of handling the IPI) when performing a wakeup.
2) The HLT VM-exit cost can be avoided.
The downside of guest side polling is that polling is performed even with other runnable tasks in the host.
Results comparing
halt_poll_nsand server/client application where a small packet is ping-ponged:host --> 31.33 halt_poll_ns=300000 / no guest busy spin --> 33.40 (93.8%) halt_poll_ns=0 / guest_halt_poll_ns=300000 --> 32.73 (95.7%)For the SAP HANA benchmarks (where
idle_spinis a parameter of the previous version of the patch, results should be the same):hpns ==
halt_poll_nsidle_spin=0/ idle_spin=800/ idle_spin=0/ hpns=200000 hpns=0 hpns=800000 DeleteC06T03 (100 thread) 1.76 1.71 (-3%) 1.78 (+1%) InsertC16T02 (100 thread) 2.14 2.07 (-3%) 2.18 (+1.8%) DeleteC00T01 (1 thread) 1.34 1.28 (-4.5%) 1.29 (-3.7%) UpdateC00T03 (1 thread) 4.72 4.18 (-12%) 4.53 (-5%)Quoted 8 lines
V2:
- Move from x86 to generic code (Paolo/Christian).
- Add auto-tuning logic (Paolo).
- Add MSR to disable host side polling (Paolo).First of all, please CC power management patches (including cpuidle, cpufreq etc) to [email protected] (there are people on that list who may want to see your changes before they go in) and CC cpuidle material (in particular) to Peter Zijlstra.
Quoted 8 lines
Second, I'm not a big fan of this approach to be honest, as it kind of is a driver trying to play the role of a governor.
We have a "polling state" already that could be used here in principle so I wonder what would be wrong with that. Also note that there seems to be at least some code duplication between your code and the "polling state" implementation, so maybe it would be possible to do some things in a common way?
Hi Rafael,
After modifying
poll_state.cto use a generic "poll time" driver callback [1] (since using a variable "target_residency" for that looks really ugly), would need a governor which does:
haltpoll_governor_select_next_state()if (prev_state was poll and evt happened on prev poll window) -> POLL. if (prev_state == HLT) -> POLL otherwise -> HLTAnd a "
default_idle" cpuidle driver that:
defaultidle_idle()if (current_clr_polling_and_test()) { local_irq_enable(); return index; } default_idle(); returnUsing such governor with any other cpuidle driver would be pointless (since it would enter the first state only and therefore not save power).
Not certain about using the
default_idledriver with other governors: one would rather use a driver that supports all states on a given machine.
Quoted 8 lines
This combination of governor/driver pair, for the sake of sharing the idle loop, seems awkward to me. And fails the governor/driver separation: one will use the pair in practice.
But i have no problem with it, so i'll proceed with that.
Let me know otherwise.
If my understanding of your argumentation is correct, it is only necessary to take the default_idle_call() branch of cpuidle_idle_call() in the VM case, so it should be sufficient to provide a suitable default_idle_call() which is what you seem to be trying to do.
I might have been confused by the terminology used in the patch series if that's the case.
Also, if that's the case, this is not cpuidle matter really. It is a matter of providing a better default_idle_call() for the arch at hand.
Thanks,
Rafael
Quoted 95 lines · Rafael J. Wysocki
Quoted 94 lines · Marcelo Tosatti
Quoted 47 lines · Rafael J. Wysocki
Quoted 33 lines · Marcelo Tosatti
The cpuidle-haltpoll driver allows the guest vcpus to poll for a specified amount of time before halting. This provides the following benefits to host side polling:
1) The POLL flag is set while polling is performed, which allows a remote vCPU to avoid sending an IPI (and the associated cost of handling the IPI) when performing a wakeup.
2) The HLT VM-exit cost can be avoided.
The downside of guest side polling is that polling is performed even with other runnable tasks in the host.
Results comparing
halt_poll_nsand server/client application where a small packet is ping-ponged:host --> 31.33 halt_poll_ns=300000 / no guest busy spin --> 33.40 (93.8%) halt_poll_ns=0 / guest_halt_poll_ns=300000 --> 32.73 (95.7%)For the SAP HANA benchmarks (where
idle_spinis a parameter of the previous version of the patch, results should be the same):hpns ==
halt_poll_nsidle_spin=0/ idle_spin=800/ idle_spin=0/ hpns=200000 hpns=0 hpns=800000 DeleteC06T03 (100 thread) 1.76 1.71 (-3%) 1.78 (+1%) InsertC16T02 (100 thread) 2.14 2.07 (-3%) 2.18 (+1.8%) DeleteC00T01 (1 thread) 1.34 1.28 (-4.5%) 1.29 (-3.7%) UpdateC00T03 (1 thread) 4.72 4.18 (-12%) 4.53 (-5%)Quoted 8 lines
V2:
- Move from x86 to generic code (Paolo/Christian).
- Add auto-tuning logic (Paolo).
- Add MSR to disable host side polling (Paolo).First of all, please CC power management patches (including cpuidle, cpufreq etc) to [email protected] (there are people on that list who may want to see your changes before they go in) and CC cpuidle material (in particular) to Peter Zijlstra.
Quoted 8 lines
Second, I'm not a big fan of this approach to be honest, as it kind of is a driver trying to play the role of a governor.
We have a "polling state" already that could be used here in principle so I wonder what would be wrong with that. Also note that there seems to be at least some code duplication between your code and the "polling state" implementation, so maybe it would be possible to do some things in a common way?
Hi Rafael,
After modifying
poll_state.cto use a generic "poll time" driver callback [1] (since using a variable "target_residency" for that looks really ugly), would need a governor which does:
haltpoll_governor_select_next_state()if (prev_state was poll and evt happened on prev poll window) -> POLL. if (prev_state == HLT) -> POLL otherwise -> HLTAnd a "
default_idle" cpuidle driver that:
defaultidle_idle()if (current_clr_polling_and_test()) { local_irq_enable(); return index; } default_idle(); returnUsing such governor with any other cpuidle driver would be pointless (since it would enter the first state only and therefore not save power).
Not certain about using the
default_idledriver with other governors: one would rather use a driver that supports all states on a given machine.This combination of governor/driver pair, for the sake of sharing the idle loop, seems awkward to me. And fails the governor/driver separation: one will use the pair in practice.
But i have no problem with it, so i'll proceed with that.
Quoted 7 lines
Quoted 1 line
Let me know otherwise.
If my understanding of your argumentation is correct, it is only necessary to take the
default_idle_call()branch ofcpuidle_idle_call()in the VM case, so it should be sufficient to provide a suitabledefault_idle_call()which is what you seem to be trying to do.
In the VM case, we need to poll before actually halting (this is because its tricky to implement MWAIT in guests, so polling for some amount of time allows the IPI avoidance optimization,
see trace_sched_wake_idle_without_ipi, to take place).
The amount of time we poll is variable and adjusted (see adjust_haltpoll_ns in the patchset).
Quoted 5 lines
I might have been confused by the terminology used in the patch series if that's the case.
Also, if that's the case, this is not cpuidle matter really. It is a matter of providing a better
default_idle_call()for the arch at hand.
Peter Zijlstra suggested a cpuidle driver for this.
Also, other architectures will use the same "poll before exiting to VM" logic (so we'd rather avoid duplicating this code): PPC, x86, S/390, MIPS... So in my POV it makes sense to unify this.
So, back to your initial suggestion:
Q) "Can you unify code with poll_state.c?"
A) Yes, but it requires a new governor, which seems overkill and unfit for the purpose.
Moreover, the logic in menu to decide whether its necessary or not to stop sched tick is useful for us (so a default_idle_call is not sufficient), because the cost of enabling/disabling the sched tick is high on VMs.
So i'll fix the comments of the cpuidle driver (which everyone seems to agree with, except your understandable distate for it) and repost.
Rafael J. Wysocki
maintainerQuoted 113 lines · Marcelo Tosatti
Quoted 95 lines · Rafael J. Wysocki
Quoted 94 lines · Marcelo Tosatti
Quoted 47 lines · Rafael J. Wysocki
Quoted 33 lines · Marcelo Tosatti
The cpuidle-haltpoll driver allows the guest vcpus to poll for a specified amount of time before halting. This provides the following benefits to host side polling:
1) The POLL flag is set while polling is performed, which allows a remote vCPU to avoid sending an IPI (and the associated cost of handling the IPI) when performing a wakeup.
2) The HLT VM-exit cost can be avoided.
The downside of guest side polling is that polling is performed even with other runnable tasks in the host.
Results comparing
halt_poll_nsand server/client application where a small packet is ping-ponged:host --> 31.33 halt_poll_ns=300000 / no guest busy spin --> 33.40 (93.8%) halt_poll_ns=0 / guest_halt_poll_ns=300000 --> 32.73 (95.7%)For the SAP HANA benchmarks (where
idle_spinis a parameter of the previous version of the patch, results should be the same):hpns ==
halt_poll_nsidle_spin=0/ idle_spin=800/ idle_spin=0/ hpns=200000 hpns=0 hpns=800000 DeleteC06T03 (100 thread) 1.76 1.71 (-3%) 1.78 (+1%) InsertC16T02 (100 thread) 2.14 2.07 (-3%) 2.18 (+1.8%) DeleteC00T01 (1 thread) 1.34 1.28 (-4.5%) 1.29 (-3.7%) UpdateC00T03 (1 thread) 4.72 4.18 (-12%) 4.53 (-5%)Quoted 8 lines
V2:
- Move from x86 to generic code (Paolo/Christian).
- Add auto-tuning logic (Paolo).
- Add MSR to disable host side polling (Paolo).First of all, please CC power management patches (including cpuidle, cpufreq etc) to [email protected] (there are people on that list who may want to see your changes before they go in) and CC cpuidle material (in particular) to Peter Zijlstra.
Quoted 8 lines
Second, I'm not a big fan of this approach to be honest, as it kind of is a driver trying to play the role of a governor.
We have a "polling state" already that could be used here in principle so I wonder what would be wrong with that. Also note that there seems to be at least some code duplication between your code and the "polling state" implementation, so maybe it would be possible to do some things in a common way?
Hi Rafael,
After modifying
poll_state.cto use a generic "poll time" driver callback [1] (since using a variable "target_residency" for that looks really ugly), would need a governor which does:
haltpoll_governor_select_next_state()if (prev_state was poll and evt happened on prev poll window) -> POLL. if (prev_state == HLT) -> POLL otherwise -> HLTAnd a "
default_idle" cpuidle driver that:
defaultidle_idle()if (current_clr_polling_and_test()) { local_irq_enable(); return index; } default_idle(); returnUsing such governor with any other cpuidle driver would be pointless (since it would enter the first state only and therefore not save power).
Not certain about using the
default_idledriver with other governors: one would rather use a driver that supports all states on a given machine.This combination of governor/driver pair, for the sake of sharing the idle loop, seems awkward to me. And fails the governor/driver separation: one will use the pair in practice.
But i have no problem with it, so i'll proceed with that.
Quoted 7 lines
Quoted 1 line
Let me know otherwise.
If my understanding of your argumentation is correct, it is only necessary to take the
default_idle_call()branch ofcpuidle_idle_call()in the VM case, so it should be sufficient to provide a suitabledefault_idle_call()which is what you seem to be trying to do.In the VM case, we need to poll before actually halting (this is because its tricky to implement MWAIT in guests, so polling for some amount of time allows the IPI avoidance optimization,
seetrace_sched_wake_idle_without_ipi, to take place).The amount of time we poll is variable and adjusted (see
adjust_haltpoll_nsin the patchset).
Quoted 7 lines
Quoted 5 lines
I might have been confused by the terminology used in the patch series if that's the case.
Also, if that's the case, this is not cpuidle matter really. It is a matter of providing a better
default_idle_call()for the arch at hand.Peter Zijlstra suggested a cpuidle driver for this.
So I wonder what his rationale was.
Quoted 3 lines
Also, other architectures will use the same "poll before exiting to VM" logic (so we'd rather avoid duplicating this code): PPC, x86, S/390, MIPS... So in my POV it makes sense to unify this.
The logic is fine IMO, but the implementation here is questionable.
Quoted 10 lines
So, back to your initial suggestion:
Q) "Can you unify code with
poll_state.c?"
A) Yes, but it requires a new governor, which seems overkill and unfit for the purpose.Moreover, the logic in menu to decide whether its necessary or not to stop sched tick is useful for us (so a
default_idle_callis not sufficient), because the cost of enabling/disabling the sched tick is high on VMs.
So in fact you need a governor, but you really only need it to decide whether or not to stop the tick for you.
menu has a quite high overhead for that. :-)
Quoted 2 lines
So i'll fix the comments of the cpuidle driver (which everyone seems to agree with, except your understandable distate for it) and repost.
Quoted 3 lines · Rafael J. Wysocki
Quoted 1 line
Peter Zijlstra suggested a cpuidle driver for this.
So I wonder what his rationale was.
I was thinking we don't need this hard-coded in the idle loop when virt can load a special driver.