x86/hyperv: Initialize clockevents earlier in CPU onlining
Subsystems (5)
PATCH v2 1/1 x86/hyperv: Initialize clockevents earlier in CPU onlining
Hyper-V has historically initialized stimer-based clockevents late in the process of onlining a CPU because clockevents depend on stimer interrupts. In the original Hyper-V design, stimer interrupts generate a VMbus message, so the VMbus machinery must be running first, and VMbus can't be initialized until relatively late. On x86/64, LAPIC timer based clockevents are used during early initialization before VMbus and stimer-based clockevents are ready, and again during CPU offlining after the stimer clockevents have been shut down.
Unfortunately, this design creates problems when offling CPUs for hibernation or other purposes. stimer-based clockevents are shut down relatively early in the offlining process, so clockevents_unbind_device() must be used to fallback to the LAPIC-based clockevents for the remainder of the offlining process. Furthermore, the late initialization and early shutdown of stimer-based clockevents doesn't work well on ARM64 since there is no other timer like the LAPIC to fallback to. So CPU onlining and offlining doesn't work properly.
Fix this by recognizing that stimer Direct Mode is the normal path for newer versions of Hyper-V on x86/64, and the only path on other architectures. With stimer Direct Mode, stimer interrupts don't require any VMbus machinery. stimer clockevents can be initialized and shut down consistent with how it is done for other clockevent devices. While the old VMbus-based stimer interrupts must still be supported for backward compatibility on x86, that mode of operation can be treated as legacy.
So add a new Hyper-V stimer entry in the CPU hotplug state list, and use that new state when in Direct Mode. Update the Hyper-V clocksource driver to allocate and initialize stimer clockevents earlier during boot. Update Hyper-V initialization and the VMbus driver to use this new design. As a result, the LAPIC timer is no longer used during boot or CPU onlining/offlining and clockevents_unbind_device() is not called. But retain the old design as a legacy implementation for older versions of Hyper-V that don't support Direct Mode.
---
Changes in v2 [all from Dexuan Cui]:
* Tweaked hv_stimer_alloc() so errors can be ignored on x86, since a fallback to LAPIC timer clockevents will happen
* Fixed handling of return value from cpuhp_setup_state()
* Fixed hv_stimer_cleanup() to skip redundant call to hv_ce_shutdown() when clockevents_unbind_device() is called
* Fixed hv_stimer_global_cleanup() to allow hv_stimer_free() and the cpuhp state mechanism stop the stimer on all CPUs instead of looping through the CPUs
* Re-added the call to hv_stimer_cleanup() in hv_crash_handler(). A separate patch will fix the problem that hv_synic_cleanup() doesn't actually do anything in many cases because a channel interrupt is assigned to the CPU.
--- arch/x86/hyperv/hv_init.c | 6 ++ drivers/clocksource/hyperv_timer.c | 154 +++++++++++++++++++++++++++++-------- drivers/hv/hv.c | 4 +- drivers/hv/vmbus_drv.c | 30 ++++---- include/clocksource/hyperv_timer.h | 7 +- include/linux/cpuhotplug.h | 1 + 6 files changed, 151 insertions(+), 51 deletions(-)
arch/x86/hyperv/hv_init.c +6 -0
@@ -319,6 +319,12 @@ void __init hyperv_init(void) hypercall_msr.guest_physical_address =vmalloc_to_pfn (hv_hypercall_pg);wrmsrl (HV_X64_MSR_HYPERCALL , hypercall_msr.as_uint64); +/* +* Ignore any errors in setting up stimer clockevents +* as we can run with the LAPIC timer as a fallback. +*/ + (void )hv_stimer_alloc (); +hv_apic_init (); x86_init.pci.arch_init = hv_pci_init;
drivers/clocksource/hyperv_timer.c +124 -30
217 lines not on this page · raw message
drivers/hv/hv.c +2 -2
@@ -202,7 +202,7 @@ int hv_synic_init(unsigned int cpu) {hv_synic_enable_regs (cpu); -hv_stimer_init (cpu); +hv_stimer_legacy_init (cpu,VMBUS_MESSAGE_SINT );return 0 ; }
@@ -277,7 +277,7 @@ int hv_synic_cleanup(unsigned int cpu)if (channel_found && vmbus_connection.conn_state ==CONNECTED )return -EBUSY ; -hv_stimer_cleanup (cpu); +hv_stimer_legacy_cleanup (cpu);hv_synic_disable_regs (cpu);
drivers/hv/vmbus_drv.c +14 -16
@@ -1342,10 +1342,6 @@ static int vmbus_bus_init(void)if (ret) goto err_alloc; - ret =hv_stimer_alloc (VMBUS_MESSAGE_SINT ); -if (ret <0 ) - goto err_alloc; -/* * Initialize the per-cpu interrupt state and stimer state. * Then connect to the host.
@@ -1402,9 +1398,8 @@ static int vmbus_bus_init(void) err_connect:cpuhp_remove_state (hyperv_cpuhp_online); err_cpuhp: -hv_stimer_free (); -err_alloc:hv_synic_free (); +err_alloc:hv_remove_vmbus_irq ();bus_unregister (&hv_bus);
@@ -2317,20 +2312,23 @@ static void hv_crash_handler(struct pt_regs *regs)static int hv_synic_suspend (void ) {/* -* When we reach here, all the non-boot CPUs have been offlined, and -* the stimers on them have been unbound in hv_synic_cleanup() -> +* When we reach here, all the non-boot CPUs have been offlined. +* If we're in a legacy configuration where stimer Direct Mode is +* not enabled, the stimers on the non-boot CPUs have been unbound +* in hv_synic_cleanup() -> hv_stimer_legacy_cleanup() -> * hv_stimer_cleanup() -> clockevents_unbind_device(). * -* hv_synic_suspend() only runs on CPU0 with interrupts disabled. Here -* we do not unbind the stimer on CPU0 because: 1) it's unnecessary -* because the interrupts remain disabled between syscore_suspend() -* and syscore_resume(): see create_image() and resume_target_kernel(); +* hv_synic_suspend() only runs on CPU0 with interrupts disabled. +* Here we do not call hv_stimer_legacy_cleanup() on CPU0 because: +* 1) it's unnecessary as interrupts remain disabled between +* syscore_suspend() and syscore_resume(): see create_image() and +* resume_target_kernel() * 2) the stimer on CPU0 is automatically disabled later by * syscore_suspend() -> timekeeping_suspend() -> tick_suspend() -> ... -* -> clockevents_shutdown() -> ... -> hv_ce_shutdown(); 3) a warning -* would be triggered if we call clockevents_unbind_device(), which -* may sleep, in an interrupts-disabled context. So, we intentionally -* don't call hv_stimer_cleanup(0) here. +* -> clockevents_shutdown() -> ... -> hv_ce_shutdown() +* 3) a warning would be triggered if we call +* clockevents_unbind_device(), which may sleep, in an +* interrupts-disabled context. */ hv_synic_disable_regs (0 );
include/clocksource/hyperv_timer.h +4 -3
@@ -21,10 +21,11 @@#define HV_MIN_DELTA_TICKS 1/* Routines called by the VMbus driver */ -extern int hv_stimer_alloc (int sint); +extern int hv_stimer_alloc (void );extern void hv_stimer_free (void ); -extern void hv_stimer_init (unsigned int cpu); -extern void hv_stimer_cleanup (unsigned int cpu); +extern int hv_stimer_cleanup (unsigned int cpu); +extern void hv_stimer_legacy_init (unsigned int cpu,int sint); +extern void hv_stimer_legacy_cleanup (unsigned int cpu);extern void hv_stimer_global_cleanup (void );extern void hv_stimer0_isr (void );
include/linux/cpuhotplug.h +1 -0
@@ -129,6 +129,7 @@ enum cpuhp_state {CPUHP_AP_ARC_TIMER_STARTING ,CPUHP_AP_RISCV_TIMER_STARTING ,CPUHP_AP_CSKY_TIMER_STARTING , +CPUHP_AP_HYPERV_TIMER_STARTING ,CPUHP_AP_KVM_STARTING ,CPUHP_AP_KVM_ARM_VGIC_INIT_STARTING , CPUHP_AP_KVM_ARM_VGIC_STARTING,
Signature · 1 line
1.8.3.1
Quoted 13 lines
From: Michael Kelley <[email protected]>
Sent: Tuesday, November 12, 2019 5:12 PM
...drivers/clocksource/hyperv_timer.c +5
-0@@ -190,14 +278,20 @@ void hv_stimer_free(void)void hv_stimer_global_cleanup (void ) {int cpu; +/* +* hv_stime_legacy_cleanup() will stop the stimer if Direct +* Mode is not enabled, and fallback to the LAPIC timer. +*/ +for_each_present_cpu (cpu) {
drivers/clocksource/hyperv_timer.c void hv_stimer_free(void) · line 296
Quoted 10 lines
drivers/clocksource/hyperv_timer.c +7
-0+hv_stimer_legacy_cleanup (cpu); } + +/* +* If Direct Mode is enabled, the cpuhp teardown callback +* (hv_stimer_cleanup) will be run on all CPUs to stop the +* stimers. +*/ hv_stimer_free (); }
In the case of direct_mode_enabled == true: When hv_vmbus unloads, vmbus_exit() -> hv_stimer_global_cleanup() -> hv_stimer_free() -> cpuhp_remove_state() disables the direct mode timers.
This does not look symmetric since hv_stimer_alloc() is called before hv_vmbus loads, but I suppose this is not a real issue because hv_vmbus should never unload in practice. :-)
From: Dexuan Cui <[email protected]> Sent: Thursday, November 14, 2019 10:49 PM
Quoted 28 lines
Quoted 13 lines
From: Michael Kelley <[email protected]>
Sent: Tuesday, November 12, 2019 5:12 PM
...drivers/clocksource/hyperv_timer.c +5
-0@@ -190,14 +278,20 @@ void hv_stimer_free(void)void hv_stimer_global_cleanup (void ) {int cpu; +/* +* hv_stime_legacy_cleanup() will stop the stimer if Direct +* Mode is not enabled, and fallback to the LAPIC timer. +*/ +for_each_present_cpu (cpu) {drivers/clocksource/hyperv_timer.c void hv_stimer_free(void) · line 296
Quoted 10 lines
drivers/clocksource/hyperv_timer.c +7
-0+hv_stimer_legacy_cleanup (cpu); } + +/* +* If Direct Mode is enabled, the cpuhp teardown callback +* (hv_stimer_cleanup) will be run on all CPUs to stop the +* stimers. +*/ hv_stimer_free (); }In the case of
direct_mode_enabled== true: Whenhv_vmbusunloads,vmbus_exit()->hv_stimer_global_cleanup()->hv_stimer_free()->cpuhp_remove_state()disables the direct mode timers.
Quoted 3 lines
This does not look symmetric since
hv_stimer_alloc()is called beforehv_vmbusloads, but I suppose this is not a real issue becausehv_vmbusshould never unload in practice. :-)
Thanks for the review. Looking at this a bit more, ideally hv_stimer_global_cleanup() should go away entirely. vmbus_exit() should let the call to cpuhp_remove_state() clean up the stimer on each CPU viahv_synic_cleanup(). But vmbus_exit() shuts down the VMbus first, which
is really the wrong order.
As you say, vmbus_exit() never gets called in practice, so it's not a current issue. But I'll look at fixing this in a future patch so that it makes logical sense.
Michael
Quoted 3 lines
Tested-by Dexuan Cui <[email protected]>Reviewed-by Dexuan Cui <[email protected]>
The following commit has been merged into the x86/hyperv branch of tip:
The following commit has been merged into the x86/hyperv branch of tip:
Commit-ID: 4df4cb9e99f83b70d54bc0e25081ac23cceafcbc Gitweb: https://git.kernel.org/tip/4df4cb9e99f83b70d54bc0e25081ac23cceafcbc Author: Michael Kelley <[email protected]> AuthorDate: Wed, 13 Nov 2019 01:11:49 Committer: Thomas Gleixner <[email protected]> CommitterDate: Fri, 15 Nov 2019 10:33:49 +01:00
x86/hyperv: Initialize clockevents earlier in CPU onlining
Hyper-V has historically initialized stimer-based clockevents late in the process of onlining a CPU because clockevents depend on stimer interrupts. In the original Hyper-V design, stimer interrupts generate a VMbus message, so the VMbus machinery must be running first, and VMbus can't be initialized until relatively late. On x86/64, LAPIC timer based clockevents are used during early initialization before VMbus and stimer-based clockevents are ready, and again during CPU offlining after the stimer clockevents have been shut down.
Unfortunately, this design creates problems when offlining CPUs for hibernation or other purposes. stimer-based clockevents are shut down relatively early in the offlining process, so clockevents_unbind_device() must be used to fallback to the LAPIC-based clockevents for the remainder of the offlining process. Furthermore, the late initialization and early shutdown of stimer-based clockevents doesn't work well on ARM64 since there is no other timer like the LAPIC to fallback to. So CPU onlining and offlining doesn't work properly.
Fix this by recognizing that stimer Direct Mode is the normal path for newer versions of Hyper-V on x86/64, and the only path on other architectures. With stimer Direct Mode, stimer interrupts don't require any VMbus machinery. stimer clockevents can be initialized and shut down consistent with how it is done for other clockevent devices. While the old VMbus-based stimer interrupts must still be supported for backward compatibility on x86, that mode of operation can be treated as legacy.
So add a new Hyper-V stimer entry in the CPU hotplug state list, and use that new state when in Direct Mode. Update the Hyper-V clocksource driver to allocate and initialize stimer clockevents earlier during boot. Update Hyper-V initialization and the VMbus driver to use this new design. As a result, the LAPIC timer is no longer used during boot or CPU onlining/offlining and clockevents_unbind_device() is not called. But retain the old design as a legacy implementation for older versions of Hyper-V that don't support Direct Mode.
--- arch/x86/hyperv/hv_init.c | 6 +- drivers/clocksource/hyperv_timer.c | 154 ++++++++++++++++++++++------ drivers/hv/hv.c | 4 +- drivers/hv/vmbus_drv.c | 30 ++--- include/clocksource/hyperv_timer.h | 7 +- include/linux/cpuhotplug.h | 1 +- 6 files changed, 151 insertions(+), 51 deletions(-)
arch/x86/hyperv/hv_init.c +6 -0
@@ -311,6 +311,12 @@ void __init hyperv_init(void) hypercall_msr.guest_physical_address =vmalloc_to_pfn (hv_hypercall_pg);wrmsrl (HV_X64_MSR_HYPERCALL , hypercall_msr.as_uint64); +/* +* Ignore any errors in setting up stimer clockevents +* as we can run with the LAPIC timer as a fallback. +*/ + (void )hv_stimer_alloc (); +hv_apic_init (); x86_init.pci.arch_init = hv_pci_init;
drivers/clocksource/hyperv_timer.c +124 -30
217 lines not on this page · raw message
drivers/hv/hv.c +2 -2
@@ -202,7 +202,7 @@ int hv_synic_init(unsigned int cpu) {hv_synic_enable_regs (cpu); -hv_stimer_init (cpu); +hv_stimer_legacy_init (cpu,VMBUS_MESSAGE_SINT );return 0 ; }
@@ -277,7 +277,7 @@ int hv_synic_cleanup(unsigned int cpu)if (channel_found && vmbus_connection.conn_state ==CONNECTED )return -EBUSY ; -hv_stimer_cleanup (cpu); +hv_stimer_legacy_cleanup (cpu);hv_synic_disable_regs (cpu);
drivers/hv/vmbus_drv.c +14 -16
@@ -1340,10 +1340,6 @@ static int vmbus_bus_init(void)if (ret) goto err_alloc; - ret =hv_stimer_alloc (VMBUS_MESSAGE_SINT ); -if (ret <0 ) - goto err_alloc; -/* * Initialize the per-cpu interrupt state and stimer state. * Then connect to the host.
@@ -1400,9 +1396,8 @@ static int vmbus_bus_init(void) err_connect:cpuhp_remove_state (hyperv_cpuhp_online); err_cpuhp: -hv_stimer_free (); -err_alloc:hv_synic_free (); +err_alloc:hv_remove_vmbus_irq ();bus_unregister (&hv_bus);
@@ -2315,20 +2310,23 @@ static void hv_crash_handler(struct pt_regs *regs)static int hv_synic_suspend (void ) {/* -* When we reach here, all the non-boot CPUs have been offlined, and -* the stimers on them have been unbound in hv_synic_cleanup() -> +* When we reach here, all the non-boot CPUs have been offlined. +* If we're in a legacy configuration where stimer Direct Mode is +* not enabled, the stimers on the non-boot CPUs have been unbound +* in hv_synic_cleanup() -> hv_stimer_legacy_cleanup() -> * hv_stimer_cleanup() -> clockevents_unbind_device(). * -* hv_synic_suspend() only runs on CPU0 with interrupts disabled. Here -* we do not unbind the stimer on CPU0 because: 1) it's unnecessary -* because the interrupts remain disabled between syscore_suspend() -* and syscore_resume(): see create_image() and resume_target_kernel(); +* hv_synic_suspend() only runs on CPU0 with interrupts disabled. +* Here we do not call hv_stimer_legacy_cleanup() on CPU0 because: +* 1) it's unnecessary as interrupts remain disabled between +* syscore_suspend() and syscore_resume(): see create_image() and +* resume_target_kernel() * 2) the stimer on CPU0 is automatically disabled later by * syscore_suspend() -> timekeeping_suspend() -> tick_suspend() -> ... -* -> clockevents_shutdown() -> ... -> hv_ce_shutdown(); 3) a warning -* would be triggered if we call clockevents_unbind_device(), which -* may sleep, in an interrupts-disabled context. So, we intentionally -* don't call hv_stimer_cleanup(0) here. +* -> clockevents_shutdown() -> ... -> hv_ce_shutdown() +* 3) a warning would be triggered if we call +* clockevents_unbind_device(), which may sleep, in an +* interrupts-disabled context. */ hv_synic_disable_regs (0 );
include/clocksource/hyperv_timer.h +4 -3
@@ -21,10 +21,11 @@#define HV_MIN_DELTA_TICKS 1/* Routines called by the VMbus driver */ -extern int hv_stimer_alloc (int sint); +extern int hv_stimer_alloc (void );extern void hv_stimer_free (void ); -extern void hv_stimer_init (unsigned int cpu); -extern void hv_stimer_cleanup (unsigned int cpu); +extern int hv_stimer_cleanup (unsigned int cpu); +extern void hv_stimer_legacy_init (unsigned int cpu,int sint); +extern void hv_stimer_legacy_cleanup (unsigned int cpu);extern void hv_stimer_global_cleanup (void );extern void hv_stimer0_isr (void );
include/linux/cpuhotplug.h +1 -0
@@ -129,6 +129,7 @@ enum cpuhp_state {CPUHP_AP_ARC_TIMER_STARTING ,CPUHP_AP_RISCV_TIMER_STARTING ,CPUHP_AP_CSKY_TIMER_STARTING , +CPUHP_AP_HYPERV_TIMER_STARTING ,CPUHP_AP_KVM_STARTING ,CPUHP_AP_KVM_ARM_VGIC_INIT_STARTING , CPUHP_AP_KVM_ARM_VGIC_STARTING,