Drivers: hv: vmbus: use cpu_hotplug_enable/disable for CPU offlining prevention
Subsystems (2)
cover Drivers: hv: vmbus: use cpu_hotplug_enable/disable for CPU offlining prevention
Export cpu_hotplug_enable/cpu_hotplug_disable functions from cpu.c and use them instead of altering the smp_ops.cpu_disable in Hyper-V vmbus module.
Vitaly Kuznetsov (2):
cpu-hotplug: export cpu_hotplug_enable/cpu_hotplug_disable
Drivers: hv: vmbus: use cpu_hotplug_enable/disable
drivers/hv/vmbus_drv.c | 38 ++++---------------------------------- kernel/cpu.c | 3 ++- 2 files changed, 6 insertions(+), 35 deletions(-)
Signature · 1 line
1.9.3
PATCH 1/2 cpu-hotplug: export cpu_hotplug_enable/cpu_hotplug_disable
Loaded Hyper-V module will use these functions to disable CPU hotplug under certain circumstances.
--- kernel/cpu.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
kernel/cpu.c +2 -1
@@ -193,6 +193,7 @@ void cpu_hotplug_disable(void) cpu_hotplug_disabled =1 ;cpu_maps_update_done (); } +EXPORT_SYMBOL_GPL (cpu_hotplug_disable);void cpu_hotplug_enable (void ) {
@@ -200,7 +201,7 @@ void cpu_hotplug_enable(void) cpu_hotplug_disabled =0 ; cpu_maps_update_done(); } - +EXPORT_SYMBOL_GPL (cpu_hotplug_enable);#endif /* CONFIG_HOTPLUG_CPU */ /* Need to know about CPUs going up/down? */
Signature · 1 line
1.9.3
Re: [PATCH 1/2] cpu-hotplug: export cpu_hotplug_enable/cpu_hotplug_disable
2015-05-19 14:15+0200, Vitaly Kuznetsov:
Quoted 13 lines
Loaded Hyper-V module will use these functions to disable CPU hotplug under certain circumstances.
Signed-off-by Vitaly Kuznetsov <[email protected]>--- kernel/cpu.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)kernel/cpu.c +0
-0
kernel/cpu.c void cpu_hotplug_disable(void) · line 193
Quoted 1 line
kernel/cpu.c +0
-0@@ -193,6 +193,7 @@ void cpu_hotplug_disable(void) cpu_hotplug_disabled =1 ;
(I think it would be safer to make this into a counter, at least, so
cpu_hotplug_disable() |
| cpu_hotplug_disable()
cpu_hotplug_enable() |wouldn't end up with enabled hotplug, when this switch is intended for hotplug-and-die situations.)
Quoted 18 lines
kernel/cpu.c +2
-1cpu_maps_update_done (); } +EXPORT_SYMBOL_GPL (cpu_hotplug_disable);void cpu_hotplug_enable (void ) {@@ -200,7 +201,7 @@ void cpu_hotplug_enable(void) cpu_hotplug_disabled =0 ; cpu_maps_update_done(); } - +EXPORT_SYMBOL_GPL (cpu_hotplug_enable);#endif /* CONFIG_HOTPLUG_CPU */ /* Need to know about CPUs going up/down? */ Signature · 1 line
1.9.3
Peter Zijlstra
maintainerRe: [PATCH 1/2] cpu-hotplug: export cpu_hotplug_enable/cpu_hotplug_disable
In reply to Vitaly KuznetsovQuoted 2 lines · Vitaly Kuznetsov
Loaded Hyper-V module will use these functions to disable CPU hotplug under certain circumstances.
What's wrong with get_online_cpus() ?
Re: [PATCH 1/2] cpu-hotplug: export cpu_hotplug_enable/cpu_hotplug_disable
Quoted 5 lines · Peter Zijlstra
Quoted 2 lines · Vitaly Kuznetsov
Loaded Hyper-V module will use these functions to disable CPU hotplug under certain circumstances.
What's wrong with
get_online_cpus()?
We need to disable cpu offlining permanently (till the module is unloaded but this is highly unlikely). If we do get_online_cpus() instead we get
# echo 0 > /sys/devices/system/cpu/cpu1/online <nothing happens, pemanent hang followed by the following:>
[ 240.247109] INFO: task bash:972 blocked for more than 120 seconds. [ 240.248937] Not tainted 4.1.0-rc3_bug1167380_test_rkr+ #602 [ 240.250515] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 240.252379] bash D ffff880021b83c18 0 972 939 0x00000004 [ 240.255068] ffff880021b83c18 0000000000000001 0000000000000046 ffff880021b83be8 [ 240.258140] ffffffff81fdcdb0 ffff880037dbcfc0 ffff880037203bd0 ffffffff81fdcdb8 [ 240.261275] ffff880021b84000 0000000000000001 0000000000000000 0000000000000001 [ 240.264402] Call Trace: [ 240.265423] [<ffffffff81aecf77>] schedule+0x37/0x90 [ 240.266748] [<ffffffff81081872>] cpu_hotplug_begin+0xd2/0xe0 [ 240.268210] [<ffffffff810817a5>] ? cpu_hotplug_begin+0x5/0xe0 [ 240.269675] [<ffffffff810cb860>] ? prepare_to_wait_event+0xf0/0xf0 [ 240.271285] [<ffffffff81ad561a>] _cpu_down+0x8a/0x300 [ 240.275294] [<ffffffff81ad58c6>] cpu_down+0x36/0x50 [ 240.276680] [<ffffffff8160f954>] cpu_subsys_offline+0x14/0x20 [ 240.278145] [<ffffffff8160a505>] device_offline+0xa5/0xd0 [ 240.279618] [<ffffffff8160a620>] online_store+0x50/0xa0 [ 240.280991] [<ffffffff81607678>] dev_attr_store+0x18/0x30
...
And if we do the cpu_hotplug_disable() call we get:
# echo 0 > /sys/devices/system/cpu/cpu1/online -bash: echo: write error: Device or resource busy
This behavior is preferable.
Signature · 1 line
Vitaly
PATCH 2/2 Drivers: hv: vmbus: use cpu_hotplug_enable/disable
In reply to Vitaly KuznetsovCommit e513229b4c38 ("Drivers: hv: vmbus: prevent cpu offlining on newer hypervisors") was alteringd smp_ops.cpu_disable to prevent CPU offlining. We can bo better by using cpu_hotplug_enable/disable functions instead of such hard-coding.
--- drivers/hv/vmbus_drv.c | 38 ++++---------------------------------- 1 file changed, 4 insertions(+), 34 deletions(-)
drivers/hv/vmbus_drv.c +4 -34
@@ -763,38 +763,6 @@ static void vmbus_isr(void) } } -#ifdef CONFIG_HOTPLUG_CPU -static int hyperv_cpu_disable (void ) -{ -return -ENOSYS ; -} - -static void hv_cpu_hotplug_quirk (bool vmbus_loaded) -{ -static void *previous_cpu_disable; - -/* -* Offlining a CPU when running on newer hypervisors (WS2012R2, Win8, -* ...) is not supported at this moment as channel interrupts are -* distributed across all of them. -*/ - -if ((vmbus_proto_version ==VERSION_WS2008 ) || - (vmbus_proto_version ==VERSION_WIN7 )) -return ; - -if (vmbus_loaded) { - previous_cpu_disable = smp_ops.cpu_disable; - smp_ops.cpu_disable = hyperv_cpu_disable; -pr_notice ("CPU offlining is not supported by hypervisor\n" ); - }else if (previous_cpu_disable) - smp_ops.cpu_disable = previous_cpu_disable; -} -#else -static void hv_cpu_hotplug_quirk (bool vmbus_loaded) -{ -} -#endif /* * vmbus_bus_init -Main vmbus driver initialization routine.
@@ -836,7 +804,8 @@ static int vmbus_bus_init(int irq)if (ret) goto err_alloc; -hv_cpu_hotplug_quirk (true); +if (vmbus_proto_version >VERSION_WIN7 ) +cpu_hotplug_disable ();/* * Only register if the crash MSRs are available
@@ -1103,7 +1072,8 @@ static void __exit vmbus_exit(void)for_each_online_cpu (cpu)smp_call_function_single (cpu, hv_synic_cleanup,NULL ,1 );acpi_bus_unregister_driver (&vmbus_acpi_driver); -hv_cpu_hotplug_quirk (false); +if (vmbus_proto_version >VERSION_WIN7 ) +cpu_hotplug_enable ();vmbus_disconnect (); }
Signature · 1 line
1.9.3
Quoted 17 lines
-----Original Message-----
From: Vitaly Kuznetsov [mailto:[email protected]]
Sent: Tuesday, May 19, 2015 5:16 AM
To: [email protected]
Cc: KY Srinivasan; Haiyang Zhang; [email protected]; Dexuan Cui; Ingo Molnar; Paul E. McKenney; Rafael J. Wysocki; Peter Zijlstra; Thomas Gleixner; Radim Krčmář
Subject: [PATCH 0/2] Drivers: hv: vmbus: use cpu_hotplug_enable/disable for CPU offlining preventionExport cpu_hotplug_enable/cpu_hotplug_disable functions from
cpu.cand use
them instead of altering the smp_ops.cpu_disable in Hyper-V vmbus module.
Quoted 7 lines
Vitaly Kuznetsov (2):
cpu-hotplug: export cpu_hotplug_enable/cpu_hotplug_disable
Drivers: hv: vmbus: use cpu_hotplug_enable/disabledrivers/hv/vmbus_drv.c | 38 ++++---------------------------------- kernel/cpu.c | 3 ++- 2 files changed, 6 insertions(+), 35 deletions(-)
Greg,
Can you take both these patches through your tree.
Regards,
K. Y
Quoted 3 lines
--
1.9.3
���{.n�+�������+%�����ݶ��w��{.n�+����{��G�����{ay�ʇڙ�,j��f���h��������z_�(�階�ݢj"���m������G������?����&���~��iO��z��v�^�m����������?�I�