Drivers: hv: vmbus: use cpu_hotplug_enable/disable for CPU offlining prevention
Subsystems (3)
v2 cover Drivers: hv: vmbus: use cpu_hotplug_enable/disable for CPU offlining prevention
Changes since v1:
- Make cpu_hotplug_disabled a counter [Radim Krčmář]
Export cpu_hotplug_enable/cpu_hotplug_disable functions from cpu.c and use them instead of altering smp_ops.cpu_disable in Hyper-V vmbus module.
Vitaly Kuznetsov (2):
cpu-hotplug: export cpu_hotplug_enable/cpu_hotplug_disable
Drivers: hv: vmbus: use cpu_hotplug_enable/disable
Documentation/power/suspend-and-cpuhotplug.txt | 6 ++-- drivers/hv/vmbus_drv.c | 38 +++----------------------- kernel/cpu.c | 13 +++++---- 3 files changed, 15 insertions(+), 42 deletions(-)
Signature · 1 line
1.9.3
PATCH v2 1/2 cpu-hotplug: export cpu_hotplug_enable/cpu_hotplug_disable
Loaded Hyper-V module will use these functions to disable CPU hotplug under certain circumstances. Convert cpu_hotplug_disabled to a counter (protected by cpu_add_remove_lock) to support e.g. disable -> disable -> enable call sequences.
--- Documentation/power/suspend-and-cpuhotplug.txt | 6 +++--- kernel/cpu.c | 13 ++++++++----- 2 files changed, 11 insertions(+), 8 deletions(-)
Documentation/power/suspend-and-cpuhotplug.txt +3 -3
@@ -72,7 +72,7 @@ More details follow: | v Disable regular cpu hotplug - by setting cpu_hotplug_disabled=1 + by increasing cpu_hotplug_disabled | v Release cpu_add_remove_lock
@@ -89,7 +89,7 @@ Resuming back is likewise, with the counterparts being (in the order of execution during resume): * enable_nonboot_cpus() which involves: | Acquire cpu_add_remove_lock - | Reset cpu_hotplug_disabled to 0, thereby enabling regular cpu hotplug + | Decrease cpu_hotplug_disabled, thereby enabling regular cpu hotplug | Call _cpu_up() [for all those cpus in the frozen_cpus mask, in a loop] | Release cpu_add_remove_lock v
@@ -120,7 +120,7 @@ after the entire cycle is complete (i.e., suspend + resume). Acquire cpu_add_remove_lock | v - If cpu_hotplug_disabled is 1 + If cpu_hotplug_disabled > 0 return gracefully | |
kernel/cpu.c +8 -5
@@ -190,17 +190,19 @@ void cpu_hotplug_done(void)void cpu_hotplug_disable (void ) {cpu_maps_update_begin (); - cpu_hotplug_disabled =1 ; + cpu_hotplug_disabled++;cpu_maps_update_done (); } +EXPORT_SYMBOL_GPL (cpu_hotplug_disable);void cpu_hotplug_enable (void ) {cpu_maps_update_begin (); - cpu_hotplug_disabled =0 ; +if (cpu_hotplug_disabled) + cpu_hotplug_disabled--;cpu_maps_update_done (); } - +EXPORT_SYMBOL_GPL (cpu_hotplug_enable);#endif /* CONFIG_HOTPLUG_CPU */ /* Need to know about CPUs going up/down? */
@@ -600,7 +602,7 @@ int disable_nonboot_cpus(void)if (!error) {BUG_ON (num_online_cpus () >1 );/* Make sure the CPUs won't be enabled by someone else */ - cpu_hotplug_disabled =1 ; + cpu_hotplug_disabled++; }else {pr_err ("Non-boot CPUs are not disabled\n" ); }
@@ -622,7 +624,8 @@ void __ref enable_nonboot_cpus(void)/* Allow everyone to use the CPU hotplug again */ cpu_maps_update_begin (); - cpu_hotplug_disabled =0 ; +if (cpu_hotplug_disabled) + cpu_hotplug_disabled--;if (cpumask_empty (frozen_cpus)) goto out;
Signature · 1 line
1.9.3
PATCH v2 2/2 Drivers: hv: vmbus: use cpu_hotplug_enable/disable
In reply to Vitaly KuznetsovCommit e513229b4c38 ("Drivers: hv: vmbus: prevent cpu offlining on newer hypervisors") was altering smp_ops.cpu_disable to prevent CPU offlining. We can bo better by using cpu_hotplug_enable/disable functions instead of such hard-coding.
--- drivers/hv/vmbus_drv.c | 38 ++++---------------------------------- 1 file changed, 4 insertions(+), 34 deletions(-)
drivers/hv/vmbus_drv.c +4 -34
@@ -763,38 +763,6 @@ static void vmbus_isr(void) } } -#ifdef CONFIG_HOTPLUG_CPU -static int hyperv_cpu_disable (void ) -{ -return -ENOSYS ; -} - -static void hv_cpu_hotplug_quirk (bool vmbus_loaded) -{ -static void *previous_cpu_disable; - -/* -* Offlining a CPU when running on newer hypervisors (WS2012R2, Win8, -* ...) is not supported at this moment as channel interrupts are -* distributed across all of them. -*/ - -if ((vmbus_proto_version ==VERSION_WS2008 ) || - (vmbus_proto_version ==VERSION_WIN7 )) -return ; - -if (vmbus_loaded) { - previous_cpu_disable = smp_ops.cpu_disable; - smp_ops.cpu_disable = hyperv_cpu_disable; -pr_notice ("CPU offlining is not supported by hypervisor\n" ); - }else if (previous_cpu_disable) - smp_ops.cpu_disable = previous_cpu_disable; -} -#else -static void hv_cpu_hotplug_quirk (bool vmbus_loaded) -{ -} -#endif /* * vmbus_bus_init -Main vmbus driver initialization routine.
@@ -836,7 +804,8 @@ static int vmbus_bus_init(int irq)if (ret) goto err_alloc; -hv_cpu_hotplug_quirk (true); +if (vmbus_proto_version >VERSION_WIN7 ) +cpu_hotplug_disable ();/* * Only register if the crash MSRs are available
@@ -1103,7 +1072,8 @@ static void __exit vmbus_exit(void)for_each_online_cpu (cpu)smp_call_function_single (cpu, hv_synic_cleanup,NULL ,1 );acpi_bus_unregister_driver (&vmbus_acpi_driver); -hv_cpu_hotplug_quirk (false); +if (vmbus_proto_version >VERSION_WIN7 ) +cpu_hotplug_enable ();vmbus_disconnect (); }
Signature · 1 line
1.9.3
RE: [PATCH v2 2/2] Drivers: hv: vmbus: use cpu_hotplug_enable/disable
Quoted 11 lines
-----Original Message-----
From: Vitaly Kuznetsov [mailto:[email protected]]
Sent: Thursday, May 21, 2015 9:22 AM
To: [email protected]
Cc: KY Srinivasan; Haiyang Zhang; [email protected]; Dexuan Cui; Ingo Molnar; Paul E. McKenney; Rafael J. Wysocki; Peter Zijlstra; Thomas Gleixner; Radim Krčmář
Subject: [PATCH v2 2/2] Drivers: hv: vmbus: use cpu_hotplug_enable/disable
Quoted 7 lines
Commit
e513229b4c38("Drivers: hv: vmbus: prevent cpu offlining on newer hypervisors") was altering smp_ops.cpu_disable to prevent CPU offlining. We can bo better by using cpu_hotplug_enable/disable functions instead of such hard-coding.Reported-by Radim Krčmář <[email protected]>Signed-off-by Vitaly Kuznetsov <[email protected]>
This patch does not apply. Vitaly, could you rebase this on Greg's tree and re-submit.
Regards,
K. Y
Quoted 69 lines
--- drivers/hv/vmbus_drv.c | 38 ++++---------------------------------- 1 file changed, 4 insertions(+), 34 deletions(-)drivers/hv/vmbus_drv.c +0
-32@@ -763,38 +763,6 @@ static void vmbus_isr(void) } } -#ifdef CONFIG_HOTPLUG_CPU -static int hyperv_cpu_disable (void ) -{ -return -ENOSYS ; -} - -static void hv_cpu_hotplug_quirk (bool vmbus_loaded) -{ -static void *previous_cpu_disable; - -/* -* Offlining a CPU when running on newer hypervisors (WS2012R2, Win8, -* ...) is not supported at this moment as channel interrupts are -* distributed across all of them. -*/ - -if ((vmbus_proto_version ==VERSION_WS2008 ) || - (vmbus_proto_version ==VERSION_WIN7 )) -return ; - -if (vmbus_loaded) { - previous_cpu_disable = smp_ops.cpu_disable; - smp_ops.cpu_disable = hyperv_cpu_disable; -pr_notice ("CPU offlining is not supported by hypervisor\n" ); - }else if (previous_cpu_disable) - smp_ops.cpu_disable = previous_cpu_disable; -} -#else -static void hv_cpu_hotplug_quirk (bool vmbus_loaded) -{ -} -#endif /* *
vmbus_bus_init-Main vmbus driver initialization routine.drivers/hv/vmbus_drv.c +4
-3@@ -836,7 +804,8 @@ static int vmbus_bus_init(int irq)if (ret) goto err_alloc; -hv_cpu_hotplug_quirk (true); +if (vmbus_proto_version >VERSION_WIN7 ) +cpu_hotplug_disable ();/* * Only register if the crash MSRs are available @@ -1103,7 +1072,8 @@ static void __exit vmbus_exit(void)for_each_online_cpu (cpu)smp_call_function_single (cpu, hv_synic_cleanup,NULL ,1 );acpi_bus_unregister_driver (&vmbus_acpi_driver); -hv_cpu_hotplug_quirk (false); +if (vmbus_proto_version >VERSION_WIN7 ) +cpu_hotplug_enable ();vmbus_disconnect (); } --
Quoted 1 line
1.9.3
���{.n�+�������+%�����ݶ��w��{.n�+����{��G�����{ay�ʇڙ�,j��f���h��������z_�(�階�ݢj"���m������G������?����&���~��iO��z��v�^�m����������?�I�
Quoted 15 lines
-----Original Message-----
From: Vitaly Kuznetsov [mailto:[email protected]]
Sent: Thursday, May 21, 2015 9:22 AM
To: [email protected]
Cc: KY Srinivasan; Haiyang Zhang; [email protected]; Dexuan Cui; Ingo Molnar; Paul E. McKenney; Rafael J. Wysocki; Peter Zijlstra; Thomas Gleixner; Radim Krčmář
Subject: [PATCH v2 0/2] Drivers: hv: vmbus: use cpu_hotplug_enable/disable for CPU offlining preventionChanges since v1:
- Makecpu_hotplug_disableda counter [Radim Krčmář]
Quoted 7 lines
Export cpu_hotplug_enable/cpu_hotplug_disable functions from
cpu.cand use
them instead of altering smp_ops.cpu_disable in Hyper-V vmbus module.Vitaly Kuznetsov (2):
cpu-hotplug: export cpu_hotplug_enable/cpu_hotplug_disable
Drivers: hv: vmbus: use cpu_hotplug_enable/disable
Vitaly,
Since there is a cross tree dependency here, could you resubmit the vmbus patch after the CPU hotplug patch has been committed.
Regards,
K. Y
Quoted 8 lines
Documentation/power/suspend-and-cpuhotplug.txt | 6 ++-- drivers/hv/vmbus_drv.c | 38 +++----------------------- kernel/cpu.c | 13 +++++---- 3 files changed, 15 insertions(+), 42 deletions(-)--
1.9.3
���{.n�+�������+%�����ݶ��w��{.n�+����{��G�����{ay�ʇڙ�,j��f���h��������z_�(�階�ݢj"���m������G������?����&���~��iO��z��v�^�m����������?�I�
Quoted 17 lines · KY Srinivasan
Quoted 17 lines
-----Original Message-----
From: Vitaly Kuznetsov [mailto:[email protected]]
Sent: Thursday, May 21, 2015 9:22 AM
To: [email protected]
Cc: KY Srinivasan; Haiyang Zhang; [email protected]; Dexuan Cui; Ingo Molnar; Paul E. McKenney; Rafael J. Wysocki; Peter Zijlstra; Thomas Gleixner; Radim Krčmář
Subject: [PATCH v2 0/2] Drivers: hv: vmbus: use cpu_hotplug_enable/disable for CPU offlining preventionChanges since v1:
- Makecpu_hotplug_disableda counter [Radim Krčmář]Export cpu_hotplug_enable/cpu_hotplug_disable functions from
cpu.cand use
them instead of altering smp_ops.cpu_disable in Hyper-V vmbus module.
Quoted 8 lines
Quoted 3 lines
Vitaly Kuznetsov (2):
cpu-hotplug: export cpu_hotplug_enable/cpu_hotplug_disable
Drivers: hv: vmbus: use cpu_hotplug_enable/disableVitaly,
Since there is a cross tree dependency here, could you resubmit the vmbus patch after the CPU hotplug patch has been committed.
I'm afraid it won't be committed without the second patch as without a single user of cpu_hotplug_enable/cpu_hotplug_disable (among modules) this patch doesn't make much sense. I would rather expect both of them to go through Greg's tree but with proper sign-offs/reviewed-bys. And of course this decision is up to tree maintainers, I'll resubmit the vmbus patch in case CPU hotplug patch gets committed separately.
Thanks,
Quoted 12 lines
Regards,
K. Y
Quoted 8 lines
Documentation/power/suspend-and-cpuhotplug.txt | 6 ++-- drivers/hv/vmbus_drv.c | 38 +++----------------------- kernel/cpu.c | 13 +++++---- 3 files changed, 15 insertions(+), 42 deletions(-)--
1.9.3
Signature · 1 line
Vitaly
Quoted 42 lines
-----Original Message-----
From: Vitaly Kuznetsov [mailto:[email protected]]
Sent: Wednesday, May 27, 2015 2:15 AM
To: KY Srinivasan
Cc: [email protected]; Haiyang Zhang; linux- [email protected]; Dexuan Cui; Ingo Molnar; Paul E. McKenney; Rafael J. Wysocki; Peter Zijlstra; Thomas Gleixner; Radim Krčmář
Subject: Re: [PATCH v2 0/2] Drivers: hv: vmbus: use cpu_hotplug_enable/disable for CPU offlining preventionQuoted 5 lines · KY Srinivasan
Quoted 5 lines
-----Original Message-----
From: Vitaly Kuznetsov [mailto:[email protected]]
Sent: Thursday, May 21, 2015 9:22 AM
To: [email protected]
Cc: KY Srinivasan; Haiyang Zhang; [email protected]; DexuanCui;
Quoted 3 lines
Quoted 3 lines
Ingo Molnar; Paul E. McKenney; Rafael J. Wysocki; Peter Zijlstra; Thomas Gleixner; Radim Krčmář
Subject: [PATCH v2 0/2] Drivers: hv: vmbus: usecpu_hotplug_enable/disable
Quoted 6 lines
Quoted 6 lines
for CPU offlining prevention
Changes since v1:
- Makecpu_hotplug_disableda counter [Radim Krčmář]Export cpu_hotplug_enable/cpu_hotplug_disable functions from
cpu.cand
Quoted 10 lines
Quoted 6 lines
use
them instead of altering smp_ops.cpu_disable in Hyper-V vmbus module.Vitaly Kuznetsov (2):
cpu-hotplug: export cpu_hotplug_enable/cpu_hotplug_disable
Drivers: hv: vmbus: use cpu_hotplug_enable/disableVitaly,
Since there is a cross tree dependency here, could you resubmit the vmbus
patch
Quoted 8 lines
Quoted 1 line
after the CPU hotplug patch has been committed.
I'm afraid it won't be committed without the second patch as without a single user of cpu_hotplug_enable/cpu_hotplug_disable (among modules) this patch doesn't make much sense. I would rather expect both of them to go through Greg's tree but with proper sign-offs/reviewed-bys. And of course this decision is up to tree maintainers, I'll resubmit the vmbus patch in case CPU hotplug patch gets committed separately.
I had asked if Greg could take both the patches and I have not heard from him yet and I assumed that he would not. This has happened to me before and I have done just that - show the need by sending patches that would use the functionality and get them into the tree in the correct order. Greg, please let us know how you want to handle this; I have a bunch of patches that I am sitting not knowing how Vitaly's patches will be merged.
Regards,
K. Y
Quoted 13 lines
Thanks,
Quoted 10 lines
Regards,
K. Y
Quoted 6 lines
Documentation/power/suspend-and-cpuhotplug.txt | 6 ++-- drivers/hv/vmbus_drv.c | 38 +++----------------------- kernel/cpu.c | 13 +++++---- 3 files changed, 15 insertions(+), 42 deletions(-)
Quoted 5 lines
Quoted 2 lines
Quoted 2 lines
--
1.9.3--
Vitaly
���{.n�+�������+%�����ݶ��w��{.n�+����{��G�����{ay�ʇڙ�,j��f���h��������z_�(�階�ݢj"���m������G������?����&���~��iO��z��v�^�m����������?�I�
Quoted 51 lines · KY Srinivasan
Quoted 42 lines
-----Original Message-----
From: Vitaly Kuznetsov [mailto:[email protected]]
Sent: Wednesday, May 27, 2015 2:15 AM
To: KY Srinivasan
Cc: [email protected]; Haiyang Zhang; linux- [email protected]; Dexuan Cui; Ingo Molnar; Paul E. McKenney; Rafael J. Wysocki; Peter Zijlstra; Thomas Gleixner; Radim Krčmář
Subject: Re: [PATCH v2 0/2] Drivers: hv: vmbus: use cpu_hotplug_enable/disable for CPU offlining preventionQuoted 5 lines · KY Srinivasan
Quoted 5 lines
-----Original Message-----
From: Vitaly Kuznetsov [mailto:[email protected]]
Sent: Thursday, May 21, 2015 9:22 AM
To: [email protected]
Cc: KY Srinivasan; Haiyang Zhang; [email protected]; DexuanCui;
Quoted 3 lines
Quoted 3 lines
Ingo Molnar; Paul E. McKenney; Rafael J. Wysocki; Peter Zijlstra; Thomas Gleixner; Radim Krčmář
Subject: [PATCH v2 0/2] Drivers: hv: vmbus: usecpu_hotplug_enable/disable
Quoted 6 lines
Quoted 6 lines
for CPU offlining prevention
Changes since v1:
- Makecpu_hotplug_disableda counter [Radim Krčmář]Export cpu_hotplug_enable/cpu_hotplug_disable functions from
cpu.cand
Quoted 10 lines
Quoted 6 lines
use
them instead of altering smp_ops.cpu_disable in Hyper-V vmbus module.Vitaly Kuznetsov (2):
cpu-hotplug: export cpu_hotplug_enable/cpu_hotplug_disable
Drivers: hv: vmbus: use cpu_hotplug_enable/disableVitaly,
Since there is a cross tree dependency here, could you resubmit the vmbus
patch
Quoted 8 lines
Quoted 1 line
after the CPU hotplug patch has been committed.
I'm afraid it won't be committed without the second patch as without a single user of cpu_hotplug_enable/cpu_hotplug_disable (among modules) this patch doesn't make much sense. I would rather expect both of them to go through Greg's tree but with proper sign-offs/reviewed-bys. And of course this decision is up to tree maintainers, I'll resubmit the vmbus patch in case CPU hotplug patch gets committed separately.
Quoted 2 lines
I had asked if Greg could take both the patches and I have not heard from him yet and I assumed that he would not.
Don't you love it when people ask you to do things, yet never even cc:
you on an email and still expect you to magically notice and respond to them?
{sigh}
You got lucky here, I happened to notice this email, but I have no email from you to me where you asked me to do anything.
As I told you before, you need to repackage up patches you want to have applied and send them to me, that's the job of a subsystem maintainer, you can't just "expect" someone else to magically notice them and apply them for you.
greg k-h
Quoted 71 lines
-----Original Message-----
From: Greg KH [mailto:[email protected]]
Sent: Wednesday, May 27, 2015 8:06 AM
To: KY Srinivasan
Cc: Vitaly Kuznetsov; Radim Krčmář; Peter Zijlstra; Haiyang Zhang; Rafael J. Wysocki; [email protected]; [email protected]; Thomas Gleixner; Paul E. McKenney; Ingo Molnar
Subject: Re: [PATCH v2 0/2] Drivers: hv: vmbus: use cpu_hotplug_enable/disable for CPU offlining preventionQuoted 8 lines · KY Srinivasan
Quoted 8 lines
-----Original Message-----
From: Vitaly Kuznetsov [mailto:[email protected]]
Sent: Wednesday, May 27, 2015 2:15 AM
To: KY Srinivasan
Cc: [email protected]; Haiyang Zhang; linux- [email protected]; Dexuan Cui; Ingo Molnar; Paul E. McKenney;Rafael
Quoted 11 lines
Quoted 11 lines
J. Wysocki; Peter Zijlstra; Thomas Gleixner; Radim Krčmář
Subject: Re: [PATCH v2 0/2] Drivers: hv: vmbus: use cpu_hotplug_enable/disable for CPU offlining preventionQuoted 5 lines · KY Srinivasan
Quoted 5 lines
-----Original Message-----
From: Vitaly Kuznetsov [mailto:[email protected]]
Sent: Thursday, May 21, 2015 9:22 AM
To: [email protected]
Cc: KY Srinivasan; Haiyang Zhang; [email protected];Dexuan
Quoted 2 lines
Quoted 2 lines
Cui;
Quoted 1 line
Quoted 1 line
Ingo Molnar; Paul E. McKenney; Rafael J. Wysocki; Peter Zijlstra;
Thomas
Quoted 9 lines
Quoted 9 lines
Quoted 2 lines
Quoted 2 lines
Gleixner; Radim Krčmář
Subject: [PATCH v2 0/2] Drivers: hv: vmbus: usecpu_hotplug_enable/disable
Quoted 6 lines
Quoted 6 lines
for CPU offlining prevention
Changes since v1:
- Makecpu_hotplug_disableda counter [Radim Krčmář]Export cpu_hotplug_enable/cpu_hotplug_disable functions from
cpu.cQuoted 3 lines
Quoted 3 lines
and
Quoted 2 lines
Quoted 2 lines
use
them instead of altering smp_ops.cpu_disable in Hyper-V vmbusmodule.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 4 lines
Vitaly Kuznetsov (2):
cpu-hotplug: export cpu_hotplug_enable/cpu_hotplug_disable
Drivers: hv: vmbus: use cpu_hotplug_enable/disableVitaly,
Since there is a cross tree dependency here, could you resubmit the
vmbus
Quoted 5 lines
Quoted 5 lines
patch
Quoted 1 line
after the CPU hotplug patch has been committed.
I'm afraid it won't be committed without the second patch as without a single user of cpu_hotplug_enable/cpu_hotplug_disable (among
modules)
Quoted 5 lines
Quoted 4 lines
this patch doesn't make much sense. I would rather expect both of them to go through Greg's tree but with proper sign-offs/reviewed-bys. And of course this decision is up to tree maintainers, I'll resubmit the vmbus patch in case CPU hotplug patch gets committed separately.
Quoted 7 lines
Quoted 1 line
I had asked if Greg could take both the patches and I have not heard from
him yet and I
Quoted 1 line
assumed that he would not.
Don't you love it when people ask you to do things, yet never even cc:
you on an email and still expect you to magically notice and respond to them?
Greg,
When I send patches, you are always on the "to" list and I was just responding to Vitaly's email (assuming that you were on the "to" list - my bad). Sorry about that.
Quoted 10 lines
{sigh}
You got lucky here, I happened to notice this email, but I have no email from you to me where you asked me to do anything.
As I told you before, you need to repackage up patches you want to have applied and send them to me, that's the job of a subsystem maintainer, you can't just "expect" someone else to magically notice them and apply them for you.
The issue is not one of packaging. Vitaly has two patches in this series each of which needs to go through a different tree. I wanted to know if you can take both patches through your tree. In any case, I will repackage and send them to you.
Thank you,
K. Y
Quoted 2 lines
greg k-h
���{.n�+�������+%�����ݶ��w��{.n�+����{��G�����{ay�ʇڙ�,j��f���h��������z_�(�階�ݢj"���m������G������?����&���~��iO��z��v�^�m����������?�I�