cpu/hotplug: Fix rollback during error-out in takedown_cpu()
Subsystems (2)
PATCH 1/1 cpu/hotplug: Fix rollback during error-out in takedown_cpu()
If takedown_cpu() fails during _cpu_down(), st->state is reset, by calling cpuhp_reset_state(). This results in an additional increment of st->state, which results in CPUHP_AP_SMPBOOT_THREADS state being skipped during rollback. Fix this by not calling cpuhp_reset_state() and doing the state reset directly in _cpu_down().
--- kernel/cpu.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
kernel/cpu.c +8 -1
@@ -970,7 +970,14 @@ static int __ref _cpu_down(unsigned int cpu, int tasks_frozen,*/ ret =cpuhp_down_callbacks (cpu, st, target);if (ret && st->state >CPUHP_TEARDOWN_CPU && st->state < prev_state) { -cpuhp_reset_state (st, prev_state); +/* +* As st->last is not set, cpuhp_reset_state() increments +* st->state, which results in CPUHP_AP_SMPBOOT_THREADS being +* skipped during rollback. So, don't use it here. +*/ + st->rollback = true; + st->target = prev_state; + st->bringup = !st->bringup;__cpuhp_kick_ap (st); }
Signature · 2 lines
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, hosted by The Linux Foundation
Quoted 20 lines · Neeraj Upadhyay
If
takedown_cpu()fails during_cpu_down(),st->stateis reset, by callingcpuhp_reset_state(). This results in an additional increment ofst->state, which results inCPUHP_AP_SMPBOOT_THREADSstate being skipped during rollback. Fix this by not callingcpuhp_reset_state()and doing the state reset directly in_cpu_down().Fixes 4dddfb5faa61 ("smp/hotplug: Rewrite AP state machine core")Signed-off-by Neeraj Upadhyay <[email protected]>--- kernel/cpu.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)kernel/cpu.c +0
-0@@ -970,7 +970,14 @@ static int __ref _cpu_down(unsigned int cpu, int tasks_frozen,*/ ret =cpuhp_down_callbacks (cpu, st, target);
kernel/cpu.c static int __ref _cpu_down(unsigned int cpu, int tasks_frozen, · line 980
Quoted 10 lines
kernel/cpu.c +8
-1if (ret && st->state >CPUHP_TEARDOWN_CPU && st->state < prev_state) { -cpuhp_reset_state (st, prev_state); +/* +* As st->last is not set, cpuhp_reset_state() increments +* st->state, which results in CPUHP_AP_SMPBOOT_THREADS being +* skipped during rollback. So, don't use it here. +*/ + st->rollback = true; + st->target = prev_state; + st->bringup = !st->bringup;
looks valid for this case.
Quoted 3 lines
kernel/cpu.c +0
-0__cpuhp_kick_ap (st); }
Quoted 20 lines · Neeraj Upadhyay
If
takedown_cpu()fails during_cpu_down(),st->stateis reset, by callingcpuhp_reset_state(). This results in an additional increment ofst->state, which results inCPUHP_AP_SMPBOOT_THREADSstate being skipped during rollback. Fix this by not callingcpuhp_reset_state()and doing the state reset directly in_cpu_down().Fixes 4dddfb5faa61 ("smp/hotplug: Rewrite AP state machine core")Signed-off-by Neeraj Upadhyay <[email protected]>--- kernel/cpu.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)kernel/cpu.c +0
-0@@ -970,7 +970,14 @@ static int __ref _cpu_down(unsigned int cpu, int tasks_frozen,*/ ret =cpuhp_down_callbacks (cpu, st, target);
kernel/cpu.c static int __ref _cpu_down(unsigned int cpu, int tasks_frozen, · line 980
Quoted 10 lines
kernel/cpu.c +8
-1if (ret && st->state >CPUHP_TEARDOWN_CPU && st->state < prev_state) { -cpuhp_reset_state (st, prev_state); +/* +* As st->last is not set, cpuhp_reset_state() increments +* st->state, which results in CPUHP_AP_SMPBOOT_THREADS being +* skipped during rollback. So, don't use it here. +*/ + st->rollback = true; + st->target = prev_state; + st->bringup = !st->bringup;
No, this is just papering over the actual problem.
The state inconsistency happens in take_cpu_down() when it returns with a failure from __cpu_disable() because that returns with state = TEARDOWN_CPU and st->state is then incremented in undo_cpu_down().
That's the real issue and we need to analyze the whole cpu_down rollback logic first.
Thanks,
tglx
Quoted 33 lines · Thomas Gleixner
Quoted 20 lines · Neeraj Upadhyay
If
takedown_cpu()fails during_cpu_down(),st->stateis reset, by callingcpuhp_reset_state(). This results in an additional increment ofst->state, which results inCPUHP_AP_SMPBOOT_THREADSstate being skipped during rollback. Fix this by not callingcpuhp_reset_state()and doing the state reset directly in_cpu_down().Fixes 4dddfb5faa61 ("smp/hotplug: Rewrite AP state machine core")Signed-off-by Neeraj Upadhyay <[email protected]>--- kernel/cpu.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)kernel/cpu.c +0
-0@@ -970,7 +970,14 @@ static int __ref _cpu_down(unsigned int cpu, int tasks_frozen,*/ ret =cpuhp_down_callbacks (cpu, st, target);kernel/cpu.c static int __ref _cpu_down(unsigned int cpu, int tasks_frozen, · line 980
Quoted 10 lines
kernel/cpu.c +8
-1if (ret && st->state >CPUHP_TEARDOWN_CPU && st->state < prev_state) { -cpuhp_reset_state (st, prev_state); +/* +* As st->last is not set, cpuhp_reset_state() increments +* st->state, which results in CPUHP_AP_SMPBOOT_THREADS being +* skipped during rollback. So, don't use it here. +*/ + st->rollback = true; + st->target = prev_state; + st->bringup = !st->bringup;No, this is just papering over the actual problem.
Quoted 6 lines
The state inconsistency happens in
take_cpu_down()when it returns with a failure from__cpu_disable()because that returns with state =TEARDOWN_CPUandst->stateis then incremented inundo_cpu_down().That's the real issue and we need to analyze the whole
cpu_downrollback logic first.
Could this be done like below ?
kernel/cpu.c +0 -0
@@ -802,17 +802,18 @@ static int take_cpu_down(void *_param)
int err, cpu = smp_processor_id();
int ret;
- /* Ensure this CPU doesn't handle any more interrupts. */ - err = __cpu_disable();
- if (err < 0)
- return err;
-
/*
* We get here while we are in CPUHP_TEARDOWN_CPU state and we must not
* do this step again.
*/
WARN_ON(st->state != CPUHP_TEARDOWN_CPU);
st->state--;
+
+ /* Ensure this CPU doesn't handle any more interrupts. */ + err = __cpu_disable();
+ if (err < 0)
+ return err;
+
/* Invoke the former CPU_DYING callbacks */
Thanks,
Mukesh
Quoted 6 lines
Quoted 6 lines · Mukesh Ojha
Quoted 5 lines · Thomas Gleixner
Review
Quoted 3 lines
(quoted diff) +3
-0+ st->rollback = true; + st->target = prev_state; + st->bringup = !st->bringup;No, this is just papering over the actual problem.
Quoted 8 lines
Quoted 6 lines
The state inconsistency happens in
take_cpu_down()when it returns with a failure from__cpu_disable()because that returns with state =TEARDOWN_CPUandst->stateis then incremented inundo_cpu_down().That's the real issue and we need to analyze the whole
cpu_downrollback logic first.Could this be done like below ?
IOW, more papering over the real root cause.
kernel/cpu.c static int take_cpu_down(void *_param) · line 803
Quoted 7 lines
kernel/cpu.c +0
-0@@ -802,17 +802,18 @@ static int take_cpu_down(void *_param) int err, cpu = smp_processor_id(); int ret;
^^^^^^^
Please fix your mailer or your editor. That patch is whitespace damaged.
Thanks,
tglx
Quoted 15 lines · Thomas Gleixner
Review
Quoted 11 lines · Neeraj Upadhyay
(quoted diff) +8
-1ret = cpuhp_down_callbacks(cpu, st, target); if (ret && st->state > CPUHP_TEARDOWN_CPU && st->state < prev_state) { - cpuhp_reset_state(st, prev_state); + /* + * As st->last is not set, cpuhp_reset_state() increments + * st->state, which results in CPUHP_AP_SMPBOOT_THREADS being + * skipped during rollback. So, don't use it here. + */ + st->rollback = true; + st->target = prev_state; + st->bringup = !st->bringup;No, this is just papering over the actual problem.
Quoted 6 lines
The state inconsistency happens in
take_cpu_down()when it returns with a failure from__cpu_disable()because that returns with state =TEARDOWN_CPUandst->stateis then incremented inundo_cpu_down().That's the real issue and we need to analyze the whole
cpu_downrollback logic first.
And looking closer this is a general issue. Just that the TEARDOWN state makes it simple to observe. It's universaly broken, when the first teardown callback fails because, st->state is only decremented _AFTER_ the callback returns success, but undo_cpu_down() increments unconditionally.
Patch below.
Thanks,
tglx
----
kernel/cpu.c +3 -2
@@ -916,7 +916,8 @@ static int cpuhp_down_callbacks(unsigned ret =cpuhp_invoke_callback (cpu, st->state, false,NULL ,NULL );if (ret) { st->target = prev_state; -undo_cpu_down (cpu, st); +if (st->state < prev_state) +undo_cpu_down (cpu, st);break ; } }
@@ -969,7 +970,7 @@ static int __ref _cpu_down(unsigned int* to do the further cleanups. */ ret =cpuhp_down_callbacks (cpu, st, target); -if (ret && st->state >CPUHP_TEARDOWN_CPU && st->state < prev_state) { +if (ret && st->state ==CPUHP_TEARDOWN_CPU && st->state < prev_state) {cpuhp_reset_state (st, prev_state);__cpuhp_kick_ap (st); }
Quoted 26 lines · Thomas Gleixner
Quoted 14 lines · Thomas Gleixner
Review
Quoted 11 lines · Neeraj Upadhyay
ret = cpuhp_down_callbacks(cpu, st, target); if (ret && st->state > CPUHP_TEARDOWN_CPU && st->state < prev_state) {(quoted diff) +8
-1- cpuhp_reset_state(st, prev_state); + /* + * As st->last is not set, cpuhp_reset_state() increments + * st->state, which results in CPUHP_AP_SMPBOOT_THREADS being + * skipped during rollback. So, don't use it here. + */ + st->rollback = true; + st->target = prev_state; + st->bringup = !st->bringup;No, this is just papering over the actual problem.
Quoted 6 lines
The state inconsistency happens in
take_cpu_down()when it returns with a failure from__cpu_disable()because that returns with state =TEARDOWN_CPUandst->stateis then incremented inundo_cpu_down().That's the real issue and we need to analyze the whole
cpu_downrollback logic first.And looking closer this is a general issue. Just that the TEARDOWN state makes it simple to observe. It's universaly broken, when the first teardown callback fails because,
st->stateis only decremented _AFTER_ the callback returns success, butundo_cpu_down()increments unconditionally.
Quoted 5 lines
Patch below.
Thanks,
tglx
As per my understanding, there are 2 problems here; one is fixed with your patch, and other is cpuhp_reset_state() is used during rollback from non-AP to AP state, which seem to result in 2 increments of st->state (one increment done by cpuhp_reset_state() and another by cpu_thread_fun()) .
Quoted 22 lines
----
kernel/cpu.c +3
-2@@ -916,7 +916,8 @@ static int cpuhp_down_callbacks(unsigned ret =cpuhp_invoke_callback (cpu, st->state, false,NULL ,NULL );if (ret) { st->target = prev_state; -undo_cpu_down (cpu, st); +if (st->state < prev_state) +undo_cpu_down (cpu, st);break ; } }@@ -969,7 +970,7 @@ static int __ref _cpu_down(unsigned int* to do the further cleanups. */ ret =cpuhp_down_callbacks (cpu, st, target); -if (ret && st->state >CPUHP_TEARDOWN_CPU && st->state < prev_state) { +if (ret && st->state ==CPUHP_TEARDOWN_CPU && st->state < prev_state) {cpuhp_reset_state (st, prev_state);__cpuhp_kick_ap (st); }
Signature · 2 lines
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, hosted by The Linux Foundation
Quoted 8 lines · Neeraj Upadhyay
Quoted 6 lines · Thomas Gleixner
And looking closer this is a general issue. Just that the TEARDOWN state makes it simple to observe. It's universaly broken, when the first teardown callback fails because,
st->stateis only decremented _AFTER_ the callback returns success, butundo_cpu_down()increments unconditionally.
Quoted 4 lines
As per my understanding, there are 2 problems here; one is fixed with your patch, and other is
cpuhp_reset_state()is used during rollback from non-AP to AP state, which seem to result in 2 increments ofst->state(one increment done bycpuhp_reset_state()and another bycpu_thread_fun()) .
And how did your hack fix that up magically? I'll have a look later today.
Thanks,
tglx
Quoted 11 lines · Thomas Gleixner
Quoted 10 lines · Neeraj Upadhyay
Quoted 5 lines · Thomas Gleixner
And looking closer this is a general issue. Just that the TEARDOWN state makes it simple to observe. It's universaly broken, when the first teardown callback fails because,
st->stateis only decremented _AFTER_ the callback returns success, butundo_cpu_down()increments unconditionally.As per my understanding, there are 2 problems here; one is fixed with your patch, and other is
cpuhp_reset_state()is used during rollback from non-AP to AP state, which seem to result in 2 increments ofst->state(one increment done bycpuhp_reset_state()and another bycpu_thread_fun()) .
Quoted 5 lines
And how did your hack fix that up magically? I'll have a look later today.
Thanks,
tglx
The hack fixes it by not calling cpuhp_reset_state() and doing rollback state reset inline in _cpu_down().
Signature · 2 lines
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, hosted by The Linux Foundation
Quoted 15 lines · Neeraj Upadhyay
Quoted 14 lines · Thomas Gleixner
Quoted 13 lines · Neeraj Upadhyay
Quoted 7 lines · Thomas Gleixner
And looking closer this is a general issue. Just that the TEARDOWN state makes it simple to observe. It's universaly broken, when the first teardown
callback fails because,st->stateis only decremented _AFTER_ the callback
returns success, butundo_cpu_down()increments unconditionally.As per my understanding, there are 2 problems here; one is fixed with your patch, and other is
cpuhp_reset_state()is used during rollback from non-AP to
AP state, which seem to result in 2 increments ofst->state(one increment done bycpuhp_reset_state()and another bycpu_thread_fun()) .
Quoted 8 lines
Quoted 5 lines
And how did your hack fix that up magically? I'll have a look later today.
Thanks,
tglx
The hack fixes it by not calling
cpuhp_reset_state()and doing rollback state reset inline in_cpu_down().
And how is that any different from the proper patch I sent? It ends up in the same state. So I have a hard time to understand your blurb above where you claim that my patch just solves one of two problems?
Thanks,
tglx
Quoted 25 lines · Thomas Gleixner
Quoted 15 lines · Neeraj Upadhyay
Quoted 14 lines · Thomas Gleixner
Quoted 13 lines · Neeraj Upadhyay
Quoted 7 lines · Thomas Gleixner
And looking closer this is a general issue. Just that the TEARDOWN state makes it simple to observe. It's universaly broken, when the first teardown
callback fails because,st->stateis only decremented _AFTER_ the callback
returns success, butundo_cpu_down()increments unconditionally.As per my understanding, there are 2 problems here; one is fixed with your patch, and other is
cpuhp_reset_state()is used during rollback from non-AP to
AP state, which seem to result in 2 increments ofst->state(one increment done bycpuhp_reset_state()and another bycpu_thread_fun()) .Quoted 8 lines
Quoted 5 lines
And how did your hack fix that up magically? I'll have a look later today.
Thanks,
tglx
The hack fixes it by not calling
cpuhp_reset_state()and doing rollback state reset inline in_cpu_down().
Quoted 8 lines
And how is that any different from the proper patch I sent? It ends up in the same state. So I have a hard time to understand your blurb above where you claim that my patch just solves one of two problems?
Thanks,
tglx
Yes, your patch solves the problem related to smpboot unparking being skipped during rollback and with the hack we end up in same state. The second thing, which I am referring to, is that there is one additional state increment. I missed the part that, it could be required, so that we reach CPUHP_AP_ONLINE_IDLE before calling __cpuhp_kick_ap(). So, it's not a problem.
* cpuhp_reset_state() does one state increment and we reach CPUHP_AP_ONLINE_IDLE.
if (ret && st->state > CPUHP_TEARDOWN_CPU && st->state < prev_state) {
cpuhp_reset_state(st, prev_state);
__cpuhp_kick_ap(st);
}
CPUHP_TEARDOWN_CPU,
CPUHP_AP_ONLINE_IDLE,
CPUHP_AP_SMPBOOT_THREADS,* cpuhp_thread_fun() does one more increment before invoking state callback (so, we skip CPUHP_AP_ONLINE_IDLE) and we reach CPUHP_AP_SMPBOOT_THREADS:
static void cpuhp_thread_fun(unsigned int cpu)
<snip>
if (bringup) {
st->state++;
state = st->state;
<snip>Signature · 2 lines
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, hosted by The Linux Foundation
Quoted 23 lines · Thomas Gleixner
Quoted 15 lines · Thomas Gleixner
Review
Quoted 11 lines · Neeraj Upadhyay
(quoted diff) +8
-1ret = cpuhp_down_callbacks(cpu, st, target); if (ret && st->state > CPUHP_TEARDOWN_CPU && st->state < prev_state) { - cpuhp_reset_state(st, prev_state); + /* + * As st->last is not set, cpuhp_reset_state() increments + * st->state, which results in CPUHP_AP_SMPBOOT_THREADS being + * skipped during rollback. So, don't use it here. + */ + st->rollback = true; + st->target = prev_state; + st->bringup = !st->bringup;No, this is just papering over the actual problem.
Quoted 6 lines
The state inconsistency happens in
take_cpu_down()when it returns with a failure from__cpu_disable()because that returns with state =TEARDOWN_CPUandst->stateis then incremented inundo_cpu_down().That's the real issue and we need to analyze the whole
cpu_downrollback logic first.
Quoted 6 lines
And looking closer this is a general issue. Just that the TEARDOWN state makes it simple to observe. It's universaly broken, when the first teardown callback fails because,
st->stateis only decremented _AFTER_ the callback returns success, butundo_cpu_down()increments unconditionally.Patch below.
This patch fixes the issue reported @[1]. Lorenzo did some debugging and I wanted to have a look at it at some point but this discussion drew my attention and sounded very similar[2]. So I did a quick test with this patch and it fixes the issue.
--
Regards,
Sudeep
[1] https://lore.kernel.org/lkml/CAMuHMdVg868LgL5xTg5Dp5rReKxoo+8fRy+ETJiMxGWZCp+hWw@mail.gmail.com/
[2] https://lore.kernel.org/lkml/20180823131505.GA31558@red-moon/
Geert Uytterhoeven
Tested-byHi Sudeep,
Thanks for the CC!
Quoted 31 lines · Sudeep Holla
Quoted 23 lines · Thomas Gleixner
Quoted 13 lines · Thomas Gleixner
Review
Quoted 11 lines · Neeraj Upadhyay
ret = cpuhp_down_callbacks(cpu, st, target); if (ret && st->state > CPUHP_TEARDOWN_CPU && st->state < prev_state) {(quoted diff) +8
-1- cpuhp_reset_state(st, prev_state); + /* + * As st->last is not set, cpuhp_reset_state() increments + * st->state, which results in CPUHP_AP_SMPBOOT_THREADS being + * skipped during rollback. So, don't use it here. + */ + st->rollback = true; + st->target = prev_state; + st->bringup = !st->bringup;Quoted 8 lines
No, this is just papering over the actual problem.
The state inconsistency happens in
take_cpu_down()when it returns with a failure from__cpu_disable()because that returns with state =TEARDOWN_CPUandst->stateis then incremented inundo_cpu_down().That's the real issue and we need to analyze the whole
cpu_downrollback logic first.Quoted 6 lines
And looking closer this is a general issue. Just that the TEARDOWN state makes it simple to observe. It's universaly broken, when the first teardown callback fails because,
st->stateis only decremented _AFTER_ the callback returns success, butundo_cpu_down()increments unconditionally.Patch below.
Quoted 7 lines
This patch fixes the issue reported @[1]. Lorenzo did some debugging and I wanted to have a look at it at some point but this discussion drew my attention and sounded very similar[2]. So I did a quick test with this patch and it fixes the issue.
[1] https://lore.kernel.org/lkml/CAMuHMdVg868LgL5xTg5Dp5rReKxoo+8fRy+ETJiMxGWZCp+hWw@mail.gmail.com/
[2] https://lore.kernel.org/lkml/20180823131505.GA31558@red-moon/
Thomas' patch fixes the issue for me:
Gr{oetje,eeting}s,
Geert
Signature · 5 lines
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected] In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that. -- Linus Torvalds
tip-bot for Thomas Gleixner
merge bot[tip:smp/urgent] cpu/hotplug: Prevent state corruption on error rollback
In reply to Thomas Gleixnercpu/hotplug: Prevent state corruption on error rollback
Commit-ID: 69fa6eb7d6a64801ea261025cce9723d9442d773 Gitweb: https://git.kernel.org/tip/69fa6eb7d6a64801ea261025cce9723d9442d773 Author: Thomas Gleixner <[email protected]> AuthorDate: Thu, 6 Sep 2018 15:21:38 +0200 Committer: Thomas Gleixner <[email protected]> CommitDate: Thu, 6 Sep 2018 15:21:38 +0200
cpu/hotplug: Prevent state corruption on error rollback
When a teardown callback fails, the CPU hotplug code brings the CPU back to the previous state. The previous state becomes the new target state. The rollback happens in undo_cpu_down() which increments the state unconditionally even if the state is already the same as the target.
As a consequence the next CPU hotplug operation will start at the wrong state. This is easily to observe when __cpu_disable() fails.
Prevent the unconditional undo by checking the state vs. target before incrementing state and fix up the consequently wrong conditional in the unplug code which handles the failure of the final CPU take down on the control CPU side.
----
--- kernel/cpu.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
kernel/cpu.c +3 -2
@@ -916,7 +916,8 @@ static int cpuhp_down_callbacks(unsigned int cpu, struct cpuhp_cpu_state *st, ret =cpuhp_invoke_callback (cpu, st->state, false,NULL ,NULL );if (ret) { st->target = prev_state; -undo_cpu_down (cpu, st); +if (st->state < prev_state) +undo_cpu_down (cpu, st);break ; } }
@@ -969,7 +970,7 @@ static int __ref _cpu_down(unsigned int cpu, int tasks_frozen,* to do the further cleanups. */ ret =cpuhp_down_callbacks (cpu, st, target); -if (ret && st->state >CPUHP_TEARDOWN_CPU && st->state < prev_state) { +if (ret && st->state ==CPUHP_TEARDOWN_CPU && st->state < prev_state) {cpuhp_reset_state (st, prev_state);__cpuhp_kick_ap (st); }