ACPI: processor: Fix CPU0 wakeup in acpi_idle_play_dead()
PATCH 1/1 ACPI: processor: Fix CPU0 wakeup in acpi_idle_play_dead()
Commit 496121c02127 ("ACPI: processor: idle: Allow probing on platforms with one ACPI C-state") broke CPU0 hotplug on certain systems, e.g. I'm observing the following on AWS Nitro (e.g r5b.xlarge but other instance types are affected as well):
# echo 0 > /sys/devices/system/cpu/cpu0/online # echo 1 > /sys/devices/system/cpu/cpu0/online <10 seconds delay> -bash: echo: write error: Input/output error
In fact, the above mentioned commit only revealed the problem and did not introduce it. On x86, to wakeup CPU an NMI is being used and hlt_play_dead()/mwait_play_dead() loops are prepared to handle it:
/* * If NMI wants to wake up CPU0, start CPU0. */ if (wakeup_cpu0()) start_cpu0();
cpuidle_play_dead() -> acpi_idle_play_dead() (which is now being called on systems where it wasn't called before the above mentioned commit) serves the same purpose but it doesn't have a path for CPU0. What happens now on wakeup is:
- NMI is sent to CPU0
- wakeup_cpu0_nmi() works as expected
- we get back to while (1) loop in acpi_idle_play_dead()
- safe_halt() puts CPU0 to sleep again.
The straightforward/minimal fix is add the special handling for CPU0 on x86 and that's what the patch is doing.
--- arch/x86/include/asm/smp.h | 1 + arch/x86/kernel/smpboot.c | 2 +- drivers/acpi/processor_idle.c | 7 +++++++ 3 files changed, 9 insertions(+), 1 deletion(-)
arch/x86/include/asm/smp.h +1 -0
@@ -132,6 +132,7 @@ void native_play_dead(void);void play_dead_common (void );void wbinvd_on_cpu (int cpu);int wbinvd_on_all_cpus (void ); +bool wakeup_cpu0 (void );void native_smp_send_reschedule (int cpu);void native_send_call_func_ipi (const struct cpumask *mask);
arch/x86/kernel/smpboot.c +1 -1
@@ -1659,7 +1659,7 @@ void play_dead_common(void)local_irq_disable (); } -static bool wakeup_cpu0 (void ) +bool wakeup_cpu0 (void ) {if (smp_processor_id () ==0 && enable_start_cpu0)return true;
drivers/acpi/processor_idle.c +7 -0
@@ -29,6 +29,7 @@*/ #ifdef CONFIG_X86 #include <asm/apic.h> +#include <asm/cpu.h> #endif #define _COMPONENT ACPI_PROCESSOR_COMPONENT
@@ -541,6 +542,12 @@ static int acpi_idle_play_dead(struct cpuidle_device *dev, int index)wait_for_freeze (); }else return -ENODEV ; + +#ifdef CONFIG_X86 +/* If NMI wants to wake up CPU0, start CPU0. */ +if (wakeup_cpu0 ()) +start_cpu0 (); +#endif }/* Never reached */
Signature · 1 line
2.30.2
Rafael J. Wysocki
maintainerQuoted 30 lines · Vitaly Kuznetsov
Commit
496121c02127("ACPI: processor: idle: Allow probing on platforms with one ACPI C-state") broke CPU0 hotplug on certain systems, e.g. I'm observing the following on AWS Nitro (e.g r5b.xlarge but other instance types are affected as well):# echo 0 > /sys/devices/system/cpu/cpu0/online # echo 1 > /sys/devices/system/cpu/cpu0/online <10 seconds delay> -bash: echo: write error: Input/output errorIn fact, the above mentioned commit only revealed the problem and did not introduce it. On x86, to wakeup CPU an NMI is being used and hlt_play_dead()/mwait_play_dead() loops are prepared to handle it:
/* * If NMI wants to wake up CPU0, start CPU0. */ if (wakeup_cpu0()) start_cpu0();
cpuidle_play_dead()->acpi_idle_play_dead()(which is now being called on systems where it wasn't called before the above mentioned commit) serves the same purpose but it doesn't have a path for CPU0. What happens now on wakeup is:
- NMI is sent to CPU0
-wakeup_cpu0_nmi()works as expected
- we get back to while (1) loop inacpi_idle_play_dead()
-safe_halt()puts CPU0 to sleep again.
Quoted 5 lines
The straightforward/minimal fix is add the special handling for CPU0 on x86 and that's what the patch is doing.
Fixes 496121c02127 ("ACPI: processor: idle: Allow probing on platforms with one ACPI C-state")Signed-off-by Vitaly Kuznetsov <[email protected]>
This looks reasonable to me, but I'll give others some time to respond.
Quoted 59 lines
--- arch/x86/include/asm/smp.h | 1 + arch/x86/kernel/smpboot.c | 2 +- drivers/acpi/processor_idle.c | 7 +++++++ 3 files changed, 9 insertions(+), 1 deletion(-)arch/x86/include/asm/smp.h +1
-0@@ -132,6 +132,7 @@ void native_play_dead(void);void play_dead_common (void );void wbinvd_on_cpu (int cpu);int wbinvd_on_all_cpus (void ); +bool wakeup_cpu0 (void );void native_smp_send_reschedule (int cpu);void native_send_call_func_ipi (const struct cpumask *mask);arch/x86/kernel/smpboot.c +1
-1@@ -1659,7 +1659,7 @@ void play_dead_common(void)local_irq_disable (); } -static bool wakeup_cpu0 (void ) +bool wakeup_cpu0 (void ) {if (smp_processor_id () ==0 && enable_start_cpu0)return true;drivers/acpi/processor_idle.c +7
-1@@ -29,6 +29,7 @@*/ #ifdef CONFIG_X86 #include <asm/apic.h> +#include <asm/cpu.h> #endif #define _COMPONENT ACPI_PROCESSOR_COMPONENT@@ -541,6 +542,12 @@ static int acpi_idle_play_dead(struct cpuidle_device *dev, int index)wait_for_freeze (); }else return -ENODEV ; + +#ifdef CONFIG_X86 +/* If NMI wants to wake up CPU0, start CPU0. */ +if (wakeup_cpu0 ()) +start_cpu0 (); +#endif }/* Never reached */ --2.30.2
Rafael J. Wysocki
maintainerQuoted 32 lines · Rafael J. Wysocki
Quoted 30 lines · Vitaly Kuznetsov
Commit
496121c02127("ACPI: processor: idle: Allow probing on platforms with one ACPI C-state") broke CPU0 hotplug on certain systems, e.g. I'm observing the following on AWS Nitro (e.g r5b.xlarge but other instance types are affected as well):# echo 0 > /sys/devices/system/cpu/cpu0/online # echo 1 > /sys/devices/system/cpu/cpu0/online <10 seconds delay> -bash: echo: write error: Input/output errorIn fact, the above mentioned commit only revealed the problem and did not introduce it. On x86, to wakeup CPU an NMI is being used and hlt_play_dead()/mwait_play_dead() loops are prepared to handle it:
/* * If NMI wants to wake up CPU0, start CPU0. */ if (wakeup_cpu0()) start_cpu0();
cpuidle_play_dead()->acpi_idle_play_dead()(which is now being called on systems where it wasn't called before the above mentioned commit) serves the same purpose but it doesn't have a path for CPU0. What happens now on wakeup is:
- NMI is sent to CPU0
-wakeup_cpu0_nmi()works as expected
- we get back to while (1) loop inacpi_idle_play_dead()
-safe_halt()puts CPU0 to sleep again.
Quoted 7 lines
Quoted 5 lines
The straightforward/minimal fix is add the special handling for CPU0 on x86 and that's what the patch is doing.
Fixes 496121c02127 ("ACPI: processor: idle: Allow probing on platforms with one ACPI C-state")Signed-off-by Vitaly Kuznetsov <[email protected]>This looks reasonable to me, but I'll give others some time to respond.
No comments, so applied as 5.12-rc material, thanks!
Quoted 59 lines
Quoted 59 lines
--- arch/x86/include/asm/smp.h | 1 + arch/x86/kernel/smpboot.c | 2 +- drivers/acpi/processor_idle.c | 7 +++++++ 3 files changed, 9 insertions(+), 1 deletion(-)arch/x86/include/asm/smp.h +1
-0@@ -132,6 +132,7 @@ void native_play_dead(void);void play_dead_common (void );void wbinvd_on_cpu (int cpu);int wbinvd_on_all_cpus (void ); +bool wakeup_cpu0 (void );void native_smp_send_reschedule (int cpu);void native_send_call_func_ipi (const struct cpumask *mask);arch/x86/kernel/smpboot.c +1
-1@@ -1659,7 +1659,7 @@ void play_dead_common(void)local_irq_disable (); } -static bool wakeup_cpu0 (void ) +bool wakeup_cpu0 (void ) {if (smp_processor_id () ==0 && enable_start_cpu0)return true;drivers/acpi/processor_idle.c +7
-1@@ -29,6 +29,7 @@*/ #ifdef CONFIG_X86 #include <asm/apic.h> +#include <asm/cpu.h> #endif #define _COMPONENT ACPI_PROCESSOR_COMPONENT@@ -541,6 +542,12 @@ static int acpi_idle_play_dead(struct cpuidle_device *dev, int index)wait_for_freeze (); }else return -ENODEV ; + +#ifdef CONFIG_X86 +/* If NMI wants to wake up CPU0, start CPU0. */ +if (wakeup_cpu0 ()) +start_cpu0 (); +#endif }/* Never reached */ --2.30.2