Allow CPU0 to be nohz full
Subsystems (8)
Nicholas Piggin
reviewerv2 cover Allow CPU0 to be nohz full
Since last time, I added a compile time option to opt-out of this if the platform does not support suspend on non-zero, and tried to improve legibility of changelogs and explain the justification better.
I have been testing this on powerpc/pseries and it seems to work fine (the firmware call to suspend can be called on any CPU and resumes where it left off), but not included here because the code has some bitrot unrelated to this series which I hacked to fix. I will discuss it and either send an acked patch to go with this series if it is small, or fix it in powerpc tree.
Thanks,
Nick
Nicholas Piggin (5):
sched/core: allow the remote scheduler tick to be started on CPU0 PM / suspend: add function to disable secondaries for suspend kernel/cpu: Allow non-zero CPU to be primary for suspend / kexec
freeze
kernel/sched/isolation: require a present CPU in housekeeping mask
nohz_full: Allow the boot CPU to be nohz_full
include/linux/cpu.h | 15 ++++++++++++ kernel/cpu.c | 10 +++++++- kernel/kexec_core.c | 4 ++-- kernel/power/Kconfig | 9 +++++++ kernel/power/hibernate.c | 12 +++++----- kernel/power/suspend.c | 4 ++-- kernel/sched/core.c | 2 +- kernel/sched/isolation.c | 18 ++++++++++---- kernel/time/tick-common.c | 50 +++++++++++++++++++++++++++++++++++---- kernel/time/tick-sched.c | 34 ++++++++++++++++++-------- 11 files changed, 131 insertions(+), 31 deletions(-)
Signature · 1 line
2.20.1
Nicholas Piggin
reviewerPATCH v2 1/5 sched/core: allow the remote scheduler tick to be started on CPU0
This has no effect yet because CPU0 will always be a housekeeping CPU until a later change.
--- kernel/sched/core.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/sched/core.c +1 -1
@@ -5918,7 +5918,7 @@ void __init sched_init_smp(void)static int __init migration_init (void ) { -sched_rq_cpu_starting (smp_processor_id ()); +sched_cpu_starting (smp_processor_id ());return 0 ; }early_initcall (migration_init);
Signature · 1 line
2.20.1
tip-bot for Nicholas Piggin
merge bot[tip:sched/core] sched/core: Allow the remote scheduler tick to be started on CPU0
sched/core: Allow the remote scheduler tick to be started on CPU0
Commit-ID: 77a5352ba977d2554643e3797e10823d0d03dcf7 Gitweb: https://git.kernel.org/tip/77a5352ba977d2554643e3797e10823d0d03dcf7 Author: Nicholas Piggin <[email protected]> AuthorDate: Thu, 11 Apr 2019 13:34:44 +1000 Committer: Ingo Molnar <[email protected]> CommitDate: Fri, 3 May 2019 12:53:14 +0200
sched/core: Allow the remote scheduler tick to be started on CPU0
This has no effect yet because CPU0 will always be a housekeeping CPU until a later change.
--- kernel/sched/core.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/sched/core.c +1 -1
@@ -5866,7 +5866,7 @@ void __init sched_init_smp(void)static int __init migration_init (void ) { -sched_rq_cpu_starting (smp_processor_id ()); +sched_cpu_starting (smp_processor_id ());return 0 ; }early_initcall (migration_init);
Nicholas Piggin
reviewerPATCH v2 2/5 PM / suspend: add function to disable secondaries for suspend
In reply to Nicholas PigginThis adds a function to disable secondary CPUs for suspend that are not necessarily non-zero / non-boot CPUs. Platforms will be able to use this to suspend using non-zero CPUs.
--- include/linux/cpu.h | 10 ++++++++++ kernel/kexec_core.c | 4 ++-- kernel/power/hibernate.c | 12 ++++++------ kernel/power/suspend.c | 4 ++-- 4 files changed, 20 insertions(+), 10 deletions(-)
include/linux/cpu.h +10 -0
@@ -137,6 +137,16 @@ static inline int disable_nonboot_cpus(void) return freeze_secondary_cpus(0); }extern void enable_nonboot_cpus (void ); + +static inline int suspend_disable_secondary_cpus (void ) +{ +return freeze_secondary_cpus (0 ); +} +static inline void suspend_enable_secondary_cpus (void ) +{ +return enable_nonboot_cpus (); +} +#else /* !CONFIG_PM_SLEEP_SMP */ static inline int disable_nonboot_cpus (void ) {return 0 ; }static inline void enable_nonboot_cpus (void ) {}
kernel/kexec_core.c +2 -2
@@ -1150,7 +1150,7 @@ int kernel_kexec(void) error =dpm_suspend_end (PMSG_FREEZE );if (error) goto Resume_devices; - error =disable_nonboot_cpus (); + error =suspend_disable_secondary_cpus ();if (error) goto Enable_cpus;local_irq_disable ();
@@ -1183,7 +1183,7 @@ int kernel_kexec(void) Enable_irqs:local_irq_enable (); Enable_cpus: -enable_nonboot_cpus (); +suspend_enable_secondary_cpus ();dpm_resume_start (PMSG_RESTORE ); Resume_devices:dpm_resume_end (PMSG_RESTORE );
kernel/power/hibernate.c +6 -6
@@ -281,7 +281,7 @@ static int create_image(int platform_mode)if (error ||hibernation_test (TEST_PLATFORM )) goto Platform_finish; - error =disable_nonboot_cpus (); + error =suspend_disable_secondary_cpus ();if (error ||hibernation_test (TEST_CPUS )) goto Enable_cpus;
@@ -323,7 +323,7 @@ static int create_image(int platform_mode)local_irq_enable (); Enable_cpus: -enable_nonboot_cpus (); +suspend_enable_secondary_cpus (); Platform_finish:platform_finish (platform_mode);
@@ -417,7 +417,7 @@ int hibernation_snapshot(int platform_mode)int __weak hibernate_resume_nonboot_cpu_disable (void ) { -return disable_nonboot_cpus (); +return suspend_disable_secondary_cpus (); }/**
@@ -486,7 +486,7 @@ static int resume_target_kernel(bool platform_mode)local_irq_enable (); Enable_cpus: -enable_nonboot_cpus (); +suspend_enable_secondary_cpus (); Cleanup:platform_restore_cleanup (platform_mode);
@@ -564,7 +564,7 @@ int hibernation_platform_enter(void)if (error) goto Platform_finish; - error =disable_nonboot_cpus (); + error =suspend_disable_secondary_cpus ();if (error) goto Enable_cpus;
@@ -586,7 +586,7 @@ int hibernation_platform_enter(void)local_irq_enable (); Enable_cpus: -enable_nonboot_cpus (); +suspend_enable_secondary_cpus (); Platform_finish: hibernation_ops->finish ();
kernel/power/suspend.c +2 -2
@@ -428,7 +428,7 @@ static int suspend_enter(suspend_state_t state, bool *wakeup)if (suspend_test (TEST_PLATFORM )) goto Platform_wake; - error =disable_nonboot_cpus (); + error =suspend_disable_secondary_cpus ();if (error ||suspend_test (TEST_CPUS )) goto Enable_cpus;
@@ -458,7 +458,7 @@ static int suspend_enter(suspend_state_t state, bool *wakeup)BUG_ON (irqs_disabled ()); Enable_cpus: -enable_nonboot_cpus (); +suspend_enable_secondary_cpus (); Platform_wake:platform_resume_noirq (state);
Signature · 1 line
2.20.1
tip-bot for Nicholas Piggin
merge bot[tip:sched/core] power/suspend: Add function to disable secondaries for suspend
power/suspend: Add function to disable secondaries for suspend
Commit-ID: c2cb30bfceceba8a2a0d5713230a250dd6140e22 Gitweb: https://git.kernel.org/tip/c2cb30bfceceba8a2a0d5713230a250dd6140e22 Author: Nicholas Piggin <[email protected]> AuthorDate: Thu, 11 Apr 2019 13:34:45 +1000 Committer: Ingo Molnar <[email protected]> CommitDate: Fri, 3 May 2019 12:53:14 +0200
power/suspend: Add function to disable secondaries for suspend
This adds a function to disable secondary CPUs for suspend that are not necessarily non-zero / non-boot CPUs. Platforms will be able to use this to suspend using non-zero CPUs.
--- include/linux/cpu.h | 10 ++++++++++ kernel/kexec_core.c | 4 ++-- kernel/power/hibernate.c | 12 ++++++------ kernel/power/suspend.c | 4 ++-- 4 files changed, 20 insertions(+), 10 deletions(-)
include/linux/cpu.h +10 -0
@@ -137,6 +137,16 @@ static inline int disable_nonboot_cpus(void) return freeze_secondary_cpus(0); }extern void enable_nonboot_cpus (void ); + +static inline int suspend_disable_secondary_cpus (void ) +{ +return freeze_secondary_cpus (0 ); +} +static inline void suspend_enable_secondary_cpus (void ) +{ +return enable_nonboot_cpus (); +} +#else /* !CONFIG_PM_SLEEP_SMP */ static inline int disable_nonboot_cpus (void ) {return 0 ; }static inline void enable_nonboot_cpus (void ) {}
kernel/kexec_core.c +2 -2
@@ -1150,7 +1150,7 @@ int kernel_kexec(void) error =dpm_suspend_end (PMSG_FREEZE );if (error) goto Resume_devices; - error =disable_nonboot_cpus (); + error =suspend_disable_secondary_cpus ();if (error) goto Enable_cpus;local_irq_disable ();
@@ -1183,7 +1183,7 @@ int kernel_kexec(void) Enable_irqs:local_irq_enable (); Enable_cpus: -enable_nonboot_cpus (); +suspend_enable_secondary_cpus ();dpm_resume_start (PMSG_RESTORE ); Resume_devices:dpm_resume_end (PMSG_RESTORE );
kernel/power/hibernate.c +6 -6
@@ -281,7 +281,7 @@ static int create_image(int platform_mode)if (error ||hibernation_test (TEST_PLATFORM )) goto Platform_finish; - error =disable_nonboot_cpus (); + error =suspend_disable_secondary_cpus ();if (error ||hibernation_test (TEST_CPUS )) goto Enable_cpus;
@@ -323,7 +323,7 @@ static int create_image(int platform_mode)local_irq_enable (); Enable_cpus: -enable_nonboot_cpus (); +suspend_enable_secondary_cpus (); Platform_finish:platform_finish (platform_mode);
@@ -417,7 +417,7 @@ int hibernation_snapshot(int platform_mode)int __weak hibernate_resume_nonboot_cpu_disable (void ) { -return disable_nonboot_cpus (); +return suspend_disable_secondary_cpus (); }/**
@@ -486,7 +486,7 @@ static int resume_target_kernel(bool platform_mode)local_irq_enable (); Enable_cpus: -enable_nonboot_cpus (); +suspend_enable_secondary_cpus (); Cleanup:platform_restore_cleanup (platform_mode);
@@ -564,7 +564,7 @@ int hibernation_platform_enter(void)if (error) goto Platform_finish; - error =disable_nonboot_cpus (); + error =suspend_disable_secondary_cpus ();if (error) goto Enable_cpus;
@@ -586,7 +586,7 @@ int hibernation_platform_enter(void)local_irq_enable (); Enable_cpus: -enable_nonboot_cpus (); +suspend_enable_secondary_cpus (); Platform_finish: hibernation_ops->finish ();
kernel/power/suspend.c +2 -2
@@ -428,7 +428,7 @@ static int suspend_enter(suspend_state_t state, bool *wakeup)if (suspend_test (TEST_PLATFORM )) goto Platform_wake; - error =disable_nonboot_cpus (); + error =suspend_disable_secondary_cpus ();if (error ||suspend_test (TEST_CPUS )) goto Enable_cpus;
@@ -458,7 +458,7 @@ static int suspend_enter(suspend_state_t state, bool *wakeup)BUG_ON (irqs_disabled ()); Enable_cpus: -enable_nonboot_cpus (); +suspend_enable_secondary_cpus (); Platform_wake:platform_resume_noirq (state);
tip-bot for Nicholas Piggin
merge bot[tip:sched/core] power/suspend: Add function to disable secondaries for suspend
In reply to Nicholas Pigginpower/suspend: Add function to disable secondaries for suspend
Commit-ID: 2f1a6fbbef7781382850c3104ecb658f21b5d460 Gitweb: https://git.kernel.org/tip/2f1a6fbbef7781382850c3104ecb658f21b5d460 Author: Nicholas Piggin <[email protected]> AuthorDate: Thu, 11 Apr 2019 13:34:45 +1000 Committer: Ingo Molnar <[email protected]> CommitDate: Fri, 3 May 2019 19:42:41 +0200
power/suspend: Add function to disable secondaries for suspend
This adds a function to disable secondary CPUs for suspend that are not necessarily non-zero / non-boot CPUs. Platforms will be able to use this to suspend using non-zero CPUs.
--- include/linux/cpu.h | 12 ++++++++++++ kernel/kexec_core.c | 4 ++-- kernel/power/hibernate.c | 12 ++++++------ kernel/power/suspend.c | 4 ++-- 4 files changed, 22 insertions(+), 10 deletions(-)
include/linux/cpu.h +12 -0
@@ -137,9 +137,21 @@ static inline int disable_nonboot_cpus(void) return freeze_secondary_cpus(0); }extern void enable_nonboot_cpus (void ); + +static inline int suspend_disable_secondary_cpus (void ) +{ +return freeze_secondary_cpus (0 ); +} +static inline void suspend_enable_secondary_cpus (void ) +{ +return enable_nonboot_cpus (); +} +#else /* !CONFIG_PM_SLEEP_SMP */ static inline int disable_nonboot_cpus (void ) {return 0 ; }static inline void enable_nonboot_cpus (void ) {} +static inline int suspend_disable_secondary_cpus (void ) {return 0 ; } +static inline void suspend_enable_secondary_cpus (void ) { }#endif /* !CONFIG_PM_SLEEP_SMP */ void cpu_startup_entry (enum cpuhp_state state);
kernel/kexec_core.c +2 -2
@@ -1150,7 +1150,7 @@ int kernel_kexec(void) error =dpm_suspend_end (PMSG_FREEZE );if (error) goto Resume_devices; - error =disable_nonboot_cpus (); + error =suspend_disable_secondary_cpus ();if (error) goto Enable_cpus;local_irq_disable ();
@@ -1183,7 +1183,7 @@ int kernel_kexec(void) Enable_irqs:local_irq_enable (); Enable_cpus: -enable_nonboot_cpus (); +suspend_enable_secondary_cpus ();dpm_resume_start (PMSG_RESTORE ); Resume_devices:dpm_resume_end (PMSG_RESTORE );
kernel/power/hibernate.c +6 -6
@@ -281,7 +281,7 @@ static int create_image(int platform_mode)if (error ||hibernation_test (TEST_PLATFORM )) goto Platform_finish; - error =disable_nonboot_cpus (); + error =suspend_disable_secondary_cpus ();if (error ||hibernation_test (TEST_CPUS )) goto Enable_cpus;
@@ -323,7 +323,7 @@ static int create_image(int platform_mode)local_irq_enable (); Enable_cpus: -enable_nonboot_cpus (); +suspend_enable_secondary_cpus (); Platform_finish:platform_finish (platform_mode);
@@ -417,7 +417,7 @@ int hibernation_snapshot(int platform_mode)int __weak hibernate_resume_nonboot_cpu_disable (void ) { -return disable_nonboot_cpus (); +return suspend_disable_secondary_cpus (); }/**
@@ -486,7 +486,7 @@ static int resume_target_kernel(bool platform_mode)local_irq_enable (); Enable_cpus: -enable_nonboot_cpus (); +suspend_enable_secondary_cpus (); Cleanup:platform_restore_cleanup (platform_mode);
@@ -564,7 +564,7 @@ int hibernation_platform_enter(void)if (error) goto Platform_finish; - error =disable_nonboot_cpus (); + error =suspend_disable_secondary_cpus ();if (error) goto Enable_cpus;
@@ -586,7 +586,7 @@ int hibernation_platform_enter(void)local_irq_enable (); Enable_cpus: -enable_nonboot_cpus (); +suspend_enable_secondary_cpus (); Platform_finish: hibernation_ops->finish ();
kernel/power/suspend.c +2 -2
@@ -428,7 +428,7 @@ static int suspend_enter(suspend_state_t state, bool *wakeup)if (suspend_test (TEST_PLATFORM )) goto Platform_wake; - error =disable_nonboot_cpus (); + error =suspend_disable_secondary_cpus ();if (error ||suspend_test (TEST_CPUS )) goto Enable_cpus;
@@ -458,7 +458,7 @@ static int suspend_enter(suspend_state_t state, bool *wakeup)BUG_ON (irqs_disabled ()); Enable_cpus: -enable_nonboot_cpus (); +suspend_enable_secondary_cpus (); Platform_wake:platform_resume_noirq (state);
Nicholas Piggin
reviewerPATCH v2 3/5 kernel/cpu: Allow non-zero CPU to be primary for suspend / kexec freeze
In reply to Nicholas PigginThis patch provides an arch option, ARCH_SUSPEND_NONZERO_CPU, to opt-in to allowing suspend to occur on one of the housekeeping CPUs rather than hardcoded CPU0.
This will allow CPU0 to be a nohz_full CPU with a later change.
It may be possible for platforms with hardware/firmware restrictions on suspend/wake effectively support this by handing off the final stage to CPU0 when kernel housekeeping is no longer required. Another option is to make housekeeping / nohz_full mask dynamic at runtime, but the complexity could not be justified at this time.
--- arch/powerpc/Kconfig | 4 ++++ include/linux/cpu.h | 7 ++++++- kernel/cpu.c | 10 +++++++++- kernel/power/Kconfig | 9 +++++++++ 4 files changed, 28 insertions(+), 2 deletions(-)
arch/powerpc/Kconfig +4 -0
@@ -318,6 +318,10 @@ config ARCH_SUSPEND_POSSIBLE (PPC_85xx && !PPC_E500MC) || PPC_86xx || PPC_PSERIES \ || 44x || 40x +config ARCH_SUSPEND_NONZERO_CPU + def_bool y + depends on PPC_POWERNV || PPC_PSERIES + config PPC_DCR_NATIVE bool
include/linux/cpu.h +6 -1
@@ -140,7 +140,12 @@ extern void enable_nonboot_cpus(void);static inline int suspend_disable_secondary_cpus (void ) { -return freeze_secondary_cpus (0 ); +int cpu =0 ; + +if (IS_ENABLED (CONFIG_PM_SLEEP_SMP_NONZERO_CPU )) + cpu =-1 ; + +return freeze_secondary_cpus (cpu); }static inline void suspend_enable_secondary_cpus (void ) {
kernel/cpu.c +9 -1
@@ -9,6 +9,7 @@#include <linux/notifier.h> #include <linux/sched/signal.h> #include <linux/sched/hotplug.h> +#include <linux/sched/isolation.h> #include <linux/sched/task.h> #include <linux/sched/smt.h> #include <linux/unistd.h>
@@ -1199,8 +1200,15 @@ int freeze_secondary_cpus(int primary)int cpu, error =0 ;cpu_maps_update_begin (); -if (!cpu_online (primary)) +if (primary ==-1 ) { primary =cpumask_first (cpu_online_mask); +if (!housekeeping_cpu (primary,HK_FLAG_TIMER )) + primary =housekeeping_any_cpu (HK_FLAG_TIMER ); + }else { +if (!cpu_online (primary)) + primary =cpumask_first (cpu_online_mask); + } +/* * We take down all of the non-boot CPUs in one shot to avoid races * with the userspace trying to use the CPU hotplug at the same time
kernel/power/Kconfig +9 -0
@@ -114,6 +114,15 @@ config PM_SLEEP_SMP depends on PM_SLEEP select HOTPLUG_CPU +config PM_SLEEP_SMP_NONZERO_CPU + def_bool y + depends on PM_SLEEP_SMP + depends on ARCH_SUSPEND_NONZERO_CPU + ---help--- + If an arch can suspend (for suspend, hibernate, kexec, etc) on a + non-zero numbered CPU, it may define ARCH_SUSPEND_NONZERO_CPU. This + will allow nohz_full mask to include CPU0. + config PM_AUTOSLEEP bool "Opportunistic sleep" depends on PM_SLEEP
Signature · 1 line
2.20.1
Peter Zijlstra
maintainerRe: [PATCH v2 3/5] kernel/cpu: Allow non-zero CPU to be primary for suspend / kexec freeze
Quoted 11 lines · Nicholas Piggin
This patch provides an arch option,
ARCH_SUSPEND_NONZERO_CPU, to opt-in to allowing suspend to occur on one of the housekeeping CPUs rather than hardcoded CPU0.This will allow CPU0 to be a
nohz_fullCPU with a later change.It may be possible for platforms with hardware/firmware restrictions on suspend/wake effectively support this by handing off the final stage to CPU0 when kernel housekeeping is no longer required. Another option is to make housekeeping /
nohz_fullmask dynamic at runtime, but the complexity could not be justified at this time.
Should we not tie this into whatever already allows an achitecture to hotplug CPU-0? For instance, x86 default disallows this but has cpu0_hotpluggable to allow this.
Presumably POWER already allows hotplugging CPU-0 ?
Nicholas Piggin
reviewerRe: [PATCH v2 3/5] kernel/cpu: Allow non-zero CPU to be primary for suspend / kexec freeze
Peter Zijlstra's on April 25, 2019 10:02 pm:
Quoted 13 lines · Nicholas Piggin
Quoted 11 lines
This patch provides an arch option,
ARCH_SUSPEND_NONZERO_CPU, to opt-in to allowing suspend to occur on one of the housekeeping CPUs rather than hardcoded CPU0.This will allow CPU0 to be a
nohz_fullCPU with a later change.It may be possible for platforms with hardware/firmware restrictions on suspend/wake effectively support this by handing off the final stage to CPU0 when kernel housekeeping is no longer required. Another option is to make housekeeping /
nohz_fullmask dynamic at runtime, but the complexity could not be justified at this time.
Quoted 3 lines
Should we not tie this into whatever already allows an achitecture to hotplug CPU-0? For instance, x86 default disallows this but has
cpu0_hotpluggableto allow this.
I didn't know about that option, but I see it still has the suspend / hibernate restriction though, which is what this patch is breaking from.
If we are to prevent suspend completely at boot time, then it's no problem to run cpu0 with nohz_full, but TPTB decided that's a bad thing.
But I have no problem with an arch adding another boot time option or hook into cpu0_hotpluggable that allows you to force nohz_full.
Quoted 1 line
Presumably POWER already allows hotplugging CPU-0 ?
Yeah it does. Suspend in pseries is actually really just used for some proprietary hypervisor partition migration scheme where you suspend, image gets saved, then resume it somewhere else. So that's easy, it's just a hypercall that appears to return exactly as it was called. No restriction on CPU number.
Thanks,
Nick
tip-bot for Nicholas Piggin
merge bot[tip:sched/core] kernel/cpu: Allow non-zero CPU to be primary for suspend / kexec freeze
In reply to Nicholas Pigginkernel/cpu: Allow non-zero CPU to be primary for suspend / kexec freeze
Commit-ID: 9bcd2cd91816020b0600a2078bab51641b1341df Gitweb: https://git.kernel.org/tip/9bcd2cd91816020b0600a2078bab51641b1341df Author: Nicholas Piggin <[email protected]> AuthorDate: Thu, 11 Apr 2019 13:34:46 +1000 Committer: Ingo Molnar <[email protected]> CommitDate: Fri, 3 May 2019 12:53:14 +0200
kernel/cpu: Allow non-zero CPU to be primary for suspend / kexec freeze
This patch provides an arch option, ARCH_SUSPEND_NONZERO_CPU, to opt-in to allowing suspend to occur on one of the housekeeping CPUs rather than hardcoded CPU0.
This will allow CPU0 to be a nohz_full CPU with a later change.
It may be possible for platforms with hardware/firmware restrictions on suspend/wake effectively support this by handing off the final stage to CPU0 when kernel housekeeping is no longer required. Another option is to make housekeeping / nohz_full mask dynamic at runtime, but the complexity could not be justified at this time.
--- arch/powerpc/Kconfig | 4 ++++ include/linux/cpu.h | 7 ++++++- kernel/cpu.c | 10 +++++++++- kernel/power/Kconfig | 9 +++++++++ 4 files changed, 28 insertions(+), 2 deletions(-)
arch/powerpc/Kconfig +4 -0
@@ -318,6 +318,10 @@ config ARCH_SUSPEND_POSSIBLE (PPC_85xx && !PPC_E500MC) || PPC_86xx || PPC_PSERIES \ || 44x || 40x +config ARCH_SUSPEND_NONZERO_CPU + def_bool y + depends on PPC_POWERNV || PPC_PSERIES + config PPC_DCR_NATIVE bool
include/linux/cpu.h +6 -1
@@ -140,7 +140,12 @@ extern void enable_nonboot_cpus(void);static inline int suspend_disable_secondary_cpus (void ) { -return freeze_secondary_cpus (0 ); +int cpu =0 ; + +if (IS_ENABLED (CONFIG_PM_SLEEP_SMP_NONZERO_CPU )) + cpu =-1 ; + +return freeze_secondary_cpus (cpu); }static inline void suspend_enable_secondary_cpus (void ) {
kernel/cpu.c +9 -1
@@ -9,6 +9,7 @@#include <linux/notifier.h> #include <linux/sched/signal.h> #include <linux/sched/hotplug.h> +#include <linux/sched/isolation.h> #include <linux/sched/task.h> #include <linux/sched/smt.h> #include <linux/unistd.h>
@@ -1199,8 +1200,15 @@ int freeze_secondary_cpus(int primary)int cpu, error =0 ;cpu_maps_update_begin (); -if (!cpu_online (primary)) +if (primary ==-1 ) { primary =cpumask_first (cpu_online_mask); +if (!housekeeping_cpu (primary,HK_FLAG_TIMER )) + primary =housekeeping_any_cpu (HK_FLAG_TIMER ); + }else { +if (!cpu_online (primary)) + primary =cpumask_first (cpu_online_mask); + } +/* * We take down all of the non-boot CPUs in one shot to avoid races * with the userspace trying to use the CPU hotplug at the same time
kernel/power/Kconfig +9 -0
@@ -114,6 +114,15 @@ config PM_SLEEP_SMP depends on PM_SLEEP select HOTPLUG_CPU +config PM_SLEEP_SMP_NONZERO_CPU + def_bool y + depends on PM_SLEEP_SMP + depends on ARCH_SUSPEND_NONZERO_CPU + ---help--- + If an arch can suspend (for suspend, hibernate, kexec, etc) on a + non-zero numbered CPU, it may define ARCH_SUSPEND_NONZERO_CPU. This + will allow nohz_full mask to include CPU0. + config PM_AUTOSLEEP bool "Opportunistic sleep" depends on PM_SLEEP
tip-bot for Nicholas Piggin
merge bot[tip:sched/core] kernel/cpu: Allow non-zero CPU to be primary for suspend / kexec freeze
In reply to Nicholas Pigginkernel/cpu: Allow non-zero CPU to be primary for suspend / kexec freeze
Commit-ID: 9ca12ac04bb7d7cfb28aa549dcd3d15761f15543 Gitweb: https://git.kernel.org/tip/9ca12ac04bb7d7cfb28aa549dcd3d15761f15543 Author: Nicholas Piggin <[email protected]> AuthorDate: Thu, 11 Apr 2019 13:34:46 +1000 Committer: Ingo Molnar <[email protected]> CommitDate: Fri, 3 May 2019 19:42:58 +0200
kernel/cpu: Allow non-zero CPU to be primary for suspend / kexec freeze
This patch provides an arch option, ARCH_SUSPEND_NONZERO_CPU, to opt-in to allowing suspend to occur on one of the housekeeping CPUs rather than hardcoded CPU0.
This will allow CPU0 to be a nohz_full CPU with a later change.
It may be possible for platforms with hardware/firmware restrictions on suspend/wake effectively support this by handing off the final stage to CPU0 when kernel housekeeping is no longer required. Another option is to make housekeeping / nohz_full mask dynamic at runtime, but the complexity could not be justified at this time.
--- arch/powerpc/Kconfig | 4 ++++ include/linux/cpu.h | 7 ++++++- kernel/cpu.c | 10 +++++++++- kernel/power/Kconfig | 9 +++++++++ 4 files changed, 28 insertions(+), 2 deletions(-)
arch/powerpc/Kconfig +4 -0
@@ -318,6 +318,10 @@ config ARCH_SUSPEND_POSSIBLE (PPC_85xx && !PPC_E500MC) || PPC_86xx || PPC_PSERIES \ || 44x || 40x +config ARCH_SUSPEND_NONZERO_CPU + def_bool y + depends on PPC_POWERNV || PPC_PSERIES + config PPC_DCR_NATIVE bool
include/linux/cpu.h +6 -1
@@ -140,7 +140,12 @@ extern void enable_nonboot_cpus(void);static inline int suspend_disable_secondary_cpus (void ) { -return freeze_secondary_cpus (0 ); +int cpu =0 ; + +if (IS_ENABLED (CONFIG_PM_SLEEP_SMP_NONZERO_CPU )) + cpu =-1 ; + +return freeze_secondary_cpus (cpu); }static inline void suspend_enable_secondary_cpus (void ) {
kernel/cpu.c +9 -1
@@ -9,6 +9,7 @@#include <linux/notifier.h> #include <linux/sched/signal.h> #include <linux/sched/hotplug.h> +#include <linux/sched/isolation.h> #include <linux/sched/task.h> #include <linux/sched/smt.h> #include <linux/unistd.h>
@@ -1199,8 +1200,15 @@ int freeze_secondary_cpus(int primary)int cpu, error =0 ;cpu_maps_update_begin (); -if (!cpu_online (primary)) +if (primary ==-1 ) { primary =cpumask_first (cpu_online_mask); +if (!housekeeping_cpu (primary,HK_FLAG_TIMER )) + primary =housekeeping_any_cpu (HK_FLAG_TIMER ); + }else { +if (!cpu_online (primary)) + primary =cpumask_first (cpu_online_mask); + } +/* * We take down all of the non-boot CPUs in one shot to avoid races * with the userspace trying to use the CPU hotplug at the same time
kernel/power/Kconfig +9 -0
@@ -114,6 +114,15 @@ config PM_SLEEP_SMP depends on PM_SLEEP select HOTPLUG_CPU +config PM_SLEEP_SMP_NONZERO_CPU + def_bool y + depends on PM_SLEEP_SMP + depends on ARCH_SUSPEND_NONZERO_CPU + ---help--- + If an arch can suspend (for suspend, hibernate, kexec, etc) on a + non-zero numbered CPU, it may define ARCH_SUSPEND_NONZERO_CPU. This + will allow nohz_full mask to include CPU0. + config PM_AUTOSLEEP bool "Opportunistic sleep" depends on PM_SLEEP
Nicholas Piggin
reviewerPATCH v2 4/5 kernel/sched/isolation: require a present CPU in housekeeping mask
In reply to Nicholas PigginDuring housekeeping mask setup, currently a possible CPU is required. That does not guarantee the CPU would be available at boot time, so check to ensure that at least one present CPU is in the mask.
--- kernel/sched/isolation.c | 18 +++++++++++++----- 1 file changed, 13 insertions(+), 5 deletions(-)
kernel/sched/isolation.c +13 -5
@@ -65,6 +65,7 @@ void __init housekeeping_init(void)static int __init housekeeping_setup (char *str,enum hk_flags flags) {cpumask_var_t non_housekeeping_mask; +cpumask_var_t tmp;int err;alloc_bootmem_cpumask_var (&non_housekeeping_mask);
@@ -75,16 +76,23 @@ static int __init housekeeping_setup(char *str, enum hk_flags flags)return 0 ; } +alloc_bootmem_cpumask_var (&tmp);if (!housekeeping_flags) {alloc_bootmem_cpumask_var (&housekeeping_mask);cpumask_andnot (housekeeping_mask, cpu_possible_mask, non_housekeeping_mask); -if (cpumask_empty (housekeeping_mask)) + +cpumask_andnot (tmp, cpu_present_mask, non_housekeeping_mask); +if (cpumask_empty (tmp)) { +pr_warn ("Housekeeping: must include one present CPU, " +"using boot CPU:%d\n" ,smp_processor_id ());__cpumask_set_cpu (smp_processor_id (), housekeeping_mask); +__cpumask_clear_cpu (smp_processor_id (), non_housekeeping_mask); + } }else { -cpumask_var_t tmp; - -alloc_bootmem_cpumask_var (&tmp); +cpumask_andnot (tmp, cpu_present_mask, non_housekeeping_mask); +if (cpumask_empty (tmp)) +__cpumask_clear_cpu (smp_processor_id (), non_housekeeping_mask);cpumask_andnot (tmp, cpu_possible_mask, non_housekeeping_mask);if (!cpumask_equal (tmp, housekeeping_mask)) {pr_warn ("Housekeeping: nohz_full= must match isolcpus=\n" );
@@ -92,8 +100,8 @@ static int __init housekeeping_setup(char *str, enum hk_flags flags)free_bootmem_cpumask_var (non_housekeeping_mask);return 0 ; } -free_bootmem_cpumask_var (tmp); } +free_bootmem_cpumask_var (tmp);if ((flags &HK_FLAG_TICK ) && !(housekeeping_flags &HK_FLAG_TICK )) {if (IS_ENABLED (CONFIG_NO_HZ_FULL )) {
Signature · 1 line
2.20.1
tip-bot for Nicholas Piggin
merge bot[tip:sched/core] sched/isolation: Require a present CPU in housekeeping mask
sched/isolation: Require a present CPU in housekeeping mask
Commit-ID: e9a140c6d20ee29951a193342ebaccf07ebc63eb Gitweb: https://git.kernel.org/tip/e9a140c6d20ee29951a193342ebaccf07ebc63eb Author: Nicholas Piggin <[email protected]> AuthorDate: Thu, 11 Apr 2019 13:34:47 +1000 Committer: Ingo Molnar <[email protected]> CommitDate: Fri, 3 May 2019 12:53:14 +0200
sched/isolation: Require a present CPU in housekeeping mask
During housekeeping mask setup, currently a possible CPU is required. That does not guarantee the CPU would be available at boot time, so check to ensure that at least one present CPU is in the mask.
--- kernel/sched/isolation.c | 18 +++++++++++++----- 1 file changed, 13 insertions(+), 5 deletions(-)
kernel/sched/isolation.c +13 -5
@@ -65,6 +65,7 @@ void __init housekeeping_init(void)static int __init housekeeping_setup (char *str,enum hk_flags flags) {cpumask_var_t non_housekeeping_mask; +cpumask_var_t tmp;int err;alloc_bootmem_cpumask_var (&non_housekeeping_mask);
@@ -75,16 +76,23 @@ static int __init housekeeping_setup(char *str, enum hk_flags flags)return 0 ; } +alloc_bootmem_cpumask_var (&tmp);if (!housekeeping_flags) {alloc_bootmem_cpumask_var (&housekeeping_mask);cpumask_andnot (housekeeping_mask, cpu_possible_mask, non_housekeeping_mask); -if (cpumask_empty (housekeeping_mask)) + +cpumask_andnot (tmp, cpu_present_mask, non_housekeeping_mask); +if (cpumask_empty (tmp)) { +pr_warn ("Housekeeping: must include one present CPU, " +"using boot CPU:%d\n" ,smp_processor_id ());__cpumask_set_cpu (smp_processor_id (), housekeeping_mask); +__cpumask_clear_cpu (smp_processor_id (), non_housekeeping_mask); + } }else { -cpumask_var_t tmp; - -alloc_bootmem_cpumask_var (&tmp); +cpumask_andnot (tmp, cpu_present_mask, non_housekeeping_mask); +if (cpumask_empty (tmp)) +__cpumask_clear_cpu (smp_processor_id (), non_housekeeping_mask);cpumask_andnot (tmp, cpu_possible_mask, non_housekeeping_mask);if (!cpumask_equal (tmp, housekeeping_mask)) {pr_warn ("Housekeeping: nohz_full= must match isolcpus=\n" );
@@ -92,8 +100,8 @@ static int __init housekeeping_setup(char *str, enum hk_flags flags)free_bootmem_cpumask_var (non_housekeeping_mask);return 0 ; } -free_bootmem_cpumask_var (tmp); } +free_bootmem_cpumask_var (tmp);if ((flags &HK_FLAG_TICK ) && !(housekeeping_flags &HK_FLAG_TICK )) {if (IS_ENABLED (CONFIG_NO_HZ_FULL )) {
tip-bot for Nicholas Piggin
merge bot[tip:sched/core] sched/isolation: Require a present CPU in housekeeping mask
In reply to Nicholas Pigginsched/isolation: Require a present CPU in housekeeping mask
Commit-ID: 9219565aa89033a9cfdae788c1940473a1253d6c Gitweb: https://git.kernel.org/tip/9219565aa89033a9cfdae788c1940473a1253d6c Author: Nicholas Piggin <[email protected]> AuthorDate: Thu, 11 Apr 2019 13:34:47 +1000 Committer: Ingo Molnar <[email protected]> CommitDate: Fri, 3 May 2019 19:42:58 +0200
sched/isolation: Require a present CPU in housekeeping mask
During housekeeping mask setup, currently a possible CPU is required. That does not guarantee the CPU would be available at boot time, so check to ensure that at least one present CPU is in the mask.
--- kernel/sched/isolation.c | 18 +++++++++++++----- 1 file changed, 13 insertions(+), 5 deletions(-)
kernel/sched/isolation.c +13 -5
@@ -65,6 +65,7 @@ void __init housekeeping_init(void)static int __init housekeeping_setup (char *str,enum hk_flags flags) {cpumask_var_t non_housekeeping_mask; +cpumask_var_t tmp;int err;alloc_bootmem_cpumask_var (&non_housekeeping_mask);
@@ -75,16 +76,23 @@ static int __init housekeeping_setup(char *str, enum hk_flags flags)return 0 ; } +alloc_bootmem_cpumask_var (&tmp);if (!housekeeping_flags) {alloc_bootmem_cpumask_var (&housekeeping_mask);cpumask_andnot (housekeeping_mask, cpu_possible_mask, non_housekeeping_mask); -if (cpumask_empty (housekeeping_mask)) + +cpumask_andnot (tmp, cpu_present_mask, non_housekeeping_mask); +if (cpumask_empty (tmp)) { +pr_warn ("Housekeeping: must include one present CPU, " +"using boot CPU:%d\n" ,smp_processor_id ());__cpumask_set_cpu (smp_processor_id (), housekeeping_mask); +__cpumask_clear_cpu (smp_processor_id (), non_housekeeping_mask); + } }else { -cpumask_var_t tmp; - -alloc_bootmem_cpumask_var (&tmp); +cpumask_andnot (tmp, cpu_present_mask, non_housekeeping_mask); +if (cpumask_empty (tmp)) +__cpumask_clear_cpu (smp_processor_id (), non_housekeeping_mask);cpumask_andnot (tmp, cpu_possible_mask, non_housekeeping_mask);if (!cpumask_equal (tmp, housekeeping_mask)) {pr_warn ("Housekeeping: nohz_full= must match isolcpus=\n" );
@@ -92,8 +100,8 @@ static int __init housekeeping_setup(char *str, enum hk_flags flags)free_bootmem_cpumask_var (non_housekeeping_mask);return 0 ; } -free_bootmem_cpumask_var (tmp); } +free_bootmem_cpumask_var (tmp);if ((flags &HK_FLAG_TICK ) && !(housekeeping_flags &HK_FLAG_TICK )) {if (IS_ENABLED (CONFIG_NO_HZ_FULL )) {
Re: [tip:sched/core] sched/isolation: Require a present CPU in housekeeping mask
Quoted 46 lines · tip-bot for Nicholas Piggin
Commit-ID: 9219565aa89033a9cfdae788c1940473a1253d6c Gitweb: https://git.kernel.org/tip/9219565aa89033a9cfdae788c1940473a1253d6c Author: Nicholas Piggin <[email protected]> AuthorDate: Thu, 11 Apr 2019 13:34:47 +1000 Committer: Ingo Molnar <[email protected]> CommitDate: Fri, 3 May 2019 19:42:58 +0200sched/isolation: Require a present CPU in housekeeping mask
During housekeeping mask setup, currently a possible CPU is required. That does not guarantee the CPU would be available at boot time, so check to ensure that at least one present CPU is in the mask.
Signed-off-by Nicholas Piggin <[email protected]>Signed-off-by Peter Zijlstra (Intel) <[email protected]>Cc Frederic Weisbecker <[email protected]>Cc Linus Torvalds <[email protected]>Cc Peter Zijlstra <[email protected]>Cc Rafael J . Wysocki <[email protected]>Cc Thomas Gleixner <[email protected]>Signed-off-by Ingo Molnar <[email protected]>--- kernel/sched/isolation.c | 18 +++++++++++++----- 1 file changed, 13 insertions(+), 5 deletions(-)kernel/sched/isolation.c +2
-0@@ -65,6 +65,7 @@ void __init housekeeping_init(void)static int __init housekeeping_setup (char *str,enum hk_flags flags) {cpumask_var_t non_housekeeping_mask; +cpumask_var_t tmp;int err;alloc_bootmem_cpumask_var (&non_housekeeping_mask);@@ -75,16 +76,23 @@ static int __init housekeeping_setup(char *str, enum hk_flags flags)return 0 ; } +alloc_bootmem_cpumask_var (&tmp);if (!housekeeping_flags) {alloc_bootmem_cpumask_var (&housekeeping_mask);
kernel/sched/isolation.c static int __init housekeeping_setup(char *str, enum hk_flags flags) · line 90
Quoted 10 lines
kernel/sched/isolation.c +6
-1cpumask_andnot (housekeeping_mask, cpu_possible_mask, non_housekeeping_mask); -if (cpumask_empty (housekeeping_mask)) + +cpumask_andnot (tmp, cpu_present_mask, non_housekeeping_mask); +if (cpumask_empty (tmp)) { +pr_warn ("Housekeeping: must include one present CPU, " +"using boot CPU:%d\n" ,smp_processor_id ());__cpumask_set_cpu (smp_processor_id (), housekeeping_mask); +__cpumask_clear_cpu (smp_processor_id (), non_housekeeping_mask);
Ah that line (along with its twin below) is actually also a fix for the upstream code. If the housekeeping mask is empty and we force the current one, we must make sure that the current CPU is excluded from the nohz_full set. Ideally it should have come as a separate patch but it's nice that you fixed it anyway.
Thanks!
Quoted 21 lines
kernel/sched/isolation.c +5
-4+ } }else { -cpumask_var_t tmp; - -alloc_bootmem_cpumask_var (&tmp); +cpumask_andnot (tmp, cpu_present_mask, non_housekeeping_mask); +if (cpumask_empty (tmp)) +__cpumask_clear_cpu (smp_processor_id (), non_housekeeping_mask);cpumask_andnot (tmp, cpu_possible_mask, non_housekeeping_mask);if (!cpumask_equal (tmp, housekeeping_mask)) {pr_warn ("Housekeeping: nohz_full= must match isolcpus=\n" );@@ -92,8 +100,8 @@ static int __init housekeeping_setup(char *str, enum hk_flags flags)free_bootmem_cpumask_var (non_housekeeping_mask);return 0 ; } -free_bootmem_cpumask_var (tmp); } +free_bootmem_cpumask_var (tmp);if ((flags &HK_FLAG_TICK ) && !(housekeeping_flags &HK_FLAG_TICK )) {if (IS_ENABLED (CONFIG_NO_HZ_FULL )) {
Frederic Weisbecker
maintainerRe: [tip:sched/core] sched/isolation: Require a present CPU in housekeeping mask
In reply to tip-bot for Nicholas PigginQuoted 7 lines · tip-bot for Nicholas Piggin
Commit-ID: 9219565aa89033a9cfdae788c1940473a1253d6c Gitweb: https://git.kernel.org/tip/9219565aa89033a9cfdae788c1940473a1253d6c Author: Nicholas Piggin <[email protected]> AuthorDate: Thu, 11 Apr 2019 13:34:47 +1000 Committer: Ingo Molnar <[email protected]> CommitDate: Fri, 3 May 2019 19:42:58 +0200
Quoted 5 lines
sched/isolation: Require a present CPU in housekeeping mask
During housekeeping mask setup, currently a possible CPU is required. That does not guarantee the CPU would be available at boot time, so check to ensure that at least one present CPU is in the mask.
I have a doubt about the requirements and semantics of cpu_present_mask. IIUC a present CPU means that it is physically plugged in (from ACPI perspective) but might not be logically plugged in (set on cpu_online_mask).
But do we have the guarantee that a present CPU _will_ be online at least once right after the boot? After all, kernel parameters such as "maxcpus=" can prevent from turning some CPUs on. I guess there are even more creative ways to achieve that.
In any case we really require the housekeeper to be forced online. Perhaps I missed that enforcement somewhere in the patchset?
Thanks.
Nicholas Piggin
reviewerRe: [tip:sched/core] sched/isolation: Require a present CPU in housekeeping mask
Frederic Weisbecker's on May 4, 2019 10:27 am:
Quoted 10 lines · tip-bot for Nicholas Piggin
Quoted 9 lines
Commit-ID: 9219565aa89033a9cfdae788c1940473a1253d6c Gitweb: https://git.kernel.org/tip/9219565aa89033a9cfdae788c1940473a1253d6c Author: Nicholas Piggin <[email protected]> AuthorDate: Thu, 11 Apr 2019 13:34:47 +1000 Committer: Ingo Molnar <[email protected]> CommitDate: Fri, 3 May 2019 19:42:58 +0200sched/isolation: Require a present CPU in housekeeping mask
Quoted 7 lines
Quoted 3 lines
During housekeeping mask setup, currently a possible CPU is required. That does not guarantee the CPU would be available at boot time, so check to ensure that at least one present CPU is in the mask.
I have a doubt about the requirements and semantics of
cpu_present_mask. IIUC a present CPU means that it is physically plugged in (from ACPI perspective) but might not be logically plugged in (set oncpu_online_mask).
Right, a superset of cpu_possible_mask, subset of cpu_online_mask. It means that CPU can be brought online at any time.
Quoted 7 lines
But do we have the guarantee that a present CPU _will_ be online at least once right after the boot? After all, kernel parameters such as "maxcpus=" can prevent from turning some CPUs on. I guess there are even more creative ways to achieve that.
In any case we really require the housekeeper to be forced online. Perhaps I missed that enforcement somewhere in the patchset?
No I think you're right, that may be able to boot without anything in the housekeeping mask. Maybe we can just cpu_up() a CPU in the housekeeping mask with a warning that it has overidden their SMP command line option. I'll take a look at it.
Thanks,
Nick
Frederic Weisbecker
maintainerRe: [tip:sched/core] sched/isolation: Require a present CPU in housekeeping mask
Quoted 27 lines · Nicholas Piggin
Frederic Weisbecker's on May 4, 2019 10:27 am:
Quoted 10 lines · tip-bot for Nicholas Piggin
Quoted 9 lines
Commit-ID: 9219565aa89033a9cfdae788c1940473a1253d6c Gitweb: https://git.kernel.org/tip/9219565aa89033a9cfdae788c1940473a1253d6c Author: Nicholas Piggin <[email protected]> AuthorDate: Thu, 11 Apr 2019 13:34:47 +1000 Committer: Ingo Molnar <[email protected]> CommitDate: Fri, 3 May 2019 19:42:58 +0200sched/isolation: Require a present CPU in housekeeping mask
Quoted 7 lines
Quoted 3 lines
During housekeeping mask setup, currently a possible CPU is required. That does not guarantee the CPU would be available at boot time, so check to ensure that at least one present CPU is in the mask.
I have a doubt about the requirements and semantics of
cpu_present_mask. IIUC a present CPU means that it is physically plugged in (from ACPI perspective) but might not be logically plugged in (set oncpu_online_mask).Right, a superset of
cpu_possible_mask, subset ofcpu_online_mask. It means that CPU can be brought online at any time.Quoted 5 lines
But do we have the guarantee that a present CPU _will_ be online at least once right after the boot? After all, kernel parameters such as "maxcpus=" can prevent from turning some CPUs on. I guess there are even more creative ways to achieve that.
Quoted 7 lines
Quoted 2 lines
In any case we really require the housekeeper to be forced online. Perhaps I missed that enforcement somewhere in the patchset?
No I think you're right, that may be able to boot without anything in the housekeeping mask. Maybe we can just
cpu_up()a CPU in the housekeeping mask with a warning that it has overidden their SMP command line option. I'll take a look at it.
But then what if cpu_up() fails? In this case I can think of only two answers:
* Force the boot CPU as the housekeeper.
* Rollback the whole thing: nohz and all isolation.
The second solution looks sane to me. After all if the user doesn't include CPU 0 in the housekeeping set, forcing it isn't going to help much.
But that means we must enhance the isolation code (nohz included) to be able to dynamically add/del CPUs to the houseeeping/isolation set. That's not going to be easy but it's a necessary evolution of that subsystem since we want to drive it through cpusets.
I should start working on that.
Nicholas Piggin
reviewerRe: [tip:sched/core] sched/isolation: Require a present CPU in housekeeping mask
Frederic Weisbecker's on May 7, 2019 1:16 am:
Quoted 36 lines · Nicholas Piggin
Quoted 27 lines
Frederic Weisbecker's on May 4, 2019 10:27 am:
Quoted 10 lines · tip-bot for Nicholas Piggin
Quoted 9 lines
Commit-ID: 9219565aa89033a9cfdae788c1940473a1253d6c Gitweb: https://git.kernel.org/tip/9219565aa89033a9cfdae788c1940473a1253d6c Author: Nicholas Piggin <[email protected]> AuthorDate: Thu, 11 Apr 2019 13:34:47 +1000 Committer: Ingo Molnar <[email protected]> CommitDate: Fri, 3 May 2019 19:42:58 +0200sched/isolation: Require a present CPU in housekeeping mask
Quoted 7 lines
Quoted 3 lines
During housekeeping mask setup, currently a possible CPU is required. That does not guarantee the CPU would be available at boot time, so check to ensure that at least one present CPU is in the mask.
I have a doubt about the requirements and semantics of
cpu_present_mask. IIUC a present CPU means that it is physically plugged in (from ACPI perspective) but might not be logically plugged in (set oncpu_online_mask).Right, a superset of
cpu_possible_mask, subset ofcpu_online_mask. It means that CPU can be brought online at any time.Quoted 5 lines
But do we have the guarantee that a present CPU _will_ be online at least once right after the boot? After all, kernel parameters such as "maxcpus=" can prevent from turning some CPUs on. I guess there are even more creative ways to achieve that.
Quoted 7 lines
Quoted 2 lines
In any case we really require the housekeeper to be forced online. Perhaps I missed that enforcement somewhere in the patchset?
No I think you're right, that may be able to boot without anything in the housekeeping mask. Maybe we can just
cpu_up()a CPU in the housekeeping mask with a warning that it has overidden their SMP command line option. I'll take a look at it.
Quoted 5 lines
But then what if
cpu_up()fails? In this case I can think of only two answers:* Force the boot CPU as the housekeeper.
* Rollback the whole thing: nohz and all isolation.
If cpu_up fails despite being in the present map and we explicitly selected it as the housekeeper? I think it would be okay to print a message telling admin to correct the config, and panic.
We try a best effort to make the system boot and limp along, but if you misconfigure it, crashing is not unreasonable. There's lots of command line option misconfiguration that will cause the same thing.
The primary problem with my patch that needs to be addressed is that the error is not explicitly caught and printed if the housekeeper does not come up, so the system might die in non-obvious ways.
Quoted 11 lines
The second solution looks sane to me. After all if the user doesn't include CPU 0 in the housekeeping set, forcing it isn't going to help much.
But that means we must enhance the isolation code (nohz included) to be able to dynamically add/del CPUs to the houseeeping/isolation set. That's not going to be easy but it's a necessary evolution of that subsystem since we want to drive it through cpusets.
I should start working on that.
I considered that when looking at the series, but couldn't justify the complexity based on my usage (which is static boot time).
If you have other uses for it, then that would solve all these boot time issues as well, which will be nice.
Thanks,
Nick
Frederic Weisbecker
maintainerRe: [tip:sched/core] sched/isolation: Require a present CPU in housekeeping mask
Quoted 47 lines · Nicholas Piggin
Frederic Weisbecker's on May 7, 2019 1:16 am:
Quoted 36 lines · Nicholas Piggin
Quoted 27 lines
Frederic Weisbecker's on May 4, 2019 10:27 am:
Quoted 10 lines · tip-bot for Nicholas Piggin
Quoted 9 lines
Commit-ID: 9219565aa89033a9cfdae788c1940473a1253d6c Gitweb: https://git.kernel.org/tip/9219565aa89033a9cfdae788c1940473a1253d6c Author: Nicholas Piggin <[email protected]> AuthorDate: Thu, 11 Apr 2019 13:34:47 +1000 Committer: Ingo Molnar <[email protected]> CommitDate: Fri, 3 May 2019 19:42:58 +0200sched/isolation: Require a present CPU in housekeeping mask
Quoted 7 lines
Quoted 3 lines
During housekeeping mask setup, currently a possible CPU is required. That does not guarantee the CPU would be available at boot time, so check to ensure that at least one present CPU is in the mask.
I have a doubt about the requirements and semantics of
cpu_present_mask. IIUC a present CPU means that it is physically plugged in (from ACPI perspective) but might not be logically plugged in (set oncpu_online_mask).Right, a superset of
cpu_possible_mask, subset ofcpu_online_mask. It means that CPU can be brought online at any time.Quoted 5 lines
But do we have the guarantee that a present CPU _will_ be online at least once right after the boot? After all, kernel parameters such as "maxcpus=" can prevent from turning some CPUs on. I guess there are even more creative ways to achieve that.
Quoted 7 lines
Quoted 2 lines
In any case we really require the housekeeper to be forced online. Perhaps I missed that enforcement somewhere in the patchset?
No I think you're right, that may be able to boot without anything in the housekeeping mask. Maybe we can just
cpu_up()a CPU in the housekeeping mask with a warning that it has overidden their SMP command line option. I'll take a look at it.Quoted 5 lines
But then what if
cpu_up()fails? In this case I can think of only two answers:* Force the boot CPU as the housekeeper.
* Rollback the whole thing: nohz and all isolation.If
cpu_upfails despite being in the present map and we explicitly selected it as the housekeeper? I think it would be okay to print a message telling admin to correct the config, and panic.
Quoted 7 lines
We try a best effort to make the system boot and limp along, but if you misconfigure it, crashing is not unreasonable. There's lots of command line option misconfiguration that will cause the same thing.
The primary problem with my patch that needs to be addressed is that the error is not explicitly caught and printed if the housekeeper does not come up, so the system might die in non-obvious ways.
I usually reserve panic and BUG_ON() to last resort when data integrity is directly threatened. But indeed I guess that's all we have for now.
If we take that path, I'd rather not call that cpu_up() and simply panic if the given CPU happens not to be online after SMP bootup.
Quoted 11 lines
Quoted 11 lines
The second solution looks sane to me. After all if the user doesn't include CPU 0 in the housekeeping set, forcing it isn't going to help much.
But that means we must enhance the isolation code (nohz included) to be able to dynamically add/del CPUs to the houseeeping/isolation set. That's not going to be easy but it's a necessary evolution of that subsystem since we want to drive it through cpusets.
Quoted 7 lines
Quoted 1 line
I should start working on that.
I considered that when looking at the series, but couldn't justify the complexity based on my usage (which is static boot time).
If you have other uses for it, then that would solve all these boot time issues as well, which will be nice.
Yeah cpuset is going to be a usecase.
I'm going to work on that so that the boot CPU is always the housekeeper in the beginning, then that duty can be later passed to any secondary CPU or the whole can be rolled back. The current situation with the temporary housekeeper that isn't a real one makes me a bit uncomfortable.
Thanks.
Nicholas Piggin
reviewerRe: [tip:sched/core] sched/isolation: Require a present CPU in housekeeping mask
Frederic Weisbecker's on May 8, 2019 10:35 am:
Quoted 52 lines · Nicholas Piggin
Quoted 51 lines
Frederic Weisbecker's on May 7, 2019 1:16 am:
Quoted 36 lines · Nicholas Piggin
Quoted 27 lines
Frederic Weisbecker's on May 4, 2019 10:27 am:
Quoted 10 lines · tip-bot for Nicholas Piggin
Quoted 9 lines
Commit-ID: 9219565aa89033a9cfdae788c1940473a1253d6c Gitweb: https://git.kernel.org/tip/9219565aa89033a9cfdae788c1940473a1253d6c Author: Nicholas Piggin <[email protected]> AuthorDate: Thu, 11 Apr 2019 13:34:47 +1000 Committer: Ingo Molnar <[email protected]> CommitDate: Fri, 3 May 2019 19:42:58 +0200sched/isolation: Require a present CPU in housekeeping mask
Quoted 7 lines
Quoted 3 lines
During housekeeping mask setup, currently a possible CPU is required. That does not guarantee the CPU would be available at boot time, so check to ensure that at least one present CPU is in the mask.
I have a doubt about the requirements and semantics of
cpu_present_mask. IIUC a present CPU means that it is physically plugged in (from ACPI perspective) but might not be logically plugged in (set oncpu_online_mask).Right, a superset of
cpu_possible_mask, subset ofcpu_online_mask. It means that CPU can be brought online at any time.Quoted 5 lines
But do we have the guarantee that a present CPU _will_ be online at least once right after the boot? After all, kernel parameters such as "maxcpus=" can prevent from turning some CPUs on. I guess there are even more creative ways to achieve that.
Quoted 7 lines
Quoted 2 lines
In any case we really require the housekeeper to be forced online. Perhaps I missed that enforcement somewhere in the patchset?
No I think you're right, that may be able to boot without anything in the housekeeping mask. Maybe we can just
cpu_up()a CPU in the housekeeping mask with a warning that it has overidden their SMP command line option. I'll take a look at it.Quoted 5 lines
But then what if
cpu_up()fails? In this case I can think of only two answers:* Force the boot CPU as the housekeeper.
* Rollback the whole thing: nohz and all isolation.If
cpu_upfails despite being in the present map and we explicitly selected it as the housekeeper? I think it would be okay to print a message telling admin to correct the config, and panic.We try a best effort to make the system boot and limp along, but if you misconfigure it, crashing is not unreasonable. There's lots of command line option misconfiguration that will cause the same thing.
Quoted 6 lines
Quoted 3 lines
The primary problem with my patch that needs to be addressed is that the error is not explicitly caught and printed if the housekeeper does not come up, so the system might die in non-obvious ways.
I usually reserve panic and
BUG_ON()to last resort when data integrity is directly threatened. But indeed I guess that's all we have for now.
Right, specifying a CPU for housekeeping that excluded from coming up at boot with maxcpus= or whatever, is not such a big deal to panic I think. Just need to have a clear error message.
Quoted 2 lines
If we take that path, I'd rather not call that
cpu_up()and simply panic if the given CPU happens not to be online after SMP bootup.
Sure that's fine by me too.
Thanks,
Nick
Nicholas Piggin
reviewerPATCH v2 5/5 nohz_full: Allow the boot CPU to be nohz_full
In reply to Nicholas PigginAllow the boot CPU / CPU0 to be nohz_full. Have the boot CPU take the do_timer duty during boot until a housekeeping CPU can take over.
This is supported when CONFIG_PM_SLEEP_SMP is not configured, or when it is configured and the arch allows suspend on non-zero CPUs.
nohz_full has been trialed at a large supercomputer site and found to significantly reduce jitter. In order to deploy it in production, they need CPU0 to be nohz_full because their job control system requires the application CPUs to start from 0, and the housekeeping CPUs are placed higher. An equivalent job scheduling that uses CPU0 for housekeeping could be achieved by modifying their system, but it is preferable if nohz_full can support their environment without modification.
--- kernel/time/tick-common.c | 50 +++++++++++++++++++++++++++++++++++---- kernel/time/tick-sched.c | 34 ++++++++++++++++++-------- 2 files changed, 70 insertions(+), 14 deletions(-)
kernel/time/tick-common.c +46 -4
@@ -46,6 +46,14 @@ ktime_t tick_period;* procedure also covers cpu hotplug. */ int tick_do_timer_cpu__read_mostly =TICK_DO_TIMER_BOOT ; +#ifdef CONFIG_NO_HZ_FULL +/* +* tick_do_timer_boot_cpu indicates the boot CPU temporarily owns +* tick_do_timer_cpu and it should be taken over by an eligible secondary +* when one comes online. +*/ +static int tick_do_timer_boot_cpu__read_mostly =-1 ; +#endif /* * Debugging: see timer_list.c
@@ -167,6 +175,26 @@ void tick_setup_periodic(struct clock_event_device *dev, int broadcast) } } +#ifdef CONFIG_NO_HZ_FULL +static void giveup_do_timer (void *info) +{ +int cpu = *(unsigned int *)info; + +WARN_ON (tick_do_timer_cpu !=smp_processor_id ()); + + tick_do_timer_cpu = cpu; +} + +static void tick_take_do_timer_from_boot (void ) +{ +int cpu =smp_processor_id (); +int from = tick_do_timer_boot_cpu; + +if (from >=0 && from != cpu) +smp_call_function_single (from, giveup_do_timer, &cpu,1 ); +} +#endif +/* * Setup the tick device */
@@ -186,12 +214,26 @@ static void tick_setup_device(struct tick_device *td,* this cpu: */ if (tick_do_timer_cpu ==TICK_DO_TIMER_BOOT ) { -if (!tick_nohz_full_cpu (cpu)) - tick_do_timer_cpu = cpu; -else - tick_do_timer_cpu =TICK_DO_TIMER_NONE ; + tick_do_timer_cpu = cpu; + tick_next_period =ktime_get (); tick_period =NSEC_PER_SEC /HZ ; +#ifdef CONFIG_NO_HZ_FULL +/* +* The boot CPU may be nohz_full, in which case set +* tick_do_timer_boot_cpu so the first housekeeping +* secondary that comes up will take do_timer from +* us. +*/ +if (tick_nohz_full_cpu (cpu)) + tick_do_timer_boot_cpu = cpu; + + }else if (tick_do_timer_boot_cpu !=-1 && + !tick_nohz_full_cpu (cpu)) { +tick_take_do_timer_from_boot (); + tick_do_timer_boot_cpu =-1 ; +WARN_ON (tick_do_timer_cpu != cpu); +#endif }/*
kernel/time/tick-sched.c +24 -10
@@ -121,10 +121,16 @@ static void tick_sched_do_timer(struct tick_sched *ts, ktime_t now)* into a long sleep. If two CPUs happen to assign themselves to * this duty, then the jiffies update is still serialized by * jiffies_lock. +* +* If nohz_full is enabled, this should not happen because the +* tick_do_timer_cpu never relinquishes. */ -if (unlikely (tick_do_timer_cpu ==TICK_DO_TIMER_NONE ) - && !tick_nohz_full_cpu (cpu)) +if (unlikely (tick_do_timer_cpu ==TICK_DO_TIMER_NONE )) { +#ifdef CONFIG_NO_HZ_FULL +WARN_ON (tick_nohz_full_running); +#endif tick_do_timer_cpu = cpu; + }#endif /* Check, if the jiffies need an update */
@@ -395,8 +401,8 @@ void __init tick_nohz_full_setup(cpumask_var_t cpumask)static int tick_nohz_cpu_down (unsigned int cpu) {/* -* The boot CPU handles housekeeping duty (unbound timers, -* workqueues, timekeeping, ...) on behalf of full dynticks +* The tick_do_timer_cpu CPU handles housekeeping duty (unbound +* timers, workqueues, timekeeping, ...) on behalf of full dynticks * CPUs. It must remain online when nohz full is enabled. */ if (tick_nohz_full_running && tick_do_timer_cpu == cpu)
@@ -423,12 +429,15 @@ void __init tick_nohz_init(void)return ; } - cpu =smp_processor_id (); +if (IS_ENABLED (CONFIG_PM_SLEEP_SMP ) && + !IS_ENABLED (CONFIG_PM_SLEEP_SMP_NONZERO_CPU )) { + cpu =smp_processor_id (); -if (cpumask_test_cpu (cpu, tick_nohz_full_mask)) { -pr_warn ("NO_HZ: Clearing %d from nohz_full range for timekeeping\n" , - cpu); -cpumask_clear_cpu (cpu, tick_nohz_full_mask); +if (cpumask_test_cpu (cpu, tick_nohz_full_mask)) { +pr_warn ("NO_HZ: Clearing %d from nohz_full range " +"for timekeeping\n" , cpu); +cpumask_clear_cpu (cpu, tick_nohz_full_mask); + } }for_each_cpu (cpu, tick_nohz_full_mask)
@@ -904,8 +913,13 @@ static bool can_stop_idle_tick(int cpu, struct tick_sched *ts)/* * Boot safety: make sure the timekeeping duty has been * assigned before entering dyntick-idle mode, +* tick_do_timer_cpu is TICK_DO_TIMER_BOOT */ -if (tick_do_timer_cpu ==TICK_DO_TIMER_NONE ) +if (unlikely (tick_do_timer_cpu ==TICK_DO_TIMER_BOOT )) +return false; + +/* Should not happen for nohz-full */ +if (WARN_ON_ONCE (tick_do_timer_cpu ==TICK_DO_TIMER_NONE ))return false; }
Signature · 1 line
2.20.1
tip-bot for Nicholas Piggin
merge bot[tip:sched/core] nohz_full: Allow the boot CPU to be nohz_full
nohz_full: Allow the boot CPU to be nohz_full
Commit-ID: 65874bd36e6ae3028539e989bfb5c28ad457368e Gitweb: https://git.kernel.org/tip/65874bd36e6ae3028539e989bfb5c28ad457368e Author: Nicholas Piggin <[email protected]> AuthorDate: Thu, 11 Apr 2019 13:34:48 +1000 Committer: Ingo Molnar <[email protected]> CommitDate: Fri, 3 May 2019 12:53:15 +0200
nohz_full: Allow the boot CPU to be nohz_full
Allow the boot CPU/CPU0 to be nohz_full. Have the boot CPU take the do_timer duty during boot until a housekeeping CPU can take over.
This is supported when CONFIG_PM_SLEEP_SMP is not configured, or when it is configured and the arch allows suspend on non-zero CPUs.
nohz_full has been trialed at a large supercomputer site and found to significantly reduce jitter. In order to deploy it in production, they need CPU0 to be nohz_full because their job control system requires the application CPUs to start from 0, and the housekeeping CPUs are placed higher. An equivalent job scheduling that uses CPU0 for housekeeping could be achieved by modifying their system, but it is preferable if nohz_full can support their environment without modification.
--- kernel/time/tick-common.c | 50 +++++++++++++++++++++++++++++++++++++++++++---- kernel/time/tick-sched.c | 34 ++++++++++++++++++++++---------- 2 files changed, 70 insertions(+), 14 deletions(-)
kernel/time/tick-common.c +46 -4
@@ -46,6 +46,14 @@ ktime_t tick_period;* procedure also covers cpu hotplug. */ int tick_do_timer_cpu__read_mostly =TICK_DO_TIMER_BOOT ; +#ifdef CONFIG_NO_HZ_FULL +/* +* tick_do_timer_boot_cpu indicates the boot CPU temporarily owns +* tick_do_timer_cpu and it should be taken over by an eligible secondary +* when one comes online. +*/ +static int tick_do_timer_boot_cpu__read_mostly =-1 ; +#endif /* * Debugging: see timer_list.c
@@ -167,6 +175,26 @@ void tick_setup_periodic(struct clock_event_device *dev, int broadcast) } } +#ifdef CONFIG_NO_HZ_FULL +static void giveup_do_timer (void *info) +{ +int cpu = *(unsigned int *)info; + +WARN_ON (tick_do_timer_cpu !=smp_processor_id ()); + + tick_do_timer_cpu = cpu; +} + +static void tick_take_do_timer_from_boot (void ) +{ +int cpu =smp_processor_id (); +int from = tick_do_timer_boot_cpu; + +if (from >=0 && from != cpu) +smp_call_function_single (from, giveup_do_timer, &cpu,1 ); +} +#endif +/* * Setup the tick device */
@@ -186,12 +214,26 @@ static void tick_setup_device(struct tick_device *td,* this cpu: */ if (tick_do_timer_cpu ==TICK_DO_TIMER_BOOT ) { -if (!tick_nohz_full_cpu (cpu)) - tick_do_timer_cpu = cpu; -else - tick_do_timer_cpu =TICK_DO_TIMER_NONE ; + tick_do_timer_cpu = cpu; + tick_next_period =ktime_get (); tick_period =NSEC_PER_SEC /HZ ; +#ifdef CONFIG_NO_HZ_FULL +/* +* The boot CPU may be nohz_full, in which case set +* tick_do_timer_boot_cpu so the first housekeeping +* secondary that comes up will take do_timer from +* us. +*/ +if (tick_nohz_full_cpu (cpu)) + tick_do_timer_boot_cpu = cpu; + + }else if (tick_do_timer_boot_cpu !=-1 && + !tick_nohz_full_cpu (cpu)) { +tick_take_do_timer_from_boot (); + tick_do_timer_boot_cpu =-1 ; +WARN_ON (tick_do_timer_cpu != cpu); +#endif }/*
kernel/time/tick-sched.c +24 -10
@@ -121,10 +121,16 @@ static void tick_sched_do_timer(struct tick_sched *ts, ktime_t now)* into a long sleep. If two CPUs happen to assign themselves to * this duty, then the jiffies update is still serialized by * jiffies_lock. +* +* If nohz_full is enabled, this should not happen because the +* tick_do_timer_cpu never relinquishes. */ -if (unlikely (tick_do_timer_cpu ==TICK_DO_TIMER_NONE ) - && !tick_nohz_full_cpu (cpu)) +if (unlikely (tick_do_timer_cpu ==TICK_DO_TIMER_NONE )) { +#ifdef CONFIG_NO_HZ_FULL +WARN_ON (tick_nohz_full_running); +#endif tick_do_timer_cpu = cpu; + }#endif /* Check, if the jiffies need an update */
@@ -395,8 +401,8 @@ void __init tick_nohz_full_setup(cpumask_var_t cpumask)static int tick_nohz_cpu_down (unsigned int cpu) {/* -* The boot CPU handles housekeeping duty (unbound timers, -* workqueues, timekeeping, ...) on behalf of full dynticks +* The tick_do_timer_cpu CPU handles housekeeping duty (unbound +* timers, workqueues, timekeeping, ...) on behalf of full dynticks * CPUs. It must remain online when nohz full is enabled. */ if (tick_nohz_full_running && tick_do_timer_cpu == cpu)
@@ -423,12 +429,15 @@ void __init tick_nohz_init(void)return ; } - cpu =smp_processor_id (); +if (IS_ENABLED (CONFIG_PM_SLEEP_SMP ) && + !IS_ENABLED (CONFIG_PM_SLEEP_SMP_NONZERO_CPU )) { + cpu =smp_processor_id (); -if (cpumask_test_cpu (cpu, tick_nohz_full_mask)) { -pr_warn ("NO_HZ: Clearing %d from nohz_full range for timekeeping\n" , - cpu); -cpumask_clear_cpu (cpu, tick_nohz_full_mask); +if (cpumask_test_cpu (cpu, tick_nohz_full_mask)) { +pr_warn ("NO_HZ: Clearing %d from nohz_full range " +"for timekeeping\n" , cpu); +cpumask_clear_cpu (cpu, tick_nohz_full_mask); + } }for_each_cpu (cpu, tick_nohz_full_mask)
@@ -904,8 +913,13 @@ static bool can_stop_idle_tick(int cpu, struct tick_sched *ts)/* * Boot safety: make sure the timekeeping duty has been * assigned before entering dyntick-idle mode, +* tick_do_timer_cpu is TICK_DO_TIMER_BOOT */ -if (tick_do_timer_cpu ==TICK_DO_TIMER_NONE ) +if (unlikely (tick_do_timer_cpu ==TICK_DO_TIMER_BOOT )) +return false; + +/* Should not happen for nohz-full */ +if (WARN_ON_ONCE (tick_do_timer_cpu ==TICK_DO_TIMER_NONE ))return false; }
tip-bot for Nicholas Piggin
merge bot[tip:sched/core] nohz_full: Allow the boot CPU to be nohz_full
In reply to Nicholas Pigginnohz_full: Allow the boot CPU to be nohz_full
Commit-ID: 08ae95f4fd3b38b257f5dc7e6507e071c27ba0d5 Gitweb: https://git.kernel.org/tip/08ae95f4fd3b38b257f5dc7e6507e071c27ba0d5 Author: Nicholas Piggin <[email protected]> AuthorDate: Thu, 11 Apr 2019 13:34:48 +1000 Committer: Ingo Molnar <[email protected]> CommitDate: Fri, 3 May 2019 19:42:58 +0200
nohz_full: Allow the boot CPU to be nohz_full
Allow the boot CPU/CPU0 to be nohz_full. Have the boot CPU take the do_timer duty during boot until a housekeeping CPU can take over.
This is supported when CONFIG_PM_SLEEP_SMP is not configured, or when it is configured and the arch allows suspend on non-zero CPUs.
nohz_full has been trialed at a large supercomputer site and found to significantly reduce jitter. In order to deploy it in production, they need CPU0 to be nohz_full because their job control system requires the application CPUs to start from 0, and the housekeeping CPUs are placed higher. An equivalent job scheduling that uses CPU0 for housekeeping could be achieved by modifying their system, but it is preferable if nohz_full can support their environment without modification.
--- kernel/time/tick-common.c | 50 +++++++++++++++++++++++++++++++++++++++++++---- kernel/time/tick-sched.c | 34 ++++++++++++++++++++++---------- 2 files changed, 70 insertions(+), 14 deletions(-)
kernel/time/tick-common.c +46 -4
@@ -46,6 +46,14 @@ ktime_t tick_period;* procedure also covers cpu hotplug. */ int tick_do_timer_cpu__read_mostly =TICK_DO_TIMER_BOOT ; +#ifdef CONFIG_NO_HZ_FULL +/* +* tick_do_timer_boot_cpu indicates the boot CPU temporarily owns +* tick_do_timer_cpu and it should be taken over by an eligible secondary +* when one comes online. +*/ +static int tick_do_timer_boot_cpu__read_mostly =-1 ; +#endif /* * Debugging: see timer_list.c
@@ -167,6 +175,26 @@ void tick_setup_periodic(struct clock_event_device *dev, int broadcast) } } +#ifdef CONFIG_NO_HZ_FULL +static void giveup_do_timer (void *info) +{ +int cpu = *(unsigned int *)info; + +WARN_ON (tick_do_timer_cpu !=smp_processor_id ()); + + tick_do_timer_cpu = cpu; +} + +static void tick_take_do_timer_from_boot (void ) +{ +int cpu =smp_processor_id (); +int from = tick_do_timer_boot_cpu; + +if (from >=0 && from != cpu) +smp_call_function_single (from, giveup_do_timer, &cpu,1 ); +} +#endif +/* * Setup the tick device */
@@ -186,12 +214,26 @@ static void tick_setup_device(struct tick_device *td,* this cpu: */ if (tick_do_timer_cpu ==TICK_DO_TIMER_BOOT ) { -if (!tick_nohz_full_cpu (cpu)) - tick_do_timer_cpu = cpu; -else - tick_do_timer_cpu =TICK_DO_TIMER_NONE ; + tick_do_timer_cpu = cpu; + tick_next_period =ktime_get (); tick_period =NSEC_PER_SEC /HZ ; +#ifdef CONFIG_NO_HZ_FULL +/* +* The boot CPU may be nohz_full, in which case set +* tick_do_timer_boot_cpu so the first housekeeping +* secondary that comes up will take do_timer from +* us. +*/ +if (tick_nohz_full_cpu (cpu)) + tick_do_timer_boot_cpu = cpu; + + }else if (tick_do_timer_boot_cpu !=-1 && + !tick_nohz_full_cpu (cpu)) { +tick_take_do_timer_from_boot (); + tick_do_timer_boot_cpu =-1 ; +WARN_ON (tick_do_timer_cpu != cpu); +#endif }/*
kernel/time/tick-sched.c +24 -10
@@ -121,10 +121,16 @@ static void tick_sched_do_timer(struct tick_sched *ts, ktime_t now)* into a long sleep. If two CPUs happen to assign themselves to * this duty, then the jiffies update is still serialized by * jiffies_lock. +* +* If nohz_full is enabled, this should not happen because the +* tick_do_timer_cpu never relinquishes. */ -if (unlikely (tick_do_timer_cpu ==TICK_DO_TIMER_NONE ) - && !tick_nohz_full_cpu (cpu)) +if (unlikely (tick_do_timer_cpu ==TICK_DO_TIMER_NONE )) { +#ifdef CONFIG_NO_HZ_FULL +WARN_ON (tick_nohz_full_running); +#endif tick_do_timer_cpu = cpu; + }#endif /* Check, if the jiffies need an update */
@@ -395,8 +401,8 @@ void __init tick_nohz_full_setup(cpumask_var_t cpumask)static int tick_nohz_cpu_down (unsigned int cpu) {/* -* The boot CPU handles housekeeping duty (unbound timers, -* workqueues, timekeeping, ...) on behalf of full dynticks +* The tick_do_timer_cpu CPU handles housekeeping duty (unbound +* timers, workqueues, timekeeping, ...) on behalf of full dynticks * CPUs. It must remain online when nohz full is enabled. */ if (tick_nohz_full_running && tick_do_timer_cpu == cpu)
@@ -423,12 +429,15 @@ void __init tick_nohz_init(void)return ; } - cpu =smp_processor_id (); +if (IS_ENABLED (CONFIG_PM_SLEEP_SMP ) && + !IS_ENABLED (CONFIG_PM_SLEEP_SMP_NONZERO_CPU )) { + cpu =smp_processor_id (); -if (cpumask_test_cpu (cpu, tick_nohz_full_mask)) { -pr_warn ("NO_HZ: Clearing %d from nohz_full range for timekeeping\n" , - cpu); -cpumask_clear_cpu (cpu, tick_nohz_full_mask); +if (cpumask_test_cpu (cpu, tick_nohz_full_mask)) { +pr_warn ("NO_HZ: Clearing %d from nohz_full range " +"for timekeeping\n" , cpu); +cpumask_clear_cpu (cpu, tick_nohz_full_mask); + } }for_each_cpu (cpu, tick_nohz_full_mask)
@@ -904,8 +913,13 @@ static bool can_stop_idle_tick(int cpu, struct tick_sched *ts)/* * Boot safety: make sure the timekeeping duty has been * assigned before entering dyntick-idle mode, +* tick_do_timer_cpu is TICK_DO_TIMER_BOOT */ -if (tick_do_timer_cpu ==TICK_DO_TIMER_NONE ) +if (unlikely (tick_do_timer_cpu ==TICK_DO_TIMER_BOOT )) +return false; + +/* Should not happen for nohz-full */ +if (WARN_ON_ONCE (tick_do_timer_cpu ==TICK_DO_TIMER_NONE ))return false; }
Peter Zijlstra
maintainerQuoted 5 lines · Nicholas Piggin
Since last time, I added a compile time option to opt-out of this if the platform does not support suspend on non-zero, and tried to improve legibility of changelogs and explain the justification better.
Quoted 7 lines
I have been testing this on powerpc/pseries and it seems to work fine (the firmware call to suspend can be called on any CPU and resumes where it left off), but not included here because the code has some bitrot unrelated to this series which I hacked to fix. I will discuss it and either send an acked patch to go with this series if it is small, or fix it in powerpc tree.
Rafael, Frederic, any comments?
Nicholas Piggin
reviewerPeter Zijlstra's on April 25, 2019 10:04 pm:
Quoted 14 lines · Nicholas Piggin
Quoted 5 lines
Since last time, I added a compile time option to opt-out of this if the platform does not support suspend on non-zero, and tried to improve legibility of changelogs and explain the justification better.
Quoted 7 lines
I have been testing this on powerpc/pseries and it seems to work fine (the firmware call to suspend can be called on any CPU and resumes where it left off), but not included here because the code has some bitrot unrelated to this series which I hacked to fix. I will discuss it and either send an acked patch to go with this series if it is small, or fix it in powerpc tree.
Quoted 2 lines
Rafael, Frederic, any comments?
Sorry to ping again, I guess people are probably busy after vacation. Any chance we could get this in next merge window? Peter are you okay with the config option as it is, then we can look at adapting it to what x86 needs as a follow up (e.g., allow nohz CPU0 for cpu0_hotpluggable case)?
Thanks,
Nick
Peter Zijlstra
maintainerQuoted 15 lines · Nicholas Piggin
Peter Zijlstra's on April 25, 2019 10:04 pm:
Quoted 14 lines · Nicholas Piggin
Quoted 12 lines
Since last time, I added a compile time option to opt-out of this if the platform does not support suspend on non-zero, and tried to improve legibility of changelogs and explain the justification better.
I have been testing this on powerpc/pseries and it seems to work fine (the firmware call to suspend can be called on any CPU and resumes where it left off), but not included here because the code has some bitrot unrelated to this series which I hacked to fix. I will discuss it and either send an acked patch to go with this series if it is small, or fix it in powerpc tree.
Quoted 8 lines
Quoted 2 lines
Rafael, Frederic, any comments?
Sorry to ping again, I guess people are probably busy after vacation. Any chance we could get this in next merge window? Peter are you okay with the config option as it is, then we can look at adapting it to what x86 needs as a follow up (e.g., allow nohz CPU0 for
cpu0_hotpluggablecase)?
Yeah, let me just queue these here patches. Not sure they'll still make the upcoming merge window, but we can try.