Skip to contentSkip to conversation

Allow CPU0 to be nohz full

PATCH In review Nicholas Piggin 4 participants 27 messages Apr 11, 2019 – May 8, 2019

Subsystems (8)
lore
Share & tools
lore

Nicholas Piggin

author reviewer
#0 lore raw

v2 cover Allow CPU0 to be nohz full

Since last time, I added a compile time option to opt-out of this if the platform does not support suspend on non-zero, and tried to improve legibility of changelogs and explain the justification better.

I have been testing this on powerpc/pseries and it seems to work fine (the firmware call to suspend can be called on any CPU and resumes where it left off), but not included here because the code has some bitrot unrelated to this series which I hacked to fix. I will discuss it and either send an acked patch to go with this series if it is small, or fix it in powerpc tree.

Thanks,
Nick

Nicholas Piggin (5):
sched/core: allow the remote scheduler tick to be started on CPU0 PM / suspend: add function to disable secondaries for suspend kernel/cpu: Allow non-zero CPU to be primary for suspend / kexec
freeze
kernel/sched/isolation: require a present CPU in housekeeping mask
nohz_full: Allow the boot CPU to be nohz_full

 include/linux/cpu.h       | 15 ++++++++++++
 kernel/cpu.c              | 10 +++++++-
 kernel/kexec_core.c       |  4 ++--
 kernel/power/Kconfig      |  9 +++++++
 kernel/power/hibernate.c  | 12 +++++-----
 kernel/power/suspend.c    |  4 ++--
 kernel/sched/core.c       |  2 +-
 kernel/sched/isolation.c  | 18 ++++++++++----
 kernel/time/tick-common.c | 50 +++++++++++++++++++++++++++++++++++----
 kernel/time/tick-sched.c  | 34 ++++++++++++++++++--------
 11 files changed, 131 insertions(+), 31 deletions(-)
Signature · 1 line
2.20.1
author reviewer
#1 lore raw

PATCH v2 1/5 sched/core: allow the remote scheduler tick to be started on CPU0

This has no effect yet because CPU0 will always be a housekeeping CPU until a later change.

---
 kernel/sched/core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/sched/core.c +1 -1
index 4778c48a7fda..10e05ec049b6 100644
@@ -5918,7 +5918,7 @@ void __init sched_init_smp(void)
 
 static int __init migration_init(void)
 {
-	sched_rq_cpu_starting(smp_processor_id());
+	sched_cpu_starting(smp_processor_id());
 	return 0;
 }
 early_initcall(migration_init);
Signature · 1 line
2.20.1
author reviewer
#3 lore raw

PATCH v2 2/5 PM / suspend: add function to disable secondaries for suspend

In reply to Nicholas Piggin

This adds a function to disable secondary CPUs for suspend that are not necessarily non-zero / non-boot CPUs. Platforms will be able to use this to suspend using non-zero CPUs.

Cc Rafael J. Wysocki <[email protected]>
---
 include/linux/cpu.h      | 10 ++++++++++
 kernel/kexec_core.c      |  4 ++--
 kernel/power/hibernate.c | 12 ++++++------
 kernel/power/suspend.c   |  4 ++--
 4 files changed, 20 insertions(+), 10 deletions(-)
include/linux/cpu.h +10 -0
index 5041357d0297..563e697e7779 100644
@@ -137,6 +137,16 @@ static inline int disable_nonboot_cpus(void)
 	return freeze_secondary_cpus(0);
 }
 extern void enable_nonboot_cpus(void);
+
+static inline int suspend_disable_secondary_cpus(void)
+{
+	return freeze_secondary_cpus(0);
+}
+static inline void suspend_enable_secondary_cpus(void)
+{
+	return enable_nonboot_cpus();
+}
+
 #else /* !CONFIG_PM_SLEEP_SMP */
 static inline int disable_nonboot_cpus(void) { return 0; }
 static inline void enable_nonboot_cpus(void) {}
kernel/kexec_core.c +2 -2
index d7140447be75..fd5c95ff9251 100644
@@ -1150,7 +1150,7 @@ int kernel_kexec(void)
 		error = dpm_suspend_end(PMSG_FREEZE);
 		if (error)
 			goto Resume_devices;
-		error = disable_nonboot_cpus();
+		error = suspend_disable_secondary_cpus();
 		if (error)
 			goto Enable_cpus;
 		local_irq_disable();
@@ -1183,7 +1183,7 @@ int kernel_kexec(void)
  Enable_irqs:
 		local_irq_enable();
  Enable_cpus:
-		enable_nonboot_cpus();
+		suspend_enable_secondary_cpus();
 		dpm_resume_start(PMSG_RESTORE);
  Resume_devices:
 		dpm_resume_end(PMSG_RESTORE);
kernel/power/hibernate.c +6 -6
index abef759de7c8..cfc7a57049e4 100644
@@ -281,7 +281,7 @@ static int create_image(int platform_mode)
 	if (error || hibernation_test(TEST_PLATFORM))
 		goto Platform_finish;
 
-	error = disable_nonboot_cpus();
+	error = suspend_disable_secondary_cpus();
 	if (error || hibernation_test(TEST_CPUS))
 		goto Enable_cpus;
 
@@ -323,7 +323,7 @@ static int create_image(int platform_mode)
 	local_irq_enable();
 
  Enable_cpus:
-	enable_nonboot_cpus();
+	suspend_enable_secondary_cpus();
 
  Platform_finish:
 	platform_finish(platform_mode);
@@ -417,7 +417,7 @@ int hibernation_snapshot(int platform_mode)
 
 int __weak hibernate_resume_nonboot_cpu_disable(void)
 {
-	return disable_nonboot_cpus();
+	return suspend_disable_secondary_cpus();
 }
 
 /**
@@ -486,7 +486,7 @@ static int resume_target_kernel(bool platform_mode)
 	local_irq_enable();
 
  Enable_cpus:
-	enable_nonboot_cpus();
+	suspend_enable_secondary_cpus();
 
  Cleanup:
 	platform_restore_cleanup(platform_mode);
@@ -564,7 +564,7 @@ int hibernation_platform_enter(void)
 	if (error)
 		goto Platform_finish;
 
-	error = disable_nonboot_cpus();
+	error = suspend_disable_secondary_cpus();
 	if (error)
 		goto Enable_cpus;
 
@@ -586,7 +586,7 @@ int hibernation_platform_enter(void)
 	local_irq_enable();
 
  Enable_cpus:
-	enable_nonboot_cpus();
+	suspend_enable_secondary_cpus();
 
  Platform_finish:
 	hibernation_ops->finish();
kernel/power/suspend.c +2 -2
index 0bd595a0b610..59b6def23046 100644
@@ -428,7 +428,7 @@ static int suspend_enter(suspend_state_t state, bool *wakeup)
 	if (suspend_test(TEST_PLATFORM))
 		goto Platform_wake;
 
-	error = disable_nonboot_cpus();
+	error = suspend_disable_secondary_cpus();
 	if (error || suspend_test(TEST_CPUS))
 		goto Enable_cpus;
 
@@ -458,7 +458,7 @@ static int suspend_enter(suspend_state_t state, bool *wakeup)
 	BUG_ON(irqs_disabled());
 
  Enable_cpus:
-	enable_nonboot_cpus();
+	suspend_enable_secondary_cpus();
 
  Platform_wake:
 	platform_resume_noirq(state);
Signature · 1 line
2.20.1
author reviewer
#6 lore raw

PATCH v2 3/5 kernel/cpu: Allow non-zero CPU to be primary for suspend / kexec freeze

In reply to Nicholas Piggin

This patch provides an arch option, ARCH_SUSPEND_NONZERO_CPU, to opt-in to allowing suspend to occur on one of the housekeeping CPUs rather than hardcoded CPU0.

This will allow CPU0 to be a nohz_full CPU with a later change.

It may be possible for platforms with hardware/firmware restrictions on suspend/wake effectively support this by handing off the final stage to CPU0 when kernel housekeeping is no longer required. Another option is to make housekeeping / nohz_full mask dynamic at runtime, but the complexity could not be justified at this time.

---
 arch/powerpc/Kconfig |  4 ++++
 include/linux/cpu.h  |  7 ++++++-
 kernel/cpu.c         | 10 +++++++++-
 kernel/power/Kconfig |  9 +++++++++
 4 files changed, 28 insertions(+), 2 deletions(-)
arch/powerpc/Kconfig +4 -0
index 2d0be82c3061..bc98b0e37a10 100644
@@ -318,6 +318,10 @@ config ARCH_SUSPEND_POSSIBLE
 		   (PPC_85xx && !PPC_E500MC) || PPC_86xx || PPC_PSERIES \
 		   || 44x || 40x
 
+config ARCH_SUSPEND_NONZERO_CPU
+	def_bool y
+	depends on PPC_POWERNV || PPC_PSERIES
+
 config PPC_DCR_NATIVE
 	bool
 
include/linux/cpu.h +6 -1
index 563e697e7779..dd3813959d62 100644
@@ -140,7 +140,12 @@ extern void enable_nonboot_cpus(void);
 
 static inline int suspend_disable_secondary_cpus(void)
 {
-	return freeze_secondary_cpus(0);
+	int cpu = 0;
+
+	if (IS_ENABLED(CONFIG_PM_SLEEP_SMP_NONZERO_CPU))
+		cpu = -1;
+
+	return freeze_secondary_cpus(cpu);
 }
 static inline void suspend_enable_secondary_cpus(void)
 {
kernel/cpu.c +9 -1
index 6754f3ecfd94..d1bf6e2b4752 100644
@@ -9,6 +9,7 @@
 #include <linux/notifier.h>
 #include <linux/sched/signal.h>
 #include <linux/sched/hotplug.h>
+#include <linux/sched/isolation.h>
 #include <linux/sched/task.h>
 #include <linux/sched/smt.h>
 #include <linux/unistd.h>
@@ -1199,8 +1200,15 @@ int freeze_secondary_cpus(int primary)
 	int cpu, error = 0;
 
 	cpu_maps_update_begin();
-	if (!cpu_online(primary))
+	if (primary == -1) {
 		primary = cpumask_first(cpu_online_mask);
+		if (!housekeeping_cpu(primary, HK_FLAG_TIMER))
+			primary = housekeeping_any_cpu(HK_FLAG_TIMER);
+	} else {
+		if (!cpu_online(primary))
+			primary = cpumask_first(cpu_online_mask);
+	}
+
 	/*
 	 * We take down all of the non-boot CPUs in one shot to avoid races
 	 * with the userspace trying to use the CPU hotplug at the same time
kernel/power/Kconfig +9 -0
index f8fe57d1022e..9bbaaab14b36 100644
@@ -114,6 +114,15 @@ config PM_SLEEP_SMP
 	depends on PM_SLEEP
 	select HOTPLUG_CPU
 
+config PM_SLEEP_SMP_NONZERO_CPU
+	def_bool y
+	depends on PM_SLEEP_SMP
+	depends on ARCH_SUSPEND_NONZERO_CPU
+	---help---
+	If an arch can suspend (for suspend, hibernate, kexec, etc) on a
+	non-zero numbered CPU, it may define ARCH_SUSPEND_NONZERO_CPU. This
+	will allow nohz_full mask to include CPU0.
+
 config PM_AUTOSLEEP
 	bool "Opportunistic sleep"
 	depends on PM_SLEEP
Signature · 1 line
2.20.1
author reviewer
#11 lore raw

PATCH v2 4/5 kernel/sched/isolation: require a present CPU in housekeeping mask

In reply to Nicholas Piggin

During housekeeping mask setup, currently a possible CPU is required. That does not guarantee the CPU would be available at boot time, so check to ensure that at least one present CPU is in the mask.

---
 kernel/sched/isolation.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)
kernel/sched/isolation.c +13 -5
index b02d148e7672..687302051a27 100644
@@ -65,6 +65,7 @@ void __init housekeeping_init(void)
 static int __init housekeeping_setup(char *str, enum hk_flags flags)
 {
 	cpumask_var_t non_housekeeping_mask;
+	cpumask_var_t tmp;
 	int err;
 
 	alloc_bootmem_cpumask_var(&non_housekeeping_mask);
@@ -75,16 +76,23 @@ static int __init housekeeping_setup(char *str, enum hk_flags flags)
 		return 0;
 	}
 
+	alloc_bootmem_cpumask_var(&tmp);
 	if (!housekeeping_flags) {
 		alloc_bootmem_cpumask_var(&housekeeping_mask);
 		cpumask_andnot(housekeeping_mask,
 			       cpu_possible_mask, non_housekeeping_mask);
-		if (cpumask_empty(housekeeping_mask))
+
+		cpumask_andnot(tmp, cpu_present_mask, non_housekeeping_mask);
+		if (cpumask_empty(tmp)) {
+			pr_warn("Housekeeping: must include one present CPU, "
+				"using boot CPU:%d\n", smp_processor_id());
 			__cpumask_set_cpu(smp_processor_id(), housekeeping_mask);
+			__cpumask_clear_cpu(smp_processor_id(), non_housekeeping_mask);
+		}
 	} else {
-		cpumask_var_t tmp;
-
-		alloc_bootmem_cpumask_var(&tmp);
+		cpumask_andnot(tmp, cpu_present_mask, non_housekeeping_mask);
+		if (cpumask_empty(tmp))
+			__cpumask_clear_cpu(smp_processor_id(), non_housekeeping_mask);
 		cpumask_andnot(tmp, cpu_possible_mask, non_housekeeping_mask);
 		if (!cpumask_equal(tmp, housekeeping_mask)) {
 			pr_warn("Housekeeping: nohz_full= must match isolcpus=\n");
@@ -92,8 +100,8 @@ static int __init housekeeping_setup(char *str, enum hk_flags flags)
 			free_bootmem_cpumask_var(non_housekeeping_mask);
 			return 0;
 		}
-		free_bootmem_cpumask_var(tmp);
 	}
+	free_bootmem_cpumask_var(tmp);
 
 	if ((flags & HK_FLAG_TICK) && !(housekeeping_flags & HK_FLAG_TICK)) {
 		if (IS_ENABLED(CONFIG_NO_HZ_FULL)) {
Signature · 1 line
2.20.1
author reviewer
#21 lore raw

PATCH v2 5/5 nohz_full: Allow the boot CPU to be nohz_full

In reply to Nicholas Piggin

Allow the boot CPU / CPU0 to be nohz_full. Have the boot CPU take the do_timer duty during boot until a housekeeping CPU can take over.

This is supported when CONFIG_PM_SLEEP_SMP is not configured, or when it is configured and the arch allows suspend on non-zero CPUs.

nohz_full has been trialed at a large supercomputer site and found to significantly reduce jitter. In order to deploy it in production, they need CPU0 to be nohz_full because their job control system requires the application CPUs to start from 0, and the housekeeping CPUs are placed higher. An equivalent job scheduling that uses CPU0 for housekeeping could be achieved by modifying their system, but it is preferable if nohz_full can support their environment without modification.

---
 kernel/time/tick-common.c | 50 +++++++++++++++++++++++++++++++++++----
 kernel/time/tick-sched.c  | 34 ++++++++++++++++++--------
 2 files changed, 70 insertions(+), 14 deletions(-)
kernel/time/tick-common.c +46 -4
index 529143b4c8d2..31146c13226e 100644
@@ -46,6 +46,14 @@ ktime_t tick_period;
  *    procedure also covers cpu hotplug.
  */
 int tick_do_timer_cpu __read_mostly = TICK_DO_TIMER_BOOT;
+#ifdef CONFIG_NO_HZ_FULL
+/*
+ * tick_do_timer_boot_cpu indicates the boot CPU temporarily owns
+ * tick_do_timer_cpu and it should be taken over by an eligible secondary
+ * when one comes online.
+ */
+static int tick_do_timer_boot_cpu __read_mostly = -1;
+#endif
 
 /*
  * Debugging: see timer_list.c
@@ -167,6 +175,26 @@ void tick_setup_periodic(struct clock_event_device *dev, int broadcast)
 	}
 }
 
+#ifdef CONFIG_NO_HZ_FULL
+static void giveup_do_timer(void *info)
+{
+	int cpu = *(unsigned int *)info;
+
+	WARN_ON(tick_do_timer_cpu != smp_processor_id());
+
+	tick_do_timer_cpu = cpu;
+}
+
+static void tick_take_do_timer_from_boot(void)
+{
+	int cpu = smp_processor_id();
+	int from = tick_do_timer_boot_cpu;
+
+	if (from >= 0 && from != cpu)
+		smp_call_function_single(from, giveup_do_timer, &cpu, 1);
+}
+#endif
+
 /*
  * Setup the tick device
  */
@@ -186,12 +214,26 @@ static void tick_setup_device(struct tick_device *td,
 		 * this cpu:
 		 */
 		if (tick_do_timer_cpu == TICK_DO_TIMER_BOOT) {
-			if (!tick_nohz_full_cpu(cpu))
-				tick_do_timer_cpu = cpu;
-			else
-				tick_do_timer_cpu = TICK_DO_TIMER_NONE;
+			tick_do_timer_cpu = cpu;
+
 			tick_next_period = ktime_get();
 			tick_period = NSEC_PER_SEC / HZ;
+#ifdef CONFIG_NO_HZ_FULL
+			/*
+			 * The boot CPU may be nohz_full, in which case set
+			 * tick_do_timer_boot_cpu so the first housekeeping
+			 * secondary that comes up will take do_timer from
+			 * us.
+			 */
+			if (tick_nohz_full_cpu(cpu))
+				tick_do_timer_boot_cpu = cpu;
+
+		} else if (tick_do_timer_boot_cpu != -1 &&
+						!tick_nohz_full_cpu(cpu)) {
+			tick_take_do_timer_from_boot();
+			tick_do_timer_boot_cpu = -1;
+			WARN_ON(tick_do_timer_cpu != cpu);
+#endif
 		}
 
 		/*
kernel/time/tick-sched.c +24 -10
index 6fa52cd6df0b..4aa917acbe1c 100644
@@ -121,10 +121,16 @@ static void tick_sched_do_timer(struct tick_sched *ts, ktime_t now)
 	 * into a long sleep. If two CPUs happen to assign themselves to
 	 * this duty, then the jiffies update is still serialized by
 	 * jiffies_lock.
+	 *
+	 * If nohz_full is enabled, this should not happen because the
+	 * tick_do_timer_cpu never relinquishes.
 	 */
-	if (unlikely(tick_do_timer_cpu == TICK_DO_TIMER_NONE)
-	    && !tick_nohz_full_cpu(cpu))
+	if (unlikely(tick_do_timer_cpu == TICK_DO_TIMER_NONE)) {
+#ifdef CONFIG_NO_HZ_FULL
+		WARN_ON(tick_nohz_full_running);
+#endif
 		tick_do_timer_cpu = cpu;
+	}
 #endif
 
 	/* Check, if the jiffies need an update */
@@ -395,8 +401,8 @@ void __init tick_nohz_full_setup(cpumask_var_t cpumask)
 static int tick_nohz_cpu_down(unsigned int cpu)
 {
 	/*
-	 * The boot CPU handles housekeeping duty (unbound timers,
-	 * workqueues, timekeeping, ...) on behalf of full dynticks
+	 * The tick_do_timer_cpu CPU handles housekeeping duty (unbound
+	 * timers, workqueues, timekeeping, ...) on behalf of full dynticks
 	 * CPUs. It must remain online when nohz full is enabled.
 	 */
 	if (tick_nohz_full_running && tick_do_timer_cpu == cpu)
@@ -423,12 +429,15 @@ void __init tick_nohz_init(void)
 		return;
 	}
 
-	cpu = smp_processor_id();
+	if (IS_ENABLED(CONFIG_PM_SLEEP_SMP) &&
+			!IS_ENABLED(CONFIG_PM_SLEEP_SMP_NONZERO_CPU)) {
+		cpu = smp_processor_id();
 
-	if (cpumask_test_cpu(cpu, tick_nohz_full_mask)) {
-		pr_warn("NO_HZ: Clearing %d from nohz_full range for timekeeping\n",
-			cpu);
-		cpumask_clear_cpu(cpu, tick_nohz_full_mask);
+		if (cpumask_test_cpu(cpu, tick_nohz_full_mask)) {
+			pr_warn("NO_HZ: Clearing %d from nohz_full range "
+				"for timekeeping\n", cpu);
+			cpumask_clear_cpu(cpu, tick_nohz_full_mask);
+		}
 	}
 
 	for_each_cpu(cpu, tick_nohz_full_mask)
@@ -904,8 +913,13 @@ static bool can_stop_idle_tick(int cpu, struct tick_sched *ts)
 		/*
 		 * Boot safety: make sure the timekeeping duty has been
 		 * assigned before entering dyntick-idle mode,
+		 * tick_do_timer_cpu is TICK_DO_TIMER_BOOT
 		 */
-		if (tick_do_timer_cpu == TICK_DO_TIMER_NONE)
+		if (unlikely(tick_do_timer_cpu == TICK_DO_TIMER_BOOT))
+			return false;
+
+		/* Should not happen for nohz-full */
+		if (WARN_ON_ONCE(tick_do_timer_cpu == TICK_DO_TIMER_NONE))
 			return false;
 	}
 
Signature · 1 line
2.20.1
maintainer
#7 lore raw

Re: [PATCH v2 3/5] kernel/cpu: Allow non-zero CPU to be primary for suspend / kexec freeze

In reply to Nicholas Piggin
Quoted 11 lines · Nicholas Piggin

This patch provides an arch option, ARCH_SUSPEND_NONZERO_CPU, to opt-in to allowing suspend to occur on one of the housekeeping CPUs rather than hardcoded CPU0.

This will allow CPU0 to be a nohz_full CPU with a later change.

It may be possible for platforms with hardware/firmware restrictions on suspend/wake effectively support this by handing off the final stage to CPU0 when kernel housekeeping is no longer required. Another option is to make housekeeping / nohz_full mask dynamic at runtime, but the complexity could not be justified at this time.

Should we not tie this into whatever already allows an achitecture to hotplug CPU-0? For instance, x86 default disallows this but has cpu0_hotpluggable to allow this.

Presumably POWER already allows hotplugging CPU-0 ?

maintainer
#24 lore raw
In reply to Nicholas Piggin
Quoted 5 lines · Nicholas Piggin

Since last time, I added a compile time option to opt-out of this if the platform does not support suspend on non-zero, and tried to improve legibility of changelogs and explain the justification better.

Quoted 7 lines

I have been testing this on powerpc/pseries and it seems to work fine (the firmware call to suspend can be called on any CPU and resumes where it left off), but not included here because the code has some bitrot unrelated to this series which I hacked to fix. I will discuss it and either send an acked patch to go with this series if it is small, or fix it in powerpc tree.

Rafael, Frederic, any comments?

author reviewer
#8 lore raw

Re: [PATCH v2 3/5] kernel/cpu: Allow non-zero CPU to be primary for suspend / kexec freeze

In reply to Peter Zijlstra

Peter Zijlstra's on April 25, 2019 10:02 pm:

Quoted 13 lines · Nicholas Piggin
Quoted 11 lines

This patch provides an arch option, ARCH_SUSPEND_NONZERO_CPU, to opt-in to allowing suspend to occur on one of the housekeeping CPUs rather than hardcoded CPU0.

This will allow CPU0 to be a nohz_full CPU with a later change.

It may be possible for platforms with hardware/firmware restrictions on suspend/wake effectively support this by handing off the final stage to CPU0 when kernel housekeeping is no longer required. Another option is to make housekeeping / nohz_full mask dynamic at runtime, but the complexity could not be justified at this time.

Quoted 3 lines

Should we not tie this into whatever already allows an achitecture to hotplug CPU-0? For instance, x86 default disallows this but has cpu0_hotpluggable to allow this.

I didn't know about that option, but I see it still has the suspend / hibernate restriction though, which is what this patch is breaking from.

If we are to prevent suspend completely at boot time, then it's no problem to run cpu0 with nohz_full, but TPTB decided that's a bad thing.

But I have no problem with an arch adding another boot time option or hook into cpu0_hotpluggable that allows you to force nohz_full.

Quoted 1 line

Presumably POWER already allows hotplugging CPU-0 ?

Yeah it does. Suspend in pseries is actually really just used for some proprietary hypervisor partition migration scheme where you suspend, image gets saved, then resume it somewhere else. So that's easy, it's just a hypercall that appears to return exactly as it was called. No restriction on CPU number.

Thanks,
Nick

author reviewer
#25 lore raw
In reply to Peter Zijlstra

Peter Zijlstra's on April 25, 2019 10:04 pm:

Quoted 14 lines · Nicholas Piggin
Quoted 5 lines

Since last time, I added a compile time option to opt-out of this if the platform does not support suspend on non-zero, and tried to improve legibility of changelogs and explain the justification better.

Quoted 7 lines

I have been testing this on powerpc/pseries and it seems to work fine (the firmware call to suspend can be called on any CPU and resumes where it left off), but not included here because the code has some bitrot unrelated to this series which I hacked to fix. I will discuss it and either send an acked patch to go with this series if it is small, or fix it in powerpc tree.

Quoted 2 lines

Rafael, Frederic, any comments?

Sorry to ping again, I guess people are probably busy after vacation. Any chance we could get this in next merge window? Peter are you okay with the config option as it is, then we can look at adapting it to what x86 needs as a follow up (e.g., allow nohz CPU0 for cpu0_hotpluggable case)?

Thanks,
Nick

maintainer
#26 lore raw
Quoted 15 lines · Nicholas Piggin

Peter Zijlstra's on April 25, 2019 10:04 pm:

Quoted 14 lines · Nicholas Piggin
Quoted 12 lines

Since last time, I added a compile time option to opt-out of this if the platform does not support suspend on non-zero, and tried to improve legibility of changelogs and explain the justification better.

I have been testing this on powerpc/pseries and it seems to work fine (the firmware call to suspend can be called on any CPU and resumes where it left off), but not included here because the code has some bitrot unrelated to this series which I hacked to fix. I will discuss it and either send an acked patch to go with this series if it is small, or fix it in powerpc tree.

Quoted 8 lines
Quoted 2 lines

Rafael, Frederic, any comments?

Sorry to ping again, I guess people are probably busy after vacation. Any chance we could get this in next merge window? Peter are you okay with the config option as it is, then we can look at adapting it to what x86 needs as a follow up (e.g., allow nohz CPU0 for cpu0_hotpluggable case)?

Yeah, let me just queue these here patches. Not sure they'll still make the upcoming merge window, but we can try.

#2 lore raw

[tip:sched/core] sched/core: Allow the remote scheduler tick to be started on CPU0

In reply to Nicholas Piggin
sched/core: Allow the remote scheduler tick to be started on CPU0
Commit-ID:  77a5352ba977d2554643e3797e10823d0d03dcf7
Gitweb:     https://git.kernel.org/tip/77a5352ba977d2554643e3797e10823d0d03dcf7
Author:     Nicholas Piggin <[email protected]>
AuthorDate: Thu, 11 Apr 2019 13:34:44 +1000
Committer:  Ingo Molnar <[email protected]>
CommitDate: Fri, 3 May 2019 12:53:14 +0200

sched/core: Allow the remote scheduler tick to be started on CPU0

This has no effect yet because CPU0 will always be a housekeeping CPU until a later change.

---
 kernel/sched/core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/sched/core.c +1 -1
index de8ab411826c..cef22c5499a8 100644
@@ -5866,7 +5866,7 @@ void __init sched_init_smp(void)
 
 static int __init migration_init(void)
 {
-	sched_rq_cpu_starting(smp_processor_id());
+	sched_cpu_starting(smp_processor_id());
 	return 0;
 }
 early_initcall(migration_init);
#4 lore raw

[tip:sched/core] power/suspend: Add function to disable secondaries for suspend

In reply to Nicholas Piggin
power/suspend: Add function to disable secondaries for suspend
Commit-ID:  c2cb30bfceceba8a2a0d5713230a250dd6140e22
Gitweb:     https://git.kernel.org/tip/c2cb30bfceceba8a2a0d5713230a250dd6140e22
Author:     Nicholas Piggin <[email protected]>
AuthorDate: Thu, 11 Apr 2019 13:34:45 +1000
Committer:  Ingo Molnar <[email protected]>
CommitDate: Fri, 3 May 2019 12:53:14 +0200

power/suspend: Add function to disable secondaries for suspend

This adds a function to disable secondary CPUs for suspend that are not necessarily non-zero / non-boot CPUs. Platforms will be able to use this to suspend using non-zero CPUs.

---
 include/linux/cpu.h      | 10 ++++++++++
 kernel/kexec_core.c      |  4 ++--
 kernel/power/hibernate.c | 12 ++++++------
 kernel/power/suspend.c   |  4 ++--
 4 files changed, 20 insertions(+), 10 deletions(-)
include/linux/cpu.h +10 -0
index 5041357d0297..563e697e7779 100644
@@ -137,6 +137,16 @@ static inline int disable_nonboot_cpus(void)
 	return freeze_secondary_cpus(0);
 }
 extern void enable_nonboot_cpus(void);
+
+static inline int suspend_disable_secondary_cpus(void)
+{
+	return freeze_secondary_cpus(0);
+}
+static inline void suspend_enable_secondary_cpus(void)
+{
+	return enable_nonboot_cpus();
+}
+
 #else /* !CONFIG_PM_SLEEP_SMP */
 static inline int disable_nonboot_cpus(void) { return 0; }
 static inline void enable_nonboot_cpus(void) {}
kernel/kexec_core.c +2 -2
index d7140447be75..fd5c95ff9251 100644
@@ -1150,7 +1150,7 @@ int kernel_kexec(void)
 		error = dpm_suspend_end(PMSG_FREEZE);
 		if (error)
 			goto Resume_devices;
-		error = disable_nonboot_cpus();
+		error = suspend_disable_secondary_cpus();
 		if (error)
 			goto Enable_cpus;
 		local_irq_disable();
@@ -1183,7 +1183,7 @@ int kernel_kexec(void)
  Enable_irqs:
 		local_irq_enable();
  Enable_cpus:
-		enable_nonboot_cpus();
+		suspend_enable_secondary_cpus();
 		dpm_resume_start(PMSG_RESTORE);
  Resume_devices:
 		dpm_resume_end(PMSG_RESTORE);
kernel/power/hibernate.c +6 -6
index abef759de7c8..cfc7a57049e4 100644
@@ -281,7 +281,7 @@ static int create_image(int platform_mode)
 	if (error || hibernation_test(TEST_PLATFORM))
 		goto Platform_finish;
 
-	error = disable_nonboot_cpus();
+	error = suspend_disable_secondary_cpus();
 	if (error || hibernation_test(TEST_CPUS))
 		goto Enable_cpus;
 
@@ -323,7 +323,7 @@ static int create_image(int platform_mode)
 	local_irq_enable();
 
  Enable_cpus:
-	enable_nonboot_cpus();
+	suspend_enable_secondary_cpus();
 
  Platform_finish:
 	platform_finish(platform_mode);
@@ -417,7 +417,7 @@ int hibernation_snapshot(int platform_mode)
 
 int __weak hibernate_resume_nonboot_cpu_disable(void)
 {
-	return disable_nonboot_cpus();
+	return suspend_disable_secondary_cpus();
 }
 
 /**
@@ -486,7 +486,7 @@ static int resume_target_kernel(bool platform_mode)
 	local_irq_enable();
 
  Enable_cpus:
-	enable_nonboot_cpus();
+	suspend_enable_secondary_cpus();
 
  Cleanup:
 	platform_restore_cleanup(platform_mode);
@@ -564,7 +564,7 @@ int hibernation_platform_enter(void)
 	if (error)
 		goto Platform_finish;
 
-	error = disable_nonboot_cpus();
+	error = suspend_disable_secondary_cpus();
 	if (error)
 		goto Enable_cpus;
 
@@ -586,7 +586,7 @@ int hibernation_platform_enter(void)
 	local_irq_enable();
 
  Enable_cpus:
-	enable_nonboot_cpus();
+	suspend_enable_secondary_cpus();
 
  Platform_finish:
 	hibernation_ops->finish();
kernel/power/suspend.c +2 -2
index 0bd595a0b610..59b6def23046 100644
@@ -428,7 +428,7 @@ static int suspend_enter(suspend_state_t state, bool *wakeup)
 	if (suspend_test(TEST_PLATFORM))
 		goto Platform_wake;
 
-	error = disable_nonboot_cpus();
+	error = suspend_disable_secondary_cpus();
 	if (error || suspend_test(TEST_CPUS))
 		goto Enable_cpus;
 
@@ -458,7 +458,7 @@ static int suspend_enter(suspend_state_t state, bool *wakeup)
 	BUG_ON(irqs_disabled());
 
  Enable_cpus:
-	enable_nonboot_cpus();
+	suspend_enable_secondary_cpus();
 
  Platform_wake:
 	platform_resume_noirq(state);
#9 lore raw

[tip:sched/core] kernel/cpu: Allow non-zero CPU to be primary for suspend / kexec freeze

In reply to Nicholas Piggin
kernel/cpu: Allow non-zero CPU to be primary for suspend / kexec freeze
Commit-ID:  9bcd2cd91816020b0600a2078bab51641b1341df
Gitweb:     https://git.kernel.org/tip/9bcd2cd91816020b0600a2078bab51641b1341df
Author:     Nicholas Piggin <[email protected]>
AuthorDate: Thu, 11 Apr 2019 13:34:46 +1000
Committer:  Ingo Molnar <[email protected]>
CommitDate: Fri, 3 May 2019 12:53:14 +0200

kernel/cpu: Allow non-zero CPU to be primary for suspend / kexec freeze

This patch provides an arch option, ARCH_SUSPEND_NONZERO_CPU, to opt-in to allowing suspend to occur on one of the housekeeping CPUs rather than hardcoded CPU0.

This will allow CPU0 to be a nohz_full CPU with a later change.

It may be possible for platforms with hardware/firmware restrictions on suspend/wake effectively support this by handing off the final stage to CPU0 when kernel housekeeping is no longer required. Another option is to make housekeeping / nohz_full mask dynamic at runtime, but the complexity could not be justified at this time.

---
 arch/powerpc/Kconfig |  4 ++++
 include/linux/cpu.h  |  7 ++++++-
 kernel/cpu.c         | 10 +++++++++-
 kernel/power/Kconfig |  9 +++++++++
 4 files changed, 28 insertions(+), 2 deletions(-)
arch/powerpc/Kconfig +4 -0
index 2d0be82c3061..bc98b0e37a10 100644
@@ -318,6 +318,10 @@ config ARCH_SUSPEND_POSSIBLE
 		   (PPC_85xx && !PPC_E500MC) || PPC_86xx || PPC_PSERIES \
 		   || 44x || 40x
 
+config ARCH_SUSPEND_NONZERO_CPU
+	def_bool y
+	depends on PPC_POWERNV || PPC_PSERIES
+
 config PPC_DCR_NATIVE
 	bool
 
include/linux/cpu.h +6 -1
index 563e697e7779..dd3813959d62 100644
@@ -140,7 +140,12 @@ extern void enable_nonboot_cpus(void);
 
 static inline int suspend_disable_secondary_cpus(void)
 {
-	return freeze_secondary_cpus(0);
+	int cpu = 0;
+
+	if (IS_ENABLED(CONFIG_PM_SLEEP_SMP_NONZERO_CPU))
+		cpu = -1;
+
+	return freeze_secondary_cpus(cpu);
 }
 static inline void suspend_enable_secondary_cpus(void)
 {
kernel/cpu.c +9 -1
index 6754f3ecfd94..d1bf6e2b4752 100644
@@ -9,6 +9,7 @@
 #include <linux/notifier.h>
 #include <linux/sched/signal.h>
 #include <linux/sched/hotplug.h>
+#include <linux/sched/isolation.h>
 #include <linux/sched/task.h>
 #include <linux/sched/smt.h>
 #include <linux/unistd.h>
@@ -1199,8 +1200,15 @@ int freeze_secondary_cpus(int primary)
 	int cpu, error = 0;
 
 	cpu_maps_update_begin();
-	if (!cpu_online(primary))
+	if (primary == -1) {
 		primary = cpumask_first(cpu_online_mask);
+		if (!housekeeping_cpu(primary, HK_FLAG_TIMER))
+			primary = housekeeping_any_cpu(HK_FLAG_TIMER);
+	} else {
+		if (!cpu_online(primary))
+			primary = cpumask_first(cpu_online_mask);
+	}
+
 	/*
 	 * We take down all of the non-boot CPUs in one shot to avoid races
 	 * with the userspace trying to use the CPU hotplug at the same time
kernel/power/Kconfig +9 -0
index f8fe57d1022e..9bbaaab14b36 100644
@@ -114,6 +114,15 @@ config PM_SLEEP_SMP
 	depends on PM_SLEEP
 	select HOTPLUG_CPU
 
+config PM_SLEEP_SMP_NONZERO_CPU
+	def_bool y
+	depends on PM_SLEEP_SMP
+	depends on ARCH_SUSPEND_NONZERO_CPU
+	---help---
+	If an arch can suspend (for suspend, hibernate, kexec, etc) on a
+	non-zero numbered CPU, it may define ARCH_SUSPEND_NONZERO_CPU. This
+	will allow nohz_full mask to include CPU0.
+
 config PM_AUTOSLEEP
 	bool "Opportunistic sleep"
 	depends on PM_SLEEP
#12 lore raw

[tip:sched/core] sched/isolation: Require a present CPU in housekeeping mask

In reply to Nicholas Piggin
sched/isolation: Require a present CPU in housekeeping mask
Commit-ID:  e9a140c6d20ee29951a193342ebaccf07ebc63eb
Gitweb:     https://git.kernel.org/tip/e9a140c6d20ee29951a193342ebaccf07ebc63eb
Author:     Nicholas Piggin <[email protected]>
AuthorDate: Thu, 11 Apr 2019 13:34:47 +1000
Committer:  Ingo Molnar <[email protected]>
CommitDate: Fri, 3 May 2019 12:53:14 +0200

sched/isolation: Require a present CPU in housekeeping mask

During housekeeping mask setup, currently a possible CPU is required. That does not guarantee the CPU would be available at boot time, so check to ensure that at least one present CPU is in the mask.

---
 kernel/sched/isolation.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)
kernel/sched/isolation.c +13 -5
index b02d148e7672..687302051a27 100644
@@ -65,6 +65,7 @@ void __init housekeeping_init(void)
 static int __init housekeeping_setup(char *str, enum hk_flags flags)
 {
 	cpumask_var_t non_housekeeping_mask;
+	cpumask_var_t tmp;
 	int err;
 
 	alloc_bootmem_cpumask_var(&non_housekeeping_mask);
@@ -75,16 +76,23 @@ static int __init housekeeping_setup(char *str, enum hk_flags flags)
 		return 0;
 	}
 
+	alloc_bootmem_cpumask_var(&tmp);
 	if (!housekeeping_flags) {
 		alloc_bootmem_cpumask_var(&housekeeping_mask);
 		cpumask_andnot(housekeeping_mask,
 			       cpu_possible_mask, non_housekeeping_mask);
-		if (cpumask_empty(housekeeping_mask))
+
+		cpumask_andnot(tmp, cpu_present_mask, non_housekeeping_mask);
+		if (cpumask_empty(tmp)) {
+			pr_warn("Housekeeping: must include one present CPU, "
+				"using boot CPU:%d\n", smp_processor_id());
 			__cpumask_set_cpu(smp_processor_id(), housekeeping_mask);
+			__cpumask_clear_cpu(smp_processor_id(), non_housekeeping_mask);
+		}
 	} else {
-		cpumask_var_t tmp;
-
-		alloc_bootmem_cpumask_var(&tmp);
+		cpumask_andnot(tmp, cpu_present_mask, non_housekeeping_mask);
+		if (cpumask_empty(tmp))
+			__cpumask_clear_cpu(smp_processor_id(), non_housekeeping_mask);
 		cpumask_andnot(tmp, cpu_possible_mask, non_housekeeping_mask);
 		if (!cpumask_equal(tmp, housekeeping_mask)) {
 			pr_warn("Housekeeping: nohz_full= must match isolcpus=\n");
@@ -92,8 +100,8 @@ static int __init housekeeping_setup(char *str, enum hk_flags flags)
 			free_bootmem_cpumask_var(non_housekeeping_mask);
 			return 0;
 		}
-		free_bootmem_cpumask_var(tmp);
 	}
+	free_bootmem_cpumask_var(tmp);
 
 	if ((flags & HK_FLAG_TICK) && !(housekeeping_flags & HK_FLAG_TICK)) {
 		if (IS_ENABLED(CONFIG_NO_HZ_FULL)) {
#22 lore raw

[tip:sched/core] nohz_full: Allow the boot CPU to be nohz_full

In reply to Nicholas Piggin
nohz_full: Allow the boot CPU to be nohz_full
Commit-ID:  65874bd36e6ae3028539e989bfb5c28ad457368e
Gitweb:     https://git.kernel.org/tip/65874bd36e6ae3028539e989bfb5c28ad457368e
Author:     Nicholas Piggin <[email protected]>
AuthorDate: Thu, 11 Apr 2019 13:34:48 +1000
Committer:  Ingo Molnar <[email protected]>
CommitDate: Fri, 3 May 2019 12:53:15 +0200

nohz_full: Allow the boot CPU to be nohz_full

Allow the boot CPU/CPU0 to be nohz_full. Have the boot CPU take the do_timer duty during boot until a housekeeping CPU can take over.

This is supported when CONFIG_PM_SLEEP_SMP is not configured, or when it is configured and the arch allows suspend on non-zero CPUs.

nohz_full has been trialed at a large supercomputer site and found to significantly reduce jitter. In order to deploy it in production, they need CPU0 to be nohz_full because their job control system requires the application CPUs to start from 0, and the housekeeping CPUs are placed higher. An equivalent job scheduling that uses CPU0 for housekeeping could be achieved by modifying their system, but it is preferable if nohz_full can support their environment without modification.

---
 kernel/time/tick-common.c | 50 +++++++++++++++++++++++++++++++++++++++++++----
 kernel/time/tick-sched.c  | 34 ++++++++++++++++++++++----------
 2 files changed, 70 insertions(+), 14 deletions(-)
kernel/time/tick-common.c +46 -4
index df401463a191..e49e8091f9ac 100644
@@ -46,6 +46,14 @@ ktime_t tick_period;
  *    procedure also covers cpu hotplug.
  */
 int tick_do_timer_cpu __read_mostly = TICK_DO_TIMER_BOOT;
+#ifdef CONFIG_NO_HZ_FULL
+/*
+ * tick_do_timer_boot_cpu indicates the boot CPU temporarily owns
+ * tick_do_timer_cpu and it should be taken over by an eligible secondary
+ * when one comes online.
+ */
+static int tick_do_timer_boot_cpu __read_mostly = -1;
+#endif
 
 /*
  * Debugging: see timer_list.c
@@ -167,6 +175,26 @@ void tick_setup_periodic(struct clock_event_device *dev, int broadcast)
 	}
 }
 
+#ifdef CONFIG_NO_HZ_FULL
+static void giveup_do_timer(void *info)
+{
+	int cpu = *(unsigned int *)info;
+
+	WARN_ON(tick_do_timer_cpu != smp_processor_id());
+
+	tick_do_timer_cpu = cpu;
+}
+
+static void tick_take_do_timer_from_boot(void)
+{
+	int cpu = smp_processor_id();
+	int from = tick_do_timer_boot_cpu;
+
+	if (from >= 0 && from != cpu)
+		smp_call_function_single(from, giveup_do_timer, &cpu, 1);
+}
+#endif
+
 /*
  * Setup the tick device
  */
@@ -186,12 +214,26 @@ static void tick_setup_device(struct tick_device *td,
 		 * this cpu:
 		 */
 		if (tick_do_timer_cpu == TICK_DO_TIMER_BOOT) {
-			if (!tick_nohz_full_cpu(cpu))
-				tick_do_timer_cpu = cpu;
-			else
-				tick_do_timer_cpu = TICK_DO_TIMER_NONE;
+			tick_do_timer_cpu = cpu;
+
 			tick_next_period = ktime_get();
 			tick_period = NSEC_PER_SEC / HZ;
+#ifdef CONFIG_NO_HZ_FULL
+			/*
+			 * The boot CPU may be nohz_full, in which case set
+			 * tick_do_timer_boot_cpu so the first housekeeping
+			 * secondary that comes up will take do_timer from
+			 * us.
+			 */
+			if (tick_nohz_full_cpu(cpu))
+				tick_do_timer_boot_cpu = cpu;
+
+		} else if (tick_do_timer_boot_cpu != -1 &&
+						!tick_nohz_full_cpu(cpu)) {
+			tick_take_do_timer_from_boot();
+			tick_do_timer_boot_cpu = -1;
+			WARN_ON(tick_do_timer_cpu != cpu);
+#endif
 		}
 
 		/*
kernel/time/tick-sched.c +24 -10
index 6fa52cd6df0b..4aa917acbe1c 100644
@@ -121,10 +121,16 @@ static void tick_sched_do_timer(struct tick_sched *ts, ktime_t now)
 	 * into a long sleep. If two CPUs happen to assign themselves to
 	 * this duty, then the jiffies update is still serialized by
 	 * jiffies_lock.
+	 *
+	 * If nohz_full is enabled, this should not happen because the
+	 * tick_do_timer_cpu never relinquishes.
 	 */
-	if (unlikely(tick_do_timer_cpu == TICK_DO_TIMER_NONE)
-	    && !tick_nohz_full_cpu(cpu))
+	if (unlikely(tick_do_timer_cpu == TICK_DO_TIMER_NONE)) {
+#ifdef CONFIG_NO_HZ_FULL
+		WARN_ON(tick_nohz_full_running);
+#endif
 		tick_do_timer_cpu = cpu;
+	}
 #endif
 
 	/* Check, if the jiffies need an update */
@@ -395,8 +401,8 @@ void __init tick_nohz_full_setup(cpumask_var_t cpumask)
 static int tick_nohz_cpu_down(unsigned int cpu)
 {
 	/*
-	 * The boot CPU handles housekeeping duty (unbound timers,
-	 * workqueues, timekeeping, ...) on behalf of full dynticks
+	 * The tick_do_timer_cpu CPU handles housekeeping duty (unbound
+	 * timers, workqueues, timekeeping, ...) on behalf of full dynticks
 	 * CPUs. It must remain online when nohz full is enabled.
 	 */
 	if (tick_nohz_full_running && tick_do_timer_cpu == cpu)
@@ -423,12 +429,15 @@ void __init tick_nohz_init(void)
 		return;
 	}
 
-	cpu = smp_processor_id();
+	if (IS_ENABLED(CONFIG_PM_SLEEP_SMP) &&
+			!IS_ENABLED(CONFIG_PM_SLEEP_SMP_NONZERO_CPU)) {
+		cpu = smp_processor_id();
 
-	if (cpumask_test_cpu(cpu, tick_nohz_full_mask)) {
-		pr_warn("NO_HZ: Clearing %d from nohz_full range for timekeeping\n",
-			cpu);
-		cpumask_clear_cpu(cpu, tick_nohz_full_mask);
+		if (cpumask_test_cpu(cpu, tick_nohz_full_mask)) {
+			pr_warn("NO_HZ: Clearing %d from nohz_full range "
+				"for timekeeping\n", cpu);
+			cpumask_clear_cpu(cpu, tick_nohz_full_mask);
+		}
 	}
 
 	for_each_cpu(cpu, tick_nohz_full_mask)
@@ -904,8 +913,13 @@ static bool can_stop_idle_tick(int cpu, struct tick_sched *ts)
 		/*
 		 * Boot safety: make sure the timekeeping duty has been
 		 * assigned before entering dyntick-idle mode,
+		 * tick_do_timer_cpu is TICK_DO_TIMER_BOOT
 		 */
-		if (tick_do_timer_cpu == TICK_DO_TIMER_NONE)
+		if (unlikely(tick_do_timer_cpu == TICK_DO_TIMER_BOOT))
+			return false;
+
+		/* Should not happen for nohz-full */
+		if (WARN_ON_ONCE(tick_do_timer_cpu == TICK_DO_TIMER_NONE))
 			return false;
 	}
 
#5 lore raw

[tip:sched/core] power/suspend: Add function to disable secondaries for suspend

In reply to Nicholas Piggin
power/suspend: Add function to disable secondaries for suspend
Commit-ID:  2f1a6fbbef7781382850c3104ecb658f21b5d460
Gitweb:     https://git.kernel.org/tip/2f1a6fbbef7781382850c3104ecb658f21b5d460
Author:     Nicholas Piggin <[email protected]>
AuthorDate: Thu, 11 Apr 2019 13:34:45 +1000
Committer:  Ingo Molnar <[email protected]>
CommitDate: Fri, 3 May 2019 19:42:41 +0200

power/suspend: Add function to disable secondaries for suspend

This adds a function to disable secondary CPUs for suspend that are not necessarily non-zero / non-boot CPUs. Platforms will be able to use this to suspend using non-zero CPUs.

---
 include/linux/cpu.h      | 12 ++++++++++++
 kernel/kexec_core.c      |  4 ++--
 kernel/power/hibernate.c | 12 ++++++------
 kernel/power/suspend.c   |  4 ++--
 4 files changed, 22 insertions(+), 10 deletions(-)
include/linux/cpu.h +12 -0
index 5041357d0297..7ab2f09c0a14 100644
@@ -137,9 +137,21 @@ static inline int disable_nonboot_cpus(void)
 	return freeze_secondary_cpus(0);
 }
 extern void enable_nonboot_cpus(void);
+
+static inline int suspend_disable_secondary_cpus(void)
+{
+	return freeze_secondary_cpus(0);
+}
+static inline void suspend_enable_secondary_cpus(void)
+{
+	return enable_nonboot_cpus();
+}
+
 #else /* !CONFIG_PM_SLEEP_SMP */
 static inline int disable_nonboot_cpus(void) { return 0; }
 static inline void enable_nonboot_cpus(void) {}
+static inline int suspend_disable_secondary_cpus(void) { return 0; }
+static inline void suspend_enable_secondary_cpus(void) { }
 #endif /* !CONFIG_PM_SLEEP_SMP */
 
 void cpu_startup_entry(enum cpuhp_state state);
kernel/kexec_core.c +2 -2
index d7140447be75..fd5c95ff9251 100644
@@ -1150,7 +1150,7 @@ int kernel_kexec(void)
 		error = dpm_suspend_end(PMSG_FREEZE);
 		if (error)
 			goto Resume_devices;
-		error = disable_nonboot_cpus();
+		error = suspend_disable_secondary_cpus();
 		if (error)
 			goto Enable_cpus;
 		local_irq_disable();
@@ -1183,7 +1183,7 @@ int kernel_kexec(void)
  Enable_irqs:
 		local_irq_enable();
  Enable_cpus:
-		enable_nonboot_cpus();
+		suspend_enable_secondary_cpus();
 		dpm_resume_start(PMSG_RESTORE);
  Resume_devices:
 		dpm_resume_end(PMSG_RESTORE);
kernel/power/hibernate.c +6 -6
index abef759de7c8..cfc7a57049e4 100644
@@ -281,7 +281,7 @@ static int create_image(int platform_mode)
 	if (error || hibernation_test(TEST_PLATFORM))
 		goto Platform_finish;
 
-	error = disable_nonboot_cpus();
+	error = suspend_disable_secondary_cpus();
 	if (error || hibernation_test(TEST_CPUS))
 		goto Enable_cpus;
 
@@ -323,7 +323,7 @@ static int create_image(int platform_mode)
 	local_irq_enable();
 
  Enable_cpus:
-	enable_nonboot_cpus();
+	suspend_enable_secondary_cpus();
 
  Platform_finish:
 	platform_finish(platform_mode);
@@ -417,7 +417,7 @@ int hibernation_snapshot(int platform_mode)
 
 int __weak hibernate_resume_nonboot_cpu_disable(void)
 {
-	return disable_nonboot_cpus();
+	return suspend_disable_secondary_cpus();
 }
 
 /**
@@ -486,7 +486,7 @@ static int resume_target_kernel(bool platform_mode)
 	local_irq_enable();
 
  Enable_cpus:
-	enable_nonboot_cpus();
+	suspend_enable_secondary_cpus();
 
  Cleanup:
 	platform_restore_cleanup(platform_mode);
@@ -564,7 +564,7 @@ int hibernation_platform_enter(void)
 	if (error)
 		goto Platform_finish;
 
-	error = disable_nonboot_cpus();
+	error = suspend_disable_secondary_cpus();
 	if (error)
 		goto Enable_cpus;
 
@@ -586,7 +586,7 @@ int hibernation_platform_enter(void)
 	local_irq_enable();
 
  Enable_cpus:
-	enable_nonboot_cpus();
+	suspend_enable_secondary_cpus();
 
  Platform_finish:
 	hibernation_ops->finish();
kernel/power/suspend.c +2 -2
index 0bd595a0b610..59b6def23046 100644
@@ -428,7 +428,7 @@ static int suspend_enter(suspend_state_t state, bool *wakeup)
 	if (suspend_test(TEST_PLATFORM))
 		goto Platform_wake;
 
-	error = disable_nonboot_cpus();
+	error = suspend_disable_secondary_cpus();
 	if (error || suspend_test(TEST_CPUS))
 		goto Enable_cpus;
 
@@ -458,7 +458,7 @@ static int suspend_enter(suspend_state_t state, bool *wakeup)
 	BUG_ON(irqs_disabled());
 
  Enable_cpus:
-	enable_nonboot_cpus();
+	suspend_enable_secondary_cpus();
 
  Platform_wake:
 	platform_resume_noirq(state);
#10 lore raw

[tip:sched/core] kernel/cpu: Allow non-zero CPU to be primary for suspend / kexec freeze

In reply to Nicholas Piggin
kernel/cpu: Allow non-zero CPU to be primary for suspend / kexec freeze
Commit-ID:  9ca12ac04bb7d7cfb28aa549dcd3d15761f15543
Gitweb:     https://git.kernel.org/tip/9ca12ac04bb7d7cfb28aa549dcd3d15761f15543
Author:     Nicholas Piggin <[email protected]>
AuthorDate: Thu, 11 Apr 2019 13:34:46 +1000
Committer:  Ingo Molnar <[email protected]>
CommitDate: Fri, 3 May 2019 19:42:58 +0200

kernel/cpu: Allow non-zero CPU to be primary for suspend / kexec freeze

This patch provides an arch option, ARCH_SUSPEND_NONZERO_CPU, to opt-in to allowing suspend to occur on one of the housekeeping CPUs rather than hardcoded CPU0.

This will allow CPU0 to be a nohz_full CPU with a later change.

It may be possible for platforms with hardware/firmware restrictions on suspend/wake effectively support this by handing off the final stage to CPU0 when kernel housekeeping is no longer required. Another option is to make housekeeping / nohz_full mask dynamic at runtime, but the complexity could not be justified at this time.

---
 arch/powerpc/Kconfig |  4 ++++
 include/linux/cpu.h  |  7 ++++++-
 kernel/cpu.c         | 10 +++++++++-
 kernel/power/Kconfig |  9 +++++++++
 4 files changed, 28 insertions(+), 2 deletions(-)
arch/powerpc/Kconfig +4 -0
index 2d0be82c3061..bc98b0e37a10 100644
@@ -318,6 +318,10 @@ config ARCH_SUSPEND_POSSIBLE
 		   (PPC_85xx && !PPC_E500MC) || PPC_86xx || PPC_PSERIES \
 		   || 44x || 40x
 
+config ARCH_SUSPEND_NONZERO_CPU
+	def_bool y
+	depends on PPC_POWERNV || PPC_PSERIES
+
 config PPC_DCR_NATIVE
 	bool
 
include/linux/cpu.h +6 -1
index 7ab2f09c0a14..73baab8535c1 100644
@@ -140,7 +140,12 @@ extern void enable_nonboot_cpus(void);
 
 static inline int suspend_disable_secondary_cpus(void)
 {
-	return freeze_secondary_cpus(0);
+	int cpu = 0;
+
+	if (IS_ENABLED(CONFIG_PM_SLEEP_SMP_NONZERO_CPU))
+		cpu = -1;
+
+	return freeze_secondary_cpus(cpu);
 }
 static inline void suspend_enable_secondary_cpus(void)
 {
kernel/cpu.c +9 -1
index 6754f3ecfd94..d1bf6e2b4752 100644
@@ -9,6 +9,7 @@
 #include <linux/notifier.h>
 #include <linux/sched/signal.h>
 #include <linux/sched/hotplug.h>
+#include <linux/sched/isolation.h>
 #include <linux/sched/task.h>
 #include <linux/sched/smt.h>
 #include <linux/unistd.h>
@@ -1199,8 +1200,15 @@ int freeze_secondary_cpus(int primary)
 	int cpu, error = 0;
 
 	cpu_maps_update_begin();
-	if (!cpu_online(primary))
+	if (primary == -1) {
 		primary = cpumask_first(cpu_online_mask);
+		if (!housekeeping_cpu(primary, HK_FLAG_TIMER))
+			primary = housekeeping_any_cpu(HK_FLAG_TIMER);
+	} else {
+		if (!cpu_online(primary))
+			primary = cpumask_first(cpu_online_mask);
+	}
+
 	/*
 	 * We take down all of the non-boot CPUs in one shot to avoid races
 	 * with the userspace trying to use the CPU hotplug at the same time
kernel/power/Kconfig +9 -0
index f8fe57d1022e..9bbaaab14b36 100644
@@ -114,6 +114,15 @@ config PM_SLEEP_SMP
 	depends on PM_SLEEP
 	select HOTPLUG_CPU
 
+config PM_SLEEP_SMP_NONZERO_CPU
+	def_bool y
+	depends on PM_SLEEP_SMP
+	depends on ARCH_SUSPEND_NONZERO_CPU
+	---help---
+	If an arch can suspend (for suspend, hibernate, kexec, etc) on a
+	non-zero numbered CPU, it may define ARCH_SUSPEND_NONZERO_CPU. This
+	will allow nohz_full mask to include CPU0.
+
 config PM_AUTOSLEEP
 	bool "Opportunistic sleep"
 	depends on PM_SLEEP
#13 lore raw

[tip:sched/core] sched/isolation: Require a present CPU in housekeeping mask

In reply to Nicholas Piggin
sched/isolation: Require a present CPU in housekeeping mask
Commit-ID:  9219565aa89033a9cfdae788c1940473a1253d6c
Gitweb:     https://git.kernel.org/tip/9219565aa89033a9cfdae788c1940473a1253d6c
Author:     Nicholas Piggin <[email protected]>
AuthorDate: Thu, 11 Apr 2019 13:34:47 +1000
Committer:  Ingo Molnar <[email protected]>
CommitDate: Fri, 3 May 2019 19:42:58 +0200

sched/isolation: Require a present CPU in housekeeping mask

During housekeeping mask setup, currently a possible CPU is required. That does not guarantee the CPU would be available at boot time, so check to ensure that at least one present CPU is in the mask.

---
 kernel/sched/isolation.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)
kernel/sched/isolation.c +13 -5
index b02d148e7672..687302051a27 100644
@@ -65,6 +65,7 @@ void __init housekeeping_init(void)
 static int __init housekeeping_setup(char *str, enum hk_flags flags)
 {
 	cpumask_var_t non_housekeeping_mask;
+	cpumask_var_t tmp;
 	int err;
 
 	alloc_bootmem_cpumask_var(&non_housekeeping_mask);
@@ -75,16 +76,23 @@ static int __init housekeeping_setup(char *str, enum hk_flags flags)
 		return 0;
 	}
 
+	alloc_bootmem_cpumask_var(&tmp);
 	if (!housekeeping_flags) {
 		alloc_bootmem_cpumask_var(&housekeeping_mask);
 		cpumask_andnot(housekeeping_mask,
 			       cpu_possible_mask, non_housekeeping_mask);
-		if (cpumask_empty(housekeeping_mask))
+
+		cpumask_andnot(tmp, cpu_present_mask, non_housekeeping_mask);
+		if (cpumask_empty(tmp)) {
+			pr_warn("Housekeeping: must include one present CPU, "
+				"using boot CPU:%d\n", smp_processor_id());
 			__cpumask_set_cpu(smp_processor_id(), housekeeping_mask);
+			__cpumask_clear_cpu(smp_processor_id(), non_housekeeping_mask);
+		}
 	} else {
-		cpumask_var_t tmp;
-
-		alloc_bootmem_cpumask_var(&tmp);
+		cpumask_andnot(tmp, cpu_present_mask, non_housekeeping_mask);
+		if (cpumask_empty(tmp))
+			__cpumask_clear_cpu(smp_processor_id(), non_housekeeping_mask);
 		cpumask_andnot(tmp, cpu_possible_mask, non_housekeeping_mask);
 		if (!cpumask_equal(tmp, housekeeping_mask)) {
 			pr_warn("Housekeeping: nohz_full= must match isolcpus=\n");
@@ -92,8 +100,8 @@ static int __init housekeeping_setup(char *str, enum hk_flags flags)
 			free_bootmem_cpumask_var(non_housekeeping_mask);
 			return 0;
 		}
-		free_bootmem_cpumask_var(tmp);
 	}
+	free_bootmem_cpumask_var(tmp);
 
 	if ((flags & HK_FLAG_TICK) && !(housekeeping_flags & HK_FLAG_TICK)) {
 		if (IS_ENABLED(CONFIG_NO_HZ_FULL)) {
#23 lore raw

[tip:sched/core] nohz_full: Allow the boot CPU to be nohz_full

In reply to Nicholas Piggin
nohz_full: Allow the boot CPU to be nohz_full
Commit-ID:  08ae95f4fd3b38b257f5dc7e6507e071c27ba0d5
Gitweb:     https://git.kernel.org/tip/08ae95f4fd3b38b257f5dc7e6507e071c27ba0d5
Author:     Nicholas Piggin <[email protected]>
AuthorDate: Thu, 11 Apr 2019 13:34:48 +1000
Committer:  Ingo Molnar <[email protected]>
CommitDate: Fri, 3 May 2019 19:42:58 +0200

nohz_full: Allow the boot CPU to be nohz_full

Allow the boot CPU/CPU0 to be nohz_full. Have the boot CPU take the do_timer duty during boot until a housekeeping CPU can take over.

This is supported when CONFIG_PM_SLEEP_SMP is not configured, or when it is configured and the arch allows suspend on non-zero CPUs.

nohz_full has been trialed at a large supercomputer site and found to significantly reduce jitter. In order to deploy it in production, they need CPU0 to be nohz_full because their job control system requires the application CPUs to start from 0, and the housekeeping CPUs are placed higher. An equivalent job scheduling that uses CPU0 for housekeeping could be achieved by modifying their system, but it is preferable if nohz_full can support their environment without modification.

---
 kernel/time/tick-common.c | 50 +++++++++++++++++++++++++++++++++++++++++++----
 kernel/time/tick-sched.c  | 34 ++++++++++++++++++++++----------
 2 files changed, 70 insertions(+), 14 deletions(-)
kernel/time/tick-common.c +46 -4
index df401463a191..e49e8091f9ac 100644
@@ -46,6 +46,14 @@ ktime_t tick_period;
  *    procedure also covers cpu hotplug.
  */
 int tick_do_timer_cpu __read_mostly = TICK_DO_TIMER_BOOT;
+#ifdef CONFIG_NO_HZ_FULL
+/*
+ * tick_do_timer_boot_cpu indicates the boot CPU temporarily owns
+ * tick_do_timer_cpu and it should be taken over by an eligible secondary
+ * when one comes online.
+ */
+static int tick_do_timer_boot_cpu __read_mostly = -1;
+#endif
 
 /*
  * Debugging: see timer_list.c
@@ -167,6 +175,26 @@ void tick_setup_periodic(struct clock_event_device *dev, int broadcast)
 	}
 }
 
+#ifdef CONFIG_NO_HZ_FULL
+static void giveup_do_timer(void *info)
+{
+	int cpu = *(unsigned int *)info;
+
+	WARN_ON(tick_do_timer_cpu != smp_processor_id());
+
+	tick_do_timer_cpu = cpu;
+}
+
+static void tick_take_do_timer_from_boot(void)
+{
+	int cpu = smp_processor_id();
+	int from = tick_do_timer_boot_cpu;
+
+	if (from >= 0 && from != cpu)
+		smp_call_function_single(from, giveup_do_timer, &cpu, 1);
+}
+#endif
+
 /*
  * Setup the tick device
  */
@@ -186,12 +214,26 @@ static void tick_setup_device(struct tick_device *td,
 		 * this cpu:
 		 */
 		if (tick_do_timer_cpu == TICK_DO_TIMER_BOOT) {
-			if (!tick_nohz_full_cpu(cpu))
-				tick_do_timer_cpu = cpu;
-			else
-				tick_do_timer_cpu = TICK_DO_TIMER_NONE;
+			tick_do_timer_cpu = cpu;
+
 			tick_next_period = ktime_get();
 			tick_period = NSEC_PER_SEC / HZ;
+#ifdef CONFIG_NO_HZ_FULL
+			/*
+			 * The boot CPU may be nohz_full, in which case set
+			 * tick_do_timer_boot_cpu so the first housekeeping
+			 * secondary that comes up will take do_timer from
+			 * us.
+			 */
+			if (tick_nohz_full_cpu(cpu))
+				tick_do_timer_boot_cpu = cpu;
+
+		} else if (tick_do_timer_boot_cpu != -1 &&
+						!tick_nohz_full_cpu(cpu)) {
+			tick_take_do_timer_from_boot();
+			tick_do_timer_boot_cpu = -1;
+			WARN_ON(tick_do_timer_cpu != cpu);
+#endif
 		}
 
 		/*
kernel/time/tick-sched.c +24 -10
index 6fa52cd6df0b..4aa917acbe1c 100644
@@ -121,10 +121,16 @@ static void tick_sched_do_timer(struct tick_sched *ts, ktime_t now)
 	 * into a long sleep. If two CPUs happen to assign themselves to
 	 * this duty, then the jiffies update is still serialized by
 	 * jiffies_lock.
+	 *
+	 * If nohz_full is enabled, this should not happen because the
+	 * tick_do_timer_cpu never relinquishes.
 	 */
-	if (unlikely(tick_do_timer_cpu == TICK_DO_TIMER_NONE)
-	    && !tick_nohz_full_cpu(cpu))
+	if (unlikely(tick_do_timer_cpu == TICK_DO_TIMER_NONE)) {
+#ifdef CONFIG_NO_HZ_FULL
+		WARN_ON(tick_nohz_full_running);
+#endif
 		tick_do_timer_cpu = cpu;
+	}
 #endif
 
 	/* Check, if the jiffies need an update */
@@ -395,8 +401,8 @@ void __init tick_nohz_full_setup(cpumask_var_t cpumask)
 static int tick_nohz_cpu_down(unsigned int cpu)
 {
 	/*
-	 * The boot CPU handles housekeeping duty (unbound timers,
-	 * workqueues, timekeeping, ...) on behalf of full dynticks
+	 * The tick_do_timer_cpu CPU handles housekeeping duty (unbound
+	 * timers, workqueues, timekeeping, ...) on behalf of full dynticks
 	 * CPUs. It must remain online when nohz full is enabled.
 	 */
 	if (tick_nohz_full_running && tick_do_timer_cpu == cpu)
@@ -423,12 +429,15 @@ void __init tick_nohz_init(void)
 		return;
 	}
 
-	cpu = smp_processor_id();
+	if (IS_ENABLED(CONFIG_PM_SLEEP_SMP) &&
+			!IS_ENABLED(CONFIG_PM_SLEEP_SMP_NONZERO_CPU)) {
+		cpu = smp_processor_id();
 
-	if (cpumask_test_cpu(cpu, tick_nohz_full_mask)) {
-		pr_warn("NO_HZ: Clearing %d from nohz_full range for timekeeping\n",
-			cpu);
-		cpumask_clear_cpu(cpu, tick_nohz_full_mask);
+		if (cpumask_test_cpu(cpu, tick_nohz_full_mask)) {
+			pr_warn("NO_HZ: Clearing %d from nohz_full range "
+				"for timekeeping\n", cpu);
+			cpumask_clear_cpu(cpu, tick_nohz_full_mask);
+		}
 	}
 
 	for_each_cpu(cpu, tick_nohz_full_mask)
@@ -904,8 +913,13 @@ static bool can_stop_idle_tick(int cpu, struct tick_sched *ts)
 		/*
 		 * Boot safety: make sure the timekeeping duty has been
 		 * assigned before entering dyntick-idle mode,
+		 * tick_do_timer_cpu is TICK_DO_TIMER_BOOT
 		 */
-		if (tick_do_timer_cpu == TICK_DO_TIMER_NONE)
+		if (unlikely(tick_do_timer_cpu == TICK_DO_TIMER_BOOT))
+			return false;
+
+		/* Should not happen for nohz-full */
+		if (WARN_ON_ONCE(tick_do_timer_cpu == TICK_DO_TIMER_NONE))
 			return false;
 	}
 
maintainer Acked-by
#14 lore raw

Re: [tip:sched/core] sched/isolation: Require a present CPU in housekeeping mask

In reply to tip-bot for Nicholas Piggin
Quoted 46 lines · tip-bot for Nicholas Piggin
Commit-ID:  9219565aa89033a9cfdae788c1940473a1253d6c
Gitweb:     https://git.kernel.org/tip/9219565aa89033a9cfdae788c1940473a1253d6c
Author:     Nicholas Piggin <[email protected]>
AuthorDate: Thu, 11 Apr 2019 13:34:47 +1000
Committer:  Ingo Molnar <[email protected]>
CommitDate: Fri, 3 May 2019 19:42:58 +0200

sched/isolation: Require a present CPU in housekeeping mask

During housekeeping mask setup, currently a possible CPU is required. That does not guarantee the CPU would be available at boot time, so check to ensure that at least one present CPU is in the mask.

---
 kernel/sched/isolation.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)
kernel/sched/isolation.c +2 -0
index b02d148e7672..687302051a27 100644
@@ -65,6 +65,7 @@ void __init housekeeping_init(void)
 static int __init housekeeping_setup(char *str, enum hk_flags flags)
 {
 	cpumask_var_t non_housekeeping_mask;
+	cpumask_var_t tmp;
 	int err;
 
 	alloc_bootmem_cpumask_var(&non_housekeeping_mask);
@@ -75,16 +76,23 @@ static int __init housekeeping_setup(char *str, enum hk_flags flags)
 		return 0;
 	}
 
+	alloc_bootmem_cpumask_var(&tmp);
 	if (!housekeeping_flags) {
 		alloc_bootmem_cpumask_var(&housekeeping_mask);

kernel/sched/isolation.c static int __init housekeeping_setup(char *str, enum hk_flags flags) · line 90

Quoted 10 lines
kernel/sched/isolation.c +6 -1
 		cpumask_andnot(housekeeping_mask,
 			       cpu_possible_mask, non_housekeeping_mask);
-		if (cpumask_empty(housekeeping_mask))
+
+		cpumask_andnot(tmp, cpu_present_mask, non_housekeeping_mask);
+		if (cpumask_empty(tmp)) {
+			pr_warn("Housekeeping: must include one present CPU, "
+				"using boot CPU:%d\n", smp_processor_id());
 			__cpumask_set_cpu(smp_processor_id(), housekeeping_mask);
+			__cpumask_clear_cpu(smp_processor_id(), non_housekeeping_mask);

Ah that line (along with its twin below) is actually also a fix for the upstream code. If the housekeeping mask is empty and we force the current one, we must make sure that the current CPU is excluded from the nohz_full set. Ideally it should have come as a separate patch but it's nice that you fixed it anyway.

Thanks!

Quoted 21 lines
kernel/sched/isolation.c +5 -4
+		}
 	} else {
-		cpumask_var_t tmp;
-
-		alloc_bootmem_cpumask_var(&tmp);
+		cpumask_andnot(tmp, cpu_present_mask, non_housekeeping_mask);
+		if (cpumask_empty(tmp))
+			__cpumask_clear_cpu(smp_processor_id(), non_housekeeping_mask);
 		cpumask_andnot(tmp, cpu_possible_mask, non_housekeeping_mask);
 		if (!cpumask_equal(tmp, housekeeping_mask)) {
 			pr_warn("Housekeeping: nohz_full= must match isolcpus=\n");
@@ -92,8 +100,8 @@ static int __init housekeeping_setup(char *str, enum hk_flags flags)
 			free_bootmem_cpumask_var(non_housekeeping_mask);
 			return 0;
 		}
-		free_bootmem_cpumask_var(tmp);
 	}
+	free_bootmem_cpumask_var(tmp);
 
 	if ((flags & HK_FLAG_TICK) && !(housekeeping_flags & HK_FLAG_TICK)) {
 		if (IS_ENABLED(CONFIG_NO_HZ_FULL)) {
#15 lore raw

Re: [tip:sched/core] sched/isolation: Require a present CPU in housekeeping mask

In reply to tip-bot for Nicholas Piggin
Quoted 7 lines · tip-bot for Nicholas Piggin
Commit-ID:  9219565aa89033a9cfdae788c1940473a1253d6c
Gitweb:     https://git.kernel.org/tip/9219565aa89033a9cfdae788c1940473a1253d6c
Author:     Nicholas Piggin <[email protected]>
AuthorDate: Thu, 11 Apr 2019 13:34:47 +1000
Committer:  Ingo Molnar <[email protected]>
CommitDate: Fri, 3 May 2019 19:42:58 +0200
Quoted 5 lines

sched/isolation: Require a present CPU in housekeeping mask

During housekeeping mask setup, currently a possible CPU is required. That does not guarantee the CPU would be available at boot time, so check to ensure that at least one present CPU is in the mask.

I have a doubt about the requirements and semantics of cpu_present_mask. IIUC a present CPU means that it is physically plugged in (from ACPI perspective) but might not be logically plugged in (set on cpu_online_mask).

But do we have the guarantee that a present CPU _will_ be online at least once right after the boot? After all, kernel parameters such as "maxcpus=" can prevent from turning some CPUs on. I guess there are even more creative ways to achieve that.

In any case we really require the housekeeper to be forced online. Perhaps I missed that enforcement somewhere in the patchset?

Thanks.

author reviewer
#16 lore raw

Re: [tip:sched/core] sched/isolation: Require a present CPU in housekeeping mask

Frederic Weisbecker's on May 4, 2019 10:27 am:

Quoted 10 lines · tip-bot for Nicholas Piggin
Quoted 9 lines
Commit-ID:  9219565aa89033a9cfdae788c1940473a1253d6c
Gitweb:     https://git.kernel.org/tip/9219565aa89033a9cfdae788c1940473a1253d6c
Author:     Nicholas Piggin <[email protected]>
AuthorDate: Thu, 11 Apr 2019 13:34:47 +1000
Committer:  Ingo Molnar <[email protected]>
CommitDate: Fri, 3 May 2019 19:42:58 +0200

sched/isolation: Require a present CPU in housekeeping mask

Quoted 7 lines
Quoted 3 lines

During housekeeping mask setup, currently a possible CPU is required. That does not guarantee the CPU would be available at boot time, so check to ensure that at least one present CPU is in the mask.

I have a doubt about the requirements and semantics of cpu_present_mask. IIUC a present CPU means that it is physically plugged in (from ACPI perspective) but might not be logically plugged in (set on cpu_online_mask).

Right, a superset of cpu_possible_mask, subset of cpu_online_mask. It means that CPU can be brought online at any time.

Quoted 7 lines

But do we have the guarantee that a present CPU _will_ be online at least once right after the boot? After all, kernel parameters such as "maxcpus=" can prevent from turning some CPUs on. I guess there are even more creative ways to achieve that.

In any case we really require the housekeeper to be forced online. Perhaps I missed that enforcement somewhere in the patchset?

No I think you're right, that may be able to boot without anything in the housekeeping mask. Maybe we can just cpu_up() a CPU in the housekeeping mask with a warning that it has overidden their SMP command line option. I'll take a look at it.

Thanks,
Nick

#17 lore raw

Re: [tip:sched/core] sched/isolation: Require a present CPU in housekeeping mask

Quoted 27 lines · Nicholas Piggin

Frederic Weisbecker's on May 4, 2019 10:27 am:

Quoted 10 lines · tip-bot for Nicholas Piggin
Quoted 9 lines
Commit-ID:  9219565aa89033a9cfdae788c1940473a1253d6c
Gitweb:     https://git.kernel.org/tip/9219565aa89033a9cfdae788c1940473a1253d6c
Author:     Nicholas Piggin <[email protected]>
AuthorDate: Thu, 11 Apr 2019 13:34:47 +1000
Committer:  Ingo Molnar <[email protected]>
CommitDate: Fri, 3 May 2019 19:42:58 +0200

sched/isolation: Require a present CPU in housekeeping mask

Quoted 7 lines
Quoted 3 lines

During housekeeping mask setup, currently a possible CPU is required. That does not guarantee the CPU would be available at boot time, so check to ensure that at least one present CPU is in the mask.

I have a doubt about the requirements and semantics of cpu_present_mask. IIUC a present CPU means that it is physically plugged in (from ACPI perspective) but might not be logically plugged in (set on cpu_online_mask).

Right, a superset of cpu_possible_mask, subset of cpu_online_mask. It means that CPU can be brought online at any time.

Quoted 5 lines

But do we have the guarantee that a present CPU _will_ be online at least once right after the boot? After all, kernel parameters such as "maxcpus=" can prevent from turning some CPUs on. I guess there are even more creative ways to achieve that.

Quoted 7 lines
Quoted 2 lines

In any case we really require the housekeeper to be forced online. Perhaps I missed that enforcement somewhere in the patchset?

No I think you're right, that may be able to boot without anything in the housekeeping mask. Maybe we can just cpu_up() a CPU in the housekeeping mask with a warning that it has overidden their SMP command line option. I'll take a look at it.

But then what if cpu_up() fails? In this case I can think of only two answers:

* Force the boot CPU as the housekeeper.
* Rollback the whole thing: nohz and all isolation.

The second solution looks sane to me. After all if the user doesn't include CPU 0 in the housekeeping set, forcing it isn't going to help much.

But that means we must enhance the isolation code (nohz included) to be able to dynamically add/del CPUs to the houseeeping/isolation set. That's not going to be easy but it's a necessary evolution of that subsystem since we want to drive it through cpusets.

I should start working on that.

author reviewer
#18 lore raw

Re: [tip:sched/core] sched/isolation: Require a present CPU in housekeeping mask

Frederic Weisbecker's on May 7, 2019 1:16 am:

Quoted 36 lines · Nicholas Piggin
Quoted 27 lines

Frederic Weisbecker's on May 4, 2019 10:27 am:

Quoted 10 lines · tip-bot for Nicholas Piggin
Quoted 9 lines
Commit-ID:  9219565aa89033a9cfdae788c1940473a1253d6c
Gitweb:     https://git.kernel.org/tip/9219565aa89033a9cfdae788c1940473a1253d6c
Author:     Nicholas Piggin <[email protected]>
AuthorDate: Thu, 11 Apr 2019 13:34:47 +1000
Committer:  Ingo Molnar <[email protected]>
CommitDate: Fri, 3 May 2019 19:42:58 +0200

sched/isolation: Require a present CPU in housekeeping mask

Quoted 7 lines
Quoted 3 lines

During housekeeping mask setup, currently a possible CPU is required. That does not guarantee the CPU would be available at boot time, so check to ensure that at least one present CPU is in the mask.

I have a doubt about the requirements and semantics of cpu_present_mask. IIUC a present CPU means that it is physically plugged in (from ACPI perspective) but might not be logically plugged in (set on cpu_online_mask).

Right, a superset of cpu_possible_mask, subset of cpu_online_mask. It means that CPU can be brought online at any time.

Quoted 5 lines

But do we have the guarantee that a present CPU _will_ be online at least once right after the boot? After all, kernel parameters such as "maxcpus=" can prevent from turning some CPUs on. I guess there are even more creative ways to achieve that.

Quoted 7 lines
Quoted 2 lines

In any case we really require the housekeeper to be forced online. Perhaps I missed that enforcement somewhere in the patchset?

No I think you're right, that may be able to boot without anything in the housekeeping mask. Maybe we can just cpu_up() a CPU in the housekeeping mask with a warning that it has overidden their SMP command line option. I'll take a look at it.

Quoted 5 lines

But then what if cpu_up() fails? In this case I can think of only two answers:

* Force the boot CPU as the housekeeper.
* Rollback the whole thing: nohz and all isolation.

If cpu_up fails despite being in the present map and we explicitly selected it as the housekeeper? I think it would be okay to print a message telling admin to correct the config, and panic.

We try a best effort to make the system boot and limp along, but if you misconfigure it, crashing is not unreasonable. There's lots of command line option misconfiguration that will cause the same thing.

The primary problem with my patch that needs to be addressed is that the error is not explicitly caught and printed if the housekeeper does not come up, so the system might die in non-obvious ways.

Quoted 11 lines

The second solution looks sane to me. After all if the user doesn't include CPU 0 in the housekeeping set, forcing it isn't going to help much.

But that means we must enhance the isolation code (nohz included) to be able to dynamically add/del CPUs to the houseeeping/isolation set. That's not going to be easy but it's a necessary evolution of that subsystem since we want to drive it through cpusets.

I should start working on that.

I considered that when looking at the series, but couldn't justify the complexity based on my usage (which is static boot time).

If you have other uses for it, then that would solve all these boot time issues as well, which will be nice.

Thanks,
Nick

#19 lore raw

Re: [tip:sched/core] sched/isolation: Require a present CPU in housekeeping mask

Quoted 47 lines · Nicholas Piggin

Frederic Weisbecker's on May 7, 2019 1:16 am:

Quoted 36 lines · Nicholas Piggin
Quoted 27 lines

Frederic Weisbecker's on May 4, 2019 10:27 am:

Quoted 10 lines · tip-bot for Nicholas Piggin
Quoted 9 lines
Commit-ID:  9219565aa89033a9cfdae788c1940473a1253d6c
Gitweb:     https://git.kernel.org/tip/9219565aa89033a9cfdae788c1940473a1253d6c
Author:     Nicholas Piggin <[email protected]>
AuthorDate: Thu, 11 Apr 2019 13:34:47 +1000
Committer:  Ingo Molnar <[email protected]>
CommitDate: Fri, 3 May 2019 19:42:58 +0200

sched/isolation: Require a present CPU in housekeeping mask

Quoted 7 lines
Quoted 3 lines

During housekeeping mask setup, currently a possible CPU is required. That does not guarantee the CPU would be available at boot time, so check to ensure that at least one present CPU is in the mask.

I have a doubt about the requirements and semantics of cpu_present_mask. IIUC a present CPU means that it is physically plugged in (from ACPI perspective) but might not be logically plugged in (set on cpu_online_mask).

Right, a superset of cpu_possible_mask, subset of cpu_online_mask. It means that CPU can be brought online at any time.

Quoted 5 lines

But do we have the guarantee that a present CPU _will_ be online at least once right after the boot? After all, kernel parameters such as "maxcpus=" can prevent from turning some CPUs on. I guess there are even more creative ways to achieve that.

Quoted 7 lines
Quoted 2 lines

In any case we really require the housekeeper to be forced online. Perhaps I missed that enforcement somewhere in the patchset?

No I think you're right, that may be able to boot without anything in the housekeeping mask. Maybe we can just cpu_up() a CPU in the housekeeping mask with a warning that it has overidden their SMP command line option. I'll take a look at it.

Quoted 5 lines

But then what if cpu_up() fails? In this case I can think of only two answers:

* Force the boot CPU as the housekeeper.
* Rollback the whole thing: nohz and all isolation.

If cpu_up fails despite being in the present map and we explicitly selected it as the housekeeper? I think it would be okay to print a message telling admin to correct the config, and panic.

Quoted 7 lines

We try a best effort to make the system boot and limp along, but if you misconfigure it, crashing is not unreasonable. There's lots of command line option misconfiguration that will cause the same thing.

The primary problem with my patch that needs to be addressed is that the error is not explicitly caught and printed if the housekeeper does not come up, so the system might die in non-obvious ways.

I usually reserve panic and BUG_ON() to last resort when data integrity is directly threatened. But indeed I guess that's all we have for now.

If we take that path, I'd rather not call that cpu_up() and simply panic if the given CPU happens not to be online after SMP bootup.

Quoted 11 lines
Quoted 11 lines

The second solution looks sane to me. After all if the user doesn't include CPU 0 in the housekeeping set, forcing it isn't going to help much.

But that means we must enhance the isolation code (nohz included) to be able to dynamically add/del CPUs to the houseeeping/isolation set. That's not going to be easy but it's a necessary evolution of that subsystem since we want to drive it through cpusets.

Quoted 7 lines
Quoted 1 line

I should start working on that.

I considered that when looking at the series, but couldn't justify the complexity based on my usage (which is static boot time).

If you have other uses for it, then that would solve all these boot time issues as well, which will be nice.

Yeah cpuset is going to be a usecase.

I'm going to work on that so that the boot CPU is always the housekeeper in the beginning, then that duty can be later passed to any secondary CPU or the whole can be rolled back. The current situation with the temporary housekeeper that isn't a real one makes me a bit uncomfortable.

Thanks.

author reviewer
#20 lore raw

Re: [tip:sched/core] sched/isolation: Require a present CPU in housekeeping mask

Frederic Weisbecker's on May 8, 2019 10:35 am:

Quoted 52 lines · Nicholas Piggin
Quoted 51 lines

Frederic Weisbecker's on May 7, 2019 1:16 am:

Quoted 36 lines · Nicholas Piggin
Quoted 27 lines

Frederic Weisbecker's on May 4, 2019 10:27 am:

Quoted 10 lines · tip-bot for Nicholas Piggin
Quoted 9 lines
Commit-ID:  9219565aa89033a9cfdae788c1940473a1253d6c
Gitweb:     https://git.kernel.org/tip/9219565aa89033a9cfdae788c1940473a1253d6c
Author:     Nicholas Piggin <[email protected]>
AuthorDate: Thu, 11 Apr 2019 13:34:47 +1000
Committer:  Ingo Molnar <[email protected]>
CommitDate: Fri, 3 May 2019 19:42:58 +0200

sched/isolation: Require a present CPU in housekeeping mask

Quoted 7 lines
Quoted 3 lines

During housekeeping mask setup, currently a possible CPU is required. That does not guarantee the CPU would be available at boot time, so check to ensure that at least one present CPU is in the mask.

I have a doubt about the requirements and semantics of cpu_present_mask. IIUC a present CPU means that it is physically plugged in (from ACPI perspective) but might not be logically plugged in (set on cpu_online_mask).

Right, a superset of cpu_possible_mask, subset of cpu_online_mask. It means that CPU can be brought online at any time.

Quoted 5 lines

But do we have the guarantee that a present CPU _will_ be online at least once right after the boot? After all, kernel parameters such as "maxcpus=" can prevent from turning some CPUs on. I guess there are even more creative ways to achieve that.

Quoted 7 lines
Quoted 2 lines

In any case we really require the housekeeper to be forced online. Perhaps I missed that enforcement somewhere in the patchset?

No I think you're right, that may be able to boot without anything in the housekeeping mask. Maybe we can just cpu_up() a CPU in the housekeeping mask with a warning that it has overidden their SMP command line option. I'll take a look at it.

Quoted 5 lines

But then what if cpu_up() fails? In this case I can think of only two answers:

* Force the boot CPU as the housekeeper.
* Rollback the whole thing: nohz and all isolation.

If cpu_up fails despite being in the present map and we explicitly selected it as the housekeeper? I think it would be okay to print a message telling admin to correct the config, and panic.

We try a best effort to make the system boot and limp along, but if you misconfigure it, crashing is not unreasonable. There's lots of command line option misconfiguration that will cause the same thing.

Quoted 6 lines
Quoted 3 lines

The primary problem with my patch that needs to be addressed is that the error is not explicitly caught and printed if the housekeeper does not come up, so the system might die in non-obvious ways.

I usually reserve panic and BUG_ON() to last resort when data integrity is directly threatened. But indeed I guess that's all we have for now.

Right, specifying a CPU for housekeeping that excluded from coming up at boot with maxcpus= or whatever, is not such a big deal to panic I think. Just need to have a clear error message.

Quoted 2 lines

If we take that path, I'd rather not call that cpu_up() and simply panic if the given CPU happens not to be online after SMP bootup.

Sure that's fine by me too.

Thanks,
Nick