Skip to contentSkip to conversation

sched: Handle CPU freeze without active domain housekeeping CPUs

RFC PATCH New Guopeng Zhang 1 participant 4 messages Jul 22

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

cover sched: Handle CPU freeze without active domain housekeeping CPUs

From: Guopeng Zhang <[email protected]>

During suspend, freeze_secondary_cpus() takes every CPU except the primary offline. With domain isolation, the primary CPU can be excluded from HK_TYPE_DOMAIN. Offlining the last domain housekeeping CPU then leaves the frozen hotplug path without an active CPU in that mask.

This exposes three related problems. The frozen cpuset callback asks for one fallback scheduler domain even though its span is empty. A user task restricted to the CPUs being frozen can exhaust select_fallback_rq() because the normal fallback mask has no active CPU. Finally, forcing such a task onto a temporary CPU clears user_cpus_ptr, so the temporary affinity can survive after the CPUs are brought back online.

This series preserves and restores user affinity across the frozen CPU fallback, permits an isolated but task-capable active CPU as a temporary last resort, and explicitly tears down scheduler domains while no active HK_TYPE_DOMAIN CPU exists.

I am sending this as an RFC because the first patch adds scheduler state for the freeze transaction and collects affected tasks during thaw. In particular, feedback would be useful on the task tracking and restore scheme, the use of task_cpu_possible_mask() as the temporary fallback, and the explicit zero-domain request passed to partition_sched_domains().

Reproduction
============

The problem was reproduced on a 32-CPU x86 physical machine booted with:

    isolcpus=domain,0,3-31

This leaves CPUs 1 and 2 in HK_TYPE_DOMAIN while the suspend primary is CPU0. On an unpatched 7.2.0-rc3-next-20260716 kernel, with panic_on_warn=0 and warn_limit=0, the following processor-stage suspend test crashed the machine:

echo deep > /sys/power/mem_sleep
echo processors > /sys/power/pm_test
echo mem > /sys/power/state

After CPU2 had been taken offline, rebuilding the scheduler domains for CPU1 produced an empty span warning, followed by a general protection fault in build_perf_domains():

    smpboot: CPU 2 is now offline
    ------------[ cut here ]------------
    WARNING: kernel/sched/topology.c:3093 at build_sched_domains+0x443/0xa60
    ...
    Call Trace:
     partition_sched_domains_locked+0x2dd/0x710
     partition_sched_domains+0x30/0x40
     cpuset_reset_sched_domains+0x25/0x40
     sched_cpu_deactivate+0x2f2/0x300
    ...
    Oops: general protection fault, probably for non-canonical address ...
    RIP: 0010:build_perf_domains+0x40/0x230
    Call Trace:
     partition_sched_domains_locked+0x3d9/0x710
     partition_sched_domains+0x30/0x40
     cpuset_reset_sched_domains+0x25/0x40
     sched_cpu_deactivate+0x2f2/0x300

Kdump captured a vmcore after the fault. This was not a panic_on_warn failure.

Testing
=======

With this series applied, the processor-stage test was repeated while the shell was restricted to the two domain housekeeping CPUs:

taskset -pc 1,2 $$
echo deep > /sys/power/mem_sleep
echo processors > /sys/power/pm_test
echo mem > /sys/power/state
ret=$?
echo none > /sys/power/pm_test
echo "return=$ret"
taskset -pc $$
cat /sys/devices/system/cpu/online

The fallback path was exercised:

process 7271 (bash) no longer affine to cpu1

The test completed successfully and the requested affinity and online CPU mask were restored:

return=0
pid 7271's current affinity list: 1,2
0-31

A real S3 cycle was then tested with:

echo deep > /sys/power/mem_sleep
echo none > /sys/power/pm_test
rtcwake -m no -s 10
echo mem > /sys/power/state
ret=$?
echo "return=$ret"
taskset -pc $$
cat /sys/devices/system/cpu/online

The machine entered S3, resumed, and brought CPUs 1-31 back online. The result was again:

return=0
pid 7271's current affinity list: 1,2
0-31

Neither test produced a scheduler-domain warning, GPF, BUG, Oops, or panic.

Each patch was build-tested in sequence with x86_64_defconfig. The final series completed a full x86_64 build. The affected objects were also build-tested with CONFIG_CPUSETS=n and CONFIG_CPU_ISOLATION=y.

Guopeng Zhang (3):
sched: Preserve user affinity across frozen CPU fallback
sched: Allow isolated CPUs as a last resort during CPU freeze sched/topology: Tear down domains without active domain housekeeping
CPUs

 include/linux/cpuset.h        |   7 +-
 include/linux/sched/hotplug.h |   2 +
 kernel/cgroup/cpuset.c        |   6 +-
 kernel/cpu.c                  |   2 +
 kernel/sched/core.c           | 191 ++++++++++++++++++++++++++++++++--
 kernel/sched/sched.h          |   4 +-
 kernel/sched/topology.c       |   9 +-
 7 files changed, 207 insertions(+), 14 deletions(-)

base-commit: 0718283ab28bc3907e10b61a6b4be6fefa1cbb2f

Signature · 1 line
2.43.0
#1 lore raw

RFC PATCH 1/3 sched: Preserve user affinity across frozen CPU fallback

From: Guopeng Zhang <[email protected]>

select_fallback_rq() uses set_cpus_allowed_force() when a task has no allowed active CPU. The forced update normally clears user_cpus_ptr because the fallback is treated as a permanent affinity override.

CPU freeze is different. freeze_secondary_cpus() removes the secondary CPUs only for the duration of suspend, and a task restricted to those CPUs may need a temporary fallback before they are brought online again. Clearing user_cpus_ptr in this case makes the temporary forced affinity survive after the secondary CPUs have been brought back online.

Preserve user_cpus_ptr for non-kthreads while a CPU freeze transaction is in progress and restore the effective affinity after the frozen CPUs have been brought back online. Stop accepting new temporary fallbacks before collecting affected tasks, copy each requested mask under the task locks, and retry the restore if a concurrent explicit affinity update replaces it. This makes the newer affinity request take precedence.

If immediate restoration fails, retain user_cpus_ptr so that a later affinity or cpuset update can reapply the requested mask.

Fixes 851a723e45d1 ("sched: Always clear user_cpus_ptr in do_set_cpus_allowed()")
---
 include/linux/sched/hotplug.h |   2 +
 kernel/cpu.c                  |   2 +
 kernel/sched/core.c           | 184 ++++++++++++++++++++++++++++++++--
 kernel/sched/sched.h          |   4 +-
 4 files changed, 185 insertions(+), 7 deletions(-)
include/linux/sched/hotplug.h +2 -0
index 17e04859b9a4..2cfd447b345a 100644
@@ -9,6 +9,8 @@
 extern int sched_cpu_starting(unsigned int cpu);
 extern int sched_cpu_activate(unsigned int cpu);
 extern int sched_cpu_deactivate(unsigned int cpu);
+void sched_cpu_fallback_begin(void);
+void sched_cpu_fallback_end(void);
 
 #ifdef CONFIG_HOTPLUG_CPU
 extern int sched_cpu_wait_empty(unsigned int cpu);
kernel/cpu.c +2 -0
index b3c8553d7bd6..67828c2158ae 100644
@@ -1902,6 +1902,7 @@ int freeze_secondary_cpus(int primary)
 	 * with the userspace trying to use the CPU hotplug at the same time
 	 */
 	cpumask_clear(frozen_cpus);
+	sched_cpu_fallback_begin();
 
 	pr_info("Disabling non-boot CPUs ...\n");
 	for (cpu = nr_cpu_ids - 1; cpu >= 0; cpu--) {
@@ -1978,6 +1979,7 @@ void thaw_secondary_cpus(void)
 
 	cpumask_clear(frozen_cpus);
 out:
+	sched_cpu_fallback_end();
 	cpu_maps_update_done();
 }
 
kernel/sched/core.c +178 -6

234 lines not on this page · raw message

kernel/sched/sched.h +3 -1
index e3e4e68bb0c5..d44dff5186e3 100644
@@ -1402,7 +1402,9 @@ static inline int cpu_of(struct rq *rq)
 	return rq->cpu;
 }
 
-#define MDF_PUSH		0x01
+#define MDF_PUSH			0x01
+#define MDF_CPUHP_FALLBACK		0x02
+#define MDF_CPUHP_AFFINITY_CHANGED	0x04
 
 static inline bool is_migration_disabled(struct task_struct *p)
 {
Signature · 1 line
2.43.0
#2 lore raw

RFC PATCH 2/3 sched: Allow isolated CPUs as a last resort during CPU freeze

In reply to Guopeng Zhang

From: Guopeng Zhang <[email protected]>

task_cpu_fallback_mask() normally excludes domain-isolated CPUs. During CPU freeze, however, every CPU except the suspend primary is taken offline. If the primary CPU is excluded from HK_TYPE_DOMAIN, the normal fallback mask eventually contains no active CPU and select_fallback_rq() can exhaust its fallback states.

When CPU freeze is in progress and the normal fallback mask contains no active CPU, use task_cpu_possible_mask() as a temporary last resort if it still contains an active CPU. select_fallback_rq() will select only an active CPU from that mask, while architecture-specific task CPU capability constraints remain intact.

Keep the normal housekeeping fallback while it contains an active CPU. The temporary affinity is preserved and restored by the preceding change.

Fixes bf5b0c27fad2 ("sched: Switch the fallback task allowed cpumask to HK_TYPE_DOMAIN")
---
 kernel/sched/core.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)
kernel/sched/core.c +9 -2
index 8160d3287ad9..5f8ffded1a20 100644
@@ -2806,8 +2806,10 @@ do_set_cpus_allowed(struct task_struct *p, struct affinity_context *ctx)
 static DEFINE_RAW_SPINLOCK(cpu_fallback_lock);
 static bool cpu_fallback_active;
 
-static bool mark_cpu_fallback(struct task_struct *p)
+static bool mark_cpu_fallback(struct task_struct *p,
+			      const struct cpumask **fallback_mask)
 {
+	const struct cpumask *possible_mask;
 	unsigned long flags;
 	bool temporary = false;
 
@@ -2817,6 +2819,11 @@ static bool mark_cpu_fallback(struct task_struct *p)
 	raw_spin_lock_irqsave(&cpu_fallback_lock, flags);
 	if (cpu_fallback_active) {
 		p->migration_flags |= MDF_CPUHP_FALLBACK;
+		if (!cpumask_intersects(*fallback_mask, cpu_active_mask)) {
+			possible_mask = task_cpu_possible_mask(p);
+			if (cpumask_intersects(possible_mask, cpu_active_mask))
+				*fallback_mask = possible_mask;
+		}
 		temporary = true;
 	}
 	raw_spin_unlock_irqrestore(&cpu_fallback_lock, flags);
@@ -2842,7 +2849,7 @@ void set_cpus_allowed_force(struct task_struct *p, const struct cpumask *new_mas
 	};
 
 	scoped_guard (__task_rq_lock, p) {
-		if (mark_cpu_fallback(p))
+		if (mark_cpu_fallback(p, &ac.new_mask))
 			ac.flags = 0;
 		do_set_cpus_allowed(p, &ac);
 	}
Signature · 1 line
2.43.0
#3 lore raw

RFC PATCH 3/3 sched/topology: Tear down domains without active domain housekeeping CPUs

In reply to Guopeng Zhang

From: Guopeng Zhang <[email protected]>

During suspend, freeze_secondary_cpus() takes every CPU except the primary offline. If the primary CPU is excluded from HK_TYPE_DOMAIN, offlining the last domain housekeeping CPU leaves no active CPU from which to build the fallback scheduler domain.

The frozen CPU-hotplug callback nevertheless requests one domain by passing a non-zero domain count with a NULL domain array. partition_sched_domains_locked() then constructs the fallback span from cpu_active_mask and housekeeping_cpumask(HK_TYPE_DOMAIN), which is empty. build_sched_domains() warns about the empty span, followed by a general protection fault in build_perf_domains().

Make cpuset_reset_sched_domains(), including its !CONFIG_CPUSETS stub, request zero domains when no active HK_TYPE_DOMAIN CPU remains. Distinguish that explicit zero-domain request from a non-zero request with a NULL domain array, for which the existing fallback-domain behavior must be retained.

Fixes d35be8bab9b0 ("CPU hotplug, cpusets, suspend: Don't modify cpusets during suspend/resume")
---
 include/linux/cpuset.h  | 7 ++++++-
 kernel/cgroup/cpuset.c  | 6 +++++-
 kernel/sched/topology.c | 9 ++++-----
 3 files changed, 15 insertions(+), 7 deletions(-)
include/linux/cpuset.h +6 -1
index a67ba1b32d16..461ba4876a8e 100644
@@ -10,6 +10,7 @@
  */
 
 #include <linux/sched.h>
+#include <linux/sched/isolation.h>
 #include <linux/sched/topology.h>
 #include <linux/sched/task.h>
 #include <linux/cpumask.h>
@@ -284,7 +285,11 @@ static inline void rebuild_sched_domains(void)
 
 static inline void cpuset_reset_sched_domains(void)
 {
-	partition_sched_domains(1, NULL, NULL);
+	if (cpumask_intersects(cpu_active_mask,
+			       housekeeping_cpumask(HK_TYPE_DOMAIN)))
+		partition_sched_domains(1, NULL, NULL);
+	else
+		partition_sched_domains(0, NULL, NULL);
 }
 
 static inline void cpuset_print_current_mems_allowed(void)
kernel/cgroup/cpuset.c +5 -1
index 03e82d6026f5..1e66723e01b4 100644
@@ -1076,7 +1076,11 @@ void rebuild_sched_domains(void)
 void cpuset_reset_sched_domains(void)
 {
 	mutex_lock(&cpuset_mutex);
-	partition_sched_domains(1, NULL, NULL);
+	if (cpumask_intersects(cpu_active_mask,
+			       housekeeping_cpumask(HK_TYPE_DOMAIN)))
+		partition_sched_domains(1, NULL, NULL);
+	else
+		partition_sched_domains(0, NULL, NULL);
 	mutex_unlock(&cpuset_mutex);
 }
 
kernel/sched/topology.c +4 -5
index 622e2e01974c..e8d34cb1ed84 100644
@@ -3365,9 +3365,8 @@ static int dattrs_equal(struct sched_domain_attr *cur, int idx_cur,
  * and partition_sched_domains() will fallback to the single partition
  * 'fallback_doms', it also forces the domains to be rebuilt.
  *
- * If doms_new == NULL it will be replaced with cpu_online_mask.
- * ndoms_new == 0 is a special case for destroying existing domains,
- * and it will not create the default domain.
+ * ndoms_new == 0 and doms_new == NULL is a special case for destroying
+ * existing domains, and it will not create the default domain.
  *
  * Call with hotplug lock and sched_domains_mutex held
  */
@@ -3387,7 +3386,7 @@ static void partition_sched_domains_locked(int ndoms_new, cpumask_var_t doms_new
 	if (new_topology)
 		asym_cpu_capacity_scan();
 
-	if (!doms_new) {
+	if (ndoms_new && !doms_new) {
 		WARN_ON_ONCE(dattr_new);
 		n = 0;
 		doms_new = alloc_sched_domains(1);
@@ -3414,7 +3413,7 @@ static void partition_sched_domains_locked(int ndoms_new, cpumask_var_t doms_new
 	}
 
 	n = ndoms_cur;
-	if (!doms_new) {
+	if (ndoms_new && !doms_new) {
 		n = 0;
 		doms_new = &fallback_doms;
 		cpumask_and(doms_new[0], cpu_active_mask,
Signature · 1 line
2.43.0