Skip to contentSkip to conversation

sched_ext: fix kfunc, sub-sched, flatcg and idle-mask issues

PATCH Applied Tao Cui 3 participants 17 messages Nov 12 – Nov 13

Subsystems (3)
lore
Share & tools
lore

Tao Cui

author
#0 lore raw

cover sched_ext: fix kfunc, sub-sched, flatcg and idle-mask issues

From: Tao Cui <[email protected]>

Hello,

This series fixes six issues in the sched_ext core, the scx_flatcg example scheduler and the CID helpers. The core.c fixes travel as a separate series.

The DSQ iterator kfuncs bpf_iter_scx_dsq_next() and bpf_iter_scx_dsq_destroy() take the user DSQ's raw spinlock but stay callable from tracing programs, so a tracing program running while that lock is already held deadlocks the machine. scx_bpf_cid_node() is documented to return NUMA_NO_NODE for an invalid cid but aborts the scheduler instead. A sub-scheduler can declare ops.cid_online()/cid_offline() and read its own caps from ops.init_cids(), but hotplug callbacks are only delivered to the root scheduler and the caps storage isn't up during a self-read, so the callbacks never run and the self-read kills the scheduler. scx_flatcg strands a cgroup off the rbtree on the ENQ_RACE path until the watchdog kills the scheduler, and the built-in idle masks aren't cleared on hotplug, leaving an offline CPU advertised as idle.

0001 rejects the two iterator kfuncs for tracing programs. 0002 makes scx_bpf_cid_node() a benign lookup that returns the documented NUMA_NO_NODE. 0003 returns -EBUSY from a pre-init self-read instead of killing the scheduler. 0004 rejects the hotplug callback declaration on sub-schedulers at load time. 0005 waits for the picker to resolve on ENQ_RACE instead of stranding the cgroup. 0006 clears the built-in idle masks in handle_hotplug() and gates the claim path on cpu_online().

All six are verified in qemu with A/B runs against a stock kernel.

Based on sched_ext/for-next (5b740a72b55c).

Tao Cui (6):
sched_ext: Reject DSQ iterator kfuncs in tracing programs
sched_ext: Make scx_bpf_cid_node() a benign lookup for invalid cids
sched_ext: Return -EBUSY from a pre-init scx_bpf_sub_caps() read
sched_ext: Reject cid_online/cid_offline on sub-schedulers tools/sched_ext: scx_flatcg: Don't strand a cgroup on the ENQ_RACE
path
sched_ext: Clear the built-in idle masks on CPU hotplug transitions

 kernel/sched/ext/cid.c           | 10 ++++++++++
 kernel/sched/ext/ext.c           | 24 ++++++++++++++++++++++--
 kernel/sched/ext/idle.c          | 21 +++++++++++++++++++++
 kernel/sched/ext/idle.h          |  1 +
 kernel/sched/ext/internal.h      |  3 +++
 kernel/sched/ext/sub.c           | 19 ++++++++++---------
 tools/sched_ext/scx_flatcg.bpf.c | 17 +++++++++++++++--
 7 files changed, 82 insertions(+), 13 deletions(-)
Signature · 1 line
2.53.0
author
#1 lore raw

PATCH 1/6 sched_ext: Reject DSQ iterator kfuncs in tracing programs

From: Tao Cui <[email protected]>

bpf_iter_scx_dsq_next() and bpf_iter_scx_dsq_destroy() take the target user DSQ's raw spinlock, but they are in scx_kfunc_ids_any, which is also registered for BPF_PROG_TYPE_TRACING.

For an active root scheduler without ops.sub_attach, the compatibility path in scx_prog_sched() lets an unassociated tracing program resolve to that scheduler. The tracing program can then call the iterator kfuncs while the target DSQ lock is already held.

For example, a scheduler which mixes FIFO and priority-queue insertions into a user DSQ triggers scx_error() from scx_dispatch_enqueue() while holding that DSQ's lock. scx_claim_exit() emits sched_ext_exit before the lock is released:

scx_dispatch_enqueue()

    raw_spin_lock(&dsq->lock)
    scx_error()
      scx_claim_exit()
        trace_sched_ext_exit()
          tracing BPF program
            bpf_iter_scx_dsq_new(dsq_id)
            bpf_for_each(scx_dsq, p, dsq_id, 0)
              bpf_iter_scx_dsq_next(it)
                raw_spin_lock_irqsave(&dsq->lock)

The tracing program recursively acquires the same raw spinlock and locks up the CPU. Because the tracepoint runs before scx_finish_exit(), disable_irq_work is not queued and the scheduler cannot enter its normal disable path.

Reject the two iterator kfuncs from tracing programs at verification time. The kfuncs remain available to SCX struct_ops and syscall programs. bpf_iter_scx_dsq_destroy() has no independent live
reproduction: its cursor only links into the DSQ after at least one successful next(), which deadlocks in the same window first. It is included for symmetry.

This extends the coverage of commit b07a7450f6ef ("sched_ext: Reject rq-lock-taking kfuncs in tracing programs") and commit 4c2e0751d5e6 ("sched_ext: Reject scx_bpf_destroy_dsq() in tracing programs").

Fixes 650ba21b131e ("sched_ext: Implement DSQ iterator")
---
 kernel/sched/ext/ext.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
kernel/sched/ext/ext.c +6 -2
index 536648378ea3..063b2964099a 100644
@@ -12166,8 +12166,10 @@ static const struct btf_kfunc_id_set scx_kfunc_set_any = {
  * when scx_locked_rq() returns NULL. Taking it while the context already holds
  * an rq lock, the same one or another cpu's, can deadlock.
  *
- * scx_bpf_destroy_dsq() takes a user DSQ lock. A tracing program can run while
- * that lock is already held and recursively acquire it.
+ * scx_bpf_destroy_dsq() takes a user DSQ lock. So do
+ * bpf_iter_scx_dsq_next() and bpf_iter_scx_dsq_destroy(), the latter only
+ * after a successful next() has linked the cursor. A tracing program can
+ * run while that lock is already held and recursively acquire it.
  *
  * Before returning, scx_bpf_sub_grant() and scx_bpf_sub_revoke() invoke
  * ops.sub_caps_updated() of the sub-schedulers whose caps changed, so those ops
@@ -12184,6 +12186,8 @@ BTF_ID_FLAGS(func, scx_bpf_cidperf_set, KF_IMPLICIT_ARGS)
 BTF_ID_FLAGS(func, scx_bpf_sub_grant, KF_IMPLICIT_ARGS)
 BTF_ID_FLAGS(func, scx_bpf_sub_revoke, KF_IMPLICIT_ARGS)
 BTF_ID_FLAGS(func, scx_bpf_destroy_dsq, KF_IMPLICIT_ARGS)
+BTF_ID_FLAGS(func, bpf_iter_scx_dsq_next, KF_ITER_NEXT | KF_RET_NULL)
+BTF_ID_FLAGS(func, bpf_iter_scx_dsq_destroy, KF_ITER_DESTROY)
 BTF_KFUNCS_END(scx_kfunc_ids_no_tracing)
 
 /*
Signature · 3 lines
2.53.0

base-commit: 5b740a72b55c4267ba4d8010f17dde26b26f52d3
author
#3 lore raw

PATCH 2/6 sched_ext: Make scx_bpf_cid_node() a benign lookup for invalid cids

In reply to Tao Cui

From: Tao Cui <[email protected]>

scx_bpf_cid_node() is documented to return NUMA_NO_NODE when @cid is invalid or there is no scheduler. Its implementation goes through scx_cid_to_cpu(), whose cid_valid() calls scx_error() on an invalid value, so an invalid @cid does end at NUMA_NO_NODE, but only after flagging the whole scheduler for exit.

Bounds-check @cid in the kfunc instead, keeping the documented benign return and leaving the aborting validation to the paths whose contracts call for it.

Fixes 5cadf7545cc6 ("sched_ext: Add a CID NUMA node lookup kfunc")
---
 kernel/sched/ext/cid.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
kernel/sched/ext/cid.c +10 -0
index 8e04a8494e11..b09b2c73ccec 100644
@@ -690,6 +690,16 @@ __bpf_kfunc s32 scx_bpf_cid_node(s32 cid, const struct bpf_prog_aux *aux)
 	sch = scx_prog_sched(aux);
 	if (unlikely(!sch))
 		return NUMA_NO_NODE;
+
+	/*
+	 * Bounds-check @cid here rather than letting scx_cid_to_cpu() do it:
+	 * its cid_valid() aborts the scheduler on an invalid value, while
+	 * this kfunc is documented as a benign lookup that returns
+	 * NUMA_NO_NODE.
+	 */
+	if (cid < 0 || cid >= num_possible_cpus())
+		return NUMA_NO_NODE;
+
 	cpu = scx_cid_to_cpu(sch, cid);
 	return cpu < 0 ? NUMA_NO_NODE : cpu_to_node(cpu);
 }
Signature · 3 lines
2.53.0

base-commit: 5b740a72b55c4267ba4d8010f17dde26b26f52d3
author
#5 lore raw

PATCH 3/6 sched_ext: Return -EBUSY from a pre-init scx_bpf_sub_caps() read

In reply to Tao Cui

From: Tao Cui <[email protected]>

scx_bpf_sub_caps() flags the scheduler for exit when the target's caps storage isn't up yet. But the guard's own comment names a legitimate way to get there: a scheduler reading its own caps from ops.init_cids(), which runs before scx_alloc_pshards() publishes the storage. An init_cids() implementation that starts by querying its caps, a reasonable thing to do, kills the scheduler it is still initializing.

Return -EBUSY instead: the caller sees a retryable "not ready yet" and the comment's expected scenario stays survivable. This mirrors how the cid path already handles the same class of pre-init window:
reads before the tables are published test the table pointer and return -EINVAL instead of killing the scheduler.

Fixes 86094b95efcf ("sched_ext: Add per-shard cap delegation for sub-schedulers")
---
 kernel/sched/ext/sub.c | 19 ++++++++++---------
 1 file changed, 10 insertions(+), 9 deletions(-)
kernel/sched/ext/sub.c +10 -9
index d2da4d2322f8..55e637e7a54a 100644
@@ -3081,8 +3081,10 @@ __bpf_kfunc void scx_bpf_sub_revoke(u64 cgroup_id, u64 caps,
  * child - the literal granted set. A sched can read only itself or a direct
  * child.
  *
- * Return 0, -ENODEV if @cgroup_id names no direct child, or -EINVAL on bad
- * inputs.
+ * Return 0, -ENODEV if @cgroup_id names no direct child or the caller has
+ * no scheduler, -EINVAL on bad inputs, -EOPNOTSUPP on the cpu form, or
+ * -EBUSY if the target's caps storage is not up yet (expected from a
+ * self-read in ops.init_cids()).
  */
 __bpf_kfunc s32 scx_bpf_sub_caps(u64 cgroup_id, u64 caps, struct scx_cmask *out__arena,
 				 const struct bpf_prog_aux *aux)
@@ -3124,15 +3126,14 @@ __bpf_kfunc s32 scx_bpf_sub_caps(u64 cgroup_id, u64 caps, struct scx_cmask *out_
 
 	/*
 	 * The target's caps storage may not be set up yet (e.g. a self-read
-	 * during ops.init_cids()). Pairs with the publish in
-	 * scx_alloc_pshards(): a non-NULL pshard has every element set and the
-	 * acquire also orders the cid table reads below against it.
+	 * during ops.init_cids()); return -EBUSY instead of aborting the
+	 * scheduler. Pairs with the publish in scx_alloc_pshards(): a
+	 * non-NULL pshard has every element set and the acquire also orders
+	 * the cid table reads below against it.
 	 */
 	pshard = smp_load_acquire(&target->pshard);
-	if (unlikely(!pshard)) {
-		scx_error(sch, "scx_bpf_sub_caps() called before caps storage is initialized");
-		return -ENODEV;
-	}
+	if (unlikely(!pshard))
+		return -EBUSY;
 
 	ret = scx_cmask_ref_init(sch, out__arena, &ref);
 	if (ret) {
Signature · 3 lines
2.53.0

base-commit: 5b740a72b55c4267ba4d8010f17dde26b26f52d3
author
#7 lore raw

PATCH 4/6 sched_ext: Reject cid_online/cid_offline on sub-schedulers

In reply to Tao Cui

From: Tao Cui <[email protected]>

A sub-sched's enable path sets the has_op bits for every callback it declares, and validate_ops doesn't stop it from declaring ops.cid_online()/cid_offline(), but handle_hotplug() delivers hotplug callbacks to the root scheduler only, so those callbacks would never run while the per-cid online masks are maintained for the sub-sched regardless.

Reject the declaration at load time like the cpu-form sub_attach/ sub_detach rejection next to it, and note the root-only delivery in the ops kerneldoc. The ecaps notifier, in contrast, is delivered per sub-sched; if per-sub hotplug delivery is wanted later, this rejection is the piece to relax.

Fixes 337ec00b1d9c ("sched_ext: Implement cgroup sub-sched enabling and disabling")
---
 kernel/sched/ext/ext.c      | 9 +++++++++
 kernel/sched/ext/internal.h | 3 +++
 2 files changed, 12 insertions(+)
kernel/sched/ext/ext.c +9 -0
index 063b2964099a..792cf0c1d582 100644
@@ -8630,6 +8630,15 @@ int scx_validate_ops(struct scx_sched *sch, const struct sched_ext_ops *ops)
 		}
 	}
 
+	/*
+	 * Hotplug callbacks are delivered to the root scheduler only, so a
+	 * sub-sched's ops.cid_online()/cid_offline() would never run.
+	 */
+	if (scx_parent(sch) && (ops->cpu_online || ops->cpu_offline)) {
+		scx_error(sch, "cpu_online/cpu_offline are root-scheduler callbacks");
+		return -EINVAL;
+	}
+
 	return 0;
 }
 
kernel/sched/ext/internal.h +3 -0
index d79a48afbff0..64d87d03e112 100644
@@ -1000,6 +1000,9 @@ struct sched_ext_ops {
 	 * @cpu just came online. @cpu will not call ops.enqueue() or
 	 * ops.dispatch(), nor run tasks associated with other CPUs beforehand.
 	 * ops.sub_child_ecaps_updated() runs for @cpu only after this op.
+	 *
+	 * Root schedulers only: hotplug callbacks are delivered to the root
+	 * scheduler and are never invoked on sub-scheds.
 	 */
 	void (*cpu_online)(s32 cpu);
 
Signature · 3 lines
2.53.0

base-commit: 5b740a72b55c4267ba4d8010f17dde26b26f52d3
author
#9 lore raw

PATCH 5/6 tools/sched_ext: scx_flatcg: Don't strand a cgroup on the ENQ_RACE path

In reply to Tao Cui

From: Tao Cui <[email protected]>

cgrp_enqueued() bails out on the ENQ_RACE path when bpf_kptr_xchg() returns NULL: the node is either back on the rbtree or in the hands of try_pick_next_cgroup(), which found this cgroup empty and is deciding whether to stash it back or re-add it to the rbtree.

The bailout relies on the assumption that a task being enqueued is already visible to try_pick_next_cgroup()'s emptiness test. That doesn't hold: inserts from ops.enqueue() are deferred till after ops.enqueue() returns, so try_pick_next_cgroup() can see an empty DSQ, clear ->queued and stash the node while ->queued stays at 1 from this enqueue, leaving the cgroup off the rbtree with every subsequent enqueue skipped; the whole group strands until it is destroyed or the scheduler is reloaded, and the kernel watchdog eventually kills the scheduler with SCX_EXIT_ERROR_STALL.

Instead of bailing out, wait for the picker to resolve and retry:
grabbing the node when it's stashed back, or bailing out when the picker has re-added the cgroup to the rbtree. Exhausted retries mean the cgroup is in the common cases either going away with its node dropped or already back on the rbtree; a narrow window with the picker still holding the node in flight can strand the cgroup the same way the unpatched code does, at a rate orders of magnitude below stock.

Verified in qemu: driving ops.enqueue() races under CPU saturation strands cgroups into scheduler-killing stalls 6/10 rounds on stock, 0/10 with the change.

Fixes a4103eacc2ab ("sched_ext: Add a cgroup scheduler which uses flattened hierarchy")
---
 tools/sched_ext/scx_flatcg.bpf.c | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)
tools/sched_ext/scx_flatcg.bpf.c +15 -2
index 1d168d492203..5f2b9d0b867a 100644
@@ -285,11 +285,24 @@ static void cgrp_enqueued(struct cgroup *cgrp, struct fcg_cgrp_ctx *cgc)
 		return;
 	}
 
-	/* NULL if the node is already on the rbtree */
+	/* NULL if the node is on the rbtree or in the picker's hands */
 	cgv_node = bpf_kptr_xchg(&stash->node, NULL);
 	if (!cgv_node) {
+		/*
+		 * The node is either back on the rbtree or in the
+		 * picker's hands; bailing out strands the cgroup. Wait
+		 * for the picker to resolve and retry.
+		 */
 		stat_inc(FCG_STAT_ENQ_RACE);
-		return;
+		bpf_repeat(CGROUP_MAX_RETRIES) {
+			cgv_node = bpf_kptr_xchg(&stash->node, NULL);
+			if (cgv_node)
+				break;
+			if (scx_bpf_dsq_nr_queued(cgid) > 0)
+				break;
+		}
+		if (!cgv_node)
+			return;
 	}
 
 	bpf_spin_lock(&cgv_tree_lock);
Signature · 3 lines
2.53.0

base-commit: 5b740a72b55c4267ba4d8010f17dde26b26f52d3
author
#12 lore raw

PATCH 6/6 sched_ext: Clear the built-in idle masks on CPU hotplug transitions

In reply to Tao Cui

From: Tao Cui <[email protected]>

The built-in idle masks are only maintained by __scx_update_idle() (task driven idle transitions) and reset_idle_masks() (ops load); nothing clears them when a CPU is hotplugged. If a CPU goes offline while idle, its bit stays set for the whole offline window:
scx_bpf_pick_idle_cpu() and the other idle kfuncs keep reporting the offline CPU as an idle candidate, and the built-in select_cpu path may claim it once before the guard layers bounce the wakeup, costing a failed pick and a bumped SCX_EV_SELECT_CPU_FALLBACK each time. This is not the documented racy and self-correcting regime of the idle SMT
handling: the stale bit is a deterministic leftover, and read-only kfunc consumers never trigger the claim-time self-correction.

handle_hotplug() now drops the CPU from the built-in idle masks on both transitions like it already does for the cid-form online_cmask. scx_idle_test_and_clear_cpu() is also gated on cpu_online(): the clear alone is not sufficient, because a dying CPU keeps re-picking its idle task until it halts and each re-pick re-sets its bit, which can happen after the hotplug paths have already run. The claim path is the one choke point every idle consumer goes through, so an offline CPU is never handed out as an idle candidate regardless of any stale bits.

Verified in qemu (4 cpus) with an idle-probe scheduler probing scx_bpf_pick_idle_cpu() every dispatch: 15/15 offline/online cycles probe zero stale entries with the change. Stock is not A/B-comparable, as every hotplug transition restarts the scheduler and reinitializes the masks; the fix rests on the path above.

Fixes f0e1a0643a59 ("sched_ext: Implement BPF extensible scheduler class")
---
 kernel/sched/ext/ext.c  |  7 +++++++
 kernel/sched/ext/idle.c | 21 +++++++++++++++++++++
 kernel/sched/ext/idle.h |  1 +
 3 files changed, 29 insertions(+)
kernel/sched/ext/ext.c +7 -0
index 792cf0c1d582..a9612d768986 100644
@@ -4356,6 +4356,13 @@ static void handle_hotplug(struct rq *rq, bool online)
 	if (scx_enabled())
 		scx_idle_update_selcpu_topology(&sch->ops);
 
+	/*
+	 * Keep the built-in idle masks in sync with the hotplug state: an
+	 * offline CPU must stop being advertised as idle and a CPU coming
+	 * back must not inherit a stale idle bit.
+	 */
+	scx_idle_clear_cpu(cpu);
+
 	/*
 	 * The tables can't be retired while this function is running as the
 	 * retirement is inside cpus_read_lock. However, scx_cpu_arg() is
kernel/sched/ext/idle.c +21 -0
index 922fbc3065cc..ff0c5402886e 100644
@@ -107,6 +107,18 @@ static bool scx_idle_test_and_clear_cpu(int cpu)
 			__cpumask_clear_cpu(cpu, idle_smts);
 	}
 
+	/*
+	 * An offline CPU is never an idle candidate, but its bit can
+	 * linger past @cpu going offline: the dying CPU keeps re-picking
+	 * its idle task until it halts, which can happen after the
+	 * hotplug paths have already run. Drop the stale bit and refuse
+	 * the claim.
+	 */
+	if (unlikely(!cpu_online(cpu))) {
+		cpumask_clear_cpu(cpu, idle_cpus);
+		return false;
+	}
+
 	return cpumask_test_and_clear_cpu(cpu, idle_cpus);
 }
 
@@ -743,6 +755,15 @@ static void update_builtin_idle(int cpu, bool idle)
 	}
 }
 
+/*
+ * Drop @cpu from the built-in idle masks; used on hotplug transitions
+ * so an offline CPU stops being advertised as idle.
+ */
+void scx_idle_clear_cpu(s32 cpu)
+{
+	update_builtin_idle(cpu, false);
+}
+
 /*
  * Notify schedulers of an idle transition on @cpu's cid, delivering to every
  * sched that holds %SCX_CAP_BASE on the cid (the root holds every cap). A real
kernel/sched/ext/idle.h +1 -0
index 87a0e58f1eb7..89180abbfd21 100644
@@ -21,6 +21,7 @@ extern struct btf_id_set8 scx_kfunc_ids_select_cpu;
 
 void scx_idle_update_selcpu_topology(struct sched_ext_ops *ops);
 void scx_idle_init_masks(void);
+void scx_idle_clear_cpu(s32 cpu);
 
 s32 scx_select_cpu_dfl(struct task_struct *p, s32 prev_cpu, u64 wake_flags,
 		       const struct cpumask *cpus_allowed, u64 flags);
Signature · 3 lines
2.53.0

base-commit: 5b740a72b55c4267ba4d8010f17dde26b26f52d3
#10 lore raw

Re: [PATCH 5/6] tools/sched_ext: scx_flatcg: Don't strand a cgroup on the ENQ_RACE path

In reply to Tao Cui
Thank you for your contribution! Sashiko AI review found 2 potential issues to consider: - [Medium] sched_ext: scx_flatcg: cgroup stranding in cgrp_enqueued() retry loop - [Medium] sched_ext…

Thank you for your contribution! Sashiko AI review found 2 potential issues to consider:
- [Medium] sched_ext: scx_flatcg: cgroup stranding in cgrp_enqueued() retry loop
- [Medium] sched_ext: scx_flatcg: 1024-iteration spin in cgrp_enqueued() after PNC_RACE
--

commit 183b4752d1564aee66142be30c8948ab1d3f9a90
Author: Tao Cui <[email protected]>

tools/sched_ext: scx_flatcg: Don't strand a cgroup on the ENQ_RACE path

This commit adds a bounded retry loop in cgrp_enqueued() when bpf_kptr_xchg(&stash->node, NULL) returns NULL, polling for either the stashed cgv_node or a non-empty cgroup DSQ before returning.

Quoted 16 lines
tools/sched_ext/scx_flatcg.bpf.c +5 -1
index 1d168d492203c..5f2b9d0b867a7 100644
@@ -285,11 +285,24 @@ static void cgrp_enqueued(struct cgroup *cgrp, struct fcg_cgrp_ctx *cgc)
 		return;
 	}
 
-	/* NULL if the node is already on the rbtree */
+	/* NULL if the node is on the rbtree or in the picker's hands */
 	cgv_node = bpf_kptr_xchg(&stash->node, NULL);
 	if (!cgv_node) {
+		/*
+		 * The node is either back on the rbtree or in the
+		 * picker's hands; bailing out strands the cgroup. Wait
+		 * for the picker to resolve and retry.

tools/sched_ext/scx_flatcg.bpf.c static void cgrp_enqueued(struct cgroup *cgrp, struct fcg_cgrp_ctx *cgc) · line 303

Quoted 10 lines
tools/sched_ext/scx_flatcg.bpf.c +8 -1
+		 */
 		stat_inc(FCG_STAT_ENQ_RACE);
-		return;
+		bpf_repeat(CGROUP_MAX_RETRIES) {
+			cgv_node = bpf_kptr_xchg(&stash->node, NULL);
+			if (cgv_node)
+				break;
+			if (scx_bpf_dsq_nr_queued(cgid) > 0)
+				break;
+		}

[Severity: Medium]
Can breaking out of the retry loop on scx_bpf_dsq_nr_queued(cgid) > 0 still strand the cgroup when try_pick_next_cgroup() has already checked scx_bpf_dsq_nr_queued(cgid) or when the queued task is consumed before it does?

In try_pick_next_cgroup(), clearing cgc->queued, checking scx_bpf_dsq_nr_queued(cgid), and stashing cgv_node are not atomic:

tools/sched_ext/scx_flatcg.bpf.c:try_pick_next_cgroup() {
    ...
	__sync_val_compare_and_swap(&cgc->queued, 1, 0);

	if (scx_bpf_dsq_nr_queued(cgid)) {
		bpf_spin_lock(&cgv_tree_lock);
		bpf_rbtree_add(&cgv_tree, &cgv_node->rb_node, cgv_node_less);
		bpf_spin_unlock(&cgv_tree_lock);
		stat_inc(FCG_STAT_PNC_RACE);
	} else {
		cgv_node = bpf_kptr_xchg(&stash->node, cgv_node);
    ...
}

If try_pick_next_cgroup() observes an empty DSQ before a second enqueue completes its deferred DSQ insert, cgrp_enqueued() sees the second task in the DSQ and bails out while the picker proceeds to stash cgv_node:

CPU0 CPU1 CPU2
try_pick_next_cgroup()
cmpxchg(&cgc->queued,

          1, 0)
  dsq_nr_queued(cgid) == 0
                           fcg_enqueue(T0)
                             cgrp_enqueued()
                               cmpxchg(&cgc->queued,
                                       0, 1)
                               kptr_xchg(&stash->node,
                                         NULL) == NULL
                                                        fcg_enqueue(T1)
                                                          cgrp_enqueued()
                                                            cgc->queued == 1
                                                        <T1 inserted in DSQ>
                               dsq_nr_queued(cgid) > 0
                               return;

kptr_xchg(&stash->node,
cgv_node) <-- cgroup stranded

The same stranding happens if T1 is already in the DSQ when CPU1 checks scx_bpf_dsq_nr_queued(cgid), and another CPU with an active slice for cgid consumes T1 via scx_bpf_dsq_move_to_local() in fcg_dispatch() before CPU0 reaches its scx_bpf_dsq_nr_queued(cgid) check. In both cases cgv_node ends up in stash->node with cgc->queued left at 1 and tasks in the DSQ, so subsequent enqueues hit FCG_STAT_ENQ_SKIP and the watchdog terminates the scheduler with SCX_EXIT_ERROR_STALL.

tools/sched_ext/scx_flatcg.bpf.c static void cgrp_enqueued(struct cgroup *cgrp, struct fcg_cgrp_ctx *cgc) · line 306

Quoted 3 lines
tools/sched_ext/scx_flatcg.bpf.c +2 -0
+		if (!cgv_node)
+			return;
 	}

[Severity: Medium]
Does this loop spin through all CGROUP_MAX_RETRIES (1024) iterations whenever cgv_node is already on cgv_tree with cgc->queued == 0 and an empty DSQ?

When try_pick_next_cgroup() hits FCG_STAT_PNC_RACE, it clears cgc->queued to 0 and re-adds cgv_node to cgv_tree without setting cgc->queued back to
1. Once the queued tasks in that DSQ are consumed, cgv_node remains on cgv_tree (so stash->node is NULL), the DSQ is empty, and cgc->queued is 0. On the next fcg_enqueue() for this cgroup, cgrp_enqueued() sets cgc->queued to 1 and enters the retry loop, but because the current task's DSQ insertion is deferred until fcg_enqueue() returns, neither bpf_kptr_xchg(&stash->node, NULL) nor scx_bpf_dsq_nr_queued(cgid) > 0 can succeed, burning 1024 atomic exchanges and kfunc calls under the rq lock and widening the window for a concurrent picker to pop and stash cgv_node after the retries expire.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
#13 lore raw

Re: [PATCH 6/6] sched_ext: Clear the built-in idle masks on CPU hotplug transitions

In reply to Tao Cui
Thank you for your contribution! Sashiko AI review found 2 potential issues to consider: - [Medium] sched_ext: offline CPU re-sets idle mask bit after scx_idle_clear_cpu() - [Medium] sched_ext…

Thank you for your contribution! Sashiko AI review found 2 potential issues to consider:
- [Medium] sched_ext: offline CPU re-sets idle mask bit after scx_idle_clear_cpu()
- [Medium] sched_ext: scx_idle_test_and_clear_cpu() claims inactive CPUs during hotplug
It also noticed 1 related pre-existing issue this patch does not introduce:
- [Medium] sched_ext: data race on shared idle_smts mask in scx_idle_test_and_clear_cpu() [1]
--

commit 0e2a7b76afc726a460e91d0e19897631e59a961f
Author: Tao Cui <[email protected]>

sched_ext: Clear the built-in idle masks on CPU hotplug transitions

This commit clears a hotplugged CPU from the built-in idle masks in handle_hotplug() and rejects offline CPUs in scx_idle_test_and_clear_cpu().

Quoted 5 lines
kernel/sched/ext/ext.c +0 -0
index 7d7e2a2fc5ae8..984d9224ab59c 100644

kernel/sched/ext/ext.c static void handle_hotplug(struct rq *rq, bool online) · line 4366

Quoted 9 lines
kernel/sched/ext/ext.c +6 -0
@@ -4358,6 +4358,13 @@ static void handle_hotplug(struct rq *rq, bool online)
 	if (scx_enabled())
 		scx_idle_update_selcpu_topology(&sch->ops);
 
+	/*
+	 * Keep the built-in idle masks in sync with the hotplug state: an
+	 * offline CPU must stop being advertised as idle and a CPU coming
+	 * back must not inherit a stale idle bit.
+	 */
+	scx_idle_clear_cpu(cpu);

[Severity: Medium]
Does the dying CPU re-set its bit in idle_cpus right after scx_idle_clear_cpu() clears it?

During CPU teardown, handle_hotplug() runs at CPUHP_AP_ACTIVE, before take_cpu_down() runs at CPUHP_TEARDOWN_CPU and parks migration/N. When migration/N parks and switches the dying CPU back to rq->idle so do_idle() can halt it, set_next_task_idle() -> __scx_update_idle() calls update_builtin_idle(cpu, true), which does not check whether the CPU or runqueue is online and sets the bit in idle_cpus again.

Because normal tasks have their affinity restricted to online CPUs and BPF schedulers typically pass task affinities to scx_bpf_pick_idle_cpu(), the claim path in scx_idle_test_and_clear_cpu() is not reached for the offline CPU. Read-only consumers such as scx_bpf_get_idle_cpumask() and the !cpumask_empty(idle_cpumask(waker_node)->cpu) check in scx_select_cpu_dfl() therefore still see the offline CPU marked idle for the entire offline window. Gating idle=true updates in __scx_update_idle() or update_builtin_idle() on the CPU/rq online state would prevent the dying CPU from re-setting the bit.

kernel/sched/ext/idle.c static bool scx_idle_test_and_clear_cpu(int cpu) · line 108

Quoted 7 lines
kernel/sched/ext/idle.c +0 -0
index 922fbc3065ccf..ff0c5402886e6 100644
@@ -107,6 +107,18 @@ static bool scx_idle_test_and_clear_cpu(int cpu)
 			__cpumask_clear_cpu(cpu, idle_smts);
 	}

[Severity: Medium]
This problem wasn't introduced by this patch, but nothing serializes
[1] concurrent updates to the shared idle_smts mask in scx_idle_test_and_clear_cpu(). Both cpumask_andnot() and __cpumask_clear_cpu() perform non-atomic read-modify-write operations on the bitmap words:

CPU0 CPU1
scx_idle_test_and_clear_cpu() scx_idle_test_and_clear_cpu()
cpumask_andnot() cpumask_andnot()

    val = *word;                        val = *word;
    val &= ~smt0;                       val &= ~smt1;
    *word = val;                        *word = val;  <-- CPU0 clear lost

A lost clear leaves a core that is no longer wholly idle marked in idle_smts, causing SMT-aware idle selection to place tasks on busy sibling threads instead of idle cores.

Quoted 5 lines
kernel/sched/ext/idle.c +5 -0
+
+	/*
+	 * An offline CPU is never an idle candidate, but its bit can
+	 * linger past @cpu going offline: the dying CPU keeps re-picking
+	 * its idle task until it halts, which can happen after the

kernel/sched/ext/idle.c static bool scx_idle_test_and_clear_cpu(int cpu) · line 123

Quoted 10 lines
kernel/sched/ext/idle.c +8 -0
+	 * hotplug paths have already run. Drop the stale bit and refuse
+	 * the claim.
+	 */
+	if (unlikely(!cpu_online(cpu))) {
+		cpumask_clear_cpu(cpu, idle_cpus);
+		return false;
+	}
+
 	return cpumask_test_and_clear_cpu(cpu, idle_cpus);
 }

[Severity: Medium]
Can scx_idle_test_and_clear_cpu() still claim a CPU while it is inactive during hotplug transitions?

On teardown, sched_cpu_deactivate() clears cpu_active(cpu) and handle_hotplug() clears SCX_RQ_ONLINE at CPUHP_AP_ACTIVE, while cpu_online(cpu) stays true until __cpu_disable() runs at CPUHP_TEARDOWN_CPU. Likewise on bringup, cpu_online(cpu) is set before the CPU reaches CPUHP_AP_ACTIVE and sets cpu_active(cpu) and SCX_RQ_ONLINE. If cpuhp/N blocks during either window (for example in synchronize_rcu() or a sleepable ops.cpu_offline() / ops.cpu_online() callback), switching to rq->idle marks the CPU in idle_cpus, and scx_idle_test_and_clear_cpu() will allow claiming it because cpu_online(cpu) is true. select_task_rq() then rejects the pick via !is_cpu_allowed() (which requires cpu_active(cpu) for non-kthreads) and falls back with SCX_EV_SELECT_CPU_FALLBACK. Checking cpu_active(cpu) (or scx_rq_online(cpu_rq(cpu))) here would avoid claiming CPUs that cannot accept tasks yet.

[1] https://sashiko.dev/#/bug/linux-ca6a3761-220e-4ce4-924a-d7356268acdd

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=6
reviewer
#15 lore raw
In reply to Tao Cui

Hello, Tao.

Quoted 3 lines · Tao Cui

This series fixes six issues in the sched_ext core, the scx_flatcg example scheduler and the CID helpers. The core.c fixes travel as a separate series.

There's one actual fix here (1/6) and the rest are rather vacuous. Can you please think more through each change and whether it makes sense in the context of actual usage? If these are coming from LLM scans, please don't just follow the details that they surface. It'd help to actually use the thing that you're producing patches for and think through the implications of the changes that you're suggesting.

Thanks.

--
tejun

reviewer Applied
#2 lore raw

Re: [PATCH 1/6] sched_ext: Reject DSQ iterator kfuncs in tracing programs

In reply to Tao Cui

Applied to sched_ext/for-7.4-fixes.

Thanks.

--
tejun

reviewer
#4 lore raw

Re: [PATCH 2/6] sched_ext: Make scx_bpf_cid_node() a benign lookup for invalid cids

In reply to Tao Cui

Hello, Tao.

Quoted 3 lines · Tao Cui

Bounds-check @cid in the kfunc instead, keeping the documented benign return and leaving the aborting validation to the paths whose contracts call for it.

If anything, it's the doc that needs updating. The other cid kfuncs trigger an error on an invalid cid and this one should stay the same. Maybe update the kerneldoc to say so instead?

Thanks.

--
tejun

reviewer
#6 lore raw

Re: [PATCH 3/6] sched_ext: Return -EBUSY from a pre-init scx_bpf_sub_caps() read

In reply to Tao Cui

Hello, Tao.

Quoted 5 lines · Tao Cui

Return -EBUSY instead: the caller sees a retryable "not ready yet" and the comment's expected scenario stays survivable. This mirrors how the cid path already handles the same class of pre-init window:
reads before the tables are published test the table pointer and return -EINVAL instead of killing the scheduler.

Why is this useful? ops.init_cids() is documented to run before caps and shards are finalized, so there's nothing for a scheduler to query or retry from there.

Thanks.

--
tejun

reviewer
#8 lore raw

Re: [PATCH 4/6] sched_ext: Reject cid_online/cid_offline on sub-schedulers

In reply to Tao Cui

Hello, Tao.

Quoted 3 lines · Tao Cui

Reject the declaration at load time like the cpu-form sub_attach/ sub_detach rejection next to it, and note the root-only delivery in the ops kerneldoc.

A scheduler can be both root and sub depending on how it's loaded. I don't think it'd be necessary to require them to use a separate op table depending on how it's loaded.

Thanks.

--
tejun

reviewer
#11 lore raw

Re: [PATCH 5/6] tools/sched_ext: scx_flatcg: Don't strand a cgroup on the ENQ_RACE path

In reply to Tao Cui

Hello, Tao.

Quoted 7 lines · Tao Cui

Instead of bailing out, wait for the picker to resolve and retry:
grabbing the node when it's stashed back, or bailing out when the picker has re-added the cgroup to the rbtree. Exhausted retries mean the cgroup is in the common cases either going away with its node dropped or already back on the rbtree; a narrow window with the picker still holding the node in flight can strand the cgroup the same way the unpatched code does, at a rate orders of magnitude below stock.

This doesn't actually fix the race. I can still reproduce the stall with the patch applied.

Maybe just add a comment noting the race? scx_flatcg isn't expected to be used for actual usage anyway, and things like this can be done a lot better now using arena data structures.

Thanks.

--
tejun

reviewer
#14 lore raw

Re: [PATCH 6/6] sched_ext: Clear the built-in idle masks on CPU hotplug transitions

In reply to Tao Cui

Hello, Tao.

Quoted 6 lines · Tao Cui

offline CPU as an idle candidate, and the built-in select_cpu path may claim it once before the guard layers bounce the wakeup, costing a failed pick and a bumped SCX_EV_SELECT_CPU_FALLBACK each time. This is not the documented racy and self-correcting regime of the idle SMT
handling: the stale bit is a deterministic leftover, and read-only kfunc consumers never trigger the claim-time self-correction.

The failure mode is benign, and the patch doesn't address even what it describes. With a scheduler which implements the hotplug ops and only reads the idle mask, the offline CPU stays set in the mask for the whole offline window with the patch applied, the same as without it.

Thanks.

--
tejun

author
#16 lore raw
In reply to Tejun Heo

Hello Tejun,

Quoted 7 lines · Tejun Heo

Hello, Tao.

Quoted 3 lines · Tao Cui

This series fixes six issues in the sched_ext core, the scx_flatcg example scheduler and the CID helpers. The core.c fixes travel as a separate series.

Quoted 7 lines

There's one actual fix here (1/6) and the rest are rather vacuous. Can you please think more through each change and whether it makes sense in the context of actual usage? If these are coming from LLM scans, please don't just follow the details that they surface. It'd help to actually use the thing that you're producing patches for and think through the implications of the changes that you're suggesting.

Thanks for the feedback.

I did use LLM as a development aid. Some of the patches started with issues it identified.

I see your point. I focused too much on the issues and not enough on whether the fixes made sense in the context of actual sched_ext usage. I'll keep that in mind for future submissions.

Thanks,
Tao

Quoted 4 lines

Thanks.

--
tejun