Skip to contentSkip to conversation

sched: Make proxy execution compatible with sched_ext

PATCH Superseded Andrea Righi 3 participants 32 messages Jul 28 – Aug 6

Subsystems (4)

v10 of 14 Superseded Latest is v14 →

lore
Share & tools
lore

Andrea Righi

author reviewer
#0 lore raw

PATCH v10 1/1 sched: Make proxy execution compatible with sched_ext

This series enables using proxy execution with sched_ext and is based on early work by John Stultz [1].

Background
==========

Proxy execution (proxy-exec) lets a waiting task ("donor") donate its scheduling context to a mutex owner, so the owner can run while the donor stays eligible on the runqueue.

Currently, proxy-exec and sched_ext are mutually exclusive at build time: we can't enable CONFIG_SCHED_PROXY_EXEC=y and CONFIG_SCHED_CLASS_EXT=y in the same kernel.

This restriction can be problematic for Linux distributions and for anyone who wants to ship one kernel and choose features at runtime.

Why are they mutually exclusive?
================================

sched_ext schedulers drive dispatch through their own interfaces. A proxy-exec handoff can run a task that the BPF scheduler never dispatched through that path. sched_ext callbacks then observe a "current" task that does not match what the BPF side considers running, so kfuncs and helper state can see an inconsistent view of the executing task.

sched_ext also tracks runnable work through Dispatch Queues (DSQs) and BPF chosen dispatch rules, while the core scheduler still maintains classic per-CPU runqueues and pick paths. A proxy handoff can therefore switch the CPU to a task that the BPF scheduler never inserted or ordered through its DSQ interface.

DSQ state, vtime, and "who is running" bookkeeping inside the BPF program can then disagree with what the core actually executes, so helpers and kfuncs that assume their dispatched task is current may observe stale or inconsistent state.

Design: supporting proxy-exec with sched_ext
============================================

Provide proxy-exec in sched_ext as optional per-scheduler capability: a BPF scheduler can set the ops flag SCX_OPS_ENQ_BLOCKED to keep mutex-blocked donors runnable and receive them through ops.enqueue() with SCX_ENQ_BLOCKED set.

This flag requires an ops.enqueue() callback and gives BPF control over whether, where, and in what order to dispatch each donor. The core then walks the mutex-owner chain and, when needed, migrates the donor's scheduling context to the owner's rq before executing the owner.

Without the SCX_OPS_ENQ_BLOCKED flag, mutex waiters block normally and do not participate in proxy-exec while owned by that scheduler.

Once BPF dispatches a donor, proxy execution may move its scheduling context to the mutex owner's rq. This move is distinct from BPF placement: wake_cpu continues to identify the donor's callback CPU, while task_cpu() identifies the proxy CPU. When the donor wakes, the normal wakeup path uses wake_cpu as prev_cpu when returning the task to the BPF scheduler, which can then choose its next placement normally.

The donor-to-owner handoff is modeled like a "function call" from the scheduler's perspective. The donor remains the running scheduling entity selected by BPF: its scheduling context, runtime and slice are consumed while the core temporarily invokes the mutex owner's code to make the critical section progress. It is not a scheduler-visible switch to the owner.

Accordingly, the donor remains the scheduling context presented to sched_ext, while the mutex owner is treated solely as the execution context selected internally by the core scheduler. Scheduling state is accounted against rq->donor where appropriate, while rq->curr identifies the execution context. The internal owner substitution does not generate synthetic sched_ext callbacks for a task that BPF did not dispatch.

Scheduler-facing current-task kfuncs follow the same rule:
scx_bpf_task_running(), scx_bpf_cpu_curr() and scx_bpf_cid_curr() report the donor selected by the scheduler rather than the mutex owner whose code happens to execute on its behalf.

The sched_ext callback bookkeeping is adjusted accordingly. A blocked donor enters a tracked ops.running()/ops.stopping() session only after proxy resolution succeeds, and ops.stopping() is only called for tasks that entered such a session. This state is independent of physical rq->curr execution, which may refer to the mutex owner instead of the donor.

NOHZ CFS bandwidth checks also follow rq->donor rather than rq->curr and use the number of queued FAIR tasks when deciding whether the tick can stop. This keeps bandwidth enforcement active for a constrained FAIR donor even when its proxy owner remains runnable in another scheduling class.

Scheduler ownership changes need special handling because a donor may already be blocked when a root scheduler is enabled, when a task moves between root and sub-schedulers, or when PI de-boosting moves it into EXT. If the incoming scheduler does not set SCX_OPS_ENQ_BLOCKED, the retained donor is removed from the runqueue and stays on the normal mutex wait path. Otherwise it remains eligible for BPF admission.

Remote DSQ transfers recheck proxy-sensitive state after locking the source runqueue. If a transfer races with proxy execution, the task is parked on the source runqueue's reject DSQ and returned to its owning BPF scheduler after proxy resolution settles. This preserves scheduler placement and sub-scheduler capability rules without migrating an active execution context.

scx_qmap has been modified with a -B option to enable queueing mutex-blocked tasks for proxy-exec. Blocked donors receive a fresh slice, are inserted at the head of their current cid's local DSQ and request immediate preemption. This is an intentionally aggressive policy for making proxy-exec easy to observe. Capability-rejection re-enqueues bypass this blocked-donor fast path and fall back to the shared DSQ, avoiding repeated rejection on a cid that no longer has the required capability. A new statistic reports blocked-donor dispatches.

A new kselftest (enq_blocked) is also introduced to validate proxy-exec with sched_ext. The test creates a priority inversion with a low-priority owner (nice +19), a high-priority donor (nice -20) and one nice 0 contender per available CPU. It exercises a same-CPU topology and a cross-CPU topology, where the donor and owner run on different CPUs. Each configuration runs with SCX_OPS_ENQ_BLOCKED first disabled and then enabled, counts blocked-donor enqueues by CPU and reports mutex hold/wait-time deltas. Access to the mutex is provided by a loadable kernel module built via TEST_GEN_MODS_DIR, with the test responsible for its lifecycle.

Example kselftest run:

===== START =====
TEST: enq_blocked
DESCRIPTION: Verify proxy donor admission under CPU-wide contention OUTPUT:

[topology=same-cpu SCX_OPS_ENQ_BLOCKED=disabled]
proxy_exec=enabled
donor_cpu=0
owner_cpu=0
nr_contenders=16
measured_trials=10
owner_nice=19
donor_nice=-20
contender_nice=0 mutex_hold_avg_ns=263587544 (263.587 ms, samples=10) mutex_wait_avg_ns=263597934 (263.597 ms, samples=10) nr_blocked_enqueues=0 nr_blocked_enqueues_donor_cpu=0 nr_blocked_enqueues_owner_cpu=0 nr_blocked_enqueues_other_cpu=0 nr_blocked_wakeups=0

[topology=same-cpu SCX_OPS_ENQ_BLOCKED=enabled]
proxy_exec=enabled
donor_cpu=0
owner_cpu=0
nr_contenders=16
measured_trials=10
owner_nice=19
donor_nice=-20
contender_nice=0 mutex_hold_avg_ns=251789178 (251.789 ms, samples=10) mutex_wait_avg_ns=209808599 (209.808 ms, samples=10) nr_blocked_enqueues=60 nr_blocked_enqueues_donor_cpu=60 nr_blocked_enqueues_owner_cpu=0 nr_blocked_enqueues_other_cpu=0 nr_blocked_wakeups=0

[topology=same-cpu delta: enabled - disabled]
mutex_hold_delta_ns=-11798366 (-4.48%)
mutex_wait_delta_ns=-53789335 (-20.41%)

[topology=cross-cpu SCX_OPS_ENQ_BLOCKED=disabled]
proxy_exec=enabled
donor_cpu=0
owner_cpu=1
nr_contenders=16
measured_trials=10
owner_nice=19
donor_nice=-20
contender_nice=0 mutex_hold_avg_ns=246794454 (246.794 ms, samples=10) mutex_wait_avg_ns=247298497 (247.298 ms, samples=10) nr_blocked_enqueues=0 nr_blocked_enqueues_donor_cpu=0 nr_blocked_enqueues_owner_cpu=0 nr_blocked_enqueues_other_cpu=0 nr_blocked_wakeups=0

[topology=cross-cpu SCX_OPS_ENQ_BLOCKED=enabled]
proxy_exec=enabled
donor_cpu=0
owner_cpu=1
nr_contenders=16
measured_trials=10
owner_nice=19
donor_nice=-20
contender_nice=0 mutex_hold_avg_ns=215486809 (215.486 ms, samples=10) mutex_wait_avg_ns=215498090 (215.498 ms, samples=10) nr_blocked_enqueues=80 nr_blocked_enqueues_donor_cpu=20 nr_blocked_enqueues_owner_cpu=60 nr_blocked_enqueues_other_cpu=0 nr_blocked_wakeups=0

[topology=cross-cpu delta: enabled - disabled]
mutex_hold_delta_ns=-31307645 (-12.69%)
mutex_wait_delta_ns=-31800407 (-12.86%)
ok 1 enq_blocked #
===== END =====

=============================

RESULTS:

PASSED:  1
SKIPPED: 0
FAILED:  0

References
==========

[1] https://lore.kernel.org/all/[email protected]

Git tree: git://git.kernel.org/pub/scm/linux/kernel/git/arighi/linux.git scx-proxy-exec

Changes in v10:
- Check constrained FAIR proxy donors before the RT fast paths in sched_can_stop_tick(), so a throttled RT owner cannot bypass bandwidth enforcement (sashiko)
- Add a preparatory fix that sets DEQUEUE_CLASS only when SCHED_FLAG_KEEP_PARAMS permits the scheduling class to change, preventing class-transition callbacks from running without an actual transition (sashiko)
- Update try_to_block_task() comment to describe the sched_ext blocked-donor admission check (sashiko)
- Link to v9: https://lore.kernel.org/all/[email protected]/

Changes in v9:
- Add an incoming-class prepare_switch() callback and use it to block retained donors before transitions into sched_ext (Tejun Heo)
- Generalize the per-rq reject DSQ and use it for proxy-raced remote transfers instead of bouncing tasks to a global DSQ (Tejun Heo)
- Restrict migration-disabled warning exception to blocked donors whose scheduling context was actually proxy-migrated (sashiko)
- Keep ops.tick() and other donor accounting inside a tracked ops.running()/ops.stopping() session (sashiko)
- Prevent BPF-directed migration of an active proxy donor and use the donor as the current scheduling context during restore (sashiko)
- Update sched_fair_update_stop_tick() to use the number of queued FAIR tasks (sashiko)
- Fold the temporary local-DSQ donor placement into the final BPF-controlled admission change
- Link to v8: https://lore.kernel.org/all/[email protected]/

Changes in v8:
- Rework remote-DSQ migration check so the locked re-check handles only proxy-exec state changes (Tejun Heo)
- Use the current-donor helper when preventing migration of an active donor (sashiko)
- Handle blocked-donor reactivation into scx schedulers without SCX_OPS_ENQ_BLOCKED (John Stultz)
- Document sched_ext callback behavior under proxy execution (sashiko)
- Avoid redispatching a rejected blocked donor to the same local DSQ in scx_qmap (sashiko)
- Add a preparatory fix to avoid false migration warnings when proxy-exec moves a migration-disabled donor
- Skip the enq_blocked kselftest when proxy execution is unavailable
- Link to v7: https://lore.kernel.org/all/[email protected]/

Changes in v7:
- Move the remote CPU check out of the lockless remote-DSQ scan and perform it only after locking the source rq, retain the locked migration recheck immediately before moving the task (sashiko)
- Complete the scheduler-context conversion from rq->curr to rq->donor for capability revocation and preemption checks, make scx_bpf_task_running(), scx_bpf_cpu_curr() and scx_bpf_cid_curr() expose the donor selected by the scheduler (sashiko)
- Handle SCX_ENQ_REENQ before scx_qmap's blocked-donor fast path to avoid an infinite reject/re-enqueue loop (sashiko)
- Add scx_qmap stat for SCX_ENQ_BLOCKED dispatches
- Link to v6: https://lore.kernel.org/all/[email protected]/

Changes in v6:
- Add a scheduler-core fix to make NOHZ CFS bandwidth checks follow the proxy donor instead of the physical execution context (sashiko)
- Block retained donors when entering sched_ext through PI de-boosting or global scheduler activation (sashiko)
- Distinguish ordinary wakeup activations from retained-donor admissions so SCX_ENQ_BLOCKED is not reported for normal wakeups; extend the selftest to detect this regression (sashiko)
- Validate unexpected-CPU blocked-donor enqueues in both same-CPU and cross-CPU test topologies (sashiko)
- Link to v5: https://lore.kernel.org/all/[email protected]/

Changes in v5:
- Split retained-donor deactivation and sched_ext's default rejection into preparatory patches so the scheduler-core changes can be routed separately
- Drop the proxy destination query kfuncs and the preparatory mutex lock-scope change (John Stultz)
- Use p->is_blocked instead of task_is_blocked() to fix a WARN triggered during scx_pair testing (John Stultz)
- Rename SCX_TASK_IS_RUNNING to SCX_TASK_RUN_TRACKED and document that it tracks an ops.running()/stopping() session rather than physical rq->curr execution
- Keep a proxy-migrated blocked donor on the owner's rq until wakeup instead of allowing BPF-directed migration to pull it back to wake_cpu and cause repeated donor migration
- Extend kselftest to cover same-CPU and cross-CPU proxy-exec switches
- Make scx_qmap insert blocked donors at the head of their current cid's local DSQ and request immediate preemption
- Link to v4: https://lore.kernel.org/all/[email protected]/

Changes in v4:
- Harden scx_bpf_task_proxy_cpu() locking and blocked-state validation, return -ENOENT for unrunnable owners and drop unnecessary donor-affinity checks (K Prateek Nayak, sashiko)
- Reschedule remote CPUs after donor deactivation, handle sched_setscheduler() admission and assert scheduler-change locking (sashiko)
- Avoid a potential KCSAN data-race report in the lockless blocked-donor migration check by using rcu_access_pointer() for rq->donor (sashiko)
- Fix tick dependency updates for incoming EXT contexts and keep the tick enabled for blocked donors (sashiko)
- Dump EXT donors in scx_dump_state() (sashiko)
- Reordered the preparatory sched_ext changes so real running-state tracking is established before donor-based accounting, and split the proxy destination query kfuncs into a separate patch
- Add compatibility wrappers for the proxy CPU/cid kfuncs and document their results as scheduling hints
- Link to v3: https://lore.kernel.org/all/[email protected]/

Changes in v3:
- Dropped the core restrictions on proxy-migrating migration-disabled and single-CPU donors: proxy execution moves the scheduling context, not the task's execution context. (Peter Zijlstra, K Prateek Nayak)
- Dropped the sched_ext-specific put_prev_task()/set_next_task() exception and fixed ops.running()/ops.stopping() pairing inside sched_ext instead; track a real running transition even when either callback is absent. (Peter Zijlstra)
- Dropped the kf_tasks[] nesting and nested ops.runnable() patches from v2; extensive proxy-exec testing did not reproduce task-op re-entry, so retain the existing non-nesting invariant.
- Replaced scx_bpf_task_is_blocked() with SCX_ENQ_BLOCKED in ops.enqueue() flags, identifying blocked-donor admission directly at enqueue time.
- Expanded the curr/donor description to cover mixed scheduling classes and fixed local preemption to expire rq->donor's slice. (Aiqun Maria Yu)
- Allow inactive blocked donors to be placed on a remote local DSQ while preventing normal migration of an active rq donor. (Aiqun Maria Yu)
- Deactivate retained donors when ownership changes to a root or sub-scheduler without SCX_OPS_ENQ_BLOCKED, and extend the selftest to cover attaching a scheduler after the donor blocks. (K Prateek Nayak)
- Harden remote DSQ consumption by rejecting active tasks and rechecking migration eligibility after locking the source rq. Fall back to the global DSQ without treating an eligibility change during the lock handoff as a BPF scheduler error.
- scx_qmap has a command line option (-B) to enable blocked-donor queueing
- Link to v2: https://lore.kernel.org/all/[email protected]/

Changes in v2:
- Rebased onto sched_ext/for-7.3 and adapted the series to the split sched_ext implementation and cid-form scheduler interfaces.
- Replaced the global sched_proxy_exec_scx boot-time opt-in with the per-scheduler SCX_OPS_ENQ_BLOCKED capability, allowing BPF to control donor admission and ordering through ops.enqueue().
- Added scx_bpf_task_is_blocked(), scx_bpf_task_proxy_cpu(), and scx_bpf_task_proxy_cid(); enforce CPU/cid API separation for cid-form schedulers.
- Added proxy exec support to scx_qmap, including optional owner-cid steering, affinity validation, and fallback to the donor's current cid.
- Added a kselftest with a kernel mutex test and a three-task priority inversion workload, test is executed with blocked task admission disabled and enabled, validates the behavior, and reports hold/wait-time deltas.
- Link to v1: https://lore.kernel.org/all/[email protected]/

Andrea Righi (15):
sched/core: Avoid false migration warning for proxy donors
sched: Make NOHZ CFS bandwidth checks follow proxy donor
sched: Add helper to block retained proxy donors
sched: Skip class callbacks with SCHED_FLAG_KEEP_PARAMS
sched: Add prepare_switch() class callback
sched: Add sched_ext hooks for proxy execution
sched_ext: Block proxy donors across scheduler transitions
sched_ext: Fix ops.running/stopping() pairing for proxy-exec donors
sched_ext: Generalize the reject DSQ reenqueue path
sched_ext: Handle proxy-exec races in remote DSQ transfers
sched_ext: Split curr|donor references properly
sched_ext: Delegate proxy donor admission to BPF schedulers
sched_ext: Add selftest for blocked donor admission
sched_ext: scx_qmap: Add proxy execution support
sched: Allow enabling proxy exec with sched_ext

 Documentation/scheduler/sched-ext.rst              |   6 +
 include/linux/sched/ext.h                          |   8 +
 init/Kconfig                                       |   2 -
 kernel/sched/core.c                                |  90 +-
 kernel/sched/ext/ext.c                             | 554 +++++++++++--
 kernel/sched/ext/ext.h                             |   6 +
 kernel/sched/ext/internal.h                        |  29 +-
 kernel/sched/ext/sub.c                             |  58 +-
 kernel/sched/ext/sub.h                             |  10 +-
 kernel/sched/fair.c                                |  12 +-
 kernel/sched/sched.h                               |  21 +-
 kernel/sched/syscalls.c                            |   7 +-
 tools/sched_ext/include/scx/compat.h               |   1 +
 tools/sched_ext/include/scx/enum_defs.autogen.h    |   1 +
 tools/sched_ext/include/scx/enums.autogen.bpf.h    |   3 +
 tools/sched_ext/include/scx/enums.autogen.h        |   1 +
 tools/sched_ext/scx_qmap.bpf.c                     |  34 +
 tools/sched_ext/scx_qmap.c                         |  13 +-
 tools/sched_ext/scx_qmap.h                         |   1 +
 tools/testing/selftests/sched_ext/.gitignore       |   4 +
 tools/testing/selftests/sched_ext/Makefile         |   2 +
 tools/testing/selftests/sched_ext/config           |   2 +
 .../testing/selftests/sched_ext/enq_blocked.bpf.c  | 116 +++
 tools/testing/selftests/sched_ext/enq_blocked.c    | 918 +++++++++++++++++++++
 tools/testing/selftests/sched_ext/enq_blocked.h    |  28 +
 .../selftests/sched_ext/test_modules/Makefile      |  13 +
 .../sched_ext/test_modules/scx_enq_blocked_test.c  | 195 +++++
 27 files changed, 1948 insertions(+), 187 deletions(-)
 create mode 100644 tools/testing/selftests/sched_ext/enq_blocked.bpf.c
 create mode 100644 tools/testing/selftests/sched_ext/enq_blocked.c
 create mode 100644 tools/testing/selftests/sched_ext/enq_blocked.h
 create mode 100644 tools/testing/selftests/sched_ext/test_modules/Makefile
 create mode 100644 tools/testing/selftests/sched_ext/test_modules/scx_enq_blocked_test.c
author reviewer
#1 lore raw

PATCH 1/15 sched/core: Avoid false migration warning for proxy donors

Proxy execution can move a blocked donor's scheduling context to the lock owner's CPU even when the donor is migration-disabled. The donor does not execute there, and its original execution CPU remains recorded in wake_cpu.

set_task_cpu() warns unconditionally for migration-disabled tasks, so a subsequent proxy migration or the wakeup path returning the donor home triggers a false positive. Moving a blocked scheduling context does not violate the migration-disabled execution context.

Exclude blocked proxy donors from the warning. The proxy wakeup path restores an executable placement before clearing the blocked state.

This is a preparatory fix for enabling proxy execution together with sched_ext.

---
 kernel/sched/core.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
kernel/sched/core.c +8 -1
index 3cc6fb1d20547..e11fac7f805e7 100644
@@ -3341,6 +3341,8 @@ void relax_compatible_cpus_allowed_ptr(struct task_struct *p)
 void set_task_cpu(struct task_struct *p, unsigned int new_cpu)
 {
 	unsigned int state = READ_ONCE(p->__state);
+	bool proxy_migrated = sched_proxy_exec() && p->is_blocked &&
+			      task_cpu(p) != p->wake_cpu;
 
 	/*
 	 * We should never call set_task_cpu() on a blocked task,
@@ -3376,7 +3378,12 @@ void set_task_cpu(struct task_struct *p, unsigned int new_cpu)
 	 */
 	WARN_ON_ONCE(!cpu_online(new_cpu));
 
-	WARN_ON_ONCE(is_migration_disabled(p));
+	/*
+	 * Proxy execution can move a blocked task's scheduling context to any
+	 * CPU without moving its migration-disabled execution context. The
+	 * wakeup path will return the task to a CPU where it can execute.
+	 */
+	WARN_ON_ONCE(is_migration_disabled(p) && !proxy_migrated);
 
 	trace_sched_migrate_task(p, new_cpu);
 
Signature · 1 line
2.55.0
author reviewer
#2 lore raw

PATCH 2/15 sched: Make NOHZ CFS bandwidth checks follow proxy donor

In reply to Andrea Righi

Proxy execution separates the scheduling context in rq->donor from the physical execution context in rq->curr. sched_can_stop_tick() checks the latter for CFS bandwidth constraints and only does so when nr_running is one.

A retained proxy donor keeps both the donor and mutex owner queued. The check therefore misses a constrained FAIR donor and may stop the tick while its runtime still needs to be enforced.

Check the selected donor instead and remove the nr_running restriction. The donor being a queued FAIR task is sufficient to require bandwidth accounting regardless of other runnable tasks.

Fixes af0c8b2bf67b ("sched: Split scheduler and execution contexts")
---
 kernel/sched/core.c | 25 +++++++++----------------
 kernel/sched/fair.c | 12 +++++++-----
 2 files changed, 16 insertions(+), 21 deletions(-)
kernel/sched/core.c +9 -16
index e11fac7f805e7..96b8b01d43101 100644
@@ -1409,11 +1409,8 @@ static void nohz_csd_func(void *info)
 #endif /* CONFIG_NO_HZ_COMMON */
 
 #ifdef CONFIG_NO_HZ_FULL
-static inline bool __need_bw_check(struct rq *rq, struct task_struct *p)
+static inline bool __need_bw_check(struct task_struct *p)
 {
-	if (rq->nr_running != 1)
-		return false;
-
 	if (p->sched_class != &fair_sched_class)
 		return false;
 
@@ -1431,6 +1428,14 @@ bool sched_can_stop_tick(struct rq *rq)
 	if (rq->dl.dl_nr_running)
 		return false;
 
+	/*
+	 * The selected scheduling context can be a constrained FAIR donor even
+	 * when rq->curr is an RT task. Check it before the RT fast paths below,
+	 * which may report that the tick can stop for a throttled RT context.
+	 */
+	if (__need_bw_check(rq->donor) && cfs_task_bw_constrained(rq->donor))
+		return false;
+
 	/*
 	 * If there are more than one RR tasks, we need the tick to affect the
 	 * actual RR behaviour.
@@ -1461,18 +1466,6 @@ bool sched_can_stop_tick(struct rq *rq)
 	if (rq->cfs.h_nr_queued > 1)
 		return false;
 
-	/*
-	 * If there is one task and it has CFS runtime bandwidth constraints
-	 * and it's on the cpu now we don't want to stop the tick.
-	 * This check prevents clearing the bit if a newly enqueued task here is
-	 * dequeued by migrating while the constrained task continues to run.
-	 * E.g. going from 2->1 without going through pick_next_task().
-	 */
-	if (__need_bw_check(rq, rq->curr)) {
-		if (cfs_task_bw_constrained(rq->curr))
-			return false;
-	}
-
 	return true;
 }
 #endif /* CONFIG_NO_HZ_FULL */
kernel/sched/fair.c +7 -5
index d78467ec6ee13..14c5b2aa01777 100644
@@ -7570,14 +7570,16 @@ static void sched_fair_update_stop_tick(struct rq *rq, struct task_struct *p)
 	if (!tick_nohz_full_cpu(cpu))
 		return;
 
-	if (rq->nr_running != 1)
+	if (rq->cfs.h_nr_queued != 1)
 		return;
 
 	/*
-	 *  We know there is only one task runnable and we've just picked it. The
-	 *  normal enqueue path will have cleared TICK_DEP_BIT_SCHED if we will
-	 *  be otherwise able to stop the tick. Just need to check if we are using
-	 *  bandwidth control.
+	 * We know there is only one FAIR task queued and we've just picked it.
+	 * The normal enqueue path will have cleared TICK_DEP_BIT_SCHED if we
+	 * will be otherwise able to stop the tick. With proxy execution, its
+	 * lock owner may remain runnable in another scheduling class, so
+	 * rq->nr_running can be greater than one. Just need to check if we are
+	 * using bandwidth control.
 	 */
 	if (cfs_task_bw_constrained(p))
 		tick_nohz_dep_set_cpu(cpu, TICK_DEP_BIT_SCHED);
Signature · 1 line
2.55.0
author reviewer
#3 lore raw

PATCH 3/15 sched: Add helper to block retained proxy donors

In reply to Andrea Righi

Scheduler ownership changes may need to turn a mutex-blocked task retained on the runqueue for proxy execution back into a normally blocked task.

Add sched_proxy_block_task() to perform that transition while holding p->pi_lock and the task's runqueue lock. Reset an active donor to the physical current task before blocking it. If DELAY_DEQUEUE keeps a FAIR donor queued, the immediately following sched_change_begin() completes the dequeue through switching_from_fair() before recording its queued state.

This is a preparatory change to support proxy execution with sched_ext.

---
 kernel/sched/core.c  | 28 ++++++++++++++++++++++++++++
 kernel/sched/sched.h |  6 ++++++
 2 files changed, 34 insertions(+)
kernel/sched/core.c +28 -0
index 96b8b01d43101..2cc6c9d16200c 100644
@@ -6764,6 +6764,34 @@ static void proxy_deactivate(struct rq *rq, struct task_struct *donor)
 	block_task(rq, donor, state);
 }
 
+/*
+ * Remove a retained proxy donor before changing its scheduler ownership.
+ * The caller holds p->pi_lock, so p cannot wake and migrate if block_task()
+ * drops it from the runqueue. If DELAY_DEQUEUE keeps a blocked fair task
+ * queued, switching_from_fair() completes the dequeue in the immediately
+ * following sched_change_begin().
+ */
+void sched_proxy_block_task(struct rq *rq, struct task_struct *p)
+{
+	unsigned long state = READ_ONCE(p->__state);
+
+	lockdep_assert_held(&p->pi_lock);
+	lockdep_assert_rq_held(rq);
+
+	if (!p->is_blocked || !task_on_rq_queued(p))
+		return;
+	if (WARN_ON_ONCE(state == TASK_RUNNING))
+		return;
+
+	if (task_current_donor(rq, p))
+		proxy_reset_donor(rq);
+
+	if (!p->se.sched_delayed)
+		block_task(rq, p, state);
+
+	WARN_ON_ONCE(task_on_rq_queued(p) && !p->se.sched_delayed);
+}
+
 static inline void proxy_release_rq_lock(struct rq *rq, struct rq_flags *rf)
 	__releases(__rq_lockp(rq))
 {
kernel/sched/sched.h +6 -0
index c0cb879d75f03..6119bfd2de60d 100644
@@ -2476,6 +2476,12 @@ static inline bool task_is_blocked(struct task_struct *p)
 	return !!p->blocked_on;
 }
 
+#ifdef CONFIG_SCHED_PROXY_EXEC
+void sched_proxy_block_task(struct rq *rq, struct task_struct *p);
+#else
+static inline void sched_proxy_block_task(struct rq *rq, struct task_struct *p) {}
+#endif
+
 static inline int task_on_cpu(struct rq *rq, struct task_struct *p)
 {
 	return p->on_cpu;
Signature · 1 line
2.55.0
author reviewer
#4 lore raw

PATCH 4/15 sched: Skip class callbacks with SCHED_FLAG_KEEP_PARAMS

In reply to Andrea Righi

SCHED_FLAG_KEEP_PARAMS suppresses scheduler parameter and class changes, but __sched_setscheduler() can still set DEQUEUE_CLASS when the class computed from the requested policy differs from the task's current class.

This causes the switching_from(), switched_from(), switching_to() and switched_to() callbacks to run even though p->sched_class remains unchanged.

Set DEQUEUE_CLASS only when SCHED_FLAG_KEEP_PARAMS allows the class to change. This also ensures that incoming-class preparation added by a later change is only performed for an actual class transition.

Fixes 637b0682821b ("sched: Fold sched_class::switch{ing,ed}_{to,from}() into the change pattern")
---
 kernel/sched/syscalls.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
kernel/sched/syscalls.c +2 -1
index b215b0ead9a60..903b47f5d0b74 100644
@@ -675,7 +675,8 @@ int __sched_setscheduler(struct task_struct *p,
 	prev_class = p->sched_class;
 	next_class = __setscheduler_class(policy, newprio);
 
-	if (prev_class != next_class)
+	if (!(attr->sched_flags & SCHED_FLAG_KEEP_PARAMS) &&
+	    prev_class != next_class)
 		queue_flags |= DEQUEUE_CLASS;
 
 	scoped_guard (sched_change, p, queue_flags) {
Signature · 1 line
2.55.0
#5 lore raw

Re: [PATCH 04/15] sched: Skip class callbacks with SCHED_FLAG_KEEP_PARAMS

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] Using SCHED_FLAG_KEEP_PARAMS with a new SCHED_DEADLINE policy on a non-deadline task causes a permanent deadline bandwidth leak in the root domain. --

--- Patch [4]: [PATCH 04/15] sched: Skip class callbacks with SCHED_FLAG_KEEP_PARAMS --- commit 7456fab35f0046cf8daada3381577cde13d7dbe1
Author: Andrea Righi <[email protected]>

sched: Skip class callbacks with SCHED_FLAG_KEEP_PARAMS

SCHED_FLAG_KEEP_PARAMS suppresses scheduler parameter and class changes, but __sched_setscheduler() can still set DEQUEUE_CLASS when the class computed from the requested policy differs from the task's current class. This commit sets DEQUEUE_CLASS only when SCHED_FLAG_KEEP_PARAMS allows the class to change.

Quoted 5 lines
kernel/sched/syscalls.c +0 -0
index b215b0ead9a60..903b47f5d0b74 100644

kernel/sched/syscalls.c int __sched_setscheduler(struct task_struct *p, · line 682

Quoted 9 lines
kernel/sched/syscalls.c +2 -1
@@ -675,7 +675,8 @@ int __sched_setscheduler(struct task_struct *p,
 	prev_class = p->sched_class;
 	next_class = __setscheduler_class(policy, newprio);
 
-	if (prev_class != next_class)
+	if (!(attr->sched_flags & SCHED_FLAG_KEEP_PARAMS) &&
+	    prev_class != next_class)
 		queue_flags |= DEQUEUE_CLASS;
 
 	scoped_guard (sched_change, p, queue_flags) {

[Severity: High]
This is a pre-existing issue and was not introduced by the patch under review, but does using SCHED_FLAG_KEEP_PARAMS with a new SCHED_DEADLINE policy on a non-deadline task cause a permanent deadline bandwidth leak in the root domain?

When a user with CAP_SYS_NICE calls sched_setattr() with attr.sched_policy = SCHED_DEADLINE and SCHED_FLAG_KEEP_PARAMS on a non-deadline task, sys_sched_setattr() fetches the current parameters:

sys_sched_setattr()
if (attr.sched_flags & SCHED_FLAG_KEEP_PARAMS)
get_params(p, &attr, 0);

This can replace attr.sched_runtime with the task's current fair slice while leaving attr.sched_period user-provided, constructing a valid deadline ratio.

Earlier in __sched_setscheduler(), this requested bandwidth is validated and permanently added to the global root domain accounting via __dl_add() inside sched_dl_overflow():

    if ((dl_policy(policy) || dl_task(p)) && sched_dl_overflow(p, policy, attr)) {

However, because SCHED_FLAG_KEEP_PARAMS is set, the parameter and class transition is skipped further down inside the scoped_guard block:

    if (!(attr->sched_flags & SCHED_FLAG_KEEP_PARAMS)) {
        __setscheduler_params(p, attr);
        p->sched_class = next_class;
        p->prio = newprio;
        __setscheduler_dl_pi(newprio, policy, p, scope);
    }

Since the task remains non-deadline and p->dl.dl_bw remains 0, when the task eventually exits, it will subtract its dl_bw (0) instead of the allocated bandwidth, leaving the globally added bandwidth permanently leaked.

Is there a way to prevent sched_dl_overflow() from allocating bandwidth if we know SCHED_FLAG_KEEP_PARAMS will skip the parameter updates?

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
author reviewer
#6 lore raw

PATCH 5/15 sched: Add prepare_switch() class callback

In reply to Andrea Righi

sched_change_begin() only invokes callbacks on the outgoing class before recording the task queueing state. An incoming class may need to prepare a task while that state is still live, before the normal dequeue and class transition take place.

Pass the incoming class to sched_change_begin() and invoke its optional prepare_switch() callback before recording the queued and running state.

This is a preparatory change to support proxy execution with sched_ext.

---
 kernel/sched/core.c     | 16 +++++++++++-----
 kernel/sched/ext/ext.c  |  7 ++++---
 kernel/sched/ext/sub.c  |  9 ++++++---
 kernel/sched/sched.h    | 10 +++++++---
 kernel/sched/syscalls.c |  4 ++--
 5 files changed, 30 insertions(+), 16 deletions(-)
kernel/sched/core.c +11 -5
index 2cc6c9d16200c..cff007d8b5e26 100644
@@ -2785,7 +2785,7 @@ void set_cpus_allowed_common(struct task_struct *p, struct affinity_context *ctx
 static void
 do_set_cpus_allowed(struct task_struct *p, struct affinity_context *ctx)
 {
-	scoped_guard (sched_change, p, DEQUEUE_SAVE)
+	scoped_guard (sched_change, p, p->sched_class, DEQUEUE_SAVE)
 		p->sched_class->set_cpus_allowed(p, ctx);
 }
 
@@ -7715,7 +7715,7 @@ void rt_mutex_setprio(struct task_struct *p, struct task_struct *pi_task)
 	if (prev_class != next_class)
 		queue_flag |= DEQUEUE_CLASS;
 
-	scoped_guard (sched_change, p, queue_flag) {
+	scoped_guard (sched_change, p, next_class, queue_flag) {
 		/*
 		 * Boosting condition are:
 		 * 1. -rt task is running and holds mutex A
@@ -8392,7 +8392,7 @@ int migrate_task_to(struct task_struct *p, int target_cpu)
 void sched_setnuma(struct task_struct *p, int nid)
 {
 	guard(task_rq_lock)(p);
-	scoped_guard (sched_change, p, DEQUEUE_SAVE)
+	scoped_guard (sched_change, p, p->sched_class, DEQUEUE_SAVE)
 		p->numa_preferred_nid = nid;
 }
 #endif /* CONFIG_NUMA_BALANCING */
@@ -9509,7 +9509,7 @@ void sched_move_task(struct task_struct *tsk, bool for_autogroup)
 	CLASS(task_rq_lock, rq_guard)(tsk);
 	rq = rq_guard.rq;
 
-	scoped_guard (sched_change, tsk, queue_flags) {
+	scoped_guard (sched_change, tsk, tsk->sched_class, queue_flags) {
 		sched_change_group(tsk);
 		if (!for_autogroup)
 			scx_cgroup_move_task(tsk);
@@ -11213,7 +11213,9 @@ static inline void sched_mm_cid_fork(struct task_struct *t) { }
 
 static DEFINE_PER_CPU(struct sched_change_ctx, sched_change_ctx);
 
-struct sched_change_ctx *sched_change_begin(struct task_struct *p, unsigned int flags)
+struct sched_change_ctx *
+sched_change_begin(struct task_struct *p, const struct sched_class *next_class,
+		   unsigned int flags)
 {
 	struct sched_change_ctx *ctx = this_cpu_ptr(&sched_change_ctx);
 	struct rq *rq = task_rq(p);
@@ -11231,6 +11233,10 @@ struct sched_change_ctx *sched_change_begin(struct task_struct *p, unsigned int
 		flags |= DEQUEUE_NOCLOCK;
 	}
 
+	if ((flags & DEQUEUE_CLASS) && next_class != p->sched_class &&
+	    next_class->prepare_switch)
+		next_class->prepare_switch(rq, p);
+
 	if ((flags & DEQUEUE_CLASS) && p->sched_class->switching_from)
 		p->sched_class->switching_from(rq, p);
 
kernel/sched/ext/ext.c +4 -3
index 312938179e524..69ab59d176ce0 100644
@@ -5861,7 +5861,8 @@ void scx_bypass(struct scx_sched *sch, bool bypass)
 				continue;
 
 			/* cycling deq/enq is enough, see the function comment */
-			scoped_guard (sched_change, p, DEQUEUE_SAVE | DEQUEUE_MOVE) {
+			scoped_guard (sched_change, p, p->sched_class,
+				      DEQUEUE_SAVE | DEQUEUE_MOVE) {
 				/* nothing */ ;
 			}
 		}
@@ -6140,7 +6141,7 @@ static void scx_root_disable(struct scx_sched *sch)
 		if (old_class != new_class)
 			queue_flags |= DEQUEUE_CLASS;
 
-		scoped_guard (sched_change, p, queue_flags) {
+		scoped_guard (sched_change, p, new_class, queue_flags) {
 			p->sched_class = new_class;
 		}
 
@@ -7508,7 +7509,7 @@ static void scx_root_enable_workfn(struct kthread_work *work)
 		if (old_class != new_class)
 			queue_flags |= DEQUEUE_CLASS;
 
-		scoped_guard (sched_change, p, queue_flags) {
+		scoped_guard (sched_change, p, new_class, queue_flags) {
 			set_task_slice(p, READ_ONCE(sch->slice_dfl));
 			p->sched_class = new_class;
 		}
kernel/sched/ext/sub.c +6 -3
index daf7fbd3d0c3b..8758317035754 100644
@@ -794,7 +794,8 @@ static void scx_rehome_task(struct scx_sched *to, struct task_struct *p)
 	lockdep_assert_held(&p->pi_lock);
 	lockdep_assert_rq_held(task_rq(p));
 
-	scoped_guard (sched_change, p, DEQUEUE_SAVE | DEQUEUE_MOVE) {
+	scoped_guard (sched_change, p, p->sched_class,
+		      DEQUEUE_SAVE | DEQUEUE_MOVE) {
 		scx_disable_and_exit_task(scx_task_sched(p), p);
 		scx_set_task_state(p, SCX_TASK_INIT_BEGIN);
 		scx_set_task_state(p, SCX_TASK_INIT);
@@ -824,7 +825,8 @@ static void scx_punt_task(struct scx_sched *to, struct task_struct *p)
 	lockdep_assert_rq_held(task_rq(p));
 	WARN_ON_ONCE(!READ_ONCE(to->bypass_depth));
 
-	scoped_guard (sched_change, p, DEQUEUE_SAVE | DEQUEUE_MOVE) {
+	scoped_guard (sched_change, p, p->sched_class,
+		      DEQUEUE_SAVE | DEQUEUE_MOVE) {
 		scx_disable_and_exit_task(scx_task_sched(p), p);
 		scx_set_task_sched(p, to);
 	}
@@ -1476,7 +1478,8 @@ void scx_sub_enable_workfn(struct kthread_work *work)
 		if (!(p->scx.flags & SCX_TASK_SUB_INIT))
 			continue;
 
-		scoped_guard (sched_change, p, DEQUEUE_SAVE | DEQUEUE_MOVE) {
+		scoped_guard (sched_change, p, p->sched_class,
+			      DEQUEUE_SAVE | DEQUEUE_MOVE) {
 			/*
 			 * $p must be either READY or ENABLED. If ENABLED,
 			 * __scx_disabled_and_exit_task() first disables and
kernel/sched/sched.h +7 -3
index 6119bfd2de60d..79daacc2389e1 100644
@@ -2704,6 +2704,7 @@ struct sched_class {
 	/*
 	 * sched_change
 	 */
+	void (*prepare_switch)(struct rq *this_rq, struct task_struct *task);
 	void (*switching_from)(struct rq *this_rq, struct task_struct *task);
 	void (*switched_from) (struct rq *this_rq, struct task_struct *task);
 	void (*switching_to)  (struct rq *this_rq, struct task_struct *task);
@@ -4213,13 +4214,16 @@ struct sched_change_ctx {
 	bool			running;
 };
 
-struct sched_change_ctx *sched_change_begin(struct task_struct *p, unsigned int flags);
+struct sched_change_ctx *
+sched_change_begin(struct task_struct *p, const struct sched_class *next_class,
+		   unsigned int flags);
 void sched_change_end(struct sched_change_ctx *ctx);
 
 DEFINE_CLASS(sched_change, struct sched_change_ctx *,
 	     sched_change_end(_T),
-	     sched_change_begin(p, flags),
-	     struct task_struct *p, unsigned int flags)
+	     sched_change_begin(p, next_class, flags),
+	     struct task_struct *p, const struct sched_class *next_class,
+	     unsigned int flags)
 
 DEFINE_CLASS_IS_UNCONDITIONAL(sched_change)
 
kernel/sched/syscalls.c +2 -2
index 903b47f5d0b74..9ab23f765f7a7 100644
@@ -85,7 +85,7 @@ void set_user_nice(struct task_struct *p, long nice)
 		return;
 	}
 
-	scoped_guard (sched_change, p, DEQUEUE_SAVE) {
+	scoped_guard (sched_change, p, p->sched_class, DEQUEUE_SAVE) {
 		p->static_prio = NICE_TO_PRIO(nice);
 		set_load_weight(p, true);
 		old_prio = p->prio;
@@ -679,7 +679,7 @@ int __sched_setscheduler(struct task_struct *p,
 	    prev_class != next_class)
 		queue_flags |= DEQUEUE_CLASS;
 
-	scoped_guard (sched_change, p, queue_flags) {
+	scoped_guard (sched_change, p, next_class, queue_flags) {
 
 		if (!(attr->sched_flags & SCHED_FLAG_KEEP_PARAMS)) {
 			__setscheduler_params(p, attr);
Signature · 1 line
2.55.0
author reviewer
#7 lore raw

PATCH 6/15 sched: Add sched_ext hooks for proxy execution

In reply to Andrea Righi

Proxy execution splits the scheduling context (the donor) from the execution context (the lock owner). sched_ext needs to observe that split at three points in __schedule():

- whether a blocked EXT task can be retained on the runqueue as a donor,
- when a donor's scheduling context starts driving a lock owner,
- once proxy resolution has settled for the current pick.

Introduce scx_allow_proxy_exec(), scx_proxy_donor_start() and scx_proxy_resolved(), and add their call sites in __schedule(). The implementations are empty here and are filled in by the sched_ext changes that follow, so that all the sched core changes needed by proxy execution stay together in the preparatory patches.

SCHED_PROXY_EXEC still depends on !SCHED_CLASS_EXT, so the new hooks are
inert: they are compiled out with CONFIG_SCHED_CLASS_EXT=n and unreachable otherwise.

This is a preparatory change to support proxy execution with sched_ext. No functional change.

---
 kernel/sched/core.c    | 12 +++++++-----
 kernel/sched/ext/ext.c | 13 +++++++++++++
 kernel/sched/ext/ext.h |  6 ++++++
 3 files changed, 26 insertions(+), 5 deletions(-)
kernel/sched/core.c +7 -5
index cff007d8b5e26..7ade08aa09dbc 100644
@@ -7156,13 +7156,12 @@ static void __sched notrace __schedule(int sched_mode)
 		}
 	} else if (!preempt && prev_state) {
 		/*
-		 * We pass task_is_blocked() as the should_block arg
-		 * in order to keep mutex-blocked tasks on the runqueue
-		 * for slection with proxy-exec (without proxy-exec
-		 * task_is_blocked() will always be false).
+		 * Keep mutex-blocked tasks on the runqueue for proxy execution
+		 * only when their scheduling class allows it. Without proxy
+		 * execution, task_is_blocked() always returns false.
 		 */
 		try_to_block_task(rq, prev, &prev_state,
-				  !task_is_blocked(prev));
+				  !task_is_blocked(prev) || !scx_allow_proxy_exec(prev));
 		switch_count = &prev->nvcsw;
 	}
 
@@ -7183,6 +7182,7 @@ static void __sched notrace __schedule(int sched_mode)
 			}
 			if (next == rq->idle) {
 				zap_balance_callbacks(rq);
+				scx_proxy_resolved(rq);
 				goto keep_resched;
 			}
 		}
@@ -7203,6 +7203,8 @@ static void __sched notrace __schedule(int sched_mode)
 			donor->sched_class->put_prev_task(rq, donor, donor);
 			donor->sched_class->set_next_task(rq, donor, true);
 		}
+		scx_proxy_donor_start(rq);
+		scx_proxy_resolved(rq);
 	} else {
 		rq_set_donor(rq, next);
 	}
kernel/sched/ext/ext.c +13 -0
index 69ab59d176ce0..8bc83070720ca 100644
@@ -24,6 +24,11 @@
 
 DEFINE_RAW_SPINLOCK(scx_sched_lock);
 
+bool scx_allow_proxy_exec(const struct task_struct *p)
+{
+	return true;
+}
+
 /*
  * NOTE: sched_ext is in the process of growing multiple scheduler support and
  * scx_root usage is in a transitional state. Naked dereferences are safe if the
@@ -1053,6 +1058,10 @@ static void schedule_deferred_locked(struct rq *rq)
 	schedule_deferred(rq);
 }
 
+void scx_proxy_resolved(struct rq *rq)
+{
+}
+
 void schedule_dsq_reenq(struct scx_sched *sch, struct scx_dispatch_q *dsq,
 			u64 reenq_flags, struct rq *locked_rq)
 {
@@ -2957,6 +2966,10 @@ static void set_next_task_scx(struct rq *rq, struct task_struct *p, bool first)
 	}
 }
 
+void scx_proxy_donor_start(struct rq *rq)
+{
+}
+
 static enum scx_cpu_preempt_reason
 preempt_reason_from_class(const struct sched_class *class)
 {
kernel/sched/ext/ext.h +6 -0
index 0b7fc46aee08c..859b9261a8a6b 100644
@@ -20,6 +20,9 @@ void scx_rq_deactivate(struct rq *rq);
 int scx_check_setscheduler(struct task_struct *p, int policy);
 bool task_should_scx(int policy);
 bool scx_allow_ttwu_queue(const struct task_struct *p);
+bool scx_allow_proxy_exec(const struct task_struct *p);
+void scx_proxy_donor_start(struct rq *rq);
+void scx_proxy_resolved(struct rq *rq);
 void init_sched_ext_class(void);
 
 static inline u32 scx_cpuperf_target(s32 cpu)
@@ -54,6 +57,9 @@ static inline void scx_rq_deactivate(struct rq *rq) {}
 static inline int scx_check_setscheduler(struct task_struct *p, int policy) { return 0; }
 static inline bool task_on_scx(const struct task_struct *p) { return false; }
 static inline bool scx_allow_ttwu_queue(const struct task_struct *p) { return true; }
+static inline bool scx_allow_proxy_exec(const struct task_struct *p) { return true; }
+static inline void scx_proxy_donor_start(struct rq *rq) {}
+static inline void scx_proxy_resolved(struct rq *rq) {}
 static inline void init_sched_ext_class(void) {}
 
 #endif	/* CONFIG_SCHED_CLASS_EXT */
Signature · 1 line
2.55.0
author reviewer
#8 lore raw

PATCH 7/15 sched_ext: Block proxy donors across scheduler transitions

In reply to Andrea Righi

Proxy execution retains mutex-blocked donors on the runqueue so their scheduling context can execute a lock owner. sched_ext cannot safely retain such donors unless the BPF scheduler explicitly participates in their admission and ordering.

Make sched_ext reject retained donors by default. Implement scx_allow_proxy_exec() to force blocked EXT tasks through the regular block path in schedule(), and fully deactivate a retained donor before sched_setscheduler(), PI de-boosting, or global activation moves it into the EXT class.

This is a preparatory change to support proxy execution with sched_ext.

---
 kernel/sched/ext/ext.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)
kernel/sched/ext/ext.c +10 -1
index 8bc83070720ca..4ee27eaea8604 100644
@@ -26,7 +26,15 @@ DEFINE_RAW_SPINLOCK(scx_sched_lock);
 
 bool scx_allow_proxy_exec(const struct task_struct *p)
 {
-	return true;
+	return p->sched_class != &ext_sched_class;
+}
+
+static void prepare_switch_scx(struct rq *rq, struct task_struct *p)
+{
+	lockdep_assert_held(&p->pi_lock);
+	lockdep_assert_rq_held(rq);
+
+	sched_proxy_block_task(rq, p);
 }
 
 /*
@@ -4748,6 +4756,7 @@ DEFINE_SCHED_CLASS(ext) = {
 
 	.task_tick		= task_tick_scx,
 
+	.prepare_switch		= prepare_switch_scx,
 	.switching_to		= switching_to_scx,
 	.switched_from		= switched_from_scx,
 	.switched_to		= switched_to_scx,
Signature · 1 line
2.55.0
reviewer
#9 lore raw

Re: [PATCH 07/15] sched_ext: Block proxy donors across scheduler transitions

On Tue, Jul 28, 2026 at 05:43:25PM +0200, Andrea Righi wrote:
...

kernel/sched/ext/ext.c DEFINE_RAW_SPINLOCK(scx_sched_lock); · line 38

Quoted 7 lines
kernel/sched/ext/ext.c +6 -0
+static void prepare_switch_scx(struct rq *rq, struct task_struct *p)
+{
+	lockdep_assert_held(&p->pi_lock);
+	lockdep_assert_rq_held(rq);
+
+	sched_proxy_block_task(rq, p);
 }

Are there use cases where proxy execution instance needs to survive across class changes (the mutex rt PI, maybe)? If not, maybe this can be the global behavior?

Thanks.

Signature · 1 line
tejun
author reviewer
#10 lore raw

Re: [PATCH 07/15] sched_ext: Block proxy donors across scheduler transitions

Hi Tejun,

Quoted 13 lines · Tejun Heo

On Tue, Jul 28, 2026 at 05:43:25PM +0200, Andrea Righi wrote:
...

kernel/sched/ext/ext.c DEFINE_RAW_SPINLOCK(scx_sched_lock); · line 38

Quoted 7 lines
kernel/sched/ext/ext.c +6 -0
+static void prepare_switch_scx(struct rq *rq, struct task_struct *p)
+{
+	lockdep_assert_held(&p->pi_lock);
+	lockdep_assert_rq_held(rq);
+
+	sched_proxy_block_task(rq, p);
 }

Are there use cases where proxy execution instance needs to survive across class changes (the mutex rt PI, maybe)? If not, maybe this can be the global behavior?

I think RT/DL PI is a use case where a proxy donor needs to survive a class change. Example:

H (RT) waits on rtmutex R owned by D
D (FAIR) waits on normal mutex M owned by O

rt_mutex_setprio(D, H) promotes D from FAIR to RT. If D remains a retained proxy donor, its new RT scheduling context can proxy-exec O, allowing O to release M, then D to release R.

If every class transition called sched_proxy_block_task(), D would be removed from the runqueue when FAIR->RT occurs. The PI boost would then stop at the ordinary mutex boundary, and O would execute only under its own scheduling context. The same issue exists for RT/DL deboost transitions.

Potentially this could be generalized in core as a per-class "supports retained proxy donors" capability, but unconditional global blocking would discard useful PI behavior.

Thanks,
-Andrea

author reviewer
#11 lore raw

Re: [PATCH 07/15] sched_ext: Block proxy donors across scheduler transitions

Quoted 27 lines · Andrea Righi

Hi Tejun,

Quoted 13 lines · Tejun Heo

On Tue, Jul 28, 2026 at 05:43:25PM +0200, Andrea Righi wrote:
...

kernel/sched/ext/ext.c DEFINE_RAW_SPINLOCK(scx_sched_lock); · line 38

Quoted 7 lines
kernel/sched/ext/ext.c +6 -0
+static void prepare_switch_scx(struct rq *rq, struct task_struct *p)
+{
+	lockdep_assert_held(&p->pi_lock);
+	lockdep_assert_rq_held(rq);
+
+	sched_proxy_block_task(rq, p);
 }

Are there use cases where proxy execution instance needs to survive across class changes (the mutex rt PI, maybe)? If not, maybe this can be the global behavior?

I think RT/DL PI is a use case where a proxy donor needs to survive a class change. Example:

H (RT) waits on rtmutex R owned by D
D (FAIR) waits on normal mutex M owned by O

rt_mutex_setprio(D, H) promotes D from FAIR to RT. If D remains a retained proxy donor, its new RT scheduling context can proxy-exec O, allowing O to release M, then D to release R.

Quoted 8 lines

If every class transition called sched_proxy_block_task(), D would be removed from the runqueue when FAIR->RT occurs. The PI boost would then stop at the ordinary mutex boundary, and O would execute only under its own scheduling context. The same issue exists for RT/DL deboost transitions.

Potentially this could be generalized in core as a per-class "supports retained proxy donors" capability, but unconditional global blocking would discard useful PI behavior.

Following up on my response to patch 12 (which essentially contradicts my earlier response here), we can probably take a more conservative approach for now and leave more selective handling for future work.

While preserving a proxy session across some RT/DL PI class transitions is useful, terminating retained proxy execution centrally in sched_change_begin(), whenever the scheduling class changes, simplifies the implementation and it can make the initial integration safer.

In the next version of this series I'll document this limitation in both the code and patch description and leave support for compatible RT/DL PI transitions as a future improvement.

Thanks,
-Andrea

author reviewer
#12 lore raw

PATCH 8/15 sched_ext: Fix ops.running/stopping() pairing for proxy-exec donors

In reply to Andrea Righi

With proxy execution, pick_next_task() can select a blocked task as the scheduling context before find_proxy_task() resolves the execution context.

From the BPF scheduler perspective, that donor is running while its scheduling context drives the lock owner; ops.tick() and other accounting must therefore remain enclosed by a matching ops.running()/ops.stopping() session.

In this scenario, the session boundaries do not always match physical task switches. Keep the "running" session open when the same donor continues on the same CPU. When proxy execution migrates a donor's scheduling context to another CPU, end its running session on the source CPU and start a new session on the destination CPU only after proxy resolution succeeds. This prevents a failed resolution from exposing a provisional ops.running() event.

Track these sessions with a new SCX_TASK_RUN_TRACKED flag.

This is a preparatory change for enabling proxy execution together with sched_ext. The explicit running-state tracking is also required by later donor-based accounting: it prevents an EXT owner executing for a non-EXT donor from being treated as the active EXT scheduling context when it is dequeued.

---
 include/linux/sched/ext.h   |  1 +
 kernel/sched/ext/ext.c      | 62 ++++++++++++++++++++++++++++---------
 kernel/sched/ext/internal.h |  6 ++++
 3 files changed, 55 insertions(+), 14 deletions(-)
include/linux/sched/ext.h +1 -0
index bd9c4059e8fc3..4785a02905ecc 100644
@@ -102,6 +102,7 @@ enum scx_ent_flags {
 	SCX_TASK_DEQD_FOR_SLEEP	= 1 << 3, /* last dequeue was for SLEEP */
 	SCX_TASK_SUB_INIT	= 1 << 4, /* task being initialized for a sub sched */
 	SCX_TASK_IMMED		= 1 << 5, /* task is on local DSQ with %SCX_ENQ_IMMED */
+	SCX_TASK_RUN_TRACKED	= 1 << 6, /* task is in an ops.running()/stopping() session */
 
 	/*
 	 * Bits 8 to 10 are used to carry task state:
kernel/sched/ext/ext.c +48 -14

101 lines not on this page · raw message

kernel/sched/ext/internal.h +6 -0
index 7501ec28958bd..c3b1f9a5cf3a9 100644
@@ -448,6 +448,12 @@ struct sched_ext_ops {
 	 * Therefore, always use scx_bpf_task_cpu(@p) to determine the
 	 * target CPU the task is going to use.
 	 *
+	 * Under proxy execution, the BPF scheduler continues to observe the
+	 * donor as the current scheduling context. A blocked donor enters a
+	 * ->running()/->stopping() session while its scheduling context drives
+	 * the lock owner. The lock owner executing on its behalf is intentionally
+	 * not reported through these callbacks.
+	 *
 	 * See ->runnable() for explanation on the task state notifiers.
 	 */
 	void (*running)(struct task_struct *p);
Signature · 1 line
2.55.0
author reviewer
#13 lore raw

PATCH 9/15 sched_ext: Generalize the reject DSQ reenqueue path

In reply to Andrea Righi

The reject DSQ is currently specific to sub-scheduler cap failures. Its storage and initialization depend on CONFIG_EXT_SUB_SCHED, and the drain path assumes every rejected task was rejected for SCX_TASK_REENQ_CAP.

Other transient placement failures need the same ability to park a task on its source rq and return it to the owning BPF scheduler. Make the reject DSQ unconditional, move its drain path into the core sched_ext implementation, and record the reenqueue reason on each parked task.

This is a preparatory change to support proxy execution with sched_ext.

---
 include/linux/sched/ext.h |  1 +
 kernel/sched/ext/ext.c    | 59 ++++++++++++++++++++++++++++++++++++---
 kernel/sched/ext/sub.c    | 47 ++-----------------------------
 kernel/sched/ext/sub.h    |  2 --
 kernel/sched/sched.h      |  4 +--
 5 files changed, 59 insertions(+), 54 deletions(-)
include/linux/sched/ext.h +1 -0
index 4785a02905ecc..0f25d99f5fdd0 100644
@@ -197,6 +197,7 @@ struct sched_ext_entity {
 	struct rb_node		dsq_priq;	/* p->scx.dsq_vtime order */
 	u32			dsq_seq;
 	u32			dsq_flags;	/* protected by DSQ lock */
+	u32			reject_reason;	/* protected by rq lock */
 	u32			flags;		/* protected by rq lock */
 	u32			weight;
 	u32			reenq_cnt;	/* reenqueues since last run */
kernel/sched/ext/ext.c +55 -4

87 lines not on this page · raw message

kernel/sched/ext/sub.c +2 -45

59 lines not on this page · raw message

kernel/sched/ext/sub.h +0 -2
index 0019b75a2560e..b5870aeb9c625 100644
@@ -37,7 +37,6 @@ void scx_discard_stale_ecaps_syncs(void);
 struct scx_dispatch_q *scx_local_or_reject_dsq(struct scx_sched *sch, struct rq *rq,
 					       struct task_struct *p, u64 *enq_flags);
 bool scx_task_reenq_on_cap_revoke(struct rq *rq, struct task_struct *p);
-void scx_reenq_reject(struct rq *rq);
 
 /*
  * cgrp->scx_sched is written by root/sub enable/disable under all of
@@ -86,7 +85,6 @@ static inline void scx_discard_ecaps_to_sync(s32 cpu, struct scx_sched_pcpu *pcp
 static inline void scx_discard_stale_ecaps_syncs(void) {}
 static inline struct scx_dispatch_q *scx_local_or_reject_dsq(struct scx_sched *sch, struct rq *rq, struct task_struct *p, u64 *enq_flags) { return &rq->scx.local_dsq; }
 static inline bool scx_task_reenq_on_cap_revoke(struct rq *rq, struct task_struct *p) { return false; }
-static inline void scx_reenq_reject(struct rq *rq) {}
 static inline void scx_dec_has_subs(struct scx_sched *sch) {}
 
 #endif	/* CONFIG_EXT_SUB_SCHED */
kernel/sched/sched.h +1 -3
index 79daacc2389e1..ade8bb393786a 100644
@@ -796,9 +796,7 @@ enum scx_rq_flags {
 
 struct scx_rq {
 	struct scx_dispatch_q	local_dsq;
-#ifdef CONFIG_EXT_SUB_SCHED
-	struct scx_dispatch_q	reject_dsq;		/* staging for cap-rejected tasks */
-#endif
+	struct scx_dispatch_q	reject_dsq;		/* staging for rejected tasks */
 	struct list_head	runnable_list;		/* runnable tasks on this rq */
 	struct list_head	ddsp_deferred_locals;	/* deferred ddsps from enq */
 	unsigned long		ops_qseq;
Signature · 1 line
2.55.0
#14 lore raw

Re: [PATCH 09/15] sched_ext: Generalize the reject DSQ reenqueue path

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] Transient failure to set a reject reason leaves the task permanently stranded on the reject DSQ. --

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Transient failure to set a reject reason leaves the task permanently stranded on the reject DSQ. --

--- Patch [9]: [PATCH 09/15] sched_ext: Generalize the reject DSQ reenqueue path --- commit b71cb24d29b22c40aa0f3482e228cd8cd043d267
Author: Andrea Righi <[email protected]>

sched_ext: Generalize the reject DSQ reenqueue path

This commit makes the reject DSQ unconditional and moves its drain path into the core sched_ext implementation. It also records the reenqueue reason on each parked task to support future proxy execution features.

Quoted 3 lines
kernel/sched/ext/ext.c +0 -0

[ ... ]

Quoted 1 line
kernel/sched/ext/ext.c +0 -0
@@ -4399,6 +4401,57 @@ static void process_deferred_reenq_users(struct rq *rq)

[ ... ]

Quoted 2 lines
kernel/sched/ext/ext.c +2 -0
+static void scx_reenq_reject(struct rq *rq)
+{

[ ... ]

kernel/sched/ext/ext.c static void process_deferred_reenq_users(struct rq *rq) · line 4431

Quoted 12 lines
kernel/sched/ext/ext.c +12 -0
+	/*
+	 * Move tasks to a private list so a task re-rejected by
+	 * scx_do_enqueue_task() below isn't revisited this round.
+	 */
+	list_for_each_entry_safe(p, n, &rq->scx.reject_dsq.list, scx.dsq_list.node) {
+		u32 reason = p->scx.reject_reason;
+
+		/* migration_pending tasks should have bypassed to local DSQ */
+		if (WARN_ON_ONCE(p->migration_pending))
+			continue;
+		if (WARN_ON_ONCE(!reason))
+			continue;

[Severity: High]
If !reason is true and this WARN_ON_ONCE triggers, will this continue statement leave the task permanently stranded on the reject DSQ list?

Because the continue bypasses scx_dispatch_dequeue() and the subsequent list_add_tail(), the task is never unlinked from rq->scx.reject_dsq.list.

Since the reject DSQ is just a staging list, the task won't be scheduled, and every subsequent run of scx_reenq_reject() will hit this continue again.

Should this error path safely recover the task (perhaps by falling back to the local DSQ) or at least unlink it instead of leaving it in an unrecoverable hung state?

Quoted 3 lines
kernel/sched/ext/ext.c +3 -0
+
+		scx_dispatch_dequeue(rq, p);
+		p->scx.reject_reason = SCX_TASK_REENQ_NONE;

[ ... ]

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=9
reviewer
#15 lore raw

Re: [PATCH 09/15] sched_ext: Generalize the reject DSQ reenqueue path

In reply to Andrea Righi

On Tue, Jul 28, 2026 at 05:43:27PM +0200, Andrea Righi wrote:
...

Quoted 19 lines
kernel/sched/ext/ext.c +19 -0
+static void scx_reenq_reject(struct rq *rq)
+{
+	LIST_HEAD(tasks);
+	struct task_struct *p, *n;
+
+	lockdep_assert_rq_held(rq);
+
+	if (list_empty(&rq->scx.reject_dsq.list))
+		return;
+
+	/*
+	 * Move tasks to a private list so a task re-rejected by
+	 * scx_do_enqueue_task() below isn't revisited this round.
+	 */
+	list_for_each_entry_safe(p, n, &rq->scx.reject_dsq.list, scx.dsq_list.node) {
+		u32 reason = p->scx.reject_reason;
+
+		/* migration_pending tasks should have bypassed to local DSQ */
+		if (WARN_ON_ONCE(p->migration_pending))

kernel/sched/ext/ext.c static void process_deferred_reenq_users(struct rq *rq) · line 4438

Quoted 10 lines
kernel/sched/ext/ext.c +10 -0
+			continue;
+		if (WARN_ON_ONCE(!reason))
+			continue;
+
+		scx_dispatch_dequeue(rq, p);
+		p->scx.reject_reason = SCX_TASK_REENQ_NONE;
+
+		if (WARN_ON_ONCE(p->scx.flags & SCX_TASK_REENQ_REASON_MASK))
+			p->scx.flags &= ~SCX_TASK_REENQ_REASON_MASK;
+		p->scx.flags |= reason;

Can you separate out code movement and actual changes into separate patches? Even for relatively simple changes, combining move and changes adds significant unnecessary review friction.

Thanks.

Signature · 1 line
tejun
reviewer
#16 lore raw

Re: [PATCH 09/15] sched_ext: Generalize the reject DSQ reenqueue path

Quoted 27 lines · Tejun Heo

On Tue, Jul 28, 2026 at 05:43:27PM +0200, Andrea Righi wrote:
...

Quoted 25 lines
kernel/sched/ext/ext.c +25 -0
+static void scx_reenq_reject(struct rq *rq)
+{
+	LIST_HEAD(tasks);
+	struct task_struct *p, *n;
+
+	lockdep_assert_rq_held(rq);
+
+	if (list_empty(&rq->scx.reject_dsq.list))
+		return;
+
+	/*
+	 * Move tasks to a private list so a task re-rejected by
+	 * scx_do_enqueue_task() below isn't revisited this round.
+	 */
+	list_for_each_entry_safe(p, n, &rq->scx.reject_dsq.list, scx.dsq_list.node) {
+		u32 reason = p->scx.reject_reason;
+
+		/* migration_pending tasks should have bypassed to local DSQ */
+		if (WARN_ON_ONCE(p->migration_pending))
+			continue;
+		if (WARN_ON_ONCE(!reason))
+			continue;
+
+		scx_dispatch_dequeue(rq, p);
+		p->scx.reject_reason = SCX_TASK_REENQ_NONE;
Quoted 8 lines

kernel/sched/ext/ext.c static void process_deferred_reenq_users(struct rq *rq) · line 4438

Quoted 4 lines
kernel/sched/ext/ext.c +4 -0
+
+		if (WARN_ON_ONCE(p->scx.flags & SCX_TASK_REENQ_REASON_MASK))
+			p->scx.flags &= ~SCX_TASK_REENQ_REASON_MASK;
+		p->scx.flags |= reason;

Can you separate out code movement and actual changes into separate patches? Even for relatively simple changes, combining move and changes adds significant unnecessary review friction.

I wonder whether the reason can be carried in p->scx.flags from the rejection site instead of bouncing through p->scx.reject_reason.

Thanks.

Signature · 1 line
tejun
author reviewer
#17 lore raw

Re: [PATCH 09/15] sched_ext: Generalize the reject DSQ reenqueue path

Quoted 37 lines · Tejun Heo
Quoted 27 lines · Tejun Heo

On Tue, Jul 28, 2026 at 05:43:27PM +0200, Andrea Righi wrote:
...

Quoted 25 lines
kernel/sched/ext/ext.c +25 -0
+static void scx_reenq_reject(struct rq *rq)
+{
+	LIST_HEAD(tasks);
+	struct task_struct *p, *n;
+
+	lockdep_assert_rq_held(rq);
+
+	if (list_empty(&rq->scx.reject_dsq.list))
+		return;
+
+	/*
+	 * Move tasks to a private list so a task re-rejected by
+	 * scx_do_enqueue_task() below isn't revisited this round.
+	 */
+	list_for_each_entry_safe(p, n, &rq->scx.reject_dsq.list, scx.dsq_list.node) {
+		u32 reason = p->scx.reject_reason;
+
+		/* migration_pending tasks should have bypassed to local DSQ */
+		if (WARN_ON_ONCE(p->migration_pending))
+			continue;
+		if (WARN_ON_ONCE(!reason))
+			continue;
+
+		scx_dispatch_dequeue(rq, p);
+		p->scx.reject_reason = SCX_TASK_REENQ_NONE;
Quoted 8 lines

kernel/sched/ext/ext.c static void process_deferred_reenq_users(struct rq *rq) · line 4438

Quoted 4 lines
kernel/sched/ext/ext.c +4 -0
+
+		if (WARN_ON_ONCE(p->scx.flags & SCX_TASK_REENQ_REASON_MASK))
+			p->scx.flags &= ~SCX_TASK_REENQ_REASON_MASK;
+		p->scx.flags |= reason;

Can you separate out code movement and actual changes into separate patches? Even for relatively simple changes, combining move and changes adds significant unnecessary review friction.

Quoted 2 lines

I wonder whether the reason can be carried in p->scx.flags from the rejection site instead of bouncing through p->scx.reject_reason.

Ack, I think it's possible. I'll split the patches and set the rejection reason directly in p->scx.flags.

Thanks,
-Andrea

author reviewer
#18 lore raw

PATCH 10/15 sched_ext: Handle proxy-exec races in remote DSQ transfers

In reply to Andrea Righi

Without proxy execution, the DSQ lock and holding_cpu handshake ensure that a task cannot be dequeued or start running during an rq-lock handoff without clearing holding_cpu.

Proxy execution is an exception: a task can start physically executing as a lock owner while its scheduling context remains on a DSQ; its on-CPU or migration-disabled state can therefore change without clearing holding_cpu.

Recheck these states after acquiring the source rq lock. If the transfer can no longer proceed, park the task on the source rq's reject DSQ and re-enqueue it through its owning scheduler. This preserves the BPF scheduler's placement policy and keeps descendant tasks within their sub-scheduler's cap grants.

Implement the scx_proxy_resolved() hook to drain the parked tasks once proxy resolution has settled and the outgoing owner has switched out.

Without this change and proxy execution enabled, stress-ng --pipeherd can trigger this race and migrate an active execution context, leading to sleeping-while-atomic warnings and subsequent lockdep corruption.

This is a preparatory change to support proxy execution with sched_ext.

---
 include/linux/sched/ext.h |   5 ++
 kernel/sched/ext/ext.c    | 154 +++++++++++++++++++++++++++++++++-----
 kernel/sched/sched.h      |   1 +
 3 files changed, 141 insertions(+), 19 deletions(-)
include/linux/sched/ext.h +5 -0
index 0f25d99f5fdd0..53da9eea0c925 100644
@@ -135,6 +135,9 @@ enum scx_ent_flags {
 	 * IMMED	reenqueued due to failed ENQ_IMMED
 	 * PREEMPTED	preempted while running
 	 * CAP		sub-sched cap miss, see p->scx.reenq_reason_*
+	 * MIGRATION_DISABLED
+	 *		migration-disabled during a remote DSQ transfer
+	 * PROXY	physically executing or donating during a remote DSQ transfer
 	 */
 	SCX_TASK_REENQ_REASON_SHIFT = 12,
 	SCX_TASK_REENQ_REASON_BITS = 3,
@@ -145,6 +148,8 @@ enum scx_ent_flags {
 	SCX_TASK_REENQ_IMMED	= 2 << SCX_TASK_REENQ_REASON_SHIFT,
 	SCX_TASK_REENQ_PREEMPTED = 3 << SCX_TASK_REENQ_REASON_SHIFT,
 	SCX_TASK_REENQ_CAP	= 4 << SCX_TASK_REENQ_REASON_SHIFT,
+	SCX_TASK_REENQ_MIGRATION_DISABLED = 5 << SCX_TASK_REENQ_REASON_SHIFT,
+	SCX_TASK_REENQ_PROXY	= 6 << SCX_TASK_REENQ_REASON_SHIFT,
 
 	/* iteration cursor, not a task */
 	SCX_TASK_CURSOR		= 1 << 31,
kernel/sched/ext/ext.c +135 -19

253 lines not on this page · raw message

kernel/sched/sched.h +1 -0
index ade8bb393786a..bbd90329585a6 100644
@@ -789,6 +789,7 @@ enum scx_rq_flags {
 	SCX_RQ_BAL_CB_PENDING	= 1 << 6, /* must queue a cb after dispatching */
 	SCX_RQ_SUB_IDLE_RENOTIFY	= 1 << 7, /* sub-scheds are owed update_idle() */
 	SCX_RQ_ROOT_IDLE_RENOTIFY	= 1 << 8, /* the root is owed update_idle() */
+	SCX_RQ_PROXY_REENQ	= 1 << 9, /* proxy-rejected tasks need reenqueue */
 
 	SCX_RQ_IN_WAKEUP	= 1 << 16,
 	SCX_RQ_IN_BALANCE	= 1 << 17,
Signature · 1 line
2.55.0
reviewer
#19 lore raw

Re: [PATCH 10/15] sched_ext: Handle proxy-exec races in remote DSQ transfers

Hello,

On Tue, Jul 28, 2026 at 05:43:28PM +0200, Andrea Righi wrote:
...

include/linux/sched/ext.h enum scx_ent_flags { · line 152

Quoted 7 lines
include/linux/sched/ext.h +2 -0
@@ -135,6 +135,9 @@ enum scx_ent_flags {
@@ -145,6 +148,8 @@ enum scx_ent_flags {
 	SCX_TASK_REENQ_IMMED	= 2 << SCX_TASK_REENQ_REASON_SHIFT,
 	SCX_TASK_REENQ_PREEMPTED = 3 << SCX_TASK_REENQ_REASON_SHIFT,
 	SCX_TASK_REENQ_CAP	= 4 << SCX_TASK_REENQ_REASON_SHIFT,
+	SCX_TASK_REENQ_MIGRATION_DISABLED = 5 << SCX_TASK_REENQ_REASON_SHIFT,
+	SCX_TASK_REENQ_PROXY	= 6 << SCX_TASK_REENQ_REASON_SHIFT,

Given that MIGRATION_DISABLED can only happen with proxy execution, it may be better to name it accordingly. Maybe that's too long. Do we have to distinguish between PROXY and MIGRATION_DISABLED? We can count both as PROXY, no?

Quoted 13 lines
kernel/sched/ext/ext.c +13 -0
+/*
+ * Proxy execution can change @p's execution and migration-disabled state
+ * without touching its DSQ entry or clearing holding_cpu. Check those states
+ * with @p's rq locked. Without proxy execution, the holding_cpu handshake is
+ * sufficient and this must not affect the existing migration path.
+ */
+static u32 task_move_reject_reason(struct task_struct *p)
+{
+	struct rq *src_rq = task_rq(p);
+
+	lockdep_assert_rq_held(src_rq);
+
+	if (!sched_proxy_exec())

kernel/sched/ext/ext.c static bool task_can_run_on_remote_rq(struct scx_sched *sch, · line 2476

Quoted 10 lines
kernel/sched/ext/ext.c +10 -0
+		return SCX_TASK_REENQ_NONE;
+
+	/* @p may be rq->curr under another task's scheduling context. */
+	if (task_on_cpu(src_rq, p))
+		return SCX_TASK_REENQ_PROXY;
+
+	/*
+	 * Reject only BPF-directed migration. proxy_migrate_task() may still
+	 * move a blocked donor's scheduling context to its lock owner's CPU.
+	 */

I have a hard time understanding this comment. Can you explain a bit more?

Quoted 17 lines
kernel/sched/ext/ext.c +17 -0
+	if (is_migration_disabled(p))
+		return SCX_TASK_REENQ_MIGRATION_DISABLED;
+
+	/* Don't move an active scheduling context off its source rq. */
+	if (task_current_donor(src_rq, p))
+		return SCX_TASK_REENQ_PROXY;
+
+	return SCX_TASK_REENQ_NONE;
+}
+
+/*
+ * Park a task whose remote transfer raced with proxy execution. Reenqueueing
+ * from the source rq makes the task's owning scheduler choose its placement
+ * again and preserves sub-scheduler containment.
+ */
+static void scx_reject_task(struct scx_sched *sch, struct rq *rq,
+			    struct task_struct *p, u64 enq_flags, u32 reason)

kernel/sched/ext/ext.c static bool task_can_run_on_remote_rq(struct scx_sched *sch, · line 2503

Quoted 10 lines
kernel/sched/ext/ext.c +10 -0
+{
+	lockdep_assert_rq_held(rq);
+	WARN_ON_ONCE(reason != SCX_TASK_REENQ_MIGRATION_DISABLED &&
+		     reason != SCX_TASK_REENQ_PROXY);
+	WARN_ON_ONCE(p->scx.reject_reason);
+
+	p->scx.holding_cpu = -1;
+	p->scx.reject_reason = reason;
+	p->scx.flags &= ~SCX_TASK_IMMED;
+	enq_flags &= ~(SCX_ENQ_IMMED | SCX_ENQ_PREEMPT);

SCX_ENQ_HEAD likely needs clearing too. After applying the rescue patchset, scx_resolve_local_dsq() has:

	/*
	 * Diverting to rescue or reject, neither of which honors IMMED, PREEMPT
	 * or HEAD - a diversion has no priority and IMMED is not allowed on
	 * non-local DSQs. Strip the enq and task flags along with the slice.
	 */
	*enq_flags &= ~(SCX_ENQ_IMMED | SCX_ENQ_PREEMPT | SCX_ENQ_HEAD |
			SCX_ENQ_APPLY_SLICE | SCX_ENQ_SLICE_DFL);
	p->scx.flags &= ~SCX_TASK_IMMED;

We should probably factor that out and use that whenever we're diverting.

kernel/sched/ext/ext.c static struct rq *move_task_between_dsqs(struct scx_sched *sch, · line 2628

Quoted 13 lines
kernel/sched/ext/ext.c +9 -0
@@ -2533,6 +2617,15 @@ static struct rq *move_task_between_dsqs(struct scx_sched *sch,
 
 	if (dst_dsq->id == SCX_DSQ_LOCAL) {
 		dst_rq = container_of(dst_dsq, struct rq, scx.local_dsq);
+		reject_reason = src_rq != dst_rq ?
+			task_move_reject_reason(p) : SCX_TASK_REENQ_NONE;
+		if (unlikely(reject_reason)) {
+			dispatch_dequeue_locked(p, src_dsq);
+			raw_spin_unlock(&src_dsq->lock);
+			scx_reject_task(sch, src_rq, p, enq_flags,
+					reject_reason);
+			return src_rq;
+		}

I wonder whether it'd be cleaner if we just mark the task for rejection and then make scx_resolve_local_dsq() resolve that to reject dsq instead of directly inserting from each site. I guess the problem is that the rejection has to be on the source rq, not the destination one. I hope there's a neater way to do this.

Quoted 29 lines
kernel/sched/ext/ext.c +8 -6
 /*
- * Drain @rq->scx.reject_dsq and reenqueue each task so that its owning BPF
- * scheduler chooses placement again.
+ * Drain ready tasks from @rq->scx.reject_dsq and reenqueue them so that their
+ * owning BPF schedulers choose placement again. Proxy-active tasks remain
+ * parked until proxy resolution schedules another drain after switch-out.
  *
  * A task can be re-rejected repeatedly. Reenqueues are bounded per task by
  * SCX_REENQ_MAX_REPEAT in scx_do_enqueue_task(), which ejects the owning
- * scheduler. The private list below prevents a task from being revisited in
- * the same round.
+ * scheduler.
  */
 static void scx_reenq_reject(struct rq *rq)
 {
 	LIST_HEAD(tasks);
 	struct task_struct *p, *n;
+	bool proxy_pending = false;
 
 	lockdep_assert_rq_held(rq);
 
-	if (list_empty(&rq->scx.reject_dsq.list))
+	if (list_empty(&rq->scx.reject_dsq.list)) {
+		rq->scx.flags &= ~SCX_RQ_PROXY_REENQ;
 		return;
+	}
 
 	/*
-	 * Move tasks to a private list so a task re-rejected by

kernel/sched/ext/ext.c static void process_deferred_reenq_users(struct rq *rq) · line 4534

Quoted 10 lines
kernel/sched/ext/ext.c +2 -2
+	 * Move ready tasks to a private list so a task re-rejected by
 	 * scx_do_enqueue_task() below isn't revisited this round.
 	 */
 	list_for_each_entry_safe(p, n, &rq->scx.reject_dsq.list, scx.dsq_list.node) {
 		u32 reason = p->scx.reject_reason;
 
 		/* migration_pending tasks should have bypassed to local DSQ */
-		if (WARN_ON_ONCE(p->migration_pending))
-			continue;
+		WARN_ON_ONCE(p->migration_pending);

Why this change? Also, wouldn't this now be allowed to happen? During task move, if it hits migration_pending due to proxy exec, the task would be put on reject DSQ, right? Reenq can trigger from other sources before migration_pending is cleared and then would see the migration_pending set. Shouldn't it just continue?

kernel/sched/ext/ext.c static void process_deferred_reenq_users(struct rq *rq) · line 4542

Quoted 8 lines
kernel/sched/ext/ext.c +5 -0
 		if (WARN_ON_ONCE(!reason))
 			continue;
 
+		if (reason == SCX_TASK_REENQ_PROXY &&
+		    (task_on_cpu(rq, p) || task_current_donor(rq, p))) {
+			proxy_pending = true;
+			continue;
+		}

ie. Shouldn't migration_pending() be one of the || conditions above? And then if it's still migraiton_pending, that should trigger a warning?

Thanks.

Signature · 1 line
tejun
author reviewer
#20 lore raw

Re: [PATCH 10/15] sched_ext: Handle proxy-exec races in remote DSQ transfers

Hi Tejun,

Quoted 4 lines · Tejun Heo

Hello,

On Tue, Jul 28, 2026 at 05:43:28PM +0200, Andrea Righi wrote:
...

Quoted 12 lines

include/linux/sched/ext.h enum scx_ent_flags { · line 152

Quoted 7 lines
include/linux/sched/ext.h +2 -0
@@ -135,6 +135,9 @@ enum scx_ent_flags {
@@ -145,6 +148,8 @@ enum scx_ent_flags {
 	SCX_TASK_REENQ_IMMED	= 2 << SCX_TASK_REENQ_REASON_SHIFT,
 	SCX_TASK_REENQ_PREEMPTED = 3 << SCX_TASK_REENQ_REASON_SHIFT,
 	SCX_TASK_REENQ_CAP	= 4 << SCX_TASK_REENQ_REASON_SHIFT,
+	SCX_TASK_REENQ_MIGRATION_DISABLED = 5 << SCX_TASK_REENQ_REASON_SHIFT,
+	SCX_TASK_REENQ_PROXY	= 6 << SCX_TASK_REENQ_REASON_SHIFT,

Given that MIGRATION_DISABLED can only happen with proxy execution, it may be better to name it accordingly. Maybe that's too long. Do we have to distinguish between PROXY and MIGRATION_DISABLED? We can count both as PROXY, no?

Ack, both conditions have the same outcome: abort the BPF-directed remote transfer, park the task on the source rq and return it to its owning scheduler.

I'll drop SCX_TASK_REENQ_MIGRATION_DISABLED and use SCX_TASK_REENQ_PROXY for all of these cases.

Quoted 18 lines
Quoted 18 lines
kernel/sched/ext/ext.c +17 -0
+/*
+ * Proxy execution can change @p's execution and migration-disabled state
+ * without touching its DSQ entry or clearing holding_cpu. Check those states
+ * with @p's rq locked. Without proxy execution, the holding_cpu handshake is
+ * sufficient and this must not affect the existing migration path.
+ */
+static u32 task_move_reject_reason(struct task_struct *p)
+{
+	struct rq *src_rq = task_rq(p);
+
+	lockdep_assert_rq_held(src_rq);
+
+	if (!sched_proxy_exec())
+		return SCX_TASK_REENQ_NONE;
+
+	/* @p may be rq->curr under another task's scheduling context. */
+	if (task_on_cpu(src_rq, p))
Quoted 8 lines

kernel/sched/ext/ext.c static bool task_can_run_on_remote_rq(struct scx_sched *sch, · line 2476

Quoted 6 lines
kernel/sched/ext/ext.c +6 -0
+		return SCX_TASK_REENQ_PROXY;
+
+	/*
+	 * Reject only BPF-directed migration. proxy_migrate_task() may still
+	 * move a blocked donor's scheduling context to its lock owner's CPU.
+	 */

I have a hard time understanding this comment. Can you explain a bit more?

The distinction here is between normal task placement and proxy donor migration.

A BPF-directed transfer to a remote local DSQ performs a normal task migration and, as we know, migration-disabled tasks can't migrate.

With proxy-exec, instead, donors can always be migrated towards the mutex owner's CPU, because they don't actually execute there; it's the owner that runs using the donated scheduling context; wake_cpu is preserved so that the donor returns to its original CPU when the owner releases the mutex. For that reason, proxy_migrate_task() should be allowed to migrate even when the donor is migration-disabled or when the target CPU outside its affinity mask.

I'll expand the comment to make this more clear.

Quoted 23 lines
Quoted 23 lines
kernel/sched/ext/ext.c +22 -0
+	if (is_migration_disabled(p))
+		return SCX_TASK_REENQ_MIGRATION_DISABLED;
+
+	/* Don't move an active scheduling context off its source rq. */
+	if (task_current_donor(src_rq, p))
+		return SCX_TASK_REENQ_PROXY;
+
+	return SCX_TASK_REENQ_NONE;
+}
+
+/*
+ * Park a task whose remote transfer raced with proxy execution. Reenqueueing
+ * from the source rq makes the task's owning scheduler choose its placement
+ * again and preserves sub-scheduler containment.
+ */
+static void scx_reject_task(struct scx_sched *sch, struct rq *rq,
+			    struct task_struct *p, u64 enq_flags, u32 reason)
+{
+	lockdep_assert_rq_held(rq);
+	WARN_ON_ONCE(reason != SCX_TASK_REENQ_MIGRATION_DISABLED &&
+		     reason != SCX_TASK_REENQ_PROXY);
+	WARN_ON_ONCE(p->scx.reject_reason);
Quoted 8 lines

kernel/sched/ext/ext.c static bool task_can_run_on_remote_rq(struct scx_sched *sch, · line 2503

Quoted 5 lines
kernel/sched/ext/ext.c +5 -0
+
+	p->scx.holding_cpu = -1;
+	p->scx.reject_reason = reason;
+	p->scx.flags &= ~SCX_TASK_IMMED;
+	enq_flags &= ~(SCX_ENQ_IMMED | SCX_ENQ_PREEMPT);

SCX_ENQ_HEAD likely needs clearing too. After applying the rescue patchset, scx_resolve_local_dsq() has:

Right, we're missing SCX_ENQ_HEAD here.

Quoted 11 lines
	/*
	 * Diverting to rescue or reject, neither of which honors IMMED, PREEMPT
	 * or HEAD - a diversion has no priority and IMMED is not allowed on
	 * non-local DSQs. Strip the enq and task flags along with the slice.
	 */
	*enq_flags &= ~(SCX_ENQ_IMMED | SCX_ENQ_PREEMPT | SCX_ENQ_HEAD |
			SCX_ENQ_APPLY_SLICE | SCX_ENQ_SLICE_DFL);
	p->scx.flags &= ~SCX_TASK_IMMED;

We should probably factor that out and use that whenever we're diverting.

I'll factor this into a common helper and use it from both scx_resolve_local_dsq() and the proxy rejection path.

Quoted 12 lines
Quoted 12 lines
kernel/sched/ext/ext.c +7 -0
@@ -2533,6 +2617,15 @@ static struct rq *move_task_between_dsqs(struct scx_sched *sch,
 
 	if (dst_dsq->id == SCX_DSQ_LOCAL) {
 		dst_rq = container_of(dst_dsq, struct rq, scx.local_dsq);
+		reject_reason = src_rq != dst_rq ?
+			task_move_reject_reason(p) : SCX_TASK_REENQ_NONE;
+		if (unlikely(reject_reason)) {
+			dispatch_dequeue_locked(p, src_dsq);
+			raw_spin_unlock(&src_dsq->lock);
+			scx_reject_task(sch, src_rq, p, enq_flags,
+					reject_reason);
Quoted 8 lines

kernel/sched/ext/ext.c static struct rq *move_task_between_dsqs(struct scx_sched *sch, · line 2628

Quoted 2 lines
kernel/sched/ext/ext.c +2 -0
+			return src_rq;
+		}

I wonder whether it'd be cleaner if we just mark the task for rejection and then make scx_resolve_local_dsq() resolve that to reject dsq instead of directly inserting from each site. I guess the problem is that the rejection has to be on the source rq, not the destination one. I hope there's a neater way to do this.

Right, the source rq requirement is the complication. By the time scx_resolve_local_dsq() is normally called, it is resolving an insertion relative to the requested destination rq. In this case the transfer must be aborted before changing task_rq() and the task must be parked on the source rq, so that its owning scheduler/sub-scheduler containment are preserved.

I can simplify this by making the race predicate boolean, making the PROXY reason implicit in the rejection helper, and using the common helper mentioned above. The individual transfer paths will still need a small branch, because they reach this point with different DSQ and rq locking states.

I don't see a better way to do this...

Quoted 36 lines
Quoted 36 lines
kernel/sched/ext/ext.c +9 -6
 /*
- * Drain @rq->scx.reject_dsq and reenqueue each task so that its owning BPF
- * scheduler chooses placement again.
+ * Drain ready tasks from @rq->scx.reject_dsq and reenqueue them so that their
+ * owning BPF schedulers choose placement again. Proxy-active tasks remain
+ * parked until proxy resolution schedules another drain after switch-out.
  *
  * A task can be re-rejected repeatedly. Reenqueues are bounded per task by
  * SCX_REENQ_MAX_REPEAT in scx_do_enqueue_task(), which ejects the owning
- * scheduler. The private list below prevents a task from being revisited in
- * the same round.
+ * scheduler.
  */
 static void scx_reenq_reject(struct rq *rq)
 {
 	LIST_HEAD(tasks);
 	struct task_struct *p, *n;
+	bool proxy_pending = false;
 
 	lockdep_assert_rq_held(rq);
 
-	if (list_empty(&rq->scx.reject_dsq.list))
+	if (list_empty(&rq->scx.reject_dsq.list)) {
+		rq->scx.flags &= ~SCX_RQ_PROXY_REENQ;
 		return;
+	}
 
 	/*
-	 * Move tasks to a private list so a task re-rejected by
+	 * Move ready tasks to a private list so a task re-rejected by
 	 * scx_do_enqueue_task() below isn't revisited this round.
 	 */
 	list_for_each_entry_safe(p, n, &rq->scx.reject_dsq.list, scx.dsq_list.node) {
 		u32 reason = p->scx.reject_reason;
Quoted 10 lines

kernel/sched/ext/ext.c static void process_deferred_reenq_users(struct rq *rq) · line 4534

Quoted 4 lines
kernel/sched/ext/ext.c +1 -2
 		/* migration_pending tasks should have bypassed to local DSQ */
-		if (WARN_ON_ONCE(p->migration_pending))
-			continue;
+		WARN_ON_ONCE(p->migration_pending);

Why this change? Also, wouldn't this now be allowed to happen? During task move, if it hits migration_pending due to proxy exec, the task would be put on reject DSQ, right? Reenq can trigger from other sources before migration_pending is cleared and then would see the migration_pending set. Shouldn't it just continue?

Yes, it should continue. Dropping the continue is wrong.

migration_pending is separate from proxy donor migration. It belongs to the generic affinity machinery, whereas proxy_migrate_task() moves only the donor's scheduling context and does not use migration_pending.

If a task on the proxy reject DSQ has migration_pending set, the affinity machinery owns its placement and will dequeue/reactivate it as necessary.

Quoted 4 lines
Quoted 4 lines
kernel/sched/ext/ext.c +0 -0
 		if (WARN_ON_ONCE(!reason))
 			continue;
Quoted 8 lines

kernel/sched/ext/ext.c static void process_deferred_reenq_users(struct rq *rq) · line 4542

Quoted 5 lines
kernel/sched/ext/ext.c +5 -0
+		if (reason == SCX_TASK_REENQ_PROXY &&
+		    (task_on_cpu(rq, p) || task_current_donor(rq, p))) {
+			proxy_pending = true;
+			continue;
+		}

ie. Shouldn't migration_pending() be one of the || conditions above? And then if it's still migraiton_pending, that should trigger a warning?

I think it should be handled by a separate early continue rather than included in this condition.

task_on_cpu() and task_current_donor() are resolved by the proxy switch-out path, so they set proxy_pending and arrange another drain from scx_proxy_resolved().

migration_pending is resolved by the affinity migration machinery, not by scx_proxy_resolved(). Treating it as proxy_pending would associate it with the wrong completion event.

Maybe we can structure it as following?

if (p->migration_pending) {
WARN_ON_ONCE(reason != SCX_TASK_REENQ_PROXY); continue;
}

  	if (reason == SCX_TASK_REENQ_PROXY &&
  	    (task_on_cpu(rq, p) || task_current_donor(rq, p))) {
  		proxy_pending = true;
  		continue;
  	}

This keeps the task parked while the affinity request is pending without restricting proxy_migrate_task(), which remains free to move blocked donors towards their mutex owners.

Thanks,
-Andrea

author reviewer
#21 lore raw

PATCH 11/15 sched_ext: Split curr|donor references properly

In reply to Andrea Righi

With proxy execution, the task selected by the scheduler and the task physically executing can differ. A blocked mutex waiter donates its scheduling context to the lock owner:

D -----------------> M -------------> O ----------------> T [donor] blocked on [mutex] owned by [owner] preempted by [task]
\_________________________________^
donates scheduling context

where:

D = blocked donor
M = mutex
O = mutex owner
T = competing runnable task

During a proxy execution switch, D supplies the scheduling class, priority, and runtime budget, while O supplies the execution context: O is the task whose code physically executes. T is a competing runnable task which may preempt the D/O proxy execution.

Consider FAIR and EXT tasks with sched_ext running in partial mode. FAIR can be replaced with a higher scheduling class such as RT or deadline without changing the class interaction described here. The possible combinations are:

1. D is EXT, O is EXT, T is EXT

D can interrupt T according to BPF scheduling policy. O executes with D's EXT priority and runtime budget, while T waits in EXT.

2. D is EXT, O is EXT, T is FAIR

D is visible to the BPF scheduler, but cannot preempt T because EXT is below FAIR. Once T stops, BPF can dispatch D and O executes with D's EXT priority and runtime budget. If T becomes runnable again, it preempts the D/O proxy execution.

3. D is EXT, O is FAIR, T is EXT

This cannot represent T preempting O because EXT is below FAIR.

4. D is EXT, O is FAIR, T is FAIR

D cannot boost O above T because EXT is below FAIR. O and T continue competing under FAIR. Once O releases M, D wakes and resumes normal EXT scheduling.

5. D is FAIR, O is EXT, T is EXT

D preempts T as the higher-class scheduling context. O executes with D's FAIR priority and runtime budget, while T waits in EXT. D is not visible to the BPF scheduler.

6. D is FAIR, O is EXT, T is FAIR

D competes with T according to its FAIR deadline. When D is selected, O executes with D's FAIR priority and runtime budget. D is not visible to the BPF scheduler.

7. D is FAIR, O is FAIR, T is EXT

This cannot represent T preempting O because EXT is below FAIR.

8. D is FAIR, O is FAIR, T is FAIR

O, T, and D all have FAIR scheduling contexts. D remains runnable as a blocked proxy donor. When CFS selects D, O executes using D's FAIR scheduling context. When CFS selects O, O executes using its own FAIR context, and when CFS selects T, T executes normally. D is not visible to the BPF scheduler.

Thus, sched_ext policy and accounting must generally use rq->donor, the scheduler-selected task which supplies the scheduling context, rather than rq->curr, the task whose code physically executes. Without proxy execution they are the same task.

On nohz_full CPUs, a blocked proxy donor must retain the scheduler tick even when it has an infinite slice. Otherwise, a full dynticks CPU could stop the tick while rq->curr and rq->donor differ, violating assumptions made by the remote NOHZ tick path.

This is a conservative compromise that keeps the change local to sched_ext, at the cost of a periodic tick while a blocked proxy donor is selected. Allowing blocked proxy donors to run tickless would require making the core scheduler's remote tick handling aware that rq->curr and rq->donor can differ.

Moreover, extend scx_dump_state() to report both contexts. Each CPU record now includes a donor= line. If an EXT donor differs from rq->curr, also emit its detailed task record. The existing '*' marker continues to identify rq->curr, while the donor= line identifies the otherwise unmarked donor record.

Note that at this point in the series, CONFIG_SCHED_PROXY_EXEC still depends on !CONFIG_SCHED_CLASS_EXT, so proxy execution and sched_ext cannot be enabled together. The scheduling changes are therefore preparatory. A later patch removes this restriction.

---
 Documentation/scheduler/sched-ext.rst |   6 ++
 kernel/sched/ext/ext.c                | 115 +++++++++++++++++---------
 kernel/sched/ext/sub.h                |   8 +-
 3 files changed, 86 insertions(+), 43 deletions(-)
Documentation/scheduler/sched-ext.rst +6 -0
index ad2fff3c05937..db1ef89ed8f2a 100644
@@ -487,6 +487,12 @@ and edge cases, to name a few examples:
   class, in which case it will exit the tick-dispatch loop even though it is runnable
   and has a non-zero slice.
 
+* Under proxy execution, sched_ext continues to observe the donor as the current
+  scheduling context. A blocked donor does not enter an ``ops.running()`` /
+  ``ops.stopping()`` session because it does not execute itself, and the lock
+  owner executing on its behalf is intentionally not reported through these
+  callbacks.
+
 See the "Scheduling Cycle" section for a more detailed description of how
 a freshly woken up task gets on a CPU.
 
kernel/sched/ext/ext.c +76 -39

292 lines not on this page · raw message

kernel/sched/ext/sub.h +4 -4
index b5870aeb9c625..0579e32015d17 100644
@@ -153,14 +153,14 @@ static inline u64 scx_caps_for_task(struct task_struct *p)
 	return SCX_CAP_ENQ;
 }
 
-/* the cap @sch needs to preempt @rq's current task, 0 if none */
+/* the cap @sch needs to preempt @rq's current scheduling context, 0 if none */
 static inline u64 scx_caps_for_preempt(struct scx_sched *sch, struct rq *rq)
 {
-	struct task_struct *curr = rq->curr;
+	struct task_struct *donor = rq->donor;
 
 	/* a non-ext task can't be preempted by ext, own-subtree needs no cap */
-	if (curr->sched_class != &ext_sched_class ||
-	    scx_is_descendant(scx_task_sched(curr), sch))
+	if (donor->sched_class != &ext_sched_class ||
+	    scx_is_descendant(scx_task_sched(donor), sch))
 		return 0;
 	return SCX_CAP_PREEMPT;
 }
Signature · 1 line
2.55.0
#22 lore raw

Re: [PATCH 11/15] sched_ext: Split curr|donor references properly

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] Documentation incorrectly claims blocked donors bypass ops.running()/ops.stopping()…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] Documentation incorrectly claims blocked donors bypass ops.running()/ops.stopping(), contradicting the implementation --

--- Patch [11]: [PATCH 11/15] sched_ext: Split curr|donor references properly --- commit 5cd9f3cfe50f459bfe5f1a7d7f7b8af97933ae95
Author: Andrea Righi <[email protected]>

sched_ext: Split curr|donor references properly

This commit updates sched_ext to use rq->donor as the scheduling context under proxy execution instead of rq->curr. It updates references in various scheduler hooks and BPF kfuncs, and extends scx_dump_state() to report both contexts.

Documentation/scheduler/sched-ext.rst and edge cases, to name a few examples: · line 494

Quoted 13 lines
Documentation/scheduler/sched-ext.rst +5 -0
index ad2fff3c05937..db1ef89ed8f2a 100644
@@ -487,6 +487,12 @@ and edge cases, to name a few examples:
   class, in which case it will exit the tick-dispatch loop even though it is runnable
   and has a non-zero slice.
 
+* Under proxy execution, sched_ext continues to observe the donor as the current
+  scheduling context. A blocked donor does not enter an ``ops.running()`` /
+  ``ops.stopping()`` session because it does not execute itself, and the lock
+  owner executing on its behalf is intentionally not reported through these
+  callbacks.

[Severity: Medium]
Is this documentation accurate regarding the implementation?

The documentation states that a blocked donor bypasses ops.running() and ops.stopping(). However, the implementation invokes ops.running() via scx_proxy_donor_start() when the proxy execution context is resolved, and ops.stopping() is invoked via put_prev_task().

Could this contradiction mislead BPF scheduler authors into believing the task lifecycle state machine is bypassed for blocked donors, when in fact the running session is maintained?

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=11
author reviewer
#23 lore raw

PATCH 12/15 sched_ext: Delegate proxy donor admission to BPF schedulers

In reply to Andrea Righi

Proxy execution keeps a mutex-blocked donor runnable so that its scheduling context can execute the mutex owner.

Define SCX_OPS_ENQ_BLOCKED as both an admission contract and an ownership rule for retained donors. Schedulers without the flag block EXT donors normally. Schedulers with the flag continue to own blocked donors and receive them through ops.enqueue() with SCX_ENQ_BLOCKED, allowing BPF to choose their DSQ, CPU and ordering.

Once BPF dispatches a donor, proxy execution may move its scheduling context to the mutex owner's rq. This move is distinct from BPF
placement: wake_cpu continues to identify the donor's callback CPU, while task_cpu() identifies the proxy CPU. When the donor wakes, the normal wakeup path uses wake_cpu as prev_cpu when returning the task to the BPF scheduler, which can then choose its next placement normally.

Apply the same admission contract across scheduler ownership changes: a retained donor moving into a scheduler without SCX_OPS_ENQ_BLOCKED is fully deactivated and remains on the mutex wait path; a donor moving into an opted-in scheduler remains queued and is handed to that scheduler. Tasks without an assigned scheduler retain the generic proxy-execution fallback during the transition.

Blocked-donor enqueueing takes precedence over the exiting and migration-disabled local-DSQ fallbacks, so that an opted-in scheduler sees every eligible donor request.

Putting these rules together, the donor admission flow is the following:

    D ------ blocked on -----> M ------ owned by -----> O
 [donor]                    [mutex]                [owner]
    |
    v
 +--------------------------+
 | SCX_OPS_ENQ_BLOCKED set? |
 +------------+-------------+
          no  |  yes
       +------+-------+
       |              |
       v              v
 block normally   D runnable
                      |
                      v
          ops.enqueue(D, SCX_ENQ_BLOCKED)
                      |
                      v
             BPF admission policy
                      |
                  D selected
                      |
                      v
             proxy-exec resolves
                 D -> M -> O
                      |
                      v
               rq->donor = D
               rq->curr  = O
                      |
                O releases M
                      |
                      v
               D returns through
                the wakeup path
                      |
                      v
              BPF handles D as
              an unblocked task
---
 include/linux/sched/ext.h                     |   1 +
 kernel/sched/ext/ext.c                        | 155 ++++++++++++++++--
 kernel/sched/ext/internal.h                   |  23 ++-
 kernel/sched/ext/sub.c                        |   6 +-
 tools/sched_ext/include/scx/compat.h          |   1 +
 .../sched_ext/include/scx/enum_defs.autogen.h |   1 +
 .../sched_ext/include/scx/enums.autogen.bpf.h |   3 +
 tools/sched_ext/include/scx/enums.autogen.h   |   1 +
 8 files changed, 171 insertions(+), 20 deletions(-)
include/linux/sched/ext.h +1 -0
index 53da9eea0c925..8944e96fd4673 100644
@@ -103,6 +103,7 @@ enum scx_ent_flags {
 	SCX_TASK_SUB_INIT	= 1 << 4, /* task being initialized for a sub sched */
 	SCX_TASK_IMMED		= 1 << 5, /* task is on local DSQ with %SCX_ENQ_IMMED */
 	SCX_TASK_RUN_TRACKED	= 1 << 6, /* task is in an ops.running()/stopping() session */
+	SCX_TASK_ENQ_WAKEUP	= 1 << 7, /* wakeup enqueue awaiting wakeup_preempt() */
 
 	/*
 	 * Bits 8 to 10 are used to carry task state:
kernel/sched/ext/ext.c +138 -17

226 lines not on this page · raw message

kernel/sched/ext/internal.h +22 -1
index c3b1f9a5cf3a9..40bc2ec217a79 100644
@@ -214,6 +214,19 @@ enum scx_ops_flags {
 	 */
 	SCX_OPS_TID_TO_TASK		= 1LLU << 8,
 
+	/*
+	 * If set, mutex-blocked tasks remain runnable as proxy donors and are
+	 * passed to ops.enqueue() with %SCX_ENQ_BLOCKED. The BPF scheduler controls
+	 * when donors are dispatched and whether they should preempt other work.
+	 *
+	 * If clear, mutex-blocked tasks are removed from the runqueue normally
+	 * and cannot donate their scheduling context through proxy execution.
+	 *
+	 * For blocked donors, this flag takes precedence over
+	 * %SCX_OPS_ENQ_EXITING and %SCX_OPS_ENQ_MIGRATION_DISABLED.
+	 */
+	SCX_OPS_ENQ_BLOCKED		= 1LLU << 9,
+
 	SCX_OPS_ALL_FLAGS		= SCX_OPS_KEEP_BUILTIN_IDLE |
 					  SCX_OPS_ENQ_LAST |
 					  SCX_OPS_ENQ_EXITING |
@@ -222,7 +235,8 @@ enum scx_ops_flags {
 					  SCX_OPS_SWITCH_PARTIAL |
 					  SCX_OPS_BUILTIN_IDLE_PER_NODE |
 					  SCX_OPS_ALWAYS_ENQ_IMMED |
-					  SCX_OPS_TID_TO_TASK,
+					  SCX_OPS_TID_TO_TASK |
+					  SCX_OPS_ENQ_BLOCKED,
 
 	/* high 8 bits are internal, don't include in SCX_OPS_ALL_FLAGS */
 	__SCX_OPS_INTERNAL_MASK		= 0xffLLU << 56,
@@ -1678,6 +1692,12 @@ enum scx_enq_flags {
 	 */
 	SCX_ENQ_LAST		= 1LLU << 41,
 
+	/*
+	 * The task is blocked on a mutex and is being kept runnable as a proxy
+	 * donor. Only passed to ops.enqueue() when %SCX_OPS_ENQ_BLOCKED is set.
+	 */
+	SCX_ENQ_BLOCKED		= 1LLU << 42,
+
 	/* high 8 bits are internal */
 	__SCX_ENQ_INTERNAL_MASK	= 0xffLLU << 56,
 
@@ -1974,6 +1994,7 @@ void scx_task_iter_start(struct scx_task_iter *iter, struct cgroup *cgrp);
 void scx_task_iter_unlock(struct scx_task_iter *iter);
 void scx_task_iter_stop(struct scx_task_iter *iter);
 struct task_struct *scx_task_iter_next_locked(struct scx_task_iter *iter);
+void scx_prepare_task_sched_change(struct task_struct *p, struct scx_sched *sch);
 void scx_dispatch_dequeue(struct rq *rq, struct task_struct *p);
 void scx_do_enqueue_task(struct rq *rq, struct task_struct *p, u64 enq_flags,
 			 int sticky_cpu);
kernel/sched/ext/sub.c +4 -2
index 640e4a4f3c07f..0fc08781e0347 100644
@@ -751,8 +751,9 @@ static void scx_rehome_task(struct scx_sched *to, struct task_struct *p)
 	lockdep_assert_held(&p->pi_lock);
 	lockdep_assert_rq_held(task_rq(p));
 
+	scx_prepare_task_sched_change(p, to);
 	scoped_guard (sched_change, p, p->sched_class,
-		      DEQUEUE_SAVE | DEQUEUE_MOVE) {
+		      DEQUEUE_SAVE | DEQUEUE_MOVE | DEQUEUE_NOCLOCK) {
 		scx_disable_and_exit_task(scx_task_sched(p), p);
 		scx_set_task_state(p, SCX_TASK_INIT_BEGIN);
 		scx_set_task_state(p, SCX_TASK_INIT);
@@ -782,8 +783,9 @@ static void scx_punt_task(struct scx_sched *to, struct task_struct *p)
 	lockdep_assert_rq_held(task_rq(p));
 	WARN_ON_ONCE(!READ_ONCE(to->bypass_depth));
 
+	scx_prepare_task_sched_change(p, to);
 	scoped_guard (sched_change, p, p->sched_class,
-		      DEQUEUE_SAVE | DEQUEUE_MOVE) {
+		      DEQUEUE_SAVE | DEQUEUE_MOVE | DEQUEUE_NOCLOCK) {
 		scx_disable_and_exit_task(scx_task_sched(p), p);
 		scx_set_task_sched(p, to);
 	}
tools/sched_ext/include/scx/compat.h +1 -0
index 7757252d52e21..0e83ebfbee2b5 100644
@@ -117,6 +117,7 @@ static inline bool __COMPAT_struct_has_field(const char *type, const char *field
 #define SCX_OPS_ALLOW_QUEUED_WAKEUP SCX_OPS_FLAG(SCX_OPS_ALLOW_QUEUED_WAKEUP)
 #define SCX_OPS_BUILTIN_IDLE_PER_NODE SCX_OPS_FLAG(SCX_OPS_BUILTIN_IDLE_PER_NODE)
 #define SCX_OPS_ALWAYS_ENQ_IMMED SCX_OPS_FLAG(SCX_OPS_ALWAYS_ENQ_IMMED)
+#define SCX_OPS_ENQ_BLOCKED SCX_OPS_FLAG(SCX_OPS_ENQ_BLOCKED)
 
 #define SCX_PICK_IDLE_FLAG(name) __COMPAT_ENUM_OR_ZERO("scx_pick_idle_cpu_flags", #name)
 
tools/sched_ext/include/scx/enum_defs.autogen.h +1 -0
index 0379eff117c9e..d0b7516a05d96 100644
@@ -73,6 +73,7 @@
 #define HAVE_SCX_ENQ_IMMED
 #define HAVE_SCX_ENQ_REENQ
 #define HAVE_SCX_ENQ_LAST
+#define HAVE_SCX_ENQ_BLOCKED
 #define HAVE___SCX_ENQ_INTERNAL_MASK
 #define HAVE_SCX_ENQ_CLEAR_OPSS
 #define HAVE_SCX_ENQ_DSQ_PRIQ
tools/sched_ext/include/scx/enums.autogen.bpf.h +3 -0
index dafccbb6b69d2..7efe7b9346b49 100644
@@ -130,6 +130,9 @@ const volatile u64 __SCX_ENQ_REENQ __weak;
 const volatile u64 __SCX_ENQ_LAST __weak;
 #define SCX_ENQ_LAST __SCX_ENQ_LAST
 
+const volatile u64 __SCX_ENQ_BLOCKED __weak;
+#define SCX_ENQ_BLOCKED __SCX_ENQ_BLOCKED
+
 const volatile u64 __SCX_ENQ_CLEAR_OPSS __weak;
 #define SCX_ENQ_CLEAR_OPSS __SCX_ENQ_CLEAR_OPSS
 
tools/sched_ext/include/scx/enums.autogen.h +1 -0
index bbd4901f4fce3..f8fbeb7fbf95b 100644
@@ -47,6 +47,7 @@
 	SCX_ENUM_SET(skel, scx_enq_flags, SCX_ENQ_IMMED); \
 	SCX_ENUM_SET(skel, scx_enq_flags, SCX_ENQ_REENQ); \
 	SCX_ENUM_SET(skel, scx_enq_flags, SCX_ENQ_LAST); \
+	SCX_ENUM_SET(skel, scx_enq_flags, SCX_ENQ_BLOCKED); \
 	SCX_ENUM_SET(skel, scx_enq_flags, SCX_ENQ_CLEAR_OPSS); \
 	SCX_ENUM_SET(skel, scx_enq_flags, SCX_ENQ_DSQ_PRIQ); \
 	SCX_ENUM_SET(skel, scx_deq_flags, SCX_DEQ_SCHED_CHANGE); \
Signature · 1 line
2.55.0
reviewer
#24 lore raw

Re: [PATCH 12/15] sched_ext: Delegate proxy donor admission to BPF schedulers

On Tue, Jul 28, 2026 at 05:43:30PM +0200, Andrea Righi wrote:
...

Quoted 6 lines
kernel/sched/ext/ext.c +6 -0
+/*
+ * Called with @p's pi and rq locks held immediately before
+ * sched_change_begin(). The caller must pass DEQUEUE_NOCLOCK so the rq clock
+ * is updated only once.
+ */
+void scx_prepare_task_sched_change(struct task_struct *p, struct scx_sched *sch)

kernel/sched/ext/ext.c DEFINE_RAW_SPINLOCK(scx_sched_lock); · line 74

Quoted 10 lines
kernel/sched/ext/ext.c +9 -0
+{
+	lockdep_assert_held(&p->pi_lock);
+	lockdep_assert_rq_held(task_rq(p));
+
+	update_rq_clock(task_rq(p));
+
+	/* Block retained donors that the incoming scheduler cannot manage. */
+	if (!(sch->ops.flags & SCX_OPS_ENQ_BLOCKED))
+		sched_proxy_block_task(task_rq(p), p);
 }

What are the cases that this one catches that scx_allow_proxy_exec() or prepare_switch_scx() doesn't?

kernel/sched/ext/ext.c static void wakeup_preempt_scx(struct rq *rq, struct task_struct *p, int wake_fl · line 2358

Quoted 11 lines
kernel/sched/ext/ext.c +5 -2
@@ -2299,11 +2351,24 @@ static void wakeup_preempt_scx(struct rq *rq, struct task_struct *p, int wake_fl
 {
 	/*
 	 * Preemption between SCX tasks is implemented by resetting the victim
-	 * task's slice to 0 and triggering reschedule on the target CPU.
-	 * Nothing to do.
+	 * task's slice to 0 and triggering reschedule on the target CPU. A
+	 * mutex-blocked task is kept queued for proxy execution, so its wakeup
+	 * doesn't go through enqueue_task_scx(). If the BPF scheduler manages
+	 * blocked donors, reschedule explicitly so that it can reconsider a
+	 * donor it declined to dispatch while blocked.

Can you make this a separate paragraph and is the comment uptodate? I'm having a difficulty understanding what "if the BPF scheduler manages blocked donors" mean.

Quoted 4 lines
kernel/sched/ext/ext.c +2 -1
 	 */
-	if (p->sched_class == &ext_sched_class)
+	if (p->sched_class == &ext_sched_class) {
+		bool enq_wakeup = p->scx.flags & SCX_TASK_ENQ_WAKEUP;

kernel/sched/ext/ext.c static void wakeup_preempt_scx(struct rq *rq, struct task_struct *p, int wake_fl · line 2371

Quoted 10 lines
kernel/sched/ext/ext.c +9 -0
+
+		p->scx.flags &= ~SCX_TASK_ENQ_WAKEUP;
+		if (!enq_wakeup && p->is_blocked) {
+			struct scx_sched *sch = scx_task_sched(p);
+
+			if (sch && (sch->ops.flags & SCX_OPS_ENQ_BLOCKED))
+				resched_curr(rq);
+		}
 		return;
+	}

My understanding of what happens here is hazy. I suppose this is for the case of an active proxy execution being preempted by another SCX task? I'm not following why resched_curr() is needed here.

Quoted 21 lines
kernel/sched/ext/ext.c +17 -0
@@ -3198,6 +3279,37 @@ static void put_prev_task_scx(struct rq *rq, struct task_struct *p,
 	if (p->scx.flags & SCX_TASK_QUEUED) {
 		set_task_runnable(rq, p);
 
+		/*
+		 * The rq lock has remained held since scx_allow_proxy_exec(), so
+		 * @p's scheduler association cannot have changed. An associated
+		 * donor stays queued only when its BPF scheduler enables
+		 * %SCX_OPS_ENQ_BLOCKED; delegate its admission to that scheduler.
+		 *
+		 * If @sch is NULL, @p is transitioning into the root scheduler. The
+		 * root is published before tasks enter EXT and cannot be cleared while
+		 * this rq is locked. Preserve generic proxy execution by placing the
+		 * donor directly on the local DSQ.
+		 */
+		if (p->is_blocked) {
+			/*
+			 * If the donor is the same and only the mutex owner
+			 * changes, avoid triggering another ops.enqueue(): the
+			 * BPF scheduler has already admitted the donor, so it
+			 * can continue running.

kernel/sched/ext/ext.c static void put_prev_task_scx(struct rq *rq, struct task_struct *p, · line 3308

Quoted 10 lines
kernel/sched/ext/ext.c +10 -0
+			 */
+			if (next == p)
+				goto switch_class;
+
+			if (sch) {
+				WARN_ON_ONCE(!(sch->ops.flags & SCX_OPS_ENQ_BLOCKED));
+				scx_do_enqueue_task(rq, p, 0, -1);
+			} else {
+				scx_dispatch_enqueue(scx_root, rq, &rq->scx.local_dsq,
+						     p, 0);

Does this else arm actually happen? Can you describe the scenario? Oh, maybe below is the counterpart.

kernel/sched/ext/ext.c static void scx_root_enable_workfn(struct kthread_work *work) · line 7881

Quoted 8 lines
kernel/sched/ext/ext.c +4 -0
@@ -7758,6 +7875,10 @@ static void scx_root_enable_workfn(struct kthread_work *work)
 
 		if (old_class != new_class)
 			queue_flags |= DEQUEUE_CLASS;
+		if (old_class == new_class && new_class == &ext_sched_class) {
+			scx_prepare_task_sched_change(p, sch);
+			queue_flags |= DEQUEUE_NOCLOCK;
+		}

I'd appreciate if there's more explanation of what happens during enable. Wouldn't it be simpler if we just do sched_proxy_block_task() on all transitions and start with a clean slate?

Thanks.

Signature · 1 line
tejun
author reviewer
#25 lore raw

Re: [PATCH 12/15] sched_ext: Delegate proxy donor admission to BPF schedulers

Hi Tejun,

Quoted 13 lines · Tejun Heo

On Tue, Jul 28, 2026 at 05:43:30PM +0200, Andrea Righi wrote:
...

Quoted 11 lines
kernel/sched/ext/ext.c +11 -0
+/*
+ * Called with @p's pi and rq locks held immediately before
+ * sched_change_begin(). The caller must pass DEQUEUE_NOCLOCK so the rq clock
+ * is updated only once.
+ */
+void scx_prepare_task_sched_change(struct task_struct *p, struct scx_sched *sch)
+{
+	lockdep_assert_held(&p->pi_lock);
+	lockdep_assert_rq_held(task_rq(p));
+
+	update_rq_clock(task_rq(p));
Quoted 8 lines

kernel/sched/ext/ext.c DEFINE_RAW_SPINLOCK(scx_sched_lock); · line 74

Quoted 5 lines
kernel/sched/ext/ext.c +4 -0
+
+	/* Block retained donors that the incoming scheduler cannot manage. */
+	if (!(sch->ops.flags & SCX_OPS_ENQ_BLOCKED))
+		sched_proxy_block_task(task_rq(p), p);
 }

What are the cases that this one catches that scx_allow_proxy_exec() or prepare_switch_scx() doesn't?

scx_allow_proxy_exec() controls whether a task is retained when it first blocks in __schedule(), it doesn't handle a donor that was already retained before its scheduler ownership changes.

prepare_switch_scx() handles a scheduling-class transition into EXT, but it isn't called for an EXT-to-EXT scheduler change. The helper was intended to cover these same-class transitions, i.e., moving between parent and child sub-schedulers.

Thinking more about this, retained proxy execution can be terminated on all class changes centrally in sched_change_begin(). In this way we can remove the .prepare_switch() class callback and prepare_switch_scx(). We would still need to terminate retained proxy execution explicitly for EXT-to-EXT scheduler ownership changes, but that shouldn't be an issue. I'll test this approach, it should simplify the transition handling considerably.

Quoted 16 lines

kernel/sched/ext/ext.c static void wakeup_preempt_scx(struct rq *rq, struct task_struct *p, int wake_fl · line 2358

Quoted 12 lines
kernel/sched/ext/ext.c +5 -2
@@ -2299,11 +2351,24 @@ static void wakeup_preempt_scx(struct rq *rq, struct task_struct *p, int wake_fl
 {
 	/*
 	 * Preemption between SCX tasks is implemented by resetting the victim
-	 * task's slice to 0 and triggering reschedule on the target CPU.
-	 * Nothing to do.
+	 * task's slice to 0 and triggering reschedule on the target CPU. A
+	 * mutex-blocked task is kept queued for proxy execution, so its wakeup
+	 * doesn't go through enqueue_task_scx(). If the BPF scheduler manages
+	 * blocked donors, reschedule explicitly so that it can reconsider a
+	 * donor it declined to dispatch while blocked.

Can you make this a separate paragraph and is the comment uptodate? I'm having a difficulty understanding what "if the BPF scheduler manages blocked donors" mean.

"manages blocked donors" means the BPF scheduler sets SCX_OPS_ENQ_BLOCKED. I'll split the comment and clarify it.

Quoted 11 lines
Quoted 11 lines
kernel/sched/ext/ext.c +8 -1
 	 */
-	if (p->sched_class == &ext_sched_class)
+	if (p->sched_class == &ext_sched_class) {
+		bool enq_wakeup = p->scx.flags & SCX_TASK_ENQ_WAKEUP;
+
+		p->scx.flags &= ~SCX_TASK_ENQ_WAKEUP;
+		if (!enq_wakeup && p->is_blocked) {
+			struct scx_sched *sch = scx_task_sched(p);
+
+			if (sch && (sch->ops.flags & SCX_OPS_ENQ_BLOCKED))
Quoted 8 lines

kernel/sched/ext/ext.c static void wakeup_preempt_scx(struct rq *rq, struct task_struct *p, int wake_fl · line 2371

Quoted 4 lines
kernel/sched/ext/ext.c +3 -0
+				resched_curr(rq);
+		}
 		return;
+	}

My understanding of what happens here is hazy. I suppose this is for the case of an active proxy execution being preempted by another SCX task? I'm not following why resched_curr() is needed here.

The relevant case is a mutex waiter receiving a wakeup while it's retained on the rq as a proxy donor. Although the task is basically blocked and cannot execute itself, its scheduling context remains on the rq, so that the mutex owner can execute through it.

There are two wakeup paths when the mutex is released:

1) If the donor was not proxy-migrated and is still on its callback rq, ttwu_runnable() handles the wakeup while the task remains on the rq. It calls wakeup_preempt() and then clears p->is_blocked, without calling enqueue_task_scx(). BPF doesn't receive any new ops.enqueue() notification. resched_curr() requests another scheduling cycle so that ops.dispatch() can reconsider the now-unblocked task (BPF scheduler may have kept the blocked donor in a BPF-managed queue).

2) If the donor was proxy-migrated to the owner's rq, proxy_needs_return() removes it from that rq and the wakeup proceeds through the full activation path. That path calls enqueue_task_scx() before wakeup_preempt(). SCX_TASK_ENQ_WAKEUP records that this enqueue already happened, preventing the additional resched_curr().

Quoted 27 lines
Quoted 27 lines
kernel/sched/ext/ext.c +22 -0
@@ -3198,6 +3279,37 @@ static void put_prev_task_scx(struct rq *rq, struct task_struct *p,
 	if (p->scx.flags & SCX_TASK_QUEUED) {
 		set_task_runnable(rq, p);
 
+		/*
+		 * The rq lock has remained held since scx_allow_proxy_exec(), so
+		 * @p's scheduler association cannot have changed. An associated
+		 * donor stays queued only when its BPF scheduler enables
+		 * %SCX_OPS_ENQ_BLOCKED; delegate its admission to that scheduler.
+		 *
+		 * If @sch is NULL, @p is transitioning into the root scheduler. The
+		 * root is published before tasks enter EXT and cannot be cleared while
+		 * this rq is locked. Preserve generic proxy execution by placing the
+		 * donor directly on the local DSQ.
+		 */
+		if (p->is_blocked) {
+			/*
+			 * If the donor is the same and only the mutex owner
+			 * changes, avoid triggering another ops.enqueue(): the
+			 * BPF scheduler has already admitted the donor, so it
+			 * can continue running.
+			 */
+			if (next == p)
+				goto switch_class;
+
+			if (sch) {
Quoted 8 lines

kernel/sched/ext/ext.c static void put_prev_task_scx(struct rq *rq, struct task_struct *p, · line 3308

Quoted 5 lines
kernel/sched/ext/ext.c +5 -0
+				WARN_ON_ONCE(!(sch->ops.flags & SCX_OPS_ENQ_BLOCKED));
+				scx_do_enqueue_task(rq, p, 0, -1);
+			} else {
+				scx_dispatch_enqueue(scx_root, rq, &rq->scx.local_dsq,
+						     p, 0);

Does this else arm actually happen? Can you describe the scenario? Oh, maybe below is the counterpart.

Right, this was intended for the root-enable transition where a task could already be on the EXT class but not yet have an associated BPF scheduler. In that window scx_allow_proxy_exec() permits generic proxy execution and sch can be NULL.

IF we call sched_proxy_block_task() unconditionally, root enable will block any retained donor before establishing the new scheduler ownership, so the NULL-scheduler fallback is then unnecessary and we can remove it.

Quoted 13 lines

kernel/sched/ext/ext.c static void scx_root_enable_workfn(struct kthread_work *work) · line 7881

Quoted 9 lines
kernel/sched/ext/ext.c +4 -0
@@ -7758,6 +7875,10 @@ static void scx_root_enable_workfn(struct kthread_work *work)
 
 		if (old_class != new_class)
 			queue_flags |= DEQUEUE_CLASS;
+		if (old_class == new_class && new_class == &ext_sched_class) {
+			scx_prepare_task_sched_change(p, sch);
+			queue_flags |= DEQUEUE_NOCLOCK;
+		}

I'd appreciate if there's more explanation of what happens during enable. Wouldn't it be simpler if we just do sched_proxy_block_task() on all transitions and start with a clean slate?

Agreed. I'll change the logic so that a proxy session never survives a scheduling-class or BPF-scheduler ownership transition.

For scheduling-class changes, we can call sched_proxy_block_task() centrally from sched_change_begin(), this makes the new .prepare_switch() sched-class callback and prepare_switch_scx() unnecessary.

Thanks,
-Andrea

author reviewer
#26 lore raw

PATCH 13/15 sched_ext: Add selftest for blocked donor admission

In reply to Andrea Righi

SCX_OPS_ENQ_BLOCKED allows BPF schedulers to receive blocked proxy donors through ops.enqueue(). SCX_ENQ_BLOCKED identifies blocked-donor admission requests. Add selftest coverage for this interface.

Exercise a priority inversion using a weighted-vruntime BPF scheduler. A nice +19 owner holds a shared mutex, a nice -20 donor blocks on it, and nice 0 CPU contenders, one per allowed CPU, keep the system busy. Test both a same-CPU topology and a cross-CPU topology with the donor and owner on different CPUs.

Treat blocked donors according to the normal BPF ordering policy and assign the default slice on every enqueue, as for other tasks. Run each CPU placement configuration with SCX_OPS_ENQ_BLOCKED first disabled and then enabled, count blocked-donor enqueues by CPU and report average mutex hold and wait times. Verify that full wakeups are never reported as blocked-donor admissions by checking that SCX_ENQ_WAKEUP and SCX_ENQ_BLOCKED are not set together.

Proxy execution coverage requires CONFIG_SCHED_PROXY_EXEC=y, which the selftest config selects. Access to the kernel mutex is provided via a loadable kernel module, built through TEST_GEN_MODS_DIR and managed by the test.

---
 tools/testing/selftests/sched_ext/.gitignore  |   4 +
 tools/testing/selftests/sched_ext/Makefile    |   2 +
 tools/testing/selftests/sched_ext/config      |   2 +
 .../selftests/sched_ext/enq_blocked.bpf.c     | 116 +++
 .../testing/selftests/sched_ext/enq_blocked.c | 918 ++++++++++++++++++
 .../testing/selftests/sched_ext/enq_blocked.h |  28 +
 .../selftests/sched_ext/test_modules/Makefile |  13 +
 .../test_modules/scx_enq_blocked_test.c       | 195 ++++
 8 files changed, 1278 insertions(+)
 create mode 100644 tools/testing/selftests/sched_ext/enq_blocked.bpf.c
 create mode 100644 tools/testing/selftests/sched_ext/enq_blocked.c
 create mode 100644 tools/testing/selftests/sched_ext/enq_blocked.h
 create mode 100644 tools/testing/selftests/sched_ext/test_modules/Makefile
 create mode 100644 tools/testing/selftests/sched_ext/test_modules/scx_enq_blocked_test.c
tools/testing/selftests/sched_ext/.gitignore +4 -0
index ae5491a114c09..54a1fd2af713d 100644
@@ -4,3 +4,7 @@
 !Makefile
 !.gitignore
 !config
+!test_modules/
+!test_modules/scx_enq_blocked_test.c
+!test_modules/Makefile
+test_modules/*.mod.c
tools/testing/selftests/sched_ext/Makefile +2 -0
index 3cfe90e0f34fa..51a16b3d32d9b 100644
@@ -5,6 +5,7 @@ include ../../../scripts/Makefile.arch
 include ../../../scripts/Makefile.include
 
 TEST_GEN_PROGS := runner
+TEST_GEN_MODS_DIR := test_modules
 
 # override lib.mk's default rules
 OVERRIDE_TARGETS := 1
@@ -164,6 +165,7 @@ all_test_bpfprogs := $(foreach prog,$(wildcard *.bpf.c),$(INCLUDE_DIR)/$(patsubs
 auto-test-targets :=			\
 	create_dsq			\
 	dequeue				\
+	enq_blocked			\
 	enq_last_no_enq_fails		\
 	ddsp_bogus_dsq_fail		\
 	ddsp_vtimelocal_fail		\
tools/testing/selftests/sched_ext/config +2 -0
index aa901b05c8ad6..affa3cf33470a 100644
@@ -6,3 +6,5 @@ CONFIG_BPF=y
 CONFIG_BPF_SYSCALL=y
 CONFIG_DEBUG_INFO=y
 CONFIG_DEBUG_INFO_BTF=y
+CONFIG_EXPERT=y
+CONFIG_SCHED_PROXY_EXEC=y
tools/testing/selftests/sched_ext/enq_blocked.bpf.c new file +116 -0

116 lines not on this page · raw message

tools/testing/selftests/sched_ext/enq_blocked.c new file +918 -0

918 lines not on this page · raw message

tools/testing/selftests/sched_ext/enq_blocked.h new file +28 -0
new file mode 100644
index 0000000000000..ef1eb97feebe6
@@ -0,0 +1,28 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES */
+#ifndef __ENQ_BLOCKED_H
+#define __ENQ_BLOCKED_H
+
+#include <linux/ioctl.h>
+#include <linux/types.h>
+
+struct enq_blocked_stats {
+	__u64 hold_time_ns;
+	__u64 wait_time_ns;
+	__u64 nr_holds;
+	__u64 nr_waits;
+};
+
+#define ENQ_BLOCKED_IOCTL_OWNER	_IO('s', 1)
+#define ENQ_BLOCKED_IOCTL_DONOR	_IO('s', 2)
+#define ENQ_BLOCKED_IOCTL_RESET_STATS	_IO('s', 3)
+#define ENQ_BLOCKED_IOCTL_GET_STATS	_IOR('s', 4, struct enq_blocked_stats)
+#define ENQ_BLOCKED_IOCTL_PREP_ATTACH	_IO('s', 5)
+#define ENQ_BLOCKED_IOCTL_ATTACH_DONE	_IO('s', 6)
+#define ENQ_BLOCKED_IOCTL_DONOR_STATE	_IO('s', 7)
+#define ENQ_BLOCKED_IOCTL_PROXY_SUPPORTED _IO('s', 8)
+
+#define ENQ_BLOCKED_DONOR_SLEEPING	(1U << 0)
+#define ENQ_BLOCKED_DONOR_ON_RQ		(1U << 1)
+
+#endif /* __ENQ_BLOCKED_H */
tools/testing/selftests/sched_ext/test_modules/Makefile new file +13 -0
new file mode 100644
index 0000000000000..a0e9e9401ead6
@@ -0,0 +1,13 @@
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES
+
+TESTMODS_DIR := $(realpath $(dir $(abspath $(lastword $(MAKEFILE_LIST)))))
+KDIR ?= $(if $(O),$(O),$(realpath ../../../../..))
+
+obj-m += scx_enq_blocked_test.o
+
+all:
+	+$(Q)$(MAKE) -C $(KDIR) M=$(TESTMODS_DIR) modules
+
+clean:
+	+$(Q)$(MAKE) -C $(KDIR) M=$(TESTMODS_DIR) clean
tools/testing/selftests/sched_ext/test_modules/scx_enq_blocked_test.c new file +195 -0

195 lines not on this page · raw message

Signature · 1 line
2.55.0
author reviewer
#27 lore raw

PATCH 14/15 sched_ext: scx_qmap: Add proxy execution support

In reply to Andrea Righi

Add a -B option to opt scx_qmap into queueing mutex-blocked tasks for proxy execution. Without the option, SCX_OPS_ENQ_BLOCKED remains clear and mutex waiters block normally. With -B, blocked donors are passed to qmap_enqueue() with SCX_ENQ_BLOCKED.

When scx_qmap receives a blocked donor, dispatch it directly to the local DSQ of its current cid with a fresh slice and SCX_ENQ_PREEMPT. This places the donor at the head of the DSQ and requests an immediate reschedule, allowing the core proxy-exec path to run the mutex owner using the donor's scheduling context as soon as the donor is selected.

The blocked policy is intentionally unfair and can strongly prioritize tasks using contended mutexes, but scx_qmap is a demo scheduler and such aggressive behavior makes proxy-exec support easy to observe. Count these dispatch attempts in nr_enq_blocked and report their per-interval delta.

---
 tools/sched_ext/scx_qmap.bpf.c | 34 ++++++++++++++++++++++++++++++++++
 tools/sched_ext/scx_qmap.c     | 13 ++++++++++---
 tools/sched_ext/scx_qmap.h     |  1 +
 3 files changed, 45 insertions(+), 3 deletions(-)
tools/sched_ext/scx_qmap.bpf.c +34 -0
index 8822ed11c0d8b..ca806ade6c013 100644
@@ -451,6 +451,40 @@ void BPF_STRUCT_OPS(qmap_enqueue, struct task_struct *p, u64 enq_flags)
 	 */
 	taskc->core_sched_seq = qa.core_sched_tail_seqs[idx]++;
 
+	/*
+	 * A blocked donor rejected by a local DSQ must not be dispatched back
+	 * to the same DSQ immediately. Move it to the global DSQ so another CPU
+	 * can pick it up and kick an idle cid to accelerate its consumption.
+	 */
+	if ((enq_flags & (SCX_ENQ_BLOCKED | SCX_ENQ_REENQ)) ==
+	    (SCX_ENQ_BLOCKED | SCX_ENQ_REENQ)) {
+		taskc->force_local = false;
+		scx_bpf_dsq_insert(p, SHARED_DSQ, 0, enq_flags);
+		cid = cmask_next_and2_set_wrap(&taskc->cpus_allowed,
+					       &qa.idle_cids.mask,
+					       &qa.self_cids.mask, 0);
+		if (cid < scx_bpf_nr_cids())
+			scx_bpf_kick_cid(cid, SCX_KICK_IDLE);
+		return;
+	}
+
+	/*
+	 * Insert a blocked mutex donor at the head of its current cid's local
+	 * DSQ with a fresh slice and %SCX_ENQ_PREEMPT, requesting an immediate
+	 * reschedule. Once selected, the core proxy-exec path can immediately
+	 * run the mutex owner using the donor's scheduling context.
+	 *
+	 * This policy is intentionally unfair and can strongly prioritize tasks
+	 * using contended mutexes; scx_qmap is a demonstration scheduler and
+	 * this behavior makes proxy-exec support easy to observe.
+	 */
+	if (enq_flags & SCX_ENQ_BLOCKED) {
+		__sync_fetch_and_add(&qa.nr_enq_blocked, 1);
+		scx_bpf_dsq_insert(p, SCX_DSQ_LOCAL_ON | scx_bpf_task_cid(p),
+				   slice_ns, enq_flags | SCX_ENQ_PREEMPT);
+		return;
+	}
+
 	/*
 	 * A node with children delegates most cids. A task of ours that can run
 	 * on none of our self cids (e.g. a per-NUMA kthread pinned to delegated
tools/sched_ext/scx_qmap.c +10 -3
index 3f54796e48bef..e27bf8dddd4eb 100644
@@ -46,7 +46,7 @@ const char help_fmt[] =
 "See the top-of-file comment in .bpf.c for the design.\n"
 "\n"
 "Usage: %s [-s SLICE_US] [-e COUNT] [-t COUNT] [-T COUNT] [-l COUNT] [-b COUNT]\n"
-"       [-N COUNT] [-P] [-M] [-H] [-c CG_PATH] [-d PID] [-D LEN] [-S] [-p] [-I]\n"
+"       [-N COUNT] [-P] [-M] [-H] [-c CG_PATH] [-d PID] [-D LEN] [-S] [-p] [-I] [-B]\n"
 "       [-F COUNT] [-i SEC] [-R MS] [-J MODE] [-v]\n"
 "\n"
 "  -s SLICE_US   Override slice duration\n"
@@ -65,6 +65,7 @@ const char help_fmt[] =
 "  -S            Suppress qmap-specific debug dump\n"
 "  -p            Switch only tasks on SCHED_EXT policy instead of all\n"
 "  -I            Turn on SCX_OPS_ALWAYS_ENQ_IMMED\n"
+"  -B            Turn on SCX_OPS_ENQ_BLOCKED\n"
 "  -F COUNT      IMMED stress: force every COUNT'th enqueue to a busy local DSQ (use with -I)\n"
 "  -C MODE       cid-override test (shuffle|bad-dup|bad-range|bad-mono)\n"
 "  -i SEC        Stats interval, seconds (default 5)\n"
@@ -105,6 +106,7 @@ struct hier_prev {
 	u64 nr_dsps[MAX_SUB_SCHEDS];
 	u64 nr_reenq_cap;
 	u64 nr_reenq_immed;
+	u64 nr_enq_blocked;
 	u64 nr_inject_attempts;
 };
 
@@ -187,13 +189,15 @@ static void print_hier(struct qmap_arena *qa, struct hier_prev *prev, u64 own_cg
 	}
 
 	format_cid_ranges(qa, CID_SHARED, ranges, sizeof(ranges));
-	printf("hier   : nsub=%llu excl=%u shared=%s rr=%s reenq cap/immed +%llu/+%llu inj=+%llu\n",
+	printf("hier   : nsub=%llu excl=%u shared=%s rr=%s reenq cap/immed +%llu/+%llu blocked=+%llu inj=+%llu\n",
 	       (unsigned long long)qa->nr_sub_scheds, qa->part.nr_excl, ranges, rr,
 	       (unsigned long long)(qa->nr_reenq_cap - prev->nr_reenq_cap),
 	       (unsigned long long)(qa->nr_reenq_immed - prev->nr_reenq_immed),
+	       (unsigned long long)(qa->nr_enq_blocked - prev->nr_enq_blocked),
 	       (unsigned long long)(qa->nr_inject_attempts - prev->nr_inject_attempts));
 	prev->nr_reenq_cap = qa->nr_reenq_cap;
 	prev->nr_reenq_immed = qa->nr_reenq_immed;
+	prev->nr_enq_blocked = qa->nr_enq_blocked;
 	prev->nr_inject_attempts = qa->nr_inject_attempts;
 
 	printf("hier   : %-4s %10s %4s %6s %8s  %s\n",
@@ -256,7 +260,7 @@ int main(int argc, char **argv)
 	skel->rodata->slice_ns = __COMPAT_ENUM_OR_ZERO("scx_public_consts", "SCX_SLICE_DFL");
 	skel->rodata->max_tasks = 16384;
 
-	while ((opt = getopt(argc, argv, "s:e:t:T:l:b:N:PMHc:d:D:SpIF:C:i:R:J:vh")) != -1) {
+	while ((opt = getopt(argc, argv, "s:e:t:T:l:b:N:PMHc:d:D:SpIBF:C:i:R:J:vh")) != -1) {
 		switch (opt) {
 		case 's':
 			skel->rodata->slice_ns = strtoull(optarg, NULL, 0) * 1000;
@@ -318,6 +322,9 @@ int main(int argc, char **argv)
 			skel->rodata->always_enq_immed = true;
 			skel->struct_ops.qmap_ops->flags |= SCX_OPS_ALWAYS_ENQ_IMMED;
 			break;
+		case 'B':
+			skel->struct_ops.qmap_ops->flags |= SCX_OPS_ENQ_BLOCKED;
+			break;
 		case 'F':
 			skel->rodata->immed_stress_nth = strtoul(optarg, NULL, 0);
 			break;
tools/sched_ext/scx_qmap.h +1 -0
index c42f7ef74b894..7f4d3b33e6133 100644
@@ -174,6 +174,7 @@ struct qmap_arena {
 	/* bpf -> userspace: stats */
 	u64 nr_reenq_cap;		/* SCX_TASK_REENQ_CAP bounces */
 	u64 nr_reenq_immed;		/* SCX_TASK_REENQ_IMMED bounces */
+	u64 nr_enq_blocked;		/* SCX_ENQ_BLOCKED dispatches */
 	u64 nr_inject_attempts;		/* fault-injection: dispatches to an unheld cid */
 	u32 inject_mode;		/* fault-injection mode (QMAP_INJ_*) */
 };
Signature · 1 line
2.55.0
reviewer
#28 lore raw

Re: [PATCH 14/15] sched_ext: scx_qmap: Add proxy execution support

tools/sched_ext/scx_qmap.bpf.c void BPF_STRUCT_OPS(qmap_enqueue, struct task_struct *p, u64 enq_flags) · line 457

Quoted 3 lines · Andrea Righi
tools/sched_ext/scx_qmap.bpf.c +3 -0
+	 * A blocked donor rejected by a local DSQ must not be dispatched back
+	 * to the same DSQ immediately. Move it to the global DSQ so another CPU
+	 * can pick it up and kick an idle cid to accelerate its consumption.

My mental picture isn't complete. Can you expand on why this block is necessary?

Thanks.

Signature · 1 line
tejun
author reviewer
#29 lore raw

Re: [PATCH 14/15] sched_ext: scx_qmap: Add proxy execution support

Quoted 7 lines · Tejun Heo

tools/sched_ext/scx_qmap.bpf.c void BPF_STRUCT_OPS(qmap_enqueue, struct task_struct *p, u64 enq_flags) · line 457

Quoted 3 lines · Andrea Righi
tools/sched_ext/scx_qmap.bpf.c +3 -0
+	 * A blocked donor rejected by a local DSQ must not be dispatched back
+	 * to the same DSQ immediately. Move it to the global DSQ so another CPU
+	 * can pick it up and kick an idle cid to accelerate its consumption.

My mental picture isn't complete. Can you expand on why this block is necessary?

This was intended to support SCX_OPS_ALWAYS_ENQ_IMMED. A blocked donor is inserted into a local DSQ, but if the donor can't run immediately, it goes back to ops.enqueue() when ENQ_IMMED is set. Maybe we can move this condition in the main SCX_ENQ_REENQ block at the top (explaining this special case with proxy exec)?

-Andrea

author reviewer
#30 lore raw

PATCH 15/15 sched: Allow enabling proxy exec with sched_ext

In reply to Andrea Righi

Now that sched_ext can handle proxy donors and BPF schedulers can opt in to blocked-donor enqueueing with SCX_OPS_ENQ_BLOCKED, remove the !SCHED_CLASS_EXT dependency from SCHED_PROXY_EXEC and allow both options to be enabled together.

---
 init/Kconfig | 2 --
 1 file changed, 2 deletions(-)
init/Kconfig +0 -2
index 5230d4879b1c8..0817e62266e03 100644
@@ -934,8 +934,6 @@ config SCHED_PROXY_EXEC
 	bool "Proxy Execution"
 	# Avoid some build failures w/ PREEMPT_RT until it can be fixed
 	depends on !PREEMPT_RT
-	# Need to investigate how to inform sched_ext of split contexts
-	depends on !SCHED_CLASS_EXT
 	# Not particularly useful until we get to multi-rq proxying
 	depends on EXPERT
 	help
Signature · 1 line
2.55.0
reviewer
#31 lore raw

Re: [PATCH 15/15] sched: Allow enabling proxy exec with sched_ext

Quoted 16 lines · Andrea Righi

Now that sched_ext can handle proxy donors and BPF schedulers can opt in to blocked-donor enqueueing with SCX_OPS_ENQ_BLOCKED, remove the !SCHED_CLASS_EXT dependency from SCHED_PROXY_EXEC and allow both options to be enabled together.

---
 init/Kconfig | 2 --
 1 file changed, 2 deletions(-)
init/Kconfig +0 -0
index 5230d4879b1c8..0817e62266e03 100644

init/Kconfig config SCHED_PROXY_EXEC · line 938

Quoted 5 lines
init/Kconfig +0 -2
@@ -934,8 +934,6 @@ config SCHED_PROXY_EXEC
 	bool "Proxy Execution"
 	# Avoid some build failures w/ PREEMPT_RT until it can be fixed
 	depends on !PREEMPT_RT
-	# Need to investigate how to inform sched_ext of split contexts
-	depends on !SCHED_CLASS_EXT

Nice.

Signature · 1 line
tejun