tools/sched_ext: Handle pinned tasks in scx_central
PATCH 1/1 tools/sched_ext: Handle pinned tasks in scx_central
scx_central only fast-pathed per-cpu kthreads (PF_KTHREAD && nr_cpus_allowed == 1) directly to the local DSQ, leaving non-kthread tasks pinned to a single CPU to go through the central queue. When dispatch_to_cpu() later dequeued such a task and found it couldn't run on the current CPU, it could trigger scx_error() and disable the scheduler:
sched_ext: central: SCX_DSQ_LOCAL[_ON] cannot move migration disabled qemu-system-x86[223212] from CPU 1 to 0
Though scx_central is a demo scheduler and meant to dispatch all tasks from a central CPU, it should remain robust under any circumstances.
Extend the handling to all tasks with nr_cpus_allowed == 1:
- select_cpu() and enqueue(): return the task's current CPU and dispatch directly to local DSQ for pinned tasks.
- dispatch_to_cpu(): use SCX_KICK_PREEMPT when kicking a task's home CPU after bouncing to the fallback DSQ, so the CPU preempts immediately rather than waiting for the next timer tick.
Test plan:
- run scx_central on the host, then launch virtme-ng in a separate terminal to create pinned tasks (qemu-system-x86), and verify stable operation.
--- tools/sched_ext/scx_central.bpf.c | 24 +++++++++++++++++++----- 1 file changed, 19 insertions(+), 5 deletions(-)
tools/sched_ext/scx_central.bpf.c +19 -5
@@ -96,7 +96,13 @@ s32 BPF_STRUCT_OPS(central_select_cpu, struct task_struct *p,* disturbing other CPUs. It's safe to blindly return the central cpu as * select_cpu() is a hint and if @p can't be on it, the kernel will * automatically pick a fallback CPU. +* +* For tasks pinned to a single CPU, return the current CPU to avoid +* an unnecessary fallback. */ +if (p->nr_cpus_allowed ==1 ) +return scx_bpf_task_cpu (p); +return central_cpu; }
@@ -107,12 +113,13 @@ void BPF_STRUCT_OPS(central_enqueue, struct task_struct *p, u64 enq_flags)__sync_fetch_and_add (&nr_total,1 );/* -* Push per-cpu kthreads at the head of local dsq's and preempt the -* corresponding CPU. This ensures that e.g. ksoftirqd isn't blocked -* behind other threads which is necessary for forward progress -* guarantee as we depend on the BPF timer which may run from ksoftirqd. +* Push tasks pinned to a single CPU at the head of local dsq's and +* preempt the corresponding CPU. This ensures that e.g. ksoftirqd +* isn't blocked behind other threads which is necessary for forward +* progress guarantee as we depend on the BPF timer which may run +* from ksoftirqd. */ -if ((p->flags &PF_KTHREAD ) && p->nr_cpus_allowed ==1 ) { +if (p->nr_cpus_allowed ==1 ) {__sync_fetch_and_add (&nr_locals,1 );scx_bpf_dsq_insert (p,SCX_DSQ_LOCAL ,SCX_SLICE_INF , enq_flags |SCX_ENQ_PREEMPT );
@@ -155,6 +162,13 @@ static bool dispatch_to_cpu(s32 cpu)if (!bpf_cpumask_test_cpu (cpu, p->cpus_ptr)) {__sync_fetch_and_add (&nr_mismatches,1 );scx_bpf_dsq_insert (p,FALLBACK_DSQ_ID ,SCX_SLICE_INF ,0 ); + +/* +* Kick the task's home CPU to pick it up from the +* fallback DSQ promptly. +*/ +scx_bpf_kick_cpu (scx_bpf_task_cpu (p),SCX_KICK_PREEMPT ); +bpf_task_release (p);/* * We might run out of dispatch buffer slots if we continue dispatching
Signature · 1 line
2.48.1
Hi Cheng-Yang,
Quoted 6 lines · Cheng-Yang Chou
tools/sched_ext/scx_central.bpf.c +0
-2@@ -107,12 +113,13 @@ void BPF_STRUCT_OPS(central_enqueue, struct task_struct *p, u64 enq_flags)__sync_fetch_and_add (&nr_total,1 );/* -* Push per-cpu kthreads at the head of local dsq's and preempt the -* corresponding CPU. This ensures that e.g. ksoftirqd isn't blocked
tools/sched_ext/scx_central.bpf.c void BPF_STRUCT_OPS(central_enqueue, struct task_struct *p, u64 enq_flags) · line 122
Quoted 10 lines
tools/sched_ext/scx_central.bpf.c +6
-3-* behind other threads which is necessary for forward progress -* guarantee as we depend on the BPF timer which may run from ksoftirqd. +* Push tasks pinned to a single CPU at the head of local dsq's and +* preempt the corresponding CPU. This ensures that e.g. ksoftirqd +* isn't blocked behind other threads which is necessary for forward +* progress guarantee as we depend on the BPF timer which may run +* from ksoftirqd. */ -if ((p->flags &PF_KTHREAD ) && p->nr_cpus_allowed ==1 ) { +if (p->nr_cpus_allowed ==1 ) {
If you want to cover all cases, then I think you should also check is_migration_disabled(p). When a task disables migration, it is not reflected in p->cpus_ptr or p->nr_cpus_allowed until the task is switched out (see migrate_disable_switch()), which happens __after__ ops.enqueue(). So here, you could see p->nr_cpus_allowed > 1 even though the task has migration disabled.
Thanks,
Kuba
Hi Kuba,
Quoted 9 lines · Kuba Piecuch
Review
Quoted 2 lines
(quoted diff) +1
-1- if ((p->flags & PF_KTHREAD) && p->nr_cpus_allowed == 1) { + if (p->nr_cpus_allowed == 1) {If you want to cover all cases, then I think you should also check
is_migration_disabled(p). When a task disables migration, it is not reflected inp->cpus_ptrorp->nr_cpus_alloweduntil the task is switched out (seemigrate_disable_switch()), which happens__after__ops.enqueue(). So here, you could seep->nr_cpus_allowed> 1 even though the task has migration disabled.
Thanks for the feedback!
I've attached the v2 patch with the updated logic.
Signature · 2 lines
Thanks, Cheng-Yang
From 038e42a0a2c6d0b23e6914eaafcb0de301b70258 Mon Sep 17 00:00:00 2001
From: Cheng-Yang Chou <[email protected]>
Date: Mon, 13 Apr 2026 15:19:59 +0800
Subject: [PATCH v2 sched_ext/for-7.1-fixes] tools/sched_ext: Handle pinned tasks in scx_central
scx_central only fast-pathed per-cpu kthreads (PF_KTHREAD && nr_cpus_allowed == 1) directly to the local DSQ, leaving non-kthread tasks pinned to a single CPU to go through the central queue. When dispatch_to_cpu() later dequeued such a task and found it couldn't run on the current CPU, it could trigger scx_error() and disable the scheduler:
sched_ext: central: SCX_DSQ_LOCAL[_ON] cannot move migration disabled qemu-system-x86[223212] from CPU 1 to 0
Though scx_central is a demo scheduler and meant to dispatch all tasks from a central CPU, it should remain robust under any circumstances.
Extend the handling to all tasks with nr_cpus_allowed == 1:
- select_cpu() and enqueue(): bypass the central queue for pinned and migration-disabled tasks. enqueue() also checks is_migration_disabled() directly because p->nr_cpus_allowed lags behind migrate_disable_switch().
- dispatch_to_cpu(): use SCX_KICK_PREEMPT when bouncing to the fallback DSQ so the home CPU preempts immediately.
Test plan:
- run scx_central on the host, then launch virtme-ng in a separate terminal to create pinned tasks (qemu-system-x86), and verify stable operation.
---
v2: Add is_migration_disabled() check in enqueue() as p->nr_cpus_allowed
is not updated until migrate_disable_switch() runs during context switch, which happens after ops.enqueue(). (Kuba Piecuch)
tools/sched_ext/scx_central.bpf.c | 28 +++++++++++++++++++++++----- 1 file changed, 23 insertions(+), 5 deletions(-)
tools/sched_ext/scx_central.bpf.c +23 -5
@@ -96,7 +96,13 @@ s32 BPF_STRUCT_OPS(central_select_cpu, struct task_struct *p,* disturbing other CPUs. It's safe to blindly return the central cpu as * select_cpu() is a hint and if @p can't be on it, the kernel will * automatically pick a fallback CPU. +* +* For tasks pinned to a single CPU, return the current CPU to avoid +* an unnecessary fallback. */ +if (p->nr_cpus_allowed ==1 ) +return scx_bpf_task_cpu (p); +return central_cpu; }
@@ -107,12 +113,17 @@ void BPF_STRUCT_OPS(central_enqueue, struct task_struct *p, u64 enq_flags)__sync_fetch_and_add (&nr_total,1 );/* -* Push per-cpu kthreads at the head of local dsq's and preempt the -* corresponding CPU. This ensures that e.g. ksoftirqd isn't blocked -* behind other threads which is necessary for forward progress -* guarantee as we depend on the BPF timer which may run from ksoftirqd. +* Push tasks pinned to a single CPU or with migration disabled at +* the head of local dsq's and preempt the corresponding CPU. This +* ensures that e.g. ksoftirqd isn't blocked behind other threads +* which is necessary for forward progress guarantee as we depend on +* the BPF timer which may run from ksoftirqd. +* +* p->nr_cpus_allowed is not updated until migrate_disable_switch() +* runs during context switch, which happens after ops.enqueue(). Check +* is_migration_disabled() directly to catch tasks in this window. */ -if ((p->flags &PF_KTHREAD ) && p->nr_cpus_allowed ==1 ) { +if (p->nr_cpus_allowed ==1 ||is_migration_disabled (p)) {__sync_fetch_and_add (&nr_locals,1 );scx_bpf_dsq_insert (p,SCX_DSQ_LOCAL ,SCX_SLICE_INF , enq_flags |SCX_ENQ_PREEMPT );
@@ -155,6 +166,13 @@ static bool dispatch_to_cpu(s32 cpu)if (!bpf_cpumask_test_cpu (cpu, p->cpus_ptr)) {__sync_fetch_and_add (&nr_mismatches,1 );scx_bpf_dsq_insert (p,FALLBACK_DSQ_ID ,SCX_SLICE_INF ,0 ); + +/* +* Kick the task's home CPU to pick it up from the +* fallback DSQ promptly. +*/ +scx_bpf_kick_cpu (scx_bpf_task_cpu (p),SCX_KICK_PREEMPT ); +bpf_task_release (p);/* * We might run out of dispatch buffer slots if we continue dispatching
Signature · 1 line
2.48.1
Tejun Heo
reviewerHello,
Quoted 5 lines · Cheng-Yang Chou
tools/sched_ext/scx_central.bpf.c +0
-0
tools/sched_ext/scx_central.bpf.c s32 BPF_STRUCT_OPS(central_select_cpu, struct task_struct *p, · line 105
Quoted 10 lines
tools/sched_ext/scx_central.bpf.c +6
-0@@ -96,7 +96,13 @@ s32 BPF_STRUCT_OPS(central_select_cpu, struct task_struct *p,* disturbing other CPUs. It's safe to blindly return the central cpu as * select_cpu() is a hint and if @p can't be on it, the kernel will * automatically pick a fallback CPU. +* +* For tasks pinned to a single CPU, return the current CPU to avoid +* an unnecessary fallback. */ +if (p->nr_cpus_allowed ==1 ) +return scx_bpf_task_cpu (p); +
This isn't necessary. select_cpu() isn't called if nr_cpus_allowed == 1.
Quoted 10 lines
tools/sched_ext/scx_central.bpf.c +2
-4@@ -107,12 +113,17 @@ void BPF_STRUCT_OPS(central_enqueue, struct task_struct *p, u64 enq_flags)__sync_fetch_and_add (&nr_total,1 );/* -* Push per-cpu kthreads at the head of local dsq's and preempt the -* corresponding CPU. This ensures that e.g. ksoftirqd isn't blocked -* behind other threads which is necessary for forward progress -* guarantee as we depend on the BPF timer which may run from ksoftirqd. +* Push tasks pinned to a single CPU or with migration disabled at +* the head of local dsq's and preempt the corresponding CPU. This
tools/sched_ext/scx_central.bpf.c void BPF_STRUCT_OPS(central_enqueue, struct task_struct *p, u64 enq_flags) · line 126
Quoted 10 lines
tools/sched_ext/scx_central.bpf.c +8
-1+* ensures that e.g. ksoftirqd isn't blocked behind other threads +* which is necessary for forward progress guarantee as we depend on +* the BPF timer which may run from ksoftirqd. +* +* p->nr_cpus_allowed is not updated until migrate_disable_switch() +* runs during context switch, which happens after ops.enqueue(). Check +* is_migration_disabled() directly to catch tasks in this window. */ -if ((p->flags &PF_KTHREAD ) && p->nr_cpus_allowed ==1 ) { +if (p->nr_cpus_allowed ==1 ||is_migration_disabled (p)) {
Let's not expand the condition here. This is to guarantee forward progress of the scheduler itself.
tools/sched_ext/scx_central.bpf.c static bool dispatch_to_cpu(s32 cpu) · line 166
Quoted 2 lines
tools/sched_ext/scx_central.bpf.c +0
-0@@ -155,6 +166,13 @@ static bool dispatch_to_cpu(s32 cpu)if (!bpf_cpumask_test_cpu (cpu, p->cpus_ptr)) {
Instead, can you expand the condition above to "if this task can't run on the target CPU for whatever reason"?
Thanks.
Signature · 1 line
tejun
Hi Tejun,
Quoted 8 lines · Tejun Heo
Hello,
Review
Quoted 3 lines
(quoted diff) +3
-0+ if (p->nr_cpus_allowed == 1) + return scx_bpf_task_cpu(p); +This isn't necessary.
select_cpu()isn't called ifnr_cpus_allowed== 1.
Ack.
Quoted 8 lines
Review void BPF_STRUCT_OPS(central_enqueue, struct task_struct *p, u64 enq_flags) · line 115
Quoted 5 lines
(quoted diff) +1
-1@@ -107,12 +113,17 @@ void BPF_STRUCT_OPS(central_enqueue, struct task_struct *p, u64 enq_flags) __sync_fetch_and_add(&nr_total, 1); - if ((p->flags & PF_KTHREAD) && p->nr_cpus_allowed == 1) { + if (p->nr_cpus_allowed == 1 || is_migration_disabled(p)) {Let's not expand the condition here. This is to guarantee forward progress of the scheduler itself.
Ack.
Quoted 5 lines
Review static bool dispatch_to_cpu(s32 cpu) · line 166
Quoted 2 lines
(quoted diff) +0
-0@@ -155,6 +166,13 @@ static bool dispatch_to_cpu(s32 cpu) if (!bpf_cpumask_test_cpu(cpu, p->cpus_ptr)) {Instead, can you expand the condition above to "if this task can't run on the target CPU for whatever reason"?
Done!
Signature · 2 lines
Thanks, Cheng-Yang
From df62cc372cb9fab7aa9b666a6a127d8248923dd8 Mon Sep 17 00:00:00 2001
From: Cheng-Yang Chou <[email protected]>
Date: Mon, 13 Apr 2026 15:19:59 +0800
Subject: [PATCH v3 sched_ext/for-7.1] tools/sched_ext: Handle migration-disabled tasks in
scx_central
When a task calls migrate_disable(), p->cpus_ptr is not updated until migrate_disable_switch() runs during context switch, so dispatch_to_cpu() may dequeue such a task and dispatch it to a CPU it cannot run on.
Extend the mismatch check in dispatch_to_cpu() to also test is_migration_disabled() alongside the cpumask check, so tasks in this window are bounced to the fallback DSQ.
---
v3:
- Drop the select_cpu() shortcut, revert enqueue() to the kthread-only forward-progress path, and expand the dispatch_to_cpu() mismatch check to cover is_migration_disabled() (Tejun Heo)
- Drop kick_cpu() on mismatch (Andrea Righi)
- Reword commit subject and message with the fix
v2:
- Add is_migration_disabled() check in enqueue() as p->nr_cpus_allowed is not updated until migrate_disable_switch() runs during context switch, which happens after ops.enqueue(). (Kuba Piecuch)
tools/sched_ext/scx_central.bpf.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-)
tools/sched_ext/scx_central.bpf.c +7 -3
@@ -149,10 +149,14 @@ static bool dispatch_to_cpu(s32 cpu) }/* -* If we can't run the task at the top, do the dumb thing and -* bounce it to the fallback dsq. +* If we can't run the task at the top for whatever reason, +* bounce it to the fallback dsq. Also check +* is_migration_disabled() explicitly as p->cpus_ptr may not +* reflect the migration-disabled state yet if +* migrate_disable_switch() hasn't run. */ -if (!bpf_cpumask_test_cpu (cpu, p->cpus_ptr)) { +if (!bpf_cpumask_test_cpu (cpu, p->cpus_ptr) || + (is_migration_disabled (p) &&scx_bpf_task_cpu (p) != cpu)) {__sync_fetch_and_add (&nr_mismatches,1 );scx_bpf_dsq_insert (p,FALLBACK_DSQ_ID ,SCX_SLICE_INF ,0 );bpf_task_release (p);
Signature · 1 line
2.48.1
Kuba Piecuch
Reviewed-byQuoted 46 lines · Cheng-Yang Chou
From
df62cc372cb9fab7aa9b666a6a127d8248923dd8Mon Sep 17 00:00:00 2001
From: Cheng-Yang Chou <[email protected]>
Date: Mon, 13 Apr 2026 15:19:59 +0800
Subject: [PATCH v3 sched_ext/for-7.1]tools/sched_ext: Handle migration-disabled tasks in
scx_centralWhen a task calls
migrate_disable(),p->cpus_ptris not updated untilmigrate_disable_switch()runs during context switch, sodispatch_to_cpu()may dequeue such a task and dispatch it to a CPU it cannot run on.Extend the mismatch check in
dispatch_to_cpu()to also testis_migration_disabled()alongside the cpumask check, so tasks in this window are bounced to the fallback DSQ.Suggested-by Andrea Righi <[email protected]>Suggested-by Tejun Heo <[email protected]>Suggested-by Kuba Piecuch <[email protected]>Signed-off-by Cheng-Yang Chou <[email protected]>---
v3:
- Drop theselect_cpu()shortcut, revertenqueue()to the kthread-only forward-progress path, and expand thedispatch_to_cpu()mismatch check to coveris_migration_disabled()(Tejun Heo)
- Dropkick_cpu()on mismatch (Andrea Righi)
- Reword commit subject and message with the fixv2:
- Addis_migration_disabled()check inenqueue()asp->nr_cpus_allowedis not updated untilmigrate_disable_switch()runs during context switch, which happens afterops.enqueue(). (Kuba Piecuch)tools/sched_ext/scx_central.bpf.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-)tools/sched_ext/scx_central.bpf.c +2
-2@@ -149,10 +149,14 @@ static bool dispatch_to_cpu(s32 cpu) }/* -* If we can't run the task at the top, do the dumb thing and -* bounce it to the fallback dsq. +* If we can't run the task at the top for whatever reason, +* bounce it to the fallback dsq. Also check
tools/sched_ext/scx_central.bpf.c static bool dispatch_to_cpu(s32 cpu) · line 162
Quoted 10 lines
tools/sched_ext/scx_central.bpf.c +5
-1+* is_migration_disabled() explicitly as p->cpus_ptr may not +* reflect the migration-disabled state yet if +* migrate_disable_switch() hasn't run. */ -if (!bpf_cpumask_test_cpu (cpu, p->cpus_ptr)) { +if (!bpf_cpumask_test_cpu (cpu, p->cpus_ptr) || + (is_migration_disabled (p) &&scx_bpf_task_cpu (p) != cpu)) {__sync_fetch_and_add (&nr_mismatches,1 );scx_bpf_dsq_insert (p,FALLBACK_DSQ_ID ,SCX_SLICE_INF ,0 );bpf_task_release (p);
Looks good to me!
Minor email-related comment: I would prefer if patch revisions (v2, v3, ...) were sent in separate emails with their own subject line, otherwise it may look like I'm reviewing the v1, if you look solely at the subject line.
To be clear, the following tag applies to v3.
Hi Kuba,
Quoted 10 lines · Kuba Piecuch
Looks good to me!
Minor email-related comment: I would prefer if patch revisions (v2, v3, ...) were sent in separate emails with their own subject line, otherwise it may look like I'm reviewing the v1, if you look solely at the subject line.
To be clear, the following tag applies to v3.
Reviewed-by Kuba Piecuch <[email protected]>
Ahh, thanks for the review and reminder!
I was just being lazy because I thought keeping it in a single thread would mean I wouldn't have to add links to the changelog (but yes, I should add them anyway).
I'll send future revisions in separate threads. Thanks!
Signature · 2 lines
Thanks, Cheng-Yang
Hello,
Quoted 2 lines
Cheng-Yang Chou (1):
tools/sched_ext: Handle migration-disabled tasks inscx_central
Applied to sched_ext/for-7.2.
Thanks.
--
tejun
Andrea Righi
reviewerHi Cheng-Yang,
On Tue, Apr 14, 2026 at 11:45:59AM +0800, Cheng-Yang Chou wrote:
...
tools/sched_ext/scx_central.bpf.c static bool dispatch_to_cpu(s32 cpu) · line 175
Quoted 11 lines
tools/sched_ext/scx_central.bpf.c +7
-0@@ -155,6 +166,13 @@ static bool dispatch_to_cpu(s32 cpu)if (!bpf_cpumask_test_cpu (cpu, p->cpus_ptr)) {__sync_fetch_and_add (&nr_mismatches,1 );scx_bpf_dsq_insert (p,FALLBACK_DSQ_ID ,SCX_SLICE_INF ,0 ); + +/* +* Kick the task's home CPU to pick it up from the +* fallback DSQ promptly. +*/ +scx_bpf_kick_cpu (scx_bpf_task_cpu (p),SCX_KICK_PREEMPT ); +
You can also pass SCX_ENQ_PREEMPT to scx_bpf_dsq_insert() and avoid calling scx_bpf_kick_cpu().
More in general, do we really need to preempt the CPU here? It'll be preempted in the central_timerfn() handler later, right? If we preempt the CPU here we may have lots of preemption events when many affinitized tasks are running (i.e., stress-ng --race-sched 0).
Thanks,
-Andrea
Quoted 6 lines
tools/sched_ext/scx_central.bpf.c +0
-0bpf_task_release (p);/* * We might run out of dispatch buffer slots if we continue dispatching Signature · 1 line
2.48.1
Hi Andrea,
Quoted 19 lines · Andrea Righi
Hi Cheng-Yang,
On Tue, Apr 14, 2026 at 11:45:59AM +0800, Cheng-Yang Chou wrote:
...tools/sched_ext/scx_central.bpf.c static bool dispatch_to_cpu(s32 cpu) · line 175
Quoted 11 lines
tools/sched_ext/scx_central.bpf.c +7
-0@@ -155,6 +166,13 @@ static bool dispatch_to_cpu(s32 cpu)if (!bpf_cpumask_test_cpu (cpu, p->cpus_ptr)) {__sync_fetch_and_add (&nr_mismatches,1 );scx_bpf_dsq_insert (p,FALLBACK_DSQ_ID ,SCX_SLICE_INF ,0 ); + +/* +* Kick the task's home CPU to pick it up from the +* fallback DSQ promptly. +*/ +scx_bpf_kick_cpu (scx_bpf_task_cpu (p),SCX_KICK_PREEMPT ); +You can also pass
SCX_ENQ_PREEMPTtoscx_bpf_dsq_insert()and avoid callingscx_bpf_kick_cpu().
Quoted 4 lines
More in general, do we really need to preempt the CPU here? It'll be preempted in the
central_timerfn()handler later, right? If we preempt the CPU here we may have lots of preemption events when many affinitized tasks are running (i.e., stress-ng --race-sched 0).
Yeah, this is wrong :/ Thanks for your explanation! Will fix this in the upcoming patch, enjoy your time in the UK!
Signature · 2 lines
Thanks, Cheng-Yang