sched_ext: Make scx_rq_online() also test cpu_active() in addition to SCX_RQ_ONLINE
PATCH 1/1 sched_ext: Make scx_rq_online() also test cpu_active() in addition to SCX_RQ_ONLINE
scx_rq_online() currently only tests SCX_RQ_ONLINE. This isn't fully correct
- e.g. consume_dispatch_q() uses task_run_on_remote_rq() which tests scx_rq_online() to see whether the current rq can run the task, and, if so, calls consume_remote_task() to migrate the task to @rq. While the test itself was done while locking @rq, @rq can be temporarily unlocked by consume_remote_task() and nothing prevents SCX_RQ_ONLINE from going offline before the migration takes place.
To address the issue, add cpu_active() test to scx_rq_online(). There is a synchronize_rcu() between cpu_active() being cleared and the rq going offline, so if an on-going scheduling operation sees cpu_active(), the associated rq is guaranteed to not go offline until the scheduling operation is complete.
--- kernel/sched/ext.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
kernel/sched/ext.c +8 -1
@@ -1818,7 +1818,14 @@ dispatch:static bool scx_rq_online (struct rq *rq) { -return likely (rq->scx.flags &SCX_RQ_ONLINE ); +/* +* Test both cpu_active() and %SCX_RQ_ONLINE. %SCX_RQ_ONLINE indicates +* the online state as seen from the BPF scheduler. cpu_active() test +* guarantees that, if this function returns %true, %SCX_RQ_ONLINE will +* stay set until the current scheduling operation is complete even if +* we aren't locking @rq. +*/ +return likely ((rq->scx.flags &SCX_RQ_ONLINE ) &&cpu_active (cpu_of (rq))); } static void do_enqueue_task(struct rq *rq, struct task_struct *p, u64 enq_flags,
Quoted 8 lines · Tejun Heo
scx_rq_online()currently only testsSCX_RQ_ONLINE. This isn't fully correct
- e.g.consume_dispatch_q()usestask_run_on_remote_rq()which testsscx_rq_online()to see whether the current rq can run the task, and, if so, callsconsume_remote_task()to migrate the task to @rq. While the test itself was done while locking @rq, @rq can be temporarily unlocked byconsume_remote_task()and nothing preventsSCX_RQ_ONLINEfrom going offline before the migration takes place.
Quoted 8 lines
To address the issue, add
cpu_active()test toscx_rq_online(). There is asynchronize_rcu()betweencpu_active()being cleared and the rq going offline, so if an on-going scheduling operation seescpu_active(), the associated rq is guaranteed to not go offline until the scheduling operation is complete.Signed-off-by Tejun Heo <[email protected]>Fixes 60c27fb59f6c ("sched_ext: Implement sched_ext_ops.cpu_online/offline()")
Tejun Heo
AppliedQuoted 8 lines · Tejun Heo
scx_rq_online()currently only testsSCX_RQ_ONLINE. This isn't fully correct
- e.g.consume_dispatch_q()usestask_run_on_remote_rq()which testsscx_rq_online()to see whether the current rq can run the task, and, if so, callsconsume_remote_task()to migrate the task to @rq. While the test itself was done while locking @rq, @rq can be temporarily unlocked byconsume_remote_task()and nothing preventsSCX_RQ_ONLINEfrom going offline before the migration takes place.
Quoted 8 lines
To address the issue, add
cpu_active()test toscx_rq_online(). There is asynchronize_rcu()betweencpu_active()being cleared and the rq going offline, so if an on-going scheduling operation seescpu_active(), the associated rq is guaranteed to not go offline until the scheduling operation is complete.Signed-off-by Tejun Heo <[email protected]>Fixes 60c27fb59f6c ("sched_ext: Implement sched_ext_ops.cpu_online/offline()")
Applied to sched_ext/for-6.12.
Thanks.
Signature · 1 line
tejun