sched_ext: don't deliver duplicate ops.cgroup_set_idle() for same value
Subsystems (3)
PATCH v2 1/1 sched_ext: don't deliver duplicate ops.cgroup_set_idle() for same value
From: Tao Cui <[email protected]>
ops.cgroup_set_idle() is documented to be invoked when a cgroup transitions between idle and non-idle states, and scx_group_set_weight() already skips value-preserving writes. scx_group_set_idle() delivers every write unconditionally, so rewriting an already-correct cpu.idle value feeds the BPF scheduler a transition callback each time, which toggle- or accounting-based schedulers miscount. Mirror the weight guard and only deliver on an actual change.
Verified with a probe scheduler printing each callback: rewriting cpu.idle=1 twice on an already-idle cgroup delivered two callbacks before and none after.
---
v1 -> v2: Fix the Link: msgid (missing @linux.dev, Andrea).
kernel/sched/ext/ext.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
kernel/sched/ext/ext.c +2 -1
@@ -4933,7 +4933,8 @@ void scx_group_set_idle(struct task_group *tg, bool idle)percpu_down_read (&scx_cgroup_ops_rwsem); sch =scx_tg_knob_sched (tg); -if (scx_cgroup_enabled && sch &&SCX_HAS_OP (sch, cgroup_set_idle)) +if (scx_cgroup_enabled && sch &&SCX_HAS_OP (sch, cgroup_set_idle) && + tg->scx.sched_idle != idle)SCX_CALL_OP (sch, cgroup_set_idle,NULL ,tg_cgrp (tg), idle);/* Update the task group's idle state */
Signature · 1 line
2.43.0
[email protected]
AI reviewThank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [High] Lockless state check introduces a Time-of-Check to Time-of-Use (TOCTOU) race that can permanently…
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] Lockless state check introduces a Time-of-Check to Time-of-Use (TOCTOU) race that can permanently desynchronize the BPF scheduler from the actual cgroup state.
- [Low] Compilation failure due to use of non-existent field sched_idle in struct scx_task_group.
--
commit 47d6f0af51f3086b39745cdfc5f053f908dbd0c7
Author: Tao Cui <[email protected]>
sched_ext: don't deliver duplicate ops.cgroup_set_idle() for same value
This commit avoids sending duplicate cgroup_set_idle() callbacks to the BPF scheduler by checking if the cgroup's idle state has actually changed.
kernel/sched/ext/ext.c void scx_group_set_idle(struct task_group *tg, bool idle) · line 4934
Quoted 11 lines
kernel/sched/ext/ext.c +2
-1@@ -4930,7 +4930,8 @@ void scx_group_set_idle(struct task_group *tg, bool idle)percpu_down_read (&scx_cgroup_ops_rwsem); sch =scx_tg_knob_sched (tg); -if (scx_cgroup_enabled && sch &&SCX_HAS_OP (sch, cgroup_set_idle)) +if (scx_cgroup_enabled && sch &&SCX_HAS_OP (sch, cgroup_set_idle) && + tg->scx.sched_idle != idle)
[Severity: Low]
Does sched_idle exist in struct scx_task_group? It looks like this might cause a compile failure when building with CONFIG_SCHED_CLASS_EXT and CONFIG_EXT_GROUP_SCHED enabled. Should this be checking tg->scx.idle instead?
[Severity: High]
Is it possible for concurrent writes to cpu.idle to permanently desynchronize the BPF scheduler from the actual cgroup state here?
Since this state check is performed locklessly before the callback is sent, if Thread A writes 1 and Thread B concurrently writes 0, could Thread B evaluate the new check while the state is still 0, and falsely skip sending its callback?
If Thread A then sends its callback for 1, and Thread B subsequently updates the state to 0, the BPF scheduler might think the state is 1 while the internal tracking state is 0.
If a user tries to manually fix this by echoing 0 again, wouldn't the lockless check see the internal state is already 0 and skip the corrective callback, leaving the BPF scheduler trapped in the wrong state?
Quoted 3 lines
kernel/sched/ext/ext.c +0
-0SCX_CALL_OP (sch, cgroup_set_idle,NULL ,tg_cgrp (tg), idle);/* Update the task group's idle state */
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Quoted 7 lines
ops.cgroup_set_idle()is documented to be invoked when a cgroup transitions between idle and non-idle states, andscx_group_set_weight()already skips value-preserving writes.scx_group_set_idle()delivers every write unconditionally, so rewriting an already-correct cpu.idle value feeds the BPF scheduler a transition callback each time, which toggle- or accounting-based schedulers miscount. Mirror the weight guard and only deliver on an actual change.
Applied to sched_ext/for-7.3-fixes with the subject capitalized and the new comparison changed to tg->scx.idle, which is the field's name there:
- tg->scx.sched_idle != idle)
+ tg->scx.idle != idle)
The rename to tg->scx.sched_idle is on for-7.4 and the for-next merge switches the comparison back.
Thanks.
--
tejun