sched_ext: pass the initial cpu.idle state in scx_cgroup_init_args
Subsystems (3)
PATCH v2 1/1 sched_ext: pass the initial cpu.idle state in scx_cgroup_init_args
From: Tao Cui <[email protected]>
scx_cgroup_init_args carries the initial weight and bandwidth control parameters of a cgroup to ops.cgroup_init(), but not its cpu.idle state. A cgroup that was already configured idle before the scheduler was loaded (or before it was onlined under it) is presented as non-idle, and the BPF scheduler only learns about it if cpu.idle is written again later.
Add the idle state to scx_cgroup_init_args and fill it in all four places that build the args: scx_tg_online() for cgroups onlined under the scheduler, scx_cgroup_init() for cgroups that already exist when the scheduler is loaded, and the sub-scheduler handover paths scx_cgroup_claim_subtree() and scx_cgroup_return_subtree().
Verified in a VM with a probe scheduler printing the init args: a cgroup configured cpu.idle=1 before loading shows idle=1 in ops.cgroup_init(), the default shows 0, and later cpu.idle writes still come through ops.cgroup_set_idle(). The sub-scheduler paths are compile-tested only.
---
v1 -> v2: Also fill .idle in the sub-scheduler handover paths in sub.c, scx_cgroup_claim_subtree() and scx_cgroup_return_subtree(), missed in v1 and pointed out by Andrea Righi and the sashiko AI review bot.
v1: https://lore.kernel.org/r/[email protected]
kernel/sched/ext/ext.c | 4 +++- kernel/sched/ext/internal.h | 3 +++ kernel/sched/ext/sub.c | 2 ++ 3 files changed, 8 insertions(+), 1 deletion(-)
kernel/sched/ext/ext.c +3 -1
@@ -4764,7 +4764,8 @@ int scx_tg_online(struct task_group *tg) { .weight = tg->scx.weight, .bw_period_us = tg->scx.bw_period_us, .bw_quota_us = tg->scx.bw_quota_us, - .bw_burst_us = tg->scx.bw_burst_us }; + .bw_burst_us = tg->scx.bw_burst_us, + .idle = tg->scx.idle }; ret =SCX_CALL_OP_RET (sch, cgroup_init,NULL , tg->css.cgroup, &args);
@@ -5185,6 +5186,7 @@ static int scx_cgroup_init(struct scx_sched *sch) .bw_period_us = tg->scx.bw_period_us, .bw_quota_us = tg->scx.bw_quota_us, .bw_burst_us = tg->scx.bw_burst_us, + .idle = tg->scx.idle, }; ret =SCX_CALL_OP_RET (sch, cgroup_init,NULL , css->cgroup, &args);
kernel/sched/ext/internal.h +3 -0
@@ -259,6 +259,9 @@ struct scx_cgroup_init_args {u64 bw_period_us;u64 bw_quota_us;u64 bw_burst_us; + +/* whether the cgroup is configured idle via cpu.idle */ +bool idle; };enum scx_cpu_preempt_reason {
kernel/sched/ext/sub.c +2 -0
@@ -1361,6 +1361,7 @@ static s32 scx_cgroup_claim_subtree(struct scx_sched *sch) .bw_period_us = tg->scx.bw_period_us, .bw_quota_us = tg->scx.bw_quota_us, .bw_burst_us = tg->scx.bw_burst_us, + .idle = tg->scx.idle, }; if (tg->scx.sched != parent ||
@@ -1464,6 +1465,7 @@ static void scx_cgroup_return_subtree(struct scx_sched *sch) .bw_period_us = tg->scx.bw_period_us, .bw_quota_us = tg->scx.bw_quota_us, .bw_burst_us = tg->scx.bw_burst_us, + .idle = tg->scx.idle, };/* the first pass must have transferred everything */
Signature · 1 line
2.43.0
Hi Tao,
Quoted 15 lines · Tao Cui
From: Tao Cui <[email protected]>
scx_cgroup_init_argscarries the initial weight and bandwidth control parameters of a cgroup toops.cgroup_init(), but not its cpu.idle state. A cgroup that was already configured idle before the scheduler was loaded (or before it was onlined under it) is presented as non-idle, and the BPF scheduler only learns about it if cpu.idle is written again later.Add the idle state to
scx_cgroup_init_argsand fill it in all four places that build the args:scx_tg_online()for cgroups onlined under the scheduler,scx_cgroup_init()for cgroups that already exist when the scheduler is loaded, and the sub-scheduler handover pathsscx_cgroup_claim_subtree()andscx_cgroup_return_subtree().
Quoted 7 lines
Verified in a VM with a probe scheduler printing the init args: a cgroup configured cpu.idle=1 before loading shows idle=1 in
ops.cgroup_init(), the default shows 0, and later cpu.idle writes still come throughops.cgroup_set_idle(). The sub-scheduler paths are compile-tested only.Signed-off-by Tao Cui <[email protected]>
We should probably add:
With that:
Thanks,
-Andrea
Quoted 72 lines
---
v1 -> v2: Also fill .idle in the sub-scheduler handover paths insub.c,scx_cgroup_claim_subtree()andscx_cgroup_return_subtree(), missed in v1 and pointed out by Andrea Righi and the sashiko AI review bot.v1: https://lore.kernel.org/r/[email protected]
kernel/sched/ext/ext.c | 4 +++- kernel/sched/ext/internal.h | 3 +++ kernel/sched/ext/sub.c | 2 ++ 3 files changed, 8 insertions(+), 1 deletion(-)kernel/sched/ext/ext.c +3
-1@@ -4764,7 +4764,8 @@ int scx_tg_online(struct task_group *tg) { .weight = tg->scx.weight, .bw_period_us = tg->scx.bw_period_us, .bw_quota_us = tg->scx.bw_quota_us, - .bw_burst_us = tg->scx.bw_burst_us }; + .bw_burst_us = tg->scx.bw_burst_us, + .idle = tg->scx.idle }; ret =SCX_CALL_OP_RET (sch, cgroup_init,NULL , tg->css.cgroup, &args);@@ -5185,6 +5186,7 @@ static int scx_cgroup_init(struct scx_sched *sch) .bw_period_us = tg->scx.bw_period_us, .bw_quota_us = tg->scx.bw_quota_us, .bw_burst_us = tg->scx.bw_burst_us, + .idle = tg->scx.idle, }; ret =SCX_CALL_OP_RET (sch, cgroup_init,NULL , css->cgroup, &args);kernel/sched/ext/internal.h +3
-0@@ -259,6 +259,9 @@ struct scx_cgroup_init_args {u64 bw_period_us;u64 bw_quota_us;u64 bw_burst_us; + +/* whether the cgroup is configured idle via cpu.idle */ +bool idle; };enum scx_cpu_preempt_reason {kernel/sched/ext/sub.c +2
-0@@ -1361,6 +1361,7 @@ static s32 scx_cgroup_claim_subtree(struct scx_sched *sch) .bw_period_us = tg->scx.bw_period_us, .bw_quota_us = tg->scx.bw_quota_us, .bw_burst_us = tg->scx.bw_burst_us, + .idle = tg->scx.idle, }; if (tg->scx.sched != parent ||@@ -1464,6 +1465,7 @@ static void scx_cgroup_return_subtree(struct scx_sched *sch) .bw_period_us = tg->scx.bw_period_us, .bw_quota_us = tg->scx.bw_quota_us, .bw_burst_us = tg->scx.bw_burst_us, + .idle = tg->scx.idle, };/* the first pass must have transferred everything */ Signature · 1 line
2.43.0
Hello, Andrea,
Quoted 26 lines · Andrea Righi
Hi Tao,
Quoted 15 lines · Tao Cui
From: Tao Cui <[email protected]>
scx_cgroup_init_argscarries the initial weight and bandwidth control parameters of a cgroup toops.cgroup_init(), but not its cpu.idle state. A cgroup that was already configured idle before the scheduler was loaded (or before it was onlined under it) is presented as non-idle, and the BPF scheduler only learns about it if cpu.idle is written again later.Add the idle state to
scx_cgroup_init_argsand fill it in all four places that build the args:scx_tg_online()for cgroups onlined under the scheduler,scx_cgroup_init()for cgroups that already exist when the scheduler is loaded, and the sub-scheduler handover pathsscx_cgroup_claim_subtree()andscx_cgroup_return_subtree().Quoted 7 lines
Verified in a VM with a probe scheduler printing the init args: a cgroup configured cpu.idle=1 before loading shows idle=1 in
ops.cgroup_init(), the default shows 0, and later cpu.idle writes still come throughops.cgroup_set_idle(). The sub-scheduler paths are compile-tested only.Signed-off-by Tao Cui <[email protected]>
Quoted 8 lines
We should probably add:
Fixes 347ed2d566da ("sched/ext: Implement cgroup_set_idle() callback"With that:
Reviewed-by Andrea Righi <[email protected]>
Thank you for the review, and for catching the sub.c handover paths earlier. v3 adds the Fixes: tag you suggested and carries your Reviewed-by. The field is also renamed to sched_idle per Tejun's review; the rename is mechanical, so I kept the tag, but I'm happy to wait for a re-review if you prefer.
Thanks,
Tao
Quoted 75 lines
Thanks,
-AndreaQuoted 72 lines
---
v1 -> v2: Also fill .idle in the sub-scheduler handover paths insub.c,scx_cgroup_claim_subtree()andscx_cgroup_return_subtree(), missed in v1 and pointed out by Andrea Righi and the sashiko AI review bot.v1: https://lore.kernel.org/r/[email protected]
kernel/sched/ext/ext.c | 4 +++- kernel/sched/ext/internal.h | 3 +++ kernel/sched/ext/sub.c | 2 ++ 3 files changed, 8 insertions(+), 1 deletion(-)kernel/sched/ext/ext.c +3
-1@@ -4764,7 +4764,8 @@ int scx_tg_online(struct task_group *tg) { .weight = tg->scx.weight, .bw_period_us = tg->scx.bw_period_us, .bw_quota_us = tg->scx.bw_quota_us, - .bw_burst_us = tg->scx.bw_burst_us }; + .bw_burst_us = tg->scx.bw_burst_us, + .idle = tg->scx.idle }; ret =SCX_CALL_OP_RET (sch, cgroup_init,NULL , tg->css.cgroup, &args);@@ -5185,6 +5186,7 @@ static int scx_cgroup_init(struct scx_sched *sch) .bw_period_us = tg->scx.bw_period_us, .bw_quota_us = tg->scx.bw_quota_us, .bw_burst_us = tg->scx.bw_burst_us, + .idle = tg->scx.idle, }; ret =SCX_CALL_OP_RET (sch, cgroup_init,NULL , css->cgroup, &args);kernel/sched/ext/internal.h +3
-0@@ -259,6 +259,9 @@ struct scx_cgroup_init_args {u64 bw_period_us;u64 bw_quota_us;u64 bw_burst_us; + +/* whether the cgroup is configured idle via cpu.idle */ +bool idle; };enum scx_cpu_preempt_reason {kernel/sched/ext/sub.c +2
-0@@ -1361,6 +1361,7 @@ static s32 scx_cgroup_claim_subtree(struct scx_sched *sch) .bw_period_us = tg->scx.bw_period_us, .bw_quota_us = tg->scx.bw_quota_us, .bw_burst_us = tg->scx.bw_burst_us, + .idle = tg->scx.idle, }; if (tg->scx.sched != parent ||@@ -1464,6 +1465,7 @@ static void scx_cgroup_return_subtree(struct scx_sched *sch) .bw_period_us = tg->scx.bw_period_us, .bw_quota_us = tg->scx.bw_quota_us, .bw_burst_us = tg->scx.bw_burst_us, + .idle = tg->scx.idle, };/* the first pass must have transferred everything */ Signature · 1 line
2.43.0
Tejun Heo
reviewerHello, Tao.
kernel/sched/ext/internal.h struct scx_cgroup_init_args { · line 264
Quoted 3 lines · Tao Cui
kernel/sched/ext/internal.h +3
-0+ +/* whether the cgroup is configured idle via cpu.idle */ +bool idle;
In sched_ext, a bare "idle" reads as CPU idle state (ops.update_idle(), idle cpumasks, scx_bpf_pick_idle_cpu()). cpu.idle is the cgroup analog of the SCHED_IDLE policy, so please name the field sched_idle instead. Renaming tg->scx.idle to match would be nice too.
Thanks.
Signature · 1 line
tejun
Hello, Tejun,
Quoted 12 lines · Tejun Heo
Hello, Tao.
kernel/sched/ext/internal.h struct scx_cgroup_init_args { · line 264
Quoted 3 lines · Tao Cui
kernel/sched/ext/internal.h +3
-0+ +/* whether the cgroup is configured idle via cpu.idle */ +bool idle;In
sched_ext, a bare "idle" reads as CPU idle state (ops.update_idle(), idle cpumasks,scx_bpf_pick_idle_cpu()). cpu.idle is the cgroup analog of theSCHED_IDLEpolicy, so please name the fieldsched_idleinstead. Renamingtg->scx.idleto match would be nice too.
Thanks for the review. Both are renamed in v3: the new scx_cgroup_init_args field, and tg->scx.idle -> tg->scx.sched_idle as a separate second patch in the series, so 1/2 stays minimal for stable backport.
Thanks,
Tao
Quoted 2 lines
Thanks.