Skip to contentSkip to conversation

sched_ext: pass the initial cpu.idle state in scx_cgroup_init_args

PATCH In review Tao Cui 3 participants 6 messages Aug 25 – Aug 31

Subsystems (3)
lore
Share & tools
lore

Tao Cui

author
#0 lore raw

v3 cover sched_ext: pass the initial cpu.idle state in scx_cgroup_init_args

From: Tao Cui <[email protected]>

This carries the initial cpu.idle state of a cgroup through ops.cgroup_init(), which was the only cpu controller knob whose initial value never reached the BPF scheduler. Patch 1 is the fix; patch 2 renames tg->scx.idle to tg->scx.sched_idle, suggested in review, as a separate cleanup.

Changes since v2:

- rename the new field to sched_idle, as a bare "idle" reads as CPU idle state in sched_ext (Tejun)
- split the tg->scx.idle rename into 2/2 so 1/2 stays minimal for stable backport
- add the Fixes: tag (Andrea)
- regenerate on top of current linux-next, which also resolves the CI conflict reported against v2

v2: https://lore.kernel.org/r/[email protected]
v1: https://lore.kernel.org/r/[email protected]

Tao Cui (2):
sched_ext: pass the initial cpu.idle state in scx_cgroup_init_args
sched_ext: rename tg->scx.idle to tg->scx.sched_idle

 include/linux/sched/ext.h   | 2 +-
 kernel/sched/ext/ext.c      | 8 +++++---
 kernel/sched/ext/internal.h | 3 +++
 kernel/sched/ext/sub.c      | 2 ++
 4 files changed, 11 insertions(+), 4 deletions(-)
Signature · 1 line
2.43.0
author
#1 lore raw

PATCH v3 1/2 sched_ext: pass the initial cpu.idle state in scx_cgroup_init_args

From: Tao Cui <[email protected]>

scx_cgroup_init_args carries the initial weight and bandwidth control parameters of a cgroup to ops.cgroup_init(), but not its cpu.idle state. A cgroup that was already configured idle before the scheduler was loaded (or before it was onlined under it) is presented as non-idle, and the BPF scheduler only learns about it if cpu.idle is written again later.

Add the sched_idle state to scx_cgroup_init_args and fill it in all four places that build the args: scx_tg_online() for cgroups onlined under the scheduler, scx_cgroup_init() for cgroups that already exist when the scheduler is loaded, and the sub-scheduler handover paths scx_cgroup_claim_subtree() and scx_cgroup_return_subtree().

Verified in a VM with a probe scheduler printing the init args: a cgroup configured cpu.idle=1 before loading shows sched_idle=1 in ops.cgroup_init(), the default shows 0, and later cpu.idle writes still come through ops.cgroup_set_idle(). The sub-scheduler paths are compile-tested only.

Fixes 347ed2d566da ("sched/ext: Implement cgroup_set_idle() callback")

---
v2 -> v3: Rename the new field to sched_idle, as a bare "idle" reads as CPU idle state in sched_ext, per Tejun. Add the Fixes: tag suggested by Andrea. Regenerated on top of current linux-next, which also resolves the CI conflict reported for v2. The tg->scx.idle rename now lives in 2/2.

v2: https://lore.kernel.org/r/[email protected]

 kernel/sched/ext/ext.c      | 4 +++-
 kernel/sched/ext/internal.h | 3 +++
 kernel/sched/ext/sub.c      | 2 ++
 3 files changed, 8 insertions(+), 1 deletion(-)
kernel/sched/ext/ext.c +3 -1
index c539d15cda63..82cf8f57e15c 100644
@@ -4764,7 +4764,8 @@ int scx_tg_online(struct task_group *tg)
 				{ .weight = tg->scx.weight,
 				  .bw_period_us = tg->scx.bw_period_us,
 				  .bw_quota_us = tg->scx.bw_quota_us,
-				  .bw_burst_us = tg->scx.bw_burst_us };
+				  .bw_burst_us = tg->scx.bw_burst_us,
+				  .sched_idle = tg->scx.idle };
 
 			ret = SCX_CALL_OP_RET(sch, cgroup_init,
 					      NULL, tg->css.cgroup, &args);
@@ -5185,6 +5186,7 @@ static int scx_cgroup_init(struct scx_sched *sch)
 				.bw_period_us = tg->scx.bw_period_us,
 				.bw_quota_us = tg->scx.bw_quota_us,
 				.bw_burst_us = tg->scx.bw_burst_us,
+				.sched_idle = tg->scx.idle,
 			};
 
 			ret = SCX_CALL_OP_RET(sch, cgroup_init, NULL, css->cgroup, &args);
kernel/sched/ext/internal.h +3 -0
index 53e136a47924..c6987a3a073b 100644
@@ -259,6 +259,9 @@ struct scx_cgroup_init_args {
 	u64			bw_period_us;
 	u64			bw_quota_us;
 	u64			bw_burst_us;
+
+	/* whether the cgroup is configured SCHED_IDLE via cpu.idle */
+	bool			sched_idle;
 };
 
 enum scx_cpu_preempt_reason {
kernel/sched/ext/sub.c +2 -0
index 0554448835bd..385302d19914 100644
@@ -1361,6 +1361,7 @@ static s32 scx_cgroup_claim_subtree(struct scx_sched *sch)
 			.bw_period_us = tg->scx.bw_period_us,
 			.bw_quota_us = tg->scx.bw_quota_us,
 			.bw_burst_us = tg->scx.bw_burst_us,
+			.sched_idle = tg->scx.idle,
 		};
 
 		if (tg->scx.sched != parent ||
@@ -1464,6 +1465,7 @@ static void scx_cgroup_return_subtree(struct scx_sched *sch)
 			.bw_period_us = tg->scx.bw_period_us,
 			.bw_quota_us = tg->scx.bw_quota_us,
 			.bw_burst_us = tg->scx.bw_burst_us,
+			.sched_idle = tg->scx.idle,
 		};
 
 		/* the first pass must have transferred everything */
Signature · 1 line
2.43.0
author
#2 lore raw

PATCH v3 2/2 sched_ext: rename tg->scx.idle to tg->scx.sched_idle

In reply to Tao Cui

From: Tao Cui <[email protected]>

A bare "idle" reads as CPU idle state in sched_ext (ops.update_idle(), idle cpumasks, scx_bpf_pick_idle_cpu()). cpu.idle is the cgroup analog of the SCHED_IDLE policy, so name the task_group field sched_idle to match scx_cgroup_init_args.sched_idle.

Pure rename, no behavior change.

---
 include/linux/sched/ext.h | 2 +-
 kernel/sched/ext/ext.c    | 8 ++++----
 kernel/sched/ext/sub.c    | 4 ++--
 3 files changed, 7 insertions(+), 7 deletions(-)
include/linux/sched/ext.h +1 -1
index 582d7cd4a983..7b9cdb65deef 100644
@@ -323,7 +323,7 @@ struct scx_task_group {
 	u64			bw_period_us;
 	u64			bw_quota_us;
 	u64			bw_burst_us;
-	bool			idle;
+	bool			sched_idle;
 #endif
 };
 
kernel/sched/ext/ext.c +4 -4
index 82cf8f57e15c..4a9c38297093 100644
@@ -4682,7 +4682,7 @@ void scx_tg_init(struct task_group *tg)
 	tg->scx.weight = CGROUP_WEIGHT_DFL;
 	tg->scx.bw_period_us = default_bw_period_us();
 	tg->scx.bw_quota_us = RUNTIME_INF;
-	tg->scx.idle = false;
+	tg->scx.sched_idle = false;
 }
 
 /**
@@ -4765,7 +4765,7 @@ int scx_tg_online(struct task_group *tg)
 				  .bw_period_us = tg->scx.bw_period_us,
 				  .bw_quota_us = tg->scx.bw_quota_us,
 				  .bw_burst_us = tg->scx.bw_burst_us,
-				  .sched_idle = tg->scx.idle };
+				  .sched_idle = tg->scx.sched_idle };
 
 			ret = SCX_CALL_OP_RET(sch, cgroup_init,
 					      NULL, tg->css.cgroup, &args);
@@ -4935,7 +4935,7 @@ void scx_group_set_idle(struct task_group *tg, bool idle)
 		SCX_CALL_OP(sch, cgroup_set_idle, NULL, tg_cgrp(tg), idle);
 
 	/* Update the task group's idle state */
-	tg->scx.idle = idle;
+	tg->scx.sched_idle = idle;
 
 	percpu_up_read(&scx_cgroup_ops_rwsem);
 }
@@ -5186,7 +5186,7 @@ static int scx_cgroup_init(struct scx_sched *sch)
 				.bw_period_us = tg->scx.bw_period_us,
 				.bw_quota_us = tg->scx.bw_quota_us,
 				.bw_burst_us = tg->scx.bw_burst_us,
-				.sched_idle = tg->scx.idle,
+				.sched_idle = tg->scx.sched_idle,
 			};
 
 			ret = SCX_CALL_OP_RET(sch, cgroup_init, NULL, css->cgroup, &args);
kernel/sched/ext/sub.c +2 -2
index 385302d19914..a17d84db93bd 100644
@@ -1361,7 +1361,7 @@ static s32 scx_cgroup_claim_subtree(struct scx_sched *sch)
 			.bw_period_us = tg->scx.bw_period_us,
 			.bw_quota_us = tg->scx.bw_quota_us,
 			.bw_burst_us = tg->scx.bw_burst_us,
-			.sched_idle = tg->scx.idle,
+			.sched_idle = tg->scx.sched_idle,
 		};
 
 		if (tg->scx.sched != parent ||
@@ -1465,7 +1465,7 @@ static void scx_cgroup_return_subtree(struct scx_sched *sch)
 			.bw_period_us = tg->scx.bw_period_us,
 			.bw_quota_us = tg->scx.bw_quota_us,
 			.bw_burst_us = tg->scx.bw_burst_us,
-			.sched_idle = tg->scx.idle,
+			.sched_idle = tg->scx.sched_idle,
 		};
 
 		/* the first pass must have transferred everything */
Signature · 1 line
2.43.0
author
#3 lore raw

Re: [PATCH v3 2/2] sched_ext: rename tg->scx.idle to tg->scx.sched_idle

Hi,

Quoted 9 lines · Tao Cui

From: Tao Cui <[email protected]>

A bare "idle" reads as CPU idle state in sched_ext (ops.update_idle(), idle cpumasks, scx_bpf_pick_idle_cpu()). cpu.idle is the cgroup analog of the SCHED_IDLE policy, so name the task_group field sched_idle to match scx_cgroup_init_args.sched_idle.

Pure rename, no behavior change.

While testing this series, I noticed that scx_group_set_idle() delivers ops.cgroup_set_idle() on every successful cpu.idle write, even rewrites of the current value, although the kerneldoc says the callback is invoked on transitions. scx_group_set_weight() and scx_group_set_bandwidth() both skip unchanged values, so this seems like an oversight.

The fix is a one-line guard, but it reads the field renamed here, so I'll post it on top of this series once it lands.

Thanks,
Tao

Quoted 82 lines
---
 include/linux/sched/ext.h | 2 +-
 kernel/sched/ext/ext.c    | 8 ++++----
 kernel/sched/ext/sub.c    | 4 ++--
 3 files changed, 7 insertions(+), 7 deletions(-)
include/linux/sched/ext.h +1 -1
index 582d7cd4a983..7b9cdb65deef 100644
@@ -323,7 +323,7 @@ struct scx_task_group {
 	u64			bw_period_us;
 	u64			bw_quota_us;
 	u64			bw_burst_us;
-	bool			idle;
+	bool			sched_idle;
 #endif
 };
 
kernel/sched/ext/ext.c +4 -4
index 82cf8f57e15c..4a9c38297093 100644
@@ -4682,7 +4682,7 @@ void scx_tg_init(struct task_group *tg)
 	tg->scx.weight = CGROUP_WEIGHT_DFL;
 	tg->scx.bw_period_us = default_bw_period_us();
 	tg->scx.bw_quota_us = RUNTIME_INF;
-	tg->scx.idle = false;
+	tg->scx.sched_idle = false;
 }
 
 /**
@@ -4765,7 +4765,7 @@ int scx_tg_online(struct task_group *tg)
 				  .bw_period_us = tg->scx.bw_period_us,
 				  .bw_quota_us = tg->scx.bw_quota_us,
 				  .bw_burst_us = tg->scx.bw_burst_us,
-				  .sched_idle = tg->scx.idle };
+				  .sched_idle = tg->scx.sched_idle };
 
 			ret = SCX_CALL_OP_RET(sch, cgroup_init,
 					      NULL, tg->css.cgroup, &args);
@@ -4935,7 +4935,7 @@ void scx_group_set_idle(struct task_group *tg, bool idle)
 		SCX_CALL_OP(sch, cgroup_set_idle, NULL, tg_cgrp(tg), idle);
 
 	/* Update the task group's idle state */
-	tg->scx.idle = idle;
+	tg->scx.sched_idle = idle;
 
 	percpu_up_read(&scx_cgroup_ops_rwsem);
 }
@@ -5186,7 +5186,7 @@ static int scx_cgroup_init(struct scx_sched *sch)
 				.bw_period_us = tg->scx.bw_period_us,
 				.bw_quota_us = tg->scx.bw_quota_us,
 				.bw_burst_us = tg->scx.bw_burst_us,
-				.sched_idle = tg->scx.idle,
+				.sched_idle = tg->scx.sched_idle,
 			};
 
 			ret = SCX_CALL_OP_RET(sch, cgroup_init, NULL, css->cgroup, &args);
kernel/sched/ext/sub.c +2 -2
index 385302d19914..a17d84db93bd 100644
@@ -1361,7 +1361,7 @@ static s32 scx_cgroup_claim_subtree(struct scx_sched *sch)
 			.bw_period_us = tg->scx.bw_period_us,
 			.bw_quota_us = tg->scx.bw_quota_us,
 			.bw_burst_us = tg->scx.bw_burst_us,
-			.sched_idle = tg->scx.idle,
+			.sched_idle = tg->scx.sched_idle,
 		};
 
 		if (tg->scx.sched != parent ||
@@ -1465,7 +1465,7 @@ static void scx_cgroup_return_subtree(struct scx_sched *sch)
 			.bw_period_us = tg->scx.bw_period_us,
 			.bw_quota_us = tg->scx.bw_quota_us,
 			.bw_burst_us = tg->scx.bw_burst_us,
-			.sched_idle = tg->scx.idle,
+			.sched_idle = tg->scx.sched_idle,
 		};
 
 		/* the first pass must have transferred everything */
reviewer Reviewed-by on the v3 series
#4 lore raw
In reply to Tao Cui
Quoted 7 lines · Tao Cui

From: Tao Cui <[email protected]>

This carries the initial cpu.idle state of a cgroup through ops.cgroup_init(), which was the only cpu controller knob whose initial value never reached the BPF scheduler. Patch 1 is the fix; patch 2 renames tg->scx.idle to tg->scx.sched_idle, suggested in review, as a separate cleanup.

This looks good to me.

Thanks,
-Andrea

Quoted 27 lines

Changes since v2:

- rename the new field to sched_idle, as a bare "idle" reads as CPU idle state in sched_ext (Tejun)
- split the tg->scx.idle rename into 2/2 so 1/2 stays minimal for stable backport
- add the Fixes: tag (Andrea)
- regenerate on top of current linux-next, which also resolves the CI conflict reported against v2

v2: https://lore.kernel.org/r/[email protected]
v1: https://lore.kernel.org/r/[email protected]

Tao Cui (2):
sched_ext: pass the initial cpu.idle state in scx_cgroup_init_args
sched_ext: rename tg->scx.idle to tg->scx.sched_idle

 include/linux/sched/ext.h   | 2 +-
 kernel/sched/ext/ext.c      | 8 +++++---
 kernel/sched/ext/internal.h | 3 +++
 kernel/sched/ext/sub.c      | 2 ++
 4 files changed, 11 insertions(+), 4 deletions(-)
Signature · 1 line
2.43.0
reviewer
#5 lore raw
In reply to Tao Cui
Quoted 3 lines

Tao Cui (2):
sched_ext: pass the initial cpu.idle state in scx_cgroup_init_args
sched_ext: rename tg->scx.idle to tg->scx.sched_idle

Applied 1 to sched_ext/for-7.3-fixes and 2 to sched_ext/for-7.4, with Andrea's Reviewed-by from the cover added to 2 and the subjects capitalized.

Thanks.

--
tejun