Skip to contentSkip to conversation

sched_ext: Don't warn on core-sched forced idle in put_prev_task_scx()

PATCH Superseded Tejun Heo 3 participants 5 messages Jun 24 – Jun 29

Subsystems (2)

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Tejun Heo

author reviewer
#0 lore raw

PATCH 1/1 sched_ext: Don't warn on core-sched forced idle in put_prev_task_scx()

put_prev_task_scx() warns when a runnable task drops to a lower sched_class without SCX_OPS_ENQ_LAST, assuming balance_one() would otherwise keep it running.

Under core scheduling that assumption is wrong: a forced-idle SMT sibling reschedules through the core_pick fast path in pick_next_task(), which skips balance() for the CPU, so balance_one() never runs and a runnable task can drop to idle with ENQ_LAST unset. Skip the warning when core scheduling is enabled.

Fixes 7c65ae81ea86 ("sched_ext: Don't call put_prev_task_scx() before picking the next task")
---
 kernel/sched/ext/ext.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
kernel/sched/ext/ext.c +3 -1
index 9c9cb9d08bca..503c4d2105ee 100644
@@ -3092,7 +3092,9 @@ static void put_prev_task_scx(struct rq *rq, struct task_struct *p,
 		 * which should trigger an explicit follow-up scheduling event.
 		 */
 		if (next && sched_class_above(&ext_sched_class, next->sched_class)) {
-			WARN_ON_ONCE(!(sch->ops.flags & SCX_OPS_ENQ_LAST));
+			/* core-sched can force cpu idle while @p is runnable */
+			if (!sched_core_enabled(rq))
+				WARN_ON_ONCE(!(sch->ops.flags & SCX_OPS_ENQ_LAST));
 			do_enqueue_task(rq, p, SCX_ENQ_LAST, -1);
 		} else {
 			do_enqueue_task(rq, p, 0, -1);
#1 lore raw

Hi Tejun,

Quoted 7 lines · Tejun Heo

put_prev_task_scx() warns when a runnable task drops to a lower sched_class without SCX_OPS_ENQ_LAST, assuming balance_one() would otherwise keep it running.

Under core scheduling that assumption is wrong: a forced-idle SMT sibling reschedules through the core_pick fast path in pick_next_task(), which skips balance() for the CPU, so balance_one() never runs and a runnable

Nit: balance_one() doesn't happen in balance() anymore, it happens in pick. So IMO it should read "... skips pick_task_scx() for the CPU, ...".

Quoted 14 lines

task can drop to idle with ENQ_LAST unset. Skip the warning when core scheduling is enabled.

Fixes 7c65ae81ea86 ("sched_ext: Don't call put_prev_task_scx() before picking the next task")
---
 kernel/sched/ext/ext.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
kernel/sched/ext/ext.c +0 -0
index 9c9cb9d08bca..503c4d2105ee 100644

kernel/sched/ext/ext.c static void put_prev_task_scx(struct rq *rq, struct task_struct *p, · line 3096

Quoted 6 lines
kernel/sched/ext/ext.c +2 -1
@@ -3092,7 +3092,9 @@ static void put_prev_task_scx(struct rq *rq, struct task_struct *p,
 		 * which should trigger an explicit follow-up scheduling event.
 		 */
 		if (next && sched_class_above(&ext_sched_class, next->sched_class)) {
-			WARN_ON_ONCE(!(sch->ops.flags & SCX_OPS_ENQ_LAST));
+			/* core-sched can force cpu idle while @p is runnable */
+			if (!sched_core_enabled(rq))

Is there a more precise check that we could do to determine if this switch is due to core-sched forcing the CPU idle? I was thinking about rq->core->core_forceidle_count, but IIUC that's the core-wide number of CPUs forced idle, so it's not a reliable signal about any particular CPU.

kernel/sched/ext/ext.c static void put_prev_task_scx(struct rq *rq, struct task_struct *p, · line 3100

Quoted 4 lines
kernel/sched/ext/ext.c +1 -0
+				WARN_ON_ONCE(!(sch->ops.flags & SCX_OPS_ENQ_LAST));
 			do_enqueue_task(rq, p, SCX_ENQ_LAST, -1);
 		} else {
 			do_enqueue_task(rq, p, 0, -1);

This patch made me think a bit about core-sched interactions and I have a concern about IMMED tasks staying on local DSQ when the CPU is forced idle. I wasn't able to quickly convince myself that an IMMED task will be reenqueued in the case where a CPU running an SCX task has an IMMED task enqueued in its local DSQ by a remote CPU, and the CPU is forced idle while the IMMED task is on the local DSQ.
Looks like we might need a call to schedule_reenq_local() somewhere in here (in a separate patch, of course). WDYT?

Thanks,
Kuba

author reviewer
#2 lore raw

Hello, Kuba.

Quoted 2 lines · Kuba Piecuch

Nit: balance_one() doesn't happen in balance() anymore, it happens in pick. So IMO it should read "... skips pick_task_scx() for the CPU, ...".

Right, fixed in v2.

Quoted 4 lines

Is there a more precise check that we could do to determine if this switch is due to core-sched forcing the CPU idle? I was thinking about rq->core->core_forceidle_count, but IIUC that's the core-wide number of CPUs forced idle, so it's not a reliable signal about any particular CPU.

I couldn't find anything with better granularity either, so v2 keeps the sched_core_enabled() gate.

Quoted 6 lines

I wasn't able to quickly convince myself that an IMMED task will be reenqueued in the case where a CPU running an SCX task has an IMMED task enqueued in its local DSQ by a remote CPU, and the CPU is forced idle while the IMMED task is on the local DSQ.
Looks like we might need a call to schedule_reenq_local() somewhere in here (in a separate patch, of course). WDYT?

Makes sense, and not too surprising. For now maybe we just note it in the comment.

Thanks.

Signature · 1 line
tejun
author reviewer
#3 lore raw

PATCH v2 1/1 sched_ext: Don't warn on core-sched forced idle in put_prev_task_scx()

In reply to Tejun Heo

put_prev_task_scx() warns when a runnable task drops to a lower sched_class without SCX_OPS_ENQ_LAST, assuming balance_one() would otherwise keep it running.

Under core scheduling that assumption is wrong: a forced-idle SMT sibling reschedules through the core_pick fast path in pick_next_task(), which skips pick_task_scx() for the CPU, so balance_one() never runs and a runnable task can drop to idle with ENQ_LAST unset. Skip the warning when core scheduling is enabled.

v2: Reworded the description per Kuba Piecuch's review. No code change.

Fixes 7c65ae81ea86 ("sched_ext: Don't call put_prev_task_scx() before picking the next task")
---
 kernel/sched/ext/ext.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
kernel/sched/ext/ext.c +3 -1
index 9c9cb9d08bca..503c4d2105ee 100644
@@ -3092,7 +3092,9 @@ static void put_prev_task_scx(struct rq *rq, struct task_struct *p,
 		 * which should trigger an explicit follow-up scheduling event.
 		 */
 		if (next && sched_class_above(&ext_sched_class, next->sched_class)) {
-			WARN_ON_ONCE(!(sch->ops.flags & SCX_OPS_ENQ_LAST));
+			/* core-sched can force cpu idle while @p is runnable */
+			if (!sched_core_enabled(rq))
+				WARN_ON_ONCE(!(sch->ops.flags & SCX_OPS_ENQ_LAST));
 			do_enqueue_task(rq, p, SCX_ENQ_LAST, -1);
 		} else {
 			do_enqueue_task(rq, p, 0, -1);
reviewer
#4 lore raw

Hi Tejun,

Quoted 23 lines · Tejun Heo

put_prev_task_scx() warns when a runnable task drops to a lower sched_class without SCX_OPS_ENQ_LAST, assuming balance_one() would otherwise keep it running.

Under core scheduling that assumption is wrong: a forced-idle SMT sibling reschedules through the core_pick fast path in pick_next_task(), which skips pick_task_scx() for the CPU, so balance_one() never runs and a runnable task can drop to idle with ENQ_LAST unset. Skip the warning when core scheduling is enabled.

v2: Reworded the description per Kuba Piecuch's review. No code change.

Fixes 7c65ae81ea86 ("sched_ext: Don't call put_prev_task_scx() before picking the next task")
---
 kernel/sched/ext/ext.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
kernel/sched/ext/ext.c +0 -0
index 9c9cb9d08bca..503c4d2105ee 100644

kernel/sched/ext/ext.c static void put_prev_task_scx(struct rq *rq, struct task_struct *p, · line 3097

Quoted 7 lines
kernel/sched/ext/ext.c +3 -1
@@ -3092,7 +3092,9 @@ static void put_prev_task_scx(struct rq *rq, struct task_struct *p,
 		 * which should trigger an explicit follow-up scheduling event.
 		 */
 		if (next && sched_class_above(&ext_sched_class, next->sched_class)) {
-			WARN_ON_ONCE(!(sch->ops.flags & SCX_OPS_ENQ_LAST));
+			/* core-sched can force cpu idle while @p is runnable */
+			if (!sched_core_enabled(rq))
+				WARN_ON_ONCE(!(sch->ops.flags & SCX_OPS_ENQ_LAST));

I was wondering if this could be a better check:

WARN_ON_ONCE(sched_cpu_cookie_match(rq, p) &&

               !(sch->ops.flags & SCX_OPS_ENQ_LAST));

With this:
- the surrounding branch establishes that runnable p is being replaced by idle (class below sched_ext),
- the core picker has already stored the selected cookie,
- if p does not match that cookie, this CPU is being forced idle by core scheduling,
- when core scheduling is disabled, sched_cpu_cookie_match() returns true, so we preserve the warning
- when core scheduling is enabled and the cookies match the warning is also preserved.

In theory it should work, unless I'm missing some other edge cases.

Thanks,
-Andrea

Quoted 3 lines
kernel/sched/ext/ext.c +0 -0
 			do_enqueue_task(rq, p, SCX_ENQ_LAST, -1);
 		} else {
 			do_enqueue_task(rq, p, 0, -1);