sched/deadline: Server stops while fair tasks are still runnable
The boost model in [1] spotted cases with fair tasks running after the server stopped. I believe this means that it's equally possible for those tasks to starve without the server even trying to start.
There are 2 related situation that can result in this:
1. dl_server_idle is set to true when the CPU is really idle
2. a fair task wakes up (nothing happens since the server is still active)
3. dl_server_timer stops the server since it's idle, although it isn't really
```
<idle>-0 d.h3. 2.521220: event_boost: -12: idle x dl_replenish -> idle
<idle>-0 d.h3. 2.521227: event_laxity: -12: zero_laxity_wait x dl_replenish_idle -> idle_wait
thread3-3-580 d..4. 3.338067: event_boost: -12: idle x dl_server_resume_throttled -> throttled
thread3-3-580 d..3. 3.338076: sched_wakeup: ksoftirqd/12:118 [120] CPU:012
thread5-5-582 d.h2. 3.471237: event_boost: -12: throttled x dl_server_stop -> stopped (final)
thread5-5-582 d.h2. 3.471240: event_laxity: -12: idle_wait x dl_server_stop -> stopped (final)
ktimers/12-117 d..3. 3.538088: error_boost: -12: event sched_switch_in not expected in the state stopped
ktimers/12-117 d..2. 3.538089: sched_switch: ktimers/12:117 [98] S ==> ksoftirqd/12:118 [120]
```1. dl_server_timer runs when the CPU is idle but after a fair task wakes up
2. the call to update_curr sets dl_server_idle since it's only checking rq->curr (which is indeed idle)
3. the server is stopped within the same timer call although a fair task just woke up
```
ktimers/13-125 d.s52 7.309878: event_boost: -12: stopped x dl_server_start -> ready
ktimers/13-125 d.s52 7.309878: event_laxity: -12: stopped x dl_server_start -> zero_laxity_wait
ktimers/13-125 d.s42 7.309879: sched_wakeup: kworker/u519:2:5559 [120] CPU:012
<idle>-0 d.h3. 7.309889: event_boost: -12: ready x dl_replenish -> ready
<idle>-0 d.h3. 7.309889: event_laxity: -12: zero_laxity_wait x dl_replenish_idle -> idle_wait
<idle>-0 d.h3. 7.309890: event_boost: -12: ready x dl_server_stop -> stopped (final)
<idle>-0 d.h3. 7.309891: event_laxity: -12: idle_wait x dl_server_stop -> stopped (final)
<idle>-0 d..3. 7.309895: error_boost: -12: event sched_switch_in not expected in the state stopped
<idle>-0 d..2. 7.309896: sched_switch: swapper/12:0 [120] R ==> kworker/u519:2:5559 [120]
```In both cases, if no new fair task wakes up, the one that woke up right before stopping the server could starve. That's a problem isn't it?
I tried a quick solution to the first case by clearing dl_server_idle every time the server started (even if it's still active, that is at every fair wakeup) and I think the second case could be avoided by relying on idle_cpu() instead of looking only on rq->curr, taking also waking tasks into account (not tried).
Is this a real issue or is the monitor just too sensitive?
Thanks,
Gabriele
Juri Lelli
maintainerHi Gabriele,
Quoted 41 lines · Gabriele Monaco
The boost model in [1] spotted cases with fair tasks running after the server stopped. I believe this means that it's equally possible for those tasks to starve without the server even trying to start.
There are 2 related situation that can result in this:
1.
dl_server_idleis set to true when the CPU is really idle
2. a fair task wakes up (nothing happens since the server is still active)
3.dl_server_timerstops the server since it's idle, although it isn't really``` <idle>-0 d.h3. 2.521220: event_boost: -12: idle x dl_replenish -> idle <idle>-0 d.h3. 2.521227: event_laxity: -12: zero_laxity_wait x dl_replenish_idle -> idle_wait thread3-3-580 d..4. 3.338067: event_boost: -12: idle x dl_server_resume_throttled -> throttled thread3-3-580 d..3. 3.338076: sched_wakeup: ksoftirqd/12:118 [120] CPU:012 thread5-5-582 d.h2. 3.471237: event_boost: -12: throttled x dl_server_stop -> stopped (final) thread5-5-582 d.h2. 3.471240: event_laxity: -12: idle_wait x dl_server_stop -> stopped (final) ktimers/12-117 d..3. 3.538088: error_boost: -12: event sched_switch_in not expected in the state stopped ktimers/12-117 d..2. 3.538089: sched_switch: ktimers/12:117 [98] S ==> ksoftirqd/12:118 [120] ```1.
dl_server_timerruns when the CPU is idle but after a fair task wakes up
2. the call toupdate_currsetsdl_server_idlesince it's only checkingrq->curr(which is indeed idle)
3. the server is stopped within the same timer call although a fair task just woke up``` ktimers/13-125 d.s52 7.309878: event_boost: -12: stopped x dl_server_start -> ready ktimers/13-125 d.s52 7.309878: event_laxity: -12: stopped x dl_server_start -> zero_laxity_wait ktimers/13-125 d.s42 7.309879: sched_wakeup: kworker/u519:2:5559 [120] CPU:012 <idle>-0 d.h3. 7.309889: event_boost: -12: ready x dl_replenish -> ready <idle>-0 d.h3. 7.309889: event_laxity: -12: zero_laxity_wait x dl_replenish_idle -> idle_wait <idle>-0 d.h3. 7.309890: event_boost: -12: ready x dl_server_stop -> stopped (final) <idle>-0 d.h3. 7.309891: event_laxity: -12: idle_wait x dl_server_stop -> stopped (final) <idle>-0 d..3. 7.309895: error_boost: -12: event sched_switch_in not expected in the state stopped <idle>-0 d..2. 7.309896: sched_switch: swapper/12:0 [120] R ==> kworker/u519:2:5559 [120] ```
Quoted 2 lines
In both cases, if no new fair task wakes up, the one that woke up right before stopping the server could starve. That's a problem isn't it?
Yeah, think so. :/
Looks like we clear dl_defer_idle only while updating the fair task entity (which didn't happen yet in both cases above).
Quoted 4 lines
I tried a quick solution to the first case by clearing
dl_server_idleevery time the server started (even if it's still active, that is at every fair wakeup) and I think the second case could be avoided by relying onidle_cpu()instead of looking only onrq->curr, taking also waking tasks into account (not tried).
Not sure if you want to wait for Peter to chime in, but if you have patches we can definitely take a look. :)
Thanks,
Juri
PATCH 1/1 sched/deadline: Fix server stopping with runnable tasks
The deadline server can currently stop due to idle although fair tasks are runnable. This happens essentially when:
* the server is set to idle, a task wakes up, the server stops
* a task wakes up, the server sets itself to idle and stops right away
Address both cases by clearing the server idle flag whenever a fair task wakes up and accounting also for pending tasks in the definition of idle.
---
This is really quick and dirty but seems to fix it according to the models. I also updated the chart to show what is happening here. I added the same event (server_resume) to the model.
kernel/sched/deadline.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-)
kernel/sched/deadline.c +8 -4
@@ -1376,7 +1376,7 @@ update_stats_dequeue_dl(struct dl_rq *dl_rq, struct sched_dl_entity *dl_se, intstatic void update_curr_dl_se (struct rq *rq,struct sched_dl_entity *dl_se,s64 delta_exec) { -bool idle = rq->curr == rq->idle; +bool idle = rq->curr == rq->idle && !rq->nr_running && !rq->ttwu_pending;s64 scaled_delta_exec;if (unlikely (delta_exec <=0 )) {
@@ -1560,8 +1560,8 @@ void dl_server_update(struct sched_dl_entity *dl_se, s64 delta_exec)* | 8 | B:zero_laxity-wait | | | * | | | <---+ | * | +--------------------------------+ | -* | | ^ ^ 2 | -* | | 7 | 2 +--------------------+ +* | | ^ ^ 2 | +* | | 7 | 2, 1 +----------------+ * | v | * | +-------------+ | * +-- | C:idle-wait | -+
@@ -1606,8 +1606,11 @@ void dl_server_update(struct sched_dl_entity *dl_se, s64 delta_exec)* dl_defer_idle = 0 * * -* [1] A->B, A->D +* [1] A->B, A->D, C->B * dl_server_start() +* dl_defer_idle = 0; +* if (dl_server_active) +* return; // [B] * dl_server_active = 1; * enqueue_dl_entity() * update_dl_entity(WAKEUP)
@@ -1741,6 +1744,7 @@ void dl_server_start(struct sched_dl_entity *dl_se) {struct rq *rq = dl_se->rq; + dl_se->dl_defer_idle =0 ;if (!dl_server (dl_se) || dl_se->dl_server_active)return ;
base-commit: 0f61b1860cc3f52aef9036d7235ed1f017632193
Signature · 1 line
2.52.0
Peter Zijlstra
maintainerRe: [PATCH] sched/deadline: Fix server stopping with runnable tasks
Quoted 24 lines · Gabriele Monaco
The deadline server can currently stop due to idle although fair tasks are runnable. This happens essentially when:
* the server is set to idle, a task wakes up, the server stops
* a task wakes up, the server sets itself to idle and stops right awayAddress both cases by clearing the server idle flag whenever a fair task wakes up and accounting also for pending tasks in the definition of idle.
Signed-off-by Gabriele Monaco <[email protected]>---
This is really quick and dirty but seems to fix it according to the models. I also updated the chart to show what is happening here. I added the same event (
server_resume) to the model.kernel/sched/deadline.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-)kernel/sched/deadline.c +0
-0
kernel/sched/deadline.c update_stats_dequeue_dl(struct dl_rq *dl_rq, struct sched_dl_entity *dl_se, int · line 1379
Quoted 5 lines
kernel/sched/deadline.c +1
-1@@ -1376,7 +1376,7 @@ update_stats_dequeue_dl(struct dl_rq *dl_rq, struct sched_dl_entity *dl_se, intstatic void update_curr_dl_se (struct rq *rq,struct sched_dl_entity *dl_se,s64 delta_exec) { -bool idle = rq->curr == rq->idle; +bool idle = rq->curr == rq->idle && !rq->nr_running && !rq->ttwu_pending;
This is idle_cpu(), perhaps we can lift that thing into sched.h or so.
Quoted 28 lines
kernel/sched/deadline.c +6
-3s64 scaled_delta_exec;if (unlikely (delta_exec <=0 )) {@@ -1560,8 +1560,8 @@ void dl_server_update(struct sched_dl_entity *dl_se, s64 delta_exec)* | 8 | B:zero_laxity-wait | | | * | | | <---+ | * | +--------------------------------+ | -* | | ^ ^ 2 | -* | | 7 | 2 +--------------------+ +* | | ^ ^ 2 | +* | | 7 | 2, 1 +----------------+ * | v | * | +-------------+ | * +-- | C:idle-wait | -+ @@ -1606,8 +1606,11 @@ void dl_server_update(struct sched_dl_entity *dl_se, s64 delta_exec)* dl_defer_idle = 0 * * -* [1] A->B, A->D +* [1] A->B, A->D, C->B * dl_server_start() +* dl_defer_idle = 0; +* if (dl_server_active) +* return; // [B] * dl_server_active = 1; * enqueue_dl_entity() * update_dl_entity(WAKEUP)
kernel/sched/deadline.c void dl_server_start(struct sched_dl_entity *dl_se) · line 1749
Quoted 6 lines
kernel/sched/deadline.c +1
-0@@ -1741,6 +1744,7 @@ void dl_server_start(struct sched_dl_entity *dl_se) {struct rq *rq = dl_se->rq; + dl_se->dl_defer_idle =0 ;if (!dl_server (dl_se) || dl_se->dl_server_active)return ;
Yeah, this seems sensible.
Let me pick it up and do that idle_cpu() thing.
Re: [PATCH] sched/deadline: Fix server stopping with runnable tasks
Quoted 9 lines · Peter Zijlstra
Quoted 6 lines
static void
update_curr_dl_se(struct rq *rq, struct sched_dl_entity *dl_se, s64 delta_exec)
{(quoted diff) +1
-1- bool idle = rq->curr == rq->idle; + bool idle = rq->curr == rq->idle && !rq->nr_running && !rq-Quoted 1 line
ttwu_pending;This is
idle_cpu(), perhaps we can lift that thing intosched.hor so.
Yeah, that's what I meant by quick and dirty.. I have idle_cpu() on the model side (and have them matching simplifies a lot of things).
I just wasn't sure if we wanted a function call in there. But we can probably do something nicer without it too.
Thanks,
Gabriele
Peter Zijlstra
maintainerRe: [PATCH] sched/deadline: Fix server stopping with runnable tasks
Quoted 8 lines · Gabriele Monaco
Quoted 7 lines · Peter Zijlstra
Quoted 6 lines
static void
update_curr_dl_se(struct rq *rq, struct sched_dl_entity *dl_se, s64 delta_exec)
{(quoted diff) +1
-1- bool idle = rq->curr == rq->idle; + bool idle = rq->curr == rq->idle && !rq->nr_running && !rq-Quoted 1 line
ttwu_pending;
Quoted 8 lines
Quoted 2 lines
This is
idle_cpu(), perhaps we can lift that thing intosched.hor so.Yeah, that's what I meant by quick and dirty.. I have
idle_cpu()on the model side (and have them matching simplifies a lot of things).I just wasn't sure if we wanted a function call in there. But we can probably do something nicer without it too.
I'm test building this...
---
include/linux/sched.h +0 -1
@@ -1869,7 +1869,6 @@ static inline int task_nice(const struct task_struct *p)extern int can_nice (const struct task_struct *p,const int nice);extern int task_curr (const struct task_struct *p);extern int idle_cpu (int cpu); -extern int available_idle_cpu (int cpu);extern int sched_setscheduler (struct task_struct *,int ,const struct sched_param *);extern int sched_setscheduler_nocheck (struct task_struct *,int ,const struct sched_param *);extern void sched_set_fifo (struct task_struct *p);
kernel/sched/deadline.c +8 -4
@@ -1420,7 +1420,7 @@ update_stats_dequeue_dl(struct dl_rq *dl_rq, struct sched_dl_entity *dl_se, intstatic void update_curr_dl_se (struct rq *rq,struct sched_dl_entity *dl_se,s64 delta_exec) { -bool idle = rq->curr == rq->idle; +bool idle =idle_rq (rq);s64 scaled_delta_exec;if (unlikely (delta_exec <=0 )) {
@@ -1603,8 +1603,8 @@ void dl_server_update(struct sched_dl_entity *dl_se, s64 delta_exec)* | 8 | B:zero_laxity-wait | | | * | | | <---+ | * | +--------------------------------+ | -* | | ^ ^ 2 | -* | | 7 | 2 +--------------------+ +* | | ^ ^ 2 | +* | | 7 | 2, 1 +----------------+ * | v | * | +-------------+ | * +-- | C:idle-wait | -+
@@ -1649,8 +1649,11 @@ void dl_server_update(struct sched_dl_entity *dl_se, s64 delta_exec)* dl_defer_idle = 0 * * -* [1] A->B, A->D +* [1] A->B, A->D, C->B * dl_server_start() +* dl_defer_idle = 0; +* if (dl_server_active) +* return; // [B] * dl_server_active = 1; * enqueue_dl_entity() * update_dl_entity(WAKEUP)
@@ -1784,6 +1787,7 @@ void dl_server_start(struct sched_dl_entity *dl_se) {struct rq *rq = dl_se->rq; + dl_se->dl_defer_idle =0 ;if (!dl_server (dl_se) || dl_se->dl_server_active)return ;
kernel/sched/sched.h +22 -0
@@ -1380,6 +1380,28 @@ static inline u32 sched_rng(void)#define cpu_curr (cpu) (cpu_rq(cpu)->curr)#define raw_rq () raw_cpu_ptr(&runqueues) +static inline bool idle_rq (struct rq *rq) +{ +return rq->curr == rq->idle && !rq->nr_running && !rq->ttwu_pending; +} + +/** +* available_idle_cpu - is a given CPU idle for enqueuing work. +* @cpu: the CPU in question. +* +* Return: 1 if the CPU is currently idle. 0 otherwise. +*/ +static inline bool available_idle_cpu (int cpu) +{ +if (!idle_rq (cpu_rq (cpu))) +return 0 ; + +if (vcpu_is_preempted (cpu)) +return 0 ; + +return 1 ; +} +#ifdef CONFIG_SCHED_PROXY_EXEC static inline void rq_set_donor (struct rq *rq,struct task_struct *t) {
kernel/sched/syscalls.c +1 -29
@@ -180,35 +180,7 @@ int task_prio(const struct task_struct *p)*/ int idle_cpu (int cpu) { -struct rq *rq =cpu_rq (cpu); - -if (rq->curr != rq->idle) -return 0 ; - -if (rq->nr_running) -return 0 ; - -if (rq->ttwu_pending) -return 0 ; - -return 1 ; -} - -/** -* available_idle_cpu - is a given CPU idle for enqueuing work. -* @cpu: the CPU in question. -* -* Return: 1 if the CPU is currently idle. 0 otherwise. -*/ -int available_idle_cpu (int cpu) -{ -if (!idle_cpu (cpu)) -return 0 ; - -if (vcpu_is_preempted (cpu)) -return 0 ; - -return 1 ; +return idle_rq (cpu_rq (cpu)); }/**
Juri Lelli
maintainerRe: [PATCH] sched/deadline: Fix server stopping with runnable tasks
In reply to Gabriele MonacoQuoted 8 lines · Gabriele Monaco
Quoted 7 lines · Peter Zijlstra
Quoted 6 lines
static void
update_curr_dl_se(struct rq *rq, struct sched_dl_entity *dl_se, s64 delta_exec)
{(quoted diff) +1
-1- bool idle = rq->curr == rq->idle; + bool idle = rq->curr == rq->idle && !rq->nr_running && !rq-Quoted 1 line
ttwu_pending;
Quoted 8 lines
Quoted 2 lines
This is
idle_cpu(), perhaps we can lift that thing intosched.hor so.Yeah, that's what I meant by quick and dirty.. I have
idle_cpu()on the model side (and have them matching simplifies a lot of things).I just wasn't sure if we wanted a function call in there. But we can probably do something nicer without it too.
Thanks for the patch (looks sensible to me too) and nice catch from the model!
[tip: sched/urgent] sched/deadline: Fix server stopping with runnable tasks
In reply to Gabriele MonacoThe following commit has been merged into the sched/urgent branch of tip:
The following commit has been merged into the sched/urgent branch of tip:
Commit-ID: ca1e8eede4fc68ce85a9fdce1a6c13ad64933318 Gitweb: https://git.kernel.org/tip/ca1e8eede4fc68ce85a9fdce1a6c13ad64933318 Author: Gabriele Monaco <[email protected]> AuthorDate: Tue, 13 Jan 2026 09:52:01 +01:00 Committer: Peter Zijlstra <[email protected]> CommitterDate: Tue, 13 Jan 2026 11:37:52 +01:00
sched/deadline: Fix server stopping with runnable tasks
The deadline server can currently stop due to idle although fair tasks are runnable. This happens essentially when:
* the server is set to idle, a task wakes up, the server stops
* a task wakes up, the server sets itself to idle and stops right away
Address both cases by clearing the server idle flag whenever a fair task wakes up and accounting also for pending tasks in the definition of idle.
--- kernel/sched/deadline.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-)
kernel/sched/deadline.c +9 -4
@@ -1420,7 +1420,7 @@ update_stats_dequeue_dl(struct dl_rq *dl_rq, struct sched_dl_entity *dl_se, intstatic void update_curr_dl_se (struct rq *rq,struct sched_dl_entity *dl_se,s64 delta_exec) { -bool idle = rq->curr == rq->idle; +bool idle =idle_rq (rq);s64 scaled_delta_exec;if (unlikely (delta_exec <=0 )) {
@@ -1603,8 +1603,8 @@ void dl_server_update(struct sched_dl_entity *dl_se, s64 delta_exec)* | 8 | B:zero_laxity-wait | | | * | | | <---+ | * | +--------------------------------+ | -* | | ^ ^ 2 | -* | | 7 | 2 +--------------------+ +* | | ^ ^ 2 | +* | | 7 | 2, 1 +----------------+ * | v | * | +-------------+ | * +-- | C:idle-wait | -+
@@ -1649,8 +1649,11 @@ void dl_server_update(struct sched_dl_entity *dl_se, s64 delta_exec)* dl_defer_idle = 0 * * -* [1] A->B, A->D +* [1] A->B, A->D, C->B * dl_server_start() +* dl_defer_idle = 0; +* if (dl_server_active) +* return; // [B] * dl_server_active = 1; * enqueue_dl_entity() * update_dl_entity(WAKEUP)
@@ -1759,6 +1762,7 @@ void dl_server_update(struct sched_dl_entity *dl_se, s64 delta_exec)* "B:zero_laxity-wait" -> "C:idle-wait" [label="7:dl_server_update_idle"] * "B:zero_laxity-wait" -> "D:running" [label="3:dl_server_timer"] * "C:idle-wait" -> "A:init" [label="8:dl_server_timer"] +* "C:idle-wait" -> "B:zero_laxity-wait" [label="1:dl_server_start"] * "C:idle-wait" -> "B:zero_laxity-wait" [label="2:dl_server_update"] * "C:idle-wait" -> "C:idle-wait" [label="7:dl_server_update_idle"] * "D:running" -> "A:init" [label="4:pick_task_dl"]
@@ -1784,6 +1788,7 @@ void dl_server_start(struct sched_dl_entity *dl_se) {struct rq *rq = dl_se->rq; + dl_se->dl_defer_idle =0 ;if (!dl_server (dl_se) || dl_se->dl_server_active)return ;