sched/fair: Clear throttle state when leaving the fair class
Subsystems (2)
PATCH 1/1 sched/fair: Clear throttle state when leaving the fair class
Task-based CFS bandwidth throttling removes a task's fair sched_entity while the scheduler core still considers the task queued. p->throttled carries the fair-specific state across dequeue/enqueue operations.
DEQUEUE_CLASS removes the task from cfs_rq->throttled_limbo_list but retains the bit. If the last finite CFS quota is disabled while the task runs in another class, it is absent from the unthrottle walk and the bit survives.
Returning to fair while cfs_bandwidth_used() is false then performs a real fair enqueue with p->throttled still set.
After bandwidth is re-enabled, a class-change dequeue treats the stale bit as limbo state and skips __dequeue_task(). p->se.on_rq therefore remains set, and put_prev_task_fair() places the entity in the EEVDF tree while the task moves to RT. On exit, the fair picker can select the TASK_DEAD task. __schedule() then returns to do_task_dead(), which hits BUG().
Keep p->throttled across migrations and group moves, but clear it on DEQUEUE_CLASS because fair-specific state must not outlive fair-class membership. A later return to fair uses the normal enqueue path. That path still respects a throttled cfs_rq.
A KASAN, LOCKDEP and DEBUG_LIST v7.3 kernel reproduced the do_task_dead() BUG in 3/3 root and 3/3 delegated uid-1000 runs with CapPrm=CapEff=0 and RLIMIT_RTPRIO=1. The fix passed both matrices plus a 3/3 class-switch control with bandwidth continuously enabled, without a scheduler warning, KASAN report, lockdep splat or crash.
An AI coding assistant analyzed the state mismatch and drafted the initial reproducer, fix candidate and this changelog. The source paths, trigger, controls and results were rechecked against the cited revisions.
--- kernel/sched/fair.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
kernel/sched/fair.c +2 -2
@@ -6957,8 +6957,8 @@ static void dequeue_throttled_task(struct task_struct *p, int flags)WARN_ON_ONCE (p->se.on_rq);list_del_init (&p->throttle_node); -/* task blocked after throttled */ -if (flags &DEQUEUE_SLEEP ) { +/* Task blocked after throttling or is leaving the fair class. */ +if (flags & (DEQUEUE_SLEEP |DEQUEUE_CLASS )) { p->throttled = false;return ; }
Signature · 1 line
2.55.0
Quoted 29 lines · Daehyeon Ko
Task-based CFS bandwidth throttling removes a task's fair
sched_entitywhile the scheduler core still considers the task queued.p->throttledcarries the fair-specific state across dequeue/enqueue operations.
DEQUEUE_CLASSremoves the task fromcfs_rq->throttled_limbo_listbut retains the bit. If the last finite CFS quota is disabled while the task runs in another class, it is absent from the unthrottle walk and the bit survives.
Returning to fair whilecfs_bandwidth_used()is false then performs a real fair enqueue withp->throttledstill set.After bandwidth is re-enabled, a class-change dequeue treats the stale bit as limbo state and skips
__dequeue_task().p->se.on_rqtherefore remains set, andput_prev_task_fair()places the entity in the EEVDF tree while the task moves to RT. On exit, the fair picker can select theTASK_DEADtask.__schedule()then returns todo_task_dead(), which hitsBUG().Keep
p->throttledacross migrations and group moves, but clear it onDEQUEUE_CLASSbecause fair-specific state must not outlive fair-class membership. A later return to fair uses the normal enqueue path. That path still respects a throttledcfs_rq.A KASAN, LOCKDEP and
DEBUG_LISTv7.3 kernel reproduced thedo_task_dead()BUG in 3/3 root and 3/3 delegated uid-1000 runs with CapPrm=CapEff=0 and RLIMIT_RTPRIO=1. The fix passed both matrices plus a 3/3 class-switch control with bandwidth continuously enabled, without a scheduler warning, KASAN report, lockdep splat or crash.
Quoted 7 lines
An AI coding assistant analyzed the state mismatch and drafted the initial reproducer, fix candidate and this changelog. The source paths, trigger, controls and results were rechecked against the cited revisions.
Fixes e1fad12dcb66 ("sched/fair: Switch to task based throttle model")Signed-off-by Daehyeon Ko <[email protected]>
Thanks!
Quoted 21 lines
--- kernel/sched/fair.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)kernel/sched/fair.c +2
-2@@ -6957,8 +6957,8 @@ static void dequeue_throttled_task(struct task_struct *p, int flags)WARN_ON_ONCE (p->se.on_rq);list_del_init (&p->throttle_node); -/* task blocked after throttled */ -if (flags &DEQUEUE_SLEEP ) { +/* Task blocked after throttling or is leaving the fair class. */ +if (flags & (DEQUEUE_SLEEP |DEQUEUE_CLASS )) { p->throttled = false;return ; }Signature · 1 line
2.55.0
Benjamin Segall
reviewerQuoted 29 lines · Daehyeon Ko
Task-based CFS bandwidth throttling removes a task's fair
sched_entitywhile the scheduler core still considers the task queued.p->throttledcarries the fair-specific state across dequeue/enqueue operations.
DEQUEUE_CLASSremoves the task fromcfs_rq->throttled_limbo_listbut retains the bit. If the last finite CFS quota is disabled while the task runs in another class, it is absent from the unthrottle walk and the bit survives.
Returning to fair whilecfs_bandwidth_used()is false then performs a real fair enqueue withp->throttledstill set.After bandwidth is re-enabled, a class-change dequeue treats the stale bit as limbo state and skips
__dequeue_task().p->se.on_rqtherefore remains set, andput_prev_task_fair()places the entity in the EEVDF tree while the task moves to RT. On exit, the fair picker can select theTASK_DEADtask.__schedule()then returns todo_task_dead(), which hitsBUG().Keep
p->throttledacross migrations and group moves, but clear it onDEQUEUE_CLASSbecause fair-specific state must not outlive fair-class membership. A later return to fair uses the normal enqueue path. That path still respects a throttledcfs_rq.A KASAN, LOCKDEP and
DEBUG_LISTv7.3 kernel reproduced thedo_task_dead()BUG in 3/3 root and 3/3 delegated uid-1000 runs with CapPrm=CapEff=0 and RLIMIT_RTPRIO=1. The fix passed both matrices plus a 3/3 class-switch control with bandwidth continuously enabled, without a scheduler warning, KASAN report, lockdep splat or crash.
Quoted 8 lines
An AI coding assistant analyzed the state mismatch and drafted the initial reproducer, fix candidate and this changelog. The source paths, trigger, controls and results were rechecked against the cited revisions.
Fixes e1fad12dcb66 ("sched/fair: Switch to task based throttle model")Signed-off-by Daehyeon Ko <[email protected]>---
Yep, sched_class switch lets you smuggle ->throttled into !cfs_bandwidth_used(), and then that check on enqueue_throttled lets you smuggle it onto actually being enqueued, which leads to corruption. That's fun.
I think we need to use switched_from_fair though - if we are currently TASK_ON_RQ_MIGRATING, sched_class changes will skip going into dequeue_task_fair at all, so we will not see the DEQUEUE_CLASS. Triggering a migration on a throttled task can still happen I'm fairly sure.
It's too bad, since it would be nice to just add to this existing touchpoint, but it's at least it's trivial:
kernel/sched/fair.c +4 -0
@@ -15500,10 +15500,14 @@ static void switching_from_fair(struct rq *rq, struct task_struct *p) }static void switched_from_fair (struct rq *rq,struct task_struct *p) {detach_task_cfs_rq (p); + +#ifdef CONFIG_CFS_BANDWIDTH + p->throttled = false; +#endif }static void switched_to_fair (struct rq *rq,struct task_struct *p) {WARN_ON_ONCE (p->se.sched_delayed);
(Or we could pull it out into its own function for cleanliness if it seems worthwhile)
Quoted 40 lines · Benjamin Segall
Quoted 29 lines · Daehyeon Ko
Task-based CFS bandwidth throttling removes a task's fair
sched_entitywhile the scheduler core still considers the task queued.p->throttledcarries the fair-specific state across dequeue/enqueue operations.
DEQUEUE_CLASSremoves the task fromcfs_rq->throttled_limbo_listbut retains the bit. If the last finite CFS quota is disabled while the task runs in another class, it is absent from the unthrottle walk and the bit survives.
Returning to fair whilecfs_bandwidth_used()is false then performs a real fair enqueue withp->throttledstill set.After bandwidth is re-enabled, a class-change dequeue treats the stale bit as limbo state and skips
__dequeue_task().p->se.on_rqtherefore remains set, andput_prev_task_fair()places the entity in the EEVDF tree while the task moves to RT. On exit, the fair picker can select theTASK_DEADtask.__schedule()then returns todo_task_dead(), which hitsBUG().Keep
p->throttledacross migrations and group moves, but clear it onDEQUEUE_CLASSbecause fair-specific state must not outlive fair-class membership. A later return to fair uses the normal enqueue path. That path still respects a throttledcfs_rq.A KASAN, LOCKDEP and
DEBUG_LISTv7.3 kernel reproduced thedo_task_dead()BUG in 3/3 root and 3/3 delegated uid-1000 runs with CapPrm=CapEff=0 and RLIMIT_RTPRIO=1. The fix passed both matrices plus a 3/3 class-switch control with bandwidth continuously enabled, without a scheduler warning, KASAN report, lockdep splat or crash.Quoted 8 lines
An AI coding assistant analyzed the state mismatch and drafted the initial reproducer, fix candidate and this changelog. The source paths, trigger, controls and results were rechecked against the cited revisions.
Fixes e1fad12dcb66 ("sched/fair: Switch to task based throttle model")Signed-off-by Daehyeon Ko <[email protected]>---
Quoted 8 lines
Yep,
sched_classswitch lets you smuggle ->throttled into !cfs_bandwidth_used(), and then that check onenqueue_throttledlets you smuggle it onto actually being enqueued, which leads to corruption. That's fun.I think we need to use
switched_from_fairthough - if we are currentlyTASK_ON_RQ_MIGRATING,sched_classchanges will skip going intodequeue_task_fairat all, so we will not see theDEQUEUE_CLASS.
sched class change serializes with migration I think, with task_rq_lock() which waits for migration to settle, so this should not happen?
Quoted 46 lines · Aaron Lu
Quoted 45 lines · Benjamin Segall
Quoted 29 lines · Daehyeon Ko
Task-based CFS bandwidth throttling removes a task's fair
sched_entitywhile the scheduler core still considers the task queued.p->throttledcarries the fair-specific state across dequeue/enqueue operations.
DEQUEUE_CLASSremoves the task fromcfs_rq->throttled_limbo_listbut retains the bit. If the last finite CFS quota is disabled while the task runs in another class, it is absent from the unthrottle walk and the bit survives.
Returning to fair whilecfs_bandwidth_used()is false then performs a real fair enqueue withp->throttledstill set.After bandwidth is re-enabled, a class-change dequeue treats the stale bit as limbo state and skips
__dequeue_task().p->se.on_rqtherefore remains set, andput_prev_task_fair()places the entity in the EEVDF tree while the task moves to RT. On exit, the fair picker can select theTASK_DEADtask.__schedule()then returns todo_task_dead(), which hitsBUG().Keep
p->throttledacross migrations and group moves, but clear it onDEQUEUE_CLASSbecause fair-specific state must not outlive fair-class membership. A later return to fair uses the normal enqueue path. That path still respects a throttledcfs_rq.A KASAN, LOCKDEP and
DEBUG_LISTv7.3 kernel reproduced thedo_task_dead()BUG in 3/3 root and 3/3 delegated uid-1000 runs with CapPrm=CapEff=0 and RLIMIT_RTPRIO=1. The fix passed both matrices plus a 3/3 class-switch control with bandwidth continuously enabled, without a scheduler warning, KASAN report, lockdep splat or crash.Quoted 8 lines
An AI coding assistant analyzed the state mismatch and drafted the initial reproducer, fix candidate and this changelog. The source paths, trigger, controls and results were rechecked against the cited revisions.
Fixes e1fad12dcb66 ("sched/fair: Switch to task based throttle model")Signed-off-by Daehyeon Ko <[email protected]>---
Yep,
sched_classswitch lets you smuggle ->throttled into !cfs_bandwidth_used(), and then that check onenqueue_throttledlets you smuggle it onto actually being enqueued, which leads to corruption. That's fun.
Quoted 6 lines
Quoted 3 lines
I think we need to use
switched_from_fairthough - if we are currentlyTASK_ON_RQ_MIGRATING,sched_classchanges will skip going intodequeue_task_fairat all, so we will not see theDEQUEUE_CLASS.sched class change serializes with migration I think, with
task_rq_lock()which waits for migration to settle, so this should not happen?
Right, of course, sigh. So then this should indeed be fine.
K Prateek Nayak
reviewerHello Ben,
Quoted 10 lines · Benjamin Segall
Yep,
sched_classswitch lets you smuggle ->throttled into !cfs_bandwidth_used(), and then that check onenqueue_throttledlets you smuggle it onto actually being enqueued, which leads to corruption. That's fun.I think we need to use
switched_from_fairthough - if we are currentlyTASK_ON_RQ_MIGRATING,sched_classchanges will skip going intodequeue_task_fairat all, so we will not see theDEQUEUE_CLASS. Triggering a migration on a throttled task can still happen I'm fairly sure.
We cannot grab the task_rq_lock() when the task is ON_RQ_MIGRATING and guard(sched_change) should never see TASK_ON_RQ_MIGRATING - the task is either fully blocked (!p->on_rq) or is queued (p->on_rq = 1) on a CPU when we are doing a class change.
Signature · 2 lines
Thanks and Regards, Prateek
The following commit has been merged into the sched/urgent branch of tip:
The following commit has been merged into the sched/urgent branch of tip:
Commit-ID: c282051fc2cae2ea54770ebe435dbea090272c25 Gitweb: https://git.kernel.org/tip/c282051fc2cae2ea54770ebe435dbea090272c25 Author: Daehyeon Ko <[email protected]> AuthorDate: Fri, 23 Oct 2026 08:52:33 +09:00 Committer: Ingo Molnar <[email protected]> CommitterDate: Fri, 06 Nov 2026 10:04:04 +01:00
sched/fair: Clear throttle state when leaving the fair class
Task-based CFS bandwidth throttling removes a task's fair sched_entity while the scheduler core still considers the task queued. p->throttled carries the fair-specific state across dequeue/enqueue operations.
DEQUEUE_CLASS removes the task from cfs_rq->throttled_limbo_list but retains the bit. If the last finite CFS quota is disabled while the task runs in another class, it is absent from the unthrottle walk and the bit survives.
Returning to fair while cfs_bandwidth_used() is false then performs a real fair enqueue with p->throttled still set.
After bandwidth is re-enabled, a class-change dequeue treats the stale bit as limbo state and skips __dequeue_task(). p->se.on_rq therefore remains set, and put_prev_task_fair() places the entity in the EEVDF tree while the task moves to RT. On exit, the fair picker can select the TASK_DEAD task. __schedule() then returns to do_task_dead(), which hits BUG().
Keep p->throttled across migrations and group moves, but clear it on DEQUEUE_CLASS because fair-specific state must not outlive fair-class membership. A later return to fair uses the normal enqueue path. That path still respects a throttled cfs_rq.
A KASAN, LOCKDEP and DEBUG_LIST v7.3 kernel reproduced the do_task_dead() BUG in 3/3 root and 3/3 delegated uid-1000 runs with CapPrm=CapEff=0 and RLIMIT_RTPRIO=1. The fix passed both matrices plus a 3/3 class-switch control with bandwidth continuously enabled, without a scheduler warning, KASAN report, lockdep splat or crash.
An AI coding assistant analyzed the state mismatch and drafted the initial reproducer, fix candidate and this changelog. The source paths, trigger, controls and results were rechecked against the cited revisions.
--- kernel/sched/fair.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
kernel/sched/fair.c +2 -2
@@ -6964,8 +6964,8 @@ static void dequeue_throttled_task(struct task_struct *p, int flags)WARN_ON_ONCE (p->se.on_rq);list_del_init (&p->throttle_node); -/* task blocked after throttled */ -if (flags &DEQUEUE_SLEEP ) { +/* Task blocked after throttling or is leaving the fair class: */ +if (flags & (DEQUEUE_SLEEP |DEQUEUE_CLASS )) { p->throttled = false;return ; }