sched/core: Tweak wait_task_inactive() to force dequeue sched_delayed tasks
RFC PATCH v2 1/1 sched/core: Tweak wait_task_inactive() to force dequeue sched_delayed tasks
It was reported that in 6.12, smpboot_create_threads() was taking much longer then in 6.6.
I narrowed down the call path to:
smpboot_create_threads()
-> kthread_create_on_cpu()
-> kthread_bind()
-> __kthread_bind_mask()
->wait_task_inactive()Where in wait_task_inactive() we were regularly hitting the queued case, which sets a 1 tick timeout, which when called multiple times in a row, accumulates quickly into a long delay.
I noticed disabling the DELAY_DEQUEUE sched feature recovered the performance, and it seems the newly create tasks are usually sched_delayed and left on the runqueue.
So in wait_task_inactive() when we see the task p->se.sched_delayed, manually dequeue the sched_delayed task with DEQUEUE_DELAYED, so we don't have to constantly wait a tick.
This seems to work, but I've only lightly tested it, so I'd love close review and feedback in case I've missed something in wait_task_inactive(), or if there is a simpler alternative approach.
NOTE: Peter did highlight[1] his general distaste for the kthread_bind() through wait_task_inactive() functions, which suggests a deeper rework might be better, but I'm not familiar enough with all its users to have a sense of how that might be done, and this fix seems to address the problem and be more easily backported to 6.12-stable, so I wanted to submit it again, as a potentially more short-term solution.
[1]: https://lore.kernel.org/lkml/[email protected]/
---
v2:
* Rework & simplify the check as suggested by K Prateek Nayak
* Added Reported-by tag for proper attribution
--- kernel/sched/core.c | 6 ++++++ 1 file changed, 6 insertions(+)
kernel/sched/core.c +6 -0
@@ -2283,6 +2283,12 @@ unsigned long wait_task_inactive(struct task_struct *p, unsigned int match_state* just go back and repeat. */ rq =task_rq_lock (p, &rf); +/* +* If task is sched_delayed, force dequeue it, to avoid always +* hitting the tick timeout in the queued case +*/ +if (p->se.sched_delayed) +dequeue_task (rq, p,DEQUEUE_SLEEP |DEQUEUE_DELAYED );trace_sched_wait_task (p); running =task_on_cpu (rq, p); queued =task_on_rq_queued (p);
Signature · 1 line
2.49.0.850.g28803427d3-goog
Hello John,
Quoted 39 lines · John Stultz
It was reported that in 6.12,
smpboot_create_threads()was taking much longer then in 6.6.I narrowed down the call path to:
smpboot_create_threads()
->kthread_create_on_cpu()-> kthread_bind() -> __kthread_bind_mask() ->wait_task_inactive()Where in
wait_task_inactive()we were regularly hitting the queued case, which sets a 1 tick timeout, which when called multiple times in a row, accumulates quickly into a long delay.I noticed disabling the
DELAY_DEQUEUEsched feature recovered the performance, and it seems the newly create tasks are usuallysched_delayedand left on the runqueue.So in
wait_task_inactive()when we see the taskp->se.sched_delayed, manually dequeue thesched_delayedtask withDEQUEUE_DELAYED, so we don't have to constantly wait a tick.This seems to work, but I've only lightly tested it, so I'd love close review and feedback in case I've missed something in
wait_task_inactive(), or if there is a simpler alternative approach.NOTE: Peter did highlight[1] his general distaste for the
kthread_bind()throughwait_task_inactive()functions, which suggests a deeper rework might be better, but I'm not familiar enough with all its users to have a sense of how that might be done, and this fix seems to address the problem and be more easily backported to 6.12-stable, so I wanted to submit it again, as a potentially more short-term solution.
Quoted 12 lines
Cc Ingo Molnar <[email protected]>Cc Peter Zijlstra <[email protected]>Cc Juri Lelli <[email protected]>Cc Vincent Guittot <[email protected]>Cc Dietmar Eggemann <[email protected]>Cc Steven Rostedt <[email protected]>Cc Ben Segall <[email protected]>Cc Mel Gorman <[email protected]>Cc Valentin Schneider <[email protected]>Cc K Prateek Nayak <[email protected]>Reported-by [email protected]
Perhaps a Fixes tag to help with backporting:
other than thank, feel free to include:
Signature · 2 lines
Thanks and Regards, Prateek
Quoted 25 lines
Signed-off-by John Stultz <[email protected]>> ---v2:
* Rework & simplify the check as suggested by K Prateek Nayak
* Added Reported-by tag for proper attribution--- kernel/sched/core.c | 6 ++++++ 1 file changed, 6 insertions(+)kernel/sched/core.c +6
-0@@ -2283,6 +2283,12 @@ unsigned long wait_task_inactive(struct task_struct *p, unsigned int match_state* just go back and repeat. */ rq =task_rq_lock (p, &rf); +/* +* If task is sched_delayed, force dequeue it, to avoid always +* hitting the tick timeout in the queued case +*/ +if (p->se.sched_delayed) +dequeue_task (rq, p,DEQUEUE_SLEEP |DEQUEUE_DELAYED );trace_sched_wait_task (p); running =task_on_cpu (rq, p); queued =task_on_rq_queued (p);