Skip to contentSkip to conversation

sched/core: Tweak wait_task_inactive() to force dequeue sched_delayed tasks

RFC PATCH Superseded John Stultz 2 participants 2 messages Apr 25, 2025 – Apr 28, 2025 Scheduler

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

RFC PATCH v2 1/1 sched/core: Tweak wait_task_inactive() to force dequeue sched_delayed tasks

It was reported that in 6.12, smpboot_create_threads() was taking much longer then in 6.6.

I narrowed down the call path to:
smpboot_create_threads()
-> kthread_create_on_cpu()

    -> kthread_bind()
       -> __kthread_bind_mask()
          ->wait_task_inactive()

Where in wait_task_inactive() we were regularly hitting the queued case, which sets a 1 tick timeout, which when called multiple times in a row, accumulates quickly into a long delay.

I noticed disabling the DELAY_DEQUEUE sched feature recovered the performance, and it seems the newly create tasks are usually sched_delayed and left on the runqueue.

So in wait_task_inactive() when we see the task p->se.sched_delayed, manually dequeue the sched_delayed task with DEQUEUE_DELAYED, so we don't have to constantly wait a tick.

This seems to work, but I've only lightly tested it, so I'd love close review and feedback in case I've missed something in wait_task_inactive(), or if there is a simpler alternative approach.

NOTE: Peter did highlight[1] his general distaste for the kthread_bind() through wait_task_inactive() functions, which suggests a deeper rework might be better, but I'm not familiar enough with all its users to have a sense of how that might be done, and this fix seems to address the problem and be more easily backported to 6.12-stable, so I wanted to submit it again, as a potentially more short-term solution.

[1]: https://lore.kernel.org/lkml/[email protected]/

Cc Ingo Molnar <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Juri Lelli <[email protected]>
Cc Vincent Guittot <[email protected]>
Cc Dietmar Eggemann <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Ben Segall <[email protected]>
Cc Mel Gorman <[email protected]>
Cc Valentin Schneider <[email protected]>
Cc K Prateek Nayak <[email protected]>
Reported-by [email protected]

---
v2:
* Rework & simplify the check as suggested by K Prateek Nayak
* Added Reported-by tag for proper attribution

---
 kernel/sched/core.c | 6 ++++++
 1 file changed, 6 insertions(+)
kernel/sched/core.c +6 -0
index c81cf642dba05..b986cd2fb19b7 100644
@@ -2283,6 +2283,12 @@ unsigned long wait_task_inactive(struct task_struct *p, unsigned int match_state
 		 * just go back and repeat.
 		 */
 		rq = task_rq_lock(p, &rf);
+		/*
+		 * If task is sched_delayed, force dequeue it, to avoid always
+		 * hitting the tick timeout in the queued case
+		 */
+		if (p->se.sched_delayed)
+			dequeue_task(rq, p, DEQUEUE_SLEEP | DEQUEUE_DELAYED);
 		trace_sched_wait_task(p);
 		running = task_on_cpu(rq, p);
 		queued = task_on_rq_queued(p);
Signature · 1 line
2.49.0.850.g28803427d3-goog
reviewer Tested-by on v2 1/1
#1 lore raw

Hello John,

Quoted 39 lines · John Stultz

It was reported that in 6.12, smpboot_create_threads() was taking much longer then in 6.6.

I narrowed down the call path to:
smpboot_create_threads()
-> kthread_create_on_cpu()

     -> kthread_bind()
        -> __kthread_bind_mask()
           ->wait_task_inactive()

Where in wait_task_inactive() we were regularly hitting the queued case, which sets a 1 tick timeout, which when called multiple times in a row, accumulates quickly into a long delay.

I noticed disabling the DELAY_DEQUEUE sched feature recovered the performance, and it seems the newly create tasks are usually sched_delayed and left on the runqueue.

So in wait_task_inactive() when we see the task p->se.sched_delayed, manually dequeue the sched_delayed task with DEQUEUE_DELAYED, so we don't have to constantly wait a tick.

This seems to work, but I've only lightly tested it, so I'd love close review and feedback in case I've missed something in wait_task_inactive(), or if there is a simpler alternative approach.

NOTE: Peter did highlight[1] his general distaste for the kthread_bind() through wait_task_inactive() functions, which suggests a deeper rework might be better, but I'm not familiar enough with all its users to have a sense of how that might be done, and this fix seems to address the problem and be more easily backported to 6.12-stable, so I wanted to submit it again, as a potentially more short-term solution.

[1]: https://lore.kernel.org/lkml/[email protected]/

Quoted 12 lines
Cc Ingo Molnar <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Juri Lelli <[email protected]>
Cc Vincent Guittot <[email protected]>
Cc Dietmar Eggemann <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Ben Segall <[email protected]>
Cc Mel Gorman <[email protected]>
Cc Valentin Schneider <[email protected]>
Cc K Prateek Nayak <[email protected]>
Reported-by [email protected]

Perhaps a Fixes tag to help with backporting:

Fixes 152e11f6df29 ("sched/fair: Implement delayed dequeue")

other than thank, feel free to include:

Signature · 2 lines
Thanks and Regards,
Prateek
Quoted 25 lines

v2:
* Rework & simplify the check as suggested by K Prateek Nayak
* Added Reported-by tag for proper attribution

---
  kernel/sched/core.c | 6 ++++++
  1 file changed, 6 insertions(+)
kernel/sched/core.c +6 -0
index c81cf642dba05..b986cd2fb19b7 100644
@@ -2283,6 +2283,12 @@ unsigned long wait_task_inactive(struct task_struct *p, unsigned int match_state
  		 * just go back and repeat.
  		 */
  		rq = task_rq_lock(p, &rf);
+		/*
+		 * If task is sched_delayed, force dequeue it, to avoid always
+		 * hitting the tick timeout in the queued case
+		 */
+		if (p->se.sched_delayed)
+			dequeue_task(rq, p, DEQUEUE_SLEEP | DEQUEUE_DELAYED);
  		trace_sched_wait_task(p);
  		running = task_on_cpu(rq, p);
  		queued = task_on_rq_queued(p);