Skip to contentSkip to conversation

sched_rt_rq_enqueue() resched idle

PATCH John Blackwood 2 participants 2 messages Aug 27, 2008

lore
Share & tools
lore
#0 lore raw
Quoted 20 lines · John Blackwood
Quoted 19 lines
Quoted 19 lines

Hi Peter,

When sysctl_sched_rt_runtime is set to something other than -1 and the CONFIG_RT_GROUP_SCHED kernel parameter is NOT enabled, we get into a state where we see one or more CPUs idling forvever even though there are real-time
tasks in their rt runqueue that are able to run (no longer throttled).

The sequence is:

- A real-time task is running when the timer sets the rt runqueue to throttled, and the rt task is resched_task()ed and switched out, and idle is switched in since there are no non-rt tasks to run on that cpu.

- Eventually the do_sched_rt_period_timer() runs and un-throttles the rt runqueue, but we just exit the timer interrupt and go back to executing the idle task in the idle loop forever.

Quoted 7 lines
Quoted 5 lines
Quoted 5 lines

If we change the sched_rt_rq_enqueue() routine to use some of the code from the CONFIG_RT_GROUP_SCHED enabled version of this same routine and resched_task() the currently executing task (idle in our case) if it is a lower priority task than the higher rt task in the now un-throttled runqueue, the problem is no longer observed.

Very good spotting, Thanks!

You're welcome.

Quoted 5 lines

However I think the patch isn't quite good, as highest_prio is only available on SMP || RT_GROUP_SCHED.

Furthermore, on !RT_GROUP_SCHED any RT task will be higher than current, so we can do the below, do you agree?

Yes, I see what you are saying.
The patch version below looks good.
I re-tested with it and it works fine.

Thanks!

Quoted 15 lines
kernel/sched_rt.c +2 -0
index 94daace..f672aee 100644
@@ -199,6 +199,8 @@ static inline struct rt_rq *group_rt_rq(struct sched_rt_entity *rt_se)
 
 static inline void sched_rt_rq_enqueue(struct rt_rq *rt_rq)
 {
+       if (rt_rq->rt_nr_running)
+               resched_task(rq_of_rt_rq(rt_rq)->curr);
 }
 
 static inline void sched_rt_rq_dequeue(struct rt_rq *rt_rq)
#1 lore raw

Ingo, please apply.

---
Subject: sched: sched_rt_rq_enqueue() resched idle
From: John Blackwood <[email protected]>
Date: Tue, 26 Aug 2008 15:09:43 -0400

When sysctl_sched_rt_runtime is set to something other than -1 and the CONFIG_RT_GROUP_SCHED kernel parameter is NOT enabled, we get into a state where we see one or more CPUs idling forvever even though there are real-time
tasks in their rt runqueue that are able to run (no longer throttled).

The sequence is:

- A real-time task is running when the timer sets the rt runqueue to throttled, and the rt task is resched_task()ed and switched out, and idle is switched in since there are no non-rt tasks to run on that cpu.

- Eventually the do_sched_rt_period_timer() runs and un-throttles the rt runqueue, but we just exit the timer interrupt and go back to executing the idle task in the idle loop forever.

If we change the sched_rt_rq_enqueue() routine to use some of the code from the CONFIG_RT_GROUP_SCHED enabled version of this same routine and resched_task() the currently executing task (idle in our case) if it is a lower priority task than the higher rt task in the now un-throttled runqueue, the problem is no longer observed.

---
 kernel/sched_rt.c |    2 ++
 1 file changed, 2 insertions(+)
kernel/sched_rt.c +2 -0
Index: linux-2.6/kernel/sched_rt.c
===================================================================
@@ -199,6 +199,8 @@ static inline struct rt_rq *group_rt_rq(
 
 static inline void sched_rt_rq_enqueue(struct rt_rq *rt_rq)
 {
+	if (rt_rq->rt_nr_running)
+		resched_task(rq_of_rt_rq(rt_rq)->curr);
 }
 
 static inline void sched_rt_rq_dequeue(struct rt_rq *rt_rq)