RT Load balance changes in sched-devel
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
[ cc'ed lkml ]
I guess, one possible load-balancing point is out of consideration -- sched_setscheduler()
(also rt_mutex_setprio()).
(1) NORMAL --> RT, when p->se.on_rq == 1 && ! task_running(rq, p)
(2) RT --> NORMAL, when task_running(rq, p) == 1
e.g. for (2) we may even get a completely idle rq (schedule() --> schedule_balance_rt() will not help due to schedule_balance_rt() having a rt_task(prev) check in place... and 'prev' is of NORMAL type when it's scheduled out).
btw., both cases would be addressed by placing load-balance points into sched_class_rt->{enqueue,dequeue}_task_rt()... push_rt_tasks() and pull_rt_tasks() respectively. As a side effect (I think, technically, it would be possible), 3 out of 4 *_balance_rt() calls (the exception: schedule_tail_balance_rt()) in schedule() would become unnecessary.
_BUT_
the enqueue/dequeue() interface would become less straightforward, logically-wise.
Something like:
rq = activate_task(rq, ...) ; /* may unlock rq and lock/return another one */
would complicate the existing use cases.
Signature · 2 lines
Best regards, Dmitry Adamushko
Hi Dmitry,
Quoted 1 line
Quoted 1 line
Quoted 1 line
On Sun, Dec 9, 2007 at 12:16 PM, in message
<[email protected]>, "Dmitry
Quoted 6 lines · Adamushko
[ cc'ed lkml ]
I guess, one possible load-balancing point is out of consideration --
sched_setscheduler()
(alsort_mutex_setprio()).
Quoted 8 lines
(1) NORMAL --> RT, when
p->se.on_rq== 1 && !task_running(rq, p)(2) RT --> NORMAL, when
task_running(rq, p)== 1e.g. for (2) we may even get a completely idle rq (
schedule()-->schedule_balance_rt()will not help due toschedule_balance_rt()having art_task(prev)check in place... and 'prev' is of NORMAL type when it's scheduled out).
Indeed. I think you are correct on both counts. This is an oversight, so good eyes!
Quoted 11 lines
btw., both cases would be addressed by placing load-balance points into sched_class_rt->{enqueue,dequeue}_task_rt()...
push_rt_tasks()andpull_rt_tasks()respectively. As a side effect (I think, technically, it would be possible), 3 out of 4*_balance_rt()calls (the exception:schedule_tail_balance_rt()) inschedule()would become unnecessary._BUT_
Quoted 9 lines
the enqueue/dequeue() interface would become less straightforward, logically-wise.
Something like:rq =
activate_task(rq, ...); /* may unlock rq and lock/return another one */would complicate the existing use cases.
I think I would prefer to just fix the setscheduler/setprio cases for the class transition than change the behavior of these enqueue/dequeue calls. But I will keep an open mind as I look into this issue.
Thanks for the review!
-Greg
Quoted 7 lines · Gregory Haskins
Quoted 7 lines
btw., both cases would be addressed by placing load-balance points into sched_class_rt->{enqueue,dequeue}_task_rt()...
push_rt_tasks()andpull_rt_tasks()respectively. As a side effect (I think, technically, it would be possible), 3 out of 4*_balance_rt()calls (the exception:schedule_tail_balance_rt()) inschedule()would become unnecessary.
Quoted 5 lines
Quoted 5 lines
_BUT_
the enqueue/dequeue() interface would become less straightforward, logically-wise.
Something like:
Also push and pull_rt use activate,deactivate as well. So this would make that code a bit more complex.
Quoted 7 lines
Quoted 5 lines
rq =
activate_task(rq, ...); /* may unlock rq and lock/return another one */would complicate the existing use cases.
I think I would prefer to just fix the setscheduler/setprio cases for the class transition than change the behavior of these enqueue/dequeue calls. But I will keep an open mind as I look into this issue.
I agree with Gregory on this. I prefer to fix the two you found. I thought about them before, but somehow they were missed :-/
Anyway, I'll be working on adding some more patches on Monday. There may be other ways to clean this up.
Quoted 2 lines
Thanks for the review!
Yeah, thanks from me too!
-- Steve
RFC PATCH 1/1 sched: Fixed missed rt-balance points on priority shifts
In reply to Gregory HaskinsHi Ingo, Steven, Dmitry,
Here is a proposed fix for the issue that Dmitry brought up today. It should apply cleanly to sched-devel (though I have a few of my other submitted fixes queued ahead of this that are not yet in sched-devel...so if you have a problem let me know and I will rebase/resubmit)
Regards,
-Greg
--------------------------
sched: Fixed missed rt-balance points on priority shifts
Dmitry Adamushko identified several holes in the rt-migration stategy relating to changing priority via sched_setscheduler or rt_mutex_setprio:
http://lkml.org/lkml/2007/12/9/94
This patch should button up those conditions.
---
kernel/sched.c | 8 ++++++++ kernel/sched_rt.c | 46 +++++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 53 insertions(+), 1 deletions(-)
kernel/sched.c +8 -0
@@ -348,6 +348,7 @@ struct rt_rq {/* highest queued rt task prio */ int highest_prio;int overloaded; +int needs_pull; };#ifdef CONFIG_SMP
@@ -4037,6 +4038,9 @@ void rt_mutex_setprio(struct task_struct *p, int prio)check_preempt_curr (rq, p); } } + +wakeup_balance_rt (rq, p); +task_rq_unlock (rq, &flags); }
@@ -4341,6 +4345,9 @@ recheck:check_preempt_curr (rq, p); } } + +wakeup_balance_rt (rq, p); +__task_rq_unlock (rq);spin_unlock_irqrestore (&p->pi_lock, flags);
@@ -6887,6 +6894,7 @@ void __init sched_init(void)INIT_LIST_HEAD (&rq->migration_queue); rq->rt.highest_prio =MAX_RT_PRIO ; rq->rt.overloaded =0 ; + rq->rt.needs_pull =0 ;#endif atomic_set (&rq->nr_iowait,0 );
kernel/sched_rt.c +45 -1
65 lines not on this page · raw message
Re: [PATCH RFC] sched: Fixed missed rt-balance points on priority shifts
Quoted 1 line
Quoted 1 line
Quoted 1 line
On Sun, Dec 9, 2007 at 9:53 PM, in message
<[email protected]>, Gregory Haskins
kernel/sched_rt.c static inline void dec_rt_tasks(struct task_struct *p, struct rq *rq) · line 142
Quoted 8 lines · <[email protected]> wrote:
kernel/sched_rt.c +8
-0+* I have no doubt that this is the proper thing to do to make +* sure RT tasks are properly balanced. What I cannot wrap my +* head around at this late hour is if issuing a reschedule() +* here may cause issues in other circumstances. TBD +*/ +if (!task_running (rq, p)) +resched_task (rq->curr); + }
It dawned on me after I sent this that a further optimization here is to predicate the reschedule on whether we are overloaded. In otherwords:
if (!task_running(rq, p) && rt_overloaded(rq))
Regards,
-Greg