sched: improve tick time missed wakeup preempt protection
PATCH 1/1 sched: improve tick time missed wakeup preempt protection
sched: improve tick time missed wakeup preempt protection
f685ceac provides protection from tasks just missing wakeup preemption, and then having to wait a full slice. However, it offers this protection to tasks which have no business receiving the benefit, namely SCHED_BATCH and SCHED_IDLE. It also treats all tasks equally, which obviously isn't true. Exclude tasks of other than SCHED_NORMAL class, and scale minimum runtime before a tick time preemption by the difference in task weights, after which, we can just use the standard wakeup preempt vruntime delta test, sysctl_sched_wakeup_granularity.
LKML-Reference: <new-submission>
--- kernel/sched_fair.c | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-)
kernel/sched_fair.c +13 -7
@@ -830,17 +830,23 @@ check_preempt_tick(struct cfs_rq *cfs_rq* narrow margin doesn't have to wait for a full slice. * This also mitigates buddy induced latencies under load. */ -if (!sched_feat (WAKEUP_PREEMPT )) +if (!sched_feat (WAKEUP_PREEMPT ) || cfs_rq->nr_running <2 )return ; - -if (delta_exec < sysctl_sched_min_granularity) -return ; - -if (cfs_rq->nr_running >1 ) { +else {struct sched_entity *se =__pick_next_entity (cfs_rq); +unsigned long min = sysctl_sched_min_granularity;s64 delta = curr->vruntime - se->vruntime; -if (delta > ideal_runtime) +if (task_of (se)->policy !=SCHED_NORMAL ) +return ; +if (delta <0 ) +return ; +if (curr->load.weight != se->load.weight) + min =calc_delta_mine (min, curr->load.weight, &se->load); +if (delta_exec < min) +return ; + +if (delta > sysctl_sched_wakeup_granularity)resched_task (rq_of (cfs_rq)->curr); } }
Quoted 44 lines · Mike Galbraith
sched: improve tick time missed wakeup preempt protection
f685ceacprovides protection from tasks just missing wakeup preemption, and then having to wait a full slice. However, it offers this protection to tasks which have no business receiving the benefit, namelySCHED_BATCHandSCHED_IDLE. It also treats all tasks equally, which obviously isn't true. Exclude tasks of other thanSCHED_NORMALclass, and scale minimum runtime before a tick time preemption by the difference in task weights, after which, we can just use the standard wakeup preempt vruntime delta test,sysctl_sched_wakeup_granularity.Signed-off-by Mike Galbraith <[email protected]>Cc Ingo Molnar <[email protected]>Cc Peter Zijlstra <[email protected]>LKML-Reference: <new-submission>
--- kernel/sched_fair.c | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-)Index:
linux-2.6/kernel/sched_fair.c
===================================================================kernel/sched_fair.c +6
-7@@ -830,17 +830,23 @@ check_preempt_tick(struct cfs_rq *cfs_rq* narrow margin doesn't have to wait for a full slice. * This also mitigates buddy induced latencies under load. */ -if (!sched_feat (WAKEUP_PREEMPT )) +if (!sched_feat (WAKEUP_PREEMPT ) || cfs_rq->nr_running <2 )return ; - -if (delta_exec < sysctl_sched_min_granularity) -return ; - -if (cfs_rq->nr_running >1 ) { +else {struct sched_entity *se =__pick_next_entity (cfs_rq); +unsigned long min = sysctl_sched_min_granularity;s64 delta = curr->vruntime - se->vruntime; -if (delta > ideal_runtime) +if (task_of (se)->policy !=SCHED_NORMAL ) +return ; +if (delta <0 )
kernel/sched_fair.c check_preempt_tick(struct cfs_rq *cfs_rq · line 852
Quoted 10 lines
kernel/sched_fair.c +7
-0+return ; +if (curr->load.weight != se->load.weight) + min =calc_delta_mine (min, curr->load.weight, &se->load); +if (delta_exec < min) +return ; + +if (delta > sysctl_sched_wakeup_granularity)resched_task (rq_of (cfs_rq)->curr); } }
You can loose the else, the if branch does an unconditional return, there's no other way to get below there than 'else' ;-)
Quoted 2 lines · Peter Zijlstra
You can loose the else, the if branch does an unconditional return, there's no other way to get below there than 'else' ;-)
Ok. Can't plug tail into a function, doesn't fit on a line, so..
sched: improve tick time missed wakeup preempt protection
f685ceac provides protection from tasks just missing wakeup preemption, and then having to wait a full slice. However, it offers this protection to tasks which have no business receiving the benefit, namely SCHED_BATCH and SCHED_IDLE. It also treats all tasks equally, which obviously isn't true. Exclude tasks of other than SCHED_NORMAL class, and scale minimum runtime before a tick time preemption by the difference in task weights, after which, we can just use the standard wakeup preempt vruntime delta test, sysctl_sched_wakeup_granularity.
LKML-Reference: <new-submission>
--- kernel/sched_fair.c | 27 +++++++++++++++++---------- 1 file changed, 17 insertions(+), 10 deletions(-)
kernel/sched_fair.c +17 -10
@@ -811,7 +811,10 @@ dequeue_entity(struct cfs_rq *cfs_rq, ststatic void check_preempt_tick (struct cfs_rq *cfs_rq,struct sched_entity *curr) { +struct sched_entity *next;unsigned long ideal_runtime, delta_exec; +unsigned long min = sysctl_sched_min_granularity; +s64 delta; ideal_runtime =sched_slice (cfs_rq, curr); delta_exec = curr->sum_exec_runtime - curr->prev_sum_exec_runtime;
@@ -825,24 +828,28 @@ check_preempt_tick(struct cfs_rq *cfs_rqreturn ; } +if (!sched_feat (WAKEUP_PREEMPT ) || cfs_rq->nr_running <2 ) +return ; +/* * Ensure that a task that missed wakeup preemption by a * narrow margin doesn't have to wait for a full slice. * This also mitigates buddy induced latencies under load. */ -if (!sched_feat (WAKEUP_PREEMPT )) -return ; + next =__pick_next_entity (cfs_rq); + delta = curr->vruntime - next->vruntime; -if (delta_exec < sysctl_sched_min_granularity) +if (task_of (next)->policy !=SCHED_NORMAL ) +return ; +if (delta <0 ) +return ; +if (curr->load.weight != next->load.weight) + min =calc_delta_mine (min, curr->load.weight, &next->load); +if (delta_exec < min)return ; -if (cfs_rq->nr_running >1 ) { -struct sched_entity *se =__pick_next_entity (cfs_rq); -s64 delta = curr->vruntime - se->vruntime; - -if (delta > ideal_runtime) -resched_task (rq_of (cfs_rq)->curr); - } +if (delta > sysctl_sched_wakeup_granularity) +resched_task (rq_of (cfs_rq)->curr); } static void