sched/fair: reduce preemption with IDLE tasks runable
PATCH 1/1 sched/fair: reduce preemption with IDLE tasks runable
From: Jiang Biao <[email protected]>
No need to preempt when there are only one runable CFS task with other IDLE tasks on runqueue. The only one CFS task would always be picked in that case.
--- kernel/sched/fair.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/sched/fair.c +1 -1
@@ -4527,7 +4527,7 @@ entity_tick(struct cfs_rq *cfs_rq, struct sched_entity *curr, int queued)return ;#endif -if (cfs_rq->nr_running >1 ) +if (cfs_rq->nr_running > cfs_rq.idle_h_nr_running +1 )check_preempt_tick (cfs_rq, curr); }
Signature · 1 line
2.21.0
Dietmar Eggemann
reviewerQuoted 16 lines · Jiang Biao
From: Jiang Biao <[email protected]>
No need to preempt when there are only one runable CFS task with other IDLE tasks on runqueue. The only one CFS task would always be picked in that case.
Signed-off-by Jiang Biao <[email protected]>--- kernel/sched/fair.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)kernel/sched/fair.c +0
-0
kernel/sched/fair.c entity_tick(struct cfs_rq *cfs_rq, struct sched_entity *curr, int queued) · line 4530
Quoted 5 lines
kernel/sched/fair.c +1
-1@@ -4527,7 +4527,7 @@ entity_tick(struct cfs_rq *cfs_rq, struct sched_entity *curr, int queued)return ;#endif -if (cfs_rq->nr_running >1 ) +if (cfs_rq->nr_running > cfs_rq.idle_h_nr_running +1 )
cfs_rq is a pointer.
kernel/sched/fair.c entity_tick(struct cfs_rq *cfs_rq, struct sched_entity *curr, int queued) · line 4532
Quoted 2 lines
kernel/sched/fair.c +0
-0check_preempt_tick (cfs_rq, curr); }
You can't compare cfs_rq->nr_running with cfs_rq->idle_h_nr_running!
There is a difference between cfs_rq->h_nr_running and cfs_rq->nr_running. The '_h_' stands for hierarchical.
The former gives you hierarchical task accounting whereas the latter is the number of sched entities (representing tasks or taskgroups) enqueued in cfs_rq.
In entity_tick(), cfs_rq->nr_running has to be used for the condition to call check_preempt_tick(). We want to check if curr has to be preempted by __pick_first_entity(cfs_rq) on this cfs_rq.
entity_tick() is called for each sched entity (and so for each cfs_rq_of(se)) of the task group hierarchy (e.g. task p running in taskgroup /A/B : se(p) -> se(A/B) -> se(A)).
Re: [PATCH] sched/fair: reduce preemption with IDLE tasks runable(Internet mail)
Quoted 24 lines · Dietmar Eggemann
Quoted 19 lines · Jiang Biao
From: Jiang Biao <[email protected]>
No need to preempt when there are only one runable CFS task with other IDLE tasks on runqueue. The only one CFS task would always be picked in that case.
Signed-off-by Jiang Biao <[email protected]>---
kernel/sched/fair.c| 2 +-1 file changed, 1 insertion(+), 1 deletion(-)kernel/sched/fair.c +0
-0@@ -4527,7 +4527,7 @@ entity_tick(struct cfs_rq *cfs_rq, struct sched_entity *curr, int queued)return ;#endif
Quoted 4 lines
kernel/sched/fair.c entity_tick(struct cfs_rq *cfs_rq, struct sched_entity *curr, int queued) · line 4530
Quoted 2 lines
kernel/sched/fair.c +1
-1-if (cfs_rq->nr_running >1 ) +if (cfs_rq->nr_running > cfs_rq.idle_h_nr_running +1 )
cfs_rqis a pointer.
It is. Sorry about that. :)
Quoted 13 lines
kernel/sched/fair.c entity_tick(struct cfs_rq *cfs_rq, struct sched_entity *curr, int queued) · line 4532
Quoted 3 lines
kernel/sched/fair.c +0
-0check_preempt_tick (cfs_rq, curr); }You can't compare
cfs_rq->nr_runningwithcfs_rq->idle_h_nr_running!There is a difference between
cfs_rq->h_nr_runningandcfs_rq->nr_running. The '_h_' stands for hierarchical.The former gives you hierarchical task accounting whereas the latter is the number of sched entities (representing tasks or taskgroups) enqueued in
cfs_rq.
Quoted 7 lines
In
entity_tick(),cfs_rq->nr_runninghas to be used for the condition to callcheck_preempt_tick(). We want to check if curr has to be preempted by__pick_first_entity(cfs_rq)on thiscfs_rq.
entity_tick()is called for each sched entity (and so for eachcfs_rq_of(se)) of the task group hierarchy (e.g. task p running in taskgroup /A/B : se(p) -> se(A/B) -> se(A)).
That’s true. I was thinking adding a new cfs_rq->idle_nr_running member to track the per cfs_rq's IDLE task number, and reducing preemption here based on that.
I’m not sure if it’s ok to do that, because the IDLE class seems not to be so pure that could tolerate starving.
We need an absolutely low priority class that could tolerate starving, which could be used to co-locate offline tasks. But IDLE class seems to be not *low* enough, if considering the fairness of CFS, and IDLE class still has a weight.
Thanks for you reply.
Regards,
Jiang
Dietmar Eggemann
reviewerRe: [PATCH] sched/fair: reduce preemption with IDLE tasks runable(Internet mail)
Quoted 47 lines · benbjiang(蒋彪)
Quoted 24 lines · Dietmar Eggemann
Quoted 19 lines · Jiang Biao
From: Jiang Biao <[email protected]>
No need to preempt when there are only one runable CFS task with other IDLE tasks on runqueue. The only one CFS task would always be picked in that case.
Signed-off-by Jiang Biao <[email protected]>---
kernel/sched/fair.c| 2 +-1 file changed, 1 insertion(+), 1 deletion(-)kernel/sched/fair.c +0
-0@@ -4527,7 +4527,7 @@ entity_tick(struct cfs_rq *cfs_rq, struct sched_entity *curr, int queued)return ;#endif Quoted 4 lines
kernel/sched/fair.c entity_tick(struct cfs_rq *cfs_rq, struct sched_entity *curr, int queued) · line 4530
Quoted 2 lines
kernel/sched/fair.c +1
-1-if (cfs_rq->nr_running >1 ) +if (cfs_rq->nr_running > cfs_rq.idle_h_nr_running +1 )
cfs_rqis a pointer.It is. Sorry about that. :)
Quoted 17 lines
kernel/sched/fair.c entity_tick(struct cfs_rq *cfs_rq, struct sched_entity *curr, int queued) · line 4532
Quoted 3 lines
kernel/sched/fair.c +0
-0check_preempt_tick (cfs_rq, curr); }You can't compare
cfs_rq->nr_runningwithcfs_rq->idle_h_nr_running!There is a difference between
cfs_rq->h_nr_runningandcfs_rq->nr_running. The '_h_' stands for hierarchical.The former gives you hierarchical task accounting whereas the latter is the number of sched entities (representing tasks or taskgroups) enqueued in
cfs_rq.In
entity_tick(),cfs_rq->nr_runninghas to be used for the condition to callcheck_preempt_tick(). We want to check if curr has to be preempted by__pick_first_entity(cfs_rq)on thiscfs_rq.
Quoted 6 lines
Quoted 3 lines
entity_tick()is called for each sched entity (and so for eachcfs_rq_of(se)) of the task group hierarchy (e.g. task p running in taskgroup /A/B : se(p) -> se(A/B) -> se(A)).That’s true. I was thinking adding a new
cfs_rq->idle_nr_runningmember to track the per cfs_rq's IDLE task number, and reducing preemption here based on that.
How would you deal with se's representing taskgroups which contain SCHED_IDLE and SCHED_NORMAL tasks or other taskgroups doing that?
Quoted 2 lines
I’m not sure if it’s ok to do that, because the IDLE class seems not to be so pure that could tolerate starving.
Not sure I understand but idle_sched_class is not the same as SCHED_IDLE (policy)?
Quoted 4 lines
We need an absolutely low priority class that could tolerate starving, which could be used to co-locate offline tasks. But IDLE class seems to be not *low* enough, if considering the fairness of CFS, and IDLE class still has a weight.
[...]
Re: [PATCH] sched/fair: reduce preemption with IDLE tasks runable(Internet mail)
Hi,
Quoted 53 lines · Dietmar Eggemann
Quoted 50 lines · benbjiang(蒋彪)
Quoted 24 lines · Dietmar Eggemann
Quoted 19 lines · Jiang Biao
From: Jiang Biao <[email protected]>
No need to preempt when there are only one runable CFS task with other IDLE tasks on runqueue. The only one CFS task would always be picked in that case.
Signed-off-by Jiang Biao <[email protected]>---
kernel/sched/fair.c| 2 +-1 file changed, 1 insertion(+), 1 deletion(-)kernel/sched/fair.c +0
-0@@ -4527,7 +4527,7 @@ entity_tick(struct cfs_rq *cfs_rq, struct sched_entity *curr, int queued)return ;#endif Quoted 4 lines
kernel/sched/fair.c entity_tick(struct cfs_rq *cfs_rq, struct sched_entity *curr, int queued) · line 4530
Quoted 2 lines
kernel/sched/fair.c +1
-1-if (cfs_rq->nr_running >1 ) +if (cfs_rq->nr_running > cfs_rq.idle_h_nr_running +1 )
cfs_rqis a pointer.It is. Sorry about that. :)
Quoted 20 lines
kernel/sched/fair.c entity_tick(struct cfs_rq *cfs_rq, struct sched_entity *curr, int queued) · line 4532
Quoted 3 lines
kernel/sched/fair.c +0
-0check_preempt_tick (cfs_rq, curr); }You can't compare
cfs_rq->nr_runningwithcfs_rq->idle_h_nr_running!There is a difference between
cfs_rq->h_nr_runningandcfs_rq->nr_running. The '_h_' stands for hierarchical.The former gives you hierarchical task accounting whereas the latter is the number of sched entities (representing tasks or taskgroups) enqueued in
cfs_rq.In
entity_tick(),cfs_rq->nr_runninghas to be used for the condition to callcheck_preempt_tick(). We want to check if curr has to be preempted by__pick_first_entity(cfs_rq)on thiscfs_rq.
entity_tick()is called for each sched entity (and so for eachcfs_rq_of(se)) of the task group hierarchy (e.g. task p running in taskgroup /A/B : se(p) -> se(A/B) -> se(A)).
Quoted 6 lines
Quoted 3 lines
That’s true. I was thinking adding a new
cfs_rq->idle_nr_runningmember to track the per cfs_rq's IDLE task number, and reducing preemption here based on that.How would you deal with se's representing taskgroups which contain
SCHED_IDLEandSCHED_NORMALtasks or other taskgroups doing that?
I’m not sure I get the point. :) How about the following patch,
kernel/sched/fair.c +8 -1
@@ -2994,6 +2994,9 @@ account_entity_enqueue(struct cfs_rq *cfs_rq, struct sched_entity *se) list_add(&se->group_node, &rq->cfs_tasks); }#endif +if (task_has_idle_policy (task_of (se))) + cfs_rq->idle_nr_running++; + cfs_rq->nr_running++; }
@@ -3007,6 +3010,9 @@ account_entity_dequeue(struct cfs_rq *cfs_rq, struct sched_entity *se) list_del_init(&se->group_node); }#endif +if (task_has_idle_policy (task_of (se))) + cfs_rq->idle_nr_running--; + cfs_rq->nr_running--; }
@@ -4527,7 +4533,7 @@ entity_tick(struct cfs_rq *cfs_rq, struct sched_entity *curr, int queued)return ;#endif -if (cfs_rq->nr_running >1 ) +if (cfs_rq->nr_running > cfs_rq->idle_nr_running +1 && + cfs_rq->h_nr_running - cfs_rq->idle_h_nr_running > cfs_rq->idle_nr_running +1 )check_preempt_tick (cfs_rq, curr); }
kernel/sched/sched.h +1 -0
@@ -500,6 +500,7 @@ struct cfs_bandwidth { };struct cfs_rq {struct load_weight load;unsigned int nr_running; +unsigned int idle_nr_running;unsigned int h_nr_running;/* SCHED_{NORMAL,BATCH,IDLE} */ unsigned int idle_h_nr_running;/* SCHED_IDLE */
Quoted 6 lines
Quoted 3 lines
I’m not sure if it’s ok to do that, because the IDLE class seems not to be so pure that could tolerate starving.
Not sure I understand but
idle_sched_classis not the same asSCHED_IDLE(policy)?
The case is that we need tasks(low priority, called offline tasks) to utilize the spare cpu left by CFS SCHED_NORMAL tasks(called online tasks) without interfering the online tasks.
Offline tasks only run when there’s no runnable online tasks, and offline tasks never preempt online tasks.
The SCHED_IDLE policy seems not to be abled to be qualified for that requirement, because it has a weight(3), even though it’s small, but it can still preempt online tasks considering the fairness. In that way, offline tasks of SCHED_IDLE policy could interfere the online tasks.
On the other hand, idle_sched_class seems not to be qualified either. It’s too simple and only used for per-cpu idle task currently.
Thx.
Regards,
Jiang
Quoted 8 lines
Quoted 5 lines
We need an absolutely low priority class that could tolerate starving, which could be used to co-locate offline tasks. But IDLE class seems to be not *low* enough, if considering the fairness of CFS, and IDLE class still has a weight.
[...]
Dietmar Eggemann
reviewerRe: [PATCH] sched/fair: reduce preemption with IDLE tasks runable(Internet mail)
Quoted 11 lines · benbjiang(蒋彪)
Hi,
Quoted 9 lines · Dietmar Eggemann
Quoted 6 lines · benbjiang(蒋彪)
Quoted 6 lines · Dietmar Eggemann
Quoted 1 line · Jiang Biao
From: Jiang Biao <[email protected]>
[...]
Quoted 43 lines
Quoted 2 lines
How would you deal with se's representing taskgroups which contain
SCHED_IDLEandSCHED_NORMALtasks or other taskgroups doing that?I’m not sure I get the point. :) How about the following patch,
kernel/sched/fair.c +8
-1@@ -2994,6 +2994,9 @@ account_entity_enqueue(struct cfs_rq *cfs_rq, struct sched_entity *se) list_add(&se->group_node, &rq->cfs_tasks); }#endif +if (task_has_idle_policy (task_of (se))) + cfs_rq->idle_nr_running++; + cfs_rq->nr_running++; }@@ -3007,6 +3010,9 @@ account_entity_dequeue(struct cfs_rq *cfs_rq, struct sched_entity *se) list_del_init(&se->group_node); }#endif +if (task_has_idle_policy (task_of (se))) + cfs_rq->idle_nr_running--; + cfs_rq->nr_running--; }@@ -4527,7 +4533,7 @@ entity_tick(struct cfs_rq *cfs_rq, struct sched_entity *curr, int queued)return ;#endif -if (cfs_rq->nr_running >1 ) +if (cfs_rq->nr_running > cfs_rq->idle_nr_running +1 && + cfs_rq->h_nr_running - cfs_rq->idle_h_nr_running > cfs_rq->idle_nr_running +1 )check_preempt_tick (cfs_rq, curr); }kernel/sched/sched.h +0
-0
kernel/sched/sched.h struct cfs_bandwidth { }; · line 505
Quoted 6 lines
kernel/sched/sched.h +1
-0@@ -500,6 +500,7 @@ struct cfs_bandwidth { };struct cfs_rq {struct load_weight load;unsigned int nr_running; +unsigned int idle_nr_running;unsigned int h_nr_running;/* SCHED_{NORMAL,BATCH,IDLE} */ unsigned int idle_h_nr_running;/* SCHED_IDLE */
/
/ | \
A n0 i0
/ \
n1 i1I don't think this will work. E.g. the patch would prevent tick preemption between 'A' and 'n0' on '/' as well
(3 > 1 + 1) && (4 - 2 > 1 + 1)
You also have to make sure that a SCHED_IDLE task can tick preempt another SCHED_IDLE task.
Quoted 5 lines
Quoted 5 lines
Quoted 2 lines
I’m not sure if it’s ok to do that, because the IDLE class seems not to be so pure that could tolerate starving.
Not sure I understand but
idle_sched_classis not the same asSCHED_IDLE(policy)?
Quoted 9 lines
The case is that we need
tasks(low priority, called offline tasks)to utilize the spare cpu left by CFSSCHED_NORMALtasks(called online tasks) without interfering the online tasks.
Offline tasks only run when there’s no runnable online tasks, and offline tasks never preempt online tasks.
TheSCHED_IDLEpolicy seems not to be abled to be qualified for that requirement, because it has a weight(3), even though it’s small, but it can still preempt online tasks considering the fairness. In that way, offline tasks ofSCHED_IDLEpolicy could interfere the online tasks.
Because of this very small weight (weight=3), compared to a SCHED_NORMAL nice 0 task (weight=1024), a SCHED_IDLE task is penalized by a huge se->vruntime value (1024/3 higher than for a SCHED_NORMAL nice 0 task). This should make sure it doesn't tick preempt a SCHED_NORMAL nice 0 task.
It's different when the SCHED_NORMAL task has nice 19 (weight=15) but that's part of the CFS design.
Quoted 2 lines
On the other hand,
idle_sched_classseems not to be qualified either. It’s too simple and only used for per-cpu idle task currently.
Yeah, leave this for the rq->idle task (swapper/X).
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
We need an absolutely low priority class that could tolerate starving, which could be used to co-locate offline tasks. But IDLE class seems to be not *low* enough, if considering the fairness of CFS, and IDLE class still has a weight.
Understood. But this (tick) preemption should happen extremely rarely, especially if you have SCHED_NORMAL nice 0 tasks, right?
Re: [PATCH] sched/fair: reduce preemption with IDLE tasks runable(Internet mail)
Hi,
Quoted 73 lines · Dietmar Eggemann
Quoted 11 lines · benbjiang(蒋彪)
Hi,
Quoted 9 lines · Dietmar Eggemann
Quoted 6 lines · benbjiang(蒋彪)
Quoted 6 lines · Dietmar Eggemann
Quoted 1 line · Jiang Biao
From: Jiang Biao <[email protected]>
[...]
Quoted 43 lines
Quoted 2 lines
How would you deal with se's representing taskgroups which contain
SCHED_IDLEandSCHED_NORMALtasks or other taskgroups doing that?I’m not sure I get the point. :) How about the following patch,
kernel/sched/fair.c +8
-1@@ -2994,6 +2994,9 @@ account_entity_enqueue(struct cfs_rq *cfs_rq, struct sched_entity *se) list_add(&se->group_node, &rq->cfs_tasks); }#endif +if (task_has_idle_policy (task_of (se))) + cfs_rq->idle_nr_running++; + cfs_rq->nr_running++; }@@ -3007,6 +3010,9 @@ account_entity_dequeue(struct cfs_rq *cfs_rq, struct sched_entity *se) list_del_init(&se->group_node); }#endif +if (task_has_idle_policy (task_of (se))) + cfs_rq->idle_nr_running--; + cfs_rq->nr_running--; }@@ -4527,7 +4533,7 @@ entity_tick(struct cfs_rq *cfs_rq, struct sched_entity *curr, int queued)return ;#endif -if (cfs_rq->nr_running >1 ) +if (cfs_rq->nr_running > cfs_rq->idle_nr_running +1 && + cfs_rq->h_nr_running - cfs_rq->idle_h_nr_running > cfs_rq->idle_nr_running +1 )check_preempt_tick (cfs_rq, curr); }kernel/sched/sched.h +0
-0kernel/sched/sched.h struct cfs_bandwidth { }; · line 505
Quoted 6 lines
kernel/sched/sched.h +1
-0@@ -500,6 +500,7 @@ struct cfs_bandwidth { };struct cfs_rq {struct load_weight load;unsigned int nr_running; +unsigned int idle_nr_running;unsigned int h_nr_running;/* SCHED_{NORMAL,BATCH,IDLE} */ unsigned int idle_h_nr_running;/* SCHED_IDLE */ / / | \ A n0 i0 / \n1 i1
Quoted 7 lines
I don't think this will work. E.g. the patch would prevent tick preemption between 'A' and 'n0' on '/' as well
(3 > 1 + 1) && (4 - 2 > 1 + 1)
You also have to make sure that a
SCHED_IDLEtask can tick preempt anotherSCHED_IDLEtask.
That’s right. :)
Quoted 16 lines
Quoted 6 lines
Quoted 6 lines
Quoted 3 lines
I’m not sure if it’s ok to do that, because the IDLE class seems not to be so pure that could tolerate starving.
Not sure I understand but
idle_sched_classis not the same asSCHED_IDLE(policy)?Quoted 9 lines
The case is that we need
tasks(low priority, called offline tasks)to utilize the spare cpu left by CFSSCHED_NORMALtasks(called online tasks) without interfering the online tasks.
Offline tasks only run when there’s no runnable online tasks, and offline tasks never preempt online tasks.
TheSCHED_IDLEpolicy seems not to be abled to be qualified for that requirement, because it has a weight(3), even though it’s small, but it can still preempt online tasks considering the fairness. In that way, offline tasks ofSCHED_IDLEpolicy could interfere the online tasks.
Quoted 4 lines
Because of this very small weight (weight=3), compared to a
SCHED_NORMALnice 0 task (weight=1024), aSCHED_IDLEtask is penalized by a hugese->vruntimevalue (1024/3 higher than for aSCHED_NORMALnice 0 task). This should make sure it doesn't tick preempt aSCHED_NORMALnice 0 task.
Could you please explain how the huge penalization of vruntime(1024/3) could make sure SCHED_IDLE not tick preempting SCHED_NORMAL nice 0 task?
Thanks a lot.
Regards,
Jiang
Quoted 8 lines
It's different when the
SCHED_NORMALtask has nice 19 (weight=15) but that's part of the CFS design.Quoted 2 lines
On the other hand,
idle_sched_classseems not to be qualified either. It’s too simple and only used for per-cpu idle task currently.Yeah, leave this for the
rq->idletask (swapper/X).
Got it.
Quoted 8 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
We need an absolutely low priority class that could tolerate starving, which could be used to co-locate offline tasks. But IDLE class seems to be not *low* enough, if considering the fairness of CFS, and IDLE class still has a weight.
Understood. But this (tick) preemption should happen extremely rarely, especially if you have
SCHED_NORMALnice 0 tasks, right?
Dietmar Eggemann
reviewerRe: [PATCH] sched/fair: reduce preemption with IDLE tasks runable(Internet mail)
Quoted 16 lines · benbjiang(蒋彪)
Hi,
Quoted 14 lines · Dietmar Eggemann
Quoted 11 lines · benbjiang(蒋彪)
Hi,
Quoted 9 lines · Dietmar Eggemann
Quoted 6 lines · benbjiang(蒋彪)
Quoted 6 lines · Dietmar Eggemann
Quoted 1 line · Jiang Biao
From: Jiang Biao <[email protected]>
[...]
Quoted 8 lines
Quoted 4 lines
Because of this very small weight (weight=3), compared to a
SCHED_NORMALnice 0 task (weight=1024), aSCHED_IDLEtask is penalized by a hugese->vruntimevalue (1024/3 higher than for aSCHED_NORMALnice 0 task). This should make sure it doesn't tick preempt aSCHED_NORMALnice 0 task.Could you please explain how the huge penalization of vruntime(1024/3) could make sure
SCHED_IDLEnot tick preemptingSCHED_NORMALnice 0 task?Thanks a lot.
Trace a run of 2 SCHED_OTHER (nice 0) tasks and 1 SCHED_IDLE task on a single CPU and trace_printk the conditions 'if (delta < 0)' and ' if (delta > ideal_runtime)' in check_preempt_tick().
Then do the same with 3 SCHED_OTHER (nice 0) tasks. You can also change the niceness of the 2 SCHED_OTHER task to 19 to see some differences in the kernelshark's task layout.
rt-app (https://github.com/scheduler-tools/rt-app) is a nice tool to craft those artificial use cases.
[...]
Re: [PATCH] sched/fair: reduce preemption with IDLE tasks runable(Internet mail)
Hi,
Quoted 35 lines · Dietmar Eggemann
Quoted 16 lines · benbjiang(蒋彪)
Hi,
Quoted 14 lines · Dietmar Eggemann
Quoted 11 lines · benbjiang(蒋彪)
Hi,
Quoted 9 lines · Dietmar Eggemann
Quoted 6 lines · benbjiang(蒋彪)
Quoted 6 lines · Dietmar Eggemann
Quoted 1 line · Jiang Biao
From: Jiang Biao <[email protected]>
[...]
Quoted 8 lines
Quoted 4 lines
Because of this very small weight (weight=3), compared to a
SCHED_NORMALnice 0 task (weight=1024), aSCHED_IDLEtask is penalized by a hugese->vruntimevalue (1024/3 higher than for aSCHED_NORMALnice 0 task). This should make sure it doesn't tick preempt aSCHED_NORMALnice 0 task.Could you please explain how the huge penalization of vruntime(1024/3) could make sure
SCHED_IDLEnot tick preemptingSCHED_NORMALnice 0 task?Thanks a lot.
Trace a run of 2
SCHED_OTHER(nice 0) tasks and 1SCHED_IDLEtask on a single CPU andtrace_printkthe conditions 'if (delta < 0)' and ' if (delta >ideal_runtime)' incheck_preempt_tick().
Quoted 6 lines
Then do the same with 3
SCHED_OTHER(nice 0) tasks. You can also change the niceness of the 2SCHED_OTHERtask to 19 to see some differences in the kernelshark's task layout.rt-app (https://github.com/scheduler-tools/rt-app) is a nice tool to craft those artificial use cases.
With rt-app tool, sched_switch traced by ftrace, the result is as what I expected,
** 1normal+1idle: idle preempt normal every 200ms **
<...>-92016 [002] d... 2398066.902477: sched_switch: prev_comm=normal0-0 prev_pid=92016 prev_prio=120 prev_state=S ==> next_comm=idle0-0 next_pid=91814 next_prio=120
<...>-91814 [002] d... 2398066.902527: sched_switch: prev_comm=idle0-0 prev_pid=91814 prev_prio=120 prev_state=R ==> next_comm=normal0-0 next_pid=92016 next_prio=120
<...>-92016 [002] d... 2398066.922472: sched_switch: prev_comm=normal0-0 prev_pid=92016 prev_prio=120 prev_state=S ==> next_comm=idle0-0 next_pid=91814 next_prio=120
<...>-91814 [002] d... 2398066.922522: sched_switch: prev_comm=idle0-0 prev_pid=91814 prev_prio=120 prev_state=R ==> next_comm=normal0-0 next_pid=92016 next_prio=120
<...>-92016 [002] d... 2398066.942292: sched_switch: prev_comm=normal0-0 prev_pid=92016 prev_prio=120 prev_state=S ==> next_comm=idle0-0 next_pid=91814 next_prio=120
<...>-91814 [002] d... 2398066.942343: sched_switch: prev_comm=idle0-0 prev_pid=91814 prev_prio=120 prev_state=R ==> next_comm=normal0-0 next_pid=92016 next_prio=120
<...>-92016 [002] d... 2398066.962331: sched_switch: prev_comm=normal0-0 prev_pid=92016 prev_prio=120 prev_state=S ==> next_comm=idle0-0 next_pid=91814 next_prio=120** 2normal+1idle: idle preempt normal every 600+ms **
<...>-49009 [002] d... 2400562.746640: sched_switch: prev_comm=normal0-0 prev_pid=49009 prev_prio=120 prev_state=R ==> next_comm=idle0-0 next_pid=187466 next_prio=120
<...>-187466 [002] d... 2400562.747502: sched_switch: prev_comm=idle0-0 prev_pid=187466 prev_prio=120 prev_state=S ==> next_comm=normal1-0 next_pid=198658 next_prio=120
<...>-198658 [002] d... 2400563.335262: sched_switch: prev_comm=normal1-0 prev_pid=198658 prev_prio=120 prev_state=R ==> next_comm=idle0-0 next_pid=187466 next_prio=120
<...>-187466 [002] d... 2400563.336258: sched_switch: prev_comm=idle0-0 prev_pid=187466 prev_prio=120 prev_state=R ==> next_comm=normal0-0 next_pid=49009 next_prio=120
<...>-198658 [002] d... 2400564.017663: sched_switch: prev_comm=normal1-0 prev_pid=198658 prev_prio=120 prev_state=R ==> next_comm=idle0-0 next_pid=187466 next_prio=120
<...>-187466 [002] d... 2400564.018661: sched_switch: prev_comm=idle0-0 prev_pid=187466 prev_prio=120 prev_state=R ==> next_comm=normal0-0 next_pid=49009 next_prio=120
<...>-198658 [002] d... 2400564.701063: sched_switch: prev_comm=normal1-0 prev_pid=198658 prev_prio=120 prev_state=R ==> next_comm=idle0-0 next_pid=187466 next_prio=120** 3normal+idle: idle preempt normal every 1000+ms **
<...>-198658 [002] d... 2400415.780701: sched_switch: prev_comm=normal1-0 prev_pid=198658 prev_prio=120 prev_state=R ==> next_comm=idle0-0 next_pid=187466 next_prio=120
<...>-187466 [002] d... 2400415.781699: sched_switch: prev_comm=idle0-0 prev_pid=187466 prev_prio=120 prev_state=R ==> next_comm=normal2-0 next_pid=46478 next_prio=120
<...>-49009 [002] d... 2400416.806298: sched_switch: prev_comm=normal0-0 prev_pid=49009 prev_prio=120 prev_state=R ==> next_comm=idle0-0 next_pid=187466 next_prio=120
<...>-187466 [002] d... 2400416.807297: sched_switch: prev_comm=idle0-0 prev_pid=187466 prev_prio=120 prev_state=R ==> next_comm=normal2-0 next_pid=46478 next_prio=120
<...>-198658 [002] d... 2400417.826910: sched_switch: prev_comm=normal1-0 prev_pid=198658 prev_prio=120 prev_state=R ==> next_comm=idle0-0 next_pid=187466 next_prio=120
<...>-187466 [002] d... 2400417.827911: sched_switch: prev_comm=idle0-0 prev_pid=187466 prev_prio=120 prev_state=R ==> next_comm=normal2-0 next_pid=46478 next_prio=120
<...>-49009 [002] d... 2400418.857497: sched_switch: prev_comm=normal0-0 prev_pid=49009 prev_prio=120 prev_state=R ==> next_comm=idle0-0 next_pid=187466 next_prio=120** 2normal(nice 19)+1idle(nice 0): idle preempt normal every 30+ms **
<...>-187466 [002] d... 2401740.134249: sched_switch: prev_comm=idle0-0 prev_pid=187466 prev_prio=120 prev_state=R ==> next_comm=normal0-0 next_pid=49009 next_prio=139
<...>-198658 [002] d... 2401740.162182: sched_switch: prev_comm=normal1-0 prev_pid=198658 prev_prio=139 prev_state=R ==> next_comm=idle0-0 next_pid=187466 next_prio=120
<...>-187466 [002] d... 2401740.165177: sched_switch: prev_comm=idle0-0 prev_pid=187466 prev_prio=120 prev_state=R ==> next_comm=normal0-0 next_pid=49009 next_prio=139
<...>-49009 [002] d... 2401740.193110: sched_switch: prev_comm=normal0-0 prev_pid=49009 prev_prio=139 prev_state=R ==> next_comm=idle0-0 next_pid=187466 next_prio=120
<...>-187466 [002] d... 2401740.196104: sched_switch: prev_comm=idle0-0 prev_pid=187466 prev_prio=120 prev_state=R ==> next_comm=normal1-0 next_pid=198658 next_prio=139
<...>-198658 [002] d... 2401740.228029: sched_switch: prev_comm=normal1-0 prev_pid=198658 prev_prio=139 prev_state=R ==> next_comm=idle0-0 next_pid=187466 next_prio=120
<...>-187466 [002] d... 2401740.231022: sched_switch: prev_comm=idle0-0 prev_pid=187466 prev_prio=120 prev_state=R ==> next_comm=normal0-0 next_pid=49009 next_prio=139
<...>-198658 [002] d... 2401740.262946: sched_switch: prev_comm=normal1-0 prev_pid=198658 prev_prio=139 prev_state=R ==> next_comm=idle0-0 next_pid=187466 next_prio=120SCHED_IDLE tasks do tick preempt rarely, but can not be avoided with a weight.
I wonder if the result is what you expected? :)
Thanks a lot.
Regards,
Jiang
Quoted 2 lines
[...]
Dietmar Eggemann
reviewerRe: [PATCH] sched/fair: reduce preemption with IDLE tasks runable(Internet mail)
Quoted 21 lines · benbjiang(蒋彪)
Hi,
Quoted 19 lines · Dietmar Eggemann
Quoted 16 lines · benbjiang(蒋彪)
Hi,
Quoted 14 lines · Dietmar Eggemann
Quoted 11 lines · benbjiang(蒋彪)
Hi,
Quoted 9 lines · Dietmar Eggemann
Quoted 6 lines · benbjiang(蒋彪)
Quoted 6 lines · Dietmar Eggemann
Quoted 1 line · Jiang Biao
From: Jiang Biao <[email protected]>
[...]
Quoted 11 lines
Quoted 10 lines
Trace a run of 2
SCHED_OTHER(nice 0) tasks and 1SCHED_IDLEtask on a single CPU andtrace_printkthe conditions 'if (delta < 0)' and ' if (delta >ideal_runtime)' incheck_preempt_tick().Then do the same with 3
SCHED_OTHER(nice 0) tasks. You can also change the niceness of the 2SCHED_OTHERtask to 19 to see some differences in the kernelshark's task layout.rt-app (https://github.com/scheduler-tools/rt-app) is a nice tool to craft those artificial use cases.
With rt-app tool,
sched_switchtraced by ftrace, the result is as what I expected,
I use:
{
"tasks" : {
"task_other" : {
"instance" : 2,
"loop" : 200,
"policy" : "SCHED_OTHER",
"run" : 8000,
"timer" : { "ref" : "unique1" , "period" : 16000, "mode" : "absolute" },
"priority" : 0
},
"task_idle" : {
"instance" : 1,
"loop" : 200,
"policy" : "SCHED_IDLE",
"run" : 8000,
"timer" : { "ref" : "unique2" , "period" : 16000, "mode" : "absolute" }
}
},
"global" : {
"calibration" : 243, <-- Has to be calibrated against the CPU you run on !!!
"default_policy" : "SCHED_OTHER",
"duration" : -1
}
}to have 2 (periodic) SCHED_OTHER and 1 SCHED_IDLE task.
Quoted 1 line
** 2normal+1idle: idle preempt normal every 600+ms **
During the 3.2s the 2 SCHED_OTHER tasks run, the SCHED_IDLE task is switched in only once, after ~2.5s.
Quoted 1 line
** 3normal+idle: idle preempt normal every 1000+ms **
Ah, this was meant to be 3 SCHED_OTHER tasks only! To see the difference in behavior.
Quoted 1 line
** 2normal(nice 19)+1idle(nice 0): idle preempt normal every 30+ms **
During the 3.2s the 2 SCHED_OTHER tasks run, the SCHED_IDLE task is switched in every ~45ms.
[...]
Re: [PATCH] sched/fair: reduce preemption with IDLE tasks runable(Internet mail)
Hi,
Quoted 66 lines · Dietmar Eggemann
Quoted 21 lines · benbjiang(蒋彪)
Hi,
Quoted 19 lines · Dietmar Eggemann
Quoted 16 lines · benbjiang(蒋彪)
Hi,
Quoted 14 lines · Dietmar Eggemann
Quoted 11 lines · benbjiang(蒋彪)
Hi,
Quoted 9 lines · Dietmar Eggemann
Quoted 6 lines · benbjiang(蒋彪)
Quoted 6 lines · Dietmar Eggemann
Quoted 1 line · Jiang Biao
From: Jiang Biao <[email protected]>
[...]
Quoted 11 lines
Quoted 10 lines
Trace a run of 2
SCHED_OTHER(nice 0) tasks and 1SCHED_IDLEtask on a single CPU andtrace_printkthe conditions 'if (delta < 0)' and ' if (delta >ideal_runtime)' incheck_preempt_tick().Then do the same with 3
SCHED_OTHER(nice 0) tasks. You can also change the niceness of the 2SCHED_OTHERtask to 19 to see some differences in the kernelshark's task layout.rt-app (https://github.com/scheduler-tools/rt-app) is a nice tool to craft those artificial use cases.
With rt-app tool,
sched_switchtraced by ftrace, the result is as what I expected,I use:
{ "tasks" : { "task_other" : { "instance" : 2, "loop" : 200, "policy" : "SCHED_OTHER", "run" : 8000, "timer" : { "ref" : "unique1" , "period" : 16000, "mode" : "absolute" }, "priority" : 0 }, "task_idle" : { "instance" : 1, "loop" : 200, "policy" : "SCHED_IDLE", "run" : 8000, "timer" : { "ref" : "unique2" , "period" : 16000, "mode" : "absolute" } } }, "global" : { "calibration" : 243, <-- Has to be calibrated against the CPU you run on !!! "default_policy" : "SCHED_OTHER", "duration" : -1 } }
Quoted 6 lines
to have 2 (periodic)
SCHED_OTHERand 1SCHED_IDLEtask.Quoted 1 line
** 2normal+1idle: idle preempt normal every 600+ms **
During the 3.2s the 2
SCHED_OTHERtasks run, theSCHED_IDLEtask is switched in only once, after ~2.5s.
Use your config with loop increased from 200 to 2000, to observe longer,
<...>-37620 [002] d... 47950.446191: sched_switch: prev_comm=task_other-1 prev_pid=37620 prev_prio=120 prev_state=S ==> next_comm=task_idle-2 next_pid=37621 next_prio=120
<...>-37619 [002] d... 47955.687709: sched_switch: prev_comm=task_other-0 prev_pid=37619 prev_prio=120 prev_state=R ==> next_comm=task_idle-2 next_pid=37621 next_prio=120
// The first preemption interval is 5.2s.
<...>-37620 [002] d... 47956.375716: sched_switch: prev_comm=task_other-1 prev_pid=37620 prev_prio=120 prev_state=R ==> next_comm=task_idle-2 next_pid=37621 next_prio=120
<...>-37619 [002] d... 47957.060722: sched_switch: prev_comm=task_other-0 prev_pid=37619 prev_prio=120 prev_state=R ==> next_comm=task_idle-2 next_pid=37621 next_prio=120
<...>-37620 [002] d... 47957.747728: sched_switch: prev_comm=task_other-1 prev_pid=37620 prev_prio=120 prev_state=R ==> next_comm=task_idle-2 next_pid=37621 next_prio=120
<...>-37620 [002] d... 47958.423734: sched_switch: prev_comm=task_other-1 prev_pid=37620 prev_prio=120 prev_state=R ==> next_comm=task_idle-2 next_pid=37621 next_prio=120
<...>-37620 [002] d... 47959.119740: sched_switch: prev_comm=task_other-1 prev_pid=37620 prev_prio=120 prev_state=R ==> next_comm=task_idle-2 next_pid=37621 next_prio=120// After the first preemption, the rest preemption intervals are all about 600ms+. :)
Quoted 5 lines
Quoted 2 lines
** 3normal+idle: idle preempt normal every 1000+ms **
Ah, this was meant to be 3
SCHED_OTHERtasks only! To see the difference in behavior.
With 3 SCHED_OHTER tasks only, the SCHED_OHTER task is switched in Every 27ms.
Quoted 5 lines
Quoted 2 lines
** 2normal(nice 19)+1idle(nice 0): idle preempt normal every 30+ms **
During the 3.2s the 2
SCHED_OTHERtasks run, theSCHED_IDLEtask is switched in every ~45ms.
That’s as what I expected. :)
Thx.
Regards,
Jiang
Quoted 2 lines
[...]
Dietmar Eggemann
reviewerRe: [PATCH] sched/fair: reduce preemption with IDLE tasks runable(Internet mail)
Quoted 26 lines · benbjiang(蒋彪)
Hi,
Quoted 24 lines · Dietmar Eggemann
Quoted 21 lines · benbjiang(蒋彪)
Hi,
Quoted 19 lines · Dietmar Eggemann
Quoted 16 lines · benbjiang(蒋彪)
Hi,
Quoted 14 lines · Dietmar Eggemann
Quoted 11 lines · benbjiang(蒋彪)
Hi,
Quoted 9 lines · Dietmar Eggemann
Quoted 6 lines · benbjiang(蒋彪)
Quoted 6 lines · Dietmar Eggemann
Quoted 1 line · Jiang Biao
From: Jiang Biao <[email protected]>
[...]
Quoted 14 lines
Quoted 4 lines
Quoted 1 line
** 2normal+1idle: idle preempt normal every 600+ms **
During the 3.2s the 2
SCHED_OTHERtasks run, theSCHED_IDLEtask is switched in only once, after ~2.5s.Use your config with loop increased from 200 to 2000, to observe longer,
<...>-37620 [002] d... 47950.446191: sched_switch: prev_comm=task_other-1 prev_pid=37620 prev_prio=120 prev_state=S ==> next_comm=task_idle-2 next_pid=37621 next_prio=120 <...>-37619 [002] d... 47955.687709: sched_switch: prev_comm=task_other-0 prev_pid=37619 prev_prio=120 prev_state=R ==> next_comm=task_idle-2 next_pid=37621 next_prio=120 // The first preemption interval is 5.2s. <...>-37620 [002] d... 47956.375716: sched_switch: prev_comm=task_other-1 prev_pid=37620 prev_prio=120 prev_state=R ==> next_comm=task_idle-2 next_pid=37621 next_prio=120 <...>-37619 [002] d... 47957.060722: sched_switch: prev_comm=task_other-0 prev_pid=37619 prev_prio=120 prev_state=R ==> next_comm=task_idle-2 next_pid=37621 next_prio=120 <...>-37620 [002] d... 47957.747728: sched_switch: prev_comm=task_other-1 prev_pid=37620 prev_prio=120 prev_state=R ==> next_comm=task_idle-2 next_pid=37621 next_prio=120 <...>-37620 [002] d... 47958.423734: sched_switch: prev_comm=task_other-1 prev_pid=37620 prev_prio=120 prev_state=R ==> next_comm=task_idle-2 next_pid=37621 next_prio=120 <...>-37620 [002] d... 47959.119740: sched_switch: prev_comm=task_other-1 prev_pid=37620 prev_prio=120 prev_state=R ==> next_comm=task_idle-2 next_pid=37621 next_prio=120
Quoted 1 line
// After the first preemption, the rest preemption intervals are all about 600ms+. :)
Are you sure about this?
The math is telling me for the:
idle task: (3 / (1024 + 1024 + 3))^(-1) * 4ms = 2735ms
normal task: (1024 / (1024 + 1024 + 3))^(-1) * 4ms = 8ms
(4ms - 250 Hz)
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
** 3normal+idle: idle preempt normal every 1000+ms **
Ah, this was meant to be 3
SCHED_OTHERtasks only! To see the difference in behavior.With 3
SCHED_OHTERtasks only, theSCHED_OHTERtask is switched in Every 27ms.
normal task: (1024 / (1024 + 1024 + 1024))^(-1) * 4ms = 12ms
Quoted 5 lines
Quoted 4 lines
Quoted 1 line
** 2normal(nice 19)+1idle(nice 0): idle preempt normal every 30+ms **
During the 3.2s the 2
SCHED_OTHERtasks run, theSCHED_IDLEtask is switched in every ~45ms.That’s as what I expected. :)
idle task: (3 / (15 + 15 + 3))^(-1) * 4ms = 44ms
normal task: (15 / (15 + 15 + 3))^(-1) * 4ms = 9ms
Re: [PATCH] sched/fair: reduce preemption with IDLE tasks runable(Internet mail)
Quoted 50 lines · Dietmar Eggemann
Quoted 26 lines · benbjiang(蒋彪)
Hi,
Quoted 24 lines · Dietmar Eggemann
Quoted 21 lines · benbjiang(蒋彪)
Hi,
Quoted 19 lines · Dietmar Eggemann
Quoted 16 lines · benbjiang(蒋彪)
Hi,
Quoted 14 lines · Dietmar Eggemann
Quoted 11 lines · benbjiang(蒋彪)
Hi,
Quoted 9 lines · Dietmar Eggemann
Quoted 6 lines · benbjiang(蒋彪)
Quoted 6 lines · Dietmar Eggemann
Quoted 1 line · Jiang Biao
From: Jiang Biao <[email protected]>
[...]
Quoted 14 lines
Quoted 4 lines
Quoted 1 line
** 2normal+1idle: idle preempt normal every 600+ms **
During the 3.2s the 2
SCHED_OTHERtasks run, theSCHED_IDLEtask is switched in only once, after ~2.5s.Use your config with loop increased from 200 to 2000, to observe longer,
<...>-37620 [002] d... 47950.446191: sched_switch: prev_comm=task_other-1 prev_pid=37620 prev_prio=120 prev_state=S ==> next_comm=task_idle-2 next_pid=37621 next_prio=120 <...>-37619 [002] d... 47955.687709: sched_switch: prev_comm=task_other-0 prev_pid=37619 prev_prio=120 prev_state=R ==> next_comm=task_idle-2 next_pid=37621 next_prio=120 // The first preemption interval is 5.2s. <...>-37620 [002] d... 47956.375716: sched_switch: prev_comm=task_other-1 prev_pid=37620 prev_prio=120 prev_state=R ==> next_comm=task_idle-2 next_pid=37621 next_prio=120 <...>-37619 [002] d... 47957.060722: sched_switch: prev_comm=task_other-0 prev_pid=37619 prev_prio=120 prev_state=R ==> next_comm=task_idle-2 next_pid=37621 next_prio=120 <...>-37620 [002] d... 47957.747728: sched_switch: prev_comm=task_other-1 prev_pid=37620 prev_prio=120 prev_state=R ==> next_comm=task_idle-2 next_pid=37621 next_prio=120 <...>-37620 [002] d... 47958.423734: sched_switch: prev_comm=task_other-1 prev_pid=37620 prev_prio=120 prev_state=R ==> next_comm=task_idle-2 next_pid=37621 next_prio=120 <...>-37620 [002] d... 47959.119740: sched_switch: prev_comm=task_other-1 prev_pid=37620 prev_prio=120 prev_state=R ==> next_comm=task_idle-2 next_pid=37621 next_prio=120Quoted 1 line
// After the first preemption, the rest preemption intervals are all about 600ms+. :)
Quoted 1 line
Are you sure about this?
Yes. :)
Quoted 8 lines
The math is telling me for the:
idle task: (3 / (1024 + 1024 + 3))^(-1) * 4ms = 2735ms
normal task: (1024 / (1024 + 1024 + 3))^(-1) * 4ms = 8ms
(4ms - 250 Hz)
My tick is 1ms - 1000HZ, which seems reasonable for 600ms? :)
Thx.
Regards,
Jiang
Quoted 19 lines
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
** 3normal+idle: idle preempt normal every 1000+ms **
Ah, this was meant to be 3
SCHED_OTHERtasks only! To see the difference in behavior.With 3
SCHED_OHTERtasks only, theSCHED_OHTERtask is switched in Every 27ms.normal task: (1024 / (1024 + 1024 + 1024))^(-1) * 4ms = 12ms
Quoted 5 lines
Quoted 4 lines
Quoted 1 line
** 2normal(nice 19)+1idle(nice 0): idle preempt normal every 30+ms **
During the 3.2s the 2
SCHED_OTHERtasks run, theSCHED_IDLEtask is switched in every ~45ms.That’s as what I expected. :)
idle task: (3 / (15 + 15 + 3))^(-1) * 4ms = 44ms
normal task: (15 / (15 + 15 + 3))^(-1) * 4ms = 9ms
Dietmar Eggemann
reviewerRe: [PATCH] sched/fair: reduce preemption with IDLE tasks runable(Internet mail)
Quoted 31 lines · benbjiang(蒋彪)
Quoted 31 lines · Dietmar Eggemann
Quoted 26 lines · benbjiang(蒋彪)
Hi,
Quoted 24 lines · Dietmar Eggemann
Quoted 21 lines · benbjiang(蒋彪)
Hi,
Quoted 19 lines · Dietmar Eggemann
Quoted 16 lines · benbjiang(蒋彪)
Hi,
Quoted 14 lines · Dietmar Eggemann
Quoted 11 lines · benbjiang(蒋彪)
Hi,
Quoted 9 lines · Dietmar Eggemann
Quoted 6 lines · benbjiang(蒋彪)
Quoted 6 lines · Dietmar Eggemann
Quoted 1 line · Jiang Biao
From: Jiang Biao <[email protected]>
[...]
Quoted 11 lines
Quoted 1 line
Are you sure about this?
Yes. :)
Quoted 8 lines
The math is telling me for the:
idle task: (3 / (1024 + 1024 + 3))^(-1) * 4ms = 2735ms
normal task: (1024 / (1024 + 1024 + 3))^(-1) * 4ms = 8ms
(4ms - 250 Hz)
My tick is 1ms - 1000HZ, which seems reasonable for 600ms? :)
OK, I see.
But here the different sched slices (check_preempt_tick()-> sched_slice()) between normal tasks and the idle task play a role to.
Normal tasks get ~3ms whereas the idle task gets <0.01ms.
So the idle task runs every ~680ms but only for 1 tick (1ms) (4 times less than the normal tasks). The condition 'if (delta_exec > ideal_runtime)' in check_preempt_tick() is only true at the 4th tick when a normal task runs even though the slice is 3ms.
In the 250 Hz example the sched slice diffs are hidden behind the 4ms tick.
[...]
Vincent Guittot
maintainerRe: [PATCH] sched/fair: reduce preemption with IDLE tasks runable(Internet mail)
Quoted 46 lines · Dietmar Eggemann
Quoted 31 lines · benbjiang(蒋彪)
Quoted 31 lines · Dietmar Eggemann
Quoted 26 lines · benbjiang(蒋彪)
Hi,
Quoted 24 lines · Dietmar Eggemann
Quoted 21 lines · benbjiang(蒋彪)
Hi,
Quoted 19 lines · Dietmar Eggemann
Quoted 16 lines · benbjiang(蒋彪)
Hi,
Quoted 14 lines · Dietmar Eggemann
Quoted 11 lines · benbjiang(蒋彪)
Hi,
Quoted 9 lines · Dietmar Eggemann
Quoted 6 lines · benbjiang(蒋彪)
Quoted 6 lines · Dietmar Eggemann
On 01/08/2020 04:32, Jiang Biao wrote:
From: Jiang Biao <[email protected]>[...]
Quoted 10 lines
Quoted 1 line
Are you sure about this?
Yes. :)
Quoted 8 lines
The math is telling me for the:
idle task: (3 / (1024 + 1024 + 3))^(-1) * 4ms = 2735ms
normal task: (1024 / (1024 + 1024 + 3))^(-1) * 4ms = 8ms
(4ms - 250 Hz)
Quoted 8 lines
Quoted 1 line
My tick is 1ms - 1000HZ, which seems reasonable for 600ms? :)
OK, I see.
But here the different sched slices (check_preempt_tick()->
sched_slice()) between normal tasks and the idle task play a role to.Normal tasks get ~3ms whereas the idle task gets <0.01ms.
In fact that depends on the number of CPUs on the system :sysctl_sched_latency = 6ms * (1 + ilog(ncpus)) . On a 8 cores system, normal task will run around 12ms in one shoot and the idle task still one tick period
Also, you can increase even more the period between 2 runs of idle task by using cgroups and min shares value : 2
Quoted 9 lines
So the idle task runs every ~680ms but only for 1 tick (1ms) (4 times less than the normal tasks). The condition 'if (
delta_exec>ideal_runtime)' incheck_preempt_tick()is only true at the 4th tick when a normal task runs even though the slice is 3ms.In the 250 Hz example the sched slice diffs are hidden behind the 4ms tick.
[...]
Dietmar Eggemann
reviewerRe: [PATCH] sched/fair: reduce preemption with IDLE tasks runable(Internet mail)
Quoted 54 lines · Vincent Guittot
Quoted 53 lines · Dietmar Eggemann
Quoted 31 lines · benbjiang(蒋彪)
Quoted 31 lines · Dietmar Eggemann
Quoted 26 lines · benbjiang(蒋彪)
Hi,
Quoted 24 lines · Dietmar Eggemann
Quoted 21 lines · benbjiang(蒋彪)
Hi,
Quoted 19 lines · Dietmar Eggemann
Quoted 16 lines · benbjiang(蒋彪)
Hi,
Quoted 14 lines · Dietmar Eggemann
Quoted 11 lines · benbjiang(蒋彪)
Hi,
Quoted 9 lines · Dietmar Eggemann
Quoted 6 lines · benbjiang(蒋彪)
On Aug 3, 2020, at 4:16 PM, Dietmar Eggemann <[email protected]> wrote:
On 01/08/2020 04:32, Jiang Biao wrote:
From: Jiang Biao <[email protected]>[...]
Quoted 11 lines
Quoted 1 line
Are you sure about this?
Yes. :)
Quoted 8 lines
The math is telling me for the:
idle task: (3 / (1024 + 1024 + 3))^(-1) * 4ms = 2735ms
normal task: (1024 / (1024 + 1024 + 3))^(-1) * 4ms = 8ms
(4ms - 250 Hz)
My tick is 1ms - 1000HZ, which seems reasonable for 600ms? :)
OK, I see.
But here the different sched slices (check_preempt_tick()->
sched_slice()) between normal tasks and the idle task play a role to.
Quoted 6 lines
Quoted 1 line
Normal tasks get ~3ms whereas the idle task gets <0.01ms.
In fact that depends on the number of CPUs on the system :sysctl_sched_latency = 6ms * (1 + ilog(ncpus)) . On a 8 cores system, normal task will run around 12ms in one shoot and the idle task still one tick period
True. This is on a single CPU.
Quoted 2 lines
Also, you can increase even more the period between 2 runs of idle task by using cgroups and min shares value : 2
Ah yes, maybe this is what Jiang wants to do then? If his runtime does not have other requirements preventing this.
[...]
Re: [PATCH] sched/fair: reduce preemption with IDLE tasks runable(Internet mail)
Quoted 59 lines · Dietmar Eggemann
Quoted 54 lines · Vincent Guittot
Quoted 53 lines · Dietmar Eggemann
Quoted 31 lines · benbjiang(蒋彪)
Quoted 31 lines · Dietmar Eggemann
Quoted 26 lines · benbjiang(蒋彪)
Hi,
Quoted 24 lines · Dietmar Eggemann
Quoted 21 lines · benbjiang(蒋彪)
Hi,
Quoted 19 lines · Dietmar Eggemann
Quoted 16 lines · benbjiang(蒋彪)
Hi,
Quoted 14 lines · Dietmar Eggemann
Quoted 11 lines · benbjiang(蒋彪)
Hi,
Quoted 9 lines · Dietmar Eggemann
On 03/08/2020 13:26, benbjiang(蒋彪) wrote:
On Aug 3, 2020, at 4:16 PM, Dietmar Eggemann <[email protected]> wrote:
On 01/08/2020 04:32, Jiang Biao wrote:
From: Jiang Biao <[email protected]>[...]
Quoted 11 lines
Quoted 1 line
Are you sure about this?
Yes. :)
Quoted 8 lines
The math is telling me for the:
idle task: (3 / (1024 + 1024 + 3))^(-1) * 4ms = 2735ms
normal task: (1024 / (1024 + 1024 + 3))^(-1) * 4ms = 8ms
(4ms - 250 Hz)
My tick is 1ms - 1000HZ, which seems reasonable for 600ms? :)
OK, I see.
But here the different sched slices (check_preempt_tick()->
sched_slice()) between normal tasks and the idle task play a role to.
Quoted 8 lines
Quoted 6 lines
Quoted 1 line
Normal tasks get ~3ms whereas the idle task gets <0.01ms.
In fact that depends on the number of CPUs on the system :sysctl_sched_latency = 6ms * (1 + ilog(ncpus)) . On a 8 cores system, normal task will run around 12ms in one shoot and the idle task still one tick period
True. This is on a single CPU.
Agree. :)
Quoted 6 lines
Quoted 3 lines
Also, you can increase even more the period between 2 runs of idle task by using cgroups and min shares value : 2
Ah yes, maybe this is what Jiang wants to do then? If his runtime does not have other requirements preventing this.
That could work for increasing the period between 2 runs. But could not reduce the single runtime of idle task I guess, which means normal task could have 1-tick schedule latency because of idle task. OTOH, cgroups(shares) could introduce extra complexity. :)
I wonder if there’s any possibility to make SCHED_IDLEs’ priorities absolutely lower than SCHED_NORMAL(OTHER), which means no weights/shares
for them, and they run only when no other task’s runnable.
I guess there may be priority inversion issue if we do that. But maybe we could avoid it by load-balance more aggressively, or it(priority inversion) could be ignored in some special case.
Thx.
Regard,
Jiang
Quoted 2 lines
[...]
Vincent Guittot
maintainerRe: [PATCH] sched/fair: reduce preemption with IDLE tasks runable(Internet mail)
Quoted 71 lines · benbjiang(蒋彪)
Quoted 60 lines · Dietmar Eggemann
Quoted 54 lines · Vincent Guittot
Quoted 53 lines · Dietmar Eggemann
Quoted 31 lines · benbjiang(蒋彪)
Quoted 31 lines · Dietmar Eggemann
Quoted 26 lines · benbjiang(蒋彪)
Hi,
Quoted 24 lines · Dietmar Eggemann
Quoted 21 lines · benbjiang(蒋彪)
Hi,
Quoted 19 lines · Dietmar Eggemann
Quoted 16 lines · benbjiang(蒋彪)
Hi,
Quoted 14 lines · Dietmar Eggemann
Quoted 11 lines · benbjiang(蒋彪)
Hi,
On Aug 6, 2020, at 9:29 PM, Dietmar Eggemann <[email protected]> wrote:
On 03/08/2020 13:26, benbjiang(蒋彪) wrote:
On Aug 3, 2020, at 4:16 PM, Dietmar Eggemann <[email protected]> wrote:
On 01/08/2020 04:32, Jiang Biao wrote:
From: Jiang Biao <[email protected]>[...]
Quoted 11 lines
Quoted 1 line
Are you sure about this?
Yes. :)
Quoted 8 lines
The math is telling me for the:
idle task: (3 / (1024 + 1024 + 3))^(-1) * 4ms = 2735ms
normal task: (1024 / (1024 + 1024 + 3))^(-1) * 4ms = 8ms
(4ms - 250 Hz)
My tick is 1ms - 1000HZ, which seems reasonable for 600ms? :)
OK, I see.
But here the different sched slices (check_preempt_tick()->
sched_slice()) between normal tasks and the idle task play a role to.Quoted 8 lines
Quoted 6 lines
Quoted 1 line
Normal tasks get ~3ms whereas the idle task gets <0.01ms.
In fact that depends on the number of CPUs on the system :sysctl_sched_latency = 6ms * (1 + ilog(ncpus)) . On a 8 cores system, normal task will run around 12ms in one shoot and the idle task still one tick period
True. This is on a single CPU.
Agree. :)
Quoted 1 line
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Also, you can increase even more the period between 2 runs of idle task by using cgroups and min shares value : 2
Ah yes, maybe this is what Jiang wants to do then? If his runtime does not have other requirements preventing this.
That could work for increasing the period between 2 runs. But could not reduce the single runtime of idle task I guess, which means normal task could have 1-tick schedule latency because of idle task.
Yes. An idle task will preempt an always running task during 1 tick every 680ms. But also you should keep in mind that a waking normal task will preempt the idle task immediately which means that it will not add scheduling latency to a normal task but "steal" 0.14% of normal task throughput (1/680) at most
Quoted 6 lines
OTOH, cgroups(shares) could introduce extra complexity. :)
I wonder if there’s any possibility to make SCHED_IDLEs’ priorities absolutely lower than
SCHED_NORMAL(OTHER), which means no weights/shares
for them, and they run only when no other task’s runnable.
I guess there may be priority inversion issue if we do that. But maybe we
Exactly, that's why we must ensure a minimum running time for sched_idle task
Quoted 10 lines
could avoid it by load-balance more aggressively, or it(priority inversion) could be ignored in some special case.
Thx.
Regard,
JiangQuoted 2 lines
[...]
Re: [PATCH] sched/fair: reduce preemption with IDLE tasks runable(Internet mail)
Quoted 85 lines · Vincent Guittot
Quoted 71 lines · benbjiang(蒋彪)
Quoted 60 lines · Dietmar Eggemann
Quoted 54 lines · Vincent Guittot
Quoted 53 lines · Dietmar Eggemann
Quoted 31 lines · benbjiang(蒋彪)
Quoted 31 lines · Dietmar Eggemann
Quoted 26 lines · benbjiang(蒋彪)
Hi,
Quoted 24 lines · Dietmar Eggemann
Quoted 21 lines · benbjiang(蒋彪)
Hi,
Quoted 19 lines · Dietmar Eggemann
Quoted 16 lines · benbjiang(蒋彪)
Hi,
Quoted 14 lines · Dietmar Eggemann
On 06/08/2020 17:52, benbjiang(蒋彪) wrote:
Hi,On Aug 6, 2020, at 9:29 PM, Dietmar Eggemann <[email protected]> wrote:
On 03/08/2020 13:26, benbjiang(蒋彪) wrote:
On Aug 3, 2020, at 4:16 PM, Dietmar Eggemann <[email protected]> wrote:
On 01/08/2020 04:32, Jiang Biao wrote:
From: Jiang Biao <[email protected]>[...]
Quoted 11 lines
Quoted 1 line
Are you sure about this?
Yes. :)
Quoted 8 lines
The math is telling me for the:
idle task: (3 / (1024 + 1024 + 3))^(-1) * 4ms = 2735ms
normal task: (1024 / (1024 + 1024 + 3))^(-1) * 4ms = 8ms
(4ms - 250 Hz)
My tick is 1ms - 1000HZ, which seems reasonable for 600ms? :)
OK, I see.
But here the different sched slices (check_preempt_tick()->
sched_slice()) between normal tasks and the idle task play a role to.Quoted 8 lines
Quoted 6 lines
Quoted 1 line
Normal tasks get ~3ms whereas the idle task gets <0.01ms.
In fact that depends on the number of CPUs on the system :sysctl_sched_latency = 6ms * (1 + ilog(ncpus)) . On a 8 cores system, normal task will run around 12ms in one shoot and the idle task still one tick period
True. This is on a single CPU.
Agree. :)
Quoted 1 line
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Also, you can increase even more the period between 2 runs of idle task by using cgroups and min shares value : 2
Ah yes, maybe this is what Jiang wants to do then? If his runtime does not have other requirements preventing this.
That could work for increasing the period between 2 runs. But could not reduce the single runtime of idle task I guess, which means normal task could have 1-tick schedule latency because of idle task.
Quoted 5 lines
Yes. An idle task will preempt an always running task during 1 tick every 680ms. But also you should keep in mind that a waking normal task will preempt the idle task immediately which means that it will not add scheduling latency to a normal task but "steal" 0.14% of normal task throughput (1/680) at most
That’s true. But in the VM case, when VM are busy(MWAIT passthrough or running cpu eating works), the 1-tick scheduling latency could be detected by cyclictest running in the VM.
OTOH, we compensate vruntime in place_entity() to boot waking without distinguish SCHED_IDLE task, do you think it’s necessary to do that? like
kernel/sched/fair.c +1 -1
@@ -4115,7 +4115,7 @@ place_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int initial) vruntime +=sched_vslice (cfs_rq, se);/* sleeps up to a single latency don't count. */ -if (!initial) { +if (!initial &&likely (!task_has_idle_policy (task_of (se)))) {unsigned long thresh = sysctl_sched_latency;
Quoted 9 lines
Quoted 7 lines
OTOH, cgroups(shares) could introduce extra complexity. :)
I wonder if there’s any possibility to make SCHED_IDLEs’ priorities absolutely lower than
SCHED_NORMAL(OTHER), which means no weights/shares
for them, and they run only when no other task’s runnable.
I guess there may be priority inversion issue if we do that. But maybe weExactly, that's why we must ensure a minimum running time for
sched_idletask
Still for VM case, different VMs have been much isolated from each other, priority inversion issue could be very rare, we’re trying to make offline tasks absoultly harmless to online tasks. :)
Thanks a lot for your time.
Regards,
Jiang
Quoted 5 lines
Quoted 5 lines
could avoid it by load-balance more aggressively, or it(priority inversion) could be ignored in some special case.
Quoted 1 line
Vincent Guittot
maintainerRe: [PATCH] sched/fair: reduce preemption with IDLE tasks runable(Internet mail)
Quoted 102 lines · benbjiang(蒋彪)
Quoted 86 lines · Vincent Guittot
Quoted 71 lines · benbjiang(蒋彪)
Quoted 60 lines · Dietmar Eggemann
Quoted 54 lines · Vincent Guittot
Quoted 53 lines · Dietmar Eggemann
Quoted 31 lines · benbjiang(蒋彪)
Quoted 31 lines · Dietmar Eggemann
Quoted 26 lines · benbjiang(蒋彪)
Hi,
Quoted 24 lines · Dietmar Eggemann
Quoted 21 lines · benbjiang(蒋彪)
Hi,
Quoted 19 lines · Dietmar Eggemann
Quoted 16 lines · benbjiang(蒋彪)
Hi,
On Aug 10, 2020, at 9:24 PM, Dietmar Eggemann <[email protected]> wrote:
On 06/08/2020 17:52, benbjiang(蒋彪) wrote:
Hi,On Aug 6, 2020, at 9:29 PM, Dietmar Eggemann <[email protected]> wrote:
On 03/08/2020 13:26, benbjiang(蒋彪) wrote:
On Aug 3, 2020, at 4:16 PM, Dietmar Eggemann <[email protected]> wrote:
On 01/08/2020 04:32, Jiang Biao wrote:
From: Jiang Biao <[email protected]>[...]
Quoted 11 lines
Quoted 1 line
Are you sure about this?
Yes. :)
Quoted 8 lines
The math is telling me for the:
idle task: (3 / (1024 + 1024 + 3))^(-1) * 4ms = 2735ms
normal task: (1024 / (1024 + 1024 + 3))^(-1) * 4ms = 8ms
(4ms - 250 Hz)
My tick is 1ms - 1000HZ, which seems reasonable for 600ms? :)
OK, I see.
But here the different sched slices (check_preempt_tick()->
sched_slice()) between normal tasks and the idle task play a role to.Quoted 8 lines
Quoted 6 lines
Quoted 1 line
Normal tasks get ~3ms whereas the idle task gets <0.01ms.
In fact that depends on the number of CPUs on the system :sysctl_sched_latency = 6ms * (1 + ilog(ncpus)) . On a 8 cores system, normal task will run around 12ms in one shoot and the idle task still one tick period
True. This is on a single CPU.
Agree. :)
Quoted 1 line
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Also, you can increase even more the period between 2 runs of idle task by using cgroups and min shares value : 2
Ah yes, maybe this is what Jiang wants to do then? If his runtime does not have other requirements preventing this.
That could work for increasing the period between 2 runs. But could not reduce the single runtime of idle task I guess, which means normal task could have 1-tick schedule latency because of idle task.
Quoted 5 lines
Yes. An idle task will preempt an always running task during 1 tick every 680ms. But also you should keep in mind that a waking normal task will preempt the idle task immediately which means that it will not add scheduling latency to a normal task but "steal" 0.14% of normal task throughput (1/680) at most
That’s true. But in the VM case, when VM are busy(MWAIT passthrough or running cpu eating works), the 1-tick scheduling latency could be detected by cyclictest running in the VM.
OTOH, we compensate vruntime in
place_entity()to boot waking without distinguishSCHED_IDLEtask, do you think it’s necessary to do that? like
kernel/sched/fair.c place_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int initial) · line 4119
Quoted 6 lines
kernel/sched/fair.c +1
-1@@ -4115,7 +4115,7 @@ place_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int initial) vruntime +=sched_vslice (cfs_rq, se);/* sleeps up to a single latency don't count. */ -if (!initial) { +if (!initial &&likely (!task_has_idle_policy (task_of (se)))) {unsigned long thresh = sysctl_sched_latency;
Yeah, this is a good improvement.
Does it solve your problem ?
Quoted 25 lines
Quoted 10 lines
Quoted 8 lines
OTOH, cgroups(shares) could introduce extra complexity. :)
I wonder if there’s any possibility to make SCHED_IDLEs’ priorities absolutely lower than
SCHED_NORMAL(OTHER), which means no weights/shares
for them, and they run only when no other task’s runnable.
I guess there may be priority inversion issue if we do that. But maybe weExactly, that's why we must ensure a minimum running time for
sched_idletaskStill for VM case, different VMs have been much isolated from each other, priority inversion issue could be very rare, we’re trying to make offline tasks absoultly harmless to online tasks. :)
Thanks a lot for your time.
Regards,
JiangQuoted 5 lines
Quoted 5 lines
could avoid it by load-balance more aggressively, or it(priority inversion) could be ignored in some special case.
Quoted 1 line
Re: [PATCH] sched/fair: reduce preemption with IDLE tasks runable(Internet mail)
Quoted 109 lines · Vincent Guittot
Quoted 104 lines · benbjiang(蒋彪)
Quoted 86 lines · Vincent Guittot
Quoted 71 lines · benbjiang(蒋彪)
Quoted 60 lines · Dietmar Eggemann
Quoted 54 lines · Vincent Guittot
Quoted 53 lines · Dietmar Eggemann
Quoted 31 lines · benbjiang(蒋彪)
Quoted 31 lines · Dietmar Eggemann
Quoted 26 lines · benbjiang(蒋彪)
Hi,
Quoted 24 lines · Dietmar Eggemann
Quoted 21 lines · benbjiang(蒋彪)
Hi,
Quoted 19 lines · Dietmar Eggemann
On 11/08/2020 02:41, benbjiang(蒋彪) wrote:
Hi,On Aug 10, 2020, at 9:24 PM, Dietmar Eggemann <[email protected]> wrote:
On 06/08/2020 17:52, benbjiang(蒋彪) wrote:
Hi,On Aug 6, 2020, at 9:29 PM, Dietmar Eggemann <[email protected]> wrote:
On 03/08/2020 13:26, benbjiang(蒋彪) wrote:
On Aug 3, 2020, at 4:16 PM, Dietmar Eggemann <[email protected]> wrote:
On 01/08/2020 04:32, Jiang Biao wrote:
From: Jiang Biao <[email protected]>[...]
Quoted 11 lines
Quoted 1 line
Are you sure about this?
Yes. :)
Quoted 8 lines
The math is telling me for the:
idle task: (3 / (1024 + 1024 + 3))^(-1) * 4ms = 2735ms
normal task: (1024 / (1024 + 1024 + 3))^(-1) * 4ms = 8ms
(4ms - 250 Hz)
My tick is 1ms - 1000HZ, which seems reasonable for 600ms? :)
OK, I see.
But here the different sched slices (check_preempt_tick()->
sched_slice()) between normal tasks and the idle task play a role to.Quoted 8 lines
Quoted 6 lines
Quoted 1 line
Normal tasks get ~3ms whereas the idle task gets <0.01ms.
In fact that depends on the number of CPUs on the system :sysctl_sched_latency = 6ms * (1 + ilog(ncpus)) . On a 8 cores system, normal task will run around 12ms in one shoot and the idle task still one tick period
True. This is on a single CPU.
Agree. :)
Quoted 1 line
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Also, you can increase even more the period between 2 runs of idle task by using cgroups and min shares value : 2
Ah yes, maybe this is what Jiang wants to do then? If his runtime does not have other requirements preventing this.
That could work for increasing the period between 2 runs. But could not reduce the single runtime of idle task I guess, which means normal task could have 1-tick schedule latency because of idle task.
Quoted 5 lines
Yes. An idle task will preempt an always running task during 1 tick every 680ms. But also you should keep in mind that a waking normal task will preempt the idle task immediately which means that it will not add scheduling latency to a normal task but "steal" 0.14% of normal task throughput (1/680) at most
That’s true. But in the VM case, when VM are busy(MWAIT passthrough or running cpu eating works), the 1-tick scheduling latency could be detected by cyclictest running in the VM.
OTOH, we compensate vruntime in
place_entity()to boot waking without distinguishSCHED_IDLEtask, do you think it’s necessary to do that? likekernel/sched/fair.c +0
-0@@ -4115,7 +4115,7 @@ place_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int initial) vruntime +=sched_vslice (cfs_rq, se);
Quoted 6 lines
kernel/sched/fair.c place_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int initial) · line 4119
Quoted 4 lines
kernel/sched/fair.c +1
-1/* sleeps up to a single latency don't count. */ -if (!initial) { +if (!initial &&likely (!task_has_idle_policy (task_of (se)))) {unsigned long thresh = sysctl_sched_latency;Yeah, this is a good improvement.
Thanks, I’ll send a patch for that. :)
Quoted 2 lines
Does it solve your problem ?
Not exactly. :) I wonder if we can make SCHED_IDLE more pure(harmless)? Or introduce a switch(or flag) to control it, and make it available for cases like us.
Thanks a lot.
Regards,
Jiang
Quoted 22 lines
Quoted 22 lines
Quoted 10 lines
Quoted 8 lines
OTOH, cgroups(shares) could introduce extra complexity. :)
I wonder if there’s any possibility to make SCHED_IDLEs’ priorities absolutely lower than
SCHED_NORMAL(OTHER), which means no weights/shares
for them, and they run only when no other task’s runnable.
I guess there may be priority inversion issue if we do that. But maybe weExactly, that's why we must ensure a minimum running time for
sched_idletaskStill for VM case, different VMs have been much isolated from each other, priority inversion issue could be very rare, we’re trying to make offline tasks absoultly harmless to online tasks. :)
Thanks a lot for your time.
Regards,
JiangQuoted 3 lines
Quoted 3 lines
could avoid it by load-balance more aggressively, or it(priority inversion) could be ignored in some special case.
Vincent Guittot
maintainerRe: [PATCH] sched/fair: reduce preemption with IDLE tasks runable(Internet mail)
Quoted 116 lines · benbjiang(蒋彪)
Quoted 110 lines · Vincent Guittot
Quoted 104 lines · benbjiang(蒋彪)
Quoted 86 lines · Vincent Guittot
Quoted 71 lines · benbjiang(蒋彪)
Quoted 60 lines · Dietmar Eggemann
Quoted 54 lines · Vincent Guittot
Quoted 53 lines · Dietmar Eggemann
Quoted 31 lines · benbjiang(蒋彪)
Quoted 31 lines · Dietmar Eggemann
Quoted 26 lines · benbjiang(蒋彪)
Hi,
Quoted 24 lines · Dietmar Eggemann
Quoted 21 lines · benbjiang(蒋彪)
Hi,
On Aug 11, 2020, at 11:54 PM, Dietmar Eggemann <[email protected]> wrote:
On 11/08/2020 02:41, benbjiang(蒋彪) wrote:
Hi,On Aug 10, 2020, at 9:24 PM, Dietmar Eggemann <[email protected]> wrote:
On 06/08/2020 17:52, benbjiang(蒋彪) wrote:
Hi,On Aug 6, 2020, at 9:29 PM, Dietmar Eggemann <[email protected]> wrote:
On 03/08/2020 13:26, benbjiang(蒋彪) wrote:
On Aug 3, 2020, at 4:16 PM, Dietmar Eggemann <[email protected]> wrote:
On 01/08/2020 04:32, Jiang Biao wrote:
From: Jiang Biao <[email protected]>[...]
Quoted 11 lines
Quoted 1 line
Are you sure about this?
Yes. :)
Quoted 8 lines
The math is telling me for the:
idle task: (3 / (1024 + 1024 + 3))^(-1) * 4ms = 2735ms
normal task: (1024 / (1024 + 1024 + 3))^(-1) * 4ms = 8ms
(4ms - 250 Hz)
My tick is 1ms - 1000HZ, which seems reasonable for 600ms? :)
OK, I see.
But here the different sched slices (check_preempt_tick()->
sched_slice()) between normal tasks and the idle task play a role to.Quoted 8 lines
Quoted 6 lines
Quoted 1 line
Normal tasks get ~3ms whereas the idle task gets <0.01ms.
In fact that depends on the number of CPUs on the system :sysctl_sched_latency = 6ms * (1 + ilog(ncpus)) . On a 8 cores system, normal task will run around 12ms in one shoot and the idle task still one tick period
True. This is on a single CPU.
Agree. :)
Quoted 1 line
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Also, you can increase even more the period between 2 runs of idle task by using cgroups and min shares value : 2
Ah yes, maybe this is what Jiang wants to do then? If his runtime does not have other requirements preventing this.
That could work for increasing the period between 2 runs. But could not reduce the single runtime of idle task I guess, which means normal task could have 1-tick schedule latency because of idle task.
Quoted 5 lines
Yes. An idle task will preempt an always running task during 1 tick every 680ms. But also you should keep in mind that a waking normal task will preempt the idle task immediately which means that it will not add scheduling latency to a normal task but "steal" 0.14% of normal task throughput (1/680) at most
That’s true. But in the VM case, when VM are busy(MWAIT passthrough or running cpu eating works), the 1-tick scheduling latency could be detected by cyclictest running in the VM.
OTOH, we compensate vruntime in
place_entity()to boot waking without distinguishSCHED_IDLEtask, do you think it’s necessary to do that? likekernel/sched/fair.c +0
-0@@ -4115,7 +4115,7 @@ place_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int initial) vruntime +=sched_vslice (cfs_rq, se);Quoted 6 lines
kernel/sched/fair.c place_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int initial) · line 4119
Quoted 4 lines
kernel/sched/fair.c +1
-1/* sleeps up to a single latency don't count. */ -if (!initial) { +if (!initial &&likely (!task_has_idle_policy (task_of (se)))) {unsigned long thresh = sysctl_sched_latency;Yeah, this is a good improvement.
Quoted 5 lines
Thanks, I’ll send a patch for that. :)
Quoted 2 lines
Does it solve your problem ?
Not exactly. :) I wonder if we can make
SCHED_IDLEmore pure(harmless)?
We can't prevent it from running time to time. Proxy execution feature could be a step for considering to relax this constraint
Quoted 29 lines
Or introduce a switch(or flag) to control it, and make it available for cases like us.
Thanks a lot.
Regards,
JiangQuoted 22 lines
Quoted 22 lines
Quoted 10 lines
Quoted 8 lines
OTOH, cgroups(shares) could introduce extra complexity. :)
I wonder if there’s any possibility to make SCHED_IDLEs’ priorities absolutely lower than
SCHED_NORMAL(OTHER), which means no weights/shares
for them, and they run only when no other task’s runnable.
I guess there may be priority inversion issue if we do that. But maybe weExactly, that's why we must ensure a minimum running time for
sched_idletaskStill for VM case, different VMs have been much isolated from each other, priority inversion issue could be very rare, we’re trying to make offline tasks absoultly harmless to online tasks. :)
Thanks a lot for your time.
Regards,
JiangQuoted 3 lines
Quoted 3 lines
could avoid it by load-balance more aggressively, or it(priority inversion) could be ignored in some special case.
Re: [PATCH] sched/fair: reduce preemption with IDLE tasks runable(Internet mail)
Quoted 118 lines · Vincent Guittot
Quoted 116 lines · benbjiang(蒋彪)
Quoted 116 lines · Vincent Guittot
Quoted 102 lines · benbjiang(蒋彪)
Quoted 86 lines · Vincent Guittot
Quoted 71 lines · benbjiang(蒋彪)
Quoted 60 lines · Dietmar Eggemann
Quoted 54 lines · Vincent Guittot
Quoted 53 lines · Dietmar Eggemann
Quoted 31 lines · benbjiang(蒋彪)
Quoted 31 lines · Dietmar Eggemann
Quoted 26 lines · benbjiang(蒋彪)
Hi,
Quoted 24 lines · Dietmar Eggemann
On 12/08/2020 05:19, benbjiang(蒋彪) wrote:
Hi,On Aug 11, 2020, at 11:54 PM, Dietmar Eggemann <[email protected]> wrote:
On 11/08/2020 02:41, benbjiang(蒋彪) wrote:
Hi,On Aug 10, 2020, at 9:24 PM, Dietmar Eggemann <[email protected]> wrote:
On 06/08/2020 17:52, benbjiang(蒋彪) wrote:
Hi,On Aug 6, 2020, at 9:29 PM, Dietmar Eggemann <[email protected]> wrote:
On 03/08/2020 13:26, benbjiang(蒋彪) wrote:
On Aug 3, 2020, at 4:16 PM, Dietmar Eggemann <[email protected]> wrote:
On 01/08/2020 04:32, Jiang Biao wrote:
From: Jiang Biao <[email protected]>[...]
Quoted 11 lines
Quoted 1 line
Are you sure about this?
Yes. :)
Quoted 8 lines
The math is telling me for the:
idle task: (3 / (1024 + 1024 + 3))^(-1) * 4ms = 2735ms
normal task: (1024 / (1024 + 1024 + 3))^(-1) * 4ms = 8ms
(4ms - 250 Hz)
My tick is 1ms - 1000HZ, which seems reasonable for 600ms? :)
OK, I see.
But here the different sched slices (check_preempt_tick()->
sched_slice()) between normal tasks and the idle task play a role to.Quoted 8 lines
Quoted 6 lines
Quoted 1 line
Normal tasks get ~3ms whereas the idle task gets <0.01ms.
In fact that depends on the number of CPUs on the system :sysctl_sched_latency = 6ms * (1 + ilog(ncpus)) . On a 8 cores system, normal task will run around 12ms in one shoot and the idle task still one tick period
True. This is on a single CPU.
Agree. :)
Quoted 1 line
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Also, you can increase even more the period between 2 runs of idle task by using cgroups and min shares value : 2
Ah yes, maybe this is what Jiang wants to do then? If his runtime does not have other requirements preventing this.
That could work for increasing the period between 2 runs. But could not reduce the single runtime of idle task I guess, which means normal task could have 1-tick schedule latency because of idle task.
Quoted 5 lines
Yes. An idle task will preempt an always running task during 1 tick every 680ms. But also you should keep in mind that a waking normal task will preempt the idle task immediately which means that it will not add scheduling latency to a normal task but "steal" 0.14% of normal task throughput (1/680) at most
That’s true. But in the VM case, when VM are busy(MWAIT passthrough or running cpu eating works), the 1-tick scheduling latency could be detected by cyclictest running in the VM.
OTOH, we compensate vruntime in
place_entity()to boot waking without distinguishSCHED_IDLEtask, do you think it’s necessary to do that? likekernel/sched/fair.c place_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int initial) · line 4119
Quoted 6 lines
kernel/sched/fair.c +1
-1@@ -4115,7 +4115,7 @@ place_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int initial) vruntime +=sched_vslice (cfs_rq, se);/* sleeps up to a single latency don't count. */ -if (!initial) { +if (!initial &&likely (!task_has_idle_policy (task_of (se)))) {unsigned long thresh = sysctl_sched_latency;Yeah, this is a good improvement.
Quoted 9 lines
Quoted 5 lines
Thanks, I’ll send a patch for that. :)
Quoted 2 lines
Does it solve your problem ?
Not exactly. :) I wonder if we can make
SCHED_IDLEmore pure(harmless)?We can't prevent it from running time to time. Proxy execution feature could be a step for considering to relax this constraint
Could you please help to explain more about the *Proxy execution feature*? I'm not sure I got the right point.
Thanks a lot.
Regards,
Jiang
Vincent Guittot
maintainerRe: [PATCH] sched/fair: reduce preemption with IDLE tasks runable(Internet mail)
Quoted 125 lines · Jiang Biao
Quoted 122 lines · Vincent Guittot
Quoted 120 lines · benbjiang(蒋彪)
Quoted 110 lines · Vincent Guittot
Quoted 104 lines · benbjiang(蒋彪)
Quoted 86 lines · Vincent Guittot
Quoted 71 lines · benbjiang(蒋彪)
Quoted 60 lines · Dietmar Eggemann
Quoted 54 lines · Vincent Guittot
Quoted 53 lines · Dietmar Eggemann
Quoted 31 lines · benbjiang(蒋彪)
Quoted 31 lines · Dietmar Eggemann
Quoted 26 lines · benbjiang(蒋彪)
Hi,
On Aug 13, 2020, at 2:39 AM, Dietmar Eggemann <[email protected]> wrote:
On 12/08/2020 05:19, benbjiang(蒋彪) wrote:
Hi,On Aug 11, 2020, at 11:54 PM, Dietmar Eggemann <[email protected]> wrote:
On 11/08/2020 02:41, benbjiang(蒋彪) wrote:
Hi,On Aug 10, 2020, at 9:24 PM, Dietmar Eggemann <[email protected]> wrote:
On 06/08/2020 17:52, benbjiang(蒋彪) wrote:
Hi,On Aug 6, 2020, at 9:29 PM, Dietmar Eggemann <[email protected]> wrote:
On 03/08/2020 13:26, benbjiang(蒋彪) wrote:
On Aug 3, 2020, at 4:16 PM, Dietmar Eggemann <[email protected]> wrote:
On 01/08/2020 04:32, Jiang Biao wrote:
From: Jiang Biao <[email protected]>[...]
Quoted 11 lines
Quoted 1 line
Are you sure about this?
Yes. :)
Quoted 8 lines
The math is telling me for the:
idle task: (3 / (1024 + 1024 + 3))^(-1) * 4ms = 2735ms
normal task: (1024 / (1024 + 1024 + 3))^(-1) * 4ms = 8ms
(4ms - 250 Hz)
My tick is 1ms - 1000HZ, which seems reasonable for 600ms? :)
OK, I see.
But here the different sched slices (check_preempt_tick()->
sched_slice()) between normal tasks and the idle task play a role to.Quoted 8 lines
Quoted 6 lines
Quoted 1 line
Normal tasks get ~3ms whereas the idle task gets <0.01ms.
In fact that depends on the number of CPUs on the system :sysctl_sched_latency = 6ms * (1 + ilog(ncpus)) . On a 8 cores system, normal task will run around 12ms in one shoot and the idle task still one tick period
True. This is on a single CPU.
Agree. :)
Quoted 1 line
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Also, you can increase even more the period between 2 runs of idle task by using cgroups and min shares value : 2
Ah yes, maybe this is what Jiang wants to do then? If his runtime does not have other requirements preventing this.
That could work for increasing the period between 2 runs. But could not reduce the single runtime of idle task I guess, which means normal task could have 1-tick schedule latency because of idle task.
Quoted 5 lines
Yes. An idle task will preempt an always running task during 1 tick every 680ms. But also you should keep in mind that a waking normal task will preempt the idle task immediately which means that it will not add scheduling latency to a normal task but "steal" 0.14% of normal task throughput (1/680) at most
That’s true. But in the VM case, when VM are busy(MWAIT passthrough or running cpu eating works), the 1-tick scheduling latency could be detected by cyclictest running in the VM.
OTOH, we compensate vruntime in
place_entity()to boot waking without distinguishSCHED_IDLEtask, do you think it’s necessary to do that? likekernel/sched/fair.c +0
-0@@ -4115,7 +4115,7 @@ place_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int initial) vruntime +=sched_vslice (cfs_rq, se);Quoted 6 lines
kernel/sched/fair.c place_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int initial) · line 4119
Quoted 4 lines
kernel/sched/fair.c +1
-1/* sleeps up to a single latency don't count. */ -if (!initial) { +if (!initial &&likely (!task_has_idle_policy (task_of (se)))) {unsigned long thresh = sysctl_sched_latency;Yeah, this is a good improvement.
Thanks, I’ll send a patch for that. :)
Quoted 2 lines
Does it solve your problem ?
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
Not exactly. :) I wonder if we can make
SCHED_IDLEmore pure(harmless)?We can't prevent it from running time to time. Proxy execution feature could be a step for considering to relax this constraint
Could you please help to explain more about the *Proxy execution feature*? I'm not sure I got the right point.
https://lwn.net/Articles/820575/
https://lwn.net/Articles/793502/
Quoted 4 lines
Thanks a lot.
Regards,
Jiang
Re: [PATCH] sched/fair: reduce preemption with IDLE tasks runable(Internet mail)
Quoted 129 lines · Vincent Guittot
Quoted 127 lines · Jiang Biao
Quoted 124 lines · Vincent Guittot
Quoted 121 lines · benbjiang(蒋彪)
Quoted 110 lines · Vincent Guittot
Quoted 104 lines · benbjiang(蒋彪)
Quoted 86 lines · Vincent Guittot
Quoted 71 lines · benbjiang(蒋彪)
Quoted 60 lines · Dietmar Eggemann
Quoted 54 lines · Vincent Guittot
Quoted 53 lines · Dietmar Eggemann
Quoted 31 lines · benbjiang(蒋彪)
Quoted 31 lines · Dietmar Eggemann
On 14/08/2020 01:55, benbjiang(蒋彪) wrote:
Hi,On Aug 13, 2020, at 2:39 AM, Dietmar Eggemann <[email protected]> wrote:
On 12/08/2020 05:19, benbjiang(蒋彪) wrote:
Hi,On Aug 11, 2020, at 11:54 PM, Dietmar Eggemann <[email protected]> wrote:
On 11/08/2020 02:41, benbjiang(蒋彪) wrote:
Hi,On Aug 10, 2020, at 9:24 PM, Dietmar Eggemann <[email protected]> wrote:
On 06/08/2020 17:52, benbjiang(蒋彪) wrote:
Hi,On Aug 6, 2020, at 9:29 PM, Dietmar Eggemann <[email protected]> wrote:
On 03/08/2020 13:26, benbjiang(蒋彪) wrote:
On Aug 3, 2020, at 4:16 PM, Dietmar Eggemann <[email protected]> wrote:
On 01/08/2020 04:32, Jiang Biao wrote:
From: Jiang Biao <[email protected]>[...]
Quoted 11 lines
Quoted 1 line
Are you sure about this?
Yes. :)
Quoted 8 lines
The math is telling me for the:
idle task: (3 / (1024 + 1024 + 3))^(-1) * 4ms = 2735ms
normal task: (1024 / (1024 + 1024 + 3))^(-1) * 4ms = 8ms
(4ms - 250 Hz)
My tick is 1ms - 1000HZ, which seems reasonable for 600ms? :)
OK, I see.
But here the different sched slices (check_preempt_tick()->
sched_slice()) between normal tasks and the idle task play a role to.Quoted 8 lines
Quoted 6 lines
Quoted 1 line
Normal tasks get ~3ms whereas the idle task gets <0.01ms.
In fact that depends on the number of CPUs on the system :sysctl_sched_latency = 6ms * (1 + ilog(ncpus)) . On a 8 cores system, normal task will run around 12ms in one shoot and the idle task still one tick period
True. This is on a single CPU.
Agree. :)
Quoted 1 line
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Also, you can increase even more the period between 2 runs of idle task by using cgroups and min shares value : 2
Ah yes, maybe this is what Jiang wants to do then? If his runtime does not have other requirements preventing this.
That could work for increasing the period between 2 runs. But could not reduce the single runtime of idle task I guess, which means normal task could have 1-tick schedule latency because of idle task.
Quoted 5 lines
Yes. An idle task will preempt an always running task during 1 tick every 680ms. But also you should keep in mind that a waking normal task will preempt the idle task immediately which means that it will not add scheduling latency to a normal task but "steal" 0.14% of normal task throughput (1/680) at most
That’s true. But in the VM case, when VM are busy(MWAIT passthrough or running cpu eating works), the 1-tick scheduling latency could be detected by cyclictest running in the VM.
OTOH, we compensate vruntime in
place_entity()to boot waking without distinguishSCHED_IDLEtask, do you think it’s necessary to do that? likekernel/sched/fair.c +0
-0@@ -4115,7 +4115,7 @@ place_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int initial) vruntime +=sched_vslice (cfs_rq, se);Quoted 6 lines
kernel/sched/fair.c place_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int initial) · line 4119
Quoted 4 lines
kernel/sched/fair.c +1
-1/* sleeps up to a single latency don't count. */ -if (!initial) { +if (!initial &&likely (!task_has_idle_policy (task_of (se)))) {unsigned long thresh = sysctl_sched_latency;Yeah, this is a good improvement.
Thanks, I’ll send a patch for that. :)
Quoted 2 lines
Does it solve your problem ?
Not exactly. :) I wonder if we can make
SCHED_IDLEmore pure(harmless)?
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
We can't prevent it from running time to time. Proxy execution feature could be a step for considering to relax this constraint
Could you please help to explain more about the *Proxy execution feature*? I'm not sure I got the right point.
https://lwn.net/Articles/820575/
https://lwn.net/Articles/793502/
Good to hear about that. I guess there would still be a long way to go. :)
Thanks again for your kindly patience.
Regard,
Jiang
Re: [PATCH] sched/fair: reduce preemption with IDLE tasks runable(Internet mail)
In reply to Dietmar EggemannQuoted 56 lines · Dietmar Eggemann
Quoted 31 lines · benbjiang(蒋彪)
Quoted 31 lines · Dietmar Eggemann
Quoted 26 lines · benbjiang(蒋彪)
Hi,
Quoted 24 lines · Dietmar Eggemann
Quoted 21 lines · benbjiang(蒋彪)
Hi,
Quoted 19 lines · Dietmar Eggemann
Quoted 16 lines · benbjiang(蒋彪)
Hi,
Quoted 14 lines · Dietmar Eggemann
Quoted 11 lines · benbjiang(蒋彪)
Hi,
Quoted 9 lines · Dietmar Eggemann
Quoted 6 lines · benbjiang(蒋彪)
Quoted 6 lines · Dietmar Eggemann
On 01/08/2020 04:32, Jiang Biao wrote:
From: Jiang Biao <[email protected]>[...]
Quoted 11 lines
Quoted 1 line
Are you sure about this?
Yes. :)
Quoted 8 lines
The math is telling me for the:
idle task: (3 / (1024 + 1024 + 3))^(-1) * 4ms = 2735ms
normal task: (1024 / (1024 + 1024 + 3))^(-1) * 4ms = 8ms
(4ms - 250 Hz)
My tick is 1ms - 1000HZ, which seems reasonable for 600ms? :)
OK, I see.
But here the different sched slices (check_preempt_tick()->
sched_slice()) between normal tasks and the idle task play a role to.
Quoted 8 lines
Normal tasks get ~3ms whereas the idle task gets <0.01ms.
So the idle task runs every ~680ms but only for 1 tick (1ms) (4 times less than the normal tasks). The condition 'if (
delta_exec>ideal_runtime)' incheck_preempt_tick()is only true at the 4th tick when a normal task runs even though the slice is 3ms.In the 250 Hz example the sched slice diffs are hidden behind the 4ms tick.
Exactly. Tick size decides the single runtime and the interval between two runs of idle task.
That makes sense for most of the test results above.
Thx.
Regards,
Jiang