sched/fair: Fix weight overly small for interactive group entity
PATCH 1/2 sched/fair: Fix weight overly small for interactive group entity
Commit 9d89c257dfb9c51a532d69 (sched/fair: Rewrite runnable load and utilization average tracking) led to overly small weight for interactive group entity. The case can be easily reproduced when a number of CPU hogs compete for the CPUs at the same time (thanks to Mike). This is largly because the task group's load average tracking cross CPUs lags behind the real changes.
We accelerate the group share distribution process by using the load.weight of the cfs_rq. This may increase the entire group's share, but we have to do so to protect the (fragile) interactive tasks from especially CPU hogs.
--- kernel/sched/fair.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
kernel/sched/fair.c +2 -2
@@ -2370,7 +2370,7 @@ static inline long calc_tg_weight(struct task_group *tg, struct cfs_rq *cfs_rq)*/ tg_weight =atomic_long_read (&tg->load_avg); tg_weight -= cfs_rq->tg_load_avg_contrib; - tg_weight +=cfs_rq_load_avg (cfs_rq); + tg_weight += cfs_rq->load.weight;return tg_weight; }
@@ -2380,7 +2380,7 @@ static long calc_cfs_shares(struct cfs_rq *cfs_rq, struct task_group *tg)long tg_weight, load, shares; tg_weight =calc_tg_weight (tg, cfs_rq); - load =cfs_rq_load_avg (cfs_rq); + load = cfs_rq->load.weight; shares = (tg->shares * load);if (tg_weight)
Signature · 1 line
2.1.4
PATCH 2/2 sched/fair: Update task group's load_avg after task migrate
When cfs_rq has removed_load_avg (when task migrates from this cfs_rq), we need to update its contribution to the group's load_avg. This should not increase tg's update too much, because in most of the cases, the cfs_rq has already decayed load_avg.
--- kernel/sched/fair.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
kernel/sched/fair.c +3 -2
@@ -2686,12 +2686,13 @@ static inline u64 cfs_rq_clock_task(struct cfs_rq *cfs_rq);static inline int update_cfs_rq_load_avg (u64 now,struct cfs_rq *cfs_rq) {struct sched_avg *sa = &cfs_rq->avg; -int decayed; +int decayed, removed =0 ;if (atomic_long_read (&cfs_rq->removed_load_avg)) {long r =atomic_long_xchg (&cfs_rq->removed_load_avg,0 ); sa->load_avg =max_t (long, sa->load_avg - r,0 ); sa->load_sum =max_t (s64, sa->load_sum - r *LOAD_AVG_MAX ,0 ); + removed =1 ; }if (atomic_long_read (&cfs_rq->removed_util_avg)) {
@@ -2708,7 +2709,7 @@ static inline int update_cfs_rq_load_avg(u64 now, struct cfs_rq *cfs_rq) cfs_rq->load_last_update_time_copy = sa->last_update_time;#endif -return decayed; +return decayed || removed; }/* Update task and its cfs_rq load average */
Signature · 1 line
2.1.4
tip-bot for Yuyang Du
merge bot[tip:sched/core] sched/fair: Update task group' s load_avg after task migration
sched/fair: Update task group's load_avg after task migration
Commit-ID: 3e386d56bafbb6d2540b49367444997fc671ea69 Gitweb: http://git.kernel.org/tip/3e386d56bafbb6d2540b49367444997fc671ea69 Author: Yuyang Du <[email protected]> AuthorDate: Tue, 13 Oct 2015 09:18:23 +0800 Committer: Ingo Molnar <[email protected]> CommitDate: Tue, 20 Oct 2015 10:13:35 +0200
sched/fair: Update task group's load_avg after task migration
When cfs_rq has cfs_rq->removed_load_avg set (when a task migrates from this cfs_rq), we need to update its contribution to the group's load_avg.
This should not increase tg's update too much, because in most cases, the cfs_rq has already decayed its load_avg.
--- kernel/sched/fair.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
kernel/sched/fair.c +3 -2
@@ -2664,13 +2664,14 @@ static inline u64 cfs_rq_clock_task(struct cfs_rq *cfs_rq);/* Group cfs_rq's load_avg is used for task_h_load and update_cfs_share */ static inline int update_cfs_rq_load_avg (u64 now,struct cfs_rq *cfs_rq) { -int decayed;struct sched_avg *sa = &cfs_rq->avg; +int decayed, removed =0 ;if (atomic_long_read (&cfs_rq->removed_load_avg)) {long r =atomic_long_xchg (&cfs_rq->removed_load_avg,0 ); sa->load_avg =max_t (long, sa->load_avg - r,0 ); sa->load_sum =max_t (s64, sa->load_sum - r *LOAD_AVG_MAX ,0 ); + removed =1 ; }if (atomic_long_read (&cfs_rq->removed_util_avg)) {
@@ -2688,7 +2689,7 @@ static inline int update_cfs_rq_load_avg(u64 now, struct cfs_rq *cfs_rq) cfs_rq->load_last_update_time_copy = sa->last_update_time;#endif -return decayed; +return decayed || removed; }/* Update task and its cfs_rq load average */
Dietmar Eggemann
reviewerHi Yuhang,
Quoted 23 lines · Yuyang Du
Commit
9d89c257dfb9c51a532d69(sched/fair: Rewrite runnable load and utilization average tracking) led to overly small weight for interactive group entity. The case can be easily reproduced when a number of CPU hogs compete for the CPUs at the same time (thanks to Mike). This is largly because the task group's load average tracking cross CPUs lags behind the real changes.We accelerate the group share distribution process by using the load.weight of the
cfs_rq. This may increase the entire group's share, but we have to do so to protect the (fragile) interactive tasks from especially CPU hogs.Reported-by Mike Galbraith <[email protected]>Signed-off-by Yuyang Du <[email protected]>--- kernel/sched/fair.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)kernel/sched/fair.c +0
-0
kernel/sched/fair.c static inline long calc_tg_weight(struct task_group *tg, struct cfs_rq *cfs_rq) · line 2370
Quoted 1 line
kernel/sched/fair.c +0
-0@@ -2370,7 +2370,7 @@ static inline long calc_tg_weight(struct task_group *tg, struct cfs_rq *cfs_rq)*/
In commit 9d89c257dfb9c51a532d69 you also changed the comment on top of this from 'CPU's actual weight' to 'CPU's real-time load'. I assume here that the former stands for cfs_rq->load.weight and the latter for cfs_rq->avg.load_avg. Not sure though ...
Quoted 9 lines
kernel/sched/fair.c +1
-1tg_weight =atomic_long_read (&tg->load_avg); tg_weight -= cfs_rq->tg_load_avg_contrib; - tg_weight +=cfs_rq_load_avg (cfs_rq); + tg_weight += cfs_rq->load.weight;return tg_weight; }
kernel/sched/fair.c static long calc_cfs_shares(struct cfs_rq *cfs_rq, struct task_group *tg) · line 2386
Quoted 8 lines
kernel/sched/fair.c +1
-1@@ -2380,7 +2380,7 @@ static long calc_cfs_shares(struct cfs_rq *cfs_rq, struct task_group *tg)long tg_weight, load, shares; tg_weight =calc_tg_weight (tg, cfs_rq); - load =cfs_rq_load_avg (cfs_rq); + load = cfs_rq->load.weight; shares = (tg->shares * load);if (tg_weight)
I get similar test results on a i7-4750HQ (1*4*2) system
Test setup: 1 cpuhog (sysbench --test cpu --num-threads 1) each in his
own cpu cgroup and each pinned to a cpu
mlayer (threads=8) in a cpu cgroup (BigBuckBunny- DivXPlusHD.mkv)
runtime: 100s
system: Ubuntu 14.04.3 LTS desktop
sum of the runtime of the mplayer threads:
commit cd126afe838d (before pelt rewrite): 55.6s
4.3.0-rc5 : 36.5s
4.3.0-rc5 + patch 1/2 and 2/2 : 55.7s
tip-bot for Yuyang Du
merge bot[tip:sched/core] sched/fair: Fix overly small weight for interactive group entities
In reply to Yuyang Dusched/fair: Fix overly small weight for interactive group entities
Commit-ID: fde7d22e01aa0d252fc5c95fa11f0dac35a4dd59 Gitweb: http://git.kernel.org/tip/fde7d22e01aa0d252fc5c95fa11f0dac35a4dd59 Author: Yuyang Du <[email protected]> AuthorDate: Tue, 13 Oct 2015 09:18:22 +0800 Committer: Ingo Molnar <[email protected]> CommitDate: Tue, 20 Oct 2015 10:13:34 +0200
sched/fair: Fix overly small weight for interactive group entities
Commit:
9d89c257dfb9 ("sched/fair: Rewrite runnable load and utilization average tracking")
led to an overly small weight for interactive group entities. The bad case can be easily reproduced when a number of CPU hogs compete for the CPUs at the same time (thanks to Mike). This is largly because the task group's load average tracking cross CPUs lags behind the real changes.
To fix this we accelerate the group share distribution process by using the load.weight of the cfs_rq. This may increase the entire group's share, but we have to do so to protect the (fragile) interactive tasks, especially from CPU hogs.
--- kernel/sched/fair.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
kernel/sched/fair.c +2 -2
@@ -2363,7 +2363,7 @@ static inline long calc_tg_weight(struct task_group *tg, struct cfs_rq *cfs_rq)*/ tg_weight =atomic_long_read (&tg->load_avg); tg_weight -= cfs_rq->tg_load_avg_contrib; - tg_weight +=cfs_rq_load_avg (cfs_rq); + tg_weight += cfs_rq->load.weight;return tg_weight; }
@@ -2373,7 +2373,7 @@ static long calc_cfs_shares(struct cfs_rq *cfs_rq, struct task_group *tg)long tg_weight, load, shares; tg_weight =calc_tg_weight (tg, cfs_rq); - load =cfs_rq_load_avg (cfs_rq); + load = cfs_rq->load.weight; shares = (tg->shares * load);if (tg_weight)