Skip to contentSkip to conversation

sched/fair: Fix weight overly small for interactive group entity

PATCH In review Yuyang Du 3 participants 5 messages Oct 13, 2015 – Oct 20, 2015 Scheduler

lore
Share & tools
lore

Yuyang Du

author
#0 lore raw

PATCH 1/2 sched/fair: Fix weight overly small for interactive group entity

Commit 9d89c257dfb9c51a532d69 (sched/fair: Rewrite runnable load and utilization average tracking) led to overly small weight for interactive group entity. The case can be easily reproduced when a number of CPU hogs compete for the CPUs at the same time (thanks to Mike). This is largly because the task group's load average tracking cross CPUs lags behind the real changes.

We accelerate the group share distribution process by using the load.weight of the cfs_rq. This may increase the entire group's share, but we have to do so to protect the (fragile) interactive tasks from especially CPU hogs.

---
 kernel/sched/fair.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
kernel/sched/fair.c +2 -2
index 700eb54..601a253 100644
@@ -2370,7 +2370,7 @@ static inline long calc_tg_weight(struct task_group *tg, struct cfs_rq *cfs_rq)
 	 */
 	tg_weight = atomic_long_read(&tg->load_avg);
 	tg_weight -= cfs_rq->tg_load_avg_contrib;
-	tg_weight += cfs_rq_load_avg(cfs_rq);
+	tg_weight += cfs_rq->load.weight;
 
 	return tg_weight;
 }
@@ -2380,7 +2380,7 @@ static long calc_cfs_shares(struct cfs_rq *cfs_rq, struct task_group *tg)
 	long tg_weight, load, shares;
 
 	tg_weight = calc_tg_weight(tg, cfs_rq);
-	load = cfs_rq_load_avg(cfs_rq);
+	load = cfs_rq->load.weight;
 
 	shares = (tg->shares * load);
 	if (tg_weight)
Signature · 1 line
2.1.4
author
#1 lore raw

PATCH 2/2 sched/fair: Update task group's load_avg after task migrate

When cfs_rq has removed_load_avg (when task migrates from this cfs_rq), we need to update its contribution to the group's load_avg. This should not increase tg's update too much, because in most of the cases, the cfs_rq has already decayed load_avg.

---
 kernel/sched/fair.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
kernel/sched/fair.c +3 -2
index 601a253..824aa9f 100644
@@ -2686,12 +2686,13 @@ static inline u64 cfs_rq_clock_task(struct cfs_rq *cfs_rq);
 static inline int update_cfs_rq_load_avg(u64 now, struct cfs_rq *cfs_rq)
 {
 	struct sched_avg *sa = &cfs_rq->avg;
-	int decayed;
+	int decayed, removed = 0;
 
 	if (atomic_long_read(&cfs_rq->removed_load_avg)) {
 		long r = atomic_long_xchg(&cfs_rq->removed_load_avg, 0);
 		sa->load_avg = max_t(long, sa->load_avg - r, 0);
 		sa->load_sum = max_t(s64, sa->load_sum - r * LOAD_AVG_MAX, 0);
+		removed = 1;
 	}
 
 	if (atomic_long_read(&cfs_rq->removed_util_avg)) {
@@ -2708,7 +2709,7 @@ static inline int update_cfs_rq_load_avg(u64 now, struct cfs_rq *cfs_rq)
 	cfs_rq->load_last_update_time_copy = sa->last_update_time;
 #endif
 
-	return decayed;
+	return decayed || removed;
 }
 
 /* Update task and its cfs_rq load average */
Signature · 1 line
2.1.4
#2 lore raw

[tip:sched/core] sched/fair: Update task group' s load_avg after task migration

sched/fair: Update task group's load_avg after task migration
Commit-ID:  3e386d56bafbb6d2540b49367444997fc671ea69
Gitweb:     http://git.kernel.org/tip/3e386d56bafbb6d2540b49367444997fc671ea69
Author:     Yuyang Du <[email protected]>
AuthorDate: Tue, 13 Oct 2015 09:18:23 +0800
Committer:  Ingo Molnar <[email protected]>
CommitDate: Tue, 20 Oct 2015 10:13:35 +0200

sched/fair: Update task group's load_avg after task migration

When cfs_rq has cfs_rq->removed_load_avg set (when a task migrates from this cfs_rq), we need to update its contribution to the group's load_avg.

This should not increase tg's update too much, because in most cases, the cfs_rq has already decayed its load_avg.

---
 kernel/sched/fair.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
kernel/sched/fair.c +3 -2
index bc62c50..9a5e60f 100644
@@ -2664,13 +2664,14 @@ static inline u64 cfs_rq_clock_task(struct cfs_rq *cfs_rq);
 /* Group cfs_rq's load_avg is used for task_h_load and update_cfs_share */
 static inline int update_cfs_rq_load_avg(u64 now, struct cfs_rq *cfs_rq)
 {
-	int decayed;
 	struct sched_avg *sa = &cfs_rq->avg;
+	int decayed, removed = 0;
 
 	if (atomic_long_read(&cfs_rq->removed_load_avg)) {
 		long r = atomic_long_xchg(&cfs_rq->removed_load_avg, 0);
 		sa->load_avg = max_t(long, sa->load_avg - r, 0);
 		sa->load_sum = max_t(s64, sa->load_sum - r * LOAD_AVG_MAX, 0);
+		removed = 1;
 	}
 
 	if (atomic_long_read(&cfs_rq->removed_util_avg)) {
@@ -2688,7 +2689,7 @@ static inline int update_cfs_rq_load_avg(u64 now, struct cfs_rq *cfs_rq)
 	cfs_rq->load_last_update_time_copy = sa->last_update_time;
 #endif
 
-	return decayed;
+	return decayed || removed;
 }
 
 /* Update task and its cfs_rq load average */
#3 lore raw
In reply to Yuyang Du

Hi Yuhang,

Quoted 23 lines · Yuyang Du

Commit 9d89c257dfb9c51a532d69 (sched/fair: Rewrite runnable load and utilization average tracking) led to overly small weight for interactive group entity. The case can be easily reproduced when a number of CPU hogs compete for the CPUs at the same time (thanks to Mike). This is largly because the task group's load average tracking cross CPUs lags behind the real changes.

We accelerate the group share distribution process by using the load.weight of the cfs_rq. This may increase the entire group's share, but we have to do so to protect the (fragile) interactive tasks from especially CPU hogs.

---
 kernel/sched/fair.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
kernel/sched/fair.c +0 -0
index 700eb54..601a253 100644

kernel/sched/fair.c static inline long calc_tg_weight(struct task_group *tg, struct cfs_rq *cfs_rq) · line 2370

Quoted 1 line
kernel/sched/fair.c +0 -0
@@ -2370,7 +2370,7 @@ static inline long calc_tg_weight(struct task_group *tg, struct cfs_rq *cfs_rq)
 	 */

In commit 9d89c257dfb9c51a532d69 you also changed the comment on top of this from 'CPU's actual weight' to 'CPU's real-time load'. I assume here that the former stands for cfs_rq->load.weight and the latter for cfs_rq->avg.load_avg. Not sure though ...

Quoted 9 lines
kernel/sched/fair.c +1 -1
 	tg_weight = atomic_long_read(&tg->load_avg);
 	tg_weight -= cfs_rq->tg_load_avg_contrib;
-	tg_weight += cfs_rq_load_avg(cfs_rq);
+	tg_weight += cfs_rq->load.weight;
 
 	return tg_weight;
 }

kernel/sched/fair.c static long calc_cfs_shares(struct cfs_rq *cfs_rq, struct task_group *tg) · line 2386

Quoted 8 lines
kernel/sched/fair.c +1 -1
@@ -2380,7 +2380,7 @@ static long calc_cfs_shares(struct cfs_rq *cfs_rq, struct task_group *tg)
 	long tg_weight, load, shares;
 
 	tg_weight = calc_tg_weight(tg, cfs_rq);
-	load = cfs_rq_load_avg(cfs_rq);
+	load = cfs_rq->load.weight;
 
 	shares = (tg->shares * load);
 	if (tg_weight)

I get similar test results on a i7-4750HQ (1*4*2) system

Test setup: 1 cpuhog (sysbench --test cpu --num-threads 1) each in his
own cpu cgroup and each pinned to a cpu
mlayer (threads=8) in a cpu cgroup (BigBuckBunny- DivXPlusHD.mkv)
runtime: 100s
system: Ubuntu 14.04.3 LTS desktop

sum of the runtime of the mplayer threads:

commit cd126afe838d (before pelt rewrite): 55.6s
4.3.0-rc5 : 36.5s
4.3.0-rc5 + patch 1/2 and 2/2 : 55.7s

#4 lore raw

[tip:sched/core] sched/fair: Fix overly small weight for interactive group entities

In reply to Yuyang Du
sched/fair: Fix overly small weight for interactive group entities
Commit-ID:  fde7d22e01aa0d252fc5c95fa11f0dac35a4dd59
Gitweb:     http://git.kernel.org/tip/fde7d22e01aa0d252fc5c95fa11f0dac35a4dd59
Author:     Yuyang Du <[email protected]>
AuthorDate: Tue, 13 Oct 2015 09:18:22 +0800
Committer:  Ingo Molnar <[email protected]>
CommitDate: Tue, 20 Oct 2015 10:13:34 +0200

sched/fair: Fix overly small weight for interactive group entities

Commit:

9d89c257dfb9 ("sched/fair: Rewrite runnable load and utilization average tracking")

led to an overly small weight for interactive group entities. The bad case can be easily reproduced when a number of CPU hogs compete for the CPUs at the same time (thanks to Mike). This is largly because the task group's load average tracking cross CPUs lags behind the real changes.

To fix this we accelerate the group share distribution process by using the load.weight of the cfs_rq. This may increase the entire group's share, but we have to do so to protect the (fragile) interactive tasks, especially from CPU hogs.

---
 kernel/sched/fair.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
kernel/sched/fair.c +2 -2
index 6e2e348..bc62c50 100644
@@ -2363,7 +2363,7 @@ static inline long calc_tg_weight(struct task_group *tg, struct cfs_rq *cfs_rq)
 	 */
 	tg_weight = atomic_long_read(&tg->load_avg);
 	tg_weight -= cfs_rq->tg_load_avg_contrib;
-	tg_weight += cfs_rq_load_avg(cfs_rq);
+	tg_weight += cfs_rq->load.weight;
 
 	return tg_weight;
 }
@@ -2373,7 +2373,7 @@ static long calc_cfs_shares(struct cfs_rq *cfs_rq, struct task_group *tg)
 	long tg_weight, load, shares;
 
 	tg_weight = calc_tg_weight(tg, cfs_rq);
-	load = cfs_rq_load_avg(cfs_rq);
+	load = cfs_rq->load.weight;
 
 	shares = (tg->shares * load);
 	if (tg_weight)