Skip to contentSkip to conversation

sched: use runnable load avg in balance

PATCH In review Alex Shi 7 participants 73 messages Jun 7, 2013 – Jun 21, 2013 Scheduler

lore
Share & tools
lore

Alex Shi

author
#0 lore raw

cover sched: use runnable load avg in balance

Peter&Ingo:

v8: add a marco div64_ul and used it in task_h_load()
v7: rebasing on tip/sched/core tree.

I tested on Intel core2, NHM, SNB, IVB, 2 and 4 sockets machines with benchmark kbuild, aim7, dbench, tbench, hackbench, oltp, and netperf loopback etc.

On SNB EP 4 sockets machine, the hackbench increased about 50%, and result become stable. on other machines, hackbench increased about 2~10%. oltp increased about 30% in NHM EX box. netperf loopback also increased on SNB EP 4 sockets box.
No clear changes on other benchmarks.

Michael Wang gotten better performance on pgbench on his box with this patchset. https://lkml.org/lkml/2013/5/16/82

And Morten tested previous version with better power consumption.
http://comments.gmane.org/gmane.linux.kernel/1463371

Changlong found ltp cgroup stress testing get faster on SNB EP machine with the last patch. https://lkml.org/lkml/2013/5/23/65
---
3.10-rc1 patch1-7 patch1-8

duration=764   duration=754   duration=750
duration=764   duration=754   duration=751
duration=763   duration=755   duration=751

duration means the seconds of testing cost.
---

Jason also found java server load benefited on his 8 sockets machine with last patch. https://lkml.org/lkml/2013/5/29/673
---
When using a 3.10-rc2 tip kernel with patches 1-8, there was about a 40% improvement in performance of the workload compared to when using the vanilla 3.10-rc2 tip kernel with no patches. When using a 3.10-rc2 tip kernel with just patches 1-7, the performance improvement of the workload over the vanilla 3.10-rc2 tip kernel was about 25%.
---

We also tried to include blocked load avg in balance. but find many benchmark performance drop a lot! Seems accumulating current blocked_load_avg into cpu load isn't a good idea. Because:
1, The blocked_load_avg is decayed same as runnable load, sometime is far bigger than runnable load, that drive tasks to other idle or slight load cpu, than cause both performance and power issue. But if the blocked load is decayed too fast, it lose its effect.
2, Another issue of blocked load is that when waking up task, we can not know blocked load proportion of the task on rq. So, the blocked load is meaningless in wake affine decision.

According to above problem, we can not figure out a right way now to use blocked_load_avg in balance.

Since using runnable load avg in balance brings much benefit on performance and power. and this patch was reviewed for long time. So seems it's time to let it be clobbered in some sub tree, like tip or linux-next. Any comments?

Regards
Alex

author
#1 lore raw

PATCH v8 1/9 Revert "sched: Introduce temporary FAIR_GROUP_SCHED dependency for load-tracking"

Remove CONFIG_FAIR_GROUP_SCHED that covers the runnable info, then we can use runnable load variables.

---
 include/linux/sched.h |  7 +------
 kernel/sched/core.c   |  7 +------
 kernel/sched/fair.c   | 13 ++-----------
 kernel/sched/sched.h  | 10 ++--------
 4 files changed, 6 insertions(+), 31 deletions(-)
include/linux/sched.h +1 -6
index 178a8d9..0019bef 100644
@@ -994,12 +994,7 @@ struct sched_entity {
 	struct cfs_rq		*my_q;
 #endif
 
-/*
- * Load-tracking only depends on SMP, FAIR_GROUP_SCHED dependency below may be
- * removed when useful for applications beyond shares distribution (e.g.
- * load-balance).
- */
-#if defined(CONFIG_SMP) && defined(CONFIG_FAIR_GROUP_SCHED)
+#ifdef CONFIG_SMP
 	/* Per-entity load-tracking */
 	struct sched_avg	avg;
 #endif
kernel/sched/core.c +1 -6
index 36f85be..b9e7036 100644
@@ -1598,12 +1598,7 @@ static void __sched_fork(struct task_struct *p)
 	p->se.vruntime			= 0;
 	INIT_LIST_HEAD(&p->se.group_node);
 
-/*
- * Load-tracking only depends on SMP, FAIR_GROUP_SCHED dependency below may be
- * removed when useful for applications beyond shares distribution (e.g.
- * load-balance).
- */
-#if defined(CONFIG_SMP) && defined(CONFIG_FAIR_GROUP_SCHED)
+#ifdef CONFIG_SMP
 	p->se.avg.runnable_avg_period = 0;
 	p->se.avg.runnable_avg_sum = 0;
 #endif
kernel/sched/fair.c +2 -11
index 3ee1c2e..f404468 100644
@@ -1128,8 +1128,7 @@ static inline void update_cfs_shares(struct cfs_rq *cfs_rq)
 }
 #endif /* CONFIG_FAIR_GROUP_SCHED */
 
-/* Only depends on SMP, FAIR_GROUP_SCHED may be removed when useful in lb */
-#if defined(CONFIG_SMP) && defined(CONFIG_FAIR_GROUP_SCHED)
+#ifdef CONFIG_SMP
 /*
  * We choose a half-life close to 1 scheduling period.
  * Note: The tables below are dependent on this value.
@@ -3436,12 +3435,6 @@ unlock:
 }
 
 /*
- * Load-tracking only depends on SMP, FAIR_GROUP_SCHED dependency below may be
- * removed when useful for applications beyond shares distribution (e.g.
- * load-balance).
- */
-#ifdef CONFIG_FAIR_GROUP_SCHED
-/*
  * Called immediately before a task is migrated to a new cpu; task_cpu(p) and
  * cfs_rq_of(p) references at time of call are still valid and identify the
  * previous cpu.  However, the caller only guarantees p->pi_lock is held; no
@@ -3464,7 +3457,6 @@ migrate_task_rq_fair(struct task_struct *p, int next_cpu)
 		atomic64_add(se->avg.load_avg_contrib, &cfs_rq->removed_load);
 	}
 }
-#endif
 #endif /* CONFIG_SMP */
 
 static unsigned long
@@ -6167,9 +6159,8 @@ const struct sched_class fair_sched_class = {
 
 #ifdef CONFIG_SMP
 	.select_task_rq		= select_task_rq_fair,
-#ifdef CONFIG_FAIR_GROUP_SCHED
 	.migrate_task_rq	= migrate_task_rq_fair,
-#endif
+
 	.rq_online		= rq_online_fair,
 	.rq_offline		= rq_offline_fair,
 
kernel/sched/sched.h +2 -8
index 74ff659..d892a9f 100644
@@ -269,12 +269,6 @@ struct cfs_rq {
 #endif
 
 #ifdef CONFIG_SMP
-/*
- * Load-tracking only depends on SMP, FAIR_GROUP_SCHED dependency below may be
- * removed when useful for applications beyond shares distribution (e.g.
- * load-balance).
- */
-#ifdef CONFIG_FAIR_GROUP_SCHED
 	/*
 	 * CFS Load tracking
 	 * Under CFS, load is tracked on a per-entity basis and aggregated up.
@@ -284,9 +278,9 @@ struct cfs_rq {
 	u64 runnable_load_avg, blocked_load_avg;
 	atomic64_t decay_counter, removed_load;
 	u64 last_decay;
-#endif /* CONFIG_FAIR_GROUP_SCHED */
-/* These always depend on CONFIG_FAIR_GROUP_SCHED */
+
 #ifdef CONFIG_FAIR_GROUP_SCHED
+	/* Required to track per-cpu representation of a task_group */
 	u32 tg_runnable_contrib;
 	u64 tg_load_contrib;
 #endif /* CONFIG_FAIR_GROUP_SCHED */
Signature · 1 line
1.7.12
author
#2 lore raw

PATCH v8 2/9 sched: move few runnable tg variables into CONFIG_SMP

In reply to Alex Shi

The following 2 variables only used under CONFIG_SMP, so better to move their definiation into CONFIG_SMP too.

        atomic64_t load_avg;
        atomic_t runnable_avg;
---
 kernel/sched/sched.h | 2 ++
 1 file changed, 2 insertions(+)
kernel/sched/sched.h +2 -0
index d892a9f..24b1503 100644
@@ -149,9 +149,11 @@ struct task_group {
 	unsigned long shares;
 
 	atomic_t load_weight;
+#ifdef	CONFIG_SMP
 	atomic64_t load_avg;
 	atomic_t runnable_avg;
 #endif
+#endif
 
 #ifdef CONFIG_RT_GROUP_SCHED
 	struct sched_rt_entity **rt_se;
Signature · 1 line
1.7.12
#3 lore raw

Re: [patch v8 2/9] sched: move few runnable tg variables into CONFIG_SMP

Quoted 7 lines · Alex Shi

The following 2 variables only used under CONFIG_SMP, so better to move their definiation into CONFIG_SMP too.

        atomic64_t load_avg;
        atomic_t runnable_avg;

Looks fine; the manipulations to the structs could perhaps be collapsed to one patch.

author
#4 lore raw

Re: [patch v8 2/9] sched: move few runnable tg variables into CONFIG_SMP

Quoted 4 lines · Paul Turner
Quoted 3 lines · Alex Shi

The following 2 variables only used under CONFIG_SMP, so better to move their definiation into CONFIG_SMP too.

Quoted 8 lines
Quoted 4 lines
        atomic64_t load_avg;
        atomic_t runnable_avg;

Looks fine; the manipulations to the structs could perhaps be collapsed to one patch.

thanks for review!

Signature · 2 lines
Thanks
    Alex
author
#5 lore raw

PATCH v8 3/9 sched: set initial value of runnable avg for new forked task

In reply to Alex Shi

We need initialize the se.avg.{decay_count, load_avg_contrib} for a new forked task.
Otherwise random values of above variables cause mess when do new task enqueue:
enqueue_task_fair
enqueue_entity
enqueue_entity_load_avg

and make forking balancing imbalance since incorrect load_avg_contrib.

Further more, Morten Rasmussen notice some tasks were not launched at once after created. So Paul and Peter suggest giving a start value for new task runnable avg time same as sched_slice().

---
 kernel/sched/core.c  |  6 ++----
 kernel/sched/fair.c  | 23 +++++++++++++++++++++++
 kernel/sched/sched.h |  2 ++
 3 files changed, 27 insertions(+), 4 deletions(-)
kernel/sched/core.c +2 -4
index b9e7036..6f226c2 100644
@@ -1598,10 +1598,6 @@ static void __sched_fork(struct task_struct *p)
 	p->se.vruntime			= 0;
 	INIT_LIST_HEAD(&p->se.group_node);
 
-#ifdef CONFIG_SMP
-	p->se.avg.runnable_avg_period = 0;
-	p->se.avg.runnable_avg_sum = 0;
-#endif
 #ifdef CONFIG_SCHEDSTATS
 	memset(&p->se.statistics, 0, sizeof(p->se.statistics));
 #endif
@@ -1745,6 +1741,8 @@ void wake_up_new_task(struct task_struct *p)
 	set_task_cpu(p, select_task_rq(p, SD_BALANCE_FORK, 0));
 #endif
 
+	/* Give new task start runnable values */
+	set_task_runnable_avg(p);
 	rq = __task_rq_lock(p);
 	activate_task(rq, p, 0);
 	p->on_rq = 1;
kernel/sched/fair.c +23 -0
index f404468..1fc30b9 100644
@@ -680,6 +680,26 @@ static u64 sched_vslice(struct cfs_rq *cfs_rq, struct sched_entity *se)
 	return calc_delta_fair(sched_slice(cfs_rq, se), se);
 }
 
+#ifdef CONFIG_SMP
+static inline void __update_task_entity_contrib(struct sched_entity *se);
+
+/* Give new task start runnable values to heavy its load in infant time */
+void set_task_runnable_avg(struct task_struct *p)
+{
+	u32 slice;
+
+	p->se.avg.decay_count = 0;
+	slice = sched_slice(task_cfs_rq(p), &p->se) >> 10;
+	p->se.avg.runnable_avg_sum = slice;
+	p->se.avg.runnable_avg_period = slice;
+	__update_task_entity_contrib(&p->se);
+}
+#else
+void set_task_runnable_avg(struct task_struct *p)
+{
+}
+#endif
+
 /*
  * Update the current task's runtime statistics. Skip current tasks that
  * are not in our scheduling class.
@@ -1527,6 +1547,9 @@ static inline void enqueue_entity_load_avg(struct cfs_rq *cfs_rq,
 	 * We track migrations using entity decay_count <= 0, on a wake-up
 	 * migration we use a negative decay count to track the remote decays
 	 * accumulated while sleeping.
+	 *
+	 * When enqueue a new forked task, the se->avg.decay_count == 0, so
+	 * we bypass update_entity_load_avg(), use avg.load_avg_contrib direct.
 	 */
 	if (unlikely(se->avg.decay_count <= 0)) {
 		se->avg.last_runnable_update = rq_clock_task(rq_of(cfs_rq));
kernel/sched/sched.h +2 -0
index 24b1503..8bc66c6 100644
@@ -1058,6 +1058,8 @@ extern void init_rt_bandwidth(struct rt_bandwidth *rt_b, u64 period, u64 runtime
 
 extern void update_idle_cpu_load(struct rq *this_rq);
 
+extern void set_task_runnable_avg(struct task_struct *p);
+
 #ifdef CONFIG_PARAVIRT
 static inline u64 steal_ticks(u64 steal)
 {
Signature · 1 line
1.7.12
Reviewed-by on v8 3/9
#6 lore raw

Re: [patch v8 3/9] sched: set initial value of runnable avg for new forked task

Quoted 8 lines · Alex Shi

We need initialize the se.avg.{decay_count, load_avg_contrib} for a new forked task.
Otherwise random values of above variables cause mess when do new task enqueue:
enqueue_task_fair
enqueue_entity
enqueue_entity_load_avg

Quoted 7 lines

and make forking balancing imbalance since incorrect load_avg_contrib.

Further more, Morten Rasmussen notice some tasks were not launched at once after created. So Paul and Peter suggest giving a start value for new task runnable avg time same as sched_slice().

Quoted 84 lines
---
 kernel/sched/core.c  |  6 ++----
 kernel/sched/fair.c  | 23 +++++++++++++++++++++++
 kernel/sched/sched.h |  2 ++
 3 files changed, 27 insertions(+), 4 deletions(-)
kernel/sched/core.c +2 -4
index b9e7036..6f226c2 100644
@@ -1598,10 +1598,6 @@ static void __sched_fork(struct task_struct *p)
 	p->se.vruntime			= 0;
 	INIT_LIST_HEAD(&p->se.group_node);
 
-#ifdef CONFIG_SMP
-	p->se.avg.runnable_avg_period = 0;
-	p->se.avg.runnable_avg_sum = 0;
-#endif
 #ifdef CONFIG_SCHEDSTATS
 	memset(&p->se.statistics, 0, sizeof(p->se.statistics));
 #endif
@@ -1745,6 +1741,8 @@ void wake_up_new_task(struct task_struct *p)
 	set_task_cpu(p, select_task_rq(p, SD_BALANCE_FORK, 0));
 #endif
 
+	/* Give new task start runnable values */
+	set_task_runnable_avg(p);
 	rq = __task_rq_lock(p);
 	activate_task(rq, p, 0);
 	p->on_rq = 1;
kernel/sched/fair.c +23 -0
index f404468..1fc30b9 100644
@@ -680,6 +680,26 @@ static u64 sched_vslice(struct cfs_rq *cfs_rq, struct sched_entity *se)
 	return calc_delta_fair(sched_slice(cfs_rq, se), se);
 }
 
+#ifdef CONFIG_SMP
+static inline void __update_task_entity_contrib(struct sched_entity *se);
+
+/* Give new task start runnable values to heavy its load in infant time */
+void set_task_runnable_avg(struct task_struct *p)
+{
+	u32 slice;
+
+	p->se.avg.decay_count = 0;
+	slice = sched_slice(task_cfs_rq(p), &p->se) >> 10;
+	p->se.avg.runnable_avg_sum = slice;
+	p->se.avg.runnable_avg_period = slice;
+	__update_task_entity_contrib(&p->se);
+}
+#else
+void set_task_runnable_avg(struct task_struct *p)
+{
+}
+#endif
+
 /*
  * Update the current task's runtime statistics. Skip current tasks that
  * are not in our scheduling class.
@@ -1527,6 +1547,9 @@ static inline void enqueue_entity_load_avg(struct cfs_rq *cfs_rq,
 	 * We track migrations using entity decay_count <= 0, on a wake-up
 	 * migration we use a negative decay count to track the remote decays
 	 * accumulated while sleeping.
+	 *
+	 * When enqueue a new forked task, the se->avg.decay_count == 0, so
+	 * we bypass update_entity_load_avg(), use avg.load_avg_contrib direct.
 	 */
 	if (unlikely(se->avg.decay_count <= 0)) {
 		se->avg.last_runnable_update = rq_clock_task(rq_of(cfs_rq));
kernel/sched/sched.h +2 -0
index 24b1503..8bc66c6 100644
@@ -1058,6 +1058,8 @@ extern void init_rt_bandwidth(struct rt_bandwidth *rt_b, u64 period, u64 runtime
 
 extern void update_idle_cpu_load(struct rq *this_rq);
 
+extern void set_task_runnable_avg(struct task_struct *p);
+
 #ifdef CONFIG_PARAVIRT
 static inline u64 steal_ticks(u64 steal)
 {
#7 lore raw

Re: [patch v8 3/9] sched: set initial value of runnable avg for new forked task

In reply to Alex Shi

Hi Alex,

Quoted 8 lines · Alex Shi

We need initialize the se.avg.{decay_count, load_avg_contrib} for a new forked task.
Otherwise random values of above variables cause mess when do new task enqueue:
enqueue_task_fair
enqueue_entity
enqueue_entity_load_avg

Quoted 5 lines

and make forking balancing imbalance since incorrect load_avg_contrib.

Further more, Morten Rasmussen notice some tasks were not launched at once after created. So Paul and Peter suggest giving a start value for new task runnable avg time same as sched_slice().

I am confused at this comment, how set slice to runnable avg would change the behavior of "some tasks were not launched at once after created"?

IMHO, I could only tell that for the new forked task, it could be run if current task already be set as need_resched, and preempt_schedule or preempt_schedule_irq
is called.

Since the set slice to avg behavior would not affect this task's vruntime, and hence cannot make current running task be need_sched, if previously it cannot.

Could you help correct if I am wrong at somewhere? ....

Thanks,
Lei

Quoted 94 lines
---
 kernel/sched/core.c  |  6 ++----
 kernel/sched/fair.c  | 23 +++++++++++++++++++++++
 kernel/sched/sched.h |  2 ++
 3 files changed, 27 insertions(+), 4 deletions(-)
kernel/sched/core.c +2 -4
index b9e7036..6f226c2 100644
@@ -1598,10 +1598,6 @@ static void __sched_fork(struct task_struct *p)
        p->se.vruntime                  = 0;
        INIT_LIST_HEAD(&p->se.group_node);
 
-#ifdef CONFIG_SMP
-       p->se.avg.runnable_avg_period = 0;
-       p->se.avg.runnable_avg_sum = 0;
-#endif
 #ifdef CONFIG_SCHEDSTATS
        memset(&p->se.statistics, 0, sizeof(p->se.statistics));
 #endif
@@ -1745,6 +1741,8 @@ void wake_up_new_task(struct task_struct *p)
        set_task_cpu(p, select_task_rq(p, SD_BALANCE_FORK, 0));
 #endif
 
+       /* Give new task start runnable values */
+       set_task_runnable_avg(p);
        rq = __task_rq_lock(p);
        activate_task(rq, p, 0);
        p->on_rq = 1;
kernel/sched/fair.c +23 -0
index f404468..1fc30b9 100644
@@ -680,6 +680,26 @@ static u64 sched_vslice(struct cfs_rq *cfs_rq, struct sched_entity *se)
        return calc_delta_fair(sched_slice(cfs_rq, se), se);
 }
 
+#ifdef CONFIG_SMP
+static inline void __update_task_entity_contrib(struct sched_entity *se);
+
+/* Give new task start runnable values to heavy its load in infant time */
+void set_task_runnable_avg(struct task_struct *p)
+{
+       u32 slice;
+
+       p->se.avg.decay_count = 0;
+       slice = sched_slice(task_cfs_rq(p), &p->se) >> 10;
+       p->se.avg.runnable_avg_sum = slice;
+       p->se.avg.runnable_avg_period = slice;
+       __update_task_entity_contrib(&p->se);
+}
+#else
+void set_task_runnable_avg(struct task_struct *p)
+{
+}
+#endif
+
 /*
  * Update the current task's runtime statistics. Skip current tasks that
  * are not in our scheduling class.
@@ -1527,6 +1547,9 @@ static inline void enqueue_entity_load_avg(struct cfs_rq *cfs_rq,
         * We track migrations using entity decay_count <= 0, on a wake-up
         * migration we use a negative decay count to track the remote decays
         * accumulated while sleeping.
+        *
+        * When enqueue a new forked task, the se->avg.decay_count == 0, so
+        * we bypass update_entity_load_avg(), use avg.load_avg_contrib direct.
         */
        if (unlikely(se->avg.decay_count <= 0)) {
                se->avg.last_runnable_update = rq_clock_task(rq_of(cfs_rq));
kernel/sched/sched.h +2 -1
index 24b1503..8bc66c6 100644
@@ -1058,6 +1058,8 @@ extern void init_rt_bandwidth(struct rt_bandwidth *rt_b, u64 period, u64 runtime
 
 extern void update_idle_cpu_load(struct rq *this_rq);
 
+extern void set_task_runnable_avg(struct task_struct *p);
+
 #ifdef CONFIG_PARAVIRT
 static inline u64 steal_ticks(u64 steal)
 {
--

1.7.12

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

author
#8 lore raw

Re: [patch v8 3/9] sched: set initial value of runnable avg for new forked task

Quoted 10 lines · Lei Wen
Quoted 8 lines
Quoted 8 lines

enqueue_entity
enqueue_entity_load_avg

and make forking balancing imbalance since incorrect load_avg_contrib.

Further more, Morten Rasmussen notice some tasks were not launched at once after created. So Paul and Peter suggest giving a start value for new task runnable avg time same as sched_slice().

I am confused at this comment, how set slice to runnable avg would change the behavior of "some tasks were not launched at once after created"?

I also don't know the details on Morten's machine. but just guess, there are much tasks on in the run queue. the minimum load avg make the new task wait its time...

Quoted 13 lines

IMHO, I could only tell that for the new forked task, it could be run if current task already be set as need_resched, and preempt_schedule or preempt_schedule_irq
is called.

Since the set slice to avg behavior would not affect this task's vruntime, and hence cannot make current running task be need_sched, if previously it cannot.

Could you help correct if I am wrong at somewhere? ....

Thanks,

Signature · 2 lines
Thanks
    Alex
#9 lore raw

Re: [patch v8 3/9] sched: set initial value of runnable avg for new forked task

Quoted 9 lines · Alex Shi
Quoted 8 lines · Lei Wen
Quoted 8 lines
Quoted 8 lines

enqueue_entity
enqueue_entity_load_avg

and make forking balancing imbalance since incorrect load_avg_contrib.

Further more, Morten Rasmussen notice some tasks were not launched at once after created. So Paul and Peter suggest giving a start value for new task runnable avg time same as sched_slice().

Quoted 6 lines
Quoted 2 lines

I am confused at this comment, how set slice to runnable avg would change the behavior of "some tasks were not launched at once after created"?

I also don't know the details on Morten's machine. but just guess, there are much tasks on in the run queue. the minimum load avg make the new task wait its time...

Is there some possibility that since task structure is allocated without being set to 0, and it cause the imbalance between runqueues. Then the new forked is migrated to other cpus, so that it cause its execution being delayed?

It is better for Morten to give us more details here. :)

Thanks,
Lei

Quoted 18 lines
Quoted 6 lines

IMHO, I could only tell that for the new forked task, it could be run if current task already be set as need_resched, and preempt_schedule or preempt_schedule_irq
is called.

Quoted 7 lines

Since the set slice to avg behavior would not affect this task's vruntime, and hence cannot make current running task be need_sched, if previously it cannot.

Could you help correct if I am wrong at somewhere? ....

Thanks,

--
Thanks
Alex

author
#10 lore raw

Re: [patch v8 3/9] sched: set initial value of runnable avg for new forked task

Quoted 8 lines · Lei Wen
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

and make forking balancing imbalance since incorrect load_avg_contrib.

Further more, Morten Rasmussen notice some tasks were not launched at once after created. So Paul and Peter suggest giving a start value for new task runnable avg time same as sched_slice().

I am confused at this comment, how set slice to runnable avg would change the behavior of "some tasks were not launched at once after created"?

Quoted 6 lines
Quoted 3 lines
Quoted 3 lines

I also don't know the details on Morten's machine. but just guess, there are much tasks on in the run queue. the minimum load avg make the new task wait its time...

Is there some possibility that since task structure is allocated without being set to 0, and it cause the imbalance between runqueues. Then the new forked is migrated to other cpus, so that it cause its execution being delayed?

Is there sth weird happens?
The task should be running a while before migration. and periodic load balance also need some time to happen. So generally, it has no such worries.

Signature · 2 lines
Thanks
    Alex
#11 lore raw

Re: [patch v8 3/9] sched: set initial value of runnable avg for new forked task

In reply to Lei Wen
Quoted 17 lines · Lei Wen
Quoted 9 lines · Alex Shi
Quoted 8 lines · Lei Wen
Quoted 8 lines
Quoted 8 lines

enqueue_entity
enqueue_entity_load_avg

and make forking balancing imbalance since incorrect load_avg_contrib.

Further more, Morten Rasmussen notice some tasks were not launched at once after created. So Paul and Peter suggest giving a start value for new task runnable avg time same as sched_slice().

Quoted 6 lines
Quoted 2 lines

I am confused at this comment, how set slice to runnable avg would change the behavior of "some tasks were not launched at once after created"?

I also don't know the details on Morten's machine. but just guess, there are much tasks on in the run queue. the minimum load avg make the new task wait its time...

Quoted 6 lines

Is there some possibility that since task structure is allocated without being set to 0, and it cause the imbalance between runqueues. Then the new forked is migrated to other cpus, so that it cause its execution being delayed?

It is better for Morten to give us more details here. :)

I think Peter's reply pretty much covers it. The problem is when a task is not running (other task has lower vruntime or blocked for other reasons) shortly after the task was created. The runnable_avg_period is very small, so the load_contrib is very sensitive.

Say if a task runs for 1 ms then is blocked for 1 ms and then runs again, the load_contrib will go from 100% to 50% instantly and then ramp back up again. So the task load may be quite different from the true load of the task depending on when you calculate the load_contrib.

Preloading runnable_avg_period should make the load_contrib a little less sensitive to this behaviour.

Morten

Quoted 22 lines

Thanks,
Lei

Quoted 18 lines
Quoted 6 lines

IMHO, I could only tell that for the new forked task, it could be run if current task already be set as need_resched, and preempt_schedule or preempt_schedule_irq
is called.

Quoted 7 lines

Since the set slice to avg behavior would not affect this task's vruntime, and hence cannot make current running task be need_sched, if previously it cannot.

Could you help correct if I am wrong at somewhere? ....

Thanks,

--
Thanks
Alex

#12 lore raw

Re: [patch v8 3/9] sched: set initial value of runnable avg for new forked task

Morten,

Quoted 30 lines · Morten Rasmussen
Quoted 17 lines · Lei Wen
Quoted 9 lines · Alex Shi
Quoted 8 lines · Lei Wen
Quoted 8 lines
Quoted 8 lines

enqueue_entity
enqueue_entity_load_avg

and make forking balancing imbalance since incorrect load_avg_contrib.

Further more, Morten Rasmussen notice some tasks were not launched at once after created. So Paul and Peter suggest giving a start value for new task runnable avg time same as sched_slice().

Quoted 6 lines
Quoted 2 lines

I am confused at this comment, how set slice to runnable avg would change the behavior of "some tasks were not launched at once after created"?

I also don't know the details on Morten's machine. but just guess, there are much tasks on in the run queue. the minimum load avg make the new task wait its time...

Quoted 6 lines

Is there some possibility that since task structure is allocated without being set to 0, and it cause the imbalance between runqueues. Then the new forked is migrated to other cpus, so that it cause its execution being delayed?

It is better for Morten to give us more details here. :)

I think Peter's reply pretty much covers it. The problem is when a task is not running (other task has lower vruntime or blocked for other reasons) shortly after the task was created. The runnable_avg_period is very small, so the load_contrib is very sensitive.

Quoted 7 lines

Say if a task runs for 1 ms then is blocked for 1 ms and then runs again, the load_contrib will go from 100% to 50% instantly and then ramp back up again. So the task load may be quite different from the true load of the task depending on when you calculate the load_contrib.

Preloading runnable_avg_period should make the load_contrib a little less sensitive to this behaviour.

Thanks for detailed explanation!
Now I could understand the preloading value prevent entity's load change quickly at its beginning.
But I cannot see why this could explain "some tasks were not launched at once after created"

Or should I understand the previous words as:
"If without preloading value, the new created task's contributed load may vary too quickly at its beginning. Like if a task runs for 1 ms then is blocked for 1 ms
and then runs again, the load_contrib will go from 100% to 50% instantly."

Thanks,
Lei

Quoted 26 lines

Morten

Quoted 22 lines

Thanks,
Lei

Quoted 18 lines
Quoted 6 lines

IMHO, I could only tell that for the new forked task, it could be run if current task already be set as need_resched, and preempt_schedule or preempt_schedule_irq
is called.

Quoted 7 lines

Since the set slice to avg behavior would not affect this task's vruntime, and hence cannot make current running task be need_sched, if previously it cannot.

Could you help correct if I am wrong at somewhere? ....

Thanks,

--
Thanks
Alex

maintainer
#13 lore raw

Re: [patch v8 3/9] sched: set initial value of runnable avg for new forked task

In reply to Lei Wen
Quoted 20 lines · Lei Wen

Hi Alex,

Quoted 8 lines · Alex Shi

We need initialize the se.avg.{decay_count, load_avg_contrib} for a new forked task.
Otherwise random values of above variables cause mess when do new task enqueue:
enqueue_task_fair
enqueue_entity
enqueue_entity_load_avg

Quoted 5 lines

and make forking balancing imbalance since incorrect load_avg_contrib.

Further more, Morten Rasmussen notice some tasks were not launched at once after created. So Paul and Peter suggest giving a start value for new task runnable avg time same as sched_slice().

I am confused at this comment, how set slice to runnable avg would change the behavior of "some tasks were not launched at once after created"?

Quoted 8 lines

IMHO, I could only tell that for the new forked task, it could be run if current task already be set as need_resched, and preempt_schedule or preempt_schedule_irq
is called.

Since the set slice to avg behavior would not affect this task's vruntime, and hence cannot make current running task be need_sched, if previously it cannot.

So the 'problem' is that our running avg is a 'floating' average; ie. it decays with time. Now we have to guess about the future of our newly spawned task -- something that is nigh impossible seeing these CPU vendors keep refusing to implement the crystal ball instruction.

So there's two asymptotic cases we want to deal well with; 1) the case where the newly spawned program will be 'nearly' idle for its lifetime; and 2) the case where its cpu-bound.

Since we have to guess, we'll go for worst case and assume its cpu-bound; now we don't want to make the avg so heavy adjusting to the near-idle case takes forever. We want to be able to quickly adjust and lower our running avg.

Now we also don't want to make our avg too light, such that it gets decremented just for the new task not having had a chance to run yet -- even if when it would run, it would be more cpu-bound than not.

So what we do is we make the initial avg of the same duration as that we guess it takes to run each task on the system at least once -- aka sched_slice().

Of course we can defeat this with wakeup/fork bombs, but in the 'normal' case it should be good enough.

Does that make sense?

#14 lore raw

Re: [patch v8 3/9] sched: set initial value of runnable avg for new forked task

Hi Peter,

Quoted 31 lines · Peter Zijlstra
Quoted 20 lines · Lei Wen

Hi Alex,

Quoted 8 lines · Alex Shi

We need initialize the se.avg.{decay_count, load_avg_contrib} for a new forked task.
Otherwise random values of above variables cause mess when do new task enqueue:
enqueue_task_fair
enqueue_entity
enqueue_entity_load_avg

Quoted 5 lines

and make forking balancing imbalance since incorrect load_avg_contrib.

Further more, Morten Rasmussen notice some tasks were not launched at once after created. So Paul and Peter suggest giving a start value for new task runnable avg time same as sched_slice().

I am confused at this comment, how set slice to runnable avg would change the behavior of "some tasks were not launched at once after created"?

Quoted 8 lines

IMHO, I could only tell that for the new forked task, it could be run if current task already be set as need_resched, and preempt_schedule or preempt_schedule_irq
is called.

Since the set slice to avg behavior would not affect this task's vruntime, and hence cannot make current running task be need_sched, if previously it cannot.

Quoted 4 lines

So the 'problem' is that our running avg is a 'floating' average; ie. it decays with time. Now we have to guess about the future of our newly spawned task -- something that is nigh impossible seeing these CPU vendors keep refusing to implement the crystal ball instruction.

I am curious at this "crystal ball instruction" saying. :) Could it be real? I mean what kind of hw mechanism could achieve such magic power? What I see, for silicon vendor they could provide more monitor unit, but to precise predict the sw's behavior, I don't think hw also this kind of power...

Quoted 18 lines

So there's two asymptotic cases we want to deal well with; 1) the case where the newly spawned program will be 'nearly' idle for its lifetime; and 2) the case where its cpu-bound.

Since we have to guess, we'll go for worst case and assume its cpu-bound; now we don't want to make the avg so heavy adjusting to the near-idle case takes forever. We want to be able to quickly adjust and lower our running avg.

Now we also don't want to make our avg too light, such that it gets decremented just for the new task not having had a chance to run yet -- even if when it would run, it would be more cpu-bound than not.

So what we do is we make the initial avg of the same duration as that we guess it takes to run each task on the system at least once -- aka sched_slice().

Quoted 5 lines

Of course we can defeat this with wakeup/fork bombs, but in the 'normal' case it should be good enough.

Does that make sense?

Thanks for your detailed explanation. Very useful indeed! :)

BTW, I have no question for the patch itself, but just confuse at the patch's comment
"some tasks were not launched at once after created".

Thanks,
Lei

maintainer
#15 lore raw

Re: [patch v8 3/9] sched: set initial value of runnable avg for new forked task

Quoted 7 lines · Lei Wen

Hi Peter,

Quoted 4 lines

So the 'problem' is that our running avg is a 'floating' average; ie. it decays with time. Now we have to guess about the future of our newly spawned task -- something that is nigh impossible seeing these CPU vendors keep refusing to implement the crystal ball instruction.

Quoted 5 lines

I am curious at this "crystal ball instruction" saying. :) Could it be real? I mean what kind of hw mechanism could achieve such magic power? What I see, for silicon vendor they could provide more monitor unit, but to precise predict the sw's behavior, I don't think hw also this kind of power...

There's indeed no known mechanism for this crystal ball instruction to really work with. Its just something I often wish for ;-) It would make life so much easier - although I have no experience with handling the resulting paradoxes, so who knows :-)

Quoted 21 lines
Quoted 21 lines

So there's two asymptotic cases we want to deal well with; 1) the case where the newly spawned program will be 'nearly' idle for its lifetime; and 2) the case where its cpu-bound.

Since we have to guess, we'll go for worst case and assume its cpu-bound; now we don't want to make the avg so heavy adjusting to the near-idle case takes forever. We want to be able to quickly adjust and lower our running avg.

Now we also don't want to make our avg too light, such that it gets decremented just for the new task not having had a chance to run yet -- even if when it would run, it would be more cpu-bound than not.

So what we do is we make the initial avg of the same duration as that we guess it takes to run each task on the system at least once -- aka sched_slice().

Of course we can defeat this with wakeup/fork bombs, but in the 'normal' case it should be good enough.

Quoted 7 lines
Quoted 1 line

Does that make sense?

Thanks for your detailed explanation. Very useful indeed! :)

BTW, I have no question for the patch itself, but just confuse at the patch's comment
"some tasks were not launched at once after created".

Right, I might edit that on applying to clarify things.

#16 lore raw

Re: [patch v8 3/9] sched: set initial value of runnable avg for new forked task

In reply to Alex Shi

Minor comments; looks good otherwise.

Quoted 37 lines · Alex Shi

We need initialize the se.avg.{decay_count, load_avg_contrib} for a new forked task.
Otherwise random values of above variables cause mess when do new task enqueue:
enqueue_task_fair
enqueue_entity
enqueue_entity_load_avg

and make forking balancing imbalance since incorrect load_avg_contrib.

Further more, Morten Rasmussen notice some tasks were not launched at once after created. So Paul and Peter suggest giving a start value for new task runnable avg time same as sched_slice().

---
 kernel/sched/core.c  |  6 ++----
 kernel/sched/fair.c  | 23 +++++++++++++++++++++++
 kernel/sched/sched.h |  2 ++
 3 files changed, 27 insertions(+), 4 deletions(-)
kernel/sched/core.c +0 -4
index b9e7036..6f226c2 100644
@@ -1598,10 +1598,6 @@ static void __sched_fork(struct task_struct *p)
        p->se.vruntime                  = 0;
        INIT_LIST_HEAD(&p->se.group_node);
 
-#ifdef CONFIG_SMP
-       p->se.avg.runnable_avg_period = 0;
-       p->se.avg.runnable_avg_sum = 0;
-#endif
 #ifdef CONFIG_SCHEDSTATS
        memset(&p->se.statistics, 0, sizeof(p->se.statistics));
 #endif

kernel/sched/core.c void wake_up_new_task(struct task_struct *p) · line 1744

Quoted 4 lines
kernel/sched/core.c +1 -0
@@ -1745,6 +1741,8 @@ void wake_up_new_task(struct task_struct *p)
        set_task_cpu(p, select_task_rq(p, SD_BALANCE_FORK, 0));
 #endif
 
+       /* Give new task start runnable values */

/* Initialize new task's runnable average */

Quoted 9 lines
kernel/sched/core.c +1 -0
+       set_task_runnable_avg(p);
        rq = __task_rq_lock(p);
        activate_task(rq, p, 0);
        p->on_rq = 1;
kernel/sched/fair.c +0 -0
index f404468..1fc30b9 100644

kernel/sched/fair.c static u64 sched_vslice(struct cfs_rq *cfs_rq, struct sched_entity *se) · line 687

Quoted 8 lines
kernel/sched/fair.c +5 -0
@@ -680,6 +680,26 @@ static u64 sched_vslice(struct cfs_rq *cfs_rq, struct sched_entity *se)
        return calc_delta_fair(sched_slice(cfs_rq, se), se);
 }
 
+#ifdef CONFIG_SMP
+static inline void __update_task_entity_contrib(struct sched_entity *se);
+
+/* Give new task start runnable values to heavy its load in infant time */
+void set_task_runnable_avg(struct task_struct *p)

init_task_runnable_average

Quoted 19 lines
kernel/sched/fair.c +15 -0
+{
+       u32 slice;
+
+       p->se.avg.decay_count = 0;
+       slice = sched_slice(task_cfs_rq(p), &p->se) >> 10;
+       p->se.avg.runnable_avg_sum = slice;
+       p->se.avg.runnable_avg_period = slice;
+       __update_task_entity_contrib(&p->se);
+}
+#else
+void set_task_runnable_avg(struct task_struct *p)
+{
+}
+#endif
+
 /*
  * Update the current task's runtime statistics. Skip current tasks that
  * are not in our scheduling class.

kernel/sched/fair.c static inline void enqueue_entity_load_avg(struct cfs_rq *cfs_rq, · line 1552

Quoted 6 lines
kernel/sched/fair.c +3 -0
@@ -1527,6 +1547,9 @@ static inline void enqueue_entity_load_avg(struct cfs_rq *cfs_rq,
         * We track migrations using entity decay_count <= 0, on a wake-up
         * migration we use a negative decay count to track the remote decays
         * accumulated while sleeping.
+        *
+        * When enqueue a new forked task, the se->avg.decay_count == 0, so
+        * we bypass update_entity_load_avg(), use avg.load_avg_contrib direct.

Newly forked tasks are enqueued with se->avg.decay_count == 0, they are seen by enqueue_entity_load_avg() as a migration with an already constructed load_avg_contrib.

Quoted 19 lines
kernel/sched/fair.c +0 -0
         */
        if (unlikely(se->avg.decay_count <= 0)) {
                se->avg.last_runnable_update = rq_clock_task(rq_of(cfs_rq));
kernel/sched/sched.h +2 -1
index 24b1503..8bc66c6 100644
@@ -1058,6 +1058,8 @@ extern void init_rt_bandwidth(struct rt_bandwidth *rt_b, u64 period, u64 runtime
 
 extern void update_idle_cpu_load(struct rq *this_rq);
 
+extern void set_task_runnable_avg(struct task_struct *p);
+
 #ifdef CONFIG_PARAVIRT
 static inline u64 steal_ticks(u64 steal)
 {
--

1.7.12

author Reviewed-by Gu Zheng
#17 lore raw

Re: [patch v8 3/9] sched: set initial value of runnable avg for new forked task

Quoted 3 lines · Paul Turner

Minor comments; looks good otherwise.

thanks a lot Paul. the patch with your input updated here:

BTW, would you like to give some comments on the last patch of this patchset?

---

Quoted 1 line

From ed35080d0bae803d68f84a3e683d34a356a5a5de Mon Sep 17 00:00:00 2001

From: Alex Shi <[email protected]>
Date: Tue, 14 May 2013 09:41:09 +0800
Subject: [PATCH 3/8] sched: set initial value of runnable avg for new forked
task

We need initialize the se.avg.{decay_count, load_avg_contrib} for a new forked task.
Otherwise random values of above variables cause mess when do new task enqueue:
enqueue_task_fair
enqueue_entity
enqueue_entity_load_avg

and make forking balancing imbalance since incorrect load_avg_contrib.

Further more, Morten Rasmussen notice some tasks were not launched at once after created. So Paul and Peter suggest giving a start value for new task runnable avg time same as sched_slice().

---
 kernel/sched/core.c  |    6 ++----
 kernel/sched/fair.c  |   24 ++++++++++++++++++++++++
 kernel/sched/sched.h |    2 ++
 3 files changed, 28 insertions(+), 4 deletions(-)
kernel/sched/core.c +2 -4
index b9e7036..c78a9e2 100644
@@ -1598,10 +1598,6 @@ static void __sched_fork(struct task_struct *p)
 	p->se.vruntime			= 0;
 	INIT_LIST_HEAD(&p->se.group_node);
 
-#ifdef CONFIG_SMP
-	p->se.avg.runnable_avg_period = 0;
-	p->se.avg.runnable_avg_sum = 0;
-#endif
 #ifdef CONFIG_SCHEDSTATS
 	memset(&p->se.statistics, 0, sizeof(p->se.statistics));
 #endif
@@ -1745,6 +1741,8 @@ void wake_up_new_task(struct task_struct *p)
 	set_task_cpu(p, select_task_rq(p, SD_BALANCE_FORK, 0));
 #endif
 
+	/* Initialize new task's runnable average */
+	init_task_runnable_average(p);
 	rq = __task_rq_lock(p);
 	activate_task(rq, p, 0);
 	p->on_rq = 1;
kernel/sched/fair.c +24 -0
index f404468..df5b8a9 100644
@@ -680,6 +680,26 @@ static u64 sched_vslice(struct cfs_rq *cfs_rq, struct sched_entity *se)
 	return calc_delta_fair(sched_slice(cfs_rq, se), se);
 }
 
+#ifdef CONFIG_SMP
+static inline void __update_task_entity_contrib(struct sched_entity *se);
+
+/* Give new task start runnable values to heavy its load in infant time */
+void init_task_runnable_average(struct task_struct *p)
+{
+	u32 slice;
+
+	p->se.avg.decay_count = 0;
+	slice = sched_slice(task_cfs_rq(p), &p->se) >> 10;
+	p->se.avg.runnable_avg_sum = slice;
+	p->se.avg.runnable_avg_period = slice;
+	__update_task_entity_contrib(&p->se);
+}
+#else
+void init_task_runnable_average(struct task_struct *p)
+{
+}
+#endif
+
 /*
  * Update the current task's runtime statistics. Skip current tasks that
  * are not in our scheduling class.
@@ -1527,6 +1547,10 @@ static inline void enqueue_entity_load_avg(struct cfs_rq *cfs_rq,
 	 * We track migrations using entity decay_count <= 0, on a wake-up
 	 * migration we use a negative decay count to track the remote decays
 	 * accumulated while sleeping.
+	 *
+	 * Newly forked tasks are enqueued with se->avg.decay_count == 0, they
+	 * are seen by enqueue_entity_load_avg() as a migration with an already
+	 * constructed load_avg_contrib.
 	 */
 	if (unlikely(se->avg.decay_count <= 0)) {
 		se->avg.last_runnable_update = rq_clock_task(rq_of(cfs_rq));
kernel/sched/sched.h +2 -0
index 24b1503..0684c26 100644
@@ -1058,6 +1058,8 @@ extern void init_rt_bandwidth(struct rt_bandwidth *rt_b, u64 period, u64 runtime
 
 extern void update_idle_cpu_load(struct rq *this_rq);
 
+extern void init_task_runnable_average(struct task_struct *p);
+
 #ifdef CONFIG_PARAVIRT
 static inline u64 steal_ticks(u64 steal)
 {
Signature · 1 line
1.7.5.4
maintainer
#18 lore raw

Re: [patch v8 3/9] sched: set initial value of runnable avg for new forked task

Quoted 24 lines · Alex Shi
Quoted 3 lines · Paul Turner

Minor comments; looks good otherwise.

thanks a lot Paul. the patch with your input updated here:

BTW, would you like to give some comments on the last patch of this patchset?

---
From ed35080d0bae803d68f84a3e683d34a356a5a5de Mon Sep 17 00:00:00 2001
From: Alex Shi <[email protected]>
Date: Tue, 14 May 2013 09:41:09 +0800
Subject: [PATCH 3/8] sched: set initial value of runnable avg for new forked
task

We need initialize the se.avg.{decay_count, load_avg_contrib} for a new forked task.
Otherwise random values of above variables cause mess when do new task enqueue:
enqueue_task_fair
enqueue_entity
enqueue_entity_load_avg

Quoted 8 lines

and make forking balancing imbalance since incorrect load_avg_contrib.

Further more, Morten Rasmussen notice some tasks were not launched at once after created. So Paul and Peter suggest giving a start value for new task runnable avg time same as sched_slice().

Should you all go read: Documentation/SubmittingPatches , or am I somehow confused on the SoB rules?

#19 lore raw

Re: [patch v8 3/9] sched: set initial value of runnable avg for new forked task

I actually did read it before, and still wasn't sure of the right tag to use.

"13) When to use Acked-by: and Cc:

The Signed-off-by: tag indicates that the signer was involved in the development of the patch, or that he/she was in the patch's delivery path.

If a person was not directly involved in the preparation or handling of a patch but wishes to signify and record their approval of it then they can arrange to have an Acked-by: line added to the patch's changelog."
https://www.kernel.org/doc/Documentation/SubmittingPatches

Acked-By seemed to fail the direct involvement test.
The definition of "delivery path" is not clear; is this strictly by inputs to Linus' tree or recipients of the original patch?

Is Reviewed-By always more appropriate here?

Quoted 36 lines · Peter Zijlstra
Quoted 24 lines · Alex Shi
Quoted 3 lines · Paul Turner

Minor comments; looks good otherwise.

thanks a lot Paul. the patch with your input updated here:

BTW, would you like to give some comments on the last patch of this patchset?

---
From ed35080d0bae803d68f84a3e683d34a356a5a5de Mon Sep 17 00:00:00 2001
From: Alex Shi <[email protected]>
Date: Tue, 14 May 2013 09:41:09 +0800
Subject: [PATCH 3/8] sched: set initial value of runnable avg for new forked
task

We need initialize the se.avg.{decay_count, load_avg_contrib} for a new forked task.
Otherwise random values of above variables cause mess when do new task enqueue:
enqueue_task_fair
enqueue_entity
enqueue_entity_load_avg

Quoted 8 lines

and make forking balancing imbalance since incorrect load_avg_contrib.

Further more, Morten Rasmussen notice some tasks were not launched at once after created. So Paul and Peter suggest giving a start value for new task runnable avg time same as sched_slice().

Should you all go read: Documentation/SubmittingPatches , or am I somehow confused on the SoB rules?

maintainer
#20 lore raw

Re: [patch v8 3/9] sched: set initial value of runnable avg for new forked task

Quoted 7 lines · Paul Turner

I actually did read it before, and still wasn't sure of the right tag to use.

"13) When to use Acked-by: and Cc:

The Signed-off-by: tag indicates that the signer was involved in the development of the patch, or that he/she was in the patch's delivery path.

Quoted 8 lines

If a person was not directly involved in the preparation or handling of a patch but wishes to signify and record their approval of it then they can arrange to have an Acked-by: line added to the patch's changelog."
https://www.kernel.org/doc/Documentation/SubmittingPatches

Acked-By seemed to fail the direct involvement test.
The definition of "delivery path" is not clear; is this strictly by inputs to Linus' tree or recipients of the original patch?

The way I interpret the delivery path is when its part of a bigger grouping. The author is always the first SOB, if the patch series isn't by the same author then he who compiles the aggregate work adds his SOB.

Similar for (sub)maintainers passing it onwards towards Linus. I compile batches of patch series and feed them to Ingo, Ingo stuffs the lot into -tip and feeds them to Linus.

So supposing you wrote a patch, gave it to Alex who composed the bigger series, which I picked up and handed to Ingo we'd get something like:

SoB: PJT
SoB: Alex
SoB: PeterZ
SoB: Mingo

and then when Linus pulls the lot he could add his SOB too, although looking at git history he typically doesn't add his sob to each and everything he pulls.

But yes, I see where the confusion stems from.

Quoted 1 line

Is Reviewed-By always more appropriate here?

Yes, or even Acked would work. The difference between reviewed and acked is the level of involvement. An ack is for something you glanced over and generally agree with, a review is for something you looked at in minute detail.

Then there's some people (and I tend to lean towards this) who can't be bothered with that distinction too much and simply always use ack.

author
#21 lore raw

Re: [patch v8 3/9] sched: set initial value of runnable avg for new forked task

In reply to Peter Zijlstra
Quoted 11 lines · Peter Zijlstra
Quoted 9 lines
Quoted 9 lines

and make forking balancing imbalance since incorrect load_avg_contrib.

Further more, Morten Rasmussen notice some tasks were not launched at once after created. So Paul and Peter suggest giving a start value for new task runnable avg time same as sched_slice().

Should you all go read: Documentation/SubmittingPatches , or am I somehow confused on the SoB rules?

has this should been right, if Paul had handed in the modified patch as he suggested? :)

Sorry for stupid, I still don't know what's SoB rule?

Signature · 2 lines
Thanks
    Alex
maintainer
#22 lore raw

Re: [patch v8 3/9] sched: set initial value of runnable avg for new forked task

Quoted 10 lines · Alex Shi
Quoted 9 lines · Peter Zijlstra
Quoted 9 lines
Quoted 9 lines

and make forking balancing imbalance since incorrect load_avg_contrib.

Further more, Morten Rasmussen notice some tasks were not launched at once after created. So Paul and Peter suggest giving a start value for new task runnable avg time same as sched_slice().

Quoted 7 lines
Quoted 2 lines

Should you all go read: Documentation/SubmittingPatches , or am I somehow confused on the SoB rules?

has this should been right, if Paul had handed in the modified patch as he suggested? :)

Sorry for stupid, I still don't know what's SoB rule?

Right, so it depends on who actually wrote the patch; the only case that's really hard is when a patch is fully co-authored -- agile dev nonsense like, 4 hands 1 keyboard situation or so.

Typically there's 1 somebody who did most work on a particular patch; that someone would be Author/From and have first SoB.

If thereafter the patch becomes part of an aggregate work; he who compiles can add another SoB; possibly with an extra [] line describing 'smallish' changes that were needed to the initial patch to make it fit the aggregate.

Example:

From: PJT

foo patch implements foo because bar; note the fubar detail.

SoB: PJT
[alex@intel: changed ponies into horses to make it fit]
SoB: Alex

The other case is where a 'simple' modification of the initial patch simply won't do; you need to change the core idea of the patch or similar. In this case I've seen things like:

From: Alex

foo patch implements foo because bar; note the fubar detail.

Based-on-patch-by: PJT
SoB: Alex

This isn't actually in the submitting patches document and I'm not sure it should be; although some clarification for these weird cases might be useful.

author
#23 lore raw

Re: [patch v8 3/9] sched: set initial value of runnable avg for new forked task

Quoted 19 lines

Example:

From: PJT

foo patch implements foo because bar; note the fubar detail.

SoB: PJT
[alex@intel: changed ponies into horses to make it fit]
SoB: Alex

The other case is where a 'simple' modification of the initial patch simply won't do; you need to change the core idea of the patch or similar. In this case I've seen things like:

From: Alex

foo patch implements foo because bar; note the fubar detail.

Quoted 7 lines

Based-on-patch-by: PJT
SoB: Alex

This isn't actually in the submitting patches document and I'm not sure it should be; although some clarification for these weird cases might be useful.

It is very very useful clarification!

Currently even some maintainers do not use the 'From' and SoB correctly. So I do suggest including it into Documentation/SubmittingPatches files. That will help everyone! :)

Signature · 2 lines
Thanks
    Alex
author
#24 lore raw

PATCH v8 4/9 sched: fix slept time double counting in enqueue entity

In reply to Alex Shi

The wakeuped migrated task will __synchronize_entity_decay(se); in migrate_task_fair, then it needs to set `se->avg.last_runnable_update -= (-se->avg.decay_count) << 20' before update_entity_load_avg, in order to avoid slept time is updated twice for se.avg.load_avg_contrib in both __syncchronize and update_entity_load_avg.

but if the slept task is waked up from self cpu, it miss the last_runnable_update before update_entity_load_avg(se, 0, 1), then the slept time was used twice in both functions.
So we need to remove the double slept time counting.

---
 kernel/sched/fair.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
kernel/sched/fair.c +2 -1
index 1fc30b9..42c7be0 100644
@@ -1570,7 +1570,8 @@ static inline void enqueue_entity_load_avg(struct cfs_rq *cfs_rq,
 		}
 		wakeup = 0;
 	} else {
-		__synchronize_entity_decay(se);
+		se->avg.last_runnable_update += __synchronize_entity_decay(se)
+							<< 20;
 	}
 
 	/* migrated tasks did not contribute to our blocked load */
Signature · 1 line
1.7.12
Reviewed-by on v8 4/9
#25 lore raw

Re: [patch v8 4/9] sched: fix slept time double counting in enqueue entity

Quoted 11 lines · Alex Shi

The wakeuped migrated task will __synchronize_entity_decay(se); in migrate_task_fair, then it needs to set `se->avg.last_runnable_update -= (-se->avg.decay_count) << 20' before update_entity_load_avg, in order to avoid slept time is updated twice for se.avg.load_avg_contrib in both __syncchronize and update_entity_load_avg.

but if the slept task is waked up from self cpu, it miss the last_runnable_update before update_entity_load_avg(se, 0, 1), then the slept time was used twice in both functions.
So we need to remove the double slept time counting.

Good catch.

Quoted 11 lines
---
 kernel/sched/fair.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
kernel/sched/fair.c +0 -0
index 1fc30b9..42c7be0 100644

kernel/sched/fair.c static inline void enqueue_entity_load_avg(struct cfs_rq *cfs_rq, · line 1574

Quoted 6 lines
kernel/sched/fair.c +2 -1
@@ -1570,7 +1570,8 @@ static inline void enqueue_entity_load_avg(struct cfs_rq *cfs_rq,
                }
                wakeup = 0;
        } else {
-               __synchronize_entity_decay(se);
+               se->avg.last_runnable_update += __synchronize_entity_decay(se)
+                                                       << 20;

Can you add something like:

+ /*
+ * Task re-woke on same cpu (or else
migrate_task_rq_fair()
+ * would have made count negative); we must be careful
to avoid
+ * double-accounting blocked time after synchronizing
decays.
+ */

author Reviewed-by Paul Turner
#26 lore raw

Re: [patch v8 4/9] sched: fix slept time double counting in enqueue entity

Quoted 12 lines · Paul Turner

Can you add something like:

(quoted diff) +2 -0
+               /*
+                * Task re-woke on same cpu (or else

migrate_task_rq_fair()
+ * would have made count negative); we must be careful
to avoid
+ * double-accounting blocked time after synchronizing
decays.
+ */

Quoted 1 line

thanks for review!
---

Quoted 1 line

From 24d9b43e7a269e6ffee5b874d39812b83812a809 Mon Sep 17 00:00:00 2001

From: Alex Shi <[email protected]>
Date: Thu, 9 May 2013 10:54:13 +0800
Subject: [PATCH] sched: fix slept time double counting in enqueue entity

The wakeuped migrated task will __synchronize_entity_decay(se); in migrate_task_fair, then it needs to set `se->avg.last_runnable_update -= (-se->avg.decay_count) << 20' before update_entity_load_avg, in order to avoid slept time is updated twice for se.avg.load_avg_contrib in both __syncchronize and update_entity_load_avg.

but if the slept task is waked up from self cpu, it miss the last_runnable_update before update_entity_load_avg(se, 0, 1), then the slept time was used twice in both functions.
So we need to remove the double slept time counting.

---
 kernel/sched/fair.c |    8 +++++++-
 1 files changed, 7 insertions(+), 1 deletions(-)
kernel/sched/fair.c +7 -1
index df5b8a9..1e5a5e6 100644
@@ -1571,7 +1571,13 @@ static inline void enqueue_entity_load_avg(struct cfs_rq *cfs_rq,
 		}
 		wakeup = 0;
 	} else {
-		__synchronize_entity_decay(se);
+		/*
+		 * Task re-woke on same cpu (or else migrate_task_rq_fair()
+		 * would have made count negative); we must be careful to avoid
+		 * double-accounting blocked time after synchronizing decays.
+		 */
+		se->avg.last_runnable_update += __synchronize_entity_decay(se)
+							<< 20;
 	}
 
 	/* migrated tasks did not contribute to our blocked load */
Signature · 1 line
1.7.5.4
#27 lore raw

Re: [patch v8 4/9] sched: fix slept time double counting in enqueue entity

Hi Alex,

Quoted 18 lines · Alex Shi
Quoted 12 lines · Paul Turner

Can you add something like:

(quoted diff) +2 -0
+               /*
+                * Task re-woke on same cpu (or else

migrate_task_rq_fair()
+ * would have made count negative); we must be careful
to avoid
+ * double-accounting blocked time after synchronizing
decays.
+ */

Quoted 1 line

thanks for review!
---

Quoted 7 lines

From 24d9b43e7a269e6ffee5b874d39812b83812a809 Mon Sep 17 00:00:00 2001
From: Alex Shi <[email protected]>
Date: Thu, 9 May 2013 10:54:13 +0800
Subject: [PATCH] sched: fix slept time double counting in enqueue entity

The wakeuped migrated task will __synchronize_entity_decay(se); in migrate_task_fair, then it needs to set

Should be migrate_task_rq_fair, right? :)

Quoted 22 lines

`se->avg.last_runnable_update -= (-se->avg.decay_count) << 20' before update_entity_load_avg, in order to avoid slept time is updated twice for se.avg.load_avg_contrib in both __syncchronize and update_entity_load_avg.

but if the slept task is waked up from self cpu, it miss the last_runnable_update before update_entity_load_avg(se, 0, 1), then the slept time was used twice in both functions.
So we need to remove the double slept time counting.

---
 kernel/sched/fair.c |    8 +++++++-
 1 files changed, 7 insertions(+), 1 deletions(-)
kernel/sched/fair.c +0 -0
index df5b8a9..1e5a5e6 100644
@@ -1571,7 +1571,13 @@ static inline void enqueue_entity_load_avg(struct cfs_rq *cfs_rq,
                }

kernel/sched/fair.c static inline void enqueue_entity_load_avg(struct cfs_rq *cfs_rq, · line 1580

Quoted 10 lines
kernel/sched/fair.c +7 -1
                wakeup = 0;
        } else {
-               __synchronize_entity_decay(se);
+               /*
+                * Task re-woke on same cpu (or else migrate_task_rq_fair()
+                * would have made count negative); we must be careful to avoid
+                * double-accounting blocked time after synchronizing decays.
+                */
+               se->avg.last_runnable_update += __synchronize_entity_decay(se)
+                                                       << 20;

I'm kind of getting confused at here, since migrate_task_rq_fair use below equation to
avoid sleep time be accounted for twice.
`se->avg.last_runnable_update -= (-se->avg.decay_count) << 20'

Why here we need to use "+=", which the reversed version comparing with previous?

Thanks,
Lei

Quoted 12 lines
kernel/sched/fair.c +0 -1
        }
 
        /* migrated tasks did not contribute to our blocked load */
--

1.7.5.4

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

author
#28 lore raw

Re: [patch v8 4/9] sched: fix slept time double counting in enqueue entity

Quoted 9 lines · Lei Wen
Quoted 7 lines
Quoted 7 lines

From 24d9b43e7a269e6ffee5b874d39812b83812a809 Mon Sep 17 00:00:00 2001
From: Alex Shi <[email protected]>
Date: Thu, 9 May 2013 10:54:13 +0800
Subject: [PATCH] sched: fix slept time double counting in enqueue entity

The wakeuped migrated task will __synchronize_entity_decay(se); in migrate_task_fair, then it needs to set

Should be migrate_task_rq_fair, right? :)

yes, thanks!

Signature · 2 lines
Thanks
    Alex
#29 lore raw

Re: [patch v8 4/9] sched: fix slept time double counting in enqueue entity

In reply to Lei Wen
Quoted 69 lines · Lei Wen

Hi Alex,

Quoted 18 lines · Alex Shi
Quoted 12 lines · Paul Turner

Can you add something like:

(quoted diff) +2 -0
+               /*
+                * Task re-woke on same cpu (or else

migrate_task_rq_fair()
+ * would have made count negative); we must be careful
to avoid
+ * double-accounting blocked time after synchronizing
decays.
+ */

Quoted 1 line

thanks for review!
---

Quoted 7 lines

From 24d9b43e7a269e6ffee5b874d39812b83812a809 Mon Sep 17 00:00:00 2001
From: Alex Shi <[email protected]>
Date: Thu, 9 May 2013 10:54:13 +0800
Subject: [PATCH] sched: fix slept time double counting in enqueue entity

The wakeuped migrated task will __synchronize_entity_decay(se); in migrate_task_fair, then it needs to set

Should be migrate_task_rq_fair, right? :)

Quoted 22 lines

`se->avg.last_runnable_update -= (-se->avg.decay_count) << 20' before update_entity_load_avg, in order to avoid slept time is updated twice for se.avg.load_avg_contrib in both __syncchronize and update_entity_load_avg.

but if the slept task is waked up from self cpu, it miss the last_runnable_update before update_entity_load_avg(se, 0, 1), then the slept time was used twice in both functions.
So we need to remove the double slept time counting.

---
 kernel/sched/fair.c |    8 +++++++-
 1 files changed, 7 insertions(+), 1 deletions(-)
kernel/sched/fair.c +0 -0
index df5b8a9..1e5a5e6 100644
@@ -1571,7 +1571,13 @@ static inline void enqueue_entity_load_avg(struct cfs_rq *cfs_rq,
                }

kernel/sched/fair.c static inline void enqueue_entity_load_avg(struct cfs_rq *cfs_rq, · line 1580

Quoted 10 lines
kernel/sched/fair.c +7 -1
                wakeup = 0;
        } else {
-               __synchronize_entity_decay(se);
+               /*
+                * Task re-woke on same cpu (or else migrate_task_rq_fair()
+                * would have made count negative); we must be careful to avoid
+                * double-accounting blocked time after synchronizing decays.
+                */
+               se->avg.last_runnable_update += __synchronize_entity_decay(se)
+                                                       << 20;

I'm kind of getting confused at here, since migrate_task_rq_fair use below equation to
avoid sleep time be accounted for twice.
`se->avg.last_runnable_update -= (-se->avg.decay_count) << 20'

Quoted 2 lines

Why here we need to use "+=", which the reversed version comparing with previous?

Here is my understanding for this reversed behavior, not sure am I right?...
1. in migrate_task_rq_fair case, it performs the decay operation directly over load_avg_contrib, but not update runnable_avg_sum and runnable_avg_period accordingly. Thus when the task migrated to different runqueue, it make its previous
decayed result can be reflected over the new runqueue, since load_avg_contrib would be calculated with original runnable_avg_sum and runnable_avg_period value,
it would mean the migrated task lost the decay in this process, as avg.last_runnable_update is aligned with new runqueue's clock_task.

So we need "-=" in the migrated case.

2. For the normal wake up case, since we run over local queue continuously, our avg.last_runnable_update is not changed, thus if we do nothing, we could get the another decay in __update_entity_runnable_avg. Since we already get one over __synchronzie_entity_decay, it should bring us a redundant one.

Thus it is right to add a "+=" here.

But here I have a question, there is another usage of __synchronzie_entity_decay in current kernel, in the switched_from_fair function.

If task being frequently switched between rt and fair class, would it also bring the
redundant issue? Do we need patch like below?

kernel/sched/fair.c +0 -0
index b5408e1..9640c66 100644
@@ -5856,7 +5856,7 @@ static void switched_from_fair(struct rq *rq,

struct task_struct *p)

        */
        if (p->se.avg.decay_count) {
                struct cfs_rq *cfs_rq = cfs_rq_of(&p->se);
-               se->avg.last_runnable_update +=
+               p->se.avg.last_runnable_update +=
                                __synchronize_entity_decay(&p->se);
                subtract_blocked_load_contrib(cfs_rq,
                                p->se.avg.load_avg_contrib);

Thanks,
Lei

Quoted 16 lines

Thanks,
Lei

Quoted 12 lines
kernel/sched/fair.c +0 -1
        }
 
        /* migrated tasks did not contribute to our blocked load */
--

1.7.5.4

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

#30 lore raw

Re: [patch v8 4/9] sched: fix slept time double counting in enqueue entity

Quoted 14 lines · Lei Wen

But here I have a question, there is another usage of __synchronzie_entity_decay in current kernel, in the switched_from_fair function.

If task being frequently switched between rt and fair class, would it also bring the
redundant issue? Do we need patch like below?

kernel/sched/fair.c +0 -0
index b5408e1..9640c66 100644

kernel/sched/fair.c static void switched_from_fair(struct rq *rq, · line 5860

Quoted 6 lines
kernel/sched/fair.c +1 -1
@@ -5856,7 +5856,7 @@ static void switched_from_fair(struct rq *rq,
 struct task_struct *p)
        */
        if (p->se.avg.decay_count) {
                struct cfs_rq *cfs_rq = cfs_rq_of(&p->se);
-               se->avg.last_runnable_update +=
+               p->se.avg.last_runnable_update +=

what tree does this patch base on?

Quoted 3 lines
kernel/sched/fair.c +0 -0
                                __synchronize_entity_decay(&p->se);
                subtract_blocked_load_contrib(cfs_rq,
                                p->se.avg.load_avg_contrib);
Signature · 2 lines
Thanks
    Alex
#31 lore raw

Re: [patch v8 4/9] sched: fix slept time double counting in enqueue entity

Hi Alex,

Quoted 10 lines · Alex Shi
Quoted 9 lines · Lei Wen

But here I have a question, there is another usage of __synchronzie_entity_decay in current kernel, in the switched_from_fair function.

If task being frequently switched between rt and fair class, would it also bring the
redundant issue? Do we need patch like below?

Quoted 13 lines

kernel/sched/fair.c static void switched_from_fair(struct rq *rq, · line 5860

Quoted 11 lines
kernel/sched/fair.c +1 -1
index b5408e1..9640c66 100644
@@ -5856,7 +5856,7 @@ static void switched_from_fair(struct rq *rq,
 struct task_struct *p)
        */
        if (p->se.avg.decay_count) {
                struct cfs_rq *cfs_rq = cfs_rq_of(&p->se);
-               se->avg.last_runnable_update +=
+               p->se.avg.last_runnable_update +=

what tree does this patch base on?

I create the patch based on v3.9 kernel.
If it is ok, I could create a formal one based on latest kernel.

Thanks,
Lei

Quoted 8 lines
Quoted 3 lines
kernel/sched/fair.c +0 -0
                                __synchronize_entity_decay(&p->se);
                subtract_blocked_load_contrib(cfs_rq,
                                p->se.avg.load_avg_contrib);

--
Thanks
Alex

#32 lore raw

Re: [patch v8 4/9] sched: fix slept time double counting in enqueue entity

Quoted 27 lines · Lei Wen

Hi Alex,

Quoted 10 lines · Alex Shi
Quoted 9 lines · Lei Wen

But here I have a question, there is another usage of __synchronzie_entity_decay in current kernel, in the switched_from_fair function.

If task being frequently switched between rt and fair class, would it also bring the
redundant issue? Do we need patch like below?

Quoted 13 lines

kernel/sched/fair.c static void switched_from_fair(struct rq *rq, · line 5860

Quoted 11 lines
kernel/sched/fair.c +1 -1
index b5408e1..9640c66 100644
@@ -5856,7 +5856,7 @@ static void switched_from_fair(struct rq *rq,
 struct task_struct *p)
        */
        if (p->se.avg.decay_count) {
                struct cfs_rq *cfs_rq = cfs_rq_of(&p->se);
-               se->avg.last_runnable_update +=
+               p->se.avg.last_runnable_update +=

what tree does this patch base on?

Quoted 2 lines

I create the patch based on v3.9 kernel.
If it is ok, I could create a formal one based on latest kernel.

In the linus tree, the first commit which introduced __synchronize_entity_decay(&p->se) here is 9ee474f55664ff63111c843099d365e7ecffb56f
+
+#if defined(CONFIG_FAIR_GROUP_SCHED) && defined(CONFIG_SMP)

+       /*
+       * Remove our load from contribution when we leave sched_fair
+       * and ensure we don't carry in an old decay_count if we
+       * switch back.
+       */
+       if (p->se.avg.decay_count) {
+               struct cfs_rq *cfs_rq = cfs_rq_of(&p->se);
+               __synchronize_entity_decay(&p->se);
+               subtract_blocked_load_contrib(cfs_rq,

+ p->se.avg.load_avg_contrib);
+ }
+#endif
}

And it is never changed from then on. So your code must based on a incorrect kernel. please do a double check.

Quoted 12 lines

Thanks,
Lei

Quoted 8 lines
Quoted 3 lines
kernel/sched/fair.c +0 -0
                                __synchronize_entity_decay(&p->se);
                subtract_blocked_load_contrib(cfs_rq,
                                p->se.avg.load_avg_contrib);

--
Thanks
Alex

Signature · 2 lines
Thanks
    Alex
#33 lore raw

Re: [patch v8 4/9] sched: fix slept time double counting in enqueue entity

Alex,

Quoted 50 lines · Alex Shi
Quoted 27 lines · Lei Wen

Hi Alex,

Quoted 10 lines · Alex Shi
Quoted 9 lines · Lei Wen

But here I have a question, there is another usage of __synchronzie_entity_decay in current kernel, in the switched_from_fair function.

If task being frequently switched between rt and fair class, would it also bring the
redundant issue? Do we need patch like below?

Quoted 13 lines

kernel/sched/fair.c static void switched_from_fair(struct rq *rq, · line 5860

Quoted 11 lines
kernel/sched/fair.c +1 -1
index b5408e1..9640c66 100644
@@ -5856,7 +5856,7 @@ static void switched_from_fair(struct rq *rq,
 struct task_struct *p)
        */
        if (p->se.avg.decay_count) {
                struct cfs_rq *cfs_rq = cfs_rq_of(&p->se);
-               se->avg.last_runnable_update +=
+               p->se.avg.last_runnable_update +=

what tree does this patch base on?

Quoted 2 lines

I create the patch based on v3.9 kernel.
If it is ok, I could create a formal one based on latest kernel.

In the linus tree, the first commit which introduced __synchronize_entity_decay(&p->se) here is 9ee474f55664ff63111c843099d365e7ecffb56f

(quoted diff) +14 -0
+
+#if defined(CONFIG_FAIR_GROUP_SCHED) && defined(CONFIG_SMP)
+       /*
+       * Remove our load from contribution when we leave sched_fair
+       * and ensure we don't carry in an old decay_count if we
+       * switch back.
+       */
+       if (p->se.avg.decay_count) {
+               struct cfs_rq *cfs_rq = cfs_rq_of(&p->se);
+               __synchronize_entity_decay(&p->se);
+               subtract_blocked_load_contrib(cfs_rq,
+                               p->se.avg.load_avg_contrib);
+       }
+#endif
 }
Quoted 2 lines

And it is never changed from then on. So your code must based on a incorrect kernel. please do a double check.

I see your point... I made the mistake that update the wrong patch... Please help check this one.

commit 5fc3d5c74f8359ef382d9a20ffe657ffc237c109

Author: Lei Wen <[email protected]>
Date:   Thu Jun 20 10:43:59 2013 +0800

sched: fix potential twice decay issue

kernel/sched/fair.c +0 -0
index c61a614..9640c66 100644
@@ -5856,7 +5856,8 @@ static void switched_from_fair(struct rq *rq,

struct task_struct *p)

        */
        if (p->se.avg.decay_count) {
                struct cfs_rq *cfs_rq = cfs_rq_of(&p->se);
-               __synchronize_entity_decay(&p->se);
+               p->se.avg.last_runnable_update +=
+                               __synchronize_entity_decay(&p->se);
                subtract_blocked_load_contrib(cfs_rq,
                                p->se.avg.load_avg_contrib);
        }

Thanks,
Lei

Quoted 17 lines
Quoted 12 lines

Thanks,
Lei

Quoted 8 lines
Quoted 3 lines
kernel/sched/fair.c +0 -0
                                __synchronize_entity_decay(&p->se);
                subtract_blocked_load_contrib(cfs_rq,
                                p->se.avg.load_avg_contrib);

--
Thanks
Alex

--
Thanks
Alex

#34 lore raw

Re: [patch v8 4/9] sched: fix slept time double counting in enqueue entity

Quoted 16 lines · Lei Wen

I see your point... I made the mistake that update the wrong patch... Please help check this one.

commit 5fc3d5c74f8359ef382d9a20ffe657ffc237c109

Author: Lei Wen <[email protected]>
Date:   Thu Jun 20 10:43:59 2013 +0800

sched: fix potential twice decay issue

kernel/sched/fair.c +0 -0
index c61a614..9640c66 100644

kernel/sched/fair.c static void switched_from_fair(struct rq *rq, · line 5861

Quoted 7 lines
kernel/sched/fair.c +2 -1
@@ -5856,7 +5856,8 @@ static void switched_from_fair(struct rq *rq,
 struct task_struct *p)
        */
        if (p->se.avg.decay_count) {
                struct cfs_rq *cfs_rq = cfs_rq_of(&p->se);
-               __synchronize_entity_decay(&p->se);
+               p->se.avg.last_runnable_update +=
+                               __synchronize_entity_decay(&p->se);

it is not needed, since the last_runnable_update will be reset if it will be switched to fair.

Signature · 2 lines
Thanks
    Alex
#35 lore raw

Re: [patch v8 4/9] sched: fix slept time double counting in enqueue entity

Alex,

Quoted 12 lines · Alex Shi
Quoted 11 lines · Lei Wen

I see your point... I made the mistake that update the wrong patch... Please help check this one.

commit 5fc3d5c74f8359ef382d9a20ffe657ffc237c109

Author: Lei Wen <[email protected]>
Date:   Thu Jun 20 10:43:59 2013 +0800

sched: fix potential twice decay issue

Quoted 15 lines

kernel/sched/fair.c static void switched_from_fair(struct rq *rq, · line 5861

Quoted 12 lines
kernel/sched/fair.c +2 -1
index c61a614..9640c66 100644
@@ -5856,7 +5856,8 @@ static void switched_from_fair(struct rq *rq,
 struct task_struct *p)
        */
        if (p->se.avg.decay_count) {
                struct cfs_rq *cfs_rq = cfs_rq_of(&p->se);
-               __synchronize_entity_decay(&p->se);
+               p->se.avg.last_runnable_update +=
+                               __synchronize_entity_decay(&p->se);

it is not needed, since the last_runnable_update will be reset if it will be switched to fair.

When it is backed to fair class, the only thing scheduler would do is to check whether it could preempt current running task, so where to reset the last_runnable_update?

Thanks,
Lei

Quoted 4 lines

--
Thanks
Alex

#36 lore raw

Re: [patch v8 4/9] sched: fix slept time double counting in enqueue entity

Quoted 17 lines
Quoted 16 lines

kernel/sched/fair.c static void switched_from_fair(struct rq *rq, · line 5861

Quoted 13 lines
kernel/sched/fair.c +2 -1
index c61a614..9640c66 100644
@@ -5856,7 +5856,8 @@ static void switched_from_fair(struct rq *rq,
 struct task_struct *p)
        */
        if (p->se.avg.decay_count) {
                struct cfs_rq *cfs_rq = cfs_rq_of(&p->se);
-               __synchronize_entity_decay(&p->se);
+               p->se.avg.last_runnable_update +=
+                               __synchronize_entity_decay(&p->se);

it is not needed, since the last_runnable_update will be reset if it will be switched to fair.

Quoted 3 lines

When it is backed to fair class, the only thing scheduler would do is to check whether it could preempt current running task, so where to reset the last_runnable_update?

here, when the task enqueue to cfs_rq.
avg.decay_count == 0, so the last line will reset last_runnable_update.

Anyway, you may consider to add a comment in the function, since it is bit hard to understand for you.

---
/* Add the load generated by se into cfs_rq's child load-average */ static inline void enqueue_entity_load_avg(struct cfs_rq *cfs_rq,
struct sched_entity *se, int wakeup)

{
        /*
         * We track migrations using entity decay_count <= 0, on a wake-up
         * migration we use a negative decay count to track the remote decays
         * accumulated while sleeping.
         */
        if (unlikely(se->avg.decay_count <= 0)) {
                se->avg.last_runnable_update = rq_of(cfs_rq)->clock_task;
Quoted 8 lines

Thanks,
Lei

Quoted 4 lines

--
Thanks
Alex

Signature · 2 lines
Thanks
    Alex
#37 lore raw

Re: [patch v8 4/9] sched: fix slept time double counting in enqueue entity

Quoted 32 lines · Alex Shi
Quoted 17 lines
Quoted 16 lines

kernel/sched/fair.c static void switched_from_fair(struct rq *rq, · line 5861

Quoted 13 lines
kernel/sched/fair.c +2 -1
index c61a614..9640c66 100644
@@ -5856,7 +5856,8 @@ static void switched_from_fair(struct rq *rq,
 struct task_struct *p)
        */
        if (p->se.avg.decay_count) {
                struct cfs_rq *cfs_rq = cfs_rq_of(&p->se);
-               __synchronize_entity_decay(&p->se);
+               p->se.avg.last_runnable_update +=
+                               __synchronize_entity_decay(&p->se);

it is not needed, since the last_runnable_update will be reset if it will be switched to fair.

Quoted 3 lines

When it is backed to fair class, the only thing scheduler would do is to check whether it could preempt current running task, so where to reset the last_runnable_update?

here, when the task enqueue to cfs_rq.
avg.decay_count == 0, so the last line will reset last_runnable_update.

Anyway, you may consider to add a comment in the function, since it is bit hard to understand for you.

---
/* Add the load generated by se into cfs_rq's child load-average */ static inline void enqueue_entity_load_avg(struct cfs_rq *cfs_rq,
struct sched_entity *se, int wakeup)

Quoted 8 lines
{
        /*
         * We track migrations using entity decay_count <= 0, on a wake-up
         * migration we use a negative decay count to track the remote decays
         * accumulated while sleeping.
         */
        if (unlikely(se->avg.decay_count <= 0)) {
                se->avg.last_runnable_update = rq_of(cfs_rq)->clock_task;

I see...
I didn't notice before the switched_from_fair, we already called dequeue and enqueue.
Since it works as expected, I have no question.

Thanks for your detailed explanation! :)

Thanks,
Lei

Quoted 14 lines
Quoted 9 lines

Thanks,
Lei

Quoted 4 lines

--
Thanks
Alex

--
Thanks
Alex

author
#38 lore raw

PATCH v8 5/9 sched: update cpu load after task_tick.

In reply to Alex Shi

To get the latest runnable info, we need do this cpuload update after task_tick.

---
 kernel/sched/core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/sched/core.c +1 -1
index 6f226c2..05176b8 100644
@@ -2152,8 +2152,8 @@ void scheduler_tick(void)
 
 	raw_spin_lock(&rq->lock);
 	update_rq_clock(rq);
-	update_cpu_load_active(rq);
 	curr->sched_class->task_tick(rq, curr, 0);
+	update_cpu_load_active(rq);
 	raw_spin_unlock(&rq->lock);
 
 	perf_event_task_tick();
Signature · 1 line
1.7.12
Reviewed-by on v8 5/9
#39 lore raw

Re: [patch v8 5/9] sched: update cpu load after task_tick.

Quoted 4 lines · Alex Shi

To get the latest runnable info, we need do this cpuload update after task_tick.

Quoted 21 lines
---
 kernel/sched/core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/sched/core.c +1 -2
index 6f226c2..05176b8 100644
@@ -2152,8 +2152,8 @@ void scheduler_tick(void)
 
        raw_spin_lock(&rq->lock);
        update_rq_clock(rq);
-       update_cpu_load_active(rq);
        curr->sched_class->task_tick(rq, curr, 0);
+       update_cpu_load_active(rq);
        raw_spin_unlock(&rq->lock);
 
        perf_event_task_tick();
--

1.7.12

author
#40 lore raw

PATCH v8 6/9 sched: compute runnable load avg in cpu_load and cpu_avg_load_per_task

In reply to Alex Shi

They are the base values in load balance, update them with rq runnable load average, then the load balance will consider runnable load avg naturally.

We also try to include the blocked_load_avg as cpu load in balancing, but that cause kbuild performance drop 6% on every Intel machine, and aim7/oltp drop on some of 4 CPU sockets machines.

---
 kernel/sched/fair.c |  5 +++--
 kernel/sched/proc.c | 17 +++++++++++++++--
 2 files changed, 18 insertions(+), 4 deletions(-)
kernel/sched/fair.c +3 -2
index 42c7be0..eadd2e7 100644
@@ -2962,7 +2962,7 @@ static void dequeue_task_fair(struct rq *rq, struct task_struct *p, int flags)
 /* Used instead of source_load when we know the type == 0 */
 static unsigned long weighted_cpuload(const int cpu)
 {
-	return cpu_rq(cpu)->load.weight;
+	return cpu_rq(cpu)->cfs.runnable_load_avg;
 }
 
 /*
@@ -3007,9 +3007,10 @@ static unsigned long cpu_avg_load_per_task(int cpu)
 {
 	struct rq *rq = cpu_rq(cpu);
 	unsigned long nr_running = ACCESS_ONCE(rq->nr_running);
+	unsigned long load_avg = rq->cfs.runnable_load_avg;
 
 	if (nr_running)
-		return rq->load.weight / nr_running;
+		return load_avg / nr_running;
 
 	return 0;
 }
kernel/sched/proc.c +15 -2
index bb3a6a0..ce5cd48 100644
@@ -501,6 +501,18 @@ static void __update_cpu_load(struct rq *this_rq, unsigned long this_load,
 	sched_avg_update(this_rq);
 }
 
+#ifdef CONFIG_SMP
+unsigned long get_rq_runnable_load(struct rq *rq)
+{
+	return rq->cfs.runnable_load_avg;
+}
+#else
+unsigned long get_rq_runnable_load(struct rq *rq)
+{
+	return rq->load.weight;
+}
+#endif
+
 #ifdef CONFIG_NO_HZ_COMMON
 /*
  * There is no sane way to deal with nohz on smp when using jiffies because the
@@ -522,7 +534,7 @@ static void __update_cpu_load(struct rq *this_rq, unsigned long this_load,
 void update_idle_cpu_load(struct rq *this_rq)
 {
 	unsigned long curr_jiffies = ACCESS_ONCE(jiffies);
-	unsigned long load = this_rq->load.weight;
+	unsigned long load = get_rq_runnable_load(this_rq);
 	unsigned long pending_updates;
 
 	/*
@@ -568,11 +580,12 @@ void update_cpu_load_nohz(void)
  */
 void update_cpu_load_active(struct rq *this_rq)
 {
+	unsigned long load = get_rq_runnable_load(this_rq);
 	/*
 	 * See the mess around update_idle_cpu_load() / update_cpu_load_nohz().
 	 */
 	this_rq->last_load_update_tick = jiffies;
-	__update_cpu_load(this_rq, this_rq->load.weight, 1);
+	__update_cpu_load(this_rq, load, 1);
 
 	calc_load_account_active(this_rq);
 }
Signature · 1 line
1.7.12
Reviewed-by on v8 6/9
#41 lore raw

Re: [patch v8 6/9] sched: compute runnable load avg in cpu_load and cpu_avg_load_per_task

Quoted 7 lines · Alex Shi

They are the base values in load balance, update them with rq runnable load average, then the load balance will consider runnable load avg naturally.

We also try to include the blocked_load_avg as cpu load in balancing, but that cause kbuild performance drop 6% on every Intel machine, and aim7/oltp drop on some of 4 CPU sockets machines.

Hi Alex,
Could you explain me why including the blocked_load_avg causes performance drop ?

Thanks,
Gu

Quoted 2 lines
Quoted 76 lines
---
 kernel/sched/fair.c |  5 +++--
 kernel/sched/proc.c | 17 +++++++++++++++--
 2 files changed, 18 insertions(+), 4 deletions(-)
kernel/sched/fair.c +3 -2
index 42c7be0..eadd2e7 100644
@@ -2962,7 +2962,7 @@ static void dequeue_task_fair(struct rq *rq, struct task_struct *p, int flags)
 /* Used instead of source_load when we know the type == 0 */
 static unsigned long weighted_cpuload(const int cpu)
 {
-	return cpu_rq(cpu)->load.weight;
+	return cpu_rq(cpu)->cfs.runnable_load_avg;
 }
 
 /*
@@ -3007,9 +3007,10 @@ static unsigned long cpu_avg_load_per_task(int cpu)
 {
 	struct rq *rq = cpu_rq(cpu);
 	unsigned long nr_running = ACCESS_ONCE(rq->nr_running);
+	unsigned long load_avg = rq->cfs.runnable_load_avg;
 
 	if (nr_running)
-		return rq->load.weight / nr_running;
+		return load_avg / nr_running;
 
 	return 0;
 }
kernel/sched/proc.c +15 -2
index bb3a6a0..ce5cd48 100644
@@ -501,6 +501,18 @@ static void __update_cpu_load(struct rq *this_rq, unsigned long this_load,
 	sched_avg_update(this_rq);
 }
 
+#ifdef CONFIG_SMP
+unsigned long get_rq_runnable_load(struct rq *rq)
+{
+	return rq->cfs.runnable_load_avg;
+}
+#else
+unsigned long get_rq_runnable_load(struct rq *rq)
+{
+	return rq->load.weight;
+}
+#endif
+
 #ifdef CONFIG_NO_HZ_COMMON
 /*
  * There is no sane way to deal with nohz on smp when using jiffies because the
@@ -522,7 +534,7 @@ static void __update_cpu_load(struct rq *this_rq, unsigned long this_load,
 void update_idle_cpu_load(struct rq *this_rq)
 {
 	unsigned long curr_jiffies = ACCESS_ONCE(jiffies);
-	unsigned long load = this_rq->load.weight;
+	unsigned long load = get_rq_runnable_load(this_rq);
 	unsigned long pending_updates;
 
 	/*
@@ -568,11 +580,12 @@ void update_cpu_load_nohz(void)
  */
 void update_cpu_load_active(struct rq *this_rq)
 {
+	unsigned long load = get_rq_runnable_load(this_rq);
 	/*
 	 * See the mess around update_idle_cpu_load() / update_cpu_load_nohz().
 	 */
 	this_rq->last_load_update_tick = jiffies;
-	__update_cpu_load(this_rq, this_rq->load.weight, 1);
+	__update_cpu_load(this_rq, load, 1);
 
 	calc_load_account_active(this_rq);
 }
author
#42 lore raw

Re: [patch v8 6/9] sched: compute runnable load avg in cpu_load and cpu_avg_load_per_task

Quoted 11 lines · Gu Zheng
Quoted 7 lines · Alex Shi
Quoted 7 lines

They are the base values in load balance, update them with rq runnable load average, then the load balance will consider runnable load avg naturally.

We also try to include the blocked_load_avg as cpu load in balancing, but that cause kbuild performance drop 6% on every Intel machine, and aim7/oltp drop on some of 4 CPU sockets machines.

Hi Alex,
Could you explain me why including the blocked_load_avg causes performance drop ?

Thanks for review!

the 9th patch has few explanation. like, after the only task got into sleep in a CPU, there is only blocked_load_avg left, it looks quite big in short time. that, block it get tasks before sleep, drive task to other cpu in periodic balance. So, it cause clear load imbalance.

Signature · 2 lines
Thanks
    Alex
#43 lore raw

Re: [patch v8 6/9] sched: compute runnable load avg in cpu_load and cpu_avg_load_per_task

Quoted 14 lines · Alex Shi
Quoted 11 lines · Gu Zheng
Quoted 7 lines · Alex Shi
Quoted 7 lines

They are the base values in load balance, update them with rq runnable load average, then the load balance will consider runnable load avg naturally.

We also try to include the blocked_load_avg as cpu load in balancing, but that cause kbuild performance drop 6% on every Intel machine, and aim7/oltp drop on some of 4 CPU sockets machines.

Hi Alex,
Could you explain me why including the blocked_load_avg causes performance drop ?

Quoted 7 lines

Thanks for review!

the 9th patch has few explanation. like, after the only task got into sleep in a CPU, there is only blocked_load_avg left, it looks quite big in short time. that, block it get tasks before sleep, drive task to other cpu in periodic balance. So, it cause clear load imbalance.

Got it. Thanks very much for your explanation.:)

Best regards,
Gu

#44 lore raw

Re: [patch v8 6/9] sched: compute runnable load avg in cpu_load and cpu_avg_load_per_task

In reply to Alex Shi
Quoted 8 lines · Alex Shi

They are the base values in load balance, update them with rq runnable load average, then the load balance will consider runnable load avg naturally.

We also try to include the blocked_load_avg as cpu load in balancing, but that cause kbuild performance drop 6% on every Intel machine, and aim7/oltp drop on some of 4 CPU sockets machines.

This looks fine.

Did you try including blocked_load_avg in only get_rq_runnable_load() [ and not weighted_cpuload() which is called by new-idle ]?

Quoted 80 lines
---
 kernel/sched/fair.c |  5 +++--
 kernel/sched/proc.c | 17 +++++++++++++++--
 2 files changed, 18 insertions(+), 4 deletions(-)
kernel/sched/fair.c +3 -2
index 42c7be0..eadd2e7 100644
@@ -2962,7 +2962,7 @@ static void dequeue_task_fair(struct rq *rq, struct task_struct *p, int flags)
 /* Used instead of source_load when we know the type == 0 */
 static unsigned long weighted_cpuload(const int cpu)
 {
-       return cpu_rq(cpu)->load.weight;
+       return cpu_rq(cpu)->cfs.runnable_load_avg;
 }
 
 /*
@@ -3007,9 +3007,10 @@ static unsigned long cpu_avg_load_per_task(int cpu)
 {
        struct rq *rq = cpu_rq(cpu);
        unsigned long nr_running = ACCESS_ONCE(rq->nr_running);
+       unsigned long load_avg = rq->cfs.runnable_load_avg;
 
        if (nr_running)
-               return rq->load.weight / nr_running;
+               return load_avg / nr_running;
 
        return 0;
 }
kernel/sched/proc.c +15 -3
index bb3a6a0..ce5cd48 100644
@@ -501,6 +501,18 @@ static void __update_cpu_load(struct rq *this_rq, unsigned long this_load,
        sched_avg_update(this_rq);
 }
 
+#ifdef CONFIG_SMP
+unsigned long get_rq_runnable_load(struct rq *rq)
+{
+       return rq->cfs.runnable_load_avg;
+}
+#else
+unsigned long get_rq_runnable_load(struct rq *rq)
+{
+       return rq->load.weight;
+}
+#endif
+
 #ifdef CONFIG_NO_HZ_COMMON
 /*
  * There is no sane way to deal with nohz on smp when using jiffies because the
@@ -522,7 +534,7 @@ static void __update_cpu_load(struct rq *this_rq, unsigned long this_load,
 void update_idle_cpu_load(struct rq *this_rq)
 {
        unsigned long curr_jiffies = ACCESS_ONCE(jiffies);
-       unsigned long load = this_rq->load.weight;
+       unsigned long load = get_rq_runnable_load(this_rq);
        unsigned long pending_updates;
 
        /*
@@ -568,11 +580,12 @@ void update_cpu_load_nohz(void)
  */
 void update_cpu_load_active(struct rq *this_rq)
 {
+       unsigned long load = get_rq_runnable_load(this_rq);
        /*
         * See the mess around update_idle_cpu_load() / update_cpu_load_nohz().
         */
        this_rq->last_load_update_tick = jiffies;
-       __update_cpu_load(this_rq, this_rq->load.weight, 1);
+       __update_cpu_load(this_rq, load, 1);
 
        calc_load_account_active(this_rq);
 }
--

1.7.12

#45 lore raw

Re: [patch v8 6/9] sched: compute runnable load avg in cpu_load and cpu_avg_load_per_task

Quoted 10 lines · Paul Turner
Quoted 8 lines · Alex Shi

They are the base values in load balance, update them with rq runnable load average, then the load balance will consider runnable load avg naturally.

We also try to include the blocked_load_avg as cpu load in balancing, but that cause kbuild performance drop 6% on every Intel machine, and aim7/oltp drop on some of 4 CPU sockets machines.

Quoted 4 lines

This looks fine.

Did you try including blocked_load_avg in only get_rq_runnable_load() [ and not weighted_cpuload() which is called by new-idle ]?

Looking at this more this feels less correct since you're taking averages of averages.

This was previously discussed at:
https://lkml.org/lkml/2013/5/6/109

And you later replied suggesting this didn't seem to hurt; what's the current status there?

Quoted 81 lines
Quoted 81 lines
---
 kernel/sched/fair.c |  5 +++--
 kernel/sched/proc.c | 17 +++++++++++++++--
 2 files changed, 18 insertions(+), 4 deletions(-)
kernel/sched/fair.c +3 -2
index 42c7be0..eadd2e7 100644
@@ -2962,7 +2962,7 @@ static void dequeue_task_fair(struct rq *rq, struct task_struct *p, int flags)
 /* Used instead of source_load when we know the type == 0 */
 static unsigned long weighted_cpuload(const int cpu)
 {
-       return cpu_rq(cpu)->load.weight;
+       return cpu_rq(cpu)->cfs.runnable_load_avg;
 }
 
 /*
@@ -3007,9 +3007,10 @@ static unsigned long cpu_avg_load_per_task(int cpu)
 {
        struct rq *rq = cpu_rq(cpu);
        unsigned long nr_running = ACCESS_ONCE(rq->nr_running);
+       unsigned long load_avg = rq->cfs.runnable_load_avg;
 
        if (nr_running)
-               return rq->load.weight / nr_running;
+               return load_avg / nr_running;
 
        return 0;
 }
kernel/sched/proc.c +15 -3
index bb3a6a0..ce5cd48 100644
@@ -501,6 +501,18 @@ static void __update_cpu_load(struct rq *this_rq, unsigned long this_load,
        sched_avg_update(this_rq);
 }
 
+#ifdef CONFIG_SMP
+unsigned long get_rq_runnable_load(struct rq *rq)
+{
+       return rq->cfs.runnable_load_avg;
+}
+#else
+unsigned long get_rq_runnable_load(struct rq *rq)
+{
+       return rq->load.weight;
+}
+#endif
+
 #ifdef CONFIG_NO_HZ_COMMON
 /*
  * There is no sane way to deal with nohz on smp when using jiffies because the
@@ -522,7 +534,7 @@ static void __update_cpu_load(struct rq *this_rq, unsigned long this_load,
 void update_idle_cpu_load(struct rq *this_rq)
 {
        unsigned long curr_jiffies = ACCESS_ONCE(jiffies);
-       unsigned long load = this_rq->load.weight;
+       unsigned long load = get_rq_runnable_load(this_rq);
        unsigned long pending_updates;
 
        /*
@@ -568,11 +580,12 @@ void update_cpu_load_nohz(void)
  */
 void update_cpu_load_active(struct rq *this_rq)
 {
+       unsigned long load = get_rq_runnable_load(this_rq);
        /*
         * See the mess around update_idle_cpu_load() / update_cpu_load_nohz().
         */
        this_rq->last_load_update_tick = jiffies;
-       __update_cpu_load(this_rq, this_rq->load.weight, 1);
+       __update_cpu_load(this_rq, load, 1);
 
        calc_load_account_active(this_rq);
 }
--

1.7.12

maintainer
#46 lore raw

Re: [patch v8 6/9] sched: compute runnable load avg in cpu_load and cpu_avg_load_per_task

Quoted 16 lines · Paul Turner
Quoted 10 lines · Paul Turner
Quoted 8 lines · Alex Shi

They are the base values in load balance, update them with rq runnable load average, then the load balance will consider runnable load avg naturally.

We also try to include the blocked_load_avg as cpu load in balancing, but that cause kbuild performance drop 6% on every Intel machine, and aim7/oltp drop on some of 4 CPU sockets machines.

Quoted 4 lines

This looks fine.

Did you try including blocked_load_avg in only get_rq_runnable_load() [ and not weighted_cpuload() which is called by new-idle ]?

Quoted 8 lines

Looking at this more this feels less correct since you're taking averages of averages.

This was previously discussed at:
https://lkml.org/lkml/2013/5/6/109

And you later replied suggesting this didn't seem to hurt; what's the current status there?

Wasn't there a follow up series (currently as RFC or so) that proposes to removes the entire *_idx stuff?

author
#47 lore raw

Re: [patch v8 6/9] sched: compute runnable load avg in cpu_load and cpu_avg_load_per_task

Quoted 9 lines · Peter Zijlstra
Quoted 9 lines
Quoted 9 lines
Quoted 5 lines
Quoted 5 lines

This looks fine.

Did you try including blocked_load_avg in only get_rq_runnable_load() [ and not weighted_cpuload() which is called by new-idle ]?

Looking at this more this feels less correct since you're taking averages of averages.

Quoted 7 lines
Quoted 5 lines
Quoted 5 lines

This was previously discussed at:
https://lkml.org/lkml/2013/5/6/109

And you later replied suggesting this didn't seem to hurt; what's the current status there?

Wasn't there a follow up series (currently as RFC or so) that proposes to removes the entire *_idx stuff?

Yes, it is in my plan. just currently I am kicked ass to do other internal Intel tasks.

Signature · 2 lines
Thanks
    Alex
author
#48 lore raw

Re: [patch v8 6/9] sched: compute runnable load avg in cpu_load and cpu_avg_load_per_task

In reply to Paul Turner
Quoted 16 lines · Paul Turner
Quoted 10 lines · Paul Turner
Quoted 8 lines · Alex Shi

They are the base values in load balance, update them with rq runnable load average, then the load balance will consider runnable load avg naturally.

We also try to include the blocked_load_avg as cpu load in balancing, but that cause kbuild performance drop 6% on every Intel machine, and aim7/oltp drop on some of 4 CPU sockets machines.

Quoted 4 lines

This looks fine.

Did you try including blocked_load_avg in only get_rq_runnable_load() [ and not weighted_cpuload() which is called by new-idle ]?

Quoted 8 lines

Looking at this more this feels less correct since you're taking averages of averages.

This was previously discussed at:
https://lkml.org/lkml/2013/5/6/109

And you later replied suggesting this didn't seem to hurt; what's the current status there?

Yes, your example show the blocked_load_avg value.
So I had given a patch for review at that time before do detailed testing. https://lkml.org/lkml/2013/5/7/66

But in detailed testing, the patch cause a big performance regression. When I look into for details. I found some cpu in kbuild just had a big blocked_load_avg, with a very small runnable_load_avg value.

Seems accumulating current blocked_load_avg into cpu load isn't a good idea. Because:
1, The blocked_load_avg is decayed same as runnable load, sometime is far bigger than runnable load, that drive tasks to other idle or slight load cpu, than cause both performance and power issue. But if the blocked load is decayed too fast, it lose its effect.
2, Another issue of blocked load is that when waking up task, we can not know blocked load proportion of the task on rq. So, the blocked load is meaningless in wake affine decision.

According to above problem, I give up to enable blocked_load_avg in balance.

Quoted 83 lines
Quoted 83 lines
Quoted 83 lines
---
 kernel/sched/fair.c |  5 +++--
 kernel/sched/proc.c | 17 +++++++++++++++--
 2 files changed, 18 insertions(+), 4 deletions(-)
kernel/sched/fair.c +3 -2
index 42c7be0..eadd2e7 100644
@@ -2962,7 +2962,7 @@ static void dequeue_task_fair(struct rq *rq, struct task_struct *p, int flags)
 /* Used instead of source_load when we know the type == 0 */
 static unsigned long weighted_cpuload(const int cpu)
 {
-       return cpu_rq(cpu)->load.weight;
+       return cpu_rq(cpu)->cfs.runnable_load_avg;
 }
 
 /*
@@ -3007,9 +3007,10 @@ static unsigned long cpu_avg_load_per_task(int cpu)
 {
        struct rq *rq = cpu_rq(cpu);
        unsigned long nr_running = ACCESS_ONCE(rq->nr_running);
+       unsigned long load_avg = rq->cfs.runnable_load_avg;
 
        if (nr_running)
-               return rq->load.weight / nr_running;
+               return load_avg / nr_running;
 
        return 0;
 }
kernel/sched/proc.c +15 -3
index bb3a6a0..ce5cd48 100644
@@ -501,6 +501,18 @@ static void __update_cpu_load(struct rq *this_rq, unsigned long this_load,
        sched_avg_update(this_rq);
 }
 
+#ifdef CONFIG_SMP
+unsigned long get_rq_runnable_load(struct rq *rq)
+{
+       return rq->cfs.runnable_load_avg;
+}
+#else
+unsigned long get_rq_runnable_load(struct rq *rq)
+{
+       return rq->load.weight;
+}
+#endif
+
 #ifdef CONFIG_NO_HZ_COMMON
 /*
  * There is no sane way to deal with nohz on smp when using jiffies because the
@@ -522,7 +534,7 @@ static void __update_cpu_load(struct rq *this_rq, unsigned long this_load,
 void update_idle_cpu_load(struct rq *this_rq)
 {
        unsigned long curr_jiffies = ACCESS_ONCE(jiffies);
-       unsigned long load = this_rq->load.weight;
+       unsigned long load = get_rq_runnable_load(this_rq);
        unsigned long pending_updates;
 
        /*
@@ -568,11 +580,12 @@ void update_cpu_load_nohz(void)
  */
 void update_cpu_load_active(struct rq *this_rq)
 {
+       unsigned long load = get_rq_runnable_load(this_rq);
        /*
         * See the mess around update_idle_cpu_load() / update_cpu_load_nohz().
         */
        this_rq->last_load_update_tick = jiffies;
-       __update_cpu_load(this_rq, this_rq->load.weight, 1);
+       __update_cpu_load(this_rq, load, 1);
 
        calc_load_account_active(this_rq);
 }
--

1.7.12

Signature · 2 lines
Thanks
    Alex
#49 lore raw

Re: [patch v8 6/9] sched: compute runnable load avg in cpu_load and cpu_avg_load_per_task

Quoted 30 lines · Alex Shi
Quoted 16 lines · Paul Turner
Quoted 10 lines · Paul Turner
Quoted 8 lines · Alex Shi

They are the base values in load balance, update them with rq runnable load average, then the load balance will consider runnable load avg naturally.

We also try to include the blocked_load_avg as cpu load in balancing, but that cause kbuild performance drop 6% on every Intel machine, and aim7/oltp drop on some of 4 CPU sockets machines.

Quoted 4 lines

This looks fine.

Did you try including blocked_load_avg in only get_rq_runnable_load() [ and not weighted_cpuload() which is called by new-idle ]?

Quoted 8 lines

Looking at this more this feels less correct since you're taking averages of averages.

This was previously discussed at:
https://lkml.org/lkml/2013/5/6/109

And you later replied suggesting this didn't seem to hurt; what's the current status there?

Yes, your example show the blocked_load_avg value.
So I had given a patch for review at that time before do detailed testing. https://lkml.org/lkml/2013/5/7/66

Quoted 6 lines

But in detailed testing, the patch cause a big performance regression. When I look into for details. I found some cpu in kbuild just had a big blocked_load_avg, with a very small runnable_load_avg value.

Seems accumulating current blocked_load_avg into cpu load isn't a good idea. Because:

So I think this describes an alternate implementation to the one suggested in:
https://lkml.org/lkml/2013/5/7/66

Specifically, we _don't_ want to accumulate into cpu-load. Taking an "average of the average" loses the mobility that the new representation allows.

Quoted 4 lines

1, The blocked_load_avg is decayed same as runnable load, sometime is far bigger than runnable load, that drive tasks to other idle or slight load cpu, than cause both performance and power issue. But if the blocked load is decayed too fast, it lose its effect.

This is why the idea would be to use an instantaneous load in weighted_cpuload() and one that incorporated averages on (wants a rename) get_rq_runnable_load().

For non-instaneous load-indexes we're pulling for stability.

Quoted 3 lines

2, Another issue of blocked load is that when waking up task, we can not know blocked load proportion of the task on rq. So, the blocked load is meaningless in wake affine decision.

I think this is confusing two things:

(a) A wake-idle wake-up
(b) A wake-affine wake-up

In (a) we do not care about the blocked load proportion, only whether a cpu is idle.

But once (a) has failed we should absolutely care how much load is blocked in (b) as:
- We know we're going to queue for bandwidth on the cpu [ otherwise we'd be in (a) ]
- Blocked load predicts how much _other_ work is expected to also share the queue with us during the quantum

Quoted 92 lines

According to above problem, I give up to enable blocked_load_avg in balance.

Quoted 78 lines
Quoted 78 lines
Quoted 78 lines
---
 kernel/sched/fair.c |  5 +++--
 kernel/sched/proc.c | 17 +++++++++++++++--
 2 files changed, 18 insertions(+), 4 deletions(-)
kernel/sched/fair.c +3 -2
index 42c7be0..eadd2e7 100644
@@ -2962,7 +2962,7 @@ static void dequeue_task_fair(struct rq *rq, struct task_struct *p, int flags)
 /* Used instead of source_load when we know the type == 0 */
 static unsigned long weighted_cpuload(const int cpu)
 {
-       return cpu_rq(cpu)->load.weight;
+       return cpu_rq(cpu)->cfs.runnable_load_avg;
 }
 
 /*
@@ -3007,9 +3007,10 @@ static unsigned long cpu_avg_load_per_task(int cpu)
 {
        struct rq *rq = cpu_rq(cpu);
        unsigned long nr_running = ACCESS_ONCE(rq->nr_running);
+       unsigned long load_avg = rq->cfs.runnable_load_avg;
 
        if (nr_running)
-               return rq->load.weight / nr_running;
+               return load_avg / nr_running;
 
        return 0;
 }
kernel/sched/proc.c +15 -2
index bb3a6a0..ce5cd48 100644
@@ -501,6 +501,18 @@ static void __update_cpu_load(struct rq *this_rq, unsigned long this_load,
        sched_avg_update(this_rq);
 }
 
+#ifdef CONFIG_SMP
+unsigned long get_rq_runnable_load(struct rq *rq)
+{
+       return rq->cfs.runnable_load_avg;
+}
+#else
+unsigned long get_rq_runnable_load(struct rq *rq)
+{
+       return rq->load.weight;
+}
+#endif
+
 #ifdef CONFIG_NO_HZ_COMMON
 /*
  * There is no sane way to deal with nohz on smp when using jiffies because the
@@ -522,7 +534,7 @@ static void __update_cpu_load(struct rq *this_rq, unsigned long this_load,
 void update_idle_cpu_load(struct rq *this_rq)
 {
        unsigned long curr_jiffies = ACCESS_ONCE(jiffies);
-       unsigned long load = this_rq->load.weight;
+       unsigned long load = get_rq_runnable_load(this_rq);
        unsigned long pending_updates;
 
        /*
@@ -568,11 +580,12 @@ void update_cpu_load_nohz(void)
  */
 void update_cpu_load_active(struct rq *this_rq)
 {
+       unsigned long load = get_rq_runnable_load(this_rq);
        /*
         * See the mess around update_idle_cpu_load() / update_cpu_load_nohz().
         */
        this_rq->last_load_update_tick = jiffies;
-       __update_cpu_load(this_rq, this_rq->load.weight, 1);
+       __update_cpu_load(this_rq, load, 1);
 
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
kernel/sched/proc.c +0 -1
        calc_load_account_active(this_rq);
 }
--

1.7.12

--
Thanks
Alex

author
#50 lore raw

Re: [patch v8 6/9] sched: compute runnable load avg in cpu_load and cpu_avg_load_per_task

Quoted 48 lines · Paul Turner
Quoted 30 lines · Alex Shi
Quoted 30 lines · Paul Turner
Quoted 16 lines
Quoted 16 lines · Paul Turner
Quoted 14 lines
Quoted 14 lines · Alex Shi
Quoted 8 lines
Quoted 8 lines

They are the base values in load balance, update them with rq runnable load average, then the load balance will consider runnable load avg naturally.

We also try to include the blocked_load_avg as cpu load in balancing, but that cause kbuild performance drop 6% on every Intel machine, and aim7/oltp drop on some of 4 CPU sockets machines.

This looks fine.

Did you try including blocked_load_avg in only get_rq_runnable_load() [ and not weighted_cpuload() which is called by new-idle ]?

Quoted 8 lines
Quoted 8 lines

Looking at this more this feels less correct since you're taking averages of averages.

This was previously discussed at:
https://lkml.org/lkml/2013/5/6/109

And you later replied suggesting this didn't seem to hurt; what's the current status there?

Yes, your example show the blocked_load_avg value.
So I had given a patch for review at that time before do detailed testing. https://lkml.org/lkml/2013/5/7/66

Quoted 6 lines
Quoted 6 lines

But in detailed testing, the patch cause a big performance regression. When I look into for details. I found some cpu in kbuild just had a big blocked_load_avg, with a very small runnable_load_avg value.

Seems accumulating current blocked_load_avg into cpu load isn't a good idea. Because:

So I think this describes an alternate implementation to the one suggested in:
https://lkml.org/lkml/2013/5/7/66

Specifically, we _don't_ want to accumulate into cpu-load. Taking an "average of the average" loses the mobility that the new representation allows.

Quoted 4 lines
Quoted 4 lines

1, The blocked_load_avg is decayed same as runnable load, sometime is far bigger than runnable load, that drive tasks to other idle or slight load cpu, than cause both performance and power issue. But if the blocked load is decayed too fast, it lose its effect.

Quoted 5 lines

This is why the idea would be to use an instantaneous load in weighted_cpuload() and one that incorporated averages on (wants a rename) get_rq_runnable_load().

For non-instaneous load-indexes we're pulling for stability.

Paul, could I summary your point here:
keep current weighted_cpu_load, but add blocked load avg in get_rq_runnable_load?

I will test this change.

Quoted 8 lines
Quoted 4 lines
Quoted 4 lines

2, Another issue of blocked load is that when waking up task, we can not know blocked load proportion of the task on rq. So, the blocked load is meaningless in wake affine decision.

I think this is confusing two things:

(a) A wake-idle wake-up
(b) A wake-affine wake-up

what's I mean the wake affine is (b). Anyway, blocked load is no help on the scenario.

Quoted 11 lines

In (a) we do not care about the blocked load proportion, only whether a cpu is idle.

But once (a) has failed we should absolutely care how much load is blocked in (b) as:
- We know we're going to queue for bandwidth on the cpu [ otherwise we'd be in (a) ]
- Blocked load predicts how much _other_ work is expected to also share the queue with us during the quantum

Signature · 2 lines
Thanks
    Alex
author
#51 lore raw

Re: [patch v8 6/9] sched: compute runnable load avg in cpu_load and cpu_avg_load_per_task

Quoted 7 lines

Paul, could I summary your point here:
keep current weighted_cpu_load, but add blocked load avg in get_rq_runnable_load?

I will test this change.

Current testing(kbuild, oltp, aim7) don't show clear different on my NHM EP box between the following and the origin patch,
the only different is get_rq_runnable_load added blocked_load_avg. in SMP will test more cases and more box.

---
 kernel/sched/fair.c |  5 +++--
 kernel/sched/proc.c | 17 +++++++++++++++--
 2 files changed, 18 insertions(+), 4 deletions(-)
kernel/sched/fair.c +3 -2
index 1e5a5e6..7d5c477 100644
@@ -2968,7 +2968,7 @@ static void dequeue_task_fair(struct rq *rq, struct task_struct *p, int flags)
 /* Used instead of source_load when we know the type == 0 */
 static unsigned long weighted_cpuload(const int cpu)
 {
-	return cpu_rq(cpu)->load.weight;
+	return cpu_rq(cpu)->cfs.runnable_load_avg;
 }
 
 /*
@@ -3013,9 +3013,10 @@ static unsigned long cpu_avg_load_per_task(int cpu)
 {
 	struct rq *rq = cpu_rq(cpu);
 	unsigned long nr_running = ACCESS_ONCE(rq->nr_running);
+	unsigned long load_avg = rq->cfs.runnable_load_avg;
 
 	if (nr_running)
-		return rq->load.weight / nr_running;
+		return load_avg / nr_running;
 
 	return 0;
 }
kernel/sched/proc.c +15 -2
index bb3a6a0..36d7db6 100644
@@ -501,6 +501,18 @@ static void __update_cpu_load(struct rq *this_rq, unsigned long this_load,
 	sched_avg_update(this_rq);
 }
 
+#ifdef CONFIG_SMP
+unsigned long get_rq_runnable_load(struct rq *rq)
+{
+	return rq->cfs.runnable_load_avg + rq->cfs.blocked_load_avg;
+}
+#else
+unsigned long get_rq_runnable_load(struct rq *rq)
+{
+	return rq->load.weight;
+}
+#endif
+
 #ifdef CONFIG_NO_HZ_COMMON
 /*
  * There is no sane way to deal with nohz on smp when using jiffies because the
@@ -522,7 +534,7 @@ static void __update_cpu_load(struct rq *this_rq, unsigned long this_load,
 void update_idle_cpu_load(struct rq *this_rq)
 {
 	unsigned long curr_jiffies = ACCESS_ONCE(jiffies);
-	unsigned long load = this_rq->load.weight;
+	unsigned long load = get_rq_runnable_load(this_rq);
 	unsigned long pending_updates;
 
 	/*
@@ -568,11 +580,12 @@ void update_cpu_load_nohz(void)
  */
 void update_cpu_load_active(struct rq *this_rq)
 {
+	unsigned long load = get_rq_runnable_load(this_rq);
 	/*
 	 * See the mess around update_idle_cpu_load() / update_cpu_load_nohz().
 	 */
 	this_rq->last_load_update_tick = jiffies;
-	__update_cpu_load(this_rq, this_rq->load.weight, 1);
+	__update_cpu_load(this_rq, load, 1);
 
 	calc_load_account_active(this_rq);
 }
Signature · 5 lines
1.7.12

-- 
Thanks
    Alex
author
#52 lore raw

Re: [patch v8 6/9] sched: compute runnable load avg in cpu_load and cpu_avg_load_per_task

Quoted 6 lines · Alex Shi
Quoted 6 lines

Paul, could I summary your point here:
keep current weighted_cpu_load, but add blocked load avg in get_rq_runnable_load?

Quoted 6 lines
Quoted 1 line

I will test this change.

Current testing(kbuild, oltp, aim7) don't show clear different on my NHM EP box between the following and the origin patch,
the only different is get_rq_runnable_load added blocked_load_avg. in SMP will test more cases and more box.

I tested the tip/sched/core, tip/sched/core with old patchset and tip/schec/core with the blocked_load_avg on Core2 2S, NHM EP, IVB EP, SNB EP 2S and SNB EP 4S box, with benchmark kbuild, sysbench oltp, hackbench, tbench, dbench.

blocked_load_avg VS origin patchset, oltp has suspicious 5% and hackbench has 3% drop on NHM EX; dbench has suspicious 6% drop on NHM EP. other benchmarks has no clear change on all other machines.

origin patchset VS sched/core, hackbench rise 20% on NHM EX, 60% on SNB EP 4S, and 30% on IVB EP. others no clear changes.

Quoted 78 lines
---
 kernel/sched/fair.c |  5 +++--
 kernel/sched/proc.c | 17 +++++++++++++++--
 2 files changed, 18 insertions(+), 4 deletions(-)
kernel/sched/fair.c +3 -2
index 1e5a5e6..7d5c477 100644
@@ -2968,7 +2968,7 @@ static void dequeue_task_fair(struct rq *rq, struct task_struct *p, int flags)
 /* Used instead of source_load when we know the type == 0 */
 static unsigned long weighted_cpuload(const int cpu)
 {
-	return cpu_rq(cpu)->load.weight;
+	return cpu_rq(cpu)->cfs.runnable_load_avg;
 }
 
 /*
@@ -3013,9 +3013,10 @@ static unsigned long cpu_avg_load_per_task(int cpu)
 {
 	struct rq *rq = cpu_rq(cpu);
 	unsigned long nr_running = ACCESS_ONCE(rq->nr_running);
+	unsigned long load_avg = rq->cfs.runnable_load_avg;
 
 	if (nr_running)
-		return rq->load.weight / nr_running;
+		return load_avg / nr_running;
 
 	return 0;
 }
kernel/sched/proc.c +15 -2
index bb3a6a0..36d7db6 100644
@@ -501,6 +501,18 @@ static void __update_cpu_load(struct rq *this_rq, unsigned long this_load,
 	sched_avg_update(this_rq);
 }
 
+#ifdef CONFIG_SMP
+unsigned long get_rq_runnable_load(struct rq *rq)
+{
+	return rq->cfs.runnable_load_avg + rq->cfs.blocked_load_avg;
+}
+#else
+unsigned long get_rq_runnable_load(struct rq *rq)
+{
+	return rq->load.weight;
+}
+#endif
+
 #ifdef CONFIG_NO_HZ_COMMON
 /*
  * There is no sane way to deal with nohz on smp when using jiffies because the
@@ -522,7 +534,7 @@ static void __update_cpu_load(struct rq *this_rq, unsigned long this_load,
 void update_idle_cpu_load(struct rq *this_rq)
 {
 	unsigned long curr_jiffies = ACCESS_ONCE(jiffies);
-	unsigned long load = this_rq->load.weight;
+	unsigned long load = get_rq_runnable_load(this_rq);
 	unsigned long pending_updates;
 
 	/*
@@ -568,11 +580,12 @@ void update_cpu_load_nohz(void)
  */
 void update_cpu_load_active(struct rq *this_rq)
 {
+	unsigned long load = get_rq_runnable_load(this_rq);
 	/*
 	 * See the mess around update_idle_cpu_load() / update_cpu_load_nohz().
 	 */
 	this_rq->last_load_update_tick = jiffies;
-	__update_cpu_load(this_rq, this_rq->load.weight, 1);
+	__update_cpu_load(this_rq, load, 1);
 
 	calc_load_account_active(this_rq);
 }
Signature · 2 lines
Thanks
    Alex
author
#53 lore raw

Re: [patch v8 6/9] sched: compute runnable load avg in cpu_load and cpu_avg_load_per_task

Quoted 23 lines · Alex Shi
Quoted 6 lines · Alex Shi
Quoted 6 lines

Paul, could I summary your point here:
keep current weighted_cpu_load, but add blocked load avg in get_rq_runnable_load?

Quoted 6 lines
Quoted 1 line

I will test this change.

Current testing(kbuild, oltp, aim7) don't show clear different on my NHM EP box between the following and the origin patch,
the only different is get_rq_runnable_load added blocked_load_avg. in SMP will test more cases and more box.

I tested the tip/sched/core, tip/sched/core with old patchset and tip/schec/core with the blocked_load_avg on Core2 2S, NHM EP, IVB EP, SNB EP 2S and SNB EP 4S box, with benchmark kbuild, sysbench oltp, hackbench, tbench, dbench.

blocked_load_avg VS origin patchset, oltp has suspicious 5% and hackbench has 3% drop on NHM EX; dbench has suspicious 6% drop on NHM EP. other benchmarks has no clear change on all other machines.

Quoted 8 lines

origin patchset VS sched/core, hackbench rise 20% on NHM EX, 60% on SNB EP 4S, and 30% on IVB EP. others no clear changes.

kernel/sched/proc.c static void __update_cpu_load(struct rq *this_rq, unsigned long this_load, · line 507

Quoted 4 lines
kernel/sched/proc.c +4 -0
+#ifdef CONFIG_SMP
+unsigned long get_rq_runnable_load(struct rq *rq)
+{
+	return rq->cfs.runnable_load_avg + rq->cfs.blocked_load_avg;

According to above testing result, with blocked_load_avg is still a slight worse than without it.

when the blocked_load_avg added here, it will impact nohz idle balance and periodic balance in update_sg_lb_stats() when the idx is not 0. As to nohz idle balance, blocked_load_avg should be too small to have big effect.
As to in update_sg_lb_stats(), since it only works when _idx is not 0, that means the blocked_load_avg was decay again in update_cpu_load. That reduce its impact.

So, could I say, at least in above testing, blocked_load_avg should be keep away from balance?

Signature · 2 lines
Thanks
    Alex
author
#54 lore raw

Re: [patch v8 6/9] sched: compute runnable load avg in cpu_load and cpu_avg_load_per_task

In reply to Paul Turner
Quoted 5 lines · Paul Turner

This looks fine.

Did you try including blocked_load_avg in only get_rq_runnable_load() [ and not weighted_cpuload() which is called by new-idle ]?

No, but blocked_load_avg also impact periodic balance much. The ideal scenario is a sleeping task will be waken up on same cpu. But like kbuild benchmark, the quick cc/ld task are forked, and then finished and disappeared quickly. blocked_load_avg cause trouble in kbuild.

Signature · 2 lines
Thanks
    Alex
author
#55 lore raw

Re: [patch v8 6/9] sched: compute runnable load avg in cpu_load and cpu_avg_load_per_task

Quoted 7 lines · Alex Shi
Quoted 4 lines
Quoted 4 lines

Did you try including blocked_load_avg in only get_rq_runnable_load() [ and not weighted_cpuload() which is called by new-idle ]?

No, but blocked_load_avg also impact periodic balance much. The ideal scenario is a sleeping task will be waken up on same cpu. But like kbuild benchmark, the quick cc/ld task are forked, and then finished and

a bit more explain, cc/ld finished cpu task then wait IO to write object files into disk.

Quoted 1 line

disappeared quickly. blocked_load_avg cause trouble in kbuild.

Signature · 2 lines
Thanks
    Alex
author
#56 lore raw

PATCH v8 7/9 math64: add div64_ul macro

In reply to Alex Shi

There are div64_long() to handle the s64/long division. But no macro do u64/ul division in kernel.
It is necessary on some scenarios. So add this macro. And fixed the coding style error.

---
 include/linux/math64.h | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
include/linux/math64.h +4 -2
index b8ba855..2913b86 100644
@@ -6,7 +6,8 @@
 
 #if BITS_PER_LONG == 64
 
-#define div64_long(x,y) div64_s64((x),(y))
+#define div64_long(x, y) div64_s64((x), (y))
+#define div64_ul(x, y)   div64_u64((x), (y))
 
 /**
  * div_u64_rem - unsigned 64bit divide with 32bit divisor with remainder
@@ -47,7 +48,8 @@ static inline s64 div64_s64(s64 dividend, s64 divisor)
 
 #elif BITS_PER_LONG == 32
 
-#define div64_long(x,y) div_s64((x),(y))
+#define div64_long(x, y) div_s64((x), (y))
+#define div64_ul(x, y)   div_u64((x), (y))
 
 #ifndef div_u64_rem
 static inline u64 div_u64_rem(u64 dividend, u32 divisor, u32 *remainder)
Signature · 1 line
1.7.12
author
#57 lore raw

PATCH v8 8/9 sched: consider runnable load average in move_tasks

In reply to Alex Shi

Except using runnable load average in background, move_tasks is also the key functions in load balance. We need consider the runnable load average in it in order to the apple to apple load comparison.

Morten had caught a div u64 bug on ARM, thanks!

---
 kernel/sched/fair.c | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)
kernel/sched/fair.c +8 -8
index eadd2e7..3aa1dc0 100644
@@ -4178,11 +4178,14 @@ static int tg_load_down(struct task_group *tg, void *data)
 	long cpu = (long)data;
 
 	if (!tg->parent) {
-		load = cpu_rq(cpu)->load.weight;
+		load = cpu_rq(cpu)->avg.load_avg_contrib;
 	} else {
+		unsigned long tmp_rla;
+		tmp_rla = tg->parent->cfs_rq[cpu]->runnable_load_avg + 1;
+
 		load = tg->parent->cfs_rq[cpu]->h_load;
-		load *= tg->se[cpu]->load.weight;
-		load /= tg->parent->cfs_rq[cpu]->load.weight + 1;
+		load *= tg->se[cpu]->avg.load_avg_contrib;
+		load /= tmp_rla;
 	}
 
 	tg->cfs_rq[cpu]->h_load = load;
@@ -4208,12 +4211,9 @@ static void update_h_load(long cpu)
 static unsigned long task_h_load(struct task_struct *p)
 {
 	struct cfs_rq *cfs_rq = task_cfs_rq(p);
-	unsigned long load;
-
-	load = p->se.load.weight;
-	load = div_u64(load * cfs_rq->h_load, cfs_rq->load.weight + 1);
 
-	return load;
+	return div64_ul(p->se.avg.load_avg_contrib * cfs_rq->h_load,
+			cfs_rq->runnable_load_avg + 1);
 }
 #else
 static inline void update_blocked_averages(int cpu)
Signature · 1 line
1.7.12
#58 lore raw

Re: [patch v8 8/9] sched: consider runnable load average in move_tasks

Quoted 20 lines · Alex Shi

Except using runnable load average in background, move_tasks is also the key functions in load balance. We need consider the runnable load average in it in order to the apple to apple load comparison.

Morten had caught a div u64 bug on ARM, thanks!

---
 kernel/sched/fair.c | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)
kernel/sched/fair.c +0 -1
index eadd2e7..3aa1dc0 100644
@@ -4178,11 +4178,14 @@ static int tg_load_down(struct task_group *tg, void *data)
        long cpu = (long)data;
 
        if (!tg->parent) {
-               load = cpu_rq(cpu)->load.weight;

kernel/sched/fair.c static int tg_load_down(struct task_group *tg, void *data) · line 4188

Quoted 10 lines
kernel/sched/fair.c +6 -2
+               load = cpu_rq(cpu)->avg.load_avg_contrib;
        } else {
+               unsigned long tmp_rla;
+               tmp_rla = tg->parent->cfs_rq[cpu]->runnable_load_avg + 1;
+
                load = tg->parent->cfs_rq[cpu]->h_load;
-               load *= tg->se[cpu]->load.weight;
-               load /= tg->parent->cfs_rq[cpu]->load.weight + 1;
+               load *= tg->se[cpu]->avg.load_avg_contrib;
+               load /= tmp_rla;

Why do we need the temporary here?

Quoted 21 lines
kernel/sched/fair.c +2 -6
        }
 
        tg->cfs_rq[cpu]->h_load = load;
@@ -4208,12 +4211,9 @@ static void update_h_load(long cpu)
 static unsigned long task_h_load(struct task_struct *p)
 {
        struct cfs_rq *cfs_rq = task_cfs_rq(p);
-       unsigned long load;
-
-       load = p->se.load.weight;
-       load = div_u64(load * cfs_rq->h_load, cfs_rq->load.weight + 1);
 
-       return load;
+       return div64_ul(p->se.avg.load_avg_contrib * cfs_rq->h_load,
+                       cfs_rq->runnable_load_avg + 1);
 }
 #else
 static inline void update_blocked_averages(int cpu)
--

1.7.12

author
#59 lore raw

Re: [patch v8 8/9] sched: consider runnable load average in move_tasks

Quoted 10 lines · Paul Turner
Quoted 8 lines

kernel/sched/fair.c static int tg_load_down(struct task_group *tg, void *data) · line 4188

Quoted 8 lines
kernel/sched/fair.c +5 -2
+               unsigned long tmp_rla;
+               tmp_rla = tg->parent->cfs_rq[cpu]->runnable_load_avg + 1;
+
                load = tg->parent->cfs_rq[cpu]->h_load;
-               load *= tg->se[cpu]->load.weight;
-               load /= tg->parent->cfs_rq[cpu]->load.weight + 1;
+               load *= tg->se[cpu]->avg.load_avg_contrib;
+               load /= tmp_rla;

Why do we need the temporary here?

you'r right. tmp_rla is not necessary here.

Signature · 2 lines
Thanks
    Alex
author
#60 lore raw

Re: [patch v8 8/9] sched: consider runnable load average in move_tasks

Quoted 9 lines · Alex Shi
Quoted 8 lines · Paul Turner
Quoted 8 lines
Quoted 8 lines
kernel/sched/fair.c +5 -2
+               unsigned long tmp_rla;
+               tmp_rla = tg->parent->cfs_rq[cpu]->runnable_load_avg + 1;
+
                load = tg->parent->cfs_rq[cpu]->h_load;
-               load *= tg->se[cpu]->load.weight;
-               load /= tg->parent->cfs_rq[cpu]->load.weight + 1;
+               load *= tg->se[cpu]->avg.load_avg_contrib;
+               load /= tmp_rla;
Quoted 5 lines
Quoted 2 lines

Why do we need the temporary here?

you'r right. tmp_rla is not necessary here.

Sorry, not, tmp_rla is a imply type casting. otherwise long /= u64; has building issue on 32 bit according to Morten.
and a clear type casting cross 2 lines.

Signature · 2 lines
Thanks
    Alex
maintainer
#61 lore raw

Re: [patch v8 8/9] sched: consider runnable load average in move_tasks

In reply to Alex Shi
Quoted 38 lines · Alex Shi

Except using runnable load average in background, move_tasks is also the key functions in load balance. We need consider the runnable load average in it in order to the apple to apple load comparison.

Morten had caught a div u64 bug on ARM, thanks!

---
 kernel/sched/fair.c | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)
kernel/sched/fair.c +6 -4
index eadd2e7..3aa1dc0 100644
@@ -4178,11 +4178,14 @@ static int tg_load_down(struct task_group *tg, void *data)
 	long cpu = (long)data;
 
 	if (!tg->parent) {
-		load = cpu_rq(cpu)->load.weight;
+		load = cpu_rq(cpu)->avg.load_avg_contrib;
 	} else {
+		unsigned long tmp_rla;
+		tmp_rla = tg->parent->cfs_rq[cpu]->runnable_load_avg + 1;
+
 		load = tg->parent->cfs_rq[cpu]->h_load;
-		load *= tg->se[cpu]->load.weight;
-		load /= tg->parent->cfs_rq[cpu]->load.weight + 1;
+		load *= tg->se[cpu]->avg.load_avg_contrib;
+		load /= tmp_rla;
 	}
 
 	tg->cfs_rq[cpu]->h_load = load;
@@ -4208,12 +4211,9 @@ static void update_h_load(long cpu)
 static unsigned long task_h_load(struct task_struct *p)
 {
 	struct cfs_rq *cfs_rq = task_cfs_rq(p);
-	unsigned long load;

kernel/sched/fair.c static void update_h_load(long cpu) · line 4219

Quoted 10 lines
kernel/sched/fair.c +2 -4
-
-	load = p->se.load.weight;
-	load = div_u64(load * cfs_rq->h_load, cfs_rq->load.weight + 1);
 
-	return load;
+	return div64_ul(p->se.avg.load_avg_contrib * cfs_rq->h_load,
+			cfs_rq->runnable_load_avg + 1);
 }
 #else
 static inline void update_blocked_averages(int cpu)

Should we not also change the !FAIR_GROUP_SCHED version of task_h_load() for this?

author
#62 lore raw

Re: [patch v8 8/9] sched: consider runnable load average in move_tasks

Quoted 4 lines · Peter Zijlstra
Quoted 2 lines
Quoted 2 lines

#else
static inline void update_blocked_averages(int cpu)

Should we not also change the !FAIR_GROUP_SCHED version of task_h_load() for this?

yes, it is needed. Sorry for omit this.
---

Quoted 1 line

From dc6a4b052c05d793938f6fbf40190d8cec959ca4 Mon Sep 17 00:00:00 2001

From: Alex Shi <[email protected]>
Date: Mon, 3 Dec 2012 23:00:53 +0800
Subject: [PATCH 1/2] sched: consider runnable load average in move_tasks

Except using runnable load average in background, move_tasks is also the key functions in load balance. We need consider the runnable load average in it in order to the apple to apple load comparison.

Morten had caught a div u64 bug on ARM, thanks!

---
 kernel/sched/fair.c |   18 +++++++++---------
 1 files changed, 9 insertions(+), 9 deletions(-)
kernel/sched/fair.c +9 -9
index 4b68f9f..0ca9cc3 100644
@@ -4179,11 +4179,14 @@ static int tg_load_down(struct task_group *tg, void *data)
 	long cpu = (long)data;
 
 	if (!tg->parent) {
-		load = cpu_rq(cpu)->load.weight;
+		load = cpu_rq(cpu)->avg.load_avg_contrib;
 	} else {
+		unsigned long tmp_rla;
+		tmp_rla = tg->parent->cfs_rq[cpu]->runnable_load_avg + 1;
+
 		load = tg->parent->cfs_rq[cpu]->h_load;
-		load *= tg->se[cpu]->load.weight;
-		load /= tg->parent->cfs_rq[cpu]->load.weight + 1;
+		load *= tg->se[cpu]->avg.load_avg_contrib;
+		load /= tmp_rla;
 	}
 
 	tg->cfs_rq[cpu]->h_load = load;
@@ -4209,12 +4212,9 @@ static void update_h_load(long cpu)
 static unsigned long task_h_load(struct task_struct *p)
 {
 	struct cfs_rq *cfs_rq = task_cfs_rq(p);
-	unsigned long load;
-
-	load = p->se.load.weight;
-	load = div_u64(load * cfs_rq->h_load, cfs_rq->load.weight + 1);
 
-	return load;
+	return div64_ul(p->se.avg.load_avg_contrib * cfs_rq->h_load,
+			cfs_rq->runnable_load_avg + 1);
 }
 #else
 static inline void update_blocked_averages(int cpu)
@@ -4227,7 +4227,7 @@ static inline void update_h_load(long cpu)
 
 static unsigned long task_h_load(struct task_struct *p)
 {
-	return p->se.load.weight;
+	return p->se.avg.load_avg_contrib;
 }
 #endif
 
Signature · 1 line
1.7.5.4
author
#63 lore raw

PATCH v8 9/9 sched/tg: remove blocked_load_avg in balance

In reply to Alex Shi

blocked_load_avg sometime is too heavy and far bigger than runnable load avg, that make balance make wrong decision. So remove it.

Changlong tested this patch, found ltp cgroup stress testing get better
performance: https://lkml.org/lkml/2013/5/23/65
---
3.10-rc1 patch1-7 patch1-8

duration=764   duration=754   duration=750
duration=764   duration=754   duration=751
duration=763   duration=755   duration=751

duration means the seconds of testing cost.
---

And Jason also tested this patchset on his 8 sockets machine:
https://lkml.org/lkml/2013/5/29/673
---
When using a 3.10-rc2 tip kernel with patches 1-8, there was about a 40% improvement in performance of the workload compared to when using the vanilla 3.10-rc2 tip kernel with no patches. When using a 3.10-rc2 tip kernel with just patches 1-7, the performance improvement of the workload over the vanilla 3.10-rc2 tip kernel was about 25%.
---

---
 kernel/sched/fair.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/sched/fair.c +1 -1
index 3aa1dc0..985d47e 100644
@@ -1358,7 +1358,7 @@ static inline void __update_cfs_rq_tg_load_contrib(struct cfs_rq *cfs_rq,
 	struct task_group *tg = cfs_rq->tg;
 	s64 tg_contrib;
 
-	tg_contrib = cfs_rq->runnable_load_avg + cfs_rq->blocked_load_avg;
+	tg_contrib = cfs_rq->runnable_load_avg;
 	tg_contrib -= cfs_rq->tg_load_contrib;
 
 	if (force_update || abs64(tg_contrib) > cfs_rq->tg_load_contrib / 8) {
Signature · 1 line
1.7.12
maintainer
#64 lore raw

Re: [patch v8 9/9] sched/tg: remove blocked_load_avg in balance

Quoted 36 lines · Alex Shi

blocked_load_avg sometime is too heavy and far bigger than runnable load avg, that make balance make wrong decision. So remove it.

Changlong tested this patch, found ltp cgroup stress testing get better
performance: https://lkml.org/lkml/2013/5/23/65
---
3.10-rc1 patch1-7 patch1-8

duration=764   duration=754   duration=750
duration=764   duration=754   duration=751
duration=763   duration=755   duration=751

duration means the seconds of testing cost.
---

And Jason also tested this patchset on his 8 sockets machine:
https://lkml.org/lkml/2013/5/29/673
---
When using a 3.10-rc2 tip kernel with patches 1-8, there was about a 40% improvement in performance of the workload compared to when using the vanilla 3.10-rc2 tip kernel with no patches. When using a 3.10-rc2 tip kernel with just patches 1-7, the performance improvement of the workload over the vanilla 3.10-rc2 tip kernel was about 25%.
---

---
 kernel/sched/fair.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/sched/fair.c +0 -0
index 3aa1dc0..985d47e 100644

kernel/sched/fair.c static inline void __update_cfs_rq_tg_load_contrib(struct cfs_rq *cfs_rq, · line 1364

Quoted 8 lines
kernel/sched/fair.c +1 -1
@@ -1358,7 +1358,7 @@ static inline void __update_cfs_rq_tg_load_contrib(struct cfs_rq *cfs_rq,
 	struct task_group *tg = cfs_rq->tg;
 	s64 tg_contrib;
 
-	tg_contrib = cfs_rq->runnable_load_avg + cfs_rq->blocked_load_avg;
+	tg_contrib = cfs_rq->runnable_load_avg;
 	tg_contrib -= cfs_rq->tg_load_contrib;
 
 	if (force_update || abs64(tg_contrib) > cfs_rq->tg_load_contrib / 8) {

PJT ping! ;-)

#65 lore raw

Re: [patch v8 9/9] sched/tg: remove blocked_load_avg in balance

In reply to Alex Shi
Quoted 2 lines · Alex Shi

blocked_load_avg sometime is too heavy and far bigger than runnable load avg, that make balance make wrong decision. So remove it.

Ok so this is going to have terrible effects on the correctness of shares distribution; I'm fairly opposed to it in its present form.

So let's see, what could be happening..

In "sched: compute runnable load avg in cpu_load and cpu_avg_load_per_task" you already update the load average weights solely based on current runnable load. While this is generally poor for stability (and I suspect the benefit is coming largely from weighted_cpuload() where you do want to use runnable_load_avg and not get_rq_runnable_load() where I suspect including blocked_load_avg() is correct in the longer term).

Ah so.. I have an inkling:
Inside weighted_cpuload() where you're trying to use only
runnable_load_avg; this is in-fact still including blocked_load_avg for a cgroup since in the cgroup case a group entities' contribution is a function of both runnable and blocked load.

Having weighted_cpuload() pull rq->load (possibly moderated by rq->avg) would reasonably avoid this since issued shares are calculated using instantaneous weights, without breaking the actual model for how much load overall that we believe the group has.

Quoted 45 lines

Changlong tested this patch, found ltp cgroup stress testing get better
performance: https://lkml.org/lkml/2013/5/23/65
---
3.10-rc1 patch1-7 patch1-8

duration=764   duration=754   duration=750
duration=764   duration=754   duration=751
duration=763   duration=755   duration=751

duration means the seconds of testing cost.
---

And Jason also tested this patchset on his 8 sockets machine:
https://lkml.org/lkml/2013/5/29/673
---
When using a 3.10-rc2 tip kernel with patches 1-8, there was about a 40% improvement in performance of the workload compared to when using the vanilla 3.10-rc2 tip kernel with no patches. When using a 3.10-rc2 tip kernel with just patches 1-7, the performance improvement of the workload over the vanilla 3.10-rc2 tip kernel was about 25%.
---

---
 kernel/sched/fair.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/sched/fair.c +1 -2
index 3aa1dc0..985d47e 100644
@@ -1358,7 +1358,7 @@ static inline void __update_cfs_rq_tg_load_contrib(struct cfs_rq *cfs_rq,
        struct task_group *tg = cfs_rq->tg;
        s64 tg_contrib;
 
-       tg_contrib = cfs_rq->runnable_load_avg + cfs_rq->blocked_load_avg;
+       tg_contrib = cfs_rq->runnable_load_avg;
        tg_contrib -= cfs_rq->tg_load_contrib;
 
        if (force_update || abs64(tg_contrib) > cfs_rq->tg_load_contrib / 8) {
--

1.7.12

maintainer
#66 lore raw

Re: [patch v8 9/9] sched/tg: remove blocked_load_avg in balance

Quoted 6 lines · Paul Turner
Quoted 2 lines · Alex Shi

blocked_load_avg sometime is too heavy and far bigger than runnable load avg, that make balance make wrong decision. So remove it.

Ok so this is going to have terrible effects on the correctness of shares distribution; I'm fairly opposed to it in its present form.

Should someone take a look at the LTP cgroup tests to make sure they cover this properly?

author
#67 lore raw

Re: [patch v8 9/9] sched/tg: remove blocked_load_avg in balance

Quoted 9 lines · Peter Zijlstra
Quoted 6 lines · Paul Turner
Quoted 6 lines · Alex Shi
Quoted 2 lines
Quoted 2 lines

blocked_load_avg sometime is too heavy and far bigger than runnable load avg, that make balance make wrong decision. So remove it.

Ok so this is going to have terrible effects on the correctness of shares distribution; I'm fairly opposed to it in its present form.

Should someone take a look at the LTP cgroup tests to make sure they cover this properly?

Seems I have no time on this in near future. :(

But I were you, I would picked up this patch, since 1, it is simple and easy revert.
2, it wins in testing.
3, it will motivate Paul to look into this problem. ;)

Signature · 2 lines
Thanks
    Alex
author
#68 lore raw

Re: [patch v8 9/9] sched/tg: remove blocked_load_avg in balance

In reply to Paul Turner
Quoted 8 lines · Paul Turner
Quoted 2 lines · Alex Shi
Quoted 2 lines

blocked_load_avg sometime is too heavy and far bigger than runnable load avg, that make balance make wrong decision. So remove it.

Ok so this is going to have terrible effects on the correctness of shares distribution; I'm fairly opposed to it in its present form.

So let's see, what could be happening..

Quoted 7 lines

In "sched: compute runnable load avg in cpu_load and cpu_avg_load_per_task" you already update the load average weights solely based on current runnable load. While this is generally poor for stability (and I suspect the benefit is coming largely from weighted_cpuload() where you do want to use runnable_load_avg and not get_rq_runnable_load() where I suspect including blocked_load_avg() is correct in the longer term).

If the 'poor stability' means your previous example of 2 40% busy task and one 90% busy task. It occasionally happens. but at least in all testing, kbuild, aim7, tbench, oltp, hackbench, ltp etc. involve blocked_load_avg is just worse, guess due to above reason.

Quoted 6 lines

Ah so.. I have an inkling:
Inside weighted_cpuload() where you're trying to use only
runnable_load_avg; this is in-fact still including blocked_load_avg for a cgroup since in the cgroup case a group entities' contribution is a function of both runnable and blocked load.

with this patch tg will not include blocked_load_avg.

Honestly, blocked_load_avg should has its meaning, like in your scenario. but just now, we only can see it bring more harm without any help on all we tested benchmarks.
I can't find a reason to enable sth that hurt performance.

Quoted 6 lines

Having weighted_cpuload() pull rq->load (possibly moderated by rq->avg) would reasonably avoid this since issued shares are calculated using instantaneous weights, without breaking the actual model for how much load overall that we believe the group has.

I considered to use rq->avg in weighted_cpuload, but when we do move_tasks to balance load between cpu, we just consider the cfs tasks not rt task, consider rq->load/avg will involved a unnecessary rt interference. So I changed to cfs load only.

Signature · 2 lines
Thanks
    Alex
author
#69 lore raw
In reply to Alex Shi
Quoted 4 lines · Alex Shi

Peter&Ingo:

v8: add a marco div64_ul and used it in task_h_load()
v7: rebasing on tip/sched/core tree.

Peter & Ingo:

IMHO, if the patch set missed in 3.11 kernel. We will lose the following widely benefit on many benchmarks, hackbench, pgbench, sysbench, anonymous java load ...

What's your opinions? :)

Quoted 60 lines

I tested on Intel core2, NHM, SNB, IVB, 2 and 4 sockets machines with benchmark kbuild, aim7, dbench, tbench, hackbench, oltp, and netperf loopback etc.

On SNB EP 4 sockets machine, the hackbench increased about 50%, and result become stable. on other machines, hackbench increased about 2~10%. oltp increased about 30% in NHM EX box. netperf loopback also increased on SNB EP 4 sockets box.
No clear changes on other benchmarks.

Michael Wang gotten better performance on pgbench on his box with this patchset. https://lkml.org/lkml/2013/5/16/82

And Morten tested previous version with better power consumption.
http://comments.gmane.org/gmane.linux.kernel/1463371

Changlong found ltp cgroup stress testing get faster on SNB EP machine with the last patch. https://lkml.org/lkml/2013/5/23/65
---
3.10-rc1 patch1-7 patch1-8

duration=764   duration=754   duration=750
duration=764   duration=754   duration=751
duration=763   duration=755   duration=751

duration means the seconds of testing cost.
---

Jason also found java server load benefited on his 8 sockets machine with last patch. https://lkml.org/lkml/2013/5/29/673
---
When using a 3.10-rc2 tip kernel with patches 1-8, there was about a 40% improvement in performance of the workload compared to when using the vanilla 3.10-rc2 tip kernel with no patches. When using a 3.10-rc2 tip kernel with just patches 1-7, the performance improvement of the workload over the vanilla 3.10-rc2 tip kernel was about 25%.
---

We also tried to include blocked load avg in balance. but find many benchmark performance drop a lot! Seems accumulating current blocked_load_avg into cpu load isn't a good idea. Because:
1, The blocked_load_avg is decayed same as runnable load, sometime is far bigger than runnable load, that drive tasks to other idle or slight load cpu, than cause both performance and power issue. But if the blocked load is decayed too fast, it lose its effect.
2, Another issue of blocked load is that when waking up task, we can not know blocked load proportion of the task on rq. So, the blocked load is meaningless in wake affine decision.

According to above problem, we can not figure out a right way now to use blocked_load_avg in balance.

Since using runnable load avg in balance brings much benefit on performance and power. and this patch was reviewed for long time. So seems it's time to let it be clobbered in some sub tree, like tip or linux-next. Any comments?

Regards
Alex

Signature · 2 lines
Thanks
    Alex
author
#70 lore raw
In reply to Alex Shi

Ingo & Peter:

How are you?
I know you are very very busy since services for wide key kernel parts. So hope this email don't disturb you. :)

PJT's runnable load was introduced into kernel from 3.8 kernel. But besides increase longer code path, it does nothing help now.

This patch try to enable the runnable load into scheduler balance. Most of this patchset are bug fixing related except 6~8th which are direct service for introducing runnable load into balance.
The patchset was talked in LKML from last Nov, adopted many suggestions from PTJ, MikeG, Morten, PeterZ etc. Also tested on x86 and arm platforms, both got positive feedback.

With this patchset, CFS scheduler will has better and stabler performance on hackbench, sysbench, pgbench. etc benchmarks. You will like the performance number which mentioned in last emails.

I appreciated for any further review and comments!

Wish you all best!
Alex

Quoted 64 lines · Alex Shi

Peter&Ingo:

v8: add a marco div64_ul and used it in task_h_load()
v7: rebasing on tip/sched/core tree.

I tested on Intel core2, NHM, SNB, IVB, 2 and 4 sockets machines with benchmark kbuild, aim7, dbench, tbench, hackbench, oltp, and netperf loopback etc.

On SNB EP 4 sockets machine, the hackbench increased about 50%, and result become stable. on other machines, hackbench increased about 2~10%. oltp increased about 30% in NHM EX box. netperf loopback also increased on SNB EP 4 sockets box.
No clear changes on other benchmarks.

Michael Wang gotten better performance on pgbench on his box with this patchset. https://lkml.org/lkml/2013/5/16/82

And Morten tested previous version with better power consumption.
http://comments.gmane.org/gmane.linux.kernel/1463371

Changlong found ltp cgroup stress testing get faster on SNB EP machine with the last patch. https://lkml.org/lkml/2013/5/23/65
---
3.10-rc1 patch1-7 patch1-8

duration=764   duration=754   duration=750
duration=764   duration=754   duration=751
duration=763   duration=755   duration=751

duration means the seconds of testing cost.
---

Jason also found java server load benefited on his 8 sockets machine with last patch. https://lkml.org/lkml/2013/5/29/673
---
When using a 3.10-rc2 tip kernel with patches 1-8, there was about a 40% improvement in performance of the workload compared to when using the vanilla 3.10-rc2 tip kernel with no patches. When using a 3.10-rc2 tip kernel with just patches 1-7, the performance improvement of the workload over the vanilla 3.10-rc2 tip kernel was about 25%.
---

We also tried to include blocked load avg in balance. but find many benchmark performance drop a lot! Seems accumulating current blocked_load_avg into cpu load isn't a good idea. Because:
1, The blocked_load_avg is decayed same as runnable load, sometime is far bigger than runnable load, that drive tasks to other idle or slight load cpu, than cause both performance and power issue. But if the blocked load is decayed too fast, it lose its effect.
2, Another issue of blocked load is that when waking up task, we can not know blocked load proportion of the task on rq. So, the blocked load is meaningless in wake affine decision.

According to above problem, we can not figure out a right way now to use blocked_load_avg in balance.

Since using runnable load avg in balance brings much benefit on performance and power. and this patch was reviewed for long time. So seems it's time to let it be clobbered in some sub tree, like tip or linux-next. Any comments?

Regards
Alex

Signature · 2 lines
Thanks
    Alex
maintainer
#71 lore raw
In reply to Alex Shi
Quoted 1 line · Alex Shi

Peter&Ingo:

I'm tempted to apply 1-7; although I want to look them over in detail once more; unfortunately (for you) I'm on PTO this week so it'll have to be early next week.

As to patch 8, I really need/want buy-in from Paul on that; it removes the entire point of the blocked load stuff.

author
#72 lore raw
Quoted 6 lines · Peter Zijlstra
Quoted 1 line · Alex Shi

Peter&Ingo:

I'm tempted to apply 1-7; although I want to look them over in detail once more; unfortunately (for you) I'm on PTO this week so it'll have to be early next week.

I am very happy to know you like them! :)

BTW, I am sorry for that previous version patch 7th, 'add div64_ul' was added into mm-tree few hours ago.

Quoted 3 lines

As to patch 8, I really need/want buy-in from Paul on that; it removes the entire point of the blocked load stuff.

'remove blocked_load_avg' should be 9th patch now. Yes, I also am waiting for Paul's comments.

Wish you all best!

Signature · 2 lines
Thanks
    Alex