sysbench+mysql(oltp, readonly) 30% regression with 2.6.26-rc1
Subsystems (2)
Comparing with kernel 2.6.25, sysbench+mysql(oltp, readonly) has many regression with 2.6.26-rc1.
1) 8-core stoakley: 28%;
2) 16-core tigerton: 20%;
3) Itanium Montvale: 50%.
Bisect located below patch.
8f1bc385cfbab474db6c27b5af1e439614f3025c is first bad commit commit 8f1bc385cfbab474db6c27b5af1e439614f3025c
Author: Peter Zijlstra <[email protected]> Date: Sat Apr 19 19:45:00 2008 +0200
sched: fair: weight calculations
In order to level the hierarchy, we need to calculate load based on the root view. That is, each task's load is in the same unit.
After I manually reverted the patch against 2.6.26-rc1 while fixing a couple of conflictions/errors, sysbench oltp regression became less than 3% on 8-core stoakley.
-yanmin
Quoted 8 lines · Zhang, Yanmin
???Comparing with kernel 2.6.25, sysbench+mysql(oltp, readonly) has many regression with 2.6.26-rc1.
1) 8-core stoakley: 28%;
2) 16-core tigerton: 20%;
3) Itanium Montvale: 50%.Bisect located below patch.
thanks Yanmin, i've queued up your reverter patch.
Ingo
Quoted 5 lines · Ingo Molnar
Quoted 3 lines · Zhang, Yanmin
???Comparing with kernel 2.6.25, sysbench+mysql(oltp, readonly) has many regression with 2.6.26-rc1.
Quoted 7 lines
Quoted 5 lines
1) 8-core stoakley: 28%;
2) 16-core tigerton: 20%;
3) Itanium Montvale: 50%.Bisect located below patch.
thanks Yanmin, i've queued up your reverter patch.
Sorry. The reverting patch has a comment-out block. I need delete it if you queue the patch officially.
kernel/sched.c +6 -3
@@ -1429,9 +1429,6 @@ static void __resched_task(struct task_s*/ #define SRR (x, y) (((x) + (1UL << ((y) - 1))) >> (y)) -/* -* delta *= weight / lw -*/ static unsigned long calc_delta_mine (unsigned long delta_exec,unsigned long weight,struct load_weight *lw)
@@ -1454,6 +1451,12 @@ calc_delta_mine(unsigned long delta_execreturn (unsigned long )min (tmp, (u64 )(unsigned long )LONG_MAX ); } +static inline unsigned long +calc_delta_fair (unsigned long delta_exec,struct load_weight *lw) +{ +return calc_delta_mine (delta_exec,NICE_0_LOAD , lw); +} +static inline void update_load_add (struct load_weight *lw,unsigned long inc) { lw->weight += inc;
kernel/sched_fair.c +33 -67
156 lines not on this page · raw message
Quoted 4 lines · Zhang, Yanmin
Quoted 1 line
thanks Yanmin, i've queued up your reverter patch.
Sorry. The reverting patch has a comment-out block. I need delete it if you queue the patch officially.
yep, i noticed that and skipped that bit already.
Ingo
Quoted 16 lines · Yanmin
Comparing with kernel 2.6.25, sysbench+mysql(oltp, readonly) has many regression with 2.6.26-rc1.
1) 8-core stoakley: 28%;
2) 16-core tigerton: 20%;
3) Itanium Montvale: 50%.Bisect located below patch.
8f1bc385cfbab474db6c27b5af1e439614f3025cis first bad commit commit8f1bc385cfbab474db6c27b5af1e439614f3025cAuthor: Peter Zijlstra <[email protected]> Date: Sat Apr 19 19:45:00 2008 +0200sched: fair: weight calculations
Quoted 8 lines
In order to level the hierarchy, we need to calculate load based on the root view. That is, each task's load is in the same unit.
After I manually reverted the patch against 2.6.26-rc1 while fixing a couple of conflictions/errors, sysbench oltp regression became less than 3% on 8-core stoakley.
Does this patch help?
---
From: Mike Galbraith <[email protected]>
Subject: sched: fix weight calculations
The conversion between virtual and real time is as follows:
dvt = rw/w * dt <=> dt = w/rw * dvt
Since we want the fair sleeper granularity to be in real time, we actually need to do:
dvt = - rw/w * l
--- kernel/sched_fair.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)
kernel/sched_fair.c +8 -3
@@ -662,10 +662,15 @@ place_entity(struct cfs_rq *cfs_rq, struif (!initial) {/* sleeps upto a single latency don't count. */ if (sched_feat (NEW_FAIR_SLEEPERS )) { +unsigned long thresh = sysctl_sched_latency; + +/* +* convert the sleeper threshold into virtual time +*/ if (sched_feat (NORMALIZED_SLEEPER )) - vruntime -=calc_delta_weight (sysctl_sched_latency, se); -else - vruntime -= sysctl_sched_latency; + thresh =calc_delta_fair (thresh, se); + + vruntime -= thresh; }/* ensure we never gain time by being placed backwards. */
Quoted 22 lines · Peter Zijlstra
Quoted 21 lines · Yanmin
Comparing with kernel 2.6.25, sysbench+mysql(oltp, readonly) has many regression with 2.6.26-rc1.
1) 8-core stoakley: 28%;
2) 16-core tigerton: 20%;
3) Itanium Montvale: 50%.Bisect located below patch.
8f1bc385cfbab474db6c27b5af1e439614f3025cis first bad commit commit8f1bc385cfbab474db6c27b5af1e439614f3025cAuthor: Peter Zijlstra <[email protected]> Date: Sat Apr 19 19:45:00 2008 +0200sched: fair: weight calculations
In order to level the hierarchy, we need to calculate load based on the root view. That is, each task's load is in the same unit.
Quoted 5 lines
Quoted 3 lines
After I manually reverted the patch against 2.6.26-rc1 while fixing a couple of conflictions/errors, sysbench oltp regression became less than 3% on 8-core stoakley.
Does this patch help?
With the patch, oltp testing result is about 50% worse than the one of pure 2.6.26-rc1.
cpu idle with 16 threads( a parameter of sysbench):
1) 2.6.25: 1%
2) 2.6.26-rc1: 33%
3) 2.6.26-rc1+new_patch: 70%.
Quoted 44 lines
---
From: Mike Galbraith <[email protected]>
Subject: sched: fix weight calculationsThe conversion between virtual and real time is as follows:
dvt = rw/w * dt <=> dt = w/rw * dvt
Since we want the fair sleeper granularity to be in real time, we actually need to do:
dvt = - rw/w * l
Signed-off-by Peter Zijlstra <[email protected]>--- kernel/sched_fair.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)Index:
linux-2.6-2/kernel/sched_fair.c
===================================================================kernel/sched_fair.c +8
-3@@ -662,10 +662,15 @@ place_entity(struct cfs_rq *cfs_rq, struif (!initial) {/* sleeps upto a single latency don't count. */ if (sched_feat (NEW_FAIR_SLEEPERS )) { +unsigned long thresh = sysctl_sched_latency; + +/* +* convert the sleeper threshold into virtual time +*/ if (sched_feat (NORMALIZED_SLEEPER )) - vruntime -=calc_delta_weight (sysctl_sched_latency, se); -else - vruntime -= sysctl_sched_latency; + thresh =calc_delta_fair (thresh, se); + + vruntime -= thresh; }/* ensure we never gain time by being placed backwards. */
Quoted 23 lines · Yanmin
Quoted 22 lines · Peter Zijlstra
Quoted 21 lines · Yanmin
Comparing with kernel 2.6.25, sysbench+mysql(oltp, readonly) has many regression with 2.6.26-rc1.
1) 8-core stoakley: 28%;
2) 16-core tigerton: 20%;
3) Itanium Montvale: 50%.Bisect located below patch.
8f1bc385cfbab474db6c27b5af1e439614f3025cis first bad commit commit8f1bc385cfbab474db6c27b5af1e439614f3025cAuthor: Peter Zijlstra <[email protected]> Date: Sat Apr 19 19:45:00 2008 +0200sched: fair: weight calculations
In order to level the hierarchy, we need to calculate load based on the root view. That is, each task's load is in the same unit.
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
After I manually reverted the patch against 2.6.26-rc1 while fixing a couple of conflictions/errors, sysbench oltp regression became less than 3% on 8-core stoakley.
Does this patch help?
With the patch, oltp testing result is about 50% worse than the one of pure 2.6.26-rc1.
Hm. I was doing some sysbench+postgress(oltp, ro) testing on my little Q6600 box this morning, and saw a different picture.
In attached pdf, .bkl refers to Linus' BKL patch, .weight is the weight fix, both are applied to git.today. The script I used is also attached.
-Mike
Quoted 9 lines · Mike Galbraith
Quoted 2 lines
With the patch, oltp testing result is about 50% worse than the one of pure 2.6.26-rc1.
Hm. I was doing some sysbench+postgress(oltp, ro) testing on my little Q6600 box this morning, and saw a different picture.
In attached pdf, .bkl refers to Linus' BKL patch, .weight is the weight fix, both are applied to git.today. The script I used is also attached.
nice numbers. I'm curious about the following detail related to Linus's BKL-spinlock patch: am i correct that it shows a small but systematic improvement with many clients? sysbench has not shown sensitivity to any BKL detail before.
Could you perhaps try the hack below that uses /proc/sys/kernel/panic as a flag whether the BKL should cause us to block or should be spun upon.
Can you confirm that with that patch too sysbench shows sensitivity to the value of the panic flag? [this is the surest way to measure such effects as flipping the sysctl only minimally impacts the system.]
if sysbench truly is dependent on the BKL then that's another argument in favor of Linus's patch.
Ingo
-------------------------->
Subject: BKL: spin on acquire
From: Ingo Molnar <[email protected]>
Date: Wed May 07 19:05:40 CEST 2008
--- lib/kernel_lock.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-)
lib/kernel_lock.c +10 -3
@@ -46,7 +46,10 @@ int __lockfunc __reacquire_kernel_lock(v task->lock_depth =-1 ;preempt_enable_no_resched (); -down (&kernel_sem); +if (panic_timeout) { +while (down_trylock (&kernel_sem)) +cpu_relax (); + }preempt_disable (); task->lock_depth = saved_lock_depth;
@@ -67,11 +70,15 @@ void __lockfunc lock_kernel(void)struct task_struct *task = current;int depth = task->lock_depth +1 ; -if (likely (!depth)) +if (likely (!depth)) {/* * No recursion worries - we set up lock_depth _after_ */ -down (&kernel_sem); +if (panic_timeout) { +while (down_trylock (&kernel_sem)) +cpu_relax (); + } + } task->lock_depth = depth; }
Review
Quoted 4 lines · Ingo Molnar
(quoted diff) +3
-1- down(&kernel_sem); + if (panic_timeout) { + while (down_trylock(&kernel_sem)) + cpu_relax();
updated one below - this will work fine in the !panic_timeout case too ;-)
Ingo
lib/kernel_lock.c +14 -3
@@ -46,7 +46,12 @@ int __lockfunc __reacquire_kernel_lock(v task->lock_depth =-1 ;preempt_enable_no_resched (); -down (&kernel_sem); +if (panic_timeout) { +while (down_trylock (&kernel_sem)) +cpu_relax (); + }else { +down (&kernel_sem); + }preempt_disable (); task->lock_depth = saved_lock_depth;
@@ -67,11 +72,17 @@ void __lockfunc lock_kernel(void)struct task_struct *task = current;int depth = task->lock_depth +1 ; -if (likely (!depth)) +if (likely (!depth)) {/* * No recursion worries - we set up lock_depth _after_ */ -down (&kernel_sem); +if (panic_timeout) { +while (down_trylock (&kernel_sem)) +cpu_relax (); + }else { +down (&kernel_sem); + } + } task->lock_depth = depth; }
Quoted 9 lines · Ingo Molnar
lib/kernel_lock.c int __lockfunc __reacquire_kernel_lock(v · line 51
Quoted 4 lines · Ingo Molnar
lib/kernel_lock.c +3
-1-down (&kernel_sem); +if (panic_timeout) { +while (down_trylock (&kernel_sem)) +cpu_relax ();updated one below - this will work fine in the !panic_timeout case too ;-)
Same result.
-Mike
Quoted 6 lines · Ingo Molnar
Could you perhaps try the hack below that uses
/proc/sys/kernel/panicas a flag whether the BKL should cause us to block or should be spun upon.Can you confirm that with that patch too sysbench shows sensitivity to the value of the panic flag? [this is the surest way to measure such effects as flipping the sysctl only minimally impacts the system.]
Difference for average of 3 runs at 128 clients is 0.3% with this patch.. iow, nope.
-Mike
Quoted 28 lines · Mike Galbraith
Quoted 27 lines · Yanmin
Quoted 26 lines · Peter Zijlstra
Quoted 24 lines · Yanmin
Comparing with kernel 2.6.25, sysbench+mysql(oltp, readonly) has many regression with 2.6.26-rc1.
1) 8-core stoakley: 28%;
2) 16-core tigerton: 20%;
3) Itanium Montvale: 50%.Bisect located below patch.
8f1bc385cfbab474db6c27b5af1e439614f3025cis first bad commit commit8f1bc385cfbab474db6c27b5af1e439614f3025cAuthor: Peter Zijlstra <[email protected]> Date: Sat Apr 19 19:45:00 2008 +0200sched: fair: weight calculations
In order to level the hierarchy, we need to calculate load based on the root view. That is, each task's load is in the same unit.
After I manually reverted the patch against 2.6.26-rc1 while fixing a couple of conflictions/errors, sysbench oltp regression became less than 3% on 8-core stoakley.
Quoted 6 lines
Quoted 3 lines
Quoted 1 line
Does this patch help?
With the patch, oltp testing result is about 50% worse than the one of pure 2.6.26-rc1.
Hm. I was doing some sysbench+postgress(oltp, ro) testing on my little Q6600 box this morning, and saw a different picture.
How many cpu are in the Q6600?
Quoted 3 lines
In attached pdf, .bkl refers to Linus' BKL patch, .weight is the weight fix, both are applied to git.today. The script I used is also attached.
With my 8-core stoakley (using mysql):
1) 2.6.25:
Number of threads: 6
read/write requests: 8025024 (66874.53 per sec.)
Number of threads: 8
read/write requests: 9132816 (76106.14 per sec.)
Number of threads: 10
read/write requests: 9244998 (77040.75 per sec.)
Number of threads: 12
read/write requests: 8994174 (74950.36 per sec.)
Number of threads: 14
read/write requests: 9051322 (75426.54 per sec.)
Number of threads: 16
read/write requests: 9015412 (75126.93 per sec.)2) 2.6.26-rc1:
Number of threads: 6
read/write requests: 5754056 (47949.87 per sec.)
Number of threads: 8
read/write requests: 6528480 (54403.29 per sec.)
Number of threads: 10
read/write requests: 6444690 (53705.16 per sec.)
Number of threads: 12
read/write requests: 6544258 (54534.23 per sec.)
Number of threads: 14
read/write requests: 6796650 (56637.65 per sec.)
Number of threads: 16
read/write requests: 6718110 (55983.18 per sec.)3) 2.6.26-rc1+weight
Number of threads: 16
read/write requests: 3219076 (26824.22 per sec.)
I'm not sure if more cpu could introduce more contention in this test.
-yanmin
Quoted 29 lines · Yanmin
Quoted 28 lines · Mike Galbraith
Quoted 27 lines · Yanmin
Quoted 26 lines · Peter Zijlstra
Quoted 24 lines · Yanmin
Comparing with kernel 2.6.25, sysbench+mysql(oltp, readonly) has many regression with 2.6.26-rc1.
1) 8-core stoakley: 28%;
2) 16-core tigerton: 20%;
3) Itanium Montvale: 50%.Bisect located below patch.
8f1bc385cfbab474db6c27b5af1e439614f3025cis first bad commit commit8f1bc385cfbab474db6c27b5af1e439614f3025cAuthor: Peter Zijlstra <[email protected]> Date: Sat Apr 19 19:45:00 2008 +0200sched: fair: weight calculations
In order to level the hierarchy, we need to calculate load based on the root view. That is, each task's load is in the same unit.
After I manually reverted the patch against 2.6.26-rc1 while fixing a couple of conflictions/errors, sysbench oltp regression became less than 3% on 8-core stoakley.
Quoted 7 lines
Quoted 6 lines
Quoted 3 lines
Quoted 1 line
Does this patch help?
With the patch, oltp testing result is about 50% worse than the one of pure 2.6.26-rc1.
Hm. I was doing some sysbench+postgress(oltp, ro) testing on my little Q6600 box this morning, and saw a different picture.
How many cpu are in the Q6600?
That's an Intel Quad core (Kentsfield).
Quoted 31 lines
Quoted 2 lines
In attached pdf, .bkl refers to Linus' BKL patch, .weight is the weight fix, both are applied to git.today. The script I used is also attached.
With my 8-core stoakley (using mysql):
1) 2.6.25:
Number of threads: 6read/write requests: 8025024 (66874.53 per sec.) Number of threads: 8 read/write requests: 9132816 (76106.14 per sec.) Number of threads: 10 read/write requests: 9244998 (77040.75 per sec.) Number of threads: 12 read/write requests: 8994174 (74950.36 per sec.) Number of threads: 14 read/write requests: 9051322 (75426.54 per sec.) Number of threads: 16 read/write requests: 9015412 (75126.93 per sec.)2) 2.6.26-rc1:
Number of threads: 6read/write requests: 5754056 (47949.87 per sec.) Number of threads: 8 read/write requests: 6528480 (54403.29 per sec.) Number of threads: 10 read/write requests: 6444690 (53705.16 per sec.) Number of threads: 12 read/write requests: 6544258 (54534.23 per sec.) Number of threads: 14 read/write requests: 6796650 (56637.65 per sec.) Number of threads: 16 read/write requests: 6718110 (55983.18 per sec.)
Quoted 5 lines
3) 2.6.26-rc1+weight
Number of threads: 16read/write requests: 3219076 (26824.22 per sec.)I'm not sure if more cpu could introduce more contention in this test.
those numbers make me sad :-(
Mike Galbraith
Tested-by Frans PopQuoted 6 lines · Yanmin
Quoted 2 lines · Mike Galbraith
Hm. I was doing some sysbench+postgress(oltp, ro) testing on my little Q6600 box this morning, and saw a different picture.
How many cpu are in the Q6600?
1.
Quoted 32 lines
Quoted 3 lines
In attached pdf, .bkl refers to Linus' BKL patch, .weight is the weight fix, both are applied to git.today. The script I used is also attached.
With my 8-core stoakley (using mysql):
1) 2.6.25:
Number of threads: 6read/write requests: 8025024 (66874.53 per sec.) Number of threads: 8 read/write requests: 9132816 (76106.14 per sec.) Number of threads: 10 read/write requests: 9244998 (77040.75 per sec.) Number of threads: 12 read/write requests: 8994174 (74950.36 per sec.) Number of threads: 14 read/write requests: 9051322 (75426.54 per sec.) Number of threads: 16 read/write requests: 9015412 (75126.93 per sec.)2) 2.6.26-rc1:
Number of threads: 6read/write requests: 5754056 (47949.87 per sec.) Number of threads: 8 read/write requests: 6528480 (54403.29 per sec.) Number of threads: 10 read/write requests: 6444690 (53705.16 per sec.) Number of threads: 12 read/write requests: 6544258 (54534.23 per sec.) Number of threads: 14 read/write requests: 6796650 (56637.65 per sec.) Number of threads: 16 read/write requests: 6718110 (55983.18 per sec.)
Quoted 5 lines
3) 2.6.26-rc1+weight
Number of threads: 16read/write requests: 3219076 (26824.22 per sec.)I'm not sure if more cpu could introduce more contention in this test.
-rc1. Do you have the fix below applied?
commit a992241de614dd2b7c97a9ba64e28c0e563f19bf
Author: Peter Zijlstra <[email protected]> Date: Mon May 5 23:56:17 2008 +0200
sched: fix normalized sleeper
Normalized sleeper uses calc_delta*() which requires that the rq load is already updated, so move account_entity_enqueue() before place_entity()
kernel/sched_fair.c +1 -1
@@ -682,6 +682,7 @@ enqueue_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int wakeup)* Update run-time statistics of the 'current'. */ update_curr (cfs_rq); +account_entity_enqueue (cfs_rq, se);if (wakeup) {place_entity (cfs_rq, se,0 );
@@ -692,7 +693,6 @@ enqueue_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int wakeup)check_spread (cfs_rq, se);if (se != cfs_rq->curr)__enqueue_entity (cfs_rq, se); -account_entity_enqueue (cfs_rq, se); }static void update_avg (u64 *avg,u64 sample)
Quoted 10 lines · Mike Galbraith
Quoted 6 lines · Yanmin
Quoted 2 lines · Mike Galbraith
Hm. I was doing some sysbench+postgress(oltp, ro) testing on my little Q6600 box this morning, and saw a different picture.
How many cpu are in the Q6600?
1.
Quoted 7 lines
Quoted 5 lines
3) 2.6.26-rc1+weight
Number of threads: 16read/write requests: 3219076 (26824.22 per sec.)I'm not sure if more cpu could introduce more contention in this test.
-rc1. Do you have the fix below applied?
No.
Quoted 36 lines
commit
a992241de614dd2b7c97a9ba64e28c0e563f19bfAuthor: Peter Zijlstra <[email protected]> Date: Mon May 5 23:56:17 2008 +0200sched: fix normalized sleeper
Normalized sleeper uses calc_delta*() which requires that the rq load is already updated, so move
account_entity_enqueue()beforeplace_entity()Tested-by Frans Pop <[email protected]>Signed-off-by Peter Zijlstra <[email protected]>Signed-off-by Ingo Molnar <[email protected]>kernel/sched_fair.c +1
-1@@ -682,6 +682,7 @@ enqueue_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int wakeup)* Update run-time statistics of the 'current'. */ update_curr (cfs_rq); +account_entity_enqueue (cfs_rq, se);if (wakeup) {place_entity (cfs_rq, se,0 );@@ -692,7 +693,6 @@ enqueue_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int wakeup)check_spread (cfs_rq, se);if (se != cfs_rq->curr)__enqueue_entity (cfs_rq, se); -account_entity_enqueue (cfs_rq, se); }static void update_avg (u64 *avg,u64 sample)
Quoted 42 lines · Mike Galbraith
Quoted 6 lines · Yanmin
Quoted 2 lines · Mike Galbraith
Hm. I was doing some sysbench+postgress(oltp, ro) testing on my little Q6600 box this morning, and saw a different picture.
How many cpu are in the Q6600?
1.
Quoted 32 lines
Quoted 3 lines
In attached pdf, .bkl refers to Linus' BKL patch, .weight is the weight fix, both are applied to git.today. The script I used is also attached.
With my 8-core stoakley (using mysql):
1) 2.6.25:
Number of threads: 6read/write requests: 8025024 (66874.53 per sec.) Number of threads: 8 read/write requests: 9132816 (76106.14 per sec.) Number of threads: 10 read/write requests: 9244998 (77040.75 per sec.) Number of threads: 12 read/write requests: 8994174 (74950.36 per sec.) Number of threads: 14 read/write requests: 9051322 (75426.54 per sec.) Number of threads: 16 read/write requests: 9015412 (75126.93 per sec.)2) 2.6.26-rc1:
Number of threads: 6read/write requests: 5754056 (47949.87 per sec.) Number of threads: 8 read/write requests: 6528480 (54403.29 per sec.) Number of threads: 10 read/write requests: 6444690 (53705.16 per sec.) Number of threads: 12 read/write requests: 6544258 (54534.23 per sec.) Number of threads: 14 read/write requests: 6796650 (56637.65 per sec.) Number of threads: 16 read/write requests: 6718110 (55983.18 per sec.)
Quoted 7 lines
Quoted 5 lines
3) 2.6.26-rc1+weight
Number of threads: 16read/write requests: 3219076 (26824.22 per sec.)I'm not sure if more cpu could introduce more contention in this test.
-rc1. Do you have the fix below applied?
Oooh - good catch, that seems to be a post -rc1 merge.
Yes this is required.
Quoted 35 lines
commit
a992241de614dd2b7c97a9ba64e28c0e563f19bfAuthor: Peter Zijlstra <[email protected]> Date: Mon May 5 23:56:17 2008 +0200sched: fix normalized sleeper
Normalized sleeper uses calc_delta*() which requires that the rq load is already updated, so move
account_entity_enqueue()beforeplace_entity()Tested-by Frans Pop <[email protected]>Signed-off-by Peter Zijlstra <[email protected]>Signed-off-by Ingo Molnar <[email protected]>kernel/sched_fair.c +1
-1@@ -682,6 +682,7 @@ enqueue_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int wakeup)* Update run-time statistics of the 'current'. */ update_curr (cfs_rq); +account_entity_enqueue (cfs_rq, se);if (wakeup) {place_entity (cfs_rq, se,0 );@@ -692,7 +693,6 @@ enqueue_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int wakeup)check_spread (cfs_rq, se);if (se != cfs_rq->curr)__enqueue_entity (cfs_rq, se); -account_entity_enqueue (cfs_rq, se); }static void update_avg (u64 *avg,u64 sample)
Quoted 47 lines · Peter Zijlstra
Quoted 46 lines · Mike Galbraith
Quoted 6 lines · Yanmin
Quoted 2 lines · Mike Galbraith
Hm. I was doing some sysbench+postgress(oltp, ro) testing on my little Q6600 box this morning, and saw a different picture.
How many cpu are in the Q6600?
1.
Quoted 36 lines
Quoted 3 lines
In attached pdf, .bkl refers to Linus' BKL patch, .weight is the weight fix, both are applied to git.today. The script I used is also attached.
With my 8-core stoakley (using mysql):
1) 2.6.25:
Number of threads: 6read/write requests: 8025024 (66874.53 per sec.) Number of threads: 8 read/write requests: 9132816 (76106.14 per sec.) Number of threads: 10 read/write requests: 9244998 (77040.75 per sec.) Number of threads: 12 read/write requests: 8994174 (74950.36 per sec.) Number of threads: 14 read/write requests: 9051322 (75426.54 per sec.) Number of threads: 16 read/write requests: 9015412 (75126.93 per sec.)2) 2.6.26-rc1:
Number of threads: 6read/write requests: 5754056 (47949.87 per sec.) Number of threads: 8 read/write requests: 6528480 (54403.29 per sec.) Number of threads: 10 read/write requests: 6444690 (53705.16 per sec.) Number of threads: 12 read/write requests: 6544258 (54534.23 per sec.) Number of threads: 14 read/write requests: 6796650 (56637.65 per sec.) Number of threads: 16 read/write requests: 6718110 (55983.18 per sec.)3) 2.6.26-rc1+weight
Number of threads: 16read/write requests: 3219076 (26824.22 per sec.)
Quoted 7 lines
Quoted 3 lines
Quoted 1 line
I'm not sure if more cpu could introduce more contention in this test.
-rc1. Do you have the fix below applied?
Oooh - good catch, that seems to be a post -rc1 merge.
Yes this is required.
With below patch+previous_fix_weight_calc, the result is improved, but is still about 10% worse than the one of pure 2.6.26-rc1.
Number of threads: 16
read/write requests: 6104336 (50867.11 per sec.)
Quoted 37 lines
Quoted 36 lines
commit
a992241de614dd2b7c97a9ba64e28c0e563f19bfAuthor: Peter Zijlstra <[email protected]> Date: Mon May 5 23:56:17 2008 +0200sched: fix normalized sleeper
Normalized sleeper uses calc_delta*() which requires that the rq load is already updated, so move
account_entity_enqueue()beforeplace_entity()Tested-by Frans Pop <[email protected]>Signed-off-by Peter Zijlstra <[email protected]>Signed-off-by Ingo Molnar <[email protected]>kernel/sched_fair.c +1
-1@@ -682,6 +682,7 @@ enqueue_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int wakeup)* Update run-time statistics of the 'current'. */ update_curr (cfs_rq); +account_entity_enqueue (cfs_rq, se);if (wakeup) {place_entity (cfs_rq, se,0 );@@ -692,7 +693,6 @@ enqueue_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int wakeup)check_spread (cfs_rq, se);if (se != cfs_rq->curr)__enqueue_entity (cfs_rq, se); -account_entity_enqueue (cfs_rq, se); }static void update_avg (u64 *avg,u64 sample)
Quoted 12 lines
Quoted 6 lines
Quoted 2 lines
-rc1. Do you have the fix below applied?
Oooh - good catch, that seems to be a post -rc1 merge.
Yes this is required.
With below patch+previous_fix_weight_calc, the result is improved, but is still about 10% worse than the one of pure 2.6.26-rc1.
Number of threads: 16read/write requests: 6104336 (50867.11 per sec.)
So puzzling - could you try -git + fix_weight_calc, as I think that is what Mike tested and proved good on his Quad.
Quoted 6 lines · Peter Zijlstra
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
-rc1. Do you have the fix below applied?
Oooh - good catch, that seems to be a post -rc1 merge.
Yes this is required.
Quoted 8 lines
Quoted 5 lines
With below patch+previous_fix_weight_calc, the result is improved, but is still about 10% worse than the one of pure 2.6.26-rc1.
Number of threads: 16read/write requests: 6104336 (50867.11 per sec.)So puzzling - could you try -git +
fix_weight_calc, as I think that is what Mike tested and proved good on his Quad.
If he still has group scheduling enabled, my box agrees that there's a regression. I got mysql running, and tested with group scheduling both enabled and disabled, results attached. (for some reason, I can only test up to 64 threads with mysql though)
-Mike
Quoted 13 lines · Mike Galbraith
Quoted 12 lines · Peter Zijlstra
Quoted 11 lines
Quoted 5 lines
Quoted 1 line
-rc1. Do you have the fix below applied?
Oooh - good catch, that seems to be a post -rc1 merge.
Yes this is required.
With below patch+previous_fix_weight_calc, the result is improved, but is still about 10% worse than the one of pure 2.6.26-rc1.
Number of threads: 16read/write requests: 6104336 (50867.11 per sec.)
Quoted 7 lines
Quoted 2 lines
So puzzling - could you try -git +
fix_weight_calc, as I think that is what Mike tested and proved good on his Quad.If he still has group scheduling enabled, my box agrees that there's a regression. I got mysql running, and tested with group scheduling both enabled and disabled, results attached. (for some reason, I can only test up to 64 threads with mysql though)
Hm, ok. Does the patch at hand improve or regress the group config for you?
Quoted 17 lines · Peter Zijlstra
Quoted 16 lines · Mike Galbraith
Quoted 14 lines · Peter Zijlstra
Quoted 11 lines
Quoted 5 lines
Quoted 1 line
-rc1. Do you have the fix below applied?
Oooh - good catch, that seems to be a post -rc1 merge.
Yes this is required.
With below patch+previous_fix_weight_calc, the result is improved, but is still about 10% worse than the one of pure 2.6.26-rc1.
Number of threads: 16read/write requests: 6104336 (50867.11 per sec.)So puzzling - could you try -git +
fix_weight_calc, as I think that is what Mike tested and proved good on his Quad.
Quoted 7 lines
Quoted 4 lines
If he still has group scheduling enabled, my box agrees that there's a regression. I got mysql running, and tested with group scheduling both enabled and disabled, results attached. (for some reason, I can only test up to 64 threads with mysql though)
Hm, ok. Does the patch at hand improve or regress the group config for you?
I'm testing got.today.
You mean 46151122e0a2e80e5a6b2889f595e371fe2b600d?
-Mike
Quoted 23 lines · Mike Galbraith
Quoted 22 lines · Peter Zijlstra
Quoted 20 lines · Mike Galbraith
Quoted 6 lines · Peter Zijlstra
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
-rc1. Do you have the fix below applied?
Oooh - good catch, that seems to be a post -rc1 merge.
Yes this is required.
Quoted 8 lines
Quoted 5 lines
With below patch+previous_fix_weight_calc, the result is improved, but is still about 10% worse than the one of pure 2.6.26-rc1.
Number of threads: 16read/write requests: 6104336 (50867.11 per sec.)So puzzling - could you try -git +
fix_weight_calc, as I think that is what Mike tested and proved good on his Quad.If he still has group scheduling enabled, my box agrees that there's a regression. I got mysql running, and tested with group scheduling both enabled and disabled, results attached. (for some reason, I can only test up to 64 threads with mysql though)
Quoted 6 lines
Quoted 2 lines
Hm, ok. Does the patch at hand improve or regress the group config for you?
I'm testing got.today.
You mean
46151122e0a2e80e5a6b2889f595e371fe2b600d?
Exactly - Yanmin says it makes it worse for him, I'm wondering if you can see a similar problem with group scheduling enabled.
Quoted 27 lines · Peter Zijlstra
Quoted 26 lines · Mike Galbraith
Quoted 24 lines · Peter Zijlstra
Quoted 13 lines · Mike Galbraith
Quoted 12 lines · Peter Zijlstra
Quoted 11 lines
Quoted 5 lines
Quoted 1 line
-rc1. Do you have the fix below applied?
Oooh - good catch, that seems to be a post -rc1 merge.
Yes this is required.
With below patch+previous_fix_weight_calc, the result is improved, but is still about 10% worse than the one of pure 2.6.26-rc1.
Number of threads: 16read/write requests: 6104336 (50867.11 per sec.)Quoted 7 lines
Quoted 2 lines
So puzzling - could you try -git +
fix_weight_calc, as I think that is what Mike tested and proved good on his Quad.If he still has group scheduling enabled, my box agrees that there's a regression. I got mysql running, and tested with group scheduling both enabled and disabled, results attached. (for some reason, I can only test up to 64 threads with mysql though)
Hm, ok. Does the patch at hand improve or regress the group config for you?
Quoted 6 lines
Quoted 3 lines
I'm testing got.today.
You mean
46151122e0a2e80e5a6b2889f595e371fe2b600d?Exactly - Yanmin says it makes it worse for him, I'm wondering if you can see a similar problem with group scheduling enabled.
Nope, same fugly curve +-jitter.
-Mike
Quoted 5 lines · Mike Galbraith
Quoted 3 lines · Peter Zijlstra
Quoted 3 lines
Quoted 2 lines
Hm, ok. Does the patch at hand improve or regress the group config for you?
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
I'm testing got.today.
You mean
46151122e0a2e80e5a6b2889f595e371fe2b600d?Exactly - Yanmin says it makes it worse for him, I'm wondering if you can see a similar problem with group scheduling enabled.
Nope, same fugly curve +-jitter.
Seems to be smpnice, see 2.6.26.git.grp.n- in attached.
-Mike
include/linux/sched.h +0 -1
@@ -766,7 +766,6 @@ struct sched_domain {struct sched_domain *child;/* bottom domain must be null terminated */ struct sched_group *groups;/* the balancing groups of the domain */ cpumask_t span;/* span of all CPUs in this domain */ -int first_cpu;/* cache of the first cpu in this domain */ unsigned long min_interval;/* Minimum balance interval ms */ unsigned long max_interval;/* Maximum balance interval ms */ unsigned int busy_factor;/* less balancing by factor if busy */
kernel/sched.c +31 -417
589 lines not on this page · raw message
kernel/sched_fair.c +44 -80
171 lines not on this page · raw message
kernel/sched_rt.c +0 -4
@@ -513,8 +513,6 @@ static void enqueue_task_rt(struct rq *r*/ for_each_sched_rt_entity (rt_se)enqueue_rt_entity (rt_se); - -inc_cpu_load (rq, p->se.load.weight); }static void dequeue_task_rt (struct rq *rq,struct task_struct *p,int sleep)
@@ -534,8 +532,6 @@ static void dequeue_task_rt(struct rq *rif (rt_rq && rt_rq->rt_nr_running)enqueue_rt_entity (rt_se); } - -dec_cpu_load (rq, p->se.load.weight); }/*
kernel/sched_debug.c +1 -1
@@ -169,7 +169,7 @@ void print_cfs_rq(struct seq_file *m, in cfs_rq->nr_spread_over);#ifdef CONFIG_FAIR_GROUP_SCHED #ifdef CONFIG_SMP -SEQ_printf (m," .%-30s: %lu\n" ,"shares" , cfs_rq->shares); +SEQ_printf (m," .%-30s: %lu\n" ,"shares" , cfs_rq->tg->shares);#endif #endif }
Quoted 17 lines · Peter Zijlstra
Quoted 16 lines · Mike Galbraith
Quoted 14 lines · Peter Zijlstra
Quoted 11 lines
Quoted 5 lines
Quoted 1 line
-rc1. Do you have the fix below applied?
Oooh - good catch, that seems to be a post -rc1 merge.
Yes this is required.
With below patch+previous_fix_weight_calc, the result is improved, but is still about 10% worse than the one of pure 2.6.26-rc1.
Number of threads: 16read/write requests: 6104336 (50867.11 per sec.)So puzzling - could you try -git +
fix_weight_calc, as I think that is what Mike tested and proved good on his Quad.
Quoted 7 lines
Quoted 4 lines
If he still has group scheduling enabled, my box agrees that there's a regression. I got mysql running, and tested with group scheduling both enabled and disabled, results attached. (for some reason, I can only test up to 64 threads with mysql though)
Hm, ok. Does the patch at hand improve or regress the group config for you?
Oh, you want me to revert it and retest group?
-Mike
Quoted 23 lines · Mike Galbraith
Quoted 22 lines · Peter Zijlstra
Quoted 20 lines · Mike Galbraith
Quoted 6 lines · Peter Zijlstra
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
-rc1. Do you have the fix below applied?
Oooh - good catch, that seems to be a post -rc1 merge.
Yes this is required.
Quoted 8 lines
Quoted 5 lines
With below patch+previous_fix_weight_calc, the result is improved, but is still about 10% worse than the one of pure 2.6.26-rc1.
Number of threads: 16read/write requests: 6104336 (50867.11 per sec.)So puzzling - could you try -git +
fix_weight_calc, as I think that is what Mike tested and proved good on his Quad.If he still has group scheduling enabled, my box agrees that there's a regression. I got mysql running, and tested with group scheduling both enabled and disabled, results attached. (for some reason, I can only test up to 64 threads with mysql though)
Quoted 4 lines
Quoted 2 lines
Hm, ok. Does the patch at hand improve or regress the group config for you?
Oh, you want me to revert it and retest group?
Yes, I guess that would be what I'm asking ;-)
Quoted 24 lines · Peter Zijlstra
Quoted 23 lines · Mike Galbraith
Quoted 22 lines · Peter Zijlstra
Quoted 20 lines · Mike Galbraith
Quoted 6 lines · Peter Zijlstra
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
-rc1. Do you have the fix below applied?
Oooh - good catch, that seems to be a post -rc1 merge.
Yes this is required.
Quoted 8 lines
Quoted 5 lines
With below patch+previous_fix_weight_calc, the result is improved, but is still about 10% worse than the one of pure 2.6.26-rc1.
Number of threads: 16read/write requests: 6104336 (50867.11 per sec.)So puzzling - could you try -git +
fix_weight_calc, as I think that is what Mike tested and proved good on his Quad.If he still has group scheduling enabled, my box agrees that there's a regression. I got mysql running, and tested with group scheduling both enabled and disabled, results attached. (for some reason, I can only test up to 64 threads with mysql though)
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
Hm, ok. Does the patch at hand improve or regress the group config for you?
Oh, you want me to revert it and retest group?
Yes, I guess that would be what I'm asking ;-)
(yeah yeah, ok, i went dense, results sent;)