Performance regression of specjbb2005/aim7 with 2.6.29-rc1
Zhang, Yanmin
Tested-by Jayson KingComparing with 2.6.28's results, specjbb2005 has about 7% regression with 2.6.29-rc1 on my a couple of x86_64 machines. aim7 has about 1.7% regression.
Ming did a quick bisect with aim7 and located below patch.
commit 0a582440ff546e2c6610d1acec325e91b4efd313
Author: Mike Galbraith <[email protected]> Date: Fri Jan 2 12:16:42 2009 +0100
sched: fix sched_slice()
Impact: fix bad-interactivity buglet
Fix sched_slice() to emit a sane result whether a task is currently enqueued or not.
After we revert the patch, aim7 regression disappeared. specjbb2005 regression becomes less than 1.5% on 8-core stokley and disappears on 16-core tigerton. I don't know what causes the last 1.5% regression.
As tbench has about 5% improvement and oltp(mysql+sysbench) has 5% improvement, we also tested to make sure such improvement isn't related to above patch. volanoMark's improvement is also not related to the patch. So it seems safe to revert it.
yanmin
Quoted 21 lines · Yanmin
Comparing with 2.6.28's results, specjbb2005 has about 7% regression with 2.6.29-rc1 on my a couple of
x86_64machines. aim7 has about 1.7% regression.Ming did a quick bisect with aim7 and located below patch.
commit
0a582440ff546e2c6610d1acec325e91b4efd313Author: Mike Galbraith <[email protected]> Date: Fri Jan 2 12:16:42 2009 +0100sched: fix
sched_slice()Impact: fix bad-interactivity buglet
Fix
sched_slice()to emit a sane result whether a task is currently enqueued or not.Signed-off-by Mike Galbraith <[email protected]>Tested-by Jayson King <[email protected]>Signed-off-by Ingo Molnar <[email protected]>
Quoted 7 lines
After we revert the patch, aim7 regression disappeared. specjbb2005 regression becomes less than 1.5% on 8-core stokley and disappears on 16-core tigerton. I don't know what causes the last 1.5% regression.
As tbench has about 5% improvement and oltp(mysql+sysbench) has 5% improvement, we also tested to make sure such improvement isn't related to above patch. volanoMark's improvement is also not related to the patch. So it seems safe to revert it.
No, it's not safe to just revert. You can replace it with something else, but as long as sched_slice() is called for unqueued tasks, it must emit sane slices, otherwise you can experience a latency-hit-from-hell.
See thread: problem with "sched: revert back to per-rq vruntime"?
-Mike
Quoted 30 lines · Mike Galbraith
Quoted 21 lines · Yanmin
Comparing with 2.6.28's results, specjbb2005 has about 7% regression with 2.6.29-rc1 on my a couple of
x86_64machines. aim7 has about 1.7% regression.Ming did a quick bisect with aim7 and located below patch.
commit
0a582440ff546e2c6610d1acec325e91b4efd313Author: Mike Galbraith <[email protected]> Date: Fri Jan 2 12:16:42 2009 +0100sched: fix
sched_slice()Impact: fix bad-interactivity buglet
Fix
sched_slice()to emit a sane result whether a task is currently enqueued or not.Signed-off-by Mike Galbraith <[email protected]>Tested-by Jayson King <[email protected]>Signed-off-by Ingo Molnar <[email protected]>Quoted 7 lines
After we revert the patch, aim7 regression disappeared. specjbb2005 regression becomes less than 1.5% on 8-core stokley and disappears on 16-core tigerton. I don't know what causes the last 1.5% regression.
As tbench has about 5% improvement and oltp(mysql+sysbench) has 5% improvement, we also tested to make sure such improvement isn't related to above patch. volanoMark's improvement is also not related to the patch. So it seems safe to revert it.
Quoted 5 lines
No, it's not safe to just revert. You can replace it with something else, but as long as
sched_slice()is called for unqueued tasks, it must emit sane slices, otherwise you can experience a latency-hit-from-hell.See thread: problem with "sched: revert back to per-rq vruntime"?
Below patch fixes aim7 regression and specjbb2005 regression becomes less than 1.5% on 8-core stokley.
Jayson,
Mike's patch fixed the latency problem you reported
would you please help to test this patch to see if it still works fine now?
kernel/sched_fair.c +2 -2
@@ -429,10 +429,10 @@ static u64 sched_slice(struct cfs_rq *cfs_rq, struct sched_entity *se)u64 slice =__sched_period (cfs_rq->nr_running + !se->on_rq);for_each_sched_entity (se) { -struct load_weight *load = &cfs_rq->load; +struct load_weight *load = &cfs_rq_of (se)->load;if (unlikely (!se->on_rq)) { -struct load_weight lw = cfs_rq->load; +struct load_weight lw =cfs_rq_of (se)->load;update_load_add (&lw, se->load.weight); load = &lw;
Thanks,
Lin Ming
Quoted 3 lines
-Mike
Quoted 52 lines · Lin Ming
Quoted 30 lines · Mike Galbraith
Quoted 21 lines · Yanmin
Comparing with 2.6.28's results, specjbb2005 has about 7% regression with 2.6.29-rc1 on my a couple of
x86_64machines. aim7 has about 1.7% regression.Ming did a quick bisect with aim7 and located below patch.
commit
0a582440ff546e2c6610d1acec325e91b4efd313Author: Mike Galbraith <[email protected]> Date: Fri Jan 2 12:16:42 2009 +0100sched: fix
sched_slice()Impact: fix bad-interactivity buglet
Fix
sched_slice()to emit a sane result whether a task is currently enqueued or not.Signed-off-by Mike Galbraith <[email protected]>Tested-by Jayson King <[email protected]>Signed-off-by Ingo Molnar <[email protected]>Quoted 7 lines
After we revert the patch, aim7 regression disappeared. specjbb2005 regression becomes less than 1.5% on 8-core stokley and disappears on 16-core tigerton. I don't know what causes the last 1.5% regression.
As tbench has about 5% improvement and oltp(mysql+sysbench) has 5% improvement, we also tested to make sure such improvement isn't related to above patch. volanoMark's improvement is also not related to the patch. So it seems safe to revert it.
Quoted 5 lines
No, it's not safe to just revert. You can replace it with something else, but as long as
sched_slice()is called for unqueued tasks, it must emit sane slices, otherwise you can experience a latency-hit-from-hell.See thread: problem with "sched: revert back to per-rq vruntime"?
Below patch fixes aim7 regression and specjbb2005 regression becomes less than 1.5% on 8-core stokley.
Jayson,
Mike's patch fixed the latency problem you reported
would you please help to test this patch to see if it still works fine now?kernel/sched_fair.c +0
-0@@ -429,10 +429,10 @@ static u64 sched_slice(struct cfs_rq *cfs_rq, struct sched_entity *se)u64 slice =__sched_period (cfs_rq->nr_running + !se->on_rq);
kernel/sched_fair.c static u64 sched_slice(struct cfs_rq *cfs_rq, struct sched_entity *se) · line 438
Quoted 10 lines
kernel/sched_fair.c +2
-2for_each_sched_entity (se) { -struct load_weight *load = &cfs_rq->load; +struct load_weight *load = &cfs_rq_of (se)->load;if (unlikely (!se->on_rq)) { -struct load_weight lw = cfs_rq->load; +struct load_weight lw =cfs_rq_of (se)->load;update_load_add (&lw, se->load.weight); load = &lw;
Ugh, that's not making sense, the thing is, if !se->on_rq it doesn't yet have a sensible cfs_rq_of().
Quoted 8 lines · Lin Ming
kernel/sched_fair.c +0
-0@@ -429,10 +429,10 @@ static u64 sched_slice(struct cfs_rq *cfs_rq, struct sched_entity *se)u64 slice =__sched_period (cfs_rq->nr_running + !se->on_rq);
kernel/sched_fair.c static u64 sched_slice(struct cfs_rq *cfs_rq, struct sched_entity *se) · line 438
Quoted 10 lines
kernel/sched_fair.c +2
-2for_each_sched_entity (se) { -struct load_weight *load = &cfs_rq->load; +struct load_weight *load = &cfs_rq_of (se)->load;if (unlikely (!se->on_rq)) { -struct load_weight lw = cfs_rq->load; +struct load_weight lw =cfs_rq_of (se)->load;update_load_add (&lw, se->load.weight); load = &lw;
I was wrong, this does look good.
Thanks
Quoted 2 lines · Lin Ming
Below patch fixes aim7 regression and specjbb2005 regression becomes less than 1.5% on 8-core stokley.
Ingo, please apply the below.
Ming, would you also provide a S-o-b line?
---
Subject: sched_slice fixlet
From: Lin Ming <[email protected]>
Date: Thu Jan 15 17:10:02 CET 2009
Mike's change: 0a582440f -- sched: fix sched_slice()) Broke group scheduling by forgetting to reload cfs_rq on each loop.
--- kernel/sched_fair.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
kernel/sched_fair.c +4 -1
@@ -429,7 +429,10 @@ static u64 sched_slice(struct cfs_rq *cfu64 slice =__sched_period (cfs_rq->nr_running + !se->on_rq);for_each_sched_entity (se) { -struct load_weight *load = &cfs_rq->load; +struct load_weight *load; + + cfs_rq =cfs_rq_of (se); + load = &cfs_rq->load;if (unlikely (!se->on_rq)) {struct load_weight lw = cfs_rq->load;
Ingo Molnar
AppliedQuoted 6 lines · Peter Zijlstra
Quoted 2 lines · Lin Ming
Below patch fixes aim7 regression and specjbb2005 regression becomes less than 1.5% on 8-core stokley.
Ingo, please apply the below.
Applied to tip/sched/urgent, thanks!
Ingo
Jayson King
Tested-byQuoted 31 lines · Peter Zijlstra
Quoted 2 lines · Lin Ming
Below patch fixes aim7 regression and specjbb2005 regression becomes less than 1.5% on 8-core stokley.
Ingo, please apply the below.
Ming, would you also provide a S-o-b line?
---
Subject:sched_slicefixlet
From: Lin Ming <[email protected]>
Date: Thu Jan 15 17:10:02 CET 2009Mike's change:
0a582440f-- sched: fixsched_slice()) Broke group scheduling by forgetting to reloadcfs_rqon each loop.Signed-off-by Peter Zijlstra <[email protected]>--- kernel/sched_fair.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)Index:
linux-2.6/kernel/sched_fair.c
===================================================================kernel/sched_fair.c +0
-0@@ -429,7 +429,10 @@ static u64 sched_slice(struct cfs_rq *cfu64 slice =__sched_period (cfs_rq->nr_running + !se->on_rq);
kernel/sched_fair.c static u64 sched_slice(struct cfs_rq *cf · line 438
Quoted 10 lines
kernel/sched_fair.c +4
-1for_each_sched_entity (se) { -struct load_weight *load = &cfs_rq->load; +struct load_weight *load; + + cfs_rq =cfs_rq_of (se); + load = &cfs_rq->load;if (unlikely (!se->on_rq)) {struct load_weight lw = cfs_rq->load;
That still works for me. You may add:
Quoted 14 lines · Jayson King
kernel/sched_fair.c static u64 sched_slice(struct cfs_rq *cf · line 438
Quoted 10 lines
kernel/sched_fair.c +4
-1-struct load_weight *load = &cfs_rq->load; +struct load_weight *load; + + cfs_rq =cfs_rq_of (se); + load = &cfs_rq->load;if (unlikely (!se->on_rq)) {struct load_weight lw = cfs_rq->load;That still works for me. You may add:
Quoted 1 line
Tested-by Jayson King <[email protected]>
I have added it to the commit - thanks Jayson for the testing,
Ingo
Quoted 8 lines · Peter Zijlstra
Quoted 2 lines · Lin Ming
Below patch fixes aim7 regression and specjbb2005 regression becomes less than 1.5% on 8-core stokley.
Ingo, please apply the below.
Ming, would you also provide a S-o-b line?
Yes,
Lin Ming
Quoted 32 lines
---
Subject:sched_slicefixlet
From: Lin Ming <[email protected]>
Date: Thu Jan 15 17:10:02 CET 2009Mike's change:
0a582440f-- sched: fixsched_slice()) Broke group scheduling by forgetting to reloadcfs_rqon each loop.Signed-off-by Peter Zijlstra <[email protected]>--- kernel/sched_fair.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)Index:
linux-2.6/kernel/sched_fair.c
===================================================================kernel/sched_fair.c +4
-1@@ -429,7 +429,10 @@ static u64 sched_slice(struct cfs_rq *cfu64 slice =__sched_period (cfs_rq->nr_running + !se->on_rq);for_each_sched_entity (se) { -struct load_weight *load = &cfs_rq->load; +struct load_weight *load; + + cfs_rq =cfs_rq_of (se); + load = &cfs_rq->load;if (unlikely (!se->on_rq)) {struct load_weight lw = cfs_rq->load;