Performance regression in 2.6.22-git1 (new sched code?)
~ 1% on 4-way x86_64
http://test.kernel.org/perf/kernbench.elm3b6.png
~ 4% on 16-way NUMA-Q (i386)
http://test.kernel.org/perf/kernbench.moe.png
~ 1.5% on 4-way i386
http://test.kernel.org/perf/kernbench.elm3b132.png
There's readprofiles and stuff available from here:
http://test.kernel.org/functional/index.html
if you drill down under the tests marked "regression", eg.
http://test.kernel.org/abat/98796/002.kernbench.test/profiling/profile.text
M.
Quoted 8 lines · Martin Bligh
~ 1% on 4-way
x86_64http://test.kernel.org/perf/kernbench.elm3b6.png
~ 4% on 16-way NUMA-Q (i386)
Quoted 8 lines
~ 1.5% on 4-way i386
http://test.kernel.org/perf/kernbench.elm3b132.png
There's readprofiles and stuff available from here:
http://test.kernel.org/functional/index.html
if you drill down under the tests marked "regression", eg.
http://test.kernel.org/abat/98796/002.kernbench.test/profiling/profile.text
Ok, I've done a crude comparison of a bunch of the jobs we run between 2.6.22 and 2.6.22-git1 kernbench runs, extracting the wallclock, user and system times for each. First column is 2.6.22, second is 2.6.22-git1, third is the delta. +ve bad, -ve good. Note that I have hand correlated these jobs so error is possible, but it matches Martin's figures I belive.
bl6-13
User 85.14 88.53 3.99 System 34.16 34.71 1.60 Total 119.30 123.24 3.30 elm3b132 User 2397.82 2431.95 1.42 System 207.41 206.78 -0.30 Total 2605.23 2638.73 1.29 elm3b6 User 352.18 357.18 1.42 System 37.06 36.61 -1.22 Total 389.24 393.79 1.17 oia5-hs20 User 645.02 653.47 1.31 System 49.30 52.45 6.37 Total 694.33 705.92 1.67 elm3b239 User 878.21 950.80 8.27 System 416.06 348.76 -16.18 Total 1294.27 1299.56 0.41
-apw
Quoted 11 lines · Martin Bligh
~ 1% on 4-way
x86_64http://test.kernel.org/perf/kernbench.elm3b6.png
~ 4% on 16-way NUMA-Q (i386)
http://test.kernel.org/perf/kernbench.moe.png
~ 1.5% on 4-way i386
thx! I'll check this tomorrow, meanwhile here are a few quick ideas.
1) does reverting this patch improve performance:
commit 9c4801cebc2add1fe514bc8eb201b16372eee11a
Author: Ingo Molnar <[email protected]>
Date: Mon Jul 9 18:52:01 2007 +0200
sched: more agressive idle balancing
2) does changing CONFIG_HZ=250 to CONFIG_HZ=100 improve the numbers? CFS has some internal tuning that depends on HZ - a higher HZ is a sign that the user wants more finegrained scheduling. So for maximum server throughput, use CONFIG_HZ=100.
3) could you turn off CONFIG_SCHED_DEBUG? The cost is small but perhaps measurable.
(these 3 suggestions could be tested together - they should have cumulative effects.)
Ingo
Quoted 3 lines · Ingo Molnar
Quoted 1 line
thx! I'll check this tomorrow, meanwhile here are a few quick ideas.
ok, i've re-tested this, and could you please try the fix attached below, combined with a CONFIG_HZ=100 run and with SCHED_DEBUG disabled? I think a fair bit of the overhead comes from granularity being capped to 10 msecs instead of the intended 100 msecs.
Ingo
--------------------------------------------->
Subject: [patch] sched: allow larger granularity
From: Ingo Molnar <[email protected]>
allow granularity up to 100 msecs, instead of 10 msecs. (needed on larger boxes)
--- kernel/sched.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/sched.c +1 -1
@@ -4772,7 +4772,7 @@ cpumask_t nohz_cpu_mask = CPU_MASK_NONE;static inline void sched_init_granularity (void ) {unsigned int factor =1 +ilog2 (num_online_cpus ()); -const unsigned long gran_limit =10000000 ; +const unsigned long gran_limit =100000000 ; sysctl_sched_granularity *= factor;if (sysctl_sched_granularity > gran_limit)