Nick's scheduler v16
Hi
http://www.kerneltrap.org/~npiggin/v16/
I'm starting to do some large SMP / NUMA testing. Fixed and changed quite a bit. It isn't too bad, although I'm only testing dbench, tbench, and volanomark at the moment.
These SMP and NUMA changes are not tied to my interactivity stuff, so its possible they could get included if they turn out well. If you find any problems with it (high end or interactivity), please let me know.
Quoted 8 lines · Nick Piggin
Hi
http://www.kerneltrap.org/~npiggin/v16/
I'm starting to do some large SMP / NUMA testing. Fixed and changed quite a bit. It isn't too bad, although I'm only testing dbench, tbench, and volanomark at the moment.
Oh, if anyone can suggest other benchmarks I could use it would be good (I'm currently getting osdl's dbt2 set up). System is 16-way 16GB ia32 NUMA and very minimum disk bandwidth. Thanks.
Quoted 7 lines
I'm starting to do some large SMP / NUMA testing. Fixed and changed quite a bit. It isn't too bad, although I'm only testing dbench, tbench, and volanomark at the moment.
These SMP and NUMA changes are not tied to my interactivity stuff, so its possible they could get included if they turn out well. If you find any problems with it (high end or interactivity), please let me know.
Interesting ... some things get getter, some worse:
Kernbench: (make -j N vmlinux, where N = 2 x num_cpus)
Elapsed System User CPU
2.6.0-test8 45.20 100.97 566.65 1476.25
2.6.0-test8-nick 44.81 93.98 568.49 1477.50
2.6.0-test8-nick2 44.78 94.69 568.81 1482.00elapsed is a tiny bit faster, system is significantly less, but with higher parallelism:
Kernbench: (make -j vmlinux, maximal tasks)
Elapsed System User CPU
2.6.0-test8 45.86 119.41 569.66 1502.00
2.6.0-test8-nick 47.00 112.75 590.40 1495.00
2.6.0-test8-nick2 47.11 112.86 590.31 1491.50elapsed is definitely worse now.
SDET is a happy bunny though:
SDET 128 (see disclaimer)
Throughput Std. Dev
2.6.0-test8 100.0% 0.3%
2.6.0-test8-nick 109.9% 0.2%Much of the changes there might just be backing out Con's interactivity changes ...
M.
Quoted 21 lines · Martin J. Bligh
Quoted 8 lines
I'm starting to do some large SMP / NUMA testing. Fixed and changed quite a bit. It isn't too bad, although I'm only testing dbench, tbench, and volanomark at the moment.
These SMP and NUMA changes are not tied to my interactivity stuff, so its possible they could get included if they turn out well. If you find any problems with it (high end or interactivity), please let me know.
Interesting ... some things get getter, some worse:
Kernbench: (make -j N vmlinux, where N = 2 x
num_cpus)Elapsed System User CPU 2.6.0-test8 45.20 100.97 566.65 1476.25 2.6.0-test8-nick 44.81 93.98 568.49 1477.50 2.6.0-test8-nick2 44.78 94.69 568.81 1482.00elapsed is a tiny bit faster, system is significantly less, but with higher parallelism:
Quoted 8 lines
Kernbench: (make -j vmlinux, maximal tasks)
Elapsed System User CPU 2.6.0-test8 45.86 119.41 569.66 1502.00 2.6.0-test8-nick 47.00 112.75 590.40 1495.00 2.6.0-test8-nick2 47.11 112.86 590.31 1491.50elapsed is definitely worse now.
I'll see if I can reproduce this and work out what is going on. Thanks Martin.
Re: Nick's scheduler v16 - reaim
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
I'm starting to do some large SMP / NUMA testing. Fixed and changed quite a bit. It isn't too bad, although I'm only testing dbench, tbench, and volanomark at the moment.
These SMP and NUMA changes are not tied to my interactivity stuff, so its possible they could get included if they turn out well. If you find any problems with it (high end or interactivity), please let me know.
Results from reaim aren't encouraging. patch was applied against 2.6.0-test8 - the result is PLM #2232
STP id kernal name Max JPM Max User Pct elevator 281932 nick_v16 4923.08 60 0.00 AS 281933 nick_v16 5196.06 68 5.54 deadline 281722 linux-2.6.0-test8 5432.77 92 9.38 deadline 281792 2.6.0-test8-mm1 5384.41 92 8.56 deadline 281790 2.6.0-test8-mm1 5392.65 88 8.7 AS
The kernel doesn't perform well at larger user numbers. Notice the different in max users.
Compare the graphs for jobs per minute, the usual graph is quite flat, see:
http://khack.osdl.org/stp/281932/results/jpm.png
With the v16 scheduler, jobs per minute falls off rapidly as user number increases giving graph with a steep slope, not good.
http://khack.osdl.org/stp/281790/results/jpm.png
Further results: http://www.developer.osdl.org/reaim/index.html
cliffw
Re: Nick's scheduler v16 - reaim
Quoted 29 lines · Cliff White
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
I'm starting to do some large SMP / NUMA testing. Fixed and changed quite a bit. It isn't too bad, although I'm only testing dbench, tbench, and volanomark at the moment.
These SMP and NUMA changes are not tied to my interactivity stuff, so its possible they could get included if they turn out well. If you find any problems with it (high end or interactivity), please let me know.
Results from reaim aren't encouraging. patch was applied against 2.6.0-test8 - the result is PLM #2232
STP id kernal name Max JPM Max User Pct elevator 281932 nick_v16 4923.08 60 0.00 AS 281933 nick_v16 5196.06 68 5.54 deadline 281722 linux-2.6.0-test8 5432.77 92 9.38 deadline 281792 2.6.0-test8-mm1 5384.41 92 8.56 deadline 281790 2.6.0-test8-mm1 5392.65 88 8.7 ASThe kernel doesn't perform well at larger user numbers. Notice the different in max users.
Compare the graphs for jobs per minute, the usual graph is quite flat, see:
http://khack.osdl.org/stp/281932/results/jpm.png
Quoted 5 lines
With the v16 scheduler, jobs per minute falls off rapidly as user number increases giving graph with a steep slope, not good.
http://khack.osdl.org/stp/281790/results/jpm.png
Further results: http://www.developer.osdl.org/reaim/index.html
Hi Cliff,
Yeah, unfortunately this graph has been a feature of my scheduler for quite a while :( I think it is caused either by too much balancing or timeslices getting too small because there are a lot of threads blocking. Anyway, if you had the time to test this one would be really good.
http://www.kerneltrap.org/~npiggin/v16/sched-rollup-nopolicy-v16.gz
Nick
Nick's scheduler v17a
In reply to Nick Pigginhttp://www.kerneltrap.org/~npiggin/v17a/
More balancing fixes. I also incorporated some of Andrew Theurer's ideas. I'm generally getting good numbers now, but using fairly synthetic benchmarks.
Now would be a good time to test if anyone is interested. Thanks.
Re: Nick's scheduler v17a
Quoted 7 lines · On Thu, 30 Oct 2003 14:41:07 +1100 Nick Piggin <[email protected]> wrote:
http://www.kerneltrap.org/~npiggin/v17a/
More balancing fixes. I also incorporated some of Andrew Theurer's ideas. I'm generally getting good numbers now, but using fairly synthetic benchmarks.
Now would be a good time to test if anyone is interested. Thanks.
well i didn't have the time to make extensive tests yet, but the behaviour improved a lot since v16, and *looks* quite better compared to test9 on a 2 way xeon p4 with hyperthreading enabled, seeing 4 cpu's.
when i get back home, I'll try to make a few benchmarks.
Re: Nick's scheduler v17a
Quoted 10 lines · Rhino
Quoted 6 lines · On Thu, 30 Oct 2003 14:41:07 +1100 Nick Piggin <[email protected]> wrote:
http://www.kerneltrap.org/~npiggin/v17a/
More balancing fixes. I also incorporated some of Andrew Theurer's ideas. I'm generally getting good numbers now, but using fairly synthetic benchmarks.
Quoted 9 lines
Quoted 2 lines
Now would be a good time to test if anyone is interested. Thanks.
well i didn't have the time to make extensive tests yet, but the behaviour improved a lot since v16, and *looks* quite better compared to test9 on a 2 way xeon p4 with hyperthreading enabled, seeing 4 cpu's.
when i get back home, I'll try to make a few benchmarks.
That would be nice. There still isn't a lot of work done in the HT department, but something simple and easy like the shared runqueues patch might be reasonable for 2.6.