2.6.21.1 vs 2.6.21-sd046 vs 2.6.21-cfs-v6
i list,
meanwhile I've redone my numbercrunching tests with the following kernels:
2.6.21.1 (mainline)
2.6.21-sd046
2.6.21-cfs-v6running on a dualcore x86_64.
[I will run the same test with 2.6.21.1-cfs-v7 over the next days, likely tonight]
The tests consist of 3 tasks (named LTMM, LTMB and LTBM). The only I/O they do is during init and for logging the results, the rest is just floating point math.
There are 3 scenarios:
j1 - all 3 tasks run sequentially
/proc/sys/kernel/sched_granularity_ns=4000000
/proc/sys/kernel/rr_interval=16
j3 - all 3 tasks run in parallel
/proc/sys/kernel/sched_granularity_ns=4000000
/proc/sys/kernel/rr_interval=16
j3big - all 3 tasks run in parallel with timeslice extended
by 2 magnitudes (not run for mainline)
/proc/sys/kernel/sched_granularity_ns=400000000
/proc/sys/kernel/rr_interval=400All 3 tasks are run while the system does nothing else except for the "normal" (KDE) daemons. The system had not been used for interactive work during the tests.
I'm giving user time as provided by the "time" cmd followed by wallclock time (all values in seconds).
LTMM
j1 j3 j3big
2.6.21-cfs-v6 5655.07/ 5682 5437.84/ 5531 5434.04/ 8072 2.6.21-sd-046 5556.44/ 5583 5446.86/ 8037 5407.50/ 8274 2.6.21.1 5417.62/ 5439 5422.37/ 7132 na /na
LTMB
j1 j3 j3big
2.6.21-cfs-v6 7729.81/ 7755 7470.10/10244 7449.16/10186 2.6.21-sd-046 7611.00/ 7626 7573.16/10109 7458.10/10316 2.6.21.1 7438.31/ 7461 7620.72/11087 na /na
LTBM
j1 j3 j3big
2.6.21-cfs-v6 7720.70/ 7746 7567.09/10362 7464.17/10335 2.6.21-sd-046 7431.06/ 7452 7539.83/10600 7474.20/10222 2.6.21.1 7452.80/ 7481 7484.19/ 9570 na /na
LTMM+LTMB+LTBM
j1 j3 j3big
2.6.21-cfs-v6 21105.58/21183 20475.03/26137 20347.37/28593 2.6.21-sd-046 20598.50/20661 20559.85/28746 20339.80/28812
2.6.21.1 20308.73/20381 20527.28/27789 na /na
User time apparently is subject to some variance. I'm particularly surprised by the wallclock time of scenario j1 and j3 for case LTMM with 2.6.21-cfs-v6. I'm not sure what to make of this, i.e. whether I had happening something else on my machine during j1 of LTMM -- that's always been the first test I ran and it might be that there were still some other jobs running after the initial boot.
Assuming scenario j1 does constitute the "true" time each task requires and also assuming each scheduler makes maximum use of the available CPUs (the tests involve very little I/O) one could compute the expected wallclock time. However since I suspect the j1 figures of LTMM to be somewhat "dirty" I'll refrain from it.
However from these figures it seems as if sd does provide for the fairest (as in equal share for all) scheduling among the 3 schedulers tested.
Best,
Michael
Signature · 5 lines
Technosis GmbH, Geschäftsführer: Michael Gerdau, Tobias Dittmar Sitz Hamburg; HRB 89145 Amtsgericht Hamburg Vote against SPAM - see http://www.politik-digital.de/spam/ Michael Gerdau email: [email protected] GPG-keys available on request or at public keyserver
Quoted 9 lines · Michael Gerdau
i list,
meanwhile I've redone my numbercrunching tests with the following kernels:
2.6.21.1 (mainline) 2.6.21-sd046 2.6.21-cfs-v6running on a dualcore
x86_64.
[I will run the same test with 2.6.21.1-cfs-v7 over the next days, likely tonight]
Thanks for testing.
Quoted 55 lines
The tests consist of 3 tasks (named LTMM, LTMB and LTBM). The only I/O they do is during init and for logging the results, the rest is just floating point math.
There are 3 scenarios:
j1 - all 3 tasks run sequentially /proc/sys/kernel/sched_granularity_ns=4000000 /proc/sys/kernel/rr_interval=16 j3 - all 3 tasks run in parallel /proc/sys/kernel/sched_granularity_ns=4000000 /proc/sys/kernel/rr_interval=16 j3big - all 3 tasks run in parallel with timeslice extended by 2 magnitudes (not run for mainline) /proc/sys/kernel/sched_granularity_ns=400000000 /proc/sys/kernel/rr_interval=400All 3 tasks are run while the system does nothing else except for the "normal" (KDE) daemons. The system had not been used for interactive work during the tests.
I'm giving user time as provided by the "time" cmd followed by wallclock time (all values in seconds).
LTMM
j1 j3 j3big2.6.21-cfs-v6 5655.07/ 5682 5437.84/ 5531 5434.04/ 8072 2.6.21-sd-046 5556.44/ 5583 5446.86/ 8037 5407.50/ 8274 2.6.21.1 5417.62/ 5439 5422.37/ 7132 na /naLTMB
j1 j3 j3big2.6.21-cfs-v6 7729.81/ 7755 7470.10/10244 7449.16/10186 2.6.21-sd-046 7611.00/ 7626 7573.16/10109 7458.10/10316 2.6.21.1 7438.31/ 7461 7620.72/11087 na /naLTBM
j1 j3 j3big2.6.21-cfs-v6 7720.70/ 7746 7567.09/10362 7464.17/10335 2.6.21-sd-046 7431.06/ 7452 7539.83/10600 7474.20/10222 2.6.21.1 7452.80/ 7481 7484.19/ 9570 na /naLTMM+LTMB+LTBM
j1 j3 j3big2.6.21-cfs-v6 21105.58/21183 20475.03/26137 20347.37/28593 2.6.21-sd-046 20598.50/20661 20559.85/28746 20339.80/288122.6.21.1 20308.73/20381 20527.28/27789 na /na
User time apparently is subject to some variance. I'm particularly surprised by the wallclock time of scenario j1 and j3 for case LTMM with 2.6.21-cfs-v6. I'm not sure what to make of this, i.e. whether I had happening something else on my machine during j1 of LTMM -- that's always been the first test I ran and it might be that there were still some other jobs running after the initial boot.
Quoted 8 lines
Assuming scenario j1 does constitute the "true" time each task requires and also assuming each scheduler makes maximum use of the available CPUs (the tests involve very little I/O) one could compute the expected wallclock time. However since I suspect the j1 figures of LTMM to be somewhat "dirty" I'll refrain from it.
However from these figures it seems as if sd does provide for the fairest (as in equal share for all) scheduling among the 3 schedulers tested.
Looks good, thanks. Ingo's been hard at work since then and has v8 out by now. SD has not changed so you wouldn't need to do the whole lot of tests on SD again unless you don't trust some of the results.
Quoted 2 lines
Best,
Michael
Signature · 1 line
-ck
Quoted 29 lines · Michael Gerdau
There are 3 scenarios:
j1 - all 3 tasks run sequentially /proc/sys/kernel/sched_granularity_ns=4000000 /proc/sys/kernel/rr_interval=16 j3 - all 3 tasks run in parallel /proc/sys/kernel/sched_granularity_ns=4000000 /proc/sys/kernel/rr_interval=16 j3big - all 3 tasks run in parallel with timeslice extended by 2 magnitudes (not run for mainline) /proc/sys/kernel/sched_granularity_ns=400000000 /proc/sys/kernel/rr_interval=400All 3 tasks are run while the system does nothing else except for the "normal" (KDE) daemons. The system had not been used for interactive work during the tests.
I'm giving user time as provided by the "time" cmd followed by wallclock time (all values in seconds).
LTMM
j1 j3 j3big
2.6.21-cfs-v6 5655.07/ 5682 5437.84/ 5531 5434.04/ 8072
LTMB
2.6.21-cfs-v6 7729.81/ 7755 7470.10/10244 7449.16/10186
LTBM
2.6.21-cfs-v6 7720.70/ 7746 7567.09/10362 7464.17/10335
LTMM+LTMB+LTBM
2.6.21-cfs-v6 21105.58/21183 20475.03/26137 20347.37/28593
Quoted 6 lines
User time apparently is subject to some variance. I'm particularly surprised by the wallclock time of scenario j1 and j3 for case LTMM with 2.6.21-cfs-v6. I'm not sure what to make of this, i.e. whether I had happening something else on my machine during j1 of LTMM -- that's always been the first test I ran and it might be that there were still some other jobs running after the initial boot.
thanks for the testing!
regarding the fairness of the different schedulers, please note the different runtimes for each component of the workload:
LTMM: 5655.07/ 5682
LTMB: 7729.81/ 7755
LTBM: 7720.70/ 7746this means that a fair scheduler would _not_ be the one that finishes them first on wall-clock time (!). A fair scheduler would run each of them at 33% capacity until the fastest one (LTMM) reaches ~5650 seconds runtime and finishes, and _then_ the remaining ~2050 seconds of runtime would be done at 50%/50% capacity between the remaining two jobs. I.e. the fair wall-clock results should be around:
LTMM: ~8500 seconds
LTMB: ~10600 seconds
LTBM: ~10600 seconds(but the IO portion of the workloads and other scheduling effects could easily shift these numbers by a few minutes.)
regarding the results: it seems the wallclock portion of LTMM/j3 is too small - even though the 3 tasks ran in parallel, in the CFS test LTMM finished just as fast as if it were running alone, right? That does not seem to be logical and indeed suggests some sort of testing artifact.
That makes it hard to judge which scheduler achieved the above 'ideal fair distribution' of the workloads better - for some of the results it was SD, for some it was CFS - but the missing LTMM/j3 number makes it hard to decide it conclusively. They are certainly both close enough and the noise of the results seems quite high.
Ingo
Quoted 14 lines
regarding the fairness of the different schedulers, please note the different runtimes for each component of the workload:
LTMM: 5655.07/ 5682 LTMB: 7729.81/ 7755 LTBM: 7720.70/ 7746this means that a fair scheduler would _not_ be the one that finishes them first on wall-clock time (!). A fair scheduler would run each of them at 33% capacity until the fastest one (LTMM) reaches ~5650 seconds runtime and finishes, and _then_ the remaining ~2050 seconds of runtime would be done at 50%/50% capacity between the remaining two jobs. I.e. the fair wall-clock results should be around:
Quoted 6 lines
LTMM: ~8500 seconds LTMB: ~10600 seconds LTBM: ~10600 seconds(but the IO portion of the workloads and other scheduling effects could easily shift these numbers by a few minutes.)
Correct. I will try to cook up some statistics for the next round of results (been redoing cfs-v6, done cfs-v7 and then redone it because of strange results and hopefully will do cfs-v8 tonight as well as sd048 during the next days -- not easy to keep up with you guys rolling out new versions ;-)
Quoted 3 lines
regarding the results: it seems the wallclock portion of LTMM/j3 is too small - even though the 3 tasks ran in parallel, in the CFS test LTMM finished just as fast as if it were running alone, right?
Right. I'm really very surprised but this remained while redoing both LTMM/j1 (twice!) and LTMM/j3 with cfs-v6.
I don't really understand that and now think it shows some hidden "fairness deficiency" that hopefully is corrected in cfs-v8. At least cfs-v7 did behave differently.
Quoted 2 lines
That does not
seem to be logical and indeed suggests some sort of testing artifact.
Since it doesn't appear with any other scheduler tested I'd not rule out a feature of the code.
Quoted 5 lines
That makes it hard to judge which scheduler achieved the above 'ideal fair distribution' of the workloads better - for some of the results it was SD, for some it was CFS - but the missing LTMM/j3 number makes it hard to decide it conclusively. They are certainly both close enough and the noise of the results seems quite high.
Oh, I'm confident both are excellent scheduler. On the other hand I find mainline isn't that bad either, at least for this type of load.
Anyway, as I wrote above I'm continuing my tests and am collecting data for the various scheduler. Presumably over the weekend I'll mail out the next round.
Best,
Michael
Signature · 5 lines
Technosis GmbH, Geschäftsführer: Michael Gerdau, Tobias Dittmar Sitz Hamburg; HRB 89145 Amtsgericht Hamburg Vote against SPAM - see http://www.politik-digital.de/spam/ Michael Gerdau email: [email protected] GPG-keys available on request or at public keyserver