IO scheduler benchmarking
Executive question: Why does 2.5.62-mm2 have higher sequential write latency than 2.5.61-mm1?
tiobench numbers on uniprocessor single disk IDE:
The cfq scheduler (2.5.62-mm2 and 2.5.61-cfq) has a big latency regression.
2.5.61-mm1 (default scheduler (anticipatory?))
2.5.61-mm1-cfq elevator=cfq
2.5.62-mm2-as anticipatory scheduler
2.5.62-mm2-dline elevator=deadline
2.5.62-mm2 elevator=cfq
Thr MB/sec CPU% avg lat max latency 2.5.61-mm1 8 15.68 54.42% 5.87 ms 2.7 seconds 2.5.61-mm1-cfq 8 9.60 15.07% 7.54 393.0 2.5.62-mm2-as 8 14.76 52.04% 6.14 4.5 2.5.62-mm2-dline 8 9.91 13.90% 9.41 .8 2.5.62-mm2 8 9.83 15.62% 7.38 408.9 2.4.21-pre3 8 10.34 27.66% 8.80 1.0 2.4.21-pre3-ac4 8 10.53 28.41% 8.83 .6 2.4.21-pre3aa1 8 18.55 71.95% 3.25 87.6
For most thread counts (8 - 128), the anticipatory scheduler has roughly 45% higher ext2 sequential read throughput. Latency was higher than deadline, but a lot lower than cfq.
For tiobench sequential writes, the max latency numbers for 2.4.21-pre3 are notably lower than 2.5.62-mm2 (but not as good as 2.5.61-mm1). This is with 16 threads.
Thr MB/sec CPU% avg lat max latency 2.5.61-mm1 16 18.30 81.12% 9.159 ms 6.1 seconds 2.5.61-mm1-cfq 16 18.03 80.71% 9.086 6.1 2.5.62-mm2-as 16 18.84 84.25% 8.620 47.7 2.5.62-mm2-dline 16 18.53 84.10% 8.967 53.4 2.5.62-mm2 16 18.46 83.28% 8.521 40.8 2.4.21-pre3 16 16.20 65.13% 9.566 8.7 2.4.21-pre3-ac4 16 18.50 83.68% 8.774 11.6 2.4.21-pre3aa1 16 18.49 88.10% 8.455 7.5
Recent uniprocessor benchmarks:
http://home.earthlink.net/~rwhron/kernel/latest.html
More uniprocessor benchmarks:
http://home.earthlink.net/~rwhron/kernel/k6-2-475.html
Signature · 4 lines
Randy Hron http://home.earthlink.net/~rwhron/kernel/bigbox.html latest quad xeon benchmarks: http://home.earthlink.net/~rwhron/kernel/blatest.html
Quoted 3 lines · [email protected] wrote:
Executive question: Why does 2.5.62-mm2 have higher sequential write latency than 2.5.61-mm1?
Well bear in mind that we sometimes need to perform reads to be able to perform writes. So the way tiobench measures it, you could be seeing read-vs-write latencies here.
And there are various odd interactions in, at least, ext3. You did not specify which filesystem was used.
Quoted 5 lines
...
Thr MB/sec CPU% avg lat max latency 2.5.62-mm2-as 8 14.76 52.04% 6.14 4.5 2.5.62-mm2-dline 8 9.91 13.90% 9.41 .8 2.5.62-mm2 8 9.83 15.62% 7.38 408.9
Fishiness. 2.5.62-mm2 _is_ 2.5.62-mm2-as. Why the 100x difference?
That 408 seconds looks suspect.
I don't know what tiobench is doing in there, really. I find it more useful to test simple things, which I can understand. If you want to test write latency, do this:
while true
do
write-and-fsync -m 200 -O -f foo
done
Maybe run a few of these. This command will cause a continuous streaming file overwrite.
then do:
time write-and-fsync -m1 -f foo
this will simply write a megabyte file, fsync it and exit.
You need to be careful with this - get it wrong and most of the runtime is actually paging the executables back in. That is why the above background load is just reusing the same pagecache over and over.
The latency which I see for the one megabyte write and fsync varies a lot.
Quoted 1 line
From one second to ten. That's with the deadline scheduler.
There is a place in VFS where one writing task could accidentally hammer a different one. I cannot trigger that, but I'll fix it up in next -mm.