Skip to contentSkip to conversation

IO scheduler benchmarking

DISCUSSION [email protected] 2 participants 2 messages Feb 25, 2003

lore
Share & tools
lore
#0 lore raw

Executive question: Why does 2.5.62-mm2 have higher sequential write latency than 2.5.61-mm1?

tiobench numbers on uniprocessor single disk IDE:
The cfq scheduler (2.5.62-mm2 and 2.5.61-cfq) has a big latency regression.

2.5.61-mm1 (default scheduler (anticipatory?))
2.5.61-mm1-cfq elevator=cfq
2.5.62-mm2-as anticipatory scheduler
2.5.62-mm2-dline elevator=deadline
2.5.62-mm2 elevator=cfq

                    Thr  MB/sec   CPU%     avg lat      max latency
2.5.61-mm1            8   15.68   54.42%     5.87 ms     2.7 seconds
2.5.61-mm1-cfq        8    9.60   15.07%     7.54      393.0
2.5.62-mm2-as         8   14.76   52.04%     6.14        4.5
2.5.62-mm2-dline      8    9.91   13.90%     9.41         .8
2.5.62-mm2            8    9.83   15.62%     7.38      408.9
2.4.21-pre3           8   10.34   27.66%     8.80        1.0
2.4.21-pre3-ac4       8   10.53   28.41%     8.83         .6
2.4.21-pre3aa1        8   18.55   71.95%     3.25       87.6

For most thread counts (8 - 128), the anticipatory scheduler has roughly 45% higher ext2 sequential read throughput. Latency was higher than deadline, but a lot lower than cfq.

For tiobench sequential writes, the max latency numbers for 2.4.21-pre3 are notably lower than 2.5.62-mm2 (but not as good as 2.5.61-mm1). This is with 16 threads.

                    Thr  MB/sec   CPU%      avg lat     max latency
2.5.61-mm1           16   18.30   81.12%     9.159 ms     6.1 seconds
2.5.61-mm1-cfq       16   18.03   80.71%     9.086        6.1
2.5.62-mm2-as        16   18.84   84.25%     8.620       47.7
2.5.62-mm2-dline     16   18.53   84.10%     8.967       53.4
2.5.62-mm2           16   18.46   83.28%     8.521       40.8
2.4.21-pre3          16   16.20   65.13%     9.566        8.7
2.4.21-pre3-ac4      16   18.50   83.68%     8.774       11.6
2.4.21-pre3aa1       16   18.49   88.10%     8.455        7.5

Recent uniprocessor benchmarks:
http://home.earthlink.net/~rwhron/kernel/latest.html

More uniprocessor benchmarks:
http://home.earthlink.net/~rwhron/kernel/k6-2-475.html

Signature · 4 lines
Randy Hron
http://home.earthlink.net/~rwhron/kernel/bigbox.html
latest quad xeon benchmarks:
http://home.earthlink.net/~rwhron/kernel/blatest.html
#1 lore raw
Quoted 3 lines · [email protected] wrote:

Executive question: Why does 2.5.62-mm2 have higher sequential write latency than 2.5.61-mm1?

Well bear in mind that we sometimes need to perform reads to be able to perform writes. So the way tiobench measures it, you could be seeing read-vs-write latencies here.

And there are various odd interactions in, at least, ext3. You did not specify which filesystem was used.

Quoted 5 lines

...

                    Thr  MB/sec   CPU%     avg lat      max latency
2.5.62-mm2-as         8   14.76   52.04%     6.14        4.5
2.5.62-mm2-dline      8    9.91   13.90%     9.41         .8
2.5.62-mm2            8    9.83   15.62%     7.38      408.9

Fishiness. 2.5.62-mm2 _is_ 2.5.62-mm2-as. Why the 100x difference?

That 408 seconds looks suspect.

I don't know what tiobench is doing in there, really. I find it more useful to test simple things, which I can understand. If you want to test write latency, do this:

while true
do
write-and-fsync -m 200 -O -f foo
done

Maybe run a few of these. This command will cause a continuous streaming file overwrite.

then do:

time write-and-fsync -m1 -f foo

this will simply write a megabyte file, fsync it and exit.

You need to be careful with this - get it wrong and most of the runtime is actually paging the executables back in. That is why the above background load is just reusing the same pagecache over and over.

The latency which I see for the one megabyte write and fsync varies a lot.

Quoted 1 line

From one second to ten. That's with the deadline scheduler.

There is a place in VFS where one writing task could accidentally hammer a different one. I cannot trigger that, but I'll fix it up in next -mm.