2.5.39-mm1
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Here follow the contest v0.41 (http://contest.kolivas.net) results for 2.5.39-mm1:
noload:
Kernel Time CPU Ratio 2.4.19 67.71 98% 1.00 2.5.38 72.38 94% 1.07 2.5.38-mm3 73.00 93% 1.08 2.5.39 73.17 93% 1.08 2.5.39-mm1 72.97 94% 1.08
process_load:
Kernel Time CPU Ratio 2.4.19 110.75 57% 1.64 2.5.38 85.71 79% 1.27 2.5.38-mm3 96.32 72% 1.42 2.5.39 88.9 75% 1.33* 2.5.39-mm1 99.0 69% 1.45*
io_load:
Kernel Time CPU Ratio 2.4.19 216.05 33% 3.19 2.5.38 887.76 8% 13.11 2.5.38-mm3 105.17 70% 1.55 2.5.39 229.4 34% 3.4 2.5.39-mm1 239.5 33% 3.4
mem_load:
Kernel Time CPU Ratio 2.4.19 105.40 70% 1.56 2.5.38 107.89 73% 1.59 2.5.38-mm3 117.09 63% 1.73 2.5.39 103.72 72% 1.53 2.5.39-mm1 104.61 73% 1.54
process_load and io_load results are averages of 6 runs.
Statistical significance in process_load performance (p=0.017), with mm1 slower. The other changes did not show statistical significance, with trends as noted above.
note: these were done with the temporary fix for the reiserfs breakage but as far as I'm aware it shouldn't affect this test
Hardware: 1133MhzP3, 224Mb Ram, IDE-ATA100 5400rpm drive with io_load tested on same disk, reiserFS. Preempt=N for all kernels.
Con
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)
iD8DBQE9mBxUF6dfvkL3i1gRAr4kAJ47cICW0qIXLmswyBL9t1ZsiyxgVwCfaHCN bXOSrZtwTjJsSibiBm5KrRo=
=XWpt
-----END PGP SIGNATURE-----
Quoted 7 lines · Con Kolivas
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1Here follow the contest v0.41 (http://contest.kolivas.net) results for 2.5.39-mm1:
Quoted 7 lines
noload:
Kernel Time CPU Ratio 2.4.19 67.71 98% 1.00 2.5.38 72.38 94% 1.07 2.5.38-mm3 73.00 93% 1.08 2.5.39 73.17 93% 1.08 2.5.39-mm1 72.97 94% 1.08
2.4.19 achieves higher CPU occupancy - you're using `make -j4', so it could be a CPU scheduler artifact, or a disk readahead latency effect.
Is the kernel source in-cache for these runs?
Quoted 7 lines
process_load:Kernel Time CPU Ratio 2.4.19 110.75 57% 1.64 2.5.38 85.71 79% 1.27 2.5.38-mm3 96.32 72% 1.42 2.5.39 88.9 75% 1.33* 2.5.39-mm1 99.0 69% 1.45*
Not sure what to make of this test. We have a bunch of tasks sending data between each other across pipes while trying to build a kernel.
It could be that with 2.4.19, those piping processes got a lot more work done.
I'd be inclined to drop this test; not sure what it means.
Quoted 7 lines
io_load:Kernel Time CPU Ratio 2.4.19 216.05 33% 3.19 2.5.38 887.76 8% 13.11 2.5.38-mm3 105.17 70% 1.55 2.5.39 229.4 34% 3.4 2.5.39-mm1 239.5 33% 3.4
I think I'll set fifo_batch to 16 again...
Quoting Andrew Morton <[email protected]>:
Quoted 16 lines
Quoted 7 lines · Con Kolivas
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1Here follow the contest v0.41 (http://contest.kolivas.net) results for 2.5.39-mm1:
Quoted 7 lines
noload:
Kernel Time CPU Ratio 2.4.19 67.71 98% 1.00 2.5.38 72.38 94% 1.07 2.5.38-mm3 73.00 93% 1.08 2.5.39 73.17 93% 1.08 2.5.39-mm1 72.97 94% 1.08
Quoted 4 lines
2.4.19 achieves higher CPU occupancy - you're using `make -j4', so it could be a CPU scheduler artifact, or a disk readahead latency effect.
Is the kernel source in-cache for these runs?
Not cached. Swap should be empty and caches flushed prior to every load test.
Quoted 9 lines
Quoted 8 lines
process_load:Kernel Time CPU Ratio 2.4.19 110.75 57% 1.64 2.5.38 85.71 79% 1.27 2.5.38-mm3 96.32 72% 1.42 2.5.39 88.9 75% 1.33* 2.5.39-mm1 99.0 69% 1.45*
Quoted 8 lines
Not sure what to make of this test. We have a bunch of tasks sending data between each other across pipes while trying to build a kernel.
It could be that with 2.4.19, those piping processes got a lot more work done.
I'd be inclined to drop this test; not sure what it means.
Err yeah...
Quoted 11 lines
Quoted 8 lines
io_load:Kernel Time CPU Ratio 2.4.19 216.05 33% 3.19 2.5.38 887.76 8% 13.11 2.5.38-mm3 105.17 70% 1.55 2.5.39 229.4 34% 3.4 2.5.39-mm1 239.5 33% 3.4I think I'll set
fifo_batchto 16 again...
And I'll happily benchmark it when you do.
Con.
Quoted 8 lines · Con Kolivas
Quoted 8 lines
Quoted 7 lines
io_load:Kernel Time CPU Ratio 2.4.19 216.05 33% 3.19 2.5.38 887.76 8% 13.11 2.5.38-mm3 105.17 70% 1.55 2.5.39 229.4 34% 3.4 2.5.39-mm1 239.5 33% 3.4
Quoted 4 lines
Quoted 2 lines
I think I'll set
fifo_batchto 16 again...And I'll happily benchmark it when you do.
Just take 2.5.39-mm1 sources, edit drivers/block/deadline-iosched.c and set
static int fifo_batch = 16;
instead of the 32. -mm has them in sysctl too, iirc.
Signature · 1 line
Jens Axboe
Quoted 9 lines · Andrew Morton
Quoted 7 lines
io_load:Kernel Time CPU Ratio 2.4.19 216.05 33% 3.19 2.5.38 887.76 8% 13.11 2.5.38-mm3 105.17 70% 1.55 2.5.39 229.4 34% 3.4 2.5.39-mm1 239.5 33% 3.4I think I'll set
fifo_batchto 16 again...
As not to compare oranges and apples, I'd very much like to see a 2.5.39-mm1 vs 2.5.39-mm1 with fifo_batch=16. Con, would you do that? Thanks!
Signature · 1 line
Jens Axboe
A message that is not in the archive (sent off-list or not yet received): [email protected]
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Quoted 10 lines · Andrew Morton
Quoted 9 lines · Jens Axboe
Quoted 8 lines · Andrew Morton
Quoted 7 lines
io_load:Kernel Time CPU Ratio 2.4.19 216.05 33% 3.19 2.5.38 887.76 8% 13.11 2.5.38-mm3 105.17 70% 1.55 2.5.39 229.4 34% 3.4 2.5.39-mm1 239.5 33% 3.4
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
I think I'll set
fifo_batchto 16 again...As not to compare oranges and apples, I'd very much like to see a 2.5.39-mm1 vs 2.5.39-mm1 with fifo_batch=16. Con, would you do that? Thanks!
The presence of
/proc/sys/vm/fifo_batchshould make that pretty easy.
Thanks. That made it a lot easier and faster, and made me curious enough to create a family or very interesting results. All these are with 2.5.39-mm1 with fifo_batch set to 1->16, average of three runs. The first result is the unmodified 2.5.39-mm1 (fifo_batch=32).
io_load:
Kernel Time CPU% Ratio 2.5.39-mm1 239.5 32 3.54 2539mm1fb16 131.2 57 1.94 2539mm1fb8 109.1 68 1.61 2539mm1fb4 146.4 51 2.16 2539mm1fb2 112.7 65 1.67 2539mm1fb1 125.4 60 1.85
What's most interesting is the variation was small until the number was <8; then the variation between runs increased. Dare I say it there appears to be a sweet spot in the results.
Con
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)
iD8DBQE9mZLPF6dfvkL3i1gRAjYnAKCGvWq43uTeClFz2tb6d8fcVe95zwCfbor2 HKO0FgK8kVsEvyQ3FwYaubg=
=bAzC
-----END PGP SIGNATURE-----
Quoted 20 lines · Con Kolivas
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1Quoted 16 lines · Andrew Morton
Quoted 14 lines · Jens Axboe
Quoted 9 lines · Andrew Morton
Quoted 7 lines
io_load:Kernel Time CPU Ratio 2.4.19 216.05 33% 3.19 2.5.38 887.76 8% 13.11 2.5.38-mm3 105.17 70% 1.55 2.5.39 229.4 34% 3.4 2.5.39-mm1 239.5 33% 3.4I think I'll set
fifo_batchto 16 again...As not to compare oranges and apples, I'd very much like to see a 2.5.39-mm1 vs 2.5.39-mm1 with fifo_batch=16. Con, would you do that? Thanks!
Quoted 6 lines
Quoted 1 line
The presence of
/proc/sys/vm/fifo_batchshould make that pretty easy.Thanks. That made it a lot easier and faster, and made me curious enough to create a family or very interesting results. All these are with 2.5.39-mm1 with
fifo_batchset to 1->16, average of three runs. The first result is the unmodified 2.5.39-mm1 (fifo_batch=32).
Ah excellent, thanks a lot!
Quoted 9 lines
io_load:Kernel Time CPU% Ratio 2.5.39-mm1 239.5 32 3.54 2539mm1fb16 131.2 57 1.94 2539mm1fb8 109.1 68 1.61 2539mm1fb4 146.4 51 2.16 2539mm1fb2 112.7 65 1.67 2539mm1fb1 125.4 60 1.85
Quoted 3 lines
What's most interesting is the variation was small until the number was <8; then the variation between runs increased. Dare I say it there appears to be a sweet spot in the results.
Yes it's an interesting curve. What makes it interesting is that 8 is better than 16. Both allow one seek to be dispatched, they only differ in the streamed amount of data we allow to dispatch. 8 will give you either 1 seek, or 8*256 == 2048 sectors == 1MiB. 16 will give you 1 seek or 2MiB of streamed I/O.
Tests with other io benchmarks need to be considered as well. And I need a bit of time to digest this :-). The 8 vs 16 numbers are not what I expected.
But the deadline io scheduler looks damn good in this test, if I have to say so myself.
Signature · 1 line
Jens Axboe
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Quoted 49 lines · Jens Axboe
Quoted 20 lines · Con Kolivas
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1Quoted 16 lines · Andrew Morton
Quoted 14 lines · Jens Axboe
Quoted 9 lines · Andrew Morton
Quoted 7 lines
io_load:Kernel Time CPU Ratio 2.4.19 216.05 33% 3.19 2.5.38 887.76 8% 13.11 2.5.38-mm3 105.17 70% 1.55 2.5.39 229.4 34% 3.4 2.5.39-mm1 239.5 33% 3.4I think I'll set
fifo_batchto 16 again...As not to compare oranges and apples, I'd very much like to see a 2.5.39-mm1 vs 2.5.39-mm1 with fifo_batch=16. Con, would you do that? Thanks!
Quoted 6 lines
Quoted 1 line
The presence of
/proc/sys/vm/fifo_batchshould make that pretty easy.Thanks. That made it a lot easier and faster, and made me curious enough to create a family or very interesting results. All these are with 2.5.39-mm1 with
fifo_batchset to 1->16, average of three runs. The first result is the unmodified 2.5.39-mm1 (fifo_batch=32).Ah excellent, thanks a lot!
Quoted 9 lines
io_load:Kernel Time CPU% Ratio 2.5.39-mm1 239.5 32 3.54 2539mm1fb16 131.2 57 1.94 2539mm1fb8 109.1 68 1.61 2539mm1fb4 146.4 51 2.16 2539mm1fb2 112.7 65 1.67 2539mm1fb1 125.4 60 1.85Quoted 3 lines
What's most interesting is the variation was small until the number was <8; then the variation between runs increased. Dare I say it there appears to be a sweet spot in the results.
Yes it's an interesting curve. What makes it interesting is that 8 is better than 16. Both allow one seek to be dispatched, they only differ in the streamed amount of data we allow to dispatch. 8 will give you either 1 seek, or 8*256 == 2048 sectors == 1MiB. 16 will give you 1 seek or 2MiB of streamed I/O.
Quoted 3 lines
Tests with other io benchmarks need to be considered as well. And I need a bit of time to digest this :-). The 8 vs 16 numbers are not what I expected.
It would seem reasonable to me to assume it may be related to the filesystem in use (in this case ReiserFS). If this is the case it is possible that each different filesystem has a different sweetspot? (and for that matter each piece of hardware, each type of ata driver etc etc..)
This was performed on an ATA100 5400rpm drive running ReiserFS. I'm afraid I don't have the hardware to do any other comparisons of different filesystems.
Quoted 2 lines
But the deadline io scheduler looks damn good in this test, if I have to say so myself.
Agree with you on that I can; Great job!
Con
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)
iD8DBQE9maa1F6dfvkL3i1gRAqX5AKCdYYuzvqe57tB+EjzwO2WNo0ik6QCfU0RS 9bQXpIFvSgYb5WKk+6T2NUU=
=Njny
-----END PGP SIGNATURE-----
Quoted 57 lines · Con Kolivas
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1Quoted 49 lines · Jens Axboe
Quoted 20 lines · Con Kolivas
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1Quoted 16 lines · Andrew Morton
Quoted 14 lines · Jens Axboe
Quoted 9 lines · Andrew Morton
Quoted 7 lines
io_load:Kernel Time CPU Ratio 2.4.19 216.05 33% 3.19 2.5.38 887.76 8% 13.11 2.5.38-mm3 105.17 70% 1.55 2.5.39 229.4 34% 3.4 2.5.39-mm1 239.5 33% 3.4I think I'll set
fifo_batchto 16 again...As not to compare oranges and apples, I'd very much like to see a 2.5.39-mm1 vs 2.5.39-mm1 with fifo_batch=16. Con, would you do that? Thanks!
Quoted 6 lines
Quoted 1 line
The presence of
/proc/sys/vm/fifo_batchshould make that pretty easy.Thanks. That made it a lot easier and faster, and made me curious enough to create a family or very interesting results. All these are with 2.5.39-mm1 with
fifo_batchset to 1->16, average of three runs. The first result is the unmodified 2.5.39-mm1 (fifo_batch=32).Ah excellent, thanks a lot!
Quoted 9 lines
io_load:Kernel Time CPU% Ratio 2.5.39-mm1 239.5 32 3.54 2539mm1fb16 131.2 57 1.94 2539mm1fb8 109.1 68 1.61 2539mm1fb4 146.4 51 2.16 2539mm1fb2 112.7 65 1.67 2539mm1fb1 125.4 60 1.85Quoted 3 lines
What's most interesting is the variation was small until the number was <8; then the variation between runs increased. Dare I say it there appears to be a sweet spot in the results.
Yes it's an interesting curve. What makes it interesting is that 8 is better than 16. Both allow one seek to be dispatched, they only differ in the streamed amount of data we allow to dispatch. 8 will give you either 1 seek, or 8*256 == 2048 sectors == 1MiB. 16 will give you 1 seek or 2MiB of streamed I/O.
Quoted 3 lines
Tests with other io benchmarks need to be considered as well. And I need a bit of time to digest this :-). The 8 vs 16 numbers are not what I expected.
Quoted 7 lines
It would seem reasonable to me to assume it may be related to the filesystem in use (in this case ReiserFS). If this is the case it is possible that each different filesystem has a different sweetspot? (and for that matter each piece of hardware, each type of ata driver etc etc..)
This was performed on an ATA100 5400rpm drive running ReiserFS. I'm afraid I don't have the hardware to do any other comparisons of different filesystems.
8 being better than 16 would seem to indicate a slower driver, which fits with what you have. The sweet spot for this particular benchmark may be 8, that may not be the sweet spot for some other benchmark though. 16 will perhaps make a good default, we shall see once more benchmarks are done.
I think you'll find that results will be similar on other file systems, the io scheduler settings will be more affected by the underlying hardware.
Signature · 1 line
Jens Axboe
Quoted 66 lines · Jens Axboe
Quoted 51 lines · Con Kolivas
Quoted 46 lines · Jens Axboe
Quoted 17 lines · Con Kolivas
Quoted 16 lines · Andrew Morton
Quoted 14 lines · Jens Axboe
Quoted 9 lines · Andrew Morton
Quoted 7 lines
io_load:Kernel Time CPU Ratio 2.4.19 216.05 33% 3.19 2.5.38 887.76 8% 13.11 2.5.38-mm3 105.17 70% 1.55 2.5.39 229.4 34% 3.4 2.5.39-mm1 239.5 33% 3.4I think I'll set
fifo_batchto 16 again...As not to compare oranges and apples, I'd very much like to see a 2.5.39-mm1 vs 2.5.39-mm1 with fifo_batch=16. Con, would you do that? Thanks!
Quoted 6 lines
Quoted 1 line
The presence of
/proc/sys/vm/fifo_batchshould make that pretty easy.Thanks. That made it a lot easier and faster, and made me curious enough to create a family or very interesting results. All these are with 2.5.39-mm1 with
fifo_batchset to 1->16, average of three runs. The first result is the unmodified 2.5.39-mm1 (fifo_batch=32).Ah excellent, thanks a lot!
Quoted 9 lines
io_load:Kernel Time CPU% Ratio 2.5.39-mm1 239.5 32 3.54 2539mm1fb16 131.2 57 1.94 2539mm1fb8 109.1 68 1.61 2539mm1fb4 146.4 51 2.16 2539mm1fb2 112.7 65 1.67 2539mm1fb1 125.4 60 1.85Quoted 3 lines
What's most interesting is the variation was small until the number was <8; then the variation between runs increased. Dare I say it there appears to be a sweet spot in the results.
Yes it's an interesting curve. What makes it interesting is that 8 is better than 16. Both allow one seek to be dispatched, they only differ in the streamed amount of data we allow to dispatch. 8 will give you either 1 seek, or 8*256 == 2048 sectors == 1MiB. 16 will give you 1 seek or 2MiB of streamed I/O.
Quoted 3 lines
Tests with other io benchmarks need to be considered as well. And I need a bit of time to digest this :-). The 8 vs 16 numbers are not what I expected.
Quoted 7 lines
It would seem reasonable to me to assume it may be related to the filesystem in use (in this case ReiserFS). If this is the case it is possible that each different filesystem has a different sweetspot? (and for that matter each piece of hardware, each type of ata driver etc etc..)
This was performed on an ATA100 5400rpm drive running ReiserFS. I'm afraid I don't have the hardware to do any other comparisons of different filesystems.
8 being better than 16 would seem to indicate a slower driver, which fits with what you have. The sweet spot for this particular benchmark may be 8, that may not be the sweet spot for some other benchmark though. 16 will perhaps make a good default, we shall see once more benchmarks are done.
Quoted 3 lines
I think you'll find that results will be similar on other file systems, the io scheduler settings will be more affected by the underlying hardware.
Just a thought but might the variability be due to physical cylinder boundaries?
1-2Mib is very close to cylider size. Getting down below that range and a significant number of requests might not cross boundaries affecting latency. Much larger requests and it will just be an issue of 1 boundary more or less.
Signature · 5 lines
________________________________________________________________ J.W. Schultz Pegasystems Technologies email address: [email protected] Remember Cernan and Schmitt
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoting Jens Axboe <[email protected]>:
Quoted 5 lines · Mike Galbraith
Quoted 4 lines
Quoted 3 lines · Con Kolivas
Quoted 2 lines · Andrew Morton
Quoted 1 line · Jens Axboe
On Mon, Sep 30 2002, Andrew Morton wrote:
[snip]
Quoted 14 lines
Quoted 10 lines
Quoted 10 lines
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
I think I'll set
fifo_batchto 16 again...As not to compare oranges and apples, I'd very much like to see a 2.5.39-mm1 vs 2.5.39-mm1 with fifo_batch=16. Con, would you do that? Thanks!
The presence of
/proc/sys/vm/fifo_batchshould make that pretty easy.Thanks. That made it a lot easier and faster, and made me curious enough
to
Quoted 1 line
Quoted 1 line
create a family or very interesting results. All these are with
2.5.39-mm1
Quoted 1 line
Quoted 1 line
with
fifo_batchset to 1->16, average of three runs. The first result is
[snip]
Quoted 7 lines
Quoted 1 line
Quoted 1 line
What's most interesting is the variation was small until the number was
<8;
Quoted 5 lines
Quoted 4 lines
then the variation between runs increased. Dare I say it there appears to
be
a sweet spot in the results.
Quoted 8 lines
Quoted 3 lines
What's more interesting (methinks) is the huge difference between 32 and
16. Have you played with values between 32 and 16? (/me wonders if there's a cliff or a gentle slope)As I wrote in response, the difference is that 16 ==
seek_cost. Sofifo_batchof 16 will allow 1 seek,fifo_batchof 32 will allow two. This is the reason for the big drop at that point. I would expect 31 to be pretty close to 16.
Ok well I've got the answer to both questions. I've removed the other results from this email for clarity, and here is a more complete family (average of 2 or 3 runs):
io_load:
Kernel Time CPU% Ratio 2539mm1fb01 125.4 60 1.85 2539mm1fb02 112.7 65 1.66 2539mm1fb04 146.4 51 2.16 2539mm1fb08 109.1 68 1.61 2539mm1fb10 204.3 63 3.02 2539mm1fb12 210.3 60 3.10 2539mm1fb14 192.6 66 2.85 2539mm1fb16 131.2 57 1.94 2539mm1fb18 209.7 61 3.10 2539mm1fb20 221.8 57 3.27 2539mm1fb22 262.3 48 3.88 2539mm1fb24 264.0 48 3.90 2539mm1fb26 258.7 50 3.82 2539mm1fb28 307.4 42 4.54 2539mm1fb30 319.4 40 4.71 2539mm1fb31 294.4 44 4.34 2539mm1fb32 239.5 32 3.54
For my machine at least it appears that falling outside the powers of 2 is not good.
Con