2.5.70-mm2 causes performance drop of random read O_DIRECT
Starting in 2.5.70-mm2 and continuing in the mm tree, there is a significant degrade in random read for block devices using O_DIRECT. The drop occurs for all block sizes and ranges from 30%-40. CPU usage is also lower although it may already be so low as to be irrelavent.
tolerance = 0.00 + 3.00% of 2.5.70-mm1
2.5.70-mm1 2.5.70-mm2
Blocksize KBs/sec KBs/sec %diff diff tolerance
---------- ------------ ------------ -------- ------------ ------------
4096 1567 924 -41.03 -643.00 47.01 *
8192 3057 1815 -40.63 -1242.00 91.71 *
16384 5745 3509 -38.92 -2236.00 172.35 *
65536 17357 11283 -34.99 -6074.00 520.71 *
262144 37537 27302 -27.27 -10235.00 1126.11 * Full results can be found at:
http://www-124.ibm.com/developerworks/oss/linuxperf/regression/2.5.70-mm2/2.5.70-mm1-vs-2.5.70-mm2/
Steve
Quoted 1 line · Steven Pratt
Starting in 2.5.70-mm2 and continuing in the mm tree, there is a significant degrade in random read for block devices using
O_DIRECT. The drop occurs for all block sizes and ranges from 30%-40. CPU usage is also lower although it may already be so low as to be irrelavent.
Hi Steven, this is quite likely to be an io scheduler problem. Is your test program rawread v2.1.5? What is the command line you are using to invoke the program?
Quoted 6 lines · Nick Piggin
Quoted 1 line · Steven Pratt
Starting in 2.5.70-mm2 and continuing in the mm tree, there is a significant degrade in random read for block devices using
O_DIRECT. The drop occurs for all block sizes and ranges from 30%-40. CPU usage is also lower although it may already be so low as to be irrelavent.Hi Steven, this is quite likely to be an io scheduler problem. Is your test program rawread v2.1.5?
This test was actually using 2.1.4, but the only difference in the 2.1.5 version is a fix for the test label array for the aio versions of the test. No functional change, just fixed the outputed test description.
Quoted 1 line
What is the command line you are using to invoke the program?
rawread -t6 -p8 -m1 -d2 -s4096 -n65536 -l1 -z -x
Which you can find if you follow either results link and look in the benchmark directory where all raw benchmark out put is stored.
Steve
Quoted 11 lines · Steven Pratt
Quoted 7 lines · Nick Piggin
Quoted 1 line · Steven Pratt
Starting in 2.5.70-mm2 and continuing in the mm tree, there is a significant degrade in random read for block devices using
O_DIRECT. The drop occurs for all block sizes and ranges from 30%-40. CPU usage is also lower although it may already be so low as to be irrelavent.Hi Steven, this is quite likely to be an io scheduler problem. Is your test program rawread v2.1.5?
Quoted 9 lines
This test was actually using 2.1.4, but the only difference in the 2.1.5 version is a fix for the test label array for the aio versions of the test. No functional change, just fixed the outputed test description.
Quoted 1 line
What is the command line you are using to invoke the program?
rawread -t6 -p8 -m1 -d2 -s4096 -n65536 -l1 -z -x
Which you can find if you follow either results link and look in the benchmark directory where all raw benchmark out put is stored.
OK thanks, I can now reproduce this! I'll work on it.
Quoted 13 lines · Nick Piggin
Quoted 11 lines · Steven Pratt
Quoted 6 lines · Nick Piggin
Quoted 1 line · Steven Pratt
Starting in 2.5.70-mm2 and continuing in the mm tree, there is a significant degrade in random read for block devices using
O_DIRECT. The drop occurs for all block sizes and ranges from 30%-40. CPU usage is also lower although it may already be so low as to be irrelavent.Hi Steven, this is quite likely to be an io scheduler problem. Is your test program rawread v2.1.5?
This test was actually using 2.1.4, but the only difference in the 2.1.5 version is a fix for the test label array for the aio versions of the test. No functional change, just fixed the outputed test description.
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
What is the command line you are using to invoke the program?
rawread -t6 -p8 -m1 -d2 -s4096 -n65536 -l1 -z -x
Which you can find if you follow either results link and look in the benchmark directory where all raw benchmark out put is stored.
OK thanks, I can now reproduce this! I'll work on it.
Looks like Andrew beat you to it. Both 2.5.70-mm7 and mm8 are back up to the previous performance levels for random reads.
Steve
Quoted 22 lines · Steven Pratt
Quoted 20 lines · Nick Piggin
Quoted 18 lines · Steven Pratt
Quoted 7 lines · Nick Piggin
Quoted 1 line · Steven Pratt
Starting in 2.5.70-mm2 and continuing in the mm tree, there is a significant degrade in random read for block devices using
O_DIRECT. The drop occurs for all block sizes and ranges from 30%-40. CPU usage is also lower although it may already be so low as to be irrelavent.Hi Steven, this is quite likely to be an io scheduler problem. Is your test program rawread v2.1.5?
This test was actually using 2.1.4, but the only difference in the 2.1.5 version is a fix for the test label array for the aio versions of the test. No functional change, just fixed the outputed test description.
Quoted 1 line
What is the command line you are using to invoke the program?
rawread -t6 -p8 -m1 -d2 -s4096 -n65536 -l1 -z -x
Quoted 8 lines
Quoted 4 lines
Quoted 1 line
Which you can find if you follow either results link and look in the benchmark directory where all raw benchmark out put is stored.
OK thanks, I can now reproduce this! I'll work on it.
Looks like Andrew beat you to it. Both 2.5.70-mm7 and mm8 are back up to the previous performance levels for random reads.
No, that was me ;)
Odd, I was still seeing a regression with mm7, but thats fixed in mm8. Thanks for testing.