An issue with fio performance on an SSD
Greetings
We are playing with some SSDs, and as usual, we want to use fio for
our testing regime. So we set up a simple streaming write:
[global]
size=16g
iodepth=32
direct=1
blocksize=1m
numjobs=1
nrfiles=1
ioengine=vsync
#iomem=shmhugegroup_reporting
create_serialize=0
create_on_open=1
rw=write
[sw1]
directory=/data/d1
With this test case, fio is getting ~65MB/s writing to this drive. 1M blocks, direct IO.
[root@localhost ~]# fio sw2.fio
sw1: (g=0): rw=write, bs=1M-1M/1M-1M, ioengine=vsync, iodepth=32 Starting 1 process
Jobs: 1 (f=1): [W] [99.6% done] [0K/62M /s] [0/61 iops] [eta 00m:01s]
sw1: (groupid=0, jobs=1): err= 0: pid=5528
write: io=16,353MB, bw=71,870KB/s, iops=70, runt=232998msec
clat (msec): min=9, max=231, avg=14.22, stdev= 8.98
bw (KB/s) : min=32188, max=99162, per=100.38%, avg=72139.10, stdev=15322.04
cpu : usr=0.06%, sys=1.56%, ctx=32769, majf=0, minf=26
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=99.8%,
Quoted 1 line
=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,
Quoted 1 line
=64=0.0%
complete : 0=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,
Quoted 1 line
=64=0.0%
issued r/w: total=0/16384, short=0/0
lat (msec): 10=11.57%, 20=83.69%, 50=4.06%, 100=0.20%, 250=0.29%Run status group 0 (all jobs):
WRITE: io=16,353MB, aggrb=71,869KB/s, minb=73,594KB/s,
maxb=73,594KB/s, mint=232998msec, maxt=232998msec
Disk stats (read/write):
sdc: ios=0/81927, merge=0/0, ticks=0/773830, in_queue=773650, util=99.43%
But when I try the simple dd version ...
[root@localhost ~]# dd if=/dev/zero of=/data/d1/big.file bs=1M count=16k oflag=direct 16384+0 records in
16384+0 records out
17179869184 bytes (17 GB) copied, 87.7913 seconds, 196 MB/s
Non-direct IO is faster with the dd, as I suspect it is overlapping operations
[root@localhost ~]# dd if=/dev/zero of=/data/d1/big.file bs=1M count=16k 16384+0 records in
16384+0 records out
17179869184 bytes (17 GB) copied, 64.0534 seconds, 268 MB/s
The same fio input deck, with direct set to 0 results in 76.5 MB/s.
Are there are some IO tunables (in the fio input) I should be using for SSD testing with fio?
This is 1.41 fio:
[root@localhost ~]# fio --version
fio 1.41
Thanks!
Joe
Signature · 9 lines
Joseph Landman, Ph.D Founder and CEO Scalable Informatics Inc. email: [email protected] web : http://scalableinformatics.com http://scalableinformatics.com/jackrabbit phone: +1 734 786 8423 x121 fax : +1 866 888 3112 cell : +1 734 612 4615
Quoted 4 lines · Joe Landman
Greetings
We are playing with some SSDs, and as usual, we want to use fio for
our testing regime. So we set up a simple streaming write:
Ok, I figured it out.
Add zero_buffers=1 to the job specification. Then we get this:
Run status group 0 (all jobs):
WRITE: io=32,706MB, aggrb=535MB/s, minb=548MB/s, maxb=548MB/s,
mint=61133msec, maxt=61133msec
Signature · 9 lines
Joseph Landman, Ph.D Founder and CEO Scalable Informatics Inc. email: [email protected] web : http://scalableinformatics.com http://scalableinformatics.com/jackrabbit phone: +1 734 786 8423 x121 fax : +1 866 888 3112 cell : +1 734 612 4615
Quoted 6 lines · Joe Landman
Quoted 4 lines · Joe Landman
Greetings
We are playing with some SSDs, and as usual, we want to use fio
for our testing regime. So we set up a simple streaming write:
Quoted 7 lines
Ok, I figured it out.
Add zero_buffers=1 to the job specification. Then we get this:
Run status group 0 (all jobs):
WRITE: io=32,706MB, aggrb=535MB/s, minb=548MB/s, maxb=548MB/s,
mint=61133msec, maxt=61133msec
Interesting. Why not initializing bufferes with zero leads to poor performance?
Vivek
Quoted 17 lines
Signature · 14 lines
-- Joseph Landman, Ph.D Founder and CEO Scalable Informatics Inc. email: [email protected] web : http://scalableinformatics.com http://scalableinformatics.com/jackrabbit phone: +1 734 786 8423 x121 fax : +1 866 888 3112 cell : +1 734 612 4615 -- To unsubscribe from this list: send the line "unsubscribe fio" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 8 lines · Vivek Goyal
Quoted 7 lines · Joe Landman
Quoted 4 lines · Joe Landman
Greetings
We are playing with some SSDs, and as usual, we want to use fio
for our testing regime. So we set up a simple streaming write:Ok, I figured it out.
Quoted 8 lines
Quoted 5 lines
Add zero_buffers=1 to the job specification. Then we get this:
Run status group 0 (all jobs):
WRITE: io=32,706MB, aggrb=535MB/s, minb=548MB/s, maxb=548MB/s,
mint=61133msec, maxt=61133msecInteresting. Why not initializing bufferes with zero leads to poor performance?
I think this is an SSD controller compression issue. That is, the Sandforce controllers want to do compression of some sort on the data. So if we start out with truly random data, which doesn't get good compression ratios, we get not so good performance.
Joe
Signature · 9 lines
Joseph Landman, Ph.D Founder and CEO Scalable Informatics Inc. email: [email protected] web : http://scalableinformatics.com http://scalableinformatics.com/jackrabbit phone: +1 734 786 8423 x121 fax : +1 866 888 3112 cell : +1 734 612 4615