Does fio write only 0x00s?
Hello,
I'm benchmarking a new FC-attached storage system which we are bringing into production, in order to see if it yields acceptable performance - and to have a baseline for potential performance trouble in the future.
One of the tools I'm using is fio, which is certainly great. Especially the fact that it includes data regarding latencies, and that it makes it easy to have a mix of reads and writes.
Now, compared to an another FC-attached storage system, and compared to a local RAID10, the new system yields fio numbers which are extremely much better. This made me have a quick and superficial look into the files which fio works with. I seems that the files contain only 0-bytes.
Does fio only write bytes with a single value?
If so: I would be afraid that some kind of compression/de-duplication/thin provisioning feature in the storage system invalidates the fio results I'm seeing. Is this a valid fear?
Signature · 3 lines
Regards, Troels Arvin <[email protected]> http://troels.arvin.dk/
Quoted 16 lines · Troels Arvin
Hello,
I'm benchmarking a new FC-attached storage system which we are bringing into production, in order to see if it yields acceptable performance - and to have a baseline for potential performance trouble in the future.
One of the tools I'm using is fio, which is certainly great. Especially the fact that it includes data regarding latencies, and that it makes it easy to have a mix of reads and writes.
Now, compared to an another FC-attached storage system, and compared to a local RAID10, the new system yields fio numbers which are extremely much better.
This made me have a quick and superficial look into the files which fio works with. I seems that the files contain only 0-bytes.
Quoted 4 lines
Does fio only write bytes with a single value?
If so: I would be afraid that some kind of compression/de-duplication/thin provisioning feature in the storage system invalidates the fio results I'm seeing. Is this a valid fear?
By default, fio will at init time randomly fill the buffer of the allocated IO units. If you are using the sync io engine, then only one buffer will be allocated and that will be repeatedly written. So yes, that'll compress very nicely. You can enable refill_buffers=1 and that'll cause fio to randomly fill it everytime it's submitted instead. That should effectively disable compression at the storage end.
But you should not be seeing zeroes only. Are you perhaps sparsely filling a file with random writes, and you are observing the holes?
Signature · 1 line
Jens Axboe
Hello,
Quoted 4 lines · Jens Axboe
By default, fio will at init time randomly fill the buffer of the allocated IO units. If you are using the sync io engine, then only one buffer will be allocated and that will be repeatedly written. So yes, that'll compress very nicely.
I'm using libaio. So the data from that shouldn't compress all that much?
Quoted 3 lines
You can enable refill_buffers=1 and
that'll cause fio to randomly fill it everytime it's submitted instead. That should effectively disable compression at the storage end.
Turning refill_buffers on or off doesn't seem to make much of a difference when compressing fil's work-file with "gzip -2" (201MB vs 198MB). But perhaps, "gzip -2" still compreses more than one can expect from a storage system?
I'm probably missing something. Here's the job description file I'm using (in this case only testing a small amount of I/O; when I was testing the storage systems, I used size=10g and six numjobs=6):
===========================================================
[global]
description=Emulation of Intel IOmeter File Server Access Pattern
[iometer] bssplit=512/10:1k/5:2k/5:4k/60:8k/2:16k/4:32k/4:64k/10 rw=randrw
rwmixread=70
direct=1
size=1g
ioengine=libaio
iodepth=256
refill_buffers=0
===========================================================
Signature · 3 lines
Regards, Troels Arvin <[email protected]> http://troels.arvin.dk/
Quoted 9 lines · Troels Arvin
Hello,
Quoted 4 lines · Jens Axboe
By default, fio will at init time randomly fill the buffer of the allocated IO units. If you are using the sync io engine, then only one buffer will be allocated and that will be repeatedly written. So yes, that'll compress very nicely.
I'm using libaio. So the data from that shouldn't compress all that much?
Each buffer used will have different contents, so it'll depend on how large the file is. If you end up writing the same 256 buffers (in your case) thousands of time, the output will still be compressible.
Quoted 8 lines
Quoted 3 lines
You can enable refill_buffers=1 and
that'll cause fio to randomly fill it everytime it's submitted instead. That should effectively disable compression at the storage end.Turning
refill_bufferson or off doesn't seem to make much of a difference when compressing fil's work-file with "gzip -2" (201MB vs 198MB). But perhaps, "gzip -2" still compreses more than one can expect from a storage system?
Hard to say really, if it's inline compression then perhaps not as strong, background data-dedup could be a lot stronger.
Quoted 7 lines
I'm probably missing something. Here's the job description file I'm using (in this case only testing a small amount of I/O; when I was testing the storage systems, I used size=10g and six numjobs=6):
===========================================================
[global]
description=Emulation of Intel IOmeter File Server Access Pattern
Quoted 10 lines
[iometer] bssplit=512/10:1k/5:2k/5:4k/60:8k/2:16k/4:32k/4:64k/10 rw=randrw
rwmixread=70
direct=1
size=1g
ioengine=libaio
iodepth=256
refill_buffers=0
===========================================================
I think what you are missing is that the random writes will create a large sparse file. The Output is 10G, and you are doing a lot of reads. So you could end up writing only 30% of the 10G, the rest would be sparse holes in the file.
Signature · 1 line
Jens Axboe
Hello,
Den 16-03-2010 09:05, skrev Jens Axboe:
Quoted 4 lines
I think what you are missing is that the random writes will create a large sparse file. The Output is 10G, and you are doing a lot of reads. So you could end up writing only 30% of the 10G, the rest would be sparse holes in the file.
Yes, that part about the file sparseness was I had to grasp.
Expressed in another way: If I filter out the null-bytes from fio's work-file (cat iometer.1.0 | tr -d '\0'), then the remainder doesn't compress at all using gzip with default options.
So I'll stop worrying about storage-side compression/de-duplication when interpreting fio's results.
Thanks.
Signature · 3 lines
Regards, Troels Arvin <[email protected]> http://troels.arvin.dk/