generating latency logs
I'm trying to generate a latency log, but no log file is ever created. I'm using the version tagged as fio-2.0.8 in git. This is how I'm invoking fio:
fio --latency-log=/tmp/log --output=/tmp/log random_write.fio
Here is the file random_write.fio:
[random-writers]
ioengine=libaio
iodepth=4
rw=randwrite
bs=4k
direct=0
size=64m
I tried to debug it a little myself by adding some print statements. In particular, at init.c:840 [1]:
if (td->o.write_lat_log) {
puts("latency logging");
setup_log(&td->lat_log, td->o.log_avg_msec);
setup_log(&td->slat_log, td->o.log_avg_msec);
setup_log(&td->clat_log, td->o.log_avg_msec);
}
else {
puts("not logging");
}[1] http://git.kernel.dk/?p=fio.git;f=init.c;hb=cf9a74#l840
When I run the command as above, I see the text "not logging". This seems odd.
Quoted 5 lines · Mike Ryan
I'm trying to generate a latency log, but no log file is ever created. I'm using the version tagged as fio-2.0.8 in git. This is how I'm invoking fio:
fio --latency-log=/tmp/log --output=/tmp/log random_write.fio
The latency/bw logs are named from the job name, so you can't actually give it a specific name with the (global) command line option. You'd need to use the job option to do that.
That said, it does look broken in that we don't inherit the global setting. Does the below make it work?
init.c +2 -0
@@ -1169,6 +1169,8 @@ static int fill_def_thread(void)fio_getaffinity (getpid (), &def_thread.o.cpumask); def_thread.o.timeout = def_timeout; + def_thread.o.write_lat_log = write_lat_log; + def_thread.o.write_bw_log = write_bw_log;/* * fill default options
Signature · 1 line
Jens Axboe
Quoted 5 lines · Jens Axboe
Quoted 1 line
fio --latency-log=/tmp/log --output=/tmp/log random_write.fioThe latency/bw logs are named from the job name, so you can't actually give it a specific name with the (global) command line option. You'd need to use the job option to do that.
The argument requires an option. It appears that option is discarded.
Quoted 2 lines
That said, it does look broken in that we don't inherit the global setting. Does the below make it work?
Yes! The files are named log_lat.log, log_slat.log, and log_clat.log. They are not named after the job.
very cool, so if I add the changes to init.c and run
fio --latency-log --output test.job
I get extra four files
-rw-r--r-- 1 me staff 14937 Aug 8 2012 fiotest_lat.log
-rw-r--r-- 1 me staff 0 Aug 8 2012 fiotest_slat.log -rw-r--r-- 1 me staff 14932 Aug 8 2012 fiotest_clat.log -rw-r--r-- 1 me staff 4194304 Aug 8 2012 fiotest.1.0
What's the format?
$ more fiotest_lat* 1, 19, 1, 4096 1, 18, 1, 4096 1, 8, 1, 4096 1, 7, 1, 4096 1, 7, 1, 4096
What's the format? OK, found http://www.spinics.net/lists/fio/msg01064.html
Format of entries is:
time, rate/latency, data direction, block size
Time: in milliseconds. Bandwidth logs are usually 500 or 1000ms apart; that can be controlled by the config file with "bwavgtime=[x ms]".
Rate/latency: for bandwidth, this is in KB/sec. For latency, it's microseconds.
Data direction: 0 is read, 1 is write.
Block size: block size of IOs in bytes.
the clat and lat look pretty much the same except clat is faster than lat (clat is completion time ie submission time plus I/O completion time - was never that clear on this)
$ paste fiotest_lat* fiotest_c* | more 1, 19, 1, 4096 1, 17, 1, 4096 1, 18, 1, 4096 1, 17, 1, 4096 1, 8, 1, 4096 1, 7, 1, 4096 1, 7, 1, 4096 1, 6, 1, 4096 1, 7, 1, 4096 1, 6, 1, 4096
what is the format of the file fiotest.1.0 ?
Is there a way to get the latency files without this file to avoid too much output.
Thanks
- Kyle
Quoted 19 lines · Mike Ryan
Quoted 5 lines · Jens Axboe
Quoted 1 line
fio --latency-log=/tmp/log --output=/tmp/log random_write.fioThe latency/bw logs are named from the job name, so you can't actually give it a specific name with the (global) command line option. You'd need to use the job option to do that.
The argument requires an option. It appears that option is discarded.
Quoted 2 lines
That said, it does look broken in that we don't inherit the global setting. Does the below make it work?
Yes! The files are named log_lat.log, log_slat.log, and log_clat.log. They are not named after the job.
--
To unsubscribe from this list: send the line "unsubscribe fio" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Signature · 7 lines
- Kyle O: +1.415.341.3430 F: +1.650.494.1676 275 Middlefield Road, Suite 50 Menlo Park, CA 94025 http://www.delphix.com
very cool, so if I add the changes to init.c and run
fio --latency-log --output test.job
I get extra four files
-rw-r--r-- 1 me staff 14937 Aug 8 2012 fiotest_lat.log
-rw-r--r-- 1 me staff 0 Aug 8 2012 fiotest_slat.log
-rw-r--r-- 1 me staff 14932 Aug 8 2012 fiotest_clat.log
-rw-r--r-- 1 me staff 4194304 Aug 8 2012 fiotest.1.0What's the format?
$ more fiotest_lat*
1, 19, 1, 4096
1, 18, 1, 4096
1, 8, 1, 4096
1, 7, 1, 4096
1, 7, 1, 4096What's the format? OK, found http://www.spinics.net/lists/fio/msg01064.html
Format of entries is:
time, rate/latency, data direction, block size
Time: in milliseconds. Bandwidth logs are usually 500 or 1000ms apart; that can be controlled by the config file with "bwavgtime=[x ms]".
Rate/latency: for bandwidth, this is in KB/sec. For latency, it's microseconds.
Data direction: 0 is read, 1 is write.
Block size: block size of IOs in bytes.
what is the format of the file fiotest.1.0 ?
Is there a way to get the latency files without this file to avoid too much output.
Thanks
- Kyle
Quoted 13 lines · Mike Ryan
Quoted 5 lines · Jens Axboe
Quoted 1 line
fio --latency-log=/tmp/log --output=/tmp/log random_write.fioThe latency/bw logs are named from the job name, so you can't actually give it a specific name with the (global) command line option. You'd need to use the job option to do that.
The argument requires an option. It appears that option is discarded.
Quoted 2 lines
That said, it does look broken in that we don't inherit the global setting. Does the below make it work?
Quoted 6 lines
Yes! The files are named log_lat.log, log_slat.log, and log_clat.log. They are not named after the job.
--
To unsubscribe from this list: send the line "unsubscribe fio" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
- Kyle
O: +1.415.341.3430
F: +1.650.494.1676
275 Middlefield Road, Suite 50
Menlo Park, CA 94025
http://www.delphix.com