Skip to contentSkip to conversation

Documentation on the binary file format?

DISCUSSION M. Edward (Ed) Borasky 2 participants 2 messages Jan 29, 2009 – Jan 30, 2009

lore
Share & tools
lore
#0 lore raw

Is there some documentation on the binary file format out of "blktrace"? I'm interested in writing some post-processing routines that extend what "blkparse" and "btt" do, and there is a substantial disk space hit if I generate the "blkparse" output and post-process that. I'm also considering a network server that takes the binary data from a "blktrace" client and does the analysis "on the fly", rather than saving off a binary trace.

A side question ... how likely are changes in the binary format? The language(s) I want to use -- Perl and/or Ruby -- are pretty flexible at re-factoring to match format changes, but I'd like to have a clue on how much maintenance activity I'm taking on.

Signature · 3 lines
M. Edward (Ed) Borasky

I've never met a happy clam. In fact, most of them were pretty steamed.
#1 lore raw
Quoted 7 lines · M. Edward (Ed) Borasky

Is there some documentation on the binary file format out of "blktrace"? I'm interested in writing some post-processing routines that extend what "blkparse" and "btt" do, and there is a substantial disk space hit if I generate the "blkparse" output and post-process that. I'm also considering a network server that takes the binary data from a "blktrace" client and does the analysis "on the fly", rather than saving off a binary trace.

Look at blktrace_api.h, that basically defines the binary format and API. In short, the binary trace is just a multiple of blk_io_trace{} structures. Note that they are always in big endian format, so the parser needs to convert them to machine local byte ordering first.

Quoted 4 lines

A side question ... how likely are changes in the binary format? The language(s) I want to use -- Perl and/or Ruby -- are pretty flexible at re-factoring to match format changes, but I'd like to have a clue on how much maintenance activity I'm taking on.

Not very likely. If a new format is incompatible with existing tools, the BLK_IO_TRACE_VERSION will change. So in your program, you basically do:

        if ((trace->magic & 0xff) != BLK_IO_TRACE_VERSION)
                err;

for starters, and then later if you want to handle different versions, you can just check for what version is reports and punt to the appropriate handler.

Signature · 1 line
Jens Axboe