driver data: blktrace pass-through support for device
cover driver data: blktrace pass-through support for device
Low-level device drivers might have additional data which complements blktrace data. For example, zfcp, a SCSI HBA driver, is capable of obtaining additional latency information per request from HBAs. Those latencies allow to further break down the overall D2C request latency provided by blktrace.
We propose an additional trace for blktrace, called "driver data". It is a sort of small binary blob, which contains a low-level driver specific struct. Blktrace would be able to filter this trace (-a option) and include it within its binary output. A small device driver specific tool on top of blktrace would then analyze "driver data" traces.
Patch 1/3 makes the blktrace kernel code provide blk_add_driver_data() for use by device drivers.
Patch 2/3 adds support for driver data traces to blktrace tools.
Patch 3/3 makes zfcp provide additional request latency and queue utilization data through blktrace.
Patches are against scsi-misc and blktrace git.
Martin
Quoted 17 lines · Martin Peschke
Low-level device drivers might have additional data which complements blktrace data. For example, zfcp, a SCSI HBA driver, is capable of obtaining additional latency information per request from HBAs. Those latencies allow to further break down the overall D2C request latency provided by blktrace.
We propose an additional trace for blktrace, called "driver data". It is a sort of small binary blob, which contains a low-level driver specific struct. Blktrace would be able to filter this trace (-a option) and include it within its binary output. A small device driver specific tool on top of blktrace would then analyze "driver data" traces.
Patch 1/3 makes the blktrace kernel code provide
blk_add_driver_data()for use by device drivers.Patch 2/3 adds support for driver data traces to blktrace tools.
Quoted 8 lines
Patch 3/3 makes zfcp provide additional request latency and queue utilization data through blktrace.
Patches are against scsi-misc and blktrace git.
Martin
Hi Martin -
Have you thought about just using blk_add_trace_msg instead?
Alan
Re: [Patch 0/3] driver data: blktrace pass-through support for
In reply to Martin PeschkeQuoted 22 lines · Alan D. Brunelle
Quoted 21 lines · Martin Peschke
Low-level device drivers might have additional data which complements blktrace data. For example, zfcp, a SCSI HBA driver, is capable of obtaining additional latency information per request from HBAs. Those latencies allow to further break down the overall D2C request latency provided by blktrace.
We propose an additional trace for blktrace, called "driver data". It is a sort of small binary blob, which contains a low-level driver specific struct. Blktrace would be able to filter this trace (-a option) and include it within its binary output. A small device driver specific tool on top of blktrace would then analyze "driver data" traces.
Patch 1/3 makes the blktrace kernel code provide
blk_add_driver_data()for use by device drivers.Patch 2/3 adds support for driver data traces to blktrace tools.
Patch 3/3 makes zfcp provide additional request latency and queue utilization data through blktrace.
Quoted 8 lines
Quoted 4 lines
Patches are against scsi-misc and blktrace git.
Martin
Hi Martin -
Have you thought about just using
blk_add_trace_msginstead?
Hi Alan,
good point. Yes, we thought about it. It actually inspired blk_add_drv_data().
blk_add_trace_msg adds information to the textual output of blkparse. There is no point in forwarding it to the binary output.
We need the opposite. We want driver data to show up in binary ouput. But there is no point in blkparse trying to generate some ascii message from low-level device driver specific data.
Besides this would lead us down the path of parsing textual output for our monitoring tool. This would burn more CPU than a c-program which consumes binary data.
Martin
Quoted 42 lines · Martin Peschke
Quoted 22 lines · Alan D. Brunelle
Quoted 21 lines · Martin Peschke
Low-level device drivers might have additional data which complements blktrace data. For example, zfcp, a SCSI HBA driver, is capable of obtaining additional latency information per request from HBAs. Those latencies allow to further break down the overall D2C request latency provided by blktrace.
We propose an additional trace for blktrace, called "driver data". It is a sort of small binary blob, which contains a low-level driver specific struct. Blktrace would be able to filter this trace (-a option) and include it within its binary output. A small device driver specific tool on top of blktrace would then analyze "driver data" traces.
Patch 1/3 makes the blktrace kernel code provide
blk_add_driver_data()for use by device drivers.Patch 2/3 adds support for driver data traces to blktrace tools.
Patch 3/3 makes zfcp provide additional request latency and queue utilization data through blktrace.
Quoted 7 lines
Quoted 4 lines
Patches are against scsi-misc and blktrace git.
Martin
Hi Martin -
Have you thought about just using
blk_add_trace_msginstead?Hi Alan,
good point. Yes, we thought about it. It actually inspiredblk_add_drv_data().
blk_add_trace_msgadds information to the textual output of blkparse. There is no point in forwarding it to the binary output.We need the opposite. We want driver data to show up in binary ouput. But there is no point in blkparse trying to generate some ascii message from low-level device driver specific data.
Quoted 7 lines
Besides this would lead us down the path of parsing textual output for our monitoring tool. This would burn more CPU than a c-program which consumes binary data.
Martin
Hm, again, if the blktrace | blkiomon (direct, no blkparse) is how you do things, I would think that it would /not/ take all that much processing power to parse character strings. (I've heard that IBM makes some pretty fast processors... :-) ) In particular, if you make the character strings more machine readable (no text, just numbers) for example it's just some atoi's going on.
Alan
Re: [Patch 0/3] driver data: blktrace pass-through support
In reply to Martin PeschkeQuoted 2 lines · Alan D. Brunelle
Hm, again, if the blktrace | blkiomon (direct, no blkparse) is how you do things,
no, it's blktrace | blkparse | blkiomon
Isn't blkparse needed to sort all those per-CPU traces by time? I want this feature.
Quoted 2 lines
I would think that it would /not/ take all that much processing power to parse character strings.
Anyway, it's more expensive.
Quoted 1 line
(I've heard that IBM makes some pretty fast processors... :-) )
Sure. But we don't make our money from wasting their CPU power :-)
Quoted 8 lines · Martin Peschke
Quoted 2 lines · Alan D. Brunelle
Hm, again, if the blktrace | blkiomon (direct, no blkparse) is how you do things,
no, it's blktrace | blkparse | blkiomon
Isn't blkparse needed to sort all those per-CPU traces by time? I want this feature.
The traces should come out pretty close from blktrace itself - I'm assuming you're interested in putting out statistics in multiples of seconds. Perhaps some test cases should be tried, but...
Quoted 5 lines
Quoted 3 lines
I would think that it would /not/ take all that much processing power to parse character strings.
Anyway, it's more expensive.
... it certainly seems to me that if one could remove blkparse completely you'd save a /tremendous/ amount of CPU at the slight cost of perhaps having some off-by-a-few counts. (Meaning: You may have a few C's that you could not match up with D's (because they come out "later" from blktrace).)
Alan
Re: [Patch 0/3] driver data: blktrace pass-through
In reply to Martin PeschkeQuoted 5 lines · Alan D. Brunelle
Quoted 4 lines · Martin Peschke
Quoted 2 lines · Alan D. Brunelle
Hm, again, if the blktrace | blkiomon (direct, no blkparse) is how you do things,
Quoted 8 lines
Quoted 4 lines
no, it's blktrace | blkparse | blkiomon
Isn't blkparse needed to sort all those per-CPU traces by time? I want this feature.
The traces should come out pretty close from blktrace itself - I'm assuming you're interested in putting out statistics in multiples of seconds. Perhaps some test cases should be tried, but...
I will give it a try
Quoted 2 lines
... it certainly seems to me that if one could remove blkparse completely you'd save a /tremendous/ amount of CPU
sounds tempting to me
Quoted 4 lines
at the slight cost of
perhaps having some off-by-a-few counts. (Meaning: You may have a few C's that you could not match up with D's (because they come out "later" from blktrace).)
I had to make provisions for this case in blkiomon anyway.
I suspect blkparse's batch handling to deliver some C's earlier than the matching D's. But I have never tracked it down.
Thanks,
Martin
Re: [Patch 0/3] driver data: blktrace pass-through support for
In reply to Martin PeschkeQuoted 5 lines · Alan D. Brunelle
Quoted 4 lines · Martin Peschke
Quoted 3 lines · Alan D. Brunelle
Hm, again, if the blktrace | blkiomon (direct, no blkparse) is how you do things,
blktrace | blkiomon works out of the box :-)
Quoted 9 lines · Martin Peschke
Quoted 4 lines · Alan D. Brunelle
Quoted 3 lines · Martin Peschke
Quoted 2 lines · Alan D. Brunelle
Hm, again, if the blktrace | blkiomon (direct, no blkparse) is how you do things,
blktrace | blkiomon works out of the box :-)
Great - that should remove a lot of CPU processing...
Alan
cover driver data: blktrace pass-through support for device
In reply to Martin PeschkeHi Jens,
may I ask what you think about this approach? Would you consider including the following patch series in blktrace?
Here is a discussion dating back to the first time I have posted these patches:
http://marc.info/?l=linux-btrace&m1620751101386&w=2
Thanks,
Martin
Low-level device drivers might have additional data which complements blktrace data. For example, zfcp, a SCSI HBA driver, is capable of obtaining additional latency information per request from HBAs. Those latencies allow to further break down the overall D2C request latency provided by blktrace.
We propose an additional trace for blktrace, called "driver data". It is a sort of small binary blob, which contains a low-level driver specific struct. Blktrace would be able to filter this trace (-a option) and include it within its binary output. A small device driver specific tool on top of blktrace would then analyze "driver data" traces.
Patch 1/3 makes the blktrace kernel code provide blk_add_driver_data() for use by device drivers.
Patch 2/3 adds support for driver data traces to blktrace tools.
Patch 3/3 makes zfcp provide additional request latency and queue utilization data through blktrace.
Patches are against scsi-misc and blktrace git.
Re: [Patch 0/3] driver data: blktrace pass-through support for
In reply to Martin PeschkeHi Jens,
this one might have slipped through your fingers.
This small extension would be really useful for us to complement blktrace data.
What do you think - yes or no, or maybe?
Thanks,
Martin
Quoted 43 lines · Martin Peschke
Hi Jens,
may I ask what you think about this approach? Would you consider including the following patch series in blktrace?Here is a discussion dating back to the first time I have posted these patches:
http://marc.info/?l=linux-btrace&m1620751101386&w=2Thanks,
MartinLow-level device drivers might have additional data which complements blktrace data. For example, zfcp, a SCSI HBA driver, is capable of obtaining additional latency information per request from HBAs. Those latencies allow to further break down the overall D2C request latency provided by blktrace.
We propose an additional trace for blktrace, called "driver data". It is a sort of small binary blob, which contains a low-level driver specific struct. Blktrace would be able to filter this trace (-a option) and include it within its binary output. A small device driver specific tool on top of blktrace would then analyze "driver data" traces.
Patch 1/3 makes the blktrace kernel code provide
blk_add_driver_data()for use by device drivers.Patch 2/3 adds support for driver data traces to blktrace tools.
Patch 3/3 makes zfcp provide additional request latency and queue utilization data through blktrace.
Patches are against scsi-misc and blktrace git.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrace" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html