blockfile access patterns logging
Jens,
For some time I've wanted to log exactly what linux is reading and writing from my harddisk - for a variety of reasons. The current reason is that my very idle laptop writes to disk every once in a while (or reads, I don't know).
Now, conceptually this should not be very hard, but I'd like to ask your thoughts on where I might insert some crude logging? There are lots of places that might be better or worse for some reason.
I'd love to be as close to the physical block device as possible, short of rewriting actual IDE drivers.
Any tips? Or is this idea crazy?
Thanks!
Signature · 2 lines
http://www.PowerDNS.com Open source, database driven DNS Software http://lartc.org Linux Advanced Routing & Traffic Control HOWTO
Quoted 7 lines · bert hubert
Jens,
For some time I've wanted to log exactly what linux is reading and writing from my harddisk - for a variety of reasons. The current reason is that my very idle laptop writes to disk every once in a while (or reads, I don't know).
Quoted 8 lines
Now, conceptually this should not be very hard, but I'd like to ask your thoughts on where I might insert some crude logging? There are lots of places that might be better or worse for some reason.
I'd love to be as close to the physical block device as possible, short of rewriting actual IDE drivers.
Any tips? Or is this idea crazy?
If you have the laptop mode patch (it's in 2.4 current, and in 2.6-mm as well), then you can enable block dump by echoing 1 to /proc/sys/vm/block_dump - this will dump all reads/writes to any device in the system. Sounds like this is what you want.
Signature · 1 line
Jens Axboe
Quoted 11 lines · bert hubert
For some time I've wanted to log exactly what linux is reading and writing from my harddisk - for a variety of reasons. The current reason is that my very idle laptop writes to disk every once in a while (or reads, I don't know).
Now, conceptually this should not be very hard, but I'd like to ask your thoughts on where I might insert some crude logging? There are lots of places that might be better or worse for some reason.
I'd love to be as close to the physical block device as possible, short of rewriting actual IDE drivers.
You probably want to do logging at a higher level. It's totally useless to find out that LBA 2234324567 got re-written. Mapping it to a partition on the disk so you know it was something on /dev/hda7 is a bit better. And being able to tell that somebody updated the atime on /var/log/messages is most informative of all.
The other problem is that unless your laptop is *VERY* idle, you will have a scrolling problem and buffering issues - so you end up writing to disk to log the buffers and... ;)
Quoted 11 lines · On Thu, Jan 08 2004, [email protected] wrote:
Quoted 9 lines · bert hubert
For some time I've wanted to log exactly what linux is reading and writing from my harddisk - for a variety of reasons. The current reason is that my very idle laptop writes to disk every once in a while (or reads, I don't know).
Now, conceptually this should not be very hard, but I'd like to ask your thoughts on where I might insert some crude logging? There are lots of places that might be better or worse for some reason.
Quoted 8 lines
Quoted 2 lines
I'd love to be as close to the physical block device as possible, short of rewriting actual IDE drivers.
You probably want to do logging at a higher level. It's totally useless to find out that LBA 2234324567 got re-written. Mapping it to a partition on the disk so you know it was something on
/dev/hda7is a bit better. And being able to tell that somebody updated the atime on/var/log/messagesis most informative of all.
For laptops, it's often most interesting to find out _what_ process dirtied what data (which in turn caused bdflush to sync it), or what process keeps doing small reads. And block_dump does exactly that (it was invented for exactly that purpose :)
Quoted 3 lines
The other problem is that unless your laptop is *VERY* idle, you will have a scrolling problem and buffering issues - so you end up writing to disk to log the buffers and... ;)
I don't think you understand what Bert is looking for. He explicitly mentions that the machine is very idle, so he's probably looking for culprits that spin up the drive occasionally.
It doesn't provide tcpdump like logs of course, that's far more invasive.
Signature · 1 line
Jens Axboe
Quoted 4 lines · Jens Axboe
For laptops, it's often most interesting to find out _what_ process dirtied what data (which in turn caused bdflush to sync it), or what process keeps doing small reads. And
block_dumpdoes exactly that (it was invented for exactly that purpose :)
I submit that "what process ID did it" is even more remote from the disk than "what inode" ;)
Quoted 3 lines
I don't think you understand what Bert is looking for. He explicitly mentions that the machine is very idle, so he's probably looking for culprits that spin up the drive occasionally.
Oh, I understand very well.. Been there myself. I'd just forgotten that the laptop patch provided the process-ID information. ;)
Quoted 9 lines · On Thu, Jan 08 2004, [email protected] wrote:
Quoted 4 lines · Jens Axboe
For laptops, it's often most interesting to find out _what_ process dirtied what data (which in turn caused bdflush to sync it), or what process keeps doing small reads. And
block_dumpdoes exactly that (it was invented for exactly that purpose :)I submit that "what process ID did it" is even more remote from the disk than "what inode" ;)
There are no inodes involved at that point, it's left the fs and is purely in area between block layer and disk driver. Besides, what would do with the inode for this problem? If you really want to find eg the file in question (if you cannot _guess_ when you know the process), then you could go back from the block number. I've never ever had a need to go that far, though.
Signature · 1 line
Jens Axboe