more efficient way to print out inode->block mappings?

I want to be able to rapidly determine the diffs between 2 volumes.

Special note 1 is an active lvm snapshot of the other -- meaning it is frozen in time, but otherwise should look identical the the file system as it was when it was snapped.

Sooo... a way of speeding checks is finding out what blocks allocated to the indoes are different, since as the new volume gets used, I had hoped that differing block numbers might give me a clue as to what had changed..

I'm thinking though, that the block numbers that will change will be the ones in lvm -- not xfs. So this probably isn't as useful as i'd hoped.

Nevertheless, trying to read the blocks allocated/inode with bmap is sorta slow. I've tried to optimize it by starting a pty session to xfs_db and issuing inode/bmap commands -- but I have to wait for a prompt to come back to know that the command has finished, and I'm not really sure it's really returning more than 1 line for any file. -- though interactively, I can find a file with a large ACL, and see it has both a data and attr bmap. I also haven't seen what output would look like if the file(or dir) was split -- as, so far, have only seen files/dirs returned that have 1 allocation/file,

So what the means is that I'm not sure about synchronization between commands output and the input I read in -- even though I read in the input after every command. -- but even with a minimal timeout of 1ms, and keeping track of commands outstanding and replies (as measured by 'prompts' recieved, I'm far from convinced it's doing the right thing -- and it still slow going 1 inode at a time over a pty interface.

I thought of trying to use blockget -v and parsing the output. I figured that would have the least latency and likely be the fastest way to dump the mappings -- BUT it seems I can't get it to work on an active file system. So how can I get that block info dumped without blockget? I've already told blockget it's in -r only mode... so it shouldn't try to repair inconsistencies...and 99.999% is going to be what I want and any inconsistencies, I can check manually be checking the files through the mounted interface.

Oddly, and likely I'm confused about something, but when i try to print the log/log_print, it says it is an invalid log 1 byte long, so even if it were played out, I don't think it would make much difference in the final results.

Is it possible to do what I want w/o writing a special util/C prog to dump this?

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

Hey Linda,

Differing block maps for the same inode on the two filesystems will give you a clue that there are changes but it isn't perfect:

Consider xfs_fsr. This might change block numbers for a file but the contents stayed the same.

Also consider an overwrite situation. The contents of the file are overwritten and have changed but the block map stayed the same. To detect that we'd need some kind of generation number on every extent, and we don't have that.

At this time I don't think we have a solid way to tell you which blocks of a file are different without actually comparing them. I think you are stuck looking at the mtime and then doing a full comparison.

Have you looked into using the xfs bulkstat ioctl interfaces? XFS_IOC_FSBULKSTAT won't get you the block map, but maybe it would be useful to you. xfs_bmap is using XFS_IOC_GETBMAP*, but it sounds like you've already considered that. Maybe a creative invocation of xfsdump?

Let me know if I misunderstood and went off the rails. ;)

HTH,
Ben

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

----
Yeah -- I figured out the downsides (like a rewrite -- but I was going
to try to use the mtime/filesize and block allocs to indicated stuff...

But neverthless-- it wouldn't be perfect, so sorta a moot point.

What I really need is to look at the block mappings of the underlying lvm -- and see what parts of the FS have been changed due to 'COW'....o

What i'm doing is taking successive daily snapshots that overlap each other and picking up the differences and putting them into a tiny static volume I'll probably keep around for a few weeks.

The reason -- is that when mounted under a 'special dir', @GMT-time/date, Windows will see those "differences" as previous versions of the files -- unlike the windows version, it keeps active snapshots going... lvm isn't quite so efficient to keep them going for weeks or so.. so I'm only keeping "sparse" trees of the differences.

The RUB in all of this is making the "difference" -- rsync takes anywhere from 80-100 minutes running local-local using no compression, and telling it to NOT try to checksum diffs, or send differences - but whole files only... (as anything that would do a checksum of the file will take longer than actually copying the file 2-3 times. It's a 1T partition with usually .8-2.5GB of change/day. I figure anything would be faster than rsync, which is why I'm trying to build up a diff-detector.

The resulting 'diff' only takes 30-100 seconds in 'cp' -- that's how little data is usually there... So I figured if any inodes or blocknumbers or sizes or timestamps change, that would be a good basis for finding any builk of changes.

It's NOT meant to be a backup mechanism, but a convenience mechanism. -- i.e. It's alot easier to browse to a file on the desktop, and right click and ask for previous versions than it is to restore the file from daily towerhanoi-xfs backups.

That's just so much effort, I really have to want the file -- but a right click and look for yesterday's copy? pretty simple.

---
Yeah, and I'm sure I can do it faster than rsync, BUT.. I'd rather take
a shortcut if I can catch most of the cases -- and leave the absolute catch everything to the dailies... But I could go either way for this type of application. Given it's known that it isn't going to keep exact backups, but is trying to keep snapshot of files that change -- if something is really written-to, as subtly as writing in place, from a windows client... and it takes 5 times as long... it might not be worth it to catch it.

But knowing how to dump the blocks could still be useful in walking
the directory and/or comparing files -- the directory walk alone -- if I know the blocks numbers of the dirs, I could walk the directory in the order of the next nearest block -- and when I encounter new dirs, do some type of binary-insertion based on the block numbers .. might get some benefit that way...

while my top speed is 1GB/s, I'm lucky to get 100MB on small random
i/o's with 20-50 being not-atypical.

Certainly an rsync that takes 90 minutes /day to run is a a bit of a pain --
though no one is really 'waiting on it' (other than me to see if I run into a new bug to fix! ;-) ).

OTOH, if I could do a snap and diff of the vol in 5 minutes... then doing
multiple snapshots in a day might be practical.

---
Yeah... my script is in perl right now ~ 2500 lines or so...mixing required 'C'
in with that would be a drag...

---
not so much -- considering how far off the rails my app is... ;-)

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs