xfs_freeze -f misbehaving under lenny / xfsprogs 2.9.8
Hi,
We have a backup script running on another machine that ssh's in to the affected server and does the following:
mkdir -p /tmp/foo;/usr/sbin/xfs_freeze -f /home;/sbin/lvcreate -s -L 20G -n snap-shot home ;/usr/sbin/xfs_freeze -u /home;
mount -o nouuid,ro /dev/data/snap-shot /tmp/foo;
It then rsyncs (over ssh) the data to the backup store from /tmp/foo
The above command set hangs at running "/sbin/lvcreate -s -L 20G -n snap-shot home;"
All I/O to /home is of course blocked at this point so for example exim starts queueing up all the mail.
As soon as I manually log in some hours later and run /usr/sbin/xfs_freeze -u /home; ... the lvcreate succeeds, the backup runs etc.
We recently upgraded this server to lenny from etch. It is still using the same kernel as it did with etch. The kernel is 2.6.26-bpo.1-amd64.
So this has never happened before and it occurring directly coincides with my upgrade to lenny on Friday.
The error occurred on Friday night
Backup proceeded normally on Saturday night
The error occurred on Sunday night
This suggests to me that there's some sort of race going on?
Previous version of xfsprogs (using etch): 2.8.11-1 Current version of xfsprogs (using lenny): 2.9.8-1lenny1
Anyone have any thoughts / is this a known issue with the 2.9.8 release?
Kind regards,
Stu.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 10 lines · Stuart Rowan
Hi,
We have a backup script running on another machine that ssh's in to the affected server and does the following:
mkdir -p/tmp/foo;/usr/sbin/xfs_freeze-f /home;/sbin/lvcreate-s -L 20G -n snap-shot home ;/usr/sbin/xfs_freeze-u /home;
mount -o nouuid,ro/dev/data/snap-shot/tmp/foo;
Quoted 7 lines
It then rsyncs (over ssh) the data to the backup store from
/tmp/fooThe above command set hangs at running "
/sbin/lvcreate-s -L 20G -n snap-shot home;"All I/O to /home is of course blocked at this point so for example exim starts queueing up all the mail.
lvcreate now does the fs freeze on its own via the snapshot ioctl, so if you run freeze manually first, you are stuck behind that first freeze.
Just drop the xfs_freeze's from the above, and all should be well.
-Eric
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Eric Sandeen wrote, on 30/03/09 14:45:
Quoted 17 lines
Quoted 9 lines · Stuart Rowan
Hi,
We have a backup script running on another machine that ssh's in to the affected server and does the following:
mkdir -p/tmp/foo;/usr/sbin/xfs_freeze-f /home;/sbin/lvcreate-s -L 20G -n snap-shot home ;/usr/sbin/xfs_freeze-u /home;
mount -o nouuid,ro/dev/data/snap-shot/tmp/foo;Quoted 5 lines
It then rsyncs (over ssh) the data to the backup store from
/tmp/fooThe above command set hangs at running "
/sbin/lvcreate-s -L 20G -n snap-shot home;"All I/O to /home is of course blocked at this point so for example exim starts queueing up all the mail.
Quoted 6 lines
lvcreate now does the fs freeze on its own via the snapshot ioctl, so if you run freeze manually first, you are stuck behind that first freeze.
Just drop the xfs_freeze's from the above, and all should be well.
-Eric
Eric,
Many thanks for your prompt reply and explanation :-)
It's good to know that there's an easy solution ... except we now have to differentiate the commands to run in the backup script based on the version of lvm on the remote system :-$
OOI when implementing the freeze ioctl, what made the developers decide that a freeze can't succeed on an already frozen filesystem ... you'd expect it to just be a no-op really?
Cheers,
Stu.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 25 lines · Stuart Rowan
Eric Sandeen wrote, on 30/03/09 14:45:
Quoted 24 lines
Quoted 10 lines · Stuart Rowan
Hi,
We have a backup script running on another machine that ssh's in to the affected server and does the following:
mkdir -p/tmp/foo;/usr/sbin/xfs_freeze-f /home;/sbin/lvcreate-s -L 20G -n snap-shot home ;/usr/sbin/xfs_freeze-u /home;
mount -o nouuid,ro/dev/data/snap-shot/tmp/foo;Quoted 7 lines
It then rsyncs (over ssh) the data to the backup store from
/tmp/fooThe above command set hangs at running "
/sbin/lvcreate-s -L 20G -n snap-shot home;"All I/O to /home is of course blocked at this point so for example exim starts queueing up all the mail.
lvcreate now does the fs freeze on its own via the snapshot ioctl, so if you run freeze manually first, you are stuck behind that first freeze.
Just drop the xfs_freeze's from the above, and all should be well.
Quoted 8 lines
Quoted 1 line
-Eric
Eric,
Many thanks for your prompt reply and explanation :-)
It's good to know that there's an easy solution ... except we now have to differentiate the commands to run in the backup script based on the version of lvm on the remote system :-$
can't be *that* bad ... ;)
Quoted 3 lines
OOI when implementing the freeze ioctl, what made the developers decide that a freeze can't succeed on an already frozen filesystem ... you'd expect it to just be a no-op really?
To complicate matters more, on newer upstream kernels w/ the freeze ioctl exposed for all filesystems, nested freezes are in fact supported.
I'm not sure why sequential freezes were serialized initially, TBH...
-Eric
Quoted 3 lines
Cheers,
Stu.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 3 lines · On Montag 30 März 2009 Stuart Rowan wrote:
OOI when implementing the freeze ioctl, what made the developers decide that a freeze can't succeed on an already frozen filesystem ... you'd expect it to just be a no-op really?
So you would break the unfreeze:
xfs_freeze -f
lvcreate -s (lvcreate does freeze;work;unfreeze)
*now you are unfrozen*xfs_freeze -u
The problem would be if you expect to be frozen still after the lvcreate command and do some processing, that might be bad.
mfg zmi
Signature · 10 lines
// Michael Monnerie, Ing.BSc ----- http://it-management.at // Tel: 0660 / 415 65 31 .network.your.ideas. // PGP Key: "curl -s http://zmi.at/zmi.asc | gpg --import" // Fingerprint: AC19 F9D5 36ED CD8A EF38 500E CE14 91F7 1C12 09B4 // Keyserver: wwwkeys.eu.pgp.net Key-ID: 1C1209B4 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs