Skip to contentSkip to conversation

snapshots of busy ext2 file system corrupt

linux-lvm Urs Thuermann 5 participants 10 messages Feb 26, 2002 – Feb 27, 2002

lore
Share & tools
lore
#0 lore raw

I tried creating a snapshot of a ext2 file system on a LV. I expected a fsck on the snapshot device to give no errors, i.e. a consistent file system. This is ok, as long as there is no (or little) load on the file system when creating the snapshot.

However, on a busy file system, the created snapshot has lots of errors found by fsck.

Is it this what the VFS lock patch is for? If so, why hasn't it been integrated into the standard kernel? Has it flaws in it? Maybe performance wise?

urs

#1 lore raw
Quoted 4 lines · Urs Thuermann

I tried creating a snapshot of a ext2 file system on a LV. I expected a fsck on the snapshot device to give no errors, i.e. a consistent file system. This is ok, as long as there is no (or little) load on the file system when creating the snapshot.

Yes.

Quoted 6 lines

However, on a busy file system, the created snapshot has lots of errors found by fsck.

Is it this what the VFS lock patch is for? If so, why hasn't it been integrated into the standard kernel? Has it flaws in it? Maybe performance wise?

The VFS lock patch only applies to journaled filesystems. There has not been any attempt to include support for sync_super_lockfs for ext2. I _think_ that if you tried to mount the ext2 snapshot, it will still work, even though it is inconsistent. Obviously it would be better to fix it so that ext2 snapshots are always consistent.

Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/

#2 lore raw
Quoted 15 lines · On Tuesday, February 26, 2002 09:59:19 AM -0700 Andreas Dilger…
Quoted 4 lines · Urs Thuermann

I tried creating a snapshot of a ext2 file system on a LV. I expected a fsck on the snapshot device to give no errors, i.e. a consistent file system. This is ok, as long as there is no (or little) load on the file system when creating the snapshot.

Yes.

Quoted 6 lines

However, on a busy file system, the created snapshot has lots of errors found by fsck.

Is it this what the VFS lock patch is for? If so, why hasn't it been integrated into the standard kernel? Has it flaws in it? Maybe performance wise?

Quoted 5 lines

The VFS lock patch only applies to journaled filesystems. There has not been any attempt to include support for sync_super_lockfs for ext2. I _think_ that if you tried to mount the ext2 snapshot, it will still work, even though it is inconsistent. Obviously it would be better to fix it so that ext2 snapshots are always consistent.

Yes, its very easy for the journaled filesystems (except XFS ;-) to do the snapshot locking, since snapshot locking is really just grab a lock on the log, finish transactions, let LVM do its thing, then release the lock.

For ext2, the best we could do would probably be a hack based on locking the super. As far as I know, nobody is planning on this, especially given how easy it is to go from ext2->ext3.

-chris

#3 lore raw
Quoted 2 lines · Chris Mason

For ext2, the best we could do would probably be a hack based on locking the super. As far as I know, nobody is planning on this,

:-(

Quoted 1 line

especially given how easy it is to go from ext2->ext3.

Some performance tests I made with ext2, ext3, and reiserfs convinced me to stay with ext2. ext3 was significantly slower than ext2 on some operations (and reiserfs even worse than that).

urs

#4 lore raw
Quoted 5 lines · On Tuesday, February 26, 2002 08:28:40 PM +0100 Urs Thuermann…
Quoted 2 lines · Chris Mason

For ext2, the best we could do would probably be a hack based on locking the super. As far as I know, nobody is planning on this,

Quoted 7 lines

:-(

Quoted 1 line

especially given how easy it is to go from ext2->ext3.

Some performance tests I made with ext2, ext3, and reiserfs convinced me to stay with ext2. ext3 was significantly slower than ext2 on some operations (and reiserfs even worse than that).

You trade speed for safety sometimes, but you can lower the performance hit by mounting ext3 with -o data=writeback, and help both filesystems by playing with bdflush values (increment the metadata timeout up from 5 seconds).

Other tuning depends on your app.

-chris

#5 lore raw
Quoted 5 lines · Chris Mason
Quoted 3 lines · On Tuesday, February 26, 2002 08:28:40 PM +0100 Urs Thuermann…
Quoted 2 lines · Chris Mason

For ext2, the best we could do would probably be a hack based on locking the super. As far as I know, nobody is planning on this,

It probably wouldn't be hard to do, just create a function "ext2_write_super_lockfs()" which is a call to lock an ext2-specific superblock semaphore and "ext2_unlockfs()" which is a call to unlock the same semaphore. You would then change all instances of "lock_super()" in the ext2 code to be ext2_lock_super() which will lock this semaphore in addition to the normal lock_super() call.

You can't just call lock_super() directly, because the write_super_lockfs() call itself calls this, and you would instantly deadlock.

Quoted 8 lines
Quoted 3 lines

Some performance tests I made with ext2, ext3, and reiserfs convinced me to stay with ext2. ext3 was significantly slower than ext2 on some operations (and reiserfs even worse than that).

You trade speed for safety sometimes, but you can lower the performance hit by mounting ext3 with -o data=writeback, and help both filesystems by playing with bdflush values (increment the metadata timeout up from 5 seconds).

Well, in many cases, data=ordered is actually faster than data=writeback because you get all of your journal I/O finished before you start writing to the FS. In some cases (e.g. mail spool, NFS) it is faster to have data=journal for the same reason (because sync I/O is very fast when it is written to the journal).

Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/

#6 lore raw
Quoted 5 lines · Andreas Dilger

Well, in many cases, data=ordered is actually faster than data=writeback because you get all of your journal I/O finished before you start writing to the FS. In some cases (e.g. mail spool, NFS) it is faster to have data=journal for the same reason (because sync I/O is very fast when it is written to the journal).

I had heard about data=journal on mail spools, but does anyone have a pointer with more performance comparisons between all 3?

Thanks,
Marc

Signature · 4 lines
Microsoft is to operating systems & security ....
                                      .... what McDonalds is to gourmet cooking
  
Home page: http://marc.merlins.org/   |   Finger [email protected] for PGP key
#7 lore raw
Quoted 9 lines · Marc MERLIN
Quoted 5 lines · Andreas Dilger

Well, in many cases, data=ordered is actually faster than data=writeback because you get all of your journal I/O finished before you start writing to the FS. In some cases (e.g. mail spool, NFS) it is faster to have data=journal for the same reason (because sync I/O is very fast when it is written to the journal).

I had heard about data=journal on mail spools, but does anyone have a pointer with more performance comparisons between all 3?

There was some discussion about it on one of (linux-kernel, ext3-users, ext2-devel), but since I read all of them, I don't know which one for sure.

I found:

http://bruce-guenter.dyndns.org/benchmarking/
http://www-106.ibm.com/developerworks/linux/library/l-fs8.html?dwzone=linux

Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/

#8 lore raw
In reply to Andreas Dilger
Quoted 2 lines · Andreas Dilger

I _think_ that if you tried to mount the ext2 snapshot, it will still work, even though it is inconsistent.

I think that wouldn't help me much, if I want to dump(8) the file system for backups. AFAIK, dump(8) opens the device file, i.e. /dev/vg0/snap which has an inconsistent image.

Quoted 2 lines

Obviously it would be better to fix it so that ext2 snapshots are always consistent.

ACK!

urs

#9 lore raw
In reply to Urs Thuermann
Quoted 11 lines · Urs Thuermann

I tried creating a snapshot of a ext2 file system on a LV. I expected a fsck on the snapshot device to give no errors, i.e. a consistent file system. This is ok, as long as there is no (or little) load on the file system when creating the snapshot.

However, on a busy file system, the created snapshot has lots of errors found by fsck.

Is it this what the VFS lock patch is for? If so, why hasn't it been integrated into the standard kernel? Has it flaws in it? Maybe performance wise?

It deadlocks for XFS on smp under heavy writeload. I use a writeable snapshot patch and replay the log afterwards.

I think, the VFS patch has some principal problems. Creating a snapshot with the VFS-lock patch applied is more or less equivalent to unmounting the file system, creating the snapshot of the device and remounting the file system. That means that all ongoing write operations must be suspended until the filesystem is in a "clean" state. This can take some time. Up to 15 minutes from my observations and that is way too long. I think the right way is: use a jornaling file system, take a snapshot, make the snapshot writeable, replay the log, make the snapshot readonly and dump it to tape or whatever you want. No races, no deadlocks, no problems.

Anselm

---
Anselm Kruis Tel. +49 (0)89-356386-74
science + computing ag FAX +49 (0)89-356386-37
Ingolstädter Str. 22 mailto: [email protected]
D-80807 München WWW: http://www.science-computing.de/

**********************************************************************

***     Wir freuen uns auf Ihren Besuch                            ***
***                                                                ***
***     CeBIT 2002                                                 ***
***     13.-20.03.2002, Messegelände Hannover, Halle 11, F 64      ***
***                                                                ***
***     Automotive Circle International                            ***
***     18.-19.04.2002, Bad Nauheim                                ***
***                                                                ***
***     VDI-Tagung: Berechnung und Simulation im Fahrzeugbau       ***

*** 01.-02.10.2002, Congress-Centrum Würzburg ***
*** ***
**********************************************************************