Data safety horror stories?
I heard that, in case of a power failure, XFS may lose data, even data that was already existing on the disk. For example, I heard horror stories of files being overwritten with zeros.
Are those stories true?
If so:
* Do you recommend not using XFS on devices that may frequently fail due to power failure?
* Is it possible to find out what files have been damaged? If not, will only files be affected that have been changed during the last couple of hours?
* Are there options to increase data safety? Should one run a regular "sync" in a cron job?
* Is it unsafe to use XFS in a virtual machine which may sometimes be terminated without proper shutdown? Suppose the virtual machine is running under Windows, and Windows may sometimes be terminated without proper shutdown.
I currently am using XFS under Ubuntu 7.10 (Kernel 2.6.22), running in a virtual machine (VMware) under Windows. The XFS file system is in a native partition on a second HDD.
Signature · 4 lines
Felix E. Klee Jabber/Google Talk: [email protected], SIP: [email protected] ICQ: 158124695, Yahoo!: feklee, AIM: [email protected] Gizmo: felixklee, Skype: felix.klee
Quoted 5 lines · Felix E. Klee
I heard that, in case of a power failure, XFS may lose data, even data that was already existing on the disk. For example, I heard horror stories of files being overwritten with zeros.
Are those stories true?
No, XFS will not lose any data that the application has committed to the disk.
Improperly written applications and/or improperly configured systems might have issues with recently written files losing data.
FWIW: I have never lost data with XFS, neither on home computers nor in SAN environments (in the presence of link/path failure).
Be sure the read the FAQ, especially the section about write cache on dekstop/consumer HDDs.
Just my opinion as an XFS user, your mileage might vary.
iustin
Hi Justin,
thanks for the info!
Quoted 2 lines · Iustin Pop
No, XFS will not lose any data that the application has committed to the disk.
OK, but just to make sure: The following FAQ entry refers only to *newly* created files - right?
http://oss.sgi.com/projects/xfs/faq.html#nulls
Quoted 2 lines
Improperly written applications and/or improperly configured systems might have issues with recently written files losing data.
Again, just to make sure that I understood you correctly: Could you name an example?
Quoted 2 lines
Be sure the read the FAQ, especially the section about write cache on dekstop/consumer HDDs.
You are probably referring to the following entry. I now disabled the write cache of the 2nd HDD in Windows (remember my configuration). I wonder though: Wouldn't it be possible to design journaling in a way so that the write cache does never cause problems? Could someone provide an example which illustrates the write cache problem in simple terms?
http://oss.sgi.com/projects/xfs/faq.html#wcache
Quoted 1 line
Just my opinion as an XFS user, your mileage might vary.
Hopefully, it's not about opinions ...
- Felix
Signature · 4 lines
Dipl.-Phys. Felix E. Klee Naunynstr. 2, 76530 Baden-Baden, Germany Tel.: +49 7221 396961, Fax: +49 7221 396960, Mobile: +49 174 1386060 http://www.linkedin.com/in/feklee
Quoted 10 lines · Felix E. Klee
I heard that, in case of a power failure, XFS may lose data, even data that was already existing on the disk. For example, I heard horror stories of files being overwritten with zeros.
Are those stories true?
If so:
* Do you recommend not using XFS on devices that may frequently fail due to power failure?
Depends on how much you care about your system and data.
I use XFS on write-cache enabled SATA drives without barriers with no UPS (yes, it's unsafe!) and I lose power at least once a week. I haven't had a data loss or corruption in over two years and tens of power failures....
Quoted 1 line
* Is it possible to find out what files have been damaged?
Not easily.
Quoted 3 lines
If not,
will only files be affected that have been changed during the last couple of hours?
Last few seconds before the power fail, actually.
Quoted 2 lines
* Are there options to increase data safety? Should one run a regular "sync" in a cron job?
If you are truly paranoid - turn off drive caching and mount with the 'wsync' option.
Quoted 2 lines
* Is it unsafe to use XFS in a virtual machine which may sometimes be terminated without proper shutdown?
I do that all the time, too. Corruption is rare and usually as a result of some bug in the code I'm testing ;)
Quoted 3 lines
I currently am using XFS under Ubuntu 7.10 (Kernel 2.6.22), running in a virtual machine (VMware) under Windows. The XFS file system is in a native partition on a second HDD.
Should be just fine. If you are really concerned - test it.
Cheers,
Dave.
Quoted 7 lines
Signature · 5 lines
-- Felix E. Klee Jabber/Google Talk: [email protected], SIP: [email protected] ICQ: 158124695, Yahoo!: feklee, AIM: [email protected] Gizmo: felixklee, Skype: felix.klee
Signature · 3 lines
Dave Chinner Principal Engineer SGI Australian Software Group
Am Montag 11 Februar 2008 schrieb Felix E. Klee:
Quoted 1 line
Hi Justin,
Hi Felix,
Quoted 5 lines
Quoted 2 lines
Improperly written applications and/or improperly configured systems might have issues with recently written files losing data.
Again, just to make sure that I understood you correctly: Could you name an example?
I recommend to read my article about write barriers and journalling filesystems for some basic understanding.
http://www.linux-magazin.de/heft_abo/sonderheft/2006/04/beschraenktes_schreiben?category=0
its in german tough. Actually I thought it should have been translated to english for Linux Magazine under the title "Imposing Order", but I do not find it on the net and it might have not been published.
Note: There is a mistake in it regarding history of write barriers. Write barriers are not supported via device mapper. Thats why I do not use LVM on my notebook for my productive data.
Quoted 3 lines
Quoted 1 line
Just my opinion as an XFS user, your mileage might vary.
Hopefully, it's not about opinions ...
I had not had any problems since quite some kernel releases. And there were some power losses involved ;). At earlier times Akregator (an KDE newsreader) said its opml was broken and a backup was restored after such a power loss... but not since quite some kernel releases.
I had one nasty problem with corrupt data in files but that was with a kernel that was patched with Con Koliva's patchset and didn't happen in mainline.
So since 2.6.17.7 XFS has been pretty stable for me. On a ThinkPad T42, a ThinkPad T23 and half a dozen of workstations at work ;-).
Ciao,
Signature · 2 lines
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7
Quoted 6 lines · Martin Steigerwald
I recommend to read my article about write barriers and journalling filesystems for some basic understanding.
http://www.linux-magazin.de/heft_abo/sonderheft/2006/04/beschraenktes_schreiben?category=0
its in german tough.
Dankeschön - Deutsch ist doch kein Problem. :-)
Signature · 4 lines
Dipl.-Phys. Felix E. Klee Naunynstr. 2, 76530 Baden-Baden, Germany Tel.: +49 7221 396961, Fax: +49 7221 396960, Mobile: +49 174 1386060 http://www.linkedin.com/in/feklee
Am Montag 18 Februar 2008 schrieb Felix E. Klee:
Quoted 6 lines
On Mon, 18 Feb 2008 22:28:45 +0100, "Martin Steigerwald"
Quoted 3 lines · <[email protected]> said:
I recommend to read my article about write barriers and journalling filesystems for some basic understanding.
Quoted 6 lines
Quoted 4 lines
http://www.linux-magazin.de/heft_abo/sonderheft/2006/04/beschraenktes
_schreiben?category=0its in german tough.
Dankeschön - Deutsch ist doch kein Problem. :-)
One thing to add: We have XFS also running on a web cluster for a customer with softraid 1 over two RAID arrays. We made sure that write caching is disabled as those RAID arrays are not equipped with NVRAM (could be added tough, but the customer did not yet). Runs marvelously. Its also running on some individual servers in that cluster as well.
Well and although the cluster is spread across two locations about 1 kilometer from each other and it was said that power supply could not fail... well it failed once for both data centers and the cluster survived that just fine ;-).
BZW I didn't broke that GPG signature. It seems the mailinglist software being used for the xfs mailinglist mangles mails.
Signature · 2 lines
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7
Quoted 4 lines · Felix E. Klee
Hi Justin,
thanks for the info!
Quoted 8 lines
Quoted 2 lines · Iustin Pop
No, XFS will not lose any data that the application has committed to the disk.
OK, but just to make sure: The following FAQ entry refers only to *newly* created files - right?
No - it can affect an file where the write is extending the file. The NULLs problem should be very rare now - we've fixed the common causes of this problem....
Quoted 5 lines
Quoted 2 lines
Improperly written applications and/or improperly configured systems might have issues with recently written files losing data.
Again, just to make sure that I understood you correctly: Could you name an example?
I suggest watching the video of this talk from the 2007 linux.conf.au:
http://lca2007.linux.org.au/talk/278.html
It's all about how applications do stupid stuff that can lose data.
Cheers,
Dave.
Signature · 3 lines
Dave Chinner Principal Engineer SGI Australian Software Group