XFS filesystem corruption: 2.6.0. Massive failure. With raid5
This has happened twice now. Massive XFS file system corruption. The system is running on a 3ware card in Raid5 config. / is XFS. Cannot mount:
XFS: log has mismatchd uuid - can't recover
XFS: failed to find log head
XFS: log mount/recovery/failed
...
xfs_repair lets me know a lot of stuff, and:
* ERROR: mismathced uuid in log
* SB: some long number
* log: a slightly different long number
It doesn't work.
xfs_logprint -t /dev/sda4 produces a lot of illegal type errors and ends up with Segmentation fault (uh oh).
I am willing to give an XFS developer access to the machine to poke around (and fix it!!!) It's just my desktop at home, not a big deal. But I have a ton of large files i'd like to recover. :)
I could fix it by forcing hte logs clean, that is what I did the first time this happened. However, I lost a lot of files last time, and this shouldn't happen. So here it is for you guys. I am hanging out in #xfs on irc.freenode.net if anybody wants to check it out.
Jerry Haltom
Feedback Plus, Inc.
In article <1072425031.737.9.camel@osaka>,
Quoted 11 lines · Jerry Haltom
This has happened twice now. Massive XFS file system corruption. The system is running on a 3ware card in Raid5 config. / is XFS. Cannot mount:
XFS: log has mismatchd uuid - can't recover
XFS: failed to find log head
XFS: log mount/recovery/failed
...
xfs_repairlets me know a lot of stuff, and:
Quoted 8 lines
* ERROR: mismathced uuid in log
* SB: some long number
* log: a slightly different long numberIt doesn't work.
xfs_logprint-t/dev/sda4produces a lot of illegal type errors and ends up with Segmentation fault (uh oh).
I'm testing the same setup here right now, and you got me a bit worried.. If you know of a way to reproduce the bug, I can see if I can replicate it on my hardware.
Mike.
Signature · 1 line
When life hands you lemons, grab the salt and pass the tequila.
Quoted 7 lines · Jerry Haltom
This has happened twice now. Massive XFS file system corruption. The system is running on a 3ware card in Raid5 config. / is XFS. Cannot mount:
XFS: log has mismatchd uuid - can't recover
XFS: failed to find log head
XFS: log mount/recovery/failed
Either a corrupt log, or a misstated external log, it seems.
Ah, on irc you said only the first part of the uuid was off, that's a bit odd. Knowing the actual numbers would be very helpful. Perhaps the raid swizzled some bits?
Quoted 5 lines
xfs_repairlets me know a lot of stuff, and:* ERROR: mismathced uuid in log
* SB: some long number
* log: a slightly different long number
Those numbers may be important, without the real output this isn't very helpful.
Quoted 4 lines
It doesn't work.
xfs_logprint-t/dev/sda4produces a lot of illegal type errors and ends up with Segmentation fault (uh oh).
Hm, so the log was in very bad shape. I don't know how it got there; it may or may not be xfs's fault, but it shouldn't segfault. If you have the core file maybe we can take a look.
Quoted 4 lines
I could fix it by forcing hte logs clean, that is what I did the first time this happened. However, I lost a lot of files last time, and this shouldn't happen. So here it is for you guys. I am hanging out in #xfs on irc.freenode.net if anybody wants to check it out.
Sounds like you've already repaired it and whacked the log, so no use now. It would also be interesting to know if any I/O or other errors occurred in the system before the problems. You might also run without the nvidia driver* for a while and see if things go better.
-Eric
*info gleaned from irc