Skip to contentSkip to conversation

How to recover data from an xfs partition

DISCUSSION Niccolò Belli 4 participants 9 messages May 12, 2012 – May 21, 2012

lore
Share & tools
lore
#0 lore raw

Hi,
I had a (software) problem with my xen server ended ruining my domU partitions. I already recovered all my ext4 partitions using fsck.ext4, but I can't recover the only xfs partition I have :(

When I try to mount I get:

root@firewall:~# mount -o loop,offset=$((97280*1024)) -t xfs /srv/xen/webserver.img /mnt
mount: Structure needs cleaning

While when trying to fsck I get:

root@firewall:~# xfs_check /dev/loop51
xfs_check: /dev/loop51 is not a valid XFS filesystem (unexpected SB magic number 0xeb639010)
xfs_check: size check failed
xfs_check: WARNING - filesystem uses v1 dirs,limited functionality provided.
cache_node_purge: refcount was 1, not zero (node=0x11f0820)
xfs_check: cannot read root inode (22)
cache_node_purge: refcount was 1, not zero (node=0x11f0970)
xfs_check: cannot read realtime bitmap inode (22)
xfs_check: size check failed
xfs_check: WARNING - filesystem uses v1 dirs,limited functionality provided.
bad superblock magic number eb639010, giving up

xfs_check:

root@firewall:~# xfs_check /dev/loop51
xfs_check: /dev/loop51 is not a valid XFS filesystem (unexpected SB magic number 0xeb639010)
xfs_check: size check failed
xfs_check: WARNING - filesystem uses v1 dirs,limited functionality provided.
cache_node_purge: refcount was 1, not zero (node=0x22f1820)
xfs_check: cannot read root inode (22)
cache_node_purge: refcount was 1, not zero (node=0x22f1970)
xfs_check: cannot read realtime bitmap inode (22)
xfs_check: size check failed
xfs_check: WARNING - filesystem uses v1 dirs,limited functionality provided.
bad superblock magic number eb639010, giving up

xfs_repair:

root@firewall:~# xfs_repair /dev/loop51
Phase 1 - find and verify superblock...
bad primary superblock - bad magic number !!!

attempting to find secondary superblock... ................................................................................found candidate secondary superblock... unable to verify superblock, continuing...
.............
[...]
....................................Sorry, could not find valid secondary superblock
Exiting now.

How can I restore my data? I cannot believe I had no problems restoring *15* ext4 partitions while I can't restore the only xfs partition I have :(

Hope you can help,
Niccolò

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#1 lore raw

Niccolo,

Quoted 15 lines · Niccolò Belli
root@firewall:~# xfs_repair /dev/loop51
Phase 1 - find and verify superblock...
bad primary superblock - bad magic number !!!

attempting to find secondary superblock... ................................................................................found candidate secondary superblock...
unable to verify superblock, continuing...
.............
[...]
....................................Sorry, could not find valid secondary superblock
Exiting now.

Quoted 1 line

How can I restore my data?

I recommend you give it a shot with the latest xfsprogs. However, it may be that your filesystem has been corrupted beyond repair.

-Ben

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#2 lore raw

Hi,

Il 18/05/2012 16:31, Ben Myers ha scritto:

Quoted 2 lines

I recommend you give it a shot with the latest xfsprogs. However, it may be that your filesystem has been corrupted beyond repair.

I succeeded recovering something with xfsprogs 3.1.8, but no more than 1GB of data (~5%).

Fortunately I managed to save some precious config files, considering I had a 6 months old backup the disaster is a bit less catastrophic.

I had a bad time replacing corrupted data from the other ten ext4 virtual machines, but they didn't give me too much troubles compared to xfs. How ironic that the data loss happened during the installation of the backup system...

To be honest I don't know if I will use xfs anymore, unfortunately I need per-directory quota and it is the only viable solution.

Niccolò

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#3 lore raw
Quoted 6 lines · Niccolò Belli

Hi,

Il 18/05/2012 16:31, Ben Myers ha scritto:

Quoted 2 lines

I recommend you give it a shot with the latest xfsprogs. However, it may be that your filesystem has been corrupted beyond repair.

Quoted 7 lines

I succeeded recovering something with xfsprogs 3.1.8, but no more than 1GB of data (~5%).

Fortunately I managed to save some precious config files, considering I had a 6 months old backup the disaster is a bit less catastrophic.

I had a bad time replacing corrupted data from the other ten ext4 virtual machines, but they didn't give me too much troubles compared to xfs. How ironic that the data loss happened during the installation of the backup system...

To be honest I don't know if I will use xfs anymore, unfortunately I need per-directory quota and it is the only viable solution.

You're welcome to follow your heart on choice of filesystem of course, but without knowing what happened to your storage, it's a little bit of a stretch to blame xfs.

Just based on:

Quoted 1 line

xfs_check: /dev/loop51 is not a valid XFS filesystem (unexpected SB magic number 0xeb639010)

it appears that something scribbled all over your storage (i.e. the file backing it - what was that stored on?) What it was that did the scribbling, and how much it scribbled is unknown, and it may or may not have been more significant than whatever hit your ext4 partitions.

I do not think you encountered an xfs bug; something else severely damaged your block device.

-Eric

Quoted 7 lines

Niccolò

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#4 lore raw

Il 18/05/2012 19:46, Eric Sandeen ha scritto:

Quoted 1 line

the file backing it - what was that stored on?

It was a raw image created with dd and stored on an SSD (ext4). Some of the virtual machines were on traditional hard disks instead (ext4).

Only the running virtual machines got damaged, the main filesystems are perfect.

Quoted 1 line

What it was that did the scribbling

Rebooting. A simple, stupid reboot after upgrading Xen. While rebooting the system I got some kind of kernel panic and I had to use magic sysrq keys (remount read-only, sync, reboot).

Niccolò

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#5 lore raw
Quoted 7 lines · Niccolò Belli

Il 18/05/2012 19:46, Eric Sandeen ha scritto:

Quoted 1 line

the file backing it - what was that stored on?

It was a raw image created with dd and stored on an SSD (ext4). Some of the virtual machines were on traditional hard disks instead (ext4).

Quoted 8 lines

Only the running virtual machines got damaged, the main filesystems are perfect.

Quoted 1 line

What it was that did the scribbling

Rebooting. A simple, stupid reboot after upgrading Xen. While rebooting the system I got some kind of kernel panic and I had to use magic sysrq keys (remount read-only, sync, reboot).

Well - somehow the first sector of the xfs image got junk written into it... hard to say how that happened, but the magic number lives in the first sector of the xfs image hosted on ext4, and it was junk post-reboot.

I have never seen xfs corrupt _itself_ this way. TBH I'd be as likely to blame the SSD, as anything. But you also have to have all of the data integrity bits set up properly all along the virt stack to survive a panic or power loss w/o corruption after the fact.

-Eric

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#6 lore raw

Il 19/05/2012 00:47, Eric Sandeen ha scritto:

Quoted 2 lines

TBH I'd be as likely to
blame the SSD, as anything.

I didn't even need to fsck the SSD's partition... Something gone mad in the Xen code and it corrupted my data, that was the problem.
I managed to rescue the vast majority of the data in my ext4 partitions, but not in the xfs one. Who should I blame for this? xfs? the fate? I really don't know it, I'm just reporting what happened compared to another fs but I cannot guarantee the bug threated all the partitions the same way :)

Niccolò

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#7 lore raw
In reply to Niccolò Belli

Am Freitag, 18. Mai 2012, 22:57:15 schrieb Niccolò Belli:

Quoted 2 lines

I had to use magic sysrq
keys (remount read-only, sync, reboot).

Shouldn't that be sync, remount r/o, reboot instead? If you remount r/o, the cache in the system wouldn't be written anymore.

Signature · 6 lines
mit freundlichen Grüssen,
Michael Monnerie, Ing. BSc

it-management Internet Services: Protéger
http://proteger.at [gesprochen: Prot-e-schee]
Tel: +43 660 / 415 6531

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#8 lore raw

Il 21/05/2012 09:44, Michael Monnerie ha scritto:

Quoted 2 lines

Shouldn't that be sync, remount r/o, reboot instead? If you remount r/o, the cache in the system wouldn't be written anymore.

Yes, that's what I did (S,U,B).

Niccolò

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs