Skip to contentSkip to conversation

fs corruption

DISCUSSION stress_buster 3 participants 5 messages Apr 12, 2011

lore
Share & tools
lore

stress_buster

author
#0 lore raw

My dmesg output shows the below trace. It repeats over and over again.

XFS internal error XFS_WANT_CORRUPTED_GOTO at line 1545 of file fs/xfs/xfs_alloc.c. Caller 0xffffffff881a8961

Call Trace:
 [<ffffffff881a6e27>] :xfs:xfs_free_ag_extent+0x19e/0x67e
 [<ffffffff881a8961>] :xfs:xfs_free_extent+0xa9/0xc9
 [<ffffffff881d96cf>] :xfs:xlog_recover_process_efi+0x112/0x16c
 [<ffffffff881f31b4>] :xfs:xfs_fs_fill_super+0x0/0x3e4
 [<ffffffff881da8c2>] :xfs:xlog_recover_process_efis+0x4f/0x8d
 [<ffffffff881da914>] :xfs:xlog_recover_finish+0x14/0xad
 [<ffffffff881f31b4>] :xfs:xfs_fs_fill_super+0x0/0x3e4
 [<ffffffff881df420>] :xfs:xfs_mountfs+0x498/0x5e2
 [<ffffffff881dfb42>] :xfs:xfs_mru_cache_create+0x113/0x143
 [<ffffffff881f33b7>] :xfs:xfs_fs_fill_super+0x203/0x3e4
 [<ffffffff800e544f>] get_sb_bdev+0x10a/0x16c
 [<ffffffff800e4dec>] vfs_kern_mount+0x93/0x11a
 [<ffffffff800e4eb5>] do_kern_mount+0x36/0x4d
 [<ffffffff800ef2ed>] do_mount+0x6a9/0x719
 [<ffffffff80008d84>] __handle_mm_fault+0x5f2/0xfaa
 [<ffffffff80022127>] __up_read+0x19/0x7f
 [<ffffffff80067b88>] do_page_fault+0x4fe/0x874
 [<ffffffff8012c580>] inode_doinit_with_dentry+0x86/0x47c
 [<ffffffff800cd378>] zone_statistics+0x3e/0x6d
 [<ffffffff8000f2ff>] __alloc_pages+0x78/0x308
 [<ffffffff8004c9fd>] sys_mount+0x8a/0xcd
 [<ffffffff8005e116>] system_call+0x7e/0x83

Failed to recover EFIs on filesystem: cciss/c0d0
XFS: log mount finish failed

Can someone shed some light on what is happening here?

Also what the next steps I need to take to repair the fs? (assuming my xfs fs is corrupted)
Will running xfs_repair be good enough in this case?

Thanks in advance

Signature · 7 lines
View this message in context: http://old.nabble.com/fs-corruption-tp31377534p31377534.html
Sent from the Xfs - General mailing list archive at Nabble.com.

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#1 lore raw
Quoted 30 lines · stress_buster

My dmesg output shows the below trace. It repeats over and over again.

XFS internal error XFS_WANT_CORRUPTED_GOTO at line 1545 of file fs/xfs/xfs_alloc.c. Caller 0xffffffff881a8961

Call Trace:
 [<ffffffff881a6e27>] :xfs:xfs_free_ag_extent+0x19e/0x67e
 [<ffffffff881a8961>] :xfs:xfs_free_extent+0xa9/0xc9
 [<ffffffff881d96cf>] :xfs:xlog_recover_process_efi+0x112/0x16c
 [<ffffffff881f31b4>] :xfs:xfs_fs_fill_super+0x0/0x3e4
 [<ffffffff881da8c2>] :xfs:xlog_recover_process_efis+0x4f/0x8d
 [<ffffffff881da914>] :xfs:xlog_recover_finish+0x14/0xad
 [<ffffffff881f31b4>] :xfs:xfs_fs_fill_super+0x0/0x3e4
 [<ffffffff881df420>] :xfs:xfs_mountfs+0x498/0x5e2
 [<ffffffff881dfb42>] :xfs:xfs_mru_cache_create+0x113/0x143
 [<ffffffff881f33b7>] :xfs:xfs_fs_fill_super+0x203/0x3e4
 [<ffffffff800e544f>] get_sb_bdev+0x10a/0x16c
 [<ffffffff800e4dec>] vfs_kern_mount+0x93/0x11a
 [<ffffffff800e4eb5>] do_kern_mount+0x36/0x4d
 [<ffffffff800ef2ed>] do_mount+0x6a9/0x719
 [<ffffffff80008d84>] __handle_mm_fault+0x5f2/0xfaa
 [<ffffffff80022127>] __up_read+0x19/0x7f
 [<ffffffff80067b88>] do_page_fault+0x4fe/0x874
 [<ffffffff8012c580>] inode_doinit_with_dentry+0x86/0x47c
 [<ffffffff800cd378>] zone_statistics+0x3e/0x6d
 [<ffffffff8000f2ff>] __alloc_pages+0x78/0x308
 [<ffffffff8004c9fd>] sys_mount+0x8a/0xcd
 [<ffffffff8005e116>] system_call+0x7e/0x83
Quoted 4 lines

Failed to recover EFIs on filesystem: cciss/c0d0
XFS: log mount finish failed

Can someone shed some light on what is happening here?

You have a corrupted free space btree. How it occurred, I have no idea.

Quoted 3 lines

Also what the next steps I need to take to repair the fs? (assuming my xfs fs is corrupted)
Will running xfs_repair be good enough in this case?

That's all you can do. If it's really important, and you don't have a backup, I'd suggest mounting with "-o ro,norecovery" and taking a backup first....

Cheers,

Dave.

Signature · 7 lines
Dave Chinner
[email protected]

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
author
#2 lore raw

You have a corrupted free space btree.

Err... apologies for my ignorance, but what is a free space btree?

I had serial trace from raid controller which i just checked and it logged some 'Loose cabling', but this was months back.....
not sure whether that can be the cause of this.. strange if that is the case since it's been a long time

________________________________
From: Dave Chinner <[email protected]>
To: stress_buster <[email protected]>
Cc: [email protected]
Sent: Tue, April 12, 2011 3:19:17 PM
Subject: Re: fs corruption

Quoted 30 lines · stress_buster

My dmesg output shows the below trace. It repeats over and over again.

XFS internal error XFS_WANT_CORRUPTED_GOTO at line 1545 of file fs/xfs/xfs_alloc.c. Caller 0xffffffff881a8961

Call Trace:
 [<ffffffff881a6e27>] :xfs:xfs_free_ag_extent+0x19e/0x67e
 [<ffffffff881a8961>] :xfs:xfs_free_extent+0xa9/0xc9
 [<ffffffff881d96cf>] :xfs:xlog_recover_process_efi+0x112/0x16c
 [<ffffffff881f31b4>] :xfs:xfs_fs_fill_super+0x0/0x3e4
 [<ffffffff881da8c2>] :xfs:xlog_recover_process_efis+0x4f/0x8d
 [<ffffffff881da914>] :xfs:xlog_recover_finish+0x14/0xad
 [<ffffffff881f31b4>] :xfs:xfs_fs_fill_super+0x0/0x3e4
 [<ffffffff881df420>] :xfs:xfs_mountfs+0x498/0x5e2
 [<ffffffff881dfb42>] :xfs:xfs_mru_cache_create+0x113/0x143
 [<ffffffff881f33b7>] :xfs:xfs_fs_fill_super+0x203/0x3e4
 [<ffffffff800e544f>] get_sb_bdev+0x10a/0x16c
 [<ffffffff800e4dec>] vfs_kern_mount+0x93/0x11a
 [<ffffffff800e4eb5>] do_kern_mount+0x36/0x4d
 [<ffffffff800ef2ed>] do_mount+0x6a9/0x719
 [<ffffffff80008d84>] __handle_mm_fault+0x5f2/0xfaa
 [<ffffffff80022127>] __up_read+0x19/0x7f
 [<ffffffff80067b88>] do_page_fault+0x4fe/0x874
 [<ffffffff8012c580>] inode_doinit_with_dentry+0x86/0x47c
 [<ffffffff800cd378>] zone_statistics+0x3e/0x6d
 [<ffffffff8000f2ff>] __alloc_pages+0x78/0x308
 [<ffffffff8004c9fd>] sys_mount+0x8a/0xcd
 [<ffffffff8005e116>] system_call+0x7e/0x83
Quoted 4 lines

Failed to recover EFIs on filesystem: cciss/c0d0
XFS: log mount finish failed

Can someone shed some light on what is happening here?

You have a corrupted free space btree. How it occurred, I have no idea.

Quoted 3 lines

Also what the next steps I need to take to repair the fs? (assuming my xfs fs is corrupted)
Will running xfs_repair be good enough in this case?

That's all you can do. If it's really important, and you don't have a backup, I'd suggest mounting with "-o ro,norecovery" and taking a backup first....

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#3 lore raw
Quoted 3 lines · Leo Davis

You have a corrupted free space btree.

Err... apologies for my ignorance, but what is a free space btree?

A tree that indexes the free space in the filesystem. Every time you write a file or remove a file you are allocating or freeing space, and these tree keep track of that free space.

If you want to know - at a high level - how XFS is structured (good for understanding what a free space tree is), read this paper:

http://oss.sgi.com/projects/xfs/papers/xfs_usenix/index.html

It's from 1996, but still correct on all the major structural details.

Quoted 4 lines

I had serial trace from raid controller which i just checked and it logged some 'Loose cabling', but this was months back..... not sure whether that can be the cause of this.. strange if that is the case since it's been a long time

it's possible that it took a couple of months to trip over a random metadata corruption. I've seen that before in directory trees and inode clusters where corruption is not detected until next time they are read from disk....

Cheers,

Dave.

Signature · 7 lines
Dave Chinner
[email protected]

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#4 lore raw

Le Tue, 12 Apr 2011 21:05:32 +1000
Dave Chinner <[email protected]> écrivait:

Quoted 5 lines

it's possible that it took a couple of months to trip over a random metadata corruption. I've seen that before in directory trees and inode clusters where corruption is not detected until next time they are read from disk....

That's why background scrubbing of RAID arrays is generally a good habit to contract :)

Signature · 11 lines
------------------------------------------------------------------------
Emmanuel Florac     |   Direction technique
                    |   Intellique
                    |	<[email protected]>
                    |   +33 1 78 94 84 02
------------------------------------------------------------------------

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs