Skip to contentSkip to conversation

XFS corruption with rsync

DISCUSSION Emmanuel Florac 2 participants 3 messages Sep 19, 2012

lore
Share & tools
lore
#0 lore raw

I'"ve seen this error showing up on the list previously: while rsync'ing a large volume (66 TB copied to a 114 TB filesystem), I've got this error showing up:

Pid: 2069, comm: rsync Not tainted 3.2.28-storiq64-opteron #1

Call Trace:
 [<ffffffff811e037f>] ? xfs_alloc_fixup_trees+0x22f/0x370
 [<ffffffff811f6ee3>] ? xfs_btree_get_rec+0x53/0x90
 [<ffffffff811e32ee>] ? xfs_alloc_ag_vextent_near+0x83e/0xbe0
 [<ffffffff811e3b4b>] ? xfs_alloc_fix_freelist+0x3cb/0x460
 [<ffffffff811e3755>] ? xfs_alloc_ag_vextent+0xc5/0xf0
 [<ffffffff811e40ba>] ? xfs_alloc_vextent+0x3aa/0x5f0
 [<ffffffff81222d4a>] ? xfs_trans_read_buf+0x22a/0x420
 [<ffffffff8120a857>] ? xfs_ialloc_ag_alloc+0x3c7/0x4d0
 [<ffffffff8120aa2b>] ? xfs_dialloc+0xcb/0x870
 [<ffffffff81077ffd>] ? up+0xd/0x40
 [<ffffffff8120d705>] ? xfs_ialloc+0x75/0x760
 [<ffffffff811df986>] ? kmem_zone_alloc+0x66/0xe0
 [<ffffffff811dbc23>] ? xfs_dir_ialloc+0x93/0x330
 [<ffffffff812198bd>] ? xfs_trans_reserve+0x9d/0x210
 [<ffffffff811df705>] ? xfs_create+0x455/0x670
 [<ffffffff811d4286>] ? xfs_vn_mknod+0xa6/0x1b0
 [<ffffffff8110f129>] ? vfs_create+0x89/0xc0
 [<ffffffff811107a0>] ? do_last+0x550/0x750
 [<ffffffff811112bb>] ? path_openat+0xdb/0x3d0
 [<ffffffff81103617>] ? do_sync_write+0xc7/0x100
 [<ffffffff810e43e6>] ? handle_mm_fault+0x226/0x270
 [<ffffffff811116dd>] ? do_filp_open+0x4d/0xc0
 [<ffffffff8111d923>] ? alloc_fd+0x43/0x130
 [<ffffffff81102fb5>] ? do_sys_open+0x105/0x1e0
 [<ffffffff81424652>] ? system_call_fastpath+0x16/0x1b

Rsync logs says that "structure needs cleaning". After running xfs_repair, there's a few files making reference to a free block.

This machine has had serious filesystem failure previously, but the RAID array and filesystem have been rebuild from the ground up, so it should be OK; however I'd like to be really sure there's no hidden hardware glitch waiting to bite us again...

Signature · 11 lines
------------------------------------------------------------------------
Emmanuel Florac     |   Direction technique
                    |   Intellique
                    |	<[email protected]>
                    |   +33 1 78 94 84 02
------------------------------------------------------------------------

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#1 lore raw
Quoted 4 lines · Emmanuel Florac

I'"ve seen this error showing up on the list previously: while rsync'ing a large volume (66 TB copied to a 114 TB filesystem), I've got this error showing up:

I think you trimmed some relevant lines before these, what actual error did it hit, there's probably a file & line number?

Quoted 29 lines

Pid: 2069, comm: rsync Not tainted 3.2.28-storiq64-opteron #1

Call Trace:
 [<ffffffff811e037f>] ? xfs_alloc_fixup_trees+0x22f/0x370
 [<ffffffff811f6ee3>] ? xfs_btree_get_rec+0x53/0x90
 [<ffffffff811e32ee>] ? xfs_alloc_ag_vextent_near+0x83e/0xbe0
 [<ffffffff811e3b4b>] ? xfs_alloc_fix_freelist+0x3cb/0x460
 [<ffffffff811e3755>] ? xfs_alloc_ag_vextent+0xc5/0xf0
 [<ffffffff811e40ba>] ? xfs_alloc_vextent+0x3aa/0x5f0
 [<ffffffff81222d4a>] ? xfs_trans_read_buf+0x22a/0x420
 [<ffffffff8120a857>] ? xfs_ialloc_ag_alloc+0x3c7/0x4d0
 [<ffffffff8120aa2b>] ? xfs_dialloc+0xcb/0x870
 [<ffffffff81077ffd>] ? up+0xd/0x40
 [<ffffffff8120d705>] ? xfs_ialloc+0x75/0x760
 [<ffffffff811df986>] ? kmem_zone_alloc+0x66/0xe0
 [<ffffffff811dbc23>] ? xfs_dir_ialloc+0x93/0x330
 [<ffffffff812198bd>] ? xfs_trans_reserve+0x9d/0x210
 [<ffffffff811df705>] ? xfs_create+0x455/0x670
 [<ffffffff811d4286>] ? xfs_vn_mknod+0xa6/0x1b0
 [<ffffffff8110f129>] ? vfs_create+0x89/0xc0
 [<ffffffff811107a0>] ? do_last+0x550/0x750
 [<ffffffff811112bb>] ? path_openat+0xdb/0x3d0
 [<ffffffff81103617>] ? do_sync_write+0xc7/0x100
 [<ffffffff810e43e6>] ? handle_mm_fault+0x226/0x270
 [<ffffffff811116dd>] ? do_filp_open+0x4d/0xc0
 [<ffffffff8111d923>] ? alloc_fd+0x43/0x130
 [<ffffffff81102fb5>] ? do_sys_open+0x105/0x1e0
 [<ffffffff81424652>] ? system_call_fastpath+0x16/0x1b
Quoted 2 lines

Rsync logs says that "structure needs cleaning". After running xfs_repair, there's a few files making reference to a free block.

Including the xfs_repair output would be helpful too.

Quoted 4 lines

This machine has had serious filesystem failure previously, but the RAID array and filesystem have been rebuild from the ground up, so it should be OK; however I'd like to be really sure there's no hidden hardware glitch waiting to bite us again...

-Eric

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#2 lore raw

Le Wed, 19 Sep 2012 12:39:53 -0500 vous écriviez:

Quoted 8 lines · Emmanuel Florac
Quoted 4 lines

I'"ve seen this error showing up on the list previously: while rsync'ing a large volume (66 TB copied to a 114 TB filesystem), I've got this error showing up:

I think you trimmed some relevant lines before these, what actual error did it hit, there's probably a file & line number?

Nothing else in dmesg but this error repeated many times. I'll check through the /var/log/messages if there is more information.

Quoted 4 lines
Quoted 2 lines

Rsync logs says that "structure needs cleaning". After running xfs_repair, there's a few files making reference to a free block.

Including the xfs_repair output would be helpful too.

Unfortunately I forgot that all of xfs_repair output is to STDERR, so I lost it... It said a lot of lines like "data fork in inode XXX claims free block YYY". However no data was dropped to lost+found, and apparently all is OK.

Signature · 11 lines
------------------------------------------------------------------------
Emmanuel Florac     |   Direction technique
                    |   Intellique
                    |	<[email protected]>
                    |   +33 1 78 94 84 02
------------------------------------------------------------------------

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs