Skip to contentSkip to conversation

any way to work backwards from xfs_inode_t to a filename?

DISCUSSION Chris Friesen 4 participants 6 messages May 15, 2013 – May 16, 2013

lore
Share & tools
lore
#0 lore raw

Hi all,

As I mentioned before I'm digging into some intermittent xfs corruption issues on an embedded system running a 2.6.27-based kernel.

In one case I've hit XFS_WANT_CORRUPTED_GOTO in xfs_free_ag_extent(). In this particular case the call chain was

xfs_free_ag_extent+0x55d/0x6e0
xfs_free_extent+0xac/0xd0
xfs_bmap_finish+0x156/0x1a0
xfs_itruncate_finish+0x137/0x340
xfs_inactive+0x386/0x4b0
xfs_fs_clear_inode+0xc6/0x120
clear_inode+0x58/0x110
generic_delete_inode+0xff/0x110
generic_drop_inode+0x16d/0x190
iput+0x5d/0x70
do_unlinkat+0x147/0x280
sys_unlinkat+0x1d/0x40

I have two questions:

1) do_unlinkat() has the filesystem path, but iput() returns void. Is there any way for me to add instrumentation to xfs_inactive() to work backwards from the xfs_inode_t pointer to print out a path to the file being deleted?

2) If this sort of corruption happens again, what's my best bet for figuring out the source of the problem? This is an embedded system, so I need to be able to automate any data gathering (after which we'll run xfs_repair so we can return to service--any manual analysis will need to happen later based on stored data).

Thanks,
Chris

Signature · 9 lines
Chris Friesen
Software Designer
www.genband.com
office:+1.343.883.2717

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#1 lore raw

Hey Chris,

Quoted 20 lines · Chris Friesen

As I mentioned before I'm digging into some intermittent xfs corruption issues on an embedded system running a 2.6.27-based kernel.

In one case I've hit XFS_WANT_CORRUPTED_GOTO in xfs_free_ag_extent(). In this particular case the call chain was

xfs_free_ag_extent+0x55d/0x6e0
xfs_free_extent+0xac/0xd0
xfs_bmap_finish+0x156/0x1a0
xfs_itruncate_finish+0x137/0x340
xfs_inactive+0x386/0x4b0
xfs_fs_clear_inode+0xc6/0x120
clear_inode+0x58/0x110
generic_delete_inode+0xff/0x110
generic_drop_inode+0x16d/0x190
iput+0x5d/0x70
do_unlinkat+0x147/0x280
sys_unlinkat+0x1d/0x40
Quoted 6 lines

I have two questions:

1) do_unlinkat() has the filesystem path, but iput() returns void. Is there any way for me to add instrumentation to xfs_inactive() to work backwards from the xfs_inode_t pointer to print out a path to the file being deleted?

Use VFS_I to get to a 'struct inode' and from there you can look at the dentries on i_dentry list and traverse back through the path by looking at d_parent. Might be easier to just print the path in do_unlinkat?

Quoted 5 lines

2) If this sort of corruption happens again, what's my best bet for figuring out the source of the problem? This is an embedded system, so I need to be able to automate any data gathering (after which we'll run xfs_repair so we can return to service--any manual analysis will need to happen later based on stored data).

Can you grab a metadump before you repair the filesystem?

Thanks,
Ben

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#2 lore raw
Quoted 25 lines · Ben Myers

Hey Chris,

Quoted 22 lines · Chris Friesen

As I mentioned before I'm digging into some intermittent xfs corruption issues on an embedded system running a 2.6.27-based kernel.

In one case I've hit XFS_WANT_CORRUPTED_GOTO in xfs_free_ag_extent(). In this particular case the call chain was

xfs_free_ag_extent+0x55d/0x6e0
xfs_free_extent+0xac/0xd0
xfs_bmap_finish+0x156/0x1a0
xfs_itruncate_finish+0x137/0x340
xfs_inactive+0x386/0x4b0
xfs_fs_clear_inode+0xc6/0x120
clear_inode+0x58/0x110
generic_delete_inode+0xff/0x110
generic_drop_inode+0x16d/0x190
iput+0x5d/0x70
do_unlinkat+0x147/0x280
sys_unlinkat+0x1d/0x40

I have two questions:

Quoted 8 lines
Quoted 4 lines

1) do_unlinkat() has the filesystem path, but iput() returns void. Is there any way for me to add instrumentation to xfs_inactive() to work backwards from the xfs_inode_t pointer to print out a path to the file being deleted?

Use VFS_I to get to a 'struct inode' and from there you can look at the dentries on i_dentry list and traverse back through the path by looking at d_parent. Might be easier to just print the path in do_unlinkat?

or just print out the inode nr (i_ino) and do a find -inum after the fact. :)

-Eric </lazy>

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#5 lore raw
In reply to Chris Friesen
Quoted 23 lines · Chris Friesen

Hi all,

As I mentioned before I'm digging into some intermittent xfs corruption issues on an embedded system running a 2.6.27-based kernel.

In one case I've hit XFS_WANT_CORRUPTED_GOTO in xfs_free_ag_extent(). In this particular case the call chain was

xfs_free_ag_extent+0x55d/0x6e0
xfs_free_extent+0xac/0xd0
xfs_bmap_finish+0x156/0x1a0
xfs_itruncate_finish+0x137/0x340
xfs_inactive+0x386/0x4b0
xfs_fs_clear_inode+0xc6/0x120
clear_inode+0x58/0x110
generic_delete_inode+0xff/0x110
generic_drop_inode+0x16d/0x190
iput+0x5d/0x70
do_unlinkat+0x147/0x280
sys_unlinkat+0x1d/0x40
Quoted 6 lines

I have two questions:

1) do_unlinkat() has the filesystem path, but iput() returns void. Is there any way for me to add instrumentation to xfs_inactive() to work backwards from the xfs_inode_t pointer to print out a path to the file being deleted?

You can't from there - the inode is being cycled out of cache, which means it does not have a dentry attached anymore and so no path can be found from there. The best you can do is print out the inode number, and then use something like "find -inum <ino>" or the xfs_db equivalent (man xfs_ncheck).

Quoted 5 lines

2) If this sort of corruption happens again, what's my best bet for figuring out the source of the problem? This is an embedded system, so I need to be able to automate any data gathering (after which we'll run xfs_repair so we can return to service--any manual analysis will need to happen later based on stored data).

Forensic analsys of the filesystem and clear information about what the state of the filesystem was in memory whenteh problem was tripped over is generally the only way to get to the bottom of these problems. i.e. kernel crash dumps triggered by setting the panic mask appropriately, and then taking an xfs_metadump of the filesystem before the filesystem is remounted, and then again after the filesystem is mounted and log recovery has run....

Cheers,

Dave.

Signature · 7 lines
Dave Chinner
[email protected]

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#3 lore raw
In reply to Eric Sandeen

Chris,

Quoted 11 lines · Eric Sandeen
Quoted 8 lines · Ben Myers
Quoted 4 lines

1) do_unlinkat() has the filesystem path, but iput() returns void. Is there any way for me to add instrumentation to xfs_inactive() to work backwards from the xfs_inode_t pointer to print out a path to the file being deleted?

Use VFS_I to get to a 'struct inode' and from there you can look at the dentries on i_dentry list and traverse back through the path by looking at d_parent. Might be easier to just print the path in do_unlinkat?

or just print out the inode nr (i_ino) and do a find -inum after the fact. :)

+1

Dave pointed out that you might not be able to get to the inode at this point in the lifecycle. Apologies if I posted misinformation. ;)

Regards,
Ben

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#4 lore raw
Quoted 13 lines · Ben Myers

Chris,

Quoted 10 lines · Eric Sandeen
Quoted 8 lines · Ben Myers
Quoted 4 lines

1) do_unlinkat() has the filesystem path, but iput() returns void. Is there any way for me to add instrumentation to xfs_inactive() to work backwards from the xfs_inode_t pointer to print out a path to the file being deleted?

Use VFS_I to get to a 'struct inode' and from there you can look at the dentries on i_dentry list and traverse back through the path by looking at d_parent. Might be easier to just print the path in do_unlinkat?

Quoted 6 lines
Quoted 1 line

or just print out the inode nr (i_ino) and do a find -inum after the fact. :)

+1

Dave pointed out that you might not be able to get to the inode at this point in the lifecycle. Apologies if I posted misinformation. ;)

Thanks guys,

I'm headed away for a few days so it'll be a while before I can dig into this again, but this should give me a good start.

Chris

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs