xfs: fix incorrect i_nlink caused by inode racing
PATCH 1/1 xfs: fix incorrect i_nlink caused by inode racing
The following error occurred during the fsstress test:
XFS: Assertion failed: VFS_I(ip)->i_nlink >= 2, file: fs/xfs/xfs_inode.c, line: 2925
The problem was that inode race condition causes incorrect i_nlink to be written to disk, and then it is read into memory. Consider the following call graph, inodes that are marked as both XFS_IFLUSHING and XFS_IRECLAIMABLE, i_nlink will be reset to 1 and then restored to original value in xfs_reinit_inode(). Therefore, the i_nlink of directory on disk may be set to 1.
xfsaild
xfs_inode_item_push
xfs_iflush_cluster
xfs_iflush
xfs_inode_to_disk
xfs_iget
xfs_iget_cache_hit
xfs_iget_recycle
xfs_reinit_inode
inode_init_always
So skip inodes that being flushed and markded as XFS_IRECLAIMABLE, prevent concurrent read and write to inodes.
--- fs/xfs/xfs_icache.c | 5 +++++ 1 file changed, 5 insertions(+)
fs/xfs/xfs_icache.c +5 -0
@@ -514,6 +514,11 @@ xfs_iget_cache_hit( (ip->i_flags & XFS_IRECLAIMABLE)) goto out_skip; +/* Skip inodes that being flushed */ +if ((ip->i_flags &XFS_IFLUSHING ) && + (ip->i_flags &XFS_IRECLAIMABLE )) + goto out_skip; +/* The inode fits the selection criteria; process it. */ if (ip->i_flags &XFS_IRECLAIMABLE ) {/* Drops i_flags_lock and RCU read lock. */
Signature · 1 line
2.31.1
Quoted 3 lines · Long Li
The following error occurred during the fsstress test:
XFS: Assertion failed: VFS_I(ip)->i_nlink >= 2, file:
fs/xfs/xfs_inode.c, line: 2925
What kernel is this? xfs_inode.c line 2925 is in the middle of xfs_rename and doesn't have any assertions on nlink.
The only assertion on nlink in the entire xfs codebase is in xfs_remove, and that's not what's going on here.
<confused>
--D
Quoted 46 lines
The problem was that inode race condition causes incorrect
i_nlinkto be written to disk, and then it is read into memory. Consider the following call graph, inodes that are marked as bothXFS_IFLUSHINGandXFS_IRECLAIMABLE,i_nlinkwill be reset to 1 and then restored to original value inxfs_reinit_inode(). Therefore, thei_nlinkof directory on disk may be set to 1.xfsaild
xfs_inode_item_push
xfs_iflush_cluster
xfs_iflush
xfs_inode_to_disk
xfs_iget
xfs_iget_cache_hit
xfs_iget_recycle
xfs_reinit_inode
inode_init_alwaysSo skip inodes that being flushed and markded as
XFS_IRECLAIMABLE, prevent concurrent read and write to inodes.Signed-off-by Long Li <[email protected]>--- fs/xfs/xfs_icache.c | 5 +++++ 1 file changed, 5 insertions(+)fs/xfs/xfs_icache.c +5
-0@@ -514,6 +514,11 @@ xfs_iget_cache_hit( (ip->i_flags & XFS_IRECLAIMABLE)) goto out_skip; +/* Skip inodes that being flushed */ +if ((ip->i_flags &XFS_IFLUSHING ) && + (ip->i_flags &XFS_IRECLAIMABLE )) + goto out_skip; +/* The inode fits the selection criteria; process it. */ if (ip->i_flags &XFS_IRECLAIMABLE ) {/* Drops i_flags_lock and RCU read lock. */ Signature · 1 line
2.31.1
Quoted 5 lines · Darrick J. Wong
Quoted 3 lines · Long Li
The following error occurred during the fsstress test:
XFS: Assertion failed: VFS_I(ip)->i_nlink >= 2, file:
fs/xfs/xfs_inode.c, line: 2925
Quoted 7 lines
What kernel is this?
xfs_inode.cline 2925 is in the middle ofxfs_renameand doesn't have any assertions on nlink.The only assertion on nlink in the entire xfs codebase is in
xfs_remove, and that's not what's going on here.<confused>
Sorry for the confusion, I found this issue in Linux 5.10, so the assertion on nlink is in xfs_remove(). I've reproduced it on the mainline kernel, the probability of this problem is very low and it is very difficult to reproduce. The mainline kernel assertion error prints is as follows:
XFS: Assertion failed: VFS_I(ip)->i_nlink >= 2, file: fs/xfs/xfs_inode.c, line: 2452
Thanks,
Long Li
Quoted 49 lines
--D
Quoted 46 lines
The problem was that inode race condition causes incorrect
i_nlinkto be written to disk, and then it is read into memory. Consider the following call graph, inodes that are marked as bothXFS_IFLUSHINGandXFS_IRECLAIMABLE,i_nlinkwill be reset to 1 and then restored to original value inxfs_reinit_inode(). Therefore, thei_nlinkof directory on disk may be set to 1.xfsaild
xfs_inode_item_push
xfs_iflush_cluster
xfs_iflush
xfs_inode_to_disk
xfs_iget
xfs_iget_cache_hit
xfs_iget_recycle
xfs_reinit_inode
inode_init_alwaysSo skip inodes that being flushed and markded as
XFS_IRECLAIMABLE, prevent concurrent read and write to inodes.Signed-off-by Long Li <[email protected]>--- fs/xfs/xfs_icache.c | 5 +++++ 1 file changed, 5 insertions(+)fs/xfs/xfs_icache.c +5
-0@@ -514,6 +514,11 @@ xfs_iget_cache_hit( (ip->i_flags & XFS_IRECLAIMABLE)) goto out_skip; +/* Skip inodes that being flushed */ +if ((ip->i_flags &XFS_IFLUSHING ) && + (ip->i_flags &XFS_IRECLAIMABLE )) + goto out_skip; +/* The inode fits the selection criteria; process it. */ if (ip->i_flags &XFS_IRECLAIMABLE ) {/* Drops i_flags_lock and RCU read lock. */ Signature · 1 line
2.31.1
Quoted 17 lines · Long Li
The following error occurred during the fsstress test:
XFS: Assertion failed: VFS_I(ip)->i_nlink >= 2, file:
fs/xfs/xfs_inode.c, line: 2925The problem was that inode race condition causes incorrect
i_nlinkto be written to disk, and then it is read into memory. Consider the following call graph, inodes that are marked as bothXFS_IFLUSHINGandXFS_IRECLAIMABLE,i_nlinkwill be reset to 1 and then restored to original value inxfs_reinit_inode(). Therefore, thei_nlinkof directory on disk may be set to 1.xfsaild
xfs_inode_item_push
xfs_iflush_cluster
xfs_iflush
xfs_inode_to_disk
Quoted 8 lines
xfs_iget
xfs_iget_cache_hit
xfs_iget_recycle
xfs_reinit_inode
inode_init_alwaysSo skip inodes that being flushed and markded as
XFS_IRECLAIMABLE, prevent concurrent read and write to inodes.
urk.
xfs_reinit_inode() needs to hold the ILOCK_EXCL as it is changing internal inode state and can race with other RCU protected inode lookups. Have a look at what xfs_iflush_cluster() does - it grabs the ILOCK_SHARED while under rcu + ip->i_flags_lock, and so xfs_iflush/xfs_inode_to_disk() are protected from racing inode updates (during transactions) by that lock.
Hence it looks to me that I_FLUSHING isn't the problem here - it's that we have a transient modified inode state in xfs_reinit_inode() that is externally visisble...
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
Quoted 36 lines · Dave Chinner
Quoted 17 lines · Long Li
The following error occurred during the fsstress test:
XFS: Assertion failed: VFS_I(ip)->i_nlink >= 2, file:
fs/xfs/xfs_inode.c, line: 2925The problem was that inode race condition causes incorrect
i_nlinkto be written to disk, and then it is read into memory. Consider the following call graph, inodes that are marked as bothXFS_IFLUSHINGandXFS_IRECLAIMABLE,i_nlinkwill be reset to 1 and then restored to original value inxfs_reinit_inode(). Therefore, thei_nlinkof directory on disk may be set to 1.xfsaild
xfs_inode_item_push
xfs_iflush_cluster
xfs_iflush
xfs_inode_to_diskQuoted 8 lines
xfs_iget
xfs_iget_cache_hit
xfs_iget_recycle
xfs_reinit_inode
inode_init_alwaysSo skip inodes that being flushed and markded as
XFS_IRECLAIMABLE, prevent concurrent read and write to inodes.urk.
xfs_reinit_inode()needs to hold theILOCK_EXCLas it is changing internal inode state and can race with other RCU protected inode lookups. Have a look at whatxfs_iflush_cluster()does - it grabs theILOCK_SHAREDwhile under rcu +ip->i_flags_lock, and so xfs_iflush/xfs_inode_to_disk() are protected from racing inode updates (during transactions) by that lock.
Quoted 3 lines
Hence it looks to me that
I_FLUSHINGisn't the problem here - it's that we have a transient modified inode state inxfs_reinit_inode()that is externally visisble...
Before xfs_reinit_inode(), XFS_IRECLAIM will be set in ip->i_flags, this looks like can prevent race with other RCU protected inode lookups. Can it be considered that don't modifying the information about the on-disk values in the VFS inode in xfs_reinit_inode()? if so lock can be avoided.
Thanks,
Long Li
Quoted 37 lines · Long Li
Quoted 36 lines · Dave Chinner
Quoted 17 lines · Long Li
The following error occurred during the fsstress test:
XFS: Assertion failed: VFS_I(ip)->i_nlink >= 2, file:
fs/xfs/xfs_inode.c, line: 2925The problem was that inode race condition causes incorrect
i_nlinkto be written to disk, and then it is read into memory. Consider the following call graph, inodes that are marked as bothXFS_IFLUSHINGandXFS_IRECLAIMABLE,i_nlinkwill be reset to 1 and then restored to original value inxfs_reinit_inode(). Therefore, thei_nlinkof directory on disk may be set to 1.xfsaild
xfs_inode_item_push
xfs_iflush_cluster
xfs_iflush
xfs_inode_to_diskQuoted 8 lines
xfs_iget
xfs_iget_cache_hit
xfs_iget_recycle
xfs_reinit_inode
inode_init_alwaysSo skip inodes that being flushed and markded as
XFS_IRECLAIMABLE, prevent concurrent read and write to inodes.urk.
xfs_reinit_inode()needs to hold theILOCK_EXCLas it is changing internal inode state and can race with other RCU protected inode lookups. Have a look at whatxfs_iflush_cluster()does - it grabs theILOCK_SHAREDwhile under rcu +ip->i_flags_lock, and so xfs_iflush/xfs_inode_to_disk() are protected from racing inode updates (during transactions) by that lock.
Quoted 6 lines
Quoted 3 lines
Hence it looks to me that
I_FLUSHINGisn't the problem here - it's that we have a transient modified inode state inxfs_reinit_inode()that is externally visisble...Before
xfs_reinit_inode(),XFS_IRECLAIMwill be set inip->i_flags, this looks like can prevent race with other RCU protected inode lookups.
That only protects against new lookups - it does not protect against the IRECLAIM flag being set *after* the lookup in xfs_iflush_cluster() whilst the inode is being flushed to the cluster buffer. That's why xfs_iflush_cluster() does:
rcu_read_lock() lookup inode spinlock(ip->i_flags_lock); check IRECLAIM|IFLUSHING
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
xfs_ilock_nowait(ip, XFS_ILOCK_SHARED) <<<<<<<<
set IFLUSHING spin_unlock(ip->i_flags_lock) rcu_read_unlock()
At this point, the only lock that is held is XFS_ILOCK_SHARED, and it's the only lock that protects the inode state outside the lookup scope against concurrent changes.
Essentially, xfs_reinit_inode() needs to add a:
xfs_ilock_nowait(ip, XFS_ILOCK_EXCL)
before it set IRECLAIM - if it fails to get the ILOCK_EXCL, then we need to skip the inode, drop out of RCU scope, delay and retry the lookup.
Quoted 2 lines
Can it be considered that don't modifying the information about the on-disk values in the VFS inode in
xfs_reinit_inode()? if so lock can be avoided.
We have to reinit the VFS inode because it has gone through ->destroy_inode and so the state has been trashed. We have to bring it back as an I_NEW inode, which requires reinitialising everything. THe issue is that we store inode state information (like nlink) in the VFS inode instead of the XFS inode portion of the structure (to minimise memory footprint), and that means xfs_reinit_inode() has a transient state where the VFS inode is not correct. We can avoid that simply by holding the XFS_ILOCK_EXCL, guaranteeing nothing in XFS should be trying to read/modify the internal metadata state while we are reinitialising the VFS inode portion of the structure...
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
Quoted 86 lines · Dave Chinner
Quoted 37 lines · Long Li
Quoted 36 lines · Dave Chinner
Quoted 17 lines · Long Li
The following error occurred during the fsstress test:
XFS: Assertion failed: VFS_I(ip)->i_nlink >= 2, file:
fs/xfs/xfs_inode.c, line: 2925The problem was that inode race condition causes incorrect
i_nlinkto be written to disk, and then it is read into memory. Consider the following call graph, inodes that are marked as bothXFS_IFLUSHINGandXFS_IRECLAIMABLE,i_nlinkwill be reset to 1 and then restored to original value inxfs_reinit_inode(). Therefore, thei_nlinkof directory on disk may be set to 1.xfsaild
xfs_inode_item_push
xfs_iflush_cluster
xfs_iflush
xfs_inode_to_diskQuoted 8 lines
xfs_iget
xfs_iget_cache_hit
xfs_iget_recycle
xfs_reinit_inode
inode_init_alwaysSo skip inodes that being flushed and markded as
XFS_IRECLAIMABLE, prevent concurrent read and write to inodes.urk.
xfs_reinit_inode()needs to hold theILOCK_EXCLas it is changing internal inode state and can race with other RCU protected inode lookups. Have a look at whatxfs_iflush_cluster()does - it grabs theILOCK_SHAREDwhile under rcu +ip->i_flags_lock, and so xfs_iflush/xfs_inode_to_disk() are protected from racing inode updates (during transactions) by that lock.Quoted 6 lines
Quoted 3 lines
Hence it looks to me that
I_FLUSHINGisn't the problem here - it's that we have a transient modified inode state inxfs_reinit_inode()that is externally visisble...Before
xfs_reinit_inode(),XFS_IRECLAIMwill be set inip->i_flags, this looks like can prevent race with other RCU protected inode lookups.That only protects against new lookups - it does not protect against the IRECLAIM flag being set *after* the lookup in
xfs_iflush_cluster()whilst the inode is being flushed to the cluster buffer. That's whyxfs_iflush_cluster()does:rcu_read_lock() lookup inode spinlock(ip->i_flags_lock); check IRECLAIM|IFLUSHINGQuoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
xfs_ilock_nowait(ip, XFS_ILOCK_SHARED) <<<<<<<<set IFLUSHING spin_unlock(ip->i_flags_lock) rcu_read_unlock()At this point, the only lock that is held is
XFS_ILOCK_SHARED, and it's the only lock that protects the inode state outside the lookup scope against concurrent changes.Essentially,
xfs_reinit_inode()needs to add a:
xfs_ilock_nowait(ip, XFS_ILOCK_EXCL)before it set IRECLAIM - if it fails to get the
ILOCK_EXCL, then we need to skip the inode, drop out of RCU scope, delay and retry the lookup.Quoted 2 lines
Can it be considered that don't modifying the information about the on-disk values in the VFS inode in
xfs_reinit_inode()? if so lock can be avoided.We have to reinit the VFS inode because it has gone through ->destroy_inode and so the state has been trashed. We have to bring it back as an
I_NEWinode, which requires reinitialising everything. THe issue is that we store inode state information (like nlink) in the VFS inode instead of the XFS inode portion of the structure (to minimise memory footprint), and that meansxfs_reinit_inode()has a transient state where the VFS inode is not correct. We can avoid that simply by holding theXFS_ILOCK_EXCL, guaranteeing nothing in XFS should be trying to read/modify the internal metadata state while we are reinitialising the VFS inode portion of the structure...
Quoted 6 lines
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
Thanks for the detailed and clear explanation, holding ILOCK_EXCL lock in xfs_reinit_inode() can solve the problem simply, I will resend a patch. :)
Thanks,
Long Li