Skip to contentSkip to conversation

xfs: fix incorrect i_nlink caused by inode racing

PATCH Superseded Long Li 2 participants 2 messages Nov 17, 2022 XFS filesystem

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Long Li

author
#0 lore raw

PATCH v2 1/1 xfs: fix incorrect i_nlink caused by inode racing

The following error occurred during the fsstress test:

XFS: Assertion failed: VFS_I(ip)->i_nlink >= 2, file: fs/xfs/xfs_inode.c, line: 2452

The problem was that inode race condition causes incorrect i_nlink to be written to disk, and then it is read into memory. Consider the following call graph, inodes that are marked as both XFS_IFLUSHING and XFS_IRECLAIMABLE, i_nlink will be reset to 1 and then restored to original value in xfs_reinit_inode(). Therefore, the i_nlink of directory on disk may be set to 1.

xfsaild
xfs_inode_item_push
xfs_iflush_cluster
xfs_iflush
xfs_inode_to_disk

xfs_iget
xfs_iget_cache_hit
xfs_iget_recycle
xfs_reinit_inode
inode_init_always

xfs_reinit_inode() needs to hold the ILOCK_EXCL as it is changing internal inode state and can race with other RCU protected inode lookups. On the read side, xfs_iflush_cluster() grabs the ILOCK_SHARED while under rcu + ip->i_flags_lock, and so xfs_iflush/xfs_inode_to_disk() are protected from racing inode updates (during transactions) by that lock.

---
v2:
- Modify the assertion error code line number
- Use ILOCK_EXCL to prevent inode racing

 fs/xfs/xfs_icache.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
fs/xfs/xfs_icache.c +6 -2
index eae7427062cf..5a1650e769e7 100644
@@ -329,7 +329,7 @@ xfs_reinit_inode(
 
 /*
  * Carefully nudge an inode whose VFS state has been torn down back into a
- * usable state.  Drops the i_flags_lock and the rcu read lock.
+ * usable state.  Drops the i_flags_lock, rcu read lock and XFS_ILOCK_EXCL.
  */
 static int
 xfs_iget_recycle(
@@ -355,6 +355,7 @@ xfs_iget_recycle(
 
 	ASSERT(!rwsem_is_locked(&inode->i_rwsem));
 	error = xfs_reinit_inode(mp, inode);
+	xfs_iunlock(ip, XFS_ILOCK_EXCL);
 	if (error) {
 		/*
 		 * Re-initializing the inode failed, and we are in deep
@@ -516,7 +517,10 @@ xfs_iget_cache_hit(
 
 	/* The inode fits the selection criteria; process it. */
 	if (ip->i_flags & XFS_IRECLAIMABLE) {
-		/* Drops i_flags_lock and RCU read lock. */
+		if (!xfs_ilock_nowait(ip, XFS_ILOCK_EXCL))
+			goto out_skip;
+
+		/* Drops i_flags_lock, RCU read lock and XFS_ILOCK_EXCL. */
 		error = xfs_iget_recycle(pag, ip);
 		if (error)
 			return error;
Signature · 1 line
2.31.1
#1 lore raw
Quoted 52 lines · Long Li

The following error occurred during the fsstress test:

XFS: Assertion failed: VFS_I(ip)->i_nlink >= 2, file: fs/xfs/xfs_inode.c, line: 2452

The problem was that inode race condition causes incorrect i_nlink to be written to disk, and then it is read into memory. Consider the following call graph, inodes that are marked as both XFS_IFLUSHING and XFS_IRECLAIMABLE, i_nlink will be reset to 1 and then restored to original value in xfs_reinit_inode(). Therefore, the i_nlink of directory on disk may be set to 1.

xfsaild
xfs_inode_item_push
xfs_iflush_cluster
xfs_iflush
xfs_inode_to_disk

xfs_iget
xfs_iget_cache_hit
xfs_iget_recycle
xfs_reinit_inode
inode_init_always

xfs_reinit_inode() needs to hold the ILOCK_EXCL as it is changing internal inode state and can race with other RCU protected inode lookups. On the read side, xfs_iflush_cluster() grabs the ILOCK_SHARED while under rcu + ip->i_flags_lock, and so xfs_iflush/xfs_inode_to_disk() are protected from racing inode updates (during transactions) by that lock.

---
v2:
- Modify the assertion error code line number
- Use ILOCK_EXCL to prevent inode racing

 fs/xfs/xfs_icache.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
fs/xfs/xfs_icache.c +1 -1
index eae7427062cf..5a1650e769e7 100644
@@ -329,7 +329,7 @@ xfs_reinit_inode(
 
 /*
  * Carefully nudge an inode whose VFS state has been torn down back into a
- * usable state.  Drops the i_flags_lock and the rcu read lock.
+ * usable state.  Drops the i_flags_lock, rcu read lock and XFS_ILOCK_EXCL.
  */
 static int
 xfs_iget_recycle(

fs/xfs/xfs_icache.c xfs_iget_recycle( · line 358

Quoted 4 lines
fs/xfs/xfs_icache.c +1 -0
@@ -355,6 +355,7 @@ xfs_iget_recycle(
 
 	ASSERT(!rwsem_is_locked(&inode->i_rwsem));
 	error = xfs_reinit_inode(mp, inode);
+	xfs_iunlock(ip, XFS_ILOCK_EXCL);

Ugh, please don't take a lock in one function and drop it in a different function. If the trylock is really necessary for this operation, have xfs_iget_recycle return EAGAIN and then make xfs_iget_cache_hit goto out_skip if recycling returns EAGAIN.

--D

Quoted 18 lines
fs/xfs/xfs_icache.c +4 -1
 	if (error) {
 		/*
 		 * Re-initializing the inode failed, and we are in deep
@@ -516,7 +517,10 @@ xfs_iget_cache_hit(
 
 	/* The inode fits the selection criteria; process it. */
 	if (ip->i_flags & XFS_IRECLAIMABLE) {
-		/* Drops i_flags_lock and RCU read lock. */
+		if (!xfs_ilock_nowait(ip, XFS_ILOCK_EXCL))
+			goto out_skip;
+
+		/* Drops i_flags_lock, RCU read lock and XFS_ILOCK_EXCL. */
 		error = xfs_iget_recycle(pag, ip);
 		if (error)
 			return error;
Signature · 1 line
2.31.1