Skip to contentSkip to conversation

ext4: fix ABBA deadlock in ext4_xattr_inode_cache_find()

PATCH Superseded Aditya Srivastava 2 participants 2 messages Jun 25 ext4 file system

v3 of 6 Superseded Latest is v6 →

lore
Share & tools
lore
#0 lore raw

PATCH v3 1/1 ext4: fix ABBA deadlock in ext4_xattr_inode_cache_find()

From: Aditya Prakash Srivastava <[email protected]>

Syzbot/stress-ng reported an ABBA deadlock in ext4 when exercising concurrent xattr workloads (using the ea_inode mount/format option).

The deadlock occurs between the running transaction and the eviction thread:
- Task 1 (stress-ng): Holds a reference to a shared mbcache_entry (ce) and calls ext4_xattr_inode_cache_find() -> ext4_iget() to retrieve the corresponding EA inode. Since the EA inode is currently being evicted, ext4_iget() blocks in __wait_on_freeing_inode() waiting for eviction to complete.
- Task 2 (eviction thread): Currently evicting the same EA inode in ext4_evict_ea_inode(). It calls mb_cache_entry_wait_unused(oe) which blocks waiting for Task 1 to release the reference to the mbcache_entry.

To break this deadlock, implement a new ext4_iget() configuration flag named EXT4_IGET_NOWAIT. When set, perform a non-blocking lookup of the inode via VFS's find_inode_nowait() API.

If the inode is currently being evicted (marked with I_FREEING or I_WILL_FREE) or created (I_CREATING), simply skip it (returning -ESTALE) rather than waiting for eviction/creation to complete, breaking the ABBA cycle. If the returned inode is I_NEW, wait for its initialization to clear via wait_on_new_inode(). Finally, standard validation checks (including is_bad_inode, EXT4_EA_INODE_FL, file_acl, and xattr flags) are executed as normal inside check_igot_inode() to fully guarantee VFS-layer safety.

In ext4_xattr_inode_cache_find(), invoke ext4_iget() with the new EXT4_IGET_NOWAIT flag to perform the non-blocking cache search.

Fixes 0a46ef234756 ("ext4: do not create EA inode under buffer lock")

---
Changes in v3:
- Implement a new ext4_iget() configuration flag named EXT4_IGET_NOWAIT to fully contain the non-blocking lookup and VFS-level validations within inode.c, as requested by Jan Kara.
- Skip inodes currently being created (I_CREATING), following Jan Kara's direct feedback.
- Remove all open-coded match helpers and VFS state-checks from xattr.c.

Changes in v2:
- Read inode state locklessly using inode_state_read_once() to resolve a lockdep assertion on cache hit.
- Manually restore essential inode/ea_inode validations on the retrieved inode (is_bad_inode, EXT4_EA_INODE_FL, file_acl, and xattr checks) to match VFS safety guarantees and prevent using corrupted/failed inodes.

 fs/ext4/ext4.h  |  3 ++-
 fs/ext4/inode.c | 41 ++++++++++++++++++++++++++++++++++++++---
 fs/ext4/xattr.c |  2 +-
 3 files changed, 41 insertions(+), 5 deletions(-)
fs/ext4/ext4.h +2 -1
index b37c136ea3ab..c76dd0bdd3d8 100644
@@ -3144,7 +3144,8 @@ typedef enum {
 	EXT4_IGET_SPECIAL =	0x0001, /* OK to iget a system inode */
 	EXT4_IGET_HANDLE = 	0x0002,	/* Inode # is from a handle */
 	EXT4_IGET_BAD =		0x0004, /* Allow to iget a bad inode */
-	EXT4_IGET_EA_INODE =	0x0008	/* Inode should contain an EA value */
+	EXT4_IGET_EA_INODE =	0x0008,	/* Inode should contain an EA value */
+	EXT4_IGET_NOWAIT =	0x0010	/* Non-blocking lookup (skip if freeing) */
 } ext4_iget_flags;
 
 extern struct inode *__ext4_iget(struct super_block *sb, unsigned long ino,
fs/ext4/inode.c +38 -3

53 lines not on this page · raw message

fs/ext4/xattr.c +1 -1
index 982a1f831e22..21b5670d8503 100644
@@ -1550,7 +1550,7 @@ ext4_xattr_inode_cache_find(struct inode *inode, const void *value,
 
 	while (ce) {
 		ea_inode = ext4_iget(inode->i_sb, ce->e_value,
-				     EXT4_IGET_EA_INODE);
+				     EXT4_IGET_EA_INODE | EXT4_IGET_NOWAIT);
 		if (IS_ERR(ea_inode))
 			goto next_entry;
 		ext4_xattr_inode_set_class(ea_inode);
Signature · 1 line
2.47.3
reviewer
#1 lore raw
Quoted 37 lines · Aditya Srivastava

From: Aditya Prakash Srivastava <[email protected]>

Syzbot/stress-ng reported an ABBA deadlock in ext4 when exercising concurrent xattr workloads (using the ea_inode mount/format option).

The deadlock occurs between the running transaction and the eviction thread:
- Task 1 (stress-ng): Holds a reference to a shared mbcache_entry (ce) and calls ext4_xattr_inode_cache_find() -> ext4_iget() to retrieve the corresponding EA inode. Since the EA inode is currently being evicted, ext4_iget() blocks in __wait_on_freeing_inode() waiting for eviction to complete.
- Task 2 (eviction thread): Currently evicting the same EA inode in ext4_evict_ea_inode(). It calls mb_cache_entry_wait_unused(oe) which blocks waiting for Task 1 to release the reference to the mbcache_entry.

To break this deadlock, implement a new ext4_iget() configuration flag named EXT4_IGET_NOWAIT. When set, perform a non-blocking lookup of the inode via VFS's find_inode_nowait() API.

If the inode is currently being evicted (marked with I_FREEING or I_WILL_FREE) or created (I_CREATING), simply skip it (returning -ESTALE) rather than waiting for eviction/creation to complete, breaking the ABBA cycle. If the returned inode is I_NEW, wait for its initialization to clear via wait_on_new_inode(). Finally, standard validation checks (including is_bad_inode, EXT4_EA_INODE_FL, file_acl, and xattr flags) are executed as normal inside check_igot_inode() to fully guarantee VFS-layer safety.

In ext4_xattr_inode_cache_find(), invoke ext4_iget() with the new EXT4_IGET_NOWAIT flag to perform the non-blocking cache search.

Fixes 0a46ef234756 ("ext4: do not create EA inode under buffer lock")

...

Quoted 24 lines
fs/ext4/inode.c +18 -0
+static int ext4_iget_match(struct inode *inode, u64 ino, void *data)
+{
+	bool *is_freeing = data;
+
+	if (inode->i_ino != ino)
+		return 0;
+	spin_lock(&inode->i_lock);
+	if (inode_state_read(inode) & (I_FREEING | I_WILL_FREE | I_CREATING)) {
+		if (is_freeing)
+			*is_freeing = true;
+		spin_unlock(&inode->i_lock);
+		return -1;
+	}
+	__iget(inode);
+	spin_unlock(&inode->i_lock);
+	return 1;
+}
+
 struct inode *__ext4_iget(struct super_block *sb, unsigned long ino,
 			  ext4_iget_flags flags, const char *function,
 			  unsigned int line)
@@ -5298,9 +5316,26 @@ struct inode *__ext4_iget(struct super_block *sb, unsigned long ino,
 		return ERR_PTR(-EFSCORRUPTED);
 	}

fs/ext4/inode.c struct inode *__ext4_iget(struct super_block *sb, unsigned long ino, · line 5324

Quoted 10 lines
fs/ext4/inode.c +6 -3
 
-	inode = iget_locked(sb, ino);
-	if (!inode)
-		return ERR_PTR(-ENOMEM);
+	if (flags & EXT4_IGET_NOWAIT) {
+		bool is_freeing = false;
+
+		inode = find_inode_nowait(sb, ino, ext4_iget_match, &is_freeing);
+		if (is_freeing)
+			return ERR_PTR(-ESTALE);

In case of EXT4_IGET_NOWAIT I just would not bother with returning errors like this. Just if the inode was not found or cannot be grabbed, I'd return say -ENOENT from __ext4_iget() and be done with it. Due to the nature of EXT4_IGET_NOWAIT it is unreliable and the caller thus has to decide what to do in case the inode cannot be grabbed.

fs/ext4/inode.c struct inode *__ext4_iget(struct super_block *sb, unsigned long ino, · line 5328

Quoted 4 lines
fs/ext4/inode.c +4 -0
+		if (!inode) {
+			inode = iget_locked(sb, ino);
+			if (!inode)
+				return ERR_PTR(-ENOMEM);

Hu, what's the point of this? It defeats the purpose of EXT4_IGET_NOWAIT as it can block...

Honza

Quoted 29 lines
fs/ext4/inode.c +10 -0
+		} else {
+			if (inode_state_read_once(inode) & I_NEW)
+				wait_on_new_inode(inode);
+		}
+	} else {
+		inode = iget_locked(sb, ino);
+		if (!inode)
+			return ERR_PTR(-ENOMEM);
+	}
+
 	if (!(inode_state_read_once(inode) & I_NEW)) {
 		ret = check_igot_inode(inode, flags, function, line);
 		if (ret) {
fs/ext4/xattr.c +1 -1
index 982a1f831e22..21b5670d8503 100644
@@ -1550,7 +1550,7 @@ ext4_xattr_inode_cache_find(struct inode *inode, const void *value,
 
 	while (ce) {
 		ea_inode = ext4_iget(inode->i_sb, ce->e_value,
-				     EXT4_IGET_EA_INODE);
+				     EXT4_IGET_EA_INODE | EXT4_IGET_NOWAIT);
 		if (IS_ERR(ea_inode))
 			goto next_entry;
 		ext4_xattr_inode_set_class(ea_inode);
Signature · 1 line
2.47.3
Signature · 2 lines
Jan Kara <[email protected]>
SUSE Labs, CR