Skip to contentSkip to conversation

ext4: fix circular lock dependency in ext4_ext_migrate

PATCH Superseded Yun Zhou 3 participants 11 messages Jun 9 – Jul 23 ext4 file system

lore
Share & tools
lore

Yun Zhou

author
#0 lore raw

PATCH 1/1 ext4: fix circular lock dependency in ext4_ext_migrate

Move iput(tmp_inode) after ext4_writepages_up_write() to avoid a circular lock dependency between s_writepages_rwsem and sb_internal (freeze protection).

The deadlock scenario:

CPU0 (EXT4_IOC_MIGRATE) CPU1 (orphan cleanup during mount)
---- ----
ext4_ext_migrate()

    ext4_writepages_down_write()
      s_writepages_rwsem (write)
                                 ext4_evict_inode()
                                   sb_start_intwrite()   [sb_internal]
                                   ...
                                     ext4_writepages()
                                       s_writepages_rwsem (read) [BLOCKED]
    iput(tmp_inode)
      ext4_evict_inode()
        sb_start_intwrite()         [BLOCKED]

The tmp_inode is a temporary inode with nlink=0 created solely for building the extent tree. Its eviction does not require s_writepages_rwsem protection, so deferring iput() until after releasing the rwsem is safe.

Reported-by [email protected]
Fixes cb85f4d23f79 ("ext4: fix race between writepages and enabling EXT4_EXTENTS_FL")
---
 fs/ext4/migrate.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
fs/ext4/migrate.c +3 -1
index 477d43d7e294..25368eb44e85 100644
@@ -464,6 +464,7 @@ int ext4_ext_migrate(struct inode *inode)
 	if (IS_ERR(tmp_inode)) {
 		retval = PTR_ERR(tmp_inode);
 		ext4_journal_stop(handle);
+		tmp_inode = NULL;
 		goto out_unlock;
 	}
 	/*
@@ -591,9 +592,10 @@ int ext4_ext_migrate(struct inode *inode)
 	ext4_journal_stop(handle);
 out_tmp_inode:
 	unlock_new_inode(tmp_inode);
-	iput(tmp_inode);
 out_unlock:
 	ext4_writepages_up_write(inode->i_sb, alloc_ctx);
+	if (tmp_inode)
+		iput(tmp_inode);
 	return retval;
 }
 
Signature · 1 line
2.43.0
reviewer Reviewed-by on v1 1/1
#1 lore raw
Quoted 25 lines · Yun Zhou

Move iput(tmp_inode) after ext4_writepages_up_write() to avoid a circular lock dependency between s_writepages_rwsem and sb_internal (freeze protection).

The deadlock scenario:

CPU0 (EXT4_IOC_MIGRATE) CPU1 (orphan cleanup during mount)
---- ----
ext4_ext_migrate()

    ext4_writepages_down_write()
      s_writepages_rwsem (write)
                                 ext4_evict_inode()
                                   sb_start_intwrite()   [sb_internal]
                                   ...
                                     ext4_writepages()
                                       s_writepages_rwsem (read) [BLOCKED]
    iput(tmp_inode)
      ext4_evict_inode()
        sb_start_intwrite()         [BLOCKED]

The tmp_inode is a temporary inode with nlink=0 created solely for building the extent tree. Its eviction does not require s_writepages_rwsem protection, so deferring iput() until after releasing the rwsem is safe.

Quoted 4 lines
Reported-by [email protected]
Fixes cb85f4d23f79 ("ext4: fix race between writepages and enabling EXT4_EXTENTS_FL")

Looks good. Feel free to add:

Just one nit below:

fs/ext4/migrate.c int ext4_ext_migrate(struct inode *inode) · line 598

Quoted 9 lines
fs/ext4/migrate.c +2 -1
@@ -591,9 +592,10 @@ int ext4_ext_migrate(struct inode *inode)
 	ext4_journal_stop(handle);
 out_tmp_inode:
 	unlock_new_inode(tmp_inode);
-	iput(tmp_inode);
 out_unlock:
 	ext4_writepages_up_write(inode->i_sb, alloc_ctx);
+	if (tmp_inode)
+		iput(tmp_inode);

iput(NULL) is properly handled so you don't need the if (tmp_inode) check here.

Honza

Signature · 2 lines
Jan Kara <[email protected]>
SUSE Labs, CR
author
#2 lore raw
Quoted 6 lines · Jan Kara

Looks good. Feel free to add:

Just one nit below:

Quoted 11 lines

fs/ext4/migrate.c int ext4_ext_migrate(struct inode *inode) · line 598

Quoted 9 lines
fs/ext4/migrate.c +2 -1
@@ -591,9 +592,10 @@ int ext4_ext_migrate(struct inode *inode)
       ext4_journal_stop(handle);
  out_tmp_inode:
       unlock_new_inode(tmp_inode);
-     iput(tmp_inode);
  out_unlock:
       ext4_writepages_up_write(inode->i_sb, alloc_ctx);
+     if (tmp_inode)
+             iput(tmp_inode);

iput(NULL) is properly handled so you don't need the if (tmp_inode) check here.

Hi Jan,

Thank you for your careful review. Should I remove this redundant check in v2?

BR,
Yun

reviewer
#3 lore raw
Quoted 20 lines · Yun
Quoted 6 lines · Jan Kara

Looks good. Feel free to add:

Just one nit below:

Quoted 11 lines

fs/ext4/migrate.c int ext4_ext_migrate(struct inode *inode) · line 598

Quoted 9 lines
fs/ext4/migrate.c +2 -1
@@ -591,9 +592,10 @@ int ext4_ext_migrate(struct inode *inode)
       ext4_journal_stop(handle);
  out_tmp_inode:
       unlock_new_inode(tmp_inode);
-     iput(tmp_inode);
  out_unlock:
       ext4_writepages_up_write(inode->i_sb, alloc_ctx);
+     if (tmp_inode)
+             iput(tmp_inode);

iput(NULL) is properly handled so you don't need the if (tmp_inode) check here.

Quoted 4 lines

Hi Jan,

Thank you for your careful review. Should I remove this redundant check in v2?

Yes, please. Thank you!

Honza

Signature · 2 lines
Jan Kara <[email protected]>
SUSE Labs, CR
author
#4 lore raw

PATCH v2 1/1 ext4: fix circular lock dependency in ext4_ext_migrate

In reply to Yun Zhou

Move iput(tmp_inode) after ext4_writepages_up_write() to avoid a circular lock dependency between s_writepages_rwsem and sb_internal (freeze protection).

The deadlock scenario:

CPU0 (EXT4_IOC_MIGRATE) CPU1 (orphan cleanup during mount)
---- ----
ext4_ext_migrate()

    ext4_writepages_down_write()
      s_writepages_rwsem (write)
                                 ext4_evict_inode()
                                   sb_start_intwrite()   [sb_internal]
                                   ...
                                     ext4_writepages()
                                       s_writepages_rwsem (read) [BLOCKED]
    iput(tmp_inode)
      ext4_evict_inode()
        sb_start_intwrite()         [BLOCKED]

The tmp_inode is a temporary inode with nlink=0 created solely for building the extent tree. Its eviction does not require s_writepages_rwsem protection, so deferring iput() until after releasing the rwsem is safe.

Reported-by [email protected]
Fixes cb85f4d23f79 ("ext4: fix race between writepages and enabling EXT4_EXTENTS_FL")

---
v2: remove redundant null pointer check for iput(tmp_inode)

 fs/ext4/migrate.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
fs/ext4/migrate.c +2 -1
index 477d43d7e294..5d60ef10fe11 100644
@@ -464,6 +464,7 @@ int ext4_ext_migrate(struct inode *inode)
 	if (IS_ERR(tmp_inode)) {
 		retval = PTR_ERR(tmp_inode);
 		ext4_journal_stop(handle);
+		tmp_inode = NULL;
 		goto out_unlock;
 	}
 	/*
@@ -591,9 +592,9 @@ int ext4_ext_migrate(struct inode *inode)
 	ext4_journal_stop(handle);
 out_tmp_inode:
 	unlock_new_inode(tmp_inode);
-	iput(tmp_inode);
 out_unlock:
 	ext4_writepages_up_write(inode->i_sb, alloc_ctx);
+	iput(tmp_inode);
 	return retval;
 }
 
Signature · 1 line
2.43.0
author
#5 lore raw

PATCH v3 1/1 ext4: fix circular lock dependency in ext4_ext_migrate

In reply to Yun Zhou

Move iput(tmp_inode) after ext4_writepages_up_write() to avoid a circular lock dependency between s_writepages_rwsem and sb_internal (freeze protection).

The deadlock scenario:

CPU0 (EXT4_IOC_MIGRATE) CPU1 (orphan cleanup during mount)
---- ----
ext4_ext_migrate()

    ext4_writepages_down_write()
      s_writepages_rwsem (write)
                                 ext4_evict_inode()
                                   sb_start_intwrite()   [sb_internal]
                                   ...
                                     ext4_writepages()
                                       s_writepages_rwsem (read) [BLOCKED]
    iput(tmp_inode)
      ext4_evict_inode()
        sb_start_intwrite()         [BLOCKED]

The tmp_inode is a temporary inode with nlink=0 created solely for building the extent tree. Its eviction does not require s_writepages_rwsem protection, so deferring iput() until after releasing the rwsem is safe.

Reported-by [email protected]
Fixes cb85f4d23f79 ("ext4: fix race between writepages and enabling EXT4_EXTENTS_FL")

---
v3: fixes Reported-by tag and Closes tag.

v2: remove redundant null pointer check for iput(tmp_inode).

 fs/ext4/migrate.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
fs/ext4/migrate.c +2 -1
index 477d43d7e294..5d60ef10fe11 100644
@@ -464,6 +464,7 @@ int ext4_ext_migrate(struct inode *inode)
 	if (IS_ERR(tmp_inode)) {
 		retval = PTR_ERR(tmp_inode);
 		ext4_journal_stop(handle);
+		tmp_inode = NULL;
 		goto out_unlock;
 	}
 	/*
@@ -591,9 +592,9 @@ int ext4_ext_migrate(struct inode *inode)
 	ext4_journal_stop(handle);
 out_tmp_inode:
 	unlock_new_inode(tmp_inode);
-	iput(tmp_inode);
 out_unlock:
 	ext4_writepages_up_write(inode->i_sb, alloc_ctx);
+	iput(tmp_inode);
 	return retval;
 }
 
Signature · 1 line
2.43.0
author
#6 lore raw
Quoted 30 lines

Move iput(tmp_inode) after ext4_writepages_up_write() to avoid a circular lock dependency between s_writepages_rwsem and sb_internal (freeze protection).

The deadlock scenario:

   CPU0 (EXT4_IOC_MIGRATE)        CPU1 (orphan cleanup during mount)
   ----                           ----
   ext4_ext_migrate()
     ext4_writepages_down_write()
       s_writepages_rwsem (write)
                                  ext4_evict_inode()
                                    sb_start_intwrite()   [sb_internal]
                                    ...
                                      ext4_writepages()
                                        s_writepages_rwsem (read) [BLOCKED]
     iput(tmp_inode)
       ext4_evict_inode()
         sb_start_intwrite()         [BLOCKED]

The tmp_inode is a temporary inode with nlink=0 created solely for building the extent tree. Its eviction does not require s_writepages_rwsem protection, so deferring iput() until after releasing the rwsem is safe.

Reported-by [email protected]
Fixes cb85f4d23f79 ("ext4: fix race between writepages and enabling EXT4_EXTENTS_FL")
Quoted 8 lines

---
v3: fixes Reported-by tag and Closes tag.

v2: remove redundant null pointer check for iput(tmp_inode).

  fs/ext4/migrate.c | 3 ++-
  1 file changed, 2 insertions(+), 1 deletion(-)

Hi Honza,

Thank you very much for taking the time to review these patches and providing your valuable suggestions. I am eager to solve these long-standing deadlock issues on Syzkaller, but I do not have much community experience. I'd like to know, regarding this patch, should I launch a new RR thread or continue waiting? BR, Yun

reviewer
#7 lore raw

Hello Yun!

Quoted 35 lines · Yun
Quoted 35 lines

Move iput(tmp_inode) after ext4_writepages_up_write() to avoid a circular lock dependency between s_writepages_rwsem and sb_internal (freeze protection).

The deadlock scenario:

   CPU0 (EXT4_IOC_MIGRATE)        CPU1 (orphan cleanup during mount)
   ----                           ----
   ext4_ext_migrate()
     ext4_writepages_down_write()
       s_writepages_rwsem (write)
                                  ext4_evict_inode()
                                    sb_start_intwrite()   [sb_internal]
                                    ...
                                      ext4_writepages()
                                        s_writepages_rwsem (read) [BLOCKED]
     iput(tmp_inode)
       ext4_evict_inode()
         sb_start_intwrite()         [BLOCKED]

The tmp_inode is a temporary inode with nlink=0 created solely for building the extent tree. Its eviction does not require s_writepages_rwsem protection, so deferring iput() until after releasing the rwsem is safe.

Reported-by [email protected]
Fixes cb85f4d23f79 ("ext4: fix race between writepages and enabling EXT4_EXTENTS_FL")

---
v3: fixes Reported-by tag and Closes tag.

v2: remove redundant null pointer check for iput(tmp_inode).

Quoted 8 lines
Quoted 2 lines
  fs/ext4/migrate.c | 3 ++-
  1 file changed, 2 insertions(+), 1 deletion(-)

Thank you very much for taking the time to review these patches and providing your valuable suggestions. I am eager to solve these long-standing deadlock issues on Syzkaller, but I do not have much community experience. I'd like to know, regarding this patch, should I launch a new RR thread or continue waiting? BR, Yun

Please keep waiting. On Sunday the merge window for 7.2-rc1 has opened. That means that about week before and at least for the next week, maintainers often aren't taking any patches in their trees. I expect Ted to pick your patch later when he collects fixes to send for 7.2-rc2 or so - he sends email about that as a reply to the patch. If nothing happens for next two weeks, I suggest you send a ping asking whether the patch didn't get lost as a reply to your patch submission. Thanks for your fixes!

Honza

Signature · 2 lines
Jan Kara <[email protected]>
SUSE Labs, CR
author
#8 lore raw
In reply to Yun Zhou

Friendly ping.

Quoted 61 lines · Yun Zhou

Move iput(tmp_inode) after ext4_writepages_up_write() to avoid a circular lock dependency between s_writepages_rwsem and sb_internal (freeze protection).

The deadlock scenario:

   CPU0 (EXT4_IOC_MIGRATE)        CPU1 (orphan cleanup during mount)
   ----                           ----
   ext4_ext_migrate()
     ext4_writepages_down_write()
       s_writepages_rwsem (write)
                                  ext4_evict_inode()
                                    sb_start_intwrite()   [sb_internal]
                                    ...
                                      ext4_writepages()
                                        s_writepages_rwsem (read) [BLOCKED]
     iput(tmp_inode)
       ext4_evict_inode()
         sb_start_intwrite()         [BLOCKED]

The tmp_inode is a temporary inode with nlink=0 created solely for building the extent tree. Its eviction does not require s_writepages_rwsem protection, so deferring iput() until after releasing the rwsem is safe.

Reported-by [email protected]
Fixes cb85f4d23f79 ("ext4: fix race between writepages and enabling EXT4_EXTENTS_FL")

---
v3: fixes Reported-by tag and Closes tag.

v2: remove redundant null pointer check for iput(tmp_inode).

  fs/ext4/migrate.c | 3 ++-
  1 file changed, 2 insertions(+), 1 deletion(-)
fs/ext4/migrate.c +2 -1
index 477d43d7e294..5d60ef10fe11 100644
@@ -464,6 +464,7 @@ int ext4_ext_migrate(struct inode *inode)
  	if (IS_ERR(tmp_inode)) {
  		retval = PTR_ERR(tmp_inode);
  		ext4_journal_stop(handle);
+		tmp_inode = NULL;
  		goto out_unlock;
  	}
  	/*
@@ -591,9 +592,9 @@ int ext4_ext_migrate(struct inode *inode)
  	ext4_journal_stop(handle);
  out_tmp_inode:
  	unlock_new_inode(tmp_inode);
-	iput(tmp_inode);
  out_unlock:
  	ext4_writepages_up_write(inode->i_sb, alloc_ctx);
+	iput(tmp_inode);
  	return retval;
  }
 
author
#9 lore raw
In reply to Yun Zhou

Hi Ted,

Gentle ping on this patch. It has been reviewed and received
Reviewed-by tags from Jan Kara a few weeks ago. I haven't heard back since then, so I wanted to check if there's anything else needed from my side.

BR,
Yun

Quoted 61 lines · Yun Zhou

Move iput(tmp_inode) after ext4_writepages_up_write() to avoid a circular lock dependency between s_writepages_rwsem and sb_internal (freeze protection).

The deadlock scenario:

   CPU0 (EXT4_IOC_MIGRATE)        CPU1 (orphan cleanup during mount)
   ----                           ----
   ext4_ext_migrate()
     ext4_writepages_down_write()
       s_writepages_rwsem (write)
                                  ext4_evict_inode()
                                    sb_start_intwrite()   [sb_internal]
                                    ...
                                      ext4_writepages()
                                        s_writepages_rwsem (read) [BLOCKED]
     iput(tmp_inode)
       ext4_evict_inode()
         sb_start_intwrite()         [BLOCKED]

The tmp_inode is a temporary inode with nlink=0 created solely for building the extent tree. Its eviction does not require s_writepages_rwsem protection, so deferring iput() until after releasing the rwsem is safe.

Reported-by [email protected]
Fixes cb85f4d23f79 ("ext4: fix race between writepages and enabling EXT4_EXTENTS_FL")

---
v3: fixes Reported-by tag and Closes tag.

v2: remove redundant null pointer check for iput(tmp_inode).

  fs/ext4/migrate.c | 3 ++-
  1 file changed, 2 insertions(+), 1 deletion(-)
fs/ext4/migrate.c +2 -1
index 477d43d7e294..5d60ef10fe11 100644
@@ -464,6 +464,7 @@ int ext4_ext_migrate(struct inode *inode)
  	if (IS_ERR(tmp_inode)) {
  		retval = PTR_ERR(tmp_inode);
  		ext4_journal_stop(handle);
+		tmp_inode = NULL;
  		goto out_unlock;
  	}
  	/*
@@ -591,9 +592,9 @@ int ext4_ext_migrate(struct inode *inode)
  	ext4_journal_stop(handle);
  out_tmp_inode:
  	unlock_new_inode(tmp_inode);
-	iput(tmp_inode);
  out_unlock:
  	ext4_writepages_up_write(inode->i_sb, alloc_ctx);
+	iput(tmp_inode);
  	return retval;
  }
 
maintainer Applied
#10 lore raw
In reply to Yun Zhou
Quoted 20 lines · Yun Zhou

Move iput(tmp_inode) after ext4_writepages_up_write() to avoid a circular lock dependency between s_writepages_rwsem and sb_internal (freeze protection).

The deadlock scenario:

CPU0 (EXT4_IOC_MIGRATE) CPU1 (orphan cleanup during mount)
---- ----
ext4_ext_migrate()

    ext4_writepages_down_write()
      s_writepages_rwsem (write)
                                 ext4_evict_inode()
                                   sb_start_intwrite()   [sb_internal]
                                   ...
                                     ext4_writepages()
                                       s_writepages_rwsem (read) [BLOCKED]
    iput(tmp_inode)
      ext4_evict_inode()
        sb_start_intwrite()         [BLOCKED]
Quoted 1 line

[...]

Applied, thanks!

[1/1] ext4: fix circular lock dependency in ext4_ext_migrate

      commit: a897682793eba5de51ee6f3152760374afa629cf

Best regards,

Signature · 1 line
Theodore Ts'o <[email protected]>