Skip to contentSkip to conversation

vfs: don't softlockup when evicting inodes

PATCH In review Josef Bacik 2 participants 3 messages Dec 19, 2014 – Dec 21, 2014 VFS

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 vfs: don't softlockup when evicting inodes

If I run an fs_mark job that creates millions of empty files and then immediately unmount the file system I will get a softlockup during unmount. This box has ~140gb of RAM so we never hit sufficient memory pressure to evict enough inodes during the runtime of the benchmark, which means I see around 80 million inodes being evicted at unmount time. With this patch my box no longer softlocks up. Thanks,

---
V1->V2:
-Still occasionally saw a softlockup in evict_inodes so add a cond_resched_lock
to that case as well.

 fs/inode.c | 2 ++
 1 file changed, 2 insertions(+)
fs/inode.c +2 -0
index ad60555..f266765 100644
@@ -581,6 +581,7 @@ static void dispose_list(struct list_head *head)
 		list_del_init(&inode->i_lru);
 
 		evict(inode);
+		cond_resched();
 	}
 }
 
@@ -613,6 +614,7 @@ void evict_inodes(struct super_block *sb)
 		inode_lru_list_del(inode);
 		spin_unlock(&inode->i_lock);
 		list_add(&inode->i_lru, &dispose);
+		cond_resched_lock(&inode_sb_list_lock);
 	}
 	spin_unlock(&inode_sb_list_lock);
 
Signature · 1 line
1.9.3
#1 lore raw
Quoted 29 lines · On Fri, 19 Dec 2014 10:34:40 -0500 Josef Bacik <[email protected]> wrote:

If I run an fs_mark job that creates millions of empty files and then immediately unmount the file system I will get a softlockup during unmount. This box has ~140gb of RAM so we never hit sufficient memory pressure to evict enough inodes during the runtime of the benchmark, which means I see around 80 million inodes being evicted at unmount time. With this patch my box no longer softlocks up. Thanks,

---
V1->V2:

(quoted diff) +0 -1
-Still occasionally saw a softlockup in evict_inodes so add a cond_resched_lock
 to that case as well.
 
 fs/inode.c | 2 ++
 1 file changed, 2 insertions(+)
fs/inode.c +1 -0
index ad60555..f266765 100644
@@ -581,6 +581,7 @@ static void dispose_list(struct list_head *head)
 		list_del_init(&inode->i_lru);
 
 		evict(inode);
+		cond_resched();
 	}
 }
 

fs/inode.c void evict_inodes(struct super_block *sb) · line 620

Quoted 7 lines
fs/inode.c +1 -0
@@ -613,6 +614,7 @@ void evict_inodes(struct super_block *sb)
 		inode_lru_list_del(inode);
 		spin_unlock(&inode->i_lock);
 		list_add(&inode->i_lru, &dispose);
+		cond_resched_lock(&inode_sb_list_lock);
 	}
 	spin_unlock(&inode_sb_list_lock);
 

Is that safe? What guarantees that the next entry in the list (that is, the one already prefetched by list_for_each_entry_safe) will still be there once you drop and reacquire the lock?

Signature · 1 line
Jeff Layton <[email protected]>
#2 lore raw

I had convinced myself it was fine but if we have a final iput of a deleted inode it could screw us, I'll fix this to restart the loop if we resched. Thanks,

Josef

Jeff Layton <[email protected]> wrote:

Quoted 29 lines · On Fri, 19 Dec 2014 10:34:40 -0500 Josef Bacik <[email protected]> wrote:

If I run an fs_mark job that creates millions of empty files and then immediately unmount the file system I will get a softlockup during unmount. This box has ~140gb of RAM so we never hit sufficient memory pressure to evict enough inodes during the runtime of the benchmark, which means I see around 80 million inodes being evicted at unmount time. With this patch my box no longer softlocks up. Thanks,

---
V1->V2:

(quoted diff) +0 -1
-Still occasionally saw a softlockup in evict_inodes so add a cond_resched_lock
 to that case as well.
 
 fs/inode.c | 2 ++
 1 file changed, 2 insertions(+)
fs/inode.c +1 -0
index ad60555..f266765 100644
@@ -581,6 +581,7 @@ static void dispose_list(struct list_head *head)
              list_del_init(&inode->i_lru);
 
              evict(inode);
+             cond_resched();
      }
 }
 

fs/inode.c void evict_inodes(struct super_block *sb) · line 620

Quoted 7 lines
fs/inode.c +1 -0
@@ -613,6 +614,7 @@ void evict_inodes(struct super_block *sb)
              inode_lru_list_del(inode);
              spin_unlock(&inode->i_lock);
              list_add(&inode->i_lru, &dispose);
+             cond_resched_lock(&inode_sb_list_lock);
      }
      spin_unlock(&inode_sb_list_lock);
 

Is that safe? What guarantees that the next entry in the list (that is, the one already prefetched by list_for_each_entry_safe) will still be there once you drop and reacquire the lock?

--
Jeff Layton <[email protected]>