mm: don't reclaim inodes with many attached pages

Spock reported that the commit 172b06c32b94 ("mm: slowly shrink slabs with a relatively small number of objects") leads to a regression on his setup: periodically the majority of the pagecache is evicted without an obvious reason, while before the change the amount of free memory was balancing around the watermark.

The reason behind is that the mentioned above change created some minimal background pressure on the inode cache. The problem is that if an inode is considered to be reclaimed, all belonging pagecache page are stripped, no matter how many of them are there. So, if a huge multi-gigabyte file is cached in the memory, and the goal is to reclaim only few slab objects (unused inodes), we still can eventually evict all gigabytes of the pagecache at once.

The workload described by Spock has few large non-mapped files in the pagecache, so it's especially noticeable.

To solve the problem let's postpone the reclaim of inodes, which have more than 1 attached page. Let's wait until the pagecache pages will be evicted naturally by scanning the corresponding LRU lists, and only then reclaim the inode structure.

...

fs/inode.c +0 -0

@@ -730,8 +730,11 @@ static enum lru_status inode_lru_isolate(struct list_head *item,
 		return LRU_REMOVED;
 	}
 

Quoted 30 lines · On Tue, 23 Oct 2018 16:43:29 +0000 Roman Gushchin <[email protected]> wrote:

Spock reported that the commit 172b06c32b94 ("mm: slowly shrink slabs with a relatively small number of objects") leads to a regression on his setup: periodically the majority of the pagecache is evicted without an obvious reason, while before the change the amount of free memory was balancing around the watermark.

The reason behind is that the mentioned above change created some minimal background pressure on the inode cache. The problem is that if an inode is considered to be reclaimed, all belonging pagecache page are stripped, no matter how many of them are there. So, if a huge multi-gigabyte file is cached in the memory, and the goal is to reclaim only few slab objects (unused inodes), we still can eventually evict all gigabytes of the pagecache at once.

The workload described by Spock has few large non-mapped files in the pagecache, so it's especially noticeable.

To solve the problem let's postpone the reclaim of inodes, which have more than 1 attached page. Let's wait until the pagecache pages will be evicted naturally by scanning the corresponding LRU lists, and only then reclaim the inode structure.

...

fs/inode.c +0 -0

@@ -730,8 +730,11 @@ static enum lru_status inode_lru_isolate(struct list_head *item,
 		return LRU_REMOVED;
 	}
 

fs/inode.c static enum lru_status inode_lru_isolate(struct list_head *item, · line 740

Quoted 10 lines

fs/inode.c +5 -2

-	/* recently referenced inodes get one more pass */
-	if (inode->i_state & I_REFERENCED) {
+	/*
+	 * Recently referenced inodes and inodes with many attached pages
+	 * get one more pass.
+	 */
+	if (inode->i_state & I_REFERENCED || inode->i_data.nrpages > 1) {
 		inode->i_state &= ~I_REFERENCED;
 		spin_unlock(&inode->i_lock);
 		return LRU_ROTATE;

hm, why "1"?

I guess one could argue that this will encompass long symlinks, but I just made that up to make "1" appear more justifiable ;)

Spock reported that the commit 172b06c32b94 ("mm: slowly shrink slabs with a relatively small number of objects") leads to a regression on his setup: periodically the majority of the pagecache is evicted without an obvious reason, while before the change the amount of free memory was balancing around the watermark.

The reason behind is that the mentioned above change created some minimal background pressure on the inode cache. The problem is that if an inode is considered to be reclaimed, all belonging pagecache page are stripped, no matter how many of them are there. So, if a huge multi-gigabyte file is cached in the memory, and the goal is to reclaim only few slab objects (unused inodes), we still can eventually evict all gigabytes of the pagecache at once.

The workload described by Spock has few large non-mapped files in the pagecache, so it's especially noticeable.

To solve the problem let's postpone the reclaim of inodes, which have more than 1 attached page. Let's wait until the pagecache pages will be evicted naturally by scanning the corresponding LRU lists, and only then reclaim the inode structure.

Quoted 17 lines · On Tue, 23 Oct 2018 16:43:29 +0000 Roman Gushchin <[email protected]> wrote:

Spock reported that the commit 172b06c32b94 ("mm: slowly shrink slabs with a relatively small number of objects") leads to a regression on his setup: periodically the majority of the pagecache is evicted without an obvious reason, while before the change the amount of free memory was balancing around the watermark.

The reason behind is that the mentioned above change created some minimal background pressure on the inode cache. The problem is that if an inode is considered to be reclaimed, all belonging pagecache page are stripped, no matter how many of them are there. So, if a huge multi-gigabyte file is cached in the memory, and the goal is to reclaim only few slab objects (unused inodes), we still can eventually evict all gigabytes of the pagecache at once.

The workload described by Spock has few large non-mapped files in the pagecache, so it's especially noticeable.

Quoted 4 lines

To solve the problem let's postpone the reclaim of inodes, which have more than 1 attached page. Let's wait until the pagecache pages will be evicted naturally by scanning the corresponding LRU lists, and only then reclaim the inode structure.

Is this regression serious enough to warrant fixing 4.19.1?

Quoted 17 lines · On Tue, 23 Oct 2018 16:43:29 +0000 Roman Gushchin <[email protected]> wrote:

Spock reported that the commit 172b06c32b94 ("mm: slowly shrink slabs with a relatively small number of objects") leads to a regression on his setup: periodically the majority of the pagecache is evicted without an obvious reason, while before the change the amount of free memory was balancing around the watermark.

The reason behind is that the mentioned above change created some minimal background pressure on the inode cache. The problem is that if an inode is considered to be reclaimed, all belonging pagecache page are stripped, no matter how many of them are there. So, if a huge multi-gigabyte file is cached in the memory, and the goal is to reclaim only few slab objects (unused inodes), we still can eventually evict all gigabytes of the pagecache at once.

The workload described by Spock has few large non-mapped files in the pagecache, so it's especially noticeable.

Quoted 4 lines

To solve the problem let's postpone the reclaim of inodes, which have more than 1 attached page. Let's wait until the pagecache pages will be evicted naturally by scanning the corresponding LRU lists, and only then reclaim the inode structure.

Is this regression serious enough to warrant fixing 4.19.1?

Quoted 24 lines · Andrew Morton

Quoted 21 lines · On Tue, 23 Oct 2018 16:43:29 +0000 Roman Gushchin <[email protected]> wrote:

Spock reported that the commit 172b06c32b94 ("mm: slowly shrink slabs with a relatively small number of objects") leads to a regression on his setup: periodically the majority of the pagecache is evicted without an obvious reason, while before the change the amount of free memory was balancing around the watermark.

The reason behind is that the mentioned above change created some minimal background pressure on the inode cache. The problem is that if an inode is considered to be reclaimed, all belonging pagecache page are stripped, no matter how many of them are there. So, if a huge multi-gigabyte file is cached in the memory, and the goal is to reclaim only few slab objects (unused inodes), we still can eventually evict all gigabytes of the pagecache at once.

The workload described by Spock has few large non-mapped files in the pagecache, so it's especially noticeable.

To solve the problem let's postpone the reclaim of inodes, which have more than 1 attached page. Let's wait until the pagecache pages will be evicted naturally by scanning the corresponding LRU lists, and only then reclaim the inode structure.

Quoted 1 line

Is this regression serious enough to warrant fixing 4.19.1?

Let's not forget about stable tree(s) which backported 172b06c32b94. I would suggest reverting there.

Quoted 25 lines · On Thu, 25 Oct 2018 11:23:52 +0200 Michal Hocko <[email protected]> wrote:

Quoted 24 lines · Andrew Morton

Quoted 21 lines · On Tue, 23 Oct 2018 16:43:29 +0000 Roman Gushchin <[email protected]> wrote:

Spock reported that the commit 172b06c32b94 ("mm: slowly shrink slabs with a relatively small number of objects") leads to a regression on his setup: periodically the majority of the pagecache is evicted without an obvious reason, while before the change the amount of free memory was balancing around the watermark.

The reason behind is that the mentioned above change created some minimal background pressure on the inode cache. The problem is that if an inode is considered to be reclaimed, all belonging pagecache page are stripped, no matter how many of them are there. So, if a huge multi-gigabyte file is cached in the memory, and the goal is to reclaim only few slab objects (unused inodes), we still can eventually evict all gigabytes of the pagecache at once.

The workload described by Spock has few large non-mapped files in the pagecache, so it's especially noticeable.

To solve the problem let's postpone the reclaim of inodes, which have more than 1 attached page. Let's wait until the pagecache pages will be evicted naturally by scanning the corresponding LRU lists, and only then reclaim the inode structure.

Quoted 4 lines

Quoted 1 line

Is this regression serious enough to warrant fixing 4.19.1?

Let's not forget about stable tree(s) which backported 172b06c32b94. I would suggest reverting there.

Yup. Sasha, can you please take care of this?

Quoted 1 line · Andrew Morton

Yup. Sasha, can you please take care of this?

Sure, I'll revert it from current stable trees.

Should 172b06c32b94 and this commit be backported once Roman confirms the issue is fixed? As far as I understand 172b06c32b94 addressed an issue FB were seeing in their fleet and needed to be fixed.

Quoted 8 lines · Sasha Levin

Quoted 1 line · Andrew Morton

Yup. Sasha, can you please take care of this?

Sure, I'll revert it from current stable trees.

Should 172b06c32b94 and this commit be backported once Roman confirms the issue is fixed? As far as I understand 172b06c32b94 addressed an issue FB were seeing in their fleet and needed to be fixed.

I'm not sure I see "FB sees an issue in their fleet" and "needs to be fixed in stable kernels" as related. FB's workload is different from most people's workloads and FB has a large and highly-skilled team of kernel engineers. Obviously I want this problem fixed in mainline, but I don't know that most people benefit from having it fixed in stable.

Quoted 32 lines · Andrew Morton

Quoted 29 lines · On Thu, 25 Oct 2018 11:23:52 +0200 Michal Hocko <[email protected]> wrote:

Quoted 19 lines · Andrew Morton

Quoted 17 lines · On Tue, 23 Oct 2018 16:43:29 +0000 Roman Gushchin <[email protected]> wrote:

Spock reported that the commit 172b06c32b94 ("mm: slowly shrink slabs with a relatively small number of objects") leads to a regression on his setup: periodically the majority of the pagecache is evicted without an obvious reason, while before the change the amount of free memory was balancing around the watermark.

The reason behind is that the mentioned above change created some minimal background pressure on the inode cache. The problem is that if an inode is considered to be reclaimed, all belonging pagecache page are stripped, no matter how many of them are there. So, if a huge multi-gigabyte file is cached in the memory, and the goal is to reclaim only few slab objects (unused inodes), we still can eventually evict all gigabytes of the pagecache at once.

The workload described by Spock has few large non-mapped files in the pagecache, so it's especially noticeable.

Quoted 6 lines

Quoted 4 lines

To solve the problem let's postpone the reclaim of inodes, which have more than 1 attached page. Let's wait until the pagecache pages will be evicted naturally by scanning the corresponding LRU lists, and only then reclaim the inode structure.

Is this regression serious enough to warrant fixing 4.19.1?

Let's not forget about stable tree(s) which backported 172b06c32b94. I would suggest reverting there.

Quoted 1 line

Yup. Sasha, can you please take care of this?

Sure, I'll revert it from current stable trees.

Should 172b06c32b94 and this commit be backported once Roman confirms the issue is fixed? As far as I understand 172b06c32b94 addressed an issue FB were seeing in their fleet and needed to be fixed.

Quoted 35 lines · Sasha Levin

Quoted 33 lines · Andrew Morton

Quoted 25 lines · On Thu, 25 Oct 2018 11:23:52 +0200 Michal Hocko <[email protected]> wrote:

Quoted 24 lines · Andrew Morton

Quoted 21 lines · On Tue, 23 Oct 2018 16:43:29 +0000 Roman Gushchin <[email protected]> wrote:

Spock reported that the commit 172b06c32b94 ("mm: slowly shrink slabs with a relatively small number of objects") leads to a regression on his setup: periodically the majority of the pagecache is evicted without an obvious reason, while before the change the amount of free memory was balancing around the watermark.

The reason behind is that the mentioned above change created some minimal background pressure on the inode cache. The problem is that if an inode is considered to be reclaimed, all belonging pagecache page are stripped, no matter how many of them are there. So, if a huge multi-gigabyte file is cached in the memory, and the goal is to reclaim only few slab objects (unused inodes), we still can eventually evict all gigabytes of the pagecache at once.

The workload described by Spock has few large non-mapped files in the pagecache, so it's especially noticeable.

To solve the problem let's postpone the reclaim of inodes, which have more than 1 attached page. Let's wait until the pagecache pages will be evicted naturally by scanning the corresponding LRU lists, and only then reclaim the inode structure.

Quoted 4 lines

Quoted 1 line

Is this regression serious enough to warrant fixing 4.19.1?

Let's not forget about stable tree(s) which backported 172b06c32b94. I would suggest reverting there.

Yup. Sasha, can you please take care of this?

Quoted 5 lines

Sure, I'll revert it from current stable trees.

Should 172b06c32b94 and this commit be backported once Roman confirms the issue is fixed? As far as I understand 172b06c32b94 addressed an issue FB were seeing in their fleet and needed to be fixed.

The memcg leak was also independently reported by several companies, so it's not only about our fleet.

The memcg css leak is fixed by a series of commits (as in the mm tree):
37e521912118 math64: prevent double calculation of DIV64_U64_ROUND_UP() arguments c6be4e82b1b3 mm: don't miss the last page because of round-off error f2e821fc8c63 mm: drain memcg stocks on css offlining
03a971b56f18 mm: rework memcg kernel stack accounting

172b06c32b94 mm: slowly shrink slabs with a relatively small number of objects

The last one by itself isn't enough, and it makes no sense to backport it without all other patches. So, I'd either backport them all (including 47036ad4032e ("mm: don't reclaim inodes with many attached pages"), either just revert 172b06c32b94.

Also 172b06c32b94 ("mm: slowly shrink slabs with a relatively small number of objects") by itself is fine, but it reveals an independent issue in inode reclaim code, which 47036ad4032e ("mm: don't reclaim inodes with many attached pages") aims to fix.

Quoted 42 lines · Roman Gushchin

Quoted 40 lines · Sasha Levin

Quoted 27 lines · Andrew Morton

Quoted 25 lines · On Thu, 25 Oct 2018 11:23:52 +0200 Michal Hocko <[email protected]> wrote:

Quoted 24 lines · Andrew Morton

Quoted 21 lines · On Tue, 23 Oct 2018 16:43:29 +0000 Roman Gushchin <[email protected]> wrote:

Spock reported that the commit 172b06c32b94 ("mm: slowly shrink slabs with a relatively small number of objects") leads to a regression on his setup: periodically the majority of the pagecache is evicted without an obvious reason, while before the change the amount of free memory was balancing around the watermark.

The reason behind is that the mentioned above change created some minimal background pressure on the inode cache. The problem is that if an inode is considered to be reclaimed, all belonging pagecache page are stripped, no matter how many of them are there. So, if a huge multi-gigabyte file is cached in the memory, and the goal is to reclaim only few slab objects (unused inodes), we still can eventually evict all gigabytes of the pagecache at once.

The workload described by Spock has few large non-mapped files in the pagecache, so it's especially noticeable.

To solve the problem let's postpone the reclaim of inodes, which have more than 1 attached page. Let's wait until the pagecache pages will be evicted naturally by scanning the corresponding LRU lists, and only then reclaim the inode structure.

Quoted 6 lines

Quoted 4 lines

Quoted 1 line

Is this regression serious enough to warrant fixing 4.19.1?

Let's not forget about stable tree(s) which backported 172b06c32b94. I would suggest reverting there.

Yup. Sasha, can you please take care of this?

Sure, I'll revert it from current stable trees.

Should 172b06c32b94 and this commit be backported once Roman confirms the issue is fixed? As far as I understand 172b06c32b94 addressed an issue FB were seeing in their fleet and needed to be fixed.

Quoted 2 lines

The memcg leak was also independently reported by several companies, so it's not only about our fleet.

By memcg leak you mean a lot of dead memcgs with small amount of memory which are staying behind and the global memory pressure removes them only very slowly or almost not at all, right?

I have avague recollection that systemd can trigger a pattern which makes this "leak" noticeable. Is that right? If yes what would be a minimal and safe fix for the stable tree? "mm: don't miss the last page because of round-off error" would sound like the candidate but I never got around to review it properly.

Quoted 5 lines

The memcg css leak is fixed by a series of commits (as in the mm tree):
37e521912118 math64: prevent double calculation of DIV64_U64_ROUND_UP() arguments c6be4e82b1b3 mm: don't miss the last page because of round-off error f2e821fc8c63 mm: drain memcg stocks on css offlining
03a971b56f18 mm: rework memcg kernel stack accounting

btw. none of these sha are refering to anything in my git tree. They all seem to be in the next tree though.

Quoted 8 lines

172b06c32b94 mm: slowly shrink slabs with a relatively small number of objects

The last one by itself isn't enough, and it makes no sense to backport it without all other patches. So, I'd either backport them all (including 47036ad4032e ("mm: don't reclaim inodes with many attached pages"), either just revert 172b06c32b94.

Quoted 3 lines

Also 172b06c32b94 ("mm: slowly shrink slabs with a relatively small number of objects") by itself is fine, but it reveals an independent issue in inode reclaim code, which 47036ad4032e ("mm: don't reclaim inodes with many attached pages") aims to fix.

To me it sounds it needs much more time to settle before it can be considered safe for the stable tree. Even if the patch itself is correct it seems too subtle and reveal a behavior which was not anticipated and that just proves it is far from straightforward.

Spock reported that the commit 172b06c32b94 ("mm: slowly shrink slabs with a relatively small number of objects") leads to a regression on his setup: periodically the majority of the pagecache is evicted without an obvious reason, while before the change the amount of free memory was balancing around the watermark.

The reason behind is that the mentioned above change created some minimal background pressure on the inode cache. The problem is that if an inode is considered to be reclaimed, all belonging pagecache page are stripped, no matter how many of them are there. So, if a huge multi-gigabyte file is cached in the memory, and the goal is to reclaim only few slab objects (unused inodes), we still can eventually evict all gigabytes of the pagecache at once.

The workload described by Spock has few large non-mapped files in the pagecache, so it's especially noticeable.

To solve the problem let's postpone the reclaim of inodes, which have more than 1 attached page. Let's wait until the pagecache pages will be evicted naturally by scanning the corresponding LRU lists, and only then reclaim the inode structure.

Reported-by Spock <[email protected]>

Signed-off-by Roman Gushchin <[email protected]>

Cc Michal Hocko <[email protected]>

Cc Rik van Riel <[email protected]>

Cc Randy Dunlap <[email protected]>

Cc Andrew Morton <[email protected]>

---
 fs/inode.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

fs/inode.c +0 -0

index 73432e64f874..0cd47fe0dbe5 100644

fs/inode.c +0 -0

 		inode->i_state &= ~I_REFERENCED;
 		spin_unlock(&inode->i_lock);
 		return LRU_ROTATE;

Signature · 1 line

2.17.2

Spock doesn't seem to be cced here - fixed now

Quoted 17 lines · Roman Gushchin

Spock reported that the commit 172b06c32b94 ("mm: slowly shrink slabs with a relatively small number of objects") leads to a regression on his setup: periodically the majority of the pagecache is evicted without an obvious reason, while before the change the amount of free memory was balancing around the watermark.

The reason behind is that the mentioned above change created some minimal background pressure on the inode cache. The problem is that if an inode is considered to be reclaimed, all belonging pagecache page are stripped, no matter how many of them are there. So, if a huge multi-gigabyte file is cached in the memory, and the goal is to reclaim only few slab objects (unused inodes), we still can eventually evict all gigabytes of the pagecache at once.

The workload described by Spock has few large non-mapped files in the pagecache, so it's especially noticeable.

Quoted 4 lines

To solve the problem let's postpone the reclaim of inodes, which have more than 1 attached page. Let's wait until the pagecache pages will be evicted naturally by scanning the corresponding LRU lists, and only then reclaim the inode structure.

Has this actually fixed/worked around the issue?

Quoted 20 lines

Spock doesn't seem to be cced here - fixed now

Quoted 17 lines · Roman Gushchin

Spock reported that the commit 172b06c32b94 ("mm: slowly shrink slabs with a relatively small number of objects") leads to a regression on his setup: periodically the majority of the pagecache is evicted without an obvious reason, while before the change the amount of free memory was balancing around the watermark.

The reason behind is that the mentioned above change created some minimal background pressure on the inode cache. The problem is that if an inode is considered to be reclaimed, all belonging pagecache page are stripped, no matter how many of them are there. So, if a huge multi-gigabyte file is cached in the memory, and the goal is to reclaim only few slab objects (unused inodes), we still can eventually evict all gigabytes of the pagecache at once.

The workload described by Spock has few large non-mapped files in the pagecache, so it's especially noticeable.

Quoted 6 lines

Quoted 4 lines

To solve the problem let's postpone the reclaim of inodes, which have more than 1 attached page. Let's wait until the pagecache pages will be evicted naturally by scanning the corresponding LRU lists, and only then reclaim the inode structure.

Has this actually fixed/worked around the issue?

Spock wrote this earlier to me directly. I believe I can quote it here:

"Patch applied, looks good so far. System behaves like it was with pre-4.18.15 kernels.
Also tried to add some user-level tests to the geneic background activity, like
- stat'ing a bunch of files
- streamed read several large files at once on ext4 and XFS
- random reads on the whole collection with a read size of 16K

I will be monitoring while fragmentation stacks up and report back if something bad happens."

Spock, please let me know if you have any new results.

Thanks!

Spock doesn't seem to be cced here - fixed now

Quoted 14 lines · Roman Gushchin

Spock reported that the commit 172b06c32b94 ("mm: slowly shrink slabs with a relatively small number of objects") leads to a regression on his setup: periodically the majority of the pagecache is evicted without an obvious reason, while before the change the amount of free memory was balancing around the watermark.

The reason behind is that the mentioned above change created some minimal background pressure on the inode cache. The problem is that if an inode is considered to be reclaimed, all belonging pagecache page are stripped, no matter how many of them are there. So, if a huge multi-gigabyte file is cached in the memory, and the goal is to reclaim only few slab objects (unused inodes), we still can eventually evict all gigabytes of the pagecache at once.

Quoted 7 lines

The workload described by Spock has few large non-mapped files in the pagecache, so it's especially noticeable.

To solve the problem let's postpone the reclaim of inodes, which have more than 1 attached page. Let's wait until the pagecache pages will be evicted naturally by scanning the corresponding LRU lists, and only then reclaim the inode structure.

Has this actually fixed/worked around the issue?

Quoted 18 lines

Reported-by Spock <[email protected]>

Signed-off-by Roman Gushchin <[email protected]>

Cc Michal Hocko <[email protected]>

Cc Rik van Riel <[email protected]>

Cc Randy Dunlap <[email protected]>

Cc Andrew Morton <[email protected]>

---
 fs/inode.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

fs/inode.c +0 -0

index 73432e64f874..0cd47fe0dbe5 100644
@@ -730,8 +730,11 @@ static enum lru_status inode_lru_isolate(struct list_head *item,
 		return LRU_REMOVED;
 	}
 

fs/inode.c static enum lru_status inode_lru_isolate(struct list_head *item, · line 737

Quoted 7 lines

fs/inode.c +5 -2

-	/* recently referenced inodes get one more pass */
-	if (inode->i_state & I_REFERENCED) {
+	/*
+	 * Recently referenced inodes and inodes with many attached pages
+	 * get one more pass.
+	 */
+	if (inode->i_state & I_REFERENCED || inode->i_data.nrpages > 1) {

The comment is just confusing. Did you mean to say s@many@any@ ?