mm: skip memcg for certain address space

Recently I'm hitting soft lockup if adding an order 2 folio to a filemap using GFP_NOFS | __GFP_NOFAIL. The softlockup happens at memcg charge code, and I guess that's exactly what __GFP_NOFAIL is expected to do, wait indefinitely until the request can be met.

On the other hand, if we do not use __GFP_NOFAIL, we can be limited by memcg at a lot of critical location, and lead to unnecessary transaction abort just due to memcg limit.

However for that specific btrfs call site, there is really no need charge the memcg, as that address space belongs to btree inode, which is not accessible to any end user, and that btree inode is a shared pool for all metadata of a btrfs.

So this patchset introduces a new address space flag, AS_NO_MEMCG, so that folios added to that address space will not trigger any memcg charge.

This would be the basis for future btrfs changes, like removing __GFP_NOFAIL completely and larger metadata folios.

Qu Wenruo (2):
mm: make lru_gen_eviction() to handle folios without memcg info
mm: allow certain address space to be not accounted by memcg

  fs/btrfs/disk-io.c      |  1 +
  include/linux/pagemap.h |  1 +
  mm/filemap.c            | 12 +++++++++---
  mm/workingset.c         |  2 +-
  4 files changed, 12 insertions(+), 4 deletions(-)

Recently I'm hitting soft lockup if adding an order 2 folio to a filemap using GFP_NOFS | __GFP_NOFAIL. The softlockup happens at memcg charge code, and I guess that's exactly what __GFP_NOFAIL is expected to do, wait indefinitely until the request can be met.

On the other hand, if we do not use __GFP_NOFAIL, we can be limited by memcg at a lot of critical location, and lead to unnecessary transaction abort just due to memcg limit.

However for that specific btrfs call site, there is really no need charge the memcg, as that address space belongs to btree inode, which is not accessible to any end user, and that btree inode is a shared pool for all metadata of a btrfs.

So this patchset introduces a new address space flag, AS_NO_MEMCG, so that folios added to that address space will not trigger any memcg charge.

This would be the basis for future btrfs changes, like removing __GFP_NOFAIL completely and larger metadata folios.

Qu Wenruo (2):
mm: make lru_gen_eviction() to handle folios without memcg info
mm: allow certain address space to be not accounted by memcg

 fs/btrfs/disk-io.c      |  1 +
 include/linux/pagemap.h |  1 +
 mm/filemap.c            | 12 +++++++++---
 mm/workingset.c         |  2 +-
 4 files changed, 12 insertions(+), 4 deletions(-)

Hi,

you should have Ccd people according to get_maintainers script to get a reply faster. Let me Cc the MEMCG section.

Quoted 4 lines · Qu Wenruo

Recently I'm hitting soft lockup if adding an order 2 folio to a filemap using GFP_NOFS | __GFP_NOFAIL. The softlockup happens at memcg charge code, and I guess that's exactly what __GFP_NOFAIL is expected to do, wait indefinitely until the request can be met.

Seems like a bug to me, as the charging of __GFP_NOFAIL in try_charge_memcg() should proceed to the force: part AFAICS and just go over the limit.

I was suspecting mem_cgroup_oom() a bit earlier return true, causing the retry loop, due to GFP_NOFS. But it seems out_of_memory() should be specifically proceeding for GFP_NOFS if it's memcg oom. But I might be missing something else. Anyway we should know what exactly is going first.

Quoted 11 lines · Vlastimil Babka (SUSE)

Hi,

you should have Ccd people according to get_maintainers script to get a reply faster. Let me Cc the MEMCG section.

Quoted 4 lines · Qu Wenruo

Recently I'm hitting soft lockup if adding an order 2 folio to a filemap using GFP_NOFS | __GFP_NOFAIL. The softlockup happens at memcg charge code, and I guess that's exactly what __GFP_NOFAIL is expected to do, wait indefinitely until the request can be met.

Quoted 8 lines

Seems like a bug to me, as the charging of __GFP_NOFAIL in try_charge_memcg() should proceed to the force: part AFAICS and just go over the limit.

I was suspecting mem_cgroup_oom() a bit earlier return true, causing the retry loop, due to GFP_NOFS. But it seems out_of_memory() should be specifically proceeding for GFP_NOFS if it's memcg oom. But I might be missing something else. Anyway we should know what exactly is going first.

Correct. memcg oom code will invoke the memcg OOM killer for NOFS requests. See out_of_memory

         /*
          * The OOM killer does not compensate for IO-less reclaim.
          * But mem_cgroup_oom() has to invoke the OOM killer even
          * if it is a GFP_NOFS allocation.
          */
         if (!(oc->gfp_mask & __GFP_FS) && !is_memcg_oom(oc))
                 return true;

That means that there will be a victim killed, charges reclaimed and forward progress made. If there is no victim then the charging path will bail out and overcharge.

Also the reclaim should have cond_rescheds in the reclaim path. If that is not sufficient it should be fixed rather than workaround.

Quoted 36 lines · Michal Hocko

Quoted 11 lines · Vlastimil Babka (SUSE)

Hi,

you should have Ccd people according to get_maintainers script to get a reply faster. Let me Cc the MEMCG section.

Quoted 4 lines · Qu Wenruo

Recently I'm hitting soft lockup if adding an order 2 folio to a filemap using GFP_NOFS | __GFP_NOFAIL. The softlockup happens at memcg charge code, and I guess that's exactly what __GFP_NOFAIL is expected to do, wait indefinitely until the request can be met.

Quoted 8 lines

Seems like a bug to me, as the charging of __GFP_NOFAIL in try_charge_memcg() should proceed to the force: part AFAICS and just go over the limit.

I was suspecting mem_cgroup_oom() a bit earlier return true, causing the retry loop, due to GFP_NOFS. But it seems out_of_memory() should be specifically proceeding for GFP_NOFS if it's memcg oom. But I might be missing something else. Anyway we should know what exactly is going first.

Correct. memcg oom code will invoke the memcg OOM killer for NOFS requests. See out_of_memory

         /*
          * The OOM killer does not compensate for IO-less reclaim.
          * But mem_cgroup_oom() has to invoke the OOM killer even
          * if it is a GFP_NOFS allocation.
          */
         if (!(oc->gfp_mask & __GFP_FS) && !is_memcg_oom(oc))
                 return true;

That means that there will be a victim killed, charges reclaimed and forward progress made. If there is no victim then the charging path will bail out and overcharge.

Quoted 2 lines

Also the reclaim should have cond_rescheds in the reclaim path. If that is not sufficient it should be fixed rather than workaround.

Another question is, I only see this hang with larger folio (order 2 vs the old order 0) when adding to the same address space.

Does the folio order has anything related to the problem or just a higher order makes it more possible?

And finally, even without the hang problem, does it make any sense to skip all the possible memcg charge completely, either to reduce latency or just to reduce GFP_NOFAIL usage, for those user inaccessible inodes?

Thanks,
Qu

On 7/18/24 12:38 AM, Qu Wenruo wrote:

Quoted 2 lines

Does the folio order has anything related to the problem or just a higher order makes it more possible?

I didn't spot anything in the memcg charge path that would depend on the order directly, hm. Also what kernel version was showing these soft lockups?

Quoted 3 lines

And finally, even without the hang problem, does it make any sense to skip all the possible memcg charge completely, either to reduce latency or just to reduce GFP_NOFAIL usage, for those user inaccessible inodes?

Quoted 1 line · Vlastimil Babka (SUSE)

On 7/18/24 12:38 AM, Qu Wenruo wrote:

[...]

Quoted 3 lines

Quoted 2 lines

Does the folio order has anything related to the problem or just a higher order makes it more possible?

Quoted 2 lines

I didn't spot anything in the memcg charge path that would depend on the order directly, hm. Also what kernel version was showing these soft lockups?

Correct. Order just defines the number of charges to be reclaimed. Unlike the page allocator path we do not have any specific requirements on the memory to be released.

Quoted 4 lines

Quoted 4 lines

And finally, even without the hang problem, does it make any sense to skip all the possible memcg charge completely, either to reduce latency or just to reduce GFP_NOFAIL usage, for those user inaccessible inodes?

Let me just add to the pile of questions. Who does own this memory?

Quoted 9 lines · Michal Hocko

Quoted 1 line · Vlastimil Babka (SUSE)

On 7/18/24 12:38 AM, Qu Wenruo wrote:

[...]

Quoted 5 lines

Quoted 2 lines

Does the folio order has anything related to the problem or just a higher order makes it more possible?

I didn't spot anything in the memcg charge path that would depend on the order directly, hm. Also what kernel version was showing these soft lockups?

Quoted 3 lines

Correct. Order just defines the number of charges to be reclaimed. Unlike the page allocator path we do not have any specific requirements on the memory to be released.

So I guess the higher folio order just brings more pressure to trigger the problem?

Quoted 5 lines

Quoted 3 lines

Quoted 3 lines

And finally, even without the hang problem, does it make any sense to skip all the possible memcg charge completely, either to reduce latency or just to reduce GFP_NOFAIL usage, for those user inaccessible inodes?

Let me just add to the pile of questions. Who does own this memory?

A special inode inside btrfs, we call it btree_inode, which is not accessible out of the btrfs module, and its lifespan is the same as the mounted btrfs filesystem.

Quoted 12 lines · Qu Wenruo

Quoted 9 lines · Michal Hocko

Quoted 1 line · Vlastimil Babka (SUSE)

On 7/18/24 12:38 AM, Qu Wenruo wrote:

[...]

Quoted 5 lines

Quoted 2 lines

Does the folio order has anything related to the problem or just a higher order makes it more possible?

I didn't spot anything in the memcg charge path that would depend on the order directly, hm. Also what kernel version was showing these soft lockups?

Quoted 6 lines

Quoted 3 lines

Correct. Order just defines the number of charges to be reclaimed. Unlike the page allocator path we do not have any specific requirements on the memory to be released.

So I guess the higher folio order just brings more pressure to trigger the problem?

It increases the reclaim target (in number of pages to reclaim). That might contribute but we are cond_resched-ing in shrink_node_memcgs and also down the path in shrink_lruvec etc. So higher target shouldn't cause soft lockups unless we have a bug there - e.g. not triggering any of those paths with empty LRUs and looping somewhere. Not sure about MGLRU state of things TBH.

Quoted 9 lines

Quoted 5 lines

Quoted 3 lines

Quoted 3 lines

And finally, even without the hang problem, does it make any sense to skip all the possible memcg charge completely, either to reduce latency or just to reduce GFP_NOFAIL usage, for those user inaccessible inodes?

Let me just add to the pile of questions. Who does own this memory?

A special inode inside btrfs, we call it btree_inode, which is not accessible out of the btrfs module, and its lifespan is the same as the mounted btrfs filesystem.

But the memory charge is attributed to the caller unless you tell otherwise. So if this is really an internal use and you use a shared infrastructure which expects the current task to be owner of the charged memory then you need to wrap the initialization into set_active_memcg scope.

Quoted 12 lines · Qu Wenruo

Quoted 9 lines · Michal Hocko

Quoted 1 line · Vlastimil Babka (SUSE)

On 7/18/24 12:38 AM, Qu Wenruo wrote:

[...]

Quoted 5 lines

Quoted 2 lines

Does the folio order has anything related to the problem or just a higher order makes it more possible?

I didn't spot anything in the memcg charge path that would depend on the order directly, hm. Also what kernel version was showing these soft lockups?

Quoted 6 lines

Quoted 3 lines

Correct. Order just defines the number of charges to be reclaimed. Unlike the page allocator path we do not have any specific requirements on the memory to be released.

So I guess the higher folio order just brings more pressure to trigger the problem?

It increases the reclaim target (in number of pages to reclaim). That might contribute but we are cond_resched-ing in shrink_node_memcgs and also down the path in shrink_lruvec etc. So higher target shouldn't cause soft lockups unless we have a bug there - e.g. not triggering any of those paths with empty LRUs and looping somewhere. Not sure about MGLRU state of things TBH.

Quoted 4 lines

Quoted 4 lines

Quoted 3 lines

Quoted 3 lines

And finally, even without the hang problem, does it make any sense to skip all the possible memcg charge completely, either to reduce latency or just to reduce GFP_NOFAIL usage, for those user inaccessible inodes?

Quoted 5 lines

Quoted 1 line

Let me just add to the pile of questions. Who does own this memory?

A special inode inside btrfs, we call it btree_inode, which is not accessible out of the btrfs module, and its lifespan is the same as the mounted btrfs filesystem.

But the memory charge is attributed to the caller unless you tell otherwise.

So if this is really an internal use and you use a shared infrastructure which expects the current task to be owner of the charged memory then you need to wrap the initialization into set_active_memcg scope.

Quoted 36 lines · Michal Hocko

Quoted 12 lines · Qu Wenruo

Quoted 9 lines · Michal Hocko

Quoted 1 line · Vlastimil Babka (SUSE)

On 7/18/24 12:38 AM, Qu Wenruo wrote:

[...]

Quoted 5 lines

Quoted 2 lines

Does the folio order has anything related to the problem or just a higher order makes it more possible?

I didn't spot anything in the memcg charge path that would depend on the order directly, hm. Also what kernel version was showing these soft lockups?

Quoted 6 lines

Quoted 3 lines

Correct. Order just defines the number of charges to be reclaimed. Unlike the page allocator path we do not have any specific requirements on the memory to be released.

So I guess the higher folio order just brings more pressure to trigger the problem?

It increases the reclaim target (in number of pages to reclaim). That might contribute but we are cond_resched-ing in shrink_node_memcgs and also down the path in shrink_lruvec etc. So higher target shouldn't cause soft lockups unless we have a bug there - e.g. not triggering any of those paths with empty LRUs and looping somewhere. Not sure about MGLRU state of things TBH.

Quoted 9 lines

Quoted 5 lines

Quoted 3 lines

Quoted 3 lines

And finally, even without the hang problem, does it make any sense to skip all the possible memcg charge completely, either to reduce latency or just to reduce GFP_NOFAIL usage, for those user inaccessible inodes?

Let me just add to the pile of questions. Who does own this memory?

A special inode inside btrfs, we call it btree_inode, which is not accessible out of the btrfs module, and its lifespan is the same as the mounted btrfs filesystem.

Quoted 2 lines

But the memory charge is attributed to the caller unless you tell otherwise.

By the caller, did you mean the user space program who triggered the filesystem operations?

Quoted 5 lines

So if this is really an internal use and you use a shared infrastructure which expects the current task to be owner of the charged memory then you need to wrap the initialization into set_active_memcg scope.

And for root cgroup I guess it means we will have no memory limits or whatever, and filemap_add_folio() should always success (except real -ENOMEM situations or -EEXIST error btrfs would handle)?

On 7/18/24 12:38 AM, Qu Wenruo wrote:

Quoted 5 lines

Another question is, I only see this hang with larger folio (order 2 vs the old order 0) when adding to the same address space.

Does the folio order has anything related to the problem or just a higher order makes it more possible?

I didn't spot anything in the memcg charge path that would depend on the order directly, hm. Also what kernel version was showing these soft lockups?

Quoted 4 lines

And finally, even without the hang problem, does it make any sense to skip all the possible memcg charge completely, either to reduce latency or just to reduce GFP_NOFAIL usage, for those user inaccessible inodes?

Is it common to even use the filemap code for such metadata that can't be really mapped to userspace?

How does it even interact with reclaim, do they
become part of the page cache and are scanned by reclaim together with data that is mapped?

How are the lru decisions handled if there are no references for PTE access bits. Or can they be even reclaimed, or because there may e.g. other open inodes pinning this metadata, the reclaim is impossible?

(sorry if the questions seem noob, I'm not that much familiar with the page cache side of mm)

Quoted 3 lines

Thanks,
Qu

Quoted 1 line · Vlastimil Babka (SUSE)

On 7/18/24 12:38 AM, Qu Wenruo wrote:

[...]

Quoted 8 lines

Quoted 5 lines

Another question is, I only see this hang with larger folio (order 2 vs the old order 0) when adding to the same address space.

Does the folio order has anything related to the problem or just a higher order makes it more possible?

I didn't spot anything in the memcg charge path that would depend on the order directly, hm. Also what kernel version was showing these soft lockups?

The previous rc kernel. IIRC it's v6.10-rc6.

But that needs extra btrfs patches, or btrfs are still only doing the order-0 allocation, then add the order-0 folio into the filemap.

The extra patch just direct btrfs to allocate an order 2 folio (matching the default 16K nodesize), then attach the folio to the metadata filemap.

With extra coding handling corner cases like different folio sizes etc.

Quoted 7 lines

Quoted 4 lines

And finally, even without the hang problem, does it make any sense to skip all the possible memcg charge completely, either to reduce latency or just to reduce GFP_NOFAIL usage, for those user inaccessible inodes?

Is it common to even use the filemap code for such metadata that can't be really mapped to userspace?

At least XFS/EXT4 doesn't use filemap to handle their metadata. One of the reason is, btrfs has pretty large metadata structure. Not only for the regular filesystem things, but also data checksum.

Even using the default CRC32C algo, it's 4 bytes per 4K data. Thus things can go crazy pretty easily, and that's the reason why btrfs is still sticking to the filemap solution.

Quoted 3 lines

How does it even interact with reclaim, do they
become part of the page cache and are scanned by reclaim together with data that is mapped?

Yes, it's handled just like all other filemaps, it's also using page cache, and all the lru/scanning things.

The major difference is, we only implement a small subset of the address operations:

- write
- release
- invalidate
- migrate
- dirty (debug only, otherwise falls back to filemap_dirty_folio())

Note there is no read operations, as it's btrfs itself triggering the metadata read, thus there is no read/readahead.
Thus we're in the full control of the page cache, e.g. determine the folio size to be added into the filemap.

The filemap infrastructure provides 2 good functionalities:

- (Page) Cache
So that we can easily determine if we really need to read from the disk, and this can save us a lot of random IOs.

- Reclaiming

And of course the page cache of the metadata inode won't be cloned/shared to any user accessible inode.

Quoted 3 lines

How are the lru decisions handled if there are no references for PTE access bits. Or can they be even reclaimed, or because there may e.g. other open inodes pinning this metadata, the reclaim is impossible?

If I understand it correctly, we have implemented release_folio() callback, which does the btrfs metadata checks to determine if we can release the current folio, and avoid releasing folios that's still under IO etc.

Quoted 3 lines

(sorry if the questions seem noob, I'm not that much familiar with the page cache side of mm)

No worry at all, I'm also a newbie on the whole mm part.

Thanks,
Qu

Quoted 4 lines

Quoted 3 lines

Thanks,
Qu

Quoted 16 lines · Qu Wenruo

Quoted 1 line · Vlastimil Babka (SUSE)

On 7/18/24 12:38 AM, Qu Wenruo wrote:

[...]

Quoted 8 lines

Quoted 5 lines

Another question is, I only see this hang with larger folio (order 2 vs the old order 0) when adding to the same address space.

Does the folio order has anything related to the problem or just a higher order makes it more possible?

I didn't spot anything in the memcg charge path that would depend on the order directly, hm. Also what kernel version was showing these soft lockups?

The previous rc kernel. IIRC it's v6.10-rc6.

Quoted 7 lines

But that needs extra btrfs patches, or btrfs are still only doing the order-0 allocation, then add the order-0 folio into the filemap.

The extra patch just direct btrfs to allocate an order 2 folio (matching the default 16K nodesize), then attach the folio to the metadata filemap.

With extra coding handling corner cases like different folio sizes etc.

Hm right, but the same code is triggered for high-order folios (at least for user mappable page cache) today by some filesystems AFAIK, so we should be seeing such lockups already? btrfs case might be special that it's for the internal node as you explain, but that makes no difference for filemap_add_folio(), right? Or is it the only user with GFP_NOFS? Also is that passed as gfp directly or are there some extra scoped gfp resctrictions involved? (memalloc_..._save()).

Quoted 4 lines

If I understand it correctly, we have implemented release_folio() callback, which does the btrfs metadata checks to determine if we can release the current folio, and avoid releasing folios that's still under IO etc.

I see, thanks. Sounds like there might be potentially some suboptimal handling in that the folio will appear inactive because there's no references that folio_check_references() can detect, unless there's some folio_mark_accessed() calls involved (I see some FGP_ACCESSED in btrfs so maybe that's fine enough) so reclaim could consider it often, only to be stopped by release_folio failing.

Quoted 14 lines

Quoted 4 lines

(sorry if the questions seem noob, I'm not that much familiar with the page cache side of mm)

No worry at all, I'm also a newbie on the whole mm part.

Thanks,
Qu

Quoted 4 lines

Quoted 3 lines

Thanks,
Qu

Quoted 12 lines · Vlastimil Babka (SUSE)

Quoted 10 lines · Qu Wenruo

The previous rc kernel. IIRC it's v6.10-rc6.

But that needs extra btrfs patches, or btrfs are still only doing the order-0 allocation, then add the order-0 folio into the filemap.

The extra patch just direct btrfs to allocate an order 2 folio (matching the default 16K nodesize), then attach the folio to the metadata filemap.

With extra coding handling corner cases like different folio sizes etc.

Quoted 7 lines

Hm right, but the same code is triggered for high-order folios (at least for user mappable page cache) today by some filesystems AFAIK, so we should be seeing such lockups already? btrfs case might be special that it's for the internal node as you explain, but that makes no difference for filemap_add_folio(), right? Or is it the only user with GFP_NOFS? Also is that passed as gfp directly or are there some extra scoped gfp resctrictions involved? (memalloc_..._save()).

I'm not sure about other fses, but for that hang case, it's very metadata heavy, and ALL folios for that btree inode filemap is in order 2, since we're always allocating the order folios using GFP_NOFAIL, and attaching that folio into the filemap using GFP_NOFAIL too.

Not sure if other fses can have such situation.

[...]

Quoted 11 lines

Quoted 4 lines

If I understand it correctly, we have implemented release_folio() callback, which does the btrfs metadata checks to determine if we can release the current folio, and avoid releasing folios that's still under IO etc.

I see, thanks. Sounds like there might be potentially some suboptimal handling in that the folio will appear inactive because there's no references that folio_check_references() can detect, unless there's some folio_mark_accessed() calls involved (I see some FGP_ACCESSED in btrfs so maybe that's fine enough) so reclaim could consider it often, only to be stopped by release_folio failing.

For the page accessed part, btrfs handles it by mark_extent_buffer_accessed() call, and it's called every time we try to grab an extent buffer structure (the structure used to represent a metadata block inside btrfs).

So the accessed flag part should be fine I guess?

Thanks,
Qu

Quoted 15 lines

Quoted 14 lines

Quoted 4 lines

(sorry if the questions seem noob, I'm not that much familiar with the page cache side of mm)

No worry at all, I'm also a newbie on the whole mm part.

Thanks,
Qu

Quoted 4 lines

Quoted 3 lines

Thanks,
Qu

Hi,

you should have Ccd people according to get_maintainers script to get a reply faster. Let me Cc the MEMCG section.

On 7/10/24 3:07 AM, Qu Wenruo wrote:

Quoted 4 lines

Recently I'm hitting soft lockup if adding an order 2 folio to a filemap using GFP_NOFS | __GFP_NOFAIL. The softlockup happens at memcg charge code, and I guess that's exactly what __GFP_NOFAIL is expected to do, wait indefinitely until the request can be met.

Seems like a bug to me, as the charging of __GFP_NOFAIL in try_charge_memcg() should proceed to the force: part AFAICS and just go over the limit.

I was suspecting mem_cgroup_oom() a bit earlier return true, causing the retry loop, due to GFP_NOFS. But it seems out_of_memory() should be specifically proceeding for GFP_NOFS if it's memcg oom. But I might be missing something else. Anyway we should know what exactly is going first.

Quoted 26 lines

On the other hand, if we do not use __GFP_NOFAIL, we can be limited by memcg at a lot of critical location, and lead to unnecessary transaction abort just due to memcg limit.

However for that specific btrfs call site, there is really no need charge the memcg, as that address space belongs to btree inode, which is not accessible to any end user, and that btree inode is a shared pool for all metadata of a btrfs.

So this patchset introduces a new address space flag, AS_NO_MEMCG, so that folios added to that address space will not trigger any memcg charge.

This would be the basis for future btrfs changes, like removing __GFP_NOFAIL completely and larger metadata folios.

Qu Wenruo (2):
mm: make lru_gen_eviction() to handle folios without memcg info
mm: allow certain address space to be not accounted by memcg

  fs/btrfs/disk-io.c      |  1 +
  include/linux/pagemap.h |  1 +
  mm/filemap.c            | 12 +++++++++---
  mm/workingset.c         |  2 +-
  4 files changed, 12 insertions(+), 4 deletions(-)