RFC memcg: effective high limit that avoids isolation issues
Get series with b4
b4 am [email protected]Versions newest first
- v1 RFC Latest7 patches 0 repliesNew
Patches in v1 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/7 | memcg: add helpers to handle memory.high overage in long operationsinclude/ mm/ | +35 | 0 replies | |
| 2/7 | mm/gup: handle memory.high overage while populatingmm/ mm/ mm/ | +43 | 0 replies | |
| 3/7 | mm, fs: handle memory.high overage in buffered reads and writesfs/ fs/ mm/ | +49 | 0 replies | |
| 4/7 | mm/shmem: handle memory.high overage in fallocate and readsmm/ | +34 | 0 replies | |
| 5/7 | mm/khugepaged: handle memory.high overage in MADV_COLLAPSEmm/ | +8 | 0 replies | |
| 6/7 | memcg: remove synchronous memory.high enforcementDocumentation/ mm/ | +16 | 0 replies | |
| 7/7 | selftests/cgroup: make test_memcg_high_sync check that throttling holdstools/ | +32 | 0 replies |
Reviews trailers per patch
No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.
Cover letter [RFC PATCH 0/7] memcg: effective high limit that avoids isolation issues
(My initial plan [1, 2] was to track kernel lock holders, let them bypass memcg limits and throttle them on unlock or on return to userspace. I was asked to look for alternative solutions, and this series is one.)
memcg limits can cause isolation problems. A task stuck in limit enforcement in the charge path keeps holding its locks, and tasks in other cgroups that need them wait for a limit that has nothing to do with them. btrfs, for example, charges its tree blocks to the root cgroup for this reason [3].
We see this in Meta's fleet. Over 28 days, on several tens of thousands of unique hosts, a bpftrace script [4] recorded sleeping locks held in memcg reclaim while another task waited for them. Most belong to the reclaiming workload, like its mmap_lock (680,579 times, up to 4.7s), but some are shared by the whole machine or a whole filesystem:
- cgroup_mutex, used by every cgroup change. mkdir charges the new memcg to the parent while holding it, and reclaims if the parent is at its limit: 350 times, up to 2.5s.
- kernfs_rwsem, used by cgroupfs lookups. The kernfs notification worker held it while queueing inotify events charged to the watcher's cgroup: 2,324 times, up to 3.3s.
- btrfs tree locks, used by every b-tree search and change. On kernels with [3], the XArray nodes that index tree blocks are still charged to the task, so tasks reclaimed with tree locks held 2,853 times, 850 of them with a root node locked, up to 2.6s.
Hung-task reports show worse cases. A container's systemd created a cgroup while its own cgroup was over memory.high, and mkdir ran memory.high reclaim with cgroup_mutex and the parent directory's lock held for over 30 minutes. The host's systemd, tasks forking into cgroups, oomd and a monitoring agent all waited, so the userspace OOM killer could not even read the cgroup tree.
memory.max is a hard limit, so this series does not change it. memory.high doesn't have to be enforced in the charge path: since commit b23afb93d317 ("memcg: punt high overage reclaim to return-to-userland path"), the charge only records the overage (usage above the limit), and the task reclaims and is throttled on return to userspace, where it holds no locks. Avoiding this priority inversion was one of the reasons for that commit.
But return to userspace is too late when one system call charges a lot, like mlock() of a large range. So commit c9afe31ec443 ("memcg:
synchronously enforce memory.high for large overcharges") handles the overage in the charge itself when the charge can block and more than a batch is pending. That brought the problem back: the task reclaims and sleeps, up to 2s per call, under whatever locks it holds. It also misses THP charges outside MADV_HUGEPAGE ranges, which can't block under the default defrag setting. RESULTS shows both problems.
This series is a first step in rethinking the memcg high limit around isolation. It makes memory.high enforcement asynchronous (to allocation and charging) again and keeps it effective for large calls. The task handles the overage on return to userspace, and also inside the operations that can charge a lot in one call, where it holds no locks or only the locks of the file it works on:
- Populating memory (mlock, mlockall, MAP_POPULATE, MAP_LOCKED, MADV_POPULATE_*): 2M at a time, with mmap_lock dropped in between.
- tmpfs fallocate, buffered writes (generic_perform_write(), iomap, btrfs) and reads into user memory (filemap_read(), shmem): once per folio or range.
- MADV_COLLAPSE: after each collapse, with mmap_lock dropped.
These points also throttle THP charges, which the synchronous step skips. A file's lock only blocks tasks using that file. Two cases hold
more: splice() from a pipe also holds the pipe's lock, and gfs2 holds its allocation glocks in writes that allocate. The synchronous step sleeps with these locks today too.
Why not keep the synchronous step, with a higher threshold, without its sleep, or only outside these operations? It would still reclaim or sleep under locks, like mmap_lock in THP faults and invalidate_lock in readahead, and it would not close the THP gap.
This also lets users drop memory.max, which still reclaims in the charge path and can stall lock holders as above. They can set only memory.high and let a userspace OOM killer like oomd [5], or a future BPF OOM policy [6], decide when to kill. Meta plans to do this.
The last two patches remove the synchronous step, update the memory.high documentation and fix test_memcg_high_sync, which missed the THP gap.
RESULTS
Tests run in a VM with 4 CPUs and THP on for anonymous memory and shmem (enabled=always, shmem_enabled=always, tmpfs huge=always, the default defrag=madvise), in a cgroup with memory.high=64M. Reads go into an untouched buffer. "No hooks" is the base plus only the patch that removes the synchronous step. Usage 18s into a 512M operation, without swap:
test base series no hooks
mlock, mlockall, MAP_POPULATE, 513-514M 84M 513-514M
MADV_POPULATE_WRITE
tmpfs fallocate and write 512-513M 84M 512-513M
reads of a tmpfs file 513M 84-85M 513M
MADV_COLLAPSE of a sparse range 96M 98M 514M
(one VMA, or one VMA per PMD)
reads of /dev/zero or /dev/urandom 513M 513M 513M
(known gaps)
Usage grows by one 2M folio per throttle, so it is 84-85M after 18s. On ext4, XFS and btrfs, 256M writes stay at 64-67M on both kernels (257-264M with no hooks), and reads with a cold page cache at 70-73M. Reads with a warm page cache, where only the buffer is charged, reach 257M on base and stay at 84-85M with the series.
With swap, the MAP_POPULATE, MADV_POPULATE_WRITE and tmpfs write tests finish in under 2s at 65-66M with the series (base: 157-169M, no hooks:
513M). tmpfs fallocate is throttled with or without swap. With swap, base fails it with -ENOMEM for small pages.
During mlock() of an 80M MADV_HUGEPAGE range, a task in another cgroup that reads the process's /proc/<pid>/cmdline every 10ms waits up to 15.3s on base and under 1ms with the series. Populating 2G with no limit takes the same time on both kernels.
All patches build with gcc and clang (W=1), with CONFIG_MEMCG=n and with btrfs as a module. A lockdep kernel showed no warnings. The memcontrol selftests pass, and the fixed test_memcg_high_sync fails on base.
KNOWN GAPS
Other paths that charge a lot in one call are now throttled only on return to userspace, and memory.max still limits them. Most can get a throttle point later, some after dropping a lock:
- /dev/zero and /dev/urandom reads, getrandom(), sendfile() to a socket, io_uring submissions, netfs and FUSE write-through writes
- MADV_WILLNEED, UFFDIO_COPY, KSM unmerge, mprotect() of mlocked mappings
- FIDEDUPERANGE, FS_IOC_ENABLE_VERITY, KVM_PRE_FAULT_MEMORY, guest_memfd fallocate, KVM page splitting
Rarer ones, like sendmmsg(), btrfs send and FITRIM, can wait. Memory pinning (VFIO, iommufd, RDMA, io_uring buffers, direct I/O and others), fork() and single large allocations (BPF maps and others) hold a lock for the whole call or can't be split, so they stay as they are.
[1] https://lore.kernel.org/[email protected]/
[2] https://lore.kernel.org/[email protected]/
[3] commit cf1dec76ba8a ("mm/filemap: add AS_KERNEL_FILE")
[4] https://github.com/bpftrace/user-tools/tree/master/reclaimlockmonitor
[5] https://github.com/facebookincubator/oomd
[6] https://lore.kernel.org/[email protected]
Based on mm-everything-2026-11-20-06-34.
Shakeel Butt (7):
memcg: add helpers to handle memory.high overage in long operations mm/gup: handle memory.high overage while populating
mm, fs: handle memory.high overage in buffered reads and writes mm/shmem: handle memory.high overage in fallocate and reads mm/khugepaged: handle memory.high overage in MADV_COLLAPSE
memcg: remove synchronous memory.high enforcement selftests/cgroup: make test_memcg_high_sync check that throttling
holds
Documentation/admin-guide/cgroup-v2.rst | 12 +++++- fs/btrfs/file.c | 9 +++++ fs/iomap/buffered-io.c | 9 +++++ include/linux/memcontrol.h | 34 ++++++++++++++++ mm/filemap.c | 31 +++++++++++++++ mm/gup.c | 9 ++++- mm/internal.h | 27 +++++++++++++ mm/khugepaged.c | 8 ++++ mm/madvise.c | 11 +++++- mm/memcontrol.c | 21 ++++------ mm/shmem.c | 37 ++++++++++++++++-- .../selftests/cgroup/test_memcontrol.c | 39 +++++++++++++++---- 12 files changed, 217 insertions(+), 30 deletions(-)
base-commit: b298839c2f77b60eea645a68d2245aa088463955
Signature · 1 line
2.53.1-Meta