mm: read root memcg stats from global counters instead of flushing
Get series with b4
b4 am [email protected]Versions newest first · linked by author and title
- v3 Latest4 patches 4 repliesIn review
- v24 patches 4 repliesSuperseded
- v13 patches 10 repliesSuperseded
Patches in v3 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/4 | mm: zswap: fix shrinker count for the root memcgmm/ | +5 | 1 reply | |
| 2/4 | mm: memcg: read root lruvec stats from the node countersmm/ | +56 | 1 reply | |
| 3/4 | mm: workingset: skip the root memcg flush in workingset_test_recent()mm/ | +4 | 1 reply | |
| 4/4 | mm/damon/core: use node counters instead of flushing the root memcgmm/ | +6 | 1 reply |
Reviews trailers per patch
All 4 patches have a Reviewed-by or Acked-by.
| Patch | Yosry Ahmed | SJ Park | Nhat Pham | Johannes Weiner | Chris Down |
|---|---|---|---|---|---|
| 1/4 | A-b | R-b | A-b | R-b | R-b T-b |
| 2/4 | R-b | ||||
| 3/4 | R-b T-b | ||||
| 4/4 | R-b | R-b T-b |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 5 reviewers.
Cover letter [PATCH v3 0/4] mm: read root memcg stats from global counters instead of flushing
Reclaim, refault detection, the zswap shrinker, and DAMON quota goals all flush the memcg rstat stats before reading them. When the target is the root memcg (eg global reclaim), that flushes every cgroup in the system, which is expensive. Most of these flushes are ratelimited today, but the stats they read can be stale for seconds.
For the root memcg, the call sites can read global counters directly instead of going through rstat. Global counters are always up to date and don't need flushing. For stats that only track memory charged to a memcg (eg LRU sizes, workingset, rotation, and reclaim writeback counts), the root memcg's hierarchical value equals the node counter.
Of the excluded stats, kernel stack, secondary page tables and WORKINGSET_NODERECLAIM only differ from the node counters because accounted kernel memory allocated by tasks in the root cgroup isn't charged to the root memcg. If the root memcg's stats should include that memory, then those could read the node counters as well. This is left for a separate subsequent patchset though since that changes what the root memcg reports and needs more upstream consensus.
Three root flushes remain. Reclaim also reads the stats of the child memcgs it scans, and count_shadow_nodes() and filemap_cachestat() need the root memcg's own (local) stats or the stats of arbitrary memcgs, which means global counters can't be used as a substitute there. This will be addressed in a subsequent patchset, as well as getting rid of ratelimiting for most call sites.
On an 80-CPU machine running proactive reclaim on the root cgroup, a root flush took 69us on average with little cgroup activity and ~0.9ms with 128 memcgs repeatedly allocating and freeing memory (wall time, including waiting for the rstat lock). Reading a node counter doesn't add any cost: it took the same amount of time as reading the rstat counter (~150ns, which includes bpftrace probe overhead).
For more context on the background of this series, please see the discussion about memcg stats flushing in [1].
Changelog
---------
Changes since v2 [3]:
- Change from denylist (patch 2) to allowlist (Sashiko, me, Michal)
- Add SJ's reviewed-by from v1 for the damon patch, which I forgot to add to v2
Changes since v1 [2]:
- Read the node counters for the root memcg by default in lruvec_page_state()/lruvec_page_state_monotonic(), with explicit exclusions, instead of adding lruvec_lru_page_state() helpers that callers opt in to (Yosry). As a result, the root memcg's memory.numa_stat now also reports the node counters for the non-excluded stats
- Split the workingset_test_recent() flush skip out into its own patch (patch 3)
- The DAMON patch (now patch 4) only skips the flush, as lruvec_page_state() reads the node counters for the root memcg
- Rebased onto current mm-new
[1] https://lore.kernel.org/linux-fsdevel/CAJnrk1aHdBm4CofvKfx7B3H-uF2xRXZUQs47J_4xeRmRZ4hcZg@mail.gmail.com/
[2] https://lore.kernel.org/linux-mm/[email protected]/
[3] https://lore.kernel.org/linux-mm/[email protected]/
Joanne Koong (4):
mm: zswap: fix shrinker count for the root memcg
mm: memcg: read root lruvec stats from the node counters
mm: workingset: skip the root memcg flush in workingset_test_recent() mm/damon/core: use node counters instead of flushing the root memcg
mm/damon/core.c | 7 +++++- mm/memcontrol.c | 58 +++++++++++++++++++++++++++++++++++++++++++++++-- mm/workingset.c | 5 ++++- mm/zswap.c | 7 ++++-- 4 files changed, 71 insertions(+), 6 deletions(-)
base-commit: 31a696f9c37defab0f28f774e00ac7f135889ec3
Signature · 1 line
2.52.1