Skip to content

cgroup: improve cgroup_file_notify() scalability

Shakeel Butt cgroups linux-mm lkml netdev 2 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v2 Latest3 patches 1 replyIn review
  2. v13 patches 7 repliesSuperseded

Changes in v2 from the cover letter

- Second patch moves rate-limiting code out of lock

Patches in v2 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/3cgroup: reduce cgroup_file_kn_lock hold time in cgroup_file_notify()kernel/cgroup/cgroup.c+8 -10 replies
2/3cgroup: add lockless fast-path checks to cgroup_file_notify()kernel/cgroup/cgroup.c+17 -130 replies
3/3cgroup: replace global cgroup_file_kn_lock with per-cgroup_file lockinclude/linux/cgroup-defs.h kernel/cgroup/cgroup.c+9 -160 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v2 yet.

Read the v2 thread

Cover letter [PATCH v2 0/3] cgroup: improve cgroup_file_notify() scalability

Jakub Kicinski reported that after the commit d929525c2e30 ("memcg:
net: track network throttling due to memcg memory pressure"), in Meta fleet, the network intensive workloads started seeing extensive spinlock contention in the rx path in the function cgroup_file_notify().

cgroup_file_notify() uses a single global spinlock to protect the cfile->kn pointer across all cgroups. On systems with many cgroups under memory pressure, this becomes a bottleneck as multiple CPUs in reclaim call __memcg_memory_event(), which walks the cgroup hierarchy calling cgroup_file_notify() at each level -- all serialized on the same lock.

This series reduces the lock hold time by moving kernfs_notify() outside the critical section, adds lockless fast-path checks to avoid the lock entirely in common cases (torn-down files and rate-limited bursts), and replaces the global lock with a per-cgroup_file lock to eliminate cross-cgroup contention.

Changes since v1:
- Second patch moves rate-limiting code out of lock

Shakeel Butt (3):
cgroup: reduce cgroup_file_kn_lock hold time in cgroup_file_notify()
cgroup: add lockless fast-path checks to cgroup_file_notify()
cgroup: replace global cgroup_file_kn_lock with per-cgroup_file lock

 include/linux/cgroup-defs.h |  1 +
 kernel/cgroup/cgroup.c      | 53 ++++++++++++++++++++-----------------
 2 files changed, 29 insertions(+), 25 deletions(-)
Signature · 1 line
2.52.0