Skip to contentSkip to conversation

cgroup: Show # of subsystem CSSes in cgroup.stat

PATCH Superseded Waiman Long 2 participants 2 messages Jul 11, 2024

Subsystems (2)

v5 of 7 Superseded Latest is v7 →

lore
Share & tools
lore
#0 lore raw

PATCH v5 1/1 cgroup: Show # of subsystem CSSes in cgroup.stat

Cgroup subsystem state (CSS) is an abstraction in the cgroup layer to help manage different structures in various cgroup subsystems by being an embedded element inside a larger structure like cpuset or mem_cgroup.

The /proc/cgroups file shows the number of cgroups for each of the subsystems. With cgroup v1, the number of CSSes is the same as the number of cgroups. That is not the case anymore with cgroup v2. The /proc/cgroups file cannot show the actual number of CSSes for the subsystems that are bound to cgroup v2.

So if a v2 cgroup subsystem is leaking cgroups (usually memory cgroup), we can't tell by looking at /proc/cgroups which cgroup subsystems may be responsible.

As cgroup v2 had deprecated the use of /proc/cgroups, the hierarchical cgroup.stat file is now being extended to show the number of live and dying CSSes associated with all the non-inhibited cgroup subsystems that have been bound to cgroup v2 as long as it is not zero. The number includes CSSes in the current cgroup as well as in all the descendants underneath it. This will help us pinpoint which subsystems are responsible for the increasing number of dying (nr_dying_descendants) cgroups.

The cgroup-v2.rst file is updated to discuss this new behavior.

With this patch applied, a sample output from root cgroup.stat file was shown below.

	nr_descendants 55
	nr_dying_descendants 35
	nr_subsys_cpuset 1
	nr_subsys_cpu 40
	nr_subsys_io 40
	nr_subsys_memory 55
	nr_dying_subsys_memory 35
	nr_subsys_perf_event 56
	nr_subsys_hugetlb 1
	nr_subsys_pids 55
	nr_subsys_rdma 1
	nr_subsys_misc 1

Another sample output from system.slice/cgroup.stat was:

	nr_descendants 32
	nr_dying_descendants 33
	nr_subsys_cpu 30
	nr_subsys_io 30
	nr_subsys_memory 32
	nr_dying_subsys_memory 33
	nr_subsys_perf_event 33
	nr_subsys_pids 32
---
 Documentation/admin-guide/cgroup-v2.rst | 14 ++++++--
 include/linux/cgroup-defs.h             |  7 ++++
 kernel/cgroup/cgroup.c                  | 47 ++++++++++++++++++++++++-
 3 files changed, 65 insertions(+), 3 deletions(-)
Documentation/admin-guide/cgroup-v2.rst +12 -2
index 52763d6b2919..356cd430c888 100644
@@ -981,6 +981,16 @@ All cgroup core files are prefixed with "cgroup."
 		A dying cgroup can consume system resources not exceeding
 		limits, which were active at the moment of cgroup deletion.
 
+	  nr_subsys_<cgroup_subsys>
+		Total number of live cgroups associated with that cgroup
+		subsystem (e.g. memory) at and beneath the current
+		cgroup.  An entry will only be shown if it is not zero.
+
+	  nr_dying_subsys_<cgroup_subsys>
+		Total number of dying cgroups associated with that cgroup
+		subsystem (e.g. memory) beneath the current cgroup.
+		An entry will only be shown if it is not zero.
+
   cgroup.freeze
 	A read-write single value file which exists on non-root cgroups.
 	Allowed values are "0" and "1". The default is "0".
@@ -2930,8 +2940,8 @@ Deprecated v1 Core Features
 
 - "cgroup.clone_children" is removed.
 
-- /proc/cgroups is meaningless for v2.  Use "cgroup.controllers" file
-  at the root instead.
+- /proc/cgroups is meaningless for v2.  Use "cgroup.controllers" or
+  "cgroup.stat" files at the root instead.
 
 
 Issues with v1 and Rationales for v2
include/linux/cgroup-defs.h +7 -0
index b36690ca0d3f..62de18874508 100644
@@ -210,6 +210,13 @@ struct cgroup_subsys_state {
 	 * fields of the containing structure.
 	 */
 	struct cgroup_subsys_state *parent;
+
+	/*
+	 * Keep track of total numbers of visible and dying descendant CSSes.
+	 * Protected by cgroup_mutex.
+	 */
+	int nr_descendants;
+	int nr_dying_descendants;
 };
 
 /*
kernel/cgroup/cgroup.c +46 -1

84 lines not on this page · raw message

Signature · 1 line
2.39.3
Reviewed-by on v5 1/1
#1 lore raw
Quoted 52 lines · Waiman Long

Cgroup subsystem state (CSS) is an abstraction in the cgroup layer to help manage different structures in various cgroup subsystems by being an embedded element inside a larger structure like cpuset or mem_cgroup.

The /proc/cgroups file shows the number of cgroups for each of the subsystems. With cgroup v1, the number of CSSes is the same as the number of cgroups. That is not the case anymore with cgroup v2. The /proc/cgroups file cannot show the actual number of CSSes for the subsystems that are bound to cgroup v2.

So if a v2 cgroup subsystem is leaking cgroups (usually memory cgroup), we can't tell by looking at /proc/cgroups which cgroup subsystems may be responsible.

As cgroup v2 had deprecated the use of /proc/cgroups, the hierarchical cgroup.stat file is now being extended to show the number of live and dying CSSes associated with all the non-inhibited cgroup subsystems that have been bound to cgroup v2 as long as it is not zero. The number includes CSSes in the current cgroup as well as in all the descendants underneath it. This will help us pinpoint which subsystems are responsible for the increasing number of dying (nr_dying_descendants) cgroups.

The cgroup-v2.rst file is updated to discuss this new behavior.

With this patch applied, a sample output from root cgroup.stat file was shown below.

	nr_descendants 55
	nr_dying_descendants 35
	nr_subsys_cpuset 1
	nr_subsys_cpu 40
	nr_subsys_io 40
	nr_subsys_memory 55
	nr_dying_subsys_memory 35
	nr_subsys_perf_event 56
	nr_subsys_hugetlb 1
	nr_subsys_pids 55
	nr_subsys_rdma 1
	nr_subsys_misc 1

Another sample output from system.slice/cgroup.stat was:

	nr_descendants 32
	nr_dying_descendants 33
	nr_subsys_cpu 30
	nr_subsys_io 30
	nr_subsys_memory 32
	nr_dying_subsys_memory 33
	nr_subsys_perf_event 33
	nr_subsys_pids 32
Quoted 2 lines
Signature · 2 lines
Thanks,
Kamalesh