Skip to contentSkip to conversation

Docs/admin-guide/cgroup: update controller interface docs

PATCH In review Guopeng Zhang 2 participants 5 messages Jul 7

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover Docs/admin-guide/cgroup: update controller interface docs

From: Guopeng Zhang <[email protected]>

Hi,

This small series updates cgroup controller documentation to match the interfaces exposed by the current code.

Patch 1 drops a stale interface-file count from the misc controller documentation.

Patch 2 documents the RDMA v1 interface files that are already exposed by the controller and already documented for cgroup v2.

Patch 3 notes that the io.latency-specific depth, avg_lat and win io.stat fields are only emitted when blkcg_debug_stats is enabled, and qualifies the usage text that suggests using avg_lat to pick a latency target.

Thanks,
Guopeng

Guopeng Zhang (3):
Docs/admin-guide/cgroup-v2: drop stale misc interface file count Docs/admin-guide/cgroup-v1: document rdma.peak, rdma.events and
rdma.events.local
Docs/admin-guide/cgroup-v2: note blkcg_debug_stats gates io.latency
stats

 Documentation/admin-guide/cgroup-v1/rdma.rst | 66 ++++++++++++++++++++
 Documentation/admin-guide/cgroup-v2.rst      | 16 +++--
 2 files changed, 76 insertions(+), 6 deletions(-)
Signature · 1 line
2.43.0
#1 lore raw

PATCH 1/3 Docs/admin-guide/cgroup-v2: drop stale misc interface file count

From: Guopeng Zhang <[email protected]>

The Miscellaneous controller documentation states it "provides 3 interface files", but misc_cg_files[] actually registers six (max, current, peak, capacity, events, events.local). Drop the stale count and let the file list that follows speak for itself.

---
 Documentation/admin-guide/cgroup-v2.rst | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
Documentation/admin-guide/cgroup-v2.rst +2 -1
index b0bd591bc4bf..df3fe7a7c6b3 100644
@@ -2968,7 +2968,8 @@ include/linux/misc_cgroup.h.
 Misc Interface Files
 ~~~~~~~~~~~~~~~~~~~~
 
-Miscellaneous controller provides 3 interface files. If two misc resources (res_a and res_b) are registered then:
+Miscellaneous controller provides the following interface files. If two misc
+resources (res_a and res_b) are registered then:
 
   misc.capacity
         A read-only flat-keyed file shown only in the root cgroup.  It shows
Signature · 1 line
2.43.0
#2 lore raw

PATCH 2/3 Docs/admin-guide/cgroup-v1: document rdma.peak, rdma.events and rdma.events.local

In reply to Guopeng Zhang

From: Guopeng Zhang <[email protected]>

The v1 RDMA controller documentation only describes rdma.max and rdma.current, but the controller exposes three more files -- rdma.peak, rdma.events and rdma.events.local -- which are already documented for v2. Mirror the v2 wording so the v1 documentation matches the files actually visible on a v1 mount.

---
 Documentation/admin-guide/cgroup-v1/rdma.rst | 66 ++++++++++++++++++++
 1 file changed, 66 insertions(+)
Documentation/admin-guide/cgroup-v1/rdma.rst +66 -0
index e69369b7252e..abddf34d2667 100644
@@ -9,6 +9,7 @@ RDMA Controller
      1-2. Why RDMA controller needed?
      1-3. How is RDMA controller implemented?
    2. Usage Examples
+   3. RDMA Interface Files
 
 1. Overview
 ===========
@@ -115,3 +116,68 @@ Following resources can be accounted by rdma controller.
 (d) Delete resource limit::
 
 	echo mlx4_0 hca_handle=max hca_object=max > /sys/fs/cgroup/rdma/1/rdma.max
+
+3. RDMA Interface Files
+========================
+
+The following interface files are available in each non-root RDMA cgroup.
+
+  rdma.max
+	A read-write file which describes the configured resource limit
+	for an RDMA/IB device.  See the Usage Examples above.
+
+  rdma.current
+	A read-only file which describes the current resource usage.
+
+  rdma.peak
+	A read-only nested-keyed file which shows the historical high
+	watermark of resource usage per device since the cgroup was created.
+
+	An example for mlx4 and ocrdma device follows::
+
+	  mlx4_0 hca_handle=1 hca_object=20
+	  ocrdma1 hca_handle=0 hca_object=23
+
+  rdma.events
+	A read-only nested-keyed file which exists on non-root cgroups
+	and contains the following keys:
+
+	  max
+		The number of times a process in this cgroup or its
+		descendants attempted an RDMA resource allocation that
+		was rejected because a rdma.max limit in the subtree
+		was reached.  This is a hierarchical counter propagated
+		upward to all ancestor cgroups.  A value change in this
+		file generates a file modified event.
+
+	  alloc_fail
+		The number of RDMA resource allocation attempts that
+		originated in this cgroup or its descendants and failed
+		due to a rdma.max limit being reached.  This is a
+		hierarchical counter propagated upward.
+
+	An example for mlx4 device follows::
+
+	  mlx4_0 hca_handle.max=5 hca_handle.alloc_fail=3 hca_object.max=0 hca_object.alloc_fail=0
+
+  rdma.events.local
+	Similar to rdma.events but the fields are local to the cgroup,
+	i.e. not hierarchical.  The file modified event generated on this
+	file reflects only the local events.
+
+	The following nested keys are defined.
+
+	  max
+		The number of times a process in this cgroup or its
+		descendants attempted an RDMA resource allocation that
+		was rejected because this cgroup's own rdma.max limit
+		was reached.
+
+	  alloc_fail
+		The number of RDMA resource allocation attempts
+		originating from this cgroup that failed due to this
+		cgroup's or an ancestor's rdma.max limit.
+
+	An example for mlx4 device follows::
+
+	  mlx4_0 hca_handle.max=5 hca_handle.alloc_fail=0 hca_object.max=0 hca_object.alloc_fail=0
Signature · 1 line
2.43.0
#3 lore raw

PATCH 3/3 Docs/admin-guide/cgroup-v2: note blkcg_debug_stats gates io.latency stats

In reply to Guopeng Zhang

From: Guopeng Zhang <[email protected]>

The io.stat section says that enabling the io.latency controller exposes the depth, avg_lat and win stats in addition to the normal ones. However, these io.latency-specific stats are debug stats and are only emitted when the blkcg_debug_stats module parameter is enabled, which is disabled by default.

Make this explicit so users do not expect these fields to appear in io.stat by default, and qualify the usage text that suggests using avg_lat to pick an io.latency target.

---
 Documentation/admin-guide/cgroup-v2.rst | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)
Documentation/admin-guide/cgroup-v2.rst +8 -5
index df3fe7a7c6b3..0df15a672cf3 100644
@@ -2278,10 +2278,11 @@ groups D and F will influence each other.  Group G will influence nobody::
 So the ideal way to configure this is to set io.latency in groups A, B, and C.
 Generally you do not want to set a value lower than the latency your device
 supports.  Experiment to find the value that works best for your workload.
-Start at higher than the expected latency for your device and watch the
-avg_lat value in io.stat for your workload group to get an idea of the
-latency you see during normal operation.  Use the avg_lat value as a basis for
-your real setting, setting at 10-15% higher than the value in io.stat.
+Start at higher than the expected latency for your device and, with
+blkcg_debug_stats enabled, watch the avg_lat value in io.stat for your
+workload group to get an idea of the latency you see during normal operation.
+Use the avg_lat value as a basis for your real setting, setting at 10-15%
+higher than the value in io.stat.
 
 How IO Latency Throttling Works
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -2319,7 +2320,9 @@ IO Latency Interface Files
 
   io.stat
 	If the controller is enabled you will see extra stats in io.stat in
-	addition to the normal ones.
+	addition to the normal ones.  These debug stats are only emitted when
+	the blkcg_debug_stats module parameter is enabled (it is disabled by
+	default).
 
 	  depth
 		This is the current queue depth for the group.
Signature · 1 line
2.43.0
maintainer
#4 lore raw
In reply to Guopeng Zhang
Quoted 2 lines · Guopeng Zhang

This small series updates cgroup controller documentation to match the interfaces exposed by the current code.

Applied 1-3 to cgroup/for-7.2-fixes.

Thanks.

--
tejun