docs: cgroup: document empty-write behavior of memory limit knobs
Subsystems (3)
PATCH v2 1/1 docs: cgroup: document empty-write behavior of memory limit knobs
From: Tao Cui <[email protected]>
A maintenance script on a cluster wrote an unset variable into memory.max of a workload cgroup; the variable expanded to an empty string, the write succeeded, and the workload in the cgroup was OOM-killed. Nothing pointed back at the write, so it took quite some time to trace the OOM kills to that script.
The memory controller documentation does not say what an empty write does; the cpuset controller documents its empty-value semantics. The actual behavior is that the empty string is accepted as 0. Reproduced on a k8s cluster (v1.29, cgroup v2, two-container pod, 384M limit):
# LIMIT=
# echo "$LIMIT" > $CG/memory.max # echo $? 0
m6demo 0/2 OOMKilled 0
Memory cgroup out of memory: Killed process 339529 (sleep) ... anon-rss:32kB
State it where the interface files are introduced, alongside the existing notes on units and page rounding.
---
Changes since v1: document the empty-write behavior instead of rejecting it, the outcome of the review discussion at the Link:
below.
--- Documentation/admin-guide/cgroup-v2.rst | 4 ++++ 1 file changed, 4 insertions(+)
Documentation/admin-guide/cgroup-v2.rst +4 -0
@@ -1321,6 +1321,10 @@ All memory amounts are in bytes. If a value which is not aligned to PAGE_SIZE is written, the value may be rounded up to the closest PAGE_SIZE multiple when read back. +For the limit files described below, an empty or all-whitespace +write is accepted and sets the limit to 0. To disable a limit, +write "max"; to set it to zero explicitly, write "0". + memory.current A read-only single value file which exists on non-root cgroups.
Signature · 1 line
2.43.0
Quoted 21 lines · Tao Cui
From: Tao Cui <[email protected]>
A maintenance script on a cluster wrote an unset variable into memory.max of a workload cgroup; the variable expanded to an empty string, the write succeeded, and the workload in the cgroup was OOM-killed. Nothing pointed back at the write, so it took quite some time to trace the OOM kills to that script.
The memory controller documentation does not say what an empty write does; the cpuset controller documents its empty-value semantics. The actual behavior is that the empty string is accepted as 0. Reproduced on a k8s cluster (v1.29, cgroup v2, two-container pod, 384M limit):
# LIMIT=
# echo "$LIMIT" > $CG/memory.max # echo $? 0m6demo 0/2 OOMKilled 0
Quoted 7 lines
Memory cgroup out of memory: Killed process 339529 (sleep) ... anon-rss:32kB
State it where the interface files are introduced, alongside the existing notes on units and page rounding.
Thanks!
Quoted 28 lines
---
Changes since v1: document the empty-write behavior instead of rejecting it, the outcome of the review discussion at the Link:
below.--- Documentation/admin-guide/cgroup-v2.rst | 4 ++++ 1 file changed, 4 insertions(+)Documentation/admin-guide/cgroup-v2.rst +4
-0@@ -1321,6 +1321,10 @@ All memory amounts are in bytes. If a value which is not aligned to PAGE_SIZE is written, the value may be rounded up to the closest PAGE_SIZE multiple when read back. +For the limit files described below, an empty or all-whitespace +write is accepted and sets the limit to 0. To disable a limit, +write "max"; to set it to zero explicitly, write "0". + memory.current A read-only single value file which exists on non-root cgroups.Signature · 1 line
2.43.0
Signature · 2 lines
Michal Hocko SUSE Labs
Quoted 21 lines · Tao Cui
From: Tao Cui <[email protected]>
A maintenance script on a cluster wrote an unset variable into memory.max of a workload cgroup; the variable expanded to an empty string, the write succeeded, and the workload in the cgroup was OOM-killed. Nothing pointed back at the write, so it took quite some time to trace the OOM kills to that script.
The memory controller documentation does not say what an empty write does; the cpuset controller documents its empty-value semantics. The actual behavior is that the empty string is accepted as 0. Reproduced on a k8s cluster (v1.29, cgroup v2, two-container pod, 384M limit):
# LIMIT=
# echo "$LIMIT" > $CG/memory.max # echo $? 0m6demo 0/2 OOMKilled 0
Quoted 7 lines
Memory cgroup out of memory: Killed process 339529 (sleep) ... anon-rss:32kB
State it where the interface files are introduced, alongside the existing notes on units and page rounding.