mm: introduce memory.min

If a cgroup has no associated tasks, invoking the OOM killer won't help release any memory, so respecting the memory.min can lead to an infinite OOM loop or system stall.

Let's ignore memory.min of unpopulated cgroups.

Signed-off-by Roman Gushchin <[email protected]>

Cc Johannes Weiner <[email protected]>

Cc Michal Hocko <[email protected]>

Cc Vladimir Davydov <[email protected]>

Cc Tejun Heo <[email protected]>

Quoted 11 lines · Roman Gushchin

If a cgroup has no associated tasks, invoking the OOM killer won't help release any memory, so respecting the memory.min can lead to an infinite OOM loop or system stall.

Let's ignore memory.min of unpopulated cgroups.

Signed-off-by Roman Gushchin <[email protected]>

Cc Johannes Weiner <[email protected]>

Cc Michal Hocko <[email protected]>

Cc Vladimir Davydov <[email protected]>

Cc Tejun Heo <[email protected]>

Acked-by Johannes Weiner <[email protected]>

I wouldn't mind merging this into the previous patch. It's fairly small, and there is no reason to introduce an infinite OOM loop scenario into the tree, even if it's just for one commit.

Memory controller implements the memory.low best-effort memory protection mechanism, which works perfectly in many cases and allows protecting working sets of important workloads from sudden reclaim.

But its semantics has a significant limitation: it works only as long as there is a supply of reclaimable memory. This makes it pretty useless against any sort of slow memory leaks or memory usage increases. This is especially true for swapless systems. If swap is enabled, memory soft protection effectively postpones problems, allowing a leaking application to fill all swap area, which makes no sense.
The only effective way to guarantee the memory protection in this case is to invoke the OOM killer.

It's possible to handle this case in userspace by reacting on MEMCG_LOW events; but there is still a place for a fail-safe in-kernel mechanism to provide stronger guarantees.

This patch introduces the memory.min interface for cgroup v2 memory controller. It works very similarly to memory.low (sharing the same hierarchical behavior), except that it's not disabled if there is no more reclaimable memory in the system.

If cgroup is not populated, its memory.min is ignored, because otherwise even the OOM killer wouldn't be able to reclaim the protected memory, and the system can stall.

...

Documentation/cgroup-v2.txt +12 -0

@@ -1002,6 +1002,29 @@ PAGE_SIZE multiple when read back.
 	The total amount of memory currently being used by the cgroup
 	and its descendants.
 
+  memory.min
+	A read-write single value file which exists on non-root
+	cgroups.  The default is "0".
+
+	Hard memory protection.  If the memory usage of a cgroup
+	is within its effective min boundary, the cgroup's memory
+	won't be reclaimed under any conditions. If there is no
+	unprotected reclaimable memory available, OOM killer
+	is invoked.
+
+	Effective low boundary is limited by memory.min values of
+	all ancestor cgroups. If there is memory.min overcommitment

Quoted 51 lines

Memory controller implements the memory.low best-effort memory protection mechanism, which works perfectly in many cases and allows protecting working sets of important workloads from sudden reclaim.

But its semantics has a significant limitation: it works only as long as there is a supply of reclaimable memory. This makes it pretty useless against any sort of slow memory leaks or memory usage increases. This is especially true for swapless systems. If swap is enabled, memory soft protection effectively postpones problems, allowing a leaking application to fill all swap area, which makes no sense.
The only effective way to guarantee the memory protection in this case is to invoke the OOM killer.

It's possible to handle this case in userspace by reacting on MEMCG_LOW events; but there is still a place for a fail-safe in-kernel mechanism to provide stronger guarantees.

This patch introduces the memory.min interface for cgroup v2 memory controller. It works very similarly to memory.low (sharing the same hierarchical behavior), except that it's not disabled if there is no more reclaimable memory in the system.

If cgroup is not populated, its memory.min is ignored, because otherwise even the OOM killer wouldn't be able to reclaim the protected memory, and the system can stall.

...

Documentation/cgroup-v2.txt +14 -0

@@ -1002,6 +1002,29 @@ PAGE_SIZE multiple when read back.
 	The total amount of memory currently being used by the cgroup
 	and its descendants.
 
+  memory.min
+	A read-write single value file which exists on non-root
+	cgroups.  The default is "0".
+
+	Hard memory protection.  If the memory usage of a cgroup
+	is within its effective min boundary, the cgroup's memory
+	won't be reclaimed under any conditions. If there is no
+	unprotected reclaimable memory available, OOM killer
+	is invoked.
+
+	Effective low boundary is limited by memory.min values of
+	all ancestor cgroups. If there is memory.min overcommitment
+	(child cgroup or cgroups are requiring more protected memory
+	than parent will allow), then each child cgroup will get

Documentation/cgroup-v2.txt PAGE_SIZE multiple when read back. · line 1026

Quoted 8 lines

Documentation/cgroup-v2.txt +8 -0

+	the part of parent's protection proportional to its
+	actual memory usage below memory.min.
+
+	Putting more memory than generally available under this
+	protection is discouraged and may lead to constant OOMs.
+
+	If a memory cgroup is not populated with processes,
+	its memory.min is ignored.

This is a copy-paste-edit of the memory.low description. Could we please carefully check that it all remains accurate? Should "Effective low boundary" be "Effective min boundary"? Does overcommit still apply to .min? etcetera.

Memory controller implements the memory.low best-effort memory protection mechanism, which works perfectly in many cases and allows protecting working sets of important workloads from sudden reclaim.

But its semantics has a significant limitation: it works only as long as there is a supply of reclaimable memory. This makes it pretty useless against any sort of slow memory leaks or memory usage increases. This is especially true for swapless systems. If swap is enabled, memory soft protection effectively postpones problems, allowing a leaking application to fill all swap area, which makes no sense.
The only effective way to guarantee the memory protection in this case is to invoke the OOM killer.

It's possible to handle this case in userspace by reacting on MEMCG_LOW events; but there is still a place for a fail-safe in-kernel mechanism to provide stronger guarantees.

This patch introduces the memory.min interface for cgroup v2 memory controller. It works very similarly to memory.low (sharing the same hierarchical behavior), except that it's not disabled if there is no more reclaimable memory in the system.

Quoted 19 lines · Roman Gushchin

Memory controller implements the memory.low best-effort memory protection mechanism, which works perfectly in many cases and allows protecting working sets of important workloads from sudden reclaim.

But its semantics has a significant limitation: it works only as long as there is a supply of reclaimable memory. This makes it pretty useless against any sort of slow memory leaks or memory usage increases. This is especially true for swapless systems. If swap is enabled, memory soft protection effectively postpones problems, allowing a leaking application to fill all swap area, which makes no sense.
The only effective way to guarantee the memory protection in this case is to invoke the OOM killer.

It's possible to handle this case in userspace by reacting on MEMCG_LOW events; but there is still a place for a fail-safe in-kernel mechanism to provide stronger guarantees.

Quoted 4 lines

This patch introduces the memory.min interface for cgroup v2 memory controller. It works very similarly to memory.low (sharing the same hierarchical behavior), except that it's not disabled if there is no more reclaimable memory in the system.

Originally I was pushing for the hard guarantee before we landed with the best effort one. The assumption back then was that properly configured systems shouldn't see problems IIRC.

It is not entirely clear to me what is the role of the low limit wrt. leaking application from the above description TBH. I presume you have a process without any low&hard limit which leaks and basically breaks the low limit expectation because of the lack of reclaimable memory and our memcg_low_reclaim fallback.

If that is the case then the hard limit should indeed protect the respective memcg from reclaim. But what is the actuall guarantee? We can reclaim that memory by the OOM killer, because there is no protection from killing a memcg under the min limit. So what is the actual semantic?

Also how is an admin supposed to configure those limits? low limit doesn't reall protect in some cases so why should it be used at all? I see how min matches max and low matches high, so there is a nice symmetry but aren't we adding additional complexity to the API? Isn't the real problem that the other guy (leaking application) doesn't have any cap?