memcg: initiate deprecation of v1 features
Subsystems (4)
Shakeel Butt
maintainerv2 cover memcg: initiate deprecation of v1 features
Let start the deprecation process of the memcg v1 features which we discussed during LSFMMBPF 2024 [1]. For now add the warnings to collect the information on how the current users are using these features. Next we will work on providing better alternatives in v2 (if needed) and fully deprecate these features.
Shakeel Butt (4):
memcg: initiate deprecation of v1 tcp accounting
memcg: initiate deprecation of v1 soft limit
memcg: initiate deprecation of oom_control
memcg: initiate deprecation of pressure_level
Changes since v1:
- Fix build (T.J. Mercier)
- Fix documentation
.../admin-guide/cgroup-v1/memory.rst | 32 +++++++++++++++---- mm/memcontrol-v1.c | 16 ++++++++++ 2 files changed, 42 insertions(+), 6 deletions(-)
Signature · 1 line
2.43.5
Shakeel Butt
maintainerPATCH v2 1/4 memcg: initiate deprecation of v1 tcp accounting
Memcg v1 provides opt-in TCP memory accounting feature. However it is mostly unused due to its performance impact on the network traffic. In v2, the TCP memory is accounted in the regular memory usage and is transparent to the users but they can observe the TCP memory usage through memcg stats.
Let's initiate the deprecation process of memcg v1's tcp accounting functionality and add warnings to gather if there are any users and if there are, collect how they are using it and plan to provide them better alternative in v2.
---
Changes since v1:
- Fix documentation
Documentation/admin-guide/cgroup-v1/memory.rst | 8 ++++++++ mm/memcontrol-v1.c | 3 +++ 2 files changed, 11 insertions(+)
Documentation/admin-guide/cgroup-v1/memory.rst +8 -0
@@ -105,10 +105,18 @@ Brief summary of control files. memory.kmem.max_usage_in_bytes show max kernel memory usage recorded memory.kmem.tcp.limit_in_bytes set/show hard limit for tcp buf memory + This knob is deprecated and shouldn't be + used. memory.kmem.tcp.usage_in_bytes show current tcp buf memory allocation + This knob is deprecated and shouldn't be + used. memory.kmem.tcp.failcnt show the number of tcp buf memory usage hits limits + This knob is deprecated and shouldn't be + used. memory.kmem.tcp.max_usage_in_bytes show max tcp buf memory usage recorded + This knob is deprecated and shouldn't be + used. ==================================== ========================================== 1. History
mm/memcontrol-v1.c +3 -0
@@ -2447,6 +2447,9 @@ static ssize_t mem_cgroup_write(struct kernfs_open_file *of, ret =0 ;break ;case _TCP: +pr_warn_once ("kmem.tcp.limit_in_bytes is deprecated and will be removed. " +"Please report your usecase to [email protected] if you " +"depend on this functionality.\n" ); ret =memcg_update_tcp_max (memcg, nr_pages);break ; }
Signature · 1 line
2.43.5
Shakeel Butt
maintainerPATCH v2 2/4 memcg: initiate deprecation of v1 soft limit
In reply to Shakeel ButtMemcg v1 provides soft limit functionality for the best effort memory sharing between multiple workloads on a system. It is usually triggered through kswapd and at the moment does not reclaim kernel memory.
Memcg v2 provide more straightforward best effort (memory.low) and hard protection (memory.min) functionalities. Let's initiate the deprecation of soft limit from v1 and gather if v2 needs something more to move the existing v1 users to v2 regarding soft limit.
---
Changes since v1:
- N/A
Documentation/admin-guide/cgroup-v1/memory.rst | 8 ++++++-- mm/memcontrol-v1.c | 3 +++ 2 files changed, 9 insertions(+), 2 deletions(-)
Documentation/admin-guide/cgroup-v1/memory.rst +6 -2
@@ -78,6 +78,8 @@ Brief summary of control files. memory.memsw.max_usage_in_bytes show max memory+Swap usage recorded memory.soft_limit_in_bytes set/show soft limit of memory usage This knob is not available on CONFIG_PREEMPT_RT systems. + This knob is deprecated and shouldn't be + used. memory.stat show various statistics memory.use_hierarchy set/show hierarchical account enabled This knob is deprecated and shouldn't be
@@ -701,8 +703,10 @@ For compatibility reasons writing 1 to memory.use_hierarchy will always pass:: # echo 1 > memory.use_hierarchy -7. Soft limits -============== +7. Soft limits (DEPRECATED) +=========================== + +THIS IS DEPRECATED! Soft limits allow for greater sharing of memory. The idea behind soft limits is to allow control groups to use as much of the memory as needed, provided
mm/memcontrol-v1.c +3 -0
@@ -2458,6 +2458,9 @@ static ssize_t mem_cgroup_write(struct kernfs_open_file *of,if (IS_ENABLED (CONFIG_PREEMPT_RT )) { ret = -EOPNOTSUPP ; }else { +pr_warn_once ("soft_limit_in_bytes is deprecated and will be removed. " +"Please report your usecase to [email protected] if you " +"depend on this functionality.\n" );WRITE_ONCE (memcg->soft_limit, nr_pages); ret =0 ; }
Signature · 1 line
2.43.5
Shakeel Butt
maintainerPATCH v2 3/4 memcg: initiate deprecation of oom_control
In reply to Shakeel ButtThe oom_control provides functionality to disable memcg oom-killer, notifications on oom-kill and reading the stats regarding oom-kills. This interface was mainly introduced to provide functionality for userspace oom-killers. However it is not robust enough and only supports OOM handling in the page fault path.
For v2, the users can use the combination of memory.events notifications and memory.high interface to provide userspace OOM-killing functionality. Let's start the deprecation process for v1 and gather the info on how the current users are using this interface and work on providing a more robust functionality in v2.
---
Changes since v1:
- Fix build (T.J. Mercier)
Documentation/admin-guide/cgroup-v1/memory.rst | 8 ++++++-- mm/memcontrol-v1.c | 7 +++++++ 2 files changed, 13 insertions(+), 2 deletions(-)
Documentation/admin-guide/cgroup-v1/memory.rst +6 -2
@@ -92,6 +92,8 @@ Brief summary of control files. This knob is deprecated and shouldn't be used. memory.oom_control set/show oom controls. + This knob is deprecated and shouldn't be + used. memory.numa_stat show the number of memory usage per numa node memory.kmem.limit_in_bytes Deprecated knob to set and read the kernel
@@ -846,8 +848,10 @@ It's applicable for root and non-root cgroup. .. _cgroup-v1-memory-oom-control: -10. OOM Control -=============== +10. OOM Control (DEPRECATED) +============================ + +THIS IS DEPRECATED! memory.oom_control file is for OOM notification and other controls.
mm/memcontrol-v1.c +7 -0
@@ -1907,6 +1907,9 @@ static ssize_t memcg_write_event_control(struct kernfs_open_file *of, event->register_event = mem_cgroup_usage_register_event; event->unregister_event = mem_cgroup_usage_unregister_event; }else if (!strcmp (name,"memory.oom_control" )) { +pr_warn_once ("oom_control is deprecated and will be removed. " +"Please report your usecase to [email protected]" +" if you depend on this functionality. \n" ); event->register_event = mem_cgroup_oom_register_event; event->unregister_event = mem_cgroup_oom_unregister_event; }else if (!strcmp (name,"memory.pressure_level" )) {
@@ -2754,6 +2757,10 @@ static int mem_cgroup_oom_control_write(struct cgroup_subsys_state *css, {struct mem_cgroup *memcg =mem_cgroup_from_css (css); +pr_warn_once ("oom_control is deprecated and will be removed. " +"Please report your usecase to [email protected] if you " +"depend on this functionality. \n" ); +/* cannot set to root cgroup and only 0 and 1 are allowed */ if (mem_cgroup_is_root (memcg) || !((val ==0 ) || (val ==1 )))return -EINVAL ;
Signature · 1 line
2.43.5
Shakeel Butt
maintainerPATCH v2 4/4 memcg: initiate deprecation of pressure_level
In reply to Shakeel ButtThe pressure_level in memcg v1 provides memory pressure notifications to the user space. At the moment it provides notifications for three levels of memory pressure i.e. low, medium and critical, which are defined based on internal memory reclaim implementation details. More specifically the ratio or scanned and reclaimed pages during a memory reclaim. However this is not robust as there are workloads with mostly unreclaimable user memory or kernel memory.
For v2, the users can use PSI for memory pressure status of the system or the cgroup. Let's start the deprecation process for pressure_level and add warnings to gather the info on how the current users are using this interface and how they can be used to PSI.
---
Changes since v1:
- Fix build (T.J. Mercier)
Documentation/admin-guide/cgroup-v1/memory.rst | 8 ++++++-- mm/memcontrol-v1.c | 3 +++ 2 files changed, 9 insertions(+), 2 deletions(-)
Documentation/admin-guide/cgroup-v1/memory.rst +6 -2
@@ -86,6 +86,8 @@ Brief summary of control files. used. memory.force_empty trigger forced page reclaim memory.pressure_level set memory pressure notifications + This knob is deprecated and shouldn't be + used. memory.swappiness set/show swappiness parameter of vmscan (See sysctl's vm.swappiness) memory.move_charge_at_immigrate set/show controls of moving charges
@@ -898,8 +900,10 @@ At reading, current status of OOM is shown. The number of processes belonging to this cgroup killed by any kind of OOM killer. -11. Memory Pressure -=================== +11. Memory Pressure (DEPRECATED) +================================ + +THIS IS DEPRECATED! The pressure level notifications can be used to monitor the memory allocation cost; based on the pressure, applications can implement
mm/memcontrol-v1.c +3 -0
@@ -1913,6 +1913,9 @@ static ssize_t memcg_write_event_control(struct kernfs_open_file *of, event->register_event = mem_cgroup_oom_register_event; event->unregister_event = mem_cgroup_oom_unregister_event; }else if (!strcmp (name,"memory.pressure_level" )) { +pr_warn_once ("pressure_level is deprecated and will be removed. " +"Please report your usecase to [email protected] " +"if you depend on this functionality. \n" ); event->register_event = vmpressure_register_event; event->unregister_event = vmpressure_unregister_event; }else if (!strcmp (name,"memory.memsw.usage_in_bytes" )) {
Signature · 1 line
2.43.5
Re: [PATCH v2 2/4] memcg: initiate deprecation of v1 soft limit
In reply to Shakeel ButtQuoted 6 lines · Shakeel Butt
Memcg v1 provides soft limit functionality for the best effort memory sharing between multiple workloads on a system. It is usually triggered through kswapd and at the moment does not reclaim kernel memory.
Memcg v2 provide more straightforward best effort (memory.low) and hard
"provides"
Quoted 5 lines
protection (memory.min) functionalities. Let's initiate the deprecation of soft limit from v1 and gather if v2 needs something more to move the existing v1 users to v2 regarding soft limit.
Signed-off-by Shakeel Butt <[email protected]>
Quoted 51 lines
---
Changes since v1:
- N/ADocumentation/admin-guide/cgroup-v1/memory.rst | 8 ++++++-- mm/memcontrol-v1.c | 3 +++ 2 files changed, 9 insertions(+), 2 deletions(-)Documentation/admin-guide/cgroup-v1/memory.rst +6
-2@@ -78,6 +78,8 @@ Brief summary of control files. memory.memsw.max_usage_in_bytes show max memory+Swap usage recorded memory.soft_limit_in_bytes set/show soft limit of memory usage This knob is not available on CONFIG_PREEMPT_RT systems. + This knob is deprecated and shouldn't be + used. memory.stat show various statistics memory.use_hierarchy set/show hierarchical account enabled This knob is deprecated and shouldn't be@@ -701,8 +703,10 @@ For compatibility reasons writing 1 to memory.use_hierarchy will always pass:: # echo 1 > memory.use_hierarchy -7. Soft limits -============== +7. Soft limits (DEPRECATED) +=========================== + +THIS IS DEPRECATED! Soft limits allow for greater sharing of memory. The idea behind soft limits is to allow control groups to use as much of the memory as needed, providedmm/memcontrol-v1.c +3
-1@@ -2458,6 +2458,9 @@ static ssize_t mem_cgroup_write(struct kernfs_open_file *of,if (IS_ENABLED (CONFIG_PREEMPT_RT )) { ret = -EOPNOTSUPP ; }else { +pr_warn_once ("soft_limit_in_bytes is deprecated and will be removed. " +"Please report your usecase to [email protected] if you " +"depend on this functionality.\n" );WRITE_ONCE (memcg->soft_limit, nr_pages); ret =0 ; } --2.43.5
Re: [PATCH v2 3/4] memcg: initiate deprecation of oom_control
In reply to Shakeel ButtQuoted 9 lines · Shakeel Butt
The
oom_controlprovides functionality to disable memcg oom-killer, notifications on oom-kill and reading the stats regarding oom-kills. This interface was mainly introduced to provide functionality for userspace oom-killers. However it is not robust enough and only supports OOM handling in the page fault path.For v2, the users can use the combination of memory.events notifications and memory.high interface to provide userspace OOM-killing functionality.
Might be worth it to mention PSI here too for userspace OOM-killing? (It's what LMKD does for Android.)
Quoted 5 lines
Let's start the deprecation process for v1 and gather the info on how the current users are using this interface and work on providing a more robust functionality in v2.
Signed-off-by Shakeel Butt <[email protected]>
Quoted 62 lines
---
Changes since v1:
- Fix build (T.J. Mercier)Documentation/admin-guide/cgroup-v1/memory.rst | 8 ++++++-- mm/memcontrol-v1.c | 7 +++++++ 2 files changed, 13 insertions(+), 2 deletions(-)Documentation/admin-guide/cgroup-v1/memory.rst +6
-2@@ -92,6 +92,8 @@ Brief summary of control files. This knob is deprecated and shouldn't be used. memory.oom_control set/show oom controls. + This knob is deprecated and shouldn't be + used. memory.numa_stat show the number of memory usage per numa node memory.kmem.limit_in_bytes Deprecated knob to set and read the kernel@@ -846,8 +848,10 @@ It's applicable for root and non-root cgroup. .. _cgroup-v1-memory-oom-control: -10. OOM Control -=============== +10. OOM Control (DEPRECATED) +============================ + +THIS IS DEPRECATED! memory.oom_control file is for OOM notification and other controls.mm/memcontrol-v1.c +7
-1@@ -1907,6 +1907,9 @@ static ssize_t memcg_write_event_control(struct kernfs_open_file *of, event->register_event = mem_cgroup_usage_register_event; event->unregister_event = mem_cgroup_usage_unregister_event; }else if (!strcmp (name,"memory.oom_control" )) { +pr_warn_once ("oom_control is deprecated and will be removed. " +"Please report your usecase to [email protected]" +" if you depend on this functionality. \n" ); event->register_event = mem_cgroup_oom_register_event; event->unregister_event = mem_cgroup_oom_unregister_event; }else if (!strcmp (name,"memory.pressure_level" )) {@@ -2754,6 +2757,10 @@ static int mem_cgroup_oom_control_write(struct cgroup_subsys_state *css, {struct mem_cgroup *memcg =mem_cgroup_from_css (css); +pr_warn_once ("oom_control is deprecated and will be removed. " +"Please report your usecase to [email protected] if you " +"depend on this functionality. \n" ); +/* cannot set to root cgroup and only 0 and 1 are allowed */ if (mem_cgroup_is_root (memcg) || !((val ==0 ) || (val ==1 )))return -EINVAL ; --2.43.5
Re: [PATCH v2 4/4] memcg: initiate deprecation of pressure_level
In reply to Shakeel ButtQuoted 6 lines · Shakeel Butt
The
pressure_levelin memcg v1 provides memory pressure notifications to the user space. At the moment it provides notifications for three levels of memory pressure i.e. low, medium and critical, which are defined based on internal memory reclaim implementation details. More specifically the ratio or scanned and reclaimed pages during a memory
"ratio of"
Quoted 9 lines
reclaim. However this is not robust as there are workloads with mostly unreclaimable user memory or kernel memory.
For v2, the users can use PSI for memory pressure status of the system or the cgroup. Let's start the deprecation process for
pressure_leveland add warnings to gather the info on how the current users are using this interface and how they can be used to PSI.Signed-off-by Shakeel Butt <[email protected]>
Quoted 51 lines
---
Changes since v1:
- Fix build (T.J. Mercier)Documentation/admin-guide/cgroup-v1/memory.rst | 8 ++++++-- mm/memcontrol-v1.c | 3 +++ 2 files changed, 9 insertions(+), 2 deletions(-)Documentation/admin-guide/cgroup-v1/memory.rst +6
-2@@ -86,6 +86,8 @@ Brief summary of control files. used. memory.force_empty trigger forced page reclaim memory.pressure_level set memory pressure notifications + This knob is deprecated and shouldn't be + used. memory.swappiness set/show swappiness parameter of vmscan (See sysctl's vm.swappiness) memory.move_charge_at_immigrate set/show controls of moving charges@@ -898,8 +900,10 @@ At reading, current status of OOM is shown. The number of processes belonging to this cgroup killed by any kind of OOM killer. -11. Memory Pressure -=================== +11. Memory Pressure (DEPRECATED) +================================ + +THIS IS DEPRECATED! The pressure level notifications can be used to monitor the memory allocation cost; based on the pressure, applications can implementmm/memcontrol-v1.c +3
-1@@ -1913,6 +1913,9 @@ static ssize_t memcg_write_event_control(struct kernfs_open_file *of, event->register_event = mem_cgroup_oom_register_event; event->unregister_event = mem_cgroup_oom_unregister_event; }else if (!strcmp (name,"memory.pressure_level" )) { +pr_warn_once ("pressure_level is deprecated and will be removed. " +"Please report your usecase to [email protected] " +"if you depend on this functionality. \n" ); event->register_event = vmpressure_register_event; event->unregister_event = vmpressure_unregister_event; }else if (!strcmp (name,"memory.memsw.usage_in_bytes" )) { --2.43.5
Shakeel Butt
maintainerRe: [PATCH v2 3/4] memcg: initiate deprecation of oom_control
In reply to T.J. MercierQuoted 8 lines · T.J. Mercier
Quoted 7 lines · Shakeel Butt
The
oom_controlprovides functionality to disable memcg oom-killer, notifications on oom-kill and reading the stats regarding oom-kills. This interface was mainly introduced to provide functionality for userspace oom-killers. However it is not robust enough and only supports OOM handling in the page fault path.
Quoted 5 lines
Quoted 2 lines
For v2, the users can use the combination of memory.events notifications and memory.high interface to provide userspace OOM-killing functionality.
Might be worth it to mention PSI here too for userspace OOM-killing? (It's what LMKD does for Android.)
I think Andrew will fix the commit messages for others, let me request to replace the first sentence of this paragraph with the following text.
"For v2, the users can use the combination of memory.events notifications, memory.high and PSI to provide userspace OOM-killing functionality. Actually LMKD in Android and OOMd in systemd and Meta infrastructure already uses PSI with combination of other stats to implement userspace OOM-killing."
Quoted 8 lines
Quoted 6 lines
Let's start the deprecation process for v1 and gather the info on how the current users are using this interface and work on providing a more robust functionality in v2.
Signed-off-by Shakeel Butt <[email protected]>Reviewed-by T.J. Mercier <[email protected]>
Thanks
Quoted 7 lines · Shakeel Butt
Let start the deprecation process of the memcg v1 features which we discussed during LSFMMBPF 2024 [1]. For now add the warnings to collect the information on how the current users are using these features. Next we will work on providing better alternatives in v2 (if needed) and fully deprecate these features.
Link https://lwn.net/Articles/974575 [1]
I will add these to our SLES kernels to catch our distribution users.
Thanks!
Quoted 17 lines
Shakeel Butt (4):
memcg: initiate deprecation of v1 tcp accounting
memcg: initiate deprecation of v1 soft limit
memcg: initiate deprecation ofoom_control
memcg: initiate deprecation ofpressure_levelChanges since v1:
- Fix build (T.J. Mercier)
- Fix documentation.../admin-guide/cgroup-v1/memory.rst | 32 +++++++++++++++---- mm/memcontrol-v1.c | 16 ++++++++++ 2 files changed, 42 insertions(+), 6 deletions(-)Signature · 1 line
2.43.5
Signature · 2 lines
Michal Hocko SUSE Labs
Quoted 6 lines · Shakeel Butt
Let start the deprecation process of the memcg v1 features which we discussed during LSFMMBPF 2024 [1]. For now add the warnings to collect the information on how the current users are using these features. Next we will work on providing better alternatives in v2 (if needed) and fully deprecate these features.
Quoted 7 lines
Link https://lwn.net/Articles/974575 [1]Shakeel Butt (4):
memcg: initiate deprecation of v1 tcp accounting
memcg: initiate deprecation of v1 soft limit
memcg: initiate deprecation ofoom_control
memcg: initiate deprecation ofpressure_level
I have a hope that we can deprecate memcg v1 altogether, but having more information about specific v1 users won't hurt.
Thanks!