memcg: net: track network throttling due to memcg memory pressure
Subsystems (7)
Shakeel Butt
maintainerPATCH 1/1 memcg: net: track network throttling due to memcg memory pressure
The kernel can throttle network sockets if the memory cgroup associated with the corresponding socket is under memory pressure. The throttling actions include clamping the transmit window, failing to expand receive or send buffers, aggressively prune out-of-order receive queue, FIN deferred to a retransmitted packet and more. Let's add memcg metric to indicate track such throttling actions.
At the moment memcg memory pressure is defined through vmpressure and in future it may be defined using PSI or we may add more flexible way for the users to define memory pressure, maybe through ebpf. However the potential throttling actions will remain the same, so this newly introduced metric will continue to track throttling actions irrespective of how memcg memory pressure is defined.
--- Documentation/admin-guide/cgroup-v2.rst | 4 ++++ include/linux/memcontrol.h | 1 + include/net/sock.h | 6 +++++- kernel/cgroup/cgroup.c | 1 + mm/memcontrol.c | 3 +++ 5 files changed, 14 insertions(+), 1 deletion(-)
Documentation/admin-guide/cgroup-v2.rst +4 -0
@@ -1515,6 +1515,10 @@ The following nested keys are defined. oom_group_kill The number of times a group OOM has occurred. + socks_throttled + The number of times network sockets associated with + this cgroup are throttled. + memory.events.local Similar to memory.events but the fields in the file are local to the cgroup i.e. not hierarchical. The file modified event
include/linux/memcontrol.h +1 -0
@@ -52,6 +52,7 @@ enum memcg_memory_event {MEMCG_SWAP_HIGH ,MEMCG_SWAP_MAX ,MEMCG_SWAP_FAIL , +MEMCG_SOCKS_THROTTLED , MEMCG_NR_MEMORY_EVENTS, };
include/net/sock.h +5 -1
@@ -2635,8 +2635,12 @@ static inline bool mem_cgroup_sk_under_memory_pressure(const struct sock *sk)#endif /* CONFIG_MEMCG_V1 */ do { -if (time_before64 (get_jiffies_64 (),mem_cgroup_get_socket_pressure (memcg))) +if (time_before64 (get_jiffies_64 (), +mem_cgroup_get_socket_pressure (memcg))) { +memcg_memory_event (mem_cgroup_from_sk (sk), +MEMCG_SOCKS_THROTTLED );return true; + } }while ((memcg =parent_mem_cgroup (memcg)));return false;
kernel/cgroup/cgroup.c +1 -0
@@ -4704,6 +4704,7 @@ void cgroup_file_notify(struct cgroup_file *cfile) }spin_unlock_irqrestore (&cgroup_file_kn_lock, flags); } +EXPORT_SYMBOL_GPL (cgroup_file_notify);/** * cgroup_file_show - show or hide a hidden cgroup file
mm/memcontrol.c +3 -0
@@ -81,6 +81,7 @@ struct cgroup_subsys memory_cgrp_subsys __read_mostly;EXPORT_SYMBOL (memory_cgrp_subsys);struct mem_cgroup *root_mem_cgroup__read_mostly ; +EXPORT_SYMBOL (root_mem_cgroup);/* Active memory cgroup to use from an interrupt context */ DEFINE_PER_CPU(struct mem_cgroup *, int_active_memcg);
@@ -4463,6 +4464,8 @@ static void __memory_events_show(struct seq_file *m, atomic_long_t *events) atomic_long_read(&events[MEMCG_OOM_KILL]));seq_printf (m,"oom_group_kill %lu\n" ,atomic_long_read (&events[MEMCG_OOM_GROUP_KILL ])); +seq_printf (m,"socks_throttled %lu\n" , +atomic_long_read (&events[MEMCG_SOCKS_THROTTLED ])); }static int memory_events_show (struct seq_file *m,void *v)
Signature · 1 line
2.47.3
Quoted 28 lines · Shakeel Butt
The kernel can throttle network sockets if the memory cgroup associated with the corresponding socket is under memory pressure. The throttling actions include clamping the transmit window, failing to expand receive or send buffers, aggressively prune out-of-order receive queue, FIN deferred to a retransmitted packet and more. Let's add memcg metric to indicate track such throttling actions.
At the moment memcg memory pressure is defined through vmpressure and in future it may be defined using PSI or we may add more flexible way for the users to define memory pressure, maybe through ebpf. However the potential throttling actions will remain the same, so this newly introduced metric will continue to track throttling actions irrespective of how memcg memory pressure is defined.
Signed-off-by Shakeel Butt <[email protected]>--- Documentation/admin-guide/cgroup-v2.rst | 4 ++++ include/linux/memcontrol.h | 1 + include/net/sock.h | 6 +++++- kernel/cgroup/cgroup.c | 1 + mm/memcontrol.c | 3 +++ 5 files changed, 14 insertions(+), 1 deletion(-)Documentation/admin-guide/cgroup-v2.rst +0
-0
Documentation/admin-guide/cgroup-v2.rst The following nested keys are defined. · line 1520
Quoted 6 lines
Documentation/admin-guide/cgroup-v2.rst +3
-0@@ -1515,6 +1515,10 @@ The following nested keys are defined. oom_group_kill The number of times a group OOM has occurred. + socks_throttled + The number of times network sockets associated with + this cgroup are throttled.
I'd prefer sockets_throttled or sock_throttled. And same for the constant name.
Otherwise,
Thanks!
Quoted 41 lines · Roman Gushchin
Quoted 28 lines · Shakeel Butt
The kernel can throttle network sockets if the memory cgroup associated with the corresponding socket is under memory pressure. The throttling actions include clamping the transmit window, failing to expand receive or send buffers, aggressively prune out-of-order receive queue, FIN deferred to a retransmitted packet and more. Let's add memcg metric to indicate track such throttling actions.
At the moment memcg memory pressure is defined through vmpressure and in future it may be defined using PSI or we may add more flexible way for the users to define memory pressure, maybe through ebpf. However the potential throttling actions will remain the same, so this newly introduced metric will continue to track throttling actions irrespective of how memcg memory pressure is defined.
Signed-off-by Shakeel Butt <[email protected]>--- Documentation/admin-guide/cgroup-v2.rst | 4 ++++ include/linux/memcontrol.h | 1 + include/net/sock.h | 6 +++++- kernel/cgroup/cgroup.c | 1 + mm/memcontrol.c | 3 +++ 5 files changed, 14 insertions(+), 1 deletion(-)Documentation/admin-guide/cgroup-v2.rst +0
-0Documentation/admin-guide/cgroup-v2.rst The following nested keys are defined. · line 1520
Quoted 6 lines
Documentation/admin-guide/cgroup-v2.rst +3
-0@@ -1515,6 +1515,10 @@ The following nested keys are defined. oom_group_kill The number of times a group OOM has occurred. + socks_throttled + The number of times network sockets associated with + this cgroup are throttled.I'd prefer
sockets_throttledorsock_throttled. And same for the constant name.
Quoted 2 lines
Otherwise,
Acked-by Roman Gushchin <[email protected]>
+1 for sock_ like "sock" in memory.stat and its MEMCG_SOCK.
Thanks!
Shakeel Butt
maintainerQuoted 42 lines · Kuniyuki Iwashima
Quoted 41 lines · Roman Gushchin
Quoted 28 lines · Shakeel Butt
The kernel can throttle network sockets if the memory cgroup associated with the corresponding socket is under memory pressure. The throttling actions include clamping the transmit window, failing to expand receive or send buffers, aggressively prune out-of-order receive queue, FIN deferred to a retransmitted packet and more. Let's add memcg metric to indicate track such throttling actions.
At the moment memcg memory pressure is defined through vmpressure and in future it may be defined using PSI or we may add more flexible way for the users to define memory pressure, maybe through ebpf. However the potential throttling actions will remain the same, so this newly introduced metric will continue to track throttling actions irrespective of how memcg memory pressure is defined.
Signed-off-by Shakeel Butt <[email protected]>--- Documentation/admin-guide/cgroup-v2.rst | 4 ++++ include/linux/memcontrol.h | 1 + include/net/sock.h | 6 +++++- kernel/cgroup/cgroup.c | 1 + mm/memcontrol.c | 3 +++ 5 files changed, 14 insertions(+), 1 deletion(-)Documentation/admin-guide/cgroup-v2.rst +0
-0Documentation/admin-guide/cgroup-v2.rst The following nested keys are defined. · line 1520
Quoted 6 lines
Documentation/admin-guide/cgroup-v2.rst +3
-0@@ -1515,6 +1515,10 @@ The following nested keys are defined. oom_group_kill The number of times a group OOM has occurred. + socks_throttled + The number of times network sockets associated with + this cgroup are throttled.I'd prefer
sockets_throttledorsock_throttled. And same for the constant name.
Quoted 7 lines
Quoted 2 lines
Otherwise,
Acked-by Roman Gushchin <[email protected]>+1 for
sock_like "sock" in memory.stat and itsMEMCG_SOCK.Reviewed-by Kuniyuki Iwashima <[email protected]>
Thanks Roman and Kuniyuki, will change the name to sock_throttled in v2.
Quoted 7 lines · Shakeel Butt
The kernel can throttle network sockets if the memory cgroup associated with the corresponding socket is under memory pressure. The throttling actions include clamping the transmit window, failing to expand receive or send buffers, aggressively prune out-of-order receive queue, FIN deferred to a retransmitted packet and more. Let's add memcg metric to indicate track such throttling actions.
Quoted 8 lines
At the moment memcg memory pressure is defined through vmpressure and in future it may be defined using PSI or we may add more flexible way for the users to define memory pressure, maybe through ebpf. However the potential throttling actions will remain the same, so this newly introduced metric will continue to track throttling actions irrespective of how memcg memory pressure is defined.
Signed-off-by Shakeel Butt <[email protected]>
I am curious how the future work will unfold. If you need help with future developments I can help you, we have hundreds of servers where this throttling is happening.
Thanks!
Daniel
Shakeel Butt
maintainerQuoted 15 lines · Daniel Sedlak
Quoted 14 lines · Shakeel Butt
The kernel can throttle network sockets if the memory cgroup associated with the corresponding socket is under memory pressure. The throttling actions include clamping the transmit window, failing to expand receive or send buffers, aggressively prune out-of-order receive queue, FIN deferred to a retransmitted packet and more. Let's add memcg metric to indicate track such throttling actions.
At the moment memcg memory pressure is defined through vmpressure and in future it may be defined using PSI or we may add more flexible way for the users to define memory pressure, maybe through ebpf. However the potential throttling actions will remain the same, so this newly introduced metric will continue to track throttling actions irrespective of how memcg memory pressure is defined.
Quoted 3 lines
Quoted 1 line
Signed-off-by Shakeel Butt <[email protected]>Reviewed-by Daniel Sedlak <[email protected]>
Thanks.
Quoted 4 lines
I am curious how the future work will unfold. If you need help with future developments I can help you, we have hundreds of servers where this throttling is happening.
I think first thing I would like to know if this patch is a good start for your use-case of observability and debugging. What else do you need for sufficient support for your use-case? I imagine that would be tracepoints to extract more information on the source of the throttling. If you don't mind, can you take a stab at that? In the long run, we want more flexible definition of memcg memory pressure. Let us know of any requirements you have for that. Thanks again for continuosly pushing this conversation.
Quoted 9 lines · Shakeel Butt
Quoted 4 lines · Daniel Sedlak
On 10/16/25 3:31 AM, Shakeel Butt wrote:
I am curious how the future work will unfold. If you need help with future developments I can help you, we have hundreds of servers where this throttling is happening.I think first thing I would like to know if this patch is a good start for your use-case of observability and debugging.What else do you need for sufficient support for your use-case?
Yes, it is a good start, we can now hook this easily into our monitoring system and detect affected servers more easily.
Quoted 3 lines
I imagine that would be
tracepoints to extract more information on the source of the throttling. If you don't mind, can you take a stab at that?
We have some tracepoints that we have used for debugging this. We would like to upstream them, if that makes sense to you?
Thanks!
Daniel
Shakeel Butt
maintainerQuoted 14 lines · Daniel Sedlak
Quoted 9 lines · Shakeel Butt
Quoted 4 lines · Daniel Sedlak
On 10/16/25 3:31 AM, Shakeel Butt wrote:
I am curious how the future work will unfold. If you need help with future developments I can help you, we have hundreds of servers where this throttling is happening.I think first thing I would like to know if this patch is a good start for your use-case of observability and debugging.What else do you need for sufficient support for your use-case?
Yes, it is a good start, we can now hook this easily into our monitoring system and detect affected servers more easily.
Quoted 6 lines
Quoted 3 lines
I imagine that would be
tracepoints to extract more information on the source of the throttling. If you don't mind, can you take a stab at that?We have some tracepoints that we have used for debugging this. We would like to upstream them, if that makes sense to you?
Yes please, send them out.