Skip to contentSkip to conversation

mm: memcg: separate legacy cgroup v1 code and put under config option

RFC PATCH In review Roman Gushchin 8 participants 32 messages May 9, 2024 – May 28, 2024

Subsystems (4)
lore
Share & tools
lore

Roman Gushchin

author maintainer
#0 lore raw

cover mm: memcg: separate legacy cgroup v1 code and put under config option

Cgroups v2 have been around for a while and many users have fully adopted them, so they never use cgroups v1 features and functionality. Yet they have to "pay" for the cgroup v1 support anyway:
1) the kernel binary contains useless cgroup v1 code,
2) some common structures like task_struct and mem_cgroup have never used cgroup v1-specific members,
3) some code paths have additional checks which are not needed.

Cgroup v1's memory controller has a number of features that are not supported by cgroup v2 and their implementation is pretty much self contained. Most notably, these features are: soft limit reclaim, oom handling in userspace, complicated event notification system, charge migration.

Cgroup v1-specific code in memcontrol.c is close to 4k lines in size and it's intervened with generic and cgroup v2-specific code. It's a burden on developers and maintainers.

This patchset aims to solve these problems by:
1) moving cgroup v1-specific memcg code to the new mm/memcontrol-v1.c file,
2) putting definitions shared by memcontrol.c and memcontrol-v1.c into the mm/internal.h header
3) introducing the CONFIG_MEMCG_V1 config option, turned on by default
4) making memcontrol-v1.c to compile only if CONFIG_MEMCG_V1 is set
5) putting unused struct memory_cgroup and task_struct members under CONFIG_MEMCG_V1 as well.

This is an RFC version, which is not 100% polished yet, so but it would be great to discuss and agree on the overall approach.

Some open questions, opinions are appreciated:
1) I consider renaming non-static functions in memcontrol-v1.c to have mem_cgroup_v1_ prefix. Is this a good idea?
2) Do we want to extend it beyond the memory controller? Should
3) Is it better to use a new include/linux/memcontrol-v1.h instead of mm/internal.h? Or mm/memcontrol-v1.h.

diffstat:

 include/linux/memcontrol.h |  165 ++++---
 include/linux/sched.h      |    5 +-
 init/Kconfig               |    7 +
 mm/Makefile                |    2 +
 mm/internal.h              |  124 +++++
 mm/memcontrol-v1.c         | 2941 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 mm/memcontrol.c            | 4121 ++++++++++++++++++++++---------------------------------------------------------------------------------------------------------------------------------
 7 files changed, 3765 insertions(+), 3600 deletions(-)

Roman Gushchin (9):
mm: memcg: introduce memcontrol-v1.c
mm: memcg: move soft limit reclaim code to memcontrol-v1.c
mm: memcg: move charge migration code to memcontrol-v1.c
mm: memcg: move legacy memcg event code into memcontrol-v1.c
mm: memcg: move cgroup v1 interface files to memcontrol-v1.c
mm: memcg: move cgroup v1 oom handling code into memcontrol-v1.c
mm: memcg: put cgroup v1-specific code under a config option
mm: memcg: put corresponding struct mem_cgroup members under
CONFIG_MEMCG_V1
mm: memcg: put cgroup v1-related members of task_struct under config
option

 include/linux/memcontrol.h |  165 +-
 include/linux/sched.h      |    5 +-
 init/Kconfig               |    7 +
 mm/Makefile                |    2 +
 mm/internal.h              |  124 ++
 mm/memcontrol-v1.c         | 2941 +++++++++++++++++++++++++
 mm/memcontrol.c            | 4121 ++++++------------------------------
 7 files changed, 3765 insertions(+), 3600 deletions(-)
 create mode 100644 mm/memcontrol-v1.c
Signature · 1 line
2.43.2
author maintainer
#1 lore raw

RFC PATCH 1/9 mm: memcg: introduce memcontrol-v1.c

This patch introduces the mm/memcontrol-v1.c source file which will be used for all legacy (cgroup v1) memory cgroup code.

As of now, let's compile it if CONFIG_MEMCG is set, similar to mm/memcontrol.c. Later on it can be switched to use a separate config option, so that the legacy code won't be compiled if not required.

---
 mm/Makefile        | 3 ++-
 mm/memcontrol-v1.c | 2 ++
 2 files changed, 4 insertions(+), 1 deletion(-)
 create mode 100644 mm/memcontrol-v1.c
mm/Makefile +2 -1
index 25da205becdd..c717a3ee612e 100644
@@ -26,6 +26,7 @@ KCOV_INSTRUMENT_page_alloc.o := n
 KCOV_INSTRUMENT_debug-pagealloc.o := n
 KCOV_INSTRUMENT_kmemleak.o := n
 KCOV_INSTRUMENT_memcontrol.o := n
+KCOV_INSTRUMENT_memcontrol-v1.o := n
 KCOV_INSTRUMENT_mmzone.o := n
 KCOV_INSTRUMENT_vmstat.o := n
 KCOV_INSTRUMENT_failslab.o := n
@@ -95,7 +96,7 @@ obj-$(CONFIG_NUMA) += memory-tiers.o
 obj-$(CONFIG_DEVICE_MIGRATION) += migrate_device.o
 obj-$(CONFIG_TRANSPARENT_HUGEPAGE) += huge_memory.o khugepaged.o
 obj-$(CONFIG_PAGE_COUNTER) += page_counter.o
-obj-$(CONFIG_MEMCG) += memcontrol.o vmpressure.o
+obj-$(CONFIG_MEMCG) += memcontrol.o memcontrol-v1.o vmpressure.o
 ifdef CONFIG_SWAP
 obj-$(CONFIG_MEMCG) += swap_cgroup.o
 endif
mm/memcontrol-v1.c new file +2 -0
new file mode 100644
index 000000000000..4bc66fc244c0
@@ -0,0 +1,2 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+
Signature · 1 line
2.43.2
author maintainer
#2 lore raw

RFC PATCH 2/9 mm: memcg: move soft limit reclaim code to memcontrol-v1.c

In reply to Roman Gushchin

From: Roman Gushchin <[email protected]>

Soft limits are cgroup v1-specific and are not supported by cgroup v2, so let's move the corresponding code into memcontrol-v1.c.

Aside from simple moving the code, this commits introduces a trivial mem_cgroup_soft_limit_reset() function to reset soft limits and also moves the global soft limit tree initialization code into a new mem_cgroup_v1_init() function.

It also moves corresponding definitions in include/memcontrol.h into a separate section at the end of the file. The idea is to group all memcg v1-specific definitions in one place and provide trivial alternatives later on to support compiling the kernel without support of the legacy memory controller.

---
 include/linux/memcontrol.h |  28 +--
 mm/internal.h              |   5 +
 mm/memcontrol-v1.c         | 347 +++++++++++++++++++++++++++++++++++++
 mm/memcontrol.c            | 336 +----------------------------------
 4 files changed, 372 insertions(+), 344 deletions(-)
include/linux/memcontrol.h +17 -11
index 030d34e9d117..f77b6fbf38fd 100644
@@ -1120,10 +1120,6 @@ static inline void memcg_memory_event_mm(struct mm_struct *mm,
 
 void split_page_memcg(struct page *head, int old_order, int new_order);
 
-unsigned long mem_cgroup_soft_limit_reclaim(pg_data_t *pgdat, int order,
-						gfp_t gfp_mask,
-						unsigned long *total_scanned);
-
 #else /* CONFIG_MEMCG */
 
 #define MEM_CGROUP_ID_SHIFT	0
@@ -1575,13 +1571,6 @@ static inline void split_page_memcg(struct page *head, int old_order, int new_or
 {
 }
 
-static inline
-unsigned long mem_cgroup_soft_limit_reclaim(pg_data_t *pgdat, int order,
-					    gfp_t gfp_mask,
-					    unsigned long *total_scanned)
-{
-	return 0;
-}
 #endif /* CONFIG_MEMCG */
 
 /*
@@ -1932,4 +1921,21 @@ static inline bool mem_cgroup_zswap_writeback_enabled(struct mem_cgroup *memcg)
 }
 #endif
 
+/* Cgroup v1-specific definitions */
+
+#ifdef CONFIG_MEMCG
+unsigned long mem_cgroup_soft_limit_reclaim(pg_data_t *pgdat, int order,
+						gfp_t gfp_mask,
+						unsigned long *total_scanned);
+#else /* CONFIG_MEMCG */
+
+static inline
+unsigned long mem_cgroup_soft_limit_reclaim(pg_data_t *pgdat, int order,
+					    gfp_t gfp_mask,
+					    unsigned long *total_scanned)
+{
+	return 0;
+}
+#endif /* CONFIG_MEMCG */
+
 #endif /* _LINUX_MEMCONTROL_H */
mm/internal.h +5 -0
index b2c75b12014e..19e96d626977 100644
@@ -1521,4 +1521,9 @@ static inline void shrinker_debugfs_remove(struct dentry *debugfs_entry,
 void workingset_update_node(struct xa_node *node);
 extern struct list_lru shadow_nodes;
 
+/* Memory cgroups v1-specific definitions */
+void mem_cgroup_update_tree(struct mem_cgroup *memcg, int nid);
+void mem_cgroup_remove_from_trees(struct mem_cgroup *memcg);
+void mem_cgroup_soft_limit_reset(struct mem_cgroup *memcg);
+
 #endif	/* __MM_INTERNAL_H */
mm/memcontrol-v1.c +347 -0

349 lines not on this page · raw message

mm/memcontrol.c +3 -333

390 lines not on this page · raw message

Signature · 1 line
2.43.2
author maintainer
#3 lore raw

RFC PATCH 3/9 mm: memcg: move charge migration code to memcontrol-v1.c

In reply to Roman Gushchin

From: Roman Gushchin <[email protected]>

Unlike the legacy cgroup v1 memory controller, cgroup v2 memory controller doesn't support moving charged pages between cgroups.

It's a fairly large and complicated code which created a number of problems in the past. Let's move this code into memcontrol-v1.c. It shaves off 1k lines from memcontrol.c. It's also another step towards making the legacy memory controller code compiled optionally.

---
 include/linux/memcontrol.h |   78 +--
 mm/internal.h              |   31 ++
 mm/memcontrol-v1.c         |  981 +++++++++++++++++++++++++++++++++++
 mm/memcontrol.c            | 1005 +-----------------------------------
 4 files changed, 1059 insertions(+), 1036 deletions(-)
include/linux/memcontrol.h +39 -39
index f77b6fbf38fd..60355f3cb67c 100644
@@ -971,29 +971,10 @@ struct mem_cgroup *mem_cgroup_get_oom_group(struct task_struct *victim,
 					    struct mem_cgroup *oom_domain);
 void mem_cgroup_print_oom_group(struct mem_cgroup *memcg);
 
-void folio_memcg_lock(struct folio *folio);
-void folio_memcg_unlock(struct folio *folio);
 
 void __mod_memcg_state(struct mem_cgroup *memcg, enum memcg_stat_item idx,
 		       int val);
 
-/* try to stablize folio_memcg() for all the pages in a memcg */
-static inline bool mem_cgroup_trylock_pages(struct mem_cgroup *memcg)
-{
-	rcu_read_lock();
-
-	if (mem_cgroup_disabled() || !atomic_read(&memcg->moving_account))
-		return true;
-
-	rcu_read_unlock();
-	return false;
-}
-
-static inline void mem_cgroup_unlock_pages(void)
-{
-	rcu_read_unlock();
-}
-
 /* idx can be of type enum memcg_stat_item or node_stat_item */
 static inline void mod_memcg_state(struct mem_cgroup *memcg,
 				   enum memcg_stat_item idx, int val)
@@ -1435,26 +1416,6 @@ mem_cgroup_print_oom_meminfo(struct mem_cgroup *memcg)
 {
 }
 
-static inline void folio_memcg_lock(struct folio *folio)
-{
-}
-
-static inline void folio_memcg_unlock(struct folio *folio)
-{
-}
-
-static inline bool mem_cgroup_trylock_pages(struct mem_cgroup *memcg)
-{
-	/* to match folio_memcg_rcu() */
-	rcu_read_lock();
-	return true;
-}
-
-static inline void mem_cgroup_unlock_pages(void)
-{
-	rcu_read_unlock();
-}
-
 static inline void mem_cgroup_handle_over_high(gfp_t gfp_mask)
 {
 }
@@ -1927,6 +1888,25 @@ static inline bool mem_cgroup_zswap_writeback_enabled(struct mem_cgroup *memcg)
 unsigned long mem_cgroup_soft_limit_reclaim(pg_data_t *pgdat, int order,
 						gfp_t gfp_mask,
 						unsigned long *total_scanned);
+void folio_memcg_lock(struct folio *folio);
+void folio_memcg_unlock(struct folio *folio);
+
+/* try to stablize folio_memcg() for all the pages in a memcg */
+static inline bool mem_cgroup_trylock_pages(struct mem_cgroup *memcg)
+{
+	rcu_read_lock();
+
+	if (mem_cgroup_disabled() || !atomic_read(&memcg->moving_account))
+		return true;
+
+	rcu_read_unlock();
+	return false;
+}
+
+static inline void mem_cgroup_unlock_pages(void)
+{
+	rcu_read_unlock();
+}
 #else /* CONFIG_MEMCG */
 
 static inline
@@ -1936,6 +1916,26 @@ unsigned long mem_cgroup_soft_limit_reclaim(pg_data_t *pgdat, int order,
 {
 	return 0;
 }
+
+static inline void folio_memcg_lock(struct folio *folio)
+{
+}
+
+static inline void folio_memcg_unlock(struct folio *folio)
+{
+}
+
+static inline bool mem_cgroup_trylock_pages(struct mem_cgroup *memcg)
+{
+	/* to match folio_memcg_rcu() */
+	rcu_read_lock();
+	return true;
+}
+
+static inline void mem_cgroup_unlock_pages(void)
+{
+	rcu_read_unlock();
+}
 #endif /* CONFIG_MEMCG */
 
 #endif /* _LINUX_MEMCONTROL_H */
mm/internal.h +31 -0
index 19e96d626977..9ffd48375ae5 100644
@@ -1521,9 +1521,40 @@ static inline void shrinker_debugfs_remove(struct dentry *debugfs_entry,
 void workingset_update_node(struct xa_node *node);
 extern struct list_lru shadow_nodes;
 
+/* Memcontrol definitions used by memory cgroups v1-specific code */
+int try_charge_memcg(struct mem_cgroup *memcg, gfp_t gfp_mask,
+		     unsigned int nr_pages);
+
+static inline int try_charge(struct mem_cgroup *memcg, gfp_t gfp_mask,
+			     unsigned int nr_pages)
+{
+	if (mem_cgroup_is_root(memcg))
+		return 0;
+
+	return try_charge_memcg(memcg, gfp_mask, nr_pages);
+}
+
+void mem_cgroup_charge_statistics(struct mem_cgroup *memcg, int nr_pages);
+void memcg_check_events(struct mem_cgroup *memcg, int nid);
+void memcg_oom_recover(struct mem_cgroup *memcg);
+void mem_cgroup_id_get_many(struct mem_cgroup *memcg, unsigned int n);
+void mem_cgroup_id_put_many(struct mem_cgroup *memcg, unsigned int n);
+
 /* Memory cgroups v1-specific definitions */
 void mem_cgroup_update_tree(struct mem_cgroup *memcg, int nid);
 void mem_cgroup_remove_from_trees(struct mem_cgroup *memcg);
 void mem_cgroup_soft_limit_reset(struct mem_cgroup *memcg);
 
+struct cftype;
+u64 mem_cgroup_move_charge_read(struct cgroup_subsys_state *css,
+				struct cftype *cft);
+int mem_cgroup_move_charge_write(struct cgroup_subsys_state *css,
+				 struct cftype *cft, u64 val);
+
+struct cgroup_taskset;
+int mem_cgroup_can_attach(struct cgroup_taskset *tset);
+void mem_cgroup_cancel_attach(struct cgroup_taskset *tset);
+bool mem_cgroup_wait_acct_move(struct mem_cgroup *memcg);
+void mem_cgroup_move_task(void);
+
 #endif	/* __MM_INTERNAL_H */
mm/memcontrol-v1.c +981 -0

1,002 lines not on this page · raw message

mm/memcontrol.c +8 -997

1,099 lines not on this page · raw message

Signature · 1 line
2.43.2
author maintainer
#4 lore raw

RFC PATCH 4/9 mm: memcg: move legacy memcg event code into memcontrol-v1.c

In reply to Roman Gushchin

Cgroup v1's memory controller contains a pretty complicated event notifications mechanism which is not used on cgroup v2. Let's move the corresponding code into memcontrol-v1.c.

Please, note, that mem_cgroup_event_ratelimit() remains in memcontrol.c, otherwise it would require exporting too many details on memcg stats outside of memcontrol.c.

---
 include/linux/memcontrol.h |  12 -
 mm/internal.h              |  55 ++-
 mm/memcontrol-v1.c         | 653 +++++++++++++++++++++++++++++++++++
 mm/memcontrol.c            | 687 +------------------------------------
 4 files changed, 712 insertions(+), 695 deletions(-)
include/linux/memcontrol.h +0 -12
index 60355f3cb67c..fc4aaa73aa5e 100644
@@ -69,18 +69,6 @@ struct mem_cgroup_id {
 	refcount_t ref;
 };
 
-/*
- * Per memcg event counter is incremented at every pagein/pageout. With THP,
- * it will be incremented by the number of pages. This counter is used
- * to trigger some periodic events. This is straightforward and better
- * than using jiffies etc. to handle periodic memcg event.
- */
-enum mem_cgroup_events_target {
-	MEM_CGROUP_TARGET_THRESH,
-	MEM_CGROUP_TARGET_SOFTLIMIT,
-	MEM_CGROUP_NTARGETS,
-};
-
 struct memcg_vmstats_percpu;
 struct memcg_vmstats;
 struct lruvec_stats_percpu;
mm/internal.h +54 -1

75 lines not on this page · raw message

mm/memcontrol-v1.c +653 -0

673 lines not on this page · raw message

mm/memcontrol.c +5 -682

755 lines not on this page · raw message

Signature · 1 line
2.43.2
author maintainer
#5 lore raw

RFC PATCH 5/9 mm: memcg: move cgroup v1 interface files to memcontrol-v1.c

In reply to Roman Gushchin

Move legacy cgroup v1 memory controller interfaces and corresponding code into memcontrol-v1.c.

---
 mm/internal.h      |  15 +-
 mm/memcontrol-v1.c | 733 +++++++++++++++++++++++++++++++++++++++++++-
 mm/memcontrol.c    | 748 +--------------------------------------------
 3 files changed, 751 insertions(+), 745 deletions(-)
mm/internal.h +13 -2
index 79104cfc08a9..533aa999a450 100644
@@ -13,6 +13,7 @@
 #include <linux/rmap.h>
 #include <linux/swap.h>
 #include <linux/swapops.h>
+#include <linux/cgroup-defs.h>
 #include <linux/tracepoint-defs.h>
 
 struct folio_batch;
@@ -1579,6 +1580,14 @@ enum mem_cgroup_events_target {
 bool mem_cgroup_event_ratelimit(struct mem_cgroup *memcg,
 				enum mem_cgroup_events_target target);
 
+int memory_stat_show(struct seq_file *m, void *v);
+unsigned long memcg_page_state_local(struct mem_cgroup *memcg, int idx);
+unsigned long memcg_page_state_output(struct mem_cgroup *memcg, int item);
+unsigned long memcg_page_state_local_output(struct mem_cgroup *memcg, int item);
+unsigned long memcg_events(struct mem_cgroup *memcg, int event);
+unsigned long memcg_events_local(struct mem_cgroup *memcg, int event);
+void drain_all_stock(struct mem_cgroup *root_memcg);
+
 /* Memory cgroups v1-specific definitions */
 void mem_cgroup_update_tree(struct mem_cgroup *memcg, int nid);
 void mem_cgroup_remove_from_trees(struct mem_cgroup *memcg);
@@ -1606,8 +1615,10 @@ enum res_type {
 
 void memcg_check_events(struct mem_cgroup *memcg, int nid);
 void mem_cgroup_oom_notify(struct mem_cgroup *memcg);
-ssize_t memcg_write_event_control(struct kernfs_open_file *of, char *buf,
-				  size_t nbytes, loff_t off);
+void memcg1_stat_format(struct mem_cgroup *memcg, struct seq_buf *s);
 void mem_cgroup_v1_offline_memcg(struct mem_cgroup *memcg);
 
+extern struct cftype memsw_files[];
+extern struct cftype mem_cgroup_legacy_files[];
+
 #endif	/* __MM_INTERNAL_H */
mm/memcontrol-v1.c +731 -2

757 lines not on this page · raw message

mm/memcontrol.c +7 -741

831 lines not on this page · raw message

Signature · 1 line
2.43.2
author maintainer
#6 lore raw

RFC PATCH 6/9 mm: memcg: move cgroup v1 oom handling code into memcontrol-v1.c

In reply to Roman Gushchin

Cgroup v1 supported a complicated OOM handling mechanism in userspace, which is not supported by cgroup v2. Let's move the corresponding code into memcontrol-v1.c.

Aside from mechanical code movement this patch introduces two new
functions: mem_cgroup_v1_oom_prepare() and mem_cgroup_v1_oom_finish(). Those are implementing cgroup v1-specific parts of the common memcg OOM handling path.

---
 mm/internal.h      |   4 +-
 mm/memcontrol-v1.c | 229 +++++++++++++++++++++++++++++++++++++++++++++
 mm/memcontrol.c    | 220 +------------------------------------------
 3 files changed, 234 insertions(+), 219 deletions(-)
mm/internal.h +3 -1
index 533aa999a450..1b94e2169e19 100644
@@ -1614,8 +1614,10 @@ enum res_type {
 };
 
 void memcg_check_events(struct mem_cgroup *memcg, int nid);
-void mem_cgroup_oom_notify(struct mem_cgroup *memcg);
 void memcg1_stat_format(struct mem_cgroup *memcg, struct seq_buf *s);
+bool mem_cgroup_v1_oom_prepare(struct mem_cgroup *memcg, gfp_t mask, int order,
+			       bool *locked);
+void mem_cgroup_v1_oom_finish(struct mem_cgroup *memcg, bool *locked);
 void mem_cgroup_v1_offline_memcg(struct mem_cgroup *memcg);
 
 extern struct cftype memsw_files[];
mm/memcontrol-v1.c +229 -0

235 lines not on this page · raw message

mm/memcontrol.c +2 -218

238 lines not on this page · raw message

Signature · 1 line
2.43.2
reviewer
#7 lore raw

Re: [PATCH rfc 6/9] mm: memcg: move cgroup v1 oom handling code into memcontrol-v1.c

On Wed 08-05-24 20:41:35, Roman Gushchin wrote:
[...]

Quoted 26 lines
mm/memcontrol.c +0 -22
@@ -1747,106 +1623,14 @@ static bool mem_cgroup_oom(struct mem_cgroup *memcg, gfp_t mask, int order)
 
 	memcg_memory_event(memcg, MEMCG_OOM);
 
-	/*
-	 * We are in the middle of the charge context here, so we
-	 * don't want to block when potentially sitting on a callstack
-	 * that holds all kinds of filesystem and mm locks.
-	 *
-	 * cgroup1 allows disabling the OOM killer and waiting for outside
-	 * handling until the charge can succeed; remember the context and put
-	 * the task to sleep at the end of the page fault when all locks are
-	 * released.
-	 *
-	 * On the other hand, in-kernel OOM killer allows for an async victim
-	 * memory reclaim (oom_reaper) and that means that we are not solely
-	 * relying on the oom victim to make a forward progress and we can
-	 * invoke the oom killer here.
-	 *
-	 * Please note that mem_cgroup_out_of_memory might fail to find a
-	 * victim and then we have to bail out from the charge path.
-	 */
-	if (READ_ONCE(memcg->oom_kill_disable)) {
-		if (current->in_user_fault) {
-			css_get(&memcg->css);
-			current->memcg_in_oom = memcg;

mm/memcontrol.c static bool mem_cgroup_oom(struct mem_cgroup *memcg, gfp_t mask, int order) · line 1780

Quoted 10 lines
mm/memcontrol.c +1 -8
-			current->memcg_oom_gfp_mask = mask;
-			current->memcg_oom_order = order;
-		}
+	if (!mem_cgroup_v1_oom_prepare(memcg, mask, order, &locked))
 		return false;
-	}
-
-	mem_cgroup_mark_under_oom(memcg);
-
-	locked = mem_cgroup_oom_trylock(memcg);

This really confused me because this looks like the oom locking is removed for v2 but this is not the case because mem_cgroup_v1_oom_prepare is not really v1 only code - in other words this is not going to be just return false for CONFIG_MEMCG_V1=n.

It makes sense to move the userspace oom handling out to the v1 file. I would keep mem_cgroup_mark_under_oom here. I am not sure about the oom locking thing because I think we can make it v1 only. For v2 I guess we can go without this locking as the oom path is already locked and it implements overkilling prevention (oom_evaluate_task) as it walks all processes in the oom hierarchy.

Quoted 13 lines
mm/memcontrol.c +1 -8
-
-	if (locked)
-		mem_cgroup_oom_notify(memcg);
-
-	mem_cgroup_unmark_under_oom(memcg);
 	ret = mem_cgroup_out_of_memory(memcg, mask, order);
-
-	if (locked)
-		mem_cgroup_oom_unlock(memcg);
+	mem_cgroup_v1_oom_finish(memcg, &locked);
 
 	return ret;
 }
Signature · 2 lines
Michal Hocko
SUSE Labs
author maintainer
#8 lore raw

Re: [PATCH rfc 6/9] mm: memcg: move cgroup v1 oom handling code into memcontrol-v1.c

Quoted 44 lines · Michal Hocko

On Wed 08-05-24 20:41:35, Roman Gushchin wrote:
[...]

Quoted 26 lines
mm/memcontrol.c +0 -22
@@ -1747,106 +1623,14 @@ static bool mem_cgroup_oom(struct mem_cgroup *memcg, gfp_t mask, int order)
 
 	memcg_memory_event(memcg, MEMCG_OOM);
 
-	/*
-	 * We are in the middle of the charge context here, so we
-	 * don't want to block when potentially sitting on a callstack
-	 * that holds all kinds of filesystem and mm locks.
-	 *
-	 * cgroup1 allows disabling the OOM killer and waiting for outside
-	 * handling until the charge can succeed; remember the context and put
-	 * the task to sleep at the end of the page fault when all locks are
-	 * released.
-	 *
-	 * On the other hand, in-kernel OOM killer allows for an async victim
-	 * memory reclaim (oom_reaper) and that means that we are not solely
-	 * relying on the oom victim to make a forward progress and we can
-	 * invoke the oom killer here.
-	 *
-	 * Please note that mem_cgroup_out_of_memory might fail to find a
-	 * victim and then we have to bail out from the charge path.
-	 */
-	if (READ_ONCE(memcg->oom_kill_disable)) {
-		if (current->in_user_fault) {
-			css_get(&memcg->css);
-			current->memcg_in_oom = memcg;

mm/memcontrol.c static bool mem_cgroup_oom(struct mem_cgroup *memcg, gfp_t mask, int order) · line 1780

Quoted 10 lines
mm/memcontrol.c +1 -8
-			current->memcg_oom_gfp_mask = mask;
-			current->memcg_oom_order = order;
-		}
+	if (!mem_cgroup_v1_oom_prepare(memcg, mask, order, &locked))
 		return false;
-	}
-
-	mem_cgroup_mark_under_oom(memcg);
-
-	locked = mem_cgroup_oom_trylock(memcg);

This really confused me because this looks like the oom locking is removed for v2 but this is not the case because mem_cgroup_v1_oom_prepare is not really v1 only code - in other words this is not going to be just return false for CONFIG_MEMCG_V1=n.

Quoted 2 lines

It makes sense to move the userspace oom handling out to the v1 file. I would keep mem_cgroup_mark_under_oom here.

Hm, I don't see any usages of memcg->under_oom outside of v1-specific context. I probably miss something, can you, please, clarify?

Quoted 5 lines

I am not sure about the oom
locking thing because I think we can make it v1 only. For v2 I guess we can go without this locking as the oom path is already locked and it implements overkilling prevention (oom_evaluate_task) as it walks all processes in the oom hierarchy.

It's a good point and not obvious if we really need anything of this on v2. I guess no, but will think a bit more.

Thank you!

author maintainer
#9 lore raw

RFC PATCH 7/9 mm: memcg: put cgroup v1-specific code under a config option

In reply to Roman Gushchin

Put legacy cgroup v1 memory controller code under a new CONFIG_MEMCG_V1 config option. The option is turned on by default as of now to keep things backward-compatible. But users who have fully adopted cgroup v2 and don't use cgroup v1 anymore can turn it off and benefit from a smaller memory footprint and small CPU wins on some memcg paths.

---
 include/linux/memcontrol.h | 13 ++++++++++---
 init/Kconfig               |  7 +++++++
 mm/Makefile                |  3 ++-
 mm/internal.h              | 24 +++++++++++++++++++++++-
 mm/memcontrol.c            | 10 +++++++---
 5 files changed, 49 insertions(+), 8 deletions(-)
include/linux/memcontrol.h +10 -3
index fc4aaa73aa5e..d2a4145b1909 100644
@@ -954,7 +954,14 @@ static inline bool task_in_memcg_oom(struct task_struct *p)
 	return p->memcg_in_oom;
 }
 
+#ifdef CONFIG_MEMCG_V1
 bool mem_cgroup_oom_synchronize(bool wait);
+#else
+static inline bool mem_cgroup_oom_synchronize(bool wait)
+{
+	return false;
+}
+#endif
 struct mem_cgroup *mem_cgroup_get_oom_group(struct task_struct *victim,
 					    struct mem_cgroup *oom_domain);
 void mem_cgroup_print_oom_group(struct mem_cgroup *memcg);
@@ -1872,7 +1879,7 @@ static inline bool mem_cgroup_zswap_writeback_enabled(struct mem_cgroup *memcg)
 
 /* Cgroup v1-specific definitions */
 
-#ifdef CONFIG_MEMCG
+#ifdef CONFIG_MEMCG_V1
 unsigned long mem_cgroup_soft_limit_reclaim(pg_data_t *pgdat, int order,
 						gfp_t gfp_mask,
 						unsigned long *total_scanned);
@@ -1895,7 +1902,7 @@ static inline void mem_cgroup_unlock_pages(void)
 {
 	rcu_read_unlock();
 }
-#else /* CONFIG_MEMCG */
+#else /* CONFIG_MEMCG_V1 */
 
 static inline
 unsigned long mem_cgroup_soft_limit_reclaim(pg_data_t *pgdat, int order,
@@ -1924,6 +1931,6 @@ static inline void mem_cgroup_unlock_pages(void)
 {
 	rcu_read_unlock();
 }
-#endif /* CONFIG_MEMCG */
+#endif /* CONFIG_MEMCG_V1 */
 
 #endif /* _LINUX_MEMCONTROL_H */
init/Kconfig +7 -0
index 10d4a638d9ae..ce9b78279627 100644
@@ -969,6 +969,13 @@ config MEMCG
 	help
 	  Provides control over the memory footprint of tasks in a cgroup.
 
+config MEMCG_V1
+	bool "Legacy memory controller"
+	depends on MEMCG
+	default y
+	help
+	  Legacy cgroup v1 memory controller.
+
 config MEMCG_KMEM
 	bool
 	depends on MEMCG
mm/Makefile +2 -1
index c717a3ee612e..4e2fe5f6637c 100644
@@ -96,7 +96,8 @@ obj-$(CONFIG_NUMA) += memory-tiers.o
 obj-$(CONFIG_DEVICE_MIGRATION) += migrate_device.o
 obj-$(CONFIG_TRANSPARENT_HUGEPAGE) += huge_memory.o khugepaged.o
 obj-$(CONFIG_PAGE_COUNTER) += page_counter.o
-obj-$(CONFIG_MEMCG) += memcontrol.o memcontrol-v1.o vmpressure.o
+obj-$(CONFIG_MEMCG_V1) += memcontrol-v1.o
+obj-$(CONFIG_MEMCG) += memcontrol.o vmpressure.o
 ifdef CONFIG_SWAP
 obj-$(CONFIG_MEMCG) += swap_cgroup.o
 endif
mm/internal.h +23 -1
index 1b94e2169e19..8c5640ef85f8 100644
@@ -1559,7 +1559,6 @@ static inline int try_charge(struct mem_cgroup *memcg, gfp_t gfp_mask,
 }
 
 void mem_cgroup_charge_statistics(struct mem_cgroup *memcg, int nr_pages);
-void memcg_oom_recover(struct mem_cgroup *memcg);
 void mem_cgroup_id_get_many(struct mem_cgroup *memcg, unsigned int n);
 void mem_cgroup_id_put_many(struct mem_cgroup *memcg, unsigned int n);
 
@@ -1589,6 +1588,7 @@ unsigned long memcg_events_local(struct mem_cgroup *memcg, int event);
 void drain_all_stock(struct mem_cgroup *root_memcg);
 
 /* Memory cgroups v1-specific definitions */
+#ifdef CONFIG_MEMCG_V1
 void mem_cgroup_update_tree(struct mem_cgroup *memcg, int nid);
 void mem_cgroup_remove_from_trees(struct mem_cgroup *memcg);
 void mem_cgroup_soft_limit_reset(struct mem_cgroup *memcg);
@@ -1618,9 +1618,31 @@ void memcg1_stat_format(struct mem_cgroup *memcg, struct seq_buf *s);
 bool mem_cgroup_v1_oom_prepare(struct mem_cgroup *memcg, gfp_t mask, int order,
 			       bool *locked);
 void mem_cgroup_v1_oom_finish(struct mem_cgroup *memcg, bool *locked);
+void memcg_oom_recover(struct mem_cgroup *memcg);
 void mem_cgroup_v1_offline_memcg(struct mem_cgroup *memcg);
 
 extern struct cftype memsw_files[];
 extern struct cftype mem_cgroup_legacy_files[];
 
+#else /* CONFIG_MEMCG_V1 */
+static inline void mem_cgroup_remove_from_trees(struct mem_cgroup *memcg) {}
+static inline void mem_cgroup_soft_limit_reset(struct mem_cgroup *memcg) {}
+
+static inline bool mem_cgroup_wait_acct_move(struct mem_cgroup *memcg)
+{
+	return false;
+}
+
+static inline void memcg_check_events(struct mem_cgroup *memcg, int nid) {}
+static inline void memcg1_stat_format(struct mem_cgroup *memcg, struct seq_buf *s) {}
+static inline bool mem_cgroup_v1_oom_prepare(struct mem_cgroup *memcg, gfp_t mask, int order,
+			       bool *locked)
+{
+	return true;
+}
+static inline void mem_cgroup_v1_oom_finish(struct mem_cgroup *memcg, bool *locked) {}
+static inline void memcg_oom_recover(struct mem_cgroup *memcg) {}
+static inline void mem_cgroup_v1_offline_memcg(struct mem_cgroup *memcg) {}
+#endif /* CONFIG_MEMCG_V1 */
+
 #endif	/* __MM_INTERNAL_H */
mm/memcontrol.c +7 -3
index 805efc98ae12..d5883f748330 100644
@@ -4435,18 +4435,20 @@ struct cgroup_subsys memory_cgrp_subsys = {
 	.css_free = mem_cgroup_css_free,
 	.css_reset = mem_cgroup_css_reset,
 	.css_rstat_flush = mem_cgroup_css_rstat_flush,
-	.can_attach = mem_cgroup_can_attach,
 #if defined(CONFIG_LRU_GEN) || defined(CONFIG_MEMCG_KMEM)
 	.attach = mem_cgroup_attach,
 #endif
-	.cancel_attach = mem_cgroup_cancel_attach,
-	.post_attach = mem_cgroup_move_task,
 #ifdef CONFIG_MEMCG_KMEM
 	.fork = mem_cgroup_fork,
 	.exit = mem_cgroup_exit,
 #endif
 	.dfl_cftypes = memory_files,
+#ifdef CONFIG_MEMCG_V1
+	.can_attach = mem_cgroup_can_attach,
+	.cancel_attach = mem_cgroup_cancel_attach,
+	.post_attach = mem_cgroup_move_task,
 	.legacy_cftypes = mem_cgroup_legacy_files,
+#endif
 	.early_init = 0,
 };
 
@@ -5618,7 +5620,9 @@ static int __init mem_cgroup_swap_init(void)
 		return 0;
 
 	WARN_ON(cgroup_add_dfl_cftypes(&memory_cgrp_subsys, swap_files));
+#ifdef CONFIG_MEMCG_V1
 	WARN_ON(cgroup_add_legacy_cftypes(&memory_cgrp_subsys, memsw_files));
+#endif
 #if defined(CONFIG_MEMCG_KMEM) && defined(CONFIG_ZSWAP)
 	WARN_ON(cgroup_add_dfl_cftypes(&memory_cgrp_subsys, zswap_files));
 #endif
Signature · 1 line
2.43.2
author maintainer
#10 lore raw

RFC PATCH 8/9 mm: memcg: put corresponding struct mem_cgroup members under CONFIG_MEMCG_V1

In reply to Roman Gushchin

Put members of struct mem_cgroup which are related to the legacy cgroup v1 memory controller under the CONFIG_MEMCG_V1 config option. Also, put the initialization and some trivial access code under the same option.

---
 include/linux/memcontrol.h | 33 ++++++++++++++++++++-------------
 mm/memcontrol.c            | 25 +++++++++++++++++++------
 2 files changed, 39 insertions(+), 19 deletions(-)
include/linux/memcontrol.h +20 -13
index d2a4145b1909..4347d6889fa0 100644
@@ -182,9 +182,11 @@ struct mem_cgroup {
 		struct page_counter memsw;	/* v1 only */
 	};
 
+#ifdef CONFIG_MEMCG_V1
 	/* Legacy consumer-oriented counters */
 	struct page_counter kmem;		/* v1 only */
 	struct page_counter tcpmem;		/* v1 only */
+#endif
 
 	/* Range enforcement for interrupt charges */
 	struct work_struct high_work;
@@ -198,24 +200,15 @@ struct mem_cgroup {
 	 */
 	bool zswap_writeback;
 #endif
-
-	unsigned long soft_limit;
-
-	/* vmpressure notifications */
-	struct vmpressure vmpressure;
+	int swappiness;
 
 	/*
 	 * Should the OOM killer kill all belonging tasks, had it kill one?
 	 */
 	bool oom_group;
 
-	/* protected by memcg_oom_lock */
-	bool		oom_lock;
-	int		under_oom;
-
-	int	swappiness;
-	/* OOM-Killer disable */
-	int		oom_kill_disable;
+	/* vmpressure notifications */
+	struct vmpressure vmpressure;
 
 	/* memory.events and memory.events.local */
 	struct cgroup_file events_file;
@@ -224,6 +217,16 @@ struct mem_cgroup {
 	/* handle for "memory.swap.events" */
 	struct cgroup_file swap_events_file;
 
+#ifdef CONFIG_MEMCG_V1
+	unsigned long soft_limit;
+
+	/* protected by memcg_oom_lock */
+	bool		oom_lock;
+	int		under_oom;
+
+	/* OOM-Killer disable */
+	int		oom_kill_disable;
+
 	/* protect arrays of thresholds */
 	struct mutex thresholds_lock;
 
@@ -244,6 +247,7 @@ struct mem_cgroup {
 	/* taken only while moving_account > 0 */
 	spinlock_t		move_lock;
 	unsigned long		move_lock_flags;
+#endif
 
 	CACHELINE_PADDING(_pad1_);
 
@@ -279,12 +283,13 @@ struct mem_cgroup {
 #endif
 
 	CACHELINE_PADDING(_pad2_);
-
+#ifdef CONFIG_MEMCG_V1
 	/*
 	 * set > 0 if pages under this cgroup are moving to other cgroup.
 	 */
 	atomic_t		moving_account;
 	struct task_struct	*move_lock_task;
+#endif
 
 	struct memcg_vmstats_percpu __percpu *vmstats_percpu;
 
@@ -294,9 +299,11 @@ struct mem_cgroup {
 	struct memcg_cgwb_frn cgwb_frn[MEMCG_CGWB_FRN_CNT];
 #endif
 
+#ifdef CONFIG_MEMCG_V1
 	/* List of events which userspace want to receive */
 	struct list_head event_list;
 	spinlock_t event_list_lock;
+#endif
 
 #ifdef CONFIG_TRANSPARENT_HUGEPAGE
 	struct deferred_split deferred_split_queue;
mm/memcontrol.c +19 -6
index d5883f748330..4dcfd1ef5d61 100644
@@ -1537,6 +1537,7 @@ void mem_cgroup_print_oom_meminfo(struct mem_cgroup *memcg)
 		pr_info("swap: usage %llukB, limit %llukB, failcnt %lu\n",
 			K((u64)page_counter_read(&memcg->swap)),
 			K((u64)READ_ONCE(memcg->swap.max)), memcg->swap.failcnt);
+#ifdef CONFIG_MEMCG_V1
 	else {
 		pr_info("memory+swap: usage %llukB, limit %llukB, failcnt %lu\n",
 			K((u64)page_counter_read(&memcg->memsw)),
@@ -1545,6 +1546,7 @@ void mem_cgroup_print_oom_meminfo(struct mem_cgroup *memcg)
 			K((u64)page_counter_read(&memcg->kmem)),
 			K((u64)memcg->kmem.max), memcg->kmem.failcnt);
 	}
+#endif
 
 	pr_info("Memory cgroup stats for ");
 	pr_cont_cgroup_path(memcg->css.cgroup);
@@ -2650,12 +2652,14 @@ struct obj_cgroup *get_obj_cgroup_from_folio(struct folio *folio)
 static void memcg_account_kmem(struct mem_cgroup *memcg, int nr_pages)
 {
 	mod_memcg_state(memcg, MEMCG_KMEM, nr_pages);
+#ifdef CONFIG_MEMCG_V1
 	if (!cgroup_subsys_on_dfl(memory_cgrp_subsys)) {
 		if (nr_pages > 0)
 			page_counter_charge(&memcg->kmem, nr_pages);
 		else
 			page_counter_uncharge(&memcg->kmem, -nr_pages);
 	}
+#endif
 }
 
 
@@ -3602,12 +3606,14 @@ static struct mem_cgroup *mem_cgroup_alloc(struct mem_cgroup *parent)
 		goto fail;
 
 	INIT_WORK(&memcg->high_work, high_work_func);
+#ifdef CONFIG_MEMCG_V1
 	INIT_LIST_HEAD(&memcg->oom_notify);
 	mutex_init(&memcg->thresholds_lock);
 	spin_lock_init(&memcg->move_lock);
-	vmpressure_init(&memcg->vmpressure);
 	INIT_LIST_HEAD(&memcg->event_list);
 	spin_lock_init(&memcg->event_list_lock);
+#endif
+	vmpressure_init(&memcg->vmpressure);
 	memcg->socket_pressure = jiffies;
 #ifdef CONFIG_MEMCG_KMEM
 	memcg->kmemcg_id = -1;
@@ -3654,20 +3660,22 @@ mem_cgroup_css_alloc(struct cgroup_subsys_state *parent_css)
 	page_counter_set_high(&memcg->swap, PAGE_COUNTER_MAX);
 	if (parent) {
 		WRITE_ONCE(memcg->swappiness, mem_cgroup_swappiness(parent));
-		WRITE_ONCE(memcg->oom_kill_disable, READ_ONCE(parent->oom_kill_disable));
-
 		page_counter_init(&memcg->memory, &parent->memory);
 		page_counter_init(&memcg->swap, &parent->swap);
+#ifdef CONFIG_MEMCG_V1
 		page_counter_init(&memcg->kmem, &parent->kmem);
 		page_counter_init(&memcg->tcpmem, &parent->tcpmem);
+		WRITE_ONCE(memcg->oom_kill_disable, READ_ONCE(parent->oom_kill_disable));
+#endif
 	} else {
 		init_memcg_stats();
 		init_memcg_events();
 		page_counter_init(&memcg->memory, NULL);
 		page_counter_init(&memcg->swap, NULL);
+#ifdef CONFIG_MEMCG_V1
 		page_counter_init(&memcg->kmem, NULL);
 		page_counter_init(&memcg->tcpmem, NULL);
-
+#endif
 		root_mem_cgroup = memcg;
 		return &memcg->css;
 	}
@@ -3802,12 +3810,14 @@ static void mem_cgroup_css_reset(struct cgroup_subsys_state *css)
 
 	page_counter_set_max(&memcg->memory, PAGE_COUNTER_MAX);
 	page_counter_set_max(&memcg->swap, PAGE_COUNTER_MAX);
+#ifdef CONFIG_MEMCG_V1
 	page_counter_set_max(&memcg->kmem, PAGE_COUNTER_MAX);
 	page_counter_set_max(&memcg->tcpmem, PAGE_COUNTER_MAX);
+	mem_cgroup_soft_limit_reset(memcg);
+#endif
 	page_counter_set_min(&memcg->memory, 0);
 	page_counter_set_low(&memcg->memory, 0);
 	page_counter_set_high(&memcg->memory, PAGE_COUNTER_MAX);
-	mem_cgroup_soft_limit_reset(memcg);
 	page_counter_set_high(&memcg->swap, PAGE_COUNTER_MAX);
 	memcg_wb_domain_size_changed(memcg);
 }
@@ -5018,6 +5028,7 @@ void mem_cgroup_sk_free(struct sock *sk)
 bool mem_cgroup_charge_skmem(struct mem_cgroup *memcg, unsigned int nr_pages,
 			     gfp_t gfp_mask)
 {
+#ifdef CONFIG_MEMCG_V1
 	if (!cgroup_subsys_on_dfl(memory_cgrp_subsys)) {
 		struct page_counter *fail;
 
@@ -5032,6 +5043,7 @@ bool mem_cgroup_charge_skmem(struct mem_cgroup *memcg, unsigned int nr_pages,
 		}
 		return false;
 	}
+#endif
 
 	if (try_charge(memcg, gfp_mask, nr_pages) == 0) {
 		mod_memcg_state(memcg, MEMCG_SOCK, nr_pages);
@@ -5048,11 +5060,12 @@ bool mem_cgroup_charge_skmem(struct mem_cgroup *memcg, unsigned int nr_pages,
  */
 void mem_cgroup_uncharge_skmem(struct mem_cgroup *memcg, unsigned int nr_pages)
 {
+#ifdef CONFIG_MEMCG_V1
 	if (!cgroup_subsys_on_dfl(memory_cgrp_subsys)) {
 		page_counter_uncharge(&memcg->tcpmem, nr_pages);
 		return;
 	}
-
+#endif
 	mod_memcg_state(memcg, MEMCG_SOCK, -nr_pages);
 
 	refill_stock(memcg, nr_pages);
Signature · 1 line
2.43.2
author maintainer
#11 lore raw

RFC PATCH 9/9 mm: memcg: put cgroup v1-related members of task_struct under config option

In reply to Roman Gushchin

Guard cgroup v1-related members of task_struct under the CONFIG_MEMCG_V1 config option, so that users who adopted cgroup v2 don't have to waste the memory for fields which are never accessed.

---
 include/linux/memcontrol.h | 11 ++++++++++-
 include/linux/sched.h      |  5 ++++-
 2 files changed, 14 insertions(+), 2 deletions(-)
include/linux/memcontrol.h +10 -1
index 4347d6889fa0..8005d749f8fc 100644
@@ -944,6 +944,7 @@ void mem_cgroup_print_oom_context(struct mem_cgroup *memcg,
 
 void mem_cgroup_print_oom_meminfo(struct mem_cgroup *memcg);
 
+#ifdef CONFIG_MEMCG_V1
 static inline void mem_cgroup_enter_user_fault(void)
 {
 	WARN_ON(current->in_user_fault);
@@ -961,9 +962,17 @@ static inline bool task_in_memcg_oom(struct task_struct *p)
 	return p->memcg_in_oom;
 }
 
-#ifdef CONFIG_MEMCG_V1
 bool mem_cgroup_oom_synchronize(bool wait);
 #else
+static inline void mem_cgroup_enter_user_fault(void) {}
+
+static inline void mem_cgroup_exit_user_fault(void) {}
+
+static inline bool task_in_memcg_oom(struct task_struct *p)
+{
+	return false;
+}
+
 static inline bool mem_cgroup_oom_synchronize(bool wait)
 {
 	return false;
include/linux/sched.h +4 -1
index 4118b3f959c3..2ecdeb7588e3 100644
@@ -1446,11 +1446,14 @@ struct task_struct {
 	unsigned int			kcov_softirq;
 #endif
 
-#ifdef CONFIG_MEMCG
+
+#ifdef CONFIG_MEMCG_V1
 	struct mem_cgroup		*memcg_in_oom;
 	gfp_t				memcg_oom_gfp_mask;
 	int				memcg_oom_order;
+#endif
 
+#ifdef CONFIG_MEMCG
 	/* Number of pages to reclaim on returning to userland: */
 	unsigned int			memcg_nr_pages_over_high;
 
Signature · 1 line
2.43.2
maintainer
#12 lore raw
In reply to Roman Gushchin
Quoted 29 lines · Roman Gushchin

Cgroups v2 have been around for a while and many users have fully adopted them, so they never use cgroups v1 features and functionality. Yet they have to "pay" for the cgroup v1 support anyway:
1) the kernel binary contains useless cgroup v1 code,
2) some common structures like task_struct and mem_cgroup have never used cgroup v1-specific members,
3) some code paths have additional checks which are not needed.

Cgroup v1's memory controller has a number of features that are not supported by cgroup v2 and their implementation is pretty much self contained. Most notably, these features are: soft limit reclaim, oom handling in userspace, complicated event notification system, charge migration.

Cgroup v1-specific code in memcontrol.c is close to 4k lines in size and it's intervened with generic and cgroup v2-specific code. It's a burden on developers and maintainers.

This patchset aims to solve these problems by:
1) moving cgroup v1-specific memcg code to the new mm/memcontrol-v1.c file,
2) putting definitions shared by memcontrol.c and memcontrol-v1.c into the mm/internal.h header
3) introducing the CONFIG_MEMCG_V1 config option, turned on by default
4) making memcontrol-v1.c to compile only if CONFIG_MEMCG_V1 is set
5) putting unused struct memory_cgroup and task_struct members under CONFIG_MEMCG_V1 as well.

This is an RFC version, which is not 100% polished yet, so but it would be great to discuss and agree on the overall approach.

Quoted 7 lines

Some open questions, opinions are appreciated:
1) I consider renaming non-static functions in memcontrol-v1.c to have mem_cgroup_v1_ prefix. Is this a good idea?
2) Do we want to extend it beyond the memory controller? Should
3) Is it better to use a new include/linux/memcontrol-v1.h instead of mm/internal.h? Or mm/memcontrol-v1.h.

Hi Roman,

A very timely and important topic and we should definitely talk about it during LSFMM as well. I have been thinking about this problem for quite sometime and I am getting more and more convinced that we should aim to completely deprecate memcg-v1.

More specifically:

1. What are the memcg-v1 features which have no alternative in memcg-v2 and are blocker for memcg-v1 users? (setting aside the cgroup v2 structual restrictions)

2. What are unused memcg-v1 features which we should start deprecating?

IMO we should systematically start deprecating memcg-v1 features and start unblocking the users stuck on memcg-v1.

Now regarding the proposal in this series, I think it can be a first step but should not give an impression that we are done. The only concern I have is the potential of "out of sight, out of mind" situation with this change but if we keep the momentum of deprecation of memcg-v1 it should be fine.

I have CCed Greg and David from Google to get their opinion on what memcg-v1 features are blocker for their memcg-v2 migration and if they have concern in deprecation of memcg-v1 features.

Anyone else still on memcg-v1, please do provide your input.

thanks,
Shakeel

author maintainer
#13 lore raw
Quoted 45 lines · Shakeel Butt
Quoted 29 lines · Roman Gushchin

Cgroups v2 have been around for a while and many users have fully adopted them, so they never use cgroups v1 features and functionality. Yet they have to "pay" for the cgroup v1 support anyway:
1) the kernel binary contains useless cgroup v1 code,
2) some common structures like task_struct and mem_cgroup have never used cgroup v1-specific members,
3) some code paths have additional checks which are not needed.

Cgroup v1's memory controller has a number of features that are not supported by cgroup v2 and their implementation is pretty much self contained. Most notably, these features are: soft limit reclaim, oom handling in userspace, complicated event notification system, charge migration.

Cgroup v1-specific code in memcontrol.c is close to 4k lines in size and it's intervened with generic and cgroup v2-specific code. It's a burden on developers and maintainers.

This patchset aims to solve these problems by:
1) moving cgroup v1-specific memcg code to the new mm/memcontrol-v1.c file,
2) putting definitions shared by memcontrol.c and memcontrol-v1.c into the mm/internal.h header
3) introducing the CONFIG_MEMCG_V1 config option, turned on by default
4) making memcontrol-v1.c to compile only if CONFIG_MEMCG_V1 is set
5) putting unused struct memory_cgroup and task_struct members under CONFIG_MEMCG_V1 as well.

This is an RFC version, which is not 100% polished yet, so but it would be great to discuss and agree on the overall approach.

Quoted 7 lines

Some open questions, opinions are appreciated:
1) I consider renaming non-static functions in memcontrol-v1.c to have mem_cgroup_v1_ prefix. Is this a good idea?
2) Do we want to extend it beyond the memory controller? Should
3) Is it better to use a new include/linux/memcontrol-v1.h instead of mm/internal.h? Or mm/memcontrol-v1.h.

Hi Roman,

A very timely and important topic and we should definitely talk about it during LSFMM as well. I have been thinking about this problem for quite sometime and I am getting more and more convinced that we should aim to completely deprecate memcg-v1.

Quoted 5 lines

More specifically:

1. What are the memcg-v1 features which have no alternative in memcg-v2 and are blocker for memcg-v1 users? (setting aside the cgroup v2 structual restrictions)

I don't think there are any, except there might be a certain cost to migrate, so some companies might be resistant to put in resources, because they don't see any immediate benefits as well.

Quoted 5 lines

2. What are unused memcg-v1 features which we should start deprecating?

IMO we should systematically start deprecating memcg-v1 features and start unblocking the users stuck on memcg-v1.

I'm not sure we want to deprecate them one-by-one - it's a lot of work and maybe we can deprecate it all together instead.

I think the only feature which we might want to deprecate separately - it's the charge migration. It's the most annoying feature as it requires a lot more synchronization, which can be dropped otherwise, so it's complicating a lot of things. Other features are more or less self-contained.

Quoted 3 lines

Now regarding the proposal in this series, I think it can be a first step but should not give an impression that we are done.

Yeah, it's really only a first step.

Quoted 4 lines

The only
concern I have is the potential of "out of sight, out of mind" situation with this change but if we keep the momentum of deprecation of memcg-v1 it should be fine.

My rough plan here:
1) move it out to a separate file and put under a config option, default on
2) clean up all remaining small bits here and there
... < wait a year >
3) flip the config option to be off by default
... < wait another year or two >
4) drop the code entirely

Quoted 4 lines

I have CCed Greg and David from Google to get their opinion on what memcg-v1 features are blocker for their memcg-v2 migration and if they have concern in deprecation of memcg-v1 features.

Thank you!

#14 lore raw
In reply to Shakeel Butt
Quoted 7 lines · Shakeel Butt

Hi Roman,

A very timely and important topic and we should definitely talk about it during LSFMM as well. I have been thinking about this problem for quite sometime and I am getting more and more convinced that we should aim to completely deprecate memcg-v1.

I think this would be a very worthwhile discussion at LSF/MM, I'm not sure if it would be too late for someone to make a formal proposal for it to be included in the schedule. Michal would know if there is a opportunity.

I say that in light of
https://lore.kernel.org/bpf/[email protected]/T/#mb6c21b09543c434dd85e718a8ecf5ca6485e6d07
as well for the whole cgroup v1 -> v2 transition.

Chris, now cc'd, would know best about all of the dependencies that Google has for memcg specifically.

Quoted 26 lines

More specifically:

1. What are the memcg-v1 features which have no alternative in memcg-v2 and are blocker for memcg-v1 users? (setting aside the cgroup v2 structual restrictions)

2. What are unused memcg-v1 features which we should start deprecating?

IMO we should systematically start deprecating memcg-v1 features and start unblocking the users stuck on memcg-v1.

Now regarding the proposal in this series, I think it can be a first step but should not give an impression that we are done. The only concern I have is the potential of "out of sight, out of mind" situation with this change but if we keep the momentum of deprecation of memcg-v1 it should be fine.

I have CCed Greg and David from Google to get their opinion on what memcg-v1 features are blocker for their memcg-v2 migration and if they have concern in deprecation of memcg-v1 features.

Anyone else still on memcg-v1, please do provide your input.

thanks,
Shakeel

#15 lore raw
Quoted 15 lines · David Rientjes
Quoted 7 lines · Shakeel Butt

Hi Roman,

A very timely and important topic and we should definitely talk about it during LSFMM as well. I have been thinking about this problem for quite sometime and I am getting more and more convinced that we should aim to completely deprecate memcg-v1.

I think this would be a very worthwhile discussion at LSF/MM, I'm not sure if it would be too late for someone to make a formal proposal for it to be included in the schedule. Michal would know if there is a opportunity.

Quoted 6 lines

I say that in light of
https://lore.kernel.org/bpf/[email protected]/T/#mb6c21b09543c434dd85e718a8ecf5ca6485e6d07
as well for the whole cgroup v1 -> v2 transition.

Chris, now cc'd, would know best about all of the dependencies that Google has for memcg specifically.

Thanks David,

Yes, I am very interested in that cgroup v1 -> v2 transition discussion.

Quoted 6 lines
Quoted 6 lines

More specifically:

1. What are the memcg-v1 features which have no alternative in memcg-v2 and are blocker for memcg-v1 users? (setting aside the cgroup v2 structual restrictions)

In the list mentioned by Roman: "soft limit reclaim, oom handling in userspace, complicated event notification system, charge migration."

The "oom.control" and leak of user space oom control is a big one for google. Some test frameworks also use "memory.force_empty".
Soft limit reclaim and charge migration is also used.

There is also the combined "memsw" limit enforcement. Google has some internal work around for V2 but it would be good if that upstream can support it directly.

BTW, I know you are not looking for the "cgroup v2 structure restrictions". Two cgroup controllers that can't have different sets of processes is a bit too restrictive.

That is what I recall right now, I might be missing some small odd items.

Anyway, glad to join the discussion if there is a session.

Chris

Chris

Quoted 21 lines
Quoted 21 lines

2. What are unused memcg-v1 features which we should start deprecating?

IMO we should systematically start deprecating memcg-v1 features and start unblocking the users stuck on memcg-v1.

Now regarding the proposal in this series, I think it can be a first step but should not give an impression that we are done. The only concern I have is the potential of "out of sight, out of mind" situation with this change but if we keep the momentum of deprecation of memcg-v1 it should be fine.

I have CCed Greg and David from Google to get their opinion on what memcg-v1 features are blocker for their memcg-v2 migration and if they have concern in deprecation of memcg-v1 features.

Anyone else still on memcg-v1, please do provide your input.

thanks,
Shakeel

reviewer
#16 lore raw
In reply to David Rientjes
Quoted 10 lines · David Rientjes
Quoted 7 lines · Shakeel Butt

Hi Roman,

A very timely and important topic and we should definitely talk about it during LSFMM as well. I have been thinking about this problem for quite sometime and I am getting more and more convinced that we should aim to completely deprecate memcg-v1.

Quoted 3 lines

I think this would be a very worthwhile discussion at LSF/MM, I'm not sure if it would be too late for someone to make a formal proposal for it to be included in the schedule. Michal would know if there is a opportunity.

yes, I think we can and should have this discussion. I will put that on the schedule. I will reference this email thread as a topic proposal with Shakeel and Roman to lead the session.

Signature · 2 lines
Michal Hocko
SUSE Labs
#17 lore raw
In reply to Shakeel Butt
Quoted 63 lines · Shakeel Butt
Quoted 29 lines · Roman Gushchin

Cgroups v2 have been around for a while and many users have fully adopted them, so they never use cgroups v1 features and functionality. Yet they have to "pay" for the cgroup v1 support anyway:
1) the kernel binary contains useless cgroup v1 code,
2) some common structures like task_struct and mem_cgroup have never used cgroup v1-specific members,
3) some code paths have additional checks which are not needed.

Cgroup v1's memory controller has a number of features that are not supported by cgroup v2 and their implementation is pretty much self contained. Most notably, these features are: soft limit reclaim, oom handling in userspace, complicated event notification system, charge migration.

Cgroup v1-specific code in memcontrol.c is close to 4k lines in size and it's intervened with generic and cgroup v2-specific code. It's a burden on developers and maintainers.

This patchset aims to solve these problems by:
1) moving cgroup v1-specific memcg code to the new mm/memcontrol-v1.c file,
2) putting definitions shared by memcontrol.c and memcontrol-v1.c into the mm/internal.h header
3) introducing the CONFIG_MEMCG_V1 config option, turned on by default
4) making memcontrol-v1.c to compile only if CONFIG_MEMCG_V1 is set
5) putting unused struct memory_cgroup and task_struct members under CONFIG_MEMCG_V1 as well.

This is an RFC version, which is not 100% polished yet, so but it would be great to discuss and agree on the overall approach.

Quoted 7 lines

Some open questions, opinions are appreciated:
1) I consider renaming non-static functions in memcontrol-v1.c to have mem_cgroup_v1_ prefix. Is this a good idea?
2) Do we want to extend it beyond the memory controller? Should
3) Is it better to use a new include/linux/memcontrol-v1.h instead of mm/internal.h? Or mm/memcontrol-v1.h.

Hi Roman,

A very timely and important topic and we should definitely talk about it during LSFMM as well. I have been thinking about this problem for quite sometime and I am getting more and more convinced that we should aim to completely deprecate memcg-v1.

More specifically:

1. What are the memcg-v1 features which have no alternative in memcg-v2 and are blocker for memcg-v1 users? (setting aside the cgroup v2 structual restrictions)

2. What are unused memcg-v1 features which we should start deprecating?

IMO we should systematically start deprecating memcg-v1 features and start unblocking the users stuck on memcg-v1.

Now regarding the proposal in this series, I think it can be a first step but should not give an impression that we are done. The only concern I have is the potential of "out of sight, out of mind" situation with this change but if we keep the momentum of deprecation of memcg-v1 it should be fine.

Quoted 5 lines

I have CCed Greg and David from Google to get their opinion on what memcg-v1 features are blocker for their memcg-v2 migration and if they have concern in deprecation of memcg-v1 features.

Anyone else still on memcg-v1, please do provide your input.

Hi Shakeel,

Hopefully I'm not too late. We are currently using memcg v1.

One specific feature we rely on in v1 is skmem accounting. In v1, we account for TCP memory usage without charging it to memcg v1, which is useful for monitoring the TCP memory usage generated by tasks running in a container. However, in memcg v2, monitoring TCP memory requires charging it to the container, which can easily cause OOM issues. It would be better if we could monitor skmem usage without charging it in the memcg v2, allowing us to account for it without the risk of triggering OOM conditions.

Signature · 2 lines
Regards
Yafang
author maintainer
#18 lore raw
Quoted 74 lines · Yafang Shao
Quoted 63 lines · Shakeel Butt
Quoted 29 lines · Roman Gushchin

Cgroups v2 have been around for a while and many users have fully adopted them, so they never use cgroups v1 features and functionality. Yet they have to "pay" for the cgroup v1 support anyway:
1) the kernel binary contains useless cgroup v1 code,
2) some common structures like task_struct and mem_cgroup have never used cgroup v1-specific members,
3) some code paths have additional checks which are not needed.

Cgroup v1's memory controller has a number of features that are not supported by cgroup v2 and their implementation is pretty much self contained. Most notably, these features are: soft limit reclaim, oom handling in userspace, complicated event notification system, charge migration.

Cgroup v1-specific code in memcontrol.c is close to 4k lines in size and it's intervened with generic and cgroup v2-specific code. It's a burden on developers and maintainers.

This patchset aims to solve these problems by:
1) moving cgroup v1-specific memcg code to the new mm/memcontrol-v1.c file,
2) putting definitions shared by memcontrol.c and memcontrol-v1.c into the mm/internal.h header
3) introducing the CONFIG_MEMCG_V1 config option, turned on by default
4) making memcontrol-v1.c to compile only if CONFIG_MEMCG_V1 is set
5) putting unused struct memory_cgroup and task_struct members under CONFIG_MEMCG_V1 as well.

This is an RFC version, which is not 100% polished yet, so but it would be great to discuss and agree on the overall approach.

Quoted 7 lines

Some open questions, opinions are appreciated:
1) I consider renaming non-static functions in memcontrol-v1.c to have mem_cgroup_v1_ prefix. Is this a good idea?
2) Do we want to extend it beyond the memory controller? Should
3) Is it better to use a new include/linux/memcontrol-v1.h instead of mm/internal.h? Or mm/memcontrol-v1.h.

Hi Roman,

A very timely and important topic and we should definitely talk about it during LSFMM as well. I have been thinking about this problem for quite sometime and I am getting more and more convinced that we should aim to completely deprecate memcg-v1.

More specifically:

1. What are the memcg-v1 features which have no alternative in memcg-v2 and are blocker for memcg-v1 users? (setting aside the cgroup v2 structual restrictions)

2. What are unused memcg-v1 features which we should start deprecating?

IMO we should systematically start deprecating memcg-v1 features and start unblocking the users stuck on memcg-v1.

Now regarding the proposal in this series, I think it can be a first step but should not give an impression that we are done. The only concern I have is the potential of "out of sight, out of mind" situation with this change but if we keep the momentum of deprecation of memcg-v1 it should be fine.

Quoted 5 lines

I have CCed Greg and David from Google to get their opinion on what memcg-v1 features are blocker for their memcg-v2 migration and if they have concern in deprecation of memcg-v1 features.

Anyone else still on memcg-v1, please do provide your input.

Hi Shakeel,

Hopefully I'm not too late. We are currently using memcg v1.

Quoted 8 lines

One specific feature we rely on in v1 is skmem accounting. In v1, we account for TCP memory usage without charging it to memcg v1, which is useful for monitoring the TCP memory usage generated by tasks running in a container. However, in memcg v2, monitoring TCP memory requires charging it to the container, which can easily cause OOM issues. It would be better if we could monitor skmem usage without charging it in the memcg v2, allowing us to account for it without the risk of triggering OOM conditions.

Hi Yafang,

the data itself is available on cgroup v2 in memory.stat:sock, however you're right, it's charged on pair with other types of memory. It was one of the main principles of cgroup v2's memory controller, so I don't think it can be changed.

So the feature you need is not skmem accounting, but something quite opposite :)

The question I have here: what makes socket memory different here?

Is it something specific to your setup (e.g. you mostly use memory.max to protect against memory leaks in the userspace code, but socket memory spikes are always caused by external traffic and are legit) or we have more fundamental problems with the socket memory handling, e.g. we can't effectively reclaim it under the memory pressure?

In the first case you can maintain a ~2-lines non-upstream patch which will disable the charging while maintaining statistics - it's not a perfect, but likely the best option here. In the second case we need collectively fix it for cgroup v2.

Thanks!

#19 lore raw
Quoted 95 lines · Roman Gushchin
Quoted 74 lines · Yafang Shao
Quoted 63 lines · Shakeel Butt
Quoted 29 lines · Roman Gushchin

Cgroups v2 have been around for a while and many users have fully adopted them, so they never use cgroups v1 features and functionality. Yet they have to "pay" for the cgroup v1 support anyway:
1) the kernel binary contains useless cgroup v1 code,
2) some common structures like task_struct and mem_cgroup have never used cgroup v1-specific members,
3) some code paths have additional checks which are not needed.

Cgroup v1's memory controller has a number of features that are not supported by cgroup v2 and their implementation is pretty much self contained. Most notably, these features are: soft limit reclaim, oom handling in userspace, complicated event notification system, charge migration.

Cgroup v1-specific code in memcontrol.c is close to 4k lines in size and it's intervened with generic and cgroup v2-specific code. It's a burden on developers and maintainers.

This patchset aims to solve these problems by:
1) moving cgroup v1-specific memcg code to the new mm/memcontrol-v1.c file,
2) putting definitions shared by memcontrol.c and memcontrol-v1.c into the mm/internal.h header
3) introducing the CONFIG_MEMCG_V1 config option, turned on by default
4) making memcontrol-v1.c to compile only if CONFIG_MEMCG_V1 is set
5) putting unused struct memory_cgroup and task_struct members under CONFIG_MEMCG_V1 as well.

This is an RFC version, which is not 100% polished yet, so but it would be great to discuss and agree on the overall approach.

Quoted 7 lines

Some open questions, opinions are appreciated:
1) I consider renaming non-static functions in memcontrol-v1.c to have mem_cgroup_v1_ prefix. Is this a good idea?
2) Do we want to extend it beyond the memory controller? Should
3) Is it better to use a new include/linux/memcontrol-v1.h instead of mm/internal.h? Or mm/memcontrol-v1.h.

Hi Roman,

A very timely and important topic and we should definitely talk about it during LSFMM as well. I have been thinking about this problem for quite sometime and I am getting more and more convinced that we should aim to completely deprecate memcg-v1.

More specifically:

1. What are the memcg-v1 features which have no alternative in memcg-v2 and are blocker for memcg-v1 users? (setting aside the cgroup v2 structual restrictions)

2. What are unused memcg-v1 features which we should start deprecating?

IMO we should systematically start deprecating memcg-v1 features and start unblocking the users stuck on memcg-v1.

Now regarding the proposal in this series, I think it can be a first step but should not give an impression that we are done. The only concern I have is the potential of "out of sight, out of mind" situation with this change but if we keep the momentum of deprecation of memcg-v1 it should be fine.

Quoted 5 lines

I have CCed Greg and David from Google to get their opinion on what memcg-v1 features are blocker for their memcg-v2 migration and if they have concern in deprecation of memcg-v1 features.

Anyone else still on memcg-v1, please do provide your input.

Hi Shakeel,

Hopefully I'm not too late. We are currently using memcg v1.

Quoted 8 lines

One specific feature we rely on in v1 is skmem accounting. In v1, we account for TCP memory usage without charging it to memcg v1, which is useful for monitoring the TCP memory usage generated by tasks running in a container. However, in memcg v2, monitoring TCP memory requires charging it to the container, which can easily cause OOM issues. It would be better if we could monitor skmem usage without charging it in the memcg v2, allowing us to account for it without the risk of triggering OOM conditions.

Hi Yafang,

the data itself is available on cgroup v2 in memory.stat:sock, however you're right, it's charged on pair with other types of memory. It was one of the main principles of cgroup v2's memory controller, so I don't think it can be changed.

So the feature you need is not skmem accounting, but something quite opposite :)

Quoted 7 lines

The question I have here: what makes socket memory different here?

Is it something specific to your setup (e.g. you mostly use memory.max to protect against memory leaks in the userspace code, but socket memory spikes are always caused by external traffic and are legit) or we have more fundamental problems with the socket memory handling, e.g. we can't effectively reclaim it under the memory pressure?

It is the first case.

Quoted 6 lines

In the first case you can maintain a ~2-lines non-upstream patch which will disable the charging while maintaining statistics - it's not a perfect, but likely the best option here. In the second case we need collectively fix it for cgroup v2.

Thank you for your advice. Currently, we do not have any immediate plans to migrate to cgroup v2. If we are required to use cgroup v2 in the future, we will need to maintain non-upstream patches.

By the way, is there any reason we cannot keep this behavior consistent with memcg v1 in the upstream kernel? That would save us from having to maintain it locally.

Signature · 2 lines
Regards
Yafang
author maintainer
#20 lore raw
Quoted 113 lines · Yafang Shao
Quoted 95 lines · Roman Gushchin
Quoted 74 lines · Yafang Shao
Quoted 63 lines · Shakeel Butt
Quoted 29 lines · Roman Gushchin

Cgroups v2 have been around for a while and many users have fully adopted them, so they never use cgroups v1 features and functionality. Yet they have to "pay" for the cgroup v1 support anyway:
1) the kernel binary contains useless cgroup v1 code,
2) some common structures like task_struct and mem_cgroup have never used cgroup v1-specific members,
3) some code paths have additional checks which are not needed.

Cgroup v1's memory controller has a number of features that are not supported by cgroup v2 and their implementation is pretty much self contained. Most notably, these features are: soft limit reclaim, oom handling in userspace, complicated event notification system, charge migration.

Cgroup v1-specific code in memcontrol.c is close to 4k lines in size and it's intervened with generic and cgroup v2-specific code. It's a burden on developers and maintainers.

This patchset aims to solve these problems by:
1) moving cgroup v1-specific memcg code to the new mm/memcontrol-v1.c file,
2) putting definitions shared by memcontrol.c and memcontrol-v1.c into the mm/internal.h header
3) introducing the CONFIG_MEMCG_V1 config option, turned on by default
4) making memcontrol-v1.c to compile only if CONFIG_MEMCG_V1 is set
5) putting unused struct memory_cgroup and task_struct members under CONFIG_MEMCG_V1 as well.

This is an RFC version, which is not 100% polished yet, so but it would be great to discuss and agree on the overall approach.

Quoted 7 lines

Some open questions, opinions are appreciated:
1) I consider renaming non-static functions in memcontrol-v1.c to have mem_cgroup_v1_ prefix. Is this a good idea?
2) Do we want to extend it beyond the memory controller? Should
3) Is it better to use a new include/linux/memcontrol-v1.h instead of mm/internal.h? Or mm/memcontrol-v1.h.

Hi Roman,

A very timely and important topic and we should definitely talk about it during LSFMM as well. I have been thinking about this problem for quite sometime and I am getting more and more convinced that we should aim to completely deprecate memcg-v1.

More specifically:

1. What are the memcg-v1 features which have no alternative in memcg-v2 and are blocker for memcg-v1 users? (setting aside the cgroup v2 structual restrictions)

2. What are unused memcg-v1 features which we should start deprecating?

IMO we should systematically start deprecating memcg-v1 features and start unblocking the users stuck on memcg-v1.

Now regarding the proposal in this series, I think it can be a first step but should not give an impression that we are done. The only concern I have is the potential of "out of sight, out of mind" situation with this change but if we keep the momentum of deprecation of memcg-v1 it should be fine.

Quoted 5 lines

I have CCed Greg and David from Google to get their opinion on what memcg-v1 features are blocker for their memcg-v2 migration and if they have concern in deprecation of memcg-v1 features.

Anyone else still on memcg-v1, please do provide your input.

Hi Shakeel,

Hopefully I'm not too late. We are currently using memcg v1.

Quoted 8 lines

One specific feature we rely on in v1 is skmem accounting. In v1, we account for TCP memory usage without charging it to memcg v1, which is useful for monitoring the TCP memory usage generated by tasks running in a container. However, in memcg v2, monitoring TCP memory requires charging it to the container, which can easily cause OOM issues. It would be better if we could monitor skmem usage without charging it in the memcg v2, allowing us to account for it without the risk of triggering OOM conditions.

Hi Yafang,

the data itself is available on cgroup v2 in memory.stat:sock, however you're right, it's charged on pair with other types of memory. It was one of the main principles of cgroup v2's memory controller, so I don't think it can be changed.

So the feature you need is not skmem accounting, but something quite opposite :)

Quoted 7 lines

The question I have here: what makes socket memory different here?

Is it something specific to your setup (e.g. you mostly use memory.max to protect against memory leaks in the userspace code, but socket memory spikes are always caused by external traffic and are legit) or we have more fundamental problems with the socket memory handling, e.g. we can't effectively reclaim it under the memory pressure?

It is the first case.

Quoted 6 lines

In the first case you can maintain a ~2-lines non-upstream patch which will disable the charging while maintaining statistics - it's not a perfect, but likely the best option here. In the second case we need collectively fix it for cgroup v2.

Quoted 7 lines

Thank you for your advice. Currently, we do not have any immediate plans to migrate to cgroup v2. If we are required to use cgroup v2 in the future, we will need to maintain non-upstream patches.

By the way, is there any reason we cannot keep this behavior consistent with memcg v1 in the upstream kernel? That would save us from having to maintain it locally.

The idea to handle various types of memory independently isn't working well for most users: it makes the configuration trickier and more fragile. It's also more expensive in terms of the accounting overhead.

The tcpmem accounting is btw quite expensive by itself. So by switching to cgroup v2 you might see (depending on your traffic and cpu load) some nice performance benefits.

Thanks!

maintainer
#21 lore raw
In reply to Yafang Shao
Quoted 3 lines · Yafang Shao
Quoted 1 line · Shakeel Butt

[...]

Quoted 4 lines

Hi Shakeel,

Hopefully I'm not too late. We are currently using memcg v1.

Quoted 9 lines

One specific feature we rely on in v1 is skmem accounting. In v1, we account for TCP memory usage without charging it to memcg v1, which is useful for monitoring the TCP memory usage generated by tasks running in a container. However, in memcg v2, monitoring TCP memory requires charging it to the container, which can easily cause OOM issues. It would be better if we could monitor skmem usage without charging it in the memcg v2, allowing us to account for it without the risk of triggering OOM conditions.

Hi Yafang,

No worries. From what I understand, you are not really using skmem charging of v1 but just the network memory usage stats and you are worried that charging network memory to cgroup memory may cause OOMs. Is that correct? Have you tried charging network memory to cgroup memory before and saw OOMs? If yes then I would really like to see OOM reports.

I have two examples where the v2's skmem charging is working fine in production namely Google and Meta. Google is still on v1 but for skmem charging, they have moved to v2 semantics. Actually I have another report from Cloudflare [0] where the tcp throttling mechanism for v2's tcp memory accounting is too much conservative for their production traffic.

Anyways this just means that we need a more flexible way to provide and enforce semantics for tcp memory pressure with a decent default behavior. I will followup on this separately.

[0] https://lore.kernel.org/lkml/CABWYdi0G7cyNFbndM-ELTDAR3x4Ngm0AehEp5aP0tfNkXUE+Uw@mail.gmail.com/

thanks,
Shakeel

#22 lore raw
Quoted 20 lines · Shakeel Butt
Quoted 3 lines · Yafang Shao
Quoted 1 line · Shakeel Butt

[...]

Quoted 4 lines

Hi Shakeel,

Hopefully I'm not too late. We are currently using memcg v1.

Quoted 9 lines

One specific feature we rely on in v1 is skmem accounting. In v1, we account for TCP memory usage without charging it to memcg v1, which is useful for monitoring the TCP memory usage generated by tasks running in a container. However, in memcg v2, monitoring TCP memory requires charging it to the container, which can easily cause OOM issues. It would be better if we could monitor skmem usage without charging it in the memcg v2, allowing us to account for it without the risk of triggering OOM conditions.

Quoted 6 lines

Hi Yafang,

No worries. From what I understand, you are not really using skmem charging of v1 but just the network memory usage stats and you are worried that charging network memory to cgroup memory may cause OOMs. Is that correct?

Correct.

Quoted 2 lines

Have you tried charging network memory to cgroup memory before and saw OOMs? If yes then I would really like to see OOM reports.

No, we don't enable the charging for TCP memory in memcg v1 and we don't have a plan to add support for it currently.

Quoted 8 lines

I have two examples where the v2's skmem charging is working fine in production namely Google and Meta. Google is still on v1 but for skmem charging, they have moved to v2 semantics. Actually I have another report from Cloudflare [0] where the tcp throttling mechanism for v2's tcp memory accounting is too much conservative for their production traffic.

Quoted 5 lines

Anyways this just means that we need a more flexible way to provide and enforce semantics for tcp memory pressure with a decent default behavior. I will followup on this separately.

[0] https://lore.kernel.org/lkml/CABWYdi0G7cyNFbndM-ELTDAR3x4Ngm0AehEp5aP0tfNkXUE+Uw@mail.gmail.com/

Thanks for your explanation.

Signature · 2 lines
Regards
Yafang
#23 lore raw
In reply to Shakeel Butt
Quoted 63 lines · Shakeel Butt
Quoted 29 lines · Roman Gushchin

Cgroups v2 have been around for a while and many users have fully adopted them, so they never use cgroups v1 features and functionality. Yet they have to "pay" for the cgroup v1 support anyway:
1) the kernel binary contains useless cgroup v1 code,
2) some common structures like task_struct and mem_cgroup have never used cgroup v1-specific members,
3) some code paths have additional checks which are not needed.

Cgroup v1's memory controller has a number of features that are not supported by cgroup v2 and their implementation is pretty much self contained. Most notably, these features are: soft limit reclaim, oom handling in userspace, complicated event notification system, charge migration.

Cgroup v1-specific code in memcontrol.c is close to 4k lines in size and it's intervened with generic and cgroup v2-specific code. It's a burden on developers and maintainers.

This patchset aims to solve these problems by:
1) moving cgroup v1-specific memcg code to the new mm/memcontrol-v1.c file,
2) putting definitions shared by memcontrol.c and memcontrol-v1.c into the mm/internal.h header
3) introducing the CONFIG_MEMCG_V1 config option, turned on by default
4) making memcontrol-v1.c to compile only if CONFIG_MEMCG_V1 is set
5) putting unused struct memory_cgroup and task_struct members under CONFIG_MEMCG_V1 as well.

This is an RFC version, which is not 100% polished yet, so but it would be great to discuss and agree on the overall approach.

Quoted 7 lines

Some open questions, opinions are appreciated:
1) I consider renaming non-static functions in memcontrol-v1.c to have mem_cgroup_v1_ prefix. Is this a good idea?
2) Do we want to extend it beyond the memory controller? Should
3) Is it better to use a new include/linux/memcontrol-v1.h instead of mm/internal.h? Or mm/memcontrol-v1.h.

Hi Roman,

A very timely and important topic and we should definitely talk about it during LSFMM as well. I have been thinking about this problem for quite sometime and I am getting more and more convinced that we should aim to completely deprecate memcg-v1.

More specifically:

1. What are the memcg-v1 features which have no alternative in memcg-v2 and are blocker for memcg-v1 users? (setting aside the cgroup v2 structual restrictions)

2. What are unused memcg-v1 features which we should start deprecating?

IMO we should systematically start deprecating memcg-v1 features and start unblocking the users stuck on memcg-v1.

Now regarding the proposal in this series, I think it can be a first step but should not give an impression that we are done. The only concern I have is the potential of "out of sight, out of mind" situation with this change but if we keep the momentum of deprecation of memcg-v1 it should be fine.

Quoted 6 lines

I have CCed Greg and David from Google to get their opinion on what memcg-v1 features are blocker for their memcg-v2 migration and if they have concern in deprecation of memcg-v1 features.

Anyone else still on memcg-v1, please do provide your input.

Hi,

Sorry for joining the discussion late, but I'd like to add some info
here: We are using the "memsw" feature a lot. It's a very useful knob for container memory overcommitting: It's a great abstraction of the "expected total memory usage" of a container, so containers can't allocate too much memory using SWAP, but still be able to SWAP out.

For a simple example, with memsw.limit == memory.limit, containers can't exceed their original memory limit, even with SWAP enabled, they get OOM killed as how they used to, but the host is now able to offload cold pages.

Similar ability seems absent with V2: With memory.swap.max == 0, the host can't use SWAP to reclaim container memory at all. But with a value larger than that, containers are able to overuse memory, causing delayed OOM kill, thrashing, CPU/Memory usage ratio could be heavily out of balance, especially with compress SWAP backends.

Cgroup accounting of ZSWAP/ZRAM doesn't really help, we want to account for the total raw usage, not the compressed usage. One example is that if a container uses tons of duplicated pages, then it can allocate much more memory than it is limited, that could cause trouble.

I saw Chris also mentioned Google has a workaround internally for it for Cgroup V2. This will be a blocker for us and a similar workaround might be needed. It will be great so see an upstream support for this.

author maintainer
#24 lore raw
Quoted 90 lines · Kairui Song
Quoted 63 lines · Shakeel Butt
Quoted 29 lines · Roman Gushchin

Cgroups v2 have been around for a while and many users have fully adopted them, so they never use cgroups v1 features and functionality. Yet they have to "pay" for the cgroup v1 support anyway:
1) the kernel binary contains useless cgroup v1 code,
2) some common structures like task_struct and mem_cgroup have never used cgroup v1-specific members,
3) some code paths have additional checks which are not needed.

Cgroup v1's memory controller has a number of features that are not supported by cgroup v2 and their implementation is pretty much self contained. Most notably, these features are: soft limit reclaim, oom handling in userspace, complicated event notification system, charge migration.

Cgroup v1-specific code in memcontrol.c is close to 4k lines in size and it's intervened with generic and cgroup v2-specific code. It's a burden on developers and maintainers.

This patchset aims to solve these problems by:
1) moving cgroup v1-specific memcg code to the new mm/memcontrol-v1.c file,
2) putting definitions shared by memcontrol.c and memcontrol-v1.c into the mm/internal.h header
3) introducing the CONFIG_MEMCG_V1 config option, turned on by default
4) making memcontrol-v1.c to compile only if CONFIG_MEMCG_V1 is set
5) putting unused struct memory_cgroup and task_struct members under CONFIG_MEMCG_V1 as well.

This is an RFC version, which is not 100% polished yet, so but it would be great to discuss and agree on the overall approach.

Quoted 7 lines

Some open questions, opinions are appreciated:
1) I consider renaming non-static functions in memcontrol-v1.c to have mem_cgroup_v1_ prefix. Is this a good idea?
2) Do we want to extend it beyond the memory controller? Should
3) Is it better to use a new include/linux/memcontrol-v1.h instead of mm/internal.h? Or mm/memcontrol-v1.h.

Hi Roman,

A very timely and important topic and we should definitely talk about it during LSFMM as well. I have been thinking about this problem for quite sometime and I am getting more and more convinced that we should aim to completely deprecate memcg-v1.

More specifically:

1. What are the memcg-v1 features which have no alternative in memcg-v2 and are blocker for memcg-v1 users? (setting aside the cgroup v2 structual restrictions)

2. What are unused memcg-v1 features which we should start deprecating?

IMO we should systematically start deprecating memcg-v1 features and start unblocking the users stuck on memcg-v1.

Now regarding the proposal in this series, I think it can be a first step but should not give an impression that we are done. The only concern I have is the potential of "out of sight, out of mind" situation with this change but if we keep the momentum of deprecation of memcg-v1 it should be fine.

Quoted 6 lines

I have CCed Greg and David from Google to get their opinion on what memcg-v1 features are blocker for their memcg-v2 migration and if they have concern in deprecation of memcg-v1 features.

Anyone else still on memcg-v1, please do provide your input.

Hi,

Sorry for joining the discussion late, but I'd like to add some info
here: We are using the "memsw" feature a lot. It's a very useful knob for container memory overcommitting: It's a great abstraction of the "expected total memory usage" of a container, so containers can't allocate too much memory using SWAP, but still be able to SWAP out.

For a simple example, with memsw.limit == memory.limit, containers can't exceed their original memory limit, even with SWAP enabled, they get OOM killed as how they used to, but the host is now able to offload cold pages.

Similar ability seems absent with V2: With memory.swap.max == 0, the host can't use SWAP to reclaim container memory at all. But with a value larger than that, containers are able to overuse memory, causing delayed OOM kill, thrashing, CPU/Memory usage ratio could be heavily out of balance, especially with compress SWAP backends.

Quoted 5 lines

Cgroup accounting of ZSWAP/ZRAM doesn't really help, we want to account for the total raw usage, not the compressed usage. One example is that if a container uses tons of duplicated pages, then it can allocate much more memory than it is limited, that could cause trouble.

So you don't need separate swap knobs, only combined, right?

Quoted 3 lines

I saw Chris also mentioned Google has a workaround internally for it for Cgroup V2. This will be a blocker for us and a similar workaround might be needed. It will be great so see an upstream support for this.

I think that _at least_ we should refactor the code so that it would be a minimal patch (e.g. one #define) to switch to the old mode.

I don't think it's reasonable to add a new interface, but having a patch/config option or even a mount option which changes the semantics of memory.swap.max to the v1-like behavior should be ok.

I'll try to do the first part (refactoring this code), and we can have a discussion from there.

Thanks!

#25 lore raw
Quoted 73 lines · Roman Gushchin
Quoted 71 lines · Kairui Song
Quoted 69 lines · Shakeel Butt
Quoted 29 lines · Roman Gushchin

Cgroups v2 have been around for a while and many users have fully adopted them, so they never use cgroups v1 features and functionality. Yet they have to "pay" for the cgroup v1 support anyway:
1) the kernel binary contains useless cgroup v1 code,
2) some common structures like task_struct and mem_cgroup have never used cgroup v1-specific members,
3) some code paths have additional checks which are not needed.

Cgroup v1's memory controller has a number of features that are not supported by cgroup v2 and their implementation is pretty much self contained. Most notably, these features are: soft limit reclaim, oom handling in userspace, complicated event notification system, charge migration.

Cgroup v1-specific code in memcontrol.c is close to 4k lines in size and it's intervened with generic and cgroup v2-specific code. It's a burden on developers and maintainers.

This patchset aims to solve these problems by:
1) moving cgroup v1-specific memcg code to the new mm/memcontrol-v1.c file,
2) putting definitions shared by memcontrol.c and memcontrol-v1.c into the mm/internal.h header
3) introducing the CONFIG_MEMCG_V1 config option, turned on by default
4) making memcontrol-v1.c to compile only if CONFIG_MEMCG_V1 is set
5) putting unused struct memory_cgroup and task_struct members under CONFIG_MEMCG_V1 as well.

This is an RFC version, which is not 100% polished yet, so but it would be great to discuss and agree on the overall approach.

Quoted 7 lines

Some open questions, opinions are appreciated:
1) I consider renaming non-static functions in memcontrol-v1.c to have mem_cgroup_v1_ prefix. Is this a good idea?
2) Do we want to extend it beyond the memory controller? Should
3) Is it better to use a new include/linux/memcontrol-v1.h instead of mm/internal.h? Or mm/memcontrol-v1.h.

Hi Roman,

A very timely and important topic and we should definitely talk about it during LSFMM as well. I have been thinking about this problem for quite sometime and I am getting more and more convinced that we should aim to completely deprecate memcg-v1.

More specifically:

1. What are the memcg-v1 features which have no alternative in memcg-v2 and are blocker for memcg-v1 users? (setting aside the cgroup v2 structual restrictions)

2. What are unused memcg-v1 features which we should start deprecating?

IMO we should systematically start deprecating memcg-v1 features and start unblocking the users stuck on memcg-v1.

Now regarding the proposal in this series, I think it can be a first step but should not give an impression that we are done. The only concern I have is the potential of "out of sight, out of mind" situation with this change but if we keep the momentum of deprecation of memcg-v1 it should be fine.

I have CCed Greg and David from Google to get their opinion on what memcg-v1 features are blocker for their memcg-v2 migration and if they have concern in deprecation of memcg-v1 features.

Anyone else still on memcg-v1, please do provide your input.

Quoted 7 lines
Quoted 7 lines

Hi,

Sorry for joining the discussion late, but I'd like to add some info
here: We are using the "memsw" feature a lot. It's a very useful knob for container memory overcommitting: It's a great abstraction of the "expected total memory usage" of a container, so containers can't allocate too much memory using SWAP, but still be able to SWAP out.

+Michal,

Just FYI, we do have companies like Tensent using the V1 combine memsw limitation as well. Google is not the only company using this API.

Quoted 18 lines
Quoted 12 lines

For a simple example, with memsw.limit == memory.limit, containers can't exceed their original memory limit, even with SWAP enabled, they get OOM killed as how they used to, but the host is now able to offload cold pages.

Similar ability seems absent with V2: With memory.swap.max == 0, the host can't use SWAP to reclaim container memory at all. But with a value larger than that, containers are able to overuse memory, causing delayed OOM kill, thrashing, CPU/Memory usage ratio could be heavily out of balance, especially with compress SWAP backends.

Quoted 5 lines

Cgroup accounting of ZSWAP/ZRAM doesn't really help, we want to account for the total raw usage, not the compressed usage. One example is that if a container uses tons of duplicated pages, then it can allocate much more memory than it is limited, that could cause trouble.

Quoted 8 lines

So you don't need separate swap knobs, only combined, right?

Quoted 3 lines

I saw Chris also mentioned Google has a workaround internally for it for Cgroup V2. This will be a blocker for us and a similar workaround might be needed. It will be great so see an upstream support for this.

I think that _at least_ we should refactor the code so that it would be a minimal patch (e.g. one #define) to switch to the old mode.

That would be great to have a path forward to allow cgroup V2 to provide the combined memsw limitations.

Quoted 4 lines

I don't think it's reasonable to add a new interface, but having a patch/config option or even a mount option which changes the semantics of memory.swap.max to the v1-like behavior should be ok.

Using sysctl or a slightly different cgroup API is fine. The feature needs to be there.

Quoted 2 lines

I'll try to do the first part (refactoring this code), and we can have a discussion from there.

Looking forward to it.

Thanks

Chris

#26 lore raw
In reply to Roman Gushchin
Quoted 92 lines · Roman Gushchin
Quoted 90 lines · Kairui Song
Quoted 63 lines · Shakeel Butt
Quoted 29 lines · Roman Gushchin

Cgroups v2 have been around for a while and many users have fully adopted them, so they never use cgroups v1 features and functionality. Yet they have to "pay" for the cgroup v1 support anyway:
1) the kernel binary contains useless cgroup v1 code,
2) some common structures like task_struct and mem_cgroup have never used cgroup v1-specific members,
3) some code paths have additional checks which are not needed.

Cgroup v1's memory controller has a number of features that are not supported by cgroup v2 and their implementation is pretty much self contained. Most notably, these features are: soft limit reclaim, oom handling in userspace, complicated event notification system, charge migration.

Cgroup v1-specific code in memcontrol.c is close to 4k lines in size and it's intervened with generic and cgroup v2-specific code. It's a burden on developers and maintainers.

This patchset aims to solve these problems by:
1) moving cgroup v1-specific memcg code to the new mm/memcontrol-v1.c file,
2) putting definitions shared by memcontrol.c and memcontrol-v1.c into the mm/internal.h header
3) introducing the CONFIG_MEMCG_V1 config option, turned on by default
4) making memcontrol-v1.c to compile only if CONFIG_MEMCG_V1 is set
5) putting unused struct memory_cgroup and task_struct members under CONFIG_MEMCG_V1 as well.

This is an RFC version, which is not 100% polished yet, so but it would be great to discuss and agree on the overall approach.

Quoted 7 lines

Some open questions, opinions are appreciated:
1) I consider renaming non-static functions in memcontrol-v1.c to have mem_cgroup_v1_ prefix. Is this a good idea?
2) Do we want to extend it beyond the memory controller? Should
3) Is it better to use a new include/linux/memcontrol-v1.h instead of mm/internal.h? Or mm/memcontrol-v1.h.

Hi Roman,

A very timely and important topic and we should definitely talk about it during LSFMM as well. I have been thinking about this problem for quite sometime and I am getting more and more convinced that we should aim to completely deprecate memcg-v1.

More specifically:

1. What are the memcg-v1 features which have no alternative in memcg-v2 and are blocker for memcg-v1 users? (setting aside the cgroup v2 structual restrictions)

2. What are unused memcg-v1 features which we should start deprecating?

IMO we should systematically start deprecating memcg-v1 features and start unblocking the users stuck on memcg-v1.

Now regarding the proposal in this series, I think it can be a first step but should not give an impression that we are done. The only concern I have is the potential of "out of sight, out of mind" situation with this change but if we keep the momentum of deprecation of memcg-v1 it should be fine.

Quoted 6 lines

I have CCed Greg and David from Google to get their opinion on what memcg-v1 features are blocker for their memcg-v2 migration and if they have concern in deprecation of memcg-v1 features.

Anyone else still on memcg-v1, please do provide your input.

Hi,

Sorry for joining the discussion late, but I'd like to add some info
here: We are using the "memsw" feature a lot. It's a very useful knob for container memory overcommitting: It's a great abstraction of the "expected total memory usage" of a container, so containers can't allocate too much memory using SWAP, but still be able to SWAP out.

For a simple example, with memsw.limit == memory.limit, containers can't exceed their original memory limit, even with SWAP enabled, they get OOM killed as how they used to, but the host is now able to offload cold pages.

Similar ability seems absent with V2: With memory.swap.max == 0, the host can't use SWAP to reclaim container memory at all. But with a value larger than that, containers are able to overuse memory, causing delayed OOM kill, thrashing, CPU/Memory usage ratio could be heavily out of balance, especially with compress SWAP backends.

Quoted 7 lines
Quoted 5 lines

Cgroup accounting of ZSWAP/ZRAM doesn't really help, we want to account for the total raw usage, not the compressed usage. One example is that if a container uses tons of duplicated pages, then it can allocate much more memory than it is limited, that could cause trouble.

So you don't need separate swap knobs, only combined, right?

Yes, currently we use either combined or separate knobs.

Quoted 7 lines
Quoted 3 lines

I saw Chris also mentioned Google has a workaround internally for it for Cgroup V2. This will be a blocker for us and a similar workaround might be needed. It will be great so see an upstream support for this.

I think that _at least_ we should refactor the code so that it would be a minimal patch (e.g. one #define) to switch to the old mode.

Quoted 6 lines

I don't think it's reasonable to add a new interface, but having a patch/config option or even a mount option which changes the semantics of memory.swap.max to the v1-like behavior should be ok.

I'll try to do the first part (refactoring this code), and we can have a discussion from there.

Thanks, that sounds like a good start.

maintainer
#27 lore raw
In reply to Roman Gushchin
Quoted 8 lines · Roman Gushchin

Cgroups v2 have been around for a while and many users have fully adopted them, so they never use cgroups v1 features and functionality. Yet they have to "pay" for the cgroup v1 support anyway:
1) the kernel binary contains useless cgroup v1 code,
2) some common structures like task_struct and mem_cgroup have never used cgroup v1-specific members,
3) some code paths have additional checks which are not needed.

Quoted 8 lines

Cgroup v1's memory controller has a number of features that are not supported by cgroup v2 and their implementation is pretty much self contained. Most notably, these features are: soft limit reclaim, oom handling in userspace, complicated event notification system, charge migration.

Cgroup v1-specific code in memcontrol.c is close to 4k lines in size and it's intervened with generic and cgroup v2-specific code. It's a burden on developers and maintainers.

Great patchset. The moves look clean and straight-forward to me on first glance.

Quoted 2 lines

This patchset aims to solve these problems by:
1) moving cgroup v1-specific memcg code to the new mm/memcontrol-v1.c file,

+1

Quoted 2 lines

2) putting definitions shared by memcontrol.c and memcontrol-v1.c into the mm/internal.h header

You proposed mm/memcontrol-v1.h below, IMO that's the best option.

Quoted 1 line

3) introducing the CONFIG_MEMCG_V1 config option, turned on by default

+1

CONFIG_MEMCG1 should also work.

Quoted 1 line

4) making memcontrol-v1.c to compile only if CONFIG_MEMCG_V1 is set

+1

Quoted 2 lines

5) putting unused struct memory_cgroup and task_struct members under CONFIG_MEMCG_V1 as well.

+1

Quoted 7 lines

This is an RFC version, which is not 100% polished yet, so but it would be great to discuss and agree on the overall approach.

Some open questions, opinions are appreciated:
1) I consider renaming non-static functions in memcontrol-v1.c to have mem_cgroup_v1_ prefix. Is this a good idea?

I think this would be great, to make it more obvious in memcontrol.c.

For core cgroup code, we used cgroup1_foo(). Maybe name them all things like memcg1_update_tree() etc.? That's short and sweet while sticking out visually pretty well.

Quoted 1 line

2) Do we want to extend it beyond the memory controller? Should

Could you please elaborate? ^_^

Quoted 2 lines

3) Is it better to use a new include/linux/memcontrol-v1.h instead of mm/internal.h? Or mm/memcontrol-v1.h.

mm/memcontrol-v1.h sounds good to me.

Quoted 1 line
 mm/memcontrol.c            | 4121 ++++++++++++++++++++++---------------------------------------------------------------------------------------------------------------------------------

Lol, awesome.

maintainer
#28 lore raw
Quoted 5 lines · Johannes Weiner
Quoted 2 lines · Roman Gushchin

3) Is it better to use a new include/linux/memcontrol-v1.h instead of mm/internal.h? Or mm/memcontrol-v1.h.

mm/memcontrol-v1.h sounds good to me.

Argh, there is a folio_memcg_lock() callsite in fs/buffer.c. I suppose include/linux/memcontrol-v1.h makes the most sense then.

author maintainer
#29 lore raw
Quoted 9 lines · Johannes Weiner
Quoted 5 lines · Johannes Weiner
Quoted 2 lines · Roman Gushchin

3) Is it better to use a new include/linux/memcontrol-v1.h instead of mm/internal.h? Or mm/memcontrol-v1.h.

mm/memcontrol-v1.h sounds good to me.

Argh, there is a folio_memcg_lock() callsite in fs/buffer.c. I suppose include/linux/memcontrol-v1.h makes the most sense then.

You mean put everything into include/linux/memcontrol-v1.h? And functions from memcontrol.c by memcontrol-v1.c into include/linux/memcontrol.h?

It's an option I considered it but the downside is that we're "leaking" a lot of internal definitions into the outside world, because memcontrol.h is included everywhere.

So maybe mm/memcontrol-v1.h for definitions shared between v1 and v2 and keep exported functions in include/linux/memcontrol.h? There are only few of them.

Thanks!

maintainer
#30 lore raw
Quoted 3 lines · Roman Gushchin

So maybe mm/memcontrol-v1.h for definitions shared between v1 and v2 and keep exported functions in include/linux/memcontrol.h? There are only few of them.

That sounds best to me.

reviewer
#31 lore raw
In reply to Roman Gushchin
Quoted 17 lines · Roman Gushchin

Cgroups v2 have been around for a while and many users have fully adopted them, so they never use cgroups v1 features and functionality. Yet they have to "pay" for the cgroup v1 support anyway:
1) the kernel binary contains useless cgroup v1 code,
2) some common structures like task_struct and mem_cgroup have never used cgroup v1-specific members,
3) some code paths have additional checks which are not needed.

Cgroup v1's memory controller has a number of features that are not supported by cgroup v2 and their implementation is pretty much self contained. Most notably, these features are: soft limit reclaim, oom handling in userspace, complicated event notification system, charge migration.

Cgroup v1-specific code in memcontrol.c is close to 4k lines in size and it's intervened with generic and cgroup v2-specific code. It's a burden on developers and maintainers.

Quoted 8 lines

This patchset aims to solve these problems by:
1) moving cgroup v1-specific memcg code to the new mm/memcontrol-v1.c file,
2) putting definitions shared by memcontrol.c and memcontrol-v1.c into the mm/internal.h header
3) introducing the CONFIG_MEMCG_V1 config option, turned on by default
4) making memcontrol-v1.c to compile only if CONFIG_MEMCG_V1 is set
5) putting unused struct memory_cgroup and task_struct members under CONFIG_MEMCG_V1 as well.

This makes sense and I have to admit I didn't think this was so much code to move. It will make the code base much esier to follow. I do not think we can drop that code anytime soon as there is still quite a lot of use of v1 out there. From my experience there is no good reason for many other than inertia and those are just waiting for somebody to move them to v2. There are some workloads which depend on v1 only features and we should discuss what to do about those.

Signature · 2 lines
Michal Hocko
SUSE Labs