Manage the top tier memory in a tiered memory

RFC PATCH In review Tim Chen 6 participants 34 messages Apr 5, 2021 – Apr 16, 2021

Subsystems (13)

Memory management Documentation MM core Driver core +9 more subsystems

lore

Share & tools

lore

Tim Chen

author

2021-04-05 17:08 UTC #0 lore raw

cover Manage the top tier memory in a tiered memory

Traditionally, all memory is DRAM. Some DRAM might be closer/faster than others NUMA wise, but a byte of media has about the same cost whether it is close or far. But, with new memory tiers such as Persistent Memory (PMEM). there is a choice between fast/expensive DRAM and slow/cheap PMEM.

The fast/expensive memory lives in the top tier of the memory hierachy.

Previously, the patchset
[PATCH 00/10] [v7] Migrate Pages in lieu of discard
https://lore.kernel.org/linux-mm/[email protected]/
provides a mechanism to demote cold pages from DRAM node into PMEM.

And the patchset
[PATCH 0/6] [RFC v6] NUMA balancing: optimize memory placement for memory tiering system
https://lore.kernel.org/linux-mm/[email protected]/
provides a mechanism to promote hot pages in PMEM to the DRAM node leveraging autonuma.

The two patchsets together keep the hot pages in DRAM and colder pages in PMEM.

To make fine grain cgroup based management of the precious top tier DRAM memory possible, this patchset adds a few new features:
1. Provides memory monitors on the amount of top tier memory used per cgroup and by the system as a whole.
2. Applies soft limits on the top tier memory each cgroup uses
3. Enables kswapd to demote top tier pages from cgroup with excess top tier memory usages.

This allows us to provision different amount of top tier memory to each cgroup according to the cgroup's latency need.

The patchset is based on cgroup v1 interface. One shortcoming of the v1 interface is the limit on the cgroup is a soft limit, so a cgroup can exceed the limit quite a bit before reclaim before page demotion reins it in.

We are also working on a cgroup v2 interface control interface that will will have a max limit on the top tier memory per cgroup but requires much additional logic to fall back and allocate from non top tier memory when a cgroup reaches the maximum limit. This simpler cgroup v1 implementation with all its warts is used to illustrate the concept of cgroup based top tier memory management and serves as a starting point of discussions.

The soft limit and soft reclaim logic in this patchset will be similar for what we would do for a cgroup v2 interface when we reach the high watermark for top tier usage in a cgroup v2 interface.

This patchset is applied on top of
[PATCH 00/10] [v7] Migrate Pages in lieu of discard
and
[PATCH 0/6] [RFC v6] NUMA balancing: optimize memory placement for memory tiering system

It is part of a larger patchset. You can play with the complete set of patches using the tree:
https://git.kernel.org/pub/scm/linux/kernel/git/vishal/tiering.git/log/?h=tiering-0.71

Tim Chen (11):
mm: Define top tier memory node mask
mm: Add soft memory limit for mem cgroup
mm: Account the top tier memory usage per cgroup
mm: Report top tier memory usage in sysfs
mm: Add soft_limit_top_tier tree for mem cgroup
mm: Handle top tier memory in cgroup soft limit memory tree utilities
mm: Account the total top tier memory in use
mm: Add toptier option for mem_cgroup_soft_limit_reclaim()
mm: Use kswapd to demote pages when toptier memory is tight
mm: Set toptier_scale_factor via sysctl
mm: Wakeup kswapd if toptier memory need soft reclaim

 Documentation/admin-guide/sysctl/vm.rst |  12 +
 drivers/base/node.c                     |   2 +
 include/linux/memcontrol.h              |  20 +-
 include/linux/mm.h                      |   4 +
 include/linux/mmzone.h                  |   7 +
 include/linux/nodemask.h                |   1 +
 include/linux/vmstat.h                  |  18 ++
 kernel/sysctl.c                         |  10 +
 mm/memcontrol.c                         | 303 +++++++++++++++++++-----
 mm/memory_hotplug.c                     |   3 +
 mm/migrate.c                            |   1 +
 mm/page_alloc.c                         |  36 ++-
 mm/vmscan.c                             |  73 +++++-
 mm/vmstat.c                             |  22 +-
 14 files changed, 444 insertions(+), 68 deletions(-)

Signature · 1 line

2.20.1

Tim Chen

author

2021-04-05 17:08 UTC #1 lore raw

RFC PATCH 1/11 mm: Define top tier memory node mask

Traditionally, all RAM is DRAM. Some DRAM might be closer/faster than others, but a byte of media has about the same cost whether it is close or far. But, with new memory tiers such as High-Bandwidth Memory or Persistent Memory, there is a choice between fast/expensive and slow/cheap.

The fast/expensive memory lives in the top tier of the memory hierachy and it is a precious resource that needs to be accounted and managed on a memory cgroup basis.

Define the top tier memory as the memory nodes that don't have demotion paths into it from higher tier memory.

Signed-off-by Tim Chen <tim.c.chen-VuQAYsv1563Yd54FQh9/[email protected]>

---
 drivers/base/node.c      | 2 ++
 include/linux/nodemask.h | 1 +
 mm/memory_hotplug.c      | 3 +++
 mm/migrate.c             | 1 +
 mm/page_alloc.c          | 5 ++++-
 5 files changed, 11 insertions(+), 1 deletion(-)

drivers/base/node.c +2 -0

index 04f71c7bc3f8..9eb214ac331f 100644
@@ -1016,6 +1016,7 @@ static struct node_attr node_state_attr[] = {
 #endif
 	[N_MEMORY] = _NODE_ATTR(has_memory, N_MEMORY),
 	[N_CPU] = _NODE_ATTR(has_cpu, N_CPU),
+	[N_TOPTIER] = _NODE_ATTR(is_toptier, N_TOPTIER),
 	[N_GENERIC_INITIATOR] = _NODE_ATTR(has_generic_initiator,
 					   N_GENERIC_INITIATOR),
 };

@@ -1029,6 +1030,7 @@ static struct attribute *node_state_attrs[] = {
 #endif
 	&node_state_attr[N_MEMORY].attr.attr,
 	&node_state_attr[N_CPU].attr.attr,
+	&node_state_attr[N_TOPTIER].attr.attr,
 	&node_state_attr[N_GENERIC_INITIATOR].attr.attr,
 	NULL
 };

include/linux/nodemask.h +1 -0

index ac398e143c9a..3003401ed7f0 100644
@@ -399,6 +399,7 @@ enum node_states {
 #endif
 	N_MEMORY,		/* The node has memory(regular, high, movable) */
 	N_CPU,		/* The node has one or more cpus */
+	N_TOPTIER,		/* Top tier node, no demotion path into node */
 	N_GENERIC_INITIATOR,	/* The node has one or more Generic Initiators */
 	NR_NODE_STATES
 };

mm/memory_hotplug.c +3 -0

index 7550b88e2432..7b21560d4c4d 100644
@@ -36,6 +36,7 @@
 #include <linux/memblock.h>
 #include <linux/compaction.h>
 #include <linux/rmap.h>
+#include <linux/node.h>
 
 #include <asm/tlbflush.h>

@@ -654,6 +655,8 @@ static void node_states_set_node(int node, struct memory_notify *arg)
 
 	if (arg->status_change_nid >= 0)
 		node_set_state(node, N_MEMORY);
+
+	node_set_state(node, N_TOPTIER);
 }
 
 static void __meminit resize_zone_range(struct zone *zone, unsigned long start_pfn,

mm/migrate.c +1 -0

index 72223fd7e623..e84aedf611da 100644
@@ -3439,6 +3439,7 @@ static int establish_migrate_target(int node, nodemask_t *used)
 		return NUMA_NO_NODE;
 
 	node_demotion[node] = migration_target;
+	node_clear_state(migration_target, N_TOPTIER);
 
 	return migration_target;
 }

mm/page_alloc.c +4 -1

index ff058941ccfa..471a2c342c4f 100644
@@ -157,6 +157,7 @@ nodemask_t node_states[NR_NODE_STATES] __read_mostly = {
 	[N_MEMORY] = { { [0] = 1UL } },
 	[N_CPU] = { { [0] = 1UL } },
 #endif	/* NUMA */
+	[N_TOPTIER] = NODE_MASK_ALL,
 };
 EXPORT_SYMBOL(node_states);

@@ -7590,8 +7591,10 @@ void __init free_area_init(unsigned long *max_zone_pfn)
 		free_area_init_node(nid);
 
 		/* Any memory on that node */
-		if (pgdat->node_present_pages)
+		if (pgdat->node_present_pages) {
 			node_set_state(nid, N_MEMORY);
+			node_set_state(nid, N_TOPTIER);
+		}
 		check_for_memory(pgdat, nid);
 	}
 }

Signature · 1 line

2.20.1

Tim Chen

author

2021-04-05 17:08 UTC #2 lore raw

RFC PATCH 2/11 mm: Add soft memory limit for mem cgroup

In reply to Tim Chen

For each memory cgroup, define a soft memory limit on its top tier memory consumption. Memory cgroups exceeding their top tier limit will be selected for demotion of their top tier memory to lower tier under memory pressure.

Signed-off-by Tim Chen <tim.c.chen-VuQAYsv1563Yd54FQh9/[email protected]>

---
 include/linux/memcontrol.h |  1 +
 mm/memcontrol.c            | 18 ++++++++++++++++++
 2 files changed, 19 insertions(+)

include/linux/memcontrol.h +1 -0

index eeb0b52203e9..25d8b9acec7c 100644
@@ -230,6 +230,7 @@ struct mem_cgroup {
 	struct work_struct high_work;
 
 	unsigned long soft_limit;
+	unsigned long toptier_soft_limit;
 
 	/* vmpressure notifications */
 	struct vmpressure vmpressure;

mm/memcontrol.c +18 -0

index 41a3f22b6639..9a9d677a6654 100644
@@ -3603,6 +3603,7 @@ enum {
 	RES_MAX_USAGE,
 	RES_FAILCNT,
 	RES_SOFT_LIMIT,
+	RES_TOPTIER_SOFT_LIMIT,
 };
 
 static u64 mem_cgroup_read_u64(struct cgroup_subsys_state *css,

@@ -3643,6 +3644,8 @@ static u64 mem_cgroup_read_u64(struct cgroup_subsys_state *css,
 		return counter->failcnt;
 	case RES_SOFT_LIMIT:
 		return (u64)memcg->soft_limit * PAGE_SIZE;
+	case RES_TOPTIER_SOFT_LIMIT:
+		return (u64)memcg->toptier_soft_limit * PAGE_SIZE;
 	default:
 		BUG();
 	}

@@ -3881,6 +3884,14 @@ static ssize_t mem_cgroup_write(struct kernfs_open_file *of,
 		memcg->soft_limit = nr_pages;
 		ret = 0;
 		break;
+	case RES_TOPTIER_SOFT_LIMIT:
+		if (mem_cgroup_is_root(memcg)) { /* Can't set limit on root */
+			ret = -EINVAL;
+			break;
+		}
+		memcg->toptier_soft_limit = nr_pages;
+		ret = 0;
+		break;
 	}
 	return ret ?: nbytes;
 }

@@ -5029,6 +5040,12 @@ static struct cftype mem_cgroup_legacy_files[] = {
 		.write = mem_cgroup_write,
 		.read_u64 = mem_cgroup_read_u64,
 	},
+	{
+		.name = "toptier_soft_limit_in_bytes",
+		.private = MEMFILE_PRIVATE(_MEM, RES_TOPTIER_SOFT_LIMIT),
+		.write = mem_cgroup_write,
+		.read_u64 = mem_cgroup_read_u64,
+	},
 	{
 		.name = "failcnt",
 		.private = MEMFILE_PRIVATE(_MEM, RES_FAILCNT),

@@ -5365,6 +5382,7 @@ mem_cgroup_css_alloc(struct cgroup_subsys_state *parent_css)
 	page_counter_set_high(&memcg->memory, PAGE_COUNTER_MAX);
 	memcg->soft_limit = PAGE_COUNTER_MAX;
 	page_counter_set_high(&memcg->swap, PAGE_COUNTER_MAX);
+	memcg->toptier_soft_limit = PAGE_COUNTER_MAX;
 	if (parent) {
 		memcg->swappiness = mem_cgroup_swappiness(parent);
 		memcg->oom_kill_disable = parent->oom_kill_disable;

Signature · 1 line

2.20.1

Tim Chen

author

2021-04-05 17:08 UTC #3 lore raw

RFC PATCH 3/11 mm: Account the top tier memory usage per cgroup

In reply to Tim Chen

For each memory cgroup, account its usage of the top tier memory at the time a top tier page is assigned and uncharged from the cgroup.

Signed-off-by Tim Chen <[email protected]>

---
 include/linux/memcontrol.h |  1 +
 mm/memcontrol.c            | 39 +++++++++++++++++++++++++++++++++++++-
 2 files changed, 39 insertions(+), 1 deletion(-)

include/linux/memcontrol.h +1 -0

index 25d8b9acec7c..609d8590950c 100644
@@ -225,6 +225,7 @@ struct mem_cgroup {
 	/* Legacy consumer-oriented counters */
 	struct page_counter kmem;		/* v1 only */
 	struct page_counter tcpmem;		/* v1 only */
+	struct page_counter toptier;
 
 	/* Range enforcement for interrupt charges */
 	struct work_struct high_work;

mm/memcontrol.c +38 -1

index 9a9d677a6654..fe7bb8613f5a 100644
@@ -253,6 +253,13 @@ struct cgroup_subsys_state *vmpressure_to_css(struct vmpressure *vmpr)
 	return &container_of(vmpr, struct mem_cgroup, vmpressure)->css;
 }
 
+static inline bool top_tier(struct page *page)
+{
+	int nid = page_to_nid(page);
+
+	return node_state(nid, N_TOPTIER);
+}
+
 #ifdef CONFIG_MEMCG_KMEM
 extern spinlock_t css_set_lock;

@@ -951,6 +958,23 @@ static void mem_cgroup_charge_statistics(struct mem_cgroup *memcg,
 	__this_cpu_add(memcg->vmstats_percpu->nr_page_events, nr_pages);
 }
 
+static inline void mem_cgroup_charge_toptier(struct mem_cgroup *memcg,
+					 struct page *page,
+					 int nr_pages)
+{
+	if (!top_tier(page))
+		return;
+
+	if (nr_pages >= 0)
+		page_counter_charge(&memcg->toptier,
+				   (unsigned long) nr_pages);
+	else {
+		nr_pages = -nr_pages;
+		page_counter_uncharge(&memcg->toptier,
+				   (unsigned long) nr_pages);
+	}
+}
+
 static bool mem_cgroup_event_ratelimit(struct mem_cgroup *memcg,
 				       enum mem_cgroup_events_target target)
 {

@@ -2932,6 +2956,7 @@ static void commit_charge(struct page *page, struct mem_cgroup *memcg)
 	 * - exclusive reference
 	 */
 	page->memcg_data = (unsigned long)memcg;
+	mem_cgroup_charge_toptier(memcg, page, thp_nr_pages(page));
 }
 
 #ifdef CONFIG_MEMCG_KMEM

@@ -3138,6 +3163,7 @@ int __memcg_kmem_charge_page(struct page *page, gfp_t gfp, int order)
 		if (!ret) {
 			page->memcg_data = (unsigned long)memcg |
 				MEMCG_DATA_KMEM;
+			mem_cgroup_charge_toptier(memcg, page, 1 << order);
 			return 0;
 		}
 		css_put(&memcg->css);

@@ -3161,6 +3187,7 @@ void __memcg_kmem_uncharge_page(struct page *page, int order)
 	VM_BUG_ON_PAGE(mem_cgroup_is_root(memcg), page);
 	__memcg_kmem_uncharge(memcg, nr_pages);
 	page->memcg_data = 0;
+	mem_cgroup_charge_toptier(memcg, page, -nr_pages);
 	css_put(&memcg->css);
 }

@@ -5389,11 +5416,13 @@ mem_cgroup_css_alloc(struct cgroup_subsys_state *parent_css)
 
 		page_counter_init(&memcg->memory, &parent->memory);
 		page_counter_init(&memcg->swap, &parent->swap);
+		page_counter_init(&memcg->toptier, &parent->toptier);
 		page_counter_init(&memcg->kmem, &parent->kmem);
 		page_counter_init(&memcg->tcpmem, &parent->tcpmem);
 	} else {
 		page_counter_init(&memcg->memory, NULL);
 		page_counter_init(&memcg->swap, NULL);
+		page_counter_init(&memcg->toptier, NULL);
 		page_counter_init(&memcg->kmem, NULL);
 		page_counter_init(&memcg->tcpmem, NULL);

@@ -5745,6 +5774,8 @@ static int mem_cgroup_move_account(struct page *page,
 	css_put(&from->css);
 
 	page->memcg_data = (unsigned long)to;
+	mem_cgroup_charge_toptier(to, page, nr_pages);
+	mem_cgroup_charge_toptier(from, page, -nr_pages);
 
 	__unlock_page_memcg(from);

@@ -6832,6 +6863,7 @@ struct uncharge_gather {
 	unsigned long nr_pages;
 	unsigned long pgpgout;
 	unsigned long nr_kmem;
+	unsigned long nr_toptier;
 	struct page *dummy_page;
 };

@@ -6846,6 +6878,7 @@ static void uncharge_batch(const struct uncharge_gather *ug)
 
 	if (!mem_cgroup_is_root(ug->memcg)) {
 		page_counter_uncharge(&ug->memcg->memory, ug->nr_pages);
+		page_counter_uncharge(&ug->memcg->toptier, ug->nr_toptier);
 		if (do_memsw_account())
 			page_counter_uncharge(&ug->memcg->memsw, ug->nr_pages);
 		if (!cgroup_subsys_on_dfl(memory_cgrp_subsys) && ug->nr_kmem)

@@ -6891,6 +6924,8 @@ static void uncharge_page(struct page *page, struct uncharge_gather *ug)
 
 	nr_pages = compound_nr(page);
 	ug->nr_pages += nr_pages;
+	if (top_tier(page))
+		ug->nr_toptier += nr_pages;
 
 	if (PageMemcgKmem(page))
 		ug->nr_kmem += nr_pages;

@@ -7216,8 +7251,10 @@ void mem_cgroup_swapout(struct page *page, swp_entry_t entry)
 
 	page->memcg_data = 0;
 
-	if (!mem_cgroup_is_root(memcg))
+	if (!mem_cgroup_is_root(memcg)) {
 		page_counter_uncharge(&memcg->memory, nr_entries);
+		mem_cgroup_charge_toptier(memcg, page, -nr_entries);
+	}
 
 	if (!cgroup_memory_noswap && memcg != swap_memcg) {
 		if (!mem_cgroup_is_root(swap_memcg))

Signature · 1 line

2.20.1

Tim Chen

author

2021-04-05 17:08 UTC #4 lore raw

RFC PATCH 4/11 mm: Report top tier memory usage in sysfs

In reply to Tim Chen

In memory cgroup's sysfs, report the memory cgroup's usage of top tier memory in a new field: "toptier_usage_in_bytes".

Signed-off-by Tim Chen <tim.c.chen-VuQAYsv1563Yd54FQh9/[email protected]>

---
 mm/memcontrol.c | 8 ++++++++
 1 file changed, 8 insertions(+)

mm/memcontrol.c +8 -0

index fe7bb8613f5a..68590f46fa76 100644
@@ -3631,6 +3631,7 @@ enum {
 	RES_FAILCNT,
 	RES_SOFT_LIMIT,
 	RES_TOPTIER_SOFT_LIMIT,
+	RES_TOPTIER_USAGE,
 };
 
 static u64 mem_cgroup_read_u64(struct cgroup_subsys_state *css,

@@ -3673,6 +3674,8 @@ static u64 mem_cgroup_read_u64(struct cgroup_subsys_state *css,
 		return (u64)memcg->soft_limit * PAGE_SIZE;
 	case RES_TOPTIER_SOFT_LIMIT:
 		return (u64)memcg->toptier_soft_limit * PAGE_SIZE;
+	case RES_TOPTIER_USAGE:
+		return (u64)page_counter_read(&memcg->toptier) * PAGE_SIZE;
 	default:
 		BUG();
 	}

@@ -5073,6 +5076,11 @@ static struct cftype mem_cgroup_legacy_files[] = {
 		.write = mem_cgroup_write,
 		.read_u64 = mem_cgroup_read_u64,
 	},
+	{
+		.name = "toptier_usage_in_bytes",
+		.private = MEMFILE_PRIVATE(_MEM, RES_TOPTIER_USAGE),
+		.read_u64 = mem_cgroup_read_u64,
+	},
 	{
 		.name = "failcnt",
 		.private = MEMFILE_PRIVATE(_MEM, RES_FAILCNT),

Signature · 1 line

2.20.1

Tim Chen

author

2021-04-05 17:08 UTC #5 lore raw

RFC PATCH 5/11 mm: Add soft_limit_top_tier tree for mem cgroup

In reply to Tim Chen

Define a per node soft_limit_top_tier red black tree that sort and track the cgroups by each group's excess over its toptier soft limit. A cgroup is added to the tree if it has exceeded its top tier soft limit and it has used pages on the node.

Signed-off-by Tim Chen <tim.c.chen-VuQAYsv1563Yd54FQh9/[email protected]>

---
 mm/memcontrol.c | 68 +++++++++++++++++++++++++++++++++++++------------
 1 file changed, 52 insertions(+), 16 deletions(-)

mm/memcontrol.c +52 ~~-16~~

index 68590f46fa76..90a78ff3fca8 100644
@@ -122,6 +122,7 @@ struct mem_cgroup_tree {
 };
 
 static struct mem_cgroup_tree soft_limit_tree __read_mostly;
+static struct mem_cgroup_tree soft_limit_toptier_tree __read_mostly;
 
 /* for OOM */
 struct mem_cgroup_eventfd_list {

@@ -590,17 +591,27 @@ mem_cgroup_page_nodeinfo(struct mem_cgroup *memcg, struct page *page)
 }
 
 static struct mem_cgroup_tree_per_node *
-soft_limit_tree_node(int nid)
-{
-	return soft_limit_tree.rb_tree_per_node[nid];
+soft_limit_tree_node(int nid, enum node_states type)
+{
+	switch (type) {
+	case N_MEMORY:
+		return soft_limit_tree.rb_tree_per_node[nid];
+	case N_TOPTIER:
+		if (node_state(nid, N_TOPTIER))
+			return soft_limit_toptier_tree.rb_tree_per_node[nid];
+		else
+			return NULL;
+	default:
+		return NULL;
+	}
 }
 
 static struct mem_cgroup_tree_per_node *
-soft_limit_tree_from_page(struct page *page)
+soft_limit_tree_from_page(struct page *page, enum node_states type)
 {
 	int nid = page_to_nid(page);
 
-	return soft_limit_tree.rb_tree_per_node[nid];
+	return soft_limit_tree_node(nid, type);
 }
 
 static void __mem_cgroup_insert_exceeded(struct mem_cgroup_per_node *mz,

@@ -661,12 +672,24 @@ static void mem_cgroup_remove_exceeded(struct mem_cgroup_per_node *mz,
 	spin_unlock_irqrestore(&mctz->lock, flags);
 }
 
-static unsigned long soft_limit_excess(struct mem_cgroup *memcg)
+static unsigned long soft_limit_excess(struct mem_cgroup *memcg, enum node_states type)
 {
-	unsigned long nr_pages = page_counter_read(&memcg->memory);
-	unsigned long soft_limit = READ_ONCE(memcg->soft_limit);
+	unsigned long nr_pages;
+	unsigned long soft_limit;
 	unsigned long excess = 0;
 
+	switch (type) {
+	case N_MEMORY:
+		nr_pages = page_counter_read(&memcg->memory);
+		soft_limit = READ_ONCE(memcg->soft_limit);
+		break;
+	case N_TOPTIER:
+		nr_pages = page_counter_read(&memcg->toptier);
+		soft_limit = READ_ONCE(memcg->toptier_soft_limit);
+		break;
+	default:
+		return 0;
+	}
 	if (nr_pages > soft_limit)
 		excess = nr_pages - soft_limit;

@@ -679,7 +702,7 @@ static void mem_cgroup_update_tree(struct mem_cgroup *memcg, struct page *page)
 	struct mem_cgroup_per_node *mz;
 	struct mem_cgroup_tree_per_node *mctz;
 
-	mctz = soft_limit_tree_from_page(page);
+	mctz = soft_limit_tree_from_page(page, N_MEMORY);
 	if (!mctz)
 		return;
 	/*

@@ -688,7 +711,7 @@ static void mem_cgroup_update_tree(struct mem_cgroup *memcg, struct page *page)
 	 */
 	for (; memcg; memcg = parent_mem_cgroup(memcg)) {
 		mz = mem_cgroup_page_nodeinfo(memcg, page);
-		excess = soft_limit_excess(memcg);
+		excess = soft_limit_excess(memcg, N_MEMORY);
 		/*
 		 * We have to update the tree if mz is on RB-tree or
 		 * mem is over its softlimit.

@@ -718,7 +741,7 @@ static void mem_cgroup_remove_from_trees(struct mem_cgroup *memcg)
 
 	for_each_node(nid) {
 		mz = mem_cgroup_nodeinfo(memcg, nid);
-		mctz = soft_limit_tree_node(nid);
+		mctz = soft_limit_tree_node(nid, N_MEMORY);
 		if (mctz)
 			mem_cgroup_remove_exceeded(mz, mctz);
 	}

@@ -742,7 +765,7 @@ __mem_cgroup_largest_soft_limit_node(struct mem_cgroup_tree_per_node *mctz)
 	 * position in the tree.
 	 */
 	__mem_cgroup_remove_exceeded(mz, mctz);
-	if (!soft_limit_excess(mz->memcg) ||
+	if (!soft_limit_excess(mz->memcg, N_MEMORY) ||
 	    !css_tryget(&mz->memcg->css))
 		goto retry;
 done:

@@ -1805,7 +1828,7 @@ static int mem_cgroup_soft_reclaim(struct mem_cgroup *root_memcg,
 		.pgdat = pgdat,
 	};
 
-	excess = soft_limit_excess(root_memcg);
+	excess = soft_limit_excess(root_memcg, N_MEMORY);
 
 	while (1) {
 		victim = mem_cgroup_iter(root_memcg, victim, &reclaim);

@@ -1834,7 +1857,7 @@ static int mem_cgroup_soft_reclaim(struct mem_cgroup *root_memcg,
 		total += mem_cgroup_shrink_node(victim, gfp_mask, false,
 					pgdat, &nr_scanned);
 		*total_scanned += nr_scanned;
-		if (!soft_limit_excess(root_memcg))
+		if (!soft_limit_excess(root_memcg, N_MEMORY))
 			break;
 	}
 	mem_cgroup_iter_break(root_memcg, victim);

@@ -3457,7 +3480,7 @@ unsigned long mem_cgroup_soft_limit_reclaim(pg_data_t *pgdat, int order,
 	if (order > 0)
 		return 0;
 
-	mctz = soft_limit_tree_node(pgdat->node_id);
+	mctz = soft_limit_tree_node(pgdat->node_id, N_MEMORY);
 
 	/*
 	 * Do not even bother to check the largest node if the root

@@ -3513,7 +3536,7 @@ unsigned long mem_cgroup_soft_limit_reclaim(pg_data_t *pgdat, int order,
 		if (!reclaimed)
 			next_mz = __mem_cgroup_largest_soft_limit_node(mctz);
 
-		excess = soft_limit_excess(mz->memcg);
+		excess = soft_limit_excess(mz->memcg, N_MEMORY);
 		/*
 		 * One school of thought says that we should not add
 		 * back the node to the tree if reclaim returns 0.

@@ -7189,6 +7212,19 @@ static int __init mem_cgroup_init(void)
 		rtpn->rb_rightmost = NULL;
 		spin_lock_init(&rtpn->lock);
 		soft_limit_tree.rb_tree_per_node[node] = rtpn;
+
+		if (!node_state(node, N_TOPTIER)) {
+			soft_limit_toptier_tree.rb_tree_per_node[node] = NULL;
+			continue;
+		}
+
+		rtpn = kzalloc_node(sizeof(*rtpn), GFP_KERNEL,
+				    node_online(node) ? node : NUMA_NO_NODE);
+
+		rtpn->rb_root = RB_ROOT;
+		rtpn->rb_rightmost = NULL;
+		spin_lock_init(&rtpn->lock);
+		soft_limit_toptier_tree.rb_tree_per_node[node] = rtpn;
 	}
 
 	return 0;

Signature · 1 line

2.20.1

Tim Chen

author

2021-04-05 17:08 UTC #6 lore raw

RFC PATCH 6/11 mm: Handle top tier memory in cgroup soft limit memory tree utilities

In reply to Tim Chen

Update the utility functions __mem_cgroup_insert_exceeded() and __mem_cgroup_remove_exceeded(), to allow addition and removal of cgroups from the new red black tree that tracks the cgroups that exceed their toptier memory limits.

Update also the function +mem_cgroup_largest_soft_limit_node(), to allow returning the cgroup that has the largest exceess usage of toptier memory.

Signed-off-by Tim Chen <tim.c.chen-VuQAYsv1563Yd54FQh9/[email protected]>

---
 include/linux/memcontrol.h |   9 +++
 mm/memcontrol.c            | 152 +++++++++++++++++++++++++++----------
 2 files changed, 122 insertions(+), 39 deletions(-)

include/linux/memcontrol.h +9 -0

index 609d8590950c..0ed8ddfd5436 100644
@@ -124,6 +124,15 @@ struct mem_cgroup_per_node {
 	unsigned long		usage_in_excess;/* Set to the value by which */
 						/* the soft limit is exceeded*/
 	bool			on_tree;
+
+	struct rb_node		toptier_tree_node;	 /* RB tree node */
+	unsigned long		toptier_usage_in_excess; /* Set to the value by which */
+						         /* the soft limit is exceeded*/
+	bool			on_toptier_tree;
+
+	bool			congested;	/* memcg has many dirty pages */
+						/* backed by a congested BDI */
+
 	struct mem_cgroup	*memcg;		/* Back pointer, we cannot */
 						/* use container_of	   */
 };

mm/memcontrol.c +113 ~~-39~~

297 lines not on this page · raw message

Signature · 1 line

2.20.1

Tim Chen

author

2021-04-05 17:08 UTC #7 lore raw

RFC PATCH 7/11 mm: Account the total top tier memory in use

In reply to Tim Chen

Track the global top tier memory usage stats. They are used as the basis of deciding when to start demoting pages from memory cgroups that have exceeded their soft limit. We start reclaiming top tier memory when the total top tier memory is low.

Signed-off-by Tim Chen <tim.c.chen-VuQAYsv1563Yd54FQh9/[email protected]>

---
 include/linux/vmstat.h | 18 ++++++++++++++++++
 mm/vmstat.c            | 20 +++++++++++++++++---
 2 files changed, 35 insertions(+), 3 deletions(-)

include/linux/vmstat.h +18 -0

index e1a4fa9abb3a..a3ad5a937fd8 100644
@@ -139,6 +139,7 @@ static inline void vm_events_fold_cpu(int cpu)
  * Zone and node-based page accounting with per cpu differentials.
  */
 extern atomic_long_t vm_zone_stat[NR_VM_ZONE_STAT_ITEMS];
+extern atomic_long_t vm_toptier_zone_stat[NR_VM_ZONE_STAT_ITEMS];
 extern atomic_long_t vm_numa_stat[NR_VM_NUMA_STAT_ITEMS];
 extern atomic_long_t vm_node_stat[NR_VM_NODE_STAT_ITEMS];

@@ -175,6 +176,8 @@ static inline void zone_page_state_add(long x, struct zone *zone,
 {
 	atomic_long_add(x, &zone->vm_stat[item]);
 	atomic_long_add(x, &vm_zone_stat[item]);
+	if (node_state(zone->zone_pgdat->node_id, N_TOPTIER))
+		atomic_long_add(x, &vm_toptier_zone_stat[item]);
 }
 
 static inline void node_page_state_add(long x, struct pglist_data *pgdat,

@@ -212,6 +215,17 @@ static inline unsigned long global_node_page_state(enum node_stat_item item)
 	return global_node_page_state_pages(item);
 }
 
+static inline unsigned long global_toptier_zone_page_state(enum zone_stat_item item)
+{
+	long x = atomic_long_read(&vm_toptier_zone_stat[item]);
+
+#ifdef CONFIG_SMP
+	if (x < 0)
+		x = 0;
+#endif
+	return x;
+}
+
 static inline unsigned long zone_page_state(struct zone *zone,
 					enum zone_stat_item item)
 {

@@ -325,6 +339,8 @@ static inline void __inc_zone_state(struct zone *zone, enum zone_stat_item item)
 {
 	atomic_long_inc(&zone->vm_stat[item]);
 	atomic_long_inc(&vm_zone_stat[item]);
+	if (node_state(zone->zone_pgdat->node_id, N_TOPTIER))
+		atomic_long_inc(&vm_toptier_zone_stat[item]);
 }
 
 static inline void __inc_node_state(struct pglist_data *pgdat, enum node_stat_item item)

@@ -337,6 +353,8 @@ static inline void __dec_zone_state(struct zone *zone, enum zone_stat_item item)
 {
 	atomic_long_dec(&zone->vm_stat[item]);
 	atomic_long_dec(&vm_zone_stat[item]);
+	if (node_state(zone->zone_pgdat->node_id, N_TOPTIER))
+		atomic_long_dec(&vm_toptier_zone_stat[item]);
 }
 
 static inline void __dec_node_state(struct pglist_data *pgdat, enum node_stat_item item)

mm/vmstat.c +17 -3

index f299d2e89acb..b59efbcaef4e 100644
@@ -161,9 +161,11 @@ void vm_events_fold_cpu(int cpu)
  * vm_stat contains the global counters
  */
 atomic_long_t vm_zone_stat[NR_VM_ZONE_STAT_ITEMS] __cacheline_aligned_in_smp;
+atomic_long_t vm_toptier_zone_stat[NR_VM_ZONE_STAT_ITEMS] __cacheline_aligned_in_smp;
 atomic_long_t vm_numa_stat[NR_VM_NUMA_STAT_ITEMS] __cacheline_aligned_in_smp;
 atomic_long_t vm_node_stat[NR_VM_NODE_STAT_ITEMS] __cacheline_aligned_in_smp;
 EXPORT_SYMBOL(vm_zone_stat);
+EXPORT_SYMBOL(vm_toptier_zone_stat);
 EXPORT_SYMBOL(vm_numa_stat);
 EXPORT_SYMBOL(vm_node_stat);

@@ -695,7 +697,7 @@ EXPORT_SYMBOL(dec_node_page_state);
  * Returns the number of counters updated.
  */
 #ifdef CONFIG_NUMA
-static int fold_diff(int *zone_diff, int *numa_diff, int *node_diff)
+static int fold_diff(int *zone_diff, int *numa_diff, int *node_diff, int *toptier_diff)
 {
 	int i;
 	int changes = 0;

@@ -717,6 +719,11 @@ static int fold_diff(int *zone_diff, int *numa_diff, int *node_diff)
 			atomic_long_add(node_diff[i], &vm_node_stat[i]);
 			changes++;
 	}
+
+	for (i = 0; i < NR_VM_ZONE_STAT_ITEMS; i++)
+		if (toptier_diff[i]) {
+			atomic_long_add(toptier_diff[i], &vm_toptier_zone_stat[i]);
+	}
 	return changes;
 }
 #else

@@ -762,6 +769,7 @@ static int refresh_cpu_vm_stats(bool do_pagesets)
 	struct zone *zone;
 	int i;
 	int global_zone_diff[NR_VM_ZONE_STAT_ITEMS] = { 0, };
+	int global_toptier_diff[NR_VM_ZONE_STAT_ITEMS] = { 0, };
 #ifdef CONFIG_NUMA
 	int global_numa_diff[NR_VM_NUMA_STAT_ITEMS] = { 0, };
 #endif

@@ -779,6 +787,9 @@ static int refresh_cpu_vm_stats(bool do_pagesets)
 
 				atomic_long_add(v, &zone->vm_stat[i]);
 				global_zone_diff[i] += v;
+				if (node_state(zone->zone_pgdat->node_id, N_TOPTIER)) {
+					global_toptier_diff[i] +=v;
+				}
 #ifdef CONFIG_NUMA
 				/* 3 seconds idle till flush */
 				__this_cpu_write(p->expire, 3);

@@ -846,7 +857,7 @@ static int refresh_cpu_vm_stats(bool do_pagesets)
 
 #ifdef CONFIG_NUMA
 	changes += fold_diff(global_zone_diff, global_numa_diff,
-			     global_node_diff);
+			     global_node_diff, global_toptier_diff);
 #else
 	changes += fold_diff(global_zone_diff, global_node_diff);
 #endif

@@ -868,6 +879,7 @@ void cpu_vm_stats_fold(int cpu)
 	int global_numa_diff[NR_VM_NUMA_STAT_ITEMS] = { 0, };
 #endif
 	int global_node_diff[NR_VM_NODE_STAT_ITEMS] = { 0, };
+	int global_toptier_diff[NR_VM_NODE_STAT_ITEMS] = { 0, };
 
 	for_each_populated_zone(zone) {
 		struct per_cpu_pageset *p;

@@ -910,11 +922,13 @@ void cpu_vm_stats_fold(int cpu)
 				p->vm_node_stat_diff[i] = 0;
 				atomic_long_add(v, &pgdat->vm_stat[i]);
 				global_node_diff[i] += v;
+				if (node_state(pgdat->node_id, N_TOPTIER))
+					global_toptier_diff[i] +=v;
 			}
 	}
 
 #ifdef CONFIG_NUMA
-	fold_diff(global_zone_diff, global_numa_diff, global_node_diff);
+	fold_diff(global_zone_diff, global_numa_diff, global_node_diff, global_toptier_diff);
 #else
 	fold_diff(global_zone_diff, global_node_diff);
 #endif

Signature · 1 line

2.20.1

Tim Chen

author

2021-04-05 17:08 UTC #8 lore raw

RFC PATCH 8/11 mm: Add toptier option for mem_cgroup_soft_limit_reclaim()

In reply to Tim Chen

Add toptier relcaim type in mem_cgroup_soft_limit_reclaim(). This option reclaims top tier memory from cgroups in the order of its excess usage of top tier memory.

Signed-off-by Tim Chen <tim.c.chen-VuQAYsv1563Yd54FQh9/[email protected]>

---
 include/linux/memcontrol.h |  9 ++++---
 mm/memcontrol.c            | 48 ++++++++++++++++++++++++--------------
 mm/vmscan.c                |  4 ++--
 3 files changed, 39 insertions(+), 22 deletions(-)

include/linux/memcontrol.h +6 -3

index 0ed8ddfd5436..c494c4b11ba2 100644
@@ -21,6 +21,7 @@
 #include <linux/vmstat.h>
 #include <linux/writeback.h>
 #include <linux/page-flags.h>
+#include <linux/nodemask.h>
 
 struct mem_cgroup;
 struct obj_cgroup;

@@ -1003,7 +1004,8 @@ static inline void mod_memcg_lruvec_state(struct lruvec *lruvec,
 
 unsigned long mem_cgroup_soft_limit_reclaim(pg_data_t *pgdat, int order,
 						gfp_t gfp_mask,
-						unsigned long *total_scanned);
+						unsigned long *total_scanned,
+						enum node_states type);
 
 void __count_memcg_events(struct mem_cgroup *memcg, enum vm_event_item idx,
 			  unsigned long count);

@@ -1421,8 +1423,9 @@ static inline void mod_lruvec_kmem_state(void *p, enum node_stat_item idx,
 
 static inline
 unsigned long mem_cgroup_soft_limit_reclaim(pg_data_t *pgdat, int order,
-					    gfp_t gfp_mask,
-					    unsigned long *total_scanned)
+						gfp_t gfp_mask,
+						unsigned long *total_scanned,
+						enum node_states type)
 {
 	return 0;
 }

mm/memcontrol.c +31 ~~-17~~

125 lines not on this page · raw message

mm/vmscan.c +2 -2

index 3b200b7170a9..11bb0c6fa524 100644
@@ -3134,7 +3134,7 @@ static void shrink_zones(struct zonelist *zonelist, struct scan_control *sc)
 			nr_soft_scanned = 0;
 			nr_soft_reclaimed = mem_cgroup_soft_limit_reclaim(zone->zone_pgdat,
 						sc->order, sc->gfp_mask,
-						&nr_soft_scanned);
+						&nr_soft_scanned, N_MEMORY);
 			sc->nr_reclaimed += nr_soft_reclaimed;
 			sc->nr_scanned += nr_soft_scanned;
 			/* need some check for avoid more shrink_zone() */

@@ -3849,7 +3849,7 @@ static int balance_pgdat(pg_data_t *pgdat, int order, int highest_zoneidx)
 		sc.nr_scanned = 0;
 		nr_soft_scanned = 0;
 		nr_soft_reclaimed = mem_cgroup_soft_limit_reclaim(pgdat, sc.order,
-						sc.gfp_mask, &nr_soft_scanned);
+						sc.gfp_mask, &nr_soft_scanned, N_MEMORY);
 		sc.nr_reclaimed += nr_soft_reclaimed;
 
 		/*

Signature · 1 line

2.20.1

Tim Chen

author

2021-04-05 17:08 UTC #9 lore raw

RFC PATCH 9/11 mm: Use kswapd to demote pages when toptier memory is tight

In reply to Tim Chen

Demote pages from memory cgroup that has excess toptier memory usage when top tier memory is tight.

When free top tier memory falls below this fraction "toptier_scale_factor/10000" of overall toptier memory in a node, kswapd reclaims top tier memory from those mem cgroups that exceeded their toptier memory soft limit by deomoting the top tier pages to lower memory tier.

Signed-off-by Tim Chen <tim.c.chen-VuQAYsv1563Yd54FQh9/[email protected]>

---
 Documentation/admin-guide/sysctl/vm.rst | 12 +++++
 include/linux/mmzone.h                  |  2 +
 mm/page_alloc.c                         | 14 +++++
 mm/vmscan.c                             | 69 ++++++++++++++++++++++++-
 4 files changed, 96 insertions(+), 1 deletion(-)

Documentation/admin-guide/sysctl/vm.rst +12 -0

index 9de3847c3469..6b49e2e90953 100644
@@ -74,6 +74,7 @@ Currently, these files are in /proc/sys/vm:
 - vfs_cache_pressure
 - watermark_boost_factor
 - watermark_scale_factor
+- toptier_scale_factor
 - zone_reclaim_mode

@@ -962,6 +963,17 @@ too small for the allocation bursts occurring in the system. This knob
 can then be used to tune kswapd aggressiveness accordingly.
 
 
+toptier_scale_factor
+====================
+
+This factor controls when kswapd wakes up to demote pages of those
+cgroups that have exceeded their memory soft limit.
+
+The unit is in fractions of 10,000. The default value of 2000 means the
+if there are less than 20% of free top tier memory in the
+node/system, we will start to demote pages of those memory cgroups
+that have exceeded their soft memory limit.
+
 zone_reclaim_mode
 =================

include/linux/mmzone.h +2 -0

index bbe649c4fdee..4ee0073d255f 100644
@@ -332,12 +332,14 @@ enum zone_watermarks {
 	WMARK_MIN,
 	WMARK_LOW,
 	WMARK_HIGH,
+	WMARK_TOPTIER,
 	NR_WMARK
 };
 
 #define min_wmark_pages(z) (z->_watermark[WMARK_MIN] + z->watermark_boost)
 #define low_wmark_pages(z) (z->_watermark[WMARK_LOW] + z->watermark_boost)
 #define high_wmark_pages(z) (z->_watermark[WMARK_HIGH] + z->watermark_boost)
+#define toptier_wmark_pages(z) (z->_watermark[WMARK_TOPTIER] + z->watermark_boost)
 #define wmark_pages(z, i) (z->_watermark[i] + z->watermark_boost)
 
 struct per_cpu_pages {

mm/page_alloc.c +14 -0

index 471a2c342c4f..20f3caee60f3 100644
@@ -7964,6 +7964,20 @@ static void __setup_per_zone_wmarks(void)
 		zone->_watermark[WMARK_LOW]  = min_wmark_pages(zone) + tmp;
 		zone->_watermark[WMARK_HIGH] = min_wmark_pages(zone) + tmp * 2;
 
+		tmp = mult_frac(zone_managed_pages(zone),
+				toptier_scale_factor, 10000);
+		/*
+		 * Clamp toptier watermark between twice high watermark
+		 * and max managed pages.
+		 */
+		if (tmp < 2 * zone->_watermark[WMARK_HIGH])
+			tmp = 2 * zone->_watermark[WMARK_HIGH];
+		if (tmp > zone_managed_pages(zone))
+			tmp = zone_managed_pages(zone);
+		zone->_watermark[WMARK_TOPTIER] = tmp;
+
+		zone->watermark_boost = 0;
+
 		spin_unlock_irqrestore(&zone->lock, flags);
 	}

mm/vmscan.c +68 -1

99 lines not on this page · raw message

Signature · 1 line

2.20.1

Tim Chen

author

2021-04-05 17:08 UTC #10 lore raw

RFC PATCH 10/11 mm: Set toptier_scale_factor via sysctl

In reply to Tim Chen

Update the toptier_scale_factor via sysctl. This variable determines when kswapd wakes up to recalaim toptier memory from those mem cgroups exceeding their toptier memory limit.

Signed-off-by Tim Chen <tim.c.chen-VuQAYsv1563Yd54FQh9/[email protected]>

---
 include/linux/mm.h     |  4 ++++
 include/linux/mmzone.h |  2 ++
 kernel/sysctl.c        | 10 ++++++++++
 mm/page_alloc.c        | 15 +++++++++++++++
 mm/vmstat.c            |  2 ++
 5 files changed, 33 insertions(+)

include/linux/mm.h +4 -0

index a43429d51fc0..af39e221d0f9 100644
@@ -3179,6 +3179,10 @@ static inline bool debug_guardpage_enabled(void) { return false; }
 static inline bool page_is_guard(struct page *page) { return false; }
 #endif /* CONFIG_DEBUG_PAGEALLOC */
 
+#ifdef CONFIG_MIGRATION
+extern int toptier_scale_factor;
+#endif
+
 #if MAX_NUMNODES > 1
 void __init setup_nr_node_ids(void);
 #else

include/linux/mmzone.h +2 -0

index 4ee0073d255f..789319dffe1c 100644
@@ -1003,6 +1003,8 @@ int min_free_kbytes_sysctl_handler(struct ctl_table *, int, void *, size_t *,
 		loff_t *);
 int watermark_scale_factor_sysctl_handler(struct ctl_table *, int, void *,
 		size_t *, loff_t *);
+int toptier_scale_factor_sysctl_handler(struct ctl_table *, int,
+					void __user *, size_t *, loff_t *);
 extern int sysctl_lowmem_reserve_ratio[MAX_NR_ZONES];
 int lowmem_reserve_ratio_sysctl_handler(struct ctl_table *, int, void *,
 		size_t *, loff_t *);

kernel/sysctl.c +10 -0

index 57f89fe1b0f2..e97c974f37b7 100644
@@ -112,6 +112,7 @@ static int sixty = 60;
 #endif
 
 static int __maybe_unused neg_one = -1;
+static int __maybe_unused one = 1;
 static int __maybe_unused two = 2;
 static int __maybe_unused three = 3;
 static int __maybe_unused four = 4;

@@ -2956,6 +2957,15 @@ static struct ctl_table vm_table[] = {
 		.extra1		= SYSCTL_ONE,
 		.extra2		= &one_thousand,
 	},
+	{
+		.procname       = "toptier_scale_factor",
+		.data           = &toptier_scale_factor,
+		.maxlen         = sizeof(toptier_scale_factor),
+		.mode           = 0644,
+		.proc_handler   = toptier_scale_factor_sysctl_handler,
+		.extra1         = &one,
+		.extra2         = &ten_thousand,
+	},
 	{
 		.procname	= "percpu_pagelist_fraction",
 		.data		= &percpu_pagelist_fraction,

mm/page_alloc.c +15 -0

index 20f3caee60f3..91212a837d8e 100644
@@ -8094,6 +8094,21 @@ int watermark_scale_factor_sysctl_handler(struct ctl_table *table, int write,
 	return 0;
 }
 
+int toptier_scale_factor_sysctl_handler(struct ctl_table *table, int write,
+	void __user *buffer, size_t *length, loff_t *ppos)
+{
+	int rc;
+
+	rc = proc_dointvec_minmax(table, write, buffer, length, ppos);
+	if (rc)
+	       return rc;
+
+	if (write)
+		setup_per_zone_wmarks();
+
+	return 0;
+}
+
 #ifdef CONFIG_NUMA
 static void setup_min_unmapped_ratio(void)
 {

mm/vmstat.c +2 -0

index b59efbcaef4e..c581753cf076 100644
@@ -1658,6 +1658,7 @@ static void zoneinfo_show_print(struct seq_file *m, pg_data_t *pgdat,
 		   "\n        min      %lu"
 		   "\n        low      %lu"
 		   "\n        high     %lu"
+		   "\n        toptier  %lu"
 		   "\n        spanned  %lu"
 		   "\n        present  %lu"
 		   "\n        managed  %lu",

@@ -1665,6 +1666,7 @@ static void zoneinfo_show_print(struct seq_file *m, pg_data_t *pgdat,
 		   min_wmark_pages(zone),
 		   low_wmark_pages(zone),
 		   high_wmark_pages(zone),
+		   toptier_wmark_pages(zone),
 		   zone->spanned_pages,
 		   zone->present_pages,
 		   zone_managed_pages(zone));

Signature · 1 line

2.20.1

Tim Chen

author

2021-04-05 17:08 UTC #11 lore raw

RFC PATCH 11/11 mm: Wakeup kswapd if toptier memory need soft reclaim

In reply to Tim Chen

Detect during page allocation whether free toptier memory is low. If so, wake up kswapd to reclaim memory from those mem cgroups that have exceeded their limit.

Signed-off-by Tim Chen <tim.c.chen-VuQAYsv1563Yd54FQh9/[email protected]>

---
 include/linux/mmzone.h | 3 +++
 mm/page_alloc.c        | 2 ++
 mm/vmscan.c            | 2 +-
 3 files changed, 6 insertions(+), 1 deletion(-)

include/linux/mmzone.h +3 -0

index 789319dffe1c..3603948e95cc 100644
@@ -886,6 +886,8 @@ bool zone_watermark_ok(struct zone *z, unsigned int order,
 		unsigned int alloc_flags);
 bool zone_watermark_ok_safe(struct zone *z, unsigned int order,
 		unsigned long mark, int highest_zoneidx);
+bool pgdat_toptier_balanced(pg_data_t *pgdat, int order, int classzone_idx);
+
 /*
  * Memory initialization context, use to differentiate memory added by
  * the platform statically or via memory hotplug interface.

@@ -1466,5 +1468,6 @@ void sparse_init(void);
 #endif
 
 #endif /* !__GENERATING_BOUNDS.H */
+
 #endif /* !__ASSEMBLY__ */
 #endif /* _LINUX_MMZONE_H */

mm/page_alloc.c +2 -0

index 91212a837d8e..ca8aa789a967 100644
@@ -3519,6 +3519,8 @@ struct page *rmqueue(struct zone *preferred_zone,
 	if (test_bit(ZONE_BOOSTED_WATERMARK, &zone->flags)) {
 		clear_bit(ZONE_BOOSTED_WATERMARK, &zone->flags);
 		wakeup_kswapd(zone, 0, 0, zone_idx(zone));
+	} else if (!pgdat_toptier_balanced(zone->zone_pgdat, order, zone_idx(zone))) {
+		wakeup_kswapd(zone, 0, 0, zone_idx(zone));
 	}
 
 	VM_BUG_ON_PAGE(page && bad_range(zone, page), page);

mm/vmscan.c +1 -1

index 270880c8baef..8fe709e3f5e4 100644
@@ -3625,7 +3625,7 @@ static bool pgdat_balanced(pg_data_t *pgdat, int order, int highest_zoneidx)
 	return false;
 }
 
-static bool pgdat_toptier_balanced(pg_data_t *pgdat, int order, int classzone_idx)
+bool pgdat_toptier_balanced(pg_data_t *pgdat, int order, int classzone_idx)
 {
 	int i;
 	unsigned long mark;

Signature · 1 line

2.20.1

Michal Hocko

reviewer

2021-04-06 09:08 UTC #12 lore raw

In reply to Tim Chen

On Mon 05-04-21 10:08:24, Tim Chen wrote:
[...]

Quoted 7 lines

To make fine grain cgroup based management of the precious top tier DRAM memory possible, this patchset adds a few new features:
1. Provides memory monitors on the amount of top tier memory used per cgroup and by the system as a whole.
2. Applies soft limits on the top tier memory each cgroup uses
3. Enables kswapd to demote top tier pages from cgroup with excess top tier memory usages.

Could you be more specific on how this interface is supposed to be used?

Quoted 7 lines

This allows us to provision different amount of top tier memory to each cgroup according to the cgroup's latency need.
The patchset is based on cgroup v1 interface. One shortcoming of the v1 interface is the limit on the cgroup is a soft limit, so a cgroup can exceed the limit quite a bit before reclaim before page demotion reins it in.

I have to say that I dislike abusing soft limit reclaim for this. In the past we have learned that the existing implementation is unfixable and changing the existing semantic impossible due to backward compatibility. So I would really prefer the soft limit just find its rest rather than see new potential usecases.

I haven't really looked into details of this patchset but from a cursory look it seems like you are actually introducing a NUMA aware limits into memcg that would control consumption from some nodes differently than other nodes. This would be rather alien concept to the existing memcg infrastructure IMO. It looks like it is fusing borders between memcg and cputset controllers.

You also seem to be basing the interface on the very specific usecase. Can we expect that there will be many different tiers requiring their own balancing?

Signature · 2 lines

Michal Hocko
SUSE Labs

Tim Chen

author

2021-04-07 22:33 UTC #13 lore raw

Quoted 10 lines · Michal Hocko

On Mon 05-04-21 10:08:24, Tim Chen wrote:
[...]
Quoted 7 lines
To make fine grain cgroup based management of the precious top tier DRAM memory possible, this patchset adds a few new features:
1. Provides memory monitors on the amount of top tier memory used per cgroup and by the system as a whole.
2. Applies soft limits on the top tier memory each cgroup uses
3. Enables kswapd to demote top tier pages from cgroup with excess top tier memory usages.

Michal,

Thanks for giving your feedback. Much appreciated.

Quoted 1 line

Could you be more specific on how this interface is supposed to be used?

We created a README section on the cgroup control part of this patchset:
https://git.kernel.org/pub/scm/linux/kernel/git/vishal/tiering.git/commit/?h=tiering-0.71&id=20f20be02671384470c7cd8f66b56a9061a4071f
to illustrate how this interface should be used.

The top tier memory used is reported in

memory.toptier_usage_in_bytes

The amount of top tier memory usable by each cgroup without triggering page reclaim is controlled by the

memory.toptier_soft_limit_in_bytes

knob for each cgroup.

We anticipate that for cgroup v2, we will have

memory_toptier.max (max allowed top tier memory) memory_toptier.high (aggressive page demotion from top tier memory) memory_toptier.min (not to page demote from top tier memory at this threshold)

this is analogous to existing controllers
memory.max, memory.high and memory.min

Quoted 9 lines

Quoted 8 lines
This allows us to provision different amount of top tier memory to each cgroup according to the cgroup's latency need.
The patchset is based on cgroup v1 interface. One shortcoming of the v1 interface is the limit on the cgroup is a soft limit, so a cgroup can exceed the limit quite a bit before reclaim before page demotion reins it in.

Quoted 5 lines

I have to say that I dislike abusing soft limit reclaim for this. In the past we have learned that the existing implementation is unfixable and changing the existing semantic impossible due to backward compatibility. So I would really prefer the soft limit just find its rest rather than see new potential usecases.

Do you think we can reuse some of the existing soft reclaim machinery for the v2 interface?

More particularly, can we treat memory_toptier.high in cgroup v2 as a soft limit? We sort how much each mem cgroup exceeds memory_toptier.high and
go after the cgroup that have largest excess first for page demotion. Will appreciate if you can shed some insights on what could go wrong with such an approach.

Quoted 7 lines

I haven't really looked into details of this patchset but from a cursory look it seems like you are actually introducing a NUMA aware limits into memcg that would control consumption from some nodes differently than other nodes. This would be rather alien concept to the existing memcg infrastructure IMO. It looks like it is fusing borders between memcg and cputset controllers.

Want to make sure I understand what you mean by NUMA aware limits. Yes, in the patch set, it does treat the NUMA nodes differently. We are putting constraint on the "top tier" RAM nodes vs the lower tier PMEM nodes. Is this what you mean? I can see it does has some flavor of cpuset controller. In this case, it doesn't explicitly set a node as allowed or forbidden as in cpuset, but put some constraints on the usage of a group of nodes.

Do you have suggestions on alternative controller for allocating tiered memory resource?

Quoted 5 lines

You also seem to be basing the interface on the very specific usecase. Can we expect that there will be many different tiers requiring their own balancing?

You mean more than two tiers of memory? We did think a bit about system that has stuff like high bandwidth memory that's faster than DRAM. Our thought is usage and freeing of those memory will require explicit assignment (not used by default), so will be outside the realm of auto balancing. So at this point, we think two tiers will be good.

Tim

Michal Hocko

reviewer

2021-04-08 11:52 UTC #14 lore raw

Quoted 16 lines · Tim Chen

Quoted 10 lines · Michal Hocko
On Mon 05-04-21 10:08:24, Tim Chen wrote:
[...]
Quoted 7 lines
To make fine grain cgroup based management of the precious top tier DRAM memory possible, this patchset adds a few new features:
1. Provides memory monitors on the amount of top tier memory used per cgroup and by the system as a whole.
2. Applies soft limits on the top tier memory each cgroup uses
3. Enables kswapd to demote top tier pages from cgroup with excess top tier memory usages.
Michal,

Quoted 7 lines

Thanks for giving your feedback. Much appreciated.
Quoted 1 line
Could you be more specific on how this interface is supposed to be used?
We created a README section on the cgroup control part of this patchset:
https://git.kernel.org/pub/scm/linux/kernel/git/vishal/tiering.git/commit/?h=tiering-0.71&id=20f20be02671384470c7cd8f66b56a9061a4071f
to illustrate how this interface should be used.

I have to confess I didn't get to look at demotion patches yet.

Quoted 8 lines

The top tier memory used is reported in
memory.toptier_usage_in_bytes
The amount of top tier memory usable by each cgroup without triggering page reclaim is controlled by the
memory.toptier_soft_limit_in_bytes

Are you trying to say that soft limit acts as some sort of guarantee? Does that mean that if the memcg is under memory pressure top tiear memory is opted out from any reclaim if the usage is not in excess?

From you previous email it sounds more like the limit is evaluated on the global memory pressure to balance specific memcgs which are in excess when trying to reclaim/demote a toptier numa node.

Soft limit reclaim has several problems. Those are historical and therefore the behavior cannot be changed. E.g. go after the biggest excessed memcg (with priority 0 - aka potential full LRU scan) and then continue with a normal reclaim. This can be really disruptive to the top user.

So you can likely define a more sane semantic. E.g. push back memcgs proporitional to their excess but then we have two different soft limits behavior which is bad as well. I am not really sure there is a sensible way out by (ab)using soft limit here.

Also I am not really sure how this is going to be used in practice. There is no soft limit by default. So opting in would effectivelly discriminate those memcgs. There has been a similar problem with the soft limit we have in general. Is this really what you are looing for? What would be a typical usecase?

[...]

Quoted 11 lines

Quoted 10 lines
Quoted 4 lines
The patchset is based on cgroup v1 interface. One shortcoming of the v1 interface is the limit on the cgroup is a soft limit, so a cgroup can exceed the limit quite a bit before reclaim before page demotion reins it in.
I have to say that I dislike abusing soft limit reclaim for this. In the past we have learned that the existing implementation is unfixable and changing the existing semantic impossible due to backward compatibility. So I would really prefer the soft limit just find its rest rather than see new potential usecases.

Quoted 4 lines

Do you think we can reuse some of the existing soft reclaim machinery for the v2 interface?
More particularly, can we treat memory_toptier.high in cgroup v2 as a soft limit?

No, you should follow existing limits semantics. High limit acts as a allocation throttling interface.

Quoted 4 lines

We sort how much each mem cgroup exceeds memory_toptier.high and go after the cgroup that have largest excess first for page demotion. Will appreciate if you can shed some insights on what could go wrong with such an approach.

This cannot work as a thorttling interface.

Quoted 11 lines

Quoted 6 lines
I haven't really looked into details of this patchset but from a cursory look it seems like you are actually introducing a NUMA aware limits into memcg that would control consumption from some nodes differently than other nodes. This would be rather alien concept to the existing memcg infrastructure IMO. It looks like it is fusing borders between memcg and cputset controllers.
Want to make sure I understand what you mean by NUMA aware limits. Yes, in the patch set, it does treat the NUMA nodes differently. We are putting constraint on the "top tier" RAM nodes vs the lower tier PMEM nodes. Is this what you mean?

What I am trying to say (and I have brought that up when demotion has been discussed at LSFMM) is that the implementation shouldn't be PMEM aware. The specific technology shouldn't be imprinted into the interface. Fundamentally you are trying to balance memory among NUMA nodes as we do not have other abstraction to use. So rather than talking about top, secondary, nth tier we have different NUMA nodes with different characteristics and you want to express your "priorities" for them.

Quoted 6 lines

I can see it does has
some flavor of cpuset controller. In this case, it doesn't explicitly set a node as allowed or forbidden as in cpuset, but put some constraints on the usage of a group of nodes.
Do you have suggestions on alternative controller for allocating tiered memory resource?

I am not really sure what would be the best interface to be honest. Maybe we want to carve this into memcg in some form of node priorities for the reclaim. Any of the existing limits is numa aware so far. Maybe we want to say hammer this node more than others if there is a memory pressure. Not sure that would help you particular usecase though.

Quoted 10 lines

Quoted 4 lines
You also seem to be basing the interface on the very specific usecase. Can we expect that there will be many different tiers requiring their own balancing?
You mean more than two tiers of memory? We did think a bit about system that has stuff like high bandwidth memory that's faster than DRAM. Our thought is usage and freeing of those memory will require explicit assignment (not used by default), so will be outside the realm of auto balancing. So at this point, we think two tiers will be good.

Please keep in mind that once there is an interface it will be impossible to change in the future. So do not bind yourself to the 2 tier setups that you have in hands right now.

Signature · 2 lines

Michal Hocko
SUSE Labs

Tim Chen

author

2021-04-09 23:26 UTC #15 lore raw

Quoted 9 lines · Michal Hocko

Quoted 8 lines
The top tier memory used is reported in
memory.toptier_usage_in_bytes
The amount of top tier memory usable by each cgroup without triggering page reclaim is controlled by the
memory.toptier_soft_limit_in_bytes

Michal,

Thanks for your comments. I will like to take a step back and look at the eventual goal we envision: a mechanism to partition the tiered memory between the cgroups.

A typical use case may be a system with two set of tasks. One set of task is very latency sensitive and we desire instantaneous response from them. Another set of tasks will be running batch jobs
were latency and performance is not critical. In this case,
we want to carve out enough top tier memory such that the working set of the latency sensitive tasks can fit entirely in the top tier memory. The rest of the top tier memory can be assigned to the background tasks.

To achieve such cgroup based tiered memory management, we probably want something like the following.

For generalization let's say that there are N tiers of memory t_0, t_1 ... t_N-1, where tier t_0 sits at the top and demotes to the lower tier.
We envision for this top tier memory t0 the following knobs and counters in the cgroup memory controller

memory_t0.current Current usage of tier 0 memory by the cgroup.

memory_t0.min If tier 0 memory used by the cgroup falls below this low
boundary, the memory will not be subjected to demotion to lower tiers to free up memory at tier 0.

memory_t0.low Above this boundary, the tier 0 memory will be subjected
to demotion. The demotion pressure will be proportional to the overage.

memory_t0.high If tier 0 memory used by the cgroup exceeds this high
boundary, allocation of tier 0 memory by the cgroup will be throttled. The tier 0 memory used by this cgroup will also be subjected to heavy demotion.

memory_t0.max This will be a hard usage limit of tier 0 memory on the cgroup.

If needed, memory_t[12...].current/min/low/high for additional tiers can be added. This follows closely with the design of the general memory controller interface.

Will such an interface looks sane and acceptable with everyone?

The patch set I posted is meant to be a straw man cgroup v1 implementation and I readily admits that it falls short of the eventual functionality we want to achieve. It is meant to solicit feedback from everyone on how the tiered memory management should work.

Quoted 1 line

Are you trying to say that soft limit acts as some sort of guarantee?

No, the soft limit does not offers guarantee. It will only serves to keep the usage of the top tier memory in the vicinity of the soft limits.

Quoted 2 lines

Does that mean that if the memcg is under memory pressure top tiear memory is opted out from any reclaim if the usage is not in excess?

In the prototype implementation, regular memory reclaim is still in effect if we are under heavy memory pressure.

Quoted 4 lines

From you previous email it sounds more like the limit is evaluated on the global memory pressure to balance specific memcgs which are in excess when trying to reclaim/demote a toptier numa node.

On a top tier node, if the free memory on the node falls below a percentage, then we will start to reclaim/demote from the node.

Quoted 6 lines

Soft limit reclaim has several problems. Those are historical and therefore the behavior cannot be changed. E.g. go after the biggest excessed memcg (with priority 0 - aka potential full LRU scan) and then continue with a normal reclaim. This can be really disruptive to the top user.

Thanks for pointing out these problems with soft limit explicitly.

Quoted 17 lines

So you can likely define a more sane semantic. E.g. push back memcgs proporitional to their excess but then we have two different soft limits behavior which is bad as well. I am not really sure there is a sensible way out by (ab)using soft limit here.
Also I am not really sure how this is going to be used in practice. There is no soft limit by default. So opting in would effectivelly discriminate those memcgs. There has been a similar problem with the soft limit we have in general. Is this really what you are looing for? What would be a typical usecase?
Quoted 4 lines
Want to make sure I understand what you mean by NUMA aware limits. Yes, in the patch set, it does treat the NUMA nodes differently. We are putting constraint on the "top tier" RAM nodes vs the lower tier PMEM nodes. Is this what you mean?

Quoted 7 lines

What I am trying to say (and I have brought that up when demotion has been discussed at LSFMM) is that the implementation shouldn't be PMEM aware. The specific technology shouldn't be imprinted into the interface. Fundamentally you are trying to balance memory among NUMA nodes as we do not have other abstraction to use. So rather than talking about top, secondary, nth tier we have different NUMA nodes with different characteristics and you want to express your "priorities" for them.

With node priorities, how would the system reserve enough high performance memory for those performance critical task cgroup?

By priority, do you mean the order of allocation of nodes for a cgroup? Or you mean that all the similar performing memory node will be grouped in the same priority?

Tim

Shakeel Butt

2021-04-12 19:20 UTC #16 lore raw

Quoted 51 lines · Tim Chen

Quoted 9 lines · Michal Hocko
Quoted 8 lines
The top tier memory used is reported in
memory.toptier_usage_in_bytes
The amount of top tier memory usable by each cgroup without triggering page reclaim is controlled by the
memory.toptier_soft_limit_in_bytes
Michal,
Thanks for your comments. I will like to take a step back and look at the eventual goal we envision: a mechanism to partition the tiered memory between the cgroups.
A typical use case may be a system with two set of tasks. One set of task is very latency sensitive and we desire instantaneous response from them. Another set of tasks will be running batch jobs
were latency and performance is not critical. In this case,
we want to carve out enough top tier memory such that the working set of the latency sensitive tasks can fit entirely in the top tier memory. The rest of the top tier memory can be assigned to the background tasks.
To achieve such cgroup based tiered memory management, we probably want something like the following.
For generalization let's say that there are N tiers of memory t_0, t_1 ... t_N-1, where tier t_0 sits at the top and demotes to the lower tier.
We envision for this top tier memory t0 the following knobs and counters in the cgroup memory controller
memory_t0.current Current usage of tier 0 memory by the cgroup.
memory_t0.min If tier 0 memory used by the cgroup falls below this low
boundary, the memory will not be subjected to demotion to lower tiers to free up memory at tier 0.
memory_t0.low Above this boundary, the tier 0 memory will be subjected
to demotion. The demotion pressure will be proportional to the overage.
memory_t0.high If tier 0 memory used by the cgroup exceeds this high
boundary, allocation of tier 0 memory by the cgroup will be throttled. The tier 0 memory used by this cgroup will also be subjected to heavy demotion.

Quoted 7 lines

memory_t0.max This will be a hard usage limit of tier 0 memory on the cgroup.
If needed, memory_t[12...].current/min/low/high for additional tiers can be added. This follows closely with the design of the general memory controller interface.
Will such an interface looks sane and acceptable with everyone?

I have a couple of questions. Let's suppose we have a two socket system. Node 0 (DRAM+CPUs), Node 1 (DRAM+CPUs), Node 2 (PMEM on socket 0 along with Node 0) and Node 3 (PMEM on socket 1 along with Node 1). Based on the tier definition of this patch series, tier_0: {node_0, node_1} and tier_1: {node_2, node_3}.

My questions are:

1) Can we assume that the cost of access within a tier will always be less than the cost of access from the tier? (node_0 <-> node_1 vs node_0 <-> node_2)
2) If yes to (1), is that assumption future proof? Will the future systems with DRAM over CXL support have the same characteristics?
3) Will the cost of access from tier_0 to tier_1 be uniform? (node_0 <-> node_2 vs node_0 <-> node_3). For jobs running on node_0, node_3 might be third tier and similarly for jobs running on node_1, node_2 might be third tier.

The reason I am asking these questions is that the statically partitioning memory nodes into tiers will inherently add platform specific assumptions in the user API.

Assumptions like:
1) Access within tier is always cheaper than across tier.
2) Access from tier_i to tier_i+1 has uniform cost.

The reason I am more inclined towards having numa centric control is that we don't have to make these assumptions. Though the usability will be more difficult. Greg (CCed) has some ideas on making it better and we will share our proposal after polishing it a bit more.

Jonathan Cameron

2021-04-14 08:59 UTC #17 lore raw

Quoted 66 lines · On Mon, 12 Apr 2021 12:20:22 -0700 Shakeel Butt…

Quoted 51 lines · Tim Chen
Quoted 9 lines · Michal Hocko
Quoted 8 lines
The top tier memory used is reported in
memory.toptier_usage_in_bytes
The amount of top tier memory usable by each cgroup without triggering page reclaim is controlled by the
memory.toptier_soft_limit_in_bytes
Michal,
Thanks for your comments. I will like to take a step back and look at the eventual goal we envision: a mechanism to partition the tiered memory between the cgroups.
A typical use case may be a system with two set of tasks. One set of task is very latency sensitive and we desire instantaneous response from them. Another set of tasks will be running batch jobs
were latency and performance is not critical. In this case,
we want to carve out enough top tier memory such that the working set of the latency sensitive tasks can fit entirely in the top tier memory. The rest of the top tier memory can be assigned to the background tasks.
To achieve such cgroup based tiered memory management, we probably want something like the following.
For generalization let's say that there are N tiers of memory t_0, t_1 ... t_N-1, where tier t_0 sits at the top and demotes to the lower tier.
We envision for this top tier memory t0 the following knobs and counters in the cgroup memory controller
memory_t0.current Current usage of tier 0 memory by the cgroup.
memory_t0.min If tier 0 memory used by the cgroup falls below this low
boundary, the memory will not be subjected to demotion to lower tiers to free up memory at tier 0.
memory_t0.low Above this boundary, the tier 0 memory will be subjected
to demotion. The demotion pressure will be proportional to the overage.
memory_t0.high If tier 0 memory used by the cgroup exceeds this high
boundary, allocation of tier 0 memory by the cgroup will be throttled. The tier 0 memory used by this cgroup will also be subjected to heavy demotion.
Quoted 7 lines
memory_t0.max This will be a hard usage limit of tier 0 memory on the cgroup.
If needed, memory_t[12...].current/min/low/high for additional tiers can be added. This follows closely with the design of the general memory controller interface.
Will such an interface looks sane and acceptable with everyone?
I have a couple of questions. Let's suppose we have a two socket system. Node 0 (DRAM+CPUs), Node 1 (DRAM+CPUs), Node 2 (PMEM on socket 0 along with Node 0) and Node 3 (PMEM on socket 1 along with Node 1). Based on the tier definition of this patch series, tier_0: {node_0, node_1} and tier_1: {node_2, node_3}.

Quoted 5 lines

My questions are:
1) Can we assume that the cost of access within a tier will always be less than the cost of access from the tier? (node_0 <-> node_1 vs node_0 <-> node_2)

No in large systems even it we can make this assumption in 2 socket ones.

Quoted 10 lines

2) If yes to (1), is that assumption future proof? Will the future systems with DRAM over CXL support have the same characteristics?
3) Will the cost of access from tier_0 to tier_1 be uniform? (node_0 <-> node_2 vs node_0 <-> node_3). For jobs running on node_0, node_3 might be third tier and similarly for jobs running on node_1, node_2 might be third tier.
The reason I am asking these questions is that the statically partitioning memory nodes into tiers will inherently add platform specific assumptions in the user API.

Absolutely agree.

Quoted 10 lines

Assumptions like:
1) Access within tier is always cheaper than across tier.
2) Access from tier_i to tier_i+1 has uniform cost.
The reason I am more inclined towards having numa centric control is that we don't have to make these assumptions. Though the usability will be more difficult. Greg (CCed) has some ideas on making it better and we will share our proposal after polishing it a bit more.

Sounds good, will look out for that.

Jonathan

Tim Chen

author

2021-04-15 00:42 UTC #18 lore raw

In reply to Shakeel Butt

Quoted 30 lines · Shakeel Butt

Quoted 16 lines
memory_t0.current Current usage of tier 0 memory by the cgroup.
memory_t0.min If tier 0 memory used by the cgroup falls below this low
boundary, the memory will not be subjected to demotion to lower tiers to free up memory at tier 0.
memory_t0.low Above this boundary, the tier 0 memory will be subjected
to demotion. The demotion pressure will be proportional to the overage.
memory_t0.high If tier 0 memory used by the cgroup exceeds this high
boundary, allocation of tier 0 memory by the cgroup will be throttled. The tier 0 memory used by this cgroup will also be subjected to heavy demotion.
Quoted 7 lines
memory_t0.max This will be a hard usage limit of tier 0 memory on the cgroup.
If needed, memory_t[12...].current/min/low/high for additional tiers can be added. This follows closely with the design of the general memory controller interface.
Will such an interface looks sane and acceptable with everyone?
I have a couple of questions. Let's suppose we have a two socket system. Node 0 (DRAM+CPUs), Node 1 (DRAM+CPUs), Node 2 (PMEM on socket 0 along with Node 0) and Node 3 (PMEM on socket 1 along with Node 1). Based on the tier definition of this patch series, tier_0: {node_0, node_1} and tier_1: {node_2, node_3}.

Quoted 5 lines

My questions are:
1) Can we assume that the cost of access within a tier will always be less than the cost of access from the tier? (node_0 <-> node_1 vs node_0 <-> node_2)

I do assume that higher tier memory offers better performance (or less access latency) than a lower tier memory. Otherwise, this defeats the whole purpose of promoting hot memory from lower tier to a higher tier, and demoting cold memory to a lower tier.

Tiers assumption is embedded once we define this promotion/demotion relationship between the numa nodes.

So if

node_m ----demotes----> node_n

         <---promotes----

then node_m is one tier higher tier than node_n. This promotion/demotion relationship between the nodes is the underpinning of Dave and Ying's demotion and promotion patch sets.

Quoted 2 lines

2) If yes to (1), is that assumption future proof? Will the future systems with DRAM over CXL support have the same characteristics?

I think if you configure a promotion/demotion relationship between DRAM over CXL and local-socket connected DRAM, you could divide them up into separate tiers. Or you don't care about the difference and you will configure them not to have a promotion/demotion relationship and they will be at the same tier. Balance within the same tier will be effected by the autonuma mechanism.

Quoted 4 lines

3) Will the cost of access from tier_0 to tier_1 be uniform? (node_0 <-> node_2 vs node_0 <-> node_3). For jobs running on node_0, node_3 might be third tier and similarly for jobs running on node_1, node_2 might be third tier.

Tier definition is an admin's choice, of where the admin think the hot memory should reside after looking at the memory performance. It falls out of how the admin construct the promotion/demotion relationship between the nodes and OS does not assume the tier relationship from memory performance directly.

Quoted 5 lines

The reason I am asking these questions is that the statically partitioning memory nodes into tiers will inherently add platform specific assumptions in the user API.

Quoted 9 lines

Assumptions like:
1) Access within tier is always cheaper than across tier.
2) Access from tier_i to tier_i+1 has uniform cost.
The reason I am more inclined towards having numa centric control is that we don't have to make these assumptions. Though the usability will be more difficult. Greg (CCed) has some ideas on making it better and we will share our proposal after polishing it a bit more.

I am still trying to understand how a numa centric control actually work. Putting limits on every numa node for each cgroup seems to make the system configuration quite complicated. Looking forward to your proposal so I can better understand that perspective.

Tim

Huang, Ying

2021-04-13 02:15 UTC #19 lore raw

In reply to Tim Chen

Quoted 40 lines · Tim Chen

Quoted 9 lines · Michal Hocko
Quoted 8 lines
The top tier memory used is reported in
memory.toptier_usage_in_bytes
The amount of top tier memory usable by each cgroup without triggering page reclaim is controlled by the
memory.toptier_soft_limit_in_bytes
Michal,
Thanks for your comments. I will like to take a step back and look at the eventual goal we envision: a mechanism to partition the tiered memory between the cgroups.
A typical use case may be a system with two set of tasks. One set of task is very latency sensitive and we desire instantaneous response from them. Another set of tasks will be running batch jobs
were latency and performance is not critical. In this case,
we want to carve out enough top tier memory such that the working set of the latency sensitive tasks can fit entirely in the top tier memory. The rest of the top tier memory can be assigned to the background tasks.
To achieve such cgroup based tiered memory management, we probably want something like the following.
For generalization let's say that there are N tiers of memory t_0, t_1 ... t_N-1, where tier t_0 sits at the top and demotes to the lower tier.
We envision for this top tier memory t0 the following knobs and counters in the cgroup memory controller
memory_t0.current Current usage of tier 0 memory by the cgroup.
memory_t0.min If tier 0 memory used by the cgroup falls below this low
boundary, the memory will not be subjected to demotion to lower tiers to free up memory at tier 0.

Quoted 8 lines

memory_t0.low Above this boundary, the tier 0 memory will be subjected
to demotion. The demotion pressure will be proportional to the overage.
memory_t0.high If tier 0 memory used by the cgroup exceeds this high
boundary, allocation of tier 0 memory by the cgroup will be throttled. The tier 0 memory used by this cgroup will also be subjected to heavy demotion.

I think we don't really need throttle here, because we can fallback to allocate memory from the t1. That will not cause something like IO device bandwidth saturation.

Best Regards,
Huang, Ying

Quoted 73 lines

memory_t0.max This will be a hard usage limit of tier 0 memory on the cgroup.
If needed, memory_t[12...].current/min/low/high for additional tiers can be added. This follows closely with the design of the general memory controller interface.
Will such an interface looks sane and acceptable with everyone?
The patch set I posted is meant to be a straw man cgroup v1 implementation and I readily admits that it falls short of the eventual functionality we want to achieve. It is meant to solicit feedback from everyone on how the tiered memory management should work.
Quoted 1 line
Are you trying to say that soft limit acts as some sort of guarantee?
No, the soft limit does not offers guarantee. It will only serves to keep the usage of the top tier memory in the vicinity of the soft limits.
Quoted 2 lines
Does that mean that if the memcg is under memory pressure top tiear memory is opted out from any reclaim if the usage is not in excess?
In the prototype implementation, regular memory reclaim is still in effect if we are under heavy memory pressure.
Quoted 4 lines
From you previous email it sounds more like the limit is evaluated on the global memory pressure to balance specific memcgs which are in excess when trying to reclaim/demote a toptier numa node.
On a top tier node, if the free memory on the node falls below a percentage, then we will start to reclaim/demote from the node.
Quoted 6 lines
Soft limit reclaim has several problems. Those are historical and therefore the behavior cannot be changed. E.g. go after the biggest excessed memcg (with priority 0 - aka potential full LRU scan) and then continue with a normal reclaim. This can be really disruptive to the top user.
Thanks for pointing out these problems with soft limit explicitly.
Quoted 17 lines
So you can likely define a more sane semantic. E.g. push back memcgs proporitional to their excess but then we have two different soft limits behavior which is bad as well. I am not really sure there is a sensible way out by (ab)using soft limit here.
Also I am not really sure how this is going to be used in practice. There is no soft limit by default. So opting in would effectivelly discriminate those memcgs. There has been a similar problem with the soft limit we have in general. Is this really what you are looing for? What would be a typical usecase?
Quoted 4 lines
Want to make sure I understand what you mean by NUMA aware limits. Yes, in the patch set, it does treat the NUMA nodes differently. We are putting constraint on the "top tier" RAM nodes vs the lower tier PMEM nodes. Is this what you mean?
Quoted 7 lines
What I am trying to say (and I have brought that up when demotion has been discussed at LSFMM) is that the implementation shouldn't be PMEM aware. The specific technology shouldn't be imprinted into the interface. Fundamentally you are trying to balance memory among NUMA nodes as we do not have other abstraction to use. So rather than talking about top, secondary, nth tier we have different NUMA nodes with different characteristics and you want to express your "priorities" for them.
With node priorities, how would the system reserve enough high performance memory for those performance critical task cgroup?
By priority, do you mean the order of allocation of nodes for a cgroup? Or you mean that all the similar performing memory node will be grouped in the same priority?
Tim

Michal Hocko

reviewer

2021-04-13 08:33 UTC #20 lore raw

In reply to Tim Chen

Quoted 10 lines · Tim Chen

Quoted 7 lines · Michal Hocko
Quoted 7 lines
The top tier memory used is reported in
memory.toptier_usage_in_bytes
The amount of top tier memory usable by each cgroup without triggering page reclaim is controlled by the

Quoted 8 lines

Quoted 2 lines
Quoted 1 line
memory.toptier_soft_limit_in_bytes
Michal,
Thanks for your comments. I will like to take a step back and look at the eventual goal we envision: a mechanism to partition the tiered memory between the cgroups.

OK, this is goot mission statemet to start with. I would expect a follow up to say what kind of granularity of control you want to achieve here. Pressumably you want more than all or nothing because that is where cpusets can be used for.

Quoted 7 lines

A typical use case may be a system with two set of tasks. One set of task is very latency sensitive and we desire instantaneous response from them. Another set of tasks will be running batch jobs
were latency and performance is not critical. In this case,
we want to carve out enough top tier memory such that the working set of the latency sensitive tasks can fit entirely in the top tier memory. The rest of the top tier memory can be assigned to the background tasks.

While from a very high level this makes sense I would be interested in more details though. Your high letency sensitive applications very likely want to be bound to high performance node, right? Can they tolerate memory reclaim? Can they consume more memory than the node size? What do you expect to happen then?

Quoted 5 lines

To achieve such cgroup based tiered memory management, we probably want something like the following.
For generalization let's say that there are N tiers of memory t_0, t_1 ... t_N-1, where tier t_0 sits at the top and demotes to the lower tier.

How is each tear defined? Is this an admin define set of NUMA nodes or is this platform specific?

[...]

Quoted 1 line

Will such an interface looks sane and acceptable with everyone?

Let's talk more about usecases first before we even start talking about the interface or which controller is the best fit for implementing it.

Quoted 4 lines

The patch set I posted is meant to be a straw man cgroup v1 implementation and I readily admits that it falls short of the eventual functionality we want to achieve. It is meant to solicit feedback from everyone on how the tiered memory management should work.

OK, fair enough. Let me then just state that I strongly believe that Soft limit based approach is a dead end and it would be better to focus on the actual usecases and try to understand what you want to achieve first.

[...]

Quoted 8 lines

Quoted 7 lines
What I am trying to say (and I have brought that up when demotion has been discussed at LSFMM) is that the implementation shouldn't be PMEM aware. The specific technology shouldn't be imprinted into the interface. Fundamentally you are trying to balance memory among NUMA nodes as we do not have other abstraction to use. So rather than talking about top, secondary, nth tier we have different NUMA nodes with different characteristics and you want to express your "priorities" for them.

Quoted 6 lines

With node priorities, how would the system reserve enough high performance memory for those performance critical task cgroup?
By priority, do you mean the order of allocation of nodes for a cgroup? Or you mean that all the similar performing memory node will be grouped in the same priority?

I have to say I do not yet have a clear idea on what those priorities would look like. I just wanted to outline that usecases you are interested about likely want to implement some form of (application transparent) control for memory distribution over several nodes. There is a long way to land on something more specific I am afraid.

Signature · 2 lines

Michal Hocko
SUSE Labs

Shakeel Butt

2021-04-12 14:03 UTC #21 lore raw

In reply to Michal Hocko

Quoted 1 line · Michal Hocko

[...]

Quoted 9 lines

What I am trying to say (and I have brought that up when demotion has been discussed at LSFMM) is that the implementation shouldn't be PMEM aware. The specific technology shouldn't be imprinted into the interface. Fundamentally you are trying to balance memory among NUMA nodes as we do not have other abstraction to use. So rather than talking about top, secondary, nth tier we have different NUMA nodes with different characteristics and you want to express your "priorities" for them.

I am also inclined towards NUMA based approach. It makes the solution more general and even existing systems with multiple numa nodes and DRAM can take advantage of this approach (if it makes sense).

Shakeel Butt

2021-04-08 17:18 UTC #22 lore raw

In reply to Tim Chen

Hi Tim,

Quoted 14 lines · Tim Chen

Traditionally, all memory is DRAM. Some DRAM might be closer/faster than others NUMA wise, but a byte of media has about the same cost whether it is close or far. But, with new memory tiers such as Persistent Memory (PMEM). there is a choice between fast/expensive DRAM and slow/cheap PMEM.
The fast/expensive memory lives in the top tier of the memory hierachy.
Previously, the patchset
[PATCH 00/10] [v7] Migrate Pages in lieu of discard
https://lore.kernel.org/linux-mm/20210401183216.443C4443-LXbPSdftPKxrdx17CPfAsdBPR1lH4CV8@public.gmane.org/
provides a mechanism to demote cold pages from DRAM node into PMEM.

Quoted 8 lines

And the patchset
[PATCH 0/6] [RFC v6] NUMA balancing: optimize memory placement for memory tiering system
https://lore.kernel.org/linux-mm/20210311081821.138467-1-ying.huang-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org/
provides a mechanism to promote hot pages in PMEM to the DRAM node leveraging autonuma.
The two patchsets together keep the hot pages in DRAM and colder pages in PMEM.

Thanks for working on this as this is becoming more and more important particularly in the data centers where memory is a big portion of the cost.

I see you have responded to Michal and I will add my more specific response there. Here I wanted to give my high level concern regarding using v1's soft limit like semantics for top tier memory.

This patch series aims to distribute/partition top tier memory between jobs of different priorities. We want high priority jobs to have preferential access to the top tier memory and we don't want low priority jobs to hog the top tier memory.

Using v1's soft limit like behavior can potentially cause high priority jobs to stall to make enough space on top tier memory on their allocation path and I think this patchset is aiming to reduce that impact by making kswapd do that work. However I think the more concerning issue is the low priority job hogging the top tier memory.

The possible ways the low priority job can hog the top tier memory are by allocating non-movable memory or by mlocking the memory. (Oh there is also pinning the memory but I don't know if there is a user api to pin memory?) For the mlocked memory, you need to either modify the reclaim code or use a different mechanism for demoting cold memory.

Basically I am saying we should put the upfront control (limit) on the usage of top tier memory by the jobs.

Yang Shi

2021-04-08 18:00 UTC #23 lore raw

Quoted 46 lines · Shakeel Butt

Hi Tim,
Quoted 14 lines · Tim Chen
Traditionally, all memory is DRAM. Some DRAM might be closer/faster than others NUMA wise, but a byte of media has about the same cost whether it is close or far. But, with new memory tiers such as Persistent Memory (PMEM). there is a choice between fast/expensive DRAM and slow/cheap PMEM.
The fast/expensive memory lives in the top tier of the memory hierachy.
Previously, the patchset
[PATCH 00/10] [v7] Migrate Pages in lieu of discard
https://lore.kernel.org/linux-mm/[email protected]/
provides a mechanism to demote cold pages from DRAM node into PMEM.
Quoted 8 lines
And the patchset
[PATCH 0/6] [RFC v6] NUMA balancing: optimize memory placement for memory tiering system
https://lore.kernel.org/linux-mm/[email protected]/
provides a mechanism to promote hot pages in PMEM to the DRAM node leveraging autonuma.
The two patchsets together keep the hot pages in DRAM and colder pages in PMEM.
Thanks for working on this as this is becoming more and more important particularly in the data centers where memory is a big portion of the cost.
I see you have responded to Michal and I will add my more specific response there. Here I wanted to give my high level concern regarding using v1's soft limit like semantics for top tier memory.
This patch series aims to distribute/partition top tier memory between jobs of different priorities. We want high priority jobs to have preferential access to the top tier memory and we don't want low priority jobs to hog the top tier memory.
Using v1's soft limit like behavior can potentially cause high priority jobs to stall to make enough space on top tier memory on their allocation path and I think this patchset is aiming to reduce that impact by making kswapd do that work. However I think the more concerning issue is the low priority job hogging the top tier memory.

Quoted 5 lines

The possible ways the low priority job can hog the top tier memory are by allocating non-movable memory or by mlocking the memory. (Oh there is also pinning the memory but I don't know if there is a user api to pin memory?) For the mlocked memory, you need to either modify the reclaim code or use a different mechanism for demoting cold memory.

Do you mean long term pin? RDMA should be able to simply pin the memory for weeks. A lot of transient pins come from Direct I/O. They should be less concerned.

The low priority jobs should be able to be restricted by cpuset, for example, just keep them on second tier memory nodes. Then all the above problems are gone.

Quoted 3 lines

Basically I am saying we should put the upfront control (limit) on the usage of top tier memory by the jobs.

This sounds similar to what I talked about in LSFMM 2019
(https://lwn.net/Articles/787418/). We used to have some potential usecase which divides DRAM:PMEM ratio for different jobs or memcgs when I was with Alibaba.

In the first place I thought about per NUMA node limit, but it was very hard to configure it correctly for users unless you know exactly about your memory usage and hot/cold memory distribution.

I'm wondering, just off the top of my head, if we could extend the semantic of low and min limit. For example, just redefine low and min to "the limit on top tier memory". Then we could have low priority jobs have 0 low/min limit.

Quoted 1 line

Shakeel Butt

2021-04-08 20:29 UTC #24 lore raw

Quoted 54 lines · Yang Shi

Quoted 46 lines · Shakeel Butt
Hi Tim,
Quoted 14 lines · Tim Chen
Traditionally, all memory is DRAM. Some DRAM might be closer/faster than others NUMA wise, but a byte of media has about the same cost whether it is close or far. But, with new memory tiers such as Persistent Memory (PMEM). there is a choice between fast/expensive DRAM and slow/cheap PMEM.
The fast/expensive memory lives in the top tier of the memory hierachy.
Previously, the patchset
[PATCH 00/10] [v7] Migrate Pages in lieu of discard
https://lore.kernel.org/linux-mm/20210401183216.443C4443-LXbPSdftPKxrdx17CPfAsdBPR1lH4CV8@public.gmane.org/
provides a mechanism to demote cold pages from DRAM node into PMEM.
Quoted 8 lines
And the patchset
[PATCH 0/6] [RFC v6] NUMA balancing: optimize memory placement for memory tiering system
https://lore.kernel.org/linux-mm/20210311081821.138467-1-ying.huang-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org/
provides a mechanism to promote hot pages in PMEM to the DRAM node leveraging autonuma.
The two patchsets together keep the hot pages in DRAM and colder pages in PMEM.
Thanks for working on this as this is becoming more and more important particularly in the data centers where memory is a big portion of the cost.
I see you have responded to Michal and I will add my more specific response there. Here I wanted to give my high level concern regarding using v1's soft limit like semantics for top tier memory.
This patch series aims to distribute/partition top tier memory between jobs of different priorities. We want high priority jobs to have preferential access to the top tier memory and we don't want low priority jobs to hog the top tier memory.
Using v1's soft limit like behavior can potentially cause high priority jobs to stall to make enough space on top tier memory on their allocation path and I think this patchset is aiming to reduce that impact by making kswapd do that work. However I think the more concerning issue is the low priority job hogging the top tier memory.
Quoted 5 lines
The possible ways the low priority job can hog the top tier memory are by allocating non-movable memory or by mlocking the memory. (Oh there is also pinning the memory but I don't know if there is a user api to pin memory?) For the mlocked memory, you need to either modify the reclaim code or use a different mechanism for demoting cold memory.

Quoted 8 lines

Do you mean long term pin? RDMA should be able to simply pin the memory for weeks. A lot of transient pins come from Direct I/O. They should be less concerned.
The low priority jobs should be able to be restricted by cpuset, for example, just keep them on second tier memory nodes. Then all the above problems are gone.

Yes that's an extreme way to overcome the issue but we can do less extreme by just (hard) limiting the top tier usage of low priority jobs.

Quoted 9 lines

Quoted 3 lines
Basically I am saying we should put the upfront control (limit) on the usage of top tier memory by the jobs.
This sounds similar to what I talked about in LSFMM 2019
(https://lwn.net/Articles/787418/). We used to have some potential usecase which divides DRAM:PMEM ratio for different jobs or memcgs when I was with Alibaba.

Quoted 9 lines

In the first place I thought about per NUMA node limit, but it was very hard to configure it correctly for users unless you know exactly about your memory usage and hot/cold memory distribution.
I'm wondering, just off the top of my head, if we could extend the semantic of low and min limit. For example, just redefine low and min to "the limit on top tier memory". Then we could have low priority jobs have 0 low/min limit.

The low and min limits have semantics similar to the v1's soft limit for this situation i.e. letting the low priority job occupy top tier memory and depending on reclaim to take back the excess top tier memory use of such jobs.

I have some thoughts on NUMA node limits which I will share in the other thread.

Yang Shi

2021-04-08 20:50 UTC #25 lore raw

Quoted 88 lines · Shakeel Butt

Quoted 54 lines · Yang Shi
Quoted 46 lines · Shakeel Butt
Hi Tim,
Quoted 14 lines · Tim Chen
Traditionally, all memory is DRAM. Some DRAM might be closer/faster than others NUMA wise, but a byte of media has about the same cost whether it is close or far. But, with new memory tiers such as Persistent Memory (PMEM). there is a choice between fast/expensive DRAM and slow/cheap PMEM.
The fast/expensive memory lives in the top tier of the memory hierachy.
Previously, the patchset
[PATCH 00/10] [v7] Migrate Pages in lieu of discard
https://lore.kernel.org/linux-mm/20210401183216.443C4443-LXbPSdftPKxrdx17CPfAsdBPR1lH4CV8@public.gmane.org/
provides a mechanism to demote cold pages from DRAM node into PMEM.
Quoted 8 lines
And the patchset
[PATCH 0/6] [RFC v6] NUMA balancing: optimize memory placement for memory tiering system
https://lore.kernel.org/linux-mm/20210311081821.138467-1-ying.huang-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org/
provides a mechanism to promote hot pages in PMEM to the DRAM node leveraging autonuma.
The two patchsets together keep the hot pages in DRAM and colder pages in PMEM.
Thanks for working on this as this is becoming more and more important particularly in the data centers where memory is a big portion of the cost.
I see you have responded to Michal and I will add my more specific response there. Here I wanted to give my high level concern regarding using v1's soft limit like semantics for top tier memory.
This patch series aims to distribute/partition top tier memory between jobs of different priorities. We want high priority jobs to have preferential access to the top tier memory and we don't want low priority jobs to hog the top tier memory.
Using v1's soft limit like behavior can potentially cause high priority jobs to stall to make enough space on top tier memory on their allocation path and I think this patchset is aiming to reduce that impact by making kswapd do that work. However I think the more concerning issue is the low priority job hogging the top tier memory.
Quoted 5 lines
The possible ways the low priority job can hog the top tier memory are by allocating non-movable memory or by mlocking the memory. (Oh there is also pinning the memory but I don't know if there is a user api to pin memory?) For the mlocked memory, you need to either modify the reclaim code or use a different mechanism for demoting cold memory.
Quoted 8 lines
Do you mean long term pin? RDMA should be able to simply pin the memory for weeks. A lot of transient pins come from Direct I/O. They should be less concerned.
The low priority jobs should be able to be restricted by cpuset, for example, just keep them on second tier memory nodes. Then all the above problems are gone.
Yes that's an extreme way to overcome the issue but we can do less extreme by just (hard) limiting the top tier usage of low priority jobs.
Quoted 9 lines
Quoted 3 lines
Basically I am saying we should put the upfront control (limit) on the usage of top tier memory by the jobs.
This sounds similar to what I talked about in LSFMM 2019
(https://lwn.net/Articles/787418/). We used to have some potential usecase which divides DRAM:PMEM ratio for different jobs or memcgs when I was with Alibaba.
Quoted 9 lines
In the first place I thought about per NUMA node limit, but it was very hard to configure it correctly for users unless you know exactly about your memory usage and hot/cold memory distribution.
I'm wondering, just off the top of my head, if we could extend the semantic of low and min limit. For example, just redefine low and min to "the limit on top tier memory". Then we could have low priority jobs have 0 low/min limit.

Quoted 4 lines

The low and min limits have semantics similar to the v1's soft limit for this situation i.e. letting the low priority job occupy top tier memory and depending on reclaim to take back the excess top tier memory use of such jobs.

I don't get why low priority jobs can *not* use top tier memory? I can think of it may incur latency overhead for high priority jobs. If it is not allowed, it could be restricted by cpuset without introducing in any new interfaces.

I'm supposed the memory utilization could be maximized by allowing all jobs allocate memory from all applicable nodes, then let reclaimer (or something new if needed) do the job to migrate the memory to proper nodes by time. We could achieve some kind of balance between memory utilization and resource isolation.

Quoted 2 lines

I have some thoughts on NUMA node limits which I will share in the other thread.

Look forward to reading it.

Shakeel Butt

2021-04-12 14:03 UTC #26 lore raw

Quoted 1 line · Yang Shi

[...]

Quoted 7 lines

Quoted 5 lines
The low and min limits have semantics similar to the v1's soft limit for this situation i.e. letting the low priority job occupy top tier memory and depending on reclaim to take back the excess top tier memory use of such jobs.
I don't get why low priority jobs can *not* use top tier memory?

I am saying low priority jobs can use top tier memory. The only difference is to limit them upfront (using limits) or reclaim from them later (using min/low/soft-limit).

Quoted 8 lines

I can
think of it may incur latency overhead for high priority jobs. If it is not allowed, it could be restricted by cpuset without introducing in any new interfaces.
I'm supposed the memory utilization could be maximized by allowing all jobs allocate memory from all applicable nodes, then let reclaimer (or something new if needed)

Most probably something new as we do want to consider unevictable memory as well.

Quoted 4 lines

do the job to migrate the memory to proper
nodes by time. We could achieve some kind of balance between memory utilization and resource isolation.

Tradeoff between utilization and isolation should be decided by the user/admin.

Michal Hocko

reviewer

2021-04-09 07:24 UTC #27 lore raw

In reply to Shakeel Butt

Quoted 1 line · Shakeel Butt

On Thu, Apr 8, 2021 at 11:01 AM Yang Shi <[email protected]> wrote:

[...]

Quoted 3 lines

Quoted 3 lines
The low priority jobs should be able to be restricted by cpuset, for example, just keep them on second tier memory nodes. Then all the above problems are gone.

Yes, if the aim is to isolate some users from certain numa node then cpuset is a good fit but as Shakeel says this is very likely not what this work is aiming for.

Quoted 3 lines

Yes that's an extreme way to overcome the issue but we can do less extreme by just (hard) limiting the top tier usage of low priority jobs.

Per numa node high/hard limit would help with a more fine grained control. The configuration would be tricky though. All low priority memcgs would have to be carefully configured to leave enough for your important processes. That includes also memory which is not accounted to any memcg.
The behavior of those limits would be quite tricky for OOM situations as well due to a lack of NUMA aware oom killer.

Signature · 2 lines

Michal Hocko
SUSE Labs

Tim Chen

author

2021-04-15 22:31 UTC #28 lore raw

Quoted 15 lines · Michal Hocko

Quoted 1 line · Shakeel Butt
On Thu, Apr 8, 2021 at 11:01 AM Yang Shi <[email protected]> wrote:
[...]
Quoted 3 lines
Quoted 3 lines
The low priority jobs should be able to be restricted by cpuset, for example, just keep them on second tier memory nodes. Then all the above problems are gone.
Yes, if the aim is to isolate some users from certain numa node then cpuset is a good fit but as Shakeel says this is very likely not what this work is aiming for.
Quoted 3 lines
Yes that's an extreme way to overcome the issue but we can do less extreme by just (hard) limiting the top tier usage of low priority jobs.

Quoted 8 lines

Per numa node high/hard limit would help with a more fine grained control. The configuration would be tricky though. All low priority memcgs would have to be carefully configured to leave enough for your important processes. That includes also memory which is not accounted to any memcg.
The behavior of those limits would be quite tricky for OOM situations as well due to a lack of NUMA aware oom killer.

Another downside of putting limits on individual NUMA
node is it would limit flexibility. For example two memory nodes are similar enough in performance, that you really only care about a cgroup not using more than a threshold of the combined capacity from the two memory nodes. But when you put a hard limit on NUMA node, then you are tied down to a fix allocation partition for each node. Perhaps there are some kernel resources that are pre-allocated primarily from one node. A cgroup may bump into the limit on the node and failed the allocation, even when it has a lot of slack in the other node. This makes getting the configuration right trickier.

There are some differences in opinion currently
on whether grouping memory nodes into tiers, and putting limit on using them by cgroup is a desirable. Many people want the management constraint placed at individual NUMA node for each cgroup, instead of at the tier level. Will appreciate feedbacks from folks who have insights on how such NUMA based control interface will work, so we at least agree here in order to move forward.

Tim

Michal Hocko

reviewer

2021-04-16 06:38 UTC #29 lore raw

Quoted 27 lines · Tim Chen

Quoted 15 lines · Michal Hocko
Quoted 1 line · Shakeel Butt
On Thu, Apr 8, 2021 at 11:01 AM Yang Shi <[email protected]> wrote:
[...]
Quoted 3 lines
Quoted 3 lines
The low priority jobs should be able to be restricted by cpuset, for example, just keep them on second tier memory nodes. Then all the above problems are gone.
Yes, if the aim is to isolate some users from certain numa node then cpuset is a good fit but as Shakeel says this is very likely not what this work is aiming for.
Quoted 3 lines
Yes that's an extreme way to overcome the issue but we can do less extreme by just (hard) limiting the top tier usage of low priority jobs.
Quoted 8 lines
Per numa node high/hard limit would help with a more fine grained control. The configuration would be tricky though. All low priority memcgs would have to be carefully configured to leave enough for your important processes. That includes also memory which is not accounted to any memcg.
The behavior of those limits would be quite tricky for OOM situations as well due to a lack of NUMA aware oom killer.

Quoted 2 lines

Another downside of putting limits on individual NUMA node is it would limit flexibility.

Let me just clarify one thing. I haven't been proposing per NUMA limits. As I've said above it would be quite tricky to use and the behavior would be tricky as well. All I am saying is that we do not want to have an interface that is tightly bound to any specific HW setup (fast RAM as a top tier and PMEM as a fallback) that you have proposed here. We want to have a generic NUMA based abstraction. How that abstraction is going to look like is an open question and it really depends on usecase that we expect to see.

Signature · 2 lines

Michal Hocko
SUSE Labs

Tim Chen

author

2021-04-14 23:22 UTC #30 lore raw

In reply to Shakeel Butt

Quoted 10 lines · Shakeel Butt

On Thu, Apr 8, 2021 at 11:01 AM Yang Shi <[email protected]> wrote:
The low and min limits have semantics similar to the v1's soft limit for this situation i.e. letting the low priority job occupy top tier memory and depending on reclaim to take back the excess top tier memory use of such jobs.
I have some thoughts on NUMA node limits which I will share in the other thread.

Shakeel,

Look forward to the proposal on NUMA node limits. Which thread are you going to post it? Want to make sure I didn't miss it.

Tim

Huang, Ying

2021-04-09 02:58 UTC #31 lore raw

In reply to Yang Shi

Quoted 53 lines · Yang Shi

Quoted 46 lines · Shakeel Butt
Hi Tim,
Quoted 14 lines · Tim Chen
Traditionally, all memory is DRAM. Some DRAM might be closer/faster than others NUMA wise, but a byte of media has about the same cost whether it is close or far. But, with new memory tiers such as Persistent Memory (PMEM). there is a choice between fast/expensive DRAM and slow/cheap PMEM.
The fast/expensive memory lives in the top tier of the memory hierachy.
Previously, the patchset
[PATCH 00/10] [v7] Migrate Pages in lieu of discard
https://lore.kernel.org/linux-mm/20210401183216.443C4443-LXbPSdftPKxrdx17CPfAsdBPR1lH4CV8@public.gmane.org/
provides a mechanism to demote cold pages from DRAM node into PMEM.
Quoted 8 lines
And the patchset
[PATCH 0/6] [RFC v6] NUMA balancing: optimize memory placement for memory tiering system
https://lore.kernel.org/linux-mm/20210311081821.138467-1-ying.huang-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org/
provides a mechanism to promote hot pages in PMEM to the DRAM node leveraging autonuma.
The two patchsets together keep the hot pages in DRAM and colder pages in PMEM.
Thanks for working on this as this is becoming more and more important particularly in the data centers where memory is a big portion of the cost.
I see you have responded to Michal and I will add my more specific response there. Here I wanted to give my high level concern regarding using v1's soft limit like semantics for top tier memory.
This patch series aims to distribute/partition top tier memory between jobs of different priorities. We want high priority jobs to have preferential access to the top tier memory and we don't want low priority jobs to hog the top tier memory.
Using v1's soft limit like behavior can potentially cause high priority jobs to stall to make enough space on top tier memory on their allocation path and I think this patchset is aiming to reduce that impact by making kswapd do that work. However I think the more concerning issue is the low priority job hogging the top tier memory.
Quoted 5 lines
The possible ways the low priority job can hog the top tier memory are by allocating non-movable memory or by mlocking the memory. (Oh there is also pinning the memory but I don't know if there is a user api to pin memory?) For the mlocked memory, you need to either modify the reclaim code or use a different mechanism for demoting cold memory.

Quoted 7 lines

Do you mean long term pin? RDMA should be able to simply pin the memory for weeks. A lot of transient pins come from Direct I/O. They should be less concerned.
The low priority jobs should be able to be restricted by cpuset, for example, just keep them on second tier memory nodes. Then all the above problems are gone.

To optimize the page placement of a process between DRAM and PMEM, we want to place the hot pages in DRAM and the cold pages in PMEM. But the memory accessing pattern changes overtime, so we need to migrate pages between DRAM and PMEM to adapt to the changing.

To avoid the hot pages be pinned in PMEM always, one way is to online the PMEM as movable zones. If so, and if the low priority jobs are restricted by cpuset to allocate from PMEM only, we may fail to run quite some workloads as being discussed in the following threads,

https://lore.kernel.org/linux-mm/1604470210-124827-1-git-send-email-feng.tang-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org/

Quoted 9 lines

Quoted 3 lines
Basically I am saying we should put the upfront control (limit) on the usage of top tier memory by the jobs.
This sounds similar to what I talked about in LSFMM 2019
(https://lwn.net/Articles/787418/). We used to have some potential usecase which divides DRAM:PMEM ratio for different jobs or memcgs when I was with Alibaba.

Quoted 8 lines

In the first place I thought about per NUMA node limit, but it was very hard to configure it correctly for users unless you know exactly about your memory usage and hot/cold memory distribution.
I'm wondering, just off the top of my head, if we could extend the semantic of low and min limit. For example, just redefine low and min to "the limit on top tier memory". Then we could have low priority jobs have 0 low/min limit.

Per my understanding, memory.low/min are for the memory protection instead of the memory limiting. memory.high is for the memory limiting.

Best Regards,
Huang, Ying

Yang Shi

2021-04-09 20:50 UTC #32 lore raw

Quoted 69 lines · Ying

Quoted 53 lines · Yang Shi
Quoted 46 lines · Shakeel Butt
Hi Tim,
Quoted 14 lines · Tim Chen
Traditionally, all memory is DRAM. Some DRAM might be closer/faster than others NUMA wise, but a byte of media has about the same cost whether it is close or far. But, with new memory tiers such as Persistent Memory (PMEM). there is a choice between fast/expensive DRAM and slow/cheap PMEM.
The fast/expensive memory lives in the top tier of the memory hierachy.
Previously, the patchset
[PATCH 00/10] [v7] Migrate Pages in lieu of discard
https://lore.kernel.org/linux-mm/20210401183216.443C4443-LXbPSdftPKxrdx17CPfAsdBPR1lH4CV8@public.gmane.org/
provides a mechanism to demote cold pages from DRAM node into PMEM.
Quoted 8 lines
And the patchset
[PATCH 0/6] [RFC v6] NUMA balancing: optimize memory placement for memory tiering system
https://lore.kernel.org/linux-mm/20210311081821.138467-1-ying.huang-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org/
provides a mechanism to promote hot pages in PMEM to the DRAM node leveraging autonuma.
The two patchsets together keep the hot pages in DRAM and colder pages in PMEM.
Thanks for working on this as this is becoming more and more important particularly in the data centers where memory is a big portion of the cost.
I see you have responded to Michal and I will add my more specific response there. Here I wanted to give my high level concern regarding using v1's soft limit like semantics for top tier memory.
This patch series aims to distribute/partition top tier memory between jobs of different priorities. We want high priority jobs to have preferential access to the top tier memory and we don't want low priority jobs to hog the top tier memory.
Using v1's soft limit like behavior can potentially cause high priority jobs to stall to make enough space on top tier memory on their allocation path and I think this patchset is aiming to reduce that impact by making kswapd do that work. However I think the more concerning issue is the low priority job hogging the top tier memory.
Quoted 5 lines
The possible ways the low priority job can hog the top tier memory are by allocating non-movable memory or by mlocking the memory. (Oh there is also pinning the memory but I don't know if there is a user api to pin memory?) For the mlocked memory, you need to either modify the reclaim code or use a different mechanism for demoting cold memory.
Quoted 7 lines
Do you mean long term pin? RDMA should be able to simply pin the memory for weeks. A lot of transient pins come from Direct I/O. They should be less concerned.
The low priority jobs should be able to be restricted by cpuset, for example, just keep them on second tier memory nodes. Then all the above problems are gone.
To optimize the page placement of a process between DRAM and PMEM, we want to place the hot pages in DRAM and the cold pages in PMEM. But the memory accessing pattern changes overtime, so we need to migrate pages between DRAM and PMEM to adapt to the changing.

Quoted 6 lines

To avoid the hot pages be pinned in PMEM always, one way is to online the PMEM as movable zones. If so, and if the low priority jobs are restricted by cpuset to allocate from PMEM only, we may fail to run quite some workloads as being discussed in the following threads,
https://lore.kernel.org/linux-mm/1604470210-124827-1-git-send-email-feng.tang-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org/

Thanks for sharing the thread. It seems the configuration of movable zone + node bind is not supported very well or need evolve to support new use cases.

Quoted 14 lines

Quoted 14 lines
Quoted 4 lines
Basically I am saying we should put the upfront control (limit) on the usage of top tier memory by the jobs.
This sounds similar to what I talked about in LSFMM 2019
(https://lwn.net/Articles/787418/). We used to have some potential usecase which divides DRAM:PMEM ratio for different jobs or memcgs when I was with Alibaba.
In the first place I thought about per NUMA node limit, but it was very hard to configure it correctly for users unless you know exactly about your memory usage and hot/cold memory distribution.

Quoted 7 lines

Quoted 4 lines
I'm wondering, just off the top of my head, if we could extend the semantic of low and min limit. For example, just redefine low and min to "the limit on top tier memory". Then we could have low priority jobs have 0 low/min limit.
Per my understanding, memory.low/min are for the memory protection instead of the memory limiting. memory.high is for the memory limiting.

Yes, it is not limit. I just misused the term, I actually do mean protection but typed "limit". Sorry for the confusion.

Quoted 3 lines

Best Regards,
Huang, Ying

Tim Chen

author

2021-04-15 22:25 UTC #33 lore raw

In reply to Shakeel Butt

Quoted 7 lines · Shakeel Butt

Using v1's soft limit like behavior can potentially cause high priority jobs to stall to make enough space on top tier memory on their allocation path and I think this patchset is aiming to reduce that impact by making kswapd do that work. However I think the more concerning issue is the low priority job hogging the top tier memory.

Quoted 9 lines

The possible ways the low priority job can hog the top tier memory are by allocating non-movable memory or by mlocking the memory. (Oh there is also pinning the memory but I don't know if there is a user api to pin memory?) For the mlocked memory, you need to either modify the reclaim code or use a different mechanism for demoting cold memory.
Basically I am saying we should put the upfront control (limit) on the usage of top tier memory by the jobs.

Circling back to your comment here.

I agree that soft limit is deficient in this scenario that you have pointed out. Eventually I was shooting for a hard limit on a memory tier for a cgroup that's similar to the v2 memory controller interface (see mail in the other thread). That interface should satisfy the hard constraint you want to place on the low priority jobs.

Tim