Add mempressure cgroup

This is an attempt to implement David Rientjes' idea of mempressure cgroup.

The main characteristics are the same to what I've tried to add to vmevent API:

Internally, it uses Mel Gorman's idea of scanned/reclaimed ratio for pressure index calculation. But we don't expose the index to the userland. Instead, there are three levels of the pressure:

o low (just reclaiming, e.g. caches are draining);
o medium (allocation cost becomes high, e.g. swapping);
o oom (about to oom very soon).

The rationale behind exposing levels and not the raw pressure index described here: http://lkml.org/lkml/2012/11/16/675

The API uses standard cgroups eventfd notifications:

  $ gcc Documentation/cgroups/cgroup_event_listener.c -o \
	cgroup_event_listener
  $ cd /sys/fs/cgroup/
  $ mkdir mempressure
  $ mount -t cgroup cgroup ./mempressure -o mempressure
  $ cd mempressure
  $ cgroup_event_listener ./mempressure.level low
  ("low", "medium", "oom" are permitted values.)

Upon hitting the threshold, you should see "/sys/fs/cgroup/mempressure
low: crossed" messages.

To test that it actually works on per-cgroup basis, I did a small trick: I moved all kswapd into a separate cgroup, and hooked the listener onto another (non-root) cgroup. The listener no longer received global reclaim pressure, which is expected.

For a task it is possible to be in both cpusets, memcg and mempressure cgroups, so by rearranging the tasks it should be possible to watch a specific pressure.

Note that while this adds the cgroups support, the code is well separated and eventually we might add a lightweight, non-cgroups API, i.e. vmevent. But this is another story.

Signed-off-by Anton Vorontsov <[email protected]>

---

mm/mempressure.c new file +32 -0

+static const uint vmpressure_level_med = 60;
+static const uint vmpressure_level_oom = 99;
+static const uint vmpressure_level_oom_prio = 4;
+
+enum vmpressure_levels {
+	VMPRESSURE_LOW = 0,
+	VMPRESSURE_MEDIUM,
+	VMPRESSURE_OOM,
+	VMPRESSURE_NUM_LEVELS,
+};
+
+static const char const *vmpressure_str_levels[] = {
+	[VMPRESSURE_LOW] = "low",
+	[VMPRESSURE_MEDIUM] = "medium",
+	[VMPRESSURE_OOM] = "oom",
+};
+
+static enum vmpressure_levels vmpressure_level(uint pressure)
+{
+	if (pressure >= vmpressure_level_oom)
+		return VMPRESSURE_OOM;
+	else if (pressure >= vmpressure_level_med)
+		return VMPRESSURE_MEDIUM;
+	return VMPRESSURE_LOW;
+}
+
+static ulong vmpressure_calc_level(uint win, uint s, uint r)
+{
+	ulong p;
+
+	if (!s)
+		return 0;

mm/mempressure.c new file +6 -0

+
+	pr_debug("%s: %3lu  (s: %6u  r: %6u)\n", __func__, p, s, r);
+
+	return vmpressure_level(p);
+}
+

mm/vmscan.c +0 -0

 		sc->nr_scanned = 0;
 		aborted_reclaim = shrink_zones(zonelist, sc);

Quoted 31 lines · Anton Vorontsov

This is an attempt to implement David Rientjes' idea of mempressure cgroup.

The main characteristics are the same to what I've tried to add to vmevent API:

Internally, it uses Mel Gorman's idea of scanned/reclaimed ratio for pressure index calculation. But we don't expose the index to the userland. Instead, there are three levels of the pressure:

o low (just reclaiming, e.g. caches are draining);
o medium (allocation cost becomes high, e.g. swapping);
o oom (about to oom very soon).

The rationale behind exposing levels and not the raw pressure index described here: http://lkml.org/lkml/2012/11/16/675

The API uses standard cgroups eventfd notifications:

  $ gcc Documentation/cgroups/cgroup_event_listener.c -o \
	cgroup_event_listener
  $ cd /sys/fs/cgroup/
  $ mkdir mempressure
  $ mount -t cgroup cgroup ./mempressure -o mempressure
  $ cd mempressure
  $ cgroup_event_listener ./mempressure.level low
  ("low", "medium", "oom" are permitted values.)

Upon hitting the threshold, you should see "/sys/fs/cgroup/mempressure
low: crossed" messages.

Quoted 4 lines

To test that it actually works on per-cgroup basis, I did a small trick: I moved all kswapd into a separate cgroup, and hooked the listener onto another (non-root) cgroup. The listener no longer received global reclaim pressure, which is expected.

Is this really expected? So you want to be notified only about the direct reclaim?

I am not sure how much useful is that. If you co-mount with e.g. memcg then the picture is different because even global memory pressure is spread among groups so it would be just a matter of the proper accounting (which can be handled similar to lruvec when your code doesn't have to care about memcg internally).
Co-mounting with cpusets makes sense as well because then you get a pressure notification based on the placement policy.

So does it make much sense to mount mempressure on its own without co-mounting with other controllers?

Quoted 3 lines

For a task it is possible to be in both cpusets, memcg and mempressure cgroups, so by rearranging the tasks it should be possible to watch a specific pressure.

Could you be more specific what you mean by rearranging? Creating a same hierarchy? Co-mounting?

Quoted 3 lines

Note that while this adds the cgroups support, the code is well separated and eventually we might add a lightweight, non-cgroups API, i.e. vmevent. But this is another story.

I think it would be nice to follow freezer and split this into 2 files. Generic and cgroup spefici.

Quoted 2 lines

Signed-off-by Anton Vorontsov <[email protected]>

---

[...]

mm/mempressure.c line 32

Quoted 2 lines

mm/mempressure.c new file +2 -0

+/* These are defaults. Might make them configurable one day. */
+static const uint vmpressure_win = SWAP_CLUSTER_MAX * 16;

I realize this is just an RFC but could you be more specific what is the meaning of vmpressure_win?

Quoted 32 lines

mm/mempressure.c new file +32 -0

+static const uint vmpressure_level_med = 60;
+static const uint vmpressure_level_oom = 99;
+static const uint vmpressure_level_oom_prio = 4;
+
+enum vmpressure_levels {
+	VMPRESSURE_LOW = 0,
+	VMPRESSURE_MEDIUM,
+	VMPRESSURE_OOM,
+	VMPRESSURE_NUM_LEVELS,
+};
+
+static const char const *vmpressure_str_levels[] = {
+	[VMPRESSURE_LOW] = "low",
+	[VMPRESSURE_MEDIUM] = "medium",
+	[VMPRESSURE_OOM] = "oom",
+};
+
+static enum vmpressure_levels vmpressure_level(uint pressure)
+{
+	if (pressure >= vmpressure_level_oom)
+		return VMPRESSURE_OOM;
+	else if (pressure >= vmpressure_level_med)
+		return VMPRESSURE_MEDIUM;
+	return VMPRESSURE_LOW;
+}
+
+static ulong vmpressure_calc_level(uint win, uint s, uint r)
+{
+	ulong p;
+
+	if (!s)
+		return 0;

mm/mempressure.c line 74

Quoted 10 lines

mm/mempressure.c new file +10 -0

+
+	/*
+	 * We calculate the ratio (in percents) of how many pages were
+	 * scanned vs. reclaimed in a given time frame (window). Note that
+	 * time is in VM reclaimer's "ticks", i.e. number of pages
+	 * scanned. This makes it possible to set desired reaction time
+	 * and serves as a ratelimit.
+	 */
+	p = win - (r * win / s);
+	p = p * 100 / win;

Do we need the win at all?

	p = 100 - (100 * r / s);

Quoted 6 lines

mm/mempressure.c new file +6 -0

+
+	pr_debug("%s: %3lu  (s: %6u  r: %6u)\n", __func__, p, s, r);
+
+	return vmpressure_level(p);
+}
+

[...]

mm/mempressure.c line 190

Quoted 9 lines

mm/mempressure.c new file +9 -0

+static int mpc_pre_destroy(struct cgroup *cg)
+{
+	struct mpc_state *mpc = cg2mpc(cg);
+	int ret = 0;
+
+	mutex_lock(&mpc->lock);
+
+	if (mpc->eventfd)
+		ret = -EBUSY;

The current cgroup's core doesn't allow pre_destroy to fail anymore. The code is marked for 3.8

[...]

Quoted 8 lines

mm/vmscan.c +0 -0

index 48550c6..430d8a5 100644
@@ -1877,6 +1877,8 @@ restart:
 		shrink_active_list(SWAP_CLUSTER_MAX, lruvec,
 				   sc, LRU_ACTIVE_ANON);
 

mm/vmscan.c restart: · line 1881

Quoted 2 lines

mm/vmscan.c +2 -0

+	vmpressure(sc->nr_scanned - nr_scanned, nr_reclaimed);
+

I think this should already report to a proper group otherwise all the global reclaim would go to a group where kswapd sits rather than to the target group as I mentioned above (so it at least wouldn't work with a co-mounted cases).

Quoted 6 lines

mm/vmscan.c +0 -0

 	/* reclaim/compaction might need reclaim to continue */
 	if (should_continue_reclaim(lruvec, nr_reclaimed,
 				    sc->nr_scanned - nr_scanned, sc))

@@ -2099,6 +2101,7 @@ static unsigned long do_try_to_free_pages(struct zonelist *zonelist,
 		count_vm_event(ALLOCSTALL);
 

mm/vmscan.c static unsigned long do_try_to_free_pages(struct zonelist *zonelist, · line 2104

Quoted 2 lines

mm/vmscan.c +1 -0

 	do {
+		vmpressure_prio(sc->priority);

Shouldn't this go into shrink_lruvec or somewhere at that level to catch also kswapd low priorities? If you insist on the direct reclaim then you should hook into __zone_reclaim as well.

The main characteristics are the same to what I've tried to add to vmevent API:

Internally, it uses Mel Gorman's idea of scanned/reclaimed ratio for pressure index calculation. But we don't expose the index to the userland. Instead, there are three levels of the pressure:

o low (just reclaiming, e.g. caches are draining);
o medium (allocation cost becomes high, e.g. swapping);
o oom (about to oom very soon).

The rationale behind exposing levels and not the raw pressure index described here: http://lkml.org/lkml/2012/11/16/675

Compare this with the shrink_slab() shrinkers. With these, the VM can query and then control the clients. If something goes wrong or is out of balance, it's the VM's problem to solve.

So I'm thinking that a better design would be one which puts the kernel VM in control of userspace scanning and freeing. Presumably with a query-and-control interface similar to the slab shrinkers.

On Wed, Nov 28, 2012 at 03:14:32PM -0800, Andrew Morton wrote:
[...]

Quoted 7 lines

Compare this with the shrink_slab() shrinkers. With these, the VM can query and then control the clients. If something goes wrong or is out of balance, it's the VM's problem to solve.

So I'm thinking that a better design would be one which puts the kernel VM in control of userspace scanning and freeing. Presumably with a query-and-control interface similar to the slab shrinkers.

Thanks for the ideas, Andrew.

Query-and-control scheme looks very attractive, and that's actually resembles my "balance" level idea, when userland tells the kernel how much reclaimable memory it has. Except the your scheme works in the reverse direction, i.e. the kernel becomes in charge.

But there is one, rather major issue: we're crossing kernel-userspace boundary. And with the scheme we'll have to cross the boundary four times:
query / reply-available / control / reply-shrunk / (and repeat if necessary, every SHRINK_BATCH pages). Plus, it has to be done somewhat synchronously (all the four stages), and/or we have to make a "userspace shrinker" thread working in parallel with the normal shrinker, and here, I'm afraid, we'll see more strange interactions. :)

But there is a good news: for these kind of fine-grained control we have a better interface, where we don't have to communicate [very often] w/ the kernel. These are "volatile ranges", where userland itself marks chunks of data as "I might need it, but I won't cry if you recycle it; but when I access it next time, let me know if you actually recycled it". Yes, userland no longer able to decide which exact page it permits to recycle, but we don't have use-cases when we actually care that much. And if we do, we'd rather introduce volatile LRUs with different priorities, or something alike.

So, we really don't need the full-fledged userland shrinker, since we can just let the in-kernel shrinker do its job. If we work with the bytes/pages granularity it is just easier (and more efficient in terms of communication) to do the volatile ranges.

For the pressure notifications use-cases, we don't even know bytes/pages
information: "activity managers" are separate processes looking after overall system performance.

So, we're not trying to make userland too smart, quite the contrary: we realized that for this interface we don't want to mess with the bytes and pages, and that's why we cut this stuff down to only three levels. Before this, we were actually trying to count bytes, we did not like it and we ran away screaming.

OTOH, your scheme makes volatile ranges unneeded, since a thread might register a shrinker hook and free stuff by itself. But again, I believe this involves more communication with the kernel.

On Wed, Nov 28, 2012 at 03:14:32PM -0800, Andrew Morton wrote:
[...]

Quoted 7 lines

Compare this with the shrink_slab() shrinkers. With these, the VM can query and then control the clients. If something goes wrong or is out of balance, it's the VM's problem to solve.

So I'm thinking that a better design would be one which puts the kernel VM in control of userspace scanning and freeing. Presumably with a query-and-control interface similar to the slab shrinkers.

Thanks for the ideas, Andrew.

Query-and-control scheme looks very attractive, and that's actually resembles my "balance" level idea, when userland tells the kernel how much reclaimable memory it has. Except the your scheme works in the reverse direction, i.e. the kernel becomes in charge.

But there is one, rather major issue: we're crossing kernel-userspace boundary. And with the scheme we'll have to cross the boundary four times:
query / reply-available / control / reply-shrunk / (and repeat if necessary, every SHRINK_BATCH pages). Plus, it has to be done somewhat synchronously (all the four stages), and/or we have to make a "userspace shrinker" thread working in parallel with the normal shrinker, and here, I'm afraid, we'll see more strange interactions. :)

But there is a good news: for these kind of fine-grained control we have a better interface, where we don't have to communicate [very often] w/ the kernel. These are "volatile ranges", where userland itself marks chunks of data as "I might need it, but I won't cry if you recycle it; but when I access it next time, let me know if you actually recycled it". Yes, userland no longer able to decide which exact page it permits to recycle, but we don't have use-cases when we actually care that much. And if we do, we'd rather introduce volatile LRUs with different priorities, or something alike.

Quoted 7 lines

But there is one, rather major issue: we're crossing kernel-userspace boundary. And with the scheme we'll have to cross the boundary four times:
query / reply-available / control / reply-shrunk / (and repeat if necessary, every SHRINK_BATCH pages). Plus, it has to be done somewhat synchronously (all the four stages), and/or we have to make a "userspace shrinker" thread working in parallel with the normal shrinker, and here, I'm afraid, we'll see more strange interactions. :)

Wouldn't this be just like kswapd?

Quoted 9 lines

But there is a good news: for these kind of fine-grained control we have a better interface, where we don't have to communicate [very often] w/ the kernel. These are "volatile ranges", where userland itself marks chunks of data as "I might need it, but I won't cry if you recycle it; but when I access it next time, let me know if you actually recycled it". Yes, userland no longer able to decide which exact page it permits to recycle, but we don't have use-cases when we actually care that much. And if we do, we'd rather introduce volatile LRUs with different priorities, or something alike.

I'm new to this stuff so please take this with a grain of salt, but I'm not sure volatile ranges would be a good fit for our use case: we want to make (kvm) guests reduce their memory when the host is getting memory pressure.

Quoted 12 lines

Query-and-control scheme looks very attractive, and that's actually resembles my "balance" level idea, when userland tells the kernel how much reclaimable memory it has. Except the your scheme works in the reverse direction, i.e. the kernel becomes in charge.

But there is one, rather major issue: we're crossing kernel-userspace boundary. And with the scheme we'll have to cross the boundary four times:
query / reply-available / control / reply-shrunk / (and repeat if necessary, every SHRINK_BATCH pages). Plus, it has to be done somewhat synchronously (all the four stages), and/or we have to make a "userspace shrinker" thread working in parallel with the normal shrinker, and here, I'm afraid, we'll see more strange interactions. :)

Andrew's idea seems to give a lot more freedom to apps, IMHO.

mm/mempressure.c new file +13 -0

+
+	mutex_unlock(&mpc->lock);
+
+	return ret;
+}
 
+static int mpc_register_level_event(struct cgroup *cg, struct cftype *cft,
+				    struct eventfd_ctx *eventfd,
+				    const char *args)
+{
+	struct mpc_state *mpc = cg2mpc(cg);
+	int i;
+	int ret;
+

mm/mempressure.c new file +15 -0

+
+	ret = -EINVAL;
+	for (i = 0; i < VMPRESSURE_NUM_LEVELS; i++) {
+		if (strcmp(vmpressure_str_levels[i], args))
+			continue;
+		mpc->eventfd = eventfd;
+		mpc->thres = i;
+		ret = 0;
+		break;
+	}
+out_unlock:
+	mutex_unlock(&mpc->lock);
+
+	return ret;
+}

mm/mempressure.c line 190

Quoted 9 lines · Anton Vorontsov

mm/mempressure.c new file +9 -0

+static int mpc_pre_destroy(struct cgroup *cg)
+{
+	struct mpc_state *mpc = cg2mpc(cg);
+	int ret = 0;
+
+	mutex_lock(&mpc->lock);
+
+	if (mpc->eventfd)
+		ret = -EBUSY;

cgroup_rmdir() will unregister all events for you. No need to handle it here.

Quoted 13 lines

mm/mempressure.c new file +13 -0

+static int mpc_register_level_event(struct cgroup *cg, struct cftype *cft,
+				    struct eventfd_ctx *eventfd,
+				    const char *args)
+{
+	struct mpc_state *mpc = cg2mpc(cg);
+	int i;
+	int ret;
+
+	mutex_lock(&mpc->lock);
+
+	/*
+	 * It's easy to implement multiple thresholds, but so far we don't
+	 * need it.

mm/mempressure.c line 242

Quoted 5 lines

mm/mempressure.c new file +5 -0

+	 */
+	if (mpc->eventfd) {
+		ret = -EBUSY;
+		goto out_unlock;
+	}

One user which listen for one threashold per cgroup?
I think it's wrong. It's essensial for API to serve multiple users.