vmevent: A bit reworked pressure attribute + docs + man page

Hi all,

So this is the second RFC. The main change is that I decided to go with discrete levels of the pressure.

When I started writing the man page, I had to describe the 'reclaimer inefficiency index', and while doing this I realized that I'm describing how the kernel is doing the memory management, which we try to avoid in the vmevent. And applications don't really care about these details:
reclaimers, its inefficiency indexes, scanning window sizes, priority levels, etc. -- it's all "not interesting", and purely kernel's stuff. So I guess Mel Gorman was right, we need some sort of levels.

What applications (well, activity managers) are really interested in is this:

1. Do we we sacrifice resources for new memory allocations (e.g. files cache)?
2. Does the new memory allocations' cost becomes too high, and the system hurts because of this?
3. Are we about to OOM soon?

And here are the answers:

1. VMEVENT_PRESSURE_LOW
2. VMEVENT_PRESSURE_MED
3. VMEVENT_PRESSURE_OOM

There is no "high" pressure, since I really don't see any definition of it, but it's possible to introduce new levels without breaking ABI. The levels described in more details in the patches, and the stuff is still tunable, but now via sysctls, not the vmevent_fd() call itself (i.e. we don't need to rebuild applications to adjust window size or other mm "details").

What I couldn't fix in this RFC is making vmevent_{scanned,reclaimed} stuff per-CPU (there's a comment describing the problem with this). But I made it lockless and tried to make it very lightweight (plus I moved the vmevent_pressure() call to a more "cold" path).

Thanks,
Anton.
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

This patch introduces VMEVENT_ATTR_PRESSURE, the attribute reports Linux virtual memory management pressure. There are three discrete levels:

VMEVENT_PRESSURE_LOW: Notifies that the system is reclaiming memory for new allocations. Monitoring reclaiming activity might be useful for maintaining overall system's cache level.

VMEVENT_PRESSURE_MED: The system is experiencing medium memory pressure, there is some mild swapping activity. Upon this event applications may decide to free any resources that can be easily reconstructed or re-read from a disk.

VMEVENT_PRESSURE_OOM: The system is actively thrashing, it is about to out of memory (OOM) or even the in-kernel OOM killer is on its way to trigger. Applications should do whatever they can to help the system.

There are three sysctls to tune the behaviour of the levels:

vmevent_window
vmevent_level_med
vmevent_level_oom

Currently vmevent pressure levels are based on the reclaimer inefficiency index (range from 0 to 100). The index shows the relative time spent by the kernel uselessly scanning pages, or, in other words, the percentage of scans of pages (vmevent_window) that were not reclaimed. The higher the index, the more it should be evident that new allocations' cost becomes higher.

The files vmevent_level_med and vmevent_level_oom accept the index values (by default set to 60 and 99 respectively). A non-existent vmevent_level_low tunable is always set to 0

When index equals to 0, this means that the kernel is reclaiming, but every scanned page has been successfully reclaimed (so the pressure is low). 100 means that the kernel is trying to reclaim, but nothing can be reclaimed (close to OOM).

Window size is used as a rate-limit tunable for VMEVENT_PRESSURE_LOW notifications and for averaging for VMEVENT_PRESSURE_{MED,OOM} levels. So, using small window sizes can cause lot of false positives for _MED and _OOM levels, but too big window size may delay notifications.

By default the window size equals to 256 pages (1MB).

---
 Documentation/sysctl/vm.txt |  37 +++++++++++++++
 include/linux/vmevent.h     |  42 +++++++++++++++++
 kernel/sysctl.c             |  24 ++++++++++
 mm/vmevent.c                | 107 ++++++++++++++++++++++++++++++++++++++++++++
 mm/vmscan.c                 |  19 ++++++++
 5 files changed, 229 insertions(+)

index 078701f..ff0023b 100644
@@ -44,6 +44,9 @@ Currently, these files are in /proc/sys/vm:
 - nr_overcommit_hugepages
 - nr_trim_pages         (only if CONFIG_MMU=n)
 - numa_zonelist_order
+- vmevent_window
+- vmevent_level_med
+- vmevent_level_oom
 - oom_dump_tasks
 - oom_kill_allocating_task
 - overcommit_memory

@@ -487,6 +490,40 @@ this is causing problems for your system/application.
 
 ==============================================================
 
+vmevent_window
+vmevent_level_med
+vmevent_level_oom
+
+These sysctls are used to tune vmevent_fd(2) behaviour.
+
+Currently vmevent pressure levels are based on the reclaimer inefficiency
+index (range from 0 to 100). The files vmevent_level_med and
+vmevent_level_oom accept the index values (by default set to 60 and 99
+respectively). A non-existent vmevent_level_low tunable is always set to 0
+
+When the system is short on idle pages, the new memory is allocated by
+reclaiming least recently used resources: kernel scans pages to be
+reclaimed (e.g. from file caches, mmap(2) volatile ranges, etc.; and
+potentially swapping some pages out), and the index shows the relative
+time spent by the kernel uselessly scanning pages, or, in other words, the
+percentage of scans of pages (vmevent_window) that were not reclaimed. The
+higher the index, the more it should be evident that new allocations' cost
+becomes higher.
+
+When index equals to 0, this means that the kernel is reclaiming, but
+every scanned page has been successfully reclaimed (so the pressure is
+low). 100 means that the kernel is trying to reclaim, but nothing can be
+reclaimed (close to OOM).
+
+Window size is used as a rate-limit tunable for VMEVENT_PRESSURE_LOW
+notifications and for averaging for VMEVENT_PRESSURE_{MED,OOM} levels. So,
+using small window sizes can cause lot of false positives for _MED and
+_OOM levels, but too big window size may delay notifications.
+
+By default the window size equals to 256 pages (1MB).
+
+==============================================================
+
 oom_dump_tasks
 
 Enables a system-wide task dump (excluding kernel threads) to be

index 87174ef..e00d3fb 100644
@@ -50,6 +50,7 @@
 #include <linux/dnotify.h>
 #include <linux/syscalls.h>
 #include <linux/vmstat.h>
+#include <linux/vmevent.h>
 #include <linux/nfs_fs.h>
 #include <linux/acpi.h>
 #include <linux/reboot.h>

@@ -1317,6 +1318,29 @@ static struct ctl_table vm_table[] = {
 		.proc_handler	= numa_zonelist_order_handler,
 	},
 #endif
+#ifdef CONFIG_VMEVENT
+	{
+		.procname	= "vmevent_window",
+		.data		= &vmevent_window,
+		.maxlen		= sizeof(vmevent_window),
+		.mode		= 0644,
+		.proc_handler	= proc_dointvec,
+	},
+	{
+		.procname	= "vmevent_level_med",
+		.data		= &vmevent_level_med,
+		.maxlen		= sizeof(vmevent_level_med),
+		.mode		= 0644,
+		.proc_handler	= proc_dointvec,
+	},
+	{
+		.procname	= "vmevent_level_oom",
+		.data		= &vmevent_level_oom,
+		.maxlen		= sizeof(vmevent_level_oom),
+		.mode		= 0644,
+		.proc_handler	= proc_dointvec,
+	},
+#endif
 #if (defined(CONFIG_X86_32) && !defined(CONFIG_UML))|| \
    (defined(CONFIG_SUPERH) && defined(CONFIG_VSYSCALL))
 	{

index 99b434b..cd3bd19 100644
@@ -20,6 +20,7 @@
 #include <linux/init.h>
 #include <linux/highmem.h>
 #include <linux/vmstat.h>
+#include <linux/vmevent.h>
 #include <linux/file.h>
 #include <linux/writeback.h>
 #include <linux/blkdev.h>

@@ -1846,6 +1847,9 @@ restart:
 		shrink_active_list(SWAP_CLUSTER_MAX, lruvec,
 				   sc, LRU_ACTIVE_ANON);
 
+	vmevent_pressure(sc->target_mem_cgroup,
+			 sc->nr_scanned - nr_scanned, nr_reclaimed);
+
 	/* reclaim/compaction might need reclaim to continue */
 	if (should_continue_reclaim(lruvec, nr_reclaimed,
 				    sc->nr_scanned - nr_scanned, sc))

@@ -2068,6 +2072,21 @@ static unsigned long do_try_to_free_pages(struct zonelist *zonelist,
 		count_vm_event(ALLOCSTALL);
 
 	do {
+		/*
+		 * OK, we're cheating. The thing is, we have to average
+		 * s/r ratio by gathering a lot of scans (otherwise we
+		 * might get some local false-positives index of '100').
+		 *
+		 * But... when we're almost OOM we might be getting the
+		 * last reclaimable pages slowly, scanning all the queues,
+		 * and so we never catch the OOM case via averaging.
+		 * Although the priority will show it for sure. 3 is an
+		 * empirically taken priority: we never observe it under
+		 * any load, except for last few allocations before OOM.
+		 */
+		if (sc->priority <= 3)
+			vmevent_pressure(sc->target_mem_cgroup,
+					 vmevent_window, 0);
 		sc->nr_scanned = 0;
 		aborted_reclaim = shrink_zones(zonelist, sc);
 

1.7.12.3
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Nit:

s/VMEVENT_PRESSURE_MED/VMEVENT_PRESSUDE_MEDIUM/

Other than that, I'm OK with this. Mel and others, what are your thoughts on this?

Anton, have you tested this with real world scenarios? How does it stack up against Android's low memory killer, for example?

Pekka
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

Hello Pekka,

Thanks for taking a look into this!

Sure thing, will change.

Yup, I was mostly testing it on a desktop. I.e. in a KVM instance I was running a full fedora17 desktop w/ a lot of apps opened. The pressure index was pretty good in the sense that it was indeed reflecting the sluggishness in the system during swap activity. It's not ideal, i.e. the index might drop slightly for some time, but we usually interested in "above some value" threshold, so it should be fine.

The _LOW level is defined very strictly, and cannot be tuned anyhow. So it's very solid, and that's what we mostly use for Android.

The _OOM level is also defined quite strict, so from the API point of view, it's also solid, and should not be a problem.

Although the problem with _OOM is delivering the event in time (i.e. we must be quick in predicting it, before OOMK triggers). Today the patch has a shortcut for _OOM level: we send _OOM notification when reclaimer's priority is below empirically found value '3' (we might make it tunable via sysctl too, but that would expose another mm detail -- although sysctl sounds not that bad as exposing something in the C API; we have plenty of mm knobs in /proc/sys/vm/ already).

The real tunable is _MED level, and this should be tuned based on the desired system's behaviour that I described in more detail in this long
post: http://lkml.org/lkml/2012/10/7/29.

Based on my observations, I wouldn't say that we have plenty of room to tune the value, though. Usual swapping activity causes index to rise to say to 30%, and when the system can't keep up, it raises to 50..90 (but we still have plenty of swap space, so the system is far away from OOM, although it is thrashing. Ideally I'd prefer to not have any sysctl, but I believe _MED level is really based on user's definition of "medium".

The LMK driver is effectively using what we call _LOW pressure notifications here, so by definition it is enough to build a full replacement for the in-kernel LMK using just the _LOW level. But in the future, we might want to use _MED as well, e.g. kill unneeded services based not on the cache level, but based on the pressure.

Thanks,
Anton.
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

The one of the concern when I see the code is that whether we should consider high order page allocation. Now OOM killer doesn't kill anyone when VM suffer from higher order allocation because it doesn't help getting physical contiguos memory in normal case. Same rule could be applied.

Absolutely. It was a biggest challenge.

Hmm, I'm not sure depending on such magic value is good idea but I have no idea so I will shut up :(

Good idea.
Thanks for keeping trying this, Anton!

Kind regards,
Minchan Kim

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

VMEVENT_FD(2) Linux Programmer's Manual VMEVENT_FD(2)

NAME
vmevent_fd - Linux virtual memory management events

SYNOPSIS

       #define _GNU_SOURCE
       #include <unistd.h>
       #include <sys/syscall.h>
       #include <asm/unistd.h>
       #include <linux/types.h>
       #include <linux/vmevent.h>

       syscall(__NR_vmevent_fd, config);

DESCRIPTION
This system call creates a new file descriptor that can be used with polling routines (e.g. poll(2)) to get notified about vari- ous in-kernel virtual memory management events that might be of interest for userspace. The interface can also be used to effe- ciently monitor memory usage (e.g. number of idle and swap pages).

Applications can make overall system's memory management more nimble by adjusting theirs resources usage upon the notifica- tions.

Attributes
Attributes are the basic concept, they are described by the fol- lowing structure:

struct vmevent_attr {
__u64 value;
__u32 type;
__u32 state;
};

type may correspond to these values:

VMEVENT_ATTR_NR_AVAIL_PAGES
The attribute reports total number of available pages in the system, not including swap space (i.e. just total
RAM). value is used to setup a threshold (in number or
pages) upon which the event will be delivered by the ker- nel.

Upon notifications kernel updates all configured
attributes, so the attribute is mostly used without any thresholds, just for getting the value together with other attributes and avoid reading and parsing /proc/vmstat.

VMEVENT_ATTR_NR_FREE_PAGES
The attribute reports total number of unused (idle) RAM in the system.

value is used to setup a threshold (in number or pages) upon which the event will be delivered by the kernel.

VMEVENT_ATTR_NR_SWAP_PAGES
The attribute reports total number of swapped pages.

value is used to setup a threshold (in number or pages) upon which the event will be delivered by the kernel.

VMEVENT_ATTR_PRESSURE
The attribute reports Linux virtual memory management pressure. There are three discrete levels:

VMEVENT_PRESSURE_LOW: By setting the threshold to this value it's possible to watch whether system is reclaiming memory for new allocations. Monitoring reclaiming activity might be useful for maintaining overall system's cache level.

VMEVENT_PRESSURE_MED: The system is experiencing medium memory pressure, there is some mild swapping activity. Upon this event applications may decide to free any resources that can be easily reconstructed or re-read from a disk.

VMEVENT_PRESSURE_OOM: The system is actively thrashing, it is about to out of memory (OOM) or even the in-kernel OOM killer is on its way to trigger. Applications should do whatever they can to help the system. See proc(5) for more
information about OOM killer and its configuration
options.

value is used to setup a threshold upon which the event will be delivered by the kernel (for algebraic compar-
isons, it is defined that VMEVENT_PRESSURE_LOW <
VMEVENT_PRESSURE_MED < VMEVENT_PRESSURE_OOM, but applica- tions should not put any meaning into the absolute val- ues.)

state is used to setup thresholds' behaviour, the following
flags can be bitwise OR'ed:

VMEVENT_ATTR_STATE_VALUE_LT
Notification will be delivered when an attribute is less than a user-specified value.

VMEVENT_ATTR_STATE_VALUE_GT
Notifications will be delivered when an attribute is greater than a user-specified value.

VMEVENT_ATTR_STATE_VALUE_EQ
Notifications will be delivered when an attribute is equal to a user-specified value.

VMEVENT_ATTR_STATE_EDGE_TRIGGER
Events will be only delivered when an attribute crosses value threshold.

Events
Upon a notification, application must read out events using read(2) system call. The events are delivered using the follow- ing structure:

struct vmevent_event {
__u32 counter;
__u32 padding;
struct vmevent_attr attrs[];
};

The counter specifies a number of reported attributes, and the attrs array contains a copy of configured attributes, with vmevent_attr's value overwritten to attribute's value.

Config
vmevent_fd(2) accepts vmevent_config structure to configure the notifications:

struct vmevent_config {
__u32 size;
__u32 counter;
__u64 sample_period_ns;
struct vmevent_attr attrs[VMEVENT_CONFIG_MAX_ATTRS];
};

size must be initialized to sizeof(struct vmevent_config).

counter specifies a number of initialized attrs elements.

sample_period_ns specifies sampling period in nanoseconds. For applications it is recommended to set this value to a highest suitable period. (Note that for some attributes the delivery tim- ing is not based on the sampling period, e.g. VMEVENT_ATTR_PRES- SURE.)

RETURN VALUE
On success, vmevent_fd() returns a new file descriptor. On error, a negative value is returned and errno is set to indicate the error.

ERRORS
vmevent_fd() can fail with errors similar to open(2).

In addition, the following errors are possible:

EINVAL The failure means that an improperly initalized config
structure has been passed to the call (this also includes improperly initialized attrs arrays).

EFAULT The failure means that the kernel was unable to read the
configuration structure, that is, config parameter points to an inaccessible memory.

VERSIONS
The system call is available on Linux since kernel 3.8. Library support is yet not provided by any glibc version.

CONFORMING TO
The system call is Linux-specific.

EXAMPLE
Examples can be found in /usr/src/linux/tools/testing/vmevent/ directory.

SEE ALSO

       poll(2), read(2), proc(5), vmstat(8)

Linux 2012-10-16 VMEVENT_FD(2)

---
 man2/vmevent_fd.2 | 235 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 235 insertions(+)
 create mode 100644 man2/vmevent_fd.2

new file mode 100644
index 0000000..b631455
@@ -0,0 +1,235 @@
+.\" Copyright (C) 2008 Michael Kerrisk <[email protected]>
+.\" Copyright (C) 2012 Linaro Ltd.
+.\" 		       Anton Vorontsov <[email protected]>
+.\" Based on ideas from:
+.\" KOSAKI Motohiro, Leonid Moiseichuk, Mel Gorman, Minchan Kim and Pekka
+.\" Enberg.
+.\"
+.\" This program is free software; you can redistribute it and/or modify
+.\" it under the terms of the GNU General Public License as published by
+.\" the Free Software Foundation; either version 2 of the License, or
+.\" (at your option) any later version.
+.\"
+.\" This program is distributed in the hope that it will be useful,
+.\" but WITHOUT ANY WARRANTY; without even the implied warranty of
+.\" MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+.\" GNU General Public License for more details.
+.\"
+.\" You should have received a copy of the GNU General Public License
+.\" along with this program; if not, write to the Free Software
+.\" Foundation, Inc., 59 Temple Place, Suite 330, Boston,
+.\" MA  02111-1307  USA
+.\"
+.TH VMEVENT_FD 2 2012-10-16 Linux "Linux Programmer's Manual"
+.SH NAME
+vmevent_fd \- Linux virtual memory management events
+.SH SYNOPSIS
+.nf
+.B #define _GNU_SOURCE
+.B #include <unistd.h>
+.B #include <sys/syscall.h>
+.B #include <asm/unistd.h>
+.B #include <linux/types.h>
+.B #include <linux/vmevent.h>
+
+.\" TODO: libc wrapper
+.BI "syscall(__NR_vmevent_fd, "config );
+.fi
+.SH DESCRIPTION
+This system call creates a new file descriptor that can be used with polling
+routines (e.g.
+.BR poll (2))
+to get notified about various in-kernel virtual memory management events
+that might be of interest for userspace. The interface can
+also be used to effeciently monitor memory usage (e.g. number of idle and
+swap pages).
+
+Applications can make overall system's memory management more nimble by
+adjusting theirs resources usage upon the notifications.
+.SS Attributes
+Attributes are the basic concept, they are described by the following
+structure:
+
+.nf
+struct vmevent_attr {
+	__u64 value;
+	__u32 type;
+	__u32 state;
+};
+.fi
+
+.I type
+may correspond to these values:
+.TP
+.B VMEVENT_ATTR_NR_AVAIL_PAGES
+The attribute reports total number of available pages in the system, not
+including swap space (i.e. just total RAM).
+.I value
+is used to setup a threshold (in number or pages) upon which the event
+will be delivered by the kernel.
+
+Upon notifications kernel updates all configured attributes, so the
+attribute is mostly used without any thresholds, just for getting the
+value together with other attributes and avoid reading and parsing
+.IR /proc/vmstat .
+.TP
+.B VMEVENT_ATTR_NR_FREE_PAGES
+The attribute reports total number of unused (idle) RAM in the system.
+
+.I value
+is used to setup a threshold (in number or pages) upon which the event
+will be delivered by the kernel.
+.TP
+.B VMEVENT_ATTR_NR_SWAP_PAGES
+The attribute reports total number of swapped pages.
+
+.I value
+is used to setup a threshold (in number or pages) upon which the event
+will be delivered by the kernel.
+.TP
+.B VMEVENT_ATTR_PRESSURE
+The attribute reports Linux virtual memory management pressure. There are
+three discrete levels:
+
+.BR VMEVENT_PRESSURE_LOW :
+By setting the threshold to this value it's possible to watch whether
+system is reclaiming memory for new allocations. Monitoring reclaiming
+activity might be useful for maintaining overall system's cache level.
+
+.BR VMEVENT_PRESSURE_MED :
+The system is experiencing medium memory pressure, there is some mild
+swapping activity. Upon this event applications may decide to free any
+resources that can be easily reconstructed or re-read from a disk.
+
+.BR VMEVENT_PRESSURE_OOM :
+The system is actively thrashing, it is about to out of memory (OOM) or
+even the in-kernel OOM killer is on its way to trigger. Applications
+should do whatever they can to help the system. See
+.BR proc (5)
+for more information about OOM killer and its configuration options.
+
+.I value
+is used to setup a threshold upon which the event will be delivered by
+the kernel (for algebraic comparisons, it is defined that
+.BR VMEVENT_PRESSURE_LOW " <"
+.BR VMEVENT_PRESSURE_MED " <"
+.BR VMEVENT_PRESSURE_OOM ,
+but applications should not put any meaning into the absolute values.)
+
+.TP
+.I state
+is used to setup thresholds' behaviour, the following flags can be bitwise
+OR'ed:
+....
+.TP
+.B VMEVENT_ATTR_STATE_VALUE_LT
+Notification will be delivered when an attribute is less than a
+user-specified
+.IR "value" .
+.TP
+.B VMEVENT_ATTR_STATE_VALUE_GT
+Notifications will be delivered when an attribute is greater than a
+user-specified
+.IR "value" .
+.TP
+.B VMEVENT_ATTR_STATE_VALUE_EQ
+Notifications will be delivered when an attribute is equal to a
+user-specified
+.IR "value" .
+.TP
+.B VMEVENT_ATTR_STATE_EDGE_TRIGGER
+Events will be only delivered when an attribute crosses
+.I value
+threshold.
+.SS Events
+Upon a notification, application must read out events using
+.BR read (2)
+system call.
+The events are delivered using the following structure:
+
+.nf
+struct vmevent_event {
+	__u32			counter;
+	__u32			padding;
+	struct vmevent_attr	attrs[];
+};
+.fi
+
+The
+.I counter
+specifies a number of reported attributes, and the
+.I attrs
+array contains a copy of configured attributes, with
+.IR "vmevent_attr" 's
+.I value
+overwritten to attribute's value.
+.SS Config
+.BR vmevent_fd (2)
+accepts
+.I vmevent_config
+structure to configure the notifications:
+
+.nf
+struct vmevent_config {
+	__u32			size;
+	__u32			counter;
+	__u64			sample_period_ns;
+	struct vmevent_attr	attrs[VMEVENT_CONFIG_MAX_ATTRS];
+};
+.fi
+
+.I size
+must be initialized to
+.IR "sizeof(struct vmevent_config)" .
+
+.I counter
+specifies a number of initialized
+.I attrs
+elements.
+
+.I sample_period_ns
+specifies sampling period in nanoseconds. For applications it is
+recommended to set this value to a highest suitable period. (Note that for
+some attributes the delivery timing is not based on the sampling period,
+e.g.
+.IR VMEVENT_ATTR_PRESSURE .)
+.SH "RETURN VALUE"
+On success,
+.BR vmevent_fd ()
+returns a new file descriptor. On error, a negative value is returned and
+.I errno
+is set to indicate the error.
+.SH ERRORS
+.BR vmevent_fd ()
+can fail with errors similar to
+.BR open (2).
+
+In addition, the following errors are possible:
+.TP
+.B EINVAL
+The failure means that an improperly initalized
+.I config
+structure has been passed to the call (this also includes improperly
+initialized
+.I attrs
+arrays).
+.TP
+.B EFAULT
+The failure means that the kernel was unable to read the configuration
+structure, that is,
+.I config
+parameter points to an inaccessible memory.
+.SH VERSIONS
+The system call is available on Linux since kernel 3.8. Library support is
+yet not provided by any glibc version.
+.SH CONFORMING TO
+The system call is Linux-specific.
+.SH EXAMPLE
+Examples can be found in
+.I /usr/src/linux/tools/testing/vmevent/
+directory.
+.SH "SEE ALSO"
+.BR poll (2),
+.BR read (2),
+.BR proc (5),
+.BR vmstat (8)

1.7.12.3

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Hi Anton,

I am very happy with that because I already have yelled it several time.

Good but I think 3 is never easy.
But early notification would be better than late notification which can kill someone.

Your description doesn't include why we need new vmevent_fd(2). Of course, it's very flexible and potential to add new VM knob easily but the thing we is about to use now is only VMEVENT_ATTR_PRESSURE. Is there any other use cases for swap or free? or potential user? Adding vmevent_fd without them is rather overkill.

And I want to avoid timer-base polling of vmevent if possbile. mem_notify of KOSAKI doesn't use such timer.

I don't object but we need rationale for adding new system call which should be maintained forever once we add it.

Kind regards,
Minchan Kim

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

What ABI would you use instead?

Agreed.
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

Hi Pekka,

I thought /dev/some_knob like mem_notify and epoll is enough but please keep in mind that I'm not against vmevent_fd strongly. My point is that description should include explain about why other candidate is not good or why vmevent_fd is better. (But at least, I don't like vmevent timer polling still and I hope we use it as last resort once we can find another)

Kind regards,
Minchan Kim

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Hello Minchan,

Thanks a lot for the email!

On Thu, Oct 25, 2012 at 03:40:09PM +0900, Minchan Kim wrote:
[...]

Well, basically these are two fixed (strictly defined) levels (low and oom) + one flexible level (med), which meaning can be slightly tuned (but we still have a meaningful definition for it).

So, I guess it's a good start. :)

Number of idle pages by itself might be not that interesting, but cache+idle level is quite interesting.

By definition, _MED happens when performance already degraded, slightly, but still -- we can be swapping.

But _LOW notifications are coming when kernel is just reclaiming, so by using _LOW notifications + watching for cache level we can very easily predict the swapping activity long before we have even _MED pressure.

E.g. if idle+cache drops below amount of memory that userland can free, we'd indeed like to start freeing stuff (this somewhat resembles current logic that we have in the in-kernel LMK).

Sure, we can read and parse /proc/vmstat upon _LOW events (and that was my backup plan), but reporting stuff together would make things much nicer.

Although, I somewhat doubt that it is OK to report raw numbers, so this needs some thinking to develop more elegant solution.

Maybe it makes sense to implement something like PRESSURE_MILD with an additional nr_pages threshold, which basically hits the kernel about how many easily reclaimable pages userland has (that would be a part of our definition for the mild pressure level). So, essentially it will be

	if (pressure_index >= oom_level)
		return PRESSURE_OOM;
	else if (pressure_index >= med_level)
		return PRESSURE_MEDIUM;
	else if (userland_reclaimable_pages >= nr_reclaimable_pages)
		return PRESSURE_MILD;
	return PRESSURE_LOW;

I must admit I like the idea more than exposing NR_FREE and stuff, but the scheme reminds me the blended attributes, which we abandoned. Although, the definition sounds better now, and we seem to be doing it in the right place.

And if we go this way, then sure, we won't need any other attributes, and so we could make the API much simpler.

For pressure notifications we don't use the timers. We also read the vmstat counters together with the pressure, so "pressure + counters" effectively turns it into non-timer based polling. :)

But yes, hopefully we can get rid of the raw counters and timers, I don't them it too.

We can do it via eventfd, or /dev/chardev (which has been discussed and people didn't like it, IIRC), or signals (which also has been discussed and there are problems with this approach as well).

I'm not sure why having a syscall is a big issue. If we're making eventfd interface, then we'd need to maintain /sys/.../ ABI the same way as we maintain the syscall. What's the difference? A dedicated syscall is just a simpler interface, we don't need to mess with opening and passing things through /sys/.../.

Personally I don't have any preference (except that I distaste chardev and ioctls :), I just want to see pros and cons of all the solutions, and so far the syscall seems like an easiest way? Anyway, I'm totally open to changing it into whatever fits best.

Thanks,
Anton.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

On Thu, Oct 25, 2012 at 02:08:14AM -0700, Anton Vorontsov wrote:
[...]

...or we can call it PRESSURE_BALANCE, just to be precise and clear.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

I mean detection of "3) Are we about to OOM soon" isn't easy.

Absolutely!

So, for seeing cache level, we need new vmevent_attr?

My concern is that user can imagine various scenario with vmstat and they might start to require new vmevent_attr in future and vmevent_fd will be bloated and mm guys should care of vmevent_vd whenever they add new vmstat. I don't like it. User can do it by just reading /proc/vmstat. So I support your backup plan.

Indeed.

That's what I want! If there isn't any user who really are willing to use it, let's drop it. Do not persuade with imaginary scenario because we should be careful to introduce new ABI.

Hmm, when I see the code, timer still works and can notify to user. No?

You and i are reaching on a conclusion, at least.

No difference. What I want is just to remove unnecessary stuff in vmevent_fd and keep it as simple. If we do via /dev/chardev, I expect we can do necessary things for VM pressure. But if we can diet with vmevent_fd, It would be better. If so, maybe we have to change vmevent_fd to lowmem_fd or vmpressure_fd.

Yeb. Interface stuff isn't a big concern for low memory notification so I'm not against it stronlgy, too.

Thanks, Anton.

Kind regards,
Minchan Kim

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

On Fri, Oct 26, 2012 at 11:37:20AM +0900, Minchan Kim wrote:
[...]

Hopefully, not. We're not interested in the raw values of the cache level, but what we want is to to tell the kernel how much "easily reclaimable pages" userland has, and get notified when kernel believes that it's good time for the userland is to help. I.e. this new _MILD level:

Yeah, I think you're right. Let's make the vmevent_fd slim first. I won't even focus on the _MILD/_BALANCE level for now, we can do it later, and we always have the /proc/vmstat even if the _MILD turns out to be a bad idea.

Reading /proc/vmstat is a bit more overhead, but it's not that much at all (especially when we don't have to timer-poll the vmstat).

Yes, I was mostly saying that it is technically not required anymore, but you're right, the code still fires the timer (it just runs needlessly for the pressure attr).

Bad wording on my side.

[..]

Sure, then I'm starting the work to slim the API down, and we'll see how things are going to look after that.

Thanks a lot!

Anton.
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html