EPOLL_KILLME: New flag to epoll_wait() that subscribes process to death row (new syscall)

It is common for services to be stateless around their main event loop. If a process passes the EPOLL_KILLME flag to epoll_wait5() then it signals to the kernel that epoll_wait5() may not complete, and the kernel may send SIGKILL if resources get tight.

See my systemd patch: https://github.com/shawnl/systemd/tree/killme

Android uses this memory model for all programs, and having it in the kernel will enable integration with the page cache (not in this series).

It is common for services to be stateless around their main event loop. If a process passes the EPOLL_KILLME flag to epoll_wait5() then it signals to the kernel that epoll_wait5() may not complete, and the kernel may send SIGKILL if resources get tight.

as the maintainer of glib2 which is used by a *lot* of things; I'm not

Quoted 1 line · Colin Walters

as the maintainer of glib2 which is used by a *lot* of things; I'm not

(I meant to say "a" maintainer)

Also, while I'm not an expert in Android, I think the "what to kill" logic
there lives in userspace, right? So it feels like we should expose this
state in e.g. /proc and allow userspace daemons (e.g. systemd, kubelet) to perform idle collection too, even if the system isn't actually low on resources from the kernel's perspective.

And doing that requires some sort of kill(pid, SIGKILL_IF_IDLE) or so?

Quoted 5 lines · Shawn Landden

It is common for services to be stateless around their main event loop. If a process passes the EPOLL_KILLME flag to epoll_wait5() then it signals to the kernel that epoll_wait5() may not complete, and the kernel may send SIGKILL if resources get tight.

I've thought about something like this in the past too and would love to see it land. Bigger picture, this also comes up in (server) container environments, see e.g.:

https://docs.openshift.com/container-platform/3.3/admin_
guide/idling_applications.html

There's going to be a long slog getting apps to actually make use of this, but I suspect if it gets wrapped up nicely in some "framework" libraries for C/C++, and be bound in the language ecosystems like golang we could see a fair amount of adoption on the order of a year or two.

However, while I understand why it feels natural to tie this to epoll, as the maintainer of glib2 which is used by a *lot* of things; I'm not sure we're going to port to epoll anytime soon.

Why not just make this a prctl()? It's not like it's really any less racy to do:

prctl(PR_SET_IDLE)
epoll()

and this also allows:

prctl(PR_SET_IDLE)
poll()

And as this is most often just going to be an optional hint it's easier to e.g. just ignore EINVAL
from the prctl().

This solves the fact that epoll_pwait() already is a 6 argument (maximum allowed) syscall. But what if the process has multiple epoll() instances in multiple threads?

Quoted 2 lines · Shawn Landden

This solves the fact that epoll_pwait() already is a 6 argument (maximum

allowed) syscall. But what if the process has multiple epoll() instances in multiple threads?

Well, that's a subset of the general question of - what is the interaction of this system call and threading? It looks like you've prototyped this out in userspace with systemd, but from a quick glance at the current git, systemd's threading is limited doing sync()/fsync() and gethostbyname() async.

But languages with a GC tend to at least use a background thread for that, and of course lots of modern userspace makes heavy use of multithreading (or variants like goroutines).

A common pattern though is to have a "main thread" that acts as a control point and runs the mainloop (particularly for anything with a GUI).
That's
going to be the thing calling prctl(SET_IDLE) - but I think its idle state should implicitly
affect the whole process, since for a lot of apps those other threads are going to
just be "background".

It'd probably then be an error to use prctl(SET_IDLE) in more than one thread
ever? (Although that might break in golang due to the way goroutines can be migrated across threads)

That'd probably be a good "generality test" - what would it take to have this system call be used for a simple golang webserver app that's e.g. socket activated by systemd, or a Kubernetes service? Or another really interesting case would be qemu; make it easy to flag VMs as always having this state (most of my testing VMs are like this; it's OK if they get
destroyed, I just reinitialize them from the gold state).

Going back to threading - a tricky thing we should handle in general is when userspace libraries create threads that are unknown to the app; the "async gethostbyname()" is a good example. To be conservative we'd likely need to "fail non-idle", but figure out some way tell the kernel for e.g. GC threads that they're still idle.

Quoted 2 lines · Shawn Landden

This solves the fact that epoll_pwait() already is a 6 argument (maximum

allowed) syscall. But what if the process has multiple epoll() instances in multiple threads?

Well, that's a subset of the general question of - what is the interaction of this system call and threading? It looks like you've prototyped this out in userspace with systemd, but from a quick glance at the current git, systemd's threading is limited doing sync()/fsync() and gethostbyname() async.

But languages with a GC tend to at least use a background thread for that, and of course lots of modern userspace makes heavy use of multithreading (or variants like goroutines).

A common pattern though is to have a "main thread" that acts as a control point and runs the mainloop (particularly for anything with a GUI).
That's
going to be the thing calling prctl(SET_IDLE) - but I think its idle state should implicitly
affect the whole process, since for a lot of apps those other threads are going to
just be "background".

It'd probably then be an error to use prctl(SET_IDLE) in more than one thread
ever? (Although that might break in golang due to the way goroutines can be migrated across threads)

That'd probably be a good "generality test" - what would it take to have this system call be used for a simple golang webserver app that's e.g. socket activated by systemd, or a Kubernetes service? Or another really interesting case would be qemu; make it easy to flag VMs as always having this state (most of my testing VMs are like this; it's OK if they get
destroyed, I just reinitialize them from the gold state).

Going back to threading - a tricky thing we should handle in general is when userspace libraries create threads that are unknown to the app; the "async gethostbyname()" is a good example. To be conservative we'd likely need to "fail non-idle", but figure out some way tell the kernel for e.g. GC threads that they're still idle.

threading is limited doing sync()/fsync() and gethostbyname() async.

But languages with a GC tend to at least use a background thread for that,
and of course lots of modern userspace makes heavy use of multithreading
(or variants like goroutines).

A common pattern though is to have a "main thread" that acts as a control
point and runs the mainloop (particularly for anything with a GUI). That's
going to be the thing calling prctl(SET_IDLE) - but I think its idle state should implicitly
affect the whole process, since for a lot of apps those other threads are going to
just be "background".

It'd probably then be an error to use prctl(SET_IDLE) in more than one thread
ever? (Although that might break in golang due to the way goroutines can
be migrated across threads)

That'd probably be a good "generality test" - what would it take to have
this system call be used for a simple golang webserver app that's e.g.
socket activated by systemd, or a Kubernetes service? Or another really interesting case would be qemu; make it easy to flag VMs as always
having this state (most of my testing VMs are like this; it's OK if they get
destroyed, I just reinitialize them from the gold state).

Going back to threading - a tricky thing we should handle in general is when userspace libraries create threads that are unknown to the app;
the "async gethostbyname()" is a good example. To be conservative we'd
likely need to "fail non-idle", but figure out some way tell the kernel
for e.g. GC threads that they're still

Quoted 5 lines · Shawn Landden

It is common for services to be stateless around their main event loop. If a process passes the EPOLL_KILLME flag to epoll_wait5() then it signals to the kernel that epoll_wait5() may not complete, and the kernel may send SIGKILL if resources get tight.

I've thought about something like this in the past too and would love to see it land. Bigger picture, this also comes up in (server) container environments, see e.g.:

https://docs.openshift.com/container-platform/3.3/admin_
guide/idling_applications.html

There's going to be a long slog getting apps to actually make use of this, but I suspect if it gets wrapped up nicely in some "framework" libraries for C/C++, and be bound in the language ecosystems like golang we could see a fair amount of adoption on the order of a year or two.

However, while I understand why it feels natural to tie this to epoll, as the maintainer of glib2 which is used by a *lot* of things; I'm not sure we're going to port to epoll anytime soon.

Why not just make this a prctl()? It's not like it's really any less racy to do:

prctl(PR_SET_IDLE)
epoll()

and this also allows:

prctl(PR_SET_IDLE)
poll()

And as this is most often just going to be an optional hint it's easier to e.g. just ignore EINVAL
from the prctl().

mm/oom_kill.c bool out_of_memory(struct oom_control *oc) · line 1037

Quoted 9 lines · Shawn Landden

mm/oom_kill.c +5 -0

@@ -1029,6 +1030,22 @@ bool out_of_memory(struct oom_control *oc)
              return true;
      }
 
+     /*
+      * Check death row.
+      */
+     if (!list_empty(eventpoll_deathrow_list())) {
+             struct list_head *l = eventpoll_deathrow_list();

Unsafe traversal. List can become empty at this moment.

mm/oom_kill.c bool out_of_memory(struct oom_control *oc) · line 1045

Quoted 8 lines

mm/oom_kill.c +8 -0

+             struct task_struct *ts = list_first_entry(l,
+                                      struct task_struct, se.deathrow);
+
+             pr_debug("Killing pid %u from EPOLL_KILLME death row.",
+                     ts->pid);
+
+             /* We use SIGKILL so as to cleanly interrupt ep_poll() */
+             kill_pid(task_pid(ts), SIGKILL, 1);

send_sig() ?

mm/oom_kill.c bool out_of_memory(struct oom_control *oc) · line 1051

Quoted 7 lines

mm/oom_kill.c +3 -0

+             return true;
+     }
+
      /*
       * The OOM killer does not compensate for IO-less reclaim.
       * pagefault_out_of_memory lost its gfp context so we have to

And why is

  static int oom_fd = open("/proc/self/oom_score_adj", O_WRONLY);

and then toggling between

  write(fd, "1000", 4);

and

  write(fd, "0", 1);

not sufficient? Adding prctl() that do this might be handy though.

It is common for services to be stateless around their main event loop. If a process passes the EPOLL_KILLME flag to epoll_wait5() then it signals to the kernel that epoll_wait5() may not complete, and the kernel may send SIGKILL if resources get tight.

See my systemd patch: https://github.com/shawnl/systemd/tree/killme

Android uses this memory model for all programs, and having it in the kernel will enable integration with the page cache (not in this series).

mm/oom_kill.c +6 -0

@@ -1029,6 +1030,22 @@ bool out_of_memory(struct oom_control *oc)
 		return true;
 	}
 
+	/*
+	 * Check death row.
+	 */
+	if (!list_empty(eventpoll_deathrow_list())) {
+		struct list_head *l = eventpoll_deathrow_list();
+		struct task_struct *ts = list_first_entry(l,

It is common for services to be stateless around their main event loop. If a process sets PR_SET_IDLE to PR_IDLE_MODE_KILLME then it signals to the kernel that epoll_wait() and friends may not complete, and the kernel may send SIGKILL if resources get tight.

See my systemd patch: https://github.com/shawnl/systemd/tree/prctl

Android uses this memory model for all programs, and having it in the kernel will enable integration with the page cache (not in this series).

16 bytes per process is kinda spendy, but I want to keep
lru behavior, which mem_score_adj does not allow. When a supervisor, like Android's user input is keeping track this can be done in user-space. It could be pulled out of task_struct if an cross-indexing additional red-black tree is added to support pid-based lookup.

mm/oom_kill.c +7 -0

@@ -1018,6 +1060,24 @@ bool out_of_memory(struct oom_control *oc)
 			return true;
 	}
 
+	/*
+	 * Check death row for current memcg or global.
+	 */
+	l = oom_target_get_queue(current);
+	if (!list_empty(l)) {
+		struct task_struct *ts = list_first_entry(l,
+				struct task_struct, se.oom_target_queue);

Quoted 7 lines · Shawn Landden

It is common for services to be stateless around their main event loop. If a process sets PR_SET_IDLE to PR_IDLE_MODE_KILLME then it signals to the kernel that epoll_wait() and friends may not complete, and the kernel may send SIGKILL if resources get tight.

See my systemd patch: https://github.com/shawnl/systemd/tree/prctl

Quoted 7 lines

Android uses this memory model for all programs, and having it in the kernel will enable integration with the page cache (not in this series).

16 bytes per process is kinda spendy, but I want to keep lru behavior, which mem_score_adj does not allow. When a supervisor, like Android's user input is keeping track this can be done in

user-space.

Quoted 2 lines

It could be pulled out of task_struct if an cross-indexing additional red-black tree is added to support pid-based lookup.

This is still an abuse and the patch is wrong. We really do have an API to use I fail to see why you do not use it.

[...]

Quoted 11 lines

mm/oom_kill.c +7 -0

@@ -1018,6 +1060,24 @@ bool out_of_memory(struct oom_control *oc)
                      return true;
      }
 
+     /*
+      * Check death row for current memcg or global.
+      */
+     l = oom_target_get_queue(current);
+     if (!list_empty(l)) {
+             struct task_struct *ts = list_first_entry(l,
+                             struct task_struct, se.oom_target_queue);

mm/oom_kill.c bool out_of_memory(struct oom_control *oc) · line 1079

Quoted 10 lines

mm/oom_kill.c +10 -0

+
+             pr_debug("Killing pid %u from EPOLL_KILLME death row.",
+                      ts->pid);
+
+             /* We use SIGKILL instead of the oom killer
+              * so as to cleanly interrupt ep_poll()
+              */
+             send_sig(SIGKILL, ts, 1);
+             return true;
+     }

Still not NUMA aware and completely backwards. If this is a memcg OOM then it is _memcg_ to evaluate not the current. The oom might happen up the hierarchy due to hard limit.

But still, you should be very clear _why_ the existing oom tuning is not appropropriate and we can think of a way to hanle it better but cramming the oom selection this way is simply not acceptable.
--
Michal Hocko
SUSE Labs

Quoted 4 lines · Michal Hocko

Quoted 3 lines · Shawn Landden

16 bytes per process is kinda spendy, but I want to keep lru behavior, which mem_score_adj does not allow. When a supervisor, like Android's user input is keeping track this can be done in

Quoted 7 lines

user-space.

Quoted 2 lines

It could be pulled out of task_struct if an cross-indexing additional red-black tree is added to support pid-based lookup.

This is still an abuse and the patch is wrong. We really do have an API to use I fail to see why you do not use it.

When I looked at wait_queue_head_t it was 20 byes.

Quoted 15 lines · Michal Hocko

Quoted 14 lines · Shawn Landden

It is common for services to be stateless around their main event loop. If a process sets PR_SET_IDLE to PR_IDLE_MODE_KILLME then it signals to the kernel that epoll_wait() and friends may not complete, and the kernel may send SIGKILL if resources get tight.

See my systemd patch: https://github.com/shawnl/systemd/tree/prctl

Android uses this memory model for all programs, and having it in the kernel will enable integration with the page cache (not in this series).

16 bytes per process is kinda spendy, but I want to keep lru behavior, which mem_score_adj does not allow. When a supervisor, like Android's user input is keeping track this can be done in

Quoted 7 lines

user-space.

Quoted 2 lines

It could be pulled out of task_struct if an cross-indexing additional red-black tree is added to support pid-based lookup.

This is still an abuse and the patch is wrong. We really do have an API to use I fail to see why you do not use it.

When I looked at wait_queue_head_t it was 20 byes.

Quoted 21 lines · Shawn Landden

Quoted 19 lines · Michal Hocko

Quoted 7 lines · Shawn Landden

It is common for services to be stateless around their main event loop. If a process sets PR_SET_IDLE to PR_IDLE_MODE_KILLME then it signals to the kernel that epoll_wait() and friends may not complete, and the kernel may send SIGKILL if resources get tight.

See my systemd patch: https://github.com/shawnl/systemd/tree/prctl

Quoted 7 lines

Android uses this memory model for all programs, and having it in the kernel will enable integration with the page cache (not in this series).

16 bytes per process is kinda spendy, but I want to keep lru behavior, which mem_score_adj does not allow. When a supervisor, like Android's user input is keeping track this can be done in

user-space.

Quoted 2 lines

It could be pulled out of task_struct if an cross-indexing additional red-black tree is added to support pid-based lookup.

Quoted 4 lines

Quoted 3 lines

This is still an abuse and the patch is wrong. We really do have an API to use I fail to see why you do not use it.

When I looked at wait_queue_head_t it was 20 byes.

I do not understand. What I meant to say is that we do have a proper user api to hint OOM killer decisions.

--
Michal Hocko
SUSE Labs

Quoted 25 lines · Michal Hocko

Quoted 24 lines · Shawn Landden

Quoted 22 lines · Michal Hocko

Quoted 7 lines · Shawn Landden

It is common for services to be stateless around their main event loop. If a process sets PR_SET_IDLE to PR_IDLE_MODE_KILLME then it signals to the kernel that epoll_wait() and friends may not complete, and the kernel may send SIGKILL if resources get tight.

See my systemd patch: https://github.com/shawnl/systemd/tree/prctl

Quoted 7 lines

Android uses this memory model for all programs, and having it in the kernel will enable integration with the page cache (not in this series).

16 bytes per process is kinda spendy, but I want to keep lru behavior, which mem_score_adj does not allow. When a supervisor, like Android's user input is keeping track this can be done in

user-space.

Quoted 2 lines

It could be pulled out of task_struct if an cross-indexing additional red-black tree is added to support pid-based lookup.

This is still an abuse and the patch is wrong. We really do have an API to use I fail to see why you do not use it.

Quoted 4 lines

Quoted 1 line

When I looked at wait_queue_head_t it was 20 byes.

I do not understand. What I meant to say is that we do have a proper user api to hint OOM killer decisions.

This is a FIFO queue, rather than a heuristic, which is all you get with the current API.

Quoted 5 lines · Shawn Landden

16 bytes per process is kinda spendy, but I want to keep
lru behavior, which mem_score_adj does not allow. When a supervisor, like Android's user input is keeping track this can be done in user-space. It could be pulled out of task_struct if an cross-indexing additional red-black tree is added to support pid-based lookup.

This is still an abuse and the patch is wrong. We really do have an API to use I fail to see why you do not use it.

It is common for services to be stateless around their main event loop. If a process sets PR_SET_IDLE to PR_IDLE_MODE_KILLME then it signals to the kernel that epoll_wait() and friends may not complete, and the kernel may send SIGKILL if resources get tight.

See my systemd patch: https://github.com/shawnl/systemd/tree/prctl

Android uses this memory model for all programs, and having it in the kernel will enable integration with the page cache (not in this series).

16 bytes per process is kinda spendy, but I want to keep
lru behavior, which mem_score_adj does not allow. When a supervisor, like Android's user input is keeping track this can be done in user-space. It could be pulled out of task_struct if an cross-indexing additional red-black tree is added to support pid-based lookup.

See my systemd patch: https://github.com/shawnl/systemd/tree/prctl

Android uses this memory model for all programs, and having it in the kernel will enable integration with the page cache (not in this series).

v2
switch to prctl, memcg support

v3
use <linux/wait.h>
put OOM after constraint checking

---
 fs/eventpoll.c             | 27 ++++++++++++++++++++
 fs/proc/array.c            |  7 ++++++
 include/linux/memcontrol.h |  3 +++
 include/linux/oom.h        |  4 +++
 include/linux/sched.h      |  1 +
 include/uapi/linux/prctl.h |  4 +++
 kernel/cgroup/cgroup.c     | 61 ++++++++++++++++++++++++++++++++++++++++++++++
 kernel/exit.c              |  1 +
 kernel/sys.c               |  9 +++++++
 mm/memcontrol.c            |  2 ++
 mm/oom_kill.c              | 47 +++++++++++++++++++++++++++++++++++
 11 files changed, 166 insertions(+)

fs/eventpoll.c +27 -0

index 2fabd19cdeea..745662f9a7e1 100644
@@ -43,6 +43,8 @@
 #include <linux/compat.h>
 #include <linux/rculist.h>
 #include <net/busy_poll.h>
+#include <linux/memcontrol.h>
+#include <linux/oom.h>
 
 /*
  * LOCKING:

@@ -1761,6 +1763,19 @@ static int ep_poll(struct eventpoll *ep, struct epoll_event __user *events,
        u64 slack = 0;
        wait_queue_entry_t wait;
        ktime_t expires, *to = NULL;
+       DEFINE_WAIT_FUNC(oom_target_wait, oom_target_callback);
+       DEFINE_WAIT_FUNC(oom_target_wait_mcg, oom_target_callback);
+
+       if (current->oom_target) {
+#ifdef CONFIG_MEMCG
+               struct mem_cgroup *mcg;
+
+               mcg = mem_cgroup_from_task(current);
+               if (mcg)
+                       add_wait_queue(&mcg->oom_target, &oom_target_wait_mcg);
+#endif
+               add_wait_queue(oom_target_get_wait(), &oom_target_wait);
+       }
 
        if (timeout > 0) {
                struct timespec64 end_time = ep_set_mstimeout(timeout);

@@ -1850,6 +1865,18 @@ static int ep_poll(struct eventpoll *ep, struct epoll_event __user *events,
            !(res = ep_send_events(ep, events, maxevents)) && !timed_out)
                goto fetch_events;
 
+       if (current->oom_target) {
+#ifdef CONFIG_MEMCG
+               struct mem_cgroup *mcg;
+
+               mcg = mem_cgroup_from_task(current);
+               if (mcg)
+                       remove_wait_queue(&mcg->oom_target,
+                                       &oom_target_wait_mcg);
+#endif
+               remove_wait_queue(oom_target_get_wait(), &oom_target_wait);
+       }
+
        return res;
 }
 

fs/proc/array.c +7 -0

index 9390032a11e1..1954ae87cb88 100644
@@ -350,6 +350,12 @@ static inline void task_seccomp(struct seq_file *m, struct task_struct *p)
        seq_putc(m, '\n');
 }
 
+static inline void task_idle(struct seq_file *m, struct task_struct *p)
+{
+       seq_put_decimal_ull(m, "Idle:\t", p->oom_target);
+       seq_putc(m, '\n');
+}
+
 static inline void task_context_switch_counts(struct seq_file *m,
                                                struct task_struct *p)
 {

@@ -381,6 +387,7 @@ int proc_pid_status(struct seq_file *m, struct pid_namespace *ns,
        task_sig(m, task);
        task_cap(m, task);
        task_seccomp(m, task);
+       task_idle(m, task);
        task_cpus_allowed(m, task);
        cpuset_task_status_allowed(m, task);
        task_context_switch_counts(m, task);

include/linux/memcontrol.h +3 -0

index 69966c461d1c..02eb92e7eff5 100644
@@ -30,6 +30,7 @@
 #include <linux/vmstat.h>
 #include <linux/writeback.h>
 #include <linux/page-flags.h>
+#include <linux/wait.h>
 
 struct mem_cgroup;
 struct page;

@@ -261,6 +262,8 @@ struct mem_cgroup {
        struct list_head event_list;
        spinlock_t event_list_lock;
 
+       wait_queue_head_t       oom_target;
+
        struct mem_cgroup_per_node *nodeinfo[0];
        /* WARNING: nodeinfo must be the last member here */
 };

include/linux/oom.h +4 -0

index 01c91d874a57..88acea9e0a59 100644
@@ -102,6 +102,10 @@ extern void oom_killer_enable(void);
 
 extern struct task_struct *find_lock_task_mm(struct task_struct *p);
 
+extern void exit_oom_target(void);
+struct wait_queue_head *oom_target_get_wait(void);
+int oom_target_callback(wait_queue_entry_t *wait, unsigned mode, int sync, void *key);
+
 /* sysctls */
 extern int sysctl_oom_dump_tasks;
 extern int sysctl_oom_kill_allocating_task;

include/linux/sched.h +1 -0

index fdf74f27acf1..51b0e5987e8c 100644
@@ -652,6 +652,7 @@ struct task_struct {
        /* disallow userland-initiated cgroup migration */
        unsigned                        no_cgroup_migration:1;
 #endif
+       unsigned                        oom_target:1;
 
        unsigned long                   atomic_flags; /* Flags requiring atomic access. */
 

include/uapi/linux/prctl.h +4 -0

index b640071421f7..94868317c6f2 100644
@@ -198,4 +198,8 @@ struct prctl_mm_map {
 # define PR_CAP_AMBIENT_LOWER          3
 # define PR_CAP_AMBIENT_CLEAR_ALL      4
 
+#define PR_SET_IDLE            48
+#define PR_GET_IDLE            49
+# define PR_IDLE_MODE_KILLME   1
+
 #endif /* _LINUX_PRCTL_H */

kernel/cgroup/cgroup.c +5 -0

index 44857278eb8a..081bcd84a8d0 100644
@@ -55,6 +55,8 @@
 #include <linux/nsproxy.h>
 #include <linux/file.h>
 #include <net/sock.h>
+#include <linux/oom.h>
+#include <linux/memcontrol.h>
 
 #define CREATE_TRACE_POINTS
 #include <trace/events/cgroup.h>

@@ -756,6 +758,9 @@ static void css_set_move_task(struct task_struct *task,
                              struct css_set *from_cset, struct css_set *to_cset,
                              bool use_mg_tasks)
 {
+#ifdef CONFIG_MEMCG
+       struct mem_cgroup *mcg;
+#endif
        lockdep_assert_held(&css_set_lock);
 
        if (to_cset && !css_set_populated(to_cset))

kernel/cgroup/cgroup.c +27 -0

+                * TODO: this is O(n), add rb-tree to make it O(logn)
+                */
+               mcg = mem_cgroup_from_task(task);
+               if (mcg) {
+                       struct wait_queue_entry *wait;
+
+                       spin_lock(&mcg->oom_target.lock);
+                       if (!waitqueue_active(&mcg->oom_target))
+                               goto empty_from;
+                       wait = list_first_entry(&mcg->oom_target.head,
+                                               wait_queue_entry_t, entry);
+                       do {
+                               struct list_head *list;
+
+                               if (wait->private == task)
+                                       __remove_wait_queue(&mcg->oom_target,
+                                                         wait);
+                               list = wait->entry.next;
+                               if (list_is_last(list, &mcg->oom_target.head))
+                                       break;
+                               wait = list_entry(list,
+                                       struct wait_queue_entry, entry);
+                       } while (1);
+empty_from:
+                       spin_unlock(&mcg->oom_target.lock);
+               }
+#endif
                if (!css_set_populated(from_cset))
                        css_set_update_populated(from_cset, false);

        } else {

kernel/cgroup/cgroup.c +27 -0

@@ -797,6 +831,33 @@ static void css_set_move_task(struct task_struct *task,
                rcu_assign_pointer(task->cgroups, to_cset);
                list_add_tail(&task->cg_list, use_mg_tasks ? &to_cset->mg_tasks :
                                                             &to_cset->tasks);
+#ifdef CONFIG_MEMCG
+               /* dequeue from memcg->oom_target */
+               mcg = mem_cgroup_from_task(task);
+               if (mcg) {
+                       struct wait_queue_entry *wait;
+
+                       spin_lock(&mcg->oom_target.lock);
+                       if (!waitqueue_active(&mcg->oom_target))
+                               goto empty_to;
+                       wait = list_first_entry(&mcg->oom_target.head,
+                                               wait_queue_entry_t, entry);
+                       do {
+                               struct list_head *list;
+
+                               if (wait->private == task)
+                                       __add_wait_queue(&mcg->oom_target,
+                                                         wait);
+                               list = wait->entry.next;
+                               if (list_is_last(list, &mcg->oom_target.head))
+                                       break;
+                               wait = list_entry(list,
+                                       struct wait_queue_entry, entry);
+                       } while (1);
+empty_to:
+                       spin_unlock(&mcg->oom_target.lock);
+               }
+#endif
        }
 }
 

kernel/exit.c +1 -0

index f6cad39f35df..2788fbdae267 100644
@@ -62,6 +62,7 @@
 #include <linux/random.h>
 #include <linux/rcuwait.h>
 #include <linux/compat.h>
+#include <linux/eventpoll.h>
 
 #include <linux/uaccess.h>
 #include <asm/unistd.h>

kernel/sys.c +9 -0

index 524a4cb9bbe2..e1eb049a85e6 100644
@@ -2386,6 +2386,15 @@ SYSCALL_DEFINE5(prctl, int, option, unsigned long, arg2, unsigned long, arg3,
        case PR_GET_FP_MODE:
                error = GET_FP_MODE(me);
                break;
+       case PR_SET_IDLE:
+               if (!((arg2 == 0) || (arg2 == PR_IDLE_MODE_KILLME)))
+                       return -EINVAL;
+               me->oom_target = arg2;
+               error = 0;
+               break;
+       case PR_GET_IDLE:
+               error = me->oom_target;
+               break;
        default:
                error = -EINVAL;
                break;

mm/memcontrol.c +2 -0

index 661f046ad318..a4e3b93aeccd 100644
@@ -4300,6 +4300,8 @@ mem_cgroup_css_alloc(struct cgroup_subsys_state *parent_css)
                        memory_cgrp_subsys.broken_hierarchy = true;
        }
 
+       init_waitqueue_head(&memcg->oom_target);
+
        /* The following stuff does not apply to the root */
        if (!parent) {
                root_mem_cgroup = memcg;

mm/oom_kill.c +47 -1

index dee0f75c3013..c5d8f5a716bc 100644
@@ -41,6 +41,9 @@
 #include <linux/kthread.h>
 #include <linux/init.h>
 #include <linux/mmu_notifier.h>
+#include <linux/eventpoll.h>
+#include <linux/wait.h>
+#include <linux/memcontrol.h>
 
 #include <asm/tlb.h>
 #include "internal.h"

@@ -54,6 +57,23 @@ int sysctl_oom_dump_tasks = 1;
 
 DEFINE_MUTEX(oom_lock);
 
+static DECLARE_WAIT_QUEUE_HEAD(oom_target);
+
+/* Clean up after a EPOLL_KILLME process quits.
+ * Called by kernel/exit.c.
+ */
+void exit_oom_target(void)
+{
+       DECLARE_WAITQUEUE(wait, current);
+
+       remove_wait_queue(&oom_target, &wait);
+}
+
+inline struct wait_queue_head *oom_target_get_wait()
+{
+       return &oom_target;
+}
+
 #ifdef CONFIG_NUMA
 /**
  * has_intersects_mems_allowed() - check task eligiblity for kill

@@ -994,6 +1014,18 @@ int unregister_oom_notifier(struct notifier_block *nb)
 }
 EXPORT_SYMBOL_GPL(unregister_oom_notifier);
 
+int oom_target_callback(wait_queue_entry_t *wait, unsigned mode, int sync, void *key)
+{
+       struct task_struct *ts = wait->private;
+
+       /* We use SIGKILL instead of the oom killer
+        * so as to cleanly interrupt ep_poll()
+        */
+       pr_info("Killing pid %u from prctl(PR_SET_IDLE) death row.\n", ts->pid);
+       send_sig(SIGKILL, ts, 1);
+       return 0;
+}
+
 /**
  * out_of_memory - kill the "best" process when we run out of memory
  * @oc: pointer to struct oom_control

@@ -1007,6 +1039,7 @@ bool out_of_memory(struct oom_control *oc)
 {
        unsigned long freed = 0;
        enum oom_constraint constraint = CONSTRAINT_NONE;
+       wait_queue_head_t *w;
 
        if (oom_killer_disabled)
                return false;

@@ -1056,6 +1089,20 @@ bool out_of_memory(struct oom_control *oc)
                return true;
        }
 
+       /*
+        * Check death row for current memcg or global.
+        */
+#ifdef CONFIG_MEMCG
+       if (is_memcg_oom(oc))
+               w = &oc->memcg->oom_target;
+       else
+#endif
+               w = oom_target_get_wait();
+       if (waitqueue_active(w)) {
+               wake_up(w);
+               return true;
+       }
+
        select_bad_process(oc);
        /* Found nothing?!?! Either we hang forever, or we panic. */
        if (!oc->chosen && !is_sysrq_oom(oc) && !is_memcg_oom(oc)) {
--

2.14.1

See my systemd patch: https://github.com/shawnl/systemd/tree/prctl

Android uses this memory model for all programs, and having it in the kernel will enable integration with the page cache (not in this series).

mm/oom_kill.c +2 -0

+}
+
 #ifdef CONFIG_NUMA
 /**
  * has_intersects_mems_allowed() - check task eligiblity for kill

mm/oom_kill.c +7 -0

+	 */
+	pr_debug("Killing pid %u from prctl(PR_SET_IDLE) death row.\n", ts->pid);
+	send_sig(SIGKILL, ts, 1);
+	return 0;
+}
+
 /**
  * out_of_memory - kill the "best" process when we run out of memory
  * @oc: pointer to struct oom_control

@@ -1007,6 +1038,7 @@ bool out_of_memory(struct oom_control *oc)
 {
 	unsigned long freed = 0;
 	enum oom_constraint constraint = CONSTRAINT_NONE;
+	wait_queue_head_t *w;
 
 	if (oom_killer_disabled)
 		return false;

@@ -1056,6 +1088,17 @@ bool out_of_memory(struct oom_control *oc)
 		return true;
 	}
 

It is common for services to be stateless around their main event loop. If a process sets PR_SET_IDLE to PR_IDLE_MODE_KILLME then it signals to the kernel that epoll_wait() and friends may not complete, and the kernel may send SIGKILL if resources get tight.

See my systemd patch: https://github.com/shawnl/systemd/tree/prctl

Android uses this memory model for all programs, and having it in the kernel will enable integration with the page cache (not in this series).

16 bytes per process is kinda spendy, but I want to keep
lru behavior, which mem_score_adj does not allow. When a supervisor, like Android's user input is keeping track this can be done in user-space. It could be pulled out of task_struct if an cross-indexing additional red-black tree is added to support pid-based lookup.