event about group change
Hello, guys,
During our work we face to problem - get events about processes die in user space (not all but some of them). In the past we had special module which solve this problem.
But now I think that this task is quite native for cgroups subsystem. Move interesting PIDs into special group and set event handler. We investigate how events made in the memcg and implement proof of concepts patch against mainline (maybe little bit old, but it's just RFC message) with the same approach to "tasks" file.
So, I see several possible use cases for others with this patch.
1) replace wait4() for processes where we dont need for return status as well as we want just know about changing status of some group of processes
2) As extension of 1, implementing some balancing mechanism in the user space between several groups. For example if some process in one group consume much system resources, processes in other groups will be protected from slowdown.
3) Extremely useful in the libcgroup (in my mind), because now it's unprotected against changing state with hands.
Need you opinion about this.
Thank you.
-- Best regards, Alex Nikiforov, Mobile SW, Advanced Software Group, Moscow R&D center, Samsung Electronics
A message that is not in the archive (sent off-list or not yet received): [email protected]
RFC 1/1 patch
Signature · 5 lines
--
Best regards,
Alex Nikiforov,
Mobile SW, Advanced Software Group,
Moscow R&D center, Samsung Electronicsinclude/linux/cgroup.h +9 -0
@@ -192,6 +192,11 @@ struct cgroup_pidlist {struct rw_semaphore mutex; }; +struct fe_eventfd_list { +struct list_head list; +struct eventfd_ctx *eventfd; +}; +struct cgroup {unsigned long flags;/* "unsigned long" so bitops work */
@@ -243,6 +248,10 @@ struct cgroup {/* List of events which userspace want to receive */ struct list_head event_list;spinlock_t event_list_lock; + +/* fork-exit event */ +struct list_head fe_notify; +spinlock_t fe_list_lock; };/*
kernel/cgroup.c +67 -0
103 lines not on this page · raw message
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [RFC] patch
On Mon, Apr 02, 2012 at 08:22:14AM +0400, Alexander Nikiforov wrote:
I think it should be part of task counter css, not core. CC list updated.
Quoted 11 lines
kernel/cgroup.c +7
-0@@ -4558,6 +4594,22 @@ void cgroup_fork(struct task_struct *child) child->cgroups = current->cgroups;get_css_set (child->cgroups);INIT_LIST_HEAD (&child->cg_list); + +struct cgroupfs_root *root; + +/* send event to the userspace */ +mutex_lock (&cgroup_mutex); +for_each_active_root (root) { +struct cgroup *cgrp;
kernel/cgroup.c void cgroup_fork(struct task_struct *child) · line 4613
Quoted 10 lines
kernel/cgroup.c +9
-0+struct fe_eventfd_list *ev; + + cgrp =task_cgroup_from_root (child, root); + +list_for_each_entry (ev, &cgrp->fe_notify, list) { +eventfd_signal (ev->eventfd,1 ); + } + } +mutex_unlock (&cgroup_mutex); }
How does it affect performance?
Quoted 18 lines
kernel/cgroup.c +5
-0/** @@ -4653,6 +4705,7 @@ void cgroup_exit(struct task_struct *tsk, int run_callbacks) {struct css_set *cg;int i; +struct cgroupfs_root *root;/* * Unlink from the css_set task list if necessary. @@ -4666,6 +4719,20 @@ void cgroup_exit(struct task_struct *tsk, int run_callbacks)write_unlock (&css_set_lock); } +/* send event to the userspace */ +mutex_lock (&cgroup_mutex); +for_each_active_root (root) { +struct cgroup *cgrp;
kernel/cgroup.c void cgroup_exit(struct task_struct *tsk, int run_callbacks) · line 4735
Quoted 10 lines
kernel/cgroup.c +10
-0+struct fe_eventfd_list *ev; + + cgrp =task_cgroup_from_root (tsk, root); + +list_for_each_entry (ev, &cgrp->fe_notify, list) { +eventfd_signal (ev->eventfd,1 ); + } + } +mutex_unlock (&cgroup_mutex); +
I think it's racy. You need to notify userspace after reassigning the task, not before.
Quoted 3 lines
kernel/cgroup.c +0
-0/* Reassign the task to the init_css_set. */ task_lock (tsk); cg = tsk->cgroups;
Signature · 1 line
Kirill A. Shutemov
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [RFC] patch
Hi Kirill, thanks for your reply
Quoted 4 lines · Kirill A. Shutemov
On Mon, Apr 02, 2012 at 08:22:14AM +0400, Alexander Nikiforov wrote:
I think it should be part of task counter css, not core. CC list updated.
Maybe, I need time to think about your suggestion.
Quoted 14 lines
Quoted 14 lines
kernel/cgroup.c +10
-0@@ -4558,6 +4594,22 @@ void cgroup_fork(struct task_struct *child) child->cgroups = current->cgroups;get_css_set (child->cgroups);INIT_LIST_HEAD (&child->cg_list); + +struct cgroupfs_root *root; + +/* send event to the userspace */ +mutex_lock (&cgroup_mutex); +for_each_active_root (root) { +struct cgroup *cgrp; +struct fe_eventfd_list *ev; + + cgrp =task_cgroup_from_root (child, root);
Quoted 8 lines
kernel/cgroup.c void cgroup_fork(struct task_struct *child) · line 4613
Quoted 7 lines
kernel/cgroup.c +6
-0+ +list_for_each_entry (ev,&cgrp->fe_notify, list) { +eventfd_signal (ev->eventfd,1 ); + } + } +mutex_unlock (&cgroup_mutex); }How does it affect performance?
One cycle with active roots, one inside task_cgroup_from_root() through cg_links and through events. I don't think what this is significant, maybe I wrong.
Quoted 15 lines
Quoted 15 lines
kernel/cgroup.c +1
-0/** @@ -4653,6 +4705,7 @@ void cgroup_exit(struct task_struct *tsk, int run_callbacks) {struct css_set *cg;int i; +struct cgroupfs_root *root;/* * Unlink from the css_set task list if necessary. @@ -4666,6 +4719,20 @@ void cgroup_exit(struct task_struct *tsk, int run_callbacks)write_unlock (&css_set_lock); }
Quoted 16 lines
Quoted 4 lines
kernel/cgroup.c +4
-0+/* send event to the userspace */ +mutex_lock (&cgroup_mutex); +for_each_active_root (root) { +struct cgroup *cgrp;kernel/cgroup.c void cgroup_exit(struct task_struct *tsk, int run_callbacks) · line 4735
Quoted 10 lines
kernel/cgroup.c +10
-0+struct fe_eventfd_list *ev; + + cgrp =task_cgroup_from_root (tsk, root); + +list_for_each_entry (ev,&cgrp->fe_notify, list) { +eventfd_signal (ev->eventfd,1 ); + } + } +mutex_unlock (&cgroup_mutex); +I think it's racy. You need to notify userspace after reassigning the task, not before.
You are right.
Quoted 4 lines
Quoted 3 lines
kernel/cgroup.c +0
-0/* Reassign the task to the init_css_set. */ task_lock (tsk); cg = tsk->cgroups;
Signature · 4 lines
Best regards,
Alex Nikiforov,
Mobile SW, Advanced Software Group,
Moscow R&D center, Samsung ElectronicsRe: [RFC] patch
In reply to a message not in the archiveHi guys, I investigate
http://lwn.net/Articles/453642/
seems that this really fit my proposal and I can make move this functionality from core to this css. But, unfortunately, Frederic git is not available on kernel.org and current Linux kernel doesn't have this patches. Could you be so kind to provide link to the kernel with this patches.
Quoted 63 lines · Kirill A. Shutemov
On Mon, Apr 02, 2012 at 08:22:14AM +0400, Alexander Nikiforov wrote:
I think it should be part of task counter css, not core. CC list updated.
Quoted 11 lines
kernel/cgroup.c +7
-0@@ -4558,6 +4594,22 @@ void cgroup_fork(struct task_struct *child) child->cgroups = current->cgroups;get_css_set (child->cgroups);INIT_LIST_HEAD (&child->cg_list); + +struct cgroupfs_root *root; + +/* send event to the userspace */ +mutex_lock (&cgroup_mutex); +for_each_active_root (root) { +struct cgroup *cgrp;kernel/cgroup.c void cgroup_fork(struct task_struct *child) · line 4613
Quoted 10 lines
kernel/cgroup.c +9
-0+struct fe_eventfd_list *ev; + + cgrp =task_cgroup_from_root (child, root); + +list_for_each_entry (ev,&cgrp->fe_notify, list) { +eventfd_signal (ev->eventfd,1 ); + } + } +mutex_unlock (&cgroup_mutex); }How does it affect performance?
Quoted 18 lines
kernel/cgroup.c +5
-0/** @@ -4653,6 +4705,7 @@ void cgroup_exit(struct task_struct *tsk, int run_callbacks) {struct css_set *cg;int i; +struct cgroupfs_root *root;/* * Unlink from the css_set task list if necessary. @@ -4666,6 +4719,20 @@ void cgroup_exit(struct task_struct *tsk, int run_callbacks)write_unlock (&css_set_lock); } +/* send event to the userspace */ +mutex_lock (&cgroup_mutex); +for_each_active_root (root) { +struct cgroup *cgrp;kernel/cgroup.c void cgroup_exit(struct task_struct *tsk, int run_callbacks) · line 4735
Quoted 10 lines
kernel/cgroup.c +10
-0+struct fe_eventfd_list *ev; + + cgrp =task_cgroup_from_root (tsk, root); + +list_for_each_entry (ev,&cgrp->fe_notify, list) { +eventfd_signal (ev->eventfd,1 ); + } + } +mutex_unlock (&cgroup_mutex); +I think it's racy. You need to notify userspace after reassigning the task, not before.
Quoted 3 lines
kernel/cgroup.c +0
-0/* Reassign the task to the init_css_set. */ task_lock (tsk); cg = tsk->cgroups;
Signature · 4 lines
Best regards,
Alex Nikiforov,
Mobile SW, Advanced Software Group,
Moscow R&D center, Samsung ElectronicsA message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [RFC] patch
2012/4/5 Alexander Nikiforov <[email protected]>:
Quoted 8 lines
Hi guys, I investigate
http://lwn.net/Articles/453642/
seems that this really fit my proposal and I can make move this functionality from core to this css. But, unfortunately, Frederic git is not available on kernel.org and current Linux kernel doesn't have this patches. Could you be so kind to provide link to the kernel with this patches.
Hi,
Last version was https://lkml.org/lkml/2012/1/31/489
I'm going to repost a new version around next week because I expect a few conflicts
with the latest selftests changes after the last merge window.
Will Cc you!
Thanks.
Quoted 82 lines · Kirill A. Shutemov
Quoted 57 lines · Alexander Nikiforov
I think it should be part of task counter css, not core. CC list updated.
Quoted 12 lines
kernel/cgroup.c +8
-0@@ -4558,6 +4594,22 @@ void cgroup_fork(struct task_struct *child) child->cgroups = current->cgroups; get_css_set(child->cgroups); INIT_LIST_HEAD(&child->cg_list); + + struct cgroupfs_root *root; + + /* send event to the userspace */ + mutex_lock(&cgroup_mutex); + for_each_active_root(root) { + struct cgroup *cgrp; + struct fe_eventfd_list *ev;kernel/cgroup.c void cgroup_fork(struct task_struct *child) · line 4614
Quoted 10 lines
kernel/cgroup.c +8
-0+ + cgrp = task_cgroup_from_root(child, root); + + list_for_each_entry(ev,&cgrp->fe_notify, list) { + eventfd_signal(ev->eventfd, 1); + } + } + mutex_unlock(&cgroup_mutex); }How does it affect performance?
Quoted 26 lines
kernel/cgroup.c +1
-0/**@@ -4653,6 +4705,7 @@ void cgroup_exit(struct task_struct *tsk, int run_callbacks) { struct css_set *cg; int i; + struct cgroupfs_root *root; /** Unlink from the
css_settask list if necessary.kernel/cgroup.c +9
-0@@ -4666,6 +4719,20 @@ void cgroup_exit(struct task_struct *tsk, int run_callbacks) write_unlock(&css_set_lock); } + /* send event to the userspace */ + mutex_lock(&cgroup_mutex); + for_each_active_root(root) { + struct cgroup *cgrp; + struct fe_eventfd_list *ev; + + cgrp = task_cgroup_from_root(tsk, root); + + list_for_each_entry(ev,&cgrp->fe_notify, list) {Quoted 14 lines
kernel/cgroup.c void cgroup_exit(struct task_struct *tsk, int run_callbacks) · line 4735
Quoted 5 lines
kernel/cgroup.c +5
-0+ eventfd_signal(ev->eventfd, 1); + } + } + mutex_unlock(&cgroup_mutex); +I think it's racy. You need to notify userspace after reassigning the task, not before.
Quoted 3 lines
kernel/cgroup.c +0
-0/* Reassign the task to the init_css_set. */task_lock(tsk);
cg =tsk->cgroups;--
Best regards,
Alex Nikiforov,
Mobile SW, Advanced Software Group,
Moscow R&D center, Samsung Electronics