cgroup: allow deletion of cgroups containing only dying processes
Subsystems (3)
PATCH 1/2 cgroup: allow deletion of cgroups containing only dying processes
A cgroup containing only dying tasks will be seen as empty when a userspace process reads its cgroup.procs or cgroup.tasks files. It should be safe to delete such a cgroup as it is considered empty. However if one of the dying tasks did not reach cgroup_exit then an attempt to delete the cgroup will fail with EBUSY because cgroup_is_populated() will not consider it empty until all tasks reach cgroup_exit. Such a condition can be triggered when a task consumes large amounts of memory and spends enough time in exit_mm to create delay between the moment it is flagged as PF_EXITING and the moment it reaches cgroup_exit.
Fix this by detecting cgroups containing only dying tasks during cgroup destruction and proceeding with it while postponing the final step of releasing the last reference until the last task reaches cgroup_exit.
--- include/linux/cgroup-defs.h | 3 ++ kernel/cgroup/cgroup.c | 65 +++++++++++++++++++++++++++++++++---- 2 files changed, 61 insertions(+), 7 deletions(-)
include/linux/cgroup-defs.h +3 -0
@@ -71,6 +71,9 @@ enum {/* Cgroup is frozen. */ CGRP_FROZEN , + +/* Cgroup is dead. */ + CGRP_DEAD, };/* cgroup_root->flags */
kernel/cgroup/cgroup.c +58 -7
107 lines not on this page · raw message
Signature · 1 line
2.25.0.rc1.283.g88dfdc4193-goog
PATCH 2/2 kselftest/cgroup: add cgroup destruction test
Add new test to verify that a cgroup with dead processes can be destroyed. The test spawns a child process which allocates and touches 100MB of RAM to ensure prolonged exit. Subsequently it kills the child, waits until the cgroup containing the child is empty and destroys the cgroup.
--- tools/testing/selftests/cgroup/test_core.c | 113 +++++++++++++++++++++ 1 file changed, 113 insertions(+)
tools/testing/selftests/cgroup/test_core.c +113 -0
@@ -2,7 +2,10 @@#include <linux/limits.h> #include <sys/types.h> +#include <sys/mman.h> +#include <sys/wait.h> #include <unistd.h> +#include <fcntl.h> #include <stdio.h> #include <errno.h> #include <signal.h>
@@ -12,6 +15,115 @@#include "../kselftest.h" #include "cgroup_util.h" +static int touch_anon (char *buf,size_t size) +{ +int fd; +char *pos = buf; + + fd =open ("/dev/urandom" ,O_RDONLY ); +if (fd <0 ) +return -1 ; + +while (size >0 ) { +ssize_t ret =read (fd, pos, size); + +if (ret <0 ) { +if (errno !=EINTR ) { +close (fd); +return -1 ; + } + }else { + pos += ret; + size -= ret; + } + } +close (fd); + +return 0 ; +} + +static int alloc_and_touch_anon_noexit (const char *cgroup,void *arg) +{ +int ppid =getppid (); +size_t size = (size_t )arg; +void *buf; + + buf =mmap (NULL , size,PROT_READ |PROT_WRITE ,MAP_PRIVATE |MAP_ANON , +0 ,0 ); +if (buf ==MAP_FAILED ) +return -1 ; + +if (touch_anon ((char *)buf, size)) { +munmap (buf, size); +return -1 ; + } + +while (getppid () == ppid) +sleep (1 ); + +munmap (buf, size); +return 0 ; +} + +/* +* Create a child process that allocates and touches 100MB, then waits to be +* killed. Wait until the child is attached to the cgroup, kill all processes +* in that cgroup and wait until "cgroup.events" is empty. At this point try to +* destroy the empty cgroup. The test helps detect race conditions between +* dying processes leaving the cgroup and cgroup destruction path. +*/ +static int test_cgcore_destroy (const char *root) +{ +int ret =KSFT_FAIL ; +char *cg_test =NULL ; +int child_pid; +char buf[PAGE_SIZE ]; + + cg_test =cg_name (root,"cg_test" ); + +if (!cg_test) + goto cleanup; + +for (int i =0 ; i <10 ; i++) { +if (cg_create (cg_test)) + goto cleanup; + + child_pid =cg_run_nowait (cg_test, alloc_and_touch_anon_noexit, + (void *)MB (100 )); + +if (child_pid <0 ) + goto cleanup; + +/* wait for the child to enter cgroup */ +if (cg_wait_for_proc_count (cg_test,1 )) + goto cleanup; + +if (cg_killall (cg_test)) + goto cleanup; + +/* wait for cgroup to be empty */ +while (1 ) { +if (cg_read (cg_test,"cgroup.procs" , buf,sizeof (buf))) + goto cleanup; +if (buf[0 ] =='\0' ) +break ; +usleep (1000 ); + } + +if (rmdir (cg_test)) + goto cleanup; + +if (waitpid (child_pid,NULL ,0 ) <0 ) + goto cleanup; + } + ret =KSFT_PASS ; +cleanup: +if (cg_test) +cg_destroy (cg_test); +free (cg_test); +return ret; +} +/* * A(0) - B(0) - C(1) * \ D(0)
@@ -512,6 +624,7 @@ struct corecg_test {T (test_cgcore_populated),T (test_cgcore_proc_migration),T (test_cgcore_thread_migration), + T(test_cgcore_destroy), };#undef T
Signature · 1 line
2.25.0.rc1.283.g88dfdc4193-goog
Michal Koutný
maintainercover cgroup: Iterate tasks that did not finish do_exit()
In reply to Suren BaghdasaryanHi,
I was looking into the issue and came up with an alternative solution that changes task iteration to be consistent with cgroup_is_populated() check and moving the responsibility to check PF_EXITING on the consumers of iterator API.
I haven't check your approach thoroughly, however, it appears to me it complicates (already non-trivial) cgroup destruction path. I ran your selftest on the iterators approach and it proved working.
Michal Koutný (2):
cgroup: Unify css_set task lists
cgroup: Iterate tasks that did not finish do_exit()
Suren Baghdasaryan (1):
kselftest/cgroup: add cgroup destruction test
include/linux/cgroup-defs.h | 15 ++- include/linux/cgroup.h | 4 +- kernel/cgroup/cgroup.c | 86 ++++++++-------- kernel/cgroup/debug.c | 16 ++- tools/testing/selftests/cgroup/test_core.c | 113 +++++++++++++++++++++ 5 files changed, 176 insertions(+), 58 deletions(-)
Signature · 1 line
2.24.1
Michal Koutný
maintainerPATCH 1/3 cgroup: Unify css_set task lists
We track tasks of css_set in three different lists. The iterators unnecessarily process each list specially.
Use an array of lists and simplify the iterator code. This is refactoring with no intended functional change.
--- include/linux/cgroup-defs.h | 15 ++++--- include/linux/cgroup.h | 4 +- kernel/cgroup/cgroup.c | 81 +++++++++++++++++++------------------ kernel/cgroup/debug.c | 16 ++++---- 4 files changed, 60 insertions(+), 56 deletions(-)
include/linux/cgroup-defs.h +10 -5
@@ -180,6 +180,13 @@ struct cgroup_subsys_state {struct cgroup_subsys_state *parent; }; +enum css_set_task_list { +CSS_SET_TASKS =0 , +CSS_SET_TASKS_MG , +CSS_SET_TASKS_DYING , +CSS_SET_TASKS_COUNT , +}; +/* * A css_set is a structure holding pointers to a set of * cgroup_subsys_state objects. This saves space in the task struct
@@ -214,14 +221,12 @@ struct css_set {/* * Lists running through all tasks using this cgroup group. -* mg_tasks lists tasks which belong to this cset but are in the +* CSS_SET_TASKS_MG lists tasks which belong to this cset but are in the * process of being migrated out or in. Protected by * css_set_rwsem, but, during migration, once tasks are moved to -* mg_tasks, it can be read safely while holding cgroup_mutex. +* CSS_SET_TASKS_MG, it can be read safely while holding cgroup_mutex. */ -struct list_head tasks; -struct list_head mg_tasks; -struct list_head dying_tasks; +struct list_head tasks[CSS_SET_TASKS_COUNT ];/* all css_task_iters currently walking this cset */ struct list_head task_iters;
include/linux/cgroup.h +1 -3
@@ -58,10 +58,8 @@ struct css_task_iter {struct list_head *tcset_head;struct list_head *task_pos; -struct list_head *tasks_head; -struct list_head *mg_tasks_head; -struct list_head *dying_tasks_head; +enum css_set_task_list cur_list;struct css_set *cur_cset;struct css_set *cur_dcset;struct task_struct *cur_task;
kernel/cgroup/cgroup.c +42 -39
188 lines not on this page · raw message
kernel/cgroup/debug.c +7 -9
@@ -124,6 +124,7 @@ static int cgroup_css_links_read(struct seq_file *seq, void *v)struct task_struct *task;int count =0 ;int refcnt =refcount_read (&cset->refcount); +enum css_set_task_list ls;/* * Print out the proc_cset and threaded_cset relationship
@@ -161,17 +162,14 @@ static int cgroup_css_links_read(struct seq_file *seq, void *v) }seq_puts (seq,"\n" ); -list_for_each_entry (task, &cset->tasks, cg_list) { -if (count++ <=MAX_TASKS_SHOWN_PER_CSS ) -seq_printf (seq," task %d\n" , -task_pid_vnr (task)); +for (ls =CSS_SET_TASKS ; ls <CSS_SET_TASKS_DYING ; ++ls) { +list_for_each_entry (task, &cset->tasks[ls], cg_list) { +if (count++ <=MAX_TASKS_SHOWN_PER_CSS ) +seq_printf (seq," task %d\n" , +task_pid_vnr (task)); + } } -list_for_each_entry (task, &cset->mg_tasks, cg_list) { -if (count++ <=MAX_TASKS_SHOWN_PER_CSS ) -seq_printf (seq," task %d\n" , -task_pid_vnr (task)); - }/* show # of overflowed tasks */ if (count >MAX_TASKS_SHOWN_PER_CSS ) seq_printf(seq, " ... (%d)\n",
Signature · 1 line
2.24.1
Tejun Heo
maintainerRe: [PATCH 1/3] cgroup: Unify css_set task lists
Quoted 5 lines · Michal Koutný
We track tasks of
css_setin three different lists. The iterators unnecessarily process each list specially.
Use an array of lists and simplify the iterator code. This is refactoring with no intended functional change.
Quoted 7 lines
Signed-off-by Michal Koutný <[email protected]>--- include/linux/cgroup-defs.h | 15 ++++--- include/linux/cgroup.h | 4 +- kernel/cgroup/cgroup.c | 81 +++++++++++++++++++------------------ kernel/cgroup/debug.c | 16 ++++---- 4 files changed, 60 insertions(+), 56 deletions(-)
So, I get the urge to move the lists into an array and thought about doing that while adding the third list for sure. However, it does make code paths which don't walk all lists wordier and the code path is already tricky like hell. Given that there aren't that many places which loop over the lists, if you really wanna clean it up, maybe add an interator over the lists so that both parts of code can look lean?
Thanks.
Signature · 1 line
tejun
Michal Koutný
maintainerPATCH 2/3 cgroup: Iterate tasks that did not finish do_exit()
In reply to Michal KoutnýPF_EXITING is set earlier than actual removal from css_set when a task is exitting. This can confuse cgroup.procs readers who see no PF_EXITING tasks, however, rmdir is checking against css_set membership so it can transitionally fail with EBUSY.
Fix this by listing tasks that weren't unlinked from css_set active lists.
It may happen that other users of the task iterator (without CSS_TASK_ITER_PROCS) spot a PF_EXITING task before cgroup_exit(). This is equal to the state before commit c03cd7738a83 ("cgroup: Include dying leaders with live threads in PROCS iterations") but it may be reviewed later.
--- kernel/cgroup/cgroup.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
kernel/cgroup/cgroup.c +3 -2
@@ -4492,11 +4492,12 @@ static void css_task_iter_advance(struct css_task_iter *it) goto repeat;/* and dying leaders w/o live member threads */ -if (!atomic_read (&task->signal->live)) +if (it->cur_list ==CSS_SET_TASKS_DYING && + !atomic_read (&task->signal->live)) goto repeat; }else {/* skip all dying ones */ -if (task->flags &PF_EXITING ) +if (it->cur_list ==CSS_SET_TASKS_DYING ) goto repeat; } }
Signature · 1 line
2.24.1
Tejun Heo
maintainerRe: [PATCH 2/3] cgroup: Iterate tasks that did not finish do_exit()
Quoted 5 lines · Michal Koutný
PF_EXITINGis set earlier than actual removal fromcss_setwhen a task is exitting. This can confuse cgroup.procs readers who see noPF_EXITINGtasks, however, rmdir is checking againstcss_setmembership so it can transitionally fail with EBUSY.
Quoted 7 lines
Fix this by listing tasks that weren't unlinked from
css_setactive lists.
It may happen that other users of the task iterator (withoutCSS_TASK_ITER_PROCS) spot aPF_EXITINGtask beforecgroup_exit(). This is equal to the state before commitc03cd7738a83("cgroup: Include dying leaders with live threads in PROCS iterations") but it may be reviewed later.
Yeah, this looks fine to me. Any chance you can order this before the clean up so that we can mark it for -stable.
Thanks.
Signature · 1 line
tejun
Suren Baghdasaryan
Tested-byRe: [PATCH 2/3] cgroup: Iterate tasks that did not finish do_exit()
Quoted 7 lines · Tejun Heo
Quoted 5 lines · Michal Koutný
PF_EXITINGis set earlier than actual removal fromcss_setwhen a task is exitting. This can confuse cgroup.procs readers who see noPF_EXITINGtasks, however, rmdir is checking againstcss_setmembership so it can transitionally fail with EBUSY.
Quoted 7 lines
Quoted 7 lines
Fix this by listing tasks that weren't unlinked from
css_setactive lists.
It may happen that other users of the task iterator (withoutCSS_TASK_ITER_PROCS) spot aPF_EXITINGtask beforecgroup_exit(). This is equal to the state before commitc03cd7738a83("cgroup: Include dying leaders with live threads in PROCS iterations") but it may be reviewed later.
Quoted 4 lines
Yeah, this looks fine to me. Any chance you can order this before the clean up so that we can mark it for -stable.
+1 for reordering. Makes it easier to backport.
Thanks,
Suren.
Quoted 8 lines
Thanks.
--
tejun--
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]
Michal Koutný
maintainerRe: [PATCH 2/3] cgroup: Iterate tasks that did not finish do_exit()
Quoted 1 line · Suren Baghdasaryan
Tested-by Suren Baghdasaryan <[email protected]>
Thanks.
Quoted 3 lines
Quoted 2 lines
Yeah, this looks fine to me. Any chance you can order this before the clean up so that we can mark it for -stable.
+1 for reordering. Makes it easier to backport.
The grounds still need to be prepared for css_task_iter to store additional information. Let me see how the preceding changes can be minimized.
Michal
Michal Koutný
maintainerv2 cover cgroup: Iterate tasks that did not finish do_exit()
Hi,
thanks for your feedback. I'm sending updated solution that changes task iteration to be consistent with cgroup_is_populated() check and moving the responsibility to check PF_EXITING on the consumers of iterator API.
Changes from v1
- v1: https://lore.kernel.org/lkml/[email protected]/
- Swap the fixing and refactoring patch (in order to make the fix easier for backport)
- Don't introduce tasks lists array because of unnecessarily long access code
- Fix comment in selftest
- I leave the CC:stable on discretion of the maintainer
Michal Koutný (2):
cgroup: Iterate tasks that did not finish do_exit()
cgroup: Clean up css_set task traversal
Suren Baghdasaryan (1):
kselftest/cgroup: add cgroup destruction test
include/linux/cgroup.h | 4 +- kernel/cgroup/cgroup.c | 60 ++++++----- tools/testing/selftests/cgroup/test_core.c | 113 +++++++++++++++++++++ 3 files changed, 146 insertions(+), 31 deletions(-)
Signature · 7 lines
2.24.1 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Michal Koutný
maintainerPATCH v2 1/3 cgroup: Iterate tasks that did not finish do_exit()
PF_EXITING is set earlier than actual removal from css_set when a task is exitting. This can confuse cgroup.procs readers who see no PF_EXITING tasks, however, rmdir is checking against css_set membership so it can transitionally fail with EBUSY.
Fix this by listing tasks that weren't unlinked from css_set active lists.
It may happen that other users of the task iterator (without CSS_TASK_ITER_PROCS) spot a PF_EXITING task before cgroup_exit(). This is equal to the state before commit c03cd7738a83 ("cgroup: Include dying leaders with live threads in PROCS iterations") but it may be reviewed later.
--- include/linux/cgroup.h | 1 + kernel/cgroup/cgroup.c | 23 ++++++++++++++++------- 2 files changed, 17 insertions(+), 7 deletions(-)
include/linux/cgroup.h +1 -0
@@ -62,6 +62,7 @@ struct css_task_iter {struct list_head *mg_tasks_head;struct list_head *dying_tasks_head; +struct list_head *cur_tasks_head;struct css_set *cur_cset;struct css_set *cur_dcset;struct task_struct *cur_task;
kernel/cgroup/cgroup.c +16 -7
@@ -4404,12 +4404,16 @@ static void css_task_iter_advance_css_set(struct css_task_iter *it) } }while (!css_set_populated (cset) &&list_empty (&cset->dying_tasks)); -if (!list_empty (&cset->tasks)) +if (!list_empty (&cset->tasks)) { it->task_pos = cset->tasks.next; -else if (!list_empty (&cset->mg_tasks)) + it->cur_tasks_head = &cset->tasks; + }else if (!list_empty (&cset->mg_tasks)) { it->task_pos = cset->mg_tasks.next; -else + it->cur_tasks_head = &cset->mg_tasks; + }else { it->task_pos = cset->dying_tasks.next; + it->cur_tasks_head = &cset->dying_tasks; + } it->tasks_head = &cset->tasks; it->mg_tasks_head = &cset->mg_tasks;
@@ -4467,10 +4471,14 @@ static void css_task_iter_advance(struct css_task_iter *it)else it->task_pos = it->task_pos->next; -if (it->task_pos == it->tasks_head) +if (it->task_pos == it->tasks_head) { it->task_pos = it->mg_tasks_head->next; -if (it->task_pos == it->mg_tasks_head) + it->cur_tasks_head = it->mg_tasks_head; + } +if (it->task_pos == it->mg_tasks_head) { it->task_pos = it->dying_tasks_head->next; + it->cur_tasks_head = it->dying_tasks_head; + }if (it->task_pos == it->dying_tasks_head)css_task_iter_advance_css_set (it); }else {
@@ -4489,11 +4497,12 @@ static void css_task_iter_advance(struct css_task_iter *it) goto repeat;/* and dying leaders w/o live member threads */ -if (!atomic_read (&task->signal->live)) +if (it->cur_tasks_head == it->dying_tasks_head && + !atomic_read (&task->signal->live)) goto repeat; }else {/* skip all dying ones */ -if (task->flags &PF_EXITING ) +if (it->cur_tasks_head == it->dying_tasks_head) goto repeat; } }
Signature · 1 line
2.24.1
Re: [PATCH v2 1/3] cgroup: Iterate tasks that did not finish do_exit()
Quoted 91 lines · Michal Koutný
PF_EXITINGis set earlier than actual removal fromcss_setwhen a task is exitting. This can confuse cgroup.procs readers who see noPF_EXITINGtasks, however, rmdir is checking againstcss_setmembership so it can transitionally fail with EBUSY.Fix this by listing tasks that weren't unlinked from
css_setactive lists.
It may happen that other users of the task iterator (withoutCSS_TASK_ITER_PROCS) spot aPF_EXITINGtask beforecgroup_exit(). This is equal to the state before commitc03cd7738a83("cgroup: Include dying leaders with live threads in PROCS iterations") but it may be reviewed later.Reported-by Suren Baghdasaryan <[email protected]>Fixes c03cd7738a83 ("cgroup: Include dying leaders with live threads in PROCS iterations")Signed-off-by Michal Koutný <[email protected]>--- include/linux/cgroup.h | 1 + kernel/cgroup/cgroup.c | 23 ++++++++++++++++------- 2 files changed, 17 insertions(+), 7 deletions(-)include/linux/cgroup.h +1
-0@@ -62,6 +62,7 @@ struct css_task_iter {struct list_head *mg_tasks_head;struct list_head *dying_tasks_head; +struct list_head *cur_tasks_head;struct css_set *cur_cset;struct css_set *cur_dcset;struct task_struct *cur_task;kernel/cgroup/cgroup.c +16
-8@@ -4404,12 +4404,16 @@ static void css_task_iter_advance_css_set(struct css_task_iter *it) } }while (!css_set_populated (cset) &&list_empty (&cset->dying_tasks)); -if (!list_empty (&cset->tasks)) +if (!list_empty (&cset->tasks)) { it->task_pos = cset->tasks.next; -else if (!list_empty (&cset->mg_tasks)) + it->cur_tasks_head = &cset->tasks; + }else if (!list_empty (&cset->mg_tasks)) { it->task_pos = cset->mg_tasks.next; -else + it->cur_tasks_head = &cset->mg_tasks; + }else { it->task_pos = cset->dying_tasks.next; + it->cur_tasks_head = &cset->dying_tasks; + } it->tasks_head = &cset->tasks; it->mg_tasks_head = &cset->mg_tasks;@@ -4467,10 +4471,14 @@ static void css_task_iter_advance(struct css_task_iter *it)else it->task_pos = it->task_pos->next; -if (it->task_pos == it->tasks_head) +if (it->task_pos == it->tasks_head) { it->task_pos = it->mg_tasks_head->next; -if (it->task_pos == it->mg_tasks_head) + it->cur_tasks_head = it->mg_tasks_head; + } +if (it->task_pos == it->mg_tasks_head) { it->task_pos = it->dying_tasks_head->next; + it->cur_tasks_head = it->dying_tasks_head; + }if (it->task_pos == it->dying_tasks_head)css_task_iter_advance_css_set (it); }else {@@ -4489,11 +4497,12 @@ static void css_task_iter_advance(struct css_task_iter *it) goto repeat;/* and dying leaders w/o live member threads */ -if (!atomic_read (&task->signal->live)) +if (it->cur_tasks_head == it->dying_tasks_head && + !atomic_read (&task->signal->live)) goto repeat; }else {/* skip all dying ones */ -if (task->flags &PF_EXITING ) +if (it->cur_tasks_head == it->dying_tasks_head) goto repeat; } } --
Quoted 2 lines
2.24.1
Thanks!
Re: [PATCH v2 1/3] cgroup: Iterate tasks that did not finish do_exit()
Quoted 98 lines · Suren Baghdasaryan
Quoted 91 lines · Michal Koutný
PF_EXITINGis set earlier than actual removal fromcss_setwhen a task is exitting. This can confuse cgroup.procs readers who see noPF_EXITINGtasks, however, rmdir is checking againstcss_setmembership so it can transitionally fail with EBUSY.Fix this by listing tasks that weren't unlinked from
css_setactive lists.
It may happen that other users of the task iterator (withoutCSS_TASK_ITER_PROCS) spot aPF_EXITINGtask beforecgroup_exit(). This is equal to the state before commitc03cd7738a83("cgroup: Include dying leaders with live threads in PROCS iterations") but it may be reviewed later.Reported-by Suren Baghdasaryan <[email protected]>Fixes c03cd7738a83 ("cgroup: Include dying leaders with live threads in PROCS iterations")Signed-off-by Michal Koutný <mkoutny-IBi9RG/[email protected]>--- include/linux/cgroup.h | 1 + kernel/cgroup/cgroup.c | 23 ++++++++++++++++------- 2 files changed, 17 insertions(+), 7 deletions(-)include/linux/cgroup.h +1
-0@@ -62,6 +62,7 @@ struct css_task_iter {struct list_head *mg_tasks_head;struct list_head *dying_tasks_head; +struct list_head *cur_tasks_head;struct css_set *cur_cset;struct css_set *cur_dcset;struct task_struct *cur_task;kernel/cgroup/cgroup.c +16
-8@@ -4404,12 +4404,16 @@ static void css_task_iter_advance_css_set(struct css_task_iter *it) } }while (!css_set_populated (cset) &&list_empty (&cset->dying_tasks)); -if (!list_empty (&cset->tasks)) +if (!list_empty (&cset->tasks)) { it->task_pos = cset->tasks.next; -else if (!list_empty (&cset->mg_tasks)) + it->cur_tasks_head = &cset->tasks; + }else if (!list_empty (&cset->mg_tasks)) { it->task_pos = cset->mg_tasks.next; -else + it->cur_tasks_head = &cset->mg_tasks; + }else { it->task_pos = cset->dying_tasks.next; + it->cur_tasks_head = &cset->dying_tasks; + } it->tasks_head = &cset->tasks; it->mg_tasks_head = &cset->mg_tasks;@@ -4467,10 +4471,14 @@ static void css_task_iter_advance(struct css_task_iter *it)else it->task_pos = it->task_pos->next; -if (it->task_pos == it->tasks_head) +if (it->task_pos == it->tasks_head) { it->task_pos = it->mg_tasks_head->next; -if (it->task_pos == it->mg_tasks_head) + it->cur_tasks_head = it->mg_tasks_head; + } +if (it->task_pos == it->mg_tasks_head) { it->task_pos = it->dying_tasks_head->next; + it->cur_tasks_head = it->dying_tasks_head; + }if (it->task_pos == it->dying_tasks_head)css_task_iter_advance_css_set (it); }else {@@ -4489,11 +4497,12 @@ static void css_task_iter_advance(struct css_task_iter *it) goto repeat;/* and dying leaders w/o live member threads */ -if (!atomic_read (&task->signal->live)) +if (it->cur_tasks_head == it->dying_tasks_head && + !atomic_read (&task->signal->live)) goto repeat; }else {/* skip all dying ones */ -if (task->flags &PF_EXITING ) +if (it->cur_tasks_head == it->dying_tasks_head) goto repeat; } } --Quoted 2 lines
2.24.1
Tested-by Suren Baghdasaryan <[email protected]>
Quoted 1 line
Thanks!
Hi Folks,
If this new version looks good could we get an Ack please? I need to start backporting this fix to Android and would like a confirmation before doing so.
Thanks!
Re: [PATCH v2 1/3] cgroup: Iterate tasks that did not finish do_exit()
In reply to Michal KoutnýQuoted 13 lines · Michal Koutný
PF_EXITINGis set earlier than actual removal fromcss_setwhen a task is exitting. This can confuse cgroup.procs readers who see noPF_EXITINGtasks, however, rmdir is checking againstcss_setmembership so it can transitionally fail with EBUSY.Fix this by listing tasks that weren't unlinked from
css_setactive lists.
It may happen that other users of the task iterator (withoutCSS_TASK_ITER_PROCS) spot aPF_EXITINGtask beforecgroup_exit(). This is equal to the state before commitc03cd7738a83("cgroup: Include dying leaders with live threads in PROCS iterations") but it may be reviewed later.
Quoted 3 lines
Reported-by Suren Baghdasaryan <[email protected]>Fixes c03cd7738a83 ("cgroup: Include dying leaders with live threads in PROCS iterations")Signed-off-by Michal Koutný <mkoutny-IBi9RG/[email protected]>
Applied to cgroup/for-5.6-fixes.
Thanks.
Signature · 1 line
tejun
Michal Koutný
maintainerPATCH v2 2/3 cgroup: Clean up css_set task traversal
In reply to Michal Koutnýcss_task_iter stores pointer to head of each iterable list, this dates back to commit 0f0a2b4fa621 ("cgroup: reorganize css_task_iter") when we did not store cur_cset. Let us utilize list heads directly in cur_cset and streamline css_task_iter_advance_css_set a bit. This is no intentional function change.
--- include/linux/cgroup.h | 3 --- kernel/cgroup/cgroup.c | 61 +++++++++++++++++++----------------------- 2 files changed, 28 insertions(+), 36 deletions(-)
include/linux/cgroup.h +0 -3
@@ -58,9 +58,6 @@ struct css_task_iter {struct list_head *tcset_head;struct list_head *task_pos; -struct list_head *tasks_head; -struct list_head *mg_tasks_head; -struct list_head *dying_tasks_head;struct list_head *cur_tasks_head;struct css_set *cur_cset;
kernel/cgroup/cgroup.c +28 -33
93 lines not on this page · raw message
Signature · 1 line
2.24.1
Michal Koutný
maintainerPATCH v2 3/3 kselftest/cgroup: add cgroup destruction test
In reply to Michal KoutnýFrom: Suren Baghdasaryan <[email protected]>
Add new test to verify that a cgroup with dead processes can be destroyed. The test spawns a child process which allocates and touches 100MB of RAM to ensure prolonged exit. Subsequently it kills the child, waits until the cgroup containing the child is empty and destroys the cgroup.
--- tools/testing/selftests/cgroup/test_core.c | 113 +++++++++++++++++++++ 1 file changed, 113 insertions(+)
tools/testing/selftests/cgroup/test_core.c +113 -0
@@ -2,7 +2,10 @@#include <linux/limits.h> #include <sys/types.h> +#include <sys/mman.h> +#include <sys/wait.h> #include <unistd.h> +#include <fcntl.h> #include <stdio.h> #include <errno.h> #include <signal.h>
@@ -12,6 +15,115 @@#include "../kselftest.h" #include "cgroup_util.h" +static int touch_anon (char *buf,size_t size) +{ +int fd; +char *pos = buf; + + fd =open ("/dev/urandom" ,O_RDONLY ); +if (fd <0 ) +return -1 ; + +while (size >0 ) { +ssize_t ret =read (fd, pos, size); + +if (ret <0 ) { +if (errno !=EINTR ) { +close (fd); +return -1 ; + } + }else { + pos += ret; + size -= ret; + } + } +close (fd); + +return 0 ; +} + +static int alloc_and_touch_anon_noexit (const char *cgroup,void *arg) +{ +int ppid =getppid (); +size_t size = (size_t )arg; +void *buf; + + buf =mmap (NULL , size,PROT_READ |PROT_WRITE ,MAP_PRIVATE |MAP_ANON , +0 ,0 ); +if (buf ==MAP_FAILED ) +return -1 ; + +if (touch_anon ((char *)buf, size)) { +munmap (buf, size); +return -1 ; + } + +while (getppid () == ppid) +sleep (1 ); + +munmap (buf, size); +return 0 ; +} + +/* +* Create a child process that allocates and touches 100MB, then waits to be +* killed. Wait until the child is attached to the cgroup, kill all processes +* in that cgroup and wait until "cgroup.procs" is empty. At this point try to +* destroy the empty cgroup. The test helps detect race conditions between +* dying processes leaving the cgroup and cgroup destruction path. +*/ +static int test_cgcore_destroy (const char *root) +{ +int ret =KSFT_FAIL ; +char *cg_test =NULL ; +int child_pid; +char buf[PAGE_SIZE ]; + + cg_test =cg_name (root,"cg_test" ); + +if (!cg_test) + goto cleanup; + +for (int i =0 ; i <10 ; i++) { +if (cg_create (cg_test)) + goto cleanup; + + child_pid =cg_run_nowait (cg_test, alloc_and_touch_anon_noexit, + (void *)MB (100 )); + +if (child_pid <0 ) + goto cleanup; + +/* wait for the child to enter cgroup */ +if (cg_wait_for_proc_count (cg_test,1 )) + goto cleanup; + +if (cg_killall (cg_test)) + goto cleanup; + +/* wait for cgroup to be empty */ +while (1 ) { +if (cg_read (cg_test,"cgroup.procs" , buf,sizeof (buf))) + goto cleanup; +if (buf[0 ] =='\0' ) +break ; +usleep (1000 ); + } + +if (rmdir (cg_test)) + goto cleanup; + +if (waitpid (child_pid,NULL ,0 ) <0 ) + goto cleanup; + } + ret =KSFT_PASS ; +cleanup: +if (cg_test) +cg_destroy (cg_test); +free (cg_test); +return ret; +} +/* * A(0) - B(0) - C(1) * \ D(0)
@@ -512,6 +624,7 @@ struct corecg_test {T (test_cgcore_populated),T (test_cgcore_proc_migration),T (test_cgcore_thread_migration), + T(test_cgcore_destroy), };#undef T
Signature · 1 line
2.24.1
Tejun Heo
maintainerRe: [PATCH v2 3/3] kselftest/cgroup: add cgroup destruction test
Quoted 11 lines · Michal Koutný
From: Suren Baghdasaryan <[email protected]>
Add new test to verify that a cgroup with dead processes can be destroyed. The test spawns a child process which allocates and touches 100MB of RAM to ensure prolonged exit. Subsequently it kills the child, waits until the cgroup containing the child is empty and destroys the cgroup.
Signed-off-by Suren Baghdasaryan <[email protected]>[mkoutny-IBi9RG/[email protected]: Fix typo in test_cgcore_destroy comment]Signed-off-by Michal Koutný <mkoutny-IBi9RG/[email protected]>
Applied 2-3 to cgroup/for-5.7.
Thanks a lot for working on this!
Signature · 1 line
tejun
Michal Koutný
maintainerPATCH 3/3 kselftest/cgroup: add cgroup destruction test
In reply to Michal KoutnýFrom: Suren Baghdasaryan <[email protected]>
Add new test to verify that a cgroup with dead processes can be destroyed. The test spawns a child process which allocates and touches 100MB of RAM to ensure prolonged exit. Subsequently it kills the child, waits until the cgroup containing the child is empty and destroys the cgroup.
--- tools/testing/selftests/cgroup/test_core.c | 113 +++++++++++++++++++++ 1 file changed, 113 insertions(+)
tools/testing/selftests/cgroup/test_core.c +113 -0
@@ -2,7 +2,10 @@#include <linux/limits.h> #include <sys/types.h> +#include <sys/mman.h> +#include <sys/wait.h> #include <unistd.h> +#include <fcntl.h> #include <stdio.h> #include <errno.h> #include <signal.h>
@@ -12,6 +15,115 @@#include "../kselftest.h" #include "cgroup_util.h" +static int touch_anon (char *buf,size_t size) +{ +int fd; +char *pos = buf; + + fd =open ("/dev/urandom" ,O_RDONLY ); +if (fd <0 ) +return -1 ; + +while (size >0 ) { +ssize_t ret =read (fd, pos, size); + +if (ret <0 ) { +if (errno !=EINTR ) { +close (fd); +return -1 ; + } + }else { + pos += ret; + size -= ret; + } + } +close (fd); + +return 0 ; +} + +static int alloc_and_touch_anon_noexit (const char *cgroup,void *arg) +{ +int ppid =getppid (); +size_t size = (size_t )arg; +void *buf; + + buf =mmap (NULL , size,PROT_READ |PROT_WRITE ,MAP_PRIVATE |MAP_ANON , +0 ,0 ); +if (buf ==MAP_FAILED ) +return -1 ; + +if (touch_anon ((char *)buf, size)) { +munmap (buf, size); +return -1 ; + } + +while (getppid () == ppid) +sleep (1 ); + +munmap (buf, size); +return 0 ; +} + +/* +* Create a child process that allocates and touches 100MB, then waits to be +* killed. Wait until the child is attached to the cgroup, kill all processes +* in that cgroup and wait until "cgroup.events" is empty. At this point try to +* destroy the empty cgroup. The test helps detect race conditions between +* dying processes leaving the cgroup and cgroup destruction path. +*/ +static int test_cgcore_destroy (const char *root) +{ +int ret =KSFT_FAIL ; +char *cg_test =NULL ; +int child_pid; +char buf[PAGE_SIZE ]; + + cg_test =cg_name (root,"cg_test" ); + +if (!cg_test) + goto cleanup; + +for (int i =0 ; i <10 ; i++) { +if (cg_create (cg_test)) + goto cleanup; + + child_pid =cg_run_nowait (cg_test, alloc_and_touch_anon_noexit, + (void *)MB (100 )); + +if (child_pid <0 ) + goto cleanup; + +/* wait for the child to enter cgroup */ +if (cg_wait_for_proc_count (cg_test,1 )) + goto cleanup; + +if (cg_killall (cg_test)) + goto cleanup; + +/* wait for cgroup to be empty */ +while (1 ) { +if (cg_read (cg_test,"cgroup.procs" , buf,sizeof (buf))) + goto cleanup; +if (buf[0 ] =='\0' ) +break ; +usleep (1000 ); + } + +if (rmdir (cg_test)) + goto cleanup; + +if (waitpid (child_pid,NULL ,0 ) <0 ) + goto cleanup; + } + ret =KSFT_PASS ; +cleanup: +if (cg_test) +cg_destroy (cg_test); +free (cg_test); +return ret; +} +/* * A(0) - B(0) - C(1) * \ D(0)
@@ -512,6 +624,7 @@ struct corecg_test {T (test_cgcore_populated),T (test_cgcore_proc_migration),T (test_cgcore_thread_migration), + T(test_cgcore_destroy), };#undef T
Signature · 1 line
2.24.1
Re: [PATCH 0/3] cgroup: Iterate tasks that did not finish do_exit()
In reply to Michal KoutnýHi Michal,
Quoted 5 lines · Michal Koutný
Hi,
I was looking into the issue and came up with an alternative solution that changes task iteration to be consistent withcgroup_is_populated()check and moving the responsibility to checkPF_EXITINGon the consumers of iterator API.
Yeah, that was my first thought which basically reverts a part of c03cd7738a83. When I first brought up this issue in the other email thread, Tejun's comment was "the right thing to do is allowing destruction of cgroups w/ only
dead processes in it". I assumed, maybe incorrectly, that the desire here is not to include dying processes into cgroup.procs but to allow cgroups with dying processes to be deleted.
To be clear, either way is fine with me since both ways solve the issue and this way the code is definitely simpler. I'll rerun the tests with your patches just to confirm the issue is gone. Thanks!
Quoted 25 lines
I haven't check your approach thoroughly, however, it appears to me it complicates (already non-trivial) cgroup destruction path. I ran your selftest on the iterators approach and it proved working.
Michal Koutný (2):
cgroup: Unifycss_settask lists
cgroup: Iterate tasks that did not finishdo_exit()Suren Baghdasaryan (1):
kselftest/cgroup: add cgroup destruction testinclude/linux/cgroup-defs.h | 15 ++- include/linux/cgroup.h | 4 +- kernel/cgroup/cgroup.c | 86 ++++++++-------- kernel/cgroup/debug.c | 16 ++- tools/testing/selftests/cgroup/test_core.c | 113 +++++++++++++++++++++ 5 files changed, 176 insertions(+), 58 deletions(-)--
2.24.1--
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].