cgroup attach task - slogging cpu
Hi Oleg, All
I saw an issue of CPU slogging posted in
https://lkml.org/lkml/2013/8/28/283, and require your valuable suggestion on a very similar issue I am facing.
We are facing the issue of cpu slogging in the function
cgroup_attach_task(struct cgroup *cgrp, struct task_struct *tsk,
bool threadgroup)
in the do-while loop which iterates over the threadgroup.
rcu_read_lock();
do {
struct task_and_cgroup ent;
/* @tsk either already exited or can't exit until the end */ if (tsk->flags & PF_EXITING)
continue;
/* as per above, nr_threads may decrease, but not increase. */ BUG_ON(i >= group_size);
ent.task = tsk;
ent.cgrp = task_cgroup_from_root(tsk, root);
/* nothing to do if this task is already in the cgroup */ if (ent.cgrp == cgrp)
continue;
/*
* saying GFP_ATOMIC has no effect here because we did prealloc
* earlier, but it's good form to communicate our expectations. */
retval = flex_array_put(group, i, &ent, GFP_ATOMIC); BUG_ON(retval != 0);
i++;
if (!threadgroup)
break;
} while_each_thread(leader, tsk);
Problem Observed:
In this loop, in case of a single thread, and argument “bool threadgroup” as “false” and
if(ent.cgrp == cgrp) is true
we will continue to the next thread instead of breaking out of the loop.
Possible Solution and Doubt:
When a break condition was added as shown in the patch attached, the cpu sluggishness was not occurring.
Can you please provide your suggestions, if this is the right way to fix the above mentioned issue.
Also, if a fix is already in for this, can you please guide me to that.
Thanks and Regards
Anjana
kernel/cgroup.c +3 -2
@@ -2002,7 +2002,9 @@ static int cgroup_attach_task(struct cgroup *cgrp, struct task_struct *tsk, ent.task = tsk; ent.cgrp =task_cgroup_from_root (tsk, root);/* nothing to do if this task is already in the cgroup */ -if (ent.cgrp == cgrp) +if (ent.cgrp == cgrp && !threadgroup) +break ; +else if (ent.cgrp == cgrp)continue ;/* * saying GFP_ATOMIC has no effect here because we did prealloc
@@ -2199,7 +2201,6 @@ retry_find_task: ret =cgroup_attach_task (cgrp, tsk, threadgroup);threadgroup_unlock (tsk); -put_task_struct (tsk); out_unlock_cgroup:mutex_unlock (&cgroup_mutex);
A message that is not in the archive (sent off-list or not yet received): CALPf4Tz+Gf_Q7wKKBufCc1mtV1qVPVrOW0S1qhHxfOv6pJa2Kg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org
Hello Anjana,
Quoted 4 lines · anjana vk
I saw an issue of CPU slogging posted in
https://lkml.org/lkml/2013/8/28/283, and require your valuable suggestion on a very similar issue I am facing.
Not sure I understand, but just in case: yes the lockless while_each_thread() is buggy and should be fixed (it should actually die eventually). And a lot of current users of while_each_thread() are themselves buggy and need the fixes no matter what we do with while_each_thread.
Oh. and just in case... I am (slowly) working on this, but didn't finish it yet, sorry.
But I do not really understand cgroup_attach_task(), and I am not sure you observed the same problem. Add Tejun.
Quoted 32 lines
We are facing the issue of cpu slogging in the function
cgroup_attach_task(struct cgroup *cgrp, structtask_struct*tsk,
bool threadgroup)
in the do-while loop which iterates over the threadgroup.
rcu_read_lock();
do {
structtask_and_cgroupent;/* @tsk either already exited or can't exit until the end */ if (
tsk->flags&PF_EXITING)
continue;/* as per above,
nr_threadsmay decrease, but not increase. */BUG_ON(i >= group_size);
ent.task = tsk;
ent.cgrp =task_cgroup_from_root(tsk, root);
/* nothing to do if this task is already in the cgroup */ if (ent.cgrp == cgrp)
continue;
/*
* sayingGFP_ATOMIChas no effect here because we did prealloc
* earlier, but it's good form to communicate our expectations. */
retval =flex_array_put(group, i, &ent, GFP_ATOMIC);BUG_ON(retval != 0);
i++;if (!threadgroup)
break;
}while_each_thread(leader, tsk);
Quoted 5 lines
Problem Observed:
In this loop, in case of a single thread, and argument “bool threadgroup” as “false” and
if(ent.cgrp == cgrp) is true
we will continue to the next thread instead of breaking out of the loop.
But in this case the loop should be terminated by while_each_thread's check?
Again, again, while_each_thread() is wrong, and we need
--- x/kernel/cgroup.c
+++ x/kernel/cgroup.c
@@ -2034,7 +2034,7 @@ static int cgroup_attach_task(struct cgr
* take an rcu_read_lock.
*/
rcu_read_lock();
- do {
+ for_each_thread(leader->signal, task) {
struct task_and_cgroup ent;
/* @tsk either already exited or can't exit until the end */
@@ -2058,7 +2058,7 @@ static int cgroup_attach_task(struct cgr
if (!threadgroup)
break;
- } while_each_thread(leader, tsk);
+ }
rcu_read_unlock();
/* remember the number of threads in the array for later. */
group_size = i;after we add for_each_thread/etc. But I am not sure we need another "threadgroup" check.
Quoted 34 lines
Possible Solution and Doubt:
When a break condition was added as shown in the patch attached, the cpu sluggishness was not occurring.
Can you please provide your suggestions, if this is the right way to fix the above mentioned issue.
Also, if a fix is already in for this, can you please guide me to that.Thanks and Regards
Anjanakernel/cgroup.c +3
-2@@ -2002,7 +2002,9 @@ static int cgroup_attach_task(struct cgroup *cgrp, struct task_struct *tsk, ent.task = tsk; ent.cgrp =task_cgroup_from_root (tsk, root);/* nothing to do if this task is already in the cgroup */ -if (ent.cgrp == cgrp) +if (ent.cgrp == cgrp && !threadgroup) +break ; +else if (ent.cgrp == cgrp)continue ;/* * saying GFP_ATOMIC has no effect here because we did prealloc @@ -2199,7 +2201,6 @@ retry_find_task: ret =cgroup_attach_task (cgrp, tsk, threadgroup);threadgroup_unlock (tsk); -put_task_struct (tsk); out_unlock_cgroup:mutex_unlock (&cgroup_mutex);
A message that is not in the archive (sent off-list or not yet received): [email protected]
Hey,
Quoted 8 lines · Oleg Nesterov
Not sure I understand, but just in case: yes the lockless
while_each_thread()is buggy and should be fixed (it should actually die eventually). And a lot of current users ofwhile_each_thread()are themselves buggy and need the fixes no matter what we do withwhile_each_thread.Oh. and just in case... I am (slowly) working on this, but didn't finish it yet, sorry.
Is there anything I can do to make the hang go away in the meantime?
Quoted 2 lines
But I do not really understand
cgroup_attach_task(), and I am not sure you observed the same problem. Add Tejun.
Sounds like the same problem to me.
Quoted 23 lines
--- x/kernel/cgroup.c +++ x/kernel/cgroup.c @@ -2034,7 +2034,7 @@ static int cgroup_attach_task(struct cgr * take an rcu_read_lock. */ rcu_read_lock(); - do { + for_each_thread(leader->signal, task) { struct task_and_cgroup ent; /* @tsk either already exited or can't exit until the end */ @@ -2058,7 +2058,7 @@ static int cgroup_attach_task(struct cgr if (!threadgroup) break; - } while_each_thread(leader, tsk); + } rcu_read_unlock(); /* remember the number of threads in the array for later. */ group_size = i;after we add for_each_thread/etc. But I am not sure we need another "threadgroup" check.
...
Quoted 5 lines
Quoted 5 lines
kernel/cgroup.c +0
-0@@ -2002,7 +2002,9 @@ static int cgroup_attach_task(struct cgroup *cgrp, struct task_struct *tsk,
Quoted 8 lines
kernel/cgroup.c static int cgroup_attach_task(struct cgroup *cgrp, struct task_struct *tsk, · line 2008
Quoted 8 lines
kernel/cgroup.c +3
-1ent.task = tsk; ent.cgrp =task_cgroup_from_root (tsk, root);/* nothing to do if this task is already in the cgroup */ -if (ent.cgrp == cgrp) +if (ent.cgrp == cgrp && !threadgroup) +break ; +else if (ent.cgrp == cgrp)continue ;
I have no idea know how this would make any difference. Hmm...
Thanks.
Signature · 1 line
tejun
A message that is not in the archive (sent off-list or not yet received): 20131007184507.GD27396-Gd/HAXX7CRxy/[email protected]
Quoted 10 lines · Tejun Heo
Hey,
Quoted 6 lines · Oleg Nesterov
Not sure I understand, but just in case: yes the lockless
while_each_thread()is buggy and should be fixed (it should actually die eventually). And a lot of current users ofwhile_each_thread()are themselves buggy and need the fixes no matter what we do withwhile_each_thread.
Quoted 4 lines
Quoted 2 lines
Oh. and just in case... I am (slowly) working on this, but didn't finish it yet, sorry.
Is there anything I can do to make the hang go away in the meantime?
well, probably tasklist, or pid_alive() check... note that this code rcu/while_each_thread() logic looks wrong even if while_each_thread() was fine. Will try to makes a patch today.
But! Anjana! I am so sorry!! I simply can't understand how I managed to misunderstand (and confuse!) you.
Tejun, could you please look at the patch from Anjana again?
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
kernel/cgroup.c +0
-0@@ -2002,7 +2002,9 @@ static int cgroup_attach_task(struct cgroup *cgrp, struct task_struct *tsk, ent.task = tsk; ent.cgrp =task_cgroup_from_root (tsk, root);
Quoted 8 lines
Quoted 6 lines
kernel/cgroup.c static int cgroup_attach_task(struct cgroup *cgrp, struct task_struct *tsk, · line 2008
Quoted 6 lines
kernel/cgroup.c +3
-1/* nothing to do if this task is already in the cgroup */ -if (ent.cgrp == cgrp) +if (ent.cgrp == cgrp && !threadgroup) +break ; +else if (ent.cgrp == cgrp)continue ;I have no idea know how this would make any difference. Hmm...
I guess the patch needs a cleanup, but this code looks indeed wrong even if we forget about rcu/while_each_thread? Really, we should not "continue" if threadgroup == T, in this case we miss the last
if (!threadgroup) break;
check in the main loop. So Anjana was right (sorry again!), and we should probably do
ent.cgrp = task_cgroup_from_root(...);
if (ent.cgrp != cgrp) {
retval = flex_array_put(...);
...
}
if (!threadgroup)
break;Or I am wrong again?
Oleg.
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 14 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
kernel/cgroup.c +0
-0@@ -2002,7 +2002,9 @@ static int cgroup_attach_task(struct cgroup *cgrp, struct task_struct *tsk, ent.task = tsk; ent.cgrp =task_cgroup_from_root (tsk, root);Quoted 8 lines
Quoted 6 lines
kernel/cgroup.c static int cgroup_attach_task(struct cgroup *cgrp, struct task_struct *tsk, · line 2008
Quoted 6 lines
kernel/cgroup.c +3
-1/* nothing to do if this task is already in the cgroup */ -if (ent.cgrp == cgrp) +if (ent.cgrp == cgrp && !threadgroup) +break ; +else if (ent.cgrp == cgrp)continue ;I have no idea know how this would make any difference. Hmm...
Quoted 7 lines
I guess the patch needs a cleanup, but this code looks indeed wrong even if we forget about rcu/while_each_thread? Really, we should not "continue" if threadgroup == T, in this case we miss the last
if (!threadgroup) break;
Yes, this is wrong.
lxc34:/mnt/tmp # echo 5207 > tasks
lxc34:/mnt/tmp # cat tasks
5207
lxc34:/mnt/tmp # echo 5207 > tasks
lxc34:/mnt/tmp # cat tasks
5207
5215
lxc34:/mnt/tmp # echo 5207 > tasks
lxc34:/mnt/tmp # cat tasks
5207
5215
5216
I wanted to attach 5207 to the cgroup, and in this case it should be a no-op because the thread was already in this cgroup, but the result was another thread got attached to it.
Anjana, could you revise the patch and send it out with proper changelog
and Signed-off-by? And please add "Cc: <[email protected]> # 3.9+"
The cpu slogging issue is another bug that we're working on.
Quoted 12 lines
check in the main loop. So Anjana was right (sorry again!), and we should probably do
ent.cgrp = task_cgroup_from_root(...); if (ent.cgrp != cgrp) { retval = flex_array_put(...); ... } if (!threadgroup) break;
Or
do {
...
if (ent.cgrp == cgrp)
goto next;
...
next:
if (!threadgroup)
break;
} while (...);Quoted 2 lines
Or I am wrong again?
No, you are not! :)
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 3 lines · Li Zefan
Anjana, could you revise the patch and send it out with proper changelog
and Signed-off-by? And please add "Cc: <[email protected]> # 3.9+"
Yes, Anjana, please!
Quoted 13 lines
Quoted 12 lines
check in the main loop. So Anjana was right (sorry again!), and we should probably do
ent.cgrp = task_cgroup_from_root(...); if (ent.cgrp != cgrp) { retval = flex_array_put(...); ... } if (!threadgroup) break;
Quoted 6 lines
Or
do { ... if (ent.cgrp == cgrp) goto next;
Or this, agreed.
Quoted 3 lines
Quoted 1 line
Or I am wrong again?
No, you are not! :)
Thanks ;)
Oleg.
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 7 lines · Oleg Nesterov
Quoted 3 lines · Li Zefan
Anjana, could you revise the patch and send it out with proper changelog
and Signed-off-by? And please add "Cc: <[email protected]> # 3.9+"Yes, Anjana, please!
Please note also that the PF_EXITING check has the same problem, it also needs "goto next".
Quoted 29 lines
Quoted 13 lines
Quoted 12 lines
check in the main loop. So Anjana was right (sorry again!), and we should probably do
ent.cgrp = task_cgroup_from_root(...); if (ent.cgrp != cgrp) { retval = flex_array_put(...); ... } if (!threadgroup) break;Quoted 6 lines
Or
do { ... if (ent.cgrp == cgrp) goto next;Or this, agreed.
Quoted 3 lines
Quoted 1 line
Or I am wrong again?
No, you are not! :)
Thanks ;)
Oleg.
cgroup_attach_task && while_each_thread (Was: cgroup attach task - slogging cpu)
And I am starting to think that this change should also fix the while_each_thread() problems in this particular case.
In generak the code like
rcu_read_lock();
task = find_get_task(...);
rcu_read_unlock();
rcu_read_lock();
t = task;
do {
...
} while_each_thread (task, t);
rcu_read_unlock();is wrong even if while_each_thread() was correct (and we have a lot of examples of this pattern). A GP can pass before the 2nd rcu-lock, and we simply can't trust ->thread_group.next.
But I didn't notice that cgroup_attach_task(tsk, threadgroup) can only be called with threadgroup == T when a) tsk is ->group_leader and b) we hold threadgroup_lock() which blocks de_thread(). IOW, in this case "tsk" can't be removed from ->thread_group list before other threads.
If next_thread() sees thread_group.next != leader, we know that the that .next thread didn't do __unhash_process() yet, and since we know that in this case "leader" didn't do this too we are safe.
In short: __unhash_process(leader) (in this) case can never change ->thread_group.next of another thread, because leader->thread_group should be already list_empty().
Quoted 42 lines · Oleg Nesterov
Quoted 7 lines · Oleg Nesterov
Quoted 3 lines · Li Zefan
Anjana, could you revise the patch and send it out with proper changelog
and Signed-off-by? And please add "Cc: <[email protected]> # 3.9+"Yes, Anjana, please!
Please note also that the
PF_EXITINGcheck has the same problem, it also needs "goto next".Quoted 29 lines
Quoted 13 lines
Quoted 12 lines
check in the main loop. So Anjana was right (sorry again!), and we should probably do
ent.cgrp = task_cgroup_from_root(...); if (ent.cgrp != cgrp) { retval = flex_array_put(...); ... } if (!threadgroup) break;Quoted 6 lines
Or
do { ... if (ent.cgrp == cgrp) goto next;Or this, agreed.
Quoted 3 lines
Quoted 1 line
Or I am wrong again?
No, you are not! :)
Thanks ;)
Oleg.
Re: cgroup_attach_task && while_each_thread (Was: cgroup attach task - slogging cpu)
Quoted 25 lines · Oleg Nesterov
And I am starting to think that this change should also fix the
while_each_thread()problems in this particular case.In generak the code like
rcu_read_lock(); task = find_get_task(...); rcu_read_unlock(); rcu_read_lock(); t = task; do { ... } while_each_thread (task, t); rcu_read_unlock();is wrong even if
while_each_thread()was correct (and we have a lot of examples of this pattern). A GP can pass before the 2nd rcu-lock, and we simply can't trust ->thread_group.next.But I didn't notice that
cgroup_attach_task(tsk, threadgroup)can only be called with threadgroup == T when a) tsk is ->group_leader and b) we holdthreadgroup_lock()which blocksde_thread(). IOW, in this case "tsk" can't be removed from ->thread_group list before other threads.
Quoted 8 lines
If
next_thread()sees thread_group.next != leader, we know that the that .next thread didn't do__unhash_process()yet, and since we know that in this case "leader" didn't do this too we are safe.In short:
__unhash_process(leader)(in this) case can never change ->thread_group.next of another thread, becauseleader->thread_groupshould be alreadylist_empty().
If threadgroup == false, and if the tsk is existing or is already in the targeted cgroup, we won't break the loop due to the bug but do this:
while_each_thread(task, t)
If @task isn't the leader, we might got stuck in the loop?
Re: cgroup_attach_task && while_each_thread (Was: cgroup attach task - slogging cpu)
Quoted 5 lines · Li Zefan
Quoted 2 lines · Oleg Nesterov
And I am starting to think that this change should also fix the
while_each_thread()problems in this particular case.
Please see below,
Quoted 31 lines
Quoted 22 lines
In generak the code like
rcu_read_lock(); task = find_get_task(...); rcu_read_unlock(); rcu_read_lock(); t = task; do { ... } while_each_thread (task, t); rcu_read_unlock();is wrong even if
while_each_thread()was correct (and we have a lot of examples of this pattern). A GP can pass before the 2nd rcu-lock, and we simply can't trust ->thread_group.next.But I didn't notice that
cgroup_attach_task(tsk, threadgroup)can only be called with threadgroup == T when a) tsk is ->group_leader and b) we holdthreadgroup_lock()which blocksde_thread(). IOW, in this case "tsk" can't be removed from ->thread_group list before other threads.Quoted 8 lines
If
next_thread()sees thread_group.next != leader, we know that the that .next thread didn't do__unhash_process()yet, and since we know that in this case "leader" didn't do this too we are safe.In short:
__unhash_process(leader)(in this) case can never change ->thread_group.next of another thread, becauseleader->thread_groupshould be alreadylist_empty().
Quoted 7 lines
If threadgroup == false, and if the tsk is existing or is already in the targeted cgroup, we won't break the loop due to the bug but do this:
while_each_thread(task, t)If @task isn't the leader, we might got stuck in the loop?
Yes, yes, sure. We need to fix the wrong "continue" logic, hopefully
I tried to say (see above) that after we do this while_each_thread() should be fine in this particular case.
Oleg.
PATCH 1/1 cgroup: fix to break the while loop in cgroup_attach_task() correctly
From: Anjana V Kumar <[email protected]>
Both Anjana and Eunki reported a stall in the while_each_thread loop in cgroup_attach_task().
It's because, when we attach a single thread to a cgroup, if the cgroup is exiting or is already in that cgroup, we won't break the loop.
If the task is already in the cgroup, the bug can lead to another thread being attached to the cgroup unexpectedly:
# echo 5207 > tasks # cat tasks 5207 # echo 5207 > tasks # cat tasks 5207 5215
What's worse, if the task to be attached isn't the leader of the thread group, we might never exit the loop, hence cpu stall. Thanks for Oleg's analysis.
This bug was introduced by commit 081aa458c38ba576bdd4265fc807fa95b48b9e79 ("cgroup: consolidate cgroup_attach_task() and cgroup_attach_proc()")
[ lizf: - fixed the first continue, pointed out by Oleg,
- rewrote changelog. ]
--- kernel/cgroup.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
kernel/cgroup.c +3 -3
@@ -2002,7 +2002,7 @@ static int cgroup_attach_task(struct cgroup *cgrp, struct task_struct *tsk,/* @tsk either already exited or can't exit until the end */ if (tsk->flags &PF_EXITING ) -continue ; + goto next;/* as per above, nr_threads may decrease, but not increase. */ BUG_ON (i >= group_size);
@@ -2010,7 +2010,7 @@ static int cgroup_attach_task(struct cgroup *cgrp, struct task_struct *tsk, ent.cgrp =task_cgroup_from_root (tsk, root);/* nothing to do if this task is already in the cgroup */ if (ent.cgrp == cgrp) -continue ; + goto next;/* * saying GFP_ATOMIC has no effect here because we did prealloc * earlier, but it's good form to communicate our expectations.
@@ -2018,7 +2018,7 @@ static int cgroup_attach_task(struct cgroup *cgrp, struct task_struct *tsk, retval =flex_array_put (group, i, &ent,GFP_ATOMIC );BUG_ON (retval !=0 ); i++; - +next:if (!threadgroup)break ; }while_each_thread (leader, tsk);
Signature · 1 line
1.8.0.2
Re: [PATCH] cgroup: fix to break the while loop in cgroup_attach_task() correctly
Quoted 26 lines · Li Zefan
From: Anjana V Kumar <[email protected]>
Both Anjana and Eunki reported a stall in the
while_each_threadloop incgroup_attach_task().It's because, when we attach a single thread to a cgroup, if the cgroup is exiting or is already in that cgroup, we won't break the loop.
If the task is already in the cgroup, the bug can lead to another thread being attached to the cgroup unexpectedly:
# echo 5207 > tasks # cat tasks 5207 # echo 5207 > tasks # cat tasks 5207 5215What's worse, if the task to be attached isn't the leader of the thread group, we might never exit the loop, hence cpu stall. Thanks for Oleg's analysis.
This bug was introduced by commit
081aa458c38ba576bdd4265fc807fa95b48b9e79("cgroup: consolidatecgroup_attach_task()andcgroup_attach_proc()")
Quoted 7 lines
Cc <[email protected]> # 3.9+Reported-by Eunki Kim <[email protected]>Reported-by Anjana V Kumar <[email protected]>Signed-off-by Anjana V Kumar <[email protected]>[ lizf: - fixed the first continue, pointed out by Oleg,
- rewrote changelog. ]Signed-off-by Li Zefan <[email protected]>
Applied to cgroup/for-3.12-fixes.
Thanks.
Signature · 1 line
tejun
Re: cgroup_attach_task && while_each_thread (Was: cgroup attach task - slogging cpu)
In reply to Oleg NesterovHi All,
Thankyou for your suggestions.
I have made the modifications as suggested.
Please find the patch below.
From fd85f093f912a160c0896ea2784dfbdd64f142ca Mon Sep 17 00:00:00 2001
From: Anjana V Kumar <[email protected]>
Date: Wed, 9 Oct 2013 16:49:22 +0530
Subject: [PATCH] Single thread break missing in cgroup_attach_task
Problem:
Issue when attaching a single thread to a cgroup if the thread was alredy in the cgroup. The check if the thread is already in cgroup in the above case, continues to the next thread instead of exciting.
Solution:
Add a check if it is a single thread or a threadgroup and
take decision accordingly. If it is a single thread break out of the do-while loop, and if it is a threadgroup goto the next thread.
--- kernel/cgroup.c | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-)
kernel/cgroup.c +0 -0
@@ -2039,7 +2039,7 @@ static int cgroup_attach_task(struct cgroup
*cgrp, struct task_struct *tsk,
/* @tsk either already exited or can't exit until the end */ if (tsk->flags & PF_EXITING)
- continue;
+ goto next_thread;
/* as per above, nr_threads may decrease, but not increase. */ BUG_ON(i >= group_size);
@@ -2047,7 +2047,7 @@ static int cgroup_attach_task(struct cgroup *cgrp, struct task_struct *tsk,
ent.cgrp = task_cgroup_from_root(tsk, root);
/* nothing to do if this task is already in the cgroup */ if (ent.cgrp == cgrp)
- continue;
+ goto next_thread;
/*
* saying GFP_ATOMIC has no effect here because we did prealloc
* earlier, but it's good form to communicate our expectations.
@@ -2055,7 +2055,7 @@ static int cgroup_attach_task(struct cgroup *cgrp, struct task_struct *tsk,
retval = flex_array_put(group, i, &ent, GFP_ATOMIC); BUG_ON(retval != 0);
i++;
-
+next_thread:
if (!threadgroup)
break;
} while_each_thread(leader, tsk);
Signature · 1 line
1.7.6
Quoted 56 lines · Oleg Nesterov
Quoted 5 lines · Li Zefan
Quoted 2 lines · Oleg Nesterov
And I am starting to think that this change should also fix the
while_each_thread()problems in this particular case.Please see below,
Quoted 31 lines
Quoted 22 lines
In generak the code like
rcu_read_lock(); task = find_get_task(...); rcu_read_unlock(); rcu_read_lock(); t = task; do { ... } while_each_thread (task, t); rcu_read_unlock();is wrong even if
while_each_thread()was correct (and we have a lot of examples of this pattern). A GP can pass before the 2nd rcu-lock, and we simply can't trust ->thread_group.next.But I didn't notice that
cgroup_attach_task(tsk, threadgroup)can only be called with threadgroup == T when a) tsk is ->group_leader and b) we holdthreadgroup_lock()which blocksde_thread(). IOW, in this case "tsk" can't be removed from ->thread_group list before other threads.Quoted 8 lines
If
next_thread()sees thread_group.next != leader, we know that the that .next thread didn't do__unhash_process()yet, and since we know that in this case "leader" didn't do this too we are safe.In short:
__unhash_process(leader)(in this) case can never change ->thread_group.next of another thread, becauseleader->thread_groupshould be alreadylist_empty().Quoted 7 lines
If threadgroup == false, and if the tsk is existing or is already in the targeted cgroup, we won't break the loop due to the bug but do this:
while_each_thread(task, t)If @task isn't the leader, we might got stuck in the loop?
Yes, yes, sure. We need to fix the wrong "continue" logic, hopefully
I tried to say (see above) that after we do this
while_each_thread()should be fine in this particular case.Oleg.
Tejun Heo
maintainerRe: cgroup_attach_task && while_each_thread (Was: cgroup attach task - slogging cpu)
Quoted 4 lines · anjana vk
Hi All,
Thankyou for your suggestions.
I have made the modifications as suggested.
I already applied the version Li posted (as you the author, of course).
Thanks a lot!
Signature · 1 line
tejun
A message that is not in the archive (sent off-list or not yet received): CAChhN7RerxpSadqyosUeSKFg+qcOpO4d-maEKBZ0rvOQGvN27g@mail.gmail.com
A message that is not in the archive (sent off-list or not yet received): CAChhN7RerxpSadqyosUeSKFg+qcOpO4d-maEKBZ0rvOQGvN27g-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org
Quoted 64 lines · Anjana V Kumar
Hi All,
Please find the attached patch making the changes for the case where thread is already in the required cgroup.
Can you please let me know if this is the right way to fix this issue, if any modifications are required.
Thanks and Regards
AnjanaQuoted 5 lines · Oleg Nesterov
Quoted 4 lines · Oleg Nesterov
Quoted 2 lines · Li Zefan
Anjana, could you revise the patch and send it out with proper
Quoted 40 lines
changelog
Quoted 1 line
Quoted 1 line
and Signed-off-by? And please add "Cc: <[email protected]> #
3.9+"
Quoted 2 lines
Yes, Anjana, please!
Please note also that the
PF_EXITINGcheck has the same problem, it also needs "goto next".Quoted 29 lines
Quoted 13 lines
Quoted 12 lines
check in the main loop. So Anjana was right (sorry again!), and we should probably do
ent.cgrp = task_cgroup_from_root(...); if (ent.cgrp != cgrp) { retval = flex_array_put(...); ... } if (!threadgroup) break;Quoted 6 lines
Or
do { ... if (ent.cgrp == cgrp) goto next;Or this, agreed.
Quoted 3 lines
Quoted 1 line
Or I am wrong again?
No, you are not! :)
Thanks ;)
Oleg.
--
Anjana
Hi Anjana,
Quoted 7 lines · Anjana V Kumar
Quoted 2 lines
Problem:
Issue when attaching a single thread to a cgroup if the thread was alredy in the cgroup. The check if the thread is already in cgroup in the above case, continues to the next thread instead of exciting.
Yes. Thanks.
Quoted 10 lines
(quoted diff) +1
-1@@ -2047,7 +2047,7 @@ static int cgroup_attach_task(struct cgroup *cgrp, struct task_struct *tsk, ent.cgrp = task_cgroup_from_root(tsk, root); /* nothing to do if this task is already in the cgroup */ if (ent.cgrp == cgrp) - continue; + goto next_thread; /* * saying GFP_ATOMIC has no effect here because we did prealloc * earlier, but it's good form to communicate our expectations.
Review static int cgroup_attach_task(struct cgroup *cgrp, struct task_struct *tsk, · line 2058
Quoted 5 lines
(quoted diff) +1
-1@@ -2055,7 +2055,7 @@ static int cgroup_attach_task(struct cgroup *cgrp, struct task_struct *tsk, retval = flex_array_put(group, i, &ent, GFP_ATOMIC); BUG_ON(retval != 0); i++; - +next_thread:
Yes, but you forgot to fix another "continue" after "ent.cgrp == cgrp" ;)
Oleg.