sched/numa: fix unsafe get_task_struct() in task_numa_assign()
And smp_rmb() beetween ifs which is pairs with rq unlocking
Quoted 96 lines
18.10.2014, 12:15, "Kirill Tkhai" <[email protected]>:
Quoted 47 lines
18.10.2014, 01:40, "Oleg Nesterov" <[email protected]>:
Quoted 29 lines
The lockless
get_task_struct(tsk)is only safe if tsk == current and didn't passexit_notify(), or if this tsk was found on a rcu protected list (say,for_each_process()orfind_task_by_vpid()). IOW, it is only safe ifrelease_task()was not called before we takercu_read_lock(), in this case we can rely on the fact thatdelayed_put_pid()can not drop the (potentially) last reference untilrcu_read_unlock().And as Kirill pointed out task_numa_compare()->task_numa_assign() path does
get_task_struct(dst_rq->curr)and this is not safe. Thetask_structitself can't go away, butrcu_read_lock()can't save us from the finalput_task_struct()infinish_task_switch(); this reference goes away without rcu gp.Reported-by Kirill Tkhai <[email protected]>Signed-off-by Oleg Nesterov <[email protected]>---
kernel/sched/fair.c| 8 +++++++-1 files changed, 7 insertions(+), 1 deletions(-)kernel/sched/fair.c +0
-1@@ -1158,7 +1158,13 @@ static void task_numa_compare(struct task_numa_env *env,rcu_read_lock (); cur =ACCESS_ONCE (dst_rq->curr); -if (cur->pid ==0 )/* idle */ kernel/sched/fair.c static void task_numa_compare(struct task_numa_env *env, · line 1170
Quoted 10 lines
kernel/sched/fair.c +7
-0+/* +* No need to move the exiting task, and this ensures that ->curr +* wasn't reaped and thus get_task_struct() in task_numa_assign() +* is safe; note that rcu_read_lock() can't protect from the final +* put_task_struct() after the last schedule(). +*/ +if (is_idle_task (cur) || (cur->flags &PF_EXITING )) cur =NULL ;/* Oleg, I've looked once again, and now it's not good for me. Where is the guarantee this memory hasn't been allocated again? If so,
PF_EXITINGis not of the task we are interesting, but it's not a task's even.Quoted 12 lines
rcu_read_lock()... ...
cur =ACCESS_ONCE(dst_rq->curr); ... ...
<interrupt>rq->curr= next; ...
<interrupt>put_prev_task()...
<interrupt>__put_prev_task...
<interrupt>kmem_cache_free()...
<interrupt> ... <alocated again>
<interrupt> ...memset(, 0, )
<interrupt> ... ...
if (cur->flags&PF_EXITING) ... ...
<no> ... ...get_task_struct()... ...How about this?
kernel/sched/fair.c +15
-2@@ -1165,7 +1165,21 @@ static void task_numa_compare(struct task_numa_env *env,rcu_read_lock (); cur =ACCESS_ONCE (dst_rq->curr); -if (cur->pid ==0 )/* idle */ +/* +* No need to move the exiting task, and this ensures that ->curr +* wasn't reaped and thus get_task_struct() in task_numa_assign() +* is safe; note that rcu_read_lock() can't protect from the final +* put_task_struct() after the last schedule(). +*/ +if (is_idle_task (cur) || (cur->flags &PF_EXITING )) + cur =NULL ; +/* +* Check once again to be sure curr is still on dst_rq. Even if +* it points on a new task, which is using the memory of freed +* cur, it's OK, because we've locked RCU before +* delayed_put_task_struct() callback is called to put its struct. +*/ +if (cur !=ACCESS_ONCE (dst_rq->curr)) cur =NULL ;/* -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/