bpf: task_group_seq_get_next: cleanup the usage of next_thread()
Subsystems (2)
PATCH 1/1 bpf: task_group_seq_get_next: cleanup the usage of next_thread()
1. find_pid_ns() + get_pid_task() under rcu_read_lock() guarantees that we can safely iterate the task->thread_group list. Even if this task exits right after get_pid_task() (or goto retry) and pid_alive() returns 0.
Kill the unnecessary pid_alive() check.
2. next_thread() simply can't return NULL, kill the bogus "if (!next_task)" check.
--- kernel/bpf/task_iter.c | 7 ------- 1 file changed, 7 deletions(-)
kernel/bpf/task_iter.c +0 -7
@@ -75,15 +75,8 @@ static struct task_struct *task_group_seq_get_next(struct bpf_iter_seq_task_commreturn NULL ; retry: -if (!pid_alive (task)) { -put_task_struct (task); -return NULL ; - } - next_task =next_thread (task);put_task_struct (task); -if (!next_task) -return NULL ; saved_tid = *tid; *tid =__task_pid_nr_ns (next_task,PIDTYPE_PID , common->ns);
Signature · 1 line
2.25.1.362.g51ebf55
Quoted 4 lines · Oleg Nesterov
1.
find_pid_ns()+get_pid_task()underrcu_read_lock()guarantees that we can safely iterate thetask->thread_grouplist. Even if this task exits right afterget_pid_task()(or goto retry) andpid_alive()returns 0 > Kill the unnecessarypid_alive()check.
This function will return next_task holding a refcount, and release the refcount until the next time calling the same function. Meanwhile, the returned task A may be killed, and its next task B may be killed after A as well, before calling this function again. However, even task B is destroyed (free), A's next is still pointing to task B. When this function is called again for the same iterator, it doesn't promise that B is still there.
Does that make sense to you?
Quoted 29 lines
2.
next_thread()simply can't return NULL, kill the bogus "if (!next_task)" check.Signed-off-by Oleg Nesterov <[email protected]>--- kernel/bpf/task_iter.c | 7 ------- 1 file changed, 7 deletions(-)kernel/bpf/task_iter.c +0
-7@@ -75,15 +75,8 @@ static struct task_struct *task_group_seq_get_next(struct bpf_iter_seq_task_commreturn NULL ; retry: -if (!pid_alive (task)) { -put_task_struct (task); -return NULL ; - } - next_task =next_thread (task);put_task_struct (task); -if (!next_task) -return NULL ; saved_tid = *tid; *tid =__task_pid_nr_ns (next_task,PIDTYPE_PID , common->ns);
Quoted 8 lines · Kui-Feng Lee
Quoted 4 lines · Oleg Nesterov
1.
find_pid_ns()+get_pid_task()underrcu_read_lock()guarantees that we can safely iterate thetask->thread_grouplist. Even if this task exits right afterget_pid_task()(or goto retry) andpid_alive()returns 0 > Kill the unnecessarypid_alive()check.
Quoted 7 lines
This function will return
next_taskholding a refcount, and release the refcount until the next time calling the same function. Meanwhile, the returned task A may be killed, and its next task B may be killed after A as well, before calling this function again. However, even task B is destroyed (free), A's next is still pointing to task B. When this function is called again for the same iterator, it doesn't promise that B is still there.
Not sure I understand...
OK, if we have a task pointer with incremented refcount and do not hold rcu lock, then yes, you can't remove the pid_alive() check in this code:
rcu_read_lock(); if (pid_alive(task)) do_something(next_thread(task)); rcu_read_unlock();
because task and then task->next can exit and do call_rcu(delayed_put_task_struct) before we take rcu_read_lock().
But if you do something like
rcu_read_lock(); task = find_task_in_some_rcu_protected_list(); do_something(next_thread(task)); rcu_read_unlock();
then next_thread(task) should be safe without pid_alive().
And iiuc task_group_seq_get_next() always does
rcu_read_lock(); // the caller does lock/unlock task = get_pid_task(pid, PIDTYPE_PID); if (!task) return; next_task = next_thread(task); rcu_read_unlock();
Yes, both task and task->next can exit right after get_pid_task(), but since can only happen after we took rcu_read_lock(), delayed_put_task_struct() can't be called until we drop rcu lock.
What have I missed?
Oleg.
So I still think the pid_alive() check should die...
and when I look at this code again I don't understand why does it abuse task_struct->usage, I'll send another patch on top of this one.
Quoted 61 lines · Oleg Nesterov
Quoted 8 lines · Kui-Feng Lee
Quoted 4 lines · Oleg Nesterov
1.
find_pid_ns()+get_pid_task()underrcu_read_lock()guarantees that we can safely iterate thetask->thread_grouplist. Even if this task exits right afterget_pid_task()(or goto retry) andpid_alive()returns 0 > Kill the unnecessarypid_alive()check.Quoted 7 lines
This function will return
next_taskholding a refcount, and release the refcount until the next time calling the same function. Meanwhile, the returned task A may be killed, and its next task B may be killed after A as well, before calling this function again. However, even task B is destroyed (free), A's next is still pointing to task B. When this function is called again for the same iterator, it doesn't promise that B is still there.Not sure I understand...
OK, if we have a task pointer with incremented refcount and do not hold rcu lock, then yes, you can't remove the
pid_alive()check in this code:rcu_read_lock(); if (pid_alive(task)) do_something(next_thread(task)); rcu_read_unlock();because task and then
task->nextcan exit and docall_rcu(delayed_put_task_struct)before we takercu_read_lock().But if you do something like
rcu_read_lock(); task = find_task_in_some_rcu_protected_list(); do_something(next_thread(task)); rcu_read_unlock();then
next_thread(task)should be safe withoutpid_alive().And iiuc
task_group_seq_get_next()always doesrcu_read_lock(); // the caller does lock/unlock task = get_pid_task(pid, PIDTYPE_PID); if (!task) return; next_task = next_thread(task); rcu_read_unlock();Yes, both task and
task->nextcan exit right afterget_pid_task(), but since can only happen after we tookrcu_read_lock(),delayed_put_task_struct()can't be called until we drop rcu lock.What have I missed?
Oleg.
Quoted 53 lines · Oleg Nesterov
Quoted 8 lines · Kui-Feng Lee
Quoted 4 lines · Oleg Nesterov
1.
find_pid_ns()+get_pid_task()underrcu_read_lock()guarantees that we can safely iterate thetask->thread_grouplist. Even if this task exits right afterget_pid_task()(or goto retry) andpid_alive()returns 0 > Kill the unnecessarypid_alive()check.Quoted 7 lines
This function will return
next_taskholding a refcount, and release the refcount until the next time calling the same function. Meanwhile, the returned task A may be killed, and its next task B may be killed after A as well, before calling this function again. However, even task B is destroyed (free), A's next is still pointing to task B. When this function is called again for the same iterator, it doesn't promise that B is still there.Not sure I understand...
OK, if we have a task pointer with incremented refcount and do not hold rcu lock, then yes, you can't remove the
pid_alive()check in this code:rcu_read_lock(); if (pid_alive(task)) do_something(next_thread(task)); rcu_read_unlock();because task and then
task->nextcan exit and docall_rcu(delayed_put_task_struct)before we takercu_read_lock().But if you do something like
rcu_read_lock(); task = find_task_in_some_rcu_protected_list(); do_something(next_thread(task)); rcu_read_unlock();then
next_thread(task)should be safe withoutpid_alive().And iiuc
task_group_seq_get_next()always doesrcu_read_lock(); // the caller does lock/unlock task = get_pid_task(pid, PIDTYPE_PID); if (!task) return; next_task = next_thread(task); rcu_read_unlock();
Quoted 5 lines
Yes, both task and
task->nextcan exit right afterget_pid_task(), but since can only happen after we tookrcu_read_lock(),delayed_put_task_struct()can't be called until we drop rcu lock.What have I missed?
Then, it makes sense to me! Thank you for the explanation.
Quoted 3 lines
Oleg.
PATCH 1/1 bpf: task_group_seq_get_next: cleanup the usage of get/put_task_struct
In reply to Oleg Nesterovget_pid_task() makes no sense, the code does put_task_struct() soon after. Use find_task_by_pid_ns() instead of find_pid_ns + get_pid_task and kill kill put_task_struct(), this allows to do get_task_struct() only once before return.
While at it, kill the unnecessary "if (!pid)" check in the "if (!*tid)" block, this matches the next usage of find_pid_ns() + get_pid_task() in this function.
--- kernel/bpf/task_iter.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-)
kernel/bpf/task_iter.c +2 -10
@@ -42,9 +42,6 @@ static struct task_struct *task_group_seq_get_next(struct bpf_iter_seq_task_commif (!*tid) {/* The first time, the iterator calls this function. */ pid =find_pid_ns (common->pid, common->ns); -if (!pid) -return NULL ; - task =get_pid_task (pid,PIDTYPE_TGID );if (!task)return NULL ;
@@ -66,17 +63,12 @@ static struct task_struct *task_group_seq_get_next(struct bpf_iter_seq_task_commreturn task; } - pid =find_pid_ns (common->pid_visiting, common->ns); -if (!pid) -return NULL ; - - task =get_pid_task (pid,PIDTYPE_PID ); + task =find_task_by_pid_ns (common->pid_visiting, common->ns);if (!task)return NULL ; retry: next_task =next_thread (task); -put_task_struct (task); saved_tid = *tid; *tid =__task_pid_nr_ns (next_task,PIDTYPE_PID , common->ns);
@@ -88,7 +80,6 @@ static struct task_struct *task_group_seq_get_next(struct bpf_iter_seq_task_commreturn NULL ; } -get_task_struct (next_task); common->pid_visiting = *tid;if (skip_if_dup_files && task->files == task->group_leader->files) {
@@ -96,6 +87,7 @@ static struct task_struct *task_group_seq_get_next(struct bpf_iter_seq_task_comm goto retry; } +get_task_struct (next_task);return next_task; }
Signature · 1 line
2.25.1.362.g51ebf55
Re: [PATCH] bpf: task_group_seq_get_next: cleanup the usage of get/put_task_struct
Quoted 34 lines · Oleg Nesterov
get_pid_task()makes no sense, the code doesput_task_struct()soon after. Usefind_task_by_pid_ns()instead offind_pid_ns+get_pid_taskand kill killput_task_struct(), this allows to doget_task_struct()only once before return.While at it, kill the unnecessary "if (!pid)" check in the "if (!*tid)" block, this matches the next usage of
find_pid_ns()+get_pid_task()in this function.Signed-off-by Oleg Nesterov <[email protected]>--- kernel/bpf/task_iter.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-)kernel/bpf/task_iter.c +0
-5@@ -42,9 +42,6 @@ static struct task_struct *task_group_seq_get_next(struct bpf_iter_seq_task_commif (!*tid) {/* The first time, the iterator calls this function. */ pid =find_pid_ns (common->pid, common->ns); -if (!pid) -return NULL ; - task =get_pid_task (pid,PIDTYPE_TGID );if (!task)return NULL ;@@ -66,17 +63,12 @@ static struct task_struct *task_group_seq_get_next(struct bpf_iter_seq_task_commreturn task; } - pid =find_pid_ns (common->pid_visiting, common->ns); -if (!pid)
kernel/bpf/task_iter.c static struct task_struct *task_group_seq_get_next(struct bpf_iter_seq_task_comm · line 79
Quoted 10 lines
kernel/bpf/task_iter.c +1
-4-return NULL ; - - task =get_pid_task (pid,PIDTYPE_PID ); + task =find_task_by_pid_ns (common->pid_visiting, common->ns);if (!task)return NULL ; retry: next_task =next_thread (task); -put_task_struct (task);
It called get_task_struct() against this task to hold a refcount at the previous time calling this function. When will it release the refcount?
Quoted 19 lines
kernel/bpf/task_iter.c +1
-1saved_tid = *tid; *tid =__task_pid_nr_ns (next_task,PIDTYPE_PID , common->ns);@@ -88,7 +80,6 @@ static struct task_struct *task_group_seq_get_next(struct bpf_iter_seq_task_commreturn NULL ; } -get_task_struct (next_task); common->pid_visiting = *tid;if (skip_if_dup_files && task->files == task->group_leader->files) {@@ -96,6 +87,7 @@ static struct task_struct *task_group_seq_get_next(struct bpf_iter_seq_task_comm goto retry; } +get_task_struct (next_task);return next_task; }
Re: [PATCH] bpf: task_group_seq_get_next: cleanup the usage of get/put_task_struct
Quoted 42 lines · Kui-Feng Lee
Quoted 39 lines · Oleg Nesterov
get_pid_task()makes no sense, the code doesput_task_struct()soon after. Usefind_task_by_pid_ns()instead offind_pid_ns+get_pid_taskand kill killput_task_struct(), this allows to doget_task_struct()only once before return.While at it, kill the unnecessary "if (!pid)" check in the "if (!*tid)" block, this matches the next usage of
find_pid_ns()+get_pid_task()in this function.Signed-off-by Oleg Nesterov <[email protected]>---
kernel/bpf/task_iter.c| 12 ++----------1 file changed, 2 insertions(+), 10 deletions(-)kernel/bpf/task_iter.c +0
-3@@ -42,9 +42,6 @@ static struct task_struct *task_group_seq_get_next(struct bpf_iter_seq_task_comm if (!*tid) { /* The first time, the iterator calls this function. */ pid = find_pid_ns(common->pid, common->ns); - if (!pid) - return NULL; - task = get_pid_task(pid, PIDTYPE_TGID); if (!task)return NULL;
kernel/bpf/task_iter.c +1
-5@@ -66,17 +63,12 @@ static struct task_struct *task_group_seq_get_next(struct bpf_iter_seq_task_comm return task; } - pid = find_pid_ns(common->pid_visiting, common->ns); - if (!pid) - return NULL; - - task = get_pid_task(pid, PIDTYPE_PID); + task = find_task_by_pid_ns(common->pid_visiting, common->ns);
Quoted 8 lines
Quoted 5 lines
kernel/bpf/task_iter.c +0
-0if (!task) return NULL;retry:
next_task=next_thread(task);
-put_task_struct(task);It called
get_task_struct()against this task to hold a refcount at the previous time calling this function. When will it release the refcount?
Oh! I missed the fact that the caller will handle it.
Quoted 17 lines
Quoted 17 lines
saved_tid= *tid;
*tid =__task_pid_nr_ns(next_task, PIDTYPE_PID, common->ns);kernel/bpf/task_iter.c +1
-1@@ -88,7 +80,6 @@ static struct task_struct *task_group_seq_get_next(struct bpf_iter_seq_task_comm return NULL; } - get_task_struct(next_task); common->pid_visiting = *tid; if (skip_if_dup_files && task->files == task->group_leader->files) {@@ -96,6 +87,7 @@ static struct task_struct *task_group_seq_get_next(struct bpf_iter_seq_task_comm goto retry; } + get_task_struct(next_task); return next_task; }
Re: [PATCH] bpf: task_group_seq_get_next: cleanup the usage of get/put_task_struct
In reply to Oleg NesterovQuoted 3 lines · Oleg Nesterov
get_pid_task()makes no sense, the code doesput_task_struct()soon after. Usefind_task_by_pid_ns()instead offind_pid_ns+get_pid_taskand kill killput_task_struct(), this allows to doget_task_struct()only once
remove the duplicated 'kill' in the above.
Quoted 7 lines
before return.
While at it, kill the unnecessary "if (!pid)" check in the "if (!*tid)" block, this matches the next usage of
find_pid_ns()+get_pid_task()in this function.Signed-off-by Oleg Nesterov <[email protected]>
LGTM.
Re: [PATCH] bpf: task_group_seq_get_next: cleanup the usage of get/put_task_struct
Quoted 8 lines · Yonghong Song
Quoted 3 lines · Oleg Nesterov
get_pid_task()makes no sense, the code doesput_task_struct()soon after. Usefind_task_by_pid_ns()instead offind_pid_ns+get_pid_taskand kill killput_task_struct(), this allows to doget_task_struct()only onceremove the duplicated 'kill' in the above.
Done,
Quoted 3 lines
LGTM.
Acked-by Yonghong Song <[email protected]>
Thanks, I'll send V2 with your ack included in a minute.
Oleg.
PATCH v2 1/1 bpf: task_group_seq_get_next: cleanup the usage of get/put_task_struct
In reply to Oleg Nesterovget_pid_task() makes no sense, the code does put_task_struct() soon after. Use find_task_by_pid_ns() instead of find_pid_ns + get_pid_task and kill put_task_struct(), this allows to do get_task_struct() only once before return.
While at it, kill the unnecessary "if (!pid)" check in the "if (!*tid)" block, this matches the next usage of find_pid_ns() + get_pid_task() in this function.
--- kernel/bpf/task_iter.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-)
kernel/bpf/task_iter.c +2 -10
@@ -42,9 +42,6 @@ static struct task_struct *task_group_seq_get_next(struct bpf_iter_seq_task_commif (!*tid) {/* The first time, the iterator calls this function. */ pid =find_pid_ns (common->pid, common->ns); -if (!pid) -return NULL ; - task =get_pid_task (pid,PIDTYPE_TGID );if (!task)return NULL ;
@@ -66,17 +63,12 @@ static struct task_struct *task_group_seq_get_next(struct bpf_iter_seq_task_commreturn task; } - pid =find_pid_ns (common->pid_visiting, common->ns); -if (!pid) -return NULL ; - - task =get_pid_task (pid,PIDTYPE_PID ); + task =find_task_by_pid_ns (common->pid_visiting, common->ns);if (!task)return NULL ; retry: next_task =next_thread (task); -put_task_struct (task); saved_tid = *tid; *tid =__task_pid_nr_ns (next_task,PIDTYPE_PID , common->ns);
@@ -88,7 +80,6 @@ static struct task_struct *task_group_seq_get_next(struct bpf_iter_seq_task_commreturn NULL ; } -get_task_struct (next_task); common->pid_visiting = *tid;if (skip_if_dup_files && task->files == task->group_leader->files) {
@@ -96,6 +87,7 @@ static struct task_struct *task_group_seq_get_next(struct bpf_iter_seq_task_comm goto retry; } +get_task_struct (next_task);return next_task; }
Signature · 1 line
2.25.1.362.g51ebf55
Daniel Borkmann
maintainerRe: [PATCH V2] bpf: task_group_seq_get_next: cleanup the usage of get/put_task_struct
Hi Oleg,
Quoted 11 lines · Oleg Nesterov
get_pid_task()makes no sense, the code doesput_task_struct()soon after. Usefind_task_by_pid_ns()instead offind_pid_ns+get_pid_taskand killput_task_struct(), this allows to doget_task_struct()only once before return.While at it, kill the unnecessary "if (!pid)" check in the "if (!*tid)" block, this matches the next usage of
find_pid_ns()+get_pid_task()in this function.Signed-off-by Oleg Nesterov <[email protected]>Acked-by Yonghong Song <[email protected]>
Could you rebase this against bpf-next tree so this can run through our BPF CI? Right now the CI cannot pick the patch up due to merge conflict [0].
Thanks,
Daniel
[0] https://patchwork.kernel.org/project/netdevbpf/patch/[email protected]/
Re: [PATCH V2] bpf: task_group_seq_get_next: cleanup the usage of get/put_task_struct
Quoted 8 lines · Daniel Borkmann
Could you rebase this against bpf-next tree so this can run through our BPF CI? Right now the CI cannot pick the patch up due to merge conflict [0].
Thanks,
Daniel[0] https://patchwork.kernel.org/project/netdevbpf/patch/[email protected]/
The merge failed because this patch depends on
[PATCH] bpf: task_group_seq_get_next: cleanup the usage of next_thread()
in this thread. But please forget.
I've sent the new series. It would be nice if you can test at least 1-5, the last 6/6 depends on
[PATCH 1/2] introduce __next_thread(), fix next_tid() vs exec() race https://lore.kernel.org/all/[email protected]/
which was not merged yet.
Oleg.
OK, it seems that you are not going to take these preparatory cleanups ;)
I'll resend along with the s/next_thread/__next_thread/ change. I was going to do the last change later, but this recent discussion
https://lore.kernel.org/all/[email protected]/
makes me think we should do this right now.
Quoted 39 lines · Oleg Nesterov
1.
find_pid_ns()+get_pid_task()underrcu_read_lock()guarantees that we can safely iterate thetask->thread_grouplist. Even if this task exits right afterget_pid_task()(or goto retry) andpid_alive()returns 0.Kill the unnecessary
pid_alive()check.2.
next_thread()simply can't return NULL, kill the bogus "if (!next_task)" check.Signed-off-by Oleg Nesterov <[email protected]>--- kernel/bpf/task_iter.c | 7 ------- 1 file changed, 7 deletions(-)kernel/bpf/task_iter.c +0
-8@@ -75,15 +75,8 @@ static struct task_struct *task_group_seq_get_next(struct bpf_iter_seq_task_commreturn NULL ; retry: -if (!pid_alive (task)) { -put_task_struct (task); -return NULL ; - } - next_task =next_thread (task);put_task_struct (task); -if (!next_task) -return NULL ; saved_tid = *tid; *tid =__task_pid_nr_ns (next_task,PIDTYPE_PID , common->ns); --2.25.1.362.g51ebf55
Eric W. Biederman
Acked-byQuoted 7 lines · Oleg Nesterov
OK, it seems that you are not going to take these preparatory cleanups ;)
I'll resend along with the s/next_thread/__next_thread/ change. I was going to do the last change later, but this recent discussion
https://lore.kernel.org/all/[email protected]/
makes me think we should do this right now.
For the record I find this code confusing, and wrong.
It looks like it wants to keep the task_struct pointer or possibly the struct pid pointer like proc does, but then it winds up keeping a userspace pid value and regenerating both the struct pid pointer and the struct task_struct pointer.
Which means that task_group_seq_get_next is unnecessarily slow and has a built in race condition which means it could wind up iterating through a different process.
This whole thing looks to be a bad (aka racy) reimplementation of first_tid and next_tid from proc. I thought the changes were to adapt to the needs of bpf, but on closer examination the code is just racy.
For this code to be correct bpf_iter_seq_task_common needs to store at a minimum a struct pid pointer.
Oleg your patch makes it easier to see what the how far this is from first_tid/next_tid in proc.
Eric
Quoted 2 lines · Eric W. Biederman
For the record I find this code confusing, and wrong.
Oh, yes...
Quoted 3 lines
and has
a built in race condition which means it could wind up iterating through a different process.
Yes, common->pid and/or common->pid_visiting can be reused
but I am not going to try to fix this ;)
Quoted 1 line
Thanks!
Oleg.