Skip to contentSkip to conversation

bpf: kernel/bpf/task_iter.c: don't abuse next_thread()

PATCH Applied Oleg Nesterov 3 participants 14 messages Nov 14, 2023 – Nov 19, 2023

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover bpf: kernel/bpf/task_iter.c: don't abuse next_thread()

Compile tested.

Every lockless usage of next_thread() was wrong, bpf/task_iter.c is the last user and is no exception.

Oleg.
---

 kernel/bpf/task_iter.c | 29 +++++++++++------------------
 1 file changed, 11 insertions(+), 18 deletions(-)
#1 lore raw

PATCH 1/3 bpf: task_group_seq_get_next: use __next_thread() rather than next_thread()

Lockless use of next_thread() should be avoided, kernel/bpf/task_iter.c is the last user and the usage is wrong.

task_group_seq_get_next() can return the group leader twice if it races with mt-thread exec which changes the group->leader's pid.

Change the main loop to use __next_thread(), kill "next_tid == common->pid" check.

__next_thread() can't loop forever, we can also change this code to retry if next_tid == 0.

---
 kernel/bpf/task_iter.c | 12 +++++-------
 1 file changed, 5 insertions(+), 7 deletions(-)
kernel/bpf/task_iter.c +5 -7
index 26082b97894d..51ae15e2b290 100644
@@ -70,15 +70,13 @@ static struct task_struct *task_group_seq_get_next(struct bpf_iter_seq_task_comm
 		return NULL;
 
 retry:
-	task = next_thread(task);
+	task = __next_thread(task);
+	if (!task)
+		return NULL;
 
 	next_tid = __task_pid_nr_ns(task, PIDTYPE_PID, common->ns);
-	if (!next_tid || next_tid == common->pid) {
-		/* Run out of tasks of a process.  The tasks of a
-		 * thread_group are linked as circular linked list.
-		 */
-		return NULL;
-	}
+	if (!next_tid)
+		goto retry;
 
 	if (skip_if_dup_files && task->files == task->group_leader->files)
 		goto retry;
Signature · 1 line
2.25.1.362.g51ebf55
#5 lore raw

PATCH 2/3 bpf: bpf_iter_task_next: use __next_thread() rather than next_thread()

In reply to Oleg Nesterov

Lockless use of next_thread() should be avoided, kernel/bpf/task_iter.c is the last user and the usage is wrong.

bpf_iter_task_next() can loop forever, "kit->pos == kit->task" can never happen if kit->pos execs. Change this code to use __next_thread().

With or without this change the usage of kit->pos/task and next_task() doesn't look nice, see the next patch.

---
 kernel/bpf/task_iter.c | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)
kernel/bpf/task_iter.c +4 -5
index 51ae15e2b290..d42e08d0d0b7 100644
@@ -1015,12 +1015,11 @@ __bpf_kfunc struct task_struct *bpf_iter_task_next(struct bpf_iter_task *it)
 	if (flags == BPF_TASK_ITER_ALL_PROCS)
 		goto get_next_task;
 
-	kit->pos = next_thread(kit->pos);
-	if (kit->pos == kit->task) {
-		if (flags == BPF_TASK_ITER_PROC_THREADS) {
-			kit->pos = NULL;
+	kit->pos = __next_thread(kit->pos);
+	if (!kit->pos) {
+		if (flags == BPF_TASK_ITER_PROC_THREADS)
 			return pos;
-		}
+		kit->pos = kit->task;
 	} else
 		return pos;
 
Signature · 1 line
2.25.1.362.g51ebf55
#7 lore raw

PATCH 3/3 bpf: bpf_iter_task_next: use next_task(kit->task) rather than next_task(kit->pos)

In reply to Oleg Nesterov

This looks more clear and simplifies the code. While at it, remove the unnecessary initialization of pos/task at the start of bpf_iter_task_new().

Note that we can even kill kit->task, we can just use pos->group_leader, but I don't understand the BUILD_BUG_ON() checks in bpf_iter_task_new().

---
 kernel/bpf/task_iter.c | 14 +++++---------
 1 file changed, 5 insertions(+), 9 deletions(-)
kernel/bpf/task_iter.c +5 -9
index d42e08d0d0b7..e5c3500443c6 100644
@@ -978,7 +978,6 @@ __bpf_kfunc int bpf_iter_task_new(struct bpf_iter_task *it,
 	BUILD_BUG_ON(__alignof__(struct bpf_iter_task_kern) !=
 					__alignof__(struct bpf_iter_task));
 
-	kit->task = kit->pos = NULL;
 	switch (flags) {
 	case BPF_TASK_ITER_ALL_THREADS:
 	case BPF_TASK_ITER_ALL_PROCS:
@@ -1016,18 +1015,15 @@ __bpf_kfunc struct task_struct *bpf_iter_task_next(struct bpf_iter_task *it)
 		goto get_next_task;
 
 	kit->pos = __next_thread(kit->pos);
-	if (!kit->pos) {
-		if (flags == BPF_TASK_ITER_PROC_THREADS)
-			return pos;
-		kit->pos = kit->task;
-	} else
+	if (kit->pos || flags == BPF_TASK_ITER_PROC_THREADS)
 		return pos;
 
 get_next_task:
-	kit->pos = next_task(kit->pos);
-	kit->task = kit->pos;
-	if (kit->pos == &init_task)
+	kit->task = next_task(kit->task);
+	if (kit->task == &init_task)
 		kit->pos = NULL;
+	else
+		kit->pos = kit->task;
 
 	return pos;
 }
Signature · 1 line
2.25.1.362.g51ebf55
maintainer
#10 lore raw
In reply to Oleg Nesterov
Quoted 4 lines · Oleg Nesterov

Compile tested.

Every lockless usage of next_thread() was wrong, bpf/task_iter.c is the last user and is no exception.

It would be great if you can give more information in the commit message about why the usage of next_thread() is wrong in bpf/task_iter.c. IIUC, some information is presented in :
https://lore.kernel.org/all/[email protected]/

Also, please add 'bpf' in the subject tag ([PATCH bpf 0/3]) to make it clear the patch should be applied to bpf tree.

Quoted 7 lines

Oleg.
---

  kernel/bpf/task_iter.c | 29 +++++++++++------------------
  1 file changed, 11 insertions(+), 18 deletions(-)
maintainer
#2 lore raw

Re: [PATCH 1/3] bpf: task_group_seq_get_next: use __next_thread() rather than next_thread()

In reply to Oleg Nesterov
Quoted 29 lines · Oleg Nesterov

Lockless use of next_thread() should be avoided, kernel/bpf/task_iter.c is the last user and the usage is wrong.

task_group_seq_get_next() can return the group leader twice if it races with mt-thread exec which changes the group->leader's pid.

Change the main loop to use __next_thread(), kill "next_tid == common->pid" check.

__next_thread() can't loop forever, we can also change this code to retry if next_tid == 0.

---
  kernel/bpf/task_iter.c | 12 +++++-------
  1 file changed, 5 insertions(+), 7 deletions(-)
kernel/bpf/task_iter.c +3 -1
index 26082b97894d..51ae15e2b290 100644
@@ -70,15 +70,13 @@ static struct task_struct *task_group_seq_get_next(struct bpf_iter_seq_task_comm
  		return NULL;
 
  retry:
-	task = next_thread(task);
+	task = __next_thread(task);
+	if (!task)
+		return NULL;

kernel/bpf/task_iter.c static struct task_struct *task_group_seq_get_next(struct bpf_iter_seq_task_comm · line 79

Quoted 10 lines
kernel/bpf/task_iter.c +2 -6
 
  	next_tid = __task_pid_nr_ns(task, PIDTYPE_PID, common->ns);
-	if (!next_tid || next_tid == common->pid) {
-		/* Run out of tasks of a process.  The tasks of a
-		 * thread_group are linked as circular linked list.
-		 */
-		return NULL;
-	}
+	if (!next_tid)
+		goto retry;

Look at the code. Looks like next_tid should never be 0 unless some task is migrated to other namespace which I think is not possible.

common->ns is assigned as below:
common->ns = get_pid_ns(task_active_pid_ns(current))
so we are searching tasks in the *current* namespace.

Look at:
pid_t pid_nr_ns(struct pid *pid, struct pid_namespace *ns)

{
        struct upid *upid;
        pid_t nr = 0;

        if (pid && ns->level <= pid->level) {
                upid = &pid->numbers[ns->level];
                if (upid->ns == ns)
                        nr = upid->nr;
        }
        return nr;
}

pid_t __task_pid_nr_ns(struct task_struct *task, enum pid_type type,
struct pid_namespace *ns)

{
        pid_t nr = 0;

        rcu_read_lock();
        if (!ns)
                ns = task_active_pid_ns(current);
        nr = pid_nr_ns(rcu_dereference(*task_pid_ptr(task, type)), ns);
        rcu_read_unlock();
        
        return nr;
}

In func pid_nr_ns(), ns->level should be equal to pid->level if pid is in input parameter 'ns'. and in this case the return value 'nr' should be none zero.

If this is the case, could you remove

	if (!next_tid)
		goto retry;

Other than above, the change looks good to me.

Quoted 3 lines
kernel/bpf/task_iter.c +0 -0
  	if (skip_if_dup_files && task->files == task->group_leader->files)
  		goto retry;
maintainer Acked-by on v1 2/3
#6 lore raw

Re: [PATCH 2/3] bpf: bpf_iter_task_next: use __next_thread() rather than next_thread()

In reply to Oleg Nesterov
Quoted 10 lines · Oleg Nesterov

Lockless use of next_thread() should be avoided, kernel/bpf/task_iter.c is the last user and the usage is wrong.

bpf_iter_task_next() can loop forever, "kit->pos == kit->task" can never happen if kit->pos execs. Change this code to use __next_thread().

With or without this change the usage of kit->pos/task and next_task() doesn't look nice, see the next patch.

maintainer Acked-by on v1 3/3
#8 lore raw

Re: [PATCH 3/3] bpf: bpf_iter_task_next: use next_task(kit->task) rather than next_task(kit->pos)

In reply to Oleg Nesterov
Quoted 5 lines · Oleg Nesterov

This looks more clear and simplifies the code. While at it, remove the unnecessary initialization of pos/task at the start of bpf_iter_task_new().

Note that we can even kill kit->task, we can just use pos->group_leader, but I don't understand the BUILD_BUG_ON() checks in bpf_iter_task_new().

Let us keep kit->task, which is used in later function bpf_iter_task_next(). The patch looks good to me.

Quoted 2 lines
#3 lore raw

Re: [PATCH 1/3] bpf: task_group_seq_get_next: use __next_thread() rather than next_thread()

In reply to Yonghong Song
Quoted 20 lines · Yonghong Song
Quoted 6 lines · Oleg Nesterov
kernel/bpf/task_iter.c +2 -1
@@ -70,15 +70,13 @@ static struct task_struct *task_group_seq_get_next(struct bpf_iter_seq_task_comm
 		return NULL;
 retry:
-	task = next_thread(task);
+	task = __next_thread(task);
+	if (!task)

kernel/bpf/task_iter.c static struct task_struct *task_group_seq_get_next(struct bpf_iter_seq_task_comm · line 77

Quoted 10 lines
kernel/bpf/task_iter.c +3 -6
+		return NULL;
 	next_tid = __task_pid_nr_ns(task, PIDTYPE_PID, common->ns);
-	if (!next_tid || next_tid == common->pid) {
-		/* Run out of tasks of a process.  The tasks of a
-		 * thread_group are linked as circular linked list.
-		 */
-		return NULL;
-	}
+	if (!next_tid)
+		goto retry;

Look at the code. Looks like next_tid should never be 0

...

Quoted 9 lines

pid_t __task_pid_nr_ns(struct task_struct *task, enum pid_type type,
struct pid_namespace *ns)

{
        pid_t nr = 0;

        rcu_read_lock();
        if (!ns)
                ns = task_active_pid_ns(current);
        nr = pid_nr_ns(rcu_dereference(*task_pid_ptr(task, type)), ns);
                                          ^^^^^^^^^^^^^^^^^^^^^^^^^

Please note that task_pid_ptr(task, type)) can return NULL if this task has already exited and called detach_pid().

detach_pid() does __change_pid(task, type, NULL), please note the

*pid_ptr = new; // NULL in this case

assignment in __change_pid().

IOW. The problem is not that ns can change, the problem is that task->thread_pid (and other pid links) can be NULL, and in this case pid_nr_ns() returns zero.

This code should be rewritten from the very beginning, it should not rely on pid_nr. If nothing else common->pid and/or pid_visiting can be reused. But currently my only concern is next_thread().

Quoted 1 line

Other than above, the change looks good to me.

Thanks for review!

Oleg.

#9 lore raw

Re: [PATCH 3/3] bpf: bpf_iter_task_next: use next_task(kit->task) rather than next_task(kit->pos)

In reply to Yonghong Song
Quoted 10 lines · Yonghong Song
Quoted 5 lines · Oleg Nesterov

This looks more clear and simplifies the code. While at it, remove the unnecessary initialization of pos/task at the start of bpf_iter_task_new().

Note that we can even kill kit->task, we can just use pos->group_leader, but I don't understand the BUILD_BUG_ON() checks in bpf_iter_task_new().

Let us keep kit->task, which is used in later function bpf_iter_task_next(). The patch looks good to me.

Yes, but it can use pos->group_leader instead of kit->task. But I agree, lets keep kit->task.

Quoted 1 line

Thanks!

Oleg.

#11 lore raw
In reply to Yonghong Song
Quoted 9 lines · Yonghong Song
Quoted 4 lines · Oleg Nesterov

Compile tested.

Every lockless usage of next_thread() was wrong, bpf/task_iter.c is the last user and is no exception.

It would be great if you can give more information in the commit message about why the usage of next_thread() is wrong in bpf/task_iter.c.

I tried to explain the problems in the changelogs:

1/3:

	task_group_seq_get_next() can return the group leader twice if it races
	with mt-thread exec which changes the group->leader's pid.

2/3:

	bpf_iter_task_next() can loop forever, "kit->pos == kit->task" can never
	happen if kit->pos execs.
Quoted 2 lines

IIUC, some information is presented in :
https://lore.kernel.org/all/[email protected]/

Yes, Linus and Eric suggest to simply kill next_thread(). I am not sure, this needs another discussion.

But as for bpf/task_iter.c... Even _if_ the usage was correct, this code simply doesn't need the "circular" next_thread(), NULL at the end simplifies the code.

Quoted 2 lines

Also, please add 'bpf' in the subject tag ([PATCH bpf 0/3]) to make it clear the patch should be applied to bpf tree.

OK, will do next time. Or should I resend this series with 'bpf' in the subject tag?

Thanks,

Oleg.

maintainer Acked-by
#4 lore raw

Re: [PATCH 1/3] bpf: task_group_seq_get_next: use __next_thread() rather than next_thread()

In reply to Oleg Nesterov
Quoted 37 lines · Oleg Nesterov
Quoted 18 lines · Yonghong Song
Quoted 6 lines · Oleg Nesterov
kernel/bpf/task_iter.c +2 -1
@@ -70,15 +70,13 @@ static struct task_struct *task_group_seq_get_next(struct bpf_iter_seq_task_comm
  		return NULL;
  retry:
-	task = next_thread(task);
+	task = __next_thread(task);
+	if (!task)

kernel/bpf/task_iter.c static struct task_struct *task_group_seq_get_next(struct bpf_iter_seq_task_comm · line 77

Quoted 10 lines
kernel/bpf/task_iter.c +3 -6
+		return NULL;
  	next_tid = __task_pid_nr_ns(task, PIDTYPE_PID, common->ns);
-	if (!next_tid || next_tid == common->pid) {
-		/* Run out of tasks of a process.  The tasks of a
-		 * thread_group are linked as circular linked list.
-		 */
-		return NULL;
-	}
+	if (!next_tid)
+		goto retry;

Look at the code. Looks like next_tid should never be 0

...

Quoted 9 lines

pid_t __task_pid_nr_ns(struct task_struct *task, enum pid_type type,
struct pid_namespace *ns)

{
         pid_t nr = 0;

         rcu_read_lock();
         if (!ns)
                 ns = task_active_pid_ns(current);
         nr = pid_nr_ns(rcu_dereference(*task_pid_ptr(task, type)), ns);
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^

Please note that task_pid_ptr(task, type)) can return NULL if this task has already exited and called detach_pid().

detach_pid() does __change_pid(task, type, NULL), please note the

Quoted 7 lines

*pid_ptr = new; // NULL in this case

assignment in __change_pid().

IOW. The problem is not that ns can change, the problem is that task->thread_pid (and other pid links) can be NULL, and in this case pid_nr_ns() returns zero.

Thanks for explanation. I certainly missed race between task iterator and __change_pid(). Then the patch looks good to me.

Quoted 11 lines

This code should be rewritten from the very beginning, it should not rely on pid_nr. If nothing else common->pid and/or pid_visiting can be reused. But currently my only concern is next_thread().

Quoted 1 line

Other than above, the change looks good to me.

Thanks for review!

Oleg.

maintainer
#12 lore raw
In reply to Oleg Nesterov
Quoted 23 lines · Oleg Nesterov
Quoted 7 lines · Yonghong Song
Quoted 4 lines · Oleg Nesterov

Compile tested.

Every lockless usage of next_thread() was wrong, bpf/task_iter.c is the last user and is no exception.

It would be great if you can give more information in the commit message about why the usage of next_thread() is wrong in bpf/task_iter.c.

I tried to explain the problems in the changelogs:

1/3:

	task_group_seq_get_next() can return the group leader twice if it races
	with mt-thread exec which changes the group->leader's pid.

2/3:

	bpf_iter_task_next() can loop forever, "kit->pos == kit->task" can never
	happen if kit->pos execs.
Quoted 2 lines

IIUC, some information is presented in :

   https://lore.kernel.org/all/[email protected]/

Yes, Linus and Eric suggest to simply kill next_thread(). I am not sure, this needs another discussion.

Quoted 8 lines

But as for bpf/task_iter.c... Even _if_ the usage was correct, this code simply doesn't need the "circular" next_thread(), NULL at the end simplifies the code.

Quoted 2 lines

Also, please add 'bpf' in the subject tag ([PATCH bpf 0/3]) to make it clear the patch should be applied to bpf tree.

OK, will do next time. Or should I resend this series with 'bpf' in the subject tag?

There is no need then. We can wait for maintainers who may or may not have additional requests.

Quoted 5 lines

Thanks,

Oleg.

#13 lore raw
In reply to Oleg Nesterov
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:

Hello:

This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:

Quoted 8 lines · On Tue, 14 Nov 2023 17:32:11 +0100 you wrote:

Compile tested.

Every lockless usage of next_thread() was wrong, bpf/task_iter.c is the last user and is no exception.

Oleg.

[...]

Here is the summary with links:
- [1/3] bpf: task_group_seq_get_next: use __next_thread() rather than next_thread()

    https://git.kernel.org/bpf/bpf-next/c/2d1618054f25
  - [2/3] bpf: bpf_iter_task_next: use __next_thread() rather than next_thread()
    https://git.kernel.org/bpf/bpf-next/c/5a34f9dabd9a
  - [3/3] bpf: bpf_iter_task_next: use next_task(kit->task) rather than next_task(kit->pos)
    https://git.kernel.org/bpf/bpf-next/c/ac8148d957f5

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html