Skip to contentSkip to conversation

bpf: task_group_seq_get_next: cleanup the usage of next_thread()

PATCH In review Oleg Nesterov 5 participants 16 messages Aug 21, 2023 – Aug 25, 2023

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 bpf: task_group_seq_get_next: cleanup the usage of next_thread()

1. find_pid_ns() + get_pid_task() under rcu_read_lock() guarantees that we can safely iterate the task->thread_group list. Even if this task exits right after get_pid_task() (or goto retry) and pid_alive() returns 0.

Kill the unnecessary pid_alive() check.

2. next_thread() simply can't return NULL, kill the bogus "if (!next_task)" check.

---
 kernel/bpf/task_iter.c | 7 -------
 1 file changed, 7 deletions(-)
kernel/bpf/task_iter.c +0 -7
index c4ab9d6cdbe9..4d1125108014 100644
@@ -75,15 +75,8 @@ static struct task_struct *task_group_seq_get_next(struct bpf_iter_seq_task_comm
 		return NULL;
 
 retry:
-	if (!pid_alive(task)) {
-		put_task_struct(task);
-		return NULL;
-	}
-
 	next_task = next_thread(task);
 	put_task_struct(task);
-	if (!next_task)
-		return NULL;
 
 	saved_tid = *tid;
 	*tid = __task_pid_nr_ns(next_task, PIDTYPE_PID, common->ns);
Signature · 1 line
2.25.1.362.g51ebf55
#1 lore raw
Quoted 4 lines · Oleg Nesterov

1. find_pid_ns() + get_pid_task() under rcu_read_lock() guarantees that we can safely iterate the task->thread_group list. Even if this task exits right after get_pid_task() (or goto retry) and pid_alive() returns 0 > Kill the unnecessary pid_alive() check.

This function will return next_task holding a refcount, and release the refcount until the next time calling the same function. Meanwhile, the returned task A may be killed, and its next task B may be killed after A as well, before calling this function again. However, even task B is destroyed (free), A's next is still pointing to task B. When this function is called again for the same iterator, it doesn't promise that B is still there.

Does that make sense to you?

Quoted 29 lines

2. next_thread() simply can't return NULL, kill the bogus "if (!next_task)" check.

---
  kernel/bpf/task_iter.c | 7 -------
  1 file changed, 7 deletions(-)
kernel/bpf/task_iter.c +0 -7
index c4ab9d6cdbe9..4d1125108014 100644
@@ -75,15 +75,8 @@ static struct task_struct *task_group_seq_get_next(struct bpf_iter_seq_task_comm
  		return NULL;
 
  retry:
-	if (!pid_alive(task)) {
-		put_task_struct(task);
-		return NULL;
-	}
-
  	next_task = next_thread(task);
  	put_task_struct(task);
-	if (!next_task)
-		return NULL;
 
  	saved_tid = *tid;
  	*tid = __task_pid_nr_ns(next_task, PIDTYPE_PID, common->ns);
#2 lore raw
Quoted 8 lines · Kui-Feng Lee
Quoted 4 lines · Oleg Nesterov

1. find_pid_ns() + get_pid_task() under rcu_read_lock() guarantees that we can safely iterate the task->thread_group list. Even if this task exits right after get_pid_task() (or goto retry) and pid_alive() returns 0 > Kill the unnecessary pid_alive() check.

Quoted 7 lines

This function will return next_task holding a refcount, and release the refcount until the next time calling the same function. Meanwhile, the returned task A may be killed, and its next task B may be killed after A as well, before calling this function again. However, even task B is destroyed (free), A's next is still pointing to task B. When this function is called again for the same iterator, it doesn't promise that B is still there.

Not sure I understand...

OK, if we have a task pointer with incremented refcount and do not hold rcu lock, then yes, you can't remove the pid_alive() check in this code:

	rcu_read_lock();
	if (pid_alive(task))
		do_something(next_thread(task));
	rcu_read_unlock();

because task and then task->next can exit and do call_rcu(delayed_put_task_struct) before we take rcu_read_lock().

But if you do something like

	rcu_read_lock();

	task = find_task_in_some_rcu_protected_list();
	do_something(next_thread(task));

	rcu_read_unlock();

then next_thread(task) should be safe without pid_alive().

And iiuc task_group_seq_get_next() always does

	rcu_read_lock();	// the caller does lock/unlock

	task = get_pid_task(pid, PIDTYPE_PID);
	if (!task)
		return;
	
	next_task = next_thread(task);

	rcu_read_unlock();

Yes, both task and task->next can exit right after get_pid_task(), but since can only happen after we took rcu_read_lock(), delayed_put_task_struct() can't be called until we drop rcu lock.

What have I missed?

Oleg.

#3 lore raw

So I still think the pid_alive() check should die...

and when I look at this code again I don't understand why does it abuse task_struct->usage, I'll send another patch on top of this one.

Quoted 61 lines · Oleg Nesterov
Quoted 8 lines · Kui-Feng Lee
Quoted 4 lines · Oleg Nesterov

1. find_pid_ns() + get_pid_task() under rcu_read_lock() guarantees that we can safely iterate the task->thread_group list. Even if this task exits right after get_pid_task() (or goto retry) and pid_alive() returns 0 > Kill the unnecessary pid_alive() check.

Quoted 7 lines

This function will return next_task holding a refcount, and release the refcount until the next time calling the same function. Meanwhile, the returned task A may be killed, and its next task B may be killed after A as well, before calling this function again. However, even task B is destroyed (free), A's next is still pointing to task B. When this function is called again for the same iterator, it doesn't promise that B is still there.

Not sure I understand...

OK, if we have a task pointer with incremented refcount and do not hold rcu lock, then yes, you can't remove the pid_alive() check in this code:

	rcu_read_lock();
	if (pid_alive(task))
		do_something(next_thread(task));
	rcu_read_unlock();

because task and then task->next can exit and do call_rcu(delayed_put_task_struct) before we take rcu_read_lock().

But if you do something like

	rcu_read_lock();

	task = find_task_in_some_rcu_protected_list();
	do_something(next_thread(task));

	rcu_read_unlock();

then next_thread(task) should be safe without pid_alive().

And iiuc task_group_seq_get_next() always does

	rcu_read_lock();	// the caller does lock/unlock

	task = get_pid_task(pid, PIDTYPE_PID);
	if (!task)
		return;

	next_task = next_thread(task);

	rcu_read_unlock();

Yes, both task and task->next can exit right after get_pid_task(), but since can only happen after we took rcu_read_lock(), delayed_put_task_struct() can't be called until we drop rcu lock.

What have I missed?

Oleg.

#4 lore raw
In reply to Oleg Nesterov
Quoted 53 lines · Oleg Nesterov
Quoted 8 lines · Kui-Feng Lee
Quoted 4 lines · Oleg Nesterov

1. find_pid_ns() + get_pid_task() under rcu_read_lock() guarantees that we can safely iterate the task->thread_group list. Even if this task exits right after get_pid_task() (or goto retry) and pid_alive() returns 0 > Kill the unnecessary pid_alive() check.

Quoted 7 lines

This function will return next_task holding a refcount, and release the refcount until the next time calling the same function. Meanwhile, the returned task A may be killed, and its next task B may be killed after A as well, before calling this function again. However, even task B is destroyed (free), A's next is still pointing to task B. When this function is called again for the same iterator, it doesn't promise that B is still there.

Not sure I understand...

OK, if we have a task pointer with incremented refcount and do not hold rcu lock, then yes, you can't remove the pid_alive() check in this code:

	rcu_read_lock();
	if (pid_alive(task))
		do_something(next_thread(task));
	rcu_read_unlock();

because task and then task->next can exit and do call_rcu(delayed_put_task_struct) before we take rcu_read_lock().

But if you do something like

	rcu_read_lock();

	task = find_task_in_some_rcu_protected_list();
	do_something(next_thread(task));

	rcu_read_unlock();

then next_thread(task) should be safe without pid_alive().

And iiuc task_group_seq_get_next() always does

	rcu_read_lock();	// the caller does lock/unlock

	task = get_pid_task(pid, PIDTYPE_PID);
	if (!task)
		return;
	
	next_task = next_thread(task);

	rcu_read_unlock();
Quoted 5 lines

Yes, both task and task->next can exit right after get_pid_task(), but since can only happen after we took rcu_read_lock(), delayed_put_task_struct() can't be called until we drop rcu lock.

What have I missed?

Then, it makes sense to me! Thank you for the explanation.

Quoted 3 lines

Oleg.

#5 lore raw

PATCH 1/1 bpf: task_group_seq_get_next: cleanup the usage of get/put_task_struct

In reply to Oleg Nesterov

get_pid_task() makes no sense, the code does put_task_struct() soon after. Use find_task_by_pid_ns() instead of find_pid_ns + get_pid_task and kill kill put_task_struct(), this allows to do get_task_struct() only once before return.

While at it, kill the unnecessary "if (!pid)" check in the "if (!*tid)" block, this matches the next usage of find_pid_ns() + get_pid_task() in this function.

---
 kernel/bpf/task_iter.c | 12 ++----------
 1 file changed, 2 insertions(+), 10 deletions(-)
kernel/bpf/task_iter.c +2 -10
index 4d1125108014..1589ec3faded 100644
@@ -42,9 +42,6 @@ static struct task_struct *task_group_seq_get_next(struct bpf_iter_seq_task_comm
 	if (!*tid) {
 		/* The first time, the iterator calls this function. */
 		pid = find_pid_ns(common->pid, common->ns);
-		if (!pid)
-			return NULL;
-
 		task = get_pid_task(pid, PIDTYPE_TGID);
 		if (!task)
 			return NULL;
@@ -66,17 +63,12 @@ static struct task_struct *task_group_seq_get_next(struct bpf_iter_seq_task_comm
 		return task;
 	}
 
-	pid = find_pid_ns(common->pid_visiting, common->ns);
-	if (!pid)
-		return NULL;
-
-	task = get_pid_task(pid, PIDTYPE_PID);
+	task = find_task_by_pid_ns(common->pid_visiting, common->ns);
 	if (!task)
 		return NULL;
 
 retry:
 	next_task = next_thread(task);
-	put_task_struct(task);
 
 	saved_tid = *tid;
 	*tid = __task_pid_nr_ns(next_task, PIDTYPE_PID, common->ns);
@@ -88,7 +80,6 @@ static struct task_struct *task_group_seq_get_next(struct bpf_iter_seq_task_comm
 		return NULL;
 	}
 
-	get_task_struct(next_task);
 	common->pid_visiting = *tid;
 
 	if (skip_if_dup_files && task->files == task->group_leader->files) {
@@ -96,6 +87,7 @@ static struct task_struct *task_group_seq_get_next(struct bpf_iter_seq_task_comm
 		goto retry;
 	}
 
+	get_task_struct(next_task);
 	return next_task;
 }
 
Signature · 1 line
2.25.1.362.g51ebf55
#6 lore raw

Re: [PATCH] bpf: task_group_seq_get_next: cleanup the usage of get/put_task_struct

Quoted 34 lines · Oleg Nesterov

get_pid_task() makes no sense, the code does put_task_struct() soon after. Use find_task_by_pid_ns() instead of find_pid_ns + get_pid_task and kill kill put_task_struct(), this allows to do get_task_struct() only once before return.

While at it, kill the unnecessary "if (!pid)" check in the "if (!*tid)" block, this matches the next usage of find_pid_ns() + get_pid_task() in this function.

---
  kernel/bpf/task_iter.c | 12 ++----------
  1 file changed, 2 insertions(+), 10 deletions(-)
kernel/bpf/task_iter.c +0 -5
index 4d1125108014..1589ec3faded 100644
@@ -42,9 +42,6 @@ static struct task_struct *task_group_seq_get_next(struct bpf_iter_seq_task_comm
  	if (!*tid) {
  		/* The first time, the iterator calls this function. */
  		pid = find_pid_ns(common->pid, common->ns);
-		if (!pid)
-			return NULL;
-
  		task = get_pid_task(pid, PIDTYPE_TGID);
  		if (!task)
  			return NULL;
@@ -66,17 +63,12 @@ static struct task_struct *task_group_seq_get_next(struct bpf_iter_seq_task_comm
  		return task;
  	}
 
-	pid = find_pid_ns(common->pid_visiting, common->ns);
-	if (!pid)

kernel/bpf/task_iter.c static struct task_struct *task_group_seq_get_next(struct bpf_iter_seq_task_comm · line 79

Quoted 10 lines
kernel/bpf/task_iter.c +1 -4
-		return NULL;
-
-	task = get_pid_task(pid, PIDTYPE_PID);
+	task = find_task_by_pid_ns(common->pid_visiting, common->ns);
  	if (!task)
  		return NULL;
 
  retry:
  	next_task = next_thread(task);
-	put_task_struct(task);

It called get_task_struct() against this task to hold a refcount at the previous time calling this function. When will it release the refcount?

Quoted 19 lines
kernel/bpf/task_iter.c +1 -1
  	saved_tid = *tid;
  	*tid = __task_pid_nr_ns(next_task, PIDTYPE_PID, common->ns);
@@ -88,7 +80,6 @@ static struct task_struct *task_group_seq_get_next(struct bpf_iter_seq_task_comm
  		return NULL;
  	}
 
-	get_task_struct(next_task);
  	common->pid_visiting = *tid;
 
  	if (skip_if_dup_files && task->files == task->group_leader->files) {
@@ -96,6 +87,7 @@ static struct task_struct *task_group_seq_get_next(struct bpf_iter_seq_task_comm
  		goto retry;
  	}
 
+	get_task_struct(next_task);
  	return next_task;
  }
 
#7 lore raw

Re: [PATCH] bpf: task_group_seq_get_next: cleanup the usage of get/put_task_struct

Quoted 42 lines · Kui-Feng Lee
Quoted 39 lines · Oleg Nesterov

get_pid_task() makes no sense, the code does put_task_struct() soon after. Use find_task_by_pid_ns() instead of find_pid_ns + get_pid_task and kill kill put_task_struct(), this allows to do get_task_struct() only once before return.

While at it, kill the unnecessary "if (!pid)" check in the "if (!*tid)" block, this matches the next usage of find_pid_ns() + get_pid_task() in this function.

---
  kernel/bpf/task_iter.c | 12 ++----------

  1 file changed, 2 insertions(+), 10 deletions(-)
kernel/bpf/task_iter.c +0 -3
index 4d1125108014..1589ec3faded 100644
@@ -42,9 +42,6 @@ static struct task_struct 
 *task_group_seq_get_next(struct bpf_iter_seq_task_comm
       if (!*tid) {
           /* The first time, the iterator calls this function. */
           pid = find_pid_ns(common->pid, common->ns);
-        if (!pid)
-            return NULL;
-
           task = get_pid_task(pid, PIDTYPE_TGID);
           if (!task)

              return NULL;

kernel/bpf/task_iter.c +1 -5
@@ -66,17 +63,12 @@ static struct task_struct 
 *task_group_seq_get_next(struct bpf_iter_seq_task_comm
           return task;
       }
-    pid = find_pid_ns(common->pid_visiting, common->ns);
-    if (!pid)
-        return NULL;
-
-    task = get_pid_task(pid, PIDTYPE_PID);
+    task = find_task_by_pid_ns(common->pid_visiting, common->ns);
Quoted 8 lines
Quoted 5 lines
kernel/bpf/task_iter.c +0 -0
       if (!task)
           return NULL;

  retry:
      next_task = next_thread(task);
-    put_task_struct(task);

It called get_task_struct() against this task to hold a refcount at the previous time calling this function. When will it release the refcount?

Oh! I missed the fact that the caller will handle it.

Quoted 17 lines
Quoted 17 lines

      saved_tid = *tid;
      *tid = __task_pid_nr_ns(next_task, PIDTYPE_PID, common->ns);

kernel/bpf/task_iter.c +1 -1
@@ -88,7 +80,6 @@ static struct task_struct 
 *task_group_seq_get_next(struct bpf_iter_seq_task_comm
           return NULL;
       }
-    get_task_struct(next_task);
       common->pid_visiting = *tid;
       if (skip_if_dup_files && task->files == task->group_leader->files) {
@@ -96,6 +87,7 @@ static struct task_struct 
 *task_group_seq_get_next(struct bpf_iter_seq_task_comm
           goto retry;
       }
+    get_task_struct(next_task);
       return next_task;
   }
maintainer Acked-by on v1 1/1
#8 lore raw

Re: [PATCH] bpf: task_group_seq_get_next: cleanup the usage of get/put_task_struct

In reply to Oleg Nesterov
Quoted 3 lines · Oleg Nesterov

get_pid_task() makes no sense, the code does put_task_struct() soon after. Use find_task_by_pid_ns() instead of find_pid_ns + get_pid_task and kill kill put_task_struct(), this allows to do get_task_struct() only once

remove the duplicated 'kill' in the above.

Quoted 7 lines

before return.

While at it, kill the unnecessary "if (!pid)" check in the "if (!*tid)" block, this matches the next usage of find_pid_ns() + get_pid_task() in this function.

LGTM.

#9 lore raw

Re: [PATCH] bpf: task_group_seq_get_next: cleanup the usage of get/put_task_struct

Quoted 8 lines · Yonghong Song
Quoted 3 lines · Oleg Nesterov

get_pid_task() makes no sense, the code does put_task_struct() soon after. Use find_task_by_pid_ns() instead of find_pid_ns + get_pid_task and kill kill put_task_struct(), this allows to do get_task_struct() only once

remove the duplicated 'kill' in the above.

Done,

Quoted 3 lines

LGTM.

Thanks, I'll send V2 with your ack included in a minute.

Oleg.

#10 lore raw

PATCH v2 1/1 bpf: task_group_seq_get_next: cleanup the usage of get/put_task_struct

In reply to Oleg Nesterov

get_pid_task() makes no sense, the code does put_task_struct() soon after. Use find_task_by_pid_ns() instead of find_pid_ns + get_pid_task and kill put_task_struct(), this allows to do get_task_struct() only once before return.

While at it, kill the unnecessary "if (!pid)" check in the "if (!*tid)" block, this matches the next usage of find_pid_ns() + get_pid_task() in this function.

---
 kernel/bpf/task_iter.c | 12 ++----------
 1 file changed, 2 insertions(+), 10 deletions(-)
kernel/bpf/task_iter.c +2 -10
index 4d1125108014..1589ec3faded 100644
@@ -42,9 +42,6 @@ static struct task_struct *task_group_seq_get_next(struct bpf_iter_seq_task_comm
 	if (!*tid) {
 		/* The first time, the iterator calls this function. */
 		pid = find_pid_ns(common->pid, common->ns);
-		if (!pid)
-			return NULL;
-
 		task = get_pid_task(pid, PIDTYPE_TGID);
 		if (!task)
 			return NULL;
@@ -66,17 +63,12 @@ static struct task_struct *task_group_seq_get_next(struct bpf_iter_seq_task_comm
 		return task;
 	}
 
-	pid = find_pid_ns(common->pid_visiting, common->ns);
-	if (!pid)
-		return NULL;
-
-	task = get_pid_task(pid, PIDTYPE_PID);
+	task = find_task_by_pid_ns(common->pid_visiting, common->ns);
 	if (!task)
 		return NULL;
 
 retry:
 	next_task = next_thread(task);
-	put_task_struct(task);
 
 	saved_tid = *tid;
 	*tid = __task_pid_nr_ns(next_task, PIDTYPE_PID, common->ns);
@@ -88,7 +80,6 @@ static struct task_struct *task_group_seq_get_next(struct bpf_iter_seq_task_comm
 		return NULL;
 	}
 
-	get_task_struct(next_task);
 	common->pid_visiting = *tid;
 
 	if (skip_if_dup_files && task->files == task->group_leader->files) {
@@ -96,6 +87,7 @@ static struct task_struct *task_group_seq_get_next(struct bpf_iter_seq_task_comm
 		goto retry;
 	}
 
+	get_task_struct(next_task);
 	return next_task;
 }
 
Signature · 1 line
2.25.1.362.g51ebf55
maintainer
#11 lore raw

Re: [PATCH V2] bpf: task_group_seq_get_next: cleanup the usage of get/put_task_struct

Hi Oleg,

Quoted 11 lines · Oleg Nesterov

get_pid_task() makes no sense, the code does put_task_struct() soon after. Use find_task_by_pid_ns() instead of find_pid_ns + get_pid_task and kill put_task_struct(), this allows to do get_task_struct() only once before return.

While at it, kill the unnecessary "if (!pid)" check in the "if (!*tid)" block, this matches the next usage of find_pid_ns() + get_pid_task() in this function.

Could you rebase this against bpf-next tree so this can run through our BPF CI? Right now the CI cannot pick the patch up due to merge conflict [0].

Thanks,
Daniel

[0] https://patchwork.kernel.org/project/netdevbpf/patch/[email protected]/

#12 lore raw

Re: [PATCH V2] bpf: task_group_seq_get_next: cleanup the usage of get/put_task_struct

Quoted 8 lines · Daniel Borkmann

Could you rebase this against bpf-next tree so this can run through our BPF CI? Right now the CI cannot pick the patch up due to merge conflict [0].

Thanks,
Daniel

[0] https://patchwork.kernel.org/project/netdevbpf/patch/[email protected]/

The merge failed because this patch depends on

[PATCH] bpf: task_group_seq_get_next: cleanup the usage of next_thread()

in this thread. But please forget.

I've sent the new series. It would be nice if you can test at least 1-5, the last 6/6 depends on

	[PATCH 1/2] introduce __next_thread(), fix next_tid() vs exec() race
	https://lore.kernel.org/all/[email protected]/

which was not merged yet.

Oleg.

#13 lore raw
In reply to Oleg Nesterov

OK, it seems that you are not going to take these preparatory cleanups ;)

I'll resend along with the s/next_thread/__next_thread/ change. I was going to do the last change later, but this recent discussion
https://lore.kernel.org/all/[email protected]/
makes me think we should do this right now.

Quoted 39 lines · Oleg Nesterov

1. find_pid_ns() + get_pid_task() under rcu_read_lock() guarantees that we can safely iterate the task->thread_group list. Even if this task exits right after get_pid_task() (or goto retry) and pid_alive() returns 0.

Kill the unnecessary pid_alive() check.

2. next_thread() simply can't return NULL, kill the bogus "if (!next_task)" check.

---
 kernel/bpf/task_iter.c | 7 -------
 1 file changed, 7 deletions(-)
kernel/bpf/task_iter.c +0 -8
index c4ab9d6cdbe9..4d1125108014 100644
@@ -75,15 +75,8 @@ static struct task_struct *task_group_seq_get_next(struct bpf_iter_seq_task_comm
 		return NULL;
 
 retry:
-	if (!pid_alive(task)) {
-		put_task_struct(task);
-		return NULL;
-	}
-
 	next_task = next_thread(task);
 	put_task_struct(task);
-	if (!next_task)
-		return NULL;
 
 	saved_tid = *tid;
 	*tid = __task_pid_nr_ns(next_task, PIDTYPE_PID, common->ns);
--

2.25.1.362.g51ebf55

#14 lore raw
Quoted 7 lines · Oleg Nesterov

OK, it seems that you are not going to take these preparatory cleanups ;)

I'll resend along with the s/next_thread/__next_thread/ change. I was going to do the last change later, but this recent discussion
https://lore.kernel.org/all/[email protected]/
makes me think we should do this right now.

For the record I find this code confusing, and wrong.

It looks like it wants to keep the task_struct pointer or possibly the struct pid pointer like proc does, but then it winds up keeping a userspace pid value and regenerating both the struct pid pointer and the struct task_struct pointer.

Which means that task_group_seq_get_next is unnecessarily slow and has a built in race condition which means it could wind up iterating through a different process.

This whole thing looks to be a bad (aka racy) reimplementation of first_tid and next_tid from proc. I thought the changes were to adapt to the needs of bpf, but on closer examination the code is just racy.

For this code to be correct bpf_iter_seq_task_common needs to store at a minimum a struct pid pointer.

Oleg your patch makes it easier to see what the how far this is from first_tid/next_tid in proc.

Eric

#15 lore raw
Quoted 2 lines · Eric W. Biederman

For the record I find this code confusing, and wrong.

Oh, yes...

Quoted 3 lines

and has
a built in race condition which means it could wind up iterating through a different process.

Yes, common->pid and/or common->pid_visiting can be reused

but I am not going to try to fix this ;)

Quoted 1 line

Thanks!

Oleg.