Skip to contentSkip to conversation

kernel: release ptraced tasks before zap_pid_ns_processes

PATCH Applied Andrei Vagin 4 participants 6 messages Jan 10, 2019 – Feb 27, 2019

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 kernel: release ptraced tasks before zap_pid_ns_processes

Currently, exit_ptrace() adds all ptraced tasks in a dead list, than zap_pid_ns_processes() waits all tasks in a current pidns, and only then tasks from the dead list are released.

zap_pid_ns_processes() can stuck on waiting tasks from the dead list. In this case, we will have one unkillable process with one or more dead children.

Thanks to Oleg for the advice to release tasks in find_child_reaper().

Fixes 7c8bd2322c7f ("exit: ptrace: shift "reap dead" code from exit_ptrace() to forget_original_parent()")
Cc "Eric W. Biederman" <[email protected]>
Cc Andrew Morton <[email protected]>

---

v2: Oleg showed that ptraced tasks can be released in find_child_reaper(). This allows to avoid additional write_lock/unlock(tasklist), and another list_for_each_entry_safe(dead) loop is called only if it is actually needed.

 kernel/exit.c | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)
kernel/exit.c +10 -2
index 2d14979577ee..5df787a497f5 100644
@@ -558,12 +558,14 @@ static struct task_struct *find_alive_thread(struct task_struct *p)
 	return NULL;
 }
 
-static struct task_struct *find_child_reaper(struct task_struct *father)
+static struct task_struct *find_child_reaper(struct task_struct *father,
+						struct list_head *dead)
 	__releases(&tasklist_lock)
 	__acquires(&tasklist_lock)
 {
 	struct pid_namespace *pid_ns = task_active_pid_ns(father);
 	struct task_struct *reaper = pid_ns->child_reaper;
+	struct task_struct *p, *n;
 
 	if (likely(reaper != father))
 		return reaper;
@@ -579,6 +581,12 @@ static struct task_struct *find_child_reaper(struct task_struct *father)
 		panic("Attempted to kill init! exitcode=0x%08x\n",
 			father->signal->group_exit_code ?: father->exit_code);
 	}
+
+	list_for_each_entry_safe(p, n, dead, ptrace_entry) {
+		list_del_init(&p->ptrace_entry);
+		release_task(p);
+	}
+
 	zap_pid_ns_processes(pid_ns);
 	write_lock_irq(&tasklist_lock);
 
@@ -668,7 +676,7 @@ static void forget_original_parent(struct task_struct *father,
 		exit_ptrace(father, dead);
 
 	/* Can drop and reacquire tasklist_lock */
-	reaper = find_child_reaper(father);
+	reaper = find_child_reaper(father, dead);
 	if (list_empty(&father->children))
 		return;
 
Signature · 1 line
2.17.2
#1 lore raw
Quoted 8 lines · On Thu, 10 Jan 2019 09:52:00 -0800 Andrei Vagin <[email protected]> wrote:

Currently, exit_ptrace() adds all ptraced tasks in a dead list, than zap_pid_ns_processes() waits all tasks in a current pidns, and only then tasks from the dead list are released.

zap_pid_ns_processes() can stuck on waiting tasks from the dead list. In this case, we will have one unkillable process with one or more dead children.

Quoted 8 lines

Thanks to Oleg for the advice to release tasks in find_child_reaper().

Fixes 7c8bd2322c7f ("exit: ptrace: shift "reap dead" code from exit_ptrace() to forget_original_parent()")
Cc "Eric W. Biederman" <[email protected]>
Cc Andrew Morton <[email protected]>

Does this warrant a -stable backport? 7c8bd2322c7f was 4+ years ago...

#3 lore raw
In reply to Andrei Vagin
Quoted 47 lines · Andrei Vagin

Currently, exit_ptrace() adds all ptraced tasks in a dead list, than zap_pid_ns_processes() waits all tasks in a current pidns, and only then tasks from the dead list are released.

zap_pid_ns_processes() can stuck on waiting tasks from the dead list. In this case, we will have one unkillable process with one or more dead children.

Thanks to Oleg for the advice to release tasks in find_child_reaper().

Fixes 7c8bd2322c7f ("exit: ptrace: shift "reap dead" code from exit_ptrace() to forget_original_parent()")
Cc "Eric W. Biederman" <[email protected]>
Cc Andrew Morton <[email protected]>

---

v2: Oleg showed that ptraced tasks can be released in find_child_reaper(). This allows to avoid additional write_lock/unlock(tasklist), and another list_for_each_entry_safe(dead) loop is called only if it is actually needed.

 kernel/exit.c | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)
kernel/exit.c +3 -1
index 2d14979577ee..5df787a497f5 100644
@@ -558,12 +558,14 @@ static struct task_struct *find_alive_thread(struct task_struct *p)
 	return NULL;
 }
 
-static struct task_struct *find_child_reaper(struct task_struct *father)
+static struct task_struct *find_child_reaper(struct task_struct *father,
+						struct list_head *dead)
 	__releases(&tasklist_lock)
 	__acquires(&tasklist_lock)
 {
 	struct pid_namespace *pid_ns = task_active_pid_ns(father);
 	struct task_struct *reaper = pid_ns->child_reaper;
+	struct task_struct *p, *n;
 
 	if (likely(reaper != father))
 		return reaper;

kernel/exit.c static struct task_struct *find_child_reaper(struct task_struct *father) · line 589

Quoted 9 lines
kernel/exit.c +6 -0
@@ -579,6 +581,12 @@ static struct task_struct *find_child_reaper(struct task_struct *father)
 		panic("Attempted to kill init! exitcode=0x%08x\n",
 			father->signal->group_exit_code ?: father->exit_code);
 	}
+
+	list_for_each_entry_safe(p, n, dead, ptrace_entry) {
+		list_del_init(&p->ptrace_entry);
+		release_task(p);
+	}
+

Hi,

from our (SUSE) QA we received a report that this patch causes a performance decline in libmicro pthread_* benchmark as reported in:
https://bugzilla.suse.com/show_bug.cgi?id=1126762

I tried myself from the repo:
https://github.com/redhat-performance/libMicro

I ran
pthread_create -B 8 -C 200 -S

and with the patch applied:
# STATISTICS usecs/call (raw) usecs/call (outliers removed)
# mean 23.38611 17.29311

Without:
# mean 41.36539 39.21347

The values vary, but they are around 23 and 42, respectively.

The benchmark seems to create 8 (-B above) pthreads, does lock/unlock in them and then the threads exit. The benchmark reaps the threads via pthread_join. This all happens 200 times (-C above).

Any idea how to restore the performance close to the previous state?

kernel/exit.c static void forget_original_parent(struct task_struct *father, · line 681

Quoted 11 lines
kernel/exit.c +1 -1
 	zap_pid_ns_processes(pid_ns);
 	write_lock_irq(&tasklist_lock);
 
@@ -668,7 +676,7 @@ static void forget_original_parent(struct task_struct *father,
 		exit_ptrace(father, dead);
 
 	/* Can drop and reacquire tasklist_lock */
-	reaper = find_child_reaper(father);
+	reaper = find_child_reaper(father, dead);
 	if (list_empty(&father->children))
 		return;

thanks,

Signature · 2 lines
js
suse labs
#4 lore raw
Quoted 33 lines · Jiri Slaby
Quoted 19 lines · Andrei Vagin
kernel/exit.c +3 -1
@@ -558,12 +558,14 @@ static struct task_struct *find_alive_thread(struct task_struct *p)
 	return NULL;
 }
 
-static struct task_struct *find_child_reaper(struct task_struct *father)
+static struct task_struct *find_child_reaper(struct task_struct *father,
+						struct list_head *dead)
 	__releases(&tasklist_lock)
 	__acquires(&tasklist_lock)
 {
 	struct pid_namespace *pid_ns = task_active_pid_ns(father);
 	struct task_struct *reaper = pid_ns->child_reaper;
+	struct task_struct *p, *n;
 
 	if (likely(reaper != father))
 		return reaper;

kernel/exit.c static struct task_struct *find_child_reaper(struct task_struct *father) · line 589

Quoted 9 lines
kernel/exit.c +6 -0
@@ -579,6 +581,12 @@ static struct task_struct *find_child_reaper(struct task_struct *father)
 		panic("Attempted to kill init! exitcode=0x%08x\n",
 			father->signal->group_exit_code ?: father->exit_code);
 	}
+
+	list_for_each_entry_safe(p, n, dead, ptrace_entry) {
+		list_del_init(&p->ptrace_entry);
+		release_task(p);
+	}
+

Hi,

Quoted 3 lines

from our (SUSE) QA we received a report that this patch causes a performance decline in libmicro pthread_* benchmark as reported in:
https://bugzilla.suse.com/show_bug.cgi?id=1126762

Access Denied

Quoted 6 lines

I tried myself from the repo:
https://github.com/redhat-performance/libMicro

I ran
pthread_create -B 8 -C 200 -S

Quoted 6 lines

and with the patch applied:
# STATISTICS usecs/call (raw) usecs/call (outliers removed)
# mean 23.38611 17.29311

Without:
# mean 41.36539 39.21347

can't reproduce, I see the same numbers with or without this patch. However, I did "./bin/pthread_create -B 8 -C 200 -S" under KVM.

Quoted 3 lines

The benchmark seems to create 8 (-B above) pthreads, does lock/unlock in them and then the threads exit. The benchmark reaps the threads via pthread_join. This all happens 200 times (-C above).

Given that this test-case doesn't use CLONE_PID, I fail to understand how this patch can make any noticeable difference performance wise...

with this patch forget_original_parent() just passes the additional argument to find_child_reaper(), nothing else.

The extra list_for_each_entry_safe/release_task loop can't happen, and even if it could it shouldn't cause any performance regression too.

Quoted 1 line

Any idea how to restore the performance close to the previous state?

maybe you can try perf to find out where does this difference come from?

Oleg.

#5 lore raw
Quoted 16 lines · Oleg Nesterov
Quoted 3 lines

from our (SUSE) QA we received a report that this patch causes a performance decline in libmicro pthread_* benchmark as reported in:
https://bugzilla.suse.com/show_bug.cgi?id=1126762

Access Denied

Quoted 10 lines

I tried myself from the repo:
https://github.com/redhat-performance/libMicro

I ran
pthread_create -B 8 -C 200 -S

and with the patch applied:
# STATISTICS usecs/call (raw) usecs/call (outliers removed)
# mean 23.38611 17.29311

Quoted 5 lines
Quoted 2 lines

Without:
# mean 41.36539 39.21347

can't reproduce, I see the same numbers with or without this patch. However, I did "./bin/pthread_create -B 8 -C 200 -S" under KVM.

Correct. I did the tests also under KVM. The above difference happens when you compare (by stupid mistake) a LOCKDEP and non-LOCKDEP kernel -- sorry. So the proper results are comparably the same -- the difference is on the noise level. So it must be their setup broken too.

Thanks for the input anyway.

Signature · 2 lines
js
suse labs