Skip to contentSkip to conversation

init's children list is long and slows reaping children.

DISCUSSION Oleg Nesterov 6 participants 23 messages Apr 6, 2007 – Apr 8, 2007

lore
Share & tools
lore
#0 lore raw
Quoted 6 lines · Robin Holt

wait_task_zombie() is taking many seconds to get through the list. For the case of a modprobe, stop_machine creates one thread per cpu (remember big number). All are parented to init and their exit will cause wait_task_zombie to scan multiple times most of the way through this very long list looking for threads which need to be reaped.

Could you try this patch

	http://marc.info/?l=linux-kernel&m=117337194209912

?

It can't solve the whole problem, but at least an exiting kernel thread won't kick init.

Oleg.

#1 lore raw
Quoted 8 lines · Oleg Nesterov
Quoted 6 lines · Robin Holt

wait_task_zombie() is taking many seconds to get through the list. For the case of a modprobe, stop_machine creates one thread per cpu (remember big number). All are parented to init and their exit will cause wait_task_zombie to scan multiple times most of the way through this very long list looking for threads which need to be reaped.

Quoted 8 lines

Could you try this patch

	http://marc.info/?l=linux-kernel&m=117337194209912

?

It can't solve the whole problem, but at least an exiting kernel thread won't kick init.

At first glance your patch looks reasonable.

Unfortunately it only applies to the rare thread that calls daemonize, and not also to kernel/kthread/kthread() which means it will miss many of our current kernel threads.

For the sgi problem I doubt it is will make a difference but the change is certainly worth making.

Eric

#2 lore raw
Quoted 17 lines · Eric W. Biederman
Quoted 14 lines · Oleg Nesterov
Quoted 6 lines · Robin Holt

wait_task_zombie() is taking many seconds to get through the list. For the case of a modprobe, stop_machine creates one thread per cpu (remember big number). All are parented to init and their exit will cause wait_task_zombie to scan multiple times most of the way through this very long list looking for threads which need to be reaped.

Could you try this patch

	http://marc.info/?l=linux-kernel&m=117337194209912

?

Quoted 8 lines
Quoted 2 lines

It can't solve the whole problem, but at least an exiting kernel thread won't kick init.

At first glance your patch looks reasonable.

Unfortunately it only applies to the rare thread that calls daemonize, and not also to kernel/kthread/kthread() which means it will miss many of our current kernel threads.

Note that a thread created by kthread_create() has ->parent == "[kthread]", not /sbin/init (unless it was created before core_initcall). So I don't really understand how stop_machine() creates the threads parented to init.

Quoted 1 line

For the sgi problem I doubt it is will make a difference

most probably you are right.

Oleg.

#3 lore raw
Quoted 9 lines · Oleg Nesterov
Quoted 5 lines

At first glance your patch looks reasonable.

Unfortunately it only applies to the rare thread that calls daemonize, and not also to kernel/kthread/kthread() which means it will miss many of our current kernel threads.

Note that a thread created by kthread_create() has ->parent == "[kthread]", not /sbin/init (unless it was created before core_initcall). So I don't really understand how stop_machine() creates the threads parented to init.

Duh. I was looking for that detail but the code was sufficiently abstracted that I missed it skimming through. Now I see it thanks. That at least explains how a handful of threads created with kthread_create have init as their parent.

My gut feel says it makes sense to change reparent_to_init to reparent_to_kthread. Which will get the all of the kernel daemons under a single parent. Further I think it makes sense to see if we can start kthread earlier. Having kthread have pid == 2 would be nice. That way we can kill the conditional that is only used during early initialization (make the code simpler and more predictable).

Since we seem to need kthread for the work of forking kernel threads, I don't see much point in not having a proper process tree, and a single extra child for pid == 1 isn't going to hurt anyone.

Eric

#4 lore raw
Quoted 9 lines · Eric W. Biederman
Quoted 6 lines · Oleg Nesterov
Quoted 5 lines

At first glance your patch looks reasonable.

Unfortunately it only applies to the rare thread that calls daemonize, and not also to kernel/kthread/kthread() which means it will miss many of our current kernel threads.

Quoted 8 lines
Quoted 3 lines

Note that a thread created by kthread_create() has ->parent == "[kthread]", not /sbin/init (unless it was created before core_initcall). So I don't really understand how stop_machine() creates the threads parented to init.

Duh. I was looking for that detail but the code was sufficiently abstracted that I missed it skimming through. Now I see it thanks. That at least explains how a handful of threads created with kthread_create have init as their parent.

Oops. I misread stop_machine(), it does kernel_thread(), not kthread_create(). So "stopmachine" threads are all re-parented to init when the caller exits. I think it makes sense to set ->exit_state = -1 in stopmachine(), regadless of any other changes.

Quoted 6 lines

My gut feel says it makes sense to change reparent_to_init to reparent_to_kthread. Which will get the all of the kernel daemons under a single parent. Further I think it makes sense to see if we can start kthread earlier. Having kthread have pid == 2 would be nice. That way we can kill the conditional that is only used during early initialization (make the code simpler and more predictable).

Yes. But we can't create helper_wq ([kthread]) before init_workqueues(). We can call helper_init() from init_workqueues() before creating "events" though.

Oleg.

#5 lore raw
Quoted 5 lines · Oleg Nesterov

Oops. I misread stop_machine(), it does kernel_thread(), not kthread_create(). So "stopmachine" threads are all re-parented to init when the caller exits. I think it makes sense to set ->exit_state = -1 in stopmachine(), regadless of any other changes.

I agree that "->exit_state = -1" makes sense, but I disagree in that I don't think it *matters*.

Most of the problematic kernel threads are long-running (as in "never exit"). Things like eventd, khelper, migration-threads etc will have init as their parent and never actually exit, so their ->exit_state doesn't matter. What matters is that they are on the children list, so when you have 1024 CPU's, and init has many thousand of these per-cpu threads as its children, then the *user* threads (that do exit) will cause problems!

I'd almost prefer to just not add kernel threads to any parent process list *at*all*.

Linus

#6 lore raw
Quoted 2 lines · Linus Torvalds

I'd almost prefer to just not add kernel threads to any parent process list *at*all*.

i think part of the problem is the legacy that the list is artificially
unified: tasks that 'will possibly exit' are on the same list as tasks that 'have already exited'. If we split it up into its natural data structure, having a list of tasks that are there and do not intend to exit, plus a separate list of tasks that are exiting and want to notify their parent, all this scanning goes away. I can see no real reason for this other than legacy - i dont think the semantics of the wait4() API force us to scan all those threads.

putting the freshly reaped tasks at the 'head' of the list is just a fancy (and incomplete) way of splitting the list up into two lists, and i'd advocate a clean split. Just like have have split the ptrace_list away from the main list too.

Ingo

#7 lore raw
Quoted 3 lines · Ingo Molnar

putting the freshly reaped tasks at the 'head' of the list is just a fancy (and incomplete) way of splitting the list up into two lists, and i'd advocate a clean split. Just like have have split the ptrace_list

                                          ^---s/have/we
Quoted 1 line

away from the main list too.

and in this sense your suggestion will happen:

Quoted 2 lines
Quoted 2 lines

I'd almost prefer to just not add kernel threads to any parent process list *at*all*.

because that way kernel threads will never show up in the to-be-reaped list, they'll only show up in the main list that represents the tree of tasks.

Ingo

#8 lore raw
In reply to Linus Torvalds
Quoted 17 lines · Linus Torvalds
Quoted 5 lines · Oleg Nesterov

Oops. I misread stop_machine(), it does kernel_thread(), not kthread_create(). So "stopmachine" threads are all re-parented to init when the caller exits. I think it makes sense to set ->exit_state = -1 in stopmachine(), regadless of any other changes.

I agree that "->exit_state = -1" makes sense, but I disagree in that I don't think it *matters*.

Most of the problematic kernel threads are long-running (as in "never exit"). Things like eventd, khelper, migration-threads etc will have init as their parent and never actually exit, so their ->exit_state doesn't matter. What matters is that they are on the children list, so when you have 1024 CPU's, and init has many thousand of these per-cpu threads as its children, then the *user* threads (that do exit) will cause problems!

Quoted 2 lines

I'd almost prefer to just not add kernel threads to any parent process list *at*all*.

Oleg is coming from a different case where it was found that exiting kernel threads were causing problems for nash when nash was run as init in an initramfs. While I think that case is likely a user space bug because nash should check the pid from waidpid before assuming the process it was waiting for returned.

I do think you can construct valid embedded cases where you can prove there that among the user space processes certain actions are safe when they exit that a spare kernel space thread could invalidate.

It just happens that the exit signal of kernel threads just matters on the other end of system size.

Eric

#9 lore raw
Quoted 7 lines · Eric W. Biederman

Oleg is coming from a different case where it was found that exiting kernel threads were causing problems for nash when nash was run as init in an initramfs. While I think that case is likely a user space bug because nash should check the pid from waidpid before assuming the process it was waiting for returned.

Are you saying waitpid() (wait4) *with a pid specified* can return another pid? That definitely sounds like a bug.

-hpa

#10 lore raw
Quoted 11 lines · H. Peter Anvin
Quoted 7 lines · Eric W. Biederman

Oleg is coming from a different case where it was found that exiting kernel threads were causing problems for nash when nash was run as init in an initramfs. While I think that case is likely a user space bug because nash should check the pid from waidpid before assuming the process it was waiting for returned.

Are you saying waitpid() (wait4) *with a pid specified* can return another pid? That definitely sounds like a bug.

No.

For the full context look back a couple of messages.

I'm guessing the issue is nash just calls wait and doesn't check the returned pid value, assuming it is the only child it forked returning. Which is valid except when you are running as pid == 1.

Eric

#11 lore raw
Quoted 4 lines · Eric W. Biederman
Quoted 3 lines
Quoted 1 line

Are you saying waitpid() (wait4) *with a pid specified* can return another pid? That definitely sounds like a bug.

Quoted 8 lines

No.

For the full context look back a couple of messages.

I'm guessing the issue is nash just calls wait and doesn't check the returned pid value, assuming it is the only child it forked returning. Which is valid except when you are running as pid == 1.

OK, userspace error :)

-hpa

#12 lore raw
In reply to Eric W. Biederman
Quoted 4 lines · Eric W. Biederman

I'm guessing the issue is nash just calls wait and doesn't check the returned pid value, assuming it is the only child it forked returning. Which is valid except when you are running as pid == 1.

Hm, that's always a bug; a process can always have children it doesn't expect (think forking, and then the parent execs).

J

#13 lore raw
Quoted 9 lines · Jeremy Fitzhardinge
Quoted 4 lines · Eric W. Biederman

I'm guessing the issue is nash just calls wait and doesn't check the returned pid value, assuming it is the only child it forked returning. Which is valid except when you are running as pid == 1.

Hm, that's always a bug; a process can always have children it doesn't expect (think forking, and then the parent execs).

Not if the process already forked.

But yes, in general this is a major screwup.

-hpa

#14 lore raw
In reply to Linus Torvalds
Quoted 18 lines · Linus Torvalds
Quoted 5 lines · Oleg Nesterov

Oops. I misread stop_machine(), it does kernel_thread(), not kthread_create(). So "stopmachine" threads are all re-parented to init when the caller exits. I think it makes sense to set ->exit_state = -1 in stopmachine(), regadless of any other changes.

I agree that "->exit_state = -1" makes sense, but I disagree in that I don't think it *matters*.

Most of the problematic kernel threads are long-running (as in "never exit"). Things like eventd, khelper, migration-threads etc will have init as their parent and never actually exit, so their ->exit_state doesn't matter. What matters is that they are on the children list, so when you have 1024 CPU's, and init has many thousand of these per-cpu threads as its children, then the *user* threads (that do exit) will cause problems!

Quoted 2 lines

I'd almost prefer to just not add kernel threads to any parent process list *at*all*.

Yes sure, I didn't argue with that. However, "->exit_state = -1" does matter, we can't detach process unless we make it auto-reap.

Perhaps,

kernel/exit.c renamed from kernel/exit.c~ +1 -4
@@ -275,10 +275,7 @@ static void reparent_to_init(void)
 	remove_parent(current);
 	current->parent = child_reaper(current);
 	current->real_parent = child_reaper(current);
-	add_parent(current);
-
-	/* Set the exit signal to SIGCHLD so we signal init on exit */
-	current->exit_signal = SIGCHLD;
+	current->exit_signal = -1;
 
 	if (!has_rt_policy(current) && (task_nice(current) < 0))
 		set_user_nice(current, 0);

is enough. init is still our parent (make ps happy), but it can't see us, we are not on ->children list.

Off course, we also need to add preparent_to_init() to kthread() and (say) stopmachine(). Or we can create kernel_thread_detached() and modify callers to use it.

Oleg.

#15 lore raw
Quoted 9 lines · Oleg Nesterov
Quoted 2 lines

I'd almost prefer to just not add kernel threads to any parent process list *at*all*.

Yes sure, I didn't argue with that. However, "->exit_state = -1" does matter, we can't detach process unless we make it auto-reap.

Off course, we also need to add preparent_to_init() to kthread() and (say) stopmachine(). Or we can create kernel_thread_detached() and modify callers to use it.

this isnt a kernel-thread special case. The right solution IMO is to first migrate wait4()'s ->children use over to a new p->exiting_children list and then to gradually get rid of all remaining uses of p->children. (the first patch of which i sent a few minutes ago)

that way wait4() will be sped up, and quite dramatically i believe. No need to deal with kthreads here at all - those just wont ever show up in the ->exiting_children list. Am i missing something?

Ingo

#16 lore raw
Quoted 13 lines · Ingo Molnar
Quoted 9 lines · Oleg Nesterov
Quoted 2 lines

I'd almost prefer to just not add kernel threads to any parent process list *at*all*.

Yes sure, I didn't argue with that. However, "->exit_state = -1" does matter, we can't detach process unless we make it auto-reap.

Off course, we also need to add preparent_to_init() to kthread() and (say) stopmachine(). Or we can create kernel_thread_detached() and modify callers to use it.

Quoted 8 lines

this isnt a kernel-thread special case. The right solution IMO is to first migrate wait4()'s ->children use over to a new p->exiting_children list and then to gradually get rid of all remaining uses of p->children. (the first patch of which i sent a few minutes ago)

that way wait4() will be sped up, and quite dramatically i believe. No need to deal with kthreads here at all - those just wont ever show up in the ->exiting_children list. Am i missing something?

Probably it is I who missed something :)

But why can't we do both changes? I think it is just ugly to use init to reap the kernel thread. Ok, wait4() can find zombie quickly if we do the ->children split. But /sbin/init could be swapped out, we still need to deliver SIGCHLD, etc.

And I personally agree with Linus, it is nice to hide the kernel threads from /sbin/init (or whatever) completely.

No?

Oleg.

#17 lore raw
Quoted 9 lines · Oleg Nesterov

Probably it is I who missed something :)

But why can't we do both changes? I think it is just ugly to use init to reap the kernel thread. Ok, wait4() can find zombie quickly if we do the ->children split. But /sbin/init could be swapped out, we still need to deliver SIGCHLD, etc.

And I personally agree with Linus, it is nice to hide the kernel threads from /sbin/init (or whatever) completely.

yeah, they should all be made auto-reap - nobody is really interested in their wait4()-driven completion. (any completion they do is via other mechanisms)

Ingo

#18 lore raw
In reply to Oleg Nesterov

kernel/exit.c static void reparent_to_init(void) · line 277

Quoted 7 lines · Oleg Nesterov
kernel/exit.c renamed from kernel/exit.c~ +0 -0
@@ -275,10 +275,7 @@ static void reparent_to_init(void)
 	remove_parent(current);
 	current->parent = child_reaper(current);
 	current->real_parent = child_reaper(current);

Just noticed. Eric, is it ok we are using child_reaper() here? This is for kernel threads only, shouldn't we always use init_pid_ns.child_reaper ? (yes, it is the same currently).

Oleg.

#19 lore raw
Quoted 12 lines · Oleg Nesterov

kernel/exit.c static void reparent_to_init(void) · line 277

Quoted 7 lines · Oleg Nesterov
kernel/exit.c renamed from kernel/exit.c~ +0 -0
@@ -275,10 +275,7 @@ static void reparent_to_init(void)
 	remove_parent(current);
 	current->parent = child_reaper(current);
 	current->real_parent = child_reaper(current);

Just noticed. Eric, is it ok we are using child_reaper() here? This is for kernel threads only, shouldn't we always use init_pid_ns.child_reaper ? (yes, it is the same currently).

Yes. It is a bug, and the description reparent_to_init is likewise buggy.

I have recently realized that a number of the inital pid namespace patches have stupid little bugs in them like this. I am upset with myself for failing to code review them closely before the went in. Live and learn I guess.

So when I saw this I added this to my growing list of things I needed to review before I could assume they were done correctly.

Feel free to fix it. I'm still a couple days out from having enough time before I can start working on pid related issues.

Eric

#20 lore raw
In reply to Oleg Nesterov
Quoted 18 lines · Oleg Nesterov

Perhaps,

kernel/exit.c renamed from kernel/exit.c~ +1 -4
@@ -275,10 +275,7 @@ static void reparent_to_init(void)
 	remove_parent(current);
 	current->parent = child_reaper(current);
 	current->real_parent = child_reaper(current);
-	add_parent(current);
-
-	/* Set the exit signal to SIGCHLD so we signal init on exit */
-	current->exit_signal = SIGCHLD;
+	current->exit_signal = -1;
 
 	if (!has_rt_policy(current) && (task_nice(current) < 0))
 		set_user_nice(current, 0);
Quoted 2 lines

is enough. init is still our parent (make ps happy), but it can't see us, we are not on ->children list.

OK, this doesn't work. A multi-threaded init may do execve(). Can we forbid exec() from non-main thread? init is special anyway. We can simplify de_thread() in this case, and kill tasklist_lock around zap_other_threads().

So, we can re-parent a kernel thread to swapper. In that case it doesn't matter if we put task on ->children list or not.

User-visible change. Acceptable?

Quoted 3 lines

Off course, we also need to add preparent_to_init() to kthread() and (say) stopmachine(). Or we can create kernel_thread_detached() and modify callers to use it.

It would be very nice to introduce CLONE_KERNEL_THREAD instead, then

	--- kernel/fork.c~	2007-04-07 20:11:14.000000000 +0400
	+++ kernel/fork.c	2007-04-07 23:40:35.000000000 +0400
	@@ -1159,7 +1159,8 @@ static struct task_struct *copy_process(
		p->parent_exec_id = p->self_exec_id;
	 
		/* ok, now we should be set up.. */
	-	p->exit_signal = (clone_flags & CLONE_THREAD) ? -1 : (clone_flags & CSIGNAL);
	+	p->exit_signal = (clone_flags & (CLONE_THREAD|CLONE_KERNEL_THREAD))
	+					? -1 : (clone_flags & CSIGNAL);
		p->pdeath_signal = 0;
		p->exit_state = 0;
	 
	@@ -1196,6 +1197,8 @@ static struct task_struct *copy_process(
		/* CLONE_PARENT re-uses the old parent */
		if (clone_flags & (CLONE_PARENT|CLONE_THREAD))
			p->parent = current->parent;
	+	else if (unlikely(clone_flags & CLONE_KERNEL_THREAD))
	+		p->parent = &init_task;
		else
			p->parent = current;

That is all, very simple. However, in that case we should introduce

	#define SYS_CLONE_MASK	(~CLONE_KERNEL_THREAD)

and change every sys_clone() implementation to filter out non-SYS_CLONE_MASK flags. This is trivial (and imho useful), but

	arch/sparc/kernel/process.c
	arch/sparc64/kernel/process.c
	arch/m68knommu/kernel/process.c
	arch/m68k/kernel/process.c
	arch/alpha/kernel/entry.S
	arch/h8300/kernel/process.c
	arch/v850/kernel/process.c
	arch/frv/kernel/kernel_thread.S
	arch/frv/kernel/kernel_thread.S
	arch/powerpc/kernel/misc_32.S
	arch/powerpc/kernel/misc_64.S

implement kthread_create() via sys_clone(). This means that without additional effort from mainteners these arches can't take advantage of CLONE_KERNEL_THREAD.

Dear CC list, any advice?

Oleg.

#21 lore raw
Quoted 16 lines · Oleg Nesterov
Quoted 15 lines · Oleg Nesterov

Perhaps,

kernel/exit.c renamed from kernel/exit.c~ +1 -4
@@ -275,10 +275,7 @@ static void reparent_to_init(void)
 	remove_parent(current);
 	current->parent = child_reaper(current);
 	current->real_parent = child_reaper(current);
-	add_parent(current);
-
-	/* Set the exit signal to SIGCHLD so we signal init on exit */
-	current->exit_signal = SIGCHLD;
+	current->exit_signal = -1;
 
Quoted 7 lines
Quoted 5 lines
kernel/exit.c renamed from kernel/exit.c~ +0 -0
 	if (!has_rt_policy(current) && (task_nice(current) < 0))
 		set_user_nice(current, 0);

is enough. init is still our parent (make ps happy), but it can't see us, we are not on ->children list.

OK, this doesn't work. A multi-threaded init may do execve().

Good catch. daemonize must die!

Quoted 2 lines

So, we can re-parent a kernel thread to swapper. In that case it doesn't matter if we put task on ->children list or not.

Yes. We can.

Quoted 1 line

User-visible change. Acceptable?

We would have user visible changes when we changed to ktrhead_create anyway, and it's none of user space's business so certainly.

Quoted 5 lines
Quoted 3 lines

Off course, we also need to add preparent_to_init() to kthread() and (say) stopmachine(). Or we can create kernel_thread_detached() and modify callers to use it.

It would be very nice to introduce CLONE_KERNEL_THREAD instead, then

If we are going to do something to copy_process and the like let's take this one step farther. Let's pass in the value of the task to copy.

Then we can add a wrapper around copy_process to build kernel_thread something like:

struct task_struct *__kernel_thread(int (*fn)(void *), void * arg,
unsigned long flags)

{
	struct task_struct *task;
	struct pt_regs regs, *reg;

        reg = kernel_thread_regs(&regs, fn, arg);
	task = copy_process(&init_task, flags, 0, reg, 0, NULL, NULL, NULL, 0);
	if (!IS_ERR(task))
		wake_up_new_task(task, flags);

	return task;
}

long kernel_thread(int (*fn)(void *), void * arg, unsigned long flags)

{
	struct task_struct *task;
	task = __kernel_thread(fn, arg, flags);
        if (IS_ERR(task))
        	return PTR_ERR(task);
	return task->pid;               
}

After that daemonize just becomes:

void daemonize(const char *name, ...)

{
	va_list args;

	va_start(args, name);
	vsnprintf(current->comm, sizeof(current->comm), name, args);
	va_end(args);
}

And kthread_create becomes:

struct task_struct *kthread_create(int (*threadfn)(void *data),

				   void *data,
				   const char namefmt[],
				   ...)
{
	struct kthread_create_info create;
	struct task_struct *task;

	create.threadfn = threadfn;
	create.data = data;

	/* We want our own signal handler (we take no signals by default). */
        task = __kernel_thread(kthread, create, CLONE_FS | CLONE_FILES | SIGCHLD);
	if (!IS_ERR(task)) {
		va_list args;
		va_start(args, namefmt);
		vsnprintf(task->comm, sizeof(task->comm),  namefmt, args);
		va_end(args);
	}
        return task;
}

If we are willing to go that far. I think it is worth it to touch the architecture specific code. As that removes an unnecessary wait queue, and ensures that kernel threads always start with a consistent state. So it is a performance, scalability and simplicity boost.

Otherwise we should just put the code in the callers of kernel_thread like we do today.

I don't have the energy to rework all of th architecture or even a noticeable fraction of them right now. I have to much on my plate. A non-architecture specific solution looks like a fairly simple patch and I might get around to it one of these times..

Eric

#22 lore raw
Quoted 18 lines · Eric W. Biederman
Quoted 15 lines · Oleg Nesterov
Quoted 14 lines · Oleg Nesterov
kernel/exit.c renamed from kernel/exit.c~ +1 -4
@@ -275,10 +275,7 @@ static void reparent_to_init(void)
 	remove_parent(current);
 	current->parent = child_reaper(current);
 	current->real_parent = child_reaper(current);
-	add_parent(current);
-
-	/* Set the exit signal to SIGCHLD so we signal init on exit */
-	current->exit_signal = SIGCHLD;
+	current->exit_signal = -1;
 
 	if (!has_rt_policy(current) && (task_nice(current) < 0))
 		set_user_nice(current, 0);
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines

is enough. init is still our parent (make ps happy), but it can't see us, we are not on ->children list.

OK, this doesn't work. A multi-threaded init may do execve().

Good catch. daemonize must die!

Well, this is not the problem of daemonize(), but I agree, daemonize() should die with the changes you proposed.

Quoted 9 lines
Quoted 2 lines

So, we can re-parent a kernel thread to swapper. In that case it doesn't matter if we put task on ->children list or not.

Yes. We can.

Quoted 1 line

User-visible change. Acceptable?

We would have user visible changes when we changed to ktrhead_create anyway, and it's none of user space's business so certainly.

OK, I'll try to do "just for review" patch. I'm afraid pstree will be confused.

Quoted 8 lines
Quoted 5 lines
Quoted 3 lines

Off course, we also need to add preparent_to_init() to kthread() and (say) stopmachine(). Or we can create kernel_thread_detached() and modify callers to use it.

It would be very nice to introduce CLONE_KERNEL_THREAD instead, then

If we are going to do something to copy_process and the like let's take this one step farther. Let's pass in the value of the task to copy.

Yes! I thought about that too. but see below,

Quoted 5 lines

Then we can add a wrapper around copy_process to build kernel_thread something like:

struct task_struct *__kernel_thread(int (*fn)(void *), void * arg,
unsigned long flags)

Quoted 11 lines
{
	struct task_struct *task;
	struct pt_regs regs, *reg;

        reg = kernel_thread_regs(&regs, fn, arg);
	task = copy_process(&init_task, flags, 0, reg, 0, NULL, NULL, NULL, 0);
	if (!IS_ERR(task))
		wake_up_new_task(task, flags);

	return task;
}

First, we still need some additional flag/parameter to set ->exit_state = -1. Let's use CLONE_KERNEL_THREAD for discussion.

Now. Can we do copy_process(an_arbitrary_task) ? If yes, we need additional locking in copy_process(). Just think about ->signal,->parent access. Nasty, and probably not so useful.

If we can only use "current" or init_task, CLONE_KERNEL_THREAD is enough. Just now it only

- sets ->exit_state = -1

- sets ->parent = &init_task

(note that the second change could be omitted, we can use CLONE_PARENT
if we know that all users of CLONE_KERNEL_THREAD are kernel threads).

Quoted 26 lines

After that daemonize just becomes:

void daemonize(const char *name, ...)

{
	va_list args;

	va_start(args, name);
	vsnprintf(current->comm, sizeof(current->comm), name, args);
	va_end(args);
}

And kthread_create becomes:

struct task_struct *kthread_create(int (*threadfn)(void *data),

				   void *data,
				   const char namefmt[],
				   ...)
{
	struct kthread_create_info create;
	struct task_struct *task;

	create.threadfn = threadfn;
	create.data = data;

	/* We want our own signal handler (we take no signals by default). */
        task = __kernel_thread(kthread, create, CLONE_FS | CLONE_FILES | SIGCHLD);
Quoted 8 lines
	if (!IS_ERR(task)) {
		va_list args;
		va_start(args, namefmt);
		vsnprintf(task->comm, sizeof(task->comm),  namefmt, args);
		va_end(args);
	}
        return task;
}

Yes, nice. This is not complete, we still have to wait for completion, because kthread_create() promises that a new thread has no CPU on return (so we can use kthread_bind() for example), but nice anyway.

Quoted 4 lines

If we are willing to go that far. I think it is worth it to touch the architecture specific code. As that removes an unnecessary wait queue, and ensures that kernel threads always start with a consistent state. So it is a performance, scalability and simplicity boost.

Yes.

But note that CLONE_KERNEL_THREAD allows us to achieve the same step by step. For example, we can extend the meaning of CLONE_KERNEL_THREAD to use init_task.mm in copy_mm(), then copy_fs(), etc.

Eric, I am far from sure I am right though. Probably it is better to do one big change as you suggested.

However, I like the idea of for-in-kernel-only CLONE_ flags. For example, we can kill the ugly "pid" parameter of copy_process() and use CLONE_IDLE instead.

Also, we can't use __kernel_thread() you propose if we want do_wait(), using CLONE_ flags a bit more flexible.

BTW. I think your idea of kernel_thread_regs() is very nice in any case. Is it enough (and possible) to implement an architecture neutral kernel_thread?

Oleg.