kernel_thread vs. zombie
Hi !
I'm changing the ADB bus reset & probe code to be in a kernel threads that is created when a bus reset is triggered and that dies of it's own death.
Everything is fine when the bus reset is triggered during kernel init as my thread is a child of init. However, when created as a result of an ioctl, the thread becomes a zombie as it's a child of the process who caused the ioctl (typically when entering sleep mode).
How do I force a kernel thread to always be a child of init and never become a zombie ?
I do call daemonize at the beginning of the thread (as it won't do anything with files, signals or whatever), but that doesn't seem to be enough.
Reagrds,
Ben.
Hi,
Quoted 7 lines
How do I force a kernel thread to always be a child of init and never become a zombie ?
I do call daemonize at the beginning of the thread (as it won't do anything with files, signals or whatever), but that doesn't seem to be enough.
Have a look at:
http://www.scs.ch/~frey/linux/kernelthreads.html
I have an example there that starts and stops kernel threads from init_module and never produced a zombie.
I use the same code also to start threads from ioctl and it works for me. I tested it on UP and SMP, Intel and Alpha, 2.2.18 and 2.4.2.
Regards,
Martin
Signature · 4 lines
Supercomputing Systems AG email: [email protected] Martin Frey web: http://www.scs.ch/~frey/ at Compaq Computer Corporation phone: +1 603 884 4266 ZKO2-3P09, 110 Spit Brook Road, Nashua, NH 03062
Quoted 7 lines
Have a look at:
http://www.scs.ch/~frey/linux/kernelthreads.html
I have an example there that starts and stops kernel threads frominit_moduleand never produced a zombie.
I use the same code also to start threads from ioctl and it works for me. I tested it on UP and SMP, Intel and Alpha, 2.2.18 and 2.4.2.
Thanks !
Could you explain me a bit why you need the lock_kernel ? My probe thread is already protected by some atomic ops, but I'm considering changing them to semaphores. Is there any need for the bkl to be taken when calling daemonize or is this just for your own syncronisation needs ?
I don't think you do more than what I currently do to prevent the zombie (except for the daemonize call, I don't see you changing anything about the parent thread or whatever).
At first I though daemonize() would do the trick, but I still see zombies on my tests. I'm running UP now so I don't since my lack of lock_kernel() could explain it.
Ben.
Hi,
Quoted 8 lines
Quoted 1 line
Could you explain me a bit why you need the
lock_kernel? My probe thread is already protected by some atomic ops, but I'm considering changing them to semaphores. Is there any need for the bkl to be taken when calling daemonize or is this just for your own syncronisation needs ?
The stuff done in daemonize() and the exit_files could need the kernel lock. At least on some 2.2.x version it does, I did not check whether it is still needed on 2.4.
On stop of the thread I need the big kernel lock to make sure the kernel thread exited (everything really done from my up() till the thread is in zombie state) before I unload the module. The comment in the code should explain in.
Note that the threads itself do not run with the kernel lock held. After setting everything up the make an unlock.
Quoted 5 lines
I don't think you do more than what I currently do to prevent the zombie (except for the daemonize call, I don't see you changing anything
about the parent thread or whatever).
No, changing the parent thread is done by daemonize().
Quoted 4 lines
At first I though
daemonize()would do the trick, but I still see zombies on my tests. I'm running UP now so I don't since my lack oflock_kernel()could explain it.
I needed the exit_files() on 2.2.x to prevent the zombies. On 2.4 daemonize() does that as well.
In the kill_thread() call I actually wait till the thread has terminated. This is assured with the semaphore and the big kernel lock. I did have problems when I just sent the termination signal and unloaded the module, but since I really block there I did not see zombies. I stress-tested the module for several days in a loop
- no zombies at all.
Regards,
Martin
Quoted 3 lines
The stuff done in
daemonize()and theexit_filescould need the kernel lock. At least on some 2.2.x version it does, I did not check whether it is still needed on 2.4.
Well, I don't really plan to backport this to 2.2.x. I'll try to see if my problem is related to the lack of kernel lock, or maybe I have just something else wrong.
Quoted 5 lines
On stop of the thread I need the big kernel lock to make sure the kernel thread exited (everything really done from my
up()till the thread is in zombie state) before I unload the module. The comment in the code should explain in.
Ok. I don't need that as I'm not in a module, no chances I ever get unloaded. At least not in 2.4. Making ADB and all the controllers and device drivers in modules would be an interesting exercise with module dependencies ;)
Quoted 2 lines
Note that the threads itself do not run with the kernel lock held. After setting everything up the make an unlock.
Ok. Well, I just have an atomic flag test&set'ed before starting the bus reset, and released at the end of the thread. No need to make sure the previous one is really dead before starting a new one. I could benefit from semaphores when starting it since if it's already running, I just loop scheduling waiting for the lock bit to be available. But that case will almost never happen in real life. ADB probes are quite rare.
Many thanks for your help,
I'll see what's wrong in my code ;)
ben.
Quoted 8 lines
Quoted 3 lines
The stuff done in
daemonize()and theexit_filescould need the kernel lock. At least on some 2.2.x version it does, I did not check whether it is still needed on 2.4.Well, I don't really plan to backport this to 2.2.x. I'll try to see if my problem is related to the lack of kernel lock, or maybe I have just something else wrong.
daemonize() makes calls that are all protected with the big kernel lock in do_exit(). All usages of daemonize have the big kernel lock held. So I guess it just needs it.
Please let me know whether you have success if it makes a difference with having it held.
Martin
Quoted 6 lines
daemonize()makes calls that are all protected with the big kernel lock indo_exit(). All usages of daemonize have the big kernel lock held. So I guess it just needs it.Please let me know whether you have success if it makes a difference with having it held.
With a bit more experiments, I have this behaviour:
(I hold the kerne lock, daemonize(), and release the kernel lock, then do my probe thing which takes a few seconds, and let the thread die by itself)
- When started during boot (low PID (9)) It becomes a zombie
- When started from a process that quits after sending the ioctl, it is correctly "garbage collected".
- When started from a process that stays around, it becomes a zombie too
So something is not working, or I'm missing something obvious, or whatever...
Any clue ?
Ben.
Quoted 13 lines · Benjamin Herrenschmidt
Quoted 7 lines
daemonize()makes calls that are all protected with the big kernel lock indo_exit(). All usages of daemonize have the big kernel lock held. So I guess it just needs it.Please let me know whether you have success if it makes a difference with having it held.
With a bit more experiments, I have this behaviour:
(I hold the kerne lock,
daemonize(), and release the kernel lock, then do my probe thing which takes a few seconds, and let the thread die by itself)
Quoted 8 lines
- When started during boot (low PID (9)) It becomes a zombie
- When started from a process that quits after sending the ioctl, it is correctly "garbage collected".
- When started from a process that stays around, it becomes a zombie tooSo something is not working, or I'm missing something obvious, or whatever...
Any clue ?
Take a look at kernel/kmod.c:call_usermodehelper(). Copy it.
This will make your thread a child of keventd. This takes care of things like chrootedness, uids, cwds, signal masks, reaping children, open files, and all the other crud which you can accidentally inherit from your caller.
something like:
struct my_struct
{
struct tq_struct tq;
void (*function)(void *);
struct semaphore sem;
<other stuff>
};/* keventd runs this */
void helper(void *data)
{
struct my_struct *my_ptr = data;
kernel_thread(my_ptr->function, my_ptr, CLONE_FLAGS|SIGCHLD);
}start_thread(struct my_struct *my_ptr)
{
my_ptr->tq.sync = 0;
INIT_LIST_HEAD(&my_ptr->tq.list);
my_ptr->routine = helper;
my_ptr->data = my_ptr;
schedule_task(&my_ptr->tq);
}Quoted 6 lines
Quoted 5 lines
- When started during boot (low PID (9)) It becomes a zombie
- When started from a process that quits after sending the ioctl, it is correctly "garbage collected".
- When started from a process that stays around, it becomes a zombie too
Quoted 7 lines
Take a look at kernel/kmod.c:call_usermodehelper(). Copy it.
This will make your thread a child of keventd. This takes care of things like chrootedness, uids, cwds, signal masks, reaping children, open files, and all the other crud which you can accidentally inherit from your caller.
So depending on the state of the caller daemonize() will not really put us into the background as we want. With being created from keventd we inherit a state as we'd like to have in a kernel thread. Did I get it right?
I will change my example and test that.
Thanks,
Martin
Quoted 9 lines · Martin Frey
Quoted 9 lines
Quoted 6 lines
- When started during boot (low PID (9)) It becomes a zombie
- When started from a process that quits after sending the ioctl, it is correctly "garbage collected".
- When started from a process that stays around, it becomes a zombie tooTake a look at kernel/kmod.c:call_usermodehelper(). Copy it.
Quoted 7 lines
Quoted 5 lines
This will make your thread a child of keventd. This takes care of things like chrootedness, uids, cwds, signal masks, reaping children, open files, and all the other crud which you can accidentally inherit from your caller.
So depending on the state of the caller
daemonize()will not really put us into the background as we want.
Well, kernel_thread() will put you in the background, in the sense that it creates an async thread. But you inherit heaps of stuff from the parent. daemonize() cleans up some of those things, but it can't clean up everything.
Kernel threads *need* to run in a well-understood and sensible environment. We went through a lot of fun late last year when there was a sudden proliferation of kernel threads and quite a few things were subtly broken.
Things like kernel threads blocking signals because that's what their user-space parent happened to do. Things like user-space applications receiving a surprise SIGCHLD from the kernel as a consequence of some system call which they happened to have executed some while beforehand.
One approach would be to tromp through your task state setting everything back where you want it. That's quite complex. Plus there's the issue of who reaps the thread when it exits.
So I think it's reasonable to use keventd as `kinit', if you like. Something which knows how to launch and reap kernel daemons, and which provides a known environment to them.
A kernel API function (`kernel_daemon'?) which does all this boilerplate is needed, I think.
-
Quoted 7 lines
So I think it's reasonable to use keventd as `kinit', if you like. Something which knows how to launch and reap kernel daemons, and which provides a known environment to them.
A kernel API function (`kernel_daemon'?) which does all this boilerplate is needed, I think.
I completely agree. I'll be on a trip for the next two weeks, but after that I can write a small example how it would look. I'll mail again when it is ready.
Thanks, Martin