Question regarding kernel threads and userlevel
Hi linux-kernel,
I have a question regarding kernel threads : Are kernel threads treated equally in terms of scheduling as normal userlevel processes ??
In my test case I have a driver for a PCI card from which I want to control access to it's memory (prefetchable PCI space). Userlevel processes can mmap this PCI memory and write directly to it (via the nopage technique). This is also possible from the kernel thread, but to avoid trashing and short bursts on the PCI bus, I protect every access to the memory space with a spin lock (a mmapped kernel memory page which the driver initializes). That means if you have a SMP system and two userlevel processes wants to write to this memory, one will have to wait for the other before doing the memcpy (yep I'm using what you can call PIO). This works great for two userlevel processes.
Now the reason for my question is; if also I have a kernel thread wanting to write to this memory space it will also have to wait for the same lock (though not mmapped since we are already in kernel space and can access the lock page directly). What happens, is that if a userlevel process holds this lock and the kernel thread gets scheduled and tries to get the same lock it will deadlock because the userlevel process never gets back control and releases the lock (kinda like when you spinlock in interrupt level on a lock wich is locked without spinlock_irq). Is this because the kernel thread has higher priority than the user level process (it has a nice level of -20) ?
Best regards,
Signature · 7 lines
Steffen Persvold Systems Engineer Email : mailto:[email protected] Scali AS (http://www.scali.com) Norway : Tel : (+47) 2262 8950 Olaf Helsets vei 6 Fax : (+47) 2262 8951 N-0621 Oslo, Norway USA : Tel : (+1) 713 706 0544 10500 Richmond Avenue, Suite 190 Houston, Texas 77042, USA
Quoted 1 line · Steffen Persvold
I have a question regarding kernel threads : Are kernel threads treated equally in terms of scheduling as normal userlevel processes ??
I'm sure someone will correct me if I am wrong but I would think that kernel threads are treated the same as the kernel, i.e. they are not pre-emptied unless you call schedule yourself or you sleep.
Quoted 1 line
kernel thread gets scheduled and tries to get the same lock it will deadlock because the userlevel process never gets back control and releases the lock.
It never gets pre-emptied according to the above so it would spin for ever. You would need a SMP system which would be running the user space code on a second CPU to unlock again. (And if your kernel thread is holding any other locks which the user mode thread will take along it's kernel code path you will deadlock, too.)
If my suggestion above is correct then you can fix this by doing:
while (!spin_trylock())
schedule();
Which will forcefully schedule to allow your processes to run and you know that as soon as you drop out of the loop you are holding the lock. If you want you could also add in a counter a display a warning or something if you don't manage to acquire a lock after several tries, perhaps just for debugging purposes.
Best regards,
Anton
Signature · 3 lines
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @) Linux NTFS Maintainer / WWW: http://sourceforge.net/projects/linux-ntfs/ ICQ: 8561279 / WWW: http://www-stu.christs.cam.ac.uk/~aia21/
Quoted 5 lines · Anton Altaparmakov
Quoted 1 line · Steffen Persvold
I have a question regarding kernel threads : Are kernel threads treated equally in terms of scheduling as normal userlevel processes ??
I'm sure someone will correct me if I am wrong but I would think that kernel threads are treated the same as the kernel, i.e. they are not pre-emptied unless you call schedule yourself or you sleep.
Quoted 8 lines
Quoted 1 line
kernel thread gets scheduled and tries to get the same lock it will deadlock because the userlevel process never gets back control and releases the lock.
It never gets pre-emptied according to the above so it would spin for ever. You would need a SMP system which would be running the user space code on a second CPU to unlock again. (And if your kernel thread is holding any other locks which the user mode thread will take along it's kernel code path you will deadlock, too.)
If my suggestion above is correct then you can fix this by doing:
while (!spin_trylock())
schedule();
Obviously dropping any other locks you hold onto before the schedule() and reacquiring them afterwards... Or you will probably deadlock anyway.
Anton
Quoted 17 lines
Which will forcefully schedule to allow your processes to run and you know that as soon as you drop out of the loop you are holding the lock. If you want you could also add in a counter a display a warning or something if you don't manage to acquire a lock after several tries, perhaps just for debugging purposes.
Best regards,
Anton
--
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @) Linux NTFS Maintainer / WWW: http://sourceforge.net/projects/linux-ntfs/
ICQ: 8561279 / WWW: http://www-stu.christs.cam.ac.uk/~aia21/-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Signature · 3 lines
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @) Linux NTFS Maintainer / WWW: http://sourceforge.net/projects/linux-ntfs/ ICQ: 8561279 / WWW: http://www-stu.christs.cam.ac.uk/~aia21/
How to time in Kernel
In reply to Anton AltaparmakovHi,
I am trying to time a portion of code inside the kernel. How do I do it?
Can I use do_gettimeofday ? or do_getitimer ? Any leads will be appreciated.
Thanks in advance,
-Srini.
RE: How to time in Kernel
Quoted 4 lines
I am trying to time a portion of code inside the kernel. How do I do it?
Can I usedo_gettimeofday? ordo_getitimer? Any leads will be appreciated.
If this is for personal debugging use only and you have a Pentium or better
x86 processor, I recommend reading the TSC directly. 'msr.h' (in the asm-i386 include directory) include 'rdtsc' and friends.
DS