tasklet latency and system calls on mips
Hello dearest all.
I have a question regarding tasklets on MIPS. I suspect that there is a bug in generic MIPS kernel, but I'm not sure yet.
Linux kernel has a couple of so called "checkpoints" when the system should check if there are tasklets to
run and run them in the following way :
if (softirq_pending(cpu))
do_softirq();
One of these places is at the end of interrupt handler (do_IRQ()), however this is not the only place. I was under
impression that this code should be called after system call too. The caveat here is that on MIPS (contrary to
other architectures, such as x86) system call is not an interrupt (it's a different exception) and has completely
different handler. So in x86 it is sufficient to call
if (softirq_pending(cpu))
do_softirq();
at the end of do_IRQ because do_IRQ handles system call too, but on MIPS it is not. Therefore I believe
these lines should be added to the end of sys_syscall function on MIPS.
What do you think ?
P.S. The whole issue started when we noticed that user process making many system calls has very
significant impact on device drivers running in tasklet mode and no impact whatsoever on device
drivers running in interrupt mode.
Quoted 20 lines · Alexander
Hello dearest all.
I have a question regarding tasklets on MIPS. I suspect that there is a bug in generic MIPS kernel, but I'm not sure yet.
Linux kernel has a couple of so called "checkpoints" when the system should check if there are tasklets to
run and run them in the following way :if (
softirq_pending(cpu))do_softirq();One of these places is at the end of interrupt handler (
do_IRQ()), however this is not the only place. I was under
impression that this code should be called after system call too. The caveat here is that on MIPS (contrary to
other architectures, such as x86) system call is not an interrupt (it's a different exception) and has completely
different handler. So in x86 it is sufficient to call
Quoted 9 lines
if (
softirq_pending(cpu))do_softirq();at the end of
do_IRQbecausedo_IRQhandles system call too, but on MIPS it is not. Therefore I believe
these lines should be added to the end ofsys_syscallfunction on MIPS.What do you think ?
softirq/tasklet/bottom_half/etc should only be raised from interrupt context. Checking at the end of do_IRQ should be good enough.
One possible mistake in MIPS porting is that if the board uses its private time interrupt routine poeple may forget to put the above two lines at the end. Check against that.
Quoted 3 lines
P.S. The whole issue started when we noticed that user process making many system calls has very
significant impact on device drivers running in tasklet mode
What kind of impact? On i386? Or on MIPS?
Jun
Hi.
See my comments below.
Jun Sun wrote:
On Wed, Aug 13, 2003 at 04:46:04PM +0300, Sirotkin, Alexander wrote:
Hello dearest all.
I have a question regarding tasklets on MIPS. I suspect that there is a
bug in generic MIPS kernel, but I'm not sure yet.
Linux kernel has a couple of so called "checkpoints" when the system
should check if there are tasklets to
run and run them in the following way :
if (softirq_pending(cpu))
do_softirq();
One of these places is at the end of interrupt handler (do_IRQ()),
however this is not the only place. I was under
impression that this code should be called after system call too. The
caveat here is that on MIPS (contrary to
other architectures, such as x86) system call is not an interrupt (it's
a different exception) and has completely
different handler. So in x86 it is sufficient to call
if (softirq_pending(cpu))
do_softirq();
at the end of do_IRQ because do_IRQ handles system call too, but on MIPS
it is not. Therefore I believe
these lines should be added to the end of sys_syscall function on MIPS.
What do you think ?
softirq/tasklet/bottom_half/etc should only be raised from interrupt
context. Checking at the end of do_IRQ should be good enough.
On mips interrupt is an exception and system call is a different exception. Different exceptions has different exception handlers, at least that's what I was able to figure from entry.S file. So the system call does not go through do_IRQ and do_softirq
is not called.
One possible mistake in MIPS porting is that if the board uses its private
time interrupt routine poeple may forget to put the above two lines
at the end. Check against that.
In our kernel port we do have these lines in the timer interrupt, that is not a problem.
P.S. The whole issue started when we noticed that user process making
many system calls has very
significant impact on device drivers running in tasklet mode
What kind of impact? On i386? Or on MIPS?
Jun
The impact is that if our driver works in tasklet mode then some user mode application making system calls
causes some (although quite small) packet loss. It does not happen if we don't use tasklet and do everything in the
ISR.
I suspect that what happens is as follows :
system call arrives and while it's being processed and interrupt to one
of the drivers arrives. This interrupt schedules a tasklet which however is not executed after the system call finishes, only after the next timer
interrupt which causes up to 10 ms latency (not all the time, only when
somebody makes a system call).
It only happens on MIPS. There is no easy way to check this on x86.
Signature · 17 lines
Alexander Sirotkin
SW Engineer
Texas Instruments
Broadband Communications Israel (BCIL)
Tel: +972-9-9706587
________________________________________________________________________
"Those who do not understand Unix are condemned to reinvent it, poorly."
-- Henry Spencer Quoted 7 lines · Alexander
I suspect that what happens is as follows :
system call arrives and while it's being processed and interrupt to one of the drivers arrives. This interrupt
schedules a tasklet which however is not executed after the system call finishes,
The tasklet should be executed at the return of interrupt handling. If not, there is a bug.
Quoted 4 lines
only after the next timer
interrupt which causes up to 10 ms latency (not all the time, only when somebody makes a system call).
BTW, make sure tasklet_schedule() is indeed called in an interrupt handler. I am not sure why will happen otherwise.
If you suspect it is a bug, you can easily trace them. You may my little tracing tool useful,
http://linux.junsun.net/patches/generic/experimental/030716.a-jstrace.patch
Jun
Jun Sun wrote:
On Thu, Aug 14, 2003 at 12:17:52PM +0300, Sirotkin, Alexander wrote:
I suspect that what happens is as follows :
system call arrives and while it's being processed and interrupt to one
of the drivers arrives. This interrupt
schedules a tasklet which however is not executed after the system call
finishes,
The tasklet should be executed at the return of interrupt handling.
If not, there is a bug.
I have a feeling that we are going in circles. Tasklets are executed at
the return of interrupt handler.
However, I suspect that this is not enough. On mips (contrary to x86),
system call is NOT an interrupt.
It's a different exception with different handler. Therefore I suspect
that tasklets are NOT called at the end of system call exception handler (which is a different handler, not do_IRQ).
only after the next timer
interrupt which causes up to 10 ms latency (not all the time, only when
somebody makes a system call).
BTW, make sure tasklet_schedule() is indeed called in an interrupt handler.
I am not sure why will happen otherwise.
If you suspect it is a bug, you can easily trace them. You may my
little tracing tool useful,
I can try to trace it, I just wanted to ensure that what I was saying makes sense. See above.
http://linux.junsun.net/patches/generic/experimental/030716.a-jstrace.pa
tch
<http://linux.junsun.net/patches/generic/experimental/030716.a-jstrace.p
atch>
Jun
Signature · 17 lines
Alexander Sirotkin
SW Engineer
Texas Instruments
Broadband Communications Israel (BCIL)
Tel: +972-9-9706587
________________________________________________________________________
"Those who do not understand Unix are condemned to reinvent it, poorly."
-- Henry Spencer Quoted 5 lines · Alexander
The tasklet should be executed at the return of interrupt handling.
Quoted 7 lines
If not, there is a bug.
I have a feeling that we are going in circles. Tasklets are executed at the return of interrupt handler.
However, I suspect that this is not enough.
If you follow this, plus "tasklet_schedule() is indeed called in an interrupt handler", you will should see "executing tasklet at the return of interrupt handler" is _obviously_ enough.
Quoted 7 lines
On mips (contrary to x86),
system call is NOT an interrupt.
It's a different exception with different handler. Therefore I suspect that tasklets are NOT called at
the end of system call exception handler (which is a different handler, notdo_IRQ).
... which is fine, if you can follow the above logic.
Jun