Skip to contentSkip to conversation

softirqd

DISCUSSION kernwek jalsl 3 participants 6 messages Nov 17, 2003 – Nov 18, 2003

lore
Share & tools
lore
#0 lore raw

Hi All;

On my embedded board I see that the preemption patch(Love etal.) is not of much use because of the following reaons:

1) calling the scheduler after the hardirq returns is not of much use because the actual work of puting the task in the run queue is done in the bottom half executed by ksoftirqd
2) ksoftirqd is not a real time thread

Will the preemption patch work better if the ksoftirqd is made a real time thread?

Another related question : has anyone thought of introducing prioirty among tasklets? Right now ksoftirqd treats them in a FIFO manner. Adding priority among the various tasklets and making sure that ksoftirqd looks ahead in the queue would ensure a real time treatment to a real time interrupt...

I am sorry in case this has been discussed earlier; in case it has been I would like someone to post me the URl for the discussion.

Regards

__________________________________
Do you Yahoo!?
Protect your identity with Yahoo! Mail AddressGuard
http://antispam.yahoo.com/whatsnewfree

#1 lore raw
Quoted 15 lines · kernwek jalsl

Hi All;

On my embedded board I see that the preemption patch(Love etal.) is not of much use because of the following reaons:

1) calling the scheduler after the hardirq returns is not of much use because the actual work of puting the task in the run queue is done in the bottom half executed by ksoftirqd
2) ksoftirqd is not a real time thread

Will the preemption patch work better if the ksoftirqd is made a real time thread?

Quoted 7 lines

Another related question : has anyone thought of introducing prioirty among tasklets? Right now ksoftirqd treats them in a FIFO manner. Adding priority among the various tasklets and making sure that ksoftirqd looks ahead in the queue would ensure a real time treatment to a real time interrupt...

What is the problem that you are attempting to fix? Stating that x is not y does not mean anything useful. 'Real-time' related to an 'interrupt' has no meaning.

An interrupt occurs as soon as hardware demands it. The software response to that interrupt, i.e., interrupt- latency is dependent upon the ISR setup code in the kernel plus the execution time of any higher-priority interrupts that may be occurring plus the execution time of any prior code sharing the same interrupt. By the time your ISR code gets control, the time is bounded by some worse-case execution times and real-time has no meaning.

Then your ISR code must handle the immediate requirements of the hardware as fast and efficiently as possible. Things that can be deferred may be handed over to the soft-IRQ stuff. If you have things that can't be deferred, you must not do it in the soft-IRQ (used to be called bottom-half) handler(s).

Specifically, the things that can be handled in a soft-IRQ are hardware configuration changes and such where here are no deadlines. If you have deadlines, i.e., you must get the data out of a device now before it gets overwritten by new data, then it must be done in the top-half handler.

Also, If you ever enable interrupts in your ISR, plan that the CPU __will__ get taken away. It may be hundreds of milliseconds before your ISR ever gets the CPU again. It all depends upon the behavior of other drivers and some network drivers even loop inside their ISR. Once they get control, you may not get the CPU until a break in network traffic!

In the context of a multi-user time-share operating system, "real-time" means "fast enough". If your complete system can keep up with the demands of these external events without ever losing data, then it is "real-time". However, you can't use these kinds of systems (fast-enough systems) in closed servo-loops because the time from an interrupting event to the completion of the necessary response to that event is not known. For those kinds of "real-time" systems, you need a "real-time kernel". Real-time kernels define the poles in the system transfer-function so that the delay is known and bounded within strict limits. Such systems are not "better". In fact, they might be much slower in response to the usual requirements for interactivity.

Realtime kernels do this by scheduling on strict time-boundaries.

Cheers,
Dick Johnson
Penguin : Linux version 2.4.22 on an i686 machine (797.90 BogoMips).
Note 96.31% of all statistics are fiction.

#2 lore raw

Hi;

Sorry in case I was not very clear with my
requirements. With real time interrupt I meant a
real time task waiting for IO from this interrupt. Assume that I have a high priority interrupt and a real time task waiting for it. Well followimg are the various latencies involved:
L1- interrupt latency
L2- hard and soft IRQ completion
L3 - scheduler latency
L4 - scheduler completion

L1 is pretty acceptable on Linux. For L3 we have the preemption and low latency patch. And for L4 the O(1) scheduler solves the problem. So I see L2 as the bootleneck especially with soft IRQ since the softIRQs get scheduled in a non real time thread and there is no wayI can tell the softIRQd that I want highest priority for the interrupt that will wake up my real time task. I was seeking a solution to this.

I know that TimeSYS has as patch for making the softIRQ a real time thread as well as giving priorities for both the top and bottom halves. Is there any place where I can get some performance figures for this patch?

Regards

--- "Richard B. Johnson" <[email protected]>

Quoted 118 lines · wrote:
Quoted 4 lines · kernwek jalsl

Hi All;

On my embedded board I see that the preemption patch(Love etal.) is not of much use because of

the

Quoted 3 lines

following reaons:

1) calling the scheduler after the hardirq returns

is

Quoted 1 line

not of much use because the actual work of puting

the

Quoted 5 lines

task in the run queue is done in the bottom half executed by ksoftirqd
2) ksoftirqd is not a real time thread

Will the preemption patch work better if the

ksoftirqd

Quoted 6 lines

is made a real time thread?

Another related question : has anyone thought of introducing prioirty among tasklets? Right now ksoftirqd treats them in a FIFO manner. Adding priority among the various tasklets and making

sure

Quoted 1 line

that ksoftirqd looks ahead in the queue would

ensure a

Quoted 2 lines

real time treatment to a real time interrupt...

What is the problem that you are attempting to fix? Stating that x is not y does not mean anything useful.
'Real-time' related to an 'interrupt' has no meaning.

An interrupt occurs as soon as hardware demands it. The software response to that interrupt, i.e., interrupt-
latency is dependent upon the ISR setup code in the kernel plus the execution time of any higher-priority
interrupts that may be occurring plus the execution time
of any prior code sharing the same interrupt. By the time
your ISR code gets control, the time is bounded by some
worse-case execution times and real-time has no meaning.

Then your ISR code must handle the immediate requirements
of the hardware as fast and efficiently as possible. Things
that can be deferred may be handed over to the soft-IRQ
stuff. If you have things that can't be deferred, you must
not do it in the soft-IRQ (used to be called bottom-half)
handler(s).

Specifically, the things that can be handled in a soft-IRQ
are hardware configuration changes and such where here are
no deadlines. If you have deadlines, i.e., you must get
the data out of a device now before it gets overwritten by
new data, then it must be done in the top-half handler.

Also, If you ever enable interrupts in your ISR, plan that
the CPU __will__ get taken away. It may be hundreds of
milliseconds before your ISR ever gets the CPU again. It
all depends upon the behavior of other drivers and some
network drivers even loop inside their ISR. Once they get
control, you may not get the CPU until a break in network
traffic!

In the context of a multi-user time-share operating system,
"real-time" means "fast enough". If your complete system can
keep up with the demands of these external events without
ever losing data, then it is "real-time". However, you can't
use these kinds of systems (fast-enough systems) in closed
servo-loops because the time from an interrupting event to
the completion of the necessary response to that event is
not known. For those kinds of "real-time" systems, you need
a "real-time kernel". Real-time kernels define the poles in
the system transfer-function so that the delay is known and
bounded within strict limits. Such systems are not "better".
In fact, they might be much slower in response to the usual
requirements for interactivity.

Realtime kernels do this by scheduling on strict time-boundaries.

Quoted 8 lines

Cheers,
Dick Johnson
Penguin : Linux version 2.4.22 on an i686 machine (797.90 BogoMips).
Note 96.31% of all statistics are
fiction.

__________________________________
Do you Yahoo!?
Protect your identity with Yahoo! Mail AddressGuard
http://antispam.yahoo.com/whatsnewfree

#3 lore raw
Quoted 14 lines · kernwek jalsl

Hi;

Sorry in case I was not very clear with my
requirements. With real time interrupt I meant a
real time task waiting for IO from this interrupt. Assume that I have a high priority interrupt and a real time task waiting for it. Well followimg are the various latencies involved:
L1- interrupt latency
L2- hard and soft IRQ completion
L3 - scheduler latency
L4 - scheduler completion

Quoted 8 lines

L1 is pretty acceptable on Linux. For L3 we have the preemption and low latency patch. And for L4 the O(1) scheduler solves the problem. So I see L2 as the bootleneck especially with soft IRQ since the softIRQs get scheduled in a non real time thread and there is no wayI can tell the softIRQd that I want highest priority for the interrupt that will wake up my real time task. I was seeking a solution to this.

The fastest would not use softIRQ at all. As previously taught, softIRQ is used for things that can be deferred. You should handle everything in the primary ISR and issue wake_up_interruptible() from the ISR. You should be waking anybody sleeping in select() and anybody sleeping in read().

Quoted 9 lines

I know that TimeSYS has as patch for making the softIRQ a real time thread as well as giving priorities for both the top and bottom halves. Is there any place where I can get some performance figures for this patch?

Regards

Search RTLinux.

Cheers,
Dick Johnson
Penguin : Linux version 2.4.22 on an i686 machine (797.90 BogoMips).
Note 96.31% of all statistics are fiction.

#4 lore raw
In reply to kernwek jalsl

Kernwek Jalsl said:

Kernwek> Sorry in case I was not very clear with my
Kernwek> requirements. With real time interrupt I meant a
Kernwek> real time task waiting for IO from this interrupt. Kernwek> Assume that I have a high priority interrupt and a Kernwek> real time task waiting for it. Well followimg are the Kernwek> various latencies involved:
Kernwek> L1- interrupt latency
Kernwek> L2- hard and soft IRQ completion
Kernwek> L3 - scheduler latency
Kernwek> L4 - scheduler completion

Kernwek> L1 is pretty acceptable on Linux.

I've been trying to measure this. On IA64 I'm measuring around 2.5microseconds (on a 900MHz machine). I personally think that this is too big, and could be reduced.

One thing I think we need to do early in 2.7 is to merge all those architecture-dependent arch/XXX/kernel/irq.c files, and try to reduce the amount of duplicated work done in the new merged file and the lower level architecture-specific files.

--
Dr Peter Chubb http://www.gelato.unsw.edu.au peterc AT gelato.unsw.edu.au The technical we do immediately, the political takes *forever*

#5 lore raw
Quoted 16 lines · Peter Chubb

Kernwek Jalsl said:

Kernwek> Sorry in case I was not very clear with my
Kernwek> requirements. With real time interrupt I meant a
Kernwek> real time task waiting for IO from this interrupt. Kernwek> Assume that I have a high priority interrupt and a Kernwek> real time task waiting for it. Well followimg are the Kernwek> various latencies involved:
Kernwek> L1- interrupt latency
Kernwek> L2- hard and soft IRQ completion
Kernwek> L3 - scheduler latency
Kernwek> L4 - scheduler completion

Quoted 5 lines

Kernwek> L1 is pretty acceptable on Linux.

I've been trying to measure this. On IA64 I'm measuring around 2.5microseconds (on a 900MHz machine). I personally think that this is too big, and could be reduced.

I have a driver which, upon getting an IRQ7 (printer-port interrupt), just reads the 'paper-out' bit and writes the result to the data-port. This allows me to toggle the bit and measure the time using a 'scope and a function generator. The complete time for everything is about 900 nanoseconds on this machine:

processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 5
model name : Pentium II (Deschutes)
stepping : 1
cpu MHz : 399.568
cache size : 512 KB
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 2
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 mmx fxsr
bogomips : 797.90

processor : 1
vendor_id : GenuineIntel
cpu family : 6
model : 5
model name : Pentium II (Deschutes)
stepping : 1
cpu MHz : 399.568
cache size : 512 KB
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 2
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 mmx fxsr
bogomips : 797.90

I have to disconnect the network before making the test because the network driver will hog too much CPU time handling broadcast messages to make any useful measurments.

Considering that it takes 150 to 200 ns to read/write the printer- port, I don't think that's too bad.

[SNIPPED...]

Cheers,
Dick Johnson
Penguin : Linux version 2.4.22 on an i686 machine (797.90 BogoMips).
Note 96.31% of all statistics are fiction.