Why no interrupt priorities?
What's the rationale for not supporting interrupt priorities in the kernel?
We're having a discussion of this in one of our CELF working groups, and it would help if someone could explain why an interrupt priority system has never been adopted in the mainstream Linux kernel. (I know that some implementations have been written and proposed).
Is there a strong policy against such a thing, or is it just that the right implementation has not come along?
Thanks,
=============================
Tim Bird
Architecture Group Co-Chair
CE Linux Forum
Senior Staff Engineer
Sony Electronics
E-mail: [email protected]
=============================
Quoted 2 lines · Tim Bird
What's the rationale for not supporting interrupt priorities in the kernel?
Interrupt priorities are supported and have been supported since the first cascaded interrupt controllers and, now with the APIC. The interrupt priorities are enforced by hardware. There are no "software interrupt priorities" because we have more than one interrupt, already prioritized by the hardware. The basic PC/AT has IRQ0 through IRQ15 interrupt sources. The IO-APIC code emulates this. The priorites go like this:
Highest priority
|
IRQ0 PIT channel 0
IRQ1 Keyboard
IRQ2 Cascade to second controller
IRQ8
IRQ9
IRQ10
IRQ11
IRQ12
IRQ13
IRQ14
IRQ15
IRQ3 Serial 1, Serial 3
IRQ4 Serial 0, Serial 2
IRQ5 Floppy disk
IRQ6
IRQ7 Printer
|
Lowest priority
You can't do software interrupt priorities with hardware interrupt controllers unless you funnel everything into one master ISR that ACKs the hardware, then sorts through some priority lists. The result is an abortion that wastes CPU cycles and throws away the hardware advantage that you already have.
If you have an architecture that has only one hardware interrupt, then you have no choice but to impliment some sort of software priority scheme. This is not what we have on the ix86.
Cheers,
Dick Johnson
Penguin : Linux version 2.4.24 on an i686 machine (797.90 BogoMips).
Note 96.31% of all statistics are fiction.
Quoted 8 lines · Richard B. Johnson
Quoted 2 lines · Tim Bird
What's the rationale for not supporting interrupt priorities in the kernel?
Interrupt priorities are supported and have been supported since the first cascaded interrupt controllers and, now with the APIC.
Please forgive my ignorance. I'm not sure what's going on with 2.6 and work queues, but do the hardware priorities allow you to control scheduling of interrupt bottom halves?
=============================
Tim Bird
Architecture Group Co-Chair
CE Linux Forum
Senior Staff Engineer
Sony Electronics
E-mail: [email protected]
=============================
Quoted 6 lines · Tim Bird
Quoted 5 lines · Richard B. Johnson
Quoted 2 lines · Tim Bird
What's the rationale for not supporting interrupt priorities in the kernel?
Quoted 7 lines
Quoted 3 lines
Interrupt priorities are supported and have been supported since the first cascaded interrupt controllers and, now with the APIC.
Please forgive my ignorance. I'm not sure what's going on with 2.6 and work queues, but do the hardware priorities allow you to control scheduling of interrupt bottom halves?
hardware IRQ priorities are useless for the linux model. In linux, the hardirq runs *very* briefly and then lets the softirq context do the longer taking work. hardware irq priorities then don't matter really because the hardirq's are hardly ever interrupted really, and when they are they cause a performance *loss* due to cache trashing. The latency added by waiting briefly is going to be really really short for any sane hardware.
Now doing priorities in softirq context... well... here again it's a case of a tiny latency hit vs a lot of cache trashing. If your softirq handler runs in 10 cachemisses (it's useless to talk about cpu cycles since most of teh time you'll be cache bound) that's not too long latency, but if you interrupt it it might get 15 or more cachemisses instead. That again will increase the delay the user context gets from irq's.... so from a userspace pov you actually increased irq latency....
Quoted 14 lines · Arjan van de Ven
Quoted 5 lines · Tim Bird
Quoted 4 lines · Richard B. Johnson
Quoted 2 lines · Tim Bird
What's the rationale for not supporting interrupt priorities in the kernel?
Quoted 7 lines
Quoted 3 lines
Interrupt priorities are supported and have been supported since the first cascaded interrupt controllers and, now with the APIC.
Please forgive my ignorance. I'm not sure what's going on with 2.6 and work queues, but do the hardware priorities allow you to control scheduling of interrupt bottom halves?
Quoted 7 lines
hardware IRQ priorities are useless for the linux model. In linux, the hardirq runs *very* briefly and then lets the softirq context do the longer taking work. hardware irq priorities then don't matter really because the hardirq's are hardly ever interrupted really, and when they are they cause a performance *loss* due to cache trashing. The latency added by waiting briefly is going to be really really short for any sane hardware.
Keep in mind the context is Linux running on non-sane hardware, sloooow CPUs, latency sensitive small io buffers etc. Losing system wide throughput to have the hardware codec not be starved is a happy trade off to make.
So far all the comments are very PC centric. I suspect the history of the no interrupt priorities goes back a lot of years. Perhaps the answer is that it never seemed to help having priorities much on PC hardware so lets not deal with the complexity in the software.
Any more historical insights?
I wonder how hard it would be add for embedded types of applications/architectures?
--mgross
Quoted 8 lines
Now doing priorities in softirq context... well... here again it's a case of a tiny latency hit vs a lot of cache trashing. If your softirq handler runs in 10 cachemisses (it's useless to talk about cpu cycles since most of teh time you'll be cache bound) that's not too long latency, but if you interrupt it it might get 15 or more cachemisses instead. That again will increase the delay the user context gets from irq's.... so from a userspace pov you actually increased irq latency....
Quoted 9 lines · Mark Gross
Quoted 7 lines
hardware IRQ priorities are useless for the linux model. In linux, the hardirq runs *very* briefly and then lets the softirq context do the longer taking work. hardware irq priorities then don't matter really because the hardirq's are hardly ever interrupted really, and when they are they cause a performance *loss* due to cache trashing. The latency added by waiting briefly is going to be really really short for any sane hardware.
Keep in mind the context is Linux running on non-sane hardware, sloooow CPUs,
50Mhz is already really really fast in this context.
Quoted 2 lines
latency sensitive small io buffers etc. Losing system wide throughput to have the hardware codec not be starved is a happy trade off to make.
The point I tried to make was that it would INCREASE latency. Unless you have misdesigned device drivers, which is something that is fixable :)
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Quoted 2 lines · Arjan van de Ven
The point I tried to make was that it would INCREASE latency. Unless you have misdesigned device drivers, which is something that is fixable :)
Not if you bought them as IP from some company. Or you did some outsourcing and external development games.
Then you are basically lost and must workaround your problems instead of fixing them.
Regards
Ingo Oeser
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
iD8DBQFAPymkU56oYWuOrkARAsWUAKCTI3ikuVsV2xfFWS/frDBYCOJetACggQfr NQcCvIS5I8jH4zWCPL5obYA=
=CoLz
-----END PGP SIGNATURE-----
Quoted 6 lines · Ingo Oeser
Quoted 2 lines · Arjan van de Ven
The point I tried to make was that it would INCREASE latency. Unless you have misdesigned device drivers, which is something that is fixable :)
Not if you bought them as IP from some company. Or you did some outsourcing and external development games.
I think that's a case of "Doctor Doctor it hurts when I do this" .. " Don't do that then"
Quoted 6 lines · Tim Bird
Quoted 5 lines · Richard B. Johnson
Quoted 2 lines · Tim Bird
What's the rationale for not supporting interrupt priorities in the kernel?
Quoted 8 lines
Quoted 3 lines
Interrupt priorities are supported and have been supported since the first cascaded interrupt controllers and, now with the APIC.
Please forgive my ignorance. I'm not sure what's going on with 2.6 and work queues, but do the hardware priorities allow you to control scheduling of interrupt bottom halves?
Only to the extent that the bottom halves will naturally occur after the hardware interrupt. There is no explicit "hold execution until something else is done" in the queue.
It is a queue, i.e., one task after another.
Quoted 9 lines
=============================
Tim Bird
Architecture Group Co-Chair
CE Linux Forum
Senior Staff Engineer
Sony Electronics
E-mail: [email protected]
=============================
Cheers,
Dick Johnson
Penguin : Linux version 2.4.24 on an i686 machine (797.90 BogoMips).
Note 96.31% of all statistics are fiction.
Quoted 2 lines · Tim Bird
What's the rationale for not supporting interrupt priorities in the kernel?
What do you actually want to do them? Linux doesn't do the traditional unix spl scheme for coplexity and performance reasons, see
www.usenix.org/publications/library/proceedings/ana97/full_papers/small/small.ps
for a related paper. Give that linux hardirq handlers should be very small there's no performance gain in that area for sane architectures, too.