i8259 and IO-APIC

I already asked this question but did not get any responses. Please condider answering.

Using the old i8259 interrupts controller, my 1-way Linux2.4.16 box livelocks when receiving a high rate UDP flow (interrupt rate is so high that the NET_RX_SOFTIRQ never gets the chance to pull the packets off the backlog queue). However, the receive livelock phenomenom completely disappears when making use of the IO-APIC. Does anyone have an explanation for this?

TIA

Eric

Hmmm

I'm a bit curious my self, in theory the IO_APIC should drastically reduce interrupt latency. An x86 has two interrupt pins, IRQ and NMI.

The first 8259 are (if remember correctly) connected to the IRQ only, and the second is connected to IRQ2 on the first 8259 (cascade). When an Int is triggered the CPU has to go out and read the registers in the two 8259's to find the triggered interrupt, and call the corresponding int handler. THe thing is that the 8259 of today are located in the south bridge, and there are an internal ISA bus in the south bridge, but still running at the amazing speed in 10MHz.

THe IO-APIC, when used, the CPU is set to use the two IRQ pins (IRQ, and NMI) as a two bit bus, and the IO_APIC send the interruot vector on this bus, saving the CPU from reading the vector over the slow ISA bus, and can call the interrupt directly.

However, I've on a couple of occations tried to measure the interrupt latency and I end up with that the latency is the same either I use 8259 or APIC. Most likely a problem with my measuring. Especially since you see better thruput using APIC. However I would like to see you get the same results with a different NIC.

Now, if our UDP packetloss are due to bursts, and not sustained traffic, and you have a NIC that uses memory descriptors (like Intel eepro100 chips, or 3com 90x, most new chips do), you can do a workaround by increasing the receive ring size in the driver.

TJ

_________________________________________________________________________

Terje Eggestad                  mailto:[email protected]
Scali Scalable Linux Systems    http://www.scali.com

Olaf Helsets Vei 6              tel:    +47 22 62 89 61 (OFFICE)
P.O.Box 150, Oppsal                     +47 975 31 574  (MOBILE)
N-0619 Oslo                     fax:    +47 22 62 89 51
NORWAY            
_________________________________________________________________________

The APIC reduces processing overhead. The APIC bus on the pentium III is pretty slow (I believe its a serial 4 wire bus or similar). On the Athlon and Pentium IV it seems to be a lot faster.

In the case of a livelock the problem is probably the cost of handling the IRQ and poking slowly at the chip. Latency is pretty immaterial here compared with irq servicing overheads.

Alan

Do you got any numbers that state that it's processing overhead, and not HW latency that is the bulk of interrupt service time? Just curious, I've been looking and can't find this "perceived fact" backed up by facts anywhere.

BTW: according to "IA-32 Intel Arch. Software Developer's Man Vol 3" both P3 and P4 the APIC bus is three wire, two data and one clock.

TJ

_________________________________________________________________________

Terje Eggestad                  mailto:[email protected]
Scali Scalable Linux Systems    http://www.scali.com

Olaf Helsets Vei 6              tel:    +47 22 62 89 61 (OFFICE)
P.O.Box 150, Oppsal                     +47 975 31 574  (MOBILE)
N-0619 Oslo                     fax:    +47 22 62 89 51
NORWAY            
_________________________________________________________________________

I think Alan had the ISR and kernel IRQ service overhead in mind. In this light, hardware interrupt latency is a drop in the ocean..

Cheers,
Zwane Mwaikambo

http://function.linuxpower.ca

It depends what the interrupt actually does. What I was saying is that the processing overhead is what hurts not the latency. If you think about a continual stream of events latency on the busses effectively skews the delivery time of the interrupt but not the rate of processing.