Skip to contentSkip to conversation

"No handler for vector" patches don't work on some systems

DISCUSSION Chuck Ebbert 2 participants 4 messages Mar 9, 2007

lore
Share & tools
lore
#0 lore raw

[sorry for the dup: this time to the right recipient]

So far I've tried the simple "survive having no handler for a vector" patch and the preliminary 3-patch series that was in -mm for a while, and neither work on the Dell PowerEdge 29xx and 19xx systems. These servers have the Intel 5000X chipset with the 6700PXH PCI Hub with dual independent PCI-X busses, each with its own I/OxAPIC with 24 interrupts. The fixes do work on "simple" systems but not on these high-end ones.

#1 lore raw
Quoted 10 lines · Chuck Ebbert

[sorry for the dup: this time to the right recipient]

So far I've tried the simple "survive having no handler for a vector" patch and the preliminary 3-patch series that was in -mm for a while, and neither work on the Dell PowerEdge 29xx and 19xx systems. These servers have the Intel 5000X chipset with the 6700PXH PCI Hub with dual independent PCI-X busses, each with its own I/OxAPIC with 24 interrupts. The fixes do work on "simple" systems but not on these high-end ones.

Ok thanks for the report. It sounds like there is another cause for the problem in the Dell case.

The simple patch drops the interrupt handler but acknowledges the hardware so if the driver can survive missing an interrupt we should be ok. With level triggered interrupts this should pretty much be guaranteed as after the acknowledgement the unhandled interrupt will be refired.

One of my internal test systems had a 6700PXH PCI hub (at least I think that was the part) the E7520 chipset. So I don't think it is just a matter of the hardware. Although I do recall Intel having an errata out on that class of hardware for occasionally reordering interrupt messages with the end of interrupt coming before the interrupt message itself. Causing various things to get confused. It would not surprise me if we were tickling some errata like that.

I would very much like to know if what I merged linus's tree helps. It is a little more conservative, than my earlier patches. I need a way to reproduce this or to work closely with someone who is, because this sounds like it has a different cause and I need to start with that assumption.

Eric

#2 lore raw
Quoted 12 lines · Eric W. Biederman
Quoted 9 lines · Chuck Ebbert

So far I've tried the simple "survive having no handler for a vector" patch and the preliminary 3-patch series that was in -mm for a while, and neither work on the Dell PowerEdge 29xx and 19xx systems. These servers have the Intel 5000X chipset with the 6700PXH PCI Hub with dual independent PCI-X busses, each with its own I/OxAPIC with 24 interrupts. The fixes do work on "simple" systems but not on these high-end ones.

Quoted 5 lines

I would very much like to know if what I merged linus's tree helps. It is a little more conservative, than my earlier patches. I need a way to reproduce this or to work closely with someone who is, because this sounds like it has a different cause and I need to start with that assumption.

Was that merged or is it still in -mm? The last thing I see in arch/x86_64/irq.c is:

[PATCH] x86-64: survive having no irq mapping for a vector

And we tried that one.

#3 lore raw
Quoted 19 lines · Chuck Ebbert
Quoted 12 lines · Eric W. Biederman
Quoted 9 lines · Chuck Ebbert

So far I've tried the simple "survive having no handler for a vector" patch and the preliminary 3-patch series that was in -mm for a while, and neither work on the Dell PowerEdge 29xx and 19xx systems. These servers have the Intel 5000X chipset with the 6700PXH PCI Hub with dual independent PCI-X busses, each with its own I/OxAPIC with 24 interrupts. The fixes do work on "simple" systems but not on these high-end ones.

Quoted 5 lines

I would very much like to know if what I merged linus's tree helps. It is a little more conservative, than my earlier patches. I need a way to reproduce this or to work closely with someone who is, because this sounds like it has a different cause and I need to start with that assumption.

Quoted 6 lines

Was that merged or is it still in -mm? The last thing I see in arch/x86_64/irq.c is:

[PATCH] x86-64: survive having no irq mapping for a vector

And we tried that one.

Look in arch/x86_64/io_apic.c. That is where most of the work happened. If you can extract that patch series for a backport more power to you.

Eric

commit 610142927b5bc149da92b03c7ab08b8b5f205b74

Author: Eric W. Biederman <[email protected]>
Date:   Fri Feb 23 04:40:58 2007 -0700

[PATCH] x86_64 irq: Safely cleanup an irq after moving it.

The problem: After moving an interrupt when is it safe to teardown the data structures for receiving the interrupt at the old location?

With a normal pci device it is possible to issue a read to a device to flush all posted writes. This does not work for the oldest ioapics because they are on a 3-wire apic bus which is a completely different data path. For some more modern ioapics when everything is using front side bus delivery you can flush interrupts by simply issuing a read to the ioapic. For other modern ioapics emperical testing has shown that this does not work.

So it appears the only reliable way to know the last of the irqs from an ioapic have been received from before the ioapic was reprogrammed is to received the first irq from the ioapic from after it was reprogrammed.

Once we know the last irq message has been received from an ioapic into a local apic we then need to know that irq message has been processed through the local apics.