Help enabling PCI interrupts on Dell/SMP and Sun/SMP systems.
Folks,
This group was instrumental in helping me get my first-ever linux/PCI-bus device driver working last year, and I'm back for some more help if you are willing.
I have a PCI card that generates an interrupt when it completes a DMA transfer to the PCs RAM.
This works just fine on a Dell 4400 running 2.6.10-1.766_FC3
When I try to run the driver on a Dell 2300 FC2/2.6.5-1.358smp or a Sun W2100Z running FC2/2.6.10-1.14_FC2smp I can see the DMA-done bit set in the device, but my interrupt service routine never gets called.
On the Sun, I booted with "noapic" option, and it booted OK, but then when my device generated an interrupt, there was a kernel message about Disabling IRQ #5 and the system was hung solidly.
I think this has something to do with the different interrupt hardware on the more advanced servers compared to my desktop Dell 4400, and I somehow need to "enable" the IOAPIC system so that my interrupt gets through to my service routine, but I don't know how.
I tried grepping through the kernel/drivers source, and I didn't find anything that jumped out at me.
The Rubini drivers book didn't help in this area either, although it's a wonderful book in other areas.
I can post source somewhere if it will help.
I can also post the essential bits from /var/log/messages about all the incredibly complicated IOAPIC configuration stuff.
Thank you for your past help, and thank you in advance for any tips you can provide.
-Alan
Signature · 1 line
- Alan Kilian <kilian(at)bobodyne.com>
On Wed, 23 Feb 2005, Alan Kilian wrote:
call pci_enable_device(dev)
... before you use the IRQ in dev->irq.
The reported IRQ is bogus until you make that call. It's a reported BUG, probably won't ever get fixed because it's considered a feature.
Also, make sure that your .config for the Dell looks something like:
CONFIG_X86_IO_APIC=y
CONFIG_X86_LOCAL_APIC=y
CONFIG_PCI=y
# CONFIG_PCI_GOBIOS is not set
# CONFIG_PCI_GODIRECT is not set
CONFIG_PCI_GOANY=y
CONFIG_PCI_BIOS=y
CONFIG_PCI_DIRECT=y
Quoted 37 lines
Folks,
This group was instrumental in helping me get my first-ever linux/PCI-bus device driver working last year, and I'm back for some more help if you are willing.
I have a PCI card that generates an interrupt when it completes a DMA transfer to the PCs RAM.
This works just fine on a Dell 4400 running 2.6.10-1.766_FC3
When I try to run the driver on a Dell 2300 FC2/2.6.5-1.358smp or a Sun W2100Z running FC2/2.6.10-1.14_FC2smp I can see the DMA-done bit set in the device, but my interrupt service routine never gets called.
On the Sun, I booted with "noapic" option, and it booted OK, but then when my device generated an interrupt, there was a kernel message about Disabling IRQ #5 and the system was hung solidly.
I think this has something to do with the different interrupt hardware on the more advanced servers compared to my desktop Dell 4400, and I somehow need to "enable" the IOAPIC system so that my interrupt gets through to my service routine, but I don't know how.
I tried grepping through the kernel/drivers source, and I didn't find anything that jumped out at me.
The Rubini drivers book didn't help in this area either, although it's a wonderful book in other areas.
I can post source somewhere if it will help.
Quoted 11 lines
I can also post the essential bits from
/var/log/messagesabout all the incredibly complicated IOAPIC configuration stuff.Thank you for your past help, and thank you in advance for any tips you can provide.
-Alan
Signature · 1 line
- Alan Kilian <kilian(at)bobodyne.com>
Cheers,
Dick Johnson
Penguin : Linux version 2.6.10 on an i686 machine (5537.79 BogoMips).
Notice : All mail here is now cached for review by Dictator Bush.
98.36% of all statistics are fiction.
Quoted 11 lines · Dick Johnson
(quoted diff) +0
-0call pci_enable_device(dev) ... before you use the IRQ in dev->irq. The reported IRQ is bogus until you make that call. It's a reported BUG, probably won't ever get fixed because it's considered a feature. Also, make sure that your .config for the Dell looks
Review
Quoted 10 lines
(quoted diff) +0
-0something like: CONFIG_X86_IO_APIC=y CONFIG_X86_LOCAL_APIC=y CONFIG_PCI=y # CONFIG_PCI_GOBIOS is not set # CONFIG_PCI_GODIRECT is not set CONFIG_PCI_GOANY=y CONFIG_PCI_BIOS=y CONFIG_PCI_DIRECT=y
Dick,
Thanks for the quick reply.
1) I call pci_enable_device(dev) immediatly after I call dev = pci_find_device(0x1492, PCI_ANY_ID, dev);
2) I have verified all the CONFIG settings you suggested.
Here is cat /proc/interrupts on my working dell:
CPU0
0: 16891629 XT-PIC timer 1: 10 XT-PIC i8042 2: 0 XT-PIC cascade 3: 2 XT-PIC parport0 5: 764 XT-PIC sse 7: 422 XT-PIC ohci_hcd 8: 1 XT-PIC rtc 9: 0 XT-PIC acpi 11: 35198 XT-PIC eth0 12: 66 XT-PIC i8042 14: 42769 XT-PIC ide0 15: 151569 XT-PIC ide1
NMI: 0
ERR: 0
My driver is called "sse" and is interrupting at IRQ #5
Here is cat /proc/interrupts on my non-working Sun:
CPU0 CPU1
0: 7302649 7417311 IO-APIC-edge timer 5: 0 0 IO-APIC-edge sse 8: 0 1 IO-APIC-edge rtc 9: 0 0 IO-APIC-level acpi 15: 1 478 IO-APIC-edge ide1 169: 0 30 IO-APIC-level aic79xx 177: 13991 18084 IO-APIC-level aic79xx 185: 0 3 IO-APIC-level ehci_hcd 193: 0 26 IO-APIC-level ohci_hcd 201: 0 21 IO-APIC-level ohci_hcd 209: 167119 87 IO-APIC-level eth0
NMI: 0 0
LOC: 14719159 14719203
ERR: 0
MIS: 0
It appears that the card is also interrupting at IRQ#5
There is an interesting message on the Sun in /var/log/messages:
Feb 23 14:01:26 sunw1200z kernel: sse: no version magic, tainting kernel. Feb 23 14:01:26 sunw1200z kernel: SSE: Found a DeCypher card. Feb 23 14:01:26 sunw1200z kernel: ACPI: PCI interrupt 0000:13:03.0[A] -> GSI 36 (level, low) -> IRQ 217 Feb 23 14:01:26 sunw1200z kernel: interrupting on line 5 Feb 23 14:01:26 sunw1200z kernel: SSE: bar[0] From 0xd2806000 to 0xd2806fff F=0x200 MEMORY space Feb 23 14:01:26 sunw1200z kernel: SSE: bar[1] From 0xd2800000 to 0xd2801fff F=0x200 MEMORY space Feb 23 14:01:26 sunw1200z kernel: SSE: bar[2] From 0xd2000000 to 0xd27fffff F=0x200 MEMORY space Feb 23 14:01:26 sunw1200z kernel: SSE: bar[0] mybase = 0xf889a000 size = 0x00001000 D'4096 Feb 23 14:01:26 sunw1200z kernel: SSE: bar[1] mybase = 0xf889c000 size = 0x00002000 D'8192 Feb 23 14:01:26 sunw1200z kernel: SSE: bar[2] mybase = 0xf8b00000 size = 0x00800000 D'8388608 Feb 23 14:01:26 sunw1200z kernel: pci_alloc_consistent returned 0xf0ded000 Feb 23 14:01:26 sunw1200z kernel: sse_read_bus_buffer 0x30ded000 Feb 23 14:01:26 sunw1200z kernel: request_irq() returned 0 Feb 23 14:01:26 sunw1200z kernel: SSE device_id 3, Rev 4 Feb 23 14:01:26 sunw1200z kernel: SSE Before: intstatus = 0x00000000 Feb 23 14:01:26 sunw1200z kernel: SSE Before: intstatus = 0x00000000 Feb 23 14:01:26 sunw1200z kernel: SSE: End of card attachment. Number of cards = 1 Feb 23 14:01:26 sunw1200z kernel: Iterating through the softp structures... Feb 23 14:01:26 sunw1200z kernel: Card at softp->mem_reg1 0xf889a000
minor = 0
The interesting bits seem to be these two lines:
kernel: SSE: Found a DeCypher card.
kernel: ACPI: PCI interrupt 0000:13:03.0[A] -> GSI 36 (level, low) ->
IRQ 217
The first message is in my driver after pci_find_device() The second is from when I do pci_enable_device(dev);
Can you decode the mysterious ACPI message?
-Alan
Signature · 1 line
- Alan Kilian <kilian(at)bobodyne.com>
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Alan" == Alan Kilian <[email protected]> writes:
Alan> kernel: SSE: Found a DeCypher card. kernel: ACPI: PCI
Alan> interrupt 0000:13:03.0[A] -> GSI 36 (level, low) -> IRQ 217
If ACPI has set this device up to use interrupt 217, why are you registering it on IRQ 5?
--
Dr Peter Chubb http://www.gelato.unsw.edu.au peterc AT gelato.unsw.edu.au The technical we do immediately, the political takes *forever*
Quoted 6 lines
Alan> kernel: SSE: Found a DeCypher card. kernel: ACPI: PCI
Alan> interrupt 0000:13:03.0[A] -> GSI 36 (level, low) -> IRQ 217If ACPI has set this device up to use interrupt 217, why are you registering it on IRQ 5?
Peter,
Maybe that's it.
I ask the card which interrupt line it was given at boot-time:
pci_read_config_byte(dev, PCI_INTERRUPT_LINE,
&softp->interrupt_line);
Then I request an IRQ:
request_irq(softp->interrupt_line, sseintr,
SA_INTERRUPT, "sse", softp);
I don't know who exactly is assigning IRQ 217, and I don't know where I could read that value.
Are you suggesting I should call request_irq(217, sseintr,
SA_INTERRUPT, "sse", softp);
Maybe you are on to the problem.
I found out that if I boot with "pci-noapic" that everyting
works on the Sun machine!!!
-Alan
Signature · 1 line
- Alan Kilian <kilian(at)bobodyne.com>
Alan> I ask the card which interrupt line it was given at Alan> boot-time:
Alan> pci_read_config_byte(dev, PCI_INTERRUPT_LINE, &softp->interrupt_line);
Alan> Then I request an IRQ:
Alan> request_irq(softp->interrupt_line, sseintr, SA_INTERRUPT, "sse", softp);
Don't do that. The kernel may need you to use a different interrupt number than you read from the PCI config header. Use dev->irq, as in
request_irq(dev->irq, sseintr, SA_SHIRQ | SA_INTERRUPT, "sse", softp);
Also, make sure that you do pci_enable_device(dev) before you look at dev->irq, since the field will not be initialized until you do that.
- R.
Quoted 7 lines · Roland Dreier
Alan> pci_read_config_byte(dev, PCI_INTERRUPT_LINE, &softp->interrupt_line); Alan> request_irq(softp->interrupt_line, sseintr, SA_INTERRUPT, "sse", softp);Don't do that. The kernel may need you to use a different interrupt number than you read from the PCI config header. Use
dev->irq, as inrequest_irq(dev->irq, sseintr, SA_SHIRQ | SA_INTERRUPT, "sse", softp);
Roland,
You are the best. Thank you so much for this information.
That was the ticket.
request_irq(dev->irq, sseintr, SA_SHIRQ | SA_INTERRUPT,
"sse", softp);
It works on all machines now without adding "pci=noapci"
I'm running again!!!
-Alan
Signature · 1 line
- Alan Kilian <kilian(at)bobodyne.com>
Where are you getting IRQ5 from? You can't "hard-code" interrupts on PCI.
Quoted 2 lines
kernel: ACPI: PCI interrupt 0000:13:03.0[A] -> GSI 36 (level, low) ->IRQ 217
^^^^^^^^^___________ This is your IRQ
It should be in dev->irq AFTER it's enabled.
[SNIPPED...]
Cheers,
Dick Johnson
Penguin : Linux version 2.6.10 on an i686 machine (5537.79 BogoMips).
Notice : All mail here is now cached for review by Dictator Bush.
98.36% of all statistics are fiction.