Skip to contentSkip to conversation

Help enabling PCI interrupts on Dell/SMP and Sun/SMP systems.

DISCUSSION Alan Kilian 4 participants 8 messages Feb 23, 2005 – Feb 24, 2005

lore
Share & tools
lore
#0 lore raw

Folks,

This group was instrumental in helping me get my first-ever linux/PCI-bus device driver working last year, and I'm back for some more help if you are willing.

I have a PCI card that generates an interrupt when it completes a DMA transfer to the PCs RAM.

This works just fine on a Dell 4400 running 2.6.10-1.766_FC3

When I try to run the driver on a Dell 2300 FC2/2.6.5-1.358smp or a Sun W2100Z running FC2/2.6.10-1.14_FC2smp I can see the DMA-done bit set in the device, but my interrupt service routine never gets called.

On the Sun, I booted with "noapic" option, and it booted OK, but then when my device generated an interrupt, there was a kernel message about Disabling IRQ #5 and the system was hung solidly.

I think this has something to do with the different interrupt hardware on the more advanced servers compared to my desktop Dell 4400, and I somehow need to "enable" the IOAPIC system so that my interrupt gets through to my service routine, but I don't know how.

I tried grepping through the kernel/drivers source, and I didn't find anything that jumped out at me.

The Rubini drivers book didn't help in this area either, although it's a wonderful book in other areas.

I can post source somewhere if it will help.

I can also post the essential bits from /var/log/messages about all the incredibly complicated IOAPIC configuration stuff.

Thank you for your past help, and thank you in advance for any tips you can provide.

-Alan

Signature · 1 line
- Alan Kilian <kilian(at)bobodyne.com>
#1 lore raw

On Wed, 23 Feb 2005, Alan Kilian wrote:

call pci_enable_device(dev)
... before you use the IRQ in dev->irq.

The reported IRQ is bogus until you make that call. It's a reported BUG, probably won't ever get fixed because it's considered a feature.

Also, make sure that your .config for the Dell looks something like:

CONFIG_X86_IO_APIC=y
CONFIG_X86_LOCAL_APIC=y
CONFIG_PCI=y
# CONFIG_PCI_GOBIOS is not set
# CONFIG_PCI_GODIRECT is not set
CONFIG_PCI_GOANY=y
CONFIG_PCI_BIOS=y
CONFIG_PCI_DIRECT=y

Quoted 37 lines

Folks,

This group was instrumental in helping me get my first-ever linux/PCI-bus device driver working last year, and I'm back for some more help if you are willing.

I have a PCI card that generates an interrupt when it completes a DMA transfer to the PCs RAM.

This works just fine on a Dell 4400 running 2.6.10-1.766_FC3

When I try to run the driver on a Dell 2300 FC2/2.6.5-1.358smp or a Sun W2100Z running FC2/2.6.10-1.14_FC2smp I can see the DMA-done bit set in the device, but my interrupt service routine never gets called.

On the Sun, I booted with "noapic" option, and it booted OK, but then when my device generated an interrupt, there was a kernel message about Disabling IRQ #5 and the system was hung solidly.

I think this has something to do with the different interrupt hardware on the more advanced servers compared to my desktop Dell 4400, and I somehow need to "enable" the IOAPIC system so that my interrupt gets through to my service routine, but I don't know how.

I tried grepping through the kernel/drivers source, and I didn't find anything that jumped out at me.

The Rubini drivers book didn't help in this area either, although it's a wonderful book in other areas.

I can post source somewhere if it will help.

Quoted 11 lines

I can also post the essential bits from /var/log/messages about all the incredibly complicated IOAPIC configuration stuff.

Thank you for your past help, and thank you in advance for any tips you can provide.

-Alan

Signature · 1 line
- Alan Kilian <kilian(at)bobodyne.com>

Cheers,
Dick Johnson
Penguin : Linux version 2.6.10 on an i686 machine (5537.79 BogoMips).
Notice : All mail here is now cached for review by Dictator Bush.
98.36% of all statistics are fiction.

#2 lore raw
Quoted 11 lines · Dick Johnson
(quoted diff) +0 -0
 	call	pci_enable_device(dev)
 	... before you use the IRQ in dev->irq.
 
 	The reported IRQ is bogus until you make that
 	call. It's a reported BUG, probably won't
 	ever get fixed because it's considered a
 	feature.
 
 	Also, make sure that your .config for the Dell looks

Review

Quoted 10 lines
(quoted diff) +0 -0
 	something like:
 
 	CONFIG_X86_IO_APIC=y
 	CONFIG_X86_LOCAL_APIC=y
 	CONFIG_PCI=y
 	# CONFIG_PCI_GOBIOS is not set
 	# CONFIG_PCI_GODIRECT is not set
 	CONFIG_PCI_GOANY=y
 	CONFIG_PCI_BIOS=y
 	CONFIG_PCI_DIRECT=y

Dick,

Thanks for the quick reply.

1) I call pci_enable_device(dev) immediatly after I call dev = pci_find_device(0x1492, PCI_ANY_ID, dev);

2) I have verified all the CONFIG settings you suggested.

Here is cat /proc/interrupts on my working dell:

CPU0

  	  0:   16891629          XT-PIC  timer
  	  1:         10          XT-PIC  i8042
  	  2:          0          XT-PIC  cascade
  	  3:          2          XT-PIC  parport0
  	  5:        764          XT-PIC  sse
  	  7:        422          XT-PIC  ohci_hcd
  	  8:          1          XT-PIC  rtc
  	  9:          0          XT-PIC  acpi
 	 11:      35198          XT-PIC  eth0
 	 12:         66          XT-PIC  i8042
 	 14:      42769          XT-PIC  ide0
 	 15:     151569          XT-PIC  ide1

NMI: 0
ERR: 0

My driver is called "sse" and is interrupting at IRQ #5

Here is cat /proc/interrupts on my non-working Sun:

CPU0 CPU1

  	  0:    7302649    7417311    IO-APIC-edge  timer
  	  5:          0          0    IO-APIC-edge  sse
  	  8:          0          1    IO-APIC-edge  rtc
  	  9:          0          0   IO-APIC-level  acpi
 	 15:          1        478    IO-APIC-edge  ide1
	169:          0         30   IO-APIC-level  aic79xx
	177:      13991      18084   IO-APIC-level  aic79xx
	185:          0          3   IO-APIC-level  ehci_hcd
	193:          0         26   IO-APIC-level  ohci_hcd
	201:          0         21   IO-APIC-level  ohci_hcd
	209:     167119         87   IO-APIC-level  eth0

NMI: 0 0
LOC: 14719159 14719203
ERR: 0
MIS: 0

It appears that the card is also interrupting at IRQ#5

There is an interesting message on the Sun in /var/log/messages:

Feb 23 14:01:26 sunw1200z kernel: sse: no version magic, tainting
kernel.
Feb 23 14:01:26 sunw1200z kernel: SSE: Found a DeCypher card.
Feb 23 14:01:26 sunw1200z kernel: ACPI: PCI interrupt 0000:13:03.0[A] ->
GSI 36 (level, low) -> IRQ 217
Feb 23 14:01:26 sunw1200z kernel: interrupting on line 5
Feb 23 14:01:26 sunw1200z kernel: SSE: bar[0] From 0xd2806000 to
0xd2806fff F=0x200 MEMORY space
Feb 23 14:01:26 sunw1200z kernel: SSE: bar[1] From 0xd2800000 to
0xd2801fff F=0x200 MEMORY space
Feb 23 14:01:26 sunw1200z kernel: SSE: bar[2] From 0xd2000000 to
0xd27fffff F=0x200 MEMORY space
Feb 23 14:01:26 sunw1200z kernel: SSE: bar[0] mybase = 0xf889a000 size =
0x00001000 D'4096
Feb 23 14:01:26 sunw1200z kernel: SSE: bar[1] mybase = 0xf889c000 size =
0x00002000 D'8192
Feb 23 14:01:26 sunw1200z kernel: SSE: bar[2] mybase = 0xf8b00000 size =
0x00800000 D'8388608
Feb 23 14:01:26 sunw1200z kernel: pci_alloc_consistent returned
0xf0ded000
Feb 23 14:01:26 sunw1200z kernel: sse_read_bus_buffer 0x30ded000
Feb 23 14:01:26 sunw1200z kernel: request_irq() returned 0
Feb 23 14:01:26 sunw1200z kernel: SSE device_id 3, Rev 4
Feb 23 14:01:26 sunw1200z kernel: SSE Before: intstatus = 0x00000000
Feb 23 14:01:26 sunw1200z kernel: SSE Before: intstatus = 0x00000000
Feb 23 14:01:26 sunw1200z kernel: SSE: End of card attachment. Number of
cards = 1
Feb 23 14:01:26 sunw1200z kernel: Iterating through the softp
structures...
Feb 23 14:01:26 sunw1200z kernel: Card at softp->mem_reg1 0xf889a000

minor = 0

The interesting bits seem to be these two lines:

kernel: SSE: Found a DeCypher card.
kernel: ACPI: PCI interrupt 0000:13:03.0[A] -> GSI 36 (level, low) ->
IRQ 217

The first message is in my driver after pci_find_device() The second is from when I do pci_enable_device(dev);

Can you decode the mysterious ACPI message?

-Alan

Signature · 1 line
- Alan Kilian <kilian(at)bobodyne.com>
#3 lore raw
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Alan" == Alan Kilian <[email protected]> writes:

Alan> kernel: SSE: Found a DeCypher card. kernel: ACPI: PCI
Alan> interrupt 0000:13:03.0[A] -> GSI 36 (level, low) -> IRQ 217

If ACPI has set this device up to use interrupt 217, why are you registering it on IRQ 5?

--
Dr Peter Chubb http://www.gelato.unsw.edu.au peterc AT gelato.unsw.edu.au The technical we do immediately, the political takes *forever*

#4 lore raw
Quoted 6 lines

Alan> kernel: SSE: Found a DeCypher card. kernel: ACPI: PCI
Alan> interrupt 0000:13:03.0[A] -> GSI 36 (level, low) -> IRQ 217

If ACPI has set this device up to use interrupt 217, why are you registering it on IRQ 5?

Peter,

Maybe that's it.

I ask the card which interrupt line it was given at boot-time:

pci_read_config_byte(dev, PCI_INTERRUPT_LINE,
&softp->interrupt_line);

Then I request an IRQ:

request_irq(softp->interrupt_line, sseintr,
SA_INTERRUPT, "sse", softp);

I don't know who exactly is assigning IRQ 217, and I don't know where I could read that value.

Are you suggesting I should call request_irq(217, sseintr,
SA_INTERRUPT, "sse", softp);

Maybe you are on to the problem.

I found out that if I boot with "pci-noapic" that everyting
works on the Sun machine!!!

-Alan

Signature · 1 line
- Alan Kilian <kilian(at)bobodyne.com>
#5 lore raw

Alan> I ask the card which interrupt line it was given at Alan> boot-time:

Alan> pci_read_config_byte(dev, PCI_INTERRUPT_LINE, &softp->interrupt_line);

Alan> Then I request an IRQ:

Alan> request_irq(softp->interrupt_line, sseintr, SA_INTERRUPT, "sse", softp);

Don't do that. The kernel may need you to use a different interrupt number than you read from the PCI config header. Use dev->irq, as in

	request_irq(dev->irq, sseintr, SA_SHIRQ | SA_INTERRUPT, "sse", softp);

Also, make sure that you do pci_enable_device(dev) before you look at dev->irq, since the field will not be initialized until you do that.

- R.

#6 lore raw
Quoted 7 lines · Roland Dreier
    Alan> 	pci_read_config_byte(dev, PCI_INTERRUPT_LINE, &softp->interrupt_line);
    Alan> 	request_irq(softp->interrupt_line, sseintr, SA_INTERRUPT, "sse", softp);

Don't do that. The kernel may need you to use a different interrupt number than you read from the PCI config header. Use dev->irq, as in

	request_irq(dev->irq, sseintr, SA_SHIRQ | SA_INTERRUPT, "sse", softp);

Roland,

You are the best. Thank you so much for this information.

That was the ticket.

request_irq(dev->irq, sseintr, SA_SHIRQ | SA_INTERRUPT,
"sse", softp);

It works on all machines now without adding "pci=noapci"

I'm running again!!!

-Alan

Signature · 1 line
- Alan Kilian <kilian(at)bobodyne.com>
#7 lore raw
In reply to Alan Kilian

Where are you getting IRQ5 from? You can't "hard-code" interrupts on PCI.

Quoted 2 lines
	kernel: ACPI: PCI interrupt 0000:13:03.0[A] -> GSI 36 (level, low) ->

IRQ 217

^^^^^^^^^___________ This is your IRQ

It should be in dev->irq AFTER it's enabled.
[SNIPPED...]

Cheers,
Dick Johnson
Penguin : Linux version 2.6.10 on an i686 machine (5537.79 BogoMips).
Notice : All mail here is now cached for review by Dictator Bush.
98.36% of all statistics are fiction.