MSI messages
Hi,
I am wondering how to handle this:
Does the kernel somehow handle MSI message readback from the PCI config space for the MSI message as described in 6.8.1 of the PCI Bus specification 2.3 or does a device specific driver have to read the Message Address and Data ?
To put it short: i am wondering how i should read the MSI messages.
Thanks,
Manu
Quoted 9 lines · Manu Abraham
Hi,
I am wondering how to handle this:
Does the kernel somehow handle MSI message readback from the PCI config space for the MSI message as described in 6.8.1 of the PCI Bus specification 2.3 or does a device specific driver have to read the Message Address and Data ?
To put it short: i am wondering how i should read the MSI messages.
The "message" is actually mapped to an the interrupt vector by the core generic interrupt handling code in the kernel.
A "GSI" (Generic Sys Interrupt?) is associated with each entry in the MSI-X table. Driver then calls request_irq() to bind an interrupt handler to each GSI. So the driver never directly sees the "message".
hth,
grant
Quoted 9 lines · Grant Grundler
Quoted 8 lines · Manu Abraham
Hi,
I am wondering how to handle this:
Does the kernel somehow handle MSI message readback from the PCI config space for the MSI message as described in 6.8.1 of the PCI Bus specification 2.3 or does a device specific driver have to read the Message Address and Data ?
Quoted 8 lines
Quoted 1 line
To put it short: i am wondering how i should read the MSI messages.
The "message" is actually mapped to an the interrupt vector by the core generic interrupt handling code in the kernel.
A "GSI" (Generic Sys Interrupt?) is associated with each entry in the MSI-X table. Driver then calls
request_irq()to bind an interrupt handler to each GSI. So the driver never directly sees the "message".
Oh, you mean the array of irq_handlers in the MSI-X table should correspond to a particular message ?
I remember sometime back somebody stated that one might not get the requested number of handlers what you requested.
In such a case, if the irq_handlers are mapped in a static fashion, one might not get any interrupt callbacks for a specific functionality of the hardware. Or is there something missing in what i understood ?
Regards,
Manu
On Sun, Dec 14, 2008 at 03:15:15PM +0400, Manu Abraham wrote:
...
Quoted 6 lines
Quoted 3 lines
A "GSI" (Generic Sys Interrupt?) is associated with each entry in the MSI-X table. Driver then calls
request_irq()to bind an interrupt handler to each GSI. So the driver never directly sees the "message".Oh, you mean the array of
irq_handlersin the MSI-X table should correspond to a particular message ?
No. I mean each MSI-X entry has an address+message pair and each MSI-X entry is associated with the parameters passed to each call of request_irq(). Pass in unique private data to each call of request_irq() and the driver can determine which message was delivered when the ISR gets called. (ISR == Interrupt Service Routine, aka driver IRQ handler)
Quoted 2 lines
I remember sometime back somebody stated that one might not get the requested number of handlers what you requested.
Correct. The limit is defined by the number of GSI's available from the HW.
Quoted 3 lines
In such a case, if the
irq_handlersare mapped in a static fashion, one might not get any interrupt callbacks for a specific functionality of the hardware. Or is there something missing in what i understood ?
Correct.
grant
Quoted 1 line · Grant Grundler
A "GSI" (Generic Sys Interrupt?)
Should be "Global System Interrupt", as described in Section 5.2.12 of ACPIspec3.0b ;-)
Quoted 9 lines · Grant Grundler
Quoted 8 lines · Manu Abraham
Hi,
I am wondering how to handle this:
Does the kernel somehow handle MSI message readback from the PCI config space for the MSI message as described in 6.8.1 of the PCI Bus specification 2.3 or does a device specific driver have to read the Message Address and Data ?
Quoted 9 lines
Quoted 1 line
To put it short: i am wondering how i should read the MSI messages.
The "message" is actually mapped to an the interrupt vector by the core generic interrupt handling code in the kernel.
A "GSI" (Generic Sys Interrupt?) is associated with each entry in the MSI-X table. Driver then calls
request_irq()to bind an interrupt handler to each GSI. So the driver never directly sees the "message".
I think "GSI (Global System Interrupt)" is for identifying the I/O APIC pin among multiple I/O APICs. Maybe you wanted to mean the interrupt number managed by kernel (frequently called "IRQ")? Or am I misunderstanding something?
Thanks,
Kenji Kaneshige
Quoted 6 lines · Kenji Kaneshige
Quoted 5 lines · Grant Grundler
Quoted 1 line
To put it short: i am wondering how i should read the MSI messages.
The "message" is actually mapped to an the interrupt vector by the core generic interrupt handling code in the kernel.
Quoted 8 lines
Quoted 4 lines
A "GSI" (Generic Sys Interrupt?) is associated with each entry in the MSI-X table. Driver then calls
request_irq()to bind an interrupt handler to each GSI. So the driver never directly sees the "message".I think "GSI (Global System Interrupt)" is for identifying the I/O APIC pin among multiple I/O APICs. Maybe you wanted to mean the interrupt number managed by kernel (frequently called "IRQ")?
I thought "GSI" and "kernel interrupt number" are equivalents. But I could easily be confusing the terms here. If "GSI" is only used for identifying APIC input lines, then that's not right term.
Quoted 1 line
Or am I misunderstanding something?
You probably understand it better than I do. :)
thanks,
grant
Quoted 5 lines · Grant Grundler
On Sun, Dec 14, 2008 at 03:15:15PM +0400, Manu Abraham wrote:
...Quoted 3 lines
Quoted 3 lines
A "GSI" (Generic Sys Interrupt?) is associated with each entry in the MSI-X table. Driver then calls
request_irq()to bind an interrupt handler to each GSI. So the driver never directly sees the "message".
Quoted 8 lines
Quoted 2 lines
Oh, you mean the array of
irq_handlersin the MSI-X table should correspond to a particular message ?No. I mean each MSI-X entry has an address+message pair and each MSI-X entry is associated with the parameters passed to each call of
request_irq(). Pass in unique private data to each call ofrequest_irq()and the driver can determine which message was delivered when the ISR gets called. (ISR == Interrupt Service Routine, aka driver IRQ handler)
Is it possible that even when the config space says that multiple messages are supported, you cannot enable MSI-X ?
To explain what i mean here, from the logs in the 3 modes of operation MSI-X, MSI, INTx, when i am requesting MSI-X mode, i find that the MSI-X mode doesn't get enabled. Is this something that's supposed to be ?
The codebase that generates the logs is http://jusst.de/hg/saa716x
the module that causes it is the saa716x module @ linux/drivers/media/dvb/saa716x
Regards,
Manu
------------- with MSI-X -------------
saa716x_pci_init (0): found a NEMO reference board PCIe card
ACPI: PCI Interrupt 0000:05:00.0[A] -> GSI 19 (level, low) -> IRQ 19
PCI: Setting latency timer of device 0000:05:00.0 to 64 saa716x_request_irq (0): Using MSI-X mode saa716x_enable_msix (0): MSI-X request failed saa716x_request_irq (0): INT-A Mode
SAA7160 Rev 1 [1131:0000], irq: 19,
mmio: 0xf9480000
SAA7160 64Bit, MSI Disabled, MSI-X=32 msgssaa716x_getbootscript_setup (0): Domain 0: Clk PSS <0x218> Divider: 0x7f80d --> N=1, M=2, freq=1250 saa716x_getbootscript_setup (0): Domain 1: Clk DCS <0x21c> Divider: 0x7f80d --> N=1, M=2, freq=1250 saa716x_getbootscript_setup (0): Domain 2: Clk SPI <0x220> Divider: 0x7f825 --> N=1, M=5, freq=500 saa716x_getbootscript_setup (0): Domain 3: Clk I2C/Boot <0x224> Divider: 0x7f84d --> N=1, M=10, freq=250 saa716x_getbootscript_setup (0): Domain 4: Clk PHI <0x228> Divider: 0x7f80d --> N=1, M=2, freq=1250 saa716x_getbootscript_setup (0): Domain 5: Clk VI0 <0x22c> Divider: 0x7f84d --> N=1, M=10, freq=250 saa716x_getbootscript_setup (0): Domain 6: Clk VI1 <0x230> Divider: 0x7f84d --> N=1, M=10, freq=250 saa716x_getbootscript_setup (0): Domain 7: Clk FGPI0 <0x234> Divider: 0x7f84d --> N=1, M=10, freq=250 saa716x_getbootscript_setup (0): Domain 8: Clk FGPI1 <0x238> Divider: 0x7f84d --> N=1, M=10, freq=250 saa716x_getbootscript_setup (0): Domain 9: Clk FGPI2 <0x23c> Divider: 0x7f84d --> N=1, M=10, freq=250 saa716x_getbootscript_setup (0): Domain 10: Clk FGPI3 <0x240> Divider: 0x7f84d --> N=1, M=10, freq=250 saa716x_getbootscript_setup (0): Domain 11: Clk AI0 <0x244> Divider: 0x7f84d --> N=1, M=10, freq=250 saa716x_getbootscript_setup (0): Domain 12: Clk AI1 <0x248> Divider: 0x7f84d --> N=1, M=10, freq=250 saa716x_getbootscript_setup (0): Domain 13: Clk Phy <0x24c> Divider: 0x7f80d --> N=1, M=2, freq=1250 saa716x_core_boot (0): Using External Boot from configsaa716x_hybrid_pci_probe (0): SAA716x Core Boot Successsaa716x_msi_init (0): Initializing MSI ..saa716x_msi_init (0): Interrupts ena_l <00> ena_h <00> sta_l <00> sta_h <00> saa716x_jetpack_init (0): SAA7160 Jetpack Successfully initializedsaa716x_i2c_init (0): Initializing SAA7160 I2C Coresaa716x_i2c_init (0): Initializing adapter (0) SAA716x I2C Core 0saa716x_i2c_init (0): Initializing Adapter (0) SAA716x I2C Core 0 @ 100k saa716x_add_irqvector (0): Adding Vector 49 <I2CINT_0>saa716x_add_irqvector (0): Interrupts ena_l <00> ena_h <20000>saa716x_i2c_init (0): Initializing adapter (1) SAA716x I2C Core 1 saa716x_hybrid_pci_irq (0): MSI STAT L=<00> H=<20000>, CTL L=<00> H=<20000>saa716x_msi_event: MSI event <I2CINT_0> Bus(0) I2C event: Status=<Idle> --> Stat=<00> Mask=<c7> saa716x_i2c_init (0): Initializing Adapter (1) SAA716x I2C Core 1 @ 100k saa716x_add_irqvector (0): Adding Vector 50 <I2CINT_1>saa716x_add_irqvector (0): Interrupts ena_l <00> ena_h <60000>saa716x_i2c_init (0): SAA7160 I2C Core succesfully initializedsaa716x_hybrid_pci_irq (0): MSI STAT L=<00> H=<40000>, CTL L=<00> H=<60000>saa716x_msi_event: MSI event <I2CINT_1> Bus(1) I2C event: Status=<Idle> --> Stat=<00> Mask=<c7> saa716x_dvb_init (0): dvb_register_adapter
DVB: registering new adapter (SAA716x dvb adapter)saa716x_dvb_init (0): dvb_dmx_initsaa716x_dvb_init (0): dvb_dmxdev_initsaa716x_dvb_init (0): Frontend Initsaa716x_frontend_power (0): Adapter (0) Power ONsaa716x_frontend_reset (0): Adapter (0) RESETsaa716x_nemo_frontend_attach (0): Adapter (0) SAA716x frontend Init saa716x_nemo_frontend_attach (0): Adapter (0) Device ID=00
------------- with MSI -------------
saa716x_pci_init (0): found a NEMO reference board PCIe card
ACPI: PCI Interrupt 0000:05:00.0[A] -> GSI 19 (level, low) -> IRQ 19
PCI: Setting latency timer of device 0000:05:00.0 to 64 saa716x_request_irq (0): Using MSI mode
SAA7160 Rev 1 [1131:0000], irq: 215,
mmio: 0xf9480000
SAA7160 64Bit, MSI Enabled, MSI-X=32 msgssaa716x_getbootscript_setup (0): Domain 0: Clk PSS <0x218> Divider: 0x7f80d --> N=1, M=2, freq=1250 saa716x_getbootscript_setup (0): Domain 1: Clk DCS <0x21c> Divider: 0x7f80d --> N=1, M=2, freq=1250 saa716x_getbootscript_setup (0): Domain 2: Clk SPI <0x220> Divider: 0x7f825 --> N=1, M=5, freq=500 saa716x_getbootscript_setup (0): Domain 3: Clk I2C/Boot <0x224> Divider: 0x7f84d --> N=1, M=10, freq=250 saa716x_getbootscript_setup (0): Domain 4: Clk PHI <0x228> Divider: 0x7f80d --> N=1, M=2, freq=1250 saa716x_getbootscript_setup (0): Domain 5: Clk VI0 <0x22c> Divider: 0x7f84d --> N=1, M=10, freq=250 saa716x_getbootscript_setup (0): Domain 6: Clk VI1 <0x230> Divider: 0x7f84d --> N=1, M=10, freq=250 saa716x_getbootscript_setup (0): Domain 7: Clk FGPI0 <0x234> Divider: 0x7f84d --> N=1, M=10, freq=250 saa716x_getbootscript_setup (0): Domain 8: Clk FGPI1 <0x238> Divider: 0x7f84d --> N=1, M=10, freq=250 saa716x_getbootscript_setup (0): Domain 9: Clk FGPI2 <0x23c> Divider: 0x7f84d --> N=1, M=10, freq=250 saa716x_getbootscript_setup (0): Domain 10: Clk FGPI3 <0x240> Divider: 0x7f84d --> N=1, M=10, freq=250 saa716x_getbootscript_setup (0): Domain 11: Clk AI0 <0x244> Divider: 0x7f84d --> N=1, M=10, freq=250 saa716x_getbootscript_setup (0): Domain 12: Clk AI1 <0x248> Divider: 0x7f84d --> N=1, M=10, freq=250 saa716x_getbootscript_setup (0): Domain 13: Clk Phy <0x24c> Divider: 0x7f80d --> N=1, M=2, freq=1250 saa716x_core_boot (0): Using External Boot from configsaa716x_hybrid_pci_probe (0): SAA716x Core Boot Successsaa716x_msi_init (0): Initializing MSI ..saa716x_msi_init (0): Interrupts ena_l <00> ena_h <00> sta_l <00> sta_h <00> saa716x_jetpack_init (0): SAA7160 Jetpack Successfully initializedsaa716x_i2c_init (0): Initializing SAA7160 I2C Coresaa716x_i2c_init (0): Initializing adapter (0) SAA716x I2C Core 0saa716x_i2c_init (0): Initializing Adapter (0) SAA716x I2C Core 0 @ 100k saa716x_add_irqvector (0): Adding Vector 49 <I2CINT_0>saa716x_add_irqvector (0): Interrupts ena_l <00> ena_h <20000>saa716x_i2c_init (0): Initializing adapter (1) SAA716x I2C Core 1
irq 64, desc: c0483e80, depth: 1, count: 0, unhandled: 0
->handle_irq(): c01550f9, handle_bad_irq+0x0/0x1a5
->chip(): c0456030, no_irq_chip+0x0/0x40
->action(): 00000000
IRQ_DISABLED set
unexpected IRQ trap at vector 40saa716x_i2c_init (0): Initializing Adapter (1) SAA716x I2C Core 1 @ 100k saa716x_add_irqvector (0): Adding Vector 50 <I2CINT_1>saa716x_add_irqvector (0): Interrupts ena_l <00> ena_h <60000>saa716x_i2c_init (0): SAA7160 I2C Core succesfully initialized
irq 64, desc: c0483e80, depth: 1, count: 0, unhandled: 0
->handle_irq(): c01550f9, handle_bad_irq+0x0/0x1a5
->chip(): c0456030, no_irq_chip+0x0/0x40
->action(): 00000000
IRQ_DISABLED set
unexpected IRQ trap at vector 40saa716x_dvb_init (0): dvb_register_adapter
DVB: registering new adapter (SAA716x dvb adapter)saa716x_dvb_init (0): dvb_dmx_initsaa716x_dvb_init (0): dvb_dmxdev_initsaa716x_dvb_init (0): Frontend Initsaa716x_frontend_power (0): Adapter (0) Power ONsaa716x_frontend_reset (0): Adapter (0) RESETsaa716x_nemo_frontend_attach (0): Adapter (0) SAA716x frontend Init saa716x_nemo_frontend_attach (0): Adapter (0) Device ID=00
-------------with INTx -------------
saa716x_pci_init (0): found a NEMO reference board PCIe card
ACPI: PCI Interrupt 0000:05:00.0[A] -> GSI 19 (level, low) -> IRQ 19
PCI: Setting latency timer of device 0000:05:00.0 to 64
SAA7160 Rev 1 [1131:0000], irq: 19,
mmio: 0xf9480000
SAA7160 64Bit, MSI Disabled, MSI-X=32 msgssaa716x_getbootscript_setup (0): Domain 0: Clk PSS <0x218> Divider: 0x7f80d --> N=1, M=2, freq=1250 saa716x_getbootscript_setup (0): Domain 1: Clk DCS <0x21c> Divider: 0x7f80d --> N=1, M=2, freq=1250 saa716x_getbootscript_setup (0): Domain 2: Clk SPI <0x220> Divider: 0x7f825 --> N=1, M=5, freq=500 saa716x_getbootscript_setup (0): Domain 3: Clk I2C/Boot <0x224> Divider: 0x7f84d --> N=1, M=10, freq=250 saa716x_getbootscript_setup (0): Domain 4: Clk PHI <0x228> Divider: 0x7f80d --> N=1, M=2, freq=1250 saa716x_getbootscript_setup (0): Domain 5: Clk VI0 <0x22c> Divider: 0x7f84d --> N=1, M=10, freq=250 saa716x_getbootscript_setup (0): Domain 6: Clk VI1 <0x230> Divider: 0x7f84d --> N=1, M=10, freq=250 saa716x_getbootscript_setup (0): Domain 7: Clk FGPI0 <0x234> Divider: 0x7f84d --> N=1, M=10, freq=250 saa716x_getbootscript_setup (0): Domain 8: Clk FGPI1 <0x238> Divider: 0x7f84d --> N=1, M=10, freq=250 saa716x_getbootscript_setup (0): Domain 9: Clk FGPI2 <0x23c> Divider: 0x7f84d --> N=1, M=10, freq=250 saa716x_getbootscript_setup (0): Domain 10: Clk FGPI3 <0x240> Divider: 0x7f84d --> N=1, M=10, freq=250 saa716x_getbootscript_setup (0): Domain 11: Clk AI0 <0x244> Divider: 0x7f84d --> N=1, M=10, freq=250 saa716x_getbootscript_setup (0): Domain 12: Clk AI1 <0x248> Divider: 0x7f84d --> N=1, M=10, freq=250 saa716x_getbootscript_setup (0): Domain 13: Clk Phy <0x24c> Divider: 0x7f80d --> N=1, M=2, freq=1250 saa716x_core_boot (0): Using External Boot from configsaa716x_hybrid_pci_probe (0): SAA716x Core Boot Successsaa716x_msi_init (0): Initializing MSI ..saa716x_msi_init (0): Interrupts ena_l <00> ena_h <00> sta_l <00> sta_h <00> saa716x_jetpack_init (0): SAA7160 Jetpack Successfully initializedsaa716x_i2c_init (0): Initializing SAA7160 I2C Coresaa716x_i2c_init (0): Initializing adapter (0) SAA716x I2C Core 0saa716x_i2c_init (0): Initializing Adapter (0) SAA716x I2C Core 0 @ 100k saa716x_add_irqvector (0): Adding Vector 49 <I2CINT_0>saa716x_add_irqvector (0): Interrupts ena_l <00> ena_h <20000>saa716x_i2c_init (0): Initializing adapter (1) SAA716x I2C Core 1 saa716x_hybrid_pci_irq (0): MSI STAT L=<00> H=<20000>, CTL L=<00> H=<20000>saa716x_msi_event: MSI event <I2CINT_0> Bus(0) I2C event: Status=<Idle> --> Stat=<00> Mask=<c7> saa716x_i2c_init (0): Initializing Adapter (1) SAA716x I2C Core 1 @ 100k saa716x_add_irqvector (0): Adding Vector 50 <I2CINT_1>saa716x_add_irqvector (0): Interrupts ena_l <00> ena_h <60000>saa716x_i2c_init (0): SAA7160 I2C Core succesfully initializedsaa716x_hybrid_pci_irq (0): MSI STAT L=<00> H=<40000>, CTL L=<00> H=<60000>saa716x_msi_event: MSI event <I2CINT_1> Bus(1) I2C event: Status=<Idle> --> Stat=<00> Mask=<c7> saa716x_dvb_init (0): dvb_register_adapter
DVB: registering new adapter (SAA716x dvb adapter)saa716x_dvb_init (0): dvb_dmx_initsaa716x_dvb_init (0): dvb_dmxdev_initsaa716x_dvb_init (0): Frontend Initsaa716x_frontend_power (0): Adapter (0) Power ONsaa716x_frontend_reset (0): Adapter (0) RESETsaa716x_nemo_frontend_attach (0): Adapter (0) SAA716x frontend Init saa716x_nemo_frontend_attach (0): Adapter (0) Device ID=00
Quoted 16 lines · Manu Abraham
Quoted 5 lines · Grant Grundler
On Sun, Dec 14, 2008 at 03:15:15PM +0400, Manu Abraham wrote:
...Quoted 3 lines
Quoted 3 lines
A "GSI" (Generic Sys Interrupt?) is associated with each entry in the MSI-X table. Driver then calls
request_irq()to bind an interrupt handler to each GSI. So the driver never directly sees the "message".Quoted 8 lines
Quoted 2 lines
Oh, you mean the array of
irq_handlersin the MSI-X table should correspond to a particular message ?No. I mean each MSI-X entry has an address+message pair and each MSI-X entry is associated with the parameters passed to each call of
request_irq(). Pass in unique private data to each call ofrequest_irq()and the driver can determine which message was delivered when the ISR gets called. (ISR == Interrupt Service Routine, aka driver IRQ handler)
Quoted 2 lines
Is it possible that even when the config space says that multiple messages are supported, you cannot enable MSI-X ?
Yes, absolutely. Have a look at pci_msi_check_device() for starters. MSI can be disabled globally, or per-device by a quirk, the bridges above your device might not support MSI and the arch code may prevent MSI/MSI-X for some reason (ie. platform doesn't support it).
There's also a check in pci_enable_msix() to make sure the entries you pass in are valid.
Quoted 7 lines
------------- with MSI-X -------------
saa716x_pci_init(0): found a NEMO reference board PCIe card
ACPI: PCI Interrupt 0000:05:00.0[A] -> GSI 19 (level, low) -> IRQ 19
PCI: Setting latency timer of device 0000:05:00.0 to 64saa716x_request_irq(0): Using MSI-X modesaa716x_enable_msix(0): MSI-X request failed
For starters you should print the error code returned by pci_enable_msix() - unfortunately it returns EINVAL for many different reasons, but it will narrow it down a bit.
cheers
Signature · 8 lines
Michael Ellerman OzLabs, IBM Australia Development Lab wwweb: http://michael.ellerman.id.au phone: +61 2 6212 1183 (tie line 70 21183) We do not inherit the earth from our ancestors, we borrow it from our children. - S.M.A.R.T Person
Quoted 34 lines · Michael Ellerman
Quoted 8 lines · Manu Abraham
Quoted 7 lines · Grant Grundler
On Sun, Dec 14, 2008 at 03:15:15PM +0400, Manu Abraham wrote:
...Quoted 5 lines
Quoted 3 lines
A "GSI" (Generic Sys Interrupt?) is associated with each entry in the MSI-X table. Driver then calls
request_irq()to bind an interrupt handler to each GSI. So the driver never directly sees the "message".Oh, you mean the array of
irq_handlersin the MSI-X table should correspond to a particular message ?Quoted 8 lines
Quoted 5 lines
No. I mean each MSI-X entry has an address+message pair and each MSI-X entry is associated with the parameters passed to each call of
request_irq(). Pass in unique private data to each call ofrequest_irq()and the driver can determine which message was delivered when the ISR gets called. (ISR == Interrupt Service Routine, aka driver IRQ handler)Is it possible that even when the config space says that multiple messages are supported, you cannot enable MSI-X ?
Yes, absolutely. Have a look at
pci_msi_check_device()for starters. MSI can be disabled globally, or per-device by a quirk, the bridges above your device might not support MSI and the arch code may prevent MSI/MSI-X for some reason (ie. platform doesn't support it).There's also a check in
pci_enable_msix()to make sure the entries you pass in are valid.Quoted 7 lines
------------- with MSI-X -------------
saa716x_pci_init(0): found a NEMO reference board PCIe card
ACPI: PCI Interrupt 0000:05:00.0[A] -> GSI 19 (level, low) -> IRQ 19
PCI: Setting latency timer of device 0000:05:00.0 to 64saa716x_request_irq(0): Using MSI-X modesaa716x_enable_msix(0): MSI-X request failed
Quoted 3 lines
For starters you should print the error code returned by
pci_enable_msix()- unfortunately it returns EINVAL for many different reasons, but it will narrow it down a bit.
It does return -22
Regards,
Manu
saa716x_request_irq (0): Using MSI-X mode saa716x_enable_msix (0): MSI-X request failed <-22> saa716x_request_irq (0): INT-A Mode
SAA7160 Rev 1 [1131:0000], irq: 19,
mmio: 0xf9480000
SAA7160 64Bit, MSI Disabled, MSI-X=32 msgsQuoted 15 lines · Michael Ellerman
Quoted 14 lines · Manu Abraham
Quoted 12 lines · Grant Grundler
On Sun, Dec 14, 2008 at 03:15:15PM +0400, Manu Abraham wrote:
...Quoted 5 lines
Quoted 3 lines
A "GSI" (Generic Sys Interrupt?) is associated with each entry in the MSI-X table. Driver then calls
request_irq()to bind an interrupt handler to each GSI. So the driver never directly sees the "message".Oh, you mean the array of
irq_handlersin the MSI-X table should correspond to a particular message ?No. I mean each MSI-X entry has an address+message pair and each MSI-X entry is associated with the parameters passed to each call of
request_irq(). Pass in unique private data to each call ofrequest_irq()and the driver can determine which message was delivered when the ISR gets called. (ISR == Interrupt Service Routine, aka driver IRQ handler)
Quoted 7 lines
Quoted 2 lines
Is it possible that even when the config space says that multiple messages are supported, you cannot enable MSI-X ?
Yes, absolutely. Have a look at
pci_msi_check_device()for starters. MSI can be disabled globally, or per-device by a quirk, the bridges above your device might not support MSI and the arch code may prevent MSI/MSI-X for some reason (ie. platform doesn't support it).
It seems that the bridge does support MSI. It looks like my video device is using MSI.
Regards,
Manu
pci 0000:01:00.0: Boot video device
PCI: Setting latency timer of device 0000:00:01.0 to 64 assign_interrupt_mode Found MSI capability
Allocate Port Service[0000:00:01.0:pcie00]
PCI: Setting latency timer of device 0000:00:1c.0 to 64 assign_interrupt_mode Found MSI capability
Allocate Port Service[0000:00:1c.0:pcie00]
Allocate Port Service[0000:00:1c.0:pcie02]
PCI: Setting latency timer of device 0000:00:1c.1 to 64 assign_interrupt_mode Found MSI capability
Allocate Port Service[0000:00:1c.1:pcie00]
Allocate Port Service[0000:00:1c.1:pcie02]
PCI: Setting latency timer of device 0000:00:1c.2 to 64 assign_interrupt_mode Found MSI capability
Allocate Port Service[0000:00:1c.2:pcie00]
Allocate Port Service[0000:00:1c.2:pcie02]
PCI: Setting latency timer of device 0000:00:1c.3 to 64 assign_interrupt_mode Found MSI capability
Allocate Port Service[0000:00:1c.3:pcie00]
Allocate Port Service[0000:00:1c.3:pcie02]
PCI: Setting latency timer of device 0000:00:1c.4 to 64 assign_interrupt_mode Found MSI capability
Allocate Port Service[0000:00:1c.4:pcie00]
Allocate Port Service[0000:00:1c.4:pcie02]
vesafb: framebuffer at 0x91000000, mapped to 0xf8880000, using 10240k, total 14336k
vesafb: mode is 1280x1024x16, linelength=2560, pages=1
04:00.0 Multimedia controller: Philips Semiconductors Device 7160 (rev 02)
Subsystem: Technotrend Systemtechnik GmbH Device 3009
Flags: bus master, fast devsel, latency 0, IRQ 11 Memory at 93b00000 (64-bit, non-prefetchable) [size=1M]
Capabilities: <access denied>
05:00.0 Multimedia controller: Philips Semiconductors Device 7160 (rev 01)
Subsystem: Philips Semiconductors Device 0000
Flags: bus master, fast devsel, latency 0, IRQ 19 Memory at 93a00000 (64-bit, non-prefetchable) [size=1M]
Capabilities: <access denied>
Kernel driver in use: SAA716x Hybrid
Quoted 35 lines · Manu Abraham
Quoted 34 lines · Michael Ellerman
Quoted 8 lines · Manu Abraham
Quoted 7 lines · Grant Grundler
On Sun, Dec 14, 2008 at 03:15:15PM +0400, Manu Abraham wrote:
...Quoted 5 lines
Quoted 3 lines
A "GSI" (Generic Sys Interrupt?) is associated with each entry in the MSI-X table. Driver then calls
request_irq()to bind an interrupt handler to each GSI. So the driver never directly sees the "message".Oh, you mean the array of
irq_handlersin the MSI-X table should correspond to a particular message ?Quoted 8 lines
Quoted 5 lines
No. I mean each MSI-X entry has an address+message pair and each MSI-X entry is associated with the parameters passed to each call of
request_irq(). Pass in unique private data to each call ofrequest_irq()and the driver can determine which message was delivered when the ISR gets called. (ISR == Interrupt Service Routine, aka driver IRQ handler)Is it possible that even when the config space says that multiple messages are supported, you cannot enable MSI-X ?
Yes, absolutely. Have a look at
pci_msi_check_device()for starters. MSI can be disabled globally, or per-device by a quirk, the bridges above your device might not support MSI and the arch code may prevent MSI/MSI-X for some reason (ie. platform doesn't support it).There's also a check in
pci_enable_msix()to make sure the entries you pass in are valid.Quoted 7 lines
------------- with MSI-X -------------
saa716x_pci_init(0): found a NEMO reference board PCIe card
ACPI: PCI Interrupt 0000:05:00.0[A] -> GSI 19 (level, low) -> IRQ 19
PCI: Setting latency timer of device 0000:05:00.0 to 64saa716x_request_irq(0): Using MSI-X modesaa716x_enable_msix(0): MSI-X request failed
Quoted 5 lines
Quoted 3 lines
For starters you should print the error code returned by
pci_enable_msix()- unfortunately it returns EINVAL for many different reasons, but it will narrow it down a bit.It does return -22
If you're curious you could try this patch.
Quoted 1 line
From
bd27f10c9acd24bb849d14b6fc373444322c7405Mon Sep 17 00:00:00 2001
From: Michael Ellerman <[email protected]>
Date: Mon, 22 Dec 2008 10:21:27 +1100
Subject: [PATCH] MSI debug
--- drivers/pci/msi.c | 52 ++++++++++++++++++++++++++++++++++++++++------------ 1 files changed, 40 insertions(+), 12 deletions(-)
drivers/pci/msi.c +40 -12
@@ -517,17 +517,26 @@ static int pci_msi_check_device(struct pci_dev* dev, int nvec, int type)struct pci_bus *bus;int ret; +if (!dev) { +printk (KERN_DEBUG ,"MSI: null device\n" ); +return -ENODEV ; + } +/* MSI must be globally enabled and supported by the device */ -if (!pci_msi_enable || !dev || dev->no_msi) +if (!pci_msi_enable || dev->no_msi) +dev_printk (KERN_DEBUG , &dev->dev,"MSI: MSI disabled\n" );return -EINVAL ; + }/* * You can't ask to have 0 or less MSIs configured. * a) it's stupid .. * b) the list manipulation code assumes nvec >= 1. */ -if (nvec <1 ) +if (nvec <1 ) { +dev_printk (KERN_DEBUG , &dev->dev,"MSI: < 1 MSI requested\n" );return -ERANGE ; + }/* Any bridge which does NOT route MSI transactions from it's * secondary bus to it's primary bus must set NO_MSI flag on
@@ -535,16 +544,24 @@ static int pci_msi_check_device(struct pci_dev* dev, int nvec, int type)* We expect only arch-specific PCI host bus controller driver * or quirks for specific PCI bridges to be setting NO_MSI. */ -for (bus = dev->bus; bus; bus = bus->parent) -if (bus->bus_flags &PCI_BUS_FLAGS_NO_MSI ) +for (bus = dev->bus; bus; bus = bus->parent) { +if (bus->bus_flags &PCI_BUS_FLAGS_NO_MSI ) { +dev_printk (KERN_DEBUG , &dev->dev, +"MSI: bus check failed\n" );return -EINVAL ; + } + } ret =arch_msi_check_device (dev, nvec, type); -if (ret) +if (ret) { +dev_printk (KERN_DEBUG , &dev->dev,"MSI: arch check failed\n" );return ret; + } -if (!pci_find_capability (dev, type)) +if (!pci_find_capability (dev, type)) { +dev_printk (KERN_DEBUG , &dev->dev,"MSI: no capability found\n" );return -EINVAL ; + }return 0 ; }
@@ -669,26 +686,37 @@ int pci_enable_msix(struct pci_dev* dev, struct msix_entry *entries, int nvec)int i, j;u16 control; -if (!entries) -return -EINVAL ; - status =pci_msi_check_device (dev, nvec,PCI_CAP_ID_MSIX );if (status)return status; +if (!entries) { +dev_printk (KERN_DEBUG , &dev->dev,"MSI: null entries\n" ); +return -EINVAL ; + } + pos =pci_find_capability (dev,PCI_CAP_ID_MSIX );pci_read_config_word (dev,msi_control_reg (pos), &control); nr_entries =multi_msix_capable (control); -if (nvec > nr_entries) +if (nvec > nr_entries) { +dev_printk (KERN_DEBUG , &dev->dev,"MSI: too many entries\n" );return -EINVAL ; + }/* Check for any invalid entries */ for (i =0 ; i < nvec; i++) { -if (entries[i].entry >= nr_entries) +if (entries[i].entry >= nr_entries) { +dev_printk (KERN_DEBUG , &dev->dev, +"MSI: invalid entry %d\n" , i);return -EINVAL ;/* invalid entry */ + }for (j = i +1 ; j < nvec; j++) { -if (entries[i].entry == entries[j].entry) +if (entries[i].entry == entries[j].entry) { +dev_printk (KERN_DEBUG , &dev->dev, +"MSI: duplicate entries %d, %d\n" , + i, j);return -EINVAL ;/* duplicate entry */ + } } }WARN_ON (!!dev->msix_enabled);
Signature · 1 line
1.5.3.7.1.g4e596e
Quoted 33 lines · Michael Ellerman
Quoted 32 lines · Manu Abraham
Quoted 31 lines · Michael Ellerman
Quoted 8 lines · Manu Abraham
Quoted 7 lines · Grant Grundler
On Sun, Dec 14, 2008 at 03:15:15PM +0400, Manu Abraham wrote:
...Quoted 5 lines
Quoted 3 lines
A "GSI" (Generic Sys Interrupt?) is associated with each entry in the MSI-X table. Driver then calls
request_irq()to bind an interrupt handler to each GSI. So the driver never directly sees the "message".Oh, you mean the array of
irq_handlersin the MSI-X table should correspond to a particular message ?Quoted 7 lines
Quoted 5 lines
No. I mean each MSI-X entry has an address+message pair and each MSI-X entry is associated with the parameters passed to each call of
request_irq(). Pass in unique private data to each call ofrequest_irq()and the driver can determine which message was delivered when the ISR gets called. (ISR == Interrupt Service Routine, aka driver IRQ handler)Is it possible that even when the config space says that multiple messages are supported, you cannot enable MSI-X ?
Yes, absolutely. Have a look at
pci_msi_check_device()for starters. MSI can be disabled globally, or per-device by a quirk, the bridges above your device might not support MSI and the arch code may prevent MSI/MSI-X for some reason (ie. platform doesn't support it).There's also a check in
pci_enable_msix()to make sure the entries you pass in are valid.Quoted 7 lines
------------- with MSI-X -------------
saa716x_pci_init(0): found a NEMO reference board PCIe card
ACPI: PCI Interrupt 0000:05:00.0[A] -> GSI 19 (level, low) -> IRQ 19
PCI: Setting latency timer of device 0000:05:00.0 to 64saa716x_request_irq(0): Using MSI-X modesaa716x_enable_msix(0): MSI-X request failed
Quoted 6 lines
Quoted 4 lines
Quoted 3 lines
For starters you should print the error code returned by
pci_enable_msix()- unfortunately it returns EINVAL for many different reasons, but it will narrow it down a bit.It does return -22
If you're curious you could try this patch.
There's a missing opening brace. I did add it and added more of kernel hacking for debugging and loaded the modules after that, which did cause a complete freeze of the machine.
Currently, digging into what would have caused that freeze.
The same module without the debug (kernel hack + pci debug) doesn't cause the freeze and results in the original log that i attached some mails earlier. (No trace/log results, results in a complete freeze immediately on module (saa716x_hybrid) load)
Regards,
Manu
Quoted 16 lines
Quoted 2 lines
From
bd27f10c9acd24bb849d14b6fc373444322c7405Mon Sep 17 00:00:00 2001From: Michael Ellerman <[email protected]>
Date: Mon, 22 Dec 2008 10:21:27 +1100
Subject: [PATCH] MSI debug--- drivers/pci/msi.c | 52 ++++++++++++++++++++++++++++++++++++++++------------ 1 files changed, 40 insertions(+), 12 deletions(-)drivers/pci/msi.c +0
-0@@ -517,17 +517,26 @@ static int pci_msi_check_device(struct pci_dev* dev, int nvec, int type)struct pci_bus *bus;
drivers/pci/msi.c static int pci_msi_check_device(struct pci_dev* dev, int nvec, int type) · line 526
Quoted 10 lines
drivers/pci/msi.c +6
-1int ret; +if (!dev) { +printk (KERN_DEBUG ,"MSI: null device\n" ); +return -ENODEV ; + } +/* MSI must be globally enabled and supported by the device */ -if (!pci_msi_enable || !dev || dev->no_msi) +if (!pci_msi_enable || dev->no_msi)
Missing opening brace in here.
Quoted 90 lines
drivers/pci/msi.c +34
-11+dev_printk (KERN_DEBUG , &dev->dev,"MSI: MSI disabled\n" );return -EINVAL ; + }/* * You can't ask to have 0 or less MSIs configured. * a) it's stupid .. * b) the list manipulation code assumes nvec >= 1. */ -if (nvec <1 ) +if (nvec <1 ) { +dev_printk (KERN_DEBUG , &dev->dev,"MSI: < 1 MSI requested\n" );return -ERANGE ; + }/* Any bridge which does NOT route MSI transactions from it's * secondary bus to it's primary bus must set NO_MSI flag on @@ -535,16 +544,24 @@ static int pci_msi_check_device(struct pci_dev* dev, int nvec, int type)* We expect only arch-specific PCI host bus controller driver * or quirks for specific PCI bridges to be setting NO_MSI. */ -for (bus = dev->bus; bus; bus = bus->parent) -if (bus->bus_flags &PCI_BUS_FLAGS_NO_MSI ) +for (bus = dev->bus; bus; bus = bus->parent) { +if (bus->bus_flags &PCI_BUS_FLAGS_NO_MSI ) { +dev_printk (KERN_DEBUG , &dev->dev, +"MSI: bus check failed\n" );return -EINVAL ; + } + } ret =arch_msi_check_device (dev, nvec, type); -if (ret) +if (ret) { +dev_printk (KERN_DEBUG , &dev->dev,"MSI: arch check failed\n" );return ret; + } -if (!pci_find_capability (dev, type)) +if (!pci_find_capability (dev, type)) { +dev_printk (KERN_DEBUG , &dev->dev,"MSI: no capability found\n" );return -EINVAL ; + }return 0 ; }@@ -669,26 +686,37 @@ int pci_enable_msix(struct pci_dev* dev, struct msix_entry *entries, int nvec)int i, j;u16 control; -if (!entries) -return -EINVAL ; - status =pci_msi_check_device (dev, nvec,PCI_CAP_ID_MSIX );if (status)return status; +if (!entries) { +dev_printk (KERN_DEBUG , &dev->dev,"MSI: null entries\n" ); +return -EINVAL ; + } + pos =pci_find_capability (dev,PCI_CAP_ID_MSIX );pci_read_config_word (dev,msi_control_reg (pos), &control); nr_entries =multi_msix_capable (control); -if (nvec > nr_entries) +if (nvec > nr_entries) { +dev_printk (KERN_DEBUG , &dev->dev,"MSI: too many entries\n" );return -EINVAL ; + }/* Check for any invalid entries */ for (i =0 ; i < nvec; i++) { -if (entries[i].entry >= nr_entries) +if (entries[i].entry >= nr_entries) { +dev_printk (KERN_DEBUG , &dev->dev, +"MSI: invalid entry %d\n" , i);return -EINVAL ;/* invalid entry */ + }for (j = i +1 ; j < nvec; j++) { -if (entries[i].entry == entries[j].entry) +if (entries[i].entry == entries[j].entry) { +dev_printk (KERN_DEBUG , &dev->dev, +"MSI: duplicate entries %d, %d\n" , + i, j);return -EINVAL ;/* duplicate entry */ + } } }WARN_ON (!!dev->msix_enabled);
Quoted 37 lines · Manu Abraham
Quoted 36 lines · Michael Ellerman
Quoted 35 lines · Manu Abraham
Quoted 34 lines · Michael Ellerman
Quoted 8 lines · Manu Abraham
Quoted 7 lines · Grant Grundler
On Sun, Dec 14, 2008 at 03:15:15PM +0400, Manu Abraham wrote:
...Quoted 5 lines
Quoted 3 lines
A "GSI" (Generic Sys Interrupt?) is associated with each entry in the MSI-X table. Driver then calls
request_irq()to bind an interrupt handler to each GSI. So the driver never directly sees the "message".Oh, you mean the array of
irq_handlersin the MSI-X table should correspond to a particular message ?Quoted 7 lines
Quoted 5 lines
No. I mean each MSI-X entry has an address+message pair and each MSI-X entry is associated with the parameters passed to each call of
request_irq(). Pass in unique private data to each call ofrequest_irq()and the driver can determine which message was delivered when the ISR gets called. (ISR == Interrupt Service Routine, aka driver IRQ handler)Is it possible that even when the config space says that multiple messages are supported, you cannot enable MSI-X ?
Yes, absolutely. Have a look at
pci_msi_check_device()for starters. MSI can be disabled globally, or per-device by a quirk, the bridges above your device might not support MSI and the arch code may prevent MSI/MSI-X for some reason (ie. platform doesn't support it).There's also a check in
pci_enable_msix()to make sure the entries you pass in are valid.Quoted 7 lines
------------- with MSI-X -------------
saa716x_pci_init(0): found a NEMO reference board PCIe card
ACPI: PCI Interrupt 0000:05:00.0[A] -> GSI 19 (level, low) -> IRQ 19
PCI: Setting latency timer of device 0000:05:00.0 to 64saa716x_request_irq(0): Using MSI-X modesaa716x_enable_msix(0): MSI-X request failedFor starters you should print the error code returned by
pci_enable_msix()- unfortunately it returns EINVAL for many different reasons, but it will narrow it down a bit.
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
It does return -22
If you're curious you could try this patch.
There's a missing opening brace. I did add it and added more of kernel hacking for debugging and loaded the modules after that, which did cause a complete freeze of the machine.
Yeah sorry I didn't compile it. I don't see how it would have caused the kernel to hang though.
cheers
Signature · 8 lines
Michael Ellerman OzLabs, IBM Australia Development Lab wwweb: http://michael.ellerman.id.au phone: +61 2 6212 1183 (tie line 70 21183) We do not inherit the earth from our ancestors, we borrow it from our children. - S.M.A.R.T Person