Message Signalled Interrupt support?

Has anybody done any work, or put any thought, into MSI support?

Would things massively break if I set up MSI manually in the driver?

I heard rumblings on lkml that Intel has done some work internally w/ MSI support in Linux, but that doesn't help me much without further details ;-)

Jeff

Work -- no. Thought? A little. Seems to me that MSIs need to be treated as a third form of interrupts (level/edge/message). The address that the MSI will write to is clearly architecture dependent (may even be irq-controller-dependent, depending on your architecture). request_irq() is an insufficient function to deal with this -- request_msi() may be needed instead. It'll need to return an address to pass to the card. (We need a mechanism to decide whether it's a 32-bit or 64-bit address).

Oh, and don't make this too PCI-specific -- native PARISC interrupts are MSI and you can see how handled it in arch/parisc/kernel/irq.c.

Might do, might not.

"It's not Hollywood.  War is real, war is primarily not about defeat or
victory, it is about death.  I've seen thousands and thousands of dead bodies.
Do you think I want to have an academic debate on this subject?" -- Robert Fisk

I've also done some thought for PPC440xx's PCI MSI support. It isn't strictly necessary to have a new request_msi() if the kernel "does the right thing". request_irq() already hooks using an interrupt value that is virtual on many platforms. In that case, the PCI subsystem would only need to provide an interface to provide the architecture/platform specific inbound MSI location. The PCI subsystem would then find all MSI capable PCI devices, and assign the appropriate number of unique messages and inbound MSI address to each device via the speced PCI MSI interface. The PCI subsystem would also be responsible for maintaining a correspondence between virtual Linux interrupt values and MSI values.

Software specific to the PCI MSI capable "Northbridge", will then route general MSI interrupt events to some PCI subsystem helper functions to verify which MSI has occurred and thus which Linux virtual interrupt.

Perhaps request_irq() just needs to be explicitly abstracted if an unsigned int is not sufficient for the entire message space or if we want messages unique only on a per-space basis i.e. PCI MSIs can be dups of RapidIO MSIs, etc.

FWIW, another interconnect that natively uses MSI is RapidIO. It's all in-band doorbell messages, no out-of-band discrete interrupts like PCI.

Regards,

Matt Porter
[email protected]

Yes, but ideally this kludge would go away...

That sounds like a lot of overhead. In particular it means we keep converting to and from `virtual IRQs'. I would hope the MSI work would allow us to tie in at a lower level than virtual interrupts. I was thinking an interface would look something like:

void *request_msi(struct device *dev,
irqreturn_t (*handler)(int, void *, struct pt_regs *), unsigned long irqflags,
void *dev_id)

You need a struct device to figure out which interrupt controller it needs.

"It's not Hollywood.  War is real, war is primarily not about defeat or
victory, it is about death.  I've seen thousands and thousands of dead bodies.
Do you think I want to have an academic debate on this subject?" -- Robert Fisk

Well, with respect to MSI, yes it could. Interrupt values are still a Linux fabrication regardless on many platforms with cascaded PICs.

Yes, it's a bit. Just pointing out that one *could* do it that way, not really that we *should* do it that way. I would personally prefer to see native msi support. However, most of the functionality I mentioned are things the PCI subsystem needs to do specific to PCI MSI anyway.

request_msi() needs an additional parameter to specify which MSI it is hooking. A device can implement many messages in order to clarify which one of many events on a device has occurred. It may be desired to hook a separate handler for each of those to avoid another read of a status register.

Regards,

Matt Porter
[email protected]

Assuming that platform specific PCI setup does reasonable real to virtual IRQ mapping, request_msi() is not needed. We can use pdev->irq as "base" vector and MSI message number as offset. Alpha works this way, BTW.

I think of something like this:
/**
* pci_using_msi - is this PCI device configured to use MSI?
* @dev: PCI device structure of device being queried
*
* Tells whether or not a PCI device is configured to use Message Signaled
* Interrupts. Returns number of allocated MSI messages, else 0. */
int
pci_using_msi(struct pci_dev *dev)

{
	int msi = pci_find_capability(dev, PCI_CAP_ID_MSI);
	u8 msgctl;

	if (!msi || !dev->irq)
		return 0;

	pci_read_config_byte(dev, msi + PCI_MSI_FLAGS, &msgctl);

	if (!(msgctl & PCI_MSI_FLAGS_ENABLE))
		return 0;
	
	return 1 << ((msgctl >> 4) & 7); /* # of messages allocated */
}

So that MSI-aware driver can do

	nummsgs = pci_using_msi(dev);
	if (!nummsgs)
		goto no_msi;
	for (msg = 0; msg < nummsgs; msg++) {
		...
		request_irq(dev->irq + msg, ...);
	}

Ivan.

From: Matthew Wilcox <[email protected]>
Date: Mon, 12 May 2003 17:53:31 +0100

Work -- no. Thought? A little. Seems to me that MSIs need to be treated as a third form of interrupts (level/edge/message).

The fact that Alpha already supports them pretty much transparently suggests that the thing to do might very well be "nothing" :-)

To be honest, MSIs are very similar to how interrupts work on sparc64, in that each device generates a unique interrupt cookie. The only different is the size of this cookie, MSIs are larger than sparc64's.

I think that MSI should be made quite explicit. If the device and platform both support MSI, then the driver should be able to ask for MSI, but kernel must be able to deny it. All devices supporting MSI must support also pin based interrupt delivery. As MSI can be costy (one interrupt vector per device for instance), it could be that a device has to work in pin based mode instead.

--Mika