AMD 8131 MSI quirk called too late, bus_flags not inherited ?
Subsystems (2)
Hi Greg,
While looking at the MSI detection, I noticed that the AMD 8131 quirk (quirk_amd_8131_ioapic) is defined as FINAL and thus executed after the PCI hierarchy is scanned. So it looks like the bus_flags won't be inherited at all. If there's a bridge behind the 8131, then the devices behind this bridge won't see the bus flags and thus might try to enable MSI anyway.
I tried to change the AMD 8131 quirk to HEADER so that it is executed during PCI scanning. But, I don't get its message in dmesg anymore. Any idea?
Thanks,
Brice
Quoted 11 lines · Brice Goglin
Hi Greg,
While looking at the MSI detection, I noticed that the AMD 8131 quirk (
quirk_amd_8131_ioapic) is defined as FINAL and thus executed after the PCI hierarchy is scanned. So it looks like thebus_flagswon't be inherited at all. If there's a bridge behind the 8131, then the devices behind this bridge won't see the bus flags and thus might try to enable MSI anyway.
I tried to change the AMD 8131 quirk to HEADER so that it is executed during PCI scanning. But, I don't get its message in dmesg anymore. Any idea?
Michael is the one who added that change, perhaps he can explain how he tested it?
thanks,
greg k-h
Quoting r. Greg KH <[email protected]>:
Quoted 15 lines
Subject: Re: AMD 8131 MSI quirk called too late,
bus_flagsnot inherited ?Quoted 11 lines · Brice Goglin
Hi Greg,
While looking at the MSI detection, I noticed that the AMD 8131 quirk (
quirk_amd_8131_ioapic) is defined as FINAL and thus executed after the PCI hierarchy is scanned. So it looks like thebus_flagswon't be inherited at all. If there's a bridge behind the 8131, then the devices behind this bridge won't see the bus flags and thus might try to enable MSI anyway.
I tried to change the AMD 8131 quirk to HEADER so that it is executed during PCI scanning. But, I don't get its message in dmesg anymore. Any idea?
Quoted 2 lines
Michael is the one who added that change, perhaps he can explain how he tested it?
Well, I just re-tested with 2.6.17-rc4 and it does not seem to work. No idea what did I do wrong when testing this patch before posting it :(. Oops. I'm sorry.
Given that its late in -rc series, that its a clear regression from 2.6.16, that disabling MSI is always safe, and that the patch was intended to enable MSI on Tyan motherboard which Roland used to have but doesn't anymore and which no one else seems to be bothered to test on either - it seems the best thing is to just revert the patch, and go back to using a global flag for now.
I'll try to revisit this for 2.6.18 if I can get my hands on such a system. Greg, what do you think?
It's 6e325a62a0a228cd0222783802b53cce04551776 in Linus' tree:
http://www.kernel.org/git/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=6e325a62a0a228cd0222783802b53cce04551776
Here's a minimal patch against 2.6.17-rc4 - I tested it by loading mthca with msi_x=1.
---
Revert MSI quirk change in 6e325a62a0a228cd0222783802b53cce04551776 - it breaks regular AMD-8131 systems, and no one seems to care enough to test on Tyan systems where it's supposed to improve performance.
drivers/pci/quirks.c +2 -5
@@ -575,11 +575,8 @@ static void __init quirk_amd_8131_ioapic {unsigned char revid, tmp; -if (dev->subordinate) { -printk (KERN_WARNING "PCI: MSI quirk detected. " -"PCI_BUS_FLAGS_NO_MSI set for subordinate bus.\n" ); - dev->subordinate->bus_flags |=PCI_BUS_FLAGS_NO_MSI ; - } + pci_msi_quirk =1 ; +printk (KERN_WARNING "PCI: MSI quirk detected. pci_msi_quirk set.\n" );if (nr_ioapics ==0 )return ;
Signature · 1 line
MST
Quoted 22 lines · Michael S. Tsirkin
Quoting r. Greg KH <[email protected]>:
Quoted 14 lines
Subject: Re: AMD 8131 MSI quirk called too late,
bus_flagsnot inherited ?Quoted 11 lines · Brice Goglin
Hi Greg,
While looking at the MSI detection, I noticed that the AMD 8131 quirk (
quirk_amd_8131_ioapic) is defined as FINAL and thus executed after the PCI hierarchy is scanned. So it looks like thebus_flagswon't be inherited at all. If there's a bridge behind the 8131, then the devices behind this bridge won't see the bus flags and thus might try to enable MSI anyway.
I tried to change the AMD 8131 quirk to HEADER so that it is executed during PCI scanning. But, I don't get its message in dmesg anymore. Any idea?Quoted 2 lines
Michael is the one who added that change, perhaps he can explain how he tested it?
Well, I just re-tested with 2.6.17-rc4 and it does not seem to work. No idea what did I do wrong when testing this patch before posting it :(. Oops. I'm sorry.
Quoted 5 lines
Given that its late in -rc series, that its a clear regression from 2.6.16, that disabling MSI is always safe, and that the patch was intended to enable MSI on Tyan motherboard which Roland used to have but doesn't anymore and which no one else seems to be bothered to test on either - it seems the best thing is to just revert the patch, and go back to using a global flag for now.
I would say that it works for devices that are directly behind the amd8131. But it won't disable MSI for devices that are behind another bridge behind the 8131. It is great for us since we have lots of Tyan motherboards without any device behind the 8131 :) I don't know how many machines have a bridge behind a 8131, this bug might not hit a lot of people.
At least, with 2.6.17-rc, we can get MSI on devices that have nothing to do with the 8131. I consider this a very good progress. Yes, we try to enable MSI on devices behing a bridge behind the 8131. But, if few people have such a PCI hierarchy, I am not sure this regression is important. What happens if you actually enable MSI there ? It fails ? Or it crashes ? Is it worth than having no MSI at all on any devices on the machine ?
Anyway, we are working on merging our MSI detection code based on HT capabilities. And we think we might have to add a couple other quirks where it will be very important there that the bus flags are inherited. So I tried to dig more in what we could do to inherit flags properly and here's what I found:
The amd8131 MSI quirk cannot be EARLY or HEADER because it would be called before dev->subordinate has been set. So PCI_BUS_FLAGS_NO_MSI could not be set in the bus flags at this time.
So we need a quirk that is called while the PCI hierarchy is scanned (ie, before FINAL or ENABLE) and after the pci child bus of the bridge is created (ie, after EARLY and HEADER).
The attached patch adds a new type of fixup that is called right after the child bus is created (ie, when dev->subordinate exists). And the patch moves the amd8131 MSI quirk to use this new quirk type. I chose the name "bridge header" but I guess somebody will come with a better name (I am not really familiar with all the PCI terminology).
Brice
drivers/pci/quirks.c +17 -6
@@ -575,12 +575,6 @@ {unsigned char revid, tmp; -if (dev->subordinate) { -printk (KERN_WARNING "PCI: MSI quirk detected. " -"PCI_BUS_FLAGS_NO_MSI set for subordinate bus.\n" ); - dev->subordinate->bus_flags |=PCI_BUS_FLAGS_NO_MSI ; - } -if (nr_ioapics ==0 )return ;
@@ -594,6 +588,16 @@ }DECLARE_PCI_FIXUP_FINAL (PCI_VENDOR_ID_AMD ,PCI_DEVICE_ID_AMD_8131_BRIDGE , quirk_amd_8131_ioapic); +static void __init quirk_amd_8131_nomsi (struct pci_dev *dev) +{ +if (dev->subordinate) { +printk (KERN_WARNING "PCI: MSI quirk detected. " +"PCI_BUS_FLAGS_NO_MSI set for subordinate bus.\n" ); + dev->subordinate->bus_flags |=PCI_BUS_FLAGS_NO_MSI ; + } +} +DECLARE_PCI_FIXUP_BRIDGE_HEADER (PCI_VENDOR_ID_AMD ,PCI_DEVICE_ID_AMD_8131_BRIDGE , quirk_amd_8131_nomsi); +static void __init quirk_svw_msi (struct pci_dev *dev) { pci_msi_quirk =1 ;
@@ -1407,6 +1411,8 @@extern struct pci_fixup __end_pci_fixups_early[];extern struct pci_fixup __start_pci_fixups_header[];extern struct pci_fixup __end_pci_fixups_header[]; +extern struct pci_fixup __start_pci_fixups_bridge_header[]; +extern struct pci_fixup __end_pci_fixups_bridge_header[];extern struct pci_fixup __start_pci_fixups_final[];extern struct pci_fixup __end_pci_fixups_final[];extern struct pci_fixup __start_pci_fixups_enable[];
@@ -1428,6 +1434,11 @@ end = __end_pci_fixups_header;break ; +case pci_fixup_bridge_header: + start = __start_pci_fixups_bridge_header; + end = __end_pci_fixups_bridge_header; +break ; +case pci_fixup_final: start = __start_pci_fixups_final; end = __end_pci_fixups_final;
drivers/pci/probe.c +1 -0
@@ -367,6 +367,7 @@ child->resource[i]->name = child->name; } bridge->subordinate = child; +pci_fixup_device (pci_fixup_bridge_header, bridge);return child; }
include/asm-generic/vmlinux.lds.h +3 -0
@@ -29,6 +29,9 @@ VMLINUX_SYMBOL(__start_pci_fixups_header) = .; \ *(.pci_fixup_header) \ VMLINUX_SYMBOL(__end_pci_fixups_header) = .; \ + VMLINUX_SYMBOL(__start_pci_fixups_bridge_header) = .; \ + *(.pci_fixup_bridge_header) \ + VMLINUX_SYMBOL(__end_pci_fixups_bridge_header) = .; \ VMLINUX_SYMBOL(__start_pci_fixups_final) = .; \ *(.pci_fixup_final) \ VMLINUX_SYMBOL(__end_pci_fixups_final) = .; \
include/linux/pci.h +4 -0
@@ -750,6 +750,7 @@enum pci_fixup_pass { pci_fixup_early,/* Before probing BARs */ pci_fixup_header,/* After reading configuration header */ + pci_fixup_bridge_header,/* After setting up the bridge subordinate */ pci_fixup_final,/* Final phase of device fixups */ pci_fixup_enable,/* pci_enable_device() time */ };
@@ -764,6 +765,9 @@#define DECLARE_PCI_FIXUP_HEADER (vendor, device, hook) \ DECLARE_PCI_FIXUP_SECTION(.pci_fixup_header, \ vendor##device##hook, vendor, device, hook) +#define DECLARE_PCI_FIXUP_BRIDGE_HEADER (vendor, device, hook) \ + DECLARE_PCI_FIXUP_SECTION(.pci_fixup_bridge_header, \ + vendor##device##hook, vendor, device, hook)#define DECLARE_PCI_FIXUP_FINAL (vendor, device, hook) \ DECLARE_PCI_FIXUP_SECTION(.pci_fixup_final, \ vendor##device##hook, vendor, device, hook)
Quoting r. Brice Goglin <[email protected]>:
Quoted 31 lines
Subject: Re: AMD 8131 MSI quirk called too late,
bus_flagsnot inherited ?Quoted 22 lines · Michael S. Tsirkin
Quoting r. Greg KH <[email protected]>:
Quoted 14 lines
Subject: Re: AMD 8131 MSI quirk called too late,
bus_flagsnot inherited ?Quoted 11 lines · Brice Goglin
Hi Greg,
While looking at the MSI detection, I noticed that the AMD 8131 quirk (
quirk_amd_8131_ioapic) is defined as FINAL and thus executed after the PCI hierarchy is scanned. So it looks like thebus_flagswon't be inherited at all. If there's a bridge behind the 8131, then the devices behind this bridge won't see the bus flags and thus might try to enable MSI anyway.
I tried to change the AMD 8131 quirk to HEADER so that it is executed during PCI scanning. But, I don't get its message in dmesg anymore. Any idea?Quoted 2 lines
Michael is the one who added that change, perhaps he can explain how he tested it?
Well, I just re-tested with 2.6.17-rc4 and it does not seem to work. No idea what did I do wrong when testing this patch before posting it :(. Oops. I'm sorry.
Quoted 5 lines
Given that its late in -rc series, that its a clear regression from 2.6.16, that disabling MSI is always safe, and that the patch was intended to enable MSI on Tyan motherboard which Roland used to have but doesn't anymore and which no one else seems to be bothered to test on either - it seems the best thing is to just revert the patch, and go back to using a global flag for now.
Quoted 4 lines
I would say that it works for devices that are directly behind the amd8131. But it won't disable MSI for devices that are behind another bridge behind the 8131. It is great for us since we have lots of Tyan motherboards without any device behind the 8131 :)
MSI is an optional feature so things are supposed to work even without MSI - are you getting that great a benefit from MSI?
Quoted 3 lines
I don't know how many
machines have a bridge behind a 8131, this bug might not hit a lot of people.
All mellanox PCI-X devices have a bridge inside them, so ...
Quoted 6 lines
At least, with 2.6.17-rc, we can get MSI on devices that have nothing to do with the 8131. I consider this a very good progress. Yes, we try to enable MSI on devices behing a bridge behind the 8131. But, if few people have such a PCI hierarchy, I am not sure this regression is important. What happens if you actually enable MSI there ? It fails ? Or it crashes ?
It fails to generate interrupts.
Quoted 2 lines
Is it worth than having no MSI at all on any devices on the machine ?
I think it is.
Quoted 19 lines
Anyway, we are working on merging our MSI detection code based on HT capabilities. And we think we might have to add a couple other quirks where it will be very important there that the bus flags are inherited. So I tried to dig more in what we could do to inherit flags properly and here's what I found:
The amd8131 MSI quirk cannot be EARLY or HEADER because it would be called before
dev->subordinatehas been set. SoPCI_BUS_FLAGS_NO_MSIcould not be set in the bus flags at this time.
So we need a quirk that is called while the PCI hierarchy is scanned (ie, before FINAL or ENABLE) and after the pci child bus of the bridge is created (ie, after EARLY and HEADER).The attached patch adds a new type of fixup that is called right after the child bus is created (ie, when
dev->subordinateexists). And the patch moves the amd8131 MSI quirk to use this new quirk type. I chose the name "bridge header" but I guess somebody will come with a better name (I am not really familiar with all the PCI terminology).
Quoted 3 lines
Signed-off-by Brice Goglin <[email protected]>Brice
Doesn't seem to work for me:
ib_mthca: Initializing 0000:04:00.0
GSI 18 sharing vector 0xB9 and IRQ 18
ACPI: PCI Interrupt 0000:04:00.0[A] -> GSI 29 (level, low) -> IRQ 185 ib_mthca 0000:04:00.0: NOP command failed to generate interrupt (IRQ 217), aborting.ib_mthca 0000:04:00.0: Try again with MSI/MSI-X disabled.
ACPI: PCI interrupt for device 0000:04:00.0 disabledib_mthca: probe of 0000:04:00.0 failed with error -16
Signature · 1 line
MST
Quoted 3 lines · Michael S. Tsirkin
MSI is an optional feature so things are supposed to work even without MSI - are you getting that great a benefit from MSI?
Not great, I would say small.
Quoted 2 lines
All mellanox PCI-X devices have a bridge inside them, so ...
Ok so you really need something for 2.6.17. What about the attached patch to fix the fact that bus flags are not inherited ?
Quoted 11 lines
Doesn't seem to work for me:
ib_mthca: Initializing 0000:04:00.0
GSI 18 sharing vector0xB9and IRQ 18
ACPI: PCI Interrupt 0000:04:00.0[A] -> GSI 29 (level, low) -> IRQ 185ib_mthca0000:04:00.0: NOP command failed to generate interrupt (IRQ 217), aborting.ib_mthca0000:04:00.0: Try again with MSI/MSI-X disabled.
ACPI: PCI interrupt for device 0000:04:00.0 disabledib_mthca: probe of 0000:04:00.0 failed with error -16
Ok. Do you at least see the quirk message ?
Thanks,
Brice
drivers/pci/msi.c +4 -2
@@ -916,6 +916,7 @@**/ int pci_enable_msi (struct pci_dev * dev) { +struct pci_bus *bus;int pos, temp, status = -EINVAL ;u16 control;
@@ -925,8 +926,9 @@if (dev->no_msi)return status; -if (dev->bus->bus_flags &PCI_BUS_FLAGS_NO_MSI ) -return -EINVAL ; +for (bus = dev->bus; bus; bus = bus->parent) +if (bus->bus_flags &PCI_BUS_FLAGS_NO_MSI ) +return -EINVAL ; temp = dev->irq;
Quoting r. Brice Goglin <[email protected]>:
Quoted 8 lines
Subject: Re: AMD 8131 MSI quirk called too late,
bus_flagsnot inherited ?Quoted 3 lines · Michael S. Tsirkin
MSI is an optional feature so things are supposed to work even without MSI - are you getting that great a benefit from MSI?
Not great, I would say small.
Me too.
Quoted 7 lines
Quoted 2 lines
All mellanox PCI-X devices have a bridge inside them, so ...
Ok so you really need something for 2.6.17. What about the attached patch to fix the fact that bus flags are not inherited ?
Signed-off-by Brice Goglin <[email protected]>
Seems to work for MSI but not for MSI-X. With MSI-X, I still see:
ib_mthca 0000:04:00.0: NOP command failed to generate interrupt (IRQ 217), aborting.ib_mthca 0000:04:00.0: Try again with MSI/MSI-X disabled.
Quoted 12 lines
Quoted 11 lines
Doesn't seem to work for me:
ib_mthca: Initializing 0000:04:00.0
GSI 18 sharing vector0xB9and IRQ 18
ACPI: PCI Interrupt 0000:04:00.0[A] -> GSI 29 (level, low) -> IRQ 185ib_mthca0000:04:00.0: NOP command failed to generate interrupt (IRQ 217), aborting.ib_mthca0000:04:00.0: Try again with MSI/MSI-X disabled.
ACPI: PCI interrupt for device 0000:04:00.0 disabledib_mthca: probe of 0000:04:00.0 failed with error -16
Quoted 1 line
Ok. Do you at least see the quirk message ?
Yes.
Quoted 20 lines
Thanks,
BriceIndex:
linux-mm/drivers/pci/msi.c
===================================================================drivers/pci/msi.c +1
-0@@ -916,6 +916,7 @@**/ int pci_enable_msi (struct pci_dev * dev) { +struct pci_bus *bus;int pos, temp, status = -EINVAL ;u16 control;@@ -925,8 +926,9 @@if (dev->no_msi)
drivers/pci/msi.c line 934
Quoted 10 lines
drivers/pci/msi.c +3
-2return status; -if (dev->bus->bus_flags &PCI_BUS_FLAGS_NO_MSI ) -return -EINVAL ; +for (bus = dev->bus; bus; bus = bus->parent) +if (bus->bus_flags &PCI_BUS_FLAGS_NO_MSI ) +return -EINVAL ; temp = dev->irq;
It seems we must add this loop to pci_enable_msix as well.
Signature · 1 line
MST
Quoted 10 lines · Michael S. Tsirkin
Quoted 10 lines
(quoted diff) +3
-2@@ -925,8 +926,9 @@ if (dev->no_msi) return status; - if (dev->bus->bus_flags & PCI_BUS_FLAGS_NO_MSI) - return -EINVAL; + for (bus = dev->bus; bus; bus = bus->parent) + if (bus->bus_flags & PCI_BUS_FLAGS_NO_MSI) + return -EINVAL;
Quoted 5 lines
Review line 934
Quoted 2 lines
(quoted diff) +0
-0temp = dev->irq;It seems we must add this loop to
pci_enable_msixas well.
Right, thanks. Greg, what do you think of putting the attached patch in 2.6.17 ?
By the way, do we need to check dev->no_msi in pci_enable_msix() too ?
For 2.6.18, I don't know what's the best. We could drop the fact that bus flags should be inherited and keep looking at parent busses. It might be good to add a pci_check_flag_in_parent_busses(dev, flag) to provide a generic way to do my for loop.
thanks,
Brice
drivers/pci/msi.c +9 -2
@@ -916,6 +916,7 @@**/ int pci_enable_msi (struct pci_dev * dev) { +struct pci_bus *bus;int pos, temp, status = -EINVAL ;u16 control;
@@ -925,8 +926,9 @@if (dev->no_msi)return status; -if (dev->bus->bus_flags &PCI_BUS_FLAGS_NO_MSI ) -return -EINVAL ; +for (bus = dev->bus; bus; bus = bus->parent) +if (bus->bus_flags &PCI_BUS_FLAGS_NO_MSI ) +return -EINVAL ; temp = dev->irq;
@@ -1162,6 +1164,7 @@**/ int pci_enable_msix (struct pci_dev * dev,struct msix_entry *entries,int nvec) { +struct pci_bus *bus;int status, pos, nr_entries, free_vectors;int i, j, temp;u16 control;
@@ -1170,6 +1173,10 @@if (!pci_msi_enable || !dev || !entries)return -EINVAL ; +for (bus = dev->bus; bus; bus = bus->parent) +if (bus->bus_flags &PCI_BUS_FLAGS_NO_MSI ) +return -EINVAL ; + status =msi_init ();if (status <0 )return status;
Quoting r. Brice Goglin <[email protected]>:
Quoted 9 lines
Subject: Re: AMD 8131 MSI quirk called too late,
bus_flagsnot inherited ?Quoted 3 lines · Michael S. Tsirkin
MSI is an optional feature so things are supposed to work even without MSI - are you getting that great a benefit from MSI?
Not great, I would say small.
Quoted 7 lines
Quoted 2 lines
All mellanox PCI-X devices have a bridge inside them, so ...
Ok so you really need something for 2.6.17. What about the attached patch to fix the fact that bus flags are not inherited ?
Signed-off-by Brice Goglin <[email protected]>
The following applies this quirk for MSI-X as well.
drivers/pci/msi.c +9 -2
@@ -865,6 +865,7 @@ static int msix_capability_init(struct p**/ int pci_enable_msi (struct pci_dev * dev) { +struct pci_bus *bus;int pos, temp, status = -EINVAL ;u16 control;
@@ -874,8 +875,9 @@ int pci_enable_msi(struct pci_dev* dev)if (dev->no_msi)return status; -if (dev->bus->bus_flags &PCI_BUS_FLAGS_NO_MSI ) -return -EINVAL ; +for (bus = dev->bus; bus; bus = bus->parent) +if (bus->bus_flags &PCI_BUS_FLAGS_NO_MSI ) +return -EINVAL ; temp = dev->irq;
@@ -1108,6 +1110,7 @@ static int reroute_msix_table(int head,**/ int pci_enable_msix (struct pci_dev * dev,struct msix_entry *entries,int nvec) { +struct pci_bus *bus;int status, pos, nr_entries, free_vectors;int i, j, temp;u16 control;
@@ -1116,6 +1119,10 @@ int pci_enable_msix(struct pci_dev* dev,if (!pci_msi_enable || !dev || !entries)return -EINVAL ; +for (bus = dev->bus; bus; bus = bus->parent) +if (bus->bus_flags &PCI_BUS_FLAGS_NO_MSI ) +return -EINVAL ; + status =msi_init ();if (status <0 )return status;
Signature · 1 line
MST
Quoted 11 lines · Brice Goglin
Quoted 10 lines · Michael S. Tsirkin
Quoted 10 lines
drivers/pci/msi.c +3
-2@@ -925,8 +926,9 @@if (dev->no_msi)return status; -if (dev->bus->bus_flags &PCI_BUS_FLAGS_NO_MSI ) -return -EINVAL ; +for (bus = dev->bus; bus; bus = bus->parent) +if (bus->bus_flags &PCI_BUS_FLAGS_NO_MSI ) +return -EINVAL ;
Quoted 8 lines
Quoted 5 lines
drivers/pci/msi.c line 934
Quoted 2 lines
drivers/pci/msi.c +0
-0temp = dev->irq;It seems we must add this loop to
pci_enable_msixas well.Right, thanks. Greg, what do you think of putting the attached patch in 2.6.17 ?
Ok, does everyone agree that this patch fixes the issues for them? I've had a few other private emails saying that the current code doesn't work properly and hadn't been able to determine what was happening. Thanks for these patches.
Quoted 1 line
By the way, do we need to check
dev->no_msiinpci_enable_msix()too ?
Yes, good catch, care to respin the patch and give it a good changelog entry?
thanks,
greg k-h
Quoted 10 lines · Greg KH
Quoted 3 lines · Brice Goglin
Right, thanks. Greg, what do you think of putting the attached patch in 2.6.17 ?
Ok, does everyone agree that this patch fixes the issues for them? I've had a few other private emails saying that the current code doesn't work properly and hadn't been able to determine what was happening. Thanks for these patches.
Quoted 4 lines
Quoted 1 line
By the way, do we need to check
dev->no_msiinpci_enable_msix()too ?Yes, good catch, care to respin the patch and give it a good changelog entry?
Here you are:
The PCI_BUS_FLAGS_NO_MSI bus flags does not appear do be inherited correctly from the amd8131 MSI quirk to its parent busses. It makes devices behind a bridge behind amd8131 try to enable MSI while the amd8131 does not support it.
We fix this by looking at flags of all parent busses in pci_enable_msi() and pci_enable_msix().
By the way, also add the missing dev->no_msi check in pci_enable_msix()
drivers/pci/msi.c +12 -2
@@ -916,6 +916,7 @@**/ int pci_enable_msi (struct pci_dev * dev) { +struct pci_bus *bus;int pos, temp, status = -EINVAL ;u16 control;
@@ -925,8 +926,9 @@if (dev->no_msi)return status; -if (dev->bus->bus_flags &PCI_BUS_FLAGS_NO_MSI ) -return -EINVAL ; +for (bus = dev->bus; bus; bus = bus->parent) +if (bus->bus_flags &PCI_BUS_FLAGS_NO_MSI ) +return -EINVAL ; temp = dev->irq;
@@ -1162,6 +1164,7 @@**/ int pci_enable_msix (struct pci_dev * dev,struct msix_entry *entries,int nvec) { +struct pci_bus *bus;int status, pos, nr_entries, free_vectors;int i, j, temp;u16 control;
@@ -1170,6 +1173,13 @@if (!pci_msi_enable || !dev || !entries)return -EINVAL ; +if (dev->no_msi) +return -EINVAL ; + +for (bus = dev->bus; bus; bus = bus->parent) +if (bus->bus_flags &PCI_BUS_FLAGS_NO_MSI ) +return -EINVAL ; + status =msi_init ();if (status <0 )return status;
Quoting r. Greg KH <[email protected]>:
Quoted 4 lines
Quoted 2 lines
Right, thanks. Greg, what do you think of putting the attached patch in 2.6.17 ?
Ok, does everyone agree that this patch fixes the issues for them?
Worked here on a PCI-X AMD-8131 based system.
----
Offtopic, something I wanted to bring up with respect to MSI, but never had the time to debug:
If I do
pci_enable_msix, pci_disable_msix
then later
pci_enable_msi
on the same device fails with the following message:
PCI: 0000:08:00.0: Can't enable MSI. Device already has MSI-X vectors assigned
This is not something new - has been happening since forever. Looks like not all MSI-X vectors get properly unassigned by pci_disable_msix.
One way to test this is by loading mthca driver with msi_x=1, unloading and loading with msi=1.
Someone has any idea what's wrong?
Signature · 1 line
MST
Quoted 10 lines · Michael S. Tsirkin
Offtopic, something I wanted to bring up with respect to MSI, but never had the time to debug:
If I do
pci_enable_msix,pci_disable_msixthen later
Quoted 8 lines
pci_enable_msion the same device fails with the following message:
PCI: 0000:08:00.0: Can't enable MSI. Device already has MSI-X vectors assignedThis is not something new - has been happening since forever. Looks like not all MSI-X vectors get properly unassigned by
pci_disable_msix.
Yes, this has been reported by others too. I've been looking at MSI code recently to fix an unrelated problem, and noticed that the code has policies about vector reservation that prevent what you're trying to do. I'm planning to clean up the MSI code shortly (patches out hopefully by next week), and will remove such policies since many people are trying to do this.
thanks,
Rajesh