New: PCI I/O range not assigned under newer kernels
A message that is not in the archive (sent off-list or not yet received): [email protected]/
Bjorn Helgaas
maintainerQuoted 13 lines · On Thu, Jan 19, 2012 at 4:22 AM, <[email protected]> wrote:
https://bugzilla.kernel.org/show_bug.cgi?id=42606
Observed a problem with a PCI Express card in a certain server PC with 8 PCI Express slots on the mainboard. There are no other PCI slots than the PCIe ones, and all slots are empty except the one with the card under test, which requires a legacy I/O address range for base address 0.
In some of the slots the card works properly, but in some other slots on the same
machine the card doesn't work since it is not accessible by the associated kernel
driver. In case it doesn't work lspci -v says for the I/O base address 0:
Quoted 5 lines
Region 0: I/O ports at <ignored>
Found out that this problem did *not* occur with kernels 2.6.32.6 and earlier, but *does* occur with kernels 2.6.36.1 and later, up to at least 3.4.1, on the same hardware.
The 2.6.32.6 dmesg is truncated and doesn't include the parts of interest (the early part that contains PCI enumeration). But this looks related to "pci=use_crs" being turned on by default, which I think happened in 2.6.34. That means that booting with "pci=nocrs" might be a workaround while we figure out a better solution.
Here's the 3.1.4 dmesg:
ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-7f]) pci_root PNP0A08:00: host bridge window [io 0xf000-0xffff]
...
ACPI: PCI Root Bridge [BR50] (domain 0000 [bus 80-fb]) pci_root PNP0A08:01: host bridge window [io 0xf000-0xffff] pci 0000:80:07.0: PCI bridge to [bus 85-85]
pci 0000:80:07.0: bridge window [io 0xf000-0xffff]
pci 0000:85:00.0: reg 10: [io 0xfc00-0xfcff]
pci 0000:80:07.0: address space collision: [io 0xf000-0xffff]
conflicts with PCI Bus 0000:00 [io 0xf000-0xffff]
pci 0000:85:00.0: no compatible bridge window for [io 0xfc00-0xfcff]
The BIOS is telling us that [io 0xf000-0xffff] is routed to both root buses (00 and 80), which Linux assumes is not possible. In the posted dmesg, it looks like your device is at 85:00.0, and it doesn't work. This is because we think the [io 0xf000-0xffff] area is already consumed by host bridge PCI0 and hence not available for bridge BR50.
What are the device addresses that work and those that don't work? My guess is the working slots might be under PCI0 (buses 00-7f) and the broken ones under BR50 (buses 80-fb). Can you collect a dmesg log with a working slot, too?
Can you boot with Windows with the card in the same slot (one that doesn't work under Linux) and use AIDA64 (free trial version at
http://www.aida64.com/downloads) to collect a log? This should show where Windows puts the device.
A message that is not in the archive (sent off-list or not yet received): CAErSpo7Gpd4OwbRPRd8ePdKDRQpZKT6E-UyEVL92as6C1Tqj7Q@mail.gmail.com
A message that is not in the archive (sent off-list or not yet received): [email protected]
Bjorn Helgaas
maintainerQuoted 17 lines · Martin Burnicki
Bjorn Helgaas wrote:
[...]Quoted 14 lines
Here's the 3.1.4 dmesg:
ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-7f])
pci_rootPNP0A08:00: host bridge window [io 0xf000-0xffff]
...
ACPI: PCI Root Bridge [BR50] (domain 0000 [bus 80-fb])pci_rootPNP0A08:01: host bridge window [io 0xf000-0xffff] pci 0000:80:07.0: PCI bridge to [bus 85-85]
pci 0000:80:07.0: bridge window [io 0xf000-0xffff]
pci 0000:85:00.0: reg 10: [io 0xfc00-0xfcff]
pci 0000:80:07.0: address space collision: [io 0xf000-0xffff] conflicts with PCI Bus 0000:00 [io 0xf000-0xffff]
pci 0000:85:00.0: no compatible bridge window for [io 0xfc00-0xfcff]
Quoted 4 lines
Quoted 2 lines
The BIOS is telling us that [io 0xf000-0xffff] is routed to both root buses (00 and 80), which Linux assumes is not possible.
Hm, so this sounds like a BIOS bug?
That was my initial assumption, but appears not to be the case; see below.
Quoted 3 lines
However, the strange thing here is that it actually *does* work if an older kernel is booted which doesn't do these checks, or even if a newer kernel is booted with "pci=nocrs".
The older kernels and newer kernels with "pci=nocrs" ignore the _CRS information (the "host bridge window" lines in dmesg), so they basically just use what the BIOS configured, which works in this case.
Quoted 8 lines
Quoted 4 lines
What are the device addresses that work and those that don't work? My guess is the working slots might be under PCI0 (buses 00-7f) and the broken ones under BR50 (buses 80-fb). Can you collect a dmesg log with a working slot, too?
The card does *not* work in the slots assigned to buses 83, 84, 85, and 86.
The card *does* work in the slots assigned to buses 02, 03, and 04.
Thanks a lot. That confirms my hypothesis that the card works under PCI0 but not under BR50.
Quoted 7 lines
Quoted 4 lines
Can you boot with Windows with the card in the same slot (one that doesn't work under Linux) and use AIDA64 (free trial version at
http://www.aida64.com/downloads) to collect a log? This should show where Windows puts the device.Done. See the attached file aida64-report.htm. The report has been generated with the default setting for a hardware report in Aida64.
Windows shows the [io 0xf000-0xffff] range as being available under both host bridges, and the fact that those ports actually work under both bridges means they must actually be routed both places.
It may be that Linux is being overly strict in rejecting that overlap.
I'm not sure what would break if we changed that. Linux has a very
hierarchical idea of resources and doesn't have a mechanism for managing duplicates like that.
I'll add this info to the bugzilla
(https://bugzilla.kernel.org/show_bug.cgi?id=42606) if you haven't already.
Bjorn
Bjorn Helgaas
maintainerQuoted 3 lines · Bjorn Helgaas
Windows shows the [io 0xf000-0xffff] range as being available under both host bridges, and the fact that those ports actually work under both bridges means they must actually be routed both places.
Oops, this is wrong. We can't tell whether the [io 0xf000-0xffff] range is routed to PCI0, because when the card under that bridge, the BIOS puts it at [io 0xdc00-0xdcff] instead.
This is probably a BIOS bug (the resource assignment part puts the card in the right area, but the ACPI _CRS doesn't match). But the fact remains that Windows does accept this _CRS and Linux doesn't but probably should.
Bjorn