Possible issue with dangling PCI BARs
While reworking the powerpc PCI resource management, to make it more x86-like and use more of the generic code in setup-bus.c and setup-res.c, I noticed something a bit fishy in the way we deal with resources that we failed to assign.
That's especially problematic for us as we have to deal with all sort of more or less crappy firmwares, so we need our PCI code to be as robust as possible vs. whatever might have been there before the kernel kicks in.
Now, let's say that we have a device for which a BAR contain a value out of whack or conflicting with something else. We "see" that in pcibios_allocate_resources() and set res->start to 0 (and offset res->end accordingly).
At this stage, we thus have a resource that is marked as unassigned. However, we haven't actually modified the BAR content. So the device is still potentially decoding the conflicting addresses (which may have been allocated to some other device).
Now, normally, pci_assign_unassigned_resources() will take care of that and assign some new address to this device. However, that may fail (bridge window too small for example). Which mean we leave a device with a dangling BAR. In fact, a driver may even successfully call pci_enable_device_bars() on it, provided that the device has several resources and that the one that is/are requested by the driver (via the bitmask argument) are not the one(s) that failed to allocate.
So not only we can have a dangling BAR, but nothing prevent us to actually go turn IO or MEM decoding on in case it wasn't already the case on that device.
Ben.
On Thu, 2007-12-13 at 14:00 +1100, Benjamin Herrenschmidt wrote:
.../...
(oops, sent too fast)
Quoted 3 lines
So not only we can have a dangling BAR, but nothing prevent us to actually go turn IO or MEM decoding on in case it wasn't already the case on that device.
And I was about to say before I clicked "send".. can't we do something like writing all ff's into the BAR at the same time as we clear res->start ? Isn't that supposed to pretty much disable decoding on that BAR ? Or not... Probably still better than leaving it to whatever dangling value it had no ?
Also, maybe we should disable IO and MEM decoding by default on devices for which one resource of that type failed to allocate ?
Ben.
Quoted 6 lines · Benjamin Herrenschmidt
On Thu, 2007-12-13 at 14:00 +1100, Benjamin Herrenschmidt wrote:
.../...
(oops, sent too fast)
Quoted 8 lines
Quoted 3 lines
So not only we can have a dangling BAR, but nothing prevent us to actually go turn IO or MEM decoding on in case it wasn't already the case on that device.
And I was about to say before I clicked "send".. can't we do something like writing all ff's into the BAR at the same time as we clear
res->start? Isn't that supposed to pretty much disable decoding on that BAR ? Or not... Probably still better than leaving it to whatever dangling value it had no ?
Ok, reading some other threads, it seems that writing all ff's will not be a very good alternative on x86 machines where MMCONFIG sits up there...
I suppose there is nothing totally safe that can be done, thanks to Intel not thinking about making BARs individually enable/disable'able (or size-able without interrupting access, among other numerous fuckups in the PCI spec).
So if a BAR is left dangling, I think we -must- disable MEM and IO decoding on the whole device. In fact, the whole trick of passing a bitmask of required BARs to pci_enable_device_bars() in the first place doesn't fly.
Yuck.
Ben.
Quoted 4 lines · Benjamin Herrenschmidt
While reworking the powerpc PCI resource management, to make it more x86-like and use more of the generic code in
setup-bus.candsetup-res.c, I noticed something a bit fishy in the way we deal with resources that we failed to assign.
On a tangent here, is there a way (other than pci=rom) to ensure greater address space/resource space for PCI resource mappings?
Jon.
Quoted 8 lines · Jon Masters
Quoted 4 lines · Benjamin Herrenschmidt
While reworking the powerpc PCI resource management, to make it more x86-like and use more of the generic code in
setup-bus.candsetup-res.c, I noticed something a bit fishy in the way we deal with resources that we failed to assign.On a tangent here, is there a way (other than pci=rom) to ensure greater address space/resource space for PCI resource mappings?
limit lowmem ? 4g/4g ? 64 bits ? :-)
Ben
Quoted 11 lines · Benjamin Herrenschmidt
Quoted 8 lines · Jon Masters
Quoted 4 lines · Benjamin Herrenschmidt
While reworking the powerpc PCI resource management, to make it more x86-like and use more of the generic code in
setup-bus.candsetup-res.c, I noticed something a bit fishy in the way we deal with resources that we failed to assign.On a tangent here, is there a way (other than pci=rom) to ensure greater address space/resource space for PCI resource mappings?
limit lowmem ? 4g/4g ? 64 bits ? :-)
Oh, I've seen resource allocation fail on x86_64, which is what bothers me, since this really shouldn't be happening.
Jon.