Probing unconfigured PCI bridge resouces

Hi folks !

I was digging around chasing a different problem when I stumbled upon this, I may be missing something but it still feels odd...

So from what I can tell, the only way that we set the flag for the 3 bridge resources of a P2P bridge (IORESOURCE_IO, IORESOURCE_MEM etc...) is via pci_read_bridge_bases().

This will in turn only set those flags if the resource has a positive size (limit >= start) ie. it wasn't closed by the firmware.

However what does that leave us in case where we either want to ignore the firmware (reconfigure everything, which we do on various embedded platforms) or if the bridge is plain uninitialized and happen to comes up with closed resources ?

either try to reallocate or reassign those windows later in the resource assignment code. IE the code in setup-bus.c will use find_free_bus_resource() which only works if the type has been set...

Or am I missing some important fact here ?

I suspect what's saved us so far on some of those embedded platforms is that uninitialized bridges tend to come up "open" with a tiny window 0...2M (for memory, I think it's 4K for IO)... but I can see problems (I had code to specifically re-open some windows on some Apple machines in the past, which I think we lost over time and might need to be brought back for example).

Cheers,
Ben.

On Tue, 2012-01-24 at 15:25 +1100, Benjamin Herrenschmidt wrote:
.../...

Ok, I think I was missing pci_bridge_check_ranges(), gotta figure now why this didn't kick in properly for me, probably some arch code crackpot.

One thing I noticed is that we do clear IORESOURCE_UNSET in setup-res.c when setting up a device resource but we don't clear it for bridges in generic code ever, so it leaks all the way through from my early arch code. Not a big deal but annoying.

Cheers,
Ben.

Maybe you can fix that when you unify your arch resource handling code with the core one day. :)

There have been a lot of changes to handle re-allocation and such, and I'm sure there's more duplication now...

Jesse Barnes, Intel Open Source Technology Center

There isn't much the my arch does that isn't just calling into core nowadays on powerpc, look at my resource survey :-)

I have some slightly different (arguably better ?) logic to deal with unassigned resources in the initial allocation pass before re-assigning (or assigning unassigned) and flags to enable ignoring the initial state of devices (which should be improved with core changes one of these days, for example we still read all bridge bases even when we know we are just going to re-assign everything etc...).

The one thing I cannot unify is actually new in 3.3, it's the code to assign resources on the new "powernv" platform which requires its own logic unfortunately and I'm not sure I can make the core do what I want without major uglyness.

The main reason is that on p7ioc and similar recent IBM host bridges, we have a domain mechanism which allows error isolation, dma isolation etc... which also encompass MMIO (in order to properly assign MMIO errors such as target aborts etc... with a domain).

The way this works is by segmenting the MMIO space, which means that all BARs of a all devices in a given domain shall be assigned to a group of segments. Now I -can- renumber 32-bit segments so in theory I could get away with alignment tricks (tho nasty ones) only but I can't always renumber 64-bit segments (tho in my current scheme I simply don't use 64-bit MMIO segments at all, in part due to that complication).

The consequence is that I need to pre-assign domains at boot based on various constraints and policy decisions which unfortunately have to be made by the kernel arbitrarily, and then tweak the MMIO resource allocation to assign BARs of devices accordingly so they fit into "segments" that can be made to correspond with the domains.

Cheers,
Ben.