PCI: try enabling "pci=use_crs" again
Subsystems (6)
cover PCI: try enabling "pci=use_crs" again
Historically, Linux has assumed a single PCI host bridge, with that bridge claiming all the address space left after RAM and legacy devices are taken out.
If the system contains multiple host bridges, we can no longer operate under that assumption. We have to know what parts of the address space are claimed by each bridge so that when we assign resources to a PCI device, we take them from a range claimed by the upstream host bridge.
We use ACPI to enumerate all the PCI host bridges in the system, and part of the host bridge description is the "_CRS" (current resource settings" property, which lists the address space used by the bridge. On x86, we currently ignore most of the _CRS information. This patch series changes this, so we will use _CRS to learn about the host bridge windows.
Since most x86 machines with multiple host bridges are relatively new, this series only turns this on for machines with BIOS dates of 2010 or newer and for a few machines that we know need it.
These apply on 0148b041be4e7, which is the current head of the linux-next branch of Jesse's pci-2.6 git tree. The first patch is just Jeff Garrett's patch to remove intel_bus.c, so that is only here for people who want to test the rest of the patches. I expect Jesse will pick up Jeff's patch via Linus' tree.
Gary and Peter have some of these problem machines, so I'm hoping they can give this a whirl.
Larry, you reported the problem the last time I tried to turn on "pci=use_crs" by default. This series shouldn't affect your machine because it's not in the whitelist, but I expect that if you boot the current kernel with "pci=use_crs", it should still fail, and if you boot with these patches and "pci=use_crs", it *should* work. I know it's a lot to ask, but it'd be great if you had a chance to try that.
Bjorn
---
Bjorn Helgaas (7):
x86/PCI: remove IOH range fetching
PCI: break out primary/secondary/subordinate for readability
PCI: split up pci_read_bridge_bases()
PCI: read bridge windows before filling in subtractive decode resources
PCI: replace bus resource table with a list
x86/PCI: use host bridge _CRS info by default on 2010 and newer machines
PCI: reference bridge window resources explicitly
Documentation/kernel-parameters.txt | 8 +- arch/ia64/include/asm/acpi.h | 1 arch/ia64/pci/pci.c | 20 ++-- arch/x86/include/asm/pci_x86.h | 1 arch/x86/pci/Makefile | 2 arch/x86/pci/acpi.c | 105 +++++++++++++++------- arch/x86/pci/bus_numa.c | 9 +- arch/x86/pci/common.c | 3 + arch/x86/pci/intel_bus.c | 94 -------------------- drivers/acpi/pci_root.c | 1 drivers/pci/bus.c | 50 ++++++++++- drivers/pci/hotplug/shpchp_sysfs.c | 15 ++- drivers/pci/pci.c | 6 + drivers/pci/probe.c | 102 +++++++++++++++------- drivers/pci/quirks.c | 4 - drivers/pci/setup-bus.c | 166 ++++++++++++++++++----------------- drivers/pcmcia/rsrc_nonstatic.c | 7 + drivers/pcmcia/yenta_socket.c | 46 ++++++---- include/acpi/acpi_drivers.h | 1 include/linux/pci.h | 27 ++++-- 20 files changed, 367 insertions(+), 301 deletions(-) delete mode 100644 arch/x86/pci/intel_bus.c
PATCH 1/7 x86/PCI: remove IOH range fetching
This is Jeff Garrett's patch to remove intel_bus.c. It's already in Linus' tree (e8e06eae4ffd), but not yet in Jesse's tree. It's only here so the subsequent patches don't have to update intel_bus.c.
---
arch/x86/pci/Makefile | 2 - arch/x86/pci/intel_bus.c | 94 ---------------------------------------------- 2 files changed, 1 insertions(+), 95 deletions(-) delete mode 100644 arch/x86/pci/intel_bus.c
arch/x86/pci/Makefile +1 -1
@@ -15,7 +15,7 @@ obj-$(CONFIG_X86_NUMAQ) += numaq_32.o obj-y += common.o early.o obj-y += amd_bus.o -obj-$(CONFIG_X86_64) += bus_numa.o intel_bus.o +obj-$(CONFIG_X86_64) += bus_numa.o ifeq ($(CONFIG_PCI_DEBUG),y) EXTRA_CFLAGS += -DDEBUG
arch/x86/pci/intel_bus.c deleted +0 -94
94 lines not on this page · raw message
PATCH 2/7 PCI: break out primary/secondary/subordinate for readability
In reply to Bjorn HelgaasNo functional change; just add names for the primary/secondary/subordinate bus numbers read from config space rather than repeatedly masking/shifting.
---
drivers/pci/probe.c | 24 ++++++++++++++---------- 1 files changed, 14 insertions(+), 10 deletions(-)
drivers/pci/probe.c +14 -10
@@ -623,16 +623,20 @@ int __devinit pci_scan_bridge(struct pci_bus *bus, struct pci_dev *dev, int max,int is_cardbus = (dev->hdr_type ==PCI_HEADER_TYPE_CARDBUS );u32 buses, i, j =0 ;u16 bctl; +u8 primary, secondary, subordinate;int broken =0 ;pci_read_config_dword (dev,PCI_PRIMARY_BUS , &buses); + primary = buses &0xFF ; + secondary = (buses >>8 ) &0xFF ; + subordinate = (buses >>16 ) &0xFF ; -dev_dbg (&dev->dev,"scanning behind bridge, config %06x, pass %d\n" , - buses &0xffffff , pass); +dev_dbg (&dev->dev,"scanning [bus %02x-%02x] behind bridge, pass %d\n" , + secondary, subordinate, pass);/* Check if setup is sensible at all */ if (!pass && - ((buses &0xff ) != bus->number || ((buses >>8 ) &0xff ) <= bus->number)) { + (primary != bus->number || secondary <= bus->number)) {dev_dbg (&dev->dev,"bus configuration invalid, reconfiguring\n" ); broken =1 ; }
@@ -643,15 +647,15 @@ int __devinit pci_scan_bridge(struct pci_bus *bus, struct pci_dev *dev, int max,pci_write_config_word (dev,PCI_BRIDGE_CONTROL , bctl & ~PCI_BRIDGE_CTL_MASTER_ABORT ); -if ((buses &0xffff00 ) && !pcibios_assign_all_busses () && !is_cardbus && !broken) { -unsigned int cmax, busnr; +if ((secondary || subordinate) && !pcibios_assign_all_busses () && + !is_cardbus && !broken) { +unsigned int cmax;/* * Bus already configured by firmware, process it in the first * pass and just note the configuration. */ if (pass) goto out; - busnr = (buses >>8 ) &0xFF ;/* * If we already got to this bus through a different bridge,
@@ -660,13 +664,13 @@ int __devinit pci_scan_bridge(struct pci_bus *bus, struct pci_dev *dev, int max,* However, we continue to descend down the hierarchy and * scan remaining child buses. */ - child =pci_find_bus (pci_domain_nr (bus), busnr); + child =pci_find_bus (pci_domain_nr (bus), secondary);if (!child) { - child =pci_add_new_bus (bus, dev, busnr); + child =pci_add_new_bus (bus, dev, secondary);if (!child) goto out; - child->primary = buses &0xFF ; - child->subordinate = (buses >>16 ) &0xFF ; + child->primary = primary; + child->subordinate = subordinate; child->bridge_ctl = bctl; }
PATCH 3/7 PCI: split up pci_read_bridge_bases()
In reply to Bjorn HelgaasNo functional change; this breaks up pci_read_bridge_bases() into separate pieces for the I/O, memory, and prefetchable memory windows, similar to how Yinghai recently split up pci_setup_bridge() in 68e84ff3bdc.
---
drivers/pci/probe.c | 54 +++++++++++++++++++++++++++++++++++++-------------- 1 files changed, 39 insertions(+), 15 deletions(-)
drivers/pci/probe.c +39 -15
@@ -281,26 +281,12 @@ static void pci_read_bases(struct pci_dev *dev, unsigned int howmany, int rom) } } -void __devinit pci_read_bridge_bases (struct pci_bus *child) +static void __devinit pci_read_bridge_io (struct pci_bus *child) {struct pci_dev *dev = child->self;u8 io_base_lo, io_limit_lo; -u16 mem_base_lo, mem_limit_lo;unsigned long base, limit;struct resource *res; -int i; - -if (pci_is_root_bus (child))/* It's a host bus, nothing to read */ -return ; - -dev_info (&dev->dev,"PCI bridge to [bus %02x-%02x]%s\n" , - child->secondary, child->subordinate, - dev->transparent ?" (subtractive decode)" :"" ); - -if (dev->transparent) { -for (i =3 ; i <PCI_BUS_NUM_RESOURCES ; i++) - child->resource[i] = child->parent->resource[i -3 ]; - } res = child->resource[0 ];pci_read_config_byte (dev,PCI_IO_BASE , &io_base_lo);
@@ -324,6 +310,14 @@ void __devinit pci_read_bridge_bases(struct pci_bus *child) res->end = limit +0xfff ;dev_printk (KERN_DEBUG , &dev->dev," bridge window %pR\n" , res); } +} + +static void __devinit pci_read_bridge_mmio (struct pci_bus *child) +{ +struct pci_dev *dev = child->self; +u16 mem_base_lo, mem_limit_lo; +unsigned long base, limit; +struct resource *res; res = child->resource[1 ];pci_read_config_word (dev,PCI_MEMORY_BASE , &mem_base_lo);
@@ -336,6 +330,14 @@ void __devinit pci_read_bridge_bases(struct pci_bus *child) res->end = limit +0xfffff ;dev_printk (KERN_DEBUG , &dev->dev," bridge window %pR\n" , res); } +} + +static void __devinit pci_read_bridge_mmio_pref (struct pci_bus *child) +{ +struct pci_dev *dev = child->self; +u16 mem_base_lo, mem_limit_lo; +unsigned long base, limit; +struct resource *res; res = child->resource[2 ];pci_read_config_word (dev,PCI_PREF_MEMORY_BASE , &mem_base_lo);
@@ -377,6 +379,28 @@ void __devinit pci_read_bridge_bases(struct pci_bus *child) } } +void __devinit pci_read_bridge_bases (struct pci_bus *child) +{ +struct pci_dev *dev = child->self; +int i; + +if (pci_is_root_bus (child))/* It's a host bus, nothing to read */ +return ; + +dev_info (&dev->dev,"PCI bridge to [bus %02x-%02x]%s\n" , + child->secondary, child->subordinate, + dev->transparent ?" (subtractive decode)" :"" ); + +if (dev->transparent) { +for (i =3 ; i <PCI_BUS_NUM_RESOURCES ; i++) + child->resource[i] = child->parent->resource[i -3 ]; + } + +pci_read_bridge_io (child); +pci_read_bridge_mmio (child); +pci_read_bridge_mmio_pref (child); +} +static struct pci_bus *pci_alloc_bus (void ) {struct pci_bus *b;
Re: [PATCH v1 3/7] PCI: split up pci_read_bridge_bases()
Quoted 8 lines · On Wed, 03 Feb 2010 16:39:11 -0700 Bjorn Helgaas <[email protected]> wrote:
No functional change; this breaks up
pci_read_bridge_bases()into separate pieces for the I/O, memory, and prefetchable memory windows, similar to how Yinghai recently split uppci_setup_bridge()in68e84ff3bdc.Signed-off-by Bjorn Helgaas <[email protected]>---
This one conflicts a bit with Yinghai's recent stuff. Can you respin against my linux-next branch? May as well do the whole series, then I'll just apply it in one go.
Thanks,
Signature · 1 line
Jesse Barnes, Intel Open Source Technology Center
PATCH 4/7 PCI: read bridge windows before filling in subtractive decode resources
In reply to Bjorn HelgaasNo functional change; this fills in the bus subtractive decode resources after reading the bridge window information rather than before. Also, print out the subtractive decode resources as we already do for the positive decode windows.
---
drivers/pci/probe.c | 15 ++++++++++----- 1 files changed, 10 insertions(+), 5 deletions(-)
drivers/pci/probe.c +10 -5
@@ -391,14 +391,19 @@ void __devinit pci_read_bridge_bases(struct pci_bus *child) child->secondary, child->subordinate, dev->transparent ? " (subtractive decode)": ""); -if (dev->transparent) { -for (i =3 ; i <PCI_BUS_NUM_RESOURCES ; i++) - child->resource[i] = child->parent->resource[i -3 ]; - } -pci_read_bridge_io (child);pci_read_bridge_mmio (child);pci_read_bridge_mmio_pref (child); + +if (dev->transparent) { +for (i =3 ; i <PCI_BUS_NUM_RESOURCES ; i++) { + child->resource[i] = child->parent->resource[i -3 ]; +if (child->resource[i]) +dev_printk (KERN_DEBUG , &dev->dev, +" bridge window %pR (subtractive decode)\n" , + child->resource[i]); + } + } }static struct pci_bus *pci_alloc_bus (void )
PATCH 5/7 PCI: replace bus resource table with a list
In reply to Bjorn HelgaasPreviously we used a table of size PCI_BUS_NUM_RESOURCES for resources forwarded to a bus by its upstream bridge. We've increased this size several times when the table overflowed.
But there's no good limit on the number of resources for a bus because host bridges can forward any number of ranges to their secondary buses (i.e., PCI root buses).
This patch replaces the fixed-size table with a list.
---
arch/ia64/pci/pci.c | 20 +++--- arch/x86/pci/acpi.c | 33 +---------- arch/x86/pci/bus_numa.c | 9 ++- drivers/pci/bus.c | 50 +++++++++++++++- drivers/pci/hotplug/shpchp_sysfs.c | 15 +++-- drivers/pci/pci.c | 6 +- drivers/pci/probe.c | 35 +++++++---- drivers/pci/setup-bus.c | 111 +++++++++++++++++------------------- drivers/pcmcia/rsrc_nonstatic.c | 7 +- drivers/pcmcia/yenta_socket.c | 7 +- include/linux/pci.h | 16 ++++- 11 files changed, 169 insertions(+), 140 deletions(-)
arch/ia64/pci/pci.c +8 -12
@@ -320,9 +320,9 @@ static __devinit acpi_status add_window(struct acpi_resource *res, void *data)static void __devinit pcibios_setup_root_windows (struct pci_bus *bus,struct pci_controller *ctrl) { -int i, j; +int i; - j =0 ; +pci_bus_remove_resources (bus);for (i =0 ; i < ctrl->windows; i++) {struct resource *res = &ctrl->window[i].resource;/* HP's firmware has a hack to work around a Windows bug.
@@ -330,13 +330,7 @@ pcibios_setup_root_windows(struct pci_bus *bus, struct pci_controller *ctrl)if ((res->flags &IORESOURCE_MEM ) && (res->end - res->start <16 ))continue ; -if (j >=PCI_BUS_NUM_RESOURCES ) { -dev_warn (&bus->dev, -"ignoring host bridge window %pR (no space)\n" , - res); -continue ; - } - bus->resource[j++] = res; +pci_bus_add_resource (bus, res,0 ); } }
@@ -451,13 +445,15 @@ EXPORT_SYMBOL(pcibios_bus_to_resource);static int __devinit is_valid_resource (struct pci_dev *dev,int idx) { -unsigned int i, type_mask =IORESOURCE_IO |IORESOURCE_MEM ; +unsigned int type_mask =IORESOURCE_IO |IORESOURCE_MEM ;struct resource *devr = &dev->resource[idx]; +struct pci_bus_resource *bus_res;if (!dev->bus)return 0 ; -for (i=0 ; i<PCI_BUS_NUM_RESOURCES ; i++) { -struct resource *busr = dev->bus->resource[i]; + +list_for_each_entry (bus_res, &dev->bus->resources, list) { +struct resource *busr = bus_res->res;if (!busr || ((busr->flags ^ devr->flags) & type_mask))continue ;
arch/x86/pci/acpi.c +4 -29
@@ -45,20 +45,6 @@ count_resource(struct acpi_resource *acpi_res, void *data)return AE_OK ; } -static int -bus_has_transparent_bridge (struct pci_bus *bus) -{ -struct pci_dev *dev; - -list_for_each_entry (dev, &bus->devices, bus_list) { -u16 class = dev->class >>8 ; - -if (class ==PCI_CLASS_BRIDGE_PCI && dev->transparent) -return true; - } -return false; -} -static void align_resource (struct acpi_device *bridge,struct resource *res) {
@@ -92,12 +78,8 @@ setup_resource(struct acpi_resource *acpi_res, void *data)acpi_status status;unsigned long flags;struct resource *root; -int max_root_bus_resources =PCI_BUS_NUM_RESOURCES ;u64 start, end; -if (bus_has_transparent_bridge (info->bus)) - max_root_bus_resources -=3 ; - status =resource_to_addr (acpi_res, &addr);if (!ACPI_SUCCESS (status))return AE_OK ;
@@ -115,15 +97,6 @@ setup_resource(struct acpi_resource *acpi_res, void *data) start = addr.minimum + addr.translation_offset; end = start + addr.address_length -1 ; -if (info->res_num >= max_root_bus_resources) { -if (pci_probe &PCI_USE__CRS ) -printk (KERN_WARNING "PCI: Failed to allocate " -"0x%lx-0x%lx from %s for %s due to _CRS " -"returning more than %d resource descriptors\n" , - (unsigned long ) start, (unsigned long ) end, - root->name, info->name, max_root_bus_resources); -return AE_OK ; - } res = &info->res[info->res_num]; res->name = info->name;
@@ -143,7 +116,7 @@ setup_resource(struct acpi_resource *acpi_res, void *data)dev_err (&info->bridge->dev,"can't allocate host bridge window %pR\n" , res); }else { - info->bus->resource[info->res_num] = res; +pci_bus_add_resource (info->bus, res,0 ); info->res_num++;if (addr.translation_offset) dev_info(&info->bridge->dev, "host bridge window %pR "
@@ -164,7 +137,9 @@ get_current_resources(struct acpi_device *device, int busnum,struct pci_root_info info;size_t size; -if (!(pci_probe &PCI_USE__CRS )) +if (pci_probe &PCI_USE__CRS ) +pci_bus_remove_resources (bus); +else dev_info (&device->dev,"ignoring host bridge windows from ACPI; " "boot with \"pci=use_crs\" to use them\n" );
arch/x86/pci/bus_numa.c +6 -3
@@ -12,10 +12,12 @@ void x86_pci_root_bus_res_quirks(struct pci_bus *b)int i;int j;struct pci_root_info *info; +struct pci_bus_resource *bus_res;/* don't go for it if _CRS is used already */ -if (b->resource[0 ] != &ioport_resource || - b->resource[1 ] != &iomem_resource) + bus_res = list_first_entry(&b->resources, struct pci_bus_resource, + list); +if (bus_res->res != &ioport_resource)return ;if (!pci_root_num)
@@ -36,13 +38,14 @@ void x86_pci_root_bus_res_quirks(struct pci_bus *b)printk (KERN_DEBUG "PCI: peer root bus %02x res updated from pci conf\n" , b->number); +pci_bus_remove_resources (b); info = &pci_root_info[i];for (j =0 ; j < info->res_num; j++) {struct resource *res;struct resource *root; res = &info->res[j]; - b->resource[j] = res; +pci_bus_add_resource (b, res,0 );if (res->flags &IORESOURCE_IO ) root = &ioport_resource;else
drivers/pci/bus.c +47 -3
68 lines not on this page · raw message
drivers/pci/hotplug/shpchp_sysfs.c +8 -7
@@ -39,16 +39,17 @@ static ssize_t show_ctrl (struct device *dev, struct device_attribute *attr, cha {struct pci_dev *pdev;char * out = buf; -int index, busnr; +int busnr;struct resource *res;struct pci_bus *bus; +struct pci_bus_resource *bus_res; pdev = container_of (dev, struct pci_dev, dev); bus = pdev->subordinate; out +=sprintf (buf,"Free resources: memory\n" ); -for (index =0 ; index <PCI_BUS_NUM_RESOURCES ; index++) { - res = bus->resource[index]; +list_for_each_entry (bus_res, &bus->resources, list) { + res = bus_res->res;if (res && (res->flags &IORESOURCE_MEM ) && !(res->flags &IORESOURCE_PREFETCH )) { out += sprintf(out, "start = %8.8llx, "
@@ -58,8 +59,8 @@ static ssize_t show_ctrl (struct device *dev, struct device_attribute *attr, cha } } out +=sprintf (out,"Free resources: prefetchable memory\n" ); -for (index =0 ; index <PCI_BUS_NUM_RESOURCES ; index++) { - res = bus->resource[index]; +list_for_each_entry (bus_res, &bus->resources, list) { + res = bus_res->res;if (res && (res->flags &IORESOURCE_MEM ) && (res->flags &IORESOURCE_PREFETCH )) { out += sprintf(out, "start = %8.8llx, "
@@ -69,8 +70,8 @@ static ssize_t show_ctrl (struct device *dev, struct device_attribute *attr, cha } } out +=sprintf (out,"Free resources: IO\n" ); -for (index =0 ; index <PCI_BUS_NUM_RESOURCES ; index++) { - res = bus->resource[index]; +list_for_each_entry (bus_res, &bus->resources, list) { + res = bus_res->res;if (res && (res->flags &IORESOURCE_IO )) { out += sprintf(out, "start = %8.8llx, " "length = %8.8llx\n",
drivers/pci/pci.c +3 -3
@@ -385,11 +385,11 @@ struct resource * pci_find_parent_resource(const struct pci_dev *dev, struct resource *res) { const struct pci_bus *bus = dev->bus; -int i; +struct pci_bus_resource *bus_res;struct resource *best =NULL ; -for (i =0 ; i <PCI_BUS_NUM_RESOURCES ; i++) { -struct resource *r = bus->resource[i]; +list_for_each_entry (bus_res, &bus->resources, list) { +struct resource *r = bus_res->res;if (!r)continue ;if (res->start && !(res->start >= r->start && res->end <= r->end))
drivers/pci/probe.c +21 -14
@@ -89,6 +89,7 @@ static void release_pcibus_dev(struct device *dev)if (pci_bus->bridge)put_device (pci_bus->bridge); +pci_bus_remove_resources (pci_bus);kfree (pci_bus); }
@@ -288,7 +289,8 @@ static void __devinit pci_read_bridge_io(struct pci_bus *child)unsigned long base, limit;struct resource *res; - res = child->resource[0 ]; + res = &dev->resource[PCI_BRIDGE_RESOURCES +0 ]; +pci_read_config_byte (dev,PCI_IO_BASE , &io_base_lo);pci_read_config_byte (dev,PCI_IO_LIMIT , &io_limit_lo); base = (io_base_lo &PCI_IO_RANGE_MASK ) <<8 ;
@@ -319,7 +321,8 @@ static void __devinit pci_read_bridge_mmio(struct pci_bus *child)unsigned long base, limit;struct resource *res; - res = child->resource[1 ]; + res = &dev->resource[PCI_BRIDGE_RESOURCES +1 ]; +pci_read_config_word (dev,PCI_MEMORY_BASE , &mem_base_lo);pci_read_config_word (dev,PCI_MEMORY_LIMIT , &mem_limit_lo); base = (mem_base_lo &PCI_MEMORY_RANGE_MASK ) <<16 ;
@@ -339,7 +342,8 @@ static void __devinit pci_read_bridge_mmio_pref(struct pci_bus *child)unsigned long base, limit;struct resource *res; - res = child->resource[2 ]; + res = &dev->resource[PCI_BRIDGE_RESOURCES +2 ]; +pci_read_config_word (dev,PCI_PREF_MEMORY_BASE , &mem_base_lo);pci_read_config_word (dev,PCI_PREF_MEMORY_LIMIT , &mem_limit_lo); base = (mem_base_lo &PCI_PREF_RANGE_MASK ) <<16 ;
@@ -382,7 +386,7 @@ static void __devinit pci_read_bridge_mmio_pref(struct pci_bus *child)void __devinit pci_read_bridge_bases (struct pci_bus *child) {struct pci_dev *dev = child->self; -int i; +struct pci_bus_resource *bus_res;if (pci_is_root_bus (child))/* It's a host bus, nothing to read */ return ;
@@ -396,12 +400,11 @@ void __devinit pci_read_bridge_bases(struct pci_bus *child)pci_read_bridge_mmio_pref (child);if (dev->transparent) { -for (i =3 ; i <PCI_BUS_NUM_RESOURCES ; i++) { - child->resource[i] = child->parent->resource[i -3 ]; -if (child->resource[i]) -dev_printk (KERN_DEBUG , &dev->dev, -" bridge window %pR (subtractive decode)\n" , - child->resource[i]); +list_for_each_entry (bus_res, &child->parent->resources, list) { +pci_bus_add_resource (child, bus_res->res,0 ); +dev_printk (KERN_DEBUG , &dev->dev, +" bridge window %pR (subtractive decode)\n" , + bus_res->res); } } }
@@ -416,6 +419,7 @@ static struct pci_bus * pci_alloc_bus(void)INIT_LIST_HEAD (&b->children);INIT_LIST_HEAD (&b->devices);INIT_LIST_HEAD (&b->slots); +INIT_LIST_HEAD (&b->resources); b->max_bus_speed =PCI_SPEED_UNKNOWN ; b->cur_bus_speed =PCI_SPEED_UNKNOWN ; }
@@ -561,6 +565,7 @@ static struct pci_bus *pci_alloc_child_bus(struct pci_bus *parent, {struct pci_bus *child;int i; +struct resource *res;/* * Allocate a new bus, and inherit stuff from the parent..
@@ -599,9 +604,11 @@ static struct pci_bus *pci_alloc_child_bus(struct pci_bus *parent,/* Set up default resource pointers and names.. */ for (i =0 ; i <PCI_BRIDGE_RESOURCE_NUM ; i++) { - child->resource[i] = &bridge->resource[PCI_BRIDGE_RESOURCES +i]; - child->resource[i]->name = child->name; + res = &bridge->resource[PCI_BRIDGE_RESOURCES + i]; + res->name = child->name; +pci_bus_add_resource (child, res,PCI_POSITIVE_DECODE ); } + bridge->subordinate = child;return child;
@@ -1427,8 +1434,8 @@ struct pci_bus * pci_create_bus(struct device *parent,pci_create_legacy_files (b); b->number = b->secondary = bus; - b->resource[0 ] = &ioport_resource; - b->resource[1 ] = &iomem_resource; +pci_bus_add_resource (b, &ioport_resource,0 ); +pci_bus_add_resource (b, &iomem_resource,0 );return b;
drivers/pci/setup-bus.c +53 -58
166 lines not on this page · raw message
drivers/pcmcia/rsrc_nonstatic.c +4 -3
@@ -786,8 +786,9 @@ static int adjust_io(struct pcmcia_socket *s, unsigned int action, unsigned long#ifdef CONFIG_PCI static int nonstatic_autoadd_resources (struct pcmcia_socket *s) { +struct pci_bus_resource *bus_res;struct resource *res; -int i, done =0 ; +int done =0 ;if (!s->cb_dev || !s->cb_dev->bus)return -ENODEV ;
@@ -803,8 +804,8 @@ static int nonstatic_autoadd_resources(struct pcmcia_socket *s)return -EINVAL ;#endif -for (i =0 ; i <PCI_BUS_NUM_RESOURCES ; i++) { - res = s->cb_dev->bus->resource[i]; +list_for_each_entry (bus_res, &s->cb_dev->bus->resources, list) { + res = bus_res->res;if (!res)continue ;
drivers/pcmcia/yenta_socket.c +4 -3
@@ -649,9 +649,10 @@ static int yenta_search_one_res(struct resource *root, struct resource *res,static int yenta_search_res (struct yenta_socket *socket,struct resource *res,u32 min) { -int i; -for (i =0 ; i <PCI_BUS_NUM_RESOURCES ; i++) { -struct resource *root = socket->dev->bus->resource[i]; +struct pci_bus_resource *bus_res; + +list_for_each_entry (bus_res, &socket->dev->bus->resources, list) { +struct resource *root = bus_res->res;if (!root)continue ;
include/linux/pci.h +11 -5
@@ -363,9 +363,13 @@ static inline void pci_add_saved_cap(struct pci_dev *pci_dev,hlist_add_head (&new_cap->next, &pci_dev->saved_cap_space); } -#ifndef PCI_BUS_NUM_RESOURCES -#define PCI_BUS_NUM_RESOURCES 16 -#endif +#define PCI_POSITIVE_DECODE 1 + +struct pci_bus_resource { +struct list_head list; +struct resource *res; +unsigned int flags; +};#define PCI_REGION_FLAG_MASK 0x0fU/* These bits of resource flags tell us the PCI region flags */
@@ -376,8 +380,7 @@ struct pci_bus {struct list_head devices;/* list of devices on this bus */ struct pci_dev *self;/* bridge device as seen by parent */ struct list_head slots;/* list of slots on this bus */ -struct resource *resource[PCI_BUS_NUM_RESOURCES ]; -/* address space routed to this bus */ +struct list_head resources;/* address space routed to this bus */ struct pci_ops *ops;/* configuration access functions */ void *sysdata;/* hook for sys-specific extension */
@@ -823,6 +826,9 @@ int pci_request_selected_regions_exclusive(struct pci_dev *, int, const char *);void pci_release_selected_regions (struct pci_dev *,int );/* drivers/pci/bus.c */ +void pci_bus_add_resource (struct pci_bus *bus,struct resource *res,unsigned int flags); +struct resource *pci_bus_get_resource (struct pci_bus *bus,unsigned long flags,int num); +void pci_bus_remove_resources (struct pci_bus *bus); int __must_check pci_bus_alloc_resource(struct pci_bus *bus, struct resource *res, resource_size_t size, resource_size_t align, resource_size_t min,
PATCH 6/7 x86/PCI: use host bridge _CRS info by default on 2010 and newer machines
In reply to Bjorn HelgaasThe main benefit of using ACPI host bridge window information is that we can do better resource allocation in systems with multiple host bridges. Most of these systems are new, so this patch turns on "pci=use_crs" only on machines with a BIOS date of 2010 or newer. In addition, it whitelists a few older machines that are known to benefit.
We previously turned on "pci=use_crs" by default on *all* machines (9e9f46c44e48), but had to revert that because of problems such as Larry Finger's: http://lkml.org/lkml/2009/6/23/715.
I think the problem Larry saw was caused by overflowing the pci_bus resource table. This patch should not affect Larry's machine directly because it is older than 2010, but the table overflow should be fixed by the previous patch in this series.
http://bugzilla.kernel.org/show_bug.cgi?id=14183
---
Documentation/kernel-parameters.txt | 8 +++- arch/ia64/include/asm/acpi.h | 1 arch/x86/include/asm/pci_x86.h | 1 arch/x86/pci/acpi.c | 76 ++++++++++++++++++++++++++++++++--- arch/x86/pci/common.c | 3 + drivers/acpi/pci_root.c | 1 include/acpi/acpi_drivers.h | 1 7 files changed, 83 insertions(+), 8 deletions(-)
Documentation/kernel-parameters.txt +6 -2
@@ -1939,8 +1939,12 @@ and is between 256 and 4096 characters. It is defined in the file IRQ routing is enabled. noacpi [X86] Do not use ACPI for IRQ routing or for PCI scanning. - use_crs [X86] Use _CRS for PCI resource - allocation. + use_crs [X86] Use PCI host bridge window information + from ACPI. On BIOSes from 2010 or later, this + is enabled by default. If you need to use this, + please report a bug. + nocrs [X86] Ignore PCI host bridge windows from ACPI. + If you need to use this, please report a bug. routeirq Do IRQ routing for all PCI devices. This is normally done in pci_enable_device(), so this option is a temporary workaround
arch/ia64/include/asm/acpi.h +1 -0
@@ -97,6 +97,7 @@ ia64_acpi_release_global_lock (unsigned int *lock)#endif #define acpi_processor_cstate_check (x) (x)/* no idle limits on IA64 :) */ static inline void disable_acpi (void ) { } +static inline void pci_acpi_crs_quirks (void ) { }const char *acpi_get_sysname (void );int acpi_request_vector (u32 int_type);
arch/x86/include/asm/pci_x86.h +1 -0
@@ -29,6 +29,7 @@#define PCI_CHECK_ENABLE_AMD_MMCONF 0x20000#define PCI_HAS_IO_ECS 0x40000#define PCI_NOASSIGN_ROMS 0x80000 +#define PCI_ROOT_NO_CRS 0x100000extern unsigned int pci_probe;extern unsigned long pirq_table_addr;
arch/x86/pci/acpi.c +70 -6
95 lines not on this page · raw message
arch/x86/pci/common.c +3 -0
@@ -520,6 +520,9 @@ char * __devinit pcibios_setup(char *str) }else if (!strcmp (str,"use_crs" )) { pci_probe |=PCI_USE__CRS ;return NULL ; + }else if (!strcmp (str,"nocrs" )) { + pci_probe |=PCI_ROOT_NO_CRS ; +return NULL ; }else if (!strcmp (str,"earlydump" )) { pci_early_dump_regs =1 ;return NULL ;
drivers/acpi/pci_root.c +1 -0
@@ -566,6 +566,7 @@ static int __init acpi_pci_root_init(void)if (acpi_pci_disabled)return 0 ; +pci_acpi_crs_quirks ();if (acpi_bus_register_driver (&acpi_pci_root_driver) <0 )return -ENODEV ;
include/acpi/acpi_drivers.h +1 -0
@@ -104,6 +104,7 @@ int acpi_pci_bind_root(struct acpi_device *device);struct pci_bus *pci_acpi_scan_root (struct acpi_device *device,int domain,int bus); +void pci_acpi_crs_quirks (void );/* -------------------------------------------------------------------------- Processor
PATCH 7/7 PCI: reference bridge window resources explicitly
In reply to Bjorn HelgaasNo functional change; just be more explicit about this mapping for bridge resources:
bridge->res[PCI_BRIDGE_RESOURCES+0]: I/O window (or CB I/O 0 window) bridge->res[PCI_BRIDGE_RESOURCES+1]: mem window (or CB I/O 1 window) bridge->res[PCI_BRIDGE_RESOURCES+2]: pref mem window (or CB mem 0 window) bridge->res[PCI_BRIDGE_RESOURCES+3]: CB mem 1 window
---
drivers/pci/probe.c | 6 ++-- drivers/pci/quirks.c | 4 +-- drivers/pci/setup-bus.c | 55 +++++++++++++++++++++++------------------ drivers/pcmcia/yenta_socket.c | 39 +++++++++++++++++------------ include/linux/pci.h | 11 +++++++- 5 files changed, 69 insertions(+), 46 deletions(-)
drivers/pci/probe.c +3 -3
@@ -289,7 +289,7 @@ static void __devinit pci_read_bridge_io(struct pci_bus *child)unsigned long base, limit;struct resource *res; - res = &dev->resource[PCI_BRIDGE_RESOURCES +0 ]; + res = &dev->resource[PCI_BRIDGE_IO_WINDOW ];pci_read_config_byte (dev,PCI_IO_BASE , &io_base_lo);pci_read_config_byte (dev,PCI_IO_LIMIT , &io_limit_lo);
@@ -321,7 +321,7 @@ static void __devinit pci_read_bridge_mmio(struct pci_bus *child)unsigned long base, limit;struct resource *res; - res = &dev->resource[PCI_BRIDGE_RESOURCES +1 ]; + res = &dev->resource[PCI_BRIDGE_MEM_WINDOW ];pci_read_config_word (dev,PCI_MEMORY_BASE , &mem_base_lo);pci_read_config_word (dev,PCI_MEMORY_LIMIT , &mem_limit_lo);
@@ -342,7 +342,7 @@ static void __devinit pci_read_bridge_mmio_pref(struct pci_bus *child)unsigned long base, limit;struct resource *res; - res = &dev->resource[PCI_BRIDGE_RESOURCES +2 ]; + res = &dev->resource[PCI_BRIDGE_PREF_MEM_WINDOW ];pci_read_config_word (dev,PCI_PREF_MEMORY_BASE , &mem_base_lo);pci_read_config_word (dev,PCI_PREF_MEMORY_LIMIT , &mem_limit_lo);
drivers/pci/quirks.c +2 -2
@@ -1890,7 +1890,7 @@ static void __devinit quirk_p64h2_1k_io(struct pci_dev *dev)u16 en1k;u8 io_base_lo, io_limit_lo;unsigned long base, limit; -struct resource *res = dev->resource +PCI_BRIDGE_RESOURCES ; +struct resource *res = &dev->resource[PCI_BRIDGE_IO_WINDOW ];pci_read_config_word (dev,0x40 , &en1k);
@@ -1917,7 +1917,7 @@ DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x1460, quirk_p64h2_1k_io);static void __devinit quirk_p64h2_1k_io_fix_iobl (struct pci_dev *dev) {u16 en1k, iobl_adr, iobl_adr_1k; -struct resource *res = dev->resource +PCI_BRIDGE_RESOURCES ; +struct resource *res = &dev->resource[PCI_BRIDGE_IO_WINDOW ];pci_read_config_word (dev,0x40 , &en1k);
drivers/pci/setup-bus.c +31 -24
108 lines not on this page · raw message
drivers/pcmcia/yenta_socket.c +23 -16
@@ -673,7 +673,7 @@ static int yenta_allocate_res(struct yenta_socket *socket, int nr, unsigned typestruct pci_bus_region region;unsigned mask; - res = dev->resource +PCI_BRIDGE_RESOURCES + nr; + res = &dev->resource[nr];/* Already allocated? */ if (res->parent)return 0 ;
@@ -690,7 +690,7 @@ static int yenta_allocate_res(struct yenta_socket *socket, int nr, unsigned type region.end =config_readl (socket, addr_end) | ~mask;if (region.start && region.end > region.start && !override_bios) {pcibios_bus_to_resource (dev, res, ®ion); -if (pci_claim_resource (dev,PCI_BRIDGE_RESOURCES + nr) ==0 ) +if (pci_claim_resource (dev, nr) ==0 )return 0 ;dev_printk (KERN_INFO , &dev->dev, "Preassigned resource %d busy or not available, "
@@ -731,32 +731,39 @@ static int yenta_allocate_res(struct yenta_socket *socket, int nr, unsigned typestatic void yenta_allocate_resources (struct yenta_socket *socket) {int program =0 ; - program +=yenta_allocate_res (socket,0 ,IORESOURCE_IO , -PCI_CB_IO_BASE_0 ,PCI_CB_IO_LIMIT_0 ); - program +=yenta_allocate_res (socket,1 ,IORESOURCE_IO , -PCI_CB_IO_BASE_1 ,PCI_CB_IO_LIMIT_1 ); - program +=yenta_allocate_res (socket,2 ,IORESOURCE_MEM |IORESOURCE_PREFETCH , + program +=yenta_allocate_res (socket,PCI_CB_BRIDGE_IO_0_WINDOW , +IORESOURCE_IO ,PCI_CB_IO_BASE_0 ,PCI_CB_IO_LIMIT_0 ); + program +=yenta_allocate_res (socket,PCI_CB_BRIDGE_IO_1_WINDOW , +IORESOURCE_IO ,PCI_CB_IO_BASE_1 ,PCI_CB_IO_LIMIT_1 ); + program +=yenta_allocate_res (socket,PCI_CB_BRIDGE_MEM_0_WINDOW , +IORESOURCE_MEM |IORESOURCE_PREFETCH ,PCI_CB_MEMORY_BASE_0 ,PCI_CB_MEMORY_LIMIT_0 ); - program +=yenta_allocate_res (socket,3 ,IORESOURCE_MEM , + program +=yenta_allocate_res (socket,PCI_CB_BRIDGE_MEM_1_WINDOW , +IORESOURCE_MEM ,PCI_CB_MEMORY_BASE_1 ,PCI_CB_MEMORY_LIMIT_1 );if (program)pci_setup_cardbus (socket->dev->subordinate); } +static void yenta_free_resource (struct yenta_socket *socket,int nr) +{ +struct resource *res; + + res = &socket->dev->resource[nr]; +if (res->start !=0 && res->end !=0 ) +release_resource (res); + res->start = res->end = res->flags =0 ; +}/* * Free the bridge mappings for the device.. */ static void yenta_free_resources (struct yenta_socket *socket) { -int i; -for (i =0 ; i <4 ; i++) { -struct resource *res; - res = socket->dev->resource +PCI_BRIDGE_RESOURCES + i; -if (res->start !=0 && res->end !=0 ) -release_resource (res); - res->start = res->end = res->flags =0 ; - } +yenta_free_resource (socket,PCI_CB_BRIDGE_IO_0_WINDOW ); +yenta_free_resource (socket,PCI_CB_BRIDGE_IO_1_WINDOW ); +yenta_free_resource (socket,PCI_CB_BRIDGE_MEM_0_WINDOW ); +yenta_free_resource (socket,PCI_CB_BRIDGE_MEM_1_WINDOW ); }
include/linux/pci.h +10 -1
@@ -101,7 +101,16 @@ enum {#endif /* resources assigned to buses behind the bridge */ -#define PCI_BRIDGE_RESOURCE_NUM 4 +#define PCI_BRIDGE_IO_WINDOW (PCI_BRIDGE_RESOURCES + 0) +#define PCI_BRIDGE_MEM_WINDOW (PCI_BRIDGE_RESOURCES + 1) +#define PCI_BRIDGE_PREF_MEM_WINDOW (PCI_BRIDGE_RESOURCES + 2) + +#define PCI_CB_BRIDGE_IO_0_WINDOW (PCI_BRIDGE_RESOURCES + 0) +#define PCI_CB_BRIDGE_IO_1_WINDOW (PCI_BRIDGE_RESOURCES + 1) +#define PCI_CB_BRIDGE_MEM_0_WINDOW (PCI_BRIDGE_RESOURCES + 2) +#define PCI_CB_BRIDGE_MEM_1_WINDOW (PCI_BRIDGE_RESOURCES + 3) + +#define PCI_BRIDGE_RESOURCE_NUM 4/* max of P2P, cardbus */ PCI_BRIDGE_RESOURCES , PCI_BRIDGE_RESOURCE_END = PCI_BRIDGE_RESOURCES +
Quoted 6 lines · Bjorn Helgaas
These apply on
0148b041be4e7, which is the current head of the linux-next branch of Jesse's pci-2.6 git tree. The first patch is just Jeff Garrett's patch to removeintel_bus.c, so that is only here for people who want to test the rest of the patches. I expect Jesse will pick up Jeff's patch via Linus' tree.
All patches look sane to me. Let's get them merged early in the next merge window, and hope for the best.
Linus
Quoted 27 lines · Bjorn Helgaas
Historically, Linux has assumed a single PCI host bridge, with that bridge claiming all the address space left after RAM and legacy devices are taken out.
If the system contains multiple host bridges, we can no longer operate under that assumption. We have to know what parts of the address space are claimed by each bridge so that when we assign resources to a PCI device, we take them from a range claimed by the upstream host bridge.
We use ACPI to enumerate all the PCI host bridges in the system, and part of the host bridge description is the "
_CRS" (current resource settings" property, which lists the address space used by the bridge. On x86, we currently ignore most of the_CRSinformation. This patch series changes this, so we will use_CRSto learn about the host bridge windows.Since most x86 machines with multiple host bridges are relatively new, this series only turns this on for machines with BIOS dates of 2010 or newer and for a few machines that we know need it.
These apply on
0148b041be4e7, which is the current head of the linux-next branch of Jesse's pci-2.6 git tree. The first patch is just Jeff Garrett's patch to removeintel_bus.c, so that is only here for people who want to test the rest of the patches. I expect Jesse will pick up Jeff's patch via Linus' tree.Gary and Peter have some of these problem machines, so I'm hoping they can give this a whirl.
Quoted 6 lines
Larry, you reported the problem the last time I tried to turn on "pci=use_crs" by default. This series shouldn't affect your machine because it's not in the whitelist, but I expect that if you boot the current kernel with "pci=use_crs", it should still fail, and if you boot with these patches and "pci=use_crs", it *should* work. I know it's a lot to ask, but it'd be great if you had a chance to try that.
On my system, "git describe" returns v2.6.33-rc6-146-gc80d292. Patch 1 does not apply and can be reverted. That is not a problem, but beginning with patch 5, these do not apply.
In addition to the above, my system now boots with "pci=use_crs", unlike when I filed the Bugzilla.
What kernel should I be running to test these patches?
Larry
Quoted 11 lines · Larry Finger
Quoted 6 lines · Bjorn Helgaas
Larry, you reported the problem the last time I tried to turn on "pci=use_crs" by default. This series shouldn't affect your machine because it's not in the whitelist, but I expect that if you boot the current kernel with "pci=use_crs", it should still fail, and if you boot with these patches and "pci=use_crs", it *should* work. I know it's a lot to ask, but it'd be great if you had a chance to try that.
On my system, "git describe" returns v2.6.33-rc6-146-gc80d292. Patch 1 does not apply and can be reverted. That is not a problem, but beginning with patch 5, these do not apply.
Looks like you're using Linus' tree. My patches go on top of Jesse's PCI linux-next tree. Here's how you can do this (assuming you have stgit as well as git):
Save all the patches in files "/tmp/use-crs.1" through "/tmp/use-crs.7". These can be plain email; you don't have to remove headers or anything.
$ cd <git repo> $ git branch $ git fetch git://git.kernel.org/pub/scm/linux/kernel/git/jbarnes/pci-2.6.git linux-next $ stg branch -c use-crs 0148b041be4e7 $ for F in `seq 7`; do stg import -m /tmp/use-crs.$F; done
Now you should have a tree with all the patches applied.
After you're done testing, to return to where you were before, use "git checkout <branch>" where <branch> is the name marked with a "*" from the very first "git branch" command.
Quoted 2 lines
In addition to the above, my system now boots with "pci=use_crs", unlike when I filed the Bugzilla.
Huh. From http://lkml.org/lkml/2009/6/24/11, I had assumed the main problem was that we overflowed the 16-entry bus resource table, but there must be more to it.
If you can build and boot the linux-next branch with my patches and collect the dmesg log, maybe it will have a clue. You can boot without "pci=use_crs"; I don't think that will make any difference on your box.
Bjorn
Quoted 21 lines · Bjorn Helgaas
Quoted 11 lines · Larry Finger
Quoted 6 lines · Bjorn Helgaas
Larry, you reported the problem the last time I tried to turn on "pci=use_crs" by default. This series shouldn't affect your machine because it's not in the whitelist, but I expect that if you boot the current kernel with "pci=use_crs", it should still fail, and if you boot with these patches and "pci=use_crs", it *should* work. I know it's a lot to ask, but it'd be great if you had a chance to try that.
On my system, "git describe" returns v2.6.33-rc6-146-gc80d292. Patch 1 does not apply and can be reverted. That is not a problem, but beginning with patch 5, these do not apply.
Looks like you're using Linus' tree. My patches go on top of Jesse's PCI linux-next tree. Here's how you can do this (assuming you have stgit as well as git):
Save all the patches in files "
/tmp/use-crs.1" through "/tmp/use-crs.7". These can be plain email; you don't have to remove headers or anything.
Quoted 7 lines
$ cd <git repo> $ git branch $ git fetch git://git.kernel.org/pub/scm/linux/kernel/git/jbarnes/pci-2.6.git linux-next $ stg branch -c use-crs 0148b041be4e7 $ for F in `seq 7`; do stg import -m /tmp/use-crs.$F; doneNow you should have a tree with all the patches applied.
That worked. I actually used quilt to apply the patches as I am more familiar with it. Patch #2 was already applied, but the rest applied cleanly.
The patched version of the linux-next kernel booted fine. I put the dmesg output as "Attachment #24914 to bug 14183".
Thanks for the help,
Larry
Quoted 5 lines · Bjorn Helgaas
Quoted 2 lines · Larry Finger
In addition to the above, my system now boots with "pci=use_crs", unlike when I filed the Bugzilla.
Quoted 7 lines
Huh. From http://lkml.org/lkml/2009/6/24/11, I had assumed the main problem was that we overflowed the 16-entry bus resource table, but there must be more to it.
If you can build and boot the linux-next branch with my patches and collect the dmesg log, maybe it will have a clue. You can boot without "pci=use_crs"; I don't think that will make any difference on your box.
Thanks for testing these and collecting the dmesg log
(http://bugzilla.kernel.org/attachment.cgi?id=24914). That log is from the PCI linux-next branch plus my patches, without using "pci=use_crs".
On the current upstream (e.g., the c80d292 kernel you started with), we have PCI_BUS_NUM_RESOURCES == 16. Your _CRS returns 17 windows (the "pci_root PNP0A08:00 host bridge window" lines), so when you boot with "pci=use_crs", we should be discarding the last window, which is an important one:
pci_root PNP0A08:00: host bridge window [mem 0xc0000000-0xfebfffff]
Can you collect the dmesg log from the c80d292 kernel with "pci=use_crs"? I'm sorry to trouble you for this, but it still looks to me like that should fail, so I'd really like to understand why it's working.
Bjorn