pci: only release that resource index is less than 3
PATCH 1/1 pci: only release that resource index is less than 3
after
| commit 308cf8e13f42f476dfd6552aeff58fdc0788e566 | | PCI: get larger bridge ranges when space is available
found one of resource of peer root bus (0x00) get released from root resource. later one hotplug device can not get big range anymore. other peer root buses is ok.
it turns out it is from transparent path.
those resources will be used for pci bridge BAR updated. so need to limit it to 3.
--- drivers/pci/setup-bus.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
drivers/pci/setup-bus.c +7 -2
@@ -344,9 +344,14 @@ static struct resource *find_free_bus_re* if there is no child under that, we should release * and use it. don't need to reset it, pbus_size_* will * set it again +* need to be less 3, otherwise can not write it to +* bridge, also need to avoid releasing it from +* transparent bus path */ -if (!r->child && !release_resource (r)) -return r; +if (i <3 && !r->child) { +if (!release_resource (r)) +return r; + } } }return NULL ;
Quoted 29 lines · Yinghai Lu
after
| commit 308cf8e13f42f476dfd6552aeff58fdc0788e566 | | PCI: get larger bridge ranges when space is availablefound one of resource of peer root bus (
0x00) get released from root resource. later one hotplug device can not get big range anymore. other peer root buses is ok.it turns out it is from transparent path.
those resources will be used for pci bridge BAR updated. so need to limit it to 3.
Signed-off-by Yinghai Lu <[email protected]>--- drivers/pci/setup-bus.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)Index:
linux-2.6/drivers/pci/setup-bus.c
===================================================================drivers/pci/setup-bus.c +0
-0@@ -344,9 +344,14 @@ static struct resource *find_free_bus_re* if there is no child under that, we should release * and use it. don't need to reset it, pbus_size_* will * set it again
drivers/pci/setup-bus.c static struct resource *find_free_bus_re · line 354
Quoted 10 lines
drivers/pci/setup-bus.c +7
-2+* need to be less 3, otherwise can not write it to +* bridge, also need to avoid releasing it from +* transparent bus path */ -if (!r->child && !release_resource (r)) -return r; +if (i <3 && !r->child) { +if (!release_resource (r)) +return r; + }
I am bewildered.
308cf8e13f42f added the release_resource() call here in find_free_bus_resource(). I don't understand why the release should be there in the first place -- it doesn't seem to logically fit there. A "release" should be connected to an event, maybe a hot-remove or a move of a device from one place to another. It shouldn't be something we do as a side-effect of searching for a free resource.
Now you're adding the magic number "3", which seems even less related to the job of "finding an available bus resource." I'm guessing the "3" is related to PCI_BRIDGE_RESOURCES or something, but that should be made explicit, and I really don't think it belongs in this function.
Bjorn
Quoted 55 lines · On Mon, 26 Oct 2009 10:32:57 -0600 Bjorn Helgaas <[email protected]> wrote:
Quoted 35 lines · Yinghai Lu
after
| commit 308cf8e13f42f476dfd6552aeff58fdc0788e566 | | PCI: get larger bridge ranges when space is availablefound one of resource of peer root bus (
0x00) get released from root resource. later one hotplug device can not get big range anymore. other peer root buses is ok.it turns out it is from transparent path.
those resources will be used for pci bridge BAR updated. so need to limit it to 3.
Signed-off-by Yinghai Lu <[email protected]>--- drivers/pci/setup-bus.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)Index:
linux-2.6/drivers/pci/setup-bus.c
===================================================================drivers/pci/setup-bus.c +0
-0@@ -344,9 +344,14 @@ static struct resource *find_free_bus_re* if there is no child under that, we should release
* and use it. don't need to reset it,
pbus_size_* willdrivers/pci/setup-bus.c +1
-0* set it again +* need to be less 3, otherwise can not write it to
drivers/pci/setup-bus.c +1
-0+* bridge, also need to avoid releasing it from
drivers/pci/setup-bus.c static struct resource *find_free_bus_re · line 354
Quoted 8 lines
drivers/pci/setup-bus.c +5
-2+* transparent bus path */ -if (!r->child && !release_resource (r)) -return r; +if (i <3 && !r->child) { +if (!release_resource (r)) +return r; + }I am bewildered.
308cf8e13f42fadded therelease_resource()call here infind_free_bus_resource(). I don't understand why the release should be there in the first place -- it doesn't seem to logically fit there. A "release" should be connected to an event, maybe a hot-remove or a move of a device from one place to another. It shouldn't be something we do as a side-effect of searching for a free resource.
Quoted 5 lines
Now you're adding the magic number "3", which seems even less related to the job of "finding an available bus resource." I'm guessing the "3" is related to
PCI_BRIDGE_RESOURCESor something, but that should be made explicit, and I really don't think it belongs in this function.
I agree, the 3 is a bit of magic. The free could probably be shuffled around a bit too, though it really is related to finding bus resources. I was a bit ambivalent about putting it in find_free_bus_resources but when I started a reply to Yinghai and looked for a better place it made some sense to leave it as-is.
But if we're going to be adding more logic here, we should probably figure out a better way of doing it, maybe by adding a new pass to do the freeing? I know we've avoided modifying bus resources too much in the past, but I think that'll be harder and harder to avoid as Windows moves to that model and platforms begin to expect it.
Signature · 1 line
Jesse Barnes, Intel Open Source Technology Center
Quoted 43 lines · Bjorn Helgaas
Quoted 29 lines · Yinghai Lu
after
| commit 308cf8e13f42f476dfd6552aeff58fdc0788e566 | | PCI: get larger bridge ranges when space is availablefound one of resource of peer root bus (
0x00) get released from root resource. later one hotplug device can not get big range anymore. other peer root buses is ok.it turns out it is from transparent path.
those resources will be used for pci bridge BAR updated. so need to limit it to 3.
Signed-off-by Yinghai Lu <[email protected]>--- drivers/pci/setup-bus.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)Index:
linux-2.6/drivers/pci/setup-bus.c
===================================================================drivers/pci/setup-bus.c +0
-0@@ -344,9 +344,14 @@ static struct resource *find_free_bus_re* if there is no child under that, we should release * and use it. don't need to reset it, pbus_size_* will * set it again drivers/pci/setup-bus.c static struct resource *find_free_bus_re · line 354
Quoted 10 lines
drivers/pci/setup-bus.c +7
-2+* need to be less 3, otherwise can not write it to +* bridge, also need to avoid releasing it from +* transparent bus path */ -if (!r->child && !release_resource (r)) -return r; +if (i <3 && !r->child) { +if (!release_resource (r)) +return r; + }I am bewildered.
Quoted 7 lines
308cf8e13f42fadded therelease_resource()call here infind_free_bus_resource(). I don't understand why the release should be there in the first place -- it doesn't seem to logically fit there. A "release" should be connected to an event, maybe a hot-remove or a move of a device from one place to another. It shouldn't be something we do as a side-effect of searching for a free resource.
system with several IOHs, and one pci bridge under one IOH,
one of device under that bridge request big range 256M, and the bridge can not find range it, so the device BAR is not initialized. and the bridge already some range preallocated ( about 8M ) for future hotplug support.
so need to release the resource used by that bridge, so it could get new one.
Quoted 6 lines
Now you're adding the magic number "3", which seems even less related to the job of "finding an available bus resource." I'm guessing the "3" is related to
PCI_BRIDGE_RESOURCESor something, but that should be made explicit, and I really don't think it belongs in this function.
find_free_bus_resource is only needed by pbus_size_io and bus_size_mem with pci bridge (not card bus) to find position of bridge BAR, so later could update corresponding BAR.
YH
Quoted 69 lines · Jesse Barnes
Quoted 54 lines · On Mon, 26 Oct 2009 10:32:57 -0600 Bjorn Helgaas <[email protected]> wrote:
Quoted 35 lines · Yinghai Lu
after
| commit 308cf8e13f42f476dfd6552aeff58fdc0788e566 | | PCI: get larger bridge ranges when space is availablefound one of resource of peer root bus (
0x00) get released from root resource. later one hotplug device can not get big range anymore. other peer root buses is ok.it turns out it is from transparent path.
those resources will be used for pci bridge BAR updated. so need to limit it to 3.
Signed-off-by Yinghai Lu <[email protected]>--- drivers/pci/setup-bus.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)Index:
linux-2.6/drivers/pci/setup-bus.c
===================================================================drivers/pci/setup-bus.c +0
-0@@ -344,9 +344,14 @@ static struct resource *find_free_bus_re* if there is no child under that, we should release
* and use it. don't need to reset it,
pbus_size_* willdrivers/pci/setup-bus.c +1
-0* set it again +* need to be less 3, otherwise can not write it to
drivers/pci/setup-bus.c +1
-0+* bridge, also need to avoid releasing it from
drivers/pci/setup-bus.c
Quoted 8 lines
drivers/pci/setup-bus.c +5
-2+* transparent bus path */ -if (!r->child && !release_resource (r)) -return r; +if (i <3 && !r->child) { +if (!release_resource (r)) +return r; + }I am bewildered.
308cf8e13f42fadded therelease_resource()call here infind_free_bus_resource(). I don't understand why the release should be there in the first place -- it doesn't seem to logically fit there. A "release" should be connected to an event, maybe a hot-remove or a move of a device from one place to another. It shouldn't be something we do as a side-effect of searching for a free resource.Quoted 5 lines
Now you're adding the magic number "3", which seems even less related to the job of "finding an available bus resource." I'm guessing the "3" is related to
PCI_BRIDGE_RESOURCESor something, but that should be made explicit, and I really don't think it belongs in this function.I agree, the 3 is a bit of magic. The free could probably be shuffled around a bit too, though it really is related to finding bus resources. I was a bit ambivalent about putting it in
find_free_bus_resourcesbut when I started a reply to Yinghai and looked for a better place it made some sense to leave it as-is.
Quoted 5 lines
But if we're going to be adding more logic here, we should probably figure out a better way of doing it, maybe by adding a new pass to do the freeing? I know we've avoided modifying bus resources too much in the past, but I think that'll be harder and harder to avoid as Windows moves to that model and platforms begin to expect it.
not sure if you like this version. it need more lines changes.
[PATCH] pci: only release that resource index is less than 3 -v2
after
| commit 308cf8e13f42f476dfd6552aeff58fdc0788e566 | | PCI: get larger bridge ranges when space is available
found one of resource of peer root bus (0x00) get released from root resource. later one hotplug device can not get big range anymore. other peer root buses is ok.
it turns out it is from transparent path.
those resources will be used for pci bridge BAR updated. so need to limit it to 3.
v2: Jesse doesn't like it is in find_free_bus_resource...
try to move out of pci_bus_size_bridges loop.
need to apply after:
[PATCH] pci: pciehp update the slot bridge res to get big range for pcie devices - v3
--- drivers/pci/hotplug/pciehp_pci.c | 1 drivers/pci/setup-bus.c | 83 +++++++++++++++++++++++++++++++++------ include/linux/pci.h | 1 3 files changed, 74 insertions(+), 11 deletions(-)
drivers/pci/setup-bus.c +72 -11
@@ -322,33 +322,54 @@ static void pci_bridge_check_ranges(stru } } -/* Helper function for sizing routines: find first available -bus resource of a given type. Note: we intentionally skip -the bus resources which have already been assigned (that is, -have non-NULL parent resource). */ -static struct resource *find_free_bus_resource (struct pci_bus *bus,unsigned long type) +void pci_bridge_release_not_used_res (struct pci_bus *bus) {int i;struct resource *r;unsigned long type_mask =IORESOURCE_IO |IORESOURCE_MEM |IORESOURCE_PREFETCH ; -for (i =0 ; i <PCI_BUS_NUM_RESOURCES ; i++) { +/* for pci bridges res only */ +for (i =0 ; i <3 ; i++) { r = bus->resource[i]; -if (r == &ioport_resource || r == &iomem_resource) -continue ; -if (r && (r->flags & type_mask) == type) { +if (r && (r->flags & type_mask)) {if (!r->parent) -return r; +continue ;/* * if there is no child under that, we should release * and use it. don't need to reset it, pbus_size_* will * set it again */ if (!r->child && !release_resource (r)) -return r; +dev_printk (KERN_DEBUG , &bus->dev, +"resource %d %s %pR released\n" , i, + (r->flags &IORESOURCE_IO ) ?"io: " : + ((r->flags &IORESOURCE_PREFETCH )? +"pref mem" :"mem:" ), + r); } } +} +EXPORT_SYMBOL (pci_bridge_release_not_used_res); + +/* Helper function for sizing routines: find first available +bus resource of a given type. Note: we intentionally skip +the bus resources which have already been assigned (that is, +have non-NULL parent resource). */ +static struct resource *find_free_bus_resource (struct pci_bus *bus,unsigned long type) +{ +int i; +struct resource *r; +unsigned long type_mask =IORESOURCE_IO |IORESOURCE_MEM | +IORESOURCE_PREFETCH ; + +for (i =0 ; i <PCI_BUS_NUM_RESOURCES ; i++) { + r = bus->resource[i]; +if (r == &ioport_resource || r == &iomem_resource) +continue ; +if (r && (r->flags & type_mask) == type && !r->parent) +return r; + }return NULL ; }
@@ -588,6 +609,41 @@ void __ref pci_bus_size_bridges(struct p }EXPORT_SYMBOL (pci_bus_size_bridges); +static void pci_bus_release_bridges_not_used_res (struct pci_bus *bus) +{ +struct pci_dev *dev; + +list_for_each_entry (dev, &bus->devices, bus_list) { +struct pci_bus *b = dev->subordinate; +if (!b) +continue ; + +switch (dev->class >>8 ) { +case PCI_CLASS_BRIDGE_CARDBUS : +break ; + +case PCI_CLASS_BRIDGE_PCI : +default : +pci_bus_release_bridges_not_used_res (b); +break ; + } + } + +/* The root bus? */ +if (!bus->self) +return ; + +switch (bus->self->class >>8 ) { +case PCI_CLASS_BRIDGE_CARDBUS : +break ; + +case PCI_CLASS_BRIDGE_PCI : +default : +pci_bridge_release_not_used_res (bus); +break ; + } +} +void __ref pci_bridge_assign_resources (const struct pci_dev *bridge) {struct pci_bus *b;
@@ -687,6 +743,11 @@ pci_assign_unassigned_resources(void) {struct pci_bus *bus; +/* Try to release bridge resources, that there is not child under it */ +list_for_each_entry (bus, &pci_root_buses, node) { +pci_bus_release_bridges_not_used_res (bus); + } +/* Depth first, calculate sizes and alignments of all subordinate buses. */ list_for_each_entry (bus, &pci_root_buses, node) {
drivers/pci/hotplug/pciehp_pci.c +1 -0
@@ -171,6 +171,7 @@ int pciehp_unconfigure_device(struct slo }pci_dev_put (temp); } +pci_bridge_release_not_used_res (parent);return rc; }
include/linux/pci.h +1 -0
@@ -759,6 +759,7 @@ int pci_vpd_truncate(struct pci_dev *devvoid pci_bridge_assign_resources (const struct pci_dev *bridge);void pci_bus_assign_resources (const struct pci_bus *bus);void pci_bus_size_bridges (struct pci_bus *bus); +void pci_bridge_release_not_used_res (struct pci_bus *bus);int pci_claim_resource (struct pci_dev *,int );void pci_assign_unassigned_resources (void );void pdev_enable_device (struct pci_dev *);
Quoted 12 lines · Yinghai Lu
===================================================================
drivers/pci/setup-bus.c +0
-5@@ -322,33 +322,54 @@ static void pci_bridge_check_ranges(stru } } -/* Helper function for sizing routines: find first available -bus resource of a given type. Note: we intentionally skip -the bus resources which have already been assigned (that is, -have non-NULL parent resource). */ -static struct resource *find_free_bus_resource (struct pci_bus *bus,unsigned long type)
drivers/pci/setup-bus.c static void pci_bridge_check_ranges(stru · line 333
Quoted 10 lines
drivers/pci/setup-bus.c +3
-1+void pci_bridge_release_not_used_res (struct pci_bus *bus) {int i;struct resource *r;unsigned long type_mask =IORESOURCE_IO |IORESOURCE_MEM |IORESOURCE_PREFETCH ; -for (i =0 ; i <PCI_BUS_NUM_RESOURCES ; i++) { +/* for pci bridges res only */ +for (i =0 ; i <3 ; i++) {
I think the "i = 0; i < 3" part is to check the bridge apertures, i.e., the I/O base/limit, the memory base/limit, and the prefetchable memory base/limit. Right? We need some way to indicate that more clearly than using a hard-coded "3".
Quoted 4 lines
drivers/pci/setup-bus.c +0
-3r = bus->resource[i]; -if (r == &ioport_resource || r == &iomem_resource) -continue ; -if (r && (r->flags & type_mask) == type) {
drivers/pci/setup-bus.c static void pci_bridge_check_ranges(stru · line 343
Quoted 10 lines
drivers/pci/setup-bus.c +2
-1+if (r && (r->flags & type_mask)) {if (!r->parent) -return r; +continue ;/* * if there is no child under that, we should release * and use it. don't need to reset it, pbus_size_* will * set it again */ if (!r->child && !release_resource (r))
We got this resource pointer out of a struct pci_bus, and we release it here. We must have previously done a request_resource(), allocate_resource(), or similar. Where does that happen? Are the requests and releases nested correctly?
I would think that somewhere, we would be doing a request_resource() and assigning the resource to pci_bus->resource[x]. But there are very few assignments to the pci_bus resources:
setup_resource (only for "pci=use_crs") pci_read_bridge_bases (just a copy from upstream bus resources) pci_alloc_child_bus (copy from upstream bridge resources) pci_create_bus (set to &ioport_resource or &iomem_resource)
My guess is that this release_resource() releases something we copied from the bridge in pci_alloc_child_bus(). But that doesn't seem right, because we aren't changing the bridge programming here.
Maybe I'm being confused by the fact that we find the resource pointer from the pci_bus table, but that's only a pointer to the actual struct resource that lives in the pci_dev of the upstream bridge.
Bjorn
Quoted 111 lines
drivers/pci/setup-bus.c +67
-1-return r; +dev_printk (KERN_DEBUG , &bus->dev, +"resource %d %s %pR released\n" , i, + (r->flags &IORESOURCE_IO ) ?"io: " : + ((r->flags &IORESOURCE_PREFETCH )? +"pref mem" :"mem:" ), + r); } } +} +EXPORT_SYMBOL (pci_bridge_release_not_used_res); + +/* Helper function for sizing routines: find first available +bus resource of a given type. Note: we intentionally skip +the bus resources which have already been assigned (that is, +have non-NULL parent resource). */ +static struct resource *find_free_bus_resource (struct pci_bus *bus,unsigned long type) +{ +int i; +struct resource *r; +unsigned long type_mask =IORESOURCE_IO |IORESOURCE_MEM | +IORESOURCE_PREFETCH ; + +for (i =0 ; i <PCI_BUS_NUM_RESOURCES ; i++) { + r = bus->resource[i]; +if (r == &ioport_resource || r == &iomem_resource) +continue ; +if (r && (r->flags & type_mask) == type && !r->parent) +return r; + }return NULL ; }@@ -588,6 +609,41 @@ void __ref pci_bus_size_bridges(struct p }EXPORT_SYMBOL (pci_bus_size_bridges); +static void pci_bus_release_bridges_not_used_res (struct pci_bus *bus) +{ +struct pci_dev *dev; + +list_for_each_entry (dev, &bus->devices, bus_list) { +struct pci_bus *b = dev->subordinate; +if (!b) +continue ; + +switch (dev->class >>8 ) { +case PCI_CLASS_BRIDGE_CARDBUS : +break ; + +case PCI_CLASS_BRIDGE_PCI : +default : +pci_bus_release_bridges_not_used_res (b); +break ; + } + } + +/* The root bus? */ +if (!bus->self) +return ; + +switch (bus->self->class >>8 ) { +case PCI_CLASS_BRIDGE_CARDBUS : +break ; + +case PCI_CLASS_BRIDGE_PCI : +default : +pci_bridge_release_not_used_res (bus); +break ; + } +} +void __ref pci_bridge_assign_resources (const struct pci_dev *bridge) {struct pci_bus *b;@@ -687,6 +743,11 @@ pci_assign_unassigned_resources(void) {struct pci_bus *bus; +/* Try to release bridge resources, that there is not child under it */ +list_for_each_entry (bus, &pci_root_buses, node) { +pci_bus_release_bridges_not_used_res (bus); + } +/* Depth first, calculate sizes and alignments of all subordinate buses. */ list_for_each_entry (bus, &pci_root_buses, node) {Index:
linux-2.6/drivers/pci/hotplug/pciehp_pci.c
===================================================================drivers/pci/hotplug/pciehp_pci.c +1
-0@@ -171,6 +171,7 @@ int pciehp_unconfigure_device(struct slo }pci_dev_put (temp); } +pci_bridge_release_not_used_res (parent);return rc; }Index:
linux-2.6/include/linux/pci.h
===================================================================include/linux/pci.h +1
-0@@ -759,6 +759,7 @@ int pci_vpd_truncate(struct pci_dev *devvoid pci_bridge_assign_resources (const struct pci_dev *bridge);void pci_bus_assign_resources (const struct pci_bus *bus);void pci_bus_size_bridges (struct pci_bus *bus); +void pci_bridge_release_not_used_res (struct pci_bus *bus);int pci_claim_resource (struct pci_dev *,int );void pci_assign_unassigned_resources (void );void pdev_enable_device (struct pci_dev *);
Quoted 21 lines · Bjorn Helgaas
Quoted 19 lines · Yinghai Lu
===================================================================
drivers/pci/setup-bus.c +1
-5@@ -322,33 +322,54 @@ static void pci_bridge_check_ranges(stru } } -/* Helper function for sizing routines: find first available -bus resource of a given type. Note: we intentionally skip -the bus resources which have already been assigned (that is, -have non-NULL parent resource). */ -static struct resource *find_free_bus_resource (struct pci_bus *bus,unsigned long type) +void pci_bridge_release_not_used_res (struct pci_bus *bus) {int i;struct resource *r;unsigned long type_mask =IORESOURCE_IO |IORESOURCE_MEM |IORESOURCE_PREFETCH ;
Quoted 8 lines
drivers/pci/setup-bus.c static void pci_bridge_check_ranges(stru · line 333
Quoted 3 lines
drivers/pci/setup-bus.c +2
-1-for (i =0 ; i <PCI_BUS_NUM_RESOURCES ; i++) { +/* for pci bridges res only */ +for (i =0 ; i <3 ; i++) {I think the "i = 0; i < 3" part is to check the bridge apertures, i.e., the I/O base/limit, the memory base/limit, and the prefetchable memory base/limit. Right? We need some way to indicate that more clearly than using a hard-coded "3".
yes. the 0x1c, 0x20, and 0x28
Quoted 29 lines
Quoted 5 lines
drivers/pci/setup-bus.c +0
-3r = bus->resource[i]; -if (r == &ioport_resource || r == &iomem_resource) -continue ; -if (r && (r->flags & type_mask) == type) {drivers/pci/setup-bus.c static void pci_bridge_check_ranges(stru · line 343
Quoted 10 lines
drivers/pci/setup-bus.c +2
-1+if (r && (r->flags & type_mask)) {if (!r->parent) -return r; +continue ;/* * if there is no child under that, we should release * and use it. don't need to reset it, pbus_size_* will * set it again */ if (!r->child && !release_resource (r))We got this resource pointer out of a struct
pci_bus, and we release it here. We must have previously done arequest_resource(),allocate_resource(), or similar. Where does that happen? Are the requests and releases nested correctly?I would think that somewhere, we would be doing a
request_resource()and assigning the resource to pci_bus->resource[x]. But there are very few assignments to thepci_busresources:
setup_resource(only for "pci=use_crs")pci_read_bridge_bases(just a copy from upstream bus resources)pci_alloc_child_bus(copy from upstream bridge resources)pci_create_bus(set to&ioport_resourceor&iomem_resource)
Quoted 3 lines
My guess is that this
release_resource()releases something we copied from the bridge inpci_alloc_child_bus(). But that doesn't seem right, because we aren't changing the bridge programming here.
in arch/x86/pci/i386.c::pcibios_resource_survey() ==> pcibios_allocate_bus_resources()
YH
Quoted 31 lines · Yinghai Lu
Quoted 21 lines · Bjorn Helgaas
Quoted 19 lines · Yinghai Lu
===================================================================
drivers/pci/setup-bus.c +1
-5@@ -322,33 +322,54 @@ static void pci_bridge_check_ranges(stru } } -/* Helper function for sizing routines: find first available -bus resource of a given type. Note: we intentionally skip -the bus resources which have already been assigned (that is, -have non-NULL parent resource). */ -static struct resource *find_free_bus_resource (struct pci_bus *bus,unsigned long type) +void pci_bridge_release_not_used_res (struct pci_bus *bus) {int i;struct resource *r;unsigned long type_mask =IORESOURCE_IO |IORESOURCE_MEM |IORESOURCE_PREFETCH ;Quoted 8 lines
drivers/pci/setup-bus.c static void pci_bridge_check_ranges(stru · line 333
Quoted 3 lines
drivers/pci/setup-bus.c +2
-1-for (i =0 ; i <PCI_BUS_NUM_RESOURCES ; i++) { +/* for pci bridges res only */ +for (i =0 ; i <3 ; i++) {I think the "i = 0; i < 3" part is to check the bridge apertures, i.e., the I/O base/limit, the memory base/limit, and the prefetchable memory base/limit. Right? We need some way to indicate that more clearly than using a hard-coded "3".
Quoted 1 line
yes. the
0x1c,0x20, and0x28
Do you have a proposal for something other than "3"? The magic number "3" does appear other places, e.g., pci_read_bridge_bases() but I don't think we should add new uses of it. Make up a descriptive #define if nothing else.
Quoted 28 lines
Quoted 28 lines
Quoted 4 lines
drivers/pci/setup-bus.c +0
-3r = bus->resource[i]; -if (r == &ioport_resource || r == &iomem_resource) -continue ; -if (r && (r->flags & type_mask) == type) {drivers/pci/setup-bus.c static void pci_bridge_check_ranges(stru · line 343
Quoted 10 lines
drivers/pci/setup-bus.c +2
-1+if (r && (r->flags & type_mask)) {if (!r->parent) -return r; +continue ;/* * if there is no child under that, we should release * and use it. don't need to reset it, pbus_size_* will * set it again */ if (!r->child && !release_resource (r))We got this resource pointer out of a struct
pci_bus, and we release it here. We must have previously done arequest_resource(),allocate_resource(), or similar. Where does that happen? Are the requests and releases nested correctly?I would think that somewhere, we would be doing a
request_resource()and assigning the resource to pci_bus->resource[x]. But there are very few assignments to thepci_busresources:
setup_resource(only for "pci=use_crs")pci_read_bridge_bases(just a copy from upstream bus resources)pci_alloc_child_bus(copy from upstream bridge resources)pci_create_bus(set to&ioport_resourceor&iomem_resource)
Quoted 6 lines
Quoted 3 lines
My guess is that this
release_resource()releases something we copied from the bridge inpci_alloc_child_bus(). But that doesn't seem right, because we aren't changing the bridge programming here.in
arch/x86/pci/i386.c::pcibios_resource_survey()==>pcibios_allocate_bus_resources()
The asymmetry here bothers me. The pci_claim_resource() in pcibios_allocate_bus_resources() is done on a *device*, while the release_resource() you added is done on a *bus*. And the claim is done in arch-specific code, while the release is done in generic code. And there's an interval where the bridge device resources don't match the hardware apertures, and it's hard to figure out when they are in sync again.
This might all work fine, but it's harder to understand than it should be.
Bjorn
[PATCH 3/4] pci: only release that resource index is less than 3 -v4
after
| commit 308cf8e13f42f476dfd6552aeff58fdc0788e566 | | PCI: get larger bridge ranges when space is available
found one of resource of peer root bus (0x00) get released from root resource. later one hotplug device can not get big range anymore. other peer root buses is ok.
it turns out it is from transparent path.
those resources will be used for pci bridge BAR updated. so need to limit it to 3.
v2: Jesse doesn't like it is in find_free_bus_resource...
try to move out of pci_bus_size_bridges loop.
need to apply after:
[PATCH] pci: pciehp update the slot bridge res to get big range for pcie devices - v4
v3: add pci_setup_bridge calling after pci_bridge_release_not_used_res.
only clear release those res for x86.
v4: Bjorn want to release use dev instead of bus.
--- arch/x86/pci/i386.c | 6 +++ drivers/pci/hotplug/pciehp_pci.c | 2 + drivers/pci/setup-bus.c | 75 ++++++++++++++++++++++++++++++++++++++- include/linux/pci.h | 2 + 4 files changed, 84 insertions(+), 1 deletion(-)
drivers/pci/setup-bus.c +74 -1
@@ -319,6 +319,42 @@ static void pci_bridge_check_ranges(stru } } +void pci_bridge_release_not_used_res (struct pci_bus *bus) +{ +int idx; +bool changed = false; +struct pci_dev *dev; +struct resource *r; +unsigned long type_mask =IORESOURCE_IO |IORESOURCE_MEM | +IORESOURCE_PREFETCH ; + +/* for pci bridges res only */ + dev = bus->self; +for (idx =PCI_BRIDGE_RESOURCES ; idx <PCI_BRIDGE_RESOURCES +3 ; + idx++) { + r = &dev->resource[idx]; +if (r->flags & type_mask) { +if (!r->parent) +continue ; +/* +* if there is no child under that, we should release +* and use it. +*/ +if (!r->child && !release_resource (r)) { +dev_info (&dev->dev, +"resource %d %pRt released\n" , + idx, r); + r->flags =0 ; + changed = true; + } + } + } + +if (changed) +pci_setup_bridge (bus); +} +EXPORT_SYMBOL (pci_bridge_release_not_used_res); +/* Helper function for sizing routines: find first available bus resource of a given type. Note: we intentionally skip the bus resources which have already been assigned (that is,
@@ -576,6 +612,42 @@ void __ref pci_bus_size_bridges(struct p }EXPORT_SYMBOL (pci_bus_size_bridges); +void __ref pci_bus_release_bridges_not_used_res (struct pci_bus *bus) +{ +struct pci_dev *dev; + +list_for_each_entry (dev, &bus->devices, bus_list) { +struct pci_bus *b = dev->subordinate; +if (!b) +continue ; + +switch (dev->class >>8 ) { +case PCI_CLASS_BRIDGE_CARDBUS : +break ; + +case PCI_CLASS_BRIDGE_PCI : +default : +pci_bus_release_bridges_not_used_res (b); +break ; + } + } + +/* The root bus? */ +if (!bus->self) +return ; + +switch (bus->self->class >>8 ) { +case PCI_CLASS_BRIDGE_CARDBUS : +break ; + +case PCI_CLASS_BRIDGE_PCI : +default : +pci_bridge_release_not_used_res (bus); +break ; + } +} +EXPORT_SYMBOL (pci_bus_release_bridges_not_used_res); +void __ref pci_bridge_assign_resources (const struct pci_dev *bridge) {struct pci_bus *b;
@@ -644,7 +716,8 @@ static void pci_bus_dump_res(struct pci_for (i =0 ; i <PCI_BUS_NUM_RESOURCES ; i++) {struct resource *res = bus->resource[i]; -if (!res || !res->end) + +if (!res || !res->end || !res->flags)continue ;dev_printk (KERN_DEBUG , &bus->dev,"resource %d %pRt\n" , i, res);
drivers/pci/hotplug/pciehp_pci.c +2 -0
@@ -98,6 +98,7 @@ int pciehp_configure_device(struct slotpci_dev_put (dev); } +pci_bridge_release_not_used_res (parent);pci_bus_size_bridges (parent);pci_clear_master (bridge);pci_bridge_assign_resources (bridge);
@@ -171,6 +172,7 @@ int pciehp_unconfigure_device(struct slo }pci_dev_put (temp); } +pci_bridge_release_not_used_res (parent);return rc; }
include/linux/pci.h +2 -0
@@ -759,6 +759,8 @@ int pci_vpd_truncate(struct pci_dev *devvoid pci_bridge_assign_resources (const struct pci_dev *bridge);void pci_bus_assign_resources (const struct pci_bus *bus);void pci_bus_size_bridges (struct pci_bus *bus); +void pci_bus_release_bridges_not_used_res (struct pci_bus *bus); +void pci_bridge_release_not_used_res (struct pci_bus *bus);int pci_claim_resource (struct pci_dev *,int );void pci_assign_unassigned_resources (void );void pdev_enable_device (struct pci_dev *);
arch/x86/pci/i386.c +6 -0
@@ -194,6 +194,7 @@ static void __init pcibios_allocate_resostatic int __init pcibios_assign_resources (void ) {struct pci_dev *dev =NULL ; +struct pci_bus *bus;struct resource *r;if (!(pci_probe &PCI_ASSIGN_ROMS )) {
@@ -213,6 +214,11 @@ static int __init pcibios_assign_resourc } } +/* Try to release bridge resources, that there is not child under it */ +list_for_each_entry (bus, &pci_root_buses, node) { +pci_bus_release_bridges_not_used_res (bus); + } +pci_assign_unassigned_resources ();return 0 ;