PCI: iproc: Use the EROM outbound window on BCMA
PATCH v3 1/1 PCI: iproc: Use the EROM outbound window on BCMA
The PCIe outbound window base on Northstar depends on the PCIe Gen2 core revision. Revision 0x01 uses 0x08000000, 0x40000000 and 0x48000000 for controllers 0 to 2, while revision 0x07 (NS-B0) uses 0x08000000, 0x20000000 and 0x28000000. Broadcom's own driver branches on the core revision for exactly this reason.
bcm-ns.dtsi is shared by every Northstar SoC, so it cannot carry a value that is correct on both. Commit 767012397976 ("ARM: dts: BCM5301X:
Describe PCIe controllers fully") gave the controllers a ranges property. The commit shipped in v7.1.
With that property present, two things go wrong with this driver:
- devm_pci_alloc_host_bridge() parses those ranges and requests them, then this driver adds its own window and requests the whole list a second time, so every controller fails to probe with -EBUSY.
- The DT window itself is only correct on core revision 0x07. On revision 0x01 it points at an address the hardware does not decode, and the first MMIO access to a BAR takes an imprecise external abort.
The enumeration ROM reports the correct base for the revision actually present, and bcma already provides it as addr_s[0]. Drop any memory window that came from the device tree and use that instead, requesting only the window this driver owns. This makes the driver correct whether or not the DT describes a window.
When a dropped window does not match what the EROM reports, print a warning naming both. The mismatch means the devicetree describes a window the hardware does not decode, and that should be fixed in the dts rather than ignored silently.
The same commit also added compatible = "brcm,iproc-pcie", so these nodes now match pcie-iproc-platform. With CONFIG_PCIE_IPROC_PLATFORM enabled, which is the default on ARCH_BCM_IPROC, that driver binds them first and this driver's probe fails inside devm_pci_alloc_host_bridge(). This patch fixes the configurations where the BCMA driver is the one in use; OpenWrt builds that way, with PCIE_IPROC_PLATFORM disabled. The platform path takes the DT window as-is and has the same wrong address on core revision 0x01, so that side needs a devicetree fix either way.
Tested on an ASUS RT-N18U (BCM47081) and a Linksys EA9200 (BCM4709), both core revision 0x01.
---
v2 -> v3: format the two problem descriptions as bullet points. Requested by Bjorn Helgaas.
v1 -> v2: print a warning for every devicetree memory window that does not match the EROM window. Requested by Arnd Bergmann:
https://lore.kernel.org/all/[email protected]/
v2: https://lore.kernel.org/all/[email protected]/
v1: https://lore.kernel.org/all/[email protected]/
drivers/pci/controller/pcie-iproc-bcma.c | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-)
drivers/pci/controller/pcie-iproc-bcma.c +16 -1
@@ -36,6 +36,7 @@ static int iproc_bcma_pcie_probe(struct bcma_device *bdev)struct device *dev = &bdev->dev;struct iproc_pcie *pcie;struct pci_host_bridge *bridge; +struct resource_entry *win, *tmp;int ret; bridge =devm_pci_alloc_host_bridge (dev,sizeof (*pcie));
@@ -59,8 +60,22 @@ static int iproc_bcma_pcie_probe(struct bcma_device *bdev) pcie->mem.end = bdev->addr_s[0 ] +SZ_128M -1 ; pcie->mem.name ="PCIe MEM space" ; pcie->mem.flags =IORESOURCE_MEM ; + +resource_list_for_each_entry_safe (win, tmp, &bridge->windows) { +if (resource_type (win->res) !=IORESOURCE_MEM ) +continue ; + +if (win->res->start != pcie->mem.start || + win->res->end != pcie->mem.end) +dev_warn (dev,"DT window %pR does not match EROM window %pR, using EROM\n" , + win->res, &pcie->mem); + +devm_release_resource (dev, win->res); +resource_list_destroy_entry (win); + } +pci_add_resource (&bridge->windows, &pcie->mem); - ret =devm_request_pci_bus_resources (dev, &bridge->windows); + ret =devm_request_resource (dev, &iomem_resource, &pcie->mem);if (ret)return ret;
Signature · 1 line
2.43.0
[email protected]
AI reviewSashiko has reviewed this patch and found no issues. It looks great!
Quoted 13 lines
The PCIe outbound window base on Northstar depends on the PCIe Gen2 core revision. Revision
0x01uses0x08000000,0x40000000and0x48000000for controllers 0 to 2, while revision0x07(NS-B0) uses0x08000000,0x20000000and0x28000000. Broadcom's own driver branches on the core revision for exactly this reason.
bcm-ns.dtsiis shared by every Northstar SoC, so it cannot carry a value that is correct on both. Commit 767012397976 ("ARM: dts: BCM5301X:
Describe PCIe controllers fully") gave the controllers a ranges property. The commit shipped in v7.1.With that property present, two things go wrong with this driver:
Quoted 8 lines
-
devm_pci_alloc_host_bridge()parses those ranges and requests them, then this driver adds its own window and requests the whole list a
[ ... ]Fixes 767012397976 ("ARM: dts: BCM5301X: Describe PCIe controllers fully")Tested-by Rani Hod <[email protected]>Cc [email protected] # v7.1+Signed-off-by Semih Baskan <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Manivannan Sadhasivam
AppliedQuoted 6 lines · Semih Baskan
The PCIe outbound window base on Northstar depends on the PCIe Gen2 core revision. Revision
0x01uses0x08000000,0x40000000and0x48000000for controllers 0 to 2, while revision0x07(NS-B0) uses0x08000000,0x20000000and0x28000000. Broadcom's own driver branches on the core revision for exactly this reason.
Quoted 6 lines
bcm-ns.dtsiis shared by every Northstar SoC, so it cannot carry a value that is correct on both. Commit 767012397976 ("ARM: dts: BCM5301X:
Describe PCIe controllers fully") gave the controllers a ranges property. The commit shipped in v7.1.[...]
Applied, thanks!
[1/1] PCI: iproc: Use the EROM outbound window on BCMA
commit: 552aa843e4c5aa92012d8c55df8471e6c5ba621c
Best regards,
Signature · 1 line
மணிவண்ணன் சதாசிவம்
[+to Rafał; probably same as [email protected], but MAINTAINERS lists [email protected] for BCMA]
Quoted 86 lines · Semih Baskan
The PCIe outbound window base on Northstar depends on the PCIe Gen2 core revision. Revision
0x01uses0x08000000,0x40000000and0x48000000for controllers 0 to 2, while revision0x07(NS-B0) uses0x08000000,0x20000000and0x28000000. Broadcom's own driver branches on the core revision for exactly this reason.
bcm-ns.dtsiis shared by every Northstar SoC, so it cannot carry a value that is correct on both. Commit 767012397976 ("ARM: dts: BCM5301X:
Describe PCIe controllers fully") gave the controllers a ranges property. The commit shipped in v7.1.With that property present, two things go wrong with this driver:
-
devm_pci_alloc_host_bridge()parses those ranges and requests them, then this driver adds its own window and requests the whole list a second time, so every controller fails to probe with-EBUSY.- The DT window itself is only correct on core revision
0x07. On revision0x01it points at an address the hardware does not decode, and the first MMIO access to a BAR takes an imprecise external abort.The enumeration ROM reports the correct base for the revision actually present, and bcma already provides it as addr_s[0]. Drop any memory window that came from the device tree and use that instead, requesting only the window this driver owns. This makes the driver correct whether or not the DT describes a window.
When a dropped window does not match what the EROM reports, print a warning naming both. The mismatch means the devicetree describes a window the hardware does not decode, and that should be fixed in the dts rather than ignored silently.
The same commit also added compatible = "brcm,iproc-pcie", so these nodes now match pcie-iproc-platform. With
CONFIG_PCIE_IPROC_PLATFORMenabled, which is the default onARCH_BCM_IPROC, that driver binds them first and this driver's probe fails insidedevm_pci_alloc_host_bridge(). This patch fixes the configurations where the BCMA driver is the one in use; OpenWrt builds that way, withPCIE_IPROC_PLATFORMdisabled. The platform path takes the DT window as-is and has the same wrong address on core revision0x01, so that side needs a devicetree fix either way.Tested on an ASUS RT-N18U (BCM47081) and a Linksys EA9200 (BCM4709), both core revision
0x01.Fixes 767012397976 ("ARM: dts: BCM5301X: Describe PCIe controllers fully")Tested-by Rani Hod <[email protected]>Cc [email protected] # v7.1+Signed-off-by Semih Baskan <[email protected]>---
v2 -> v3: format the two problem descriptions as bullet points. Requested by Bjorn Helgaas.v1 -> v2: print a warning for every devicetree memory window that does not match the EROM window. Requested by Arnd Bergmann:
https://lore.kernel.org/all/[email protected]/v2: https://lore.kernel.org/all/[email protected]/
v1: https://lore.kernel.org/all/[email protected]/drivers/pci/controller/pcie-iproc-bcma.c | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-)drivers/pci/controller/pcie-iproc-bcma.c +8
-0@@ -36,6 +36,7 @@ static int iproc_bcma_pcie_probe(struct bcma_device *bdev)struct device *dev = &bdev->dev;struct iproc_pcie *pcie;struct pci_host_bridge *bridge; +struct resource_entry *win, *tmp;int ret; bridge =devm_pci_alloc_host_bridge (dev,sizeof (*pcie));@@ -59,8 +60,22 @@ static int iproc_bcma_pcie_probe(struct bcma_device *bdev) pcie->mem.end = bdev->addr_s[0 ] +SZ_128M -1 ; pcie->mem.name ="PCIe MEM space" ; pcie->mem.flags =IORESOURCE_MEM ; + +resource_list_for_each_entry_safe (win, tmp, &bridge->windows) { +if (resource_type (win->res) !=IORESOURCE_MEM ) +continue ; + +if (win->res->start != pcie->mem.start || + win->res->end != pcie->mem.end)
drivers/pci/controller/pcie-iproc-bcma.c static int iproc_bcma_pcie_probe(struct bcma_device *bdev) · line 78
Quoted 10 lines
drivers/pci/controller/pcie-iproc-bcma.c +8
-1+dev_warn (dev,"DT window %pR does not match EROM window %pR, using EROM\n" , + win->res, &pcie->mem); + +devm_release_resource (dev, win->res); +resource_list_destroy_entry (win); + } +pci_add_resource (&bridge->windows, &pcie->mem); - ret =devm_request_pci_bus_resources (dev, &bridge->windows); + ret =devm_request_resource (dev, &iomem_resource, &pcie->mem);
If the EROM supplies the information the driver needs, why does this BCMA driver use DT at all?
AFAICS, pcie-iproc-bcma.c doesn't use any of the information from DT. devm_pci_alloc_host_bridge() *looks* in DT, parses it, and fills in bridge->windows, and requests those windows, but I don't think iproc-bcma ever uses them.
iproc_bcma_pcie_probe() passes &bridge->windows to iproc_pcie_setup(), where I think it's ignored because pcie->need_ob_cfg is never set for iproc_bcma.
It looks like iproc_bcma_pcie_probe() should just call pci_alloc_host_bridge() directly and skip devm_pci_alloc_host_bridge() and the DT things it does.
Quoted 6 lines
drivers/pci/controller/pcie-iproc-bcma.c +0
-0if (ret)return ret;Signature · 1 line
2.43.0
Quoted 4 lines · Bjorn Helgaas
If the EROM supplies the information the driver needs, why does this BCMA driver use DT at all?
AFAICS,
pcie-iproc-bcma.cdoesn't use any of the information from DT.
It doesn't. need_ob_cfg is set only in pcie-iproc-platform.c, so the windows devm_pci_alloc_host_bridge() parses never reach iproc_pcie_map_ranges() here; the applied patch only had to get them out of the way of the EROM window.
The follow-up keeps bridge->dev.parent set by hand, because the wifi nodes under pcie_bridge0 in bcm4709-netgear-r8000.dts are resolved through the root bus, and frees the bridge itself on the error paths and in remove(). The one DT dependency left is the IRQ, and that goes through bcma_core_irq(), not through this driver.
The follow-up is in reply to this mail: pci_alloc_host_bridge(), nothing from DT requested or handed to the PCI core, request_resource() and release_resource() instead of devm since pcie->mem sits inside the bridge allocation. The ranges property is still read, but only to compare it with the EROM and warn. That is the warning Arnd asked for before v2: the same property feeds the platform driver on the same nodes, and on core revision 0x01 it is wrong, which only a BCMA boot can notice. If you would rather the driver did not open the DT at all, that check is one separate function and comes out cleanly. Both shapes are tested on the RT-N18U (core revision 0x01): the warning lines match the applied version, /proc/iomem and the enumerated devices are unchanged.
One thing changes with it. With PCIE_IPROC_PLATFORM and PCIE_IPROC_BCMA both enabled, a core revision 0x01 board now gets a second probe from this driver: the DT window sits elsewhere there, so the collision that stopped it before is gone. Revision 0x07 still collides. That configuration does not work on revision 0x01 either way, since the DT window there is one the hardware does not decode, and Rafał's DT commit was tested with the platform driver, so I take that as the intended path on mainline. It is in the commit log.
If it is better squashed into 552aa843e4c5 while that is still on the topic branch, I can send it that way instead.
Best regards,
Semih
PATCH 1/1 PCI: iproc: Use pci_alloc_host_bridge() on BCMA
The BCMA driver takes its register base and, since commit 552aa843e4c5 ("PCI: iproc: Use the EROM outbound window on BCMA"), its outbound window from what bcma read out of the enumeration ROM. It still allocates its host bridge with devm_pci_alloc_host_bridge(), which since commit 669cbc708122 ("PCI: Move DT resource setup into devm_pci_alloc_host_bridge()") parses ranges, dma-ranges and bus-range from the device's OF node and requests the windows it finds. None of that reaches the hardware here: need_ob_cfg is only ever set by the platform driver, so iproc_pcie_setup() never maps the parsed windows, and the EROM commit above had to throw them away again to keep them from colliding with its own window.
Allocate the bridge with pci_alloc_host_bridge() instead, so nothing from the devicetree is requested or handed to the PCI core, and free it on the error paths and in remove(). The driver now sets bridge->dev.parent itself, as devm_pci_alloc_host_bridge() did, since the wifi nodes under pcie_bridge0 in bcm4709-netgear-r8000.dts are resolved through the root bus. The window request
moves from devm to request_resource() and release_resource() because the resource lives inside the bridge allocation and has to be released before the bridge is freed.
The ranges property is still read, but only to compare. bcm-ns.dtsi describes the same window for the platform driver, and on core revision 0x01 it points at an address the hardware does not decode. The warning from the EROM commit stays for that reason: a wrong dts is visible on BCMA boots, where nothing else would show it. Without bus-range the root bus also logs "No busn resource found for root bus, will use [bus 00-ff]" again, which changes nothing else.
With PCIE_IPROC_PLATFORM and PCIE_IPROC_BCMA both enabled, the platform driver binds the same nodes first and claims the devicetree window. On core revision 0x07 that is the EROM window, so this driver's request still fails and the probe backs out as before. On revision 0x01 the devicetree window is elsewhere, so this driver now probes as well, next to a platform driver instance whose window the hardware does not decode. That instance does not work either; the only difference is that the second probe is no longer stopped by the collision.
Tested on an ASUS RT-N18U (BCM47081, core revision 0x01): the warning lines are identical to the applied version, /proc/iomem and the enumerated devices are unchanged.
--- drivers/pci/controller/pcie-iproc-bcma.c | 70 +++++++++++++++++------- 1 file changed, 49 insertions(+), 21 deletions(-)
drivers/pci/controller/pcie-iproc-bcma.c +49 -21
@@ -11,6 +11,7 @@#include <linux/phy/phy.h> #include <linux/bcma/bcma.h> #include <linux/ioport.h> +#include <linux/of_address.h> #include "pcie-iproc.h"
@@ -31,18 +32,42 @@ static int iproc_bcma_pcie_map_irq(const struct pci_dev *dev, u8 slot, u8 pin)return bcma_core_irq (bdev,5 ); } +static void iproc_bcma_pcie_check_dt_window (struct iproc_pcie *pcie) +{ +struct device_node *np = pcie->dev->of_node; +struct of_pci_range_parser parser; +struct of_pci_range range; +struct resource res; + +if (!np ||of_pci_range_parser_init (&parser, np)) +return ; + +for_each_of_pci_range (&parser, &range) { +if ((range.flags &IORESOURCE_TYPE_BITS ) !=IORESOURCE_MEM ) +continue ; + +if (of_pci_range_to_resource (&range, np, &res)) +continue ; + +if (res.start != pcie->mem.start || res.end != pcie->mem.end) +dev_warn (pcie->dev,"DT window %pR does not match EROM window %pR, using EROM\n" , + &res, &pcie->mem); + } +} +static int iproc_bcma_pcie_probe (struct bcma_device *bdev) {struct device *dev = &bdev->dev;struct iproc_pcie *pcie;struct pci_host_bridge *bridge; -struct resource_entry *win, *tmp;int ret; - bridge =devm_pci_alloc_host_bridge (dev,sizeof (*pcie)); + bridge =pci_alloc_host_bridge (sizeof (*pcie));if (!bridge)return -ENOMEM ; + bridge->dev.parent = dev; + pcie =pci_host_bridge_priv (bridge); pcie->dev = dev;
@@ -51,7 +76,8 @@ static int iproc_bcma_pcie_probe(struct bcma_device *bdev) pcie->base = bdev->io_addr;if (!pcie->base) {dev_err (dev,"no controller registers\n" ); -return -ENOMEM ; + ret = -ENOMEM ; + goto err_free_bridge; } pcie->base_addr = bdev->addr;
@@ -60,37 +86,39 @@ static int iproc_bcma_pcie_probe(struct bcma_device *bdev) pcie->mem.end = bdev->addr_s[0 ] +SZ_128M -1 ; pcie->mem.name ="PCIe MEM space" ; pcie->mem.flags =IORESOURCE_MEM ; - -resource_list_for_each_entry_safe (win, tmp, &bridge->windows) { -if (resource_type (win->res) !=IORESOURCE_MEM ) -continue ; - -if (win->res->start != pcie->mem.start || - win->res->end != pcie->mem.end) -dev_warn (dev,"DT window %pR does not match EROM window %pR, using EROM\n" , - win->res, &pcie->mem); - -devm_release_resource (dev, win->res); -resource_list_destroy_entry (win); - } - +iproc_bcma_pcie_check_dt_window (pcie);pci_add_resource (&bridge->windows, &pcie->mem); - ret =devm_request_resource (dev, &iomem_resource, &pcie->mem); -if (ret) -return ret; + ret =request_resource (&iomem_resource, &pcie->mem); +if (ret) { +dev_err (dev,"can't claim %pR\n" , &pcie->mem); + goto err_free_bridge; + } bridge->map_irq = iproc_bcma_pcie_map_irq;bcma_set_drvdata (bdev, pcie); -return iproc_pcie_setup (pcie, &bridge->windows); + ret =iproc_pcie_setup (pcie, &bridge->windows); +if (ret) + goto err_release_mem; + +return 0 ; + +err_release_mem: +release_resource (&pcie->mem); +err_free_bridge: +pci_free_host_bridge (bridge); +return ret; }static void iproc_bcma_pcie_remove (struct bcma_device *bdev) {struct iproc_pcie *pcie =bcma_get_drvdata (bdev); +struct pci_host_bridge *bridge =pci_host_bridge_from_priv (pcie);iproc_pcie_remove (pcie); +release_resource (&pcie->mem); +pci_free_host_bridge (bridge); }static const struct bcma_device_id iproc_bcma_pcie_table[] = {
Signature · 1 line
2.43.0
Re: [PATCH] PCI: iproc: Use pci_alloc_host_bridge() on BCMA
Quoted 30 lines · Semih Baskan
The BCMA driver takes its register base and, since commit
552aa843e4c5("PCI: iproc: Use the EROM outbound window on BCMA"), its outbound window from what bcma read out of the enumeration ROM. It still allocates its host bridge withdevm_pci_alloc_host_bridge(), which since commit669cbc708122("PCI: Move DT resource setup intodevm_pci_alloc_host_bridge()") parses ranges, dma-ranges and bus-range from the device's OF node and requests the windows it finds. None of that reaches the hardware here:need_ob_cfgis only ever set by the platform driver, soiproc_pcie_setup()never maps the parsed windows, and the EROM commit above had to throw them away again to keep them from colliding with its own window.Allocate the bridge with
pci_alloc_host_bridge()instead, so nothing from the devicetree is requested or handed to the PCI core, and free it on the error paths and inremove(). The driver now setsbridge->dev.parentitself, asdevm_pci_alloc_host_bridge()did, since the wifi nodes underpcie_bridge0inbcm4709-netgear-r8000.dtsare resolved through the root bus. The window request
moves from devm torequest_resource()andrelease_resource()because the resource lives inside the bridge allocation and has to be released before the bridge is freed.The ranges property is still read, but only to compare.
bcm-ns.dtsidescribes the same window for the platform driver, and on core revision0x01it points at an address the hardware does not decode. The warning from the EROM commit stays for that reason: a wrong dts is visible on BCMA boots, where nothing else would show it. Without bus-range the root bus also logs "No busn resource found for root bus, will use [bus 00-ff]" again, which changes nothing else.
Quoted 8 lines
With
PCIE_IPROC_PLATFORMandPCIE_IPROC_BCMAboth enabled, the platform driver binds the same nodes first and claims the devicetree window. On core revision0x07that is the EROM window, so this driver's request still fails and the probe backs out as before. On revision0x01the devicetree window is elsewhere, so this driver now probes as well, next to a platform driver instance whose window the hardware does not decode. That instance does not work either; the only difference is that the second probe is no longer stopped by the collision.
I guess this goes back to 767012397976 ("ARM: dts: BCM5301X: Describe PCIe controllers fully"), but I'm confused about this. Why are two incompatible devices (rev 0x01 and 0x07) described with the same DT with address ranges that are wrong for rev 0x01? I thought DT was supposed to be matched with the hardware in the box?
And I guess I missed this part about the platform and the bcma drivers both trying to claim the same device. That seems like something that should be solved somewhere in the bus drivers (platform, bcma), not in pcie-iproc-bcma.c and pcie-iproc-platform.c.
Why is this not a problem for other BCMA devices (bgmac_bcma_driver, b43_bcma_driver, brcms_bcma_driver, bcma_hcd_driver)?
Is this another consequence of using a DT that describes "brcm,iproc-pcie" controllers that don't match the hardware? Why don't we have different DTs for these two kinds of hardware?
Quoted 139 lines
Tested on an ASUS RT-N18U (BCM47081, core revision
0x01): the warning lines are identical to the applied version,/proc/iomemand the enumerated devices are unchanged.Suggested-by Bjorn Helgaas <[email protected]>Signed-off-by Semih Baskan <[email protected]>--- drivers/pci/controller/pcie-iproc-bcma.c | 70 +++++++++++++++++------- 1 file changed, 49 insertions(+), 21 deletions(-)drivers/pci/controller/pcie-iproc-bcma.c +49
-21@@ -11,6 +11,7 @@#include <linux/phy/phy.h> #include <linux/bcma/bcma.h> #include <linux/ioport.h> +#include <linux/of_address.h> #include "pcie-iproc.h" @@ -31,18 +32,42 @@ static int iproc_bcma_pcie_map_irq(const struct pci_dev *dev, u8 slot, u8 pin)return bcma_core_irq (bdev,5 ); } +static void iproc_bcma_pcie_check_dt_window (struct iproc_pcie *pcie) +{ +struct device_node *np = pcie->dev->of_node; +struct of_pci_range_parser parser; +struct of_pci_range range; +struct resource res; + +if (!np ||of_pci_range_parser_init (&parser, np)) +return ; + +for_each_of_pci_range (&parser, &range) { +if ((range.flags &IORESOURCE_TYPE_BITS ) !=IORESOURCE_MEM ) +continue ; + +if (of_pci_range_to_resource (&range, np, &res)) +continue ; + +if (res.start != pcie->mem.start || res.end != pcie->mem.end) +dev_warn (pcie->dev,"DT window %pR does not match EROM window %pR, using EROM\n" , + &res, &pcie->mem); + } +} +static int iproc_bcma_pcie_probe (struct bcma_device *bdev) {struct device *dev = &bdev->dev;struct iproc_pcie *pcie;struct pci_host_bridge *bridge; -struct resource_entry *win, *tmp;int ret; - bridge =devm_pci_alloc_host_bridge (dev,sizeof (*pcie)); + bridge =pci_alloc_host_bridge (sizeof (*pcie));if (!bridge)return -ENOMEM ; + bridge->dev.parent = dev; + pcie =pci_host_bridge_priv (bridge); pcie->dev = dev;@@ -51,7 +76,8 @@ static int iproc_bcma_pcie_probe(struct bcma_device *bdev) pcie->base = bdev->io_addr;if (!pcie->base) {dev_err (dev,"no controller registers\n" ); -return -ENOMEM ; + ret = -ENOMEM ; + goto err_free_bridge; } pcie->base_addr = bdev->addr;@@ -60,37 +86,39 @@ static int iproc_bcma_pcie_probe(struct bcma_device *bdev) pcie->mem.end = bdev->addr_s[0 ] +SZ_128M -1 ; pcie->mem.name ="PCIe MEM space" ; pcie->mem.flags =IORESOURCE_MEM ; - -resource_list_for_each_entry_safe (win, tmp, &bridge->windows) { -if (resource_type (win->res) !=IORESOURCE_MEM ) -continue ; - -if (win->res->start != pcie->mem.start || - win->res->end != pcie->mem.end) -dev_warn (dev,"DT window %pR does not match EROM window %pR, using EROM\n" , - win->res, &pcie->mem); - -devm_release_resource (dev, win->res); -resource_list_destroy_entry (win); - } - +iproc_bcma_pcie_check_dt_window (pcie);pci_add_resource (&bridge->windows, &pcie->mem); - ret =devm_request_resource (dev, &iomem_resource, &pcie->mem); -if (ret) -return ret; + ret =request_resource (&iomem_resource, &pcie->mem); +if (ret) { +dev_err (dev,"can't claim %pR\n" , &pcie->mem); + goto err_free_bridge; + } bridge->map_irq = iproc_bcma_pcie_map_irq;bcma_set_drvdata (bdev, pcie); -return iproc_pcie_setup (pcie, &bridge->windows); + ret =iproc_pcie_setup (pcie, &bridge->windows); +if (ret) + goto err_release_mem; + +return 0 ; + +err_release_mem: +release_resource (&pcie->mem); +err_free_bridge: +pci_free_host_bridge (bridge); +return ret; }static void iproc_bcma_pcie_remove (struct bcma_device *bdev) {struct iproc_pcie *pcie =bcma_get_drvdata (bdev); +struct pci_host_bridge *bridge =pci_host_bridge_from_priv (pcie);iproc_pcie_remove (pcie); +release_resource (&pcie->mem); +pci_free_host_bridge (bridge); }static const struct bcma_device_id iproc_bcma_pcie_table[] = {Signature · 1 line
2.43.0
Re: [PATCH] PCI: iproc: Use pci_alloc_host_bridge() on BCMA
Quoted 5 lines · Bjorn Helgaas
I guess this goes back to 767012397976 ("ARM: dts: BCM5301X: Describe PCIe controllers fully"), but I'm confused about this. Why are two incompatible devices (rev
0x01and0x07) described with the same DT with address ranges that are wrong for rev0x01? I thought DT was supposed to be matched with the hardware in the box?
Yes, it goes back to that commit. The PCIe nodes are in bcm-ns.dtsi, the include behind bcm4708.dtsi, bcm47081.dtsi, bcm4709.dtsi and bcm47094.dtsi. Before 767012397976 they had reg and the cell sizes only. That commit added the compatible, the interrupt maps, bus-range and one set of ranges, and widened the axi node's ranges to the same three windows. Its log says it was tested on BCM47094 with the platform driver.
Both revisions are the same PCIe Gen 2 core, BCMA_CORE_NS_PCIEG2, and pcie-iproc-bcma matches it at BCMA_ANY_REV. The difference this driver meets is the fixed outbound window base. The first controller decodes 0x08000000 on both. The second and third decode 0x20000000/0x28000000 on revision 0x07 (BCM47094) and 0x40000000/0x48000000 on revision 0x01 (BCM47081 and BCM4709, measured). The values in the DT are the revision 0x07 ones. Broadcom's own driver in the 2.6.36 vendor kernels, arch/arm/plat-brcm/bcm5301x_pcie.c, has 0x40000000 and 0x48000000 as its default table and switches to 0x20000000 and 0x28000000 only when the core revision reads 0x7. The enumeration ROM reports the same base per core, and that is what the applied patch reads through bcma.
Quoted 4 lines
And I guess I missed this part about the platform and the bcma drivers both trying to claim the same device. That seems like something that should be solved somewhere in the bus drivers (platform, bcma), not in
pcie-iproc-bcma.candpcie-iproc-platform.c.
Yes. Both drivers have bound these nodes since 767012397976 in any build with both enabled, and neither the applied patch nor the follow-up changes which one binds first. The follow-up only stops this driver from requesting windows it never programs. It is in the commit log because that changes what happens after the collision on revision 0x01, and multi_v7_defconfig builds both drivers (both symbols default to y under ARCH_BCM_5301X). OpenWrt builds only the BCMA one.
Quoted 2 lines
Why is this not a problem for other BCMA devices (
bgmac_bcma_driver,b43_bcma_driver,brcms_bcma_driver,bcma_hcd_driver)?
Because the pcie nodes are the only children of the axi node in bcm-ns.dtsi with a compatible at all. bcma_bus_register() runs of_platform_default_populate() on the axi node before it registers its cores, so a core whose node has a compatible gets a device from both sides.
The gmac nodes have no compatible, so only bgmac-bcma binds them; bgmac-platform matches brcm,amac, brcm,nsp-amac and brcm,ns2-amac, none of which appear there. The usb2 and usb3 nodes have no compatible either; bcma-hcd claims the core and populates the generic-ehci, ohci and xhci children itself. b43 and brcmsmac have no platform counterpart.
Quoted 3 lines
Is this another consequence of using a DT that describes "brcm,iproc-pcie" controllers that don't match the hardware? Why don't we have different DTs for these two kinds of hardware?
The double claim comes from the compatible. Revision 0x07, where the DT window matches the EROM, gets the two probes as well, so the ranges play no part in it. bcma attaches the node to the core by its reg either way, and the wifi child nodes in bcm4709-netgear-r8000.dts hang off it, so both drivers see the same node however the ranges are split.
The DT is already split per SoC family, and BCM47094 has its own dtsi. The shared part is the include. The per-SoC files can carry the window difference; today they do not.
Best regards,
Semih
Re: [PATCH] PCI: iproc: Use pci_alloc_host_bridge() on BCMA
Quoted 61 lines · Semih Baskan
Quoted 5 lines · Bjorn Helgaas
I guess this goes back to 767012397976 ("ARM: dts: BCM5301X: Describe PCIe controllers fully"), but I'm confused about this. Why are two incompatible devices (rev
0x01and0x07) described with the same DT with address ranges that are wrong for rev0x01? I thought DT was supposed to be matched with the hardware in the box?Yes, it goes back to that commit. The PCIe nodes are in
bcm-ns.dtsi, the include behindbcm4708.dtsi,bcm47081.dtsi,bcm4709.dtsiandbcm47094.dtsi. Before 767012397976 they had reg and the cell sizes only. That commit added the compatible, the interrupt maps, bus-range and one set of ranges, and widened the axi node's ranges to the same three windows. Its log says it was tested on BCM47094 with the platform driver.Both revisions are the same PCIe Gen 2 core,
BCMA_CORE_NS_PCIEG2, and pcie-iproc-bcma matches it atBCMA_ANY_REV. The difference this driver meets is the fixed outbound window base. The first controller decodes0x08000000on both. The second and third decode 0x20000000/0x28000000 on revision0x07(BCM47094) and 0x40000000/0x48000000 on revision0x01(BCM47081 and BCM4709, measured). The values in the DT are the revision0x07ones. Broadcom's own driver in the 2.6.36 vendor kernels,arch/arm/plat-brcm/bcm5301x_pcie.c, has0x40000000and0x48000000as its default table and switches to0x20000000and0x28000000only when the core revision reads0x7. The enumeration ROM reports the same base per core, and that is what the applied patch reads through bcma.Quoted 4 lines
And I guess I missed this part about the platform and the bcma drivers both trying to claim the same device. That seems like something that should be solved somewhere in the bus drivers (platform, bcma), not in
pcie-iproc-bcma.candpcie-iproc-platform.c.Yes. Both drivers have bound these nodes since 767012397976 in any build with both enabled, and neither the applied patch nor the follow-up changes which one binds first. The follow-up only stops this driver from requesting windows it never programs. It is in the commit log because that changes what happens after the collision on revision
0x01, andmulti_v7_defconfigbuilds both drivers (both symbols default to y underARCH_BCM_5301X). OpenWrt builds only the BCMA one.Quoted 2 lines
Why is this not a problem for other BCMA devices (
bgmac_bcma_driver,b43_bcma_driver,brcms_bcma_driver,bcma_hcd_driver)?Because the pcie nodes are the only children of the axi node in
bcm-ns.dtsiwith a compatible at all.bcma_bus_register()runsof_platform_default_populate()on the axi node before it registers its cores, so a core whose node has a compatible gets a device from both sides.The gmac nodes have no compatible, so only bgmac-bcma binds them; bgmac-platform matches brcm,amac, brcm,nsp-amac and brcm,ns2-amac, none of which appear there. The usb2 and usb3 nodes have no compatible either; bcma-hcd claims the core and populates the generic-ehci, ohci and xhci children itself. b43 and brcmsmac have no platform counterpart.
Quoted 3 lines
Is this another consequence of using a DT that describes "brcm,iproc-pcie" controllers that don't match the hardware? Why don't we have different DTs for these two kinds of hardware?
Quoted 6 lines
The double claim comes from the compatible. Revision
0x07, where the DT window matches the EROM, gets the two probes as well, so the ranges play no part in it. bcma attaches the node to the core by its reg either way, and the wifi child nodes inbcm4709-netgear-r8000.dtshang off it, so both drivers see the same node however the ranges are split.
If bcma finds both of these devices via EROM, why do they need a "brcm,iproc-pcie" compatible? If we omitted that compatible, what would break?
Sorry to be dense, I'm probably asking dumb questions because I'm not a DT expert. I just imagine DT as being a substitute for native enumeration protocols (e.g., PCI, ACPI, EROM), and as specific to a piece of hardware, so this single DT that describes enumerable devices with incompatible addresses doesn't fit my simple mental model.
Bjorn
Re: [PATCH] PCI: iproc: Use pci_alloc_host_bridge() on BCMA
Quoted 3 lines · Bjorn Helgaas
If bcma finds both of these devices via EROM, why do they need a "brcm,iproc-pcie" compatible? If we omitted that compatible, what would break?
bcma does not need it. pcie-iproc-platform does.
bcma attaches a child node to a core by comparing the node's reg with the core's base address, in bcma_of_find_child_device(), and the only compatible it looks at is the bus's own "brcm,bus-axi". The PCI devices below then find their nodes by devfn under the core's node, through pci_set_bus_of_node() and of_pci_find_child_device(). The wifi nodes in bcm4709-netgear-r8000.dts go back to 5d1f2d2c2530 ("ARM: dts:
BCM5301X: Set 5 GHz wireless frequency limits on Netgear R8000") in 2017, which added pcie0 and pcie1 to the dtsi with reg and nothing else. The core IRQs have come from the axi node's interrupt-map since 1f80de6863ca ("ARM: BCM5301X: add IRQ numbers for PCIe controller") in 2015.
The only code that matches the compatible is the pcie-iproc-platform match table. The compatible is also what makes of_platform_default_populate() create a platform device for the node. Without one the node would be skipped and the platform driver would never bind it. That driver reads the ranges, bus-range and the pcie node's own interrupt-map, through devm_pci_alloc_host_bridge() and of_irq_parse_and_map_pci(). The BCMA driver does not depend on any of them. It has taken its window from the EROM since it was added, 552aa843e4c5 made it drop the DT one, and its interrupt comes from bcma. With the follow-up it reads ranges only to compare them with the EROM window.
What would break is a build with PCIE_IPROC_PLATFORM and without PCIE_IPROC_BCMA. No in-tree defconfig sets either symbol. Both default to y under ARCH_BCM_5301X, so multi_v7_defconfig builds both and would still get PCIe through bcma. OpenWrt builds only the BCMA one.
The dtbs_check warnings in 767012397976's log would not come back. The schema behind them, pci-bus.yaml in that log and pci-bus-common.yaml in current dtschema, is selected by the node name, "^pcie?@", and requires device_type, ranges and the cell sizes, not a compatible. I checked with dt-validate from dtschema 2026.6: a pcie@12000 node with reg and the cell sizes only gives the same two warnings, and the same node with device_type and ranges added and still no compatible gives none.
Quoted 5 lines
Sorry to be dense, I'm probably asking dumb questions because I'm not a DT expert. I just imagine DT as being a substitute for native enumeration protocols (e.g., PCI, ACPI, EROM), and as specific to a piece of hardware, so this single DT that describes enumerable devices with incompatible addresses doesn't fit my simple mental model.
The bcma binding reads the same way. bindings/bus/brcm,bus-axi.txt says the cores are detected by bcma with the memory ranges they use. The child nodes are there for what cannot be detected, IRQ numbers in particular, and for cores that do extra things, such as ChipCommon and its GPIO chip. Under that binding a pcie node is attached to a core the EROM found, by reg.
Best regards,
Semih
Re: [PATCH] PCI: iproc: Use pci_alloc_host_bridge() on BCMA
Quoted 6 lines · Semih Baskan
Quoted 3 lines · Bjorn Helgaas
If bcma finds both of these devices via EROM, why do they need a "brcm,iproc-pcie" compatible? If we omitted that compatible, what would break?
bcma does not need it. pcie-iproc-platform does.
And why do we need pcie-iproc-platform, since pcie-iproc-bcma.c will claim them?
Is there a requirement that pcie-iproc-platform must be able to claim the PCI host bridge if CONFIG_PCIE_IPROC_BCMA is not enabled?
Re: [PATCH] PCI: iproc: Use pci_alloc_host_bridge() on BCMA
Quoted 2 lines · Bjorn Helgaas
And why do we need pcie-iproc-platform, since
pcie-iproc-bcma.cwill claim them?
I do not see anything on Northstar that needs it. I think pcie-iproc-platform is the front end for the iProc SoCs that have no bcma bus.
Quoted 2 lines
Is there a requirement that pcie-iproc-platform must be able to claim the PCI host bridge if
CONFIG_PCIE_IPROC_BCMAis not enabled?
None that I can find. Neither symbol depends on the other, no in-tree defconfig sets either one, and multi_v7_defconfig gets both from their defaults. OpenWrt builds only the BCMA one. The one use of the platform driver on Northstar I know of is Rafał's: 767012397976 says it was tested with it on BCM47094, and his test in the thread on the axi ranges ran both drivers on his board. The platform driver enumerated the two BCM4366 devices on its own:
https://lore.kernel.org/all/[email protected]/
I have not looked outside the tree.
Best regards,
Semih
Re: [PATCH] PCI: iproc: Use pci_alloc_host_bridge() on BCMA
Quoted 11 lines · Semih Baskan
Quoted 2 lines · Bjorn Helgaas
And why do we need pcie-iproc-platform, since
pcie-iproc-bcma.cwill claim them?I do not see anything on Northstar that needs it. I think pcie-iproc-platform is the front end for the iProc SoCs that have no bcma bus.
Quoted 2 lines
Is there a requirement that pcie-iproc-platform must be able to claim the PCI host bridge if
CONFIG_PCIE_IPROC_BCMAis not enabled?
Quoted 9 lines
None that I can find. Neither symbol depends on the other, no in-tree defconfig sets either one, and
multi_v7_defconfiggets both from their defaults. OpenWrt builds only the BCMA one. The one use of the platform driver on Northstar I know of is Rafał's: 767012397976 says it was tested with it on BCM47094, and his test in the thread on the axi ranges ran both drivers on his board. The platform driver enumerated the two BCM4366 devices on its own:
https://lore.kernel.org/all/[email protected]/
I have not looked outside the tree.
Then it seems like we should remove the "brcm,iproc-pcie" compatible from DTs for platforms where bcma can enumerate it via EROM. Those platforms can enable CONFIG_PCIE_IPROC_BCMA. They still need the PCIe controller node in DT so they can use the wifi nodes below it, but if there's no "brcm,iproc-pcie" compatible, pcie-iproc-platform shouldn't do anything.
Arnd, Rafał, jump in if I'm in the weeds here.
Re: [PATCH] PCI: iproc: Use pci_alloc_host_bridge() on BCMA
Quoted 11 lines · Bjorn Helgaas
Quoted 1 line · Semih Baskan
On Thu, Sep 17, 2026 at 10:59:29AM -0500, Bjorn Helgaas wrote:
Then it seems like we should remove the "brcm,iproc-pcie" compatible from DTs for platforms where bcma can enumerate it via EROM. Those platforms can enable
CONFIG_PCIE_IPROC_BCMA. They still need the PCIe controller node in DT so they can use the wifi nodes below it, but if there's no "brcm,iproc-pcie" compatible, pcie-iproc-platform shouldn't do anything.Arnd, Rafał, jump in if I'm in the weeds here.
It feels wrong to me to change the devicetee file when that arguably describes the device correctly, and we still need the node for the bcma bus probe in the end.
Normally, there should be a list of compatible strings so the driver can make a more informed decision, e.g. the platform driver could skip a device if it identifies the compatible string as one that can be probed using the bcma bus when that driver is also enabled.
Unfortunately, it appears that all of these only have the generic "brcm,iproc-pcie" string (or only the paxc variant), so that doesn't work here.
Would it be help to change the probe order so the bcma driver always comes before the platform driver?
Arnd
Re: [PATCH] PCI: iproc: Use pci_alloc_host_bridge() on BCMA
Quoted 11 lines · Arnd Bergmann
Quoted 10 lines · Bjorn Helgaas
Quoted 1 line · Semih Baskan
On Thu, Sep 17, 2026 at 10:59:29AM -0500, Bjorn Helgaas wrote:
Then it seems like we should remove the "brcm,iproc-pcie" compatible from DTs for platforms where bcma can enumerate it via EROM. Those platforms can enable
CONFIG_PCIE_IPROC_BCMA. They still need the PCIe controller node in DT so they can use the wifi nodes below it, but if there's no "brcm,iproc-pcie" compatible, pcie-iproc-platform shouldn't do anything.
Quoted 5 lines
Quoted 1 line
Arnd, Rafał, jump in if I'm in the weeds here.
It feels wrong to me to change the devicetee file when that arguably describes the device correctly, and we still need the node for the bcma bus probe in the end.
I don't think pcie-iproc-bcma.c needs anything from DT at all. Everything it needs is in EROM.
Some PCI drivers, e.g., wifi, need DT nodes, and I think those need to be enclosed in a node for the PCI host controller, but pcie-iproc-bcma.c itself doesn't care.
Bjorn
Re: [PATCH] PCI: iproc: Use pci_alloc_host_bridge() on BCMA
Quoted 7 lines · Bjorn Helgaas
Quoted 5 lines · Arnd Bergmann
On Thu, Sep 17, 2026, at 20:36, Bjorn Helgaas wrote:
It feels wrong to me to change the devicetee file when that arguably describes the device correctly, and we still need the node for the bcma bus probe in the end.
Quoted 6 lines
I don't think
pcie-iproc-bcma.cneeds anything from DT at all. Everything it needs is in EROM.Some PCI drivers, e.g., wifi, need DT nodes, and I think those need to be enclosed in a node for the PCI host controller, but
pcie-iproc-bcma.citself doesn't care.
I understand that it just works without that, my only concerns is that having a DT node (which is required for adding child nodes) without a compatible string is going to cause problems later,
From the DT perspective, it would be better to add a second (or third) compatible string to identify the device better and keep the existing generic string as a fallback. The pcie-iproc-platform driver can then skip the device based on the more specific string.
Simply replacing the existing compatible string with a new one would solve the probe order problem and at least leave a well-formed device node, but this would still count as an incompatible binding update, which we try hard to avoid.
Arnd
Re: [PATCH] PCI: iproc: Use pci_alloc_host_bridge() on BCMA
In reply to Arnd BergmannQuoted 2 lines · Arnd Bergmann
Would it be help to change the probe order so the bcma driver always comes before the platform driver?
Only where the DT window matches the EROM one, because that is where the platform driver backs out.
In the tree as it is, the platform driver binds first. bcma_bus_register() populates the axi node's children before it registers the cores (drivers/bcma/main.c), and with both drivers built in the two PCIe drivers are already registered by then (both are device_initcall and drivers/pci links before drivers/bcma), so the pcie platform devices are bound as soon as they are created. The order of the two objects in drivers/pci/controller/Makefile does not change that.
I tried both orders on an RT-N18U (BCM47081, revision 0x01) with both drivers built in, on an OpenWrt 6.18 kernel with pcie-iproc-bcma.c as in this patch and no wifi driver built.
With the platform driver first, the board did not come up, on two flashes. The same tree built with only the BCMA driver boots it.
With of_platform_default_populate() moved after bcma_register_devices() as a test, it boots. pcie-iproc-bcma takes the first controller and enumerates the BCM4360 as before. The platform driver then probes the same node and fails in devm_pci_alloc_host_bridge(), which it reports as -ENOMEM:
iproc-pcie 18012000.pcie: resource collision: [mem
0x08000000-0x0fffffff] conflicts with PCIe MEM space [mem 0x08000000-0x0fffffff]
On the second and third controllers the DT window is 0x20000000/0x28000000 and the EROM one 0x40000000/0x48000000, so the platform driver's request goes through and its probe carries on with the DT window until the link check, which fails because nothing is connected to those two on this board. That is as far as this board can show.
On the first controller the DT window and the EROM one are the same, so the window plays no part in the no-boot. With PCIE_IPROC_BCMA not set at all the board did not come up either. Clearing has_apb_err_disable for IPROC_PCIE_PAXB, one line, makes that platform-only kernel boot and enumerate the BCM4360; nothing else differs between the two builds. The flag makes the driver read and write APB_ERR_EN (0xf40) around every config access to bus 1 and above (538928fd6ce8). In the bcma-first build the platform driver never reached bus 1. The PAXB_BCMA register table has no such register, and 404349c5c806 ("PCI: iproc: Add BCMA type") describes the BCMA-based NS as a legacy PAXB with some registers missing. On Rafał's BCM47094 (revision 0x07) the platform driver as shipped enumerates both BCM4366s. I could not see what the access to 0xf40 does on revision 0x01, only that the board comes up without it.
Best regards,
Semih
Re: [PATCH] PCI: iproc: Use pci_alloc_host_bridge() on BCMA
Quoted 6 lines · Semih Baskan
Quoted 2 lines · Arnd Bergmann
Would it be help to change the probe order so the bcma driver always comes before the platform driver?
Only where the DT window matches the EROM one, because that is where the platform driver backs out.
I'm probably missing one of the steps here, sorry I'm not too familiar with the platform.
If the DT window doesn't match, how does the bcma driver find the device node that corresponds to the EROM entry to populate the child nodes?
Quoted 9 lines
With
of_platform_default_populate()moved afterbcma_register_devices()as a test, it boots. pcie-iproc-bcma takes the first controller and enumerates the BCM4360 as before. The platform driver then probes the same node and fails indevm_pci_alloc_host_bridge(), which it reports as-ENOMEM:iproc-pcie 18012000.pcie: resource collision: [mem
0x08000000-0x0fffffff] conflicts with PCIe MEM space [mem 0x08000000-0x0fffffff]
The driver should probably check earlier if the device is already bound to the bcma bus, rather than just trying to request the bus windows. If anything, I would have expected the conflict on the 18012000 address for the register base.
I see an ioremap() of bdev->addr in drivers/bcma/scan.c but don't see a corresponding request_resource() for it, so maybe an easy fix is to add this to the bcma bus scan and make sure that always happens before the pci driver probe? That way, the platform driver would just not be able to probe on a bcma based SoC regardless of the memory window settings.
Quoted 6 lines
On the second and third controllers the DT window is 0x20000000/0x28000000 and the EROM one 0x40000000/0x48000000, so the platform driver's request goes through and its probe carries on with the DT window until the link check, which fails because nothing is connected to those two on this board. That is as far as this board can show.
A few more questions, for my understanding:
If a board has nothing connected to one of the host bridges, why is the pcie host bridge still marked as status="ok"? Do you need a driver to put it into low-power mode, or should this just be marked as disabled?
Does the platform driver reprogram the windows to match the DT description, or does it rely on the boot loader to ensure they ranges/dma-ranges properties match the register contents?
When using the bcma driver, does that update the ranges properties in memory to match the EROM contents?
Arnd
Re: [PATCH] PCI: iproc: Use pci_alloc_host_bridge() on BCMA
Quoted 3 lines · Arnd Bergmann
If the DT window doesn't match, how does the bcma driver find the device node that corresponds to the EROM entry to populate the child nodes?
By reg. bcma_of_find_child_device() in drivers/bcma/main.c compares the first reg address of each child of the axi node with the core's register base from the EROM. That base is 0x18012000, 0x18013000 and 0x18014000 on both revisions; only the outbound window differs. The pcie node's ranges is not looked at. The reg is translated through the axi node's first ranges entry. The PCI core then matches the wifi nodes below that node by devfn.
Quoted 10 lines
The driver should probably check earlier if the device is already bound to the bcma bus, rather than just trying to request the bus windows. If anything, I would have expected the conflict on the 18012000 address for the register base.
I see an
ioremap()ofbdev->addrindrivers/bcma/scan.cbut don't see a correspondingrequest_resource()for it, so maybe an easy fix is to add this to the bcma bus scan and make sure that always happens before the pci driver probe?
In the tree as it is, neither side claims the register base. bcma only ioremaps it in scan.c, and pcie-iproc-platform maps its reg with devm_pci_remap_cfgspace(), which does not request the region either. That is why the window is the first thing the two collide on. On the ordering, bcma_bus_register() calls bcma_bus_scan(), then of_platform_default_populate(), then bcma_register_devices(), so the scan is done before the pcie platform devices are created and the bcma core devices are registered after that.
Quoted 4 lines
If a board has nothing connected to one of the host bridges, why is the pcie host bridge still marked as status="ok"? Do you need a driver to put it into low-power mode, or should this just be marked as disabled?
The three nodes are in bcm-ns.dtsi without a status property, and no Northstar board file disables one. Which controllers carry a device differs per board: the RT-N18U has one on the first, the R8000 on the first, and on the second through a switch with two radios behind it. bcma takes the cores from the EROM, and bcma_register_devices() skips a core whose node is not available, so a node marked status = "disabled" is skipped there too, and neither driver would take that core. I do not know whether an unused core needs a driver for power.
Quoted 3 lines
Does the platform driver reprogram the windows to match the DT description, or does it rely on the boot loader to ensure they ranges/dma-ranges properties match the register contents?
It programs OARR/OMAP from ranges only when the node has "brcm,pcie-ob", and the inbound side only when it has dma-ranges. The bcm-ns.dtsi nodes have neither, so it programs nothing and runs with what is in the registers at boot. The PAXB_BCMA register table has no OARR/OMAP at all, and the bcma driver never asks for an outbound mapping, so it does not program them either. OARR0/OMAP0 are in the PAXB table the platform driver uses, at 0xd20 and 0xd40; I have not written them on revision 0x01. Broadcom's 2.6.36 driver writes OMAP/OARR 1:1 from its own table: 0x08000000, 0x40000000 and 0x48000000, with the last two replaced by 0x20000000 and 0x28000000 when the core revision reads 0x07.
Quoted 2 lines
When using the bcma driver, does that update the ranges properties in memory to match the EROM contents?
No. 552aa843e4c5 drops the windows parsed from ranges out of the bridge's resource list and adds the EROM one; the patch in this thread reads ranges only to compare them with the EROM window. The property in the live tree is not touched, so /proc/device-tree keeps the dtsi values and the mismatch shows up only as the warning.
Best regards,
Semih
Re: [PATCH] PCI: iproc: Use pci_alloc_host_bridge() on BCMA
Quoted 5 lines · Semih Baskan
Quoted 3 lines · Arnd Bergmann
If the DT window doesn't match, how does the bcma driver find the device node that corresponds to the EROM entry to populate the child nodes?
Quoted 7 lines
By reg.
bcma_of_find_child_device()indrivers/bcma/main.ccompares the first reg address of each child of the axi node with the core's register base from the EROM. That base is0x18012000,0x18013000and0x18014000on both revisions; only the outbound window differs. The pcie node's ranges is not looked at. The reg is translated through the axi node's first ranges entry. The PCI core then matches the wifi nodes below that node by devfn.
Ok, makes sense.
Quoted 11 lines
Quoted 10 lines
The driver should probably check earlier if the device is already bound to the bcma bus, rather than just trying to request the bus windows. If anything, I would have expected the conflict on the 18012000 address for the register base.
I see an
ioremap()ofbdev->addrindrivers/bcma/scan.cbut don't see a correspondingrequest_resource()for it, so maybe an easy fix is to add this to the bcma bus scan and make sure that always happens before the pci driver probe?
Quoted 8 lines
In the tree as it is, neither side claims the register base. bcma only ioremaps it in
scan.c, and pcie-iproc-platform maps its reg withdevm_pci_remap_cfgspace(), which does not request the region either. That is why the window is the first thing the two collide on. On the ordering,bcma_bus_register()callsbcma_bus_scan(), thenof_platform_default_populate(), thenbcma_register_devices(), so the scan is done before the pcie platform devices are created and the bcma core devices are registered after that.
Ok, so adding the regular resource management to both sides here is probably a good idea anyway, though it is not sufficient as long the platform driver gets probed first.
Quoted 5 lines
Quoted 4 lines
If a board has nothing connected to one of the host bridges, why is the pcie host bridge still marked as status="ok"? Do you need a driver to put it into low-power mode, or should this just be marked as disabled?
Quoted 8 lines
The three nodes are in
bcm-ns.dtsiwithout a status property, and no Northstar board file disables one. Which controllers carry a device differs per board: the RT-N18U has one on the first, the R8000 on the first, and on the second through a switch with two radios behind it. bcma takes the cores from the EROM, andbcma_register_devices()skips a core whose node is not available, so a node marked status = "disabled" is skipped there too, and neither driver would take that core. I do not know whether an unused core needs a driver for power.
Ok
Quoted 4 lines
Quoted 3 lines
Does the platform driver reprogram the windows to match the DT description, or does it rely on the boot loader to ensure they ranges/dma-ranges properties match the register contents?
Quoted 11 lines
It programs OARR/OMAP from ranges only when the node has "brcm,pcie-ob", and the inbound side only when it has dma-ranges. The
bcm-ns.dtsinodes have neither, so it programs nothing and runs with what is in the registers at boot. ThePAXB_BCMAregister table has no OARR/OMAP at all, and the bcma driver never asks for an outbound mapping, so it does not program them either. OARR0/OMAP0 are in the PAXB table the platform driver uses, at0xd20and0xd40; I have not written them on revision0x01. Broadcom's 2.6.36 driver writes OMAP/OARR 1:1 from its own table:0x08000000,0x40000000and0x48000000, with the last two replaced by0x20000000and0x28000000when the core revision reads0x07.
Ah, interesting. So if the problem with the platform driver is that the outbound windows are mismatched between the register settings and the ranges property, wouldn't it make sense to always set "brcm,pcie-ob" for boards that are known to have more than one possible setting in their firmware?
That way I would hope the platform driver can just work in the cases that currently fail.
Quoted 8 lines
Quoted 2 lines
When using the bcma driver, does that update the ranges properties in memory to match the EROM contents?
No.
552aa843e4c5drops the windows parsed from ranges out of the bridge's resource list and adds the EROM one; the patch in this thread reads ranges only to compare them with the EROM window. The property in the live tree is not touched, so/proc/device-treekeeps the dtsi values and the mismatch shows up only as the warning.
Ok. I suspect this may cause problems when someone uses an MFD device that needs to translate internal registers of child device through the ranges. It's probably not a big deal since those devices are rare, just something to keep in mind.
Arnd
Re: [PATCH] PCI: iproc: Use pci_alloc_host_bridge() on BCMA
Quoted 7 lines · Arnd Bergmann
Ah, interesting. So if the problem with the platform driver is that the outbound windows are mismatched between the register settings and the ranges property, wouldn't it make sense to always set "brcm,pcie-ob" for boards that are known to have more than one possible setting in their firmware?
That way I would hope the platform driver can just work in the cases that currently fail.
On the RT-N18U the window is not what the board fails on. The device is on the first controller, where ranges and the EROM agree (0x08000000 on both revisions), and the board still does not come up with pcie-iproc-platform on it. Clearing has_apb_err_disable for IPROC_PCIE_PAXB, one line, so the driver never touches APB_ERR_EN (0xf40), makes the same platform-only kernel boot and enumerate the BCM4360. The flag is set for that type whether or not "brcm,pcie-ob" is present, and with the property the driver would program OARR in iproc_pcie_setup() before the enumeration that touches APB_ERR_EN.
The base follows the core revision rather than the firmware. On revision 0x01 the EROM window of the second and third controller is 0x40000000 and 0x48000000, and Broadcom's driver picks its OMAP/OARR values by core revision the same way. With "brcm,pcie-ob" the platform driver would write the ranges value into OARR, which in bcm-ns.dtsi is the revision 0x07 one for every SoC. Whether a revision 0x01 core decodes 0x20000000 once OARR says so I have not tried. In the tree only ns2.dtsi sets the property.
Best regards,
Semih
Re: [PATCH] PCI: iproc: Use pci_alloc_host_bridge() on BCMA
In reply to Semih Baskan[cc->to: Mani]
Quoted 39 lines · Semih Baskan
The BCMA driver takes its register base and, since commit
552aa843e4c5("PCI: iproc: Use the EROM outbound window on BCMA"), its outbound window from what bcma read out of the enumeration ROM. It still allocates its host bridge withdevm_pci_alloc_host_bridge(), which since commit669cbc708122("PCI: Move DT resource setup intodevm_pci_alloc_host_bridge()") parses ranges, dma-ranges and bus-range from the device's OF node and requests the windows it finds. None of that reaches the hardware here:need_ob_cfgis only ever set by the platform driver, soiproc_pcie_setup()never maps the parsed windows, and the EROM commit above had to throw them away again to keep them from colliding with its own window.Allocate the bridge with
pci_alloc_host_bridge()instead, so nothing from the devicetree is requested or handed to the PCI core, and free it on the error paths and inremove(). The driver now setsbridge->dev.parentitself, asdevm_pci_alloc_host_bridge()did, since the wifi nodes underpcie_bridge0inbcm4709-netgear-r8000.dtsare resolved through the root bus. The window request
moves from devm torequest_resource()andrelease_resource()because the resource lives inside the bridge allocation and has to be released before the bridge is freed.The ranges property is still read, but only to compare.
bcm-ns.dtsidescribes the same window for the platform driver, and on core revision0x01it points at an address the hardware does not decode. The warning from the EROM commit stays for that reason: a wrong dts is visible on BCMA boots, where nothing else would show it. Without bus-range the root bus also logs "No busn resource found for root bus, will use [bus 00-ff]" again, which changes nothing else.With
PCIE_IPROC_PLATFORMandPCIE_IPROC_BCMAboth enabled, the platform driver binds the same nodes first and claims the devicetree window. On core revision0x07that is the EROM window, so this driver's request still fails and the probe backs out as before. On revision0x01the devicetree window is elsewhere, so this driver now probes as well, next to a platform driver instance whose window the hardware does not decode. That instance does not work either; the only difference is that the second probe is no longer stopped by the collision.
Quoted 3 lines
Tested on an ASUS RT-N18U (BCM47081, core revision
0x01): the warning lines are identical to the applied version,/proc/iomemand the enumerated devices are unchanged.
What do you think about this, Mani?
I still don't quite understand the issue with the DT and discovering this via both platform and bcma bus drivers, but we should make progress.
I *think* that if we squash this patch into 552aa843e4c5 ("PCI: iproc:
Use the EROM outbound window on BCMA") and use devm_add_action_or_reset(), it will simplify down to this, which is less intrusive and would need less explanation:
drivers/pci/controller/pcie-iproc-bcma.c +38 -1
@@ -11,6 +11,7 @@#include <linux/phy/phy.h> #include <linux/bcma/bcma.h> #include <linux/ioport.h> +#include <linux/of_address.h> #include "pcie-iproc.h"
@@ -31,6 +32,34 @@ static int iproc_bcma_pcie_map_irq(const struct pci_dev *dev, u8 slot, u8 pin)return bcma_core_irq (bdev,5 ); } +static void iproc_bcma_pcie_check_dt_window (struct iproc_pcie *pcie) +{ +struct device_node *np = pcie->dev->of_node; +struct of_pci_range_parser parser; +struct of_pci_range range; +struct resource res; + +if (!np ||of_pci_range_parser_init (&parser, np)) +return ; + +for_each_of_pci_range (&parser, &range) { +if ((range.flags &IORESOURCE_TYPE_BITS ) !=IORESOURCE_MEM ) +continue ; + +if (of_pci_range_to_resource (&range, np, &res)) +continue ; + +if (res.start != pcie->mem.start || res.end != pcie->mem.end) +dev_warn (pcie->dev,"DT window %pR does not match EROM window %pR, using EROM\n" , + &res, &pcie->mem); + } +} + +static void devm_pci_alloc_host_bridge_release (void *data) +{ +pci_free_host_bridge (data); +} +static int iproc_bcma_pcie_probe (struct bcma_device *bdev) {struct device *dev = &bdev->dev;
@@ -38,10 +67,17 @@ static int iproc_bcma_pcie_probe(struct bcma_device *bdev)struct pci_host_bridge *bridge;int ret; - bridge =devm_pci_alloc_host_bridge (dev,sizeof (*pcie)); + bridge =pci_alloc_host_bridge (sizeof (*pcie));if (!bridge)return -ENOMEM ; + bridge->dev.parent = dev; + + ret =devm_add_action_or_reset (dev, devm_pci_alloc_host_bridge_release, + bridge); +if (ret) +return -ENOMEM ; + pcie =pci_host_bridge_priv (bridge); pcie->dev = dev;
@@ -59,6 +95,7 @@ static int iproc_bcma_pcie_probe(struct bcma_device *bdev) pcie->mem.end = bdev->addr_s[0 ] +SZ_128M -1 ; pcie->mem.name ="PCIe MEM space" ; pcie->mem.flags =IORESOURCE_MEM ; +iproc_bcma_pcie_check_dt_window (pcie);pci_add_resource (&bridge->windows, &pcie->mem); ret =devm_request_pci_bus_resources (dev, &bridge->windows);if (ret)
Re: [PATCH] PCI: iproc: Use pci_alloc_host_bridge() on BCMA
Quoted 46 lines · Bjorn Helgaas
[
cc->to: Mani]Quoted 39 lines · Semih Baskan
The BCMA driver takes its register base and, since commit
552aa843e4c5("PCI: iproc: Use the EROM outbound window on BCMA"), its outbound window from what bcma read out of the enumeration ROM. It still allocates its host bridge withdevm_pci_alloc_host_bridge(), which since commit669cbc708122("PCI: Move DT resource setup intodevm_pci_alloc_host_bridge()") parses ranges, dma-ranges and bus-range from the device's OF node and requests the windows it finds. None of that reaches the hardware here:need_ob_cfgis only ever set by the platform driver, soiproc_pcie_setup()never maps the parsed windows, and the EROM commit above had to throw them away again to keep them from colliding with its own window.Allocate the bridge with
pci_alloc_host_bridge()instead, so nothing from the devicetree is requested or handed to the PCI core, and free it on the error paths and inremove(). The driver now setsbridge->dev.parentitself, asdevm_pci_alloc_host_bridge()did, since the wifi nodes underpcie_bridge0inbcm4709-netgear-r8000.dtsare resolved through the root bus. The window request
moves from devm torequest_resource()andrelease_resource()because the resource lives inside the bridge allocation and has to be released before the bridge is freed.The ranges property is still read, but only to compare.
bcm-ns.dtsidescribes the same window for the platform driver, and on core revision0x01it points at an address the hardware does not decode. The warning from the EROM commit stays for that reason: a wrong dts is visible on BCMA boots, where nothing else would show it. Without bus-range the root bus also logs "No busn resource found for root bus, will use [bus 00-ff]" again, which changes nothing else.With
PCIE_IPROC_PLATFORMandPCIE_IPROC_BCMAboth enabled, the platform driver binds the same nodes first and claims the devicetree window. On core revision0x07that is the EROM window, so this driver's request still fails and the probe backs out as before. On revision0x01the devicetree window is elsewhere, so this driver now probes as well, next to a platform driver instance whose window the hardware does not decode. That instance does not work either; the only difference is that the second probe is no longer stopped by the collision.Quoted 3 lines
Tested on an ASUS RT-N18U (BCM47081, core revision
0x01): the warning lines are identical to the applied version,/proc/iomemand the enumerated devices are unchanged.
Quoted 6 lines
What do you think about this, Mani?
I still don't quite understand the issue with the DT and discovering this via both platform and bcma bus drivers, but we should make progress.
Sorry for the delay. Vacation + LPC kept me off for some time...
I read through the whole thread, and compared the driver + devicetree. Let me try to explain what I understood on this issue.
Before commit 767012397976, every Northstar SoC relied on the drivers/pci/controller/pcie-iproc-bcma.c driver to initialize the PCIe controllers present in the SoC. This driver only relied on the bcma bus driver for probing and getting the Host bridge resources such as MEM range, register base address etc... and ignored devicetree. So the devm_pci_alloc_host_bridge() call was only allocating the host bridge struct and devm_of_pci_bridge_init() ended up being a NOP.
But with the advent of the commit 767012397976, the PCIe nodes present in the bcm-ns.dtsi got compatibles and resource properties. Unfortunately, the values provided in the resource properties (ranges specifically), only matched the BCM47094 SoC. What the author overlooked was, they added these properties to the common bcm-ns.dtsi which was included by several other SoCs as well. Typically, the resources are only added to the common dtsi when they are common to all SoCs inheriting the common dtsi. Otherwise, they get added to the specific SoC dtsi. So this started all the issues as the added ranges properties were only valid for BCM47094 SoC and not on other SoCs using bcm-ns.dtsi. I believe the author was not aware of this incompatibility.
But then there was one more issue that the author of commit 767012397976 overlooked. When the compatible got added to the PCIe nodes, they got matched with the pcie-iproc-platform driver and that driver started driving the PCIe controller instead of the existing pcie-iproc-bcma driver. By probe ordering, I think the pcie-iproc-platform driver got probed first and it made use of the devicetree resources like the ranges property. Since the ranges value matched the BCM47094 SoC, the pcie-iproc-platform probed fine and the controllers were initialized properly.
But here is the catch, the author must've failed to notice the issue coming from pcie-iproc-bcma driver when it tried probing later. When the pcie-iproc-bcma driver probed, devm_of_pci_bridge_init() inside devm_pci_alloc_host_bridge() would've tried parsing the ranges property again and should've failed with -EBUSY as the memory was already mapped by the pcie-iproc-platform driver. Either the author didn't notice this or the pcie-iproc-bcma driver could've been disabled on their setup.
Now, when the same devicetree was used on other Northstar SoCs like BCM47081, BCM4709, the ranges value were invalid and the pcie-iproc-platform driver's probe got failed. Then the pcie-iproc-bcma driver also got probed later (it should be noted that these two are separate, so platform driver probe doesn't affect this bus driver probe), it also ended up parsing the wrong ranges from the devicetree. And this is what fixed by the patch from Semih [1] which forced the driver to use the ranges from EROM only and ignore devicetree. The reason why EROM parsing was added to the brcm driver instead of the platform driver was, only the brcm driver got access to the EROM region as it gets populated by the brcm bus code.
Even though [1] allowed the PCIe controllers to be working again, it didn't prevent the platform driver from probing first and claiming the invalid resource. For fixing this specific issue, I think we need two fixes:
1. Override the ranges property in the SoC dtsi with correct values.
2. Prevent the pcie-iproc-platform driver from probing for the Northstar SoCs and just rely on pcie-iproc-bcma driver to initialize the PCIe controllers. This can be achieved by the below patch:
drivers/pci/controller/pcie-iproc-platform.c +3 -0
@@ -45,6 +45,9 @@ static int iproc_pltfm_pcie_probe(struct platform_device *pdev)struct pci_host_bridge *bridge;int ret; +if (of_device_is_compatible (np->parent,"brcm,bus-axi" )) +return -EINVAL ; + bridge =devm_pci_alloc_host_bridge (dev,sizeof (*pcie));if (!bridge)return -ENOMEM ;
IMO, we need both fixes because, devicetree has to describe the hardware properly and we should not have two drivers claiming the same resource.
[1] https://lore.kernel.org/linux-pci/[email protected]
Quoted 5 lines
I *think* that if we squash this patch into
552aa843e4c5("PCI: iproc:
Use the EROM outbound window on BCMA") and usedevm_add_action_or_reset(), it will simplify down to this, which is less intrusive and would need less explanation:
This LGTM!
- Mani