Skip to contentSkip to conversation

PCI: dwc: Make Link Up IRQ logic handle already powered on PCIe switches

PATCH Superseded Niklas Cassel 5 participants 5 messages Nov 27, 2025 – Nov 29, 2025

Subsystems (6)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 PCI: dwc: Make Link Up IRQ logic handle already powered on PCIe switches

The DWC glue drivers always call pci_host_probe() during probe(), which will allocate upstream bridge resources and enumerate the bus.

For controllers without Link Up IRQ support, pci_host_probe() is called after dw_pcie_wait_for_link(), which will also wait the time required by the PCIe specification before performing PCI Configuration Space reads.

For controllers with Link Up IRQ support, the pci_host_probe() call (which will perform PCI Configuration Space reads) is done without any of the delays mandated by the PCIe specification.

For controllers with Link Up IRQ support, since the pci_host_probe() call is done without any delay (link training might still be ongoing), it is very unlikely that this scan will find any devices. Once the Link Up IRQ triggers, the Link Up IRQ handler will call pci_rescan_bus().

This works fine for PCIe endpoints connected to the Root Port, since they don't extend the bus. However, if the pci_rescan_bus() call detects a PCIe switch, then there will be a problem when the downstream busses starts showing up, because the PCIe controller is not hotplug capable, so we are not allowed to extend the subordinate bus number after the initial scan, resulting in error messages such as:

pci_bus 0004:43: busn_res: can not insert [bus 43-41] under [bus 42-41] (conflicts with (null) [bus 42-41]) pci_bus 0004:43: busn_res: [bus 43-41] end is updated to 43
pci_bus 0004:43: busn_res: can not insert [bus 43] under [bus 42-41] (conflicts with (null) [bus 42-41]) pci 0004:42:00.0: devices behind bridge are unusable because [bus 43] cannot be assigned for them pci_bus 0004:44: busn_res: can not insert [bus 44-41] under [bus 42-41] (conflicts with (null) [bus 42-41]) pci_bus 0004:44: busn_res: [bus 44-41] end is updated to 44
pci_bus 0004:44: busn_res: can not insert [bus 44] under [bus 42-41] (conflicts with (null) [bus 42-41]) pci 0004:42:02.0: devices behind bridge are unusable because [bus 44] cannot be assigned for them pci_bus 0004:45: busn_res: can not insert [bus 45-41] under [bus 42-41] (conflicts with (null) [bus 42-41]) pci_bus 0004:45: busn_res: [bus 45-41] end is updated to 45
pci_bus 0004:45: busn_res: can not insert [bus 45] under [bus 42-41] (conflicts with (null) [bus 42-41]) pci 0004:42:06.0: devices behind bridge are unusable because [bus 45] cannot be assigned for them pci_bus 0004:46: busn_res: can not insert [bus 46-41] under [bus 42-41] (conflicts with (null) [bus 42-41]) pci_bus 0004:46: busn_res: [bus 46-41] end is updated to 46
pci_bus 0004:46: busn_res: can not insert [bus 46] under [bus 42-41] (conflicts with (null) [bus 42-41]) pci 0004:42:0e.0: devices behind bridge are unusable because [bus 46] cannot be assigned for them pci_bus 0004:42: busn_res: [bus 42-41] end is updated to 46
pci_bus 0004:42: busn_res: can not insert [bus 42-46] under [bus 41] (conflicts with (null) [bus 41]) pci 0004:41:00.0: devices behind bridge are unusable because [bus 42-46] cannot be assigned for them pcieport 0004:40:00.0: bridge has subordinate 41 but max busn 46

While we would like to set the is_hotplug_bridge flag (quirk_hotplug_bridge()), many embedded SoCs that use the DWC controller have synthesized the controller without hot-plug support. Thus, the Link Up IRQ logic is only mimicking hot-plug functionality, i.e. it is not compliant with the PCI Hot-Plug Specification, so we cannot make use of the is_hotplug_bridge flag.

In order to let the Link Up IRQ logic handle PCIe switches that are already powered on (PCIe switches that not powered on already need to implement a pwrctrl driver), don't perform a pci_host_probe() call during probe() (which disregards the delays required by the PCIe specification).

Instead let the first Link Up IRQ call pci_host_probe(). Any follow up Link Up IRQ will call pci_rescan_bus().

Fixes ec9fd499b9c6 ("PCI: dw-rockchip: Don't wait for link since we can detect Link Up")
Fixes 0e0b45ab5d77 ("PCI: dw-rockchip: Enumerate endpoints based on dll_link_up IRQ")
---
 .../pci/controller/dwc/pcie-designware-host.c | 70 ++++++++++++++++---
 drivers/pci/controller/dwc/pcie-designware.h  |  5 ++
 drivers/pci/controller/dwc/pcie-dw-rockchip.c |  5 +-
 drivers/pci/controller/dwc/pcie-qcom.c        |  5 +-
 4 files changed, 68 insertions(+), 17 deletions(-)
drivers/pci/controller/dwc/pcie-designware-host.c +61 -9
index e92513c5bda51..8654346729574 100644
@@ -565,6 +565,59 @@ static int dw_pcie_host_get_resources(struct dw_pcie_rp *pp)
 	return 0;
 }
 
+static int dw_pcie_host_initial_scan(struct dw_pcie_rp *pp)
+{
+	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
+	struct pci_host_bridge *bridge = pp->bridge;
+	int ret;
+
+	ret = pci_host_probe(bridge);
+	if (ret)
+		return ret;
+
+	if (pp->ops->post_init)
+		pp->ops->post_init(pp);
+
+	dwc_pcie_debugfs_init(pci, DW_PCIE_RC_TYPE);
+
+	return 0;
+}
+
+void dw_pcie_handle_link_up_irq(struct dw_pcie_rp *pp)
+{
+	if (!pp->initial_linkup_irq_done) {
+		int ret;
+
+		ret = dw_pcie_host_initial_scan(pp);
+		if (ret) {
+			struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
+			struct device *dev = pci->dev;
+
+			dev_err(dev, "Initial scan from IRQ failed: %d\n", ret);
+
+			dw_pcie_stop_link(pci);
+
+			dw_pcie_edma_remove(pci);
+
+			if (pp->has_msi_ctrl)
+				dw_pcie_free_msi(pp);
+
+			if (pp->ops->deinit)
+				pp->ops->deinit(pp);
+
+			if (pp->cfg)
+				pci_ecam_free(pp->cfg);
+		} else {
+			pp->initial_linkup_irq_done = true;
+		}
+	} else {
+		/* Rescan the bus to enumerate endpoint devices */
+		pci_lock_rescan_remove();
+		pci_rescan_bus(pp->bridge->bus);
+		pci_unlock_rescan_remove();
+	}
+}
+
 int dw_pcie_host_init(struct dw_pcie_rp *pp)
 {
 	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
@@ -669,18 +722,17 @@ int dw_pcie_host_init(struct dw_pcie_rp *pp)
 	 * If there is no Link Up IRQ, we should not bypass the delay
 	 * because that would require users to manually rescan for devices.
 	 */
-	if (!pp->use_linkup_irq)
+	if (!pp->use_linkup_irq) {
 		/* Ignore errors, the link may come up later */
 		dw_pcie_wait_for_link(pci);
 
-	ret = pci_host_probe(bridge);
-	if (ret)
-		goto err_stop_link;
-
-	if (pp->ops->post_init)
-		pp->ops->post_init(pp);
-
-	dwc_pcie_debugfs_init(pci, DW_PCIE_RC_TYPE);
+		/*
+		 * For platforms with Link Up IRQ, initial scan will be done
+		 * on first Link Up IRQ.
+		 */
+		if (dw_pcie_host_initial_scan(pp))
+			goto err_stop_link;
+	}
 
 	return 0;
 
drivers/pci/controller/dwc/pcie-designware.h +5 -0
index e995f692a1ecd..a31bd93490dcd 100644
@@ -427,6 +427,7 @@ struct dw_pcie_rp {
 	int			msg_atu_index;
 	struct resource		*msg_res;
 	bool			use_linkup_irq;
+	bool			initial_linkup_irq_done;
 	struct pci_eq_presets	presets;
 	struct pci_config_window *cfg;
 	bool			ecam_enabled;
@@ -807,6 +808,7 @@ void dw_pcie_msi_init(struct dw_pcie_rp *pp);
 int dw_pcie_msi_host_init(struct dw_pcie_rp *pp);
 void dw_pcie_free_msi(struct dw_pcie_rp *pp);
 int dw_pcie_setup_rc(struct dw_pcie_rp *pp);
+void dw_pcie_handle_link_up_irq(struct dw_pcie_rp *pp);
 int dw_pcie_host_init(struct dw_pcie_rp *pp);
 void dw_pcie_host_deinit(struct dw_pcie_rp *pp);
 int dw_pcie_allocate_domains(struct dw_pcie_rp *pp);
@@ -844,6 +846,9 @@ static inline int dw_pcie_setup_rc(struct dw_pcie_rp *pp)
 	return 0;
 }
 
+static inline void dw_pcie_handle_link_up_irq(struct dw_pcie_rp *pp)
+{ }
+
 static inline int dw_pcie_host_init(struct dw_pcie_rp *pp)
 {
 	return 0;
drivers/pci/controller/dwc/pcie-dw-rockchip.c +1 -4
index 3e2752c7dd096..8f2cc1ef25e3d 100644
@@ -466,10 +466,7 @@ static irqreturn_t rockchip_pcie_rc_sys_irq_thread(int irq, void *arg)
 		if (rockchip_pcie_link_up(pci)) {
 			msleep(PCIE_RESET_CONFIG_WAIT_MS);
 			dev_dbg(dev, "Received Link up event. Starting enumeration!\n");
-			/* Rescan the bus to enumerate endpoint devices */
-			pci_lock_rescan_remove();
-			pci_rescan_bus(pp->bridge->bus);
-			pci_unlock_rescan_remove();
+			dw_pcie_handle_link_up_irq(pp);
 		}
 	}
 
drivers/pci/controller/dwc/pcie-qcom.c +1 -4
index c48a20602d7fa..2d8aca6630949 100644
@@ -1617,10 +1617,7 @@ static irqreturn_t qcom_pcie_global_irq_thread(int irq, void *data)
 	if (FIELD_GET(PARF_INT_ALL_LINK_UP, status)) {
 		msleep(PCIE_RESET_CONFIG_WAIT_MS);
 		dev_dbg(dev, "Received Link up event. Starting enumeration!\n");
-		/* Rescan the bus to enumerate endpoint devices */
-		pci_lock_rescan_remove();
-		pci_rescan_bus(pp->bridge->bus);
-		pci_unlock_rescan_remove();
+		dw_pcie_handle_link_up_irq(pp);
 
 		qcom_pcie_icc_opp_update(pcie);
 	} else {
Signature · 1 line
2.52.0
Tested-by on v1 1/1
#1 lore raw
Quoted 51 lines · Niklas Cassel

The DWC glue drivers always call pci_host_probe() during probe(), which will allocate upstream bridge resources and enumerate the bus.

For controllers without Link Up IRQ support, pci_host_probe() is called after dw_pcie_wait_for_link(), which will also wait the time required by the PCIe specification before performing PCI Configuration Space reads.

For controllers with Link Up IRQ support, the pci_host_probe() call (which will perform PCI Configuration Space reads) is done without any of the delays mandated by the PCIe specification.

For controllers with Link Up IRQ support, since the pci_host_probe() call is done without any delay (link training might still be ongoing), it is very unlikely that this scan will find any devices. Once the Link Up IRQ triggers, the Link Up IRQ handler will call pci_rescan_bus().

This works fine for PCIe endpoints connected to the Root Port, since they don't extend the bus. However, if the pci_rescan_bus() call detects a PCIe switch, then there will be a problem when the downstream busses starts showing up, because the PCIe controller is not hotplug capable, so we are not allowed to extend the subordinate bus number after the initial scan, resulting in error messages such as:

pci_bus 0004:43: busn_res: can not insert [bus 43-41] under [bus 42-41] (conflicts with (null) [bus 42-41]) pci_bus 0004:43: busn_res: [bus 43-41] end is updated to 43
pci_bus 0004:43: busn_res: can not insert [bus 43] under [bus 42-41] (conflicts with (null) [bus 42-41]) pci 0004:42:00.0: devices behind bridge are unusable because [bus 43] cannot be assigned for them pci_bus 0004:44: busn_res: can not insert [bus 44-41] under [bus 42-41] (conflicts with (null) [bus 42-41]) pci_bus 0004:44: busn_res: [bus 44-41] end is updated to 44
pci_bus 0004:44: busn_res: can not insert [bus 44] under [bus 42-41] (conflicts with (null) [bus 42-41]) pci 0004:42:02.0: devices behind bridge are unusable because [bus 44] cannot be assigned for them pci_bus 0004:45: busn_res: can not insert [bus 45-41] under [bus 42-41] (conflicts with (null) [bus 42-41]) pci_bus 0004:45: busn_res: [bus 45-41] end is updated to 45
pci_bus 0004:45: busn_res: can not insert [bus 45] under [bus 42-41] (conflicts with (null) [bus 42-41]) pci 0004:42:06.0: devices behind bridge are unusable because [bus 45] cannot be assigned for them pci_bus 0004:46: busn_res: can not insert [bus 46-41] under [bus 42-41] (conflicts with (null) [bus 42-41]) pci_bus 0004:46: busn_res: [bus 46-41] end is updated to 46
pci_bus 0004:46: busn_res: can not insert [bus 46] under [bus 42-41] (conflicts with (null) [bus 42-41]) pci 0004:42:0e.0: devices behind bridge are unusable because [bus 46] cannot be assigned for them pci_bus 0004:42: busn_res: [bus 42-41] end is updated to 46
pci_bus 0004:42: busn_res: can not insert [bus 42-46] under [bus 41] (conflicts with (null) [bus 41]) pci 0004:41:00.0: devices behind bridge are unusable because [bus 42-46] cannot be assigned for them pcieport 0004:40:00.0: bridge has subordinate 41 but max busn 46

While we would like to set the is_hotplug_bridge flag (quirk_hotplug_bridge()), many embedded SoCs that use the DWC controller have synthesized the controller without hot-plug support. Thus, the Link Up IRQ logic is only mimicking hot-plug functionality, i.e. it is not compliant with the PCI Hot-Plug Specification, so we cannot make use of the is_hotplug_bridge flag.

Quoted 8 lines

In order to let the Link Up IRQ logic handle PCIe switches that are already powered on (PCIe switches that not powered on already need to implement a pwrctrl driver), don't perform a pci_host_probe() call during probe() (which disregards the delays required by the PCIe specification).

Instead let the first Link Up IRQ call pci_host_probe(). Any follow up Link Up IRQ will call pci_rescan_bus().

Thanks for fixing this, I'v tested it on RK3588S-EVB1 with both of PCIe switch + 2 NVMe and NVMe directly connected.

FWIW,

Quoted 165 lines
Fixes ec9fd499b9c6 ("PCI: dw-rockchip: Don't wait for link since we can detect Link Up")
Fixes 0e0b45ab5d77 ("PCI: dw-rockchip: Enumerate endpoints based on dll_link_up IRQ")
---
  .../pci/controller/dwc/pcie-designware-host.c | 70 ++++++++++++++++---
  drivers/pci/controller/dwc/pcie-designware.h  |  5 ++
  drivers/pci/controller/dwc/pcie-dw-rockchip.c |  5 +-
  drivers/pci/controller/dwc/pcie-qcom.c        |  5 +-
  4 files changed, 68 insertions(+), 17 deletions(-)
drivers/pci/controller/dwc/pcie-designware-host.c +61 -9
index e92513c5bda51..8654346729574 100644
@@ -565,6 +565,59 @@ static int dw_pcie_host_get_resources(struct dw_pcie_rp *pp)
  	return 0;
  }
 
+static int dw_pcie_host_initial_scan(struct dw_pcie_rp *pp)
+{
+	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
+	struct pci_host_bridge *bridge = pp->bridge;
+	int ret;
+
+	ret = pci_host_probe(bridge);
+	if (ret)
+		return ret;
+
+	if (pp->ops->post_init)
+		pp->ops->post_init(pp);
+
+	dwc_pcie_debugfs_init(pci, DW_PCIE_RC_TYPE);
+
+	return 0;
+}
+
+void dw_pcie_handle_link_up_irq(struct dw_pcie_rp *pp)
+{
+	if (!pp->initial_linkup_irq_done) {
+		int ret;
+
+		ret = dw_pcie_host_initial_scan(pp);
+		if (ret) {
+			struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
+			struct device *dev = pci->dev;
+
+			dev_err(dev, "Initial scan from IRQ failed: %d\n", ret);
+
+			dw_pcie_stop_link(pci);
+
+			dw_pcie_edma_remove(pci);
+
+			if (pp->has_msi_ctrl)
+				dw_pcie_free_msi(pp);
+
+			if (pp->ops->deinit)
+				pp->ops->deinit(pp);
+
+			if (pp->cfg)
+				pci_ecam_free(pp->cfg);
+		} else {
+			pp->initial_linkup_irq_done = true;
+		}
+	} else {
+		/* Rescan the bus to enumerate endpoint devices */
+		pci_lock_rescan_remove();
+		pci_rescan_bus(pp->bridge->bus);
+		pci_unlock_rescan_remove();
+	}
+}
+
  int dw_pcie_host_init(struct dw_pcie_rp *pp)
  {
  	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
@@ -669,18 +722,17 @@ int dw_pcie_host_init(struct dw_pcie_rp *pp)
  	 * If there is no Link Up IRQ, we should not bypass the delay
  	 * because that would require users to manually rescan for devices.
  	 */
-	if (!pp->use_linkup_irq)
+	if (!pp->use_linkup_irq) {
  		/* Ignore errors, the link may come up later */
  		dw_pcie_wait_for_link(pci);
 
-	ret = pci_host_probe(bridge);
-	if (ret)
-		goto err_stop_link;
-
-	if (pp->ops->post_init)
-		pp->ops->post_init(pp);
-
-	dwc_pcie_debugfs_init(pci, DW_PCIE_RC_TYPE);
+		/*
+		 * For platforms with Link Up IRQ, initial scan will be done
+		 * on first Link Up IRQ.
+		 */
+		if (dw_pcie_host_initial_scan(pp))
+			goto err_stop_link;
+	}
 
  	return 0;
 
drivers/pci/controller/dwc/pcie-designware.h +5 -0
index e995f692a1ecd..a31bd93490dcd 100644
@@ -427,6 +427,7 @@ struct dw_pcie_rp {
  	int			msg_atu_index;
  	struct resource		*msg_res;
  	bool			use_linkup_irq;
+	bool			initial_linkup_irq_done;
  	struct pci_eq_presets	presets;
  	struct pci_config_window *cfg;
  	bool			ecam_enabled;
@@ -807,6 +808,7 @@ void dw_pcie_msi_init(struct dw_pcie_rp *pp);
  int dw_pcie_msi_host_init(struct dw_pcie_rp *pp);
  void dw_pcie_free_msi(struct dw_pcie_rp *pp);
  int dw_pcie_setup_rc(struct dw_pcie_rp *pp);
+void dw_pcie_handle_link_up_irq(struct dw_pcie_rp *pp);
  int dw_pcie_host_init(struct dw_pcie_rp *pp);
  void dw_pcie_host_deinit(struct dw_pcie_rp *pp);
  int dw_pcie_allocate_domains(struct dw_pcie_rp *pp);
@@ -844,6 +846,9 @@ static inline int dw_pcie_setup_rc(struct dw_pcie_rp *pp)
  	return 0;
  }
 
+static inline void dw_pcie_handle_link_up_irq(struct dw_pcie_rp *pp)
+{ }
+
  static inline int dw_pcie_host_init(struct dw_pcie_rp *pp)
  {
  	return 0;
drivers/pci/controller/dwc/pcie-dw-rockchip.c +1 -4
index 3e2752c7dd096..8f2cc1ef25e3d 100644
@@ -466,10 +466,7 @@ static irqreturn_t rockchip_pcie_rc_sys_irq_thread(int irq, void *arg)
  		if (rockchip_pcie_link_up(pci)) {
  			msleep(PCIE_RESET_CONFIG_WAIT_MS);
  			dev_dbg(dev, "Received Link up event. Starting enumeration!\n");
-			/* Rescan the bus to enumerate endpoint devices */
-			pci_lock_rescan_remove();
-			pci_rescan_bus(pp->bridge->bus);
-			pci_unlock_rescan_remove();
+			dw_pcie_handle_link_up_irq(pp);
  		}
  	}
 
drivers/pci/controller/dwc/pcie-qcom.c +1 -4
index c48a20602d7fa..2d8aca6630949 100644
@@ -1617,10 +1617,7 @@ static irqreturn_t qcom_pcie_global_irq_thread(int irq, void *data)
  	if (FIELD_GET(PARF_INT_ALL_LINK_UP, status)) {
  		msleep(PCIE_RESET_CONFIG_WAIT_MS);
  		dev_dbg(dev, "Received Link up event. Starting enumeration!\n");
-		/* Rescan the bus to enumerate endpoint devices */
-		pci_lock_rescan_remove();
-		pci_rescan_bus(pp->bridge->bus);
-		pci_unlock_rescan_remove();
+		dw_pcie_handle_link_up_irq(pp);
 
  		qcom_pcie_icc_opp_update(pcie);
  	} else {
Tested-by on v1 1/1
#2 lore raw
In reply to Niklas Cassel

Hi Niklas,

Quoted 51 lines · Niklas Cassel

The DWC glue drivers always call pci_host_probe() during probe(), which will allocate upstream bridge resources and enumerate the bus.

For controllers without Link Up IRQ support, pci_host_probe() is called after dw_pcie_wait_for_link(), which will also wait the time required by the PCIe specification before performing PCI Configuration Space reads.

For controllers with Link Up IRQ support, the pci_host_probe() call (which will perform PCI Configuration Space reads) is done without any of the delays mandated by the PCIe specification.

For controllers with Link Up IRQ support, since the pci_host_probe() call is done without any delay (link training might still be ongoing), it is very unlikely that this scan will find any devices. Once the Link Up IRQ triggers, the Link Up IRQ handler will call pci_rescan_bus().

This works fine for PCIe endpoints connected to the Root Port, since they don't extend the bus. However, if the pci_rescan_bus() call detects a PCIe switch, then there will be a problem when the downstream busses starts showing up, because the PCIe controller is not hotplug capable, so we are not allowed to extend the subordinate bus number after the initial scan, resulting in error messages such as:

pci_bus 0004:43: busn_res: can not insert [bus 43-41] under [bus 42-41] (conflicts with (null) [bus 42-41]) pci_bus 0004:43: busn_res: [bus 43-41] end is updated to 43
pci_bus 0004:43: busn_res: can not insert [bus 43] under [bus 42-41] (conflicts with (null) [bus 42-41]) pci 0004:42:00.0: devices behind bridge are unusable because [bus 43] cannot be assigned for them pci_bus 0004:44: busn_res: can not insert [bus 44-41] under [bus 42-41] (conflicts with (null) [bus 42-41]) pci_bus 0004:44: busn_res: [bus 44-41] end is updated to 44
pci_bus 0004:44: busn_res: can not insert [bus 44] under [bus 42-41] (conflicts with (null) [bus 42-41]) pci 0004:42:02.0: devices behind bridge are unusable because [bus 44] cannot be assigned for them pci_bus 0004:45: busn_res: can not insert [bus 45-41] under [bus 42-41] (conflicts with (null) [bus 42-41]) pci_bus 0004:45: busn_res: [bus 45-41] end is updated to 45
pci_bus 0004:45: busn_res: can not insert [bus 45] under [bus 42-41] (conflicts with (null) [bus 42-41]) pci 0004:42:06.0: devices behind bridge are unusable because [bus 45] cannot be assigned for them pci_bus 0004:46: busn_res: can not insert [bus 46-41] under [bus 42-41] (conflicts with (null) [bus 42-41]) pci_bus 0004:46: busn_res: [bus 46-41] end is updated to 46
pci_bus 0004:46: busn_res: can not insert [bus 46] under [bus 42-41] (conflicts with (null) [bus 42-41]) pci 0004:42:0e.0: devices behind bridge are unusable because [bus 46] cannot be assigned for them pci_bus 0004:42: busn_res: [bus 42-41] end is updated to 46
pci_bus 0004:42: busn_res: can not insert [bus 42-46] under [bus 41] (conflicts with (null) [bus 41]) pci 0004:41:00.0: devices behind bridge are unusable because [bus 42-46] cannot be assigned for them pcieport 0004:40:00.0: bridge has subordinate 41 but max busn 46

While we would like to set the is_hotplug_bridge flag (quirk_hotplug_bridge()), many embedded SoCs that use the DWC controller have synthesized the controller without hot-plug support. Thus, the Link Up IRQ logic is only mimicking hot-plug functionality, i.e. it is not compliant with the PCI Hot-Plug Specification, so we cannot make use of the is_hotplug_bridge flag.

Quoted 7 lines

In order to let the Link Up IRQ logic handle PCIe switches that are already powered on (PCIe switches that not powered on already need to implement a pwrctrl driver), don't perform a pci_host_probe() call during probe() (which disregards the delays required by the PCIe specification).

Instead let the first Link Up IRQ call pci_host_probe(). Any follow up Link Up IRQ will call pci_rescan_bus().

I'm pleased to inform you that your patch worked perfectly with devices behind the ASM2806 switch (Radxa Dual 2.5G Router HAT) connected to the PCIe 2.1 bus of the Radxa ROCK 5A, 5B, and 5C Lite.

So,

Thank you very much for your work, and I apologize for any confusion caused by my test report.

Best regards,

--
FUKAUMI Naoki
Radxa Computer (Shenzhen) Co., Ltd.

Quoted 165 lines
Fixes ec9fd499b9c6 ("PCI: dw-rockchip: Don't wait for link since we can detect Link Up")
Fixes 0e0b45ab5d77 ("PCI: dw-rockchip: Enumerate endpoints based on dll_link_up IRQ")
---
  .../pci/controller/dwc/pcie-designware-host.c | 70 ++++++++++++++++---
  drivers/pci/controller/dwc/pcie-designware.h  |  5 ++
  drivers/pci/controller/dwc/pcie-dw-rockchip.c |  5 +-
  drivers/pci/controller/dwc/pcie-qcom.c        |  5 +-
  4 files changed, 68 insertions(+), 17 deletions(-)
drivers/pci/controller/dwc/pcie-designware-host.c +61 -9
index e92513c5bda51..8654346729574 100644
@@ -565,6 +565,59 @@ static int dw_pcie_host_get_resources(struct dw_pcie_rp *pp)
  	return 0;
  }
 
+static int dw_pcie_host_initial_scan(struct dw_pcie_rp *pp)
+{
+	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
+	struct pci_host_bridge *bridge = pp->bridge;
+	int ret;
+
+	ret = pci_host_probe(bridge);
+	if (ret)
+		return ret;
+
+	if (pp->ops->post_init)
+		pp->ops->post_init(pp);
+
+	dwc_pcie_debugfs_init(pci, DW_PCIE_RC_TYPE);
+
+	return 0;
+}
+
+void dw_pcie_handle_link_up_irq(struct dw_pcie_rp *pp)
+{
+	if (!pp->initial_linkup_irq_done) {
+		int ret;
+
+		ret = dw_pcie_host_initial_scan(pp);
+		if (ret) {
+			struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
+			struct device *dev = pci->dev;
+
+			dev_err(dev, "Initial scan from IRQ failed: %d\n", ret);
+
+			dw_pcie_stop_link(pci);
+
+			dw_pcie_edma_remove(pci);
+
+			if (pp->has_msi_ctrl)
+				dw_pcie_free_msi(pp);
+
+			if (pp->ops->deinit)
+				pp->ops->deinit(pp);
+
+			if (pp->cfg)
+				pci_ecam_free(pp->cfg);
+		} else {
+			pp->initial_linkup_irq_done = true;
+		}
+	} else {
+		/* Rescan the bus to enumerate endpoint devices */
+		pci_lock_rescan_remove();
+		pci_rescan_bus(pp->bridge->bus);
+		pci_unlock_rescan_remove();
+	}
+}
+
  int dw_pcie_host_init(struct dw_pcie_rp *pp)
  {
  	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
@@ -669,18 +722,17 @@ int dw_pcie_host_init(struct dw_pcie_rp *pp)
  	 * If there is no Link Up IRQ, we should not bypass the delay
  	 * because that would require users to manually rescan for devices.
  	 */
-	if (!pp->use_linkup_irq)
+	if (!pp->use_linkup_irq) {
  		/* Ignore errors, the link may come up later */
  		dw_pcie_wait_for_link(pci);
 
-	ret = pci_host_probe(bridge);
-	if (ret)
-		goto err_stop_link;
-
-	if (pp->ops->post_init)
-		pp->ops->post_init(pp);
-
-	dwc_pcie_debugfs_init(pci, DW_PCIE_RC_TYPE);
+		/*
+		 * For platforms with Link Up IRQ, initial scan will be done
+		 * on first Link Up IRQ.
+		 */
+		if (dw_pcie_host_initial_scan(pp))
+			goto err_stop_link;
+	}
 
  	return 0;
 
drivers/pci/controller/dwc/pcie-designware.h +5 -0
index e995f692a1ecd..a31bd93490dcd 100644
@@ -427,6 +427,7 @@ struct dw_pcie_rp {
  	int			msg_atu_index;
  	struct resource		*msg_res;
  	bool			use_linkup_irq;
+	bool			initial_linkup_irq_done;
  	struct pci_eq_presets	presets;
  	struct pci_config_window *cfg;
  	bool			ecam_enabled;
@@ -807,6 +808,7 @@ void dw_pcie_msi_init(struct dw_pcie_rp *pp);
  int dw_pcie_msi_host_init(struct dw_pcie_rp *pp);
  void dw_pcie_free_msi(struct dw_pcie_rp *pp);
  int dw_pcie_setup_rc(struct dw_pcie_rp *pp);
+void dw_pcie_handle_link_up_irq(struct dw_pcie_rp *pp);
  int dw_pcie_host_init(struct dw_pcie_rp *pp);
  void dw_pcie_host_deinit(struct dw_pcie_rp *pp);
  int dw_pcie_allocate_domains(struct dw_pcie_rp *pp);
@@ -844,6 +846,9 @@ static inline int dw_pcie_setup_rc(struct dw_pcie_rp *pp)
  	return 0;
  }
 
+static inline void dw_pcie_handle_link_up_irq(struct dw_pcie_rp *pp)
+{ }
+
  static inline int dw_pcie_host_init(struct dw_pcie_rp *pp)
  {
  	return 0;
drivers/pci/controller/dwc/pcie-dw-rockchip.c +1 -4
index 3e2752c7dd096..8f2cc1ef25e3d 100644
@@ -466,10 +466,7 @@ static irqreturn_t rockchip_pcie_rc_sys_irq_thread(int irq, void *arg)
  		if (rockchip_pcie_link_up(pci)) {
  			msleep(PCIE_RESET_CONFIG_WAIT_MS);
  			dev_dbg(dev, "Received Link up event. Starting enumeration!\n");
-			/* Rescan the bus to enumerate endpoint devices */
-			pci_lock_rescan_remove();
-			pci_rescan_bus(pp->bridge->bus);
-			pci_unlock_rescan_remove();
+			dw_pcie_handle_link_up_irq(pp);
  		}
  	}
 
drivers/pci/controller/dwc/pcie-qcom.c +1 -4
index c48a20602d7fa..2d8aca6630949 100644
@@ -1617,10 +1617,7 @@ static irqreturn_t qcom_pcie_global_irq_thread(int irq, void *data)
  	if (FIELD_GET(PARF_INT_ALL_LINK_UP, status)) {
  		msleep(PCIE_RESET_CONFIG_WAIT_MS);
  		dev_dbg(dev, "Received Link up event. Starting enumeration!\n");
-		/* Rescan the bus to enumerate endpoint devices */
-		pci_lock_rescan_remove();
-		pci_rescan_bus(pp->bridge->bus);
-		pci_unlock_rescan_remove();
+		dw_pcie_handle_link_up_irq(pp);
 
  		qcom_pcie_icc_opp_update(pcie);
  	} else {
build bot
#3 lore raw
In reply to Niklas Cassel
BCC: [email protected] CC: [email protected] In-Reply-To: <[email protected]> References: <[email protected]> TO: Niklas Cassel…

BCC: [email protected]
CC: [email protected]
In-Reply-To: <[email protected]>
References: <[email protected]>
TO: Niklas Cassel <[email protected]>
TO: Jingoo Han <[email protected]>
TO: Manivannan Sadhasivam <[email protected]>
TO: Lorenzo Pieralisi <[email protected]>
TO: "Krzysztof Wilczyński" <[email protected]>
TO: Rob Herring <[email protected]>
TO: Bjorn Helgaas <[email protected]>
TO: Heiko Stuebner <[email protected]>
TO: Niklas Cassel <[email protected]>
CC: FUKAUMI Naoki <[email protected]>
CC: [email protected]
CC: [email protected]
CC: [email protected]
CC: [email protected]

Hi Niklas,

kernel test robot noticed the following build warnings:

[auto build test WARNING on pci/next]
[also build test WARNING on pci/for-linus mani-mhi/mhi-next linus/master v6.18-rc7 next-20251128] [If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Niklas-Cassel/PCI-dwc-Make-Link-Up-IRQ-logic-handle-already-powered-on-PCIe-switches/20251127-214649
base:   https://git.kernel.org/pub/scm/linux/kernel/git/pci/pci.git next
patch link:    https://lore.kernel.org/r/20251127134318.3655052-2-cassel%40kernel.org
patch subject: [PATCH] PCI: dwc: Make Link Up IRQ logic handle already powered on PCIe switches

:::::: branch date: 29 hours ago
:::::: commit date: 29 hours ago
config: x86_64-randconfig-r071-20251128 (https://download.01.org/0day-ci/archive/20251129/[email protected]/config)
compiler: gcc-14 (Debian 14.2.0-19) 14.2.0

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags

| Reported-by: kernel test robot <[email protected]>
| Reported-by: Dan Carpenter <[email protected]>
| Closes: https://lore.kernel.org/r/[email protected]/

smatch warnings:
drivers/pci/controller/dwc/pcie-designware-host.c:737 dw_pcie_host_init() warn: missing error code 'ret'

vim +/ret +737 drivers/pci/controller/dwc/pcie-designware-host.c

cd723d3dce14ac drivers/pci/controller/dwc/pcie-designware-host.c Niklas Cassel             2025-11-27  623  
2ce107e064574a drivers/pci/controller/dwc/pcie-designware-host.c Bjorn Helgaas             2025-03-15  624  int dw_pcie_host_init(struct dw_pcie_rp *pp)
2ce107e064574a drivers/pci/controller/dwc/pcie-designware-host.c Bjorn Helgaas             2025-03-15  625  {
2ce107e064574a drivers/pci/controller/dwc/pcie-designware-host.c Bjorn Helgaas             2025-03-15  626  	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
2ce107e064574a drivers/pci/controller/dwc/pcie-designware-host.c Bjorn Helgaas             2025-03-15  627  	struct device *dev = pci->dev;
2ce107e064574a drivers/pci/controller/dwc/pcie-designware-host.c Bjorn Helgaas             2025-03-15  628  	struct device_node *np = dev->of_node;
2ce107e064574a drivers/pci/controller/dwc/pcie-designware-host.c Bjorn Helgaas             2025-03-15  629  	struct pci_host_bridge *bridge;
2ce107e064574a drivers/pci/controller/dwc/pcie-designware-host.c Bjorn Helgaas             2025-03-15  630  	int ret;
2ce107e064574a drivers/pci/controller/dwc/pcie-designware-host.c Bjorn Helgaas             2025-03-15  631  
2ce107e064574a drivers/pci/controller/dwc/pcie-designware-host.c Bjorn Helgaas             2025-03-15  632  	raw_spin_lock_init(&pp->lock);
2ce107e064574a drivers/pci/controller/dwc/pcie-designware-host.c Bjorn Helgaas             2025-03-15  633  
2ce107e064574a drivers/pci/controller/dwc/pcie-designware-host.c Bjorn Helgaas             2025-03-15  634  	bridge = devm_pci_alloc_host_bridge(dev, 0);
2ce107e064574a drivers/pci/controller/dwc/pcie-designware-host.c Bjorn Helgaas             2025-03-15  635  	if (!bridge)
2ce107e064574a drivers/pci/controller/dwc/pcie-designware-host.c Bjorn Helgaas             2025-03-15  636  		return -ENOMEM;
2ce107e064574a drivers/pci/controller/dwc/pcie-designware-host.c Bjorn Helgaas             2025-03-15  637  
2ce107e064574a drivers/pci/controller/dwc/pcie-designware-host.c Bjorn Helgaas             2025-03-15  638  	pp->bridge = bridge;
2ce107e064574a drivers/pci/controller/dwc/pcie-designware-host.c Bjorn Helgaas             2025-03-15  639  
2ce107e064574a drivers/pci/controller/dwc/pcie-designware-host.c Bjorn Helgaas             2025-03-15  640  	ret = dw_pcie_host_get_resources(pp);
2ce107e064574a drivers/pci/controller/dwc/pcie-designware-host.c Bjorn Helgaas             2025-03-15  641  	if (ret)
2ce107e064574a drivers/pci/controller/dwc/pcie-designware-host.c Bjorn Helgaas             2025-03-15  642  		return ret;
2ce107e064574a drivers/pci/controller/dwc/pcie-designware-host.c Bjorn Helgaas             2025-03-15  643  
aea370b2aec9d3 drivers/pci/controller/dwc/pcie-designware-host.c Yoshihiro Shimoda         2023-12-20  644  	if (pp->ops->init) {
aea370b2aec9d3 drivers/pci/controller/dwc/pcie-designware-host.c Yoshihiro Shimoda         2023-12-20  645  		ret = pp->ops->init(pp);
7e919677bb392c drivers/pci/controller/dwc/pcie-designware-host.c Bjorn Andersson           2021-08-23  646  		if (ret)
f6fd357f7afbeb drivers/pci/controller/dwc/pcie-designware-host.c Krishna Chaitanya Chundru 2025-09-23  647  			goto err_free_ecam;
7e919677bb392c drivers/pci/controller/dwc/pcie-designware-host.c Bjorn Andersson           2021-08-23  648  	}
7e919677bb392c drivers/pci/controller/dwc/pcie-designware-host.c Bjorn Andersson           2021-08-23  649  
9e2b5de5604a6f drivers/pci/controller/dwc/pcie-designware-host.c Jisheng Zhang             2019-03-29  650  	if (pci_msi_enabled()) {
aea370b2aec9d3 drivers/pci/controller/dwc/pcie-designware-host.c Yoshihiro Shimoda         2023-12-20  651  		pp->has_msi_ctrl = !(pp->ops->msi_init ||
154fc1f6420840 drivers/pci/controller/dwc/pcie-designware-host.c Rob Herring (Arm          2024-11-04  652) 				     of_property_present(np, "msi-parent") ||
154fc1f6420840 drivers/pci/controller/dwc/pcie-designware-host.c Rob Herring (Arm          2024-11-04  653) 				     of_property_present(np, "msi-map"));
f78f02638af594 drivers/pci/controller/dwc/pcie-designware-host.c Rob Herring               2020-11-05  654  
cd761378e62c26 drivers/pci/controller/dwc/pcie-designware-host.c Dmitry Baryshkov          2022-07-07  655  		/*
cd761378e62c26 drivers/pci/controller/dwc/pcie-designware-host.c Dmitry Baryshkov          2022-07-07  656  		 * For the has_msi_ctrl case the default assignment is handled
cd761378e62c26 drivers/pci/controller/dwc/pcie-designware-host.c Dmitry Baryshkov          2022-07-07  657  		 * in the dw_pcie_msi_host_init().
cd761378e62c26 drivers/pci/controller/dwc/pcie-designware-host.c Dmitry Baryshkov          2022-07-07  658  		 */
cd761378e62c26 drivers/pci/controller/dwc/pcie-designware-host.c Dmitry Baryshkov          2022-07-07  659  		if (!pp->has_msi_ctrl && !pp->num_vectors) {
7c5925afbc58c6 drivers/pci/dwc/pcie-designware-host.c            Gustavo Pimentel          2018-03-06  660  			pp->num_vectors = MSI_DEF_NUM_VECTORS;
331e9bcead5252 drivers/pci/controller/dwc/pcie-designware-host.c Rob Herring               2020-11-05  661  		} else if (pp->num_vectors > MAX_MSI_IRQS) {
331e9bcead5252 drivers/pci/controller/dwc/pcie-designware-host.c Rob Herring               2020-11-05  662  			dev_err(dev, "Invalid number of vectors\n");
c6481d51dc65f2 drivers/pci/controller/dwc/pcie-designware-host.c Serge Semin               2022-06-24  663  			ret = -EINVAL;
c6481d51dc65f2 drivers/pci/controller/dwc/pcie-designware-host.c Serge Semin               2022-06-24  664  			goto err_deinit_host;
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I    2017-02-15  665  		}
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I    2017-02-15  666  
aea370b2aec9d3 drivers/pci/controller/dwc/pcie-designware-host.c Yoshihiro Shimoda         2023-12-20  667  		if (pp->ops->msi_init) {
aea370b2aec9d3 drivers/pci/controller/dwc/pcie-designware-host.c Yoshihiro Shimoda         2023-12-20  668  			ret = pp->ops->msi_init(pp);
f78f02638af594 drivers/pci/controller/dwc/pcie-designware-host.c Rob Herring               2020-11-05  669  			if (ret < 0)
c6481d51dc65f2 drivers/pci/controller/dwc/pcie-designware-host.c Serge Semin               2022-06-24  670  				goto err_deinit_host;
f78f02638af594 drivers/pci/controller/dwc/pcie-designware-host.c Rob Herring               2020-11-05  671  		} else if (pp->has_msi_ctrl) {
226ec087497a6a drivers/pci/controller/dwc/pcie-designware-host.c Dmitry Baryshkov          2022-07-22  672  			ret = dw_pcie_msi_host_init(pp);
226ec087497a6a drivers/pci/controller/dwc/pcie-designware-host.c Dmitry Baryshkov          2022-07-22  673  			if (ret < 0)
c6481d51dc65f2 drivers/pci/controller/dwc/pcie-designware-host.c Serge Semin               2022-06-24  674  				goto err_deinit_host;
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I    2017-02-15  675  		}
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I    2017-02-15  676  	}
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I    2017-02-15  677  
13e9d3900c2024 drivers/pci/controller/dwc/pcie-designware-host.c Serge Semin               2022-06-24  678  	dw_pcie_version_detect(pci);
13e9d3900c2024 drivers/pci/controller/dwc/pcie-designware-host.c Serge Semin               2022-06-24  679  
8bcca26585585a drivers/pci/controller/dwc/pcie-designware-host.c Hou Zhiqiang              2021-04-13  680  	dw_pcie_iatu_detect(pci);
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I    2017-02-15  681  
f9eb654fb194e7 drivers/pci/controller/dwc/pcie-designware-host.c Krishna Chaitanya Chundru 2025-03-28  682  	if (pci->num_lanes < 1)
f9eb654fb194e7 drivers/pci/controller/dwc/pcie-designware-host.c Krishna Chaitanya Chundru 2025-03-28  683  		pci->num_lanes = dw_pcie_link_get_max_link_width(pci);
f9eb654fb194e7 drivers/pci/controller/dwc/pcie-designware-host.c Krishna Chaitanya Chundru 2025-03-28  684  
09483959e34d25 drivers/pci/controller/dwc/pcie-designware-host.c Krishna Chaitanya Chundru 2025-03-28  685  	ret = of_pci_get_equalization_presets(dev, &pp->presets, pci->num_lanes);
09483959e34d25 drivers/pci/controller/dwc/pcie-designware-host.c Krishna Chaitanya Chundru 2025-03-28  686  	if (ret)
09483959e34d25 drivers/pci/controller/dwc/pcie-designware-host.c Krishna Chaitanya Chundru 2025-03-28  687  		goto err_free_msi;
09483959e34d25 drivers/pci/controller/dwc/pcie-designware-host.c Krishna Chaitanya Chundru 2025-03-28  688  
f6fd357f7afbeb drivers/pci/controller/dwc/pcie-designware-host.c Krishna Chaitanya Chundru 2025-09-23  689  	if (pp->ecam_enabled) {
f6fd357f7afbeb drivers/pci/controller/dwc/pcie-designware-host.c Krishna Chaitanya Chundru 2025-09-23  690  		ret = dw_pcie_config_ecam_iatu(pp);
f6fd357f7afbeb drivers/pci/controller/dwc/pcie-designware-host.c Krishna Chaitanya Chundru 2025-09-23  691  		if (ret) {
f6fd357f7afbeb drivers/pci/controller/dwc/pcie-designware-host.c Krishna Chaitanya Chundru 2025-09-23  692  			dev_err(dev, "Failed to configure iATU in ECAM mode\n");
f6fd357f7afbeb drivers/pci/controller/dwc/pcie-designware-host.c Krishna Chaitanya Chundru 2025-09-23  693  			goto err_free_msi;
f6fd357f7afbeb drivers/pci/controller/dwc/pcie-designware-host.c Krishna Chaitanya Chundru 2025-09-23  694  		}
f6fd357f7afbeb drivers/pci/controller/dwc/pcie-designware-host.c Krishna Chaitanya Chundru 2025-09-23  695  	}
f6fd357f7afbeb drivers/pci/controller/dwc/pcie-designware-host.c Krishna Chaitanya Chundru 2025-09-23  696  
e1a4ec1a9520bf drivers/pci/controller/dwc/pcie-designware-host.c Frank Li                  2024-04-18  697  	/*
e1a4ec1a9520bf drivers/pci/controller/dwc/pcie-designware-host.c Frank Li                  2024-04-18  698  	 * Allocate the resource for MSG TLP before programming the iATU
e1a4ec1a9520bf drivers/pci/controller/dwc/pcie-designware-host.c Frank Li                  2024-04-18  699  	 * outbound window in dw_pcie_setup_rc(). Since the allocation depends
e1a4ec1a9520bf drivers/pci/controller/dwc/pcie-designware-host.c Frank Li                  2024-04-18  700  	 * on the value of 'region_align', this has to be done after
e1a4ec1a9520bf drivers/pci/controller/dwc/pcie-designware-host.c Frank Li                  2024-04-18  701  	 * dw_pcie_iatu_detect().
e1a4ec1a9520bf drivers/pci/controller/dwc/pcie-designware-host.c Frank Li                  2024-04-18  702  	 *
e1a4ec1a9520bf drivers/pci/controller/dwc/pcie-designware-host.c Frank Li                  2024-04-18  703  	 * Glue drivers need to set 'use_atu_msg' before dw_pcie_host_init() to
e1a4ec1a9520bf drivers/pci/controller/dwc/pcie-designware-host.c Frank Li                  2024-04-18  704  	 * make use of the generic MSG TLP implementation.
e1a4ec1a9520bf drivers/pci/controller/dwc/pcie-designware-host.c Frank Li                  2024-04-18  705  	 */
e1a4ec1a9520bf drivers/pci/controller/dwc/pcie-designware-host.c Frank Li                  2024-04-18  706  	if (pp->use_atu_msg)
e1a4ec1a9520bf drivers/pci/controller/dwc/pcie-designware-host.c Frank Li                  2024-04-18  707  		dw_pcie_host_request_msg_tlp_res(pp);
e1a4ec1a9520bf drivers/pci/controller/dwc/pcie-designware-host.c Frank Li                  2024-04-18  708  
939fbcd568fd29 drivers/pci/controller/dwc/pcie-designware-host.c Serge Semin               2023-01-13  709  	ret = dw_pcie_edma_detect(pci);
ce06bf570390fb drivers/pci/controller/dwc/pcie-designware-host.c Serge Semin               2022-06-24  710  	if (ret)
ce06bf570390fb drivers/pci/controller/dwc/pcie-designware-host.c Serge Semin               2022-06-24  711  		goto err_free_msi;
59fbab1ae40eb0 drivers/pci/controller/dwc/pcie-designware-host.c Rob Herring               2020-11-05  712  
939fbcd568fd29 drivers/pci/controller/dwc/pcie-designware-host.c Serge Semin               2023-01-13  713  	ret = dw_pcie_setup_rc(pp);
939fbcd568fd29 drivers/pci/controller/dwc/pcie-designware-host.c Serge Semin               2023-01-13  714  	if (ret)
939fbcd568fd29 drivers/pci/controller/dwc/pcie-designware-host.c Serge Semin               2023-01-13  715  		goto err_remove_edma;
939fbcd568fd29 drivers/pci/controller/dwc/pcie-designware-host.c Serge Semin               2023-01-13  716  
c5097b9869a136 drivers/pci/controller/dwc/pcie-designware-host.c Johan Hovold              2023-07-06  717  	if (!dw_pcie_link_up(pci)) {
a37beefbde8802 drivers/pci/controller/dwc/pcie-designware-host.c Serge Semin               2022-06-24  718  		ret = dw_pcie_start_link(pci);
886a9c1347558f drivers/pci/controller/dwc/pcie-designware-host.c Rob Herring               2020-11-05  719  		if (ret)
939fbcd568fd29 drivers/pci/controller/dwc/pcie-designware-host.c Serge Semin               2023-01-13  720  			goto err_remove_edma;
da56a1bfbab551 drivers/pci/controller/dwc/pcie-designware-host.c Ajay Agarwal              2023-04-12  721  	}
886a9c1347558f drivers/pci/controller/dwc/pcie-designware-host.c Rob Herring               2020-11-05  722  
8d3bf19f1b585a drivers/pci/controller/dwc/pcie-designware-host.c Krishna chaitanya chundru 2024-11-23  723  	/*
8d3bf19f1b585a drivers/pci/controller/dwc/pcie-designware-host.c Krishna chaitanya chundru 2024-11-23  724  	 * Note: Skip the link up delay only when a Link Up IRQ is present.
8d3bf19f1b585a drivers/pci/controller/dwc/pcie-designware-host.c Krishna chaitanya chundru 2024-11-23  725  	 * If there is no Link Up IRQ, we should not bypass the delay
8d3bf19f1b585a drivers/pci/controller/dwc/pcie-designware-host.c Krishna chaitanya chundru 2024-11-23  726  	 * because that would require users to manually rescan for devices.
8d3bf19f1b585a drivers/pci/controller/dwc/pcie-designware-host.c Krishna chaitanya chundru 2024-11-23  727  	 */
cd723d3dce14ac drivers/pci/controller/dwc/pcie-designware-host.c Niklas Cassel             2025-11-27  728  	if (!pp->use_linkup_irq) {
c5097b9869a136 drivers/pci/controller/dwc/pcie-designware-host.c Johan Hovold              2023-07-06  729  		/* Ignore errors, the link may come up later */
c5097b9869a136 drivers/pci/controller/dwc/pcie-designware-host.c Johan Hovold              2023-07-06  730  		dw_pcie_wait_for_link(pci);
c5097b9869a136 drivers/pci/controller/dwc/pcie-designware-host.c Johan Hovold              2023-07-06  731  
cd723d3dce14ac drivers/pci/controller/dwc/pcie-designware-host.c Niklas Cassel             2025-11-27  732  		/*
cd723d3dce14ac drivers/pci/controller/dwc/pcie-designware-host.c Niklas Cassel             2025-11-27  733  		 * For platforms with Link Up IRQ, initial scan will be done
cd723d3dce14ac drivers/pci/controller/dwc/pcie-designware-host.c Niklas Cassel             2025-11-27  734  		 * on first Link Up IRQ.
cd723d3dce14ac drivers/pci/controller/dwc/pcie-designware-host.c Niklas Cassel             2025-11-27  735  		 */
cd723d3dce14ac drivers/pci/controller/dwc/pcie-designware-host.c Niklas Cassel             2025-11-27  736  		if (dw_pcie_host_initial_scan(pp))
113fa857b74c94 drivers/pci/controller/dwc/pcie-designware-host.c Serge Semin               2022-06-24 @737  			goto err_stop_link;
cd723d3dce14ac drivers/pci/controller/dwc/pcie-designware-host.c Niklas Cassel             2025-11-27  738  	}
4fbfa17f9a0755 drivers/pci/controller/dwc/pcie-designware-host.c Shradha Todi              2025-02-21  739  
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I    2017-02-15  740  	return 0;
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I    2017-02-15  741  
113fa857b74c94 drivers/pci/controller/dwc/pcie-designware-host.c Serge Semin               2022-06-24  742  err_stop_link:
a37beefbde8802 drivers/pci/controller/dwc/pcie-designware-host.c Serge Semin               2022-06-24  743  	dw_pcie_stop_link(pci);
113fa857b74c94 drivers/pci/controller/dwc/pcie-designware-host.c Serge Semin               2022-06-24  744  
939fbcd568fd29 drivers/pci/controller/dwc/pcie-designware-host.c Serge Semin               2023-01-13  745  err_remove_edma:
939fbcd568fd29 drivers/pci/controller/dwc/pcie-designware-host.c Serge Semin               2023-01-13  746  	dw_pcie_edma_remove(pci);
939fbcd568fd29 drivers/pci/controller/dwc/pcie-designware-host.c Serge Semin               2023-01-13  747  
9e2b5de5604a6f drivers/pci/controller/dwc/pcie-designware-host.c Jisheng Zhang             2019-03-29  748  err_free_msi:
f78f02638af594 drivers/pci/controller/dwc/pcie-designware-host.c Rob Herring               2020-11-05  749  	if (pp->has_msi_ctrl)
9e2b5de5604a6f drivers/pci/controller/dwc/pcie-designware-host.c Jisheng Zhang             2019-03-29  750  		dw_pcie_free_msi(pp);
c6481d51dc65f2 drivers/pci/controller/dwc/pcie-designware-host.c Serge Semin               2022-06-24  751  
c6481d51dc65f2 drivers/pci/controller/dwc/pcie-designware-host.c Serge Semin               2022-06-24  752  err_deinit_host:
aea370b2aec9d3 drivers/pci/controller/dwc/pcie-designware-host.c Yoshihiro Shimoda         2023-12-20  753  	if (pp->ops->deinit)
aea370b2aec9d3 drivers/pci/controller/dwc/pcie-designware-host.c Yoshihiro Shimoda         2023-12-20  754  		pp->ops->deinit(pp);
c6481d51dc65f2 drivers/pci/controller/dwc/pcie-designware-host.c Serge Semin               2022-06-24  755  
f6fd357f7afbeb drivers/pci/controller/dwc/pcie-designware-host.c Krishna Chaitanya Chundru 2025-09-23  756  err_free_ecam:
f6fd357f7afbeb drivers/pci/controller/dwc/pcie-designware-host.c Krishna Chaitanya Chundru 2025-09-23  757  	if (pp->cfg)
f6fd357f7afbeb drivers/pci/controller/dwc/pcie-designware-host.c Krishna Chaitanya Chundru 2025-09-23  758  		pci_ecam_free(pp->cfg);
f6fd357f7afbeb drivers/pci/controller/dwc/pcie-designware-host.c Krishna Chaitanya Chundru 2025-09-23  759  
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I    2017-02-15  760  	return ret;
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I    2017-02-15  761  }
ca98329d3b58ef drivers/pci/controller/dwc/pcie-designware-host.c Vidya Sagar               2019-06-25  762  EXPORT_SYMBOL_GPL(dw_pcie_host_init);
feb85d9b1c47ea drivers/pci/dwc/pcie-designware-host.c            Kishon Vijay Abraham I    2017-02-15  763  
Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
#4 lore raw
In reply to Niklas Cassel
Quoted 148 lines · Niklas Cassel

The DWC glue drivers always call pci_host_probe() during probe(), which will allocate upstream bridge resources and enumerate the bus.

For controllers without Link Up IRQ support, pci_host_probe() is called after dw_pcie_wait_for_link(), which will also wait the time required by the PCIe specification before performing PCI Configuration Space reads.

For controllers with Link Up IRQ support, the pci_host_probe() call (which will perform PCI Configuration Space reads) is done without any of the delays mandated by the PCIe specification.

For controllers with Link Up IRQ support, since the pci_host_probe() call is done without any delay (link training might still be ongoing), it is very unlikely that this scan will find any devices. Once the Link Up IRQ triggers, the Link Up IRQ handler will call pci_rescan_bus().

This works fine for PCIe endpoints connected to the Root Port, since they don't extend the bus. However, if the pci_rescan_bus() call detects a PCIe switch, then there will be a problem when the downstream busses starts showing up, because the PCIe controller is not hotplug capable, so we are not allowed to extend the subordinate bus number after the initial scan, resulting in error messages such as:

pci_bus 0004:43: busn_res: can not insert [bus 43-41] under [bus 42-41] (conflicts with (null) [bus 42-41]) pci_bus 0004:43: busn_res: [bus 43-41] end is updated to 43
pci_bus 0004:43: busn_res: can not insert [bus 43] under [bus 42-41] (conflicts with (null) [bus 42-41]) pci 0004:42:00.0: devices behind bridge are unusable because [bus 43] cannot be assigned for them pci_bus 0004:44: busn_res: can not insert [bus 44-41] under [bus 42-41] (conflicts with (null) [bus 42-41]) pci_bus 0004:44: busn_res: [bus 44-41] end is updated to 44
pci_bus 0004:44: busn_res: can not insert [bus 44] under [bus 42-41] (conflicts with (null) [bus 42-41]) pci 0004:42:02.0: devices behind bridge are unusable because [bus 44] cannot be assigned for them pci_bus 0004:45: busn_res: can not insert [bus 45-41] under [bus 42-41] (conflicts with (null) [bus 42-41]) pci_bus 0004:45: busn_res: [bus 45-41] end is updated to 45
pci_bus 0004:45: busn_res: can not insert [bus 45] under [bus 42-41] (conflicts with (null) [bus 42-41]) pci 0004:42:06.0: devices behind bridge are unusable because [bus 45] cannot be assigned for them pci_bus 0004:46: busn_res: can not insert [bus 46-41] under [bus 42-41] (conflicts with (null) [bus 42-41]) pci_bus 0004:46: busn_res: [bus 46-41] end is updated to 46
pci_bus 0004:46: busn_res: can not insert [bus 46] under [bus 42-41] (conflicts with (null) [bus 42-41]) pci 0004:42:0e.0: devices behind bridge are unusable because [bus 46] cannot be assigned for them pci_bus 0004:42: busn_res: [bus 42-41] end is updated to 46
pci_bus 0004:42: busn_res: can not insert [bus 42-46] under [bus 41] (conflicts with (null) [bus 41]) pci 0004:41:00.0: devices behind bridge are unusable because [bus 42-46] cannot be assigned for them pcieport 0004:40:00.0: bridge has subordinate 41 but max busn 46

While we would like to set the is_hotplug_bridge flag (quirk_hotplug_bridge()), many embedded SoCs that use the DWC controller have synthesized the controller without hot-plug support. Thus, the Link Up IRQ logic is only mimicking hot-plug functionality, i.e. it is not compliant with the PCI Hot-Plug Specification, so we cannot make use of the is_hotplug_bridge flag.

In order to let the Link Up IRQ logic handle PCIe switches that are already powered on (PCIe switches that not powered on already need to implement a pwrctrl driver), don't perform a pci_host_probe() call during probe() (which disregards the delays required by the PCIe specification).

Instead let the first Link Up IRQ call pci_host_probe(). Any follow up Link Up IRQ will call pci_rescan_bus().

Fixes ec9fd499b9c6 ("PCI: dw-rockchip: Don't wait for link since we can detect Link Up")
Fixes 0e0b45ab5d77 ("PCI: dw-rockchip: Enumerate endpoints based on dll_link_up IRQ")
---
 .../pci/controller/dwc/pcie-designware-host.c | 70 ++++++++++++++++---
 drivers/pci/controller/dwc/pcie-designware.h  |  5 ++
 drivers/pci/controller/dwc/pcie-dw-rockchip.c |  5 +-
 drivers/pci/controller/dwc/pcie-qcom.c        |  5 +-
 4 files changed, 68 insertions(+), 17 deletions(-)
drivers/pci/controller/dwc/pcie-designware-host.c +54 -5
index e92513c5bda51..8654346729574 100644
@@ -565,6 +565,59 @@ static int dw_pcie_host_get_resources(struct dw_pcie_rp *pp)
 	return 0;
 }
 
+static int dw_pcie_host_initial_scan(struct dw_pcie_rp *pp)
+{
+	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
+	struct pci_host_bridge *bridge = pp->bridge;
+	int ret;
+
+	ret = pci_host_probe(bridge);
+	if (ret)
+		return ret;
+
+	if (pp->ops->post_init)
+		pp->ops->post_init(pp);
+
+	dwc_pcie_debugfs_init(pci, DW_PCIE_RC_TYPE);
+
+	return 0;
+}
+
+void dw_pcie_handle_link_up_irq(struct dw_pcie_rp *pp)
+{
+	if (!pp->initial_linkup_irq_done) {
+		int ret;
+
+		ret = dw_pcie_host_initial_scan(pp);
+		if (ret) {
+			struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
+			struct device *dev = pci->dev;
+
+			dev_err(dev, "Initial scan from IRQ failed: %d\n", ret);
+
+			dw_pcie_stop_link(pci);
+
+			dw_pcie_edma_remove(pci);
+
+			if (pp->has_msi_ctrl)
+				dw_pcie_free_msi(pp);
+
+			if (pp->ops->deinit)
+				pp->ops->deinit(pp);
+
+			if (pp->cfg)
+				pci_ecam_free(pp->cfg);
+		} else {
+			pp->initial_linkup_irq_done = true;
+		}
+	} else {
+		/* Rescan the bus to enumerate endpoint devices */
+		pci_lock_rescan_remove();
+		pci_rescan_bus(pp->bridge->bus);
+		pci_unlock_rescan_remove();
+	}
+}
+
 int dw_pcie_host_init(struct dw_pcie_rp *pp)
 {
 	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
@@ -669,18 +722,17 @@ int dw_pcie_host_init(struct dw_pcie_rp *pp)
 	 * If there is no Link Up IRQ, we should not bypass the delay
 	 * because that would require users to manually rescan for devices.
 	 */
-	if (!pp->use_linkup_irq)
+	if (!pp->use_linkup_irq) {
 		/* Ignore errors, the link may come up later */
 		dw_pcie_wait_for_link(pci);
 
-	ret = pci_host_probe(bridge);
-	if (ret)
-		goto err_stop_link;
-

drivers/pci/controller/dwc/pcie-designware-host.c int dw_pcie_host_init(struct dw_pcie_rp *pp) · line 734

Quoted 10 lines
drivers/pci/controller/dwc/pcie-designware-host.c +6 -4
-	if (pp->ops->post_init)
-		pp->ops->post_init(pp);
-
-	dwc_pcie_debugfs_init(pci, DW_PCIE_RC_TYPE);
+		/*
+		 * For platforms with Link Up IRQ, initial scan will be done
+		 * on first Link Up IRQ.
+		 */
+		if (dw_pcie_host_initial_scan(pp))
+			goto err_stop_link;

This is causing NULL ptr dereference on Qcom platforms as 'pp->bridge->bus' is dereferenced after dw_pcie_host_init() for registering the IRQ.

I still feel that a revert would be the safest option.

- Mani

Signature · 1 line
மணிவண்ணன் சதாசிவம்