Skip to contentSkip to conversation

Fixes to pcie-tegra194 driver

PATCH Superseded Manikanta Maddireddy 4 participants 31 messages Feb 23 – Mar 2

Subsystems (4)

v6 of 8 Superseded Latest is v8 →

lore
Share & tools
lore
#0 lore raw

v6 cover Fixes to pcie-tegra194 driver

This series[1] was originally posted by Vidya Sagar, and I have rebased it onto 6.19.0-rc6-next. I addressed review comments and split this into two series, one for fixes(current) and the other is for enhancements. I verified these patches on Jetson AGX Orin(Tegra234 SoC).

I added below four new patches to fix bugs, commit message of each patch has the details on the bug and fix.
- PCI: tegra194: Use HW version number
- PCI: tegra194: Fix CBB timeout caused by DBI access before core power-on
- PCI: tegra194: Disable PERST IRQ only in Endpoint mode

Rest of the patches are same as the original V3 series, just rebased them on 6.19.0-rc6-next.

I fixed the missing PATCH prefix in the subject in V6 and also reordered the patches with regard to fixes tag.

Verification details.
- I verified both Root port and Endpoint mode controllers on Tegra234 SoC.
- Basic sanity Link up, configuration space access and BAR access are verified.
- I verified that ASPM L1.2 capability is disabled for Endpoint mode.
- I verified suspend to RAM tests with Endpoint mode.

[1] https://patchwork.kernel.org/project/linux-pci/patch/[email protected]/

Manikanta Maddireddy (3):
PCI: tegra194: Disable PERST IRQ only in Endpoint mode
PCI: tegra194: Use HW version number
PCI: tegra194: Fix CBB timeout caused by DBI access before core
power-on

Vidya Sagar (10):
PCI: tegra194: Fix polling delay for L2 state
PCI: tegra194: Refactor LTSSM state polling on surprise down
PCI: tegra194: Don't force the device into the D0 state before L2
PCI: tegra194: Use devm_gpiod_get_optional() to parse
"nvidia,refclk-select"
PCI: tegra194: Disable direct speed change for EP
PCI: tegra194: Set LTR message request before PCIe link up
PCI: tegra194: Apply pinctrl settings for both PCIe RP and EP
PCI: tegra194: Allow system suspend when the Endpoint link is not up
PCI: tegra194: Free up EP resources during remove()
PCI: tegra194: Free resources during controller deinitialization

 drivers/pci/controller/dwc/pcie-designware.c |   2 +-
 drivers/pci/controller/dwc/pcie-designware.h |   2 +
 drivers/pci/controller/dwc/pcie-tegra194.c   | 191 +++++++++----------
 3 files changed, 95 insertions(+), 100 deletions(-)
Signature · 1 line
2.34.1
#1 lore raw

PATCH v6 1/13 PCI: tegra194: Fix polling delay for L2 state

From: Vidya Sagar <[email protected]>

As per PCIe spec r6.0, sec 5.3.3.2.1, after sending PME_Turn_Off message, Root port should wait for 1~10 msec for PME_TO_Ack message. Currently, driver is polling for 10 msec with 1 usec delay which is aggressive. Change it to 10 msec polling with 100 usec delay. Since this function is used in non-atomic context only, use non-atomic poll function.

Fixes 56e15a238d92 ("PCI: tegra: Add Tegra194 PCIe support")

---
Changes V1 -> V6: None

 drivers/pci/controller/dwc/pcie-tegra194.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)
drivers/pci/controller/dwc/pcie-tegra194.c +5 -4
index 0ddeef70726d..96d38571a7e7 100644
@@ -198,7 +198,8 @@
 #define CAP_SPCIE_CAP_OFF_USP_TX_PRESET0_MASK	GENMASK(11, 8)
 #define CAP_SPCIE_CAP_OFF_USP_TX_PRESET0_SHIFT	8
 
-#define PME_ACK_TIMEOUT 10000
+#define PME_ACK_DELAY		100   /* 100 us */
+#define PME_ACK_TIMEOUT		10000 /* 10 ms */
 
 #define LTSSM_TIMEOUT 50000	/* 50ms */
 
@@ -1553,9 +1554,9 @@ static int tegra_pcie_try_link_l2(struct tegra_pcie_dw *pcie)
 	val |= APPL_PM_XMT_TURNOFF_STATE;
 	appl_writel(pcie, val, APPL_RADM_STATUS);
 
-	return readl_poll_timeout_atomic(pcie->appl_base + APPL_DEBUG, val,
-				 val & APPL_DEBUG_PM_LINKST_IN_L2_LAT,
-				 1, PME_ACK_TIMEOUT);
+	return readl_poll_timeout(pcie->appl_base + APPL_DEBUG, val,
+				  val & APPL_DEBUG_PM_LINKST_IN_L2_LAT,
+				  PME_ACK_DELAY, PME_ACK_TIMEOUT);
 }
 
 static void tegra_pcie_dw_pme_turnoff(struct tegra_pcie_dw *pcie)
Signature · 1 line
2.34.1
Reviewed-by on v6 1/13
#2 lore raw

Re: [PATCH v6 01/13] PCI: tegra194: Fix polling delay for L2 state

Quoted 34 lines · Manikanta Maddireddy

From: Vidya Sagar <[email protected]>

As per PCIe spec r6.0, sec 5.3.3.2.1, after sending PME_Turn_Off message, Root port should wait for 1~10 msec for PME_TO_Ack message. Currently, driver is polling for 10 msec with 1 usec delay which is aggressive. Change it to 10 msec polling with 100 usec delay. Since this function is used in non-atomic context only, use non-atomic poll function.

Fixes 56e15a238d92 ("PCI: tegra: Add Tegra194 PCIe support")

---
Changes V1 -> V6: None

 drivers/pci/controller/dwc/pcie-tegra194.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)
drivers/pci/controller/dwc/pcie-tegra194.c +2 -1
index 0ddeef70726d..96d38571a7e7 100644
@@ -198,7 +198,8 @@
 #define CAP_SPCIE_CAP_OFF_USP_TX_PRESET0_MASK	GENMASK(11, 8)
 #define CAP_SPCIE_CAP_OFF_USP_TX_PRESET0_SHIFT	8
 
-#define PME_ACK_TIMEOUT 10000
+#define PME_ACK_DELAY		100   /* 100 us */
+#define PME_ACK_TIMEOUT		10000 /* 10 ms */
 
 #define LTSSM_TIMEOUT 50000	/* 50ms */
 
@@ -1553,9 +1554,9 @@ static int tegra_pcie_try_link_l2(struct tegra_pcie_dw *pcie)
 	val |= APPL_PM_XMT_TURNOFF_STATE;
 	appl_writel(pcie, val, APPL_RADM_STATUS);

drivers/pci/controller/dwc/pcie-tegra194.c static int tegra_pcie_try_link_l2(struct tegra_pcie_dw *pcie) · line 1562

Quoted 10 lines
drivers/pci/controller/dwc/pcie-tegra194.c +3 -3
 
-	return readl_poll_timeout_atomic(pcie->appl_base + APPL_DEBUG, val,
-				 val & APPL_DEBUG_PM_LINKST_IN_L2_LAT,
-				 1, PME_ACK_TIMEOUT);
+	return readl_poll_timeout(pcie->appl_base + APPL_DEBUG, val,
+				  val & APPL_DEBUG_PM_LINKST_IN_L2_LAT,
+				  PME_ACK_DELAY, PME_ACK_TIMEOUT);
 }
 
 static void tegra_pcie_dw_pme_turnoff(struct tegra_pcie_dw *pcie)
#3 lore raw

Re: [PATCH v6 01/13] PCI: tegra194: Fix polling delay for L2 state

In reply to Manikanta Maddireddy
Quoted 22 lines · Manikanta Maddireddy

From: Vidya Sagar <[email protected]>

As per PCIe spec r6.0, sec 5.3.3.2.1, after sending PME_Turn_Off message, Root port should wait for 1~10 msec for PME_TO_Ack message. Currently, driver is polling for 10 msec with 1 usec delay which is aggressive. Change it to 10 msec polling with 100 usec delay. Since this function is used in non-atomic context only, use non-atomic poll function.

Fixes 56e15a238d92 ("PCI: tegra: Add Tegra194 PCIe support")

---
Changes V1 -> V6: None

 drivers/pci/controller/dwc/pcie-tegra194.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)
drivers/pci/controller/dwc/pcie-tegra194.c +0 -0
index 0ddeef70726d..96d38571a7e7 100644

drivers/pci/controller/dwc/pcie-tegra194.c line 202

Quoted 6 lines
drivers/pci/controller/dwc/pcie-tegra194.c +2 -1
@@ -198,7 +198,8 @@
 #define CAP_SPCIE_CAP_OFF_USP_TX_PRESET0_MASK	GENMASK(11, 8)
 #define CAP_SPCIE_CAP_OFF_USP_TX_PRESET0_SHIFT	8
 
-#define PME_ACK_TIMEOUT 10000
+#define PME_ACK_DELAY		100   /* 100 us */
+#define PME_ACK_TIMEOUT		10000 /* 10 ms */

Can you add "_US" and "_MS" suffixes to these to indicate the units?

Unless they are something specific to tegra194, the #defines should probably go in drivers/pci/pci.h so they can be shared.

It'd be nice to update the PCIe spec citation to r7.0.

#4 lore raw

PATCH v6 2/13 PCI: tegra194: Refactor LTSSM state polling on surprise down

In reply to Manikanta Maddireddy

From: Vidya Sagar <[email protected]>

On surprise down, LTSSM state transits from L0 -> Recovery.RcvrLock -> Recovery.RcvrSpeed -> Gen1 Recovery.RcvrLock -> Detect. Recovery.RcvrLock and Recovery.RcvrSpeed transit times are 24 msec and 48 msec respectively. So, the total time taken to transit from L0 to detect state is ~96 msec. Hence, increase the poll time to 120 msec.

Disable the LTSSM state after it transits to detect to avoid LTSSM toggling between polling and detect states.

tegra_pcie_dw_pme_turnoff() function is called in non-atomic context only, so use the non-atomic poll function.

Fixes 56e15a238d92 ("PCI: tegra: Add Tegra194 PCIe support")

---
Changes V5 -> V6: Retain only one fixes tag
Changes V1 -> V5: None

 drivers/pci/controller/dwc/pcie-tegra194.c | 55 +++++++++++++---------
 1 file changed, 32 insertions(+), 23 deletions(-)
drivers/pci/controller/dwc/pcie-tegra194.c +32 -23
index 96d38571a7e7..4ac6b1cea13f 100644
@@ -137,7 +137,11 @@
 #define APPL_DEBUG_PM_LINKST_IN_L0		0x11
 #define APPL_DEBUG_LTSSM_STATE_MASK		GENMASK(8, 3)
 #define APPL_DEBUG_LTSSM_STATE_SHIFT		3
-#define LTSSM_STATE_PRE_DETECT			5
+#define LTSSM_STATE_DETECT_QUIET		0x00
+#define LTSSM_STATE_DETECT_ACT			0x08
+#define LTSSM_STATE_PRE_DETECT_QUIET		0x28
+#define LTSSM_STATE_DETECT_WAIT			0x30
+#define LTSSM_STATE_L2_IDLE			0xa8
 
 #define APPL_RADM_STATUS			0xE4
 #define APPL_PM_XMT_TURNOFF_STATE		BIT(0)
@@ -201,7 +205,8 @@
 #define PME_ACK_DELAY		100   /* 100 us */
 #define PME_ACK_TIMEOUT		10000 /* 10 ms */
 
-#define LTSSM_TIMEOUT 50000	/* 50ms */
+#define LTSSM_DELAY		10000	/* 10 ms */
+#define LTSSM_TIMEOUT		120000	/* 120 ms */
 
 #define GEN3_GEN4_EQ_PRESET_INIT	5
 
@@ -1591,23 +1596,22 @@ static void tegra_pcie_dw_pme_turnoff(struct tegra_pcie_dw *pcie)
 		data &= ~APPL_PINMUX_PEX_RST;
 		appl_writel(pcie, data, APPL_PINMUX);
 
+		err = readl_poll_timeout(pcie->appl_base + APPL_DEBUG, data,
+			((data & APPL_DEBUG_LTSSM_STATE_MASK) == LTSSM_STATE_DETECT_QUIET) ||
+			((data & APPL_DEBUG_LTSSM_STATE_MASK) == LTSSM_STATE_DETECT_ACT) ||
+			((data & APPL_DEBUG_LTSSM_STATE_MASK) == LTSSM_STATE_PRE_DETECT_QUIET) ||
+			((data & APPL_DEBUG_LTSSM_STATE_MASK) == LTSSM_STATE_DETECT_WAIT),
+			LTSSM_DELAY, LTSSM_TIMEOUT);
+		if (err)
+			dev_info(pcie->dev, "Link didn't go to detect state\n");
+
 		/*
-		 * Some cards do not go to detect state even after de-asserting
-		 * PERST#. So, de-assert LTSSM to bring link to detect state.
+		 * Deassert LTSSM state to stop the state toggling between
+		 * polling and detect.
 		 */
 		data = readl(pcie->appl_base + APPL_CTRL);
 		data &= ~APPL_CTRL_LTSSM_EN;
 		writel(data, pcie->appl_base + APPL_CTRL);
-
-		err = readl_poll_timeout_atomic(pcie->appl_base + APPL_DEBUG,
-						data,
-						((data &
-						APPL_DEBUG_LTSSM_STATE_MASK) >>
-						APPL_DEBUG_LTSSM_STATE_SHIFT) ==
-						LTSSM_STATE_PRE_DETECT,
-						1, LTSSM_TIMEOUT);
-		if (err)
-			dev_info(pcie->dev, "Link didn't go to detect state\n");
 	}
 	/*
 	 * DBI registers may not be accessible after this as PLL-E would be
@@ -1681,19 +1685,24 @@ static void pex_ep_event_pex_rst_assert(struct tegra_pcie_dw *pcie)
 	if (pcie->ep_state == EP_STATE_DISABLED)
 		return;
 
-	/* Disable LTSSM */
+	ret = readl_poll_timeout(pcie->appl_base + APPL_DEBUG, val,
+		((val & APPL_DEBUG_LTSSM_STATE_MASK) == LTSSM_STATE_DETECT_QUIET) ||
+		((val & APPL_DEBUG_LTSSM_STATE_MASK) == LTSSM_STATE_DETECT_ACT) ||
+		((val & APPL_DEBUG_LTSSM_STATE_MASK) == LTSSM_STATE_PRE_DETECT_QUIET) ||
+		((val & APPL_DEBUG_LTSSM_STATE_MASK) == LTSSM_STATE_DETECT_WAIT) ||
+		((val & APPL_DEBUG_LTSSM_STATE_MASK) == LTSSM_STATE_L2_IDLE),
+		LTSSM_DELAY, LTSSM_TIMEOUT);
+	if (ret)
+		dev_err(pcie->dev, "LTSSM state: 0x%x timeout: %d\n", val, ret);
+
+	/*
+	 * Deassert LTSSM state to stop the state toggling between
+	 * polling and detect.
+	 */
 	val = appl_readl(pcie, APPL_CTRL);
 	val &= ~APPL_CTRL_LTSSM_EN;
 	appl_writel(pcie, val, APPL_CTRL);
 
-	ret = readl_poll_timeout(pcie->appl_base + APPL_DEBUG, val,
-				 ((val & APPL_DEBUG_LTSSM_STATE_MASK) >>
-				 APPL_DEBUG_LTSSM_STATE_SHIFT) ==
-				 LTSSM_STATE_PRE_DETECT,
-				 1, LTSSM_TIMEOUT);
-	if (ret)
-		dev_err(pcie->dev, "Failed to go Detect state: %d\n", ret);
-
 	reset_control_assert(pcie->core_rst);
 
 	tegra_pcie_disable_phy(pcie);
Signature · 1 line
2.34.1
Reviewed-by on v6 2/13
#5 lore raw

Re: [PATCH v6 02/13] PCI: tegra194: Refactor LTSSM state polling on surprise down

Quoted 110 lines · Manikanta Maddireddy

From: Vidya Sagar <[email protected]>

On surprise down, LTSSM state transits from L0 -> Recovery.RcvrLock -> Recovery.RcvrSpeed -> Gen1 Recovery.RcvrLock -> Detect. Recovery.RcvrLock and Recovery.RcvrSpeed transit times are 24 msec and 48 msec respectively. So, the total time taken to transit from L0 to detect state is ~96 msec. Hence, increase the poll time to 120 msec.

Disable the LTSSM state after it transits to detect to avoid LTSSM toggling between polling and detect states.

tegra_pcie_dw_pme_turnoff() function is called in non-atomic context only, so use the non-atomic poll function.

Fixes 56e15a238d92 ("PCI: tegra: Add Tegra194 PCIe support")

---
Changes V5 -> V6: Retain only one fixes tag
Changes V1 -> V5: None

 drivers/pci/controller/dwc/pcie-tegra194.c | 55 +++++++++++++---------
 1 file changed, 32 insertions(+), 23 deletions(-)
drivers/pci/controller/dwc/pcie-tegra194.c +32 -16
index 96d38571a7e7..4ac6b1cea13f 100644
@@ -137,7 +137,11 @@
 #define APPL_DEBUG_PM_LINKST_IN_L0		0x11
 #define APPL_DEBUG_LTSSM_STATE_MASK		GENMASK(8, 3)
 #define APPL_DEBUG_LTSSM_STATE_SHIFT		3
-#define LTSSM_STATE_PRE_DETECT			5
+#define LTSSM_STATE_DETECT_QUIET		0x00
+#define LTSSM_STATE_DETECT_ACT			0x08
+#define LTSSM_STATE_PRE_DETECT_QUIET		0x28
+#define LTSSM_STATE_DETECT_WAIT			0x30
+#define LTSSM_STATE_L2_IDLE			0xa8
 
 #define APPL_RADM_STATUS			0xE4
 #define APPL_PM_XMT_TURNOFF_STATE		BIT(0)
@@ -201,7 +205,8 @@
 #define PME_ACK_DELAY		100   /* 100 us */
 #define PME_ACK_TIMEOUT		10000 /* 10 ms */
 
-#define LTSSM_TIMEOUT 50000	/* 50ms */
+#define LTSSM_DELAY		10000	/* 10 ms */
+#define LTSSM_TIMEOUT		120000	/* 120 ms */
 
 #define GEN3_GEN4_EQ_PRESET_INIT	5
 
@@ -1591,23 +1596,22 @@ static void tegra_pcie_dw_pme_turnoff(struct tegra_pcie_dw *pcie)
 		data &= ~APPL_PINMUX_PEX_RST;
 		appl_writel(pcie, data, APPL_PINMUX);
 
+		err = readl_poll_timeout(pcie->appl_base + APPL_DEBUG, data,
+			((data & APPL_DEBUG_LTSSM_STATE_MASK) == LTSSM_STATE_DETECT_QUIET) ||
+			((data & APPL_DEBUG_LTSSM_STATE_MASK) == LTSSM_STATE_DETECT_ACT) ||
+			((data & APPL_DEBUG_LTSSM_STATE_MASK) == LTSSM_STATE_PRE_DETECT_QUIET) ||
+			((data & APPL_DEBUG_LTSSM_STATE_MASK) == LTSSM_STATE_DETECT_WAIT),
+			LTSSM_DELAY, LTSSM_TIMEOUT);
+		if (err)
+			dev_info(pcie->dev, "Link didn't go to detect state\n");
+
 		/*
-		 * Some cards do not go to detect state even after de-asserting
-		 * PERST#. So, de-assert LTSSM to bring link to detect state.
+		 * Deassert LTSSM state to stop the state toggling between
+		 * polling and detect.
 		 */
 		data = readl(pcie->appl_base + APPL_CTRL);
 		data &= ~APPL_CTRL_LTSSM_EN;
 		writel(data, pcie->appl_base + APPL_CTRL);
-
-		err = readl_poll_timeout_atomic(pcie->appl_base + APPL_DEBUG,
-						data,
-						((data &
-						APPL_DEBUG_LTSSM_STATE_MASK) >>
-						APPL_DEBUG_LTSSM_STATE_SHIFT) ==
-						LTSSM_STATE_PRE_DETECT,
-						1, LTSSM_TIMEOUT);
-		if (err)
-			dev_info(pcie->dev, "Link didn't go to detect state\n");
 	}
 	/*
 	 * DBI registers may not be accessible after this as PLL-E would be
@@ -1681,19 +1685,24 @@ static void pex_ep_event_pex_rst_assert(struct tegra_pcie_dw *pcie)
 	if (pcie->ep_state == EP_STATE_DISABLED)
 		return;
 
-	/* Disable LTSSM */
+	ret = readl_poll_timeout(pcie->appl_base + APPL_DEBUG, val,
+		((val & APPL_DEBUG_LTSSM_STATE_MASK) == LTSSM_STATE_DETECT_QUIET) ||
+		((val & APPL_DEBUG_LTSSM_STATE_MASK) == LTSSM_STATE_DETECT_ACT) ||
+		((val & APPL_DEBUG_LTSSM_STATE_MASK) == LTSSM_STATE_PRE_DETECT_QUIET) ||
+		((val & APPL_DEBUG_LTSSM_STATE_MASK) == LTSSM_STATE_DETECT_WAIT) ||
+		((val & APPL_DEBUG_LTSSM_STATE_MASK) == LTSSM_STATE_L2_IDLE),
+		LTSSM_DELAY, LTSSM_TIMEOUT);
+	if (ret)
+		dev_err(pcie->dev, "LTSSM state: 0x%x timeout: %d\n", val, ret);
+
+	/*
+	 * Deassert LTSSM state to stop the state toggling between
+	 * polling and detect.
+	 */
 	val = appl_readl(pcie, APPL_CTRL);
 	val &= ~APPL_CTRL_LTSSM_EN;
 	appl_writel(pcie, val, APPL_CTRL);
 
-	ret = readl_poll_timeout(pcie->appl_base + APPL_DEBUG, val,

drivers/pci/controller/dwc/pcie-tegra194.c static void pex_ep_event_pex_rst_assert(struct tegra_pcie_dw *pcie) · line 1708

Quoted 10 lines
drivers/pci/controller/dwc/pcie-tegra194.c +0 -7
-				 ((val & APPL_DEBUG_LTSSM_STATE_MASK) >>
-				 APPL_DEBUG_LTSSM_STATE_SHIFT) ==
-				 LTSSM_STATE_PRE_DETECT,
-				 1, LTSSM_TIMEOUT);
-	if (ret)
-		dev_err(pcie->dev, "Failed to go Detect state: %d\n", ret);
-
 	reset_control_assert(pcie->core_rst);
 
 	tegra_pcie_disable_phy(pcie);
#6 lore raw

PATCH v6 3/13 PCI: tegra194: Don't force the device into the D0 state before L2

In reply to Manikanta Maddireddy

From: Vidya Sagar <[email protected]>

As per PCIe CEM spec rev 4.0 ver 1.0 sec 2.3, the PCIe endpoint device should be in D3 state to assert wake# pin. This takes precedence over PCI Express Base r4.0 v1.0 September 27-2017, 5.2 Link State Power Management which states that the device can be put into D0 state before taking the link to L2 state. So, to enable the wake functionality for endpoints, do not force the devices to D0 state before taking the link to L2 state. There is no functional issue with the endpoints where the link doesn't go into L2 state (the reason why the earlier change was made in the first place) as the root port proceeds with the usual flow post PME timeout.

Fixes 56e15a238d92 ("PCI: tegra: Add Tegra194 PCIe support")

---
Changes V1 -> V6: None

 drivers/pci/controller/dwc/pcie-tegra194.c | 41 ----------------------
 1 file changed, 41 deletions(-)
drivers/pci/controller/dwc/pcie-tegra194.c +0 -41
index 4ac6b1cea13f..808a1e213b79 100644
@@ -1261,44 +1261,6 @@ static int tegra_pcie_bpmp_set_pll_state(struct tegra_pcie_dw *pcie,
 	return 0;
 }
 
-static void tegra_pcie_downstream_dev_to_D0(struct tegra_pcie_dw *pcie)
-{
-	struct dw_pcie_rp *pp = &pcie->pci.pp;
-	struct pci_bus *child, *root_port_bus = NULL;
-	struct pci_dev *pdev;
-
-	/*
-	 * link doesn't go into L2 state with some of the endpoints with Tegra
-	 * if they are not in D0 state. So, need to make sure that immediate
-	 * downstream devices are in D0 state before sending PME_TurnOff to put
-	 * link into L2 state.
-	 * This is as per PCI Express Base r4.0 v1.0 September 27-2017,
-	 * 5.2 Link State Power Management (Page #428).
-	 */
-
-	list_for_each_entry(child, &pp->bridge->bus->children, node) {
-		if (child->parent == pp->bridge->bus) {
-			root_port_bus = child;
-			break;
-		}
-	}
-
-	if (!root_port_bus) {
-		dev_err(pcie->dev, "Failed to find downstream bus of Root Port\n");
-		return;
-	}
-
-	/* Bring downstream devices to D0 if they are not already in */
-	list_for_each_entry(pdev, &root_port_bus->devices, bus_list) {
-		if (PCI_SLOT(pdev->devfn) == 0) {
-			if (pci_set_power_state(pdev, PCI_D0))
-				dev_err(pcie->dev,
-					"Failed to transition %s to D0 state\n",
-					dev_name(&pdev->dev));
-		}
-	}
-}
-
 static int tegra_pcie_get_slot_regulators(struct tegra_pcie_dw *pcie)
 {
 	pcie->slot_ctl_3v3 = devm_regulator_get_optional(pcie->dev, "vpcie3v3");
@@ -1627,7 +1589,6 @@ static void tegra_pcie_dw_pme_turnoff(struct tegra_pcie_dw *pcie)
 
 static void tegra_pcie_deinit_controller(struct tegra_pcie_dw *pcie)
 {
-	tegra_pcie_downstream_dev_to_D0(pcie);
 	dw_pcie_host_deinit(&pcie->pci.pp);
 	tegra_pcie_dw_pme_turnoff(pcie);
 	tegra_pcie_unconfig_controller(pcie);
@@ -2337,7 +2298,6 @@ static int tegra_pcie_dw_suspend_noirq(struct device *dev)
 	if (!pcie->link_state)
 		return 0;
 
-	tegra_pcie_downstream_dev_to_D0(pcie);
 	tegra_pcie_dw_pme_turnoff(pcie);
 	tegra_pcie_unconfig_controller(pcie);
 
@@ -2411,7 +2371,6 @@ static void tegra_pcie_dw_shutdown(struct platform_device *pdev)
 			return;
 
 		debugfs_remove_recursive(pcie->debugfs);
-		tegra_pcie_downstream_dev_to_D0(pcie);
 
 		disable_irq(pcie->pci.pp.irq);
 		if (IS_ENABLED(CONFIG_PCI_MSI))
Signature · 1 line
2.34.1
Reviewed-by on v6 3/13
#7 lore raw

Re: [PATCH v6 03/13] PCI: tegra194: Don't force the device into the D0 state before L2

Quoted 87 lines · Manikanta Maddireddy

From: Vidya Sagar <[email protected]>

As per PCIe CEM spec rev 4.0 ver 1.0 sec 2.3, the PCIe endpoint device should be in D3 state to assert wake# pin. This takes precedence over PCI Express Base r4.0 v1.0 September 27-2017, 5.2 Link State Power Management which states that the device can be put into D0 state before taking the link to L2 state. So, to enable the wake functionality for endpoints, do not force the devices to D0 state before taking the link to L2 state. There is no functional issue with the endpoints where the link doesn't go into L2 state (the reason why the earlier change was made in the first place) as the root port proceeds with the usual flow post PME timeout.

Fixes 56e15a238d92 ("PCI: tegra: Add Tegra194 PCIe support")

---
Changes V1 -> V6: None

 drivers/pci/controller/dwc/pcie-tegra194.c | 41 ----------------------
 1 file changed, 41 deletions(-)
drivers/pci/controller/dwc/pcie-tegra194.c +0 -40
index 4ac6b1cea13f..808a1e213b79 100644
@@ -1261,44 +1261,6 @@ static int tegra_pcie_bpmp_set_pll_state(struct tegra_pcie_dw *pcie,
 	return 0;
 }
 
-static void tegra_pcie_downstream_dev_to_D0(struct tegra_pcie_dw *pcie)
-{
-	struct dw_pcie_rp *pp = &pcie->pci.pp;
-	struct pci_bus *child, *root_port_bus = NULL;
-	struct pci_dev *pdev;
-
-	/*
-	 * link doesn't go into L2 state with some of the endpoints with Tegra
-	 * if they are not in D0 state. So, need to make sure that immediate
-	 * downstream devices are in D0 state before sending PME_TurnOff to put
-	 * link into L2 state.
-	 * This is as per PCI Express Base r4.0 v1.0 September 27-2017,
-	 * 5.2 Link State Power Management (Page #428).
-	 */
-
-	list_for_each_entry(child, &pp->bridge->bus->children, node) {
-		if (child->parent == pp->bridge->bus) {
-			root_port_bus = child;
-			break;
-		}
-	}
-
-	if (!root_port_bus) {
-		dev_err(pcie->dev, "Failed to find downstream bus of Root Port\n");
-		return;
-	}
-
-	/* Bring downstream devices to D0 if they are not already in */
-	list_for_each_entry(pdev, &root_port_bus->devices, bus_list) {
-		if (PCI_SLOT(pdev->devfn) == 0) {
-			if (pci_set_power_state(pdev, PCI_D0))
-				dev_err(pcie->dev,
-					"Failed to transition %s to D0 state\n",
-					dev_name(&pdev->dev));
-		}
-	}
-}
-
 static int tegra_pcie_get_slot_regulators(struct tegra_pcie_dw *pcie)
 {
 	pcie->slot_ctl_3v3 = devm_regulator_get_optional(pcie->dev, "vpcie3v3");
@@ -1627,7 +1589,6 @@ static void tegra_pcie_dw_pme_turnoff(struct tegra_pcie_dw *pcie)
 
 static void tegra_pcie_deinit_controller(struct tegra_pcie_dw *pcie)
 {
-	tegra_pcie_downstream_dev_to_D0(pcie);
 	dw_pcie_host_deinit(&pcie->pci.pp);
 	tegra_pcie_dw_pme_turnoff(pcie);
 	tegra_pcie_unconfig_controller(pcie);
@@ -2337,7 +2298,6 @@ static int tegra_pcie_dw_suspend_noirq(struct device *dev)
 	if (!pcie->link_state)
 		return 0;
 
-	tegra_pcie_downstream_dev_to_D0(pcie);
 	tegra_pcie_dw_pme_turnoff(pcie);
 	tegra_pcie_unconfig_controller(pcie);
 

drivers/pci/controller/dwc/pcie-tegra194.c static void tegra_pcie_dw_shutdown(struct platform_device *pdev) · line 2376

Quoted 7 lines
drivers/pci/controller/dwc/pcie-tegra194.c +0 -1
@@ -2411,7 +2371,6 @@ static void tegra_pcie_dw_shutdown(struct platform_device *pdev)
 			return;
 
 		debugfs_remove_recursive(pcie->debugfs);
-		tegra_pcie_downstream_dev_to_D0(pcie);
 
 		disable_irq(pcie->pci.pp.irq);
 		if (IS_ENABLED(CONFIG_PCI_MSI))
#8 lore raw

PATCH v6 4/13 PCI: tegra194: Disable PERST IRQ only in Endpoint mode

In reply to Manikanta Maddireddy

The PERST (PERST#) GPIO interrupt is only registered when the controller is operating in Endpoint mode. In Root Port mode, the PERST GPIO is configured as an output to control downstream devices, and no interrupt is registered for it.

Currently, tegra_pcie_dw_stop_link() unconditionally calls disable_irq() on pex_rst_irq, which causes issues in Root Port mode where this IRQ is not registered.

Fix this by only disabling the PERST IRQ when operating in Endpoint mode, where the interrupt is actually registered and used to detect PERST assertion/deassertion from the host.

Fixes c57247f940e8 ("PCI: tegra: Add support for PCIe endpoint mode in Tegra194")

---
Changes V1 -> V6: None

 drivers/pci/controller/dwc/pcie-tegra194.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/pci/controller/dwc/pcie-tegra194.c +2 -1
index 808a1e213b79..9455456028bc 100644
@@ -1028,7 +1028,8 @@ static void tegra_pcie_dw_stop_link(struct dw_pcie *pci)
 {
 	struct tegra_pcie_dw *pcie = to_tegra_pcie(pci);
 
-	disable_irq(pcie->pex_rst_irq);
+	if (pcie->of_data->mode == DW_PCIE_EP_TYPE)
+		disable_irq(pcie->pex_rst_irq);
 }
 
 static const struct dw_pcie_ops tegra_dw_pcie_ops = {
Signature · 1 line
2.34.1
Reviewed-by on v6 4/13
#9 lore raw

Re: [PATCH v6 04/13] PCI: tegra194: Disable PERST IRQ only in Endpoint mode

Quoted 26 lines · Manikanta Maddireddy

The PERST (PERST#) GPIO interrupt is only registered when the controller is operating in Endpoint mode. In Root Port mode, the PERST GPIO is configured as an output to control downstream devices, and no interrupt is registered for it.

Currently, tegra_pcie_dw_stop_link() unconditionally calls disable_irq() on pex_rst_irq, which causes issues in Root Port mode where this IRQ is not registered.

Fix this by only disabling the PERST IRQ when operating in Endpoint mode, where the interrupt is actually registered and used to detect PERST assertion/deassertion from the host.

Fixes c57247f940e8 ("PCI: tegra: Add support for PCIe endpoint mode in Tegra194")

---
Changes V1 -> V6: None

 drivers/pci/controller/dwc/pcie-tegra194.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/pci/controller/dwc/pcie-tegra194.c +0 -0
index 808a1e213b79..9455456028bc 100644

drivers/pci/controller/dwc/pcie-tegra194.c static void tegra_pcie_dw_stop_link(struct dw_pcie *pci) · line 1035

Quoted 9 lines
drivers/pci/controller/dwc/pcie-tegra194.c +2 -1
@@ -1028,7 +1028,8 @@ static void tegra_pcie_dw_stop_link(struct dw_pcie *pci)
 {
 	struct tegra_pcie_dw *pcie = to_tegra_pcie(pci);
 
-	disable_irq(pcie->pex_rst_irq);
+	if (pcie->of_data->mode == DW_PCIE_EP_TYPE)
+		disable_irq(pcie->pex_rst_irq);
 }
 
 static const struct dw_pcie_ops tegra_dw_pcie_ops = {
#10 lore raw

PATCH v6 5/13 PCI: tegra194: Use devm_gpiod_get_optional() to parse "nvidia,refclk-select"

In reply to Manikanta Maddireddy

From: Vidya Sagar <[email protected]>

The GPIO DT property "nvidia,refclk-select" to select the PCIe reference clock is optional. Use devm_gpiod_get_optional() to get it.

Fixes c57247f940e8 ("PCI: tegra: Add support for PCIe endpoint mode in Tegra194")

---
Changes V5 -> V6: None
Changes V4 -> V5: Added Fixes tag
Changes V1 -> V4: None

 drivers/pci/controller/dwc/pcie-tegra194.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
drivers/pci/controller/dwc/pcie-tegra194.c +3 -3
index 9455456028bc..8f69c5c5a099 100644
@@ -1170,9 +1170,9 @@ static int tegra_pcie_dw_parse_dt(struct tegra_pcie_dw *pcie)
 		return err;
 	}
 
-	pcie->pex_refclk_sel_gpiod = devm_gpiod_get(pcie->dev,
-						    "nvidia,refclk-select",
-						    GPIOD_OUT_HIGH);
+	pcie->pex_refclk_sel_gpiod = devm_gpiod_get_optional(pcie->dev,
+							     "nvidia,refclk-select",
+							     GPIOD_OUT_HIGH);
 	if (IS_ERR(pcie->pex_refclk_sel_gpiod)) {
 		int err = PTR_ERR(pcie->pex_refclk_sel_gpiod);
 		const char *level = KERN_ERR;
Signature · 1 line
2.34.1
Reviewed-by on v6 5/13
#11 lore raw

Re: [PATCH v6 05/13] PCI: tegra194: Use devm_gpiod_get_optional() to parse "nvidia,refclk-select"

Quoted 23 lines · Manikanta Maddireddy

From: Vidya Sagar <[email protected]>

The GPIO DT property "nvidia,refclk-select" to select the PCIe reference clock is optional. Use devm_gpiod_get_optional() to get it.

Fixes c57247f940e8 ("PCI: tegra: Add support for PCIe endpoint mode in Tegra194")

---
Changes V5 -> V6: None
Changes V4 -> V5: Added Fixes tag
Changes V1 -> V4: None

 drivers/pci/controller/dwc/pcie-tegra194.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
drivers/pci/controller/dwc/pcie-tegra194.c +0 -0
index 9455456028bc..8f69c5c5a099 100644
@@ -1170,9 +1170,9 @@ static int tegra_pcie_dw_parse_dt(struct tegra_pcie_dw *pcie)
 		return err;
 	}

drivers/pci/controller/dwc/pcie-tegra194.c static int tegra_pcie_dw_parse_dt(struct tegra_pcie_dw *pcie) · line 1178

Quoted 10 lines
drivers/pci/controller/dwc/pcie-tegra194.c +3 -3
 
-	pcie->pex_refclk_sel_gpiod = devm_gpiod_get(pcie->dev,
-						    "nvidia,refclk-select",
-						    GPIOD_OUT_HIGH);
+	pcie->pex_refclk_sel_gpiod = devm_gpiod_get_optional(pcie->dev,
+							     "nvidia,refclk-select",
+							     GPIOD_OUT_HIGH);
 	if (IS_ERR(pcie->pex_refclk_sel_gpiod)) {
 		int err = PTR_ERR(pcie->pex_refclk_sel_gpiod);
 		const char *level = KERN_ERR;
#12 lore raw

PATCH v6 6/13 PCI: tegra194: Disable direct speed change for EP

In reply to Manikanta Maddireddy

From: Vidya Sagar <[email protected]>

Disable direct speed change for the endpoint to prevent it from initiating the speed change post physical layer link up at gen1. This leaves the speed change ownership with the host.

Fixes c57247f940e8 ("PCI: tegra: Add support for PCIe endpoint mode in Tegra194")

---
Changes V1 -> V6: None

 drivers/pci/controller/dwc/pcie-tegra194.c | 4 ++++
 1 file changed, 4 insertions(+)
drivers/pci/controller/dwc/pcie-tegra194.c +4 -0
index 8f69c5c5a099..a6868b77e3b7 100644
@@ -1807,6 +1807,10 @@ static void pex_ep_event_pex_rst_deassert(struct tegra_pcie_dw *pcie)
 
 	reset_control_deassert(pcie->core_rst);
 
+	val = dw_pcie_readl_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL);
+	val &= ~PORT_LOGIC_SPEED_CHANGE;
+	dw_pcie_writel_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL, val);
+
 	if (pcie->update_fc_fixup) {
 		val = dw_pcie_readl_dbi(pci, CFG_TIMER_CTRL_MAX_FUNC_NUM_OFF);
 		val |= 0x1 << CFG_TIMER_CTRL_ACK_NAK_SHIFT;
Signature · 1 line
2.34.1
Reviewed-by on v6 6/13
#13 lore raw

Re: [PATCH v6 06/13] PCI: tegra194: Disable direct speed change for EP

Quoted 20 lines · Manikanta Maddireddy

From: Vidya Sagar <[email protected]>

Disable direct speed change for the endpoint to prevent it from initiating the speed change post physical layer link up at gen1. This leaves the speed change ownership with the host.

Fixes c57247f940e8 ("PCI: tegra: Add support for PCIe endpoint mode in Tegra194")

---
Changes V1 -> V6: None

 drivers/pci/controller/dwc/pcie-tegra194.c | 4 ++++
 1 file changed, 4 insertions(+)
drivers/pci/controller/dwc/pcie-tegra194.c +0 -0
index 8f69c5c5a099..a6868b77e3b7 100644

drivers/pci/controller/dwc/pcie-tegra194.c static void pex_ep_event_pex_rst_deassert(struct tegra_pcie_dw *pcie) · line 1816

Quoted 10 lines
drivers/pci/controller/dwc/pcie-tegra194.c +4 -0
@@ -1807,6 +1807,10 @@ static void pex_ep_event_pex_rst_deassert(struct tegra_pcie_dw *pcie)
 
 	reset_control_deassert(pcie->core_rst);
 
+	val = dw_pcie_readl_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL);
+	val &= ~PORT_LOGIC_SPEED_CHANGE;
+	dw_pcie_writel_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL, val);
+
 	if (pcie->update_fc_fixup) {
 		val = dw_pcie_readl_dbi(pci, CFG_TIMER_CTRL_MAX_FUNC_NUM_OFF);
 		val |= 0x1 << CFG_TIMER_CTRL_ACK_NAK_SHIFT;
#14 lore raw

PATCH v6 7/13 PCI: tegra194: Set LTR message request before PCIe link up

In reply to Manikanta Maddireddy

From: Vidya Sagar <[email protected]>

LTR message should be sent as soon as the root port enables LTR in the endpoint. Set snoop & no snoop LTR timing and LTR message request before PCIe links up. This ensures that LTR message is sent upstream as soon as LTR is enabled.

Fixes c57247f940e8 ("PCI: tegra: Add support for PCIe endpoint mode in Tegra194")

---
Changes V1 -> V6: None

 drivers/pci/controller/dwc/pcie-tegra194.c | 15 ++++++---------
 1 file changed, 6 insertions(+), 9 deletions(-)
drivers/pci/controller/dwc/pcie-tegra194.c +6 -9
index a6868b77e3b7..ad1056d68d6d 100644
@@ -124,6 +124,7 @@
 
 #define APPL_LTR_MSG_1				0xC4
 #define LTR_MSG_REQ				BIT(15)
+#define LTR_MST_NO_SNOOP_SHIFT			16
 #define LTR_NOSNOOP_MSG_REQ			BIT(31)
 
 #define APPL_LTR_MSG_2				0xC8
@@ -488,15 +489,6 @@ static irqreturn_t tegra_pcie_ep_irq_thread(int irq, void *arg)
 	if (val & PCI_COMMAND_MASTER) {
 		ktime_t timeout;
 
-		/* 110us for both snoop and no-snoop */
-		val = FIELD_PREP(PCI_LTR_VALUE_MASK, 110) |
-		      FIELD_PREP(PCI_LTR_SCALE_MASK, 2) |
-		      LTR_MSG_REQ |
-		      FIELD_PREP(PCI_LTR_NOSNOOP_VALUE, 110) |
-		      FIELD_PREP(PCI_LTR_NOSNOOP_SCALE, 2) |
-		      LTR_NOSNOOP_MSG_REQ;
-		appl_writel(pcie, val, APPL_LTR_MSG_1);
-
 		/* Send LTR upstream */
 		val = appl_readl(pcie, APPL_LTR_MSG_2);
 		val |= APPL_LTR_MSG_2_LTR_MSG_REQ_STATE;
@@ -1805,6 +1797,11 @@ static void pex_ep_event_pex_rst_deassert(struct tegra_pcie_dw *pcie)
 	val |= APPL_INTR_EN_L1_0_0_RDLH_LINK_UP_INT_EN;
 	appl_writel(pcie, val, APPL_INTR_EN_L1_0_0);
 
+	/* 110us for both snoop and no-snoop */
+	val = 110 | (2 << PCI_LTR_SCALE_SHIFT) | LTR_MSG_REQ;
+	val |= (val << LTR_MST_NO_SNOOP_SHIFT);
+	appl_writel(pcie, val, APPL_LTR_MSG_1);
+
 	reset_control_deassert(pcie->core_rst);
 
 	val = dw_pcie_readl_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL);
Signature · 1 line
2.34.1
Reviewed-by on v6 7/13
#15 lore raw

Re: [PATCH v6 07/13] PCI: tegra194: Set LTR message request before PCIe link up

Quoted 46 lines · Manikanta Maddireddy

From: Vidya Sagar <[email protected]>

LTR message should be sent as soon as the root port enables LTR in the endpoint. Set snoop & no snoop LTR timing and LTR message request before PCIe links up. This ensures that LTR message is sent upstream as soon as LTR is enabled.

Fixes c57247f940e8 ("PCI: tegra: Add support for PCIe endpoint mode in Tegra194")

---
Changes V1 -> V6: None

 drivers/pci/controller/dwc/pcie-tegra194.c | 15 ++++++---------
 1 file changed, 6 insertions(+), 9 deletions(-)
drivers/pci/controller/dwc/pcie-tegra194.c +1 -9
index a6868b77e3b7..ad1056d68d6d 100644
@@ -124,6 +124,7 @@
 
 #define APPL_LTR_MSG_1				0xC4
 #define LTR_MSG_REQ				BIT(15)
+#define LTR_MST_NO_SNOOP_SHIFT			16
 #define LTR_NOSNOOP_MSG_REQ			BIT(31)
 
 #define APPL_LTR_MSG_2				0xC8
@@ -488,15 +489,6 @@ static irqreturn_t tegra_pcie_ep_irq_thread(int irq, void *arg)
 	if (val & PCI_COMMAND_MASTER) {
 		ktime_t timeout;
 
-		/* 110us for both snoop and no-snoop */
-		val = FIELD_PREP(PCI_LTR_VALUE_MASK, 110) |
-		      FIELD_PREP(PCI_LTR_SCALE_MASK, 2) |
-		      LTR_MSG_REQ |
-		      FIELD_PREP(PCI_LTR_NOSNOOP_VALUE, 110) |
-		      FIELD_PREP(PCI_LTR_NOSNOOP_SCALE, 2) |
-		      LTR_NOSNOOP_MSG_REQ;
-		appl_writel(pcie, val, APPL_LTR_MSG_1);
-
 		/* Send LTR upstream */
 		val = appl_readl(pcie, APPL_LTR_MSG_2);
 		val |= APPL_LTR_MSG_2_LTR_MSG_REQ_STATE;
@@ -1805,6 +1797,11 @@ static void pex_ep_event_pex_rst_deassert(struct tegra_pcie_dw *pcie)
 	val |= APPL_INTR_EN_L1_0_0_RDLH_LINK_UP_INT_EN;

drivers/pci/controller/dwc/pcie-tegra194.c static void pex_ep_event_pex_rst_deassert(struct tegra_pcie_dw *pcie) · line 1807

Quoted 10 lines
drivers/pci/controller/dwc/pcie-tegra194.c +5 -0
 	appl_writel(pcie, val, APPL_INTR_EN_L1_0_0);
 
+	/* 110us for both snoop and no-snoop */
+	val = 110 | (2 << PCI_LTR_SCALE_SHIFT) | LTR_MSG_REQ;
+	val |= (val << LTR_MST_NO_SNOOP_SHIFT);
+	appl_writel(pcie, val, APPL_LTR_MSG_1);
+
 	reset_control_deassert(pcie->core_rst);
 
 	val = dw_pcie_readl_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL);
#16 lore raw

Re: [PATCH v6 07/13] PCI: tegra194: Set LTR message request before PCIe link up

In reply to Manikanta Maddireddy
Quoted 21 lines · Manikanta Maddireddy

From: Vidya Sagar <[email protected]>

LTR message should be sent as soon as the root port enables LTR in the endpoint. Set snoop & no snoop LTR timing and LTR message request before PCIe links up. This ensures that LTR message is sent upstream as soon as LTR is enabled.

Fixes c57247f940e8 ("PCI: tegra: Add support for PCIe endpoint mode in Tegra194")

---
Changes V1 -> V6: None

 drivers/pci/controller/dwc/pcie-tegra194.c | 15 ++++++---------
 1 file changed, 6 insertions(+), 9 deletions(-)
drivers/pci/controller/dwc/pcie-tegra194.c +0 -0
index a6868b77e3b7..ad1056d68d6d 100644

drivers/pci/controller/dwc/pcie-tegra194.c line 127

Quoted 4 lines
drivers/pci/controller/dwc/pcie-tegra194.c +1 -0
@@ -124,6 +124,7 @@
 
 #define APPL_LTR_MSG_1				0xC4
 #define LTR_MSG_REQ				BIT(15)
+#define LTR_MST_NO_SNOOP_SHIFT			16

Would prefer to avoid adding more _SHIFT #defines, see below.

Quoted 20 lines
drivers/pci/controller/dwc/pcie-tegra194.c +0 -9
 #define LTR_NOSNOOP_MSG_REQ			BIT(31)
 
 #define APPL_LTR_MSG_2				0xC8
@@ -488,15 +489,6 @@ static irqreturn_t tegra_pcie_ep_irq_thread(int irq, void *arg)
 	if (val & PCI_COMMAND_MASTER) {
 		ktime_t timeout;
 
-		/* 110us for both snoop and no-snoop */
-		val = FIELD_PREP(PCI_LTR_VALUE_MASK, 110) |
-		      FIELD_PREP(PCI_LTR_SCALE_MASK, 2) |
-		      LTR_MSG_REQ |
-		      FIELD_PREP(PCI_LTR_NOSNOOP_VALUE, 110) |
-		      FIELD_PREP(PCI_LTR_NOSNOOP_SCALE, 2) |
-		      LTR_NOSNOOP_MSG_REQ;
-		appl_writel(pcie, val, APPL_LTR_MSG_1);
-
 		/* Send LTR upstream */
 		val = appl_readl(pcie, APPL_LTR_MSG_2);
 		val |= APPL_LTR_MSG_2_LTR_MSG_REQ_STATE;

drivers/pci/controller/dwc/pcie-tegra194.c static void pex_ep_event_pex_rst_deassert(struct tegra_pcie_dw *pcie) · line 1801

Quoted 5 lines
drivers/pci/controller/dwc/pcie-tegra194.c +2 -0
@@ -1805,6 +1797,11 @@ static void pex_ep_event_pex_rst_deassert(struct tegra_pcie_dw *pcie)
 	val |= APPL_INTR_EN_L1_0_0_RDLH_LINK_UP_INT_EN;
 	appl_writel(pcie, val, APPL_INTR_EN_L1_0_0);
 
+	/* 110us for both snoop and no-snoop */
+	val = 110 | (2 << PCI_LTR_SCALE_SHIFT) | LTR_MSG_REQ;

I get that we're moving this LTR setup from tegra_pcie_ep_irq_thread() to pex_ep_event_pex_rst_deassert(), but I don't understand why we lost the FIELD_PREP() usage. It's nice to be able to grep for PCI_LTR_VALUE_MASK, etc and find all the uses.

Quoted 10 lines
drivers/pci/controller/dwc/pcie-tegra194.c +3 -0
+	val |= (val << LTR_MST_NO_SNOOP_SHIFT);
 
+	appl_writel(pcie, val, APPL_LTR_MSG_1);
+
 	reset_control_deassert(pcie->core_rst);
 

val = dw_pcie_readl_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL);

Signature · 1 line
2.34.1
#17 lore raw

PATCH v6 8/13 PCI: tegra194: Apply pinctrl settings for both PCIe RP and EP

In reply to Manikanta Maddireddy

From: Vidya Sagar <[email protected]>

PERST# and CLKREQ# pinctrl settings should be applied for both root port and endpoint mode. Move pinctrl_pm_select_default_state() function call from root port specific configuration function to probe().

Fixes c57247f940e8 ("PCI: tegra: Add support for PCIe endpoint mode in Tegra194")

---
Changes V5 -> V6: None
Changes V4 -> V5: Use dev_err_probe() function
Changes V1 -> V4: None

 drivers/pci/controller/dwc/pcie-tegra194.c | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)
drivers/pci/controller/dwc/pcie-tegra194.c +4 -6
index ad1056d68d6d..61cbb4f230e1 100644
@@ -1601,12 +1601,6 @@ static int tegra_pcie_config_rp(struct tegra_pcie_dw *pcie)
 		goto fail_pm_get_sync;
 	}
 
-	ret = pinctrl_pm_select_default_state(dev);
-	if (ret < 0) {
-		dev_err(dev, "Failed to configure sideband pins: %d\n", ret);
-		goto fail_pm_get_sync;
-	}
-
 	ret = tegra_pcie_init_controller(pcie);
 	if (ret < 0) {
 		dev_err(dev, "Failed to initialize controller: %d\n", ret);
@@ -2076,6 +2070,10 @@ static int tegra_pcie_dw_probe(struct platform_device *pdev)
 	pp = &pci->pp;
 	pp->num_vectors = MAX_MSI_IRQS;
 
+	ret = pinctrl_pm_select_default_state(dev);
+	if (ret < 0)
+		return dev_err_probe(dev, ret, "Failed to configure sideband pins: %d\n", ret);
+
 	ret = tegra_pcie_dw_parse_dt(pcie);
 	if (ret < 0) {
 		const char *level = KERN_ERR;
Signature · 1 line
2.34.1
Reviewed-by on v6 8/13
#18 lore raw

Re: [PATCH v6 08/13] PCI: tegra194: Apply pinctrl settings for both PCIe RP and EP

Quoted 35 lines · Manikanta Maddireddy

From: Vidya Sagar <[email protected]>

PERST# and CLKREQ# pinctrl settings should be applied for both root port and endpoint mode. Move pinctrl_pm_select_default_state() function call from root port specific configuration function to probe().

Fixes c57247f940e8 ("PCI: tegra: Add support for PCIe endpoint mode in Tegra194")

---
Changes V5 -> V6: None
Changes V4 -> V5: Use dev_err_probe() function
Changes V1 -> V4: None

 drivers/pci/controller/dwc/pcie-tegra194.c | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)
drivers/pci/controller/dwc/pcie-tegra194.c +0 -6
index ad1056d68d6d..61cbb4f230e1 100644
@@ -1601,12 +1601,6 @@ static int tegra_pcie_config_rp(struct tegra_pcie_dw *pcie)
 		goto fail_pm_get_sync;
 	}
 
-	ret = pinctrl_pm_select_default_state(dev);
-	if (ret < 0) {
-		dev_err(dev, "Failed to configure sideband pins: %d\n", ret);
-		goto fail_pm_get_sync;
-	}
-
 	ret = tegra_pcie_init_controller(pcie);
 	if (ret < 0) {
 		dev_err(dev, "Failed to initialize controller: %d\n", ret);

drivers/pci/controller/dwc/pcie-tegra194.c static int tegra_pcie_dw_probe(struct platform_device *pdev) · line 2079

Quoted 10 lines
drivers/pci/controller/dwc/pcie-tegra194.c +4 -0
@@ -2076,6 +2070,10 @@ static int tegra_pcie_dw_probe(struct platform_device *pdev)
 	pp = &pci->pp;
 	pp->num_vectors = MAX_MSI_IRQS;
 
+	ret = pinctrl_pm_select_default_state(dev);
+	if (ret < 0)
+		return dev_err_probe(dev, ret, "Failed to configure sideband pins: %d\n", ret);
+
 	ret = tegra_pcie_dw_parse_dt(pcie);
 	if (ret < 0) {
 		const char *level = KERN_ERR;
#19 lore raw

PATCH v6 9/13 PCI: tegra194: Allow system suspend when the Endpoint link is not up

In reply to Manikanta Maddireddy

From: Vidya Sagar <[email protected]>

Only a Root port initiates the L2 sequence. PCIe link is kept in L2 state during suspend. If Endpoint mode is enabled and the link is up, the software cannot proceed with suspend. However, when the PCIe Endpoint driver is probed, but the PCIe link is not up, Tegra can go into suspend state. So, allow system to suspend in this case.

Fixes de2bbf2b71bb ("PCI: tegra194: Don't allow suspend when Tegra PCIe is in EP mode")

---
Changes V1 -> V6: None

 drivers/pci/controller/dwc/pcie-tegra194.c | 31 +++++++++++++++++-----
 1 file changed, 25 insertions(+), 6 deletions(-)
drivers/pci/controller/dwc/pcie-tegra194.c +25 -6
index 61cbb4f230e1..58069b1615b8 100644
@@ -2266,16 +2266,28 @@ static void tegra_pcie_dw_remove(struct platform_device *pdev)
 		gpiod_set_value(pcie->pex_refclk_sel_gpiod, 0);
 }
 
-static int tegra_pcie_dw_suspend_late(struct device *dev)
+static int tegra_pcie_dw_suspend(struct device *dev)
 {
 	struct tegra_pcie_dw *pcie = dev_get_drvdata(dev);
-	u32 val;
 
 	if (pcie->of_data->mode == DW_PCIE_EP_TYPE) {
-		dev_err(dev, "Failed to Suspend as Tegra PCIe is in EP mode\n");
-		return -EPERM;
+		if (pcie->ep_state == EP_STATE_ENABLED) {
+			dev_err(dev, "Tegra PCIe is in EP mode, suspend not allowed\n");
+			return -EPERM;
+		}
+
+		disable_irq(pcie->pex_rst_irq);
+		return 0;
 	}
 
+	return 0;
+}
+
+static int tegra_pcie_dw_suspend_late(struct device *dev)
+{
+	struct tegra_pcie_dw *pcie = dev_get_drvdata(dev);
+	u32 val;
+
 	if (!pcie->link_state)
 		return 0;
 
@@ -2295,6 +2307,9 @@ static int tegra_pcie_dw_suspend_noirq(struct device *dev)
 {
 	struct tegra_pcie_dw *pcie = dev_get_drvdata(dev);
 
+	if (pcie->of_data->mode == DW_PCIE_EP_TYPE)
+		return 0;
+
 	if (!pcie->link_state)
 		return 0;
 
@@ -2309,6 +2324,9 @@ static int tegra_pcie_dw_resume_noirq(struct device *dev)
 	struct tegra_pcie_dw *pcie = dev_get_drvdata(dev);
 	int ret;
 
+	if (pcie->of_data->mode == DW_PCIE_EP_TYPE)
+		return 0;
+
 	if (!pcie->link_state)
 		return 0;
 
@@ -2341,8 +2359,8 @@ static int tegra_pcie_dw_resume_early(struct device *dev)
 	u32 val;
 
 	if (pcie->of_data->mode == DW_PCIE_EP_TYPE) {
-		dev_err(dev, "Suspend is not supported in EP mode");
-		return -ENOTSUPP;
+		enable_irq(pcie->pex_rst_irq);
+		return 0;
 	}
 
 	if (!pcie->link_state)
@@ -2447,6 +2465,7 @@ static const struct of_device_id tegra_pcie_dw_of_match[] = {
 };
 
 static const struct dev_pm_ops tegra_pcie_dw_pm_ops = {
+	.suspend = tegra_pcie_dw_suspend,
 	.suspend_late = tegra_pcie_dw_suspend_late,
 	.suspend_noirq = tegra_pcie_dw_suspend_noirq,
 	.resume_noirq = tegra_pcie_dw_resume_noirq,
Signature · 1 line
2.34.1
Reviewed-by on v6 9/13
#20 lore raw

Re: [PATCH v6 09/13] PCI: tegra194: Allow system suspend when the Endpoint link is not up

Quoted 86 lines · Manikanta Maddireddy

From: Vidya Sagar <[email protected]>

Only a Root port initiates the L2 sequence. PCIe link is kept in L2 state during suspend. If Endpoint mode is enabled and the link is up, the software cannot proceed with suspend. However, when the PCIe Endpoint driver is probed, but the PCIe link is not up, Tegra can go into suspend state. So, allow system to suspend in this case.

Fixes de2bbf2b71bb ("PCI: tegra194: Don't allow suspend when Tegra PCIe is in EP mode")

---
Changes V1 -> V6: None

 drivers/pci/controller/dwc/pcie-tegra194.c | 31 +++++++++++++++++-----
 1 file changed, 25 insertions(+), 6 deletions(-)
drivers/pci/controller/dwc/pcie-tegra194.c +24 -6
index 61cbb4f230e1..58069b1615b8 100644
@@ -2266,16 +2266,28 @@ static void tegra_pcie_dw_remove(struct platform_device *pdev)
 		gpiod_set_value(pcie->pex_refclk_sel_gpiod, 0);
 }
 
-static int tegra_pcie_dw_suspend_late(struct device *dev)
+static int tegra_pcie_dw_suspend(struct device *dev)
 {
 	struct tegra_pcie_dw *pcie = dev_get_drvdata(dev);
-	u32 val;
 
 	if (pcie->of_data->mode == DW_PCIE_EP_TYPE) {
-		dev_err(dev, "Failed to Suspend as Tegra PCIe is in EP mode\n");
-		return -EPERM;
+		if (pcie->ep_state == EP_STATE_ENABLED) {
+			dev_err(dev, "Tegra PCIe is in EP mode, suspend not allowed\n");
+			return -EPERM;
+		}
+
+		disable_irq(pcie->pex_rst_irq);
+		return 0;
 	}
 
+	return 0;
+}
+
+static int tegra_pcie_dw_suspend_late(struct device *dev)
+{
+	struct tegra_pcie_dw *pcie = dev_get_drvdata(dev);
+	u32 val;
+
 	if (!pcie->link_state)
 		return 0;
 
@@ -2295,6 +2307,9 @@ static int tegra_pcie_dw_suspend_noirq(struct device *dev)
 {
 	struct tegra_pcie_dw *pcie = dev_get_drvdata(dev);
 
+	if (pcie->of_data->mode == DW_PCIE_EP_TYPE)
+		return 0;
+
 	if (!pcie->link_state)
 		return 0;
 
@@ -2309,6 +2324,9 @@ static int tegra_pcie_dw_resume_noirq(struct device *dev)
 	struct tegra_pcie_dw *pcie = dev_get_drvdata(dev);
 	int ret;
 
+	if (pcie->of_data->mode == DW_PCIE_EP_TYPE)
+		return 0;
+
 	if (!pcie->link_state)
 		return 0;
 
@@ -2341,8 +2359,8 @@ static int tegra_pcie_dw_resume_early(struct device *dev)
 	u32 val;
 
 	if (pcie->of_data->mode == DW_PCIE_EP_TYPE) {
-		dev_err(dev, "Suspend is not supported in EP mode");
-		return -ENOTSUPP;
+		enable_irq(pcie->pex_rst_irq);
+		return 0;
 	}
 
 	if (!pcie->link_state)

drivers/pci/controller/dwc/pcie-tegra194.c static const struct of_device_id tegra_pcie_dw_of_match[] = { · line 2471

Quoted 7 lines
drivers/pci/controller/dwc/pcie-tegra194.c +1 -0
@@ -2447,6 +2465,7 @@ static const struct of_device_id tegra_pcie_dw_of_match[] = {
 };
 
 static const struct dev_pm_ops tegra_pcie_dw_pm_ops = {
+	.suspend = tegra_pcie_dw_suspend,
 	.suspend_late = tegra_pcie_dw_suspend_late,
 	.suspend_noirq = tegra_pcie_dw_suspend_noirq,
 	.resume_noirq = tegra_pcie_dw_resume_noirq,
#21 lore raw

Re: [PATCH v6 09/13] PCI: tegra194: Allow system suspend when the Endpoint link is not up

In reply to Manikanta Maddireddy
Quoted 7 lines · Manikanta Maddireddy

From: Vidya Sagar <[email protected]>

Only a Root port initiates the L2 sequence. PCIe link is kept in L2 state during suspend. If Endpoint mode is enabled and the link is up, the software cannot proceed with suspend. However, when the PCIe Endpoint driver is probed, but the PCIe link is not up, Tegra can go into suspend state. So, allow system to suspend in this case.

Would be nice to consistently use "Root Port" throughout the series instead of the current mix of "root port", "Root port", "Root Port". Using "Endpoint" is also nice because these are proper nouns defined by the PCIe spec.

#22 lore raw

PATCH v6 10/13 PCI: tegra194: Free up EP resources during remove()

In reply to Manikanta Maddireddy

From: Vidya Sagar <[email protected]>

Free up the resources during remove() that were acquired by the DesignWare driver for the endpoint mode during probe().

Fixes bb617cbd8151 ("PCI: tegra194: Clean up the exit path for Endpoint mode")

---
Changes V1 -> V6: None

 drivers/pci/controller/dwc/pcie-tegra194.c | 2 ++
 1 file changed, 2 insertions(+)
drivers/pci/controller/dwc/pcie-tegra194.c +2 -0
index 58069b1615b8..d225165a3aa2 100644
@@ -2247,6 +2247,7 @@ static int tegra_pcie_dw_probe(struct platform_device *pdev)
 static void tegra_pcie_dw_remove(struct platform_device *pdev)
 {
 	struct tegra_pcie_dw *pcie = platform_get_drvdata(pdev);
+	struct dw_pcie_ep *ep = &pcie->pci.ep;
 
 	if (pcie->of_data->mode == DW_PCIE_RC_TYPE) {
 		if (!pcie->link_state)
@@ -2258,6 +2259,7 @@ static void tegra_pcie_dw_remove(struct platform_device *pdev)
 	} else {
 		disable_irq(pcie->pex_rst_irq);
 		pex_ep_event_pex_rst_assert(pcie);
+		dw_pcie_ep_deinit(ep);
 	}
 
 	pm_runtime_disable(pcie->dev);
Signature · 1 line
2.34.1
Reviewed-by on v6 10/13
#23 lore raw

Re: [PATCH v6 10/13] PCI: tegra194: Free up EP resources during remove()

Quoted 27 lines · Manikanta Maddireddy

From: Vidya Sagar <[email protected]>

Free up the resources during remove() that were acquired by the DesignWare driver for the endpoint mode during probe().

Fixes bb617cbd8151 ("PCI: tegra194: Clean up the exit path for Endpoint mode")

---
Changes V1 -> V6: None

 drivers/pci/controller/dwc/pcie-tegra194.c | 2 ++
 1 file changed, 2 insertions(+)
drivers/pci/controller/dwc/pcie-tegra194.c +1 -0
index 58069b1615b8..d225165a3aa2 100644
@@ -2247,6 +2247,7 @@ static int tegra_pcie_dw_probe(struct platform_device *pdev)
 static void tegra_pcie_dw_remove(struct platform_device *pdev)
 {
 	struct tegra_pcie_dw *pcie = platform_get_drvdata(pdev);
+	struct dw_pcie_ep *ep = &pcie->pci.ep;
 
 	if (pcie->of_data->mode == DW_PCIE_RC_TYPE) {
 		if (!pcie->link_state)

drivers/pci/controller/dwc/pcie-tegra194.c static void tegra_pcie_dw_remove(struct platform_device *pdev) · line 2265

Quoted 7 lines
drivers/pci/controller/dwc/pcie-tegra194.c +1 -0
@@ -2258,6 +2259,7 @@ static void tegra_pcie_dw_remove(struct platform_device *pdev)
 	} else {
 		disable_irq(pcie->pex_rst_irq);
 		pex_ep_event_pex_rst_assert(pcie);
+		dw_pcie_ep_deinit(ep);
 	}
 
 	pm_runtime_disable(pcie->dev);
#24 lore raw

PATCH v6 11/13 PCI: tegra194: Use HW version number

In reply to Manikanta Maddireddy

Tegra194 PCIe driver uses custom version number to detect Tegra194 and Tegra234 IPs. With version detect logic added, version check results in mismatch warnings.

Use HW version numbers in Tegra194 driver to avoid this kernel warnings.

Fixed version check to enable ecrc for Tegra194.
Existing 490A check is left intact in case any HW relying on existing check.

Fixes a54e19073718 ("PCI: tegra194: Add Tegra234 PCIe support")

---
Changes V1 -> V6: None

 drivers/pci/controller/dwc/pcie-designware.c | 2 +-
 drivers/pci/controller/dwc/pcie-designware.h | 2 ++
 drivers/pci/controller/dwc/pcie-tegra194.c   | 4 ++--
 3 files changed, 5 insertions(+), 3 deletions(-)
drivers/pci/controller/dwc/pcie-designware.c +1 -1
index 345365ea97c7..0dac5d2f5a83 100644
@@ -559,7 +559,7 @@ int dw_pcie_prog_outbound_atu(struct dw_pcie *pci,
 	if (upper_32_bits(limit_addr) > upper_32_bits(parent_bus_addr) &&
 	    dw_pcie_ver_is_ge(pci, 460A))
 		val |= PCIE_ATU_INCREASE_REGION_SIZE;
-	if (dw_pcie_ver_is(pci, 490A))
+	if (dw_pcie_ver_is(pci, 490A) || dw_pcie_ver_is(pci, 500A))
 		val = dw_pcie_enable_ecrc(val);
 	dw_pcie_writel_atu_ob(pci, atu->index, PCIE_ATU_REGION_CTRL1, val);
 
drivers/pci/controller/dwc/pcie-designware.h +2 -0
index f4cf1602cc99..5bceadbd2c9f 100644
@@ -34,8 +34,10 @@
 #define DW_PCIE_VER_470A		0x3437302a
 #define DW_PCIE_VER_480A		0x3438302a
 #define DW_PCIE_VER_490A		0x3439302a
+#define DW_PCIE_VER_500A		0x3530302a
 #define DW_PCIE_VER_520A		0x3532302a
 #define DW_PCIE_VER_540A		0x3534302a
+#define DW_PCIE_VER_562A		0x3536322a
 
 #define __dw_pcie_ver_cmp(_pci, _ver, _op) \
 	((_pci)->version _op DW_PCIE_VER_ ## _ver)
drivers/pci/controller/dwc/pcie-tegra194.c +2 -2
index d225165a3aa2..f107f2eb98fd 100644
@@ -35,8 +35,8 @@
 #include <soc/tegra/bpmp-abi.h>
 #include "../../pci.h"
 
-#define TEGRA194_DWC_IP_VER			0x490A
-#define TEGRA234_DWC_IP_VER			0x562A
+#define TEGRA194_DWC_IP_VER			DW_PCIE_VER_500A
+#define TEGRA234_DWC_IP_VER			DW_PCIE_VER_562A
 
 #define APPL_PINMUX				0x0
 #define APPL_PINMUX_PEX_RST			BIT(0)
Signature · 1 line
2.34.1
Reviewed-by on v6 11/13
#25 lore raw

Re: [PATCH v6 11/13] PCI: tegra194: Use HW version number

Quoted 52 lines · Manikanta Maddireddy

Tegra194 PCIe driver uses custom version number to detect Tegra194 and Tegra234 IPs. With version detect logic added, version check results in mismatch warnings.

Use HW version numbers in Tegra194 driver to avoid this kernel warnings.

Fixed version check to enable ecrc for Tegra194.
Existing 490A check is left intact in case any HW relying on existing check.

Fixes a54e19073718 ("PCI: tegra194: Add Tegra234 PCIe support")

---
Changes V1 -> V6: None

 drivers/pci/controller/dwc/pcie-designware.c | 2 +-
 drivers/pci/controller/dwc/pcie-designware.h | 2 ++
 drivers/pci/controller/dwc/pcie-tegra194.c   | 4 ++--
 3 files changed, 5 insertions(+), 3 deletions(-)
drivers/pci/controller/dwc/pcie-designware.c +1 -1
index 345365ea97c7..0dac5d2f5a83 100644
@@ -559,7 +559,7 @@ int dw_pcie_prog_outbound_atu(struct dw_pcie *pci,
 	if (upper_32_bits(limit_addr) > upper_32_bits(parent_bus_addr) &&
 	    dw_pcie_ver_is_ge(pci, 460A))
 		val |= PCIE_ATU_INCREASE_REGION_SIZE;
-	if (dw_pcie_ver_is(pci, 490A))
+	if (dw_pcie_ver_is(pci, 490A) || dw_pcie_ver_is(pci, 500A))
 		val = dw_pcie_enable_ecrc(val);
 	dw_pcie_writel_atu_ob(pci, atu->index, PCIE_ATU_REGION_CTRL1, val);
 
drivers/pci/controller/dwc/pcie-designware.h +2 -0
index f4cf1602cc99..5bceadbd2c9f 100644
@@ -34,8 +34,10 @@
 #define DW_PCIE_VER_470A		0x3437302a
 #define DW_PCIE_VER_480A		0x3438302a
 #define DW_PCIE_VER_490A		0x3439302a
+#define DW_PCIE_VER_500A		0x3530302a
 #define DW_PCIE_VER_520A		0x3532302a
 #define DW_PCIE_VER_540A		0x3534302a
+#define DW_PCIE_VER_562A		0x3536322a
 
 #define __dw_pcie_ver_cmp(_pci, _ver, _op) \
 	((_pci)->version _op DW_PCIE_VER_ ## _ver)
drivers/pci/controller/dwc/pcie-tegra194.c +0 -0
index d225165a3aa2..f107f2eb98fd 100644

drivers/pci/controller/dwc/pcie-tegra194.c line 42

Quoted 10 lines
drivers/pci/controller/dwc/pcie-tegra194.c +2 -2
@@ -35,8 +35,8 @@
 #include <soc/tegra/bpmp-abi.h>
 #include "../../pci.h"
 
-#define TEGRA194_DWC_IP_VER			0x490A
-#define TEGRA234_DWC_IP_VER			0x562A
+#define TEGRA194_DWC_IP_VER			DW_PCIE_VER_500A
+#define TEGRA234_DWC_IP_VER			DW_PCIE_VER_562A
 
 #define APPL_PINMUX				0x0
 #define APPL_PINMUX_PEX_RST			BIT(0)
#26 lore raw

PATCH v6 12/13 PCI: tegra194: Fix CBB timeout caused by DBI access before core power-on

In reply to Manikanta Maddireddy

When PERST# is deasserted twice (assert -> deassert -> assert -> deassert), a CBB (Control Backbone) timeout occurs at DBI register offset 0x8bc (PCIE_MISC_CONTROL_1_OFF). This happens because pci_epc_deinit_notify() and dw_pcie_ep_cleanup() are called before reset_control_deassert() powers on the controller core.

The call chain that causes the timeout:
pex_ep_event_pex_rst_deassert()

    pci_epc_deinit_notify()
      pci_epf_test_epc_deinit()
        pci_epf_test_clear_bar()
          pci_epc_clear_bar()
            dw_pcie_ep_clear_bar()
              __dw_pcie_ep_reset_bar()
                dw_pcie_dbi_ro_wr_en()  <- Accesses 0x8bc DBI register
    reset_control_deassert(pcie->core_rst)  <- Core powered on HERE

The DBI registers, including PCIE_MISC_CONTROL_1_OFF (0x8bc), are only accessible after the controller core is powered on via reset_control_deassert(pcie->core_rst). Accessing them before this point results in a CBB timeout because the hardware is not yet operational.

Fix this by moving pci_epc_deinit_notify() and dw_pcie_ep_cleanup() to after reset_control_deassert(pcie->core_rst), ensuring the controller is fully powered on before any DBI register accesses occur.

Fixes 40e2125381dc ("PCI: tegra194: Move controller cleanups to pex_ep_event_pex_rst_deassert()")

---
Changes V1 -> V6: None

 drivers/pci/controller/dwc/pcie-tegra194.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
drivers/pci/controller/dwc/pcie-tegra194.c +4 -4
index f107f2eb98fd..256a5d1eba16 100644
@@ -1729,10 +1729,6 @@ static void pex_ep_event_pex_rst_deassert(struct tegra_pcie_dw *pcie)
 		goto fail_phy;
 	}
 
-	/* Perform cleanup that requires refclk */
-	pci_epc_deinit_notify(pcie->pci.ep.epc);
-	dw_pcie_ep_cleanup(&pcie->pci.ep);
-
 	/* Clear any stale interrupt statuses */
 	appl_writel(pcie, 0xFFFFFFFF, APPL_INTR_STATUS_L0);
 	appl_writel(pcie, 0xFFFFFFFF, APPL_INTR_STATUS_L1_0_0);
@@ -1798,6 +1794,10 @@ static void pex_ep_event_pex_rst_deassert(struct tegra_pcie_dw *pcie)
 
 	reset_control_deassert(pcie->core_rst);
 
+	/* Perform cleanup that requires refclk and core reset deasserted */
+	pci_epc_deinit_notify(pcie->pci.ep.epc);
+	dw_pcie_ep_cleanup(&pcie->pci.ep);
+
 	val = dw_pcie_readl_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL);
 	val &= ~PORT_LOGIC_SPEED_CHANGE;
 	dw_pcie_writel_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL, val);
Signature · 1 line
2.34.1
Reviewed-by on v6 12/13
#27 lore raw

Re: [PATCH v6 12/13] PCI: tegra194: Fix CBB timeout caused by DBI access before core power-on

Quoted 50 lines · Manikanta Maddireddy

When PERST# is deasserted twice (assert -> deassert -> assert -> deassert), a CBB (Control Backbone) timeout occurs at DBI register offset 0x8bc (PCIE_MISC_CONTROL_1_OFF). This happens because pci_epc_deinit_notify() and dw_pcie_ep_cleanup() are called before reset_control_deassert() powers on the controller core.

The call chain that causes the timeout:
pex_ep_event_pex_rst_deassert()

    pci_epc_deinit_notify()
      pci_epf_test_epc_deinit()
        pci_epf_test_clear_bar()
          pci_epc_clear_bar()
            dw_pcie_ep_clear_bar()
              __dw_pcie_ep_reset_bar()
                dw_pcie_dbi_ro_wr_en()  <- Accesses 0x8bc DBI register
    reset_control_deassert(pcie->core_rst)  <- Core powered on HERE

The DBI registers, including PCIE_MISC_CONTROL_1_OFF (0x8bc), are only accessible after the controller core is powered on via reset_control_deassert(pcie->core_rst). Accessing them before this point results in a CBB timeout because the hardware is not yet operational.

Fix this by moving pci_epc_deinit_notify() and dw_pcie_ep_cleanup() to after reset_control_deassert(pcie->core_rst), ensuring the controller is fully powered on before any DBI register accesses occur.

Fixes 40e2125381dc ("PCI: tegra194: Move controller cleanups to pex_ep_event_pex_rst_deassert()")

---
Changes V1 -> V6: None

 drivers/pci/controller/dwc/pcie-tegra194.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
drivers/pci/controller/dwc/pcie-tegra194.c +0 -4
index f107f2eb98fd..256a5d1eba16 100644
@@ -1729,10 +1729,6 @@ static void pex_ep_event_pex_rst_deassert(struct tegra_pcie_dw *pcie)
 		goto fail_phy;
 	}
 
-	/* Perform cleanup that requires refclk */
-	pci_epc_deinit_notify(pcie->pci.ep.epc);
-	dw_pcie_ep_cleanup(&pcie->pci.ep);
-
 	/* Clear any stale interrupt statuses */
 	appl_writel(pcie, 0xFFFFFFFF, APPL_INTR_STATUS_L0);
 	appl_writel(pcie, 0xFFFFFFFF, APPL_INTR_STATUS_L1_0_0);

drivers/pci/controller/dwc/pcie-tegra194.c static void pex_ep_event_pex_rst_deassert(struct tegra_pcie_dw *pcie) · line 1803

Quoted 10 lines
drivers/pci/controller/dwc/pcie-tegra194.c +4 -0
@@ -1798,6 +1794,10 @@ static void pex_ep_event_pex_rst_deassert(struct tegra_pcie_dw *pcie)
 
 	reset_control_deassert(pcie->core_rst);
 
+	/* Perform cleanup that requires refclk and core reset deasserted */
+	pci_epc_deinit_notify(pcie->pci.ep.epc);
+	dw_pcie_ep_cleanup(&pcie->pci.ep);
+
 	val = dw_pcie_readl_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL);
 	val &= ~PORT_LOGIC_SPEED_CHANGE;
 	dw_pcie_writel_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL, val);
#28 lore raw

PATCH v6 13/13 PCI: tegra194: Free resources during controller deinitialization

In reply to Manikanta Maddireddy

From: Vidya Sagar <[email protected]>

Call pci_epc_deinit_notify() during controller deinitialization to free the resources allocated by Endpoint function driver. This is safe to call during PCIe assert sequence because we don't expect Endpoint function driver to touch hardware in deinit function.

Fixes 40e2125381dc ("PCI: tegra194: Move controller cleanups to pex_ep_event_pex_rst_deassert()")

---
Changes V5 -> V6: None
Changes V4 -> V5: Remove pci_epc_deinit_notify() call from pex_ep_event_pex_rst_deassert() Changes V1 -> V4: None

 drivers/pci/controller/dwc/pcie-tegra194.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
drivers/pci/controller/dwc/pcie-tegra194.c +4 -1
index 256a5d1eba16..9883d14f7f97 100644
@@ -1627,12 +1627,16 @@ static int tegra_pcie_config_rp(struct tegra_pcie_dw *pcie)
 
 static void pex_ep_event_pex_rst_assert(struct tegra_pcie_dw *pcie)
 {
+	struct dw_pcie *pci = &pcie->pci;
+	struct dw_pcie_ep *ep = &pci->ep;
 	u32 val;
 	int ret;
 
 	if (pcie->ep_state == EP_STATE_DISABLED)
 		return;
 
+	pci_epc_deinit_notify(ep->epc);
+
 	ret = readl_poll_timeout(pcie->appl_base + APPL_DEBUG, val,
 		((val & APPL_DEBUG_LTSSM_STATE_MASK) == LTSSM_STATE_DETECT_QUIET) ||
 		((val & APPL_DEBUG_LTSSM_STATE_MASK) == LTSSM_STATE_DETECT_ACT) ||
@@ -1795,7 +1799,6 @@ static void pex_ep_event_pex_rst_deassert(struct tegra_pcie_dw *pcie)
 	reset_control_deassert(pcie->core_rst);
 
 	/* Perform cleanup that requires refclk and core reset deasserted */
-	pci_epc_deinit_notify(pcie->pci.ep.epc);
 	dw_pcie_ep_cleanup(&pcie->pci.ep);
 
 	val = dw_pcie_readl_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL);
Signature · 1 line
2.34.1
Reviewed-by on v6 13/13
#29 lore raw

Re: [PATCH v6 13/13] PCI: tegra194: Free resources during controller deinitialization

Quoted 40 lines · Manikanta Maddireddy

From: Vidya Sagar <[email protected]>

Call pci_epc_deinit_notify() during controller deinitialization to free the resources allocated by Endpoint function driver. This is safe to call during PCIe assert sequence because we don't expect Endpoint function driver to touch hardware in deinit function.

Fixes 40e2125381dc ("PCI: tegra194: Move controller cleanups to pex_ep_event_pex_rst_deassert()")

---
Changes V5 -> V6: None
Changes V4 -> V5: Remove pci_epc_deinit_notify() call from pex_ep_event_pex_rst_deassert() Changes V1 -> V4: None

 drivers/pci/controller/dwc/pcie-tegra194.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
drivers/pci/controller/dwc/pcie-tegra194.c +4 -0
index 256a5d1eba16..9883d14f7f97 100644
@@ -1627,12 +1627,16 @@ static int tegra_pcie_config_rp(struct tegra_pcie_dw *pcie)
 
 static void pex_ep_event_pex_rst_assert(struct tegra_pcie_dw *pcie)
 {
+	struct dw_pcie *pci = &pcie->pci;
+	struct dw_pcie_ep *ep = &pci->ep;
 	u32 val;
 	int ret;
 
 	if (pcie->ep_state == EP_STATE_DISABLED)
 		return;
 
+	pci_epc_deinit_notify(ep->epc);
+
 	ret = readl_poll_timeout(pcie->appl_base + APPL_DEBUG, val,
 		((val & APPL_DEBUG_LTSSM_STATE_MASK) == LTSSM_STATE_DETECT_QUIET) ||
 		((val & APPL_DEBUG_LTSSM_STATE_MASK) == LTSSM_STATE_DETECT_ACT) ||

drivers/pci/controller/dwc/pcie-tegra194.c static void pex_ep_event_pex_rst_deassert(struct tegra_pcie_dw *pcie) · line 1804

Quoted 7 lines
drivers/pci/controller/dwc/pcie-tegra194.c +0 -1
@@ -1795,7 +1799,6 @@ static void pex_ep_event_pex_rst_deassert(struct tegra_pcie_dw *pcie)
 	reset_control_deassert(pcie->core_rst);
 
 	/* Perform cleanup that requires refclk and core reset deasserted */
-	pci_epc_deinit_notify(pcie->pci.ep.epc);
 	dw_pcie_ep_cleanup(&pcie->pci.ep);
 
 	val = dw_pcie_readl_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL);
maintainer Reviewed-by Tested-by on the v6 series
#30 lore raw
In reply to Manikanta Maddireddy
Quoted 44 lines · Manikanta Maddireddy

This series[1] was originally posted by Vidya Sagar, and I have rebased it onto 6.19.0-rc6-next. I addressed review comments and split this into two series, one for fixes(current) and the other is for enhancements. I verified these patches on Jetson AGX Orin(Tegra234 SoC).

I added below four new patches to fix bugs, commit message of each patch has the details on the bug and fix.
- PCI: tegra194: Use HW version number
- PCI: tegra194: Fix CBB timeout caused by DBI access before core power-on
- PCI: tegra194: Disable PERST IRQ only in Endpoint mode

Rest of the patches are same as the original V3 series, just rebased them on 6.19.0-rc6-next.

I fixed the missing PATCH prefix in the subject in V6 and also reordered the patches with regard to fixes tag.

Verification details.
- I verified both Root port and Endpoint mode controllers on Tegra234 SoC.
- Basic sanity Link up, configuration space access and BAR access are verified.
- I verified that ASPM L1.2 capability is disabled for Endpoint mode.
- I verified suspend to RAM tests with Endpoint mode.

[1] https://patchwork.kernel.org/project/linux-pci/patch/[email protected]/

Manikanta Maddireddy (3):
PCI: tegra194: Disable PERST IRQ only in Endpoint mode
PCI: tegra194: Use HW version number
PCI: tegra194: Fix CBB timeout caused by DBI access before core
power-on

Vidya Sagar (10):
PCI: tegra194: Fix polling delay for L2 state
PCI: tegra194: Refactor LTSSM state polling on surprise down
PCI: tegra194: Don't force the device into the D0 state before L2
PCI: tegra194: Use devm_gpiod_get_optional() to parse
"nvidia,refclk-select"
PCI: tegra194: Disable direct speed change for EP
PCI: tegra194: Set LTR message request before PCIe link up
PCI: tegra194: Apply pinctrl settings for both PCIe RP and EP
PCI: tegra194: Allow system suspend when the Endpoint link is not up
PCI: tegra194: Free up EP resources during remove()
PCI: tegra194: Free resources during controller deinitialization

Quoted 5 lines
  drivers/pci/controller/dwc/pcie-designware.c |   2 +-
  drivers/pci/controller/dwc/pcie-designware.h |   2 +
  drivers/pci/controller/dwc/pcie-tegra194.c   | 191 +++++++++----------
  3 files changed, 95 insertions(+), 100 deletions(-)

For the series ...

Thanks
Jon

Signature · 1 line
nvpublic