Enhancements to pcie-tegra194 driver
Subsystems (5)
v5 cover Enhancements to pcie-tegra194 driver
This series[1] was originally posted by Vidya Sagar, and I have rebased it onto 6.19.0-rc6-next. I addressed review comments and split this into two series, one for fixes and the other is for enhancements(current). I verified these patches on Jetson AGX Orin(Tegra234 SoC).
Disabling L1SS capability based on support-clkreq device tree property is moved to common DWC driver, so I reworked below patch to disable just L1.2 capability on Tegra234 SoC.
- PCI: tegra194: Disable L1.2 capability of Tegra234 EP
I added below new patch to tune the ASPM parameters, commit message of the patch has the details on the bug and fix.
- PCI: tegra194: Add ASPM L1 entrance latency config
I added more context in the commit message for below patch based on review comment in V3.
- PCI: tegra194: Remove IRQF_ONESHOT flag during Endpoint interrupt registration
Rest of the patches are same as the original V3 series, just rebased them on 6.19.0-rc6-next.
Verification details.
- I verified both Root port and Endpoint mode controllers on Tegra234 SoC.
- Basic sanity Link up, configuration space access and BAR access are verified.
- I verified that ASPM L1.2 capability is disabled for Endpoint mode.
- I verified suspend to RAM tests with Endpoint mode.
[1] https://patchwork.kernel.org/project/linux-pci/patch/[email protected]/
Manikanta Maddireddy (1):
PCI: tegra194: Add ASPM L1 entrance latency config
Vidya Sagar (8):
PCI: tegra194: Drive CLKREQ signal low explicitly
PCI: tegra194: Calibrate P2U for endpoint mode
PCI: tegra194: Remove IRQF_ONESHOT flag during Endpoint interrupt
registration
PCI: tegra194: Enable DMA interrupt
PCI: tegra194: Enable hardware hot reset mode in Endpoint
PCI: tegra194: Disable L1.2 capability of Tegra234 EP
dt-bindings: PCI: tegra194: Add monitor clock support
PCI: tegra194: Add core monitor clock support
.../bindings/pci/nvidia,tegra194-pcie-ep.yaml | 6 +- .../bindings/pci/nvidia,tegra194-pcie.yaml | 6 +- drivers/pci/controller/dwc/pcie-tegra194.c | 67 ++++++++++++++++++- 3 files changed, 76 insertions(+), 3 deletions(-)
Signature · 1 line
2.34.1
PATCH v5 1/9 PCI: tegra194: Drive CLKREQ signal low explicitly
From: Vidya Sagar <[email protected]>
Currently, the default setting is that CLKREQ signal of a Root Port is internally overridden to '0' to enable REFCLK to flow out to the slot. It is observed that one of the PCIe switches (case in point Broadcom PCIe Gen4 switch) is propagating the CLKREQ signal of the root port to the downstream side of the switch and expecting the endpoints to pull it low so that it (PCIe switch) can give out the REFCLK although the Switch as such doesn't support CLK-PM or ASPM-L1SS. So, as a workaround, this patch drives the CLKREQ of the Root Port itself low to avoid link up issues between PCIe switch downstream port and endpoints. This is not a wrong thing to do after all the CLKREQ is anyway being overridden to '0' internally and now it is just that the same is being propagated outside also.
---
V5:
* None
V4:
* None
V3:
* None
V2:
* None
drivers/pci/controller/dwc/pcie-tegra194.c | 2 ++ 1 file changed, 2 insertions(+)
drivers/pci/controller/dwc/pcie-tegra194.c +2 -0
@@ -44,6 +44,7 @@#define APPL_PINMUX_CLKREQ_OVERRIDE BIT(3)#define APPL_PINMUX_CLK_OUTPUT_IN_OVERRIDE_EN BIT(4)#define APPL_PINMUX_CLK_OUTPUT_IN_OVERRIDE BIT(5) +#define APPL_PINMUX_CLKREQ_DEFAULT_VALUE BIT(13)#define APPL_CTRL 0x4#define APPL_CTRL_SYS_PRE_DET_STATE BIT(6)
@@ -1415,6 +1416,7 @@ static int tegra_pcie_config_controller(struct tegra_pcie_dw *pcie, val =appl_readl (pcie,APPL_PINMUX ); val |=APPL_PINMUX_CLKREQ_OVERRIDE_EN ; val &= ~APPL_PINMUX_CLKREQ_OVERRIDE ; + val &= ~APPL_PINMUX_CLKREQ_DEFAULT_VALUE ;appl_writel (pcie, val,APPL_PINMUX ); }
Signature · 1 line
2.34.1
PATCH v5 2/9 PCI: tegra194: Calibrate P2U for endpoint mode
In reply to Manikanta MaddireddyFrom: Vidya Sagar <[email protected]>
Calibrate P2U for endpoint controller to request UPHY PLL rate change to Gen1 during initialization. This helps to reset stale PLL state from the previous bad link state.
---
V5:
* None
V4:
* None
V3:
* None
V2:
* None
drivers/pci/controller/dwc/pcie-tegra194.c | 3 +++ 1 file changed, 3 insertions(+)
drivers/pci/controller/dwc/pcie-tegra194.c +3 -0
@@ -1058,6 +1058,9 @@ static int tegra_pcie_enable_phy(struct tegra_pcie_dw *pcie) ret =phy_power_on (pcie->phys[i]);if (ret <0 ) goto phy_exit; + +if (pcie->of_data->mode ==DW_PCIE_EP_TYPE ) +phy_calibrate (pcie->phys[i]); }return 0 ;
Signature · 1 line
2.34.1
PATCH v5 3/9 PCI: tegra194: Remove IRQF_ONESHOT flag during Endpoint interrupt registration
In reply to Manikanta MaddireddyFrom: Vidya Sagar <[email protected]>
The Tegra PCIe Endpoint controller has a single interrupt line that is shared between multiple interrupt sources:
1. PCIe link state events (link up, hot reset done)
2. Configuration space events (Bus Master Enable changes)
3. DMA completion events
Currently, the interrupt is registered with IRQF_ONESHOT, which keeps the interrupt line masked until the threaded handler completes. This prevents the DMA driver from sharing the same interrupt line, as the DMA completion interrupts would be blocked while the threaded handler processes link state events.
Removing IRQF_ONESHOT is safe for the following reasons:
1. The hard IRQ handler (tegra_pcie_ep_hard_irq) properly acknowledges and clears all interrupt status bits in hardware before returning. This prevents interrupt storms and ensures the interrupt controller can re-enable the interrupt line immediately.
2. The hard IRQ handler explicitly checks for DMA interrupts (APPL_INTR_STATUS_L1_8_0_EDMA_INT_MASK) and marks them as handled, allowing the DMA driver's handler to process them separately.
3. The threaded handler (tegra_pcie_ep_irq_thread) only processes link-up notifications and LTR message sending. These operations don't conflict with DMA interrupt processing and don't require the interrupt line to remain masked.
This change enables the DMA driver to share the interrupt line with the PCIe endpoint driver, allowing both drivers to process their respective events without blocking each other.
---
V5:
* None
V4:
* None
V3:
* None
V2:
* None
drivers/pci/controller/dwc/pcie-tegra194.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/pci/controller/dwc/pcie-tegra194.c +1 -1
@@ -2226,7 +2226,7 @@ static int tegra_pcie_dw_probe(struct platform_device *pdev) ret =devm_request_threaded_irq (dev, pp->irq, tegra_pcie_ep_hard_irq, tegra_pcie_ep_irq_thread, -IRQF_SHARED |IRQF_ONESHOT , +IRQF_SHARED ,"tegra-pcie-ep-intr" , pcie);if (ret) { dev_err(dev, "Failed to request IRQ %d: %d\n", pp->irq,
Signature · 1 line
2.34.1
PATCH v5 4/9 PCI: tegra194: Enable DMA interrupt
In reply to Manikanta MaddireddyFrom: Vidya Sagar <[email protected]>
Enable DMA interrupt to support Tegra PCIe DMA in both Root port and Endpoint modes.
---
V5:
* None
V4:
* None
V3:
* None
V2:
* None
drivers/pci/controller/dwc/pcie-tegra194.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
drivers/pci/controller/dwc/pcie-tegra194.c +14 -0
@@ -91,6 +91,7 @@#define APPL_INTR_EN_L1_8_0 0x44#define APPL_INTR_EN_L1_8_BW_MGT_INT_EN BIT(2)#define APPL_INTR_EN_L1_8_AUTO_BW_INT_EN BIT(3) +#define APPL_INTR_EN_L1_8_EDMA_INT_EN BIT(6)#define APPL_INTR_EN_L1_8_INTX_EN BIT(11)#define APPL_INTR_EN_L1_8_AER_INT_EN BIT(15)
@@ -547,6 +548,13 @@ static irqreturn_t tegra_pcie_ep_hard_irq(int irq, void *arg) spurious =0 ; } +if (status_l0 &APPL_INTR_STATUS_L0_INT_INT ) { + status_l1 =appl_readl (pcie,APPL_INTR_STATUS_L1_8_0 ); +/* Interrupt is handled by dma driver, don't treat it as spurious */ +if (status_l1 &APPL_INTR_STATUS_L1_8_0_EDMA_INT_MASK ) + spurious =0 ; + } +if (spurious) {dev_warn (pcie->dev,"Random interrupt (STATUS = 0x%08X)\n" , status_l0);
@@ -766,6 +774,7 @@ static void tegra_pcie_enable_intx_interrupts(struct dw_pcie_rp *pp) val |=APPL_INTR_EN_L1_8_INTX_EN ; val |=APPL_INTR_EN_L1_8_AUTO_BW_INT_EN ; val |=APPL_INTR_EN_L1_8_BW_MGT_INT_EN ; + val |=APPL_INTR_EN_L1_8_EDMA_INT_EN ;if (IS_ENABLED (CONFIG_PCIEAER )) val |=APPL_INTR_EN_L1_8_AER_INT_EN ;appl_writel (pcie, val,APPL_INTR_EN_L1_8_0 );
@@ -1789,6 +1798,7 @@ static void pex_ep_event_pex_rst_deassert(struct tegra_pcie_dw *pcie) val |=APPL_INTR_EN_L0_0_SYS_INTR_EN ; val |=APPL_INTR_EN_L0_0_LINK_STATE_INT_EN ; val |=APPL_INTR_EN_L0_0_PCI_CMD_EN_INT_EN ; + val |=APPL_INTR_EN_L0_0_INT_INT_EN ;appl_writel (pcie, val,APPL_INTR_EN_L0_0 ); val =appl_readl (pcie,APPL_INTR_EN_L1_0_0 );
@@ -1796,6 +1806,10 @@ static void pex_ep_event_pex_rst_deassert(struct tegra_pcie_dw *pcie) val |=APPL_INTR_EN_L1_0_0_RDLH_LINK_UP_INT_EN ;appl_writel (pcie, val,APPL_INTR_EN_L1_0_0 ); + val =appl_readl (pcie,APPL_INTR_EN_L1_8_0 ); + val |=APPL_INTR_EN_L1_8_EDMA_INT_EN ; +appl_writel (pcie, val,APPL_INTR_EN_L1_8_0 ); +/* 110us for both snoop and no-snoop */ val =110 | (2 <<PCI_LTR_SCALE_SHIFT ) |LTR_MSG_REQ ; val |= (val <<LTR_MST_NO_SNOOP_SHIFT );
Signature · 1 line
2.34.1
PATCH v5 5/9 PCI: tegra194: Enable hardware hot reset mode in Endpoint
In reply to Manikanta MaddireddyFrom: Vidya Sagar <[email protected]>
When PCIe link goes down, hardware can retrain the link and try to link up. To enable this feature, program the APPL_CTRL register with hardware hot reset with immediate LTSSM enable mode.
---
V5:
* None
V4:
* None
V3:
* None
V2:
* None
drivers/pci/controller/dwc/pcie-tegra194.c | 2 ++ 1 file changed, 2 insertions(+)
drivers/pci/controller/dwc/pcie-tegra194.c +2 -0
@@ -1775,6 +1775,8 @@ static void pex_ep_event_pex_rst_deassert(struct tegra_pcie_dw *pcie) val =appl_readl (pcie,APPL_CTRL ); val |=APPL_CTRL_SYS_PRE_DET_STATE ; val |=APPL_CTRL_HW_HOT_RST_EN ; + val &= ~(APPL_CTRL_HW_HOT_RST_MODE_MASK <<APPL_CTRL_HW_HOT_RST_MODE_SHIFT ); + val |= (APPL_CTRL_HW_HOT_RST_MODE_IMDT_RST_LTSSM_EN <<APPL_CTRL_HW_HOT_RST_MODE_SHIFT );appl_writel (pcie, val,APPL_CTRL ); val =appl_readl (pcie,APPL_CFG_MISC );
Signature · 1 line
2.34.1
PATCH v5 6/9 PCI: tegra194: Disable L1.2 capability of Tegra234 EP
In reply to Manikanta MaddireddyFrom: Vidya Sagar <[email protected]>
When Tegra234 is operating in the endpoint mode with L1.2 enabled, PCIe link goes down during L1.2 exit. This is because Tegra234 is powering up UPHY PLL immediately without making sure that the REFCLK is stable. This is causing UPHY PLL to not lock to the correct frequency and leading to link going down. There is no hardware fix for this, hence do not advertise the L1.2 capability in the endpoint mode.
---
V5:
* None
V4:
* None
V3:
* None
V2:
* None
drivers/pci/controller/dwc/pcie-tegra194.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+)
drivers/pci/controller/dwc/pcie-tegra194.c +18 -0
@@ -240,6 +240,7 @@ struct tegra_pcie_dw_of_data {bool has_sbr_reset_fix;bool has_l1ss_exit_fix;bool has_ltr_req_fix; +bool disable_l1_2;u32 cdm_chk_int_en_bit;u32 gen4_preset_vec;u8 n_fts[2 ];
@@ -692,6 +693,22 @@ static void init_host_aspm(struct tegra_pcie_dw *pcie)if (pcie->supports_clkreq) pci->l1ss_support = true; +/* +* Disable L1.2 capability advertisement for Tegra234 Endpoint mode. +* Tegra234 has a hardware bug where during L1.2 exit, the UPHY PLL is +* powered up immediately without waiting for REFCLK to stabilize. This +* causes the PLL to fail to lock to the correct frequency, resulting in +* PCIe link loss. Since there is no hardware fix available, we prevent +* the Endpoint from advertising L1.2 support by clearing the L1.2 bits +* in the L1 PM Substates Capabilities register. This ensures the host +* will not attempt to enter L1.2 state with this Endpoint. +*/ +if (pcie->of_data->disable_l1_2 && pcie->of_data->mode ==DW_PCIE_EP_TYPE ) { + val =dw_pcie_readl_dbi (pci, l1ss +PCI_L1SS_CAP ); + val &= ~(PCI_L1SS_CAP_PCIPM_L1_2 |PCI_L1SS_CAP_ASPM_L1_2 ); +dw_pcie_writel_dbi (pci, l1ss +PCI_L1SS_CAP , val); + } +/* Program L0s and L1 entrance latencies */ val =dw_pcie_readl_dbi (pci,PCIE_PORT_AFR ); val &= ~PORT_AFR_L0S_ENTRANCE_LAT_MASK ;
@@ -2464,6 +2481,7 @@ static const struct tegra_pcie_dw_of_data tegra234_pcie_dw_ep_of_data = { .mode = DW_PCIE_EP_TYPE, .has_l1ss_exit_fix = true, .has_ltr_req_fix = true, + .disable_l1_2 = true, .cdm_chk_int_en_bit = BIT(18),/* Gen4 - 6, 8 and 9 presets enabled */ .gen4_preset_vec = 0x340,
Signature · 1 line
2.34.1
PATCH v5 7/9 dt-bindings: PCI: tegra194: Add monitor clock support
In reply to Manikanta MaddireddyFrom: Vidya Sagar <[email protected]>
Tegra supports PCIe core clock monitoring for any rate changes that may be happening because of the link speed changes. This is useful in tracking any changes in the core clock that are not initiated by the software.
---
V5:
* Fixed review comments
V4:
* None
V3:
* None
V2:
* None
.../devicetree/bindings/pci/nvidia,tegra194-pcie-ep.yaml | 6 +++++- .../devicetree/bindings/pci/nvidia,tegra194-pcie.yaml | 6 +++++- 2 files changed, 10 insertions(+), 2 deletions(-)
Documentation/devicetree/bindings/pci/nvidia,tegra194-pcie-ep.yaml +5 -1
@@ -55,12 +55,16 @@ properties: - const: intr clocks: + minItems: 1 items: - - description: module clock + - description: core clock + - description: monitor clock clock-names: + minItems: 1 items: - const: core + - const: core_m resets: items:
Documentation/devicetree/bindings/pci/nvidia,tegra194-pcie.yaml +5 -1
@@ -58,12 +58,16 @@ properties: - const: msi clocks: + minItems: 1 items: - - description: module clock + - description: core clock + - description: monitor clock clock-names: + minItems: 1 items: - const: core + - const: core_m resets: items:
Signature · 1 line
2.34.1
PATCH v5 8/9 PCI: tegra194: Add core monitor clock support
In reply to Manikanta MaddireddyFrom: Vidya Sagar <[email protected]>
Tegra supports PCIe core clock monitoring for any rate changes that may be happening because of the link speed changes. This is useful in tracking any changes in the core clock that are not initiated by the software. This patch adds support to parse the monitor clock info from device-tree and enable it if present.
---
V5:
* None
V4:
* None
V3:
* None
V2:
* None
drivers/pci/controller/dwc/pcie-tegra194.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+)
drivers/pci/controller/dwc/pcie-tegra194.c +18 -0
@@ -253,6 +253,7 @@ struct tegra_pcie_dw {struct resource *atu_dma_res;void __iomem *appl_base;struct clk *core_clk; +struct clk *core_clk_m;struct reset_control *core_apb_rst;struct reset_control *core_rst;struct dw_pcie pci;
@@ -949,6 +950,8 @@ static int tegra_pcie_dw_host_init(struct dw_pcie_rp *pp) }clk_set_rate (pcie->core_clk,GEN4_CORE_CLK_FREQ ); +if (clk_prepare_enable (pcie->core_clk_m)) +dev_err (pci->dev,"Failed to enable core monitor clock\n" );return 0 ; }
@@ -1021,6 +1024,12 @@ static int tegra_pcie_dw_start_link(struct dw_pcie *pci) val &= ~PCI_DLF_EXCHANGE_ENABLE ;dw_pcie_writel_dbi (pci, offset +PCI_DLF_CAP , val); +/* +* core_clk_m is enabled as part of host_init callback in +* dw_pcie_host_init(). Disable the clock since below +* tegra_pcie_dw_host_init() will enable it again. +*/ +clk_disable_unprepare (pcie->core_clk_m);tegra_pcie_dw_host_init (pp);dw_pcie_setup_rc (pp);
@@ -1613,6 +1622,7 @@ static void tegra_pcie_dw_pme_turnoff(struct tegra_pcie_dw *pcie)static void tegra_pcie_deinit_controller (struct tegra_pcie_dw *pcie) { +clk_disable_unprepare (pcie->core_clk_m);dw_pcie_host_deinit (&pcie->pci.pp);tegra_pcie_dw_pme_turnoff (pcie);tegra_pcie_unconfig_controller (pcie);
@@ -2160,6 +2170,13 @@ static int tegra_pcie_dw_probe(struct platform_device *pdev)return PTR_ERR (pcie->core_clk); } + pcie->core_clk_m =devm_clk_get_optional (dev,"core_m" ); +if (IS_ERR (pcie->core_clk_m)) { +dev_err (dev,"Failed to get monitor clock: %ld\n" , +PTR_ERR (pcie->core_clk_m)); +return PTR_ERR (pcie->core_clk_m); + } + pcie->appl_res =platform_get_resource_byname (pdev,IORESOURCE_MEM ,"appl" );if (!pcie->appl_res) {
@@ -2356,6 +2373,7 @@ static int tegra_pcie_dw_suspend_noirq(struct device *dev)if (!pcie->link_state)return 0 ; +clk_disable_unprepare (pcie->core_clk_m);tegra_pcie_dw_pme_turnoff (pcie);tegra_pcie_unconfig_controller (pcie);
Signature · 1 line
2.34.1
PATCH v5 9/9 PCI: tegra194: Add ASPM L1 entrance latency config
In reply to Manikanta MaddireddyFor Tegra234, the HW PHY team conducted experiments and determined the optimal ASPM L1 entrance latency values: 8 us for Root Port mode and 16 us for Endpoint mode. Update the default ASPM L1 entrance latency configuration accordingly.
---
V5:
* None
V4:
* None
V3:
* None
V2:
* None
drivers/pci/controller/dwc/pcie-tegra194.c | 8 ++++++++ 1 file changed, 8 insertions(+)
drivers/pci/controller/dwc/pcie-tegra194.c +8 -0
@@ -244,6 +244,8 @@ struct tegra_pcie_dw_of_data {u32 cdm_chk_int_en_bit;u32 gen4_preset_vec;u8 n_fts[2 ]; +/* L1 Latency entrance values(Rest/Prod) */ +u32 aspm_l1_enter_lat; };struct tegra_pcie_dw {
@@ -714,6 +716,8 @@ static void init_host_aspm(struct tegra_pcie_dw *pcie) val =dw_pcie_readl_dbi (pci,PCIE_PORT_AFR ); val &= ~PORT_AFR_L0S_ENTRANCE_LAT_MASK ; val |= (pcie->aspm_l0s_enter_lat <<PORT_AFR_L0S_ENTRANCE_LAT_SHIFT ); + val &= ~PORT_AFR_L1_ENTRANCE_LAT_MASK ; + val |= (pcie->of_data->aspm_l1_enter_lat <<PORT_AFR_L1_ENTRANCE_LAT_SHIFT ); val |=PORT_AFR_ENTER_ASPM ;dw_pcie_writel_dbi (pci,PCIE_PORT_AFR , val); }
@@ -2471,6 +2475,7 @@ static const struct tegra_pcie_dw_of_data tegra194_pcie_dw_rc_of_data = {/* Gen4 - 5, 6, 8 and 9 presets enabled */ .gen4_preset_vec = 0x360, .n_fts = { 52, 52 }, + .aspm_l1_enter_lat = 3, }; static const struct tegra_pcie_dw_of_data tegra194_pcie_dw_ep_of_data = {
@@ -2480,6 +2485,7 @@ static const struct tegra_pcie_dw_of_data tegra194_pcie_dw_ep_of_data = {/* Gen4 - 5, 6, 8 and 9 presets enabled */ .gen4_preset_vec = 0x360, .n_fts = { 52, 52 }, + .aspm_l1_enter_lat = 3, }; static const struct tegra_pcie_dw_of_data tegra234_pcie_dw_rc_of_data = {
@@ -2492,6 +2498,7 @@ static const struct tegra_pcie_dw_of_data tegra234_pcie_dw_rc_of_data = {/* Gen4 - 6, 8 and 9 presets enabled */ .gen4_preset_vec = 0x340, .n_fts = { 52, 80 }, + .aspm_l1_enter_lat = 4, }; static const struct tegra_pcie_dw_of_data tegra234_pcie_dw_ep_of_data = {
@@ -2504,6 +2511,7 @@ static const struct tegra_pcie_dw_of_data tegra234_pcie_dw_ep_of_data = {/* Gen4 - 6, 8 and 9 presets enabled */ .gen4_preset_vec = 0x340, .n_fts = { 52, 80 }, + .aspm_l1_enter_lat = 5, }; static const struct of_device_id tegra_pcie_dw_of_match[] = {
Signature · 1 line
2.34.1