PCI: dw-rockchip: Disable L1 substates
PATCH v2 1/1 PCI: dw-rockchip: Disable L1 substates
The L1 substates support requires additional steps to work, see e.g. section '11.6.6.4 L1 Substate' in the RK3588 TRM V1.0.
These steps are currently missing from the driver.
While this has always been a problem when using e.g. CONFIG_PCIEASPM_POWER_SUPERSAVE=y, the problem became more apparent after commit f3ac2ff14834 ("PCI/ASPM: Enable all ClockPM and ASPM states for devicetree platforms"), which enabled ASPM also for CONFIG_PCIEASPM_DEFAULT=y.
Disable L1 substates until proper support is added.
---
Changes since v1:
-Remove superfluous dw_pcie_readl_dbi()
drivers/pci/controller/dwc/pcie-dw-rockchip.c | 21 +++++++++++++++++++ 1 file changed, 21 insertions(+)
drivers/pci/controller/dwc/pcie-dw-rockchip.c +21 -0
@@ -200,6 +200,25 @@ static bool rockchip_pcie_link_up(struct dw_pcie *pci)return FIELD_GET (PCIE_LINKUP_MASK , val) ==PCIE_LINKUP ; } +/* +* See e.g. section '11.6.6.4 L1 Substate' in the RK3588 TRM V1.0 for the steps +* needed to support L1 substates. Currently, not a single rockchip platform +* performs these steps, so disable L1 substates until there is proper support. +*/ +static void rockchip_pcie_disable_l1sub (struct dw_pcie *pci) +{ +u32 cap, l1subcap; + + cap =dw_pcie_find_ext_capability (pci,PCI_EXT_CAP_ID_L1SS ); +if (cap) { + l1subcap =dw_pcie_readl_dbi (pci, cap +PCI_L1SS_CAP ); + l1subcap &= ~(PCI_L1SS_CAP_L1_PM_SS |PCI_L1SS_CAP_ASPM_L1_1 | +PCI_L1SS_CAP_ASPM_L1_2 |PCI_L1SS_CAP_PCIPM_L1_1 | +PCI_L1SS_CAP_PCIPM_L1_2 ); +dw_pcie_writel_dbi (pci, cap +PCI_L1SS_CAP , l1subcap); + } +} +static void rockchip_pcie_enable_l0s (struct dw_pcie *pci) {u32 cap, lnkcap;
@@ -264,6 +283,7 @@ static int rockchip_pcie_host_init(struct dw_pcie_rp *pp)irq_set_chained_handler_and_data (irq, rockchip_pcie_intx_handler, rockchip); +rockchip_pcie_disable_l1sub (pci);rockchip_pcie_enable_l0s (pci);return 0 ;
@@ -301,6 +321,7 @@ static void rockchip_pcie_ep_init(struct dw_pcie_ep *ep)struct dw_pcie *pci =to_dw_pcie_from_ep (ep);enum pci_barno bar; +rockchip_pcie_disable_l1sub (pci);rockchip_pcie_enable_l0s (pci);rockchip_pcie_ep_hide_broken_ats_cap_rk3588 (ep);
Signature · 1 line
2.51.0
Quoted 4 lines · Niklas Cassel
The L1 substates support requires additional steps to work, see e.g. section '11.6.6.4 L1 Substate' in the RK3588 TRM V1.0.
These steps are currently missing from the driver.
Yes, we could add them later if concerns about supports-clkreq is fully discussed.
Quoted 9 lines
While this has always been a problem when using e.g.
CONFIG_PCIEASPM_POWER_SUPERSAVE=y, the problem became more apparent after commitf3ac2ff14834("PCI/ASPM: Enable all ClockPM and ASPM states for devicetree platforms"), which enabled ASPM also forCONFIG_PCIEASPM_DEFAULT=y.Disable L1 substates until proper support is added.
Thanks for the patch.
Quoted 57 lines
Fixes 0e898eb8df4e ("PCI: rockchip-dwc: Add Rockchip RK356X host controller driver")Fixes f3ac2ff14834 ("PCI/ASPM: Enable all ClockPM and ASPM states for devicetree platforms")Signed-off-by Niklas Cassel <[email protected]>---
Changes since v1:(quoted diff) +0
-1-Remove superfluous dw_pcie_readl_dbi() drivers/pci/controller/dwc/pcie-dw-rockchip.c | 21 +++++++++++++++++++ 1 file changed, 21 insertions(+)drivers/pci/controller/dwc/pcie-dw-rockchip.c +21
-0@@ -200,6 +200,25 @@ static bool rockchip_pcie_link_up(struct dw_pcie *pci)return FIELD_GET (PCIE_LINKUP_MASK , val) ==PCIE_LINKUP ; } +/* +* See e.g. section '11.6.6.4 L1 Substate' in the RK3588 TRM V1.0 for the steps +* needed to support L1 substates. Currently, not a single rockchip platform +* performs these steps, so disable L1 substates until there is proper support. +*/ +static void rockchip_pcie_disable_l1sub (struct dw_pcie *pci) +{ +u32 cap, l1subcap; + + cap =dw_pcie_find_ext_capability (pci,PCI_EXT_CAP_ID_L1SS ); +if (cap) { + l1subcap =dw_pcie_readl_dbi (pci, cap +PCI_L1SS_CAP ); + l1subcap &= ~(PCI_L1SS_CAP_L1_PM_SS |PCI_L1SS_CAP_ASPM_L1_1 | +PCI_L1SS_CAP_ASPM_L1_2 |PCI_L1SS_CAP_PCIPM_L1_1 | +PCI_L1SS_CAP_PCIPM_L1_2 ); +dw_pcie_writel_dbi (pci, cap +PCI_L1SS_CAP , l1subcap); + } +} +static void rockchip_pcie_enable_l0s (struct dw_pcie *pci) {u32 cap, lnkcap;@@ -264,6 +283,7 @@ static int rockchip_pcie_host_init(struct dw_pcie_rp *pp)irq_set_chained_handler_and_data (irq, rockchip_pcie_intx_handler, rockchip); +rockchip_pcie_disable_l1sub (pci);rockchip_pcie_enable_l0s (pci);return 0 ;@@ -301,6 +321,7 @@ static void rockchip_pcie_ep_init(struct dw_pcie_ep *ep)struct dw_pcie *pci =to_dw_pcie_from_ep (ep);enum pci_barno bar; +rockchip_pcie_disable_l1sub (pci);rockchip_pcie_enable_l0s (pci);rockchip_pcie_ep_hide_broken_ats_cap_rk3588 (ep);
Quoted 4 lines · Niklas Cassel
The L1 substates support requires additional steps to work, see e.g. section '11.6.6.4 L1 Substate' in the RK3588 TRM V1.0.
These steps are currently missing from the driver.
Can we outline here specifically what is missing?
Quoted 5 lines
While this has always been a problem when using e.g.
CONFIG_PCIEASPM_POWER_SUPERSAVE=y, the problem became more apparent after commitf3ac2ff14834("PCI/ASPM: Enable all ClockPM and ASPM states for devicetree platforms"), which enabled ASPM also forCONFIG_PCIEASPM_DEFAULT=y.
Should also be able to trigger this problem regardless of CONFIG_PCIEASPM_* by using /sys/bus/pci/devices/.../link/l1_2_aspm.
Quoted 1 line
Disable L1 substates until proper support is added.
I would word this more like "prevent advertising L1 Substates support" since we're not actually *disabling* anything here.
If the RK3588 TRM is publicly available, a URL here would be helpful.
Quoted 24 lines
Fixes 0e898eb8df4e ("PCI: rockchip-dwc: Add Rockchip RK356X host controller driver")Fixes f3ac2ff14834 ("PCI/ASPM: Enable all ClockPM and ASPM states for devicetree platforms")Signed-off-by Niklas Cassel <[email protected]>---
Changes since v1:(quoted diff) +0
-1-Remove superfluous dw_pcie_readl_dbi() drivers/pci/controller/dwc/pcie-dw-rockchip.c | 21 +++++++++++++++++++ 1 file changed, 21 insertions(+)drivers/pci/controller/dwc/pcie-dw-rockchip.c +5
-0@@ -200,6 +200,25 @@ static bool rockchip_pcie_link_up(struct dw_pcie *pci)return FIELD_GET (PCIE_LINKUP_MASK , val) ==PCIE_LINKUP ; } +/* +* See e.g. section '11.6.6.4 L1 Substate' in the RK3588 TRM V1.0 for the steps +* needed to support L1 substates. Currently, not a single rockchip platform +* performs these steps, so disable L1 substates until there is proper support. +*/
drivers/pci/controller/dwc/pcie-dw-rockchip.c static bool rockchip_pcie_link_up(struct dw_pcie *pci) · line 217
Quoted 10 lines
drivers/pci/controller/dwc/pcie-dw-rockchip.c +10
-0+static void rockchip_pcie_disable_l1sub (struct dw_pcie *pci) +{ +u32 cap, l1subcap; + + cap =dw_pcie_find_ext_capability (pci,PCI_EXT_CAP_ID_L1SS ); +if (cap) { + l1subcap =dw_pcie_readl_dbi (pci, cap +PCI_L1SS_CAP ); + l1subcap &= ~(PCI_L1SS_CAP_L1_PM_SS |PCI_L1SS_CAP_ASPM_L1_1 | +PCI_L1SS_CAP_ASPM_L1_2 |PCI_L1SS_CAP_PCIPM_L1_1 | +PCI_L1SS_CAP_PCIPM_L1_2 );
I suspect this problem is specifically related to L1.2 and CLKREQ#, and L1.1 might work fine. If so, can we update this so we still advertise L1.1 support?
Quoted 26 lines
drivers/pci/controller/dwc/pcie-dw-rockchip.c +6
-0+dw_pcie_writel_dbi (pci, cap +PCI_L1SS_CAP , l1subcap); + } +} +static void rockchip_pcie_enable_l0s (struct dw_pcie *pci) {u32 cap, lnkcap;@@ -264,6 +283,7 @@ static int rockchip_pcie_host_init(struct dw_pcie_rp *pp)irq_set_chained_handler_and_data (irq, rockchip_pcie_intx_handler, rockchip); +rockchip_pcie_disable_l1sub (pci);rockchip_pcie_enable_l0s (pci);return 0 ;@@ -301,6 +321,7 @@ static void rockchip_pcie_ep_init(struct dw_pcie_ep *ep)struct dw_pcie *pci =to_dw_pcie_from_ep (ep);enum pci_barno bar; +rockchip_pcie_disable_l1sub (pci);rockchip_pcie_enable_l0s (pci);rockchip_pcie_ep_hide_broken_ats_cap_rk3588 (ep);Signature · 1 line
2.51.0
Quoted 4 lines · Bjorn Helgaas
Quoted 2 lines · Niklas Cassel
The L1 substates support requires additional steps to work, see e.g. section '11.6.6.4 L1 Substate' in the RK3588 TRM V1.0.
Quoted 14 lines
drivers/pci/controller/dwc/pcie-dw-rockchip.c static bool rockchip_pcie_link_up(struct dw_pcie *pci) · line 217
Quoted 10 lines
drivers/pci/controller/dwc/pcie-dw-rockchip.c +10
-0+static void rockchip_pcie_disable_l1sub (struct dw_pcie *pci) +{ +u32 cap, l1subcap; + + cap =dw_pcie_find_ext_capability (pci,PCI_EXT_CAP_ID_L1SS ); +if (cap) { + l1subcap =dw_pcie_readl_dbi (pci, cap +PCI_L1SS_CAP ); + l1subcap &= ~(PCI_L1SS_CAP_L1_PM_SS |PCI_L1SS_CAP_ASPM_L1_1 | +PCI_L1SS_CAP_ASPM_L1_2 |PCI_L1SS_CAP_PCIPM_L1_1 | +PCI_L1SS_CAP_PCIPM_L1_2 );I suspect this problem is specifically related to L1.2 and CLKREQ#, and L1.1 might work fine. If so, can we update this so we still advertise L1.1 support?
Hmm, looking again at the spec (PCIe r7.0, sec 5.5), it looks like CLKREQ# is required for both L1.1 and L1.2 (but not L1.0, the basic L1), so ignore my comment here.
Quoted 7 lines · Bjorn Helgaas
Quoted 4 lines · Niklas Cassel
The L1 substates support requires additional steps to work, see e.g. section '11.6.6.4 L1 Substate' in the RK3588 TRM V1.0.
These steps are currently missing from the driver.
Can we outline here specifically what is missing?
Sure.
Quoted 7 lines
Quoted 6 lines
While this has always been a problem when using e.g.
CONFIG_PCIEASPM_POWER_SUPERSAVE=y, the problem became more apparent after commitf3ac2ff14834("PCI/ASPM: Enable all ClockPM and ASPM states for devicetree platforms"), which enabled ASPM also forCONFIG_PCIEASPM_DEFAULT=y.
Quoted 7 lines
Should also be able to trigger this problem regardless of CONFIG_PCIEASPM_* by using
/sys/bus/pci/devices/.../link/l1_2_aspm.Quoted 1 line
Disable L1 substates until proper support is added.
I would word this more like "prevent advertising L1 Substates support" since we're not actually *disabling* anything here.
Sure.
Quoted 2 lines
If the RK3588 TRM is publicly available, a URL here would be helpful.
I couldn't find it on any official rockchip or radxa URL.
Will submit a v3 that fixes your comments.
Kind regards,
Niklas