arm64: dts: qcom: sc8280xp: PCIe fixes and GICv3 ITS enable
Subsystems (7)
v2 cover arm64: dts: qcom: sc8280xp: PCIe fixes and GICv3 ITS enable
This series addresses a few problems with the sc8280xp PCIe implementation.
The DWC PCIe controller can either use its internal MSI controller or an external one such as the GICv3 ITS. Enabling the latter allows for assigning affinity to individual interrupts, but results in a large amount of Correctable Errors being logged on both the Lenovo ThinkPad X13s and the sc8280xp-crd reference design.
It turns out that these errors are always generated, but for some yet to be determined reason, the AER interrupts are never received when using the internal MSI controller, which makes the link errors harder to notice.
On the X13s, there is a large number of errors generated when bringing up the link on boot. This is related to the fact that UEFI firmware has already enabled the Wi-Fi PCIe link at Gen2 speed and restarting the link at Gen3 generates a massive amount of errors until the Wi-Fi firmware is restarted. This has now also been shown to cause the Wi-Fi to sometimes not start at all on boot for some users.
A recent commit enabling ASPM on certain Qualcomm platforms introduced further errors when using the Wi-Fi on the X13s as well as when accessing the NVMe on the CRD. The exact reason for this has not yet been identified, but disabling ASPM L0s makes the errors go away. This could suggest that either the current ASPM implementation is incomplete or that L0s is not supported with these devices.
Note that the X13s and CRD use the same Wi-Fi controller, but the errors are only generated on the X13s. The NVMe controller on my X13s does not support L0s so there are no issues there, unlike on the CRD which uses a different controller. The modem on the CRD does not generate any errors, but both the NVMe and modem keeps bouncing in and out of L0s/L1 also when not used, which could indicate that there are bigger problems with the ASPM implementation. I don't have a modem on my X13s so I have not been able to test whether L0s causes any trouble there.
Enabling AER error reporting on sc8280xp could similarly also reveal existing problems with the related sa8295p and sa8540p platforms as they share the base dtsi.
After discussing this with Bjorn Andersson at Qualcomm we have decided to go ahead and disable L0s for all controllers on the CRD and the X13s.
Note that disabling ASPM L0s for the X13s Wi-Fi does not seem to have a significant impact on the power consumption (and there are indications that this applies generally for L0s on these platforms).
***
As this series fixes a regression in 6.7 (which enabled ASPM) and fixes a user-reported problem with the Wi-Fi often not starting at boot, I think we should merge this series for the 6.8 cycle. The final patch enabling the GIC ITS should wait for 6.9.
The DT bindings and PCI patch are expected to go through the PCI tree, while Bjorn A takes the devicetree updates through the Qualcomm tree.
Note that the devicetree binding updates will conflict with the following series which was just merged for 6.9 today:
https://lore.kernel.org/lkml/[email protected]/
The resolution is simply to add 'required-opps' and 'aspm-no-l0s' to the new 'qcom,pcie-common.yaml' schema file instead of 'qcom,pcie.yaml', while removing 'msi-map-mask'.
***
Johan
Changes in v2
- drop RFC from ASPM patches and add stable tags
- reorder patches and move ITS patch last
- fix s/GB/MB/ typo in Gen2 speed commit messages
- fix an incorrect Fixes tag
- amend commit message X13 wifi link speed patch after user confirmation that this fixes the wifi startup issue
- disable L0s also for modem and wifi on CRD
- disable L0s also for nvme and modem on X13s
Johan Hovold (12):
dt-bindings: PCI: qcom: Allow 'required-opps'
dt-bindings: PCI: qcom: Do not require 'msi-map-mask'
dt-bindings: PCI: qcom: Allow 'aspm-no-l0s'
PCI: qcom: Add support for disabling ASPM L0s in devicetree
arm64: dts: qcom: sc8280xp: add missing PCIe minimum OPP
arm64: dts: qcom: sc8280xp-crd: limit pcie4 link speed
arm64: dts: qcom: sc8280xp-x13s: limit pcie4 link speed
arm64: dts: qcom: sc8280xp-crd: disable ASPM L0s for NVMe
arm64: dts: qcom: sc8280xp-crd: disable ASPM L0s for modem and Wi-Fi
arm64: dts: qcom: sc8280xp-x13s: disable ASPM L0s for Wi-Fi
arm64: dts: qcom: sc8280xp-x13s: disable ASPM L0s for NVMe and modem
arm64: dts: qcom: sc8280xp: enable GICv3 ITS for PCIe
.../devicetree/bindings/pci/qcom,pcie.yaml | 6 +++++- arch/arm64/boot/dts/qcom/sc8280xp-crd.dts | 7 +++++++ .../qcom/sc8280xp-lenovo-thinkpad-x13s.dts | 7 +++++++ arch/arm64/boot/dts/qcom/sc8280xp.dtsi | 17 +++++++++++++++- drivers/pci/controller/dwc/pcie-qcom.c | 20 +++++++++++++++++++ 5 files changed, 55 insertions(+), 2 deletions(-)
Signature · 1 line
2.43.0
PATCH v2 1/12 dt-bindings: PCI: qcom: Allow 'required-opps'
Some Qualcomm SoCs require a minimum performance level for the power domain so add 'required-opps' to the binding.
--- Documentation/devicetree/bindings/pci/qcom,pcie.yaml | 3 +++ 1 file changed, 3 insertions(+)
Documentation/devicetree/bindings/pci/qcom,pcie.yaml +3 -0
@@ -123,6 +123,9 @@ properties: description: GPIO controlled connection to PERST# signal maxItems: 1 + required-opps: + maxItems: 1 + wake-gpios: description: GPIO controlled connection to WAKE# signal maxItems: 1
Signature · 1 line
2.43.0
PATCH v2 2/12 dt-bindings: PCI: qcom: Do not require 'msi-map-mask'
In reply to Johan HovoldWhether the 'msi-map-mask' property is needed or not depends on how the MSI interrupts are mapped and it should therefore not be described as required.
Note that the current schema fails to detect omissions of the mask property if the internal MSI controller properties are also present.
--- Documentation/devicetree/bindings/pci/qcom,pcie.yaml | 1 - 1 file changed, 1 deletion(-)
Documentation/devicetree/bindings/pci/qcom,pcie.yaml +0 -1
@@ -146,7 +146,6 @@ anyOf: - "#interrupt-cells" - required: - msi-map - - msi-map-mask allOf: - $ref: /schemas/pci/pci-bus.yaml#
Signature · 1 line
2.43.0
PATCH v2 3/12 dt-bindings: PCI: qcom: Allow 'aspm-no-l0s'
In reply to Johan HovoldAdd 'aspm-no-l0s', which can be used to indicate that ASPM L0s is not supported, to the binding.
--- Documentation/devicetree/bindings/pci/qcom,pcie.yaml | 2 ++ 1 file changed, 2 insertions(+)
Documentation/devicetree/bindings/pci/qcom,pcie.yaml +2 -0
@@ -130,6 +130,8 @@ properties: description: GPIO controlled connection to WAKE# signal maxItems: 1 + aspm-no-l0s: true + required: - compatible - reg
Signature · 1 line
2.43.0
Re: [PATCH v2 03/12] dt-bindings: PCI: qcom: Allow 'aspm-no-l0s'
Quoted 8 lines · Johan Hovold
Add 'aspm-no-l0s', which can be used to indicate that ASPM L0s is not supported, to the binding.
Signed-off-by Johan Hovold <[email protected]>--- Documentation/devicetree/bindings/pci/qcom,pcie.yaml | 2 ++ 1 file changed, 2 insertions(+)
PATCH v2 4/12 PCI: qcom: Add support for disabling ASPM L0s in devicetree
In reply to Johan HovoldCommit 9f4f3dfad8cf ("PCI: qcom: Enable ASPM for platforms supporting 1.9.0 ops") started enabling ASPM unconditionally when the hardware claims to support it. This triggers Correctable Errors for some PCIe devices on machines like the Lenovo ThinkPad X13s, which could indicate an incomplete driver ASPM implementation or that the hardware does in fact not support L0s.
Add support for disabling ASPM L0s in the devicetree when it is not supported on a particular machine and controller.
Note that only the 1.9.0 ops enable ASPM currently.
--- drivers/pci/controller/dwc/pcie-qcom.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+)
drivers/pci/controller/dwc/pcie-qcom.c +20 -0
@@ -273,6 +273,25 @@ static int qcom_pcie_start_link(struct dw_pcie *pci)return 0 ; } +static void qcom_pcie_clear_aspm_l0s (struct dw_pcie *pci) +{ +u16 offset; +u32 val; + +if (!of_property_read_bool (pci->dev->of_node,"aspm-no-l0s" )) +return ; + + offset =dw_pcie_find_capability (pci,PCI_CAP_ID_EXP ); + +dw_pcie_dbi_ro_wr_en (pci); + + val =readl (pci->dbi_base + offset +PCI_EXP_LNKCAP ); + val &= ~PCI_EXP_LNKCAP_ASPM_L0S ; +writel (val, pci->dbi_base + offset +PCI_EXP_LNKCAP ); + +dw_pcie_dbi_ro_wr_dis (pci); +} +static void qcom_pcie_clear_hpc (struct dw_pcie *pci) {u16 offset =dw_pcie_find_capability (pci,PCI_CAP_ID_EXP );
@@ -962,6 +981,7 @@ static int qcom_pcie_init_2_7_0(struct qcom_pcie *pcie)static int qcom_pcie_post_init_2_7_0 (struct qcom_pcie *pcie) { +qcom_pcie_clear_aspm_l0s (pcie->pci);qcom_pcie_clear_hpc (pcie->pci);return 0 ;
Signature · 1 line
2.43.0
Re: [PATCH v2 04/12] PCI: qcom: Add support for disabling ASPM L0s in devicetree
Quoted 6 lines · Johan Hovold
Commit
9f4f3dfad8cf("PCI: qcom: Enable ASPM for platforms supporting 1.9.0 ops") started enabling ASPM unconditionally when the hardware claims to support it. This triggers Correctable Errors for some PCIe devices on machines like the Lenovo ThinkPad X13s, which could indicate an incomplete driver ASPM implementation or that the hardware does in fact not support L0s.
Are there any more details about this? Do the errors occur around suspend/resume, a power state transition, or some other event? Might other DWC-based devices be susceptible? Is there a specific driver you suspect might be incomplete?
Do you want the DT approach because the problem is believed to be platform-specific? Otherwise, maybe we should consider reverting 9f4f3dfad8cf until the problem is understood?
Could this be done via a quirk like quirk_disable_aspm_l0s()? That currently uses pci_disable_link_state(), which I don't think is completely safe because it leaves the possibility that drivers or users could re-enable L0s, e.g., via sysfs.
This patch is nice because IIUC it directly changes PCI_EXP_LNKCAP, which avoids that issue, but quirk_disable_aspm_l0s() could conceivably be reimplemented to cache PCI_EXP_LNKCAP in struct pci_dev so quirks could override it, as we do with struct pci_dev.devcap.
Quoted 53 lines
Add support for disabling ASPM L0s in the devicetree when it is not supported on a particular machine and controller.
Note that only the 1.9.0 ops enable ASPM currently.
Fixes 9f4f3dfad8cf ("PCI: qcom: Enable ASPM for platforms supporting 1.9.0 ops")Cc [email protected] # 6.7Signed-off-by Johan Hovold <[email protected]>--- drivers/pci/controller/dwc/pcie-qcom.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+)drivers/pci/controller/dwc/pcie-qcom.c +20
-0@@ -273,6 +273,25 @@ static int qcom_pcie_start_link(struct dw_pcie *pci)return 0 ; } +static void qcom_pcie_clear_aspm_l0s (struct dw_pcie *pci) +{ +u16 offset; +u32 val; + +if (!of_property_read_bool (pci->dev->of_node,"aspm-no-l0s" )) +return ; + + offset =dw_pcie_find_capability (pci,PCI_CAP_ID_EXP ); + +dw_pcie_dbi_ro_wr_en (pci); + + val =readl (pci->dbi_base + offset +PCI_EXP_LNKCAP ); + val &= ~PCI_EXP_LNKCAP_ASPM_L0S ; +writel (val, pci->dbi_base + offset +PCI_EXP_LNKCAP ); + +dw_pcie_dbi_ro_wr_dis (pci); +} +static void qcom_pcie_clear_hpc (struct dw_pcie *pci) {u16 offset =dw_pcie_find_capability (pci,PCI_CAP_ID_EXP );@@ -962,6 +981,7 @@ static int qcom_pcie_init_2_7_0(struct qcom_pcie *pcie)static int qcom_pcie_post_init_2_7_0 (struct qcom_pcie *pcie) { +qcom_pcie_clear_aspm_l0s (pcie->pci);qcom_pcie_clear_hpc (pcie->pci);return 0 ;Signature · 1 line
2.43.0
Re: [PATCH v2 04/12] PCI: qcom: Add support for disabling ASPM L0s in devicetree
Quoted 8 lines · Bjorn Helgaas
Quoted 6 lines · Johan Hovold
Commit
9f4f3dfad8cf("PCI: qcom: Enable ASPM for platforms supporting 1.9.0 ops") started enabling ASPM unconditionally when the hardware claims to support it. This triggers Correctable Errors for some PCIe devices on machines like the Lenovo ThinkPad X13s, which could indicate an incomplete driver ASPM implementation or that the hardware does in fact not support L0s.
Quoted 4 lines
Are there any more details about this? Do the errors occur around suspend/resume, a power state transition, or some other event? Might other DWC-based devices be susceptible? Is there a specific driver you suspect might be incomplete?
I see these errors when the devices in question are active as well as idle (not during suspend/resume). For example, when running iperf3 or fio to test the wifi and nvme, but I also see this occasionally for a wifi device which is (supposedly) not active (e.g. a handful errors over night).
I skimmed Qualcomm's driver and noted that there are some registers related to ASPM which that driver updates, while the mainline driver leaves them at their default settings, but I essentially only mentioned that the ASPM implementation may be incomplete as a theoretical possibility. The somewhat erratic ASPM behaviour for one of the modems also suggests that some further tweak/quirk may be needed, and I was hoping to catch Mani's interest by reporting it.
But based on what I've since heard from Qualcomm, it seems like these correctable error may be a known issue with the hardware (e.g. seen also with Windows), which is also why we decided to disable it for all controllers on these two platforms where I've seen this in v2.
Quoted 3 lines
Do you want the DT approach because the problem is believed to be platform-specific? Otherwise, maybe we should consider reverting
9f4f3dfad8cfuntil the problem is understood?
Enabling ASPM gave a very significant improvement in battery life on the Lenovo ThinkPad X13s, from 10.5 h to 15 h, so reverting is not really an option there.
And with L0s disabled, the AER error reports about correctable errors (that prevent enabling the GIC ITS and possibly degrades performance somewhat) are gone.
I don't know for sure if there are further Qualcomm platform that are affected by this so I also don't want to use a too big of a hammer. The devicetree property allows us to disable L0s only after confirming that it's needed, and we can always extend this to broader classes of device when/if we learn more.
Quoted 4 lines
Could this be done via a quirk like
quirk_disable_aspm_l0s()? That currently usespci_disable_link_state(), which I don't think is completely safe because it leaves the possibility that drivers or users could re-enable L0s, e.g., via sysfs.
That was my first approach, thinking that it was the endpoint devices which did not really support L0s. But initially it seemed like the wifi controller on the CRD was not affected by this, while the same controller on the X13s was. That made me conclude that this is not just a property of the device but (also) of the controller and/or machine.
I then noticed that we already had some controller drivers implementing 'aspm-no-l0s' and decided to go with that.
Quoted 4 lines
This patch is nice because IIUC it directly changes
PCI_EXP_LNKCAP, which avoids that issue, butquirk_disable_aspm_l0s()could conceivably be reimplemented to cachePCI_EXP_LNKCAPin structpci_devso quirks could override it, as we do with struct pci_dev.devcap.
Johan
Re: [PATCH v2 04/12] PCI: qcom: Add support for disabling ASPM L0s in devicetree
Quoted 33 lines · Johan Hovold
Quoted 8 lines · Bjorn Helgaas
Quoted 6 lines · Johan Hovold
Commit
9f4f3dfad8cf("PCI: qcom: Enable ASPM for platforms supporting 1.9.0 ops") started enabling ASPM unconditionally when the hardware claims to support it. This triggers Correctable Errors for some PCIe devices on machines like the Lenovo ThinkPad X13s, which could indicate an incomplete driver ASPM implementation or that the hardware does in fact not support L0s.Quoted 4 lines
Are there any more details about this? Do the errors occur around suspend/resume, a power state transition, or some other event? Might other DWC-based devices be susceptible? Is there a specific driver you suspect might be incomplete?
I see these errors when the devices in question are active as well as idle (not during suspend/resume). For example, when running iperf3 or fio to test the wifi and nvme, but I also see this occasionally for a wifi device which is (supposedly) not active (e.g. a handful errors over night).
I skimmed Qualcomm's driver and noted that there are some registers related to ASPM which that driver updates, while the mainline driver leaves them at their default settings, but I essentially only mentioned that the ASPM implementation may be incomplete as a theoretical possibility. The somewhat erratic ASPM behaviour for one of the modems also suggests that some further tweak/quirk may be needed, and I was hoping to catch Mani's interest by reporting it.
But based on what I've since heard from Qualcomm, it seems like these correctable error may be a known issue with the hardware (e.g. seen also with Windows), which is also why we decided to disable it for all controllers on these two platforms where I've seen this in v2.
Quoted 7 lines
Quoted 3 lines
Do you want the DT approach because the problem is believed to be platform-specific? Otherwise, maybe we should consider reverting
9f4f3dfad8cfuntil the problem is understood?Enabling ASPM gave a very significant improvement in battery life on the Lenovo ThinkPad X13s, from 10.5 h to 15 h, so reverting is not really an option there.
Ah, I missed that you're only disabling L0s, but leaving L1 enabled, thanks!
And given that the v1.9.0 ops that enable ASPM are used on a bunch of platforms, and L0s seems to work fine on most of them, we wouldn't want to disable L0s for everybody, so this seems like the right solution.
Bjorn
PATCH v2 5/12 arm64: dts: qcom: sc8280xp: add missing PCIe minimum OPP
In reply to Johan HovoldAdd the missing PCIe CX performance level votes to avoid relying on other drivers (e.g. USB or UFS) to maintain the nominal performance level required for Gen3 speeds.
--- arch/arm64/boot/dts/qcom/sc8280xp.dtsi | 5 +++++ 1 file changed, 5 insertions(+)
arch/arm64/boot/dts/qcom/sc8280xp.dtsi +5 -0
@@ -1780,6 +1780,7 @@ pcie4: pcie@1c00000 { reset-names = "pci"; power-domains = <&gcc PCIE_4_GDSC>; + required-opps = <&rpmhpd_opp_nom>; phys = <&pcie4_phy>; phy-names = "pciephy";
@@ -1878,6 +1879,7 @@ pcie3b: pcie@1c08000 { reset-names = "pci"; power-domains = <&gcc PCIE_3B_GDSC>; + required-opps = <&rpmhpd_opp_nom>; phys = <&pcie3b_phy>; phy-names = "pciephy";
@@ -1976,6 +1978,7 @@ pcie3a: pcie@1c10000 { reset-names = "pci"; power-domains = <&gcc PCIE_3A_GDSC>; + required-opps = <&rpmhpd_opp_nom>; phys = <&pcie3a_phy>; phy-names = "pciephy";
@@ -2077,6 +2080,7 @@ pcie2b: pcie@1c18000 { reset-names = "pci"; power-domains = <&gcc PCIE_2B_GDSC>; + required-opps = <&rpmhpd_opp_nom>; phys = <&pcie2b_phy>; phy-names = "pciephy";
@@ -2175,6 +2179,7 @@ pcie2a: pcie@1c20000 { reset-names = "pci"; power-domains = <&gcc PCIE_2A_GDSC>; + required-opps = <&rpmhpd_opp_nom>; phys = <&pcie2a_phy>; phy-names = "pciephy";
Signature · 1 line
2.43.0
Re: [PATCH v2 05/12] arm64: dts: qcom: sc8280xp: add missing PCIe minimum OPP
Quoted 8 lines · Johan Hovold
Add the missing PCIe CX performance level votes to avoid relying on other drivers (e.g. USB or UFS) to maintain the nominal performance level required for Gen3 speeds.
Fixes 813e83157001 ("arm64: dts: qcom: sc8280xp/sa8540p: add PCIe2-4 nodes")Cc [email protected] # 6.2Signed-off-by Johan Hovold <[email protected]>---
Please remember that we should remove these when OPP support lands :D
Konrad
PATCH v2 6/12 arm64: dts: qcom: sc8280xp-crd: limit pcie4 link speed
In reply to Johan HovoldLimit the WiFi PCIe link speed to Gen2 speed (500 MB/s), which is the speed that Windows uses.
--- arch/arm64/boot/dts/qcom/sc8280xp-crd.dts | 2 ++ 1 file changed, 2 insertions(+)
arch/arm64/boot/dts/qcom/sc8280xp-crd.dts +2 -0
@@ -563,6 +563,8 @@ &pcie3a_phy { }; &pcie4 { + max-link-speed = <2>; + perst-gpios = <&tlmm 141 GPIO_ACTIVE_LOW>; wake-gpios = <&tlmm 139 GPIO_ACTIVE_LOW>;
Signature · 1 line
2.43.0
Re: [PATCH v2 06/12] arm64: dts: qcom: sc8280xp-crd: limit pcie4 link speed
Quoted 5 lines · Johan Hovold
Limit the WiFi PCIe link speed to Gen2 speed (500 MB/s), which is the speed that Windows uses.
Signed-off-by Johan Hovold <[email protected]>---
Konrad
PATCH v2 7/12 arm64: dts: qcom: sc8280xp-x13s: limit pcie4 link speed
In reply to Johan HovoldLimit the WiFi PCIe link speed to Gen2 speed (500 MB/s), which is the speed that the boot firmware has brought up the link at (and that Windows uses).
This is specifically needed to avoid a large amount of link errors when restarting the link during boot (but which are currently not reported).
This also appears to fix intermittent failures to download the ath11k firmware during boot which can be seen when there is a longer delay between restarting the link and loading the WiFi driver (e.g. when using full disk encryption).
--- arch/arm64/boot/dts/qcom/sc8280xp-lenovo-thinkpad-x13s.dts | 2 ++ 1 file changed, 2 insertions(+)
arch/arm64/boot/dts/qcom/sc8280xp-lenovo-thinkpad-x13s.dts +2 -0
@@ -768,6 +768,8 @@ &pcie3a_phy { }; &pcie4 { + max-link-speed = <2>; + perst-gpios = <&tlmm 141 GPIO_ACTIVE_LOW>; wake-gpios = <&tlmm 139 GPIO_ACTIVE_LOW>;
Signature · 1 line
2.43.0
Re: [PATCH v2 07/12] arm64: dts: qcom: sc8280xp-x13s: limit pcie4 link speed
Quoted 12 lines · Johan Hovold
Limit the WiFi PCIe link speed to Gen2 speed (500 MB/s), which is the speed that the boot firmware has brought up the link at (and that Windows uses).
This is specifically needed to avoid a large amount of link errors when restarting the link during boot (but which are currently not reported).
This also appears to fix intermittent failures to download the ath11k firmware during boot which can be seen when there is a longer delay between restarting the link and loading the WiFi driver (e.g. when using full disk encryption).
Quoted 4 lines
Fixes 123b30a75623 ("arm64: dts: qcom: sc8280xp-x13s: enable WiFi controller")Cc [email protected] # 6.2Signed-off-by Johan Hovold <[email protected]>---
Konrad
PATCH v2 8/12 arm64: dts: qcom: sc8280xp-crd: disable ASPM L0s for NVMe
In reply to Johan HovoldEnabling ASPM L0s on the CRD results in a large amount of Correctable Errors (Timeout) when accessing the NVMe controller so disable it for now.
--- arch/arm64/boot/dts/qcom/sc8280xp-crd.dts | 2 ++ 1 file changed, 2 insertions(+)
arch/arm64/boot/dts/qcom/sc8280xp-crd.dts +2 -0
@@ -525,6 +525,8 @@ keyboard@68 { }; &pcie2a { + aspm-no-l0s; + perst-gpios = <&tlmm 143 GPIO_ACTIVE_LOW>; wake-gpios = <&tlmm 145 GPIO_ACTIVE_LOW>;
Signature · 1 line
2.43.0
Re: [PATCH v2 08/12] arm64: dts: qcom: sc8280xp-crd: disable ASPM L0s for NVMe
Quoted 8 lines · Johan Hovold
Enabling ASPM L0s on the CRD results in a large amount of Correctable Errors (Timeout) when accessing the NVMe controller so disable it for now.
Fixes 9f4f3dfad8cf ("PCI: qcom: Enable ASPM for platforms supporting 1.9.0 ops")Cc [email protected] # 6.7Signed-off-by Johan Hovold <[email protected]>---
Konrad
PATCH v2 9/12 arm64: dts: qcom: sc8280xp-crd: disable ASPM L0s for modem and Wi-Fi
In reply to Johan HovoldThere are indications that ASPM L0s is not working very well on this machine so disable it also for the modem and Wi-Fi controllers for now.
This specifically avoids having the modem and Wi-Fi controllers bounce in an out of L0s when not used (the modem still bounces in and out of L1) as well as intermittent Correctable errors on the Wi-Fi link when not used.
--- arch/arm64/boot/dts/qcom/sc8280xp-crd.dts | 3 +++ 1 file changed, 3 insertions(+)
arch/arm64/boot/dts/qcom/sc8280xp-crd.dts +3 -0
@@ -546,6 +546,8 @@ &pcie2a_phy { }; &pcie3a { + aspm-no-l0s; + perst-gpios = <&tlmm 151 GPIO_ACTIVE_LOW>; wake-gpios = <&tlmm 148 GPIO_ACTIVE_LOW>;
@@ -566,6 +568,7 @@ &pcie3a_phy { &pcie4 { max-link-speed = <2>; + aspm-no-l0s; perst-gpios = <&tlmm 141 GPIO_ACTIVE_LOW>; wake-gpios = <&tlmm 139 GPIO_ACTIVE_LOW>;
Signature · 1 line
2.43.0
PATCH v2 10/12 arm64: dts: qcom: sc8280xp-x13s: disable ASPM L0s for Wi-Fi
In reply to Johan HovoldEnabling ASPM L0s on the Lenovo Thinkpad X13s results in Correctable Errors (BadTLP, Timeout) when accessing the Wi-Fi controller so disable it for now.
--- arch/arm64/boot/dts/qcom/sc8280xp-lenovo-thinkpad-x13s.dts | 1 + 1 file changed, 1 insertion(+)
arch/arm64/boot/dts/qcom/sc8280xp-lenovo-thinkpad-x13s.dts +1 -0
@@ -769,6 +769,7 @@ &pcie3a_phy { &pcie4 { max-link-speed = <2>; + aspm-no-l0s; perst-gpios = <&tlmm 141 GPIO_ACTIVE_LOW>; wake-gpios = <&tlmm 139 GPIO_ACTIVE_LOW>;
Signature · 1 line
2.43.0
PATCH v2 11/12 arm64: dts: qcom: sc8280xp-x13s: disable ASPM L0s for NVMe and modem
In reply to Johan HovoldThere are indications that ASPM L0s is not working very well on this machine so disable it also for the NVMe and modem controllers for now.
Note that this is done as a precaution based on problems with the Wi-Fi on the X13s as well as the NVMe, modem and Wi-Fi on the sc8280xp-crd reference design (the NVMe controller on my X13s does not support L0 and the machine lacks a modem).
--- arch/arm64/boot/dts/qcom/sc8280xp-lenovo-thinkpad-x13s.dts | 4 ++++ 1 file changed, 4 insertions(+)
arch/arm64/boot/dts/qcom/sc8280xp-lenovo-thinkpad-x13s.dts +4 -0
@@ -730,6 +730,8 @@ keyboard@68 { }; &pcie2a { + aspm-no-l0s; + perst-gpios = <&tlmm 143 GPIO_ACTIVE_LOW>; wake-gpios = <&tlmm 145 GPIO_ACTIVE_LOW>;
@@ -749,6 +751,8 @@ &pcie2a_phy { }; &pcie3a { + aspm-no-l0s; + perst-gpios = <&tlmm 151 GPIO_ACTIVE_LOW>; wake-gpios = <&tlmm 148 GPIO_ACTIVE_LOW>;
Signature · 1 line
2.43.0
PATCH v2 12/12 arm64: dts: qcom: sc8280xp: enable GICv3 ITS for PCIe
In reply to Johan HovoldThe DWC PCIe controller can be used with its internal MSI controller or with an external one such as the GICv3 Interrupt Translation Service (ITS).
Add the msi-map properties needed to use the GIC ITS. This will also make Linux switch to the ITS implementation, which allows for assigning affinity to individual MSIs.
Note that using the GIC ITS on SC8280XP will cause Advanced Error Reporting (AER) interrupts to be received on errors unlike when using the internal MSI controller. This will specifically lead to notifications about Correctable Errors being logged for the Wi-Fi controller on the Lenovo ThinkPad X13s when ASPM L0s is enabled.
--- arch/arm64/boot/dts/qcom/sc8280xp.dtsi | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)
arch/arm64/boot/dts/qcom/sc8280xp.dtsi +11 -1
@@ -1737,6 +1737,8 @@ pcie4: pcie@1c00000 { linux,pci-domain = <6>; num-lanes = <1>; + msi-map = <0x0 &its 0xe0000 0x10000>; + interrupts = <GIC_SPI 141 IRQ_TYPE_LEVEL_HIGH>, <GIC_SPI 142 IRQ_TYPE_LEVEL_HIGH>, <GIC_SPI 143 IRQ_TYPE_LEVEL_HIGH>,
@@ -1838,6 +1840,8 @@ pcie3b: pcie@1c08000 { linux,pci-domain = <5>; num-lanes = <2>; + msi-map = <0x0 &its 0xd0000 0x10000>; + interrupts = <GIC_SPI 145 IRQ_TYPE_LEVEL_HIGH>, <GIC_SPI 146 IRQ_TYPE_LEVEL_HIGH>, <GIC_SPI 147 IRQ_TYPE_LEVEL_HIGH>,
@@ -1937,6 +1941,8 @@ pcie3a: pcie@1c10000 { linux,pci-domain = <4>; num-lanes = <4>; + msi-map = <0x0 &its 0xc0000 0x10000>; + interrupts = <GIC_SPI 312 IRQ_TYPE_LEVEL_HIGH>, <GIC_SPI 313 IRQ_TYPE_LEVEL_HIGH>, <GIC_SPI 314 IRQ_TYPE_LEVEL_HIGH>,
@@ -2039,6 +2045,8 @@ pcie2b: pcie@1c18000 { linux,pci-domain = <3>; num-lanes = <2>; + msi-map = <0x0 &its 0xb0000 0x10000>; + interrupts = <GIC_SPI 306 IRQ_TYPE_LEVEL_HIGH>, <GIC_SPI 307 IRQ_TYPE_LEVEL_HIGH>, <GIC_SPI 308 IRQ_TYPE_LEVEL_HIGH>,
@@ -2138,6 +2146,8 @@ pcie2a: pcie@1c20000 { linux,pci-domain = <2>; num-lanes = <4>; + msi-map = <0x0 &its 0xa0000 0x10000>; + interrupts = <GIC_SPI 86 IRQ_TYPE_LEVEL_HIGH>, <GIC_SPI 523 IRQ_TYPE_LEVEL_HIGH>, <GIC_SPI 524 IRQ_TYPE_LEVEL_HIGH>,
@@ -4426,7 +4436,7 @@ intc: interrupt-controller@17a00000 { #size-cells = <2>; ranges; - msi-controller@17a40000 { + its: msi-controller@17a40000 { compatible = "arm,gic-v3-its"; reg = <0 0x17a40000 0 0x20000>; msi-controller;
Signature · 1 line
2.43.0
[+to Mani]
Quoted 21 lines · Johan Hovold
This series addresses a few problems with the sc8280xp PCIe implementation.
...A recent commit enabling ASPM on certain Qualcomm platforms introduced further errors when using the Wi-Fi on the X13s as well as when accessing the NVMe on the CRD. The exact reason for this has not yet been identified, but disabling ASPM L0s makes the errors go away. This could suggest that either the current ASPM implementation is incomplete or that L0s is not supported with these devices.
...As this series fixes a regression in 6.7 (which enabled ASPM) and fixes a user-reported problem with the Wi-Fi often not starting at boot, I think we should merge this series for the 6.8 cycle. The final patch enabling the GIC ITS should wait for 6.9.
The DT bindings and PCI patch are expected to go through the PCI tree, while Bjorn A takes the devicetree updates through the Qualcomm tree.
...
Quoted 5 lines
Johan Hovold (12):
dt-bindings: PCI: qcom: Allow 'required-opps'
dt-bindings: PCI: qcom: Do not require 'msi-map-mask'
dt-bindings: PCI: qcom: Allow 'aspm-no-l0s'
PCI: qcom: Add support for disabling ASPM L0s in devicetree
The ASPM patches fix a v6.7 regression, so it would be good to fix that in v6.8.
Mani, if you are OK with them, I can add them to for-linus for v6.8.
What about the 'required-opps' and 'msi-map-mask' patches? If they're important, I can merge them for v6.8, too, but it's late in the cycle and it's not clear from the commit logs why they shouldn't wait for v6.9.
Bjorn
Quoted 33 lines · Bjorn Helgaas
[+to Mani]
Quoted 21 lines · Johan Hovold
This series addresses a few problems with the sc8280xp PCIe implementation.
...A recent commit enabling ASPM on certain Qualcomm platforms introduced further errors when using the Wi-Fi on the X13s as well as when accessing the NVMe on the CRD. The exact reason for this has not yet been identified, but disabling ASPM L0s makes the errors go away. This could suggest that either the current ASPM implementation is incomplete or that L0s is not supported with these devices.
...As this series fixes a regression in 6.7 (which enabled ASPM) and fixes a user-reported problem with the Wi-Fi often not starting at boot, I think we should merge this series for the 6.8 cycle. The final patch enabling the GIC ITS should wait for 6.9.
The DT bindings and PCI patch are expected to go through the PCI tree, while Bjorn A takes the devicetree updates through the Qualcomm tree.
...Quoted 5 lines
Johan Hovold (12):
dt-bindings: PCI: qcom: Allow 'required-opps'
dt-bindings: PCI: qcom: Do not require 'msi-map-mask'
dt-bindings: PCI: qcom: Allow 'aspm-no-l0s'
PCI: qcom: Add support for disabling ASPM L0s in devicetreeThe ASPM patches fix a v6.7 regression, so it would be good to fix that in v6.8.
Quoted 6 lines
Mani, if you are OK with them, I can add them to for-linus for v6.8.
What about the 'required-opps' and 'msi-map-mask' patches? If they're important, I can merge them for v6.8, too, but it's late in the cycle and it's not clear from the commit logs why they shouldn't wait for v6.9.
Either way, I believe they should go through the qcom tree, as it's a very hot one with lots of different changes to a given file.
Unless the soc-late window is already closed... Bjorn A will know.
Konrad
Quoted 9 lines · Konrad Dybcio
Quoted 7 lines · Bjorn Helgaas
Quoted 5 lines · Johan Hovold
Johan Hovold (12):
dt-bindings: PCI: qcom: Allow 'required-opps'
dt-bindings: PCI: qcom: Do not require 'msi-map-mask'
dt-bindings: PCI: qcom: Allow 'aspm-no-l0s'
PCI: qcom: Add support for disabling ASPM L0s in devicetree
Quoted 7 lines
Quoted 4 lines
What about the 'required-opps' and 'msi-map-mask' patches? If they're important, I can merge them for v6.8, too, but it's late in the cycle and it's not clear from the commit logs why they shouldn't wait for v6.9.
Either way, I believe they should go through the qcom tree, as it's a very hot one with lots of different changes to a given file.
I think Bjorn was just referring to the three binding patches listed above and which should go through the PCI tree (unlike the later DT fixes which will go through the Qualcomm SoC tree).
Johan
Quoted 33 lines · Bjorn Helgaas
[+to Mani]
Quoted 21 lines · Johan Hovold
This series addresses a few problems with the sc8280xp PCIe implementation.
...A recent commit enabling ASPM on certain Qualcomm platforms introduced further errors when using the Wi-Fi on the X13s as well as when accessing the NVMe on the CRD. The exact reason for this has not yet been identified, but disabling ASPM L0s makes the errors go away. This could suggest that either the current ASPM implementation is incomplete or that L0s is not supported with these devices.
...As this series fixes a regression in 6.7 (which enabled ASPM) and fixes a user-reported problem with the Wi-Fi often not starting at boot, I think we should merge this series for the 6.8 cycle. The final patch enabling the GIC ITS should wait for 6.9.
The DT bindings and PCI patch are expected to go through the PCI tree, while Bjorn A takes the devicetree updates through the Qualcomm tree.
...Quoted 5 lines
Johan Hovold (12):
dt-bindings: PCI: qcom: Allow 'required-opps'
dt-bindings: PCI: qcom: Do not require 'msi-map-mask'
dt-bindings: PCI: qcom: Allow 'aspm-no-l0s'
PCI: qcom: Add support for disabling ASPM L0s in devicetreeThe ASPM patches fix a v6.7 regression, so it would be good to fix that in v6.8.
Quoted 6 lines
Mani, if you are OK with them, I can add them to for-linus for v6.8.
What about the 'required-opps' and 'msi-map-mask' patches? If they're important, I can merge them for v6.8, too, but it's late in the cycle and it's not clear from the commit logs why they shouldn't wait for v6.9.
The 'required-opps' binding patch is a prerequisite for the corresponding DT fix, which is tagged for stable and that should go in 6.8.
The 'msi-map-mask' binding update is strictly only needed for enabling the ITS, which is 6.9 material, even if it's arguably also a fix for the current binding. So this one could possibly wait for 6.9 even if it may make sense to just take all three now for 6.8 to only have to deal with the mentioned binding conflict once.
Johan
Quoted 33 lines · Bjorn Helgaas
[+to Mani]
Quoted 21 lines · Johan Hovold
This series addresses a few problems with the sc8280xp PCIe implementation.
...A recent commit enabling ASPM on certain Qualcomm platforms introduced further errors when using the Wi-Fi on the X13s as well as when accessing the NVMe on the CRD. The exact reason for this has not yet been identified, but disabling ASPM L0s makes the errors go away. This could suggest that either the current ASPM implementation is incomplete or that L0s is not supported with these devices.
...As this series fixes a regression in 6.7 (which enabled ASPM) and fixes a user-reported problem with the Wi-Fi often not starting at boot, I think we should merge this series for the 6.8 cycle. The final patch enabling the GIC ITS should wait for 6.9.
The DT bindings and PCI patch are expected to go through the PCI tree, while Bjorn A takes the devicetree updates through the Qualcomm tree.
...Quoted 5 lines
Johan Hovold (12):
dt-bindings: PCI: qcom: Allow 'required-opps'
dt-bindings: PCI: qcom: Do not require 'msi-map-mask'
dt-bindings: PCI: qcom: Allow 'aspm-no-l0s'
PCI: qcom: Add support for disabling ASPM L0s in devicetreeThe ASPM patches fix a v6.7 regression, so it would be good to fix that in v6.8.
Quoted 7 lines
Mani, if you are OK with them, I can add them to for-linus for v6.8.
What about the 'required-opps' and 'msi-map-mask' patches? If they're important, I can merge them for v6.8, too, but it's late in the cycle and it's not clear from the commit logs why they shouldn't wait for v6.9.
I'm checking with Qcom HW team on the ASPM behavior. So please hold off the ASPM related patches until I get an answer. But 'required-opps' and 'msi-map-mask' patches can be applied for 6.9 (not strictly fixing anything in 6.8).
- Mani
Signature · 1 line
மணிவண்ணன் சதாசிவம்
Quoted 20 lines · Manivannan Sadhasivam
Quoted 11 lines · Bjorn Helgaas
Quoted 5 lines · Johan Hovold
Johan Hovold (12):
dt-bindings: PCI: qcom: Allow 'required-opps'
dt-bindings: PCI: qcom: Do not require 'msi-map-mask'
dt-bindings: PCI: qcom: Allow 'aspm-no-l0s'
PCI: qcom: Add support for disabling ASPM L0s in devicetreeThe ASPM patches fix a v6.7 regression, so it would be good to fix that in v6.8.
Quoted 7 lines
Mani, if you are OK with them, I can add them to for-linus for v6.8.
What about the 'required-opps' and 'msi-map-mask' patches? If they're important, I can merge them for v6.8, too, but it's late in the cycle and it's not clear from the commit logs why they shouldn't wait for v6.9.
Quoted 3 lines
I'm checking with Qcom HW team on the ASPM behavior. So please hold off the ASPM related patches until I get an answer. But 'required-opps' and 'msi-map-mask' patches can be applied for 6.9 (not strictly fixing anything in 6.8).
As I mentioned, the 'required-opps' binding update is needed to fix the missing OPP vote so blocking the binding patch would block merging the DT fix which could otherwise go into 6.8.
The 'msi-map-mask' is arguably a fix of the binding which should never have had that property, but sure, it's strictly only needed for 6.9.
And Bjorn A has already checked with the Qualcomm PCI team regarding ASPM. It's also been two weeks since you said you were going to check with your contacts. Is it really worth waiting more for an answer from that part of the team? We can always amend the ASPM fixes later when/if we learn more.
Note that this is also a blocker for merging ITS support for 6.9.
Johan
Quoted 21 lines · Johan Hovold
Quoted 20 lines · Manivannan Sadhasivam
Quoted 18 lines · Bjorn Helgaas
Quoted 5 lines · Johan Hovold
Johan Hovold (12):
dt-bindings: PCI: qcom: Allow 'required-opps'
dt-bindings: PCI: qcom: Do not require 'msi-map-mask'
dt-bindings: PCI: qcom: Allow 'aspm-no-l0s'
PCI: qcom: Add support for disabling ASPM L0s in devicetreeThe ASPM patches fix a v6.7 regression, so it would be good to fix that in v6.8.
Mani, if you are OK with them, I can add them to for-linus for v6.8.
What about the 'required-opps' and 'msi-map-mask' patches? If they're important, I can merge them for v6.8, too, but it's late in the cycle and it's not clear from the commit logs why they shouldn't wait for v6.9.
Quoted 8 lines
Quoted 3 lines
I'm checking with Qcom HW team on the ASPM behavior. So please hold off the ASPM related patches until I get an answer. But 'required-opps' and 'msi-map-mask' patches can be applied for 6.9 (not strictly fixing anything in 6.8).
As I mentioned, the 'required-opps' binding update is needed to fix the missing OPP vote so blocking the binding patch would block merging the DT fix which could otherwise go into 6.8.
I agree that the fix gets the priority. But some maintainers perfer to merge fix patches _only_ if they are fixing the issue introduced in the ongoing release. But if Bjorn has no issues in merging these for 6.8, then it is fine.
Quoted 11 lines
The 'msi-map-mask' is arguably a fix of the binding which should never have had that property, but sure, it's strictly only needed for 6.9.
And Bjorn A has already checked with the Qualcomm PCI team regarding ASPM. It's also been two weeks since you said you were going to check with your contacts. Is it really worth waiting more for an answer from that part of the team? We can always amend the ASPM fixes later when/if we learn more.
Note that this is also a blocker for merging ITS support for 6.9.
I got it, but we cannot just merge the patches without finding the rootcause. I heard from Qcom that this AER error could also be due to PHY init sequence as spotted on some other platforms, so if that is the case then the DT property is not correct.
Since this is not the hot target now (for Qcom), it takes time to check things.
- Mani
Signature · 1 line
மணிவண்ணன் சதாசிவம்
Quoted 9 lines · Manivannan Sadhasivam
Quoted 3 lines · Johan Hovold
As I mentioned, the 'required-opps' binding update is needed to fix the missing OPP vote so blocking the binding patch would block merging the DT fix which could otherwise go into 6.8.
I agree that the fix gets the priority. But some maintainers perfer to merge fix patches _only_ if they are fixing the issue introduced in the ongoing release. But if Bjorn has no issues in merging these for 6.8, then it is fine.
It also depends on the severity of the issue and to some extent the complexity of the fix. These binding fixes are certainly low risk. :)
Quoted 9 lines
Quoted 9 lines
The 'msi-map-mask' is arguably a fix of the binding which should never have had that property, but sure, it's strictly only needed for 6.9.
And Bjorn A has already checked with the Qualcomm PCI team regarding ASPM. It's also been two weeks since you said you were going to check with your contacts. Is it really worth waiting more for an answer from that part of the team? We can always amend the ASPM fixes later when/if we learn more.
Quoted 6 lines
Quoted 1 line
Note that this is also a blocker for merging ITS support for 6.9.
I got it, but we cannot just merge the patches without finding the rootcause. I heard from Qcom that this AER error could also be due to PHY init sequence as spotted on some other platforms, so if that is the case then the DT property is not correct.
I've verified the PHY sequences both against what the UEFI firmware (and hence Windows) uses as well as against the internal Qualcomm documentation (with the help of Bjorn A). And Qualcomm did say that such errors are also seen under Windows on these platforms.
But the fact that the symptoms differ between the CRD and X13s, which use the same Atheros Wi-Fi controller (and the same PHY initialisation) also suggests that this may to some extent be due to some property of the machine.
Notably, on the X13s there are lot of errors when pushing data (e.g. using iperf3), whereas on the CRD the are no errors when running such tests.
When leaving the CRD on for long periods of time with the Wi-Fi disconnected, I do however see occasional correctable errors.
Quoted 2 lines
Since this is not the hot target now (for Qcom), it takes time to check things.
I think that based on the available data it's reasonable to go ahead and merge these patches. In the event that this turns out to be a configuration issue, we can just drop the 'aspm-no-l0s' properties again.
Johan
Quoted 7 lines · Johan Hovold
Quoted 5 lines · Manivannan Sadhasivam
Quoted 3 lines · Johan Hovold
As I mentioned, the 'required-opps' binding update is needed to fix the missing OPP vote so blocking the binding patch would block merging the DT fix which could otherwise go into 6.8.
Quoted 7 lines
Quoted 3 lines
I agree that the fix gets the priority. But some maintainers perfer to merge fix patches _only_ if they are fixing the issue introduced in the ongoing release. But if Bjorn has no issues in merging these for 6.8, then it is fine.
It also depends on the severity of the issue and to some extent the complexity of the fix. These binding fixes are certainly low risk. :)
Right.
Quoted 16 lines
Quoted 9 lines
Quoted 9 lines
The 'msi-map-mask' is arguably a fix of the binding which should never have had that property, but sure, it's strictly only needed for 6.9.
And Bjorn A has already checked with the Qualcomm PCI team regarding ASPM. It's also been two weeks since you said you were going to check with your contacts. Is it really worth waiting more for an answer from that part of the team? We can always amend the ASPM fixes later when/if we learn more.
Quoted 6 lines
Quoted 1 line
Note that this is also a blocker for merging ITS support for 6.9.
I got it, but we cannot just merge the patches without finding the rootcause. I heard from Qcom that this AER error could also be due to PHY init sequence as spotted on some other platforms, so if that is the case then the DT property is not correct.
Quoted 5 lines
I've verified the PHY sequences both against what the UEFI firmware (and hence Windows) uses as well as against the internal Qualcomm documentation (with the help of Bjorn A). And Qualcomm did say that such errors are also seen under Windows on these platforms.
Well, sometimes the init sequence published by qualcomm could turn out to be faulty. That's why they publish v2 sequence and such. And I want to rule out that possibility in this case.
Quoted 5 lines
But the fact that the symptoms differ between the CRD and X13s, which use the same Atheros Wi-Fi controller (and the same PHY initialisation) also suggests that this may to some extent be due to some property of the machine.
Quoted 7 lines
Notably, on the X13s there are lot of errors when pushing data (e.g. using iperf3), whereas on the CRD the are no errors when running such tests.
When leaving the CRD on for long periods of time with the Wi-Fi disconnected, I do however see occasional correctable errors.
This may be due to hardware design that I agree (possibly influenced by driver defect).
Quoted 8 lines
Quoted 2 lines
Since this is not the hot target now (for Qcom), it takes time to check things.
I think that based on the available data it's reasonable to go ahead and merge these patches. In the event that this turns out to be a configuration issue, we can just drop the 'aspm-no-l0s' properties again.
Well the problem is, if you are not sure, then adding the DT properties is certainly not correct. As that implies a hardware defect, but it may not be. So let's wait for some time to find out the actual issue.
- Mani
Signature · 1 line
மணிவண்ணன் சதாசிவம்
Quoted 10 lines · Manivannan Sadhasivam
Quoted 4 lines · Johan Hovold
I think that based on the available data it's reasonable to go ahead and merge these patches. In the event that this turns out to be a configuration issue, we can just drop the 'aspm-no-l0s' properties again.
Well the problem is, if you are not sure, then adding the DT properties is certainly not correct. As that implies a hardware defect, but it may not be. So let's wait for some time to find out the actual issue.
Our devicetrees are always going to be a tentative description of the hardware, and this especially true for Qualcomm that don't publish any documentation so that people are forced to rely on informed guesses based on downstream devicetrees and drivers and reverse engineering.
As far as I can tell, after having spent a lot of time on this and checking with sources inside Qualcomm, the hardware is to blame here. If this turns out not to be true, we can always revise later. We do this all the time, as you know.
I'm all for digging further into this issue with the help of Qualcomm, but I don't think that should block this series as that would leave the link errors that we hit since 6.7 in place and effectively prevent us from enabling the ITS in 6.9.
Johan
Quoted 22 lines · Johan Hovold
Quoted 10 lines · Manivannan Sadhasivam
Quoted 4 lines · Johan Hovold
I think that based on the available data it's reasonable to go ahead and merge these patches. In the event that this turns out to be a configuration issue, we can just drop the 'aspm-no-l0s' properties again.
Well the problem is, if you are not sure, then adding the DT properties is certainly not correct. As that implies a hardware defect, but it may not be. So let's wait for some time to find out the actual issue.
Our devicetrees are always going to be a tentative description of the hardware, and this especially true for Qualcomm that don't publish any documentation so that people are forced to rely on informed guesses based on downstream devicetrees and drivers and reverse engineering.
As far as I can tell, after having spent a lot of time on this and checking with sources inside Qualcomm, the hardware is to blame here. If this turns out not to be true, we can always revise later. We do this all the time, as you know.
Quoted 5 lines
I'm all for digging further into this issue with the help of Qualcomm, but I don't think that should block this series as that would leave the link errors that we hit since 6.7 in place and effectively prevent us from enabling the ITS in 6.9.
Sounds fair. I will report back, perhaps with a fix based on what I get to know.
But I think it is better to disable L0s in the SoC dtsi itself. That's not only because there are patches to essentially disable L0s in 2 of the available platforms making use of this Soc, but also you are enabling GIC ITS in the SoC dtsi and that may affect sa8540p which is making use of this dtsi.
The users of that SoC may have not noticed the errors as you did before, but enabling GIC ITS will certainly make the issue visible to them (more likely).
Also, if it turns out to be a hardware IP issue, then we can leave the patches as it is, otherwise we can revert them easily.
- Mani
Signature · 1 line
மணிவண்ணன் சதாசிவம்
Quoted 8 lines · Manivannan Sadhasivam
Quoted 4 lines · Johan Hovold
I'm all for digging further into this issue with the help of Qualcomm, but I don't think that should block this series as that would leave the link errors that we hit since 6.7 in place and effectively prevent us from enabling the ITS in 6.9.
Sounds fair. I will report back, perhaps with a fix based on what I get to know.
Sounds good, thanks.
Quoted 4 lines
But I think it is better to disable L0s in the SoC dtsi itself. That's not only because there are patches to essentially disable L0s in 2 of the available platforms making use of this Soc, but also you are enabling GIC ITS in the SoC dtsi and that may affect sa8540p which is making use of this dtsi.
I did not do so on purpose as I'm only disabling L0s on machines where I've confirmed the issue. And the assumption for now is that this is a machine-level issue.
Quoted 2 lines
The users of that SoC may have not noticed the errors as you did before, but enabling GIC ITS will certainly make the issue visible to them (more likely).
Sure and that would be good to know as that would give us another data point which may help determine where the problem lies. Enabling the ITS will (hopefully) be done in 6.9 so we'll have a whole cycle to disable L0s where needed. I don't think this should be done before then.
Johan
Manivannan Sadhasivam
Reviewed-byQuoted 21 lines · Johan Hovold
Quoted 8 lines · Manivannan Sadhasivam
Quoted 4 lines · Johan Hovold
I'm all for digging further into this issue with the help of Qualcomm, but I don't think that should block this series as that would leave the link errors that we hit since 6.7 in place and effectively prevent us from enabling the ITS in 6.9.
Sounds fair. I will report back, perhaps with a fix based on what I get to know.
Sounds good, thanks.
Quoted 4 lines
But I think it is better to disable L0s in the SoC dtsi itself. That's not only because there are patches to essentially disable L0s in 2 of the available platforms making use of this Soc, but also you are enabling GIC ITS in the SoC dtsi and that may affect sa8540p which is making use of this dtsi.
I did not do so on purpose as I'm only disabling L0s on machines where I've confirmed the issue. And the assumption for now is that this is a machine-level issue.
Quoted 8 lines
Quoted 2 lines
The users of that SoC may have not noticed the errors as you did before, but enabling GIC ITS will certainly make the issue visible to them (more likely).
Sure and that would be good to know as that would give us another data point which may help determine where the problem lies. Enabling the ITS will (hopefully) be done in 6.9 so we'll have a whole cycle to disable L0s where needed. I don't think this should be done before then.
Ok. Let's see what happens :)
For the series:
- Mani
Signature · 1 line
மணிவண்ணன் சதாசிவம்
Quoted 3 lines · Manivannan Sadhasivam
For the series:
Reviewed-by Manivannan Sadhasivam <[email protected]>
Thanks for reviewing.
Johan
Quoted 11 lines · Johan Hovold
Quoted 10 lines · Manivannan Sadhasivam
Quoted 3 lines · Johan Hovold
As I mentioned, the 'required-opps' binding update is needed to fix the missing OPP vote so blocking the binding patch would block merging the DT fix which could otherwise go into 6.8.
I agree that the fix gets the priority. But some maintainers perfer to merge fix patches _only_ if they are fixing the issue introduced in the ongoing release. But if Bjorn has no issues in merging these for 6.8, then it is fine.
I do prefer to merge only regression and important fixes after the merge window, so I want to be able to provide justification.
Quoted 3 lines
It also depends on the severity of the issue and to some extent the complexity of the fix. These binding fixes are certainly low risk. :)
IIUC we're talking about:
arm64: dts: qcom: sc8280xp: add missing PCIe minimum OPP
dt-bindings: PCI: qcom: Allow 'required-opps'
These don't look like a regression fix (correct me if I'm wrong), and I can't tell whether they fix a user-visible problem, since sc8280xp.dtsi does already contain 'required-opps' for ufs_mem_hc, usb_0, and usb_1, which are mentioned in the commit log as covering up the issue.
If these patches wait until v6.9, what badness ensues?
Bjorn
Quoted 10 lines · Bjorn Helgaas
Quoted 3 lines · Johan Hovold
It also depends on the severity of the issue and to some extent the complexity of the fix. These binding fixes are certainly low risk. :)
IIUC we're talking about:
arm64: dts: qcom: sc8280xp: add missing PCIe minimum OPP
dt-bindings: PCI: qcom: Allow 'required-opps'
Right.
Quoted 5 lines
These don't look like a regression fix (correct me if I'm wrong), and I can't tell whether they fix a user-visible problem, since
sc8280xp.dtsidoes already contain 'required-opps' forufs_mem_hc,usb_0, andusb_1, which are mentioned in the commit log as covering up the issue.
The issue has been there since PCIe support was added for this platform and does not cause any issues until the USB and UFS controllers are runtime suspended.
When that happens nothing is currently making sure that we have enough power to run PCIe at gen3 speeds, something which can potentially result in system instability (e.g. resets).
Quoted 1 line
If these patches wait until v6.9, what badness ensues?
We'd have a few more weeks where users enabling runtime PM for USB on the X13s could hit this before we can get the fix backported to stable.
I could have put some more details in the commit message for the DT patch, but I did not think that amending the PCIe binding would be controversial. (I guess we can also take the DT fix without waiting for the binding update as it has been acked by a DT maintainer even if that would result in some DT checker warnings until things are aligned again.)
Let me know what you decide regarding getting the whole series into 6.8, and then I can spend some more time on rewording, splitting and rebasing this series if needed.
Johan
Bjorn Andersson
maintainerQuoted 16 lines · Bjorn Helgaas
Quoted 11 lines · Johan Hovold
Quoted 10 lines · Manivannan Sadhasivam
Quoted 3 lines · Johan Hovold
As I mentioned, the 'required-opps' binding update is needed to fix the missing OPP vote so blocking the binding patch would block merging the DT fix which could otherwise go into 6.8.
I agree that the fix gets the priority. But some maintainers perfer to merge fix patches _only_ if they are fixing the issue introduced in the ongoing release. But if Bjorn has no issues in merging these for 6.8, then it is fine.
I do prefer to merge only regression and important fixes after the merge window, so I want to be able to provide justification.
Quoted 7 lines
Quoted 3 lines
It also depends on the severity of the issue and to some extent the complexity of the fix. These binding fixes are certainly low risk. :)
IIUC we're talking about:
arm64: dts: qcom: sc8280xp: add missing PCIe minimum OPP
I'd prefer to take this one through my tree. I will double check the hardware documentation (there are differences in sc8280xp here) and decide how to proceed...
Quoted 1 line
dt-bindings: PCI: qcom: Allow 'required-opps'
Picking this for v6.9 is fine, no practical badness ensues. We would temporarily have a few additional DeviceTree validation warnings in the v6.8 release...
Regards,
Bjorn
Quoted 10 lines
These don't look like a regression fix (correct me if I'm wrong), and I can't tell whether they fix a user-visible problem, since
sc8280xp.dtsidoes already contain 'required-opps' forufs_mem_hc,usb_0, andusb_1, which are mentioned in the commit log as covering up the issue.If these patches wait until v6.9, what badness ensues?
Bjorn
Re: (subset) [PATCH v2 00/12] arm64: dts: qcom: sc8280xp: PCIe fixes and GICv3 ITS enable
In reply to Johan HovoldQuoted 10 lines · Johan Hovold
This series addresses a few problems with the sc8280xp PCIe implementation.
The DWC PCIe controller can either use its internal MSI controller or an external one such as the GICv3 ITS. Enabling the latter allows for assigning affinity to individual interrupts, but results in a large amount of Correctable Errors being logged on both the Lenovo ThinkPad X13s and the sc8280xp-crd reference design.
[...]
Applied, thanks!
[06/12] arm64: dts: qcom: sc8280xp-crd: limit pcie4 link speed
commit: db8138845cebcdd0c709570b8217bd052757b8df
[07/12] arm64: dts: qcom: sc8280xp-x13s: limit pcie4 link speed
commit: 7a1c6a8bf47b0b290c79b9cc3ba6ee68be5522e8Best regards,
Signature · 1 line
Bjorn Andersson <[email protected]>