PCI: Fix the issue of failed speed limit lifting
v4 cover PCI: Fix the issue of failed speed limit lifting
From: Jiwei Sun <[email protected]>
Since commit de9a6c8d5dbf ("PCI/bwctrl: Add pcie_set_target_speed() to set PCIe Link Speed"), there are two potential issues in the function pcie_failed_link_retrain().
(1) The macro PCIE_LNKCTL2_TLS2SPEED() and PCIE_LNKCAP_SLS2SPEED() just use the link speed field of the registers. However, there are many other different function fields in the Link Control 2 Register or the Link Capabilities Register. If the register value is directly used by the two macros, it may cause getting an error link speed value (PCI_SPEED_UNKNOWN).
(2) In the pcie_failed_link_retrain(), the local variable lnkctl2 is not changed after reading from PCI_EXP_LNKCTL2. It might cause that the removing 2.5GT/s downstream link speed restriction codes are not executed.
In order to avoid the above-mentioned potential issues, only keep link speed field of the two registers before using and reread the Link Control 2 Register before using.
This series focuses on the first patch of the original series [1]. The second one of the original series will submitted via the other single patch.
[1] https://lore.kernel.org/linux-pci/[email protected]/
---
v4 changes:
- rename the variable name in the macro
v3 changes:
- add fix tag in the commit messages of first patch
- add an empty line after the local variable definition in the macro
- adjust the position of reading the Link Control 2 register in the code
v2 changes:
- divide the two issues into different patches
- get fixed inside the macros
Jiwei Sun (2):
PCI: Fix the wrong reading of register fields
PCI: Adjust the position of reading the Link Control 2 register
drivers/pci/pci.h | 32 +++++++++++++++++++------------- drivers/pci/quirks.c | 6 ++++-- 2 files changed, 23 insertions(+), 15 deletions(-)
Signature · 1 line
2.34.1
PATCH v4 1/2 PCI: Fix the wrong reading of register fields
From: Jiwei Sun <[email protected]>
The macro PCIE_LNKCTL2_TLS2SPEED() and PCIE_LNKCAP_SLS2SPEED() just use the link speed field of the registers. However, there are many other different function fields in the Link Control 2 Register or the Link Capabilities Register. If the register value is directly used by the two macros, it may cause getting an error link speed value (PCI_SPEED_UNKNOWN).
In order to avoid the above-mentioned potential issue, only keep link speed field of the two registers before using.
--- drivers/pci/pci.h | 32 +++++++++++++++++++------------- 1 file changed, 19 insertions(+), 13 deletions(-)
drivers/pci/pci.h +19 -13
@@ -337,12 +337,14 @@ void pci_bus_put(struct pci_bus *bus);#define PCIE_LNKCAP_SLS2SPEED (lnkcap) \ ({ \ - ((lnkcap) == PCI_EXP_LNKCAP_SLS_64_0GB ? PCIE_SPEED_64_0GT : \ - (lnkcap) == PCI_EXP_LNKCAP_SLS_32_0GB ? PCIE_SPEED_32_0GT : \ - (lnkcap) == PCI_EXP_LNKCAP_SLS_16_0GB ? PCIE_SPEED_16_0GT : \ - (lnkcap) == PCI_EXP_LNKCAP_SLS_8_0GB ? PCIE_SPEED_8_0GT : \ - (lnkcap) == PCI_EXP_LNKCAP_SLS_5_0GB ? PCIE_SPEED_5_0GT : \ - (lnkcap) == PCI_EXP_LNKCAP_SLS_2_5GB ? PCIE_SPEED_2_5GT : \ + u32 lnkcap_sls = (lnkcap) & PCI_EXP_LNKCAP_SLS; \ + \ + (lnkcap_sls == PCI_EXP_LNKCAP_SLS_64_0GB ? PCIE_SPEED_64_0GT : \ + lnkcap_sls == PCI_EXP_LNKCAP_SLS_32_0GB ? PCIE_SPEED_32_0GT : \ + lnkcap_sls == PCI_EXP_LNKCAP_SLS_16_0GB ? PCIE_SPEED_16_0GT : \ + lnkcap_sls == PCI_EXP_LNKCAP_SLS_8_0GB ? PCIE_SPEED_8_0GT : \ + lnkcap_sls == PCI_EXP_LNKCAP_SLS_5_0GB ? PCIE_SPEED_5_0GT : \ + lnkcap_sls == PCI_EXP_LNKCAP_SLS_2_5GB ? PCIE_SPEED_2_5GT : \ PCI_SPEED_UNKNOWN); \ })
@@ -357,13 +359,17 @@ void pci_bus_put(struct pci_bus *bus); PCI_SPEED_UNKNOWN)#define PCIE_LNKCTL2_TLS2SPEED (lnkctl2) \ - ((lnkctl2) == PCI_EXP_LNKCTL2_TLS_64_0GT ? PCIE_SPEED_64_0GT : \ - (lnkctl2) == PCI_EXP_LNKCTL2_TLS_32_0GT ? PCIE_SPEED_32_0GT : \ - (lnkctl2) == PCI_EXP_LNKCTL2_TLS_16_0GT ? PCIE_SPEED_16_0GT : \ - (lnkctl2) == PCI_EXP_LNKCTL2_TLS_8_0GT ? PCIE_SPEED_8_0GT : \ - (lnkctl2) == PCI_EXP_LNKCTL2_TLS_5_0GT ? PCIE_SPEED_5_0GT : \ - (lnkctl2) == PCI_EXP_LNKCTL2_TLS_2_5GT ? PCIE_SPEED_2_5GT : \ - PCI_SPEED_UNKNOWN) +({ \ + u16 lnkctl2_tls = (lnkctl2) & PCI_EXP_LNKCTL2_TLS; \ + \ + (lnkctl2_tls == PCI_EXP_LNKCTL2_TLS_64_0GT ? PCIE_SPEED_64_0GT : \ + lnkctl2_tls == PCI_EXP_LNKCTL2_TLS_32_0GT ? PCIE_SPEED_32_0GT : \ + lnkctl2_tls == PCI_EXP_LNKCTL2_TLS_16_0GT ? PCIE_SPEED_16_0GT : \ + lnkctl2_tls == PCI_EXP_LNKCTL2_TLS_8_0GT ? PCIE_SPEED_8_0GT : \ + lnkctl2_tls == PCI_EXP_LNKCTL2_TLS_5_0GT ? PCIE_SPEED_5_0GT : \ + lnkctl2_tls == PCI_EXP_LNKCTL2_TLS_2_5GT ? PCIE_SPEED_2_5GT : \ + PCI_SPEED_UNKNOWN); \ +})/* PCIe speed to Mb/s reduced by encoding overhead */ #define PCIE_SPEED2MBS_ENC (speed) \
Signature · 1 line
2.34.1
PATCH v4 2/2 PCI: Adjust the position of reading the Link Control 2 register
In reply to Jiwei SunFrom: Jiwei Sun <[email protected]>
In the commit a89c82249c37 ("PCI: Work around PCIe link training failures"), if the speed limit is set to 2.5 GT/s and the retraining is successful, an attempt will be made to lift the speed limit. One condition for lifting the speed limit is to check whether the link speed field of the Link Control 2 register is PCI_EXP_LNKCTL2_TLS_2_5GT.
However, since the commit de9a6c8d5dbf ("PCI/bwctrl: Add pcie_set_target_speed() to set PCIe Link Speed"), the lnkctl2 local variable does not undergo any changes during the speed limit setting and retraining process. As a result, the code intended to lift the speed limit is not executed.
To address this issue, adjust the position of the Link Control 2 register read operation in the code and place it before its use.
--- drivers/pci/quirks.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
drivers/pci/quirks.c +4 -2
@@ -108,13 +108,13 @@ int pcie_failed_link_retrain(struct pci_dev *dev) !pcie_cap_has_lnkctl2(dev) || !dev->link_active_reporting)return ret; -pcie_capability_read_word (dev,PCI_EXP_LNKCTL2 , &lnkctl2);pcie_capability_read_word (dev,PCI_EXP_LNKSTA , &lnksta);if (!(lnksta &PCI_EXP_LNKSTA_DLLLA ) &&pcie_lbms_seen (dev, lnksta)) { -u16 oldlnkctl2 = lnkctl2; +u16 oldlnkctl2;pci_info (dev,"broken device, retraining non-functional downstream link at 2.5GT/s\n" ); +pcie_capability_read_word (dev,PCI_EXP_LNKCTL2 , &oldlnkctl2); ret =pcie_set_target_speed (dev,PCIE_SPEED_2_5GT , false);if (ret) {pci_info (dev,"retraining failed\n" );
@@ -126,6 +126,8 @@ int pcie_failed_link_retrain(struct pci_dev *dev)pcie_capability_read_word (dev,PCI_EXP_LNKSTA , &lnksta); } +pcie_capability_read_word (dev,PCI_EXP_LNKCTL2 , &lnkctl2); +if ((lnksta &PCI_EXP_LNKSTA_DLLLA ) && (lnkctl2 &PCI_EXP_LNKCTL2_TLS ) ==PCI_EXP_LNKCTL2_TLS_2_5GT &&pci_match_id (ids, dev)) {
Signature · 1 line
2.34.1
Quoted 38 lines · Jiwei Sun
From: Jiwei Sun <[email protected]>
Since commit
de9a6c8d5dbf("PCI/bwctrl: Addpcie_set_target_speed()to set PCIe Link Speed"), there are two potential issues in the functionpcie_failed_link_retrain().(1) The macro
PCIE_LNKCTL2_TLS2SPEED()andPCIE_LNKCAP_SLS2SPEED()just use the link speed field of the registers. However, there are many other different function fields in the Link Control 2 Register or the Link Capabilities Register. If the register value is directly used by the two macros, it may cause getting an error link speed value (PCI_SPEED_UNKNOWN).(2) In the
pcie_failed_link_retrain(), the local variable lnkctl2 is not changed after reading fromPCI_EXP_LNKCTL2. It might cause that the removing 2.5GT/s downstream link speed restriction codes are not executed.In order to avoid the above-mentioned potential issues, only keep link speed field of the two registers before using and reread the Link Control 2 Register before using.
This series focuses on the first patch of the original series [1]. The second one of the original series will submitted via the other single patch.
[1] https://lore.kernel.org/linux-pci/[email protected]/
---
v4 changes:
- rename the variable name in the macrov3 changes:
- add fix tag in the commit messages of first patch
- add an empty line after the local variable definition in the macro
- adjust the position of reading the Link Control 2 register in the codev2 changes:
- divide the two issues into different patches
- get fixed inside the macros
Quoted 8 lines
Jiwei Sun (2):
PCI: Fix the wrong reading of register fields
PCI: Adjust the position of reading the Link Control 2 registerdrivers/pci/pci.h | 32 +++++++++++++++++++------------- drivers/pci/quirks.c | 6 ++++-- 2 files changed, 23 insertions(+), 15 deletions(-)
Gentle ping.
Thanks,
Regards,
Jiwei
[+cc Andrew, Matthew, Sathy]
Quoted 38 lines · Jiwei Sun
From: Jiwei Sun <[email protected]>
Since commit
de9a6c8d5dbf("PCI/bwctrl: Addpcie_set_target_speed()to set PCIe Link Speed"), there are two potential issues in the functionpcie_failed_link_retrain().(1) The macro
PCIE_LNKCTL2_TLS2SPEED()andPCIE_LNKCAP_SLS2SPEED()just use the link speed field of the registers. However, there are many other different function fields in the Link Control 2 Register or the Link Capabilities Register. If the register value is directly used by the two macros, it may cause getting an error link speed value (PCI_SPEED_UNKNOWN).(2) In the
pcie_failed_link_retrain(), the local variable lnkctl2 is not changed after reading fromPCI_EXP_LNKCTL2. It might cause that the removing 2.5GT/s downstream link speed restriction codes are not executed.In order to avoid the above-mentioned potential issues, only keep link speed field of the two registers before using and reread the Link Control 2 Register before using.
This series focuses on the first patch of the original series [1]. The second one of the original series will submitted via the other single patch.
[1] https://lore.kernel.org/linux-pci/[email protected]/
---
v4 changes:
- rename the variable name in the macrov3 changes:
- add fix tag in the commit messages of first patch
- add an empty line after the local variable definition in the macro
- adjust the position of reading the Link Control 2 register in the codev2 changes:
- divide the two issues into different patches
- get fixed inside the macros
Quoted 7 lines
Jiwei Sun (2):
PCI: Fix the wrong reading of register fields
PCI: Adjust the position of reading the Link Control 2 registerdrivers/pci/pci.h | 32 +++++++++++++++++++------------- drivers/pci/quirks.c | 6 ++++-- 2 files changed, 23 insertions(+), 15 deletions(-)
Sorry, this totally slipped through the cracks. I applied both of these to pci/enumeration for v6.17.
Andrew reported tripping over this issue fixed by the first patch, and Lukas also posted a similar patch [1] to fix it, so I updated the commit log as below to include details of Andrew's report.
As Lukas did, I added a stable tag but made it for v6.13+ (not v6.12+) because I think the actual problem showed up with de9a6c8d5dbf ("PCI/bwctrl: Add pcie_set_target_speed() to set PCIe Link Speed"), not with f68dea13405c ("PCI: Revert to the original speed after PCIe failed link retraining").
[1] https://lore.kernel.org/r/1c92ef6bcb314ee6977839b46b393282e4f52e74.1750684771.git.lukas@wunner.de
PCI: Fix link speed calculation on retrain failure
When pcie_failed_link_retrain() fails to retrain, it tries to revert to the previous link speed. However it calculates that speed from the Link Control 2 register without masking out non-speed bits first.
PCIE_LNKCTL2_TLS2SPEED() converts such incorrect values to PCI_SPEED_UNKNOWN (0xff), which in turn causes a WARN splat in pcie_set_target_speed():
pci 0000:00:01.1: [1022:14ed] type 01 class 0x060400 PCIe Root Port pci 0000:00:01.1: broken device, retraining non-functional downstream link at 2.5GT/s pci 0000:00:01.1: retraining failed
WARNING: CPU: 1 PID: 1 at drivers/pci/pcie/bwctrl.c:168 pcie_set_target_speed
RDX: 0000000000000001 RSI: 00000000000000ff RDI: ffff9acd82efa000 pcie_failed_link_retrain
pci_device_add
pci_scan_single_device
Mask out the non-speed bits in PCIE_LNKCTL2_TLS2SPEED() and PCIE_LNKCAP_SLS2SPEED() so they don't incorrectly return PCI_SPEED_UNKNOWN.
Quoted 15 lines · Bjorn Helgaas
Quoted 7 lines
Jiwei Sun (2):
PCI: Fix the wrong reading of register fields
PCI: Adjust the position of reading the Link Control 2 registerdrivers/pci/pci.h | 32 +++++++++++++++++++------------- drivers/pci/quirks.c | 6 ++++-- 2 files changed, 23 insertions(+), 15 deletions(-)Sorry, this totally slipped through the cracks. I applied both of these to pci/enumeration for v6.17.
Andrew reported tripping over this issue fixed by the first patch, and Lukas also posted a similar patch [1] to fix it, so I updated the commit log as below to include details of Andrew's report.
Quoted 5 lines
As Lukas did, I added a stable tag but made it for v6.13+ (not v6.12+) because I think the actual problem showed up with
de9a6c8d5dbf("PCI/bwctrl: Addpcie_set_target_speed()to set PCIe Link Speed"), not withf68dea13405c("PCI: Revert to the original speed after PCIe failed link retraining").
Thank you for taking care of these issues while I've been distracted with
no resources available to actually try things in my lab.
Maciej