Skip to contentSkip to conversation

lspci: Don't report PCIe link downgrades for downstream ports

PATCH In review Bjorn Helgaas 4 participants 6 messages Mar 18, 2021 – Jan 21, 2022 PCI

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 lspci: Don't report PCIe link downgrades for downstream ports

From: Bjorn Helgaas <[email protected]>

After b47b5bd408e1 ("lspci: Report if the PCIe link speed/width is full or downgraded"), we report "downgraded" or "ok" for PCIe links operating at lower speed or width than they're capable of:

LnkCap: Port #1, Speed 8GT/s, Width x1, ASPM L1, Exit Latency L1 <16us
LnkSta: Speed 5GT/s (downgraded), Width x1 (ok)

Previously we did this for both ends of the link, but I don't think it's very useful for Downstream Ports (the upstream end of the link) because we claim the link is downgraded even if (1) there's no device on the other end or (2) the other device doesn't support anything faster/wider.

Drop the "downgraded" reporting for Downstream Ports. If there is a device below, we'll still complain at that end if it supports a faster/wider link than is available.

---
 ls-caps.c | 17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)
ls-caps.c +10 -7
index db56556..dd17c6b 100644
@@ -758,13 +758,16 @@ static char *link_speed(int speed)
     }
 }
 
-static char *link_compare(int sta, int cap)
+static char *link_compare(int type, int sta, int cap)
 {
+  if ((type == PCI_EXP_TYPE_ROOT_PORT) || (type == PCI_EXP_TYPE_DOWNSTREAM) ||
+      (type == PCI_EXP_TYPE_PCIE_BRIDGE))
+    return "";
   if (sta < cap)
-    return "downgraded";
+    return " (downgraded)";
   if (sta > cap)
-    return "strange";
-  return "ok";
+    return " (strange)";
+  return " (ok)";
 }
 
 static char *aspm_support(int code)
@@ -837,11 +840,11 @@ static void cap_express_link(struct device *d, int where, int type)
   w = get_conf_word(d, where + PCI_EXP_LNKSTA);
   sta_speed = w & PCI_EXP_LNKSTA_SPEED;
   sta_width = (w & PCI_EXP_LNKSTA_WIDTH) >> 4;
-  printf("\t\tLnkSta:\tSpeed %s (%s), Width x%d (%s)\n",
+  printf("\t\tLnkSta:\tSpeed %s%s, Width x%d%s\n",
 	link_speed(sta_speed),
-	link_compare(sta_speed, cap_speed),
+	link_compare(type, sta_speed, cap_speed),
 	sta_width,
-	link_compare(sta_width, cap_width));
+	link_compare(type, sta_width, cap_width));
   printf("\t\t\tTrErr%c Train%c SlotClk%c DLActive%c BWMgmt%c ABWMgmt%c\n",
 	FLAG(w, PCI_EXP_LNKSTA_TR_ERR),
 	FLAG(w, PCI_EXP_LNKSTA_TRAIN),
Signature · 1 line
2.25.1
#1 lore raw
Quoted 35 lines · Bjorn Helgaas

From: Bjorn Helgaas <[email protected]>

After b47b5bd408e1 ("lspci: Report if the PCIe link speed/width is full or downgraded"), we report "downgraded" or "ok" for PCIe links operating at lower speed or width than they're capable of:

LnkCap: Port #1, Speed 8GT/s, Width x1, ASPM L1, Exit Latency L1 <16us
LnkSta: Speed 5GT/s (downgraded), Width x1 (ok)

Previously we did this for both ends of the link, but I don't think it's very useful for Downstream Ports (the upstream end of the link) because we claim the link is downgraded even if (1) there's no device on the other end or (2) the other device doesn't support anything faster/wider.

Drop the "downgraded" reporting for Downstream Ports. If there is a device below, we'll still complain at that end if it supports a faster/wider link than is available.

---
 ls-caps.c | 17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)
ls-caps.c +2 -1
index db56556..dd17c6b 100644
@@ -758,13 +758,16 @@ static char *link_speed(int speed)
     }
 }
 
-static char *link_compare(int sta, int cap)
+static char *link_compare(int type, int sta, int cap)
 {
+  if ((type == PCI_EXP_TYPE_ROOT_PORT) || (type == PCI_EXP_TYPE_DOWNSTREAM) ||

ls-caps.c static char *link_speed(int speed) · line 770

Quoted 10 lines
ls-caps.c +5 -3
+      (type == PCI_EXP_TYPE_PCIE_BRIDGE))
+    return "";
   if (sta < cap)
-    return "downgraded";
+    return " (downgraded)";
   if (sta > cap)
-    return "strange";
-  return "ok";
+    return " (strange)";
+  return " (ok)";

Personally I would probably just return "" by default and drop the "(ok)" notes because they don't really seem necessary. But either way is OK.

Quoted 21 lines
ls-caps.c +3 -3
 }
 
 static char *aspm_support(int code)
@@ -837,11 +840,11 @@ static void cap_express_link(struct device *d, int where, int type)
   w = get_conf_word(d, where + PCI_EXP_LNKSTA);
   sta_speed = w & PCI_EXP_LNKSTA_SPEED;
   sta_width = (w & PCI_EXP_LNKSTA_WIDTH) >> 4;
-  printf("\t\tLnkSta:\tSpeed %s (%s), Width x%d (%s)\n",
+  printf("\t\tLnkSta:\tSpeed %s%s, Width x%d%s\n",
 	link_speed(sta_speed),
-	link_compare(sta_speed, cap_speed),
+	link_compare(type, sta_speed, cap_speed),
 	sta_width,
-	link_compare(sta_width, cap_width));
+	link_compare(type, sta_width, cap_width));
   printf("\t\t\tTrErr%c Train%c SlotClk%c DLActive%c BWMgmt%c ABWMgmt%c\n",
 	FLAG(w, PCI_EXP_LNKSTA_TR_ERR),
 	FLAG(w, PCI_EXP_LNKSTA_TRAIN),
Signature · 1 line
2.25.1
#2 lore raw
In reply to Bjorn Helgaas
Quoted 3 lines · Bjorn Helgaas

Drop the "downgraded" reporting for Downstream Ports. If there is a device below, we'll still complain at that end if it supports a faster/wider link than is available.

This makes sense, but I think we should still report if training has gone horribly wrong. Maybe something like this ...

Quoted 10 lines

+++ b/ls-caps.c

ls-caps.c +3 -1
@@ -758,13 +758,16 @@ static char *link_speed(int speed)
     }
 }
 
-static char *link_compare(int sta, int cap)
+static char *link_compare(int type, int sta, int cap)
 {
+  if ((type == PCI_EXP_TYPE_ROOT_PORT) || (type == PCI_EXP_TYPE_DOWNSTREAM) ||
+      (type == PCI_EXP_TYPE_PCIE_BRIDGE))

ls-caps.c static char *link_speed(int speed) · line 771

Quoted 10 lines
ls-caps.c +4 -3
+    return "";
   if (sta < cap)
-    return "downgraded";
+    return " (downgraded)";
   if (sta > cap)
-    return "strange";
-  return "ok";
+    return " (strange)";
+  return " (ok)";
 }

{
if (sta > cap)

    return " (overdriven)";
  if (sta == cap)
    return "";
  if ((type == PCI_EXP_TYPE_ROOT_PORT) ||
      ((type == PCI_EXP_TYPE_DOWNSTREAM) ||
      ((type == PCI_EXP_TYPE_PCIE_BRIDGE))
    return "";

return " (downgraded)";
}

(i don't know if the PCIe spec has a better word than overdriven for this situation, but i don't like "strange". "invalid", maybe?)

The reason i say we should report it on the downstream port is that we probably can't get to the config data on the upstream port/device, so this may be our best chance to find out what's wrong.

#3 lore raw

Hello!

This fell through the cracks in my mailbox, but I finally assembled pieces and here is what I applied.

Martin

commit 9f7681202fcfaefd02e202eb64c01eb9e962729d

Author: Martin Mares <[email protected]>
Date:   Fri Jan 21 14:16:37 2022 +0100

lspci: Improvements to PCIe link speed downgrade reporting

Do not report PCIe link downgrades for downstream ports.

Changed wording so that "overdriven" is reported instead of "strange" for speeds greater than the maximum supported one.

Also report nothing instead of "ok".

Inspired by patches by Bjorn Helgaas and Matthew Wilcox.

ls-caps.c +11 -8
index 91acb59..79b61cd 100644
@@ -771,13 +771,16 @@ static char *link_speed(int speed)
     }
 }
 
-static char *link_compare(int sta, int cap)
+static char *link_compare(int type, int sta, int cap)
 {
-  if (sta < cap)
-    return "downgraded";
   if (sta > cap)
-    return "strange";
-  return "ok";
+    return " (overdriven)";
+  if (sta == cap)
+    return "";
+  if ((type == PCI_EXP_TYPE_ROOT_PORT) || (type == PCI_EXP_TYPE_DOWNSTREAM) ||
+      (type == PCI_EXP_TYPE_PCIE_BRIDGE))
+    return "";
+  return " (downgraded)";
 }
 
 static char *aspm_support(int code)
@@ -850,11 +853,11 @@ static void cap_express_link(struct device *d, int where, int type)
   w = get_conf_word(d, where + PCI_EXP_LNKSTA);
   sta_speed = w & PCI_EXP_LNKSTA_SPEED;
   sta_width = (w & PCI_EXP_LNKSTA_WIDTH) >> 4;
-  printf("\t\tLnkSta:\tSpeed %s (%s), Width x%d (%s)\n",
+  printf("\t\tLnkSta:\tSpeed %s%s, Width x%d%s\n",
 	link_speed(sta_speed),
-	link_compare(sta_speed, cap_speed),
+	link_compare(type, sta_speed, cap_speed),
 	sta_width,
-	link_compare(sta_width, cap_width));
+	link_compare(type, sta_width, cap_width));
   printf("\t\t\tTrErr%c Train%c SlotClk%c DLActive%c BWMgmt%c ABWMgmt%c\n",
 	FLAG(w, PCI_EXP_LNKSTA_TR_ERR),
 	FLAG(w, PCI_EXP_LNKSTA_TRAIN),
#4 lore raw
In reply to Bjorn Helgaas
Quoted 6 lines · On Thursday 18 March 2021 12:02:44 Bjorn Helgaas wrote:

From: Bjorn Helgaas <[email protected]>

After b47b5bd408e1 ("lspci: Report if the PCIe link speed/width is full or downgraded"), we report "downgraded" or "ok" for PCIe links operating at lower speed or width than they're capable of:

Quoted 7 lines

LnkCap: Port #1, Speed 8GT/s, Width x1, ASPM L1, Exit Latency L1 <16us
LnkSta: Speed 5GT/s (downgraded), Width x1 (ok)

Previously we did this for both ends of the link, but I don't think it's very useful for Downstream Ports (the upstream end of the link) because we claim the link is downgraded even if (1) there's no device on the other end or (2) the other device doesn't support anything faster/wider.

If there is no device under Root or Downstream port then PCIe Bridge reports initial speed (2.5 GT/s) independently of the maximal speed. So I agree that we should not report "downgraded" state in this case as the link is not technically downgraded. For hotplugging PCIe Bridge it is just waiting until some PCIe card is inserted back.

Different thing is when you have Gen2 PCIe Bridge which can operate at 5 GT/s but you connect only Gen1 PCIe card under it (which operates only at 2.5 GT/s).

In this case Root or Downstream port of PCIe Bridge is running at lower "downgraded" speed but card (on the upstream end of the link) is running at maximal speed (not downgraded).

So in this case proposed patch does not report "downgraded" state neither on Root/Downstream Bridge part nor on card part.

Is it correct? Should not lspci report in this case _somewhere_ that link is downgraded and is not running at the full speed?

I can image even more complicated PCIe topology with two PCIe packet switches and one WiFi card where information about "downgraded" state with proposed change would not be reported too:

     CPU / Host
         |
         |
    Root 8 GT/s --> Upstream 8 GT/s
                          |
                          |
                          v
                  Downstream 5 GT/s --> Upstream 5 GT/s
                                               |
                                               |
                                               v
                                   Downstream 2.5 GT/s --> WiFi card 2.5 GT/s

If we do not report "downgraded" state in Downstream or Root part then in this topology no node would be marked as "downgraded" even when Host or Root Bridge is capable of 8 GT/s and WiFi card is slow and its maximal speed is just 2.5 GT/s.

I'm not sure but I think it could be possible to create somehow very complicated topology with more PCIe to PCI Bridges and PCI to PCIe bridges where information about "downgraded" state is lost when lspci would not report "downgraded" state for Downstream parts of the link.

Quoted 52 lines

Drop the "downgraded" reporting for Downstream Ports. If there is a device below, we'll still complain at that end if it supports a faster/wider link than is available.

---
 ls-caps.c | 17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)
ls-caps.c +10 -7
index db56556..dd17c6b 100644
@@ -758,13 +758,16 @@ static char *link_speed(int speed)
     }
 }
 
-static char *link_compare(int sta, int cap)
+static char *link_compare(int type, int sta, int cap)
 {
+  if ((type == PCI_EXP_TYPE_ROOT_PORT) || (type == PCI_EXP_TYPE_DOWNSTREAM) ||
+      (type == PCI_EXP_TYPE_PCIE_BRIDGE))
+    return "";
   if (sta < cap)
-    return "downgraded";
+    return " (downgraded)";
   if (sta > cap)
-    return "strange";
-  return "ok";
+    return " (strange)";
+  return " (ok)";
 }
 
 static char *aspm_support(int code)
@@ -837,11 +840,11 @@ static void cap_express_link(struct device *d, int where, int type)
   w = get_conf_word(d, where + PCI_EXP_LNKSTA);
   sta_speed = w & PCI_EXP_LNKSTA_SPEED;
   sta_width = (w & PCI_EXP_LNKSTA_WIDTH) >> 4;
-  printf("\t\tLnkSta:\tSpeed %s (%s), Width x%d (%s)\n",
+  printf("\t\tLnkSta:\tSpeed %s%s, Width x%d%s\n",
 	link_speed(sta_speed),
-	link_compare(sta_speed, cap_speed),
+	link_compare(type, sta_speed, cap_speed),
 	sta_width,
-	link_compare(sta_width, cap_width));
+	link_compare(type, sta_width, cap_width));
   printf("\t\t\tTrErr%c Train%c SlotClk%c DLActive%c BWMgmt%c ABWMgmt%c\n",
 	FLAG(w, PCI_EXP_LNKSTA_TR_ERR),
 	FLAG(w, PCI_EXP_LNKSTA_TRAIN),
Signature · 1 line
2.25.1
#5 lore raw
Quoted 8 lines · Pali Rohár

Different thing is when you have Gen2 PCIe Bridge which can operate at 5 GT/s but you connect only Gen1 PCIe card under it (which operates only at 2.5 GT/s).

In this case Root or Downstream port of PCIe Bridge is running at lower "downgraded" speed but card (on the upstream end of the link) is running at maximal speed (not downgraded).

Quoted 5 lines

So in this case proposed patch does not report "downgraded" state neither on Root/Downstream Bridge part nor on card part.

Is it correct? Should not lspci report in this case _somewhere_ that link is downgraded and is not running at the full speed?

I suppose it depends why you think it's important that "downgraded" is reported. If you want it to mean "you've plugged a slow card into a port that's capable of more", then you're right. If you want it to mean "autonegotiation went wrong", then you actually don't want downgraded to be reported in the scenarios you outlined.

Dmitry? What did you /want/ it to mean?