Skip to contentSkip to conversation

pciutils: Add basic decode support for CXL DVSEC

PATCH Superseded Sean V Kelley 2 participants 5 messages Apr 13, 2020 – Apr 14, 2020

Subsystems (2)

v4 of 7 Superseded Latest is v7 →

lore
Share & tools
lore
#0 lore raw

v4 cover pciutils: Add basic decode support for CXL DVSEC

Changes since v3 [1]:

- Non-functional changes, subject and typo corrections

Changes since v2 [2]:

- Separate DVSEC capabilities from CXL and provide a means to identify. (Bjorn Helgaas)

Changes since v1 [3]:

- Use consistent syntax to match other lspci capability output. (Bjorn Helgaas)

[1] https://lore.kernel.org/linux-pci/[email protected]/
[2] https://lore.kernel.org/linux-pci/[email protected]/
[3] https://lore.kernel.org/linux-pci/[email protected]/

This patch series adds support for basic lspci decode of Compute eXpress Link[4], a new CPU interconnect building upon PCIe. As a foundation for the CXL support it adds separate Designated Vendor-Specific Capability (DVSEC) defines and a cap function so as to align with PCIe r5.0, sec 7.9.6.2 terms and provide available details. It makes use of the Vendor ID so as to identify a Flex Bus capable port for purposes of CXL support.

[4] https://www.computeexpresslink.org/

Sean V Kelley (2):
pciutils: Decode available DVSEC details
pciutils: Decode Compute eXpress Link DVSEC

 lib/header.h        |  24 ++++
 ls-ecaps.c          |  73 +++++++++-
 tests/cap-dvsec     | 340 ++++++++++++++++++++++++++++++++++++++++++++
 tests/cap-dvsec-cxl | 340 ++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 776 insertions(+), 1 deletion(-)
 create mode 100644 tests/cap-dvsec
 create mode 100644 tests/cap-dvsec-cxl

--
2.26.0

#1 lore raw

PATCH v4 1/2 pciutils: Decode available DVSEC details

Instead of current generic 'unknown' output for DVSEC, decode details on Vendor ID, Rev, etc.

---
 lib/header.h    |   4 +
 ls-ecaps.c      |  25 +++-
 tests/cap-dvsec | 340 ++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 368 insertions(+), 1 deletion(-)
 create mode 100644 tests/cap-dvsec
lib/header.h +4 -0
index bfdcc80..daeddeb 100644
@@ -1042,6 +1042,10 @@
 #define PCI_EVNDR_HEADER	4	/* Vendor-Specific Header */
 #define PCI_EVNDR_REGISTERS	8	/* Vendor-Specific Registers */
 
+/* PCIe Designated Vendor-Specific Capability */
+#define PCI_DVSEC_HEADER1	4	/* Designated Vendor-Specific Header 1 */
+#define PCI_DVSEC_HEADER2	8	/* Designated Vendor-Specific Header 2 */
+
 /* Access Control Services */
 #define PCI_ACS_CAP		0x04	/* ACS Capability Register */
 #define PCI_ACS_CAP_VALID	0x0001	/* ACS Source Validation */
ls-ecaps.c +24 -1
index 0021734..b82d37e 100644
@@ -634,6 +634,29 @@ cap_rclink(struct device *d, int where)
     }
 }
 
+static void
+cap_dvsec(struct device *d, int where)
+{
+  u32 hdr;
+
+  printf("Designated Vendor-Specific:\n");
+  if (!config_fetch(d, where + PCI_DVSEC_HEADER1, 8))
+    {
+      printf("<unreadable>\n");
+      return;
+    }
+
+  hdr = get_conf_long(d, where + PCI_DVSEC_HEADER1);
+  printf("\t\tDVSEC Vendor ID=%04x Rev=%d Len=%03x <?>\n",
+    BITS(hdr, 0, 16),
+    BITS(hdr, 16, 4),
+    BITS(hdr, 20, 12));
+
+  hdr = get_conf_long(d, where + PCI_DVSEC_HEADER2);
+  printf("\t\tDVSEC ID=%04x <?>\n",
+    BITS(hdr, 0, 16));
+}
+
 static void
 cap_evendor(struct device *d, int where)
 {
@@ -924,7 +947,7 @@ show_ext_caps(struct device *d, int type)
 	    printf("Readiness Time Reporting <?>\n");
 	    break;
 	  case PCI_EXT_CAP_ID_DVSEC:
-	    printf("Designated Vendor-Specific <?>\n");
+	    cap_dvsec(d, where);
 	    break;
 	  case PCI_EXT_CAP_ID_VF_REBAR:
 	    printf("VF Resizable BAR <?>\n");
tests/cap-dvsec new file +340 -0

340 lines not on this page · raw message

Signature · 1 line
2.26.0
#2 lore raw

PATCH v4 2/2 pciutils: Decode Compute eXpress Link DVSEC

In reply to Sean V Kelley

Compute eXpress Link[1] is a new CPU interconnect created with workload accelerators in mind. The interconnect relies on PCIe electrical and physical interconnect for communication via a Flex Bus port which allows designs to choose between providing PCIe or CXL.

This patch introduces basic support for lspci decode of CXL and builds upon the existing Designated Vendor-Specific support in lspci through identification of a Flex Bus capable Vendor ID.

[1] https://www.computeexpresslink.org/

---
 lib/header.h        |  20 +++
 ls-ecaps.c          |  50 ++++++-
 tests/cap-dvsec-cxl | 340 ++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 409 insertions(+), 1 deletion(-)
 create mode 100644 tests/cap-dvsec-cxl
lib/header.h +20 -0
index daeddeb..30b4eba 100644
@@ -1045,6 +1045,26 @@
 /* PCIe Designated Vendor-Specific Capability */
 #define PCI_DVSEC_HEADER1	4	/* Designated Vendor-Specific Header 1 */
 #define PCI_DVSEC_HEADER2	8	/* Designated Vendor-Specific Header 2 */
+#define PCI_DVSEC_FLEXBUS_ID	0	/* Flex Bus feature capability ID */
+
+/* PCIe CXL Designated Vendor-Specific Capabilities, Control, Status */
+#define PCI_CXL_CAP		0x0a
+#define  PCI_CXL_CAP_CACHE	0x0001
+#define  PCI_CXL_CAP_IO		0x0002
+#define  PCI_CXL_CAP_MEM	0x0004
+#define  PCI_CXL_CAP_MEM_HWINIT	0x0008
+#define  PCI_CXL_CAP_HDM_CNT(x)	(((x) & (3 << 4)) >> 4)
+#define  PCI_CXL_CAP_VIRAL	0x4000
+#define PCI_CXL_CTRL		0x0c
+#define  PCI_CXL_CTRL_CACHE	0x0001
+#define  PCI_CXL_CTRL_IO	0x0002
+#define  PCI_CXL_CTRL_MEM	0x0004
+#define  PCI_CXL_CTRL_CACHE_SF_COV(x)	(((x) & (0x1f << 3)) >> 3)
+#define  PCI_CXL_CTRL_CACHE_SF_GRAN(x)	(((x) & (0x7 << 8)) >> 8)
+#define  PCI_CXL_CTRL_CACHE_CLN	0x0800
+#define  PCI_CXL_CTRL_VIRAL	0x4000
+#define PCI_CXL_STATUS		0x0e
+#define  PCI_CXL_STATUS_VIRAL	0x4000
 
 /* Access Control Services */
 #define PCI_ACS_CAP		0x04	/* ACS Capability Register */
ls-ecaps.c +49 -1

62 lines not on this page · raw message

tests/cap-dvsec-cxl new file +340 -0

340 lines not on this page · raw message

Signature · 1 line
2.26.0
#3 lore raw

Re: [PATCH v4 2/2] pciutils: Decode Compute eXpress Link DVSEC

Quoted 8 lines · Sean V Kelley

Compute eXpress Link[1] is a new CPU interconnect created with workload accelerators in mind. The interconnect relies on PCIe electrical and physical interconnect for communication via a Flex Bus port which allows designs to choose between providing PCIe or CXL.

This patch introduces basic support for lspci decode of CXL and builds upon the existing Designated Vendor-Specific support in lspci through identification of a Flex Bus capable Vendor ID.

I don't think this is quite right. Isn't the Flex Bus ID the "DVSEC ID" (not the "DVSEC Vendor ID")?

ls-ecaps.c cap_rclink(struct device *d, int where) · line 671

Quoted 8 lines
ls-ecaps.c +8 -0
+static int
+is_flexbus_cap(struct device *d, int where)
+{
+  u32 hdr;
+  u16 w;
+
+  if (!config_fetch(d, where + PCI_DVSEC_HEADER1, 8))
+    return 0;

And here, I think we need to check the "DVSEC Vendor ID" first, i.e., in the log below, I guess we'd look for the Intel Vendor ID (0x8086).

We can only decode the capability if *both* the DVSEC Vendor ID (HEADER1) and the DVSEC ID (HEADER2) match.

Quoted 36 lines
ls-ecaps.c +23 -1
+  /* Check for Designated Vendor-Specific Flex Bus Capable ID */
+  hdr = get_conf_long(d, where + PCI_DVSEC_HEADER2);
+  w = BITS(hdr, 0, 16);
+  if (w == PCI_DVSEC_FLEXBUS_ID)
+    return 1;
+
+  return 0;
+}
+
 static void
 cap_dvsec(struct device *d, int where)
 {
@@ -947,7 +992,10 @@ show_ext_caps(struct device *d, int type)
 	    printf("Readiness Time Reporting <?>\n");
 	    break;
 	  case PCI_EXT_CAP_ID_DVSEC:
-	    cap_dvsec(d, where);
+	    if (is_flexbus_cap(d, where))
+	      cap_cxl(d, where);
+	    else
+	      cap_dvsec(d, where);
 	    break;
 	  case PCI_EXT_CAP_ID_VF_REBAR:
 	    printf("VF Resizable BAR <?>\n");
 
+        Capabilities: [e00 v1] CXL Designated Vendor-Specific:
+                CXLCap: Cache+ IO+ Mem+ Mem HW Init+ HDMCount 1 Viral-
+                CXLCtl: Cache+ IO+ Mem- Cache SF Cov 0 Cache SF Gran 0 Cache Clean- Viral-
+                CXLSta: Viral-
+        Capabilities: [e38 v1] Device Serial Number 12-34-56-78-90-00-00-00
 
+e00: 23 00 81 e3 86 80 80 03 00 00 1f 00 03 00 00 00
+e10: 00 00 00 00 00 00 00 00 00 00 00 00 03 01 00 08
+e20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
+e30: 00 00 00 00 00 00 00 00 03 00 01 00 00 00 00 90
+e40: 78 56 34 12 00 00 00 00 00 00 00 00 00 00 00 00
#4 lore raw

Re: [PATCH v4 2/2] pciutils: Decode Compute eXpress Link DVSEC

Quoted 6 lines · Bjorn Helgaas
Quoted 5 lines · Sean V Kelley

Compute eXpress Link[1] is a new CPU interconnect created with workload accelerators in mind. The interconnect relies on PCIe electrical and physical interconnect for communication via a Flex Bus port which allows designs to choose between providing PCIe or CXL.

Quoted 6 lines
Quoted 3 lines

This patch introduces basic support for lspci decode of CXL and builds upon the existing Designated Vendor-Specific support in lspci through identification of a Flex Bus capable Vendor ID.

I don't think this is quite right. Isn't the Flex Bus ID the "DVSEC ID" (not the "DVSEC Vendor ID")?

Correct. It is not really a Flex Bus ID but really a DVSEC ID. I was leaning too heavily on the Spec’s wording which stated: “The DVSEC ID is set to 0x0 to advertise that this is an Flex Bus feature capability structure.”

I’ll correct. I was trying to note that CXL depends on Flex Bus capability which is identified through the DVSEC ID. But I confused things by also mentioning the Vendor ID.

Quoted 12 lines

ls-ecaps.c cap_rclink(struct device *d, int where) · line 671

Quoted 9 lines
ls-ecaps.c +8 -0
+static int
+is_flexbus_cap(struct device *d, int where)
+{
+  u32 hdr;
+  u16 w;
+
+  if (!config_fetch(d, where + PCI_DVSEC_HEADER1, 8))
+    return 0;

And here, I think we need to check the "DVSEC Vendor ID" first, i.e., in the log below, I guess we'd look for the Intel Vendor ID (0x8086).

Agree. Will do.

Quoted 3 lines

We can only decode the capability if *both* the DVSEC Vendor ID (HEADER1) and the DVSEC ID (HEADER2) match.

Got it.

Thanks,

Sean

Quoted 40 lines
Quoted 40 lines
ls-ecaps.c +15 -1
+  /* Check for Designated Vendor-Specific Flex Bus Capable ID */
+  hdr = get_conf_long(d, where + PCI_DVSEC_HEADER2);
+  w = BITS(hdr, 0, 16);
+  if (w == PCI_DVSEC_FLEXBUS_ID)
+    return 1;
+
+  return 0;
+}
+
 static void
 cap_dvsec(struct device *d, int where)
 {
@@ -947,7 +992,10 @@ show_ext_caps(struct device *d, int type)
 	    printf("Readiness Time Reporting <?>\n");
 	    break;
 	  case PCI_EXT_CAP_ID_DVSEC:
-	    cap_dvsec(d, where);
+	    if (is_flexbus_cap(d, where))
+	      cap_cxl(d, where);
+	    else
+	      cap_dvsec(d, where);
 	    break;
 	  case PCI_EXT_CAP_ID_VF_REBAR:
 	    printf("VF Resizable BAR <?>\n");
 
+        Capabilities: [e00 v1] CXL Designated Vendor-Specific:
+                CXLCap: Cache+ IO+ Mem+ Mem HW Init+ HDMCount 1 

Viral-
+ CXLCtl: Cache+ IO+ Mem- Cache SF Cov 0 Cache SF Gran
0 Cache Clean- Viral-

(quoted diff) +2 -0
+                CXLSta: Viral-
+        Capabilities: [e38 v1] Device Serial Number 

12-34-56-78-90-00-00-00

tests/cap-dvsec-cxl new file +5 -0
+e00: 23 00 81 e3 86 80 80 03 00 00 1f 00 03 00 00 00
+e10: 00 00 00 00 00 00 00 00 00 00 00 00 03 01 00 08
+e20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
+e30: 00 00 00 00 00 00 00 00 03 00 01 00 00 00 00 90
+e40: 78 56 34 12 00 00 00 00 00 00 00 00 00 00 00 00