pciutils: Add basic decode support for CXL DVSEC
Subsystems (2)
v5 cover pciutils: Add basic decode support for CXL DVSEC
Changes since v4 [1]:
- Describe CXL in terms of both the DVSEC Vendor ID and DVSEC ID. (Bjorn Helgaas)
- Check the DVSEC Vendor ID first. Both the DVSEC Vendor ID and DVSEC ID must match for CXL identification.
(Bjorn Helgaas)
- Renamed is_flexbus_cap() to is_cxl_cap() to more accurately reflect use.
- Added descriptions to the CXL support/control/status register bitfields.
[1] https://lore.kernel.org/linux-pci/[email protected]/
This patch series adds support for basic lspci decode of Compute eXpress Link[2], a new CPU interconnect building upon PCIe. As a foundation for the CXL support it adds separate Designated Vendor-Specific Capability (DVSEC) defines and a cap function so as to align with PCIe r5.0, sec 7.9.6.2 terms and provide available details. It makes use of the DVSEC Vendor ID and DVSEC ID so as to identify a CXL capable device.
[2] https://www.computeexpresslink.org/
Sean V Kelley (2):
pciutils: Decode available DVSEC details
pciutils: Decode Compute eXpress Link DVSEC
lib/header.h | 24 ++++ ls-ecaps.c | 79 +++++++++- tests/cap-dvsec | 340 ++++++++++++++++++++++++++++++++++++++++++++ tests/cap-dvsec-cxl | 340 ++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 782 insertions(+), 1 deletion(-) create mode 100644 tests/cap-dvsec create mode 100644 tests/cap-dvsec-cxl
--
2.26.0
PATCH v5 1/2 pciutils: Decode available DVSEC details
Instead of current generic 'unknown' output for DVSEC, decode details on Vendor ID, Rev, etc.
--- lib/header.h | 4 + ls-ecaps.c | 25 +++- tests/cap-dvsec | 340 ++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 368 insertions(+), 1 deletion(-) create mode 100644 tests/cap-dvsec
lib/header.h +4 -0
@@ -1042,6 +1042,10 @@#define PCI_EVNDR_HEADER 4/* Vendor-Specific Header */ #define PCI_EVNDR_REGISTERS 8/* Vendor-Specific Registers */ +/* PCIe Designated Vendor-Specific Capability */ +#define PCI_DVSEC_HEADER1 4/* Designated Vendor-Specific Header 1 */ +#define PCI_DVSEC_HEADER2 8/* Designated Vendor-Specific Header 2 */ +/* Access Control Services */ #define PCI_ACS_CAP 0x04/* ACS Capability Register */ #define PCI_ACS_CAP_VALID 0x0001/* ACS Source Validation */
ls-ecaps.c +24 -1
@@ -634,6 +634,29 @@ cap_rclink(struct device *d, int where) } } +static void +cap_dvsec (struct device *d,int where) +{ +u32 hdr; + +printf ("Designated Vendor-Specific:\n" ); +if (!config_fetch (d, where +PCI_DVSEC_HEADER1 ,8 )) + { +printf ("<unreadable>\n" ); +return ; + } + + hdr =get_conf_long (d, where +PCI_DVSEC_HEADER1 ); +printf ("\t\tDVSEC Vendor ID=%04x Rev=%d Len=%03x <?>\n" , +BITS (hdr,0 ,16 ), +BITS (hdr,16 ,4 ), +BITS (hdr,20 ,12 )); + + hdr =get_conf_long (d, where +PCI_DVSEC_HEADER2 ); +printf ("\t\tDVSEC ID=%04x <?>\n" , +BITS (hdr,0 ,16 )); +} +static void cap_evendor (struct device *d,int where) {
@@ -924,7 +947,7 @@ show_ext_caps(struct device *d, int type)printf ("Readiness Time Reporting <?>\n" );break ;case PCI_EXT_CAP_ID_DVSEC : -printf ("Designated Vendor-Specific <?>\n" ); +cap_dvsec (d, where);break ;case PCI_EXT_CAP_ID_VF_REBAR :printf ("VF Resizable BAR <?>\n" );
tests/cap-dvsec new file +340 -0
340 lines not on this page · raw message
Signature · 1 line
2.26.0
PATCH v5 2/2 pciutils: Decode Compute eXpress Link DVSEC
In reply to Sean V KelleyCompute eXpress Link[1] is a new CPU interconnect created with workload accelerators in mind. The interconnect relies on PCIe electrical and physical interconnect for communication via a Flex Bus port which allows designs to choose between providing PCIe or CXL.
This patch introduces basic support for lspci decode of CXL and builds upon the existing Designated Vendor-Specific support in lspci through identification of a supporting CXL device using DVSEC Vendor ID and DVSEC ID.
[1] https://www.computeexpresslink.org/
--- lib/header.h | 20 +++ ls-ecaps.c | 56 +++++++- tests/cap-dvsec-cxl | 340 ++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 415 insertions(+), 1 deletion(-) create mode 100644 tests/cap-dvsec-cxl
lib/header.h +20 -0
@@ -1045,6 +1045,26 @@/* PCIe Designated Vendor-Specific Capability */ #define PCI_DVSEC_HEADER1 4/* Designated Vendor-Specific Header 1 */ #define PCI_DVSEC_HEADER2 8/* Designated Vendor-Specific Header 2 */ +#define PCI_DVSEC_ID 0/* Designated Vendor-Specific ID */ + +/* PCIe CXL Designated Vendor-Specific Capabilities, Control, Status */ +#define PCI_CXL_CAP 0x0a/* CXL Capability Register */ +#define PCI_CXL_CAP_CACHE 0x0001/* CXL.cache Protocol Support */ +#define PCI_CXL_CAP_IO 0x0002/* CXL.io Protocol Support */ +#define PCI_CXL_CAP_MEM 0x0004/* CXL.mem Protocol Support */ +#define PCI_CXL_CAP_MEM_HWINIT 0x0008/* CXL.mem Initalizes with HW/FW Support */ +#define PCI_CXL_CAP_HDM_CNT (x) (((x) & (3 << 4)) >> 4)/* CXL Number of HDM ranges */ +#define PCI_CXL_CAP_VIRAL 0x4000/* CXL Viral Handling Support */ +#define PCI_CXL_CTRL 0x0c/* CXL Control Register */ +#define PCI_CXL_CTRL_CACHE 0x0001/* CXL.cache Protocol Enable */ +#define PCI_CXL_CTRL_IO 0x0002/* CXL.io Protocol Enable */ +#define PCI_CXL_CTRL_MEM 0x0004/* CXL.mem Protocol Enable */ +#define PCI_CXL_CTRL_CACHE_SF_COV (x) (((x) & (0x1f << 3)) >> 3)/* Snoop Filter Coverage */ +#define PCI_CXL_CTRL_CACHE_SF_GRAN (x) (((x) & (0x7 << 8)) >> 8)/* Snoop Filter Granularity */ +#define PCI_CXL_CTRL_CACHE_CLN 0x0800/* CXL.cache Performance Hint on Clean Evictions */ +#define PCI_CXL_CTRL_VIRAL 0x4000/* CXL Viral Handling Enable */ +#define PCI_CXL_STATUS 0x0e/* CXL Status Register */ +#define PCI_CXL_STATUS_VIRAL 0x4000/* CXL Viral Handling Status */ /* Access Control Services */ #define PCI_ACS_CAP 0x04/* ACS Capability Register */
ls-ecaps.c +55 -1
68 lines not on this page · raw message
tests/cap-dvsec-cxl new file +340 -0
340 lines not on this page · raw message
Signature · 1 line
2.26.0
Re: [PATCH v5 2/2] pciutils: Decode Compute eXpress Link DVSEC
Quoted 11 lines · Sean V Kelley
[1] https://www.computeexpresslink.org/
Signed-off-by Sean V Kelley <[email protected]>--- lib/header.h | 20 +++ls-ecaps.c +4
-0+ +static int +is_cxl_cap (struct device *d,int where) +{
ls-ecaps.c cap_rclink(struct device *d, int where) · line 676
Quoted 10 lines
ls-ecaps.c +10
-0+u32 hdr; +u16 w; + +if (!config_fetch (d, where +PCI_DVSEC_HEADER1 ,8 )) +return 0 ; + +/* Check for supported Vendor */ + hdr =get_conf_long (d, where +PCI_DVSEC_HEADER1 ); + w =BITS (hdr,0 ,16 ); +if (w !=PCI_VENDOR_ID_INTEL )
I don't think here checking is quite right. Does only Intel support CXL? Other Vendors should also be considered.
Thanks
Quoted 34 lines
ls-ecaps.c +15
-1+return 0 ; + +/* Check for Designated Vendor-Specific ID */ + hdr =get_conf_long (d, where +PCI_DVSEC_HEADER2 ); + w =BITS (hdr,0 ,16 ); +if (w ==PCI_DVSEC_ID ) +return 1 ; + +return 0 ; +} +static void cap_dvsec (struct device *d,int where) {@@ -947,7 +998,10 @@ show_ext_caps(struct device *d, int type)printf ("Readiness Time Reporting <?>\n" );break ;case PCI_EXT_CAP_ID_DVSEC : -cap_dvsec (d, where); +if (is_cxl_cap (d, where)) +cap_cxl (d, where); +else +cap_dvsec (d, where);break ;case PCI_EXT_CAP_ID_VF_REBAR :printf ("VF Resizable BAR <?>\n" );tests/cap-dvsec-cxl new file +2
-0@@ -0,0 +1,340 @@ +6b:00.0 Unassigned class [ff00]: Intel Corporation Device 0d93 + Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
Re: [PATCH v5 2/2] pciutils: Decode Compute eXpress Link DVSEC
Hi Jay,
Thank you for reviewing. My comments below:
Quoted 9 lines · Jay Fang
Quoted 6 lines · Sean V Kelley
[1] https://www.computeexpresslink.org/
Signed-off-by Sean V Kelley <[email protected]>--- lib/header.h | 20 +++
Quoted 16 lines
Quoted 4 lines
ls-ecaps.c +4
-0+ +static int +is_cxl_cap (struct device *d,int where) +{ls-ecaps.c cap_rclink(struct device *d, int where) · line 676
Quoted 10 lines
ls-ecaps.c +10
-0+u32 hdr; +u16 w; + +if (!config_fetch (d, where +PCI_DVSEC_HEADER1 ,8 )) +return 0 ; + +/* Check for supported Vendor */ + hdr =get_conf_long (d, where +PCI_DVSEC_HEADER1 ); + w =BITS (hdr,0 ,16 ); +if (w !=PCI_VENDOR_ID_INTEL )I don't think here checking is quite right. Does only Intel support CXL? Other Vendors should also be considered.
In the absence of currently available hardware, I was attempting to limit false positive noise. I’m happy to avoid the check on the vendor if there were to exist a definitive supported list. Bjorn suggested that I check for vendor ID with DVSEC ID for now. As hardware enters the market, I can surely revise this with an update when the CXL group publishes a list.
Best regards,
Sean
Quoted 37 lines
Thanks
Quoted 35 lines
ls-ecaps.c +15
-1+return 0 ; + +/* Check for Designated Vendor-Specific ID */ + hdr =get_conf_long (d, where +PCI_DVSEC_HEADER2 ); + w =BITS (hdr,0 ,16 ); +if (w ==PCI_DVSEC_ID ) +return 1 ; + +return 0 ; +} +static void cap_dvsec (struct device *d,int where) {@@ -947,7 +998,10 @@ show_ext_caps(struct device *d, int type)printf ("Readiness Time Reporting <?>\n" );break ;case PCI_EXT_CAP_ID_DVSEC : -cap_dvsec (d, where); +if (is_cxl_cap (d, where)) +cap_cxl (d, where); +else +cap_dvsec (d, where);break ;case PCI_EXT_CAP_ID_VF_REBAR :printf ("VF Resizable BAR <?>\n" );tests/cap-dvsec-cxl new file +2
-0@@ -0,0 +1,340 @@ +6b:00.0 Unassigned class [ff00]: Intel Corporation Device 0d93 + Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop-ParErr+ Stepping- SERR+ FastB2B- DisINTx-
Re: [PATCH v5 2/2] pciutils: Decode Compute eXpress Link DVSEC
Hi Jay, thanks for taking a look at this!
Quoted 27 lines · Sean V Kelley
Quoted 17 lines · Jay Fang
Quoted 16 lines · Sean V Kelley
[1] https://www.computeexpresslink.org/
Signed-off-by Sean V Kelley <[email protected]>--- lib/header.h | 20 +++ls-ecaps.c +9
-0+ +static int +is_cxl_cap (struct device *d,int where) +{ +u32 hdr; +u16 w; + +if (!config_fetch (d, where +PCI_DVSEC_HEADER1 ,8 )) +return 0 ;Quoted 8 lines
ls-ecaps.c cap_rclink(struct device *d, int where) · line 676
Quoted 5 lines
ls-ecaps.c +5
-0+ +/* Check for supported Vendor */ + hdr =get_conf_long (d, where +PCI_DVSEC_HEADER1 ); + w =BITS (hdr,0 ,16 ); +if (w !=PCI_VENDOR_ID_INTEL )I don't think here checking is quite right. Does only Intel support CXL? Other Vendors should also be considered.
Quoted 5 lines
In the absence of currently available hardware, I was attempting to limit false positive noise. I’m happy to avoid the check on the vendor if there were to exist a definitive supported list. Bjorn suggested that I check for vendor ID with DVSEC ID for now. As hardware enters the market, I can surely revise this with an update when the CXL group publishes a list.
The Vendor ID check cannot be avoided. Vendor IDs are allocated by the PCI-SIG, but the DVSEC ID is vendor-defined so there cannot be a global "CXL capability" DVSEC ID.
CXL *could* work through the PCI-SIG and define a new CXL Extended Capability that could make this generic. But apparently they've chosen to use the DVSEC mechanism instead.
Quoted 4 lines
Quoted 4 lines
ls-ecaps.c cap_rclink(struct device *d, int where) · line 682
Quoted 4 lines
ls-ecaps.c +4
-0+/* Check for Designated Vendor-Specific ID */ + hdr =get_conf_long (d, where +PCI_DVSEC_HEADER2 ); + w =BITS (hdr,0 ,16 ); +if (w ==PCI_DVSEC_ID )
However, this is slightly wrong. The name "PCI_DVSEC_ID" implies that there can only be one DVSEC ID and it is vendor-independent, but neither is true.
This value is allocated by Intel, so we must check for the Intel Vendor ID first. And Intel may define several DVSEC capabilities for different purposes, so the name should also mention CXL.
So this should be "PCI_DVSEC_INTEL_CXL" or something similar.
But that doesn't mean other vendors need to define their own DVSEC IDs for CXL. The spec (PCIe r5.0, sec 7.9.6) says:
[The DVSEC Capability] allows PCI Express component vendors to use the Extended Capability mechanism to expose vendor-specific registers that can be present in components by a variety of vendors.
Any vendor that implements this CXL DVSEC the same way Intel does can tag it with PCI_VENDOR_ID_INTEL and PCI_DVSEC_INTEL_CXL.
Note that there are two types of vendor-specific capabilities:
1) Vendor-Specific Extended Capability (VSEC). The structure and definition are defined by the *Function* Vendor ID (from offset 0 in config space) and the VSEC ID in the capability.
2) Designated Vendor-Specific Extended Capability (DVSEC). The structure and definition are defined by the *DVSEC* Vendor ID (from the DVSEC capability, *not* the vendor of the Function) and the DVSEC ID in the capability.
This CXL capability is the latter, of course.
Bjorn
Re: [PATCH v5 2/2] pciutils: Decode Compute eXpress Link DVSEC
Quoted 62 lines · Bjorn Helgaas
Hi Jay, thanks for taking a look at this!
Quoted 27 lines · Sean V Kelley
Quoted 17 lines · Jay Fang
Quoted 16 lines · Sean V Kelley
[1] https://www.computeexpresslink.org/
Signed-off-by Sean V Kelley <[email protected]>--- lib/header.h | 20 +++ls-ecaps.c +9
-0+ +static int +is_cxl_cap (struct device *d,int where) +{ +u32 hdr; +u16 w; + +if (!config_fetch (d, where +PCI_DVSEC_HEADER1 ,8 )) +return 0 ;Quoted 8 lines
ls-ecaps.c cap_rclink(struct device *d, int where) · line 676
Quoted 5 lines
ls-ecaps.c +5
-0+ +/* Check for supported Vendor */ + hdr =get_conf_long (d, where +PCI_DVSEC_HEADER1 ); + w =BITS (hdr,0 ,16 ); +if (w !=PCI_VENDOR_ID_INTEL )I don't think here checking is quite right. Does only Intel support CXL? Other Vendors should also be considered.
Quoted 5 lines
In the absence of currently available hardware, I was attempting to limit false positive noise. I’m happy to avoid the check on the vendor if there were to exist a definitive supported list. Bjorn suggested that I check for vendor ID with DVSEC ID for now. As hardware enters the market, I can surely revise this with an update when the CXL group publishes a list.
The Vendor ID check cannot be avoided. Vendor IDs are allocated by the PCI-SIG, but the DVSEC ID is vendor-defined so there cannot be a global "CXL capability" DVSEC ID.
CXL *could* work through the PCI-SIG and define a new CXL Extended Capability that could make this generic. But apparently they've chosen to use the DVSEC mechanism instead.
Quoted 4 lines
Quoted 4 lines
ls-ecaps.c cap_rclink(struct device *d, int where) · line 682
Quoted 4 lines
ls-ecaps.c +4
-0+/* Check for Designated Vendor-Specific ID */ + hdr =get_conf_long (d, where +PCI_DVSEC_HEADER2 ); + w =BITS (hdr,0 ,16 ); +if (w ==PCI_DVSEC_ID )However, this is slightly wrong. The name "
PCI_DVSEC_ID" implies that there can only be one DVSEC ID and it is vendor-independent, but neither is true.This value is allocated by Intel, so we must check for the Intel Vendor ID first. And Intel may define several DVSEC capabilities for different purposes, so the name should also mention CXL.
So this should be "
PCI_DVSEC_INTEL_CXL" or something similar.But that doesn't mean other vendors need to define their own DVSEC IDs for CXL. The spec (PCIe r5.0, sec 7.9.6) says:
Quoted 7 lines
[The DVSEC Capability] allows PCI Express component vendors to use the Extended Capability mechanism to expose vendor-specific registers that can be present in components by a variety of vendors.
Any vendor that implements this CXL DVSEC the same way Intel does can tag it with
PCI_VENDOR_ID_INTELandPCI_DVSEC_INTEL_CXL.
Good point. I will revise.
Thanks,
Sean
Quoted 15 lines
Note that there are two types of vendor-specific capabilities:
1) Vendor-Specific Extended Capability (VSEC). The structure and definition are defined by the *Function* Vendor ID (from offset 0 in config space) and the VSEC ID in the capability.
2) Designated Vendor-Specific Extended Capability (DVSEC). The structure and definition are defined by the *DVSEC* Vendor ID (from the DVSEC capability, *not* the vendor of the Function) and the DVSEC ID in the capability.
This CXL capability is the latter, of course.
Bjorn
Re: [PATCH v5 2/2] pciutils: Decode Compute eXpress Link DVSEC
In reply to Bjorn HelgaasGot it.
Bjorn, thanks.
Jay
Quoted 87 lines · Bjorn Helgaas
Hi Jay, thanks for taking a look at this!
Quoted 27 lines · Sean V Kelley
Quoted 17 lines · Jay Fang
Quoted 16 lines · Sean V Kelley
[1] https://www.computeexpresslink.org/
Signed-off-by Sean V Kelley <[email protected]>--- lib/header.h | 20 +++ls-ecaps.c +9
-0+ +static int +is_cxl_cap (struct device *d,int where) +{ +u32 hdr; +u16 w; + +if (!config_fetch (d, where +PCI_DVSEC_HEADER1 ,8 )) +return 0 ;Quoted 8 lines
ls-ecaps.c cap_rclink(struct device *d, int where) · line 676
Quoted 5 lines
ls-ecaps.c +5
-0+ +/* Check for supported Vendor */ + hdr =get_conf_long (d, where +PCI_DVSEC_HEADER1 ); + w =BITS (hdr,0 ,16 ); +if (w !=PCI_VENDOR_ID_INTEL )I don't think here checking is quite right. Does only Intel support CXL? Other Vendors should also be considered.
Quoted 5 lines
In the absence of currently available hardware, I was attempting to limit false positive noise. I’m happy to avoid the check on the vendor if there were to exist a definitive supported list. Bjorn suggested that I check for vendor ID with DVSEC ID for now. As hardware enters the market, I can surely revise this with an update when the CXL group publishes a list.
The Vendor ID check cannot be avoided. Vendor IDs are allocated by the PCI-SIG, but the DVSEC ID is vendor-defined so there cannot be a global "CXL capability" DVSEC ID.
CXL *could* work through the PCI-SIG and define a new CXL Extended Capability that could make this generic. But apparently they've chosen to use the DVSEC mechanism instead.
Quoted 4 lines
Quoted 4 lines
ls-ecaps.c cap_rclink(struct device *d, int where) · line 682
Quoted 4 lines
ls-ecaps.c +4
-0+/* Check for Designated Vendor-Specific ID */ + hdr =get_conf_long (d, where +PCI_DVSEC_HEADER2 ); + w =BITS (hdr,0 ,16 ); +if (w ==PCI_DVSEC_ID )However, this is slightly wrong. The name "
PCI_DVSEC_ID" implies that there can only be one DVSEC ID and it is vendor-independent, but neither is true.This value is allocated by Intel, so we must check for the Intel Vendor ID first. And Intel may define several DVSEC capabilities for different purposes, so the name should also mention CXL.
So this should be "
PCI_DVSEC_INTEL_CXL" or something similar.But that doesn't mean other vendors need to define their own DVSEC IDs for CXL. The spec (PCIe r5.0, sec 7.9.6) says:
[The DVSEC Capability] allows PCI Express component vendors to use the Extended Capability mechanism to expose vendor-specific registers that can be present in components by a variety of vendors.
Any vendor that implements this CXL DVSEC the same way Intel does can tag it with
PCI_VENDOR_ID_INTELandPCI_DVSEC_INTEL_CXL.Note that there are two types of vendor-specific capabilities:
1) Vendor-Specific Extended Capability (VSEC). The structure and definition are defined by the *Function* Vendor ID (from offset 0 in config space) and the VSEC ID in the capability.
2) Designated Vendor-Specific Extended Capability (DVSEC). The structure and definition are defined by the *DVSEC* Vendor ID (from the DVSEC capability, *not* the vendor of the Function) and the DVSEC ID in the capability.
This CXL capability is the latter, of course.
Bjorn
.