PCI: Add basic Compute eXpress Link DVSEC decode
v2 cover PCI: Add basic Compute eXpress Link DVSEC decode
Changes since v1 [1]:
- Make use pci_info() and FLAG(), as in pcie_init().
- Wrap new cxl_cap in pci_dev struct within #ifdef PCI_CXL. (Bjorn Helgaas)
- Added functionality for some CXL.mem and CXL.cache helper functions. Snoop filter helper functions along with a performance hint as well as a toggle for viral self-isolation mode could be implemented. However, in the absence of CXL devices with their associated drivers, it perhaps makes best sense to be in a holding pattern on this until we have something to exercise it with.
Thoughts?
[1] https://lore.kernel.org/linux-pci/[email protected]/
This patch series implements basic Designated Vendor-Specific Extended Capabilities (DVSEC) decode for Compute eXpress Link devices, a new CPU interconnect building upon PCIe. As a basis for the CXL support it provides PCI init handling for detection, decode, and caching of CXL device capabilities. Moreover, it makes use of the DVSEC Vendor ID and DVSEC ID so as to identify a CXL capable device. (PCIe r5.0, sec 7.9.6.2)
DocLink: https://www.computeexpresslink.org/
For your reference, a parallel series of patches have been submitted to enable lspci decode of CXL DVSEC and may be tracked.
This patch makes use of pending DVSEC related header additions and the first patch of that series is included here. It can be sorted out when the upstream merge is done.
Sample dmesg output of a CXL Type 3 device (CXL.io, CXL.mem):
[ 2.997177] pci 0000:6b:00.0: CXL: Cache- IO+ Mem+ Viral- HDMCount 1 [ 2.997188] pci 0000:6b:00.0: CXL: cap ctrl status ctrl2 status2 lock [ 2.997201] pci 0000:6b:00.0: CXL: 001e 0002 0000 0000 0000 0000
David E. Box (1):
pci: Add Designated Vendor Specific Capability
Sean V Kelley (2):
PCI: Add basic Compute eXpress Link DVSEC decode
PCI: Add helpers to enable/disable CXL.mem and CXL.cache
drivers/pci/Kconfig | 9 ++ drivers/pci/Makefile | 1 + drivers/pci/cxl.c | 175 ++++++++++++++++++++++++++++++++++ drivers/pci/pci.h | 15 +++ drivers/pci/probe.c | 1 + include/linux/pci.h | 3 + include/uapi/linux/pci_regs.h | 5 + 7 files changed, 209 insertions(+) create mode 100644 drivers/pci/cxl.c
--
2.26.2
PATCH v2 1/3 pci: Add Designated Vendor Specific Capability
From: "David E. Box" <[email protected]>
Add pcie dvsec extended capability id along with helper macros to retrieve information from the headers.
https://members.pcisig.com/wg/PCI-SIG/document/12335
--- include/uapi/linux/pci_regs.h | 5 +++++ 1 file changed, 5 insertions(+)
include/uapi/linux/pci_regs.h +5 -0
@@ -720,6 +720,7 @@#define PCI_EXT_CAP_ID_DPC 0x1D/* Downstream Port Containment */ #define PCI_EXT_CAP_ID_L1SS 0x1E/* L1 PM Substates */ #define PCI_EXT_CAP_ID_PTM 0x1F/* Precision Time Measurement */ +#define PCI_EXT_CAP_ID_DVSEC 0x23/* Desinated Vendor-Specific */ #define PCI_EXT_CAP_ID_DLF 0x25/* Data Link Feature */ #define PCI_EXT_CAP_ID_PL_16GT 0x26/* Physical Layer 16.0 GT/s */ #define PCI_EXT_CAP_ID_MAX PCI_EXT_CAP_ID_PL_16GT
@@ -1062,6 +1063,10 @@#define PCI_L1SS_CTL1_LTR_L12_TH_SCALE 0xe0000000/* LTR_L1.2_THRESHOLD_Scale */ #define PCI_L1SS_CTL2 0x0c/* Control 2 Register */ +/* Designated Vendor-Specific (DVSEC, PCI_EXT_CAP_ID_DVSEC) */ +#define PCI_DVSEC_HEADER1 0x4/* Vendor-Specific Header1 */ +#define PCI_DVSEC_HEADER2 0x8/* Vendor-Specific Header2 */ +/* Data Link Feature */ #define PCI_DLF_CAP 0x04/* Capabilities Register */ #define PCI_DLF_EXCHANGE_ENABLE 0x80000000/* Data Link Feature Exchange Enable */
Signature · 1 line
2.26.2
PATCH v2 2/3 PCI: Add basic Compute eXpress Link DVSEC decode
In reply to Sean V KelleyCompute eXpress Link is a new CPU interconnect created with workload accelerators in mind. The interconnect relies on PCIe Electrical and Physical interconnect for communication. CXL devices enumerate to the OS as an ACPI-described PCIe Root Complex Integrated Endpoint.
This patch introduces the bare minimum support by simply looking for and caching the DVSEC CXL Extended Capability. Currently, only CXL.io (which is mandatory to be configured by BIOS) is enabled. In future, we will also add support for CXL.cache and CXL.mem.
DocLink: https://www.computeexpresslink.org/
--- drivers/pci/Kconfig | 9 +++++ drivers/pci/Makefile | 1 + drivers/pci/cxl.c | 92 ++++++++++++++++++++++++++++++++++++++++++++ drivers/pci/pci.h | 7 ++++ drivers/pci/probe.c | 1 + include/linux/pci.h | 3 ++ 6 files changed, 113 insertions(+) create mode 100644 drivers/pci/cxl.c
drivers/pci/Kconfig +9 -0
@@ -115,6 +115,15 @@ config XEN_PCIDEV_FRONTEND The PCI device frontend driver allows the kernel to import arbitrary PCI devices from a PCI backend to support PCI driver domains. +config PCI_CXL + bool "Enable PCI Compute eXpress Link" + depends on PCI + help + Say Y here if you want the PCI core to detect CXL devices, decode, and + cache the DVSEC CXL Extended Capability as configured by BIOS. + + When in doubt, say N. + config PCI_ATS bool
drivers/pci/Makefile +1 -0
@@ -30,6 +30,7 @@ obj-$(CONFIG_PCI_PF_STUB) += pci-pf-stub.o obj-$(CONFIG_PCI_ECAM) += ecam.o obj-$(CONFIG_PCI_P2PDMA) += p2pdma.o obj-$(CONFIG_XEN_PCIDEV_FRONTEND) += xen-pcifront.o +obj-$(CONFIG_PCI_CXL) += cxl.o # Endpoint library must be initialized before its users obj-$(CONFIG_PCI_ENDPOINT) += endpoint/
drivers/pci/cxl.c new file +92 -0
92 lines not on this page · raw message
drivers/pci/pci.h +7 -0
@@ -469,6 +469,13 @@ static inline void pci_ats_init(struct pci_dev *d) { }static inline void pci_restore_ats_state (struct pci_dev *dev) { }#endif /* CONFIG_PCI_ATS */ +#ifdef CONFIG_PCI_CXL +/* Compute eXpress Link */ +void pci_cxl_init (struct pci_dev *dev); +#else +static inline void pci_cxl_init (struct pci_dev *dev) { } +#endif +#ifdef CONFIG_PCI_PRI void pci_pri_init (struct pci_dev *dev);void pci_restore_pri_state (struct pci_dev *pdev);
drivers/pci/probe.c +1 -0
@@ -2371,6 +2371,7 @@ static void pci_init_capabilities(struct pci_dev *dev)pci_ptm_init (dev);/* Precision Time Measurement */ pci_aer_init (dev);/* Advanced Error Reporting */ pci_dpc_init (dev);/* Downstream Port Containment */ +pci_cxl_init (dev);/* Compute eXpress Link */ pcie_report_downtraining (dev);
include/linux/pci.h +3 -0
@@ -314,6 +314,9 @@ struct pci_dev {#ifdef CONFIG_PCIEAER u16 aer_cap;/* AER capability offset */ struct aer_stats *aer_stats;/* AER stats for this device */ +#endif +#ifdef CONFIG_PCI_CXL +u16 cxl_cap;/* CXL capability offset */ #endif u8 pcie_cap;/* PCIe capability offset */ u8 msi_cap;/* MSI capability offset */
Signature · 1 line
2.26.2
PATCH v2 3/3 PCI: Add helpers to enable/disable CXL.mem and CXL.cache
In reply to Sean V KelleyWith these helpers, a device driver can enable/disable access to CXL.mem and CXL.cache. Note that the device driver is responsible for managing the memory area.
--- drivers/pci/cxl.c | 93 ++++++++++++++++++++++++++++++++++++++++++++--- drivers/pci/pci.h | 8 ++++ 2 files changed, 96 insertions(+), 5 deletions(-)
drivers/pci/cxl.c +88 -5
@@ -24,6 +24,88 @@#define PCI_CXL_HDM_COUNT (reg) (((reg) & (3 << 4)) >> 4)#define PCI_CXL_VIRAL BIT(14) +#define PCI_CXL_CONFIG_LOCK BIT(0) + +static void pci_cxl_unlock (struct pci_dev *dev) +{ +int pos = dev->cxl_cap; +u16 lock; + +pci_read_config_word (dev, pos +PCI_CXL_LOCK , &lock); + lock &= ~PCI_CXL_CONFIG_LOCK ; +pci_write_config_word (dev, pos +PCI_CXL_LOCK , lock); +} + +static void pci_cxl_lock (struct pci_dev *dev) +{ +int pos = dev->cxl_cap; +u16 lock; + +pci_read_config_word (dev, pos +PCI_CXL_LOCK , &lock); + lock |=PCI_CXL_CONFIG_LOCK ; +pci_write_config_word (dev, pos +PCI_CXL_LOCK , lock); +} + +static int pci_cxl_enable_disable_feature (struct pci_dev *dev,int enable, +u16 feature) +{ +int pos = dev->cxl_cap; +int ret; +u16 reg; + +if (!dev->cxl_cap) +return -EINVAL ; + +/* Only for PCIe */ +if (!pci_is_pcie (dev)) +return -EINVAL ; + +/* Only for Device 0 Function 0, Root Complex Integrated Endpoints */ +if (dev->devfn !=0 || (pci_pcie_type (dev) !=PCI_EXP_TYPE_RC_END )) +return -EINVAL ; + +pci_cxl_unlock (dev); + ret =pci_read_config_word (dev, pos +PCI_CXL_CTRL , ®); +if (ret) + goto lock; + +if (enable) + reg |= feature; +else + reg &= ~feature; + + ret =pci_write_config_word (dev, pos +PCI_CXL_CTRL , reg); + +lock: +pci_cxl_lock (dev); + +return ret; +} + +int pci_cxl_mem_enable (struct pci_dev *dev) +{ +return pci_cxl_enable_disable_feature (dev, true,PCI_CXL_MEM ); +} +EXPORT_SYMBOL_GPL (pci_cxl_mem_enable); + +void pci_cxl_mem_disable (struct pci_dev *dev) +{ +pci_cxl_enable_disable_feature (dev, false,PCI_CXL_MEM ); +} +EXPORT_SYMBOL_GPL (pci_cxl_mem_disable); + +int pci_cxl_cache_enable (struct pci_dev *dev) +{ +return pci_cxl_enable_disable_feature (dev, true,PCI_CXL_CACHE ); +} +EXPORT_SYMBOL_GPL (pci_cxl_cache_enable); + +void pci_cxl_cache_disable (struct pci_dev *dev) +{ +pci_cxl_enable_disable_feature (dev, false,PCI_CXL_CACHE ); +} +EXPORT_SYMBOL_GPL (pci_cxl_cache_disable); +/* * pci_find_cxl_capability - Identify and return offset to Vendor-Specific * capabilities.
@@ -73,11 +155,6 @@ void pci_cxl_init(struct pci_dev *dev) dev->cxl_cap = pos;pci_read_config_word (dev, pos +PCI_CXL_CAP , &cap); -pci_read_config_word (dev, pos +PCI_CXL_CTRL , &ctrl); -pci_read_config_word (dev, pos +PCI_CXL_STS , &status); -pci_read_config_word (dev, pos +PCI_CXL_CTRL2 , &ctrl2); -pci_read_config_word (dev, pos +PCI_CXL_STS2 , &status2); -pci_read_config_word (dev, pos +PCI_CXL_LOCK , &lock); pci_info(dev, "CXL: Cache%c IO%c Mem%c Viral%c HDMCount %d\n", FLAG(cap, PCI_CXL_CACHE),
@@ -86,6 +163,12 @@ void pci_cxl_init(struct pci_dev *dev)FLAG (cap,PCI_CXL_VIRAL ), PCI_CXL_HDM_COUNT(cap)); +pci_read_config_word (dev, pos +PCI_CXL_CTRL , &ctrl); +pci_read_config_word (dev, pos +PCI_CXL_STS , &status); +pci_read_config_word (dev, pos +PCI_CXL_CTRL2 , &ctrl2); +pci_read_config_word (dev, pos +PCI_CXL_STS2 , &status2); +pci_read_config_word (dev, pos +PCI_CXL_LOCK , &lock); +pci_info (dev,"CXL: cap ctrl status ctrl2 status2 lock\n" );pci_info (dev,"CXL: %04x %04x %04x %04x %04x %04x\n" , cap, ctrl, status, ctrl2, status2, lock);
drivers/pci/pci.h +8 -0
@@ -472,8 +472,16 @@ static inline void pci_restore_ats_state(struct pci_dev *dev) { }#ifdef CONFIG_PCI_CXL /* Compute eXpress Link */ void pci_cxl_init (struct pci_dev *dev); +int pci_cxl_mem_enable (struct pci_dev *dev); +void pci_cxl_mem_disable (struct pci_dev *dev); +int pci_cxl_cache_enable (struct pci_dev *dev); +void pci_cxl_cache_disable (struct pci_dev *dev);#else static inline void pci_cxl_init (struct pci_dev *dev) { } +static inline int pci_cxl_mem_enable (struct pci_dev *dev) {} +static inline void pci_cxl_mem_disable (struct pci_dev *dev) {} +static inline int pci_cxl_cache_enable (struct pci_dev *dev) {} +static inline void pci_cxl_cache_disable (struct pci_dev *dev) {}#endif #ifdef CONFIG_PCI_PRI
Signature · 1 line
2.26.2
Re: [PATCH V2 3/3] PCI: Add helpers to enable/disable CXL.mem and CXL.cache
Quoted 29 lines · Sean V Kelley
With these helpers, a device driver can enable/disable access to CXL.mem and CXL.cache. Note that the device driver is responsible for managing the memory area.
Signed-off-by Sean V Kelley <[email protected]>--- drivers/pci/cxl.c | 93 ++++++++++++++++++++++++++++++++++++++++++++--- drivers/pci/pci.h | 8 ++++ 2 files changed, 96 insertions(+), 5 deletions(-)drivers/pci/cxl.c +11
-0@@ -24,6 +24,88 @@#define PCI_CXL_HDM_COUNT (reg) (((reg) & (3 << 4)) >> 4)#define PCI_CXL_VIRAL BIT(14) +#define PCI_CXL_CONFIG_LOCK BIT(0) + +static void pci_cxl_unlock (struct pci_dev *dev) +{ +int pos = dev->cxl_cap; +u16 lock; + +pci_read_config_word (dev, pos +PCI_CXL_LOCK , &lock); + lock &= ~PCI_CXL_CONFIG_LOCK ; +pci_write_config_word (dev, pos +PCI_CXL_LOCK , lock); +}
drivers/pci/cxl.c line 47
Quoted 10 lines
drivers/pci/cxl.c +10
-0+ +static void pci_cxl_lock (struct pci_dev *dev) +{ +int pos = dev->cxl_cap; +u16 lock; + +pci_read_config_word (dev, pos +PCI_CXL_LOCK , &lock); + lock |=PCI_CXL_CONFIG_LOCK ; +pci_write_config_word (dev, pos +PCI_CXL_LOCK , lock); +}
Maybe a tiny comment about what PCI_CXL_CONFIG_LOCK does? I guess these functions don't enforce mutual exclusion (as I first expected a "lock" function to do); they're simply telling the device to accept CTRL changes or something?
drivers/pci/cxl.c line 57
Quoted 9 lines
drivers/pci/cxl.c +9
-0+static int pci_cxl_enable_disable_feature (struct pci_dev *dev,int enable, +u16 feature) +{ +int pos = dev->cxl_cap; +int ret; +u16 reg; + +if (!dev->cxl_cap) +return -EINVAL ;
"pos"
"pos" is the typical name for things like this, but sometimes I think the code would be more descriptive if we used things like "cxl", "aer", etc.
drivers/pci/cxl.c line 61
Quoted 4 lines
drivers/pci/cxl.c +4
-0+ +/* Only for PCIe */ +if (!pci_is_pcie (dev)) +return -EINVAL ;
Probably not necessary here. We already checked it in pci_cxi_init().
Quoted 62 lines
drivers/pci/cxl.c +46
-5+/* Only for Device 0 Function 0, Root Complex Integrated Endpoints */ +if (dev->devfn !=0 || (pci_pcie_type (dev) !=PCI_EXP_TYPE_RC_END )) +return -EINVAL ; + +pci_cxl_unlock (dev); + ret =pci_read_config_word (dev, pos +PCI_CXL_CTRL , ®); +if (ret) + goto lock; + +if (enable) + reg |= feature; +else + reg &= ~feature; + + ret =pci_write_config_word (dev, pos +PCI_CXL_CTRL , reg); + +lock: +pci_cxl_lock (dev); + +return ret; +} + +int pci_cxl_mem_enable (struct pci_dev *dev) +{ +return pci_cxl_enable_disable_feature (dev, true,PCI_CXL_MEM ); +} +EXPORT_SYMBOL_GPL (pci_cxl_mem_enable); + +void pci_cxl_mem_disable (struct pci_dev *dev) +{ +pci_cxl_enable_disable_feature (dev, false,PCI_CXL_MEM ); +} +EXPORT_SYMBOL_GPL (pci_cxl_mem_disable); + +int pci_cxl_cache_enable (struct pci_dev *dev) +{ +return pci_cxl_enable_disable_feature (dev, true,PCI_CXL_CACHE ); +} +EXPORT_SYMBOL_GPL (pci_cxl_cache_enable); + +void pci_cxl_cache_disable (struct pci_dev *dev) +{ +pci_cxl_enable_disable_feature (dev, false,PCI_CXL_CACHE ); +} +EXPORT_SYMBOL_GPL (pci_cxl_cache_disable); +/* * pci_find_cxl_capability - Identify and return offset to Vendor-Specific * capabilities. @@ -73,11 +155,6 @@ void pci_cxl_init(struct pci_dev *dev) dev->cxl_cap = pos;pci_read_config_word (dev, pos +PCI_CXL_CAP , &cap); -pci_read_config_word (dev, pos +PCI_CXL_CTRL , &ctrl); -pci_read_config_word (dev, pos +PCI_CXL_STS , &status); -pci_read_config_word (dev, pos +PCI_CXL_CTRL2 , &ctrl2); -pci_read_config_word (dev, pos +PCI_CXL_STS2 , &status2); -pci_read_config_word (dev, pos +PCI_CXL_LOCK , &lock); pci_info(dev, "CXL: Cache%c IO%c Mem%c Viral%c HDMCount %d\n", FLAG(cap, PCI_CXL_CACHE),
drivers/pci/cxl.c void pci_cxl_init(struct pci_dev *dev) · line 170
Quoted 8 lines
drivers/pci/cxl.c +5
-0@@ -86,6 +163,12 @@ void pci_cxl_init(struct pci_dev *dev)FLAG (cap,PCI_CXL_VIRAL ), PCI_CXL_HDM_COUNT(cap)); +pci_read_config_word (dev, pos +PCI_CXL_CTRL , &ctrl); +pci_read_config_word (dev, pos +PCI_CXL_STS , &status); +pci_read_config_word (dev, pos +PCI_CXL_CTRL2 , &ctrl2); +pci_read_config_word (dev, pos +PCI_CXL_STS2 , &status2); +pci_read_config_word (dev, pos +PCI_CXL_LOCK , &lock);
This looks like it's just moving these reads around and isn't actually related to this patch. Put them where you want them in the first place so the move doesn't clutter this patch.
Quoted 11 lines
drivers/pci/cxl.c +0
-0pci_info (dev,"CXL: cap ctrl status ctrl2 status2 lock\n" );pci_info (dev,"CXL: %04x %04x %04x %04x %04x %04x\n" , cap, ctrl, status, ctrl2, status2, lock);drivers/pci/pci.h +0
-0@@ -472,8 +472,16 @@ static inline void pci_restore_ats_state(struct pci_dev *dev) { }#ifdef CONFIG_PCI_CXL /* Compute eXpress Link */ void pci_cxl_init (struct pci_dev *dev);
drivers/pci/pci.h static inline void pci_restore_ats_state(struct pci_dev *dev) { } · line 484
Quoted 10 lines
drivers/pci/pci.h +8
-0+int pci_cxl_mem_enable (struct pci_dev *dev); +void pci_cxl_mem_disable (struct pci_dev *dev); +int pci_cxl_cache_enable (struct pci_dev *dev); +void pci_cxl_cache_disable (struct pci_dev *dev);#else static inline void pci_cxl_init (struct pci_dev *dev) { } +static inline int pci_cxl_mem_enable (struct pci_dev *dev) {} +static inline void pci_cxl_mem_disable (struct pci_dev *dev) {} +static inline int pci_cxl_cache_enable (struct pci_dev *dev) {} +static inline void pci_cxl_cache_disable (struct pci_dev *dev) {}
The stubs in this file aren't completely consistent, but you can be at least locally consistent by including a space between the "{ }".
Quoted 6 lines
drivers/pci/pci.h +0
-0#endif #ifdef CONFIG_PCI_PRI Signature · 1 line
2.26.2
Re: [PATCH V2 3/3] PCI: Add helpers to enable/disable CXL.mem and CXL.cache
Quoted 38 lines · Bjorn Helgaas
Quoted 37 lines · Sean V Kelley
With these helpers, a device driver can enable/disable access to CXL.mem and CXL.cache. Note that the device driver is responsible for managing the memory area.
Signed-off-by Sean V Kelley <[email protected]>--- drivers/pci/cxl.c | 93 ++++++++++++++++++++++++++++++++++++++++++++--- drivers/pci/pci.h | 8 ++++ 2 files changed, 96 insertions(+), 5 deletions(-)drivers/pci/cxl.c +18
-0@@ -24,6 +24,88 @@#define PCI_CXL_HDM_COUNT (reg) (((reg) & (3 << 4)) >> 4)#define PCI_CXL_VIRAL BIT(14) +#define PCI_CXL_CONFIG_LOCK BIT(0) + +static void pci_cxl_unlock (struct pci_dev *dev) +{ +int pos = dev->cxl_cap; +u16 lock; + +pci_read_config_word (dev, pos +PCI_CXL_LOCK , &lock); + lock &= ~PCI_CXL_CONFIG_LOCK ; +pci_write_config_word (dev, pos +PCI_CXL_LOCK , lock); +} + +static void pci_cxl_lock (struct pci_dev *dev) +{ +int pos = dev->cxl_cap; +u16 lock; + +pci_read_config_word (dev, pos +PCI_CXL_LOCK , &lock);
Quoted 8 lines
drivers/pci/cxl.c line 47
Quoted 3 lines
drivers/pci/cxl.c +3
-0+ lock |=PCI_CXL_CONFIG_LOCK ; +pci_write_config_word (dev, pos +PCI_CXL_LOCK , lock); +}Maybe a tiny comment about what
PCI_CXL_CONFIG_LOCKdoes? I guess these functions don't enforce mutual exclusion (as I first expected a "lock" function to do); they're simply telling the device to accept CTRL changes or something?
Agree will add. DVSEC Lock is intended to signify whether the CTRL bits may be changed. It locks certain registers and makes them RO. This lock is designed to prevent future changes to configuration. Not to be confused with enforcing mutual exclusion.
Quoted 14 lines
drivers/pci/cxl.c line 57
Quoted 11 lines
drivers/pci/cxl.c +9
-0+static int pci_cxl_enable_disable_feature (struct pci_dev *dev,int enable, +u16 feature) +{ +int pos = dev->cxl_cap; +int ret; +u16 reg; + +if (!dev->cxl_cap) +return -EINVAL ;"pos"
Quoted 3 lines
"pos" is the typical name for things like this, but sometimes I think the code would be more descriptive if we used things like "cxl", "aer", etc.
Will do, makes sense.
Quoted 7 lines
drivers/pci/cxl.c line 61
Quoted 5 lines
drivers/pci/cxl.c +4
-0+ +/* Only for PCIe */ +if (!pci_is_pcie (dev)) +return -EINVAL ;Probably not necessary here. We already checked it in
pci_cxi_init().
You are correct. Will remove.
Quoted 68 lines
Quoted 68 lines
drivers/pci/cxl.c +46
-0+/* Only for Device 0 Function 0, Root Complex Integrated Endpoints */ +if (dev->devfn !=0 || (pci_pcie_type (dev) !=PCI_EXP_TYPE_RC_END )) +return -EINVAL ; + +pci_cxl_unlock (dev); + ret =pci_read_config_word (dev, pos +PCI_CXL_CTRL , ®); +if (ret) + goto lock; + +if (enable) + reg |= feature; +else + reg &= ~feature; + + ret =pci_write_config_word (dev, pos +PCI_CXL_CTRL , reg); + +lock: +pci_cxl_lock (dev); + +return ret; +} + +int pci_cxl_mem_enable (struct pci_dev *dev) +{ +return pci_cxl_enable_disable_feature (dev, true,PCI_CXL_MEM ); +} +EXPORT_SYMBOL_GPL (pci_cxl_mem_enable); + +void pci_cxl_mem_disable (struct pci_dev *dev) +{ +pci_cxl_enable_disable_feature (dev, false,PCI_CXL_MEM ); +} +EXPORT_SYMBOL_GPL (pci_cxl_mem_disable); + +int pci_cxl_cache_enable (struct pci_dev *dev) +{ +return pci_cxl_enable_disable_feature (dev, true,PCI_CXL_CACHE ); +} +EXPORT_SYMBOL_GPL (pci_cxl_cache_enable); + +void pci_cxl_cache_disable (struct pci_dev *dev) +{ +pci_cxl_enable_disable_feature (dev, false,PCI_CXL_CACHE ); +} +EXPORT_SYMBOL_GPL (pci_cxl_cache_disable); +/* * pci_find_cxl_capability - Identify and return offset to Vendor-Specific
* capabilities.drivers/pci/cxl.c +0
-5@@ -73,11 +155,6 @@ void pci_cxl_init(struct pci_dev *dev) dev->cxl_cap = pos;pci_read_config_word (dev, pos +PCI_CXL_CAP , &cap); -pci_read_config_word (dev, pos +PCI_CXL_CTRL , &ctrl); -pci_read_config_word (dev, pos +PCI_CXL_STS , &status); -pci_read_config_word (dev, pos +PCI_CXL_CTRL2 , &ctrl2); -pci_read_config_word (dev, pos +PCI_CXL_STS2 , &status2); -pci_read_config_word (dev, pos +PCI_CXL_LOCK , &lock); pci_info(dev, "CXL: Cache%c IO%c Mem%c Viral%c HDMCount %d\n", FLAG(cap, PCI_CXL_CACHE),@@ -86,6 +163,12 @@ void pci_cxl_init(struct pci_dev *dev)FLAG (cap,PCI_CXL_VIRAL ), PCI_CXL_HDM_COUNT(cap));
Quoted 9 lines
drivers/pci/cxl.c void pci_cxl_init(struct pci_dev *dev) · line 170
Quoted 5 lines
drivers/pci/cxl.c +5
-0+pci_read_config_word (dev, pos +PCI_CXL_CTRL , &ctrl); +pci_read_config_word (dev, pos +PCI_CXL_STS , &status); +pci_read_config_word (dev, pos +PCI_CXL_CTRL2 , &ctrl2); +pci_read_config_word (dev, pos +PCI_CXL_STS2 , &status2); +pci_read_config_word (dev, pos +PCI_CXL_LOCK , &lock);This looks like it's just moving these reads around and isn't actually related to this patch. Put them where you want them in the first place so the move doesn't clutter this patch.
Looks that way. I will clean it up.
Quoted 18 lines
Quoted 18 lines
drivers/pci/cxl.c +0
-0pci_info (dev,"CXL: cap ctrl status ctrl2 status2 lock\n" );pci_info (dev,"CXL: %04x %04x %04x %04x %04x %04x\n" , cap, ctrl, status, ctrl2, status2, lock);drivers/pci/pci.h +4
-0@@ -472,8 +472,16 @@ static inline void pci_restore_ats_state(struct pci_dev *dev) { }#ifdef CONFIG_PCI_CXL /* Compute eXpress Link */ void pci_cxl_init (struct pci_dev *dev); +int pci_cxl_mem_enable (struct pci_dev *dev); +void pci_cxl_mem_disable (struct pci_dev *dev); +int pci_cxl_cache_enable (struct pci_dev *dev); +void pci_cxl_cache_disable (struct pci_dev *dev);#else
Quoted 8 lines
drivers/pci/pci.h static inline void pci_restore_ats_state(struct pci_dev *dev) { } · line 484
Quoted 5 lines
drivers/pci/pci.h +4
-0static inline void pci_cxl_init (struct pci_dev *dev) { } +static inline int pci_cxl_mem_enable (struct pci_dev *dev) {} +static inline void pci_cxl_mem_disable (struct pci_dev *dev) {} +static inline int pci_cxl_cache_enable (struct pci_dev *dev) {} +static inline void pci_cxl_cache_disable (struct pci_dev *dev) {}The stubs in this file aren't completely consistent, but you can be at least locally consistent by including a space between the "{ }".
Will do.
Thanks,
Sean
Quoted 7 lines
Quoted 7 lines
drivers/pci/pci.h +0
-0#endif #ifdef
CONFIG_PCI_PRISignature · 1 line
2.26.2
kbuild test robot
build botRe: [PATCH V2 3/3] PCI: Add helpers to enable/disable CXL.mem and CXL.cache
In reply to Sean V KelleyThank you for the patch! Perhaps something to improve:
Hi Sean,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on pci/next]
[also build test WARNING on v5.7-rc6 next-20200518]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please see https://stackoverflow.com/a/37406982]
url: https://github.com/0day-ci/linux/commits/Sean-V-Kelley/PCI-Add-basic-Compute-eXpress-Link-DVSEC-decode/20200519-004029 base: https://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git next config: arc-defconfig (attached as .config) compiler: arc-elf-gcc (GCC) 9.3.0
reproduce:
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=arc
If you fix the issue, kindly add following tag as appropriate
All warnings (new ones prefixed by >>, old ones prefixed by <<):
In file included from drivers/pci/of.c:16:drivers/pci/pci.h: In function 'pci_cxl_mem_enable':
>> drivers/pci/pci.h:481:45: warning: no return statement in function returning non-void [-Wreturn-type]
481 | static inline int pci_cxl_mem_enable(struct pci_dev *dev) {}
| ^~~~~~~drivers/pci/pci.h: In function 'pci_cxl_cache_enable':drivers/pci/pci.h:483:47: warning: no return statement in function returning non-void [-Wreturn-type] 483 | static inline int pci_cxl_cache_enable(struct pci_dev *dev) {}
| ^~~~~~~
vim +481 drivers/pci/pci.h
471
472 #ifdef CONFIG_PCI_CXL
473 /* Compute eXpress Link */
474 void pci_cxl_init(struct pci_dev *dev);
475 int pci_cxl_mem_enable(struct pci_dev *dev);
476 void pci_cxl_mem_disable(struct pci_dev *dev);
477 int pci_cxl_cache_enable(struct pci_dev *dev);
478 void pci_cxl_cache_disable(struct pci_dev *dev);
479 #else
480 static inline void pci_cxl_init(struct pci_dev *dev) { }
> 481 static inline int pci_cxl_mem_enable(struct pci_dev *dev) {} 482 static inline void pci_cxl_mem_disable(struct pci_dev *dev) {}
483 static inline int pci_cxl_cache_enable(struct pci_dev *dev) {}
484 static inline void pci_cxl_cache_disable(struct pci_dev *dev) {}
485 #endif
486
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
kbuild test robot
build botRe: [PATCH V2 3/3] PCI: Add helpers to enable/disable CXL.mem and CXL.cache
In reply to Sean V KelleyThank you for the patch! Perhaps something to improve:
Hi Sean,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on pci/next]
[also build test WARNING on v5.7-rc6 next-20200518]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please see https://stackoverflow.com/a/37406982]
url: https://github.com/0day-ci/linux/commits/Sean-V-Kelley/PCI-Add-basic-Compute-eXpress-Link-DVSEC-decode/20200519-004029 base: https://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git next config: powerpc-defconfig (attached as .config) compiler: powerpc64-linux-gcc (GCC) 9.3.0
reproduce:
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=powerpc
If you fix the issue, kindly add following tag as appropriate
All warnings (new ones prefixed by >>, old ones prefixed by <<):
In file included from drivers/pci/hotplug/pci_hotplug_core.c:32:drivers/pci/hotplug/../pci.h: In function 'pci_cxl_mem_enable':
>> drivers/pci/hotplug/../pci.h:481:45: warning: no return statement in function returning non-void [-Wreturn-type]
481 | static inline int pci_cxl_mem_enable(struct pci_dev *dev) {}
| ^~~~~~~drivers/pci/hotplug/../pci.h: In function 'pci_cxl_cache_enable':drivers/pci/hotplug/../pci.h:483:47: warning: no return statement in function returning non-void [-Wreturn-type] 483 | static inline int pci_cxl_cache_enable(struct pci_dev *dev) {}
| ^~~~~~~
vim +481 drivers/pci/hotplug/../pci.h
471
472 #ifdef CONFIG_PCI_CXL
473 /* Compute eXpress Link */
474 void pci_cxl_init(struct pci_dev *dev);
475 int pci_cxl_mem_enable(struct pci_dev *dev);
476 void pci_cxl_mem_disable(struct pci_dev *dev);
477 int pci_cxl_cache_enable(struct pci_dev *dev);
478 void pci_cxl_cache_disable(struct pci_dev *dev);
479 #else
480 static inline void pci_cxl_init(struct pci_dev *dev) { }
> 481 static inline int pci_cxl_mem_enable(struct pci_dev *dev) {} 482 static inline void pci_cxl_mem_disable(struct pci_dev *dev) {}
483 static inline int pci_cxl_cache_enable(struct pci_dev *dev) {}
484 static inline void pci_cxl_cache_disable(struct pci_dev *dev) {}
485 #endif
486
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
Re: [PATCH V2 3/3] PCI: Add helpers to enable/disable CXL.mem and CXL.cache
In reply to Sean V KelleyQuoted 3 lines · Sean V Kelley
With these helpers, a device driver can enable/disable access to CXL.mem and CXL.cache. Note that the device driver is responsible for managing the memory area.
Who is going to call these? Please don't submit new APIs without users.
Quoted 6 lines · Sean V Kelley
Changes since v1 [1]:
- Make use
pci_info()andFLAG(), as inpcie_init().
- Wrap newcxl_capinpci_devstruct within #ifdefPCI_CXL. (Bjorn Helgaas)
Quoted 6 lines
- Added functionality for some CXL.mem and CXL.cache helper functions. Snoop filter helper functions along with a performance hint as well as a toggle for viral self-isolation mode could be implemented. However, in the absence of CXL devices with their associated drivers, it perhaps makes best sense to be in a holding pattern on this until we have something to exercise it with.
Right, until these add functionality we can actually use, e.g., drivers that call these new functions, I think we're in a holding pattern.
Bjorn