CXL, ACPI, APEI, EINJ: Update EINJ for CXL error types
Subsystems (4)
v6 cover CXL, ACPI, APEI, EINJ: Update EINJ for CXL error types
v6 Changes:
- Reworked to have CXL error types under /sys/kernel/debug/cxl (Dan)
- Removed CXL error types from legacy EINJ interface in favor of new interface
- Removed cxl_rcrb_addr file
- Added optional patch for CXL error type #defines (patch 2/5)
- Changes to documentation updates to match rework
- Change base to cxl-fixes branch
The new CXL error types will use the Memory Address field in the SET_ERROR_TYPE_WITH_ADDRESS structure in order to target a CXL 1.1 compliant memory-mapped downstream port. The value of the memory address will be in the port's MMIO range, and it will not represent physical (normal or persistent) memory.
Add the functionality for injecting CXL 1.1 errors to the EINJ module, but not through the EINJ legacy interface under /sys/kernel/debug/apei/einj. Instead, make the error types available under /sys/kernel/debug/cxl. This allows for validating the MMIO address for a CXL 1.1 error type while also not making the user responsible for finding it.
Ben Cheatham (5):
cxl/port: Add EINJ debugfs files and callback support
ACPI: Add CXL protocol error defines
EINJ: Separate CXL errors from other EINJ errors cxl/port, EINJ: Add CXL EINJ callback functions
EINJ: Update EINJ documentation
Documentation/ABI/testing/debugfs-cxl | 27 ++++ .../firmware-guide/acpi/apei/einj.rst | 12 ++ drivers/acpi/apei/Kconfig | 3 + drivers/acpi/apei/einj.c | 149 ++++++++++++++++-- drivers/cxl/core/port.c | 84 ++++++++++ drivers/cxl/cxl.h | 10 ++ include/acpi/actbl1.h | 6 + 7 files changed, 281 insertions(+), 10 deletions(-)
base-commit: c66650d29764e228eba40b7a59fdb70fa6567daa
Signature · 1 line
2.34.1
PATCH v6 1/5 cxl/port: Add EINJ debugfs files and callback support
Add creation of debugfs directories for ports and dports under /sys/kernel/debug/cxl when EINJ support is enabled. The dport directories will contain files for injecting CXL protocol errors. These files are only usable once the EINJ module has loaded and registered callback functions with the CXL core module, before that occurs (or if the EINJ module isn't loaded) the files will do nothing and return an ENODEV error.
--- Documentation/ABI/testing/debugfs-cxl | 27 +++++++++ drivers/cxl/core/port.c | 84 +++++++++++++++++++++++++++ drivers/cxl/cxl.h | 10 ++++ 3 files changed, 121 insertions(+)
Documentation/ABI/testing/debugfs-cxl +27 -0
@@ -33,3 +33,30 @@ Description: device cannot clear poison from the address, -ENXIO is returned. The clear_poison attribute is only visible for devices supporting the capability. + +What: /sys/kernel/debug/cxl/portX/dportY/einj_types +Date: October, 2023 +KernelVersion: v6.7 +Contact: [email protected] +Description: + (RO) Prints the CXL protocol error types made available by + the platform in the format "0x<error number> <error type>". + The <error number> can be written to einj_inject to inject + <error type> into dportY. This file is only visible if + CONFIG_ACPI_APEI_EINJ is enabled, and the EINJ module must + be able to reach one (or both) of the CXL_ACPI or CXL_PORT + modules to be functional. + +What: /sys/kernel/debug/cxl/portX/dportY/einj_inject +Date: October, 2023 +KernelVersion: v6.7 +Contact: [email protected] +Description: + (WO) Writing an integer to this file injects the corresponding + CXL protocol error into dportY (integer to type mapping is + available by reading from einj_types). If the dport was + enumerated in RCH mode, a CXL 1.1 error is injected, otherwise + a CXL 2.0 error is injected. This file is only visible if + CONFIG_ACPI_APEI_EINJ is enabled, and the EINJ module must + be able to reach one (or both) of the CXL_ACPI or CXL_PORT + modules to be functional.
drivers/cxl/core/port.c +84 -0
126 lines not on this page · raw message
drivers/cxl/cxl.h +10 -0
@@ -581,6 +581,7 @@ struct cxl_dax_region {* @depth: How deep this port is relative to the root. depth 0 is the root. * @cdat: Cached CDAT data * @cdat_available: Should a CDAT attribute be available in sysfs +* @debug_dir: dentry for port in cxl debugfs (optional) */ struct cxl_port {struct device dev;
@@ -604,6 +605,7 @@ struct cxl_port {size_t length; } cdat;bool cdat_available; +struct dentry *debug_dir; }; static inline struct cxl_dport *
@@ -795,6 +797,14 @@ bool is_cxl_nvdimm_bridge(struct device *dev);int devm_cxl_add_nvdimm (struct cxl_memdev *cxlmd);struct cxl_nvdimm_bridge *cxl_find_nvdimm_bridge (struct cxl_memdev *cxlmd); +struct cxl_einj_ops { +int (*einj_type)(struct seq_file *f,void *data); +int (*einj_inject)(struct cxl_dport *dport,u64 type); +}; + +void cxl_einj_set_ops_cbs (struct cxl_einj_ops *ops); + +#ifdef CONFIG_CXL_REGION bool is_cxl_pmem_region (struct device *dev);struct cxl_pmem_region *to_cxl_pmem_region (struct device *dev);
Signature · 1 line
2.34.1
PATCH v6 2/5 ACPI: Add CXL protocol error defines
In reply to Ben CheathamAdd CXL protocol error defines to include/actbl1.h to be used in later commits.
---
I made a pull request for adding this support into the acpica project that has already been merged [1], Feel free to discard this commit and use the acpica changes instead.
[1]:
include/acpi/actbl1.h | 6 ++++++ 1 file changed, 6 insertions(+)
include/acpi/actbl1.h +6 -0
@@ -1093,6 +1093,12 @@ enum acpi_einj_command_status {#define ACPI_EINJ_PLATFORM_CORRECTABLE (1<<9)#define ACPI_EINJ_PLATFORM_UNCORRECTABLE (1<<10)#define ACPI_EINJ_PLATFORM_FATAL (1<<11) +#define ACPI_EINJ_CXL_CACHE_CORRECTABLE (1<<12) +#define ACPI_EINJ_CXL_CACHE_UNCORRECTABLE (1<<13) +#define ACPI_EINJ_CXL_CACHE_FATAL (1<<14) +#define ACPI_EINJ_CXL_MEM_CORRECTABLE (1<<15) +#define ACPI_EINJ_CXL_MEM_UNCORRECTABLE (1<<16) +#define ACPI_EINJ_CXL_MEM_FATAL (1<<17)#define ACPI_EINJ_VENDOR_DEFINED (1<<31)/*******************************************************************************
Signature · 1 line
2.34.1
PATCH v6 3/5 EINJ: Separate CXL errors from other EINJ errors
In reply to Ben CheathamSeparate CXL error types from other EINJ error types and disallow them in the legacy EINJ interface under /sys/kernel/debug/apei/einj. Support for the CXL error types will be added under /sys/kernel/debug/cxl in the next commit.
--- drivers/acpi/apei/einj.c | 56 +++++++++++++++++++++++++++++----------- 1 file changed, 41 insertions(+), 15 deletions(-)
drivers/acpi/apei/einj.c +41 -15
@@ -36,6 +36,12 @@#define MEM_ERROR_MASK (ACPI_EINJ_MEMORY_CORRECTABLE | \ ACPI_EINJ_MEMORY_UNCORRECTABLE | \ ACPI_EINJ_MEMORY_FATAL) +#define CXL_ERROR_MASK (ACPI_EINJ_CXL_CACHE_CORRECTABLE | \ + ACPI_EINJ_CXL_CACHE_UNCORRECTABLE | \ + ACPI_EINJ_CXL_CACHE_FATAL | \ + ACPI_EINJ_CXL_MEM_CORRECTABLE | \ + ACPI_EINJ_CXL_MEM_UNCORRECTABLE | \ + ACPI_EINJ_CXL_MEM_FATAL)/* * ACPI version 5 provides a SET_ERROR_TYPE_WITH_ADDRESS action.
@@ -537,8 +543,11 @@ static int einj_error_inject(u32 type, u32 flags, u64 param1, u64 param2,if (type &ACPI5_VENDOR_BIT ) {if (vendor_flags !=SETWA_FLAGS_MEM ) goto inject; - }else if (!(type &MEM_ERROR_MASK ) && !(flags &SETWA_FLAGS_MEM )) + }else if (!(type &MEM_ERROR_MASK ) && !(flags &SETWA_FLAGS_MEM )) { goto inject; + }else if ((type &CXL_ERROR_MASK ) && (flags &SETWA_FLAGS_MEM )) { + goto inject; + }/* * Disallow crazy address masks that give BIOS leeway to pick
@@ -590,6 +599,9 @@ static const char * const einj_error_type_string[] = {"0x00000200\tPlatform Correctable\n" ,"0x00000400\tPlatform Uncorrectable non-fatal\n" , "0x00000800\tPlatform Uncorrectable fatal\n", +}; + +static const char *const einj_cxl_error_type_string[] = {"0x00001000\tCXL.cache Protocol Correctable\n" ,"0x00002000\tCXL.cache Protocol Uncorrectable non-fatal\n" , "0x00004000\tCXL.cache Protocol Uncorrectable fatal\n",
@@ -615,29 +627,21 @@ static int available_error_type_show(struct seq_file *m, void *v)DEFINE_SHOW_ATTRIBUTE (available_error_type); -static int error_type_get (void *data,u64 *val) -{ - *val = error_type; - -return 0 ; -} - -static int error_type_set (void *data,u64 val) +static int validate_error_type (u64 type) { +u32 tval, vendor, available_error_type =0 ;int rc; -u32 available_error_type =0 ; -u32 tval, vendor;/* Only low 32 bits for error type are valid */ -if (val &GENMASK_ULL (63 ,32 )) +if (type &GENMASK_ULL (63 ,32 ))return -EINVAL ;/* * Vendor defined types have 0x80000000 bit set, and * are not enumerated by ACPI_EINJ_GET_ERROR_TYPE */ - vendor = val &ACPI5_VENDOR_BIT ; - tval = val &0x7fffffff ; + vendor = type &ACPI5_VENDOR_BIT ; + tval = type &0x7fffffff ;/* Only one error type can be specified */ if (tval & (tval -1 ))
@@ -646,9 +650,31 @@ static int error_type_set(void *data, u64 val) rc =einj_get_available_error_type (&available_error_type);if (rc)return rc; -if (!(val & available_error_type)) +if (!(type & available_error_type))return -EINVAL ; } + +return 0 ; +} + +static int error_type_get (void *data,u64 *val) +{ + *val = error_type; + +return 0 ; +} + +static int error_type_set (void *data,u64 val) +{ +int rc; + +if (val &CXL_ERROR_MASK && !(val &ACPI5_VENDOR_BIT )) +return -EINVAL ; + + rc =validate_error_type (val); +if (rc) +return rc; + error_type = val;return 0 ;
Signature · 1 line
2.34.1
PATCH v6 4/5 cxl/port, EINJ: Add CXL EINJ callback functions
In reply to Ben CheathamAdd functions to the EINJ module to be used in the CXL module for CXL protocol error injection. The callbacks implement the einj_types and einj_inject files under /sys/kernel/debug/cxl/portX/dportY. These two files work in the same way as the available_error_types and error_inject files under /sys/kernel/debug/apei/einj, but only for CXL error types. If the dport is enumerated in RCH (CXL 1.1) mode, a CXL 1.1 error is injected into the dport; a CXL 2.0 error is injected otherwise.
--- drivers/acpi/apei/Kconfig | 3 ++ drivers/acpi/apei/einj.c | 103 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 106 insertions(+)
drivers/acpi/apei/Kconfig +3 -0
@@ -11,6 +11,9 @@ config ACPI_APEI select PSTORE select UEFI_CPER depends on HAVE_ACPI_APEI + imply CXL_BUS + imply CXL_ACPI + imply CXL_PORT help APEI allows to report errors (for example from the chipset) to the operating system. This improves NMI handling
drivers/acpi/apei/einj.c +103 -0
154 lines not on this page · raw message
Signature · 1 line
2.34.1
kernel test robot
build botRe: [PATCH v6 4/5] cxl/port, EINJ: Add CXL EINJ callback functions
kernel test robot noticed the following build warnings:
Hi Ben,
kernel test robot noticed the following build warnings:
[auto build test WARNING on c66650d29764e228eba40b7a59fdb70fa6567daa]
url: https://github.com/intel-lab-lkp/linux/commits/Ben-Cheatham/cxl-port-Add-EINJ-debugfs-files-and-callback-support/20231011-040503 base: c66650d29764e228eba40b7a59fdb70fa6567daa patch link: https://lore.kernel.org/r/20231010200254.764273-5-Benjamin.Cheatham%40amd.com patch subject: [PATCH v6 4/5] cxl/port, EINJ: Add CXL EINJ callback functions reproduce: (https://download.01.org/0day-ci/archive/20231011/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
# many are suggestions rather than must-fix
ERROR:SPACING: space required after that ',' (ctx:VxV) #98: FILE: drivers/acpi/apei/einj.c:699:
+ devfn = (PCI_DEVFN(PCI_SLOT(pdev->devfn),PCI_FUNC(pdev->devfn)) << 8);
^
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
PATCH v6 5/5 EINJ: Update EINJ documentation
In reply to Ben CheathamUpdate EINJ documentation with build requirements for CXL error types and how to inject CXL error types.
--- Documentation/firmware-guide/acpi/apei/einj.rst | 12 ++++++++++++ 1 file changed, 12 insertions(+)
Documentation/firmware-guide/acpi/apei/einj.rst +12 -0
@@ -181,6 +181,18 @@ You should see something like this in dmesg:: [22715.834759] EDAC sbridge MC3: PROCESSOR 0:306e7 TIME 1422553404 SOCKET 0 APIC 0 [22716.616173] EDAC MC3: 1 CE memory read error on CPU_SrcID#0_Channel#0_DIMM#0 (channel:0 slot:0 page:0x12345 offset:0x0 grain:32 syndrome:0x0 - area:DRAM err_code:0001:0090 socket:0 channel_mask:1 rank:0) +CXL error types are supported from ACPI 6.5 onwards. These error types +are not available in the legacy interface at /sys/kernel/debug/apei/einj, +but are instead at /sys/kernel/debug/cxl/portX/dportY. Inside the dportY +directory are two files, einj_types and einj_inject. These files work the +same as the available_error_type and error_inject files (read the error +types from einj_types and write the type to inject to einj_inject). + +To use these error types one of (or both) ``CONFIG_CXL_ACPI`` or +``CONFIG_CXL_PORT`` must be reachable by the EINJ module; if +``CONFIG_ACPI_APEI_EINJ`` == y/m, then at least one of ``CONFIG_CXL_ACPI`` +or ``CONFIG_CXL_PORT`` must also be set to y/m. + Special notes for injection into SGX enclaves: There may be a separate BIOS setup option to enable SGX injection.
Signature · 1 line
2.34.1
Quoted 8 lines · On Tue, 10 Oct 2023 15:02:49 -0500 Ben Cheatham <[email protected]>…
v6 Changes:
- Reworked to have CXL error types under/sys/kernel/debug/cxl(Dan)
- Removed CXL error types from legacy EINJ interface in favor of new interface
- Removedcxl_rcrb_addrfile
- Added optional patch for CXL error type #defines (patch 2/5)
- Changes to documentation updates to match rework
- Change base to cxl-fixes branch
New approach looks good to me and I took a look at the implementation. Couldn't find any problems to call out in the individual patches. Feel free to add
Quoted 30 lines
The new CXL error types will use the Memory Address field in the
SET_ERROR_TYPE_WITH_ADDRESSstructure in order to target a CXL 1.1 compliant memory-mapped downstream port. The value of the memory address will be in the port's MMIO range, and it will not represent physical (normal or persistent) memory.Add the functionality for injecting CXL 1.1 errors to the EINJ module, but not through the EINJ legacy interface under
/sys/kernel/debug/apei/einj. Instead, make the error types available under/sys/kernel/debug/cxl. This allows for validating the MMIO address for a CXL 1.1 error type while also not making the user responsible for finding it.Ben Cheatham (5):
cxl/port: Add EINJ debugfs files and callback support
ACPI: Add CXL protocol error defines
EINJ: Separate CXL errors from other EINJ errors cxl/port, EINJ: Add CXL EINJ callback functions
EINJ: Update EINJ documentationDocumentation/ABI/testing/debugfs-cxl | 27 ++++ .../firmware-guide/acpi/apei/einj.rst | 12 ++ drivers/acpi/apei/Kconfig | 3 + drivers/acpi/apei/einj.c | 149 ++++++++++++++++-- drivers/cxl/core/port.c | 84 ++++++++++ drivers/cxl/cxl.h | 10 ++ include/acpi/actbl1.h | 6 + 7 files changed, 281 insertions(+), 10 deletions(-)base-commit:
c66650d29764e228eba40b7a59fdb70fa6567daa