CXL, ACPI, APEI, EINJ: Update EINJ for CXL error types
Subsystems (4)
v10 cover CXL, ACPI, APEI, EINJ: Update EINJ for CXL error types
v10 Changes:
- Fixups in EINJ module initializtion (Dan)
- Add include/linux/einj-cxl.h to MAINTAINERS under CXL subsystem (Dan)
- Replace usages of IS_ENABLED(CONFIG_CXL_EINJ) with new einj_is_initialized() function in cxl/core/port.c (Dan)
- Fix typo in EINJ documentation (Dan)
v9 Changes:
- Remove wrapper __init function in EINJ module and use a platform driver instead (Dan & Jonathan)
- Move cxl/einj.h to include/linux/einj-cxl.h (Dan)
- Rework header file to remove cxl_einj_* functions (Dan)
- Add IS_ENABLED() guards around EINJ debugfs functionality in cxl/core/port.c (Dan)
- Rework cxl/Kconfig to be more concise (Dan)
- Small fixups to cxl-debugfs documentation (Dan)
- Add check to not add einj files to CXL 2.0+ dports that aren't represented by a pci_dev
- Bump version number in debugfs-cxl documentation
The new CXL error types will use the Memory Address field in the SET_ERROR_TYPE_WITH_ADDRESS structure in order to target a CXL 1.1 compliant memory-mapped downstream port. The value of the memory address will be in the port's MMIO range, and it will not represent physical (normal or persistent) memory.
Add the functionality for injecting CXL 1.1 errors to the EINJ module, but not through the EINJ legacy interface under /sys/kernel/debug/apei/einj. Instead, make the error types available under /sys/kernel/debug/cxl. This allows for validating the MMIO address for a CXL 1.1 error type while also not making the user responsible for finding it.
Ben Cheatham (5):
cxl, ACPI, APEI, EINJ: Add CXL_EINJ Kconfig option
ACPI: Add CXL protocol error defines
EINJ: Migrate to a platform driver
cxl/core, EINJ: Add EINJ CXL debugfs files and EINJ helper functions EINJ, Documentation: Update EINJ kernel doc
Documentation/ABI/testing/debugfs-cxl | 22 ++ .../firmware-guide/acpi/apei/einj.rst | 19 ++ MAINTAINERS | 1 + drivers/acpi/apei/einj.c | 194 ++++++++++++++++-- drivers/cxl/Kconfig | 12 ++ drivers/cxl/core/port.c | 39 ++++ include/acpi/actbl1.h | 6 + include/linux/einj-cxl.h | 45 ++++ 8 files changed, 326 insertions(+), 12 deletions(-) create mode 100644 include/linux/einj-cxl.h
Signature · 1 line
2.34.1
PATCH v10 1/5 cxl, ACPI, APEI, EINJ: Add CXL_EINJ Kconfig option
Add CONFIG_CXL_EINJ to cxl/Kconfig. This option will allow for the CXL core module to access helpers inside the EINJ module, while also giving users the option of disabling CXL EINJ error types at build time.
Also update CONFIG_ACPI_APEI_EINJ to set CONFIG_CXL_EINJ by default.
--- drivers/cxl/Kconfig | 12 ++++++++++++ 1 file changed, 12 insertions(+)
drivers/cxl/Kconfig +12 -0
@@ -157,4 +157,16 @@ config CXL_PMU monitoring units and provide standard perf based interfaces. If unsure say 'm'. + +config CXL_EINJ + bool "CXL Error INJection Support" + default ACPI_APEI_EINJ + depends on ACPI_APEI_EINJ >= CXL_BUS + help + Support for CXL protocol Error INJection through debugfs/cxl. + Availability and which errors are supported is dependent on + the host platform. Look to ACPI v6.5 section 18.6.4 and kernel + EINJ documentation for more information. + + If unsure say 'n' endif
Signature · 1 line
2.34.1
PATCH v10 2/5 ACPI: Add CXL protocol error defines
In reply to Ben CheathamAdd CXL protocol error defines to include/actbl1.h.
---
I made a pull request for this support in the ACPICA project which has been accepted (link below), so this patch is temporary and I expect it to be dropped once the kernel updates from ACPICA.
[1]:
--- include/acpi/actbl1.h | 6 ++++++ 1 file changed, 6 insertions(+)
include/acpi/actbl1.h +6 -0
@@ -1096,6 +1096,12 @@ enum acpi_einj_command_status {#define ACPI_EINJ_PLATFORM_CORRECTABLE (1<<9)#define ACPI_EINJ_PLATFORM_UNCORRECTABLE (1<<10)#define ACPI_EINJ_PLATFORM_FATAL (1<<11) +#define ACPI_EINJ_CXL_CACHE_CORRECTABLE (1<<12) +#define ACPI_EINJ_CXL_CACHE_UNCORRECTABLE (1<<13) +#define ACPI_EINJ_CXL_CACHE_FATAL (1<<14) +#define ACPI_EINJ_CXL_MEM_CORRECTABLE (1<<15) +#define ACPI_EINJ_CXL_MEM_UNCORRECTABLE (1<<16) +#define ACPI_EINJ_CXL_MEM_FATAL (1<<17)#define ACPI_EINJ_VENDOR_DEFINED (1<<31)/*******************************************************************************
Signature · 1 line
2.34.1
Re: [PATCH v10 2/5] ACPI: Add CXL protocol error defines
Quoted 6 lines · Ben Cheatham
Add CXL protocol error defines to
include/actbl1.h.Signed-off-by Ben Cheatham <[email protected]>---
I made a pull request for this support in the ACPICA project which has
Hmm, it has been a few months with no movement and nothing in v6.8.
Quoted 2 lines
been accepted (link below), so this patch is temporary and I expect it to be dropped once the kernel updates from ACPICA.
Ben, not sure what is happening with the ACPICA process right now, but since the ACPI_EINJ driver is the only consumer of these defines I would say that in the spirit of the policy in "include/linux/pci_ids.h":
"Do not add new entries to this file unless the definitions
are shared between multiple drivers."
...go ahead and defines these locally in einj.c.
#ifndef ACPI_EINJ_CXL_CACHE_CORRECTABLE
#define ACPI_EINJ_CXL_CACHE_CORRECTABLE (1<<12) #define ACPI_EINJ_CXL_CACHE_UNCORRECTABLE (1<<13) #define ACPI_EINJ_CXL_CACHE_FATAL (1<<14) #define ACPI_EINJ_CXL_MEM_CORRECTABLE (1<<15) #define ACPI_EINJ_CXL_MEM_UNCORRECTABLE (1<<16) #define ACPI_EINJ_CXL_MEM_FATAL (1<<17)
#endif
Then you can delete them later if they ever get duplicated in actbl1.h.
Re: [PATCH v10 2/5] ACPI: Add CXL protocol error defines
Quoted 29 lines · Dan Williams
Quoted 6 lines · Ben Cheatham
Add CXL protocol error defines to
include/actbl1.h.Signed-off-by Ben Cheatham <[email protected]>---
I made a pull request for this support in the ACPICA project which has
Hmm, it has been a few months with no movement and nothing in v6.8.
Quoted 2 lines
been accepted (link below), so this patch is temporary and I expect it to be dropped once the kernel updates from ACPICA.
Ben, not sure what is happening with the ACPICA process right now, but since the
ACPI_EINJdriver is the only consumer of these defines I would say that in the spirit of the policy in "include/linux/pci_ids.h":"Do not add new entries to this file unless the definitions
are shared between multiple drivers."...go ahead and defines these locally in
einj.c.#ifndef
ACPI_EINJ_CXL_CACHE_CORRECTABLE#define ACPI_EINJ_CXL_CACHE_CORRECTABLE (1<<12) #define ACPI_EINJ_CXL_CACHE_UNCORRECTABLE (1<<13) #define ACPI_EINJ_CXL_CACHE_FATAL (1<<14) #define ACPI_EINJ_CXL_MEM_CORRECTABLE (1<<15) #define ACPI_EINJ_CXL_MEM_UNCORRECTABLE (1<<16) #define ACPI_EINJ_CXL_MEM_FATAL (1<<17)
Quoted 3 lines
#endif
Then you can delete them later if they ever get duplicated in
actbl1.h.
Sorry for the late response, but will do! I should be able to send a v11 this afternoon with this change.
Thanks,
Ben
PATCH v10 3/5 EINJ: Migrate to a platform driver
In reply to Ben CheathamChange the EINJ module to install a platform device/driver on module init and move the module init() and exit() functions to driver probe and remove. This change allows the EINJ module to load regardless of whether setting up EINJ succeeds, which allows dependent modules to still load (i.e. the CXL core).
Since EINJ may no longer be initialized when the module loads, any functions that are called from dependent/external modules should check the einj_initialized variable before calling any EINJ functions.
--- drivers/acpi/apei/einj.c | 44 ++++++++++++++++++++++++++++++++++++++-- 1 file changed, 42 insertions(+), 2 deletions(-)
drivers/acpi/apei/einj.c +42 -2
@@ -21,6 +21,7 @@#include <linux/nmi.h> #include <linux/delay.h> #include <linux/mm.h> +#include <linux/platform_device.h> #include <asm/unaligned.h> #include "apei-internal.h"
@@ -136,6 +137,11 @@ static struct apei_exec_ins_type einj_ins_type[] = {*/ static DEFINE_MUTEX (einj_mutex); +/* +* Exported APIs use this flag to exit early if einj_probe() failed. +*/ +static bool einj_initialized__ro_after_init ; +static void *einj_param;static void einj_exec_ctx_init (struct apei_exec_context *ctx)
@@ -684,7 +690,7 @@ static int einj_check_table(struct acpi_table_einj *einj_tab)return 0 ; } -static int __init einj_init (void ) +static int __init einj_probe (struct platform_device *pdev) {int rc;acpi_status status;
@@ -782,7 +788,7 @@ static int __init einj_init(void)return rc; } -static void __exit einj_exit(void) +static void __exit einj_remove(struct platform_device *pdev) {struct apei_exec_context ctx;
@@ -801,6 +807,40 @@ static void __exit einj_exit(void)acpi_put_table ((struct acpi_table_header *)einj_tab); } +static struct platform_device *einj_dev; +struct platform_driver einj_driver = { + .remove_new = einj_remove, + .driver = { + .name ="acpi-einj" , + }, +}; + +static int __init einj_init (void ) +{ +struct platform_device_info einj_dev_info = { + .name ="acpi-einj" , + .id =-1 , + }; +int rc; + + einj_dev =platform_device_register_full (&einj_dev_info); +if (IS_ERR_OR_NULL (einj_dev)) +return PTR_ERR (einj_dev); + + rc =platform_driver_probe (&einj_driver, einj_probe); + einj_initialized = rc ==0 ; + +return 0 ; +} + +static void __exit einj_exit(void) +{ +if (einj_initialized) +platform_driver_unregister (&einj_driver); + +platform_device_del (einj_dev); +} +module_init (einj_init);module_exit (einj_exit);
Signature · 1 line
2.34.1
PATCH v10 4/5 cxl/core, EINJ: Add EINJ CXL debugfs files and EINJ helper functions
In reply to Ben CheathamImplement CXL helper functions in the EINJ module for getting/injecting available CXL protocol error types and export them to sysfs under kernel/debug/cxl.
The kernel/debug/cxl/einj_types file will print the available CXL protocol errors in the same format as the available_error_types file provided by the EINJ module. The kernel/debug/cxl/$dport_dev/einj_inject is functionally the same as the error_type and error_inject files provided by the EINJ module, i.e.:
writing an error type into $dport_dev/einj_inject will inject said error type into the CXL dport represented by $dport_dev.
--- Documentation/ABI/testing/debugfs-cxl | 22 ++++ MAINTAINERS | 1 + drivers/acpi/apei/einj.c | 150 ++++++++++++++++++++++++-- drivers/cxl/core/port.c | 39 +++++++ include/linux/einj-cxl.h | 45 ++++++++ 5 files changed, 247 insertions(+), 10 deletions(-) create mode 100644 include/linux/einj-cxl.h
Documentation/ABI/testing/debugfs-cxl +22 -0
@@ -33,3 +33,25 @@ Description: device cannot clear poison from the address, -ENXIO is returned. The clear_poison attribute is only visible for devices supporting the capability. + +What: /sys/kernel/debug/cxl/einj_types +Date: January, 2024 +KernelVersion: v6.9 +Contact: [email protected] +Description: + (RO) Prints the CXL protocol error types made available by + the platform in the format "0x<error number> <error type>". + The <error number> can be written to einj_inject to inject + <error type> into a chosen dport. + +What: /sys/kernel/debug/cxl/$dport_dev/einj_inject +Date: January, 2024 +KernelVersion: v6.9 +Contact: [email protected] +Description: + (WO) Writing an integer to this file injects the corresponding + CXL protocol error into $dport_dev ($dport_dev will be a device + name from /sys/bus/pci/devices). The integer to type mapping for + injection can be found by reading from einj_types. If the dport + was enumerated in RCH mode, a CXL 1.1 error is injected, otherwise + a CXL 2.0 error is injected.
MAINTAINERS +1 -0
@@ -5246,6 +5246,7 @@ L: [email protected] S: Maintained F: drivers/cxl/ F: include/uapi/linux/cxl_mem.h +F: include/linux/einj-cxl.h F: tools/testing/cxl/ COMPUTE EXPRESS LINK PMU (CPMU)
drivers/acpi/apei/einj.c +140 -10
210 lines not on this page · raw message
drivers/cxl/core/port.c +39 -0
@@ -3,6 +3,7 @@#include <linux/platform_device.h> #include <linux/memregion.h> #include <linux/workqueue.h> +#include <linux/einj-cxl.h> #include <linux/debugfs.h> #include <linux/device.h> #include <linux/module.h>
@@ -797,6 +798,37 @@ static int cxl_dport_setup_regs(struct device *host, struct cxl_dport *dport,return rc; } +DEFINE_SHOW_ATTRIBUTE (einj_cxl_available_error_type); + +static int cxl_einj_inject (void *data,u64 type) +{ +struct cxl_dport *dport = data; + +if (dport->rch) +return einj_cxl_inject_rch_error (dport->rcrb.base, type); + +return einj_cxl_inject_error (to_pci_dev (dport->dport_dev), type); +} +DEFINE_DEBUGFS_ATTRIBUTE (cxl_einj_inject_fops,NULL , cxl_einj_inject,"%llx\n" ); + +static void cxl_debugfs_create_dport_dir (struct cxl_dport *dport) +{ +struct dentry *dir; + +/* +* dport_dev needs to be a PCIe port for CXL 2.0+ ports because +* EINJ expects a dport SBDF to be specified for 2.0 error injection. +*/ +if (!einj_is_initialized () || + (!dport->rch && !dev_is_pci (dport->dport_dev))) +return ; + + dir =cxl_debugfs_create_dir (dev_name (dport->dport_dev)); + +debugfs_create_file ("einj_inject" ,0200 , dir, dport, + &cxl_einj_inject_fops); +} + static struct cxl_port *__devm_cxl_add_port(struct device *host, struct device *uport_dev, resource_size_t component_reg_phys,
@@ -1144,6 +1176,8 @@ __devm_cxl_add_dport(struct cxl_port *port, struct device *dport_dev,if (dev_is_pci (dport_dev)) dport->link_latency =cxl_pci_get_latency (to_pci_dev (dport_dev)); +cxl_debugfs_create_dport_dir (dport); +return dport; }
@@ -2220,6 +2254,11 @@ static __init int cxl_core_init(void) cxl_debugfs =debugfs_create_dir ("cxl" ,NULL ); +if (einj_is_initialized ()) { +debugfs_create_file ("einj_types" ,0400 , cxl_debugfs,NULL , + &einj_cxl_available_error_type_fops); + } +cxl_mbox_init (); rc =cxl_memdev_init ();
include/linux/einj-cxl.h new file +45 -0
45 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH v10 5/5 EINJ, Documentation: Update EINJ kernel doc
In reply to Ben CheathamUpdate EINJ kernel document to include how to inject CXL protocol error types, build the kernel to include CXL error types, and give an example injection.
--- .../firmware-guide/acpi/apei/einj.rst | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)
Documentation/firmware-guide/acpi/apei/einj.rst +19 -0
@@ -181,6 +181,25 @@ You should see something like this in dmesg:: [22715.834759] EDAC sbridge MC3: PROCESSOR 0:306e7 TIME 1422553404 SOCKET 0 APIC 0 [22716.616173] EDAC MC3: 1 CE memory read error on CPU_SrcID#0_Channel#0_DIMM#0 (channel:0 slot:0 page:0x12345 offset:0x0 grain:32 syndrome:0x0 - area:DRAM err_code:0001:0090 socket:0 channel_mask:1 rank:0) +CXL error types are supported from ACPI 6.5 onwards. These error types +are not available in the legacy interface at /sys/kernel/debug/apei/einj, +and are instead at /sys/kernel/debug/cxl/. There is a file under debug/cxl +called "einj_type" that is analogous to available_error_type under debug/cxl. +There is also a "einj_inject" file in each $dport_dev directory under debug/cxl +that will inject a given error into the dport represented by $dport_dev. +For example, to inject a CXL.mem protocol correctable error into +$dport_dev=pci0000:0c:: + + # cd /sys/kernel/debug/cxl/ + # cat einj_type # See which error can be injected + 0x00008000 CXL.mem Protocol Correctable + 0x00010000 CXL.mem Protocol Uncorrectable non-fatal + 0x00020000 CXL.mem Protocol Uncorrectable fatal + # cd 0000:e0:01.1 # Navigate to dport to inject into + # echo 0x8000 > einj_inject # Inject error + +To use CXL error types, ``CONFIG_CXL_EINJ`` will need to be enabled. + Special notes for injection into SGX enclaves: There may be a separate BIOS setup option to enable SGX injection.
Signature · 1 line
2.34.1
[ add ACPI APEI reviewers ]
Quoted 40 lines · Ben Cheatham
v10 Changes:
- Fixups in EINJ module initializtion (Dan)
- Addinclude/linux/einj-cxl.htoMAINTAINERSunder CXL subsystem (Dan)
- Replace usages ofIS_ENABLED(CONFIG_CXL_EINJ)with neweinj_is_initialized()function incxl/core/port.c(Dan)
- Fix typo in EINJ documentation (Dan)v9 Changes:
- Remove wrapper__initfunction in EINJ module and use a platform driver instead (Dan & Jonathan)
- Movecxl/einj.htoinclude/linux/einj-cxl.h(Dan)
- Rework header file to remove cxl_einj_* functions (Dan)
- AddIS_ENABLED()guards around EINJ debugfs functionality incxl/core/port.c(Dan)
- Rework cxl/Kconfig to be more concise (Dan)
- Small fixups to cxl-debugfs documentation (Dan)
- Add check to not add einj files to CXL 2.0+ dports that aren't represented by apci_dev
- Bump version number in debugfs-cxl documentationThe new CXL error types will use the Memory Address field in the
SET_ERROR_TYPE_WITH_ADDRESSstructure in order to target a CXL 1.1 compliant memory-mapped downstream port. The value of the memory address will be in the port's MMIO range, and it will not represent physical (normal or persistent) memory.Add the functionality for injecting CXL 1.1 errors to the EINJ module, but not through the EINJ legacy interface under
/sys/kernel/debug/apei/einj. Instead, make the error types available under/sys/kernel/debug/cxl. This allows for validating the MMIO address for a CXL 1.1 error type while also not making the user responsible for finding it.Ben Cheatham (5):
cxl, ACPI, APEI, EINJ: AddCXL_EINJKconfigoption
ACPI: Add CXL protocol error defines
EINJ: Migrate to a platform driver
cxl/core, EINJ: Add EINJ CXL debugfs files and EINJ helper functions EINJ, Documentation: Update EINJ kernel doc
Quoted 10 lines
Documentation/ABI/testing/debugfs-cxl | 22 ++ .../firmware-guide/acpi/apei/einj.rst | 19 ++ MAINTAINERS | 1 + drivers/acpi/apei/einj.c | 194 ++++++++++++++++-- drivers/cxl/Kconfig | 12 ++ drivers/cxl/core/port.c | 39 ++++ include/acpi/actbl1.h | 6 + include/linux/einj-cxl.h | 45 ++++ 8 files changed, 326 insertions(+), 12 deletions(-) create mode 100644 include/linux/einj-cxl.h
Looks good to me, for the series:
Next steps are for the ACPICA process to pull in the protocol error definitions so that patch2 can be dropped, and for one of the APEI reviewers to ack the einj.c changes.
As for merging, the balance of the changes are in drivers/acpi/, so I am happy if this goes through acpi.git, but otherwise circle back in a couple weeks after the merge window completes to track this into linux-next. Thanks for all the updates, Ben!