Add L1 and L2 error detection for A72
Subsystems (2)
v11 cover Add L1 and L2 error detection for A72
This is an attempt to revive [v5] series. I have attempted to address comments and suggestions from Marc Zyngier since [v5]. Additionally, I have limited the support only for A72 processors per [v8] discussion. Testing the driver on a problematic A72 SoC has led to the detection of Correctable Errors (CEs). Below are logs captured from the problematic SoC during various boot instances.
[ 876.896022] EDAC DEVICE0: CE: cortex-arm64-edac instance: cpu2 block: L1 count: 1 'L1-D Data RAM correctable error(s) on CPU 2'
[ 3700.978086] EDAC DEVICE0: CE: cortex-arm64-edac instance: cpu2 block: L1 count: 1 'L1-D Data RAM correctable error(s) on CPU 2'
[ 976.956158] EDAC DEVICE0: CE: cortex-arm64-edac instance: cpu2 block: L1 count: 1 'L1-D Data RAM correctable error(s) on CPU 2'
[ 1427.933606] EDAC DEVICE0: CE: cortex-arm64-edac instance: cpu2 block: L1 count: 1 'L1-D Data RAM correctable error(s) on CPU 2'
[ 192.959911] EDAC DEVICE0: CE: cortex-arm64-edac instance: cpu2 block: L1 count: 1 'L1-D Data RAM correctable error(s) on CPU 2'
Testing our product kernel involved adding the 'edac-enabled' property to CPU nodes in the DTS. For mainline sanity checks, we tested under QEMU by extracting the default DTB and modifying the DTS to include the 'edac-enabled' property. We then verified the presence of /sysfs nodes for CE and UE counts for the emulated A72 CPUs.
Our primary focus is on A72. We have a significant number of A72-based systems in our fleet, and timely replacements via monitoring CEs will be instrumental in managing them effectively.
I am eager to hear your suggestions and feedback on this series.
Thanks,
Vijay
[v5] https://lore.kernel.org/all/[email protected]/#t
[v6] https://lore.kernel.org/all/[email protected]/
[v7] https://lore.kernel.org/all/[email protected]/#t
[v8] https://lore.kernel.org/all/[email protected]/
[v9] https://lore.kernel.org/all/[email protected]/
[v10] https://lore.kernel.org/all/[email protected]/
Changes since v10:
- edac_a72.c: copyright line add (Jonathan)
- cpus.yaml: drop stale comment line (Krzysztof)
- added "Reviewed-by" tags
Changes since v9:
- commit title, message and prefix update (Boris)
- fix spelling in Kconfig help text (Boris)
- prepared patches against edac-for-next (Boris)
- struct naming update from "merrsr" to "mem_err_synd_reg" (Boris)
- grouping of all defines (Boris)
- function variable declarations in reverse fir tree order (Boris)
- simplify naming of static functions (Boris)
- protect smp_call_function_single() against CPU hotplug (Boris)
- "CPU" in visible string instead of "cpu" (Boris)
- error message reflects "edac_a72" driver name (Boris)
- fixed the issues with device_node release using scope exit (Jonathan)
- of_cpu_device_node_get() instead of of_get_cpu_node() (Jonathan)
- make dt-binding update applicable only for A72 using if/then schema (Rob)
Changes since v8:
- removed support for A53 and A57
- added entry to MAINTAINERS
- added missing module exit point to enable unload
Changes since v7:
- v5 was based on the internal product kernel, identified following upon review
- correct format specifier to print CPUID/WAY
- removal of unused dynamic attributes for edac_device_alloc_ctl_info()
- driver remove callback return type is void
Changes since v6:
- restore the change made in [v5] to clear CPU/L2 syndrome registers back to read_errors()
- upon detecting a valid error, clear syndrome registers immediately to avoid clobbering between the read and write (Marc)
- NULL return check for of_get_cpu_node() (Tyler)
- of_node_put() to avoid refcount issue (Tyler)
- quotes are dropped in yaml file (Krzysztof)
Changes since v5:
- rebase on v6.15-rc1
- the syndrome registers for CPU/L2 memory errors are cleared only upon detecting an error and an isb() after for synchronization (Marc)
- "edac-enabled" hunk moved to initial patch to avoid breaking virtual environments (Marc)
- to ensure compatibility across all three families, we are not reporting "L1 Dirty RAM," documented only in the A53 TRM
- above prompted changing default CPU L1 error meesage from "unknown" to "Unspecified"
- capturing CPUID/WAY information in L2 memory error log (Marc)
- module license from "GPL v2" to "GPL" (checkpatch.pl warning)
- extend support for A72
Sascha Hauer (2):
EDAC: Add EDAC driver for ARM Cortex A72 cores
dt-bindings: arm: cpus: Add edac-enabled property
.../devicetree/bindings/arm/cpus.yaml | 50 ++-- MAINTAINERS | 7 + drivers/edac/Kconfig | 8 + drivers/edac/Makefile | 1 + drivers/edac/edac_a72.c | 230 ++++++++++++++++++ 5 files changed, 280 insertions(+), 16 deletions(-) create mode 100644 drivers/edac/edac_a72.c
base-commit: 855b5de2e562c07d6cda4deb08d09dc2e0e2b18d
Signature · 1 line
2.49.0
PATCH v11 1/2 EDAC: Add EDAC driver for ARM Cortex A72 cores
From: Sascha Hauer <[email protected]>
The driver is designed to support error detection and reporting for Cortex A72 cores, specifically within their L1 and L2 cache systems. The errors are detected by reading CPU/L2 memory error syndrome registers.
Unfortunately there is no robust way to inject errors into the caches, so this driver doesn't contain any code to actually test it. It has been tested though with code taken from an older version [1] of this driver. For reasons stated in thread [1], the error injection code is not suitable for mainline, so it is removed from the driver.
[1] https://lore.kernel.org/all/[email protected]/#t
--- MAINTAINERS | 7 ++ drivers/edac/Kconfig | 8 ++ drivers/edac/Makefile | 1 + drivers/edac/edac_a72.c | 230 ++++++++++++++++++++++++++++++++++++++++ 4 files changed, 246 insertions(+) create mode 100644 drivers/edac/edac_a72.c
MAINTAINERS +7 -0
@@ -8429,6 +8429,13 @@ F: Documentation/driver-api/edac.rst F: drivers/edac/ F: include/linux/edac.h +EDAC-A72 +M: Vijay Balakrishna <[email protected]> +M: Tyler Hicks <[email protected]> +L: [email protected] +S: Supported +F: drivers/edac/edac_a72.c + EDAC-DMC520 M: Lei Wang <[email protected]> L: [email protected]
drivers/edac/Kconfig +8 -0
@@ -576,4 +576,12 @@ config EDAC_LOONGSON errors (CE) only. Loongson-3A5000/3C5000/3D5000/3A6000/3C6000 are compatible. +config EDAC_CORTEX_A72 + tristate "ARM Cortex A72" + depends on ARM64 + help + Support for L1/L2 cache error detection for ARM Cortex A72 processor. + The detected and reported errors are from reading CPU/L2 memory error + syndrome registers. + endif # EDAC
drivers/edac/Makefile +1 -0
@@ -88,3 +88,4 @@ obj-$(CONFIG_EDAC_NPCM) += npcm_edac.o obj-$(CONFIG_EDAC_ZYNQMP) += zynqmp_edac.o obj-$(CONFIG_EDAC_VERSAL) += versal_edac.o obj-$(CONFIG_EDAC_LOONGSON) += loongson_edac.o +obj-$(CONFIG_EDAC_CORTEX_A72) += edac_a72.o
drivers/edac/edac_a72.c new file +230 -0
230 lines not on this page · raw message
Signature · 1 line
2.49.0
Borislav Petkov
maintainerRe: [v11 PATCH 1/2] EDAC: Add EDAC driver for ARM Cortex A72 cores
drivers/edac/Makefile obj-$(CONFIG_EDAC_NPCM) += npcm_edac.o · line 91
Quoted 9 lines · Vijay Balakrishna
drivers/edac/Makefile +1
-0@@ -88,3 +88,4 @@ obj-$(CONFIG_EDAC_NPCM) += npcm_edac.o obj-$(CONFIG_EDAC_ZYNQMP) += zynqmp_edac.o obj-$(CONFIG_EDAC_VERSAL) += versal_edac.o obj-$(CONFIG_EDAC_LOONGSON) += loongson_edac.o +obj-$(CONFIG_EDAC_CORTEX_A72) += edac_a72.o
The drivers filename format is
edac_<something>.c
So a72_edac.c
Quoted 35 lines
drivers/edac/edac_a72.c new file +29
-0@@ -0,0 +1,230 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Cortex A72 EDAC L1 and L2 cache error detection +* +* Copyright (c) 2020 Pengutronix, Sascha Hauer <[email protected]> +* Copyright (c) 2025 Microsoft Corporation, <[email protected]> +* +* Based on Code from: +* Copyright (c) 2018, NXP Semiconductor +* Author: York Sun <[email protected]> +*/ + +#include <linux/module.h> +#include <linux/of.h> +#include <linux/bitfield.h> +#include <asm/smp_plat.h> + +#include "edac_module.h" + +#define DRVNAME "edac-a72" + +#define SYS_CPUMERRSR_EL1 sys_reg(3, 1, 15, 2, 2) +#define SYS_L2MERRSR_EL1 sys_reg(3, 1, 15, 2, 3) + +#define CPUMERRSR_EL1_RAMID GENMASK(30, 24) +#define L2MERRSR_EL1_CPUID_WAY GENMASK(21, 18) + +#define CPUMERRSR_EL1_VALID BIT(31) +#define CPUMERRSR_EL1_FATAL BIT(63)
drivers/edac/edac_a72.c line 39
Quoted 10 lines
drivers/edac/edac_a72.c new file +10
-0+#define L2MERRSR_EL1_VALID BIT(31) +#define L2MERRSR_EL1_FATAL BIT(63) + +#define L1_I_TAG_RAM 0x00 +#define L1_I_DATA_RAM 0x01 +#define L1_D_TAG_RAM 0x08 +#define L1_D_DATA_RAM 0x09 +#define TLB_RAM 0x18 + +#define MESSAGE_SIZE 64
I had written
"Please group all defines together, align them vertically and then put other definitions below. Look at other drivers for inspiration."
in my previous review
Message-ID: <20250519085130.GFaCrxEnZvaoETKrao@fat_crate.local>
but seems like this got ignored.
Oh well, I'll ignore your submission too until you address *all* my review feedback.
Thx.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquettePATCH v11 2/2 dt-bindings: arm: cpus: Add edac-enabled property
In reply to Vijay BalakrishnaFrom: Sascha Hauer <[email protected]>
Some ARM Cortex CPUs including A72 have Error Detection And Correction (EDAC) support on their L1 and L2 caches. This is implemented in implementation defined registers, so usage of this functionality is not safe in virtualized environments or when EL3 already uses these registers. This patch adds a edac-enabled flag which can be explicitly set when EDAC can be used.
--- .../devicetree/bindings/arm/cpus.yaml | 50 +++++++++++++------ 1 file changed, 34 insertions(+), 16 deletions(-)
Documentation/devicetree/bindings/arm/cpus.yaml +34 -16
@@ -331,6 +331,12 @@ properties: corresponding to the index of an SCMI performance domain provider, must be "perf". + edac-enabled: + $ref: /schemas/types.yaml#/definitions/flag + description: + A72 CPUs support Error Detection And Correction (EDAC) on their L1 and + L2 caches. This flag marks this function as usable. + qcom,saw: $ref: /schemas/types.yaml#/definitions/phandle description: |
@@ -378,22 +384,34 @@ properties: formed by encoding the target CPU id into the low bits of the physical start address it should jump to. -if: - # If the enable-method property contains one of those values - properties: - enable-method: - contains: - enum: - - brcm,bcm11351-cpu-method - - brcm,bcm23550 - - brcm,bcm-nsp-smp - # and if enable-method is present - required: - - enable-method - -then: - required: - - secondary-boot-reg +allOf: + - if: + not: + properties: + compatible: + contains: + const: arm,cortex-a72 + then: + # Allow edac-enabled only for Cortex A72 + properties: + edac-enabled: false + + - if: + # If the enable-method property contains one of those values + properties: + enable-method: + contains: + enum: + - brcm,bcm11351-cpu-method + - brcm,bcm23550 + - brcm,bcm-nsp-smp + # and if enable-method is present + required: + - enable-method + + then: + required: + - secondary-boot-reg required: - device_type
Signature · 1 line
2.49.0
Borislav Petkov
maintainerQuoted 4 lines · Vijay Balakrishna
Changes since v10:
-edac_a72.c: copyright line add (Jonathan)
-cpus.yaml: drop stale comment line (Krzysztof)
- added "Reviewed-by" tags
It seems you're new to kernel development:
From: Documentation/process/submitting-patches.rst
"Don't get discouraged - or impatient
------------------------------------
After you have submitted your change, be patient and wait. Reviewers are busy people and may not get to your patch right away.
Once upon a time, patches used to disappear into the void without comment, but the development process works more smoothly than that now. You should receive comments within a week or so; if that does not happen, make sure that you have sent your patches to the right place. Wait for a minimum of one week before resubmitting or pinging reviewers - possibly longer during busy times like merge windows."
And we have a merge window right now.
Instead of sending patches every day, you should spend some time perusing the documentation about how this all works:
https://kernel.org/doc/html/latest/process/development-process.html
Thx.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteQuoted 16 lines · Borislav Petkov
Quoted 4 lines · Vijay Balakrishna
Changes since v10:
-edac_a72.c: copyright line add (Jonathan)
-cpus.yaml: drop stale comment line (Krzysztof)
- added "Reviewed-by" tagsIt seems you're new to kernel development:
From:
Documentation/process/submitting-patches.rst"Don't get discouraged - or impatient
------------------------------------After you have submitted your change, be patient and wait. Reviewers are busy people and may not get to your patch right away.
Quoted 9 lines
Once upon a time, patches used to disappear into the void without comment, but the development process works more smoothly than that now. You should receive comments within a week or so; if that does not happen, make sure that you have sent your patches to the right place. Wait for a minimum of one week before resubmitting or pinging reviewers - possibly longer during busy times like merge windows."
And we have a merge window right now.
Hi Boris,
I apologize for sending v10 and v11 in quick succession during the merge window last month. Our goal was to provide you with the latest patch series, ensuring all comments were addressed and ready for your review at your convenience. Please share your feedback so I can address it accordingly. Thank you for your time.
Vijay