Add L1 and L2 error detection for A53, A57 and A72

Hello,

This is an attempt to revive [v5] series. I have attempted to address comments and suggestions from Marc Zyngier since [v5]. Additionally, I have extended support for A72 processors. Testing the driver on a problematic A72 SoC has led to the detection of Correctable Errors (CEs). Below are logs captured from the problematic SoC during various boot instances.

[  876.896022] EDAC DEVICE0: CE: cortex-arm64-edac instance: cpu2 block: L1 count: 1 'L1-D Data RAM correctable error(s) on CPU 2'

[ 3700.978086] EDAC DEVICE0: CE: cortex-arm64-edac instance: cpu2 block: L1 count: 1 'L1-D Data RAM correctable error(s) on CPU 2'

[  976.956158] EDAC DEVICE0: CE: cortex-arm64-edac instance: cpu2 block: L1 count: 1 'L1-D Data RAM correctable error(s) on CPU 2'

[ 1427.933606] EDAC DEVICE0: CE: cortex-arm64-edac instance: cpu2 block: L1 count: 1 'L1-D Data RAM correctable error(s) on CPU 2'

[  192.959911] EDAC DEVICE0: CE: cortex-arm64-edac instance: cpu2 block: L1 count: 1 'L1-D Data RAM correctable error(s) on CPU 2'

Our primary focus is on A72. We have a significant number of A72-based systems in our fleet, and timely replacements via monitoring CEs will be instrumental in managing them effectively.

I am eager to hear your suggestions and feedback on this series.

Thanks,
Vijay

[v5] https://lore.kernel.org/all/[email protected]/#t
[v6] https://lore.kernel.org/all/[email protected]/
[v7] https://lore.kernel.org/all/[email protected]/#t

Changes since v7:
- v5 was based on the internal product kernel, identified following upon review
- correct format specifier to print CPUID/WAY
- removal of unused dynamic attributes for edac_device_alloc_ctl_info()
- driver remove callback return type is void

Changes since v6:
- restore the change made in [v5] to clear CPU/L2 syndrome registers back to read_errors()
- upon detecting a valid error, clear syndrome registers immediately to avoid clobbering between the read and write (Marc)
- NULL return check for of_get_cpu_node() (Tyler)
- of_node_put() to avoid refcount issue (Tyler)
- quotes are dropped in yaml file (Krzysztof)

Changes since v5:
- rebase on v6.15-rc1
- the syndrome registers for CPU/L2 memory errors are cleared only upon detecting an error and an isb() after for synchronization (Marc)
- "edac-enabled" hunk moved to initial patch to avoid breaking virtual environments (Marc)
- to ensure compatibility across all three families, we are not reporting "L1 Dirty RAM," documented only in the A53 TRM
- above prompted changing default CPU L1 error meesage from "unknown" to "Unspecified"
- capturing CPUID/WAY information in L2 memory error log (Marc)
- module license from "GPL v2" to "GPL" (checkpatch.pl warning)
- extend support for A72

Changes since v4:
- Rebase on v5.12-rc5

Changes since v3:
- Add edac-enabled property to make EDAC 3support optional

Changes since v2:
- drop usage of virtual dt node (Robh)
- use read_sysreg_s instead of open coded variant (James Morse)
- separate error retrieving from error reporting
- use smp_call_function_single rather than smp_call_function_single_async
- make driver single instance and register all 'cpu' hierarchy up front once

Changes since v1:
- Split dt-binding into separate patch
- Sort local function variables in reverse-xmas tree order
- drop unnecessary comparison and make variable bool

Sascha Hauer (2):
drivers/edac: Add L1 and L2 error detection for A53, A57 and A72
dt-bindings: arm: cpus: Add edac-enabled property

 .../devicetree/bindings/arm/cpus.yaml         |   6 +
 drivers/edac/Kconfig                          |   9 +
 drivers/edac/Makefile                         |   1 +
 drivers/edac/cortex_arm64_l1_l2.c             | 229 ++++++++++++++++++
 4 files changed, 245 insertions(+)
 create mode 100644 drivers/edac/cortex_arm64_l1_l2.c

base-commit: 59c9ab3e8cc7f56cd65608f6e938b5ae96eb9cd2

2.49.0

From: Sascha Hauer <[email protected]>

The Cortex A53, A57 and A72 cores have error detection capabilities for the L1/L2 Caches, this patch adds a driver for them. The selected errors to detect/report have compatible bit assignments concerning CPU/L2 memory error syndrome registers.

Unfortunately there is no robust way to inject errors into the caches, so this driver doesn't contain any code to actually test it. It has been tested though with code taken from an older version [1] of this driver. For reasons stated in thread [1], the error injection code is not suitable for mainline, so it is removed from the driver.

[1] https://lore.kernel.org/all/[email protected]/#t

---
 drivers/edac/Kconfig              |   9 ++
 drivers/edac/Makefile             |   1 +
 drivers/edac/cortex_arm64_l1_l2.c | 229 ++++++++++++++++++++++++++++++
 3 files changed, 239 insertions(+)
 create mode 100644 drivers/edac/cortex_arm64_l1_l2.c

index 19ad3c3b675d..20c15a18437f 100644
@@ -576,4 +576,13 @@ config EDAC_LOONGSON
 	  errors (CE) only. Loongson-3A5000/3C5000/3D5000/3A6000/3C6000
 	  are compatible.
 
+config EDAC_CORTEX_ARM64_L1_L2
+	tristate "ARM Cortex A72/A57/A53"
+	depends on ARM64
+	help
+	  Support for L1/L2 cache error detection is provided for ARM Cortex
+	  A72, A57, and A53 processors. The selected subset of errors is common
+	  across these three MPCore processors, featuring compatible bit
+	  assignments in their CPU/L2 memory error syndrome registers.
+
 endif # EDAC

index a8f2d8f6c894..157062410e61 100644
@@ -88,3 +88,4 @@ obj-$(CONFIG_EDAC_NPCM)			+= npcm_edac.o
 obj-$(CONFIG_EDAC_ZYNQMP)		+= zynqmp_edac.o
 obj-$(CONFIG_EDAC_VERSAL)		+= versal_edac.o
 obj-$(CONFIG_EDAC_LOONGSON)		+= loongson_edac.o
+obj-$(CONFIG_EDAC_CORTEX_ARM64_L1_L2)	+= cortex_arm64_l1_l2.o

2.49.0

From: Sascha Hauer <[email protected]>

Some ARM Cortex CPUs like the A53, A57 and A72 have Error Detection And Correction (EDAC) support on their L1 and L2 caches. This is implemented in implementation defined registers, so usage of this functionality is not safe in virtualized environments or when EL3 already uses these registers. This patch adds a edac-enabled flag which can be explicitly set when EDAC can be used.

---
 Documentation/devicetree/bindings/arm/cpus.yaml | 6 ++++++
 1 file changed, 6 insertions(+)

index 2e666b2a4dcd..d1dc0a843d07 100644
@@ -331,6 +331,12 @@ properties:
       corresponding to the index of an SCMI performance domain provider, must be
       "perf".
 
+  edac-enabled:
+    $ref: /schemas/types.yaml#/definitions/flag
+    description:
+      Some CPUs support Error Detection And Correction (EDAC) on their L1 and
+      L2 caches. This flag marks this function as usable.
+
   qcom,saw:
     $ref: /schemas/types.yaml#/definitions/phandle
     description: |

2.49.0

I'd like to hear from ARM folks here, whether this makes sense to have still.

Then zap the support for the other CPUs as supporting those is futile.

cortex_arm64_l1_l2.c - I don't want an EDAC driver per RAS functional unit. Call this edac_a72 or whatever, which will contain all A72 RAS functionality support. ARM folks will give you a good idea here if you don't have.

Also, I'd need at least a reviewer entry to MAINTAINERS for patches to this driver because you'll be the only ones testing this as you have vested interest in this working.

The dt patch needs a reviewed-by too.

Once that is addressed, I'll take a look.

Thx.

Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Thank you, Boris.

I will soon be posting a new series featuring only A72 functionality. Could the ARM folks on Cc please comment on additional changes we can include for A72?

Tyler and I can serve as joint reviewers, and I'll update the MAINTAINERS file accordingly.

Krzysztof, I would appreciate your reviewed-by for the DT patch when I post the next version.

Thanks,
Vijay