Add L1 and L2 error detection for A72

This is an attempt to revive [v5] series. I have attempted to address comments and suggestions from Marc Zyngier since [v5]. Additionally, I have limited the support only for A72 processors per [v8] discussion. Testing the driver on a problematic A72 SoC has led to the detection of Correctable Errors (CEs). Below are logs captured from the problematic SoC during various boot instances.

[  876.896022] EDAC DEVICE0: CE: cortex-arm64-edac instance: cpu2 block: L1 count: 1 'L1-D Data RAM correctable error(s) on CPU 2'

[ 3700.978086] EDAC DEVICE0: CE: cortex-arm64-edac instance: cpu2 block: L1 count: 1 'L1-D Data RAM correctable error(s) on CPU 2'

[  976.956158] EDAC DEVICE0: CE: cortex-arm64-edac instance: cpu2 block: L1 count: 1 'L1-D Data RAM correctable error(s) on CPU 2'

[ 1427.933606] EDAC DEVICE0: CE: cortex-arm64-edac instance: cpu2 block: L1 count: 1 'L1-D Data RAM correctable error(s) on CPU 2'

[  192.959911] EDAC DEVICE0: CE: cortex-arm64-edac instance: cpu2 block: L1 count: 1 'L1-D Data RAM correctable error(s) on CPU 2'

Testing our product kernel involved adding the 'edac-enabled' property to CPU nodes in the DTS. For mainline sanity checks, we tested under QEMU by extracting the default DTB and modifying the DTS to include the 'edac-enabled' property. We then verified the presence of /sysfs nodes for CE and UE counts for the emulated A72 CPUs.

Our primary focus is on A72. We have a significant number of A72-based systems in our fleet, and timely replacements via monitoring CEs will be instrumental in managing them effectively.

I am eager to hear your suggestions and feedback on this series.

Thanks,
Vijay

[v5] https://lore.kernel.org/all/[email protected]/#t
[v6] https://lore.kernel.org/all/[email protected]/
[v7] https://lore.kernel.org/all/[email protected]/#t
[v8] https://lore.kernel.org/all/[email protected]/
[v9] https://lore.kernel.org/all/[email protected]/
[v10] https://lore.kernel.org/all/[email protected]/
[v11] https://lore.kernel.org/all/[email protected]/

Changes since v11:
- driver file name change: edac_a72.c -> a72_edac.c (Boris)
- align defines (Boris)
- the device is registered before driver
- in module exit unregister the device

Changes since v10:
- edac_a72.c: copyright line add (Jonathan)
- cpus.yaml: drop stale comment line (Krzysztof)
- added "Reviewed-by" tags

Changes since v9:
- commit title, message and prefix update (Boris)
- fix spelling in Kconfig help text (Boris)
- prepared patches against edac-for-next (Boris)
- struct naming update from "merrsr" to "mem_err_synd_reg" (Boris)
- grouping of all defines (Boris)
- function variable declarations in reverse fir tree order (Boris)
- simplify naming of static functions (Boris)
- protect smp_call_function_single() against CPU hotplug (Boris)
- "CPU" in visible string instead of "cpu" (Boris)
- error message reflects "edac_a72" driver name (Boris)
- fixed the issues with device_node release using scope exit (Jonathan)
- of_cpu_device_node_get() instead of of_get_cpu_node() (Jonathan)
- make dt-binding update applicable only for A72 using if/then schema (Rob)

Changes since v8:
- removed support for A53 and A57
- added entry to MAINTAINERS
- added missing module exit point to enable unload

Changes since v7:
- v5 was based on the internal product kernel, identified following upon review
- correct format specifier to print CPUID/WAY
- removal of unused dynamic attributes for edac_device_alloc_ctl_info()
- driver remove callback return type is void

Changes since v6:
- restore the change made in [v5] to clear CPU/L2 syndrome registers back to read_errors()
- upon detecting a valid error, clear syndrome registers immediately to avoid clobbering between the read and write (Marc)
- NULL return check for of_get_cpu_node() (Tyler)
- of_node_put() to avoid refcount issue (Tyler)
- quotes are dropped in yaml file (Krzysztof)

Changes since v5:
- rebase on v6.15-rc1
- the syndrome registers for CPU/L2 memory errors are cleared only upon detecting an error and an isb() after for synchronization (Marc)
- "edac-enabled" hunk moved to initial patch to avoid breaking virtual environments (Marc)
- to ensure compatibility across all three families, we are not reporting "L1 Dirty RAM," documented only in the A53 TRM
- above prompted changing default CPU L1 error meesage from "unknown" to "Unspecified"
- capturing CPUID/WAY information in L2 memory error log (Marc)
- module license from "GPL v2" to "GPL" (checkpatch.pl warning)
- extend support for A72

Sascha Hauer (2):
EDAC: Add EDAC driver for ARM Cortex A72 cores
dt-bindings: arm: cpus: Add edac-enabled property

 .../devicetree/bindings/arm/cpus.yaml         |  17 ++
 MAINTAINERS                                   |   7 +
 drivers/edac/Kconfig                          |   8 +
 drivers/edac/Makefile                         |   1 +
 drivers/edac/a72_edac.c                       | 230 ++++++++++++++++++
 5 files changed, 263 insertions(+)
 create mode 100644 drivers/edac/a72_edac.c

base-commit: 6beda760db7eb6b06cdbf77d4749af0bf9aca1e0

2.49.0

From: Sascha Hauer <[email protected]>

The driver is designed to support error detection and reporting for Cortex A72 cores, specifically within their L1 and L2 cache systems. The errors are detected by reading CPU/L2 memory error syndrome registers.

Unfortunately there is no robust way to inject errors into the caches, so this driver doesn't contain any code to actually test it. It has been tested though with code taken from an older version [1] of this driver. For reasons stated in thread [1], the error injection code is not suitable for mainline, so it is removed from the driver.

[1] https://lore.kernel.org/all/[email protected]/#t

---
 MAINTAINERS             |   7 ++
 drivers/edac/Kconfig    |   8 ++
 drivers/edac/Makefile   |   1 +
 drivers/edac/a72_edac.c | 227 ++++++++++++++++++++++++++++++++++++++++
 4 files changed, 243 insertions(+)
 create mode 100644 drivers/edac/a72_edac.c

index c3f7fbd0d67a..d237ab92fdb0 100644
@@ -8588,6 +8588,13 @@ F:	Documentation/driver-api/edac.rst
 F:	drivers/edac/
 F:	include/linux/edac.h
 
+EDAC-A72
+M:	Vijay Balakrishna <[email protected]>
+M:	Tyler Hicks <[email protected]>
+L:	[email protected]
+S:	Supported
+F:	drivers/edac/a72_edac.c
+
 EDAC-DMC520
 M:	Lei Wang <[email protected]>
 L:	[email protected]

index 19ad3c3b675d..b824472208c4 100644
@@ -576,4 +576,12 @@ config EDAC_LOONGSON
 	  errors (CE) only. Loongson-3A5000/3C5000/3D5000/3A6000/3C6000
 	  are compatible.
 
+config EDAC_CORTEX_A72
+	tristate "ARM Cortex A72"
+	depends on ARM64
+	help
+	  Support for L1/L2 cache error detection for ARM Cortex A72 processor.
+	  The detected and reported errors are from reading CPU/L2 memory error
+	  syndrome registers.
+
 endif # EDAC

index a8f2d8f6c894..c1736f264320 100644
@@ -88,3 +88,4 @@ obj-$(CONFIG_EDAC_NPCM)			+= npcm_edac.o
 obj-$(CONFIG_EDAC_ZYNQMP)		+= zynqmp_edac.o
 obj-$(CONFIG_EDAC_VERSAL)		+= versal_edac.o
 obj-$(CONFIG_EDAC_LOONGSON)		+= loongson_edac.o
+obj-$(CONFIG_EDAC_CORTEX_A72)		+= a72_edac.o

2.49.0

From: Sascha Hauer <[email protected]>

Some ARM Cortex CPUs including A72 have Error Detection And Correction (EDAC) support on their L1 and L2 caches. This is implemented in implementation defined registers, so usage of this functionality is not safe in virtualized environments or when EL3 already uses these registers. This patch adds a edac-enabled flag which can be explicitly set when EDAC can be used.

---
 Documentation/devicetree/bindings/arm/cpus.yaml | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

index 2e9ab9583005..67eaa6df5959 100644
@@ -352,6 +352,12 @@ properties:
     $ref: /schemas/types.yaml#/definitions/phandle
     description: Link to Mediatek Cache Coherent Interconnect
 
+  edac-enabled:
+    $ref: /schemas/types.yaml#/definitions/flag
+    description:
+      A72 CPUs support Error Detection And Correction (EDAC) on their L1 and
+      L2 caches. This flag marks this function as usable.
+
   qcom,saw:
     $ref: /schemas/types.yaml#/definitions/phandle
     description:

@@ -398,6 +404,17 @@ properties:
 allOf:
   - $ref: /schemas/cpu.yaml#
   - $ref: /schemas/opp/opp-v1.yaml#
+  - if:
+      not:
+        properties:
+          compatible:
+            contains:
+              const: arm,cortex-a72
+    then:
+      # Allow edac-enabled only for Cortex A72
+      properties:
+        edac-enabled: false
+
   - if:
       # If the enable-method property contains one of those values
       properties:

2.49.0

Applied, thanks.

Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette