Fix issues with ARM Processor CPER records

This is needed for both kernelspace and userspace properly handle ARM processor CPER events.

Patch 1 of this series fix the UEFI 2.6+ implementation of the ARM trace event, as the original implementation was incomplete. Changeset e9279e83ad1f ("trace, ras: add ARM processor error trace event") added such event, but it reports only some fields of the CPER record defined on UEFI 2.6+ appendix N, table N.16. Those are not enough actually parse such events on userspace, as not even the event type is exported.

Patch 2 fixes a compilation breakage when W=1;

Patch 3 adds a new helper function to be used by cper and ghes drivers to display CPER bitmaps;

Patch 4 fixes CPER logic according with UEFI 2.9A errata. Before it, there was no description about how processor type field was encoded. The errata defines it as a bitmask, and provides the information about how it should be encoded.

Patch 5 adds CPER functions to Kernel-doc.

This series was validated with the help of an ARM EINJ code for QEMU:

	https://gitlab.com/mchehab_kernel/qemu/-/tree/qemu_submission_v5.1?ref_type=heads

	$ ./scripts/ghes_inject.py -d arm -m 2 4 -t tlb-error bus-error,micro-arch
	GUID: e19e3d16-bc11-11e4-9caa-c2051d5d46b0
	CPER:
	      00000000  04 00 00 00 02 00 00 00 68 00 00 00 00 00 00 00   ........h.......
	      00000010  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
	      00000020  00 00 00 00 00 00 00 00 00 20 05 00 04 02 00 03   ......... ......
	      00000030  7f 00 54 00 00 00 00 00 ef be ad de 00 00 00 00   ..T.............
	      00000040  ad 0b ba ab 00 00 00 00 00 20 00 00 18 01 00 03   ......... ......
	      00000050  00 00 00 00 00 00 00 00 ef be ad de 00 00 00 00   ................
	      00000060  ad 0b ba ab 00 00 00 00                           ........

	Error injected.

The CPER event is now properly handled:

[   83.807957] {1}[Hardware Error]: Hardware error from APEI Generic Hardware Error Source: 1
[   83.808314] {1}[Hardware Error]: event severity: recoverable
[   83.808545] {1}[Hardware Error]:  Error 0, type: recoverable
[   83.808806] {1}[Hardware Error]:   section_type: ARM processor error
[   83.809045] {1}[Hardware Error]:   MIDR: 0x0000000000000000
[   83.809262] {1}[Hardware Error]:   running state: 0x0
[   83.809441] {1}[Hardware Error]:   Power State Coordination Interface state: 0
[   83.809691] {1}[Hardware Error]:   Error info structure 0:
[   83.809885] {1}[Hardware Error]:   num errors: 3
[   83.810128] {1}[Hardware Error]:    error_type: 0x04: TLB error
[   83.810364] {1}[Hardware Error]:    error_info: 0x000000000054007f
[   83.810595] {1}[Hardware Error]:     transaction type: Instruction
[   83.810821] {1}[Hardware Error]:     TLB error, operation type: Instruction fetch
[   83.811095] {1}[Hardware Error]:     TLB level: 1
[   83.811283] {1}[Hardware Error]:     processor context not corrupted
[   83.811501] {1}[Hardware Error]:     the error has not been corrected
[   83.811721] {1}[Hardware Error]:     PC is imprecise
[   83.811924] {1}[Hardware Error]:   Error info structure 1:
[   83.812105] {1}[Hardware Error]:   num errors: 2
[   83.812263] {1}[Hardware Error]:    error_type: 0x18: bus error|micro-architectural error
[   83.812956] [Firmware Warn]: GHES: Unhandled processor error type 0x04: TLB error
[   83.813212] [Firmware Warn]: GHES: Unhandled processor error type 0x18: bus error|micro-architectural error

-

I also tested the ghes and cper reports both with and without this change, using different versions of rasdaemon, with and without support for the extended trace event. Those are a summary of the test results:

- adding more fields to the trace events didn't break userspace API:
both versions of rasdaemon handled it;

- the rasdaemon patches to handle the new trace report was missing a backward-compatibility logic. I fixed already. So, rasdaemon can now handle both old and new trace events.

Btw, rasdaemon has gained support for the extended trace since its version 0.5.8 (released in 2021). I didn't saw any issues there complain about troubles on it, so either distros used on ARM servers are using an old version of rasdaemon, or they're carrying on the trace event changes as well.

---

v3:
- rebased on the top of 6.11-rc1;
- test example updated to reflect latest error-inj patch series submitted to qemu-devel

v2:
- removed an uneeded patch adding #ifdef for CONFIG_ARM/ARM64;
- cper_bits_to_str() now returns the number of chars filled at the buffer;
- did a cosmetic (blank lines) improvement at include/linux/ras.h;
- arm_event trace dynamic arrays renamed to pei_buf/ctx_buf/oem_buf.

Daniel Ferguson (1):
RAS: Report all ARM processor CPER information to userspace

Mauro Carvalho Chehab (4):
efi/cper: Adjust infopfx size to accept an extra space efi/cper: Add a new helper function to print bitmasks efi/cper: align ARM CPER type with UEFI 2.9A/2.10 specs
docs: efi: add CPER functions to driver-api

 .../driver-api/firmware/efi/index.rst         | 11 ++--
 drivers/acpi/apei/ghes.c                      | 26 +++++-----
 drivers/firmware/efi/cper-arm.c               | 52 +++++++++----------
 drivers/firmware/efi/cper.c                   | 45 +++++++++++++++-
 drivers/ras/ras.c                             | 45 +++++++++++++++-
 include/linux/cper.h                          | 12 +++--
 include/linux/ras.h                           | 16 ++++--
 include/ras/ras_event.h                       | 48 +++++++++++++++--
 8 files changed, 196 insertions(+), 59 deletions(-)

2.45.2

From: Daniel Ferguson <[email protected]>

The ARM processor CPER record was added at UEFI 2.6, and hasn't any changes up to UEFI 2.10 on its struct.

Yet, the original arm_event trace code added on changeset e9279e83ad1f ("trace, ras: add ARM processor error trace event") is incomplete, as it only traces some fields of UAPI 2.6 table N.16, not exporting at all any information from tables N.17 to N.29 of the record.

This is not enough for user to take appropriate action or to log what exactly happened.

According to UEFI_2_9 specification chapter N2.4.4, the ARM processor error section includes:

- several (ERR_INFO_NUM) ARM processor error information structures (Tables N.17 to N.20);
- several (CONTEXT_INFO_NUM) ARM processor context information structures (Tables N.21 to N.29);
- several vendor specific error information structures. The size is given by Section Length minus the size of the other fields.

In addition to those data, it also exports two fields that are parsed by the GHES driver when firmware reports it, e. g.:

- error severity
- cpu logical index

Report all of these information to userspace via trace uAPI, So that userspace can properly record the error and take decisions related to cpu core isolation according to error severity and other info.

After this patch, all the data from ARM Processor record from table N.16 are directly or indirectly visible on userspace:

====================================== =============================
UEFI field on table N.16 ARM Processor trace fields
====================================== =============================
Validation handled when filling data for
affinity MPIDR and running state.
ERR_INFO_NUM pei_len
CONTEXT_INFO_NUM ctx_len
Section Length indirectly reported by
pei_len, ctx_len and oem_len
Error affinity level affinity
MPIDR_EL1 mpidr
MIDR_EL1 midr
Running State running_state
PSCI State psci_state
Processor Error Information Structure pei_err - count at pei_len
Processor Context ctx_err- count at ctx_len
Vendor Specific Error Info oem - count at oem_len
====================================== =============================

It should be noticed that decoding of tables N.17 to N.29, if needed, will be handled on userspace. That gives more flexibility, as there won't be any need to flood the Kernel with micro-architecture specific error decoding).
Also, decoding the other fields require a complex logic, and should be done for each of the several values inside the record field. So, let userspace daemons like rasdaemon decode them, parsing such tables and having vendor-specific micro-architecture-specific decoders.

---
 drivers/acpi/apei/ghes.c | 11 ++++-----
 drivers/ras/ras.c        | 45 +++++++++++++++++++++++++++++++++++--
 include/linux/ras.h      | 16 +++++++++++---
 include/ras/ras_event.h  | 48 +++++++++++++++++++++++++++++++++++-----
 4 files changed, 103 insertions(+), 17 deletions(-)

index 623cc0cb4a65..06d9351a9abc 100644
@@ -529,7 +529,7 @@ static bool ghes_handle_memory_failure(struct acpi_hest_generic_data *gdata,
 }
 
 static bool ghes_handle_arm_hw_error(struct acpi_hest_generic_data *gdata,
-				       int sev, bool sync)
+				     int sev, bool sync)
 {
 	struct cper_sec_proc_arm *err = acpi_hest_get_payload(gdata);
 	int flags = sync ? MF_ACTION_REQUIRED : 0;

@@ -537,9 +537,8 @@ static bool ghes_handle_arm_hw_error(struct acpi_hest_generic_data *gdata,
 	int sec_sev, i;
 	char *p;
 
-	log_arm_hw_error(err);
-
 	sec_sev = ghes_severity(gdata->error_severity);
+	log_arm_hw_error(err, sec_sev);
 	if (sev != GHES_SEV_RECOVERABLE || sec_sev != GHES_SEV_RECOVERABLE)
 		return false;
 

@@ -773,11 +772,9 @@ static bool ghes_do_proc(struct ghes *ghes,
 
 			arch_apei_report_mem_error(sev, mem_err);
 			queued = ghes_handle_memory_failure(gdata, sev, sync);
-		}
-		else if (guid_equal(sec_type, &CPER_SEC_PCIE)) {
+		} else if (guid_equal(sec_type, &CPER_SEC_PCIE)) {
 			ghes_handle_aer(gdata);
-		}
-		else if (guid_equal(sec_type, &CPER_SEC_PROC_ARM)) {
+		} else if (guid_equal(sec_type, &CPER_SEC_PROC_ARM)) {
 			queued = ghes_handle_arm_hw_error(gdata, sev, sync);
 		} else if (guid_equal(sec_type, &CPER_SEC_CXL_GEN_MEDIA_GUID)) {
 			struct cxl_cper_event_rec *rec = acpi_hest_get_payload(gdata);

index a64182bc72ad..df444492b434 100644
@@ -24,8 +24,7 @@ int __init parse_cec_param(char *str);
 void log_non_standard_event(const guid_t *sec_type,
 			    const guid_t *fru_id, const char *fru_text,
 			    const u8 sev, const u8 *err, const u32 len);
-void log_arm_hw_error(struct cper_sec_proc_arm *err);
-
+void log_arm_hw_error(struct cper_sec_proc_arm *err, const u8 sev);
 #else
 static inline void
 log_non_standard_event(const guid_t *sec_type,

@@ -33,7 +32,7 @@ log_non_standard_event(const guid_t *sec_type,
 		       const u8 sev, const u8 *err, const u32 len)
 { return; }
 static inline void
-log_arm_hw_error(struct cper_sec_proc_arm *err) { return; }
+log_arm_hw_error(struct cper_sec_proc_arm *err, const u8 sev) { return; }
 #endif
 
 struct atl_err {

@@ -53,4 +52,15 @@ static inline unsigned long
 amd_convert_umc_mca_addr_to_sys_addr(struct atl_err *err) { return -EINVAL; }
 #endif /* CONFIG_AMD_ATL */
 
+#if defined(CONFIG_ARM) || defined(CONFIG_ARM64)
+#include <asm/smp_plat.h>
+/*
+ * Include ARM specific SMP header which provides a function mapping mpidr to
+ * cpu logical index.
+ */
+#define GET_LOGICAL_INDEX(mpidr) get_logical_index(mpidr & MPIDR_HWID_BITMASK)
+#else
+#define GET_LOGICAL_INDEX(mpidr) -EINVAL
+#endif /* CONFIG_ARM || CONFIG_ARM64 */
+
 #endif /* __RAS_H__ */

2.45.2

Compiling with W=1 with werror enabled produces an error:

drivers/firmware/efi/cper-arm.c: In function ‘cper_print_proc_arm’:
drivers/firmware/efi/cper-arm.c:298:64: error: ‘snprintf’ output may be truncated before the last format character [-Werror=format-truncation=]
298 | snprintf(infopfx, sizeof(infopfx), "%s ", newpfx);
| ^
drivers/firmware/efi/cper-arm.c:298:25: note: ‘snprintf’ output between 2 and 65 bytes into a destination of size 64

  298 |                         snprintf(infopfx, sizeof(infopfx), "%s ", newpfx);
      |                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

As the logic there adds an space at the end of infopx buffer. Add an extra space to avoid such warning.

---
 drivers/firmware/efi/cper-arm.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

index fa9c1c3bf168..eb7ee6af55f2 100644
@@ -240,7 +240,7 @@ void cper_print_proc_arm(const char *pfx,
 	int i, len, max_ctx_type;
 	struct cper_arm_err_info *err_info;
 	struct cper_arm_ctx_info *ctx_info;
-	char newpfx[64], infopfx[64];
+	char newpfx[64], infopfx[ARRAY_SIZE(newpfx) + 1];
 
 	printk("%sMIDR: 0x%016llx\n", pfx, proc->midr);
 

2.45.2

Sometimes it is desired to produce a single log line for errors. Add a new helper function for such purpose.

Reviewed-by; Jonathan Cameron <[email protected]>

---
 drivers/firmware/efi/cper.c | 43 +++++++++++++++++++++++++++++++++++++
 include/linux/cper.h        |  2 ++
 2 files changed, 45 insertions(+)

index 7d2cdd9e2227..462d739e8dd1 100644
@@ -106,6 +106,49 @@ void cper_print_bits(const char *pfx, unsigned int bits,
 		printk("%s\n", buf);
 }
 
+/**
+ * cper_bits_to_str - return a string for set bits
+ * @buf: buffer to store the output string
+ * @buf_size: size of the output string buffer
+ * @bits: bit mask
+ * @strs: string array, indexed by bit position
+ * @strs_size: size of the string array: @strs
+ *
+ * Add to @buf the bitmask in hexadecimal. Then, for each set bit in @bits,
+ * add the corresponding string describing the bit in @strs to @buf.
+ *
+ * Return: number of bytes stored or an error code if lower than zero.
+ */
+int cper_bits_to_str(char *buf, int buf_size, unsigned long bits,
+		     const char * const strs[], unsigned int strs_size)
+{
+	int len = buf_size;
+	char *str = buf;
+	int i, size;
+
+	*buf = '\0';
+
+	for_each_set_bit(i, &bits, strs_size) {
+		if (!(bits & (1U << (i))))
+			continue;
+
+		if (*buf && len > 0) {
+			*str = '|';
+			len--;
+			str++;
+		}
+
+		size = strscpy(str, strs[i], len);
+		if (size < 0)
+			return size;
+
+		len -= size;
+		str += size;
+	}
+	return len - buf_size;
+}
+EXPORT_SYMBOL_GPL(cper_bits_to_str);
+
 static const char * const proc_type_strs[] = {
 	"IA32/X64",
 	"IA64",

index 265b0f8fc0b3..25858a7608b7 100644
@@ -584,6 +584,8 @@ const char *cper_mem_err_type_str(unsigned int);
 const char *cper_mem_err_status_str(u64 status);
 void cper_print_bits(const char *prefix, unsigned int bits,
 		     const char * const strs[], unsigned int strs_size);
+int cper_bits_to_str(char *buf, int buf_size, unsigned long bits,
+		     const char * const strs[], unsigned int strs_size);
 void cper_mem_err_pack(const struct cper_sec_mem_err *,
 		       struct cper_mem_err_compact *);
 const char *cper_mem_err_unpack(struct trace_seq *,

2.45.2

Up to UEFI spec 2.9, the type byte of CPER struct for ARM processor was defined simply as:

Type at byte offset 4:

- Cache error
- TLB Error
- Bus Error
- Micro-architectural Error
All other values are reserved

Yet, there was no information about how this would be encoded.

Spec 2.9A errata corrected it by defining:

- Bit 1 - Cache Error
- Bit 2 - TLB Error
- Bit 3 - Bus Error
- Bit 4 - Micro-architectural Error
All other values are reserved

That actually aligns with the values already defined on older versions at N.2.4.1. Generic Processor Error Section.

Spec 2.10 also preserve the same encoding as 2.9A.

Adjust CPER and GHES handling code for both generic and ARM processors to properly handle UEFI 2.9A and 2.10 encoding.

---
 drivers/acpi/apei/ghes.c        | 15 ++++++----
 drivers/firmware/efi/cper-arm.c | 50 ++++++++++++++++-----------------
 include/linux/cper.h            | 10 +++----
 3 files changed, 38 insertions(+), 37 deletions(-)

index 06d9351a9abc..d7d78d546f68 100644
@@ -533,6 +533,7 @@ static bool ghes_handle_arm_hw_error(struct acpi_hest_generic_data *gdata,
 {
 	struct cper_sec_proc_arm *err = acpi_hest_get_payload(gdata);
 	int flags = sync ? MF_ACTION_REQUIRED : 0;
+	char error_type[120];
 	bool queued = false;
 	int sec_sev, i;
 	char *p;

@@ -545,9 +546,8 @@ static bool ghes_handle_arm_hw_error(struct acpi_hest_generic_data *gdata,
 	p = (char *)(err + 1);
 	for (i = 0; i < err->err_info_num; i++) {
 		struct cper_arm_err_info *err_info = (struct cper_arm_err_info *)p;
-		bool is_cache = (err_info->type == CPER_ARM_CACHE_ERROR);
+		bool is_cache = err_info->type & CPER_ARM_CACHE_ERROR;
 		bool has_pa = (err_info->validation_bits & CPER_ARM_INFO_VALID_PHYSICAL_ADDR);
-		const char *error_type = "unknown error";
 
 		/*
 		 * The field (err_info->error_info & BIT(26)) is fixed to set to

@@ -561,12 +561,15 @@ static bool ghes_handle_arm_hw_error(struct acpi_hest_generic_data *gdata,
 			continue;
 		}
 
-		if (err_info->type < ARRAY_SIZE(cper_proc_error_type_strs))
-			error_type = cper_proc_error_type_strs[err_info->type];
+		cper_bits_to_str(error_type, sizeof(error_type),
+				 FIELD_GET(CPER_ARM_ERR_TYPE_MASK, err_info->type),
+				 cper_proc_error_type_strs,
+				 ARRAY_SIZE(cper_proc_error_type_strs));
 
 		pr_warn_ratelimited(FW_WARN GHES_PFX
-				    "Unhandled processor error type: %s\n",
-				    error_type);
+				    "Unhandled processor error type 0x%02x: %s%s\n",
+				    err_info->type, error_type,
+				    (err_info->type & ~CPER_ARM_ERR_TYPE_MASK) ? " with reserved bit(s)" : "");
 		p += err_info->length;
 	}
 

index 25858a7608b7..3670b866ac11 100644
@@ -293,11 +293,11 @@ enum {
 #define CPER_ARM_INFO_FLAGS_PROPAGATED		BIT(2)
 #define CPER_ARM_INFO_FLAGS_OVERFLOW		BIT(3)
 
-#define CPER_ARM_CACHE_ERROR			0
-#define CPER_ARM_TLB_ERROR			1
-#define CPER_ARM_BUS_ERROR			2
-#define CPER_ARM_VENDOR_ERROR			3
-#define CPER_ARM_MAX_TYPE			CPER_ARM_VENDOR_ERROR
+#define CPER_ARM_ERR_TYPE_MASK			GENMASK(4,1)
+#define CPER_ARM_CACHE_ERROR			BIT(1)
+#define CPER_ARM_TLB_ERROR			BIT(2)
+#define CPER_ARM_BUS_ERROR			BIT(3)
+#define CPER_ARM_VENDOR_ERROR			BIT(4)
 
 #define CPER_ARM_ERR_VALID_TRANSACTION_TYPE	BIT(0)
 #define CPER_ARM_ERR_VALID_OPERATION_TYPE	BIT(1)

2.45.2

There are two kernel-doc like descriptions at cper, which is used by other parts of cper and on ghes driver. They both have kernel-doc like descriptions.

Change the tags for them to be actual kernel-doc tags and add them to the driver-api documentaion at the UEFI section.

---
 Documentation/driver-api/firmware/efi/index.rst | 11 ++++++++---
 drivers/firmware/efi/cper.c                     |  2 +-
 2 files changed, 9 insertions(+), 4 deletions(-)

index 4fe8abba9fc6..5a6b6229592c 100644
@@ -1,11 +1,16 @@
 .. SPDX-License-Identifier: GPL-2.0
 
-============
-UEFI Support
-============
+====================================================
+Unified Extensible Firmware Interface (UEFI) Support
+====================================================
 
 UEFI stub library functions
 ===========================
 
 .. kernel-doc:: drivers/firmware/efi/libstub/mem.c
    :internal:
+
+UEFI Common Platform Error Record (CPER) functions
+==================================================
+
+.. kernel-doc:: drivers/firmware/efi/cper.c

index 462d739e8dd1..295f6f339575 100644
@@ -69,7 +69,7 @@ const char *cper_severity_str(unsigned int severity)
 }
 EXPORT_SYMBOL_GPL(cper_severity_str);
 
-/*
+/**
  * cper_print_bits - print strings for set bits
  * @pfx: prefix for each line, including log level and prefix string
  * @bits: bit mask

2.45.2