arm64: erratum: Workaround for Kryo reserved system register read

The ARMv8.0 architecture reserves several system register encodings for future use. These encodings should behave as read-only and always return zero on a read. The Kryo core errantly causes an instruction abort upon an AArch64 read attempt to the following system register encodings using the MRS instruction:

	 3, 0, C0, [C4-C7], [2-3, 6-7]
	 3, 0, C0, C3, [3-7]
	 3, 0, C0, [C4,C6,C7], [4-5]
	 3, 0, C0, C2, [6-7]

All system register encodings above use the following form Op0, Op1, CRn, CRm, Op2.
Note that some of the encodings listed above include the system register space reserved for the following identification registers which may appear in future revisions of the ARM architecture beyond ARMv8.0.

This space includes:

	ID_AA64PFR[2-7]_EL1
	ID_AA64DFR[2-3]_EL1
	ID_AA64AFR[2-3]_EL1
	ID_AA64ISAR[2-7]_EL1
	ID_AA64MMFR[2-7]_EL1

Workaround:
Software must not rely on a zero value returned from any of the system register encodings listed above when attempting to determine support for certain architectural options. Software should instead infer architectural support when unaffected variant is in use by reading the MIDR_EL1.

System Impact
None. The required software workaround does not result in any reduction in functionality nor does it have any relevant impact on performance or power.

This change uses the ARMv8 implementer introduced in
https://www.mail-archive.com/kvm at vger.kernel.org/msg116473.html

Change-Id Id53e335b6c4524c730b95e5dd7279cf870bb82f6

Signed-off-by Naveen Kaje <[email protected]>

---
 Documentation/arm64/silicon-errata.txt | 29 +++++++++++++++--------------
 arch/arm64/Kconfig                     | 29 +++++++++++++++++++++++++++++
 arch/arm64/include/asm/cpufeature.h    |  3 ++-
 arch/arm64/kernel/cpu_errata.c         |  8 ++++++++
 arch/arm64/kernel/cpuinfo.c            | 10 +++++++---
 5 files changed, 61 insertions(+), 18 deletions(-)

Documentation/arm64/silicon-errata.txt +15 -14

index 58b71dd..4e16be5 100644
@@ -42,17 +42,18 @@ file acts as a registry of software workarounds in the Linux Kernel and
 will be updated when new workarounds are committed and backported to
 stable kernels.
 
-| Implementor    | Component       | Erratum ID      | Kconfig                 |
-+----------------+-----------------+-----------------+-------------------------+
-| ARM            | Cortex-A53      | #826319         | ARM64_ERRATUM_826319    |
-| ARM            | Cortex-A53      | #827319         | ARM64_ERRATUM_827319    |
-| ARM            | Cortex-A53      | #824069         | ARM64_ERRATUM_824069    |
-| ARM            | Cortex-A53      | #819472         | ARM64_ERRATUM_819472    |
-| ARM            | Cortex-A53      | #845719         | ARM64_ERRATUM_845719    |
-| ARM            | Cortex-A53      | #843419         | ARM64_ERRATUM_843419    |
-| ARM            | Cortex-A57      | #832075         | ARM64_ERRATUM_832075    |
-| ARM            | Cortex-A57      | #852523         | N/A                     |
-| ARM            | Cortex-A57      | #834220         | ARM64_ERRATUM_834220    |
-|                |                 |                 |                         |
-| Cavium         | ThunderX ITS    | #22375, #24313  | CAVIUM_ERRATUM_22375    |
-| Cavium         | ThunderX GICv3  | #23154          | CAVIUM_ERRATUM_23154    |
+| Implementor		 | Component       | Erratum ID      | Kconfig                 |
++------------------------+-----------------+-----------------+-------------------------+
+| ARM			 | Cortex-A53      | #826319         | ARM64_ERRATUM_826319    |
+| ARM			 | Cortex-A53      | #827319         | ARM64_ERRATUM_827319    |
+| ARM			 | Cortex-A53      | #824069         | ARM64_ERRATUM_824069    |
+| ARM			 | Cortex-A53      | #819472         | ARM64_ERRATUM_819472    |
+| ARM			 | Cortex-A53      | #845719         | ARM64_ERRATUM_845719    |
+| ARM			 | Cortex-A53      | #843419         | ARM64_ERRATUM_843419    |
+| ARM			 | Cortex-A57      | #832075         | ARM64_ERRATUM_832075    |
+| ARM			 | Cortex-A57      | #852523         | N/A                     |
+| ARM			 | Cortex-A57      | #834220         | ARM64_ERRATUM_834220    |
+|			 |                 |                 |                         |
+| Cavium		 | ThunderX ITS    | #22375, #24313  | CAVIUM_ERRATUM_22375    |
+| Cavium		 | ThunderX GICv3  | #23154          | CAVIUM_ERRATUM_23154    |
+| Qualcomm Technologies	 | Kryo		   | #94	     | KRYO_ERRATUM_94	       |

arch/arm64/Kconfig +29 -0

index e6e61dc..004e998 100644
@@ -443,6 +443,35 @@ config CAVIUM_ERRATUM_23154
 
 	  If unsure, say Y.
 
+config KRYO_ERRATUM_94
+	bool "ERRATUM to KRYO System Register Read"
+	help
+	The ARMv8.0 specification reserves several system registers
+	for future use. These registers should behave read-only and
+	return zero on a read. The Kryo core errantly causes
+	an instruction abort upon AArch64 read attempt of the following
+	system register encodings using the MRS instruction.
+
+	3, 0, C0, [C4-C7], [2-3, 6-7]
+	3, 0, C0, C3, [3-7]
+	3, 0, C0, [C4,C6,C7], [4-5]
+	3, 0, C0, C2, [6-7]
+
+	All system register encodings above use the form
+
+	Op0, Op1, CRn, CRm, Op2.
+
+	Note that some of the encodings listed above include
+	the system register space reserved for the following
+	identification registers which may appear in future revisions
+	of the ARM architecture beyond ARMv8.0.
+	This space includes:
+	ID_AA64PFR[2-7]_EL1
+	ID_AA64DFR[2-3]_EL1
+	ID_AA64AFR[2-3]_EL1
+	ID_AA64ISAR[2-7]_EL1
+	ID_AA64MMFR[2-7]_EL1
+
 endmenu
 
 

arch/arm64/include/asm/cpufeature.h +0 -0

index d28e8b2..448a637 100644

arch/arm64/include/asm/cpufeature.h +0 -0

 #ifndef __ASSEMBLY__
 

arch/arm64/kernel/cpu_errata.c +8 -0

index feb6b4e..27f5846 100644
@@ -24,6 +24,7 @@
 #define MIDR_CORTEX_A53 MIDR_CPU_PART(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A53)
 #define MIDR_CORTEX_A57 MIDR_CPU_PART(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A57)
 #define MIDR_THUNDERX	MIDR_CPU_PART(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX)
+#define MIDR_QCOM_KRYO  MIDR_CPU_PART(ARM_CPU_IMP_QCOM, QCOM_CPU_PART_KRYO)
 
 #define CPU_MODEL_MASK (MIDR_IMPLEMENTOR_MASK | MIDR_PARTNUM_MASK | \
 			MIDR_ARCHITECTURE_MASK)

@@ -100,6 +101,13 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
 		MIDR_RANGE(MIDR_THUNDERX, 0x00, 0x01),
 	},
 #endif
+#ifdef CONFIG_KRYO_ERRATUM_94
+	{
+		.desc = "Kryo Erratum 94",
+		.capability = ARM64_WORKAROUND_KRYO_94,
+		MIDR_RANGE(MIDR_QCOM_KRYO, 0x00, 0x01),
+	},
+#endif
 	{
 	}
 };

arch/arm64/kernel/cpuinfo.c +0 -0

index 966fbd5..9921fac 100644

arch/arm64/kernel/cpuinfo.c +4 -1

 	info->reg_id_aa64dfr0 = read_cpuid(SYS_ID_AA64DFR0_EL1);
 	info->reg_id_aa64dfr1 = read_cpuid(SYS_ID_AA64DFR1_EL1);
 	info->reg_id_aa64isar0 = read_cpuid(SYS_ID_AA64ISAR0_EL1);
 	info->reg_id_aa64isar1 = read_cpuid(SYS_ID_AA64ISAR1_EL1);
 	info->reg_id_aa64mmfr0 = read_cpuid(SYS_ID_AA64MMFR0_EL1);
 	info->reg_id_aa64mmfr1 = read_cpuid(SYS_ID_AA64MMFR1_EL1);
-	info->reg_id_aa64mmfr2 = read_cpuid(SYS_ID_AA64MMFR2_EL1);
+	if (cpus_have_cap(ARM64_WORKAROUND_KRYO_94))
+		info->reg_id_aa64mmfr2 = 0;
+	else
+		info->reg_id_aa64mmfr2 = read_cpuid(SYS_ID_AA64MMFR2_EL1);
 	info->reg_id_aa64pfr0 = read_cpuid(SYS_ID_AA64PFR0_EL1);
 	info->reg_id_aa64pfr1 = read_cpuid(SYS_ID_AA64PFR1_EL1);
 

Quoted 6 lines

arch/arm64/Kconfig +6 -0

+	All system register encodings above use the form
+
+	Op0, Op1, CRn, CRm, Op2.
+
+	Note that some of the encodings listed above include
+	the system register space reserved for the following

arch/arm64/Kconfig config CAVIUM_ERRATUM_23154 · line 473

Quoted 8 lines

arch/arm64/Kconfig +8 -0

+	identification registers which may appear in future revisions
+	of the ARM architecture beyond ARMv8.0.
+	This space includes:
+	ID_AA64PFR[2-7]_EL1
+	ID_AA64DFR[2-3]_EL1
+	ID_AA64AFR[2-3]_EL1
+	ID_AA64ISAR[2-7]_EL1
+	ID_AA64MMFR[2-7]_EL1

arch/arm64/kernel/cpuinfo.c static void __cpuinfo_store_cpu(struct cpuinfo_arm64 *info) · line 208

Quoted 3 lines

arch/arm64/kernel/cpuinfo.c +3 -0

+
+	check_local_cpu_errata();
+

What is the impact of moving this around? Suzuki, was there any particular reason why this check was done later rather than earlier?

Quoted 10 lines · Marc Zyngier

Quoted 10 lines

arch/arm64/Kconfig +10 -0

+	All system register encodings above use the form
+
+	Op0, Op1, CRn, CRm, Op2.
+
+	Note that some of the encodings listed above include
+	the system register space reserved for the following
+	identification registers which may appear in future revisions
+	of the ARM architecture beyond ARMv8.0.
+	This space includes:
+	ID_AA64PFR[2-7]_EL1

Quoted 4 lines

arch/arm64/Kconfig config CAVIUM_ERRATUM_23154 · line 473

Quoted 4 lines

arch/arm64/Kconfig +4 -0

+	ID_AA64DFR[2-3]_EL1
+	ID_AA64AFR[2-3]_EL1
+	ID_AA64ISAR[2-7]_EL1
+	ID_AA64MMFR[2-7]_EL1

AFAIK, the id space is unassigned. So the naming above could cause confusion if the register is named something else.

Quoted 12 lines · Suzuki K Poulose

Quoted 10 lines · Marc Zyngier

Quoted 10 lines

arch/arm64/Kconfig +10 -0

+	All system register encodings above use the form
+
+	Op0, Op1, CRn, CRm, Op2.
+
+	Note that some of the encodings listed above include
+	the system register space reserved for the following
+	identification registers which may appear in future revisions
+	of the ARM architecture beyond ARMv8.0.
+	This space includes:
+	ID_AA64PFR[2-7]_EL1

Quoted 8 lines

Quoted 4 lines

arch/arm64/Kconfig config CAVIUM_ERRATUM_23154 · line 473

Quoted 4 lines

arch/arm64/Kconfig +4 -0

+	ID_AA64DFR[2-3]_EL1
+	ID_AA64AFR[2-3]_EL1
+	ID_AA64ISAR[2-7]_EL1
+	ID_AA64MMFR[2-7]_EL1

AFAIK, the id space is unassigned. So the naming above could cause confusion if the register is named something else.

It is reserved *at the moment*, but already has a defined behaviour. My

worry is that when some new architecture revision comes around, we start using these registers without thinking much about it (because we should be able to). At this point, your SoC will catch fire and nobody will have a clue about the problem because it is not apparent in the code.

I'd really like to see something a bit more forward looking that covers that space for good.

Quoted 22 lines · Marc Zyngier

Quoted 20 lines · Suzuki K Poulose

Quoted 6 lines · Marc Zyngier

Quoted 6 lines

arch/arm64/Kconfig +6 -0

+	All system register encodings above use the form
+
+	Op0, Op1, CRn, CRm, Op2.
+
+	Note that some of the encodings listed above include
+	the system register space reserved for the following

Quoted 8 lines

arch/arm64/Kconfig config CAVIUM_ERRATUM_23154 · line 473

Quoted 8 lines

arch/arm64/Kconfig +8 -0

+	identification registers which may appear in future revisions
+	of the ARM architecture beyond ARMv8.0.
+	This space includes:
+	ID_AA64PFR[2-7]_EL1
+	ID_AA64DFR[2-3]_EL1
+	ID_AA64AFR[2-3]_EL1
+	ID_AA64ISAR[2-7]_EL1
+	ID_AA64MMFR[2-7]_EL1

AFAIK, the id space is unassigned. So the naming above could cause confusion if the register is named something else.

Quoted 1 line

It is reserved *at the moment*, but already has a defined behaviour. My

Absolutely, they do need to be RAZ. My point was assigning names to the reserved space where the names are unassigned.

Quoted 12 lines · Suzuki K Poulose

Quoted 10 lines · Marc Zyngier

Quoted 10 lines

arch/arm64/Kconfig +10 -0

+	All system register encodings above use the form
+
+	Op0, Op1, CRn, CRm, Op2.
+
+	Note that some of the encodings listed above include
+	the system register space reserved for the following
+	identification registers which may appear in future revisions
+	of the ARM architecture beyond ARMv8.0.
+	This space includes:
+	ID_AA64PFR[2-7]_EL1

Quoted 8 lines

Quoted 4 lines

arch/arm64/Kconfig config CAVIUM_ERRATUM_23154 · line 473

Quoted 4 lines

arch/arm64/Kconfig +4 -0

+	ID_AA64DFR[2-3]_EL1
+	ID_AA64AFR[2-3]_EL1
+	ID_AA64ISAR[2-7]_EL1
+	ID_AA64MMFR[2-7]_EL1

AFAIK, the id space is unassigned. So the naming above could cause confusion if the register is named something else.

It is reserved *at the moment*, but already has a defined behaviour. My worry is that when some new architecture revision comes around, we start using these registers without thinking much about it (because we should be able to). At this point, your SoC will catch fire and nobody will have a clue about the problem because it is not apparent in the code.

I'd really like to see something a bit more forward looking that covers that space for good.

Quoted 22 lines · Marc Zyngier

Quoted 12 lines · Suzuki K Poulose

Quoted 10 lines · Marc Zyngier

Quoted 10 lines

arch/arm64/Kconfig +10 -0

+	All system register encodings above use the form
+
+	Op0, Op1, CRn, CRm, Op2.
+
+	Note that some of the encodings listed above include
+	the system register space reserved for the following
+	identification registers which may appear in future revisions
+	of the ARM architecture beyond ARMv8.0.
+	This space includes:
+	ID_AA64PFR[2-7]_EL1

Quoted 8 lines

Quoted 4 lines

arch/arm64/Kconfig config CAVIUM_ERRATUM_23154 · line 473

Quoted 4 lines

arch/arm64/Kconfig +4 -0

+	ID_AA64DFR[2-3]_EL1
+	ID_AA64AFR[2-3]_EL1
+	ID_AA64ISAR[2-7]_EL1
+	ID_AA64MMFR[2-7]_EL1

AFAIK, the id space is unassigned. So the naming above could cause confusion if the register is named something else.

Quoted 8 lines

It is reserved *at the moment*, but already has a defined behaviour. My worry is that when some new architecture revision comes around, we start using these registers without thinking much about it (because we should be able to). At this point, your SoC will catch fire and nobody will have a clue about the problem because it is not apparent in the code.

I'd really like to see something a bit more forward looking that covers that space for good.

At the risk of volunteering...
Registering these instructions with the undef hooks would be ideal, but they won't catch this instruction abort. I guess refactor them to be generic faulting instruction hooks, and have a list for the existing undef cases, and a new one for this instruction abort.

This won't cover early code in head.S, or KVM code that runs at EL2. Is this sufficient, or should any approach cover those too?

The ARMv8.0 architecture reserves several system register encodings for future use. These encodings should behave as read-only and always return zero on a read. The Kryo core errantly causes an instruction abort upon an AArch64 read attempt to the following system register encodings using the MRS instruction:

	 3, 0, C0, [C4-C7], [2-3, 6-7]
	 3, 0, C0, C3, [3-7]
	 3, 0, C0, [C4,C6,C7], [4-5]
	 3, 0, C0, C2, [6-7]

All system register encodings above use the following form Op0, Op1, CRn, CRm, Op2.
Note that some of the encodings listed above include the system register space reserved for the following identification registers which may appear in future revisions of the ARM architecture beyond ARMv8.0.