Skip to contentSkip to conversation

KVM: arm64: Enforce MTE disablement at EL2

PATCH Applied Fuad Tabba 2 participants 6 messages Jan 22 – Jan 23

Subsystems (2)
lore
Share & tools
lore

Fuad Tabba

author
#0 lore raw

v3 cover KVM: arm64: Enforce MTE disablement at EL2

Changes since v2 [1]:
- Trap accesses to GMID_EL1 by setting HCR_EL2.TID5 when MTE is disabled (Marc)
- Dropped patch refactoring enter_exception64() in favor of open-coded logic in inject_undef64() in patch 3/4 (Marc)
- Based on Linux 6.19-rc6

pKVM never exposes MTE to protected guests (pVM), but we must also ensure that a malicious host cannot use MTE to attack the hypervisor or a protected VM.

If MTE is supported by the hardware (and is enabled at EL3), it remains available to lower exception levels by default. Disabling it in the host kernel (e.g., via 'arm64.nomte') only stops the kernel from advertising the feature; it does not physically disable MTE in the hardware.

The ability to disable MTE in the host kernel is used by some systems, such as Android, so that the physical memory otherwise used as tag storage can be used for other things (i.e. treated just like the rest of memory). In this scenario, a malicious host could still access tags in pages donated to a guest using MTE instructions (e.g., STG and LDG), bypassing the kernel's configuration.

To prevent this, explicitly disable MTE at EL2 (by clearing HCR_EL2.ATA and setting HCR_EL2.TID5) when the host has MTE disabled. This causes MTE instructions to trap to the hypervisor.

Cheers,
/fuad

[1] https://lore.kernel.org/all/[email protected]/

Fuad Tabba (4):
KVM: arm64: Remove dead code resetting HCR_EL2 for pKVM
KVM: arm64: Trap MTE access and discovery when MTE is disabled
KVM: arm64: Inject UNDEF when accessing MTE sysregs with MTE disabled
KVM: arm64: Use kvm_has_mte() in pKVM trap initialization

 arch/arm64/include/asm/kvm_arm.h   |  2 +-
 arch/arm64/kernel/head.S           |  2 +-
 arch/arm64/kvm/arm.c               |  6 +++
 arch/arm64/kvm/hyp/nvhe/hyp-init.S |  5 ---
 arch/arm64/kvm/hyp/nvhe/hyp-main.c | 67 ++++++++++++++++++++++++++++++
 arch/arm64/kvm/hyp/nvhe/pkvm.c     |  2 +-
 6 files changed, 76 insertions(+), 8 deletions(-)

base-commit: 24d479d26b25bce5faea3ddd9fa8f3a6c3129ea7

Signature · 1 line
2.52.0.457.g6b5491de43-goog
author
#1 lore raw

PATCH v3 1/4 KVM: arm64: Remove dead code resetting HCR_EL2 for pKVM

The pKVM lifecycle does not support tearing down the hypervisor and returning to the hyp stub once initialized. The transition to protected mode is one-way.

Consequently, the code path in hyp-init.S responsible for resetting EL2 registers (triggered by kexec or hibernation) is unreachable in protected mode.

Remove the dead code handling HCR_EL2 reset for
ARM64_KVM_PROTECTED_MODE.

No functional change intended.

---
 arch/arm64/kvm/hyp/nvhe/hyp-init.S | 5 -----
 1 file changed, 5 deletions(-)
arch/arm64/kvm/hyp/nvhe/hyp-init.S +0 -5
index aada42522e7b..0d42eedc7167 100644
@@ -260,11 +260,6 @@ reset:
 	msr	sctlr_el2, x5
 	isb
 
-alternative_if ARM64_KVM_PROTECTED_MODE
-	mov_q	x5, HCR_HOST_NVHE_FLAGS
-	msr_hcr_el2 x5
-alternative_else_nop_endif
-
 	/* Install stub vectors */
 	adr_l	x5, __hyp_stub_vectors
 	msr	vbar_el2, x5
Signature · 1 line
2.52.0.457.g6b5491de43-goog
author
#2 lore raw

PATCH v3 2/4 KVM: arm64: Trap MTE access and discovery when MTE is disabled

In reply to Fuad Tabba

If MTE is not supported by the hardware, or is disabled in the kernel configuration (CONFIG_ARM64_MTE=n) or command line (arm64.nomte), the kernel stops advertising MTE to userspace and avoids using MTE instructions. However, this is a software-level disable only.

When MTE hardware is present and enabled by EL3 firmware, leaving HCR_EL2.ATA set allows the host to execute MTE instructions (STG, LDG, etc.) and access allocation tags in physical memory.

Prevent this by clearing HCR_EL2.ATA when MTE is disabled. Remove it from the HCR_HOST_NVHE_FLAGS default, and conditionally set it in cpu_prepare_hyp_mode() only when system_supports_mte() returns true. This causes MTE instructions to trap to EL2 when HCR_EL2.ATA is cleared.

Additionally, set HCR_EL2.TID5 when MTE is disabled. This traps reads of GMID_EL1 (Multiple tag transfer ID register) to EL2, preventing the discovery of MTE parameters (such as tag block size) when the feature is suppressed.

Early boot code in head.S temporarily keeps HCR_ATA set to avoid special-casing initialization paths. This is safe because this code executes before untrusted code runs and will clear HCR_ATA if MTE is disabled.

---
 arch/arm64/include/asm/kvm_arm.h | 2 +-
 arch/arm64/kernel/head.S         | 2 +-
 arch/arm64/kvm/arm.c             | 6 ++++++
 3 files changed, 8 insertions(+), 2 deletions(-)
arch/arm64/include/asm/kvm_arm.h +1 -1
index e500600e4b9b..752e3e1604e8 100644
@@ -101,7 +101,7 @@
 			 HCR_BSU_IS | HCR_FB | HCR_TACR | \
 			 HCR_AMO | HCR_SWIO | HCR_TIDCP | HCR_RW | HCR_TLOR | \
 			 HCR_FMO | HCR_IMO | HCR_PTW | HCR_TID3 | HCR_TID1)
-#define HCR_HOST_NVHE_FLAGS (HCR_RW | HCR_API | HCR_APK | HCR_ATA)
+#define HCR_HOST_NVHE_FLAGS (HCR_RW | HCR_API | HCR_APK)
 #define HCR_HOST_NVHE_PROTECTED_FLAGS (HCR_HOST_NVHE_FLAGS | HCR_TSC)
 #define HCR_HOST_VHE_FLAGS (HCR_RW | HCR_TGE | HCR_E2H | HCR_AMO | HCR_IMO | HCR_FMO)
 
arch/arm64/kernel/head.S +1 -1
index ca04b338cb0d..87a822e5c4ca 100644
@@ -299,7 +299,7 @@ SYM_INNER_LABEL(init_el2, SYM_L_LOCAL)
 	isb
 0:
 
-	init_el2_hcr	HCR_HOST_NVHE_FLAGS
+	init_el2_hcr	HCR_HOST_NVHE_FLAGS | HCR_ATA
 	init_el2_state
 
 	/* Hypervisor stub */
arch/arm64/kvm/arm.c +6 -0
index 4f80da0c0d1d..aeac113e5e74 100644
@@ -2044,6 +2044,12 @@ static void __init cpu_prepare_hyp_mode(int cpu, u32 hyp_va_bits)
 		params->hcr_el2 = HCR_HOST_NVHE_PROTECTED_FLAGS;
 	else
 		params->hcr_el2 = HCR_HOST_NVHE_FLAGS;
+
+	if (system_supports_mte())
+		params->hcr_el2 |= HCR_ATA;
+	else
+		params->hcr_el2 |= HCR_TID5;
+
 	if (cpus_have_final_cap(ARM64_KVM_HVHE))
 		params->hcr_el2 |= HCR_E2H;
 	params->vttbr = params->vtcr = 0;
Signature · 1 line
2.52.0.457.g6b5491de43-goog
author
#3 lore raw

PATCH v3 3/4 KVM: arm64: Inject UNDEF when accessing MTE sysregs with MTE disabled

In reply to Fuad Tabba

When MTE hardware is present but disabled via software (arm64.nomte or CONFIG_ARM64_MTE=n), the kernel clears HCR_EL2.ATA and sets HCR_EL2.TID5, to prevent the use of MTE instructions.

Additionally, accesses to certain MTE system registers trap to EL2 with exception class ESR_ELx_EC_SYS64. To emulate hardware without MTE (where such accesses would cause an Undefined Instruction exception), inject UNDEF into the host.

---
 arch/arm64/kvm/hyp/nvhe/hyp-main.c | 67 ++++++++++++++++++++++++++++++
 1 file changed, 67 insertions(+)
arch/arm64/kvm/hyp/nvhe/hyp-main.c +67 -0
index a7c689152f68..faed1b38e6cc 100644
@@ -687,6 +687,69 @@ static void handle_host_smc(struct kvm_cpu_context *host_ctxt)
 	kvm_skip_host_instr();
 }
 
+/*
+ * Inject an Undefined Instruction exception into the host.
+ *
+ * This is open-coded to allow control over PSTATE construction without
+ * complicating the generic exception entry helpers.
+ */
+static void inject_undef64(void)
+{
+	u64 spsr_mask, vbar, sctlr, old_spsr, new_spsr, esr, offset;
+
+	spsr_mask = PSR_N_BIT | PSR_Z_BIT | PSR_C_BIT | PSR_V_BIT | PSR_DIT_BIT | PSR_PAN_BIT;
+
+	vbar = read_sysreg_el1(SYS_VBAR);
+	sctlr = read_sysreg_el1(SYS_SCTLR);
+	old_spsr = read_sysreg_el2(SYS_SPSR);
+
+	new_spsr = old_spsr & spsr_mask;
+	new_spsr |= PSR_D_BIT | PSR_A_BIT | PSR_I_BIT | PSR_F_BIT;
+	new_spsr |= PSR_MODE_EL1h;
+
+	if (!(sctlr & SCTLR_EL1_SPAN))
+		new_spsr |= PSR_PAN_BIT;
+
+	if (sctlr & SCTLR_ELx_DSSBS)
+		new_spsr |= PSR_SSBS_BIT;
+
+	if (system_supports_mte())
+		new_spsr |= PSR_TCO_BIT;
+
+	esr = (ESR_ELx_EC_UNKNOWN << ESR_ELx_EC_SHIFT) | ESR_ELx_IL;
+	offset = CURRENT_EL_SP_ELx_VECTOR + except_type_sync;
+
+	write_sysreg_el1(esr, SYS_ESR);
+	write_sysreg_el1(read_sysreg_el2(SYS_ELR), SYS_ELR);
+	write_sysreg_el1(old_spsr, SYS_SPSR);
+	write_sysreg_el2(vbar + offset, SYS_ELR);
+	write_sysreg_el2(new_spsr, SYS_SPSR);
+}
+
+static bool handle_host_mte(u64 esr)
+{
+	switch (esr_sys64_to_sysreg(esr)) {
+	case SYS_RGSR_EL1:
+	case SYS_GCR_EL1:
+	case SYS_TFSR_EL1:
+	case SYS_TFSRE0_EL1:
+		/* If we're here for any reason other than MTE, it's a bug. */
+		if (read_sysreg(HCR_EL2) & HCR_ATA)
+			return false;
+		break;
+	case SYS_GMID_EL1:
+		/* If we're here for any reason other than MTE, it's a bug. */
+		if (!(read_sysreg(HCR_EL2) & HCR_TID5))
+			return false;
+		break;
+	default:
+		return false;
+	}
+
+	inject_undef64();
+	return true;
+}
+
 void handle_trap(struct kvm_cpu_context *host_ctxt)
 {
 	u64 esr = read_sysreg_el2(SYS_ESR);
@@ -702,6 +765,10 @@ void handle_trap(struct kvm_cpu_context *host_ctxt)
 	case ESR_ELx_EC_DABT_LOW:
 		handle_host_mem_abort(host_ctxt);
 		break;
+	case ESR_ELx_EC_SYS64:
+		if (handle_host_mte(esr))
+			break;
+		fallthrough;
 	default:
 		BUG();
 	}
Signature · 1 line
2.52.0.457.g6b5491de43-goog
author
#4 lore raw

PATCH v3 4/4 KVM: arm64: Use kvm_has_mte() in pKVM trap initialization

In reply to Fuad Tabba

When initializing HCR traps in protected mode, use kvm_has_mte() to check for MTE support rather than kvm_has_feat(kvm, ID_AA64PFR1_EL1, MTE, IMP).

kvm_has_mte() provides a more comprehensive check:
- kvm_has_feat() only checks if MTE is in the guest's ID register view (i.e., what we advertise to the guest)
- kvm_has_mte() checks both system_supports_mte() AND whether KVM_ARCH_FLAG_MTE_ENABLED is set for this VM instance

---
 arch/arm64/kvm/hyp/nvhe/pkvm.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/arm64/kvm/hyp/nvhe/pkvm.c +1 -1
index 8911338961c5..9b933424e70a 100644
@@ -82,7 +82,7 @@ static void pvm_init_traps_hcr(struct kvm_vcpu *vcpu)
 	if (!kvm_has_feat(kvm, ID_AA64PFR0_EL1, AMU, IMP))
 		val &= ~(HCR_AMVOFFEN);
 
-	if (!kvm_has_feat(kvm, ID_AA64PFR1_EL1, MTE, IMP)) {
+	if (!kvm_has_mte(kvm)) {
 		val |= HCR_TID5;
 		val &= ~(HCR_DCT | HCR_ATA);
 	}
Signature · 1 line
2.52.0.457.g6b5491de43-goog
maintainer Applied
#5 lore raw
In reply to Fuad Tabba
Quoted 8 lines · Fuad Tabba

Changes since v2 [1]:
- Trap accesses to GMID_EL1 by setting HCR_EL2.TID5 when MTE is disabled (Marc)
- Dropped patch refactoring enter_exception64() in favor of open-coded logic in inject_undef64() in patch 3/4 (Marc)
- Based on Linux 6.19-rc6

[...]

Applied to next, thanks!

[1/4] KVM: arm64: Remove dead code resetting HCR_EL2 for pKVM

      commit: c103c2dfe4975da31b67a0fcb95761359f30992d
[2/4] KVM: arm64: Trap MTE access and discovery when MTE is disabled
      commit: f35abcbb8a084db4c24b66ccc8db0405c08e2f61
[3/4] KVM: arm64: Inject UNDEF when accessing MTE sysregs with MTE disabled
      commit: 5ee8ad69da07d0e2cffa0ce2f2339c9ad2d587f2
[4/4] KVM: arm64: Use kvm_has_mte() in pKVM trap initialization
      commit: 230b080623fec2e1302df2afe2cf2dcb34f9c89b

Cheers,

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.