KVM: arm64: Enforce MTE disablement at EL2
v2 cover KVM: arm64: Enforce MTE disablement at EL2
pKVM never exposes MTE to protected guests (pVM), but we must also ensure a malicious host cannot use MTE to attack the hypervisor or a pVM.
If MTE is supported by the hardware (and is enabled at EL3), it remains available to lower exception levels by default. Disabling it in the host kernel (e.g., via 'arm64.nomte') only stops the kernel from advertising the feature; it does not physically disable MTE in the hardware.
The ability to disable MTE in the host kernel is used by some systems, such as Android, so that the physical memory otherwise used as tag storage can be used for other things (i.e. treated just like the rest of memory).
In this scenario, a malicious host could still access tags in pages donated to a guest using MTE instructions (e.g., STG and LDG), bypassing the kernel's configuration.
To prevent this, explicitly disable MTE at EL2 (by clearing HCR_EL2.ATA) when the host has MTE disabled. This causes any MTE instruction usage to generate a Data Abort (trap) to the hypervisor.
Additionally, to faithfully mimic hardware that does not support MTE, trap accesses to MTE system registers (e.g., GCR_EL1) and inject an Undefined Instruction exception back to the host.
This logic is applied in all non-VHE modes. For non-protected modes, this remains beneficial as it prevents unpredictable behavior caused by accessing allocation tags when the system considers them disabled.
Note that this ties into my other outgoing patch series [1], which also has some MTE-related fixes, but is not dependent on it.
Based on Linux 6.18
Cheers,
/fuad
[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/
Fuad Tabba (4):
arm64: Remove dead code resetting HCR_EL2 for pKVM
arm64: Clear HCR_EL2.ATA when MTE is not supported or disabled
arm64: Inject UNDEF when accessing MTE sysregs with MTE disabled
KVM: arm64: Use kvm_has_mte() in pKVM trap initialization
Quentin Perret (1):
KVM: arm64: Refactor enter_exception64()
arch/arm64/include/asm/kvm_arm.h | 2 +- arch/arm64/include/asm/kvm_emulate.h | 5 ++ arch/arm64/kernel/head.S | 2 +- arch/arm64/kvm/arm.c | 4 ++ arch/arm64/kvm/hyp/exception.c | 100 ++++++++++++++++----------- arch/arm64/kvm/hyp/nvhe/hyp-init.S | 5 -- arch/arm64/kvm/hyp/nvhe/hyp-main.c | 44 ++++++++++++ arch/arm64/kvm/hyp/nvhe/pkvm.c | 2 +- 8 files changed, 114 insertions(+), 50 deletions(-)
base-commit: 7d0a66e4bb9081d75c82ec4957c50034cb0ea449
Signature · 1 line
2.52.0.223.gf5cc29aaa4-goog
PATCH v2 1/5 arm64: Remove dead code resetting HCR_EL2 for pKVM
The pKVM lifecycle does not support tearing down the hypervisor and returning to the hyp stub once initialized. The transition to protected mode is one-way.
Consequently, the code path in hyp-init.S responsible for resetting EL2 registers (triggered by kexec or hibernation) is unreachable in protected mode.
Remove the dead code handling HCR_EL2 reset forARM64_KVM_PROTECTED_MODE.
No functional change intended.
--- arch/arm64/kvm/hyp/nvhe/hyp-init.S | 5 ----- 1 file changed, 5 deletions(-)
arch/arm64/kvm/hyp/nvhe/hyp-init.S +0 -5
@@ -260,11 +260,6 @@ reset: msr sctlr_el2, x5 isb -alternative_if ARM64_KVM_PROTECTED_MODE - mov_q x5, HCR_HOST_NVHE_FLAGS - msr_hcr_el2 x5 -alternative_else_nop_endif - /* Install stub vectors */ adr_l x5, __hyp_stub_vectors msr vbar_el2, x5
Signature · 1 line
2.52.0.223.gf5cc29aaa4-goog
PATCH v2 2/5 arm64: Clear HCR_EL2.ATA when MTE is not supported or disabled
In reply to Fuad TabbaIf MTE is not supported by the hardware, or is disabled in the kernel configuration (CONFIG_ARM64_MTE=n) or command line (arm64.nomte), the kernel stops advertising MTE to userspace and avoids using MTE instructions. However, this is a software-level disable only.
When MTE hardware is present and enabled by EL3 firmware, leaving HCR_EL2.ATA set allows the host to execute MTE instructions (STG, LDG, etc.) and access allocation tags in physical memory. This creates a security risk where a malicious or buggy host could lead to system crashes, undefined behavior, or compromise guests.
Prevent this by clearing HCR_EL2.ATA when MTE is disabled. Remove it from the HCR_HOST_NVHE_FLAGS default, and conditionally set it in cpu_prepare_hyp_mode() only when system_supports_mte() returns true. This causes MTE instructions to trap to EL2 when HCR_EL2.ATA is cleared.
Early boot code in head.S temporarily keeps HCR_ATA set to avoid special-casing initialization paths. This is safe because this code executes before untrusted code runs and will clear HCR_ATA if MTE is disabled.
--- arch/arm64/include/asm/kvm_arm.h | 2 +- arch/arm64/kernel/head.S | 2 +- arch/arm64/kvm/arm.c | 4 ++++ 3 files changed, 6 insertions(+), 2 deletions(-)
arch/arm64/include/asm/kvm_arm.h +1 -1
@@ -101,7 +101,7 @@ HCR_BSU_IS | HCR_FB | HCR_TACR | \ HCR_AMO | HCR_SWIO | HCR_TIDCP | HCR_RW | HCR_TLOR | \ HCR_FMO | HCR_IMO | HCR_PTW | HCR_TID3 | HCR_TID1) -#define HCR_HOST_NVHE_FLAGS (HCR_RW | HCR_API | HCR_APK | HCR_ATA) +#define HCR_HOST_NVHE_FLAGS (HCR_RW | HCR_API | HCR_APK)#define HCR_HOST_NVHE_PROTECTED_FLAGS (HCR_HOST_NVHE_FLAGS | HCR_TSC)#define HCR_HOST_VHE_FLAGS (HCR_RW | HCR_TGE | HCR_E2H | HCR_AMO | HCR_IMO | HCR_FMO)
arch/arm64/kernel/head.S +1 -1
@@ -299,7 +299,7 @@ SYM_INNER_LABEL(init_el2, SYM_L_LOCAL) isb 0: - init_el2_hcr HCR_HOST_NVHE_FLAGS + init_el2_hcr HCR_HOST_NVHE_FLAGS | HCR_ATA init_el2_state /* Hypervisor stub */
arch/arm64/kvm/arm.c +4 -0
@@ -2030,6 +2030,10 @@ static void __init cpu_prepare_hyp_mode(int cpu, u32 hyp_va_bits) params->hcr_el2 =HCR_HOST_NVHE_PROTECTED_FLAGS ; else params->hcr_el2 = HCR_HOST_NVHE_FLAGS; + +if (system_supports_mte ()) + params->hcr_el2 |=HCR_ATA ; +if (cpus_have_final_cap (ARM64_KVM_HVHE )) params->hcr_el2 |=HCR_E2H ; params->vttbr = params->vtcr =0 ;
Signature · 1 line
2.52.0.223.gf5cc29aaa4-goog
Marc Zyngier
maintainerRe: [PATCH v2 2/5] arm64: Clear HCR_EL2.ATA when MTE is not supported or disabled
Quoted 60 lines · Fuad Tabba
If MTE is not supported by the hardware, or is disabled in the kernel configuration (
CONFIG_ARM64_MTE=n) or command line (arm64.nomte), the kernel stops advertising MTE to userspace and avoids using MTE instructions. However, this is a software-level disable only.When MTE hardware is present and enabled by EL3 firmware, leaving HCR_EL2.ATA set allows the host to execute MTE instructions (STG, LDG, etc.) and access allocation tags in physical memory. This creates a security risk where a malicious or buggy host could lead to system crashes, undefined behavior, or compromise guests.
Prevent this by clearing HCR_EL2.ATA when MTE is disabled. Remove it from the
HCR_HOST_NVHE_FLAGSdefault, and conditionally set it incpu_prepare_hyp_mode()only whensystem_supports_mte()returns true. This causes MTE instructions to trap to EL2 when HCR_EL2.ATA is cleared.Early boot code in
head.Stemporarily keepsHCR_ATAset to avoid special-casing initialization paths. This is safe because this code executes before untrusted code runs and will clearHCR_ATAif MTE is disabled.Signed-off-by Fuad Tabba <[email protected]>--- arch/arm64/include/asm/kvm_arm.h | 2 +- arch/arm64/kernel/head.S | 2 +- arch/arm64/kvm/arm.c | 4 ++++ 3 files changed, 6 insertions(+), 2 deletions(-)arch/arm64/include/asm/kvm_arm.h +1
-1@@ -101,7 +101,7 @@ HCR_BSU_IS | HCR_FB | HCR_TACR | \ HCR_AMO | HCR_SWIO | HCR_TIDCP | HCR_RW | HCR_TLOR | \ HCR_FMO | HCR_IMO | HCR_PTW | HCR_TID3 | HCR_TID1) -#define HCR_HOST_NVHE_FLAGS (HCR_RW | HCR_API | HCR_APK | HCR_ATA) +#define HCR_HOST_NVHE_FLAGS (HCR_RW | HCR_API | HCR_APK)#define HCR_HOST_NVHE_PROTECTED_FLAGS (HCR_HOST_NVHE_FLAGS | HCR_TSC)#define HCR_HOST_VHE_FLAGS (HCR_RW | HCR_TGE | HCR_E2H | HCR_AMO | HCR_IMO | HCR_FMO)arch/arm64/kernel/head.S +1
-1@@ -299,7 +299,7 @@ SYM_INNER_LABEL(init_el2, SYM_L_LOCAL) isb 0: - init_el2_hcr HCR_HOST_NVHE_FLAGS + init_el2_hcr HCR_HOST_NVHE_FLAGS | HCR_ATA init_el2_state /* Hypervisor stub */arch/arm64/kvm/arm.c +0
-0
arch/arm64/kvm/arm.c static void __init cpu_prepare_hyp_mode(int cpu, u32 hyp_va_bits) · line 2036
Quoted 7 lines
arch/arm64/kvm/arm.c +4
-0@@ -2030,6 +2030,10 @@ static void __init cpu_prepare_hyp_mode(int cpu, u32 hyp_va_bits) params->hcr_el2 =HCR_HOST_NVHE_PROTECTED_FLAGS ; else params->hcr_el2 = HCR_HOST_NVHE_FLAGS; + +if (system_supports_mte ()) + params->hcr_el2 |=HCR_ATA ; +
How about TID5? If you really want to hide MTE, you also need to catch accesses to GMID_EL1.
Thanks,
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
PATCH v2 3/5 KVM: arm64: Refactor enter_exception64()
In reply to Fuad TabbaFrom: Quentin Perret <[email protected]>
To simplify the injection of exceptions into the host in pKVM context, refactor enter_exception64() to split out the logic for calculating the exception vector offset and the target CPSR.
Extract two new helper functions:
- get_except64_offset(): Calculates exception vector offset based on current/target exception levels and exception type
- get_except64_cpsr(): Computes the new CPSR/PSTATE when taking an exception
A subsequent patch will use these helpers to inject UNDEF exceptions into the host when MTE system registers are accessed with MTE disabled. Extracting the helpers allows that code to reuse the exception entry logic without duplicating the CPSR and vector offset calculations.
No functional change intended.
--- arch/arm64/include/asm/kvm_emulate.h | 5 ++ arch/arm64/kvm/hyp/exception.c | 100 ++++++++++++++++----------- 2 files changed, 63 insertions(+), 42 deletions(-)
arch/arm64/include/asm/kvm_emulate.h +5 -0
@@ -71,6 +71,11 @@ static inline int kvm_inject_serror(struct kvm_vcpu *vcpu)return kvm_inject_serror_esr (vcpu, ESR_ELx_ISV); } +unsigned long get_except64_offset (unsigned long psr,unsigned long target_mode, +enum exception_type type); +unsigned long get_except64_cpsr (unsigned long old,bool has_mte, +unsigned long sctlr,unsigned long mode); +void kvm_vcpu_wfi (struct kvm_vcpu *vcpu);void kvm_emulate_nested_eret (struct kvm_vcpu *vcpu);
arch/arm64/kvm/hyp/exception.c +58 -42
126 lines not on this page · raw message
Signature · 1 line
2.52.0.223.gf5cc29aaa4-goog
Marc Zyngier
maintainerRe: [PATCH v2 3/5] KVM: arm64: Refactor enter_exception64()
Quoted 32 lines · Fuad Tabba
From: Quentin Perret <[email protected]>
To simplify the injection of exceptions into the host in pKVM context, refactor
enter_exception64()to split out the logic for calculating the exception vector offset and the target CPSR.Extract two new helper functions:
-get_except64_offset(): Calculates exception vector offset based on current/target exception levels and exception type
-get_except64_cpsr(): Computes the new CPSR/PSTATE when taking an exceptionA subsequent patch will use these helpers to inject UNDEF exceptions into the host when MTE system registers are accessed with MTE disabled. Extracting the helpers allows that code to reuse the exception entry logic without duplicating the CPSR and vector offset calculations.
No functional change intended.
Signed-off-by Quentin Perret <[email protected]>Signed-off-by Fuad Tabba <[email protected]>--- arch/arm64/include/asm/kvm_emulate.h | 5 ++ arch/arm64/kvm/hyp/exception.c | 100 ++++++++++++++++----------- 2 files changed, 63 insertions(+), 42 deletions(-)arch/arm64/include/asm/kvm_emulate.h +0
-0
arch/arm64/include/asm/kvm_emulate.h static inline int kvm_inject_serror(struct kvm_vcpu *vcpu) · line 77
Quoted 7 lines
arch/arm64/include/asm/kvm_emulate.h +4
-0@@ -71,6 +71,11 @@ static inline int kvm_inject_serror(struct kvm_vcpu *vcpu)return kvm_inject_serror_esr (vcpu, ESR_ELx_ISV); } +unsigned long get_except64_offset (unsigned long psr,unsigned long target_mode, +enum exception_type type); +unsigned long get_except64_cpsr (unsigned long old,bool has_mte, +unsigned long sctlr,unsigned long mode);
s/cpsr/pstate/ as we don't need to introduce more 32bit terminology.
Quoted 40 lines
arch/arm64/include/asm/kvm_emulate.h +1
-0+void kvm_vcpu_wfi (struct kvm_vcpu *vcpu);void kvm_emulate_nested_eret (struct kvm_vcpu *vcpu);arch/arm64/kvm/hyp/exception.c +18
-5@@ -65,12 +65,25 @@ static void __vcpu_write_spsr_und(struct kvm_vcpu *vcpu, u64 val) vcpu->arch.ctxt.spsr_und = val; } +unsigned long get_except64_offset (unsigned long psr,unsigned long target_mode, +enum exception_type type) +{ +u64 mode = psr & (PSR_MODE_MASK |PSR_MODE32_BIT ); +u64 exc_offset; + +if (mode == target_mode) + exc_offset = CURRENT_EL_SP_ELx_VECTOR; +else if ((mode |PSR_MODE_THREAD_BIT ) == target_mode) + exc_offset =CURRENT_EL_SP_EL0_VECTOR ; +else if (!(mode &PSR_MODE32_BIT )) + exc_offset = LOWER_EL_AArch64_VECTOR; +else + exc_offset = LOWER_EL_AArch32_VECTOR; + +return exc_offset + type; +} +/* -* This performs the exception entry at a given EL (@target_mode), stashing PC -* and PSTATE into ELR and SPSR respectively, and compute the new PC/PSTATE. -* The EL passed to this function *must* be a non-secure, privileged mode with -* bit 0 being set (PSTATE.SP == 1). -* * When an exception is taken, most PSTATE fields are left unchanged in the * handler. However, some are explicitly overridden (e.g. M[4:0]). Luckily all * of the inherited bits have the same position in the AArch64/AArch32 SPSR_ELx
arch/arm64/kvm/hyp/exception.c static void __vcpu_write_spsr_und(struct kvm_vcpu *vcpu, u64 val) · line 99
Quoted 7 lines
arch/arm64/kvm/hyp/exception.c +2
-2@@ -82,50 +95,17 @@ static void __vcpu_write_spsr_und(struct kvm_vcpu *vcpu, u64 val)* Here we manipulate the fields in order of the AArch64 SPSR_ELx layout, from * MSB to LSB. */ -static void enter_exception64 (struct kvm_vcpu *vcpu,unsigned long target_mode, -enum exception_type type) +unsigned long get_except64_cpsr (unsigned long old,bool has_mte, +unsigned long sctlr,unsigned long target_mode)
I really dislike the has_mte and sctlr thing.
The main reason is that it will not scale as we end-up hiding more feature on the host (think PM and FEAT_EBEP, for example). Even worse, some bits are not necessarily sourced from PSTATE, nor SCTLR (EXLOCK depends on GCSCR_ELx, for example).
I think you really need to turn this into something that is flexible enough that it can work for both host and guest, with an actual abstraction. It is likely to look like a list of register accessors to get to the correct data.
But it could well be that open-coding it is the least horrid solution.
Thanks,
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
PATCH v2 4/5 arm64: Inject UNDEF when accessing MTE sysregs with MTE disabled
In reply to Fuad TabbaWhen MTE hardware is present but disabled via software (arm64.nomte or CONFIG_ARM64_MTE=n), HCR_EL2.ATA is cleared to prevent use of MTE instructions. However, this alone doesn't fully emulate hardware that lacks MTE support.
With HCR_EL2.ATA cleared, accesses to certain MTE system registers trap to EL2 with exception class ESR_ELx_EC_SYS64. To faithfully emulate hardware without MTE (where such accesses would cause an Undefined Instruction exception), inject UNDEF into the host.
The ability to disable MTE in the host kernel is used by some systems, such as Android, so that the physical memory otherwise used as tag storage can be used for other things (i.e. treated just like the rest of memory).
--- arch/arm64/kvm/hyp/nvhe/hyp-main.c | 44 ++++++++++++++++++++++++++++++ 1 file changed, 44 insertions(+)
arch/arm64/kvm/hyp/nvhe/hyp-main.c +44 -0
@@ -686,6 +686,46 @@ static void handle_host_smc(struct kvm_cpu_context *host_ctxt)kvm_skip_host_instr (); } +static void inject_undef64 (void ) +{ +unsigned long sctlr, vbar, old, new; +u64 offset, esr; + + vbar =read_sysreg_el1 (SYS_VBAR ); + sctlr =read_sysreg_el1 (SYS_SCTLR ); + old =read_sysreg_el2 (SYS_SPSR ); + new =get_except64_cpsr (old,system_supports_mte (), sctlr, PSR_MODE_EL1h); + offset =get_except64_offset (old, PSR_MODE_EL1h, except_type_sync); + esr = (ESR_ELx_EC_UNKNOWN << ESR_ELx_EC_SHIFT) | ESR_ELx_IL; + +write_sysreg_el1 (esr,SYS_ESR ); +write_sysreg_el1 (read_sysreg_el2 (SYS_ELR ),SYS_ELR ); +write_sysreg_el1 (old,SYS_SPSR ); +write_sysreg_el2 (vbar + offset,SYS_ELR ); +write_sysreg_el2 (new,SYS_SPSR ); +} + +static bool handle_host_mte (u64 esr) +{ +/* If we're here for any reason other than MTE, then it's a bug. */ + +if (read_sysreg (HCR_EL2 ) &HCR_ATA ) +return false; + +switch (esr_sys64_to_sysreg (esr)) { +case SYS_RGSR_EL1 : +case SYS_GCR_EL1 : +case SYS_TFSR_EL1 : +case SYS_TFSRE0_EL1 : +break ; +default : +return false; + } + +inject_undef64 (); +return true; +} +void handle_trap (struct kvm_cpu_context *host_ctxt) {u64 esr =read_sysreg_el2 (SYS_ESR );
@@ -701,6 +741,10 @@ void handle_trap(struct kvm_cpu_context *host_ctxt)case ESR_ELx_EC_DABT_LOW:handle_host_mem_abort (host_ctxt);break ; +case ESR_ELx_EC_SYS64: +if (handle_host_mte (esr)) +break ; + fallthrough;default :BUG (); }
Signature · 1 line
2.52.0.223.gf5cc29aaa4-goog
PATCH v2 5/5 KVM: arm64: Use kvm_has_mte() in pKVM trap initialization
In reply to Fuad TabbaWhen initializing HCR traps in protected mode, use kvm_has_mte() to check for MTE support rather than kvm_has_feat(kvm, ID_AA64PFR1_EL1, MTE, IMP).
kvm_has_mte() provides a more comprehensive check:
- kvm_has_feat() only checks if MTE is in the guest's ID register view (i.e., what we advertise to the guest)
- kvm_has_mte() checks both system_supports_mte() AND whether KVM_ARCH_FLAG_MTE_ENABLED is set for this VM instance
--- arch/arm64/kvm/hyp/nvhe/pkvm.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/arm64/kvm/hyp/nvhe/pkvm.c +1 -1
@@ -82,7 +82,7 @@ static void pvm_init_traps_hcr(struct kvm_vcpu *vcpu)if (!kvm_has_feat (kvm,ID_AA64PFR0_EL1 ,AMU ,IMP )) val &= ~(HCR_AMVOFFEN ); -if (!kvm_has_feat (kvm,ID_AA64PFR1_EL1 ,MTE ,IMP )) { +if (!kvm_has_mte (kvm)) { val |=HCR_TID5 ; val &= ~(HCR_DCT |HCR_ATA ); }
Signature · 1 line
2.52.0.223.gf5cc29aaa4-goog
Hi,
I was wondering if there's anything else you would like me to do for this series. Should I rebase it on 6.19-rc6?
Cheers,
/fuad
Quoted 66 lines · Fuad Tabba
pKVM never exposes MTE to protected guests (pVM), but we must also ensure a malicious host cannot use MTE to attack the hypervisor or a pVM.
If MTE is supported by the hardware (and is enabled at EL3), it remains available to lower exception levels by default. Disabling it in the host kernel (e.g., via 'arm64.nomte') only stops the kernel from advertising the feature; it does not physically disable MTE in the hardware.
The ability to disable MTE in the host kernel is used by some systems, such as Android, so that the physical memory otherwise used as tag storage can be used for other things (i.e. treated just like the rest of memory).
In this scenario, a malicious host could still access tags in pages donated to a guest using MTE instructions (e.g., STG and LDG), bypassing the kernel's configuration.
To prevent this, explicitly disable MTE at EL2 (by clearing HCR_EL2.ATA) when the host has MTE disabled. This causes any MTE instruction usage to generate a Data Abort (trap) to the hypervisor.
Additionally, to faithfully mimic hardware that does not support MTE, trap accesses to MTE system registers (e.g.,
GCR_EL1) and inject an Undefined Instruction exception back to the host.This logic is applied in all non-VHE modes. For non-protected modes, this remains beneficial as it prevents unpredictable behavior caused by accessing allocation tags when the system considers them disabled.
Note that this ties into my other outgoing patch series [1], which also has some MTE-related fixes, but is not dependent on it.
Based on Linux 6.18
Cheers,
/fuad[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/Fuad Tabba (4):
arm64: Remove dead code resettingHCR_EL2for pKVM
arm64: Clear HCR_EL2.ATA when MTE is not supported or disabled
arm64: Inject UNDEF when accessing MTE sysregs with MTE disabled
KVM: arm64: Usekvm_has_mte()in pKVM trap initializationQuentin Perret (1):
KVM: arm64: Refactorenter_exception64()arch/arm64/include/asm/kvm_arm.h | 2 +- arch/arm64/include/asm/kvm_emulate.h | 5 ++ arch/arm64/kernel/head.S | 2 +- arch/arm64/kvm/arm.c | 4 ++ arch/arm64/kvm/hyp/exception.c | 100 ++++++++++++++++----------- arch/arm64/kvm/hyp/nvhe/hyp-init.S | 5 -- arch/arm64/kvm/hyp/nvhe/hyp-main.c | 44 ++++++++++++ arch/arm64/kvm/hyp/nvhe/pkvm.c | 2 +- 8 files changed, 114 insertions(+), 50 deletions(-)base-commit:
7d0a66e4bb9081d75c82ec4957c50034cb0ea449--
2.52.0.223.gf5cc29aaa4-goog