Skip to contentSkip to conversation

KVM: arm: Refuse to enable KVM on systems with SME but not FGT

PATCH In review Mark Brown 4 participants 7 messages Oct 27, 2022 – Oct 31, 2022

Subsystems (2)
lore
Share & tools
lore

Mark Brown

author
#0 lore raw

cover KVM: arm: Refuse to enable KVM on systems with SME but not FGT

The architecture requires that any system which implements SME also has fine grained traps since SME is a v9.2 feature, meaning that v8.7 must be implemented, and FGT is mandatory from v8.6. SME relies on fine grained traps to control access to SMPRI_EL1 and in nVHE mode to TPIDR2_EL0, without traps SMPRI_EL1.Priority and TPIDR2_EL0 can be used as side channels.

This series adds support for detecting FGT and refuses to allow KVM to be used in architecturally invalid configurations which have SME but not FGT, without detection the issue presents as faults due to EL2 attempting to access the FGT registers which isn't obvious to users. Currently fine grained traps are only used in nVHE but but a series "arm64/sme: Fix SMPRI_EL1 traps for KVM guests" sent along with this will add usage for VHE mode too making the issue more pressing.

Mark Brown (2):
arm64/cpufeature: Add feature detection for fine grained traps
KVM: arm: Refuse to enable KVM on systems with FEAT_SME but not
FEAT_FGT

 arch/arm64/kernel/cpufeature.c | 11 +++++++++++
 arch/arm64/kvm/arm.c           | 15 +++++++++++++++
 arch/arm64/tools/cpucaps       |  1 +
 3 files changed, 27 insertions(+)

base-commit: 9abf2313adc1ca1b6180c508c25f22f9395cc780

Signature · 1 line
2.30.2
author
#1 lore raw

PATCH 1/2 arm64/cpufeature: Add feature detection for fine grained traps

Virtualisation support for SME depends on FEAT_FGT since it uses fine grained traps to control access to SMPRI_EL1 and (for nVHE) TPIDR2_EL0 but systems have been encountered which implement SME without VHE. In order to work around such systems we need to detect the absence of FEAT_FGT, add detection in the cpufeature code.

---
 arch/arm64/kernel/cpufeature.c | 11 +++++++++++
 arch/arm64/tools/cpucaps       |  1 +
 2 files changed, 12 insertions(+)
arch/arm64/kernel/cpufeature.c +11 -0
index 6062454a9067..8ded72a8ccee 100644
@@ -2590,6 +2590,17 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
 		.matches = has_cpuid_feature,
 		.min_field_value = 1,
 	},
+	{
+		.desc = "Fine Grained Traps",
+		.type = ARM64_CPUCAP_SYSTEM_FEATURE,
+		.capability = ARM64_HAS_FGT,
+		.sys_reg = SYS_ID_AA64MMFR0_EL1,
+		.sign = FTR_UNSIGNED,
+		.field_pos = ID_AA64MMFR0_EL1_FGT_SHIFT,
+		.field_width = ID_AA64MMFR0_EL1_FGT_WIDTH,
+		.min_field_value = ID_AA64MMFR0_EL1_FGT_IMP,
+		.matches = has_cpuid_feature,
+	},
 #ifdef CONFIG_ARM64_SME
 	{
 		.desc = "Scalable Matrix Extension",
arch/arm64/tools/cpucaps +1 -0
index f1c0347ec31a..2553c7559412 100644
@@ -23,6 +23,7 @@ HAS_DCPOP
 HAS_E0PD
 HAS_ECV
 HAS_EPAN
+HAS_FGT
 HAS_GENERIC_AUTH
 HAS_GENERIC_AUTH_ARCH_QARMA3
 HAS_GENERIC_AUTH_ARCH_QARMA5
Signature · 1 line
2.30.2
author
#2 lore raw

PATCH 2/2 KVM: arm: Refuse to enable KVM on systems with FEAT_SME but not FEAT_FGT

In reply to Mark Brown

The architecture requires that any system which implements SME also has fine grained traps since SME is a v9.2 feature, meaning that v8.7 must be implemented, and FGT is mandatory from v8.6. Virtualisation support for SME relies on fine grained traps to control access to SMPRI_EL1 and in nVHE mode to TPIDR2_EL0, without traps SMPRI_EL1.Priority and TPIDR2_EL0 can be used as side channels even if SME support is not exposed to the guest.

---
 arch/arm64/kvm/arm.c | 15 +++++++++++++++
 1 file changed, 15 insertions(+)
arch/arm64/kvm/arm.c +15 -0
index 94d33e296e10..4662407ee789 100644
@@ -2183,6 +2183,21 @@ int kvm_arch_init(void *opaque)
 		return -ENODEV;
 	}
 
+	/*
+	 * SME without fine grained traps is an architecturally
+	 * invalid configuration since SME is a v9.2 feature and FGT
+	 * is required from v8.6 but virtual platforms have been
+	 * encountered which don't respect this. Without FGT we can't
+	 * trap access to TPIDR2_EL0 in nVHE mode or SMPRI_EL1 in any
+	 * mode, making this conditional in the code would lead to
+	 * side channels on these out of spec systems.
+	 */
+	if (cpus_have_final_cap(ARM64_SME) &&
+	    !cpus_have_final_cap(ARM64_HAS_FGT)) {
+		kvm_err("KVM disabled since system has SME without FGT\n");
+		return -ENODEV;
+	}
+
 	if (kvm_get_mode() == KVM_MODE_NONE) {
 		kvm_info("KVM disabled from command line\n");
 		return -ENODEV;
Signature · 1 line
2.30.2
Reviewed-by on the v1 series
#3 lore raw
In reply to Mark Brown
Quoted 7 lines · Mark Brown

The architecture requires that any system which implements SME also has fine grained traps since SME is a v9.2 feature, meaning that v8.7 must be implemented, and FGT is mandatory from v8.6. SME relies on fine grained traps to control access to SMPRI_EL1 and in nVHE mode to TPIDR2_EL0, without traps SMPRI_EL1.Priority and TPIDR2_EL0 can be used as side channels.

Quoted 7 lines

This series adds support for detecting FGT and refuses to allow KVM to be used in architecturally invalid configurations which have SME but not FGT, without detection the issue presents as faults due to EL2 attempting to access the FGT registers which isn't obvious to users. Currently fine grained traps are only used in nVHE but but a series "arm64/sme: Fix SMPRI_EL1 traps for KVM guests" sent along with this will add usage for VHE mode too making the issue more pressing.

Series:

r~

maintainer
#4 lore raw
In reply to Mark Brown
Quoted 8 lines · Mark Brown

The architecture requires that any system which implements SME also has fine grained traps since SME is a v9.2 feature, meaning that v8.7 must be implemented, and FGT is mandatory from v8.6. SME relies on fine grained traps to control access to SMPRI_EL1 and in nVHE mode to TPIDR2_EL0, without traps SMPRI_EL1.Priority and TPIDR2_EL0 can be used as side channels.

Quoted 7 lines

This series adds support for detecting FGT and refuses to allow KVM to be used in architecturally invalid configurations which have SME but not FGT, without detection the issue presents as faults due to EL2 attempting to access the FGT registers which isn't obvious to users. Currently fine grained traps are only used in nVHE but but a series "arm64/sme: Fix SMPRI_EL1 traps for KVM guests" sent along with this will add usage for VHE mode too making the issue more pressing.

I think this goes the wrong way around. SME without FGT is invalid, and yet you keep SME around and device to kill virtualisation support.

I'd rather it is SME that gets disabled when the kernel boots at EL2. Furthermore, this is only working around a QEMU issue which can be fixed (as opposed to HW that is forever baked).

To me, it looks like the most reasonable course of action is a mention in the QEMU release notes that virtualization and SME are currently incompatible, and that the user needs to chose one or the other.

We had similar issues in the past where QEMU would ignore certain trap bits (HCR_EL2.TID{1,2,3}), leading to KVM misbehaving. Did we disable KVM? No, we fixed QEMU instead. This case isn't different.

Thanks,

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
maintainer
#5 lore raw
Quoted 18 lines · Marc Zyngier
Quoted 8 lines · Mark Brown

The architecture requires that any system which implements SME also has fine grained traps since SME is a v9.2 feature, meaning that v8.7 must be implemented, and FGT is mandatory from v8.6. SME relies on fine grained traps to control access to SMPRI_EL1 and in nVHE mode to TPIDR2_EL0, without traps SMPRI_EL1.Priority and TPIDR2_EL0 can be used as side channels.

Quoted 7 lines

This series adds support for detecting FGT and refuses to allow KVM to be used in architecturally invalid configurations which have SME but not FGT, without detection the issue presents as faults due to EL2 attempting to access the FGT registers which isn't obvious to users. Currently fine grained traps are only used in nVHE but but a series "arm64/sme: Fix SMPRI_EL1 traps for KVM guests" sent along with this will add usage for VHE mode too making the issue more pressing.

Quoted 4 lines

I think this goes the wrong way around. SME without FGT is invalid, and yet you keep SME around and device to kill virtualisation support.

I'd rather it is SME that gets disabled when the kernel boots at EL2.

I agree, I rather we did it the other way around.

Signature · 1 line
Catalin
author
#6 lore raw
In reply to Marc Zyngier
Quoted 6 lines · Marc Zyngier

I think this goes the wrong way around. SME without FGT is invalid, and yet you keep SME around and device to kill virtualisation support.

I'd rather it is SME that gets disabled when the kernel boots at EL2. Furthermore, this is only working around a QEMU issue which can be fixed (as opposed to HW that is forever baked).

I do agree that it would be better if we were able to do this the other way around however access to SMPRI_EL1 at EL1 is not controlled by the SME enable control for EL2, it is controlled by either the fine grained trap or the EL3 SME enable trap. This was the primary driver here rather than the issues with writing to the FGT registers.

Quoted 3 lines

To me, it looks like the most reasonable course of action is a mention in the QEMU release notes that virtualization and SME are currently incompatible, and that the user needs to chose one or the other.

Trying to resolve it at the qemu level would certainly be fine, though note that qemu has already released versions with problems so there's a bit of stable door/horse issue and it'd just be a note on the web site and any point releases.

Quoted 3 lines

We had similar issues in the past where QEMU would ignore certain trap bits (HCR_EL2.TID{1,2,3}), leading to KVM misbehaving. Did we disable KVM? No, we fixed QEMU instead. This case isn't different.

OK.