KVM: arm64: NV userspace ABI

PATCH Applied Marc Zyngier 6 participants 23 messages Feb 20, 2025 – Feb 24, 2025

Subsystems (4)

Documentation KVM arm64 KVM arm64

lore

Share & tools

lore

Marc Zyngier

author maintainer

2025-02-20 13:49 UTC #0 lore raw

v2 cover KVM: arm64: NV userspace ABI

Since the previous incarnation of the ABI was proved to be subtly wrong, I have reworked it to be more in line with the current way KVM operates.

No more late NV-specific adjustment nor writable ID_AA64MMFR0_EL1.VH. The NV configuration is now entirely selected from the vcpu flags. I've preserved the KVM_ARM_VCPU_EL2 flag which enables NV with VHE, and added KVM_ARM_VCPU_EL2_E2H0 which alters the NV behaviour to only allow nVHE guests without recursive NV support.

This series is actually very little new code. The bulk of it is converting the feature downgrade to be per-idreg, essentially going back to the state before 44241f34fac96 ("KVM: arm64: nv: Use accessors for modifying ID registers"), only slightly modernised. This then becomes part of the reset value computing.

The rest is simply what you'd expect in terms of being able to write the ID_AA64MMFR4_EL1.NV_frac field, making the correct bits RES0 when needed, probing for capabilities and handling the init flags.

Patches on top of -rc3, with the integration branch at the usual location.

* From v1 [1]

- Fixed mishandling of ID_UNALLOCATED(), resulting in extra consolidation and simplify the macro maze a bit

- Picked up Oliver's RBs (thanks!)

- Rebased on top of -rc3

[1] https://lore.kernel.org/r/[email protected]

Marc Zyngier (14):
arm64: cpufeature: Handle NV_frac as a synonym of NV2
KVM: arm64: Hide ID_AA64MMFR2_EL1.NV from guest and userspace
KVM: arm64: Mark HCR.EL2.E2H RES0 when ID_AA64MMFR1_EL1.VH is zero
KVM: arm64: Mark HCR.EL2.{NV*,AT} RES0 when ID_AA64MMFR4_EL1.NV_frac
is 0
KVM: arm64: Advertise NV2 in the boot messages
KVM: arm64: Consolidate idreg callbacks
KVM: arm64: Make ID_REG_LIMIT_FIELD_ENUM() more widely available
KVM: arm64: Enforce NV limits on a per-idregs basis
KVM: arm64: Move NV-specific capping to idreg sanitisation
KVM: arm64: Allow userspace to limit NV support to nVHE
KVM: arm64: Make ID_AA64MMFR4_EL1.NV_frac writable
KVM: arm64: Advertise FEAT_ECV when possible
KVM: arm64: Allow userspace to request KVM_ARM_VCPU_EL2*
KVM: arm64: Document NV caps and vcpu flags

 Documentation/virt/kvm/api.rst      |  14 +-
 arch/arm64/include/asm/kvm_host.h   |   2 +-
 arch/arm64/include/asm/kvm_nested.h |   1 +
 arch/arm64/include/uapi/asm/kvm.h   |   1 +
 arch/arm64/kernel/cpufeature.c      |  15 +-
 arch/arm64/kvm/arm.c                |  11 +-
 arch/arm64/kvm/nested.c             | 285 +++++++++++++++-------------
 arch/arm64/kvm/sys_regs.c           |  44 ++---
 arch/arm64/kvm/sys_regs.h           |  10 +
 include/uapi/linux/kvm.h            |   2 +
 10 files changed, 217 insertions(+), 168 deletions(-)

Signature · 1 line

2.39.2

Marc Zyngier

author maintainer

2025-02-20 13:49 UTC #1 lore raw

PATCH v2 1/14 arm64: cpufeature: Handle NV_frac as a synonym of NV2

With ARMv9.5, an implementation supporting Nested Virtualization is allowed to only support NV2, and to avoid supporting the old (and useless) ARMv8.3 variant.

This is indicated by ID_AA64MMFR2_EL1.NV being 0 (as if NV wasn't implemented) and ID_AA64MMDR4_EL1.NV_frac being 1 (indicating that NV2 is actually supported).

Given that KVM only deals with NV2 and refuses to use the old NV, detecting NV2 or NV_frac is what we need to enable it.

Signed-off-by Marc Zyngier <[email protected]>

---
 arch/arm64/kernel/cpufeature.c | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

arch/arm64/kernel/cpufeature.c +13 -2

index d561cf3b8ac7b..2c198cd4f9405 100644
@@ -497,6 +497,7 @@ static const struct arm64_ftr_bits ftr_id_aa64mmfr3[] = {
 
 static const struct arm64_ftr_bits ftr_id_aa64mmfr4[] = {
 	S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR4_EL1_E2H0_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR4_EL1_NV_frac_SHIFT, 4, 0),
 	ARM64_FTR_END,
 };

@@ -2162,7 +2163,7 @@ static bool has_nested_virt_support(const struct arm64_cpu_capabilities *cap,
 	if (kvm_get_mode() != KVM_MODE_NV)
 		return false;
 
-	if (!has_cpuid_feature(cap, scope)) {
+	if (!cpucap_multi_entry_cap_matches(cap, scope)) {
 		pr_warn("unavailable: %s\n", cap->desc);
 		return false;
 	}

@@ -2519,7 +2520,17 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
 		.capability = ARM64_HAS_NESTED_VIRT,
 		.type = ARM64_CPUCAP_SYSTEM_FEATURE,
 		.matches = has_nested_virt_support,
-		ARM64_CPUID_FIELDS(ID_AA64MMFR2_EL1, NV, NV2)
+		.match_list = (const struct arm64_cpu_capabilities []){
+			{
+				.matches = has_cpuid_feature,
+				ARM64_CPUID_FIELDS(ID_AA64MMFR2_EL1, NV, NV2)
+			},
+			{
+				.matches = has_cpuid_feature,
+				ARM64_CPUID_FIELDS(ID_AA64MMFR4_EL1, NV_frac, NV2_ONLY)
+			},
+			{ /* Sentinel */ }
+		},
 	},
 	{
 		.capability = ARM64_HAS_32BIT_EL0_DO_NOT_USE,

Signature · 1 line

2.39.2

Joey Gouly

reviewer

2025-02-20 14:03 UTC #2 lore raw

Re: [PATCH v2 01/14] arm64: cpufeature: Handle NV_frac as a synonym of NV2

Quoted 6 lines · Marc Zyngier

With ARMv9.5, an implementation supporting Nested Virtualization is allowed to only support NV2, and to avoid supporting the old (and useless) ARMv8.3 variant.
This is indicated by ID_AA64MMFR2_EL1.NV being 0 (as if NV wasn't implemented) and ID_AA64MMDR4_EL1.NV_frac being 1 (indicating that

typo: ID_AA64MMDR4_EL1 -> ID_AA64MMFR4_EL1

Quoted 53 lines

NV2 is actually supported).

Given that KVM only deals with NV2 and refuses to use the old NV, detecting NV2 or NV_frac is what we need to enable it.

Signed-off-by Marc Zyngier <[email protected]>

---
 arch/arm64/kernel/cpufeature.c | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

arch/arm64/kernel/cpufeature.c +13 -2

index d561cf3b8ac7b..2c198cd4f9405 100644
@@ -497,6 +497,7 @@ static const struct arm64_ftr_bits ftr_id_aa64mmfr3[] = {
 
 static const struct arm64_ftr_bits ftr_id_aa64mmfr4[] = {
 	S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR4_EL1_E2H0_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR4_EL1_NV_frac_SHIFT, 4, 0),
 	ARM64_FTR_END,
 };

@@ -2162,7 +2163,7 @@ static bool has_nested_virt_support(const struct arm64_cpu_capabilities *cap,
 	if (kvm_get_mode() != KVM_MODE_NV)
 		return false;
 
-	if (!has_cpuid_feature(cap, scope)) {
+	if (!cpucap_multi_entry_cap_matches(cap, scope)) {
 		pr_warn("unavailable: %s\n", cap->desc);
 		return false;
 	}

@@ -2519,7 +2520,17 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
 		.capability = ARM64_HAS_NESTED_VIRT,
 		.type = ARM64_CPUCAP_SYSTEM_FEATURE,
 		.matches = has_nested_virt_support,
-		ARM64_CPUID_FIELDS(ID_AA64MMFR2_EL1, NV, NV2)
+		.match_list = (const struct arm64_cpu_capabilities []){
+			{
+				.matches = has_cpuid_feature,
+				ARM64_CPUID_FIELDS(ID_AA64MMFR2_EL1, NV, NV2)
+			},
+			{
+				.matches = has_cpuid_feature,
+				ARM64_CPUID_FIELDS(ID_AA64MMFR4_EL1, NV_frac, NV2_ONLY)
+			},
+			{ /* Sentinel */ }
+		},
 	},
 	{
 		.capability = ARM64_HAS_32BIT_EL0_DO_NOT_USE,

Signature · 1 line

2.39.2

Marc Zyngier

author maintainer

2025-02-20 13:49 UTC #3 lore raw

PATCH v2 2/14 KVM: arm64: Hide ID_AA64MMFR2_EL1.NV from guest and userspace

In reply to Marc Zyngier

Since our take on FEAT_NV is to only support FEAT_NV2, we should never expose ID_AA64MMFR2_EL1.NV to a guest nor userspace.

Make sure we mask this field for good.

Signed-off-by Marc Zyngier <[email protected]>

---
 arch/arm64/kvm/sys_regs.c | 1 +
 1 file changed, 1 insertion(+)

arch/arm64/kvm/sys_regs.c +1 -0

index 82430c1e1dd02..9f10dbd26e348 100644
@@ -1627,6 +1627,7 @@ static u64 __kvm_read_sanitised_id_reg(const struct kvm_vcpu *vcpu,
 		break;
 	case SYS_ID_AA64MMFR2_EL1:
 		val &= ~ID_AA64MMFR2_EL1_CCIDX_MASK;
+		val &= ~ID_AA64MMFR2_EL1_NV;
 		break;
 	case SYS_ID_AA64MMFR3_EL1:
 		val &= ID_AA64MMFR3_EL1_TCRX | ID_AA64MMFR3_EL1_S1POE |

Signature · 1 line

2.39.2

Sebastian Ott

2025-02-20 17:36 UTC #4 lore raw

Re: [PATCH v2 02/14] KVM: arm64: Hide ID_AA64MMFR2_EL1.NV from guest and userspace

Hi Marc,

Quoted 15 lines · Marc Zyngier

Since our take on FEAT_NV is to only support FEAT_NV2, we should never expose ID_AA64MMFR2_EL1.NV to a guest nor userspace.
Make sure we mask this field for good.
Signed-off-by Marc Zyngier <[email protected]>
---
arch/arm64/kvm/sys_regs.c | 1 +
1 file changed, 1 insertion(+)
arch/arm64/kvm/sys_regs.c +0 -0
index 82430c1e1dd02..9f10dbd26e348 100644

arch/arm64/kvm/sys_regs.c static u64 __kvm_read_sanitised_id_reg(const struct kvm_vcpu *vcpu, · line 1631

Quoted 5 lines

arch/arm64/kvm/sys_regs.c +1 -0

@@ -1627,6 +1627,7 @@ static u64 __kvm_read_sanitised_id_reg(const struct kvm_vcpu *vcpu,
 		break;
 	case SYS_ID_AA64MMFR2_EL1:
 		val &= ~ID_AA64MMFR2_EL1_CCIDX_MASK;
+		val &= ~ID_AA64MMFR2_EL1_NV;
 		break;

This would cause issues when you update the host kernel while keeping the guests register state. Could we allow to write (but ignore) the previously valid value? Like it was handled in:
6685f5d572c2 KVM: arm64: Disable MPAM visibility by default and ignore VMM writes

Sebastian

Marc Zyngier

author maintainer

2025-02-20 19:46 UTC #5 lore raw

Re: [PATCH v2 02/14] KVM: arm64: Hide ID_AA64MMFR2_EL1.NV from guest and userspace

Quoted 14 lines · Sebastian Ott

Hi Marc,
Quoted 10 lines · Marc Zyngier
Since our take on FEAT_NV is to only support FEAT_NV2, we should never expose ID_AA64MMFR2_EL1.NV to a guest nor userspace.
Make sure we mask this field for good.
Signed-off-by Marc Zyngier <[email protected]>
---
arch/arm64/kvm/sys_regs.c | 1 +
1 file changed, 1 insertion(+)

Quoted 15 lines

arch/arm64/kvm/sys_regs.c static u64 __kvm_read_sanitised_id_reg(const struct kvm_vcpu *vcpu, · line 1631
Quoted 10 lines
arch/arm64/kvm/sys_regs.c +1 -0
index 82430c1e1dd02..9f10dbd26e348 100644
@@ -1627,6 +1627,7 @@ static u64 __kvm_read_sanitised_id_reg(const struct kvm_vcpu *vcpu,
 		break;
 	case SYS_ID_AA64MMFR2_EL1:
 		val &= ~ID_AA64MMFR2_EL1_CCIDX_MASK;
+		val &= ~ID_AA64MMFR2_EL1_NV;
 		break;
This would cause issues when you update the host kernel while keeping the guests register state. Could we allow to write (but ignore) the previously valid value? Like it was handled in:
6685f5d572c2 KVM: arm64: Disable MPAM visibility by default and ignore VMM writes

Yeah, this falls into the same "shouldn't have exposed this the first place" bucket. Annoying. Something like the diff below?

M.

arch/arm64/kvm/sys_regs.c +18 -1

index 2e14562b5841f..87269de31a642 100644
@@ -1946,6 +1946,22 @@ static int set_id_aa64pfr1_el1(struct kvm_vcpu *vcpu,
 	return set_id_reg(vcpu, rd, user_val);
 }
 
+static int set_id_aa64mmfr2_el1(struct kvm_vcpu *vcpu,
+				const struct sys_reg_desc *rd, u64 user_val)
+{
+	u64 hw_val = read_sanitised_ftr_reg(SYS_ID_AA64MMFR2_EL1);
+	u64 nv_mask = ID_AA64MMFR2_EL1_NV_MASK;
+
+	/*
+	 * We made the mistake to expose the now deprecated NV field,
+	 * so allow userspace to write it, but silently ignore it.
+	 */
+	if ((hw_val & nv_mask) == (user_val & nv_mask))
+		user_val &= ~nv_mask;
+
+	return set_id_reg(vcpu, rd, user_val);
+}
+
 static int set_ctr_el0(struct kvm_vcpu *vcpu,
 		       const struct sys_reg_desc *rd, u64 user_val)
 {

@@ -2730,7 +2746,8 @@ static const struct sys_reg_desc sys_reg_descs[] = {
 					ID_AA64MMFR1_EL1_XNX |
 					ID_AA64MMFR1_EL1_VH |
 					ID_AA64MMFR1_EL1_VMIDBits)),
-	ID_WRITABLE(ID_AA64MMFR2_EL1, ~(ID_AA64MMFR2_EL1_RES0 |
+	ID_FILTERED(ID_AA64MMFR2_EL1,
+		    id_aa64mmfr2_el1, ~(ID_AA64MMFR2_EL1_RES0 |
 					ID_AA64MMFR2_EL1_EVT |
 					ID_AA64MMFR2_EL1_FWB |
 					ID_AA64MMFR2_EL1_IDS |

Signature · 1 line

Without deviation from the norm, progress is not possible.

Sebastian Ott

2025-02-21 07:52 UTC #6 lore raw

Re: [PATCH v2 02/14] KVM: arm64: Hide ID_AA64MMFR2_EL1.NV from guest and userspace

Quoted 29 lines · Marc Zyngier

Quoted 11 lines · Sebastian Ott
Quoted 10 lines · Marc Zyngier
Since our take on FEAT_NV is to only support FEAT_NV2, we should never expose ID_AA64MMFR2_EL1.NV to a guest nor userspace.
Make sure we mask this field for good.
Signed-off-by Marc Zyngier <[email protected]>
---
arch/arm64/kvm/sys_regs.c | 1 +
1 file changed, 1 insertion(+)
Quoted 15 lines
arch/arm64/kvm/sys_regs.c static u64 __kvm_read_sanitised_id_reg(const struct kvm_vcpu *vcpu, · line 1631
Quoted 10 lines
arch/arm64/kvm/sys_regs.c +1 -0
index 82430c1e1dd02..9f10dbd26e348 100644
@@ -1627,6 +1627,7 @@ static u64 __kvm_read_sanitised_id_reg(const struct kvm_vcpu *vcpu,
 		break;
 	case SYS_ID_AA64MMFR2_EL1:
 		val &= ~ID_AA64MMFR2_EL1_CCIDX_MASK;
+		val &= ~ID_AA64MMFR2_EL1_NV;
 		break;
This would cause issues when you update the host kernel while keeping the guests register state. Could we allow to write (but ignore) the previously valid value? Like it was handled in:
6685f5d572c2 KVM: arm64: Disable MPAM visibility by default and ignore VMM writes

Quoted 2 lines

Yeah, this falls into the same "shouldn't have exposed this the first place" bucket. Annoying. Something like the diff below?

Yes, thanks!

Marc Zyngier

author maintainer

2025-02-20 13:49 UTC #7 lore raw

PATCH v2 3/14 KVM: arm64: Mark HCR.EL2.E2H RES0 when ID_AA64MMFR1_EL1.VH is zero

In reply to Marc Zyngier

Enforce HCR_EL2.E2H being RES0 when VHE is disabled, so that we can actually rely on that bit never being flipped behind our back.

Signed-off-by Marc Zyngier <[email protected]>

---
 arch/arm64/kvm/nested.c | 2 ++
 1 file changed, 2 insertions(+)

arch/arm64/kvm/nested.c +2 -0

index 0c9387d2f5070..ed3add7d32f66 100644
@@ -1034,6 +1034,8 @@ int kvm_init_nv_sysregs(struct kvm_vcpu *vcpu)
 		res0 |= (HCR_TEA | HCR_TERR);
 	if (!kvm_has_feat(kvm, ID_AA64MMFR1_EL1, LO, IMP))
 		res0 |= HCR_TLOR;
+	if (!kvm_has_feat(kvm, ID_AA64MMFR1_EL1, VH, IMP))
+		res0 |= HCR_E2H;
 	if (!kvm_has_feat(kvm, ID_AA64MMFR4_EL1, E2H0, IMP))
 		res1 |= HCR_E2H;
 	set_sysreg_masks(kvm, HCR_EL2, res0, res1);

Signature · 1 line

2.39.2

Marc Zyngier

author maintainer

2025-02-20 13:49 UTC #8 lore raw

PATCH v2 4/14 KVM: arm64: Mark HCR.EL2.{NV*,AT} RES0 when ID_AA64MMFR4_EL1.NV_frac is 0

In reply to Marc Zyngier

Enforce HCR_EL2.{NV*,AT} being RES0 when NV2 is disabled, so that we can actually rely on these bits never being flipped behind our back.

This of course relies on our earlier ID reg sanitising.

Signed-off-by Marc Zyngier <[email protected]>

---
 arch/arm64/kvm/nested.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

arch/arm64/kvm/nested.c +5 -4

index ed3add7d32f66..9f140560a6f5d 100644
@@ -1021,10 +1021,11 @@ int kvm_init_nv_sysregs(struct kvm_vcpu *vcpu)
 		res0 |= HCR_FIEN;
 	if (!kvm_has_feat(kvm, ID_AA64MMFR2_EL1, FWB, IMP))
 		res0 |= HCR_FWB;
-	if (!kvm_has_feat(kvm, ID_AA64MMFR2_EL1, NV, NV2))
-		res0 |= HCR_NV2;
-	if (!kvm_has_feat(kvm, ID_AA64MMFR2_EL1, NV, IMP))
-		res0 |= (HCR_AT | HCR_NV1 | HCR_NV);
+	/* Implementation choice: NV2 is the only supported config */
+	if (!kvm_has_feat(kvm, ID_AA64MMFR4_EL1, NV_frac, NV2_ONLY))
+		res0 |= (HCR_NV2 | HCR_NV | HCR_AT);
+	if (!kvm_has_feat(kvm, ID_AA64MMFR4_EL1, E2H0, NI))
+		res0 |= HCR_NV1;
 	if (!(kvm_vcpu_has_feature(kvm, KVM_ARM_VCPU_PTRAUTH_ADDRESS) &&
 	      kvm_vcpu_has_feature(kvm, KVM_ARM_VCPU_PTRAUTH_GENERIC)))
 		res0 |= (HCR_API | HCR_APK);

Signature · 1 line

2.39.2

Marc Zyngier

author maintainer

2025-02-20 13:49 UTC #9 lore raw

PATCH v2 5/14 KVM: arm64: Advertise NV2 in the boot messages

In reply to Marc Zyngier

Make it a bit easier to understand what people are running by adding a +NV2 string to the successful KVM initialisation.

Signed-off-by Marc Zyngier <[email protected]>

---
 arch/arm64/kvm/arm.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

arch/arm64/kvm/arm.c +3 -2

index b8e55a441282f..5ea09f13c7bc0 100644
@@ -2806,11 +2806,12 @@ static __init int kvm_arm_init(void)
 	if (err)
 		goto out_hyp;
 
-	kvm_info("%s%sVHE mode initialized successfully\n",
+	kvm_info("%s%sVHE%s mode initialized successfully\n",
 		 in_hyp_mode ? "" : (is_protected_kvm_enabled() ?
 				     "Protected " : "Hyp "),
 		 in_hyp_mode ? "" : (cpus_have_final_cap(ARM64_KVM_HVHE) ?
-				     "h" : "n"));
+				     "h" : "n"),
+		 cpus_have_final_cap(ARM64_HAS_NESTED_VIRT) ? "+NV2": "");
 
 	/*
 	 * FIXME: Do something reasonable if kvm_init() fails after pKVM

Signature · 1 line

2.39.2

Marc Zyngier

author maintainer

2025-02-20 13:49 UTC #10 lore raw

PATCH v2 6/14 KVM: arm64: Consolidate idreg callbacks

In reply to Marc Zyngier

Most of the ID_DESC() users use the same callbacks, with only a few overrides. Consolidate the common callbacks in a macro, and consistently use it everywhere.

Whilst we're at it, give ID_UNALLOCATED() a .name string, so that we can easily decode traces.

Signed-off-by Marc Zyngier <[email protected]>

---
 arch/arm64/kvm/sys_regs.c | 28 ++++++++++------------------
 1 file changed, 10 insertions(+), 18 deletions(-)

arch/arm64/kvm/sys_regs.c +10 ~~-18~~

index 9f10dbd26e348..678213dc15513 100644
@@ -2267,35 +2267,33 @@ static bool bad_redir_trap(struct kvm_vcpu *vcpu,
  * from userspace.
  */
 
+#define ID_DESC_DEFAULT_CALLBACKS		\
+	.access	= access_id_reg,		\
+	.get_user = get_id_reg,			\
+	.set_user = set_id_reg,			\
+	.visibility = id_visibility,		\
+	.reset = kvm_read_sanitised_id_reg
+
 #define ID_DESC(name)				\
 	SYS_DESC(SYS_##name),			\
-	.access	= access_id_reg,		\
-	.get_user = get_id_reg			\
+	ID_DESC_DEFAULT_CALLBACKS
 
 /* sys_reg_desc initialiser for known cpufeature ID registers */
 #define ID_SANITISED(name) {			\
 	ID_DESC(name),				\
-	.set_user = set_id_reg,			\
-	.visibility = id_visibility,		\
-	.reset = kvm_read_sanitised_id_reg,	\
 	.val = 0,				\
 }
 
 /* sys_reg_desc initialiser for known cpufeature ID registers */
 #define AA32_ID_SANITISED(name) {		\
 	ID_DESC(name),				\
-	.set_user = set_id_reg,			\
 	.visibility = aa32_id_visibility,	\
-	.reset = kvm_read_sanitised_id_reg,	\
 	.val = 0,				\
 }
 
 /* sys_reg_desc initialiser for writable ID registers */
 #define ID_WRITABLE(name, mask) {		\
 	ID_DESC(name),				\
-	.set_user = set_id_reg,			\
-	.visibility = id_visibility,		\
-	.reset = kvm_read_sanitised_id_reg,	\
 	.val = mask,				\
 }

@@ -2303,8 +2301,6 @@ static bool bad_redir_trap(struct kvm_vcpu *vcpu,
 #define ID_FILTERED(sysreg, name, mask) {	\
 	ID_DESC(sysreg),				\
 	.set_user = set_##name,				\
-	.visibility = id_visibility,			\
-	.reset = kvm_read_sanitised_id_reg,		\
 	.val = (mask),					\
 }

@@ -2314,12 +2310,10 @@ static bool bad_redir_trap(struct kvm_vcpu *vcpu,
  * (1 <= crm < 8, 0 <= Op2 < 8).
  */
 #define ID_UNALLOCATED(crm, op2) {			\
+	.name = "S3_0_0_" #crm "_" #op2,		\
 	Op0(3), Op1(0), CRn(0), CRm(crm), Op2(op2),	\
-	.access = access_id_reg,			\
-	.get_user = get_id_reg,				\
-	.set_user = set_id_reg,				\
+	ID_DESC_DEFAULT_CALLBACKS,			\
 	.visibility = raz_visibility,			\
-	.reset = kvm_read_sanitised_id_reg,		\
 	.val = 0,					\
 }

@@ -2330,9 +2324,7 @@ static bool bad_redir_trap(struct kvm_vcpu *vcpu,
  */
 #define ID_HIDDEN(name) {			\
 	ID_DESC(name),				\
-	.set_user = set_id_reg,			\
 	.visibility = raz_visibility,		\
-	.reset = kvm_read_sanitised_id_reg,	\
 	.val = 0,				\
 }

Signature · 1 line

2.39.2

Ganapatrao Kulkarni

Reviewed-by on v2 6/14

2025-02-20 15:44 UTC #11 lore raw

Re: [PATCH v2 06/14] KVM: arm64: Consolidate idreg callbacks

Quoted 85 lines · Marc Zyngier

Most of the ID_DESC() users use the same callbacks, with only a few overrides. Consolidate the common callbacks in a macro, and consistently use it everywhere.

Whilst we're at it, give ID_UNALLOCATED() a .name string, so that we can easily decode traces.

Signed-off-by Marc Zyngier <[email protected]>

---
  arch/arm64/kvm/sys_regs.c | 28 ++++++++++------------------
  1 file changed, 10 insertions(+), 18 deletions(-)

arch/arm64/kvm/sys_regs.c +10 ~~-16~~

index 9f10dbd26e348..678213dc15513 100644
@@ -2267,35 +2267,33 @@ static bool bad_redir_trap(struct kvm_vcpu *vcpu,
   * from userspace.
   */
 
+#define ID_DESC_DEFAULT_CALLBACKS		\
+	.access	= access_id_reg,		\
+	.get_user = get_id_reg,			\
+	.set_user = set_id_reg,			\
+	.visibility = id_visibility,		\
+	.reset = kvm_read_sanitised_id_reg
+
  #define ID_DESC(name)				\
  	SYS_DESC(SYS_##name),			\
-	.access	= access_id_reg,		\
-	.get_user = get_id_reg			\
+	ID_DESC_DEFAULT_CALLBACKS
 
  /* sys_reg_desc initialiser for known cpufeature ID registers */
  #define ID_SANITISED(name) {			\
  	ID_DESC(name),				\
-	.set_user = set_id_reg,			\
-	.visibility = id_visibility,		\
-	.reset = kvm_read_sanitised_id_reg,	\
  	.val = 0,				\
  }
 
  /* sys_reg_desc initialiser for known cpufeature ID registers */
  #define AA32_ID_SANITISED(name) {		\
  	ID_DESC(name),				\
-	.set_user = set_id_reg,			\
  	.visibility = aa32_id_visibility,	\
-	.reset = kvm_read_sanitised_id_reg,	\
  	.val = 0,				\
  }
 
  /* sys_reg_desc initialiser for writable ID registers */
  #define ID_WRITABLE(name, mask) {		\
  	ID_DESC(name),				\
-	.set_user = set_id_reg,			\
-	.visibility = id_visibility,		\
-	.reset = kvm_read_sanitised_id_reg,	\
  	.val = mask,				\
  }

@@ -2303,8 +2301,6 @@ static bool bad_redir_trap(struct kvm_vcpu *vcpu,
  #define ID_FILTERED(sysreg, name, mask) {	\
  	ID_DESC(sysreg),				\
  	.set_user = set_##name,				\
-	.visibility = id_visibility,			\
-	.reset = kvm_read_sanitised_id_reg,		\
  	.val = (mask),					\
  }

@@ -2314,12 +2310,10 @@ static bool bad_redir_trap(struct kvm_vcpu *vcpu,
   * (1 <= crm < 8, 0 <= Op2 < 8).
   */
  #define ID_UNALLOCATED(crm, op2) {			\
+	.name = "S3_0_0_" #crm "_" #op2,		\
  	Op0(3), Op1(0), CRn(0), CRm(crm), Op2(op2),	\
-	.access = access_id_reg,			\
-	.get_user = get_id_reg,				\
-	.set_user = set_id_reg,				\
+	ID_DESC_DEFAULT_CALLBACKS,			\
  	.visibility = raz_visibility,			\
-	.reset = kvm_read_sanitised_id_reg,		\
  	.val = 0,					\
  }

arch/arm64/kvm/sys_regs.c static bool bad_redir_trap(struct kvm_vcpu *vcpu, · line 2330

Quoted 9 lines

arch/arm64/kvm/sys_regs.c +0 -2

@@ -2330,9 +2324,7 @@ static bool bad_redir_trap(struct kvm_vcpu *vcpu,
   */
  #define ID_HIDDEN(name) {			\
  	ID_DESC(name),				\
-	.set_user = set_id_reg,			\
  	.visibility = raz_visibility,		\
-	.reset = kvm_read_sanitised_id_reg,	\
  	.val = 0,				\
  }

Reviewed-by Ganapatrao Kulkarni <[email protected]>

Signature · 2 lines

Thanks,
Ganapat/GK

Marc Zyngier

author maintainer

2025-02-20 13:49 UTC #12 lore raw

PATCH v2 7/14 KVM: arm64: Make ID_REG_LIMIT_FIELD_ENUM() more widely available

In reply to Marc Zyngier

ID_REG_LIMIT_FIELD_ENUM() is a useful macro to limit the idreg features exposed to guest and userspace, and the NV code can make use of it.

Signed-off-by Marc Zyngier <[email protected]>

---
 arch/arm64/kvm/sys_regs.c | 10 ----------
 arch/arm64/kvm/sys_regs.h | 10 ++++++++++
 2 files changed, 10 insertions(+), 10 deletions(-)

arch/arm64/kvm/sys_regs.c +0 ~~-10~~

index 678213dc15513..d14daadb8a7c0 100644
@@ -1803,16 +1803,6 @@ static u64 sanitise_id_aa64pfr0_el1(const struct kvm_vcpu *vcpu, u64 val)
 	return val;
 }
 
-#define ID_REG_LIMIT_FIELD_ENUM(val, reg, field, limit)			       \
-({									       \
-	u64 __f_val = FIELD_GET(reg##_##field##_MASK, val);		       \
-	(val) &= ~reg##_##field##_MASK;					       \
-	(val) |= FIELD_PREP(reg##_##field##_MASK,			       \
-			    min(__f_val,				       \
-				(u64)SYS_FIELD_VALUE(reg, field, limit)));     \
-	(val);								       \
-})
-
 static u64 sanitise_id_aa64dfr0_el1(const struct kvm_vcpu *vcpu, u64 val)
 {
 	val = ID_REG_LIMIT_FIELD_ENUM(val, ID_AA64DFR0_EL1, DebugVer, V8P8);

arch/arm64/kvm/sys_regs.h +10 -0

index 1d94ed6efad2c..cc6338d387663 100644
@@ -247,4 +247,14 @@ int kvm_finalize_sys_regs(struct kvm_vcpu *vcpu);
 	CRn(sys_reg_CRn(reg)), CRm(sys_reg_CRm(reg)),	\
 	Op2(sys_reg_Op2(reg))
 
+#define ID_REG_LIMIT_FIELD_ENUM(val, reg, field, limit)			       \
+({									       \
+	u64 __f_val = FIELD_GET(reg##_##field##_MASK, val);		       \
+	(val) &= ~reg##_##field##_MASK;					       \
+	(val) |= FIELD_PREP(reg##_##field##_MASK,			       \
+			    min(__f_val,				       \
+				(u64)SYS_FIELD_VALUE(reg, field, limit)));     \
+	(val);								       \
+})
+
 #endif /* __ARM64_KVM_SYS_REGS_LOCAL_H__ */

Signature · 1 line

2.39.2

Marc Zyngier

author maintainer

2025-02-20 13:49 UTC #13 lore raw

PATCH v2 8/14 KVM: arm64: Enforce NV limits on a per-idregs basis

In reply to Marc Zyngier

As we are about to change the way the idreg reset values are computed, move all the NV limits into a function that initialises one register at a time.

This will be most useful in the upcoming patches. We take this opportunity to remove the NV_FTR() macro and rely on the generated names instead.

Signed-off-by Marc Zyngier <[email protected]>

---
 arch/arm64/kvm/nested.c | 239 +++++++++++++++++++++++-----------------
 1 file changed, 136 insertions(+), 103 deletions(-)

arch/arm64/kvm/nested.c +136 ~~-103~~

index 9f140560a6f5d..2cc82e69ab523 100644
@@ -16,9 +16,6 @@
 
 #include "sys_regs.h"
 
-/* Protection against the sysreg repainting madness... */
-#define NV_FTR(r, f)		ID_AA64##r##_EL1_##f
-
 /*
  * Ratio of live shadow S2 MMU per vcpu. This is a trade-off between
  * memory usage and potential number of different sets of S2 PTs in

@@ -807,133 +804,169 @@ void kvm_arch_flush_shadow_all(struct kvm *kvm)
  * This list should get updated as new features get added to the NV
  * support, and new extension to the architecture.
  */
+static u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val)
+{
+	switch (reg) {
+	case SYS_ID_AA64ISAR0_EL1:
+		/* Support everything but TME */
+		val &= ~ID_AA64ISAR0_EL1_TME;
+		break;
+
+	case SYS_ID_AA64ISAR1_EL1:
+		/* Support everything but LS64 and Spec Invalidation */
+		val &= ~(ID_AA64ISAR1_EL1_LS64	|
+			 ID_AA64ISAR1_EL1_SPECRES);
+		break;
+
+	case SYS_ID_AA64PFR0_EL1:
+		/* No RME, AMU, MPAM, S-EL2, or RAS */
+		val &= ~(ID_AA64PFR0_EL1_RME	|
+			 ID_AA64PFR0_EL1_AMU	|
+			 ID_AA64PFR0_EL1_MPAM	|
+			 ID_AA64PFR0_EL1_SEL2	|
+			 ID_AA64PFR0_EL1_RAS	|
+			 ID_AA64PFR0_EL1_EL3	|
+			 ID_AA64PFR0_EL1_EL2	|
+			 ID_AA64PFR0_EL1_EL1	|
+			 ID_AA64PFR0_EL1_EL0);
+		/* 64bit only at any EL */
+		val |= SYS_FIELD_PREP_ENUM(ID_AA64PFR0_EL1, EL0, IMP);
+		val |= SYS_FIELD_PREP_ENUM(ID_AA64PFR0_EL1, EL1, IMP);
+		val |= SYS_FIELD_PREP_ENUM(ID_AA64PFR0_EL1, EL2, IMP);
+		val |= SYS_FIELD_PREP_ENUM(ID_AA64PFR0_EL1, EL3, IMP);
+		break;
+
+	case SYS_ID_AA64PFR1_EL1:
+		/* Only support BTI, SSBS, CSV2_frac */
+		val &= (ID_AA64PFR1_EL1_BT	|
+			ID_AA64PFR1_EL1_SSBS	|
+			ID_AA64PFR1_EL1_CSV2_frac);
+		break;
+
+	case SYS_ID_AA64MMFR0_EL1:
+		/* Hide ECV, ExS, Secure Memory */
+		val &= ~(ID_AA64MMFR0_EL1_EVC		|
+			 ID_AA64MMFR0_EL1_EXS		|
+			 ID_AA64MMFR0_EL1_TGRAN4_2	|
+			 ID_AA64MMFR0_EL1_TGRAN16_2	|
+			 ID_AA64MMFR0_EL1_TGRAN64_2	|
+			 ID_AA64MMFR0_EL1_SNSMEM);
+
+		/* Disallow unsupported S2 page sizes */
+		switch (PAGE_SIZE) {
+		case SZ_64K:
+			val |= SYS_FIELD_PREP_ENUM(ID_AA64MMFR0_EL1, TGRAN16_2, NI);
+			fallthrough;
+		case SZ_16K:
+			val |= SYS_FIELD_PREP_ENUM(ID_AA64MMFR0_EL1, TGRAN4_2, NI);
+			fallthrough;
+		case SZ_4K:
+			/* Support everything */
+			break;
+		}
+
+		/*
+		 * Since we can't support a guest S2 page size smaller
+		 * than the host's own page size (due to KVM only
+		 * populating its own S2 using the kernel's page
+		 * size), advertise the limitation using FEAT_GTG.
+		 */
+		switch (PAGE_SIZE) {
+		case SZ_4K:
+			val |= SYS_FIELD_PREP_ENUM(ID_AA64MMFR0_EL1, TGRAN4_2, IMP);
+			fallthrough;
+		case SZ_16K:
+			val |= SYS_FIELD_PREP_ENUM(ID_AA64MMFR0_EL1, TGRAN16_2, IMP);
+			fallthrough;
+		case SZ_64K:
+			val |= SYS_FIELD_PREP_ENUM(ID_AA64MMFR0_EL1, TGRAN64_2, IMP);
+			break;
+		}
+
+		/* Cap PARange to 48bits */
+		val = ID_REG_LIMIT_FIELD_ENUM(val, ID_AA64MMFR0_EL1, PARANGE, 48);
+		break;
+
+	case SYS_ID_AA64MMFR1_EL1:
+		val &= (ID_AA64MMFR1_EL1_HCX	|
+			ID_AA64MMFR1_EL1_PAN	|
+			ID_AA64MMFR1_EL1_LO	|
+			ID_AA64MMFR1_EL1_HPDS	|
+			ID_AA64MMFR1_EL1_VH	|
+			ID_AA64MMFR1_EL1_VMIDBits);
+		break;
+
+	case SYS_ID_AA64MMFR2_EL1:
+		val &= ~(ID_AA64MMFR2_EL1_BBM	|
+			 ID_AA64MMFR2_EL1_TTL	|
+			 GENMASK_ULL(47, 44)	|
+			 ID_AA64MMFR2_EL1_ST	|
+			 ID_AA64MMFR2_EL1_CCIDX	|
+			 ID_AA64MMFR2_EL1_VARange);
+
+		/* Force TTL support */
+		val |= SYS_FIELD_PREP_ENUM(ID_AA64MMFR2_EL1, TTL, IMP);
+		break;
+
+	case SYS_ID_AA64MMFR4_EL1:
+		val = SYS_FIELD_PREP_ENUM(ID_AA64MMFR4_EL1, NV_frac, NV2_ONLY);
+		val |= SYS_FIELD_PREP_ENUM(ID_AA64MMFR4_EL1, E2H0, NI_NV1);
+		break;
+
+	case SYS_ID_AA64DFR0_EL1:
+		/* Only limited support for PMU, Debug, BPs, WPs, and HPMN0 */
+		val &= (ID_AA64DFR0_EL1_PMUVer	|
+			ID_AA64DFR0_EL1_WRPs	|
+			ID_AA64DFR0_EL1_BRPs	|
+			ID_AA64DFR0_EL1_DebugVer|
+			ID_AA64DFR0_EL1_HPMN0);
+
+		/* Cap Debug to ARMv8.1 */
+		val = ID_REG_LIMIT_FIELD_ENUM(val, ID_AA64DFR0_EL1, DebugVer, VHE);
+		break;
+	}
+
+	return val;
+}
+
 static void limit_nv_id_regs(struct kvm *kvm)
 {
-	u64 val, tmp;
+	u64 val;
 
-	/* Support everything but TME */
 	val = kvm_read_vm_id_reg(kvm, SYS_ID_AA64ISAR0_EL1);
-	val &= ~NV_FTR(ISAR0, TME);
+	val = limit_nv_id_reg(kvm, SYS_ID_AA64ISAR0_EL1, val);
 	kvm_set_vm_id_reg(kvm, SYS_ID_AA64ISAR0_EL1, val);
 
-	/* Support everything but Spec Invalidation and LS64 */
 	val = kvm_read_vm_id_reg(kvm, SYS_ID_AA64ISAR1_EL1);
-	val &= ~(NV_FTR(ISAR1, LS64)	|
-		 NV_FTR(ISAR1, SPECRES));
+	val = limit_nv_id_reg(kvm, SYS_ID_AA64ISAR1_EL1, val);
 	kvm_set_vm_id_reg(kvm, SYS_ID_AA64ISAR1_EL1, val);
 
-	/* No AMU, MPAM, S-EL2, or RAS */
 	val = kvm_read_vm_id_reg(kvm, SYS_ID_AA64PFR0_EL1);
-	val &= ~(GENMASK_ULL(55, 52)	|
-		 NV_FTR(PFR0, AMU)	|
-		 NV_FTR(PFR0, MPAM)	|
-		 NV_FTR(PFR0, SEL2)	|
-		 NV_FTR(PFR0, RAS)	|
-		 NV_FTR(PFR0, EL3)	|
-		 NV_FTR(PFR0, EL2)	|
-		 NV_FTR(PFR0, EL1)	|
-		 NV_FTR(PFR0, EL0));
-	/* 64bit only at any EL */
-	val |= FIELD_PREP(NV_FTR(PFR0, EL0), 0b0001);
-	val |= FIELD_PREP(NV_FTR(PFR0, EL1), 0b0001);
-	val |= FIELD_PREP(NV_FTR(PFR0, EL2), 0b0001);
-	val |= FIELD_PREP(NV_FTR(PFR0, EL3), 0b0001);
+	val = limit_nv_id_reg(kvm, SYS_ID_AA64PFR0_EL1, val);
 	kvm_set_vm_id_reg(kvm, SYS_ID_AA64PFR0_EL1, val);
 
-	/* Only support BTI, SSBS, CSV2_frac */
 	val = kvm_read_vm_id_reg(kvm, SYS_ID_AA64PFR1_EL1);
-	val &= (NV_FTR(PFR1, BT)	|
-		NV_FTR(PFR1, SSBS)	|
-		NV_FTR(PFR1, CSV2_frac));
+	val = limit_nv_id_reg(kvm, SYS_ID_AA64PFR1_EL1, val);
 	kvm_set_vm_id_reg(kvm, SYS_ID_AA64PFR1_EL1, val);
 
-	/* Hide ECV, ExS, Secure Memory */
 	val = kvm_read_vm_id_reg(kvm, SYS_ID_AA64MMFR0_EL1);
-	val &= ~(NV_FTR(MMFR0, ECV)		|
-		 NV_FTR(MMFR0, EXS)		|
-		 NV_FTR(MMFR0, TGRAN4_2)	|
-		 NV_FTR(MMFR0, TGRAN16_2)	|
-		 NV_FTR(MMFR0, TGRAN64_2)	|
-		 NV_FTR(MMFR0, SNSMEM));
-
-	/* Disallow unsupported S2 page sizes */
-	switch (PAGE_SIZE) {
-	case SZ_64K:
-		val |= FIELD_PREP(NV_FTR(MMFR0, TGRAN16_2), 0b0001);
-		fallthrough;
-	case SZ_16K:
-		val |= FIELD_PREP(NV_FTR(MMFR0, TGRAN4_2), 0b0001);
-		fallthrough;
-	case SZ_4K:
-		/* Support everything */
-		break;
-	}
-	/*
-	 * Since we can't support a guest S2 page size smaller than
-	 * the host's own page size (due to KVM only populating its
-	 * own S2 using the kernel's page size), advertise the
-	 * limitation using FEAT_GTG.
-	 */
-	switch (PAGE_SIZE) {
-	case SZ_4K:
-		val |= FIELD_PREP(NV_FTR(MMFR0, TGRAN4_2), 0b0010);
-		fallthrough;
-	case SZ_16K:
-		val |= FIELD_PREP(NV_FTR(MMFR0, TGRAN16_2), 0b0010);
-		fallthrough;
-	case SZ_64K:
-		val |= FIELD_PREP(NV_FTR(MMFR0, TGRAN64_2), 0b0010);
-		break;
-	}
-	/* Cap PARange to 48bits */
-	tmp = FIELD_GET(NV_FTR(MMFR0, PARANGE), val);
-	if (tmp > 0b0101) {
-		val &= ~NV_FTR(MMFR0, PARANGE);
-		val |= FIELD_PREP(NV_FTR(MMFR0, PARANGE), 0b0101);
-	}
+	val = limit_nv_id_reg(kvm, SYS_ID_AA64MMFR0_EL1, val);
 	kvm_set_vm_id_reg(kvm, SYS_ID_AA64MMFR0_EL1, val);
 
 	val = kvm_read_vm_id_reg(kvm, SYS_ID_AA64MMFR1_EL1);
-	val &= (NV_FTR(MMFR1, HCX)	|
-		NV_FTR(MMFR1, PAN)	|
-		NV_FTR(MMFR1, LO)	|
-		NV_FTR(MMFR1, HPDS)	|
-		NV_FTR(MMFR1, VH)	|
-		NV_FTR(MMFR1, VMIDBits));
+	val = limit_nv_id_reg(kvm, SYS_ID_AA64MMFR1_EL1, val);
 	kvm_set_vm_id_reg(kvm, SYS_ID_AA64MMFR1_EL1, val);
 
 	val = kvm_read_vm_id_reg(kvm, SYS_ID_AA64MMFR2_EL1);
-	val &= ~(NV_FTR(MMFR2, BBM)	|
-		 NV_FTR(MMFR2, TTL)	|
-		 GENMASK_ULL(47, 44)	|
-		 NV_FTR(MMFR2, ST)	|
-		 NV_FTR(MMFR2, CCIDX)	|
-		 NV_FTR(MMFR2, VARange));
-
-	/* Force TTL support */
-	val |= FIELD_PREP(NV_FTR(MMFR2, TTL), 0b0001);
+	val = limit_nv_id_reg(kvm, SYS_ID_AA64MMFR2_EL1, val);
 	kvm_set_vm_id_reg(kvm, SYS_ID_AA64MMFR2_EL1, val);
 
-	val = 0;
-	if (!cpus_have_final_cap(ARM64_HAS_HCR_NV1))
-		val |= FIELD_PREP(NV_FTR(MMFR4, E2H0),
-				  ID_AA64MMFR4_EL1_E2H0_NI_NV1);
+	val = kvm_read_vm_id_reg(kvm, SYS_ID_AA64MMFR4_EL1);
+	val = limit_nv_id_reg(kvm, SYS_ID_AA64MMFR4_EL1, val);
 	kvm_set_vm_id_reg(kvm, SYS_ID_AA64MMFR4_EL1, val);
 
-	/* Only limited support for PMU, Debug, BPs, WPs, and HPMN0 */
 	val = kvm_read_vm_id_reg(kvm, SYS_ID_AA64DFR0_EL1);
-	val &= (NV_FTR(DFR0, PMUVer)	|
-		NV_FTR(DFR0, WRPs)	|
-		NV_FTR(DFR0, BRPs)	|
-		NV_FTR(DFR0, DebugVer)	|
-		NV_FTR(DFR0, HPMN0));
-
-	/* Cap Debug to ARMv8.1 */
-	tmp = FIELD_GET(NV_FTR(DFR0, DebugVer), val);
-	if (tmp > 0b0111) {
-		val &= ~NV_FTR(DFR0, DebugVer);
-		val |= FIELD_PREP(NV_FTR(DFR0, DebugVer), 0b0111);
-	}
+	val = limit_nv_id_reg(kvm, SYS_ID_AA64DFR0_EL1, val);
 	kvm_set_vm_id_reg(kvm, SYS_ID_AA64DFR0_EL1, val);
 }

Signature · 1 line

2.39.2

Marc Zyngier

author maintainer

2025-02-20 13:49 UTC #14 lore raw

PATCH v2 9/14 KVM: arm64: Move NV-specific capping to idreg sanitisation

In reply to Marc Zyngier

Instead of applying the NV idreg limits at run time, switch to doing it at the same time as the reset of the VM initialisation.

This will make things much simpler once we introduce vcpu-driven variants of NV.

Signed-off-by Marc Zyngier <[email protected]>

---
 arch/arm64/include/asm/kvm_nested.h |  1 +
 arch/arm64/kvm/nested.c             | 45 +----------------------------
 arch/arm64/kvm/sys_regs.c           |  3 ++
 3 files changed, 5 insertions(+), 44 deletions(-)

arch/arm64/include/asm/kvm_nested.h +1 -0

index 56c4bcd35e2e5..692f403c1896e 100644
@@ -188,6 +188,7 @@ static inline bool kvm_supported_tlbi_s1e2_op(struct kvm_vcpu *vpcu, u32 instr)
 }
 
 int kvm_init_nv_sysregs(struct kvm_vcpu *vcpu);
+u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val);
 
 #ifdef CONFIG_ARM64_PTR_AUTH
 bool kvm_auth_eretax(struct kvm_vcpu *vcpu, u64 *elr);

arch/arm64/kvm/nested.c +1 ~~-44~~

63 lines not on this page · raw message

arch/arm64/kvm/sys_regs.c +3 -0

index d14daadb8a7c0..87f165289fb9f 100644
@@ -1638,6 +1638,9 @@ static u64 __kvm_read_sanitised_id_reg(const struct kvm_vcpu *vcpu,
 		break;
 	}
 
+	if (vcpu_has_nv(vcpu))
+		val = limit_nv_id_reg(vcpu->kvm, id, val);
+
 	return val;
 }

Signature · 1 line

2.39.2

Marc Zyngier

author maintainer

2025-02-20 13:49 UTC #15 lore raw

PATCH v2 10/14 KVM: arm64: Allow userspace to limit NV support to nVHE

In reply to Marc Zyngier

NV is hard. No kidding.

In order to make things simpler, we have established that NV would support two mutually exclusive configurations:

- VHE-only, and supporting recursive virtualisation

- mVHE-only, and not supporting recursive virtualisation

For that purpose, introduce a new vcpu feature flag that denotes the second configuration. We use this flag to limit the idregs further.

Signed-off-by Marc Zyngier <[email protected]>

---
 arch/arm64/include/uapi/asm/kvm.h |  1 +
 arch/arm64/kvm/nested.c           | 28 ++++++++++++++++++++++++++--
 2 files changed, 27 insertions(+), 2 deletions(-)

arch/arm64/include/uapi/asm/kvm.h +1 -0

index 568bf858f3198..3bcab2a106c98 100644
@@ -105,6 +105,7 @@ struct kvm_regs {
 #define KVM_ARM_VCPU_PTRAUTH_ADDRESS	5 /* VCPU uses address authentication */
 #define KVM_ARM_VCPU_PTRAUTH_GENERIC	6 /* VCPU uses generic authentication */
 #define KVM_ARM_VCPU_HAS_EL2		7 /* Support nested virtualization */
+#define KVM_ARM_VCPU_HAS_EL2_E2H0	8 /* Limit NV support to E2H RES0 */
 
 struct kvm_vcpu_init {
 	__u32 target;

arch/arm64/kvm/nested.c +26 -2

index 96d1d300e79f9..5ec5acb6310e9 100644
@@ -51,6 +51,10 @@ int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu)
 	struct kvm_s2_mmu *tmp;
 	int num_mmus, ret = 0;
 
+	if (test_bit(KVM_ARM_VCPU_HAS_EL2_E2H0, kvm->arch.vcpu_features) &&
+	    !cpus_have_final_cap(ARM64_HAS_HCR_NV1))
+		return -EINVAL;
+
 	/*
 	 * Let's treat memory allocation failures as benign: If we fail to
 	 * allocate anything, return an error and keep the allocated array

@@ -894,6 +898,9 @@ u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val)
 			ID_AA64MMFR1_EL1_HPDS	|
 			ID_AA64MMFR1_EL1_VH	|
 			ID_AA64MMFR1_EL1_VMIDBits);
+		/* FEAT_E2H0 implies no VHE */
+		if (test_bit(KVM_ARM_VCPU_HAS_EL2_E2H0, kvm->arch.vcpu_features))
+			val &= ~ID_AA64MMFR1_EL1_VH;
 		break;
 
 	case SYS_ID_AA64MMFR2_EL1:

@@ -909,8 +916,25 @@ u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val)
 		break;
 
 	case SYS_ID_AA64MMFR4_EL1:
-		val = SYS_FIELD_PREP_ENUM(ID_AA64MMFR4_EL1, NV_frac, NV2_ONLY);
-		val |= SYS_FIELD_PREP_ENUM(ID_AA64MMFR4_EL1, E2H0, NI_NV1);
+		/*
+		 * You get EITHER
+		 *
+		 * - FEAT_VHE without FEAT_E2H0
+		 * - FEAT_NV limited to FEAT_NV2
+		 * - HCR_EL2.NV1 being RES0
+		 *
+		 * OR
+		 *
+		 * - FEAT_E2H0 without FEAT_VHE nor FEAT_NV
+		 *
+		 * Life is too short for anything else.
+		 */
+		if (test_bit(KVM_ARM_VCPU_HAS_EL2_E2H0, kvm->arch.vcpu_features)) {
+			val = 0;
+		} else {
+			val = SYS_FIELD_PREP_ENUM(ID_AA64MMFR4_EL1, NV_frac, NV2_ONLY);
+			val |= SYS_FIELD_PREP_ENUM(ID_AA64MMFR4_EL1, E2H0, NI_NV1);
+		}
 		break;
 
 	case SYS_ID_AA64DFR0_EL1:

Signature · 1 line

2.39.2

Aneesh Kumar K.V

2025-02-24 09:44 UTC #16 lore raw

Re: [PATCH v2 10/14] KVM: arm64: Allow userspace to limit NV support to nVHE

Quoted 9 lines · Marc Zyngier

NV is hard. No kidding.
In order to make things simpler, we have established that NV would support two mutually exclusive configurations:
- VHE-only, and supporting recursive virtualisation
- mVHE-only, and not supporting recursive virtualisation

mVHE-only -> nVHE-only

-aneesh

Marc Zyngier

author maintainer

2025-02-20 13:49 UTC #17 lore raw

PATCH v2 11/14 KVM: arm64: Make ID_AA64MMFR4_EL1.NV_frac writable

In reply to Marc Zyngier

We want to make sure that it is possible for userspace to configure whether recursive NV is possible. Make NV_frac writable for that purpose.

Signed-off-by Marc Zyngier <[email protected]>

---
 arch/arm64/kvm/sys_regs.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

arch/arm64/kvm/sys_regs.c +1 -1

index 87f165289fb9f..5de542b045ac8 100644
@@ -2666,7 +2666,7 @@ static const struct sys_reg_desc sys_reg_descs[] = {
 	ID_WRITABLE(ID_AA64MMFR3_EL1, (ID_AA64MMFR3_EL1_TCRX	|
 				       ID_AA64MMFR3_EL1_S1PIE   |
 				       ID_AA64MMFR3_EL1_S1POE)),
-	ID_SANITISED(ID_AA64MMFR4_EL1),
+	ID_WRITABLE(ID_AA64MMFR4_EL1, ID_AA64MMFR4_EL1_NV_frac),
 	ID_UNALLOCATED(7,5),
 	ID_UNALLOCATED(7,6),
 	ID_UNALLOCATED(7,7),

Signature · 1 line

2.39.2

Marc Zyngier

author maintainer

2025-02-20 13:49 UTC #18 lore raw

PATCH v2 12/14 KVM: arm64: Advertise FEAT_ECV when possible

In reply to Marc Zyngier

We can advertise support for FEAT_ECV if supported on the HW as long as we limit it to the basic trap bits, and not advertise CNTPOFF_EL2 support, even if the host has it (the short story being that CNTPOFF_EL2 is not virtualisable).

Signed-off-by Marc Zyngier <[email protected]>

---
 arch/arm64/kvm/nested.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

arch/arm64/kvm/nested.c +5 -3

index 5ec5acb6310e9..d55c296fcb27a 100644
@@ -848,14 +848,16 @@ u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val)
 		break;
 
 	case SYS_ID_AA64MMFR0_EL1:
-		/* Hide ECV, ExS, Secure Memory */
-		val &= ~(ID_AA64MMFR0_EL1_EVC		|
-			 ID_AA64MMFR0_EL1_EXS		|
+		/* Hide ExS, Secure Memory */
+		val &= ~(ID_AA64MMFR0_EL1_EXS		|
 			 ID_AA64MMFR0_EL1_TGRAN4_2	|
 			 ID_AA64MMFR0_EL1_TGRAN16_2	|
 			 ID_AA64MMFR0_EL1_TGRAN64_2	|
 			 ID_AA64MMFR0_EL1_SNSMEM);
 
+		/* Hide CNTPOFF if present */
+		val = ID_REG_LIMIT_FIELD_ENUM(val, ID_AA64MMFR0_EL1, ECV, IMP);
+
 		/* Disallow unsupported S2 page sizes */
 		switch (PAGE_SIZE) {
 		case SZ_64K:

Signature · 1 line

2.39.2

Marc Zyngier

author maintainer

2025-02-20 13:49 UTC #19 lore raw

PATCH v2 13/14 KVM: arm64: Allow userspace to request KVM_ARM_VCPU_EL2*

In reply to Marc Zyngier

Since we're (almost) feature complete, let's allow userspace to request KVM_ARM_VCPU_EL2* by bumping KVM_VCPU_MAX_FEATURES up.

We also now advertise the features to userspace with new capabilities.

It's going to be great...

Reviewed-by Oliver Upton <[email protected]>

Signed-off-by Marc Zyngier <[email protected]>

---
 arch/arm64/include/asm/kvm_host.h | 2 +-
 arch/arm64/kvm/arm.c              | 6 ++++++
 include/uapi/linux/kvm.h          | 2 ++
 3 files changed, 9 insertions(+), 1 deletion(-)

arch/arm64/include/asm/kvm_host.h +1 -1

index 3a7ec98ef1238..e6b9a601258fd 100644
@@ -39,7 +39,7 @@
 
 #define KVM_MAX_VCPUS VGIC_V3_MAX_CPUS
 
-#define KVM_VCPU_MAX_FEATURES 7
+#define KVM_VCPU_MAX_FEATURES 9
 #define KVM_VCPU_VALID_FEATURES	(BIT(KVM_VCPU_MAX_FEATURES) - 1)
 
 #define KVM_REQ_SLEEP \

arch/arm64/kvm/arm.c +6 -0

index 5ea09f13c7bc0..1bc79f9e5b23a 100644
@@ -359,6 +359,12 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 	case KVM_CAP_ARM_EL1_32BIT:
 		r = cpus_have_final_cap(ARM64_HAS_32BIT_EL1);
 		break;
+	case KVM_CAP_ARM_EL2:
+		r = cpus_have_final_cap(ARM64_HAS_NESTED_VIRT);
+		break;
+	case KVM_CAP_ARM_EL2_E2H0:
+		r = cpus_have_final_cap(ARM64_HAS_HCR_NV1);
+		break;
 	case KVM_CAP_GUEST_DEBUG_HW_BPS:
 		r = get_num_brps();
 		break;

include/uapi/linux/kvm.h +2 -0

index 45e6d8fca9b99..9a6674f51b8be 100644
@@ -929,6 +929,8 @@ struct kvm_enable_cap {
 #define KVM_CAP_PRE_FAULT_MEMORY 236
 #define KVM_CAP_X86_APIC_BUS_CYCLES_NS 237
 #define KVM_CAP_X86_GUEST_MODE 238
+#define KVM_CAP_ARM_EL2 239
+#define KVM_CAP_ARM_EL2_E2H0 240
 
 struct kvm_irq_routing_irqchip {
 	__u32 irqchip;

Signature · 1 line

2.39.2

Marc Zyngier

author maintainer

2025-02-20 13:49 UTC #20 lore raw

PATCH v2 14/14 KVM: arm64: Document NV caps and vcpu flags

In reply to Marc Zyngier

Describe the two new vcpu flags that control NV, together with the capabilities that advertise them.

Reviewed-by Oliver Upton <[email protected]>

Signed-off-by Marc Zyngier <[email protected]>

---
 Documentation/virt/kvm/api.rst | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)

Documentation/virt/kvm/api.rst +13 -1

index 2b52eb77e29cb..2d7b516ae408d 100644
@@ -3456,7 +3456,8 @@ The initial values are defined as:
 	- FPSIMD/NEON registers: set to 0
 	- SVE registers: set to 0
 	- System registers: Reset to their architecturally defined
-	  values as for a warm reset to EL1 (resp. SVC)
+	  values as for a warm reset to EL1 (resp. SVC) or EL2 (in the
+	  case of EL2 being enabled).
 
 Note that because some registers reflect machine topology, all vcpus
 should be created before this ioctl is invoked.

@@ -3523,6 +3524,17 @@ Possible features:
 	      - the KVM_REG_ARM64_SVE_VLS pseudo-register is immutable, and can
 	        no longer be written using KVM_SET_ONE_REG.
 
+	- KVM_ARM_VCPU_HAS_EL2: Enable Nested Virtualisation support,
+	  booting the guest from EL2 instead of EL1.
+	  Depends on KVM_CAP_ARM_EL2.
+	  The VM is running with HCR_EL2.E2H being RES1 (VHE) unless
+	  KVM_ARM_VCPU_HAS_EL2_E2H0 is also set.
+
+	- KVM_ARM_VCPU_HAS_EL2_E2H0: Restrict Nested Virtualisation
+	  support to HCR_EL2.E2H being RES0 (non-VHE).
+	  Depends on KVM_CAP_ARM_EL2_E2H0.
+	  KVM_ARM_VCPU_HAS_EL2 must also be set.
+
 4.83 KVM_ARM_PREFERRED_TARGET
 -----------------------------

Signature · 1 line

2.39.2

Joey Gouly

reviewer Reviewed-by on the v2 series

2025-02-20 14:05 UTC #21 lore raw

In reply to Marc Zyngier

Quoted 51 lines · Marc Zyngier

Since the previous incarnation of the ABI was proved to be subtly wrong, I have reworked it to be more in line with the current way KVM operates.
No more late NV-specific adjustment nor writable ID_AA64MMFR0_EL1.VH. The NV configuration is now entirely selected from the vcpu flags. I've preserved the KVM_ARM_VCPU_EL2 flag which enables NV with VHE, and added KVM_ARM_VCPU_EL2_E2H0 which alters the NV behaviour to only allow nVHE guests without recursive NV support.
This series is actually very little new code. The bulk of it is converting the feature downgrade to be per-idreg, essentially going back to the state before 44241f34fac96 ("KVM: arm64: nv: Use accessors for modifying ID registers"), only slightly modernised. This then becomes part of the reset value computing.
The rest is simply what you'd expect in terms of being able to write the ID_AA64MMFR4_EL1.NV_frac field, making the correct bits RES0 when needed, probing for capabilities and handling the init flags.
Patches on top of -rc3, with the integration branch at the usual location.
* From v1 [1]
- Fixed mishandling of ID_UNALLOCATED(), resulting in extra consolidation and simplify the macro maze a bit
- Picked up Oliver's RBs (thanks!)
- Rebased on top of -rc3
[1] https://lore.kernel.org/r/[email protected]
Marc Zyngier (14):
arm64: cpufeature: Handle NV_frac as a synonym of NV2
KVM: arm64: Hide ID_AA64MMFR2_EL1.NV from guest and userspace
KVM: arm64: Mark HCR.EL2.E2H RES0 when ID_AA64MMFR1_EL1.VH is zero
KVM: arm64: Mark HCR.EL2.{NV*,AT} RES0 when ID_AA64MMFR4_EL1.NV_frac
is 0
KVM: arm64: Advertise NV2 in the boot messages
KVM: arm64: Consolidate idreg callbacks
KVM: arm64: Make ID_REG_LIMIT_FIELD_ENUM() more widely available
KVM: arm64: Enforce NV limits on a per-idregs basis
KVM: arm64: Move NV-specific capping to idreg sanitisation
KVM: arm64: Allow userspace to limit NV support to nVHE
KVM: arm64: Make ID_AA64MMFR4_EL1.NV_frac writable
KVM: arm64: Advertise FEAT_ECV when possible
KVM: arm64: Allow userspace to request KVM_ARM_VCPU_EL2*
KVM: arm64: Document NV caps and vcpu flags

Quoted 12 lines

 Documentation/virt/kvm/api.rst      |  14 +-
 arch/arm64/include/asm/kvm_host.h   |   2 +-
 arch/arm64/include/asm/kvm_nested.h |   1 +
 arch/arm64/include/uapi/asm/kvm.h   |   1 +
 arch/arm64/kernel/cpufeature.c      |  15 +-
 arch/arm64/kvm/arm.c                |  11 +-
 arch/arm64/kvm/nested.c             | 285 +++++++++++++++-------------
 arch/arm64/kvm/sys_regs.c           |  44 ++---
 arch/arm64/kvm/sys_regs.h           |  10 +
 include/uapi/linux/kvm.h            |   2 +
 10 files changed, 217 insertions(+), 168 deletions(-)

Reviewed-by Joey Gouly <[email protected]>

Oliver Upton

Applied

2025-02-24 19:53 UTC #22 lore raw

Re: (subset) [PATCH v2 00/14] KVM: arm64: NV userspace ABI

In reply to Marc Zyngier

Quoted 11 lines · Marc Zyngier

Since the previous incarnation of the ABI was proved to be subtly wrong, I have reworked it to be more in line with the current way KVM operates.
No more late NV-specific adjustment nor writable ID_AA64MMFR0_EL1.VH. The NV configuration is now entirely selected from the vcpu flags. I've preserved the KVM_ARM_VCPU_EL2 flag which enables NV with VHE, and added KVM_ARM_VCPU_EL2_E2H0 which alters the NV behaviour to only allow nVHE guests without recursive NV support.
[...]

I fixed up the typos, including the one that breaks bisection. Can always back this out if anything is out of place.

Applied to next, thanks!

[01/14] arm64: cpufeature: Handle NV_frac as a synonym of NV2

        https://git.kernel.org/kvmarm/kvmarm/c/88aea41b9bc5
[02/14] KVM: arm64: Hide ID_AA64MMFR2_EL1.NV from guest and userspace
        https://git.kernel.org/kvmarm/kvmarm/c/9d6745572899
[03/14] KVM: arm64: Mark HCR.EL2.E2H RES0 when ID_AA64MMFR1_EL1.VH is zero
        https://git.kernel.org/kvmarm/kvmarm/c/d9f943f76506
[04/14] KVM: arm64: Mark HCR.EL2.{NV*,AT} RES0 when ID_AA64MMFR4_EL1.NV_frac is 0
        https://git.kernel.org/kvmarm/kvmarm/c/8f8d6084f5b5
[05/14] KVM: arm64: Advertise NV2 in the boot messages
        https://git.kernel.org/kvmarm/kvmarm/c/2cd9542a375a
[06/14] KVM: arm64: Consolidate idreg callbacks
        https://git.kernel.org/kvmarm/kvmarm/c/57e7de2650c8
[07/14] KVM: arm64: Make ID_REG_LIMIT_FIELD_ENUM() more widely available
        https://git.kernel.org/kvmarm/kvmarm/c/179fd7e30f04
[08/14] KVM: arm64: Enforce NV limits on a per-idregs basis
        https://git.kernel.org/kvmarm/kvmarm/c/e7ef6ed4583e
[09/14] KVM: arm64: Move NV-specific capping to idreg sanitisation
        https://git.kernel.org/kvmarm/kvmarm/c/94f296dcd6d9
[10/14] KVM: arm64: Allow userspace to limit NV support to nVHE
        https://git.kernel.org/kvmarm/kvmarm/c/f83c41fb3ddd
[11/14] KVM: arm64: Make ID_AA64MMFR4_EL1.NV_frac writable
        https://git.kernel.org/kvmarm/kvmarm/c/642c23ea8b45
[12/14] KVM: arm64: Advertise FEAT_ECV when possible
        https://git.kernel.org/kvmarm/kvmarm/c/8b0b98ebf34d

--
Best,
Oliver