Skip to contentSkip to conversation

KVM: arm64: Enable 'writable' ID registers

PATCH Applied Oliver Upton 2 participants 22 messages Oct 3, 2023 – Oct 4, 2023

Subsystems (5)
Documentation KVM Kselftest arm64 +1 more subsystems
lore
Share & tools
lore
#0 lore raw

v11 cover KVM: arm64: Enable 'writable' ID registers

Few more fixes that I threw on top:

v10 -> v11:
- Drop the custom handling of FEAT_BC as it is now fixed on the arm64 side (Kristina)
- Bikeshed on the naming of the masks ioctl to keep things in the KVM_ namespace
- Apply more bikeshedding to the ioctl documentation, spinning off separate blocks for the 'generic' description and the Feature ID documentation
- Fix referencing in the vCPU features doc
- Fix use of uninitialized data in selftest

Jing Zhang (7):
KVM: arm64: Allow userspace to get the writable masks for feature ID
registers
KVM: arm64: Document KVM_ARM_GET_REG_WRITABLE_MASKS
KVM: arm64: Use guest ID register values for the sake of emulation
KVM: arm64: Allow userspace to change ID_AA64MMFR{0-2}_EL1
KVM: arm64: Allow userspace to change ID_AA64PFR0_EL1
KVM: arm64: selftests: Import automatic generation of sysreg defs
KVM: arm64: selftests: Test for setting ID register from usersapce

Oliver Upton (5):
KVM: arm64: Reject attempts to set invalid debug arch version
KVM: arm64: Bump up the default KVM sanitised debug version to v8p8
KVM: arm64: Allow userspace to change ID_AA64ISAR{0-2}_EL1
KVM: arm64: Allow userspace to change ID_AA64ZFR0_EL1
KVM: arm64: Document vCPU feature selection UAPIs

 Documentation/virt/kvm/api.rst                |   52 +
 Documentation/virt/kvm/arm/index.rst          |    1 +
 Documentation/virt/kvm/arm/vcpu-features.rst  |   48 +
 arch/arm64/include/asm/kvm_host.h             |    2 +
 arch/arm64/include/uapi/asm/kvm.h             |   32 +
 arch/arm64/kvm/arm.c                          |   10 +
 arch/arm64/kvm/sys_regs.c                     |  181 +-
 include/uapi/linux/kvm.h                      |    2 +
 tools/arch/arm64/include/.gitignore           |    1 +
 tools/arch/arm64/include/asm/gpr-num.h        |   26 +
 tools/arch/arm64/include/asm/sysreg.h         |  839 ++----
 tools/arch/arm64/tools/gen-sysreg.awk         |  336 +++
 tools/arch/arm64/tools/sysreg                 | 2497 +++++++++++++++++
 tools/testing/selftests/kvm/Makefile          |   15 +-
 .../selftests/kvm/aarch64/aarch32_id_regs.c   |    4 +-
 .../selftests/kvm/aarch64/debug-exceptions.c  |   12 +-
 .../selftests/kvm/aarch64/page_fault_test.c   |    6 +-
 .../selftests/kvm/aarch64/set_id_regs.c       |  479 ++++
 .../selftests/kvm/lib/aarch64/processor.c     |    6 +-
 19 files changed, 3860 insertions(+), 689 deletions(-)
 create mode 100644 Documentation/virt/kvm/arm/vcpu-features.rst
 create mode 100644 tools/arch/arm64/include/.gitignore
 create mode 100644 tools/arch/arm64/include/asm/gpr-num.h
 create mode 100755 tools/arch/arm64/tools/gen-sysreg.awk
 create mode 100644 tools/arch/arm64/tools/sysreg
 create mode 100644 tools/testing/selftests/kvm/aarch64/set_id_regs.c

base-commit: 6465e260f48790807eef06b583b38ca9789b6072

Signature · 1 line
2.42.0.609.gbb76f46606-goog
#1 lore raw

PATCH v11 1/12 KVM: arm64: Allow userspace to get the writable masks for feature ID registers

From: Jing Zhang <[email protected]>

While the Feature ID range is well defined and pretty large, it isn't inconceivable that the architecture will eventually grow some other ranges that will need to similarly be described to userspace.

Add a VM ioctl to allow userspace to get writable masks for feature ID registers in below system register space:
op0 = 3, op1 = {0, 1, 3}, CRn = 0, CRm = {0 - 7}, op2 = {0 - 7} This is used to support mix-and-match userspace and kernels for writable ID registers, where userspace may want to know upfront whether it can actually tweak the contents of an idreg or not.

Add a new capability (KVM_CAP_ARM_SUPPORTED_FEATURE_ID_RANGES) that returns a bitmap of the valid ranges, which can subsequently be retrieved, one at a time by setting the index of the set bit as the range identifier.

---
 arch/arm64/include/asm/kvm_host.h |  2 +
 arch/arm64/include/uapi/asm/kvm.h | 32 +++++++++++++++
 arch/arm64/kvm/arm.c              | 10 +++++
 arch/arm64/kvm/sys_regs.c         | 66 +++++++++++++++++++++++++++++++
 include/uapi/linux/kvm.h          |  2 +
 5 files changed, 112 insertions(+)
arch/arm64/include/asm/kvm_host.h +2 -0
index af06ccb7ee34..5e3f778f81db 100644
@@ -1078,6 +1078,8 @@ int kvm_vm_ioctl_mte_copy_tags(struct kvm *kvm,
 			       struct kvm_arm_copy_mte_tags *copy_tags);
 int kvm_vm_ioctl_set_counter_offset(struct kvm *kvm,
 				    struct kvm_arm_counter_offset *offset);
+int kvm_vm_ioctl_get_reg_writable_masks(struct kvm *kvm,
+					struct reg_mask_range *range);
 
 /* Guest/host FPSIMD coordination helpers */
 int kvm_arch_vcpu_run_map_fp(struct kvm_vcpu *vcpu);
arch/arm64/include/uapi/asm/kvm.h +32 -0
index f7ddd73a8c0f..89d2fc872d9f 100644
@@ -505,6 +505,38 @@ struct kvm_smccc_filter {
 #define KVM_HYPERCALL_EXIT_SMC		(1U << 0)
 #define KVM_HYPERCALL_EXIT_16BIT	(1U << 1)
 
+/*
+ * Get feature ID registers userspace writable mask.
+ *
+ * From DDI0487J.a, D19.2.66 ("ID_AA64MMFR2_EL1, AArch64 Memory Model
+ * Feature Register 2"):
+ *
+ * "The Feature ID space is defined as the System register space in
+ * AArch64 with op0==3, op1=={0, 1, 3}, CRn==0, CRm=={0-7},
+ * op2=={0-7}."
+ *
+ * This covers all currently known R/O registers that indicate
+ * anything useful feature wise, including the ID registers.
+ *
+ * If we ever need to introduce a new range, it will be described as
+ * such in the range field.
+ */
+#define KVM_ARM_FEATURE_ID_RANGE_IDX(op0, op1, crn, crm, op2)		\
+	({								\
+		__u64 __op1 = (op1) & 3;				\
+		__op1 -= (__op1 == 3);					\
+		(__op1 << 6 | ((crm) & 7) << 3 | (op2));		\
+	})
+
+#define KVM_ARM_FEATURE_ID_RANGE	0
+#define KVM_ARM_FEATURE_ID_RANGE_SIZE	(3 * 8 * 8)
+
+struct reg_mask_range {
+	__u64 addr;		/* Pointer to mask array */
+	__u32 range;		/* Requested range */
+	__u32 reserved[13];
+};
+
 #endif
 
 #endif /* __ARM_KVM_H__ */
arch/arm64/kvm/arm.c +10 -0
index 4866b3f7b4ea..f48430ed5b8b 100644
@@ -317,6 +317,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 	case KVM_CAP_ARM_SUPPORTED_BLOCK_SIZES:
 		r = kvm_supported_block_sizes();
 		break;
+	case KVM_CAP_ARM_SUPPORTED_REG_MASK_RANGES:
+		r = BIT(0);
+		break;
 	default:
 		r = 0;
 	}
@@ -1629,6 +1632,13 @@ int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg)
 
 		return kvm_vm_set_attr(kvm, &attr);
 	}
+	case KVM_ARM_GET_REG_WRITABLE_MASKS: {
+		struct reg_mask_range range;
+
+		if (copy_from_user(&range, argp, sizeof(range)))
+			return -EFAULT;
+		return kvm_vm_ioctl_get_reg_writable_masks(kvm, &range);
+	}
 	default:
 		return -EINVAL;
 	}
arch/arm64/kvm/sys_regs.c +66 -0

78 lines not on this page · raw message

include/uapi/linux/kvm.h +2 -0
index 13065dd96132..f49c84487200 100644
@@ -1192,6 +1192,7 @@ struct kvm_ppc_resize_hpt {
 #define KVM_CAP_COUNTER_OFFSET 227
 #define KVM_CAP_ARM_EAGER_SPLIT_CHUNK_SIZE 228
 #define KVM_CAP_ARM_SUPPORTED_BLOCK_SIZES 229
+#define KVM_CAP_ARM_SUPPORTED_REG_MASK_RANGES 230
 
 #ifdef KVM_CAP_IRQ_ROUTING
 
@@ -1562,6 +1563,7 @@ struct kvm_s390_ucas_mapping {
 #define KVM_ARM_MTE_COPY_TAGS	  _IOR(KVMIO,  0xb4, struct kvm_arm_copy_mte_tags)
 /* Available with KVM_CAP_COUNTER_OFFSET */
 #define KVM_ARM_SET_COUNTER_OFFSET _IOW(KVMIO,  0xb5, struct kvm_arm_counter_offset)
+#define KVM_ARM_GET_REG_WRITABLE_MASKS _IOR(KVMIO,  0xb6, struct reg_mask_range)
 
 /* ioctl for vm fd */
 #define KVM_CREATE_DEVICE	  _IOWR(KVMIO,  0xe0, struct kvm_create_device)
Signature · 1 line
2.42.0.609.gbb76f46606-goog
#2 lore raw

PATCH v11 2/12 KVM: arm64: Document KVM_ARM_GET_REG_WRITABLE_MASKS

In reply to Oliver Upton

From: Jing Zhang <[email protected]>

Add some basic documentation on how to get feature ID register writable masks from userspace.

---
 Documentation/virt/kvm/api.rst | 48 ++++++++++++++++++++++++++++++++++
 1 file changed, 48 insertions(+)
Documentation/virt/kvm/api.rst +48 -0
index 21a7578142a1..d55c2b68c0a9 100644
@@ -6070,6 +6070,54 @@ writes to the CNTVCT_EL0 and CNTPCT_EL0 registers using the SET_ONE_REG
 interface. No error will be returned, but the resulting offset will not be
 applied.
 
+4.139 KVM_ARM_GET_REG_WRITABLE_MASKS
+-------------------------------------------
+
+:Capability: KVM_CAP_ARM_SUPPORTED_REG_MASK_RANGES
+:Architectures: arm64
+:Type: vm ioctl
+:Parameters: struct reg_mask_range (in/out)
+:Returns: 0 on success, < 0 on error
+
+
+::
+
+        #define KVM_ARM_FEATURE_ID_RANGE	0
+        #define KVM_ARM_FEATURE_ID_RANGE_SIZE	(3 * 8 * 8)
+
+        struct reg_mask_range {
+                __u64 addr;             /* Pointer to mask array */
+                __u32 range;            /* Requested range */
+                __u32 reserved[13];
+        };
+
+This ioctl copies the writable masks for a selected range of registers to
+userspace.
+
+The ``addr`` field is a pointer to the destination array where KVM copies
+the writable masks.
+
+The ``range`` field indicates the requested range of registers.
+``KVM_CHECK_EXTENSION`` for the ``KVM_CAP_ARM_SUPPORTED_REG_MASK_RANGES``
+capability returns the supported ranges, expressed as a set of flags. Each
+flag's bit index represents a possible value for the ``range`` field.
+All other values are reserved for future use and KVM may return an error.
+
+The ``reserved[13]`` array is reserved for future use and should be 0, or
+KVM may return an error.
+
+KVM_ARM_FEATURE_ID_RANGE (0)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The Feature ID range is defined as the AArch64 System register space with
+op0==3, op1=={0, 1, 3}, CRn==0, CRm=={0-7}, op2=={0-7}.
+
+The mask returned array pointed to by ``addr`` is indexed by the macro
+``ARM64_FEATURE_ID_RANGE_IDX(op0, op1, crn, crm, op2)``, allowing userspace
+to know what fields can be changed for the system register described by
+``op0, op1, crn, crm, op2``. KVM rejects ID register values that describe a
+superset of the features supported by the system.
+
 5. The kvm_run structure
 ========================
 
Signature · 1 line
2.42.0.609.gbb76f46606-goog
#3 lore raw

PATCH v11 3/12 KVM: arm64: Use guest ID register values for the sake of emulation

In reply to Oliver Upton

From: Jing Zhang <[email protected]>

Since KVM now supports per-VM ID registers, use per-VM ID register values for the sake of emulation for DBGDIDR and LORegion.

---
 arch/arm64/kvm/sys_regs.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
arch/arm64/kvm/sys_regs.c +3 -3
index 5b0e94367f74..b73f9ff0899b 100644
@@ -379,7 +379,7 @@ static bool trap_loregion(struct kvm_vcpu *vcpu,
 			  struct sys_reg_params *p,
 			  const struct sys_reg_desc *r)
 {
-	u64 val = read_sanitised_ftr_reg(SYS_ID_AA64MMFR1_EL1);
+	u64 val = IDREG(vcpu->kvm, SYS_ID_AA64MMFR1_EL1);
 	u32 sr = reg_to_encoding(r);
 
 	if (!(val & (0xfUL << ID_AA64MMFR1_EL1_LO_SHIFT))) {
@@ -2445,8 +2445,8 @@ static bool trap_dbgdidr(struct kvm_vcpu *vcpu,
 	if (p->is_write) {
 		return ignore_write(vcpu, p);
 	} else {
-		u64 dfr = read_sanitised_ftr_reg(SYS_ID_AA64DFR0_EL1);
-		u64 pfr = read_sanitised_ftr_reg(SYS_ID_AA64PFR0_EL1);
+		u64 dfr = IDREG(vcpu->kvm, SYS_ID_AA64DFR0_EL1);
+		u64 pfr = IDREG(vcpu->kvm, SYS_ID_AA64PFR0_EL1);
 		u32 el3 = !!cpuid_feature_extract_unsigned_field(pfr, ID_AA64PFR0_EL1_EL3_SHIFT);
 
 		p->regval = ((((dfr >> ID_AA64DFR0_EL1_WRPs_SHIFT) & 0xf) << 28) |
Signature · 1 line
2.42.0.609.gbb76f46606-goog
#4 lore raw

PATCH v11 4/12 KVM: arm64: Reject attempts to set invalid debug arch version

In reply to Oliver Upton

The debug architecture is mandatory in ARMv8, so KVM should not allow userspace to configure a vCPU with less than that. Of course, this isn't handled elegantly by the generic ID register plumbing, as the respective ID register fields have a nonzero starting value.

Add an explicit check for debug versions less than v8 of the architecture.

---
 arch/arm64/kvm/sys_regs.c | 32 +++++++++++++++++++++++++++++---
 1 file changed, 29 insertions(+), 3 deletions(-)
arch/arm64/kvm/sys_regs.c +29 -3
index b73f9ff0899b..8fbfe61fe7bc 100644
@@ -1216,8 +1216,14 @@ static s64 kvm_arm64_ftr_safe_value(u32 id, const struct arm64_ftr_bits *ftrp,
 	/* Some features have different safe value type in KVM than host features */
 	switch (id) {
 	case SYS_ID_AA64DFR0_EL1:
-		if (kvm_ftr.shift == ID_AA64DFR0_EL1_PMUVer_SHIFT)
+		switch (kvm_ftr.shift) {
+		case ID_AA64DFR0_EL1_PMUVer_SHIFT:
 			kvm_ftr.type = FTR_LOWER_SAFE;
+			break;
+		case ID_AA64DFR0_EL1_DebugVer_SHIFT:
+			kvm_ftr.type = FTR_LOWER_SAFE;
+			break;
+		}
 		break;
 	case SYS_ID_DFR0_EL1:
 		if (kvm_ftr.shift == ID_DFR0_EL1_PerfMon_SHIFT)
@@ -1476,14 +1482,22 @@ static u64 read_sanitised_id_aa64pfr0_el1(struct kvm_vcpu *vcpu,
 	return val;
 }
 
+#define ID_REG_LIMIT_FIELD_ENUM(val, reg, field, limit)			       \
+({									       \
+	u64 __f_val = FIELD_GET(reg##_##field##_MASK, val);		       \
+	(val) &= ~reg##_##field##_MASK;					       \
+	(val) |= FIELD_PREP(reg##_##field##_MASK,			       \
+			min(__f_val, (u64)reg##_##field##_##limit));	       \
+	(val);								       \
+})
+
 static u64 read_sanitised_id_aa64dfr0_el1(struct kvm_vcpu *vcpu,
 					  const struct sys_reg_desc *rd)
 {
 	u64 val = read_sanitised_ftr_reg(SYS_ID_AA64DFR0_EL1);
 
 	/* Limit debug to ARMv8.0 */
-	val &= ~ID_AA64DFR0_EL1_DebugVer_MASK;
-	val |= SYS_FIELD_PREP_ENUM(ID_AA64DFR0_EL1, DebugVer, IMP);
+	val = ID_REG_LIMIT_FIELD_ENUM(val, ID_AA64DFR0_EL1, DebugVer, IMP);
 
 	/*
 	 * Only initialize the PMU version if the vCPU was configured with one.
@@ -1503,6 +1517,7 @@ static int set_id_aa64dfr0_el1(struct kvm_vcpu *vcpu,
 			       const struct sys_reg_desc *rd,
 			       u64 val)
 {
+	u8 debugver = SYS_FIELD_GET(ID_AA64DFR0_EL1, DebugVer, val);
 	u8 pmuver = SYS_FIELD_GET(ID_AA64DFR0_EL1, PMUVer, val);
 
 	/*
@@ -1522,6 +1537,13 @@ static int set_id_aa64dfr0_el1(struct kvm_vcpu *vcpu,
 	if (pmuver == ID_AA64DFR0_EL1_PMUVer_IMP_DEF)
 		val &= ~ID_AA64DFR0_EL1_PMUVer_MASK;
 
+	/*
+	 * ID_AA64DFR0_EL1.DebugVer is one of those awkward fields with a
+	 * nonzero minimum safe value.
+	 */
+	if (debugver < ID_AA64DFR0_EL1_DebugVer_IMP)
+		return -EINVAL;
+
 	return set_id_reg(vcpu, rd, val);
 }
 
@@ -1543,6 +1565,7 @@ static int set_id_dfr0_el1(struct kvm_vcpu *vcpu,
 			   u64 val)
 {
 	u8 perfmon = SYS_FIELD_GET(ID_DFR0_EL1, PerfMon, val);
+	u8 copdbg = SYS_FIELD_GET(ID_DFR0_EL1, CopDbg, val);
 
 	if (perfmon == ID_DFR0_EL1_PerfMon_IMPDEF) {
 		val &= ~ID_DFR0_EL1_PerfMon_MASK;
@@ -1558,6 +1581,9 @@ static int set_id_dfr0_el1(struct kvm_vcpu *vcpu,
 	if (perfmon != 0 && perfmon < ID_DFR0_EL1_PerfMon_PMUv3)
 		return -EINVAL;
 
+	if (copdbg < ID_DFR0_EL1_CopDbg_Armv8)
+		return -EINVAL;
+
 	return set_id_reg(vcpu, rd, val);
 }
 
Signature · 1 line
2.42.0.609.gbb76f46606-goog
#5 lore raw

PATCH v11 5/12 KVM: arm64: Bump up the default KVM sanitised debug version to v8p8

In reply to Oliver Upton

Since ID_AA64DFR0_EL1 and ID_DFR0_EL1 are now writable from userspace, it is safe to bump up the default KVM sanitised debug version to v8p8.

---
 arch/arm64/kvm/sys_regs.c | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)
arch/arm64/kvm/sys_regs.c +7 -4
index 8fbfe61fe7bc..b342c96e08f4 100644
@@ -1496,8 +1496,7 @@ static u64 read_sanitised_id_aa64dfr0_el1(struct kvm_vcpu *vcpu,
 {
 	u64 val = read_sanitised_ftr_reg(SYS_ID_AA64DFR0_EL1);
 
-	/* Limit debug to ARMv8.0 */
-	val = ID_REG_LIMIT_FIELD_ENUM(val, ID_AA64DFR0_EL1, DebugVer, IMP);
+	val = ID_REG_LIMIT_FIELD_ENUM(val, ID_AA64DFR0_EL1, DebugVer, V8P8);
 
 	/*
 	 * Only initialize the PMU version if the vCPU was configured with one.
@@ -1557,6 +1556,8 @@ static u64 read_sanitised_id_dfr0_el1(struct kvm_vcpu *vcpu,
 	if (kvm_vcpu_has_pmu(vcpu))
 		val |= SYS_FIELD_PREP(ID_DFR0_EL1, PerfMon, perfmon);
 
+	val = ID_REG_LIMIT_FIELD_ENUM(val, ID_DFR0_EL1, CopDbg, Debugv8p8);
+
 	return val;
 }
 
@@ -2013,7 +2014,8 @@ static const struct sys_reg_desc sys_reg_descs[] = {
 	  .set_user = set_id_dfr0_el1,
 	  .visibility = aa32_id_visibility,
 	  .reset = read_sanitised_id_dfr0_el1,
-	  .val = ID_DFR0_EL1_PerfMon_MASK, },
+	  .val = ID_DFR0_EL1_PerfMon_MASK |
+		 ID_DFR0_EL1_CopDbg_MASK, },
 	ID_HIDDEN(ID_AFR0_EL1),
 	AA32_ID_SANITISED(ID_MMFR0_EL1),
 	AA32_ID_SANITISED(ID_MMFR1_EL1),
@@ -2062,7 +2064,8 @@ static const struct sys_reg_desc sys_reg_descs[] = {
 	  .get_user = get_id_reg,
 	  .set_user = set_id_aa64dfr0_el1,
 	  .reset = read_sanitised_id_aa64dfr0_el1,
-	  .val = ID_AA64DFR0_EL1_PMUVer_MASK, },
+	  .val = ID_AA64DFR0_EL1_PMUVer_MASK |
+		 ID_AA64DFR0_EL1_DebugVer_MASK, },
 	ID_SANITISED(ID_AA64DFR1_EL1),
 	ID_UNALLOCATED(5,2),
 	ID_UNALLOCATED(5,3),
Signature · 1 line
2.42.0.609.gbb76f46606-goog
maintainer
#6 lore raw

Re: [PATCH v11 05/12] KVM: arm64: Bump up the default KVM sanitised debug version to v8p8

Quoted 6 lines · Oliver Upton

Since ID_AA64DFR0_EL1 and ID_DFR0_EL1 are now writable from userspace, it is safe to bump up the default KVM sanitised debug version to v8p8.

The SoB sequence looks odd. Either you're the author, and Jing's SoB shouldn't be there without a Co-DB tag, or you've lost Jing's attribution (which sometimes happens when rebasing and squashing patches together).

Quoted 19 lines
---
 arch/arm64/kvm/sys_regs.c | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)
arch/arm64/kvm/sys_regs.c +1 -2
index 8fbfe61fe7bc..b342c96e08f4 100644
@@ -1496,8 +1496,7 @@ static u64 read_sanitised_id_aa64dfr0_el1(struct kvm_vcpu *vcpu,
 {
 	u64 val = read_sanitised_ftr_reg(SYS_ID_AA64DFR0_EL1);
 
-	/* Limit debug to ARMv8.0 */
-	val = ID_REG_LIMIT_FIELD_ENUM(val, ID_AA64DFR0_EL1, DebugVer, IMP);
+	val = ID_REG_LIMIT_FIELD_ENUM(val, ID_AA64DFR0_EL1, DebugVer, V8P8);
 
 	/*
 	 * Only initialize the PMU version if the vCPU was configured with one.

arch/arm64/kvm/sys_regs.c static u64 read_sanitised_id_dfr0_el1(struct kvm_vcpu *vcpu, · line 1560

Quoted 5 lines
arch/arm64/kvm/sys_regs.c +2 -0
@@ -1557,6 +1556,8 @@ static u64 read_sanitised_id_dfr0_el1(struct kvm_vcpu *vcpu,
 	if (kvm_vcpu_has_pmu(vcpu))
 		val |= SYS_FIELD_PREP(ID_DFR0_EL1, PerfMon, perfmon);
 
+	val = ID_REG_LIMIT_FIELD_ENUM(val, ID_DFR0_EL1, CopDbg, Debugv8p8);
+

For consistency, you should also repaint DBGDIDR, which has a hardcoded '6' (ARMv8) as the supported debug version.

Thanks,

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
#7 lore raw

Re: [PATCH v11 05/12] KVM: arm64: Bump up the default KVM sanitised debug version to v8p8

Quoted 6 lines · Marc Zyngier
Quoted 4 lines · Oliver Upton

Since ID_AA64DFR0_EL1 and ID_DFR0_EL1 are now writable from userspace, it is safe to bump up the default KVM sanitised debug version to v8p8.

Quoted 7 lines
Quoted 2 lines

The SoB sequence looks odd. Either you're the author, and Jing's SoB shouldn't be there without a Co-DB tag, or you've lost Jing's attribution (which sometimes happens when rebasing and squashing patches together).

This is an artifact of me applying Jing's version of the series and hacking on top of it.

Quoted 22 lines
Quoted 22 lines
---
 arch/arm64/kvm/sys_regs.c | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)
arch/arm64/kvm/sys_regs.c +1 -2
index 8fbfe61fe7bc..b342c96e08f4 100644
@@ -1496,8 +1496,7 @@ static u64 read_sanitised_id_aa64dfr0_el1(struct kvm_vcpu *vcpu,
 {
 	u64 val = read_sanitised_ftr_reg(SYS_ID_AA64DFR0_EL1);
 
-	/* Limit debug to ARMv8.0 */
-	val = ID_REG_LIMIT_FIELD_ENUM(val, ID_AA64DFR0_EL1, DebugVer, IMP);
+	val = ID_REG_LIMIT_FIELD_ENUM(val, ID_AA64DFR0_EL1, DebugVer, V8P8);
 
 	/*
 	 * Only initialize the PMU version if the vCPU was configured with one.
@@ -1557,6 +1556,8 @@ static u64 read_sanitised_id_dfr0_el1(struct kvm_vcpu *vcpu,
 	if (kvm_vcpu_has_pmu(vcpu))
 		val |= SYS_FIELD_PREP(ID_DFR0_EL1, PerfMon, perfmon);
 
Quoted 5 lines

arch/arm64/kvm/sys_regs.c static u64 read_sanitised_id_dfr0_el1(struct kvm_vcpu *vcpu, · line 1560

Quoted 2 lines
arch/arm64/kvm/sys_regs.c +2 -0
+	val = ID_REG_LIMIT_FIELD_ENUM(val, ID_DFR0_EL1, CopDbg, Debugv8p8);
+

For consistency, you should also repaint DBGDIDR, which has a hardcoded '6' (ARMv8) as the supported debug version.

Ah, good point. I'll extract the value from the ID register much like we do for other fields in DBGDIDR:

commit b92565ca433f611ea0901a6098d72f91be84cdb0

Author: Oliver Upton <[email protected]>
Date:   Wed Oct 4 17:03:17 2023 +0000

KVM: arm64: Advertise selected DebugVer in DBGDIDR.Version

Much like we do for other fields, extract the Debug architecture version from the ID register to populate the corresponding field in DBGDIDR. Rewrite the existing sysreg field extractors to use SYS_FIELD_GET() for consistency.

arch/arm64/kvm/sys_regs.c +6 -5
index cf1b2def53db..57c8190d5438 100644
@@ -2510,12 +2510,13 @@ static bool trap_dbgdidr(struct kvm_vcpu *vcpu,
 	} else {
 		u64 dfr = IDREG(vcpu->kvm, SYS_ID_AA64DFR0_EL1);
 		u64 pfr = IDREG(vcpu->kvm, SYS_ID_AA64PFR0_EL1);
-		u32 el3 = !!cpuid_feature_extract_unsigned_field(pfr, ID_AA64PFR0_EL1_EL3_SHIFT);
+		u32 el3 = !!SYS_FIELD_GET(ID_AA64PFR0_EL1, EL3, pfr);
 
-		p->regval = ((((dfr >> ID_AA64DFR0_EL1_WRPs_SHIFT) & 0xf) << 28) |
-			     (((dfr >> ID_AA64DFR0_EL1_BRPs_SHIFT) & 0xf) << 24) |
-			     (((dfr >> ID_AA64DFR0_EL1_CTX_CMPs_SHIFT) & 0xf) << 20)
-			     | (6 << 16) | (1 << 15) | (el3 << 14) | (el3 << 12));
+		p->regval = ((SYS_FIELD_GET(ID_AA64DFR0_EL1, WRPs, dfr) << 28) |
+			     (SYS_FIELD_GET(ID_AA64DFR0_EL1, BRPs, dfr) << 24) |
+			     (SYS_FIELD_GET(ID_AA64DFR0_EL1, CTX_CMPs, dfr) << 20) |
+			     (SYS_FIELD_GET(ID_AA64DFR0_EL1, DebugVer, dfr) << 16) |
+			     (1 << 15) | (el3 << 14) | (el3 << 12));
 		return true;
 	}
 }
Signature · 2 lines
Thanks,
Oliver
maintainer Reviewed-by
#8 lore raw

Re: [PATCH v11 05/12] KVM: arm64: Bump up the default KVM sanitised debug version to v8p8

Quoted 56 lines · Oliver Upton
Quoted 6 lines · Marc Zyngier
Quoted 4 lines · Oliver Upton

Since ID_AA64DFR0_EL1 and ID_DFR0_EL1 are now writable from userspace, it is safe to bump up the default KVM sanitised debug version to v8p8.

Quoted 7 lines
Quoted 2 lines

The SoB sequence looks odd. Either you're the author, and Jing's SoB shouldn't be there without a Co-DB tag, or you've lost Jing's attribution (which sometimes happens when rebasing and squashing patches together).

This is an artifact of me applying Jing's version of the series and hacking on top of it.

Quoted 22 lines
Quoted 22 lines
---
 arch/arm64/kvm/sys_regs.c | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)
arch/arm64/kvm/sys_regs.c +1 -2
index 8fbfe61fe7bc..b342c96e08f4 100644
@@ -1496,8 +1496,7 @@ static u64 read_sanitised_id_aa64dfr0_el1(struct kvm_vcpu *vcpu,
 {
 	u64 val = read_sanitised_ftr_reg(SYS_ID_AA64DFR0_EL1);
 
-	/* Limit debug to ARMv8.0 */
-	val = ID_REG_LIMIT_FIELD_ENUM(val, ID_AA64DFR0_EL1, DebugVer, IMP);
+	val = ID_REG_LIMIT_FIELD_ENUM(val, ID_AA64DFR0_EL1, DebugVer, V8P8);
 
 	/*
 	 * Only initialize the PMU version if the vCPU was configured with one.
@@ -1557,6 +1556,8 @@ static u64 read_sanitised_id_dfr0_el1(struct kvm_vcpu *vcpu,
 	if (kvm_vcpu_has_pmu(vcpu))
 		val |= SYS_FIELD_PREP(ID_DFR0_EL1, PerfMon, perfmon);
 
Quoted 5 lines

arch/arm64/kvm/sys_regs.c static u64 read_sanitised_id_dfr0_el1(struct kvm_vcpu *vcpu, · line 1560

Quoted 2 lines
arch/arm64/kvm/sys_regs.c +2 -0
+	val = ID_REG_LIMIT_FIELD_ENUM(val, ID_DFR0_EL1, CopDbg, Debugv8p8);
+

For consistency, you should also repaint DBGDIDR, which has a hardcoded '6' (ARMv8) as the supported debug version.

Ah, good point. I'll extract the value from the ID register much like we do for other fields in DBGDIDR:

commit b92565ca433f611ea0901a6098d72f91be84cdb0

Author: Oliver Upton <[email protected]>
Date:   Wed Oct 4 17:03:17 2023 +0000

KVM: arm64: Advertise selected DebugVer in DBGDIDR.Version

Quoted 7 lines

Much like we do for other fields, extract the Debug architecture version from the ID register to populate the corresponding field in DBGDIDR. Rewrite the existing sysreg field extractors to use SYS_FIELD_GET() for consistency.

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
#9 lore raw

PATCH v11 6/12 KVM: arm64: Allow userspace to change ID_AA64ISAR{0-2}_EL1

In reply to Oliver Upton

Almost all of the features described by the ISA registers have no KVM involvement. Allow userspace to change the value of these registers with a couple exceptions:

- MOPS is not writable as KVM does not currently virtualize FEAT_MOPS.

- The PAuth fields are not writable as KVM requires both address and generic authentication be enabled.

---
 arch/arm64/kvm/sys_regs.c | 38 ++++++++++++++++++++++++++------------
 1 file changed, 26 insertions(+), 12 deletions(-)
arch/arm64/kvm/sys_regs.c +26 -12
index b342c96e08f4..09be55338e0a 100644
@@ -1851,11 +1851,14 @@ static unsigned int elx2_visibility(const struct kvm_vcpu *vcpu,
  * from userspace.
  */
 
-/* sys_reg_desc initialiser for known cpufeature ID registers */
-#define ID_SANITISED(name) {			\
+#define ID_DESC(name)				\
 	SYS_DESC(SYS_##name),			\
 	.access	= access_id_reg,		\
-	.get_user = get_id_reg,			\
+	.get_user = get_id_reg			\
+
+/* sys_reg_desc initialiser for known cpufeature ID registers */
+#define ID_SANITISED(name) {			\
+	ID_DESC(name),				\
 	.set_user = set_id_reg,			\
 	.visibility = id_visibility,		\
 	.reset = kvm_read_sanitised_id_reg,	\
@@ -1864,15 +1867,22 @@ static unsigned int elx2_visibility(const struct kvm_vcpu *vcpu,
 
 /* sys_reg_desc initialiser for known cpufeature ID registers */
 #define AA32_ID_SANITISED(name) {		\
-	SYS_DESC(SYS_##name),			\
-	.access	= access_id_reg,		\
-	.get_user = get_id_reg,			\
+	ID_DESC(name),				\
 	.set_user = set_id_reg,			\
 	.visibility = aa32_id_visibility,	\
 	.reset = kvm_read_sanitised_id_reg,	\
 	.val = 0,				\
 }
 
+/* sys_reg_desc initialiser for writable ID registers */
+#define ID_WRITABLE(name, mask) {		\
+	ID_DESC(name),				\
+	.set_user = set_id_reg,			\
+	.visibility = id_visibility,		\
+	.reset = kvm_read_sanitised_id_reg,	\
+	.val = mask,				\
+}
+
 /*
  * sys_reg_desc initialiser for architecturally unallocated cpufeature ID
  * register with encoding Op0=3, Op1=0, CRn=0, CRm=crm, Op2=op2
@@ -1894,9 +1904,7 @@ static unsigned int elx2_visibility(const struct kvm_vcpu *vcpu,
  * RAZ for the guest.
  */
 #define ID_HIDDEN(name) {			\
-	SYS_DESC(SYS_##name),			\
-	.access = access_id_reg,		\
-	.get_user = get_id_reg,			\
+	ID_DESC(name),				\
 	.set_user = set_id_reg,			\
 	.visibility = raz_visibility,		\
 	.reset = kvm_read_sanitised_id_reg,	\
@@ -2075,9 +2083,15 @@ static const struct sys_reg_desc sys_reg_descs[] = {
 	ID_UNALLOCATED(5,7),
 
 	/* CRm=6 */
-	ID_SANITISED(ID_AA64ISAR0_EL1),
-	ID_SANITISED(ID_AA64ISAR1_EL1),
-	ID_SANITISED(ID_AA64ISAR2_EL1),
+	ID_WRITABLE(ID_AA64ISAR0_EL1, ~ID_AA64ISAR0_EL1_RES0),
+	ID_WRITABLE(ID_AA64ISAR1_EL1, ~(ID_AA64ISAR1_EL1_GPI |
+					ID_AA64ISAR1_EL1_GPA |
+					ID_AA64ISAR1_EL1_API |
+					ID_AA64ISAR1_EL1_APA)),
+	ID_WRITABLE(ID_AA64ISAR2_EL1, ~(ID_AA64ISAR2_EL1_RES0 |
+					ID_AA64ISAR2_EL1_MOPS |
+					ID_AA64ISAR2_EL1_APA3 |
+					ID_AA64ISAR2_EL1_GPA3)),
 	ID_UNALLOCATED(6,3),
 	ID_UNALLOCATED(6,4),
 	ID_UNALLOCATED(6,5),
Signature · 1 line
2.42.0.609.gbb76f46606-goog
#10 lore raw

PATCH v11 7/12 KVM: arm64: Allow userspace to change ID_AA64MMFR{0-2}_EL1

In reply to Oliver Upton

From: Jing Zhang <[email protected]>

Allow userspace to modify the guest-visible values of these ID registers. Prevent changes to any of the virtualization features until KVM picks up support for nested and we have a handle on managing NV features.

[oliver: prevent changes to EL2 features for now]
---
 arch/arm64/kvm/sys_regs.c | 20 +++++++++++++++++---
 1 file changed, 17 insertions(+), 3 deletions(-)
arch/arm64/kvm/sys_regs.c +17 -3
index 09be55338e0a..9e31887ee0f4 100644
@@ -2099,9 +2099,23 @@ static const struct sys_reg_desc sys_reg_descs[] = {
 	ID_UNALLOCATED(6,7),
 
 	/* CRm=7 */
-	ID_SANITISED(ID_AA64MMFR0_EL1),
-	ID_SANITISED(ID_AA64MMFR1_EL1),
-	ID_SANITISED(ID_AA64MMFR2_EL1),
+	ID_WRITABLE(ID_AA64MMFR0_EL1, ~(ID_AA64MMFR0_EL1_RES0 |
+					ID_AA64MMFR0_EL1_TGRAN4_2 |
+					ID_AA64MMFR0_EL1_TGRAN64_2 |
+					ID_AA64MMFR0_EL1_TGRAN16_2)),
+	ID_WRITABLE(ID_AA64MMFR1_EL1, ~(ID_AA64MMFR1_EL1_RES0 |
+					ID_AA64MMFR1_EL1_HCX |
+					ID_AA64MMFR1_EL1_XNX |
+					ID_AA64MMFR1_EL1_TWED |
+					ID_AA64MMFR1_EL1_XNX |
+					ID_AA64MMFR1_EL1_VH |
+					ID_AA64MMFR1_EL1_VMIDBits)),
+	ID_WRITABLE(ID_AA64MMFR2_EL1, ~(ID_AA64MMFR2_EL1_RES0 |
+					ID_AA64MMFR2_EL1_EVT |
+					ID_AA64MMFR2_EL1_FWB |
+					ID_AA64MMFR2_EL1_IDS |
+					ID_AA64MMFR2_EL1_NV |
+					ID_AA64MMFR2_EL1_CCIDX)),
 	ID_SANITISED(ID_AA64MMFR3_EL1),
 	ID_UNALLOCATED(7,4),
 	ID_UNALLOCATED(7,5),
Signature · 1 line
2.42.0.609.gbb76f46606-goog
#11 lore raw

PATCH v11 8/12 KVM: arm64: Allow userspace to change ID_AA64PFR0_EL1

In reply to Oliver Upton

From: Jing Zhang <[email protected]>

Allow userspace to change the guest-visible value of the register with some severe limitation:

- No changes to features not virtualized by KVM (AMU, MPAM, RAS)

- Short of full GICv2 emulation in kernel, hiding GICv3 from the guest makes absolutely no sense.

- FP is effectively assumed for KVM VMs.

[oliver: restrict features that are illogical to change]
---
 arch/arm64/kvm/sys_regs.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
arch/arm64/kvm/sys_regs.c +7 -1
index 9e31887ee0f4..24825b73d365 100644
@@ -2057,7 +2057,13 @@ static const struct sys_reg_desc sys_reg_descs[] = {
 	  .get_user = get_id_reg,
 	  .set_user = set_id_reg,
 	  .reset = read_sanitised_id_aa64pfr0_el1,
-	  .val = ID_AA64PFR0_EL1_CSV2_MASK | ID_AA64PFR0_EL1_CSV3_MASK, },
+	  .val = ~(ID_AA64PFR0_EL1_AMU |
+		   ID_AA64PFR0_EL1_MPAM |
+		   ID_AA64PFR0_EL1_SVE |
+		   ID_AA64PFR0_EL1_RAS |
+		   ID_AA64PFR0_EL1_GIC |
+		   ID_AA64PFR0_EL1_AdvSIMD |
+		   ID_AA64PFR0_EL1_FP), },
 	ID_SANITISED(ID_AA64PFR1_EL1),
 	ID_UNALLOCATED(4,2),
 	ID_UNALLOCATED(4,3),
Signature · 1 line
2.42.0.609.gbb76f46606-goog
#12 lore raw

PATCH v11 9/12 KVM: arm64: Allow userspace to change ID_AA64ZFR0_EL1

In reply to Oliver Upton

All known fields in ID_AA64ZFR0_EL1 describe the unprivileged instructions supported by the PE's SVE implementation. Allow userspace to pick and choose the advertised feature set, though nothing stops the guest from using undisclosed instructions.

---
 arch/arm64/kvm/sys_regs.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/arm64/kvm/sys_regs.c +1 -1
index 24825b73d365..cf1b2def53db 100644
@@ -2067,7 +2067,7 @@ static const struct sys_reg_desc sys_reg_descs[] = {
 	ID_SANITISED(ID_AA64PFR1_EL1),
 	ID_UNALLOCATED(4,2),
 	ID_UNALLOCATED(4,3),
-	ID_SANITISED(ID_AA64ZFR0_EL1),
+	ID_WRITABLE(ID_AA64ZFR0_EL1, ~ID_AA64ZFR0_EL1_RES0),
 	ID_HIDDEN(ID_AA64SMFR0_EL1),
 	ID_UNALLOCATED(4,6),
 	ID_UNALLOCATED(4,7),
Signature · 1 line
2.42.0.609.gbb76f46606-goog
#13 lore raw

PATCH v11 10/12 KVM: arm64: Document vCPU feature selection UAPIs

In reply to Oliver Upton

KVM/arm64 has a couple schemes for handling vCPU feature selection now, which is a lot to put on userspace. Add some documentation about how these interact and provide some recommendations for how to use the writable ID register scheme.

---
 Documentation/virt/kvm/api.rst               |  4 ++
 Documentation/virt/kvm/arm/index.rst         |  1 +
 Documentation/virt/kvm/arm/vcpu-features.rst | 48 ++++++++++++++++++++
 3 files changed, 53 insertions(+)
 create mode 100644 Documentation/virt/kvm/arm/vcpu-features.rst
Documentation/virt/kvm/api.rst +4 -0
index d55c2b68c0a9..8d4050eedb26 100644
@@ -3370,6 +3370,8 @@ return indicates the attribute is implemented.  It does not necessarily
 indicate that the attribute can be read or written in the device's
 current state.  "addr" is ignored.
 
+.. _KVM_ARM_VCPU_INIT:
+
 4.82 KVM_ARM_VCPU_INIT
 ----------------------
 
@@ -6070,6 +6072,8 @@ writes to the CNTVCT_EL0 and CNTPCT_EL0 registers using the SET_ONE_REG
 interface. No error will be returned, but the resulting offset will not be
 applied.
 
+.. _KVM_ARM_GET_REG_WRITABLE_MASKS:
+
 4.139 KVM_ARM_GET_REG_WRITABLE_MASKS
 -------------------------------------------
 
Documentation/virt/kvm/arm/index.rst +1 -0
index e84848432158..7f231c724e16 100644
@@ -11,3 +11,4 @@ ARM
    hypercalls
    pvtime
    ptp_kvm
+   vcpu-features
Documentation/virt/kvm/arm/vcpu-features.rst new file +48 -0

48 lines not on this page · raw message

Signature · 1 line
2.42.0.609.gbb76f46606-goog
maintainer
#14 lore raw

Re: [PATCH v11 10/12] KVM: arm64: Document vCPU feature selection UAPIs

Quoted 69 lines · Oliver Upton

KVM/arm64 has a couple schemes for handling vCPU feature selection now, which is a lot to put on userspace. Add some documentation about how these interact and provide some recommendations for how to use the writable ID register scheme.

---
 Documentation/virt/kvm/api.rst               |  4 ++
 Documentation/virt/kvm/arm/index.rst         |  1 +
 Documentation/virt/kvm/arm/vcpu-features.rst | 48 ++++++++++++++++++++
 3 files changed, 53 insertions(+)
 create mode 100644 Documentation/virt/kvm/arm/vcpu-features.rst
Documentation/virt/kvm/api.rst +4 -0
index d55c2b68c0a9..8d4050eedb26 100644
@@ -3370,6 +3370,8 @@ return indicates the attribute is implemented.  It does not necessarily
 indicate that the attribute can be read or written in the device's
 current state.  "addr" is ignored.
 
+.. _KVM_ARM_VCPU_INIT:
+
 4.82 KVM_ARM_VCPU_INIT
 ----------------------
 
@@ -6070,6 +6072,8 @@ writes to the CNTVCT_EL0 and CNTPCT_EL0 registers using the SET_ONE_REG
 interface. No error will be returned, but the resulting offset will not be
 applied.
 
+.. _KVM_ARM_GET_REG_WRITABLE_MASKS:
+
 4.139 KVM_ARM_GET_REG_WRITABLE_MASKS
 -------------------------------------------
 
Documentation/virt/kvm/arm/index.rst +1 -0
index e84848432158..7f231c724e16 100644
@@ -11,3 +11,4 @@ ARM
    hypercalls
    pvtime
    ptp_kvm
+   vcpu-features
Documentation/virt/kvm/arm/vcpu-features.rst new file +18 -0
new file mode 100644
index 000000000000..2d2f89c5781f
@@ -0,0 +1,48 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+===============================
+vCPU feature selection on arm64
+===============================
+
+KVM/arm64 provides two mechanisms that allow userspace to configure
+the CPU features presented to the guest.
+
+KVM_ARM_VCPU_INIT
+=================
+
+The ``KVM_ARM_VCPU_INIT`` ioctl accepts a bitmap of feature flags
+(``struct kvm_vcpu_init::features``). Features enabled by this interface are
+*opt-in* and may change/extend UAPI. See :ref:`KVM_ARM_VCPU_INIT` for complete
+documentation of the features controlled by the ioctl.
+
+Otherwise, all CPU features supported by KVM are described by the architected

Documentation/virt/kvm/arm/vcpu-features.rst line 28

Quoted 10 lines
Documentation/virt/kvm/arm/vcpu-features.rst new file +10 -0
+ID registers.
+
+The ID Registers
+================
+
+The Arm architecture specifies a range of *ID Registers* that describe the set
+of architectural features supported by the CPU implementation. KVM initializes
+the guest's ID registers to the maximum set of CPU features supported by the
+system. The ID register values are VM-scoped in KVM, meaning that the values
+are identical for all vCPUs in a VM.

I'm a bit reluctant to give this guarantee. Case in point: MPIDR_EL1 is part of the Feature ID space, and is definitely *not* a register that we can make global, even on a fully homogeneous system.

I'd also like to give us more flexibility to change the implementation in the future without having to change the API again. IMO, the fact that we make our life simpler by only tracking a single copy is an implementation detail, not something that userspace should rely on.

I would simply turn the "The ID register values are VM-scoped" into "The ID register values may be VM-scoped", which gives us that flexibility.

Documentation/virt/kvm/arm/vcpu-features.rst line 39

Quoted 11 lines
Documentation/virt/kvm/arm/vcpu-features.rst new file +11 -0
+
+KVM allows userspace to *opt-out* of certain CPU features described by the ID
+registers by writing values to them via the ``KVM_SET_ONE_REG`` ioctl. The ID
+registers are mutable until the VM has started, i.e. userspace has called
+``KVM_RUN`` on at least one vCPU in the VM. Userspace can discover what fields
+are mutable in the ID registers using the ``KVM_ARM_GET_REG_WRITABLE_MASKS``.
+See the :ref:`ioctl documentation <KVM_ARM_GET_REG_WRITABLE_MASKS>` for more
+details.
+
+Userspace is allowed to *limit* or *mask* CPU features according to the rules
+outlined by the architecture in DDI0487J 'D19.1.3 Principles of the ID scheme

nit: consider spelling out the *full* version of the ARM ARM (DDI 0487J.a), just in case we get a J.b this side of Xmas and that this reference is renumbered...

Documentation/virt/kvm/arm/vcpu-features.rst line 48

Quoted 9 lines
Documentation/virt/kvm/arm/vcpu-features.rst new file +9 -0
+for fields in ID register'. KVM does not allow ID register values that exceed
+the capabilities of the system.
+
+.. warning::
+   It is **strongly recommended** that userspace modify the ID register values
+   before accessing the rest of the vCPU's CPU register state. KVM may use the
+   ID register values to control feature emulation. Interleaving ID register
+   modification with other system register accesses may lead to unpredictable
+   behavior.

Thanks,

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
#15 lore raw

Re: [PATCH v11 10/12] KVM: arm64: Document vCPU feature selection UAPIs

Quoted 2 lines · Marc Zyngier

On Wed, 04 Oct 2023 00:04:06 +0100,
Oliver Upton <[email protected]> wrote:

[...]

Quoted 4 lines
Quoted 4 lines
(quoted diff) +4 -0
+The ID Registers
+================
+
+The Arm architecture specifies a range of *ID Registers* that describe the set
Quoted 8 lines

Review

Quoted 4 lines
(quoted diff) +4 -0
+of architectural features supported by the CPU implementation. KVM initializes
+the guest's ID registers to the maximum set of CPU features supported by the
+system. The ID register values are VM-scoped in KVM, meaning that the values
+are identical for all vCPUs in a VM.

I'm a bit reluctant to give this guarantee. Case in point: MPIDR_EL1 is part of the Feature ID space, and is definitely *not* a register that we can make global, even on a fully homogeneous system.

Oh, very good point.

Quoted 8 lines

I'd also like to give us more flexibility to change the implementation in the future without having to change the API again. IMO, the fact that we make our life simpler by only tracking a single copy is an implementation detail, not something that userspace should rely on.

I would simply turn the "The ID register values are VM-scoped" into "The ID register values may be VM-scoped", which gives us that flexibility.

Agreed, I'm happy to duck behind some vague language here :)

Quoted 7 lines
Quoted 7 lines
(quoted diff) +7 -0
+
+KVM allows userspace to *opt-out* of certain CPU features described by the ID
+registers by writing values to them via the ``KVM_SET_ONE_REG`` ioctl. The ID
+registers are mutable until the VM has started, i.e. userspace has called
+``KVM_RUN`` on at least one vCPU in the VM. Userspace can discover what fields
+are mutable in the ID registers using the ``KVM_ARM_GET_REG_WRITABLE_MASKS``.
+See the :ref:`ioctl documentation <KVM_ARM_GET_REG_WRITABLE_MASKS>` for more
Quoted 8 lines

Review

Quoted 4 lines
(quoted diff) +4 -0
+details.
+
+Userspace is allowed to *limit* or *mask* CPU features according to the rules
+outlined by the architecture in DDI0487J 'D19.1.3 Principles of the ID scheme

nit: consider spelling out the *full* version of the ARM ARM (DDI 0487J.a), just in case we get a J.b this side of Xmas and that this reference is renumbered...

Going to fix both of these with the following diff:

Documentation/virt/kvm/arm/vcpu-features.rst +5 -5
index 2d2f89c5781f..f7cc6d8d8b74 100644
@@ -24,8 +24,8 @@ The ID Registers
 The Arm architecture specifies a range of *ID Registers* that describe the set
 of architectural features supported by the CPU implementation. KVM initializes
 the guest's ID registers to the maximum set of CPU features supported by the
-system. The ID register values are VM-scoped in KVM, meaning that the values
-are identical for all vCPUs in a VM.
+system. The ID register values may be VM-scoped in KVM, meaning that the
+values could be shared for all vCPUs in a VM.
 
 KVM allows userspace to *opt-out* of certain CPU features described by the ID
 registers by writing values to them via the ``KVM_SET_ONE_REG`` ioctl. The ID
@@ -36,9 +36,9 @@ See the :ref:`ioctl documentation <KVM_ARM_GET_REG_WRITABLE_MASKS>` for more
 details.
 
 Userspace is allowed to *limit* or *mask* CPU features according to the rules
-outlined by the architecture in DDI0487J 'D19.1.3 Principles of the ID scheme
-for fields in ID register'. KVM does not allow ID register values that exceed
-the capabilities of the system.
+outlined by the architecture in DDI0487J.a D19.1.3 'Principles of the ID
+scheme for fields in ID register'. KVM does not allow ID register values that
+exceed the capabilities of the system.
 
 .. warning::
    It is **strongly recommended** that userspace modify the ID register values
Signature · 2 lines
Thanks,
Oliver
maintainer
#16 lore raw

Re: [PATCH v11 10/12] KVM: arm64: Document vCPU feature selection UAPIs

Quoted 70 lines · Oliver Upton
Quoted 2 lines · Marc Zyngier

On Wed, 04 Oct 2023 00:04:06 +0100,
Oliver Upton <[email protected]> wrote:

[...]

Quoted 4 lines
Quoted 4 lines
(quoted diff) +4 -0
+The ID Registers
+================
+
+The Arm architecture specifies a range of *ID Registers* that describe the set
Quoted 8 lines

Review

Quoted 4 lines
(quoted diff) +4 -0
+of architectural features supported by the CPU implementation. KVM initializes
+the guest's ID registers to the maximum set of CPU features supported by the
+system. The ID register values are VM-scoped in KVM, meaning that the values
+are identical for all vCPUs in a VM.

I'm a bit reluctant to give this guarantee. Case in point: MPIDR_EL1 is part of the Feature ID space, and is definitely *not* a register that we can make global, even on a fully homogeneous system.

Oh, very good point.

Quoted 8 lines

I'd also like to give us more flexibility to change the implementation in the future without having to change the API again. IMO, the fact that we make our life simpler by only tracking a single copy is an implementation detail, not something that userspace should rely on.

I would simply turn the "The ID register values are VM-scoped" into "The ID register values may be VM-scoped", which gives us that flexibility.

Agreed, I'm happy to duck behind some vague language here :)

Quoted 7 lines
Quoted 7 lines
(quoted diff) +7 -0
+
+KVM allows userspace to *opt-out* of certain CPU features described by the ID
+registers by writing values to them via the ``KVM_SET_ONE_REG`` ioctl. The ID
+registers are mutable until the VM has started, i.e. userspace has called
+``KVM_RUN`` on at least one vCPU in the VM. Userspace can discover what fields
+are mutable in the ID registers using the ``KVM_ARM_GET_REG_WRITABLE_MASKS``.
+See the :ref:`ioctl documentation <KVM_ARM_GET_REG_WRITABLE_MASKS>` for more
Quoted 8 lines

Review

Quoted 4 lines
(quoted diff) +4 -0
+details.
+
+Userspace is allowed to *limit* or *mask* CPU features according to the rules
+outlined by the architecture in DDI0487J 'D19.1.3 Principles of the ID scheme

nit: consider spelling out the *full* version of the ARM ARM (DDI 0487J.a), just in case we get a J.b this side of Xmas and that this reference is renumbered...

Going to fix both of these with the following diff:

Documentation/virt/kvm/arm/vcpu-features.rst +2 -2
index 2d2f89c5781f..f7cc6d8d8b74 100644
@@ -24,8 +24,8 @@ The ID Registers
 The Arm architecture specifies a range of *ID Registers* that describe the set
 of architectural features supported by the CPU implementation. KVM initializes
 the guest's ID registers to the maximum set of CPU features supported by the
-system. The ID register values are VM-scoped in KVM, meaning that the values
-are identical for all vCPUs in a VM.
+system. The ID register values may be VM-scoped in KVM, meaning that the
+values could be shared for all vCPUs in a VM.
 
 KVM allows userspace to *opt-out* of certain CPU features described by the ID
 registers by writing values to them via the ``KVM_SET_ONE_REG`` ioctl. The ID
@@ -36,9 +36,9 @@ See the :ref:`ioctl documentation <KVM_ARM_GET_REG_WRITABLE_MASKS>` for more
 details.
 

Documentation/virt/kvm/arm/vcpu-features.rst See the :ref:`ioctl documentation <KVM_ARM_GET_REG_WRITABLE_MASKS>` for more · line 44

Quoted 10 lines
Documentation/virt/kvm/arm/vcpu-features.rst +3 -3
 Userspace is allowed to *limit* or *mask* CPU features according to the rules
-outlined by the architecture in DDI0487J 'D19.1.3 Principles of the ID scheme
-for fields in ID register'. KVM does not allow ID register values that exceed
-the capabilities of the system.
+outlined by the architecture in DDI0487J.a D19.1.3 'Principles of the ID
+scheme for fields in ID register'. KVM does not allow ID register values that
+exceed the capabilities of the system.
 
 .. warning::
    It is **strongly recommended** that userspace modify the ID register values

Yup, looks good.

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
#17 lore raw

PATCH v11 11/12 KVM: arm64: selftests: Import automatic generation of sysreg defs

In reply to Oliver Upton

From: Jing Zhang <[email protected]>

Import automatic system register definition generation from kernel and update system register usage accordingly.

---
 tools/arch/arm64/include/.gitignore           |    1 +
 tools/arch/arm64/include/asm/gpr-num.h        |   26 +
 tools/arch/arm64/include/asm/sysreg.h         |  839 ++----
 tools/arch/arm64/tools/gen-sysreg.awk         |  336 +++
 tools/arch/arm64/tools/sysreg                 | 2497 +++++++++++++++++
 tools/testing/selftests/kvm/Makefile          |   14 +-
 .../selftests/kvm/aarch64/aarch32_id_regs.c   |    4 +-
 .../selftests/kvm/aarch64/debug-exceptions.c  |   12 +-
 .../selftests/kvm/aarch64/page_fault_test.c   |    6 +-
 .../selftests/kvm/lib/aarch64/processor.c     |    6 +-
 10 files changed, 3078 insertions(+), 663 deletions(-)
 create mode 100644 tools/arch/arm64/include/.gitignore
 create mode 100644 tools/arch/arm64/include/asm/gpr-num.h
 create mode 100755 tools/arch/arm64/tools/gen-sysreg.awk
 create mode 100644 tools/arch/arm64/tools/sysreg
tools/arch/arm64/include/.gitignore new file +1 -0
new file mode 100644
index 000000000000..9ab870da897d
@@ -0,0 +1 @@
+generated/
tools/arch/arm64/include/asm/gpr-num.h new file +26 -0
new file mode 100644
index 000000000000..05da4a7c5788
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef __ASM_GPR_NUM_H
+#define __ASM_GPR_NUM_H
+
+#ifdef __ASSEMBLY__
+
+	.irp	num,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30
+	.equ	.L__gpr_num_x\num, \num
+	.equ	.L__gpr_num_w\num, \num
+	.endr
+	.equ	.L__gpr_num_xzr, 31
+	.equ	.L__gpr_num_wzr, 31
+
+#else /* __ASSEMBLY__ */
+
+#define __DEFINE_ASM_GPR_NUMS					\
+"	.irp	num,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30\n" \
+"	.equ	.L__gpr_num_x\\num, \\num\n"			\
+"	.equ	.L__gpr_num_w\\num, \\num\n"			\
+"	.endr\n"						\
+"	.equ	.L__gpr_num_xzr, 31\n"				\
+"	.equ	.L__gpr_num_wzr, 31\n"
+
+#endif /* __ASSEMBLY__ */
+
+#endif /* __ASM_GPR_NUM_H */
tools/arch/arm64/include/asm/sysreg.h +192 -647

1,107 lines not on this page · raw message

tools/arch/arm64/tools/gen-sysreg.awk new file +336 -0

336 lines not on this page · raw message

tools/arch/arm64/tools/sysreg new file +2497 -0

1,530 lines not on this page · raw message

tools/testing/selftests/kvm/Makefile +12 -2
index a3bb36fb3cfc..1cdefbbc728c 100644

Lines of this file are not shown. View the raw message.

tools/testing/selftests/kvm/aarch64/aarch32_id_regs.c +2 -2
index b90580840b22..8e5bd07a3727 100644

Lines of this file are not shown. View the raw message.

tools/testing/selftests/kvm/aarch64/debug-exceptions.c +6 -6
index f5b6cb3a0019..866002917441 100644

Lines of this file are not shown. View the raw message.

tools/testing/selftests/kvm/aarch64/page_fault_test.c +3 -3
index 47bb914ab2fa..975d28be3cca 100644

Lines of this file are not shown. View the raw message.

tools/testing/selftests/kvm/lib/aarch64/processor.c +3 -3
index 3a0259e25335..6fe12e985ba5 100644

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

Signature · 1 line
2.42.0.609.gbb76f46606-goog

Part of the diff is not shown here. View the raw message.

#18 lore raw

PATCH v11 12/12 KVM: arm64: selftests: Test for setting ID register from usersapce

In reply to Oliver Upton

From: Jing Zhang <[email protected]>

Add tests to verify setting ID registers from userspace is handled correctly by KVM. Also add a test case to use ioctl KVM_ARM_GET_REG_WRITABLE_MASKS to get writable masks.

---
 tools/testing/selftests/kvm/Makefile          |   1 +
 .../selftests/kvm/aarch64/set_id_regs.c       | 479 ++++++++++++++++++
 2 files changed, 480 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/aarch64/set_id_regs.c
tools/testing/selftests/kvm/Makefile +1 -0
index 1cdefbbc728c..fdeb1abfdc8b 100644
@@ -145,6 +145,7 @@ TEST_GEN_PROGS_aarch64 += aarch64/debug-exceptions
 TEST_GEN_PROGS_aarch64 += aarch64/hypercalls
 TEST_GEN_PROGS_aarch64 += aarch64/page_fault_test
 TEST_GEN_PROGS_aarch64 += aarch64/psci_test
+TEST_GEN_PROGS_aarch64 += aarch64/set_id_regs
 TEST_GEN_PROGS_aarch64 += aarch64/smccc_filter
 TEST_GEN_PROGS_aarch64 += aarch64/vcpu_width_config
 TEST_GEN_PROGS_aarch64 += aarch64/vgic_init
tools/testing/selftests/kvm/aarch64/set_id_regs.c new file +479 -0

479 lines not on this page · raw message

Signature · 1 line
2.42.0.609.gbb76f46606-goog
maintainer Reviewed-by on the v11 series
#19 lore raw
In reply to Oliver Upton
Quoted 13 lines · Oliver Upton

Few more fixes that I threw on top:

v10 -> v11:
- Drop the custom handling of FEAT_BC as it is now fixed on the arm64 side (Kristina)
- Bikeshed on the naming of the masks ioctl to keep things in the KVM_ namespace
- Apply more bikeshedding to the ioctl documentation, spinning off separate blocks for the 'generic' description and the Feature ID documentation
- Fix referencing in the vCPU features doc
- Fix use of uninitialized data in selftest

We'll probably need another bit on top to deal with Kirstina's FEAT_MOPS series and make that field writable.

The minor nitpicks I had notwithstanding:

Thanks,

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
#20 lore raw
Quoted 16 lines · Marc Zyngier
Quoted 13 lines · Oliver Upton

Few more fixes that I threw on top:

v10 -> v11:
- Drop the custom handling of FEAT_BC as it is now fixed on the arm64 side (Kristina)
- Bikeshed on the naming of the masks ioctl to keep things in the KVM_ namespace
- Apply more bikeshedding to the ioctl documentation, spinning off separate blocks for the 'generic' description and the Feature ID documentation
- Fix referencing in the vCPU features doc
- Fix use of uninitialized data in selftest

Quoted 2 lines

We'll probably need another bit on top to deal with Kirstina's FEAT_MOPS series and make that field writable.

Yep, I was planning on a one-liner to unmask MOPS once I get Kristina's patches applied.

Quoted 3 lines

The minor nitpicks I had notwithstanding:

Appreciated!

Signature · 2 lines
Thanks,
Oliver
author Applied
#21 lore raw
In reply to Oliver Upton
Quoted 13 lines · Oliver Upton

Few more fixes that I threw on top:

v10 -> v11:
- Drop the custom handling of FEAT_BC as it is now fixed on the arm64 side (Kristina)
- Bikeshed on the naming of the masks ioctl to keep things in the KVM_ namespace
- Apply more bikeshedding to the ioctl documentation, spinning off separate blocks for the 'generic' description and the Feature ID documentation
- Fix referencing in the vCPU features doc
- Fix use of uninitialized data in selftest

Quoted 1 line

[...]

Applied to kvmarm/next, thanks!

[01/12] KVM: arm64: Allow userspace to get the writable masks for feature ID registers

        https://git.kernel.org/kvmarm/kvmarm/c/3f9cd0ca8484
[02/12] KVM: arm64: Document KVM_ARM_GET_REG_WRITABLE_MASKS
        https://git.kernel.org/kvmarm/kvmarm/c/6656cda0f3b2
[03/12] KVM: arm64: Use guest ID register values for the sake of emulation
        https://git.kernel.org/kvmarm/kvmarm/c/8b6958d6ace1
[04/12] KVM: arm64: Reject attempts to set invalid debug arch version
        https://git.kernel.org/kvmarm/kvmarm/c/a9bc4a1c1e0c
[05/12] KVM: arm64: Bump up the default KVM sanitised debug version to v8p8
        https://git.kernel.org/kvmarm/kvmarm/c/9f9917bc71b0
[06/12] KVM: arm64: Allow userspace to change ID_AA64ISAR{0-2}_EL1
        https://git.kernel.org/kvmarm/kvmarm/c/56d77aa8bdf5
[07/12] KVM: arm64: Allow userspace to change ID_AA64MMFR{0-2}_EL1
        https://git.kernel.org/kvmarm/kvmarm/c/d5a32b60dc18
[08/12] KVM: arm64: Allow userspace to change ID_AA64PFR0_EL1
        https://git.kernel.org/kvmarm/kvmarm/c/8cfd5be88ebe
[09/12] KVM: arm64: Allow userspace to change ID_AA64ZFR0_EL1
        https://git.kernel.org/kvmarm/kvmarm/c/f89fbb350dd7
[10/12] KVM: arm64: Document vCPU feature selection UAPIs
        https://git.kernel.org/kvmarm/kvmarm/c/dafa493dd01d
[11/12] KVM: arm64: selftests: Import automatic generation of sysreg defs
        https://git.kernel.org/kvmarm/kvmarm/c/6a4c6c6a56c1
[12/12] KVM: arm64: selftests: Test for setting ID register from usersapce
        https://git.kernel.org/kvmarm/kvmarm/c/3b44c2008bf0

--
Best,
Oliver