Support writable CPU ID registers from userspace
v12 cover Support writable CPU ID registers from userspace
Hello folks,
In the interest of expediting the acceptance of the generalized ID register infrastructure, I have taken Jing's series and applied fixes according to the feedback of reviewers and my own interpretation.
The laundry list:
- Split and reorganize the changes to avoid introducing intermediate regressions in the series. Don't want to break bisection! Sadly, this requires introducing a bit of churn but to me is an acceptable tradeoff.
- Split up monster changes into more bite-sized chunks.
- Actually delete the old masking logic when we no longer use it.
- Reinterpret IMP_DEF PMUs as NI at the time of write
- Move locking back into ID reg specific code. Let it be known: if you want to do custom ->set_user() logic for an ID register, it better find its way into set_id_reg().
- Rewrite kvm_vcpu_set_target() into something that's a bit more cognizant of the existence of a bitmap.
- Prevent userspace from configuring asymmetric vCPU features. Yep, this is technically user-visible breakage, but has been a long time coming. Asymmetry in KVM has always worked by chance, and is inconsistent with the overall handling of asymmetry in the kernel. If this breaks you: now's your chance to plead your usecase.
- Rip out the PMUVer <-> PerfMon bidirectional pipe. Making things consistent between the 32bit and 64bit worlds is needlessly complicated, and the only thing we care about in KVM is that the advertised feature set aligns with the limits of the system/VM.
- Allow the value of KVM_ARM_VCPU_POWER_OFF to change between calls. It doesn't make sense in the context of a VM-wide feature set, and we may as well let userspace do whatever it wants with the bit.
- Opportunistically rewrite sysreg definitions to use the new syntax.
Tested with selftests, kvm-unit-tests, as well as booting an actual VM with kvmtool + QEMU.
v11: https://lore.kernel.org/kvmarm/[email protected]/
Jing Zhang (5):
KVM: arm64: Reuse fields of sys_reg_desc for idreg
KVM: arm64: Save ID registers' sanitized value per guest
KVM: arm64: Use arm64_ftr_bits to sanitise ID register writes
KVM: arm64: Use generic sanitisation for ID_(AA64)DFR0_EL1
KVM: arm64: Use generic sanitisation for ID_AA64PFR0_EL1
Oliver Upton (6):
KVM: arm64: Separate out feature sanitisation and initialisation
KVM: arm64: Relax invariance of KVM_ARM_VCPU_POWER_OFF
KVM: arm64: Make vCPU feature flags consistent VM-wide
KVM: arm64: Rewrite IMPDEF PMU version as NI
KVM: arm64: Handle ID register reads using the VM-wide values
KVM: arm64: Rip out the vestiges of the 'old' ID register scheme
arch/arm64/include/asm/cpufeature.h | 1 + arch/arm64/include/asm/kvm_emulate.h | 7 +- arch/arm64/include/asm/kvm_host.h | 45 +-- arch/arm64/kernel/cpufeature.c | 2 +- arch/arm64/kvm/arm.c | 134 +++++--- arch/arm64/kvm/reset.c | 58 ---- arch/arm64/kvm/sys_regs.c | 493 +++++++++++++++++++-------- arch/arm64/kvm/sys_regs.h | 22 +- include/kvm/arm_pmu.h | 8 +- 9 files changed, 478 insertions(+), 292 deletions(-)
base-commit: 7877cb91f1081754a1487c144d85dc0d2e2e7fc4
Signature · 1 line
2.41.0.162.gfafddb0af9-goog
PATCH v12 1/11 KVM: arm64: Separate out feature sanitisation and initialisation
kvm_vcpu_set_target() iteratively sanitises and copies feature flags in one go. This is rather odd, especially considering the fact that bitmap accessors can do the heavy lifting. A subsequent change will make vCPU features VM-wide, and fitting that into the present implementation is just a chore.
Rework the whole thing to use bitmap accessors to sanitise and copy flags.
--- arch/arm64/include/asm/kvm_host.h | 1 + arch/arm64/kvm/arm.c | 75 +++++++++++++++++++------------ 2 files changed, 48 insertions(+), 28 deletions(-)
arch/arm64/include/asm/kvm_host.h +1 -0
@@ -39,6 +39,7 @@#define KVM_MAX_VCPUS VGIC_V3_MAX_CPUS#define KVM_VCPU_MAX_FEATURES 7 +#define KVM_VCPU_VALID_FEATURES (BIT(KVM_VCPU_MAX_FEATURES) - 1)#define KVM_REQ_SLEEP \ KVM_ARCH_REQ_FLAGS(0, KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP)
arch/arm64/kvm/arm.c +47 -28
95 lines not on this page · raw message
Signature · 1 line
2.41.0.162.gfafddb0af9-goog
PATCH v12 2/11 KVM: arm64: Relax invariance of KVM_ARM_VCPU_POWER_OFF
In reply to Oliver UptonAllow the value of KVM_ARM_VCPU_POWER_OFF to differ between calls to KVM_ARM_VCPU_INIT. Userspace can already change the state of the vCPU through the KVM_SET_MP_STATE ioctl, so making the bit invariant seems needlessly restrictive.
--- arch/arm64/kvm/arm.c | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-)
arch/arm64/kvm/arm.c +12 -1
@@ -1236,8 +1236,19 @@ static int kvm_vcpu_set_target(struct kvm_vcpu *vcpu,static int kvm_arch_vcpu_ioctl_vcpu_init (struct kvm_vcpu *vcpu,struct kvm_vcpu_init *init) { +bool power_off = false;int ret; +/* +* Treat the power-off vCPU feature as ephemeral. Clear the bit to avoid +* reflecting it in the finalized feature set, thus limiting its scope +* to a single KVM_ARM_VCPU_INIT call. +*/ +if (init->features[0 ] &KVM_ARM_VCPU_POWER_OFF ) { + init->features[0 ] &= ~KVM_ARM_VCPU_POWER_OFF ; + power_off = true; + } + ret =kvm_vcpu_set_target (vcpu, init);if (ret)return ret;
@@ -1266,7 +1277,7 @@ static int kvm_arch_vcpu_ioctl_vcpu_init(struct kvm_vcpu *vcpu,*/ spin_lock (&vcpu->arch.mp_state_lock); -if (test_bit (KVM_ARM_VCPU_POWER_OFF , vcpu->arch.features)) +if (power_off)__kvm_arm_vcpu_power_off (vcpu);else WRITE_ONCE (vcpu->arch.mp_state.mp_state,KVM_MP_STATE_RUNNABLE );
Signature · 1 line
2.41.0.162.gfafddb0af9-goog
PATCH v12 3/11 KVM: arm64: Make vCPU feature flags consistent VM-wide
In reply to Oliver UptonTo date KVM has allowed userspace to construct asymmetric VMs where particular features may only be supported on a subset of vCPUs. This wasn't really the intened usage pattern, and it is a total pain in the ass to keep working in the kernel. What's more, this is at odds with CPU features in host userspace, where asymmetric features are largely hidden or disabled.
It's time to put an end to the whole game. Require all vCPUs in the VM to have the same feature set, rejecting deviants in the KVM_ARM_VCPU_INIT ioctl. Preserve some of the vestiges of per-vCPU feature flags in case we need to reinstate the old behavior for some limited configurations. Yes, this is a sign of cowardice around a user-visibile change.
Hoist all of the 32-bit limitations into kvm_vcpu_init_check_features() to avoid nested attempts to acquire the config_lock, which won't end well.
--- arch/arm64/include/asm/kvm_emulate.h | 7 +--- arch/arm64/include/asm/kvm_host.h | 22 +++++------ arch/arm64/kvm/arm.c | 31 ++++++++++++++- arch/arm64/kvm/reset.c | 58 ---------------------------- 4 files changed, 40 insertions(+), 78 deletions(-)
arch/arm64/include/asm/kvm_emulate.h +1 -6
@@ -62,12 +62,7 @@ static __always_inline bool vcpu_el1_is_32bit(struct kvm_vcpu *vcpu)#else static __always_inline bool vcpu_el1_is_32bit(struct kvm_vcpu *vcpu) { -struct kvm *kvm = vcpu->kvm; - -WARN_ON_ONCE (!test_bit (KVM_ARCH_FLAG_REG_WIDTH_CONFIGURED , - &kvm->arch.flags)); - -return test_bit (KVM_ARCH_FLAG_EL1_32BIT , &kvm->arch.flags); +return test_bit (KVM_ARM_VCPU_EL1_32BIT , vcpu->arch.features); }#endif
arch/arm64/include/asm/kvm_host.h +9 -13
@@ -215,25 +215,21 @@ struct kvm_arch {#define KVM_ARCH_FLAG_MTE_ENABLED 1/* At least one vCPU has ran in the VM */ #define KVM_ARCH_FLAG_HAS_RAN_ONCE 2 -/* -* The following two bits are used to indicate the guest's EL1 -* register width configuration. A value of KVM_ARCH_FLAG_EL1_32BIT -* bit is valid only when KVM_ARCH_FLAG_REG_WIDTH_CONFIGURED is set. -* Otherwise, the guest's EL1 register width has not yet been -* determined yet. -*/ -#define KVM_ARCH_FLAG_REG_WIDTH_CONFIGURED 3 -#define KVM_ARCH_FLAG_EL1_32BIT 4 +/* The vCPU feature set for the VM is configured */ +#define KVM_ARCH_FLAG_VCPU_FEATURES_CONFIGURED 3/* PSCI SYSTEM_SUSPEND enabled for the guest */ -#define KVM_ARCH_FLAG_SYSTEM_SUSPEND_ENABLED 5 +#define KVM_ARCH_FLAG_SYSTEM_SUSPEND_ENABLED 4/* VM counter offset */ -#define KVM_ARCH_FLAG_VM_COUNTER_OFFSET 6 +#define KVM_ARCH_FLAG_VM_COUNTER_OFFSET 5/* Timer PPIs made immutable */ -#define KVM_ARCH_FLAG_TIMER_PPIS_IMMUTABLE 7 +#define KVM_ARCH_FLAG_TIMER_PPIS_IMMUTABLE 6/* SMCCC filter initialized for the VM */ -#define KVM_ARCH_FLAG_SMCCC_FILTER_CONFIGURED 8 +#define KVM_ARCH_FLAG_SMCCC_FILTER_CONFIGURED 7unsigned long flags; +/* VM-wide vCPU feature set */ +DECLARE_BITMAP (vcpu_features,KVM_VCPU_MAX_FEATURES ); +/* * VM-wide PMU filter, implemented as a bitmap and big enough for * up to 2^10 events (ARMv8.0) or 2^16 events (ARMv8.1+).
arch/arm64/kvm/arm.c +30 -1
@@ -170,6 +170,8 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)*/ kvm->arch.dfr0_pmuver.imp =kvm_arm_pmu_get_pmuver_limit (); +bitmap_zero (kvm->arch.vcpu_features,KVM_VCPU_MAX_FEATURES ); +return 0 ; err_free_cpumask:
@@ -1181,6 +1183,20 @@ static int kvm_vcpu_init_check_features(struct kvm_vcpu *vcpu,return -ENOENT ; } +if (!test_bit (KVM_ARM_VCPU_EL1_32BIT , &features)) +return 0 ; + +if (!cpus_have_const_cap (ARM64_HAS_32BIT_EL1 )) +return -EINVAL ; + +/* MTE is incompatible with AArch32 */ +if (kvm_has_mte (vcpu->kvm)) +return -EINVAL ; + +/* NV is incompatible with AArch32 */ +if (test_bit (KVM_ARM_VCPU_HAS_EL2 , &features)) +return -EINVAL ; +return 0 ; }
@@ -1197,7 +1213,14 @@ static int __kvm_vcpu_set_target(struct kvm_vcpu *vcpu, const struct kvm_vcpu_init *init) {unsigned long features = init->features[0 ]; -int ret; +struct kvm *kvm = vcpu->kvm; +int ret = -EINVAL ; + +mutex_lock (&kvm->arch.config_lock); + +if (test_bit (KVM_ARCH_FLAG_VCPU_FEATURES_CONFIGURED , &kvm->arch.flags) && + !bitmap_equal (kvm->arch.vcpu_features, &features,KVM_VCPU_MAX_FEATURES )) + goto out_unlock; vcpu->arch.target = init->target;bitmap_copy (vcpu->arch.features, &features,KVM_VCPU_MAX_FEATURES );
@@ -1207,8 +1230,14 @@ static int __kvm_vcpu_set_target(struct kvm_vcpu *vcpu,if (ret) { vcpu->arch.target =-1 ;bitmap_zero (vcpu->arch.features,KVM_VCPU_MAX_FEATURES ); + goto out_unlock; } +bitmap_copy (kvm->arch.vcpu_features, &features,KVM_VCPU_MAX_FEATURES ); +set_bit (KVM_ARCH_FLAG_VCPU_FEATURES_CONFIGURED , &kvm->arch.flags); + +out_unlock: +mutex_unlock (&kvm->arch.config_lock);return ret; }
arch/arm64/kvm/reset.c +0 -58
70 lines not on this page · raw message
Signature · 1 line
2.41.0.162.gfafddb0af9-goog
PATCH v12 4/11 KVM: arm64: Rewrite IMPDEF PMU version as NI
In reply to Oliver UptonKVM allows userspace to write an IMPDEF PMU version to the corresponding 32bit and 64bit ID register fields for the sake of backwards compatibility with kernels that lacked commit 3d0dba5764b9 ("KVM: arm64:
PMU: Move the ID_AA64DFR0_EL1.PMUver limit to VM creation"). Plumbing that IMPDEF PMU version through to the gues is getting in the way of progress, and really doesn't any sense in the first place.
Bite the bullet and reinterpret the IMPDEF PMU version as NI (0) for userspace writes. Additionally, spill the dirty details into a comment.
--- arch/arm64/include/asm/kvm_host.h | 5 +-- arch/arm64/kvm/arm.c | 2 +- arch/arm64/kvm/sys_regs.c | 56 +++++++++++++++++++------------ include/kvm/arm_pmu.h | 2 +- 4 files changed, 38 insertions(+), 27 deletions(-)
arch/arm64/include/asm/kvm_host.h +1 -4
@@ -241,10 +241,7 @@ struct kvm_arch {u8 pfr0_csv2;u8 pfr0_csv3; -struct { -u8 imp:4 ; -u8 unimp:4 ; - } dfr0_pmuver; +u8 dfr0_pmuver;/* Hypercall features firmware registers' descriptor */ struct kvm_smccc_features smccc_feat;
arch/arm64/kvm/arm.c +1 -1
@@ -168,7 +168,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)* Initialise the default PMUver before there is a chance to * create an actual PMU. */ - kvm->arch.dfr0_pmuver.imp =kvm_arm_pmu_get_pmuver_limit (); + kvm->arch.dfr0_pmuver =kvm_arm_pmu_get_pmuver_limit ();bitmap_zero (kvm->arch.vcpu_features,KVM_VCPU_MAX_FEATURES );
arch/arm64/kvm/sys_regs.c +35 -21
102 lines not on this page · raw message
include/kvm/arm_pmu.h +1 -1
@@ -93,7 +93,7 @@ void kvm_vcpu_pmu_restore_host(struct kvm_vcpu *vcpu);* Evaluates as true when emulating PMUv3p5, and false otherwise. */ #define kvm_pmu_is_3p5 (vcpu) \ - (vcpu->kvm->arch.dfr0_pmuver.imp >= ID_AA64DFR0_EL1_PMUVer_V3P5) + (vcpu->kvm->arch.dfr0_pmuver >= ID_AA64DFR0_EL1_PMUVer_V3P5)u8 kvm_arm_pmu_get_pmuver_limit (void );
Signature · 1 line
2.41.0.162.gfafddb0af9-goog
PATCH v12 5/11 KVM: arm64: Reuse fields of sys_reg_desc for idreg
In reply to Oliver UptonFrom: Jing Zhang <[email protected]>
sys_reg_desc::{reset, val} are presently unused for ID register descriptors. Repurpose these fields to support user-configurable ID registers.
Use the ::reset() function pointer to return the sanitised value of a given ID register, optionally with KVM-specific feature sanitisation. Additionally, keep a mask of writable register fields in ::val.
--- arch/arm64/kvm/sys_regs.c | 94 ++++++++++++++++++++++++++++++--------- arch/arm64/kvm/sys_regs.h | 15 ++++--- 2 files changed, 84 insertions(+), 25 deletions(-)
arch/arm64/kvm/sys_regs.c +74 -20
@@ -540,10 +540,11 @@ static int get_bvr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd,return 0 ; } -static void reset_bvr (struct kvm_vcpu *vcpu, +static u64 reset_bvr (struct kvm_vcpu *vcpu,const struct sys_reg_desc *rd) { vcpu->arch.vcpu_debug_state.dbg_bvr[rd->CRm] = rd->val; +return rd->val; } static bool trap_bcr(struct kvm_vcpu *vcpu,
@@ -576,10 +577,11 @@ static int get_bcr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd,return 0 ; } -static void reset_bcr (struct kvm_vcpu *vcpu, +static u64 reset_bcr (struct kvm_vcpu *vcpu,const struct sys_reg_desc *rd) { vcpu->arch.vcpu_debug_state.dbg_bcr[rd->CRm] = rd->val; +return rd->val; } static bool trap_wvr(struct kvm_vcpu *vcpu,
@@ -613,10 +615,11 @@ static int get_wvr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd,return 0 ; } -static void reset_wvr (struct kvm_vcpu *vcpu, +static u64 reset_wvr (struct kvm_vcpu *vcpu,const struct sys_reg_desc *rd) { vcpu->arch.vcpu_debug_state.dbg_wvr[rd->CRm] = rd->val; +return rd->val; } static bool trap_wcr(struct kvm_vcpu *vcpu,
@@ -649,25 +652,28 @@ static int get_wcr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd,return 0 ; } -static void reset_wcr (struct kvm_vcpu *vcpu, +static u64 reset_wcr (struct kvm_vcpu *vcpu,const struct sys_reg_desc *rd) { vcpu->arch.vcpu_debug_state.dbg_wcr[rd->CRm] = rd->val; +return rd->val; } -static void reset_amair_el1 (struct kvm_vcpu *vcpu,const struct sys_reg_desc *r) +static u64 reset_amair_el1 (struct kvm_vcpu *vcpu,const struct sys_reg_desc *r) {u64 amair =read_sysreg (amair_el1);vcpu_write_sys_reg (vcpu, amair,AMAIR_EL1 ); +return amair; } -static void reset_actlr (struct kvm_vcpu *vcpu,const struct sys_reg_desc *r) +static u64 reset_actlr (struct kvm_vcpu *vcpu,const struct sys_reg_desc *r) {u64 actlr =read_sysreg (actlr_el1);vcpu_write_sys_reg (vcpu, actlr,ACTLR_EL1 ); +return actlr; } -static void reset_mpidr (struct kvm_vcpu *vcpu,const struct sys_reg_desc *r) +static u64 reset_mpidr (struct kvm_vcpu *vcpu,const struct sys_reg_desc *r) {u64 mpidr;
@@ -681,7 +687,10 @@ static void reset_mpidr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r) mpidr = (vcpu->vcpu_id &0x0f ) <<MPIDR_LEVEL_SHIFT (0 ); mpidr |= ((vcpu->vcpu_id >>4 ) &0xff ) <<MPIDR_LEVEL_SHIFT (1 ); mpidr |= ((vcpu->vcpu_id >>12 ) &0xff ) <<MPIDR_LEVEL_SHIFT (2 ); -vcpu_write_sys_reg (vcpu, (1ULL <<31 ) | mpidr,MPIDR_EL1 ); + mpidr |= (1ULL <<31 ); +vcpu_write_sys_reg (vcpu, mpidr,MPIDR_EL1 ); + +return mpidr; } static unsigned int pmu_visibility(const struct kvm_vcpu *vcpu,
@@ -693,13 +702,13 @@ static unsigned int pmu_visibility(const struct kvm_vcpu *vcpu,return REG_HIDDEN ; } -static void reset_pmu_reg (struct kvm_vcpu *vcpu,const struct sys_reg_desc *r) +static u64 reset_pmu_reg (struct kvm_vcpu *vcpu,const struct sys_reg_desc *r) {u64 n, mask =BIT (ARMV8_PMU_CYCLE_IDX );/* No PMU available, any PMU reg may UNDEF... */ if (!kvm_arm_support_pmu_v3 ()) -return ; +return 0 ; n =read_sysreg (pmcr_el0) >>ARMV8_PMU_PMCR_N_SHIFT ; n &=ARMV8_PMU_PMCR_N_MASK ;
@@ -708,33 +717,41 @@ static void reset_pmu_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r)reset_unknown (vcpu, r);__vcpu_sys_reg (vcpu, r->reg) &= mask; + +return __vcpu_sys_reg (vcpu, r->reg); } -static void reset_pmevcntr (struct kvm_vcpu *vcpu,const struct sys_reg_desc *r) +static u64 reset_pmevcntr (struct kvm_vcpu *vcpu,const struct sys_reg_desc *r) {reset_unknown (vcpu, r);__vcpu_sys_reg (vcpu, r->reg) &=GENMASK (31 ,0 ); + +return __vcpu_sys_reg (vcpu, r->reg); } -static void reset_pmevtyper (struct kvm_vcpu *vcpu,const struct sys_reg_desc *r) +static u64 reset_pmevtyper (struct kvm_vcpu *vcpu,const struct sys_reg_desc *r) {reset_unknown (vcpu, r);__vcpu_sys_reg (vcpu, r->reg) &=ARMV8_PMU_EVTYPE_MASK ; + +return __vcpu_sys_reg (vcpu, r->reg); } -static void reset_pmselr (struct kvm_vcpu *vcpu,const struct sys_reg_desc *r) +static u64 reset_pmselr (struct kvm_vcpu *vcpu,const struct sys_reg_desc *r) {reset_unknown (vcpu, r);__vcpu_sys_reg (vcpu, r->reg) &=ARMV8_PMU_COUNTER_MASK ; + +return __vcpu_sys_reg (vcpu, r->reg); } -static void reset_pmcr (struct kvm_vcpu *vcpu,const struct sys_reg_desc *r) +static u64 reset_pmcr (struct kvm_vcpu *vcpu,const struct sys_reg_desc *r) {u64 pmcr;/* No PMU available, PMCR_EL0 may UNDEF... */ if (!kvm_arm_support_pmu_v3 ()) -return ; +return 0 ;/* Only preserve PMCR_EL0.N, and reset the rest to 0 */ pmcr =read_sysreg (pmcr_el0) & (ARMV8_PMU_PMCR_N_MASK <<ARMV8_PMU_PMCR_N_SHIFT );
@@ -742,6 +759,8 @@ static void reset_pmcr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r) pmcr |=ARMV8_PMU_PMCR_LC ;__vcpu_sys_reg (vcpu, r->reg) = pmcr; + +return __vcpu_sys_reg (vcpu, r->reg); }static bool check_pmu_access_disabled (struct kvm_vcpu *vcpu,u64 flags)
@@ -1209,7 +1228,8 @@ static u8 pmuver_to_perfmon(u8 pmuver) }/* Read a sanitised cpufeature ID register by sys_reg_desc */ -static u64 read_id_reg (const struct kvm_vcpu *vcpu,struct sys_reg_desc const *r) +static u64 __kvm_read_sanitised_id_reg (const struct kvm_vcpu *vcpu, +const struct sys_reg_desc *r) {u32 id =reg_to_encoding (r);u64 val;
@@ -1280,6 +1300,17 @@ static u64 read_id_reg(const struct kvm_vcpu *vcpu, struct sys_reg_desc const *rreturn val; } +static u64 kvm_read_sanitised_id_reg (struct kvm_vcpu *vcpu, +const struct sys_reg_desc *r) +{ +return __kvm_read_sanitised_id_reg (vcpu, r); +} + +static u64 read_id_reg (const struct kvm_vcpu *vcpu,const struct sys_reg_desc *r) +{ +return __kvm_read_sanitised_id_reg (vcpu, r); +} +static unsigned int id_visibility (const struct kvm_vcpu *vcpu,const struct sys_reg_desc *r) {
@@ -1530,7 +1561,7 @@ static bool access_clidr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,* Fabricate a CLIDR_EL1 value instead of using the real value, which can vary * by the physical CPU which the vcpu currently resides in. */ -static void reset_clidr (struct kvm_vcpu *vcpu,const struct sys_reg_desc *r) +static u64 reset_clidr (struct kvm_vcpu *vcpu,const struct sys_reg_desc *r) {u64 ctr_el0 =read_sanitised_ftr_reg (SYS_CTR_EL0 );u64 clidr;
@@ -1578,6 +1609,8 @@ static void reset_clidr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r) clidr |=2 <<CLIDR_TTYPE_SHIFT (loc);__vcpu_sys_reg (vcpu, r->reg) = clidr; + +return __vcpu_sys_reg (vcpu, r->reg); } static int set_clidr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd,
@@ -1677,6 +1710,17 @@ static unsigned int elx2_visibility(const struct kvm_vcpu *vcpu, .visibility = elx2_visibility, \ } +/* +* Since reset() callback and field val are not used for idregs, they will be +* used for specific purposes for idregs. +* The reset() would return KVM sanitised register value. The value would be the +* same as the host kernel sanitised value if there is no KVM sanitisation. +* The val would be used as a mask indicating writable fields for the idreg. +* Only bits with 1 are writable from userspace. This mask might not be +* necessary in the future whenever all ID registers are enabled as writable +* from userspace. +*/ +/* sys_reg_desc initialiser for known cpufeature ID registers */ #define ID_SANITISED (name) { \ SYS_DESC(SYS_##name), \
@@ -1684,6 +1728,8 @@ static unsigned int elx2_visibility(const struct kvm_vcpu *vcpu, .get_user = get_id_reg, \ .set_user = set_id_reg, \ .visibility = id_visibility, \ + .reset = kvm_read_sanitised_id_reg, \ + .val = 0, \ }/* sys_reg_desc initialiser for known cpufeature ID registers */
@@ -1693,6 +1739,8 @@ static unsigned int elx2_visibility(const struct kvm_vcpu *vcpu, .get_user = get_id_reg, \ .set_user = set_id_reg, \ .visibility = aa32_id_visibility, \ + .reset = kvm_read_sanitised_id_reg, \ + .val = 0, \ }/*
@@ -1705,7 +1753,9 @@ static unsigned int elx2_visibility(const struct kvm_vcpu *vcpu, .access = access_id_reg, \ .get_user = get_id_reg, \ .set_user = set_id_reg, \ - .visibility = raz_visibility \ + .visibility = raz_visibility, \ + .reset = kvm_read_sanitised_id_reg, \ + .val = 0, \ }/*
@@ -1719,6 +1769,8 @@ static unsigned int elx2_visibility(const struct kvm_vcpu *vcpu, .get_user = get_id_reg, \ .set_user = set_id_reg, \ .visibility = raz_visibility, \ + .reset = kvm_read_sanitised_id_reg, \ + .val = 0, \ } static bool access_sp_el1(struct kvm_vcpu *vcpu,
@@ -3055,19 +3107,21 @@ id_to_sys_reg_desc(struct kvm_vcpu *vcpu, u64 id,*/ #define FUNCTION_INVARIANT (reg) \ - static void get_##reg(struct kvm_vcpu *v, \ + static u64 get_##reg(struct kvm_vcpu *v, \ const struct sys_reg_desc *r) \ { \ ((struct sys_reg_desc *)r)->val = read_sysreg(reg); \ + return ((struct sys_reg_desc *)r)->val; \ }FUNCTION_INVARIANT (midr_el1)FUNCTION_INVARIANT (revidr_el1 )FUNCTION_INVARIANT (aidr_el1) -static void get_ctr_el0(struct kvm_vcpu *v, const struct sys_reg_desc *r) +static u64 get_ctr_el0(struct kvm_vcpu *v, const struct sys_reg_desc *r) { ((struct sys_reg_desc *)r)->val =read_sanitised_ftr_reg (SYS_CTR_EL0 ); +return ((struct sys_reg_desc *)r)->val; }/* ->val is filled in by kvm_sys_reg_table_init() */
arch/arm64/kvm/sys_regs.h +10 -5
@@ -64,13 +64,16 @@ struct sys_reg_desc { struct sys_reg_params *, const struct sys_reg_desc *); -/* Initialization for vcpu. */ -void (*reset)(struct kvm_vcpu *,const struct sys_reg_desc *); +/* +* Initialization for vcpu. Return initialized value, or KVM +* sanitized value for ID registers. +*/ +u64 (*reset)(struct kvm_vcpu *,const struct sys_reg_desc *);/* Index into sys_reg[], or 0 if we don't need to save it. */ int reg; -/* Value (usually reset value) */ +/* Value (usually reset value), or write mask for idregs */ u64 val;/* Custom get/set_user functions, fallback to generic if NULL */
@@ -123,19 +126,21 @@ static inline bool read_zero(struct kvm_vcpu *vcpu, }/* Reset functions */ -static inline void reset_unknown (struct kvm_vcpu *vcpu, +static inline u64 reset_unknown (struct kvm_vcpu *vcpu,const struct sys_reg_desc *r) {BUG_ON (!r->reg);BUG_ON (r->reg >=NR_SYS_REGS );__vcpu_sys_reg (vcpu, r->reg) =0x1de7ec7edbadc0deULL ; +return __vcpu_sys_reg (vcpu, r->reg); } -static inline void reset_val (struct kvm_vcpu *vcpu,const struct sys_reg_desc *r) +static inline u64 reset_val (struct kvm_vcpu *vcpu,const struct sys_reg_desc *r) {BUG_ON (!r->reg);BUG_ON (r->reg >=NR_SYS_REGS );__vcpu_sys_reg (vcpu, r->reg) = r->val; +return __vcpu_sys_reg (vcpu, r->reg); } static inline unsigned int sysreg_visibility(const struct kvm_vcpu *vcpu,
Signature · 1 line
2.41.0.162.gfafddb0af9-goog
PATCH v12 6/11 KVM: arm64: Save ID registers' sanitized value per guest
In reply to Oliver UptonFrom: Jing Zhang <[email protected]>
Initialize the default ID register values upon the first call to KVM_ARM_VCPU_INIT. The vCPU feature flags are finalized at that point, so it is possible to determine the maximum feature set supported by a particular VM configuration. Do nothing with these values for now, as we need to rework the plumbing of what's already writable to be compatible with the generic infrastructure.
[Oliver: Hoist everything into KVM_ARM_VCPU_INIT time, so the features
are final]
--- arch/arm64/include/asm/kvm_host.h | 15 +++++++++ arch/arm64/kvm/sys_regs.c | 56 +++++++++++++++++++++++++++++-- arch/arm64/kvm/sys_regs.h | 7 ++++ 3 files changed, 75 insertions(+), 3 deletions(-)
arch/arm64/include/asm/kvm_host.h +15 -0
@@ -225,6 +225,8 @@ struct kvm_arch {#define KVM_ARCH_FLAG_TIMER_PPIS_IMMUTABLE 6/* SMCCC filter initialized for the VM */ #define KVM_ARCH_FLAG_SMCCC_FILTER_CONFIGURED 7 +/* Initial ID reg values loaded */ +#define KVM_ARCH_FLAG_ID_REGS_INITIALIZED 8unsigned long flags;/* VM-wide vCPU feature set */
@@ -247,6 +249,19 @@ struct kvm_arch {struct kvm_smccc_features smccc_feat;struct maple_tree smccc_filter; +/* +* Emulated CPU ID registers per VM +* (Op0, Op1, CRn, CRm, Op2) of the ID registers to be saved in it +* is (3, 0, 0, crm, op2), where 1<=crm<8, 0<=op2<8. +* +* These emulated idregs are VM-wide, but accessed from the context of a vCPU. +* Atomic access to multiple idregs are guarded by kvm_arch.config_lock. +*/ +#define IDREG_IDX (id) (((sys_reg_CRm(id) - 1) << 3) | sys_reg_Op2(id)) +#define IDREG (kvm, id) ((kvm)->arch.id_regs[IDREG_IDX(id)]) +#define KVM_ARM_ID_REG_NUM (IDREG_IDX(sys_reg(3, 0, 0, 7, 7)) + 1) +u64 id_regs[KVM_ARM_ID_REG_NUM ]; +/* * For an untrusted host VM, 'pkvm.handle' is used to lookup * the associated pKVM instance in the hypervisor.
arch/arm64/kvm/sys_regs.c +53 -3
91 lines not on this page · raw message
arch/arm64/kvm/sys_regs.h +7 -0
@@ -27,6 +27,13 @@ struct sys_reg_params {bool is_write; }; +#define encoding_to_params (reg) \ + ((struct sys_reg_params){ .Op0 = sys_reg_Op0(reg), \ + .Op1 = sys_reg_Op1(reg), \ + .CRn = sys_reg_CRn(reg), \ + .CRm = sys_reg_CRm(reg), \ + .Op2 = sys_reg_Op2(reg) }) +#define esr_sys64_to_params (esr) \ ((struct sys_reg_params){ .Op0 = ((esr) >> 20) & 3, \ .Op1 = ((esr) >> 14) & 0x7, \
Signature · 1 line
2.41.0.162.gfafddb0af9-goog
PATCH v12 7/11 KVM: arm64: Use arm64_ftr_bits to sanitise ID register writes
In reply to Oliver UptonFrom: Jing Zhang <[email protected]>
Rather than reinventing the wheel in KVM to do ID register sanitisation we can rely on the work already done in the core kernel. Implement a generalized sanitisation of ID registers based on the combination of the arm64_ftr_bits definitions from the core kernel and (optionally) a set of KVM-specific overrides.
This all amounts to absolutely nothing for now, but will be used in subsequent changes to realize user-configurable ID registers.
[Oliver: split off from monster patch, rewrote commit description,
reworked RAZ handling]
--- arch/arm64/include/asm/cpufeature.h | 1 + arch/arm64/kernel/cpufeature.c | 2 +- arch/arm64/kvm/sys_regs.c | 113 +++++++++++++++++++++++++++- 3 files changed, 111 insertions(+), 5 deletions(-)
arch/arm64/include/asm/cpufeature.h +1 -0
@@ -915,6 +915,7 @@ static inline unsigned int get_vmid_bits(u64 mmfr1)return 8 ; } +s64 arm64_ftr_safe_value (const struct arm64_ftr_bits *ftrp,s64 new,s64 cur);struct arm64_ftr_reg *get_arm64_ftr_reg (u32 sys_id);extern struct arm64_ftr_override id_aa64mmfr1_override;
arch/arm64/kernel/cpufeature.c +1 -1
@@ -798,7 +798,7 @@ static u64 arm64_ftr_set_value(const struct arm64_ftr_bits *ftrp, s64 reg,return reg; } -static s64 arm64_ftr_safe_value (const struct arm64_ftr_bits *ftrp,s64 new, +s64 arm64_ftr_safe_value (const struct arm64_ftr_bits *ftrp,s64 new,s64 cur) {s64 ret =0 ;
arch/arm64/kvm/sys_regs.c +109 -4
126 lines not on this page · raw message
Signature · 1 line
2.41.0.162.gfafddb0af9-goog
Marc Zyngier
maintainerRe: [PATCH v12 07/11] KVM: arm64: Use arm64_ftr_bits to sanitise ID register writes
Hi Oliver,
Quoted 95 lines · Oliver Upton
From: Jing Zhang <[email protected]>
Rather than reinventing the wheel in KVM to do ID register sanitisation we can rely on the work already done in the core kernel. Implement a generalized sanitisation of ID registers based on the combination of the
arm64_ftr_bitsdefinitions from the core kernel and (optionally) a set of KVM-specific overrides.This all amounts to absolutely nothing for now, but will be used in subsequent changes to realize user-configurable ID registers.
Signed-off-by Jing Zhang <[email protected]>[Oliver: split off from monster patch, rewrote commit description,
reworked RAZ handling]Signed-off-by Oliver Upton <[email protected]>--- arch/arm64/include/asm/cpufeature.h | 1 + arch/arm64/kernel/cpufeature.c | 2 +- arch/arm64/kvm/sys_regs.c | 113 +++++++++++++++++++++++++++- 3 files changed, 111 insertions(+), 5 deletions(-)arch/arm64/include/asm/cpufeature.h +1
-0@@ -915,6 +915,7 @@ static inline unsigned int get_vmid_bits(u64 mmfr1)return 8 ; } +s64 arm64_ftr_safe_value (const struct arm64_ftr_bits *ftrp,s64 new,s64 cur);struct arm64_ftr_reg *get_arm64_ftr_reg (u32 sys_id);extern struct arm64_ftr_override id_aa64mmfr1_override;arch/arm64/kernel/cpufeature.c +1
-1@@ -798,7 +798,7 @@ static u64 arm64_ftr_set_value(const struct arm64_ftr_bits *ftrp, s64 reg,return reg; } -static s64 arm64_ftr_safe_value (const struct arm64_ftr_bits *ftrp,s64 new, +s64 arm64_ftr_safe_value (const struct arm64_ftr_bits *ftrp,s64 new,s64 cur) {s64 ret =0 ;arch/arm64/kvm/sys_regs.c +40
-0@@ -1201,6 +1201,91 @@ static u8 vcpu_pmuver(const struct kvm_vcpu *vcpu)return 0 ; } +static s64 kvm_arm64_ftr_safe_value (u32 id,const struct arm64_ftr_bits *ftrp, +s64 new,s64 cur) +{ +struct arm64_ftr_bits kvm_ftr = *ftrp; + +/* Some features have different safe value type in KVM than host features */ +switch (id) { +case SYS_ID_AA64DFR0_EL1 : +if (kvm_ftr.shift == ID_AA64DFR0_EL1_PMUVer_SHIFT) + kvm_ftr.type =FTR_LOWER_SAFE ; +break ; +case SYS_ID_DFR0_EL1 : +if (kvm_ftr.shift == ID_DFR0_EL1_PerfMon_SHIFT) + kvm_ftr.type =FTR_LOWER_SAFE ; +break ; + } + +return arm64_ftr_safe_value (&kvm_ftr, new, cur); +} + +/** +* arm64_check_features() - Check if a feature register value constitutes +* a subset of features indicated by the idreg's KVM sanitised limit. +* +* This function will check if each feature field of @val is the "safe" value +* against idreg's KVM sanitised limit return from reset() callback. +* If a field value in @val is the same as the one in limit, it is always +* considered the safe value regardless For register fields that are not in +* writable, only the value in limit is considered the safe value. +* +* Return: 0 if all the fields are safe. Otherwise, return negative errno. +*/ +static int arm64_check_features (struct kvm_vcpu *vcpu, +const struct sys_reg_desc *rd, +u64 val) +{ +const struct arm64_ftr_reg *ftr_reg; +const struct arm64_ftr_bits *ftrp =NULL ; +u32 id =reg_to_encoding (rd); +u64 writable_mask = rd->val;
arch/arm64/kvm/sys_regs.c static u8 vcpu_pmuver(const struct kvm_vcpu *vcpu) · line 1253
Quoted 10 lines
arch/arm64/kvm/sys_regs.c +10
-0+u64 limit = rd->reset (vcpu, rd); +u64 mask =0 ; + +/* +* Hidden and unallocated ID registers may not have a corresponding +* struct arm64_ftr_reg. Of course, if the register is RAZ we know the +* only safe value is 0. +*/ +if (sysreg_visible_as_raz (vcpu, rd)) +return val ? -E2BIG :0 ;
This -E2BIG is certainly reflecting the error here. However...
Quoted 61 lines
arch/arm64/kvm/sys_regs.c +49
-4+ + ftr_reg =get_arm64_ftr_reg (id); +if (!ftr_reg) +return -EINVAL ; + + ftrp = ftr_reg->ftr_bits; + +for (; ftrp && ftrp->width; ftrp++) { +s64 f_val, f_lim, safe_val; +u64 ftr_mask; + + ftr_mask =arm64_ftr_mask (ftrp); +if ((ftr_mask & writable_mask) != ftr_mask) +continue ; + + f_val =arm64_ftr_value (ftrp, val); + f_lim =arm64_ftr_value (ftrp, limit); + mask |= ftr_mask; + +if (f_val == f_lim) + safe_val = f_val; +else + safe_val =kvm_arm64_ftr_safe_value (id, ftrp, f_val, f_lim); + +if (safe_val != f_val) +return -E2BIG ; + } + +/* For fields that are not writable, values in limit are the safe values. */ +if ((val & ~mask) != (limit & ~mask)) +return -E2BIG ; + +return 0 ; +} +static u8 perfmon_to_pmuver (u8 perfmon) {switch (perfmon) {@@ -1528,11 +1613,31 @@ static int get_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd,static int set_id_reg (struct kvm_vcpu *vcpu,const struct sys_reg_desc *rd,u64 val) { -/* This is what we mean by invariant: you can't change it. */ -if (val !=read_id_reg (vcpu, rd)) -return -EINVAL ; +u32 id =reg_to_encoding (rd); +int ret; -return 0 ; +mutex_lock (&vcpu->kvm->arch.config_lock); + +/* +* Once the VM has started the ID registers are immutable. Reject any +* write that does not match the final register value. +*/ +if (kvm_vm_has_ran_once (vcpu->kvm)) { +if (val !=read_id_reg (vcpu, rd)) + ret = -EBUSY ; +else + ret =0 ; +
arch/arm64/kvm/sys_regs.c static int get_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd, · line 1640
Quoted 10 lines
arch/arm64/kvm/sys_regs.c +10
-0+mutex_unlock (&vcpu->kvm->arch.config_lock); +return ret; + } + + ret =arm64_check_features (vcpu, rd, val); +if (!ret) +IDREG (vcpu->kvm, id) = val; + +mutex_unlock (&vcpu->kvm->arch.config_lock); +return ret;
... we now end-up with a *new* error code that userspace was never able to see so far.
This may not be a big deal, but I'd rather err on the side of caution by keeping the current, slightly less precise error code.
Thoughts?
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Re: [PATCH v12 07/11] KVM: arm64: Use arm64_ftr_bits to sanitise ID register writes
Hey Marc,
Quoted 37 lines · Marc Zyngier
Quoted 23 lines
arch/arm64/kvm/sys_regs.c +14
-4@@ -1528,11 +1613,31 @@ static int get_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd,static int set_id_reg (struct kvm_vcpu *vcpu,const struct sys_reg_desc *rd,u64 val) { -/* This is what we mean by invariant: you can't change it. */ -if (val !=read_id_reg (vcpu, rd)) -return -EINVAL ; +u32 id =reg_to_encoding (rd); +int ret; -return 0 ; +mutex_lock (&vcpu->kvm->arch.config_lock); + +/* +* Once the VM has started the ID registers are immutable. Reject any +* write that does not match the final register value. +*/ +if (kvm_vm_has_ran_once (vcpu->kvm)) { +if (val !=read_id_reg (vcpu, rd)) + ret = -EBUSY ; +else + ret =0 ; +arch/arm64/kvm/sys_regs.c static int get_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd, · line 1640
Quoted 10 lines
arch/arm64/kvm/sys_regs.c +10
-0+mutex_unlock (&vcpu->kvm->arch.config_lock); +return ret; + } + + ret =arm64_check_features (vcpu, rd, val); +if (!ret) +IDREG (vcpu->kvm, id) = val; + +mutex_unlock (&vcpu->kvm->arch.config_lock); +return ret;... we now end-up with a *new* error code that userspace was never able to see so far.
Quoted 2 lines
This may not be a big deal, but I'd rather err on the side of caution by keeping the current, slightly less precise error code.
I completely agree, thanks for catching this. There's already enough deliberate (theorectical) breakage brought about by this series, want to avoid any unintended fallout :)
I plan on taking this, and I'll apply a fix on top to dumb down the return.
Signature · 2 lines
Thanks, Oliver
PATCH v12 8/11 KVM: arm64: Use generic sanitisation for ID_(AA64)DFR0_EL1
In reply to Oliver UptonFrom: Jing Zhang <[email protected]>
KVM allows userspace to specify a PMU version for the guest by writing to the corresponding ID registers. Currently the validation of these writes is done manuallly, but there's no reason we can't switch over to the generic sanitisation infrastructure.
Start screening user writes through arm64_check_features() to prevent userspace from over-promising in terms of vPMU support. Leave the old masking in place for now, as we aren't completely ready to serve reads from the VM-wide values.
[Oliver: split off from monster patch, cleaned up handling of NI vPMU
values, wrote commit description]
--- arch/arm64/kvm/sys_regs.c | 106 ++++++++++++++++++++++---------------- 1 file changed, 61 insertions(+), 45 deletions(-)
arch/arm64/kvm/sys_regs.c +61 -45
@@ -42,6 +42,8 @@*/ static u64 sys_reg_to_index (const struct sys_reg_desc *reg); +static int set_id_reg (struct kvm_vcpu *vcpu,const struct sys_reg_desc *rd, +u64 val); static bool read_from_write_only(struct kvm_vcpu *vcpu, struct sys_reg_params *params,
@@ -1503,15 +1505,35 @@ static int set_id_aa64pfr0_el1(struct kvm_vcpu *vcpu,return 0 ; } +static u64 read_sanitised_id_aa64dfr0_el1 (struct kvm_vcpu *vcpu, +const struct sys_reg_desc *rd) +{ +u64 val =read_sanitised_ftr_reg (SYS_ID_AA64DFR0_EL1 ); + +/* Limit debug to ARMv8.0 */ + val &= ~ID_AA64DFR0_EL1_DebugVer_MASK; + val |=SYS_FIELD_PREP_ENUM (ID_AA64DFR0_EL1 , DebugVer,IMP ); + +/* +* Only initialize the PMU version if the vCPU was configured with one. +*/ + val &= ~ID_AA64DFR0_EL1_PMUVer_MASK; +if (kvm_vcpu_has_pmu (vcpu)) + val |=SYS_FIELD_PREP (ID_AA64DFR0_EL1 , PMUVer, +kvm_arm_pmu_get_pmuver_limit ()); + +/* Hide SPE from guests */ + val &= ~ID_AA64DFR0_EL1_PMSVer_MASK; + +return val; +} +static int set_id_aa64dfr0_el1 (struct kvm_vcpu *vcpu,const struct sys_reg_desc *rd,u64 val) { -u8 pmuver, host_pmuver; -bool valid_pmu; - - host_pmuver =kvm_arm_pmu_get_pmuver_limit (); - pmuver =SYS_FIELD_GET (ID_AA64DFR0_EL1 , PMUVer, val); +u8 pmuver =SYS_FIELD_GET (ID_AA64DFR0_EL1 , PMUVer, val); +int r;/* * Prior to commit 3d0dba5764b9 ("KVM: arm64: PMU: Move the
@@ -1532,38 +1554,33 @@ static int set_id_aa64dfr0_el1(struct kvm_vcpu *vcpu, pmuver =0 ; } -/* -* Allow AA64DFR0_EL1.PMUver to be set from userspace as long -* as it doesn't promise more than what the HW gives us. -*/ -if (pmuver > host_pmuver) -return -EINVAL ; + r =set_id_reg (vcpu, rd, val); +if (r) +return r; - valid_pmu = pmuver; + vcpu->kvm->arch.dfr0_pmuver = pmuver; +return 0 ; +} -/* Make sure view register and PMU support do match */ -if (kvm_vcpu_has_pmu (vcpu) != valid_pmu) -return -EINVAL ; +static u64 read_sanitised_id_dfr0_el1 (struct kvm_vcpu *vcpu, +const struct sys_reg_desc *rd) +{ +u8 perfmon =pmuver_to_perfmon (kvm_arm_pmu_get_pmuver_limit ()); +u64 val =read_sanitised_ftr_reg (SYS_ID_DFR0_EL1 ); -/* We can only differ with PMUver, and anything else is an error */ - val ^=read_id_reg (vcpu, rd); - val &= ~ARM64_FEATURE_MASK (ID_AA64DFR0_EL1_PMUVer); -if (val) -return -EINVAL ; + val &= ~ID_DFR0_EL1_PerfMon_MASK; +if (kvm_vcpu_has_pmu (vcpu)) + val |=SYS_FIELD_PREP (ID_DFR0_EL1 , PerfMon, perfmon); - vcpu->kvm->arch.dfr0_pmuver = pmuver; -return 0 ; +return val; }static int set_id_dfr0_el1 (struct kvm_vcpu *vcpu,const struct sys_reg_desc *rd,u64 val) { -u8 perfmon, host_perfmon; -bool valid_pmu; - - host_perfmon =pmuver_to_perfmon (kvm_arm_pmu_get_pmuver_limit ()); - perfmon =SYS_FIELD_GET (ID_DFR0_EL1 , PerfMon, val); +u8 perfmon =SYS_FIELD_GET (ID_DFR0_EL1 , PerfMon, val); +int r;if (perfmon == ID_DFR0_EL1_PerfMon_IMPDEF) { val &= ~ID_DFR0_EL1_PerfMon_MASK;
@@ -1576,21 +1593,12 @@ static int set_id_dfr0_el1(struct kvm_vcpu *vcpu,* AArch64 side (as everything is emulated with that), and * that this is a PMUv3. */ -if (perfmon > host_perfmon || - (perfmon !=0 && perfmon < ID_DFR0_EL1_PerfMon_PMUv3)) -return -EINVAL ; - - valid_pmu = perfmon; - -/* Make sure view register and PMU support do match */ -if (kvm_vcpu_has_pmu (vcpu) != valid_pmu) +if (perfmon !=0 && perfmon < ID_DFR0_EL1_PerfMon_PMUv3)return -EINVAL ; -/* We can only differ with PerfMon, and anything else is an error */ - val ^=read_id_reg (vcpu, rd); - val &= ~ARM64_FEATURE_MASK (ID_DFR0_EL1_PerfMon); -if (val) -return -EINVAL ; + r =set_id_reg (vcpu, rd, val); +if (r) +return r; vcpu->kvm->arch.dfr0_pmuver =perfmon_to_pmuver (perfmon);return 0 ;
@@ -1988,9 +1996,13 @@ static const struct sys_reg_desc sys_reg_descs[] = {/* CRm=1 */ AA32_ID_SANITISED (ID_PFR0_EL1 ),AA32_ID_SANITISED (ID_PFR1_EL1 ), - { SYS_DESC(SYS_ID_DFR0_EL1), .access = access_id_reg, - .get_user = get_id_reg, .set_user = set_id_dfr0_el1, - .visibility = aa32_id_visibility, }, + { SYS_DESC(SYS_ID_DFR0_EL1), + .access = access_id_reg, + .get_user = get_id_reg, + .set_user = set_id_dfr0_el1, + .visibility = aa32_id_visibility, + .reset = read_sanitised_id_dfr0_el1, + .val = ID_DFR0_EL1_PerfMon_MASK, },ID_HIDDEN (ID_AFR0_EL1 ),AA32_ID_SANITISED (ID_MMFR0_EL1 ), AA32_ID_SANITISED(ID_MMFR1_EL1),
@@ -2030,8 +2042,12 @@ static const struct sys_reg_desc sys_reg_descs[] = {ID_UNALLOCATED (4 ,7 ),/* CRm=5 */ - { SYS_DESC(SYS_ID_AA64DFR0_EL1), .access = access_id_reg, - .get_user = get_id_reg, .set_user = set_id_aa64dfr0_el1, }, + { SYS_DESC(SYS_ID_AA64DFR0_EL1), + .access = access_id_reg, + .get_user = get_id_reg, + .set_user = set_id_aa64dfr0_el1, + .reset = read_sanitised_id_aa64dfr0_el1, + .val = ID_AA64DFR0_EL1_PMUVer_MASK, },ID_SANITISED (ID_AA64DFR1_EL1 ),ID_UNALLOCATED (5 ,2 ), ID_UNALLOCATED(5,3),
Signature · 1 line
2.41.0.162.gfafddb0af9-goog
PATCH v12 9/11 KVM: arm64: Use generic sanitisation for ID_AA64PFR0_EL1
In reply to Oliver UptonFrom: Jing Zhang <[email protected]>
KVM allows userspace to write to the CSV2 and CSV3 fields of ID_AA64PFR0_EL1 so long as it doesn't over-promise on the Spectre/Meltdown mitigation state. Switch over to the new way of the world for screening user writes. Leave the old plumbing in place until we actually start handling ID register reads from the VM-wide values.
--- arch/arm64/kvm/sys_regs.c | 66 ++++++++++++++++++++++++++------------- 1 file changed, 45 insertions(+), 21 deletions(-)
arch/arm64/kvm/sys_regs.c +45 -21
@@ -1470,34 +1470,54 @@ static unsigned int sve_visibility(const struct kvm_vcpu *vcpu,return REG_HIDDEN ; } +static u64 read_sanitised_id_aa64pfr0_el1 (struct kvm_vcpu *vcpu, +const struct sys_reg_desc *rd) +{ +u64 val =read_sanitised_ftr_reg (SYS_ID_AA64PFR0_EL1 ); + +if (!vcpu_has_sve (vcpu)) + val &= ~ID_AA64PFR0_EL1_SVE_MASK ; + +/* +* The default is to expose CSV2 == 1 if the HW isn't affected. +* Although this is a per-CPU feature, we make it global because +* asymmetric systems are just a nuisance. +* +* Userspace can override this as long as it doesn't promise +* the impossible. +*/ +if (arm64_get_spectre_v2_state () ==SPECTRE_UNAFFECTED ) { + val &= ~ID_AA64PFR0_EL1_CSV2_MASK ; + val |=SYS_FIELD_PREP_ENUM (ID_AA64PFR0_EL1 ,CSV2 ,IMP ); + } +if (arm64_get_meltdown_state () ==SPECTRE_UNAFFECTED ) { + val &= ~ID_AA64PFR0_EL1_CSV3_MASK ; + val |=SYS_FIELD_PREP_ENUM (ID_AA64PFR0_EL1 ,CSV3 ,IMP ); + } + +if (kvm_vgic_global_state.type ==VGIC_V3 ) { + val &= ~ID_AA64PFR0_EL1_GIC_MASK ; + val |=SYS_FIELD_PREP_ENUM (ID_AA64PFR0_EL1 ,GIC ,IMP ); + } + + val &= ~ID_AA64PFR0_EL1_AMU_MASK ; + +return val; +} +static int set_id_aa64pfr0_el1 (struct kvm_vcpu *vcpu,const struct sys_reg_desc *rd,u64 val) {u8 csv2, csv3; +int r; -/* -* Allow AA64PFR0_EL1.CSV2 to be set from userspace as long as -* it doesn't promise more than what is actually provided (the -* guest could otherwise be covered in ectoplasmic residue). -*/ csv2 =cpuid_feature_extract_unsigned_field (val,ID_AA64PFR0_EL1_CSV2_SHIFT ); -if (csv2 >1 || - (csv2 &&arm64_get_spectre_v2_state () !=SPECTRE_UNAFFECTED )) -return -EINVAL ; - -/* Same thing for CSV3 */ csv3 =cpuid_feature_extract_unsigned_field (val,ID_AA64PFR0_EL1_CSV3_SHIFT ); -if (csv3 >1 || - (csv3 &&arm64_get_meltdown_state () !=SPECTRE_UNAFFECTED )) -return -EINVAL ; -/* We can only differ with CSV[23], and anything else is an error */ - val ^=read_id_reg (vcpu, rd); - val &= ~(ARM64_FEATURE_MASK (ID_AA64PFR0_EL1_CSV2 ) | -ARM64_FEATURE_MASK (ID_AA64PFR0_EL1_CSV3 )); -if (val) -return -EINVAL ; + r =set_id_reg (vcpu, rd, val); +if (r) +return r; vcpu->kvm->arch.pfr0_csv2 = csv2; vcpu->kvm->arch.pfr0_csv3 = csv3;
@@ -2031,8 +2051,12 @@ static const struct sys_reg_desc sys_reg_descs[] = {/* AArch64 ID registers */ /* CRm=4 */ - { SYS_DESC(SYS_ID_AA64PFR0_EL1), .access = access_id_reg, - .get_user = get_id_reg, .set_user = set_id_aa64pfr0_el1, }, + { SYS_DESC(SYS_ID_AA64PFR0_EL1), + .access = access_id_reg, + .get_user = get_id_reg, + .set_user = set_id_aa64pfr0_el1, + .reset = read_sanitised_id_aa64pfr0_el1, + .val = ID_AA64PFR0_EL1_CSV2_MASK | ID_AA64PFR0_EL1_CSV3_MASK, },ID_SANITISED (ID_AA64PFR1_EL1 ),ID_UNALLOCATED (4 ,2 ), ID_UNALLOCATED(4,3),
Signature · 1 line
2.41.0.162.gfafddb0af9-goog
PATCH v12 10/11 KVM: arm64: Handle ID register reads using the VM-wide values
In reply to Oliver UptonEverything is in place now to use the generic ID register infrastructure. Use the VM-wide values to service ID register reads. The ID registers are invariant after the VM has started, so there is no need for locking in that case. This is rather desirable for VM live migration, as the needless lock contention could prolong the VM blackout period.
--- arch/arm64/kvm/sys_regs.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-)
arch/arm64/kvm/sys_regs.c +13 -1
@@ -1395,7 +1395,7 @@ static u64 kvm_read_sanitised_id_reg(struct kvm_vcpu *vcpu,static u64 read_id_reg (const struct kvm_vcpu *vcpu,const struct sys_reg_desc *r) { -return __kvm_read_sanitised_id_reg (vcpu, r); +return IDREG (vcpu->kvm,reg_to_encoding (r)); }/*
@@ -1634,7 +1634,19 @@ static int set_id_dfr0_el1(struct kvm_vcpu *vcpu,static int get_id_reg (struct kvm_vcpu *vcpu,const struct sys_reg_desc *rd,u64 *val) { +/* +* Avoid locking if the VM has already started, as the ID registers are +* guaranteed to be invariant at that point. +*/ +if (kvm_vm_has_ran_once (vcpu->kvm)) { + *val =read_id_reg (vcpu, rd); +return 0 ; + } + +mutex_lock (&vcpu->kvm->arch.config_lock); *val =read_id_reg (vcpu, rd); +mutex_unlock (&vcpu->kvm->arch.config_lock); +return 0 ; }
Signature · 1 line
2.41.0.162.gfafddb0af9-goog
PATCH v12 11/11 KVM: arm64: Rip out the vestiges of the 'old' ID register scheme
In reply to Oliver UptonThere's no longer a need for the baggage of the old scheme for handling configurable ID register fields. Rip it all out in favor of the generalized infrastructure.
--- arch/arm64/include/asm/kvm_host.h | 4 -- arch/arm64/kvm/arm.c | 23 -------- arch/arm64/kvm/sys_regs.c | 92 ++----------------------------- include/kvm/arm_pmu.h | 8 ++- 4 files changed, 10 insertions(+), 117 deletions(-)
arch/arm64/include/asm/kvm_host.h +0 -4
@@ -241,10 +241,6 @@ struct kvm_arch {cpumask_var_t supported_cpus; -u8 pfr0_csv2; -u8 pfr0_csv3; -u8 dfr0_pmuver; -/* Hypercall features firmware registers' descriptor */ struct kvm_smccc_features smccc_feat;struct maple_tree smccc_filter;
arch/arm64/kvm/arm.c +0 -23
@@ -102,22 +102,6 @@ static int kvm_arm_default_max_vcpus(void)return vgic_present ?kvm_vgic_get_max_vcpus () :KVM_MAX_VCPUS ; } -static void set_default_spectre (struct kvm *kvm) -{ -/* -* The default is to expose CSV2 == 1 if the HW isn't affected. -* Although this is a per-CPU feature, we make it global because -* asymmetric systems are just a nuisance. -* -* Userspace can override this as long as it doesn't promise -* the impossible. -*/ -if (arm64_get_spectre_v2_state () ==SPECTRE_UNAFFECTED ) - kvm->arch.pfr0_csv2 =1 ; -if (arm64_get_meltdown_state () ==SPECTRE_UNAFFECTED ) - kvm->arch.pfr0_csv3 =1 ; -} -/** * kvm_arch_init_vm - initializes a VM data structure * @kvm: pointer to the KVM struct
@@ -161,15 +145,8 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)/* The maximum number of VCPUs is limited by the host's GIC model */ kvm->max_vcpus =kvm_arm_default_max_vcpus (); -set_default_spectre (kvm);kvm_arm_init_hypercalls (kvm); -/* -* Initialise the default PMUver before there is a chance to -* create an actual PMU. -*/ - kvm->arch.dfr0_pmuver =kvm_arm_pmu_get_pmuver_limit (); -bitmap_zero (kvm->arch.vcpu_features,KVM_VCPU_MAX_FEATURES );return 0 ;
arch/arm64/kvm/sys_regs.c +4 -88
154 lines not on this page · raw message
include/kvm/arm_pmu.h +6 -2
@@ -92,8 +92,12 @@ void kvm_vcpu_pmu_restore_host(struct kvm_vcpu *vcpu);/* * Evaluates as true when emulating PMUv3p5, and false otherwise. */ -#define kvm_pmu_is_3p5 (vcpu) \ - (vcpu->kvm->arch.dfr0_pmuver >= ID_AA64DFR0_EL1_PMUVer_V3P5) +#define kvm_pmu_is_3p5 (vcpu) ({ \ + u64 val = IDREG(vcpu->kvm, SYS_ID_AA64DFR0_EL1); \ + u8 pmuver = SYS_FIELD_GET(ID_AA64DFR0_EL1, PMUVer, val); \ + \ + pmuver >= ID_AA64DFR0_EL1_PMUVer_V3P5; \ +})u8 kvm_arm_pmu_get_pmuver_limit (void );
Signature · 1 line
2.41.0.162.gfafddb0af9-goog
Hey,
For the new arm64 folks I've added to the conversation, apologies for the rather vague cover letter title. I kept the original author's title, but this is solely focused on KVM support for changing guest ID register values.
As always, feedback appreciated on the whole series, but the most interesting interaction comes from patch 7 where KVM leverages some of the cpufeature infrastructure for ID register validation.
Signature · 2 lines
Thanks, Oliver
Quoted 5 lines · Oliver Upton
Hello folks,
In the interest of expediting the acceptance of the generalized ID register infrastructure, I have taken Jing's series and applied fixes according to the feedback of reviewers and my own interpretation.
[...]
Oliver Upton
AppliedQuoted 11 lines · Oliver Upton
In the interest of expediting the acceptance of the generalized ID register infrastructure, I have taken Jing's series and applied fixes according to the feedback of reviewers and my own interpretation.
The laundry list:
- Split and reorganize the changes to avoid introducing intermediate regressions in the series. Don't want to break bisection! Sadly, this requires introducing a bit of churn but to me is an acceptable tradeoff.[...]
Applied to kvmarm/next, thanks!
[01/11] KVM: arm64: Separate out feature sanitisation and initialisation
https://git.kernel.org/kvmarm/kvmarm/c/a7a2c72ae014 [02/11] KVM: arm64: Relax invariance of KVM_ARM_VCPU_POWER_OFF https://git.kernel.org/kvmarm/kvmarm/c/e3c1c0cae31e [03/11] KVM: arm64: Make vCPU feature flags consistent VM-wide https://git.kernel.org/kvmarm/kvmarm/c/2251e9ff1573 [04/11] KVM: arm64: Rewrite IMPDEF PMU version as NI https://git.kernel.org/kvmarm/kvmarm/c/f90f9360c3d7 [05/11] KVM: arm64: Reuse fields of sys_reg_desc for idreg https://git.kernel.org/kvmarm/kvmarm/c/d86cde6e335f [06/11] KVM: arm64: Save ID registers' sanitized value per guest https://git.kernel.org/kvmarm/kvmarm/c/473341469042 [07/11] KVM: arm64: Use arm64_ftr_bits to sanitise ID register writes https://git.kernel.org/kvmarm/kvmarm/c/2e8bf0cbd058 [08/11] KVM: arm64: Use generic sanitisation for ID_(AA64)DFR0_EL1 https://git.kernel.org/kvmarm/kvmarm/c/c118cead07a7 [09/11] KVM: arm64: Use generic sanitisation for ID_AA64PFR0_EL1 https://git.kernel.org/kvmarm/kvmarm/c/c39f5974d38f [10/11] KVM: arm64: Handle ID register reads using the VM-wide values https://git.kernel.org/kvmarm/kvmarm/c/6db7af0d5b2b [11/11] KVM: arm64: Rip out the vestiges of the 'old' ID register scheme https://git.kernel.org/kvmarm/kvmarm/c/686672407e6e
--
Best,
Oliver
Quoted 6 lines · Oliver Upton
Hello folks,
In the interest of expediting the acceptance of the generalized ID register infrastructure, I have taken Jing's series and applied fixes according to the feedback of reviewers and my own interpretation.
With the small nit I mentioned earlier:
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.