KVM: arm64: mixed-width check should be skipped for uninitialized vCPUs
v5 cover KVM: arm64: mixed-width check should be skipped for uninitialized vCPUs
KVM allows userspace to configure either all EL1 32bit or 64bit vCPUs for a guest. At vCPU reset, vcpu_allowed_register_width() checks if the vcpu's register width is consistent with all other vCPUs'. Since the checking is done even against vCPUs that are not initialized (KVM_ARM_VCPU_INIT has not been done) yet, the uninitialized vCPUs are erroneously treated as 64bit vCPU, which causes the function to incorrectly detect a mixed-width VM.
This series will fix this problem by introducing a new VM flag that indicates the guest needs to be configured with all 32bit or 64bit vCPUs and checking vcpu's register width against the new flag at the vcpu's KVM_ARM_VCPU_INIT (instead of against other vCPUs' register width).
Patch-1 introduces KVM_ARCH_FLAG_EL1_32BIT and KVM_ARCH_FLAG_REG_WIDTH_CONFIGURED bits for kvm->arch.flags and uses them to check vcpu's register width to fix the problem.
Patch-2 introduces a selftest that can test non-mixed-width vCPUs (all 64bit vCPUs or all 32bit vcPUs) can be configured, and mixed-width vCPUs cannot be configured.
The series is based on kvmarm/next's at tag: kvmarm-5.18.
v5:
- Rebase to kvmarm/next (and drop the patch-1 "KVM: arm64: Generalise VM features into a set of flags")
- Use kernel-doc style comments for kvm_set_vm_width() [Oliver]
- Change kvm_set_vm_width() to use if/else instead of a ternary operator for KVM_ARCH_FLAG_EL1_32BIT check [Oliver]
v4: https://lore.kernel.org/all/[email protected]/
- Use different implementation of vcpu_el1_is_32bit() depending on the context. [Marc]
- Rename kvm_register_width_check_or_init() to kvm_set_vm_width(), and call it from kvm_rest_vcpu() instead of from kvm_vcpu_set_target()
- Remove vcpu_allowed_register_width(), and does the same checking in kvm_set_vm_width() instead.
v3: https://lore.kernel.org/all/[email protected]/
- Introduced 'flags' to kvm_arch, and use bits of the flags for a set of booleans for VM feature.
- Changed 'el1_reg_width' to two bits of 'flags' of kvm_arch.
v2: https://lore.kernel.org/all/[email protected]/
- Introduced 'el1_reg_width' for kvm_arch and use it to check vcpu's register width against the flag at the vcpu's KVM_ARM_VCPU_INIT.
v1: https://lore.kernel.org/all/[email protected]/
[1] https://lore.kernel.org/all/[email protected]/
Reiji Watanabe (2):
KVM: arm64: mixed-width check should be skipped for uninitialized
vCPUs
KVM: arm64: selftests: Introduce vcpu_width_config
arch/arm64/include/asm/kvm_emulate.h | 27 ++-- arch/arm64/include/asm/kvm_host.h | 10 ++ arch/arm64/kvm/reset.c | 65 ++++++--- tools/testing/selftests/kvm/.gitignore | 1 + tools/testing/selftests/kvm/Makefile | 1 + .../selftests/kvm/aarch64/vcpu_width_config.c | 125 ++++++++++++++++++ 6 files changed, 199 insertions(+), 30 deletions(-) create mode 100644 tools/testing/selftests/kvm/aarch64/vcpu_width_config.c
Signature · 1 line
2.35.1.894.gb6a874cedc-goog
PATCH v5 1/2 KVM: arm64: mixed-width check should be skipped for uninitialized vCPUs
KVM allows userspace to configure either all EL1 32bit or 64bit vCPUs for a guest. At vCPU reset, vcpu_allowed_register_width() checks if the vcpu's register width is consistent with all other vCPUs'. Since the checking is done even against vCPUs that are not initialized (KVM_ARM_VCPU_INIT has not been done) yet, the uninitialized vCPUs are erroneously treated as 64bit vCPU, which causes the function to incorrectly detect a mixed-width VM.
Introduce KVM_ARCH_FLAG_EL1_32BIT and KVM_ARCH_FLAG_REG_WIDTH_CONFIGURED bits for kvm->arch.flags. A value of the EL1_32BIT bit indicates that the guest needs to be configured with all 32bit or 64bit vCPUs, and a value of the REG_WIDTH_CONFIGURED bit indicates if a value of the EL1_32BIT bit is valid (already set up). Values in those bits are set at the first KVM_ARM_VCPU_INIT for the guest based on KVM_ARM_VCPU_EL1_32BIT configuration for the vCPU.
Check vcpu's register width against those new bits at the vcpu's KVM_ARM_VCPU_INIT (instead of against other vCPUs' register width).
--- arch/arm64/include/asm/kvm_emulate.h | 27 ++++++++---- arch/arm64/include/asm/kvm_host.h | 10 +++++ arch/arm64/kvm/reset.c | 65 ++++++++++++++++++---------- 3 files changed, 72 insertions(+), 30 deletions(-)
arch/arm64/include/asm/kvm_emulate.h +19 -8
@@ -43,10 +43,22 @@ void kvm_inject_pabt(struct kvm_vcpu *vcpu, unsigned long addr);void kvm_vcpu_wfi (struct kvm_vcpu *vcpu); +#if defined(__KVM_VHE_HYPERVISOR__) || defined(__KVM_NVHE_HYPERVISOR__) static __always_inline bool vcpu_el1_is_32bit(struct kvm_vcpu *vcpu) {return !(vcpu->arch.hcr_el2 &HCR_RW ); } +#else +static __always_inline bool vcpu_el1_is_32bit(struct kvm_vcpu *vcpu) +{ +struct kvm *kvm = vcpu->kvm; + +WARN_ON_ONCE (!test_bit (KVM_ARCH_FLAG_REG_WIDTH_CONFIGURED , + &kvm->arch.flags)); + +return test_bit (KVM_ARCH_FLAG_EL1_32BIT , &kvm->arch.flags); +} +#endif static inline void vcpu_reset_hcr (struct kvm_vcpu *vcpu) {
@@ -72,15 +84,14 @@ static inline void vcpu_reset_hcr(struct kvm_vcpu *vcpu) vcpu->arch.hcr_el2 |=HCR_TVM ; } -if (test_bit (KVM_ARM_VCPU_EL1_32BIT , vcpu->arch.features)) +if (vcpu_el1_is_32bit (vcpu)) vcpu->arch.hcr_el2 &= ~HCR_RW ; - -/* -* TID3: trap feature register accesses that we virtualise. -* For now this is conditional, since no AArch32 feature regs -* are currently virtualised. -*/ -if (!vcpu_el1_is_32bit (vcpu)) +else +/* +* TID3: trap feature register accesses that we virtualise. +* For now this is conditional, since no AArch32 feature regs +* are currently virtualised. +*/ vcpu->arch.hcr_el2 |=HCR_TID3 ; if (cpus_have_const_cap(ARM64_MISMATCHED_CACHE_TYPE) ||
arch/arm64/include/asm/kvm_host.h +10 -0
@@ -127,6 +127,16 @@ struct kvm_arch {#define KVM_ARCH_FLAG_MTE_ENABLED 1/* At least one vCPU has ran in the VM */ #define KVM_ARCH_FLAG_HAS_RAN_ONCE 2 +/* +* The following two bits are used to indicate the guest's EL1 +* register width configuration. A value of KVM_ARCH_FLAG_EL1_32BIT +* bit is valid only when KVM_ARCH_FLAG_REG_WIDTH_CONFIGURED is set. +* Otherwise, the guest's EL1 register width has not yet been +* determined yet. +*/ +#define KVM_ARCH_FLAG_REG_WIDTH_CONFIGURED 3 +#define KVM_ARCH_FLAG_EL1_32BIT 4 +unsigned long flags;/*
arch/arm64/kvm/reset.c +43 -22
94 lines not on this page · raw message
Signature · 6 lines
2.35.1.894.gb6a874cedc-goog _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH v5 2/2 KVM: arm64: selftests: Introduce vcpu_width_config
In reply to Reiji WatanabeIntroduce a test for aarch64 that ensures non-mixed-width vCPUs (all 64bit vCPUs or all 32bit vcPUs) can be configured, and mixed-width vCPUs cannot be configured.
--- tools/testing/selftests/kvm/.gitignore | 1 + tools/testing/selftests/kvm/Makefile | 1 + .../selftests/kvm/aarch64/vcpu_width_config.c | 125 ++++++++++++++++++ 3 files changed, 127 insertions(+) create mode 100644 tools/testing/selftests/kvm/aarch64/vcpu_width_config.c
tools/testing/selftests/kvm/.gitignore +1 -0
@@ -3,6 +3,7 @@ /aarch64/debug-exceptions /aarch64/get-reg-list /aarch64/psci_cpu_on_test +/aarch64/vcpu_width_config /aarch64/vgic_init /aarch64/vgic_irq /s390x/memop
tools/testing/selftests/kvm/Makefile +1 -0
@@ -104,6 +104,7 @@ TEST_GEN_PROGS_aarch64 += aarch64/arch_timer TEST_GEN_PROGS_aarch64 += aarch64/debug-exceptions TEST_GEN_PROGS_aarch64 += aarch64/get-reg-list TEST_GEN_PROGS_aarch64 += aarch64/psci_cpu_on_test +TEST_GEN_PROGS_aarch64 += aarch64/vcpu_width_config TEST_GEN_PROGS_aarch64 += aarch64/vgic_init TEST_GEN_PROGS_aarch64 += aarch64/vgic_irq TEST_GEN_PROGS_aarch64 += demand_paging_test
tools/testing/selftests/kvm/aarch64/vcpu_width_config.c new file +125 -0
125 lines not on this page · raw message
Signature · 6 lines
2.35.1.894.gb6a874cedc-goog _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Re: [PATCH v5 2/2] KVM: arm64: selftests: Introduce vcpu_width_config
Hi Reiji,
Quoted 6 lines · Reiji Watanabe
Introduce a test for aarch64 that ensures non-mixed-width vCPUs (all 64bit vCPUs or all 32bit vcPUs) can be configured, and mixed-width vCPUs cannot be configured.
Reviewed-by Andrew Jones <[email protected]>Signed-off-by Reiji Watanabe <[email protected]>
Tiny nits, but looks fine to me. Only bother addressing if you do another spin, otherwise:
Quoted 39 lines
--- tools/testing/selftests/kvm/.gitignore | 1 + tools/testing/selftests/kvm/Makefile | 1 + .../selftests/kvm/aarch64/vcpu_width_config.c | 125 ++++++++++++++++++ 3 files changed, 127 insertions(+) create mode 100644 tools/testing/selftests/kvm/aarch64/vcpu_width_config.ctools/testing/selftests/kvm/.gitignore +1
-0@@ -3,6 +3,7 @@ /aarch64/debug-exceptions /aarch64/get-reg-list /aarch64/psci_cpu_on_test +/aarch64/vcpu_width_config /aarch64/vgic_init /aarch64/vgic_irq /s390x/memoptools/testing/selftests/kvm/Makefile +1
-0@@ -104,6 +104,7 @@ TEST_GEN_PROGS_aarch64 += aarch64/arch_timer TEST_GEN_PROGS_aarch64 += aarch64/debug-exceptions TEST_GEN_PROGS_aarch64 += aarch64/get-reg-list TEST_GEN_PROGS_aarch64 += aarch64/psci_cpu_on_test +TEST_GEN_PROGS_aarch64 += aarch64/vcpu_width_config TEST_GEN_PROGS_aarch64 += aarch64/vgic_init TEST_GEN_PROGS_aarch64 += aarch64/vgic_irq TEST_GEN_PROGS_aarch64 += demand_paging_testtools/testing/selftests/kvm/aarch64/vcpu_width_config.c new file +2
-0@@ -0,0 +1,125 @@ +// SPDX-License-Identifier: GPL-2.0-only +/*
tools/testing/selftests/kvm/aarch64/vcpu_width_config.c line 12
Quoted 10 lines
tools/testing/selftests/kvm/aarch64/vcpu_width_config.c new file +10
-0+* vcpu_width_config - Test KVM_ARM_VCPU_INIT() with KVM_ARM_VCPU_EL1_32BIT. +* +* Copyright (c) 2022 Google LLC. +* +* This is a test that ensures that non-mixed-width vCPUs (all 64bit vCPUs +* or all 32bit vcPUs) can be configured and mixed-width vCPUs cannot be +* configured. +*/ + +#define _GNU_SOURCE
In other instances where we define _GNU_SOURCE, it is said we do it for program_invocation_short_name. Nonetheless, I cannot find anywhere that the symbol is actually being used.
This looks to be some leftover crud from our internal test library before we upstreamed KVM selftests a few years ago.
Quoted 49 lines
tools/testing/selftests/kvm/aarch64/vcpu_width_config.c new file +49
-0+#include "kvm_util.h" +#include "processor.h" +#include "test_util.h" + + +/* +* Add a vCPU, run KVM_ARM_VCPU_INIT with @init1, and then +* add another vCPU, and run KVM_ARM_VCPU_INIT with @init2. +*/ +static int add_init_2vcpus (struct kvm_vcpu_init *init1, +struct kvm_vcpu_init *init2) +{ +struct kvm_vm *vm; +int ret; + + vm =vm_create (VM_MODE_DEFAULT ,DEFAULT_GUEST_PHY_PAGES ,O_RDWR ); + +vm_vcpu_add (vm,0 ); + ret =_vcpu_ioctl (vm,0 ,KVM_ARM_VCPU_INIT , init1); +if (ret) + goto free_exit; + +vm_vcpu_add (vm,1 ); + ret =_vcpu_ioctl (vm,1 ,KVM_ARM_VCPU_INIT , init2); + +free_exit: +kvm_vm_free (vm); +return ret; +} + +/* +* Add two vCPUs, then run KVM_ARM_VCPU_INIT for one vCPU with @init1, +* and run KVM_ARM_VCPU_INIT for another vCPU with @init2. +*/ +static int add_2vcpus_init_2vcpus (struct kvm_vcpu_init *init1, +struct kvm_vcpu_init *init2) +{ +struct kvm_vm *vm; +int ret; + + vm =vm_create (VM_MODE_DEFAULT ,DEFAULT_GUEST_PHY_PAGES ,O_RDWR ); + +vm_vcpu_add (vm,0 ); +vm_vcpu_add (vm,1 ); + + ret =_vcpu_ioctl (vm,0 ,KVM_ARM_VCPU_INIT , init1); +if (ret) + goto free_exit; +
tools/testing/selftests/kvm/aarch64/vcpu_width_config.c line 72
Quoted 10 lines
tools/testing/selftests/kvm/aarch64/vcpu_width_config.c new file +10
-0+ ret =_vcpu_ioctl (vm,1 ,KVM_ARM_VCPU_INIT , init2); + +free_exit: +kvm_vm_free (vm); +return ret; +} + +/* +* Tests that two 64bit vCPUs can be configured, two 32bit vCPUs can be +* configured, and two mixed-witgh vCPUs cannot be configured.
mixed-width
tools/testing/selftests/kvm/aarch64/vcpu_width_config.c line 85
Quoted 13 lines
tools/testing/selftests/kvm/aarch64/vcpu_width_config.c new file +13
-0+* Each of those three cases, configure vCPUs in two different orders. +* The one is running KVM_CREATE_VCPU for 2 vCPUs, and then running +* KVM_ARM_VCPU_INIT for them. +* The other is running KVM_CREATE_VCPU and KVM_ARM_VCPU_INIT for a vCPU, +* and then run those commands for another vCPU. +*/ +int main (void ) +{ +struct kvm_vcpu_init init1, init2; +struct kvm_vm *vm; +int ret; + +if (kvm_check_cap (KVM_CAP_ARM_EL1_32BIT ) <=0 ) {
nit: this is fine, but KVM_CHECK_EXTENSION returns exactly 0 if a capability is not supported.
tools/testing/selftests/kvm/aarch64/vcpu_width_config.c line 94
Quoted 9 lines
tools/testing/selftests/kvm/aarch64/vcpu_width_config.c new file +9
-0+print_skip ("KVM_CAP_ARM_EL1_32BIT is not supported" ); +exit (KSFT_SKIP ); + } + +/* Get the preferred target type and copy that to init2 */ + vm =vm_create (VM_MODE_DEFAULT ,DEFAULT_GUEST_PHY_PAGES ,O_RDWR ); +vm_ioctl (vm,KVM_ARM_PREFERRED_TARGET , &init1); +kvm_vm_free (vm); +memcpy (&init2, &init1,sizeof (init2));
nit: I think you can just assign init2 = init1.
tools/testing/selftests/kvm/aarch64/vcpu_width_config.c line 100
Quoted 5 lines
tools/testing/selftests/kvm/aarch64/vcpu_width_config.c new file +5
-0+/* Test with 64bit vCPUs */ + ret =add_init_2vcpus (&init1, &init2); +TEST_ASSERT (ret ==0 , +"Configuring 64bit EL1 vCPUs failed unexpectedly" ); + ret =add_2vcpus_init_2vcpus (&init1, &init2);
nit: for the homogeneous vCPU configuration tests, you could pass two pointers to the same init structure to really drive the point home that the vCPUs are the same. The underlying ioctl does not write back anything to userspace.
Quoted 28 lines
tools/testing/selftests/kvm/aarch64/vcpu_width_config.c new file +25
-0+TEST_ASSERT (ret ==0 , +"Configuring 64bit EL1 vCPUs failed unexpectedly" ); + +/* Test with 32bit vCPUs */ + init1.features[0 ] = (1 <<KVM_ARM_VCPU_EL1_32BIT ); + init2.features[0 ] = (1 <<KVM_ARM_VCPU_EL1_32BIT ); + ret =add_init_2vcpus (&init1, &init2); +TEST_ASSERT (ret ==0 , +"Configuring 32bit EL1 vCPUs failed unexpectedly" ); + ret =add_2vcpus_init_2vcpus (&init1, &init2); +TEST_ASSERT (ret ==0 , +"Configuring 32bit EL1 vCPUs failed unexpectedly" ); + +/* Test with mixed-width vCPUs */ + init1.features[0 ] =0 ; + init2.features[0 ] = (1 <<KVM_ARM_VCPU_EL1_32BIT ); + ret =add_init_2vcpus (&init1, &init2); +TEST_ASSERT (ret !=0 , +"Configuring mixed-width vCPUs worked unexpectedly" ); + ret =add_2vcpus_init_2vcpus (&init1, &init2); +TEST_ASSERT (ret !=0 , +"Configuring mixed-width vCPUs worked unexpectedly" ); + +return 0 ; +}Signature · 1 line
2.35.1.894.gb6a874cedc-goog
Re: [PATCH v5 2/2] KVM: arm64: selftests: Introduce vcpu_width_config
Quoted 3 lines · Oliver Upton
Hi Reiji,
On Sun, Mar 20, 2022 at 10:08:04PM -0700, Reiji Watanabe wrote:
[...]
Quoted 9 lines
Quoted 1 line
+#define
_GNU_SOURCEIn other instances where we define
_GNU_SOURCE, it is said we do it forprogram_invocation_short_name. Nonetheless, I cannot find anywhere that the symbol is actually being used.This looks to be some leftover crud from our internal test library before we upstreamed KVM selftests a few years ago.
Ah, it's because we're actually using program_invocation_name. This already gets defined in lib/kvm_util.c, so except for a few oddball tests that directly call kvm_vm_elf_load(), this is unnecessary.
--
Thanks,
Oliver
Re: [PATCH v5 2/2] KVM: arm64: selftests: Introduce vcpu_width_config
In reply to Oliver UptonHi Oliver,
Quoted 8 lines · Oliver Upton
Hi Reiji,
Quoted 4 lines · Reiji Watanabe
Introduce a test for aarch64 that ensures non-mixed-width vCPUs (all 64bit vCPUs or all 32bit vcPUs) can be configured, and mixed-width vCPUs cannot be configured.
Quoted 7 lines
Quoted 2 lines
Reviewed-by Andrew Jones <[email protected]>Signed-off-by Reiji Watanabe <[email protected]>Tiny nits, but looks fine to me. Only bother addressing if you do another spin, otherwise:
Reviewed-by Oliver Upton <[email protected]>
Thank you for the review!
Since I would like to fix one of your comments (the typo) at least, I will respin the series (and will address all the comments).
Thanks,
Reiji
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm