KVM: arm64: Fix some races in CPU_ON PSCI call
v2 cover KVM: arm64: Fix some races in CPU_ON PSCI call
The CPU_ON PSCI call requires careful coordination between vCPUs in KVM, as it allows callers to send a payload (pc, context id) to another vCPU to start execution. There are a couple of races in the handling of CPU_ON:
- KVM uses the kvm->lock to serialize the write-side of a vCPU's reset state. However, kvm_vcpu_reset() doesn't take the lock on the read-size, meaning the vCPU could be reset with interleaved state from two separate CPU_ON calls.
- If a targeted vCPU never enters the guest again (say, the VMM was getting ready to migrate), then the reset payload is never actually folded in to the vCPU's registers. Despite this, the calling vCPU has already made the target runnable. Migrating the target vCPU at this time will result in execution from its old PC, not execution coming out of the reset state at the requested address.
Patch 1 addresses the read-side race in KVM's CPU_ON implementation.
Patch 2 fixes the KVM/VMM race by resetting a vCPU (if requested) whenever the VMM tries to read out its registers. Gross, but it avoids exposing the vcpu_reset_state structure through some other UAPI. That is undesirable, as we really are only trying to paper over the implementation details of PSCI in KVM.
Patch 3 is unrelated, and is based on my own reading of the PSCI specification. In short, if you invoke PSCI_ON from AArch64, then you must set the Aff3 bits. This is impossible if you use the 32 bit function, since the arguments are only 32 bits. Just return INVALID_PARAMS to the guest in this case.
This series cleanly applies to v5.14-rc6
The series was tested with the included KVM selftest on an Ampere Mt. Jade system. Broken behavior was verified using the same test on v5.14-rc6, sans this series.
v1: http://lore.kernel.org/r/[email protected]
v1 -> v2:
- avoid memcpy for reading reset state (Marc)
- promote reset_state.reset = false to WRITE_ONCE() (Marc)
- rephrase comment, commit msg (Marc)
- cite the PSCI spec precisely (Marc)
- drop unnecessary mask-down to 32 bits in CPU_ON (Marc)
- rebased on top of v5.14-rc6
Oliver Upton (4):
KVM: arm64: Fix read-side race on updates to vcpu reset state
KVM: arm64: Handle PSCI resets before userspace touches vCPU state
KVM: arm64: Enforce reserved bits for PSCI target affinities
selftests: KVM: Introduce psci_cpu_on_test
arch/arm64/kvm/arm.c | 9 ++ arch/arm64/kvm/psci.c | 20 ++- arch/arm64/kvm/reset.c | 16 ++- tools/testing/selftests/kvm/.gitignore | 1 + tools/testing/selftests/kvm/Makefile | 1 + .../selftests/kvm/aarch64/psci_cpu_on_test.c | 121 ++++++++++++++++++ .../selftests/kvm/include/aarch64/processor.h | 3 + 7 files changed, 162 insertions(+), 9 deletions(-) create mode 100644 tools/testing/selftests/kvm/aarch64/psci_cpu_on_test.c
Signature · 1 line
2.33.0.rc1.237.g0d66db33f3-goog
PATCH v2 1/4 KVM: arm64: Fix read-side race on updates to vcpu reset state
KVM correctly serializes writes to a vCPU's reset state, however since we do not take the KVM lock on the read side it is entirely possible to read state from two different reset requests.
Cure the race for now by taking the KVM lock when reading the reset_state structure.
--- arch/arm64/kvm/reset.c | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-)
arch/arm64/kvm/reset.c +10 -6
@@ -210,10 +210,16 @@ static bool vcpu_allowed_register_width(struct kvm_vcpu *vcpu)*/ int kvm_reset_vcpu (struct kvm_vcpu *vcpu) { +struct vcpu_reset_state reset_state;int ret;bool loaded;u32 pstate; +mutex_lock (&vcpu->kvm->lock); + reset_state = vcpu->arch.reset_state; +WRITE_ONCE (vcpu->arch.reset_state.reset, false); +mutex_unlock (&vcpu->kvm->lock); +/* Reset PMU outside of the non-preemptible section */ kvm_pmu_vcpu_reset (vcpu);
@@ -276,8 +282,8 @@ int kvm_reset_vcpu(struct kvm_vcpu *vcpu)* Additional reset state handling that PSCI may have imposed on us. * Must be done after all the sys_reg reset. */ -if (vcpu->arch.reset_state.reset) { -unsigned long target_pc = vcpu->arch.reset_state.pc; +if (reset_state.reset) { +unsigned long target_pc = reset_state.pc;/* Gracefully handle Thumb2 entry point */ if (vcpu_mode_is_32bit (vcpu) && (target_pc &1 )) {
@@ -286,13 +292,11 @@ int kvm_reset_vcpu(struct kvm_vcpu *vcpu) }/* Propagate caller endianness */ -if (vcpu->arch.reset_state.be) +if (reset_state.be)kvm_vcpu_set_be (vcpu); *vcpu_pc (vcpu) = target_pc; -vcpu_set_reg (vcpu,0 , vcpu->arch.reset_state.r0); - - vcpu->arch.reset_state.reset = false; +vcpu_set_reg (vcpu,0 , reset_state.r0); }/* Reset timer */
Signature · 1 line
2.33.0.rc1.237.g0d66db33f3-goog
PATCH v2 2/4 KVM: arm64: Handle PSCI resets before userspace touches vCPU state
In reply to Oliver UptonThe CPU_ON PSCI call takes a payload that KVM uses to configure a destination vCPU to run. This payload is non-architectural state and not exposed through any existing UAPI. Effectively, we have a race between CPU_ON and userspace saving/restoring a guest: if the target vCPU isn't ran again before the VMM saves its state, the requested PC and context ID are lost. When restored, the target vCPU will be runnable and start executing at its old PC.
We can avoid this race by making sure the reset payload is serviced before userspace can access a vCPU's state.
--- arch/arm64/kvm/arm.c | 8 ++++++++ 1 file changed, 8 insertions(+)
arch/arm64/kvm/arm.c +8 -0
@@ -1219,6 +1219,14 @@ long kvm_arch_vcpu_ioctl(struct file *filp,if (copy_from_user (®, argp,sizeof (reg)))break ; +/* +* We could owe a reset due to PSCI. Handle the pending reset +* here to ensure userspace register accesses are ordered after +* the reset. +*/ +if (kvm_check_request (KVM_REQ_VCPU_RESET , vcpu)) +kvm_reset_vcpu (vcpu); +if (ioctl ==KVM_SET_ONE_REG ) r =kvm_arm_set_reg (vcpu, ®);else
Signature · 1 line
2.33.0.rc1.237.g0d66db33f3-goog
PATCH v2 3/4 KVM: arm64: Enforce reserved bits for PSCI target affinities
In reply to Oliver UptonAccording to the PSCI specification, ARM DEN 0022D, 5.1.4 "CPU_ON", the CPU_ON function takes a target_cpu argument that is bit-compatible with the affinity fields in MPIDR_EL1. All other bits in the argument are RES0. Note that the same constraints apply to the target_affinity argument for the AFFINITY_INFO call.
Enforce the spec by returning INVALID_PARAMS if a guest incorrectly sets a RES0 bit.
--- arch/arm64/kvm/psci.c | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-)
arch/arm64/kvm/psci.c +12 -3
@@ -59,6 +59,12 @@ static void kvm_psci_vcpu_off(struct kvm_vcpu *vcpu)kvm_vcpu_kick (vcpu); } +static inline bool kvm_psci_valid_affinity (struct kvm_vcpu *vcpu, +unsigned long affinity) +{ +return !(affinity & ~MPIDR_HWID_BITMASK ); +} +static unsigned long kvm_psci_vcpu_on (struct kvm_vcpu *source_vcpu) {struct vcpu_reset_state *reset_state;
@@ -66,9 +72,9 @@ static unsigned long kvm_psci_vcpu_on(struct kvm_vcpu *source_vcpu)struct kvm_vcpu *vcpu =NULL ;unsigned long cpu_id; - cpu_id =smccc_get_arg1 (source_vcpu) &MPIDR_HWID_BITMASK ; -if (vcpu_mode_is_32bit (source_vcpu)) - cpu_id &= ~((u32 )0 ); + cpu_id =smccc_get_arg1 (source_vcpu); +if (!kvm_psci_valid_affinity (source_vcpu, cpu_id)) +return PSCI_RET_INVALID_PARAMS ; vcpu =kvm_mpidr_to_vcpu (kvm, cpu_id);
@@ -126,6 +132,9 @@ static unsigned long kvm_psci_vcpu_affinity_info(struct kvm_vcpu *vcpu) target_affinity =smccc_get_arg1 (vcpu); lowest_affinity_level =smccc_get_arg2 (vcpu); +if (!kvm_psci_valid_affinity (vcpu, target_affinity)) +return PSCI_RET_INVALID_PARAMS ; +/* Determine target affinity mask */ target_affinity_mask =psci_affinity_mask (lowest_affinity_level);if (!target_affinity_mask)
Signature · 1 line
2.33.0.rc1.237.g0d66db33f3-goog
PATCH v2 4/4 selftests: KVM: Introduce psci_cpu_on_test
In reply to Oliver UptonIntroduce a test for aarch64 that ensures CPU resets induced by PSCI are reflected in the target vCPU's state, even if the target is never run again. This is a regression test for a race between vCPU migration and PSCI.
--- tools/testing/selftests/kvm/.gitignore | 1 + tools/testing/selftests/kvm/Makefile | 1 + .../selftests/kvm/aarch64/psci_cpu_on_test.c | 121 ++++++++++++++++++ .../selftests/kvm/include/aarch64/processor.h | 3 + 4 files changed, 126 insertions(+) create mode 100644 tools/testing/selftests/kvm/aarch64/psci_cpu_on_test.c
tools/testing/selftests/kvm/.gitignore +1 -0
@@ -1,6 +1,7 @@ # SPDX-License-Identifier: GPL-2.0-only /aarch64/debug-exceptions /aarch64/get-reg-list +/aarch64/psci_cpu_on_test /aarch64/vgic_init /s390x/memop /s390x/resets
tools/testing/selftests/kvm/Makefile +1 -0
@@ -86,6 +86,7 @@ TEST_GEN_PROGS_x86_64 += kvm_binary_stats_test TEST_GEN_PROGS_aarch64 += aarch64/debug-exceptions TEST_GEN_PROGS_aarch64 += aarch64/get-reg-list +TEST_GEN_PROGS_aarch64 += aarch64/psci_cpu_on_test TEST_GEN_PROGS_aarch64 += aarch64/vgic_init TEST_GEN_PROGS_aarch64 += demand_paging_test TEST_GEN_PROGS_aarch64 += dirty_log_test
tools/testing/selftests/kvm/aarch64/psci_cpu_on_test.c new file +121 -0
121 lines not on this page · raw message
tools/testing/selftests/kvm/include/aarch64/processor.h +3 -0
@@ -17,6 +17,7 @@#define CPACR_EL1 3, 0, 1, 0, 2#define TCR_EL1 3, 0, 2, 0, 2#define MAIR_EL1 3, 0, 10, 2, 0 +#define MPIDR_EL1 3, 0, 0, 0, 5#define TTBR0_EL1 3, 0, 2, 0, 0#define SCTLR_EL1 3, 0, 1, 0, 0#define VBAR_EL1 3, 0, 12, 0, 0
@@ -40,6 +41,8 @@ (0xfful << (4 * 8)) | \ (0xbbul << (5 * 8))) +#define MPIDR_HWID_BITMASK (0xff00fffffful) +static inline void get_reg (struct kvm_vm *vm,uint32_t vcpuid,uint64_t id,uint64_t *addr) {struct kvm_one_reg reg;
Signature · 1 line
2.33.0.rc1.237.g0d66db33f3-goog
Quoted 11 lines · Oliver Upton
The
CPU_ONPSCI call requires careful coordination between vCPUs in KVM, as it allows callers to send a payload (pc, context id) to another vCPU to start execution. There are a couple of races in the handling ofCPU_ON:- KVM uses the
kvm->lockto serialize the write-side of a vCPU's reset state. However,kvm_vcpu_reset()doesn't take the lock on the read-size, meaning the vCPU could be reset with interleaved state from two separateCPU_ONcalls.[...]
Applied to next, thanks!
[1/4] KVM: arm64: Fix read-side race on updates to vcpu reset state
commit: 6654f9dfcb88fea3b9affc180dc3c04333d0f306
[2/4] KVM: arm64: Handle PSCI resets before userspace touches vCPU state
commit: 6826c6849b46aaa91300201213701eb861af4ba0
[3/4] KVM: arm64: Enforce reserved bits for PSCI target affinities
commit: e10ecb4d6c0761ca545b3946df1707a41f9f845e
[4/4] selftests: KVM: Introduce psci_cpu_on_test
commit: cb97cf95c44021278b7637731bc0928026bc29abCheers,
M.
Signature · 7 lines
Without deviation from the norm, progress is not possible. _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm