KVM: Serialize vCPU creation and revert vcpu_ids tracking
Sean Christopherson
maintainerv2 cover KVM: Serialize vCPU creation and revert vcpu_ids tracking
Serialize vCPU creation by holding kvm->lock for the entirety of kvm_vm_ioctl_create_vcpu(), and then revert the now-redundant tracking adding by commit 97d65b544f48 ("KVM: Check for duplicate vcpu_id as early as possible"). I botched the math when justifying the vcpu_ids tracking; it's not an extra 256 bytes, it's an extra 2048 bytes. Roughly doubling the size of "struct kvm" tripped x86's KVM_SANITY_CHECK_VM_STRUCT_SIZE, and obviously isn't something we want to do in general.
The TL;DR of why it's a-ok to serialize vCPU creation is that no VMM actually does parallel vCPU creation. As with so many things, KVM's current behavior is the result of decades-old cruft, not intentional, deliberate design.
Patches 1-3 are a tangentially related cleanups and bug fixes; I included them here because holding kvm->lock for all of vCPU creation allows WARNing if KVM attempts to lock all vCPUs if vCPU creation is in-progress (the caller must hold kvm->lock).
v2:
- Tweak patch 1's changelog to clarify that that only x86's manual checks are dropped. [Sashiko]
- Add patches to convert additional arm64 and RISC-V usage to kvm_is_vcpu_creation_in_progress(). [Sashiko]
- Remove acquisition of kvm->lock from s390 and PPC vCPU creation flows. [Christian, Sashiko]
- Add Jean-Christophe's Tested-by to the revert.
v1: https://lore.kernel.org/all/[email protected]
Sean Christopherson (7):
KVM: Reject attempts to lock all vCPUs if vCPU creation is in-progress
KVM: arm64: vgic: Rely on vCPU creation check in "trylock all vCPUs"
KVM: RISC-V: Use kvm_is_vcpu_creation_in_progress() instead of
open-coded equivalent
KVM: Protect all of kvm_vm_ioctl_create_vcpu() with kvm->lock
KVM: Move check for existing vCPU ID to the top of vCPU creation Revert "KVM: Check for duplicate vcpu_id as early as possible"
KVM: WARN if vCPU creation is in-progress when locking all vCPUs
arch/arm64/kvm/vgic/vgic-init.c | 12 ++++------- arch/powerpc/kvm/book3s_hv.c | 2 -- arch/riscv/kvm/aia_device.c | 2 +- arch/s390/kvm/s390/s390.c | 5 +---- arch/x86/kvm/svm/sev.c | 10 ---------- arch/x86/kvm/vmx/tdx.c | 5 ----- include/linux/kvm_host.h | 1 - virt/kvm/kvm_main.c | 35 +++++++++++---------------------- 8 files changed, 17 insertions(+), 55 deletions(-)
base-commit: d599822bdb66aeec5ec76297b0fc6efaaeefe07c
Signature · 1 line
2.55.0.1082.g2b9226bbc0-goog
Sean Christopherson
maintainerPATCH v2 1/7 KVM: Reject attempts to lock all vCPUs if vCPU creation is in-progress
Reject locking of all vCPUs if vCPU creation is in-progress, i.e. if the number of "created" vCPUs doesn't match the number of "onlined" vCPUs. It's simply not possible to guarantee that KVM has truly locked all vCPUs if one or more vCPUs are actively being created. Holding kvm->lock does prevent in-flight vCPUs from being fully onlined, but it's infeasible for common KVM to know whether or not that provides sufficient protection.
In practice, this is likely a minor bug fix for the ARM and RISC-V usage of kvm_trylock_all_vcpus(), and a glorified nop for everything else. E.g. ARM's kvm_timer_vcpu_init() can race kvm_vm_ioctl_set_counter_offset() with respect to observing KVM_ARCH_FLAG_VM_COUNTER_OFFSET.
Opportunistically drop x86's existing manual checks on vCPU creation being in-progress as all of x86's checks immediately precede or follow locking of all vCPUs. Leave arm64 and RISC-V alone for the moment, as their checks aren't as obviously redundant/equivalent.
--- arch/x86/kvm/svm/sev.c | 10 ---------- arch/x86/kvm/vmx/tdx.c | 5 ----- virt/kvm/kvm_main.c | 6 ++++++ 3 files changed, 6 insertions(+), 15 deletions(-)
arch/x86/kvm/svm/sev.c +0 -10
@@ -1125,9 +1125,6 @@ static int sev_launch_update_vmsa(struct kvm *kvm, struct kvm_sev_cmd *argp)if (!sev_es_guest (kvm))return -ENOTTY ; -if (kvm_is_vcpu_creation_in_progress (kvm)) -return -EBUSY ; - ret =kvm_lock_all_vcpus (kvm);if (ret)return ret;
@@ -2115,10 +2112,6 @@ static int sev_check_source_vcpus(struct kvm *dst, struct kvm *src)struct kvm_vcpu *src_vcpu;unsigned long i; -if (kvm_is_vcpu_creation_in_progress (src) || -kvm_is_vcpu_creation_in_progress (dst)) -return -EBUSY ; -if (!sev_es_guest (src))return 0 ;
@@ -2510,9 +2503,6 @@ static int snp_launch_update_vmsa(struct kvm *kvm, struct kvm_sev_cmd *argp)unsigned long i;int ret; -if (kvm_is_vcpu_creation_in_progress (kvm)) -return -EBUSY ; - ret =kvm_lock_all_vcpus (kvm);if (ret)return ret;
arch/x86/kvm/vmx/tdx.c +0 -5
@@ -2728,11 +2728,6 @@ static tdx_vm_state_guard_t tdx_acquire_vm_state_locks(struct kvm *kvm)mutex_lock (&kvm->lock); -if (kvm->created_vcpus !=atomic_read (&kvm->online_vcpus)) { - r = -EBUSY ; - goto out_err; - } - r =kvm_lock_all_vcpus (kvm);if (r) goto out_err;
virt/kvm/kvm_main.c +6 -0
@@ -1363,6 +1363,9 @@ int kvm_trylock_all_vcpus(struct kvm *kvm)lockdep_assert_held (&kvm->lock); +if (kvm_is_vcpu_creation_in_progress (kvm)) +return -EBUSY ; +kvm_for_each_vcpu (i, vcpu, kvm)if (!mutex_trylock_nest_lock (&vcpu->mutex, &kvm->lock)) goto out_unlock;
@@ -1386,6 +1389,9 @@ int kvm_lock_all_vcpus(struct kvm *kvm)lockdep_assert_held (&kvm->lock); +if (kvm_is_vcpu_creation_in_progress (kvm)) +return -EBUSY ; +kvm_for_each_vcpu (i, vcpu, kvm) { r =mutex_lock_killable_nest_lock (&vcpu->mutex, &kvm->lock);if (r)
Signature · 1 line
2.55.0.1082.g2b9226bbc0-goog
Sean Christopherson
maintainerPATCH v2 2/7 KVM: arm64: vgic: Rely on vCPU creation check in "trylock all vCPUs"
In reply to Sean ChristophersonNow that KVM's APIs for locking all vCPUs return -EBUSY if vCPU creation is in-progress, drop the manual check for the same from vGIC creation, and update the comments accordingly.
Note, while KVM arm64 guards many vGIC operations with its arch-specific config_lock, holding kvm->lock is sufficient to guarantee a stable result for "is vCPU creation in-progress". So, no functional change intended.
Note #2, the open coded check in vgic_init() is racy when called without kvm->lock held, e.g. via vgic_lazy_init(). I.e. that check needs to stay open coded to avoid triggering a lockdep assert. Whether or not the race is "fine" is a problem for a different day.
--- arch/arm64/kvm/vgic/vgic-init.c | 12 ++++-------- 1 file changed, 4 insertions(+), 8 deletions(-)
arch/arm64/kvm/vgic/vgic-init.c +4 -8
@@ -97,6 +97,9 @@ int kvm_vgic_create(struct kvm *kvm, u32 type)/* * - Acquiring the vCPU mutex for every *online* vCPU to prevent * concurrent vCPU ioctls for vCPUs already visible to userspace. +* This also ensures KVM isn't in the middle of creating a vCPU, +* i.e. that there are no vCPUs that have been created but aren't +* yet fully online. */ ret = -EBUSY ;if (kvm_trylock_all_vcpus (kvm))
@@ -105,18 +108,11 @@ int kvm_vgic_create(struct kvm *kvm, u32 type)/* * - Taking the config_lock which protects VGIC data structures such * as the per-vCPU arrays of private IRQs (SGIs, PPIs). -*/ -mutex_lock (&kvm->arch.config_lock); - -/* -* - Bailing on the entire thing if a vCPU is in the middle of creation, -* dropped the kvm->lock, but hasn't reached kvm_arch_vcpu_create(). * * The whole combination of this guarantees that no vCPU can get into * KVM with a VGIC configuration inconsistent with the VM's VGIC. */ -if (kvm->created_vcpus !=atomic_read (&kvm->online_vcpus)) - goto out_unlock; +mutex_lock (&kvm->arch.config_lock);if (irqchip_in_kernel (kvm)) { ret = -EEXIST ;
Signature · 1 line
2.55.0.1082.g2b9226bbc0-goog
Sean Christopherson
maintainerPATCH v2 3/7 KVM: RISC-V: Use kvm_is_vcpu_creation_in_progress() instead of open-coded equivalent
In reply to Sean ChristophersonUse kvm_is_vcpu_creation_in_progress() instead of an open-coded equivalent during AIA initialization. Unlike similar vGIC code in arm64, the relevant RISC-V code runs under kvm->lock, i.e. can use the standard API without hitting lockdep false positive.
No functional change intended.
--- arch/riscv/kvm/aia_device.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/riscv/kvm/aia_device.c +1 -1
@@ -237,7 +237,7 @@ static int aia_init(struct kvm *kvm)return -EBUSY ;/* We might be in the middle of creating a VCPU? */ -if (kvm->created_vcpus !=atomic_read (&kvm->online_vcpus)) +if (kvm_is_vcpu_creation_in_progress (kvm))return -EBUSY ;/* Number of sources should be less than or equals number of IDs */
Signature · 1 line
2.55.0.1082.g2b9226bbc0-goog
Sean Christopherson
maintainerPATCH v2 4/7 KVM: Protect all of kvm_vm_ioctl_create_vcpu() with kvm->lock
In reply to Sean ChristophersonWhen creating a vCPU, don't drop kvm->lock to when doing the bulk of actual vCPU creation, as allowing multiple vCPUs to be created in parallel adds significant complexity in KVM (as evidenced by the many related bugs), and all known VMMs fully serialize vCPU creation. Remove all manually locking of kvm->lock from kvm_arch_vcpu_{post,}create() for obvious reasons.
For many years, "everyone" has assumed that dropping kvm->lock was done for performance reasons optimization, e.g. to allow userspace to create all vCPUs concurrently for latency purposes. But as above, no known VMM does that. Looking at the history of this code, before commit 11ec28047118 ("KVM: Convert vm lock to a mutex"), kvm->lock was a spinlock. I.e. KVM *had* to drop kvm->lock when doing the bulk of vCPU creation, otherwise KVM couldn't do normal memory allocations. When kvm->lock got turned into a mutex for unrelated reasons, no one took advantage updated of the change to simplify vCPU creation. And 19 years later, everyone just assumed that KVM continued to deal with the complexity for performance reasons.
Furthermore, naively parallelizing vCPU creation in userspace is likely a net negative due to the overheads of task creation. Unless a VMM carefully avoids the extra overhead related to parallelization, e.g. spawns each vCPU's thread before creating the vCPU, creating vCPUs concurrently is a net *negative* up until about ~64 vCPUs, after which the times are a wash.
The absolute speed of light _is_ faster if KVM doesn't hold kvm-lock, but at vCPU counts of ~16 or less, it's probably in the noise when considering total VM creation time, as the added latency is less than 1ms up until 16 or so vCPUs.
On top of all that, KVM has had a *lot* of fatal bugs (most often found by syzkaller) related to vCPUs being created while trying to do per-VM operations (basically, see every flow that locks all vCPUs). I.e. the parallel vCPU creation "support" is actively harmful as the only "use case" is for misbehaving userspace to exploit KVM bugs.
Serializing vCPU creation will allow reverting commit 97d65b544f48 ("KVM:
Check for duplicate vcpu_id as early as possible"), which had "minor" math
error: the worst case scenario isn't "256 bytes per VM", it's "256 unsigned longs per VM", i.e. 2048 bytes per VM, which doubles the size of each VM and pushes several architectures into order-1 allocations.
--- arch/powerpc/kvm/book3s_hv.c | 2 -- arch/s390/kvm/s390/s390.c | 5 +---- virt/kvm/kvm_main.c | 22 +++++----------------- 3 files changed, 6 insertions(+), 23 deletions(-)
arch/powerpc/kvm/book3s_hv.c +0 -2
@@ -3058,7 +3058,6 @@ static int kvmppc_core_vcpu_create_hv(struct kvm_vcpu *vcpu)init_waitqueue_head (&vcpu->arch.cpu_run); -mutex_lock (&kvm->lock); vcore =NULL ; err = -EINVAL ;if (cpu_has_feature (CPU_FTR_ARCH_300 )) {
@@ -3091,7 +3090,6 @@ static int kvmppc_core_vcpu_create_hv(struct kvm_vcpu *vcpu)mutex_unlock (&kvm->arch.mmu_setup_lock); } } -mutex_unlock (&kvm->lock);if (!vcore)return err;
arch/s390/kvm/s390/s390.c +1 -4
@@ -3579,12 +3579,11 @@ void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)void kvm_arch_vcpu_postcreate (struct kvm_vcpu *vcpu) { -mutex_lock (&vcpu->kvm->lock);preempt_disable (); vcpu->arch.sie_block->epoch = vcpu->kvm->arch.epoch; vcpu->arch.sie_block->epdx = vcpu->kvm->arch.epdx;preempt_enable (); -mutex_unlock (&vcpu->kvm->lock); +if (!kvm_is_ucontrol (vcpu->kvm)) { vcpu->arch.gmap = vcpu->kvm->arch.gmap;sca_add_vcpu (vcpu);
@@ -3757,13 +3756,11 @@ static int kvm_s390_vcpu_setup(struct kvm_vcpu *vcpu)kvm_s390_vcpu_pci_setup (vcpu); -mutex_lock (&vcpu->kvm->lock);if (kvm_s390_pv_is_protected (vcpu->kvm)) { rc =kvm_s390_pv_create_cpu (vcpu, &uvrc, &uvrrc);if (rc)kvm_s390_vcpu_unsetup_cmma (vcpu); } -mutex_unlock (&vcpu->kvm->lock);return rc; }
virt/kvm/kvm_main.c +5 -17
@@ -4165,6 +4165,8 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id)struct kvm_vcpu *vcpu;struct page *page; +guard (mutex)(&kvm->lock); +/* * KVM tracks vCPU IDs as 'int', be kind to userspace and reject * too-large values instead of silently truncating.
@@ -4177,26 +4179,18 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id)if (id >=KVM_MAX_VCPU_IDS )return -EINVAL ; -mutex_lock (&kvm->lock); -if (kvm->created_vcpus >= kvm->max_vcpus) { -mutex_unlock (&kvm->lock); +if (kvm->created_vcpus >= kvm->max_vcpus)return -EINVAL ; - } -if (test_bit (id, kvm->vcpu_ids)) { -mutex_unlock (&kvm->lock); +if (test_bit (id, kvm->vcpu_ids))return -EEXIST ; - } r =kvm_arch_vcpu_precreate (kvm, id); -if (r) { -mutex_unlock (&kvm->lock); +if (r)return r; - } kvm->created_vcpus++;__set_bit (id, kvm->vcpu_ids); -mutex_unlock (&kvm->lock); vcpu =kmem_cache_zalloc (kvm_vcpu_cache,GFP_KERNEL_ACCOUNT );if (!vcpu) {
@@ -4227,8 +4221,6 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id) goto arch_vcpu_destroy; } -mutex_lock (&kvm->lock); -if (WARN_ON_ONCE (kvm_get_vcpu_by_id (kvm, id))) { r = -EEXIST ; goto unlock_vcpu_destroy;
@@ -4267,7 +4259,6 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id)atomic_inc (&kvm->online_vcpus);mutex_unlock (&vcpu->mutex); -mutex_unlock (&kvm->lock);kvm_arch_vcpu_postcreate (vcpu);kvm_create_vcpu_debugfs (vcpu);return r;
@@ -4278,7 +4269,6 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id)xa_erase (&kvm->vcpu_array, vcpu->vcpu_idx); unlock_vcpu_destroy: vcpu->vcpu_idx =-1 ; -mutex_unlock (&kvm->lock);kvm_dirty_ring_free (&vcpu->dirty_ring); arch_vcpu_destroy:kvm_arch_vcpu_destroy (vcpu);
@@ -4287,10 +4277,8 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id) vcpu_free:kmem_cache_free (kvm_vcpu_cache, vcpu); vcpu_decrement: -mutex_lock (&kvm->lock); kvm->created_vcpus--;__clear_bit (id, kvm->vcpu_ids); -mutex_unlock (&kvm->lock);return r; }
Signature · 1 line
2.55.0.1082.g2b9226bbc0-goog
[email protected]
AI reviewRe: [PATCH v2 4/7] KVM: Protect all of kvm_vm_ioctl_create_vcpu() with kvm->lock
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] kvm: mixing goto-based error handling with guard() in kvm_vm_ioctl_create_vcpu --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] kvm: mixing goto-based error handling with guard() in kvm_vm_ioctl_create_vcpu --
commit 01402bb146d901e6f480f96bad9c9a7b5314ecb9
Author: Sean Christopherson <[email protected]>
KVM: Protect all of kvm_vm_ioctl_create_vcpu() with kvm->lock
This commit serializes vCPU creation by holding the kvm->lock for the entirety of kvm_vm_ioctl_create_vcpu(). It removes the manual lock/unlock calls that previously existed around parts of the creation process and replaces them with a scoped guard.
Quoted 11 lines
virt/kvm/kvm_main.c +2
-0@@ -4165,6 +4165,8 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id)struct kvm_vcpu *vcpu;struct page *page; +guard (mutex)(&kvm->lock); +/* * KVM tracks vCPU IDs as 'int', be kind to userspace and reject
[ ... ]
virt/kvm/kvm_main.c static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id) · line 4283
Quoted 10 lines
virt/kvm/kvm_main.c +0
-2@@ -4287,10 +4277,8 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id) vcpu_free:kmem_cache_free (kvm_vcpu_cache, vcpu); vcpu_decrement: -mutex_lock (&kvm->lock); kvm->created_vcpus--;__clear_bit (id, kvm->vcpu_ids); -mutex_unlock (&kvm->lock);return r; }
[Severity: Low]
Does kvm_vm_ioctl_create_vcpu() in virt/kvm/kvm_main.c mix goto-based error handling with the new scope-based guard()?
The cleanup subsystem guidelines forbid mixing scope-based guard() with legacy goto error handling in the same function, as it creates confusing ownership semantics and risks future double-free or resource leak bugs during maintenance.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
Sean Christopherson
maintainerPATCH v2 5/7 KVM: Move check for existing vCPU ID to the top of vCPU creation
In reply to Sean ChristophersonNow that kvm->lock is held for the entirety of vCPU creation, check for a conflicting vCPU ID at the begnning of vCPU creation, before the arch precreate() hook is invoked. This will allow reverting commit 97d65b544f48 ("KVM: Check for duplicate vcpu_id as early as possible"). For now, keep the redundant vcpu_ids tracking as a sanity check.
No functional change intended (absent KVM bugs, checking vcpu_ids and walking kvm_get_vcpu_by_id() should yield the same result).
--- virt/kvm/kvm_main.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-)
virt/kvm/kvm_main.c +4 -6
@@ -4182,7 +4182,10 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id)if (kvm->created_vcpus >= kvm->max_vcpus)return -EINVAL ; -if (test_bit (id, kvm->vcpu_ids)) +if (kvm_get_vcpu_by_id (kvm, id)) +return -EEXIST ; + +if (WARN_ON_ONCE (test_bit (id, kvm->vcpu_ids)))return -EEXIST ; r =kvm_arch_vcpu_precreate (kvm, id);
@@ -4221,11 +4224,6 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id) goto arch_vcpu_destroy; } -if (WARN_ON_ONCE (kvm_get_vcpu_by_id (kvm, id))) { - r = -EEXIST ; - goto unlock_vcpu_destroy; - } -/* * Set the vCPU's index *before* the vCPU is reachable by other tasks. * Unwind the index back to -1 on failure so that KVM can use the index
Signature · 1 line
2.55.0.1082.g2b9226bbc0-goog
Sean Christopherson
maintainerPATCH v2 6/7 Revert "KVM: Check for duplicate vcpu_id as early as possible"
In reply to Sean ChristophersonNow that KVM uses kvm_get_vcpu_by_id() to check for an existing vCPU ID before doing any meaningful work, which was made possible by holding kvm->lock for the entirety of vCPU creation, revert the now-redundant "early" vCPU ID tracking. The claims about the impact of kvm->vcpu_ids on the memory footprint were a wee bit wrong: the worst case scenario isn't 256 bytes per VM, it's 256 "unsigned longs" per VM, i.e. 2048 bytes per VM.
Increasing the size of "struct kvm" by 2048 nearly doubled the total size on many architectures, and tripped x86's KVM_SANITY_CHECK_VM_STRUCT_SIZE, which was added to detect this *exact* scenario, where a single change significantly increased the size of "struct kvm". I.e. attempting to build KVM with CONFIG_DEBUG_KERNEL=n fails on x86 (the build failures got missed because all build bots apparently test only CONFIG_DEBUG_KERNEL=y kernels, and maintainers' test flows were similarly lacking).
This reverts commit 97d65b544f48b2ee49f6aea32145e3e7969955dc.
--- include/linux/kvm_host.h | 1 - virt/kvm/kvm_main.c | 5 ----- 2 files changed, 6 deletions(-)
include/linux/kvm_host.h +0 -1
@@ -791,7 +791,6 @@ struct kvm {/* The current active memslot set for each address space */ struct kvm_memslots __rcu *memslots[KVM_MAX_NR_ADDRESS_SPACES ];struct xarray vcpu_array; -DECLARE_BITMAP (vcpu_ids,KVM_MAX_VCPU_IDS );/* * Protected by slots_lock, but can be read outside if an * incorrect answer is acceptable.
virt/kvm/kvm_main.c +0 -5
@@ -4185,15 +4185,11 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id)if (kvm_get_vcpu_by_id (kvm, id))return -EEXIST ; -if (WARN_ON_ONCE (test_bit (id, kvm->vcpu_ids))) -return -EEXIST ; - r =kvm_arch_vcpu_precreate (kvm, id);if (r)return r; kvm->created_vcpus++; -__set_bit (id, kvm->vcpu_ids); vcpu =kmem_cache_zalloc (kvm_vcpu_cache,GFP_KERNEL_ACCOUNT );if (!vcpu) {
@@ -4276,7 +4272,6 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id)kmem_cache_free (kvm_vcpu_cache, vcpu); vcpu_decrement: kvm->created_vcpus--; -__clear_bit (id, kvm->vcpu_ids);return r; }
Signature · 1 line
2.55.0.1082.g2b9226bbc0-goog
Sean Christopherson
maintainerPATCH v2 7/7 KVM: WARN if vCPU creation is in-progress when locking all vCPUs
In reply to Sean ChristophersonNow that KVM holds kvm->lock for the entirety of vCPU creation, from when created_vcpus is incremented until the new vCPU is fully onlined, WARN if the impossible happens and KVM somehow sees a discrepancy between the number of vCPUs "created" and "onlined".
--- virt/kvm/kvm_main.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
virt/kvm/kvm_main.c +2 -2
@@ -1363,7 +1363,7 @@ int kvm_trylock_all_vcpus(struct kvm *kvm)lockdep_assert_held (&kvm->lock); -if (kvm_is_vcpu_creation_in_progress (kvm)) +if (WARN_ON_ONCE (kvm_is_vcpu_creation_in_progress (kvm)))return -EBUSY ;kvm_for_each_vcpu (i, vcpu, kvm)
@@ -1389,7 +1389,7 @@ int kvm_lock_all_vcpus(struct kvm *kvm)lockdep_assert_held (&kvm->lock); -if (kvm_is_vcpu_creation_in_progress (kvm)) +if (WARN_ON_ONCE (kvm_is_vcpu_creation_in_progress (kvm)))return -EBUSY ;kvm_for_each_vcpu (i, vcpu, kvm) {
Signature · 1 line
2.55.0.1082.g2b9226bbc0-goog
Hi Sean,
Just like for the first version, I tested the full V2 series with the configuration that originally triggered the build failure (CONFIG_DEBUG_KERNEL=n, CONFIG_KASAN=n).
The kernel builds successfully on adm64, and I booted the resulting kernel and started two QEMU/KVM VMs without any issues.
Thanks again !
Cheers,
Jean-Christophe Guillain
Signature · 3 lines
kvm-riscv mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kvm-riscv
Quoted 28 lines · Sean Christopherson
Serialize vCPU creation by holding
kvm->lockfor the entirety ofkvm_vm_ioctl_create_vcpu(), and then revert the now-redundant tracking adding by commit97d65b544f48("KVM: Check for duplicatevcpu_idas early as possible"). I botched the math when justifying thevcpu_idstracking; it's not an extra 256 bytes, it's an extra 2048 bytes. Roughly doubling the size of "struct kvm" tripped x86'sKVM_SANITY_CHECK_VM_STRUCT_SIZE, and obviously isn't something we want to do in general.The TL;DR of why it's a-ok to serialize vCPU creation is that no VMM actually does parallel vCPU creation. As with so many things, KVM's current behavior is the result of decades-old cruft, not intentional, deliberate design.
Patches 1-3 are a tangentially related cleanups and bug fixes; I included them here because holding
kvm->lockfor all of vCPU creation allows WARNing if KVM attempts to lock all vCPUs if vCPU creation is in-progress (the caller must holdkvm->lock).v2:
- Tweak patch 1's changelog to clarify that that only x86's manual checks are dropped. [Sashiko]
- Add patches to convert additional arm64 and RISC-V usage tokvm_is_vcpu_creation_in_progress(). [Sashiko]
- Remove acquisition ofkvm->lockfrom s390 and PPC vCPU creation flows. [Christian, Sashiko]
- Add Jean-Christophe's Tested-by to the revert.
Quoted 9 lines
Sean Christopherson (7):
KVM: Reject attempts to lock all vCPUs if vCPU creation is in-progress
KVM: arm64: vgic: Rely on vCPU creation check in "trylock all vCPUs"
KVM: RISC-V: Usekvm_is_vcpu_creation_in_progress()instead of
open-coded equivalent
KVM: Protect all ofkvm_vm_ioctl_create_vcpu()withkvm->lock
KVM: Move check for existing vCPU ID to the top of vCPU creation Revert "KVM: Check for duplicatevcpu_idas early as possible"
KVM: WARN if vCPU creation is in-progress when locking all vCPUs
This addresses the Secure TSC splat I reported previously:
https://lore.kernel.org/all/apgj5l7DZGsScsNc@blrnaveerao1/
I booted a SNP guest with Secure TSC enabled and didn't see any lockdep reports. For what that's worth:
- Naveen
Signature · 3 lines
kvm-riscv mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kvm-riscv
Queued, thanks.
Paolo