Skip to contentSkip to conversation

KVM: Serialize vCPU creation and revert vcpu_ids tracking

PATCH Superseded Sean Christopherson 6 participants 13 messages Sep 14 – Sep 21

Subsystems (6)
x86 KVM s390 KVM x86 +2 more subsystems

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Sean Christopherson

author maintainer
#0 lore raw

cover KVM: Serialize vCPU creation and revert vcpu_ids tracking

Serialize vCPU creation by holding kvm->lock for the entirety of kvm_vm_ioctl_create_vcpu(), and then revert the now-redundant tracking adding by commit 97d65b544f48 ("KVM: Check for duplicate vcpu_id as early as possible"). I botched the math when justifying the vcpu_ids tracking; it's not an extra 256 bytes, it's an extra 2048 bytes. Roughly doubling the size of "struct kvm" tripped x86's KVM_SANITY_CHECK_VM_STRUCT_SIZE, and obviously isn't something we want to do in general.

The TL;DR of why it's a-ok to serialize vCPU creation is that no VMM actually does parallel vCPU creation. As with so many things, KVM's current behavior is the result of decades-old cruft, not intentional, deliberate design.

Patch 1 is a tangentially related bug fix; I included it here because holding kvm->lock for all of vCPU creation allows WARNing if KVM attempts to lock all vCPUs if vCPU creation is in-progress (the caller is must hold kvm->lock).

Sean Christopherson (5):
KVM: Reject attempts to lock all vCPUs if vCPU creation is in-progress
KVM: Protect all of kvm_vm_ioctl_create_vcpu() with kvm->lock
KVM: Move check for existing vCPU ID to the top of vCPU creation Revert "KVM: Check for duplicate vcpu_id as early as possible"
KVM: WARN if vCPU creation is in-progress when locking all vCPUs

 arch/x86/kvm/svm/sev.c   | 10 ----------
 arch/x86/kvm/vmx/tdx.c   |  5 -----
 include/linux/kvm_host.h |  1 -
 virt/kvm/kvm_main.c      | 35 +++++++++++------------------------
 4 files changed, 11 insertions(+), 40 deletions(-)

base-commit: d599822bdb66aeec5ec76297b0fc6efaaeefe07c

Signature · 1 line
2.55.0.1032.g73a4cd73de-goog
author maintainer
#1 lore raw

PATCH 1/5 KVM: Reject attempts to lock all vCPUs if vCPU creation is in-progress

Reject locking of all vCPUs if vCPU creation is in-progress, i.e. if the number of "created" vCPUs doesn't match the number of "onlined" vCPUs. It's simply not possible to guarantee that KVM has truly locked all vCPUs if one or more vCPUs are actively being created. Holding kvm->lock does prevent in-flight vCPUs from being fully onlined, but it's infeasible for common KVM to know whether or not that provides sufficient protection.

In practice, this is likely a minor bug fix for the ARM and RISC-V usage of kvm_trylock_all_vcpus(), and a glorified nop for everything else. E.g. ARM's kvm_timer_vcpu_init() can race kvm_vm_ioctl_set_counter_offset() with respect to observing KVM_ARCH_FLAG_VM_COUNTER_OFFSET.

Opportunistically drop all existing manual checks on vCPU creation being in-progress as all such checks immediately precede or follow locking of all vCPUs.

---
 arch/x86/kvm/svm/sev.c | 10 ----------
 arch/x86/kvm/vmx/tdx.c |  5 -----
 virt/kvm/kvm_main.c    |  6 ++++++
 3 files changed, 6 insertions(+), 15 deletions(-)
arch/x86/kvm/svm/sev.c +0 -10
index 5705723f1f41..068f8a236a35 100644
@@ -1125,9 +1125,6 @@ static int sev_launch_update_vmsa(struct kvm *kvm, struct kvm_sev_cmd *argp)
 	if (!sev_es_guest(kvm))
 		return -ENOTTY;
 
-	if (kvm_is_vcpu_creation_in_progress(kvm))
-		return -EBUSY;
-
 	ret = kvm_lock_all_vcpus(kvm);
 	if (ret)
 		return ret;
@@ -2115,10 +2112,6 @@ static int sev_check_source_vcpus(struct kvm *dst, struct kvm *src)
 	struct kvm_vcpu *src_vcpu;
 	unsigned long i;
 
-	if (kvm_is_vcpu_creation_in_progress(src) ||
-	    kvm_is_vcpu_creation_in_progress(dst))
-		return -EBUSY;
-
 	if (!sev_es_guest(src))
 		return 0;
 
@@ -2510,9 +2503,6 @@ static int snp_launch_update_vmsa(struct kvm *kvm, struct kvm_sev_cmd *argp)
 	unsigned long i;
 	int ret;
 
-	if (kvm_is_vcpu_creation_in_progress(kvm))
-		return -EBUSY;
-
 	ret = kvm_lock_all_vcpus(kvm);
 	if (ret)
 		return ret;
arch/x86/kvm/vmx/tdx.c +0 -5
index b272c20586a7..58c255256e4c 100644
@@ -2728,11 +2728,6 @@ static tdx_vm_state_guard_t tdx_acquire_vm_state_locks(struct kvm *kvm)
 
 	mutex_lock(&kvm->lock);
 
-	if (kvm->created_vcpus != atomic_read(&kvm->online_vcpus)) {
-		r = -EBUSY;
-		goto out_err;
-	}
-
 	r = kvm_lock_all_vcpus(kvm);
 	if (r)
 		goto out_err;
virt/kvm/kvm_main.c +6 -0
index 65eb26a0520d..78cc090435be 100644
@@ -1363,6 +1363,9 @@ int kvm_trylock_all_vcpus(struct kvm *kvm)
 
 	lockdep_assert_held(&kvm->lock);
 
+	if (kvm_is_vcpu_creation_in_progress(kvm))
+		return -EBUSY;
+
 	kvm_for_each_vcpu(i, vcpu, kvm)
 		if (!mutex_trylock_nest_lock(&vcpu->mutex, &kvm->lock))
 			goto out_unlock;
@@ -1386,6 +1389,9 @@ int kvm_lock_all_vcpus(struct kvm *kvm)
 
 	lockdep_assert_held(&kvm->lock);
 
+	if (kvm_is_vcpu_creation_in_progress(kvm))
+		return -EBUSY;
+
 	kvm_for_each_vcpu(i, vcpu, kvm) {
 		r = mutex_lock_killable_nest_lock(&vcpu->mutex, &kvm->lock);
 		if (r)
Signature · 1 line
2.55.0.1032.g73a4cd73de-goog
#2 lore raw

Re: [PATCH 1/5] KVM: Reject attempts to lock all vCPUs if vCPU creation is in-progress

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] Redundant manual check for vCPU creation left behind in ARM VGIC initialization. --

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] Redundant manual check for vCPU creation left behind in ARM VGIC initialization. --

commit 62db20a1fe5b1f9f44f668c005957e7cf68b7547
Author: Sean Christopherson <[email protected]>

KVM: Reject attempts to lock all vCPUs if vCPU creation is in-progress

This commit updates KVM to reject attempts to lock all vCPUs while vCPU creation is in progress by adding checks inside kvm_trylock_all_vcpus() and kvm_lock_all_vcpus(). It also opportunistically drops existing manual checks that precede or follow locking all vCPUs.

virt/kvm/kvm_main.c int kvm_trylock_all_vcpus(struct kvm *kvm) · line 1368

Quoted 11 lines
virt/kvm/kvm_main.c +3 -0
index 65eb26a0520d8..78cc090435be5 100644
@@ -1363,6 +1363,9 @@ int kvm_trylock_all_vcpus(struct kvm *kvm)
 
 	lockdep_assert_held(&kvm->lock);
 
+	if (kvm_is_vcpu_creation_in_progress(kvm))
+		return -EBUSY;
+

[Severity: Low]
Was the manual check in arch/arm64/kvm/vgic/vgic-init.c missed during the cleanup of redundant in-progress vCPU creation checks?

The commit message notes that existing manual checks are being dropped, but in kvm_vgic_create(), there is still a manual check immediately following the kvm_trylock_all_vcpus() call:

arch/arm64/kvm/vgic/vgic-init.c:kvm_vgic_create() {
        ...
        ret = kvm_trylock_all_vcpus(kvm);
        if (ret)
                return ret;

        ret = -EBUSY;
        if (kvm->created_vcpus != atomic_read(&kvm->online_vcpus))
                goto out_unlock;
        ...
}

Because this condition is now evaluated inside the preceding kvm_trylock_all_vcpus() call, this manual check in kvm_vgic_create() appears to be left as dead code and will never evaluate to true.

Quoted 3 lines
virt/kvm/kvm_main.c +0 -0
 	kvm_for_each_vcpu(i, vcpu, kvm)
 		if (!mutex_trylock_nest_lock(&vcpu->mutex, &kvm->lock))
 			goto out_unlock;
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
author maintainer
#3 lore raw

PATCH 2/5 KVM: Protect all of kvm_vm_ioctl_create_vcpu() with kvm->lock

In reply to Sean Christopherson

When creating a vCPU, don't drop kvm->lock to when doing the bulk of actual vCPU creation, as allowing multiple vCPUs to be created in parallel adds significant complexity in KVM (as evidenced by the many related bugs), and all known VMMs fully serialize vCPU creation.

For many years, "everyone" has assumed that dropping kvm->lock was done for performance reasons optimization, e.g. to allow userspace to create all vCPUs concurrently for latency purposes. But as above, no known VMM does that. Looking at the history of this code, before commit 11ec28047118 ("KVM: Convert vm lock to a mutex"), kvm->lock was a spinlock. I.e. KVM *had* to drop kvm->lock when doing the bulk of vCPU creation, otherwise KVM couldn't do normal memory allocations. When kvm->lock got turned into a mutex for unrelated reasons, no one took advantage updated of the change to simplify vCPU creation. And 19 years later, everyone just assumed that KVM continued to deal with the complexity for performance reasons.

Furthermore, naively parallelizing vCPU creation in userspace is likely a net negative due to the overheads of task creation. Unless a VMM carefully avoids the extra overhead related to parallelization, e.g. spawns each vCPU's thread before creating the vCPU, creating vCPUs concurrently is a net *negative* up until about ~64 vCPUs, after which the times are a wash.

The absolute speed of light _is_ faster if KVM doesn't hold kvm-lock, but at vCPU counts of ~16 or less, it's probably in the noise when considering total VM creation time, as the added latency is less than 1ms up until 16 or so vCPUs.

On top of all that, KVM has had a *lot* of fatal bugs (most often found by syzkaller) related to vCPUs being created while trying to do per-VM operations (basically, see every flow that locks all vCPUs). I.e. the parallel vCPU creation "support" is actively harmful as the only "use case" is for misbehaving userspace to exploit KVM bugs.

Serializing vCPU creation will allow reverting commit 97d65b544f48 ("KVM:
Check for duplicate vcpu_id as early as possible"), which had "minor" math
error: the worst case scenario isn't "256 bytes per VM", it's "256 unsigned longs per VM", i.e. 2048 bytes per VM, which doubles the size of each VM and pushes several architectures into order-1 allocations.

---
 virt/kvm/kvm_main.c | 22 +++++-----------------
 1 file changed, 5 insertions(+), 17 deletions(-)
virt/kvm/kvm_main.c +5 -17
index 78cc090435be..c17cc8dd371b 100644
@@ -4165,6 +4165,8 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id)
 	struct kvm_vcpu *vcpu;
 	struct page *page;
 
+	guard(mutex)(&kvm->lock);
+
 	/*
 	 * KVM tracks vCPU IDs as 'int', be kind to userspace and reject
 	 * too-large values instead of silently truncating.
@@ -4177,26 +4179,18 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id)
 	if (id >= KVM_MAX_VCPU_IDS)
 		return -EINVAL;
 
-	mutex_lock(&kvm->lock);
-	if (kvm->created_vcpus >= kvm->max_vcpus) {
-		mutex_unlock(&kvm->lock);
+	if (kvm->created_vcpus >= kvm->max_vcpus)
 		return -EINVAL;
-	}
 
-	if (test_bit(id, kvm->vcpu_ids)) {
-		mutex_unlock(&kvm->lock);
+	if (test_bit(id, kvm->vcpu_ids))
 		return -EEXIST;
-	}
 
 	r = kvm_arch_vcpu_precreate(kvm, id);
-	if (r) {
-		mutex_unlock(&kvm->lock);
+	if (r)
 		return r;
-	}
 
 	kvm->created_vcpus++;
 	__set_bit(id, kvm->vcpu_ids);
-	mutex_unlock(&kvm->lock);
 
 	vcpu = kmem_cache_zalloc(kvm_vcpu_cache, GFP_KERNEL_ACCOUNT);
 	if (!vcpu) {
@@ -4227,8 +4221,6 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id)
 			goto arch_vcpu_destroy;
 	}
 
-	mutex_lock(&kvm->lock);
-
 	if (WARN_ON_ONCE(kvm_get_vcpu_by_id(kvm, id))) {
 		r = -EEXIST;
 		goto unlock_vcpu_destroy;
@@ -4267,7 +4259,6 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id)
 	atomic_inc(&kvm->online_vcpus);
 	mutex_unlock(&vcpu->mutex);
 
-	mutex_unlock(&kvm->lock);
 	kvm_arch_vcpu_postcreate(vcpu);
 	kvm_create_vcpu_debugfs(vcpu);
 	return r;
@@ -4278,7 +4269,6 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id)
 	xa_erase(&kvm->vcpu_array, vcpu->vcpu_idx);
 unlock_vcpu_destroy:
 	vcpu->vcpu_idx = -1;
-	mutex_unlock(&kvm->lock);
 	kvm_dirty_ring_free(&vcpu->dirty_ring);
 arch_vcpu_destroy:
 	kvm_arch_vcpu_destroy(vcpu);
@@ -4287,10 +4277,8 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id)
 vcpu_free:
 	kmem_cache_free(kvm_vcpu_cache, vcpu);
 vcpu_decrement:
-	mutex_lock(&kvm->lock);
 	kvm->created_vcpus--;
 	__clear_bit(id, kvm->vcpu_ids);
-	mutex_unlock(&kvm->lock);
 	return r;
 }
 
Signature · 1 line
2.55.0.1032.g73a4cd73de-goog
#4 lore raw

Re: [PATCH 2/5] KVM: Protect all of kvm_vm_ioctl_create_vcpu() with kvm->lock

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [Critical] Self-deadlock in KVM vCPU creation due to recursive `kvm->lock` acquisition on PowerPC and…

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Critical] Self-deadlock in KVM vCPU creation due to recursive kvm->lock acquisition on PowerPC and s390.
- [Low] Mixing goto-based error handling and guard() cleanup in the same function.
--

commit de58d2836963a7ea1b99cd6c2ad5be4f18984b33
Author: Sean Christopherson <[email protected]>
KVM: Protect all of kvm_vm_ioctl_create_vcpu() with kvm->lock

This commit serializes vCPU creation by holding kvm->lock throughout the entirety of kvm_vm_ioctl_create_vcpu(). It prevents concurrent vCPU creations to simplify the code and prevent race conditions.

virt/kvm/kvm_main.c static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id) · line 4169

Quoted 10 lines
virt/kvm/kvm_main.c +2 -0
index 78cc090435be5..c17cc8dd371b6 100644
@@ -4165,6 +4165,8 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id)
 	struct kvm_vcpu *vcpu;
 	struct page *page;
 
+	guard(mutex)(&kvm->lock);
+

[Severity: Critical]
Could this cause a self-deadlock on PowerPC and s390? The guard acquires kvm->lock for the entire scope of kvm_vm_ioctl_create_vcpu(), but this function subsequently calls architecture-specific hooks like kvm_arch_vcpu_create() and kvm_arch_vcpu_postcreate().

On PowerPC, kvm_arch_vcpu_create() eventually calls kvmppc_core_vcpu_create_hv() in arch/powerpc/kvm/book3s_hv.c, which explicitly attempts to acquire kvm->lock via mutex_lock(&kvm->lock).

Similarly, on s390, both kvm_s390_vcpu_setup() and kvm_arch_vcpu_postcreate() in arch/s390/kvm/s390/s390.c call mutex_lock(&vcpu->kvm->lock).

Since standard Linux mutexes are not recursive, it appears any userspace process invoking the KVM_CREATE_VCPU ioctl on these architectures will indefinitely hang the creating thread.

[ ... ]

Quoted 9 lines
virt/kvm/kvm_main.c +0 -1
@@ -4267,7 +4259,6 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id)
 	atomic_inc(&kvm->online_vcpus);
 	mutex_unlock(&vcpu->mutex);
 
-	mutex_unlock(&kvm->lock);
 	kvm_arch_vcpu_postcreate(vcpu);
 	kvm_create_vcpu_debugfs(vcpu);
 	return r;

virt/kvm/kvm_main.c static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id) · line 4273

Quoted 6 lines
virt/kvm/kvm_main.c +0 -1
@@ -4278,7 +4269,6 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id)
 	xa_erase(&kvm->vcpu_array, vcpu->vcpu_idx);
 unlock_vcpu_destroy:
 	vcpu->vcpu_idx = -1;
-	mutex_unlock(&kvm->lock);
 	kvm_dirty_ring_free(&vcpu->dirty_ring);
 arch_vcpu_destroy:

[Severity: Low]
Does this mix goto-based error handling and scope-based cleanup within the same function? The cleanup subsystem guidelines specify that a function should either use goto for cleanup or scope-based cleanup helpers, but never both.

While the lock is safely released on return here, introducing guard(mutex)(&kvm->lock) at the top of kvm_vm_ioctl_create_vcpu() while retaining manual goto cleanup labels like unlock_vcpu_destroy creates confusing ownership semantics.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
author maintainer
#5 lore raw

PATCH 3/5 KVM: Move check for existing vCPU ID to the top of vCPU creation

In reply to Sean Christopherson

Now that kvm->lock is held for the entirety of vCPU creation, check for a conflicting vCPU ID at the begnning of vCPU creation, before the arch precreate() hook is invoked. This will allow reverting commit 97d65b544f48 ("KVM: Check for duplicate vcpu_id as early as possible"). For now, keep the redundant vcpu_ids tracking as a sanity check.

No functional change intended (absent KVM bugs, checking vcpu_ids and walking kvm_get_vcpu_by_id() should yield the same result).

---
 virt/kvm/kvm_main.c | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)
virt/kvm/kvm_main.c +4 -6
index c17cc8dd371b..d5524ac8c5cf 100644
@@ -4182,7 +4182,10 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id)
 	if (kvm->created_vcpus >= kvm->max_vcpus)
 		return -EINVAL;
 
-	if (test_bit(id, kvm->vcpu_ids))
+	if (kvm_get_vcpu_by_id(kvm, id))
+		return -EEXIST;
+
+	if (WARN_ON_ONCE(test_bit(id, kvm->vcpu_ids)))
 		return -EEXIST;
 
 	r = kvm_arch_vcpu_precreate(kvm, id);
@@ -4221,11 +4224,6 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id)
 			goto arch_vcpu_destroy;
 	}
 
-	if (WARN_ON_ONCE(kvm_get_vcpu_by_id(kvm, id))) {
-		r = -EEXIST;
-		goto unlock_vcpu_destroy;
-	}
-
 	/*
 	 * Set the vCPU's index *before* the vCPU is reachable by other tasks.
 	 * Unwind the index back to -1 on failure so that KVM can use the index
Signature · 1 line
2.55.0.1032.g73a4cd73de-goog
author maintainer
#6 lore raw

PATCH 4/5 Revert "KVM: Check for duplicate vcpu_id as early as possible"

In reply to Sean Christopherson

Now that KVM uses kvm_get_vcpu_by_id() to check for an existing vCPU ID before doing any meaningful work, which was made possible by holding kvm->lock for the entirety of vCPU creation, revert the now-redundant "early" vCPU ID tracking. The claims about the impact of kvm->vcpu_ids on the memory footprint were a wee bit wrong: the worst case scenario isn't 256 bytes per VM, it's 256 "unsigned longs" per VM, i.e. 2048 bytes per VM.

Increasing the size of "struct kvm" by 2048 nearly doubled the total size on many architectures, and tripped x86's KVM_SANITY_CHECK_VM_STRUCT_SIZE, which was added to detect this *exact* scenario, where a single change significantly increased the size of "struct kvm". I.e. attempting to build KVM with CONFIG_DEBUG_KERNEL=n fails on x86 (the build failures got missed because all build bots apparently test only CONFIG_DEBUG_KERNEL=y kernels, and maintainers' test flows were similarly lacking).

This reverts commit 97d65b544f48b2ee49f6aea32145e3e7969955dc.

---
 include/linux/kvm_host.h | 1 -
 virt/kvm/kvm_main.c      | 5 -----
 2 files changed, 6 deletions(-)
include/linux/kvm_host.h +0 -1
index 03bfc92864b6..6aab167bf482 100644
@@ -791,7 +791,6 @@ struct kvm {
 	/* The current active memslot set for each address space */
 	struct kvm_memslots __rcu *memslots[KVM_MAX_NR_ADDRESS_SPACES];
 	struct xarray vcpu_array;
-	DECLARE_BITMAP(vcpu_ids, KVM_MAX_VCPU_IDS);
 	/*
 	 * Protected by slots_lock, but can be read outside if an
 	 * incorrect answer is acceptable.
virt/kvm/kvm_main.c +0 -5
index d5524ac8c5cf..985af39b980a 100644
@@ -4185,15 +4185,11 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id)
 	if (kvm_get_vcpu_by_id(kvm, id))
 		return -EEXIST;
 
-	if (WARN_ON_ONCE(test_bit(id, kvm->vcpu_ids)))
-		return -EEXIST;
-
 	r = kvm_arch_vcpu_precreate(kvm, id);
 	if (r)
 		return r;
 
 	kvm->created_vcpus++;
-	__set_bit(id, kvm->vcpu_ids);
 
 	vcpu = kmem_cache_zalloc(kvm_vcpu_cache, GFP_KERNEL_ACCOUNT);
 	if (!vcpu) {
@@ -4276,7 +4272,6 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id)
 	kmem_cache_free(kvm_vcpu_cache, vcpu);
 vcpu_decrement:
 	kvm->created_vcpus--;
-	__clear_bit(id, kvm->vcpu_ids);
 	return r;
 }
 
Signature · 1 line
2.55.0.1032.g73a4cd73de-goog
author maintainer
#7 lore raw

PATCH 5/5 KVM: WARN if vCPU creation is in-progress when locking all vCPUs

In reply to Sean Christopherson

Now that KVM holds kvm->lock for the entirety of vCPU creation, from when created_vcpus is incremented until the new vCPU is fully onlined, WARN if the impossible happens and KVM somehow sees a discrepancy between the number of vCPUs "created" and "onlined".

---
 virt/kvm/kvm_main.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
virt/kvm/kvm_main.c +2 -2
index 985af39b980a..e66d9761ee49 100644
@@ -1363,7 +1363,7 @@ int kvm_trylock_all_vcpus(struct kvm *kvm)
 
 	lockdep_assert_held(&kvm->lock);
 
-	if (kvm_is_vcpu_creation_in_progress(kvm))
+	if (WARN_ON_ONCE(kvm_is_vcpu_creation_in_progress(kvm)))
 		return -EBUSY;
 
 	kvm_for_each_vcpu(i, vcpu, kvm)
@@ -1389,7 +1389,7 @@ int kvm_lock_all_vcpus(struct kvm *kvm)
 
 	lockdep_assert_held(&kvm->lock);
 
-	if (kvm_is_vcpu_creation_in_progress(kvm))
+	if (WARN_ON_ONCE(kvm_is_vcpu_creation_in_progress(kvm)))
 		return -EBUSY;
 
 	kvm_for_each_vcpu(i, vcpu, kvm) {
Signature · 1 line
2.55.0.1032.g73a4cd73de-goog
#8 lore raw
In reply to Sean Christopherson

Am 14.09.26 um 20:12 schrieb Sean Christopherson:

Quoted 12 lines

Serialize vCPU creation by holding kvm->lock for the entirety of kvm_vm_ioctl_create_vcpu(), and then revert the now-redundant tracking adding by commit 97d65b544f48 ("KVM: Check for duplicate vcpu_id as early as possible"). I botched the math when justifying the vcpu_ids tracking; it's not an extra 256 bytes, it's an extra 2048 bytes. Roughly doubling the size of "struct kvm" tripped x86's KVM_SANITY_CHECK_VM_STRUCT_SIZE, and obviously isn't something we want to do in general.

The TL;DR of why it's a-ok to serialize vCPU creation is that no VMM actually does parallel vCPU creation. As with so many things, KVM's current behavior is the result of decades-old cruft, not intentional, deliberate design.

Quoted 7 lines

Patch 1 is a tangentially related bug fix; I included it here because holding kvm->lock for all of vCPU creation allows WARNing if KVM attempts to lock all vCPUs if vCPU creation is in-progress (the caller is must hold kvm->lock).

Sean Christopherson (5):
KVM: Reject attempts to lock all vCPUs if vCPU creation is in-progress
KVM: Protect all of kvm_vm_ioctl_create_vcpu() with kvm->lock

Interesting, that would allow to simplify several aspects in s390 kvm code as well. We might also be able to move most things from postcreate into create.

Before that, the series as is needs some s390 fixups.
s390 takes the kvm->lock in
- kvm_s390_vcpu_setup() locks kvm->lock around kvm_s390_pv_create_cpu() at arch/s390/kvm/s390/s390.c:3760 (added with 29b40f105ec8)
- kvm_arch_vcpu_postcreate() locks kvm->lock around the epoch copy at arch/s390/kvm/s390/s390.c:3582. That dates back to the TOD attribute commit 72f250206f0f. as far as I can tell.

#10 lore raw
In reply to Sean Christopherson
Quoted 7 lines · Sean Christopherson

Serialize vCPU creation by holding kvm->lock for the entirety of kvm_vm_ioctl_create_vcpu(), and then revert the now-redundant tracking adding by commit 97d65b544f48 ("KVM: Check for duplicate vcpu_id as early as possible"). I botched the math when justifying the vcpu_ids tracking; it's not an extra 256 bytes, it's an extra 2048 bytes. Roughly doubling the size of "struct kvm" tripped x86's KVM_SANITY_CHECK_VM_STRUCT_SIZE, and obviously isn't something we want to do in general.

Hi Sean! Thx for handling this and sorry for pestering you, but I have to ask, as this fixes a regression that was already reported two times and introduced this cycle:

What's the plan to merge this? Is this aimed for the current or the next merge window? Ciao, Thorsten

Quoted 23 lines

The TL;DR of why it's a-ok to serialize vCPU creation is that no VMM actually does parallel vCPU creation. As with so many things, KVM's current behavior is the result of decades-old cruft, not intentional, deliberate design.

Patch 1 is a tangentially related bug fix; I included it here because holding kvm->lock for all of vCPU creation allows WARNing if KVM attempts to lock all vCPUs if vCPU creation is in-progress (the caller is must hold kvm->lock).

Sean Christopherson (5):
KVM: Reject attempts to lock all vCPUs if vCPU creation is in-progress
KVM: Protect all of kvm_vm_ioctl_create_vcpu() with kvm->lock
KVM: Move check for existing vCPU ID to the top of vCPU creation Revert "KVM: Check for duplicate vcpu_id as early as possible"
KVM: WARN if vCPU creation is in-progress when locking all vCPUs

 arch/x86/kvm/svm/sev.c   | 10 ----------
 arch/x86/kvm/vmx/tdx.c   |  5 -----
 include/linux/kvm_host.h |  1 -
 virt/kvm/kvm_main.c      | 35 +++++++++++------------------------
 4 files changed, 11 insertions(+), 40 deletions(-)

base-commit: d599822bdb66aeec5ec76297b0fc6efaaeefe07c

maintainer
#11 lore raw
Quoted 9 lines · Thorsten Leemhuis
Quoted 7 lines · Sean Christopherson

Serialize vCPU creation by holding kvm->lock for the entirety of kvm_vm_ioctl_create_vcpu(), and then revert the now-redundant tracking adding by commit 97d65b544f48 ("KVM: Check for duplicate vcpu_id as early as possible"). I botched the math when justifying the vcpu_ids tracking; it's not an extra 256 bytes, it's an extra 2048 bytes. Roughly doubling the size of "struct kvm" tripped x86's KVM_SANITY_CHECK_VM_STRUCT_SIZE, and obviously isn't something we want to do in general.

Quoted 6 lines

Hi Sean! Thx for handling this and sorry for pestering you, but I have to ask, as this fixes a regression that was already reported two times and introduced this cycle:

What's the plan to merge this? Is this aimed for the current or the next merge window? Ciao, Thorsten

For this one, but after fixing the PPC and s390 cases. Sean, if you're busy I can send the v2 myself.

Paolo

Quoted 25 lines
Quoted 23 lines

The TL;DR of why it's a-ok to serialize vCPU creation is that no VMM actually does parallel vCPU creation. As with so many things, KVM's current behavior is the result of decades-old cruft, not intentional, deliberate design.

Patch 1 is a tangentially related bug fix; I included it here because holding kvm->lock for all of vCPU creation allows WARNing if KVM attempts to lock all vCPUs if vCPU creation is in-progress (the caller is must hold kvm->lock).

Sean Christopherson (5):
KVM: Reject attempts to lock all vCPUs if vCPU creation is in-progress
KVM: Protect all of kvm_vm_ioctl_create_vcpu() with kvm->lock
KVM: Move check for existing vCPU ID to the top of vCPU creation Revert "KVM: Check for duplicate vcpu_id as early as possible"
KVM: WARN if vCPU creation is in-progress when locking all vCPUs

  arch/x86/kvm/svm/sev.c   | 10 ----------
  arch/x86/kvm/vmx/tdx.c   |  5 -----
  include/linux/kvm_host.h |  1 -
  virt/kvm/kvm_main.c      | 35 +++++++++++------------------------
  4 files changed, 11 insertions(+), 40 deletions(-)

base-commit: d599822bdb66aeec5ec76297b0fc6efaaeefe07c

author maintainer
#12 lore raw
Quoted 10 lines · Paolo Bonzini
Quoted 9 lines · Thorsten Leemhuis
Quoted 7 lines · Sean Christopherson

Serialize vCPU creation by holding kvm->lock for the entirety of kvm_vm_ioctl_create_vcpu(), and then revert the now-redundant tracking adding by commit 97d65b544f48 ("KVM: Check for duplicate vcpu_id as early as possible"). I botched the math when justifying the vcpu_ids tracking; it's not an extra 256 bytes, it's an extra 2048 bytes. Roughly doubling the size of "struct kvm" tripped x86's KVM_SANITY_CHECK_VM_STRUCT_SIZE, and obviously isn't something we want to do in general.

Quoted 8 lines
Quoted 6 lines

Hi Sean! Thx for handling this and sorry for pestering you, but I have to ask, as this fixes a regression that was already reported two times and introduced this cycle:

What's the plan to merge this? Is this aimed for the current or the next merge window? Ciao, Thorsten

For this one, but after fixing the PPC and s390 cases.

Ya.

Quoted 1 line

Sean, if you're busy I can send the v2 myself.

I'll get it out today. I was traveling last week and got behind one a "few" things.