KVM: VMX: Postpone IPIv setup after successful vCPU creation
PATCH 1/1 KVM: VMX: Postpone IPIv setup after successful vCPU creation
vCPU creation in kvm_vm_ioctl_create_vcpu() may fail after kvm_arch_vcpu_create() -> vmx_vcpu_create() already succeeded. In such case kvm_vm_ioctl_create_vcpu() destroys the newly created vCPU in the failure path. However, that leaves a side effect: the IPIv pid_table entry remains configured with this vCPU's pi_desc address. As a result, when another vCPU sends an IPI to the APIC ID of this failed-to-create vCPU, it will cause HW to write to this (freed!) pi_desc memory. [*]
An easy fix would be to clear the pid_table entry in vmx_vcpu_free(). However that would be still problematic, for the following reason:
userspace may try to create a vCPU with the same vcpu_id as an existing one; vmx_vcpu_create() will succeed, and only after that kvm_vm_ioctl_create_vcpu() will check for the duplicate vcpu_id and fail with -EEXIST and then free the vCPU in the failure path. So in this failure path, vmx_vcpu_free() would clear the pid_table entry for that already existing good vCPU, i.e. effectively disable IPIv for that vCPU.
So instead fix the issue by moving the pid_table entry setup from .vcpu_create() to the newly introduced .vcpu_postcreate() callback called at the end of kvm_vm_ioctl_create_vcpu(), when we are sure that the vCPU creation succeeded.
[*] Although, since this memory is freed into the kvm_vcpu_cache kmem
cache which is only used for allocating kvm_vcpus, _maybe_ this memory will only be reused for pi_desc of another vCPU, not for anything else. So _maybe_ this will only result in delivering the IPI to a wrong vCPU (possibly of another VM) in the worst case, not in a random corruption of kernel memory.
--- arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/vmx/main.c | 9 +++++++++ arch/x86/kvm/vmx/vmx.c | 11 +++++++---- arch/x86/kvm/vmx/x86_ops.h | 1 + arch/x86/kvm/x86.c | 1 + 6 files changed, 20 insertions(+), 4 deletions(-)
arch/x86/include/asm/kvm-x86-ops.h +1 -0
@@ -25,6 +25,7 @@ KVM_X86_OP_OPTIONAL(vm_destroy) KVM_X86_OP_OPTIONAL(vm_pre_destroy) KVM_X86_OP_OPTIONAL_RET0(vcpu_precreate) KVM_X86_OP(vcpu_create) +KVM_X86_OP_OPTIONAL(vcpu_postcreate) KVM_X86_OP(vcpu_free) KVM_X86_OP(vcpu_reset) KVM_X86_OP(prepare_switch_to_guest)
arch/x86/include/asm/kvm_host.h +1 -0
@@ -1803,6 +1803,7 @@ struct kvm_x86_ops {/* Create, but do not attach this VCPU */ int (*vcpu_precreate)(struct kvm *kvm);int (*vcpu_create)(struct kvm_vcpu *vcpu); +void (*vcpu_postcreate)(struct kvm_vcpu *vcpu);void (*vcpu_free)(struct kvm_vcpu *vcpu);void (*vcpu_reset)(struct kvm_vcpu *vcpu,bool init_event);
arch/x86/kvm/vmx/main.c +9 -0
@@ -78,6 +78,14 @@ static int vt_vcpu_create(struct kvm_vcpu *vcpu)return vmx_vcpu_create (vcpu); } +static void vt_vcpu_postcreate (struct kvm_vcpu *vcpu) +{ +if (is_td_vcpu (vcpu)) +return ; + +vmx_vcpu_postcreate (vcpu); +} +static void vt_vcpu_free (struct kvm_vcpu *vcpu) {if (is_td_vcpu (vcpu)) {
@@ -898,6 +906,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .vcpu_precreate = vt_op(vcpu_precreate), .vcpu_create = vt_op(vcpu_create), + .vcpu_postcreate = vt_op(vcpu_postcreate), .vcpu_free = vt_op(vcpu_free), .vcpu_reset = vt_op(vcpu_reset),
arch/x86/kvm/vmx/vmx.c +7 -4
@@ -7752,10 +7752,6 @@ int vmx_vcpu_create(struct kvm_vcpu *vcpu) vmx->ve_info =page_to_virt (page); } -if (vmx_can_use_ipiv (vcpu)) -WRITE_ONCE (to_kvm_vmx (vcpu->kvm)->pid_table[vcpu->vcpu_id], -__pa (&vmx->vt.pi_desc) |PID_TABLE_ENTRY_VALID ); -return 0 ; free_vmcs:
@@ -7767,6 +7763,13 @@ int vmx_vcpu_create(struct kvm_vcpu *vcpu)return err; } +void vmx_vcpu_postcreate (struct kvm_vcpu *vcpu) +{ +if (vmx_can_use_ipiv (vcpu)) +WRITE_ONCE (to_kvm_vmx (vcpu->kvm)->pid_table[vcpu->vcpu_id], +__pa (&to_vmx (vcpu)->vt.pi_desc) |PID_TABLE_ENTRY_VALID ); +} +#define L1TF_MSG_SMT "L1TF CPU bug present and SMT on, data leak possible. See CVE-2018-3646 and https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/l1tf.html for details.\n"#define L1TF_MSG_L1D "L1TF CPU bug present and virtualization mitigation disabled, data leak possible. See CVE-2018-3646 and https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/l1tf.html for details.\n"
arch/x86/kvm/vmx/x86_ops.h +1 -0
@@ -21,6 +21,7 @@ int vmx_vm_init(struct kvm *kvm);void vmx_vm_destroy (struct kvm *kvm);int vmx_vcpu_precreate (struct kvm *kvm);int vmx_vcpu_create (struct kvm_vcpu *vcpu); +void vmx_vcpu_postcreate (struct kvm_vcpu *vcpu);int vmx_vcpu_pre_run (struct kvm_vcpu *vcpu);fastpath_t vmx_vcpu_run (struct kvm_vcpu *vcpu,u64 run_flags);void vmx_vcpu_free (struct kvm_vcpu *vcpu);
arch/x86/kvm/x86.c +1 -0
@@ -12873,6 +12873,7 @@ void kvm_arch_vcpu_postcreate(struct kvm_vcpu *vcpu)return ;vcpu_load (vcpu);kvm_synchronize_tsc (vcpu,NULL ); +kvm_x86_call (vcpu_postcreate)(vcpu);vcpu_put (vcpu);/* poll control enabled by default */
Signature · 1 line
2.55.0.141.g00534a21ce-goog
[email protected]
AI reviewThank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [Critical] The patch fixes a use-after-free in VMX IPI virtualization caused by KVM_CREATE_VCPU destroying a vCPU on duplicate ID, but fails to fix the exact same issue in AMD SVM AVIC. --
--- Patch [1]: [PATCH] KVM: VMX: Postpone IPIv setup after successful vCPU creation --- commit c59610ef1e5c50c0016e2a7b4db23402a3147ffa
Author: Dmytro Maluka <[email protected]>
KVM: VMX: Postpone IPIv setup after successful vCPU creation
vCPU creation in kvm_vm_ioctl_create_vcpu() may fail after kvm_arch_vcpu_create() -> vmx_vcpu_create() already succeeded. In such case kvm_vm_ioctl_create_vcpu() destroys the newly created vCPU in the failure path. However, that leaves a side effect: the IPIv pid_table entry remains configured with this vCPU's pi_desc address.
arch/x86/include/asm/kvm-x86-ops.h KVM_X86_OP_OPTIONAL(vm_destroy) · line 27
Quoted 9 lines
arch/x86/include/asm/kvm-x86-ops.h +1
-0@@ -24,6 +24,7 @@ KVM_X86_OP_OPTIONAL(vm_destroy) KVM_X86_OP_OPTIONAL(vm_pre_destroy) KVM_X86_OP_OPTIONAL_RET0(vcpu_precreate) KVM_X86_OP(vcpu_create) +KVM_X86_OP_OPTIONAL(vcpu_postcreate)
[Severity: Critical]
This isn't a bug introduced by this patch, but does this exact same vulnerability exist in the AMD SVM AVIC code?
It looks like the SVM code suffers from the same use-after-free pattern when userspace creates a duplicate vCPU ID.
In arch/x86/kvm/svm/avic.c:avic_init_backing_page(), the newly allocated APIC backing page address is written into the VM-wide hardware IPI table before the vCPU ID duplicate check:
WRITE_ONCE(kvm_svm->avic_physical_id_table[id], new_entry);
If the vCPU ID check later fails in kvm_vm_ioctl_create_vcpu(), the new vCPU is destroyed and kvm_free_lapic() frees the backing page:
free_page((unsigned long)apic->regs);
However, the AVIC table is never updated to revert the overwrite. This leaves the physical address of the freed page in the hardware table, which could cause a hardware use-after-free if a guest subsequently sends an IPI to that ID.
Could we also implement vcpu_postcreate for SVM to postpone the AVIC table update, or do we need a separate fix for AMD?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Quoted 6 lines · On Thu, Jul 16, 2026 at 04:29:00PM +0000, [email protected] wrote:
[Severity: Critical]
This isn't a bug introduced by this patch, but does this exact same vulnerability exist in the AMD SVM AVIC code?It looks like the SVM code suffers from the same use-after-free pattern when userspace creates a duplicate vCPU ID.
Seems plausible. Can someone with AMD expertise look into this?
Quoted 21 lines
In arch/x86/kvm/svm/avic.c:avic_init_backing_page(), the newly allocated APIC backing page address is written into the VM-wide hardware IPI table before the vCPU ID duplicate check:
WRITE_ONCE(kvm_svm->avic_physical_id_table[id], new_entry);If the vCPU ID check later fails in
kvm_vm_ioctl_create_vcpu(), the new vCPU is destroyed andkvm_free_lapic()frees the backing page:free_page((unsigned long)apic->regs);However, the AVIC table is never updated to revert the overwrite. This leaves the physical address of the freed page in the hardware table, which could cause a hardware use-after-free if a guest subsequently sends an IPI to that ID.
Could we also implement
vcpu_postcreatefor SVM to postpone the AVIC table update, or do we need a separate fix for AMD?Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Quoted 15 lines · Dmytro Maluka
Quoted 6 lines · On Thu, Jul 16, 2026 at 04:29:00PM +0000, [email protected] wrote:
[Severity: Critical]
This isn't a bug introduced by this patch, but does this exact same vulnerability exist in the AMD SVM AVIC code?It looks like the SVM code suffers from the same use-after-free pattern when userspace creates a duplicate vCPU ID.
Seems plausible. Can someone with AMD expertise look into this?
Quoted 5 lines
In arch/x86/kvm/svm/avic.c:avic_init_backing_page(), the newly allocated APIC backing page address is written into the VM-wide hardware IPI table before the vCPU ID duplicate check:
WRITE_ONCE(kvm_svm->avic_physical_id_table[id], new_entry);
Yes, this write (or just setting AVIC_PHYSICAL_ID_ENTRY_VALID_MASK) will need to be delayed to a post-create phase similar to VMX.
The alternative is to simply skip setting the entry valid during vCPU creation and to instead do it unconditionally during vcpu_load(), where we anyway have to update the entry.
Quoted 6 lines
Quoted 6 lines
If the vCPU ID check later fails in
kvm_vm_ioctl_create_vcpu(), the new vCPU is destroyed andkvm_free_lapic()frees the backing page:free_page((unsigned long)apic->regs);
Quoted 7 lines
Quoted 7 lines
However, the AVIC table is never updated to revert the overwrite. This leaves the physical address of the freed page in the hardware table, which could cause a hardware use-after-free if a guest subsequently sends an IPI to that ID.
Could we also implement
vcpu_postcreatefor SVM to postpone the AVIC table update, or do we need a separate fix for AMD?
Thanks,
Naveen
Sean Christopherson
maintainerQuoted 15 lines · Naveen N Rao
Quoted 14 lines · Dmytro Maluka
Quoted 6 lines · On Thu, Jul 16, 2026 at 04:29:00PM +0000, [email protected] wrote:
[Severity: Critical]
This isn't a bug introduced by this patch, but does this exact same vulnerability exist in the AMD SVM AVIC code?It looks like the SVM code suffers from the same use-after-free pattern when userspace creates a duplicate vCPU ID.
Seems plausible. Can someone with AMD expertise look into this?
Quoted 4 lines
In arch/x86/kvm/svm/avic.c:avic_init_backing_page(), the newly allocated APIC backing page address is written into the VM-wide hardware IPI table before the vCPU ID duplicate check:
Quoted 8 lines
Quoted 1 line
Quoted 1 line
WRITE_ONCE(kvm_svm->avic_physical_id_table[id], new_entry);Yes, this write (or just setting
AVIC_PHYSICAL_ID_ENTRY_VALID_MASK) will need to be delayed to a post-create phase similar to VMX.The alternative is to simply skip setting the entry valid during vCPU creation and to instead do it unconditionally during
vcpu_load(), where we anyway have to update the entry.
Won't work, vcpu_load() is called during vCPU creation. :-(
Quoted 9 lines
An easy fix would be to clear the
pid_tableentry invmx_vcpu_free(). However that would be still problematic, for the following reason:
userspace may try to create a vCPU with the samevcpu_idas an existing one;vmx_vcpu_create()will succeed, and only after thatkvm_vm_ioctl_create_vcpu()will check for the duplicatevcpu_idand fail with-EEXISTand then free the vCPU in the failure path. So in this failure path,vmx_vcpu_free()would clear thepid_tableentry for that already existing good vCPU, i.e. effectively disable IPIv for that vCPU.
IMHO this seems a bit fragile? If something similar to pid_table coming up in the future, we could end up with a similar problem.
The "duplicated vcpu_id check" seems the ones that should happen as early as possible. Is it better to move the "duplicated vcpu_id check" earlier before vmx_vcpu_create(), e.g., even before the kvm_arch_vcpu_precreate()? In this case we can do the pid_table cleanup in vmx_vcpu_free() I think.
The downside is this is common KVM code change for all archs, but it doesn't seem this will bring problem to other archs.
Quoted 10 lines · Kai
Quoted 9 lines
An easy fix would be to clear the
pid_tableentry invmx_vcpu_free(). However that would be still problematic, for the following reason:
userspace may try to create a vCPU with the samevcpu_idas an existing one;vmx_vcpu_create()will succeed, and only after thatkvm_vm_ioctl_create_vcpu()will check for the duplicatevcpu_idand fail with-EEXISTand then free the vCPU in the failure path. So in this failure path,vmx_vcpu_free()would clear thepid_tableentry for that already existing good vCPU, i.e. effectively disable IPIv for that vCPU.
Quoted 7 lines
IMHO this seems a bit fragile? If something similar to
pid_tablecoming up in the future, we could end up with a similar problem.The "duplicated
vcpu_idcheck" seems the ones that should happen as early as possible. Is it better to move the "duplicatedvcpu_idcheck" earlier beforevmx_vcpu_create(), e.g., even before thekvm_arch_vcpu_precreate()? In this case we can do thepid_tablecleanup invmx_vcpu_free()I think.
Unfortunately it is not that simple. As I understand, the reason why the "duplicated vcpu_id check" is done later is that it needs to be done atomically together with inserting the vCPU into kvm->vcpu_array after it, i.e. both the check and the insertion need to be done with kvm->lock held (rather than releasing kvm->lock and taking it again between the two operations).
IOW, if we just move the "duplicated vcpu_id check" earlier (before the first unlock of kvm->lock), we have a race:
1. vCPU A is being created but not installed in kvm->vcpu_array yet.
2. vCPU B with the same vcpu_id is being created. It passes the duplicated vcpu_id check, since the check doesn't find vCPU A in kvm->vcpu_array.
3. vCPU A is installed in kvm->vcpu_array, vCPU creation succeeds.
4. vCPU B with the same vcpu_id is installed in kvm->vcpu_array, vCPU creation succeeds.
And we cannot just move the kvm->vcpu_array insertion earlier (before the first unlock of kvm->lock), at least because the vCPU is not even allocated at that point.
I guess we could track the used vcpu_ids separately in another xarray (or a bitmap) which could be checked and updated by the early "duplicated vcpu_id check" before the first unlock of kvm->lock. But do we want to pay the memory price for that?
Huang, Kai
Reviewed-byQuoted 37 lines · Dmytro Maluka
Quoted 10 lines · Kai
Quoted 9 lines
An easy fix would be to clear the
pid_tableentry invmx_vcpu_free(). However that would be still problematic, for the following reason:
userspace may try to create a vCPU with the samevcpu_idas an existing one;vmx_vcpu_create()will succeed, and only after thatkvm_vm_ioctl_create_vcpu()will check for the duplicatevcpu_idand fail with-EEXISTand then free the vCPU in the failure path. So in this failure path,vmx_vcpu_free()would clear thepid_tableentry for that already existing good vCPU, i.e. effectively disable IPIv for that vCPU.Quoted 7 lines
IMHO this seems a bit fragile? If something similar to
pid_tablecoming up in the future, we could end up with a similar problem.The "duplicated
vcpu_idcheck" seems the ones that should happen as early as possible. Is it better to move the "duplicatedvcpu_idcheck" earlier beforevmx_vcpu_create(), e.g., even before thekvm_arch_vcpu_precreate()? In this case we can do thepid_tablecleanup invmx_vcpu_free()I think.Unfortunately it is not that simple. As I understand, the reason why the "duplicated
vcpu_idcheck" is done later is that it needs to be done atomically together with inserting the vCPU intokvm->vcpu_arrayafter it, i.e. both the check and the insertion need to be done withkvm->lockheld (rather than releasingkvm->lockand taking it again between the two operations).IOW, if we just move the "duplicated
vcpu_idcheck" earlier (before the first unlock ofkvm->lock), we have a race:1. vCPU A is being created but not installed in
kvm->vcpu_arrayyet.
2. vCPU B with the samevcpu_idis being created. It passes the duplicatedvcpu_idcheck, since the check doesn't find vCPU A inkvm->vcpu_array.
3. vCPU A is installed inkvm->vcpu_array, vCPU creation succeeds.
4. vCPU B with the samevcpu_idis installed inkvm->vcpu_array, vCPU creation succeeds.
Quoted 3 lines
And we cannot just move the
kvm->vcpu_arrayinsertion earlier (before the first unlock ofkvm->lock), at least because the vCPU is not even allocated at that point.
Hmm right, and AFACIT vCPU A needs to actually bump up the kvm->online_vcpus in order for the vCPU B to detect the vCPU of same vcpu_id has been created because kvm_get_vcpu_by_id() will only scan online vCPUs.
Thanks for pointing out!
Quoted 5 lines
I guess we could track the used
vcpu_idsseparately in another xarray (or a bitmap) which could be checked and updated by the early "duplicatedvcpu_idcheck" before the first unlock ofkvm->lock. But do we want to pay the memory price for that?
I don't think we should do that, and your approach is much safer:
Sean Christopherson
maintainerQuoted 42 lines · Kai Huang
Quoted 37 lines · Dmytro Maluka
Quoted 10 lines · Kai
Quoted 9 lines
An easy fix would be to clear the
pid_tableentry invmx_vcpu_free(). However that would be still problematic, for the following reason:
userspace may try to create a vCPU with the samevcpu_idas an existing one;vmx_vcpu_create()will succeed, and only after thatkvm_vm_ioctl_create_vcpu()will check for the duplicatevcpu_idand fail with-EEXISTand then free the vCPU in the failure path. So in this failure path,vmx_vcpu_free()would clear thepid_tableentry for that already existing good vCPU, i.e. effectively disable IPIv for that vCPU.Quoted 7 lines
IMHO this seems a bit fragile? If something similar to
pid_tablecoming up in the future, we could end up with a similar problem.The "duplicated
vcpu_idcheck" seems the ones that should happen as early as possible. Is it better to move the "duplicatedvcpu_idcheck" earlier beforevmx_vcpu_create(), e.g., even before thekvm_arch_vcpu_precreate()? In this case we can do thepid_tablecleanup invmx_vcpu_free()I think.Unfortunately it is not that simple. As I understand, the reason why the "duplicated
vcpu_idcheck" is done later is that it needs to be done atomically together with inserting the vCPU intokvm->vcpu_arrayafter it, i.e. both the check and the insertion need to be done withkvm->lockheld (rather than releasingkvm->lockand taking it again between the two operations).IOW, if we just move the "duplicated
vcpu_idcheck" earlier (before the first unlock ofkvm->lock), we have a race:1. vCPU A is being created but not installed in
kvm->vcpu_arrayyet.
2. vCPU B with the samevcpu_idis being created. It passes the duplicatedvcpu_idcheck, since the check doesn't find vCPU A inkvm->vcpu_array.
3. vCPU A is installed inkvm->vcpu_array, vCPU creation succeeds.
4. vCPU B with the samevcpu_idis installed inkvm->vcpu_array, vCPU creation succeeds.Quoted 3 lines
And we cannot just move the
kvm->vcpu_arrayinsertion earlier (before the first unlock ofkvm->lock), at least because the vCPU is not even allocated at that point.
Quoted 10 lines
Hmm right, and AFACIT vCPU A needs to actually bump up the
kvm->online_vcpusin order for the vCPU B to detect the vCPU of samevcpu_idhas been created becausekvm_get_vcpu_by_id()will only scan online vCPUs.Thanks for pointing out!
Quoted 4 lines
I guess we could track the used
vcpu_idsseparately in another xarray (or a bitmap) which could be checked and updated by the early "duplicatedvcpu_idcheck" before the first unlock ofkvm->lock. But do we want to pay the memory price for that?
I'm comfortable paying the cost for a bitmap. If a system can actually support thousands of vCPU IDs, then burning a few hundred bytes per VM is all but guaranteed to be a non-issue. And for smaller systems, e.g. on arm64 where a VM can have at most 512 vCPU IDs, the bitmap costs a measely 64 bytes.
And if the wastefulness of a persistent bitmap is a concern, we could easly use an xarray to track only "pending" vCPUs, so that the steady state cost is ~zero. Actually, given how easy that is (famous last words), unless removing from an xarray doesn't free memory soon-ish, I'd say go straight to a semi-permanent xarray to avoid bikeshedding over the cost of the bitmap.
Quoted 1 line
I don't think we should do that, and your approach is much safer:
I disagree. I actually arrived at the bitmap solution before reading this. My concern isn't so much about IPI virtualization, it's about the lurking danger of an unverified vcpu_id. E.g. see Naveen's suggestion in this thread of simply letting vcpu_load() initialize the per-VM table, which doesn't work because x86 calls vcpu_load() as part of vCPU creation. I wouldn't be all that surprised if there's another bug or two in KVM where colliding
Ha! Case in point, s390 had what is effectively the *exact* same bug and fixed it in the *exact* same way (hooking vcpu_postcreate()) over a decade ago in commit 255088244929 ("KVM: s390: fix SCA related races and double use"), and that obviously did nothing to help x86 from repeating the same mistake.
In other words, I want to fix this entire class of bugs, not play a game of whack-a-mole with bugs that humans are all but guaranteed to overlook.
My other concern with the proposed change is that the vCPU becomes reachable before long before kvm_arch_vcpu_postcreate(). Which should be fine" for IPI virtualization, but sets a precedence I'd rather not exist, because initializing vCPU state _after_ it's reachable is rarely correct. E.g. msr_kvm_poll_control is initialized in postcreate for some reason, and that's technically buggy because it's possible, albeit extremely unlikely, that MSR_KVM_POLL_CONTROL could be written by userspace before postcreate() runs.
E.g. for the xarray approacy (sketch only, completely untested):
include/linux/kvm_host.h +1 -0
@@ -791,6 +791,7 @@ struct kvm {/* The current active memslot set for each address space */ struct kvm_memslots __rcu *memslots[KVM_MAX_NR_ADDRESS_SPACES ];struct xarray vcpu_array; +struct xarray pending_vcpu_ids;/* * Protected by slots_lock, but can be read outside if an * incorrect answer is acceptable.
virt/kvm/kvm_main.c +8 -0
@@ -4179,6 +4179,12 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id)return r; } + r =xa_insert (&kvm->pending_vcpu_ids, id,xa_mk_value (id),GFP_KERNEL ); +if (r) { +mutex_unlock (&kvm->lock); +return r; + } + kvm->created_vcpus++;mutex_unlock (&kvm->lock);
@@ -4249,6 +4255,7 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id)*/ smp_wmb ();atomic_inc (&kvm->online_vcpus); +xa_erase (&kvm->pending_vcpu_ids, id);mutex_unlock (&vcpu->mutex);mutex_unlock (&kvm->lock);
@@ -4273,6 +4280,7 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id) vcpu_decrement:mutex_lock (&kvm->lock); kvm->created_vcpus--; +xa_erase (&kvm->pending_vcpu_ids, id);mutex_unlock (&kvm->lock);return r; }
or if that doesn't work, the more naive bitmap approach:
include/linux/kvm_host.h +2 -0
@@ -791,6 +791,8 @@ struct kvm {/* The current active memslot set for each address space */ struct kvm_memslots __rcu *memslots[KVM_MAX_NR_ADDRESS_SPACES ];struct xarray vcpu_array; +DECLARE_BITMAP (vcpu_ids,KVM_MAX_VCPU_IDS ); +/* * Protected by slots_lock, but can be read outside if an * incorrect answer is acceptable.
virt/kvm/kvm_main.c +7 -1
@@ -4179,6 +4179,11 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id)return r; } +if (__test_and_set_bit (id, kvm->vcpu_ids)) { +mutex_unlock (&kvm->lock); +return -EEXIST ; + } + kvm->created_vcpus++;mutex_unlock (&kvm->lock);
@@ -4213,7 +4218,7 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id)mutex_lock (&kvm->lock); -if (kvm_get_vcpu_by_id (kvm, id)) { +if (WARN_ON_ONCE (kvm_get_vcpu_by_id (kvm, id))) { r = -EEXIST ; goto unlock_vcpu_destroy; }
@@ -4273,6 +4278,7 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id) vcpu_decrement:mutex_lock (&kvm->lock); kvm->created_vcpus--; +__clear_bit (id, kvm->vcpu_ids);mutex_unlock (&kvm->lock);return r; }
Quoted 3 lines
In other words, I want to fix this entire class of bugs, not play a game of whack-a-mole with bugs that humans are all but guaranteed to overlook.
I was thinking yet another xarray could make the overall logic more complicated, but yeah, agreed that rejecting the duplicated vcpu_id asap is the right solution to avoid such bugs (my first glance too in the first reply).
Quoted 8 lines
My other concern with the proposed change is that the vCPU becomes reachable before long before
kvm_arch_vcpu_postcreate(). Which should be fine" for IPI virtualization, but sets a precedence I'd rather not exist, because initializing vCPU state _after_ it's reachable is rarely correct. E.g.msr_kvm_poll_controlis initialized in postcreate for some reason, and that's technically buggy because it's possible, albeit extremely unlikely, thatMSR_KVM_POLL_CONTROLcould be written by userspace beforepostcreate()runs.
Fair enough. This didn't ring a bell to me. Thanks for mentioning it.
Quoted 20 lines
E.g. for the xarray approacy (sketch only, completely untested):
include/linux/kvm_host.h +1
-0@@ -791,6 +791,7 @@ struct kvm {/* The current active memslot set for each address space */ struct kvm_memslots __rcu *memslots[KVM_MAX_NR_ADDRESS_SPACES ];struct xarray vcpu_array; +struct xarray pending_vcpu_ids;/* * Protected by slots_lock, but can be read outside if an * incorrect answer is acceptable. virt/kvm/kvm_main.c +0
-0
virt/kvm/kvm_main.c static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id) · line 4187
Quoted 9 lines
virt/kvm/kvm_main.c +6
-0@@ -4179,6 +4179,12 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id)return r; } + r =xa_insert (&kvm->pending_vcpu_ids, id,xa_mk_value (id),GFP_KERNEL ); +if (r) { +mutex_unlock (&kvm->lock); +return r; + } +
(No preference between xarray vs bitmap)
Should we move this before kvm_arch_vcpu_precreate() to avoid any "precreate" for duplicated vcpu_id?
Sean Christopherson
maintainerQuoted 31 lines · Kai Huang
Quoted 19 lines
E.g. for the xarray approacy (sketch only, completely untested):
include/linux/kvm_host.h +1
-0@@ -791,6 +791,7 @@ struct kvm {/* The current active memslot set for each address space */ struct kvm_memslots __rcu *memslots[KVM_MAX_NR_ADDRESS_SPACES ];struct xarray vcpu_array; +struct xarray pending_vcpu_ids;/* * Protected by slots_lock, but can be read outside if an * incorrect answer is acceptable. virt/kvm/kvm_main.c +0
-0virt/kvm/kvm_main.c static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id) · line 4187
Quoted 9 lines
virt/kvm/kvm_main.c +6
-0@@ -4179,6 +4179,12 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id)return r; } + r =xa_insert (&kvm->pending_vcpu_ids, id,xa_mk_value (id),GFP_KERNEL ); +if (r) { +mutex_unlock (&kvm->lock); +return r; + } +(No preference between xarray vs bitmap)
Quoted 2 lines
Should we move this before
kvm_arch_vcpu_precreate()to avoid any "precreate" for duplicatedvcpu_id?
Hmm, I'm not sure. I'm leaning "yes". At a glance, I like the idea. My only concern is that we could consume a garbage @id if kvm_arch_vcpu_precreate() performs additional checks on @id, as x86's implementation does. However, I don't think that's a meaningful concern since kvm_vm_ioctl_create_vcpu() *must* perform at least basic sanity checks.
So yeah, I think moving the check before precreate() makes sense.
Quoted 5 lines · Sean Christopherson
My only
concern is that we could consume a garbage @id ifkvm_arch_vcpu_precreate()performs additional checks on @id, as x86's implementation does. However, I don't think that's a meaningful concern sincekvm_vm_ioctl_create_vcpu()*must* perform at least basic sanity checks.
Agreed.
Quoted 11 lines · Sean Christopherson
Quoted 5 lines · Kai Huang
Quoted 4 lines · Dmytro Maluka
I guess we could track the used
vcpu_idsseparately in another xarray (or a bitmap) which could be checked and updated by the early "duplicatedvcpu_idcheck" before the first unlock ofkvm->lock. But do we want to pay the memory price for that?I'm comfortable paying the cost for a bitmap. If a system can actually support thousands of vCPU IDs, then burning a few hundred bytes per VM is all but guaranteed to be a non-issue. And for smaller systems, e.g. on arm64 where a VM can have at most 512 vCPU IDs, the bitmap costs a measely 64 bytes.
I was just a bit concerned that, for example, on my laptop, even though there is only a dozen of physical CPUs, there is still CONFIG_KVM_MAX_NR_VCPUS=4096 in my Debian's default kernel config, and KVM_VCPU_ID_RATIO is still hardcoded to 4, so the bitmap size would be 4096 * 4 / 8 = 2KB which just feels "disproportionate" for such a corner case as this sanity check.
But yeah, that doesn't seem to be a real problem (it's not like we routinely run thousands of VMs on such systems).
Quoted 5 lines
And if the wastefulness of a persistent bitmap is a concern, we could easly use an xarray to track only "pending" vCPUs, so that the steady state cost is ~zero. Actually, given how easy that is (famous last words), unless removing from an xarray doesn't free memory soon-ish, I'd say go straight to a semi-permanent xarray to avoid bikeshedding over the cost of the bitmap.
After some thinking, I think I prefer the bitmap. The pending_vcpu_ids xarray feels like a premature optimization.
Quoted 17 lines
Quoted 1 line
I don't think we should do that, and your approach is much safer:
I disagree. I actually arrived at the bitmap solution before reading this. My concern isn't so much about IPI virtualization, it's about the lurking danger of an unverified
vcpu_id. E.g. see Naveen's suggestion in this thread of simply lettingvcpu_load()initialize the per-VM table, which doesn't work because x86 callsvcpu_load()as part of vCPU creation. I wouldn't be all that surprised if there's another bug or two in KVM where collidingHa! Case in point, s390 had what is effectively the *exact* same bug and fixed it in the *exact* same way (hooking
vcpu_postcreate()) over a decade ago in commit 255088244929 ("KVM: s390: fix SCA related races and double use"), and that obviously did nothing to help x86 from repeating the same mistake.In other words, I want to fix this entire class of bugs, not play a game of whack-a-mole with bugs that humans are all but guaranteed to overlook.
Quoted 7 lines
My other concern with the proposed change is that the vCPU becomes reachable before long before
kvm_arch_vcpu_postcreate(). Which should be fine" for IPI virtualization, but sets a precedence I'd rather not exist, because initializing vCPU state _after_ it's reachable is rarely correct. E.g.msr_kvm_poll_controlis initialized in postcreate for some reason, and that's technically buggy because it's possible, albeit extremely unlikely, thatMSR_KVM_POLL_CONTROLcould be written by userspace beforepostcreate()runs.
Yeah, agree on both points. I myself was a bit uncomfortable with my vcpu_postcreate approach implying some unwritten ill-defined rules on what things _must_ be done in vcpu_postcreate (rather than in vcpu_create) and what things _must not_ be done in it.
Quoted 97 lines
E.g. for the xarray approacy (sketch only, completely untested):
include/linux/kvm_host.h +1
-0@@ -791,6 +791,7 @@ struct kvm {/* The current active memslot set for each address space */ struct kvm_memslots __rcu *memslots[KVM_MAX_NR_ADDRESS_SPACES ];struct xarray vcpu_array; +struct xarray pending_vcpu_ids;/* * Protected by slots_lock, but can be read outside if an * incorrect answer is acceptable. virt/kvm/kvm_main.c +8
-0@@ -4179,6 +4179,12 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id)return r; } + r =xa_insert (&kvm->pending_vcpu_ids, id,xa_mk_value (id),GFP_KERNEL ); +if (r) { +mutex_unlock (&kvm->lock); +return r; + } + kvm->created_vcpus++;mutex_unlock (&kvm->lock);@@ -4249,6 +4255,7 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id)*/ smp_wmb ();atomic_inc (&kvm->online_vcpus); +xa_erase (&kvm->pending_vcpu_ids, id);mutex_unlock (&vcpu->mutex);mutex_unlock (&kvm->lock);@@ -4273,6 +4280,7 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id) vcpu_decrement:mutex_lock (&kvm->lock); kvm->created_vcpus--; +xa_erase (&kvm->pending_vcpu_ids, id);mutex_unlock (&kvm->lock);return r; }or if that doesn't work, the more naive bitmap approach:
include/linux/kvm_host.h +2
-0@@ -791,6 +791,8 @@ struct kvm {/* The current active memslot set for each address space */ struct kvm_memslots __rcu *memslots[KVM_MAX_NR_ADDRESS_SPACES ];struct xarray vcpu_array; +DECLARE_BITMAP (vcpu_ids,KVM_MAX_VCPU_IDS ); +/* * Protected by slots_lock, but can be read outside if an * incorrect answer is acceptable. virt/kvm/kvm_main.c +7
-1@@ -4179,6 +4179,11 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id)return r; } +if (__test_and_set_bit (id, kvm->vcpu_ids)) { +mutex_unlock (&kvm->lock); +return -EEXIST ; + } + kvm->created_vcpus++;mutex_unlock (&kvm->lock);@@ -4213,7 +4218,7 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id)mutex_lock (&kvm->lock); -if (kvm_get_vcpu_by_id (kvm, id)) { +if (WARN_ON_ONCE (kvm_get_vcpu_by_id (kvm, id))) { r = -EEXIST ; goto unlock_vcpu_destroy; }@@ -4273,6 +4278,7 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id) vcpu_decrement:mutex_lock (&kvm->lock); kvm->created_vcpus--; +__clear_bit (id, kvm->vcpu_ids);mutex_unlock (&kvm->lock);return r; }
Quoted 29 lines · Dmytro Maluka
vCPU creation in
kvm_vm_ioctl_create_vcpu()may fail afterkvm_arch_vcpu_create()->vmx_vcpu_create()already succeeded. In such casekvm_vm_ioctl_create_vcpu()destroys the newly created vCPU in the failure path. However, that leaves a side effect: the IPIvpid_tableentry remains configured with this vCPU'spi_descaddress. As a result, when another vCPU sends an IPI to the APIC ID of this failed-to-create vCPU, it will cause HW to write to this (freed!)pi_descmemory. [*]An easy fix would be to clear the
pid_tableentry invmx_vcpu_free(). However that would be still problematic, for the following reason:
userspace may try to create a vCPU with the samevcpu_idas an existing one;vmx_vcpu_create()will succeed, and only after thatkvm_vm_ioctl_create_vcpu()will check for the duplicatevcpu_idand fail with-EEXISTand then free the vCPU in the failure path. So in this failure path,vmx_vcpu_free()would clear thepid_tableentry for that already existing good vCPU, i.e. effectively disable IPIv for that vCPU.So instead fix the issue by moving the
pid_tableentry setup from .vcpu_create() to the newly introduced .vcpu_postcreate() callback called at the end ofkvm_vm_ioctl_create_vcpu(), when we are sure that the vCPU creation succeeded.[*] Although, since this memory is freed into the
kvm_vcpu_cachekmem
cache which is only used for allocatingkvm_vcpus, _maybe_ this memory will only be reused forpi_descof another vCPU, not for anything else. So _maybe_ this will only result in delivering the IPI to a wrong vCPU (possibly of another VM) in the worst case, not in a random corruption of kernel memory.
Quoted 2 lines
Fixes d588bb9be1da ("KVM: VMX: enable IPI virtualization")Signed-off-by Dmytro Maluka <[email protected]>