KVM: Various fixes and improvements around kicking vCPUs
Subsystems (4)
Vitaly Kuznetsov
maintainerv5 cover KVM: Various fixes and improvements around kicking vCPUs
Changes since v4 (Sean):
- Add Reviewed-by: tag to PATCHes 3 and 5.
- Drop unneeded 'vcpu' initialization from PATCH4.
- Return -ENOMEM from kvm_init() when cpumask allocation fails and drop unnecessary braces (PATCH 7).
- Drop cpumask_available() check from kvm_kick_many_cpus() and convert kvm_make_vcpu_request()'s parameter from 'cpumask_var_t' to 'struct cpumask *' (PATCH 8)
This series is a continuation to Sean's "[PATCH 0/2] VM: Fix a benign race in kicking vCPUs" work and v2 for my "KVM: Optimize kvm_make_vcpus_request_mask() a bit"/"KVM: x86: Fix stack-out-of-bounds memory access from ioapic_write_indirect()" patchset.
From Sean:
"Fix benign races when kicking vCPUs where the task doing the kicking can consume a stale vcpu->cpu. The races are benign because of the impliciations of task migration with respect to interrupts and being in guest mode, but IMO they're worth fixing if only as an excuse to document the flows.
Patch 2 is a tangentially related cleanup to prevent future me from trying to get rid of the NULL check on the cpumask parameters, which _looks_ like it can't ever be NULL, but has a subtle edge case due to the way CONFIG_CPUMASK_OFFSTACK=y handles cpumasks."
Patch3 is a preparation to untangling kvm_make_all_cpus_request_except() and kvm_make_vcpus_request_mask().
Patch4 is a minor optimization for kvm_make_vcpus_request_mask() for big guests.
Patch5 is a minor cleanup.
Patch6 fixes a real problem with ioapic_write_indirect() KVM does out-of-bounds access to stack memory.
Patches7 and 8 get rid of dynamic cpumask allocation for kicking vCPUs.
Sean Christopherson (2):
KVM: Clean up benign vcpu->cpu data races when kicking vCPUs
KVM: KVM: Use cpumask_available() to check for NULL cpumask when
kicking vCPUs
Vitaly Kuznetsov (6):
KVM: x86: hyper-v: Avoid calling kvm_make_vcpus_request_mask() with
vcpu_mask==NULL
KVM: Optimize kvm_make_vcpus_request_mask() a bit
KVM: Drop 'except' parameter from kvm_make_vcpus_request_mask()
KVM: x86: Fix stack-out-of-bounds memory access from
ioapic_write_indirect()
KVM: Pre-allocate cpumasks for kvm_make_all_cpus_request_except()
KVM: Make kvm_make_vcpus_request_mask() use pre-allocated
cpu_kick_mask
arch/x86/include/asm/kvm_host.h | 1 - arch/x86/kvm/hyperv.c | 18 ++--- arch/x86/kvm/ioapic.c | 10 +-- arch/x86/kvm/x86.c | 8 +-- include/linux/kvm_host.h | 3 +- virt/kvm/kvm_main.c | 115 +++++++++++++++++++++++--------- 6 files changed, 101 insertions(+), 54 deletions(-)
Signature · 1 line
2.31.1
Vitaly Kuznetsov
maintainerPATCH v5 1/8 KVM: Clean up benign vcpu->cpu data races when kicking vCPUs
From: Sean Christopherson <[email protected]>
Fix a benign data race reported by syzbot+KCSAN[*] by ensuring vcpu->cpu is read exactly once, and by ensuring the vCPU is booted from guest mode if kvm_arch_vcpu_should_kick() returns true. Fix a similar race in kvm_make_vcpus_request_mask() by ensuring the vCPU is interrupted if kvm_request_needs_ipi() returns true.
Reading vcpu->cpu before vcpu->mode (via kvm_arch_vcpu_should_kick() or kvm_request_needs_ipi()) means the target vCPU could get migrated (change vcpu->cpu) and enter !OUTSIDE_GUEST_MODE between reading vcpu->cpud and reading vcpu->mode. If that happens, the kick/IPI will be sent to the old pCPU, not the new pCPU that is now running the vCPU or reading SPTEs.
Although failing to kick the vCPU is not exactly ideal, practically speaking it cannot cause a functional issue unless there is also a bug in the caller, and any such bug would exist regardless of kvm_vcpu_kick()'s behavior.
The purpose of sending an IPI is purely to get a vCPU into the host (or out of reading SPTEs) so that the vCPU can recognize a change in state, e.g. a KVM_REQ_* request. If vCPU's handling of the state change is required for correctness, KVM must ensure either the vCPU sees the change before entering the guest, or that the sender sees the vCPU as running in guest mode. All architectures handle this by (a) sending the request before calling kvm_vcpu_kick() and (b) checking for requests _after_ setting vcpu->mode.
x86's READING_SHADOW_PAGE_TABLES has similar requirements; KVM needs to ensure it kicks and waits for vCPUs that started reading SPTEs _before_ MMU changes were finalized, but any vCPU that starts reading after MMU changes were finalized will see the new state and can continue on uninterrupted.
For uses of kvm_vcpu_kick() that are not paired with a KVM_REQ_*, e.g. x86's kvm_arch_sync_dirty_log(), the order of the kick must not be relied upon for functional correctness, e.g. in the dirty log case, userspace cannot assume it has a 100% complete log if vCPUs are still running.
All that said, eliminate the benign race since the cost of doing so is an "extra" atomic cmpxchg() in the case where the target vCPU is loaded by the current pCPU or is not loaded at all. I.e. the kick will be skipped due to kvm_vcpu_exiting_guest_mode() seeing a compatible vcpu->mode as opposed to the kick being skipped because of the cpu checks.
Keep the "cpu != me" checks even though they appear useless/impossible at first glance. x86 processes guest IPI writes in a fast path that runs in IN_GUEST_MODE, i.e. can call kvm_vcpu_kick() from IN_GUEST_MODE. And calling kvm_vm_bugged()->kvm_make_vcpus_request_mask() from IN_GUEST or READING_SHADOW_PAGE_TABLES is perfectly reasonable.
Note, a race with the cpu_online() check in kvm_vcpu_kick() likely persists, e.g. the vCPU could exit guest mode and get offlined between the cpu_online() check and the sending of smp_send_reschedule(). But, the online check appears to exist only to avoid a WARN in x86's native_smp_send_reschedule() that fires if the target CPU is not online. The reschedule WARN exists because CPU offlining takes the CPU out of the scheduling pool, i.e. the WARN is intended to detect the case where the kernel attempts to schedule a task on an offline CPU. The actual sending of the IPI is a non-issue as at worst it will simpy be dropped on the floor. In other words, KVM's usurping of the reschedule IPI could theoretically trigger a WARN if the stars align, but there will be no loss of functionality.
[*] https://syzkaller.appspot.com/bug?extid=cd4154e502f43f10808a
--- virt/kvm/kvm_main.c | 36 ++++++++++++++++++++++++++++-------- 1 file changed, 28 insertions(+), 8 deletions(-)
virt/kvm/kvm_main.c +28 -8
@@ -273,14 +273,26 @@ bool kvm_make_vcpus_request_mask(struct kvm *kvm, unsigned int req,continue ;kvm_make_request (req, vcpu); - cpu = vcpu->cpu;if (!(req &KVM_REQUEST_NO_WAKEUP ) &&kvm_vcpu_wake_up (vcpu))continue ; -if (tmp !=NULL && cpu !=-1 && cpu != me && -kvm_request_needs_ipi (vcpu, req)) -__cpumask_set_cpu (cpu, tmp); +/* +* Note, the vCPU could get migrated to a different pCPU at any +* point after kvm_request_needs_ipi(), which could result in +* sending an IPI to the previous pCPU. But, that's ok because +* the purpose of the IPI is to ensure the vCPU returns to +* OUTSIDE_GUEST_MODE, which is satisfied if the vCPU migrates. +* Entering READING_SHADOW_PAGE_TABLES after this point is also +* ok, as the requirement is only that KVM wait for vCPUs that +* were reading SPTEs _before_ any changes were finalized. See +* kvm_vcpu_kick() for more details on handling requests. +*/ +if (tmp !=NULL &&kvm_request_needs_ipi (vcpu, req)) { + cpu =READ_ONCE (vcpu->cpu); +if (cpu !=-1 && cpu != me) +__cpumask_set_cpu (cpu, tmp); + } } called =kvm_kick_many_cpus (tmp, !!(req &KVM_REQUEST_WAIT ));
@@ -3309,16 +3321,24 @@ EXPORT_SYMBOL_GPL(kvm_vcpu_wake_up);*/ void kvm_vcpu_kick (struct kvm_vcpu *vcpu) { -int me; -int cpu = vcpu->cpu; +int me, cpu;if (kvm_vcpu_wake_up (vcpu))return ; +/* +* Note, the vCPU could get migrated to a different pCPU at any point +* after kvm_arch_vcpu_should_kick(), which could result in sending an +* IPI to the previous pCPU. But, that's ok because the purpose of the +* IPI is to force the vCPU to leave IN_GUEST_MODE, and migrating the +* vCPU also requires it to leave IN_GUEST_MODE. +*/ me =get_cpu (); -if (cpu != me && (unsigned )cpu < nr_cpu_ids &&cpu_online (cpu)) -if (kvm_arch_vcpu_should_kick (vcpu)) +if (kvm_arch_vcpu_should_kick (vcpu)) { + cpu =READ_ONCE (vcpu->cpu); +if (cpu != me && (unsigned )cpu < nr_cpu_ids &&cpu_online (cpu))smp_send_reschedule (cpu); + }put_cpu (); }EXPORT_SYMBOL_GPL (kvm_vcpu_kick);
Signature · 1 line
2.31.1
Re: [PATCH v5 1/8] KVM: Clean up benign vcpu->cpu data races when kicking vCPUs
Quoted 65 lines · Vitaly Kuznetsov
From: Sean Christopherson <[email protected]>
Fix a benign data race reported by syzbot+KCSAN[*] by ensuring
vcpu->cpuis read exactly once, and by ensuring the vCPU is booted from guest mode ifkvm_arch_vcpu_should_kick()returns true. Fix a similar race inkvm_make_vcpus_request_mask()by ensuring the vCPU is interrupted ifkvm_request_needs_ipi()returns true.Reading
vcpu->cpubeforevcpu->mode(viakvm_arch_vcpu_should_kick()orkvm_request_needs_ipi()) means the target vCPU could get migrated (changevcpu->cpu) and enter !OUTSIDE_GUEST_MODE between readingvcpu->cpudand readingvcpu->mode. If that happens, the kick/IPI will be sent to the old pCPU, not the new pCPU that is now running the vCPU or reading SPTEs.Although failing to kick the vCPU is not exactly ideal, practically speaking it cannot cause a functional issue unless there is also a bug in the caller, and any such bug would exist regardless of kvm_vcpu_kick()'s behavior.
The purpose of sending an IPI is purely to get a vCPU into the host (or out of reading SPTEs) so that the vCPU can recognize a change in state, e.g. a KVM_REQ_* request. If vCPU's handling of the state change is required for correctness, KVM must ensure either the vCPU sees the change before entering the guest, or that the sender sees the vCPU as running in guest mode. All architectures handle this by (a) sending the request before calling
kvm_vcpu_kick()and (b) checking for requests _after_ settingvcpu->mode.x86's
READING_SHADOW_PAGE_TABLEShas similar requirements; KVM needs to ensure it kicks and waits for vCPUs that started reading SPTEs _before_ MMU changes were finalized, but any vCPU that starts reading after MMU changes were finalized will see the new state and can continue on uninterrupted.For uses of
kvm_vcpu_kick()that are not paired with a KVM_REQ_*, e.g. x86'skvm_arch_sync_dirty_log(), the order of the kick must not be relied upon for functional correctness, e.g. in the dirty log case, userspace cannot assume it has a 100% complete log if vCPUs are still running.All that said, eliminate the benign race since the cost of doing so is an "extra" atomic
cmpxchg()in the case where the target vCPU is loaded by the current pCPU or is not loaded at all. I.e. the kick will be skipped due tokvm_vcpu_exiting_guest_mode()seeing a compatiblevcpu->modeas opposed to the kick being skipped because of the cpu checks.Keep the "cpu != me" checks even though they appear useless/impossible at first glance. x86 processes guest IPI writes in a fast path that runs in
IN_GUEST_MODE, i.e. can callkvm_vcpu_kick()fromIN_GUEST_MODE. And calling kvm_vm_bugged()->kvm_make_vcpus_request_mask() fromIN_GUESTorREADING_SHADOW_PAGE_TABLESis perfectly reasonable.Note, a race with the
cpu_online()check inkvm_vcpu_kick()likely persists, e.g. the vCPU could exit guest mode and get offlined between thecpu_online()check and the sending ofsmp_send_reschedule(). But, the online check appears to exist only to avoid a WARN in x86'snative_smp_send_reschedule()that fires if the target CPU is not online. The reschedule WARN exists because CPU offlining takes the CPU out of the scheduling pool, i.e. the WARN is intended to detect the case where the kernel attempts to schedule a task on an offline CPU. The actual sending of the IPI is a non-issue as at worst it will simpy be dropped on the floor. In other words, KVM's usurping of the reschedule IPI could theoretically trigger a WARN if the stars align, but there will be no loss of functionality.
Quoted 7 lines
[*] https://syzkaller.appspot.com/bug?extid=cd4154e502f43f10808a
Cc Venkatesh Srinivas <[email protected]>Cc Vitaly Kuznetsov <[email protected]>Fixes 97222cc83163 ("KVM: Emulate local APIC in kernel")Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Vitaly Kuznetsov <[email protected]>
Quoted 71 lines
--- virt/kvm/kvm_main.c | 36 ++++++++++++++++++++++++++++-------- 1 file changed, 28 insertions(+), 8 deletions(-)virt/kvm/kvm_main.c +28
-9@@ -273,14 +273,26 @@ bool kvm_make_vcpus_request_mask(struct kvm *kvm, unsigned int req,continue ;kvm_make_request (req, vcpu); - cpu = vcpu->cpu;if (!(req &KVM_REQUEST_NO_WAKEUP ) &&kvm_vcpu_wake_up (vcpu))continue ; -if (tmp !=NULL && cpu !=-1 && cpu != me && -kvm_request_needs_ipi (vcpu, req)) -__cpumask_set_cpu (cpu, tmp); +/* +* Note, the vCPU could get migrated to a different pCPU at any +* point after kvm_request_needs_ipi(), which could result in +* sending an IPI to the previous pCPU. But, that's ok because +* the purpose of the IPI is to ensure the vCPU returns to +* OUTSIDE_GUEST_MODE, which is satisfied if the vCPU migrates. +* Entering READING_SHADOW_PAGE_TABLES after this point is also +* ok, as the requirement is only that KVM wait for vCPUs that +* were reading SPTEs _before_ any changes were finalized. See +* kvm_vcpu_kick() for more details on handling requests. +*/ +if (tmp !=NULL &&kvm_request_needs_ipi (vcpu, req)) { + cpu =READ_ONCE (vcpu->cpu); +if (cpu !=-1 && cpu != me) +__cpumask_set_cpu (cpu, tmp); + } } called =kvm_kick_many_cpus (tmp, !!(req &KVM_REQUEST_WAIT ));@@ -3309,16 +3321,24 @@ EXPORT_SYMBOL_GPL(kvm_vcpu_wake_up);*/ void kvm_vcpu_kick (struct kvm_vcpu *vcpu) { -int me; -int cpu = vcpu->cpu; +int me, cpu;if (kvm_vcpu_wake_up (vcpu))return ; +/* +* Note, the vCPU could get migrated to a different pCPU at any point +* after kvm_arch_vcpu_should_kick(), which could result in sending an +* IPI to the previous pCPU. But, that's ok because the purpose of the +* IPI is to force the vCPU to leave IN_GUEST_MODE, and migrating the +* vCPU also requires it to leave IN_GUEST_MODE. +*/ me =get_cpu (); -if (cpu != me && (unsigned )cpu < nr_cpu_ids &&cpu_online (cpu)) -if (kvm_arch_vcpu_should_kick (vcpu)) +if (kvm_arch_vcpu_should_kick (vcpu)) { + cpu =READ_ONCE (vcpu->cpu); +if (cpu != me && (unsigned )cpu < nr_cpu_ids &&cpu_online (cpu))smp_send_reschedule (cpu); + }put_cpu (); }EXPORT_SYMBOL_GPL (kvm_vcpu_kick); --2.31.1
Vitaly Kuznetsov
maintainerPATCH v5 2/8 KVM: KVM: Use cpumask_available() to check for NULL cpumask when kicking vCPUs
In reply to Vitaly KuznetsovFrom: Sean Christopherson <[email protected]>
Check for a NULL cpumask_var_t when kicking multiple vCPUs via cpumask_available(), which performs a !NULL check if and only if cpumasks are configured to be allocated off-stack. This is a meaningless optimization, e.g. avoids a TEST+Jcc and TEST+CMOV on x86, but more importantly helps document that the NULL check is necessary even though all callers pass in a local variable.
No functional change intended.
--- virt/kvm/kvm_main.c | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-)
virt/kvm/kvm_main.c +15 -3
@@ -245,9 +245,13 @@ static void ack_flush(void *_completed) { } -static inline bool kvm_kick_many_cpus (const struct cpumask *cpus,bool wait) +static inline bool kvm_kick_many_cpus (cpumask_var_t tmp,bool wait) { -if (unlikely (!cpus)) +const struct cpumask *cpus; + +if (likely (cpumask_available (tmp))) + cpus = tmp; +else cpus = cpu_online_mask;if (cpumask_empty (cpus))
@@ -277,6 +281,14 @@ bool kvm_make_vcpus_request_mask(struct kvm *kvm, unsigned int req,if (!(req &KVM_REQUEST_NO_WAKEUP ) &&kvm_vcpu_wake_up (vcpu))continue ; +/* +* tmp can be "unavailable" if cpumasks are allocated off stack +* as allocation of the mask is deliberately not fatal and is +* handled by falling back to kicking all online CPUs. +*/ +if (!cpumask_available (tmp)) +continue ; +/* * Note, the vCPU could get migrated to a different pCPU at any * point after kvm_request_needs_ipi(), which could result in
@@ -288,7 +300,7 @@ bool kvm_make_vcpus_request_mask(struct kvm *kvm, unsigned int req,* were reading SPTEs _before_ any changes were finalized. See * kvm_vcpu_kick() for more details on handling requests. */ -if (tmp !=NULL &&kvm_request_needs_ipi (vcpu, req)) { +if (kvm_request_needs_ipi (vcpu, req)) { cpu =READ_ONCE (vcpu->cpu);if (cpu !=-1 && cpu != me)__cpumask_set_cpu (cpu, tmp);
Signature · 1 line
2.31.1
Vitaly Kuznetsov
maintainerPATCH v5 3/8 KVM: x86: hyper-v: Avoid calling kvm_make_vcpus_request_mask() with vcpu_mask==NULL
In reply to Vitaly KuznetsovIn preparation to making kvm_make_vcpus_request_mask() use for_each_set_bit() switch kvm_hv_flush_tlb() to calling kvm_make_all_cpus_request() for 'all cpus' case.
Note: kvm_make_all_cpus_request() (unlike kvm_make_vcpus_request_mask()) currently dynamically allocates cpumask on each call and this is suboptimal. Both kvm_make_all_cpus_request() and kvm_make_vcpus_request_mask() are going to be switched to using pre-allocated per-cpu masks.
--- arch/x86/kvm/hyperv.c | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-)
arch/x86/kvm/hyperv.c +9 -6
@@ -1839,16 +1839,19 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, boolcpumask_clear (&hv_vcpu->tlb_flush); - vcpu_mask = all_cpus ?NULL : -sparse_set_to_vcpu_mask (kvm, sparse_banks, valid_bank_mask, - vp_bitmap, vcpu_bitmap); -/* * vcpu->arch.cr3 may not be up-to-date for running vCPUs so we can't * analyze it here, flush TLB regardless of the specified address space. */ -kvm_make_vcpus_request_mask (kvm,KVM_REQ_TLB_FLUSH_GUEST , -NULL , vcpu_mask, &hv_vcpu->tlb_flush); +if (all_cpus) { +kvm_make_all_cpus_request (kvm,KVM_REQ_TLB_FLUSH_GUEST ); + }else { + vcpu_mask =sparse_set_to_vcpu_mask (kvm, sparse_banks, valid_bank_mask, + vp_bitmap, vcpu_bitmap); + +kvm_make_vcpus_request_mask (kvm,KVM_REQ_TLB_FLUSH_GUEST , +NULL , vcpu_mask, &hv_vcpu->tlb_flush); + } ret_success:/* We always do full TLB flush, set 'Reps completed' = 'Rep Count' */
Signature · 1 line
2.31.1
Vitaly Kuznetsov
maintainerPATCH v5 4/8 KVM: Optimize kvm_make_vcpus_request_mask() a bit
In reply to Vitaly KuznetsovIterating over set bits in 'vcpu_bitmap' should be faster than going through all vCPUs, especially when just a few bits are set.
Drop kvm_make_vcpus_request_mask() call from kvm_make_all_cpus_request_except() to avoid handling the special case when 'vcpu_bitmap' is NULL, move the code to kvm_make_all_cpus_request_except() itself.
--- virt/kvm/kvm_main.c | 88 +++++++++++++++++++++++++++------------------ 1 file changed, 53 insertions(+), 35 deletions(-)
virt/kvm/kvm_main.c +53 -35
@@ -261,50 +261,57 @@ static inline bool kvm_kick_many_cpus(cpumask_var_t tmp, bool wait)return true; } +static void kvm_make_vcpu_request (struct kvm *kvm,struct kvm_vcpu *vcpu, +unsigned int req,cpumask_var_t tmp, +int current_cpu) +{ +int cpu; + +kvm_make_request (req, vcpu); + +if (!(req &KVM_REQUEST_NO_WAKEUP ) &&kvm_vcpu_wake_up (vcpu)) +return ; + +/* +* tmp can be "unavailable" if cpumasks are allocated off stack as +* allocation of the mask is deliberately not fatal and is handled by +* falling back to kicking all online CPUs. +*/ +if (!cpumask_available (tmp)) +return ; + +/* +* Note, the vCPU could get migrated to a different pCPU at any point +* after kvm_request_needs_ipi(), which could result in sending an IPI +* to the previous pCPU. But, that's OK because the purpose of the IPI +* is to ensure the vCPU returns to OUTSIDE_GUEST_MODE, which is +* satisfied if the vCPU migrates. Entering READING_SHADOW_PAGE_TABLES +* after this point is also OK, as the requirement is only that KVM wait +* for vCPUs that were reading SPTEs _before_ any changes were +* finalized. See kvm_vcpu_kick() for more details on handling requests. +*/ +if (kvm_request_needs_ipi (vcpu, req)) { + cpu =READ_ONCE (vcpu->cpu); +if (cpu !=-1 && cpu != current_cpu) +__cpumask_set_cpu (cpu, tmp); + } +} +bool kvm_make_vcpus_request_mask (struct kvm *kvm,unsigned int req,struct kvm_vcpu *except,unsigned long *vcpu_bitmap,cpumask_var_t tmp) { -int i, cpu, me;struct kvm_vcpu *vcpu; +int i, me;bool called; me =get_cpu (); -kvm_for_each_vcpu (i, vcpu, kvm) { -if ((vcpu_bitmap && !test_bit (i, vcpu_bitmap)) || - vcpu == except) +for_each_set_bit (i, vcpu_bitmap,KVM_MAX_VCPUS ) { + vcpu =kvm_get_vcpu (kvm, i); +if (!vcpu || vcpu == except)continue ; - -kvm_make_request (req, vcpu); - -if (!(req &KVM_REQUEST_NO_WAKEUP ) &&kvm_vcpu_wake_up (vcpu)) -continue ; - -/* -* tmp can be "unavailable" if cpumasks are allocated off stack -* as allocation of the mask is deliberately not fatal and is -* handled by falling back to kicking all online CPUs. -*/ -if (!cpumask_available (tmp)) -continue ; - -/* -* Note, the vCPU could get migrated to a different pCPU at any -* point after kvm_request_needs_ipi(), which could result in -* sending an IPI to the previous pCPU. But, that's ok because -* the purpose of the IPI is to ensure the vCPU returns to -* OUTSIDE_GUEST_MODE, which is satisfied if the vCPU migrates. -* Entering READING_SHADOW_PAGE_TABLES after this point is also -* ok, as the requirement is only that KVM wait for vCPUs that -* were reading SPTEs _before_ any changes were finalized. See -* kvm_vcpu_kick() for more details on handling requests. -*/ -if (kvm_request_needs_ipi (vcpu, req)) { - cpu =READ_ONCE (vcpu->cpu); -if (cpu !=-1 && cpu != me) -__cpumask_set_cpu (cpu, tmp); - } +kvm_make_vcpu_request (kvm, vcpu, req, tmp, me); } called =kvm_kick_many_cpus (tmp, !!(req &KVM_REQUEST_WAIT ));
@@ -316,12 +323,23 @@ bool kvm_make_vcpus_request_mask(struct kvm *kvm, unsigned int req,bool kvm_make_all_cpus_request_except (struct kvm *kvm,unsigned int req,struct kvm_vcpu *except) { +struct kvm_vcpu *vcpu;cpumask_var_t cpus;bool called; +int i, me;zalloc_cpumask_var (&cpus,GFP_ATOMIC ); - called =kvm_make_vcpus_request_mask (kvm, req, except,NULL , cpus); + me =get_cpu (); + +kvm_for_each_vcpu (i, vcpu, kvm) { +if (vcpu == except) +continue ; +kvm_make_vcpu_request (kvm, vcpu, req, cpus, me); + } + + called =kvm_kick_many_cpus (cpus, !!(req &KVM_REQUEST_WAIT )); +put_cpu ();free_cpumask_var (cpus);return called;
Signature · 1 line
2.31.1
Re: [PATCH v5 4/8] KVM: Optimize kvm_make_vcpus_request_mask() a bit
Quoted 9 lines · Vitaly Kuznetsov
Iterating over set bits in '
vcpu_bitmap' should be faster than going through all vCPUs, especially when just a few bits are set.Drop
kvm_make_vcpus_request_mask()call fromkvm_make_all_cpus_request_except()to avoid handling the special case when 'vcpu_bitmap' is NULL, move the code tokvm_make_all_cpus_request_except()itself.Signed-off-by Vitaly Kuznetsov <[email protected]>---
virt/kvm/kvm_main.c static inline bool kvm_kick_many_cpus(cpumask_var_t tmp, bool wait) · line 306
Quoted 8 lines
bool kvm_make_vcpus_request_mask(struct kvm *kvm, unsigned int req,
virt/kvm/kvm_main.c +1
-1struct kvm_vcpu *except, unsigned long *vcpu_bitmap, cpumask_var_t tmp) { - int i, cpu, me; struct kvm_vcpu *vcpu; +int i, me;bool called;
Uber nit, if you're moving "int i, me;" to get reverse fir tree ordering, it should be moved below "bool called;" as well, which you amusingly did do in the below function :-)
Quoted 3 lines
me =
get_cpu();
...
Quoted 8 lines
virt/kvm/kvm_main.c +2
-0@@ -316,12 +323,23 @@ bool kvm_make_vcpus_request_mask(struct kvm *kvm, unsigned int req,bool kvm_make_all_cpus_request_except (struct kvm *kvm,unsigned int req,struct kvm_vcpu *except) { +struct kvm_vcpu *vcpu;cpumask_var_t cpus;bool called; +int i, me;
Vitaly Kuznetsov
maintainerPATCH v5 5/8 KVM: Drop 'except' parameter from kvm_make_vcpus_request_mask()
In reply to Vitaly KuznetsovBoth remaining callers of kvm_make_vcpus_request_mask() pass 'NULL' for 'except' parameter so it can just be dropped.
No functional change intended ©.
--- arch/x86/kvm/hyperv.c | 2 +- arch/x86/kvm/x86.c | 2 +- include/linux/kvm_host.h | 1 - virt/kvm/kvm_main.c | 3 +-- 4 files changed, 3 insertions(+), 5 deletions(-)
arch/x86/kvm/hyperv.c +1 -1
@@ -1850,7 +1850,7 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, bool vp_bitmap, vcpu_bitmap);kvm_make_vcpus_request_mask (kvm,KVM_REQ_TLB_FLUSH_GUEST , -NULL , vcpu_mask, &hv_vcpu->tlb_flush); + vcpu_mask, &hv_vcpu->tlb_flush); } ret_success:
arch/x86/kvm/x86.c +1 -1
@@ -9229,7 +9229,7 @@ void kvm_make_scan_ioapic_request_mask(struct kvm *kvm,zalloc_cpumask_var (&cpus,GFP_ATOMIC );kvm_make_vcpus_request_mask (kvm,KVM_REQ_SCAN_IOAPIC , -NULL , vcpu_bitmap, cpus); + vcpu_bitmap, cpus);free_cpumask_var (cpus); }
include/linux/kvm_host.h +0 -1
@@ -160,7 +160,6 @@ static inline bool is_error_page(struct page *page)#define KVM_ARCH_REQ (nr) KVM_ARCH_REQ_FLAGS(nr, 0)bool kvm_make_vcpus_request_mask (struct kvm *kvm,unsigned int req, -struct kvm_vcpu *except,unsigned long *vcpu_bitmap,cpumask_var_t tmp);bool kvm_make_all_cpus_request (struct kvm *kvm,unsigned int req); bool kvm_make_all_cpus_request_except(struct kvm *kvm, unsigned int req,
virt/kvm/kvm_main.c +1 -2
@@ -298,7 +298,6 @@ static void kvm_make_vcpu_request(struct kvm *kvm, struct kvm_vcpu *vcpu, }bool kvm_make_vcpus_request_mask (struct kvm *kvm,unsigned int req, -struct kvm_vcpu *except,unsigned long *vcpu_bitmap,cpumask_var_t tmp) {struct kvm_vcpu *vcpu;
@@ -309,7 +308,7 @@ bool kvm_make_vcpus_request_mask(struct kvm *kvm, unsigned int req,for_each_set_bit (i, vcpu_bitmap,KVM_MAX_VCPUS ) { vcpu =kvm_get_vcpu (kvm, i); -if (!vcpu || vcpu == except) +if (!vcpu)continue ;kvm_make_vcpu_request (kvm, vcpu, req, tmp, me); }
Signature · 1 line
2.31.1
Vitaly Kuznetsov
maintainerPATCH v5 6/8 KVM: x86: Fix stack-out-of-bounds memory access from ioapic_write_indirect()
In reply to Vitaly KuznetsovKASAN reports the following issue:
BUG: KASAN: stack-out-of-bounds in kvm_make_vcpus_request_mask+0x174/0x440 [kvm] Read of size 8 at addr ffffc9001364f638 by task qemu-kvm/4798 CPU: 0 PID: 4798 Comm: qemu-kvm Tainted: G X --------- --- Hardware name: AMD Corporation DAYTONA_X/DAYTONA_X, BIOS RYM0081C 07/13/2020 Call Trace: dump_stack+0xa5/0xe6 print_address_description.constprop.0+0x18/0x130 ? kvm_make_vcpus_request_mask+0x174/0x440 [kvm] __kasan_report.cold+0x7f/0x114 ? kvm_make_vcpus_request_mask+0x174/0x440 [kvm] kasan_report+0x38/0x50 kasan_check_range+0xf5/0x1d0 kvm_make_vcpus_request_mask+0x174/0x440 [kvm] kvm_make_scan_ioapic_request_mask+0x84/0xc0 [kvm] ? kvm_arch_exit+0x110/0x110 [kvm] ? sched_clock+0x5/0x10 ioapic_write_indirect+0x59f/0x9e0 [kvm] ? static_obj+0xc0/0xc0 ? __lock_acquired+0x1d2/0x8c0 ? kvm_ioapic_eoi_inject_work+0x120/0x120 [kvm]
The problem appears to be that 'vcpu_bitmap' is allocated as a single long on stack and it should really be KVM_MAX_VCPUS long. We also seem to clear the lower 16 bits of it with bitmap_zero() for no particular reason (my guess would be that 'bitmap' and 'vcpu_bitmap' variables in kvm_bitmap_or_dest_vcpus() caused the confusion: while the later is indeed 16-bit long, the later should accommodate all possible vCPUs).
--- arch/x86/kvm/ioapic.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
arch/x86/kvm/ioapic.c +5 -5
@@ -319,8 +319,8 @@ static void ioapic_write_indirect(struct kvm_ioapic *ioapic, u32 val)unsigned index;bool mask_before, mask_after;union kvm_ioapic_redirect_entry *e; -unsigned long vcpu_bitmap;int old_remote_irr, old_delivery_status, old_dest_id, old_dest_mode; +DECLARE_BITMAP (vcpu_bitmap,KVM_MAX_VCPUS );switch (ioapic->ioregsel) {case IOAPIC_REG_VERSION :
@@ -384,9 +384,9 @@ static void ioapic_write_indirect(struct kvm_ioapic *ioapic, u32 val) irq.shorthand =APIC_DEST_NOSHORT ; irq.dest_id = e->fields.dest_id; irq.msi_redir_hint = false; -bitmap_zero (&vcpu_bitmap,16 ); +bitmap_zero (vcpu_bitmap,KVM_MAX_VCPUS );kvm_bitmap_or_dest_vcpus (ioapic->kvm, &irq, - &vcpu_bitmap); + vcpu_bitmap);if (old_dest_mode != e->fields.dest_mode || old_dest_id != e->fields.dest_id) {/*
@@ -399,10 +399,10 @@ static void ioapic_write_indirect(struct kvm_ioapic *ioapic, u32 val)kvm_lapic_irq_dest_mode ( !!e->fields.dest_mode);kvm_bitmap_or_dest_vcpus (ioapic->kvm, &irq, - &vcpu_bitmap); + vcpu_bitmap); }kvm_make_scan_ioapic_request_mask (ioapic->kvm, - &vcpu_bitmap); + vcpu_bitmap); }else {kvm_make_scan_ioapic_request (ioapic->kvm); }
Signature · 1 line
2.31.1
Vitaly Kuznetsov
maintainerPATCH v5 7/8 KVM: Pre-allocate cpumasks for kvm_make_all_cpus_request_except()
In reply to Vitaly KuznetsovAllocating cpumask dynamically in zalloc_cpumask_var() is not ideal. Allocation is somewhat slow and can (in theory and when CPUMASK_OFFSTACK) fail. kvm_make_all_cpus_request_except() already disables preemption so we can use pre-allocated per-cpu cpumasks instead.
--- virt/kvm/kvm_main.c | 29 +++++++++++++++++++++++------ 1 file changed, 23 insertions(+), 6 deletions(-)
virt/kvm/kvm_main.c +23 -6
@@ -155,6 +155,8 @@ static void kvm_uevent_notify_change(unsigned int type, struct kvm *kvm);static unsigned long long kvm_createvm_count;static unsigned long long kvm_active_vms; +static DEFINE_PER_CPU(cpumask_var_t, cpu_kick_mask); +__weak void kvm_arch_mmu_notifier_invalidate_range (struct kvm *kvm,unsigned long start,unsigned long end) {
@@ -323,14 +325,15 @@ bool kvm_make_all_cpus_request_except(struct kvm *kvm, unsigned int req, struct kvm_vcpu *except) {struct kvm_vcpu *vcpu; -cpumask_var_t cpus; +struct cpumask *cpus;bool called;int i, me; -zalloc_cpumask_var (&cpus,GFP_ATOMIC ); - me =get_cpu (); + cpus =this_cpu_cpumask_var_ptr (cpu_kick_mask); +cpumask_clear (cpus); +kvm_for_each_vcpu (i, vcpu, kvm) {if (vcpu == except)continue ;
@@ -340,7 +343,6 @@ bool kvm_make_all_cpus_request_except(struct kvm *kvm, unsigned int req, called =kvm_kick_many_cpus (cpus, !!(req &KVM_REQUEST_WAIT ));put_cpu (); -free_cpumask_var (cpus);return called; }
@@ -5581,9 +5583,17 @@ int kvm_init(void *opaque, unsigned vcpu_size, unsigned vcpu_align, goto out_free_3; } +for_each_possible_cpu (cpu) { +if (!alloc_cpumask_var_node (&per_cpu (cpu_kick_mask, cpu), +GFP_KERNEL ,cpu_to_node (cpu))) { + r = -ENOMEM ; + goto out_free_4; + } + } + r =kvm_async_pf_init ();if (r) - goto out_free; + goto out_free_5; kvm_chardev_ops.owner = module; kvm_vm_fops.owner = module;
@@ -5609,7 +5619,10 @@ int kvm_init(void *opaque, unsigned vcpu_size, unsigned vcpu_align, out_unreg:kvm_async_pf_deinit (); -out_free: +out_free_5: +for_each_possible_cpu (cpu) +free_cpumask_var (per_cpu (cpu_kick_mask, cpu)); +out_free_4:kmem_cache_destroy (kvm_vcpu_cache); out_free_3:unregister_reboot_notifier (&kvm_reboot_notifier);
@@ -5629,8 +5642,12 @@ EXPORT_SYMBOL_GPL(kvm_init);void kvm_exit (void ) { +int cpu; +debugfs_remove_recursive (kvm_debugfs_dir);misc_deregister (&kvm_dev); +for_each_possible_cpu (cpu) +free_cpumask_var (per_cpu (cpu_kick_mask, cpu));kmem_cache_destroy (kvm_vcpu_cache);kvm_async_pf_deinit ();unregister_syscore_ops (&kvm_syscore_ops);
Signature · 1 line
2.31.1
Re: [PATCH v5 7/8] KVM: Pre-allocate cpumasks for kvm_make_all_cpus_request_except()
Quoted 7 lines · Vitaly Kuznetsov
Allocating cpumask dynamically in
zalloc_cpumask_var()is not ideal. Allocation is somewhat slow and can (in theory and whenCPUMASK_OFFSTACK) fail.kvm_make_all_cpus_request_except()already disables preemption so we can use pre-allocated per-cpu cpumasks instead.Signed-off-by Vitaly Kuznetsov <[email protected]>---
Vitaly Kuznetsov
maintainerPATCH v5 8/8 KVM: Make kvm_make_vcpus_request_mask() use pre-allocated cpu_kick_mask
In reply to Vitaly Kuznetsovkvm_make_vcpus_request_mask() already disables preemption so just like kvm_make_all_cpus_request_except() it can be switched to using pre-allocated per-cpu cpumasks. This allows for improvements for both users of the function: in Hyper-V emulation code 'tlb_flush' can now be dropped from 'struct kvm_vcpu_hv' and kvm_make_scan_ioapic_request_mask() gets rid of dynamic allocation.
cpumask_available() checks in kvm_make_vcpu_request() and kvm_kick_many_cpus() can now be dropped as they checks for an impossible
condition: kvm_init() makes sure per-cpu masks are allocated.
--- arch/x86/include/asm/kvm_host.h | 1 - arch/x86/kvm/hyperv.c | 5 +---- arch/x86/kvm/x86.c | 8 +------- include/linux/kvm_host.h | 2 +- virt/kvm/kvm_main.c | 29 +++++++++-------------------- 5 files changed, 12 insertions(+), 33 deletions(-)
arch/x86/include/asm/kvm_host.h +0 -1
@@ -569,7 +569,6 @@ struct kvm_vcpu_hv {struct kvm_hyperv_exit exit;struct kvm_vcpu_hv_stimer stimer[HV_SYNIC_STIMER_COUNT ];DECLARE_BITMAP (stimer_pending_bitmap,HV_SYNIC_STIMER_COUNT ); -cpumask_t tlb_flush;bool enforce_cpuid;struct {u32 features_eax;/* HYPERV_CPUID_FEATURES.EAX */
arch/x86/kvm/hyperv.c +1 -4
@@ -1755,7 +1755,6 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, boolint i;gpa_t gpa;struct kvm *kvm = vcpu->kvm; -struct kvm_vcpu_hv *hv_vcpu =to_hv_vcpu (vcpu);struct hv_tlb_flush_ex flush_ex;struct hv_tlb_flush flush;u64 vp_bitmap[KVM_HV_MAX_SPARSE_VCPU_SET_BITS ];
@@ -1837,8 +1836,6 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, bool } } -cpumask_clear (&hv_vcpu->tlb_flush); -/* * vcpu->arch.cr3 may not be up-to-date for running vCPUs so we can't * analyze it here, flush TLB regardless of the specified address space.
@@ -1850,7 +1847,7 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, bool vp_bitmap, vcpu_bitmap);kvm_make_vcpus_request_mask (kvm,KVM_REQ_TLB_FLUSH_GUEST , - vcpu_mask, &hv_vcpu->tlb_flush); + vcpu_mask); } ret_success:
arch/x86/kvm/x86.c +1 -7
@@ -9224,14 +9224,8 @@ static void process_smi(struct kvm_vcpu *vcpu)void kvm_make_scan_ioapic_request_mask (struct kvm *kvm,unsigned long *vcpu_bitmap) { -cpumask_var_t cpus; - -zalloc_cpumask_var (&cpus,GFP_ATOMIC ); -kvm_make_vcpus_request_mask (kvm,KVM_REQ_SCAN_IOAPIC , - vcpu_bitmap, cpus); - -free_cpumask_var (cpus); + vcpu_bitmap); }void kvm_make_scan_ioapic_request (struct kvm *kvm)
include/linux/kvm_host.h +1 -1
@@ -160,7 +160,7 @@ static inline bool is_error_page(struct page *page)#define KVM_ARCH_REQ (nr) KVM_ARCH_REQ_FLAGS(nr, 0)bool kvm_make_vcpus_request_mask (struct kvm *kvm,unsigned int req, -unsigned long *vcpu_bitmap,cpumask_var_t tmp); +unsigned long *vcpu_bitmap);bool kvm_make_all_cpus_request (struct kvm *kvm,unsigned int req);bool kvm_make_all_cpus_request_except (struct kvm *kvm,unsigned int req,struct kvm_vcpu *except);
virt/kvm/kvm_main.c +9 -20
@@ -247,15 +247,8 @@ static void ack_flush(void *_completed) { } -static inline bool kvm_kick_many_cpus (cpumask_var_t tmp,bool wait) +static inline bool kvm_kick_many_cpus (struct cpumask *cpus,bool wait) { -const struct cpumask *cpus; - -if (likely (cpumask_available (tmp))) - cpus = tmp; -else - cpus = cpu_online_mask; -if (cpumask_empty (cpus))return false;
@@ -264,7 +257,7 @@ static inline bool kvm_kick_many_cpus(cpumask_var_t tmp, bool wait) }static void kvm_make_vcpu_request (struct kvm *kvm,struct kvm_vcpu *vcpu, -unsigned int req,cpumask_var_t tmp, +unsigned int req,struct cpumask *tmp,int current_cpu) {int cpu;
@@ -274,14 +267,6 @@ static void kvm_make_vcpu_request(struct kvm *kvm, struct kvm_vcpu *vcpu,if (!(req &KVM_REQUEST_NO_WAKEUP ) &&kvm_vcpu_wake_up (vcpu))return ; -/* -* tmp can be "unavailable" if cpumasks are allocated off stack as -* allocation of the mask is deliberately not fatal and is handled by -* falling back to kicking all online CPUs. -*/ -if (!cpumask_available (tmp)) -return ; -/* * Note, the vCPU could get migrated to a different pCPU at any point * after kvm_request_needs_ipi(), which could result in sending an IPI
@@ -300,22 +285,26 @@ static void kvm_make_vcpu_request(struct kvm *kvm, struct kvm_vcpu *vcpu, }bool kvm_make_vcpus_request_mask (struct kvm *kvm,unsigned int req, -unsigned long *vcpu_bitmap,cpumask_var_t tmp) +unsigned long *vcpu_bitmap) {struct kvm_vcpu *vcpu; +struct cpumask *cpus;int i, me;bool called; me =get_cpu (); + cpus =this_cpu_cpumask_var_ptr (cpu_kick_mask); +cpumask_clear (cpus); +for_each_set_bit (i, vcpu_bitmap,KVM_MAX_VCPUS ) { vcpu =kvm_get_vcpu (kvm, i);if (!vcpu)continue ; -kvm_make_vcpu_request (kvm, vcpu, req, tmp, me); +kvm_make_vcpu_request (kvm, vcpu, req, cpus, me); } - called =kvm_kick_many_cpus (tmp, !!(req &KVM_REQUEST_WAIT )); + called =kvm_kick_many_cpus (cpus, !!(req &KVM_REQUEST_WAIT ));put_cpu ();return called;
Signature · 1 line
2.31.1
Re: [PATCH v5 8/8] KVM: Make kvm_make_vcpus_request_mask() use pre-allocated cpu_kick_mask
Quoted 7 lines · Vitaly Kuznetsov
kvm_make_vcpus_request_mask()already disables preemption so just likekvm_make_all_cpus_request_except()it can be switched to using pre-allocated per-cpu cpumasks. This allows for improvements for both users of the function: in Hyper-V emulation code 'tlb_flush' can now be dropped from 'structkvm_vcpu_hv' andkvm_make_scan_ioapic_request_mask()gets rid of dynamic allocation.
Quoted 6 lines
cpumask_available()checks inkvm_make_vcpu_request()andkvm_kick_many_cpus()can now be dropped as they checks for an impossible
condition:kvm_init()makes sure per-cpu masks are allocated.Signed-off-by Vitaly Kuznetsov <[email protected]>---
Quoted 7 lines
arch/x86/kvm/x86.c +0
-0@@ -9224,14 +9224,8 @@ static void process_smi(struct kvm_vcpu *vcpu)void kvm_make_scan_ioapic_request_mask (struct kvm *kvm,unsigned long *vcpu_bitmap)
arch/x86/kvm/x86.c static void process_smi(struct kvm_vcpu *vcpu) · line 9228
Quoted 10 lines
arch/x86/kvm/x86.c +1
-7{ -cpumask_var_t cpus; - -zalloc_cpumask_var (&cpus,GFP_ATOMIC ); -kvm_make_vcpus_request_mask (kvm,KVM_REQ_SCAN_IOAPIC , - vcpu_bitmap, cpus); - -free_cpumask_var (cpus); + vcpu_bitmap);
Nit, this can all go on a single line.
Quoted 3 lines
arch/x86/kvm/x86.c +0
-0}void kvm_make_scan_ioapic_request (struct kvm *kvm)
Vitaly Kuznetsov
maintainerRe: [PATCH v5 8/8] KVM: Make kvm_make_vcpus_request_mask() use pre-allocated cpu_kick_mask
Quoted 8 lines · Sean Christopherson
Quoted 7 lines · Vitaly Kuznetsov
kvm_make_vcpus_request_mask()already disables preemption so just likekvm_make_all_cpus_request_except()it can be switched to using pre-allocated per-cpu cpumasks. This allows for improvements for both users of the function: in Hyper-V emulation code 'tlb_flush' can now be dropped from 'structkvm_vcpu_hv' andkvm_make_scan_ioapic_request_mask()gets rid of dynamic allocation.
Quoted 9 lines
Quoted 6 lines
cpumask_available()checks inkvm_make_vcpu_request()andkvm_kick_many_cpus()can now be dropped as they checks for an impossible
condition:kvm_init()makes sure per-cpu masks are allocated.Signed-off-by Vitaly Kuznetsov <[email protected]>---
Reviewed-by Sean Christopherson <[email protected]>
Thanks!
Quoted 11 lines
Quoted 11 lines
arch/x86/kvm/x86.c +0
-3@@ -9224,14 +9224,8 @@ static void process_smi(struct kvm_vcpu *vcpu)void kvm_make_scan_ioapic_request_mask (struct kvm *kvm,unsigned long *vcpu_bitmap) { -cpumask_var_t cpus; - -zalloc_cpumask_var (&cpus,GFP_ATOMIC );
Quoted 9 lines
arch/x86/kvm/x86.c static void process_smi(struct kvm_vcpu *vcpu) · line 9228
Quoted 6 lines
arch/x86/kvm/x86.c +1
-4-kvm_make_vcpus_request_mask (kvm,KVM_REQ_SCAN_IOAPIC , - vcpu_bitmap, cpus); - -free_cpumask_var (cpus); + vcpu_bitmap);Nit, this can all go on a single line.
Sorry, you've mentioned this on v4 but I forgot. Hope this can be fixed upon commit...
Quoted 4 lines
Quoted 3 lines
arch/x86/kvm/x86.c +0
-0}void kvm_make_scan_ioapic_request (struct kvm *kvm)
Signature · 1 line
Vitaly
Vitaly Kuznetsov
maintainerQuoted 2 lines · Vitaly Kuznetsov
Patch6 fixes a real problem with
ioapic_write_indirect()KVM does out-of-bounds access to stack memory.
Paolo,
while the rest of the series is certainly not urgent, PATCH6 seems to be fixing a real problem introduced in 5.10. Would it be possible to send it for one of the upcoming 5.15 rcs (and probably to stable@)?
Thanks!
Signature · 1 line
Vitaly
Paolo Bonzini
maintainerQuoted 8 lines · Vitaly Kuznetsov
Quoted 3 lines
Patch6 fixes a real problem with
ioapic_write_indirect()KVM does out-of-bounds access to stack memory.Paolo,
while the rest of the series is certainly not urgent, PATCH6 seems to be fixing a real problem introduced in 5.10. Would it be possible to send it for one of the upcoming 5.15 rcs (and probably to stable@)?
Yep, I'm back to KVM now.
Paolo
Paolo Bonzini
maintainerQuoted 9 lines · Vitaly Kuznetsov
Changes since v4 (Sean):
- Add Reviewed-by: tag to PATCHes 3 and 5.
- Drop unneeded 'vcpu' initialization from PATCH4.
- Return-ENOMEMfromkvm_init()when cpumask allocation fails and drop unnecessary braces (PATCH 7).
- Dropcpumask_available()check fromkvm_kick_many_cpus()and convert kvm_make_vcpu_request()'s parameter from 'cpumask_var_t' to 'struct cpumask *' (PATCH 8)
Quoted 4 lines
This series is a continuation to Sean's "[PATCH 0/2] VM: Fix a benign race in kicking vCPUs" work and v2 for my "KVM: Optimize
kvm_make_vcpus_request_mask()a bit"/"KVM: x86: Fix stack-out-of-bounds memory access fromioapic_write_indirect()" patchset.
Now queued 3-4-5-7-8 as well, thanks.
Paolo
Quoted 50 lines
From Sean:
"Fix benign races when kicking vCPUs where the task doing the kicking can consume a stale
vcpu->cpu. The races are benign because of the impliciations of task migration with respect to interrupts and being in guest mode, but IMO they're worth fixing if only as an excuse to document the flows.Patch 2 is a tangentially related cleanup to prevent future me from trying to get rid of the NULL check on the cpumask parameters, which _looks_ like it can't ever be NULL, but has a subtle edge case due to the way
CONFIG_CPUMASK_OFFSTACK=yhandles cpumasks."Patch3 is a preparation to untangling
kvm_make_all_cpus_request_except()andkvm_make_vcpus_request_mask().Patch4 is a minor optimization for
kvm_make_vcpus_request_mask()for big guests.Patch5 is a minor cleanup.
Patch6 fixes a real problem with
ioapic_write_indirect()KVM does out-of-bounds access to stack memory.Patches7 and 8 get rid of dynamic cpumask allocation for kicking vCPUs.
Sean Christopherson (2):
KVM: Clean up benignvcpu->cpudata races when kicking vCPUs
KVM: KVM: Usecpumask_available()to check for NULL cpumask when
kicking vCPUsVitaly Kuznetsov (6):
KVM: x86: hyper-v: Avoid callingkvm_make_vcpus_request_mask()withvcpu_mask==NULLKVM: Optimize
kvm_make_vcpus_request_mask()a bit
KVM: Drop 'except' parameter fromkvm_make_vcpus_request_mask()
KVM: x86: Fix stack-out-of-bounds memory access from
ioapic_write_indirect()
KVM: Pre-allocate cpumasks forkvm_make_all_cpus_request_except()
KVM: Makekvm_make_vcpus_request_mask()use pre-allocated
cpu_kick_maskarch/x86/include/asm/kvm_host.h | 1 - arch/x86/kvm/hyperv.c | 18 ++--- arch/x86/kvm/ioapic.c | 10 +-- arch/x86/kvm/x86.c | 8 +-- include/linux/kvm_host.h | 3 +- virt/kvm/kvm_main.c | 115 +++++++++++++++++++++++--------- 6 files changed, 101 insertions(+), 54 deletions(-)