Fix the duplicate PMI injections in vPMU
cover Fix the duplicate PMI injections in vPMU
When we do stress test on KVM vPMU using Intel vtune, we find the following warning kernel message in the guest VM:
[ 1437.487320] Uhhuh. NMI received for unknown reason 20 on CPU 3. [ 1437.487330] Dazed and confused, but trying to continue
The Problem
===========
The above issue indicates that there are more NMIs injected than guest could recognize. After a month of investigation, we discovered that the bug happened due to minor glitches in two separate parts of the KVM: 1) KVM vPMU mistakenly fires a PMI due to emulated counter overflow even though the overflow has already been fired by the PMI handler on the host [1]. 2) KVM APIC allows multiple injections of PMI at one VM entry which violates Intel SDM. Both glitches contributes to extra injection of PMIs and thus confuses PMI handler in guest VM and causes the above warning messages.
The Fixes
=========
The patches disallow the multi-PMI injection fundamentally at APIC level. In addition, they also simplify the PMI injection process by removing irq_work and only use KVM_REQ_PMI.
The Testing
===========
With the series applied, we do not see the above warning messages when stress testing VM with Intel vtune. In addition, we add some kernel printing, all emulated counter overflow happens when hardware counter value is 0 and emulated counter value is 1 (prev_counter is -1). We never observed unexpected prev_counter values we saw in [2].
Note that this series does break the upstream kvm-unit-tests/pmu with the following error:
FAIL: Intel: emulated instruction: instruction counter overflow
FAIL: Intel: full-width writes: emulated instruction: instruction counter overflow
This is a test bug and apply the following diff should fix the issue:
x86/pmu.c +1 -0
@@ -68,6 +68,7 @@ volatile uint64_t irq_received;static void cnt_overflow (isr_regs_t *regs) { »......irq_received++; +»......apic_write(APIC_LVTPC, apic_read(APIC_LVTPC) & ~APIC_LVT_MASKED); »......apic_write(APIC_EOI, 0); }
We will post the above change soon.
[1] commit 9cd803d496e7 ("KVM: x86: Update vPMCs when retiring instructions")
[2] https://lore.kernel.org/all/CAL715WL9T8Ucnj_1AygwMgDjOJrttNZHRP9o-KUNfpx1aYZnog@mail.gmail.com/
Versioning
==========
The series is in v1. We made some changes:
- drop Dapeng's reviewed-by, since code changes.
- applies fix up in kvm_apic_local_deliver(). [seanjc]
- remove pmc->prev_counter. [seanjc]
Previous version (v0) shown as follows:
- [APIC patches v0]: https://lore.kernel.org/all/[email protected]/
- [vPMU patch v0]: https://lore.kernel.org/all/[email protected]/
Jim Mattson (2):
KVM: x86: Synthesize at most one PMI per VM-exit
KVM: x86: Mask LVTPC when handling a PMI
arch/x86/include/asm/kvm_host.h | 1 - arch/x86/kvm/lapic.c | 8 ++++++-- arch/x86/kvm/pmu.c | 27 +-------------------------- arch/x86/kvm/x86.c | 3 +++ 4 files changed, 10 insertions(+), 29 deletions(-)
base-commit: 6de2ccc169683bf81feba163834dae7cdebdd826
Signature · 1 line
2.42.0.515.g380fc7ccd1-goog
PATCH 1/2 KVM: x86: Synthesize at most one PMI per VM-exit
From: Jim Mattson <[email protected]>
When the irq_work callback, kvm_pmi_trigger_fn(), is invoked during a VM-exit that also invokes __kvm_perf_overflow() as a result of instruction emulation, kvm_pmu_deliver_pmi() will be called twice before the next VM-entry.
That shouldn't be a problem. The local APIC is supposed to automatically set the mask flag in LVTPC when it handles a PMI, so the second PMI should be inhibited. However, KVM's local APIC emulation fails to set the mask flag in LVTPC when it handles a PMI, so two PMIs are delivered via the local APIC. In the common case, where LVTPC is configured to deliver an NMI, the first NMI is vectored through the guest IDT, and the second one is held pending. When the NMI handler returns, the second NMI is vectored through the IDT. For Linux guests, this results in the "dazed and confused" spurious NMI message.
Though the obvious fix is to set the mask flag in LVTPC when handling a PMI, KVM's logic around synthesizing a PMI is unnecessarily convoluted.
Remove the irq_work callback for synthesizing a PMI, and all of the logic for invoking it. Instead, to prevent a vcpu from leaving C0 with a PMI pending, add a check for KVM_REQ_PMI to kvm_vcpu_has_events().
--- arch/x86/include/asm/kvm_host.h | 1 - arch/x86/kvm/pmu.c | 27 +-------------------------- arch/x86/kvm/x86.c | 3 +++ 3 files changed, 4 insertions(+), 27 deletions(-)
arch/x86/include/asm/kvm_host.h +0 -1
@@ -528,7 +528,6 @@ struct kvm_pmu {u64 raw_event_mask;struct kvm_pmc gp_counters[KVM_INTEL_PMC_MAX_GENERIC ];struct kvm_pmc fixed_counters[KVM_PMC_MAX_FIXED ]; -struct irq_work irq_work;/* * Overlay the bitmap with a 64-bit atomic so that all bits can be
arch/x86/kvm/pmu.c +1 -26
@@ -93,14 +93,6 @@ void kvm_pmu_ops_update(const struct kvm_pmu_ops *pmu_ops)#undef __KVM_X86_PMU_OP } -static void kvm_pmi_trigger_fn (struct irq_work *irq_work) -{ - struct kvm_pmu *pmu = container_of(irq_work, struct kvm_pmu, irq_work); -struct kvm_vcpu *vcpu =pmu_to_vcpu (pmu); - -kvm_pmu_deliver_pmi (vcpu); -} -static inline void __kvm_perf_overflow (struct kvm_pmc *pmc,bool in_pmi) {struct kvm_pmu *pmu =pmc_to_pmu (pmc);
@@ -124,20 +116,7 @@ static inline void __kvm_perf_overflow(struct kvm_pmc *pmc, bool in_pmi)__set_bit (pmc->idx, (unsigned long *)&pmu->global_status); } -if (!pmc->intr || skip_pmi) -return ; - -/* -* Inject PMI. If vcpu was in a guest mode during NMI PMI -* can be ejected on a guest mode re-entry. Otherwise we can't -* be sure that vcpu wasn't executing hlt instruction at the -* time of vmexit and is not going to re-enter guest mode until -* woken up. So we should wake it, but this is impossible from -* NMI context. Do it from irq work instead. -*/ -if (in_pmi && !kvm_handling_nmi_from_guest (pmc->vcpu)) -irq_work_queue (&pmc_to_pmu (pmc)->irq_work); -else +if (pmc->intr && !skip_pmi)kvm_make_request (KVM_REQ_PMI , pmc->vcpu); }
@@ -675,9 +654,6 @@ void kvm_pmu_refresh(struct kvm_vcpu *vcpu)void kvm_pmu_reset (struct kvm_vcpu *vcpu) { -struct kvm_pmu *pmu =vcpu_to_pmu (vcpu); - -irq_work_sync (&pmu->irq_work);static_call (kvm_x86_pmu_reset)(vcpu); }
@@ -687,7 +663,6 @@ void kvm_pmu_init(struct kvm_vcpu *vcpu)memset (pmu,0 ,sizeof (*pmu));static_call (kvm_x86_pmu_init)(vcpu); -init_irq_work (&pmu->irq_work, kvm_pmi_trigger_fn); pmu->event_count =0 ; pmu->need_cleanup = false;kvm_pmu_refresh (vcpu);
arch/x86/kvm/x86.c +3 -0
@@ -12820,6 +12820,9 @@ static inline bool kvm_vcpu_has_events(struct kvm_vcpu *vcpu)return true;#endif +if (kvm_test_request (KVM_REQ_PMI , vcpu)) +return true; +if (kvm_arch_interrupt_allowed (vcpu) && (kvm_cpu_has_interrupt (vcpu) ||kvm_guest_apic_has_interrupt (vcpu)))
Signature · 1 line
2.42.0.515.g380fc7ccd1-goog
Sean Christopherson
maintainerRe: [PATCH 1/2] KVM: x86: Synthesize at most one PMI per VM-exit
Quoted 17 lines · Mingwei Zhang
From: Jim Mattson <[email protected]>
When the
irq_workcallback,kvm_pmi_trigger_fn(), is invoked during a VM-exit that also invokes__kvm_perf_overflow()as a result of instruction emulation,kvm_pmu_deliver_pmi()will be called twice before the next VM-entry.That shouldn't be a problem. The local APIC is supposed to automatically set the mask flag in LVTPC when it handles a PMI, so the second PMI should be inhibited. However, KVM's local APIC emulation fails to set the mask flag in LVTPC when it handles a PMI, so two PMIs are delivered via the local APIC. In the common case, where LVTPC is configured to deliver an NMI, the first NMI is vectored through the guest IDT, and the second one is held pending. When the NMI handler returns, the second NMI is vectored through the IDT. For Linux guests, this results in the "dazed and confused" spurious NMI message.
Quoted 3 lines
Though the obvious fix is to set the mask flag in LVTPC when handling a PMI, KVM's logic around synthesizing a PMI is unnecessarily convoluted.
Unless Jim outright objects, I strongly prefer placing this patch second, with the above two paragraphs replaced with my suggestion (or something similar):
Calling kvm_pmu_deliver_pmi() twice is unlikely to be problematic now that KVM sets the LVTPC mask bit when delivering a PMI. But using IRQ work to trigger the PMI is still broken, albeit very theoretically.
E.g. if the self-IPI to trigger IRQ work is be delayed long enough for the vCPU to be migrated to a different pCPU, then it's possible for kvm_pmi_trigger_fn() to race with the kvm_pmu_deliver_pmi() from KVM_REQ_PMI and still generate two PMIs.
KVM could set the mask bit using an atomic operation, but that'd just be piling on unnecessary code to workaround what is effectively a hack. The *only* reason KVM uses IRQ work is to ensure the PMI is treated as a wake event, e.g. if the vCPU just executed HLT.
I understand Jim's desire for the patch to be more obviously valuable, but the people that need convincing are already convinced that the patch is worth taking.
Quoted 8 lines
Remove the
irq_workcallback for synthesizing a PMI, and all of the logic for invoking it. Instead, to prevent a vcpu from leaving C0 with a PMI pending, add a check forKVM_REQ_PMItokvm_vcpu_has_events().Fixes 9cd803d496e7 ("KVM: x86: Update vPMCs when retiring instructions")Signed-off-by Jim Mattson <[email protected]>Tested-by Mingwei Zhang <[email protected]>Tested-by Dapeng Mi <[email protected]>
Needs your SoB
Re: [PATCH 1/2] KVM: x86: Synthesize at most one PMI per VM-exit
Quoted 47 lines · Sean Christopherson
Quoted 17 lines · Mingwei Zhang
From: Jim Mattson <[email protected]>
When the
irq_workcallback,kvm_pmi_trigger_fn(), is invoked during a VM-exit that also invokes__kvm_perf_overflow()as a result of instruction emulation,kvm_pmu_deliver_pmi()will be called twice before the next VM-entry.That shouldn't be a problem. The local APIC is supposed to automatically set the mask flag in LVTPC when it handles a PMI, so the second PMI should be inhibited. However, KVM's local APIC emulation fails to set the mask flag in LVTPC when it handles a PMI, so two PMIs are delivered via the local APIC. In the common case, where LVTPC is configured to deliver an NMI, the first NMI is vectored through the guest IDT, and the second one is held pending. When the NMI handler returns, the second NMI is vectored through the IDT. For Linux guests, this results in the "dazed and confused" spurious NMI message.
Quoted 3 lines
Though the obvious fix is to set the mask flag in LVTPC when handling a PMI, KVM's logic around synthesizing a PMI is unnecessarily convoluted.
Unless Jim outright objects, I strongly prefer placing this patch second, with the above two paragraphs replaced with my suggestion (or something similar):
Calling
kvm_pmu_deliver_pmi()twice is unlikely to be problematic now that KVM sets the LVTPC mask bit when delivering a PMI. But using IRQ work to trigger the PMI is still broken, albeit very theoretically.E.g. if the self-IPI to trigger IRQ work is be delayed long enough for the vCPU to be migrated to a different pCPU, then it's possible for
kvm_pmi_trigger_fn()to race with thekvm_pmu_deliver_pmi()fromKVM_REQ_PMIand still generate two PMIs.KVM could set the mask bit using an atomic operation, but that'd just be piling on unnecessary code to workaround what is effectively a hack. The *only* reason KVM uses IRQ work is to ensure the PMI is treated as a wake event, e.g. if the vCPU just executed HLT.
I understand Jim's desire for the patch to be more obviously valuable, but the people that need convincing are already convinced that the patch is worth taking.
Quoted 4 lines
Remove the
irq_workcallback for synthesizing a PMI, and all of the logic for invoking it. Instead, to prevent a vcpu from leaving C0 with a PMI pending, add a check forKVM_REQ_PMItokvm_vcpu_has_events().
Quoted 6 lines
Quoted 4 lines
Fixes 9cd803d496e7 ("KVM: x86: Update vPMCs when retiring instructions")Signed-off-by Jim Mattson <[email protected]>Tested-by Mingwei Zhang <[email protected]>Tested-by Dapeng Mi <[email protected]>Needs your SoB
Sean Christopherson
maintainerRe: [PATCH 1/2] KVM: x86: Synthesize at most one PMI per VM-exit
Quoted 48 lines · Mingwei Zhang
Quoted 47 lines · Sean Christopherson
Quoted 17 lines · Mingwei Zhang
From: Jim Mattson <[email protected]>
When the
irq_workcallback,kvm_pmi_trigger_fn(), is invoked during a VM-exit that also invokes__kvm_perf_overflow()as a result of instruction emulation,kvm_pmu_deliver_pmi()will be called twice before the next VM-entry.That shouldn't be a problem. The local APIC is supposed to automatically set the mask flag in LVTPC when it handles a PMI, so the second PMI should be inhibited. However, KVM's local APIC emulation fails to set the mask flag in LVTPC when it handles a PMI, so two PMIs are delivered via the local APIC. In the common case, where LVTPC is configured to deliver an NMI, the first NMI is vectored through the guest IDT, and the second one is held pending. When the NMI handler returns, the second NMI is vectored through the IDT. For Linux guests, this results in the "dazed and confused" spurious NMI message.
Quoted 3 lines
Though the obvious fix is to set the mask flag in LVTPC when handling a PMI, KVM's logic around synthesizing a PMI is unnecessarily convoluted.
Unless Jim outright objects, I strongly prefer placing this patch second, with the above two paragraphs replaced with my suggestion (or something similar):
Calling
kvm_pmu_deliver_pmi()twice is unlikely to be problematic now that KVM sets the LVTPC mask bit when delivering a PMI. But using IRQ work to trigger the PMI is still broken, albeit very theoretically.E.g. if the self-IPI to trigger IRQ work is be delayed long enough for the vCPU to be migrated to a different pCPU, then it's possible for
kvm_pmi_trigger_fn()to race with thekvm_pmu_deliver_pmi()fromKVM_REQ_PMIand still generate two PMIs.KVM could set the mask bit using an atomic operation, but that'd just be piling on unnecessary code to workaround what is effectively a hack. The *only* reason KVM uses IRQ work is to ensure the PMI is treated as a wake event, e.g. if the vCPU just executed HLT.
I understand Jim's desire for the patch to be more obviously valuable, but the people that need convincing are already convinced that the patch is worth taking.
Quoted 4 lines
Remove the
irq_workcallback for synthesizing a PMI, and all of the logic for invoking it. Instead, to prevent a vcpu from leaving C0 with a PMI pending, add a check forKVM_REQ_PMItokvm_vcpu_has_events().
Quoted 8 lines
Quoted 6 lines
Quoted 4 lines
Fixes 9cd803d496e7 ("KVM: x86: Update vPMCs when retiring instructions")Signed-off-by Jim Mattson <[email protected]>Tested-by Mingwei Zhang <[email protected]>Tested-by Dapeng Mi <[email protected]>Needs your SoB
Signed-off-by Mingwei Zhang <[email protected]>
Thanks!
Jim gave his blessing off-list for swapping the order, I'll do that and massage the changelogs when applying, i.e. no need for a v3.
PATCH 2/2 KVM: x86: Mask LVTPC when handling a PMI
In reply to Mingwei ZhangFrom: Jim Mattson <[email protected]>
Per the SDM, "When the local APIC handles a performance-monitoring counters interrupt, it automatically sets the mask flag in the LVT performance counter register."
Add this behavior to KVM's local APIC emulation, to reduce the incidence of "dazed and confused" spurious NMI warnings in Linux guests (at least, those that use a PMI handler with "late_ack").
--- arch/x86/kvm/lapic.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
arch/x86/kvm/lapic.c +6 -2
@@ -2729,13 +2729,17 @@ int kvm_apic_local_deliver(struct kvm_lapic *apic, int lvt_type) {u32 reg =kvm_lapic_get_reg (apic, lvt_type);int vector, mode, trig_mode; +int r;if (kvm_apic_hw_enabled (apic) && !(reg &APIC_LVT_MASKED )) { vector = reg &APIC_VECTOR_MASK ; mode = reg &APIC_MODE_MASK ; trig_mode = reg &APIC_LVT_LEVEL_TRIGGER ; -return __apic_accept_irq (apic, mode, vector,1 , trig_mode, -NULL ); + + r =__apic_accept_irq (apic, mode, vector,1 , trig_mode,NULL ); +if (r && lvt_type ==APIC_LVTPC ) +kvm_lapic_set_reg (apic, lvt_type, reg |APIC_LVT_MASKED ); +return r; }return 0 ; }
Signature · 1 line
2.42.0.515.g380fc7ccd1-goog
Sean Christopherson
maintainerRe: [PATCH 2/2] KVM: x86: Mask LVTPC when handling a PMI
Quoted 11 lines · Mingwei Zhang
From: Jim Mattson <[email protected]>
Per the SDM, "When the local APIC handles a performance-monitoring counters interrupt, it automatically sets the mask flag in the LVT performance counter register."
Add this behavior to KVM's local APIC emulation, to reduce the incidence of "dazed and confused" spurious NMI warnings in Linux guests (at least, those that use a PMI handler with "
late_ack").Fixes 23930f9521c9 ("KVM: x86: Enable NMI Watchdog via in-kernel PIT source")
This Fixes is wrong. Prior to commit f5132b01386b ("KVM: Expose a version 2 architectural PMU to a guests"), KVM didn't ever deliver interrupts via the LVTPC entry. E.g. prior to that commit, the only reference to APIC_LVTPC is in kvm_lapic_reg_write:
arch/x86/kvm $ git grep APIC_LVTPC f5132b01386b^ f5132b01386b^:lapic.c: case APIC_LVTPC:
Commit 23930f9521c9 definitely set the PMU support up to fail, but the bug would never have existed if kvm_deliver_pmi() had been written as:
void kvm_deliver_pmi(struct kvm_vcpu *vcpu)
{
struct kvm_lapic *apic = vcpu->arch.apic;
if (apic && kvm_apic_local_deliver(apic, APIC_LVTPC))
kvm_lapic_set_reg(apic, APIC_LVTPC,
kvm_lapic_get_reg(apic, LVTPC) | APIC_LVT_MASKED);
}And this needs an explicit Cc: to stable because KVM opts out of AUTOSEL.
So
Quoted 2 lines
Signed-off-by Jim Mattson <[email protected]>Tested-by Mingwei Zhang <[email protected]>
When posting patches on behalf of others, you need to provide your SoB.
Quoted 14 lines
--- arch/x86/kvm/lapic.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)arch/x86/kvm/lapic.c +1
-0@@ -2729,13 +2729,17 @@ int kvm_apic_local_deliver(struct kvm_lapic *apic, int lvt_type) {u32 reg =kvm_lapic_get_reg (apic, lvt_type);int vector, mode, trig_mode; +int r;
arch/x86/kvm/lapic.c int kvm_apic_local_deliver(struct kvm_lapic *apic, int lvt_type) · line 2741
Quoted 10 lines
arch/x86/kvm/lapic.c +4
-2if (kvm_apic_hw_enabled (apic) && !(reg &APIC_LVT_MASKED )) { vector = reg &APIC_VECTOR_MASK ; mode = reg &APIC_MODE_MASK ; trig_mode = reg &APIC_LVT_LEVEL_TRIGGER ; -return __apic_accept_irq (apic, mode, vector,1 , trig_mode, -NULL ); + + r =__apic_accept_irq (apic, mode, vector,1 , trig_mode,NULL ); +if (r && lvt_type ==APIC_LVTPC ) +kvm_lapic_set_reg (apic, lvt_type, reg |APIC_LVT_MASKED );
Belated feedback, I think I'd prefer to write this as
kvm_lapic_set_reg(apic, APIC_LVTPC, reg | APIC_LVT_MASKED);
so that this code will show up when searching for APIC_LVTPC.
Quoted 7 lines
arch/x86/kvm/lapic.c +1
-0+return r; }return 0 ; }Signature · 1 line
2.42.0.515.g380fc7ccd1-goog
Re: [PATCH 2/2] KVM: x86: Mask LVTPC when handling a PMI
Quoted 73 lines · Sean Christopherson
Quoted 11 lines · Mingwei Zhang
From: Jim Mattson <[email protected]>
Per the SDM, "When the local APIC handles a performance-monitoring counters interrupt, it automatically sets the mask flag in the LVT performance counter register."
Add this behavior to KVM's local APIC emulation, to reduce the incidence of "dazed and confused" spurious NMI warnings in Linux guests (at least, those that use a PMI handler with "
late_ack").Fixes 23930f9521c9 ("KVM: x86: Enable NMI Watchdog via in-kernel PIT source")This Fixes is wrong. Prior to commit
f5132b01386b("KVM: Expose a version 2 architectural PMU to a guests"), KVM didn't ever deliver interrupts via the LVTPC entry. E.g. prior to that commit, the only reference toAPIC_LVTPCis inkvm_lapic_reg_write:
arch/x86/kvm$ git grepAPIC_LVTPCf5132b01386b^ f5132b01386b^:lapic.c: caseAPIC_LVTPC:Commit
23930f9521c9definitely set the PMU support up to fail, but the bug would never have existed ifkvm_deliver_pmi()had been written as:void
kvm_deliver_pmi(struct kvm_vcpu *vcpu){ struct kvm_lapic *apic = vcpu->arch.apic; if (apic && kvm_apic_local_deliver(apic, APIC_LVTPC)) kvm_lapic_set_reg(apic, APIC_LVTPC, kvm_lapic_get_reg(apic, LVTPC) | APIC_LVT_MASKED); }And this needs an explicit Cc: to stable because KVM opts out of AUTOSEL.
So
Fixes f5132b01386b ("KVM: Expose a version 2 architectural PMU to a guests")Quoted 2 lines
Signed-off-by Jim Mattson <[email protected]>Tested-by Mingwei Zhang <[email protected]>When posting patches on behalf of others, you need to provide your SoB.
Quoted 14 lines
--- arch/x86/kvm/lapic.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)arch/x86/kvm/lapic.c +1
-0@@ -2729,13 +2729,17 @@ int kvm_apic_local_deliver(struct kvm_lapic *apic, int lvt_type) {u32 reg =kvm_lapic_get_reg (apic, lvt_type);int vector, mode, trig_mode; +int r;arch/x86/kvm/lapic.c int kvm_apic_local_deliver(struct kvm_lapic *apic, int lvt_type) · line 2741
Quoted 10 lines
arch/x86/kvm/lapic.c +4
-2if (kvm_apic_hw_enabled (apic) && !(reg &APIC_LVT_MASKED )) { vector = reg &APIC_VECTOR_MASK ; mode = reg &APIC_MODE_MASK ; trig_mode = reg &APIC_LVT_LEVEL_TRIGGER ; -return __apic_accept_irq (apic, mode, vector,1 , trig_mode, -NULL ); + + r =__apic_accept_irq (apic, mode, vector,1 , trig_mode,NULL ); +if (r && lvt_type ==APIC_LVTPC ) +kvm_lapic_set_reg (apic, lvt_type, reg |APIC_LVT_MASKED );Belated feedback, I think I'd prefer to write this as
Quoted 11 lines
kvm_lapic_set_reg(apic, APIC_LVTPC, reg | APIC_LVT_MASKED);so that this code will show up when searching for
APIC_LVTPC.Quoted 7 lines
arch/x86/kvm/lapic.c +1
-1+return r; }return 0 ; } --2.42.0.515.g380fc7ccd1-goog
Quoted 10 lines · Mingwei Zhang
When we do stress test on KVM vPMU using Intel vtune, we find the following warning kernel message in the guest VM:
[ 1437.487320] Uhhuh. NMI received for unknown reason 20 on CPU 3. [ 1437.487330] Dazed and confused, but trying to continueThe Problem
===========[...]
Applied to kvm-x86 pmu, with the order swapped and a bit of changelog massaging. Thanks!
[1/2] KVM: x86: Mask LVTPC when handling a PMI
https://github.com/kvm-x86/linux/commit/a16eb25b09c0 [2/2] KVM: x86/pmu: Synthesize at most one PMI per VM-exit https://github.com/kvm-x86/linux/commit/73554b29bd70