Skip to content

KVM: nVMX: Fix IPIv vs. nested posted interrupts

Sean Christopherson kvm lkml 2 versions – Applied

Status from this message: Applied to kvm-x86 vmx.

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v2 Latest7 patches 3 repliesApplied
  2. v16 patches 8 repliesApplied

Changes in v2 from the cover letter

- Split kvm_get_apic_interrupt() into has+ack to avoid marking the IRQ as in-service in vmcs02 instead of vmcs01. [Nathan]
- Gather reviews, but only for the patches that didn't meaningful change (all two of them). [Chao]
- Drop Cc: stable@ from all patches. For real world hypervisors, this is unlikely to cause functional issues, only loss of IPI virtualization performance due to the unnecessary VM-Exit. Whereas evidenced by my screwup in v1, this code is plenty subtle enough to introduce bugs.
- Drop the patch to store nested.posted_intr_nv as an int, as there is no need to explicitly match -1 (as a signed int) in this approach.
- Add a patch to assert vcpu->mutex is held when getting vmcs12, as I was "this" close to yanking out nested.posted_intr_nv, until I realized that accessing a different vCPU's vmcs12 in the IPI path is unsafe.

Patches in v2 Applied

#Subject and filesSizeRepliesApplied
1/7KVM: x86: Move "ack" phase of local APIC IRQ delivery to separate APIarch/x86/kvm/lapic.c arch/x86/kvm/lapic.h+14 -40 replies
2/7KVM: nVMX: Get to-be-acknowledge IRQ for nested VM-Exit at injection sitearch/x86/include/asm/kvm_host.h arch/x86/kvm/irq.c arch/x86/kvm/vmx/nested.c+30 -100 replies
3/7KVM: nVMX: Suppress external interrupt VM-Exit injection if there's no IRQarch/x86/kvm/vmx/nested.c+2 -10 replies
4/7KVM: nVMX: Detect nested posted interrupt NV at nested VM-Exit injectionarch/x86/kvm/vmx/nested.c+14 -00 replies
5/7KVM: x86: Fold kvm_get_apic_interrupt() into kvm_cpu_get_interrupt()arch/x86/kvm/irq.c arch/x86/kvm/lapic.c arch/x86/kvm/lapic.h+5 -130 replies
6/7KVM: nVMX: Explicitly invalidate posted_intr_nv if PI is disabled at VM-Enterarch/x86/kvm/vmx/nested.c arch/x86/kvm/vmx/vmx.c+11 -20 replies
7/7KVM: nVMX: Assert that vcpu->mutex is held when accessing secondary VMCSesarch/x86/kvm/vmx/nested.h+6 -00 replies

Reviews trailers per patch

2 of 7 patches have a Reviewed-by or Acked-by.

PatchChao Gao
1/7
2/7
3/7R-b
4/7R-b
5/7
6/7
7/7

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v2 thread

Cover letter [PATCH v2 0/7] KVM: nVMX: Fix IPIv vs. nested posted interrupts

Fix a bug where KVM injects L2's nested posted interrupt into L1 as a nested VM-Exit instead of triggering PI processing. The actual bug is technically a generic nested posted interrupts problem, but due to the way that KVM handles interrupt delivery, the issue is mostly limited to to IPI virtualization being enabled.

Found by the nested posted interrupt KUT test on SPR.

If it weren't for an annoying TOCTOU bug waiting to happen, the fix would be quite simple, e.g. it's really just:

arch/x86/kvm/vmx/nested.c +8 -0
index f7dde74ff565..b07805daedf5 100644
@@ -4288,6 +4288,15 @@ static int vmx_check_nested_events(struct kvm_vcpu *vcpu)
                        return -EBUSY;
                if (!nested_exit_on_intr(vcpu))
                        goto no_vmexit;
+
+               if (nested_cpu_has_posted_intr(get_vmcs12(vcpu)) &&
+                   kvm_apic_has_interrupt(vcpu) == vmx->nested.posted_intr_nv) {
+                       vmx->nested.pi_pending = true;
+                       kvm_apic_clear_irr(vcpu, vmx->nested.posted_intr_nv);
+                       goto no_vmexit;
+               }
+
                nested_vmx_vmexit(vcpu, EXIT_REASON_EXTERNAL_INTERRUPT, 0, 0);
                return 0;
        }

v2:
- Split kvm_get_apic_interrupt() into has+ack to avoid marking the IRQ as in-service in vmcs02 instead of vmcs01. [Nathan]
- Gather reviews, but only for the patches that didn't meaningful change (all two of them). [Chao]
- Drop Cc: stable@ from all patches. For real world hypervisors, this is unlikely to cause functional issues, only loss of IPI virtualization performance due to the unnecessary VM-Exit. Whereas evidenced by my screwup in v1, this code is plenty subtle enough to introduce bugs.
- Drop the patch to store nested.posted_intr_nv as an int, as there is no need to explicitly match -1 (as a signed int) in this approach.
- Add a patch to assert vcpu->mutex is held when getting vmcs12, as I was "this" close to yanking out nested.posted_intr_nv, until I realized that accessing a different vCPU's vmcs12 in the IPI path is unsafe.

v1: https://lore.kernel.org/all/[email protected]

Sean Christopherson (7):
KVM: x86: Move "ack" phase of local APIC IRQ delivery to separate API
KVM: nVMX: Get to-be-acknowledge IRQ for nested VM-Exit at injection
site
KVM: nVMX: Suppress external interrupt VM-Exit injection if there's no
IRQ
KVM: nVMX: Detect nested posted interrupt NV at nested VM-Exit
injection
KVM: x86: Fold kvm_get_apic_interrupt() into kvm_cpu_get_interrupt()
KVM: nVMX: Explicitly invalidate posted_intr_nv if PI is disabled at
VM-Enter
KVM: nVMX: Assert that vcpu->mutex is held when accessing secondary
VMCSes

 arch/x86/include/asm/kvm_host.h |  1 +
 arch/x86/kvm/irq.c              | 10 ++++--
 arch/x86/kvm/lapic.c            |  9 +++---
 arch/x86/kvm/lapic.h            |  2 +-
 arch/x86/kvm/vmx/nested.c       | 57 ++++++++++++++++++++++++++-------
 arch/x86/kvm/vmx/nested.h       |  6 ++++
 arch/x86/kvm/vmx/vmx.c          |  7 ++++
 7 files changed, 72 insertions(+), 20 deletions(-)

base-commit: 332d2c1d713e232e163386c35a3ba0c1b90df83f

Signature · 1 line
2.46.0.469.g59c65b2a67-goog