Fix a race between posted interrupt delivery and migration in a nested VM
v2 cover Fix a race between posted interrupt delivery and migration in a nested VM
This patch set aims to fix a race condition between posted interrupt delivery and migration for a nested VM. In particular, we proves that when a nested vCPU is halted and just migrated, it will lose a posted interrupt from another vCPU in the same VM.
Changelog:
v1 -> v2:
- Replace the original vmcs12 bug fix patch into one that processes nested state pages request in a common function [paolo].
- Update the commit messages [seanjc, oupton].
- Remove vcpu_run_interruptable(), use __vcpu_run() instead [seanjc].
- Fix format issue in prepare_posted_intr_desc() [seanjc].
- Rebase to kvm/queue.
v1 link:
- https://lore.kernel.org/lkml/[email protected]/t/
Jim Mattson (1):
KVM: selftests: Test if posted interrupt delivery race with migration
Mingwei Zhang (3):
KVM: x86: move the event handling of KVM_REQ_GET_VMCS12_PAGES into a
common function
KVM: selftests: Save/restore vAPIC state in migration tests
KVM: selftests: Add support for posted interrupt handling in L2
arch/x86/kvm/x86.c | 29 +- tools/testing/selftests/kvm/.gitignore | 1 + tools/testing/selftests/kvm/Makefile | 1 + .../selftests/kvm/include/kvm_util_base.h | 10 + .../selftests/kvm/include/x86_64/processor.h | 1 + .../selftests/kvm/include/x86_64/vmx.h | 10 + .../selftests/kvm/lib/x86_64/processor.c | 2 + tools/testing/selftests/kvm/lib/x86_64/vmx.c | 14 + .../kvm/x86_64/vmx_migrate_pi_pending.c | 291 ++++++++++++++++++ 9 files changed, 353 insertions(+), 6 deletions(-) create mode 100644 tools/testing/selftests/kvm/x86_64/vmx_migrate_pi_pending.c
base-commit: 372d07084593dc7a399bf9bee815711b1fb1bcf2
Signature · 1 line
2.37.2.672.g94769d06f0-goog
PATCH v2 1/4 KVM: x86: move the event handling of KVM_REQ_GET_VMCS12_PAGES into a common function
Create a common function to handle kvm request in the vcpu_run loop. KVM implicitly assumes the virtual APIC page being present + mapped into the kernel address space when executing vmx_guest_apic_has_interrupts(). However, with demand paging KVM breaks the assumption, as the KVM_REQ_GET_VMCS12_PAGES event isn't assessed before entering vcpu_block.
Fix this by getting vmcs12 pages before inspecting the guest's APIC page. Because of this fix, the event handling code of KVM_REQ_GET_NESTED_STATE_PAGES becomes a common code path for both vcpu_enter_guest() and vcpu_block(). Thus, put this code snippet into a common helper function to avoid code duplication.
--- arch/x86/kvm/x86.c | 29 +++++++++++++++++++++++------ 1 file changed, 23 insertions(+), 6 deletions(-)
arch/x86/kvm/x86.c +23 -6
@@ -10261,12 +10261,6 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) r = -EIO ; goto out; } -if (kvm_check_request (KVM_REQ_GET_NESTED_STATE_PAGES , vcpu)) { -if (unlikely (!kvm_x86_ops.nested_ops->get_nested_state_pages (vcpu))) { - r =0 ; - goto out; - } - }if (kvm_check_request (KVM_REQ_MMU_FREE_OBSOLETE_ROOTS , vcpu))kvm_mmu_free_obsolete_roots (vcpu);if (kvm_check_request (KVM_REQ_MIGRATE_TIMER , vcpu))
@@ -10666,6 +10660,23 @@ static inline bool kvm_vcpu_running(struct kvm_vcpu *vcpu) !vcpu->arch.apf.halted); } +static int kvm_vcpu_handle_common_requests (struct kvm_vcpu *vcpu) +{ +if (kvm_request_pending (vcpu)) { +/* +* Get the vmcs12 pages before checking for interrupts that +* might unblock the guest if L1 is using virtual-interrupt +* delivery. +*/ +if (kvm_check_request (KVM_REQ_GET_NESTED_STATE_PAGES , vcpu)) { +if (unlikely (!kvm_x86_ops.nested_ops->get_nested_state_pages (vcpu))) +return 0 ; + } + } + +return 1 ; +} +/* Called within kvm->srcu read side. */ static int vcpu_run (struct kvm_vcpu *vcpu) {
@@ -10681,6 +10692,12 @@ static int vcpu_run(struct kvm_vcpu *vcpu)* this point can start executing an instruction. */ vcpu->arch.at_instruction_boundary = false; + +/* Process common request regardless of vcpu state. */ + r =kvm_vcpu_handle_common_requests (vcpu); +if (r <=0 ) +break ; +if (kvm_vcpu_running (vcpu)) { r =vcpu_enter_guest (vcpu); }else {
Signature · 1 line
2.37.2.672.g94769d06f0-goog
Sean Christopherson
maintainerRe: [PATCH v2 1/4] KVM: x86: move the event handling of KVM_REQ_GET_VMCS12_PAGES into a common function
Quoted 5 lines · Mingwei Zhang
Create a common function to handle kvm request in the
vcpu_runloop. KVM implicitly assumes the virtual APIC page being present + mapped into the kernel address space when executingvmx_guest_apic_has_interrupts(). However, with demand paging KVM breaks the assumption, as theKVM_REQ_GET_VMCS12_PAGESevent isn't assessed before enteringvcpu_block.
KVM_REQ_GET_VMCS12_PAGES doesn't exist upstream.
Quoted 10 lines
Fix this by getting vmcs12 pages before inspecting the guest's APIC page. Because of this fix, the event handling code of
KVM_REQ_GET_NESTED_STATE_PAGESbecomes a common code path for bothvcpu_enter_guest()andvcpu_block(). Thus, put this code snippet into a common helper function to avoid code duplication.Cc Maxim Levitsky <[email protected]>Cc Vitaly Kuznetsov <[email protected]>Originally-by Oliver Upton <[email protected]>Signed-off-by Oliver Upton <[email protected]>
If you drop someone as author, then their SOB also needs to be jettisoned.
Quoted 23 lines
Signed-off-by Mingwei Zhang <[email protected]>--- arch/x86/kvm/x86.c | 29 +++++++++++++++++++++++------ 1 file changed, 23 insertions(+), 6 deletions(-)arch/x86/kvm/x86.c +0
-6@@ -10261,12 +10261,6 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) r = -EIO ; goto out; } -if (kvm_check_request (KVM_REQ_GET_NESTED_STATE_PAGES , vcpu)) { -if (unlikely (!kvm_x86_ops.nested_ops->get_nested_state_pages (vcpu))) { - r =0 ; - goto out; - } - }if (kvm_check_request (KVM_REQ_MMU_FREE_OBSOLETE_ROOTS , vcpu))kvm_mmu_free_obsolete_roots (vcpu);if (kvm_check_request (KVM_REQ_MIGRATE_TIMER , vcpu))
arch/x86/kvm/x86.c static inline bool kvm_vcpu_running(struct kvm_vcpu *vcpu) · line 10665
Quoted 6 lines
arch/x86/kvm/x86.c +3
-0@@ -10666,6 +10660,23 @@ static inline bool kvm_vcpu_running(struct kvm_vcpu *vcpu) !vcpu->arch.apf.halted); } +static int kvm_vcpu_handle_common_requests (struct kvm_vcpu *vcpu) +{ +if (kvm_request_pending (vcpu)) {
Probably going to be a moot point, but write this as
if (!kvm_request_pending(vcpu)) return 1;
to reduce indentation.
arch/x86/kvm/x86.c static inline bool kvm_vcpu_running(struct kvm_vcpu *vcpu) · line 10672
Quoted 7 lines
arch/x86/kvm/x86.c +7
-0+/* +* Get the vmcs12 pages before checking for interrupts that +* might unblock the guest if L1 is using virtual-interrupt +* delivery. +*/ +if (kvm_check_request (KVM_REQ_GET_NESTED_STATE_PAGES , vcpu)) { +if (unlikely (!kvm_x86_ops.nested_ops->get_nested_state_pages (vcpu)))
Similarly
if (kvm_check_request(KVM_REQ_GET_NESTED_STATE_PAGES, vcpu) && unlikely(!kvm_x86_ops.nested_ops->get_nested_state_pages(vcpu))) return 0;
though I can see the argument for fully isolating each request. But again, likely a moot point.
Quoted 11 lines
arch/x86/kvm/x86.c +7
-0+return 0 ; + } + } + +return 1 ; +} +/* Called within kvm->srcu read side. */ static int vcpu_run (struct kvm_vcpu *vcpu) {
arch/x86/kvm/x86.c static int vcpu_run(struct kvm_vcpu *vcpu) · line 10697
Quoted 6 lines
arch/x86/kvm/x86.c +3
-0@@ -10681,6 +10692,12 @@ static int vcpu_run(struct kvm_vcpu *vcpu)* this point can start executing an instruction. */ vcpu->arch.at_instruction_boundary = false; + +/* Process common request regardless of vcpu state. */ + r =kvm_vcpu_handle_common_requests (vcpu);
IMO this is subtly a dangerous hook. It implies that both vcpu_enter_guest() and vcpu_block() correctly handle requests becoming pending after the "common" check, but that's not actually the case. If a request _needs_ to be handled before vcpu_block(), then ideally it should be explicitly queried in kvm_vcpu_check_block(). KVM_REQ_GET_NESTED_STATE_PAGES doesn't have issues because it's only ever set from the vCPU itself.
Following that train of thought, KVM_REQ_GET_NESTED_STATE_PAGES really shouldn't even be a request. Aha! And we can do that in a way that would magically fix this bug, and would ensure we don't leave a trap for future us.
KVM already provides KVM_REQ_UNBLOCK to prevent blocking the vCPU without actaully waking the vCPU, i.e. to kick the vCPU back into the vcpu_run() loop. The request is provided specifically for scenarios like this where KVM needs to do work before blocking.
Normally I'd say we should do this over multiple patches so that the "blocking" bug is fixed before doing the rework/cleanup, but I'm ok if we want to skip straight to the rework since we're obviously carrying an internal patch and no one else is likely to need the fix. But I also wouldn't object to including an intermediate patch to fix the bug so that there's a better paper trail.
E.g. as a very partial conversion:
--- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/vmx/nested.c | 2 +- arch/x86/kvm/x86.c | 12 ++++++++++++ arch/x86/kvm/x86.h | 10 ++++++++++ 4 files changed, 25 insertions(+), 1 deletion(-)
arch/x86/include/asm/kvm_host.h +2 -0
@@ -939,6 +939,8 @@ struct kvm_vcpu_arch {*/ bool pdptrs_from_userspace; +bool nested_get_pages_pending; +#if IS_ENABLED (CONFIG_HYPERV )hpa_t hv_root_tdp;#endif
arch/x86/kvm/vmx/nested.c +1 -1
@@ -3446,7 +3446,7 @@ enum nvmx_vmentry_status nested_vmx_enter_non_root_mode(struct kvm_vcpu *vcpu,* to nested_get_vmcs12_pages before the next VM-entry. The MSRs * have already been set at vmentry time and should not be reset. */ -kvm_make_request (KVM_REQ_GET_NESTED_STATE_PAGES , vcpu); +kvm_nested_get_pages_set_pending (vcpu); }/*
arch/x86/kvm/x86.c +12 -0
@@ -9650,6 +9650,12 @@ int kvm_check_nested_events(struct kvm_vcpu *vcpu)return kvm_x86_ops.nested_ops->check_events (vcpu); } +static int kvm_get_nested_state_pages (struct kvm_vcpu *vcpu) +{ + vcpu->arch.nested_get_pages_pending = false; +return kvm_x86_ops.nested_ops->get_nested_state_pages (vcpu); +} +static void kvm_inject_exception (struct kvm_vcpu *vcpu) { trace_kvm_inj_exception(vcpu->arch.exception.nr,
@@ -10700,6 +10706,12 @@ static int vcpu_run(struct kvm_vcpu *vcpu)if (kvm_cpu_has_pending_timer (vcpu))kvm_inject_pending_timer_irqs (vcpu); +if (vcpu->arch.nested_get_pages_pending) { + r =kvm_get_nested_state_pages (vcpu); +if (r <=0 ) +break ; + } +if (dm_request_for_irq_injection (vcpu) &&kvm_vcpu_ready_for_interrupt_injection (vcpu)) { r =0 ;
arch/x86/kvm/x86.h +10 -0
@@ -481,4 +481,14 @@ int kvm_sev_es_string_io(struct kvm_vcpu *vcpu, unsigned int size, unsigned int port, void *data, unsigned int count, int in); +static inline void kvm_nested_get_pages_set_pending (struct kvm_vcpu *vcpu) +{ +/* +* Here is a comment explaining why KVM needs to prevent the vCPU from +* blocking until the vCPU's nested pages have been loaded. +*/ + vcpu->arch.nested_get_pages_pending = true; +kvm_make_request (KVM_REQ_UNBLOCK , vcpu); +} +#endif
base-commit: 14a47a98151834c5bd2f6d8d592b01108a3f882a --
Re: [PATCH v2 1/4] KVM: x86: move the event handling of KVM_REQ_GET_VMCS12_PAGES into a common function
Quoted 8 lines · Sean Christopherson
Quoted 5 lines · Mingwei Zhang
Create a common function to handle kvm request in the
vcpu_runloop. KVM implicitly assumes the virtual APIC page being present + mapped into the kernel address space when executingvmx_guest_apic_has_interrupts(). However, with demand paging KVM breaks the assumption, as theKVM_REQ_GET_VMCS12_PAGESevent isn't assessed before enteringvcpu_block.
KVM_REQ_GET_VMCS12_PAGESdoesn't exist upstream.
ack.
Quoted 7 lines
Quoted 7 lines
Fix this by getting vmcs12 pages before inspecting the guest's APIC page. Because of this fix, the event handling code of
KVM_REQ_GET_NESTED_STATE_PAGESbecomes a common code path for bothvcpu_enter_guest()andvcpu_block(). Thus, put this code snippet into a common helper function to avoid code duplication.
Quoted 7 lines
Quoted 4 lines
Cc Maxim Levitsky <[email protected]>Cc Vitaly Kuznetsov <[email protected]>Originally-by Oliver Upton <[email protected]>Signed-off-by Oliver Upton <[email protected]>If you drop someone as author, then their SOB also needs to be jettisoned.
ack.
Quoted 79 lines
Quoted 23 lines
Signed-off-by Mingwei Zhang <[email protected]>--- arch/x86/kvm/x86.c | 29 +++++++++++++++++++++++------ 1 file changed, 23 insertions(+), 6 deletions(-)arch/x86/kvm/x86.c +0
-6@@ -10261,12 +10261,6 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) r = -EIO ; goto out; } -if (kvm_check_request (KVM_REQ_GET_NESTED_STATE_PAGES , vcpu)) { -if (unlikely (!kvm_x86_ops.nested_ops->get_nested_state_pages (vcpu))) { - r =0 ; - goto out; - } - }if (kvm_check_request (KVM_REQ_MMU_FREE_OBSOLETE_ROOTS , vcpu))kvm_mmu_free_obsolete_roots (vcpu);if (kvm_check_request (KVM_REQ_MIGRATE_TIMER , vcpu))arch/x86/kvm/x86.c static inline bool kvm_vcpu_running(struct kvm_vcpu *vcpu) · line 10665
Quoted 6 lines
arch/x86/kvm/x86.c +3
-0@@ -10666,6 +10660,23 @@ static inline bool kvm_vcpu_running(struct kvm_vcpu *vcpu) !vcpu->arch.apf.halted); } +static int kvm_vcpu_handle_common_requests (struct kvm_vcpu *vcpu) +{ +if (kvm_request_pending (vcpu)) {Probably going to be a moot point, but write this as
if (!kvm_request_pending(vcpu)) return 1;to reduce indentation.
arch/x86/kvm/x86.c static inline bool kvm_vcpu_running(struct kvm_vcpu *vcpu) · line 10672
Quoted 7 lines
arch/x86/kvm/x86.c +7
-0+/* +* Get the vmcs12 pages before checking for interrupts that +* might unblock the guest if L1 is using virtual-interrupt +* delivery. +*/ +if (kvm_check_request (KVM_REQ_GET_NESTED_STATE_PAGES , vcpu)) { +if (unlikely (!kvm_x86_ops.nested_ops->get_nested_state_pages (vcpu)))Similarly
if (kvm_check_request(KVM_REQ_GET_NESTED_STATE_PAGES, vcpu) && unlikely(!kvm_x86_ops.nested_ops->get_nested_state_pages(vcpu))) return 0;though I can see the argument for fully isolating each request. But again, likely a moot point.
Quoted 11 lines
arch/x86/kvm/x86.c +7
-0+return 0 ; + } + } + +return 1 ; +} +/* Called within kvm->srcu read side. */ static int vcpu_run (struct kvm_vcpu *vcpu) {arch/x86/kvm/x86.c static int vcpu_run(struct kvm_vcpu *vcpu) · line 10697
Quoted 6 lines
arch/x86/kvm/x86.c +3
-0@@ -10681,6 +10692,12 @@ static int vcpu_run(struct kvm_vcpu *vcpu)* this point can start executing an instruction. */ vcpu->arch.at_instruction_boundary = false; + +/* Process common request regardless of vcpu state. */ + r =kvm_vcpu_handle_common_requests (vcpu);IMO this is subtly a dangerous hook. It implies that both
vcpu_enter_guest()andvcpu_block()correctly handle requests becoming pending after the "common" check, but that's not actually the case. If a request _needs_ to be handled beforevcpu_block(), then ideally it should be explicitly queried inkvm_vcpu_check_block().KVM_REQ_GET_NESTED_STATE_PAGESdoesn't have issues because it's only ever set from the vCPU itself.
Quoted 9 lines
Following that train of thought,
KVM_REQ_GET_NESTED_STATE_PAGESreally shouldn't even be a request. Aha! And we can do that in a way that would magically fix this bug, and would ensure we don't leave a trap for future us.KVM already provides
KVM_REQ_UNBLOCKto prevent blocking the vCPU without actaully waking the vCPU, i.e. to kick the vCPU back into thevcpu_run()loop. The request is provided specifically for scenarios like this where KVM needs to do work before blocking.
hmm. I think this won't work. The warning is happening at this trace (although the dynamic trace does not show the full stack trace in source code):
WARN_ON_ONCE(!vmx->nested.virtual_apic_map.gfn))vmx_guest_apic_has_interrupt()kvm_guest_apic_has_interrupt()kvm_vcpu_has_events()kvm_arch_vcpu_runnablea()kvm_vcpu_check_block()
If you go to kvm_vcpu_check_block(), the check of KVM_REQ_UNBLOCK is behind check of kvm_arch_vcpu_runnable(). So, with the diff you pointed out, we will still see the warning.
Maybe what we can do is to re-order the kvm_check_request(KVM_REQ_UNBLOCK, vcpu) to the beginning of the kvm_vcpu_check_block()? But I am not sure.
Thanks.
-Mingwei
Quoted 93 lines
Normally I'd say we should do this over multiple patches so that the "blocking" bug is fixed before doing the rework/cleanup, but I'm ok if we want to skip straight to the rework since we're obviously carrying an internal patch and no one else is likely to need the fix. But I also wouldn't object to including an intermediate patch to fix the bug so that there's a better paper trail.
E.g. as a very partial conversion:
--- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/vmx/nested.c | 2 +- arch/x86/kvm/x86.c | 12 ++++++++++++ arch/x86/kvm/x86.h | 10 ++++++++++ 4 files changed, 25 insertions(+), 1 deletion(-)arch/x86/include/asm/kvm_host.h +2
-0@@ -939,6 +939,8 @@ struct kvm_vcpu_arch {*/ bool pdptrs_from_userspace; +bool nested_get_pages_pending; +#if IS_ENABLED (CONFIG_HYPERV )hpa_t hv_root_tdp;#endif arch/x86/kvm/vmx/nested.c +1
-1@@ -3446,7 +3446,7 @@ enum nvmx_vmentry_status nested_vmx_enter_non_root_mode(struct kvm_vcpu *vcpu,* to nested_get_vmcs12_pages before the next VM-entry. The MSRs * have already been set at vmentry time and should not be reset. */ -kvm_make_request (KVM_REQ_GET_NESTED_STATE_PAGES , vcpu); +kvm_nested_get_pages_set_pending (vcpu); }/* arch/x86/kvm/x86.c +12
-0@@ -9650,6 +9650,12 @@ int kvm_check_nested_events(struct kvm_vcpu *vcpu)return kvm_x86_ops.nested_ops->check_events (vcpu); } +static int kvm_get_nested_state_pages (struct kvm_vcpu *vcpu) +{ + vcpu->arch.nested_get_pages_pending = false; +return kvm_x86_ops.nested_ops->get_nested_state_pages (vcpu); +} +static void kvm_inject_exception (struct kvm_vcpu *vcpu) { trace_kvm_inj_exception(vcpu->arch.exception.nr,@@ -10700,6 +10706,12 @@ static int vcpu_run(struct kvm_vcpu *vcpu)if (kvm_cpu_has_pending_timer (vcpu))kvm_inject_pending_timer_irqs (vcpu); +if (vcpu->arch.nested_get_pages_pending) { + r =kvm_get_nested_state_pages (vcpu); +if (r <=0 ) +break ; + } +if (dm_request_for_irq_injection (vcpu) &&kvm_vcpu_ready_for_interrupt_injection (vcpu)) { r =0 ;arch/x86/kvm/x86.h +10
-0@@ -481,4 +481,14 @@ int kvm_sev_es_string_io(struct kvm_vcpu *vcpu, unsigned int size, unsigned int port, void *data, unsigned int count, int in); +static inline void kvm_nested_get_pages_set_pending (struct kvm_vcpu *vcpu) +{ +/* +* Here is a comment explaining why KVM needs to prevent the vCPU from +* blocking until the vCPU's nested pages have been loaded. +*/ + vcpu->arch.nested_get_pages_pending = true; +kvm_make_request (KVM_REQ_UNBLOCK , vcpu); +} +#endif base-commit:
14a47a98151834c5bd2f6d8d592b01108a3f882a--
Re: [PATCH v2 1/4] KVM: x86: move the event handling of KVM_REQ_GET_VMCS12_PAGES into a common function
In reply to Sean ChristophersonQuoted 169 lines · Sean Christopherson
Quoted 5 lines · Mingwei Zhang
Create a common function to handle kvm request in the
vcpu_runloop. KVM implicitly assumes the virtual APIC page being present + mapped into the kernel address space when executingvmx_guest_apic_has_interrupts(). However, with demand paging KVM breaks the assumption, as theKVM_REQ_GET_VMCS12_PAGESevent isn't assessed before enteringvcpu_block.
KVM_REQ_GET_VMCS12_PAGESdoesn't exist upstream.Quoted 10 lines
Fix this by getting vmcs12 pages before inspecting the guest's APIC page. Because of this fix, the event handling code of
KVM_REQ_GET_NESTED_STATE_PAGESbecomes a common code path for bothvcpu_enter_guest()andvcpu_block(). Thus, put this code snippet into a common helper function to avoid code duplication.Cc Maxim Levitsky <[email protected]>Cc Vitaly Kuznetsov <[email protected]>Originally-by Oliver Upton <[email protected]>Signed-off-by Oliver Upton <[email protected]>If you drop someone as author, then their SOB also needs to be jettisoned.
Quoted 23 lines
Signed-off-by Mingwei Zhang <[email protected]>--- arch/x86/kvm/x86.c | 29 +++++++++++++++++++++++------ 1 file changed, 23 insertions(+), 6 deletions(-)arch/x86/kvm/x86.c +0
-6@@ -10261,12 +10261,6 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) r = -EIO ; goto out; } -if (kvm_check_request (KVM_REQ_GET_NESTED_STATE_PAGES , vcpu)) { -if (unlikely (!kvm_x86_ops.nested_ops->get_nested_state_pages (vcpu))) { - r =0 ; - goto out; - } - }if (kvm_check_request (KVM_REQ_MMU_FREE_OBSOLETE_ROOTS , vcpu))kvm_mmu_free_obsolete_roots (vcpu);if (kvm_check_request (KVM_REQ_MIGRATE_TIMER , vcpu))arch/x86/kvm/x86.c static inline bool kvm_vcpu_running(struct kvm_vcpu *vcpu) · line 10665
Quoted 6 lines
arch/x86/kvm/x86.c +3
-0@@ -10666,6 +10660,23 @@ static inline bool kvm_vcpu_running(struct kvm_vcpu *vcpu) !vcpu->arch.apf.halted); } +static int kvm_vcpu_handle_common_requests (struct kvm_vcpu *vcpu) +{ +if (kvm_request_pending (vcpu)) {Probably going to be a moot point, but write this as
if (!kvm_request_pending(vcpu)) return 1;to reduce indentation.
arch/x86/kvm/x86.c static inline bool kvm_vcpu_running(struct kvm_vcpu *vcpu) · line 10672
Quoted 7 lines
arch/x86/kvm/x86.c +7
-0+/* +* Get the vmcs12 pages before checking for interrupts that +* might unblock the guest if L1 is using virtual-interrupt +* delivery. +*/ +if (kvm_check_request (KVM_REQ_GET_NESTED_STATE_PAGES , vcpu)) { +if (unlikely (!kvm_x86_ops.nested_ops->get_nested_state_pages (vcpu)))Similarly
if (kvm_check_request(KVM_REQ_GET_NESTED_STATE_PAGES, vcpu) && unlikely(!kvm_x86_ops.nested_ops->get_nested_state_pages(vcpu))) return 0;though I can see the argument for fully isolating each request. But again, likely a moot point.
Quoted 11 lines
arch/x86/kvm/x86.c +7
-0+return 0 ; + } + } + +return 1 ; +} +/* Called within kvm->srcu read side. */ static int vcpu_run (struct kvm_vcpu *vcpu) {arch/x86/kvm/x86.c static int vcpu_run(struct kvm_vcpu *vcpu) · line 10697
Quoted 6 lines
arch/x86/kvm/x86.c +3
-0@@ -10681,6 +10692,12 @@ static int vcpu_run(struct kvm_vcpu *vcpu)* this point can start executing an instruction. */ vcpu->arch.at_instruction_boundary = false; + +/* Process common request regardless of vcpu state. */ + r =kvm_vcpu_handle_common_requests (vcpu);IMO this is subtly a dangerous hook. It implies that both
vcpu_enter_guest()andvcpu_block()correctly handle requests becoming pending after the "common" check, but that's not actually the case. If a request _needs_ to be handled beforevcpu_block(), then ideally it should be explicitly queried inkvm_vcpu_check_block().KVM_REQ_GET_NESTED_STATE_PAGESdoesn't have issues because it's only ever set from the vCPU itself.Following that train of thought,
KVM_REQ_GET_NESTED_STATE_PAGESreally shouldn't even be a request. Aha! And we can do that in a way that would magically fix this bug, and would ensure we don't leave a trap for future us.KVM already provides
KVM_REQ_UNBLOCKto prevent blocking the vCPU without actaully waking the vCPU, i.e. to kick the vCPU back into thevcpu_run()loop. The request is provided specifically for scenarios like this where KVM needs to do work before blocking.Normally I'd say we should do this over multiple patches so that the "blocking" bug is fixed before doing the rework/cleanup, but I'm ok if we want to skip straight to the rework since we're obviously carrying an internal patch and no one else is likely to need the fix. But I also wouldn't object to including an intermediate patch to fix the bug so that there's a better paper trail.
E.g. as a very partial conversion:
--- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/vmx/nested.c | 2 +- arch/x86/kvm/x86.c | 12 ++++++++++++ arch/x86/kvm/x86.h | 10 ++++++++++ 4 files changed, 25 insertions(+), 1 deletion(-)arch/x86/include/asm/kvm_host.h +2
-0@@ -939,6 +939,8 @@ struct kvm_vcpu_arch {*/ bool pdptrs_from_userspace; +bool nested_get_pages_pending; +#if IS_ENABLED (CONFIG_HYPERV )hpa_t hv_root_tdp;#endif arch/x86/kvm/vmx/nested.c +1
-1@@ -3446,7 +3446,7 @@ enum nvmx_vmentry_status nested_vmx_enter_non_root_mode(struct kvm_vcpu *vcpu,* to nested_get_vmcs12_pages before the next VM-entry. The MSRs * have already been set at vmentry time and should not be reset. */ -kvm_make_request (KVM_REQ_GET_NESTED_STATE_PAGES , vcpu); +kvm_nested_get_pages_set_pending (vcpu); }/* arch/x86/kvm/x86.c +6
-0@@ -9650,6 +9650,12 @@ int kvm_check_nested_events(struct kvm_vcpu *vcpu)return kvm_x86_ops.nested_ops->check_events (vcpu); } +static int kvm_get_nested_state_pages (struct kvm_vcpu *vcpu) +{ + vcpu->arch.nested_get_pages_pending = false; +return kvm_x86_ops.nested_ops->get_nested_state_pages (vcpu); +} +static void kvm_inject_exception (struct kvm_vcpu *vcpu) { trace_kvm_inj_exception(vcpu->arch.exception.nr,
arch/x86/kvm/x86.c static int vcpu_run(struct kvm_vcpu *vcpu) · line 10714
Quoted 9 lines
arch/x86/kvm/x86.c +6
-0@@ -10700,6 +10706,12 @@ static int vcpu_run(struct kvm_vcpu *vcpu)if (kvm_cpu_has_pending_timer (vcpu))kvm_inject_pending_timer_irqs (vcpu); +if (vcpu->arch.nested_get_pages_pending) { + r =kvm_get_nested_state_pages (vcpu); +if (r <=0 ) +break ; + } +
Will this leads to skip the get_nested_state_pages for L2 first time vmentry in every L2 running iteration ? Because with above changes KVM_REQ_GET_NESTED_STATE_PAGES is not set in nested_vmx_enter_non_root_mode() and vcpu->arch.nested_get_pages_pending is not checked in vcpu_enter_guest().
Quoted 25 lines
arch/x86/kvm/x86.c +0
-0if (dm_request_for_irq_injection (vcpu) &&kvm_vcpu_ready_for_interrupt_injection (vcpu)) { r =0 ;arch/x86/kvm/x86.h +10
-0@@ -481,4 +481,14 @@ int kvm_sev_es_string_io(struct kvm_vcpu *vcpu, unsigned int size, unsigned int port, void *data, unsigned int count, int in); +static inline void kvm_nested_get_pages_set_pending (struct kvm_vcpu *vcpu) +{ +/* +* Here is a comment explaining why KVM needs to prevent the vCPU from +* blocking until the vCPU's nested pages have been loaded. +*/ + vcpu->arch.nested_get_pages_pending = true; +kvm_make_request (KVM_REQ_UNBLOCK , vcpu); +} +#endif base-commit:
14a47a98151834c5bd2f6d8d592b01108a3f882a--
Re: [PATCH v2 1/4] KVM: x86: move the event handling of KVM_REQ_GET_VMCS12_PAGES into a common function
Quoted 4 lines
Quoted 4 lines
arch/x86/kvm/x86.c +0
-0@@ -10700,6 +10706,12 @@ static int vcpu_run(struct kvm_vcpu *vcpu)if (kvm_cpu_has_pending_timer (vcpu))kvm_inject_pending_timer_irqs (vcpu);
Quoted 14 lines
arch/x86/kvm/x86.c static int vcpu_run(struct kvm_vcpu *vcpu) · line 10714
Quoted 6 lines
arch/x86/kvm/x86.c +6
-0+if (vcpu->arch.nested_get_pages_pending) { + r =kvm_get_nested_state_pages (vcpu); +if (r <=0 ) +break ; + } +Will this leads to skip the
get_nested_state_pagesfor L2 first time vmentry in every L2 running iteration ? Because with above changesKVM_REQ_GET_NESTED_STATE_PAGESis not set innested_vmx_enter_non_root_mode()andvcpu->arch.nested_get_pages_pendingis not checked invcpu_enter_guest().
Good catch. I think the diff won't work when vcpu is runnable. It only tries to catch the vcpu block case. Even for the vcpu block case, the check of KVM_REQ_UNBLOCK is way too late. Ah, kvm_vcpu_check_block() is called by kvm_vcpu_block() which is called by vcpu_block(). The warning is triggered at the very beginning of vcpu_block(), i.e., within kvm_arch_vcpu_runnable(). So, please ignore the trace in my previous email.
In addition, my minor push back for that is vcpu->arch.nested_get_pages_pending seems to be another KVM_REQ_GET_NESTED_STATE_PAGES.
Thanks.
-Mingwei
-Mingwei
Re: [PATCH v2 1/4] KVM: x86: move the event handling of KVM_REQ_GET_VMCS12_PAGES into a common function
Quoted 26 lines · Mingwei Zhang
Quoted 4 lines
Quoted 4 lines
arch/x86/kvm/x86.c +0
-0@@ -10700,6 +10706,12 @@ static int vcpu_run(struct kvm_vcpu *vcpu)if (kvm_cpu_has_pending_timer (vcpu))kvm_inject_pending_timer_irqs (vcpu);Quoted 14 lines
arch/x86/kvm/x86.c static int vcpu_run(struct kvm_vcpu *vcpu) · line 10714
Quoted 6 lines
arch/x86/kvm/x86.c +6
-0+if (vcpu->arch.nested_get_pages_pending) { + r =kvm_get_nested_state_pages (vcpu); +if (r <=0 ) +break ; + } +Will this leads to skip the
get_nested_state_pagesfor L2 first time vmentry in every L2 running iteration ? Because with above changesKVM_REQ_GET_NESTED_STATE_PAGESis not set innested_vmx_enter_non_root_mode()andvcpu->arch.nested_get_pages_pendingis not checked invcpu_enter_guest().Good catch. I think the diff won't work when vcpu is runnable. It only tries to catch the vcpu block case. Even for the vcpu block case, the check of
KVM_REQ_UNBLOCKis way too late. Ah,kvm_vcpu_check_block()is called bykvm_vcpu_block()which is called byvcpu_block(). The warning is triggered at the very beginning ofvcpu_block(), i.e., withinkvm_arch_vcpu_runnable(). So, please ignore the trace in my previous email.
Quoted 3 lines
In addition, my minor push back for that is
vcpu->arch.nested_get_pages_pendingseems to be anotherKVM_REQ_GET_NESTED_STATE_PAGES.
Yeah, but in concept level it's not a REQ mask lives in the vcpu->requests which can be cached by e.g. kvm_request_pending(). It's necessary to check vcpu->arch.nested_get_pages_pending in vcpu_enter_guest() if Sean's idea is to replace KVM_REQ_GET_NESTED_STATE_PAGES with nested_get_pages_pending.
Quoted 6 lines
Thanks.
(quoted diff) +0
-2-Mingwei -Mingwei
Sean Christopherson
maintainerRe: [PATCH v2 1/4] KVM: x86: move the event handling of KVM_REQ_GET_VMCS12_PAGES into a common function
Quoted 19 lines · Yuan Yao
Quoted 18 lines · Mingwei Zhang
Quoted 18 lines
arch/x86/kvm/x86.c static int vcpu_run(struct kvm_vcpu *vcpu) · line 10714
Quoted 10 lines
arch/x86/kvm/x86.c +6
-0@@ -10700,6 +10706,12 @@ static int vcpu_run(struct kvm_vcpu *vcpu)if (kvm_cpu_has_pending_timer (vcpu))kvm_inject_pending_timer_irqs (vcpu); +if (vcpu->arch.nested_get_pages_pending) { + r =kvm_get_nested_state_pages (vcpu); +if (r <=0 ) +break ; + } +Will this leads to skip the
get_nested_state_pagesfor L2 first time vmentry in every L2 running iteration ? Because with above changesKVM_REQ_GET_NESTED_STATE_PAGESis not set innested_vmx_enter_non_root_mode()andvcpu->arch.nested_get_pages_pendingis not checked invcpu_enter_guest().
Quoted 1 line
Quoted 1 line
Good catch. I think the diff won't work when vcpu is runnable.
It works, but it's inefficient if the request comes from KVM_SET_NESTED_STATE. The pending KVM_REQ_UNBLOCK that comes with the flag will prevent actually running the guest. Specifically, this chunk of code will detect the pending request and bail out of vcpu_enter_guest().
if (kvm_vcpu_exit_request(vcpu)) {
vcpu->mode = OUTSIDE_GUEST_MODE;
smp_wmb();
local_irq_enable();
preempt_enable();
kvm_vcpu_srcu_read_lock(vcpu);
r = 1;
goto cancel_injection;
}But the inefficiency is a non-issue since "true" emulation of VM-Enter will flow through this path (the VMRESUME/VMLAUNCH/VMRUN exit handler runs at the end of vcpu_enter_guest().
Quoted 10 lines
Quoted 9 lines
It only tries to catch the vcpu block case. Even for the vcpu block case, the check of
KVM_REQ_UNBLOCKis way too late. Ah,kvm_vcpu_check_block()is called bykvm_vcpu_block()which is called byvcpu_block(). The warning is triggered at the very beginning ofvcpu_block(), i.e., withinkvm_arch_vcpu_runnable(). So, please ignore the trace in my previous email.In addition, my minor push back for that is
vcpu->arch.nested_get_pages_pendingseems to be anotherKVM_REQ_GET_NESTED_STATE_PAGES.
Quoted 5 lines
Yeah, but in concept level it's not a REQ mask lives in the
vcpu->requestswhich can be cached by e.g.kvm_request_pending(). It's necessary to checkvcpu->arch.nested_get_pages_pendinginvcpu_enter_guest()if Sean's idea is to replaceKVM_REQ_GET_NESTED_STATE_PAGESwithnested_get_pages_pending.
Yes, they key is that it's not a request. Requests have implicit properties:
e.g. as above, effectively prevent running the vCPU until the request goes away, they can be pended from other vCPUs, etc... And the property that is most relevant to this bug: except for special cases, requests only need to be serviced before running vCPU.
And the number of requests is limited due to them being stored in a bitmap. x86 still has plenty of room due to kvm_vcpu.requests being a u64, but it's still preferable to avoid using a request unless absolutely necessary.
For this case, since using a request isn't strictly needed and using a request would require special casing that request, my strong preference is to not use a request.
So yes, my idea is to "just" replace the request with a flag, but there are subtly quite a few impliciations in not using a request.
Sean Christopherson
maintainerRe: [PATCH v2 1/4] KVM: x86: move the event handling of KVM_REQ_GET_VMCS12_PAGES into a common function
Quoted 37 lines · Sean Christopherson
Quoted 19 lines · Yuan Yao
Quoted 18 lines · Mingwei Zhang
Quoted 18 lines
arch/x86/kvm/x86.c static int vcpu_run(struct kvm_vcpu *vcpu) · line 10714
Quoted 10 lines
arch/x86/kvm/x86.c +6
-0@@ -10700,6 +10706,12 @@ static int vcpu_run(struct kvm_vcpu *vcpu)if (kvm_cpu_has_pending_timer (vcpu))kvm_inject_pending_timer_irqs (vcpu); +if (vcpu->arch.nested_get_pages_pending) { + r =kvm_get_nested_state_pages (vcpu); +if (r <=0 ) +break ; + } +Will this leads to skip the
get_nested_state_pagesfor L2 first time vmentry in every L2 running iteration ? Because with above changesKVM_REQ_GET_NESTED_STATE_PAGESis not set innested_vmx_enter_non_root_mode()andvcpu->arch.nested_get_pages_pendingis not checked invcpu_enter_guest().Quoted 1 line
Quoted 1 line
Good catch. I think the diff won't work when vcpu is runnable.
It works, but it's inefficient if the request comes from
KVM_SET_NESTED_STATE. The pendingKVM_REQ_UNBLOCKthat comes with the flag will prevent actually running the guest. Specifically, this chunk of code will detect the pending request and bail out ofvcpu_enter_guest().if (kvm_vcpu_exit_request(vcpu)) { vcpu->mode = OUTSIDE_GUEST_MODE; smp_wmb(); local_irq_enable(); preempt_enable(); kvm_vcpu_srcu_read_lock(vcpu); r = 1; goto cancel_injection; }
Quoted 3 lines
But the inefficiency is a non-issue since "true" emulation of VM-Enter will flow through this path (the VMRESUME/VMLAUNCH/VMRUN exit handler runs at the end of
vcpu_enter_guest().
Actually, nested VM-Enter doesn't use this path at all. The above holds true for emulated RSM, but that's largely a moot point since RSM isn't exactly a hot path.
PATCH v2 2/4 KVM: selftests: Save/restore vAPIC state in migration tests
In reply to Mingwei ZhangSave/restore vAPIC state as part of vCPU save/load so that it is preserved across VM (copyless) migration. This wil allow testing the posted interrupts are properly handled across VM migration.
--- tools/testing/selftests/kvm/include/kvm_util_base.h | 10 ++++++++++ tools/testing/selftests/kvm/include/x86_64/processor.h | 1 + tools/testing/selftests/kvm/lib/x86_64/processor.c | 2 ++ 3 files changed, 13 insertions(+)
tools/testing/selftests/kvm/include/kvm_util_base.h +10 -0
@@ -457,6 +457,16 @@ static inline void vcpu_fpu_set(struct kvm_vcpu *vcpu, struct kvm_fpu *fpu)vcpu_ioctl (vcpu,KVM_SET_FPU , fpu); } +static inline void vcpu_apic_get (struct kvm_vcpu *vcpu,struct kvm_lapic_state *apic) +{ +vcpu_ioctl (vcpu,KVM_GET_LAPIC , apic); +} + +static inline void vcpu_apic_set (struct kvm_vcpu *vcpu,struct kvm_lapic_state *apic) +{ +vcpu_ioctl (vcpu,KVM_SET_LAPIC , apic); +} +static inline int __vcpu_get_reg (struct kvm_vcpu *vcpu,uint64_t id,void *addr) {struct kvm_one_reg reg = { .id = id, .addr = (uint64_t )addr };
tools/testing/selftests/kvm/include/x86_64/processor.h +1 -0
@@ -225,6 +225,7 @@ struct kvm_x86_state {struct kvm_nested_state nested;char nested_[16384 ]; }; +struct kvm_lapic_state apic;struct kvm_msrs msrs; };
tools/testing/selftests/kvm/lib/x86_64/processor.c +2 -0
@@ -980,6 +980,7 @@ struct kvm_x86_state *vcpu_save_state(struct kvm_vcpu *vcpu)vcpu_msrs_get (vcpu, &state->msrs);vcpu_debugregs_get (vcpu, &state->debugregs); +vcpu_apic_get (vcpu, &state->apic);return state; }
@@ -997,6 +998,7 @@ void vcpu_load_state(struct kvm_vcpu *vcpu, struct kvm_x86_state *state)vcpu_mp_state_set (vcpu, &state->mp_state);vcpu_debugregs_set (vcpu, &state->debugregs);vcpu_regs_set (vcpu, &state->regs); +vcpu_apic_set (vcpu, &state->apic);if (state->nested.size)vcpu_nested_state_set (vcpu, &state->nested);
Signature · 1 line
2.37.2.672.g94769d06f0-goog
PATCH v2 3/4 KVM: selftests: Add support for posted interrupt handling in L2
In reply to Mingwei ZhangAdd support for posted interrupt handling in L2. This is done by adding needed data structures in vmx_pages and APIs to allow an L2 receive posted interrupts.
--- tools/testing/selftests/kvm/include/x86_64/vmx.h | 10 ++++++++++ tools/testing/selftests/kvm/lib/x86_64/vmx.c | 14 ++++++++++++++ 2 files changed, 24 insertions(+)
tools/testing/selftests/kvm/include/x86_64/vmx.h +10 -0
@@ -577,6 +577,14 @@ struct vmx_pages {void *apic_access_hva;uint64_t apic_access_gpa;void *apic_access; + +void *virtual_apic_hva; +uint64_t virtual_apic_gpa; +void *virtual_apic; + +void *posted_intr_desc_hva; +uint64_t posted_intr_desc_gpa; +void *posted_intr_desc; };union vmx_basic {
@@ -620,5 +628,7 @@ void nested_identity_map_1g(struct vmx_pages *vmx, struct kvm_vm *vm,void prepare_eptp (struct vmx_pages *vmx,struct kvm_vm *vm,uint32_t eptp_memslot);void prepare_virtualize_apic_accesses (struct vmx_pages *vmx,struct kvm_vm *vm); +void prepare_virtual_apic (struct vmx_pages *vmx,struct kvm_vm *vm); +void prepare_posted_intr_desc (struct vmx_pages *vmx,struct kvm_vm *vm);#endif /* SELFTEST_KVM_VMX_H */
tools/testing/selftests/kvm/lib/x86_64/vmx.c +14 -0
@@ -556,3 +556,17 @@ void prepare_virtualize_apic_accesses(struct vmx_pages *vmx, struct kvm_vm *vm) vmx->apic_access_hva =addr_gva2hva (vm, (uintptr_t )vmx->apic_access); vmx->apic_access_gpa =addr_gva2gpa (vm, (uintptr_t )vmx->apic_access); } + +void prepare_virtual_apic (struct vmx_pages *vmx,struct kvm_vm *vm) +{ + vmx->virtual_apic = (void *)vm_vaddr_alloc_page (vm); + vmx->virtual_apic_hva =addr_gva2hva (vm, (uintptr_t )vmx->virtual_apic); + vmx->virtual_apic_gpa =addr_gva2gpa (vm, (uintptr_t )vmx->virtual_apic); +} + +void prepare_posted_intr_desc (struct vmx_pages *vmx,struct kvm_vm *vm) +{ + vmx->posted_intr_desc = (void *)vm_vaddr_alloc_page (vm); + vmx->posted_intr_desc_hva =addr_gva2hva (vm, vmx->posted_intr_desc); + vmx->posted_intr_desc_gpa =addr_gva2gpa (vm, vmx->posted_intr_desc); +}
Signature · 1 line
2.37.2.672.g94769d06f0-goog
Sean Christopherson
maintainerRe: [PATCH v2 3/4] KVM: selftests: Add support for posted interrupt handling in L2
Quoted 17 lines · Mingwei Zhang
Add support for posted interrupt handling in L2. This is done by adding needed data structures in
vmx_pagesand APIs to allow an L2 receive posted interrupts.Cc Jim Mattson <[email protected]>Signed-off-by Mingwei Zhang <[email protected]>--- tools/testing/selftests/kvm/include/x86_64/vmx.h | 10 ++++++++++ tools/testing/selftests/kvm/lib/x86_64/vmx.c | 14 ++++++++++++++ 2 files changed, 24 insertions(+)tools/testing/selftests/kvm/include/x86_64/vmx.h +0
-0@@ -577,6 +577,14 @@ struct vmx_pages {void *apic_access_hva;
tools/testing/selftests/kvm/include/x86_64/vmx.h struct vmx_pages { · line 587
Quoted 10 lines
tools/testing/selftests/kvm/include/x86_64/vmx.h +8
-0uint64_t apic_access_gpa;void *apic_access; + +void *virtual_apic_hva; +uint64_t virtual_apic_gpa; +void *virtual_apic; + +void *posted_intr_desc_hva; +uint64_t posted_intr_desc_gpa; +void *posted_intr_desc;
Can you add a prep patch to dedup the absurd amount of copy-paste code related to vmx_pages?
I.e. take this and give all the other triplets the same treatment.
--- tools/testing/selftests/kvm/include/x86_64/vmx.h | 10 +++++++--- tools/testing/selftests/kvm/lib/x86_64/vmx.c | 15 ++++++++++----- 2 files changed, 17 insertions(+), 8 deletions(-)
tools/testing/selftests/kvm/include/x86_64/vmx.h +7 -3
@@ -537,10 +537,14 @@ static inline uint32_t vmcs_revision(void)return rdmsr (MSR_IA32_VMX_BASIC ); } +struct vmx_page { +void *gva; +uint64_t gpa; +void *hva; +}; +struct vmx_pages { -void *vmxon_hva; -uint64_t vmxon_gpa; -void *vmxon; +struct vmx_page vmxon;void *vmcs_hva;uint64_t vmcs_gpa;
tools/testing/selftests/kvm/lib/x86_64/vmx.c +10 -5
@@ -58,6 +58,13 @@ int vcpu_enable_evmcs(struct kvm_vcpu *vcpu)return evmcs_ver; } +static void vcpu_alloc_vmx_page (struct kvm_vm *vm,struct vmx_page *page) +{ + page->gva = (void *)vm_vaddr_alloc_page (vm); + page->hva =addr_gva2hva (vm, (uintptr_t )page->gva); + page->gpa =addr_gva2gpa (vm, (uintptr_t )page->gva); +} +/* Allocate memory regions for nested VMX tests. * * Input Args:
@@ -76,9 +83,7 @@ vcpu_alloc_vmx(struct kvm_vm *vm, vm_vaddr_t *p_vmx_gva)struct vmx_pages *vmx =addr_gva2hva (vm, vmx_gva);/* Setup of a region of guest memory for the vmxon region. */ - vmx->vmxon = (void *)vm_vaddr_alloc_page (vm); - vmx->vmxon_hva =addr_gva2hva (vm, (uintptr_t )vmx->vmxon); - vmx->vmxon_gpa =addr_gva2gpa (vm, (uintptr_t )vmx->vmxon); +vcpu_alloc_vmx_page (vm, &vmx->vmxon);/* Setup of a region of guest memory for a vmcs. */ vmx->vmcs = (void *)vm_vaddr_alloc_page (vm);
@@ -160,8 +165,8 @@ bool prepare_for_vmx_operation(struct vmx_pages *vmx)wrmsr (MSR_IA32_FEAT_CTL , feature_control | required);/* Enter VMX root operation. */ - *(uint32_t *)(vmx->vmxon) =vmcs_revision (); -if (vmxon (vmx->vmxon_gpa)) + *(uint32_t *)(vmx->vmxon.gva) =vmcs_revision (); +if (vmxon (vmx->vmxon.gpa))return false;return true;
base-commit: 372d07084593dc7a399bf9bee815711b1fb1bcf2 --
Re: [PATCH v2 3/4] KVM: selftests: Add support for posted interrupt handling in L2
Quoted 32 lines · Sean Christopherson
Quoted 17 lines · Mingwei Zhang
Add support for posted interrupt handling in L2. This is done by adding needed data structures in
vmx_pagesand APIs to allow an L2 receive posted interrupts.Cc Jim Mattson <[email protected]>Signed-off-by Mingwei Zhang <[email protected]>--- tools/testing/selftests/kvm/include/x86_64/vmx.h | 10 ++++++++++ tools/testing/selftests/kvm/lib/x86_64/vmx.c | 14 ++++++++++++++ 2 files changed, 24 insertions(+)tools/testing/selftests/kvm/include/x86_64/vmx.h +0
-0@@ -577,6 +577,14 @@ struct vmx_pages {void *apic_access_hva;tools/testing/selftests/kvm/include/x86_64/vmx.h struct vmx_pages { · line 587
Quoted 10 lines
tools/testing/selftests/kvm/include/x86_64/vmx.h +8
-0uint64_t apic_access_gpa;void *apic_access; + +void *virtual_apic_hva; +uint64_t virtual_apic_gpa; +void *virtual_apic; + +void *posted_intr_desc_hva; +uint64_t posted_intr_desc_gpa; +void *posted_intr_desc;Can you add a prep patch to dedup the absurd amount of copy-paste code related to
vmx_pages?
Quoted 1 line
I.e. take this and give all the other triplets the same treatment.
Will do.
Quoted 72 lines
--- tools/testing/selftests/kvm/include/x86_64/vmx.h | 10 +++++++--- tools/testing/selftests/kvm/lib/x86_64/vmx.c | 15 ++++++++++----- 2 files changed, 17 insertions(+), 8 deletions(-)tools/testing/selftests/kvm/include/x86_64/vmx.h +7
-3@@ -537,10 +537,14 @@ static inline uint32_t vmcs_revision(void)return rdmsr (MSR_IA32_VMX_BASIC ); } +struct vmx_page { +void *gva; +uint64_t gpa; +void *hva; +}; +struct vmx_pages { -void *vmxon_hva; -uint64_t vmxon_gpa; -void *vmxon; +struct vmx_page vmxon;void *vmcs_hva;uint64_t vmcs_gpa;tools/testing/selftests/kvm/lib/x86_64/vmx.c +10
-5@@ -58,6 +58,13 @@ int vcpu_enable_evmcs(struct kvm_vcpu *vcpu)return evmcs_ver; } +static void vcpu_alloc_vmx_page (struct kvm_vm *vm,struct vmx_page *page) +{ + page->gva = (void *)vm_vaddr_alloc_page (vm); + page->hva =addr_gva2hva (vm, (uintptr_t )page->gva); + page->gpa =addr_gva2gpa (vm, (uintptr_t )page->gva); +} +/* Allocate memory regions for nested VMX tests. * * Input Args: @@ -76,9 +83,7 @@ vcpu_alloc_vmx(struct kvm_vm *vm, vm_vaddr_t *p_vmx_gva)struct vmx_pages *vmx =addr_gva2hva (vm, vmx_gva);/* Setup of a region of guest memory for the vmxon region. */ - vmx->vmxon = (void *)vm_vaddr_alloc_page (vm); - vmx->vmxon_hva =addr_gva2hva (vm, (uintptr_t )vmx->vmxon); - vmx->vmxon_gpa =addr_gva2gpa (vm, (uintptr_t )vmx->vmxon); +vcpu_alloc_vmx_page (vm, &vmx->vmxon);/* Setup of a region of guest memory for a vmcs. */ vmx->vmcs = (void *)vm_vaddr_alloc_page (vm);@@ -160,8 +165,8 @@ bool prepare_for_vmx_operation(struct vmx_pages *vmx)wrmsr (MSR_IA32_FEAT_CTL , feature_control | required);/* Enter VMX root operation. */ - *(uint32_t *)(vmx->vmxon) =vmcs_revision (); -if (vmxon (vmx->vmxon_gpa)) + *(uint32_t *)(vmx->vmxon.gva) =vmcs_revision (); +if (vmxon (vmx->vmxon.gpa))return false;return true;base-commit:
372d07084593dc7a399bf9bee815711b1fb1bcf2--
PATCH v2 4/4 KVM: selftests: Test if posted interrupt delivery race with migration
In reply to Mingwei ZhangFrom: Jim Mattson <[email protected]>
Test if posted interrupt delivery race with migration. Add a selftest to demonstrate a race condition between migration and posted interrupt for a nested VM. The consequence of this race condition causes the loss of a posted interrupt for a nested vCPU after migration and triggers a warning for unpatched kernel.
The selftest demonstrates that if a L2 vCPU is in halted state before migration, then after migration, it is not able to receive a posted interrupt from another vCPU within the same VM.
The fundamental problem is deeply buried in the kernel logic where vcpu_block() will directly check vmcs12 related mappings before having a valid vmcs12 ready. Because of that, it fails to process the posted interrupt and triggers the warning in vmx_guest_apic_has_interrupt()
static bool vmx_guest_apic_has_interrupt(struct kvm_vcpu *vcpu)
{
...
if (WARN_ON_ONCE(!is_guest_mode(vcpu)) ||
!nested_cpu_has_vid(get_vmcs12(vcpu)) ||
WARN_ON_ONCE(!vmx->nested.virtual_apic_map.gfn)) <= HERE
return false;
...
}--- tools/testing/selftests/kvm/.gitignore | 1 + tools/testing/selftests/kvm/Makefile | 1 + .../kvm/x86_64/vmx_migrate_pi_pending.c | 291 ++++++++++++++++++ 3 files changed, 293 insertions(+) create mode 100644 tools/testing/selftests/kvm/x86_64/vmx_migrate_pi_pending.c
tools/testing/selftests/kvm/.gitignore +1 -0
@@ -51,6 +51,7 @@ /x86_64/vmx_exception_with_invalid_guest_state /x86_64/vmx_invalid_nested_guest_state /x86_64/vmx_msrs_test +/x86_64/vmx_migrate_pi_pending /x86_64/vmx_preemption_timer_test /x86_64/vmx_set_nested_state_test /x86_64/vmx_tsc_adjust_test
tools/testing/selftests/kvm/Makefile +1 -0
@@ -110,6 +110,7 @@ TEST_GEN_PROGS_x86_64 += x86_64/vmx_dirty_log_test TEST_GEN_PROGS_x86_64 += x86_64/vmx_exception_with_invalid_guest_state TEST_GEN_PROGS_x86_64 += x86_64/vmx_msrs_test TEST_GEN_PROGS_x86_64 += x86_64/vmx_invalid_nested_guest_state +TEST_GEN_PROGS_x86_64 += x86_64/vmx_migrate_pi_pending TEST_GEN_PROGS_x86_64 += x86_64/vmx_set_nested_state_test TEST_GEN_PROGS_x86_64 += x86_64/vmx_tsc_adjust_test TEST_GEN_PROGS_x86_64 += x86_64/vmx_nested_tsc_scaling_test
tools/testing/selftests/kvm/x86_64/vmx_migrate_pi_pending.c new file +291 -0
291 lines not on this page · raw message
Signature · 1 line
2.37.2.672.g94769d06f0-goog
Sean Christopherson
maintainerRe: [PATCH v2 4/4] KVM: selftests: Test if posted interrupt delivery race with migration
Quoted 11 lines · Mingwei Zhang
From: Jim Mattson <[email protected]>
Test if posted interrupt delivery race with migration. Add a selftest to demonstrate a race condition between migration and posted interrupt for a nested VM. The consequence of this race condition causes the loss of a posted interrupt for a nested vCPU after migration and triggers a warning for unpatched kernel.
The selftest demonstrates that if a L2 vCPU is in halted state before migration, then after migration, it is not able to receive a posted interrupt from another vCPU within the same VM.
For tests, try to phrase the changelog in terms of what architectural behavior is being tested, as opposed to stating what exact KVM bug is being targeted. It's definitely helpful to call out the KVM bug, but do that _after_ explaining the test itself. The problem with talking about a specific KVM bug is that
IIUC, this can be:
Add a test to verify that a posted interrupt wakes the target vCPU from a halted state if the VM is migrated while the vCPU is halted.
and then something like this to call out the known KVM bug
This test exposes a bug where KVM checks the vAPIC page before loading nested pages when the vCPU is blocking.
Quoted 14 lines
The fundamental problem is deeply buried in the kernel logic where
vcpu_block()will directly check vmcs12 related mappings before having a valid vmcs12 ready. Because of that, it fails to process the posted interrupt and triggers the warning invmx_guest_apic_has_interrupt()static bool
vmx_guest_apic_has_interrupt(struct kvm_vcpu *vcpu){ ... if (WARN_ON_ONCE(!is_guest_mode(vcpu)) || !nested_cpu_has_vid(get_vmcs12(vcpu)) || WARN_ON_ONCE(!vmx->nested.virtual_apic_map.gfn)) <= HERE return false; ... }
Quoted 1 line
+static void
vcpu0_ipi_handler(struct ex_regs *regs)
This needs to clarify that it's an L2 handler, that info is _extremely_ important to understanding this sequence. And even if that info were superfluous, it's still a good habit to use consistent namespacing.
tools/testing/selftests/kvm/x86_64/vmx_migrate_pi_pending.c line 49
Quoted 3 lines
tools/testing/selftests/kvm/x86_64/vmx_migrate_pi_pending.c new file +3
-0+{ + asmvolatile ("inb %%dx, %%al" + : : [port]"d" (PORT_L0_EXIT ) :"rax" );
Can't this use GUEST_SYNC()?
tools/testing/selftests/kvm/x86_64/vmx_migrate_pi_pending.c line 50
Quoted 1 line
tools/testing/selftests/kvm/x86_64/vmx_migrate_pi_pending.c new file +1
-0+ asmvolatile ("vmcall" );
A comment would be helpful, but not as necessary if this is vcpu0_l2_ipi_handler().
tools/testing/selftests/kvm/x86_64/vmx_migrate_pi_pending.c line 53
Quoted 3 lines
tools/testing/selftests/kvm/x86_64/vmx_migrate_pi_pending.c new file +3
-0+} + +static void l2_vcpu0_guest_code (void )
I have a slight preference for
vcpu0_l2_ipi_handler() vcpu0_l2_guest_code() vcpu1_l1_guest_code()
because the "vcpu0 vs. vcpu1" is the broader scope, and then "l1 vs. l2" further clarifies the exact scope of the function.
tools/testing/selftests/kvm/x86_64/vmx_migrate_pi_pending.c line 56
Quoted 3 lines
tools/testing/selftests/kvm/x86_64/vmx_migrate_pi_pending.c new file +3
-0+{ + asmvolatile ("cli" ); + asmvolatile ("sti; nop; hlt" );
What is this code trying to do? Assuming the intent is to ensure the posted IRQ arrives after "hlt", this needs to be:
GUEST_ASSERT(!irqs_enabled());
asm volatile("sti; hlt");because if interrupts are enabled when l2_vcpu0_guest_code() starts running, then the IRQ can arrive before CLI. And the "nop" needs to go because the nop will consume the STI shadow, i.e. the IRQ can arrive before the "hlt". irqs_enabled() needs to be defined, but it's fairly straightfoward; something like this:
static __always_inline bool irqs_enabled(void)
{
unsigned long flags;
asm volatile("pushf ; pop %0"
: "=rm" (flags)
:
: "memory");
return flags & X86_EFLAGS_IF;
}If my assuming is wrong, then this needs a very verbose comment.
tools/testing/selftests/kvm/x86_64/vmx_migrate_pi_pending.c line 106
Quoted 9 lines
tools/testing/selftests/kvm/x86_64/vmx_migrate_pi_pending.c new file +9
-0+static void post_intr (u8 vector,void *pi_desc) +{ +set_bit (vector, pi_desc); +set_bit (PI_ON_BIT , pi_desc); +} + +static void l1_vcpu1_guest_code (void *vcpu0_pi_desc) +{ +post_intr (L2_INTR , vcpu0_pi_desc);
Open code post_intr() here. I would expect a "post_intr()" helper to actually do the notification. Separating the two things confused me.
Quoted 1 line
+
x2apic_enable();
Why take a dependency on x2APIC? Either way, enable x2APIC, then post the interrupt. Again, it's weird splitting "set info in PI descriptor" from the notification.
Quoted 4 lines
tools/testing/selftests/kvm/x86_64/vmx_migrate_pi_pending.c new file +4
-0+x2apic_write_reg (APIC_ICR , ((u64 )VCPU_ID0 <<32 ) | +APIC_DEST_PHYSICAL |APIC_DM_FIXED |PI_NV ); +GUEST_DONE (); +}
...
Quoted 5 lines
tools/testing/selftests/kvm/x86_64/vmx_migrate_pi_pending.c new file +5
-0+void *create_and_run_vcpu1 (void *arg) +{ +struct ucall uc; +struct kvm_run *run; +struct kvm_mp_state vcpu0_mp_state;
tools/testing/selftests/kvm/x86_64/vmx_migrate_pi_pending.c line 148
Quoted 10 lines
tools/testing/selftests/kvm/x86_64/vmx_migrate_pi_pending.c new file +10
-0+ + pthread_cpu1 =pthread_self (); + +/* Keep trying to kick out vcpu0 until it is in halted state. */ +for (;;) { +WRITE_ONCE (vcpu0_can_run, true); +sleep (1 ); +WRITE_ONCE (vcpu0_can_run, false); +pthread_kill (pthread_cpu0,SIGUSR1 ); +printf ("vcpu1: Sent SIGUSR1 to vcpu0\n" );
Use pr_debug(), this has the potential to spam the console. And then probably use pr_info() for the other printfs so that they can be turned off via QUIET.
tools/testing/selftests/kvm/x86_64/vmx_migrate_pi_pending.c line 151
Quoted 3 lines
tools/testing/selftests/kvm/x86_64/vmx_migrate_pi_pending.c new file +3
-0+ +while (READ_ONCE (vcpu0_running)) + ;
vcpu0_running is unnecessary. KVM needs to acquire vcpu->mutex to do KVM_GET_MP_STATE, i.e. vcpu_mp_state_get() will block until KVM_RUN completes. Nothing guarantees vcpu0_running will be set from main() before this gets to the while-loop, so vcpu_mp_state_get() racing with KVM_RUN is already possible.
tools/testing/selftests/kvm/x86_64/vmx_migrate_pi_pending.c line 163
Quoted 12 lines
tools/testing/selftests/kvm/x86_64/vmx_migrate_pi_pending.c new file +12
-0+ +vcpu_mp_state_get (vcpu0, &vcpu0_mp_state); +if (vcpu0_mp_state.mp_state ==KVM_MP_STATE_HALTED ) +break ; + } + +printf ("vcpu1: Kicked out vcpu0 and ensure vcpu0 is halted\n" ); + +/* Use save_restore_vm() to simulate a VM migration. */ +save_restore_vm (vm); + +printf ("vcpu1: Finished save and restore vm.\n" );
Uber nit, be consistent on whether or not the test uses punctionation.
Quoted 7 lines
tools/testing/selftests/kvm/x86_64/vmx_migrate_pi_pending.c new file +7
-0+ vcpu1 =vm_vcpu_add (vm,VCPU_ID1 , l1_vcpu1_guest_code); +vcpu_args_set (vcpu1,1 , vmx->posted_intr_desc); + +/* Start an L1 in vcpu1 and send a posted interrupt to halted L2 in vcpu0. */ +for (;;) { + run = vcpu1->run; +vcpu_run (vcpu1);
tools/testing/selftests/kvm/x86_64/vmx_migrate_pi_pending.c line 180
Quoted 10 lines
tools/testing/selftests/kvm/x86_64/vmx_migrate_pi_pending.c new file +10
-0+ +TEST_ASSERT (run->exit_reason ==KVM_EXIT_IO , +"vcpu1: Got exit_reason other than KVM_EXIT_IO: %u (%s)\n" , + run->exit_reason, +exit_reason_str (run->exit_reason)); + +switch (get_ucall (vcpu1, &uc)) { +case UCALL_ABORT : +TEST_FAIL ("%s" , (const char *)uc.args[0 ]); +/* NOT REACHED */
REPORT_GUEST_ASSERT(...)
Quoted 5 lines
tools/testing/selftests/kvm/x86_64/vmx_migrate_pi_pending.c new file +5
-0+case UCALL_DONE : +printf ("vcpu1: Successfully send a posted interrupt to vcpu0\n" ); + goto done; +default : +TEST_FAIL ("vcpu1: Unknown ucall %lu" , uc.cmd);
tools/testing/selftests/kvm/x86_64/vmx_migrate_pi_pending.c line 195
Quoted 10 lines
tools/testing/selftests/kvm/x86_64/vmx_migrate_pi_pending.c new file +10
-0+ } + } + +done: +/* +* Allow vcpu0 resume execution from L0 userspace and check if the +* posted interrupt get executed. +*/ +WRITE_ONCE (vcpu0_can_run, true); +sleep (1 );
What guarantees that sleep(1) is sufficient for vCPU to get back into the guest? This might be a good candidate for a sempahore?
Quoted 5 lines
tools/testing/selftests/kvm/x86_64/vmx_migrate_pi_pending.c new file +5
-0+TEST_ASSERT (READ_ONCE (pi_executed), +"vcpu0 did not execute the posted interrupt.\n" ); + +return NULL ; +}
...
tools/testing/selftests/kvm/x86_64/vmx_migrate_pi_pending.c line 226
Quoted 2 lines
tools/testing/selftests/kvm/x86_64/vmx_migrate_pi_pending.c new file +2
-0+TEST_REQUIRE (kvm_has_cap (KVM_CAP_NESTED_STATE )); +TEST_REQUIRE (kvm_cpu_has (X86_FEATURE_VMX ));
This also requires APICv / posted interrupts, and x2APIC if that hardcoded behavior is kept.
tools/testing/selftests/kvm/x86_64/vmx_migrate_pi_pending.c line 254
Quoted 13 lines
tools/testing/selftests/kvm/x86_64/vmx_migrate_pi_pending.c new file +13
-0+for (;;) { +struct kvm_run *run = vcpu0->run; +struct ucall uc; +int rc; + +while (!READ_ONCE (vcpu0_can_run)) + ; + +WRITE_ONCE (vcpu0_running, true); + + rc =__vcpu_run (vcpu0); + + vcpu0->run->immediate_exit =0 ;
Why? vcpu_run_complete_io() is the only thing that sets immediate_exit, and it clears the flag after doing __vcpu_run().
Quoted 7 lines
tools/testing/selftests/kvm/x86_64/vmx_migrate_pi_pending.c new file +7
-0+ +/* +* When vCPU is kicked out by a signal, ensure a consistent vCPU +* state to prepare for migration before setting the +* vcpu_running flag to false. +*/ +if (rc ==-1 && run->exit_reason ==KVM_EXIT_INTR ) {
tools/testing/selftests/kvm/x86_64/vmx_migrate_pi_pending.c line 271
Quoted 10 lines
tools/testing/selftests/kvm/x86_64/vmx_migrate_pi_pending.c new file +10
-0+vcpu_run_complete_io (vcpu0); + +WRITE_ONCE (vcpu0_running, false); + +continue ; + } + +WRITE_ONCE (vcpu0_running, false); + +if (run->io.port ==PORT_L0_EXIT ) {
One of the motivations for using GUEST_SYNC() instead of PORT_L0_EXIT is that this test could (very theoretically) get a false pass if GUEST_DONE() is reached before PORT_L0_EXIT is encountered.
tools/testing/selftests/kvm/x86_64/vmx_migrate_pi_pending.c line 279
Quoted 8 lines
tools/testing/selftests/kvm/x86_64/vmx_migrate_pi_pending.c new file +8
-0+printf ("vcpu0: Executed the posted interrupt\n" ); +WRITE_ONCE (pi_executed, true); +continue ; + } + +switch (get_ucall (vcpu0, &uc)) { +case UCALL_ABORT : +TEST_FAIL ("%s" , (const char *)uc.args[0 ]);
REPORT_GUEST_ASSERT(...)
tools/testing/selftests/kvm/x86_64/vmx_migrate_pi_pending.c line 183
Quoted 3 lines
tools/testing/selftests/kvm/x86_64/vmx_migrate_pi_pending.c new file +3
-0+/* NOT REACHED */ +case UCALL_DONE : + goto done;
Just break?
Quoted 12 lines
tools/testing/selftests/kvm/x86_64/vmx_migrate_pi_pending.c new file +9
-0+default : +TEST_FAIL ("vcpu0: Unknown ucall %lu" , uc.cmd); + } + } + +done: +kvm_vm_free (vm); +return 0 ; +}Signature · 1 line
2.37.2.672.g94769d06f0-goog