KVM: VMX: Clear GUEST_ACTIVITY_STATE when userspace makes a vCPU RUNNABLE
Subsystems (4)
PATCH 1/1 KVM: VMX: Clear GUEST_ACTIVITY_STATE when userspace makes a vCPU RUNNABLE
Force a vCPU out of its hardware-tracked halted state when userspace explicitly declares the vCPU RUNNABLE via KVM_SET_MP_STATE, i.e. clear VMCS.GUEST_ACTIVITY_STATE if it says the vCPU is halted. Add an optional kvm_x86_ops hook to do the clearing, as SVM has no equivalent VMCB field.
When HLT-exiting is disabled for a VM (KVM_CAP_X86_DISABLE_EXITS with KVM_X86_DISABLE_EXITS_HLT, e.g. QEMU's "-overcommit cpu-pm=on"), a guest HLT halts the physical CPU instead of exiting to KVM, and hardware saves GUEST_ACTIVITY_STATE=HLT into the VMCS on the next VM-Exit. That field is
sticky: it survives VM-Exit/VM-Enter and is only cleared by vmx_clear_hlt() on event injection, or by vmx_vcpu_reset() on INIT / vCPU creation.
Nothing clears it on a userspace-driven state change. KVM_SET_REGS only writes the software register cache and KVM_SET_MP_STATE only writes vcpu->arch.mp_state; kvm_vcpu_running() likewise consults software state only. A VMM that emulates a machine reset therefore ends up with a vCPU that KVM happily VM-Enters while hardware refuses to fetch instructions.
Reproduce with a Linux guest by triggering a panic/kdump on a non-boot
vCPU: nmi_shootdown_cpus() parks the other vCPUs -- including vCPU0 -- in crash_nmi_callback(), which does local_irq_disable() followed by a bare HLT. The capture kernel then resets the machine via port 0xCF9. QEMU rewrites RIP to 0xfff0 and sets mp_state to RUNNABLE, but vCPU0's GUEST_ACTIVITY_STATE is still HLT, so the BSP never executes the reset vector, never sends SIPIs, and the entire VM hangs at "reboot: machine restart" forever. Only destroying and recreating the VM recovers it.
Clearing the state is always safe: waking from HLT is architecturally permitted to be spurious, and every HLT in the kernel is inside a loop. Hook KVM_SET_MP_STATE rather than the VM-Enter path so that the clearing is driven by an explicit userspace declaration, and so that no work is added to vmx_vcpu_run().
Note, vmx_clear_hlt() loses its "static" qualifier as the kvm_x86_ops table now lives in vmx/main.c. TDX cannot disable HLT-exiting and KVM cannot access a TD's VMCS, so vt_clear_hlt() short-circuits for TD vCPUs, following the existing vt_*() wrapper pattern.
--- arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 7 +++++++ arch/x86/kvm/vmx/main.c | 13 +++++++++++++ arch/x86/kvm/vmx/vmx.c | 2 +- arch/x86/kvm/vmx/x86_ops.h | 1 + arch/x86/kvm/x86.c | 13 +++++++++++++ 6 files changed, 36 insertions(+), 1 deletion(-)
arch/x86/include/asm/kvm-x86-ops.h +1 -0
@@ -82,6 +82,7 @@ KVM_X86_OP(interrupt_allowed) KVM_X86_OP(nmi_allowed) KVM_X86_OP(get_nmi_mask) KVM_X86_OP(set_nmi_mask) +KVM_X86_OP_OPTIONAL(clear_hlt) KVM_X86_OP(enable_nmi_window) KVM_X86_OP(enable_irq_window) KVM_X86_OP_OPTIONAL(update_cr8_intercept)
arch/x86/include/asm/kvm_host.h +7 -0
@@ -1607,6 +1607,13 @@ struct kvm_x86_ops {int (*nmi_allowed)(struct kvm_vcpu *vcpu,bool for_injection);bool (*get_nmi_mask)(struct kvm_vcpu *vcpu);void (*set_nmi_mask)(struct kvm_vcpu *vcpu,bool masked); +/* +* Force the vCPU out of any hardware-tracked halted/inactive state so +* that it will fetch and execute instructions on the next VM-Enter. +* Only needed by VMX, where VMCS.GUEST_ACTIVITY_STATE persists across +* VM-Exit/VM-Enter; SVM has no equivalent VMCB field. +*/ +void (*clear_hlt)(struct kvm_vcpu *vcpu);/* Whether or not a virtual NMI is pending in hardware. */ bool (*is_vnmi_pending)(struct kvm_vcpu *vcpu);/*
arch/x86/kvm/vmx/main.c +13 -0
@@ -601,6 +601,18 @@ static void vt_set_nmi_mask(struct kvm_vcpu *vcpu, bool masked)vmx_set_nmi_mask (vcpu, masked); } +static void vt_clear_hlt (struct kvm_vcpu *vcpu) +{ +/* +* TDX doesn't support disabling HLT-exiting, and KVM can't access a +* TD's VMCS, so there is never any hardware halted state to clear. +*/ +if (is_td_vcpu (vcpu)) +return ; + +vmx_clear_hlt (vcpu); +} +static void vt_enable_nmi_window (struct kvm_vcpu *vcpu) {/* Refer to the comments in tdx_inject_nmi(). */
@@ -964,6 +976,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .nmi_allowed = vt_op(nmi_allowed), .get_nmi_mask = vt_op(get_nmi_mask), .set_nmi_mask = vt_op(set_nmi_mask), + .clear_hlt = vt_op(clear_hlt), .enable_nmi_window = vt_op(enable_nmi_window), .enable_irq_window = vt_op(enable_irq_window), .update_cr8_intercept = vt_op(update_cr8_intercept),
arch/x86/kvm/vmx/vmx.c +1 -1
@@ -1909,7 +1909,7 @@ int vmx_skip_emulated_instruction(struct kvm_vcpu *vcpu)return skip_emulated_instruction (vcpu); } -static void vmx_clear_hlt (struct kvm_vcpu *vcpu) +void vmx_clear_hlt (struct kvm_vcpu *vcpu) {/* * Ensure that we clear the HLT state in the VMCS. We don't need to
arch/x86/kvm/vmx/x86_ops.h +1 -0
@@ -95,6 +95,7 @@ int vmx_interrupt_allowed(struct kvm_vcpu *vcpu, bool for_injection);int vmx_nmi_allowed (struct kvm_vcpu *vcpu,bool for_injection);bool vmx_get_nmi_mask (struct kvm_vcpu *vcpu);void vmx_set_nmi_mask (struct kvm_vcpu *vcpu,bool masked); +void vmx_clear_hlt (struct kvm_vcpu *vcpu);void vmx_enable_nmi_window (struct kvm_vcpu *vcpu);void vmx_enable_irq_window (struct kvm_vcpu *vcpu);void vmx_update_cr8_intercept (struct kvm_vcpu *vcpu,int tpr,int irr);
arch/x86/kvm/x86.c +13 -0
@@ -9058,6 +9058,19 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu, }kvm_set_mp_state (vcpu, mp_state->mp_state); + +/* +* Force the vCPU out of any hardware-tracked inactive state, e.g. VMX's +* GUEST_ACTIVITY_STATE=HLT. That state is sticky across VM-Exit and +* VM-Enter and is not touched by any other ioctl, so a vCPU that halted +* with HLT-exiting disabled (KVM_X86_DISABLE_EXITS_HLT) stays wedged +* even after userspace declares it RUNNABLE and rewrites its registers, +* e.g. when a VMM emulates a machine reset. Waking from HLT is +* architecturally allowed to be spurious, so clearing it is always safe. +*/ +if (vcpu->arch.mp_state ==KVM_MP_STATE_RUNNABLE ) +kvm_x86_call (clear_hlt)(vcpu); +kvm_make_request (KVM_REQ_EVENT , vcpu); ret =0 ;
base-commit: 1b731e5ded480bd1e5546aed35584238661ce72e
Signature · 1 line
2.24.3
Quoted 33 lines · Keqiang Duan
Force a vCPU out of its hardware-tracked halted state when userspace explicitly declares the vCPU RUNNABLE via
KVM_SET_MP_STATE, i.e. clear VMCS.GUEST_ACTIVITY_STATE if it says the vCPU is halted. Add an optionalkvm_x86_opshook to do the clearing, as SVM has no equivalent VMCB field.When HLT-exiting is disabled for a VM (
KVM_CAP_X86_DISABLE_EXITSwithKVM_X86_DISABLE_EXITS_HLT, e.g. QEMU's "-overcommit cpu-pm=on"), a guest HLT halts the physical CPU instead of exiting to KVM, and hardware saves GUEST_ACTIVITY_STATE=HLT into the VMCS on the next VM-Exit. That field is
sticky: it survives VM-Exit/VM-Enter and is only cleared byvmx_clear_hlt()on event injection, or byvmx_vcpu_reset()on INIT / vCPU creation.Nothing clears it on a userspace-driven state change.
KVM_SET_REGSonly writes the software register cache andKVM_SET_MP_STATEonly writesvcpu->arch.mp_state;kvm_vcpu_running()likewise consults software state only. A VMM that emulates a machine reset therefore ends up with a vCPU that KVM happily VM-Enters while hardware refuses to fetch instructions.Reproduce with a Linux guest by triggering a panic/kdump on a non-boot
vCPU:nmi_shootdown_cpus()parks the other vCPUs -- including vCPU0 -- incrash_nmi_callback(), which doeslocal_irq_disable()followed by a bare HLT. The capture kernel then resets the machine via port0xCF9. QEMU rewrites RIP to0xfff0and setsmp_stateto RUNNABLE, but vCPU0'sGUEST_ACTIVITY_STATEis still HLT, so the BSP never executes the reset vector, never sends SIPIs, and the entire VM hangs at "reboot: machine restart" forever. Only destroying and recreating the VM recovers it.Clearing the state is always safe: waking from HLT is architecturally permitted to be spurious, and every HLT in the kernel is inside a loop. Hook
KVM_SET_MP_STATErather than the VM-Enter path so that the clearing is driven by an explicit userspace declaration, and so that no work is added tovmx_vcpu_run().
Quoted 8 lines
Note,
vmx_clear_hlt()loses its "static" qualifier as thekvm_x86_opstable now lives invmx/main.c. TDX cannot disable HLT-exiting and KVM cannot access a TD's VMCS, sovt_clear_hlt()short-circuits for TD vCPUs, following the existing vt_*() wrapper pattern.Fixes caa057a2cad6 ("KVM: X86: Provide a capability to disable HLT intercepts")Signed-off-by Keqiang Duan <[email protected]>
There was an earlier attempt to fix this issue:
https://lore.kernel.org/kvm/[email protected]
Sean suggested doing exactly this there, i.e. clearing the activity state in kvm_arch_vcpu_ioctl_set_mpstate() via a new kvm_x86_ops hook:
Sean Christopherson
maintainerQuoted 43 lines · Chao Gao
Quoted 33 lines · Keqiang Duan
Force a vCPU out of its hardware-tracked halted state when userspace explicitly declares the vCPU RUNNABLE via
KVM_SET_MP_STATE, i.e. clear VMCS.GUEST_ACTIVITY_STATE if it says the vCPU is halted. Add an optionalkvm_x86_opshook to do the clearing, as SVM has no equivalent VMCB field.When HLT-exiting is disabled for a VM (
KVM_CAP_X86_DISABLE_EXITSwithKVM_X86_DISABLE_EXITS_HLT, e.g. QEMU's "-overcommit cpu-pm=on"), a guest HLT halts the physical CPU instead of exiting to KVM, and hardware saves GUEST_ACTIVITY_STATE=HLT into the VMCS on the next VM-Exit. That field is
sticky: it survives VM-Exit/VM-Enter and is only cleared byvmx_clear_hlt()on event injection, or byvmx_vcpu_reset()on INIT / vCPU creation.Nothing clears it on a userspace-driven state change.
KVM_SET_REGSonly writes the software register cache andKVM_SET_MP_STATEonly writesvcpu->arch.mp_state;kvm_vcpu_running()likewise consults software state only. A VMM that emulates a machine reset therefore ends up with a vCPU that KVM happily VM-Enters while hardware refuses to fetch instructions.Reproduce with a Linux guest by triggering a panic/kdump on a non-boot
vCPU:nmi_shootdown_cpus()parks the other vCPUs -- including vCPU0 -- incrash_nmi_callback(), which doeslocal_irq_disable()followed by a bare HLT. The capture kernel then resets the machine via port0xCF9. QEMU rewrites RIP to0xfff0and setsmp_stateto RUNNABLE, but vCPU0'sGUEST_ACTIVITY_STATEis still HLT, so the BSP never executes the reset vector, never sends SIPIs, and the entire VM hangs at "reboot: machine restart" forever. Only destroying and recreating the VM recovers it.Clearing the state is always safe: waking from HLT is architecturally permitted to be spurious, and every HLT in the kernel is inside a loop. Hook
KVM_SET_MP_STATErather than the VM-Enter path so that the clearing is driven by an explicit userspace declaration, and so that no work is added tovmx_vcpu_run().Quoted 8 lines
Note,
vmx_clear_hlt()loses its "static" qualifier as thekvm_x86_opstable now lives invmx/main.c. TDX cannot disable HLT-exiting and KVM cannot access a TD's VMCS, sovt_clear_hlt()short-circuits for TD vCPUs, following the existing vt_*() wrapper pattern.Fixes caa057a2cad6 ("KVM: X86: Provide a capability to disable HLT intercepts")Signed-off-by Keqiang Duan <[email protected]>
Quoted 8 lines
There was an earlier attempt to fix this issue:
https://lore.kernel.org/kvm/[email protected]
Sean suggested doing exactly this there, i.e. clearing the activity state in
kvm_arch_vcpu_ioctl_set_mpstate()via a newkvm_x86_opshook:
Thanks Chao, I knew this sounded familiar!
As suggested in the above link, the call into vendor code should be gated on kvm_hlt_in_guest(). Yes, vmx_clear_hlt() already does that, but it's a cheap check and makes the common x86 code easier to understand.
The other difference between my suggestion and this patch is clearing HLT if and only if the vCPU is being made RUNNABLE, versus clearing HLT if the vCPU is being put into any state other than HALTED. In practice it probably doesn't matter?
AP_RESET_HOLD is unsupported for VMX, SIPI_RECEIVED is deprecated and is now just a variant off INIT_RECEIVED, and getting out of INIT_RECIEVED and UNINITIALIZED requires stuffing RUNNABLE or going through vmx_vcpu_reset(), which explicitly sets the activity state to GUEST_ACTIVITY_ACTIVE.
But I think my vote would still be to do:
mp_state->mp_state != KVM_MP_STATE_HALTED
because I can't think of any reason not to?
Side topic, I think we should officially take an erratum in Documentation/virt/kvm/x86/errata.rst to state that KVM doesn't support save/restore of HLT state when HLT-exiting is disabled. AFAIK, there's simply no sane solution for SVM, and that would justify why the kvm_x86_ops would be clear_hlt() as opposed to set_mp_state(), i.e. why KVM only handles taking vCPUs out of HLT, not putting them into HLT.
Quoted 4 lines · Sean Christopherson
Thanks Chao, I knew this sounded familiar!
As suggested in the above link, the call into vendor code should be gated onkvm_hlt_in_guest(). Yes,vmx_clear_hlt()already does that, but it's a cheap check and makes the common x86 code easier to understand.
Done in v2.
Quoted 9 lines
The other difference between my suggestion and this patch is clearing HLT if and only if the vCPU is being made RUNNABLE, versus clearing HLT if the vCPU is being put into any state other than HALTED. In practice it probably doesn't matter?
AP_RESET_HOLDis unsupported for VMX,SIPI_RECEIVEDis deprecated and is now just a variant offINIT_RECEIVED, and getting out ofINIT_RECIEVEDand UNINITIALIZED requires stuffing RUNNABLE or going throughvmx_vcpu_reset(), which explicitly sets the activity state toGUEST_ACTIVITY_ACTIVE.
Quoted 5 lines
But I think my vote would still be to do:
mp_state->mp_state != KVM_MP_STATE_HALTEDbecause I can't think of any reason not to?
Agreed, and I can't think of one either. HALTED is the only target state for which retaining the hardware halted state is meaningful, and as you note, every other transition either can't be reached on VMX or goes through vmx_vcpu_reset(), which stuffs GUEST_ACTIVITY_ACTIVE anyways.
Keying off "not HALTED" also states the intent more directly, i.e. that KVM only ever drops the hardware halted state and never installs it.
Switched to "mp_state->mp_state != KVM_MP_STATE_HALTED" in v2.
Quoted 6 lines
Side topic, I think we should officially take an erratum in
Documentation/virt/kvm/x86/errata.rstto state that KVM doesn't support save/restore of HLT state when HLT-exiting is disabled. AFAIK, there's simply no sane solution for SVM, and that would justify why thekvm_x86_opswould beclear_hlt()as opposed toset_mp_state(), i.e. why KVM only handles taking vCPUs out of HLT, not putting them into HLT.
Agreed, documenting the limitation makes sense, and it does explain why the hook only takes vCPUs out of HLT.
Thanks for the quick review!