KVM: VMX: Clear GUEST_ACTIVITY_STATE when userspace makes a vCPU RUNNABLE
Subsystems (3)
PATCH 1/1 KVM: VMX: Clear GUEST_ACTIVITY_STATE when userspace makes a vCPU RUNNABLE
Force a vCPU out of its hardware-tracked halted state when userspace explicitly declares the vCPU RUNNABLE via KVM_SET_MP_STATE, i.e. clear VMCS.GUEST_ACTIVITY_STATE if it says the vCPU is halted. Add an optional kvm_x86_ops hook to do the clearing, as SVM has no equivalent VMCB field.
When HLT-exiting is disabled for a VM (KVM_CAP_X86_DISABLE_EXITS with KVM_X86_DISABLE_EXITS_HLT, e.g. QEMU's "-overcommit cpu-pm=on"), a guest HLT halts the physical CPU instead of exiting to KVM, and hardware saves GUEST_ACTIVITY_STATE=HLT into the VMCS on the next VM-Exit. That field is
sticky: it survives VM-Exit/VM-Enter and is only cleared by vmx_clear_hlt() on event injection, or by vmx_vcpu_reset() on INIT / vCPU creation.
Nothing clears it on a userspace-driven state change. KVM_SET_REGS only writes the software register cache and KVM_SET_MP_STATE only writes vcpu->arch.mp_state; kvm_vcpu_running() likewise consults software state only. A VMM that emulates a machine reset therefore ends up with a vCPU that KVM happily VM-Enters while hardware refuses to fetch instructions.
Reproduce with a Linux guest by triggering a panic/kdump on a non-boot
vCPU: nmi_shootdown_cpus() parks the other vCPUs -- including vCPU0 -- in crash_nmi_callback(), which does local_irq_disable() followed by a bare HLT. The capture kernel then resets the machine via port 0xCF9. QEMU rewrites RIP to 0xfff0 and sets mp_state to RUNNABLE, but vCPU0's GUEST_ACTIVITY_STATE is still HLT, so the BSP never executes the reset vector, never sends SIPIs, and the entire VM hangs at "reboot: machine restart" forever. Only destroying and recreating the VM recovers it.
Clearing the state is always safe: waking from HLT is architecturally permitted to be spurious, and every HLT in the kernel is inside a loop. Hook KVM_SET_MP_STATE rather than the VM-Enter path so that the clearing is driven by an explicit userspace declaration, and so that no work is added to vmx_vcpu_run().
Note, vmx_clear_hlt() loses its "static" qualifier as the kvm_x86_ops table now lives in vmx/main.c. TDX cannot disable HLT-exiting and KVM cannot access a TD's VMCS, so vt_clear_hlt() short-circuits for TD vCPUs, following the existing vt_*() wrapper pattern.
--- arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 7 +++++++ arch/x86/kvm/vmx/main.c | 13 +++++++++++++ arch/x86/kvm/vmx/vmx.c | 2 +- arch/x86/kvm/vmx/x86_ops.h | 1 + arch/x86/kvm/x86.c | 13 +++++++++++++ 6 files changed, 36 insertions(+), 1 deletion(-)
arch/x86/include/asm/kvm-x86-ops.h +1 -0
@@ -82,6 +82,7 @@ KVM_X86_OP(interrupt_allowed) KVM_X86_OP(nmi_allowed) KVM_X86_OP(get_nmi_mask) KVM_X86_OP(set_nmi_mask) +KVM_X86_OP_OPTIONAL(clear_hlt) KVM_X86_OP(enable_nmi_window) KVM_X86_OP(enable_irq_window) KVM_X86_OP_OPTIONAL(update_cr8_intercept)
arch/x86/include/asm/kvm_host.h +7 -0
@@ -1607,6 +1607,13 @@ struct kvm_x86_ops {int (*nmi_allowed)(struct kvm_vcpu *vcpu,bool for_injection);bool (*get_nmi_mask)(struct kvm_vcpu *vcpu);void (*set_nmi_mask)(struct kvm_vcpu *vcpu,bool masked); +/* +* Force the vCPU out of any hardware-tracked halted/inactive state so +* that it will fetch and execute instructions on the next VM-Enter. +* Only needed by VMX, where VMCS.GUEST_ACTIVITY_STATE persists across +* VM-Exit/VM-Enter; SVM has no equivalent VMCB field. +*/ +void (*clear_hlt)(struct kvm_vcpu *vcpu);/* Whether or not a virtual NMI is pending in hardware. */ bool (*is_vnmi_pending)(struct kvm_vcpu *vcpu);/*
arch/x86/kvm/vmx/main.c +13 -0
@@ -601,6 +601,18 @@ static void vt_set_nmi_mask(struct kvm_vcpu *vcpu, bool masked)vmx_set_nmi_mask (vcpu, masked); } +static void vt_clear_hlt (struct kvm_vcpu *vcpu) +{ +/* +* TDX doesn't support disabling HLT-exiting, and KVM can't access a +* TD's VMCS, so there is never any hardware halted state to clear. +*/ +if (is_td_vcpu (vcpu)) +return ; + +vmx_clear_hlt (vcpu); +} +static void vt_enable_nmi_window (struct kvm_vcpu *vcpu) {/* Refer to the comments in tdx_inject_nmi(). */
@@ -964,6 +976,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .nmi_allowed = vt_op(nmi_allowed), .get_nmi_mask = vt_op(get_nmi_mask), .set_nmi_mask = vt_op(set_nmi_mask), + .clear_hlt = vt_op(clear_hlt), .enable_nmi_window = vt_op(enable_nmi_window), .enable_irq_window = vt_op(enable_irq_window), .update_cr8_intercept = vt_op(update_cr8_intercept),
arch/x86/kvm/vmx/vmx.c +1 -1
@@ -1909,7 +1909,7 @@ int vmx_skip_emulated_instruction(struct kvm_vcpu *vcpu)return skip_emulated_instruction (vcpu); } -static void vmx_clear_hlt (struct kvm_vcpu *vcpu) +void vmx_clear_hlt (struct kvm_vcpu *vcpu) {/* * Ensure that we clear the HLT state in the VMCS. We don't need to
arch/x86/kvm/vmx/x86_ops.h +1 -0
@@ -95,6 +95,7 @@ int vmx_interrupt_allowed(struct kvm_vcpu *vcpu, bool for_injection);int vmx_nmi_allowed (struct kvm_vcpu *vcpu,bool for_injection);bool vmx_get_nmi_mask (struct kvm_vcpu *vcpu);void vmx_set_nmi_mask (struct kvm_vcpu *vcpu,bool masked); +void vmx_clear_hlt (struct kvm_vcpu *vcpu);void vmx_enable_nmi_window (struct kvm_vcpu *vcpu);void vmx_enable_irq_window (struct kvm_vcpu *vcpu);void vmx_update_cr8_intercept (struct kvm_vcpu *vcpu,int tpr,int irr);
arch/x86/kvm/x86.c +13 -0
@@ -9058,6 +9058,19 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu, }kvm_set_mp_state (vcpu, mp_state->mp_state); + +/* +* Force the vCPU out of any hardware-tracked inactive state, e.g. VMX's +* GUEST_ACTIVITY_STATE=HLT. That state is sticky across VM-Exit and +* VM-Enter and is not touched by any other ioctl, so a vCPU that halted +* with HLT-exiting disabled (KVM_X86_DISABLE_EXITS_HLT) stays wedged +* even after userspace declares it RUNNABLE and rewrites its registers, +* e.g. when a VMM emulates a machine reset. Waking from HLT is +* architecturally allowed to be spurious, so clearing it is always safe. +*/ +if (vcpu->arch.mp_state ==KVM_MP_STATE_RUNNABLE ) +kvm_x86_call (clear_hlt)(vcpu); +kvm_make_request (KVM_REQ_EVENT , vcpu); ret =0 ;
base-commit: 1b731e5ded480bd1e5546aed35584238661ce72e
Signature · 1 line
2.24.3