KVM: SVM: Zero DEBUGCTL before VMRUN if necessary
Sean Christopherson
maintainercover KVM: SVM: Zero DEBUGCTL before VMRUN if necessary
PeterZ,
Can you confirm that the last patch (snapshot and restore DEBUGCTL with IRQs disabled) is actually necessary? I'm 99% certain it is, but I'm holding out hope that it somehow isn't, because I don't love the idea of adding a RDMSR to every VM-Entry.
Assuming DEBUGCTL can indeed get modified in IRQ context, it probably makes sense to add a per-CPU cache to eliminate the RDMSR. Unfortunately, there are quite a few open-coded WRMSRs, so it's not a trivial change.
On to the main event...
Fix a long-lurking bug in SVM where KVM runs the guest with the host's DEBUGCTL if LBR virtualization is disabled. AMD CPUs rather stupidly context switch DEBUGCTL if and only if LBR virtualization is enabled (not just supported, but fully enabled).
The bug has gone unnoticed because until recently, the only bits that KVM would leave set were things like BTF, which are guest visible but won't cause functional problems unless guest software is being especially particular about #DBs.
The bug was exposed by the addition of BusLockTrap ("Detect" in the kernel), as the resulting #DBs due to split-lock accesses in guest userspace (lol Steam) get reflected into the guest by KVM.
Sean Christopherson (3):
KVM: x86: Snapshot the host's DEBUGCTL in common x86
KVM: SVM: Manually zero/restore DEBUGCTL if LBR virtualization is
disabled
KVM: x86: Snapshot the host's DEBUGCTL after disabling IRQs
arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/svm/svm.c | 14 ++++++++++++++ arch/x86/kvm/vmx/vmx.c | 8 ++------ arch/x86/kvm/vmx/vmx.h | 2 -- arch/x86/kvm/x86.c | 2 ++ 5 files changed, 19 insertions(+), 8 deletions(-)
base-commit: fed48e2967f402f561d80075a20c5c9e16866e53
Signature · 1 line
2.48.1.658.g4767266eb4-goog
Sean Christopherson
maintainerPATCH 1/3 KVM: x86: Snapshot the host's DEBUGCTL in common x86
Move KVM's snapshot of DEBUGCTL to kvm_vcpu_arch and take the snapshot in common x86, so that SVM can also use the snapshot.
Opportunistically change the field to a u64. While bits 63:32 are reserved on AMD, not mentioned at all in Intel's SDM, and managed as an "unsigned long" by the kernel, DEBUGCTL is an MSR and therefore a 64-bit value.
--- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/vmx/vmx.c | 8 ++------ arch/x86/kvm/vmx/vmx.h | 2 -- arch/x86/kvm/x86.c | 1 + 4 files changed, 4 insertions(+), 8 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -0
@@ -781,6 +781,7 @@ struct kvm_vcpu_arch {u32 pkru;u32 hflags;u64 efer; +u64 host_debugctl;u64 apic_base;struct kvm_lapic *apic;/* kernel irqchip context */ bool load_eoi_exitmap_pending;
arch/x86/kvm/vmx/vmx.c +2 -6
@@ -1514,16 +1514,12 @@ void vmx_vcpu_load_vmcs(struct kvm_vcpu *vcpu, int cpu,*/ void vmx_vcpu_load (struct kvm_vcpu *vcpu,int cpu) { -struct vcpu_vmx *vmx =to_vmx (vcpu); -if (vcpu->scheduled_out && !kvm_pause_in_guest (vcpu->kvm))shrink_ple_window (vcpu);vmx_vcpu_load_vmcs (vcpu, cpu,NULL );vmx_vcpu_pi_load (vcpu, cpu); - - vmx->host_debugctlmsr =get_debugctlmsr (); }void vmx_vcpu_put (struct kvm_vcpu *vcpu)
@@ -7458,8 +7454,8 @@ fastpath_t vmx_vcpu_run(struct kvm_vcpu *vcpu, bool force_immediate_exit) }/* MSR_IA32_DEBUGCTLMSR is zeroed on vmexit. Restore it if needed */ -if (vmx->host_debugctlmsr) -update_debugctlmsr (vmx->host_debugctlmsr); +if (vcpu->arch.host_debugctl) +update_debugctlmsr (vcpu->arch.host_debugctl);#ifndef CONFIG_X86_64 /*
arch/x86/kvm/vmx/vmx.h +0 -2
@@ -340,8 +340,6 @@ struct vcpu_vmx {/* apic deadline value in host tsc */ u64 hv_deadline_tsc; -unsigned long host_debugctlmsr; -/* * Only bits masked by msr_ia32_feature_control_valid_bits can be set in * msr_ia32_feature_control. FEAT_CTL_LOCKED is always included
arch/x86/kvm/x86.c +1 -0
@@ -4991,6 +4991,7 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)/* Save host pkru register if supported */ vcpu->arch.host_pkru =read_pkru (); + vcpu->arch.host_debugctl =get_debugctlmsr ();/* Apply any externally detected TSC adjustments (due to suspend) */ if (unlikely (vcpu->arch.tsc_offset_adjustment)) {
Signature · 1 line
2.48.1.658.g4767266eb4-goog
Re: [PATCH 1/3] KVM: x86: Snapshot the host's DEBUGCTL in common x86
Quoted 9 lines · Sean Christopherson
Move KVM's snapshot of DEBUGCTL to
kvm_vcpu_archand take the snapshot in common x86, so that SVM can also use the snapshot.Opportunistically change the field to a u64. While bits 63:32 are reserved on AMD, not mentioned at all in Intel's SDM, and managed as an "unsigned long" by the kernel, DEBUGCTL is an MSR and therefore a 64-bit value.
Signed-off-by Sean Christopherson <[email protected]>
Quoted 76 lines
--- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/vmx/vmx.c | 8 ++------ arch/x86/kvm/vmx/vmx.h | 2 -- arch/x86/kvm/x86.c | 1 + 4 files changed, 4 insertions(+), 8 deletions(-)arch/x86/include/asm/kvm_host.h +1
-0@@ -781,6 +781,7 @@ struct kvm_vcpu_arch {u32 pkru;u32 hflags;u64 efer; +u64 host_debugctl;u64 apic_base;struct kvm_lapic *apic;/* kernel irqchip context */ bool load_eoi_exitmap_pending;arch/x86/kvm/vmx/vmx.c +2
-6@@ -1514,16 +1514,12 @@ void vmx_vcpu_load_vmcs(struct kvm_vcpu *vcpu, int cpu,*/ void vmx_vcpu_load (struct kvm_vcpu *vcpu,int cpu) { -struct vcpu_vmx *vmx =to_vmx (vcpu); -if (vcpu->scheduled_out && !kvm_pause_in_guest (vcpu->kvm))shrink_ple_window (vcpu);vmx_vcpu_load_vmcs (vcpu, cpu,NULL );vmx_vcpu_pi_load (vcpu, cpu); - - vmx->host_debugctlmsr =get_debugctlmsr (); }void vmx_vcpu_put (struct kvm_vcpu *vcpu)@@ -7458,8 +7454,8 @@ fastpath_t vmx_vcpu_run(struct kvm_vcpu *vcpu, bool force_immediate_exit) }/* MSR_IA32_DEBUGCTLMSR is zeroed on vmexit. Restore it if needed */ -if (vmx->host_debugctlmsr) -update_debugctlmsr (vmx->host_debugctlmsr); +if (vcpu->arch.host_debugctl) +update_debugctlmsr (vcpu->arch.host_debugctl);#ifndef CONFIG_X86_64 /* arch/x86/kvm/vmx/vmx.h +0
-2@@ -340,8 +340,6 @@ struct vcpu_vmx {/* apic deadline value in host tsc */ u64 hv_deadline_tsc; -unsigned long host_debugctlmsr; -/* * Only bits masked by msr_ia32_feature_control_valid_bits can be set in * msr_ia32_feature_control. FEAT_CTL_LOCKED is always included arch/x86/kvm/x86.c +1
-0@@ -4991,6 +4991,7 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)/* Save host pkru register if supported */ vcpu->arch.host_pkru =read_pkru (); + vcpu->arch.host_debugctl =get_debugctlmsr ();/* Apply any externally detected TSC adjustments (due to suspend) */ if (unlikely (vcpu->arch.tsc_offset_adjustment)) {
Sean Christopherson
maintainerPATCH 2/3 KVM: SVM: Manually zero/restore DEBUGCTL if LBR virtualization is disabled
In reply to Sean ChristophersonManually zero DEBUGCTL prior to VMRUN if the host's value is non-zero and LBR virtualization is disabled, as hardware only context switches DEBUGCTL if LBR virtualization is fully enabled. Running the guest with the host's value has likely been mildly problematic for quite some time, e.g. it will result in undesirable behavior if host is running with BTF=1.
But the bug became fatal with the introduction of Bus Lock Trap ("Detect" in kernel paralance) support for AMD (commit 408eb7417a92 ("x86/bus_lock: Add support for AMD")), as a bus lock in the guest will trigger an unexpected #DB.
Note, suppressing the bus lock #DB, i.e. simply resuming the guest without injecting a #DB, is not an option. It wouldn't address the general issue with DEBUGCTL, e.g. for things like BTF, and there are other guest-visible side effects if BusLockTrap is left enabled.
If BusLockTrap is disabled, then DR6.BLD is reserved-to-1; any attempts to clear it by software are ignored. But if BusLockTrap is enabled, software can clear DR6.BLD:
Software enables bus lock trap by setting DebugCtl MSR[BLCKDB] (bit 2) to 1. When bus lock trap is enabled, ... The processor indicates that this #DB was caused by a bus lock by clearing DR6[BLD] (bit 11). DR6[11] previously had been defined to be always 1.
and clearing DR6.BLD is "sticky" in that it's not set (i.e. lowered) by other #DBs:
All other #DB exceptions leave DR6[BLD] unmodified
E.g. leaving BusLockTrap enable can confuse a legacy guest that writes '0' to reset DR6.
--- arch/x86/kvm/svm/svm.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
arch/x86/kvm/svm/svm.c +14 -0
@@ -4253,6 +4253,16 @@ static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu,clgi ();kvm_load_guest_xsave_state (vcpu); +/* +* Hardware only context switches DEBUGCTL if LBR virtualization is +* enabled. Manually zero DEBUGCTL if necessary (and restore it after +* VM-Exit), as running with the host's DEBUGCTL can negatively affect +* guest state and can even be fatal, e.g. due to Bus Lock Detect. +*/ +if (vcpu->arch.host_debugctl && + !(svm->vmcb->control.virt_ext &LBR_CTL_ENABLE_MASK )) +update_debugctlmsr (0 ); +kvm_wait_lapic_expire (vcpu);/*
@@ -4280,6 +4290,10 @@ static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu,if (unlikely (svm->vmcb->control.exit_code ==SVM_EXIT_NMI ))kvm_before_interrupt (vcpu,KVM_HANDLING_NMI ); +if (vcpu->arch.host_debugctl && + !(svm->vmcb->control.virt_ext &LBR_CTL_ENABLE_MASK )) +update_debugctlmsr (vcpu->arch.host_debugctl); +kvm_load_host_xsave_state (vcpu);stgi ();
Signature · 1 line
2.48.1.658.g4767266eb4-goog
Re: [PATCH 2/3] KVM: SVM: Manually zero/restore DEBUGCTL if LBR virtualization is disabled
Hi Sean,
Quoted 25 lines · Sean Christopherson
Manually zero DEBUGCTL prior to VMRUN if the host's value is non-zero and LBR virtualization is disabled, as hardware only context switches DEBUGCTL if LBR virtualization is fully enabled. Running the guest with the host's value has likely been mildly problematic for quite some time, e.g. it will result in undesirable behavior if host is running with BTF=1.
But the bug became fatal with the introduction of Bus Lock Trap ("Detect" in kernel paralance) support for AMD (commit
408eb7417a92("x86/bus_lock: Add support for AMD")), as a bus lock in the guest will trigger an unexpected #DB.Note, suppressing the bus lock #DB, i.e. simply resuming the guest without injecting a #DB, is not an option. It wouldn't address the general issue with DEBUGCTL, e.g. for things like BTF, and there are other guest-visible side effects if BusLockTrap is left enabled.
If BusLockTrap is disabled, then DR6.BLD is reserved-to-1; any attempts to clear it by software are ignored. But if BusLockTrap is enabled, software can clear DR6.BLD:
Software enables bus lock trap by setting DebugCtl MSR[BLCKDB] (bit 2) to 1. When bus lock trap is enabled, ... The processor indicates that this #DB was caused by a bus lock by clearing DR6[BLD] (bit 11). DR6[11] previously had been defined to be always 1.
Quoted 7 lines
and clearing DR6.BLD is "sticky" in that it's not set (i.e. lowered) by other #DBs:
All other #DB exceptions leave DR6[BLD] unmodified
E.g. leaving BusLockTrap enable can confuse a legacy guest that writes '0' to reset DR6.
What if guest sets DEBUGCTL[BusLockTrapEn] and runs an application which causes a bus lock? Guest will receive #DB due to bus lock, even though guest CPUID says BusLockTrap isn't supported. Should KVM prevent guest to write to DEBUGCTL[BusLockTrapEn]? Something like:
---
@@ -3168,6 +3168,10 @@ static int svm_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr)
if (data & DEBUGCTL_RESERVED_BITS)
return 1;
+ if ((data & DEBUGCTLMSR_BUS_LOCK_DETECT) &&
+ !guest_cpu_cap_has(vcpu, X86_FEATURE_BUS_LOCK_DETECT))
+ return 1;
+
svm_get_lbr_vmcb(svm)->save.dbgctl = data; svm_update_lbrv(vcpu);
break;
---
Thanks,
Ravi
Sean Christopherson
maintainerRe: [PATCH 2/3] KVM: SVM: Manually zero/restore DEBUGCTL if LBR virtualization is disabled
Quoted 33 lines · Ravi Bangoria
Hi Sean,
Quoted 30 lines · Sean Christopherson
Manually zero DEBUGCTL prior to VMRUN if the host's value is non-zero and LBR virtualization is disabled, as hardware only context switches DEBUGCTL if LBR virtualization is fully enabled. Running the guest with the host's value has likely been mildly problematic for quite some time, e.g. it will result in undesirable behavior if host is running with BTF=1.
But the bug became fatal with the introduction of Bus Lock Trap ("Detect" in kernel paralance) support for AMD (commit
408eb7417a92("x86/bus_lock: Add support for AMD")), as a bus lock in the guest will trigger an unexpected #DB.Note, suppressing the bus lock #DB, i.e. simply resuming the guest without injecting a #DB, is not an option. It wouldn't address the general issue with DEBUGCTL, e.g. for things like BTF, and there are other guest-visible side effects if BusLockTrap is left enabled.
If BusLockTrap is disabled, then DR6.BLD is reserved-to-1; any attempts to clear it by software are ignored. But if BusLockTrap is enabled, software can clear DR6.BLD:
Software enables bus lock trap by setting DebugCtl MSR[BLCKDB] (bit 2) to 1. When bus lock trap is enabled, ... The processor indicates that this #DB was caused by a bus lock by clearing DR6[BLD] (bit 11). DR6[11] previously had been defined to be always 1.
and clearing DR6.BLD is "sticky" in that it's not set (i.e. lowered) by other #DBs:
All other #DB exceptions leave DR6[BLD] unmodified
Quoted 7 lines
Quoted 2 lines
E.g. leaving BusLockTrap enable can confuse a legacy guest that writes '0' to reset DR6.
What if guest sets DEBUGCTL[BusLockTrapEn] and runs an application which causes a bus lock? Guest will receive #DB due to bus lock, even though guest CPUID says BusLockTrap isn't supported. Should KVM prevent guest to write to DEBUGCTL[BusLockTrapEn]? Something like:
Ugh, right, AMD's legacy DEBUGCTL_RESERVED_BITS weirdness. Ideally, KVM would make bits 5:2 reserved. I suspect we could get away with that, because VMX has rejected all bits except BTF and LBR since the beginning. But I really, really don't want to deal with more guest breakage due to sending such a change to stable kernels, so for an immediate fix, I'll add a patch to drop those bits. That'll still be a guest-visible change, e.g. if the guest is enabling LBRs *and* the legacy PBi bits, then the state of the PBi bits would be accurate. But given KVM's craptastic handling of DEBUGCTL, I highly doubt dropping bits 5:2 will break anything.
*sigh*
And that's exposes yet another bug in this code. Zeroing DEBUGCTL before VMRUN is wrong if the guest has enabled BTF. KVM should *load* the guest's desired value if DEBUGCTL == BTF, i.e. if BTF is enabled but LBRs are not.
Quoted 16 lines
---
(quoted diff) +4
-1@@ -3168,6 +3168,10 @@ static int svm_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr) if (data & DEBUGCTL_RESERVED_BITS) return 1; + if ((data & DEBUGCTLMSR_BUS_LOCK_DETECT) && + !guest_cpu_cap_has(vcpu, X86_FEATURE_BUS_LOCK_DETECT)) + return 1; + svm_get_lbr_vmcb(svm)->save.dbgctl = data; svm_update_lbrv(vcpu); break; ---Thanks,
Ravi
Sean Christopherson
maintainerPATCH 3/3 KVM: x86: Snapshot the host's DEBUGCTL after disabling IRQs
In reply to Sean ChristophersonSnapshot the host's DEBUGCTL after disabling IRQs, as perf can toggle debugctl bits from IRQ context, e.g. when enabling/disabling events via smp_call_function_single(). Taking the snapshot (long) before IRQs are disabled could result in KVM effectively clobbering DEBUGCTL due to using a stale snapshot.
--- arch/x86/kvm/x86.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
arch/x86/kvm/x86.c +2 -1
@@ -4991,7 +4991,6 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)/* Save host pkru register if supported */ vcpu->arch.host_pkru =read_pkru (); - vcpu->arch.host_debugctl =get_debugctlmsr ();/* Apply any externally detected TSC adjustments (due to suspend) */ if (unlikely (vcpu->arch.tsc_offset_adjustment)) {
@@ -10984,6 +10983,8 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)set_debugreg (0 ,7 ); } + vcpu->arch.host_debugctl =get_debugctlmsr (); +guest_timing_enter_irqoff ();for (;;) {
Signature · 1 line
2.48.1.658.g4767266eb4-goog
Quoted 6 lines · Sean Christopherson
PeterZ,
Can you confirm that the last patch (snapshot and restore DEBUGCTL with IRQs disabled) is actually necessary? I'm 99% certain it is, but I'm holding out hope that it somehow isn't, because I don't love the idea of adding a RDMSR to every VM-Entry.
I think you're right. I mean, I'd have to go double check and trace the various call paths again, but I'd be very surprised if we can't change DEBUGCTL from NMI context.
Quoted 3 lines
Assuming DEBUGCTL can indeed get modified in IRQ context, it probably makes sense to add a per-CPU cache to eliminate the RDMSR. Unfortunately, there are quite a few open-coded WRMSRs, so it's not a trivial change.
This, I'm surprised we've not yet done that.
Quoted 7 lines
On to the main event...
Fix a long-lurking bug in SVM where KVM runs the guest with the host's DEBUGCTL if LBR virtualization is disabled. AMD CPUs rather stupidly context switch DEBUGCTL if and only if LBR virtualization is enabled (not just supported, but fully enabled).
Quoted 8 lines
The bug has gone unnoticed because until recently, the only bits that KVM would leave set were things like BTF, which are guest visible but won't cause functional problems unless guest software is being especially particular about #DBs.
The bug was exposed by the addition of BusLockTrap ("Detect" in the kernel), as the resulting #DBs due to split-lock accesses in guest userspace (lol Steam) get reflected into the guest by KVM.
Hehe, yeah, games. Yeah we ran into that with bus-lock on intel too :-)