KVM: x86: Improve #DB handling in the emulator
Sean Christopherson
maintainerv3 cover KVM: x86: Improve #DB handling in the emulator
Hou's series of fixes and cleanups related to #DB handling in the emulator.
v3:
- Fix the GUEST_PENDING_DBG_EXCEPTIONS.BS consistency check issue by stuffing the VMCS during injection, not during emulation.
- Drop a prep that is no longer necessary.
- Massage a few shortlogs/changelogs.
- Fix the selftest that takes an IRQ in the STI-shadow after IRET.
- Fix an intermediate bug where kvm_queue_exception_e() was being used instead of kvm_queue_exception_p().
v2:
- https://lore.kernel.org/all/[email protected]
- cleanup in inject_emulated_exception().
- rename 'set_pending_dbg' callback as 'refresh_pending_dbg_exceptions'.
- fold refresh_pending_dbg_exceptions() call into kvm_vcpu_do_singlestep().
- Split the change to move up kvm_set_rflags() into a single patch.
- Move the #DB and IRQ handler registration after guest debug testcases.
v1: https://lore.kernel.org/all/[email protected]
Hou Wenlong (7):
KVM: x86: Capture "struct x86_exception" in
inject_emulated_exception()
KVM: x86: Set guest DR6 by kvm_queue_exception_p() in instruction
emulation
KVM: x86: Honor KVM_GUESTDBG_USE_HW_BP when emulating MOV DR (in
emulator)
KVM: x86: Honor KVM_GUESTDBG_USE_HW_BP when checking for code
breakpoints in emulation
KVM: x86: Move KVM_GUESTDBG_SINGLESTEP handling into
kvm_inject_emulated_db()
KVM: selftests: Verify guest debug DR7.GD checking during instruction
emulation
KVM: selftests: Verify VMX's GUEST_PENDING_DBG_EXCEPTIONS.BS
Consistency Check
Sean Christopherson (3):
KVM: VMX: Refresh GUEST_PENDING_DBG_EXCEPTIONS.BS on all injected #DBs
KVM: x86: Drop kvm_vcpu_do_singlestep() now that it's been gutted
KVM: selftests: Add all (known) EFLAGS bit definitions
arch/x86/kvm/emulate.c | 14 +-- arch/x86/kvm/kvm_emulate.h | 7 +- arch/x86/kvm/vmx/vmx.c | 35 +++--- arch/x86/kvm/x86.c | 111 +++++++++--------- .../selftests/kvm/include/x86/processor.h | 19 ++- .../testing/selftests/kvm/lib/x86/processor.c | 2 +- tools/testing/selftests/kvm/lib/x86/vmx.c | 2 +- tools/testing/selftests/kvm/x86/debug_regs.c | 83 ++++++++++++- 8 files changed, 184 insertions(+), 89 deletions(-)
base-commit: b7fbe9a1bf9ee6c967ef77d366ca58c35fcf1887
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Sean Christopherson
maintainerPATCH v3 1/10 KVM: VMX: Refresh GUEST_PENDING_DBG_EXCEPTIONS.BS on all injected #DBs
Move KVM's stuffing of GUEST_PENDING_DBG_EXCEPTIONS.BS when RFLAGS.TF=1 and MOV/POP SS or STI blocking is active into the exception injection code so that KVM fixes up the VMCS for all injected #DBs, not only those that are reflected back into the guest after #DB interception. E.g. if KVM queues a #DB in the emulator, or more importantly if userspace does save/restore exactly on the #DB+shadow boundary, then KVM needs to massage the VMCS to avoid the VM-Entry consistency check.
Opportunistically update the wording of the comment to describe the behavior as a workaround of flawed CPU behavior/architecture, to make it clear that the *only* thing KVM is doing is fudging around a consistency check. Per the SDM:
There are no pending debug exceptions after VM entry if any of the following are true:
* The VM entry is vectoring with one of the following interruption
types: external interrupt, non-maskable interrupt (NMI), hardware exception, or privileged software exception.
I.e. forcing GUEST_PENDING_DBG_EXCEPTIONS.BS does *not* impact guest- visible behavior.
--- arch/x86/kvm/vmx/vmx.c | 35 ++++++++++++++++++----------------- 1 file changed, 18 insertions(+), 17 deletions(-)
arch/x86/kvm/vmx/vmx.c +18 -17
@@ -1909,6 +1909,24 @@ void vmx_inject_exception(struct kvm_vcpu *vcpu)u32 intr_info = ex->vector |INTR_INFO_VALID_MASK ;struct vcpu_vmx *vmx =to_vmx (vcpu); +/* +* When injecting a #DB, single-stepping is enabled in RFLAGS, and STI +* or MOV-SS blocking is active, set vmcs.PENDING_DBG_EXCEPTIONS.BS to +* prevent a false positive from VM-Entry consistency check. VM-Entry +* asserts that a single-step #DB _must_ be pending in this scenario, +* as the previous instruction cannot have toggled RFLAGS.TF 0=>1 +* (because STI and POP/MOV don't modify RFLAGS), therefore the one +* instruction delay when activating single-step breakpoints must have +* already expired. However, the CPU isn't smart enough to peek at +* vmcs.VM_ENTRY_INTR_INFO_FIELD and so doesn't realize that yes, there +* is indeed a #DB pending/imminent. +*/ +if (ex->vector ==DB_VECTOR && + (vmx_get_rflags (vcpu) &X86_EFLAGS_TF ) && +vmx_get_interrupt_shadow (vcpu)) +vmcs_writel (GUEST_PENDING_DBG_EXCEPTIONS , +vmcs_readl (GUEST_PENDING_DBG_EXCEPTIONS ) |DR6_BS ); +kvm_deliver_exception_payload (vcpu, ex);if (ex->has_error_code) {
@@ -5485,26 +5503,9 @@ static int handle_exception_nmi(struct kvm_vcpu *vcpu)* avoid single-step #DB and MTF updates, as ICEBP is * higher priority. Note, skipping ICEBP still clears * STI and MOVSS blocking. -* -* For all other #DBs, set vmcs.PENDING_DBG_EXCEPTIONS.BS -* if single-step is enabled in RFLAGS and STI or MOVSS -* blocking is active, as the CPU doesn't set the bit -* on VM-Exit due to #DB interception. VM-Entry has a -* consistency check that a single-step #DB is pending -* in this scenario as the previous instruction cannot -* have toggled RFLAGS.TF 0=>1 (because STI and POP/MOV -* don't modify RFLAGS), therefore the one instruction -* delay when activating single-step breakpoints must -* have already expired. Note, the CPU sets/clears BS -* as appropriate for all other VM-Exits types. */ if (is_icebp (intr_info))WARN_ON (!skip_emulated_instruction (vcpu)); -else if ((vmx_get_rflags (vcpu) &X86_EFLAGS_TF ) && - (vmcs_read32 (GUEST_INTERRUPTIBILITY_INFO ) & - (GUEST_INTR_STATE_STI |GUEST_INTR_STATE_MOV_SS ))) -vmcs_writel (GUEST_PENDING_DBG_EXCEPTIONS , -vmcs_readl (GUEST_PENDING_DBG_EXCEPTIONS ) |DR6_BS );kvm_queue_exception_p (vcpu,DB_VECTOR , dr6);return 1 ;
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Re: [PATCH v3 01/10] KVM: VMX: Refresh GUEST_PENDING_DBG_EXCEPTIONS.BS on all injected #DBs
Quoted 48 lines · Sean Christopherson
Move KVM's stuffing of GUEST_PENDING_DBG_EXCEPTIONS.BS when RFLAGS.TF=1 and MOV/POP SS or STI blocking is active into the exception injection code so that KVM fixes up the VMCS for all injected #DBs, not only those that are reflected back into the guest after #DB interception. E.g. if KVM queues a #DB in the emulator, or more importantly if userspace does save/restore exactly on the #DB+shadow boundary, then KVM needs to massage the VMCS to avoid the VM-Entry consistency check.
Opportunistically update the wording of the comment to describe the behavior as a workaround of flawed CPU behavior/architecture, to make it clear that the *only* thing KVM is doing is fudging around a consistency check. Per the SDM:
There are no pending debug exceptions after VM entry if any of the following are true:
* The VM entry is vectoring with one of the following interruption
types: external interrupt, non-maskable interrupt (NMI), hardware exception, or privileged software exception.I.e. forcing GUEST_PENDING_DBG_EXCEPTIONS.BS does *not* impact guest- visible behavior.
Fixes b9bed78e2fa9 ("KVM: VMX: Set vmcs.PENDING_DBG.BS on #DB in STI/MOVSS blocking shadow")Reported-by Hou Wenlong <[email protected]>Closes https://lore.kernel.org/all/b1a294bc9ed4dae532474a5dc6c8cb6e5962de7c.1757416809.git.houwenlong.hwl@antgroup.com open hereSigned-off-by Sean Christopherson <[email protected]>--- arch/x86/kvm/vmx/vmx.c | 35 ++++++++++++++++++----------------- 1 file changed, 18 insertions(+), 17 deletions(-)arch/x86/kvm/vmx/vmx.c +8
-0@@ -1909,6 +1909,24 @@ void vmx_inject_exception(struct kvm_vcpu *vcpu)u32 intr_info = ex->vector |INTR_INFO_VALID_MASK ;struct vcpu_vmx *vmx =to_vmx (vcpu); +/* +* When injecting a #DB, single-stepping is enabled in RFLAGS, and STI +* or MOV-SS blocking is active, set vmcs.PENDING_DBG_EXCEPTIONS.BS to +* prevent a false positive from VM-Entry consistency check. VM-Entry +* asserts that a single-step #DB _must_ be pending in this scenario, +* as the previous instruction cannot have toggled RFLAGS.TF 0=>1 +* (because STI and POP/MOV don't modify RFLAGS), therefore the one +* instruction delay when activating single-step breakpoints must have
arch/x86/kvm/vmx/vmx.c void vmx_inject_exception(struct kvm_vcpu *vcpu) · line 1929
Quoted 10 lines
arch/x86/kvm/vmx/vmx.c +10
-0+* already expired. However, the CPU isn't smart enough to peek at +* vmcs.VM_ENTRY_INTR_INFO_FIELD and so doesn't realize that yes, there +* is indeed a #DB pending/imminent. +*/ +if (ex->vector ==DB_VECTOR && + (vmx_get_rflags (vcpu) &X86_EFLAGS_TF ) && +vmx_get_interrupt_shadow (vcpu)) +vmcs_writel (GUEST_PENDING_DBG_EXCEPTIONS , +vmcs_readl (GUEST_PENDING_DBG_EXCEPTIONS ) |DR6_BS ); +
This matches a version of the modification I had considered earlier.
Thanks!
Quoted 32 lines
arch/x86/kvm/vmx/vmx.c +0
-17kvm_deliver_exception_payload (vcpu, ex);if (ex->has_error_code) {@@ -5485,26 +5503,9 @@ static int handle_exception_nmi(struct kvm_vcpu *vcpu)* avoid single-step #DB and MTF updates, as ICEBP is * higher priority. Note, skipping ICEBP still clears * STI and MOVSS blocking. -* -* For all other #DBs, set vmcs.PENDING_DBG_EXCEPTIONS.BS -* if single-step is enabled in RFLAGS and STI or MOVSS -* blocking is active, as the CPU doesn't set the bit -* on VM-Exit due to #DB interception. VM-Entry has a -* consistency check that a single-step #DB is pending -* in this scenario as the previous instruction cannot -* have toggled RFLAGS.TF 0=>1 (because STI and POP/MOV -* don't modify RFLAGS), therefore the one instruction -* delay when activating single-step breakpoints must -* have already expired. Note, the CPU sets/clears BS -* as appropriate for all other VM-Exits types. */ if (is_icebp (intr_info))WARN_ON (!skip_emulated_instruction (vcpu)); -else if ((vmx_get_rflags (vcpu) &X86_EFLAGS_TF ) && - (vmcs_read32 (GUEST_INTERRUPTIBILITY_INFO ) & - (GUEST_INTR_STATE_STI |GUEST_INTR_STATE_MOV_SS ))) -vmcs_writel (GUEST_PENDING_DBG_EXCEPTIONS , -vmcs_readl (GUEST_PENDING_DBG_EXCEPTIONS ) |DR6_BS );kvm_queue_exception_p (vcpu,DB_VECTOR , dr6);return 1 ;Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Sean Christopherson
maintainerRe: [PATCH v3 01/10] KVM: VMX: Refresh GUEST_PENDING_DBG_EXCEPTIONS.BS on all injected #DBs
In reply to Sean ChristophersonQuoted 47 lines · Sean Christopherson
Move KVM's stuffing of GUEST_PENDING_DBG_EXCEPTIONS.BS when RFLAGS.TF=1 and MOV/POP SS or STI blocking is active into the exception injection code so that KVM fixes up the VMCS for all injected #DBs, not only those that are reflected back into the guest after #DB interception. E.g. if KVM queues a #DB in the emulator, or more importantly if userspace does save/restore exactly on the #DB+shadow boundary, then KVM needs to massage the VMCS to avoid the VM-Entry consistency check.
Opportunistically update the wording of the comment to describe the behavior as a workaround of flawed CPU behavior/architecture, to make it clear that the *only* thing KVM is doing is fudging around a consistency check. Per the SDM:
There are no pending debug exceptions after VM entry if any of the following are true:
* The VM entry is vectoring with one of the following interruption
types: external interrupt, non-maskable interrupt (NMI), hardware exception, or privileged software exception.I.e. forcing GUEST_PENDING_DBG_EXCEPTIONS.BS does *not* impact guest- visible behavior.
Fixes b9bed78e2fa9 ("KVM: VMX: Set vmcs.PENDING_DBG.BS on #DB in STI/MOVSS blocking shadow")Reported-by Hou Wenlong <[email protected]>Closes https://lore.kernel.org/all/b1a294bc9ed4dae532474a5dc6c8cb6e5962de7c.1757416809.git.houwenlong.hwl@antgroup.com open hereSigned-off-by Sean Christopherson <[email protected]>--- arch/x86/kvm/vmx/vmx.c | 35 ++++++++++++++++++----------------- 1 file changed, 18 insertions(+), 17 deletions(-)arch/x86/kvm/vmx/vmx.c +7
-0@@ -1909,6 +1909,24 @@ void vmx_inject_exception(struct kvm_vcpu *vcpu)u32 intr_info = ex->vector |INTR_INFO_VALID_MASK ;struct vcpu_vmx *vmx =to_vmx (vcpu); +/* +* When injecting a #DB, single-stepping is enabled in RFLAGS, and STI +* or MOV-SS blocking is active, set vmcs.PENDING_DBG_EXCEPTIONS.BS to +* prevent a false positive from VM-Entry consistency check. VM-Entry +* asserts that a single-step #DB _must_ be pending in this scenario, +* as the previous instruction cannot have toggled RFLAGS.TF 0=>1 +* (because STI and POP/MOV don't modify RFLAGS), therefore the one
arch/x86/kvm/vmx/vmx.c void vmx_inject_exception(struct kvm_vcpu *vcpu) · line 1928
Quoted 10 lines
arch/x86/kvm/vmx/vmx.c +10
-0+* instruction delay when activating single-step breakpoints must have +* already expired. However, the CPU isn't smart enough to peek at +* vmcs.VM_ENTRY_INTR_INFO_FIELD and so doesn't realize that yes, there +* is indeed a #DB pending/imminent. +*/ +if (ex->vector ==DB_VECTOR && + (vmx_get_rflags (vcpu) &X86_EFLAGS_TF ) && +vmx_get_interrupt_shadow (vcpu)) +vmcs_writel (GUEST_PENDING_DBG_EXCEPTIONS , +vmcs_readl (GUEST_PENDING_DBG_EXCEPTIONS ) |DR6_BS );
Pulling in a Sashiko comment:
: By restricting this workaround to only when a #DB is injected, does this : leave the VM vulnerable to a VM-Entry failure regression after live migration? :
: KVM does not export GUEST_PENDING_DBG_EXCEPTIONS to userspace via : KVM_GET_VCPU_EVENTS. Therefore, upon migration, the destination KVM : initializes the VMCS with GUEST_PENDING_DBG_EXCEPTIONS=0.
:
: If a live migration occurs when the guest is in an active interrupt shadow : with RFLAGS.TF=1, but a different event is pending (or no event is pending : due to a host timer preemption), this DB_VECTOR check is skipped or : vmx_inject_exception() is never called.
:
: Consequently, KVM will attempt VM-Entry with TF=1, shadow=1, and BS=0. : The Intel SDM mandates that if RFLAGS.TF=1 and STI or MOV SS blocking is : active, the VM-Entry consistency check requires
: GUEST_PENDING_DBG_EXCEPTIONS.BS=1. The hardware VM-Entry will fail due to : invalid guest state.
:
: Since vmx_guest_state_valid() does not check the GUEST_PENDING_DBG_EXCEPTIONS : field, KVM's emulation_required flag evaluates to false. KVM then falls : into the error path in __vmx_handle_exit(), dumping the VMCS and crashing : the guest by returning KVM_EXIT_FAIL_ENTRY to userspace.
:
: Does KVM need to handle the BS bit requirement in a broader context to : account for live migration when no #DB is being injected?
Yes, but that's a different problem entirely[*], and isn't even solvable on AMD because SVM lacks an equivalent for GUEST_PENDING_DBG_EXCEPTIONS. Note, only MOV/POP-SS blocking matters, because STI blocking doesn't prevent single-step #DBs, and single-step #DBs have higher priority than IRQs.
Re: [PATCH v3 01/10] KVM: VMX: Refresh GUEST_PENDING_DBG_EXCEPTIONS.BS on all injected #DBs
Quoted 61 lines · Sean Christopherson
Quoted 47 lines · Sean Christopherson
Move KVM's stuffing of GUEST_PENDING_DBG_EXCEPTIONS.BS when RFLAGS.TF=1 and MOV/POP SS or STI blocking is active into the exception injection code so that KVM fixes up the VMCS for all injected #DBs, not only those that are reflected back into the guest after #DB interception. E.g. if KVM queues a #DB in the emulator, or more importantly if userspace does save/restore exactly on the #DB+shadow boundary, then KVM needs to massage the VMCS to avoid the VM-Entry consistency check.
Opportunistically update the wording of the comment to describe the behavior as a workaround of flawed CPU behavior/architecture, to make it clear that the *only* thing KVM is doing is fudging around a consistency check. Per the SDM:
There are no pending debug exceptions after VM entry if any of the following are true:
* The VM entry is vectoring with one of the following interruption
types: external interrupt, non-maskable interrupt (NMI), hardware exception, or privileged software exception.I.e. forcing GUEST_PENDING_DBG_EXCEPTIONS.BS does *not* impact guest- visible behavior.
Fixes b9bed78e2fa9 ("KVM: VMX: Set vmcs.PENDING_DBG.BS on #DB in STI/MOVSS blocking shadow")Reported-by Hou Wenlong <[email protected]>Closes https://lore.kernel.org/all/b1a294bc9ed4dae532474a5dc6c8cb6e5962de7c.1757416809.git.houwenlong.hwl@antgroup.com open hereSigned-off-by Sean Christopherson <[email protected]>--- arch/x86/kvm/vmx/vmx.c | 35 ++++++++++++++++++----------------- 1 file changed, 18 insertions(+), 17 deletions(-)arch/x86/kvm/vmx/vmx.c +7
-0@@ -1909,6 +1909,24 @@ void vmx_inject_exception(struct kvm_vcpu *vcpu)u32 intr_info = ex->vector |INTR_INFO_VALID_MASK ;struct vcpu_vmx *vmx =to_vmx (vcpu); +/* +* When injecting a #DB, single-stepping is enabled in RFLAGS, and STI +* or MOV-SS blocking is active, set vmcs.PENDING_DBG_EXCEPTIONS.BS to +* prevent a false positive from VM-Entry consistency check. VM-Entry +* asserts that a single-step #DB _must_ be pending in this scenario, +* as the previous instruction cannot have toggled RFLAGS.TF 0=>1 +* (because STI and POP/MOV don't modify RFLAGS), therefore the one arch/x86/kvm/vmx/vmx.c void vmx_inject_exception(struct kvm_vcpu *vcpu) · line 1928
Quoted 10 lines
arch/x86/kvm/vmx/vmx.c +10
-0+* instruction delay when activating single-step breakpoints must have +* already expired. However, the CPU isn't smart enough to peek at +* vmcs.VM_ENTRY_INTR_INFO_FIELD and so doesn't realize that yes, there +* is indeed a #DB pending/imminent. +*/ +if (ex->vector ==DB_VECTOR && + (vmx_get_rflags (vcpu) &X86_EFLAGS_TF ) && +vmx_get_interrupt_shadow (vcpu)) +vmcs_writel (GUEST_PENDING_DBG_EXCEPTIONS , +vmcs_readl (GUEST_PENDING_DBG_EXCEPTIONS ) |DR6_BS );Pulling in a Sashiko comment:
Quoted 12 lines
: By restricting this workaround to only when a #DB is injected, does this : leave the VM vulnerable to a VM-Entry failure regression after live migration? :
: KVM does not exportGUEST_PENDING_DBG_EXCEPTIONSto userspace via :KVM_GET_VCPU_EVENTS. Therefore, upon migration, the destination KVM : initializes the VMCS with GUEST_PENDING_DBG_EXCEPTIONS=0.
:
: If a live migration occurs when the guest is in an active interrupt shadow : with RFLAGS.TF=1, but a different event is pending (or no event is pending : due to a host timer preemption), thisDB_VECTORcheck is skipped or :vmx_inject_exception()is never called.
:
Nice AI review. This reminds me that before writing v1 I was wondering:
if KVM_GUESTDBG_SINGLESTEP is enabled and there is no #DB injection, then single-stepping over STI should fail as well. Then I noticed the following code:
```
/* When single-stepping over STI and MOV SS, we must clear the
* corresponding interruptibility bits in the guest state. Otherwise
* vmentry fails as it then expects bit 14 (BS) in pending debug
* exceptions being set, but that's not correct for the guest
* debugging
* case. */
if (vcpu->guest_debug & KVM_GUESTDBG_SINGLESTEP)
vmx_set_interrupt_shadow(vcpu, 0);
```However, at that time I was missing the live-migration case.
Quoted 14 lines
: Crnsequently, KVM will attempt VM-Entry with TF=1, shadow=1, and BS=0. : The Intel SDM mandates that if RFLAGS.TF=1 and STI or MOV SS blocking is : active, the VM-Entry consistency check requires
: GUEST_PENDING_DBG_EXCEPTIONS.BS=1. The hardware VM-Entry will fail due to : invalid guest state.
:
: Sincevmx_guest_state_valid()does not check theGUEST_PENDING_DBG_EXCEPTIONS: field, KVM'semulation_requiredflag evaluates to false. KVM then falls : into the error path in__vmx_handle_exit(), dumping the VMCS and crashing : the guest by returningKVM_EXIT_FAIL_ENTRYto userspace.
:
: Does KVM need to handle the BS bit requirement in a broader context to : account for live migration when no #DB is being injected?
Quoted 5 lines
Yes, but that's a different problem entirely[*], and isn't even solvable on AMD because SVM lacks an equivalent for
GUEST_PENDING_DBG_EXCEPTIONS. Note, only MOV/POP-SS blocking matters, because STI blocking doesn't prevent single-step #DBs, and single-step #DBs have higher priority than IRQs.
Besides that, if MOV/POP SS is emulated with single-stepping, the emulator currently injects a #DB. As a result, the guest would observe a single-step #DB during the MOV/POP SS shadow, as stated in Intel SDM 28.7.1:
"If the VM entry is vectoring, there is no blocking by STI or by MOV SS following the VM entry, regardless of the contents of the interruptibility-state field."
If we try to fix this by not injecting #DB, then VM entry fails again. Since emulating MOV/POP SS only happens with force emulation, perhaps we can simply document this behavior?
Thanks!
Quoted 1 line
Sean Christopherson
maintainerPATCH v3 2/10 KVM: x86: Capture "struct x86_exception" in inject_emulated_exception()
In reply to Sean ChristophersonFrom: Hou Wenlong <[email protected]>
As all callers in inject_emulated_exception() use "struct x86_exception" directly, capture it locally instead of using the context.
No functional change intended.
--- arch/x86/kvm/x86.c | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-)
arch/x86/kvm/x86.c +6 -7
@@ -8978,15 +8978,14 @@ static void toggle_interruptibility(struct kvm_vcpu *vcpu, u32 mask)static void inject_emulated_exception (struct kvm_vcpu *vcpu) { -struct x86_emulate_ctxt *ctxt = vcpu->arch.emulate_ctxt; +struct x86_exception *ex = &vcpu->arch.emulate_ctxt->exception; -if (ctxt->exception.vector ==PF_VECTOR ) -kvm_inject_emulated_page_fault (vcpu, &ctxt->exception); -else if (ctxt->exception.error_code_valid) -kvm_queue_exception_e (vcpu, ctxt->exception.vector, - ctxt->exception.error_code); +if (ex->vector ==PF_VECTOR ) +kvm_inject_emulated_page_fault (vcpu, ex); +else if (ex->error_code_valid) +kvm_queue_exception_e (vcpu, ex->vector, ex->error_code);else -kvm_queue_exception (vcpu, ctxt->exception.vector); +kvm_queue_exception (vcpu, ex->vector); }static struct x86_emulate_ctxt *alloc_emulate_ctxt (struct kvm_vcpu *vcpu)
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Re: [PATCH v3 02/10] KVM: x86: Capture "struct x86_exception" in inject_emulated_exception()
Quoted 10 lines · Sean Christopherson
From: Hou Wenlong <[email protected]>
As all callers in
inject_emulated_exception()use "structx86_exception" directly, capture it locally instead of using the context.No functional change intended.
Suggested-by Sean Christopherson <[email protected]>Signed-off-by Hou Wenlong <[email protected]>Signed-off-by Sean Christopherson <[email protected]>
Sean Christopherson
maintainerPATCH v3 3/10 KVM: x86: Set guest DR6 by kvm_queue_exception_p() in instruction emulation
In reply to Sean ChristophersonFrom: Hou Wenlong <[email protected]>
Record DR6 in emulate_db() and use kvm_queue_exception_p() to set DR6 instead of directly using kvm_set_dr6() in emulation, which keeps the handling of DR6 during #DB injection consistent with other code paths.
No functional change intended.
--- arch/x86/kvm/emulate.c | 14 ++++---------- arch/x86/kvm/kvm_emulate.h | 6 +++++- arch/x86/kvm/x86.c | 10 +++++++++- 3 files changed, 18 insertions(+), 12 deletions(-)
arch/x86/kvm/emulate.c +4 -10
@@ -540,8 +540,9 @@ static int emulate_exception(struct x86_emulate_ctxt *ctxt, int vec,return X86EMUL_PROPAGATE_FAULT ; } -static int emulate_db (struct x86_emulate_ctxt *ctxt) +static int emulate_db (struct x86_emulate_ctxt *ctxt,unsigned long dr6) { + ctxt->exception.dr6 = dr6;return emulate_exception (ctxt,DB_VECTOR ,0 , false); }
@@ -3847,15 +3848,8 @@ static int check_dr_read(struct x86_emulate_ctxt *ctxt)if ((cr4 &X86_CR4_DE ) && (dr ==4 || dr ==5 ))return emulate_ud (ctxt); -if (ctxt->ops->get_dr (ctxt,7 ) &DR7_GD ) { -ulong dr6; - - dr6 = ctxt->ops->get_dr (ctxt,6 ); - dr6 &= ~DR_TRAP_BITS ; - dr6 |=DR6_BD |DR6_ACTIVE_LOW ; - ctxt->ops->set_dr (ctxt,6 , dr6); -return emulate_db (ctxt); - } +if (ctxt->ops->get_dr (ctxt,7 ) &DR7_GD ) +return emulate_db (ctxt,DR6_BD );return X86EMUL_CONTINUE ; }
arch/x86/kvm/kvm_emulate.h +5 -1
@@ -24,7 +24,11 @@ struct x86_exception {bool error_code_valid;u16 error_code;bool nested_page_fault; -u64 address;/* cr2 or nested page fault gpa */ +union { +u64 address;/* cr2 or nested page fault gpa */ +unsigned long dr6; +u64 payload; + };u8 async_page_fault;unsigned long exit_qualification; };
arch/x86/kvm/x86.c +9 -1
@@ -8976,11 +8976,18 @@ static void toggle_interruptibility(struct kvm_vcpu *vcpu, u32 mask) } } +static void kvm_inject_emulated_db (struct kvm_vcpu *vcpu,unsigned long dr6) +{ +kvm_queue_exception_p (vcpu,DB_VECTOR , dr6); +} +static void inject_emulated_exception (struct kvm_vcpu *vcpu) {struct x86_exception *ex = &vcpu->arch.emulate_ctxt->exception; -if (ex->vector ==PF_VECTOR ) +if (ex->vector ==DB_VECTOR ) +kvm_inject_emulated_db (vcpu, ex->dr6); +else if (ex->vector ==PF_VECTOR )kvm_inject_emulated_page_fault (vcpu, ex);else if (ex->error_code_valid)kvm_queue_exception_e (vcpu, ex->vector, ex->error_code);
@@ -9025,6 +9032,7 @@ static void init_emulate_ctxt(struct kvm_vcpu *vcpu) ctxt->interruptibility =0 ; ctxt->have_exception = false; ctxt->exception.vector =-1 ; + ctxt->exception.payload =0 ; ctxt->perm_ok = false;init_decode_cache (ctxt);
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Re: [PATCH v3 03/10] KVM: x86: Set guest DR6 by kvm_queue_exception_p() in instruction emulation
Quoted 37 lines · Sean Christopherson
From: Hou Wenlong <[email protected]>
Record DR6 in
emulate_db()and usekvm_queue_exception_p()to set DR6 instead of directly usingkvm_set_dr6()in emulation, which keeps the handling of DR6 during #DB injection consistent with other code paths.No functional change intended.
Signed-off-by Hou Wenlong <[email protected]>[sean: fix e vs. p goof, add kvm_inject_emulated_db() right away]Signed-off-by Sean Christopherson <[email protected]>--- arch/x86/kvm/emulate.c | 14 ++++---------- arch/x86/kvm/kvm_emulate.h | 6 +++++- arch/x86/kvm/x86.c | 10 +++++++++- 3 files changed, 18 insertions(+), 12 deletions(-)arch/x86/kvm/emulate.c +2
-2@@ -540,8 +540,9 @@ static int emulate_exception(struct x86_emulate_ctxt *ctxt, int vec,return X86EMUL_PROPAGATE_FAULT ; } -static int emulate_db (struct x86_emulate_ctxt *ctxt) +static int emulate_db (struct x86_emulate_ctxt *ctxt,unsigned long dr6) { + ctxt->exception.dr6 = dr6;return emulate_exception (ctxt,DB_VECTOR ,0 , false); }@@ -3847,15 +3848,8 @@ static int check_dr_read(struct x86_emulate_ctxt *ctxt)if ((cr4 &X86_CR4_DE ) && (dr ==4 || dr ==5 ))return emulate_ud (ctxt); -if (ctxt->ops->get_dr (ctxt,7 ) &DR7_GD ) {
arch/x86/kvm/emulate.c static int check_dr_read(struct x86_emulate_ctxt *ctxt) · line 3852
Quoted 10 lines
arch/x86/kvm/emulate.c +2
-8-ulong dr6; - - dr6 = ctxt->ops->get_dr (ctxt,6 ); - dr6 &= ~DR_TRAP_BITS ; - dr6 |=DR6_BD |DR6_ACTIVE_LOW ; - ctxt->ops->set_dr (ctxt,6 , dr6); -return emulate_db (ctxt); - } +if (ctxt->ops->get_dr (ctxt,7 ) &DR7_GD ) +return emulate_db (ctxt,DR6_BD );
For other readers of this patch, the manipulations done to DR6 here are no longer needed because kvm_deliver_exception_payload() will apply them when delivering the exception.
Not including that in the changelog and leaving it as an exercise to the reader to chase it down is just mean :P
Anyway, it looks correct:
Sean Christopherson
maintainerPATCH v3 4/10 KVM: x86: Honor KVM_GUESTDBG_USE_HW_BP when emulating MOV DR (in emulator)
In reply to Sean ChristophersonFrom: Hou Wenlong <[email protected]>
When emulating a MOV DR instruction, honor KVM_GUESTDBG_USE_HW_BP when checking DR7.GD, and if there is a general-detect #DB, route it to host userspace as appropriate. Consulting only the guest's actual DR7 causes KVM to fail to report a DR access to userspace (assuming the guest itself doesn't have DR7.GD=1).
--- arch/x86/kvm/emulate.c | 2 +- arch/x86/kvm/kvm_emulate.h | 1 + arch/x86/kvm/x86.c | 41 ++++++++++++++++++++++++++++++-------- 3 files changed, 35 insertions(+), 9 deletions(-)
arch/x86/kvm/emulate.c +1 -1
@@ -3848,7 +3848,7 @@ static int check_dr_read(struct x86_emulate_ctxt *ctxt)if ((cr4 &X86_CR4_DE ) && (dr ==4 || dr ==5 ))return emulate_ud (ctxt); -if (ctxt->ops->get_dr (ctxt,7 ) &DR7_GD ) +if (ctxt->ops->get_eff_dr7 (ctxt) &DR7_GD )return emulate_db (ctxt,DR6_BD );return X86EMUL_CONTINUE ;
arch/x86/kvm/kvm_emulate.h +1 -0
@@ -215,6 +215,7 @@ struct x86_emulate_ops {ulong (*get_cr)(struct x86_emulate_ctxt *ctxt,int cr);int (*set_cr)(struct x86_emulate_ctxt *ctxt,int cr,ulong val);int (*cpl)(struct x86_emulate_ctxt *ctxt); +ulong (*get_eff_dr7)(struct x86_emulate_ctxt *ctxt);ulong (*get_dr)(struct x86_emulate_ctxt *ctxt,int dr);int (*set_dr)(struct x86_emulate_ctxt *ctxt,int dr,ulong value);int (*set_msr_with_filter)(struct x86_emulate_ctxt *ctxt,u32 msr_index,u64 data);
arch/x86/kvm/x86.c +33 -8
90 lines not on this page · raw message
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Re: [PATCH v3 04/10] KVM: x86: Honor KVM_GUESTDBG_USE_HW_BP when emulating MOV DR (in emulator)
Quoted 37 lines · Sean Christopherson
From: Hou Wenlong <[email protected]>
When emulating a MOV DR instruction, honor
KVM_GUESTDBG_USE_HW_BPwhen checking DR7.GD, and if there is a general-detect #DB, route it to host userspace as appropriate. Consulting only the guest's actual DR7 causes KVM to fail to report a DR access to userspace (assuming the guest itself doesn't have DR7.GD=1).Fixes ae675ef01cd8 ("KVM: x86: Wire-up hardware breakpoints for guest debugging")Suggested-by Lai Jiangshan <[email protected]>Signed-off-by Hou Wenlong <[email protected]>[sean: only expose effective DR7 to emulator, massage changelog]Signed-off-by Sean Christopherson <[email protected]>--- arch/x86/kvm/emulate.c | 2 +- arch/x86/kvm/kvm_emulate.h | 1 + arch/x86/kvm/x86.c | 41 ++++++++++++++++++++++++++++++-------- 3 files changed, 35 insertions(+), 9 deletions(-)arch/x86/kvm/emulate.c +1
-1@@ -3848,7 +3848,7 @@ static int check_dr_read(struct x86_emulate_ctxt *ctxt)if ((cr4 &X86_CR4_DE ) && (dr ==4 || dr ==5 ))return emulate_ud (ctxt); -if (ctxt->ops->get_dr (ctxt,7 ) &DR7_GD ) +if (ctxt->ops->get_eff_dr7 (ctxt) &DR7_GD )return emulate_db (ctxt,DR6_BD );return X86EMUL_CONTINUE ;arch/x86/kvm/kvm_emulate.h +0
-0
arch/x86/kvm/kvm_emulate.h struct x86_emulate_ops { · line 218
Quoted 4 lines
arch/x86/kvm/kvm_emulate.h +1
-0@@ -215,6 +215,7 @@ struct x86_emulate_ops {ulong (*get_cr)(struct x86_emulate_ctxt *ctxt,int cr);int (*set_cr)(struct x86_emulate_ctxt *ctxt,int cr,ulong val);int (*cpl)(struct x86_emulate_ctxt *ctxt); +ulong (*get_eff_dr7)(struct x86_emulate_ctxt *ctxt);
I would name this get_effective_dr7, and the callback kvm_get_effective_dr7(). I don't think saving a few characters here offsets the readability losses.
Sean Christopherson
maintainerPATCH v3 5/10 KVM: x86: Honor KVM_GUESTDBG_USE_HW_BP when checking for code breakpoints in emulation
In reply to Sean ChristophersonFrom: Hou Wenlong <[email protected]>
When KVM_GUESTDBG_USE_HW_BP is enabled, i.e. userspace is usurping the guest's hardware debug registers, the guest's effective breakpoints are controlled by userspace rather than by the guest itself. Honor the KVM_GUESTDBG_USE_HW_BP behavior when handling code #DBs in the emulator so that userspace (and the guest) gets consistent behavior for code #DBs regardless of whether an instruction is executed natively or emulated by KVM.
To aid in userspace debug, don't treat code breakpoints as inhibited if KVM_GUESTDBG_USE_HW_BP is enabled as accurately emulating x86 architecture is obviously a non-goal of guest-debug.
--- arch/x86/kvm/x86.c | 35 ++++++++++------------------------- 1 file changed, 10 insertions(+), 25 deletions(-)
arch/x86/kvm/x86.c +10 -25
@@ -9319,6 +9319,9 @@ EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_skip_emulated_instruction);static bool kvm_is_code_breakpoint_inhibited (struct kvm_vcpu *vcpu) { +if (vcpu->guest_debug &KVM_GUESTDBG_USE_HW_BP ) +return false; +if (kvm_get_rflags (vcpu) &X86_EFLAGS_RF )return true;
@@ -9335,6 +9338,8 @@ static bool kvm_is_code_breakpoint_inhibited(struct kvm_vcpu *vcpu)static bool kvm_vcpu_check_code_breakpoint (struct kvm_vcpu *vcpu,int emulation_type,int *r) { +unsigned long dr7 =kvm_get_eff_dr7 (vcpu); +WARN_ON_ONCE (emulation_type &EMULTYPE_NO_DECODE );/*
@@ -9355,34 +9360,14 @@ static bool kvm_vcpu_check_code_breakpoint(struct kvm_vcpu *vcpu, EMULTYPE_TRAP_UD | EMULTYPE_VMWARE_GP | EMULTYPE_PF))return false; -if (unlikely (vcpu->guest_debug &KVM_GUESTDBG_USE_HW_BP ) && - (vcpu->arch.guest_debug_dr7 &DR7_BP_EN_MASK )) { -struct kvm_run *kvm_run = vcpu->run; -unsigned long eip =kvm_get_linear_rip (vcpu); -u32 dr6 =kvm_vcpu_check_hw_bp (eip,0 , - vcpu->arch.guest_debug_dr7, - vcpu->arch.eff_db); - -if (dr6 !=0 ) { - kvm_run->debug.arch.dr6 = dr6 |DR6_ACTIVE_LOW ; - kvm_run->debug.arch.pc = eip; - kvm_run->debug.arch.exception =DB_VECTOR ; - kvm_run->exit_reason =KVM_EXIT_DEBUG ; - *r =0 ; -return true; - } - } - -if (unlikely (vcpu->arch.dr7 &DR7_BP_EN_MASK ) && +if (unlikely (dr7 &DR7_BP_EN_MASK ) && !kvm_is_code_breakpoint_inhibited (vcpu)) {unsigned long eip =kvm_get_linear_rip (vcpu); -u32 dr6 =kvm_vcpu_check_hw_bp (eip,0 , - vcpu->arch.dr7, - vcpu->arch.db); +u32 dr6 =kvm_vcpu_check_hw_bp (eip,0 , dr7, + vcpu->arch.eff_db); -if (dr6 !=0 ) { -kvm_queue_exception_p (vcpu,DB_VECTOR , dr6); - *r =1 ; +if (dr6) { + *r =kvm_inject_emulated_db (vcpu, dr6);return true; } }
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Sean Christopherson
maintainerPATCH v3 6/10 KVM: x86: Move KVM_GUESTDBG_SINGLESTEP handling into kvm_inject_emulated_db()
In reply to Sean ChristophersonFrom: Hou Wenlong <[email protected]>
Move KVM_GUESTDBG_SINGLESTEP handling from kvm_vcpu_do_singlestep() into kvm_inject_emulated_db() to dedup the USE_HW_BP vs. SINGLESTEP logic, and to allow for removing kvm_vcpu_do_singlestep() entirely.
No functional change intended.
--- arch/x86/kvm/x86.c | 14 ++------------ 1 file changed, 2 insertions(+), 12 deletions(-)
arch/x86/kvm/x86.c +2 -12
@@ -8994,7 +8994,7 @@ static int kvm_inject_emulated_db(struct kvm_vcpu *vcpu, unsigned long dr6) {struct kvm_run *kvm_run = vcpu->run; -if (vcpu->guest_debug &KVM_GUESTDBG_USE_HW_BP ) { +if (vcpu->guest_debug & (KVM_GUESTDBG_USE_HW_BP |KVM_GUESTDBG_SINGLESTEP )) { kvm_run->debug.arch.dr6 = dr6 |DR6_ACTIVE_LOW ; kvm_run->debug.arch.pc =kvm_get_linear_rip (vcpu); kvm_run->debug.arch.exception =DB_VECTOR ;
@@ -9279,17 +9279,7 @@ static int kvm_vcpu_check_hw_bp(unsigned long addr, u32 type, u32 dr7,static int kvm_vcpu_do_singlestep (struct kvm_vcpu *vcpu) { -struct kvm_run *kvm_run = vcpu->run; - -if (vcpu->guest_debug &KVM_GUESTDBG_SINGLESTEP ) { - kvm_run->debug.arch.dr6 =DR6_BS |DR6_ACTIVE_LOW ; - kvm_run->debug.arch.pc =kvm_get_linear_rip (vcpu); - kvm_run->debug.arch.exception =DB_VECTOR ; - kvm_run->exit_reason =KVM_EXIT_DEBUG ; -return 0 ; - } -kvm_queue_exception_p (vcpu,DB_VECTOR ,DR6_BS ); -return 1 ; +return kvm_inject_emulated_db (vcpu,DR6_BS ); }int kvm_skip_emulated_instruction (struct kvm_vcpu *vcpu)
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Re: [PATCH v3 06/10] KVM: x86: Move KVM_GUESTDBG_SINGLESTEP handling into kvm_inject_emulated_db()
Quoted 6 lines · Sean Christopherson
From: Hou Wenlong <[email protected]>
Move
KVM_GUESTDBG_SINGLESTEPhandling fromkvm_vcpu_do_singlestep()intokvm_inject_emulated_db()to dedup theUSE_HW_BPvs. SINGLESTEP logic, and to allow for removingkvm_vcpu_do_singlestep()entirely.
Quoted 6 lines
No functional change intended.
Suggested-by Lai Jiangshan <[email protected]>Signed-off-by Hou Wenlong <[email protected]>[sean: massage changelog]Signed-off-by Sean Christopherson <[email protected]>
Sean Christopherson
maintainerPATCH v3 7/10 KVM: x86: Drop kvm_vcpu_do_singlestep() now that it's been gutted
In reply to Sean ChristophersonNow that all of kvm_vcpu_do_singlestep()'s previously-unique functionality has been moved into kvm_inject_emulated_db(), drop the one-line wrapper.
No functional change intended.
--- arch/x86/kvm/x86.c | 10 ++-------- 1 file changed, 2 insertions(+), 8 deletions(-)
arch/x86/kvm/x86.c +2 -8
@@ -133,7 +133,6 @@ static void process_nmi(struct kvm_vcpu *vcpu);static void __kvm_set_rflags (struct kvm_vcpu *vcpu,unsigned long rflags);static void store_regs (struct kvm_vcpu *vcpu);static int sync_regs (struct kvm_vcpu *vcpu); -static int kvm_vcpu_do_singlestep (struct kvm_vcpu *vcpu);static int __set_sregs2 (struct kvm_vcpu *vcpu,struct kvm_sregs2 *sregs2);static void __get_sregs2 (struct kvm_vcpu *vcpu,struct kvm_sregs2 *sregs2);
@@ -9277,11 +9276,6 @@ static int kvm_vcpu_check_hw_bp(unsigned long addr, u32 type, u32 dr7,return dr6; } -static int kvm_vcpu_do_singlestep (struct kvm_vcpu *vcpu) -{ -return kvm_inject_emulated_db (vcpu,DR6_BS ); -} -int kvm_skip_emulated_instruction (struct kvm_vcpu *vcpu) {unsigned long rflags =kvm_x86_call (get_rflags)(vcpu);
@@ -9302,7 +9296,7 @@ int kvm_skip_emulated_instruction(struct kvm_vcpu *vcpu)* that sets the TF flag". */ if (unlikely (rflags &X86_EFLAGS_TF )) - r =kvm_vcpu_do_singlestep (vcpu); + r =kvm_inject_emulated_db (vcpu,DR6_BS );return r; }EXPORT_SYMBOL_FOR_KVM_INTERNAL (kvm_skip_emulated_instruction);
@@ -9641,7 +9635,7 @@ int x86_emulate_instruction(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa,kvm_pmu_branch_retired (vcpu);kvm_rip_write (vcpu, ctxt->eip);if (r && (ctxt->tf || (vcpu->guest_debug &KVM_GUESTDBG_SINGLESTEP ))) - r =kvm_vcpu_do_singlestep (vcpu); + r =kvm_inject_emulated_db (vcpu,DR6_BS );kvm_x86_call (update_emulated_instruction)(vcpu);__kvm_set_rflags (vcpu, ctxt->eflags); }
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Re: [PATCH v3 07/10] KVM: x86: Drop kvm_vcpu_do_singlestep() now that it's been gutted
Quoted 6 lines · Sean Christopherson
Now that all of kvm_vcpu_do_singlestep()'s previously-unique functionality has been moved into
kvm_inject_emulated_db(), drop the one-line wrapper.No functional change intended.
Signed-off-by Sean Christopherson <[email protected]>
Sean Christopherson
maintainerPATCH v3 8/10 KVM: selftests: Add all (known) EFLAGS bit definitions
In reply to Sean ChristophersonAdd #defines for all known EFLAGS bit, e.g. so that tests can use things like EFLAGS.TF to validate single-stepping behavior. Opportunistically use X86_EFLAGS_FIXED instead of an open-coded equivalent when stuffing initial vCPU state.
No functional change intended.
--- .../selftests/kvm/include/x86/processor.h | 19 ++++++++++++++++++- .../testing/selftests/kvm/lib/x86/processor.c | 2 +- tools/testing/selftests/kvm/lib/x86/vmx.c | 2 +- 3 files changed, 20 insertions(+), 3 deletions(-)
tools/testing/selftests/kvm/include/x86/processor.h +18 -1
@@ -38,7 +38,24 @@ extern u64 guest_tsc_khz;const char *ex_str (int vector); -#define X86_EFLAGS_FIXED (1u << 1) +#define X86_EFLAGS_CF BIT(0)/* Carry Flag */ +#define X86_EFLAGS_FIXED BIT(1)/* Bit 1 - always on */ +#define X86_EFLAGS_PF BIT(2)/* Parity Flag */ +#define X86_EFLAGS_AF BIT(4)/* Auxiliary carry Flag */ +#define X86_EFLAGS_ZF BIT(6)/* Zero Flag */ +#define X86_EFLAGS_SF BIT(7)/* Sign Flag */ +#define X86_EFLAGS_TF BIT(8)/* Trap Flag */ +#define X86_EFLAGS_IF BIT(9)/* Interrupt Flag */ +#define X86_EFLAGS_DF BIT(10)/* Direction Flag */ +#define X86_EFLAGS_OF BIT(11)/* Overflow Flag */ +#define X86_EFLAGS_IOPL BIT(12)/* I/O Privilege Level (2 bits) */ +#define X86_EFLAGS_NT BIT(14)/* Nested Task */ +#define X86_EFLAGS_RF BIT(16)/* Resume Flag */ +#define X86_EFLAGS_VM BIT(17)/* Virtual Mode */ +#define X86_EFLAGS_AC BIT(18)/* Alignment Check/Access Control */ +#define X86_EFLAGS_VIF BIT(19)/* Virtual Interrupt Flag */ +#define X86_EFLAGS_VIP BIT(20)/* Virtual Interrupt Pending */ +#define X86_EFLAGS_ID BIT(21)/* CPUID detection */ #define X86_CR4_VME (1ul << 0)#define X86_CR4_PVI (1ul << 1)
tools/testing/selftests/kvm/lib/x86/processor.c +1 -1
@@ -848,7 +848,7 @@ struct kvm_vcpu *vm_arch_vcpu_add(struct kvm_vm *vm, u32 vcpu_id)/* Setup guest general purpose registers */ vcpu_regs_get (vcpu, ®s); - regs.rflags = regs.rflags |0x2 ; + regs.rflags = regs.rflags |X86_EFLAGS_FIXED ; regs.rsp = stack_gva;vcpu_regs_set (vcpu, ®s);
tools/testing/selftests/kvm/lib/x86/vmx.c +1 -1
@@ -360,7 +360,7 @@ static inline void init_vmcs_guest_state(void *rip, void *rsp)vmwrite (GUEST_DR7 ,0x400 );vmwrite (GUEST_RSP , (u64 )rsp);vmwrite (GUEST_RIP , (u64 )rip); -vmwrite (GUEST_RFLAGS ,2 ); +vmwrite (GUEST_RFLAGS ,X86_EFLAGS_FIXED );vmwrite (GUEST_PENDING_DBG_EXCEPTIONS ,0 );vmwrite (GUEST_SYSENTER_ESP ,vmreadz (HOST_IA32_SYSENTER_ESP ));vmwrite (GUEST_SYSENTER_EIP ,vmreadz (HOST_IA32_SYSENTER_EIP ));
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Sean Christopherson
maintainerPATCH v3 9/10 KVM: selftests: Verify guest debug DR7.GD checking during instruction emulation
In reply to Sean ChristophersonFrom: Hou Wenlong <[email protected]>
Similar to the global disable test case in x86's debug_regs test, use 'KVM_FEP' to trigger instruction emulation in order to verify the guest debug DR7.GD checking during instruction emulation.
--- tools/testing/selftests/kvm/x86/debug_regs.c | 23 +++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-)
tools/testing/selftests/kvm/x86/debug_regs.c +22 -1
@@ -19,6 +19,7 @@u32 guest_value;extern unsigned char sw_bp, hw_bp, write_data, ss_start, bd_start; +extern unsigned char fep_bd_start;static void guest_code (void ) {
@@ -64,6 +65,10 @@ static void guest_code(void)/* DR6.BD test */ asmvolatile ("bd_start: mov %%dr0, %%rax" : : :"rax" ); + +if (is_forced_emulation_enabled) + asmvolatile (KVM_FEP "fep_bd_start: mov %%dr0, %%rax" : : :"rax" ); +GUEST_DONE (); }
@@ -185,7 +190,7 @@ int main(void) target_dr6); } -/* Finally test global disable */ +/* test global disable */ memset (&debug,0 ,sizeof (debug)); debug.control =KVM_GUESTDBG_ENABLE |KVM_GUESTDBG_USE_HW_BP ; debug.arch.debugreg[7 ] =0x400 |DR7_GD ;
@@ -202,6 +207,22 @@ int main(void) run->debug.arch.pc, target_rip, run->debug.arch.dr6, target_dr6); +/* test global disable in emulation */ +if (is_forced_emulation_enabled) { +/* Skip the 3-bytes "mov dr0" */ +vcpu_skip_insn (vcpu,3 ); +vcpu_run (vcpu); +TEST_ASSERT (run->exit_reason ==KVM_EXIT_DEBUG && + run->debug.arch.exception ==DB_VECTOR && + run->debug.arch.pc ==CAST_TO_RIP (fep_bd_start) && + run->debug.arch.dr6 == target_dr6, +"DR7.GD: exit %d exception %d rip 0x%llx " +"(should be 0x%llx) dr6 0x%llx (should be 0x%llx)" , + run->exit_reason, run->debug.arch.exception, + run->debug.arch.pc, target_rip, run->debug.arch.dr6, + target_dr6); + } +/* Disable all debug controls, run to the end */ memset (&debug,0 ,sizeof (debug));vcpu_guest_debug_set (vcpu, &debug);
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Sean Christopherson
maintainerRe: [PATCH v3 09/10] KVM: selftests: Verify guest debug DR7.GD checking during instruction emulation
Quoted 7 lines · Sean Christopherson
tools/testing/selftests/kvm/x86/debug_regs.c +3
-0@@ -202,6 +207,22 @@ int main(void) run->debug.arch.pc, target_rip, run->debug.arch.dr6, target_dr6); +/* test global disable in emulation */ +if (is_forced_emulation_enabled) { +/* Skip the 3-bytes "mov dr0" */
tools/testing/selftests/kvm/x86/debug_regs.c int main(void) · line 222
Quoted 10 lines
tools/testing/selftests/kvm/x86/debug_regs.c +10
-0+vcpu_skip_insn (vcpu,3 ); +vcpu_run (vcpu); + TEST_ASSERT(run->exit_reason == KVM_EXIT_DEBUG && + run->debug.arch.exception == DB_VECTOR && + run->debug.arch.pc == CAST_TO_RIP(fep_bd_start) && + run->debug.arch.dr6 == target_dr6, + "DR7.GD: exit %d exception %d rip 0x%llx " + "(should be 0x%llx) dr6 0x%llx (should be 0x%llx)", + run->exit_reason, run->debug.arch.exception, + run->debug.arch.pc, target_rip, run->debug.arch.dr6,
Per Sashiko[*], this should print CAST_TO_RIP(fep_bd_start), not target_rip. I'll fixup when applying.
[*] https://sashiko.dev/#/patchset/20260515222638.1949982-1-seanjc%40google.com
Quoted 9 lines
tools/testing/selftests/kvm/x86/debug_regs.c +3
-0+ target_dr6); + } +/* Disable all debug controls, run to the end */ memset (&debug,0 ,sizeof (debug));vcpu_guest_debug_set (vcpu, &debug);Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Sean Christopherson
maintainerPATCH v3 10/10 KVM: selftests: Verify VMX's GUEST_PENDING_DBG_EXCEPTIONS.BS Consistency Check
In reply to Sean ChristophersonFrom: Hou Wenlong <[email protected]>
In x86's debug_regs test, add a test case to cover the scenario where a single-step #DB occurs in an STI-shadow, in which case KVM needs to stuff vmcs.GUEST_PENDING_DBG_EXCEPTIONS.BS in order to satisfy a flawed VM-Entry Consistency Check.
Wire up an IRQ handler to gain a bit of bonus coverage, as the subsequent IRET from the #DB sets RFLAGS.IF, but *without* STI-blocking, and so the pending IRQ is expected on the instruction immediately following STI.
--- tools/testing/selftests/kvm/x86/debug_regs.c | 64 ++++++++++++++++++-- 1 file changed, 60 insertions(+), 4 deletions(-)
tools/testing/selftests/kvm/x86/debug_regs.c +60 -4
@@ -15,11 +15,46 @@#define IRQ_VECTOR 0xAA +#define CAST_TO_RIP (v) ((unsigned long long)&(v)) +/* For testing data access debug BP */ u32 guest_value;extern unsigned char sw_bp, hw_bp, write_data, ss_start, bd_start; -extern unsigned char fep_bd_start; +extern unsigned char fep_bd_start, fep_sti_start, fep_sti_end; + +static void guest_db_handler (struct ex_regs *regs) +{ +static int count; +unsigned long target_rips[2 ] = { +CAST_TO_RIP (fep_sti_start), +CAST_TO_RIP (fep_sti_end), + }; + +__GUEST_ASSERT (regs->rip == target_rips[count], +"STI[%u]: unexpected rip 0x%lx (should be 0x%lx)" , + count, regs->rip, target_rips[count]); + regs->rflags &= ~X86_EFLAGS_TF ; + count++; +} + +static void guest_irq_handler (struct ex_regs *regs) +{ +/* +* The pending IRQ should finally be take when KVM_GUESTDBG_BLOCKIRQ is +* cleared and IRQs are enabled. Note, the IRQ is expected to arrive +* on the instruction immediately after STI, even though its in an STI +* shadow. Because the next instruction has a coincident #DB, and #DBs +* are not subject to STI-blocking, the #DB will push RFLAGS.IF=1 on +* the stack, and the eventual IRET will unmask IRQs and obliterate the +* STI shadow in the process. +*/ +unsigned long target_rip =CAST_TO_RIP (fep_sti_start); + +__GUEST_ASSERT (regs->rip == target_rip, +"IRQ: unexpected rip 0x%lx (should be 0x%lx)" , + regs->rip, target_rip); +}static void guest_code (void ) {
@@ -66,14 +101,32 @@ static void guest_code(void)/* DR6.BD test */ asmvolatile ("bd_start: mov %%dr0, %%rax" : : :"rax" ); -if (is_forced_emulation_enabled) +/* +* Note, the IRET from the #DB that occurs in the below STI-shadow will +* unmask IRQs, i.e. the pending interrupt will be delivered after #DB +* handling, on the CLI! +*/ +if (is_forced_emulation_enabled) { asmvolatile (KVM_FEP "fep_bd_start: mov %%dr0, %%rax" : : :"rax" ); +/* pending debug exceptions for emulation */ + asmvolatile ("pushf\n\t" + "orq $" __stringify(X86_EFLAGS_TF) ", (%rsp)\n\t" +"popf\n\t" +"sti\n\t" +"fep_sti_start:" +"cli\n\t" +"pushf\n\t" + "orq $" __stringify(X86_EFLAGS_TF) ", (%rsp)\n\t" +"popf\n\t" +KVM_FEP "sti\n\t" +"fep_sti_end:" + "cli\n\t"); + } +GUEST_DONE (); } -#define CAST_TO_RIP (v) ((unsigned long long)&(v)) -static void vcpu_skip_insn (struct kvm_vcpu *vcpu,int insn_len) {struct kvm_regs regs;
@@ -227,6 +280,9 @@ int main(void)memset (&debug,0 ,sizeof (debug));vcpu_guest_debug_set (vcpu, &debug); +vm_install_exception_handler (vm,DB_VECTOR , guest_db_handler); +vm_install_exception_handler (vm,IRQ_VECTOR , guest_irq_handler); +vcpu_run (vcpu);TEST_ASSERT_KVM_EXIT_REASON (vcpu,KVM_EXIT_IO ); cmd =get_ucall (vcpu, &uc);
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Sean Christopherson
maintainerRe: [PATCH v3 10/10] KVM: selftests: Verify VMX's GUEST_PENDING_DBG_EXCEPTIONS.BS Consistency Check
Quoted 56 lines · Sean Christopherson
From: Hou Wenlong <[email protected]>
In x86's
debug_regstest, add a test case to cover the scenario where a single-step #DB occurs in an STI-shadow, in which case KVM needs to stuff vmcs.GUEST_PENDING_DBG_EXCEPTIONS.BS in order to satisfy a flawed VM-Entry Consistency Check.Wire up an IRQ handler to gain a bit of bonus coverage, as the subsequent IRET from the #DB sets RFLAGS.IF, but *without* STI-blocking, and so the pending IRQ is expected on the instruction immediately following STI.
Signed-off-by Hou Wenlong <[email protected]>[sean: expect the IRQ on the CLI, and explain why]Signed-off-by Sean Christopherson <[email protected]>--- tools/testing/selftests/kvm/x86/debug_regs.c | 64 ++++++++++++++++++-- 1 file changed, 60 insertions(+), 4 deletions(-)tools/testing/selftests/kvm/x86/debug_regs.c +25
-1@@ -15,11 +15,46 @@#define IRQ_VECTOR 0xAA +#define CAST_TO_RIP (v) ((unsigned long long)&(v)) +/* For testing data access debug BP */ u32 guest_value;extern unsigned char sw_bp, hw_bp, write_data, ss_start, bd_start; -extern unsigned char fep_bd_start; +extern unsigned char fep_bd_start, fep_sti_start, fep_sti_end; + +static void guest_db_handler (struct ex_regs *regs) +{ +static int count; +unsigned long target_rips[2 ] = { +CAST_TO_RIP (fep_sti_start), +CAST_TO_RIP (fep_sti_end), + }; + +__GUEST_ASSERT (regs->rip == target_rips[count], +"STI[%u]: unexpected rip 0x%lx (should be 0x%lx)" , + count, regs->rip, target_rips[count]); + regs->rflags &= ~X86_EFLAGS_TF ; + count++; +} + +static void guest_irq_handler (struct ex_regs *regs) +{ +/* +* The pending IRQ should finally be take when KVM_GUESTDBG_BLOCKIRQ is +* cleared and IRQs are enabled. Note, the IRQ is expected to arrive +* on the instruction immediately after STI, even though its in an STI
tools/testing/selftests/kvm/x86/debug_regs.c line 56
Quoted 10 lines
tools/testing/selftests/kvm/x86/debug_regs.c +10
-0+* shadow. Because the next instruction has a coincident #DB, and #DBs +* are not subject to STI-blocking, the #DB will push RFLAGS.IF=1 on +* the stack, and the eventual IRET will unmask IRQs and obliterate the +* STI shadow in the process. +*/ +unsigned long target_rip =CAST_TO_RIP (fep_sti_start); + +__GUEST_ASSERT (regs->rip == target_rip, +"IRQ: unexpected rip 0x%lx (should be 0x%lx)" , + regs->rip, target_rip);
From Sashiko:
: Is an End of Interrupt (EOI) acknowledgment required here?
:
: The handler is triggered by an APIC interrupt injected earlier in the test, : but does not write to the APIC_EOI register to acknowledge it. :
: While it might not fail this specific test since it ends immediately after, : does failing to send an EOI leave the interrupt permanently marked as : in-service in the local APIC state machine?
Yes, this will leave the IRQ dangling in the ISR. It doesn't really matter, but I'll throw in a:
x2apic_write_reg(APIC_EOI, 0);
when applying.
Quoted 22 lines
tools/testing/selftests/kvm/x86/debug_regs.c +12
-1+}static void guest_code (void ) {@@ -66,14 +101,32 @@ static void guest_code(void)/* DR6.BD test */ asmvolatile ("bd_start: mov %%dr0, %%rax" : : :"rax" ); -if (is_forced_emulation_enabled) +/* +* Note, the IRET from the #DB that occurs in the below STI-shadow will +* unmask IRQs, i.e. the pending interrupt will be delivered after #DB +* handling, on the CLI! +*/ +if (is_forced_emulation_enabled) { asmvolatile (KVM_FEP "fep_bd_start: mov %%dr0, %%rax" : : :"rax" ); +/* pending debug exceptions for emulation */ + asmvolatile ("pushf\n\t" + "orq $" __stringify(X86_EFLAGS_TF) ", (%rsp)\n\t" +"popf\n\t" +"sti\n\t"
tools/testing/selftests/kvm/x86/debug_regs.c static void guest_code(void) · line 126
Quoted 10 lines
tools/testing/selftests/kvm/x86/debug_regs.c +10
-0+"fep_sti_start:" +"cli\n\t" +"pushf\n\t" + "orq $" __stringify(X86_EFLAGS_TF) ", (%rsp)\n\t" +"popf\n\t" +KVM_FEP "sti\n\t" +"fep_sti_end:" + "cli\n\t"); + } +
Also from Sashiko:
: Does the test verify that the expected #DB and IRQ events actually occurred? :
: The assertions are placed inside guest_db_handler() and guest_irq_handler(), : but if a bug causes the exceptions to be dropped completely, the guest : will simply proceed to execute GUEST_DONE() without error.
:
: Should there be a shared state variable to assert that the handlers fired : the expected number of times before concluding the test?
Another "yes". I'll add an assert that the IRQ actually fired.
Quoted 11 lines · Sean Christopherson
Hou's series of fixes and cleanups related to #DB handling in the emulator.
v3:
- Fix the GUEST_PENDING_DBG_EXCEPTIONS.BS consistency check issue by stuffing the VMCS during injection, not during emulation.
- Drop a prep that is no longer necessary.
- Massage a few shortlogs/changelogs.
- Fix the selftest that takes an IRQ in the STI-shadow after IRET.
- Fix an intermediate bug wherekvm_queue_exception_e()was being used instead ofkvm_queue_exception_p().
Quoted 1 line
[...]
Applied to kvm-x86 misc, thanks!
[01/10] KVM: VMX: Refresh GUEST_PENDING_DBG_EXCEPTIONS.BS on all injected #DBs
https://github.com/kvm-x86/linux/commit/c5bad4fa2d5d [02/10] KVM: x86: Capture "struct x86_exception" in inject_emulated_exception() https://github.com/kvm-x86/linux/commit/17afe9b750f2 [03/10] KVM: x86: Set guest DR6 by kvm_queue_exception_p() in instruction emulation https://github.com/kvm-x86/linux/commit/dfc1c4687e9c [04/10] KVM: x86: Honor KVM_GUESTDBG_USE_HW_BP when emulating MOV DR (in emulator) https://github.com/kvm-x86/linux/commit/51543660c521 [05/10] KVM: x86: Honor KVM_GUESTDBG_USE_HW_BP when checking for code breakpoints in emulation https://github.com/kvm-x86/linux/commit/00b6520669bc [06/10] KVM: x86: Move KVM_GUESTDBG_SINGLESTEP handling into kvm_inject_emulated_db() https://github.com/kvm-x86/linux/commit/6d79302b0834 [07/10] KVM: x86: Drop kvm_vcpu_do_singlestep() now that it's been gutted https://github.com/kvm-x86/linux/commit/a1e7aeb1be11 [08/10] KVM: selftests: Add all (known) EFLAGS bit definitions https://github.com/kvm-x86/linux/commit/35c08038cbdc [09/10] KVM: selftests: Verify guest debug DR7.GD checking during instruction emulation https://github.com/kvm-x86/linux/commit/063a22451fd9 [10/10] KVM: selftests: Verify VMX's GUEST_PENDING_DBG_EXCEPTIONS.BS Consistency Check https://github.com/kvm-x86/linux/commit/b8284279b732