KVM: x86: Remove emulation_result enums
v2 cover KVM: x86: Remove emulation_result enums
Rework the emulator and its users to handle failure scenarios entirely within the emulator.
{x86,kvm}_emulate_instruction() currently returns a tri-state value to indicate success/continue, userspace exit needed, and failure. The intent of returning EMULATE_FAIL is to let the caller handle failure in a manner that is appropriate for the current context. In practice, the emulator has ended up with a mixture of failure handling, i.e. whether or not the emulator takes action on failure is dependent on the specific flavor of emulation.
The mixed handling has proven to be rather fragile, e.g. many flows incorrectly assume their specific flavor of emulation cannot fail or that the emulator sets state to report the failure back to userspace.
Move everything inside the emulator, piece by piece, so that the emulation routines can return '0' for exit to userspace and '1' for resume the guest, just like every other VM-Exit handler.
Patch 13/14 is a tangentially related bug fix that conflicts heavily with this series, so I tacked it on here.
Patch 14/14 documents the emulation types. I added it as a separate patch at the very end so that the comments could reference the final state of the code base, e.g. incorporate the rule change for using EMULTYPE_SKIP that is introduced in patch 13/14.
v1:
- https://patchwork.kernel.org/cover/11110331/
v2:
- Collect reviews. [Vitaly and Liran]
- Squash VMware emultype changes into a single patch. [Liran]
- Add comments in VMX/SVM for VMware #GP handling. [Vitaly]
- Tack on the EPT misconfig bug fix.
- Add a patch to comment/document the emultypes. [Liran]
Sean Christopherson (14):
KVM: x86: Relocate MMIO exit stats counting
KVM: x86: Clean up handle_emulation_failure()
KVM: x86: Refactor kvm_vcpu_do_singlestep() to remove out param
KVM: x86: Don't attempt VMWare emulation on #GP with non-zero error
code
KVM: x86: Move #GP injection for VMware into x86_emulate_instruction()
KVM: x86: Add explicit flag for forced emulation on #UD
KVM: x86: Move #UD injection for failed emulation into emulation code
KVM: x86: Exit to userspace on emulation skip failure
KVM: x86: Handle emulation failure directly in kvm_task_switch()
KVM: x86: Move triple fault request into RM int injection
KVM: VMX: Remove EMULATE_FAIL handling in handle_invalid_guest_state()
KVM: x86: Remove emulation_result enums, EMULATE_{DONE,FAIL,USER_EXIT}
KVM: VMX: Handle single-step #DB for EMULTYPE_SKIP on EPT misconfig
KVM: x86: Add comments to document various emulation types
arch/x86/include/asm/kvm_host.h | 40 +++++++-- arch/x86/kvm/mmu.c | 16 +--- arch/x86/kvm/svm.c | 62 ++++++-------- arch/x86/kvm/vmx/vmx.c | 147 +++++++++++++------------------- arch/x86/kvm/x86.c | 133 ++++++++++++++++------------- arch/x86/kvm/x86.h | 2 +- 6 files changed, 195 insertions(+), 205 deletions(-)
Signature · 1 line
2.22.0
PATCH v2 1/14 KVM: x86: Relocate MMIO exit stats counting
Move the stat.mmio_exits update into x86_emulate_instruction(). This is both a bug fix, e.g. the current update flows will incorrectly increment mmio_exits on emulation failure, and a preparatory change to set the stage for eliminating EMULATE_DONE and company.
--- arch/x86/kvm/mmu.c | 2 -- arch/x86/kvm/vmx/vmx.c | 1 - arch/x86/kvm/x86.c | 2 ++ 3 files changed, 2 insertions(+), 3 deletions(-)
arch/x86/kvm/mmu.c +0 -2
@@ -5437,8 +5437,6 @@ int kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gva_t cr2, u64 error_code,case EMULATE_DONE :return 1 ;case EMULATE_USER_EXIT : - ++vcpu->stat.mmio_exits; -/* fall through */ case EMULATE_FAIL :return 0 ;default :
arch/x86/kvm/vmx/vmx.c +0 -1
@@ -5200,7 +5200,6 @@ static int handle_invalid_guest_state(struct kvm_vcpu *vcpu) err =kvm_emulate_instruction (vcpu,0 );if (err ==EMULATE_USER_EXIT ) { - ++vcpu->stat.mmio_exits; ret =0 ; goto out; }
arch/x86/kvm/x86.c +2 -0
@@ -6598,6 +6598,8 @@ int x86_emulate_instruction(struct kvm_vcpu *vcpu, } r =EMULATE_USER_EXIT ; }else if (vcpu->mmio_needed) { + ++vcpu->stat.mmio_exits; +if (!vcpu->mmio_is_write) writeback = false; r =EMULATE_USER_EXIT ;
Signature · 1 line
2.22.0
PATCH v2 2/14 KVM: x86: Clean up handle_emulation_failure()
In reply to Sean ChristophersonWhen handling emulation failure, return the emulation result directly instead of capturing it in a local variable. Future patches will move additional cases into handle_emulation_failure(), clean up the cruft before so there isn't an ugly mix of setting a local variable and returning directly.
--- arch/x86/kvm/x86.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-)
arch/x86/kvm/x86.c +4 -6
@@ -6207,24 +6207,22 @@ EXPORT_SYMBOL_GPL(kvm_inject_realmode_interrupt);static int handle_emulation_failure (struct kvm_vcpu *vcpu,int emulation_type) { -int r =EMULATE_DONE ; - ++vcpu->stat.insn_emulation_fail;trace_kvm_emulate_insn_failed (vcpu);if (emulation_type &EMULTYPE_NO_UD_ON_FAIL )return EMULATE_FAIL ; +kvm_queue_exception (vcpu,UD_VECTOR ); +if (!is_guest_mode (vcpu) && kvm_x86_ops->get_cpl (vcpu) ==0 ) { vcpu->run->exit_reason =KVM_EXIT_INTERNAL_ERROR ; vcpu->run->internal.suberror =KVM_INTERNAL_ERROR_EMULATION ; vcpu->run->internal.ndata =0 ; - r =EMULATE_USER_EXIT ; +return EMULATE_USER_EXIT ; } -kvm_queue_exception (vcpu,UD_VECTOR ); - -return r; +return EMULATE_DONE ; } static bool reexecute_instruction(struct kvm_vcpu *vcpu, gva_t cr2,
Signature · 1 line
2.22.0
PATCH v2 3/14 KVM: x86: Refactor kvm_vcpu_do_singlestep() to remove out param
In reply to Sean ChristophersonReturn the single-step emulation result directly instead of via an out param. Presumably at some point in the past kvm_vcpu_do_singlestep() could be called with *r==EMULATE_USER_EXIT, but that is no longer the case, i.e. all callers are happy to overwrite their own return variable.
--- arch/x86/kvm/x86.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)
arch/x86/kvm/x86.c +6 -6
@@ -6377,7 +6377,7 @@ static int kvm_vcpu_check_hw_bp(unsigned long addr, u32 type, u32 dr7,return dr6; } -static void kvm_vcpu_do_singlestep (struct kvm_vcpu *vcpu,int *r) +static int kvm_vcpu_do_singlestep (struct kvm_vcpu *vcpu) {struct kvm_run *kvm_run = vcpu->run;
@@ -6386,10 +6386,10 @@ static void kvm_vcpu_do_singlestep(struct kvm_vcpu *vcpu, int *r) kvm_run->debug.arch.pc = vcpu->arch.singlestep_rip; kvm_run->debug.arch.exception =DB_VECTOR ; kvm_run->exit_reason =KVM_EXIT_DEBUG ; - *r =EMULATE_USER_EXIT ; - }else { -kvm_queue_exception_p (vcpu,DB_VECTOR ,DR6_BS ); +return EMULATE_USER_EXIT ; } +kvm_queue_exception_p (vcpu,DB_VECTOR ,DR6_BS ); +return EMULATE_DONE ; }int kvm_skip_emulated_instruction (struct kvm_vcpu *vcpu)
@@ -6410,7 +6410,7 @@ int kvm_skip_emulated_instruction(struct kvm_vcpu *vcpu)* that sets the TF flag". */ if (unlikely (rflags &X86_EFLAGS_TF )) -kvm_vcpu_do_singlestep (vcpu, &r); + r =kvm_vcpu_do_singlestep (vcpu);return r ==EMULATE_DONE ; }EXPORT_SYMBOL_GPL (kvm_skip_emulated_instruction);
@@ -6613,7 +6613,7 @@ int x86_emulate_instruction(struct kvm_vcpu *vcpu, vcpu->arch.emulate_regs_need_sync_to_vcpu = false;kvm_rip_write (vcpu, ctxt->eip);if (r ==EMULATE_DONE && ctxt->tf) -kvm_vcpu_do_singlestep (vcpu, &r); + r =kvm_vcpu_do_singlestep (vcpu);if (!ctxt->have_exception ||exception_type (ctxt->exception.vector) ==EXCPT_TRAP )__kvm_set_rflags (vcpu, ctxt->eflags);
Signature · 1 line
2.22.0
Paolo Bonzini
maintainerRe: [PATCH v2 03/14] KVM: x86: Refactor kvm_vcpu_do_singlestep() to remove out param
Quoted 4 lines · Sean Christopherson
Return the single-step emulation result directly instead of via an out param. Presumably at some point in the past
kvm_vcpu_do_singlestep()could be called with *r==EMULATE_USER_EXIT, but that is no longer the case, i.e. all callers are happy to overwrite their own return variable.
It was actually done for consistency with kvm_vcpu_check_breakpoint. It's okay to change it.
Paolo
Quoted 53 lines
Reviewed-by Vitaly Kuznetsov <[email protected]>Reviewed-by Liran Alon <[email protected]>Signed-off-by Sean Christopherson <[email protected]>--- arch/x86/kvm/x86.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)arch/x86/kvm/x86.c +6
-6@@ -6377,7 +6377,7 @@ static int kvm_vcpu_check_hw_bp(unsigned long addr, u32 type, u32 dr7,return dr6; } -static void kvm_vcpu_do_singlestep (struct kvm_vcpu *vcpu,int *r) +static int kvm_vcpu_do_singlestep (struct kvm_vcpu *vcpu) {struct kvm_run *kvm_run = vcpu->run;@@ -6386,10 +6386,10 @@ static void kvm_vcpu_do_singlestep(struct kvm_vcpu *vcpu, int *r) kvm_run->debug.arch.pc = vcpu->arch.singlestep_rip; kvm_run->debug.arch.exception =DB_VECTOR ; kvm_run->exit_reason =KVM_EXIT_DEBUG ; - *r =EMULATE_USER_EXIT ; - }else { -kvm_queue_exception_p (vcpu,DB_VECTOR ,DR6_BS ); +return EMULATE_USER_EXIT ; } +kvm_queue_exception_p (vcpu,DB_VECTOR ,DR6_BS ); +return EMULATE_DONE ; }int kvm_skip_emulated_instruction (struct kvm_vcpu *vcpu)@@ -6410,7 +6410,7 @@ int kvm_skip_emulated_instruction(struct kvm_vcpu *vcpu)* that sets the TF flag". */ if (unlikely (rflags &X86_EFLAGS_TF )) -kvm_vcpu_do_singlestep (vcpu, &r); + r =kvm_vcpu_do_singlestep (vcpu);return r ==EMULATE_DONE ; }EXPORT_SYMBOL_GPL (kvm_skip_emulated_instruction);@@ -6613,7 +6613,7 @@ int x86_emulate_instruction(struct kvm_vcpu *vcpu, vcpu->arch.emulate_regs_need_sync_to_vcpu = false;kvm_rip_write (vcpu, ctxt->eip);if (r ==EMULATE_DONE && ctxt->tf) -kvm_vcpu_do_singlestep (vcpu, &r); + r =kvm_vcpu_do_singlestep (vcpu);if (!ctxt->have_exception ||exception_type (ctxt->exception.vector) ==EXCPT_TRAP )__kvm_set_rflags (vcpu, ctxt->eflags);
PATCH v2 4/14 KVM: x86: Don't attempt VMWare emulation on #GP with non-zero error code
In reply to Sean ChristophersonThe VMware backdoor hooks #GP faults on IN{S}, OUT{S}, and RDPMC, none of which generate a non-zero error code for their #GP. Re-injecting #GP instead of attempting emulation on a non-zero error code will allow a future patch to move #GP injection (for emulation failure) into kvm_emulate_instruction() without having to plumb in the error code.
--- arch/x86/kvm/svm.c | 10 +++++++++- arch/x86/kvm/vmx/vmx.c | 12 +++++++++++- 2 files changed, 20 insertions(+), 2 deletions(-)
arch/x86/kvm/svm.c +9 -1
@@ -2772,12 +2772,20 @@ static int gp_interception(struct vcpu_svm *svm)WARN_ON_ONCE (!enable_vmware_backdoor); +/* +* VMware backdoor emulation on #GP interception only handles IN{S}, +* OUT{S}, and RDPMC, none of which generate a non-zero error code. +*/ +if (error_code) { +kvm_queue_exception_e (vcpu,GP_VECTOR , error_code); +return 1 ; + } er =kvm_emulate_instruction (vcpu,EMULTYPE_VMWARE |EMULTYPE_NO_UD_ON_FAIL );if (er ==EMULATE_USER_EXIT )return 0 ;else if (er !=EMULATE_DONE ) -kvm_queue_exception_e (vcpu,GP_VECTOR , error_code); +kvm_queue_exception_e (vcpu,GP_VECTOR ,0 );return 1 ; }
arch/x86/kvm/vmx/vmx.c +11 -1
@@ -4509,12 +4509,22 @@ static int handle_exception_nmi(struct kvm_vcpu *vcpu)if (!vmx->rmode.vm86_active &&is_gp_fault (intr_info)) {WARN_ON_ONCE (!enable_vmware_backdoor); + +/* +* VMware backdoor emulation on #GP interception only handles +* IN{S}, OUT{S}, and RDPMC, none of which generate a non-zero +* error code on #GP. +*/ +if (error_code) { +kvm_queue_exception_e (vcpu,GP_VECTOR , error_code); +return 1 ; + } er =kvm_emulate_instruction (vcpu,EMULTYPE_VMWARE |EMULTYPE_NO_UD_ON_FAIL );if (er ==EMULATE_USER_EXIT )return 0 ;else if (er !=EMULATE_DONE ) -kvm_queue_exception_e (vcpu,GP_VECTOR , error_code); +kvm_queue_exception_e (vcpu,GP_VECTOR ,0 );return 1 ; }
Signature · 1 line
2.22.0
PATCH v2 5/14 KVM: x86: Move #GP injection for VMware into x86_emulate_instruction()
In reply to Sean ChristophersonImmediately inject a #GP when VMware emulation fails and return EMULATE_DONE instead of propagating EMULATE_FAIL up the stack. This helps pave the way for removing EMULATE_FAIL altogether.
Rename EMULTYPE_VMWARE to EMULTYPE_VMWARE_GP to document that the x86 emulator is called to handle VMware #GP interception, e.g. why a #GP is injected on emulation failure for EMULTYPE_VMWARE_GP.
Drop EMULTYPE_NO_UD_ON_FAIL as a standalone type. The "no #UD on fail" is used only in the VMWare case and is obsoleted by having the emulator itself reinject #GP.
--- arch/x86/include/asm/kvm_host.h | 3 +-- arch/x86/kvm/svm.c | 10 ++-------- arch/x86/kvm/vmx/vmx.c | 10 ++-------- arch/x86/kvm/x86.c | 14 +++++++++----- 4 files changed, 14 insertions(+), 23 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -2
@@ -1318,8 +1318,7 @@ enum emulation_result {#define EMULTYPE_TRAP_UD (1 << 1)#define EMULTYPE_SKIP (1 << 2)#define EMULTYPE_ALLOW_RETRY (1 << 3) -#define EMULTYPE_NO_UD_ON_FAIL (1 << 4) -#define EMULTYPE_VMWARE (1 << 5) +#define EMULTYPE_VMWARE_GP (1 << 5)int kvm_emulate_instruction (struct kvm_vcpu *vcpu,int emulation_type);int kvm_emulate_instruction_from_buffer (struct kvm_vcpu *vcpu,void *insn,int insn_len);
arch/x86/kvm/svm.c +2 -8
@@ -2768,7 +2768,6 @@ static int gp_interception(struct vcpu_svm *svm) {struct kvm_vcpu *vcpu = &svm->vcpu;u32 error_code = svm->vmcb->control.exit_info_1; -int er;WARN_ON_ONCE (!enable_vmware_backdoor);
@@ -2780,13 +2779,8 @@ static int gp_interception(struct vcpu_svm *svm)kvm_queue_exception_e (vcpu,GP_VECTOR , error_code);return 1 ; } - er =kvm_emulate_instruction (vcpu, -EMULTYPE_VMWARE |EMULTYPE_NO_UD_ON_FAIL ); -if (er ==EMULATE_USER_EXIT ) -return 0 ; -else if (er !=EMULATE_DONE ) -kvm_queue_exception_e (vcpu,GP_VECTOR ,0 ); -return 1 ; +return kvm_emulate_instruction (vcpu,EMULTYPE_VMWARE_GP ) != +EMULATE_USER_EXIT ; }static bool is_erratum_383 (void )
arch/x86/kvm/vmx/vmx.c +2 -8
@@ -4492,7 +4492,6 @@ static int handle_exception_nmi(struct kvm_vcpu *vcpu)u32 intr_info, ex_no, error_code;unsigned long cr2, rip, dr6;u32 vect_info; -enum emulation_result er; vect_info = vmx->idt_vectoring_info; intr_info = vmx->exit_intr_info;
@@ -4519,13 +4518,8 @@ static int handle_exception_nmi(struct kvm_vcpu *vcpu)kvm_queue_exception_e (vcpu,GP_VECTOR , error_code);return 1 ; } - er =kvm_emulate_instruction (vcpu, -EMULTYPE_VMWARE |EMULTYPE_NO_UD_ON_FAIL ); -if (er ==EMULATE_USER_EXIT ) -return 0 ; -else if (er !=EMULATE_DONE ) -kvm_queue_exception_e (vcpu,GP_VECTOR ,0 ); -return 1 ; +return kvm_emulate_instruction (vcpu,EMULTYPE_VMWARE_GP ) != +EMULATE_USER_EXIT ; }/*
arch/x86/kvm/x86.c +9 -5
@@ -6210,8 +6210,10 @@ static int handle_emulation_failure(struct kvm_vcpu *vcpu, int emulation_type) ++vcpu->stat.insn_emulation_fail;trace_kvm_emulate_insn_failed (vcpu); -if (emulation_type &EMULTYPE_NO_UD_ON_FAIL ) -return EMULATE_FAIL ; +if (emulation_type &EMULTYPE_VMWARE_GP ) { +kvm_queue_exception_e (vcpu,GP_VECTOR ,0 ); +return EMULATE_DONE ; + }kvm_queue_exception (vcpu,UD_VECTOR );
@@ -6543,9 +6545,11 @@ int x86_emulate_instruction(struct kvm_vcpu *vcpu, } } -if ((emulation_type &EMULTYPE_VMWARE ) && - !is_vmware_backdoor_opcode (ctxt)) -return EMULATE_FAIL ; +if ((emulation_type &EMULTYPE_VMWARE_GP ) && + !is_vmware_backdoor_opcode (ctxt)) { +kvm_queue_exception_e (vcpu,GP_VECTOR ,0 ); +return EMULATE_DONE ; + }if (emulation_type &EMULTYPE_SKIP ) {kvm_rip_write (vcpu, ctxt->_eip);
Signature · 1 line
2.22.0
Re: [PATCH v2 05/14] KVM: x86: Move #GP injection for VMware into x86_emulate_instruction()
Quoted 11 lines · Sean Christopherson
Immediately inject a #GP when VMware emulation fails and return
EMULATE_DONEinstead of propagatingEMULATE_FAILup the stack. This helps pave the way for removingEMULATE_FAILaltogether.Rename
EMULTYPE_VMWAREtoEMULTYPE_VMWARE_GPto document that the x86 emulator is called to handle VMware #GP interception, e.g. why a #GP is injected on emulation failure forEMULTYPE_VMWARE_GP.
Quoted 5 lines
Drop
EMULTYPE_NO_UD_ON_FAILas a standalone type. The "no #UD on fail" is used only in the VMWare case and is obsoleted by having the emulator itself reinject #GP.Signed-off-by Sean Christopherson <[email protected]>
Quoted 112 lines
---
arch/x86/include/asm/kvm_host.h | 3 +-- arch/x86/kvm/svm.c | 10 ++-------- arch/x86/kvm/vmx/vmx.c | 10 ++-------- arch/x86/kvm/x86.c | 14 +++++++++----- 4 files changed, 14 insertions(+), 23 deletions(-)arch/x86/include/asm/kvm_host.h +1
-2@@ -1318,8 +1318,7 @@ enum emulation_result {#define EMULTYPE_TRAP_UD (1 << 1)#define EMULTYPE_SKIP (1 << 2)#define EMULTYPE_ALLOW_RETRY (1 << 3) -#define EMULTYPE_NO_UD_ON_FAIL (1 << 4) -#define EMULTYPE_VMWARE (1 << 5) +#define EMULTYPE_VMWARE_GP (1 << 5)int kvm_emulate_instruction (struct kvm_vcpu *vcpu,int emulation_type);int kvm_emulate_instruction_from_buffer (struct kvm_vcpu *vcpu,void *insn,int insn_len);arch/x86/kvm/svm.c +2
-8@@ -2768,7 +2768,6 @@ static int gp_interception(struct vcpu_svm *svm) {struct kvm_vcpu *vcpu = &svm->vcpu;u32 error_code = svm->vmcb->control.exit_info_1; -int er;WARN_ON_ONCE (!enable_vmware_backdoor);@@ -2780,13 +2779,8 @@ static int gp_interception(struct vcpu_svm *svm)kvm_queue_exception_e (vcpu,GP_VECTOR , error_code);return 1 ; } - er =kvm_emulate_instruction (vcpu, -EMULTYPE_VMWARE |EMULTYPE_NO_UD_ON_FAIL ); -if (er ==EMULATE_USER_EXIT ) -return 0 ; -else if (er !=EMULATE_DONE ) -kvm_queue_exception_e (vcpu,GP_VECTOR ,0 ); -return 1 ; +return kvm_emulate_instruction (vcpu,EMULTYPE_VMWARE_GP ) != +EMULATE_USER_EXIT ; }static bool is_erratum_383 (void )arch/x86/kvm/vmx/vmx.c +2
-8@@ -4492,7 +4492,6 @@ static int handle_exception_nmi(struct kvm_vcpu *vcpu)u32 intr_info, ex_no, error_code;unsigned long cr2, rip, dr6;u32 vect_info; -enum emulation_result er; vect_info = vmx->idt_vectoring_info; intr_info = vmx->exit_intr_info;@@ -4519,13 +4518,8 @@ static int handle_exception_nmi(struct kvm_vcpu *vcpu)kvm_queue_exception_e (vcpu,GP_VECTOR , error_code);return 1 ; } - er =kvm_emulate_instruction (vcpu, -EMULTYPE_VMWARE |EMULTYPE_NO_UD_ON_FAIL ); -if (er ==EMULATE_USER_EXIT ) -return 0 ; -else if (er !=EMULATE_DONE ) -kvm_queue_exception_e (vcpu,GP_VECTOR ,0 ); -return 1 ; +return kvm_emulate_instruction (vcpu,EMULTYPE_VMWARE_GP ) != +EMULATE_USER_EXIT ; }/* arch/x86/kvm/x86.c +9
-5@@ -6210,8 +6210,10 @@ static int handle_emulation_failure(struct kvm_vcpu *vcpu, int emulation_type) ++vcpu->stat.insn_emulation_fail;trace_kvm_emulate_insn_failed (vcpu); -if (emulation_type &EMULTYPE_NO_UD_ON_FAIL ) -return EMULATE_FAIL ; +if (emulation_type &EMULTYPE_VMWARE_GP ) { +kvm_queue_exception_e (vcpu,GP_VECTOR ,0 ); +return EMULATE_DONE ; + }kvm_queue_exception (vcpu,UD_VECTOR );@@ -6543,9 +6545,11 @@ int x86_emulate_instruction(struct kvm_vcpu *vcpu, } } -if ((emulation_type &EMULTYPE_VMWARE ) && - !is_vmware_backdoor_opcode (ctxt)) -return EMULATE_FAIL ; +if ((emulation_type &EMULTYPE_VMWARE_GP ) && + !is_vmware_backdoor_opcode (ctxt)) { +kvm_queue_exception_e (vcpu,GP_VECTOR ,0 ); +return EMULATE_DONE ; + }if (emulation_type &EMULTYPE_SKIP ) {kvm_rip_write (vcpu, ctxt->_eip);Signature · 1 line
2.22.0
PATCH v2 6/14 KVM: x86: Add explicit flag for forced emulation on #UD
In reply to Sean ChristophersonAdd an explicit emulation type for forced #UD emulation and use it to detect that KVM should unconditionally inject a #UD instead of falling into its standard emulation failure handling.
--- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/x86.c | 5 +++-- 2 files changed, 4 insertions(+), 2 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -0
@@ -1318,6 +1318,7 @@ enum emulation_result {#define EMULTYPE_TRAP_UD (1 << 1)#define EMULTYPE_SKIP (1 << 2)#define EMULTYPE_ALLOW_RETRY (1 << 3) +#define EMULTYPE_TRAP_UD_FORCED (1 << 4)#define EMULTYPE_VMWARE_GP (1 << 5)int kvm_emulate_instruction (struct kvm_vcpu *vcpu,int emulation_type); int kvm_emulate_instruction_from_buffer(struct kvm_vcpu *vcpu,
arch/x86/kvm/x86.c +3 -2
@@ -5337,7 +5337,7 @@ int handle_ud(struct kvm_vcpu *vcpu) sig, sizeof(sig), &e) == 0 && memcmp(sig, "\xf\xbkvm", sizeof(sig)) == 0) {kvm_rip_write (vcpu,kvm_rip_read (vcpu) +sizeof (sig)); - emul_type =0 ; + emul_type =EMULTYPE_TRAP_UD_FORCED ; } er =kvm_emulate_instruction (vcpu, emul_type);
@@ -6532,7 +6532,8 @@ int x86_emulate_instruction(struct kvm_vcpu *vcpu,trace_kvm_emulate_insn_start (vcpu); ++vcpu->stat.insn_emulation;if (r !=EMULATION_OK ) { -if (emulation_type &EMULTYPE_TRAP_UD ) +if ((emulation_type &EMULTYPE_TRAP_UD ) || + (emulation_type &EMULTYPE_TRAP_UD_FORCED ))return EMULATE_FAIL ;if (reexecute_instruction (vcpu, cr2, write_fault_to_spt, emulation_type))
Signature · 1 line
2.22.0
PATCH v2 7/14 KVM: x86: Move #UD injection for failed emulation into emulation code
In reply to Sean ChristophersonImmediately inject a #UD and return EMULATE done if emulation fails when handling an intercepted #UD. This helps pave the way for removing EMULATE_FAIL altogether.
--- arch/x86/kvm/x86.c | 14 +++++--------- 1 file changed, 5 insertions(+), 9 deletions(-)
arch/x86/kvm/x86.c +5 -9
@@ -5328,7 +5328,6 @@ EXPORT_SYMBOL_GPL(kvm_write_guest_virt_system);int handle_ud (struct kvm_vcpu *vcpu) {int emul_type =EMULTYPE_TRAP_UD ; -enum emulation_result er;char sig[5 ];/* ud2; .ascii "kvm" */ struct x86_exception e;
@@ -5340,12 +5339,7 @@ int handle_ud(struct kvm_vcpu *vcpu) emul_type =EMULTYPE_TRAP_UD_FORCED ; } - er =kvm_emulate_instruction (vcpu, emul_type); -if (er ==EMULATE_USER_EXIT ) -return 0 ; -if (er !=EMULATE_DONE ) -kvm_queue_exception (vcpu,UD_VECTOR ); -return 1 ; +return kvm_emulate_instruction (vcpu, emul_type) !=EMULATE_USER_EXIT ; }EXPORT_SYMBOL_GPL (handle_ud);
@@ -6533,8 +6527,10 @@ int x86_emulate_instruction(struct kvm_vcpu *vcpu, ++vcpu->stat.insn_emulation;if (r !=EMULATION_OK ) {if ((emulation_type &EMULTYPE_TRAP_UD ) || - (emulation_type &EMULTYPE_TRAP_UD_FORCED )) -return EMULATE_FAIL ; + (emulation_type &EMULTYPE_TRAP_UD_FORCED )) { +kvm_queue_exception (vcpu,UD_VECTOR ); +return EMULATE_DONE ; + }if (reexecute_instruction (vcpu, cr2, write_fault_to_spt, emulation_type))return EMULATE_DONE ;
Signature · 1 line
2.22.0
PATCH v2 8/14 KVM: x86: Exit to userspace on emulation skip failure
In reply to Sean ChristophersonKill a few birds with one stone by forcing an exit to userspace on skip emulation failure. This removes a reference to EMULATE_FAIL, fixes a bug in handle_ept_misconfig() where it would exit to userspace without setting run->exit_reason, and fixes a theoretical bug in SVM's task_switch_interception() where it would overwrite run->exit_reason on a return of EMULATE_USER_EXIT.
Note, this technically doesn't fully fix task_switch_interception() as it now incorrectly handles EMULATE_FAIL, but in practice there is no bug as EMULATE_FAIL will never be returned for EMULTYPE_SKIP.
--- arch/x86/kvm/svm.c | 4 ++-- arch/x86/kvm/x86.c | 9 +++++++-- 2 files changed, 9 insertions(+), 4 deletions(-)
arch/x86/kvm/svm.c +2 -2
@@ -3885,8 +3885,8 @@ static int task_switch_interception(struct vcpu_svm *svm) int_type == SVM_EXITINTINFO_TYPE_SOFT || (int_type == SVM_EXITINTINFO_TYPE_EXEPT && (int_vec == OF_VECTOR || int_vec == BP_VECTOR))) { -if (skip_emulated_instruction (&svm->vcpu) !=EMULATE_DONE ) - goto fail; +if (skip_emulated_instruction (&svm->vcpu) ==EMULATE_USER_EXIT ) +return 0 ; }if (int_type !=SVM_EXITINTINFO_TYPE_SOFT )
arch/x86/kvm/x86.c +7 -2
@@ -6209,6 +6209,13 @@ static int handle_emulation_failure(struct kvm_vcpu *vcpu, int emulation_type)return EMULATE_DONE ; } +if (emulation_type &EMULTYPE_SKIP ) { + vcpu->run->exit_reason =KVM_EXIT_INTERNAL_ERROR ; + vcpu->run->internal.suberror =KVM_INTERNAL_ERROR_EMULATION ; + vcpu->run->internal.ndata =0 ; +return EMULATE_USER_EXIT ; + } +kvm_queue_exception (vcpu,UD_VECTOR );if (!is_guest_mode (vcpu) && kvm_x86_ops->get_cpl (vcpu) ==0 ) {
@@ -6536,8 +6543,6 @@ int x86_emulate_instruction(struct kvm_vcpu *vcpu,return EMULATE_DONE ;if (ctxt->have_exception &&inject_emulated_exception (vcpu))return EMULATE_DONE ; -if (emulation_type &EMULTYPE_SKIP ) -return EMULATE_FAIL ;return handle_emulation_failure (vcpu, emulation_type); } }
Signature · 1 line
2.22.0
PATCH v2 9/14 KVM: x86: Handle emulation failure directly in kvm_task_switch()
In reply to Sean ChristophersonConsolidate the reporting of emulation failure into kvm_task_switch() so that it can return EMULATE_USER_EXIT. This helps pave the way for removing EMULATE_FAIL altogether.
This also fixes a theoretical bug where task switch interception could suppress an EMULATE_USER_EXIT return.
--- arch/x86/kvm/svm.c | 11 ++--------- arch/x86/kvm/vmx/vmx.c | 14 +++----------- arch/x86/kvm/x86.c | 9 ++++++--- 3 files changed, 11 insertions(+), 23 deletions(-)
arch/x86/kvm/svm.c +2 -9
@@ -3892,17 +3892,10 @@ static int task_switch_interception(struct vcpu_svm *svm)if (int_type !=SVM_EXITINTINFO_TYPE_SOFT ) int_vec =-1 ; -if (kvm_task_switch (&svm->vcpu, tss_selector, int_vec, reason, - has_error_code, error_code) ==EMULATE_FAIL ) - goto fail; -return 1 ; -fail: - svm->vcpu.run->exit_reason =KVM_EXIT_INTERNAL_ERROR ; - svm->vcpu.run->internal.suberror =KVM_INTERNAL_ERROR_EMULATION ; - svm->vcpu.run->internal.ndata =0 ; -return 0 ; +return kvm_task_switch (&svm->vcpu, tss_selector, int_vec, reason, + has_error_code, error_code) !=EMULATE_USER_EXIT ; }static int cpuid_interception (struct vcpu_svm *svm)
arch/x86/kvm/vmx/vmx.c +3 -11
@@ -5073,21 +5073,13 @@ static int handle_task_switch(struct kvm_vcpu *vcpu) type != INTR_TYPE_NMI_INTR))skip_emulated_instruction (vcpu); -if (kvm_task_switch (vcpu, tss_selector, - type ==INTR_TYPE_SOFT_INTR ? idt_index :-1 , reason, - has_error_code, error_code) ==EMULATE_FAIL ) { - vcpu->run->exit_reason =KVM_EXIT_INTERNAL_ERROR ; - vcpu->run->internal.suberror =KVM_INTERNAL_ERROR_EMULATION ; - vcpu->run->internal.ndata =0 ; -return 0 ; - } -/* * TODO: What about debug traps on tss switch? * Are we supposed to inject them and update dr6? */ - -return 1 ; +return kvm_task_switch (vcpu, tss_selector, + type ==INTR_TYPE_SOFT_INTR ? idt_index :-1 , + reason, has_error_code, error_code) !=EMULATE_USER_EXIT ; }static int handle_ept_violation (struct kvm_vcpu *vcpu)
arch/x86/kvm/x86.c +6 -3
@@ -8587,9 +8587,12 @@ int kvm_task_switch(struct kvm_vcpu *vcpu, u16 tss_selector, int idt_index, ret =emulator_task_switch (ctxt, tss_selector, idt_index, reason, has_error_code, error_code); - -if (ret) -return EMULATE_FAIL ; +if (ret) { + vcpu->run->exit_reason =KVM_EXIT_INTERNAL_ERROR ; + vcpu->run->internal.suberror =KVM_INTERNAL_ERROR_EMULATION ; + vcpu->run->internal.ndata =0 ; +return EMULATE_USER_EXIT ; + }kvm_rip_write (vcpu, ctxt->eip);kvm_set_rflags (vcpu, ctxt->eflags);
Signature · 1 line
2.22.0
PATCH v2 10/14 KVM: x86: Move triple fault request into RM int injection
In reply to Sean ChristophersonRequest triple fault in kvm_inject_realmode_interrupt() instead of returning EMULATE_FAIL and deferring to the caller. All existing callers request triple fault and it's highly unlikely Real Mode is going to acquire new features. While this consolidates a small amount of code, the real goal is to remove the last reference to EMULATE_FAIL.
No functional change intended.
--- arch/x86/kvm/vmx/vmx.c | 9 +++------ arch/x86/kvm/x86.c | 17 ++++++++--------- arch/x86/kvm/x86.h | 2 +- 3 files changed, 12 insertions(+), 16 deletions(-)
arch/x86/kvm/vmx/vmx.c +3 -6
@@ -1527,8 +1527,7 @@ static void vmx_queue_exception(struct kvm_vcpu *vcpu)int inc_eip =0 ;if (kvm_exception_is_soft (nr)) inc_eip = vcpu->arch.event_exit_inst_len; -if (kvm_inject_realmode_interrupt (vcpu, nr, inc_eip) !=EMULATE_DONE ) -kvm_make_request (KVM_REQ_TRIPLE_FAULT , vcpu); +kvm_inject_realmode_interrupt (vcpu, nr, inc_eip);return ; }
@@ -4276,8 +4275,7 @@ static void vmx_inject_irq(struct kvm_vcpu *vcpu)int inc_eip =0 ;if (vcpu->arch.interrupt.soft) inc_eip = vcpu->arch.event_exit_inst_len; -if (kvm_inject_realmode_interrupt (vcpu, irq, inc_eip) !=EMULATE_DONE ) -kvm_make_request (KVM_REQ_TRIPLE_FAULT , vcpu); +kvm_inject_realmode_interrupt (vcpu, irq, inc_eip);return ; } intr = irq |INTR_INFO_VALID_MASK ;
@@ -4313,8 +4311,7 @@ static void vmx_inject_nmi(struct kvm_vcpu *vcpu) vmx->loaded_vmcs->nmi_known_unmasked = false;if (vmx->rmode.vm86_active) { -if (kvm_inject_realmode_interrupt (vcpu,NMI_VECTOR ,0 ) !=EMULATE_DONE ) -kvm_make_request (KVM_REQ_TRIPLE_FAULT , vcpu); +kvm_inject_realmode_interrupt (vcpu,NMI_VECTOR ,0 );return ; }
arch/x86/kvm/x86.c +8 -9
@@ -6176,7 +6176,7 @@ static void init_emulate_ctxt(struct kvm_vcpu *vcpu) vcpu->arch.emulate_regs_need_sync_from_vcpu = false; } -int kvm_inject_realmode_interrupt (struct kvm_vcpu *vcpu,int irq,int inc_eip) +void kvm_inject_realmode_interrupt (struct kvm_vcpu *vcpu,int irq,int inc_eip) {struct x86_emulate_ctxt *ctxt = &vcpu->arch.emulate_ctxt;int ret;
@@ -6188,14 +6188,13 @@ int kvm_inject_realmode_interrupt(struct kvm_vcpu *vcpu, int irq, int inc_eip) ctxt->_eip = ctxt->eip + inc_eip; ret =emulate_int_real (ctxt, irq); -if (ret !=X86EMUL_CONTINUE ) -return EMULATE_FAIL ; - - ctxt->eip = ctxt->_eip; -kvm_rip_write (vcpu, ctxt->eip); -kvm_set_rflags (vcpu, ctxt->eflags); - -return EMULATE_DONE ; +if (ret !=X86EMUL_CONTINUE ) { +kvm_make_request (KVM_REQ_TRIPLE_FAULT , vcpu); + }else { + ctxt->eip = ctxt->_eip; +kvm_rip_write (vcpu, ctxt->eip); +kvm_set_rflags (vcpu, ctxt->eflags); + } }EXPORT_SYMBOL_GPL (kvm_inject_realmode_interrupt);
arch/x86/kvm/x86.h +1 -1
@@ -261,7 +261,7 @@ static inline bool kvm_check_has_quirk(struct kvm *kvm, u64 quirk) }void kvm_set_pending_timer (struct kvm_vcpu *vcpu); -int kvm_inject_realmode_interrupt (struct kvm_vcpu *vcpu,int irq,int inc_eip); +void kvm_inject_realmode_interrupt (struct kvm_vcpu *vcpu,int irq,int inc_eip);void kvm_write_tsc (struct kvm_vcpu *vcpu,struct msr_data *msr);u64 get_kvmclock_ns (struct kvm *kvm);
Signature · 1 line
2.22.0
PATCH v2 11/14 KVM: VMX: Remove EMULATE_FAIL handling in handle_invalid_guest_state()
In reply to Sean ChristophersonNow that EMULATE_FAIL is completely unused, remove the last remaning usage where KVM does something functional in response to EMULATE_FAIL. Leave the check in place as a WARN_ON_ONCE to provide a better paper trail when EMULATE_{DONE,FAIL,USER_EXIT} are completely removed.
Opportunistically remove the gotos in handle_invalid_guest_state(). With the EMULATE_FAIL handling gone there is no need to have a common handler for emulation failure and the gotos only complicate things, e.g. the signal_pending() check always returns '1', but this is far from obvious when glancing through the code.
--- arch/x86/kvm/vmx/vmx.c | 39 +++++++++++++++++++-------------------- 1 file changed, 19 insertions(+), 20 deletions(-)
arch/x86/kvm/vmx/vmx.c +19 -20
@@ -5169,7 +5169,6 @@ static int handle_invalid_guest_state(struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu);enum emulation_result err =EMULATE_DONE ; -int ret =1 ;bool intr_window_requested;unsigned count =130 ;
@@ -5192,38 +5191,38 @@ static int handle_invalid_guest_state(struct kvm_vcpu *vcpu) err =kvm_emulate_instruction (vcpu,0 ); -if (err ==EMULATE_USER_EXIT ) { - ret =0 ; - goto out; - } +if (err ==EMULATE_USER_EXIT ) +return 0 ; -if (err !=EMULATE_DONE ) - goto emulation_error; +if (WARN_ON_ONCE (err ==EMULATE_FAIL )) +return 1 ;if (vmx->emulation_required && !vmx->rmode.vm86_active && - vcpu->arch.exception.pending) - goto emulation_error; + vcpu->arch.exception.pending) { + vcpu->run->exit_reason =KVM_EXIT_INTERNAL_ERROR ; + vcpu->run->internal.suberror = +KVM_INTERNAL_ERROR_EMULATION ; + vcpu->run->internal.ndata =0 ; +return 0 ; + }if (vcpu->arch.halt_request) { vcpu->arch.halt_request =0 ; - ret =kvm_vcpu_halt (vcpu); - goto out; +return kvm_vcpu_halt (vcpu); } +/* +* Note, return 1 and not 0, vcpu_run() is responsible for +* morphing the pending signal into the proper return code. +*/ if (signal_pending (current)) - goto out; +return 1 ; +if (need_resched ())schedule (); } -out: -return ret; - -emulation_error: - vcpu->run->exit_reason =KVM_EXIT_INTERNAL_ERROR ; - vcpu->run->internal.suberror =KVM_INTERNAL_ERROR_EMULATION ; - vcpu->run->internal.ndata =0 ; -return 0 ; +return 1 ; }static void grow_ple_window (struct kvm_vcpu *vcpu)
Signature · 1 line
2.22.0
PATCH v2 12/14 KVM: x86: Remove emulation_result enums, EMULATE_{DONE,FAIL,USER_EXIT}
In reply to Sean ChristophersonDeferring emulation failure handling (in some cases) to the caller of x86_emulate_instruction() has proven fragile, e.g. multiple instances of KVM not setting run->exit_reason on EMULATE_FAIL, largely due to it being difficult to discern what emulation types can return what result, and which combination of types and results are handled where.
Now that x86_emulate_instruction() always handles emulation failure, i.e. EMULATION_FAIL is only referenced in callers, remove the emulation_result enums entirely. Per KVM's existing exit handling conventions, return '0' and '1' for "exit to userspace" and "resume guest" respectively. Doing so cleans up many callers, e.g. they can return kvm_emulate_instruction() directly instead of having to interpret its result.
--- arch/x86/include/asm/kvm_host.h | 6 ---- arch/x86/kvm/mmu.c | 14 ++------ arch/x86/kvm/svm.c | 22 ++++++------- arch/x86/kvm/vmx/vmx.c | 28 ++++++---------- arch/x86/kvm/x86.c | 57 ++++++++++++++++----------------- 5 files changed, 49 insertions(+), 78 deletions(-)
arch/x86/include/asm/kvm_host.h +0 -6
@@ -1308,12 +1308,6 @@ extern u64 kvm_default_tsc_scaling_ratio;extern u64 kvm_mce_cap_supported; -enum emulation_result { -EMULATE_DONE ,/* no further processing */ -EMULATE_USER_EXIT ,/* kvm_run ready for userspace exit */ -EMULATE_FAIL ,/* can't emulate this instruction */ -}; -#define EMULTYPE_NO_DECODE (1 << 0)#define EMULTYPE_TRAP_UD (1 << 1)#define EMULTYPE_SKIP (1 << 2)
arch/x86/kvm/mmu.c +2 -12
@@ -5364,7 +5364,6 @@ int kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gva_t cr2, u64 error_code, void *insn, int insn_len) {int r, emulation_type =0 ; -enum emulation_result er;bool direct = vcpu->arch.mmu->direct_map;/* With shadow page tables, fault_address contains a GVA or nGPA. */
@@ -5431,17 +5430,8 @@ int kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gva_t cr2, u64 error_code,return 1 ; } - er =x86_emulate_instruction (vcpu, cr2, emulation_type, insn, insn_len); - -switch (er) { -case EMULATE_DONE : -return 1 ; -case EMULATE_USER_EXIT : -case EMULATE_FAIL : -return 0 ; -default : -BUG (); - } +return x86_emulate_instruction (vcpu, cr2, emulation_type, insn, + insn_len); }EXPORT_SYMBOL_GPL (kvm_mmu_page_fault);
arch/x86/kvm/svm.c +9 -13
@@ -787,7 +787,7 @@ static int skip_emulated_instruction(struct kvm_vcpu *vcpu)kvm_rip_write (vcpu, svm->next_rip);svm_set_interrupt_shadow (vcpu,0 ); -return EMULATE_DONE ; +return 1 ; }static void svm_queue_exception (struct kvm_vcpu *vcpu)
@@ -2779,8 +2779,7 @@ static int gp_interception(struct vcpu_svm *svm)kvm_queue_exception_e (vcpu,GP_VECTOR , error_code);return 1 ; } -return kvm_emulate_instruction (vcpu,EMULTYPE_VMWARE_GP ) != -EMULATE_USER_EXIT ; +return kvm_emulate_instruction (vcpu,EMULTYPE_VMWARE_GP ); }static bool is_erratum_383 (void )
@@ -2878,7 +2877,7 @@ static int io_interception(struct vcpu_svm *svm) string = (io_info &SVM_IOIO_STR_MASK ) !=0 ; in = (io_info &SVM_IOIO_TYPE_MASK ) !=0 ;if (string) -return kvm_emulate_instruction (vcpu,0 ) ==EMULATE_DONE ; +return kvm_emulate_instruction (vcpu,0 ); port = io_info >>16 ; size = (io_info &SVM_IOIO_SIZE_MASK ) >>SVM_IOIO_SIZE_SHIFT ;
@@ -3885,17 +3884,15 @@ static int task_switch_interception(struct vcpu_svm *svm) int_type == SVM_EXITINTINFO_TYPE_SOFT || (int_type == SVM_EXITINTINFO_TYPE_EXEPT && (int_vec == OF_VECTOR || int_vec == BP_VECTOR))) { -if (skip_emulated_instruction (&svm->vcpu) ==EMULATE_USER_EXIT ) +if (!skip_emulated_instruction (&svm->vcpu))return 0 ; }if (int_type !=SVM_EXITINTINFO_TYPE_SOFT ) int_vec =-1 ; - -return kvm_task_switch (&svm->vcpu, tss_selector, int_vec, reason, - has_error_code, error_code) !=EMULATE_USER_EXIT ; + has_error_code, error_code); }static int cpuid_interception (struct vcpu_svm *svm)
@@ -3916,7 +3913,7 @@ static int iret_interception(struct vcpu_svm *svm)static int invlpg_interception (struct vcpu_svm *svm) {if (!static_cpu_has (X86_FEATURE_DECODEASSISTS )) -return kvm_emulate_instruction (&svm->vcpu,0 ) ==EMULATE_DONE ; +return kvm_emulate_instruction (&svm->vcpu,0 );kvm_mmu_invlpg (&svm->vcpu, svm->vmcb->control.exit_info_1);return kvm_skip_emulated_instruction (&svm->vcpu);
@@ -3924,13 +3921,12 @@ static int invlpg_interception(struct vcpu_svm *svm)static int emulate_on_interception (struct vcpu_svm *svm) { -return kvm_emulate_instruction (&svm->vcpu,0 ) ==EMULATE_DONE ; +return kvm_emulate_instruction (&svm->vcpu,0 ); }static int rsm_interception (struct vcpu_svm *svm) { -return kvm_emulate_instruction_from_buffer (&svm->vcpu, - rsm_ins_bytes,2 ) ==EMULATE_DONE ; +return kvm_emulate_instruction_from_buffer (&svm->vcpu, rsm_ins_bytes,2 ); }static int rdpmc_interception (struct vcpu_svm *svm)
@@ -4749,7 +4745,7 @@ static int avic_unaccelerated_access_interception(struct vcpu_svm *svm) ret =avic_unaccel_trap_write (svm); }else {/* Handling Fault */ - ret = (kvm_emulate_instruction (&svm->vcpu,0 ) ==EMULATE_DONE ); + ret =kvm_emulate_instruction (&svm->vcpu,0 ); }return ret;
arch/x86/kvm/vmx/vmx.c +10 -18
@@ -1487,7 +1487,7 @@ static int __skip_emulated_instruction(struct kvm_vcpu *vcpu)/* skipping an emulated instruction also counts */ vmx_set_interrupt_shadow (vcpu,0 ); -return EMULATE_DONE ; +return 1 ; }static inline void skip_emulated_instruction (struct kvm_vcpu *vcpu)
@@ -4438,7 +4438,7 @@ static int handle_rmode_exception(struct kvm_vcpu *vcpu,* Cause the #SS fault with 0 error code in VM86 mode. */ if (((vec ==GP_VECTOR ) || (vec ==SS_VECTOR )) && err_code ==0 ) { -if (kvm_emulate_instruction (vcpu,0 ) ==EMULATE_DONE ) { +if (kvm_emulate_instruction (vcpu,0 )) {if (vcpu->arch.halt_request) { vcpu->arch.halt_request =0 ;return kvm_vcpu_halt (vcpu);
@@ -4515,8 +4515,7 @@ static int handle_exception_nmi(struct kvm_vcpu *vcpu)kvm_queue_exception_e (vcpu,GP_VECTOR , error_code);return 1 ; } -return kvm_emulate_instruction (vcpu,EMULTYPE_VMWARE_GP ) != -EMULATE_USER_EXIT ; +return kvm_emulate_instruction (vcpu,EMULTYPE_VMWARE_GP ); }/*
@@ -4613,7 +4612,7 @@ static int handle_io(struct kvm_vcpu *vcpu) ++vcpu->stat.io_exits;if (string) -return kvm_emulate_instruction (vcpu,0 ) ==EMULATE_DONE ; +return kvm_emulate_instruction (vcpu,0 ); port = exit_qualification >>16 ; size = (exit_qualification &7 ) +1 ;
@@ -4687,7 +4686,7 @@ static int handle_set_cr4(struct kvm_vcpu *vcpu, unsigned long val)static int handle_desc (struct kvm_vcpu *vcpu) {WARN_ON (!(vcpu->arch.cr4 &X86_CR4_UMIP )); -return kvm_emulate_instruction (vcpu,0 ) ==EMULATE_DONE ; +return kvm_emulate_instruction (vcpu,0 ); }static int handle_cr (struct kvm_vcpu *vcpu)
@@ -4932,7 +4931,7 @@ static int handle_vmcall(struct kvm_vcpu *vcpu)static int handle_invd (struct kvm_vcpu *vcpu) { -return kvm_emulate_instruction (vcpu,0 ) ==EMULATE_DONE ; +return kvm_emulate_instruction (vcpu,0 ); }static int handle_invlpg (struct kvm_vcpu *vcpu)
@@ -4999,7 +4998,7 @@ static int handle_apic_access(struct kvm_vcpu *vcpu)return kvm_skip_emulated_instruction (vcpu); } } -return kvm_emulate_instruction (vcpu,0 ) ==EMULATE_DONE ; +return kvm_emulate_instruction (vcpu,0 ); }static int handle_apic_eoi_induced (struct kvm_vcpu *vcpu)
@@ -5076,7 +5075,7 @@ static int handle_task_switch(struct kvm_vcpu *vcpu)*/ return kvm_task_switch (vcpu, tss_selector, type ==INTR_TYPE_SOFT_INTR ? idt_index :-1 , - reason, has_error_code, error_code) !=EMULATE_USER_EXIT ; + reason, has_error_code, error_code); }static int handle_ept_violation (struct kvm_vcpu *vcpu)
@@ -5148,8 +5147,7 @@ static int handle_ept_misconfig(struct kvm_vcpu *vcpu)if (!static_cpu_has (X86_FEATURE_HYPERVISOR ))return kvm_skip_emulated_instruction (vcpu);else -return kvm_emulate_instruction (vcpu,EMULTYPE_SKIP ) == -EMULATE_DONE ; +return kvm_emulate_instruction (vcpu,EMULTYPE_SKIP ); }return kvm_mmu_page_fault (vcpu, gpa,PFERR_RSVD_MASK ,NULL ,0 );
@@ -5168,7 +5166,6 @@ static int handle_nmi_window(struct kvm_vcpu *vcpu)static int handle_invalid_guest_state (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu); -enum emulation_result err =EMULATE_DONE ;bool intr_window_requested;unsigned count =130 ;
@@ -5189,14 +5186,9 @@ static int handle_invalid_guest_state(struct kvm_vcpu *vcpu)if (kvm_test_request (KVM_REQ_EVENT , vcpu))return 1 ; - err =kvm_emulate_instruction (vcpu,0 ); - -if (err ==EMULATE_USER_EXIT ) +if (!kvm_emulate_instruction (vcpu,0 ))return 0 ; -if (WARN_ON_ONCE (err ==EMULATE_FAIL )) -return 1 ; -if (vmx->emulation_required && !vmx->rmode.vm86_active && vcpu->arch.exception.pending) { vcpu->run->exit_reason =KVM_EXIT_INTERNAL_ERROR ;
arch/x86/kvm/x86.c +28 -29
196 lines not on this page · raw message
Signature · 1 line
2.22.0
PATCH v2 13/14 KVM: VMX: Handle single-step #DB for EMULTYPE_SKIP on EPT misconfig
In reply to Sean ChristophersonVMX's EPT misconfig flow to handle fast-MMIO path falls back to decoding the instruction to determine the instruction length when running as a guest (Hyper-V doesn't fill VMCS.VM_EXIT_INSTRUCTION_LEN because it's technically not defined for EPT misconfigs). Rather than implement the slow skip in VMX's generic skip_emulated_instruction(), handle_ept_misconfig() directly calls kvm_emulate_instruction() with EMULTYPE_SKIP, which intentionally doesn't do single-step detection, and so handle_ept_misconfig() misses a single-step #DB.
Rework the EPT misconfig fallback case to route it through kvm_skip_emulated_instruction() so that single-step #DBs and interrupt shadow updates are handled automatically. I.e. make VMX's slow skip logic match SVM's and have the SVM flow not intentionally avoid the shadow update.
Alternatively, the handle_ept_misconfig() could manually handle single- step detection, but that results in EMULTYPE_SKIP having split logic for the interrupt shadow vs. single-step #DBs, and split emulator logic is largely what led to this mess in the first place.
Modifying SVM to mirror VMX flow isn't really an option as SVM's case isn't limited to a specific exit reason, i.e. handling the slow skip in skip_emulated_instruction() is mandatory for all intents and purposes.
Drop VMX's skip_emulated_instruction() wrapper since it can now fail, and instead WARN if it fails unexpectedly, e.g. if exit_reason somehow becomes corrupted.
--- arch/x86/kvm/svm.c | 17 +++++++------- arch/x86/kvm/vmx/vmx.c | 52 ++++++++++++++++++------------------------ arch/x86/kvm/x86.c | 6 ++++- 3 files changed, 36 insertions(+), 39 deletions(-)
arch/x86/kvm/svm.c +9 -8
@@ -777,14 +777,15 @@ static int skip_emulated_instruction(struct kvm_vcpu *vcpu) svm->next_rip = svm->vmcb->control.next_rip; } -if (!svm->next_rip) -return kvm_emulate_instruction (vcpu,EMULTYPE_SKIP ); - -if (svm->next_rip -kvm_rip_read (vcpu) >MAX_INST_SIZE ) -printk (KERN_ERR "%s: ip 0x%lx next 0x%llx\n" , - __func__,kvm_rip_read (vcpu), svm->next_rip); - -kvm_rip_write (vcpu, svm->next_rip); +if (!svm->next_rip) { +if (!kvm_emulate_instruction (vcpu,EMULTYPE_SKIP )) +return 0 ; + }else { +if (svm->next_rip -kvm_rip_read (vcpu) >MAX_INST_SIZE ) +pr_err ("%s: ip 0x%lx next 0x%llx\n" , + __func__,kvm_rip_read (vcpu), svm->next_rip); +kvm_rip_write (vcpu, svm->next_rip); + }svm_set_interrupt_shadow (vcpu,0 );return 1 ;
arch/x86/kvm/vmx/vmx.c +22 -30
91 lines not on this page · raw message
arch/x86/kvm/x86.c +5 -1
@@ -6552,11 +6552,15 @@ int x86_emulate_instruction(struct kvm_vcpu *vcpu,return 1 ; } +/* +* Note, EMULTYPE_SKIP is intended for use *only* by vendor callbacks +* for kvm_skip_emulated_instruction(). The caller is responsible for +* updating interruptibility state and injecting single-step #DBs. +*/ if (emulation_type &EMULTYPE_SKIP ) {kvm_rip_write (vcpu, ctxt->_eip);if (ctxt->eflags &X86_EFLAGS_RF )kvm_set_rflags (vcpu, ctxt->eflags & ~X86_EFLAGS_RF ); - kvm_x86_ops->set_interrupt_shadow (vcpu,0 );return 1 ; }
Signature · 1 line
2.22.0
PATCH v2 14/14 KVM: x86: Add comments to document various emulation types
In reply to Sean ChristophersonDocument the intended usage of each emulation type as each exists to handle an edge case of one kind or another and can be easily misinterpreted at first glance.
--- arch/x86/include/asm/kvm_host.h | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+)
arch/x86/include/asm/kvm_host.h +30 -0
@@ -1308,6 +1308,36 @@ extern u64 kvm_default_tsc_scaling_ratio;extern u64 kvm_mce_cap_supported; +/* +* EMULTYPE_NO_DECODE - Set when re-emulating an instruction (after completing +* userspace I/O) to indicate that the emulation context +* should be resued as is, i.e. skip initialization of +* emulation context, instruction fetch and decode. +* +* EMULTYPE_TRAP_UD - Set when emulating an intercepted #UD from hardware. +* Indicates that only select instructions (tagged with +* EmulateOnUD) should be emulated (to minimize the emulator +* attack surface). See also EMULTYPE_TRAP_UD_FORCED. +* +* EMULTYPE_SKIP - Set when emulating solely to skip an instruction, i.e. to +* decode the instruction length. For use *only* by +* kvm_x86_ops->skip_emulated_instruction() implementations. +* +* EMULTYPE_ALLOW_RETRY - Set when the emulator should resume the guest to +* retry native execution under certain conditions. +* +* EMULTYPE_TRAP_UD_FORCED - Set when emulating an intercepted #UD that was +* triggered by KVM's magic "force emulation" prefix, +* which is opt in via module param (off by default). +* Bypasses EmulateOnUD restriction despite emulating +* due to an intercepted #UD (see EMULTYPE_TRAP_UD). +* Used to test the full emulator from userspace. +* +* EMULTYPE_VMWARE_GP - Set when emulating an intercepted #GP for VMware +* backdoor emulation, which is opt in via module param. +* VMware backoor emulation handles select instructions +* and reinjects the #GP for all other cases. +*/ #define EMULTYPE_NO_DECODE (1 << 0)#define EMULTYPE_TRAP_UD (1 << 1)#define EMULTYPE_SKIP (1 << 2)
Signature · 1 line
2.22.0
Quoted 54 lines · Sean Christopherson
Rework the emulator and its users to handle failure scenarios entirely within the emulator.
{x86,kvm}_emulate_instruction() currently returns a tri-state value to indicate success/continue, userspace exit needed, and failure. The intent of returning
EMULATE_FAILis to let the caller handle failure in a manner that is appropriate for the current context. In practice, the emulator has ended up with a mixture of failure handling, i.e. whether or not the emulator takes action on failure is dependent on the specific flavor of emulation.The mixed handling has proven to be rather fragile, e.g. many flows incorrectly assume their specific flavor of emulation cannot fail or that the emulator sets state to report the failure back to userspace.
Move everything inside the emulator, piece by piece, so that the emulation routines can return '0' for exit to userspace and '1' for resume the guest, just like every other VM-Exit handler.
Patch 13/14 is a tangentially related bug fix that conflicts heavily with this series, so I tacked it on here.
Patch 14/14 documents the emulation types. I added it as a separate patch at the very end so that the comments could reference the final state of the code base, e.g. incorporate the rule change for using
EMULTYPE_SKIPthat is introduced in patch 13/14.v1:
- https://patchwork.kernel.org/cover/11110331/v2:
- Collect reviews. [Vitaly and Liran]
- Squash VMware emultype changes into a single patch. [Liran]
- Add comments in VMX/SVM for VMware #GP handling. [Vitaly]
- Tack on the EPT misconfig bug fix.
- Add a patch to comment/document the emultypes. [Liran]Sean Christopherson (14):
KVM: x86: Relocate MMIO exit stats counting
KVM: x86: Clean uphandle_emulation_failure()
KVM: x86: Refactorkvm_vcpu_do_singlestep()to remove out param
KVM: x86: Don't attempt VMWare emulation on #GP with non-zero error
code
KVM: x86: Move #GP injection for VMware intox86_emulate_instruction()
KVM: x86: Add explicit flag for forced emulation on #UD
KVM: x86: Move #UD injection for failed emulation into emulation code
KVM: x86: Exit to userspace on emulation skip failure
KVM: x86: Handle emulation failure directly inkvm_task_switch()
KVM: x86: Move triple fault request into RM int injection
KVM: VMX: RemoveEMULATE_FAILhandling inhandle_invalid_guest_state()
KVM: x86: Removeemulation_resultenums, EMULATE_{DONE,FAIL,USER_EXIT}
KVM: VMX: Handle single-step #DB forEMULTYPE_SKIPon EPT misconfig
KVM: x86: Add comments to document various emulation types
Quoted 8 lines
arch/x86/include/asm/kvm_host.h | 40 +++++++-- arch/x86/kvm/mmu.c | 16 +--- arch/x86/kvm/svm.c | 62 ++++++-------- arch/x86/kvm/vmx/vmx.c | 147 +++++++++++++------------------- arch/x86/kvm/x86.c | 133 ++++++++++++++++------------- arch/x86/kvm/x86.h | 2 +- 6 files changed, 195 insertions(+), 205 deletions(-)
Queued, thanks (a couple conflicts had to be sorted out, but nothing requiring a respin).
Paolo
Quoted 64 lines · Paolo Bonzini
Quoted 54 lines · Sean Christopherson
Rework the emulator and its users to handle failure scenarios entirely within the emulator.
{x86,kvm}_emulate_instruction() currently returns a tri-state value to indicate success/continue, userspace exit needed, and failure. The intent of returning
EMULATE_FAILis to let the caller handle failure in a manner that is appropriate for the current context. In practice, the emulator has ended up with a mixture of failure handling, i.e. whether or not the emulator takes action on failure is dependent on the specific flavor of emulation.The mixed handling has proven to be rather fragile, e.g. many flows incorrectly assume their specific flavor of emulation cannot fail or that the emulator sets state to report the failure back to userspace.
Move everything inside the emulator, piece by piece, so that the emulation routines can return '0' for exit to userspace and '1' for resume the guest, just like every other VM-Exit handler.
Patch 13/14 is a tangentially related bug fix that conflicts heavily with this series, so I tacked it on here.
Patch 14/14 documents the emulation types. I added it as a separate patch at the very end so that the comments could reference the final state of the code base, e.g. incorporate the rule change for using
EMULTYPE_SKIPthat is introduced in patch 13/14.v1:
- https://patchwork.kernel.org/cover/11110331/v2:
- Collect reviews. [Vitaly and Liran]
- Squash VMware emultype changes into a single patch. [Liran]
- Add comments in VMX/SVM for VMware #GP handling. [Vitaly]
- Tack on the EPT misconfig bug fix.
- Add a patch to comment/document the emultypes. [Liran]Sean Christopherson (14):
KVM: x86: Relocate MMIO exit stats counting
KVM: x86: Clean uphandle_emulation_failure()
KVM: x86: Refactorkvm_vcpu_do_singlestep()to remove out param
KVM: x86: Don't attempt VMWare emulation on #GP with non-zero error
code
KVM: x86: Move #GP injection for VMware intox86_emulate_instruction()
KVM: x86: Add explicit flag for forced emulation on #UD
KVM: x86: Move #UD injection for failed emulation into emulation code
KVM: x86: Exit to userspace on emulation skip failure
KVM: x86: Handle emulation failure directly inkvm_task_switch()
KVM: x86: Move triple fault request into RM int injection
KVM: VMX: RemoveEMULATE_FAILhandling inhandle_invalid_guest_state()
KVM: x86: Removeemulation_resultenums, EMULATE_{DONE,FAIL,USER_EXIT}
KVM: VMX: Handle single-step #DB forEMULTYPE_SKIPon EPT misconfig
KVM: x86: Add comments to document various emulation typesQuoted 8 lines
arch/x86/include/asm/kvm_host.h | 40 +++++++-- arch/x86/kvm/mmu.c | 16 +--- arch/x86/kvm/svm.c | 62 ++++++-------- arch/x86/kvm/vmx/vmx.c | 147 +++++++++++++------------------- arch/x86/kvm/x86.c | 133 ++++++++++++++++------------- arch/x86/kvm/x86.h | 2 +- 6 files changed, 195 insertions(+), 205 deletions(-)
Quoted 2 lines
Queued, thanks (a couple conflicts had to be sorted out, but nothing requiring a respin).
Ugh, I just stumbled over this commit. Is this really the right direction to move towards?
I appreciate the move to reduce the emulator logic from the many-fold enum into a simple binary "worked" or "needs a user space exit". But are "0" and "1" really the right names for that? I find the readability of the current intercept handlers bad enough, trickling that into even more code sounds like a situation that will decrease readability even more.
Why can't we just use names throughout? Something like
enum kvm_return {
KVM_RET_USER_EXIT = 0,
KVM_RET_GUEST = 1,
};and then consistently use them as return values? That way anyone who has not worked on kvm before can still make sense of the code.
Alex
Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Ralf Herbrich Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879
Quoted 65 lines · Alexander Graf
Quoted 64 lines · Paolo Bonzini
Quoted 62 lines · Sean Christopherson
Rework the emulator and its users to handle failure scenarios entirely within the emulator.
{x86,kvm}_emulate_instruction() currently returns a tri-state value to indicate success/continue, userspace exit needed, and failure. The intent of returning
EMULATE_FAILis to let the caller handle failure in a manner that is appropriate for the current context. In practice, the emulator has ended up with a mixture of failure handling, i.e. whether or not the emulator takes action on failure is dependent on the specific flavor of emulation.The mixed handling has proven to be rather fragile, e.g. many flows incorrectly assume their specific flavor of emulation cannot fail or that the emulator sets state to report the failure back to userspace.
Move everything inside the emulator, piece by piece, so that the emulation routines can return '0' for exit to userspace and '1' for resume the guest, just like every other VM-Exit handler.
Patch 13/14 is a tangentially related bug fix that conflicts heavily with this series, so I tacked it on here.
Patch 14/14 documents the emulation types. I added it as a separate patch at the very end so that the comments could reference the final state of the code base, e.g. incorporate the rule change for using
EMULTYPE_SKIPthat is introduced in patch 13/14.v1:
- https://patchwork.kernel.org/cover/11110331/v2:
- Collect reviews. [Vitaly and Liran]
- Squash VMware emultype changes into a single patch. [Liran]
- Add comments in VMX/SVM for VMware #GP handling. [Vitaly]
- Tack on the EPT misconfig bug fix.
- Add a patch to comment/document the emultypes. [Liran]Sean Christopherson (14):
KVM: x86: Relocate MMIO exit stats counting
KVM: x86: Clean uphandle_emulation_failure()
KVM: x86: Refactorkvm_vcpu_do_singlestep()to remove out param
KVM: x86: Don't attempt VMWare emulation on #GP with non-zero error
code
KVM: x86: Move #GP injection for VMware intox86_emulate_instruction()
KVM: x86: Add explicit flag for forced emulation on #UD
KVM: x86: Move #UD injection for failed emulation into emulation code
KVM: x86: Exit to userspace on emulation skip failure
KVM: x86: Handle emulation failure directly inkvm_task_switch()
KVM: x86: Move triple fault request into RM int injection
KVM: VMX: RemoveEMULATE_FAILhandling inhandle_invalid_guest_state()
KVM: x86: Removeemulation_resultenums, EMULATE_{DONE,FAIL,USER_EXIT}
KVM: VMX: Handle single-step #DB forEMULTYPE_SKIPon EPT misconfig
KVM: x86: Add comments to document various emulation typesarch/x86/include/asm/kvm_host.h | 40 +++++++-- arch/x86/kvm/mmu.c | 16 +--- arch/x86/kvm/svm.c | 62 ++++++-------- arch/x86/kvm/vmx/vmx.c | 147 +++++++++++++------------------- arch/x86/kvm/x86.c | 133 ++++++++++++++++------------- arch/x86/kvm/x86.h | 2 +- 6 files changed, 195 insertions(+), 205 deletions(-)
Quoted 5 lines
Quoted 2 lines
Queued, thanks (a couple conflicts had to be sorted out, but nothing requiring a respin).
Ugh, I just stumbled over this commit. Is this really the right direction to move towards?
As you basically surmised below, removing the enum was just a side effect of cleaning up the emulation error handling, it wasn't really a goal in and of itself.
Quoted 8 lines
I appreciate the move to reduce the emulator logic from the many-fold enum into a simple binary "worked" or "needs a user space exit". But are "0" and "1" really the right names for that? I find the readability of the current intercept handlers bad enough, trickling that into even more code sounds like a situation that will decrease readability even more.
Why can't we just use names throughout? Something like
Quoted 7 lines
enum kvm_return { KVM_RET_USER_EXIT = 0, KVM_RET_GUEST = 1, };and then consistently use them as return values? That way anyone who has not worked on kvm before can still make sense of the code.
Hmm, I think it'd make more sense to use #define instead of enum to hopefully make it clear that they aren't the *only* values that can be returned. That'd also prevent anyone from changing the return types from 'int' to 'enum kvm_return', which IMO would hurt readability overall.
And maybe KVM_EXIT_TO_USERSPACE and KVM_RETURN_TO_GUEST?
Paolo Bonzini
maintainerQuoted 5 lines · Sean Christopherson
Quoted 5 lines
enum kvm_return { KVM_RET_USER_EXIT = 0, KVM_RET_GUEST = 1, };
Quoted 8 lines
Quoted 2 lines
and then consistently use them as return values? That way anyone who has not worked on kvm before can still make sense of the code.
Hmm, I think it'd make more sense to use #define instead of enum to hopefully make it clear that they aren't the *only* values that can be returned. That'd also prevent anyone from changing the return types from 'int' to 'enum
kvm_return', which IMO would hurt readability overall.And maybe
KVM_EXIT_TO_USERSPACEandKVM_RETURN_TO_GUEST?
That would be quite some work. Right now there is some consistency between all of:
- x86_emulate_instruction and its callers
- vcpu->arch.complete_userspace_io
- vcpu_enter_guest/vcpu_block
- kvm_x86_ops->handle_exit
so it would be very easy to end up with a half-int-half-enum state that is more confusing than before...
I'm more worried about cases where we have functions returning either 0 or -errno, but 0 lets you enter the guest. I'm not sure if the only one is kvm_mmu_reload or there are others.
Paolo
Quoted 20 lines · Paolo Bonzini
Quoted 5 lines · Sean Christopherson
Quoted 5 lines
enum kvm_return { KVM_RET_USER_EXIT = 0, KVM_RET_GUEST = 1, };Quoted 8 lines
Quoted 2 lines
and then consistently use them as return values? That way anyone who has not worked on kvm before can still make sense of the code.
Hmm, I think it'd make more sense to use #define instead of enum to hopefully make it clear that they aren't the *only* values that can be returned. That'd also prevent anyone from changing the return types from 'int' to 'enum
kvm_return', which IMO would hurt readability overall.And maybe
KVM_EXIT_TO_USERSPACEandKVM_RETURN_TO_GUEST?That would be quite some work. Right now there is some consistency between all of:
-
x86_emulate_instructionand its callers
Quoted 8 lines
-
vcpu->arch.complete_userspace_io- vcpu_enter_guest/vcpu_block
-
kvm_x86_ops->handle_exitso it would be very easy to end up with a half-int-half-enum state that is more confusing than before...
Ya, my thought was to update the obvious cases, essentially the ones you listed above, to use the define. So almost intentionally end up in a half-n-half state, at least in the short term, the thought being that the extra annotation would do more harm than good. But there's really no way to determine whether or not it would actually be a net positive without writing the code...
Quoted 3 lines
I'm more worried about cases where we have functions returning either 0 or -errno, but 0 lets you enter the guest. I'm not sure if the only one is
kvm_mmu_reloador there are others.
There are lots of those, e.g. basically all of the helpers for nested consistency checks.