KVM: x86: Convey the exit reason, etc. to user-space on emulation failure
v6 cover KVM: x86: Convey the exit reason, etc. to user-space on emulation failure
To help when debugging failures in the field, if instruction emulation fails, report the VM exit reason, etc. to userspace in order that it can be recorded.
The SGX changes here are compiled but untested.
v6:
- More Reviewed-by (Sean).
- Fix "From" (d'oh!).
v5:
- Add some Reviewed-by (Sean).
- Build-time complaint about sizing rather than run-time calculation (Sean).
- Clarify that the format of the auxiliary debug data is undefined (Sean).
- ndata_start -> info_start (Sean).
- sizeof(variable) rather than sizeof(type) (Sean).
v4:
- Update the API for preparing emulation failure report (Sean)
- sgx uses the provided API in all relevant cases (Sean)
- Clarify the intended layout of kvm_run.emulation_failure.
v3:
- Convey any debug data un-flagged after the ABI specified data in struct emulation_failure (Sean)
- Obey the ABI protocol in sgx_handle_emulation_failure() (Sean)
v2:
- Improve patch comments (dmatlock)
- Intel should provide the full exit reason (dmatlock)
- Pass a boolean rather than flags (dmatlock)
- Use the helper in kvm_task_switch() and kvm_handle_memory_failure() (dmatlock)
- Describe the exit_reason field of the emulation_failure structure (dmatlock)
David Edmondson (4):
KVM: x86: Clarify the kvm_run.emulation_failure structure layout
KVM: x86: Get exit_reason as part of kvm_x86_ops.get_exit_info
KVM: x86: On emulation failure, convey the exit reason, etc. to
userspace
KVM: x86: SGX must obey the KVM_INTERNAL_ERROR_EMULATION protocol
arch/x86/include/asm/kvm_host.h | 10 +++-- arch/x86/kvm/svm/svm.c | 8 ++-- arch/x86/kvm/trace.h | 9 ++-- arch/x86/kvm/vmx/nested.c | 2 +- arch/x86/kvm/vmx/sgx.c | 16 +++----- arch/x86/kvm/vmx/vmx.c | 11 +++-- arch/x86/kvm/x86.c | 73 ++++++++++++++++++++++++++------- include/uapi/linux/kvm.h | 14 ++++++- 8 files changed, 99 insertions(+), 44 deletions(-)
Signature · 1 line
2.33.0
PATCH v6 1/4 KVM: x86: Clarify the kvm_run.emulation_failure structure layout
Until more flags for kvm_run.emulation_failure flags are defined, it is undetermined whether new payload elements corresponding to those flags will be additive or alternative. As a hint to userspace that an alternative is possible, wrap the current payload elements in a union.
--- include/uapi/linux/kvm.h | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
include/uapi/linux/kvm.h +6 -2
@@ -402,8 +402,12 @@ struct kvm_run {__u32 suberror;__u32 ndata;__u64 flags; -__u8 insn_size; -__u8 insn_bytes[15 ]; +union { +struct { +__u8 insn_size; +__u8 insn_bytes[15 ]; + }; + }; } emulation_failure;/* KVM_EXIT_OSI */ struct {
Signature · 1 line
2.33.0
Re: [PATCH v6 1/4] KVM: x86: Clarify the kvm_run.emulation_failure structure layout
Quoted 8 lines · David Edmondson
Until more flags for kvm_run.emulation_failure flags are defined, it is undetermined whether new payload elements corresponding to those flags will be additive or alternative. As a hint to userspace that an alternative is possible, wrap the current payload elements in a union.
Suggested-by Sean Christopherson <[email protected]>Signed-off-by David Edmondson <[email protected]>---
To complete the set... :-)
Re: [PATCH v6 1/4] KVM: x86: Clarify the kvm_run.emulation_failure structure layout
Quoted 6 lines · Sean Christopherson
Quoted 5 lines · David Edmondson
Until more flags for kvm_run.emulation_failure flags are defined, it is undetermined whether new payload elements corresponding to those flags will be additive or alternative. As a hint to userspace that an alternative is possible, wrap the current payload elements in a union.
Quoted 7 lines
Quoted 3 lines
Suggested-by Sean Christopherson <[email protected]>Signed-off-by David Edmondson <[email protected]>---
To complete the set... :-)
Reviewed-by Sean Christopherson <[email protected]>
Thanks!
Re: [PATCH v6 1/4] KVM: x86: Clarify the kvm_run.emulation_failure structure layout
Quoted 12 lines · David Edmondson
Quoted 10 lines · Sean Christopherson
Quoted 8 lines · David Edmondson
Until more flags for kvm_run.emulation_failure flags are defined, it is undetermined whether new payload elements corresponding to those flags will be additive or alternative. As a hint to userspace that an alternative is possible, wrap the current payload elements in a union.
Suggested-by Sean Christopherson <[email protected]>Signed-off-by David Edmondson <[email protected]>---
Quoted 6 lines
Quoted 3 lines
To complete the set... :-)
Reviewed-by Sean Christopherson <[email protected]>Thanks!
Queued, thanks!
Paolo
PATCH v6 2/4 KVM: x86: Get exit_reason as part of kvm_x86_ops.get_exit_info
In reply to David EdmondsonExtend the get_exit_info static call to provide the reason for the VM exit. Modify relevant trace points to use this rather than extracting the reason in the caller.
--- arch/x86/include/asm/kvm_host.h | 7 ++++--- arch/x86/kvm/svm/svm.c | 8 +++++--- arch/x86/kvm/trace.h | 9 +++++---- arch/x86/kvm/vmx/nested.c | 2 +- arch/x86/kvm/vmx/vmx.c | 6 ++++-- 5 files changed, 19 insertions(+), 13 deletions(-)
arch/x86/include/asm/kvm_host.h +4 -3
@@ -1405,10 +1405,11 @@ struct kvm_x86_ops {void (*write_tsc_multiplier)(struct kvm_vcpu *vcpu,u64 multiplier);/* -* Retrieve somewhat arbitrary exit information. Intended to be used -* only from within tracepoints to avoid VMREADs when tracing is off. +* Retrieve somewhat arbitrary exit information. Intended to +* be used only from within tracepoints or error paths. */ -void (*get_exit_info)(struct kvm_vcpu *vcpu,u64 *info1,u64 *info2, +void (*get_exit_info)(struct kvm_vcpu *vcpu,u32 *reason, +u64 *info1,u64 *info2,u32 *exit_int_info,u32 *exit_int_info_err_code); int (*check_intercept)(struct kvm_vcpu *vcpu,
arch/x86/kvm/svm/svm.c +5 -3
@@ -3272,11 +3272,13 @@ int svm_invoke_exit_handler(struct kvm_vcpu *vcpu, u64 exit_code)return svm_exit_handlers[exit_code](vcpu); } -static void svm_get_exit_info (struct kvm_vcpu *vcpu,u64 *info1,u64 *info2, +static void svm_get_exit_info (struct kvm_vcpu *vcpu,u32 *reason, +u64 *info1,u64 *info2,u32 *intr_info,u32 *error_code) {struct vmcb_control_area *control = &to_svm (vcpu)->vmcb->control; + *reason = control->exit_code; *info1 = control->exit_info_1; *info2 = control->exit_info_2; *intr_info = control->exit_int_info;
@@ -3293,7 +3295,7 @@ static int handle_exit(struct kvm_vcpu *vcpu, fastpath_t exit_fastpath)struct kvm_run *kvm_run = vcpu->run;u32 exit_code = svm->vmcb->control.exit_code; -trace_kvm_exit (exit_code, vcpu,KVM_ISA_SVM ); +trace_kvm_exit (vcpu,KVM_ISA_SVM );/* SEV-ES guests must use the CR write traps to track CR registers. */ if (!sev_es_guest (vcpu->kvm)) {
@@ -3306,7 +3308,7 @@ static int handle_exit(struct kvm_vcpu *vcpu, fastpath_t exit_fastpath)if (is_guest_mode (vcpu)) {int vmexit; -trace_kvm_nested_vmexit (exit_code, vcpu,KVM_ISA_SVM ); +trace_kvm_nested_vmexit (vcpu,KVM_ISA_SVM ); vmexit =nested_svm_exit_special (svm);
arch/x86/kvm/trace.h +5 -4
@@ -288,8 +288,8 @@ TRACE_EVENT(kvm_apic,#define TRACE_EVENT_KVM_EXIT (name) \ TRACE_EVENT(name, \ - TP_PROTO(unsigned int exit_reason, struct kvm_vcpu *vcpu, u32 isa), \ - TP_ARGS(exit_reason, vcpu, isa), \ + TP_PROTO(struct kvm_vcpu *vcpu, u32 isa), \ + TP_ARGS(vcpu, isa), \ \ TP_STRUCT__entry( \ __field( unsigned int, exit_reason ) \
@@ -303,11 +303,12 @@ TRACE_EVENT(name, \ ), \ \TP_fast_assign ( \ - __entry->exit_reason = exit_reason; \ __entry->guest_rip = kvm_rip_read(vcpu); \ __entry->isa = isa; \ __entry->vcpu_id = vcpu->vcpu_id; \ -static_call (kvm_x86_get_exit_info)(vcpu, &__entry->info1, \ +static_call (kvm_x86_get_exit_info)(vcpu, \ + &__entry->exit_reason, \ + &__entry->info1, \ &__entry->info2, \ &__entry->intr_info, \ &__entry->error_code); \
arch/x86/kvm/vmx/nested.c +1 -1
@@ -6059,7 +6059,7 @@ bool nested_vmx_reflect_vmexit(struct kvm_vcpu *vcpu) goto reflect_vmexit; } -trace_kvm_nested_vmexit (exit_reason.full, vcpu,KVM_ISA_VMX ); +trace_kvm_nested_vmexit (vcpu,KVM_ISA_VMX );/* If L0 (KVM) wants the exit, it trumps L1's desires. */ if (nested_vmx_l0_wants_exit (vcpu, exit_reason))
arch/x86/kvm/vmx/vmx.c +4 -2
@@ -5628,11 +5628,13 @@ static int (*kvm_vmx_exit_handlers[])(struct kvm_vcpu *vcpu) = {static const int kvm_vmx_max_exit_handlers =ARRAY_SIZE (kvm_vmx_exit_handlers); -static void vmx_get_exit_info (struct kvm_vcpu *vcpu,u64 *info1,u64 *info2, +static void vmx_get_exit_info (struct kvm_vcpu *vcpu,u32 *reason, +u64 *info1,u64 *info2,u32 *intr_info,u32 *error_code) {struct vcpu_vmx *vmx =to_vmx (vcpu); + *reason = vmx->exit_reason.full; *info1 =vmx_get_exit_qual (vcpu);if (!(vmx->exit_reason.failed_vmentry)) { *info2 = vmx->idt_vectoring_info;
@@ -6769,7 +6771,7 @@ static fastpath_t vmx_vcpu_run(struct kvm_vcpu *vcpu)if (likely (!vmx->exit_reason.failed_vmentry)) vmx->idt_vectoring_info =vmcs_read32 (IDT_VECTORING_INFO_FIELD ); -trace_kvm_exit (vmx->exit_reason.full, vcpu,KVM_ISA_VMX ); +trace_kvm_exit (vcpu,KVM_ISA_VMX );if (unlikely (vmx->exit_reason.failed_vmentry))return EXIT_FASTPATH_NONE ;
Signature · 1 line
2.33.0
PATCH v6 3/4 KVM: x86: On emulation failure, convey the exit reason, etc. to userspace
In reply to David EdmondsonShould instruction emulation fail, include the VM exit reason, etc. in the emulation_failure data passed to userspace, in order that the VMM can report it as a debugging aid when describing the failure.
--- arch/x86/include/asm/kvm_host.h | 3 ++ arch/x86/kvm/vmx/vmx.c | 5 +-- arch/x86/kvm/x86.c | 73 ++++++++++++++++++++++++++------- include/uapi/linux/kvm.h | 6 +++ 4 files changed, 69 insertions(+), 18 deletions(-)
arch/x86/include/asm/kvm_host.h +3 -0
@@ -1658,6 +1658,9 @@ extern u64 kvm_mce_cap_supported;int kvm_emulate_instruction (struct kvm_vcpu *vcpu,int emulation_type);int kvm_emulate_instruction_from_buffer (struct kvm_vcpu *vcpu,void *insn,int insn_len); +void __kvm_prepare_emulation_failure_exit (struct kvm_vcpu *vcpu, +u64 *data,u8 ndata); +void kvm_prepare_emulation_failure_exit (struct kvm_vcpu *vcpu);void kvm_enable_efer_bits (u64 );bool kvm_valid_efer (struct kvm_vcpu *vcpu,u64 efer);
arch/x86/kvm/vmx/vmx.c +1 -4
@@ -5378,10 +5378,7 @@ static int handle_invalid_guest_state(struct kvm_vcpu *vcpu)if (vmx->emulation_required && !vmx->rmode.vm86_active && vcpu->arch.exception.pending) { - vcpu->run->exit_reason =KVM_EXIT_INTERNAL_ERROR ; - vcpu->run->internal.suberror = -KVM_INTERNAL_ERROR_EMULATION ; - vcpu->run->internal.ndata =0 ; +kvm_prepare_emulation_failure_exit (vcpu);return 0 ; }
arch/x86/kvm/x86.c +59 -14
113 lines not on this page · raw message
include/uapi/linux/kvm.h +6 -0
@@ -397,6 +397,11 @@ struct kvm_run {* "ndata" is correct, that new fields are enumerated in "flags", * and that each flag enumerates fields that are 64-bit aligned * and sized (so that ndata+internal.data[] is valid/accurate). +* +* Space beyond the defined fields may be used to store arbitrary +* debug information relating to the emulation failure. It is +* accounted for in "ndata" but the format is unspecified and is +* not represented in "flags". Any such information is *not* ABI! */ struct {__u32 suberror;
@@ -408,6 +413,7 @@ struct kvm_run {__u8 insn_bytes[15 ]; }; }; +/* Arbitrary debug data may follow. */ } emulation_failure;/* KVM_EXIT_OSI */ struct {
Signature · 1 line
2.33.0
PATCH v6 4/4 KVM: x86: SGX must obey the KVM_INTERNAL_ERROR_EMULATION protocol
In reply to David EdmondsonWhen passing the failing address and size out to user space, SGX must ensure not to trample on the earlier fields of the emulation_failure sub-union of struct kvm_run.
--- arch/x86/kvm/vmx/sgx.c | 16 +++++----------- 1 file changed, 5 insertions(+), 11 deletions(-)
arch/x86/kvm/vmx/sgx.c +5 -11
@@ -53,11 +53,9 @@ static int sgx_get_encls_gva(struct kvm_vcpu *vcpu, unsigned long offset,static void sgx_handle_emulation_failure (struct kvm_vcpu *vcpu,u64 addr,unsigned int size) { - vcpu->run->exit_reason =KVM_EXIT_INTERNAL_ERROR ; - vcpu->run->internal.suberror =KVM_INTERNAL_ERROR_EMULATION ; - vcpu->run->internal.ndata =2 ; - vcpu->run->internal.data[0 ] = addr; - vcpu->run->internal.data[1 ] = size; +uint64_t data[2 ] = { addr, size }; + +__kvm_prepare_emulation_failure_exit (vcpu, data,ARRAY_SIZE (data)); } static int sgx_read_hva(struct kvm_vcpu *vcpu, unsigned long hva, void *data,
@@ -112,9 +110,7 @@ static int sgx_inject_fault(struct kvm_vcpu *vcpu, gva_t gva, int trapnr)* but the error code isn't (yet) plumbed through the ENCLS helpers. */ if (trapnr ==PF_VECTOR && !boot_cpu_has (X86_FEATURE_SGX2 )) { - vcpu->run->exit_reason =KVM_EXIT_INTERNAL_ERROR ; - vcpu->run->internal.suberror =KVM_INTERNAL_ERROR_EMULATION ; - vcpu->run->internal.ndata =0 ; +kvm_prepare_emulation_failure_exit (vcpu);return 0 ; }
@@ -155,9 +151,7 @@ static int __handle_encls_ecreate(struct kvm_vcpu *vcpu, sgx_12_0 =kvm_find_cpuid_entry (vcpu,0x12 ,0 ); sgx_12_1 =kvm_find_cpuid_entry (vcpu,0x12 ,1 );if (!sgx_12_0 || !sgx_12_1) { - vcpu->run->exit_reason =KVM_EXIT_INTERNAL_ERROR ; - vcpu->run->internal.suberror =KVM_INTERNAL_ERROR_EMULATION ; - vcpu->run->internal.ndata =0 ; +kvm_prepare_emulation_failure_exit (vcpu);return 0 ; }
Signature · 1 line
2.33.0