kvm: x86: Add logical CPU to KVM_EXIT_FAIL_ENTRY info
Subsystems (4)
PATCH v2 1/1 kvm: x86: Add logical CPU to KVM_EXIT_FAIL_ENTRY info
More often than not, a failed VM-entry in a production environment is the result of a defective CPU (at least, insofar as Intel x86 is concerned). To aid in identifying the bad hardware, add the logical CPU to the information provided to userspace on a KVM exit with reason KVM_EXIT_FAIL_ENTRY. The presence of this additional information is indicated by a new capability, KVM_CAP_FAILED_ENTRY_CPU.
--- Documentation/virt/kvm/api.txt | 1 + arch/x86/kvm/svm.c | 1 + arch/x86/kvm/vmx/vmx.c | 2 ++ arch/x86/kvm/x86.c | 1 + include/uapi/linux/kvm.h | 2 ++ 5 files changed, 7 insertions(+)
Documentation/virt/kvm/api.txt +1 -0
@@ -4245,6 +4245,7 @@ hardware_exit_reason. /* KVM_EXIT_FAIL_ENTRY */ struct { __u64 hardware_entry_failure_reason; + __u32 cpu; /* if KVM_CAP_FAILED_ENTRY_CPU */ } fail_entry; If exit_reason is KVM_EXIT_FAIL_ENTRY, the vcpu could not be run due
arch/x86/kvm/svm.c +1 -0
@@ -4980,6 +4980,7 @@ static int handle_exit(struct kvm_vcpu *vcpu) kvm_run->exit_reason =KVM_EXIT_FAIL_ENTRY ; kvm_run->fail_entry.hardware_entry_failure_reason = svm->vmcb->control.exit_code; + kvm_run->fail_entry.cpu = vcpu->cpu;dump_vmcb (vcpu);return 0 ; }
arch/x86/kvm/vmx/vmx.c +2 -0
@@ -5846,6 +5846,7 @@ static int vmx_handle_exit(struct kvm_vcpu *vcpu) vcpu->run->exit_reason =KVM_EXIT_FAIL_ENTRY ; vcpu->run->fail_entry.hardware_entry_failure_reason = exit_reason; + vcpu->run->fail_entry.cpu = vcpu->cpu;return 0 ; }
@@ -5854,6 +5855,7 @@ static int vmx_handle_exit(struct kvm_vcpu *vcpu) vcpu->run->exit_reason =KVM_EXIT_FAIL_ENTRY ; vcpu->run->fail_entry.hardware_entry_failure_reason =vmcs_read32 (VM_INSTRUCTION_ERROR ); + vcpu->run->fail_entry.cpu = vcpu->cpu;return 0 ; }
arch/x86/kvm/x86.c +1 -0
@@ -3273,6 +3273,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)case KVM_CAP_GET_MSR_FEATURES :case KVM_CAP_MSR_PLATFORM_INFO :case KVM_CAP_EXCEPTION_PAYLOAD : +case KVM_CAP_FAILED_ENTRY_CPU : r =1 ;break ;case KVM_CAP_SYNC_REGS :
include/uapi/linux/kvm.h +2 -0
@@ -277,6 +277,7 @@ struct kvm_run {/* KVM_EXIT_FAIL_ENTRY */ struct {__u64 hardware_entry_failure_reason; +__u32 cpu; } fail_entry;/* KVM_EXIT_EXCEPTION */ struct {
@@ -1009,6 +1010,7 @@ struct kvm_ppc_resize_hpt {#define KVM_CAP_PPC_GUEST_DEBUG_SSTEP 176#define KVM_CAP_ARM_NISV_TO_USER 177#define KVM_CAP_ARM_INJECT_EXT_DABT 178 +#define KVM_CAP_FAILED_ENTRY_CPU 179#ifdef KVM_CAP_IRQ_ROUTING
Signature · 1 line
2.24.1.735.g03f4e72817-goog
Quoted 11 lines · Jim Mattson
More often than not, a failed VM-entry in a production environment is the result of a defective CPU (at least, insofar as Intel x86 is concerned). To aid in identifying the bad hardware, add the logical CPU to the information provided to userspace on a KVM exit with reason
KVM_EXIT_FAIL_ENTRY. The presence of this additional information is indicated by a new capability,KVM_CAP_FAILED_ENTRY_CPU.
Quoted 4 lines
Signed-off-by Jim Mattson <[email protected]>Reviewed-by Oliver Upton <[email protected]>Cc Liran Alon <[email protected]>Cc Paolo Bonzini <[email protected]>
BTW, one could argue that receiving an unexpected exit-reason (i.e. KVM_INTERNAL_ERROR_UNEXPECTED_EXIT_REASON) could also only occur in production either from a KVM bug or from a defective CPU. Similar to failed VM-entry. Should we add similar behaviour to that as-well?
-Liran
Quoted 87 lines
---
Documentation/virt/kvm/api.txt | 1 + arch/x86/kvm/svm.c | 1 + arch/x86/kvm/vmx/vmx.c | 2 ++ arch/x86/kvm/x86.c | 1 + include/uapi/linux/kvm.h | 2 ++ 5 files changed, 7 insertions(+)Documentation/virt/kvm/api.txt +1
-0@@ -4245,6 +4245,7 @@ hardware_exit_reason. /* KVM_EXIT_FAIL_ENTRY */ struct { __u64 hardware_entry_failure_reason; + __u32 cpu; /* if KVM_CAP_FAILED_ENTRY_CPU */ } fail_entry;If
exit_reasonisKVM_EXIT_FAIL_ENTRY, the vcpu could not be run duearch/x86/kvm/svm.c +1
-0@@ -4980,6 +4980,7 @@ static int handle_exit(struct kvm_vcpu *vcpu) kvm_run->exit_reason =KVM_EXIT_FAIL_ENTRY ; kvm_run->fail_entry.hardware_entry_failure_reason = svm->vmcb->control.exit_code; + kvm_run->fail_entry.cpu = vcpu->cpu;dump_vmcb (vcpu);return 0 ; }arch/x86/kvm/vmx/vmx.c +2
-0@@ -5846,6 +5846,7 @@ static int vmx_handle_exit(struct kvm_vcpu *vcpu) vcpu->run->exit_reason =KVM_EXIT_FAIL_ENTRY ; vcpu->run->fail_entry.hardware_entry_failure_reason = exit_reason; + vcpu->run->fail_entry.cpu = vcpu->cpu;return 0 ; }@@ -5854,6 +5855,7 @@ static int vmx_handle_exit(struct kvm_vcpu *vcpu) vcpu->run->exit_reason =KVM_EXIT_FAIL_ENTRY ; vcpu->run->fail_entry.hardware_entry_failure_reason =vmcs_read32 (VM_INSTRUCTION_ERROR ); + vcpu->run->fail_entry.cpu = vcpu->cpu;return 0 ; }arch/x86/kvm/x86.c +1
-0@@ -3273,6 +3273,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)case KVM_CAP_GET_MSR_FEATURES :case KVM_CAP_MSR_PLATFORM_INFO :case KVM_CAP_EXCEPTION_PAYLOAD : +case KVM_CAP_FAILED_ENTRY_CPU : r =1 ;break ;case KVM_CAP_SYNC_REGS :include/uapi/linux/kvm.h +2
-0@@ -277,6 +277,7 @@ struct kvm_run {/* KVM_EXIT_FAIL_ENTRY */ struct {__u64 hardware_entry_failure_reason; +__u32 cpu; } fail_entry;/* KVM_EXIT_EXCEPTION */ struct {@@ -1009,6 +1010,7 @@ struct kvm_ppc_resize_hpt {#define KVM_CAP_PPC_GUEST_DEBUG_SSTEP 176#define KVM_CAP_ARM_NISV_TO_USER 177#define KVM_CAP_ARM_INJECT_EXT_DABT 178 +#define KVM_CAP_FAILED_ENTRY_CPU 179#ifdef KVM_CAP_IRQ_ROUTING Signature · 1 line
2.24.1.735.g03f4e72817-goog
Quoted 17 lines · Liran Alon
Quoted 12 lines · Jim Mattson
More often than not, a failed VM-entry in a production environment is the result of a defective CPU (at least, insofar as Intel x86 is concerned). To aid in identifying the bad hardware, add the logical CPU to the information provided to userspace on a KVM exit with reason
KVM_EXIT_FAIL_ENTRY. The presence of this additional information is indicated by a new capability,KVM_CAP_FAILED_ENTRY_CPU.Quoted 4 lines
Signed-off-by Jim Mattson <[email protected]>Reviewed-by Oliver Upton <[email protected]>Cc Liran Alon <[email protected]>Cc Paolo Bonzini <[email protected]>
Quoted 7 lines
Reviewed-by Liran Alon <[email protected]>BTW, one could argue that receiving an unexpected exit-reason (i.e.
KVM_INTERNAL_ERROR_UNEXPECTED_EXIT_REASON) could also only occur in production either from a KVM bug or from a defective CPU. Similar to failed VM-entry. Should we add similar behaviour to that as-well?-Liran
That's a good point. We had one case of numerous VM-exits for INIT, and I'm pretty sure that was a defective CPU too.
Paolo Bonzini
maintainerQuoted 2 lines · Jim Mattson
That's a good point. We had one case of numerous VM-exits for INIT, and I'm pretty sure that was a defective CPU too.
We too, and that's the only conclusion we could reach. And the other one that I can remember was KVM_INTERNAL_ERROR_DELIVERY_EV with EPT_VIOLATION exits, so I would add it to all KVM_EXIT_INTERNAL_ERROR.
Nowadays KVM_EXIT_FAIL_ENTRY would probably also be an internal error, however it was somewhat more frequent back before Intel CPUs had unrestricted guest support.
Paolo