accel/kvm: notify management of vCPU run errors
cover accel/kvm: notify management of vCPU run errors
From: Tao Cui <[email protected]>
Today, when a KVM vCPU exits with KVM_EXIT_INTERNAL_ERROR, KVM_EXIT_UNKNOWN, or a failed KVM_RUN ioctl, QEMU logs to stderr and calls vm_stop(RUN_STATE_INTERNAL_ERROR). The management layer then only sees the derived STOP event; it cannot tell which vCPU failed nor obtain the kernel suberror / extra data. This gap is even called out by a FIXME in kvm_handle_internal_error().
This series adds a KVM_VCPU_ERROR QMP event, emitted before the VM is stopped, that carries the failing vCPU (cpu-index, qom-path), an error category (KvmVcpuErrorReason) and, for KVM_EXIT_INTERNAL_ERROR, the kernel suberror and extra data words.
Patch 1 defines the event and the KvmVcpuErrorReason enum in
qapi/run-state.json.
Patch 2 wires it up at the three unrecoverable-exit points in
accel/kvm/kvm-all.c and removes the FIXME.
Tao Cui (2):
qapi: add KVM_VCPU_ERROR event
accel/kvm: emit KVM_VCPU_ERROR on unrecoverable vCPU exits
accel/kvm/kvm-all.c | 46 +++++++++++++++++++++++++++++++++--- qapi/run-state.json | 57 +++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 100 insertions(+), 3 deletions(-)
Signature · 1 line
2.43.0
PATCH 1/2 qapi: add KVM_VCPU_ERROR event
From: Tao Cui <[email protected]>
When a KVM vCPU exits with an unrecoverable error (KVM_EXIT_INTERNAL_ERROR, KVM_EXIT_UNKNOWN, or a failed KVM_RUN ioctl), QEMU currently only logs to stderr and stops the VM with run state "internal-error". The management layer must infer what happened from the subsequent STOP event, with no indication of which vCPU failed or why.
Add a KVM_VCPU_ERROR event, emitted before the VM is stopped, that carries the failing vCPU (cpu-index, qom-path), an error category (KvmVcpuErrorReason) and, for KVM_EXIT_INTERNAL_ERROR, the kernel suberror and extra data words.
--- qapi/run-state.json | 57 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 57 insertions(+)
qapi/run-state.json +57 -0
@@ -732,6 +732,63 @@ 'data': { 'action-required': 'bool', 'recursive': 'bool'} } +## +# @KvmVcpuErrorReason: +# +# Categorizes a KVM vCPU run error reported through @KVM_VCPU_ERROR. +# +# @ioctl-failed: the KVM_RUN ioctl failed; see stderr for the errno. +# +# @unknown: KVM_EXIT_UNKNOWN; the hardware exit reason is logged to +# stderr. +# +# @internal-error: KVM_EXIT_INTERNAL_ERROR; see @suberror and +# @exit-data for details. +# +# Since: 11.2 +## +{ 'enum': 'KvmVcpuErrorReason', + 'data': [ 'ioctl-failed', 'unknown', 'internal-error' ] } + +## +# @KVM_VCPU_ERROR: +# +# Emitted when a KVM vCPU exits with an unrecoverable error, before +# the VM is stopped with run state ``internal-error``. Lets the +# management layer learn which vCPU failed and why, instead of +# inferring it from the subsequent STOP event. +# +# @cpu-index: index of the vCPU that failed. +# +# @qom-path: the QOM path of the vCPU that failed. +# +# @reason: error category. +# +# @suberror: KVM internal suberror. Present only when @reason is +# @internal-error. +# +# @exit-data: extra KVM data words. Present only when @reason is +# @internal-error and the kernel provided extra data. +# +# Since: 11.2 +# +# .. qmp-example:: +# +# <- { "event": "KVM_VCPU_ERROR", +# "data": { "cpu-index": 0, +# "qom-path": "/machine/unattached/device[0]", +# "reason": "internal-error", +# "suberror": 1, +# "exit-data": [ 0 ] }, +# "timestamp": { "seconds": 1267061043, "microseconds": 959568 } } +## +{ 'event': 'KVM_VCPU_ERROR', + 'data': { 'cpu-index': 'int', + 'qom-path': 'str', + 'reason': 'KvmVcpuErrorReason', + '*suberror': 'int', + '*exit-data': ['uint64'] } } + ## # @NotifyVmexitOption: #
Signature · 1 line
2.43.0
PATCH 2/2 accel/kvm: emit KVM_VCPU_ERROR on unrecoverable vCPU exits
In reply to Tao CuiFrom: Tao Cui <[email protected]>
Emit the KVM_VCPU_ERROR event at the three points where a vCPU run becomes unrecoverable:
- KVM_RUN ioctl failure (reason "ioctl-failed") - KVM_EXIT_UNKNOWN (reason "unknown") - KVM_EXIT_INTERNAL_ERROR (reason "internal-error",
with suberror and exit-data)
The KVM_RUN loop in kvm_cpu_exec() runs without the BQL, but QMP event emission goes through the monitor and requires it; take and release the lock around the send, mirroring the KVM_SYSTEM_EVENT_CRASH path.
This resolves the long-standing FIXME in kvm_handle_internal_error().
--- accel/kvm/kvm-all.c | 46 ++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 43 insertions(+), 3 deletions(-)
accel/kvm/kvm-all.c +43 -3
@@ -46,6 +46,7 @@#include "qapi/visitor.h" #include "qapi/qapi-types-common.h" #include "qapi/qapi-visit-common.h" +#include "qapi/qapi-events-run-state.h" #include "system/reset.h" #include "qemu/guest-random.h" #include "system/hw_accel.h"
@@ -3140,6 +3141,38 @@ static void kvm_handle_io(uint16_t port, MemTxAttrs attrs, void *data, int direc } } +/* +* Notify the management layer that a vCPU hit an unrecoverable KVM run +* error. The KVM_RUN loop in kvm_cpu_exec() runs without the BQL, but +* emitting a QMP event goes through the monitor and therefore requires +* it; take/release the lock around the send. +*/ +static void kvm_emit_vcpu_error (CPUState *cpu,KvmVcpuErrorReason reason, +bool has_suberror,int64_t suberror, +uint32_t ndata,const uint64_t *data) +{ + g_autofree char *qom_path = object_get_canonical_path(OBJECT(cpu)); +uint64List *exit_data =NULL ; +uint64List **tail = &exit_data; +uint32_t i; + +for (i =0 ; i < ndata; i++) { +uint64List *node =g_new (uint64List,1 ); + node->value = data[i]; + node->next =NULL ; + *tail = node; + tail = &node->next; + } + +bql_lock (); +qapi_event_send_kvm_vcpu_error (cpu->cpu_index, qom_path, reason, + has_suberror, suberror, + !!ndata, exit_data); +bql_unlock (); + +qapi_free_uint64List (exit_data); +} +static int kvm_handle_internal_error (CPUState *cpu,struct kvm_run *run) {int i;
@@ -3151,6 +3184,12 @@ static int kvm_handle_internal_error(CPUState *cpu, struct kvm_run *run)fprintf (stderr,"extra data[%d]: 0x%016" PRIx64"\n" , i, (uint64_t )run->internal.data[i]); } + +kvm_emit_vcpu_error (cpu,KVM_VCPU_ERROR_REASON_INTERNAL_ERROR , + true, run->internal.suberror, + run->internal.ndata, + (const uint64_t *)run->internal.data); +if (run->internal.suberror ==KVM_INTERNAL_ERROR_EMULATION ) {fprintf (stderr,"emulation failure\n" );if (!kvm_arch_stop_on_emulation_error (cpu)) {
@@ -3158,9 +3197,6 @@ static int kvm_handle_internal_error(CPUState *cpu, struct kvm_run *run)return EXCP_INTERRUPT ; } } -/* FIXME: Should trigger a qmp message to let management know -* something went wrong. -*/ return -1 ; }
@@ -3508,6 +3544,8 @@ int kvm_cpu_exec(CPUState *cpu) "secondary threads offline.\n"); }#endif +kvm_emit_vcpu_error (cpu,KVM_VCPU_ERROR_REASON_IOCTL_FAILED , + false,0 ,0 ,NULL ); ret =-1 ;break ; }
@@ -3543,6 +3581,8 @@ int kvm_cpu_exec(CPUState *cpu)case KVM_EXIT_UNKNOWN :fprintf (stderr,"KVM: unknown exit, hardware reason %" PRIx64"\n" , (uint64_t )run->hw.hardware_exit_reason); +kvm_emit_vcpu_error (cpu,KVM_VCPU_ERROR_REASON_UNKNOWN , + false,0 ,0 ,NULL ); ret =-1 ;break ;case KVM_EXIT_INTERNAL_ERROR :
Signature · 1 line
2.43.0