KVM, SEV: Add KVM_EXIT_SHUTDOWN metadata for SEV-ES
Subsystems (4)
PATCH v3 1/1 KVM, SEV: Add KVM_EXIT_SHUTDOWN metadata for SEV-ES
SEV-ES guests can request termination using the GHCB's MSR protocol. See AMD's GHCB spec section '4.1.13 Termination Request'. Currently when a guest does this the userspace VMM sees an KVM_EXIT_UNKNOWN (-EVINAL) return code from KVM_RUN. By adding a KVM_EXIT_SHUTDOWN_ENTRY to kvm_run struct the userspace VMM can clearly see the guest has requested a SEV-ES termination including the termination reason code set and reason code.
---
V3
* Add Documentation/ update.
* Updated other KVM_EXIT_SHUTDOWN exits to clear ndata and set reason to KVM_SHUTDOWN_REQ.
V2
* Add KVM_CAP_EXIT_SHUTDOWN_REASON check for KVM_CHECK_EXTENSION.
Tested by making an SEV-ES guest call sev_es_terminate() with hardcoded reason code set and reason code and then observing the codes from the userspace VMM in the kvm_run.shutdown.data fields.
Change-Id: I55dcdf0f42bfd70d0e59829ae70c2fb067b60809
--- Documentation/virt/kvm/api.rst | 12 ++++++++++++ arch/x86/kvm/svm/sev.c | 9 +++++++-- arch/x86/kvm/svm/svm.c | 2 ++ arch/x86/kvm/vmx/vmx.c | 2 ++ arch/x86/kvm/x86.c | 2 ++ include/uapi/linux/kvm.h | 13 +++++++++++++ virt/kvm/kvm_main.c | 1 + 7 files changed, 39 insertions(+), 2 deletions(-)
Documentation/virt/kvm/api.rst +12 -0
@@ -7834,3 +7834,15 @@ only be invoked on a VM prior to the creation of VCPUs. At this time, KVM_PMU_CAP_DISABLE is the only capability. Setting this capability will disable PMU virtualization for that VM. Usermode should adjust CPUID leaf 0xA to reflect that the PMU is disabled. + +8.36 KVM_CAP_EXIT_SHUTDOWN_REASON +--------------------------- + +:Capability KVM_CAP_EXIT_SHUTDOWN_REASON +:Architectures: x86 +:Type: vm + +This capability means shutdown metadata may be included in +kvm_run.shutdown when a vCPU exits with KVM_EXIT_SHUTDOWN. This +may help userspace determine the guest's reason for termination and +if the guest should be restarted or an error caused the shutdown.
arch/x86/kvm/svm/sev.c +7 -2
@@ -2735,8 +2735,13 @@ static int sev_handle_vmgexit_msr_protocol(struct vcpu_svm *svm)pr_info ("SEV-ES guest requested termination: %#llx:%#llx\n" , reason_set, reason_code); - ret = -EINVAL ; -break ; + vcpu->run->exit_reason =KVM_EXIT_SHUTDOWN ; + vcpu->run->shutdown.reason =KVM_SHUTDOWN_SEV_TERM ; + vcpu->run->shutdown.ndata =2 ; + vcpu->run->shutdown.data[0 ] = reason_set; + vcpu->run->shutdown.data[1 ] = reason_code; + +return 0 ; }default :/* Error, keep GHCB MSR value as-is */
arch/x86/kvm/svm/svm.c +2 -0
@@ -1953,6 +1953,8 @@ static int shutdown_interception(struct kvm_vcpu *vcpu)kvm_vcpu_reset (vcpu, true); kvm_run->exit_reason =KVM_EXIT_SHUTDOWN ; + vcpu->run->shutdown.reason =KVM_SHUTDOWN_REQ ; + vcpu->run->shutdown.ndata =0 ;return 0 ; }
arch/x86/kvm/vmx/vmx.c +2 -0
@@ -4988,6 +4988,8 @@ static __always_inline int handle_external_interrupt(struct kvm_vcpu *vcpu)static int handle_triple_fault (struct kvm_vcpu *vcpu) { vcpu->run->exit_reason =KVM_EXIT_SHUTDOWN ; + vcpu->run->shutdown.reason =KVM_SHUTDOWN_REQ ; + vcpu->run->shutdown.ndata =0 ; vcpu->mmio_needed =0 ;return 0 ; }
arch/x86/kvm/x86.c +2 -0
@@ -9999,6 +9999,8 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) kvm_x86_ops.nested_ops->triple_fault (vcpu); }else { vcpu->run->exit_reason =KVM_EXIT_SHUTDOWN ; + vcpu->run->shutdown.reason =KVM_SHUTDOWN_REQ ; + vcpu->run->shutdown.ndata =0 ; vcpu->mmio_needed =0 ; r =0 ; goto out;
include/uapi/linux/kvm.h +13 -0
@@ -271,6 +271,12 @@ struct kvm_xen_exit {#define KVM_EXIT_XEN 34#define KVM_EXIT_RISCV_SBI 35 +/* For KVM_EXIT_SHUTDOWN */ +/* Standard VM shutdown request. No additional metadata provided. */ +#define KVM_SHUTDOWN_REQ 0 +/* SEV-ES termination request */ +#define KVM_SHUTDOWN_SEV_TERM 1 +/* For KVM_EXIT_INTERNAL_ERROR */ /* Emulate instruction failed. */ #define KVM_INTERNAL_ERROR_EMULATION 1
@@ -311,6 +317,12 @@ struct kvm_run {struct {__u64 hardware_exit_reason; } hw; +/* KVM_EXIT_SHUTDOWN */ +struct { +__u64 reason; +__u32 ndata; +__u64 data[16 ]; + } shutdown;/* KVM_EXIT_FAIL_ENTRY */ struct {__u64 hardware_entry_failure_reason;
@@ -1145,6 +1157,7 @@ struct kvm_ppc_resize_hpt {#define KVM_CAP_PMU_CAPABILITY 212#define KVM_CAP_DISABLE_QUIRKS2 213#define KVM_CAP_VM_TSC_CONTROL 214 +#define KVM_CAP_EXIT_SHUTDOWN_REASON 215#ifdef KVM_CAP_IRQ_ROUTING
virt/kvm/kvm_main.c +1 -0
@@ -4299,6 +4299,7 @@ static long kvm_vm_ioctl_check_extension_generic(struct kvm *kvm, long arg)case KVM_CAP_CHECK_EXTENSION_VM :case KVM_CAP_ENABLE_CAP_VM :case KVM_CAP_HALT_POLL : +case KVM_CAP_EXIT_SHUTDOWN_REASON :return 1 ;#ifdef CONFIG_KVM_MMIO case KVM_CAP_COALESCED_MMIO :
Signature · 1 line
2.35.1.1094.g7c7d902a7c-goog
Quoted 164 lines · Peter Gonda
SEV-ES guests can request termination using the GHCB's MSR protocol. See AMD's GHCB spec section '4.1.13 Termination Request'. Currently when a guest does this the userspace VMM sees an
KVM_EXIT_UNKNOWN(-EVINAL) return code fromKVM_RUN. By adding aKVM_EXIT_SHUTDOWN_ENTRYtokvm_runstruct the userspace VMM can clearly see the guest has requested a SEV-ES termination including the termination reason code set and reason code.Signed-off-by Peter Gonda <[email protected]>---
V3
* AddDocumentation/update.
* Updated otherKVM_EXIT_SHUTDOWNexits to clear ndata and set reason toKVM_SHUTDOWN_REQ.V2
* AddKVM_CAP_EXIT_SHUTDOWN_REASONcheck forKVM_CHECK_EXTENSION.Tested by making an SEV-ES guest call
sev_es_terminate()with hardcoded reason code set and reason code and then observing the codes from the userspace VMM in the kvm_run.shutdown.data fields.Change-Id: I55dcdf0f42bfd70d0e59829ae70c2fb067b60809
--- Documentation/virt/kvm/api.rst | 12 ++++++++++++ arch/x86/kvm/svm/sev.c | 9 +++++++-- arch/x86/kvm/svm/svm.c | 2 ++ arch/x86/kvm/vmx/vmx.c | 2 ++ arch/x86/kvm/x86.c | 2 ++ include/uapi/linux/kvm.h | 13 +++++++++++++ virt/kvm/kvm_main.c | 1 + 7 files changed, 39 insertions(+), 2 deletions(-)Documentation/virt/kvm/api.rst +12
-0@@ -7834,3 +7834,15 @@ only be invoked on a VM prior to the creation of VCPUs. At this time, KVM_PMU_CAP_DISABLE is the only capability. Setting this capability will disable PMU virtualization for that VM. Usermode should adjust CPUID leaf 0xA to reflect that the PMU is disabled. + +8.36 KVM_CAP_EXIT_SHUTDOWN_REASON +--------------------------- + +:Capability KVM_CAP_EXIT_SHUTDOWN_REASON +:Architectures: x86 +:Type: vm + +This capability means shutdown metadata may be included in +kvm_run.shutdown when a vCPU exits with KVM_EXIT_SHUTDOWN. This +may help userspace determine the guest's reason for termination and +if the guest should be restarted or an error caused the shutdown.arch/x86/kvm/svm/sev.c +7
-2@@ -2735,8 +2735,13 @@ static int sev_handle_vmgexit_msr_protocol(struct vcpu_svm *svm)pr_info ("SEV-ES guest requested termination: %#llx:%#llx\n" , reason_set, reason_code); - ret = -EINVAL ; -break ; + vcpu->run->exit_reason =KVM_EXIT_SHUTDOWN ; + vcpu->run->shutdown.reason =KVM_SHUTDOWN_SEV_TERM ; + vcpu->run->shutdown.ndata =2 ; + vcpu->run->shutdown.data[0 ] = reason_set; + vcpu->run->shutdown.data[1 ] = reason_code; + +return 0 ; }default :/* Error, keep GHCB MSR value as-is */ arch/x86/kvm/svm/svm.c +2
-0@@ -1953,6 +1953,8 @@ static int shutdown_interception(struct kvm_vcpu *vcpu)kvm_vcpu_reset (vcpu, true); kvm_run->exit_reason =KVM_EXIT_SHUTDOWN ; + vcpu->run->shutdown.reason =KVM_SHUTDOWN_REQ ; + vcpu->run->shutdown.ndata =0 ;return 0 ; }arch/x86/kvm/vmx/vmx.c +2
-0@@ -4988,6 +4988,8 @@ static __always_inline int handle_external_interrupt(struct kvm_vcpu *vcpu)static int handle_triple_fault (struct kvm_vcpu *vcpu) { vcpu->run->exit_reason =KVM_EXIT_SHUTDOWN ; + vcpu->run->shutdown.reason =KVM_SHUTDOWN_REQ ; + vcpu->run->shutdown.ndata =0 ; vcpu->mmio_needed =0 ;return 0 ; }arch/x86/kvm/x86.c +2
-0@@ -9999,6 +9999,8 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) kvm_x86_ops.nested_ops->triple_fault (vcpu); }else { vcpu->run->exit_reason =KVM_EXIT_SHUTDOWN ; + vcpu->run->shutdown.reason =KVM_SHUTDOWN_REQ ; + vcpu->run->shutdown.ndata =0 ; vcpu->mmio_needed =0 ; r =0 ; goto out;include/uapi/linux/kvm.h +13
-0@@ -271,6 +271,12 @@ struct kvm_xen_exit {#define KVM_EXIT_XEN 34#define KVM_EXIT_RISCV_SBI 35 +/* For KVM_EXIT_SHUTDOWN */ +/* Standard VM shutdown request. No additional metadata provided. */ +#define KVM_SHUTDOWN_REQ 0 +/* SEV-ES termination request */ +#define KVM_SHUTDOWN_SEV_TERM 1 +/* For KVM_EXIT_INTERNAL_ERROR */ /* Emulate instruction failed. */ #define KVM_INTERNAL_ERROR_EMULATION 1@@ -311,6 +317,12 @@ struct kvm_run {struct {__u64 hardware_exit_reason; } hw; +/* KVM_EXIT_SHUTDOWN */ +struct { +__u64 reason; +__u32 ndata; +__u64 data[16 ]; + } shutdown;/* KVM_EXIT_FAIL_ENTRY */ struct {__u64 hardware_entry_failure_reason;@@ -1145,6 +1157,7 @@ struct kvm_ppc_resize_hpt {#define KVM_CAP_PMU_CAPABILITY 212#define KVM_CAP_DISABLE_QUIRKS2 213#define KVM_CAP_VM_TSC_CONTROL 214 +#define KVM_CAP_EXIT_SHUTDOWN_REASON 215#ifdef KVM_CAP_IRQ_ROUTING virt/kvm/kvm_main.c +1
-1@@ -4299,6 +4299,7 @@ static long kvm_vm_ioctl_check_extension_generic(struct kvm *kvm, long arg)case KVM_CAP_CHECK_EXTENSION_VM :case KVM_CAP_ENABLE_CAP_VM :case KVM_CAP_HALT_POLL : +case KVM_CAP_EXIT_SHUTDOWN_REASON :return 1 ;#ifdef CONFIG_KVM_MMIO case KVM_CAP_COALESCED_MMIO : --
Quoted 2 lines
2.35.1.1094.g7c7d902a7c-goog
Sean Christopherson
maintainer+Paolo and Vitaly
In the future, I highly recommend using scripts/get_maintainers.pl.
Quoted 15 lines · Peter Gonda
SEV-ES guests can request termination using the GHCB's MSR protocol. See AMD's GHCB spec section '4.1.13 Termination Request'. Currently when a guest does this the userspace VMM sees an
KVM_EXIT_UNKNOWN(-EVINAL) return code fromKVM_RUN. By adding aKVM_EXIT_SHUTDOWN_ENTRYtokvm_runstruct the userspace VMM can clearly see the guest has requested a SEV-ES termination including the termination reason code set and reason code.Signed-off-by Peter Gonda <[email protected]>---
arch/x86/kvm/svm/sev.c +0
-0
arch/x86/kvm/svm/sev.c static int sev_handle_vmgexit_msr_protocol(struct vcpu_svm *svm) · line 2736
Quoted 2 lines
arch/x86/kvm/svm/sev.c +0
-0@@ -2735,8 +2735,13 @@ static int sev_handle_vmgexit_msr_protocol(struct vcpu_svm *svm)pr_info ("SEV-ES guest requested termination: %#llx:%#llx\n" , reason_set, reason_code);
This pr_info() should be removed. A malicious usersepace could spam the kernel by constantly running a vCPU that requests termination.
arch/x86/kvm/svm/sev.c static int sev_handle_vmgexit_msr_protocol(struct vcpu_svm *svm) · line 2742
Quoted 7 lines
arch/x86/kvm/svm/sev.c +5
-2- ret = -EINVAL ; -break ; + vcpu->run->exit_reason =KVM_EXIT_SHUTDOWN ; + vcpu->run->shutdown.reason =KVM_SHUTDOWN_SEV_TERM ; + vcpu->run->shutdown.ndata =2 ; + vcpu->run->shutdown.data[0 ] = reason_set; + vcpu->run->shutdown.data[1 ] = reason_code;
Does KVM really need to split the reason_set and reason_code? Without reading the spec, it's not even clear what "set" means. Assuming it's something like "the reason code is valid", then I don't see any reason (lol) to split the two. If we do split them, then arguably the reason_code should be filled if and only if reason_set is true, and that's just extra work.
Dumping the entire raw "MSR" would simplify this code and the helper to fill the termination request.
Though I'm not actually sure KVM_EXIT_SHUTDOWN is the best exit reason. The exit reason used for Hyper-V's paravirt stuff is arguably more appropriate, because in this case the guest hasn't actually encountered shutdown, rather it has requested termination.
/* KVM_EXIT_SYSTEM_EVENT */
struct {
#define KVM_SYSTEM_EVENT_SHUTDOWN 1
#define KVM_SYSTEM_EVENT_RESET 2
#define KVM_SYSTEM_EVENT_CRASH 3
__u32 type;
__u64 flags;
} system_event;Though looking at system_event, isn't that missing padding after type? Ah, KVM doesn't actually populate flags, wonderful. Maybe we can get away with tweaking it to:
struct {
__u32 type;
__u32 ndata;
__u64 data[16];
} system_event;Quoted 23 lines
arch/x86/kvm/svm/sev.c +2
-0+ +return 0 ; }default :/* Error, keep GHCB MSR value as-is */ arch/x86/kvm/svm/svm.c +2
-0@@ -1953,6 +1953,8 @@ static int shutdown_interception(struct kvm_vcpu *vcpu)kvm_vcpu_reset (vcpu, true); kvm_run->exit_reason =KVM_EXIT_SHUTDOWN ; + vcpu->run->shutdown.reason =KVM_SHUTDOWN_REQ ; + vcpu->run->shutdown.ndata =0 ;return 0 ; }arch/x86/kvm/vmx/vmx.c +0
-0
arch/x86/kvm/vmx/vmx.c static __always_inline int handle_external_interrupt(struct kvm_vcpu *vcpu) · line 4992
Quoted 5 lines
arch/x86/kvm/vmx/vmx.c +2
-0@@ -4988,6 +4988,8 @@ static __always_inline int handle_external_interrupt(struct kvm_vcpu *vcpu)static int handle_triple_fault (struct kvm_vcpu *vcpu) { vcpu->run->exit_reason =KVM_EXIT_SHUTDOWN ; + vcpu->run->shutdown.reason =KVM_SHUTDOWN_REQ ; + vcpu->run->shutdown.ndata =0 ;
If we do piggyback KVM_EXIT_SHUTDOWN, a helper to fill the shutdown reason is warranted, similar to prepare_emulation_failure_exit(). If the raw GHCB MSR is dumped, the helper can be:
void kvm_prepare_shutdown_exit(struct kvm_vcpu *vcpu, u64 *data);
where a NULL @data means ndata=0, and a non-NULL @data means ndata=1.
Quoted 21 lines
arch/x86/kvm/vmx/vmx.c +0
-0vcpu->mmio_needed =0 ;return 0 ; }arch/x86/kvm/x86.c +2
-0@@ -9999,6 +9999,8 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) kvm_x86_ops.nested_ops->triple_fault (vcpu); }else { vcpu->run->exit_reason =KVM_EXIT_SHUTDOWN ; + vcpu->run->shutdown.reason =KVM_SHUTDOWN_REQ ; + vcpu->run->shutdown.ndata =0 ; vcpu->mmio_needed =0 ; r =0 ; goto out;include/uapi/linux/kvm.h +0
-0
include/uapi/linux/kvm.h struct kvm_xen_exit { · line 276
Quoted 6 lines
include/uapi/linux/kvm.h +3
-0@@ -271,6 +271,12 @@ struct kvm_xen_exit {#define KVM_EXIT_XEN 34#define KVM_EXIT_RISCV_SBI 35 +/* For KVM_EXIT_SHUTDOWN */ +/* Standard VM shutdown request. No additional metadata provided. */ +#define KVM_SHUTDOWN_REQ 0
I dislike the "REQ" part. In a triple fault scenario, the guest isn't requesting anything. KVM does "request" shutdown, but that's not a request from the guest's perspective and is purely a KVM implementation detail. I'm having trouble coming up with a decent alternative, but that's probably another indicator that piggybacking KVM_EXIT_SHUTDOWN may not be appropriate.
Quoted 7 lines
include/uapi/linux/kvm.h +3
-0+/* SEV-ES termination request */ +#define KVM_SHUTDOWN_SEV_TERM 1 +/* For KVM_EXIT_INTERNAL_ERROR */ /* Emulate instruction failed. */ #define KVM_INTERNAL_ERROR_EMULATION 1
include/uapi/linux/kvm.h struct kvm_run { · line 323
Quoted 7 lines
include/uapi/linux/kvm.h +4
-0@@ -311,6 +317,12 @@ struct kvm_run {struct {__u64 hardware_exit_reason; } hw; +/* KVM_EXIT_SHUTDOWN */ +struct { +__u64 reason; +__u32 ndata;
This needs
__u32 pad;
to ensure data[16] is aligned regardless of compiler. Though it's simpler to just do:
struct {
__u32 reason;
__u32 ndata;
__u64 data[16];
}because 4 billion reasons is probably enough :-)
Quoted 28 lines
include/uapi/linux/kvm.h +3
-0+__u64 data[16 ]; + } shutdown;/* KVM_EXIT_FAIL_ENTRY */ struct {__u64 hardware_entry_failure_reason;@@ -1145,6 +1157,7 @@ struct kvm_ppc_resize_hpt {#define KVM_CAP_PMU_CAPABILITY 212#define KVM_CAP_DISABLE_QUIRKS2 213#define KVM_CAP_VM_TSC_CONTROL 214 +#define KVM_CAP_EXIT_SHUTDOWN_REASON 215#ifdef KVM_CAP_IRQ_ROUTING virt/kvm/kvm_main.c +1
-0@@ -4299,6 +4299,7 @@ static long kvm_vm_ioctl_check_extension_generic(struct kvm *kvm, long arg)case KVM_CAP_CHECK_EXTENSION_VM :case KVM_CAP_ENABLE_CAP_VM :case KVM_CAP_HALT_POLL : +case KVM_CAP_EXIT_SHUTDOWN_REASON :return 1 ;#ifdef CONFIG_KVM_MMIO case KVM_CAP_COALESCED_MMIO :Signature · 1 line
2.35.1.1094.g7c7d902a7c-goog
Paolo Bonzini
maintainerQuoted 13 lines · Sean Christopherson
/* KVM_EXIT_SYSTEM_EVENT */ struct { #define KVM_SYSTEM_EVENT_SHUTDOWN 1 #define KVM_SYSTEM_EVENT_RESET 2 #define KVM_SYSTEM_EVENT_CRASH 3 __u32 type; __u64 flags; } system_event;Though looking at
system_event, isn't that missing padding after type? Ah, KVM doesn't actually populate flags, wonderful. Maybe we can get away with tweaking it to:
Quoted 5 lines
struct {
__u32 type;
__u32 ndata;
__u64 data[16];
} system_event;Yes, you can do that and say that the ndata is only valid e.g. if bit 31 is set in type.
I agree with reusing KVM_EXIT_SYSTEM_EVENT, that would be a much smaller patch. Sorry about that Peter.
Paolo
Quoted 21 lines · Paolo Bonzini
Quoted 13 lines · Sean Christopherson
/* KVM_EXIT_SYSTEM_EVENT */ struct { #define KVM_SYSTEM_EVENT_SHUTDOWN 1 #define KVM_SYSTEM_EVENT_RESET 2 #define KVM_SYSTEM_EVENT_CRASH 3 __u32 type; __u64 flags; } system_event;Though looking at
system_event, isn't that missing padding after type? Ah, KVM doesn't actually populate flags, wonderful. Maybe we can get away with tweaking it to:Quoted 5 lines
struct { __u32 type; __u32 ndata; __u64 data[16]; } system_event;
Quoted 5 lines
Yes, you can do that and say that the ndata is only valid e.g. if bit 31 is set in type.
I agree with reusing
KVM_EXIT_SYSTEM_EVENT, that would be a much smaller patch. Sorry about that Peter.
KVM_EXIT_SYSTEM_EVENT makes sense. No worries, I appreciate the very detailed feedback. I'll update this for a V4 and have a second patch to address the pr_info()
Quoted 3 lines
Paolo
Quoted 15 lines · Sean Christopherson
+Paolo and Vitaly
In the future, I highly recommend using
scripts/get_maintainers.pl.Quoted 9 lines · Peter Gonda
SEV-ES guests can request termination using the GHCB's MSR protocol. See AMD's GHCB spec section '4.1.13 Termination Request'. Currently when a guest does this the userspace VMM sees an
KVM_EXIT_UNKNOWN(-EVINAL) return code fromKVM_RUN. By adding aKVM_EXIT_SHUTDOWN_ENTRYtokvm_runstruct the userspace VMM can clearly see the guest has requested a SEV-ES termination including the termination reason code set and reason code.Signed-off-by Peter Gonda <[email protected]>
Quoted 11 lines
arch/x86/kvm/svm/sev.c static int sev_handle_vmgexit_msr_protocol(struct vcpu_svm *svm) · line 2736
Quoted 8 lines
---
arch/x86/kvm/svm/sev.c +0
-0@@ -2735,8 +2735,13 @@ static int sev_handle_vmgexit_msr_protocol(struct vcpu_svm *svm)pr_info ("SEV-ES guest requested termination: %#llx:%#llx\n" , reason_set, reason_code);This
pr_info()should be removed. A malicious usersepace could spam the kernel by constantly running a vCPU that requests termination.
Ah, good catch. But actually, I've found this specific pr_info _very_ useful in debugging. Sean, would you be OK to convert it to a rate limited print?
Quoted 8 lines
arch/x86/kvm/svm/sev.c static int sev_handle_vmgexit_msr_protocol(struct vcpu_svm *svm) · line 2742
Quoted 7 lines
arch/x86/kvm/svm/sev.c +5
-2- ret = -EINVAL ; -break ; + vcpu->run->exit_reason =KVM_EXIT_SHUTDOWN ; + vcpu->run->shutdown.reason =KVM_SHUTDOWN_SEV_TERM ; + vcpu->run->shutdown.ndata =2 ; + vcpu->run->shutdown.data[0 ] = reason_set; + vcpu->run->shutdown.data[1 ] = reason_code;
Quoted 5 lines
Does KVM really need to split the
reason_setandreason_code? Without reading the spec, it's not even clear what "set" means. Assuming it's something like "the reason code is valid", then I don't see any reason (lol) to split the two. If we do split them, then arguably thereason_codeshould be filled if and only ifreason_setis true, and that's just extra work.
I think KVM should split the reason_set and reason_code. This code is based on the GHCB spec after all. And reason_set and reason_code are both a part of the GHCB spec. But I agree, folks shouldn't have to go to the spec to understand what reason_set and reason_code are. Rather than not splitting them up, can we add comments in the source to explain what they mean?
Also, my understanding from reading the spec is that reason_code should always be filled, even when reason_set is 0. See below. But basically, you can have reason_set: 0 and reason_code: non-zero.
Quoting the spec:
The reason code set is meant to provide hypervisors with their own termination SEV-ES Guest-Hypervisor Communication Block Standardization reason codes. This document defines and owns reason code set 0x0 and the following reason codes (GHCBData[23:16]):0x00 – General termination request0x01 – SEV-ES / GHCB Protocol range is not supported. 0x02 – SEV-SNP features not supported
Quoted 17 lines · Marc Orr
Quoted 15 lines · Sean Christopherson
+Paolo and Vitaly
In the future, I highly recommend using
scripts/get_maintainers.pl.Quoted 9 lines · Peter Gonda
SEV-ES guests can request termination using the GHCB's MSR protocol. See AMD's GHCB spec section '4.1.13 Termination Request'. Currently when a guest does this the userspace VMM sees an
KVM_EXIT_UNKNOWN(-EVINAL) return code fromKVM_RUN. By adding aKVM_EXIT_SHUTDOWN_ENTRYtokvm_runstruct the userspace VMM can clearly see the guest has requested a SEV-ES termination including the termination reason code set and reason code.Signed-off-by Peter Gonda <[email protected]>
Quoted 11 lines
Quoted 11 lines
arch/x86/kvm/svm/sev.c static int sev_handle_vmgexit_msr_protocol(struct vcpu_svm *svm) · line 2736
Quoted 8 lines
---
arch/x86/kvm/svm/sev.c +0
-0@@ -2735,8 +2735,13 @@ static int sev_handle_vmgexit_msr_protocol(struct vcpu_svm *svm)pr_info ("SEV-ES guest requested termination: %#llx:%#llx\n" , reason_set, reason_code);This
pr_info()should be removed. A malicious usersepace could spam the kernel by constantly running a vCPU that requests termination.
Though... this patch makes this pr_info redundant! Since we'll now report this in userspace. Actually, I'd be OK to remove it.
Quoted 29 lines
Ah, good catch. But actually, I've found this specific
pr_info_very_ useful in debugging. Sean, would you be OK to convert it to a rate limited print?Quoted 8 lines
arch/x86/kvm/svm/sev.c static int sev_handle_vmgexit_msr_protocol(struct vcpu_svm *svm) · line 2742
Quoted 7 lines
arch/x86/kvm/svm/sev.c +5
-2- ret = -EINVAL ; -break ; + vcpu->run->exit_reason =KVM_EXIT_SHUTDOWN ; + vcpu->run->shutdown.reason =KVM_SHUTDOWN_SEV_TERM ; + vcpu->run->shutdown.ndata =2 ; + vcpu->run->shutdown.data[0 ] = reason_set; + vcpu->run->shutdown.data[1 ] = reason_code;Quoted 5 lines
Does KVM really need to split the
reason_setandreason_code? Without reading the spec, it's not even clear what "set" means. Assuming it's something like "the reason code is valid", then I don't see any reason (lol) to split the two. If we do split them, then arguably thereason_codeshould be filled if and only ifreason_setis true, and that's just extra work.I think KVM should split the
reason_setandreason_code. This code is based on the GHCB spec after all. Andreason_setandreason_codeare both a part of the GHCB spec. But I agree, folks shouldn't have to go to the spec to understand whatreason_setandreason_codeare. Rather than not splitting them up, can we add comments in the source to explain what they mean?Also, my understanding from reading the spec is that
reason_codeshould always be filled, even whenreason_setis 0. See below. But basically, you can havereason_set: 0 andreason_code: non-zero.
Quoted 8 lines
Quoting the spec:
The reason code set is meant to provide hypervisors with their own termination SEV-ES Guest-Hypervisor Communication Block Standardization reason codes. This document defines and owns reason code set0x0and the following reason codes (GHCBData[23:16]):0x00– General termination request0x01– SEV-ES / GHCB Protocol range is not supported.0x02– SEV-SNP features not supported
Reading this again, I now see that "reason_set" sounds like "The reason code is set". I bet that's how Sean read it during his review. So yeah, this needs comments :-)!
Quoted 31 lines · Marc Orr
Quoted 17 lines · Marc Orr
Quoted 15 lines · Sean Christopherson
+Paolo and Vitaly
In the future, I highly recommend using
scripts/get_maintainers.pl.Quoted 9 lines · Peter Gonda
SEV-ES guests can request termination using the GHCB's MSR protocol. See AMD's GHCB spec section '4.1.13 Termination Request'. Currently when a guest does this the userspace VMM sees an
KVM_EXIT_UNKNOWN(-EVINAL) return code fromKVM_RUN. By adding aKVM_EXIT_SHUTDOWN_ENTRYtokvm_runstruct the userspace VMM can clearly see the guest has requested a SEV-ES termination including the termination reason code set and reason code.Signed-off-by Peter Gonda <[email protected]>Quoted 11 lines
Quoted 11 lines
arch/x86/kvm/svm/sev.c static int sev_handle_vmgexit_msr_protocol(struct vcpu_svm *svm) · line 2736
Quoted 8 lines
---
arch/x86/kvm/svm/sev.c +0
-0@@ -2735,8 +2735,13 @@ static int sev_handle_vmgexit_msr_protocol(struct vcpu_svm *svm)pr_info ("SEV-ES guest requested termination: %#llx:%#llx\n" , reason_set, reason_code);This
pr_info()should be removed. A malicious usersepace could spam the kernel by constantly running a vCPU that requests termination.
Quoted 2 lines
Though... this patch makes this
pr_inforedundant! Since we'll now report this in userspace. Actually, I'd be OK to remove it.
I'll make this 2 patches. This current patch and another to rate limit this pr_info() I think this patch is doing a lot already so would prefer to just add a second. Is that reasonable?
Quoted 39 lines
Quoted 30 lines
Ah, good catch. But actually, I've found this specific
pr_info_very_ useful in debugging. Sean, would you be OK to convert it to a rate limited print?Quoted 8 lines
arch/x86/kvm/svm/sev.c static int sev_handle_vmgexit_msr_protocol(struct vcpu_svm *svm) · line 2742
Quoted 7 lines
arch/x86/kvm/svm/sev.c +5
-2- ret = -EINVAL ; -break ; + vcpu->run->exit_reason =KVM_EXIT_SHUTDOWN ; + vcpu->run->shutdown.reason =KVM_SHUTDOWN_SEV_TERM ; + vcpu->run->shutdown.ndata =2 ; + vcpu->run->shutdown.data[0 ] = reason_set; + vcpu->run->shutdown.data[1 ] = reason_code;Quoted 5 lines
Does KVM really need to split the
reason_setandreason_code? Without reading the spec, it's not even clear what "set" means. Assuming it's something like "the reason code is valid", then I don't see any reason (lol) to split the two. If we do split them, then arguably thereason_codeshould be filled if and only ifreason_setis true, and that's just extra work.I think KVM should split the
reason_setandreason_code. This code is based on the GHCB spec after all. Andreason_setandreason_codeare both a part of the GHCB spec. But I agree, folks shouldn't have to go to the spec to understand whatreason_setandreason_codeare. Rather than not splitting them up, can we add comments in the source to explain what they mean?Also, my understanding from reading the spec is that
reason_codeshould always be filled, even whenreason_setis 0. See below. But basically, you can havereason_set: 0 andreason_code: non-zero.Quoted 8 lines
Quoting the spec:
The reason code set is meant to provide hypervisors with their own termination SEV-ES Guest-Hypervisor Communication Block Standardization reason codes. This document defines and owns reason code set0x0and the following reason codes (GHCBData[23:16]):0x00– General termination request0x01– SEV-ES / GHCB Protocol range is not supported.0x02– SEV-SNP features not supported
Quoted 3 lines
Reading this again, I now see that "
reason_set" sounds like "The reason code is set". I bet that's how Sean read it during his review. So yeah, this needs comments :-)!
I'll add comments but I agree with Marc. These are part of the GHCB spec so for the very specific SEV-ES termination reason we should include all the data the spec allows. Sounds OK?
Sean Christopherson
maintainerQuoted 32 lines · Peter Gonda
Quoted 31 lines · Marc Orr
Quoted 28 lines · Marc Orr
Quoted 26 lines · Sean Christopherson
+Paolo and Vitaly
In the future, I highly recommend using
scripts/get_maintainers.pl.Quoted 15 lines · Peter Gonda
SEV-ES guests can request termination using the GHCB's MSR protocol. See AMD's GHCB spec section '4.1.13 Termination Request'. Currently when a guest does this the userspace VMM sees an
KVM_EXIT_UNKNOWN(-EVINAL) return code fromKVM_RUN. By adding aKVM_EXIT_SHUTDOWN_ENTRYtokvm_runstruct the userspace VMM can clearly see the guest has requested a SEV-ES termination including the termination reason code set and reason code.Signed-off-by Peter Gonda <[email protected]>---
arch/x86/kvm/svm/sev.c +0
-0arch/x86/kvm/svm/sev.c static int sev_handle_vmgexit_msr_protocol(struct vcpu_svm *svm) · line 2736
Quoted 2 lines
arch/x86/kvm/svm/sev.c +0
-0@@ -2735,8 +2735,13 @@ static int sev_handle_vmgexit_msr_protocol(struct vcpu_svm *svm)pr_info ("SEV-ES guest requested termination: %#llx:%#llx\n" , reason_set, reason_code);This
pr_info()should be removed. A malicious usersepace could spam the kernel by constantly running a vCPU that requests termination.
Quoted 6 lines
Quoted 2 lines
Though... this patch makes this
pr_inforedundant! Since we'll now report this in userspace. Actually, I'd be OK to remove it.I'll make this 2 patches. This current patch and another to rate limit this
pr_info()I think this patch is doing a lot already so would prefer to just add a second. Is that reasonable?
I strongly prefer removing the pr_info() entirely. As Marc pointed out, the info is redundant when KVM properly reports the issue. And worse, the info is useless unless there's exactly one VM running. Even then, it doesn't capture which vCPU failed. This is exactly why Jim, myself, and others, have been pushing to avoid using dmesg to report guest errors. They're helpful for initial development, but dead weight for production, and if they're helpful for development then odds are good that having proper reporting in production would also be valuable.
Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
Quoting the spec:
The reason code set is meant to provide hypervisors with their own termination SEV-ES Guest-Hypervisor Communication Block Standardization reason codes. This document defines and owns reason code set0x0and the following reason codes (GHCBData[23:16]):0x00– General termination request0x01– SEV-ES / GHCB Protocol range is not supported.0x02– SEV-SNP features not supported
Quoted 7 lines
Quoted 3 lines
Reading this again, I now see that "
reason_set" sounds like "The reason code is set". I bet that's how Sean read it during his review. So yeah, this needs comments :-)!I'll add comments but I agree with Marc. These are part of the GHCB spec so for the very specific SEV-ES termination reason we should include all the data the spec allows. Sounds OK?
Ugh, so "set" means "set of reason codes"? That's heinous naming. I don't have a strong objection to splitting, but at the same time, why not punt it to userspace? Userspace is obviously going to have to understand what the hell "set" means anyways...
Quoted 41 lines · Sean Christopherson
Quoted 32 lines · Peter Gonda
Quoted 31 lines · Marc Orr
Quoted 28 lines · Marc Orr
Quoted 26 lines · Sean Christopherson
+Paolo and Vitaly
In the future, I highly recommend using
scripts/get_maintainers.pl.Quoted 15 lines · Peter Gonda
SEV-ES guests can request termination using the GHCB's MSR protocol. See AMD's GHCB spec section '4.1.13 Termination Request'. Currently when a guest does this the userspace VMM sees an
KVM_EXIT_UNKNOWN(-EVINAL) return code fromKVM_RUN. By adding aKVM_EXIT_SHUTDOWN_ENTRYtokvm_runstruct the userspace VMM can clearly see the guest has requested a SEV-ES termination including the termination reason code set and reason code.Signed-off-by Peter Gonda <[email protected]>---
arch/x86/kvm/svm/sev.c +0
-0arch/x86/kvm/svm/sev.c static int sev_handle_vmgexit_msr_protocol(struct vcpu_svm *svm) · line 2736
Quoted 2 lines
arch/x86/kvm/svm/sev.c +0
-0@@ -2735,8 +2735,13 @@ static int sev_handle_vmgexit_msr_protocol(struct vcpu_svm *svm)pr_info ("SEV-ES guest requested termination: %#llx:%#llx\n" , reason_set, reason_code);This
pr_info()should be removed. A malicious usersepace could spam the kernel by constantly running a vCPU that requests termination.Quoted 6 lines
Quoted 2 lines
Though... this patch makes this
pr_inforedundant! Since we'll now report this in userspace. Actually, I'd be OK to remove it.I'll make this 2 patches. This current patch and another to rate limit this
pr_info()I think this patch is doing a lot already so would prefer to just add a second. Is that reasonable?
Quoted 7 lines
I strongly prefer removing the
pr_info()entirely. As Marc pointed out, the info is redundant when KVM properly reports the issue. And worse, the info is useless unless there's exactly one VM running. Even then, it doesn't capture which vCPU failed. This is exactly why Jim, myself, and others, have been pushing to avoid using dmesg to report guest errors. They're helpful for initial development, but dead weight for production, and if they're helpful for development then odds are good that having proper reporting in production would also be valuable.
Sounds good to me. Is a second patch OK with you? I think we get a lot of cryptic cpu run exit reasons so fixing this up when we remove pr_infos would be good. This would be a good example without this pr_info or this change you'd have no idea whats going on.
Quoted 14 lines
Quoted 14 lines
Quoted 13 lines
Quoted 9 lines
Quoting the spec:
The reason code set is meant to provide hypervisors with their own termination SEV-ES Guest-Hypervisor Communication Block Standardization reason codes. This document defines and owns reason code set0x0and the following reason codes (GHCBData[23:16]):0x00– General termination request0x01– SEV-ES / GHCB Protocol range is not supported.0x02– SEV-SNP features not supportedReading this again, I now see that "
reason_set" sounds like "The reason code is set". I bet that's how Sean read it during his review. So yeah, this needs comments :-)!
Quoted 8 lines
Quoted 3 lines
I'll add comments but I agree with Marc. These are part of the GHCB spec so for the very specific SEV-ES termination reason we should include all the data the spec allows. Sounds OK?
Ugh, so "set" means "set of reason codes"? That's heinous naming. I don't have a strong objection to splitting, but at the same time, why not punt it to userspace? Userspace is obviously going to have to understand what the hell "set" means anyways...
I am fine just copying the entire MSR to userspace.
Sean Christopherson
maintainerQuoted 15 lines · Peter Gonda
Quoted 6 lines · Sean Christopherson
Quoted 3 lines · Peter Gonda
I'll make this 2 patches. This current patch and another to rate limit this
pr_info()I think this patch is doing a lot already so would prefer to just add a second. Is that reasonable?Quoted 7 lines
I strongly prefer removing the
pr_info()entirely. As Marc pointed out, the info is redundant when KVM properly reports the issue. And worse, the info is useless unless there's exactly one VM running. Even then, it doesn't capture which vCPU failed. This is exactly why Jim, myself, and others, have been pushing to avoid using dmesg to report guest errors. They're helpful for initial development, but dead weight for production, and if they're helpful for development then odds are good that having proper reporting in production would also be valuable.
Quoted 4 lines
Sounds good to me. Is a second patch OK with you? I think we get a lot of cryptic cpu run exit reasons so fixing this up when we remove
pr_infoswould be good. This would be a good example without thispr_infoor this change you'd have no idea whats going on.
As in, a second patch to remove the pr_info? Yeah, no objection.
Quoted 16 lines
Quoted 16 lines
Quoted 15 lines
Quoted 11 lines
Quoted 7 lines
The reason code set is meant to provide hypervisors with their own termination SEV-ES Guest-Hypervisor Communication Block Standardization reason codes. This document defines and owns reason code set
0x0and the following reason codes (GHCBData[23:16]):0x00– General termination request0x01– SEV-ES / GHCB Protocol range is not supported.0x02– SEV-SNP features not supportedReading this again, I now see that "
reason_set" sounds like "The reason code is set". I bet that's how Sean read it during his review. So yeah, this needs comments :-)!I'll add comments but I agree with Marc. These are part of the GHCB spec so for the very specific SEV-ES termination reason we should include all the data the spec allows. Sounds OK?
Quoted 6 lines
Quoted 4 lines
Ugh, so "set" means "set of reason codes"? That's heinous naming. I don't have a strong objection to splitting, but at the same time, why not punt it to userspace? Userspace is obviously going to have to understand what the hell "set" means anyways...
I am fine just copying the entire MSR to userspace.
I'm fine with it too. Thanks for the feedback, Sean!