Skip to contentSkip to conversation

Add KVM_EXIT_SHUTDOWN metadata for SEV-ES

PATCH Superseded Peter Gonda 3 participants 9 messages Mar 21, 2022 – Mar 25, 2022

Subsystems (3)

v1 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 Add KVM_EXIT_SHUTDOWN metadata for SEV-ES

SEV-ES guests can request termination using the GHCB's MSR protocol. See AMD's GHCB spec section '4.1.13 Termination Request'. Currently when a guest does this the userspace VMM sees an KVM_EXIT_UNKNOWN (-EVINAL) return code from KVM_RUN. By adding a KVM_EXIT_SHUTDOWN_ENTRY to kvm_run struct the userspace VMM can clearly see the guest has requested a SEV-ES termination including the termination reason code set and reason code.

Cc Borislav Petkov <[email protected]>
Cc Tom Lendacky <[email protected]>
Cc Brijesh Singh <[email protected]>
Cc Joerg Roedel <[email protected]>
Cc Marc Orr <[email protected]>
Cc Sean Christopherson <[email protected]>

---

Tested by making an SEV-ES guest call sev_es_terminate() with hardcoded reason code set and reason code and then observing the codes from the userspace VMM in the kvm_run.shutdown.data fields.

---
 arch/x86/kvm/svm/sev.c   |  9 +++++++--
 include/uapi/linux/kvm.h | 12 ++++++++++++
 2 files changed, 19 insertions(+), 2 deletions(-)
arch/x86/kvm/svm/sev.c +7 -2
index 75fa6dd268f0..5f9d37dd3f6f 100644
@@ -2735,8 +2735,13 @@ static int sev_handle_vmgexit_msr_protocol(struct vcpu_svm *svm)
 		pr_info("SEV-ES guest requested termination: %#llx:%#llx\n",
 			reason_set, reason_code);
 
-		ret = -EINVAL;
-		break;
+		vcpu->run->exit_reason = KVM_EXIT_SHUTDOWN;
+		vcpu->run->shutdown.reason = KVM_SHUTDOWN_SEV_TERM;
+		vcpu->run->shutdown.ndata = 2;
+		vcpu->run->shutdown.data[0] = reason_set;
+		vcpu->run->shutdown.data[1] = reason_code;
+
+		return 0;
 	}
 	default:
 		/* Error, keep GHCB MSR value as-is */
include/uapi/linux/kvm.h +12 -0
index 8616af85dc5d..12138b8f290c 100644
@@ -271,6 +271,12 @@ struct kvm_xen_exit {
 #define KVM_EXIT_XEN              34
 #define KVM_EXIT_RISCV_SBI        35
 
+/* For KVM_EXIT_SHUTDOWN */
+/* Standard VM shutdown request. No additional metadata provided. */
+#define KVM_SHUTDOWN_REQ	0
+/* SEV-ES termination request */
+#define KVM_SHUTDOWN_SEV_TERM	1
+
 /* For KVM_EXIT_INTERNAL_ERROR */
 /* Emulate instruction failed. */
 #define KVM_INTERNAL_ERROR_EMULATION	1
@@ -311,6 +317,12 @@ struct kvm_run {
 		struct {
 			__u64 hardware_exit_reason;
 		} hw;
+		/* KVM_EXIT_SHUTDOWN_ENTRY */
+		struct {
+			__u64 reason;
+			__u32 ndata;
+			__u64 data[16];
+		} shutdown;
 		/* KVM_EXIT_FAIL_ENTRY */
 		struct {
 			__u64 hardware_entry_failure_reason;
Signature · 1 line
2.35.1.894.gb6a874cedc-goog
maintainer
#1 lore raw
Quoted 7 lines · Peter Gonda

SEV-ES guests can request termination using the GHCB's MSR protocol. See AMD's GHCB spec section '4.1.13 Termination Request'. Currently when a guest does this the userspace VMM sees an KVM_EXIT_UNKNOWN (-EVINAL) return code from KVM_RUN. By adding a KVM_EXIT_SHUTDOWN_ENTRY to kvm_run struct the userspace VMM can clearly see the guest has requested a SEV-ES termination including the termination reason code set and reason code.

Quoted 9 lines
Cc Borislav Petkov <[email protected]>
Cc Tom Lendacky <[email protected]>
Cc Brijesh Singh <[email protected]>
Cc Joerg Roedel <[email protected]>
Cc Marc Orr <[email protected]>
Cc Sean Christopherson <[email protected]>

Looks good, but it has to also add a capability.

include/uapi/linux/kvm.h struct kvm_run { · line 320

Quoted 1 line
include/uapi/linux/kvm.h +1 -0
+		/* KVM_EXIT_SHUTDOWN_ENTRY */

Just KVM_EXIT_SHUTDOWN.

Paolo

Quoted 8 lines
include/uapi/linux/kvm.h +5 -0
+		struct {
+			__u64 reason;
+			__u32 ndata;
+			__u64 data[16];
+		} shutdown;
  		/* KVM_EXIT_FAIL_ENTRY */
  		struct {
  			__u64 hardware_entry_failure_reason;
#2 lore raw
Quoted 19 lines · Paolo Bonzini
Quoted 7 lines · Peter Gonda

SEV-ES guests can request termination using the GHCB's MSR protocol. See AMD's GHCB spec section '4.1.13 Termination Request'. Currently when a guest does this the userspace VMM sees an KVM_EXIT_UNKNOWN (-EVINAL) return code from KVM_RUN. By adding a KVM_EXIT_SHUTDOWN_ENTRY to kvm_run struct the userspace VMM can clearly see the guest has requested a SEV-ES termination including the termination reason code set and reason code.

Quoted 9 lines
Cc Borislav Petkov <[email protected]>
Cc Tom Lendacky <[email protected]>
Cc Brijesh Singh <[email protected]>
Cc Joerg Roedel <[email protected]>
Cc Marc Orr <[email protected]>
Cc Sean Christopherson <[email protected]>
Quoted 1 line

Looks good, but it has to also add a capability.

Thanks for the quick review! Just so I understand. I should add KVM_CAP_SEV_TERM or something, then if that has been enabled do the new functionality, else keep the old functionality?

Quoted 5 lines
Quoted 2 lines

+ /* KVM_EXIT_SHUTDOWN_ENTRY */

Just KVM_EXIT_SHUTDOWN.

Will do.

maintainer
#3 lore raw
Quoted 20 lines · Peter Gonda
Quoted 19 lines · Paolo Bonzini
Quoted 16 lines · Peter Gonda

SEV-ES guests can request termination using the GHCB's MSR protocol. See AMD's GHCB spec section '4.1.13 Termination Request'. Currently when a guest does this the userspace VMM sees an KVM_EXIT_UNKNOWN (-EVINAL) return code from KVM_RUN. By adding a KVM_EXIT_SHUTDOWN_ENTRY to kvm_run struct the userspace VMM can clearly see the guest has requested a SEV-ES termination including the termination reason code set and reason code.

Cc Borislav Petkov <[email protected]>
Cc Tom Lendacky <[email protected]>
Cc Brijesh Singh <[email protected]>
Cc Joerg Roedel <[email protected]>
Cc Marc Orr <[email protected]>
Cc Sean Christopherson <[email protected]>
Quoted 5 lines
Quoted 1 line

Looks good, but it has to also add a capability.

Thanks for the quick review! Just so I understand. I should add KVM_CAP_SEV_TERM or something, then if that has been enabled do the new functionality, else keep the old functionality?

No, much simpler; just something for which KVM_CHECK_EXTENSION returns 1, so that userspace knows that there is a "shutdown" member to be filled by KVM_EXIT_SHUTDOWN. e.g. KVM_CAP_EXIT_SHUTDOWN_REASON.

Paolo

#4 lore raw
Quoted 24 lines · Paolo Bonzini
Quoted 22 lines · Peter Gonda
Quoted 20 lines · Paolo Bonzini
Quoted 7 lines · Peter Gonda

SEV-ES guests can request termination using the GHCB's MSR protocol. See AMD's GHCB spec section '4.1.13 Termination Request'. Currently when a guest does this the userspace VMM sees an KVM_EXIT_UNKNOWN (-EVINAL) return code from KVM_RUN. By adding a KVM_EXIT_SHUTDOWN_ENTRY to kvm_run struct the userspace VMM can clearly see the guest has requested a SEV-ES termination including the termination reason code set and reason code.

Quoted 9 lines
Cc Borislav Petkov <[email protected]>
Cc Tom Lendacky <[email protected]>
Cc Brijesh Singh <[email protected]>
Cc Joerg Roedel <[email protected]>
Cc Marc Orr <[email protected]>
Cc Sean Christopherson <[email protected]>

Looks good, but it has to also add a capability.

Quoted 7 lines
Quoted 3 lines

Thanks for the quick review! Just so I understand. I should add KVM_CAP_SEV_TERM or something, then if that has been enabled do the new functionality, else keep the old functionality?

No, much simpler; just something for which KVM_CHECK_EXTENSION returns 1, so that userspace knows that there is a "shutdown" member to be filled by KVM_EXIT_SHUTDOWN. e.g. KVM_CAP_EXIT_SHUTDOWN_REASON.

Makes sense, thanks for help. Will do for V2.

Quoted 3 lines

Paolo

#5 lore raw
In reply to Peter Gonda
Quoted 36 lines · Peter Gonda

SEV-ES guests can request termination using the GHCB's MSR protocol. See AMD's GHCB spec section '4.1.13 Termination Request'. Currently when a guest does this the userspace VMM sees an KVM_EXIT_UNKNOWN (-EVINAL) return code from KVM_RUN. By adding a KVM_EXIT_SHUTDOWN_ENTRY to kvm_run struct the userspace VMM can clearly see the guest has requested a SEV-ES termination including the termination reason code set and reason code.

Cc Borislav Petkov <[email protected]>
Cc Tom Lendacky <[email protected]>
Cc Brijesh Singh <[email protected]>
Cc Joerg Roedel <[email protected]>
Cc Marc Orr <[email protected]>
Cc Sean Christopherson <[email protected]>

---

Tested by making an SEV-ES guest call sev_es_terminate() with hardcoded reason code set and reason code and then observing the codes from the userspace VMM in the kvm_run.shutdown.data fields.

---
 arch/x86/kvm/svm/sev.c   |  9 +++++++--
 include/uapi/linux/kvm.h | 12 ++++++++++++
 2 files changed, 19 insertions(+), 2 deletions(-)
arch/x86/kvm/svm/sev.c +0 -0
index 75fa6dd268f0..5f9d37dd3f6f 100644
@@ -2735,8 +2735,13 @@ static int sev_handle_vmgexit_msr_protocol(struct vcpu_svm *svm)
                pr_info("SEV-ES guest requested termination: %#llx:%#llx\n",
                        reason_set, reason_code);

arch/x86/kvm/svm/sev.c static int sev_handle_vmgexit_msr_protocol(struct vcpu_svm *svm) · line 2744

Quoted 10 lines
arch/x86/kvm/svm/sev.c +7 -2
 
-               ret = -EINVAL;
-               break;
+               vcpu->run->exit_reason = KVM_EXIT_SHUTDOWN;
+               vcpu->run->shutdown.reason = KVM_SHUTDOWN_SEV_TERM;
+               vcpu->run->shutdown.ndata = 2;
+               vcpu->run->shutdown.data[0] = reason_set;
+               vcpu->run->shutdown.data[1] = reason_code;
+
+               return 0;

Maybe I'm missing something, but don't we want to keep returning an error?

rationale: Current behavior: return -EINVAL to userpsace, but userpsace cannot infer where the -EINVAL came from. After this patch:
We should still return -EINVAL to userspace, but now userspace can parse this new info in the KVM run struct to properly terminate.

Quoted 36 lines
arch/x86/kvm/svm/sev.c +0 -0
        }
        default:
                /* Error, keep GHCB MSR value as-is */
include/uapi/linux/kvm.h +12 -1
index 8616af85dc5d..12138b8f290c 100644
@@ -271,6 +271,12 @@ struct kvm_xen_exit {
 #define KVM_EXIT_XEN              34
 #define KVM_EXIT_RISCV_SBI        35
 
+/* For KVM_EXIT_SHUTDOWN */
+/* Standard VM shutdown request. No additional metadata provided. */
+#define KVM_SHUTDOWN_REQ       0
+/* SEV-ES termination request */
+#define KVM_SHUTDOWN_SEV_TERM  1
+
 /* For KVM_EXIT_INTERNAL_ERROR */
 /* Emulate instruction failed. */
 #define KVM_INTERNAL_ERROR_EMULATION   1
@@ -311,6 +317,12 @@ struct kvm_run {
                struct {
                        __u64 hardware_exit_reason;
                } hw;
+               /* KVM_EXIT_SHUTDOWN_ENTRY */
+               struct {
+                       __u64 reason;
+                       __u32 ndata;
+                       __u64 data[16];
+               } shutdown;
                /* KVM_EXIT_FAIL_ENTRY */
                struct {
                        __u64 hardware_entry_failure_reason;
--

2.35.1.894.gb6a874cedc-goog

#6 lore raw
Quoted 20 lines
Quoted 7 lines
arch/x86/kvm/svm/sev.c +0 -0
index 75fa6dd268f0..5f9d37dd3f6f 100644
@@ -2735,8 +2735,13 @@ static int sev_handle_vmgexit_msr_protocol(struct vcpu_svm *svm)
                pr_info("SEV-ES guest requested termination: %#llx:%#llx\n",
                        reason_set, reason_code);

arch/x86/kvm/svm/sev.c static int sev_handle_vmgexit_msr_protocol(struct vcpu_svm *svm) · line 2744

Quoted 10 lines
arch/x86/kvm/svm/sev.c +7 -2
 
-               ret = -EINVAL;
-               break;
+               vcpu->run->exit_reason = KVM_EXIT_SHUTDOWN;
+               vcpu->run->shutdown.reason = KVM_SHUTDOWN_SEV_TERM;
+               vcpu->run->shutdown.ndata = 2;
+               vcpu->run->shutdown.data[0] = reason_set;
+               vcpu->run->shutdown.data[1] = reason_code;
+
+               return 0;

Maybe I'm missing something, but don't we want to keep returning an error?

Quoted 5 lines

rationale: Current behavior: return -EINVAL to userpsace, but userpsace cannot infer where the -EINVAL came from. After this patch:
We should still return -EINVAL to userspace, but now userspace can parse this new info in the KVM run struct to properly terminate.

I removed the error return code here since an SEV guest may request a termination due to no fault of the host at all. This is now inline with any other shutdown requested by the guest. I don't have a strong preference here but EINVAL doesn't seem correct in all cases, do others have any thoughts on this?

#7 lore raw
Quoted 27 lines · Peter Gonda
Quoted 21 lines
Quoted 8 lines
arch/x86/kvm/svm/sev.c +0 -0
index 75fa6dd268f0..5f9d37dd3f6f 100644
@@ -2735,8 +2735,13 @@ static int sev_handle_vmgexit_msr_protocol(struct vcpu_svm *svm)
                pr_info("SEV-ES guest requested termination: %#llx:%#llx\n",
                        reason_set, reason_code);

arch/x86/kvm/svm/sev.c static int sev_handle_vmgexit_msr_protocol(struct vcpu_svm *svm) · line 2744

Quoted 10 lines
arch/x86/kvm/svm/sev.c +7 -2
 
-               ret = -EINVAL;
-               break;
+               vcpu->run->exit_reason = KVM_EXIT_SHUTDOWN;
+               vcpu->run->shutdown.reason = KVM_SHUTDOWN_SEV_TERM;
+               vcpu->run->shutdown.ndata = 2;
+               vcpu->run->shutdown.data[0] = reason_set;
+               vcpu->run->shutdown.data[1] = reason_code;
+
+               return 0;

Maybe I'm missing something, but don't we want to keep returning an error?

Quoted 5 lines

rationale: Current behavior: return -EINVAL to userpsace, but userpsace cannot infer where the -EINVAL came from. After this patch:
We should still return -EINVAL to userspace, but now userspace can parse this new info in the KVM run struct to properly terminate.

Quoted 5 lines

I removed the error return code here since an SEV guest may request a termination due to no fault of the host at all. This is now inline with any other shutdown requested by the guest. I don't have a strong preference here but EINVAL doesn't seem correct in all cases, do others have any thoughts on this?

Makes sense. Yeah, let's see if others have an opinion. Otherwise, I'm fine either way. Now that you mention it, returning an error to userspace when the guest triggered the self-termination, and could've done so for reasons outside the host's control, is a little odd. But at the same time, it's just as likely that the guest is self-terminating due to a host-side error. So I guess it's not clear whether returning an error here is "right" or "wrong".

#8 lore raw
Quoted 35 lines · Marc Orr
Quoted 27 lines · Peter Gonda
Quoted 21 lines
Quoted 8 lines
arch/x86/kvm/svm/sev.c +0 -0
index 75fa6dd268f0..5f9d37dd3f6f 100644
@@ -2735,8 +2735,13 @@ static int sev_handle_vmgexit_msr_protocol(struct vcpu_svm *svm)
                pr_info("SEV-ES guest requested termination: %#llx:%#llx\n",
                        reason_set, reason_code);

arch/x86/kvm/svm/sev.c static int sev_handle_vmgexit_msr_protocol(struct vcpu_svm *svm) · line 2744

Quoted 10 lines
arch/x86/kvm/svm/sev.c +7 -2
 
-               ret = -EINVAL;
-               break;
+               vcpu->run->exit_reason = KVM_EXIT_SHUTDOWN;
+               vcpu->run->shutdown.reason = KVM_SHUTDOWN_SEV_TERM;
+               vcpu->run->shutdown.ndata = 2;
+               vcpu->run->shutdown.data[0] = reason_set;
+               vcpu->run->shutdown.data[1] = reason_code;
+
+               return 0;

Maybe I'm missing something, but don't we want to keep returning an error?

Quoted 5 lines

rationale: Current behavior: return -EINVAL to userpsace, but userpsace cannot infer where the -EINVAL came from. After this patch:
We should still return -EINVAL to userspace, but now userspace can parse this new info in the KVM run struct to properly terminate.

Quoted 5 lines

I removed the error return code here since an SEV guest may request a termination due to no fault of the host at all. This is now inline with any other shutdown requested by the guest. I don't have a strong preference here but EINVAL doesn't seem correct in all cases, do others have any thoughts on this?

Quoted 7 lines

Makes sense. Yeah, let's see if others have an opinion. Otherwise, I'm fine either way. Now that you mention it, returning an error to userspace when the guest triggered the self-termination, and could've done so for reasons outside the host's control, is a little odd. But at the same time, it's just as likely that the guest is self-terminating due to a host-side error. So I guess it's not clear whether returning an error here is "right" or "wrong".

Since no one has expressed a strong opinion have have left this part of the patch unchanged in the V2. Happy to revise again if people prefer something else.