Skip to contentSkip to conversation

KVM: SEV: Update SEV-ES shutdown intercepts with more metadata

PATCH Superseded Peter Gonda 3 participants 7 messages Sep 6, 2023

Subsystems (3)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 KVM: SEV: Update SEV-ES shutdown intercepts with more metadata

Currently if an SEV-ES VM shuts down userspace sees KVM_RUN struct with only the INVALID_ARGUMENT. This is a very limited amount of information to debug the situation. Instead KVM can return a KVM_EXIT_SHUTDOWN to alert userspace the VM is shutting down and is not usable any further.

Cc Paolo Bonzini <[email protected]>
Cc Sean Christopherson <[email protected]>
Cc Tom Lendacky <[email protected]>
Cc Joerg Roedel <[email protected]>
Cc Borislav Petkov <[email protected]>
---
 arch/x86/kvm/svm/svm.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
arch/x86/kvm/svm/svm.c +5 -3
index 956726d867aa..cecf6a528c9b 100644
@@ -2131,12 +2131,14 @@ static int shutdown_interception(struct kvm_vcpu *vcpu)
 	 * The VM save area has already been encrypted so it
 	 * cannot be reinitialized - just terminate.
 	 */
-	if (sev_es_guest(vcpu->kvm))
-		return -EINVAL;
+	if (sev_es_guest(vcpu->kvm)) {
+		kvm_run->exit_reason = KVM_EXIT_SHUTDOWN;
+		return 0;
+	}
 
 	/*
 	 * VMCB is undefined after a SHUTDOWN intercept.  INIT the vCPU to put
-	 * the VMCB in a known good state.  Unfortuately, KVM doesn't have
+	 * the VMCB in a known good state.  Unfortunately, KVM doesn't have
 	 * KVM_MP_STATE_SHUTDOWN and can't add it without potentially breaking
 	 * userspace.  At a platform view, INIT is acceptable behavior as
 	 * there exist bare metal platforms that automatically INIT the CPU
Signature · 1 line
2.42.0.283.g2d96d420d3-goog
Acked-by on v2 1/1
#1 lore raw
Quoted 1 line · Peter Gonda

Currently if an SEV-ES VM shuts down userspace sees KVM_RUN struct with

s/down userspace/down, userspace/

Quoted 24 lines

only the INVALID_ARGUMENT. This is a very limited amount of information to debug the situation. Instead KVM can return a KVM_EXIT_SHUTDOWN to alert userspace the VM is shutting down and is not usable any further.

Cc Paolo Bonzini <[email protected]>
Cc Sean Christopherson <[email protected]>
Cc Tom Lendacky <[email protected]>
Cc Joerg Roedel <[email protected]>
Cc Borislav Petkov <[email protected]>
---
  arch/x86/kvm/svm/svm.c | 8 +++++---
  1 file changed, 5 insertions(+), 3 deletions(-)
arch/x86/kvm/svm/svm.c +0 -0
index 956726d867aa..cecf6a528c9b 100644

arch/x86/kvm/svm/svm.c static int shutdown_interception(struct kvm_vcpu *vcpu) · line 2137

Quoted 9 lines
arch/x86/kvm/svm/svm.c +4 -2
@@ -2131,12 +2131,14 @@ static int shutdown_interception(struct kvm_vcpu *vcpu)
  	 * The VM save area has already been encrypted so it
  	 * cannot be reinitialized - just terminate.
  	 */
-	if (sev_es_guest(vcpu->kvm))
-		return -EINVAL;
+	if (sev_es_guest(vcpu->kvm)) {
+		kvm_run->exit_reason = KVM_EXIT_SHUTDOWN;
+		return 0;
+	}

Just a nit... feel free to ignore, but, since KVM_EXIT_SHUTDOWN is also set at the end of the function and I don't think kvm_vcpu_reset() clears the value from kvm_run, you could just set kvm_run->exit_reason on entry and just return 0 early for an SEV-ES guest.

Overall, though:

Thanks,
Tom

Quoted 8 lines
arch/x86/kvm/svm/svm.c +1 -1
  	/*
  	 * VMCB is undefined after a SHUTDOWN intercept.  INIT the vCPU to put
-	 * the VMCB in a known good state.  Unfortuately, KVM doesn't have
+	 * the VMCB in a known good state.  Unfortunately, KVM doesn't have
  	 * KVM_MP_STATE_SHUTDOWN and can't add it without potentially breaking
  	 * userspace.  At a platform view, INIT is acceptable behavior as
  	 * there exist bare metal platforms that automatically INIT the CPU
#2 lore raw
Quoted 4 lines · Tom Lendacky
Quoted 1 line · Peter Gonda

Currently if an SEV-ES VM shuts down userspace sees KVM_RUN struct with

s/down userspace/down, userspace/

Heh, yeah, I read that the same way you did.

Quoted 30 lines
Quoted 30 lines

only the INVALID_ARGUMENT. This is a very limited amount of information to debug the situation. Instead KVM can return a KVM_EXIT_SHUTDOWN to alert userspace the VM is shutting down and is not usable any further.

Cc Paolo Bonzini <[email protected]>
Cc Sean Christopherson <[email protected]>
Cc Tom Lendacky <[email protected]>
Cc Joerg Roedel <[email protected]>
Cc Borislav Petkov <[email protected]>
---
  arch/x86/kvm/svm/svm.c | 8 +++++---
  1 file changed, 5 insertions(+), 3 deletions(-)
arch/x86/kvm/svm/svm.c +1 -2
index 956726d867aa..cecf6a528c9b 100644
@@ -2131,12 +2131,14 @@ static int shutdown_interception(struct kvm_vcpu *vcpu)
  	 * The VM save area has already been encrypted so it
  	 * cannot be reinitialized - just terminate.
  	 */
-	if (sev_es_guest(vcpu->kvm))
-		return -EINVAL;
+	if (sev_es_guest(vcpu->kvm)) {
Quoted 8 lines

arch/x86/kvm/svm/svm.c static int shutdown_interception(struct kvm_vcpu *vcpu) · line 2137

Quoted 3 lines
arch/x86/kvm/svm/svm.c +3 -0
+		kvm_run->exit_reason = KVM_EXIT_SHUTDOWN;
+		return 0;
+	}

Just a nit... feel free to ignore, but, since KVM_EXIT_SHUTDOWN is also set at the end of the function and I don't think kvm_vcpu_reset() clears the value from kvm_run, you could just set kvm_run->exit_reason on entry and just return 0 early for an SEV-ES guest.

kvm_run is writable by userspace though, so KVM can't rely on kvm_run->exit_reason for correctness.

And IIUC, the VMSA is also toast, i.e. doing anything other than marking the VM dead is futile, no?

#3 lore raw
Quoted 47 lines · Sean Christopherson
Quoted 4 lines · Tom Lendacky
Quoted 1 line · Peter Gonda

Currently if an SEV-ES VM shuts down userspace sees KVM_RUN struct with

s/down userspace/down, userspace/

Heh, yeah, I read that the same way you did.

Quoted 30 lines
Quoted 30 lines

only the INVALID_ARGUMENT. This is a very limited amount of information to debug the situation. Instead KVM can return a KVM_EXIT_SHUTDOWN to alert userspace the VM is shutting down and is not usable any further.

Cc Paolo Bonzini <[email protected]>
Cc Sean Christopherson <[email protected]>
Cc Tom Lendacky <[email protected]>
Cc Joerg Roedel <[email protected]>
Cc Borislav Petkov <[email protected]>
---
   arch/x86/kvm/svm/svm.c | 8 +++++---
   1 file changed, 5 insertions(+), 3 deletions(-)
arch/x86/kvm/svm/svm.c +1 -2
index 956726d867aa..cecf6a528c9b 100644
@@ -2131,12 +2131,14 @@ static int shutdown_interception(struct kvm_vcpu *vcpu)
   	 * The VM save area has already been encrypted so it
   	 * cannot be reinitialized - just terminate.
   	 */
-	if (sev_es_guest(vcpu->kvm))
-		return -EINVAL;
+	if (sev_es_guest(vcpu->kvm)) {
Quoted 8 lines

arch/x86/kvm/svm/svm.c static int shutdown_interception(struct kvm_vcpu *vcpu) · line 2137

Quoted 3 lines
arch/x86/kvm/svm/svm.c +3 -0
+		kvm_run->exit_reason = KVM_EXIT_SHUTDOWN;
+		return 0;
+	}

Just a nit... feel free to ignore, but, since KVM_EXIT_SHUTDOWN is also set at the end of the function and I don't think kvm_vcpu_reset() clears the value from kvm_run, you could just set kvm_run->exit_reason on entry and just return 0 early for an SEV-ES guest.

Quoted 5 lines

kvm_run is writable by userspace though, so KVM can't rely on kvm_run->exit_reason for correctness.

And IIUC, the VMSA is also toast, i.e. doing anything other than marking the VM dead is futile, no?

I was just saying that "kvm_run->exit_reason = KVM_EXIT_SHUTDOWN;" is in the shutdown_interception() function twice now (at both exit points of the function) and can probably just be moved to the top of the function and be common for both exit points, now, right?

I'm not saying to get rid of it, just set it sooner.

Thanks,
Tom

#4 lore raw
Quoted 29 lines · Tom Lendacky
Quoted 22 lines · Sean Christopherson
Quoted 12 lines · Tom Lendacky
Quoted 11 lines · Peter Gonda
arch/x86/kvm/svm/svm.c +1 -2
index 956726d867aa..cecf6a528c9b 100644
@@ -2131,12 +2131,14 @@ static int shutdown_interception(struct kvm_vcpu *vcpu)
   	 * The VM save area has already been encrypted so it
   	 * cannot be reinitialized - just terminate.
   	 */
-	if (sev_es_guest(vcpu->kvm))
-		return -EINVAL;
+	if (sev_es_guest(vcpu->kvm)) {
Quoted 8 lines

arch/x86/kvm/svm/svm.c static int shutdown_interception(struct kvm_vcpu *vcpu) · line 2137

Quoted 3 lines
arch/x86/kvm/svm/svm.c +3 -0
+		kvm_run->exit_reason = KVM_EXIT_SHUTDOWN;
+		return 0;
+	}

Just a nit... feel free to ignore, but, since KVM_EXIT_SHUTDOWN is also set at the end of the function and I don't think kvm_vcpu_reset() clears the value from kvm_run, you could just set kvm_run->exit_reason on entry and just return 0 early for an SEV-ES guest.

Quoted 5 lines

kvm_run is writable by userspace though, so KVM can't rely on kvm_run->exit_reason for correctness.

And IIUC, the VMSA is also toast, i.e. doing anything other than marking the VM dead is futile, no?

Quoted 6 lines

I was just saying that "kvm_run->exit_reason = KVM_EXIT_SHUTDOWN;" is in the shutdown_interception() function twice now (at both exit points of the function) and can probably just be moved to the top of the function and be common for both exit points, now, right?

I'm not saying to get rid of it, just set it sooner.

Ah, I thought you were saying bail early from kvm_vcpu_reset(). I agree that not having completely split logic would be ideal. What about this?

	/*
	 * VMCB is undefined after a SHUTDOWN intercept.  INIT the vCPU to put
	 * the VMCB in a known good state.  Unfortuately, KVM doesn't have
	 * KVM_MP_STATE_SHUTDOWN and can't add it without potentially breaking
	 * userspace.  At a platform view, INIT is acceptable behavior as
	 * there exist bare metal platforms that automatically INIT the CPU
	 * in response to shutdown.
	 *
	 * The VM save area for SEV-ES guests has already been encrypted so it
	 * cannot be reinitialized, i.e. synthesizing INIT is futile.
	 */
	if (!sev_es_guest(vcpu->kvm)) {
		clear_page(svm->vmcb);
		kvm_vcpu_reset(vcpu, true);
	}

	kvm_run->exit_reason = KVM_EXIT_SHUTDOWN;
	return 0;
#5 lore raw
Quoted 51 lines · Sean Christopherson
Quoted 29 lines · Tom Lendacky
Quoted 22 lines · Sean Christopherson
Quoted 12 lines · Tom Lendacky
Quoted 11 lines · Peter Gonda
arch/x86/kvm/svm/svm.c +1 -2
index 956726d867aa..cecf6a528c9b 100644
@@ -2131,12 +2131,14 @@ static int shutdown_interception(struct kvm_vcpu *vcpu)
         * The VM save area has already been encrypted so it
         * cannot be reinitialized - just terminate.
         */
-       if (sev_es_guest(vcpu->kvm))
-               return -EINVAL;
+       if (sev_es_guest(vcpu->kvm)) {
Quoted 8 lines

arch/x86/kvm/svm/svm.c static int shutdown_interception(struct kvm_vcpu *vcpu) · line 2137

Quoted 3 lines
arch/x86/kvm/svm/svm.c +3 -0
+               kvm_run->exit_reason = KVM_EXIT_SHUTDOWN;
+               return 0;
+       }

Just a nit... feel free to ignore, but, since KVM_EXIT_SHUTDOWN is also set at the end of the function and I don't think kvm_vcpu_reset() clears the value from kvm_run, you could just set kvm_run->exit_reason on entry and just return 0 early for an SEV-ES guest.

Quoted 5 lines

kvm_run is writable by userspace though, so KVM can't rely on kvm_run->exit_reason for correctness.

And IIUC, the VMSA is also toast, i.e. doing anything other than marking the VM dead is futile, no?

Quoted 6 lines

I was just saying that "kvm_run->exit_reason = KVM_EXIT_SHUTDOWN;" is in the shutdown_interception() function twice now (at both exit points of the function) and can probably just be moved to the top of the function and be common for both exit points, now, right?

I'm not saying to get rid of it, just set it sooner.

Ah, I thought you were saying bail early from kvm_vcpu_reset(). I agree that not having completely split logic would be ideal. What about this?

        /*
         * VMCB is undefined after a SHUTDOWN intercept.  INIT the vCPU to put
         * the VMCB in a known good state.  Unfortuately, KVM doesn't have
         * KVM_MP_STATE_SHUTDOWN and can't add it without potentially breaking
         * userspace.  At a platform view, INIT is acceptable behavior as
         * there exist bare metal platforms that automatically INIT the CPU
         * in response to shutdown.
         *
         * The VM save area for SEV-ES guests has already been encrypted so it
         * cannot be reinitialized, i.e. synthesizing INIT is futile.
Quoted 8 lines
         */
        if (!sev_es_guest(vcpu->kvm)) {
                clear_page(svm->vmcb);
                kvm_vcpu_reset(vcpu, true);
        }

        kvm_run->exit_reason = KVM_EXIT_SHUTDOWN;
        return 0;

Looks better to me. Thanks!

#6 lore raw
In reply to Sean Christopherson
Quoted 50 lines · Sean Christopherson
Quoted 29 lines · Tom Lendacky
Quoted 22 lines · Sean Christopherson
Quoted 12 lines · Tom Lendacky
Quoted 11 lines · Peter Gonda
arch/x86/kvm/svm/svm.c +1 -2
index 956726d867aa..cecf6a528c9b 100644
@@ -2131,12 +2131,14 @@ static int shutdown_interception(struct kvm_vcpu *vcpu)
    	 * The VM save area has already been encrypted so it
    	 * cannot be reinitialized - just terminate.
    	 */
-	if (sev_es_guest(vcpu->kvm))
-		return -EINVAL;
+	if (sev_es_guest(vcpu->kvm)) {
Quoted 8 lines

arch/x86/kvm/svm/svm.c static int shutdown_interception(struct kvm_vcpu *vcpu) · line 2137

Quoted 3 lines
arch/x86/kvm/svm/svm.c +3 -0
+		kvm_run->exit_reason = KVM_EXIT_SHUTDOWN;
+		return 0;
+	}

Just a nit... feel free to ignore, but, since KVM_EXIT_SHUTDOWN is also set at the end of the function and I don't think kvm_vcpu_reset() clears the value from kvm_run, you could just set kvm_run->exit_reason on entry and just return 0 early for an SEV-ES guest.

Quoted 5 lines

kvm_run is writable by userspace though, so KVM can't rely on kvm_run->exit_reason for correctness.

And IIUC, the VMSA is also toast, i.e. doing anything other than marking the VM dead is futile, no?

Quoted 6 lines

I was just saying that "kvm_run->exit_reason = KVM_EXIT_SHUTDOWN;" is in the shutdown_interception() function twice now (at both exit points of the function) and can probably just be moved to the top of the function and be common for both exit points, now, right?

I'm not saying to get rid of it, just set it sooner.

Ah, I thought you were saying bail early from kvm_vcpu_reset(). I agree that not having completely split logic would be ideal. What about this?

	/*
	 * VMCB is undefined after a SHUTDOWN intercept.  INIT the vCPU to put
	 * the VMCB in a known good state.  Unfortuately, KVM doesn't have
	 * KVM_MP_STATE_SHUTDOWN and can't add it without potentially breaking
	 * userspace.  At a platform view, INIT is acceptable behavior as
	 * there exist bare metal platforms that automatically INIT the CPU
	 * in response to shutdown.
	 *
	 * The VM save area for SEV-ES guests has already been encrypted so it
	 * cannot be reinitialized, i.e. synthesizing INIT is futile.
Quoted 8 lines
	 */
	if (!sev_es_guest(vcpu->kvm)) {
		clear_page(svm->vmcb);
		kvm_vcpu_reset(vcpu, true);
	}

	kvm_run->exit_reason = KVM_EXIT_SHUTDOWN;
	return 0;

That looks good to me!

Thanks,
Tom