Skip to contentSkip to conversation

KVM: SEV: Add support for the ALLOWED_SEV_FEATURES feature

PATCH Superseded Kim Phillips 4 participants 10 messages Aug 2, 2024 – Aug 20, 2024

Subsystems (3)

v1 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

cover KVM: SEV: Add support for the ALLOWED_SEV_FEATURES feature

AMD EPYC 5th generation processors have introduced a feature that allows the hypervisor to control the SEV_FEATURES that are set for or by a guest. The ALLOWED_SEV_FEATURES can be used by the hypervisor to enforce that SEV-ES and SEV-SNP guests cannot enable features that the hypervisor does not want to be enabled.

Patch 1 adds support to detect the feature.

Patch 2 configures the ALLOWED_SEV_FEATURES field in the VMCB according to the features the hypervisor supports.

Based on tip/master commit a2767e7f31ad ("Merge branch into tip/master: 'x86/timers'")

Kishon Vijay Abraham I (2):
x86/cpufeatures: Add "Allowed SEV Features" Feature
KVM: SEV: Configure "ALLOWED_SEV_FEATURES" VMCB Field

 arch/x86/include/asm/cpufeatures.h | 1 +
 arch/x86/include/asm/svm.h         | 6 +++++-
 arch/x86/kvm/svm/sev.c             | 5 +++++
 3 files changed, 11 insertions(+), 1 deletion(-)
Signature · 1 line
2.34.1
#1 lore raw

PATCH 1/2 x86/cpufeatures: Add "Allowed SEV Features" Feature

From: Kishon Vijay Abraham I <[email protected]>

Add CPU feature detection for "Allowed SEV Features" to allow the Hypervisor to enforce that SEV-ES and SEV-SNP guest VMs cannot enable features (via SEV_FEATURES) that the Hypervisor does not support or wish to be enabled.

---
 arch/x86/include/asm/cpufeatures.h | 1 +
 1 file changed, 1 insertion(+)
arch/x86/include/asm/cpufeatures.h +1 -0
index dd4682857c12..0c73da91a041 100644
@@ -447,6 +447,7 @@
 #define X86_FEATURE_V_TSC_AUX		(19*32+ 9) /* Virtual TSC_AUX */
 #define X86_FEATURE_SME_COHERENT	(19*32+10) /* AMD hardware-enforced cache coherency */
 #define X86_FEATURE_DEBUG_SWAP		(19*32+14) /* "debug_swap" AMD SEV-ES full debug state swap support */
+#define X86_FEATURE_ALLOWED_SEV_FEATURES (19*32+27) /* AMD Allowed SEV Features */
 #define X86_FEATURE_SVSM		(19*32+28) /* "svsm" SVSM present */
 
 /* AMD-defined Extended Feature 2 EAX, CPUID level 0x80000021 (EAX), word 20 */
Signature · 1 line
2.34.1
#2 lore raw

Re: [PATCH 1/2] x86/cpufeatures: Add "Allowed SEV Features" Feature

Quoted 6 lines · Kim Phillips

From: Kishon Vijay Abraham I <[email protected]>

Add CPU feature detection for "Allowed SEV Features" to allow the Hypervisor to enforce that SEV-ES and SEV-SNP guest VMs cannot enable features (via SEV_FEATURES) that the Hypervisor does not support or wish to be enabled.

Can you please add this new feature bit to the CPUID database

    https://gitlab.com/x86-cpuid.org/x86-cpuid-db

Thanks,

tglx

#3 lore raw

PATCH 2/2 KVM: SEV: Configure "ALLOWED_SEV_FEATURES" VMCB Field

In reply to Kim Phillips

From: Kishon Vijay Abraham I <[email protected]>

AMD EPYC 5th generation processors have introduced a feature that allows the hypervisor to control the SEV_FEATURES that are set for or by a guest [1]. The ALLOWED_SEV_FEATURES feature can be used by the hypervisor to enforce that SEV-ES and SEV-SNP guests cannot enable features that the hypervisor does not want to be enabled.

When ALLOWED_SEV_FEATURES is enabled, a VMRUN will fail if any non-reserved bits are 1 in SEV_FEATURES but are 0 in ALLOWED_SEV_FEATURES.

[1] Section 15.36.20 "Allowed SEV Features", AMD64 Architecture

    Programmer's Manual, Pub. 24593 Rev. 3.42 - March 2024:
    https://bugzilla.kernel.org/attachment.cgi?id=306250
---
 arch/x86/include/asm/svm.h | 6 +++++-
 arch/x86/kvm/svm/sev.c     | 5 +++++
 2 files changed, 10 insertions(+), 1 deletion(-)
arch/x86/include/asm/svm.h +5 -1
index f0dea3750ca9..59516ad2028b 100644
@@ -158,7 +158,9 @@ struct __attribute__ ((__packed__)) vmcb_control_area {
 	u64 avic_physical_id;	/* Offset 0xf8 */
 	u8 reserved_7[8];
 	u64 vmsa_pa;		/* Used for an SEV-ES guest */
-	u8 reserved_8[720];
+	u8 reserved_8[40];
+	u64 allowed_sev_features;	/* Offset 0x138 */
+	u8 reserved_9[672];
 	/*
 	 * Offset 0x3e0, 32 bytes reserved
 	 * for use by hypervisor/software.
@@ -294,6 +296,8 @@ static_assert((X2AVIC_MAX_PHYSICAL_ID & AVIC_PHYSICAL_MAX_INDEX_MASK) == X2AVIC_
 	(SVM_SEV_FEAT_RESTRICTED_INJECTION |	\
 	 SVM_SEV_FEAT_ALTERNATE_INJECTION)
 
+#define VMCB_ALLOWED_SEV_FEATURES_VALID		BIT_ULL(63)
+
 struct vmcb_seg {
 	u16 selector;
 	u16 attrib;
arch/x86/kvm/svm/sev.c +5 -0
index a16c873b3232..d12b4d615b32 100644
@@ -899,6 +899,7 @@ static int sev_es_sync_vmsa(struct vcpu_svm *svm)
 static int __sev_launch_update_vmsa(struct kvm *kvm, struct kvm_vcpu *vcpu,
 				    int *error)
 {
+	struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
 	struct sev_data_launch_update_vmsa vmsa;
 	struct vcpu_svm *svm = to_svm(vcpu);
 	int ret;
@@ -908,6 +909,10 @@ static int __sev_launch_update_vmsa(struct kvm *kvm, struct kvm_vcpu *vcpu,
 		return -EINVAL;
 	}
 
+	if (cpu_feature_enabled(X86_FEATURE_ALLOWED_SEV_FEATURES))
+		svm->vmcb->control.allowed_sev_features = VMCB_ALLOWED_SEV_FEATURES_VALID |
+							  sev->vmsa_features;
+
 	/* Perform some pre-encryption checks against the VMSA */
 	ret = sev_es_sync_vmsa(svm);
 	if (ret)
Signature · 1 line
2.34.1
Reviewed-by on v1 2/2
#4 lore raw

Re: [PATCH 2/2] KVM: SEV: Configure "ALLOWED_SEV_FEATURES" VMCB Field

Quoted 12 lines · Kim Phillips

From: Kishon Vijay Abraham I <[email protected]>

AMD EPYC 5th generation processors have introduced a feature that allows the hypervisor to control the SEV_FEATURES that are set for or by a guest [1]. The ALLOWED_SEV_FEATURES feature can be used by the hypervisor to enforce that SEV-ES and SEV-SNP guests cannot enable features that the hypervisor does not want to be enabled.

When ALLOWED_SEV_FEATURES is enabled, a VMRUN will fail if any non-reserved bits are 1 in SEV_FEATURES but are 0 in ALLOWED_SEV_FEATURES.

Quoted 6 lines

[1] Section 15.36.20 "Allowed SEV Features", AMD64 Architecture

    Programmer's Manual, Pub. 24593 Rev. 3.42 - March 2024:
    https://bugzilla.kernel.org/attachment.cgi?id=306250

Looks good to me.

Quoted 52 lines
---
 arch/x86/include/asm/svm.h | 6 +++++-
 arch/x86/kvm/svm/sev.c     | 5 +++++
 2 files changed, 10 insertions(+), 1 deletion(-)
arch/x86/include/asm/svm.h +5 -1
index f0dea3750ca9..59516ad2028b 100644
@@ -158,7 +158,9 @@ struct __attribute__ ((__packed__)) vmcb_control_area {
 	u64 avic_physical_id;	/* Offset 0xf8 */
 	u8 reserved_7[8];
 	u64 vmsa_pa;		/* Used for an SEV-ES guest */
-	u8 reserved_8[720];
+	u8 reserved_8[40];
+	u64 allowed_sev_features;	/* Offset 0x138 */
+	u8 reserved_9[672];
 	/*
 	 * Offset 0x3e0, 32 bytes reserved
 	 * for use by hypervisor/software.
@@ -294,6 +296,8 @@ static_assert((X2AVIC_MAX_PHYSICAL_ID & AVIC_PHYSICAL_MAX_INDEX_MASK) == X2AVIC_
 	(SVM_SEV_FEAT_RESTRICTED_INJECTION |	\
 	 SVM_SEV_FEAT_ALTERNATE_INJECTION)
 
+#define VMCB_ALLOWED_SEV_FEATURES_VALID		BIT_ULL(63)
+
 struct vmcb_seg {
 	u16 selector;
 	u16 attrib;
arch/x86/kvm/svm/sev.c +5 -0
index a16c873b3232..d12b4d615b32 100644
@@ -899,6 +899,7 @@ static int sev_es_sync_vmsa(struct vcpu_svm *svm)
 static int __sev_launch_update_vmsa(struct kvm *kvm, struct kvm_vcpu *vcpu,
 				    int *error)
 {
+	struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
 	struct sev_data_launch_update_vmsa vmsa;
 	struct vcpu_svm *svm = to_svm(vcpu);
 	int ret;
@@ -908,6 +909,10 @@ static int __sev_launch_update_vmsa(struct kvm *kvm, struct kvm_vcpu *vcpu,
 		return -EINVAL;
 	}
 
+	if (cpu_feature_enabled(X86_FEATURE_ALLOWED_SEV_FEATURES))
+		svm->vmcb->control.allowed_sev_features = VMCB_ALLOWED_SEV_FEATURES_VALID |
+							  sev->vmsa_features;
+
 	/* Perform some pre-encryption checks against the VMSA */
 	ret = sev_es_sync_vmsa(svm);
 	if (ret)
#5 lore raw

Re: [PATCH 2/2] KVM: SEV: Configure "ALLOWED_SEV_FEATURES" VMCB Field

In reply to Kim Phillips
Quoted 7 lines · Kim Phillips

From: Kishon Vijay Abraham I <[email protected]>

AMD EPYC 5th generation processors have introduced a feature that allows the hypervisor to control the SEV_FEATURES that are set for or by a guest [1]. The ALLOWED_SEV_FEATURES feature can be used by the hypervisor to enforce that SEV-ES and SEV-SNP guests cannot enable features that the hypervisor does not want to be enabled.

How does the host communicate to the guest which features are allowed? And based on this blurb:

Some SEV features can only be used if the Allowed SEV Features Mask is enabled, and the mask is configured to permit the corresponding feature. If the Allowed SEV Features Mask is not enabled, these features are not available (see SEV_FEATURES in Appendix B, Table B-4).

and the appendix, this only applies to PmcVirtualization and SecureAvic. Adding that info in the changelog would be *very* helpful.

And I see that SVM_SEV_FEAT_DEBUG_SWAP, a.k.a. DebugVirtualization, is a guest controlled feature and doesn't honor ALLOWED_SEV_FEATURES. Doesn't that mean sev_vcpu_has_debug_swap() is broken, i.e. that KVM must assume the guest can DebugVirtualization on and off at will? Or am I missing something?

#6 lore raw

Re: [PATCH 2/2] KVM: SEV: Configure "ALLOWED_SEV_FEATURES" VMCB Field

Quoted 10 lines · Sean Christopherson
Quoted 7 lines · Kim Phillips

From: Kishon Vijay Abraham I <[email protected]>

AMD EPYC 5th generation processors have introduced a feature that allows the hypervisor to control the SEV_FEATURES that are set for or by a guest [1]. The ALLOWED_SEV_FEATURES feature can be used by the hypervisor to enforce that SEV-ES and SEV-SNP guests cannot enable features that the hypervisor does not want to be enabled.

How does the host communicate to the guest which features are allowed?

I'm not familiar with any future plans to negotiate with the guest directly, but since commit ac5c48027bac ("KVM: SEV: publish supported VMSA features"), userspace can retrieve sev_supported_vmsa_features via an ioctl.

Quoted 9 lines

And based on this blurb:

Some SEV features can only be used if the Allowed SEV Features Mask is enabled, and the mask is configured to permit the corresponding feature. If the Allowed SEV Features Mask is not enabled, these features are not available (see SEV_FEATURES in Appendix B, Table B-4).

and the appendix, this only applies to PmcVirtualization and SecureAvic. Adding that info in the changelog would be *very* helpful.

Ok, how about adding:

"The PmcVirtualization and SecureAvic features explicitly require ALLOWED_SEV_FEATURES to enable them before they can be used."

Quoted 4 lines

And I see that SVM_SEV_FEAT_DEBUG_SWAP, a.k.a. DebugVirtualization, is a guest controlled feature and doesn't honor ALLOWED_SEV_FEATURES. Doesn't that mean sev_vcpu_has_debug_swap() is broken, i.e. that KVM must assume the guest can DebugVirtualization on and off at will? Or am I missing something?

My understanding is that users control KVM's DEBUG_SWAP setting with a module parameter since commit 4dd5ecacb9a4 ("KVM: SEV: allow SEV-ES DebugSwap again"). If the module parameter is not set, with this patch, VMRUN will fail since the host doesn't allow DEBUG_SWAP.

Thanks for your review!

Kim

#7 lore raw

Re: [PATCH 2/2] KVM: SEV: Configure "ALLOWED_SEV_FEATURES" VMCB Field

Quoted 10 lines · Kim Phillips
Quoted 9 lines · Sean Christopherson
Quoted 7 lines · Kim Phillips

From: Kishon Vijay Abraham I <[email protected]>

AMD EPYC 5th generation processors have introduced a feature that allows the hypervisor to control the SEV_FEATURES that are set for or by a guest [1]. The ALLOWED_SEV_FEATURES feature can be used by the hypervisor to enforce that SEV-ES and SEV-SNP guests cannot enable features that the hypervisor does not want to be enabled.

Quoted 3 lines
Quoted 1 line

How does the host communicate to the guest which features are allowed?

I'm not familiar with any future plans to negotiate with the guest directly,

I feel like I'm missing something. What happens if the guest wants to enable PmcVirtualization and it's unexpectedly disallowed? Does the guest simply panic?

Quoted 23 lines

but since commit ac5c48027bac ("KVM: SEV: publish supported VMSA features"), userspace can retrieve sev_supported_vmsa_features via an ioctl.

Quoted 9 lines

And based on this blurb:

Some SEV features can only be used if the Allowed SEV Features Mask is enabled, and the mask is configured to permit the corresponding feature. If the Allowed SEV Features Mask is not enabled, these features are not available (see SEV_FEATURES in Appendix B, Table B-4).

and the appendix, this only applies to PmcVirtualization and SecureAvic. Adding that info in the changelog would be *very* helpful.

Ok, how about adding:

"The PmcVirtualization and SecureAvic features explicitly require ALLOWED_SEV_FEATURES to enable them before they can be used."

Quoted 4 lines

And I see that SVM_SEV_FEAT_DEBUG_SWAP, a.k.a. DebugVirtualization, is a guest controlled feature and doesn't honor ALLOWED_SEV_FEATURES. Doesn't that mean sev_vcpu_has_debug_swap() is broken, i.e. that KVM must assume the guest can DebugVirtualization on and off at will? Or am I missing something?

Quoted 4 lines

My understanding is that users control KVM's DEBUG_SWAP setting with a module parameter since commit 4dd5ecacb9a4 ("KVM: SEV: allow SEV-ES DebugSwap again"). If the module parameter is not set, with this patch, VMRUN will fail since the host doesn't allow DEBUG_SWAP.

But that's just KVM's view of vmsa_features. With SNP's wonderful SVM_VMGEXIT_AP_CREATE, can't the guest create a VMSA with whatever sev_features it wants, so long as they aren't host-controllable, i.e. aren't PmcVirtualization or SecureAvic?

#8 lore raw

Re: [PATCH 2/2] KVM: SEV: Configure "ALLOWED_SEV_FEATURES" VMCB Field

Quoted 11 lines · Sean Christopherson
Quoted 10 lines · Kim Phillips
Quoted 9 lines · Sean Christopherson
Quoted 7 lines · Kim Phillips

From: Kishon Vijay Abraham I <[email protected]>

AMD EPYC 5th generation processors have introduced a feature that allows the hypervisor to control the SEV_FEATURES that are set for or by a guest [1]. The ALLOWED_SEV_FEATURES feature can be used by the hypervisor to enforce that SEV-ES and SEV-SNP guests cannot enable features that the hypervisor does not want to be enabled.

Quoted 6 lines
Quoted 3 lines
Quoted 1 line

How does the host communicate to the guest which features are allowed?

I'm not familiar with any future plans to negotiate with the guest directly,

I feel like I'm missing something. What happens if the guest wants to enable PmcVirtualization and it's unexpectedly disallowed? Does the guest simply panic?

In SNP, VMRUN will return with an exit code of VMEXIT_INVALID (0xffffffff) if the guest tries to set it.

In SEV-ES, the hypervisor can set it, and the same thing will happen to VMRUN.

In both cases, SEV_FEATURES is saved to VMCB field GUEST_SEV_FEATURES at offset 140h on the VMEXIT, indicating to the host which feature was attempted but caught as not allowed.

Quoted 28 lines
Quoted 23 lines

but since commit ac5c48027bac ("KVM: SEV: publish supported VMSA features"), userspace can retrieve sev_supported_vmsa_features via an ioctl.

Quoted 9 lines

And based on this blurb:

Some SEV features can only be used if the Allowed SEV Features Mask is enabled, and the mask is configured to permit the corresponding feature. If the Allowed SEV Features Mask is not enabled, these features are not available (see SEV_FEATURES in Appendix B, Table B-4).

and the appendix, this only applies to PmcVirtualization and SecureAvic. Adding that info in the changelog would be *very* helpful.

Ok, how about adding:

"The PmcVirtualization and SecureAvic features explicitly require ALLOWED_SEV_FEATURES to enable them before they can be used."

Quoted 4 lines

And I see that SVM_SEV_FEAT_DEBUG_SWAP, a.k.a. DebugVirtualization, is a guest controlled feature and doesn't honor ALLOWED_SEV_FEATURES. Doesn't that mean sev_vcpu_has_debug_swap() is broken, i.e. that KVM must assume the guest can DebugVirtualization on and off at will? Or am I missing something?

Quoted 4 lines

My understanding is that users control KVM's DEBUG_SWAP setting with a module parameter since commit 4dd5ecacb9a4 ("KVM: SEV: allow SEV-ES DebugSwap again"). If the module parameter is not set, with this patch, VMRUN will fail since the host doesn't allow DEBUG_SWAP.

Quoted 4 lines

But that's just KVM's view of vmsa_features. With SNP's wonderful SVM_VMGEXIT_AP_CREATE, can't the guest create a VMSA with whatever sev_features it wants, so long as they aren't host-controllable, i.e. aren't PmcVirtualization or SecureAvic?

No, as above, if the guest tries any silly business the host will get a VMEXIT_INVALID, no matter if using the feature *requires* ALLOWED_SEV_FEATURES to be enabled and explicitly allow it (currently PmcVirtualization or SecureAvic).

Kim

#9 lore raw

Re: [PATCH 2/2] KVM: SEV: Configure "ALLOWED_SEV_FEATURES" VMCB Field

Quoted 35 lines · Kim Phillips
Quoted 29 lines · Sean Christopherson
Quoted 23 lines · Kim Phillips

but since commit ac5c48027bac ("KVM: SEV: publish supported VMSA features"), userspace can retrieve sev_supported_vmsa_features via an ioctl.

Quoted 9 lines

And based on this blurb:

Some SEV features can only be used if the Allowed SEV Features Mask is enabled, and the mask is configured to permit the corresponding feature. If the Allowed SEV Features Mask is not enabled, these features are not available (see SEV_FEATURES in Appendix B, Table B-4).

and the appendix, this only applies to PmcVirtualization and SecureAvic. Adding that info in the changelog would be *very* helpful.

Ok, how about adding:

"The PmcVirtualization and SecureAvic features explicitly require ALLOWED_SEV_FEATURES to enable them before they can be used."

Quoted 4 lines

And I see that SVM_SEV_FEAT_DEBUG_SWAP, a.k.a. DebugVirtualization, is a guest controlled feature and doesn't honor ALLOWED_SEV_FEATURES. Doesn't that mean sev_vcpu_has_debug_swap() is broken, i.e. that KVM must assume the guest can DebugVirtualization on and off at will? Or am I missing something?

Quoted 4 lines

My understanding is that users control KVM's DEBUG_SWAP setting with a module parameter since commit 4dd5ecacb9a4 ("KVM: SEV: allow SEV-ES DebugSwap again"). If the module parameter is not set, with this patch, VMRUN will fail since the host doesn't allow DEBUG_SWAP.

Quoted 4 lines

But that's just KVM's view of vmsa_features. With SNP's wonderful SVM_VMGEXIT_AP_CREATE, can't the guest create a VMSA with whatever sev_features it wants, so long as they aren't host-controllable, i.e. aren't PmcVirtualization or SecureAvic?

Quoted 4 lines

No, as above, if the guest tries any silly business the host will get a VMEXIT_INVALID, no matter if using the feature *requires* ALLOWED_SEV_FEATURES to be enabled and explicitly allow it (currently PmcVirtualization or SecureAvic).

Oooh, I finally get it. PmcVirtualization and SecureAvic require an opt-in via ALLOWED_SEV_FEATURES, i.e. are off-by-default, whereas all other features are effectively on-by-default, but still honor ALLOWED_SEV_FEATURES.