Skip to contentSkip to conversation

x86/sev: Add SEV-SNP guest feature negotiation support

PATCH Superseded Nikunj A Dadhania 3 participants 11 messages Jan 2, 2023 – Jan 4, 2023

Subsystems (3)

v3 of 7 Superseded Latest is v7 →

lore
Share & tools
lore
#0 lore raw

PATCH v3 1/1 x86/sev: Add SEV-SNP guest feature negotiation support

The hypervisor can enable various new features (SEV_FEATURES[1:63]) and start the SNP guest. Some of these features need guest side implementation. If any of these features are enabled without guest side implementation, the behavior of the SNP guest will be undefined. The SNP guest boot may fail in a non-obvious way making it difficult to debug.

Instead of allowing the guest to continue and have it fail randomly later, detect this early and fail gracefully.

SEV_STATUS MSR indicates features which hypervisor has enabled. While booting, SNP guests should ascertain that all the enabled features have guest side implementation. In case any feature is not implemented in the guest, the guest terminates booting with SNP feature unsupported exit code.

More details in AMD64 APM[1] Vol 2: 15.34.10 SEV_STATUS MSR

[1] https://www.amd.com/system/files/TechDocs/40332_4.05.pdf

Fixes cbd3d4f7c4e5 ("x86/sev: Check SEV-SNP features support")
CC Borislav Petkov <[email protected]>
CC Michael Roth <[email protected]>
CC Tom Lendacky <[email protected]>

---

Changes:
v2:
* Updated Documentation/x86/amd-memory-encryption.rst
* Address review feedback from Boris/Tom

v1:
* Dropped _ENABLED from the feature bits
* Use approprate macro/function names and move closer to the function where it is used.
* More details added to the commit message and comments
* Fixed compilation issue

---
 Documentation/x86/amd-memory-encryption.rst | 35 +++++++++++++++++++
 arch/x86/boot/compressed/sev.c              | 37 +++++++++++++++++++++
 arch/x86/include/asm/msr-index.h            | 20 +++++++++++
 arch/x86/include/asm/sev-common.h           |  1 +
 4 files changed, 93 insertions(+)
Documentation/x86/amd-memory-encryption.rst +35 -0
index a1940ebe7be5..b8b6b87be995 100644
@@ -95,3 +95,38 @@ by supplying mem_encrypt=on on the kernel command line.  However, if BIOS does
 not enable SME, then Linux will not be able to activate memory encryption, even
 if configured to do so by default or the mem_encrypt=on command line parameter
 is specified.
+
+Secure Nested Paging (SNP):
+===========================
+SEV-SNP introduces new features (SEV_FEATURES[1:63]) which can be enabled
+by the hypervisor for security enhancements. Some of these features need
+guest side implementation to function correctly. The below table lists the
+expected guest behavior with various possible scenarios of guest/hypervisor
+SNP feature support.
+
++---------------+---------------+---------------+---------------+
+|Feature Enabled|  Guest needs  |   Guest has   |  Guest boot   |
+|     by HV     |implementation |implementation |   behavior    |
++---------------+---------------+---------------+---------------+
+|      No       |      No       |      No       |     Boot      |
+|               |               |               |               |
++---------------+---------------+---------------+---------------+
+|      No       |      Yes      |      No       |     Boot      |
+|               |               |               |               |
++---------------+---------------+---------------+---------------+
+|      No       |      Yes      |      Yes      |     Boot      |
+|               |               |               |               |
++---------------+---------------+---------------+---------------+
+|      Yes      |      No       |      No       |   Boot with   |
+|               |               |               |feature enabled|
++---------------+---------------+---------------+---------------+
+|      Yes      |      Yes      |      No       | Graceful Boot |
+|               |               |               |    Failure    |
++---------------+---------------+---------------+---------------+
+|      Yes      |      Yes      |      Yes      |   Boot with   |
+|               |               |               |feature enabled|
++---------------+---------------+---------------+---------------+
+
+More details in AMD64 APM[1] Vol 2: 15.34.10 SEV_STATUS MSR
+
+[1] https://www.amd.com/system/files/TechDocs/40332_4.05.pdf
arch/x86/boot/compressed/sev.c +37 -0
index c93930d5ccbd..43793f46cfc1 100644
@@ -270,6 +270,36 @@ static void enforce_vmpl0(void)
 		sev_es_terminate(SEV_TERM_SET_LINUX, GHCB_TERM_NOT_VMPL0);
 }
 
+/*
+ * SNP_FEATURES_IMPL_REQ is the mask of SNP features that will need
+ * guest side implementation for proper functioning of the guest. If any
+ * of these features are enabled in the hypervisor but are lacking guest
+ * side implementation, the behavior of the guest will be undefined. The
+ * guest could fail in non-obvious way making it difficult to debug.
+ *
+ * As the behavior of reserved feature bits is unknown to be on the
+ * safe side add them to the required features mask.
+ */
+#define SNP_FEATURES_IMPL_REQ	(MSR_AMD64_SNP_VTOM |			\
+				MSR_AMD64_SNP_REFLECT_VC |		\
+				MSR_AMD64_SNP_RESTRICTED_INJ |		\
+				MSR_AMD64_SNP_ALT_INJ |			\
+				MSR_AMD64_SNP_DEBUG_SWAP |		\
+				MSR_AMD64_SNP_VMPL_SSS |		\
+				MSR_AMD64_SNP_SECURE_TSC |		\
+				MSR_AMD64_SNP_VMGEXIT_PARAM |		\
+				MSR_AMD64_SNP_VMSA_REG_PROTECTION |	\
+				MSR_AMD64_SNP_RESERVED_BIT13 |		\
+				MSR_AMD64_SNP_RESERVED_BIT15 |		\
+				MSR_AMD64_SNP_RESERVED_MASK)
+
+/*
+ * SNP_FEATURES_PRESENT is the mask of SNP features that are implemented
+ * by the guest kernel. As and when a new feature is implemented in the
+ * guest kernel, a corresponding bit should be added to the mask.
+ */
+#define SNP_FEATURES_PRESENT (0)
+
 void sev_enable(struct boot_params *bp)
 {
 	unsigned int eax, ebx, ecx, edx;
@@ -335,6 +365,13 @@ void sev_enable(struct boot_params *bp)
 		if (!(get_hv_features() & GHCB_HV_FT_SNP))
 			sev_es_terminate(SEV_TERM_SET_GEN, GHCB_SNP_UNSUPPORTED);
 
+		/*
+		 * Terminate the boot if hypervisor has enabled any feature
+		 * lacking guest side implementation.
+		 */
+		if (sev_status & SNP_FEATURES_IMPL_REQ & ~SNP_FEATURES_PRESENT)
+			sev_es_terminate(SEV_TERM_SET_GEN, GHCB_SNP_FEAT_NOT_IMPLEMENTED);
+
 		enforce_vmpl0();
 	}
 
arch/x86/include/asm/msr-index.h +20 -0
index 37ff47552bcb..d3fe82c5d6b6 100644
@@ -566,6 +566,26 @@
 #define MSR_AMD64_SEV_ES_ENABLED	BIT_ULL(MSR_AMD64_SEV_ES_ENABLED_BIT)
 #define MSR_AMD64_SEV_SNP_ENABLED	BIT_ULL(MSR_AMD64_SEV_SNP_ENABLED_BIT)
 
+/* SNP feature bits enabled by the hypervisor */
+#define MSR_AMD64_SNP_VTOM			BIT_ULL(3)
+#define MSR_AMD64_SNP_REFLECT_VC		BIT_ULL(4)
+#define MSR_AMD64_SNP_RESTRICTED_INJ		BIT_ULL(5)
+#define MSR_AMD64_SNP_ALT_INJ			BIT_ULL(6)
+#define MSR_AMD64_SNP_DEBUG_SWAP		BIT_ULL(7)
+#define MSR_AMD64_SNP_PREVENT_HOST_IBS		BIT_ULL(8)
+#define MSR_AMD64_SNP_BTB_ISOLATION		BIT_ULL(9)
+#define MSR_AMD64_SNP_VMPL_SSS			BIT_ULL(10)
+#define MSR_AMD64_SNP_SECURE_TSC		BIT_ULL(11)
+#define MSR_AMD64_SNP_VMGEXIT_PARAM		BIT_ULL(12)
+#define MSR_AMD64_SNP_IBS_VIRT			BIT_ULL(14)
+#define MSR_AMD64_SNP_VMSA_REG_PROTECTION	BIT_ULL(16)
+#define MSR_AMD64_SNP_SMT_PROTECTION		BIT_ULL(17)
+
+/* SNP feature bits reserved for future use. */
+#define MSR_AMD64_SNP_RESERVED_BIT13		BIT_ULL(13)
+#define MSR_AMD64_SNP_RESERVED_BIT15		BIT_ULL(15)
+#define MSR_AMD64_SNP_RESERVED_MASK		GENMASK_ULL(63, 18)
+
 #define MSR_AMD64_VIRT_SPEC_CTRL	0xc001011f
 
 /* AMD Collaborative Processor Performance Control MSRs */
arch/x86/include/asm/sev-common.h +1 -0
index b8357d6ecd47..db60cbb01b31 100644
@@ -148,6 +148,7 @@ struct snp_psc_desc {
 #define GHCB_SEV_ES_GEN_REQ		0
 #define GHCB_SEV_ES_PROT_UNSUPPORTED	1
 #define GHCB_SNP_UNSUPPORTED		2
+#define GHCB_SNP_FEAT_NOT_IMPLEMENTED	3
 
 /* Linux-specific reason codes (used with reason set 1) */
 #define SEV_TERM_SET_LINUX		1
Signature · 1 line
2.32.0
#1 lore raw
Quoted 77 lines · Nikunj A Dadhania

The hypervisor can enable various new features (SEV_FEATURES[1:63]) and start the SNP guest. Some of these features need guest side implementation. If any of these features are enabled without guest side implementation, the behavior of the SNP guest will be undefined. The SNP guest boot may fail in a non-obvious way making it difficult to debug.

Instead of allowing the guest to continue and have it fail randomly later, detect this early and fail gracefully.

SEV_STATUS MSR indicates features which hypervisor has enabled. While booting, SNP guests should ascertain that all the enabled features have guest side implementation. In case any feature is not implemented in the guest, the guest terminates booting with SNP feature unsupported exit code.

More details in AMD64 APM[1] Vol 2: 15.34.10 SEV_STATUS MSR

[1] https://www.amd.com/system/files/TechDocs/40332_4.05.pdf

Fixes cbd3d4f7c4e5 ("x86/sev: Check SEV-SNP features support")
CC Borislav Petkov <[email protected]>
CC Michael Roth <[email protected]>
CC Tom Lendacky <[email protected]>

---

Changes:
v2:
* Updated Documentation/x86/amd-memory-encryption.rst
* Address review feedback from Boris/Tom

v1:
* Dropped _ENABLED from the feature bits
* Use approprate macro/function names and move closer to the function where it is used.
* More details added to the commit message and comments
* Fixed compilation issue

---
 Documentation/x86/amd-memory-encryption.rst | 35 +++++++++++++++++++
 arch/x86/boot/compressed/sev.c              | 37 +++++++++++++++++++++
 arch/x86/include/asm/msr-index.h            | 20 +++++++++++
 arch/x86/include/asm/sev-common.h           |  1 +
 4 files changed, 93 insertions(+)
Documentation/x86/amd-memory-encryption.rst +22 -0
index a1940ebe7be5..b8b6b87be995 100644
@@ -95,3 +95,38 @@ by supplying mem_encrypt=on on the kernel command line.  However, if BIOS does
 not enable SME, then Linux will not be able to activate memory encryption, even
 if configured to do so by default or the mem_encrypt=on command line parameter
 is specified.
+
+Secure Nested Paging (SNP):
+===========================
+SEV-SNP introduces new features (SEV_FEATURES[1:63]) which can be enabled
+by the hypervisor for security enhancements. Some of these features need
+guest side implementation to function correctly. The below table lists the
+expected guest behavior with various possible scenarios of guest/hypervisor
+SNP feature support.
+
++---------------+---------------+---------------+---------------+
+|Feature Enabled|  Guest needs  |   Guest has   |  Guest boot   |
+|     by HV     |implementation |implementation |   behavior    |
++---------------+---------------+---------------+---------------+
+|      No       |      No       |      No       |     Boot      |
+|               |               |               |               |
++---------------+---------------+---------------+---------------+
+|      No       |      Yes      |      No       |     Boot      |
+|               |               |               |               |
++---------------+---------------+---------------+---------------+
+|      No       |      Yes      |      Yes      |     Boot      |
+|               |               |               |               |
++---------------+---------------+---------------+---------------+

Documentation/x86/amd-memory-encryption.rst by supplying mem_encrypt=on on the kernel command line. However, if BIOS does · line 129

Quoted 10 lines
Documentation/x86/amd-memory-encryption.rst +10 -0
+|      Yes      |      No       |      No       |   Boot with   |
+|               |               |               |feature enabled|
++---------------+---------------+---------------+---------------+
+|      Yes      |      Yes      |      No       | Graceful Boot |
+|               |               |               |    Failure    |
++---------------+---------------+---------------+---------------+
+|      Yes      |      Yes      |      Yes      |   Boot with   |
+|               |               |               |feature enabled|
++---------------+---------------+---------------+---------------+
+

Couple things:

- I'd assume that the documentation would refer the reader to the SNP_FEATURES_IMPL_REQ mask that defines whether the guest is required to have a specific feature or not.

- Don't hate me, but I feel wanting more from this, such as a rationale for why certain features are required in the guest. When a guest fails to boot and the reasoning provided by a cloud provider is "your guest doesn't support ${feature}", the natural question to ask will be "why do I need ${feature}?"

Quoted 10 lines
Documentation/x86/amd-memory-encryption.rst +3 -0
+More details in AMD64 APM[1] Vol 2: 15.34.10 SEV_STATUS MSR
+
+[1] https://www.amd.com/system/files/TechDocs/40332_4.05.pdf
arch/x86/boot/compressed/sev.c +0 -0
index c93930d5ccbd..43793f46cfc1 100644
@@ -270,6 +270,36 @@ static void enforce_vmpl0(void)
 		sev_es_terminate(SEV_TERM_SET_LINUX, GHCB_TERM_NOT_VMPL0);
 }

arch/x86/boot/compressed/sev.c static void enforce_vmpl0(void) · line 281

Quoted 10 lines
arch/x86/boot/compressed/sev.c +9 -0
 
+/*
+ * SNP_FEATURES_IMPL_REQ is the mask of SNP features that will need
+ * guest side implementation for proper functioning of the guest. If any
+ * of these features are enabled in the hypervisor but are lacking guest
+ * side implementation, the behavior of the guest will be undefined. The
+ * guest could fail in non-obvious way making it difficult to debug.
+ *
+ * As the behavior of reserved feature bits is unknown to be on the
+ * safe side add them to the required features mask.

If one or more of the reserved feature bits turns out to not be required for the SNP guest to function correctly, is this the correct choice?

IIUC, legacy guests would fail to boot correctly (and unnecessarily, because of this change) if they do not have an implentation for a non-required feature. Absent this change, however, they would proceed just fine.

Quoted 25 lines
arch/x86/boot/compressed/sev.c +21 -0
+ */
+#define SNP_FEATURES_IMPL_REQ	(MSR_AMD64_SNP_VTOM |			\
+				MSR_AMD64_SNP_REFLECT_VC |		\
+				MSR_AMD64_SNP_RESTRICTED_INJ |		\
+				MSR_AMD64_SNP_ALT_INJ |			\
+				MSR_AMD64_SNP_DEBUG_SWAP |		\
+				MSR_AMD64_SNP_VMPL_SSS |		\
+				MSR_AMD64_SNP_SECURE_TSC |		\
+				MSR_AMD64_SNP_VMGEXIT_PARAM |		\
+				MSR_AMD64_SNP_VMSA_REG_PROTECTION |	\
+				MSR_AMD64_SNP_RESERVED_BIT13 |		\
+				MSR_AMD64_SNP_RESERVED_BIT15 |		\
+				MSR_AMD64_SNP_RESERVED_MASK)
+
+/*
+ * SNP_FEATURES_PRESENT is the mask of SNP features that are implemented
+ * by the guest kernel. As and when a new feature is implemented in the
+ * guest kernel, a corresponding bit should be added to the mask.
+ */
+#define SNP_FEATURES_PRESENT (0)
+
 void sev_enable(struct boot_params *bp)
 {
 	unsigned int eax, ebx, ecx, edx;

arch/x86/boot/compressed/sev.c void sev_enable(struct boot_params *bp) · line 373

Quoted 9 lines
arch/x86/boot/compressed/sev.c +6 -0
@@ -335,6 +365,13 @@ void sev_enable(struct boot_params *bp)
 		if (!(get_hv_features() & GHCB_HV_FT_SNP))
 			sev_es_terminate(SEV_TERM_SET_GEN, GHCB_SNP_UNSUPPORTED);
 
+		/*
+		 * Terminate the boot if hypervisor has enabled any feature
+		 * lacking guest side implementation.
+		 */
+		if (sev_status & SNP_FEATURES_IMPL_REQ & ~SNP_FEATURES_PRESENT)
+			sev_es_terminate(SEV_TERM_SET_GEN, GHCB_SNP_FEAT_NOT_IMPLEMENTED);

We can't help out by specifying which feature(s)?

Quoted 51 lines
arch/x86/boot/compressed/sev.c +1 -0
+
 		enforce_vmpl0();
 	}
 
arch/x86/include/asm/msr-index.h +20 -0
index 37ff47552bcb..d3fe82c5d6b6 100644
@@ -566,6 +566,26 @@
 #define MSR_AMD64_SEV_ES_ENABLED	BIT_ULL(MSR_AMD64_SEV_ES_ENABLED_BIT)
 #define MSR_AMD64_SEV_SNP_ENABLED	BIT_ULL(MSR_AMD64_SEV_SNP_ENABLED_BIT)
 
+/* SNP feature bits enabled by the hypervisor */
+#define MSR_AMD64_SNP_VTOM			BIT_ULL(3)
+#define MSR_AMD64_SNP_REFLECT_VC		BIT_ULL(4)
+#define MSR_AMD64_SNP_RESTRICTED_INJ		BIT_ULL(5)
+#define MSR_AMD64_SNP_ALT_INJ			BIT_ULL(6)
+#define MSR_AMD64_SNP_DEBUG_SWAP		BIT_ULL(7)
+#define MSR_AMD64_SNP_PREVENT_HOST_IBS		BIT_ULL(8)
+#define MSR_AMD64_SNP_BTB_ISOLATION		BIT_ULL(9)
+#define MSR_AMD64_SNP_VMPL_SSS			BIT_ULL(10)
+#define MSR_AMD64_SNP_SECURE_TSC		BIT_ULL(11)
+#define MSR_AMD64_SNP_VMGEXIT_PARAM		BIT_ULL(12)
+#define MSR_AMD64_SNP_IBS_VIRT			BIT_ULL(14)
+#define MSR_AMD64_SNP_VMSA_REG_PROTECTION	BIT_ULL(16)
+#define MSR_AMD64_SNP_SMT_PROTECTION		BIT_ULL(17)
+
+/* SNP feature bits reserved for future use. */
+#define MSR_AMD64_SNP_RESERVED_BIT13		BIT_ULL(13)
+#define MSR_AMD64_SNP_RESERVED_BIT15		BIT_ULL(15)
+#define MSR_AMD64_SNP_RESERVED_MASK		GENMASK_ULL(63, 18)
+
 #define MSR_AMD64_VIRT_SPEC_CTRL	0xc001011f
 
 /* AMD Collaborative Processor Performance Control MSRs */
arch/x86/include/asm/sev-common.h +1 -0
index b8357d6ecd47..db60cbb01b31 100644
@@ -148,6 +148,7 @@ struct snp_psc_desc {
 #define GHCB_SEV_ES_GEN_REQ		0
 #define GHCB_SEV_ES_PROT_UNSUPPORTED	1
 #define GHCB_SNP_UNSUPPORTED		2
+#define GHCB_SNP_FEAT_NOT_IMPLEMENTED	3
 
 /* Linux-specific reason codes (used with reason set 1) */
 #define SEV_TERM_SET_LINUX		1
Signature · 1 line
2.32.0
#2 lore raw
Quoted 92 lines · David Rientjes
Quoted 77 lines · Nikunj A Dadhania

The hypervisor can enable various new features (SEV_FEATURES[1:63]) and start the SNP guest. Some of these features need guest side implementation. If any of these features are enabled without guest side implementation, the behavior of the SNP guest will be undefined. The SNP guest boot may fail in a non-obvious way making it difficult to debug.

Instead of allowing the guest to continue and have it fail randomly later, detect this early and fail gracefully.

SEV_STATUS MSR indicates features which hypervisor has enabled. While booting, SNP guests should ascertain that all the enabled features have guest side implementation. In case any feature is not implemented in the guest, the guest terminates booting with SNP feature unsupported exit code.

More details in AMD64 APM[1] Vol 2: 15.34.10 SEV_STATUS MSR

[1] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.amd.com%2Fsystem%2Ffiles%2FTechDocs%2F40332_4.05.pdf&data=05%7C01%7Cnikunj.dadhania%40amd.com%7Cd4d286d55435487f7d4f08daecb3d3e6%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C638082554474351848%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=TLBCkg20WCaF77YrOzGC%2Bn9V3JFgVl2vs1XMPmSWHcM%3D&reserved=0

Fixes cbd3d4f7c4e5 ("x86/sev: Check SEV-SNP features support")
CC Borislav Petkov <[email protected]>
CC Michael Roth <[email protected]>
CC Tom Lendacky <[email protected]>

---

Changes:
v2:
* Updated Documentation/x86/amd-memory-encryption.rst
* Address review feedback from Boris/Tom

v1:
* Dropped _ENABLED from the feature bits
* Use approprate macro/function names and move closer to the function where it is used.
* More details added to the commit message and comments
* Fixed compilation issue

---
 Documentation/x86/amd-memory-encryption.rst | 35 +++++++++++++++++++
 arch/x86/boot/compressed/sev.c              | 37 +++++++++++++++++++++
 arch/x86/include/asm/msr-index.h            | 20 +++++++++++
 arch/x86/include/asm/sev-common.h           |  1 +
 4 files changed, 93 insertions(+)
Documentation/x86/amd-memory-encryption.rst +22 -0
index a1940ebe7be5..b8b6b87be995 100644
@@ -95,3 +95,38 @@ by supplying mem_encrypt=on on the kernel command line.  However, if BIOS does
 not enable SME, then Linux will not be able to activate memory encryption, even
 if configured to do so by default or the mem_encrypt=on command line parameter
 is specified.
+
+Secure Nested Paging (SNP):
+===========================
+SEV-SNP introduces new features (SEV_FEATURES[1:63]) which can be enabled
+by the hypervisor for security enhancements. Some of these features need
+guest side implementation to function correctly. The below table lists the
+expected guest behavior with various possible scenarios of guest/hypervisor
+SNP feature support.
+
++---------------+---------------+---------------+---------------+
+|Feature Enabled|  Guest needs  |   Guest has   |  Guest boot   |
+|     by HV     |implementation |implementation |   behavior    |
++---------------+---------------+---------------+---------------+
+|      No       |      No       |      No       |     Boot      |
+|               |               |               |               |
++---------------+---------------+---------------+---------------+
+|      No       |      Yes      |      No       |     Boot      |
+|               |               |               |               |
++---------------+---------------+---------------+---------------+
+|      No       |      Yes      |      Yes      |     Boot      |
+|               |               |               |               |
++---------------+---------------+---------------+---------------+

Documentation/x86/amd-memory-encryption.rst by supplying mem_encrypt=on on the kernel command line. However, if BIOS does · line 129

Quoted 10 lines
Documentation/x86/amd-memory-encryption.rst +10 -0
+|      Yes      |      No       |      No       |   Boot with   |
+|               |               |               |feature enabled|
++---------------+---------------+---------------+---------------+
+|      Yes      |      Yes      |      No       | Graceful Boot |
+|               |               |               |    Failure    |
++---------------+---------------+---------------+---------------+
+|      Yes      |      Yes      |      Yes      |   Boot with   |
+|               |               |               |feature enabled|
++---------------+---------------+---------------+---------------+
+

Couple things:

Quoted 5 lines

- I'd assume that the documentation would refer the reader to the SNP_FEATURES_IMPL_REQ mask that defines whether the guest is required to have a specific feature or not.>
- Don't hate me, but I feel wanting more from this, such as a rationale for why certain features are required in the guest.

The guest can boot without any of these features provided the hypervisor has not enabled any of the SNP_FEATURES_IMPL_REQ features. But in case the hypervisor does enable them, the guest needs to react in a predictable manner.

These are optional security features provided for SNP guests which the hypervisor can enable.

Quoted 3 lines
   When a guest fails to boot and the reasoning provided by a cloud provider 
   is "your guest doesn't support ${feature}", the natural question to ask 
   will be "why do I need ${feature}?"

I think the "why" part depends on the user. Whether or not the user needs a certain feature enabled for the confidential guest.

If the cloud provider(hypervisor) enables the feature on user request, the guest terminates with GHCB_SNP_FEAT_NOT_IMPLEMENTED when guest kernel does have corresponding code/implementation.

Quoted 12 lines
Quoted 12 lines
Documentation/x86/amd-memory-encryption.rst +3 -0
+More details in AMD64 APM[1] Vol 2: 15.34.10 SEV_STATUS MSR
+
+[1] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.amd.com%2Fsystem%2Ffiles%2FTechDocs%2F40332_4.05.pdf&data=05%7C01%7Cnikunj.dadhania%40amd.com%7Cd4d286d55435487f7d4f08daecb3d3e6%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C638082554474351848%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=TLBCkg20WCaF77YrOzGC%2Bn9V3JFgVl2vs1XMPmSWHcM%3D&reserved=0
arch/x86/boot/compressed/sev.c +0 -0
index c93930d5ccbd..43793f46cfc1 100644
@@ -270,6 +270,36 @@ static void enforce_vmpl0(void)
 		sev_es_terminate(SEV_TERM_SET_LINUX, GHCB_TERM_NOT_VMPL0);
 }
 
Quoted 12 lines

arch/x86/boot/compressed/sev.c static void enforce_vmpl0(void) · line 281

Quoted 9 lines
arch/x86/boot/compressed/sev.c +9 -0
+/*
+ * SNP_FEATURES_IMPL_REQ is the mask of SNP features that will need
+ * guest side implementation for proper functioning of the guest. If any
+ * of these features are enabled in the hypervisor but are lacking guest
+ * side implementation, the behavior of the guest will be undefined. The
+ * guest could fail in non-obvious way making it difficult to debug.
+ *
+ * As the behavior of reserved feature bits is unknown to be on the
+ * safe side add them to the required features mask.

If one or more of the reserved feature bits turns out to not be required for the SNP guest to function correctly, is this the correct choice?

Yes, I think so. As they are undefined at the moment, it is safe to assume that a guest implementation is required.

Quoted 5 lines

IIUC, legacy guests would fail to boot correctly (and unnecessarily, because of this change) if they do not have an implentation for a non-required feature. Absent this change, however, they would proceed just fine.

True, but the other way around guest behaviour may be undefined.

Quoted 29 lines
Quoted 29 lines
arch/x86/boot/compressed/sev.c +21 -0
+ */
+#define SNP_FEATURES_IMPL_REQ	(MSR_AMD64_SNP_VTOM |			\
+				MSR_AMD64_SNP_REFLECT_VC |		\
+				MSR_AMD64_SNP_RESTRICTED_INJ |		\
+				MSR_AMD64_SNP_ALT_INJ |			\
+				MSR_AMD64_SNP_DEBUG_SWAP |		\
+				MSR_AMD64_SNP_VMPL_SSS |		\
+				MSR_AMD64_SNP_SECURE_TSC |		\
+				MSR_AMD64_SNP_VMGEXIT_PARAM |		\
+				MSR_AMD64_SNP_VMSA_REG_PROTECTION |	\
+				MSR_AMD64_SNP_RESERVED_BIT13 |		\
+				MSR_AMD64_SNP_RESERVED_BIT15 |		\
+				MSR_AMD64_SNP_RESERVED_MASK)
+
+/*
+ * SNP_FEATURES_PRESENT is the mask of SNP features that are implemented
+ * by the guest kernel. As and when a new feature is implemented in the
+ * guest kernel, a corresponding bit should be added to the mask.
+ */
+#define SNP_FEATURES_PRESENT (0)
+
 void sev_enable(struct boot_params *bp)
 {
 	unsigned int eax, ebx, ecx, edx;
@@ -335,6 +365,13 @@ void sev_enable(struct boot_params *bp)
 		if (!(get_hv_features() & GHCB_HV_FT_SNP))
 			sev_es_terminate(SEV_TERM_SET_GEN, GHCB_SNP_UNSUPPORTED);
 
Quoted 8 lines

arch/x86/boot/compressed/sev.c void sev_enable(struct boot_params *bp) · line 373

Quoted 6 lines
arch/x86/boot/compressed/sev.c +6 -0
+		/*
+		 * Terminate the boot if hypervisor has enabled any feature
+		 * lacking guest side implementation.
+		 */
+		if (sev_status & SNP_FEATURES_IMPL_REQ & ~SNP_FEATURES_PRESENT)
+			sev_es_terminate(SEV_TERM_SET_GEN, GHCB_SNP_FEAT_NOT_IMPLEMENTED);

We can't help out by specifying which feature(s)?

The purpose of SNP_FEATURES_PRESENT is just that, at present no features that need guest implementation is part of the kernel. For e.g. I will be posting patches with SecureTSC enabled, that will make the following change.

arch/x86/boot/compressed/sev.c +1 -1
index 43793f46cfc1..cd0948c6db50 100644
@@ -298,7 +298,7 @@ static void enforce_vmpl0(void)
  * by the guest kernel. As and when a new feature is implemented in the
  * guest kernel, a corresponding bit should be added to the mask.
  */
-#define SNP_FEATURES_PRESENT (0)
+#define SNP_FEATURES_PRESENT (MSR_AMD64_SNP_SECURE_TSC)
 
 void sev_enable(struct boot_params *bp)
 {

Regards,
Nikunj

maintainer
#3 lore raw
Quoted 6 lines · Nikunj A. Dadhania

I think the "why" part depends on the user. Whether or not the user needs a certain feature enabled for the confidential guest.

If the cloud provider(hypervisor) enables the feature on user request, the guest terminates with GHCB_SNP_FEAT_NOT_IMPLEMENTED when guest kernel does have corresponding code/implementation.

I think you mean "does not have" here.

In any case, I think this whole handling of SEV features could go both ways:

* Cloud provider could say: we've enabled features X, Y and Z and if the guest doesn't have support for them, then it would fail booting.

There would optimally be some text sowewhere in the cloud provider documentation stating why those features are enabled and thus required to be supported by the guest.

* Guest owner could require a minimal subset of features which must be present in the HV in order to even boot on that HV.

Of course, I'm only speculating here. How it ends up really playing out in reality we will have to see...

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
#4 lore raw
Quoted 9 lines · Borislav Petkov
Quoted 6 lines · Nikunj A. Dadhania

I think the "why" part depends on the user. Whether or not the user needs a certain feature enabled for the confidential guest.

If the cloud provider(hypervisor) enables the feature on user request, the guest terminates with GHCB_SNP_FEAT_NOT_IMPLEMENTED when guest kernel does have corresponding code/implementation.

I think you mean "does not have" here.

Yes, that is correct.

Quoted 16 lines

In any case, I think this whole handling of SEV features could go both ways:

* Cloud provider could say: we've enabled features X, Y and Z and if the guest doesn't have support for them, then it would fail booting.

There would optimally be some text sowewhere in the cloud provider documentation stating why those features are enabled and thus required to be supported by the guest.

* Guest owner could require a minimal subset of features which must be present in the HV in order to even boot on that HV.

Of course, I'm only speculating here. How it ends up really playing out in reality we will have to see...

maintainer
#5 lore raw
In reply to Nikunj A. Dadhania
Quoted 9 lines · Nikunj A. Dadhania
Quoted 8 lines

Review

Quoted 6 lines
(quoted diff) +6 -0
+		/*
+		 * Terminate the boot if hypervisor has enabled any feature
+		 * lacking guest side implementation.
+		 */
+		if (sev_status & SNP_FEATURES_IMPL_REQ & ~SNP_FEATURES_PRESENT)
+			sev_es_terminate(SEV_TERM_SET_GEN, GHCB_SNP_FEAT_NOT_IMPLEMENTED);

We can't help out by specifying which feature(s)?

Quoted 3 lines

The purpose of SNP_FEATURES_PRESENT is just that, at present no features that need guest implementation is part of the kernel. For e.g. I will be posting patches with SecureTSC enabled, that will make the following change.

I think what David means is, can we have sev_es_terminate() say exactly which feature wasn't implemented instead of users having to dig out which one exactly wasn't by trying to find out what their SNP_FEATURES_IMPL_REQ and SNP_FEATURES_PRESENT masks are.

Looking at the GHCB protocol, where GHCB_SNP_FEAT_NOT_IMPLEMENTED reason code goes is GHCBData[23:16] which is not enough... And the VMSA has SEV_FEATURES but that's guest-only.

I guess we need a way to communicate those masks in a more user-friendly way so that it is exactly clear because of which missing feature(s) has the guest terminated.

Hmm.

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
#6 lore raw
Quoted 14 lines · Borislav Petkov
Quoted 9 lines · Nikunj A. Dadhania
Quoted 8 lines

Review

Quoted 6 lines
(quoted diff) +6 -0
+		/*
+		 * Terminate the boot if hypervisor has enabled any feature
+		 * lacking guest side implementation.
+		 */
+		if (sev_status & SNP_FEATURES_IMPL_REQ & ~SNP_FEATURES_PRESENT)
+			sev_es_terminate(SEV_TERM_SET_GEN, GHCB_SNP_FEAT_NOT_IMPLEMENTED);

We can't help out by specifying which feature(s)?

Quoted 3 lines

The purpose of SNP_FEATURES_PRESENT is just that, at present no features that need guest implementation is part of the kernel. For e.g. I will be posting patches with SecureTSC enabled, that will make the following change.

Quoted 8 lines

I think what David means is, can we have sev_es_terminate() say exactly which feature wasn't implemented instead of users having to dig out which one exactly wasn't by trying to find out what their SNP_FEATURES_IMPL_REQ and SNP_FEATURES_PRESENT masks are.

Looking at the GHCB protocol, where GHCB_SNP_FEAT_NOT_IMPLEMENTED reason code goes is GHCBData[23:16] which is not enough... And the VMSA has SEV_FEATURES but that's guest-only.

Currently, GHCBData[24:63] is unused. If we intend to use the bit range(40bits), GHCB spec will need to be updated. And probably would not be enough.

Quoted 3 lines

I guess we need a way to communicate those masks in a more user-friendly way so that it is exactly clear because of which missing feature(s) has the guest terminated.

As the termination request is done using GHCB MSR protocol, exit codes cannot be used.

Regards,
Nikunj

maintainer
#7 lore raw
Quoted 2 lines · Nikunj A. Dadhania

Currently, GHCBData[24:63] is unused. If we intend to use the bit range(40bits), GHCB spec will need to be updated. And probably would not be enough.

My fear too...

Quoted 1 line

As the termination request is done using GHCB MSR protocol, exit codes cannot be used.

We need to figure out some other way of communicating to the guest owner because of which feature the guest refused booting.

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
#8 lore raw
Quoted 11 lines · Borislav Petkov
Quoted 2 lines · Nikunj A. Dadhania

Currently, GHCBData[24:63] is unused. If we intend to use the bit range(40bits), GHCB spec will need to be updated. And probably would not be enough.

My fear too...

Quoted 1 line

As the termination request is done using GHCB MSR protocol, exit codes cannot be used.

We need to figure out some other way of communicating to the guest owner because of which feature the guest refused booting.

Yeah, this was exactly the thought. If the guest terminates because SNP_FEAT_NOT_IMPLEMENTED, it seems necessary to provide information on which feature(s) the guest is lacking an implementation for that the hypervisor has enabled. Otherwise this becomes a frustrating trial and error for the customer.

maintainer
#9 lore raw
In reply to Nikunj A Dadhania
Quoted 11 lines · Nikunj A Dadhania

The hypervisor can enable various new features (SEV_FEATURES[1:63]) and start the SNP guest. Some of these features need guest side implementation. If any of these features are enabled without guest side implementation, the behavior of the SNP guest will be undefined. The SNP guest boot may fail in a non-obvious way making it difficult to debug.

Instead of allowing the guest to continue and have it fail randomly later, detect this early and fail gracefully.

SEV_STATUS MSR indicates features which hypervisor has enabled. While

					 ^
					 the
Quoted 15 lines

booting, SNP guests should ascertain that all the enabled features have guest side implementation. In case any feature is not implemented in the guest, the guest terminates booting with SNP feature unsupported exit code.

More details in AMD64 APM[1] Vol 2: 15.34.10 SEV_STATUS MSR

[1] https://www.amd.com/system/files/TechDocs/40332_4.05.pdf

Fixes cbd3d4f7c4e5 ("x86/sev: Check SEV-SNP features support")
CC Borislav Petkov <[email protected]>
CC Michael Roth <[email protected]>
CC Tom Lendacky <[email protected]>

...

Documentation/x86/amd-memory-encryption.rst by supplying mem_encrypt=on on the kernel command line. However, if BIOS does · line 99

Quoted 10 lines
Documentation/x86/amd-memory-encryption.rst +2 -0
index a1940ebe7be5..b8b6b87be995 100644
@@ -95,3 +95,38 @@ by supplying mem_encrypt=on on the kernel command line.  However, if BIOS does
 not enable SME, then Linux will not be able to activate memory encryption, even
 if configured to do so by default or the mem_encrypt=on command line parameter
 is specified.
+
+Secure Nested Paging (SNP):

No ":"

Documentation/x86/amd-memory-encryption.rst by supplying mem_encrypt=on on the kernel command line. However, if BIOS does · line 100

Quoted 1 line
Documentation/x86/amd-memory-encryption.rst +1 -0
+===========================

<---- newline here.

Quoted 18 lines
Documentation/x86/amd-memory-encryption.rst +18 -0
+SEV-SNP introduces new features (SEV_FEATURES[1:63]) which can be enabled
+by the hypervisor for security enhancements. Some of these features need
+guest side implementation to function correctly. The below table lists the
+expected guest behavior with various possible scenarios of guest/hypervisor
+SNP feature support.
+
++---------------+---------------+---------------+---------------+
+|Feature Enabled|  Guest needs  |   Guest has   |  Guest boot   |
+|     by HV     |implementation |implementation |   behavior    |
++---------------+---------------+---------------+---------------+
+|      No       |      No       |      No       |     Boot      |
+|               |               |               |               |
++---------------+---------------+---------------+---------------+
+|      No       |      Yes      |      No       |     Boot      |
+|               |               |               |               |
++---------------+---------------+---------------+---------------+
+|      No       |      Yes      |      Yes      |     Boot      |
+|               |               |               |               |

Documentation/x86/amd-memory-encryption.rst by supplying mem_encrypt=on on the kernel command line. However, if BIOS does · line 128

Quoted 10 lines
Documentation/x86/amd-memory-encryption.rst +10 -0
++---------------+---------------+---------------+---------------+
+|      Yes      |      No       |      No       |   Boot with   |
+|               |               |               |feature enabled|
++---------------+---------------+---------------+---------------+
+|      Yes      |      Yes      |      No       | Graceful Boot |
+|               |               |               |    Failure    |
++---------------+---------------+---------------+---------------+
+|      Yes      |      Yes      |      Yes      |   Boot with   |
+|               |               |               |feature enabled|
++---------------+---------------+---------------+---------------+

sphinx is not happy about that table for some reason. I always find the error messages cryptic though:

Documentation/x86/amd-memory-encryption.rst:110: WARNING: Block quote ends without a blank line; unexpected unindent. Documentation/x86/amd-memory-encryption.rst:110: WARNING: Block quote ends without a blank line; unexpected unindent. Documentation/x86/amd-memory-encryption.rst:122: WARNING: Block quote ends without a blank line; unexpected unindent. Documentation/x86/amd-memory-encryption.rst:128: WARNING: Block quote ends without a blank line; unexpected unindent.

You can repro by doing "make htmldocs".

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
#10 lore raw
Quoted 8 lines · Borislav Petkov
Quoted 7 lines · Nikunj A Dadhania

The hypervisor can enable various new features (SEV_FEATURES[1:63]) and start the SNP guest. Some of these features need guest side implementation. If any of these features are enabled without guest side implementation, the behavior of the SNP guest will be undefined. The SNP guest boot may fail in a non-obvious way making it difficult to debug.

Quoted 6 lines
Quoted 4 lines

Instead of allowing the guest to continue and have it fail randomly later, detect this early and fail gracefully.

SEV_STATUS MSR indicates features which hypervisor has enabled. While

					 ^
					 the

Sure.

Quoted 19 lines
Quoted 16 lines

booting, SNP guests should ascertain that all the enabled features have guest side implementation. In case any feature is not implemented in the guest, the guest terminates booting with SNP feature unsupported exit code.

More details in AMD64 APM[1] Vol 2: 15.34.10 SEV_STATUS MSR

[1] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.amd.com%2Fsystem%2Ffiles%2FTechDocs%2F40332_4.05.pdf&data=05%7C01%7Cnikunj.dadhania%40amd.com%7C6575db7c0d8f4f136d1f08daed902274%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C638083500715058552%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=weTrXWfPxDBdu9OsFZ4FxlvlgbhhG%2F985%2Bii%2BM8vh6I%3D&reserved=0

Fixes cbd3d4f7c4e5 ("x86/sev: Check SEV-SNP features support")
CC Borislav Petkov <[email protected]>
CC Michael Roth <[email protected]>
CC Tom Lendacky <[email protected]>

...

Quoted 13 lines

Documentation/x86/amd-memory-encryption.rst by supplying mem_encrypt=on on the kernel command line. However, if BIOS does · line 99

Quoted 10 lines
Documentation/x86/amd-memory-encryption.rst +2 -0
index a1940ebe7be5..b8b6b87be995 100644
@@ -95,3 +95,38 @@ by supplying mem_encrypt=on on the kernel command line.  However, if BIOS does
 not enable SME, then Linux will not be able to activate memory encryption, even
 if configured to do so by default or the mem_encrypt=on command line parameter
 is specified.
+
+Secure Nested Paging (SNP):

No ":"

Done

Quoted 3 lines

Documentation/x86/amd-memory-encryption.rst by supplying mem_encrypt=on on the kernel command line. However, if BIOS does · line 100

Quoted 1 line
Documentation/x86/amd-memory-encryption.rst +1 -0
+===========================

<---- newline here.

Done

Quoted 24 lines
Quoted 24 lines
Documentation/x86/amd-memory-encryption.rst +23 -0
+SEV-SNP introduces new features (SEV_FEATURES[1:63]) which can be enabled
+by the hypervisor for security enhancements. Some of these features need
+guest side implementation to function correctly. The below table lists the
+expected guest behavior with various possible scenarios of guest/hypervisor
+SNP feature support.
+
++---------------+---------------+---------------+---------------+
+|Feature Enabled|  Guest needs  |   Guest has   |  Guest boot   |
+|     by HV     |implementation |implementation |   behavior    |
++---------------+---------------+---------------+---------------+
+|      No       |      No       |      No       |     Boot      |
+|               |               |               |               |
++---------------+---------------+---------------+---------------+
+|      No       |      Yes      |      No       |     Boot      |
+|               |               |               |               |
++---------------+---------------+---------------+---------------+
+|      No       |      Yes      |      Yes      |     Boot      |
+|               |               |               |               |
++---------------+---------------+---------------+---------------+
+|      Yes      |      No       |      No       |   Boot with   |
+|               |               |               |feature enabled|
++---------------+---------------+---------------+---------------+
+|      Yes      |      Yes      |      No       | Graceful Boot |
Quoted 8 lines

Documentation/x86/amd-memory-encryption.rst by supplying mem_encrypt=on on the kernel command line. However, if BIOS does · line 128

Quoted 5 lines
Documentation/x86/amd-memory-encryption.rst +5 -0
+|               |               |               |    Failure    |
++---------------+---------------+---------------+---------------+
+|      Yes      |      Yes      |      Yes      |   Boot with   |
+|               |               |               |feature enabled|
++---------------+---------------+---------------+---------------+

sphinx is not happy about that table for some reason. I always find the error messages cryptic though:

sphinx uses spaces before multi-line text as block quote. Also, found that after the table header it needs line with "=" and not "-".

Quoted 6 lines

Documentation/x86/amd-memory-encryption.rst:110: WARNING: Block quote ends without a blank line; unexpected unindent. Documentation/x86/amd-memory-encryption.rst:110: WARNING: Block quote ends without a blank line; unexpected unindent. Documentation/x86/amd-memory-encryption.rst:122: WARNING: Block quote ends without a blank line; unexpected unindent. Documentation/x86/amd-memory-encryption.rst:128: WARNING: Block quote ends without a blank line; unexpected unindent.

You can repro by doing "make htmldocs".

Fixed,

+-----------------+---------------+---------------+------------------+
| Feature Enabled | Guest needs   | Guest has     | Guest boot       |
| by the HV       | implementation| implementation| behaviour        |
+=================+===============+===============+==================+
|      No         |      No       |      No       |     Boot         |
|                 |               |               |                  |
+-----------------+---------------+---------------+------------------+
|      No         |      Yes      |      No       |     Boot         |
|                 |               |               |                  |
+-----------------+---------------+---------------+------------------+
|      No         |      Yes      |      Yes      |     Boot         |
|                 |               |               |                  |
+-----------------+---------------+---------------+------------------+
|      Yes        |      No       |      No       | Boot with        |
|                 |               |               | feature enabled  |
+-----------------+---------------+---------------+------------------+
|      Yes        |      Yes      |      No       | Graceful boot    |
|                 |               |               | failure          |
+-----------------+---------------+---------------+------------------+
|      Yes        |      Yes      |      Yes      | Boot with        |
|                 |               |               | feature enabled  |
+-----------------+---------------+---------------+------------------+

Regards
Nikunj