Skip to contentSkip to conversation

Add AMD Secure Nested Paging (SEV-SNP)

PATCH In review Ashish Kalra 22 participants 306 messages Jun 20, 2022 – Jan 5, 2023 Page 1 of 3

Subsystems (12)
lore
Share & tools
lore

Ashish Kalra

author maintainer
#0 lore raw

v6 cover Add AMD Secure Nested Paging (SEV-SNP)

From: Ashish Kalra <[email protected]>

This part of the Secure Encrypted Paging (SEV-SNP) series focuses on the changes required in a host OS for SEV-SNP support. The series builds upon SEV-SNP Guest Support now part of mainline.

This series provides the basic building blocks to support booting the SEV-SNP VMs, it does not cover all the security enhancement introduced by the SEV-SNP such as interrupt protection.

The CCP driver is enhanced to provide new APIs that use the SEV-SNP specific commands defined in the SEV-SNP firmware specification. The KVM driver uses those APIs to create and managed the SEV-SNP guests.

The GHCB specification version 2 introduces new set of NAE's that is used by the SEV-SNP guest to communicate with the hypervisor. The series provides support to handle the following new NAE events:
- Register GHCB GPA
- Page State Change Request
- Hypevisor feature
- Guest message request

The RMP check is enforced as soon as SEV-SNP is enabled. Not every memory access requires an RMP check. In particular, the read accesses from the hypervisor do not require RMP checks because the data confidentiality is already protected via memory encryption. When hardware encounters an RMP checks failure, it raises a page-fault exception. If RMP check failure is due to the page-size mismatch, then split the large page to resolve the fault.

The series does not provide support for the interrupt security and migration and those feature will be added after the base support.

Please note that some areas, such as how private guest pages are managed/pinned/protected, are likely to change once Unmapped Private Memory support is further along in development/design and can be incorporated into this series. We are posting these patches without UPM support for now to hopefully get some review on other aspects of the series in the meantime.

Here is a link to latest UPM v6 patches:
https://lore.kernel.org/linux-mm/[email protected]/

A branch containing these patches is available here:
https://github.com/AMDESE/linux/tree/sev-snp-5.18-rc3-v3

Changes since v5:
* Rebase to 5.18.0-rc3, these patches are just for review so they are based on 5.18.0-rc3 linux-next release as this included the SNP guest patches which weren't in mainline then.
* Using kvm_write_guest() to sync the GHCB scratch buffer can fail due to host mapping being 2M, but RMP being 4K. The page fault handling in do_user_addr_fault() fails to split the 2M page to handle RMP fault due it being called in a non-preemptible context. Instead, use the already kernel mapped ghcb to sync the scratch buffer when the scratch buffer is contained within the GHCB.
* warn and retry failed rmpupdates.
* Fix for stale per-cpu pointer due to cond_resched due during ghcb mapping.
* Multiple fixes for SEV-SNP AP Creation.
* Remove SRCU to synchronize the PSC and gfn mapping replacing it with a spinlock.
* Remove generic post_{map,unmap}_gfn ops, need to revisit these later with respect to UPM support.
* Fix kvm_mmu_get_tdp_walk() to handle "suspicious RCU usage" warning.
* Fix sev_snp_init() to do WBINVD/DF_FLUSH command after SNP_INIT command has been issued.
* Fix sev_free_vcpu() to flush the VMSA page after it is transitioned back to hypervisor state and restored in the kernel direct map.

Changes since v4:
* Move the RMP entry definition to x86 specific header file.
* Move the dump RMP entry function to SEV specific file.
* Use BIT_ULL while defining the #PF bit fields.
* Add helper function to check the IOMMU support for SEV-SNP feature.
* Add helper functions for the page state transition.
* Map and unmap the pages from the direct map after page is added or removed in RMP table.
* Enforce the minimum SEV-SNP firmware version.
* Extend the LAUNCH_UPDATE to accept the base_gfn and remove the logic to calculate the gfn from the hva.
* Add a check in LAUNCH_UPDATE to ensure that all the pages are shared before calling the PSP.
* Mark the memory failure when failing to remove the page from the RMP table or clearing the immutable bit.
* Exclude the encrypted hva range from the KSM.
* Remove the gfn tracking during the kvm_gfn_map() and use SRCU to syncronize the PSC and gfn mapping.
* Allow PSC on the registered hva range only.
* Add support for the Preferred GPA VMGEXIT.
* Simplify the PSC handling routines.
* Use the static_call() for the newly added kvm_x86_ops.
* Remove the long-lived GHCB map.
* Move the snp enable module parameter to the end of the file.
* Remove the kvm_x86_op for the RMP fault handling. Call the fault handler directly from the #NPF interception.

Changes since v3:
* Add support for extended guest message request.
* Add ioctl to query the SNP Platform status.
* Add ioctl to get and set the SNP config.
* Add check to verify that memory reserved for the RMP covers the full system RAM.
* Start the SNP specific commands from 256 instead of 255.
* Multiple cleanup and fixes based on the review feedback.

Changes since v2:
* Add AP creation support.
* Drop the patch to handle the RMP fault for the kernel address.
* Add functions to track the write access from the hypervisor.
* Do not enable the SNP feature when IOMMU is disabled or is in passthrough mode.
* Dump the RMP entry on RMP violation for the debug.
* Shorten the GHCB macro names.
* Start the SNP_INIT command id from 255 to give some gap for the legacy SEV.
* Sync the header with the latest 0.9 SNP spec.

Changes since v1:
* Add AP reset MSR protocol VMGEXIT NAE.
* Add Hypervisor features VMGEXIT NAE.
* Move the RMP table initialization and RMPUPDATE/PSMASH helper in arch/x86/kernel/sev.c.
* Add support to map/unmap SEV legacy command buffer to firmware state when SNP is active.
* Enhance PSP driver to provide helper to allocate/free memory used for the firmware context page.
* Add support to handle RMP fault for the kernel address.
* Add support to handle GUEST_REQUEST NAE event for attestation.
* Rename RMP table lookup helper.
* Drop typedef from rmpentry struct definition.
* Drop SNP static key and use cpu_feature_enabled() to check whether SEV-SNP is active.
* Multiple cleanup/fixes to address Boris review feedback.

Ashish Kalra (1):
KVM: SVM: Sync the GHCB scratch buffer using already mapped ghcb

Brijesh Singh (42):
x86/cpufeatures: Add SEV-SNP CPU feature
iommu/amd: Introduce function to check SEV-SNP support x86/sev: Add the host SEV-SNP initialization support x86/sev: set SYSCFG.MFMD
x86/sev: Add RMP entry lookup helpers
x86/sev: Add helper functions for RMPUPDATE and PSMASH instruction x86/sev: Invalid pages from direct map when adding it to RMP table x86/traps: Define RMP violation #PF error code
x86/fault: Add support to handle the RMP fault for user address x86/fault: Add support to dump RMP entry on fault crypto:ccp: Define the SEV-SNP commands
crypto: ccp: Add support to initialize the AMD-SP for SEV-SNP crypto:ccp: Provide APIs to issue SEV-SNP commands
crypto: ccp: Handle the legacy TMR allocation when SNP is enabled
crypto: ccp: Handle the legacy SEV command when SNP is enabled
crypto: ccp: Add the SNP_PLATFORM_STATUS command
crypto: ccp: Add the SNP_{SET,GET}_EXT_CONFIG command
crypto: ccp: Provide APIs to query extended attestation report
KVM: SVM: Provide the Hypervisor Feature support VMGEXIT
KVM: SVM: Make AVIC backing, VMSA and VMCB memory allocation SNP safe
KVM: SVM: Add initial SEV-SNP support
KVM: SVM: Add KVM_SNP_INIT command
KVM: SVM: Add KVM_SEV_SNP_LAUNCH_START command
KVM: SVM: Disallow registering memory range from HugeTLB for SNP guest
KVM: SVM: Add KVM_SEV_SNP_LAUNCH_UPDATE command
KVM: SVM: Mark the private vma unmerable for SEV-SNP guests
KVM: SVM: Add KVM_SEV_SNP_LAUNCH_FINISH command
KVM: X86: Keep the NPT and RMP page level in sync
KVM: x86: Introduce kvm_mmu_get_tdp_walk() for SEV-SNP use
KVM: x86: Define RMP page fault error bits for #NPF
KVM: x86: Update page-fault trace to log full 64-bit error code
KVM: SVM: Do not use long-lived GHCB map while setting scratch area
KVM: SVM: Remove the long-lived GHCB host map
KVM: SVM: Add support to handle GHCB GPA register VMGEXIT
KVM: SVM: Add support to handle MSR based Page State Change VMGEXIT
KVM: SVM: Add support to handle Page State Change VMGEXIT
KVM: SVM: Introduce ops for the post gfn map and unmap
KVM: x86: Export the kvm_zap_gfn_range() for the SNP use
KVM: SVM: Add support to handle the RMP nested page fault
KVM: SVM: Provide support for SNP_GUEST_REQUEST NAE event
KVM: SVM: Add module parameter to enable the SEV-SNP
ccp: add support to decrypt the page

Michael Roth (2):
*fix for stale per-cpu pointer due to cond_resched during ghcb
mapping
*debug: warn and retry failed rmpupdates

Sean Christopherson (1):
KVM: x86/mmu: Introduce kvm_mmu_map_tdp_page() for use by TDX and SNP

Tom Lendacky (3):
KVM: SVM: Add support to handle AP reset MSR protocol
KVM: SVM: Use a VMSA physical address variable for populating VMCB
KVM: SVM: Support SEV-SNP AP Creation NAE event

 Documentation/virt/coco/sevguest.rst          |   54 +
 .../virt/kvm/x86/amd-memory-encryption.rst    |  102 +
 arch/x86/include/asm/cpufeatures.h            |    1 +
 arch/x86/include/asm/disabled-features.h      |    8 +-
 arch/x86/include/asm/kvm-x86-ops.h            |    2 +
 arch/x86/include/asm/kvm_host.h               |   15 +
 arch/x86/include/asm/msr-index.h              |    9 +
 arch/x86/include/asm/sev-common.h             |   28 +
 arch/x86/include/asm/sev.h                    |   45 +
 arch/x86/include/asm/svm.h                    |    6 +
 arch/x86/include/asm/trap_pf.h                |   18 +-
 arch/x86/kernel/cpu/amd.c                     |    3 +-
 arch/x86/kernel/sev.c                         |  400 ++++
 arch/x86/kvm/lapic.c                          |    5 +-
 arch/x86/kvm/mmu.h                            |    7 +-
 arch/x86/kvm/mmu/mmu.c                        |   90 +
 arch/x86/kvm/svm/sev.c                        | 1703 ++++++++++++++++-
 arch/x86/kvm/svm/svm.c                        |   62 +-
 arch/x86/kvm/svm/svm.h                        |   75 +-
 arch/x86/kvm/trace.h                          |   40 +-
 arch/x86/kvm/x86.c                            |   10 +-
 arch/x86/mm/fault.c                           |   84 +-
 drivers/crypto/ccp/sev-dev.c                  |  908 ++++++++-
 drivers/crypto/ccp/sev-dev.h                  |   17 +
 drivers/iommu/amd/init.c                      |   30 +
 include/linux/iommu.h                         |    9 +
 include/linux/mm.h                            |    3 +-
 include/linux/mm_types.h                      |    3 +
 include/linux/psp-sev.h                       |  346 ++++
 include/linux/sev.h                           |   32 +
 include/uapi/linux/kvm.h                      |   56 +
 include/uapi/linux/psp-sev.h                  |   60 +
 mm/memory.c                                   |   13 +
 tools/arch/x86/include/asm/cpufeatures.h      |    1 +
 34 files changed, 4090 insertions(+), 155 deletions(-)
 create mode 100644 include/linux/sev.h
Signature · 1 line
2.25.1
author maintainer
#1 lore raw

PATCH v6 1/49 x86/cpufeatures: Add SEV-SNP CPU feature

From: Brijesh Singh <[email protected]>

Add CPU feature detection for Secure Encrypted Virtualization with Secure Nested Paging. This feature adds a strong memory integrity protection to help prevent malicious hypervisor-based attacks like data replay, memory re-mapping, and more.

---
 arch/x86/include/asm/cpufeatures.h       | 1 +
 arch/x86/kernel/cpu/amd.c                | 3 ++-
 tools/arch/x86/include/asm/cpufeatures.h | 1 +
 3 files changed, 4 insertions(+), 1 deletion(-)
arch/x86/include/asm/cpufeatures.h +1 -0
index 30da1341f226..1cba0217669f 100644
@@ -407,6 +407,7 @@
 #define X86_FEATURE_SEV			(19*32+ 1) /* AMD Secure Encrypted Virtualization */
 #define X86_FEATURE_VM_PAGE_FLUSH	(19*32+ 2) /* "" VM Page Flush MSR is supported */
 #define X86_FEATURE_SEV_ES		(19*32+ 3) /* AMD Secure Encrypted Virtualization - Encrypted State */
+#define X86_FEATURE_SEV_SNP		(19*32+4)  /* AMD Secure Encrypted Virtualization - Secure Nested Paging */
 #define X86_FEATURE_SME_COHERENT	(19*32+10) /* "" AMD hardware-enforced cache coherency */
 
 /*
arch/x86/kernel/cpu/amd.c +2 -1
index 0c0b09796ced..2e87015a9d69 100644
@@ -559,7 +559,7 @@ static void early_detect_mem_encrypt(struct cpuinfo_x86 *c)
 	 *	      If the kernel has not enabled SME via any means then
 	 *	      don't advertise the SME feature.
 	 *   For SEV: If BIOS has not enabled SEV then don't advertise the
-	 *            SEV and SEV_ES feature (set in scattered.c).
+	 *            SEV, SEV_ES and SEV_SNP feature.
 	 *
 	 *   In all cases, since support for SME and SEV requires long mode,
 	 *   don't advertise the feature under CONFIG_X86_32.
@@ -594,6 +594,7 @@ static void early_detect_mem_encrypt(struct cpuinfo_x86 *c)
 clear_sev:
 		setup_clear_cpu_cap(X86_FEATURE_SEV);
 		setup_clear_cpu_cap(X86_FEATURE_SEV_ES);
+		setup_clear_cpu_cap(X86_FEATURE_SEV_SNP);
 	}
 }
 
tools/arch/x86/include/asm/cpufeatures.h +1 -0
index 73e643ae94b6..a636342ecb26 100644
@@ -405,6 +405,7 @@
 #define X86_FEATURE_SEV			(19*32+ 1) /* AMD Secure Encrypted Virtualization */
 #define X86_FEATURE_VM_PAGE_FLUSH	(19*32+ 2) /* "" VM Page Flush MSR is supported */
 #define X86_FEATURE_SEV_ES		(19*32+ 3) /* AMD Secure Encrypted Virtualization - Encrypted State */
+#define X86_FEATURE_SEV_SNP		(19*32+4)  /* AMD Secure Encrypted Virtualization - Secure Nested Paging */
 #define X86_FEATURE_SME_COHERENT	(19*32+10) /* "" AMD hardware-enforced cache coherency */
 
 /*
Signature · 1 line
2.25.1
author maintainer
#3 lore raw

PATCH v6 2/49 iommu/amd: Introduce function to check SEV-SNP support

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

The SEV-SNP support requires that IOMMU must to enabled, see the IOMMU spec section 2.12 for further details. If IOMMU is not enabled or the SNPSup extended feature register is not set then the SNP_INIT command (used for initializing firmware) will fail.

The iommu_sev_snp_supported() can be used to check if IOMMU supports the SEV-SNP feature.

---
 drivers/iommu/amd/init.c | 30 ++++++++++++++++++++++++++++++
 include/linux/iommu.h    |  9 +++++++++
 2 files changed, 39 insertions(+)
drivers/iommu/amd/init.c +30 -0
index 1a3ad58ba846..82be8067ddf5 100644
@@ -3361,3 +3361,33 @@ int amd_iommu_pc_set_reg(struct amd_iommu *iommu, u8 bank, u8 cntr, u8 fxn, u64
 
 	return iommu_pc_get_set_reg(iommu, bank, cntr, fxn, value, true);
 }
+
+bool iommu_sev_snp_supported(void)
+{
+	struct amd_iommu *iommu;
+
+	/*
+	 * The SEV-SNP support requires that IOMMU must be enabled, and is
+	 * not configured in the passthrough mode.
+	 */
+	if (no_iommu || iommu_default_passthrough()) {
+		pr_err("SEV-SNP: IOMMU is either disabled or configured in passthrough mode.\n");
+		return false;
+	}
+
+	/*
+	 * Iterate through all the IOMMUs and verify the SNPSup feature is
+	 * enabled.
+	 */
+	for_each_iommu(iommu) {
+		if (!iommu_feature(iommu, FEATURE_SNP)) {
+			pr_err("SNPSup is disabled (devid: %02x:%02x.%x)\n",
+			       PCI_BUS_NUM(iommu->devid), PCI_SLOT(iommu->devid),
+			       PCI_FUNC(iommu->devid));
+			return false;
+		}
+	}
+
+	return true;
+}
+EXPORT_SYMBOL_GPL(iommu_sev_snp_supported);
include/linux/iommu.h +9 -0
index 9208eca4b0d1..fecb72e1b11b 100644
@@ -675,6 +675,12 @@ struct iommu_sva *iommu_sva_bind_device(struct device *dev,
 void iommu_sva_unbind_device(struct iommu_sva *handle);
 u32 iommu_sva_get_pasid(struct iommu_sva *handle);
 
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+bool iommu_sev_snp_supported(void);
+#else
+static inline bool iommu_sev_snp_supported(void) { return false; }
+#endif
+
 #else /* CONFIG_IOMMU_API */
 
 struct iommu_ops {};
@@ -1031,6 +1037,9 @@ static inline struct iommu_fwspec *dev_iommu_fwspec_get(struct device *dev)
 {
 	return NULL;
 }
+
+static inline bool iommu_sev_snp_supported(void) { return false; }
+
 #endif /* CONFIG_IOMMU_API */
 
 /**
Signature · 1 line
2.25.1
author maintainer
#16 lore raw

PATCH v6 3/49 x86/sev: Add the host SEV-SNP initialization support

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

The memory integrity guarantees of SEV-SNP are enforced through a new structure called the Reverse Map Table (RMP). The RMP is a single data structure shared across the system that contains one entry for every 4K page of DRAM that may be used by SEV-SNP VMs. The goal of RMP is to track the owner of each page of memory. Pages of memory can be owned by the hypervisor, owned by a specific VM or owned by the AMD-SP. See APM2 section 15.36.3 for more detail on RMP.

The RMP table is used to enforce access control to memory. The table itself is not directly writable by the software. New CPU instructions (RMPUPDATE, PVALIDATE, RMPADJUST) are used to manipulate the RMP entries.

Based on the platform configuration, the BIOS reserves the memory used for the RMP table. The start and end address of the RMP table must be queried by reading the RMP_BASE and RMP_END MSRs. If the RMP_BASE and RMP_END are not set then disable the SEV-SNP feature.

The SEV-SNP feature is enabled only after the RMP table is successfully initialized.

---
 arch/x86/include/asm/disabled-features.h |   8 +-
 arch/x86/include/asm/msr-index.h         |   6 +
 arch/x86/kernel/sev.c                    | 144 +++++++++++++++++++++++
 3 files changed, 157 insertions(+), 1 deletion(-)
arch/x86/include/asm/disabled-features.h +7 -1
index 36369e76cc63..c1be3091a383 100644
@@ -68,6 +68,12 @@
 # define DISABLE_TDX_GUEST	(1 << (X86_FEATURE_TDX_GUEST & 31))
 #endif
 
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+# define DISABLE_SEV_SNP	0
+#else
+# define DISABLE_SEV_SNP	(1 << (X86_FEATURE_SEV_SNP & 31))
+#endif
+
 /*
  * Make sure to add features to the correct mask
  */
@@ -91,7 +97,7 @@
 			 DISABLE_ENQCMD)
 #define DISABLED_MASK17	0
 #define DISABLED_MASK18	0
-#define DISABLED_MASK19	0
+#define DISABLED_MASK19	(DISABLE_SEV_SNP)
 #define DISABLED_MASK_CHECK BUILD_BUG_ON_ZERO(NCAPINTS != 20)
 
 #endif /* _ASM_X86_DISABLED_FEATURES_H */
arch/x86/include/asm/msr-index.h +6 -0
index 9e2e7185fc1d..57a8280e283a 100644
@@ -507,6 +507,8 @@
 #define MSR_AMD64_SEV_ENABLED		BIT_ULL(MSR_AMD64_SEV_ENABLED_BIT)
 #define MSR_AMD64_SEV_ES_ENABLED	BIT_ULL(MSR_AMD64_SEV_ES_ENABLED_BIT)
 #define MSR_AMD64_SEV_SNP_ENABLED	BIT_ULL(MSR_AMD64_SEV_SNP_ENABLED_BIT)
+#define MSR_AMD64_RMP_BASE		0xc0010132
+#define MSR_AMD64_RMP_END		0xc0010133
 
 #define MSR_AMD64_VIRT_SPEC_CTRL	0xc001011f
 
@@ -581,6 +583,10 @@
 #define MSR_AMD64_SYSCFG		0xc0010010
 #define MSR_AMD64_SYSCFG_MEM_ENCRYPT_BIT	23
 #define MSR_AMD64_SYSCFG_MEM_ENCRYPT	BIT_ULL(MSR_AMD64_SYSCFG_MEM_ENCRYPT_BIT)
+#define MSR_AMD64_SYSCFG_SNP_EN_BIT		24
+#define MSR_AMD64_SYSCFG_SNP_EN		BIT_ULL(MSR_AMD64_SYSCFG_SNP_EN_BIT)
+#define MSR_AMD64_SYSCFG_SNP_VMPL_EN_BIT	25
+#define MSR_AMD64_SYSCFG_SNP_VMPL_EN	BIT_ULL(MSR_AMD64_SYSCFG_SNP_VMPL_EN_BIT)
 #define MSR_K8_INT_PENDING_MSG		0xc0010055
 /* C1E active bits in int pending message */
 #define K8_INTP_C1E_ACTIVE_MASK		0x18000000
arch/x86/kernel/sev.c +144 -0

171 lines not on this page · raw message

Signature · 1 line
2.25.1
author maintainer
#25 lore raw

PATCH v6 4/49 x86/sev: set SYSCFG.MFMD

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

SEV-SNP FW >= 1.51 requires that SYSCFG.MFMD must be set.

Subsequent CCP patches while require 1.51 as the minimum SEV-SNP firmware version.

---
 arch/x86/include/asm/msr-index.h |  3 +++
 arch/x86/kernel/sev.c            | 24 ++++++++++++++++++++++++
 2 files changed, 27 insertions(+)
arch/x86/include/asm/msr-index.h +3 -0
index 57a8280e283a..1e36f16daa56 100644
@@ -587,6 +587,9 @@
 #define MSR_AMD64_SYSCFG_SNP_EN		BIT_ULL(MSR_AMD64_SYSCFG_SNP_EN_BIT)
 #define MSR_AMD64_SYSCFG_SNP_VMPL_EN_BIT	25
 #define MSR_AMD64_SYSCFG_SNP_VMPL_EN	BIT_ULL(MSR_AMD64_SYSCFG_SNP_VMPL_EN_BIT)
+#define MSR_AMD64_SYSCFG_MFDM_BIT		19
+#define MSR_AMD64_SYSCFG_MFDM		BIT_ULL(MSR_AMD64_SYSCFG_MFDM_BIT)
+
 #define MSR_K8_INT_PENDING_MSG		0xc0010055
 /* C1E active bits in int pending message */
 #define K8_INTP_C1E_ACTIVE_MASK		0x18000000
arch/x86/kernel/sev.c +24 -0
index 3a233b5d47c5..25c7feb367f6 100644
@@ -2257,6 +2257,27 @@ static __init void snp_enable(void *arg)
 	__snp_enable(smp_processor_id());
 }
 
+static int __mfdm_enable(unsigned int cpu)
+{
+	u64 val;
+
+	if (!cpu_feature_enabled(X86_FEATURE_SEV_SNP))
+		return 0;
+
+	rdmsrl(MSR_AMD64_SYSCFG, val);
+
+	val |= MSR_AMD64_SYSCFG_MFDM;
+
+	wrmsrl(MSR_AMD64_SYSCFG, val);
+
+	return 0;
+}
+
+static __init void mfdm_enable(void *arg)
+{
+	__mfdm_enable(smp_processor_id());
+}
+
 static bool get_rmptable_info(u64 *start, u64 *len)
 {
 	u64 calc_rmp_sz, rmp_sz, rmp_base, rmp_end, nr_pages;
@@ -2325,6 +2346,9 @@ static __init int __snp_rmptable_init(void)
 	/* Flush the caches to ensure that data is written before SNP is enabled. */
 	wbinvd_on_all_cpus();
 
+	/* MFDM must be enabled on all the CPUs prior to enabling SNP. */
+	on_each_cpu(mfdm_enable, NULL, 1);
+
 	/* Enable SNP on all CPUs. */
 	on_each_cpu(snp_enable, NULL, 1);
 
Signature · 1 line
2.25.1
author maintainer
#29 lore raw

PATCH v6 5/49 x86/sev: Add RMP entry lookup helpers

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

The snp_lookup_page_in_rmptable() can be used by the host to read the RMP entry for a given page. The RMP entry format is documented in AMD PPR, see
https://bugzilla.kernel.org/attachment.cgi?id=296015.

---
 arch/x86/include/asm/sev.h | 27 ++++++++++++++++++++++++
 arch/x86/kernel/sev.c      | 43 ++++++++++++++++++++++++++++++++++++++
 include/linux/sev.h        | 30 ++++++++++++++++++++++++++
 3 files changed, 100 insertions(+)
 create mode 100644 include/linux/sev.h
arch/x86/include/asm/sev.h +27 -0
index 9c2d33f1cfee..cb16f0e5b585 100644
@@ -9,6 +9,7 @@
 #define __ASM_ENCRYPTED_STATE_H
 
 #include <linux/types.h>
+#include <linux/sev.h>
 #include <asm/insn.h>
 #include <asm/sev-common.h>
 #include <asm/bootparam.h>
@@ -84,6 +85,32 @@ extern bool handle_vc_boot_ghcb(struct pt_regs *regs);
 
 /* RMP page size */
 #define RMP_PG_SIZE_4K			0
+#define RMP_TO_X86_PG_LEVEL(level)	(((level) == RMP_PG_SIZE_4K) ? PG_LEVEL_4K : PG_LEVEL_2M)
+
+/*
+ * The RMP entry format is not architectural. The format is defined in PPR
+ * Family 19h Model 01h, Rev B1 processor.
+ */
+struct __packed rmpentry {
+	union {
+		struct {
+			u64	assigned	: 1,
+				pagesize	: 1,
+				immutable	: 1,
+				rsvd1		: 9,
+				gpa		: 39,
+				asid		: 10,
+				vmsa		: 1,
+				validated	: 1,
+				rsvd2		: 1;
+		} info;
+		u64 low;
+	};
+	u64 high;
+};
+
+#define rmpentry_assigned(x)	((x)->info.assigned)
+#define rmpentry_pagesize(x)	((x)->info.pagesize)
 
 #define RMPADJUST_VMSA_PAGE_BIT		BIT(16)
 
arch/x86/kernel/sev.c +43 -0

52 lines not on this page · raw message

include/linux/sev.h new file +30 -0
new file mode 100644
index 000000000000..1a68842789e1
@@ -0,0 +1,30 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * AMD Secure Encrypted Virtualization
+ *
+ * Author: Brijesh Singh <[email protected]>
+ */
+
+#ifndef __LINUX_SEV_H
+#define __LINUX_SEV_H
+
+/* RMUPDATE detected 4K page and 2MB page overlap. */
+#define RMPUPDATE_FAIL_OVERLAP		7
+
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+int snp_lookup_rmpentry(u64 pfn, int *level);
+int psmash(u64 pfn);
+int rmp_make_private(u64 pfn, u64 gpa, enum pg_level level, int asid, bool immutable);
+int rmp_make_shared(u64 pfn, enum pg_level level);
+#else
+static inline int snp_lookup_rmpentry(u64 pfn, int *level) { return 0; }
+static inline int psmash(u64 pfn) { return -ENXIO; }
+static inline int rmp_make_private(u64 pfn, u64 gpa, enum pg_level level, int asid,
+				   bool immutable)
+{
+	return -ENODEV;
+}
+static inline int rmp_make_shared(u64 pfn, enum pg_level level) { return -ENODEV; }
+
+#endif /* CONFIG_AMD_MEM_ENCRYPT */
+#endif /* __LINUX_SEV_H */
Signature · 1 line
2.25.1
author maintainer
#57 lore raw

PATCH v6 6/49 x86/sev: Add helper functions for RMPUPDATE and PSMASH instruction

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

The RMPUPDATE instruction writes a new RMP entry in the RMP Table. The hypervisor will use the instruction to add pages to the RMP table. See APM3 for details on the instruction operations.

The PSMASH instruction expands a 2MB RMP entry into a corresponding set of contiguous 4KB-Page RMP entries. The hypervisor will use this instruction to adjust the RMP entry without invalidating the previous RMP entry.

---
 arch/x86/include/asm/sev.h | 11 ++++++
 arch/x86/kernel/sev.c      | 72 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 83 insertions(+)
arch/x86/include/asm/sev.h +11 -0
index cb16f0e5b585..6ab872311544 100644
@@ -85,7 +85,9 @@ extern bool handle_vc_boot_ghcb(struct pt_regs *regs);
 
 /* RMP page size */
 #define RMP_PG_SIZE_4K			0
+#define RMP_PG_SIZE_2M			1
 #define RMP_TO_X86_PG_LEVEL(level)	(((level) == RMP_PG_SIZE_4K) ? PG_LEVEL_4K : PG_LEVEL_2M)
+#define X86_TO_RMP_PG_LEVEL(level)	(((level) == PG_LEVEL_4K) ? RMP_PG_SIZE_4K : RMP_PG_SIZE_2M)
 
 /*
  * The RMP entry format is not architectural. The format is defined in PPR
@@ -126,6 +128,15 @@ struct snp_guest_platform_data {
 	u64 secrets_gpa;
 };
 
+struct rmpupdate {
+	u64 gpa;
+	u8 assigned;
+	u8 pagesize;
+	u8 immutable;
+	u8 rsvd;
+	u32 asid;
+} __packed;
+
 #ifdef CONFIG_AMD_MEM_ENCRYPT
 extern struct static_key_false sev_es_enable_key;
 extern void __sev_es_ist_enter(struct pt_regs *regs);
arch/x86/kernel/sev.c +72 -0

75 lines not on this page · raw message

Signature · 1 line
2.25.1
author maintainer
#75 lore raw

PATCH v6 7/49 x86/sev: Invalid pages from direct map when adding it to RMP table

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

The integrity guarantee of SEV-SNP is enforced through the RMP table. The RMP is used with standard x86 and IOMMU page tables to enforce memory restrictions and page access rights. The RMP check is enforced as soon as SEV-SNP is enabled globally in the system. When hardware encounters an RMP checks failure, it raises a page-fault exception.

The rmp_make_private() and rmp_make_shared() helpers are used to add or remove the pages from the RMP table. Improve the rmp_make_private() to invalid state so that pages cannot be used in the direct-map after its added in the RMP table, and restore to its default valid permission after the pages are removed from the RMP table.

---
 arch/x86/kernel/sev.c | 61 ++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 60 insertions(+), 1 deletion(-)
arch/x86/kernel/sev.c +60 -1
index f6c64a722e94..734cddd837f5 100644
@@ -2451,10 +2451,42 @@ int psmash(u64 pfn)
 }
 EXPORT_SYMBOL_GPL(psmash);
 
+static int restore_direct_map(u64 pfn, int npages)
+{
+	int i, ret = 0;
+
+	for (i = 0; i < npages; i++) {
+		ret = set_direct_map_default_noflush(pfn_to_page(pfn + i));
+		if (ret)
+			goto cleanup;
+	}
+
+cleanup:
+	WARN(ret > 0, "Failed to restore direct map for pfn 0x%llx\n", pfn + i);
+	return ret;
+}
+
+static int invalid_direct_map(unsigned long pfn, int npages)
+{
+	int i, ret = 0;
+
+	for (i = 0; i < npages; i++) {
+		ret = set_direct_map_invalid_noflush(pfn_to_page(pfn + i));
+		if (ret)
+			goto cleanup;
+	}
+
+	return 0;
+
+cleanup:
+	restore_direct_map(pfn, i);
+	return ret;
+}
+
 static int rmpupdate(u64 pfn, struct rmpupdate *val)
 {
 	unsigned long paddr = pfn << PAGE_SHIFT;
-	int ret;
+	int ret, level, npages;
 
 	if (!pfn_valid(pfn))
 		return -EINVAL;
@@ -2462,11 +2494,38 @@ static int rmpupdate(u64 pfn, struct rmpupdate *val)
 	if (!cpu_feature_enabled(X86_FEATURE_SEV_SNP))
 		return -ENXIO;
 
+	level = RMP_TO_X86_PG_LEVEL(val->pagesize);
+	npages = page_level_size(level) / PAGE_SIZE;
+
+	/*
+	 * If page is getting assigned in the RMP table then unmap it from the
+	 * direct map.
+	 */
+	if (val->assigned) {
+		if (invalid_direct_map(pfn, npages)) {
+			pr_err("Failed to unmap pfn 0x%llx pages %d from direct_map\n",
+			       pfn, npages);
+			return -EFAULT;
+		}
+	}
+
 	/* Binutils version 2.36 supports the RMPUPDATE mnemonic. */
 	asm volatile(".byte 0xF2, 0x0F, 0x01, 0xFE"
 		     : "=a"(ret)
 		     : "a"(paddr), "c"((unsigned long)val)
 		     : "memory", "cc");
+
+	/*
+	 * Restore the direct map after the page is removed from the RMP table.
+	 */
+	if (!ret && !val->assigned) {
+		if (restore_direct_map(pfn, npages)) {
+			pr_err("Failed to map pfn 0x%llx pages %d in direct_map\n",
+			       pfn, npages);
+			return -EFAULT;
+		}
+	}
+
 	return ret;
 }
 
Signature · 1 line
2.25.1
author maintainer
#91 lore raw

PATCH v6 8/49 x86/traps: Define RMP violation #PF error code

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

Bit 31 in the page fault-error bit will be set when processor encounters an RMP violation.

While at it, use the BIT_ULL() macro.

---
 arch/x86/include/asm/trap_pf.h | 18 +++++++++++-------
 arch/x86/mm/fault.c            |  1 +
 2 files changed, 12 insertions(+), 7 deletions(-)
arch/x86/include/asm/trap_pf.h +11 -7
index 10b1de500ab1..89b705114b3f 100644
@@ -2,6 +2,8 @@
 #ifndef _ASM_X86_TRAP_PF_H
 #define _ASM_X86_TRAP_PF_H
 
+#include <linux/bits.h>  /* BIT() macro */
+
 /*
  * Page fault error code bits:
  *
@@ -12,15 +14,17 @@
  *   bit 4 ==				1: fault was an instruction fetch
  *   bit 5 ==				1: protection keys block access
  *   bit 15 ==				1: SGX MMU page-fault
+ *   bit 31 ==				1: fault was due to RMP violation
  */
 enum x86_pf_error_code {
-	X86_PF_PROT	=		1 << 0,
-	X86_PF_WRITE	=		1 << 1,
-	X86_PF_USER	=		1 << 2,
-	X86_PF_RSVD	=		1 << 3,
-	X86_PF_INSTR	=		1 << 4,
-	X86_PF_PK	=		1 << 5,
-	X86_PF_SGX	=		1 << 15,
+	X86_PF_PROT	=		BIT_ULL(0),
+	X86_PF_WRITE	=		BIT_ULL(1),
+	X86_PF_USER	=		BIT_ULL(2),
+	X86_PF_RSVD	=		BIT_ULL(3),
+	X86_PF_INSTR	=		BIT_ULL(4),
+	X86_PF_PK	=		BIT_ULL(5),
+	X86_PF_SGX	=		BIT_ULL(15),
+	X86_PF_RMP	=		BIT_ULL(31),
 };
 
 #endif /* _ASM_X86_TRAP_PF_H */
arch/x86/mm/fault.c +1 -0
index fad8faa29d04..a4c270e99f7f 100644
@@ -546,6 +546,7 @@ show_fault_oops(struct pt_regs *regs, unsigned long error_code, unsigned long ad
 		 !(error_code & X86_PF_PROT) ? "not-present page" :
 		 (error_code & X86_PF_RSVD)  ? "reserved bit violation" :
 		 (error_code & X86_PF_PK)    ? "protection keys violation" :
+		 (error_code & X86_PF_RMP)   ? "RMP violation" :
 					       "permissions violation");
 
 	if (!(error_code & X86_PF_USER) && user_mode(regs)) {
Signature · 1 line
2.25.1
author maintainer
#93 lore raw

PATCH v6 9/49 x86/fault: Add support to handle the RMP fault for user address

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

When SEV-SNP is enabled globally, a write from the host goes through the RMP check. When the host writes to pages, hardware checks the following conditions at the end of page walk:

1. Assigned bit in the RMP table is zero (i.e page is shared).
2. If the page table entry that gives the sPA indicates that the target page size is a large page, then all RMP entries for the 4KB constituting pages of the target must have the assigned bit 0.
3. Immutable bit in the RMP table is not zero.

The hardware will raise page fault if one of the above conditions is not met. Try resolving the fault instead of taking fault again and again. If the host attempts to write to the guest private memory then send the SIGBUS signal to kill the process. If the page level between the host and RMP entry does not match, then split the address to keep the RMP and host page levels in sync.

---
 arch/x86/mm/fault.c      | 66 ++++++++++++++++++++++++++++++++++++++++
 include/linux/mm.h       |  3 +-
 include/linux/mm_types.h |  3 ++
 mm/memory.c              | 13 ++++++++
 4 files changed, 84 insertions(+), 1 deletion(-)
arch/x86/mm/fault.c +66 -0
index a4c270e99f7f..f5de9673093a 100644
@@ -19,6 +19,7 @@
 #include <linux/uaccess.h>		/* faulthandler_disabled()	*/
 #include <linux/efi.h>			/* efi_crash_gracefully_on_page_fault()*/
 #include <linux/mm_types.h>
+#include <linux/sev.h>			/* snp_lookup_rmpentry()	*/
 
 #include <asm/cpufeature.h>		/* boot_cpu_has, ...		*/
 #include <asm/traps.h>			/* dotraplinkage, ...		*/
@@ -1209,6 +1210,60 @@ do_kern_addr_fault(struct pt_regs *regs, unsigned long hw_error_code,
 }
 NOKPROBE_SYMBOL(do_kern_addr_fault);
 
+static inline size_t pages_per_hpage(int level)
+{
+	return page_level_size(level) / PAGE_SIZE;
+}
+
+/*
+ * Return 1 if the caller need to retry, 0 if it the address need to be split
+ * in order to resolve the fault.
+ */
+static int handle_user_rmp_page_fault(struct pt_regs *regs, unsigned long error_code,
+				      unsigned long address)
+{
+	int rmp_level, level;
+	pte_t *pte;
+	u64 pfn;
+
+	pte = lookup_address_in_mm(current->mm, address, &level);
+
+	/*
+	 * It can happen if there was a race between an unmap event and
+	 * the RMP fault delivery.
+	 */
+	if (!pte || !pte_present(*pte))
+		return 1;
+
+	pfn = pte_pfn(*pte);
+
+	/* If its large page then calculte the fault pfn */
+	if (level > PG_LEVEL_4K) {
+		unsigned long mask;
+
+		mask = pages_per_hpage(level) - pages_per_hpage(level - 1);
+		pfn |= (address >> PAGE_SHIFT) & mask;
+	}
+
+	/*
+	 * If its a guest private page, then the fault cannot be resolved.
+	 * Send a SIGBUS to terminate the process.
+	 */
+	if (snp_lookup_rmpentry(pfn, &rmp_level)) {
+		do_sigbus(regs, error_code, address, VM_FAULT_SIGBUS);
+		return 1;
+	}
+
+	/*
+	 * The backing page level is higher than the RMP page level, request
+	 * to split the page.
+	 */
+	if (level > rmp_level)
+		return 0;
+
+	return 1;
+}
+
 /*
  * Handle faults in the user portion of the address space.  Nothing in here
  * should check X86_PF_USER without a specific justification: for almost
@@ -1306,6 +1361,17 @@ void do_user_addr_fault(struct pt_regs *regs,
 	if (error_code & X86_PF_INSTR)
 		flags |= FAULT_FLAG_INSTRUCTION;
 
+	/*
+	 * If its an RMP violation, try resolving it.
+	 */
+	if (error_code & X86_PF_RMP) {
+		if (handle_user_rmp_page_fault(regs, error_code, address))
+			return;
+
+		/* Ask to split the page */
+		flags |= FAULT_FLAG_PAGE_SPLIT;
+	}
+
 #ifdef CONFIG_X86_64
 	/*
 	 * Faults in the vsyscall page might need emulation.  The
include/linux/mm.h +2 -1
index de32c0383387..2ccc562d166f 100644
@@ -463,7 +463,8 @@ static inline bool fault_flag_allow_retry_first(enum fault_flag flags)
 	{ FAULT_FLAG_USER,		"USER" }, \
 	{ FAULT_FLAG_REMOTE,		"REMOTE" }, \
 	{ FAULT_FLAG_INSTRUCTION,	"INSTRUCTION" }, \
-	{ FAULT_FLAG_INTERRUPTIBLE,	"INTERRUPTIBLE" }
+	{ FAULT_FLAG_INTERRUPTIBLE,	"INTERRUPTIBLE" }, \
+	{ FAULT_FLAG_PAGE_SPLIT,	"PAGESPLIT" }
 
 /*
  * vm_fault is filled by the pagefault handler and passed to the vma's
include/linux/mm_types.h +3 -0
index 6dfaf271ebf8..aa2d8d48ce3e 100644
@@ -818,6 +818,8 @@ typedef struct {
  *                      mapped R/O.
  * @FAULT_FLAG_ORIG_PTE_VALID: whether the fault has vmf->orig_pte cached.
  *                        We should only access orig_pte if this flag set.
+ * @FAULT_FLAG_PAGE_SPLIT: The fault was due page size mismatch, split the
+ *                         region to smaller page size and retry.
  *
  * About @FAULT_FLAG_ALLOW_RETRY and @FAULT_FLAG_TRIED: we can specify
  * whether we would allow page faults to retry by specifying these two
@@ -855,6 +857,7 @@ enum fault_flag {
 	FAULT_FLAG_INTERRUPTIBLE =	1 << 9,
 	FAULT_FLAG_UNSHARE =		1 << 10,
 	FAULT_FLAG_ORIG_PTE_VALID =	1 << 11,
+	FAULT_FLAG_PAGE_SPLIT =		1 << 12,
 };
 
 typedef unsigned int __bitwise zap_flags_t;
mm/memory.c +13 -0
index 7274f2b52bca..c2187ffcbb8e 100644
@@ -4945,6 +4945,15 @@ static vm_fault_t handle_pte_fault(struct vm_fault *vmf)
 	return 0;
 }
 
+static int handle_split_page_fault(struct vm_fault *vmf)
+{
+	if (!IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT))
+		return VM_FAULT_SIGBUS;
+
+	__split_huge_pmd(vmf->vma, vmf->pmd, vmf->address, false, NULL);
+	return 0;
+}
+
 /*
  * By the time we get here, we already hold the mm semaphore
  *
@@ -5024,6 +5033,10 @@ static vm_fault_t __handle_mm_fault(struct vm_area_struct *vma,
 				pmd_migration_entry_wait(mm, vmf.pmd);
 			return 0;
 		}
+
+		if (flags & FAULT_FLAG_PAGE_SPLIT)
+			return handle_split_page_fault(&vmf);
+
 		if (pmd_trans_huge(vmf.orig_pmd) || pmd_devmap(vmf.orig_pmd)) {
 			if (pmd_protnone(vmf.orig_pmd) && vma_is_accessible(vma))
 				return do_huge_pmd_numa_page(&vmf);
Signature · 1 line
2.25.1
author maintainer
#125 lore raw

PATCH v6 10/49 x86/fault: Add support to dump RMP entry on fault

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

When SEV-SNP is enabled globally, a write from the host goes through the RMP check. If the hardware encounters the check failure, then it raises the #PF (with RMP set). Dump the RMP entry at the faulting pfn to help the debug.

---
 arch/x86/include/asm/sev.h |  7 +++++++
 arch/x86/kernel/sev.c      | 43 ++++++++++++++++++++++++++++++++++++++
 arch/x86/mm/fault.c        | 17 +++++++++++----
 include/linux/sev.h        |  2 ++
 4 files changed, 65 insertions(+), 4 deletions(-)
arch/x86/include/asm/sev.h +7 -0
index 6ab872311544..c0c4df817159 100644
@@ -113,6 +113,11 @@ struct __packed rmpentry {
 
 #define rmpentry_assigned(x)	((x)->info.assigned)
 #define rmpentry_pagesize(x)	((x)->info.pagesize)
+#define rmpentry_vmsa(x)	((x)->info.vmsa)
+#define rmpentry_asid(x)	((x)->info.asid)
+#define rmpentry_validated(x)	((x)->info.validated)
+#define rmpentry_gpa(x)		((unsigned long)(x)->info.gpa)
+#define rmpentry_immutable(x)	((x)->info.immutable)
 
 #define RMPADJUST_VMSA_PAGE_BIT		BIT(16)
 
@@ -205,6 +210,7 @@ void snp_set_wakeup_secondary_cpu(void);
 bool snp_init(struct boot_params *bp);
 void snp_abort(void);
 int snp_issue_guest_request(u64 exit_code, struct snp_req_data *input, unsigned long *fw_err);
+void dump_rmpentry(u64 pfn);
 #else
 static inline void sev_es_ist_enter(struct pt_regs *regs) { }
 static inline void sev_es_ist_exit(void) { }
@@ -229,6 +235,7 @@ static inline int snp_issue_guest_request(u64 exit_code, struct snp_req_data *in
 {
 	return -ENOTTY;
 }
+static inline void dump_rmpentry(u64 pfn) {}
 #endif
 
 #endif
arch/x86/kernel/sev.c +43 -0

49 lines not on this page · raw message

arch/x86/mm/fault.c +13 -4
index f5de9673093a..25896a6ba04a 100644
@@ -34,6 +34,7 @@
 #include <asm/kvm_para.h>		/* kvm_handle_async_pf		*/
 #include <asm/vdso.h>			/* fixup_vdso_exception()	*/
 #include <asm/irq_stack.h>
+#include <asm/sev.h>			/* dump_rmpentry()		*/
 
 #define CREATE_TRACE_POINTS
 #include <asm/trace/exceptions.h>
@@ -290,7 +291,7 @@ static bool low_pfn(unsigned long pfn)
 	return pfn < max_low_pfn;
 }
 
-static void dump_pagetable(unsigned long address)
+static void dump_pagetable(unsigned long address, bool show_rmpentry)
 {
 	pgd_t *base = __va(read_cr3_pa());
 	pgd_t *pgd = &base[pgd_index(address)];
@@ -346,10 +347,11 @@ static int bad_address(void *p)
 	return get_kernel_nofault(dummy, (unsigned long *)p);
 }
 
-static void dump_pagetable(unsigned long address)
+static void dump_pagetable(unsigned long address, bool show_rmpentry)
 {
 	pgd_t *base = __va(read_cr3_pa());
 	pgd_t *pgd = base + pgd_index(address);
+	unsigned long pfn;
 	p4d_t *p4d;
 	pud_t *pud;
 	pmd_t *pmd;
@@ -367,6 +369,7 @@ static void dump_pagetable(unsigned long address)
 	if (bad_address(p4d))
 		goto bad;
 
+	pfn = p4d_pfn(*p4d);
 	pr_cont("P4D %lx ", p4d_val(*p4d));
 	if (!p4d_present(*p4d) || p4d_large(*p4d))
 		goto out;
@@ -375,6 +378,7 @@ static void dump_pagetable(unsigned long address)
 	if (bad_address(pud))
 		goto bad;
 
+	pfn = pud_pfn(*pud);
 	pr_cont("PUD %lx ", pud_val(*pud));
 	if (!pud_present(*pud) || pud_large(*pud))
 		goto out;
@@ -383,6 +387,7 @@ static void dump_pagetable(unsigned long address)
 	if (bad_address(pmd))
 		goto bad;
 
+	pfn = pmd_pfn(*pmd);
 	pr_cont("PMD %lx ", pmd_val(*pmd));
 	if (!pmd_present(*pmd) || pmd_large(*pmd))
 		goto out;
@@ -391,9 +396,13 @@ static void dump_pagetable(unsigned long address)
 	if (bad_address(pte))
 		goto bad;
 
+	pfn = pte_pfn(*pte);
 	pr_cont("PTE %lx", pte_val(*pte));
 out:
 	pr_cont("\n");
+
+	if (show_rmpentry)
+		dump_rmpentry(pfn);
 	return;
 bad:
 	pr_info("BAD\n");
@@ -579,7 +588,7 @@ show_fault_oops(struct pt_regs *regs, unsigned long error_code, unsigned long ad
 		show_ldttss(&gdt, "TR", tr);
 	}
 
-	dump_pagetable(address);
+	dump_pagetable(address, error_code & X86_PF_RMP);
 }
 
 static noinline void
@@ -596,7 +605,7 @@ pgtable_bad(struct pt_regs *regs, unsigned long error_code,
 
 	printk(KERN_ALERT "%s: Corrupted page table at address %lx\n",
 	       tsk->comm, address);
-	dump_pagetable(address);
+	dump_pagetable(address, false);
 
 	if (__die("Bad pagetable", regs, error_code))
 		sig = 0;
include/linux/sev.h +2 -0
index 1a68842789e1..734b13a69c54 100644
@@ -16,6 +16,7 @@ int snp_lookup_rmpentry(u64 pfn, int *level);
 int psmash(u64 pfn);
 int rmp_make_private(u64 pfn, u64 gpa, enum pg_level level, int asid, bool immutable);
 int rmp_make_shared(u64 pfn, enum pg_level level);
+void dump_rmpentry(u64 pfn);
 #else
 static inline int snp_lookup_rmpentry(u64 pfn, int *level) { return 0; }
 static inline int psmash(u64 pfn) { return -ENXIO; }
@@ -25,6 +26,7 @@ static inline int rmp_make_private(u64 pfn, u64 gpa, enum pg_level level, int as
 	return -ENODEV;
 }
 static inline int rmp_make_shared(u64 pfn, enum pg_level level) { return -ENODEV; }
+static inline void dump_rmpentry(u64 pfn) { }
 
 #endif /* CONFIG_AMD_MEM_ENCRYPT */
 #endif /* __LINUX_SEV_H */
Signature · 1 line
2.25.1
author maintainer
#130 lore raw

PATCH v6 11/49 crypto:ccp: Define the SEV-SNP commands

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

AMD introduced the next generation of SEV called SEV-SNP (Secure Nested Paging). SEV-SNP builds upon existing SEV and SEV-ES functionality while adding new hardware security protection.

Define the commands and structures used to communicate with the AMD-SP when creating and managing the SEV-SNP guests. The SEV-SNP firmware spec is available at developer.amd.com/sev.

---
 drivers/crypto/ccp/sev-dev.c |  14 +++
 include/linux/psp-sev.h      | 222 +++++++++++++++++++++++++++++++++++
 include/uapi/linux/psp-sev.h |  42 +++++++
 3 files changed, 278 insertions(+)
drivers/crypto/ccp/sev-dev.c +14 -0
index fd928199bf1e..9cb3265f3bef 100644
@@ -153,6 +153,20 @@ static int sev_cmd_buffer_len(int cmd)
 	case SEV_CMD_GET_ID:			return sizeof(struct sev_data_get_id);
 	case SEV_CMD_ATTESTATION_REPORT:	return sizeof(struct sev_data_attestation_report);
 	case SEV_CMD_SEND_CANCEL:		return sizeof(struct sev_data_send_cancel);
+	case SEV_CMD_SNP_GCTX_CREATE:		return sizeof(struct sev_data_snp_gctx_create);
+	case SEV_CMD_SNP_LAUNCH_START:		return sizeof(struct sev_data_snp_launch_start);
+	case SEV_CMD_SNP_LAUNCH_UPDATE:		return sizeof(struct sev_data_snp_launch_update);
+	case SEV_CMD_SNP_ACTIVATE:		return sizeof(struct sev_data_snp_activate);
+	case SEV_CMD_SNP_DECOMMISSION:		return sizeof(struct sev_data_snp_decommission);
+	case SEV_CMD_SNP_PAGE_RECLAIM:		return sizeof(struct sev_data_snp_page_reclaim);
+	case SEV_CMD_SNP_GUEST_STATUS:		return sizeof(struct sev_data_snp_guest_status);
+	case SEV_CMD_SNP_LAUNCH_FINISH:		return sizeof(struct sev_data_snp_launch_finish);
+	case SEV_CMD_SNP_DBG_DECRYPT:		return sizeof(struct sev_data_snp_dbg);
+	case SEV_CMD_SNP_DBG_ENCRYPT:		return sizeof(struct sev_data_snp_dbg);
+	case SEV_CMD_SNP_PAGE_UNSMASH:		return sizeof(struct sev_data_snp_page_unsmash);
+	case SEV_CMD_SNP_PLATFORM_STATUS:	return sizeof(struct sev_data_snp_platform_status_buf);
+	case SEV_CMD_SNP_GUEST_REQUEST:		return sizeof(struct sev_data_snp_guest_request);
+	case SEV_CMD_SNP_CONFIG:		return sizeof(struct sev_user_data_snp_config);
 	default:				return 0;
 	}
 
include/linux/psp-sev.h +222 -0

234 lines not on this page · raw message

include/uapi/linux/psp-sev.h +42 -0

54 lines not on this page · raw message

Signature · 1 line
2.25.1
author maintainer
#134 lore raw

PATCH v6 12/49 crypto: ccp: Add support to initialize the AMD-SP for SEV-SNP

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

Before SNP VMs can be launched, the platform must be appropriately configured and initialized. Platform initialization is accomplished via the SNP_INIT command. Make sure to do a WBINVD and issue DF_FLUSH command to prepare for the first SNP guest launch after INIT.

Signed-off by: Ashish Kalra <[email protected]>

---
 drivers/crypto/ccp/sev-dev.c | 121 +++++++++++++++++++++++++++++++++++
 drivers/crypto/ccp/sev-dev.h |   2 +
 include/linux/psp-sev.h      |  16 +++++
 3 files changed, 139 insertions(+)
drivers/crypto/ccp/sev-dev.c +121 -0
index 9cb3265f3bef..f1173221d0b9 100644
@@ -33,6 +33,10 @@
 #define SEV_FW_FILE		"amd/sev.fw"
 #define SEV_FW_NAME_SIZE	64
 
+/* Minimum firmware version required for the SEV-SNP support */
+#define SNP_MIN_API_MAJOR	1
+#define SNP_MIN_API_MINOR	51
+
 static DEFINE_MUTEX(sev_cmd_mutex);
 static struct sev_misc_dev *misc_dev;
 
@@ -775,6 +779,98 @@ static int sev_update_firmware(struct device *dev)
 	return ret;
 }
 
+static void snp_set_hsave_pa(void *arg)
+{
+	wrmsrl(MSR_VM_HSAVE_PA, 0);
+}
+
+static int __sev_snp_init_locked(int *error)
+{
+	struct psp_device *psp = psp_master;
+	struct sev_device *sev;
+	int rc = 0;
+
+	if (!psp || !psp->sev_data)
+		return -ENODEV;
+
+	sev = psp->sev_data;
+
+	if (sev->snp_inited)
+		return 0;
+
+	/*
+	 * The SNP_INIT requires the MSR_VM_HSAVE_PA must be set to 0h
+	 * across all cores.
+	 */
+	on_each_cpu(snp_set_hsave_pa, NULL, 1);
+
+	/* Issue the SNP_INIT firmware command. */
+	rc = __sev_do_cmd_locked(SEV_CMD_SNP_INIT, NULL, error);
+	if (rc)
+		return rc;
+
+	/* Prepare for first SNP guest launch after INIT */
+	wbinvd_on_all_cpus();
+	rc = __sev_do_cmd_locked(SEV_CMD_SNP_DF_FLUSH, NULL, error);
+	if (rc)
+		return rc;
+
+	sev->snp_inited = true;
+	dev_dbg(sev->dev, "SEV-SNP firmware initialized\n");
+
+	return rc;
+}
+
+int sev_snp_init(int *error)
+{
+	int rc;
+
+	if (!cpu_feature_enabled(X86_FEATURE_SEV_SNP))
+		return -ENODEV;
+
+	mutex_lock(&sev_cmd_mutex);
+	rc = __sev_snp_init_locked(error);
+	mutex_unlock(&sev_cmd_mutex);
+
+	return rc;
+}
+EXPORT_SYMBOL_GPL(sev_snp_init);
+
+static int __sev_snp_shutdown_locked(int *error)
+{
+	struct sev_device *sev = psp_master->sev_data;
+	int ret;
+
+	if (!sev->snp_inited)
+		return 0;
+
+	/* SHUTDOWN requires the DF_FLUSH */
+	wbinvd_on_all_cpus();
+	__sev_do_cmd_locked(SEV_CMD_SNP_DF_FLUSH, NULL, NULL);
+
+	ret = __sev_do_cmd_locked(SEV_CMD_SNP_SHUTDOWN, NULL, error);
+	if (ret) {
+		dev_err(sev->dev, "SEV-SNP firmware shutdown failed\n");
+		return ret;
+	}
+
+	sev->snp_inited = false;
+	dev_dbg(sev->dev, "SEV-SNP firmware shutdown\n");
+
+	return ret;
+}
+
+static int sev_snp_shutdown(int *error)
+{
+	int rc;
+
+	mutex_lock(&sev_cmd_mutex);
+	rc = __sev_snp_shutdown_locked(NULL);
+	mutex_unlock(&sev_cmd_mutex);
+
+	return rc;
+}
+
 static int sev_ioctl_do_pek_import(struct sev_issue_cmd *argp, bool writable)
 {
 	struct sev_device *sev = psp_master->sev_data;
@@ -1231,6 +1327,8 @@ static void sev_firmware_shutdown(struct sev_device *sev)
 			   get_order(NV_LENGTH));
 		sev_init_ex_buffer = NULL;
 	}
+
+	sev_snp_shutdown(NULL);
 }
 
 void sev_dev_destroy(struct psp_device *psp)
@@ -1287,6 +1385,26 @@ void sev_pci_init(void)
 		}
 	}
 
+	/*
+	 * If boot CPU supports the SNP, then first attempt to initialize
+	 * the SNP firmware.
+	 */
+	if (cpu_feature_enabled(X86_FEATURE_SEV_SNP)) {
+		if (!sev_version_greater_or_equal(SNP_MIN_API_MAJOR, SNP_MIN_API_MINOR)) {
+			dev_err(sev->dev, "SEV-SNP support requires firmware version >= %d:%d\n",
+				SNP_MIN_API_MAJOR, SNP_MIN_API_MINOR);
+		} else {
+			rc = sev_snp_init(&error);
+			if (rc) {
+				/*
+				 * If we failed to INIT SNP then don't abort the probe.
+				 * Continue to initialize the legacy SEV firmware.
+				 */
+				dev_err(sev->dev, "SEV-SNP: failed to INIT error %#x\n", error);
+			}
+		}
+	}
+
 	/* Obtain the TMR memory area for SEV-ES use */
 	sev_es_tmr = sev_fw_alloc(SEV_ES_TMR_SIZE);
 	if (!sev_es_tmr)
@@ -1302,6 +1420,9 @@ void sev_pci_init(void)
 		dev_err(sev->dev, "SEV: failed to INIT error %#x, rc %d\n",
 			error, rc);
 
+	dev_info(sev->dev, "SEV%s API:%d.%d build:%d\n", sev->snp_inited ?
+		"-SNP" : "", sev->api_major, sev->api_minor, sev->build);
+
 	return;
 
 err:
drivers/crypto/ccp/sev-dev.h +2 -0
index 666c21eb81ab..186ad20cbd24 100644
@@ -52,6 +52,8 @@ struct sev_device {
 	u8 build;
 
 	void *cmd_buf;
+
+	bool snp_inited;
 };
 
 int sev_dev_init(struct psp_device *psp);
include/linux/psp-sev.h +16 -0
index 01ba9dc46ca3..ef4d42e8c96e 100644
@@ -769,6 +769,20 @@ struct sev_data_snp_init_ex {
  */
 int sev_platform_init(int *error);
 
+/**
+ * sev_snp_init - perform SEV SNP_INIT command
+ *
+ * @error: SEV command return code
+ *
+ * Returns:
+ * 0 if the SEV successfully processed the command
+ * -%ENODEV    if the SEV device is not available
+ * -%ENOTSUPP  if the SEV does not support SEV
+ * -%ETIMEDOUT if the SEV command timed out
+ * -%EIO       if the SEV returned a non-zero return code
+ */
+int sev_snp_init(int *error);
+
 /**
  * sev_platform_status - perform SEV PLATFORM_STATUS command
  *
@@ -876,6 +890,8 @@ sev_platform_status(struct sev_user_data_status *status, int *error) { return -E
 
 static inline int sev_platform_init(int *error) { return -ENODEV; }
 
+static inline int sev_snp_init(int *error) { return -ENODEV; }
+
 static inline int
 sev_guest_deactivate(struct sev_data_deactivate *data, int *error) { return -ENODEV; }
 
Signature · 1 line
2.25.1
author maintainer
#143 lore raw

PATCH v6 13/49 crypto:ccp: Provide APIs to issue SEV-SNP commands

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

Provide the APIs for the hypervisor to manage an SEV-SNP guest. The commands for SEV-SNP is defined in the SEV-SNP firmware specification.

---
 drivers/crypto/ccp/sev-dev.c | 24 ++++++++++++
 include/linux/psp-sev.h      | 73 ++++++++++++++++++++++++++++++++++++
 2 files changed, 97 insertions(+)
drivers/crypto/ccp/sev-dev.c +24 -0
index f1173221d0b9..35d76333e120 100644
@@ -1205,6 +1205,30 @@ int sev_guest_df_flush(int *error)
 }
 EXPORT_SYMBOL_GPL(sev_guest_df_flush);
 
+int snp_guest_decommission(struct sev_data_snp_decommission *data, int *error)
+{
+	return sev_do_cmd(SEV_CMD_SNP_DECOMMISSION, data, error);
+}
+EXPORT_SYMBOL_GPL(snp_guest_decommission);
+
+int snp_guest_df_flush(int *error)
+{
+	return sev_do_cmd(SEV_CMD_SNP_DF_FLUSH, NULL, error);
+}
+EXPORT_SYMBOL_GPL(snp_guest_df_flush);
+
+int snp_guest_page_reclaim(struct sev_data_snp_page_reclaim *data, int *error)
+{
+	return sev_do_cmd(SEV_CMD_SNP_PAGE_RECLAIM, data, error);
+}
+EXPORT_SYMBOL_GPL(snp_guest_page_reclaim);
+
+int snp_guest_dbg_decrypt(struct sev_data_snp_dbg *data, int *error)
+{
+	return sev_do_cmd(SEV_CMD_SNP_DBG_DECRYPT, data, error);
+}
+EXPORT_SYMBOL_GPL(snp_guest_dbg_decrypt);
+
 static void sev_exit(struct kref *ref)
 {
 	misc_deregister(&misc_dev->misc);
include/linux/psp-sev.h +73 -0

85 lines not on this page · raw message

Signature · 1 line
2.25.1
author maintainer
#156 lore raw

PATCH v6 14/49 crypto: ccp: Handle the legacy TMR allocation when SNP is enabled

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

The behavior and requirement for the SEV-legacy command is altered when the SNP firmware is in the INIT state. See SEV-SNP firmware specification for more details.

Allocate the Trusted Memory Region (TMR) as a 2mb sized/aligned region when SNP is enabled to satify new requirements for the SNP. Continue allocating a 1mb region for !SNP configuration.

While at it, provide API that can be used by others to allocate a page that can be used by the firmware. The immediate user for this API will be the KVM driver. The KVM driver to need to allocate a firmware context page during the guest creation. The context page need to be updated by the firmware. See the SEV-SNP specification for further details.

---
 drivers/crypto/ccp/sev-dev.c | 173 +++++++++++++++++++++++++++++++++--
 include/linux/psp-sev.h      |  11 +++
 2 files changed, 178 insertions(+), 6 deletions(-)
drivers/crypto/ccp/sev-dev.c +167 -6
index 35d76333e120..0dbd99f29b25 100644
@@ -79,6 +79,14 @@ static void *sev_es_tmr;
 #define NV_LENGTH (32 * 1024)
 static void *sev_init_ex_buffer;
 
+/* When SEV-SNP is enabled the TMR needs to be 2MB aligned and 2MB size. */
+#define SEV_SNP_ES_TMR_SIZE	(2 * 1024 * 1024)
+
+static size_t sev_es_tmr_size = SEV_ES_TMR_SIZE;
+
+static int __sev_do_cmd_locked(int cmd, void *data, int *psp_ret);
+static int sev_do_cmd(int cmd, void *data, int *psp_ret);
+
 static inline bool sev_version_greater_or_equal(u8 maj, u8 min)
 {
 	struct sev_device *sev = psp_master->sev_data;
@@ -177,11 +185,161 @@ static int sev_cmd_buffer_len(int cmd)
 	return 0;
 }
 
+static void snp_leak_pages(unsigned long pfn, unsigned int npages)
+{
+	WARN(1, "psc failed, pfn 0x%lx pages %d (leaking)\n", pfn, npages);
+	while (npages--) {
+		memory_failure(pfn, 0);
+		dump_rmpentry(pfn);
+		pfn++;
+	}
+}
+
+static int snp_reclaim_pages(unsigned long pfn, unsigned int npages, bool locked)
+{
+	struct sev_data_snp_page_reclaim data;
+	int ret, err, i, n = 0;
+
+	for (i = 0; i < npages; i++) {
+		memset(&data, 0, sizeof(data));
+		data.paddr = pfn << PAGE_SHIFT;
+
+		if (locked)
+			ret = __sev_do_cmd_locked(SEV_CMD_SNP_PAGE_RECLAIM, &data, &err);
+		else
+			ret = sev_do_cmd(SEV_CMD_SNP_PAGE_RECLAIM, &data, &err);
+		if (ret)
+			goto cleanup;
+
+		ret = rmp_make_shared(pfn, PG_LEVEL_4K);
+		if (ret)
+			goto cleanup;
+
+		pfn++;
+		n++;
+	}
+
+	return 0;
+
+cleanup:
+	/*
+	 * If failed to reclaim the page then page is no longer safe to
+	 * be released, leak it.
+	 */
+	snp_leak_pages(pfn, npages - n);
+	return ret;
+}
+
+static inline int rmp_make_firmware(unsigned long pfn, int level)
+{
+	return rmp_make_private(pfn, 0, level, 0, true);
+}
+
+static int snp_set_rmp_state(unsigned long paddr, unsigned int npages, bool to_fw, bool locked,
+			     bool need_reclaim)
+{
+	unsigned long pfn = __sme_clr(paddr) >> PAGE_SHIFT; /* Cbit maybe set in the paddr */
+	int rc, n = 0, i;
+
+	for (i = 0; i < npages; i++) {
+		if (to_fw)
+			rc = rmp_make_firmware(pfn, PG_LEVEL_4K);
+		else
+			rc = need_reclaim ? snp_reclaim_pages(pfn, 1, locked) :
+					    rmp_make_shared(pfn, PG_LEVEL_4K);
+		if (rc)
+			goto cleanup;
+
+		pfn++;
+		n++;
+	}
+
+	return 0;
+
+cleanup:
+	/* Try unrolling the firmware state changes */
+	if (to_fw) {
+		/*
+		 * Reclaim the pages which were already changed to the
+		 * firmware state.
+		 */
+		snp_reclaim_pages(paddr >> PAGE_SHIFT, n, locked);
+
+		return rc;
+	}
+
+	/*
+	 * If failed to change the page state to shared, then its not safe
+	 * to release the page back to the system, leak it.
+	 */
+	snp_leak_pages(pfn, npages - n);
+
+	return rc;
+}
+
+static struct page *__snp_alloc_firmware_pages(gfp_t gfp_mask, int order, bool locked)
+{
+	unsigned long npages = 1ul << order, paddr;
+	struct sev_device *sev;
+	struct page *page;
+
+	if (!psp_master || !psp_master->sev_data)
+		return NULL;
+
+	page = alloc_pages(gfp_mask, order);
+	if (!page)
+		return NULL;
+
+	/* If SEV-SNP is initialized then add the page in RMP table. */
+	sev = psp_master->sev_data;
+	if (!sev->snp_inited)
+		return page;
+
+	paddr = __pa((unsigned long)page_address(page));
+	if (snp_set_rmp_state(paddr, npages, true, locked, false))
+		return NULL;
+
+	return page;
+}
+
+void *snp_alloc_firmware_page(gfp_t gfp_mask)
+{
+	struct page *page;
+
+	page = __snp_alloc_firmware_pages(gfp_mask, 0, false);
+
+	return page ? page_address(page) : NULL;
+}
+EXPORT_SYMBOL_GPL(snp_alloc_firmware_page);
+
+static void __snp_free_firmware_pages(struct page *page, int order, bool locked)
+{
+	unsigned long paddr, npages = 1ul << order;
+
+	if (!page)
+		return;
+
+	paddr = __pa((unsigned long)page_address(page));
+	if (snp_set_rmp_state(paddr, npages, false, locked, true))
+		return;
+
+	__free_pages(page, order);
+}
+
+void snp_free_firmware_page(void *addr)
+{
+	if (!addr)
+		return;
+
+	__snp_free_firmware_pages(virt_to_page(addr), 0, false);
+}
+EXPORT_SYMBOL(snp_free_firmware_page);
+
 static void *sev_fw_alloc(unsigned long len)
 {
 	struct page *page;
 
-	page = alloc_pages(GFP_KERNEL, get_order(len));
+	page = __snp_alloc_firmware_pages(GFP_KERNEL, get_order(len), false);
 	if (!page)
 		return NULL;
 
@@ -393,7 +551,7 @@ static int __sev_init_locked(int *error)
 		data.tmr_address = __pa(sev_es_tmr);
 
 		data.flags |= SEV_INIT_FLAGS_SEV_ES;
-		data.tmr_len = SEV_ES_TMR_SIZE;
+		data.tmr_len = sev_es_tmr_size;
 	}
 
 	return __sev_do_cmd_locked(SEV_CMD_INIT, &data, error);
@@ -421,7 +579,7 @@ static int __sev_init_ex_locked(int *error)
 		data.tmr_address = __pa(sev_es_tmr);
 
 		data.flags |= SEV_INIT_FLAGS_SEV_ES;
-		data.tmr_len = SEV_ES_TMR_SIZE;
+		data.tmr_len = sev_es_tmr_size;
 	}
 
 	return __sev_do_cmd_locked(SEV_CMD_INIT_EX, &data, error);
@@ -818,6 +976,8 @@ static int __sev_snp_init_locked(int *error)
 	sev->snp_inited = true;
 	dev_dbg(sev->dev, "SEV-SNP firmware initialized\n");
 
+	sev_es_tmr_size = SEV_SNP_ES_TMR_SIZE;
+
 	return rc;
 }
 
@@ -1341,8 +1501,9 @@ static void sev_firmware_shutdown(struct sev_device *sev)
 		/* The TMR area was encrypted, flush it from the cache */
 		wbinvd_on_all_cpus();
 
-		free_pages((unsigned long)sev_es_tmr,
-			   get_order(SEV_ES_TMR_SIZE));
+		__snp_free_firmware_pages(virt_to_page(sev_es_tmr),
+					  get_order(sev_es_tmr_size),
+					  false);
 		sev_es_tmr = NULL;
 	}
 
@@ -1430,7 +1591,7 @@ void sev_pci_init(void)
 	}
 
 	/* Obtain the TMR memory area for SEV-ES use */
-	sev_es_tmr = sev_fw_alloc(SEV_ES_TMR_SIZE);
+	sev_es_tmr = sev_fw_alloc(sev_es_tmr_size);
 	if (!sev_es_tmr)
 		dev_warn(sev->dev,
 			 "SEV: TMR allocation failed, SEV-ES support unavailable\n");
include/linux/psp-sev.h +11 -0
index 9f921d221b75..a3bb792bb842 100644
@@ -12,6 +12,8 @@
 #ifndef __PSP_SEV_H__
 #define __PSP_SEV_H__
 
+#include <linux/sev.h>
+
 #include <uapi/linux/psp-sev.h>
 
 #ifdef CONFIG_X86
@@ -940,6 +942,8 @@ int snp_guest_page_reclaim(struct sev_data_snp_page_reclaim *data, int *error);
 int snp_guest_dbg_decrypt(struct sev_data_snp_dbg *data, int *error);
 
 void *psp_copy_user_blob(u64 uaddr, u32 len);
+void *snp_alloc_firmware_page(gfp_t mask);
+void snp_free_firmware_page(void *addr);
 
 #else	/* !CONFIG_CRYPTO_DEV_SP_PSP */
 
@@ -981,6 +985,13 @@ static inline int snp_guest_dbg_decrypt(struct sev_data_snp_dbg *data, int *erro
 	return -ENODEV;
 }
 
+static inline void *snp_alloc_firmware_page(gfp_t mask)
+{
+	return NULL;
+}
+
+static inline void snp_free_firmware_page(void *addr) { }
+
 #endif	/* CONFIG_CRYPTO_DEV_SP_PSP */
 
 #endif	/* __PSP_SEV_H__ */
Signature · 1 line
2.25.1
author maintainer
#198 lore raw

PATCH v6 15/49 crypto: ccp: Handle the legacy SEV command when SNP is enabled

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

The behavior of the SEV-legacy commands is altered when the SNP firmware is in the INIT state. When SNP is in INIT state, all the SEV-legacy commands that cause the firmware to write to memory must be in the firmware state before issuing the command..

A command buffer may contains a system physical address that the firmware may write to. There are two cases that need to be handled:

1) system physical address points to a guest memory
2) system physical address points to a host memory

To handle the case #1, change the page state to the firmware in the RMP table before issuing the command and restore the state to shared after the command completes.

For the case #2, use a bounce buffer to complete the request.

---
 drivers/crypto/ccp/sev-dev.c | 346 ++++++++++++++++++++++++++++++++++-
 drivers/crypto/ccp/sev-dev.h |  12 ++
 2 files changed, 348 insertions(+), 10 deletions(-)
drivers/crypto/ccp/sev-dev.c +336 -10
index 0dbd99f29b25..75f5c4ed9ac3 100644
@@ -441,12 +441,295 @@ static void sev_write_init_ex_file_if_required(int cmd_id)
 	sev_write_init_ex_file();
 }
 
+static int alloc_snp_host_map(struct sev_device *sev)
+{
+	struct page *page;
+	int i;
+
+	for (i = 0; i < MAX_SNP_HOST_MAP_BUFS; i++) {
+		struct snp_host_map *map = &sev->snp_host_map[i];
+
+		memset(map, 0, sizeof(*map));
+
+		page = alloc_pages(GFP_KERNEL_ACCOUNT, get_order(SEV_FW_BLOB_MAX_SIZE));
+		if (!page)
+			return -ENOMEM;
+
+		map->host = page_address(page);
+	}
+
+	return 0;
+}
+
+static void free_snp_host_map(struct sev_device *sev)
+{
+	int i;
+
+	for (i = 0; i < MAX_SNP_HOST_MAP_BUFS; i++) {
+		struct snp_host_map *map = &sev->snp_host_map[i];
+
+		if (map->host) {
+			__free_pages(virt_to_page(map->host), get_order(SEV_FW_BLOB_MAX_SIZE));
+			memset(map, 0, sizeof(*map));
+		}
+	}
+}
+
+static int map_firmware_writeable(u64 *paddr, u32 len, bool guest, struct snp_host_map *map)
+{
+	unsigned int npages = PAGE_ALIGN(len) >> PAGE_SHIFT;
+
+	map->active = false;
+
+	if (!paddr || !len)
+		return 0;
+
+	map->paddr = *paddr;
+	map->len = len;
+
+	/* If paddr points to a guest memory then change the page state to firmwware. */
+	if (guest) {
+		if (snp_set_rmp_state(*paddr, npages, true, true, false))
+			return -EFAULT;
+
+		goto done;
+	}
+
+	if (!map->host)
+		return -ENOMEM;
+
+	/* Check if the pre-allocated buffer can be used to fullfil the request. */
+	if (len > SEV_FW_BLOB_MAX_SIZE)
+		return -EINVAL;
+
+	/* Transition the pre-allocated buffer to the firmware state. */
+	if (snp_set_rmp_state(__pa(map->host), npages, true, true, false))
+		return -EFAULT;
+
+	/* Set the paddr to use pre-allocated firmware buffer */
+	*paddr = __psp_pa(map->host);
+
+done:
+	map->active = true;
+	return 0;
+}
+
+static int unmap_firmware_writeable(u64 *paddr, u32 len, bool guest, struct snp_host_map *map)
+{
+	unsigned int npages = PAGE_ALIGN(len) >> PAGE_SHIFT;
+
+	if (!map->active)
+		return 0;
+
+	/* If paddr points to a guest memory then restore the page state to hypervisor. */
+	if (guest) {
+		if (snp_set_rmp_state(*paddr, npages, false, true, true))
+			return -EFAULT;
+
+		goto done;
+	}
+
+	/*
+	 * Transition the pre-allocated buffer to hypervisor state before the access.
+	 *
+	 * This is because while changing the page state to firmware, the kernel unmaps
+	 * the pages from the direct map, and to restore the direct map we must
+	 * transition the pages to shared state.
+	 */
+	if (snp_set_rmp_state(__pa(map->host), npages, false, true, true))
+		return -EFAULT;
+
+	/* Copy the response data firmware buffer to the callers buffer. */
+	memcpy(__va(__sme_clr(map->paddr)), map->host, min_t(size_t, len, map->len));
+	*paddr = map->paddr;
+
+done:
+	map->active = false;
+	return 0;
+}
+
+static bool sev_legacy_cmd_buf_writable(int cmd)
+{
+	switch (cmd) {
+	case SEV_CMD_PLATFORM_STATUS:
+	case SEV_CMD_GUEST_STATUS:
+	case SEV_CMD_LAUNCH_START:
+	case SEV_CMD_RECEIVE_START:
+	case SEV_CMD_LAUNCH_MEASURE:
+	case SEV_CMD_SEND_START:
+	case SEV_CMD_SEND_UPDATE_DATA:
+	case SEV_CMD_SEND_UPDATE_VMSA:
+	case SEV_CMD_PEK_CSR:
+	case SEV_CMD_PDH_CERT_EXPORT:
+	case SEV_CMD_GET_ID:
+	case SEV_CMD_ATTESTATION_REPORT:
+		return true;
+	default:
+		return false;
+	}
+}
+
+#define prep_buffer(name, addr, len, guest, map) \
+	func(&((typeof(name *))cmd_buf)->addr, ((typeof(name *))cmd_buf)->len, guest, map)
+
+static int __snp_cmd_buf_copy(int cmd, void *cmd_buf, bool to_fw, int fw_err)
+{
+	int (*func)(u64 *paddr, u32 len, bool guest, struct snp_host_map *map);
+	struct sev_device *sev = psp_master->sev_data;
+	bool from_fw = !to_fw;
+
+	/*
+	 * After the command is completed, change the command buffer memory to
+	 * hypervisor state.
+	 *
+	 * The immutable bit is automatically cleared by the firmware, so
+	 * no not need to reclaim the page.
+	 */
+	if (from_fw && sev_legacy_cmd_buf_writable(cmd)) {
+		if (snp_set_rmp_state(__pa(cmd_buf), 1, false, true, false))
+			return -EFAULT;
+
+		/* No need to go further if firmware failed to execute command. */
+		if (fw_err)
+			return 0;
+	}
+
+	if (to_fw)
+		func = map_firmware_writeable;
+	else
+		func = unmap_firmware_writeable;
+
+	/*
+	 * A command buffer may contains a system physical address. If the address
+	 * points to a host memory then use an intermediate firmware page otherwise
+	 * change the page state in the RMP table.
+	 */
+	switch (cmd) {
+	case SEV_CMD_PDH_CERT_EXPORT:
+		if (prep_buffer(struct sev_data_pdh_cert_export, pdh_cert_address,
+				pdh_cert_len, false, &sev->snp_host_map[0]))
+			goto err;
+		if (prep_buffer(struct sev_data_pdh_cert_export, cert_chain_address,
+				cert_chain_len, false, &sev->snp_host_map[1]))
+			goto err;
+		break;
+	case SEV_CMD_GET_ID:
+		if (prep_buffer(struct sev_data_get_id, address, len,
+				false, &sev->snp_host_map[0]))
+			goto err;
+		break;
+	case SEV_CMD_PEK_CSR:
+		if (prep_buffer(struct sev_data_pek_csr, address, len,
+				false, &sev->snp_host_map[0]))
+			goto err;
+		break;
+	case SEV_CMD_LAUNCH_UPDATE_DATA:
+		if (prep_buffer(struct sev_data_launch_update_data, address, len,
+				true, &sev->snp_host_map[0]))
+			goto err;
+		break;
+	case SEV_CMD_LAUNCH_UPDATE_VMSA:
+		if (prep_buffer(struct sev_data_launch_update_vmsa, address, len,
+				true, &sev->snp_host_map[0]))
+			goto err;
+		break;
+	case SEV_CMD_LAUNCH_MEASURE:
+		if (prep_buffer(struct sev_data_launch_measure, address, len,
+				false, &sev->snp_host_map[0]))
+			goto err;
+		break;
+	case SEV_CMD_LAUNCH_UPDATE_SECRET:
+		if (prep_buffer(struct sev_data_launch_secret, guest_address, guest_len,
+				true, &sev->snp_host_map[0]))
+			goto err;
+		break;
+	case SEV_CMD_DBG_DECRYPT:
+		if (prep_buffer(struct sev_data_dbg, dst_addr, len, false,
+				&sev->snp_host_map[0]))
+			goto err;
+		break;
+	case SEV_CMD_DBG_ENCRYPT:
+		if (prep_buffer(struct sev_data_dbg, dst_addr, len, true,
+				&sev->snp_host_map[0]))
+			goto err;
+		break;
+	case SEV_CMD_ATTESTATION_REPORT:
+		if (prep_buffer(struct sev_data_attestation_report, address, len,
+				false, &sev->snp_host_map[0]))
+			goto err;
+		break;
+	case SEV_CMD_SEND_START:
+		if (prep_buffer(struct sev_data_send_start, session_address,
+				session_len, false, &sev->snp_host_map[0]))
+			goto err;
+		break;
+	case SEV_CMD_SEND_UPDATE_DATA:
+		if (prep_buffer(struct sev_data_send_update_data, hdr_address, hdr_len,
+				false, &sev->snp_host_map[0]))
+			goto err;
+		if (prep_buffer(struct sev_data_send_update_data, trans_address,
+				trans_len, false, &sev->snp_host_map[1]))
+			goto err;
+		break;
+	case SEV_CMD_SEND_UPDATE_VMSA:
+		if (prep_buffer(struct sev_data_send_update_vmsa, hdr_address, hdr_len,
+				false, &sev->snp_host_map[0]))
+			goto err;
+		if (prep_buffer(struct sev_data_send_update_vmsa, trans_address,
+				trans_len, false, &sev->snp_host_map[1]))
+			goto err;
+		break;
+	case SEV_CMD_RECEIVE_UPDATE_DATA:
+		if (prep_buffer(struct sev_data_receive_update_data, guest_address,
+				guest_len, true, &sev->snp_host_map[0]))
+			goto err;
+		break;
+	case SEV_CMD_RECEIVE_UPDATE_VMSA:
+		if (prep_buffer(struct sev_data_receive_update_vmsa, guest_address,
+				guest_len, true, &sev->snp_host_map[0]))
+			goto err;
+		break;
+	default:
+		break;
+	}
+
+	/* The command buffer need to be in the firmware state. */
+	if (to_fw && sev_legacy_cmd_buf_writable(cmd)) {
+		if (snp_set_rmp_state(__pa(cmd_buf), 1, true, true, false))
+			return -EFAULT;
+	}
+
+	return 0;
+
+err:
+	return -EINVAL;
+}
+
+static inline bool need_firmware_copy(int cmd)
+{
+	struct sev_device *sev = psp_master->sev_data;
+
+	/* After SNP is INIT'ed, the behavior of legacy SEV command is changed. */
+	return ((cmd < SEV_CMD_SNP_INIT) && sev->snp_inited) ? true : false;
+}
+
+static int snp_aware_copy_to_firmware(int cmd, void *data)
+{
+	return __snp_cmd_buf_copy(cmd, data, true, 0);
+}
+
+static int snp_aware_copy_from_firmware(int cmd, void *data, int fw_err)
+{
+	return __snp_cmd_buf_copy(cmd, data, false, fw_err);
+}
+
 static int __sev_do_cmd_locked(int cmd, void *data, int *psp_ret)
 {
 	struct psp_device *psp = psp_master;
 	struct sev_device *sev;
 	unsigned int phys_lsb, phys_msb;
 	unsigned int reg, ret = 0;
+	void *cmd_buf;
 	int buf_len;
 
 	if (!psp || !psp->sev_data)
@@ -466,12 +749,28 @@ static int __sev_do_cmd_locked(int cmd, void *data, int *psp_ret)
 	 * work for some memory, e.g. vmalloc'd addresses, and @data may not be
 	 * physically contiguous.
 	 */
-	if (data)
-		memcpy(sev->cmd_buf, data, buf_len);
+	if (data) {
+		if (sev->cmd_buf_active > 2)
+			return -EBUSY;
+
+		cmd_buf = sev->cmd_buf_active ? sev->cmd_buf_backup : sev->cmd_buf;
+
+		memcpy(cmd_buf, data, buf_len);
+		sev->cmd_buf_active++;
+
+		/*
+		 * The behavior of the SEV-legacy commands is altered when the
+		 * SNP firmware is in the INIT state.
+		 */
+		if (need_firmware_copy(cmd) && snp_aware_copy_to_firmware(cmd, sev->cmd_buf))
+			return -EFAULT;
+	} else {
+		cmd_buf = sev->cmd_buf;
+	}
 
 	/* Get the physical address of the command buffer */
-	phys_lsb = data ? lower_32_bits(__psp_pa(sev->cmd_buf)) : 0;
-	phys_msb = data ? upper_32_bits(__psp_pa(sev->cmd_buf)) : 0;
+	phys_lsb = data ? lower_32_bits(__psp_pa(cmd_buf)) : 0;
+	phys_msb = data ? upper_32_bits(__psp_pa(cmd_buf)) : 0;
 
 	dev_dbg(sev->dev, "sev command id %#x buffer 0x%08x%08x timeout %us\n",
 		cmd, phys_msb, phys_lsb, psp_timeout);
@@ -514,15 +813,24 @@ static int __sev_do_cmd_locked(int cmd, void *data, int *psp_ret)
 		sev_write_init_ex_file_if_required(cmd);
 	}
 
-	print_hex_dump_debug("(out): ", DUMP_PREFIX_OFFSET, 16, 2, data,
-			     buf_len, false);
-
 	/*
 	 * Copy potential output from the PSP back to data.  Do this even on
 	 * failure in case the caller wants to glean something from the error.
 	 */
-	if (data)
-		memcpy(data, sev->cmd_buf, buf_len);
+	if (data) {
+		/*
+		 * Restore the page state after the command completes.
+		 */
+		if (need_firmware_copy(cmd) &&
+		    snp_aware_copy_from_firmware(cmd, cmd_buf, ret))
+			return -EFAULT;
+
+		memcpy(data, cmd_buf, buf_len);
+		sev->cmd_buf_active--;
+	}
+
+	print_hex_dump_debug("(out): ", DUMP_PREFIX_OFFSET, 16, 2, data,
+			     buf_len, false);
 
 	return ret;
 }
@@ -1451,10 +1759,12 @@ int sev_dev_init(struct psp_device *psp)
 	if (!sev)
 		goto e_err;
 
-	sev->cmd_buf = (void *)devm_get_free_pages(dev, GFP_KERNEL, 0);
+	sev->cmd_buf = (void *)devm_get_free_pages(dev, GFP_KERNEL, 1);
 	if (!sev->cmd_buf)
 		goto e_sev;
 
+	sev->cmd_buf_backup = (uint8_t *)sev->cmd_buf + PAGE_SIZE;
+
 	psp->sev_data = sev;
 
 	sev->dev = dev;
@@ -1513,6 +1823,12 @@ static void sev_firmware_shutdown(struct sev_device *sev)
 		sev_init_ex_buffer = NULL;
 	}
 
+	/*
+	 * The host map need to clear the immutable bit so it must be free'd before the
+	 * SNP firmware shutdown.
+	 */
+	free_snp_host_map(sev);
+
 	sev_snp_shutdown(NULL);
 }
 
@@ -1588,6 +1904,14 @@ void sev_pci_init(void)
 				dev_err(sev->dev, "SEV-SNP: failed to INIT error %#x\n", error);
 			}
 		}
+
+		/*
+		 * Allocate the intermediate buffers used for the legacy command handling.
+		 */
+		if (alloc_snp_host_map(sev)) {
+			dev_notice(sev->dev, "Failed to alloc host map (disabling legacy SEV)\n");
+			goto skip_legacy;
+		}
 	}
 
 	/* Obtain the TMR memory area for SEV-ES use */
@@ -1605,12 +1929,14 @@ void sev_pci_init(void)
 		dev_err(sev->dev, "SEV: failed to INIT error %#x, rc %d\n",
 			error, rc);
 
+skip_legacy:
 	dev_info(sev->dev, "SEV%s API:%d.%d build:%d\n", sev->snp_inited ?
 		"-SNP" : "", sev->api_major, sev->api_minor, sev->build);
 
 	return;
 
 err:
+	free_snp_host_map(sev);
 	psp_master->sev_data = NULL;
 }
 
drivers/crypto/ccp/sev-dev.h +12 -0
index 186ad20cbd24..fe5d7a3ebace 100644
@@ -29,11 +29,20 @@
 #define SEV_CMDRESP_CMD_SHIFT		16
 #define SEV_CMDRESP_IOC			BIT(0)
 
+#define MAX_SNP_HOST_MAP_BUFS		2
+
 struct sev_misc_dev {
 	struct kref refcount;
 	struct miscdevice misc;
 };
 
+struct snp_host_map {
+	u64 paddr;
+	u32 len;
+	void *host;
+	bool active;
+};
+
 struct sev_device {
 	struct device *dev;
 	struct psp_device *psp;
@@ -52,8 +61,11 @@ struct sev_device {
 	u8 build;
 
 	void *cmd_buf;
+	void *cmd_buf_backup;
+	int cmd_buf_active;
 
 	bool snp_inited;
+	struct snp_host_map snp_host_map[MAX_SNP_HOST_MAP_BUFS];
 };
 
 int sev_dev_init(struct psp_device *psp);
Signature · 1 line
2.25.1
author maintainer
#199 lore raw

PATCH v6 16/49 crypto: ccp: Add the SNP_PLATFORM_STATUS command

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

The command can be used by the userspace to query the SNP platform status report. See the SEV-SNP spec for more details.

---
 Documentation/virt/coco/sevguest.rst | 27 +++++++++++++++++
 drivers/crypto/ccp/sev-dev.c         | 45 ++++++++++++++++++++++++++++
 include/uapi/linux/psp-sev.h         |  1 +
 3 files changed, 73 insertions(+)
Documentation/virt/coco/sevguest.rst +27 -0
index bf593e88cfd9..11ea67c944df 100644
@@ -61,6 +61,22 @@ counter (e.g. counter overflow), then -EIO will be returned.
                 __u64 fw_err;
         };
 
+The host ioctl should be called to /dev/sev device. The ioctl accepts command
+id and command input structure.
+
+::
+        struct sev_issue_cmd {
+                /* Command ID */
+                __u32 cmd;
+
+                /* Command request structure */
+                __u64 data;
+
+                /* firmware error code on failure (see psp-sev.h) */
+                __u32 error;
+        };
+
+
 2.1 SNP_GET_REPORT
 ------------------
 
@@ -118,6 +134,17 @@ be updated with the expected value.
 
 See GHCB specification for further detail on how to parse the certificate blob.
 
+2.4 SNP_PLATFORM_STATUS
+-----------------------
+:Technology: sev-snp
+:Type: hypervisor ioctl cmd
+:Parameters (in): struct sev_data_snp_platform_status
+:Returns (out): 0 on success, -negative on error
+
+The SNP_PLATFORM_STATUS command is used to query the SNP platform status. The
+status includes API major, minor version and more. See the SEV-SNP
+specification for further details.
+
 3. SEV-SNP CPUID Enforcement
 ============================
 
drivers/crypto/ccp/sev-dev.c +45 -0

57 lines not on this page · raw message

include/uapi/linux/psp-sev.h +1 -0
index bed65a891223..ffd60e8b0a31 100644
@@ -28,6 +28,7 @@ enum {
 	SEV_PEK_CERT_IMPORT,
 	SEV_GET_ID,	/* This command is deprecated, use SEV_GET_ID2 */
 	SEV_GET_ID2,
+	SNP_PLATFORM_STATUS,
 
 	SEV_MAX,
 };
Signature · 1 line
2.25.1
author maintainer
#200 lore raw

PATCH v6 17/49 crypto: ccp: Add the SNP_{SET,GET}_EXT_CONFIG command

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

The SEV-SNP firmware provides the SNP_CONFIG command used to set the system-wide configuration value for SNP guests. The information includes the TCB version string to be reported in guest attestation reports.

Version 2 of the GHCB specification adds an NAE (SNP extended guest request) that a guest can use to query the reports that include additional certificates.

In both cases, userspace provided additional data is included in the attestation reports. The userspace will use the SNP_SET_EXT_CONFIG command to give the certificate blob and the reported TCB version string at once. Note that the specification defines certificate blob with a specific GUID format; the userspace is responsible for building the proper certificate blob. The ioctl treats it an opaque blob.

While it is not defined in the spec, but let's add SNP_GET_EXT_CONFIG command that can be used to obtain the data programmed through the SNP_SET_EXT_CONFIG.

---
 Documentation/virt/coco/sevguest.rst |  27 +++++++
 drivers/crypto/ccp/sev-dev.c         | 115 +++++++++++++++++++++++++++
 drivers/crypto/ccp/sev-dev.h         |   3 +
 include/uapi/linux/psp-sev.h         |  17 ++++
 4 files changed, 162 insertions(+)
Documentation/virt/coco/sevguest.rst +27 -0
index 11ea67c944df..3014de47e4ce 100644
@@ -145,6 +145,33 @@ The SNP_PLATFORM_STATUS command is used to query the SNP platform status. The
 status includes API major, minor version and more. See the SEV-SNP
 specification for further details.
 
+2.5 SNP_SET_EXT_CONFIG
+----------------------
+:Technology: sev-snp
+:Type: hypervisor ioctl cmd
+:Parameters (in): struct sev_data_snp_ext_config
+:Returns (out): 0 on success, -negative on error
+
+The SNP_SET_EXT_CONFIG is used to set the system-wide configuration such as
+reported TCB version in the attestation report. The command is similar to
+SNP_CONFIG command defined in the SEV-SNP spec. The main difference is the
+command also accepts an additional certificate blob defined in the GHCB
+specification.
+
+If the certs_address is zero, then previous certificate blob will deleted.
+For more information on the certificate blob layout, see the GHCB spec
+(extended guest request message).
+
+2.6 SNP_GET_EXT_CONFIG
+----------------------
+:Technology: sev-snp
+:Type: hypervisor ioctl cmd
+:Parameters (in): struct sev_data_snp_ext_config
+:Returns (out): 0 on success, -negative on error
+
+The SNP_SET_EXT_CONFIG is used to query the system-wide configuration set
+through the SNP_SET_EXT_CONFIG.
+
 3. SEV-SNP CPUID Enforcement
 ============================
 
drivers/crypto/ccp/sev-dev.c +115 -0

133 lines not on this page · raw message

drivers/crypto/ccp/sev-dev.h +3 -0
index fe5d7a3ebace..d2fe1706311a 100644
@@ -66,6 +66,9 @@ struct sev_device {
 
 	bool snp_inited;
 	struct snp_host_map snp_host_map[MAX_SNP_HOST_MAP_BUFS];
+	void *snp_certs_data;
+	u32 snp_certs_len;
+	struct sev_user_data_snp_config snp_config;
 };
 
 int sev_dev_init(struct psp_device *psp);
include/uapi/linux/psp-sev.h +17 -0
index ffd60e8b0a31..60e7a8d1a18e 100644
@@ -29,6 +29,8 @@ enum {
 	SEV_GET_ID,	/* This command is deprecated, use SEV_GET_ID2 */
 	SEV_GET_ID2,
 	SNP_PLATFORM_STATUS,
+	SNP_SET_EXT_CONFIG,
+	SNP_GET_EXT_CONFIG,
 
 	SEV_MAX,
 };
@@ -190,6 +192,21 @@ struct sev_user_data_snp_config {
 	__u8 rsvd[52];
 } __packed;
 
+/**
+ * struct sev_data_snp_ext_config - system wide configuration value for SNP.
+ *
+ * @config_address: address of the struct sev_user_data_snp_config or 0 when
+ *		reported_tcb does not need to be updated.
+ * @certs_address: address of extended guest request certificate chain or
+ *              0 when previous certificate should be removed on SNP_SET_EXT_CONFIG.
+ * @certs_len: length of the certs
+ */
+struct sev_user_data_ext_snp_config {
+	__u64 config_address;		/* In */
+	__u64 certs_address;		/* In */
+	__u32 certs_len;		/* In */
+};
+
 /**
  * struct sev_issue_cmd - SEV ioctl parameters
  *
Signature · 1 line
2.25.1
author maintainer
#207 lore raw

PATCH v6 18/49 crypto: ccp: Provide APIs to query extended attestation report

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

Version 2 of the GHCB specification defines VMGEXIT that is used to get the extended attestation report. The extended attestation report includes the certificate blobs provided through the SNP_SET_EXT_CONFIG.

The snp_guest_ext_guest_request() will be used by the hypervisor to get the extended attestation report. See the GHCB specification for more details.

---
 drivers/crypto/ccp/sev-dev.c | 43 ++++++++++++++++++++++++++++++++++++
 include/linux/psp-sev.h      | 24 ++++++++++++++++++++
 2 files changed, 67 insertions(+)
drivers/crypto/ccp/sev-dev.c +43 -0
index 97b479d5aa86..f6306b820b86 100644
@@ -25,6 +25,7 @@
 #include <linux/fs.h>
 
 #include <asm/smp.h>
+#include <asm/sev.h>
 
 #include "psp-dev.h"
 #include "sev-dev.h"
@@ -1857,6 +1858,48 @@ int snp_guest_dbg_decrypt(struct sev_data_snp_dbg *data, int *error)
 }
 EXPORT_SYMBOL_GPL(snp_guest_dbg_decrypt);
 
+int snp_guest_ext_guest_request(struct sev_data_snp_guest_request *data,
+				unsigned long vaddr, unsigned long *npages, unsigned long *fw_err)
+{
+	unsigned long expected_npages;
+	struct sev_device *sev;
+	int rc;
+
+	if (!psp_master || !psp_master->sev_data)
+		return -ENODEV;
+
+	sev = psp_master->sev_data;
+
+	if (!sev->snp_inited)
+		return -EINVAL;
+
+	/*
+	 * Check if there is enough space to copy the certificate chain. Otherwise
+	 * return ERROR code defined in the GHCB specification.
+	 */
+	expected_npages = sev->snp_certs_len >> PAGE_SHIFT;
+	if (*npages < expected_npages) {
+		*npages = expected_npages;
+		*fw_err = SNP_GUEST_REQ_INVALID_LEN;
+		return -EINVAL;
+	}
+
+	rc = sev_do_cmd(SEV_CMD_SNP_GUEST_REQUEST, data, (int *)&fw_err);
+	if (rc)
+		return rc;
+
+	/* Copy the certificate blob */
+	if (sev->snp_certs_data) {
+		*npages = expected_npages;
+		memcpy((void *)vaddr, sev->snp_certs_data, *npages << PAGE_SHIFT);
+	} else {
+		*npages = 0;
+	}
+
+	return rc;
+}
+EXPORT_SYMBOL_GPL(snp_guest_ext_guest_request);
+
 static void sev_exit(struct kref *ref)
 {
 	misc_deregister(&misc_dev->misc);
include/linux/psp-sev.h +24 -0
index a3bb792bb842..cd37ccd1fa1f 100644
@@ -945,6 +945,23 @@ void *psp_copy_user_blob(u64 uaddr, u32 len);
 void *snp_alloc_firmware_page(gfp_t mask);
 void snp_free_firmware_page(void *addr);
 
+/**
+ * snp_guest_ext_guest_request - perform the SNP extended guest request command
+ *  defined in the GHCB specification.
+ *
+ * @data: the input guest request structure
+ * @vaddr: address where the certificate blob need to be copied.
+ * @npages: number of pages for the certificate blob.
+ *    If the specified page count is less than the certificate blob size, then the
+ *    required page count is returned with error code defined in the GHCB spec.
+ *    If the specified page count is more than the certificate blob size, then
+ *    page count is updated to reflect the amount of valid data copied in the
+ *    vaddr.
+ */
+int snp_guest_ext_guest_request(struct sev_data_snp_guest_request *data,
+				unsigned long vaddr, unsigned long *npages,
+				unsigned long *error);
+
 #else	/* !CONFIG_CRYPTO_DEV_SP_PSP */
 
 static inline int
@@ -992,6 +1009,13 @@ static inline void *snp_alloc_firmware_page(gfp_t mask)
 
 static inline void snp_free_firmware_page(void *addr) { }
 
+static inline int snp_guest_ext_guest_request(struct sev_data_snp_guest_request *data,
+					      unsigned long vaddr, unsigned long *n,
+					      unsigned long *error)
+{
+	return -ENODEV;
+}
+
 #endif	/* CONFIG_CRYPTO_DEV_SP_PSP */
 
 #endif	/* __PSP_SEV_H__ */
Signature · 1 line
2.25.1
author maintainer
#210 lore raw

PATCH v6 19/49 KVM: SVM: Add support to handle AP reset MSR protocol

In reply to Ashish Kalra

From: Tom Lendacky <[email protected]>

Add support for AP Reset Hold being invoked using the GHCB MSR protocol, available in version 2 of the GHCB specification.

---
 arch/x86/include/asm/sev-common.h |  2 ++
 arch/x86/kvm/svm/sev.c            | 56 ++++++++++++++++++++++++++-----
 arch/x86/kvm/svm/svm.h            |  1 +
 3 files changed, 51 insertions(+), 8 deletions(-)
arch/x86/include/asm/sev-common.h +2 -0
index b8357d6ecd47..e15548d88f2a 100644
@@ -56,6 +56,8 @@
 /* AP Reset Hold */
 #define GHCB_MSR_AP_RESET_HOLD_REQ	0x006
 #define GHCB_MSR_AP_RESET_HOLD_RESP	0x007
+#define GHCB_MSR_AP_RESET_HOLD_RESULT_POS	12
+#define GHCB_MSR_AP_RESET_HOLD_RESULT_MASK	GENMASK_ULL(51, 0)
 
 /* GHCB GPA Register */
 #define GHCB_MSR_REG_GPA_REQ		0x012
arch/x86/kvm/svm/sev.c +48 -8

85 lines not on this page · raw message

arch/x86/kvm/svm/svm.h +1 -0
index bb9ec9139af3..9f7eb1f18893 100644
@@ -186,6 +186,7 @@ struct vcpu_sev_es_state {
 	struct ghcb *ghcb;
 	struct kvm_host_map ghcb_map;
 	bool received_first_sipi;
+	unsigned int ap_reset_hold_type;
 
 	/* SEV-ES scratch area support */
 	void *ghcb_sa;
Signature · 1 line
2.25.1
author maintainer
#211 lore raw

PATCH v6 20/49 KVM: SVM: Provide the Hypervisor Feature support VMGEXIT

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

Version 2 of the GHCB specification introduced advertisement of features that are supported by the Hypervisor.

Now that KVM supports version 2 of the GHCB specification, bump the maximum supported protocol version.

---
 arch/x86/include/asm/sev-common.h |  2 ++
 arch/x86/kvm/svm/sev.c            | 14 ++++++++++++++
 arch/x86/kvm/svm/svm.h            |  3 ++-
 3 files changed, 18 insertions(+), 1 deletion(-)
arch/x86/include/asm/sev-common.h +2 -0
index e15548d88f2a..539de6b93420 100644
@@ -101,6 +101,8 @@ enum psc_op {
 /* GHCB Hypervisor Feature Request/Response */
 #define GHCB_MSR_HV_FT_REQ		0x080
 #define GHCB_MSR_HV_FT_RESP		0x081
+#define GHCB_MSR_HV_FT_POS		12
+#define GHCB_MSR_HV_FT_MASK		GENMASK_ULL(51, 0)
 #define GHCB_MSR_HV_FT_RESP_VAL(v)			\
 	/* GHCBData[63:12] */				\
 	(((u64)(v) & GENMASK_ULL(63, 12)) >> 12)
arch/x86/kvm/svm/sev.c +14 -0
index a1318236acd2..b49c370d5ae9 100644
@@ -2480,6 +2480,7 @@ static int sev_es_validate_vmgexit(struct vcpu_svm *svm)
 	case SVM_VMGEXIT_AP_HLT_LOOP:
 	case SVM_VMGEXIT_AP_JUMP_TABLE:
 	case SVM_VMGEXIT_UNSUPPORTED_EVENT:
+	case SVM_VMGEXIT_HV_FEATURES:
 		break;
 	default:
 		reason = GHCB_ERR_INVALID_EVENT;
@@ -2746,6 +2747,13 @@ static int sev_handle_vmgexit_msr_protocol(struct vcpu_svm *svm)
 				  GHCB_MSR_INFO_MASK,
 				  GHCB_MSR_INFO_POS);
 		break;
+	case GHCB_MSR_HV_FT_REQ: {
+		set_ghcb_msr_bits(svm, GHCB_HV_FT_SUPPORTED,
+				  GHCB_MSR_HV_FT_MASK, GHCB_MSR_HV_FT_POS);
+		set_ghcb_msr_bits(svm, GHCB_MSR_HV_FT_RESP,
+				  GHCB_MSR_INFO_MASK, GHCB_MSR_INFO_POS);
+		break;
+	}
 	case GHCB_MSR_TERM_REQ: {
 		u64 reason_set, reason_code;
 
@@ -2871,6 +2879,12 @@ int sev_handle_vmgexit(struct kvm_vcpu *vcpu)
 		ret = 1;
 		break;
 	}
+	case SVM_VMGEXIT_HV_FEATURES: {
+		ghcb_set_sw_exit_info_2(ghcb, GHCB_HV_FT_SUPPORTED);
+
+		ret = 1;
+		break;
+	}
 	case SVM_VMGEXIT_UNSUPPORTED_EVENT:
 		vcpu_unimpl(vcpu,
 			    "vmgexit: unsupported event - exit_info_1=%#llx, exit_info_2=%#llx\n",
arch/x86/kvm/svm/svm.h +2 -1
index 9f7eb1f18893..1f4a8bd09c9e 100644
@@ -629,9 +629,10 @@ unsigned long avic_vcpu_get_apicv_inhibit_reasons(struct kvm_vcpu *vcpu);
 
 /* sev.c */
 
-#define GHCB_VERSION_MAX	1ULL
+#define GHCB_VERSION_MAX	2ULL
 #define GHCB_VERSION_MIN	1ULL
 
+#define GHCB_HV_FT_SUPPORTED	0
 
 extern unsigned int max_sev_asid;
 
Signature · 1 line
2.25.1
author maintainer
#212 lore raw

PATCH v6 21/49 KVM: SVM: Make AVIC backing, VMSA and VMCB memory allocation SNP safe

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

Implement a workaround for an SNP erratum where the CPU will incorrectly signal an RMP violation #PF if a hugepage (2mb or 1gb) collides with the RMP entry of a VMCB, VMSA or AVIC backing page.

When SEV-SNP is globally enabled, the CPU marks the VMCB, VMSA, and AVIC
backing pages as "in-use" in the RMP after a successful VMRUN. This
is done for _all_ VMs, not just SNP-Active VMs.

If the hypervisor accesses an in-use page through a writable translation, the CPU will throw an RMP violation #PF. On early SNP hardware, if an in-use page is 2mb aligned and software accesses any part of the associated 2mb region with a hupage, the CPU will incorrectly treat the entire 2mb region as in-use and signal a spurious RMP violation #PF.

The recommended is to not use the hugepage for the VMCB, VMSA or AVIC backing page. Add a generic allocator that will ensure that the page returns is not hugepage (2mb or 1gb) and is safe to be used when SEV-SNP is enabled.

---
 arch/x86/include/asm/kvm-x86-ops.h |  1 +
 arch/x86/include/asm/kvm_host.h    |  2 ++
 arch/x86/kvm/lapic.c               |  5 ++++-
 arch/x86/kvm/svm/sev.c             | 35 ++++++++++++++++++++++++++++++
 arch/x86/kvm/svm/svm.c             | 16 ++++++++++++--
 arch/x86/kvm/svm/svm.h             |  1 +
 6 files changed, 57 insertions(+), 3 deletions(-)
arch/x86/include/asm/kvm-x86-ops.h +1 -0
index da47f60a4650..a66292dae698 100644
@@ -128,6 +128,7 @@ KVM_X86_OP(msr_filter_changed)
 KVM_X86_OP(complete_emulated_msr)
 KVM_X86_OP(vcpu_deliver_sipi_vector)
 KVM_X86_OP_OPTIONAL_RET0(vcpu_get_apicv_inhibit_reasons);
+KVM_X86_OP(alloc_apic_backing_page)
 
 #undef KVM_X86_OP
 #undef KVM_X86_OP_OPTIONAL
arch/x86/include/asm/kvm_host.h +2 -0
index c24a72ddc93b..0205e2944067 100644
@@ -1512,6 +1512,8 @@ struct kvm_x86_ops {
 	 * Returns vCPU specific APICv inhibit reasons
 	 */
 	unsigned long (*vcpu_get_apicv_inhibit_reasons)(struct kvm_vcpu *vcpu);
+
+	void *(*alloc_apic_backing_page)(struct kvm_vcpu *vcpu);
 };
 
 struct kvm_x86_nested_ops {
arch/x86/kvm/lapic.c +4 -1
index 66b0eb0bda94..7c7fc6c4a7f9 100644
@@ -2506,7 +2506,10 @@ int kvm_create_lapic(struct kvm_vcpu *vcpu, int timer_advance_ns)
 
 	vcpu->arch.apic = apic;
 
-	apic->regs = (void *)get_zeroed_page(GFP_KERNEL_ACCOUNT);
+	if (kvm_x86_ops.alloc_apic_backing_page)
+		apic->regs = static_call(kvm_x86_alloc_apic_backing_page)(vcpu);
+	else
+		apic->regs = (void *)get_zeroed_page(GFP_KERNEL_ACCOUNT);
 	if (!apic->regs) {
 		printk(KERN_ERR "malloc apic regs error for vcpu %x\n",
 		       vcpu->vcpu_id);
arch/x86/kvm/svm/sev.c +35 -0
index b49c370d5ae9..93365996bd59 100644
@@ -3030,3 +3030,38 @@ void sev_vcpu_deliver_sipi_vector(struct kvm_vcpu *vcpu, u8 vector)
 		break;
 	}
 }
+
+struct page *snp_safe_alloc_page(struct kvm_vcpu *vcpu)
+{
+	unsigned long pfn;
+	struct page *p;
+
+	if (!cpu_feature_enabled(X86_FEATURE_SEV_SNP))
+		return alloc_page(GFP_KERNEL_ACCOUNT | __GFP_ZERO);
+
+	/*
+	 * Allocate an SNP safe page to workaround the SNP erratum where
+	 * the CPU will incorrectly signal an RMP violation  #PF if a
+	 * hugepage (2mb or 1gb) collides with the RMP entry of VMCB, VMSA
+	 * or AVIC backing page. The recommeded workaround is to not use the
+	 * hugepage.
+	 *
+	 * Allocate one extra page, use a page which is not 2mb aligned
+	 * and free the other.
+	 */
+	p = alloc_pages(GFP_KERNEL_ACCOUNT | __GFP_ZERO, 1);
+	if (!p)
+		return NULL;
+
+	split_page(p, 1);
+
+	pfn = page_to_pfn(p);
+	if (IS_ALIGNED(__pfn_to_phys(pfn), PMD_SIZE)) {
+		pfn++;
+		__free_page(p);
+	} else {
+		__free_page(pfn_to_page(pfn + 1));
+	}
+
+	return pfn_to_page(pfn);
+}
arch/x86/kvm/svm/svm.c +14 -2
index efc7623d0f90..b4bd64f94d3a 100644
@@ -1260,7 +1260,7 @@ static int svm_vcpu_create(struct kvm_vcpu *vcpu)
 	svm = to_svm(vcpu);
 
 	err = -ENOMEM;
-	vmcb01_page = alloc_page(GFP_KERNEL_ACCOUNT | __GFP_ZERO);
+	vmcb01_page = snp_safe_alloc_page(vcpu);
 	if (!vmcb01_page)
 		goto out;
 
@@ -1269,7 +1269,7 @@ static int svm_vcpu_create(struct kvm_vcpu *vcpu)
 		 * SEV-ES guests require a separate VMSA page used to contain
 		 * the encrypted register state of the guest.
 		 */
-		vmsa_page = alloc_page(GFP_KERNEL_ACCOUNT | __GFP_ZERO);
+		vmsa_page = snp_safe_alloc_page(vcpu);
 		if (!vmsa_page)
 			goto error_free_vmcb_page;
 
@@ -4598,6 +4598,16 @@ static int svm_vm_init(struct kvm *kvm)
 	return 0;
 }
 
+static void *svm_alloc_apic_backing_page(struct kvm_vcpu *vcpu)
+{
+	struct page *page = snp_safe_alloc_page(vcpu);
+
+	if (!page)
+		return NULL;
+
+	return page_address(page);
+}
+
 static struct kvm_x86_ops svm_x86_ops __initdata = {
 	.name = "kvm_amd",
 
@@ -4722,6 +4732,8 @@ static struct kvm_x86_ops svm_x86_ops __initdata = {
 
 	.vcpu_deliver_sipi_vector = svm_vcpu_deliver_sipi_vector,
 	.vcpu_get_apicv_inhibit_reasons = avic_vcpu_get_apicv_inhibit_reasons,
+
+	.alloc_apic_backing_page = svm_alloc_apic_backing_page,
 };
 
 /*
arch/x86/kvm/svm/svm.h +1 -0
index 1f4a8bd09c9e..9672e25a338d 100644
@@ -659,6 +659,7 @@ void sev_es_vcpu_reset(struct vcpu_svm *svm);
 void sev_vcpu_deliver_sipi_vector(struct kvm_vcpu *vcpu, u8 vector);
 void sev_es_prepare_switch_to_guest(struct sev_es_save_area *hostsa);
 void sev_es_unmap_ghcb(struct vcpu_svm *svm);
+struct page *snp_safe_alloc_page(struct kvm_vcpu *vcpu);
 
 /* vmenter.S */
 
Signature · 1 line
2.25.1
author maintainer
#214 lore raw

PATCH v6 22/49 KVM: SVM: Add initial SEV-SNP support

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

The next generation of SEV is called SEV-SNP (Secure Nested Paging). SEV-SNP builds upon existing SEV and SEV-ES functionality while adding new hardware based security protection. SEV-SNP adds strong memory encryption integrity protection to help prevent malicious hypervisor-based attacks such as data replay, memory re-mapping, and more, to create an isolated execution environment.

The SNP feature is added incrementally, the later patches adds a new module parameters that can be used to enabled SEV-SNP in the KVM.

---
 arch/x86/kvm/svm/sev.c | 10 +++++++++-
 arch/x86/kvm/svm/svm.h |  8 ++++++++
 2 files changed, 17 insertions(+), 1 deletion(-)
arch/x86/kvm/svm/sev.c +9 -1
index 93365996bd59..dc1f69a28aa7 100644
@@ -56,6 +56,9 @@ module_param_named(sev_es, sev_es_enabled, bool, 0444);
 #define sev_es_enabled false
 #endif /* CONFIG_KVM_AMD_SEV */
 
+/* enable/disable SEV-SNP support */
+static bool sev_snp_enabled;
+
 #define AP_RESET_HOLD_NONE		0
 #define AP_RESET_HOLD_NAE_EVENT		1
 #define AP_RESET_HOLD_MSR_PROTO		2
@@ -2120,6 +2123,7 @@ void __init sev_hardware_setup(void)
 {
 #ifdef CONFIG_KVM_AMD_SEV
 	unsigned int eax, ebx, ecx, edx, sev_asid_count, sev_es_asid_count;
+	bool sev_snp_supported = false;
 	bool sev_es_supported = false;
 	bool sev_supported = false;
 
@@ -2190,12 +2194,16 @@ void __init sev_hardware_setup(void)
 	if (misc_cg_set_capacity(MISC_CG_RES_SEV_ES, sev_es_asid_count))
 		goto out;
 
-	pr_info("SEV-ES supported: %u ASIDs\n", sev_es_asid_count);
 	sev_es_supported = true;
+	sev_snp_supported = sev_snp_enabled && cpu_feature_enabled(X86_FEATURE_SEV_SNP);
+
+	pr_info("SEV-ES %ssupported: %u ASIDs\n",
+		sev_snp_supported ? "and SEV-SNP " : "", sev_es_asid_count);
 
 out:
 	sev_enabled = sev_supported;
 	sev_es_enabled = sev_es_supported;
+	sev_snp_enabled = sev_snp_supported;
 #endif
 }
 
arch/x86/kvm/svm/svm.h +8 -0
index 9672e25a338d..edecc5066517 100644
@@ -75,6 +75,7 @@ enum {
 struct kvm_sev_info {
 	bool active;		/* SEV enabled guest */
 	bool es_active;		/* SEV-ES enabled guest */
+	bool snp_active;	/* SEV-SNP enabled guest */
 	unsigned int asid;	/* ASID used for this guest */
 	unsigned int handle;	/* SEV firmware handle */
 	int fd;			/* SEV device fd */
@@ -314,6 +315,13 @@ static __always_inline bool sev_es_guest(struct kvm *kvm)
 #endif
 }
 
+static inline bool sev_snp_guest(struct kvm *kvm)
+{
+	struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
+
+	return sev_es_guest(kvm) && sev->snp_active;
+}
+
 static inline void vmcb_mark_all_dirty(struct vmcb *vmcb)
 {
 	vmcb->control.clean = 0;
Signature · 1 line
2.25.1
author maintainer
#215 lore raw

PATCH v6 23/49 KVM: SVM: Add KVM_SNP_INIT command

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

The KVM_SNP_INIT command is used by the hypervisor to initialize the SEV-SNP platform context. In a typical workflow, this command should be the first command issued. When creating SEV-SNP guest, the VMM must use this command instead of the KVM_SEV_INIT or KVM_SEV_ES_INIT.

The flags value must be zero, it will be extended in future SNP support to communicate the optional features (such as restricted INT injection etc).

---
 .../virt/kvm/x86/amd-memory-encryption.rst    | 27 ++++++++++++
 arch/x86/include/asm/svm.h                    |  1 +
 arch/x86/kvm/svm/sev.c                        | 44 ++++++++++++++++++-
 arch/x86/kvm/svm/svm.h                        |  4 ++
 include/uapi/linux/kvm.h                      | 13 ++++++
 5 files changed, 87 insertions(+), 2 deletions(-)
Documentation/virt/kvm/x86/amd-memory-encryption.rst +27 -0
index 2d307811978c..903023f524af 100644
@@ -435,6 +435,33 @@ issued by the hypervisor to make the guest ready for execution.
 
 Returns: 0 on success, -negative on error
 
+18. KVM_SNP_INIT
+----------------
+
+The KVM_SNP_INIT command can be used by the hypervisor to initialize SEV-SNP
+context. In a typical workflow, this command should be the first command issued.
+
+Parameters (in/out): struct kvm_snp_init
+
+Returns: 0 on success, -negative on error
+
+::
+
+        struct kvm_snp_init {
+                __u64 flags;
+        };
+
+The flags bitmap is defined as::
+
+   /* enable the restricted injection */
+   #define KVM_SEV_SNP_RESTRICTED_INJET   (1<<0)
+
+   /* enable the restricted injection timer */
+   #define KVM_SEV_SNP_RESTRICTED_TIMER_INJET   (1<<1)
+
+If the specified flags is not supported then return -EOPNOTSUPP, and the supported
+flags are returned.
+
 References
 ==========
 
arch/x86/include/asm/svm.h +1 -0
index 1b07fba11704..284a8113227e 100644
@@ -263,6 +263,7 @@ enum avic_ipi_failure_cause {
 #define AVIC_HPA_MASK	~((0xFFFULL << 52) | 0xFFF)
 #define VMCB_AVIC_APIC_BAR_MASK		0xFFFFFFFFFF000ULL
 
+#define SVM_SEV_FEAT_SNP_ACTIVE		BIT(0)
 
 struct vmcb_seg {
 	u16 selector;
arch/x86/kvm/svm/sev.c +42 -2

79 lines not on this page · raw message

arch/x86/kvm/svm/svm.h +4 -0
index edecc5066517..2f45589ee596 100644
@@ -72,6 +72,9 @@ enum {
 /* TPR and CR2 are always written before VMRUN */
 #define VMCB_ALWAYS_DIRTY_MASK	((1U << VMCB_INTR) | (1U << VMCB_CR2))
 
+/* Supported init feature flags */
+#define SEV_SNP_SUPPORTED_FLAGS		0x0
+
 struct kvm_sev_info {
 	bool active;		/* SEV enabled guest */
 	bool es_active;		/* SEV-ES enabled guest */
@@ -87,6 +90,7 @@ struct kvm_sev_info {
 	struct list_head mirror_entry; /* Use as a list entry of mirrors */
 	struct misc_cg *misc_cg; /* For misc cgroup accounting */
 	atomic_t migration_in_progress;
+	u64 snp_init_flags;
 };
 
 struct kvm_svm {
include/uapi/linux/kvm.h +13 -0
index 68ce07185f03..0f912cefc544 100644
@@ -1810,6 +1810,9 @@ enum sev_cmd_id {
 	/* Guest Migration Extension */
 	KVM_SEV_SEND_CANCEL,
 
+	/* SNP specific commands */
+	KVM_SEV_SNP_INIT,
+
 	KVM_SEV_NR_MAX,
 };
 
@@ -1906,6 +1909,16 @@ struct kvm_sev_receive_update_data {
 	__u32 trans_len;
 };
 
+/* enable the restricted injection */
+#define KVM_SEV_SNP_RESTRICTED_INJET   (1 << 0)
+
+/* enable the restricted injection timer */
+#define KVM_SEV_SNP_RESTRICTED_TIMER_INJET   (1 << 1)
+
+struct kvm_snp_init {
+	__u64 flags;
+};
+
 #define KVM_DEV_ASSIGN_ENABLE_IOMMU	(1 << 0)
 #define KVM_DEV_ASSIGN_PCI_2_3		(1 << 1)
 #define KVM_DEV_ASSIGN_MASK_INTX	(1 << 2)
Signature · 1 line
2.25.1
author maintainer
#216 lore raw

PATCH v6 24/49 KVM: SVM: Add KVM_SEV_SNP_LAUNCH_START command

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

KVM_SEV_SNP_LAUNCH_START begins the launch process for an SEV-SNP guest. The command initializes a cryptographic digest context used to construct the measurement of the guest. If the guest is expected to be migrated, the command also binds a migration agent (MA) to the guest.

For more information see the SEV-SNP specification.

---
 .../virt/kvm/x86/amd-memory-encryption.rst    |  24 ++++
 arch/x86/kvm/svm/sev.c                        | 115 +++++++++++++++++-
 arch/x86/kvm/svm/svm.h                        |   1 +
 include/uapi/linux/kvm.h                      |  10 ++
 4 files changed, 147 insertions(+), 3 deletions(-)
Documentation/virt/kvm/x86/amd-memory-encryption.rst +24 -0
index 903023f524af..878711f2dca6 100644
@@ -462,6 +462,30 @@ The flags bitmap is defined as::
 If the specified flags is not supported then return -EOPNOTSUPP, and the supported
 flags are returned.
 
+19. KVM_SNP_LAUNCH_START
+------------------------
+
+The KVM_SNP_LAUNCH_START command is used for creating the memory encryption
+context for the SEV-SNP guest. To create the encryption context, user must
+provide a guest policy, migration agent (if any) and guest OS visible
+workarounds value as defined SEV-SNP specification.
+
+Parameters (in): struct  kvm_snp_launch_start
+
+Returns: 0 on success, -negative on error
+
+::
+
+        struct kvm_sev_snp_launch_start {
+                __u64 policy;           /* Guest policy to use. */
+                __u64 ma_uaddr;         /* userspace address of migration agent */
+                __u8 ma_en;             /* 1 if the migtation agent is enabled */
+                __u8 imi_en;            /* set IMI to 1. */
+                __u8 gosvw[16];         /* guest OS visible workarounds */
+        };
+
+See the SEV-SNP specification for further detail on the launch input.
+
 References
 ==========
 
arch/x86/kvm/svm/sev.c +112 -3

161 lines not on this page · raw message

arch/x86/kvm/svm/svm.h +1 -0
index 2f45589ee596..71c011af098e 100644
@@ -91,6 +91,7 @@ struct kvm_sev_info {
 	struct misc_cg *misc_cg; /* For misc cgroup accounting */
 	atomic_t migration_in_progress;
 	u64 snp_init_flags;
+	void *snp_context;      /* SNP guest context page */
 };
 
 struct kvm_svm {
include/uapi/linux/kvm.h +10 -0
index 0f912cefc544..0cb119d66ae5 100644
@@ -1812,6 +1812,7 @@ enum sev_cmd_id {
 
 	/* SNP specific commands */
 	KVM_SEV_SNP_INIT,
+	KVM_SEV_SNP_LAUNCH_START,
 
 	KVM_SEV_NR_MAX,
 };
@@ -1919,6 +1920,15 @@ struct kvm_snp_init {
 	__u64 flags;
 };
 
+struct kvm_sev_snp_launch_start {
+	__u64 policy;
+	__u64 ma_uaddr;
+	__u8 ma_en;
+	__u8 imi_en;
+	__u8 gosvw[16];
+	__u8 pad[6];
+};
+
 #define KVM_DEV_ASSIGN_ENABLE_IOMMU	(1 << 0)
 #define KVM_DEV_ASSIGN_PCI_2_3		(1 << 1)
 #define KVM_DEV_ASSIGN_MASK_INTX	(1 << 2)
Signature · 1 line
2.25.1
author maintainer
#220 lore raw

PATCH v6 25/49 KVM: SVM: Disallow registering memory range from HugeTLB for SNP guest

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

While creating the VM, userspace call the KVM_MEMORY_ENCRYPT_REG_REGION ioctl to register the memory regions for the guest. This registered memory region is typically used as a guest RAM. Later, the guest may issue the page state change (PSC) request that will require splitting the large page into smaller page. If the memory is allocated from the HugeTLB then hypervisor will not be able to split it.

Do not allow registering the memory range backed by the HugeTLB until the hypervisor support is added to handle the case.

---
 arch/x86/kvm/svm/sev.c | 37 +++++++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)
arch/x86/kvm/svm/sev.c +37 -0
index 9e6fc7a94ed7..41b83aa6b5f4 100644
@@ -17,6 +17,7 @@
 #include <linux/misc_cgroup.h>
 #include <linux/processor.h>
 #include <linux/trace_events.h>
+#include <linux/hugetlb.h>
 
 #include <asm/pkru.h>
 #include <asm/trapnr.h>
@@ -2007,6 +2008,35 @@ int sev_mem_enc_ioctl(struct kvm *kvm, void __user *argp)
 	return r;
 }
 
+static bool is_range_hugetlb(struct kvm *kvm, struct kvm_enc_region *range)
+{
+	struct vm_area_struct *vma;
+	u64 start, end;
+	bool ret = true;
+
+	start = range->addr;
+	end = start + range->size;
+
+	mmap_read_lock(kvm->mm);
+
+	do {
+		vma = find_vma_intersection(kvm->mm, start, end);
+		if (!vma)
+			goto unlock;
+
+		if (is_vm_hugetlb_page(vma))
+			goto unlock;
+
+		start = vma->vm_end;
+	} while (end > vma->vm_end);
+
+	ret = false;
+
+unlock:
+	mmap_read_unlock(kvm->mm);
+	return ret;
+}
+
 int sev_mem_enc_register_region(struct kvm *kvm,
 				struct kvm_enc_region *range)
 {
@@ -2024,6 +2054,13 @@ int sev_mem_enc_register_region(struct kvm *kvm,
 	if (range->addr > ULONG_MAX || range->size > ULONG_MAX)
 		return -EINVAL;
 
+	/*
+	 * SEV-SNP does not support the backing pages from the HugeTLB. Verify
+	 * that the registered memory range is not from the HugeTLB.
+	 */
+	if (sev_snp_guest(kvm) && is_range_hugetlb(kvm, range))
+		return -EINVAL;
+
 	region = kzalloc(sizeof(*region), GFP_KERNEL_ACCOUNT);
 	if (!region)
 		return -ENOMEM;
Signature · 1 line
2.25.1
author maintainer
#222 lore raw

PATCH v6 26/49 KVM: SVM: Add KVM_SEV_SNP_LAUNCH_UPDATE command

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

The KVM_SEV_SNP_LAUNCH_UPDATE command can be used to insert data into the guest's memory. The data is encrypted with the cryptographic context created with the KVM_SEV_SNP_LAUNCH_START.

In addition to the inserting data, it can insert a two special pages into the guests memory: the secrets page and the CPUID page.

While terminating the guest, reclaim the guest pages added in the RMP table. If the reclaim fails, then the page is no longer safe to be released back to the system and leak them.

For more information see the SEV-SNP specification.

---
 .../virt/kvm/x86/amd-memory-encryption.rst    |  29 +++
 arch/x86/kvm/svm/sev.c                        | 187 ++++++++++++++++++
 include/uapi/linux/kvm.h                      |  19 ++
 3 files changed, 235 insertions(+)
Documentation/virt/kvm/x86/amd-memory-encryption.rst +29 -0
index 878711f2dca6..62abd5c1f72b 100644
@@ -486,6 +486,35 @@ Returns: 0 on success, -negative on error
 
 See the SEV-SNP specification for further detail on the launch input.
 
+20. KVM_SNP_LAUNCH_UPDATE
+-------------------------
+
+The KVM_SNP_LAUNCH_UPDATE is used for encrypting a memory region. It also
+calculates a measurement of the memory contents. The measurement is a signature
+of the memory contents that can be sent to the guest owner as an attestation
+that the memory was encrypted correctly by the firmware.
+
+Parameters (in): struct  kvm_snp_launch_update
+
+Returns: 0 on success, -negative on error
+
+::
+
+        struct kvm_sev_snp_launch_update {
+                __u64 start_gfn;        /* Guest page number to start from. */
+                __u64 uaddr;            /* userspace address need to be encrypted */
+                __u32 len;              /* length of memory region */
+                __u8 imi_page;          /* 1 if memory is part of the IMI */
+                __u8 page_type;         /* page type */
+                __u8 vmpl3_perms;       /* VMPL3 permission mask */
+                __u8 vmpl2_perms;       /* VMPL2 permission mask */
+                __u8 vmpl1_perms;       /* VMPL1 permission mask */
+        };
+
+See the SEV-SNP spec for further details on how to build the VMPL permission
+mask and page type.
+
+
 References
 ==========
 
arch/x86/kvm/svm/sev.c +187 -0

217 lines not on this page · raw message

include/uapi/linux/kvm.h +19 -0
index 0cb119d66ae5..9b36b07414ea 100644
@@ -1813,6 +1813,7 @@ enum sev_cmd_id {
 	/* SNP specific commands */
 	KVM_SEV_SNP_INIT,
 	KVM_SEV_SNP_LAUNCH_START,
+	KVM_SEV_SNP_LAUNCH_UPDATE,
 
 	KVM_SEV_NR_MAX,
 };
@@ -1929,6 +1930,24 @@ struct kvm_sev_snp_launch_start {
 	__u8 pad[6];
 };
 
+#define KVM_SEV_SNP_PAGE_TYPE_NORMAL		0x1
+#define KVM_SEV_SNP_PAGE_TYPE_VMSA		0x2
+#define KVM_SEV_SNP_PAGE_TYPE_ZERO		0x3
+#define KVM_SEV_SNP_PAGE_TYPE_UNMEASURED	0x4
+#define KVM_SEV_SNP_PAGE_TYPE_SECRETS		0x5
+#define KVM_SEV_SNP_PAGE_TYPE_CPUID		0x6
+
+struct kvm_sev_snp_launch_update {
+	__u64 start_gfn;
+	__u64 uaddr;
+	__u32 len;
+	__u8 imi_page;
+	__u8 page_type;
+	__u8 vmpl3_perms;
+	__u8 vmpl2_perms;
+	__u8 vmpl1_perms;
+};
+
 #define KVM_DEV_ASSIGN_ENABLE_IOMMU	(1 << 0)
 #define KVM_DEV_ASSIGN_PCI_2_3		(1 << 1)
 #define KVM_DEV_ASSIGN_MASK_INTX	(1 << 2)
Signature · 1 line
2.25.1
author maintainer
#228 lore raw

PATCH v6 27/49 KVM: SVM: Mark the private vma unmerable for SEV-SNP guests

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

When SEV-SNP is enabled, the guest private pages are added in the RMP table; while adding the pages, the rmp_make_private() unmaps the pages from the direct map. If KSM attempts to access those unmapped pages then it will trigger #PF (page-not-present).

Encrypted guest pages cannot be shared between the process, so an userspace should not mark the region mergeable but to be safe, mark the process vma unmerable before adding the pages in the RMP table.

---
 arch/x86/kvm/svm/sev.c | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)
arch/x86/kvm/svm/sev.c +32 -0
index b5f0707d7ed6..a9461d352eda 100644
@@ -19,11 +19,13 @@
 #include <linux/trace_events.h>
 #include <linux/hugetlb.h>
 #include <linux/sev.h>
+#include <linux/ksm.h>
 
 #include <asm/pkru.h>
 #include <asm/trapnr.h>
 #include <asm/fpu/xcr.h>
 #include <asm/sev.h>
+#include <asm/mman.h>
 
 #include "x86.h"
 #include "svm.h"
@@ -1965,6 +1967,30 @@ static bool is_hva_registered(struct kvm *kvm, hva_t hva, size_t len)
 	return false;
 }
 
+static int snp_mark_unmergable(struct kvm *kvm, u64 start, u64 size)
+{
+	struct vm_area_struct *vma;
+	u64 end = start + size;
+	int ret;
+
+	do {
+		vma = find_vma_intersection(kvm->mm, start, end);
+		if (!vma) {
+			ret = -EINVAL;
+			break;
+		}
+
+		ret = ksm_madvise(vma, vma->vm_start, vma->vm_end,
+				  MADV_UNMERGEABLE, &vma->vm_flags);
+		if (ret)
+			break;
+
+		start = vma->vm_end;
+	} while (end > vma->vm_end);
+
+	return ret;
+}
+
 static int snp_launch_update(struct kvm *kvm, struct kvm_sev_cmd *argp)
 {
 	struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
@@ -1989,6 +2015,12 @@ static int snp_launch_update(struct kvm *kvm, struct kvm_sev_cmd *argp)
 	if (!is_hva_registered(kvm, params.uaddr, params.len))
 		return -EINVAL;
 
+	mmap_write_lock(kvm->mm);
+	ret = snp_mark_unmergable(kvm, params.uaddr, params.len);
+	mmap_write_unlock(kvm->mm);
+	if (ret)
+		return -EFAULT;
+
 	/*
 	 * The userspace memory is already locked so technically we don't
 	 * need to lock it again. Later part of the function needs to know
Signature · 1 line
2.25.1
author maintainer
#231 lore raw

PATCH v6 28/49 KVM: SVM: Add KVM_SEV_SNP_LAUNCH_FINISH command

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

The KVM_SEV_SNP_LAUNCH_FINISH finalize the cryptographic digest and stores it as the measurement of the guest at launch.

While finalizing the launch flow, it also issues the LAUNCH_UPDATE command to encrypt the VMSA pages.

If its an SNP guest, then VMSA was added in the RMP entry as a guest owned page and also removed from the kernel direct map so flush it later after it is transitioned back to hypervisor state and restored in the direct map.

---
 .../virt/kvm/x86/amd-memory-encryption.rst    |  22 ++++
 arch/x86/kvm/svm/sev.c                        | 119 ++++++++++++++++++
 include/uapi/linux/kvm.h                      |  14 +++
 3 files changed, 155 insertions(+)
Documentation/virt/kvm/x86/amd-memory-encryption.rst +22 -0
index 62abd5c1f72b..750162cff87b 100644
@@ -514,6 +514,28 @@ Returns: 0 on success, -negative on error
 See the SEV-SNP spec for further details on how to build the VMPL permission
 mask and page type.
 
+21. KVM_SNP_LAUNCH_FINISH
+-------------------------
+
+After completion of the SNP guest launch flow, the KVM_SNP_LAUNCH_FINISH command can be
+issued to make the guest ready for the execution.
+
+Parameters (in): struct kvm_sev_snp_launch_finish
+
+Returns: 0 on success, -negative on error
+
+::
+
+        struct kvm_sev_snp_launch_finish {
+                __u64 id_block_uaddr;
+                __u64 id_auth_uaddr;
+                __u8 id_block_en;
+                __u8 auth_key_en;
+                __u8 host_data[32];
+        };
+
+
+See SEV-SNP specification for further details on launch finish input parameters.
 
 References
 ==========
arch/x86/kvm/svm/sev.c +119 -0

142 lines not on this page · raw message

include/uapi/linux/kvm.h +14 -0
index 9b36b07414ea..5a4662716b6a 100644
@@ -1814,6 +1814,7 @@ enum sev_cmd_id {
 	KVM_SEV_SNP_INIT,
 	KVM_SEV_SNP_LAUNCH_START,
 	KVM_SEV_SNP_LAUNCH_UPDATE,
+	KVM_SEV_SNP_LAUNCH_FINISH,
 
 	KVM_SEV_NR_MAX,
 };
@@ -1948,6 +1949,19 @@ struct kvm_sev_snp_launch_update {
 	__u8 vmpl1_perms;
 };
 
+#define KVM_SEV_SNP_ID_BLOCK_SIZE	96
+#define KVM_SEV_SNP_ID_AUTH_SIZE	4096
+#define KVM_SEV_SNP_FINISH_DATA_SIZE	32
+
+struct kvm_sev_snp_launch_finish {
+	__u64 id_block_uaddr;
+	__u64 id_auth_uaddr;
+	__u8 id_block_en;
+	__u8 auth_key_en;
+	__u8 host_data[KVM_SEV_SNP_FINISH_DATA_SIZE];
+	__u8 pad[6];
+};
+
 #define KVM_DEV_ASSIGN_ENABLE_IOMMU	(1 << 0)
 #define KVM_DEV_ASSIGN_PCI_2_3		(1 << 1)
 #define KVM_DEV_ASSIGN_MASK_INTX	(1 << 2)
Signature · 1 line
2.25.1
author maintainer
#244 lore raw

PATCH v6 29/49 KVM: X86: Keep the NPT and RMP page level in sync

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

When running an SEV-SNP VM, the sPA used to index the RMP entry is obtained through the NPT translation (gva->gpa->spa). The NPT page level is checked against the page level programmed in the RMP entry. If the page level does not match, then it will cause a nested page fault with the RMP bit set to indicate the RMP violation.

---
 arch/x86/include/asm/kvm-x86-ops.h |  1 +
 arch/x86/include/asm/kvm_host.h    |  1 +
 arch/x86/kvm/mmu/mmu.c             |  5 ++++
 arch/x86/kvm/svm/sev.c             | 46 ++++++++++++++++++++++++++++++
 arch/x86/kvm/svm/svm.c             |  1 +
 arch/x86/kvm/svm/svm.h             |  1 +
 6 files changed, 55 insertions(+)
arch/x86/include/asm/kvm-x86-ops.h +1 -0
index a66292dae698..e0068e702692 100644
@@ -129,6 +129,7 @@ KVM_X86_OP(complete_emulated_msr)
 KVM_X86_OP(vcpu_deliver_sipi_vector)
 KVM_X86_OP_OPTIONAL_RET0(vcpu_get_apicv_inhibit_reasons);
 KVM_X86_OP(alloc_apic_backing_page)
+KVM_X86_OP_OPTIONAL(rmp_page_level_adjust)
 
 #undef KVM_X86_OP
 #undef KVM_X86_OP_OPTIONAL
arch/x86/include/asm/kvm_host.h +1 -0
index 0205e2944067..2748c69609e3 100644
@@ -1514,6 +1514,7 @@ struct kvm_x86_ops {
 	unsigned long (*vcpu_get_apicv_inhibit_reasons)(struct kvm_vcpu *vcpu);
 
 	void *(*alloc_apic_backing_page)(struct kvm_vcpu *vcpu);
+	void (*rmp_page_level_adjust)(struct kvm *kvm, kvm_pfn_t pfn, int *level);
 };
 
 struct kvm_x86_nested_ops {
arch/x86/kvm/mmu/mmu.c +5 -0
index c623019929a7..997318ecebd1 100644
@@ -43,6 +43,7 @@
 #include <linux/hash.h>
 #include <linux/kern_levels.h>
 #include <linux/kthread.h>
+#include <linux/sev.h>
 
 #include <asm/page.h>
 #include <asm/memtype.h>
@@ -2824,6 +2825,10 @@ static int host_pfn_mapping_level(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn,
 	if (unlikely(!pte))
 		return PG_LEVEL_4K;
 
+	/* Adjust the page level based on the SEV-SNP RMP page level. */
+	if (kvm_x86_ops.rmp_page_level_adjust)
+		static_call(kvm_x86_rmp_page_level_adjust)(kvm, pfn, &level);
+
 	return level;
 }
 
arch/x86/kvm/svm/sev.c +46 -0

49 lines not on this page · raw message

arch/x86/kvm/svm/svm.c +1 -0
index b4bd64f94d3a..18e2cd4d9559 100644
@@ -4734,6 +4734,7 @@ static struct kvm_x86_ops svm_x86_ops __initdata = {
 	.vcpu_get_apicv_inhibit_reasons = avic_vcpu_get_apicv_inhibit_reasons,
 
 	.alloc_apic_backing_page = svm_alloc_apic_backing_page,
+	.rmp_page_level_adjust = sev_rmp_page_level_adjust,
 };
 
 /*
arch/x86/kvm/svm/svm.h +1 -0
index 71c011af098e..7782312a1cda 100644
@@ -673,6 +673,7 @@ void sev_vcpu_deliver_sipi_vector(struct kvm_vcpu *vcpu, u8 vector);
 void sev_es_prepare_switch_to_guest(struct sev_es_save_area *hostsa);
 void sev_es_unmap_ghcb(struct vcpu_svm *svm);
 struct page *snp_safe_alloc_page(struct kvm_vcpu *vcpu);
+void sev_rmp_page_level_adjust(struct kvm *kvm, kvm_pfn_t pfn, int *level);
 
 /* vmenter.S */
 
Signature · 1 line
2.25.1
author maintainer
#246 lore raw

PATCH v6 30/49 KVM: x86/mmu: Introduce kvm_mmu_map_tdp_page() for use by TDX and SNP

In reply to Ashish Kalra

From: Sean Christopherson <[email protected]>

Introduce a helper to directly (pun intended) fault-in a TDP page without having to go through the full page fault path. This allows TDX to get the resulting pfn and also allows the RET_PF_* enums to stay in mmu.c where they belong.

---
 arch/x86/kvm/mmu.h     |  3 +++
 arch/x86/kvm/mmu/mmu.c | 51 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 54 insertions(+)
arch/x86/kvm/mmu.h +3 -0
index e6cae6f22683..c99b15e97a0a 100644
@@ -204,6 +204,9 @@ static inline int kvm_mmu_do_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa,
 	return vcpu->arch.mmu->page_fault(vcpu, &fault);
 }
 
+kvm_pfn_t kvm_mmu_map_tdp_page(struct kvm_vcpu *vcpu, gpa_t gpa,
+			       u32 error_code, int max_level);
+
 /*
  * Check if a given access (described through the I/D, W/R and U/S bits of a
  * page fault error code pfec) causes a permission fault with the given PTE
arch/x86/kvm/mmu/mmu.c +51 -0

57 lines not on this page · raw message

Signature · 1 line
2.25.1
author maintainer
#247 lore raw

PATCH v6 31/49 KVM: x86: Introduce kvm_mmu_get_tdp_walk() for SEV-SNP use

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

The SEV-SNP VMs may call the page state change VMGEXIT to add the GPA as private or shared in the RMP table. The page state change VMGEXIT will contain the RMP page level to be used in the RMP entry. If the page level between the TDP and RMP does not match then, it will result in nested-page-fault (RMP violation).

The SEV-SNP VMGEXIT handler will use the kvm_mmu_get_tdp_walk() to get the current page-level in the TDP for the given GPA and calculate a workable page level. If a GPA is mapped as a 4K-page in the TDP, but the guest requested to add the GPA as a 2M in the RMP entry then the 2M request will be broken into 4K-pages to keep the RMP and TDP page-levels in sync.

TDP SPTEs are RCU protected so need to put kvm_mmu_get_tdp_walk() in RCU read-side critical section by using walk_shadow_page_lockless_begin() and walk_lockless_shadow_page_lockless_end(). This fixes the "suspicious RCU usage" message seen with lockdep kernel build.

Signed-off by: Ashish Kalra <[email protected]>

---
 arch/x86/kvm/mmu.h     |  2 ++
 arch/x86/kvm/mmu/mmu.c | 33 +++++++++++++++++++++++++++++++++
 2 files changed, 35 insertions(+)
arch/x86/kvm/mmu.h +2 -0
index c99b15e97a0a..d55b5166389a 100644
@@ -178,6 +178,8 @@ static inline bool is_nx_huge_page_enabled(void)
 	return READ_ONCE(nx_huge_pages);
 }
 
+bool kvm_mmu_get_tdp_walk(struct kvm_vcpu *vcpu, gpa_t gpa, kvm_pfn_t *pfn, int *level);
+
 static inline int kvm_mmu_do_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa,
 					u32 err, bool prefetch)
 {
arch/x86/kvm/mmu/mmu.c +33 -0
index 569021af349a..c1ac486e096e 100644
@@ -4151,6 +4151,39 @@ kvm_pfn_t kvm_mmu_map_tdp_page(struct kvm_vcpu *vcpu, gpa_t gpa,
 }
 EXPORT_SYMBOL_GPL(kvm_mmu_map_tdp_page);
 
+bool kvm_mmu_get_tdp_walk(struct kvm_vcpu *vcpu, gpa_t gpa, kvm_pfn_t *pfn, int *level)
+{
+	u64 sptes[PT64_ROOT_MAX_LEVEL + 1];
+	int leaf, root;
+
+	walk_shadow_page_lockless_begin(vcpu);
+
+	if (is_tdp_mmu(vcpu->arch.mmu))
+		leaf = kvm_tdp_mmu_get_walk(vcpu, gpa, sptes, &root);
+	else
+		leaf = get_walk(vcpu, gpa, sptes, &root);
+
+	walk_shadow_page_lockless_end(vcpu);
+
+	if (unlikely(leaf < 0))
+		return false;
+
+	/* Check if the leaf SPTE is present */
+	if (!is_shadow_present_pte(sptes[leaf]))
+		return false;
+
+	*pfn = spte_to_pfn(sptes[leaf]);
+	if (leaf > PG_LEVEL_4K) {
+		u64 page_mask = KVM_PAGES_PER_HPAGE(leaf) - KVM_PAGES_PER_HPAGE(leaf - 1);
+		*pfn |= (gpa_to_gfn(gpa) & page_mask);
+	}
+
+	*level = leaf;
+
+	return true;
+}
+EXPORT_SYMBOL_GPL(kvm_mmu_get_tdp_walk);
+
 static void nonpaging_init_context(struct kvm_mmu *context)
 {
 	context->page_fault = nonpaging_page_fault;
Signature · 1 line
2.25.1
author maintainer
#249 lore raw

PATCH v6 32/49 KVM: x86: Define RMP page fault error bits for #NPF

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

When SEV-SNP is enabled globally, the hardware places restrictions on all memory accesses based on the RMP entry, whether the hypervisor or a VM, performs the accesses. When hardware encounters an RMP access violation during a guest access, it will cause a #VMEXIT(NPF).

See APM2 section 16.36.10 for more details.

---
 arch/x86/include/asm/kvm_host.h | 8 ++++++++
 1 file changed, 8 insertions(+)
arch/x86/include/asm/kvm_host.h +8 -0
index 2748c69609e3..49b217dc8d7e 100644
@@ -247,9 +247,13 @@ enum x86_intercept_stage;
 #define PFERR_FETCH_BIT 4
 #define PFERR_PK_BIT 5
 #define PFERR_SGX_BIT 15
+#define PFERR_GUEST_RMP_BIT 31
 #define PFERR_GUEST_FINAL_BIT 32
 #define PFERR_GUEST_PAGE_BIT 33
 #define PFERR_IMPLICIT_ACCESS_BIT 48
+#define PFERR_GUEST_ENC_BIT 34
+#define PFERR_GUEST_SIZEM_BIT 35
+#define PFERR_GUEST_VMPL_BIT 36
 
 #define PFERR_PRESENT_MASK (1U << PFERR_PRESENT_BIT)
 #define PFERR_WRITE_MASK (1U << PFERR_WRITE_BIT)
@@ -261,6 +265,10 @@ enum x86_intercept_stage;
 #define PFERR_GUEST_FINAL_MASK (1ULL << PFERR_GUEST_FINAL_BIT)
 #define PFERR_GUEST_PAGE_MASK (1ULL << PFERR_GUEST_PAGE_BIT)
 #define PFERR_IMPLICIT_ACCESS (1ULL << PFERR_IMPLICIT_ACCESS_BIT)
+#define PFERR_GUEST_RMP_MASK (1ULL << PFERR_GUEST_RMP_BIT)
+#define PFERR_GUEST_ENC_MASK (1ULL << PFERR_GUEST_ENC_BIT)
+#define PFERR_GUEST_SIZEM_MASK (1ULL << PFERR_GUEST_SIZEM_BIT)
+#define PFERR_GUEST_VMPL_MASK (1ULL << PFERR_GUEST_VMPL_BIT)
 
 #define PFERR_NESTED_GUEST_PAGE (PFERR_GUEST_PAGE_MASK |	\
 				 PFERR_WRITE_MASK |		\
Signature · 1 line
2.25.1
author maintainer
#250 lore raw

PATCH v6 33/49 KVM: x86: Update page-fault trace to log full 64-bit error code

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

The #NPT error code is a 64-bit value but the trace prints only the lower 32-bits. Some of the fault error code (e.g PFERR_GUEST_FINAL_MASK) are available in the upper 32-bits.

---
 arch/x86/kvm/trace.h | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
arch/x86/kvm/trace.h +3 -3
index e3a24b8f04be..9b9bc5468103 100644
@@ -383,12 +383,12 @@ TRACE_EVENT(kvm_inj_exception,
  * Tracepoint for page fault.
  */
 TRACE_EVENT(kvm_page_fault,
-	TP_PROTO(unsigned long fault_address, unsigned int error_code),
+	TP_PROTO(unsigned long fault_address, u64 error_code),
 	TP_ARGS(fault_address, error_code),
 
 	TP_STRUCT__entry(
 		__field(	unsigned long,	fault_address	)
-		__field(	unsigned int,	error_code	)
+		__field(	u64,		error_code	)
 	),
 
 	TP_fast_assign(
@@ -396,7 +396,7 @@ TRACE_EVENT(kvm_page_fault,
 		__entry->error_code	= error_code;
 	),
 
-	TP_printk("address %lx error_code %x",
+	TP_printk("address %lx error_code %llx",
 		  __entry->fault_address, __entry->error_code)
 );
 
Signature · 1 line
2.25.1
author maintainer
#252 lore raw

PATCH v6 34/49 KVM: SVM: Do not use long-lived GHCB map while setting scratch area

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

The setup_vmgexit_scratch() function may rely on a long-lived GHCB mapping if the GHCB shared buffer area was used for the scratch area. In preparation for eliminating the long-lived GHCB mapping, always allocate a buffer for the scratch area so it can be accessed without the GHCB mapping.

---
 arch/x86/kvm/svm/sev.c | 74 +++++++++++++++++++-----------------------
 arch/x86/kvm/svm/svm.h |  3 +-
 2 files changed, 36 insertions(+), 41 deletions(-)
arch/x86/kvm/svm/sev.c +34 -40
index 91d3d24e60d2..01ea257e17d6 100644
@@ -2820,8 +2820,7 @@ void sev_free_vcpu(struct kvm_vcpu *vcpu)
 	__free_page(virt_to_page(svm->sev_es.vmsa));
 
 skip_vmsa_free:
-	if (svm->sev_es.ghcb_sa_free)
-		kvfree(svm->sev_es.ghcb_sa);
+	kvfree(svm->sev_es.ghcb_sa);
 }
 
 static void dump_ghcb(struct vcpu_svm *svm)
@@ -2909,6 +2908,9 @@ static void sev_es_sync_from_ghcb(struct vcpu_svm *svm)
 	control->exit_info_1 = ghcb_get_sw_exit_info_1(ghcb);
 	control->exit_info_2 = ghcb_get_sw_exit_info_2(ghcb);
 
+	/* Copy the GHCB scratch area GPA */
+	svm->sev_es.ghcb_sa_gpa = ghcb_get_sw_scratch(ghcb);
+
 	/* Clear the valid entries fields */
 	memset(ghcb->save.valid_bitmap, 0, sizeof(ghcb->save.valid_bitmap));
 }
@@ -3054,23 +3056,12 @@ void sev_es_unmap_ghcb(struct vcpu_svm *svm)
 	if (!svm->sev_es.ghcb)
 		return;
 
-	if (svm->sev_es.ghcb_sa_free) {
-		/*
-		 * The scratch area lives outside the GHCB, so there is a
-		 * buffer that, depending on the operation performed, may
-		 * need to be synced, then freed.
-		 */
-		if (svm->sev_es.ghcb_sa_sync) {
-			kvm_write_guest(svm->vcpu.kvm,
-					ghcb_get_sw_scratch(svm->sev_es.ghcb),
-					svm->sev_es.ghcb_sa,
-					svm->sev_es.ghcb_sa_len);
-			svm->sev_es.ghcb_sa_sync = false;
-		}
-
-		kvfree(svm->sev_es.ghcb_sa);
-		svm->sev_es.ghcb_sa = NULL;
-		svm->sev_es.ghcb_sa_free = false;
+	 /* Sync the scratch buffer area. */
+	if (svm->sev_es.ghcb_sa_sync) {
+		kvm_write_guest(svm->vcpu.kvm,
+				ghcb_get_sw_scratch(svm->sev_es.ghcb),
+				svm->sev_es.ghcb_sa, svm->sev_es.ghcb_sa_len);
+		svm->sev_es.ghcb_sa_sync = false;
 	}
 
 	trace_kvm_vmgexit_exit(svm->vcpu.vcpu_id, svm->sev_es.ghcb);
@@ -3111,9 +3102,8 @@ static int setup_vmgexit_scratch(struct vcpu_svm *svm, bool sync, u64 len)
 	struct ghcb *ghcb = svm->sev_es.ghcb;
 	u64 ghcb_scratch_beg, ghcb_scratch_end;
 	u64 scratch_gpa_beg, scratch_gpa_end;
-	void *scratch_va;
 
-	scratch_gpa_beg = ghcb_get_sw_scratch(ghcb);
+	scratch_gpa_beg = svm->sev_es.ghcb_sa_gpa;
 	if (!scratch_gpa_beg) {
 		pr_err("vmgexit: scratch gpa not provided\n");
 		goto e_scratch;
@@ -3143,9 +3133,6 @@ static int setup_vmgexit_scratch(struct vcpu_svm *svm, bool sync, u64 len)
 			       scratch_gpa_beg, scratch_gpa_end);
 			goto e_scratch;
 		}
-
-		scratch_va = (void *)svm->sev_es.ghcb;
-		scratch_va += (scratch_gpa_beg - control->ghcb_gpa);
 	} else {
 		/*
 		 * The guest memory must be read into a kernel buffer, so
@@ -3156,29 +3143,36 @@ static int setup_vmgexit_scratch(struct vcpu_svm *svm, bool sync, u64 len)
 			       len, GHCB_SCRATCH_AREA_LIMIT);
 			goto e_scratch;
 		}
-		scratch_va = kvzalloc(len, GFP_KERNEL_ACCOUNT);
-		if (!scratch_va)
-			return -ENOMEM;
+	}
 
-		if (kvm_read_guest(svm->vcpu.kvm, scratch_gpa_beg, scratch_va, len)) {
-			/* Unable to copy scratch area from guest */
-			pr_err("vmgexit: kvm_read_guest for scratch area failed\n");
+	if (svm->sev_es.ghcb_sa_alloc_len < len) {
+		void *scratch_va = kvzalloc(len, GFP_KERNEL_ACCOUNT);
 
-			kvfree(scratch_va);
-			return -EFAULT;
-		}
+		if (!scratch_va)
+			return -ENOMEM;
 
 		/*
-		 * The scratch area is outside the GHCB. The operation will
-		 * dictate whether the buffer needs to be synced before running
-		 * the vCPU next time (i.e. a read was requested so the data
-		 * must be written back to the guest memory).
+		 * Free the old scratch area and switch to using newly
+		 * allocated.
 		 */
-		svm->sev_es.ghcb_sa_sync = sync;
-		svm->sev_es.ghcb_sa_free = true;
+		kvfree(svm->sev_es.ghcb_sa);
+
+		svm->sev_es.ghcb_sa_alloc_len = len;
+		svm->sev_es.ghcb_sa = scratch_va;
 	}
 
-	svm->sev_es.ghcb_sa = scratch_va;
+	if (kvm_read_guest(svm->vcpu.kvm, scratch_gpa_beg, svm->sev_es.ghcb_sa, len)) {
+		/* Unable to copy scratch area from guest */
+		pr_err("vmgexit: kvm_read_guest for scratch area failed\n");
+		return -EFAULT;
+	}
+
+	/*
+	 * The operation will dictate whether the buffer needs to be synced
+	 * before running the vCPU next time (i.e. a read was requested so
+	 * the data must be written back to the guest memory).
+	 */
+	svm->sev_es.ghcb_sa_sync = sync;
 	svm->sev_es.ghcb_sa_len = len;
 
 	return 0;
arch/x86/kvm/svm/svm.h +2 -1
index 7782312a1cda..bd0db4d4a61e 100644
@@ -197,8 +197,9 @@ struct vcpu_sev_es_state {
 	/* SEV-ES scratch area support */
 	void *ghcb_sa;
 	u32 ghcb_sa_len;
+	u64 ghcb_sa_gpa;
+	u32 ghcb_sa_alloc_len;
 	bool ghcb_sa_sync;
-	bool ghcb_sa_free;
 };
 
 struct vcpu_svm {
Signature · 1 line
2.25.1
author maintainer
#253 lore raw

PATCH v6 35/49 KVM: SVM: Remove the long-lived GHCB host map

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

On VMGEXIT, sev_handle_vmgexit() creates a host mapping for the GHCB GPA, and unmaps it just before VM-entry. This long-lived GHCB map is used by the VMGEXIT handler through accessors such as ghcb_{set_get}_xxx().

A long-lived GHCB map can cause issue when SEV-SNP is enabled. When SEV-SNP is enabled the mapped GPA needs to be protected against a page state change.

To eliminate the long-lived GHCB mapping, update the GHCB sync operations to explicitly map the GHCB before access and unmap it after access is complete. This requires that the setting of the GHCBs sw_exit_info_{1,2} fields be done during sev_es_sync_to_ghcb(), so create two new fields in the vcpu_svm struct to hold these values when required to be set outside of the GHCB mapping.

---
 arch/x86/kvm/svm/sev.c | 131 ++++++++++++++++++++++++++---------------
 arch/x86/kvm/svm/svm.c |  12 ++--
 arch/x86/kvm/svm/svm.h |  24 +++++++-
 3 files changed, 111 insertions(+), 56 deletions(-)
arch/x86/kvm/svm/sev.c +83 -48
index 01ea257e17d6..c70f3f7e06a8 100644
@@ -2823,15 +2823,40 @@ void sev_free_vcpu(struct kvm_vcpu *vcpu)
 	kvfree(svm->sev_es.ghcb_sa);
 }
 
+static inline int svm_map_ghcb(struct vcpu_svm *svm, struct kvm_host_map *map)
+{
+	struct vmcb_control_area *control = &svm->vmcb->control;
+	u64 gfn = gpa_to_gfn(control->ghcb_gpa);
+
+	if (kvm_vcpu_map(&svm->vcpu, gfn, map)) {
+		/* Unable to map GHCB from guest */
+		pr_err("error mapping GHCB GFN [%#llx] from guest\n", gfn);
+		return -EFAULT;
+	}
+
+	return 0;
+}
+
+static inline void svm_unmap_ghcb(struct vcpu_svm *svm, struct kvm_host_map *map)
+{
+	kvm_vcpu_unmap(&svm->vcpu, map, true);
+}
+
 static void dump_ghcb(struct vcpu_svm *svm)
 {
-	struct ghcb *ghcb = svm->sev_es.ghcb;
+	struct kvm_host_map map;
 	unsigned int nbits;
+	struct ghcb *ghcb;
+
+	if (svm_map_ghcb(svm, &map))
+		return;
+
+	ghcb = map.hva;
 
 	/* Re-use the dump_invalid_vmcb module parameter */
 	if (!dump_invalid_vmcb) {
 		pr_warn_ratelimited("set kvm_amd.dump_invalid_vmcb=1 to dump internal KVM state.\n");
-		return;
+		goto e_unmap;
 	}
 
 	nbits = sizeof(ghcb->save.valid_bitmap) * 8;
@@ -2846,12 +2871,21 @@ static void dump_ghcb(struct vcpu_svm *svm)
 	pr_err("%-20s%016llx is_valid: %u\n", "sw_scratch",
 	       ghcb->save.sw_scratch, ghcb_sw_scratch_is_valid(ghcb));
 	pr_err("%-20s%*pb\n", "valid_bitmap", nbits, ghcb->save.valid_bitmap);
+
+e_unmap:
+	svm_unmap_ghcb(svm, &map);
 }
 
-static void sev_es_sync_to_ghcb(struct vcpu_svm *svm)
+static bool sev_es_sync_to_ghcb(struct vcpu_svm *svm)
 {
 	struct kvm_vcpu *vcpu = &svm->vcpu;
-	struct ghcb *ghcb = svm->sev_es.ghcb;
+	struct kvm_host_map map;
+	struct ghcb *ghcb;
+
+	if (svm_map_ghcb(svm, &map))
+		return false;
+
+	ghcb = map.hva;
 
 	/*
 	 * The GHCB protocol so far allows for the following data
@@ -2865,13 +2899,24 @@ static void sev_es_sync_to_ghcb(struct vcpu_svm *svm)
 	ghcb_set_rbx(ghcb, vcpu->arch.regs[VCPU_REGS_RBX]);
 	ghcb_set_rcx(ghcb, vcpu->arch.regs[VCPU_REGS_RCX]);
 	ghcb_set_rdx(ghcb, vcpu->arch.regs[VCPU_REGS_RDX]);
+
+	/*
+	 * Copy the return values from the exit_info_{1,2}.
+	 */
+	ghcb_set_sw_exit_info_1(ghcb, svm->sev_es.ghcb_sw_exit_info_1);
+	ghcb_set_sw_exit_info_2(ghcb, svm->sev_es.ghcb_sw_exit_info_2);
+
+	trace_kvm_vmgexit_exit(svm->vcpu.vcpu_id, ghcb);
+
+	svm_unmap_ghcb(svm, &map);
+
+	return true;
 }
 
-static void sev_es_sync_from_ghcb(struct vcpu_svm *svm)
+static void sev_es_sync_from_ghcb(struct vcpu_svm *svm, struct ghcb *ghcb)
 {
 	struct vmcb_control_area *control = &svm->vmcb->control;
 	struct kvm_vcpu *vcpu = &svm->vcpu;
-	struct ghcb *ghcb = svm->sev_es.ghcb;
 	u64 exit_code;
 
 	/*
@@ -2915,20 +2960,25 @@ static void sev_es_sync_from_ghcb(struct vcpu_svm *svm)
 	memset(ghcb->save.valid_bitmap, 0, sizeof(ghcb->save.valid_bitmap));
 }
 
-static int sev_es_validate_vmgexit(struct vcpu_svm *svm)
+static int sev_es_validate_vmgexit(struct vcpu_svm *svm, u64 *exit_code)
 {
-	struct kvm_vcpu *vcpu;
+	struct kvm_vcpu *vcpu = &svm->vcpu;
+	struct kvm_host_map map;
 	struct ghcb *ghcb;
-	u64 exit_code;
 	u64 reason;
 
-	ghcb = svm->sev_es.ghcb;
+	if (svm_map_ghcb(svm, &map))
+		return -EFAULT;
+
+	ghcb = map.hva;
+
+	trace_kvm_vmgexit_enter(vcpu->vcpu_id, ghcb);
 
 	/*
 	 * Retrieve the exit code now even though it may not be marked valid
 	 * as it could help with debugging.
 	 */
-	exit_code = ghcb_get_sw_exit_code(ghcb);
+	*exit_code = ghcb_get_sw_exit_code(ghcb);
 
 	/* Only GHCB Usage code 0 is supported */
 	if (ghcb->ghcb_usage) {
@@ -3021,6 +3071,9 @@ static int sev_es_validate_vmgexit(struct vcpu_svm *svm)
 		goto vmgexit_err;
 	}
 
+	sev_es_sync_from_ghcb(svm, ghcb);
+
+	svm_unmap_ghcb(svm, &map);
 	return 0;
 
 vmgexit_err:
@@ -3031,10 +3084,10 @@ static int sev_es_validate_vmgexit(struct vcpu_svm *svm)
 			    ghcb->ghcb_usage);
 	} else if (reason == GHCB_ERR_INVALID_EVENT) {
 		vcpu_unimpl(vcpu, "vmgexit: exit code %#llx is not valid\n",
-			    exit_code);
+			    *exit_code);
 	} else {
 		vcpu_unimpl(vcpu, "vmgexit: exit code %#llx input is not valid\n",
-			    exit_code);
+			    *exit_code);
 		dump_ghcb(svm);
 	}
 
@@ -3044,6 +3097,8 @@ static int sev_es_validate_vmgexit(struct vcpu_svm *svm)
 	ghcb_set_sw_exit_info_1(ghcb, 2);
 	ghcb_set_sw_exit_info_2(ghcb, reason);
 
+	svm_unmap_ghcb(svm, &map);
+
 	/* Resume the guest to "return" the error code. */
 	return 1;
 }
@@ -3053,23 +3108,20 @@ void sev_es_unmap_ghcb(struct vcpu_svm *svm)
 	/* Clear any indication that the vCPU is in a type of AP Reset Hold */
 	svm->sev_es.ap_reset_hold_type = AP_RESET_HOLD_NONE;
 
-	if (!svm->sev_es.ghcb)
+	if (!svm->sev_es.ghcb_in_use)
 		return;
 
 	 /* Sync the scratch buffer area. */
 	if (svm->sev_es.ghcb_sa_sync) {
 		kvm_write_guest(svm->vcpu.kvm,
-				ghcb_get_sw_scratch(svm->sev_es.ghcb),
+				svm->sev_es.ghcb_sa_gpa,
 				svm->sev_es.ghcb_sa, svm->sev_es.ghcb_sa_len);
 		svm->sev_es.ghcb_sa_sync = false;
 	}
 
-	trace_kvm_vmgexit_exit(svm->vcpu.vcpu_id, svm->sev_es.ghcb);
-
 	sev_es_sync_to_ghcb(svm);
 
-	kvm_vcpu_unmap(&svm->vcpu, &svm->sev_es.ghcb_map, true);
-	svm->sev_es.ghcb = NULL;
+	svm->sev_es.ghcb_in_use = false;
 }
 
 void pre_sev_run(struct vcpu_svm *svm, int cpu)
@@ -3099,7 +3151,6 @@ void pre_sev_run(struct vcpu_svm *svm, int cpu)
 static int setup_vmgexit_scratch(struct vcpu_svm *svm, bool sync, u64 len)
 {
 	struct vmcb_control_area *control = &svm->vmcb->control;
-	struct ghcb *ghcb = svm->sev_es.ghcb;
 	u64 ghcb_scratch_beg, ghcb_scratch_end;
 	u64 scratch_gpa_beg, scratch_gpa_end;
 
@@ -3178,8 +3229,8 @@ static int setup_vmgexit_scratch(struct vcpu_svm *svm, bool sync, u64 len)
 	return 0;
 
 e_scratch:
-	ghcb_set_sw_exit_info_1(ghcb, 2);
-	ghcb_set_sw_exit_info_2(ghcb, GHCB_ERR_INVALID_SCRATCH_AREA);
+	svm_set_ghcb_sw_exit_info_1(&svm->vcpu, 2);
+	svm_set_ghcb_sw_exit_info_2(&svm->vcpu, GHCB_ERR_INVALID_SCRATCH_AREA);
 
 	return 1;
 }
@@ -3316,7 +3367,6 @@ int sev_handle_vmgexit(struct kvm_vcpu *vcpu)
 	struct vcpu_svm *svm = to_svm(vcpu);
 	struct vmcb_control_area *control = &svm->vmcb->control;
 	u64 ghcb_gpa, exit_code;
-	struct ghcb *ghcb;
 	int ret;
 
 	/* Validate the GHCB */
@@ -3331,29 +3381,14 @@ int sev_handle_vmgexit(struct kvm_vcpu *vcpu)
 		return 1;
 	}
 
-	if (kvm_vcpu_map(vcpu, ghcb_gpa >> PAGE_SHIFT, &svm->sev_es.ghcb_map)) {
-		/* Unable to map GHCB from guest */
-		vcpu_unimpl(vcpu, "vmgexit: error mapping GHCB [%#llx] from guest\n",
-			    ghcb_gpa);
-
-		/* Without a GHCB, just return right back to the guest */
-		return 1;
-	}
-
-	svm->sev_es.ghcb = svm->sev_es.ghcb_map.hva;
-	ghcb = svm->sev_es.ghcb_map.hva;
-
-	trace_kvm_vmgexit_enter(vcpu->vcpu_id, ghcb);
-
-	exit_code = ghcb_get_sw_exit_code(ghcb);
-
-	ret = sev_es_validate_vmgexit(svm);
+	ret = sev_es_validate_vmgexit(svm, &exit_code);
 	if (ret)
 		return ret;
 
-	sev_es_sync_from_ghcb(svm);
-	ghcb_set_sw_exit_info_1(ghcb, 0);
-	ghcb_set_sw_exit_info_2(ghcb, 0);
+	svm->sev_es.ghcb_in_use = true;
+
+	svm_set_ghcb_sw_exit_info_1(vcpu, 0);
+	svm_set_ghcb_sw_exit_info_2(vcpu, 0);
 
 	switch (exit_code) {
 	case SVM_VMGEXIT_MMIO_READ:
@@ -3393,20 +3428,20 @@ int sev_handle_vmgexit(struct kvm_vcpu *vcpu)
 			break;
 		case 1:
 			/* Get AP jump table address */
-			ghcb_set_sw_exit_info_2(ghcb, sev->ap_jump_table);
+			svm_set_ghcb_sw_exit_info_2(vcpu, sev->ap_jump_table);
 			break;
 		default:
 			pr_err("svm: vmgexit: unsupported AP jump table request - exit_info_1=%#llx\n",
 			       control->exit_info_1);
-			ghcb_set_sw_exit_info_1(ghcb, 2);
-			ghcb_set_sw_exit_info_2(ghcb, GHCB_ERR_INVALID_INPUT);
+			svm_set_ghcb_sw_exit_info_1(vcpu, 2);
+			svm_set_ghcb_sw_exit_info_2(vcpu, GHCB_ERR_INVALID_INPUT);
 		}
 
 		ret = 1;
 		break;
 	}
 	case SVM_VMGEXIT_HV_FEATURES: {
-		ghcb_set_sw_exit_info_2(ghcb, GHCB_HV_FT_SUPPORTED);
+		svm_set_ghcb_sw_exit_info_2(vcpu, GHCB_HV_FT_SUPPORTED);
 
 		ret = 1;
 		break;
@@ -3537,7 +3572,7 @@ void sev_vcpu_deliver_sipi_vector(struct kvm_vcpu *vcpu, u8 vector)
 		 * Return from an AP Reset Hold VMGEXIT, where the guest will
 		 * set the CS and RIP. Set SW_EXIT_INFO_2 to a non-zero value.
 		 */
-		ghcb_set_sw_exit_info_2(svm->sev_es.ghcb, 1);
+		svm_set_ghcb_sw_exit_info_2(vcpu, 1);
 		break;
 	case AP_RESET_HOLD_MSR_PROTO:
 		/*
arch/x86/kvm/svm/svm.c +6 -6
index 18e2cd4d9559..b24e0171cbf2 100644
@@ -2720,14 +2720,14 @@ static int svm_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 static int svm_complete_emulated_msr(struct kvm_vcpu *vcpu, int err)
 {
 	struct vcpu_svm *svm = to_svm(vcpu);
-	if (!err || !sev_es_guest(vcpu->kvm) || WARN_ON_ONCE(!svm->sev_es.ghcb))
+	if (!err || !sev_es_guest(vcpu->kvm) || WARN_ON_ONCE(!svm->sev_es.ghcb_in_use))
 		return kvm_complete_insn_gp(vcpu, err);
 
-	ghcb_set_sw_exit_info_1(svm->sev_es.ghcb, 1);
-	ghcb_set_sw_exit_info_2(svm->sev_es.ghcb,
-				X86_TRAP_GP |
-				SVM_EVTINJ_TYPE_EXEPT |
-				SVM_EVTINJ_VALID);
+	svm_set_ghcb_sw_exit_info_1(vcpu, 1);
+	svm_set_ghcb_sw_exit_info_2(vcpu,
+				    X86_TRAP_GP |
+				    SVM_EVTINJ_TYPE_EXEPT |
+				    SVM_EVTINJ_VALID);
 	return 1;
 }
 
arch/x86/kvm/svm/svm.h +22 -2
index bd0db4d4a61e..c80352c9c0d6 100644
@@ -189,8 +189,7 @@ struct svm_nested_state {
 struct vcpu_sev_es_state {
 	/* SEV-ES support */
 	struct sev_es_save_area *vmsa;
-	struct ghcb *ghcb;
-	struct kvm_host_map ghcb_map;
+	bool ghcb_in_use;
 	bool received_first_sipi;
 	unsigned int ap_reset_hold_type;
 
@@ -200,6 +199,13 @@ struct vcpu_sev_es_state {
 	u64 ghcb_sa_gpa;
 	u32 ghcb_sa_alloc_len;
 	bool ghcb_sa_sync;
+
+	/*
+	 * SEV-ES support to hold the sw_exit_info return values to be
+	 * sync'ed to the GHCB when mapped.
+	 */
+	u64 ghcb_sw_exit_info_1;
+	u64 ghcb_sw_exit_info_2;
 };
 
 struct vcpu_svm {
@@ -614,6 +620,20 @@ void nested_sync_control_from_vmcb02(struct vcpu_svm *svm);
 void nested_vmcb02_compute_g_pat(struct vcpu_svm *svm);
 void svm_switch_vmcb(struct vcpu_svm *svm, struct kvm_vmcb_info *target_vmcb);
 
+static inline void svm_set_ghcb_sw_exit_info_1(struct kvm_vcpu *vcpu, u64 val)
+{
+	struct vcpu_svm *svm = to_svm(vcpu);
+
+	svm->sev_es.ghcb_sw_exit_info_1 = val;
+}
+
+static inline void svm_set_ghcb_sw_exit_info_2(struct kvm_vcpu *vcpu, u64 val)
+{
+	struct vcpu_svm *svm = to_svm(vcpu);
+
+	svm->sev_es.ghcb_sw_exit_info_2 = val;
+}
+
 extern struct kvm_x86_nested_ops svm_nested_ops;
 
 /* avic.c */
Signature · 1 line
2.25.1
author maintainer
#260 lore raw

PATCH v6 36/49 KVM: SVM: Add support to handle GHCB GPA register VMGEXIT

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

SEV-SNP guests are required to perform a GHCB GPA registration. Before using a GHCB GPA for a vCPU the first time, a guest must register the vCPU GHCB GPA. If hypervisor can work with the guest requested GPA then it must respond back with the same GPA otherwise return -1.

On VMEXIT, Verify that GHCB GPA matches with the registered value. If a mismatch is detected then abort the guest.

---
 arch/x86/include/asm/sev-common.h |  8 ++++++++
 arch/x86/kvm/svm/sev.c            | 27 +++++++++++++++++++++++++++
 arch/x86/kvm/svm/svm.h            |  7 +++++++
 3 files changed, 42 insertions(+)
arch/x86/include/asm/sev-common.h +8 -0
index 539de6b93420..0a9055cdfae2 100644
@@ -59,6 +59,14 @@
 #define GHCB_MSR_AP_RESET_HOLD_RESULT_POS	12
 #define GHCB_MSR_AP_RESET_HOLD_RESULT_MASK	GENMASK_ULL(51, 0)
 
+/* Preferred GHCB GPA Request */
+#define GHCB_MSR_PREF_GPA_REQ		0x010
+#define GHCB_MSR_GPA_VALUE_POS		12
+#define GHCB_MSR_GPA_VALUE_MASK		GENMASK_ULL(51, 0)
+
+#define GHCB_MSR_PREF_GPA_RESP		0x011
+#define GHCB_MSR_PREF_GPA_NONE		0xfffffffffffff
+
 /* GHCB GPA Register */
 #define GHCB_MSR_REG_GPA_REQ		0x012
 #define GHCB_MSR_REG_GPA_REQ_VAL(v)			\
arch/x86/kvm/svm/sev.c +27 -0
index c70f3f7e06a8..6de48130e414 100644
@@ -3331,6 +3331,27 @@ static int sev_handle_vmgexit_msr_protocol(struct vcpu_svm *svm)
 				  GHCB_MSR_INFO_MASK, GHCB_MSR_INFO_POS);
 		break;
 	}
+	case GHCB_MSR_PREF_GPA_REQ: {
+		set_ghcb_msr_bits(svm, GHCB_MSR_PREF_GPA_NONE, GHCB_MSR_GPA_VALUE_MASK,
+				  GHCB_MSR_GPA_VALUE_POS);
+		set_ghcb_msr_bits(svm, GHCB_MSR_PREF_GPA_RESP, GHCB_MSR_INFO_MASK,
+				  GHCB_MSR_INFO_POS);
+		break;
+	}
+	case GHCB_MSR_REG_GPA_REQ: {
+		u64 gfn;
+
+		gfn = get_ghcb_msr_bits(svm, GHCB_MSR_GPA_VALUE_MASK,
+					GHCB_MSR_GPA_VALUE_POS);
+
+		svm->sev_es.ghcb_registered_gpa = gfn_to_gpa(gfn);
+
+		set_ghcb_msr_bits(svm, gfn, GHCB_MSR_GPA_VALUE_MASK,
+				  GHCB_MSR_GPA_VALUE_POS);
+		set_ghcb_msr_bits(svm, GHCB_MSR_REG_GPA_RESP, GHCB_MSR_INFO_MASK,
+				  GHCB_MSR_INFO_POS);
+		break;
+	}
 	case GHCB_MSR_TERM_REQ: {
 		u64 reason_set, reason_code;
 
@@ -3381,6 +3402,12 @@ int sev_handle_vmgexit(struct kvm_vcpu *vcpu)
 		return 1;
 	}
 
+	/* SEV-SNP guest requires that the GHCB GPA must be registered */
+	if (sev_snp_guest(svm->vcpu.kvm) && !ghcb_gpa_is_registered(svm, ghcb_gpa)) {
+		vcpu_unimpl(&svm->vcpu, "vmgexit: GHCB GPA [%#llx] is not registered.\n", ghcb_gpa);
+		return -EINVAL;
+	}
+
 	ret = sev_es_validate_vmgexit(svm, &exit_code);
 	if (ret)
 		return ret;
arch/x86/kvm/svm/svm.h +7 -0
index c80352c9c0d6..54ff56cb6125 100644
@@ -206,6 +206,8 @@ struct vcpu_sev_es_state {
 	 */
 	u64 ghcb_sw_exit_info_1;
 	u64 ghcb_sw_exit_info_2;
+
+	u64 ghcb_registered_gpa;
 };
 
 struct vcpu_svm {
@@ -334,6 +336,11 @@ static inline bool sev_snp_guest(struct kvm *kvm)
 	return sev_es_guest(kvm) && sev->snp_active;
 }
 
+static inline bool ghcb_gpa_is_registered(struct vcpu_svm *svm, u64 val)
+{
+	return svm->sev_es.ghcb_registered_gpa == val;
+}
+
 static inline void vmcb_mark_all_dirty(struct vmcb *vmcb)
 {
 	vmcb->control.clean = 0;
Signature · 1 line
2.25.1
author maintainer
#262 lore raw

PATCH v6 37/49 KVM: SVM: Add support to handle MSR based Page State Change VMGEXIT

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

SEV-SNP VMs can ask the hypervisor to change the page state in the RMP table to be private or shared using the Page State Change MSR protocol as defined in the GHCB specification.

Before changing the page state in the RMP entry, lookup the page in the NPT to make sure that there is a valid mapping for it. If the mapping exist then try to find a workable page level between the NPT and RMP for the page. If the page is not mapped in the NPT, then create a fault such that it gets mapped before we change the page state in the RMP entry.

---
 arch/x86/include/asm/sev-common.h |   9 ++
 arch/x86/kvm/svm/sev.c            | 197 ++++++++++++++++++++++++++++++
 arch/x86/kvm/trace.h              |  34 ++++++
 arch/x86/kvm/x86.c                |   1 +
 4 files changed, 241 insertions(+)
arch/x86/include/asm/sev-common.h +9 -0
index 0a9055cdfae2..ee38f7408470 100644
@@ -93,6 +93,10 @@ enum psc_op {
 };
 
 #define GHCB_MSR_PSC_REQ		0x014
+#define GHCB_MSR_PSC_GFN_POS		12
+#define GHCB_MSR_PSC_GFN_MASK		GENMASK_ULL(39, 0)
+#define GHCB_MSR_PSC_OP_POS		52
+#define GHCB_MSR_PSC_OP_MASK		0xf
 #define GHCB_MSR_PSC_REQ_GFN(gfn, op)			\
 	/* GHCBData[55:52] */				\
 	(((u64)((op) & 0xf) << 52) |			\
@@ -102,6 +106,11 @@ enum psc_op {
 	GHCB_MSR_PSC_REQ)
 
 #define GHCB_MSR_PSC_RESP		0x015
+#define GHCB_MSR_PSC_ERROR_POS		32
+#define GHCB_MSR_PSC_ERROR_MASK		GENMASK_ULL(31, 0)
+#define GHCB_MSR_PSC_ERROR		GENMASK_ULL(31, 0)
+#define GHCB_MSR_PSC_RSVD_POS		12
+#define GHCB_MSR_PSC_RSVD_MASK		GENMASK_ULL(19, 0)
 #define GHCB_MSR_PSC_RESP_VAL(val)			\
 	/* GHCBData[63:32] */				\
 	(((u64)(val) & GENMASK_ULL(63, 32)) >> 32)
arch/x86/kvm/svm/sev.c +197 -0

215 lines not on this page · raw message

arch/x86/kvm/trace.h +34 -0
index 9b9bc5468103..79801e50344a 100644
@@ -7,6 +7,7 @@
 #include <asm/svm.h>
 #include <asm/clocksource.h>
 #include <asm/pvclock-abi.h>
+#include <asm/sev-common.h>
 
 #undef TRACE_SYSTEM
 #define TRACE_SYSTEM kvm
@@ -1755,6 +1756,39 @@ TRACE_EVENT(kvm_vmgexit_msr_protocol_exit,
 		  __entry->vcpu_id, __entry->ghcb_gpa, __entry->result)
 );
 
+/*
+ * Tracepoint for the SEV-SNP page state change processing
+ */
+#define psc_operation					\
+	{SNP_PAGE_STATE_PRIVATE, "private"},		\
+	{SNP_PAGE_STATE_SHARED,  "shared"}		\
+
+TRACE_EVENT(kvm_snp_psc,
+	TP_PROTO(unsigned int vcpu_id, u64 pfn, u64 gpa, u8 op, int level),
+	TP_ARGS(vcpu_id, pfn, gpa, op, level),
+
+	TP_STRUCT__entry(
+		__field(int, vcpu_id)
+		__field(u64, pfn)
+		__field(u64, gpa)
+		__field(u8, op)
+		__field(int, level)
+	),
+
+	TP_fast_assign(
+		__entry->vcpu_id = vcpu_id;
+		__entry->pfn = pfn;
+		__entry->gpa = gpa;
+		__entry->op = op;
+		__entry->level = level;
+	),
+
+	TP_printk("vcpu %u, pfn %llx, gpa %llx, op %s, level %d",
+		  __entry->vcpu_id, __entry->pfn, __entry->gpa,
+		  __print_symbolic(__entry->op, psc_operation),
+		  __entry->level)
+);
+
 #endif /* _TRACE_KVM_H */
 
 #undef TRACE_INCLUDE_PATH
arch/x86/kvm/x86.c +1 -0
index 50fff5202e7e..4a1d16231e30 100644
@@ -13066,6 +13066,7 @@ EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_vmgexit_enter);
 EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_vmgexit_exit);
 EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_vmgexit_msr_protocol_enter);
 EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_vmgexit_msr_protocol_exit);
+EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_snp_psc);
 
 static int __init kvm_x86_init(void)
 {
Signature · 1 line
2.25.1
author maintainer
#272 lore raw

PATCH v6 38/49 KVM: SVM: Add support to handle Page State Change VMGEXIT

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

SEV-SNP VMs can ask the hypervisor to change the page state in the RMP table to be private or shared using the Page State Change NAE event as defined in the GHCB specification version 2.

---
 arch/x86/include/asm/sev-common.h |  7 +++
 arch/x86/kvm/svm/sev.c            | 79 +++++++++++++++++++++++++++++--
 2 files changed, 81 insertions(+), 5 deletions(-)
arch/x86/include/asm/sev-common.h +7 -0
index ee38f7408470..1b111cde8c82 100644
@@ -130,6 +130,13 @@ enum psc_op {
 /* SNP Page State Change NAE event */
 #define VMGEXIT_PSC_MAX_ENTRY		253
 
+/* The page state change hdr structure in not valid */
+#define PSC_INVALID_HDR			1
+/* The hdr.cur_entry or hdr.end_entry is not valid */
+#define PSC_INVALID_ENTRY		2
+/* Page state change encountered undefined error */
+#define PSC_UNDEF_ERR			3
+
 struct psc_hdr {
 	u16 cur_entry;
 	u16 end_entry;
arch/x86/kvm/svm/sev.c +74 -5

126 lines not on this page · raw message

Signature · 1 line
2.25.1
author maintainer
#273 lore raw

PATCH v6 39/49 KVM: SVM: Introduce ops for the post gfn map and unmap

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

When SEV-SNP is enabled in the guest VM, the guest memory pages can either be a private or shared. A write from the hypervisor goes through the RMP checks. If hardware sees that hypervisor is attempting to write to a guest private page, then it triggers an RMP violation #PF.

To avoid the RMP violation with GHCB pages, added new post_{map,unmap}_gfn functions to verify if its safe to map GHCB pages. Uses a spinlock to protect against the page state change for existing mapped pages.

Need to add generic post_{map,unmap}_gfn() ops that can be used to verify that its safe to map a given guest page in the hypervisor.

This patch will need to be revisited later after consensus is reached on how to manage guest private memory as probably UPM private memslots will be able to handle this page state change more gracefully.

Signed-off by: Ashish Kalra <[email protected]>

---
 arch/x86/include/asm/kvm-x86-ops.h |  1 +
 arch/x86/include/asm/kvm_host.h    |  3 ++
 arch/x86/kvm/svm/sev.c             | 48 ++++++++++++++++++++++++++++--
 arch/x86/kvm/svm/svm.c             |  3 ++
 arch/x86/kvm/svm/svm.h             | 11 +++++++
 5 files changed, 64 insertions(+), 2 deletions(-)
arch/x86/include/asm/kvm-x86-ops.h +1 -0
index e0068e702692..2dd2bc0cf4c3 100644
@@ -130,6 +130,7 @@ KVM_X86_OP(vcpu_deliver_sipi_vector)
 KVM_X86_OP_OPTIONAL_RET0(vcpu_get_apicv_inhibit_reasons);
 KVM_X86_OP(alloc_apic_backing_page)
 KVM_X86_OP_OPTIONAL(rmp_page_level_adjust)
+KVM_X86_OP(update_protected_guest_state)
 
 #undef KVM_X86_OP
 #undef KVM_X86_OP_OPTIONAL
arch/x86/include/asm/kvm_host.h +3 -0
index 49b217dc8d7e..8abc0e724f5c 100644
@@ -1522,7 +1522,10 @@ struct kvm_x86_ops {
 	unsigned long (*vcpu_get_apicv_inhibit_reasons)(struct kvm_vcpu *vcpu);
 
 	void *(*alloc_apic_backing_page)(struct kvm_vcpu *vcpu);
+
 	void (*rmp_page_level_adjust)(struct kvm *kvm, kvm_pfn_t pfn, int *level);
+
+	int (*update_protected_guest_state)(struct kvm_vcpu *vcpu);
 };
 
 struct kvm_x86_nested_ops {
arch/x86/kvm/svm/sev.c +46 -2

86 lines not on this page · raw message

arch/x86/kvm/svm/svm.c +3 -0
index b24e0171cbf2..1c8e035ba011 100644
@@ -4734,7 +4734,10 @@ static struct kvm_x86_ops svm_x86_ops __initdata = {
 	.vcpu_get_apicv_inhibit_reasons = avic_vcpu_get_apicv_inhibit_reasons,
 
 	.alloc_apic_backing_page = svm_alloc_apic_backing_page,
+
 	.rmp_page_level_adjust = sev_rmp_page_level_adjust,
+
+	.update_protected_guest_state = sev_snp_update_protected_guest_state,
 };
 
 /*
arch/x86/kvm/svm/svm.h +11 -0
index 54ff56cb6125..3fd95193ed8d 100644
@@ -79,19 +79,25 @@ struct kvm_sev_info {
 	bool active;		/* SEV enabled guest */
 	bool es_active;		/* SEV-ES enabled guest */
 	bool snp_active;	/* SEV-SNP enabled guest */
+
 	unsigned int asid;	/* ASID used for this guest */
 	unsigned int handle;	/* SEV firmware handle */
 	int fd;			/* SEV device fd */
+
 	unsigned long pages_locked; /* Number of pages locked */
 	struct list_head regions_list;  /* List of registered regions */
+
 	u64 ap_jump_table;	/* SEV-ES AP Jump Table address */
+
 	struct kvm *enc_context_owner; /* Owner of copied encryption context */
 	struct list_head mirror_vms; /* List of VMs mirroring */
 	struct list_head mirror_entry; /* Use as a list entry of mirrors */
 	struct misc_cg *misc_cg; /* For misc cgroup accounting */
 	atomic_t migration_in_progress;
+
 	u64 snp_init_flags;
 	void *snp_context;      /* SNP guest context page */
+	spinlock_t psc_lock;
 };
 
 struct kvm_svm {
@@ -702,6 +708,11 @@ void sev_es_prepare_switch_to_guest(struct sev_es_save_area *hostsa);
 void sev_es_unmap_ghcb(struct vcpu_svm *svm);
 struct page *snp_safe_alloc_page(struct kvm_vcpu *vcpu);
 void sev_rmp_page_level_adjust(struct kvm *kvm, kvm_pfn_t pfn, int *level);
+int sev_post_map_gfn(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn);
+void sev_post_unmap_gfn(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn);
+void handle_rmp_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u64 error_code);
+void sev_snp_init_protected_guest_state(struct kvm_vcpu *vcpu);
+int sev_snp_update_protected_guest_state(struct kvm_vcpu *vcpu);
 
 /* vmenter.S */
 
Signature · 1 line
2.25.1
author maintainer
#278 lore raw

PATCH v6 40/49 KVM: x86: Export the kvm_zap_gfn_range() for the SNP use

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

While resolving the RMP page fault, we may run into cases where the page level between the RMP entry and TDP does not match and the 2M RMP entry must be split into 4K RMP entries. Or a 2M TDP page need to be broken into multiple of 4K pages.

To keep the RMP and TDP page level in sync, we will zap the gfn range after splitting the pages in the RMP entry. The zap should force the TDP to gets rebuilt with the new page level.

---
 arch/x86/include/asm/kvm_host.h | 2 ++
 arch/x86/kvm/mmu.h              | 2 --
 arch/x86/kvm/mmu/mmu.c          | 1 +
 3 files changed, 3 insertions(+), 2 deletions(-)
arch/x86/include/asm/kvm_host.h +2 -0
index 8abc0e724f5c..1db4d178eb1d 100644
@@ -1627,6 +1627,8 @@ void kvm_mmu_slot_leaf_clear_dirty(struct kvm *kvm,
 void kvm_mmu_zap_all(struct kvm *kvm);
 void kvm_mmu_invalidate_mmio_sptes(struct kvm *kvm, u64 gen);
 void kvm_mmu_change_mmu_pages(struct kvm *kvm, unsigned long kvm_nr_mmu_pages);
+void kvm_zap_gfn_range(struct kvm *kvm, gfn_t gfn_start, gfn_t gfn_end);
+
 
 int load_pdptrs(struct kvm_vcpu *vcpu, unsigned long cr3);
 
arch/x86/kvm/mmu.h +0 -2
index d55b5166389a..c5044958a0fa 100644
@@ -267,8 +267,6 @@ static inline u8 permission_fault(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu,
 	return -(u32)fault & errcode;
 }
 
-void kvm_zap_gfn_range(struct kvm *kvm, gfn_t gfn_start, gfn_t gfn_end);
-
 int kvm_arch_write_log_dirty(struct kvm_vcpu *vcpu);
 
 int kvm_mmu_post_init_vm(struct kvm *kvm);
arch/x86/kvm/mmu/mmu.c +1 -0
index c1ac486e096e..67120bfeb667 100644
@@ -6084,6 +6084,7 @@ static bool kvm_mmu_zap_collapsible_spte(struct kvm *kvm,
 
 	return need_tlb_flush;
 }
+EXPORT_SYMBOL_GPL(kvm_zap_gfn_range);
 
 void kvm_mmu_zap_collapsible_sptes(struct kvm *kvm,
 				   const struct kvm_memory_slot *slot)
Signature · 1 line
2.25.1
author maintainer
#279 lore raw

PATCH v6 41/49 KVM: SVM: Add support to handle the RMP nested page fault

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

When SEV-SNP is enabled in the guest, the hardware places restrictions on all memory accesses based on the contents of the RMP table. When hardware encounters RMP check failure caused by the guest memory access it raises the #NPF. The error code contains additional information on the access type. See the APM volume 2 for additional information.

---
 arch/x86/kvm/svm/sev.c | 76 ++++++++++++++++++++++++++++++++++++++++++
 arch/x86/kvm/svm/svm.c | 14 +++++---
 2 files changed, 86 insertions(+), 4 deletions(-)
arch/x86/kvm/svm/sev.c +76 -0
index 4ed90331bca0..7fc0fad87054 100644
@@ -4009,3 +4009,79 @@ void sev_post_unmap_gfn(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn)
 
 	spin_unlock(&sev->psc_lock);
 }
+
+void handle_rmp_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u64 error_code)
+{
+	int rmp_level, npt_level, rc, assigned;
+	struct kvm *kvm = vcpu->kvm;
+	gfn_t gfn = gpa_to_gfn(gpa);
+	bool need_psc = false;
+	enum psc_op psc_op;
+	kvm_pfn_t pfn;
+	bool private;
+
+	write_lock(&kvm->mmu_lock);
+
+	if (unlikely(!kvm_mmu_get_tdp_walk(vcpu, gpa, &pfn, &npt_level)))
+		goto unlock;
+
+	assigned = snp_lookup_rmpentry(pfn, &rmp_level);
+	if (unlikely(assigned < 0))
+		goto unlock;
+
+	private = !!(error_code & PFERR_GUEST_ENC_MASK);
+
+	/*
+	 * If the fault was due to size mismatch, or NPT and RMP page level's
+	 * are not in sync, then use PSMASH to split the RMP entry into 4K.
+	 */
+	if ((error_code & PFERR_GUEST_SIZEM_MASK) ||
+	    (npt_level == PG_LEVEL_4K && rmp_level == PG_LEVEL_2M && private)) {
+		rc = snp_rmptable_psmash(kvm, pfn);
+		if (rc)
+			pr_err_ratelimited("psmash failed, gpa 0x%llx pfn 0x%llx rc %d\n",
+					   gpa, pfn, rc);
+		goto out;
+	}
+
+	/*
+	 * If it's a private access, and the page is not assigned in the
+	 * RMP table, create a new private RMP entry. This can happen if
+	 * guest did not use the PSC VMGEXIT to transition the page state
+	 * before the access.
+	 */
+	if (!assigned && private) {
+		need_psc = 1;
+		psc_op = SNP_PAGE_STATE_PRIVATE;
+		goto out;
+	}
+
+	/*
+	 * If it's a shared access, but the page is private in the RMP table
+	 * then make the page shared in the RMP table. This can happen if
+	 * the guest did not use the PSC VMGEXIT to transition the page
+	 * state before the access.
+	 */
+	if (assigned && !private) {
+		need_psc = 1;
+		psc_op = SNP_PAGE_STATE_SHARED;
+	}
+
+out:
+	write_unlock(&kvm->mmu_lock);
+
+	if (need_psc)
+		rc = __snp_handle_page_state_change(vcpu, psc_op, gpa, PG_LEVEL_4K);
+
+	/*
+	 * The fault handler has updated the RMP pagesize, zap the existing
+	 * rmaps for large entry ranges so that nested page table gets rebuilt
+	 * with the updated RMP pagesize.
+	 */
+	gfn = gpa_to_gfn(gpa) & ~(KVM_PAGES_PER_HPAGE(PG_LEVEL_2M) - 1);
+	kvm_zap_gfn_range(kvm, gfn, gfn + PTRS_PER_PMD);
+	return;
+
+unlock:
+	write_unlock(&kvm->mmu_lock);
+}
arch/x86/kvm/svm/svm.c +10 -4
index 1c8e035ba011..7742bc986afc 100644
@@ -1866,15 +1866,21 @@ static int pf_interception(struct kvm_vcpu *vcpu)
 static int npf_interception(struct kvm_vcpu *vcpu)
 {
 	struct vcpu_svm *svm = to_svm(vcpu);
+	int rc;
 
 	u64 fault_address = svm->vmcb->control.exit_info_2;
 	u64 error_code = svm->vmcb->control.exit_info_1;
 
 	trace_kvm_page_fault(fault_address, error_code);
-	return kvm_mmu_page_fault(vcpu, fault_address, error_code,
-			static_cpu_has(X86_FEATURE_DECODEASSISTS) ?
-			svm->vmcb->control.insn_bytes : NULL,
-			svm->vmcb->control.insn_len);
+	rc = kvm_mmu_page_fault(vcpu, fault_address, error_code,
+				static_cpu_has(X86_FEATURE_DECODEASSISTS) ?
+				svm->vmcb->control.insn_bytes : NULL,
+				svm->vmcb->control.insn_len);
+
+	if (error_code & PFERR_GUEST_RMP_MASK)
+		handle_rmp_page_fault(vcpu, fault_address, error_code);
+
+	return rc;
 }
 
 static int db_interception(struct kvm_vcpu *vcpu)
Signature · 1 line
2.25.1
author maintainer
#288 lore raw

PATCH v6 42/49 KVM: SVM: Provide support for SNP_GUEST_REQUEST NAE event

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

Version 2 of GHCB specification added the support for two SNP Guest Request Message NAE events. The events allows for an SEV-SNP guest to make request to the SEV-SNP firmware through hypervisor using the SNP_GUEST_REQUEST API define in the SEV-SNP firmware specification.

The SNP_EXT_GUEST_REQUEST is similar to SNP_GUEST_REQUEST with the difference of an additional certificate blob that can be passed through the SNP_SET_CONFIG ioctl defined in the CCP driver. The CCP driver provides snp_guest_ext_guest_request() that is used by the KVM to get both the report and certificate data at once.

---
 arch/x86/kvm/svm/sev.c | 196 +++++++++++++++++++++++++++++++++++++++--
 arch/x86/kvm/svm/svm.h |   2 +
 2 files changed, 192 insertions(+), 6 deletions(-)
arch/x86/kvm/svm/sev.c +190 -6
index 7fc0fad87054..089af21a4efe 100644
@@ -343,6 +343,7 @@ static int sev_guest_init(struct kvm *kvm, struct kvm_sev_cmd *argp)
 
 		spin_lock_init(&sev->psc_lock);
 		ret = sev_snp_init(&argp->error);
+		mutex_init(&sev->guest_req_lock);
 	} else {
 		ret = sev_platform_init(&argp->error);
 	}
@@ -1884,23 +1885,39 @@ int sev_vm_move_enc_context_from(struct kvm *kvm, unsigned int source_fd)
 
 static void *snp_context_create(struct kvm *kvm, struct kvm_sev_cmd *argp)
 {
+	void *context = NULL, *certs_data = NULL, *resp_page = NULL;
+	struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
 	struct sev_data_snp_gctx_create data = {};
-	void *context;
 	int rc;
 
+	/* Allocate memory used for the certs data in SNP guest request */
+	certs_data = kmalloc(SEV_FW_BLOB_MAX_SIZE, GFP_KERNEL_ACCOUNT);
+	if (!certs_data)
+		return NULL;
+
 	/* Allocate memory for context page */
 	context = snp_alloc_firmware_page(GFP_KERNEL_ACCOUNT);
 	if (!context)
-		return NULL;
+		goto e_free;
+
+	/* Allocate a firmware buffer used during the guest command handling. */
+	resp_page = snp_alloc_firmware_page(GFP_KERNEL_ACCOUNT);
+	if (!resp_page)
+		goto e_free;
 
 	data.gctx_paddr = __psp_pa(context);
 	rc = __sev_issue_cmd(argp->sev_fd, SEV_CMD_SNP_GCTX_CREATE, &data, &argp->error);
-	if (rc) {
-		snp_free_firmware_page(context);
-		return NULL;
-	}
+	if (rc)
+		goto e_free;
+
+	sev->snp_certs_data = certs_data;
 
 	return context;
+
+e_free:
+	snp_free_firmware_page(context);
+	kfree(certs_data);
+	return NULL;
 }
 
 static int snp_bind_asid(struct kvm *kvm, int *error)
@@ -2565,6 +2582,8 @@ static int snp_decommission_context(struct kvm *kvm)
 	snp_free_firmware_page(sev->snp_context);
 	sev->snp_context = NULL;
 
+	kfree(sev->snp_certs_data);
+
 	return 0;
 }
 
@@ -3077,6 +3096,8 @@ static int sev_es_validate_vmgexit(struct vcpu_svm *svm, u64 *exit_code)
 	case SVM_VMGEXIT_UNSUPPORTED_EVENT:
 	case SVM_VMGEXIT_HV_FEATURES:
 	case SVM_VMGEXIT_PSC:
+	case SVM_VMGEXIT_GUEST_REQUEST:
+	case SVM_VMGEXIT_EXT_GUEST_REQUEST:
 		break;
 	default:
 		reason = GHCB_ERR_INVALID_EVENT;
@@ -3502,6 +3523,155 @@ static unsigned long snp_handle_page_state_change(struct vcpu_svm *svm)
 	return rc ? map_to_psc_vmgexit_code(rc) : 0;
 }
 
+static unsigned long snp_setup_guest_buf(struct vcpu_svm *svm,
+					 struct sev_data_snp_guest_request *data,
+					 gpa_t req_gpa, gpa_t resp_gpa)
+{
+	struct kvm_vcpu *vcpu = &svm->vcpu;
+	struct kvm *kvm = vcpu->kvm;
+	kvm_pfn_t req_pfn, resp_pfn;
+	struct kvm_sev_info *sev;
+
+	sev = &to_kvm_svm(kvm)->sev_info;
+
+	if (!IS_ALIGNED(req_gpa, PAGE_SIZE) || !IS_ALIGNED(resp_gpa, PAGE_SIZE))
+		return SEV_RET_INVALID_PARAM;
+
+	req_pfn = gfn_to_pfn(kvm, gpa_to_gfn(req_gpa));
+	if (is_error_noslot_pfn(req_pfn))
+		return SEV_RET_INVALID_ADDRESS;
+
+	resp_pfn = gfn_to_pfn(kvm, gpa_to_gfn(resp_gpa));
+	if (is_error_noslot_pfn(resp_pfn))
+		return SEV_RET_INVALID_ADDRESS;
+
+	if (rmp_make_private(resp_pfn, 0, PG_LEVEL_4K, 0, true))
+		return SEV_RET_INVALID_ADDRESS;
+
+	data->gctx_paddr = __psp_pa(sev->snp_context);
+	data->req_paddr = __sme_set(req_pfn << PAGE_SHIFT);
+	data->res_paddr = __sme_set(resp_pfn << PAGE_SHIFT);
+
+	return 0;
+}
+
+static void snp_cleanup_guest_buf(struct sev_data_snp_guest_request *data, unsigned long *rc)
+{
+	u64 pfn = __sme_clr(data->res_paddr) >> PAGE_SHIFT;
+	int ret;
+
+	ret = snp_page_reclaim(pfn);
+	if (ret)
+		*rc = SEV_RET_INVALID_ADDRESS;
+
+	ret = rmp_make_shared(pfn, PG_LEVEL_4K);
+	if (ret)
+		*rc = SEV_RET_INVALID_ADDRESS;
+}
+
+static void snp_handle_guest_request(struct vcpu_svm *svm, gpa_t req_gpa, gpa_t resp_gpa)
+{
+	struct sev_data_snp_guest_request data = {0};
+	struct kvm_vcpu *vcpu = &svm->vcpu;
+	struct kvm *kvm = vcpu->kvm;
+	struct kvm_sev_info *sev;
+	unsigned long rc;
+	int err;
+
+	if (!sev_snp_guest(vcpu->kvm)) {
+		rc = SEV_RET_INVALID_GUEST;
+		goto e_fail;
+	}
+
+	sev = &to_kvm_svm(kvm)->sev_info;
+
+	mutex_lock(&sev->guest_req_lock);
+
+	rc = snp_setup_guest_buf(svm, &data, req_gpa, resp_gpa);
+	if (rc)
+		goto unlock;
+
+	rc = sev_issue_cmd(kvm, SEV_CMD_SNP_GUEST_REQUEST, &data, &err);
+	if (rc)
+		/* use the firmware error code */
+		rc = err;
+
+	snp_cleanup_guest_buf(&data, &rc);
+
+unlock:
+	mutex_unlock(&sev->guest_req_lock);
+
+e_fail:
+	svm_set_ghcb_sw_exit_info_2(vcpu, rc);
+}
+
+static void snp_handle_ext_guest_request(struct vcpu_svm *svm, gpa_t req_gpa, gpa_t resp_gpa)
+{
+	struct sev_data_snp_guest_request req = {0};
+	struct kvm_vcpu *vcpu = &svm->vcpu;
+	struct kvm *kvm = vcpu->kvm;
+	unsigned long data_npages;
+	struct kvm_sev_info *sev;
+	unsigned long rc, err;
+	u64 data_gpa;
+
+	if (!sev_snp_guest(vcpu->kvm)) {
+		rc = SEV_RET_INVALID_GUEST;
+		goto e_fail;
+	}
+
+	sev = &to_kvm_svm(kvm)->sev_info;
+
+	data_gpa = vcpu->arch.regs[VCPU_REGS_RAX];
+	data_npages = vcpu->arch.regs[VCPU_REGS_RBX];
+
+	if (!IS_ALIGNED(data_gpa, PAGE_SIZE)) {
+		rc = SEV_RET_INVALID_ADDRESS;
+		goto e_fail;
+	}
+
+	/* Verify that requested blob will fit in certificate buffer */
+	if ((data_npages << PAGE_SHIFT) > SEV_FW_BLOB_MAX_SIZE) {
+		rc = SEV_RET_INVALID_PARAM;
+		goto e_fail;
+	}
+
+	mutex_lock(&sev->guest_req_lock);
+
+	rc = snp_setup_guest_buf(svm, &req, req_gpa, resp_gpa);
+	if (rc)
+		goto unlock;
+
+	rc = snp_guest_ext_guest_request(&req, (unsigned long)sev->snp_certs_data,
+					 &data_npages, &err);
+	if (rc) {
+		/*
+		 * If buffer length is small then return the expected
+		 * length in rbx.
+		 */
+		if (err == SNP_GUEST_REQ_INVALID_LEN)
+			vcpu->arch.regs[VCPU_REGS_RBX] = data_npages;
+
+		/* pass the firmware error code */
+		rc = err;
+		goto cleanup;
+	}
+
+	/* Copy the certificate blob in the guest memory */
+	if (data_npages &&
+	    kvm_write_guest(kvm, data_gpa, sev->snp_certs_data, data_npages << PAGE_SHIFT))
+		rc = SEV_RET_INVALID_ADDRESS;
+
+cleanup:
+	snp_cleanup_guest_buf(&req, &rc);
+
+unlock:
+	mutex_unlock(&sev->guest_req_lock);
+
+e_fail:
+	svm_set_ghcb_sw_exit_info_2(vcpu, rc);
+}
+
 static int sev_handle_vmgexit_msr_protocol(struct vcpu_svm *svm)
 {
 	struct vmcb_control_area *control = &svm->vmcb->control;
@@ -3753,6 +3923,20 @@ int sev_handle_vmgexit(struct kvm_vcpu *vcpu)
 		svm_set_ghcb_sw_exit_info_2(vcpu, rc);
 		break;
 	}
+	case SVM_VMGEXIT_GUEST_REQUEST: {
+		snp_handle_guest_request(svm, control->exit_info_1, control->exit_info_2);
+
+		ret = 1;
+		break;
+	}
+	case SVM_VMGEXIT_EXT_GUEST_REQUEST: {
+		snp_handle_ext_guest_request(svm,
+					     control->exit_info_1,
+					     control->exit_info_2);
+
+		ret = 1;
+		break;
+	}
 	case SVM_VMGEXIT_UNSUPPORTED_EVENT:
 		vcpu_unimpl(vcpu,
 			    "vmgexit: unsupported event - exit_info_1=%#llx, exit_info_2=%#llx\n",
arch/x86/kvm/svm/svm.h +2 -0
index 3fd95193ed8d..3be24da1a743 100644
@@ -98,6 +98,8 @@ struct kvm_sev_info {
 	u64 snp_init_flags;
 	void *snp_context;      /* SNP guest context page */
 	spinlock_t psc_lock;
+	void *snp_certs_data;
+	struct mutex guest_req_lock;
 };
 
 struct kvm_svm {
Signature · 1 line
2.25.1
author maintainer
#297 lore raw

PATCH v6 43/49 KVM: SVM: Use a VMSA physical address variable for populating VMCB

In reply to Ashish Kalra

From: Tom Lendacky <[email protected]>

In preparation to support SEV-SNP AP Creation, use a variable that holds the VMSA physical address rather than converting the virtual address. This will allow SEV-SNP AP Creation to set the new physical address that will be used should the vCPU reset path be taken.

---
 arch/x86/kvm/svm/sev.c | 5 ++---
 arch/x86/kvm/svm/svm.c | 9 ++++++++-
 arch/x86/kvm/svm/svm.h | 1 +
 3 files changed, 11 insertions(+), 4 deletions(-)
arch/x86/kvm/svm/sev.c +2 -3
index 089af21a4efe..d5584551f3dd 100644
@@ -3980,10 +3980,9 @@ void sev_es_init_vmcb(struct vcpu_svm *svm)
 
 	/*
 	 * An SEV-ES guest requires a VMSA area that is a separate from the
-	 * VMCB page. Do not include the encryption mask on the VMSA physical
-	 * address since hardware will access it using the guest key.
+	 * VMCB page.
 	 */
-	svm->vmcb->control.vmsa_pa = __pa(svm->sev_es.vmsa);
+	svm->vmcb->control.vmsa_pa = svm->sev_es.vmsa_pa;
 
 	/* Can't intercept CR register access, HV can't modify CR registers */
 	svm_clr_intercept(svm, INTERCEPT_CR0_READ);
arch/x86/kvm/svm/svm.c +8 -1
index 7742bc986afc..f7155abe7567 100644
@@ -1296,9 +1296,16 @@ static int svm_vcpu_create(struct kvm_vcpu *vcpu)
 	svm->vmcb01.pa = __sme_set(page_to_pfn(vmcb01_page) << PAGE_SHIFT);
 	svm_switch_vmcb(svm, &svm->vmcb01);
 
-	if (vmsa_page)
+	if (vmsa_page) {
 		svm->sev_es.vmsa = page_address(vmsa_page);
 
+		/*
+		 * Do not include the encryption mask on the VMSA physical
+		 * address since hardware will access it using the guest key.
+		 */
+		svm->sev_es.vmsa_pa = __pa(svm->sev_es.vmsa);
+	}
+
 	svm->guest_state_loaded = false;
 
 	return 0;
arch/x86/kvm/svm/svm.h +1 -0
index 3be24da1a743..46790bab07a8 100644
@@ -197,6 +197,7 @@ struct svm_nested_state {
 struct vcpu_sev_es_state {
 	/* SEV-ES support */
 	struct sev_es_save_area *vmsa;
+	hpa_t vmsa_pa;
 	bool ghcb_in_use;
 	bool received_first_sipi;
 	unsigned int ap_reset_hold_type;
Signature · 1 line
2.25.1
author maintainer
#298 lore raw

PATCH v6 44/49 KVM: SVM: Support SEV-SNP AP Creation NAE event

In reply to Ashish Kalra

From: Tom Lendacky <[email protected]>

Add support for the SEV-SNP AP Creation NAE event. This allows SEV-SNP guests to alter the register state of the APs on their own. This allows the guest a way of simulating INIT-SIPI.

A new event, KVM_REQ_UPDATE_PROTECTED_GUEST_STATE, is created and used so as to avoid updating the VMSA pointer while the vCPU is running.

For CREATE
The guest supplies the GPA of the VMSA to be used for the vCPU with the specified APIC ID. The GPA is saved in the svm struct of the target vCPU, the KVM_REQ_UPDATE_PROTECTED_GUEST_STATE event is added to the vCPU and then the vCPU is kicked.

For CREATE_ON_INIT:
The guest supplies the GPA of the VMSA to be used for the vCPU with the specified APIC ID the next time an INIT is performed. The GPA is saved in the svm struct of the target vCPU.

For DESTROY:
The guest indicates it wishes to stop the vCPU. The GPA is cleared from the svm struct, the KVM_REQ_UPDATE_PROTECTED_GUEST_STATE event is added to vCPU and then the vCPU is kicked.

The KVM_REQ_UPDATE_PROTECTED_GUEST_STATE event handler will be invoked as a result of the event or as a result of an INIT. The handler sets the vCPU to the KVM_MP_STATE_UNINITIALIZED state, so that any errors will leave the vCPU as not runnable. Any previous VMSA pages that were installed as part of an SEV-SNP AP Creation NAE event are un-pinned. If a new VMSA is to be installed, the VMSA guest page is pinned and set as the VMSA in the vCPU VMCB and the vCPU state is set to KVM_MP_STATE_RUNNABLE. If a new VMSA is not to be installed, the VMSA is cleared in the vCPU VMCB and the vCPU state is left as KVM_MP_STATE_UNINITIALIZED to prevent it from being run.

---
 arch/x86/include/asm/kvm-x86-ops.h |   1 -
 arch/x86/include/asm/kvm_host.h    |   3 +-
 arch/x86/include/asm/svm.h         |   7 +-
 arch/x86/kvm/svm/sev.c             | 197 +++++++++++++++++++++++++++++
 arch/x86/kvm/svm/svm.c             |   5 +-
 arch/x86/kvm/svm/svm.h             |   6 +
 arch/x86/kvm/x86.c                 |   9 +-
 7 files changed, 221 insertions(+), 7 deletions(-)
arch/x86/include/asm/kvm-x86-ops.h +0 -1
index 2dd2bc0cf4c3..e0068e702692 100644
@@ -130,7 +130,6 @@ KVM_X86_OP(vcpu_deliver_sipi_vector)
 KVM_X86_OP_OPTIONAL_RET0(vcpu_get_apicv_inhibit_reasons);
 KVM_X86_OP(alloc_apic_backing_page)
 KVM_X86_OP_OPTIONAL(rmp_page_level_adjust)
-KVM_X86_OP(update_protected_guest_state)
 
 #undef KVM_X86_OP
 #undef KVM_X86_OP_OPTIONAL
arch/x86/include/asm/kvm_host.h +1 -2
index 1db4d178eb1d..660cf39344fb 100644
@@ -105,6 +105,7 @@
 	KVM_ARCH_REQ_FLAGS(30, KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP)
 #define KVM_REQ_MMU_FREE_OBSOLETE_ROOTS \
 	KVM_ARCH_REQ_FLAGS(31, KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP)
+#define KVM_REQ_UPDATE_PROTECTED_GUEST_STATE	KVM_ARCH_REQ(32)
 
 #define CR0_RESERVED_BITS                                               \
 	(~(unsigned long)(X86_CR0_PE | X86_CR0_MP | X86_CR0_EM | X86_CR0_TS \
@@ -1524,8 +1525,6 @@ struct kvm_x86_ops {
 	void *(*alloc_apic_backing_page)(struct kvm_vcpu *vcpu);
 
 	void (*rmp_page_level_adjust)(struct kvm *kvm, kvm_pfn_t pfn, int *level);
-
-	int (*update_protected_guest_state)(struct kvm_vcpu *vcpu);
 };
 
 struct kvm_x86_nested_ops {
arch/x86/include/asm/svm.h +6 -1
index 284a8113227e..a69b6da71a65 100644
@@ -263,7 +263,12 @@ enum avic_ipi_failure_cause {
 #define AVIC_HPA_MASK	~((0xFFFULL << 52) | 0xFFF)
 #define VMCB_AVIC_APIC_BAR_MASK		0xFFFFFFFFFF000ULL
 
-#define SVM_SEV_FEAT_SNP_ACTIVE		BIT(0)
+#define SVM_SEV_FEAT_SNP_ACTIVE			BIT(0)
+#define SVM_SEV_FEAT_RESTRICTED_INJECTION	BIT(3)
+#define SVM_SEV_FEAT_ALTERNATE_INJECTION	BIT(4)
+#define SVM_SEV_FEAT_INT_INJ_MODES		\
+	(SVM_SEV_FEAT_RESTRICTED_INJECTION |	\
+	 SVM_SEV_FEAT_ALTERNATE_INJECTION)
 
 struct vmcb_seg {
 	u16 selector;
arch/x86/kvm/svm/sev.c +197 -0

233 lines not on this page · raw message

arch/x86/kvm/svm/svm.c +3 -2
index f7155abe7567..fced6ea423ad 100644
@@ -1237,6 +1237,9 @@ static void svm_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event)
 	svm->spec_ctrl = 0;
 	svm->virt_spec_ctrl = 0;
 
+	if (init_event)
+		sev_snp_init_protected_guest_state(vcpu);
+
 	init_vmcb(vcpu);
 
 	if (!init_event)
@@ -4749,8 +4752,6 @@ static struct kvm_x86_ops svm_x86_ops __initdata = {
 	.alloc_apic_backing_page = svm_alloc_apic_backing_page,
 
 	.rmp_page_level_adjust = sev_rmp_page_level_adjust,
-
-	.update_protected_guest_state = sev_snp_update_protected_guest_state,
 };
 
 /*
arch/x86/kvm/svm/svm.h +6 -0
index 46790bab07a8..971ff4e949fd 100644
@@ -100,6 +100,8 @@ struct kvm_sev_info {
 	spinlock_t psc_lock;
 	void *snp_certs_data;
 	struct mutex guest_req_lock;
+
+	u64 sev_features;	/* Features set at VMSA creation */
 };
 
 struct kvm_svm {
@@ -217,6 +219,10 @@ struct vcpu_sev_es_state {
 	u64 ghcb_sw_exit_info_2;
 
 	u64 ghcb_registered_gpa;
+
+	struct mutex snp_vmsa_mutex;
+	gpa_t snp_vmsa_gpa;
+	bool snp_ap_create;
 };
 
 struct vcpu_svm {
arch/x86/kvm/x86.c +8 -1
index 4a1d16231e30..c649d15efae3 100644
@@ -10095,6 +10095,12 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)
 
 		if (kvm_check_request(KVM_REQ_UPDATE_CPU_DIRTY_LOGGING, vcpu))
 			static_call(kvm_x86_update_cpu_dirty_logging)(vcpu);
+
+		if (kvm_check_request(KVM_REQ_UPDATE_PROTECTED_GUEST_STATE, vcpu)) {
+			kvm_vcpu_reset(vcpu, true);
+			if (vcpu->arch.mp_state != KVM_MP_STATE_RUNNABLE)
+				goto out;
+		}
 	}
 
 	if (kvm_check_request(KVM_REQ_EVENT, vcpu) || req_int_win ||
@@ -12219,7 +12225,8 @@ static inline bool kvm_vcpu_has_events(struct kvm_vcpu *vcpu)
 	if (!list_empty_careful(&vcpu->async_pf.done))
 		return true;
 
-	if (kvm_apic_has_events(vcpu))
+	if (kvm_apic_has_events(vcpu) ||
+	    kvm_test_request(KVM_REQ_UPDATE_PROTECTED_GUEST_STATE, vcpu))
 		return true;
 
 	if (vcpu->arch.pv.pv_unhalted)
Signature · 1 line
2.25.1
author maintainer
#299 lore raw

PATCH v6 45/49 KVM: SVM: Add module parameter to enable the SEV-SNP

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

Add a module parameter than can be used to enable or disable the SEV-SNP feature. Now that KVM contains the support for the SNP set the GHCB hypervisor feature flag to indicate that SNP is supported.

---
 arch/x86/kvm/svm/sev.c | 7 ++++---
 arch/x86/kvm/svm/svm.h | 2 +-
 2 files changed, 5 insertions(+), 4 deletions(-)
arch/x86/kvm/svm/sev.c +4 -3
index bb7d4547df81..2c88215a111f 100644
@@ -57,14 +57,15 @@ module_param_named(sev, sev_enabled, bool, 0444);
 /* enable/disable SEV-ES support */
 static bool sev_es_enabled = true;
 module_param_named(sev_es, sev_es_enabled, bool, 0444);
+
+/* enable/disable SEV-SNP support */
+static bool sev_snp_enabled = true;
+module_param_named(sev_snp, sev_snp_enabled, bool, 0444);
 #else
 #define sev_enabled false
 #define sev_es_enabled false
 #endif /* CONFIG_KVM_AMD_SEV */
 
-/* enable/disable SEV-SNP support */
-static bool sev_snp_enabled;
-
 #define AP_RESET_HOLD_NONE		0
 #define AP_RESET_HOLD_NAE_EVENT		1
 #define AP_RESET_HOLD_MSR_PROTO		2
arch/x86/kvm/svm/svm.h +1 -1
index 971ff4e949fd..7b14b5ef1f8c 100644
@@ -688,7 +688,7 @@ unsigned long avic_vcpu_get_apicv_inhibit_reasons(struct kvm_vcpu *vcpu);
 #define GHCB_VERSION_MAX	2ULL
 #define GHCB_VERSION_MIN	1ULL
 
-#define GHCB_HV_FT_SUPPORTED	0
+#define GHCB_HV_FT_SUPPORTED	(GHCB_HV_FT_SNP | GHCB_HV_FT_SNP_AP_CREATION)
 
 extern unsigned int max_sev_asid;
 
Signature · 1 line
2.25.1
author maintainer
#300 lore raw

PATCH v6 46/49 ccp: add support to decrypt the page

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

Add support to decrypt guest encrypted memory, these API interfaces can be used for example to dump VMCBs on SNP guest exit.

---
 drivers/crypto/ccp/sev-dev.c | 33 ++++++++++++++++++++++++++++++---
 include/linux/psp-sev.h      |  6 +++---
 2 files changed, 33 insertions(+), 6 deletions(-)
drivers/crypto/ccp/sev-dev.c +30 -3
index f6306b820b86..9896350e7f56 100644
@@ -1852,11 +1852,38 @@ int snp_guest_page_reclaim(struct sev_data_snp_page_reclaim *data, int *error)
 }
 EXPORT_SYMBOL_GPL(snp_guest_page_reclaim);
 
-int snp_guest_dbg_decrypt(struct sev_data_snp_dbg *data, int *error)
+int snp_guest_dbg_decrypt_page(u64 gctx_pfn, u64 src_pfn, u64 dst_pfn, int *error)
 {
-	return sev_do_cmd(SEV_CMD_SNP_DBG_DECRYPT, data, error);
+	struct sev_data_snp_dbg data = {0};
+	struct sev_device *sev;
+	int ret;
+
+	if (!psp_master || !psp_master->sev_data)
+		return -ENODEV;
+
+	sev = psp_master->sev_data;
+
+	if (!sev->snp_inited)
+		return -EINVAL;
+
+	data.gctx_paddr = sme_me_mask | (gctx_pfn << PAGE_SHIFT);
+	data.src_addr = sme_me_mask | (src_pfn << PAGE_SHIFT);
+	data.dst_addr = sme_me_mask | (dst_pfn << PAGE_SHIFT);
+	data.len = PAGE_SIZE;
+
+	/* The destination page must be in the firmware state. */
+	if (snp_set_rmp_state(data.dst_addr, 1, true, false, false))
+		return -EIO;
+
+	ret = sev_do_cmd(SEV_CMD_SNP_DBG_DECRYPT, &data, error);
+
+	/* Restore the page state */
+	if (snp_set_rmp_state(data.dst_addr, 1, false, false, true))
+		ret = -EIO;
+
+	return ret;
 }
-EXPORT_SYMBOL_GPL(snp_guest_dbg_decrypt);
+EXPORT_SYMBOL_GPL(snp_guest_dbg_decrypt_page);
 
 int snp_guest_ext_guest_request(struct sev_data_snp_guest_request *data,
 				unsigned long vaddr, unsigned long *npages, unsigned long *fw_err)
include/linux/psp-sev.h +3 -3
index cd37ccd1fa1f..8d2565c70c39 100644
@@ -928,7 +928,7 @@ int snp_guest_decommission(struct sev_data_snp_decommission *data, int *error);
 int snp_guest_page_reclaim(struct sev_data_snp_page_reclaim *data, int *error);
 
 /**
- * snp_guest_dbg_decrypt - perform SEV SNP_DBG_DECRYPT command
+ * snp_guest_dbg_decrypt_page - perform SEV SNP_DBG_DECRYPT command
  *
  * @sev_ret: sev command return code
  *
@@ -939,7 +939,7 @@ int snp_guest_page_reclaim(struct sev_data_snp_page_reclaim *data, int *error);
  * -%ETIMEDOUT if the sev command timed out
  * -%EIO       if the sev returned a non-zero return code
  */
-int snp_guest_dbg_decrypt(struct sev_data_snp_dbg *data, int *error);
+int snp_guest_dbg_decrypt_page(u64 gctx_pfn, u64 src_pfn, u64 dst_pfn, int *error);
 
 void *psp_copy_user_blob(u64 uaddr, u32 len);
 void *snp_alloc_firmware_page(gfp_t mask);
@@ -997,7 +997,7 @@ static inline int snp_guest_page_reclaim(struct sev_data_snp_page_reclaim *data,
 	return -ENODEV;
 }
 
-static inline int snp_guest_dbg_decrypt(struct sev_data_snp_dbg *data, int *error)
+static inline int snp_guest_dbg_decrypt_page(u64 gctx_pfn, u64 src_pfn, u64 dst_pfn, int *error)
 {
 	return -ENODEV;
 }
Signature · 1 line
2.25.1
author maintainer
#301 lore raw

PATCH v6 47/49 *fix for stale per-cpu pointer due to cond_resched during ghcb mapping

In reply to Ashish Kalra

From: Michael Roth <[email protected]>

---
 arch/x86/kvm/svm/svm.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
arch/x86/kvm/svm/svm.c +5 -1
index fced6ea423ad..f78e3b1bde0e 100644
@@ -1352,7 +1352,7 @@ static void svm_vcpu_free(struct kvm_vcpu *vcpu)
 static void svm_prepare_switch_to_guest(struct kvm_vcpu *vcpu)
 {
 	struct vcpu_svm *svm = to_svm(vcpu);
-	struct svm_cpu_data *sd = per_cpu(svm_data, vcpu->cpu);
+	struct svm_cpu_data *sd;
 
 	if (sev_es_guest(vcpu->kvm))
 		sev_es_unmap_ghcb(svm);
@@ -1360,6 +1360,10 @@ static void svm_prepare_switch_to_guest(struct kvm_vcpu *vcpu)
 	if (svm->guest_state_loaded)
 		return;
 
+	/* sev_es_unmap_ghcb() can resched, so grab per-cpu pointer afterward. */
+	barrier();
+	sd = per_cpu(svm_data, vcpu->cpu);
+
 	/*
 	 * Save additional host state that will be restored on VMEXIT (sev-es)
 	 * or subsequent vmload of host save area.
Signature · 1 line
2.25.1
author maintainer
#304 lore raw

PATCH v6 48/49 *debug: warn and retry failed rmpupdates

In reply to Ashish Kalra

From: Michael Roth <[email protected]>

In some cases on B0 hardware exhibits something like the following behavior (where M < 512):

  Guest A                        | Guest B
 |-------------------------------|----------------------------------|
 |                               | rc = rmpupdate pfn=N*512,4K,priv
 | rmpupdate pfn=N*512+M,4K,priv |

| rc = FAIL_OVERLAP | rc = SUCCESS

The FAIL_OVERLAP might possible be the result of hardware temporarily treating Guest B's rmpupdate for pfn=N*512 as a 2M update, causing the subsequent update from Guest A for pfn=N*512+M to report FAIL_OVERLAP at that particular instance. Retrying the update for N*512+M immediately afterward seems to resolve the FAIL_OVERLAP issue reliably however.

A similar failure has also been observed when transitioning pages back to shared during VM destroy. In this case repeating the rmpupdate does not always seem to resolve the failure immediately.

Both situations are much more likely to occur if THP is disabled, or if it is enabled/disabled while guests are actively being started/stopped.

Include some debug/error information to get a better idea of the behavior on different hardware, and add the rmpupdate retry as a workaround for Milan B0 testing.

---
 arch/x86/kernel/sev.c | 15 +++++++++++++++
 1 file changed, 15 insertions(+)
arch/x86/kernel/sev.c +15 -0
index 6640a639fffc..5ae8c9f853c8 100644
@@ -2530,6 +2530,7 @@ static int rmpupdate(u64 pfn, struct rmpupdate *val)
 {
 	unsigned long paddr = pfn << PAGE_SHIFT;
 	int ret, level, npages;
+	int retries = 0;
 
 	if (!pfn_valid(pfn))
 		return -EINVAL;
@@ -2552,12 +2553,26 @@ static int rmpupdate(u64 pfn, struct rmpupdate *val)
 		}
 	}
 
+retry:
 	/* Binutils version 2.36 supports the RMPUPDATE mnemonic. */
 	asm volatile(".byte 0xF2, 0x0F, 0x01, 0xFE"
 		     : "=a"(ret)
 		     : "a"(paddr), "c"((unsigned long)val)
 		     : "memory", "cc");
 
+	if (ret) {
+		if (!retries) {
+			pr_err("rmpupdate failed, ret: %d, pfn: %llx, npages: %d, level: %d, retrying (max: %d)...\n",
+			       ret, pfn, npages, level, 2 * num_present_cpus());
+			dump_stack();
+		}
+		retries++;
+		if (retries < 2 * num_present_cpus())
+			goto retry;
+	} else if (retries > 0) {
+		pr_err("rmpupdate for pfn %llx succeeded after %d retries\n", pfn, retries);
+	}
+
 	/*
 	 * Restore the direct map after the page is removed from the RMP table.
 	 */
Signature · 1 line
2.25.1
author maintainer
#305 lore raw

PATCH v6 49/49 KVM: SVM: Sync the GHCB scratch buffer using already mapped ghcb

In reply to Ashish Kalra

From: Ashish Kalra <[email protected]>

Using kvm_write_guest() to sync the GHCB scratch buffer can fail due to host mapping being 2M, but RMP being 4K. The page fault handling in do_user_addr_fault() fails to split the 2M page to handle RMP fault due to it being called here in a non-preemptible context. Instead use the already kernel mapped ghcb to sync the scratch buffer when the scratch buffer is contained within the GHCB.

---
 arch/x86/kvm/svm/sev.c | 29 +++++++++++++++++++++--------
 arch/x86/kvm/svm/svm.h |  2 ++
 2 files changed, 23 insertions(+), 8 deletions(-)
arch/x86/kvm/svm/sev.c +21 -8
index 2c88215a111f..e1dd67e12774 100644
@@ -2944,6 +2944,24 @@ static bool sev_es_sync_to_ghcb(struct vcpu_svm *svm)
 	ghcb_set_sw_exit_info_1(ghcb, svm->sev_es.ghcb_sw_exit_info_1);
 	ghcb_set_sw_exit_info_2(ghcb, svm->sev_es.ghcb_sw_exit_info_2);
 
+	/* Sync the scratch buffer area. */
+	if (svm->sev_es.ghcb_sa_sync) {
+		if (svm->sev_es.ghcb_sa_contained) {
+			memcpy(ghcb->shared_buffer + svm->sev_es.ghcb_sa_offset,
+			       svm->sev_es.ghcb_sa, svm->sev_es.ghcb_sa_len);
+		} else {
+			int ret;
+
+			ret = kvm_write_guest(svm->vcpu.kvm,
+					      svm->sev_es.ghcb_sa_gpa,
+					      svm->sev_es.ghcb_sa, svm->sev_es.ghcb_sa_len);
+			if (ret)
+				pr_warn_ratelimited("unmap_ghcb: kvm_write_guest failed while syncing scratch area, gpa: %llx, ret: %d\n",
+						    svm->sev_es.ghcb_sa_gpa, ret);
+		}
+		svm->sev_es.ghcb_sa_sync = false;
+	}
+
 	trace_kvm_vmgexit_exit(svm->vcpu.vcpu_id, ghcb);
 
 	svm_unmap_ghcb(svm, &map);
@@ -3156,14 +3174,6 @@ void sev_es_unmap_ghcb(struct vcpu_svm *svm)
 	if (!svm->sev_es.ghcb_in_use)
 		return;
 
-	 /* Sync the scratch buffer area. */
-	if (svm->sev_es.ghcb_sa_sync) {
-		kvm_write_guest(svm->vcpu.kvm,
-				svm->sev_es.ghcb_sa_gpa,
-				svm->sev_es.ghcb_sa, svm->sev_es.ghcb_sa_len);
-		svm->sev_es.ghcb_sa_sync = false;
-	}
-
 	sev_es_sync_to_ghcb(svm);
 
 	svm->sev_es.ghcb_in_use = false;
@@ -3229,6 +3239,8 @@ static int setup_vmgexit_scratch(struct vcpu_svm *svm, bool sync, u64 len)
 			       scratch_gpa_beg, scratch_gpa_end);
 			goto e_scratch;
 		}
+		svm->sev_es.ghcb_sa_contained = true;
+		svm->sev_es.ghcb_sa_offset = scratch_gpa_beg - ghcb_scratch_beg;
 	} else {
 		/*
 		 * The guest memory must be read into a kernel buffer, so
@@ -3239,6 +3251,7 @@ static int setup_vmgexit_scratch(struct vcpu_svm *svm, bool sync, u64 len)
 			       len, GHCB_SCRATCH_AREA_LIMIT);
 			goto e_scratch;
 		}
+		svm->sev_es.ghcb_sa_contained = false;
 	}
 
 	if (svm->sev_es.ghcb_sa_alloc_len < len) {
arch/x86/kvm/svm/svm.h +2 -0
index 7b14b5ef1f8c..2cdfc79bf2cf 100644
@@ -210,6 +210,8 @@ struct vcpu_sev_es_state {
 	u64 ghcb_sa_gpa;
 	u32 ghcb_sa_alloc_len;
 	bool ghcb_sa_sync;
+	bool ghcb_sa_contained;
+	u32 ghcb_sa_offset;
 
 	/*
 	 * SEV-ES support to hold the sw_exit_info return values to be
Signature · 1 line
2.25.1
maintainer
#2 lore raw

Re: [PATCH Part2 v6 01/49] x86/cpufeatures: Add SEV-SNP CPU feature

In reply to Ashish Kalra
Quoted 8 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

Add CPU feature detection for Secure Encrypted Virtualization with Secure Nested Paging. This feature adds a strong memory integrity protection to help prevent malicious hypervisor-based attacks like data replay, memory re-mapping, and more.

verify_tags: Warning: Sender Ashish Kalra <[email protected]> hasn't signed off on the patch!

When you send someone else's patch, you need to add your SOB underneath it to state that you have handled that patch too, on its way mainline.

While waiting for review, please brush up on the development process by perusing the documentation in Documentation/process/ and especially

Documentation/process/submitting-patches.rst

Quoted 11 lines
---
 arch/x86/include/asm/cpufeatures.h       | 1 +
 arch/x86/kernel/cpu/amd.c                | 3 ++-
 tools/arch/x86/include/asm/cpufeatures.h | 1 +
 3 files changed, 4 insertions(+), 1 deletion(-)
arch/x86/include/asm/cpufeatures.h +0 -0
index 30da1341f226..1cba0217669f 100644

arch/x86/include/asm/cpufeatures.h line 410

Quoted 4 lines
arch/x86/include/asm/cpufeatures.h +1 -0
@@ -407,6 +407,7 @@
 #define X86_FEATURE_SEV			(19*32+ 1) /* AMD Secure Encrypted Virtualization */
 #define X86_FEATURE_VM_PAGE_FLUSH	(19*32+ 2) /* "" VM Page Flush MSR is supported */
 #define X86_FEATURE_SEV_ES		(19*32+ 3) /* AMD Secure Encrypted Virtualization - Encrypted State */
+#define X86_FEATURE_SEV_SNP		(19*32+4)  /* AMD Secure Encrypted Virtualization - Secure Nested Paging */

Do you not see how there's a space between the '+' and the single-digit number so that the vertical formatting works?

arch/x86/kernel/cpu/amd.c static void early_detect_mem_encrypt(struct cpuinfo_x86 *c) · line 562

Quoted 13 lines
arch/x86/include/asm/cpufeatures.h +0 -0
 #define X86_FEATURE_SME_COHERENT	(19*32+10) /* "" AMD hardware-enforced cache coherency */
 
 /*
arch/x86/kernel/cpu/amd.c +1 -1
index 0c0b09796ced..2e87015a9d69 100644
@@ -559,7 +559,7 @@ static void early_detect_mem_encrypt(struct cpuinfo_x86 *c)
 	 *	      If the kernel has not enabled SME via any means then
 	 *	      don't advertise the SME feature.
 	 *   For SEV: If BIOS has not enabled SEV then don't advertise the
-	 *            SEV and SEV_ES feature (set in scattered.c).
+	 *            SEV, SEV_ES and SEV_SNP feature.

Let's generalize that so that it doesn't get updated with every feature:

"... then don't advertize SEV and any additional functionality based on it."

Quoted 16 lines
arch/x86/kernel/cpu/amd.c +1 -0
 	 *
 	 *   In all cases, since support for SME and SEV requires long mode,
 	 *   don't advertise the feature under CONFIG_X86_32.
@@ -594,6 +594,7 @@ static void early_detect_mem_encrypt(struct cpuinfo_x86 *c)
 clear_sev:
 		setup_clear_cpu_cap(X86_FEATURE_SEV);
 		setup_clear_cpu_cap(X86_FEATURE_SEV_ES);
+		setup_clear_cpu_cap(X86_FEATURE_SEV_SNP);
 	}
 }
 
tools/arch/x86/include/asm/cpufeatures.h +0 -0
index 73e643ae94b6..a636342ecb26 100644

tools/arch/x86/include/asm/cpufeatures.h line 409

Quoted 5 lines
tools/arch/x86/include/asm/cpufeatures.h +1 -0
@@ -405,6 +405,7 @@
 #define X86_FEATURE_SEV			(19*32+ 1) /* AMD Secure Encrypted Virtualization */
 #define X86_FEATURE_VM_PAGE_FLUSH	(19*32+ 2) /* "" VM Page Flush MSR is supported */
 #define X86_FEATURE_SEV_ES		(19*32+ 3) /* AMD Secure Encrypted Virtualization - Encrypted State */
+#define X86_FEATURE_SEV_SNP		(19*32+4)  /* AMD Secure Encrypted Virtualization - Secure Nested Paging */
 #define X86_FEATURE_SME_COHERENT	(19*32+10) /* "" AMD hardware-enforced cache coherency */

Ditto.

Thx.

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
#4 lore raw

Re: [PATCH Part2 v6 02/49] iommu/amd: Introduce function to check SEV-SNP support

In reply to Ashish Kalra
Quoted 26 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

The SEV-SNP support requires that IOMMU must to enabled, see the IOMMU spec section 2.12 for further details. If IOMMU is not enabled or the SNPSup extended feature register is not set then the SNP_INIT command (used for initializing firmware) will fail.

The iommu_sev_snp_supported() can be used to check if IOMMU supports the SEV-SNP feature.

---
 drivers/iommu/amd/init.c | 30 ++++++++++++++++++++++++++++++
 include/linux/iommu.h    |  9 +++++++++
 2 files changed, 39 insertions(+)
drivers/iommu/amd/init.c +1 -0
index 1a3ad58ba846..82be8067ddf5 100644
@@ -3361,3 +3361,33 @@ int amd_iommu_pc_set_reg(struct amd_iommu *iommu, u8 bank, u8 cntr, u8 fxn, u64
 
        return iommu_pc_get_set_reg(iommu, bank, cntr, fxn, value, true);
 }
+

drivers/iommu/amd/init.c int amd_iommu_pc_set_reg(struct amd_iommu *iommu, u8 bank, u8 cntr, u8 fxn, u64 · line 3374

Quoted 10 lines
drivers/iommu/amd/init.c +10 -0
+bool iommu_sev_snp_supported(void)
+{
+       struct amd_iommu *iommu;
+
+       /*
+        * The SEV-SNP support requires that IOMMU must be enabled, and is
+        * not configured in the passthrough mode.
+        */
+       if (no_iommu || iommu_default_passthrough()) {
+               pr_err("SEV-SNP: IOMMU is either disabled or configured in passthrough mode.\n");

Like below could this say something like snp support is disabled because of iommu settings.

drivers/iommu/amd/init.c int amd_iommu_pc_set_reg(struct amd_iommu *iommu, u8 bank, u8 cntr, u8 fxn, u64 · line 3384

Quoted 10 lines
drivers/iommu/amd/init.c +10 -0
+               return false;
+       }
+
+       /*
+        * Iterate through all the IOMMUs and verify the SNPSup feature is
+        * enabled.
+        */
+       for_each_iommu(iommu) {
+               if (!iommu_feature(iommu, FEATURE_SNP)) {
+                       pr_err("SNPSup is disabled (devid: %02x:%02x.%x)\n",

SNPSup might not be obvious to readers, what about " SNP Support is disabled ...".

Also should this have the "SEV-SNP:" prefix like the above log?

Quoted 39 lines
drivers/iommu/amd/init.c +9 -0
+                              PCI_BUS_NUM(iommu->devid), PCI_SLOT(iommu->devid),
+                              PCI_FUNC(iommu->devid));
+                       return false;
+               }
+       }
+
+       return true;
+}
+EXPORT_SYMBOL_GPL(iommu_sev_snp_supported);
include/linux/iommu.h +9 -1
index 9208eca4b0d1..fecb72e1b11b 100644
@@ -675,6 +675,12 @@ struct iommu_sva *iommu_sva_bind_device(struct device *dev,
 void iommu_sva_unbind_device(struct iommu_sva *handle);
 u32 iommu_sva_get_pasid(struct iommu_sva *handle);
 
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+bool iommu_sev_snp_supported(void);
+#else
+static inline bool iommu_sev_snp_supported(void) { return false; }
+#endif
+
 #else /* CONFIG_IOMMU_API */
 
 struct iommu_ops {};
@@ -1031,6 +1037,9 @@ static inline struct iommu_fwspec *dev_iommu_fwspec_get(struct device *dev)
 {
        return NULL;
 }
+
+static inline bool iommu_sev_snp_supported(void) { return false; }
+
 #endif /* CONFIG_IOMMU_API */
 
 /**
--

2.25.1

#17 lore raw

Re: [PATCH Part2 v6 03/49] x86/sev: Add the host SEV-SNP initialization support

In reply to Ashish Kalra
Quoted 219 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

The memory integrity guarantees of SEV-SNP are enforced through a new structure called the Reverse Map Table (RMP). The RMP is a single data structure shared across the system that contains one entry for every 4K page of DRAM that may be used by SEV-SNP VMs. The goal of RMP is to track the owner of each page of memory. Pages of memory can be owned by the hypervisor, owned by a specific VM or owned by the AMD-SP. See APM2 section 15.36.3 for more detail on RMP.

The RMP table is used to enforce access control to memory. The table itself is not directly writable by the software. New CPU instructions (RMPUPDATE, PVALIDATE, RMPADJUST) are used to manipulate the RMP entries.

Based on the platform configuration, the BIOS reserves the memory used for the RMP table. The start and end address of the RMP table must be queried by reading the RMP_BASE and RMP_END MSRs. If the RMP_BASE and RMP_END are not set then disable the SEV-SNP feature.

The SEV-SNP feature is enabled only after the RMP table is successfully initialized.

---
 arch/x86/include/asm/disabled-features.h |   8 +-
 arch/x86/include/asm/msr-index.h         |   6 +
 arch/x86/kernel/sev.c                    | 144 +++++++++++++++++++++++
 3 files changed, 157 insertions(+), 1 deletion(-)
arch/x86/include/asm/disabled-features.h +7 -1
index 36369e76cc63..c1be3091a383 100644
@@ -68,6 +68,12 @@
 # define DISABLE_TDX_GUEST     (1 << (X86_FEATURE_TDX_GUEST & 31))
 #endif
 
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+# define DISABLE_SEV_SNP       0
+#else
+# define DISABLE_SEV_SNP       (1 << (X86_FEATURE_SEV_SNP & 31))
+#endif
+
 /*
  * Make sure to add features to the correct mask
  */
@@ -91,7 +97,7 @@
                         DISABLE_ENQCMD)
 #define DISABLED_MASK17        0
 #define DISABLED_MASK18        0
-#define DISABLED_MASK19        0
+#define DISABLED_MASK19        (DISABLE_SEV_SNP)
 #define DISABLED_MASK_CHECK BUILD_BUG_ON_ZERO(NCAPINTS != 20)
 
 #endif /* _ASM_X86_DISABLED_FEATURES_H */
arch/x86/include/asm/msr-index.h +6 -0
index 9e2e7185fc1d..57a8280e283a 100644
@@ -507,6 +507,8 @@
 #define MSR_AMD64_SEV_ENABLED          BIT_ULL(MSR_AMD64_SEV_ENABLED_BIT)
 #define MSR_AMD64_SEV_ES_ENABLED       BIT_ULL(MSR_AMD64_SEV_ES_ENABLED_BIT)
 #define MSR_AMD64_SEV_SNP_ENABLED      BIT_ULL(MSR_AMD64_SEV_SNP_ENABLED_BIT)
+#define MSR_AMD64_RMP_BASE             0xc0010132
+#define MSR_AMD64_RMP_END              0xc0010133
 
 #define MSR_AMD64_VIRT_SPEC_CTRL       0xc001011f
 
@@ -581,6 +583,10 @@
 #define MSR_AMD64_SYSCFG               0xc0010010
 #define MSR_AMD64_SYSCFG_MEM_ENCRYPT_BIT       23
 #define MSR_AMD64_SYSCFG_MEM_ENCRYPT   BIT_ULL(MSR_AMD64_SYSCFG_MEM_ENCRYPT_BIT)
+#define MSR_AMD64_SYSCFG_SNP_EN_BIT            24
+#define MSR_AMD64_SYSCFG_SNP_EN                BIT_ULL(MSR_AMD64_SYSCFG_SNP_EN_BIT)
+#define MSR_AMD64_SYSCFG_SNP_VMPL_EN_BIT       25
+#define MSR_AMD64_SYSCFG_SNP_VMPL_EN   BIT_ULL(MSR_AMD64_SYSCFG_SNP_VMPL_EN_BIT)
 #define MSR_K8_INT_PENDING_MSG         0xc0010055
 /* C1E active bits in int pending message */
 #define K8_INTP_C1E_ACTIVE_MASK                0x18000000
arch/x86/kernel/sev.c +103 -0
index f01f4550e2c6..3a233b5d47c5 100644
@@ -22,6 +22,8 @@
 #include <linux/efi.h>
 #include <linux/platform_device.h>
 #include <linux/io.h>
+#include <linux/cpumask.h>
+#include <linux/iommu.h>
 
 #include <asm/cpu_entry_area.h>
 #include <asm/stacktrace.h>
@@ -38,6 +40,7 @@
 #include <asm/apic.h>
 #include <asm/cpuid.h>
 #include <asm/cmdline.h>
+#include <asm/iommu.h>
 
 #define DR7_RESET_VALUE        0x400
 
@@ -57,6 +60,12 @@
 #define AP_INIT_CR0_DEFAULT            0x60000010
 #define AP_INIT_MXCSR_DEFAULT          0x1f80
 
+/*
+ * The first 16KB from the RMP_BASE is used by the processor for the
+ * bookkeeping, the range need to be added during the RMP entry lookup.
+ */
+#define RMPTABLE_CPU_BOOKKEEPING_SZ    0x4000
+
 /* For early boot hypervisor communication in SEV-ES enabled guests */
 static struct ghcb boot_ghcb_page __bss_decrypted __aligned(PAGE_SIZE);
 
@@ -69,6 +78,10 @@ static struct ghcb *boot_ghcb __section(".data");
 /* Bitmap of SEV features supported by the hypervisor */
 static u64 sev_hv_features __ro_after_init;
 
+static unsigned long rmptable_start __ro_after_init;
+static unsigned long rmptable_end __ro_after_init;
+
+
 /* #VC handler runtime per-CPU data */
 struct sev_es_runtime_data {
        struct ghcb ghcb_page;
@@ -2218,3 +2231,134 @@ static int __init snp_init_platform_device(void)
        return 0;
 }
 device_initcall(snp_init_platform_device);
+
+#undef pr_fmt
+#define pr_fmt(fmt)    "SEV-SNP: " fmt
+
+static int __snp_enable(unsigned int cpu)
+{
+       u64 val;
+
+       if (!cpu_feature_enabled(X86_FEATURE_SEV_SNP))
+               return 0;
+
+       rdmsrl(MSR_AMD64_SYSCFG, val);
+
+       val |= MSR_AMD64_SYSCFG_SNP_EN;
+       val |= MSR_AMD64_SYSCFG_SNP_VMPL_EN;
+
+       wrmsrl(MSR_AMD64_SYSCFG, val);
+
+       return 0;
+}
+
+static __init void snp_enable(void *arg)
+{
+       __snp_enable(smp_processor_id());
+}
+
+static bool get_rmptable_info(u64 *start, u64 *len)
+{
+       u64 calc_rmp_sz, rmp_sz, rmp_base, rmp_end, nr_pages;
+
+       rdmsrl(MSR_AMD64_RMP_BASE, rmp_base);
+       rdmsrl(MSR_AMD64_RMP_END, rmp_end);
+
+       if (!rmp_base || !rmp_end) {
+               pr_info("Memory for the RMP table has not been reserved by BIOS\n");
+               return false;
+       }
+
+       rmp_sz = rmp_end - rmp_base + 1;
+
+       /*
+        * Calculate the amount the memory that must be reserved by the BIOS to
+        * address the full system RAM. The reserved memory should also cover the
+        * RMP table itself.
+        *
+        * See PPR Family 19h Model 01h, Revision B1 section 2.1.4.2 for more
+        * information on memory requirement.
+        */
+       nr_pages = totalram_pages();
+       calc_rmp_sz = (((rmp_sz >> PAGE_SHIFT) + nr_pages) << 4) + RMPTABLE_CPU_BOOKKEEPING_SZ;
+
+       if (calc_rmp_sz > rmp_sz) {
+               pr_info("Memory reserved for the RMP table does not cover full system RAM (expected 0x%llx got 0x%llx)\n",
+                       calc_rmp_sz, rmp_sz);
+               return false;
+       }
+
+       *start = rmp_base;
+       *len = rmp_sz;
+
+       pr_info("RMP table physical address 0x%016llx - 0x%016llx\n", rmp_base, rmp_end);
+
+       return true;
+}
+
+static __init int __snp_rmptable_init(void)
+{
+       u64 rmp_base, sz;
+       void *start;
+       u64 val;
+
+       if (!get_rmptable_info(&rmp_base, &sz))
+               return 1;
+
+       start = memremap(rmp_base, sz, MEMREMAP_WB);
+       if (!start) {
+               pr_err("Failed to map RMP table 0x%llx+0x%llx\n", rmp_base, sz);
+               return 1;
+       }
+
+       /*
+        * Check if SEV-SNP is already enabled, this can happen if we are coming from
+        * kexec boot.
+        */
+       rdmsrl(MSR_AMD64_SYSCFG, val);
+       if (val & MSR_AMD64_SYSCFG_SNP_EN)
+               goto skip_enable;
+
+       /* Initialize the RMP table to zero */
+       memset(start, 0, sz);

arch/x86/kernel/sev.c static int __init snp_init_platform_device(void) · line 2333

Quoted 10 lines
arch/x86/kernel/sev.c +10 -0
+
+       /* Flush the caches to ensure that data is written before SNP is enabled. */
+       wbinvd_on_all_cpus();
+
+       /* Enable SNP on all CPUs. */
+       on_each_cpu(snp_enable, NULL, 1);
+
+skip_enable:
+       rmptable_start = (unsigned long)start;
+       rmptable_end = rmptable_start + sz;

Since in get_rmptable_info() rmp_sz = rmp_end - rmp_base + 1; should this be rmptable_end = rmptable_start + sz - 1;?

Quoted 34 lines
arch/x86/kernel/sev.c +31 -1
+
+       return 0;
+}
+
+static int __init snp_rmptable_init(void)
+{
+       if (!boot_cpu_has(X86_FEATURE_SEV_SNP))
+               return 0;
+
+       if (!iommu_sev_snp_supported())
+               goto nosnp;
+
+       if (__snp_rmptable_init())
+               goto nosnp;
+
+       cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "x86/rmptable_init:online", __snp_enable, NULL);
+
+       return 0;
+
+nosnp:
+       setup_clear_cpu_cap(X86_FEATURE_SEV_SNP);
+       return 1;
+}
+
+/*
+ * This must be called after the PCI subsystem. This is because before enabling
+ * the SNP feature we need to ensure that IOMMU supports the SEV-SNP feature.
+ * The iommu_sev_snp_support() is used for checking the feature, and it is
+ * available after subsys_initcall().
+ */
+fs_initcall(snp_rmptable_init);
--

2.25.1

#58 lore raw

Re: [PATCH Part2 v6 06/49] x86/sev: Add helper functions for RMPUPDATE and PSMASH instruction

In reply to Ashish Kalra
Quoted 32 lines · * Ashish Kalra ([email protected]) wrote:

From: Brijesh Singh <[email protected]>

The RMPUPDATE instruction writes a new RMP entry in the RMP Table. The hypervisor will use the instruction to add pages to the RMP table. See APM3 for details on the instruction operations.

The PSMASH instruction expands a 2MB RMP entry into a corresponding set of contiguous 4KB-Page RMP entries. The hypervisor will use this instruction to adjust the RMP entry without invalidating the previous RMP entry.

---
 arch/x86/include/asm/sev.h | 11 ++++++
 arch/x86/kernel/sev.c      | 72 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 83 insertions(+)
arch/x86/include/asm/sev.h +2 -0
index cb16f0e5b585..6ab872311544 100644
@@ -85,7 +85,9 @@ extern bool handle_vc_boot_ghcb(struct pt_regs *regs);
 
 /* RMP page size */
 #define RMP_PG_SIZE_4K			0
+#define RMP_PG_SIZE_2M			1
 #define RMP_TO_X86_PG_LEVEL(level)	(((level) == RMP_PG_SIZE_4K) ? PG_LEVEL_4K : PG_LEVEL_2M)
+#define X86_TO_RMP_PG_LEVEL(level)	(((level) == PG_LEVEL_4K) ? RMP_PG_SIZE_4K : RMP_PG_SIZE_2M)
 
 /*
  * The RMP entry format is not architectural. The format is defined in PPR
@@ -126,6 +128,15 @@ struct snp_guest_platform_data {
 	u64 secrets_gpa;

arch/x86/include/asm/sev.h struct snp_guest_platform_data { · line 138

Quoted 10 lines
arch/x86/include/asm/sev.h +8 -0
 };
 
+struct rmpupdate {
+	u64 gpa;
+	u8 assigned;
+	u8 pagesize;
+	u8 immutable;
+	u8 rsvd;
+	u32 asid;
+} __packed;

I see above it says the RMP entry format isn't architectural; is this 'rmpupdate' structure? If not how is this going to get handled when we have a couple of SNP capable CPUs with different layouts?

Dave

Quoted 86 lines
arch/x86/include/asm/sev.h +0 -0
 #ifdef CONFIG_AMD_MEM_ENCRYPT
 extern struct static_key_false sev_es_enable_key;
 extern void __sev_es_ist_enter(struct pt_regs *regs);
arch/x86/kernel/sev.c +72 -0
index 59e7ec6b0326..f6c64a722e94 100644
@@ -2429,3 +2429,75 @@ int snp_lookup_rmpentry(u64 pfn, int *level)
 	return !!rmpentry_assigned(e);
 }
 EXPORT_SYMBOL_GPL(snp_lookup_rmpentry);
+
+int psmash(u64 pfn)
+{
+	unsigned long paddr = pfn << PAGE_SHIFT;
+	int ret;
+
+	if (!pfn_valid(pfn))
+		return -EINVAL;
+
+	if (!cpu_feature_enabled(X86_FEATURE_SEV_SNP))
+		return -ENXIO;
+
+	/* Binutils version 2.36 supports the PSMASH mnemonic. */
+	asm volatile(".byte 0xF3, 0x0F, 0x01, 0xFF"
+		      : "=a"(ret)
+		      : "a"(paddr)
+		      : "memory", "cc");
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(psmash);
+
+static int rmpupdate(u64 pfn, struct rmpupdate *val)
+{
+	unsigned long paddr = pfn << PAGE_SHIFT;
+	int ret;
+
+	if (!pfn_valid(pfn))
+		return -EINVAL;
+
+	if (!cpu_feature_enabled(X86_FEATURE_SEV_SNP))
+		return -ENXIO;
+
+	/* Binutils version 2.36 supports the RMPUPDATE mnemonic. */
+	asm volatile(".byte 0xF2, 0x0F, 0x01, 0xFE"
+		     : "=a"(ret)
+		     : "a"(paddr), "c"((unsigned long)val)
+		     : "memory", "cc");
+	return ret;
+}
+
+int rmp_make_private(u64 pfn, u64 gpa, enum pg_level level, int asid, bool immutable)
+{
+	struct rmpupdate val;
+
+	if (!pfn_valid(pfn))
+		return -EINVAL;
+
+	memset(&val, 0, sizeof(val));
+	val.assigned = 1;
+	val.asid = asid;
+	val.immutable = immutable;
+	val.gpa = gpa;
+	val.pagesize = X86_TO_RMP_PG_LEVEL(level);
+
+	return rmpupdate(pfn, &val);
+}
+EXPORT_SYMBOL_GPL(rmp_make_private);
+
+int rmp_make_shared(u64 pfn, enum pg_level level)
+{
+	struct rmpupdate val;
+
+	if (!pfn_valid(pfn))
+		return -EINVAL;
+
+	memset(&val, 0, sizeof(val));
+	val.pagesize = X86_TO_RMP_PG_LEVEL(level);
+
+	return rmpupdate(pfn, &val);
+}
+EXPORT_SYMBOL_GPL(rmp_make_shared);
Signature · 1 line
2.25.1
Signature · 1 line
Dr. David Alan Gilbert / [email protected] / Manchester, UK
author maintainer
#59 lore raw

RE: [PATCH Part2 v6 06/49] x86/sev: Add helper functions for RMPUPDATE and PSMASH instruction

[AMD Official Use Only - General]

Hello Dave,

Quoted 6 lines
Quoted 6 lines

/*
* The RMP entry format is not architectural. The format is defined
in PPR @@ -126,6 +128,15 @@ struct snp_guest_platform_data {
u64 secrets_gpa;
};

Quoted 10 lines

arch/x86/include/asm/sev.h struct snp_guest_platform_data { · line 138

Quoted 8 lines
arch/x86/include/asm/sev.h +8 -0
+struct rmpupdate {
+	u64 gpa;
+	u8 assigned;
+	u8 pagesize;
+	u8 immutable;
+	u8 rsvd;
+	u32 asid;
+} __packed;

I see above it says the RMP entry format isn't architectural; is this 'rmpupdate' structure? If not how is this going to get handled when we have a couple of SNP capable CPUs with different layouts?

Architectural implies that it is defined in the APM and shouldn't change in such a way as to not be backward compatible. I probably think the wording here should be architecture independent or more precisely platform independent.

Thanks,
Ashish

author maintainer
#5 lore raw

RE: [PATCH Part2 v6 02/49] iommu/amd: Introduce function to check SEV-SNP support

In reply to Peter Gonda

[AMD Official Use Only - General]

Hello Peter,

Quoted 5 lines
Quoted 5 lines
drivers/iommu/amd/init.c +5 -0
+bool iommu_sev_snp_supported(void)
+{
+       struct amd_iommu *iommu;
+
+       /*
Quoted 8 lines

drivers/iommu/amd/init.c int amd_iommu_pc_set_reg(struct amd_iommu *iommu, u8 bank, u8 cntr, u8 fxn, u64

Quoted 6 lines
drivers/iommu/amd/init.c +6 -0
+        * The SEV-SNP support requires that IOMMU must be enabled, and is
+        * not configured in the passthrough mode.
+        */
+       if (no_iommu || iommu_default_passthrough()) {
+               pr_err("SEV-SNP: IOMMU is either disabled or 
+ configured in passthrough mode.\n");

Like below could this say something like snp support is disabled because of iommu settings.

Here we may need to be more precise with the error information indicating why SNP is not enabled. Please note that this patch may actually become part of the IOMMU + SNP patch series, where additional checks are done, for example, not enabling SNP if IOMMU v2 page tables are enabled, so precise error information will be useful here.

Quoted 12 lines

drivers/iommu/amd/init.c int amd_iommu_pc_set_reg(struct amd_iommu *iommu, u8 bank, u8 cntr, u8 fxn, u64

Quoted 11 lines
drivers/iommu/amd/init.c +11 -0
+               return false;
+       }
+
+       /*
+        * Iterate through all the IOMMUs and verify the SNPSup feature is
+        * enabled.
+        */
+       for_each_iommu(iommu) {
+               if (!iommu_feature(iommu, FEATURE_SNP)) {
+                       pr_err("SNPSup is disabled (devid: 
+ %02x:%02x.%x)\n",
Quoted 1 line

SNPSup might not be obvious to readers, what about " SNP Support is disabled ...".

Yes, that makes sense.

Quoted 1 line

Also should this have the "SEV-SNP:" prefix like the above log?

Probably, we want to be more consistent with the SNP guest patches and replace SEV-SNP with SNP everywhere, including function names, etc.

Thanks,
Ashish

#6 lore raw

Re: [PATCH Part2 v6 02/49] iommu/amd: Introduce function to check SEV-SNP support

Quoted 19 lines · Ashish

[AMD Official Use Only - General]

Hello Peter,

Quoted 5 lines
Quoted 5 lines
drivers/iommu/amd/init.c +5 -0
+bool iommu_sev_snp_supported(void)
+{
+       struct amd_iommu *iommu;
+
+       /*
Quoted 8 lines

drivers/iommu/amd/init.c int amd_iommu_pc_set_reg(struct amd_iommu *iommu, u8 bank, u8 cntr, u8 fxn, u64

Quoted 6 lines
drivers/iommu/amd/init.c +6 -0
+        * The SEV-SNP support requires that IOMMU must be enabled, and is
+        * not configured in the passthrough mode.
+        */
+       if (no_iommu || iommu_default_passthrough()) {
+               pr_err("SEV-SNP: IOMMU is either disabled or
+ configured in passthrough mode.\n");

Like below could this say something like snp support is disabled because of iommu settings.

Quoted 4 lines

Here we may need to be more precise with the error information indicating why SNP is not enabled. Please note that this patch may actually become part of the IOMMU + SNP patch series, where additional checks are done, for example, not enabling SNP if IOMMU v2 page tables are enabled, so precise error information will be useful here.

I agree we should be more precise. I just thought we should explicitly state something like: "SEV-SNP: IOMMU is either disabled or configured in passthrough mode, SNP cannot be supported".

author maintainer
#18 lore raw

RE: [PATCH Part2 v6 03/49] x86/sev: Add the host SEV-SNP initialization support

In reply to Peter Gonda

[Public]

Hello Peter,

Quoted 28 lines
Quoted 28 lines
arch/x86/kernel/sev.c +28 -0
+static __init int __snp_rmptable_init(void) {
+       u64 rmp_base, sz;
+       void *start;
+       u64 val;
+
+       if (!get_rmptable_info(&rmp_base, &sz))
+               return 1;
+
+       start = memremap(rmp_base, sz, MEMREMAP_WB);
+       if (!start) {
+               pr_err("Failed to map RMP table 0x%llx+0x%llx\n", rmp_base, sz);
+               return 1;
+       }
+
+       /*
+        * Check if SEV-SNP is already enabled, this can happen if we are coming from
+        * kexec boot.
+        */
+       rdmsrl(MSR_AMD64_SYSCFG, val);
+       if (val & MSR_AMD64_SYSCFG_SNP_EN)
+               goto skip_enable;
+
+       /* Initialize the RMP table to zero */
+       memset(start, 0, sz);
+
+       /* Flush the caches to ensure that data is written before SNP is enabled. */
+       wbinvd_on_all_cpus();
+
Quoted 8 lines

arch/x86/kernel/sev.c static int __init snp_init_platform_device(void) · line 2333

Quoted 6 lines
arch/x86/kernel/sev.c +6 -0
+       /* Enable SNP on all CPUs. */
+       on_each_cpu(snp_enable, NULL, 1);
+
+skip_enable:
+       rmptable_start = (unsigned long)start;
+       rmptable_end = rmptable_start + sz;

Since in get_rmptable_info() rmp_sz = rmp_end - rmp_base + 1; should this be rmptable_end = rmptable_start + sz - 1;?

Yes, it should be.

Thanks,
Ashish

#157 lore raw

Re: [PATCH Part2 v6 14/49] crypto: ccp: Handle the legacy TMR allocation when SNP is enabled

In reply to Ashish Kalra
Quoted 9 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

The behavior and requirement for the SEV-legacy command is altered when the SNP firmware is in the INIT state. See SEV-SNP firmware specification for more details.

Allocate the Trusted Memory Region (TMR) as a 2mb sized/aligned region when SNP is enabled to satify new requirements for the SNP. Continue

satisfy

Quoted 19 lines

allocating a 1mb region for !SNP configuration.

While at it, provide API that can be used by others to allocate a page that can be used by the firmware. The immediate user for this API will be the KVM driver. The KVM driver to need to allocate a firmware context page during the guest creation. The context page need to be updated by the firmware. See the SEV-SNP specification for further details.

---
 drivers/crypto/ccp/sev-dev.c | 173 +++++++++++++++++++++++++++++++++--
 include/linux/psp-sev.h      |  11 +++
 2 files changed, 178 insertions(+), 6 deletions(-)
drivers/crypto/ccp/sev-dev.c +0 -0
index 35d76333e120..0dbd99f29b25 100644

drivers/crypto/ccp/sev-dev.c static void *sev_es_tmr; · line 85

Quoted 7 lines
drivers/crypto/ccp/sev-dev.c +4 -0
@@ -79,6 +79,14 @@ static void *sev_es_tmr;
 #define NV_LENGTH (32 * 1024)
 static void *sev_init_ex_buffer;
 
+/* When SEV-SNP is enabled the TMR needs to be 2MB aligned and 2MB size. */
+#define SEV_SNP_ES_TMR_SIZE    (2 * 1024 * 1024)
+
+static size_t sev_es_tmr_size = SEV_ES_TMR_SIZE;

Why not keep all this TMR stuff together near the SEV_ES_TMR_SIZE define?

Quoted 17 lines
drivers/crypto/ccp/sev-dev.c +10 -0
+
+static int __sev_do_cmd_locked(int cmd, void *data, int *psp_ret);
+static int sev_do_cmd(int cmd, void *data, int *psp_ret);
+
 static inline bool sev_version_greater_or_equal(u8 maj, u8 min)
 {
        struct sev_device *sev = psp_master->sev_data;
@@ -177,11 +185,161 @@ static int sev_cmd_buffer_len(int cmd)
        return 0;
 }
 
+static void snp_leak_pages(unsigned long pfn, unsigned int npages)
+{
+       WARN(1, "psc failed, pfn 0x%lx pages %d (leaking)\n", pfn, npages);
+       while (npages--) {
+               memory_failure(pfn, 0);
+               dump_rmpentry(pfn);

drivers/crypto/ccp/sev-dev.c static int sev_cmd_buffer_len(int cmd) · line 203

Quoted 10 lines
drivers/crypto/ccp/sev-dev.c +10 -0
+               pfn++;
+       }
+}
+
+static int snp_reclaim_pages(unsigned long pfn, unsigned int npages, bool locked)
+{
+       struct sev_data_snp_page_reclaim data;
+       int ret, err, i, n = 0;
+
+       for (i = 0; i < npages; i++) {

What about setting |n| here too, also the other increments.

for (i = 0, n = 0; i < npages; i++, n++, pfn++)

drivers/crypto/ccp/sev-dev.c static int sev_cmd_buffer_len(int cmd) · line 210

Quoted 7 lines
drivers/crypto/ccp/sev-dev.c +7 -0
+               memset(&data, 0, sizeof(data));
+               data.paddr = pfn << PAGE_SHIFT;
+
+               if (locked)
+                       ret = __sev_do_cmd_locked(SEV_CMD_SNP_PAGE_RECLAIM, &data, &err);
+               else
+                       ret = sev_do_cmd(SEV_CMD_SNP_PAGE_RECLAIM, &data, &err);

Can we change sev_cmd_mutex to some sort of nesting lock type? That could clean up this if (locked) code.

Quoted 19 lines
drivers/crypto/ccp/sev-dev.c +19 -0
+               if (ret)
+                       goto cleanup;
+
+               ret = rmp_make_shared(pfn, PG_LEVEL_4K);
+               if (ret)
+                       goto cleanup;
+
+               pfn++;
+               n++;
+       }
+
+       return 0;
+
+cleanup:
+       /*
+        * If failed to reclaim the page then page is no longer safe to
+        * be released, leak it.
+        */
+       snp_leak_pages(pfn, npages - n);

drivers/crypto/ccp/sev-dev.c static int sev_cmd_buffer_len(int cmd) · line 239

Quoted 10 lines
drivers/crypto/ccp/sev-dev.c +10 -0
+       return ret;
+}
+
+static inline int rmp_make_firmware(unsigned long pfn, int level)
+{
+       return rmp_make_private(pfn, 0, level, 0, true);
+}
+
+static int snp_set_rmp_state(unsigned long paddr, unsigned int npages, bool to_fw, bool locked,
+                            bool need_reclaim)

This function can do a lot and when I read the call sites its hard to see what its doing since we have a combination of arguments which tell us what behavior is happening, some of which are not valid (ex: to_fw == true and need_reclaim == true is an invalid argument combination). Also this for loop over |npages| is duplicated from snp_reclaim_pages(). One improvement here is that on the current snp_reclaim_pages() if we fail to reclaim a page we assume we cannot reclaim the next pages, this may cause us to snp_leak_pages() more pages than we actually need too.

What about something like this?

static snp_leak_page(u64 pfn, enum pg_level level) {
memory_failure(pfn, 0); dump_rmpentry(pfn);
}

static int snp_reclaim_page(u64 pfn, enum pg_level level) {
int ret;
struct sev_data_snp_page_reclaim data;

  ret = sev_do_cmd(SEV_CMD_SNP_PAGE_RECLAIM, &data, &err);
  if (ret)
    goto cleanup;

  ret = rmp_make_shared(pfn, level);
  if (ret)
    goto cleanup;

return 0;

cleanup:
snp_leak_page(pfn, level)
}

typedef int (*rmp_state_change_func) (u64 pfn, enum pg_level level);

static int snp_set_rmp_state(unsigned long paddr, unsigned int npages, rmp_state_change_func state_change, rmp_state_change_func cleanup) {
struct sev_data_snp_page_reclaim data;
int ret, err, i, n = 0;

  for (i = 0, n = 0; i < npages; i++, n++, pfn++) {
    ret = state_change(pfn, PG_LEVEL_4K)
    if (ret)
      goto cleanup;
  }

return 0;

cleanup:

  for (; i>= 0; i--, n--, pfn--) {
    cleanup(pfn, PG_LEVEL_4K);

}

return ret;
}

Then inside of __snp_alloc_firmware_pages():

snp_set_rmp_state(paddr, npages, rmp_make_firmware, snp_reclaim_page);

And inside of __snp_free_firmware_pages():

snp_set_rmp_state(paddr, npages, snp_reclaim_page, snp_leak_page);

Just a suggestion feel free to ignore. The readability comment could be addressed much less invasively by just making separate functions for each valid combination of arguments here. Like snp_set_rmp_fw_state(), snp_set_rmp_shared_state(), snp_set_rmp_release_state() or something.

Quoted 51 lines
drivers/crypto/ccp/sev-dev.c +51 -0
+{
+       unsigned long pfn = __sme_clr(paddr) >> PAGE_SHIFT; /* Cbit maybe set in the paddr */
+       int rc, n = 0, i;
+
+       for (i = 0; i < npages; i++) {
+               if (to_fw)
+                       rc = rmp_make_firmware(pfn, PG_LEVEL_4K);
+               else
+                       rc = need_reclaim ? snp_reclaim_pages(pfn, 1, locked) :
+                                           rmp_make_shared(pfn, PG_LEVEL_4K);
+               if (rc)
+                       goto cleanup;
+
+               pfn++;
+               n++;
+       }
+
+       return 0;
+
+cleanup:
+       /* Try unrolling the firmware state changes */
+       if (to_fw) {
+               /*
+                * Reclaim the pages which were already changed to the
+                * firmware state.
+                */
+               snp_reclaim_pages(paddr >> PAGE_SHIFT, n, locked);
+
+               return rc;
+       }
+
+       /*
+        * If failed to change the page state to shared, then its not safe
+        * to release the page back to the system, leak it.
+        */
+       snp_leak_pages(pfn, npages - n);
+
+       return rc;
+}
+
+static struct page *__snp_alloc_firmware_pages(gfp_t gfp_mask, int order, bool locked)
+{
+       unsigned long npages = 1ul << order, paddr;
+       struct sev_device *sev;
+       struct page *page;
+
+       if (!psp_master || !psp_master->sev_data)
+               return NULL;
+
+       page = alloc_pages(gfp_mask, order);
+       if (!page)

drivers/crypto/ccp/sev-dev.c static int sev_cmd_buffer_len(int cmd) · line 300

Quoted 10 lines
drivers/crypto/ccp/sev-dev.c +10 -0
+               return NULL;
+
+       /* If SEV-SNP is initialized then add the page in RMP table. */
+       sev = psp_master->sev_data;
+       if (!sev->snp_inited)
+               return page;
+
+       paddr = __pa((unsigned long)page_address(page));
+       if (snp_set_rmp_state(paddr, npages, true, locked, false))
+               return NULL;

So what about the case where snp_set_rmp_state() fails but we were able to reclaim all the pages? Should we be able to signal that to callers so that we could free |page| here? But given this is an error path already maybe we can optimize this in a follow up series.

Quoted 14 lines
drivers/crypto/ccp/sev-dev.c +14 -0
+
+       return page;
+}
+
+void *snp_alloc_firmware_page(gfp_t gfp_mask)
+{
+       struct page *page;
+
+       page = __snp_alloc_firmware_pages(gfp_mask, 0, false);
+
+       return page ? page_address(page) : NULL;
+}
+EXPORT_SYMBOL_GPL(snp_alloc_firmware_page);
+

drivers/crypto/ccp/sev-dev.c static int sev_cmd_buffer_len(int cmd) · line 324

Quoted 10 lines
drivers/crypto/ccp/sev-dev.c +10 -0
+static void __snp_free_firmware_pages(struct page *page, int order, bool locked)
+{
+       unsigned long paddr, npages = 1ul << order;
+
+       if (!page)
+               return;
+
+       paddr = __pa((unsigned long)page_address(page));
+       if (snp_set_rmp_state(paddr, npages, false, locked, true))
+               return;

Here we may be able to free some of |page| depending how where inside of snp_set_rmp_state() we failed. But again given this is an error path already maybe we can optimize this in a follow up series.

Quoted 109 lines
drivers/crypto/ccp/sev-dev.c +22 -6
+
+       __free_pages(page, order);
+}
+
+void snp_free_firmware_page(void *addr)
+{
+       if (!addr)
+               return;
+
+       __snp_free_firmware_pages(virt_to_page(addr), 0, false);
+}
+EXPORT_SYMBOL(snp_free_firmware_page);
+
 static void *sev_fw_alloc(unsigned long len)
 {
        struct page *page;
 
-       page = alloc_pages(GFP_KERNEL, get_order(len));
+       page = __snp_alloc_firmware_pages(GFP_KERNEL, get_order(len), false);
        if (!page)
                return NULL;
 
@@ -393,7 +551,7 @@ static int __sev_init_locked(int *error)
                data.tmr_address = __pa(sev_es_tmr);
 
                data.flags |= SEV_INIT_FLAGS_SEV_ES;
-               data.tmr_len = SEV_ES_TMR_SIZE;
+               data.tmr_len = sev_es_tmr_size;
        }
 
        return __sev_do_cmd_locked(SEV_CMD_INIT, &data, error);
@@ -421,7 +579,7 @@ static int __sev_init_ex_locked(int *error)
                data.tmr_address = __pa(sev_es_tmr);
 
                data.flags |= SEV_INIT_FLAGS_SEV_ES;
-               data.tmr_len = SEV_ES_TMR_SIZE;
+               data.tmr_len = sev_es_tmr_size;
        }
 
        return __sev_do_cmd_locked(SEV_CMD_INIT_EX, &data, error);
@@ -818,6 +976,8 @@ static int __sev_snp_init_locked(int *error)
        sev->snp_inited = true;
        dev_dbg(sev->dev, "SEV-SNP firmware initialized\n");
 
+       sev_es_tmr_size = SEV_SNP_ES_TMR_SIZE;
+
        return rc;
 }
 
@@ -1341,8 +1501,9 @@ static void sev_firmware_shutdown(struct sev_device *sev)
                /* The TMR area was encrypted, flush it from the cache */
                wbinvd_on_all_cpus();
 
-               free_pages((unsigned long)sev_es_tmr,
-                          get_order(SEV_ES_TMR_SIZE));
+               __snp_free_firmware_pages(virt_to_page(sev_es_tmr),
+                                         get_order(sev_es_tmr_size),
+                                         false);
                sev_es_tmr = NULL;
        }
 
@@ -1430,7 +1591,7 @@ void sev_pci_init(void)
        }
 
        /* Obtain the TMR memory area for SEV-ES use */
-       sev_es_tmr = sev_fw_alloc(SEV_ES_TMR_SIZE);
+       sev_es_tmr = sev_fw_alloc(sev_es_tmr_size);
        if (!sev_es_tmr)
                dev_warn(sev->dev,
                         "SEV: TMR allocation failed, SEV-ES support unavailable\n");
include/linux/psp-sev.h +11 -1
index 9f921d221b75..a3bb792bb842 100644
@@ -12,6 +12,8 @@
 #ifndef __PSP_SEV_H__
 #define __PSP_SEV_H__
 
+#include <linux/sev.h>
+
 #include <uapi/linux/psp-sev.h>
 
 #ifdef CONFIG_X86
@@ -940,6 +942,8 @@ int snp_guest_page_reclaim(struct sev_data_snp_page_reclaim *data, int *error);
 int snp_guest_dbg_decrypt(struct sev_data_snp_dbg *data, int *error);
 
 void *psp_copy_user_blob(u64 uaddr, u32 len);
+void *snp_alloc_firmware_page(gfp_t mask);
+void snp_free_firmware_page(void *addr);
 
 #else  /* !CONFIG_CRYPTO_DEV_SP_PSP */
 
@@ -981,6 +985,13 @@ static inline int snp_guest_dbg_decrypt(struct sev_data_snp_dbg *data, int *erro
        return -ENODEV;
 }
 
+static inline void *snp_alloc_firmware_page(gfp_t mask)
+{
+       return NULL;
+}
+
+static inline void snp_free_firmware_page(void *addr) { }
+
 #endif /* CONFIG_CRYPTO_DEV_SP_PSP */
 
 #endif /* __PSP_SEV_H__ */
--

2.25.1

author maintainer
#158 lore raw

RE: [PATCH Part2 v6 14/49] crypto: ccp: Handle the legacy TMR allocation when SNP is enabled

[Public]

Hello Peter,

Quoted 10 lines

drivers/crypto/ccp/sev-dev.c static int sev_cmd_buffer_len(int cmd) · line 203

Quoted 6 lines
drivers/crypto/ccp/sev-dev.c +6 -0
+static int snp_reclaim_pages(unsigned long pfn, unsigned int npages, 
+bool locked) {
+       struct sev_data_snp_page_reclaim data;
+       int ret, err, i, n = 0;
+
+       for (i = 0; i < npages; i++) {

What about setting |n| here too, also the other increments.

for (i = 0, n = 0; i < npages; i++, n++, pfn++)

Yes that is simpler.

Quoted 17 lines

drivers/crypto/ccp/sev-dev.c static int sev_cmd_buffer_len(int cmd)

Quoted 8 lines
drivers/crypto/ccp/sev-dev.c +8 -0
+               memset(&data, 0, sizeof(data));
+               data.paddr = pfn << PAGE_SHIFT;
+
+               if (locked)
+                       ret = __sev_do_cmd_locked(SEV_CMD_SNP_PAGE_RECLAIM, &data, &err);
+               else
+                       ret = sev_do_cmd(SEV_CMD_SNP_PAGE_RECLAIM, 
+ &data, &err);

Can we change sev_cmd_mutex to some sort of nesting lock type? That could clean up this if (locked) code.

drivers/crypto/ccp/sev-dev.c +5 -0
+static inline int rmp_make_firmware(unsigned long pfn, int level) {
+       return rmp_make_private(pfn, 0, level, 0, true); }
+
+static int snp_set_rmp_state(unsigned long paddr, unsigned int npages, bool to_fw, bool locked,
+                            bool need_reclaim)
Quoted 1 line

This function can do a lot and when I read the call sites its hard to see what its doing since we have a combination of arguments which tell us what behavior is happening, some of which are not valid (ex: to_fw == true and need_reclaim == true is an >invalid argument combination).

to_fw is used to make a firmware page and need_reclaim is for freeing the firmware page, so they are going to be mutually exclusive.

I actually can connect with it quite logically with the callers :
snp_alloc_firmware_pages will call with to_fw = true and need_reclaim = false and snp_free_firmware_pages will do the opposite, to_fw = false and need_reclaim = true.

That seems straightforward to look at.

Quoted 2 lines

Also this for loop over |npages| is duplicated from snp_reclaim_pages(). One improvement here is that on the current
snp_reclaim_pages() if we fail to reclaim a page we assume we cannot reclaim the next pages, this may cause us to snp_leak_pages() more pages than we actually need too.

Yes that is true.

Quoted 73 lines

What about something like this?

static snp_leak_page(u64 pfn, enum pg_level level) {
memory_failure(pfn, 0); dump_rmpentry(pfn);
}

static int snp_reclaim_page(u64 pfn, enum pg_level level) {
int ret;
struct sev_data_snp_page_reclaim data;

ret = sev_do_cmd(SEV_CMD_SNP_PAGE_RECLAIM, &data, &err); if (ret)
goto cleanup;

ret = rmp_make_shared(pfn, level);
if (ret)
goto cleanup;

return 0;

cleanup:
snp_leak_page(pfn, level)
}

typedef int (*rmp_state_change_func) (u64 pfn, enum pg_level level);

static int snp_set_rmp_state(unsigned long paddr, unsigned int npages, rmp_state_change_func state_change, rmp_state_change_func cleanup) {
struct sev_data_snp_page_reclaim data;
int ret, err, i, n = 0;

for (i = 0, n = 0; i < npages; i++, n++, pfn++) {
ret = state_change(pfn, PG_LEVEL_4K)
if (ret)

     goto cleanup;

}

return 0;

cleanup:

 for (; i>= 0; i--, n--, pfn--) {
   cleanup(pfn, PG_LEVEL_4K);

}

return ret;
}

Then inside of __snp_alloc_firmware_pages():

snp_set_rmp_state(paddr, npages, rmp_make_firmware, snp_reclaim_page);

And inside of __snp_free_firmware_pages():

snp_set_rmp_state(paddr, npages, snp_reclaim_page, snp_leak_page);

Just a suggestion feel free to ignore. The readability comment could be addressed much less invasively by just making separate functions for each valid combination of arguments here. Like snp_set_rmp_fw_state(), snp_set_rmp_shared_state(), snp_set_rmp_release_state() or something.

Quoted 15 lines
drivers/crypto/ccp/sev-dev.c +15 -0
+static struct page *__snp_alloc_firmware_pages(gfp_t gfp_mask, int 
+order, bool locked) {
+       unsigned long npages = 1ul << order, paddr;
+       struct sev_device *sev;
+       struct page *page;
+
+       if (!psp_master || !psp_master->sev_data)
+               return NULL;
+
+       page = alloc_pages(gfp_mask, order);
+       if (!page)
+               return NULL;
+
+       /* If SEV-SNP is initialized then add the page in RMP table. */
+       sev = psp_master->sev_data;
Quoted 8 lines

drivers/crypto/ccp/sev-dev.c static int sev_cmd_buffer_len(int cmd) · line 300

Quoted 6 lines
drivers/crypto/ccp/sev-dev.c +6 -0
+       if (!sev->snp_inited)
+               return page;
+
+       paddr = __pa((unsigned long)page_address(page));
+       if (snp_set_rmp_state(paddr, npages, true, locked, false))
+               return NULL;

So what about the case where snp_set_rmp_state() fails but we were able to reclaim all the pages? Should we be able to signal that to callers so that we could free |page| here? But given this is an error path already maybe we can optimize this in a >follow up series.

Yes, we should actually tie in to snp_reclaim_pages() success or failure here in the case we were able to successfully unroll some or all of the firmware state change.

Quoted 23 lines
drivers/crypto/ccp/sev-dev.c +22 -0
+
+       return page;
+}
+
+void *snp_alloc_firmware_page(gfp_t gfp_mask) {
+       struct page *page;
+
+       page = __snp_alloc_firmware_pages(gfp_mask, 0, false);
+
+       return page ? page_address(page) : NULL; } 
+EXPORT_SYMBOL_GPL(snp_alloc_firmware_page);
+
+static void __snp_free_firmware_pages(struct page *page, int order, 
+bool locked) {
+       unsigned long paddr, npages = 1ul << order;
+
+       if (!page)
+               return;
+
+       paddr = __pa((unsigned long)page_address(page));
+       if (snp_set_rmp_state(paddr, npages, false, locked, true))
+               return;
Quoted 1 line

Here we may be able to free some of |page| depending how where inside of snp_set_rmp_state() we failed. But again given this is an error path already maybe we can optimize this in a follow up series.

Yes, we probably should be able to free some of the page(s) depending on how many page(s) got reclaimed in snp_set_rmp_state().
But these reclamation failures may not be very common, so any failure is indicative of a bigger issue, it might be the case when there is a single page reclamation error it might happen with all the subsequent pages and so follow a simple recovery procedure, then handling a more complex recovery for a chunk of pages being reclaimed and another chunk not.

Thanks,
Ashish

#144 lore raw

Re: [PATCH Part2 v6 13/49] crypto:ccp: Provide APIs to issue SEV-SNP commands

In reply to Ashish Kalra

(

Quoted 21 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

Provide the APIs for the hypervisor to manage an SEV-SNP guest. The commands for SEV-SNP is defined in the SEV-SNP firmware specification.

---
 drivers/crypto/ccp/sev-dev.c | 24 ++++++++++++
 include/linux/psp-sev.h      | 73 ++++++++++++++++++++++++++++++++++++
 2 files changed, 97 insertions(+)
drivers/crypto/ccp/sev-dev.c +1 -0
index f1173221d0b9..35d76333e120 100644
@@ -1205,6 +1205,30 @@ int sev_guest_df_flush(int *error)
 }
 EXPORT_SYMBOL_GPL(sev_guest_df_flush);
 
+int snp_guest_decommission(struct sev_data_snp_decommission *data, int *error)

drivers/crypto/ccp/sev-dev.c int sev_guest_df_flush(int *error) · line 1218

Quoted 10 lines
drivers/crypto/ccp/sev-dev.c +10 -0
+{
+       return sev_do_cmd(SEV_CMD_SNP_DECOMMISSION, data, error);
+}
+EXPORT_SYMBOL_GPL(snp_guest_decommission);
+
+int snp_guest_df_flush(int *error)
+{
+       return sev_do_cmd(SEV_CMD_SNP_DF_FLUSH, NULL, error);
+}
+EXPORT_SYMBOL_GPL(snp_guest_df_flush);

Why not instead change sev_guest_df_flush() to be SNP aware? That way callers get the right behavior without having to know if SNP is enabled or not.

int sev_guest_df_flush(int *error)
{
if (!psp_master || !psp_master->sev_data)
return -EINVAL;

  if (psp_master->sev_data->snp_inited)
    return sev_do_cmd(SEV_CMD_SNP_DF_FLUSH, NULL, error);

return sev_do_cmd(SEV_CMD_DF_FLUSH, NULL, error);
}

Quoted 109 lines
drivers/crypto/ccp/sev-dev.c +12 -0
+int snp_guest_page_reclaim(struct sev_data_snp_page_reclaim *data, int *error)
+{
+       return sev_do_cmd(SEV_CMD_SNP_PAGE_RECLAIM, data, error);
+}
+EXPORT_SYMBOL_GPL(snp_guest_page_reclaim);
+
+int snp_guest_dbg_decrypt(struct sev_data_snp_dbg *data, int *error)
+{
+       return sev_do_cmd(SEV_CMD_SNP_DBG_DECRYPT, data, error);
+}
+EXPORT_SYMBOL_GPL(snp_guest_dbg_decrypt);
+
 static void sev_exit(struct kref *ref)
 {
        misc_deregister(&misc_dev->misc);
include/linux/psp-sev.h +73 -1
index ef4d42e8c96e..9f921d221b75 100644
@@ -881,6 +881,64 @@ int sev_guest_df_flush(int *error);
  */
 int sev_guest_decommission(struct sev_data_decommission *data, int *error);
 
+/**
+ * snp_guest_df_flush - perform SNP DF_FLUSH command
+ *
+ * @sev_ret: sev command return code
+ *
+ * Returns:
+ * 0 if the sev successfully processed the command
+ * -%ENODEV    if the sev device is not available
+ * -%ENOTSUPP  if the sev does not support SEV
+ * -%ETIMEDOUT if the sev command timed out
+ * -%EIO       if the sev returned a non-zero return code
+ */
+int snp_guest_df_flush(int *error);
+
+/**
+ * snp_guest_decommission - perform SNP_DECOMMISSION command
+ *
+ * @decommission: sev_data_decommission structure to be processed
+ * @sev_ret: sev command return code
+ *
+ * Returns:
+ * 0 if the sev successfully processed the command
+ * -%ENODEV    if the sev device is not available
+ * -%ENOTSUPP  if the sev does not support SEV
+ * -%ETIMEDOUT if the sev command timed out
+ * -%EIO       if the sev returned a non-zero return code
+ */
+int snp_guest_decommission(struct sev_data_snp_decommission *data, int *error);
+
+/**
+ * snp_guest_page_reclaim - perform SNP_PAGE_RECLAIM command
+ *
+ * @decommission: sev_snp_page_reclaim structure to be processed
+ * @sev_ret: sev command return code
+ *
+ * Returns:
+ * 0 if the sev successfully processed the command
+ * -%ENODEV    if the sev device is not available
+ * -%ENOTSUPP  if the sev does not support SEV
+ * -%ETIMEDOUT if the sev command timed out
+ * -%EIO       if the sev returned a non-zero return code
+ */
+int snp_guest_page_reclaim(struct sev_data_snp_page_reclaim *data, int *error);
+
+/**
+ * snp_guest_dbg_decrypt - perform SEV SNP_DBG_DECRYPT command
+ *
+ * @sev_ret: sev command return code
+ *
+ * Returns:
+ * 0 if the sev successfully processed the command
+ * -%ENODEV    if the sev device is not available
+ * -%ENOTSUPP  if the sev does not support SEV
+ * -%ETIMEDOUT if the sev command timed out
+ * -%EIO       if the sev returned a non-zero return code
+ */
+int snp_guest_dbg_decrypt(struct sev_data_snp_dbg *data, int *error);
+
 void *psp_copy_user_blob(u64 uaddr, u32 len);
 
 #else  /* !CONFIG_CRYPTO_DEV_SP_PSP */
@@ -908,6 +966,21 @@ sev_issue_cmd_external_user(struct file *filep, unsigned int id, void *data, int
 
 static inline void *psp_copy_user_blob(u64 __user uaddr, u32 len) { return ERR_PTR(-EINVAL); }
 
+static inline int
+snp_guest_decommission(struct sev_data_snp_decommission *data, int *error) { return -ENODEV; }
+
+static inline int snp_guest_df_flush(int *error) { return -ENODEV; }
+
+static inline int snp_guest_page_reclaim(struct sev_data_snp_page_reclaim *data, int *error)
+{
+       return -ENODEV;
+}
+
+static inline int snp_guest_dbg_decrypt(struct sev_data_snp_dbg *data, int *error)
+{
+       return -ENODEV;
+}
+
 #endif /* CONFIG_CRYPTO_DEV_SP_PSP */
 
 #endif /* __PSP_SEV_H__ */
--

2.25.1

#201 lore raw

Re: [PATCH Part2 v6 17/49] crypto: ccp: Add the SNP_{SET,GET}_EXT_CONFIG command

In reply to Ashish Kalra
Quoted 42 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

The SEV-SNP firmware provides the SNP_CONFIG command used to set the system-wide configuration value for SNP guests. The information includes the TCB version string to be reported in guest attestation reports.

Version 2 of the GHCB specification adds an NAE (SNP extended guest request) that a guest can use to query the reports that include additional certificates.

In both cases, userspace provided additional data is included in the attestation reports. The userspace will use the SNP_SET_EXT_CONFIG command to give the certificate blob and the reported TCB version string at once. Note that the specification defines certificate blob with a specific GUID format; the userspace is responsible for building the proper certificate blob. The ioctl treats it an opaque blob.

While it is not defined in the spec, but let's add SNP_GET_EXT_CONFIG command that can be used to obtain the data programmed through the SNP_SET_EXT_CONFIG.

---
 Documentation/virt/coco/sevguest.rst |  27 +++++++
 drivers/crypto/ccp/sev-dev.c         | 115 +++++++++++++++++++++++++++
 drivers/crypto/ccp/sev-dev.h         |   3 +
 include/uapi/linux/psp-sev.h         |  17 ++++
 4 files changed, 162 insertions(+)
Documentation/virt/coco/sevguest.rst +4 -0
index 11ea67c944df..3014de47e4ce 100644
@@ -145,6 +145,33 @@ The SNP_PLATFORM_STATUS command is used to query the SNP platform status. The
 status includes API major, minor version and more. See the SEV-SNP
 specification for further details.
 
+2.5 SNP_SET_EXT_CONFIG
+----------------------
+:Technology: sev-snp
+:Type: hypervisor ioctl cmd

Documentation/virt/coco/sevguest.rst The SNP_PLATFORM_STATUS command is used to query the SNP platform status. The · line 161

Quoted 10 lines
Documentation/virt/coco/sevguest.rst +10 -0
+:Parameters (in): struct sev_data_snp_ext_config
+:Returns (out): 0 on success, -negative on error
+
+The SNP_SET_EXT_CONFIG is used to set the system-wide configuration such as
+reported TCB version in the attestation report. The command is similar to
+SNP_CONFIG command defined in the SEV-SNP spec. The main difference is the
+command also accepts an additional certificate blob defined in the GHCB
+specification.
+
+If the certs_address is zero, then previous certificate blob will deleted.

... then the previous certificate blob will be deleted.

Quoted 32 lines
Documentation/virt/coco/sevguest.rst +13 -0
+For more information on the certificate blob layout, see the GHCB spec
+(extended guest request message).
+
+2.6 SNP_GET_EXT_CONFIG
+----------------------
+:Technology: sev-snp
+:Type: hypervisor ioctl cmd
+:Parameters (in): struct sev_data_snp_ext_config
+:Returns (out): 0 on success, -negative on error
+
+The SNP_SET_EXT_CONFIG is used to query the system-wide configuration set
+through the SNP_SET_EXT_CONFIG.
+
 3. SEV-SNP CPUID Enforcement
 ============================
 
drivers/crypto/ccp/sev-dev.c +4 -0
index b9b6fab31a82..97b479d5aa86 100644
@@ -1312,6 +1312,10 @@ static int __sev_snp_shutdown_locked(int *error)
        if (!sev->snp_inited)
                return 0;
 
+       /* Free the memory used for caching the certificate data */
+       kfree(sev->snp_certs_data);
+       sev->snp_certs_data = NULL;
+
        /* SHUTDOWN requires the DF_FLUSH */
        wbinvd_on_all_cpus();
        __sev_do_cmd_locked(SEV_CMD_SNP_DF_FLUSH, NULL, NULL);

drivers/crypto/ccp/sev-dev.c static int sev_ioctl_snp_platform_status(struct sev_issue_cmd *argp) · line 1626

Quoted 7 lines
drivers/crypto/ccp/sev-dev.c +4 -0
@@ -1616,6 +1620,111 @@ static int sev_ioctl_snp_platform_status(struct sev_issue_cmd *argp)
        return ret;
 }
 
+static int sev_ioctl_snp_get_config(struct sev_issue_cmd *argp)
+{
+       struct sev_device *sev = psp_master->sev_data;
+       struct sev_user_data_ext_snp_config input;

Lets memset |input| to zero to avoid leaking kernel memory, see "crypto: ccp - Use kzalloc for sev ioctl interfaces to prevent kernel memory leak"

Quoted 52 lines
drivers/crypto/ccp/sev-dev.c +52 -0
+       int ret;
+
+       if (!sev->snp_inited || !argp->data)
+               return -EINVAL;
+
+       if (copy_from_user(&input, (void __user *)argp->data, sizeof(input)))
+               return -EFAULT;
+
+       /* Copy the TCB version programmed through the SET_CONFIG to userspace */
+       if (input.config_address) {
+               if (copy_to_user((void * __user)input.config_address,
+                                &sev->snp_config, sizeof(struct sev_user_data_snp_config)))
+                       return -EFAULT;
+       }
+
+       /* Copy the extended certs programmed through the SNP_SET_CONFIG */
+       if (input.certs_address && sev->snp_certs_data) {
+               if (input.certs_len < sev->snp_certs_len) {
+                       /* Return the certs length to userspace */
+                       input.certs_len = sev->snp_certs_len;
+
+                       ret = -ENOSR;
+                       goto e_done;
+               }
+
+               if (copy_to_user((void * __user)input.certs_address,
+                                sev->snp_certs_data, sev->snp_certs_len))
+                       return -EFAULT;
+       }
+
+       ret = 0;
+
+e_done:
+       if (copy_to_user((void __user *)argp->data, &input, sizeof(input)))
+               ret = -EFAULT;
+
+       return ret;
+}
+
+static int sev_ioctl_snp_set_config(struct sev_issue_cmd *argp, bool writable)
+{
+       struct sev_device *sev = psp_master->sev_data;
+       struct sev_user_data_ext_snp_config input;
+       struct sev_user_data_snp_config config;
+       void *certs = NULL;
+       int ret = 0;
+
+       if (!sev->snp_inited || !argp->data)
+               return -EINVAL;
+
+       if (!writable)
+               return -EPERM;

drivers/crypto/ccp/sev-dev.c static int sev_ioctl_snp_platform_status(struct sev_issue_cmd *argp) · line 1688

Quoted 10 lines
drivers/crypto/ccp/sev-dev.c +10 -0
+
+       if (copy_from_user(&input, (void __user *)argp->data, sizeof(input)))
+               return -EFAULT;
+
+       /* Copy the certs from userspace */
+       if (input.certs_address) {
+               if (!input.certs_len || !IS_ALIGNED(input.certs_len, PAGE_SIZE))
+                       return -EINVAL;
+
+               certs = psp_copy_user_blob(input.certs_address, input.certs_len);

I see that psp_copy_user_blob() uses memdup_user() which tracks the allocated memory to GFP_USER. Given this memory is long lived and now belongs to the PSP driver in perpetuity, should this be tracked with GFP_KERNEL?

Quoted 21 lines
drivers/crypto/ccp/sev-dev.c +21 -0
+               if (IS_ERR(certs))
+                       return PTR_ERR(certs);
+       }
+
+       /* Issue the PSP command to update the TCB version using the SNP_CONFIG. */
+       if (input.config_address) {
+               if (copy_from_user(&config,
+                                  (void __user *)input.config_address, sizeof(config))) {
+                       ret = -EFAULT;
+                       goto e_free;
+               }
+
+               ret = __sev_do_cmd_locked(SEV_CMD_SNP_CONFIG, &config, &argp->error);
+               if (ret)
+                       goto e_free;
+
+               memcpy(&sev->snp_config, &config, sizeof(config));
+       }
+
+       /*
+        * If the new certs are passed then cache it else free the old certs.

drivers/crypto/ccp/sev-dev.c static int sev_ioctl_snp_platform_status(struct sev_issue_cmd *argp) · line 1719

Quoted 10 lines
drivers/crypto/ccp/sev-dev.c +10 -0
+        */
+       if (certs) {
+               kfree(sev->snp_certs_data);
+               sev->snp_certs_data = certs;
+               sev->snp_certs_len = input.certs_len;
+       } else {
+               kfree(sev->snp_certs_data);
+               sev->snp_certs_data = NULL;
+               sev->snp_certs_len = 0;
+       }

Do we need another lock here? When I look at 18/49 it seems like snp_guest_ext_guest_request() it seems like we have a race for |sev->snp_certs_data|

Quoted 29 lines
drivers/crypto/ccp/sev-dev.c +14 -0
+
+       return 0;
+
+e_free:
+       kfree(certs);
+       return ret;
+}
+
 static long sev_ioctl(struct file *file, unsigned int ioctl, unsigned long arg)
 {
        void __user *argp = (void __user *)arg;
@@ -1670,6 +1779,12 @@ static long sev_ioctl(struct file *file, unsigned int ioctl, unsigned long arg)
        case SNP_PLATFORM_STATUS:
                ret = sev_ioctl_snp_platform_status(&input);
                break;
+       case SNP_SET_EXT_CONFIG:
+               ret = sev_ioctl_snp_set_config(&input, writable);
+               break;
+       case SNP_GET_EXT_CONFIG:
+               ret = sev_ioctl_snp_get_config(&input);
+               break;
        default:
                ret = -EINVAL;
                goto out;
drivers/crypto/ccp/sev-dev.h +0 -0
index fe5d7a3ebace..d2fe1706311a 100644

drivers/crypto/ccp/sev-dev.h struct sev_device { · line 71

Quoted 6 lines
drivers/crypto/ccp/sev-dev.h +3 -0
@@ -66,6 +66,9 @@ struct sev_device {
 
        bool snp_inited;
        struct snp_host_map snp_host_map[MAX_SNP_HOST_MAP_BUFS];
+       void *snp_certs_data;
+       u32 snp_certs_len;
+       struct sev_user_data_snp_config snp_config;

Since this gets copy_to_user'd can we memset this to 0 to prevent leaking kernel uninitialized memory? Similar to recent patches with kzalloc and __GPF_ZERO usage.

Quoted 41 lines
drivers/crypto/ccp/sev-dev.h +0 -0
 };
 
 int sev_dev_init(struct psp_device *psp);
include/uapi/linux/psp-sev.h +17 -1
index ffd60e8b0a31..60e7a8d1a18e 100644
@@ -29,6 +29,8 @@ enum {
        SEV_GET_ID,     /* This command is deprecated, use SEV_GET_ID2 */
        SEV_GET_ID2,
        SNP_PLATFORM_STATUS,
+       SNP_SET_EXT_CONFIG,
+       SNP_GET_EXT_CONFIG,
 
        SEV_MAX,
 };
@@ -190,6 +192,21 @@ struct sev_user_data_snp_config {
        __u8 rsvd[52];
 } __packed;
 
+/**
+ * struct sev_data_snp_ext_config - system wide configuration value for SNP.
+ *
+ * @config_address: address of the struct sev_user_data_snp_config or 0 when
+ *             reported_tcb does not need to be updated.
+ * @certs_address: address of extended guest request certificate chain or
+ *              0 when previous certificate should be removed on SNP_SET_EXT_CONFIG.
+ * @certs_len: length of the certs
+ */
+struct sev_user_data_ext_snp_config {
+       __u64 config_address;           /* In */
+       __u64 certs_address;            /* In */
+       __u32 certs_len;                /* In */
+};
+
 /**
  * struct sev_issue_cmd - SEV ioctl parameters
  *
--

2.25.1

#208 lore raw

Re: [PATCH Part2 v6 18/49] crypto: ccp: Provide APIs to query extended attestation report

In reply to Ashish Kalra
Quoted 50 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

Version 2 of the GHCB specification defines VMGEXIT that is used to get the extended attestation report. The extended attestation report includes the certificate blobs provided through the SNP_SET_EXT_CONFIG.

The snp_guest_ext_guest_request() will be used by the hypervisor to get the extended attestation report. See the GHCB specification for more details.

---
 drivers/crypto/ccp/sev-dev.c | 43 ++++++++++++++++++++++++++++++++++++
 include/linux/psp-sev.h      | 24 ++++++++++++++++++++
 2 files changed, 67 insertions(+)
drivers/crypto/ccp/sev-dev.c +18 -0
index 97b479d5aa86..f6306b820b86 100644
@@ -25,6 +25,7 @@
 #include <linux/fs.h>
 
 #include <asm/smp.h>
+#include <asm/sev.h>
 
 #include "psp-dev.h"
 #include "sev-dev.h"
@@ -1857,6 +1858,48 @@ int snp_guest_dbg_decrypt(struct sev_data_snp_dbg *data, int *error)
 }
 EXPORT_SYMBOL_GPL(snp_guest_dbg_decrypt);
 
+int snp_guest_ext_guest_request(struct sev_data_snp_guest_request *data,
+                               unsigned long vaddr, unsigned long *npages, unsigned long *fw_err)
+{
+       unsigned long expected_npages;
+       struct sev_device *sev;
+       int rc;
+
+       if (!psp_master || !psp_master->sev_data)
+               return -ENODEV;
+
+       sev = psp_master->sev_data;
+
+       if (!sev->snp_inited)
+               return -EINVAL;
+
+       /*
+        * Check if there is enough space to copy the certificate chain. Otherwise

drivers/crypto/ccp/sev-dev.c int snp_guest_dbg_decrypt(struct sev_data_snp_dbg *data, int *error) · line 1887

Quoted 10 lines
drivers/crypto/ccp/sev-dev.c +10 -0
+        * return ERROR code defined in the GHCB specification.
+        */
+       expected_npages = sev->snp_certs_len >> PAGE_SHIFT;
+       if (*npages < expected_npages) {
+               *npages = expected_npages;
+               *fw_err = SNP_GUEST_REQ_INVALID_LEN;
+               return -EINVAL;
+       }
+
+       rc = sev_do_cmd(SEV_CMD_SNP_GUEST_REQUEST, data, (int *)&fw_err);

We can just pass |fw_error| here (with the cast) here right? Not need to do &fw_err.

      rc = sev_do_cmd(SEV_CMD_SNP_GUEST_REQUEST, data, (int *)fw_err);

drivers/crypto/ccp/sev-dev.c int snp_guest_dbg_decrypt(struct sev_data_snp_dbg *data, int *error) · line 1894

Quoted 7 lines
drivers/crypto/ccp/sev-dev.c +7 -0
+       if (rc)
+               return rc;
+
+       /* Copy the certificate blob */
+       if (sev->snp_certs_data) {
+               *npages = expected_npages;
+               memcpy((void *)vaddr, sev->snp_certs_data, *npages << PAGE_SHIFT);

Why don't we just make |vaddr| into a void* instead of an unsigned long?

Quoted 56 lines
drivers/crypto/ccp/sev-dev.c +8 -0
+       } else {
+               *npages = 0;
+       }
+
+       return rc;
+}
+EXPORT_SYMBOL_GPL(snp_guest_ext_guest_request);
+
 static void sev_exit(struct kref *ref)
 {
        misc_deregister(&misc_dev->misc);
include/linux/psp-sev.h +24 -1
index a3bb792bb842..cd37ccd1fa1f 100644
@@ -945,6 +945,23 @@ void *psp_copy_user_blob(u64 uaddr, u32 len);
 void *snp_alloc_firmware_page(gfp_t mask);
 void snp_free_firmware_page(void *addr);
 
+/**
+ * snp_guest_ext_guest_request - perform the SNP extended guest request command
+ *  defined in the GHCB specification.
+ *
+ * @data: the input guest request structure
+ * @vaddr: address where the certificate blob need to be copied.
+ * @npages: number of pages for the certificate blob.
+ *    If the specified page count is less than the certificate blob size, then the
+ *    required page count is returned with error code defined in the GHCB spec.
+ *    If the specified page count is more than the certificate blob size, then
+ *    page count is updated to reflect the amount of valid data copied in the
+ *    vaddr.
+ */
+int snp_guest_ext_guest_request(struct sev_data_snp_guest_request *data,
+                               unsigned long vaddr, unsigned long *npages,
+                               unsigned long *error);
+
 #else  /* !CONFIG_CRYPTO_DEV_SP_PSP */
 
 static inline int
@@ -992,6 +1009,13 @@ static inline void *snp_alloc_firmware_page(gfp_t mask)
 
 static inline void snp_free_firmware_page(void *addr) { }
 
+static inline int snp_guest_ext_guest_request(struct sev_data_snp_guest_request *data,
+                                             unsigned long vaddr, unsigned long *n,
+                                             unsigned long *error)
+{
+       return -ENODEV;
+}
+
 #endif /* CONFIG_CRYPTO_DEV_SP_PSP */
 
 #endif /* __PSP_SEV_H__ */
--

2.25.1

author maintainer
#145 lore raw

RE: [PATCH Part2 v6 13/49] crypto:ccp: Provide APIs to issue SEV-SNP commands

In reply to Peter Gonda

[Public]

Quoted 8 lines

drivers/crypto/ccp/sev-dev.c int sev_guest_df_flush(int *error) · line 1218

Quoted 6 lines
drivers/crypto/ccp/sev-dev.c +6 -0
+EXPORT_SYMBOL_GPL(snp_guest_decommission);
+
+int snp_guest_df_flush(int *error)
+{
+       return sev_do_cmd(SEV_CMD_SNP_DF_FLUSH, NULL, error); } 
+EXPORT_SYMBOL_GPL(snp_guest_df_flush);

Why not instead change sev_guest_df_flush() to be SNP aware? That way callers get the right behavior without having to know if SNP is enabled or not.

It can be done, and actually both DF_FLUSH commands do exactly the same thing.

But as with other API interfaces here, I think it is better to differentiate between snp and sev API interfaces and the callers be aware of which interface they are invoking.

Thanks,
Ashish

author maintainer
#202 lore raw

RE: [PATCH Part2 v6 17/49] crypto: ccp: Add the SNP_{SET,GET}_EXT_CONFIG command

In reply to Peter Gonda

[AMD Official Use Only - General]

Hello Peter,

Quoted 6 lines

drivers/crypto/ccp/sev-dev.c static int sev_ioctl_snp_platform_status(struct sev_issue_cmd *argp) · line 1626

Quoted 3 lines
drivers/crypto/ccp/sev-dev.c +3 -0
+static int sev_ioctl_snp_get_config(struct sev_issue_cmd *argp) {
+       struct sev_device *sev = psp_master->sev_data;
+       struct sev_user_data_ext_snp_config input;

Lets memset |input| to zero to avoid leaking kernel memory, see "crypto: ccp - Use kzalloc for sev ioctl interfaces to prevent kernel memory leak"

Yes.

Quoted 18 lines
Quoted 18 lines
drivers/crypto/ccp/sev-dev.c +18 -0
+static int sev_ioctl_snp_set_config(struct sev_issue_cmd *argp, bool 
+writable) {
+       struct sev_device *sev = psp_master->sev_data;
+       struct sev_user_data_ext_snp_config input;
+       struct sev_user_data_snp_config config;
+       void *certs = NULL;
+       int ret = 0;
+
+       if (!sev->snp_inited || !argp->data)
+               return -EINVAL;
+
+       if (!writable)
+               return -EPERM;
+
+       if (copy_from_user(&input, (void __user *)argp->data, sizeof(input)))
+               return -EFAULT;
+
+       /* Copy the certs from userspace */
Quoted 8 lines

drivers/crypto/ccp/sev-dev.c static int sev_ioctl_snp_platform_status(struct sev_issue_cmd *argp)

Quoted 6 lines
drivers/crypto/ccp/sev-dev.c +6 -0
+       if (input.certs_address) {
+               if (!input.certs_len || !IS_ALIGNED(input.certs_len, PAGE_SIZE))
+                       return -EINVAL;
+
+               certs = psp_copy_user_blob(input.certs_address, 
+ input.certs_len);

I see that psp_copy_user_blob() uses memdup_user() which tracks the allocated memory to GFP_USER. Given this memory is long lived and now belongs to the PSP driver in perpetuity, should this be tracked with GFP_KERNEL?

But we need to copy from user space address, so what is the alternative here ?

Quoted 13 lines

drivers/crypto/ccp/sev-dev.c static int sev_ioctl_snp_platform_status(struct sev_issue_cmd *argp) · line 1719

Quoted 12 lines
drivers/crypto/ccp/sev-dev.c +12 -0
+       /*
+        * If the new certs are passed then cache it else free the old certs.
+        */
+       if (certs) {
+               kfree(sev->snp_certs_data);
+               sev->snp_certs_data = certs;
+               sev->snp_certs_len = input.certs_len;
+       } else {
+               kfree(sev->snp_certs_data);
+               sev->snp_certs_data = NULL;
+               sev->snp_certs_len = 0;
+       }
Quoted 3 lines

Do we need another lock here? When I look at 18/49 it seems like snp_guest_ext_guest_request() it seems like we have a race for |sev->snp_certs_data|

The certificate blob in snp_guest_ext_guest_request() will depend on the certificate blob provided here by SNP_SET_EXT_CONFIG. There might be a potential race with the SNP extended guest request NAE, let me have a look at it.

Quoted 7 lines

drivers/crypto/ccp/sev-dev.h struct sev_device { · line 71

Quoted 5 lines
        bool snp_inited;
        struct snp_host_map snp_host_map[MAX_SNP_HOST_MAP_BUFS];
drivers/crypto/ccp/sev-dev.h +3 -0
+       void *snp_certs_data;
+       u32 snp_certs_len;
+       struct sev_user_data_snp_config snp_config;

Since this gets copy_to_user'd can we memset this to 0 to prevent leaking kernel uninitialized memory? Similar to recent patches with kzalloc and __GPF_ZERO usage.

Yes.

Thanks,
Ashish

#7 lore raw

Re: [PATCH Part2 v6 02/49] iommu/amd: Introduce function to check SEV-SNP support

In reply to Peter Gonda
Quoted 20 lines · Peter Gonda
Quoted 19 lines · Ashish

[AMD Official Use Only - General]

Hello Peter,

Quoted 13 lines

drivers/iommu/amd/init.c int amd_iommu_pc_set_reg(struct amd_iommu *iommu, u8 bank, u8 cntr, u8 fxn, u64

Quoted 11 lines
drivers/iommu/amd/init.c +11 -0
+bool iommu_sev_snp_supported(void)
+{
+       struct amd_iommu *iommu;
+
+       /*
+        * The SEV-SNP support requires that IOMMU must be enabled, and is
+        * not configured in the passthrough mode.
+        */
+       if (no_iommu || iommu_default_passthrough()) {
+               pr_err("SEV-SNP: IOMMU is either disabled or
+ configured in passthrough mode.\n");

Like below could this say something like snp support is disabled because of iommu settings.

Quoted 8 lines
Quoted 4 lines

Here we may need to be more precise with the error information indicating why SNP is not enabled. Please note that this patch may actually become part of the IOMMU + SNP patch series, where additional checks are done, for example, not enabling SNP if IOMMU v2 page tables are enabled, so precise error information will be useful here.

I agree we should be more precise. I just thought we should explicitly state something like: "SEV-SNP: IOMMU is either disabled or configured in passthrough mode, SNP cannot be supported".

I can update this in the other patch series here

https://lists.linuxfoundation.org/pipermail/iommu/2022-June/066399.html

Thank you,
Suravee

#229 lore raw

Re: [PATCH Part2 v6 27/49] KVM: SVM: Mark the private vma unmerable for SEV-SNP guests

In reply to Ashish Kalra
Quoted 10 lines · * Ashish Kalra ([email protected]) wrote:

From: Brijesh Singh <[email protected]>

When SEV-SNP is enabled, the guest private pages are added in the RMP table; while adding the pages, the rmp_make_private() unmaps the pages from the direct map. If KSM attempts to access those unmapped pages then it will trigger #PF (page-not-present).

Encrypted guest pages cannot be shared between the process, so an userspace should not mark the region mergeable but to be safe, mark the process vma unmerable before adding the pages in the RMP table.

^
Typo 'unmergable' (also in title)

Quoted 70 lines
---
 arch/x86/kvm/svm/sev.c | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)
arch/x86/kvm/svm/sev.c +32 -0
index b5f0707d7ed6..a9461d352eda 100644
@@ -19,11 +19,13 @@
 #include <linux/trace_events.h>
 #include <linux/hugetlb.h>
 #include <linux/sev.h>
+#include <linux/ksm.h>
 
 #include <asm/pkru.h>
 #include <asm/trapnr.h>
 #include <asm/fpu/xcr.h>
 #include <asm/sev.h>
+#include <asm/mman.h>
 
 #include "x86.h"
 #include "svm.h"
@@ -1965,6 +1967,30 @@ static bool is_hva_registered(struct kvm *kvm, hva_t hva, size_t len)
 	return false;
 }
 
+static int snp_mark_unmergable(struct kvm *kvm, u64 start, u64 size)
+{
+	struct vm_area_struct *vma;
+	u64 end = start + size;
+	int ret;
+
+	do {
+		vma = find_vma_intersection(kvm->mm, start, end);
+		if (!vma) {
+			ret = -EINVAL;
+			break;
+		}
+
+		ret = ksm_madvise(vma, vma->vm_start, vma->vm_end,
+				  MADV_UNMERGEABLE, &vma->vm_flags);
+		if (ret)
+			break;
+
+		start = vma->vm_end;
+	} while (end > vma->vm_end);
+
+	return ret;
+}
+
 static int snp_launch_update(struct kvm *kvm, struct kvm_sev_cmd *argp)
 {
 	struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
@@ -1989,6 +2015,12 @@ static int snp_launch_update(struct kvm *kvm, struct kvm_sev_cmd *argp)
 	if (!is_hva_registered(kvm, params.uaddr, params.len))
 		return -EINVAL;
 
+	mmap_write_lock(kvm->mm);
+	ret = snp_mark_unmergable(kvm, params.uaddr, params.len);
+	mmap_write_unlock(kvm->mm);
+	if (ret)
+		return -EFAULT;
+
 	/*
 	 * The userspace memory is already locked so technically we don't
 	 * need to lock it again. Later part of the function needs to know
Signature · 1 line
2.25.1
Signature · 1 line
Dr. David Alan Gilbert / [email protected] / Manchester, UK
#30 lore raw

Re: [PATCH Part2 v6 05/49] x86/sev: Add RMP entry lookup helpers

In reply to Ashish Kalra
Quoted 4 lines · Ashish Kalra

+/*
+ * The RMP entry format is not architectural. The format is defined in PPR
+ * Family 19h Model 01h, Rev B1 processor.
+ */

Let's say that Family 20h comes out and has a new RMP entry format. What keeps an old kernel from attempting to use this old format on that new CPU?

author maintainer
#31 lore raw

RE: [PATCH Part2 v6 05/49] x86/sev: Add RMP entry lookup helpers

[AMD Official Use Only - General]

Quoted 8 lines

arch/x86/include/asm/sev.h extern bool handle_vc_boot_ghcb(struct pt_regs *regs); · line 93

Quoted 5 lines
arch/x86/include/asm/sev.h +5 -0
+/*
+ * The RMP entry format is not architectural. The format is defined 
+in PPR
+ * Family 19h Model 01h, Rev B1 processor.
+ */

Let's say that Family 20h comes out and has a new RMP entry format. What keeps an old kernel from attempting to use this old format on that new CPU?

As I replied previously on the same subject:
Architectural implies that it is defined in the APM and shouldn't change in such a way as to not be backward compatible. I probably think the wording here should be architecture independent or more precisely platform independent.

Thanks,
Ashish

#68 lore raw

Re: [PATCH Part2 v6 06/49] x86/sev: Add helper functions for RMPUPDATE and PSMASH instruction

In reply to Ashish Kalra
Quoted 10 lines · Ashish Kalra
arch/x86/kernel/sev.c +10 -0
+int psmash(u64 pfn)
+{
+	unsigned long paddr = pfn << PAGE_SHIFT;
+	int ret;
+
+	if (!pfn_valid(pfn))
+		return -EINVAL;
+
+	if (!cpu_feature_enabled(X86_FEATURE_SEV_SNP))
+		return -ENXIO;

arch/x86/kernel/sev.c int snp_lookup_rmpentry(u64 pfn, int *level) · line 2452

Quoted 10 lines
arch/x86/kernel/sev.c +10 -0
+
+	/* Binutils version 2.36 supports the PSMASH mnemonic. */
+	asm volatile(".byte 0xF3, 0x0F, 0x01, 0xFF"
+		      : "=a"(ret)
+		      : "a"(paddr)
+		      : "memory", "cc");
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(psmash);

If a function gets an EXPORT_SYMBOL_GPL(), the least we can do is reasonably document it. We don't need full kerneldoc nonsense, but a one-line about what this does would be quite helpful. That goes for all the functions here.

It would also be extremely helpful to have the changelog explain why these functions are exported and how the exports will be used.

As a general rule, please push cpu_feature_enabled() checks as early as you reasonably can. They are *VERY* cheap and can even enable the compiler to completely zap code like an #ifdef.

There also seem to be a lot of pfn_valid() checks in here that aren't very well thought out. For instance, there's a pfn_valid() check here:

+int rmp_make_shared(u64 pfn, enum pg_level level) +{
+ struct rmpupdate val;
+
+ if (!pfn_valid(pfn))
+ return -EINVAL;
...
+ return rmpupdate(pfn, &val);
+}

and in rmpupdate():

+static int rmpupdate(u64 pfn, struct rmpupdate *val) +{
+ unsigned long paddr = pfn << PAGE_SHIFT;
+ int ret;
+
+ if (!pfn_valid(pfn))
+ return -EINVAL;
...

This is (at best) wasteful. Could it be refactored?

#32 lore raw

Re: [PATCH Part2 v6 05/49] x86/sev: Add RMP entry lookup helpers

In reply to Kalra, Ashish
Quoted 5 lines · Ashish

As I replied previously on the same subject: Architectural implies that it is defined in the APM and shouldn't change in such a way as to not be backward compatible. I probably think the wording here should be architecture independent or more precisely platform independent.

Yeah, arch-independent and non-architectural are quite different concepts.

At Intel, at least, when someone says "not architectural" mean that the behavior is implementation-specific. That, combined with the model/family/stepping gave me the wrong impression about what was going on.

Some more clarity would be greatly appreciated.

#94 lore raw

Re: [PATCH Part2 v6 09/49] x86/fault: Add support to handle the RMP fault for user address

In reply to Ashish Kalra
Quoted 73 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

When SEV-SNP is enabled globally, a write from the host goes through the RMP check. When the host writes to pages, hardware checks the following conditions at the end of page walk:

1. Assigned bit in the RMP table is zero (i.e page is shared).
2. If the page table entry that gives the sPA indicates that the target page size is a large page, then all RMP entries for the 4KB constituting pages of the target must have the assigned bit 0.
3. Immutable bit in the RMP table is not zero.

The hardware will raise page fault if one of the above conditions is not met. Try resolving the fault instead of taking fault again and again. If the host attempts to write to the guest private memory then send the SIGBUS signal to kill the process. If the page level between the host and RMP entry does not match, then split the address to keep the RMP and host page levels in sync.

---
 arch/x86/mm/fault.c      | 66 ++++++++++++++++++++++++++++++++++++++++
 include/linux/mm.h       |  3 +-
 include/linux/mm_types.h |  3 ++
 mm/memory.c              | 13 ++++++++
 4 files changed, 84 insertions(+), 1 deletion(-)
arch/x86/mm/fault.c +31 -0
index a4c270e99f7f..f5de9673093a 100644
@@ -19,6 +19,7 @@
 #include <linux/uaccess.h>		/* faulthandler_disabled()	*/
 #include <linux/efi.h>			/* efi_crash_gracefully_on_page_fault()*/
 #include <linux/mm_types.h>
+#include <linux/sev.h>			/* snp_lookup_rmpentry()	*/
 
 #include <asm/cpufeature.h>		/* boot_cpu_has, ...		*/
 #include <asm/traps.h>			/* dotraplinkage, ...		*/
@@ -1209,6 +1210,60 @@ do_kern_addr_fault(struct pt_regs *regs, unsigned long hw_error_code,
 }
 NOKPROBE_SYMBOL(do_kern_addr_fault);
 
+static inline size_t pages_per_hpage(int level)
+{
+	return page_level_size(level) / PAGE_SIZE;
+}
+
+/*
+ * Return 1 if the caller need to retry, 0 if it the address need to be split
+ * in order to resolve the fault.
+ */
+static int handle_user_rmp_page_fault(struct pt_regs *regs, unsigned long error_code,
+				      unsigned long address)
+{
+	int rmp_level, level;
+	pte_t *pte;
+	u64 pfn;
+
+	pte = lookup_address_in_mm(current->mm, address, &level);
+
+	/*
+	 * It can happen if there was a race between an unmap event and
+	 * the RMP fault delivery.
+	 */
+	if (!pte || !pte_present(*pte))
+		return 1;
+
+	pfn = pte_pfn(*pte);
+
+	/* If its large page then calculte the fault pfn */
+	if (level > PG_LEVEL_4K) {
+		unsigned long mask;

arch/x86/mm/fault.c do_kern_addr_fault(struct pt_regs *regs, unsigned long hw_error_code, · line 1252

Quoted 10 lines
arch/x86/mm/fault.c +10 -0
+
+		mask = pages_per_hpage(level) - pages_per_hpage(level - 1);
+		pfn |= (address >> PAGE_SHIFT) & mask;
+	}
+
+	/*
+	 * If its a guest private page, then the fault cannot be resolved.
+	 * Send a SIGBUS to terminate the process.
+	 */
+	if (snp_lookup_rmpentry(pfn, &rmp_level)) {

snp_lookup_rmpentry returns 0, 1 or -errno, so this should likely be:

  if (snp_lookup_rmpentry(pfn, &rmp_level) != 1)) {
Quoted 104 lines
arch/x86/mm/fault.c +25 -0
+		do_sigbus(regs, error_code, address, VM_FAULT_SIGBUS);
+		return 1;
+	}
+
+	/*
+	 * The backing page level is higher than the RMP page level, request
+	 * to split the page.
+	 */
+	if (level > rmp_level)
+		return 0;
+
+	return 1;
+}
+
 /*
  * Handle faults in the user portion of the address space.  Nothing in here
  * should check X86_PF_USER without a specific justification: for almost
@@ -1306,6 +1361,17 @@ void do_user_addr_fault(struct pt_regs *regs,
 	if (error_code & X86_PF_INSTR)
 		flags |= FAULT_FLAG_INSTRUCTION;
 
+	/*
+	 * If its an RMP violation, try resolving it.
+	 */
+	if (error_code & X86_PF_RMP) {
+		if (handle_user_rmp_page_fault(regs, error_code, address))
+			return;
+
+		/* Ask to split the page */
+		flags |= FAULT_FLAG_PAGE_SPLIT;
+	}
+
 #ifdef CONFIG_X86_64
 	/*
 	 * Faults in the vsyscall page might need emulation.  The
include/linux/mm.h +2 -1
index de32c0383387..2ccc562d166f 100644
@@ -463,7 +463,8 @@ static inline bool fault_flag_allow_retry_first(enum fault_flag flags)
 	{ FAULT_FLAG_USER,		"USER" }, \
 	{ FAULT_FLAG_REMOTE,		"REMOTE" }, \
 	{ FAULT_FLAG_INSTRUCTION,	"INSTRUCTION" }, \
-	{ FAULT_FLAG_INTERRUPTIBLE,	"INTERRUPTIBLE" }
+	{ FAULT_FLAG_INTERRUPTIBLE,	"INTERRUPTIBLE" }, \
+	{ FAULT_FLAG_PAGE_SPLIT,	"PAGESPLIT" }
 
 /*
  * vm_fault is filled by the pagefault handler and passed to the vma's
include/linux/mm_types.h +3 -0
index 6dfaf271ebf8..aa2d8d48ce3e 100644
@@ -818,6 +818,8 @@ typedef struct {
  *                      mapped R/O.
  * @FAULT_FLAG_ORIG_PTE_VALID: whether the fault has vmf->orig_pte cached.
  *                        We should only access orig_pte if this flag set.
+ * @FAULT_FLAG_PAGE_SPLIT: The fault was due page size mismatch, split the
+ *                         region to smaller page size and retry.
  *
  * About @FAULT_FLAG_ALLOW_RETRY and @FAULT_FLAG_TRIED: we can specify
  * whether we would allow page faults to retry by specifying these two
@@ -855,6 +857,7 @@ enum fault_flag {
 	FAULT_FLAG_INTERRUPTIBLE =	1 << 9,
 	FAULT_FLAG_UNSHARE =		1 << 10,
 	FAULT_FLAG_ORIG_PTE_VALID =	1 << 11,
+	FAULT_FLAG_PAGE_SPLIT =		1 << 12,
 };
 
 typedef unsigned int __bitwise zap_flags_t;
mm/memory.c +13 -0
index 7274f2b52bca..c2187ffcbb8e 100644
@@ -4945,6 +4945,15 @@ static vm_fault_t handle_pte_fault(struct vm_fault *vmf)
 	return 0;
 }
 
+static int handle_split_page_fault(struct vm_fault *vmf)
+{
+	if (!IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT))
+		return VM_FAULT_SIGBUS;
+
+	__split_huge_pmd(vmf->vma, vmf->pmd, vmf->address, false, NULL);
+	return 0;
+}
+
 /*
  * By the time we get here, we already hold the mm semaphore
  *
@@ -5024,6 +5033,10 @@ static vm_fault_t __handle_mm_fault(struct vm_area_struct *vma,
 				pmd_migration_entry_wait(mm, vmf.pmd);
 			return 0;
 		}
+
+		if (flags & FAULT_FLAG_PAGE_SPLIT)
+			return handle_split_page_fault(&vmf);
+
 		if (pmd_trans_huge(vmf.orig_pmd) || pmd_devmap(vmf.orig_pmd)) {
 			if (pmd_protnone(vmf.orig_pmd) && vma_is_accessible(vma))
 				return do_huge_pmd_numa_page(&vmf);
Signature · 1 line
2.25.1
#126 lore raw

Re: [PATCH Part2 v6 10/49] x86/fault: Add support to dump RMP entry on fault

In reply to Ashish Kalra
Quoted 53 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

When SEV-SNP is enabled globally, a write from the host goes through the RMP check. If the hardware encounters the check failure, then it raises the #PF (with RMP set). Dump the RMP entry at the faulting pfn to help the debug.

---
 arch/x86/include/asm/sev.h |  7 +++++++
 arch/x86/kernel/sev.c      | 43 ++++++++++++++++++++++++++++++++++++++
 arch/x86/mm/fault.c        | 17 +++++++++++----
 include/linux/sev.h        |  2 ++
 4 files changed, 65 insertions(+), 4 deletions(-)
arch/x86/include/asm/sev.h +7 -0
index 6ab872311544..c0c4df817159 100644
@@ -113,6 +113,11 @@ struct __packed rmpentry {
 
 #define rmpentry_assigned(x)	((x)->info.assigned)
 #define rmpentry_pagesize(x)	((x)->info.pagesize)
+#define rmpentry_vmsa(x)	((x)->info.vmsa)
+#define rmpentry_asid(x)	((x)->info.asid)
+#define rmpentry_validated(x)	((x)->info.validated)
+#define rmpentry_gpa(x)		((unsigned long)(x)->info.gpa)
+#define rmpentry_immutable(x)	((x)->info.immutable)
 
 #define RMPADJUST_VMSA_PAGE_BIT		BIT(16)
 
@@ -205,6 +210,7 @@ void snp_set_wakeup_secondary_cpu(void);
 bool snp_init(struct boot_params *bp);
 void snp_abort(void);
 int snp_issue_guest_request(u64 exit_code, struct snp_req_data *input, unsigned long *fw_err);
+void dump_rmpentry(u64 pfn);
 #else
 static inline void sev_es_ist_enter(struct pt_regs *regs) { }
 static inline void sev_es_ist_exit(void) { }
@@ -229,6 +235,7 @@ static inline int snp_issue_guest_request(u64 exit_code, struct snp_req_data *in
 {
 	return -ENOTTY;
 }
+static inline void dump_rmpentry(u64 pfn) {}
 #endif
 
 #endif
arch/x86/kernel/sev.c +0 -0
index 734cddd837f5..6640a639fffc 100644
@@ -2414,6 +2414,49 @@ static struct rmpentry *__snp_lookup_rmpentry(u64 pfn, int *level)
 	return entry;

arch/x86/kernel/sev.c static struct rmpentry *__snp_lookup_rmpentry(u64 pfn, int *level) · line 2424

Quoted 10 lines
arch/x86/kernel/sev.c +8 -0
 }
 
+void dump_rmpentry(u64 pfn)
+{
+	unsigned long pfn_end;
+	struct rmpentry *e;
+	int level;
+
+	e = __snp_lookup_rmpentry(pfn, &level);
+	if (!e) {

__snp_lookup_rmpentry may return -errno so this should be:

if (e != 1)

Quoted 15 lines
arch/x86/kernel/sev.c +15 -0
+		pr_alert("failed to read RMP entry pfn 0x%llx\n", pfn);
+		return;
+	}
+
+	if (rmpentry_assigned(e)) {
+		pr_alert("RMPEntry paddr 0x%llx [assigned=%d immutable=%d pagesize=%d gpa=0x%lx"
+			" asid=%d vmsa=%d validated=%d]\n", pfn << PAGE_SHIFT,
+			rmpentry_assigned(e), rmpentry_immutable(e), rmpentry_pagesize(e),
+			rmpentry_gpa(e), rmpentry_asid(e), rmpentry_vmsa(e),
+			rmpentry_validated(e));
+		return;
+	}
+
+	/*
+	 * If the RMP entry at the faulting pfn was not assigned, then we do not

arch/x86/kernel/sev.c static struct rmpentry *__snp_lookup_rmpentry(u64 pfn, int *level) · line 2449

Quoted 10 lines
arch/x86/kernel/sev.c +10 -0
+	 * know what caused the RMP violation. To get some useful debug information,
+	 * let iterate through the entire 2MB region, and dump the RMP entries if
+	 * one of the bit in the RMP entry is set.
+	 */
+	pfn = pfn & ~(PTRS_PER_PMD - 1);
+	pfn_end = pfn + PTRS_PER_PMD;
+
+	while (pfn < pfn_end) {
+		e = __snp_lookup_rmpentry(pfn, &level);
+		if (!e)

if (e != 1)

Quoted 126 lines
arch/x86/kernel/sev.c +10 -0
+			return;
+
+		if (e->low || e->high)
+			pr_alert("RMPEntry paddr 0x%llx: [high=0x%016llx low=0x%016llx]\n",
+				 pfn << PAGE_SHIFT, e->high, e->low);
+		pfn++;
+	}
+}
+EXPORT_SYMBOL_GPL(dump_rmpentry);
+
 /*
  * Return 1 if the RMP entry is assigned, 0 if it exists but is not assigned,
  * and -errno if there is no corresponding RMP entry.
arch/x86/mm/fault.c +13 -4
index f5de9673093a..25896a6ba04a 100644
@@ -34,6 +34,7 @@
 #include <asm/kvm_para.h>		/* kvm_handle_async_pf		*/
 #include <asm/vdso.h>			/* fixup_vdso_exception()	*/
 #include <asm/irq_stack.h>
+#include <asm/sev.h>			/* dump_rmpentry()		*/
 
 #define CREATE_TRACE_POINTS
 #include <asm/trace/exceptions.h>
@@ -290,7 +291,7 @@ static bool low_pfn(unsigned long pfn)
 	return pfn < max_low_pfn;
 }
 
-static void dump_pagetable(unsigned long address)
+static void dump_pagetable(unsigned long address, bool show_rmpentry)
 {
 	pgd_t *base = __va(read_cr3_pa());
 	pgd_t *pgd = &base[pgd_index(address)];
@@ -346,10 +347,11 @@ static int bad_address(void *p)
 	return get_kernel_nofault(dummy, (unsigned long *)p);
 }
 
-static void dump_pagetable(unsigned long address)
+static void dump_pagetable(unsigned long address, bool show_rmpentry)
 {
 	pgd_t *base = __va(read_cr3_pa());
 	pgd_t *pgd = base + pgd_index(address);
+	unsigned long pfn;
 	p4d_t *p4d;
 	pud_t *pud;
 	pmd_t *pmd;
@@ -367,6 +369,7 @@ static void dump_pagetable(unsigned long address)
 	if (bad_address(p4d))
 		goto bad;
 
+	pfn = p4d_pfn(*p4d);
 	pr_cont("P4D %lx ", p4d_val(*p4d));
 	if (!p4d_present(*p4d) || p4d_large(*p4d))
 		goto out;
@@ -375,6 +378,7 @@ static void dump_pagetable(unsigned long address)
 	if (bad_address(pud))
 		goto bad;
 
+	pfn = pud_pfn(*pud);
 	pr_cont("PUD %lx ", pud_val(*pud));
 	if (!pud_present(*pud) || pud_large(*pud))
 		goto out;
@@ -383,6 +387,7 @@ static void dump_pagetable(unsigned long address)
 	if (bad_address(pmd))
 		goto bad;
 
+	pfn = pmd_pfn(*pmd);
 	pr_cont("PMD %lx ", pmd_val(*pmd));
 	if (!pmd_present(*pmd) || pmd_large(*pmd))
 		goto out;
@@ -391,9 +396,13 @@ static void dump_pagetable(unsigned long address)
 	if (bad_address(pte))
 		goto bad;
 
+	pfn = pte_pfn(*pte);
 	pr_cont("PTE %lx", pte_val(*pte));
 out:
 	pr_cont("\n");
+
+	if (show_rmpentry)
+		dump_rmpentry(pfn);
 	return;
 bad:
 	pr_info("BAD\n");
@@ -579,7 +588,7 @@ show_fault_oops(struct pt_regs *regs, unsigned long error_code, unsigned long ad
 		show_ldttss(&gdt, "TR", tr);
 	}
 
-	dump_pagetable(address);
+	dump_pagetable(address, error_code & X86_PF_RMP);
 }
 
 static noinline void
@@ -596,7 +605,7 @@ pgtable_bad(struct pt_regs *regs, unsigned long error_code,
 
 	printk(KERN_ALERT "%s: Corrupted page table at address %lx\n",
 	       tsk->comm, address);
-	dump_pagetable(address);
+	dump_pagetable(address, false);
 
 	if (__die("Bad pagetable", regs, error_code))
 		sig = 0;
include/linux/sev.h +2 -0
index 1a68842789e1..734b13a69c54 100644
@@ -16,6 +16,7 @@ int snp_lookup_rmpentry(u64 pfn, int *level);
 int psmash(u64 pfn);
 int rmp_make_private(u64 pfn, u64 gpa, enum pg_level level, int asid, bool immutable);
 int rmp_make_shared(u64 pfn, enum pg_level level);
+void dump_rmpentry(u64 pfn);
 #else
 static inline int snp_lookup_rmpentry(u64 pfn, int *level) { return 0; }
 static inline int psmash(u64 pfn) { return -ENXIO; }
@@ -25,6 +26,7 @@ static inline int rmp_make_private(u64 pfn, u64 gpa, enum pg_level level, int as
 	return -ENODEV;
 }
 static inline int rmp_make_shared(u64 pfn, enum pg_level level) { return -ENODEV; }
+static inline void dump_rmpentry(u64 pfn) { }
 
 #endif /* CONFIG_AMD_MEM_ENCRYPT */
 #endif /* __LINUX_SEV_H */
Signature · 1 line
2.25.1
#127 lore raw

Re: [PATCH Part2 v6 10/49] x86/fault: Add support to dump RMP entry on fault

Quoted 67 lines · Jeremi Piotrowski
Quoted 53 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

When SEV-SNP is enabled globally, a write from the host goes through the RMP check. If the hardware encounters the check failure, then it raises the #PF (with RMP set). Dump the RMP entry at the faulting pfn to help the debug.

---
 arch/x86/include/asm/sev.h |  7 +++++++
 arch/x86/kernel/sev.c      | 43 ++++++++++++++++++++++++++++++++++++++
 arch/x86/mm/fault.c        | 17 +++++++++++----
 include/linux/sev.h        |  2 ++
 4 files changed, 65 insertions(+), 4 deletions(-)
arch/x86/include/asm/sev.h +7 -0
index 6ab872311544..c0c4df817159 100644
@@ -113,6 +113,11 @@ struct __packed rmpentry {
 
 #define rmpentry_assigned(x)	((x)->info.assigned)
 #define rmpentry_pagesize(x)	((x)->info.pagesize)
+#define rmpentry_vmsa(x)	((x)->info.vmsa)
+#define rmpentry_asid(x)	((x)->info.asid)
+#define rmpentry_validated(x)	((x)->info.validated)
+#define rmpentry_gpa(x)		((unsigned long)(x)->info.gpa)
+#define rmpentry_immutable(x)	((x)->info.immutable)
 
 #define RMPADJUST_VMSA_PAGE_BIT		BIT(16)
 
@@ -205,6 +210,7 @@ void snp_set_wakeup_secondary_cpu(void);
 bool snp_init(struct boot_params *bp);
 void snp_abort(void);
 int snp_issue_guest_request(u64 exit_code, struct snp_req_data *input, unsigned long *fw_err);
+void dump_rmpentry(u64 pfn);
 #else
 static inline void sev_es_ist_enter(struct pt_regs *regs) { }
 static inline void sev_es_ist_exit(void) { }
@@ -229,6 +235,7 @@ static inline int snp_issue_guest_request(u64 exit_code, struct snp_req_data *in
 {
 	return -ENOTTY;
 }
+static inline void dump_rmpentry(u64 pfn) {}
 #endif
 
 #endif
arch/x86/kernel/sev.c +0 -0
index 734cddd837f5..6640a639fffc 100644
@@ -2414,6 +2414,49 @@ static struct rmpentry *__snp_lookup_rmpentry(u64 pfn, int *level)
 	return entry;

arch/x86/kernel/sev.c static struct rmpentry *__snp_lookup_rmpentry(u64 pfn, int *level) · line 2424

Quoted 10 lines
arch/x86/kernel/sev.c +8 -0
 }
 
+void dump_rmpentry(u64 pfn)
+{
+	unsigned long pfn_end;
+	struct rmpentry *e;
+	int level;
+
+	e = __snp_lookup_rmpentry(pfn, &level);
+	if (!e) {

__snp_lookup_rmpentry may return -errno so this should be:

Quoted 1 line

if (e != 1)

Sorry, actually it should be:

  if (IS_ERR_OR_NULL(e)) {
Quoted 20 lines
Quoted 20 lines
arch/x86/kernel/sev.c +19 -0
+		pr_alert("failed to read RMP entry pfn 0x%llx\n", pfn);
+		return;
+	}
+
+	if (rmpentry_assigned(e)) {
+		pr_alert("RMPEntry paddr 0x%llx [assigned=%d immutable=%d pagesize=%d gpa=0x%lx"
+			" asid=%d vmsa=%d validated=%d]\n", pfn << PAGE_SHIFT,
+			rmpentry_assigned(e), rmpentry_immutable(e), rmpentry_pagesize(e),
+			rmpentry_gpa(e), rmpentry_asid(e), rmpentry_vmsa(e),
+			rmpentry_validated(e));
+		return;
+	}
+
+	/*
+	 * If the RMP entry at the faulting pfn was not assigned, then we do not
+	 * know what caused the RMP violation. To get some useful debug information,
+	 * let iterate through the entire 2MB region, and dump the RMP entries if
+	 * one of the bit in the RMP entry is set.
+	 */
Quoted 9 lines

arch/x86/kernel/sev.c static struct rmpentry *__snp_lookup_rmpentry(u64 pfn, int *level) · line 2449

Quoted 6 lines
arch/x86/kernel/sev.c +6 -0
+	pfn = pfn & ~(PTRS_PER_PMD - 1);
+	pfn_end = pfn + PTRS_PER_PMD;
+
+	while (pfn < pfn_end) {
+		e = __snp_lookup_rmpentry(pfn, &level);
+		if (!e)

if (e != 1)

and this too:

if (IS_ERR_OR_NULL(e))

Quoted 126 lines
Quoted 126 lines
arch/x86/kernel/sev.c +10 -0
+			return;
+
+		if (e->low || e->high)
+			pr_alert("RMPEntry paddr 0x%llx: [high=0x%016llx low=0x%016llx]\n",
+				 pfn << PAGE_SHIFT, e->high, e->low);
+		pfn++;
+	}
+}
+EXPORT_SYMBOL_GPL(dump_rmpentry);
+
 /*
  * Return 1 if the RMP entry is assigned, 0 if it exists but is not assigned,
  * and -errno if there is no corresponding RMP entry.
arch/x86/mm/fault.c +13 -4
index f5de9673093a..25896a6ba04a 100644
@@ -34,6 +34,7 @@
 #include <asm/kvm_para.h>		/* kvm_handle_async_pf		*/
 #include <asm/vdso.h>			/* fixup_vdso_exception()	*/
 #include <asm/irq_stack.h>
+#include <asm/sev.h>			/* dump_rmpentry()		*/
 
 #define CREATE_TRACE_POINTS
 #include <asm/trace/exceptions.h>
@@ -290,7 +291,7 @@ static bool low_pfn(unsigned long pfn)
 	return pfn < max_low_pfn;
 }
 
-static void dump_pagetable(unsigned long address)
+static void dump_pagetable(unsigned long address, bool show_rmpentry)
 {
 	pgd_t *base = __va(read_cr3_pa());
 	pgd_t *pgd = &base[pgd_index(address)];
@@ -346,10 +347,11 @@ static int bad_address(void *p)
 	return get_kernel_nofault(dummy, (unsigned long *)p);
 }
 
-static void dump_pagetable(unsigned long address)
+static void dump_pagetable(unsigned long address, bool show_rmpentry)
 {
 	pgd_t *base = __va(read_cr3_pa());
 	pgd_t *pgd = base + pgd_index(address);
+	unsigned long pfn;
 	p4d_t *p4d;
 	pud_t *pud;
 	pmd_t *pmd;
@@ -367,6 +369,7 @@ static void dump_pagetable(unsigned long address)
 	if (bad_address(p4d))
 		goto bad;
 
+	pfn = p4d_pfn(*p4d);
 	pr_cont("P4D %lx ", p4d_val(*p4d));
 	if (!p4d_present(*p4d) || p4d_large(*p4d))
 		goto out;
@@ -375,6 +378,7 @@ static void dump_pagetable(unsigned long address)
 	if (bad_address(pud))
 		goto bad;
 
+	pfn = pud_pfn(*pud);
 	pr_cont("PUD %lx ", pud_val(*pud));
 	if (!pud_present(*pud) || pud_large(*pud))
 		goto out;
@@ -383,6 +387,7 @@ static void dump_pagetable(unsigned long address)
 	if (bad_address(pmd))
 		goto bad;
 
+	pfn = pmd_pfn(*pmd);
 	pr_cont("PMD %lx ", pmd_val(*pmd));
 	if (!pmd_present(*pmd) || pmd_large(*pmd))
 		goto out;
@@ -391,9 +396,13 @@ static void dump_pagetable(unsigned long address)
 	if (bad_address(pte))
 		goto bad;
 
+	pfn = pte_pfn(*pte);
 	pr_cont("PTE %lx", pte_val(*pte));
 out:
 	pr_cont("\n");
+
+	if (show_rmpentry)
+		dump_rmpentry(pfn);
 	return;
 bad:
 	pr_info("BAD\n");
@@ -579,7 +588,7 @@ show_fault_oops(struct pt_regs *regs, unsigned long error_code, unsigned long ad
 		show_ldttss(&gdt, "TR", tr);
 	}
 
-	dump_pagetable(address);
+	dump_pagetable(address, error_code & X86_PF_RMP);
 }
 
 static noinline void
@@ -596,7 +605,7 @@ pgtable_bad(struct pt_regs *regs, unsigned long error_code,
 
 	printk(KERN_ALERT "%s: Corrupted page table at address %lx\n",
 	       tsk->comm, address);
-	dump_pagetable(address);
+	dump_pagetable(address, false);
 
 	if (__die("Bad pagetable", regs, error_code))
 		sig = 0;
include/linux/sev.h +2 -0
index 1a68842789e1..734b13a69c54 100644
@@ -16,6 +16,7 @@ int snp_lookup_rmpentry(u64 pfn, int *level);
 int psmash(u64 pfn);
 int rmp_make_private(u64 pfn, u64 gpa, enum pg_level level, int asid, bool immutable);
 int rmp_make_shared(u64 pfn, enum pg_level level);
+void dump_rmpentry(u64 pfn);
 #else
 static inline int snp_lookup_rmpentry(u64 pfn, int *level) { return 0; }
 static inline int psmash(u64 pfn) { return -ENXIO; }
@@ -25,6 +26,7 @@ static inline int rmp_make_private(u64 pfn, u64 gpa, enum pg_level level, int as
 	return -ENODEV;
 }
 static inline int rmp_make_shared(u64 pfn, enum pg_level level) { return -ENODEV; }
+static inline void dump_rmpentry(u64 pfn) { }
 
 #endif /* CONFIG_AMD_MEM_ENCRYPT */
 #endif /* __LINUX_SEV_H */
Signature · 1 line
2.25.1
author maintainer
#69 lore raw

RE: [PATCH Part2 v6 06/49] x86/sev: Add helper functions for RMPUPDATE and PSMASH instruction

In reply to Dave Hansen

[AMD Official Use Only - General]

Quoted 23 lines
Quoted 10 lines
arch/x86/kernel/sev.c +10 -0
+int psmash(u64 pfn)
+{
+	unsigned long paddr = pfn << PAGE_SHIFT;
+	int ret;
+
+	if (!pfn_valid(pfn))
+		return -EINVAL;
+
+	if (!cpu_feature_enabled(X86_FEATURE_SEV_SNP))
+		return -ENXIO;

arch/x86/kernel/sev.c int snp_lookup_rmpentry(u64 pfn, int *level) · line 2452

Quoted 10 lines
arch/x86/kernel/sev.c +10 -0
+
+	/* Binutils version 2.36 supports the PSMASH mnemonic. */
+	asm volatile(".byte 0xF3, 0x0F, 0x01, 0xFF"
+		      : "=a"(ret)
+		      : "a"(paddr)
+		      : "memory", "cc");
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(psmash);

If a function gets an EXPORT_SYMBOL_GPL(), the least we can do is reasonably document it. We don't need full kerneldoc nonsense, but a one-line about what this does would be quite helpful. That goes for all the functions here.

Quoted 1 line

It would also be extremely helpful to have the changelog explain why these functions are exported and how the exports will be used.

I will add basic descriptions for all these exported functions.

Thanks,
Ashish

Quoted 1 line

As a general rule, please push cpu_feature_enabled() checks as early as you reasonably can. They are *VERY* cheap and can even enable the compiler to completely zap code like an #ifdef.

There also seem to be a lot of pfn_valid() checks in here that aren't very well thought out. For instance, there's a pfn_valid() check here:

+int rmp_make_shared(u64 pfn, enum pg_level level) {
+ struct rmpupdate val;
+
+ if (!pfn_valid(pfn))
+ return -EINVAL;
...
+ return rmpupdate(pfn, &val);
+}

and in rmpupdate():

+static int rmpupdate(u64 pfn, struct rmpupdate *val) {
+ unsigned long paddr = pfn << PAGE_SHIFT;
+ int ret;
+
+ if (!pfn_valid(pfn))
+ return -EINVAL;
...

This is (at best) wasteful. Could it be refactored?

author maintainer
#128 lore raw

RE: [PATCH Part2 v6 10/49] x86/fault: Add support to dump RMP entry on fault

In reply to Jeremi Piotrowski

[AMD Official Use Only - General]

Quoted 11 lines
Quoted 11 lines

arch/x86/kernel/sev.c static struct rmpentry *__snp_lookup_rmpentry(u64 pfn, int *level) · line 2424

Quoted 8 lines
arch/x86/kernel/sev.c +8 -0
+void dump_rmpentry(u64 pfn)
+{
+	unsigned long pfn_end;
+	struct rmpentry *e;
+	int level;
+
+	e = __snp_lookup_rmpentry(pfn, &level);
+	if (!e) {

__snp_lookup_rmpentry may return -errno so this should be:

Quoted 5 lines
Quoted 1 line

if (e != 1)

Sorry, actually it should be:

if (IS_ERR_OR_NULL(e)) {

I will fix this accordingly.

Quoted 11 lines
Quoted 7 lines

arch/x86/kernel/sev.c static struct rmpentry *__snp_lookup_rmpentry(u64 pfn, int *level) · line 2449

Quoted 4 lines
arch/x86/kernel/sev.c +4 -0
+
+	while (pfn < pfn_end) {
+		e = __snp_lookup_rmpentry(pfn, &level);
+		if (!e)

if (e != 1)

and this too:

if (IS_ERR_OR_NULL(e))

Same here.

Thanks,
Ashish

author maintainer
#33 lore raw

RE: [PATCH Part2 v6 05/49] x86/sev: Add RMP entry lookup helpers

In reply to Dave Hansen

[Public]

Quoted 9 lines · Ashish
Quoted 4 lines

As I replied previously on the same subject: Architectural implies that it is defined in the APM and shouldn't change in such a way as to not be backward compatible. I probably think the wording here should be architecture independent or more precisely platform independent.

Yeah, arch-independent and non-architectural are quite different concepts.

At Intel, at least, when someone says "not architectural" mean that the behavior is implementation-specific. That, combined with the model/family/stepping gave me the wrong impression about what was going on.

Some more clarity would be greatly appreciated.

Actually, the PPR for family 19h Model 01h, Rev B1 defines the RMP entry format as below:

2.1.4.2 RMP Entry Format
Architecturally the format of RMP entries are not specified in APM. In order to assist software, the following table specifies select portions of the RMP entry format for this specific product. Each RMP entry is 16B in size and is formatted as follows. Software should not rely on any field definitions not specified in this table and the format of an RMP entry may change in future processors.

Architectural implies that it is defined in the APM and shouldn't change in such a way as to not be backward compatible. So non-architectural in this context means that it is only defined in our PPR.

So actually this RPM entry definition is platform dependent and will need to be changed for different AMD processors and that change has to be handled correspondingly in the dump_rmpentry() code.

Thanks,
Ashish

author maintainer
#60 lore raw

RE: [PATCH Part2 v6 06/49] x86/sev: Add helper functions for RMPUPDATE and PSMASH instruction

In reply to Kalra, Ashish

[AMD Official Use Only - General]

Quoted 17 lines
Quoted 6 lines
Quoted 6 lines

/*
* The RMP entry format is not architectural. The format is defined
in PPR @@ -126,6 +128,15 @@ struct snp_guest_platform_data {
u64 secrets_gpa;
};

Quoted 10 lines

arch/x86/include/asm/sev.h struct snp_guest_platform_data { · line 138

Quoted 8 lines
arch/x86/include/asm/sev.h +8 -0
+struct rmpupdate {
+	u64 gpa;
+	u8 assigned;
+	u8 pagesize;
+	u8 immutable;
+	u8 rsvd;
+	u32 asid;
+} __packed;

I see above it says the RMP entry format isn't architectural; is this 'rmpupdate' structure? If not how is this going to get handled when we have a couple >of SNP capable CPUs with different layouts?

Quoted 2 lines

Architectural implies that it is defined in the APM and shouldn't change in such a way as to not be backward compatible. I probably think the wording here should be architecture independent or more precisely platform independent.

Some more clarity on this:

Actually, the PPR for family 19h Model 01h, Rev B1 defines the RMP entry format as below:

2.1.4.2 RMP Entry Format
Architecturally the format of RMP entries are not specified in APM. In order to assist software, the following table specifies select portions of the RMP entry format for this specific product. Each RMP entry is 16B in size and is formatted as follows. Software should not rely on any field definitions not specified in this table and the format of an RMP entry may change in future processors.

Architectural implies that it is defined in the APM and shouldn't change in such a way as to not be backward compatible. So non-architectural in this context means that it is only defined in our PPR.

So actually this RPM entry definition is platform dependent and will need to be changed for different AMD processors and that change has to be handled correspondingly in the dump_rmpentry() code.

Thanks,
Ashish

#34 lore raw

Re: [PATCH Part2 v6 05/49] x86/sev: Add RMP entry lookup helpers

In reply to Kalra, Ashish
Quoted 3 lines · Ashish

So actually this RPM entry definition is platform dependent and will need to be changed for different AMD processors and that change has to be handled correspondingly in the dump_rmpentry() code.

So, if the RMP entry format changes in future processors, how do we make sure that the kernel does not try to use *this* code on those processors?

author maintainer
#35 lore raw

RE: [PATCH Part2 v6 05/49] x86/sev: Add RMP entry lookup helpers

[AMD Official Use Only - General]

-----Original Message-----
From: Dave Hansen <[email protected]>
Sent: Wednesday, June 22, 2022 1:18 PM
To: Kalra, Ashish <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Lendacky, Thomas <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Roth, Michael <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH Part2 v6 05/49] x86/sev: Add RMP entry lookup helpers

Quoted 5 lines · Ashish

So actually this RPM entry definition is platform dependent and will need to be changed for different AMD processors and that change has to be handled correspondingly in the dump_rmpentry() code.

So, if the RMP entry format changes in future processors, how do we make sure that the kernel does not try to use *this* code on those processors?

Functions snp_lookup_rmpentry() and dump_rmpentry() which rely on this structure definition will need to handle it accordingly.

Thanks,
Ashish

#36 lore raw

Re: [PATCH Part2 v6 05/49] x86/sev: Add RMP entry lookup helpers

Quoted 5 lines · Ashish
Quoted 3 lines

So, if the RMP entry format changes in future processors, how do we make sure that the kernel does not try to use *this* code on those processors?

Functions snp_lookup_rmpentry() and dump_rmpentry() which rely on this structure definition will need to handle it accordingly.

In other words, old kernels will break on new hardware?

I think that needs to be fixed. It should be as simple as a model/family check, though. If someone (for example) attempts to use SNP (and thus snp_lookup_rmpentry() and dump_rmpentry()) code on a newer CPU, the kernel should refuse.

author maintainer
#37 lore raw

Re: [PATCH Part2 v6 05/49] x86/sev: Add RMP entry lookup helpers

[AMD Official Use Only - General]

[AMD Official Use Only - General]

________________________________
From: Dave Hansen <[email protected]>
Sent: Wednesday, June 22, 2022 1:43 PM
To: Kalra, Ashish <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>
Cc: [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; Lendacky, Thomas <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; Roth, Michael <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>
Subject: Re: [PATCH Part2 v6 05/49] x86/sev: Add RMP entry lookup helpers

Quoted 6 lines · Ashish
Quoted 3 lines

So, if the RMP entry format changes in future processors, how do we make sure that the kernel does not try to use *this* code on those processors?

Functions snp_lookup_rmpentry() and dump_rmpentry() which rely on this structure definition will need to handle it accordingly.

Quoted 6 lines

In other words, old kernels will break on new >hardware?

I think that needs to be fixed. It should be as >simple as a model/family check, though. If someone (for >example) attempts to use SNP (and thus snp_lookup_rmpentry() and >dump_rmpentry()) code on a newer CPU, the kernel should refuse.

More specifically I am thinking of adding RMP entry field accessors so that they can do this cpu model/family check and return the correct field as per processor architecture.

Thanks,
Ashish

author maintainer
#38 lore raw

RE: [PATCH Part2 v6 05/49] x86/sev: Add RMP entry lookup helpers

In reply to Dave Hansen

[AMD Official Use Only - General]

-----Original Message-----
From: Dave Hansen <[email protected]>
Sent: Wednesday, June 22, 2022 1:43 PM
To: Kalra, Ashish <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Lendacky, Thomas <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Roth, Michael <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH Part2 v6 05/49] x86/sev: Add RMP entry lookup helpers

Quoted 9 lines · Ashish
Quoted 5 lines
Quoted 3 lines

So, if the RMP entry format changes in future processors, how do we make sure that the kernel does not try to use *this* code on those processors?

Functions snp_lookup_rmpentry() and dump_rmpentry() which rely on this structure definition will need to handle it accordingly.

In other words, old kernels will break on new hardware?

I think that needs to be fixed. It should be as simple as a model/family check, though. If someone (for example) attempts to use SNP (and thus snp_lookup_rmpentry() and dump_rmpentry()) code on a newer CPU, the kernel should refuse.

More specifically I am thinking of adding RMP entry field accessors so that they can do this cpu model/family check and return the correct field as per processor architecture.

Thanks,
Ashish

#39 lore raw

Re: [PATCH Part2 v6 05/49] x86/sev: Add RMP entry lookup helpers

Quoted 7 lines · Ashish
Quoted 4 lines

I think that needs to be fixed. It should be as simple as a model/family check, though. If someone (for example) attempts to use SNP (and thus snp_lookup_rmpentry() and dump_rmpentry()) code on a newer CPU, the kernel should refuse.

More specifically I am thinking of adding RMP entry field accessors so that they can do this cpu model/family check and return the correct field as per processor architecture.

That will be helpful down the road when there's more than one format. But, the real issue is that the kernel doesn't *support* a different RMP format. So, the SNP support should be disabled when encountering a model/family other than the known good one.

author maintainer
#40 lore raw

RE: [PATCH Part2 v6 05/49] x86/sev: Add RMP entry lookup helpers

[AMD Official Use Only - General]

From: Dave Hansen <[email protected]>
Sent: Wednesday, June 22, 2022 2:50 PM
To: Kalra, Ashish <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Lendacky, Thomas <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Roth, Michael <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH Part2 v6 05/49] x86/sev: Add RMP entry lookup helpers

Quoted 10 lines · Ashish
Quoted 7 lines
Quoted 4 lines

I think that needs to be fixed. It should be as simple as a model/family check, though. If someone (for example) attempts to use SNP (and thus snp_lookup_rmpentry() and dump_rmpentry()) code on a newer CPU, the kernel should refuse.

More specifically I am thinking of adding RMP entry field accessors so that they can do this cpu model/family check and return the correct field as per processor architecture.

That will be helpful down the road when there's more than one format.
But, the real issue is that the kernel doesn't *support* a different RMP format. So, the SNP support should be disabled when encountering a model/family other than the known good one.

Yes, that makes sense, will add an additional check in snp_rmptable_init().

Thanks,
Ashish

author maintainer
#41 lore raw

RE: [PATCH Part2 v6 05/49] x86/sev: Add RMP entry lookup helpers

[Public]

From: Dave Hansen <[email protected]>
Sent: Wednesday, June 22, 2022 2:50 PM
To: Kalra, Ashish <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Lendacky, Thomas <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Roth, Michael <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH Part2 v6 05/49] x86/sev: Add RMP entry lookup helpers

Quoted 4 lines · Ashish
Quoted 4 lines
Quoted 4 lines

I think that needs to be fixed. It should be as simple as a model/family check, though. If someone (for example) attempts to use SNP (and thus snp_lookup_rmpentry() and dump_rmpentry()) code on a newer CPU, the kernel should refuse.

Quoted 8 lines
Quoted 3 lines

More specifically I am thinking of adding RMP entry field accessors so that they can do this cpu model/family check and return the correct field as per processor architecture.

That will be helpful down the road when there's more than one format.
But, the real issue is that the kernel doesn't *support* a different RMP format. So, the SNP support should be disabled when encountering a model/family other than the known good one.

Yes, that makes sense, will add an additional check in snp_rmptable_init().

Also to add here, additionally we may create an architectural way to read the RMP entry in the future.

Thanks,
Ashish

#19 lore raw

Re: [PATCH Part2 v6 03/49] x86/sev: Add the host SEV-SNP initialization support

In reply to Ashish Kalra
Quoted 70 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

The memory integrity guarantees of SEV-SNP are enforced through a new structure called the Reverse Map Table (RMP). The RMP is a single data structure shared across the system that contains one entry for every 4K page of DRAM that may be used by SEV-SNP VMs. The goal of RMP is to track the owner of each page of memory. Pages of memory can be owned by the hypervisor, owned by a specific VM or owned by the AMD-SP. See APM2 section 15.36.3 for more detail on RMP.

The RMP table is used to enforce access control to memory. The table itself is not directly writable by the software. New CPU instructions (RMPUPDATE, PVALIDATE, RMPADJUST) are used to manipulate the RMP entries.

Based on the platform configuration, the BIOS reserves the memory used for the RMP table. The start and end address of the RMP table must be queried by reading the RMP_BASE and RMP_END MSRs. If the RMP_BASE and RMP_END are not set then disable the SEV-SNP feature.

The SEV-SNP feature is enabled only after the RMP table is successfully initialized.

---
 arch/x86/include/asm/disabled-features.h |   8 +-
 arch/x86/include/asm/msr-index.h         |   6 +
 arch/x86/kernel/sev.c                    | 144 +++++++++++++++++++++++
 3 files changed, 157 insertions(+), 1 deletion(-)
arch/x86/include/asm/disabled-features.h +7 -1
index 36369e76cc63..c1be3091a383 100644
@@ -68,6 +68,12 @@
 # define DISABLE_TDX_GUEST     (1 << (X86_FEATURE_TDX_GUEST & 31))
 #endif
 
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+# define DISABLE_SEV_SNP       0
+#else
+# define DISABLE_SEV_SNP       (1 << (X86_FEATURE_SEV_SNP & 31))
+#endif
+
 /*
  * Make sure to add features to the correct mask
  */
@@ -91,7 +97,7 @@
                         DISABLE_ENQCMD)
 #define DISABLED_MASK17        0
 #define DISABLED_MASK18        0
-#define DISABLED_MASK19        0
+#define DISABLED_MASK19        (DISABLE_SEV_SNP)
 #define DISABLED_MASK_CHECK BUILD_BUG_ON_ZERO(NCAPINTS != 20)
 
 #endif /* _ASM_X86_DISABLED_FEATURES_H */
arch/x86/include/asm/msr-index.h +2 -0
index 9e2e7185fc1d..57a8280e283a 100644
@@ -507,6 +507,8 @@
 #define MSR_AMD64_SEV_ENABLED          BIT_ULL(MSR_AMD64_SEV_ENABLED_BIT)
 #define MSR_AMD64_SEV_ES_ENABLED       BIT_ULL(MSR_AMD64_SEV_ES_ENABLED_BIT)
 #define MSR_AMD64_SEV_SNP_ENABLED      BIT_ULL(MSR_AMD64_SEV_SNP_ENABLED_BIT)
+#define MSR_AMD64_RMP_BASE             0xc0010132
+#define MSR_AMD64_RMP_END              0xc0010133
 
 #define MSR_AMD64_VIRT_SPEC_CTRL       0xc001011f
 

arch/x86/include/asm/msr-index.h line 589

Quoted 7 lines
arch/x86/include/asm/msr-index.h +4 -0
@@ -581,6 +583,10 @@
 #define MSR_AMD64_SYSCFG               0xc0010010
 #define MSR_AMD64_SYSCFG_MEM_ENCRYPT_BIT       23
 #define MSR_AMD64_SYSCFG_MEM_ENCRYPT   BIT_ULL(MSR_AMD64_SYSCFG_MEM_ENCRYPT_BIT)
+#define MSR_AMD64_SYSCFG_SNP_EN_BIT            24
+#define MSR_AMD64_SYSCFG_SNP_EN                BIT_ULL(MSR_AMD64_SYSCFG_SNP_EN_BIT)
+#define MSR_AMD64_SYSCFG_SNP_VMPL_EN_BIT       25
+#define MSR_AMD64_SYSCFG_SNP_VMPL_EN   BIT_ULL(MSR_AMD64_SYSCFG_SNP_VMPL_EN_BIT)

nit: The alignment here looks off. The rest of the file left-aligns the macro definition column under a comment header. The bad alignment can be viewed on the github version of this patch:
https://github.com/AMDESE/linux/commit/5101daef92f448c046207b701c0c420b1fce3eaf

Quoted 164 lines
arch/x86/include/asm/msr-index.h +0 -0
 #define MSR_K8_INT_PENDING_MSG         0xc0010055
 /* C1E active bits in int pending message */
 #define K8_INTP_C1E_ACTIVE_MASK                0x18000000
arch/x86/kernel/sev.c +125 -0
index f01f4550e2c6..3a233b5d47c5 100644
@@ -22,6 +22,8 @@
 #include <linux/efi.h>
 #include <linux/platform_device.h>
 #include <linux/io.h>
+#include <linux/cpumask.h>
+#include <linux/iommu.h>
 
 #include <asm/cpu_entry_area.h>
 #include <asm/stacktrace.h>
@@ -38,6 +40,7 @@
 #include <asm/apic.h>
 #include <asm/cpuid.h>
 #include <asm/cmdline.h>
+#include <asm/iommu.h>
 
 #define DR7_RESET_VALUE        0x400
 
@@ -57,6 +60,12 @@
 #define AP_INIT_CR0_DEFAULT            0x60000010
 #define AP_INIT_MXCSR_DEFAULT          0x1f80
 
+/*
+ * The first 16KB from the RMP_BASE is used by the processor for the
+ * bookkeeping, the range need to be added during the RMP entry lookup.
+ */
+#define RMPTABLE_CPU_BOOKKEEPING_SZ    0x4000
+
 /* For early boot hypervisor communication in SEV-ES enabled guests */
 static struct ghcb boot_ghcb_page __bss_decrypted __aligned(PAGE_SIZE);
 
@@ -69,6 +78,10 @@ static struct ghcb *boot_ghcb __section(".data");
 /* Bitmap of SEV features supported by the hypervisor */
 static u64 sev_hv_features __ro_after_init;
 
+static unsigned long rmptable_start __ro_after_init;
+static unsigned long rmptable_end __ro_after_init;
+
+
 /* #VC handler runtime per-CPU data */
 struct sev_es_runtime_data {
        struct ghcb ghcb_page;
@@ -2218,3 +2231,134 @@ static int __init snp_init_platform_device(void)
        return 0;
 }
 device_initcall(snp_init_platform_device);
+
+#undef pr_fmt
+#define pr_fmt(fmt)    "SEV-SNP: " fmt
+
+static int __snp_enable(unsigned int cpu)
+{
+       u64 val;
+
+       if (!cpu_feature_enabled(X86_FEATURE_SEV_SNP))
+               return 0;
+
+       rdmsrl(MSR_AMD64_SYSCFG, val);
+
+       val |= MSR_AMD64_SYSCFG_SNP_EN;
+       val |= MSR_AMD64_SYSCFG_SNP_VMPL_EN;
+
+       wrmsrl(MSR_AMD64_SYSCFG, val);
+
+       return 0;
+}
+
+static __init void snp_enable(void *arg)
+{
+       __snp_enable(smp_processor_id());
+}
+
+static bool get_rmptable_info(u64 *start, u64 *len)
+{
+       u64 calc_rmp_sz, rmp_sz, rmp_base, rmp_end, nr_pages;
+
+       rdmsrl(MSR_AMD64_RMP_BASE, rmp_base);
+       rdmsrl(MSR_AMD64_RMP_END, rmp_end);
+
+       if (!rmp_base || !rmp_end) {
+               pr_info("Memory for the RMP table has not been reserved by BIOS\n");
+               return false;
+       }
+
+       rmp_sz = rmp_end - rmp_base + 1;
+
+       /*
+        * Calculate the amount the memory that must be reserved by the BIOS to
+        * address the full system RAM. The reserved memory should also cover the
+        * RMP table itself.
+        *
+        * See PPR Family 19h Model 01h, Revision B1 section 2.1.4.2 for more
+        * information on memory requirement.
+        */
+       nr_pages = totalram_pages();
+       calc_rmp_sz = (((rmp_sz >> PAGE_SHIFT) + nr_pages) << 4) + RMPTABLE_CPU_BOOKKEEPING_SZ;
+
+       if (calc_rmp_sz > rmp_sz) {
+               pr_info("Memory reserved for the RMP table does not cover full system RAM (expected 0x%llx got 0x%llx)\n",
+                       calc_rmp_sz, rmp_sz);
+               return false;
+       }
+
+       *start = rmp_base;
+       *len = rmp_sz;
+
+       pr_info("RMP table physical address 0x%016llx - 0x%016llx\n", rmp_base, rmp_end);
+
+       return true;
+}
+
+static __init int __snp_rmptable_init(void)
+{
+       u64 rmp_base, sz;
+       void *start;
+       u64 val;
+
+       if (!get_rmptable_info(&rmp_base, &sz))
+               return 1;
+
+       start = memremap(rmp_base, sz, MEMREMAP_WB);
+       if (!start) {
+               pr_err("Failed to map RMP table 0x%llx+0x%llx\n", rmp_base, sz);
+               return 1;
+       }
+
+       /*
+        * Check if SEV-SNP is already enabled, this can happen if we are coming from
+        * kexec boot.
+        */
+       rdmsrl(MSR_AMD64_SYSCFG, val);
+       if (val & MSR_AMD64_SYSCFG_SNP_EN)
+               goto skip_enable;
+
+       /* Initialize the RMP table to zero */
+       memset(start, 0, sz);
+
+       /* Flush the caches to ensure that data is written before SNP is enabled. */
+       wbinvd_on_all_cpus();
+
+       /* Enable SNP on all CPUs. */
+       on_each_cpu(snp_enable, NULL, 1);
+
+skip_enable:
+       rmptable_start = (unsigned long)start;
+       rmptable_end = rmptable_start + sz;
+
+       return 0;
+}
+
+static int __init snp_rmptable_init(void)
+{
+       if (!boot_cpu_has(X86_FEATURE_SEV_SNP))
+               return 0;
+
+       if (!iommu_sev_snp_supported())
+               goto nosnp;
+

arch/x86/kernel/sev.c static int __init snp_init_platform_device(void) · line 2355

Quoted 10 lines
arch/x86/kernel/sev.c +10 -0
+       if (__snp_rmptable_init())
+               goto nosnp;
+
+       cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "x86/rmptable_init:online", __snp_enable, NULL);
+
+       return 0;
+
+nosnp:
+       setup_clear_cpu_cap(X86_FEATURE_SEV_SNP);
+       return 1;

Seems odd that we're returning 1 here, rather than 0. I tried to figure out how the initcall return values are used and failed. My impression was 0 means success and a negative number means failure. But maybe this is normal.

Quoted 12 lines
arch/x86/kernel/sev.c +9 -1
+}
+
+/*
+ * This must be called after the PCI subsystem. This is because before enabling
+ * the SNP feature we need to ensure that IOMMU supports the SEV-SNP feature.
+ * The iommu_sev_snp_support() is used for checking the feature, and it is
+ * available after subsys_initcall().
+ */
+fs_initcall(snp_rmptable_init);
--

2.25.1

#26 lore raw

Re: [PATCH Part2 v6 04/49] x86/sev: set SYSCFG.MFMD

In reply to Ashish Kalra
Quoted 19 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

SEV-SNP FW >= 1.51 requires that SYSCFG.MFMD must be set.

Subsequent CCP patches while require 1.51 as the minimum SEV-SNP firmware version.

---
 arch/x86/include/asm/msr-index.h |  3 +++
 arch/x86/kernel/sev.c            | 24 ++++++++++++++++++++++++
 2 files changed, 27 insertions(+)
arch/x86/include/asm/msr-index.h +0 -0
index 57a8280e283a..1e36f16daa56 100644

arch/x86/include/asm/msr-index.h line 591

Quoted 5 lines
arch/x86/include/asm/msr-index.h +2 -0
@@ -587,6 +587,9 @@
 #define MSR_AMD64_SYSCFG_SNP_EN                BIT_ULL(MSR_AMD64_SYSCFG_SNP_EN_BIT)
 #define MSR_AMD64_SYSCFG_SNP_VMPL_EN_BIT       25
 #define MSR_AMD64_SYSCFG_SNP_VMPL_EN   BIT_ULL(MSR_AMD64_SYSCFG_SNP_VMPL_EN_BIT)
+#define MSR_AMD64_SYSCFG_MFDM_BIT              19
+#define MSR_AMD64_SYSCFG_MFDM          BIT_ULL(MSR_AMD64_SYSCFG_MFDM_BIT)

nit: Similar to the previous patch, the alignment here doesn't look right. The bad alignment can be viewed on the github version of this patch:
https://github.com/AMDESE/linux/commit/6d4469b86f90e67119ff110230857788a0d9dbd0

Quoted 12 lines
arch/x86/include/asm/msr-index.h +1 -0
+
 #define MSR_K8_INT_PENDING_MSG         0xc0010055
 /* C1E active bits in int pending message */
 #define K8_INTP_C1E_ACTIVE_MASK                0x18000000
arch/x86/kernel/sev.c +0 -0
index 3a233b5d47c5..25c7feb367f6 100644
@@ -2257,6 +2257,27 @@ static __init void snp_enable(void *arg)
        __snp_enable(smp_processor_id());
 }
 

arch/x86/kernel/sev.c static __init void snp_enable(void *arg) · line 2269

Quoted 10 lines
arch/x86/kernel/sev.c +10 -0
+static int __mfdm_enable(unsigned int cpu)
+{
+       u64 val;
+
+       if (!cpu_feature_enabled(X86_FEATURE_SEV_SNP))
+               return 0;
+
+       rdmsrl(MSR_AMD64_SYSCFG, val);
+
+       val |= MSR_AMD64_SYSCFG_MFDM;

Can we do this inside __snp_enable(), above? Then, we'll execute if a hotplug event happens as well.

static int __snp_enable(unsigned int cpu)

{
     u64 val;

     if (!cpu_feature_enabled(X86_FEATURE_SEV_SNP))
          return 0;

     rdmsrl(MSR_AMD64_SYSCFG, val);

     val |= MSR_AMD64_SYSCFG_SNP_EN;
     val |= MSR_AMD64_SYSCFG_SNP_VMPL_EN;
     val |= MSR_AMD64_SYSCFG_MFDM;

     wrmsrl(MSR_AMD64_SYSCFG, val);

     return 0;
}
Quoted 27 lines
arch/x86/kernel/sev.c +14 -1
+
+       wrmsrl(MSR_AMD64_SYSCFG, val);
+
+       return 0;
+}
+
+static __init void mfdm_enable(void *arg)
+{
+       __mfdm_enable(smp_processor_id());
+}
+
 static bool get_rmptable_info(u64 *start, u64 *len)
 {
        u64 calc_rmp_sz, rmp_sz, rmp_base, rmp_end, nr_pages;
@@ -2325,6 +2346,9 @@ static __init int __snp_rmptable_init(void)
        /* Flush the caches to ensure that data is written before SNP is enabled. */
        wbinvd_on_all_cpus();
 
+       /* MFDM must be enabled on all the CPUs prior to enabling SNP. */
+       on_each_cpu(mfdm_enable, NULL, 1);
+
        /* Enable SNP on all CPUs. */
        on_each_cpu(snp_enable, NULL, 1);
 
--

2.25.1

#52 lore raw

Re: [PATCH Part2 v6 05/49] x86/sev: Add RMP entry lookup helpers

In reply to Ashish Kalra
Quoted 78 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

The snp_lookup_page_in_rmptable() can be used by the host to read the RMP entry for a given page. The RMP entry format is documented in AMD PPR, see
https://bugzilla.kernel.org/attachment.cgi?id=296015.

---
 arch/x86/include/asm/sev.h | 27 ++++++++++++++++++++++++
 arch/x86/kernel/sev.c      | 43 ++++++++++++++++++++++++++++++++++++++
 include/linux/sev.h        | 30 ++++++++++++++++++++++++++
 3 files changed, 100 insertions(+)
 create mode 100644 include/linux/sev.h
arch/x86/include/asm/sev.h +27 -0
index 9c2d33f1cfee..cb16f0e5b585 100644
@@ -9,6 +9,7 @@
 #define __ASM_ENCRYPTED_STATE_H
 
 #include <linux/types.h>
+#include <linux/sev.h>
 #include <asm/insn.h>
 #include <asm/sev-common.h>
 #include <asm/bootparam.h>
@@ -84,6 +85,32 @@ extern bool handle_vc_boot_ghcb(struct pt_regs *regs);
 
 /* RMP page size */
 #define RMP_PG_SIZE_4K                 0
+#define RMP_TO_X86_PG_LEVEL(level)     (((level) == RMP_PG_SIZE_4K) ? PG_LEVEL_4K : PG_LEVEL_2M)
+
+/*
+ * The RMP entry format is not architectural. The format is defined in PPR
+ * Family 19h Model 01h, Rev B1 processor.
+ */
+struct __packed rmpentry {
+       union {
+               struct {
+                       u64     assigned        : 1,
+                               pagesize        : 1,
+                               immutable       : 1,
+                               rsvd1           : 9,
+                               gpa             : 39,
+                               asid            : 10,
+                               vmsa            : 1,
+                               validated       : 1,
+                               rsvd2           : 1;
+               } info;
+               u64 low;
+       };
+       u64 high;
+};
+
+#define rmpentry_assigned(x)   ((x)->info.assigned)
+#define rmpentry_pagesize(x)   ((x)->info.pagesize)
 
 #define RMPADJUST_VMSA_PAGE_BIT                BIT(16)
 
arch/x86/kernel/sev.c +3 -0
index 25c7feb367f6..59e7ec6b0326 100644
@@ -65,6 +65,8 @@
  * bookkeeping, the range need to be added during the RMP entry lookup.
  */
 #define RMPTABLE_CPU_BOOKKEEPING_SZ    0x4000
+#define RMPENTRY_SHIFT                 8
+#define rmptable_page_offset(x)        (RMPTABLE_CPU_BOOKKEEPING_SZ + (((unsigned long)x) >> RMPENTRY_SHIFT))
 
 /* For early boot hypervisor communication in SEV-ES enabled guests */
 static struct ghcb boot_ghcb_page __bss_decrypted __aligned(PAGE_SIZE);
@@ -2386,3 +2388,44 @@ static int __init snp_rmptable_init(void)
  * available after subsys_initcall().
  */
 fs_initcall(snp_rmptable_init);
+

arch/x86/kernel/sev.c static int __init snp_rmptable_init(void) · line 2401

Quoted 10 lines
arch/x86/kernel/sev.c +10 -0
+static struct rmpentry *__snp_lookup_rmpentry(u64 pfn, int *level)
+{
+       unsigned long vaddr, paddr = pfn << PAGE_SHIFT;
+       struct rmpentry *entry, *large_entry;
+
+       if (!pfn_valid(pfn))
+               return ERR_PTR(-EINVAL);
+
+       if (!cpu_feature_enabled(X86_FEATURE_SEV_SNP))
+               return ERR_PTR(-ENXIO);

nit: I think we should check if SNP is enabled first, before doing anything else. In other words, I think we should move this check above the !pfn_valid() check.

arch/x86/kernel/sev.c static int __init snp_rmptable_init(void) · line 2405

Quoted 4 lines
arch/x86/kernel/sev.c +4 -0
+
+       vaddr = rmptable_start + rmptable_page_offset(paddr);
+       if (unlikely(vaddr > rmptable_end))
+               return ERR_PTR(-ENXIO);

nit: It would be nice to use a different error code here, from the SNP feature check. That way, if this function fails, it's easier to diagnose where the function failed from the error code.

Quoted 40 lines
arch/x86/kernel/sev.c +26 -0
+
+       entry = (struct rmpentry *)vaddr;
+
+       /* Read a large RMP entry to get the correct page level used in RMP entry. */
+       vaddr = rmptable_start + rmptable_page_offset(paddr & PMD_MASK);
+       large_entry = (struct rmpentry *)vaddr;
+       *level = RMP_TO_X86_PG_LEVEL(rmpentry_pagesize(large_entry));
+
+       return entry;
+}
+
+/*
+ * Return 1 if the RMP entry is assigned, 0 if it exists but is not assigned,
+ * and -errno if there is no corresponding RMP entry.
+ */
+int snp_lookup_rmpentry(u64 pfn, int *level)
+{
+       struct rmpentry *e;
+
+       e = __snp_lookup_rmpentry(pfn, level);
+       if (IS_ERR(e))
+               return PTR_ERR(e);
+
+       return !!rmpentry_assigned(e);
+}
+EXPORT_SYMBOL_GPL(snp_lookup_rmpentry);
include/linux/sev.h new file +8 -0
new file mode 100644
index 000000000000..1a68842789e1
@@ -0,0 +1,30 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * AMD Secure Encrypted Virtualization
+ *
+ * Author: Brijesh Singh <[email protected]>
+ */
+
+#ifndef __LINUX_SEV_H

include/linux/sev.h line 18

Quoted 10 lines
include/linux/sev.h new file +10 -0
+#define __LINUX_SEV_H
+
+/* RMUPDATE detected 4K page and 2MB page overlap. */
+#define RMPUPDATE_FAIL_OVERLAP         7
+
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+int snp_lookup_rmpentry(u64 pfn, int *level);
+int psmash(u64 pfn);
+int rmp_make_private(u64 pfn, u64 gpa, enum pg_level level, int asid, bool immutable);
+int rmp_make_shared(u64 pfn, enum pg_level level);

nit: I think the declarations for psmash(), rmp_make_private(), and rmp_make_shared() should be introduced in the patches that have their definitions.

Quoted 15 lines
include/linux/sev.h new file +12 -1
+#else
+static inline int snp_lookup_rmpentry(u64 pfn, int *level) { return 0; }
+static inline int psmash(u64 pfn) { return -ENXIO; }
+static inline int rmp_make_private(u64 pfn, u64 gpa, enum pg_level level, int asid,
+                                  bool immutable)
+{
+       return -ENODEV;
+}
+static inline int rmp_make_shared(u64 pfn, enum pg_level level) { return -ENODEV; }
+
+#endif /* CONFIG_AMD_MEM_ENCRYPT */
+#endif /* __LINUX_SEV_H */
--

2.25.1

author maintainer
#20 lore raw

RE: [PATCH Part2 v6 03/49] x86/sev: Add the host SEV-SNP initialization support

In reply to Marc Orr

[AMD Official Use Only - General]

Quoted 13 lines
Quoted 13 lines
arch/x86/kernel/sev.c +13 -0
+static int __init snp_rmptable_init(void) {
+       if (!boot_cpu_has(X86_FEATURE_SEV_SNP))
+               return 0;
+
+       if (!iommu_sev_snp_supported())
+               goto nosnp;
+
+       if (__snp_rmptable_init())
+               goto nosnp;
+
+       cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, 
+ "x86/rmptable_init:online", __snp_enable, NULL);
+
Quoted 8 lines

arch/x86/kernel/sev.c static int __init snp_init_platform_device(void) · line 2355

Quoted 5 lines
arch/x86/kernel/sev.c +5 -0
+       return 0;
+
+nosnp:
+       setup_clear_cpu_cap(X86_FEATURE_SEV_SNP);
+       return 1;

Seems odd that we're returning 1 here, rather than 0. I tried to figure out how the initcall return values are used and failed. My impression was 0 means success and a negative number means failure. But maybe this is normal.

I think that initcall values are typically ignored, but it should return 0 on success and negative on error. So probably should fix this to return something like -ENOSYS instead of 1.

Thanks,
Ashish

#42 lore raw

Re: [PATCH Part2 v6 05/49] x86/sev: Add RMP entry lookup helpers

In reply to Kalra, Ashish
Quoted 9 lines · Ashish
Quoted 7 lines · Ashish
Quoted 7 lines
Quoted 4 lines

I think that needs to be fixed. It should be as simple as a model/family check, though. If someone (for example) attempts to use SNP (and thus snp_lookup_rmpentry() and dump_rmpentry()) code on a newer CPU, the kernel should refuse.

More specifically I am thinking of adding RMP entry field accessors so that they can do this cpu model/family check and return the correct field as per processor architecture.

Quoted 6 lines
Quoted 4 lines

That will be helpful down the road when there's more than one format. But, the real issue is that the kernel doesn't *support* a different RMP format. So, the SNP support should be disabled when encountering a model/family other than the known good one.

Yes, that makes sense, will add an additional check in snp_rmptable_init().

And as I suggested in v5[*], bury the microarchitectural struct in sev.c so that nothing outside of the few bits of SNP code that absolutely need to know the layout of the struct should even be aware that there's a struct overlay for RMP entries.

[*] https://lore.kernel.org/all/[email protected]

author maintainer
#43 lore raw

RE: [PATCH Part2 v6 05/49] x86/sev: Add RMP entry lookup helpers

[AMD Official Use Only - General]

Quoted 9 lines
Quoted 9 lines · Ashish
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines

I think that needs to be fixed. It should be as simple as a model/family check, though. If someone (for example) attempts to use SNP (and thus snp_lookup_rmpentry() and dump_rmpentry()) code on a newer CPU, the kernel should refuse.

More specifically I am thinking of adding RMP entry field accessors so that they can do this cpu model/family check and return the correct field as per processor architecture.

Quoted 8 lines
Quoted 6 lines
Quoted 4 lines

That will be helpful down the road when there's more than one format. But, the real issue is that the kernel doesn't *support* a different RMP format. So, the SNP support should be disabled when encountering a model/family other than the known good one.

Yes, that makes sense, will add an additional check in snp_rmptable_init().

And as I suggested in v5[*], bury the microarchitectural struct in sev.c so that nothing outside of the few bits of SNP code that absolutely need to know the layout of the struct should even be aware that there's a struct overlay for RMP entries.

Yes, that's a nice way to hide it from the rest of the kernel which does not require access to this structure anyway, in essence, it becomes a private structure.

Thanks,
Ashish

Quoted 1 line

[*] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kernel.org%2Fall%2FYPCAZaROOHNskGlO%40google.com&amp;data=05%7C01%7CAshish.Kalra%40amd.com%7Ce210ec383f654556348c08da5568ca81%>7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637916205851843411%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=6TOpchjhgFg%>2F5JTa%2FqSviiTuehNoZgvTVBuZv6JxsXc%3D&amp;reserved=0

#76 lore raw

Re: [PATCH Part2 v6 07/49] x86/sev: Invalid pages from direct map when adding it to RMP table

In reply to Ashish Kalra
Quoted 8 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

The integrity guarantee of SEV-SNP is enforced through the RMP table. The RMP is used with standard x86 and IOMMU page tables to enforce memory restrictions and page access rights. The RMP check is enforced as soon as SEV-SNP is enabled globally in the system. When hardware encounters an RMP checks failure, it raises a page-fault exception.

nit: "RMP checks ..." -> "RMP-check ..."

Quoted 4 lines

The rmp_make_private() and rmp_make_shared() helpers are used to add or remove the pages from the RMP table. Improve the rmp_make_private() to invalid state so that pages cannot be used in the direct-map after its

nit: "invalid state ..." -> "invalidate state ..."
nit: "... after its" -> "... after they're"

(Here, and in the patch subject too.)

Quoted 1 line

added in the RMP table, and restore to its default valid permission after

nit: "... restore to its ..." -> "... restored to their ..."

Quoted 21 lines

the pages are removed from the RMP table.

---
 arch/x86/kernel/sev.c | 61 ++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 60 insertions(+), 1 deletion(-)
arch/x86/kernel/sev.c +6 -0
index f6c64a722e94..734cddd837f5 100644
@@ -2451,10 +2451,42 @@ int psmash(u64 pfn)
 }
 EXPORT_SYMBOL_GPL(psmash);
 
+static int restore_direct_map(u64 pfn, int npages)
+{
+       int i, ret = 0;
+
+       for (i = 0; i < npages; i++) {
+               ret = set_direct_map_default_noflush(pfn_to_page(pfn + i));

arch/x86/kernel/sev.c int psmash(u64 pfn) · line 2469

Quoted 10 lines
arch/x86/kernel/sev.c +10 -0
+               if (ret)
+                       goto cleanup;
+       }
+
+cleanup:
+       WARN(ret > 0, "Failed to restore direct map for pfn 0x%llx\n", pfn + i);
+       return ret;
+}
+
+static int invalid_direct_map(unsigned long pfn, int npages)

I think we should rename this function to "invalidate_direct_map()".

Quoted 67 lines
arch/x86/kernel/sev.c +44 -2
+{
+       int i, ret = 0;
+
+       for (i = 0; i < npages; i++) {
+               ret = set_direct_map_invalid_noflush(pfn_to_page(pfn + i));
+               if (ret)
+                       goto cleanup;
+       }
+
+       return 0;
+
+cleanup:
+       restore_direct_map(pfn, i);
+       return ret;
+}
+
 static int rmpupdate(u64 pfn, struct rmpupdate *val)
 {
        unsigned long paddr = pfn << PAGE_SHIFT;
-       int ret;
+       int ret, level, npages;
 
        if (!pfn_valid(pfn))
                return -EINVAL;
@@ -2462,11 +2494,38 @@ static int rmpupdate(u64 pfn, struct rmpupdate *val)
        if (!cpu_feature_enabled(X86_FEATURE_SEV_SNP))
                return -ENXIO;
 
+       level = RMP_TO_X86_PG_LEVEL(val->pagesize);
+       npages = page_level_size(level) / PAGE_SIZE;
+
+       /*
+        * If page is getting assigned in the RMP table then unmap it from the
+        * direct map.
+        */
+       if (val->assigned) {
+               if (invalid_direct_map(pfn, npages)) {
+                       pr_err("Failed to unmap pfn 0x%llx pages %d from direct_map\n",
+                              pfn, npages);
+                       return -EFAULT;
+               }
+       }
+
        /* Binutils version 2.36 supports the RMPUPDATE mnemonic. */
        asm volatile(".byte 0xF2, 0x0F, 0x01, 0xFE"
                     : "=a"(ret)
                     : "a"(paddr), "c"((unsigned long)val)
                     : "memory", "cc");
+
+       /*
+        * Restore the direct map after the page is removed from the RMP table.
+        */
+       if (!ret && !val->assigned) {
+               if (restore_direct_map(pfn, npages)) {
+                       pr_err("Failed to map pfn 0x%llx pages %d in direct_map\n",
+                              pfn, npages);
+                       return -EFAULT;
+               }
+       }
+
        return ret;
 }
 
--

2.25.1

#159 lore raw

Re: [PATCH Part2 v6 14/49] crypto: ccp: Handle the legacy TMR allocation when SNP is enabled

In reply to Kalra, Ashish
Quoted 37 lines · Ashish

[Public]

Hello Peter,

Quoted 10 lines

drivers/crypto/ccp/sev-dev.c static int sev_cmd_buffer_len(int cmd) · line 203

Quoted 6 lines
drivers/crypto/ccp/sev-dev.c +6 -0
+static int snp_reclaim_pages(unsigned long pfn, unsigned int npages,
+bool locked) {
+       struct sev_data_snp_page_reclaim data;
+       int ret, err, i, n = 0;
+
+       for (i = 0; i < npages; i++) {

What about setting |n| here too, also the other increments.

for (i = 0, n = 0; i < npages; i++, n++, pfn++)

Yes that is simpler.

Quoted 17 lines

drivers/crypto/ccp/sev-dev.c static int sev_cmd_buffer_len(int cmd)

Quoted 8 lines
drivers/crypto/ccp/sev-dev.c +8 -0
+               memset(&data, 0, sizeof(data));
+               data.paddr = pfn << PAGE_SHIFT;
+
+               if (locked)
+                       ret = __sev_do_cmd_locked(SEV_CMD_SNP_PAGE_RECLAIM, &data, &err);
+               else
+                       ret = sev_do_cmd(SEV_CMD_SNP_PAGE_RECLAIM,
+ &data, &err);

Can we change sev_cmd_mutex to some sort of nesting lock type? That could clean up this if (locked) code.

drivers/crypto/ccp/sev-dev.c +5 -0
+static inline int rmp_make_firmware(unsigned long pfn, int level) {
+       return rmp_make_private(pfn, 0, level, 0, true); }
+
+static int snp_set_rmp_state(unsigned long paddr, unsigned int npages, bool to_fw, bool locked,
+                            bool need_reclaim)
Quoted 1 line

This function can do a lot and when I read the call sites its hard to see what its doing since we have a combination of arguments which tell us what behavior is happening, some of which are not valid (ex: to_fw == true and need_reclaim == true is an >invalid argument combination).

Quoted 7 lines

to_fw is used to make a firmware page and need_reclaim is for freeing the firmware page, so they are going to be mutually exclusive.

I actually can connect with it quite logically with the callers :
snp_alloc_firmware_pages will call with to_fw = true and need_reclaim = false and snp_free_firmware_pages will do the opposite, to_fw = false and need_reclaim = true.

That seems straightforward to look at.

This might be a preference thing but I find it not straightforward. When I am reading through unmap_firmware_writeable() and I see

/* Transition the pre-allocated buffer to the firmware state. */ if (snp_set_rmp_state(__pa(map->host), npages, true, true, false))
return -EFAULT;

I don't actually know what snp_set_rmp_state() is doing unless I go look at the definition and see what all those booleans mean. This is unlike the rmp_make_shared() and rmp_make_private() functions, each of which tells me a lot more about what the function will do just from the name.

Quoted 124 lines
Quoted 3 lines

Also this for loop over |npages| is duplicated from snp_reclaim_pages(). One improvement here is that on the current
snp_reclaim_pages() if we fail to reclaim a page we assume we cannot reclaim the next pages, this may cause us to snp_leak_pages() more pages than we actually need too.

Yes that is true.

Quoted 73 lines

What about something like this?

static snp_leak_page(u64 pfn, enum pg_level level) {
memory_failure(pfn, 0); dump_rmpentry(pfn);
}

static int snp_reclaim_page(u64 pfn, enum pg_level level) {
int ret;
struct sev_data_snp_page_reclaim data;

ret = sev_do_cmd(SEV_CMD_SNP_PAGE_RECLAIM, &data, &err); if (ret)
goto cleanup;

ret = rmp_make_shared(pfn, level);
if (ret)
goto cleanup;

return 0;

cleanup:
snp_leak_page(pfn, level)
}

typedef int (*rmp_state_change_func) (u64 pfn, enum pg_level level);

static int snp_set_rmp_state(unsigned long paddr, unsigned int npages, rmp_state_change_func state_change, rmp_state_change_func cleanup) {
struct sev_data_snp_page_reclaim data;
int ret, err, i, n = 0;

for (i = 0, n = 0; i < npages; i++, n++, pfn++) {
ret = state_change(pfn, PG_LEVEL_4K)
if (ret)

     goto cleanup;

}

return 0;

cleanup:

 for (; i>= 0; i--, n--, pfn--) {
   cleanup(pfn, PG_LEVEL_4K);

}

return ret;
}

Then inside of __snp_alloc_firmware_pages():

snp_set_rmp_state(paddr, npages, rmp_make_firmware, snp_reclaim_page);

And inside of __snp_free_firmware_pages():

snp_set_rmp_state(paddr, npages, snp_reclaim_page, snp_leak_page);

Just a suggestion feel free to ignore. The readability comment could be addressed much less invasively by just making separate functions for each valid combination of arguments here. Like snp_set_rmp_fw_state(), snp_set_rmp_shared_state(), snp_set_rmp_release_state() or something.

Quoted 15 lines
drivers/crypto/ccp/sev-dev.c +15 -0
+static struct page *__snp_alloc_firmware_pages(gfp_t gfp_mask, int
+order, bool locked) {
+       unsigned long npages = 1ul << order, paddr;
+       struct sev_device *sev;
+       struct page *page;
+
+       if (!psp_master || !psp_master->sev_data)
+               return NULL;
+
+       page = alloc_pages(gfp_mask, order);
+       if (!page)
+               return NULL;
+
+       /* If SEV-SNP is initialized then add the page in RMP table. */
+       sev = psp_master->sev_data;
Quoted 8 lines

drivers/crypto/ccp/sev-dev.c static int sev_cmd_buffer_len(int cmd) · line 300

Quoted 6 lines
drivers/crypto/ccp/sev-dev.c +6 -0
+       if (!sev->snp_inited)
+               return page;
+
+       paddr = __pa((unsigned long)page_address(page));
+       if (snp_set_rmp_state(paddr, npages, true, locked, false))
+               return NULL;

So what about the case where snp_set_rmp_state() fails but we were able to reclaim all the pages? Should we be able to signal that to callers so that we could free |page| here? But given this is an error path already maybe we can optimize this in a >follow up series.

Yes, we should actually tie in to snp_reclaim_pages() success or failure here in the case we were able to successfully unroll some or all of the firmware state change.

Quoted 23 lines
drivers/crypto/ccp/sev-dev.c +22 -0
+
+       return page;
+}
+
+void *snp_alloc_firmware_page(gfp_t gfp_mask) {
+       struct page *page;
+
+       page = __snp_alloc_firmware_pages(gfp_mask, 0, false);
+
+       return page ? page_address(page) : NULL; }
+EXPORT_SYMBOL_GPL(snp_alloc_firmware_page);
+
+static void __snp_free_firmware_pages(struct page *page, int order,
+bool locked) {
+       unsigned long paddr, npages = 1ul << order;
+
+       if (!page)
+               return;
+
+       paddr = __pa((unsigned long)page_address(page));
+       if (snp_set_rmp_state(paddr, npages, false, locked, true))
+               return;
Quoted 1 line

Here we may be able to free some of |page| depending how where inside of snp_set_rmp_state() we failed. But again given this is an error path already maybe we can optimize this in a follow up series.

Yes, we probably should be able to free some of the page(s) depending on how many page(s) got reclaimed in snp_set_rmp_state().
But these reclamation failures may not be very common, so any failure is indicative of a bigger issue, it might be the case when there is a single page reclamation error it might happen with all the subsequent pages and so follow a simple recovery procedure, then handling a more complex recovery for a chunk of pages being reclaimed and another chunk not.

Thanks,
Ashish

#223 lore raw

Re: [PATCH Part2 v6 26/49] KVM: SVM: Add KVM_SEV_SNP_LAUNCH_UPDATE command

In reply to Ashish Kalra
Quoted 82 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

The KVM_SEV_SNP_LAUNCH_UPDATE command can be used to insert data into the guest's memory. The data is encrypted with the cryptographic context created with the KVM_SEV_SNP_LAUNCH_START.

In addition to the inserting data, it can insert a two special pages into the guests memory: the secrets page and the CPUID page.

While terminating the guest, reclaim the guest pages added in the RMP table. If the reclaim fails, then the page is no longer safe to be released back to the system and leak them.

For more information see the SEV-SNP specification.

---
 .../virt/kvm/x86/amd-memory-encryption.rst    |  29 +++
 arch/x86/kvm/svm/sev.c                        | 187 ++++++++++++++++++
 include/uapi/linux/kvm.h                      |  19 ++
 3 files changed, 235 insertions(+)
Documentation/virt/kvm/x86/amd-memory-encryption.rst +29 -0
index 878711f2dca6..62abd5c1f72b 100644
@@ -486,6 +486,35 @@ Returns: 0 on success, -negative on error
 
 See the SEV-SNP specification for further detail on the launch input.
 
+20. KVM_SNP_LAUNCH_UPDATE
+-------------------------
+
+The KVM_SNP_LAUNCH_UPDATE is used for encrypting a memory region. It also
+calculates a measurement of the memory contents. The measurement is a signature
+of the memory contents that can be sent to the guest owner as an attestation
+that the memory was encrypted correctly by the firmware.
+
+Parameters (in): struct  kvm_snp_launch_update
+
+Returns: 0 on success, -negative on error
+
+::
+
+        struct kvm_sev_snp_launch_update {
+                __u64 start_gfn;        /* Guest page number to start from. */
+                __u64 uaddr;            /* userspace address need to be encrypted */
+                __u32 len;              /* length of memory region */
+                __u8 imi_page;          /* 1 if memory is part of the IMI */
+                __u8 page_type;         /* page type */
+                __u8 vmpl3_perms;       /* VMPL3 permission mask */
+                __u8 vmpl2_perms;       /* VMPL2 permission mask */
+                __u8 vmpl1_perms;       /* VMPL1 permission mask */
+        };
+
+See the SEV-SNP spec for further details on how to build the VMPL permission
+mask and page type.
+
+
 References
 ==========
 
arch/x86/kvm/svm/sev.c +4 -0
index 41b83aa6b5f4..b5f0707d7ed6 100644
@@ -18,6 +18,7 @@
 #include <linux/processor.h>
 #include <linux/trace_events.h>
 #include <linux/hugetlb.h>
+#include <linux/sev.h>
 
 #include <asm/pkru.h>
 #include <asm/trapnr.h>
@@ -233,6 +234,49 @@ static void sev_decommission(unsigned int handle)
        sev_guest_decommission(&decommission, NULL);
 }
 
+static inline void snp_leak_pages(u64 pfn, enum pg_level level)
+{
+       unsigned int npages = page_level_size(level) >> PAGE_SHIFT;

arch/x86/kvm/svm/sev.c static void sev_decommission(unsigned int handle) · line 249

Quoted 10 lines
arch/x86/kvm/svm/sev.c +10 -0
+
+       WARN(1, "psc failed pfn 0x%llx pages %d (leaking)\n", pfn, npages);
+
+       while (npages) {
+               memory_failure(pfn, 0);
+               dump_rmpentry(pfn);
+               npages--;
+               pfn++;
+       }
+}

Should this be deduplicated with the snp_leak_pages() in "crypto: ccp:
Handle the legacy TMR allocation when SNP is enabled" ?

Quoted 42 lines
arch/x86/kvm/svm/sev.c +35 -0
+
+static int snp_page_reclaim(u64 pfn)
+{
+       struct sev_data_snp_page_reclaim data = {0};
+       int err, rc;
+
+       data.paddr = __sme_set(pfn << PAGE_SHIFT);
+       rc = snp_guest_page_reclaim(&data, &err);
+       if (rc) {
+               /*
+                * If the reclaim failed, then page is no longer safe
+                * to use.
+                */
+               snp_leak_pages(pfn, PG_LEVEL_4K);
+       }
+
+       return rc;
+}
+
+static int host_rmp_make_shared(u64 pfn, enum pg_level level, bool leak)
+{
+       int rc;
+
+       rc = rmp_make_shared(pfn, level);
+       if (rc && leak)
+               snp_leak_pages(pfn, level);
+
+       return rc;
+}
+
 static void sev_unbind_asid(struct kvm *kvm, unsigned int handle)
 {
        struct sev_data_deactivate deactivate;
@@ -1902,6 +1946,123 @@ static int snp_launch_start(struct kvm *kvm, struct kvm_sev_cmd *argp)
        return rc;
 }
 
+static bool is_hva_registered(struct kvm *kvm, hva_t hva, size_t len)
+{
+       struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
+       struct list_head *head = &sev->regions_list;
+       struct enc_region *i;

arch/x86/kvm/svm/sev.c static int snp_launch_start(struct kvm *kvm, struct kvm_sev_cmd *argp) · line 1963

Quoted 10 lines
arch/x86/kvm/svm/sev.c +10 -0
+
+       lockdep_assert_held(&kvm->lock);
+
+       list_for_each_entry(i, head, list) {
+               u64 start = i->uaddr;
+               u64 end = start + i->size;
+
+               if (start <= hva && end >= (hva + len))
+                       return true;
+       }

Given that usersapce could load sev->regions_list with any # of any sized regions. Should we add a cond_resched() like in sev_vm_destroy()?

Quoted 75 lines
arch/x86/kvm/svm/sev.c +75 -0
+
+       return false;
+}
+
+static int snp_launch_update(struct kvm *kvm, struct kvm_sev_cmd *argp)
+{
+       struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
+       struct sev_data_snp_launch_update data = {0};
+       struct kvm_sev_snp_launch_update params;
+       unsigned long npages, pfn, n = 0;
+       int *error = &argp->error;
+       struct page **inpages;
+       int ret, i, level;
+       u64 gfn;
+
+       if (!sev_snp_guest(kvm))
+               return -ENOTTY;
+
+       if (!sev->snp_context)
+               return -EINVAL;
+
+       if (copy_from_user(&params, (void __user *)(uintptr_t)argp->data, sizeof(params)))
+               return -EFAULT;
+
+       /* Verify that the specified address range is registered. */
+       if (!is_hva_registered(kvm, params.uaddr, params.len))
+               return -EINVAL;
+
+       /*
+        * The userspace memory is already locked so technically we don't
+        * need to lock it again. Later part of the function needs to know
+        * pfn so call the sev_pin_memory() so that we can get the list of
+        * pages to iterate through.
+        */
+       inpages = sev_pin_memory(kvm, params.uaddr, params.len, &npages, 1);
+       if (!inpages)
+               return -ENOMEM;
+
+       /*
+        * Verify that all the pages are marked shared in the RMP table before
+        * going further. This is avoid the cases where the userspace may try
+        * updating the same page twice.
+        */
+       for (i = 0; i < npages; i++) {
+               if (snp_lookup_rmpentry(page_to_pfn(inpages[i]), &level) != 0) {
+                       sev_unpin_memory(kvm, inpages, npages);
+                       return -EFAULT;
+               }
+       }
+
+       gfn = params.start_gfn;
+       level = PG_LEVEL_4K;
+       data.gctx_paddr = __psp_pa(sev->snp_context);
+
+       for (i = 0; i < npages; i++) {
+               pfn = page_to_pfn(inpages[i]);
+
+               ret = rmp_make_private(pfn, gfn << PAGE_SHIFT, level, sev_get_asid(kvm), true);
+               if (ret) {
+                       ret = -EFAULT;
+                       goto e_unpin;
+               }
+
+               n++;
+               data.address = __sme_page_pa(inpages[i]);
+               data.page_size = X86_TO_RMP_PG_LEVEL(level);
+               data.page_type = params.page_type;
+               data.vmpl3_perms = params.vmpl3_perms;
+               data.vmpl2_perms = params.vmpl2_perms;
+               data.vmpl1_perms = params.vmpl1_perms;
+               ret = __sev_issue_cmd(argp->sev_fd, SEV_CMD_SNP_LAUNCH_UPDATE, &data, error);
+               if (ret) {
+                       /*
+                        * If the command failed then need to reclaim the page.
+                        */

arch/x86/kvm/svm/sev.c static int snp_launch_start(struct kvm *kvm, struct kvm_sev_cmd *argp) · line 2048

Quoted 10 lines
arch/x86/kvm/svm/sev.c +10 -0
+                       snp_page_reclaim(pfn);
+                       goto e_unpin;
+               }
+
+               gfn++;
+       }
+
+e_unpin:
+       /* Content of memory is updated, mark pages dirty */
+       for (i = 0; i < n; i++) {

Since |n| is not only a loop variable but actually carries the number of private pages over to e_unpin can we use a more descriptive name? How about something like 'nprivate_pages'?

Quoted 100 lines
arch/x86/kvm/svm/sev.c +43 -0
+               set_page_dirty_lock(inpages[i]);
+               mark_page_accessed(inpages[i]);
+
+               /*
+                * If its an error, then update RMP entry to change page ownership
+                * to the hypervisor.
+                */
+               if (ret)
+                       host_rmp_make_shared(pfn, level, true);
+       }
+
+       /* Unlock the user pages */
+       sev_unpin_memory(kvm, inpages, npages);
+
+       return ret;
+}
+
 int sev_mem_enc_ioctl(struct kvm *kvm, void __user *argp)
 {
        struct kvm_sev_cmd sev_cmd;
@@ -1995,6 +2156,9 @@ int sev_mem_enc_ioctl(struct kvm *kvm, void __user *argp)
        case KVM_SEV_SNP_LAUNCH_START:
                r = snp_launch_start(kvm, &sev_cmd);
                break;
+       case KVM_SEV_SNP_LAUNCH_UPDATE:
+               r = snp_launch_update(kvm, &sev_cmd);
+               break;
        default:
                r = -EINVAL;
                goto out;
@@ -2113,6 +2277,29 @@ find_enc_region(struct kvm *kvm, struct kvm_enc_region *range)
 static void __unregister_enc_region_locked(struct kvm *kvm,
                                           struct enc_region *region)
 {
+       unsigned long i, pfn;
+       int level;
+
+       /*
+        * The guest memory pages are assigned in the RMP table. Unassign it
+        * before releasing the memory.
+        */
+       if (sev_snp_guest(kvm)) {
+               for (i = 0; i < region->npages; i++) {
+                       pfn = page_to_pfn(region->pages[i]);
+
+                       if (!snp_lookup_rmpentry(pfn, &level))
+                               continue;
+
+                       cond_resched();
+
+                       if (level > PG_LEVEL_4K)
+                               pfn &= ~(KVM_PAGES_PER_HPAGE(PG_LEVEL_2M) - 1);
+
+                       host_rmp_make_shared(pfn, level, true);
+               }
+       }
+
        sev_unpin_memory(kvm, region->pages, region->npages);
        list_del(&region->list);
        kfree(region);
include/uapi/linux/kvm.h +19 -1
index 0cb119d66ae5..9b36b07414ea 100644
@@ -1813,6 +1813,7 @@ enum sev_cmd_id {
        /* SNP specific commands */
        KVM_SEV_SNP_INIT,
        KVM_SEV_SNP_LAUNCH_START,
+       KVM_SEV_SNP_LAUNCH_UPDATE,
 
        KVM_SEV_NR_MAX,
 };
@@ -1929,6 +1930,24 @@ struct kvm_sev_snp_launch_start {
        __u8 pad[6];
 };
 
+#define KVM_SEV_SNP_PAGE_TYPE_NORMAL           0x1
+#define KVM_SEV_SNP_PAGE_TYPE_VMSA             0x2
+#define KVM_SEV_SNP_PAGE_TYPE_ZERO             0x3
+#define KVM_SEV_SNP_PAGE_TYPE_UNMEASURED       0x4
+#define KVM_SEV_SNP_PAGE_TYPE_SECRETS          0x5
+#define KVM_SEV_SNP_PAGE_TYPE_CPUID            0x6
+
+struct kvm_sev_snp_launch_update {
+       __u64 start_gfn;
+       __u64 uaddr;
+       __u32 len;
+       __u8 imi_page;
+       __u8 page_type;
+       __u8 vmpl3_perms;
+       __u8 vmpl2_perms;
+       __u8 vmpl1_perms;
+};
+
 #define KVM_DEV_ASSIGN_ENABLE_IOMMU    (1 << 0)
 #define KVM_DEV_ASSIGN_PCI_2_3         (1 << 1)
 #define KVM_DEV_ASSIGN_MASK_INTX       (1 << 2)
--

2.25.1

#217 lore raw

Re: [PATCH Part2 v6 24/49] KVM: SVM: Add KVM_SEV_SNP_LAUNCH_START command

In reply to Ashish Kalra
Quoted 9 lines

+19. KVM_SNP_LAUNCH_START
+------------------------

Documentation/virt/kvm/x86/amd-memory-encryption.rst +6 -0
+
+The KVM_SNP_LAUNCH_START command is used for creating the memory encryption
+context for the SEV-SNP guest. To create the encryption context, user must
+provide a guest policy, migration agent (if any) and guest OS visible
+workarounds value as defined SEV-SNP specification.
+

Documentation/virt/kvm/x86/amd-memory-encryption.rst The flags bitmap is defined as:: · line 482

Quoted 10 lines
Documentation/virt/kvm/x86/amd-memory-encryption.rst +10 -0
+Parameters (in): struct  kvm_snp_launch_start
+
+Returns: 0 on success, -negative on error
+
+::
+
+        struct kvm_sev_snp_launch_start {
+                __u64 policy;           /* Guest policy to use. */
+                __u64 ma_uaddr;         /* userspace address of migration agent */
+                __u8 ma_en;             /* 1 if the migtation agent is enabled */

migration

Quoted 13 lines
(quoted diff) +8 -0
+                __u8 imi_en;            /* set IMI to 1. */
+                __u8 gosvw[16];         /* guest OS visible workarounds */
+        };
+
+See the SEV-SNP specification for further detail on the launch input.
+
 References
 ==========
 
 
 
+static int snp_decommission_context(struct kvm *kvm)
+{

Review

Quoted 10 lines
(quoted diff) +10 -0
+       struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
+       struct sev_data_snp_decommission data = {};
+       int ret;
+
+       /* If context is not created then do nothing */
+       if (!sev->snp_context)
+               return 0;
+
+       data.gctx_paddr = __sme_pa(sev->snp_context);
+       ret = snp_guest_decommission(&data, NULL);

Do we have a similar race like in sev_unbind_asid() with DEACTIVATE and WBINVD/DF_FLUSH? The SNP_DECOMMISSION spec looks quite similar to DEACTIVATE.

Quoted 13 lines
arch/x86/kvm/svm/sev.c +10 -0
+       if (WARN_ONCE(ret, "failed to release guest context"))
+               return ret;
+
+       /* free the context page now */
+       snp_free_firmware_page(sev->snp_context);
+       sev->snp_context = NULL;
+
+       return 0;
+}
+
 void sev_vm_destroy(struct kvm *kvm)
 {
        struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
#254 lore raw

Re: [PATCH Part2 v6 35/49] KVM: SVM: Remove the long-lived GHCB host map

In reply to Ashish Kalra
Quoted 36 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

On VMGEXIT, sev_handle_vmgexit() creates a host mapping for the GHCB GPA, and unmaps it just before VM-entry. This long-lived GHCB map is used by the VMGEXIT handler through accessors such as ghcb_{set_get}_xxx().

A long-lived GHCB map can cause issue when SEV-SNP is enabled. When SEV-SNP is enabled the mapped GPA needs to be protected against a page state change.

To eliminate the long-lived GHCB mapping, update the GHCB sync operations to explicitly map the GHCB before access and unmap it after access is complete. This requires that the setting of the GHCBs sw_exit_info_{1,2} fields be done during sev_es_sync_to_ghcb(), so create two new fields in the vcpu_svm struct to hold these values when required to be set outside of the GHCB mapping.

---
 arch/x86/kvm/svm/sev.c | 131 ++++++++++++++++++++++++++---------------
 arch/x86/kvm/svm/svm.c |  12 ++--
 arch/x86/kvm/svm/svm.h |  24 +++++++-
 3 files changed, 111 insertions(+), 56 deletions(-)
arch/x86/kvm/svm/sev.c +3 -0
index 01ea257e17d6..c70f3f7e06a8 100644
@@ -2823,15 +2823,40 @@ void sev_free_vcpu(struct kvm_vcpu *vcpu)
        kvfree(svm->sev_es.ghcb_sa);
 }
 
+static inline int svm_map_ghcb(struct vcpu_svm *svm, struct kvm_host_map *map)
+{
+       struct vmcb_control_area *control = &svm->vmcb->control;

arch/x86/kvm/svm/sev.c void sev_free_vcpu(struct kvm_vcpu *vcpu) · line 2838

Quoted 10 lines
arch/x86/kvm/svm/sev.c +10 -0
+       u64 gfn = gpa_to_gfn(control->ghcb_gpa);
+
+       if (kvm_vcpu_map(&svm->vcpu, gfn, map)) {
+               /* Unable to map GHCB from guest */
+               pr_err("error mapping GHCB GFN [%#llx] from guest\n", gfn);
+               return -EFAULT;
+       }
+
+       return 0;
+}

There is a perf cost to this suggestion but it might make accessing the GHCB safer for KVM. Have you thought about just using kvm_read_guest() or copy_from_user() to fully copy out the GCHB into a KVM owned buffer, then copying it back before the VMRUN. That way the KVM doesn't need to guard against page_state_changes on the GHCBs, that could be a perf improvement in a follow up.

Since we cannot unmap GHCBs I don't think UPM will help here so we probably want to make these patches safe against malicious guests making GHCBs private. But maybe UPM does help?

#289 lore raw

Re: [PATCH Part2 v6 42/49] KVM: SVM: Provide support for SNP_GUEST_REQUEST NAE event

In reply to Ashish Kalra
Quoted 33 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

Version 2 of GHCB specification added the support for two SNP Guest Request Message NAE events. The events allows for an SEV-SNP guest to make request to the SEV-SNP firmware through hypervisor using the SNP_GUEST_REQUEST API define in the SEV-SNP firmware specification.

The SNP_EXT_GUEST_REQUEST is similar to SNP_GUEST_REQUEST with the difference of an additional certificate blob that can be passed through the SNP_SET_CONFIG ioctl defined in the CCP driver. The CCP driver provides snp_guest_ext_guest_request() that is used by the KVM to get both the report and certificate data at once.

---
 arch/x86/kvm/svm/sev.c | 196 +++++++++++++++++++++++++++++++++++++++--
 arch/x86/kvm/svm/svm.h |   2 +
 2 files changed, 192 insertions(+), 6 deletions(-)
arch/x86/kvm/svm/sev.c +1 -0
index 7fc0fad87054..089af21a4efe 100644
@@ -343,6 +343,7 @@ static int sev_guest_init(struct kvm *kvm, struct kvm_sev_cmd *argp)
 
                spin_lock_init(&sev->psc_lock);
                ret = sev_snp_init(&argp->error);
+               mutex_init(&sev->guest_req_lock);
        } else {
                ret = sev_platform_init(&argp->error);
        }

arch/x86/kvm/svm/sev.c int sev_vm_move_enc_context_from(struct kvm *kvm, unsigned int source_fd) · line 1888

Quoted 4 lines
arch/x86/kvm/svm/sev.c +1 -0
@@ -1884,23 +1885,39 @@ int sev_vm_move_enc_context_from(struct kvm *kvm, unsigned int source_fd)
 
 static void *snp_context_create(struct kvm *kvm, struct kvm_sev_cmd *argp)
 {
+       void *context = NULL, *certs_data = NULL, *resp_page = NULL;

Is the NULL setting here unnecessary since all of these are set via functions snp_alloc_firmware_page(), kmalloc(), and snp_alloc_firmware_page() respectively?

arch/x86/kvm/svm/sev.c int sev_vm_move_enc_context_from(struct kvm *kvm, unsigned int source_fd) · line 1896

Quoted 9 lines
arch/x86/kvm/svm/sev.c +5 -1
+       struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
        struct sev_data_snp_gctx_create data = {};
-       void *context;
        int rc;
 
+       /* Allocate memory used for the certs data in SNP guest request */
+       certs_data = kmalloc(SEV_FW_BLOB_MAX_SIZE, GFP_KERNEL_ACCOUNT);
+       if (!certs_data)
+               return NULL;

I think we want to use kzalloc() here to ensure we never give the guest uninitialized kernel memory.

arch/x86/kvm/svm/sev.c int sev_vm_move_enc_context_from(struct kvm *kvm, unsigned int source_fd) · line 1906

Quoted 11 lines
arch/x86/kvm/svm/sev.c +7 -1
+
        /* Allocate memory for context page */
        context = snp_alloc_firmware_page(GFP_KERNEL_ACCOUNT);
        if (!context)
-               return NULL;
+               goto e_free;
+
+       /* Allocate a firmware buffer used during the guest command handling. */
+       resp_page = snp_alloc_firmware_page(GFP_KERNEL_ACCOUNT);
+       if (!resp_page)
+               goto e_free;

|resp_page| doesn't appear to be used anywhere?

Quoted 43 lines
arch/x86/kvm/svm/sev.c +13 -4
        data.gctx_paddr = __psp_pa(context);
        rc = __sev_issue_cmd(argp->sev_fd, SEV_CMD_SNP_GCTX_CREATE, &data, &argp->error);
-       if (rc) {
-               snp_free_firmware_page(context);
-               return NULL;
-       }
+       if (rc)
+               goto e_free;
+
+       sev->snp_certs_data = certs_data;
 
        return context;
+
+e_free:
+       snp_free_firmware_page(context);
+       kfree(certs_data);
+       return NULL;
 }
 
 static int snp_bind_asid(struct kvm *kvm, int *error)
@@ -2565,6 +2582,8 @@ static int snp_decommission_context(struct kvm *kvm)
        snp_free_firmware_page(sev->snp_context);
        sev->snp_context = NULL;
 
+       kfree(sev->snp_certs_data);
+
        return 0;
 }
 
@@ -3077,6 +3096,8 @@ static int sev_es_validate_vmgexit(struct vcpu_svm *svm, u64 *exit_code)
        case SVM_VMGEXIT_UNSUPPORTED_EVENT:
        case SVM_VMGEXIT_HV_FEATURES:
        case SVM_VMGEXIT_PSC:
+       case SVM_VMGEXIT_GUEST_REQUEST:
+       case SVM_VMGEXIT_EXT_GUEST_REQUEST:
                break;
        default:
                reason = GHCB_ERR_INVALID_EVENT;
@@ -3502,6 +3523,155 @@ static unsigned long snp_handle_page_state_change(struct vcpu_svm *svm)
        return rc ? map_to_psc_vmgexit_code(rc) : 0;
 }
 

arch/x86/kvm/svm/sev.c static unsigned long snp_handle_page_state_change(struct vcpu_svm *svm) · line 3535

Quoted 10 lines
arch/x86/kvm/svm/sev.c +10 -0
+static unsigned long snp_setup_guest_buf(struct vcpu_svm *svm,
+                                        struct sev_data_snp_guest_request *data,
+                                        gpa_t req_gpa, gpa_t resp_gpa)
+{
+       struct kvm_vcpu *vcpu = &svm->vcpu;
+       struct kvm *kvm = vcpu->kvm;
+       kvm_pfn_t req_pfn, resp_pfn;
+       struct kvm_sev_info *sev;
+
+       sev = &to_kvm_svm(kvm)->sev_info;

This is normally done at declaration in this file. Why not here?

      struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
Quoted 20 lines
arch/x86/kvm/svm/sev.c +20 -0
+
+       if (!IS_ALIGNED(req_gpa, PAGE_SIZE) || !IS_ALIGNED(resp_gpa, PAGE_SIZE))
+               return SEV_RET_INVALID_PARAM;
+
+       req_pfn = gfn_to_pfn(kvm, gpa_to_gfn(req_gpa));
+       if (is_error_noslot_pfn(req_pfn))
+               return SEV_RET_INVALID_ADDRESS;
+
+       resp_pfn = gfn_to_pfn(kvm, gpa_to_gfn(resp_gpa));
+       if (is_error_noslot_pfn(resp_pfn))
+               return SEV_RET_INVALID_ADDRESS;
+
+       if (rmp_make_private(resp_pfn, 0, PG_LEVEL_4K, 0, true))
+               return SEV_RET_INVALID_ADDRESS;
+
+       data->gctx_paddr = __psp_pa(sev->snp_context);
+       data->req_paddr = __sme_set(req_pfn << PAGE_SHIFT);
+       data->res_paddr = __sme_set(resp_pfn << PAGE_SHIFT);
+
+       return 0;

arch/x86/kvm/svm/sev.c static unsigned long snp_handle_page_state_change(struct vcpu_svm *svm) · line 3565

Quoted 10 lines
arch/x86/kvm/svm/sev.c +10 -0
+}
+
+static void snp_cleanup_guest_buf(struct sev_data_snp_guest_request *data, unsigned long *rc)
+{
+       u64 pfn = __sme_clr(data->res_paddr) >> PAGE_SHIFT;
+       int ret;
+
+       ret = snp_page_reclaim(pfn);
+       if (ret)
+               *rc = SEV_RET_INVALID_ADDRESS;

Do we need a diff error code here? This means the page the guest gives us is now "stuck" in the FW owned state. How would the guest know this is the case? We return the exact same error in snp_setup_guest_buf() if the resp_gpa isn't page aligned so now if the guest ever sees a SEV_RET_INVALID_ADDRESS I think its only safe option is to either try and page_state_change it to a know state or mark it as unusable memory.

arch/x86/kvm/svm/sev.c static unsigned long snp_handle_page_state_change(struct vcpu_svm *svm) · line 3569

Quoted 4 lines
arch/x86/kvm/svm/sev.c +4 -0
+
+       ret = rmp_make_shared(pfn, PG_LEVEL_4K);
+       if (ret)
+               *rc = SEV_RET_INVALID_ADDRESS;

Ditto here I think we need some way to signal to the guest what state this page is on return to guest execution.

Also these errors shadow over FW successes, this means the guest's guest-request-sequence-numbers are now out of sync meaning this VMPCK is unusable less the guest risk reusing the AES IV (which would break the confidentiality/integrity). Should we have a way to signal to the guest the FW has successfully run your command but we could not change the page states back correctly, so the guest should increment their sequence numbers.

Quoted 7 lines
arch/x86/kvm/svm/sev.c +7 -0
+}
+
+static void snp_handle_guest_request(struct vcpu_svm *svm, gpa_t req_gpa, gpa_t resp_gpa)
+{
+       struct sev_data_snp_guest_request data = {0};
+       struct kvm_vcpu *vcpu = &svm->vcpu;
+       struct kvm *kvm = vcpu->kvm;

arch/x86/kvm/svm/sev.c static unsigned long snp_handle_page_state_change(struct vcpu_svm *svm) · line 3586

Quoted 10 lines
arch/x86/kvm/svm/sev.c +10 -0
+       struct kvm_sev_info *sev;
+       unsigned long rc;
+       int err;
+
+       if (!sev_snp_guest(vcpu->kvm)) {
+               rc = SEV_RET_INVALID_GUEST;
+               goto e_fail;
+       }
+
+       sev = &to_kvm_svm(kvm)->sev_info;

Ditto why not due this above?

Quoted 67 lines
arch/x86/kvm/svm/sev.c +67 -0
+
+       mutex_lock(&sev->guest_req_lock);
+
+       rc = snp_setup_guest_buf(svm, &data, req_gpa, resp_gpa);
+       if (rc)
+               goto unlock;
+
+       rc = sev_issue_cmd(kvm, SEV_CMD_SNP_GUEST_REQUEST, &data, &err);
+       if (rc)
+               /* use the firmware error code */
+               rc = err;
+
+       snp_cleanup_guest_buf(&data, &rc);
+
+unlock:
+       mutex_unlock(&sev->guest_req_lock);
+
+e_fail:
+       svm_set_ghcb_sw_exit_info_2(vcpu, rc);
+}
+
+static void snp_handle_ext_guest_request(struct vcpu_svm *svm, gpa_t req_gpa, gpa_t resp_gpa)
+{
+       struct sev_data_snp_guest_request req = {0};
+       struct kvm_vcpu *vcpu = &svm->vcpu;
+       struct kvm *kvm = vcpu->kvm;
+       unsigned long data_npages;
+       struct kvm_sev_info *sev;
+       unsigned long rc, err;
+       u64 data_gpa;
+
+       if (!sev_snp_guest(vcpu->kvm)) {
+               rc = SEV_RET_INVALID_GUEST;
+               goto e_fail;
+       }
+
+       sev = &to_kvm_svm(kvm)->sev_info;
+
+       data_gpa = vcpu->arch.regs[VCPU_REGS_RAX];
+       data_npages = vcpu->arch.regs[VCPU_REGS_RBX];
+
+       if (!IS_ALIGNED(data_gpa, PAGE_SIZE)) {
+               rc = SEV_RET_INVALID_ADDRESS;
+               goto e_fail;
+       }
+
+       /* Verify that requested blob will fit in certificate buffer */
+       if ((data_npages << PAGE_SHIFT) > SEV_FW_BLOB_MAX_SIZE) {
+               rc = SEV_RET_INVALID_PARAM;
+               goto e_fail;
+       }
+
+       mutex_lock(&sev->guest_req_lock);
+
+       rc = snp_setup_guest_buf(svm, &req, req_gpa, resp_gpa);
+       if (rc)
+               goto unlock;
+
+       rc = snp_guest_ext_guest_request(&req, (unsigned long)sev->snp_certs_data,
+                                        &data_npages, &err);
+       if (rc) {
+               /*
+                * If buffer length is small then return the expected
+                * length in rbx.
+                */
+               if (err == SNP_GUEST_REQ_INVALID_LEN)
+                       vcpu->arch.regs[VCPU_REGS_RBX] = data_npages;

arch/x86/kvm/svm/sev.c static unsigned long snp_handle_page_state_change(struct vcpu_svm *svm) · line 3663

Quoted 10 lines
arch/x86/kvm/svm/sev.c +10 -0
+
+               /* pass the firmware error code */
+               rc = err;
+               goto cleanup;
+       }
+
+       /* Copy the certificate blob in the guest memory */
+       if (data_npages &&
+           kvm_write_guest(kvm, data_gpa, sev->snp_certs_data, data_npages << PAGE_SHIFT))
+               rc = SEV_RET_INVALID_ADDRESS;

Since at this point the PSP FW has correctly executed the command and incremented the VMPCK sequence number I think we need another error signal here since this will tell the guest the PSP had an error so it will not know if the VMPCK sequence number should be incremented.

Quoted 51 lines
arch/x86/kvm/svm/sev.c +25 -0
+
+cleanup:
+       snp_cleanup_guest_buf(&req, &rc);
+
+unlock:
+       mutex_unlock(&sev->guest_req_lock);
+
+e_fail:
+       svm_set_ghcb_sw_exit_info_2(vcpu, rc);
+}
+
 static int sev_handle_vmgexit_msr_protocol(struct vcpu_svm *svm)
 {
        struct vmcb_control_area *control = &svm->vmcb->control;
@@ -3753,6 +3923,20 @@ int sev_handle_vmgexit(struct kvm_vcpu *vcpu)
                svm_set_ghcb_sw_exit_info_2(vcpu, rc);
                break;
        }
+       case SVM_VMGEXIT_GUEST_REQUEST: {
+               snp_handle_guest_request(svm, control->exit_info_1, control->exit_info_2);
+
+               ret = 1;
+               break;
+       }
+       case SVM_VMGEXIT_EXT_GUEST_REQUEST: {
+               snp_handle_ext_guest_request(svm,
+                                            control->exit_info_1,
+                                            control->exit_info_2);
+
+               ret = 1;
+               break;
+       }
        case SVM_VMGEXIT_UNSUPPORTED_EVENT:
                vcpu_unimpl(vcpu,
                            "vmgexit: unsupported event - exit_info_1=%#llx, exit_info_2=%#llx\n",
arch/x86/kvm/svm/svm.h +2 -1
index 3fd95193ed8d..3be24da1a743 100644
@@ -98,6 +98,8 @@ struct kvm_sev_info {
        u64 snp_init_flags;
        void *snp_context;      /* SNP guest context page */
        spinlock_t psc_lock;
+       void *snp_certs_data;
+       struct mutex guest_req_lock;
 };
 
 struct kvm_svm {
--

2.25.1

#302 lore raw

Re: [PATCH Part2 v6 47/49] *fix for stale per-cpu pointer due to cond_resched during ghcb mapping

In reply to Ashish Kalra
Quoted 4 lines · Ashish Kalra

From: Michael Roth <[email protected]>

Can you add a commit description here? Is this a fix for existing SEV-ES support or should this be incorporated into a patch in this series which adds this issue?

Quoted 31 lines
---
 arch/x86/kvm/svm/svm.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
arch/x86/kvm/svm/svm.c +5 -2
index fced6ea423ad..f78e3b1bde0e 100644
@@ -1352,7 +1352,7 @@ static void svm_vcpu_free(struct kvm_vcpu *vcpu)
 static void svm_prepare_switch_to_guest(struct kvm_vcpu *vcpu)
 {
        struct vcpu_svm *svm = to_svm(vcpu);
-       struct svm_cpu_data *sd = per_cpu(svm_data, vcpu->cpu);
+       struct svm_cpu_data *sd;
 
        if (sev_es_guest(vcpu->kvm))
                sev_es_unmap_ghcb(svm);
@@ -1360,6 +1360,10 @@ static void svm_prepare_switch_to_guest(struct kvm_vcpu *vcpu)
        if (svm->guest_state_loaded)
                return;
 
+       /* sev_es_unmap_ghcb() can resched, so grab per-cpu pointer afterward. */
+       barrier();
+       sd = per_cpu(svm_data, vcpu->cpu);
+
        /*
         * Save additional host state that will be restored on VMEXIT (sev-es)
         * or subsequent vmload of host save area.
--

2.25.1

author maintainer
#303 lore raw

RE: [PATCH Part2 v6 47/49] *fix for stale per-cpu pointer due to cond_resched during ghcb mapping

[AMD Official Use Only - General]

Hello Peter,

Quoted 6 lines
Quoted 4 lines

From: Michael Roth <[email protected]>

Can you add a commit description here? Is this a fix for existing SEV-ES support or should this be incorporated into a patch in this series which adds this issue?

This actually fixes issues caused due to preemption happening in svm_prepare_switch_to_guest() when kvm_vcpu_map() is called to map in the GHCB before entering the guest.

This is a temporary fix and what we need to do is to prevent getting preempted after vcpu_enter_guest() has disabled preemption, have some ideas about using gfn_to_pfn_cache() infrastructure to re-use the already mapped GHCB at guest exit, so that we can avoid calling kvm_vcpu_map() to re-map the GHCB.

Thanks,
Ashish

Quoted 30 lines
---
 arch/x86/kvm/svm/svm.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
arch/x86/kvm/svm/svm.c index renamed from arch/x86/kvm/svm/svm.c +0 -0

fced6ea423ad..f78e3b1bde0e 100644

arch/x86/kvm/svm/svm.c +1 -1
@@ -1352,7 +1352,7 @@ static void svm_vcpu_free(struct kvm_vcpu *vcpu)  
 static void svm_prepare_switch_to_guest(struct kvm_vcpu *vcpu)  {
        struct vcpu_svm *svm = to_svm(vcpu);
-       struct svm_cpu_data *sd = per_cpu(svm_data, vcpu->cpu);
+       struct svm_cpu_data *sd;
 
        if (sev_es_guest(vcpu->kvm))
                sev_es_unmap_ghcb(svm); @@ -1360,6 +1360,10 @@ static 
 void svm_prepare_switch_to_guest(struct kvm_vcpu *vcpu)
        if (svm->guest_state_loaded)
                return;
arch/x86/kvm/svm/svm.c +4 -1
+       /* sev_es_unmap_ghcb() can resched, so grab per-cpu pointer afterward. */
+       barrier();
+       sd = per_cpu(svm_data, vcpu->cpu);
+
        /*
         * Save additional host state that will be restored on VMEXIT (sev-es)
         * or subsequent vmload of host save area.
--

2.25.1

author maintainer
#218 lore raw

RE: [PATCH Part2 v6 24/49] KVM: SVM: Add KVM_SEV_SNP_LAUNCH_START command

In reply to Peter Gonda

[AMD Official Use Only - General]

Quoted 5 lines
Quoted 5 lines
arch/x86/kvm/svm/sev.c +5 -0
+static int snp_decommission_context(struct kvm *kvm) {
+       struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
+       struct sev_data_snp_decommission data = {};
+       int ret;
+
Quoted 8 lines

arch/x86/kvm/svm/sev.c int sev_vm_copy_enc_context_from(struct kvm *kvm, unsigned int source_fd) · line 2199

Quoted 6 lines
arch/x86/kvm/svm/sev.c +6 -0
+       /* If context is not created then do nothing */
+       if (!sev->snp_context)
+               return 0;
+
+       data.gctx_paddr = __sme_pa(sev->snp_context);
+       ret = snp_guest_decommission(&data, NULL);

Do we have a similar race like in sev_unbind_asid() with DEACTIVATE and WBINVD/DF_FLUSH? The SNP_DECOMMISSION spec looks quite similar to DEACTIVATE.

Yes, SNP_DECOMMISION also marks the ASID as invalid and require a WBINVD/DF_FLUSH before the ASID is re-used/re-cycled, so we need to prevent against DECOMMISION and ASID re-cycling happening at the same time. Can reuse the same RWSEM (sev_deactivate_lock) here too.

Thanks,
Ashish

Quoted 13 lines
arch/x86/kvm/svm/sev.c +10 -0
+       if (WARN_ONCE(ret, "failed to release guest context"))
+               return ret;
+
+       /* free the context page now */
+       snp_free_firmware_page(sev->snp_context);
+       sev->snp_context = NULL;
+
+       return 0;
+}
+
 void sev_vm_destroy(struct kvm *kvm)
 {
        struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
author maintainer
#255 lore raw

RE: [PATCH Part2 v6 35/49] KVM: SVM: Remove the long-lived GHCB host map

In reply to Peter Gonda

[AMD Official Use Only - General]

Hello Peter,

Quoted 33 lines
Quoted 33 lines

From: Brijesh Singh <[email protected]>

On VMGEXIT, sev_handle_vmgexit() creates a host mapping for the GHCB GPA, and unmaps it just before VM-entry. This long-lived GHCB map is used by the VMGEXIT handler through accessors such as ghcb_{set_get}_xxx().

A long-lived GHCB map can cause issue when SEV-SNP is enabled. When SEV-SNP is enabled the mapped GPA needs to be protected against a page state change.

To eliminate the long-lived GHCB mapping, update the GHCB sync operations to explicitly map the GHCB before access and unmap it after access is complete. This requires that the setting of the GHCBs sw_exit_info_{1,2} fields be done during sev_es_sync_to_ghcb(), so create two new fields in the vcpu_svm struct to hold these values when required to be set outside of the GHCB mapping.

---
 arch/x86/kvm/svm/sev.c | 131 
++++++++++++++++++++++++++---------------
 arch/x86/kvm/svm/svm.c |  12 ++--
 arch/x86/kvm/svm/svm.h |  24 +++++++-
 3 files changed, 111 insertions(+), 56 deletions(-)
arch/x86/kvm/svm/sev.c index renamed from arch/x86/kvm/svm/sev.c +0 -0

01ea257e17d6..c70f3f7e06a8 100644

arch/x86/kvm/svm/sev.c +0 -0
@@ -2823,15 +2823,40 @@ void sev_free_vcpu(struct kvm_vcpu *vcpu)
        kvfree(svm->sev_es.ghcb_sa);
 }
 
Quoted 16 lines

arch/x86/kvm/svm/sev.c void sev_free_vcpu(struct kvm_vcpu *vcpu) · line 2838

Quoted 13 lines
arch/x86/kvm/svm/sev.c +13 -0
+static inline int svm_map_ghcb(struct vcpu_svm *svm, struct 
+kvm_host_map *map) {
+       struct vmcb_control_area *control = &svm->vmcb->control;
+       u64 gfn = gpa_to_gfn(control->ghcb_gpa);
+
+       if (kvm_vcpu_map(&svm->vcpu, gfn, map)) {
+               /* Unable to map GHCB from guest */
+               pr_err("error mapping GHCB GFN [%#llx] from guest\n", gfn);
+               return -EFAULT;
+       }
+
+       return 0;
+}

There is a perf cost to this suggestion but it might make accessing the GHCB safer for KVM. Have you thought about just using
kvm_read_guest() or copy_from_user() to fully copy out the GCHB into a KVM owned buffer, then copying it back before the VMRUN. That way the KVM doesn't need to guard against page_state_changes on the GHCBs, that could be a perf ?>improvement in a follow up.

Along with the performance costs you mentioned, the main concern here will be the GHCB write-back path (copying it back) before VMRUN: this will again hit the issue we have currently with kvm_write_guest() / copy_to_user(), when we use it to sync the scratch buffer back to GHCB. This can fail if guest RAM is mapped using huge-page(s) and RMP is 4K. Please refer to the patch/fix mentioned below, kvm_write_guest() potentially can fail before VMRUN in case of SNP :

commit 94ed878c2669532ebae8eb9b4503f19aa33cd7aa

Author: Ashish Kalra <[email protected]>
Date:   Mon Jun 6 22:28:01 2022 +0000

KVM: SVM: Sync the GHCB scratch buffer using already mapped ghcb

Using kvm_write_guest() to sync the GHCB scratch buffer can fail due to host mapping being 2M, but RMP being 4K. The page fault handling in do_user_addr_fault() fails to split the 2M page to handle RMP fault due to it being called here in a non-preemptible context. Instead use the already kernel mapped ghcb to sync the scratch buffer when the scratch buffer is contained within the GHCB.

Thanks,
Ashish

Quoted 1 line

Since we cannot unmap GHCBs I don't think UPM will help here so we probably want to make these patches safe against malicious guests making GHCBs private. But maybe UPM does help?

author maintainer
#290 lore raw

RE: [PATCH Part2 v6 42/49] KVM: SVM: Provide support for SNP_GUEST_REQUEST NAE event

In reply to Peter Gonda

[Public]

Hello Peter,

-----Original Message-----
From: Peter Gonda <[email protected]>
Sent: Friday, June 24, 2022 11:25 AM
To: Kalra, Ashish <[email protected]>
Cc: the arch/x86 maintainers <[email protected]>; LKML <[email protected]>; kvm list <[email protected]>; [email protected]; [email protected]; Linux Crypto Mailing List <[email protected]>; Thomas Gleixner <[email protected]>; Ingo Molnar <[email protected]>; Joerg Roedel <[email protected]>; Lendacky, Thomas <[email protected]>; H. Peter Anvin <[email protected]>; Ard Biesheuvel <[email protected]>; Paolo Bonzini <[email protected]>; Sean Christopherson <[email protected]>; Vitaly Kuznetsov <[email protected]>; Jim Mattson <[email protected]>; Andy Lutomirski <[email protected]>; Dave Hansen <[email protected]>; Sergio Lopez <[email protected]>; Peter Zijlstra <[email protected]>; Srinivas Pandruvada <[email protected]>; David Rientjes <[email protected]>; Dov Murik <[email protected]>; Tobin Feldman-Fitzthum <[email protected]>; Borislav Petkov <[email protected]>; Roth, Michael <[email protected]>; Vlastimil Babka <[email protected]>; Kirill A . Shutemov <[email protected]>; Andi Kleen <[email protected]>; Tony Luck <[email protected]>; Marc Orr <[email protected]>; Sathyanarayanan Kuppuswamy <[email protected]>; Alper Gun <[email protected]>; Dr. David Alan Gilbert <[email protected]>; [email protected]
Subject: Re: [PATCH Part2 v6 42/49] KVM: SVM: Provide support for SNP_GUEST_REQUEST NAE event

Quoted 40 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

Version 2 of GHCB specification added the support for two SNP Guest Request Message NAE events. The events allows for an SEV-SNP guest to make request to the SEV-SNP firmware through hypervisor using the SNP_GUEST_REQUEST API define in the SEV-SNP firmware specification.

The SNP_EXT_GUEST_REQUEST is similar to SNP_GUEST_REQUEST with the difference of an additional certificate blob that can be passed through the SNP_SET_CONFIG ioctl defined in the CCP driver. The CCP driver provides snp_guest_ext_guest_request() that is used by the KVM to get both the report and certificate data at once.

---
 arch/x86/kvm/svm/sev.c | 196 +++++++++++++++++++++++++++++++++++++++--
 arch/x86/kvm/svm/svm.h |   2 +
 2 files changed, 192 insertions(+), 6 deletions(-)
arch/x86/kvm/svm/sev.c index renamed from arch/x86/kvm/svm/sev.c +0 -0

7fc0fad87054..089af21a4efe 100644

arch/x86/kvm/svm/sev.c +1 -0
@@ -343,6 +343,7 @@ static int sev_guest_init(struct kvm *kvm, struct 
 kvm_sev_cmd *argp)
 
                spin_lock_init(&sev->psc_lock);
                ret = sev_snp_init(&argp->error);
+               mutex_init(&sev->guest_req_lock);
        } else {
                ret = sev_platform_init(&argp->error);
        }
arch/x86/kvm/svm/sev.c +1 -0
@@ -1884,23 +1885,39 @@ int sev_vm_move_enc_context_from(struct kvm 
 *kvm, unsigned int source_fd)
 
 static void *snp_context_create(struct kvm *kvm, struct kvm_sev_cmd 
 *argp)  {
+       void *context = NULL, *certs_data = NULL, *resp_page = NULL;
 
Quoted 2 lines

Is the NULL setting here unnecessary since all of these are set via functions snp_alloc_firmware_page(), kmalloc(), and snp_alloc_firmware_page() respectively?

Yes, they don't need to be set to NULL.

Quoted 11 lines
arch/x86/kvm/svm/sev.c +5 -1
+       struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
        struct sev_data_snp_gctx_create data = {};
-       void *context;
        int rc;
 
+       /* Allocate memory used for the certs data in SNP guest request */
+       certs_data = kmalloc(SEV_FW_BLOB_MAX_SIZE, GFP_KERNEL_ACCOUNT);
+       if (!certs_data)
+               return NULL;

I think we want to use kzalloc() here to ensure we never give the guest uninitialized kernel memory.

Yes.

Quoted 67 lines
arch/x86/kvm/svm/sev.c +7 -1
+
        /* Allocate memory for context page */
        context = snp_alloc_firmware_page(GFP_KERNEL_ACCOUNT);
        if (!context)
-               return NULL;
+               goto e_free;
+
+       /* Allocate a firmware buffer used during the guest command handling. */
+       resp_page = snp_alloc_firmware_page(GFP_KERNEL_ACCOUNT);
+       if (!resp_page)
+               goto e_free;

|resp_page| doesn't appear to be used anywhere?

        data.gctx_paddr = __psp_pa(context);
        rc = __sev_issue_cmd(argp->sev_fd, SEV_CMD_SNP_GCTX_CREATE, &data, &argp->error);
arch/x86/kvm/svm/sev.c +14 -4
-       if (rc) {
-               snp_free_firmware_page(context);
-               return NULL;
-       }
+       if (rc)
+               goto e_free;
+
+       sev->snp_certs_data = certs_data;
 
        return context;
+
+e_free:
+       snp_free_firmware_page(context);
+       kfree(certs_data);
+       return NULL;
 }
 
 static int snp_bind_asid(struct kvm *kvm, int *error) @@ -2565,6 
+2582,8 @@ static int snp_decommission_context(struct kvm *kvm)
        snp_free_firmware_page(sev->snp_context);
        sev->snp_context = NULL;
 
+       kfree(sev->snp_certs_data);
+
        return 0;
 }
@@ -3077,6 +3096,8 @@ static int sev_es_validate_vmgexit(struct vcpu_svm *svm, u64 *exit_code)
        case SVM_VMGEXIT_UNSUPPORTED_EVENT:
        case SVM_VMGEXIT_HV_FEATURES:
        case SVM_VMGEXIT_PSC:
+       case SVM_VMGEXIT_GUEST_REQUEST:
+       case SVM_VMGEXIT_EXT_GUEST_REQUEST:
                break;
        default:
                reason = GHCB_ERR_INVALID_EVENT; @@ -3502,6 +3523,155 

@@ static unsigned long snp_handle_page_state_change(struct vcpu_svm *svm)

        return rc ? map_to_psc_vmgexit_code(rc) : 0;  }
arch/x86/kvm/svm/sev.c +10 -0
+static unsigned long snp_setup_guest_buf(struct vcpu_svm *svm,
+                                        struct sev_data_snp_guest_request *data,
+                                        gpa_t req_gpa, gpa_t 
+resp_gpa) {
+       struct kvm_vcpu *vcpu = &svm->vcpu;
+       struct kvm *kvm = vcpu->kvm;
+       kvm_pfn_t req_pfn, resp_pfn;
+       struct kvm_sev_info *sev;
+
+       sev = &to_kvm_svm(kvm)->sev_info;
Quoted 3 lines

This is normally done at declaration in this file. Why not here?

     struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;

Ok.

Quoted 31 lines
arch/x86/kvm/svm/sev.c +30 -0
+
+       if (!IS_ALIGNED(req_gpa, PAGE_SIZE) || !IS_ALIGNED(resp_gpa, PAGE_SIZE))
+               return SEV_RET_INVALID_PARAM;
+
+       req_pfn = gfn_to_pfn(kvm, gpa_to_gfn(req_gpa));
+       if (is_error_noslot_pfn(req_pfn))
+               return SEV_RET_INVALID_ADDRESS;
+
+       resp_pfn = gfn_to_pfn(kvm, gpa_to_gfn(resp_gpa));
+       if (is_error_noslot_pfn(resp_pfn))
+               return SEV_RET_INVALID_ADDRESS;
+
+       if (rmp_make_private(resp_pfn, 0, PG_LEVEL_4K, 0, true))
+               return SEV_RET_INVALID_ADDRESS;
+
+       data->gctx_paddr = __psp_pa(sev->snp_context);
+       data->req_paddr = __sme_set(req_pfn << PAGE_SHIFT);
+       data->res_paddr = __sme_set(resp_pfn << PAGE_SHIFT);
+
+       return 0;
+}
+
+static void snp_cleanup_guest_buf(struct sev_data_snp_guest_request 
+*data, unsigned long *rc) {
+       u64 pfn = __sme_clr(data->res_paddr) >> PAGE_SHIFT;
+       int ret;
+
+       ret = snp_page_reclaim(pfn);
+       if (ret)
+               *rc = SEV_RET_INVALID_ADDRESS;
Quoted 1 line

Do we need a diff error code here? This means the page the guest gives us is now "stuck" in the FW owned state. How would the guest know this is the case? We return the exact same error in snp_setup_guest_buf() if the resp_gpa isn't page aligned >so now if the guest ever sees a SEV_RET_INVALID_ADDRESS I think its only safe option is to either try and page_state_change it to a know state or mark it as unusable memory.

If snp_page_reclaim() fails, it will invoke snp_leak_pages() which will indicate memory failure and trigger memory recovery mechanisms and that should drop the pages or mark them unusable.

Quoted 8 lines
arch/x86/kvm/svm/sev.c +4 -0
+
+       ret = rmp_make_shared(pfn, PG_LEVEL_4K);
+       if (ret)
+               *rc = SEV_RET_INVALID_ADDRESS;

Ditto here I think we need some way to signal to the guest what state this page is on return to guest execution.

Also these errors shadow over FW successes, this means the guest's guest-request-sequence-numbers are now out of sync meaning this VMPCK is unusable less the guest risk reusing the AES IV (which would break the confidentiality/integrity). >Should we have a way to signal to the guest the FW has successfully run your command but we could not change the page states back correctly, so the guest should increment their sequence numbers.

Yes, that is an important observation, and the sequence numbers are now out of sync.
But as this is an error path, so what's the guarantee that the next guest message request will succeed completely, isn’t it better to let the FW reject any subsequent guest messages once it has detected that the sequence numbers are out of sync ?

Quoted 18 lines
arch/x86/kvm/svm/sev.c +17 -0
+}
+
+static void snp_handle_guest_request(struct vcpu_svm *svm, gpa_t 
+req_gpa, gpa_t resp_gpa) {
+       struct sev_data_snp_guest_request data = {0};
+       struct kvm_vcpu *vcpu = &svm->vcpu;
+       struct kvm *kvm = vcpu->kvm;
+       struct kvm_sev_info *sev;
+       unsigned long rc;
+       int err;
+
+       if (!sev_snp_guest(vcpu->kvm)) {
+               rc = SEV_RET_INVALID_GUEST;
+               goto e_fail;
+       }
+
+       sev = &to_kvm_svm(kvm)->sev_info;
Quoted 1 line

Ditto why not due this above?

Ok.

Quoted 77 lines
arch/x86/kvm/svm/sev.c +76 -0
+
+       mutex_lock(&sev->guest_req_lock);
+
+       rc = snp_setup_guest_buf(svm, &data, req_gpa, resp_gpa);
+       if (rc)
+               goto unlock;
+
+       rc = sev_issue_cmd(kvm, SEV_CMD_SNP_GUEST_REQUEST, &data, &err);
+       if (rc)
+               /* use the firmware error code */
+               rc = err;
+
+       snp_cleanup_guest_buf(&data, &rc);
+
+unlock:
+       mutex_unlock(&sev->guest_req_lock);
+
+e_fail:
+       svm_set_ghcb_sw_exit_info_2(vcpu, rc); }
+
+static void snp_handle_ext_guest_request(struct vcpu_svm *svm, gpa_t 
+req_gpa, gpa_t resp_gpa) {
+       struct sev_data_snp_guest_request req = {0};
+       struct kvm_vcpu *vcpu = &svm->vcpu;
+       struct kvm *kvm = vcpu->kvm;
+       unsigned long data_npages;
+       struct kvm_sev_info *sev;
+       unsigned long rc, err;
+       u64 data_gpa;
+
+       if (!sev_snp_guest(vcpu->kvm)) {
+               rc = SEV_RET_INVALID_GUEST;
+               goto e_fail;
+       }
+
+       sev = &to_kvm_svm(kvm)->sev_info;
+
+       data_gpa = vcpu->arch.regs[VCPU_REGS_RAX];
+       data_npages = vcpu->arch.regs[VCPU_REGS_RBX];
+
+       if (!IS_ALIGNED(data_gpa, PAGE_SIZE)) {
+               rc = SEV_RET_INVALID_ADDRESS;
+               goto e_fail;
+       }
+
+       /* Verify that requested blob will fit in certificate buffer */
+       if ((data_npages << PAGE_SHIFT) > SEV_FW_BLOB_MAX_SIZE) {
+               rc = SEV_RET_INVALID_PARAM;
+               goto e_fail;
+       }
+
+       mutex_lock(&sev->guest_req_lock);
+
+       rc = snp_setup_guest_buf(svm, &req, req_gpa, resp_gpa);
+       if (rc)
+               goto unlock;
+
+       rc = snp_guest_ext_guest_request(&req, (unsigned long)sev->snp_certs_data,
+                                        &data_npages, &err);
+       if (rc) {
+               /*
+                * If buffer length is small then return the expected
+                * length in rbx.
+                */
+               if (err == SNP_GUEST_REQ_INVALID_LEN)
+                       vcpu->arch.regs[VCPU_REGS_RBX] = data_npages;
+
+               /* pass the firmware error code */
+               rc = err;
+               goto cleanup;
+       }
+
+       /* Copy the certificate blob in the guest memory */
+       if (data_npages &&
+           kvm_write_guest(kvm, data_gpa, sev->snp_certs_data, data_npages << PAGE_SHIFT))
+               rc = SEV_RET_INVALID_ADDRESS;
Quoted 1 line

Since at this point the PSP FW has correctly executed the command and incremented the VMPCK sequence number I think we need another error signal here since this will tell the guest the PSP had an error so it will not know if the VMPCK sequence >number should be incremented.

Similarly as above, as this is an error path, so what's the guarantee that the next guest message request will succeed completely, isn’t it better to let the FW reject any subsequent guest messages once it has detected that the sequence numbers are out of sync ?

Quoted 51 lines
arch/x86/kvm/svm/sev.c +25 -1
+
+cleanup:
+       snp_cleanup_guest_buf(&req, &rc);
+
+unlock:
+       mutex_unlock(&sev->guest_req_lock);
+
+e_fail:
+       svm_set_ghcb_sw_exit_info_2(vcpu, rc); }
+
 static int sev_handle_vmgexit_msr_protocol(struct vcpu_svm *svm)  {
        struct vmcb_control_area *control = &svm->vmcb->control; @@ 
-3753,6 +3923,20 @@ int sev_handle_vmgexit(struct kvm_vcpu *vcpu)
                svm_set_ghcb_sw_exit_info_2(vcpu, rc);
                break;
        }
+       case SVM_VMGEXIT_GUEST_REQUEST: {
+               snp_handle_guest_request(svm, control->exit_info_1, 
+ control->exit_info_2);
+
+               ret = 1;
+               break;
+       }
+       case SVM_VMGEXIT_EXT_GUEST_REQUEST: {
+               snp_handle_ext_guest_request(svm,
+                                            control->exit_info_1,
+                                            control->exit_info_2);
+
+               ret = 1;
+               break;
+       }
        case SVM_VMGEXIT_UNSUPPORTED_EVENT:
                vcpu_unimpl(vcpu,
                            "vmgexit: unsupported event - 

exit_info_1=%#llx, exit_info_2=%#llx\n", diff --git a/arch/x86/kvm/svm/svm.h b/arch/x86/kvm/svm/svm.h index 3fd95193ed8d..3be24da1a743 100644

arch/x86/kvm/svm/svm.h +2 -1
@@ -98,6 +98,8 @@ struct kvm_sev_info {
        u64 snp_init_flags;
        void *snp_context;      /* SNP guest context page */
        spinlock_t psc_lock;
+       void *snp_certs_data;
+       struct mutex guest_req_lock;
 };
 
 struct kvm_svm {
--

2.25.1

#61 lore raw

Re: [PATCH Part2 v6 06/49] x86/sev: Add helper functions for RMPUPDATE and PSMASH instruction

In reply to Kalra, Ashish
Quoted 24 lines · * Kalra, Ashish ([email protected]) wrote:

[AMD Official Use Only - General]

Quoted 17 lines
Quoted 6 lines
Quoted 6 lines

/*
* The RMP entry format is not architectural. The format is defined
in PPR @@ -126,6 +128,15 @@ struct snp_guest_platform_data {
u64 secrets_gpa;
};

Quoted 10 lines

arch/x86/include/asm/sev.h struct snp_guest_platform_data { · line 138

Quoted 8 lines
arch/x86/include/asm/sev.h +8 -0
+struct rmpupdate {
+	u64 gpa;
+	u8 assigned;
+	u8 pagesize;
+	u8 immutable;
+	u8 rsvd;
+	u32 asid;
+} __packed;

I see above it says the RMP entry format isn't architectural; is this 'rmpupdate' structure? If not how is this going to get handled when we have a couple >of SNP capable CPUs with different layouts?

Quoted 2 lines

Architectural implies that it is defined in the APM and shouldn't change in such a way as to not be backward compatible. I probably think the wording here should be architecture independent or more precisely platform independent.

Some more clarity on this:

Quoted 8 lines

Actually, the PPR for family 19h Model 01h, Rev B1 defines the RMP entry format as below:

2.1.4.2 RMP Entry Format
Architecturally the format of RMP entries are not specified in APM. In order to assist software, the following table specifies select portions of the RMP entry format for this specific product. Each RMP entry is 16B in size and is formatted as follows. Software should not rely on any field definitions not specified in this table and the format of an RMP entry may change in future processors.

Architectural implies that it is defined in the APM and shouldn't change in such a way as to not be backward compatible. So non-architectural in this context means that it is only defined in our PPR.

So actually this RPM entry definition is platform dependent and will need to be changed for different AMD processors and that change has to be handled correspondingly in the dump_rmpentry() code.

You'll need a way to make that fail cleanly when run on a newer CPU with different layout, and a way to build kernels that can handle more than one layout.

Dave

Quoted 3 lines

Thanks,
Ashish

Signature · 1 line
Dr. David Alan Gilbert / [email protected] / Manchester, UK
#261 lore raw

Re: [PATCH Part2 v6 36/49] KVM: SVM: Add support to handle GHCB GPA register VMGEXIT

In reply to Ashish Kalra
Quoted 22 lines · * Ashish Kalra ([email protected]) wrote:

From: Brijesh Singh <[email protected]>

SEV-SNP guests are required to perform a GHCB GPA registration. Before using a GHCB GPA for a vCPU the first time, a guest must register the vCPU GHCB GPA. If hypervisor can work with the guest requested GPA then it must respond back with the same GPA otherwise return -1.

On VMEXIT, Verify that GHCB GPA matches with the registered value. If a mismatch is detected then abort the guest.

---
 arch/x86/include/asm/sev-common.h |  8 ++++++++
 arch/x86/kvm/svm/sev.c            | 27 +++++++++++++++++++++++++++
 arch/x86/kvm/svm/svm.h            |  7 +++++++
 3 files changed, 42 insertions(+)
arch/x86/include/asm/sev-common.h +0 -0
index 539de6b93420..0a9055cdfae2 100644

arch/x86/include/asm/sev-common.h line 65

Quoted 7 lines
arch/x86/include/asm/sev-common.h +4 -0
@@ -59,6 +59,14 @@
 #define GHCB_MSR_AP_RESET_HOLD_RESULT_POS	12
 #define GHCB_MSR_AP_RESET_HOLD_RESULT_MASK	GENMASK_ULL(51, 0)
 
+/* Preferred GHCB GPA Request */
+#define GHCB_MSR_PREF_GPA_REQ		0x010
+#define GHCB_MSR_GPA_VALUE_POS		12
+#define GHCB_MSR_GPA_VALUE_MASK		GENMASK_ULL(51, 0)

Are the magic 51's in here fixed ?

Dave

Quoted 79 lines
arch/x86/include/asm/sev-common.h +3 -0
+#define GHCB_MSR_PREF_GPA_RESP		0x011
+#define GHCB_MSR_PREF_GPA_NONE		0xfffffffffffff
+
 /* GHCB GPA Register */
 #define GHCB_MSR_REG_GPA_REQ		0x012
 #define GHCB_MSR_REG_GPA_REQ_VAL(v)			\
arch/x86/kvm/svm/sev.c +27 -0
index c70f3f7e06a8..6de48130e414 100644
@@ -3331,6 +3331,27 @@ static int sev_handle_vmgexit_msr_protocol(struct vcpu_svm *svm)
 				  GHCB_MSR_INFO_MASK, GHCB_MSR_INFO_POS);
 		break;
 	}
+	case GHCB_MSR_PREF_GPA_REQ: {
+		set_ghcb_msr_bits(svm, GHCB_MSR_PREF_GPA_NONE, GHCB_MSR_GPA_VALUE_MASK,
+				  GHCB_MSR_GPA_VALUE_POS);
+		set_ghcb_msr_bits(svm, GHCB_MSR_PREF_GPA_RESP, GHCB_MSR_INFO_MASK,
+				  GHCB_MSR_INFO_POS);
+		break;
+	}
+	case GHCB_MSR_REG_GPA_REQ: {
+		u64 gfn;
+
+		gfn = get_ghcb_msr_bits(svm, GHCB_MSR_GPA_VALUE_MASK,
+					GHCB_MSR_GPA_VALUE_POS);
+
+		svm->sev_es.ghcb_registered_gpa = gfn_to_gpa(gfn);
+
+		set_ghcb_msr_bits(svm, gfn, GHCB_MSR_GPA_VALUE_MASK,
+				  GHCB_MSR_GPA_VALUE_POS);
+		set_ghcb_msr_bits(svm, GHCB_MSR_REG_GPA_RESP, GHCB_MSR_INFO_MASK,
+				  GHCB_MSR_INFO_POS);
+		break;
+	}
 	case GHCB_MSR_TERM_REQ: {
 		u64 reason_set, reason_code;
 
@@ -3381,6 +3402,12 @@ int sev_handle_vmgexit(struct kvm_vcpu *vcpu)
 		return 1;
 	}
 
+	/* SEV-SNP guest requires that the GHCB GPA must be registered */
+	if (sev_snp_guest(svm->vcpu.kvm) && !ghcb_gpa_is_registered(svm, ghcb_gpa)) {
+		vcpu_unimpl(&svm->vcpu, "vmgexit: GHCB GPA [%#llx] is not registered.\n", ghcb_gpa);
+		return -EINVAL;
+	}
+
 	ret = sev_es_validate_vmgexit(svm, &exit_code);
 	if (ret)
 		return ret;
arch/x86/kvm/svm/svm.h +7 -0
index c80352c9c0d6..54ff56cb6125 100644
@@ -206,6 +206,8 @@ struct vcpu_sev_es_state {
 	 */
 	u64 ghcb_sw_exit_info_1;
 	u64 ghcb_sw_exit_info_2;
+
+	u64 ghcb_registered_gpa;
 };
 
 struct vcpu_svm {
@@ -334,6 +336,11 @@ static inline bool sev_snp_guest(struct kvm *kvm)
 	return sev_es_guest(kvm) && sev->snp_active;
 }
 
+static inline bool ghcb_gpa_is_registered(struct vcpu_svm *svm, u64 val)
+{
+	return svm->sev_es.ghcb_registered_gpa == val;
+}
+
 static inline void vmcb_mark_all_dirty(struct vmcb *vmcb)
 {
 	vmcb->control.clean = 0;
Signature · 1 line
2.25.1
Signature · 1 line
Dr. David Alan Gilbert / [email protected] / Manchester, UK
author maintainer
#62 lore raw

RE: [PATCH Part2 v6 06/49] x86/sev: Add helper functions for RMPUPDATE and PSMASH instruction

In reply to Dr. David Alan Gilbert

[AMD Official Use Only - General]

Hello Dave,

-----Original Message-----
From: Dr. David Alan Gilbert <[email protected]>
Sent: Tuesday, June 28, 2022 5:51 AM
To: Kalra, Ashish <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Lendacky, Thomas <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Roth, Michael <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH Part2 v6 06/49] x86/sev: Add helper functions for RMPUPDATE and PSMASH instruction

Quoted 26 lines · * Kalra, Ashish ([email protected]) wrote:

[AMD Official Use Only - General]

Quoted 17 lines
Quoted 6 lines
Quoted 6 lines

/*
* The RMP entry format is not architectural. The format is
defined in PPR @@ -126,6 +128,15 @@ struct snp_guest_platform_data {
u64 secrets_gpa;
};

Quoted 10 lines

arch/x86/include/asm/sev.h struct snp_guest_platform_data { · line 138

Quoted 8 lines
arch/x86/include/asm/sev.h +8 -0
+struct rmpupdate {
+	u64 gpa;
+	u8 assigned;
+	u8 pagesize;
+	u8 immutable;
+	u8 rsvd;
+	u32 asid;
+} __packed;

I see above it says the RMP entry format isn't architectural; is this 'rmpupdate' structure? If not how is this going to get handled when we have a couple >of SNP capable CPUs with different layouts?

Quoted 2 lines

Architectural implies that it is defined in the APM and shouldn't change in such a way as to not be backward compatible. I probably think the wording here should be architecture independent or more precisely platform independent.

Some more clarity on this:

Actually, the PPR for family 19h Model 01h, Rev B1 defines the RMP entry format as below:

Quoted 8 lines

2.1.4.2 RMP Entry Format
Architecturally the format of RMP entries are not specified in APM. In order to assist software, the following table specifies select portions of the RMP entry format for this specific product. Each RMP entry is 16B in size and is formatted as follows. Software should not rely on any field definitions not specified in this table and the format of an RMP entry may change in future processors.

Architectural implies that it is defined in the APM and shouldn't change in such a way as to not be backward compatible. So non-architectural in this context means that it is only defined in our PPR.

So actually this RPM entry definition is platform dependent and will need to be changed for different AMD processors and that change has to be handled correspondingly in the dump_rmpentry() code.

You'll need a way to make that fail cleanly when run on a newer CPU with different layout, and a way to build kernels that can handle more than one layout.

Yes, I will be adding a check for CPU family/model as following :

static int __init snp_rmptable_init(void)
{
+ int family, model;

      if (!boot_cpu_has(X86_FEATURE_SEV_SNP))
               return 0;

+ family = boot_cpu_data.x86;
+ model = boot_cpu_data.x86_model;

+ /*

+        * RMP table entry format is not architectural and it can vary by processor and
+        * is defined by the per-processor PPR. Restrict SNP support on the known CPU
+        * model and family for which the RMP table entry format is currently defined for.
+        */

+ if (family != 0x19 || model > 0xaf)
+ goto nosnp;
+

This way SNP will only be enabled specifically on the platforms for which this RMP entry format is defined in those processor's PPR. This will work for Milan and Genoa as of now.

Additionally as per Sean's suggestion, I will be moving the RMP structure definition to sev.c, which will make it a private structure and not exposed to other parts of the kernel.

Also in the future we will have an architectural interface to read the RMP table entry, we will first check for it's availability and if not available fall back to the RMP table entry structure definition.

Thanks,
Ashish

#63 lore raw

Re: [PATCH Part2 v6 06/49] x86/sev: Add helper functions for RMPUPDATE and PSMASH instruction

Quoted 56 lines · * Kalra, Ashish ([email protected]) wrote:

[AMD Official Use Only - General]

Hello Dave,

-----Original Message-----
From: Dr. David Alan Gilbert <[email protected]>
Sent: Tuesday, June 28, 2022 5:51 AM
To: Kalra, Ashish <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Lendacky, Thomas <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Roth, Michael <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH Part2 v6 06/49] x86/sev: Add helper functions for RMPUPDATE and PSMASH instruction

Quoted 26 lines · * Kalra, Ashish ([email protected]) wrote:

[AMD Official Use Only - General]

Quoted 17 lines
Quoted 6 lines
Quoted 6 lines

/*
* The RMP entry format is not architectural. The format is
defined in PPR @@ -126,6 +128,15 @@ struct snp_guest_platform_data {
u64 secrets_gpa;
};

Quoted 10 lines

arch/x86/include/asm/sev.h struct snp_guest_platform_data { · line 138

Quoted 8 lines
arch/x86/include/asm/sev.h +8 -0
+struct rmpupdate {
+	u64 gpa;
+	u8 assigned;
+	u8 pagesize;
+	u8 immutable;
+	u8 rsvd;
+	u32 asid;
+} __packed;

I see above it says the RMP entry format isn't architectural; is this 'rmpupdate' structure? If not how is this going to get handled when we have a couple >of SNP capable CPUs with different layouts?

Quoted 2 lines

Architectural implies that it is defined in the APM and shouldn't change in such a way as to not be backward compatible. I probably think the wording here should be architecture independent or more precisely platform independent.

Some more clarity on this:

Actually, the PPR for family 19h Model 01h, Rev B1 defines the RMP entry format as below:

Quoted 8 lines

2.1.4.2 RMP Entry Format
Architecturally the format of RMP entries are not specified in APM. In order to assist software, the following table specifies select portions of the RMP entry format for this specific product. Each RMP entry is 16B in size and is formatted as follows. Software should not rely on any field definitions not specified in this table and the format of an RMP entry may change in future processors.

Architectural implies that it is defined in the APM and shouldn't change in such a way as to not be backward compatible. So non-architectural in this context means that it is only defined in our PPR.

So actually this RPM entry definition is platform dependent and will need to be changed for different AMD processors and that change has to be handled correspondingly in the dump_rmpentry() code.

You'll need a way to make that fail cleanly when run on a newer CPU with different layout, and a way to build kernels that can handle more than one layout.

Yes, I will be adding a check for CPU family/model as following :

static int __init snp_rmptable_init(void)
{

(quoted diff) +1 -0
+       int family, model;
 
      if (!boot_cpu_has(X86_FEATURE_SEV_SNP))
               return 0;
 

Review

Quoted 10 lines
(quoted diff) +9 -0
+       family = boot_cpu_data.x86;
+       model  = boot_cpu_data.x86_model;
 
+       /*
+        * RMP table entry format is not architectural and it can vary by processor and
+        * is defined by the per-processor PPR. Restrict SNP support on the known CPU
+        * model and family for which the RMP table entry format is currently defined for.
+        */
+       if (family != 0x19 || model > 0xaf)
+               goto nosnp;

please add a print there to say why you're not enabling SNP.

It would be great if your firmware could give you an 'rmpentry version'; and then if a new model came out that happened to have the same layout everything would just carryon working by checking that rather than the actual family/model.

Quoted 11 lines

+

This way SNP will only be enabled specifically on the platforms for which this RMP entry format is defined in those processor's PPR. This will work for Milan and Genoa as of now.

Additionally as per Sean's suggestion, I will be moving the RMP structure definition to sev.c, which will make it a private structure and not exposed to other parts of the kernel.

Also in the future we will have an architectural interface to read the RMP table entry, we will first check for it's availability and if not available fall back to the RMP table entry structure definition.

Dave

Quoted 3 lines

Thanks,
Ashish

Signature · 1 line
Dr. David Alan Gilbert / [email protected] / Manchester, UK
#64 lore raw

Re: [PATCH Part2 v6 06/49] x86/sev: Add helper functions for RMPUPDATE and PSMASH instruction

In reply to Kalra, Ashish
Quoted 11 lines · Ashish
(quoted diff) +8 -0
+       /*
+        * RMP table entry format is not architectural and it can vary by processor and
+        * is defined by the per-processor PPR. Restrict SNP support on the known CPU
+        * model and family for which the RMP table entry format is currently defined for.
+        */
+       if (family != 0x19 || model > 0xaf)
+               goto nosnp;
+

This way SNP will only be enabled specifically on the platforms for which this RMP entry format is defined in those processor's PPR. This will work for Milan and Genoa as of now.

At some point, it would be really nice if the AMD side of things could work to kick the magic number habit on these things. This:

arch/x86/include/asm/intel-family.h

has been really handy. It lets you do things like

grep INTEL_FAM6_SKYLAKE arch/x86

That's a *LOT* more precise than:

	egrep -i '0x5E|94' arch/x86
author maintainer
#224 lore raw

RE: [PATCH Part2 v6 26/49] KVM: SVM: Add KVM_SEV_SNP_LAUNCH_UPDATE command

In reply to Peter Gonda

[Public]

Quoted 8 lines
Quoted 8 lines
arch/x86/kvm/svm/sev.c +8 -0
+static inline void snp_leak_pages(u64 pfn, enum pg_level level) {
+       unsigned int npages = page_level_size(level) >> PAGE_SHIFT;
+
+       WARN(1, "psc failed pfn 0x%llx pages %d (leaking)\n", pfn, 
+ npages);
+
+       while (npages) {
+               memory_failure(pfn, 0);
Quoted 8 lines

arch/x86/kvm/svm/sev.c static void sev_decommission(unsigned int handle) · line 249

Quoted 5 lines
arch/x86/kvm/svm/sev.c +5 -0
+               dump_rmpentry(pfn);
+               npages--;
+               pfn++;
+       }
+}

Should this be deduplicated with the snp_leak_pages() in "crypto: ccp:
Handle the legacy TMR allocation when SNP is enabled" ?

Yes, probably should.

Quoted 9 lines
Quoted 9 lines
arch/x86/kvm/svm/sev.c +9 -0
+static bool is_hva_registered(struct kvm *kvm, hva_t hva, size_t len) 
+{
+       struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
+       struct list_head *head = &sev->regions_list;
+       struct enc_region *i;
+
+       lockdep_assert_held(&kvm->lock);
+
+       list_for_each_entry(i, head, list) {
Quoted 8 lines

arch/x86/kvm/svm/sev.c static int snp_launch_start(struct kvm *kvm, struct kvm_sev_cmd *argp) · line 1963

Quoted 6 lines
arch/x86/kvm/svm/sev.c +6 -0
+               u64 start = i->uaddr;
+               u64 end = start + i->size;
+
+               if (start <= hva && end >= (hva + len))
+                       return true;
+       }

Given that usersapce could load sev->regions_list with any # of any sized regions. Should we add a cond_resched() like in sev_vm_destroy()?

Actually, is_hva_registered() is also called from PSC handler with kvm->lock mutex held. Even though it is a mutex, I am not really sure if it is a good idea to do cond_resched() with the kvm->lock mutex held ?

Quoted 6 lines

arch/x86/kvm/svm/sev.c static int snp_launch_start(struct kvm *kvm, struct kvm_sev_cmd *argp) · line 2048

Quoted 3 lines
arch/x86/kvm/svm/sev.c +3 -0
+e_unpin:
+       /* Content of memory is updated, mark pages dirty */
+       for (i = 0; i < n; i++) {

Since |n| is not only a loop variable but actually carries the number of private pages over to e_unpin can we use a more descriptive name? How about something like 'nprivate_pages'?

Yes.

Thanks,
Ashish

author maintainer
#291 lore raw

RE: [PATCH Part2 v6 42/49] KVM: SVM: Provide support for SNP_GUEST_REQUEST NAE event

In reply to Kalra, Ashish

[Public]

Quoted 57 lines
Quoted 56 lines
arch/x86/kvm/svm/sev.c +56 -0
+static void snp_handle_ext_guest_request(struct vcpu_svm *svm, gpa_t 
+req_gpa, gpa_t resp_gpa) {
+       struct sev_data_snp_guest_request req = {0};
+       struct kvm_vcpu *vcpu = &svm->vcpu;
+       struct kvm *kvm = vcpu->kvm;
+       unsigned long data_npages;
+       struct kvm_sev_info *sev;
+       unsigned long rc, err;
+       u64 data_gpa;
+
+       if (!sev_snp_guest(vcpu->kvm)) {
+               rc = SEV_RET_INVALID_GUEST;
+               goto e_fail;
+       }
+
+       sev = &to_kvm_svm(kvm)->sev_info;
+
+       data_gpa = vcpu->arch.regs[VCPU_REGS_RAX];
+       data_npages = vcpu->arch.regs[VCPU_REGS_RBX];
+
+       if (!IS_ALIGNED(data_gpa, PAGE_SIZE)) {
+               rc = SEV_RET_INVALID_ADDRESS;
+               goto e_fail;
+       }
+
+       /* Verify that requested blob will fit in certificate buffer */
+       if ((data_npages << PAGE_SHIFT) > SEV_FW_BLOB_MAX_SIZE) {
+               rc = SEV_RET_INVALID_PARAM;
+               goto e_fail;
+       }
+
+       mutex_lock(&sev->guest_req_lock);
+
+       rc = snp_setup_guest_buf(svm, &req, req_gpa, resp_gpa);
+       if (rc)
+               goto unlock;
+
+       rc = snp_guest_ext_guest_request(&req, (unsigned long)sev->snp_certs_data,
+                                        &data_npages, &err);
+       if (rc) {
+               /*
+                * If buffer length is small then return the expected
+                * length in rbx.
+                */
+               if (err == SNP_GUEST_REQ_INVALID_LEN)
+                       vcpu->arch.regs[VCPU_REGS_RBX] = data_npages;
+
+               /* pass the firmware error code */
+               rc = err;
+               goto cleanup;
+       }
+
+       /* Copy the certificate blob in the guest memory */
+       if (data_npages &&
+           kvm_write_guest(kvm, data_gpa, sev->snp_certs_data, data_npages << PAGE_SHIFT))
+               rc = SEV_RET_INVALID_ADDRESS;
Quoted 4 lines
Quoted 1 line

Since at this point the PSP FW has correctly executed the command and incremented the VMPCK sequence number I think we need another error signal here since this will tell the guest the PSP had an error so it will not know if the VMPCK sequence >number should be incremented.

Similarly as above, as this is an error path, so what's the guarantee that the next guest message request will succeed completely, isn’t it better to let the FW reject any subsequent guest messages once it has detected that the sequence numbers are out of sync ?

Alternately, we probably can return SEV_RET_INVALID_PAGE_STATE/SEV_RET_INVALID_PAGE_OWNER here, but that still does not indicate to the guest
that the FW has successfully executed the command and the error occurred during cleanup/result phase and it needs to increment the VMPCK sequence number. There is nothing as such defined in SNP FW API specs to indicate such kind of failures to guest. As I mentioned earlier, this is probably indicative of a bigger system failure and it is better to let the FW reject subsequent guest messages/requests once it has detected that the sequence numbers are out of sync.

Thanks,
Ashish

maintainer
#12 lore raw

Re: [PATCH Part2 v6 02/49] iommu/amd: Introduce function to check SEV-SNP support

In reply to Ashish Kalra
Quoted 19 lines · Ashish Kalra
drivers/iommu/amd/init.c +19 -0
+bool iommu_sev_snp_supported(void)
+{
+	struct amd_iommu *iommu;
+
+	/*
+	 * The SEV-SNP support requires that IOMMU must be enabled, and is
+	 * not configured in the passthrough mode.
+	 */
+	if (no_iommu || iommu_default_passthrough()) {
+		pr_err("SEV-SNP: IOMMU is either disabled or configured in passthrough mode.\n");
+		return false;
+	}
+
+	/*
+	 * Iterate through all the IOMMUs and verify the SNPSup feature is
+	 * enabled.
+	 */
+	for_each_iommu(iommu) {
+		if (!iommu_feature(iommu, FEATURE_SNP)) {

drivers/iommu/amd/init.c int amd_iommu_pc_set_reg(struct amd_iommu *iommu, u8 bank, u8 cntr, u8 fxn, u64 · line 3393

Quoted 10 lines
drivers/iommu/amd/init.c +10 -0
+			pr_err("SNPSup is disabled (devid: %02x:%02x.%x)\n",
+			       PCI_BUS_NUM(iommu->devid), PCI_SLOT(iommu->devid),
+			       PCI_FUNC(iommu->devid));
+			return false;
+		}
+	}
+
+	return true;
+}
+EXPORT_SYMBOL_GPL(iommu_sev_snp_supported);

Why is this function exported?

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
author maintainer
#13 lore raw

RE: [PATCH Part2 v6 02/49] iommu/amd: Introduce function to check SEV-SNP support

[AMD Official Use Only - General]

Hello Boris,

Quoted 23 lines
Quoted 23 lines
drivers/iommu/amd/init.c +23 -0
+bool iommu_sev_snp_supported(void)
+{
+	struct amd_iommu *iommu;
+
+	/*
+	 * The SEV-SNP support requires that IOMMU must be enabled, and is
+	 * not configured in the passthrough mode.
+	 */
+	if (no_iommu || iommu_default_passthrough()) {
+		pr_err("SEV-SNP: IOMMU is either disabled or configured in passthrough mode.\n");
+		return false;
+	}
+
+	/*
+	 * Iterate through all the IOMMUs and verify the SNPSup feature is
+	 * enabled.
+	 */
+	for_each_iommu(iommu) {
+		if (!iommu_feature(iommu, FEATURE_SNP)) {
+			pr_err("SNPSup is disabled (devid: %02x:%02x.%x)\n",
+			       PCI_BUS_NUM(iommu->devid), PCI_SLOT(iommu->devid),
+			       PCI_FUNC(iommu->devid));
+			return false;
Quoted 8 lines

drivers/iommu/amd/init.c int amd_iommu_pc_set_reg(struct amd_iommu *iommu, u8 bank, u8 cntr, u8 fxn, u64 · line 3393

Quoted 6 lines
drivers/iommu/amd/init.c +6 -0
+		}
+	}
+
+	return true;
+}
+EXPORT_SYMBOL_GPL(iommu_sev_snp_supported);

Why is this function exported?

This function is required to ensure that IOMMU supports the SEV-SNP feature before enabling the SNP feature and calling SNP_INIT. This IOMMU support check is done in the AMD IOMMU driver with the iommu_sev_snp_supported() function so it is exported by the IOMMU driver and called by sev module later for SNP initialization in snp_rmptable_init().

Thanks,
Ashish

maintainer
#14 lore raw

Re: [PATCH Part2 v6 02/49] iommu/amd: Introduce function to check SEV-SNP support

Quoted 5 lines · Ashish

This function is required to ensure that IOMMU supports the SEV-SNP feature before enabling the SNP feature and calling SNP_INIT. This IOMMU support check is done in the AMD IOMMU driver with the iommu_sev_snp_supported() function so it is exported by the IOMMU driver and called by sev module

What sev module?

The call to iommu_sev_snp_supported() is done by snp_rmptable_init() which is in arch/x86/kernel/sev.c. AFAICT.

And that is not a module. But function exports are done for modules.

So that export looks superfluous.

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
author maintainer
#15 lore raw

RE: [PATCH Part2 v6 02/49] iommu/amd: Introduce function to check SEV-SNP support

[AMD Official Use Only - General]

Hello Boris,

Quoted 7 lines
Quoted 5 lines

This function is required to ensure that IOMMU supports the SEV-SNP feature before enabling the SNP feature and calling SNP_INIT. This IOMMU support check is done in the AMD IOMMU driver with the iommu_sev_snp_supported() function so it is exported by the IOMMU driver and called by sev module

What sev module?

I meant the kvm-amd module.

Quoted 5 lines

The call to iommu_sev_snp_supported() is done by snp_rmptable_init() which is in arch/x86/kernel/sev.c. AFAICT.

And that is not a module. But function exports are done for modules.

So that export looks superfluous.

Yes realized this is called in arch/x86/kernel/sev.c, so yes the export is not needed and will remove it.

Thanks,
Ashish

#256 lore raw

Re: [PATCH Part2 v6 35/49] KVM: SVM: Remove the long-lived GHCB host map

In reply to Kalra, Ashish
Quoted 63 lines · Ashish

[AMD Official Use Only - General]

Hello Peter,

Quoted 33 lines
Quoted 33 lines

From: Brijesh Singh <[email protected]>

On VMGEXIT, sev_handle_vmgexit() creates a host mapping for the GHCB GPA, and unmaps it just before VM-entry. This long-lived GHCB map is used by the VMGEXIT handler through accessors such as ghcb_{set_get}_xxx().

A long-lived GHCB map can cause issue when SEV-SNP is enabled. When SEV-SNP is enabled the mapped GPA needs to be protected against a page state change.

To eliminate the long-lived GHCB mapping, update the GHCB sync operations to explicitly map the GHCB before access and unmap it after access is complete. This requires that the setting of the GHCBs sw_exit_info_{1,2} fields be done during sev_es_sync_to_ghcb(), so create two new fields in the vcpu_svm struct to hold these values when required to be set outside of the GHCB mapping.

---
 arch/x86/kvm/svm/sev.c | 131
++++++++++++++++++++++++++---------------
 arch/x86/kvm/svm/svm.c |  12 ++--
 arch/x86/kvm/svm/svm.h |  24 +++++++-
 3 files changed, 111 insertions(+), 56 deletions(-)
arch/x86/kvm/svm/sev.c index renamed from arch/x86/kvm/svm/sev.c +0 -0

01ea257e17d6..c70f3f7e06a8 100644

arch/x86/kvm/svm/sev.c +0 -0
@@ -2823,15 +2823,40 @@ void sev_free_vcpu(struct kvm_vcpu *vcpu)
        kvfree(svm->sev_es.ghcb_sa);
 }
 
Quoted 16 lines

arch/x86/kvm/svm/sev.c void sev_free_vcpu(struct kvm_vcpu *vcpu) · line 2838

Quoted 13 lines
arch/x86/kvm/svm/sev.c +13 -0
+static inline int svm_map_ghcb(struct vcpu_svm *svm, struct
+kvm_host_map *map) {
+       struct vmcb_control_area *control = &svm->vmcb->control;
+       u64 gfn = gpa_to_gfn(control->ghcb_gpa);
+
+       if (kvm_vcpu_map(&svm->vcpu, gfn, map)) {
+               /* Unable to map GHCB from guest */
+               pr_err("error mapping GHCB GFN [%#llx] from guest\n", gfn);
+               return -EFAULT;
+       }
+
+       return 0;
+}

There is a perf cost to this suggestion but it might make accessing the GHCB safer for KVM. Have you thought about just using
kvm_read_guest() or copy_from_user() to fully copy out the GCHB into a KVM owned buffer, then copying it back before the VMRUN. That way the KVM doesn't need to guard against page_state_changes on the GHCBs, that could be a perf ?>improvement in a follow up.

Along with the performance costs you mentioned, the main concern here will be the GHCB write-back path (copying it back) before VMRUN: this will again hit the issue we have currently with kvm_write_guest() / copy_to_user(), when we use it to sync the scratch buffer back to GHCB. This can fail if guest RAM is mapped using huge-page(s) and RMP is 4K. Please refer to the patch/fix mentioned below, kvm_write_guest() potentially can fail before VMRUN in case of SNP :

commit 94ed878c2669532ebae8eb9b4503f19aa33cd7aa

Author: Ashish Kalra <[email protected]>
Date:   Mon Jun 6 22:28:01 2022 +0000
Quoted 8 lines

KVM: SVM: Sync the GHCB scratch buffer using already mapped ghcb

Using kvm_write_guest() to sync the GHCB scratch buffer can fail due to host mapping being 2M, but RMP being 4K. The page fault handling in do_user_addr_fault() fails to split the 2M page to handle RMP fault due to it being called here in a non-preemptible context. Instead use the already kernel mapped ghcb to sync the scratch buffer when the scratch buffer is contained within the GHCB.

Ah I didn't see that issue thanks for the pointer.

The patch description says "When SEV-SNP is enabled the mapped GPA needs to be protected against a page state change." since if the guest were to convert the GHCB page to private when the host is using the GHCB the host could get an RMP violation right? That RMP violation would cause the host to crash unless we use some copy_to_user() type protections. I don't see anything mechanism for this patch to add the page state change protection discussed. Can't another vCPU still convert the GHCB to private?

I was wrong about the importance of this though seanjc@ walked me through how UPM will solve this issue so no worries about this until the series is rebased on to UPM.

Quoted 5 lines

Thanks,
Ashish

Quoted 1 line

Since we cannot unmap GHCBs I don't think UPM will help here so we probably want to make these patches safe against malicious guests making GHCBs private. But maybe UPM does help?

author maintainer
#257 lore raw

RE: [PATCH Part2 v6 35/49] KVM: SVM: Remove the long-lived GHCB host map

[AMD Official Use Only - General]

Hello Peter,

Quoted 23 lines
Quoted 14 lines
Quoted 3 lines

There is a perf cost to this suggestion but it might make accessing
the GHCB safer for KVM. Have you thought about just using
kvm_read_guest() or copy_from_user() to fully copy out the GCHB into a KVM owned buffer, then copying it back before the VMRUN. That way the KVM doesn't need to guard against page_state_changes on the GHCBs, that could be a perf ?>>improvement in a follow up.

Along with the performance costs you mentioned, the main concern here
will be the GHCB write-back path (copying it back) before VMRUN: this
will again hit the issue we have currently with
kvm_write_guest() / copy_to_user(), when we use it to sync the scratch
buffer back to GHCB. This can fail if guest RAM is mapped using huge-page(s) and RMP is 4K. Please refer to the patch/fix mentioned below, kvm_write_guest() potentially can fail before VMRUN in case of SNP :

commit 94ed878c2669532ebae8eb9b4503f19aa33cd7aa

Author: Ashish Kalra <[email protected]>
Date:   Mon Jun 6 22:28:01 2022 +0000
Quoted 8 lines

KVM: SVM: Sync the GHCB scratch buffer using already mapped ghcb

Using kvm_write_guest() to sync the GHCB scratch buffer can fail
due to host mapping being 2M, but RMP being 4K. The page fault handling in do_user_addr_fault() fails to split the 2M page to handle RMP fault due to it being called here in a non-preemptible context. Instead use the already kernel mapped ghcb to sync the scratch buffer when the scratch buffer is contained within the GHCB.

Quoted 3 lines

Ah I didn't see that issue thanks for the pointer.

The patch description says "When SEV-SNP is enabled the mapped GPA needs to be protected against a page state change." since if the guest were to convert the GHCB page to private when the host is using the GHCB the host could get an RMP violation right?

Right.

Quoted 1 line

That RMP violation would cause the host to crash unless we use some copy_to_user() type protections.

As such copy_to_user() will only swallow the RMP violation and return failure, so the host can retry the write.

Quoted 1 line

I don't see anything mechanism for this patch to add the page state change protection discussed. Can't another vCPU still convert the GHCB to private?

We do have the protections for GHCB getting mapped to private specifically, there are new post_{map|unmap}_gfn functions added to verify if it is safe to map GHCB pages. There is a PSC spinlock added which protects again page state change for these mapped pages.
Below is the reference to this patch:
https://lore.kernel.org/lkml/[email protected]/T/#mafcaac7296eb9a92c0ea58730dbd3ca47a8e0756

But do note that there is protection only for GHCB pages and there is a need to add generic post_{map,unmap}_gfn() ops that can be used to verify that it's safe to map a given guest page in the hypervisor. This is a TODO right now and probably this is something which UPM can address more cleanly.

Quoted 1 line

I was wrong about the importance of this though seanjc@ walked me through how UPM will solve this issue so no worries about this until the series is rebased on to UPM.

Thanks,
Ashish

#292 lore raw

Re: [PATCH Part2 v6 42/49] KVM: SVM: Provide support for SNP_GUEST_REQUEST NAE event

In reply to Kalra, Ashish
Quoted 61 lines · Ashish

[Public]

Quoted 56 lines
Quoted 56 lines
arch/x86/kvm/svm/sev.c +56 -0
+static void snp_handle_ext_guest_request(struct vcpu_svm *svm, gpa_t
+req_gpa, gpa_t resp_gpa) {
+       struct sev_data_snp_guest_request req = {0};
+       struct kvm_vcpu *vcpu = &svm->vcpu;
+       struct kvm *kvm = vcpu->kvm;
+       unsigned long data_npages;
+       struct kvm_sev_info *sev;
+       unsigned long rc, err;
+       u64 data_gpa;
+
+       if (!sev_snp_guest(vcpu->kvm)) {
+               rc = SEV_RET_INVALID_GUEST;
+               goto e_fail;
+       }
+
+       sev = &to_kvm_svm(kvm)->sev_info;
+
+       data_gpa = vcpu->arch.regs[VCPU_REGS_RAX];
+       data_npages = vcpu->arch.regs[VCPU_REGS_RBX];
+
+       if (!IS_ALIGNED(data_gpa, PAGE_SIZE)) {
+               rc = SEV_RET_INVALID_ADDRESS;
+               goto e_fail;
+       }
+
+       /* Verify that requested blob will fit in certificate buffer */
+       if ((data_npages << PAGE_SHIFT) > SEV_FW_BLOB_MAX_SIZE) {
+               rc = SEV_RET_INVALID_PARAM;
+               goto e_fail;
+       }
+
+       mutex_lock(&sev->guest_req_lock);
+
+       rc = snp_setup_guest_buf(svm, &req, req_gpa, resp_gpa);
+       if (rc)
+               goto unlock;
+
+       rc = snp_guest_ext_guest_request(&req, (unsigned long)sev->snp_certs_data,
+                                        &data_npages, &err);
+       if (rc) {
+               /*
+                * If buffer length is small then return the expected
+                * length in rbx.
+                */
+               if (err == SNP_GUEST_REQ_INVALID_LEN)
+                       vcpu->arch.regs[VCPU_REGS_RBX] = data_npages;
+
+               /* pass the firmware error code */
+               rc = err;
+               goto cleanup;
+       }
+
+       /* Copy the certificate blob in the guest memory */
+       if (data_npages &&
+           kvm_write_guest(kvm, data_gpa, sev->snp_certs_data, data_npages << PAGE_SHIFT))
+               rc = SEV_RET_INVALID_ADDRESS;
Quoted 8 lines
Quoted 4 lines
Quoted 1 line

Since at this point the PSP FW has correctly executed the command and incremented the VMPCK sequence number I think we need another error signal here since this will tell the guest the PSP had an error so it will not know if the VMPCK sequence >number should be incremented.

Similarly as above, as this is an error path, so what's the guarantee that the next guest message request will succeed completely, isn’t it better to let the FW reject any subsequent guest messages once it has detected that the sequence numbers are out of sync ?

Alternately, we probably can return SEV_RET_INVALID_PAGE_STATE/SEV_RET_INVALID_PAGE_OWNER here, but that still does not indicate to the guest
that the FW has successfully executed the command and the error occurred during cleanup/result phase and it needs to increment the VMPCK sequence number. There is nothing as such defined in SNP FW API specs to indicate such kind of failures to guest. As I mentioned earlier, this is probably indicative of a bigger system failure and it is better to let the FW reject subsequent guest messages/requests once it has detected that the sequence numbers are out of sync.

Hmm I think the guest must be careful here because the guest could not trust the hypervisor here to be truthful about the sequence numbers incrementing. That's unfortunate since this means if these operations do fail with a well behaved hypervisor the guest cannot use that VMPCK again. But there is no harm in the guest re-issuing the SNP_GUEST_REQUEST (or extended version) with the exact same request just in at a different address. The GHCB spec actually calls this out " It is recommended that the hypervisor validate the guest physical address of the response page before invoking the SNP_GUEST_REQUEST API so that the sequence numbers do not get out of sync for the guest, possibly resulting in all successive requests failing".

Currently SVM_VMGEXIT_GUEST_REQUEST and SVM_VMGEXIT_EXT_GUEST_REQUEST have different hypervisor -> guest usage for SW_EXITINFO2. I think they both should be defined as what SVM_VMGEXIT_EXT_GUEST_REQUEST is
now: the high 32bits are the hypervisor error code, the low 32bits are the FW error code. This would allow for both NAEs to have some signal to the guest say SEV_RET_INVALID_REQ_ADDRESS. The hypervisor can use this error code when doing the validation on the request and response regions, if some is wrong with them the guest can retry with the exact same request (so no IV reuse) in a corrected region.

But another reason I think SVM_VMGEXIT_GUEST_REQUEST SW_EXITINFO2 hypervisor->guest state should include this change is because in this patch we are currently overloading the lower 32bits with hypervisor error codes. In snp_handle_guest_request() if sev_snp_guest(), snp_setup_guest_buf(), or snp_cleanup_guest_buf() fails we use the low 32bits of SW_EXITINFO2 to return hypervisor errors to the guest.

Quoted 3 lines

Thanks,
Ashish

#258 lore raw

Re: [PATCH Part2 v6 35/49] KVM: SVM: Remove the long-lived GHCB host map

In reply to Kalra, Ashish
Quoted 40 lines · Ashish

[AMD Official Use Only - General]

Hello Peter,

Quoted 23 lines
Quoted 14 lines
Quoted 3 lines

There is a perf cost to this suggestion but it might make accessing
the GHCB safer for KVM. Have you thought about just using
kvm_read_guest() or copy_from_user() to fully copy out the GCHB into a KVM owned buffer, then copying it back before the VMRUN. That way the KVM doesn't need to guard against page_state_changes on the GHCBs, that could be a perf ?>>improvement in a follow up.

Along with the performance costs you mentioned, the main concern here
will be the GHCB write-back path (copying it back) before VMRUN: this
will again hit the issue we have currently with
kvm_write_guest() / copy_to_user(), when we use it to sync the scratch
buffer back to GHCB. This can fail if guest RAM is mapped using huge-page(s) and RMP is 4K. Please refer to the patch/fix mentioned below, kvm_write_guest() potentially can fail before VMRUN in case of SNP :

commit 94ed878c2669532ebae8eb9b4503f19aa33cd7aa

Author: Ashish Kalra <[email protected]>
Date:   Mon Jun 6 22:28:01 2022 +0000
Quoted 8 lines

KVM: SVM: Sync the GHCB scratch buffer using already mapped ghcb

Using kvm_write_guest() to sync the GHCB scratch buffer can fail
due to host mapping being 2M, but RMP being 4K. The page fault handling in do_user_addr_fault() fails to split the 2M page to handle RMP fault due to it being called here in a non-preemptible context. Instead use the already kernel mapped ghcb to sync the scratch buffer when the scratch buffer is contained within the GHCB.

Quoted 3 lines

Ah I didn't see that issue thanks for the pointer.

The patch description says "When SEV-SNP is enabled the mapped GPA needs to be protected against a page state change." since if the guest were to convert the GHCB page to private when the host is using the GHCB the host could get an RMP violation right?

Right.

Quoted 1 line

That RMP violation would cause the host to crash unless we use some copy_to_user() type protections.

As such copy_to_user() will only swallow the RMP violation and return failure, so the host can retry the write.

Quoted 1 line

I don't see anything mechanism for this patch to add the page state change protection discussed. Can't another vCPU still convert the GHCB to private?

Quoted 7 lines

We do have the protections for GHCB getting mapped to private specifically, there are new post_{map|unmap}_gfn functions added to verify if it is safe to map GHCB pages. There is a PSC spinlock added which protects again page state change for these mapped pages.
Below is the reference to this patch:
https://lore.kernel.org/lkml/[email protected]/T/#mafcaac7296eb9a92c0ea58730dbd3ca47a8e0756

But do note that there is protection only for GHCB pages and there is a need to add generic post_{map,unmap}_gfn() ops that can be used to verify that it's safe to map a given guest page in the hypervisor. This is a TODO right now and probably this is something which UPM can address more cleanly.

Thank you Ashish. I had missed that.

Can you help me understand why its OK to use kvm_write_guest() for the |snp_certs_data| inside of snp_handle_ext_guest_request() in patch 42/49? I would have thought we'd have the same 2M vs 4K mapping issues.

Quoted 5 lines
Quoted 2 lines

I was wrong about the importance of this though seanjc@ walked me through how UPM will solve this issue so no worries about this until the series is rebased on to UPM.

Thanks,
Ashish

author maintainer
#259 lore raw

RE: [PATCH Part2 v6 35/49] KVM: SVM: Remove the long-lived GHCB host map

[AMD Official Use Only - General]

Hello Peter,

Quoted 14 lines
Quoted 14 lines
Quoted 1 line

I don't see anything mechanism for this patch to add the page state change protection discussed. Can't another vCPU still convert the GHCB to private?

We do have the protections for GHCB getting mapped to private
specifically, there are new post_{map|unmap}_gfn functions added to verify if it is safe to map GHCB pages. There is a PSC spinlock added which protects again page state change for these mapped pages. Below is the reference to this patch:
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore
.kernel.org%2Flkml%2Fcover.1655761627.git.ashish.kalra%40amd.com%2FT%2
F%23mafcaac7296eb9a92c0ea58730dbd3ca47a8e0756&amp;data=05%7C01%7CAshis
h.Kalra%40amd.com%7C647218cdb2a040bf354e08da60fa2968%7C3dd8961fe4884e6
08e11a82d994e183d%7C0%7C0%7C637928924845082803%7CUnknown%7CTWFpbGZsb3d
8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C
3000%7C%7C%7C&amp;sdata=ss8%2F5qualccXQero9phARIG2wvYhtp8SMdve3GglZeU%
3D&amp;reserved=0

Quoted 8 lines
Quoted 2 lines

But do note that there is protection only for GHCB pages and there is
a need to add generic post_{map,unmap}_gfn() ops that can be used to verify that it's safe to map a given guest page in the hypervisor. This is a TODO right now and probably this is something which UPM can address more cleanly.

Thank you Ashish. I had missed that.

Can you help me understand why its OK to use kvm_write_guest() for the |snp_certs_data| inside of snp_handle_ext_guest_request() in patch 42/49? I would have thought we'd have the same 2M vs 4K mapping issues.

Preemption is not disabled there, hence the RMP page fault handler can do the split of 2M to 4K on host pages without any issues.

Thanks,
Ashish