Skip to contentSkip to conversation

KVM: SVM: Support direct setting of VMSA for SEV-SNP guests

PATCH Superseded Jörg Rödel 3 participants 17 messages Sep 8 – Sep 18

Subsystems (5)
Documentation x86 KVM Kselftest +1 more subsystems

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

v2 cover KVM: SVM: Support direct setting of VMSA for SEV-SNP guests

From: Joerg Roedel <[email protected]>

Hi,

here is the updated version of KVM-AMD changes to allow user-space to directly set the boot VMSA (direct-vmsa) for an SEV-SNP virtual machine.

The direct-vmsa support is required for stable launch measurements when loading the initial SEV-SNP guest image from an IGVM file. Without this support KVM measures a VMSA at a fixed GPA for every VCPU, which makes the launch measurement dependent on the number of VCPUs. Besides that the current KVM VMSA-allocation behavior poses challenges for live migration of SEV-SNP VMs. The current behavior is fully preserved when direct-vmsa is not used.

For a longer justification have a look at this email from the v1 thread:

https://lore.kernel.org/kvm/[email protected]/

One of the main concerns with the previous version was the user-space API, which has changed significantly in v2. The API now consists of three parts:

- A new KVM_CAP_SNP_DIRECT_VMSA VM-level capability which can be queried and enabled from user-space. When user-space enabled the capability, KVM will no longer allocate VMSA pages when creating VCPUs. This means that the capability can only be enabled before any VCPUs are created.

- Allow VMSA pages in SNP launch updates. This was already present in the previous version, but has different semantics now. In this version all it does is adding the VMSA pages to the initial image and the launch measurement.

- A new per-VCPU MEM_ENC operation to get and set the VMSA and GHCB addresses of a VCPU. User-space will use this operation to set the GPA of the launch VMSA of the VCPUs, but can also query and set the addresses during runtime.

This operation is designed to support the requirements of future live-migration support, which needs to be able to set and get VMSA as well as GHCB addresses. Since this functionality is a superset of the operation needed for direct-vmsa support, it goes the extra mile and allows import and export of all SEV-SNP specific VCPU state.

For v2 I heavily used AI (codex with gpt-5.6-sol medium) and countless iterations with manual review and testing. All changes are reviewed by me, tested with KVM selftests, and booting an SVSM (as far as it gets without planes support). Lockdep was enabled during testing as well.

Please review.

-Joerg

Joerg Roedel (8):
KVM: SEV: Document SNP direct VMSA user-space ABI
KVM: SVM: Implement GET_AP_APIC_IDS NAE event
KVM: SVM: Hold SRCU while reloading guest-owned VMSAs
KVM: SEV: Add direct VMSA capability
KVM: SEV: Allow VMSA pages in SNP launch updates
KVM: SEV: Add SNP vCPU state get and set commands
KVM: selftests: Test the SNP APIC-ID-list GHCB request
KVM: selftests: Test SNP vCPU state and direct VMSA launch

 .../virt/kvm/x86/amd-memory-encryption.rst    |  66 ++-
 arch/x86/include/asm/kvm-x86-ops.h            |   1 +
 arch/x86/include/asm/kvm_host.h               |   1 +
 arch/x86/include/asm/sev-common.h             |   1 +
 arch/x86/include/uapi/asm/kvm.h               |  13 +
 arch/x86/include/uapi/asm/svm.h               |   1 +
 arch/x86/kvm/svm/sev.c                        | 287 +++++++++++-
 arch/x86/kvm/svm/svm.c                        |  23 +
 arch/x86/kvm/svm/svm.h                        |   2 +
 arch/x86/kvm/x86.c                            |   6 +
 include/uapi/linux/kvm.h                      |   2 +
 tools/testing/selftests/kvm/Makefile.kvm      |   2 +
 tools/testing/selftests/kvm/include/x86/sev.h |  49 ++
 .../selftests/kvm/x86/sev_snp_apic_id_test.c  | 180 ++++++++
 .../kvm/x86/sev_snp_direct_vmsa_test.c        | 421 ++++++++++++++++++
 15 files changed, 1039 insertions(+), 16 deletions(-)
 create mode 100644 tools/testing/selftests/kvm/x86/sev_snp_apic_id_test.c
 create mode 100644 tools/testing/selftests/kvm/x86/sev_snp_direct_vmsa_test.c

base-commit: df2908090cda368b01ff43709f51890076c56157

Signature · 1 line
2.53.0
#1 lore raw

PATCH v2 1/8 KVM: SEV: Document SNP direct VMSA userspace ABI

From: Joerg Roedel <[email protected]>

Document the VMSA launch-update page type, the vCPU-scoped state get and set commands, and the VM-scoped direct-VMSA capability in one place.

Describe the ordering and argument requirements for enabling direct-VMSA mode, and contrast its VMSA allocation and measurement behavior with the legacy launch flow.

Assisted-by LLM
---
 .../virt/kvm/x86/amd-memory-encryption.rst    | 66 ++++++++++++++++++-
 1 file changed, 64 insertions(+), 2 deletions(-)
Documentation/virt/kvm/x86/amd-memory-encryption.rst +64 -2
index bd04a908a8db..427b673c3832 100644
@@ -536,6 +536,7 @@ Returns: 0 on success, < 0 on error, -EAGAIN if caller should retry
 where the allowed values for page_type are #define'd as::
 
         KVM_SEV_SNP_PAGE_TYPE_NORMAL
+        KVM_SEV_SNP_PAGE_TYPE_VMSA
         KVM_SEV_SNP_PAGE_TYPE_ZERO
         KVM_SEV_SNP_PAGE_TYPE_UNMEASURED
         KVM_SEV_SNP_PAGE_TYPE_SECRETS
@@ -544,7 +545,68 @@ where the allowed values for page_type are #define'd as::
 See the SEV-SNP spec [snp-fw-abi]_ for further details on how each page type is
 used/measured.
 
-20. KVM_SEV_SNP_LAUNCH_FINISH
+``KVM_SEV_SNP_PAGE_TYPE_VMSA`` creates VMSA pages as part of the measured
+initial image.  A request must contain exactly one 4 KiB VMSA page, but the
+command may be used multiple times.  Creating a VMSA page does not associate it
+with a vCPU; use ``KVM_SEV_SNP_SET_VCPU_STATE`` on the intended vCPU file
+descriptor before launch finish to make that association.  KVM treats the
+VMSA contents as guest-owned data, but requires VMPL 0 and a ``sev_features``
+value that matches the VM's configured VMSA features.
+
+20. KVM_SEV_SNP_GET_VCPU_STATE / KVM_SEV_SNP_SET_VCPU_STATE
+------------------------------------------------------------
+
+These commands get or set the VMSA and GHCB addresses for the vCPU on whose
+file descriptor the command is issued.  Unlike the other SEV commands,
+userspace must issue KVM_MEMORY_ENCRYPT_OP on a vCPU file descriptor.  The
+capability is reported as ``KVM_CAP_SNP_VCPU_STATE``.
+
+Parameters (in/out): struct kvm_sev_snp_vcpu_state
+
+Returns: 0 on success, -negative on error
+
+::
+
+        #define KVM_SEV_SNP_VCPU_STATE_VMSA_VALID        _BITULL(0)
+        #define KVM_SEV_SNP_VCPU_STATE_GHCB_VALID        _BITULL(1)
+
+        struct kvm_sev_snp_vcpu_state {
+                __u64 valid_fields;
+                __u64 vmsa_gpa;
+                __u64 ghcb_gpa;
+                __u64 pad[5];           /* Must be zero */
+        };
+
+``KVM_SEV_SNP_GET_VCPU_STATE`` returns the current addresses and sets the
+corresponding bit in ``valid_fields`` for each valid address.
+
+``KVM_SEV_SNP_SET_VCPU_STATE`` sets addresses whose validity bits are present
+and invalidates addresses whose bits are absent.  The command must be issued
+after launch start and before KVM_SEV_SNP_LAUNCH_FINISH, and the VM must have
+enabled ``KVM_CAP_SNP_DIRECT_VMSA``.  A valid VMSA GPA must be backed by
+guest_memfd and populated.  The GPA must be 4-KiB aligned.  A valid GHCB
+address is copied without inspecting its backing page.  Nonzero reserved
+fields or unknown validity bits are rejected.
+
+``KVM_CAP_SNP_DIRECT_VMSA`` is a VM-scoped capability that selects direct-VMSA
+mode.  Userspace enables it with ``KVM_ENABLE_CAP`` on an SNP VM before
+creating any vCPUs.  ``flags`` and all elements of ``args`` must be zero.
+Enabling the capability on a non-SNP VM or after creating a vCPU is rejected.
+
+In direct-VMSA mode, KVM does not allocate a KVM-owned VMSA when a vCPU is
+created and does not generate or measure one at launch finish.  All launch
+VMSAs are owned and supplied by userspace.  Valid VMSAs selected with
+``KVM_SEV_SNP_SET_VCPU_STATE`` are preserved, while vCPUs without a valid VMSA
+have no runnable VMSA until the guest uses SNP AP creation to supply one.  If
+the capability is not enabled, launch finish retains the legacy behavior of
+generating and measuring a KVM-owned VMSA for every vCPU.  VMSAs that were
+measured but not selected remain ordinary valid pages in the initial image.
+
+Direct VMSAs make the launch measurement independent of KVM's selected VMSA
+GPA and of the configured vCPU count.  This gives VMMs a stable launch
+measurement across hypervisors.
+
+21. KVM_SEV_SNP_LAUNCH_FINISH
 -----------------------------
 
 After completion of the SNP guest launch flow, the KVM_SEV_SNP_LAUNCH_FINISH
@@ -572,7 +634,7 @@ Returns: 0 on success, -negative on error
 See SNP_LAUNCH_FINISH in the SEV-SNP specification [snp-fw-abi]_ for further
 details on the input parameters in ``struct kvm_sev_snp_launch_finish``.
 
-21. KVM_SEV_SNP_ENABLE_REQ_CERTS
+22. KVM_SEV_SNP_ENABLE_REQ_CERTS
 --------------------------------
 
 The KVM_SEV_SNP_ENABLE_REQ_CERTS command will configure KVM to exit to
Signature · 1 line
2.53.0
#2 lore raw

PATCH v2 2/8 KVM: SVM: Implement GET_AP_APIC_IDS NAE event

In reply to Jörg Rödel

From: Joerg Roedel <[email protected]>

Advertise and handle the GHCB APIC ID list feature so that SNP guests can discover the vCPUs for which they need to create VMSAs without relying on INIT-SIPI.

Return the required page count for undersized buffers. For sufficiently large requests, validate the GPA range and use KVM's SRCU-protected guest write path to copy an exact snapshot of the online vCPU APIC IDs.

Assisted-by LLM
---
 arch/x86/include/asm/sev-common.h |  1 +
 arch/x86/include/uapi/asm/svm.h   |  1 +
 arch/x86/kvm/svm/sev.c            | 67 ++++++++++++++++++++++++++++++-
 3 files changed, 67 insertions(+), 2 deletions(-)
arch/x86/include/asm/sev-common.h +1 -0
index 01a6e4dbe423..c041f85dfd5c 100644
@@ -136,6 +136,7 @@ enum psc_op {
 
 #define GHCB_HV_FT_SNP			BIT_ULL(0)
 #define GHCB_HV_FT_SNP_AP_CREATION	BIT_ULL(1)
+#define GHCB_HV_FT_APIC_ID_LIST		BIT_ULL(4)
 #define GHCB_HV_FT_SNP_MULTI_VMPL	BIT_ULL(5)
 
 /*
arch/x86/include/uapi/asm/svm.h +1 -0
index 010a45c9f614..05e5e31e23e1 100644
@@ -117,6 +117,7 @@
 #define SVM_VMGEXIT_AP_CREATE_ON_INIT		0
 #define SVM_VMGEXIT_AP_CREATE			1
 #define SVM_VMGEXIT_AP_DESTROY			2
+#define SVM_VMGEXIT_GET_APIC_IDS		0x80000017ull
 #define SVM_VMGEXIT_SNP_RUN_VMPL		0x80000018ull
 #define SVM_VMGEXIT_SAVIC			0x8000001aull
 #define SVM_VMGEXIT_SAVIC_REGISTER_GPA		0
arch/x86/kvm/svm/sev.c +65 -2

97 lines not on this page · raw message

Signature · 1 line
2.53.0
#3 lore raw

Re: [PATCH v2 2/8] KVM: SVM: Implement GET_AP_APIC_IDS NAE event

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] The new SVM_VMGEXIT_GET_APIC_IDS feature is unconditionally enabled and advertised to the guest…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] The new SVM_VMGEXIT_GET_APIC_IDS feature is unconditionally enabled and advertised to the guest, breaking the default-off policy and live migration capability. --

commit b3798f7802c495f4483b09adbdba116cdb452f71
Author: Joerg Roedel <[email protected]>

KVM: SVM: Implement GET_AP_APIC_IDS NAE event

This commit implements the GHCB APIC ID list feature to allow SNP guests to discover vCPUs needing VMSAs without INIT-SIPI. It handles undersized buffers and copies online vCPU APIC IDs to guest memory.

arch/x86/kvm/svm/sev.c line 45

Quoted 12 lines
arch/x86/kvm/svm/sev.c +3 -1
index 5705723f1f412..c16eca9983f37 100644
@@ -40,7 +40,9 @@
 #define GHCB_VERSION_MAX	2ULL
 #define GHCB_VERSION_MIN	1ULL
 
-#define GHCB_HV_FT_SUPPORTED	(GHCB_HV_FT_SNP | GHCB_HV_FT_SNP_AP_CREATION)
+#define GHCB_HV_FT_SUPPORTED	(GHCB_HV_FT_SNP | \
+				 GHCB_HV_FT_SNP_AP_CREATION | \
+				 GHCB_HV_FT_APIC_ID_LIST)

[Severity: High]
Does unconditionally adding GHCB_HV_FT_APIC_ID_LIST bypass userspace capability negotiation?

KVM subsystem guidelines mandate that any new behavior the guest can observe must be off by default and discoverable through the architecture's standard enumeration interface.

If a VM boots on a kernel with this patch, observes the feature via the GHCB_MSR_HV_FT_REQ MSR, and is then live-migrated to an older kernel that lacks support, could it cause the VM to crash when it attempts to use the NAE event? Should this feature require explicit opt-in from host userspace (e.g., via a KVM_CAP_ check) before advertising it to the guest?

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
#4 lore raw

PATCH v2 3/8 KVM: SVM: Hold SRCU while reloading guest-owned VMSAs

In reply to Jörg Rödel

From: Joerg Roedel <[email protected]>

Take a dedicated KVM SRCU read lock around the guest-owned VMSA memslot and guest_memfd lookup. This makes the reload path self-contained and prevents it from racing a userspace memslot replacement.

Keep the lock through VMCB installation, and release it on all lookup failure paths.

Assisted-by LLM
---
 arch/x86/kvm/svm/sev.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)
arch/x86/kvm/svm/sev.c +8 -4
index c16eca9983f3..ee1bf9b4cc53 100644
@@ -4037,6 +4037,7 @@ static void __sev_snp_reload_vmsa(struct kvm_vcpu *vcpu, gpa_t gpa)
 	unsigned long mmu_seq;
 	struct page *page;
 	kvm_pfn_t pfn;
+	int idx;
 
 	lockdep_assert_held(&svm->sev_es.snp_vmsa_mutex);
 
@@ -4068,9 +4069,10 @@ static void __sev_snp_reload_vmsa(struct kvm_vcpu *vcpu, gpa_t gpa)
 	if (!VALID_PAGE(gpa))
 		return;
 
-	slot = gfn_to_memslot(vcpu->kvm, gfn);
+	idx = srcu_read_lock(&kvm->srcu);
+	slot = gfn_to_memslot(kvm, gfn);
 	if (!slot)
-		return;
+		goto out_unlock;
 
 	mmu_seq = kvm->mmu_invalidate_seq;
 	smp_rmb();
@@ -4079,8 +4081,8 @@ static void __sev_snp_reload_vmsa(struct kvm_vcpu *vcpu, gpa_t gpa)
 	 * The new VMSA will be private memory guest memory, so retrieve the
 	 * PFN from the gmem backend.
 	 */
-	if (kvm_gmem_get_pfn(vcpu->kvm, slot, gfn, &pfn, &page, NULL))
-		return;
+	if (kvm_gmem_get_pfn(kvm, slot, gfn, &pfn, &page, NULL))
+		goto out_unlock;
 
 	read_lock(&kvm->mmu_lock);
 	/*
@@ -4097,6 +4099,8 @@ static void __sev_snp_reload_vmsa(struct kvm_vcpu *vcpu, gpa_t gpa)
 	read_unlock(&kvm->mmu_lock);
 
 	kvm_release_page_clean(page);
+out_unlock:
+	srcu_read_unlock(&kvm->srcu, idx);
 }
 
 /*
Signature · 1 line
2.53.0
#5 lore raw

PATCH v2 4/8 KVM: SEV: Add direct VMSA capability

In reply to Jörg Rödel

From: Joerg Roedel <[email protected]>

Direct VMSA mode is currently selected implicitly when userspace installs a guest-owned VMSA on any vCPU. That makes the launch model a side effect of per-vCPU state and applies it retroactively to every vCPU.

Add KVM_CAP_SNP_DIRECT_VMSA as a VM-scoped settable capability. Require userspace to enable it on an SNP VM before creating vCPUs, and reject nonzero arguments.

When enabled, skip allocating KVM-owned VMSAs during vCPU creation and skip generating and measuring KVM-owned VMSAs at launch finish. Keep legacy allocation and measurement behavior when it is not enabled.

Assisted-by LLM
---
 arch/x86/include/asm/kvm-x86-ops.h |  1 +
 arch/x86/include/asm/kvm_host.h    |  1 +
 arch/x86/kvm/svm/sev.c             | 28 ++++++++++++++++++++--------
 arch/x86/kvm/svm/svm.c             | 22 ++++++++++++++++++++++
 arch/x86/kvm/svm/svm.h             |  1 +
 arch/x86/kvm/x86.c                 |  5 +++++
 include/uapi/linux/kvm.h           |  1 +
 7 files changed, 51 insertions(+), 8 deletions(-)
arch/x86/include/asm/kvm-x86-ops.h +1 -0
index e213c9ae3e30..a48259696ca6 100644
@@ -23,6 +23,7 @@ KVM_X86_OP(vcpu_after_set_cpuid)
 KVM_X86_OP(vm_init)
 KVM_X86_OP_OPTIONAL(vm_destroy)
 KVM_X86_OP_OPTIONAL(vm_pre_destroy)
+KVM_X86_OP_OPTIONAL(enable_vm_cap)
 KVM_X86_OP_OPTIONAL_RET0(vcpu_precreate)
 KVM_X86_OP(vcpu_create)
 KVM_X86_OP(vcpu_free)
arch/x86/include/asm/kvm_host.h +1 -0
index 683bb8bf43a9..3bfc02bb7998 100644
@@ -1519,6 +1519,7 @@ struct kvm_x86_ops {
 	int (*vm_init)(struct kvm *kvm);
 	void (*vm_destroy)(struct kvm *kvm);
 	void (*vm_pre_destroy)(struct kvm *kvm);
+	int (*enable_vm_cap)(struct kvm *kvm, struct kvm_enable_cap *cap);
 
 	/* Create, but do not attach this VCPU */
 	int (*vcpu_precreate)(struct kvm *kvm);
arch/x86/kvm/svm/sev.c +20 -8
index ee1bf9b4cc53..5a282aff04a8 100644
@@ -2524,8 +2524,15 @@ static int snp_launch_update_vmsa(struct kvm *kvm, struct kvm_sev_cmd *argp)
 
 	kvm_for_each_vcpu(i, vcpu, kvm) {
 		struct vcpu_svm *svm = to_svm(vcpu);
-		u64 pfn = __pa(svm->sev_es.vmsa) >> PAGE_SHIFT;
+		u64 pfn;
+
+		if (sev->snp_direct_vmsa) {
+			if (!svm->sev_es.snp_has_guest_vmsa)
+				svm->vmcb->control.vmsa_pa = INVALID_PAGE;
+			goto protect_vcpu;
+		}
 
+		pfn = __pa(svm->sev_es.vmsa) >> PAGE_SHIFT;
 		ret = sev_es_sync_vmsa(svm);
 		if (ret)
 			goto out;
@@ -2545,6 +2552,7 @@ static int snp_launch_update_vmsa(struct kvm *kvm, struct kvm_sev_cmd *argp)
 			goto out;
 		}
 
+protect_vcpu:
 		svm->vcpu.arch.guest_state_protected = true;
 		/*
 		 * SEV-ES (and thus SNP) guest mandates LBR Virtualization to
@@ -3559,6 +3567,9 @@ void sev_free_vcpu(struct kvm_vcpu *vcpu)
 	 * a guest-owned page. Transition the page to hypervisor state before
 	 * releasing it back to the system.
 	 */
+	if (!svm->sev_es.vmsa)
+		goto skip_vmsa_free;
+
 	if (is_sev_snp_guest(vcpu)) {
 		u64 pfn = __pa(svm->sev_es.vmsa) >> PAGE_SHIFT;
 
@@ -4056,13 +4067,9 @@ static void __sev_snp_reload_vmsa(struct kvm_vcpu *vcpu, gpa_t gpa)
 	vmcb_mark_all_dirty(svm->vmcb);
 
 	/*
-	 * From this point forward, the VMSA will always be a guest-mapped page
-	 * rather than the initial one allocated by KVM in svm->sev_es.vmsa. In
-	 * theory, svm->sev_es.vmsa could be free'd and cleaned up here, but
-	 * that involves cleanups like flushing caches, which would ideally be
-	 * handled during teardown rather than guest boot.  Deferring that also
-	 * allows the existing logic for SEV-ES VMSAs to be re-used with
-	 * minimal SNP-specific changes.
+	 * From this point forward, the VMSA will always be a guest-mapped page.
+	 * If KVM allocated an initial VMSA, keep it until teardown to defer
+	 * cache flushing and other cleanup out of the guest boot path.
 	 */
 	svm->sev_es.snp_has_guest_vmsa = true;
 
@@ -4929,12 +4936,15 @@ void sev_init_vmcb(struct vcpu_svm *svm, bool init_event)
 int sev_vcpu_create(struct kvm_vcpu *vcpu)
 {
 	struct vcpu_svm *svm = to_svm(vcpu);
+	struct kvm_sev_info *sev = to_kvm_sev_info(vcpu->kvm);
 	struct page *vmsa_page;
 
 	mutex_init(&svm->sev_es.snp_vmsa_mutex);
 
 	if (!is_sev_es_guest(vcpu))
 		return 0;
+	if (is_sev_snp_guest(vcpu) && sev->snp_direct_vmsa)
+		goto init_vmsa_state;
 
 	/*
 	 * SEV-ES guests require a separate (from the VMCB) VMSA page used to
@@ -4945,6 +4955,8 @@ int sev_vcpu_create(struct kvm_vcpu *vcpu)
 		return -ENOMEM;
 
 	svm->sev_es.vmsa = page_address(vmsa_page);
+
+init_vmsa_state:
 	svm->sev_es.snp_pending_vmsa_gpa = INVALID_PAGE;
 	svm->sev_es.snp_guest_vmsa_gpa = INVALID_PAGE;
 
arch/x86/kvm/svm/svm.c +22 -0
index 7d59d301e1e5..98c5cffe2c22 100644
@@ -5325,6 +5325,27 @@ static void *svm_alloc_apic_backing_page(struct kvm_vcpu *vcpu)
 	return page_address(page);
 }
 
+static int svm_enable_vm_cap(struct kvm *kvm, struct kvm_enable_cap *cap)
+{
+	switch (cap->cap) {
+#ifdef CONFIG_KVM_AMD_SEV
+	case KVM_CAP_SNP_DIRECT_VMSA:
+		if (memchr_inv(cap->args, 0, sizeof(cap->args)) ||
+		    kvm->arch.vm_type != KVM_X86_SNP_VM)
+			return -EINVAL;
+
+		guard(mutex)(&kvm->lock);
+		if (kvm->created_vcpus)
+			return -EINVAL;
+
+		to_kvm_sev_info(kvm)->snp_direct_vmsa = true;
+		return 0;
+#endif
+	default:
+		return -EINVAL;
+	}
+}
+
 struct kvm_x86_ops svm_x86_ops __initdata = {
 	.name = KBUILD_MODNAME,
 
@@ -5345,6 +5366,7 @@ struct kvm_x86_ops svm_x86_ops __initdata = {
 	.vm_init = svm_vm_init,
 	.vm_pre_destroy = avic_vm_pre_destroy,
 	.vm_destroy = svm_vm_destroy,
+	.enable_vm_cap = svm_enable_vm_cap,
 
 	.prepare_switch_to_guest = svm_prepare_switch_to_guest,
 	.vcpu_load = svm_vcpu_load,
arch/x86/kvm/svm/svm.h +1 -0
index e958943b8162..881d662aaab2 100644
@@ -121,6 +121,7 @@ struct kvm_sev_info {
 	struct mutex guest_req_mutex; /* Must acquire before using bounce buffers */
 	cpumask_var_t have_run_cpus; /* CPUs that have done VMRUN for this VM. */
 	bool snp_certs_enabled;	/* SNP certificate-fetching support. */
+	bool snp_direct_vmsa;	/* Userspace provides and measures VMSA pages. */
 };
 #endif
 
arch/x86/kvm/x86.c +5 -0
index 79468ddfe473..fb4857eca6c6 100644
@@ -2404,6 +2404,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 	case KVM_CAP_VM_TYPES:
 		r = kvm_caps.supported_vm_types;
 		break;
+	case KVM_CAP_SNP_DIRECT_VMSA:
+		r = !!(kvm_caps.supported_vm_types & BIT(KVM_X86_SNP_VM));
+		break;
 	case KVM_CAP_READONLY_MEM:
 		r = kvm ? kvm_arch_has_readonly_mem(kvm) : 1;
 		break;
@@ -4212,6 +4215,8 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm,
 	}
 	default:
 		r = -EINVAL;
+		if (kvm_x86_ops.enable_vm_cap)
+			r = kvm_x86_call(enable_vm_cap)(kvm, cap);
 		break;
 	}
 	return r;
include/uapi/linux/kvm.h +1 -0
index ac2d77d14963..283e881b2634 100644
@@ -999,6 +999,7 @@ struct kvm_enable_cap {
 #define KVM_CAP_S390_HPAGE_2G 249
 #define KVM_CAP_PPC_COMPAT_CAPS 250
 #define KVM_CAP_ARM_PMU_V3_STRICT 251
+#define KVM_CAP_SNP_DIRECT_VMSA 252
 
 struct kvm_irq_routing_irqchip {
 	__u32 irqchip;
Signature · 1 line
2.53.0
#6 lore raw

Re: [PATCH v2 4/8] KVM: SEV: Add direct VMSA capability

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] Using `guard(mutex)` inside a `switch` case without enclosing braces causes a compile-time error…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] Using guard(mutex) inside a switch case without enclosing braces causes a compile-time error and bypasses variable initialization. --

commit 14ba20ca988537a92ef023f074770d517f738803
Author: Joerg Roedel <[email protected]>

KVM: SEV: Add direct VMSA capability

This commit introduces the KVM_CAP_SNP_DIRECT_VMSA capability, allowing userspace to manage VMSA pages directly for SNP VMs. When enabled, it skips KVM-owned VMSA allocation during vCPU creation and measurement at launch.

Quoted 4 lines
arch/x86/kvm/svm/svm.c +0 -0
index 7d59d301e1e54..98c5cffe2c227 100644

[ ... ]

arch/x86/kvm/svm/svm.c static void *svm_alloc_apic_backing_page(struct kvm_vcpu *vcpu) · line 5337

Quoted 10 lines
arch/x86/kvm/svm/svm.c +10 -0
+static int svm_enable_vm_cap(struct kvm *kvm, struct kvm_enable_cap *cap)
+{
+	switch (cap->cap) {
+#ifdef CONFIG_KVM_AMD_SEV
+	case KVM_CAP_SNP_DIRECT_VMSA:
+		if (memchr_inv(cap->args, 0, sizeof(cap->args)) ||
+		    kvm->arch.vm_type != KVM_X86_SNP_VM)
+			return -EINVAL;
+
+		guard(mutex)(&kvm->lock);

[Severity: Low]
Does this code cause a build failure? Using guard(mutex) inside a switch case without enclosing braces declares a variable in the wider scope of the switch block.

arch/x86/kvm/svm/svm.c static void *svm_alloc_apic_backing_page(struct kvm_vcpu *vcpu) · line 5345

Quoted 8 lines
arch/x86/kvm/svm/svm.c +8 -0
+		if (kvm->created_vcpus)
+			return -EINVAL;
+
+		to_kvm_sev_info(kvm)->snp_direct_vmsa = true;
+		return 0;
+#endif
+	default:
+		return -EINVAL;

If execution jumps to this default label, it bypasses the initialization of the cleanup variable declared by the guard in the case above. The cleanup function will then attempt to run on an uninitialized pointer upon return.

Could the KVM_CAP_SNP_DIRECT_VMSA case be enclosed in curly braces to limit the scope of the guard?

Quoted 2 lines
arch/x86/kvm/svm/svm.c +2 -0
+	}
+}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
#7 lore raw

PATCH v2 5/8 KVM: SEV: Allow VMSA pages in SNP launch updates

In reply to Jörg Rödel

From: Joerg Roedel <[email protected]>

Pass the firmware VMSA page type through KVM_SEV_SNP_LAUNCH_UPDATE. Only accept VMSA pages when userspace has enabled KVM_CAP_SNP_DIRECT_VMSA.
Require each request to describe exactly one 4-KiB VMSA page. Allow repeated requests and keep VMSA creation independent of association with a vCPU.

The VMSA's VMPL and SEV features define its execution context. They must agree with KVM's VM-wide configuration. Before passing a VMSA to firmware, require VMPL 0. Require sev_features to exactly match the VM's configured VMSA features. Treat the remaining contents as guest-owned data.

Assisted-by LLM
---
 arch/x86/include/uapi/asm/kvm.h |  1 +
 arch/x86/kvm/svm/sev.c          | 16 ++++++++++++++++
 2 files changed, 17 insertions(+)
arch/x86/include/uapi/asm/kvm.h +1 -0
index 1585ec804066..69dcd044583f 100644
@@ -887,6 +887,7 @@ struct kvm_sev_snp_launch_start {
 /* Kept in sync with firmware values for simplicity. */
 #define KVM_SEV_PAGE_TYPE_INVALID		0x0
 #define KVM_SEV_SNP_PAGE_TYPE_NORMAL		0x1
+#define KVM_SEV_SNP_PAGE_TYPE_VMSA		0x2
 #define KVM_SEV_SNP_PAGE_TYPE_ZERO		0x3
 #define KVM_SEV_SNP_PAGE_TYPE_UNMEASURED	0x4
 #define KVM_SEV_SNP_PAGE_TYPE_SECRETS		0x5
arch/x86/kvm/svm/sev.c +16 -0
index 5a282aff04a8..7a9ef1bc54e9 100644
@@ -2346,6 +2346,7 @@ struct sev_gmem_populate_args {
 	__u8 type;
 	int sev_fd;
 	int fw_error;
+	bool vmsa_invalid;
 };
 
 static int sev_gmem_post_populate(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn,
@@ -2369,11 +2370,20 @@ static int sev_gmem_post_populate(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn,
 	if (src_page) {
 		void *src_vaddr = kmap_local_page(src_page);
 		void *dst_vaddr = kmap_local_pfn(pfn);
+		struct sev_es_save_area *vmsa = dst_vaddr;
 
 		memcpy(dst_vaddr, src_vaddr, PAGE_SIZE);
+		if (sev_populate_args->type == KVM_SEV_SNP_PAGE_TYPE_VMSA &&
+		    (vmsa->vmpl || vmsa->sev_features != sev->vmsa_features)) {
+			sev_populate_args->vmsa_invalid = true;
+			ret = -EINVAL;
+		}
 
 		kunmap_local(dst_vaddr);
 		kunmap_local(src_vaddr);
+
+		if (ret)
+			goto out;
 	}
 
 	ret = rmp_make_private(pfn, gfn << PAGE_SHIFT, PG_LEVEL_4K,
@@ -2439,7 +2449,10 @@ static int snp_launch_update(struct kvm *kvm, struct kvm_sev_cmd *argp)
 		 params.gfn_start, params.len, params.type, params.flags);
 
 	if (!params.len || !PAGE_ALIGNED(params.len) || params.flags ||
+	    (params.type == KVM_SEV_SNP_PAGE_TYPE_VMSA &&
+	     (!sev->snp_direct_vmsa || params.len != PAGE_SIZE)) ||
 	    (params.type != KVM_SEV_SNP_PAGE_TYPE_NORMAL &&
+	     params.type != KVM_SEV_SNP_PAGE_TYPE_VMSA &&
 	     params.type != KVM_SEV_SNP_PAGE_TYPE_ZERO &&
 	     params.type != KVM_SEV_SNP_PAGE_TYPE_UNMEASURED &&
 	     params.type != KVM_SEV_SNP_PAGE_TYPE_SECRETS &&
@@ -2487,6 +2500,9 @@ static int snp_launch_update(struct kvm *kvm, struct kvm_sev_cmd *argp)
 				  params.type == KVM_SEV_SNP_PAGE_TYPE_CPUID,
 				  sev_gmem_post_populate, &sev_populate_args);
 	if (count < 0) {
+		if (sev_populate_args.vmsa_invalid)
+			return -EINVAL;
+
 		argp->error = sev_populate_args.fw_error;
 		pr_debug("%s: kvm_gmem_populate failed, ret %ld (fw_error %d)\n",
 			 __func__, count, argp->error);
Signature · 1 line
2.53.0
#8 lore raw

PATCH v2 6/8 KVM: SEV: Add SNP vCPU state get and set commands

In reply to Jörg Rödel

From: Joerg Roedel <[email protected]>

Add KVM_SEV_SNP_GET_VCPU_STATE and KVM_SEV_SNP_SET_VCPU_STATE as vCPU-scoped KVM_MEMORY_ENCRYPT_OP commands and advertise KVM_CAP_SNP_VCPU_STATE. The shared, extensible argument reports the VMSA and GHCB GPAs and has validity bits for both addresses.

For SET, validate the argument, launch ordering, VMSA alignment, and that a valid VMSA is backed by guest_memfd. Install or invalidate the VMSA and GHCB as requested. GET returns the currently tracked guest-owned VMSA GPA and the GHCB address.

Require KVM_CAP_SNP_DIRECT_VMSA for SET. The VM-scoped capability fixes the launch model before vCPU creation; the per-vCPU command only selects the guest-owned VMSA and GHCB addresses for that vCPU.

Route the commands through the locked vCPU ioctl hook. The vCPU mutex serializes the commands against launch finalization, which locks all vCPUs before inspecting their state.

Assisted-by LLM
---
 arch/x86/include/uapi/asm/kvm.h |  12 +++
 arch/x86/kvm/svm/sev.c          | 164 ++++++++++++++++++++++++++++++++
 arch/x86/kvm/svm/svm.c          |   1 +
 arch/x86/kvm/svm/svm.h          |   1 +
 arch/x86/kvm/x86.c              |   1 +
 include/uapi/linux/kvm.h        |   1 +
 6 files changed, 180 insertions(+)
arch/x86/include/uapi/asm/kvm.h +12 -0
index 69dcd044583f..5ae041e617b8 100644
@@ -748,6 +748,8 @@ enum sev_cmd_id {
 	KVM_SEV_SNP_LAUNCH_UPDATE,
 	KVM_SEV_SNP_LAUNCH_FINISH,
 	KVM_SEV_SNP_ENABLE_REQ_CERTS,
+	KVM_SEV_SNP_GET_VCPU_STATE,
+	KVM_SEV_SNP_SET_VCPU_STATE,
 
 	KVM_SEV_NR_MAX,
 };
@@ -904,6 +906,16 @@ struct kvm_sev_snp_launch_update {
 	__u64 pad2[4];
 };
 
+#define KVM_SEV_SNP_VCPU_STATE_VMSA_VALID	_BITULL(0)
+#define KVM_SEV_SNP_VCPU_STATE_GHCB_VALID	_BITULL(1)
+
+struct kvm_sev_snp_vcpu_state {
+	__u64 valid_fields;
+	__u64 vmsa_gpa;
+	__u64 ghcb_gpa;
+	__u64 pad[5];
+};
+
 #define KVM_SEV_SNP_ID_BLOCK_SIZE	96
 #define KVM_SEV_SNP_ID_AUTH_SIZE	4096
 #define KVM_SEV_SNP_FINISH_DATA_SIZE	32
arch/x86/kvm/svm/sev.c +164 -0

188 lines not on this page · raw message

arch/x86/kvm/svm/svm.c +1 -0
index 98c5cffe2c22..378d94409134 100644
@@ -5466,6 +5466,7 @@ struct kvm_x86_ops svm_x86_ops __initdata = {
 	.vcpu_needs_initialization = sev_vcpu_needs_initialization,
 	.dev_get_attr = sev_dev_get_attr,
 	.mem_enc_ioctl = sev_mem_enc_ioctl,
+	.vcpu_mem_enc_ioctl = sev_vcpu_mem_enc_ioctl,
 	.mem_enc_register_region = sev_mem_enc_register_region,
 	.mem_enc_unregister_region = sev_mem_enc_unregister_region,
 	.guest_memory_reclaimed = sev_guest_memory_reclaimed,
arch/x86/kvm/svm/svm.h +1 -0
index 881d662aaab2..c920c797b7ec 100644
@@ -983,6 +983,7 @@ void sev_es_unmap_ghcb(struct vcpu_svm *svm);
 #ifdef CONFIG_KVM_AMD_SEV
 bool sev_vcpu_needs_initialization(struct kvm_vcpu *vcpu);
 int sev_mem_enc_ioctl(struct kvm *kvm, void __user *argp);
+int sev_vcpu_mem_enc_ioctl(struct kvm_vcpu *vcpu, void __user *argp);
 int sev_mem_enc_register_region(struct kvm *kvm,
 				struct kvm_enc_region *range);
 int sev_mem_enc_unregister_region(struct kvm *kvm,
arch/x86/kvm/x86.c +1 -0
index fb4857eca6c6..232507ae504a 100644
@@ -2404,6 +2404,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 	case KVM_CAP_VM_TYPES:
 		r = kvm_caps.supported_vm_types;
 		break;
+	case KVM_CAP_SNP_VCPU_STATE:
 	case KVM_CAP_SNP_DIRECT_VMSA:
 		r = !!(kvm_caps.supported_vm_types & BIT(KVM_X86_SNP_VM));
 		break;
include/uapi/linux/kvm.h +1 -0
index 283e881b2634..8c6765f78c1c 100644
@@ -1000,6 +1000,7 @@ struct kvm_enable_cap {
 #define KVM_CAP_PPC_COMPAT_CAPS 250
 #define KVM_CAP_ARM_PMU_V3_STRICT 251
 #define KVM_CAP_SNP_DIRECT_VMSA 252
+#define KVM_CAP_SNP_VCPU_STATE 253
 
 struct kvm_irq_routing_irqchip {
 	__u32 irqchip;
Signature · 1 line
2.53.0
#9 lore raw

Re: [PATCH v2 6/8] KVM: SEV: Add SNP vCPU state get and set commands

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [High] KVM_SEV_SNP_GET_VCPU_STATE silently hides the configured VMSA GPA during transient MMU…

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] KVM_SEV_SNP_GET_VCPU_STATE silently hides the configured VMSA GPA during transient MMU invalidations, causing state loss during live migration.
- [High] KVM_SEV_SNP_SET_VCPU_STATE incorrectly rejects valid 2MB-aligned VMSA GPAs.
--

commit ccc907edf743bac6d650ebf0b64bb61800fedfea
Author: Joerg Roedel <[email protected]>

KVM: SEV: Add SNP vCPU state get and set commands

Add KVM_SEV_SNP_GET_VCPU_STATE and KVM_SEV_SNP_SET_VCPU_STATE as vCPU-scoped KVM_MEMORY_ENCRYPT_OP commands and advertise KVM_CAP_SNP_VCPU_STATE.

Quoted 4 lines
arch/x86/kvm/svm/sev.c +0 -0
index 7a9ef1bc54e9d..3e0727e251a93 100644

[ ... ]

Quoted 10 lines
arch/x86/kvm/svm/sev.c +6 -0
@@ -2585,6 +2587,89 @@ static int snp_launch_update_vmsa(struct kvm *kvm, struct kvm_sev_cmd *argp)
 	return ret;
 }
 
+static int snp_get_vcpu_state(struct kvm_vcpu *vcpu,
+			      struct kvm_sev_cmd *argp)
+{
+	struct vcpu_svm *svm = to_svm(vcpu);
+	struct kvm *kvm = vcpu->kvm;
+	struct kvm_sev_snp_vcpu_state state = {};

arch/x86/kvm/svm/sev.c static int snp_launch_update_vmsa(struct kvm *kvm, struct kvm_sev_cmd *argp) · line 2605

Quoted 10 lines
arch/x86/kvm/svm/sev.c +10 -0
+
+	if (!is_sev_snp_guest(vcpu))
+		return -ENOTTY;
+	if (!to_kvm_sev_info(kvm)->snp_context)
+		return -EINVAL;
+
+	guard(mutex)(&svm->sev_es.snp_vmsa_mutex);
+
+	if (VALID_PAGE(svm->sev_es.snp_guest_vmsa_gpa) &&
+	    VALID_PAGE(svm->vmcb->control.vmsa_pa)) {

[Severity: High]
Does this safely handle transient MMU invalidations?

When a concurrent host MMU invalidation (e.g., from page migration) overlaps with a VMSA reload, __sev_snp_reload_vmsa() will hit the retry path and set vmsa_pa to INVALID_PAGE while correctly queuing KVM_REQ_VMSA_PAGE_RELOAD.

If userspace queries the state to migrate the VM during this window, this check evaluates to false because vmsa_pa is INVALID_PAGE. Consequently, it returns VMSA_VALID = 0 even though snp_guest_vmsa_gpa is correctly configured. Can this cause the configured VMSA GPA to be silently dropped and lead to state loss on the destination VM?

Quoted 3 lines
arch/x86/kvm/svm/sev.c +3 -0
+		state.vmsa_gpa = svm->sev_es.snp_guest_vmsa_gpa;
+		state.valid_fields |= KVM_SEV_SNP_VCPU_STATE_VMSA_VALID;
+	}

[ ... ]

Quoted 3 lines
arch/x86/kvm/svm/sev.c +3 -0
+static int snp_set_vcpu_state(struct kvm_vcpu *vcpu,
+			      struct kvm_sev_cmd *argp)
+{

[ ... ]

arch/x86/kvm/svm/sev.c static int snp_launch_update_vmsa(struct kvm *kvm, struct kvm_sev_cmd *argp) · line 2648

Quoted 4 lines
arch/x86/kvm/svm/sev.c +4 -0
+	if (state.valid_fields & KVM_SEV_SNP_VCPU_STATE_VMSA_VALID) {
+		if (!PAGE_ALIGNED(state.vmsa_gpa) ||
+		    !page_address_valid(vcpu, state.vmsa_gpa) ||
+		    IS_ALIGNED(state.vmsa_gpa, PMD_SIZE))

[Severity: High]
Is it intended to reject perfectly valid 4KB-aligned VMSA GPAs if they happen to fall on a 2MB boundary?

A guest OS or userspace VMM linearly allocating memory might naturally land on a 2MB-aligned address (e.g., 0x200000). The IS_ALIGNED(state.vmsa_gpa, PMD_SIZE) condition evaluates to true for these addresses and rejects them with -EINVAL, which can cause sporadic VM boot or migration failures.

Quoted 2 lines
arch/x86/kvm/svm/sev.c +2 -0
+			return -EINVAL;
+	}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=6
#10 lore raw

Re: [PATCH v2 6/8] KVM: SEV: Add SNP vCPU state get and set commands

In reply to Jörg Rödel
Quoted 3 lines · Jörg Rödel

From: Joerg Roedel <[email protected]>

arch/x86/kvm/svm/sev.c +0 -0

...

Quoted 73 lines
arch/x86/kvm/svm/sev.c +73 -0
+static int snp_get_vcpu_state(struct kvm_vcpu *vcpu,
+			      struct kvm_sev_cmd *argp)
+{
+	struct vcpu_svm *svm = to_svm(vcpu);
+	struct kvm *kvm = vcpu->kvm;
+	struct kvm_sev_snp_vcpu_state state = {};
+
+	if (!is_sev_snp_guest(vcpu))
+		return -ENOTTY;
+	if (!to_kvm_sev_info(kvm)->snp_context)
+		return -EINVAL;
+
+	guard(mutex)(&svm->sev_es.snp_vmsa_mutex);
+
+	if (VALID_PAGE(svm->sev_es.snp_guest_vmsa_gpa) &&
+	    VALID_PAGE(svm->vmcb->control.vmsa_pa)) {
+		state.vmsa_gpa = svm->sev_es.snp_guest_vmsa_gpa;
+		state.valid_fields |= KVM_SEV_SNP_VCPU_STATE_VMSA_VALID;
+	}
+
+	if (VALID_PAGE(svm->vmcb->control.ghcb_gpa)) {
+		state.ghcb_gpa = svm->vmcb->control.ghcb_gpa;
+		state.valid_fields |= KVM_SEV_SNP_VCPU_STATE_GHCB_VALID;
+	}
+
+	if (copy_to_user(u64_to_user_ptr(argp->data), &state, sizeof(state)))
+		return -EFAULT;
+
+	return 0;
+}
+
+static int snp_set_vcpu_state(struct kvm_vcpu *vcpu,
+			      struct kvm_sev_cmd *argp)
+{
+	struct vcpu_svm *svm = to_svm(vcpu);
+	struct kvm *kvm = vcpu->kvm;
+	struct kvm_sev_info *sev = to_kvm_sev_info(kvm);
+	struct kvm_sev_snp_vcpu_state state;
+	int ret;
+
+	if (!is_sev_snp_guest(vcpu))
+		return -ENOTTY;
+	if (!sev->snp_direct_vmsa)
+		return -EINVAL;
+	if (!sev->snp_context || kvm->arch.pre_fault_allowed)
+		return -EINVAL;
+
+	if (copy_from_user(&state, u64_to_user_ptr(argp->data), sizeof(state)))
+		return -EFAULT;
+
+	if (memchr_inv(state.pad, 0, sizeof(state.pad)) ||
+	    state.valid_fields & ~(KVM_SEV_SNP_VCPU_STATE_VMSA_VALID |
+				   KVM_SEV_SNP_VCPU_STATE_GHCB_VALID))
+		return -EINVAL;
+
+	if (state.valid_fields & KVM_SEV_SNP_VCPU_STATE_VMSA_VALID) {
+		if (!PAGE_ALIGNED(state.vmsa_gpa) ||
+		    !page_address_valid(vcpu, state.vmsa_gpa) ||
+		    IS_ALIGNED(state.vmsa_gpa, PMD_SIZE))
+			return -EINVAL;
+	}
+
+	guard(mutex)(&svm->sev_es.snp_vmsa_mutex);
+
+	if (state.valid_fields & KVM_SEV_SNP_VCPU_STATE_VMSA_VALID) {
+		ret = sev_snp_install_guest_vmsa(svm, state.vmsa_gpa);
+		if (ret)
+			return ret;
+	} else {
+		svm->sev_es.snp_has_guest_vmsa = true;
+		svm->sev_es.snp_guest_vmsa_gpa = INVALID_PAGE;
+		svm->vmcb->control.vmsa_pa = INVALID_PAGE;
+	}

arch/x86/kvm/svm/sev.c static int snp_launch_update_vmsa(struct kvm *kvm, struct kvm_sev_cmd *argp) · line 2672

Quoted 10 lines
arch/x86/kvm/svm/sev.c +10 -0
+
+	if (state.valid_fields & KVM_SEV_SNP_VCPU_STATE_GHCB_VALID)
+		svm->vmcb->control.ghcb_gpa = state.ghcb_gpa;
+	else
+		svm->vmcb->control.ghcb_gpa = INVALID_PAGE;
+
+	vmcb_mark_all_dirty(svm->vmcb);
+	return 0;
+}
+

Nice, looks like snp_get/set_vcpu_state() could quite easily make use of KVM_EXPORT/IMPORT_VCPU?

#11 lore raw

Re: [PATCH v2 6/8] KVM: SEV: Add SNP vCPU state get and set commands

Hi Tony,

Quoted 2 lines · Tony Lindgren

Nice, looks like snp_get/set_vcpu_state() could quite easily make use of KVM_EXPORT/IMPORT_VCPU?

That depends on whether KVM_EXPORT/IMPORT_VCPU will migrate the confidential VCPU state (registers) only or hypervisor state as well. The get/set_state helpers here only export/import the hypervisor tracking state for the guest VCPUs.

On AMD the confidential VCPU state is migrated as part of the memory image when guest execution has stopped on the source side.

-Joerg

#12 lore raw

Re: [PATCH v2 6/8] KVM: SEV: Add SNP vCPU state get and set commands

Hi Jörg,

Quoted 8 lines · Jörg Rödel
Quoted 2 lines · Tony Lindgren

Nice, looks like snp_get/set_vcpu_state() could quite easily make use of KVM_EXPORT/IMPORT_VCPU?

That depends on whether KVM_EXPORT/IMPORT_VCPU will migrate the confidential VCPU state (registers) only or hypervisor state as well. The get/set_state helpers here only export/import the hypervisor tracking state for the guest VCPUs.

Oh OK, so calling for both hypervisor and confidential VCPU state would be needed. The difference could be specified in the struct kvm_vcpu_transfer flags?

Quoted 2 lines

On AMD the confidential VCPU state is migrated as part of the memory image when guest execution has stopped on the source side.

This part for the confidential VCPU transfer matches what we are doing for TDX.

Regards,

Tony

#13 lore raw

Re: [PATCH v2 6/8] KVM: SEV: Add SNP vCPU state get and set commands

Hi Tony,

Quoted 3 lines · Tony Lindgren

Oh OK, so calling for both hypervisor and confidential VCPU state would be needed. The difference could be specified in the struct kvm_vcpu_transfer flags?

Or hypervisor state is migrated outside of the CVM live migration interface? From QEMUs perspective it is not different to other state like for emulated devices and such. Does TDX also have per-cpu hypervisor state that needs to be migrated?

In any case, for SNP this state must be settable from QEMU to correctly support IGVM loading, so an interface outside of live migration is needed :)

Regards,

Joerg

#14 lore raw

Re: [PATCH v2 6/8] KVM: SEV: Add SNP vCPU state get and set commands

Quoted 7 lines · Jörg Rödel

Hi Tony,

Quoted 3 lines · Tony Lindgren

Oh OK, so calling for both hypervisor and confidential VCPU state would be needed. The difference could be specified in the struct kvm_vcpu_transfer flags?

Quoted 6 lines

Or hypervisor state is migrated outside of the CVM live migration interface? From QEMUs perspective it is not different to other state like for emulated devices and such. Does TDX also have per-cpu hypervisor state that needs to be migrated?

In any case, for SNP this state must be settable from QEMU to correctly support IGVM loading, so an interface outside of live migration is needed :)

Oh OK so the per-cpu hypervisor state is part of the IGVM loading.

Yes the per-vcpu migration is a bit different for TDX. No two separate steps like you're describing for vCPU migration (per-cpu for vCPU registers and per-cpu for hypervisor state).

After stopping the TDX source, the VM state is migrated. And then the per-cpu state. The per-vcpu state is just a QEMU VMState for TDX too, and it needs the help of KVM to export the state into an encrypted blob.

#15 lore raw

PATCH v2 7/8 KVM: selftests: Test the SNP APIC-ID-list GHCB request

In reply to Jörg Rödel

From: Joerg Roedel <[email protected]>

Exercise the GET_AP_APIC_IDS NAE event from an SNP guest using a test-local GHCB encoder. Verify the exact count and APIC IDs, missing RAX validity, zero-sized requests, invalid GPAs, and the 1024-vCPU descriptor boundary when supported.

Assisted-by LLM
---
 tools/testing/selftests/kvm/Makefile.kvm      |   1 +
 .../selftests/kvm/x86/sev_snp_apic_id_test.c  | 180 ++++++++++++++++++
 2 files changed, 181 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/x86/sev_snp_apic_id_test.c
tools/testing/selftests/kvm/Makefile.kvm +1 -0
index 96bab7002d39..92eb3dff13ff 100644
@@ -149,6 +149,7 @@ TEST_GEN_PROGS_x86 += x86/xen_vmcall_test
 TEST_GEN_PROGS_x86 += x86/sev_dbg_test
 TEST_GEN_PROGS_x86 += x86/sev_init2_tests
 TEST_GEN_PROGS_x86 += x86/sev_migrate_tests
+TEST_GEN_PROGS_x86 += x86/sev_snp_apic_id_test
 TEST_GEN_PROGS_x86 += x86/sev_smoke_test
 TEST_GEN_PROGS_x86 += x86/amx_test
 TEST_GEN_PROGS_x86 += x86/max_vcpuid_cap_test
tools/testing/selftests/kvm/x86/sev_snp_apic_id_test.c new file +180 -0

180 lines not on this page · raw message

Signature · 1 line
2.53.0
#16 lore raw

PATCH v2 8/8 KVM: selftests: Test SNP vCPU state and direct VMSA launch

In reply to Jörg Rödel

From: Joerg Roedel <[email protected]>

Add vCPU-scoped SEV ioctl helpers and exercise the SNP vCPU state get and set commands. Enable KVM_CAP_SNP_DIRECT_VMSA before creating vCPUs and verify that invalid capability arguments and late enable attempts fail.

Boot through a handcrafted BSP VMSA, verify VMSA and GHCB address round trips and validity bits, rebind to the selected GPA, and verify an extra unassociated AP remains uninitialized.

Reject multi-page launch updates and VMSAs with nonzero VMPL or mismatched SEV features. Also verify the legacy launch path and reject non-SNP use, bad or reserved arguments, unsafe alignment, absent pages, wrong ordering, and post-launch SET calls. Verify that SET accepts both shared and private guestmemfd-backed VMSA GPAs without consulting memory attributes or the RMP.

Assisted-by LLM
---
 tools/testing/selftests/kvm/Makefile.kvm      |   1 +
 tools/testing/selftests/kvm/include/x86/sev.h |  49 ++
 .../kvm/x86/sev_snp_direct_vmsa_test.c        | 421 ++++++++++++++++++
 3 files changed, 471 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/x86/sev_snp_direct_vmsa_test.c
tools/testing/selftests/kvm/Makefile.kvm +1 -0
index 92eb3dff13ff..4cf0a8297b04 100644
@@ -150,6 +150,7 @@ TEST_GEN_PROGS_x86 += x86/sev_dbg_test
 TEST_GEN_PROGS_x86 += x86/sev_init2_tests
 TEST_GEN_PROGS_x86 += x86/sev_migrate_tests
 TEST_GEN_PROGS_x86 += x86/sev_snp_apic_id_test
+TEST_GEN_PROGS_x86 += x86/sev_snp_direct_vmsa_test
 TEST_GEN_PROGS_x86 += x86/sev_smoke_test
 TEST_GEN_PROGS_x86 += x86/amx_test
 TEST_GEN_PROGS_x86 += x86/max_vcpuid_cap_test
tools/testing/selftests/kvm/include/x86/sev.h +49 -0

61 lines not on this page · raw message

tools/testing/selftests/kvm/x86/sev_snp_direct_vmsa_test.c new file +421 -0

421 lines not on this page · raw message

Signature · 1 line
2.53.0