Skip to contentSkip to conversation

Add AMD SEV and SEV-ES intra host migration support

PATCH Superseded Peter Gonda 2 participants 8 messages Oct 5, 2021 – Oct 8, 2021

Subsystems (5)
Documentation x86 KVM Kselftest +1 more subsystems

v9 of 11 Superseded Latest is v11 →

lore
Share & tools
lore
#0 lore raw

v9 cover Add AMD SEV and SEV-ES intra host migration support

Intra host migration provides a low-cost mechanism for userspace VMM upgrades. It is an alternative to traditional (i.e., remote) live migration. Whereas remote migration handles moving a guest to a new host, intra host migration only handles moving a guest to a new userspace VMM within a host. This can be used to update, rollback, change flags of the VMM, etc. The lower cost compared to live migration comes from the fact that the guest's memory does not need to be copied between processes. A handle to the guest memory simply gets passed to the new VMM, this could be done via /dev/shm with share=on or similar feature.

The guest state can be transferred from an old VMM to a new VMM as follows:
1. Export guest state from KVM to the old user-space VMM via a getter user-space/kernel API 2. Transfer guest state from old VMM to new VMM via IPC communication 3. Import guest state into KVM from the new user-space VMM via a setter user-space/kernel API VMMs by exporting from KVM using getters, sending that data to the new VMM, then setting it again in KVM.

In the common case for intra host migration, we can rely on the normal ioctls for passing data from one VMM to the next. SEV, SEV-ES, and other confidential compute environments make most of this information opaque, and render KVM ioctls such as "KVM_GET_REGS" irrelevant. As a result, we need the ability to pass this opaque metadata from one VMM to the next. The easiest way to do this is to leave this data in the kernel, and transfer ownership of the metadata from one KVM VM (or vCPU) to the next. For example, we need to move the SEV enabled ASID, VMSAs, and GHCB metadata from one VMM to the next. In general, we need to be able to hand off any data that would be unsafe/impossible for the kernel to hand directly to userspace (and cannot be reproduced using data that can be handed safely to userspace).

V9
* Fix sev_lock_vcpus_for_migration from unlocking the vCPU mutex it failed to unlock.

V8
* Update to require that @dst is not SEV or SEV-ES enabled.
* Address selftest feedback.

V7
* Address selftest feedback.

V6
* Add selftest.

V5:
* Fix up locking scheme
* Address marcorr@ comments.

V4:
* Move to seanjc@'s suggestion of source VM FD based single ioctl design.

v3:
* Fix memory leak found by dan.carpenter@

v2:
* Added marcorr@ reviewed by tag
* Renamed function introduced in 1/3
* Edited with seanjc@'s review comments
** Cleaned up WARN usage
** Userspace makes random token now
* Edited with brijesh.singh@'s review comments ** Checks for different LAUNCH_* states in send function

v1: https://lore.kernel.org/kvm/[email protected]/

base-commit: 680c7e3be6a3

Cc Marc Orr <[email protected]>
Cc Paolo Bonzini <[email protected]>
Cc Sean Christopherson <[email protected]>
Cc David Rientjes <[email protected]>
Cc Dr. David Alan Gilbert <[email protected]>
Cc Brijesh Singh <[email protected]>
Cc Vitaly Kuznetsov <[email protected]>
Cc Wanpeng Li <[email protected]>
Cc Jim Mattson <[email protected]>
Cc Joerg Roedel <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Borislav Petkov <[email protected]>
Cc "H. Peter Anvin" <[email protected]>

Peter Gonda (4):
KVM: SEV: Add support for SEV intra host migration
KVM: SEV: Add support for SEV-ES intra host migration
selftest: KVM: Add open sev dev helper
selftest: KVM: Add intra host migration tests

 Documentation/virt/kvm/api.rst                |  15 ++
 arch/x86/include/asm/kvm_host.h               |   1 +
 arch/x86/kvm/svm/sev.c                        | 187 ++++++++++++++++
 arch/x86/kvm/svm/svm.c                        |   1 +
 arch/x86/kvm/svm/svm.h                        |   2 +
 arch/x86/kvm/x86.c                            |   6 +
 include/uapi/linux/kvm.h                      |   1 +
 tools/testing/selftests/kvm/Makefile          |   1 +
 .../testing/selftests/kvm/include/kvm_util.h  |   1 +
 .../selftests/kvm/include/x86_64/svm_util.h   |   2 +
 tools/testing/selftests/kvm/lib/kvm_util.c    |  24 ++-
 tools/testing/selftests/kvm/lib/x86_64/svm.c  |  13 ++
 .../selftests/kvm/x86_64/sev_vm_tests.c       | 203 ++++++++++++++++++
 13 files changed, 447 insertions(+), 10 deletions(-)
 create mode 100644 tools/testing/selftests/kvm/x86_64/sev_vm_tests.c
Signature · 1 line
2.33.0.309.g3052b89438-goog
#1 lore raw

PATCH v9 1/4 KVM: SEV: Add support for SEV intra host migration

For SEV to work with intra host migration, contents of the SEV info struct such as the ASID (used to index the encryption key in the AMD SP) and the list of memory regions need to be transferred to the target VM. This change adds a commands for a target VMM to get a source SEV VM's sev info.

Cc Marc Orr <[email protected]>
Cc Paolo Bonzini <[email protected]>
Cc Sean Christopherson <[email protected]>
Cc David Rientjes <[email protected]>
Cc Dr. David Alan Gilbert <[email protected]>
Cc Brijesh Singh <[email protected]>
Cc Vitaly Kuznetsov <[email protected]>
Cc Wanpeng Li <[email protected]>
Cc Jim Mattson <[email protected]>
Cc Joerg Roedel <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Borislav Petkov <[email protected]>
Cc "H. Peter Anvin" <[email protected]>
---
 Documentation/virt/kvm/api.rst  |  15 ++++
 arch/x86/include/asm/kvm_host.h |   1 +
 arch/x86/kvm/svm/sev.c          | 137 ++++++++++++++++++++++++++++++++
 arch/x86/kvm/svm/svm.c          |   1 +
 arch/x86/kvm/svm/svm.h          |   2 +
 arch/x86/kvm/x86.c              |   6 ++
 include/uapi/linux/kvm.h        |   1 +
 7 files changed, 163 insertions(+)
Documentation/virt/kvm/api.rst +15 -0
index 3b093d6dbe22..d9797c6d4b1d 100644
@@ -6911,6 +6911,21 @@ MAP_SHARED mmap will result in an -EINVAL return.
 When enabled the VMM may make use of the ``KVM_ARM_MTE_COPY_TAGS`` ioctl to
 perform a bulk copy of tags to/from the guest.
 
+7.29 KVM_CAP_VM_MIGRATE_PROTECTED_VM_FROM
+-------------------------------------
+
+Architectures: x86 SEV enabled
+Type: vm
+Parameters: args[0] is the fd of the source vm
+Returns: 0 on success
+
+This capability enables userspace to migrate the encryption context from the VM
+indicated by the fd to the VM this is called on.
+
+This is intended to support intra-host migration of VMs between userspace VMMs.
+in-guest workloads scheduled by the host. This allows for upgrading the VMM
+process without interrupting the guest.
+
 8. Other capabilities.
 ======================
 
arch/x86/include/asm/kvm_host.h +1 -0
index 88f0326c184a..a334e6b36309 100644
@@ -1481,6 +1481,7 @@ struct kvm_x86_ops {
 	int (*mem_enc_reg_region)(struct kvm *kvm, struct kvm_enc_region *argp);
 	int (*mem_enc_unreg_region)(struct kvm *kvm, struct kvm_enc_region *argp);
 	int (*vm_copy_enc_context_from)(struct kvm *kvm, unsigned int source_fd);
+	int (*vm_migrate_protected_vm_from)(struct kvm *kvm, unsigned int source_fd);
 
 	int (*get_msr_feature)(struct kvm_msr_entry *entry);
 
arch/x86/kvm/svm/sev.c +137 -0

143 lines not on this page · raw message

arch/x86/kvm/svm/svm.c +1 -0
index 89077160d463..1bda39844773 100644
@@ -4697,6 +4697,7 @@ static struct kvm_x86_ops svm_x86_ops __initdata = {
 	.mem_enc_unreg_region = svm_unregister_enc_region,
 
 	.vm_copy_enc_context_from = svm_vm_copy_asid_from,
+	.vm_migrate_protected_vm_from = svm_vm_migrate_from,
 
 	.can_emulate_instruction = svm_can_emulate_instruction,
 
arch/x86/kvm/svm/svm.h +2 -0
index 0d7bbe548ac3..064e7c7f2834 100644
@@ -80,6 +80,7 @@ struct kvm_sev_info {
 	u64 ap_jump_table;	/* SEV-ES AP Jump Table address */
 	struct kvm *enc_context_owner; /* Owner of copied encryption context */
 	struct misc_cg *misc_cg; /* For misc cgroup accounting */
+	atomic_t migration_in_progress;
 };
 
 struct kvm_svm {
@@ -558,6 +559,7 @@ int svm_register_enc_region(struct kvm *kvm,
 int svm_unregister_enc_region(struct kvm *kvm,
 			      struct kvm_enc_region *range);
 int svm_vm_copy_asid_from(struct kvm *kvm, unsigned int source_fd);
+int svm_vm_migrate_from(struct kvm *kvm, unsigned int source_fd);
 void pre_sev_run(struct vcpu_svm *svm, int cpu);
 void __init sev_set_cpu_caps(void);
 void __init sev_hardware_setup(void);
arch/x86/kvm/x86.c +6 -0
index 196ac33ef958..093deb784b6b 100644
@@ -5806,6 +5806,12 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm,
 		if (kvm_x86_ops.vm_copy_enc_context_from)
 			r = kvm_x86_ops.vm_copy_enc_context_from(kvm, cap->args[0]);
 		return r;
+	case KVM_CAP_VM_MIGRATE_PROTECTED_VM_FROM:
+		r = -EINVAL;
+		if (kvm_x86_ops.vm_migrate_protected_vm_from)
+			r = kvm_x86_ops.vm_migrate_protected_vm_from(
+				kvm, cap->args[0]);
+		return r;
 	case KVM_CAP_EXIT_HYPERCALL:
 		if (cap->args[0] & ~KVM_EXIT_HYPERCALL_VALID_MASK) {
 			r = -EINVAL;
include/uapi/linux/kvm.h +1 -0
index 5ca5ffe16cb4..dabd143aad8f 100644
@@ -1120,6 +1120,7 @@ struct kvm_ppc_resize_hpt {
 #define KVM_CAP_BINARY_STATS_FD 203
 #define KVM_CAP_EXIT_ON_EMULATION_FAILURE 204
 #define KVM_CAP_ARM_MTE 205
+#define KVM_CAP_VM_MIGRATE_PROTECTED_VM_FROM 206
 
 #ifdef KVM_CAP_IRQ_ROUTING
 
Signature · 1 line
2.33.0.800.g4c38ced690-goog
#2 lore raw

PATCH v9 2/4 KVM: SEV: Add support for SEV-ES intra host migration

In reply to Peter Gonda

For SEV-ES to work with intra host migration the VMSAs, GHCB metadata, and other SEV-ES info needs to be preserved along with the guest's memory.

Cc Marc Orr <[email protected]>
Cc Paolo Bonzini <[email protected]>
Cc Sean Christopherson <[email protected]>
Cc David Rientjes <[email protected]>
Cc Dr. David Alan Gilbert <[email protected]>
Cc Brijesh Singh <[email protected]>
Cc Vitaly Kuznetsov <[email protected]>
Cc Wanpeng Li <[email protected]>
Cc Jim Mattson <[email protected]>
Cc Joerg Roedel <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Borislav Petkov <[email protected]>
Cc "H. Peter Anvin" <[email protected]>
---
 arch/x86/kvm/svm/sev.c | 53 +++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 52 insertions(+), 1 deletion(-)
arch/x86/kvm/svm/sev.c +52 -1
index 6fc1935b52ea..321b55654f36 100644
@@ -1576,6 +1576,51 @@ static void sev_migrate_from(struct kvm_sev_info *dst,
 	list_replace_init(&src->regions_list, &dst->regions_list);
 }
 
+static int sev_es_migrate_from(struct kvm *dst, struct kvm *src)
+{
+	int i;
+	struct kvm_vcpu *dst_vcpu, *src_vcpu;
+	struct vcpu_svm *dst_svm, *src_svm;
+
+	if (atomic_read(&src->online_vcpus) != atomic_read(&dst->online_vcpus))
+		return -EINVAL;
+
+	kvm_for_each_vcpu(i, src_vcpu, src) {
+		if (!src_vcpu->arch.guest_state_protected)
+			return -EINVAL;
+	}
+
+	kvm_for_each_vcpu(i, src_vcpu, src) {
+		src_svm = to_svm(src_vcpu);
+		dst_vcpu = dst->vcpus[i];
+		dst_vcpu = kvm_get_vcpu(dst, i);
+		dst_svm = to_svm(dst_vcpu);
+
+		/*
+		 * Transfer VMSA and GHCB state to the destination.  Nullify and
+		 * clear source fields as appropriate, the state now belongs to
+		 * the destination.
+		 */
+		dst_vcpu->vcpu_id = src_vcpu->vcpu_id;
+		dst_svm->vmsa = src_svm->vmsa;
+		src_svm->vmsa = NULL;
+		dst_svm->ghcb = src_svm->ghcb;
+		src_svm->ghcb = NULL;
+		dst_svm->vmcb->control.ghcb_gpa = src_svm->vmcb->control.ghcb_gpa;
+		dst_svm->ghcb_sa = src_svm->ghcb_sa;
+		src_svm->ghcb_sa = NULL;
+		dst_svm->ghcb_sa_len = src_svm->ghcb_sa_len;
+		src_svm->ghcb_sa_len = 0;
+		dst_svm->ghcb_sa_sync = src_svm->ghcb_sa_sync;
+		src_svm->ghcb_sa_sync = false;
+		dst_svm->ghcb_sa_free = src_svm->ghcb_sa_free;
+		src_svm->ghcb_sa_free = false;
+	}
+	to_kvm_svm(src)->sev_info.es_active = false;
+
+	return 0;
+}
+
 int svm_vm_migrate_from(struct kvm *kvm, unsigned int source_fd)
 {
 	struct kvm_sev_info *dst_sev = &to_kvm_svm(kvm)->sev_info;
@@ -1604,7 +1649,7 @@ int svm_vm_migrate_from(struct kvm *kvm, unsigned int source_fd)
 	if (ret)
 		goto out_fput;
 
-	if (!sev_guest(source_kvm) || sev_es_guest(source_kvm)) {
+	if (!sev_guest(source_kvm)) {
 		ret = -EINVAL;
 		goto out_source;
 	}
@@ -1615,6 +1660,12 @@ int svm_vm_migrate_from(struct kvm *kvm, unsigned int source_fd)
 	if (ret)
 		goto out_source_vcpu;
 
+	if (sev_es_guest(source_kvm)) {
+		ret = sev_es_migrate_from(kvm, source_kvm);
+		if (ret)
+			goto out_source_vcpu;
+	}
+
 	sev_migrate_from(dst_sev, &to_kvm_svm(source_kvm)->sev_info);
 	kvm_for_each_vcpu (i, vcpu, source_kvm) {
 		kvm_vcpu_reset(vcpu, /* init_event= */ false);
Signature · 1 line
2.33.0.309.g3052b89438-goog
#3 lore raw

Re: [PATCH 2/4 V9] KVM: SEV: Add support for SEV-ES intra host migration

Quoted 42 lines · Peter Gonda

For SEV-ES to work with intra host migration the VMSAs, GHCB metadata, and other SEV-ES info needs to be preserved along with the guest's memory.

Cc Marc Orr <[email protected]>
Cc Paolo Bonzini <[email protected]>
Cc Sean Christopherson <[email protected]>
Cc David Rientjes <[email protected]>
Cc Dr. David Alan Gilbert <[email protected]>
Cc Brijesh Singh <[email protected]>
Cc Vitaly Kuznetsov <[email protected]>
Cc Wanpeng Li <[email protected]>
Cc Jim Mattson <[email protected]>
Cc Joerg Roedel <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Borislav Petkov <[email protected]>
Cc "H. Peter Anvin" <[email protected]>
---
  arch/x86/kvm/svm/sev.c | 53 +++++++++++++++++++++++++++++++++++++++++-
  1 file changed, 52 insertions(+), 1 deletion(-)
arch/x86/kvm/svm/sev.c +8 -0
index 6fc1935b52ea..321b55654f36 100644
@@ -1576,6 +1576,51 @@ static void sev_migrate_from(struct kvm_sev_info *dst,
  	list_replace_init(&src->regions_list, &dst->regions_list);
  }
 
+static int sev_es_migrate_from(struct kvm *dst, struct kvm *src)
+{
+	int i;
+	struct kvm_vcpu *dst_vcpu, *src_vcpu;
+	struct vcpu_svm *dst_svm, *src_svm;
+
+	if (atomic_read(&src->online_vcpus) != atomic_read(&dst->online_vcpus))
+		return -EINVAL;

arch/x86/kvm/svm/sev.c static void sev_migrate_from(struct kvm_sev_info *dst, · line 1596

Quoted 10 lines
arch/x86/kvm/svm/sev.c +10 -0
+
+	kvm_for_each_vcpu(i, src_vcpu, src) {
+		if (!src_vcpu->arch.guest_state_protected)
+			return -EINVAL;
+	}
+
+	kvm_for_each_vcpu(i, src_vcpu, src) {
+		src_svm = to_svm(src_vcpu);
+		dst_vcpu = dst->vcpus[i];
+		dst_vcpu = kvm_get_vcpu(dst, i);

One of these assignments of dst_vcpu can be deleted.

Quoted 11 lines
arch/x86/kvm/svm/sev.c +11 -0
+		dst_svm = to_svm(dst_vcpu);
+
+		/*
+		 * Transfer VMSA and GHCB state to the destination.  Nullify and
+		 * clear source fields as appropriate, the state now belongs to
+		 * the destination.
+		 */
+		dst_vcpu->vcpu_id = src_vcpu->vcpu_id;
+		dst_svm->vmsa = src_svm->vmsa;
+		src_svm->vmsa = NULL;
+		dst_svm->ghcb = src_svm->ghcb;

arch/x86/kvm/svm/sev.c static void sev_migrate_from(struct kvm_sev_info *dst, · line 1617

Quoted 10 lines
arch/x86/kvm/svm/sev.c +10 -0
+		src_svm->ghcb = NULL;
+		dst_svm->vmcb->control.ghcb_gpa = src_svm->vmcb->control.ghcb_gpa;
+		dst_svm->ghcb_sa = src_svm->ghcb_sa;
+		src_svm->ghcb_sa = NULL;
+		dst_svm->ghcb_sa_len = src_svm->ghcb_sa_len;
+		src_svm->ghcb_sa_len = 0;
+		dst_svm->ghcb_sa_sync = src_svm->ghcb_sa_sync;
+		src_svm->ghcb_sa_sync = false;
+		dst_svm->ghcb_sa_free = src_svm->ghcb_sa_free;
+		src_svm->ghcb_sa_free = false;

Would it make sense to have a pre-patch that puts these fields into a struct? Then you can just copy the struct and zero it after. If anything is ever added for any reason, then it could/should be added to the struct and this code wouldn't have to change. It might be more churn than it's worth, just a thought.

Thanks,
Tom

Quoted 32 lines
arch/x86/kvm/svm/sev.c +13 -1
+	}
+	to_kvm_svm(src)->sev_info.es_active = false;
+
+	return 0;
+}
+
  int svm_vm_migrate_from(struct kvm *kvm, unsigned int source_fd)
  {
  	struct kvm_sev_info *dst_sev = &to_kvm_svm(kvm)->sev_info;
@@ -1604,7 +1649,7 @@ int svm_vm_migrate_from(struct kvm *kvm, unsigned int source_fd)
  	if (ret)
  		goto out_fput;
 
-	if (!sev_guest(source_kvm) || sev_es_guest(source_kvm)) {
+	if (!sev_guest(source_kvm)) {
  		ret = -EINVAL;
  		goto out_source;
  	}
@@ -1615,6 +1660,12 @@ int svm_vm_migrate_from(struct kvm *kvm, unsigned int source_fd)
  	if (ret)
  		goto out_source_vcpu;
 
+	if (sev_es_guest(source_kvm)) {
+		ret = sev_es_migrate_from(kvm, source_kvm);
+		if (ret)
+			goto out_source_vcpu;
+	}
+
  	sev_migrate_from(dst_sev, &to_kvm_svm(source_kvm)->sev_info);
  	kvm_for_each_vcpu (i, vcpu, source_kvm) {
  		kvm_vcpu_reset(vcpu, /* init_event= */ false);
#4 lore raw

Re: [PATCH 2/4 V9] KVM: SEV: Add support for SEV-ES intra host migration

Quoted 48 lines · Tom Lendacky
Quoted 46 lines · Peter Gonda

For SEV-ES to work with intra host migration the VMSAs, GHCB metadata, and other SEV-ES info needs to be preserved along with the guest's memory.

Cc Marc Orr <[email protected]>
Cc Paolo Bonzini <[email protected]>
Cc Sean Christopherson <[email protected]>
Cc David Rientjes <[email protected]>
Cc Dr. David Alan Gilbert <[email protected]>
Cc Brijesh Singh <[email protected]>
Cc Vitaly Kuznetsov <[email protected]>
Cc Wanpeng Li <[email protected]>
Cc Jim Mattson <[email protected]>
Cc Joerg Roedel <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Borislav Petkov <[email protected]>
Cc "H. Peter Anvin" <[email protected]>
---
  arch/x86/kvm/svm/sev.c | 53 +++++++++++++++++++++++++++++++++++++++++-
  1 file changed, 52 insertions(+), 1 deletion(-)
arch/x86/kvm/svm/sev.c +12 -0
index 6fc1935b52ea..321b55654f36 100644
@@ -1576,6 +1576,51 @@ static void sev_migrate_from(struct kvm_sev_info *dst,
      list_replace_init(&src->regions_list, &dst->regions_list);
  }
 
+static int sev_es_migrate_from(struct kvm *dst, struct kvm *src)
+{
+     int i;
+     struct kvm_vcpu *dst_vcpu, *src_vcpu;
+     struct vcpu_svm *dst_svm, *src_svm;
+
+     if (atomic_read(&src->online_vcpus) != atomic_read(&dst->online_vcpus))
+             return -EINVAL;
+
+     kvm_for_each_vcpu(i, src_vcpu, src) {
+             if (!src_vcpu->arch.guest_state_protected)
+                     return -EINVAL;
Quoted 8 lines

arch/x86/kvm/svm/sev.c static void sev_migrate_from(struct kvm_sev_info *dst, · line 1596

Quoted 6 lines
arch/x86/kvm/svm/sev.c +6 -0
+     }
+
+     kvm_for_each_vcpu(i, src_vcpu, src) {
+             src_svm = to_svm(src_vcpu);
+             dst_vcpu = dst->vcpus[i];
+             dst_vcpu = kvm_get_vcpu(dst, i);

One of these assignments of dst_vcpu can be deleted.

Good catch I'll remove the dst_vcpu = dst->vcpus[i]; line.

Quoted 23 lines
Quoted 12 lines
arch/x86/kvm/svm/sev.c +11 -0
+             dst_svm = to_svm(dst_vcpu);
+
+             /*
+              * Transfer VMSA and GHCB state to the destination.  Nullify and
+              * clear source fields as appropriate, the state now belongs to
+              * the destination.
+              */
+             dst_vcpu->vcpu_id = src_vcpu->vcpu_id;
+             dst_svm->vmsa = src_svm->vmsa;
+             src_svm->vmsa = NULL;
+             dst_svm->ghcb = src_svm->ghcb;

arch/x86/kvm/svm/sev.c static void sev_migrate_from(struct kvm_sev_info *dst, · line 1617

Quoted 10 lines
arch/x86/kvm/svm/sev.c +10 -0
+             src_svm->ghcb = NULL;
+             dst_svm->vmcb->control.ghcb_gpa = src_svm->vmcb->control.ghcb_gpa;
+             dst_svm->ghcb_sa = src_svm->ghcb_sa;
+             src_svm->ghcb_sa = NULL;
+             dst_svm->ghcb_sa_len = src_svm->ghcb_sa_len;
+             src_svm->ghcb_sa_len = 0;
+             dst_svm->ghcb_sa_sync = src_svm->ghcb_sa_sync;
+             src_svm->ghcb_sa_sync = false;
+             dst_svm->ghcb_sa_free = src_svm->ghcb_sa_free;
+             src_svm->ghcb_sa_free = false;
Quoted 6 lines

Would it make sense to have a pre-patch that puts these fields into a struct? Then you can just copy the struct and zero it after. If anything is ever added for any reason, then it could/should be added to the struct and this code wouldn't have to change. It might be more churn than it's worth, just a thought.

That sounds like a good idea to me. I'll add a new patch to the start of the series which adds in something like:

struct vcpu_sev_es_state {
/* SEV-ES support */
struct vmcb_save_area *vmsa;
struct ghcb *ghcb;
struct kvm_host_map ghcb_map;
bool received_first_sipi;
/* SEV-ES scratch area support */
void *ghcb_sa;
u64 ghcb_sa_len;
bool ghcb_sa_sync;
bool ghcb_sa_free;
};

struct vcpu_svm {
...
struct vcpu_sev_es_state sev_es_state;
...
};

I think that will make this less tedious / error prone code. Names sound OK or better suggestion?

#5 lore raw

Re: [PATCH 2/4 V9] KVM: SEV: Add support for SEV-ES intra host migration

Quoted 25 lines · Peter Gonda
Quoted 7 lines · Tom Lendacky

Would it make sense to have a pre-patch that puts these fields into a struct? Then you can just copy the struct and zero it after. If anything is ever added for any reason, then it could/should be added to the struct and this code wouldn't have to change. It might be more churn than it's worth, just a thought.

That sounds like a good idea to me. I'll add a new patch to the start of the series which adds in something like:

struct vcpu_sev_es_state {
   /* SEV-ES support */
   struct vmcb_save_area *vmsa;
   struct ghcb *ghcb;
   struct kvm_host_map ghcb_map;
   bool received_first_sipi;
   /* SEV-ES scratch area support */
   void *ghcb_sa;
   u64 ghcb_sa_len;
   bool ghcb_sa_sync;
   bool ghcb_sa_free;
};
Quoted 8 lines

struct vcpu_svm {
...
struct vcpu_sev_es_state sev_es_state;
...
};

I think that will make this less tedious / error prone code. Names sound OK or better suggestion?

Those names seem fine to me. If you want to shorten them, you could always drop the "_state" portion.

Thanks,
Tom

Quoted 1 line
#6 lore raw

PATCH v9 3/4 selftest: KVM: Add open sev dev helper

In reply to Peter Gonda

Refactors out open path support from open_kvm_dev_path_or_exit() and adds new helper for SEV device path.

Cc Marc Orr <[email protected]>
Cc Paolo Bonzini <[email protected]>
Cc Sean Christopherson <[email protected]>
Cc David Rientjes <[email protected]>
Cc Dr. David Alan Gilbert <[email protected]>
Cc Brijesh Singh <[email protected]>
Cc Vitaly Kuznetsov <[email protected]>
Cc Wanpeng Li <[email protected]>
Cc Jim Mattson <[email protected]>
Cc Joerg Roedel <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Borislav Petkov <[email protected]>
Cc "H. Peter Anvin" <[email protected]>
---
 .../testing/selftests/kvm/include/kvm_util.h  |  1 +
 .../selftests/kvm/include/x86_64/svm_util.h   |  2 ++
 tools/testing/selftests/kvm/lib/kvm_util.c    | 24 +++++++++++--------
 tools/testing/selftests/kvm/lib/x86_64/svm.c  | 13 ++++++++++
 4 files changed, 30 insertions(+), 10 deletions(-)
tools/testing/selftests/kvm/include/kvm_util.h +1 -0
index 010b59b13917..368e88305046 100644
@@ -80,6 +80,7 @@ struct vm_guest_mode_params {
 };
 extern const struct vm_guest_mode_params vm_guest_mode_params[];
 
+int open_path_or_exit(const char *path, int flags);
 int open_kvm_dev_path_or_exit(void);
 int kvm_check_cap(long cap);
 int vm_enable_cap(struct kvm_vm *vm, struct kvm_enable_cap *cap);
tools/testing/selftests/kvm/include/x86_64/svm_util.h +2 -0
index b7531c83b8ae..587fbe408b99 100644
@@ -46,4 +46,6 @@ static inline bool cpu_has_svm(void)
 	return ecx & CPUID_SVM;
 }
 
+int open_sev_dev_path_or_exit(void);
+
 #endif /* SELFTEST_KVM_SVM_UTILS_H */
tools/testing/selftests/kvm/lib/kvm_util.c +14 -10
index 10a8ed691c66..06a6c04010fb 100644
@@ -31,6 +31,19 @@ static void *align(void *x, size_t size)
 	return (void *) (((size_t) x + mask) & ~mask);
 }
 
+int open_path_or_exit(const char *path, int flags)
+{
+	int fd;
+
+	fd = open(path, flags);
+	if (fd < 0) {
+		print_skip("%s not available (errno: %d)", path, errno);
+		exit(KSFT_SKIP);
+	}
+
+	return fd;
+}
+
 /*
  * Open KVM_DEV_PATH if available, otherwise exit the entire program.
  *
@@ -42,16 +55,7 @@ static void *align(void *x, size_t size)
  */
 static int _open_kvm_dev_path_or_exit(int flags)
 {
-	int fd;
-
-	fd = open(KVM_DEV_PATH, flags);
-	if (fd < 0) {
-		print_skip("%s not available, is KVM loaded? (errno: %d)",
-			   KVM_DEV_PATH, errno);
-		exit(KSFT_SKIP);
-	}
-
-	return fd;
+	return open_path_or_exit(KVM_DEV_PATH, flags);
 }
 
 int open_kvm_dev_path_or_exit(void)
tools/testing/selftests/kvm/lib/x86_64/svm.c +13 -0
index 2ac98d70d02b..14a8618efa9c 100644
@@ -13,6 +13,8 @@
 #include "processor.h"
 #include "svm_util.h"
 
+#define SEV_DEV_PATH "/dev/sev"
+
 struct gpr64_regs guest_regs;
 u64 rflags;
 
@@ -160,3 +162,14 @@ void nested_svm_check_supported(void)
 		exit(KSFT_SKIP);
 	}
 }
+
+/*
+ * Open SEV_DEV_PATH if available, otherwise exit the entire program.
+ *
+ * Return:
+ *   The opened file descriptor of /dev/sev.
+ */
+int open_sev_dev_path_or_exit(void)
+{
+	return open_path_or_exit(SEV_DEV_PATH, 0);
+}
Signature · 1 line
2.33.0.309.g3052b89438-goog
#7 lore raw

PATCH v9 4/4 selftest: KVM: Add intra host migration tests

In reply to Peter Gonda

Adds testcases for intra host migration for SEV and SEV-ES. Also adds locking test to confirm no deadlock exists.

---
 tools/testing/selftests/kvm/Makefile          |   1 +
 .../selftests/kvm/x86_64/sev_vm_tests.c       | 203 ++++++++++++++++++
 2 files changed, 204 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/x86_64/sev_vm_tests.c
tools/testing/selftests/kvm/Makefile +1 -0
index c103873531e0..44fd3566fb51 100644
@@ -72,6 +72,7 @@ TEST_GEN_PROGS_x86_64 += x86_64/vmx_pmu_msrs_test
 TEST_GEN_PROGS_x86_64 += x86_64/xen_shinfo_test
 TEST_GEN_PROGS_x86_64 += x86_64/xen_vmcall_test
 TEST_GEN_PROGS_x86_64 += x86_64/vmx_pi_mmio_test
+TEST_GEN_PROGS_x86_64 += x86_64/sev_vm_tests
 TEST_GEN_PROGS_x86_64 += access_tracking_perf_test
 TEST_GEN_PROGS_x86_64 += demand_paging_test
 TEST_GEN_PROGS_x86_64 += dirty_log_test
tools/testing/selftests/kvm/x86_64/sev_vm_tests.c new file +203 -0

203 lines not on this page · raw message

Signature · 1 line
2.33.0.309.g3052b89438-goog