Skip to contentSkip to conversation

Add TDX intra-host migration support

RFC PATCH In review Ryan Afranji 3 participants 13 messages Jun 11, 2025 – Mar 9

Subsystems (5)
x86 KVM Kselftest KVM x86 +1 more subsystems
lore
Share & tools
lore
#0 lore raw

v2 cover Add TDX intra-host migration support

Hello,

This is RFC v2 for the TDX intra-host migration patch series. It addresses comments in RFC v1 [1] and is rebased onto the latest kvm/next (v6.16-rc1).

This patchset was built on top of the latest TDX selftests [2] and gmem linking [3] RFC patch series.

Here is the series stitched together for your convenience:
https://github.com/googleprodkernel/linux-cc/tree/tdx-copyless-rfc-v2

Changes from RFC v1:
+ Added patch to prevent deadlock warnings by re-ordering locking order.
+ Added patch to allow vCPUs to be created for uninitialized VMs.
+ Minor optimizations to TDX intra-host migration core logic.
+ Moved lapic state transfer into TDX intra-host migration core logic.
+ Added logic to handle posted interrupts that are injected during migration.
+ Added selftests.
+ Addressed comments from RFC v1.
+ Various small changes to make patchset compatible with latest version of kvm/next.

[1] https://lore.kernel.org/lkml/[email protected]
[2] https://lore.kernel.org/lkml/[email protected]
[3] https://lore.kernel.org/all/[email protected]

Ackerley Tng (2):
KVM: selftests: Add TDX support for ucalls
KVM: selftests: Add irqfd/interrupts test for TDX with migration

Ryan Afranji (3):
KVM: x86: Adjust locking order in move_enc_context_from
KVM: TDX: Allow vCPUs to be created for migration
KVM: selftests: Refactor userspace_mem_region creation out of
vm_mem_add

Sagi Shahar (5):
KVM: Split tdp_mmu_pages to mirror and direct counters
KVM: TDX: Add base implementation for tdx_vm_move_enc_context_from
KVM: TDX: Implement moving mirror pages between 2 TDs
KVM: TDX: Add core logic for TDX intra-host migration
KVM: selftests: TDX: Add tests for TDX in-place migration

 arch/x86/include/asm/kvm_host.h               |   7 +-
 arch/x86/kvm/mmu.h                            |   2 +
 arch/x86/kvm/mmu/mmu.c                        |  66 ++++
 arch/x86/kvm/mmu/tdp_mmu.c                    |  72 +++-
 arch/x86/kvm/mmu/tdp_mmu.h                    |   6 +
 arch/x86/kvm/svm/sev.c                        |  13 +-
 arch/x86/kvm/vmx/main.c                       |  12 +-
 arch/x86/kvm/vmx/tdx.c                        | 236 +++++++++++-
 arch/x86/kvm/vmx/x86_ops.h                    |   1 +
 arch/x86/kvm/x86.c                            |  14 +-
 tools/testing/selftests/kvm/Makefile.kvm      |   2 +
 .../testing/selftests/kvm/include/kvm_util.h  |  25 ++
 .../selftests/kvm/include/x86/tdx/tdx_util.h  |   3 +
 .../selftests/kvm/include/x86/tdx/test_util.h |   5 +
 .../testing/selftests/kvm/include/x86/ucall.h |   4 +-
 tools/testing/selftests/kvm/lib/kvm_util.c    | 222 ++++++++----
 .../testing/selftests/kvm/lib/ucall_common.c  |   2 +-
 .../selftests/kvm/lib/x86/tdx/tdx_util.c      |  63 +++-
 .../selftests/kvm/lib/x86/tdx/test_util.c     |  17 +
 tools/testing/selftests/kvm/lib/x86/ucall.c   | 108 ++++--
 .../kvm/x86/tdx_irqfd_migrate_test.c          | 264 ++++++++++++++
 .../selftests/kvm/x86/tdx_migrate_tests.c     | 337 ++++++++++++++++++
 22 files changed, 1349 insertions(+), 132 deletions(-)
 create mode 100644 tools/testing/selftests/kvm/x86/tdx_irqfd_migrate_test.c
 create mode 100644 tools/testing/selftests/kvm/x86/tdx_migrate_tests.c
Signature · 1 line
2.50.0.rc1.591.g9c95f17f64-goog
#1 lore raw

RFC PATCH v2 1/10 KVM: Split tdp_mmu_pages to mirror and direct counters

From: Sagi Shahar <[email protected]>

tdp_mmu_pages counts all the active pages used by the mmu. When we transfer the state during intra-host migration we need to transfer the mirror pages but not the direct ones. The direct pages are going to be re-faulted as needed on the destination, but that approach doesn't work for mirrored pages which stores information in the secure EPT.

Keeping them in separate counters makes this transfer more efficient.

---
 arch/x86/include/asm/kvm_host.h |  7 +++++--
 arch/x86/kvm/mmu/tdp_mmu.c      | 11 +++++++++--
 2 files changed, 14 insertions(+), 4 deletions(-)
arch/x86/include/asm/kvm_host.h +5 -2
index 999872c13722..b9966394acda 100644
@@ -1484,10 +1484,13 @@ struct kvm_arch {
 #ifdef CONFIG_X86_64
 #ifdef CONFIG_KVM_PROVE_MMU
 	/*
-	 * The number of TDP MMU pages across all roots.  Used only to sanity
-	 * check that KVM isn't leaking TDP MMU pages.
+	 * The number of non-mirrored TDP MMU pages across all roots.
+	 * Used only to sanity check that KVM isn't leaking TDP MMU pages.
 	 */
 	atomic64_t tdp_mmu_pages;
+
+	/* Same as tdp_mmu_pages but only for mirror pages. */
+	atomic64_t tdp_mirror_mmu_pages;
 #endif
 
 	/*
arch/x86/kvm/mmu/tdp_mmu.c +9 -2
index 7f3d7229b2c1..115af5e4c5ed 100644
@@ -42,6 +42,7 @@ void kvm_mmu_uninit_tdp_mmu(struct kvm *kvm)
 
 #ifdef CONFIG_KVM_PROVE_MMU
 	KVM_MMU_WARN_ON(atomic64_read(&kvm->arch.tdp_mmu_pages));
+	KVM_MMU_WARN_ON(atomic64_read(&kvm->arch.tdp_mirror_mmu_pages));
 #endif
 	WARN_ON(!list_empty(&kvm->arch.tdp_mmu_roots));
 
@@ -328,7 +329,10 @@ static void tdp_account_mmu_page(struct kvm *kvm, struct kvm_mmu_page *sp)
 {
 	kvm_account_pgtable_pages((void *)sp->spt, +1);
 #ifdef CONFIG_KVM_PROVE_MMU
-	atomic64_inc(&kvm->arch.tdp_mmu_pages);
+	if (sp->role.is_mirror)
+		atomic64_inc(&kvm->arch.tdp_mirror_mmu_pages);
+	else
+		atomic64_inc(&kvm->arch.tdp_mmu_pages);
 #endif
 }
 
@@ -336,7 +340,10 @@ static void tdp_unaccount_mmu_page(struct kvm *kvm, struct kvm_mmu_page *sp)
 {
 	kvm_account_pgtable_pages((void *)sp->spt, -1);
 #ifdef CONFIG_KVM_PROVE_MMU
-	atomic64_dec(&kvm->arch.tdp_mmu_pages);
+	if (sp->role.is_mirror)
+		atomic64_dec(&kvm->arch.tdp_mirror_mmu_pages);
+	else
+		atomic64_dec(&kvm->arch.tdp_mmu_pages);
 #endif
 }
 
Signature · 1 line
2.50.0.rc1.591.g9c95f17f64-goog
#2 lore raw

RFC PATCH v2 2/10 KVM: x86: Adjust locking order in move_enc_context_from

In reply to Ryan Afranji

Previously, the order for acquiring the locks required for the migration function move_enc_context_from() was: 1) memslot lock 2) vCPU lock. This can trigger a deadlock warning because a vCPU IOCTL modifying memslots will acquire the locks in reverse order: 1) vCPU lock 2) memslot lock.

This patch adjusts move_enc_context_from() to match vCPU IOCTL’s locking order to prevent deadlock warnings.

---
 arch/x86/kvm/svm/sev.c | 13 +------------
 arch/x86/kvm/x86.c     | 14 +++++++++++++-
 2 files changed, 14 insertions(+), 13 deletions(-)
arch/x86/kvm/svm/sev.c +1 -12
index 402543994b0b..380d5951f8dd 100644
@@ -1961,26 +1961,15 @@ int sev_vm_move_enc_context_from(struct kvm *kvm, struct kvm *source_kvm)
 		charged = true;
 	}
 
-	ret = kvm_lock_all_vcpus(kvm);
-	if (ret)
-		goto out_dst_cgroup;
-	ret = kvm_lock_all_vcpus(source_kvm);
-	if (ret)
-		goto out_dst_vcpu;
-
 	ret = sev_check_source_vcpus(kvm, source_kvm);
 	if (ret)
-		goto out_source_vcpu;
+		goto out_dst_cgroup;
 
 	sev_migrate_from(kvm, source_kvm);
 	kvm_vm_dead(source_kvm);
 	cg_cleanup_sev = src_sev;
 	ret = 0;
 
-out_source_vcpu:
-	kvm_unlock_all_vcpus(source_kvm);
-out_dst_vcpu:
-	kvm_unlock_all_vcpus(kvm);
 out_dst_cgroup:
 	/* Operates on the source on success, on the destination on failure.  */
 	if (charged)
arch/x86/kvm/x86.c +13 -1
index b1672379a16b..c28fa28a8e42 100644
@@ -6743,10 +6743,18 @@ static int kvm_vm_move_enc_context_from(struct kvm *kvm, unsigned int source_fd)
 	if (r)
 		goto out_mark_migration_done;
 
-	r = kvm_lock_vm_memslots(kvm, source_kvm);
+	r = kvm_lock_all_vcpus(kvm);
 	if (r)
 		goto out_unlock;
 
+	r = kvm_lock_all_vcpus(source_kvm);
+	if (r)
+		goto out_unlock_vcpus;
+
+	r = kvm_lock_vm_memslots(kvm, source_kvm);
+	if (r)
+		goto out_unlock_source_vcpus;
+
 	r = kvm_move_memory_ctxt_from(kvm, source_kvm);
 	if (r)
 		goto out_unlock_memslots;
@@ -6762,6 +6770,10 @@ static int kvm_vm_move_enc_context_from(struct kvm *kvm, unsigned int source_fd)
 
 out_unlock_memslots:
 	kvm_unlock_vm_memslots(kvm, source_kvm);
+out_unlock_source_vcpus:
+	kvm_unlock_all_vcpus(source_kvm);
+out_unlock_vcpus:
+	kvm_unlock_all_vcpus(kvm);
 out_unlock:
 	kvm_unlock_two_vms(kvm, source_kvm);
 out_mark_migration_done:
Signature · 1 line
2.50.0.rc1.591.g9c95f17f64-goog
#3 lore raw

RFC PATCH v2 3/10 KVM: TDX: Add base implementation for tdx_vm_move_enc_context_from

In reply to Ryan Afranji

From: Sagi Shahar <[email protected]>

This should mostly match the logic in sev_vm_move_enc_context_from.

---
 arch/x86/kvm/vmx/main.c    | 12 +++++++++++-
 arch/x86/kvm/vmx/tdx.c     | 24 ++++++++++++++++++++++++
 arch/x86/kvm/vmx/x86_ops.h |  1 +
 3 files changed, 36 insertions(+), 1 deletion(-)
arch/x86/kvm/vmx/main.c +11 -1
index d1e02e567b57..125af25fd09a 100644
@@ -879,6 +879,14 @@ static int vt_gmem_private_max_mapping_level(struct kvm *kvm, kvm_pfn_t pfn)
 	return 0;
 }
 
+static int vt_move_enc_context_from(struct kvm *kvm, struct kvm *source_kvm)
+{
+	if (!is_td(kvm))
+		return -ENOTTY;
+
+	return tdx_vm_move_enc_context_from(kvm, source_kvm);
+}
+
 #define vt_op(name) vt_##name
 #define vt_op_tdx_only(name) vt_##name
 #else /* CONFIG_KVM_INTEL_TDX */
@@ -1044,7 +1052,9 @@ struct kvm_x86_ops vt_x86_ops __initdata = {
 	.mem_enc_ioctl = vt_op_tdx_only(mem_enc_ioctl),
 	.vcpu_mem_enc_ioctl = vt_op_tdx_only(vcpu_mem_enc_ioctl),
 
-	.private_max_mapping_level = vt_op_tdx_only(gmem_private_max_mapping_level)
+	.private_max_mapping_level = vt_op_tdx_only(gmem_private_max_mapping_level),
+
+	.vm_move_enc_context_from = vt_move_enc_context_from
 };
 
 struct kvm_x86_init_ops vt_init_ops __initdata = {
arch/x86/kvm/vmx/tdx.c +24 -0
index b952bc673271..07583a11d6e3 100644
@@ -626,6 +626,7 @@ int tdx_vm_init(struct kvm *kvm)
 	kvm->arch.has_protected_state = true;
 	kvm->arch.has_private_mem = true;
 	kvm->arch.disabled_quirks |= KVM_X86_QUIRK_IGNORE_GUEST_PAT;
+	kvm->arch.use_vm_enc_ctxt_op = true;
 
 	/*
 	 * Because guest TD is protected, VMM can't parse the instruction in TD.
@@ -3524,3 +3525,26 @@ int __init tdx_bringup(void)
 	enable_tdx = 0;
 	return 0;
 }
+
+static __always_inline bool tdx_finalized(struct kvm *kvm)
+{
+	struct kvm_tdx *tdx_kvm = to_kvm_tdx(kvm);
+
+	return tdx_kvm->state == TD_STATE_RUNNABLE;
+}
+
+static int tdx_migrate_from(struct kvm *dst, struct kvm *src)
+{
+	return -EINVAL;
+}
+
+int tdx_vm_move_enc_context_from(struct kvm *kvm, struct kvm *src_kvm)
+{
+	if (!is_td(kvm) || !is_td(src_kvm))
+		return -EINVAL;
+
+	if (tdx_finalized(kvm) || !tdx_finalized(src_kvm))
+		return -EINVAL;
+
+	return tdx_migrate_from(kvm, src_kvm);
+}
arch/x86/kvm/vmx/x86_ops.h +1 -0
index b4596f651232..001f1540a560 100644
@@ -164,6 +164,7 @@ void tdx_flush_tlb_current(struct kvm_vcpu *vcpu);
 void tdx_flush_tlb_all(struct kvm_vcpu *vcpu);
 void tdx_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa, int root_level);
 int tdx_gmem_private_max_mapping_level(struct kvm *kvm, kvm_pfn_t pfn);
+int tdx_vm_move_enc_context_from(struct kvm *kvm, struct kvm *source_kvm);
 #endif
 
 #endif /* __KVM_X86_VMX_X86_OPS_H */
Signature · 1 line
2.50.0.rc1.591.g9c95f17f64-goog
#4 lore raw

RFC PATCH v2 4/10 KVM: TDX: Implement moving mirror pages between 2 TDs

In reply to Ryan Afranji

From: Sagi Shahar <[email protected]>

Added functionality for moving the mirror EPT table from one TD to a new one.

This function moves the root of the mirror EPT table and overwrites the root of the destination.

---
 arch/x86/kvm/mmu.h         |  2 ++
 arch/x86/kvm/mmu/mmu.c     | 66 ++++++++++++++++++++++++++++++++++++++
 arch/x86/kvm/mmu/tdp_mmu.c | 61 ++++++++++++++++++++++++++++++++---
 arch/x86/kvm/mmu/tdp_mmu.h |  6 ++++
 4 files changed, 130 insertions(+), 5 deletions(-)
arch/x86/kvm/mmu.h +2 -0
index b4b6860ab971..b43d770daa05 100644
@@ -102,6 +102,8 @@ void kvm_mmu_sync_roots(struct kvm_vcpu *vcpu);
 void kvm_mmu_sync_prev_roots(struct kvm_vcpu *vcpu);
 void kvm_mmu_track_write(struct kvm_vcpu *vcpu, gpa_t gpa, const u8 *new,
 			 int bytes);
+int kvm_mmu_move_mirror_pages_from(struct kvm_vcpu *vcpu,
+				   struct kvm_vcpu *src_vcpu);
 
 static inline int kvm_mmu_reload(struct kvm_vcpu *vcpu)
 {
arch/x86/kvm/mmu/mmu.c +66 -0

72 lines not on this page · raw message

arch/x86/kvm/mmu/tdp_mmu.c +56 -5

79 lines not on this page · raw message

arch/x86/kvm/mmu/tdp_mmu.h +6 -0
index 52acf99d40a0..abb1a84d8b1c 100644
@@ -63,6 +63,12 @@ static inline struct kvm_mmu_page *tdp_mmu_get_root(struct kvm_vcpu *vcpu,
 	return root_to_sp(vcpu->arch.mmu->root.hpa);
 }
 
+struct kvm_mmu_page *
+kvm_tdp_mmu_get_vcpu_root(struct kvm_vcpu *vcpu,
+			  union kvm_mmu_page_role role);
+hpa_t kvm_tdp_mmu_move_mirror_pages_from(struct kvm_vcpu *vcpu,
+					  struct kvm_vcpu *src_vcpu);
+
 bool kvm_tdp_mmu_zap_leafs(struct kvm *kvm, gfn_t start, gfn_t end, bool flush);
 bool kvm_tdp_mmu_zap_sp(struct kvm *kvm, struct kvm_mmu_page *sp);
 void kvm_tdp_mmu_zap_all(struct kvm *kvm);
Signature · 1 line
2.50.0.rc1.591.g9c95f17f64-goog
#5 lore raw

RFC PATCH v2 5/10 KVM: TDX: Allow vCPUs to be created for migration

In reply to Ryan Afranji

During migration, vCPUs need to be created for an uninitialized VM.

This commit moves the TDX vCPU setup that requires an initialized VM out of tdx_vcpu_create() and into tdx_td_vcpu_init().

---
 arch/x86/kvm/vmx/tdx.c | 21 +++++++++++----------
 1 file changed, 11 insertions(+), 10 deletions(-)
arch/x86/kvm/vmx/tdx.c +11 -10
index 07583a11d6e3..4582f94175b7 100644
@@ -664,9 +664,6 @@ int tdx_vcpu_create(struct kvm_vcpu *vcpu)
 	struct kvm_tdx *kvm_tdx = to_kvm_tdx(vcpu->kvm);
 	struct vcpu_tdx *tdx = to_tdx(vcpu);
 
-	if (kvm_tdx->state != TD_STATE_INITIALIZED)
-		return -EIO;
-
 	/*
 	 * TDX module mandates APICv, which requires an in-kernel local APIC.
 	 * Disallow an in-kernel I/O APIC, because level-triggered interrupts
@@ -692,12 +689,6 @@ int tdx_vcpu_create(struct kvm_vcpu *vcpu)
 	vcpu->arch.tsc_scaling_ratio = kvm_tdx->tsc_multiplier;
 	vcpu->arch.l1_tsc_scaling_ratio = kvm_tdx->tsc_multiplier;
 
-	vcpu->arch.guest_state_protected =
-		!(to_kvm_tdx(vcpu->kvm)->attributes & TDX_TD_ATTR_DEBUG);
-
-	if ((kvm_tdx->xfam & XFEATURE_MASK_XTILE) == XFEATURE_MASK_XTILE)
-		vcpu->arch.xfd_no_write_intercept = true;
-
 	tdx->vt.pi_desc.nv = POSTED_INTR_VECTOR;
 	__pi_set_sn(&tdx->vt.pi_desc);
 
@@ -3003,8 +2994,9 @@ static int tdx_vcpu_get_cpuid(struct kvm_vcpu *vcpu, struct kvm_tdx_cmd *cmd)
 
 static int tdx_vcpu_init(struct kvm_vcpu *vcpu, struct kvm_tdx_cmd *cmd)
 {
-	u64 apic_base;
+	struct kvm_tdx *kvm_tdx = to_kvm_tdx(vcpu->kvm);
 	struct vcpu_tdx *tdx = to_tdx(vcpu);
+	u64 apic_base;
 	int ret;
 
 	if (cmd->flags)
@@ -3013,6 +3005,15 @@ static int tdx_vcpu_init(struct kvm_vcpu *vcpu, struct kvm_tdx_cmd *cmd)
 	if (tdx->state != VCPU_TD_STATE_UNINITIALIZED)
 		return -EINVAL;
 
+	if (kvm_tdx->state != TD_STATE_INITIALIZED)
+		return -EIO;
+
+	vcpu->arch.guest_state_protected = !(kvm_tdx->attributes &
+					     TDX_TD_ATTR_DEBUG);
+
+	if ((kvm_tdx->xfam & XFEATURE_MASK_XTILE) == XFEATURE_MASK_XTILE)
+		vcpu->arch.xfd_no_write_intercept = true;
+
 	/*
 	 * TDX requires X2APIC, userspace is responsible for configuring guest
 	 * CPUID accordingly.
Signature · 1 line
2.50.0.rc1.591.g9c95f17f64-goog
#6 lore raw

RFC PATCH v2 6/10 KVM: TDX: Add core logic for TDX intra-host migration

In reply to Ryan Afranji

From: Sagi Shahar <[email protected]>

Adds the core logic for transferring state between source and destination TDs during intra-host migration.

---
 arch/x86/kvm/vmx/tdx.c | 193 ++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 192 insertions(+), 1 deletion(-)
arch/x86/kvm/vmx/tdx.c +192 -1
index 4582f94175b7..268aca28d878 100644
@@ -3534,9 +3534,200 @@ static __always_inline bool tdx_finalized(struct kvm *kvm)
 	return tdx_kvm->state == TD_STATE_RUNNABLE;
 }
 
+#define MAX_APIC_VECTOR 256
+
+static int tdx_migrate_vcpus(struct kvm *dst, struct kvm *src)
+{
+	struct kvm_vcpu *src_vcpu;
+	struct kvm_tdx *dst_tdx;
+	unsigned long i;
+
+	dst_tdx = to_kvm_tdx(dst);
+
+	kvm_for_each_vcpu(i, src_vcpu, src)
+		tdx_flush_vp_on_cpu(src_vcpu);
+
+	/* Copy per-vCPU state. */
+	kvm_for_each_vcpu(i, src_vcpu, src) {
+		struct vcpu_tdx *dst_tdx_vcpu, *src_tdx_vcpu;
+		struct kvm_lapic_state src_lapic_state;
+		struct kvm_vcpu *dst_vcpu;
+		u64 apic_base;
+		u32 vector;
+		int ret;
+
+		src_tdx_vcpu = to_tdx(src_vcpu);
+		dst_vcpu = kvm_get_vcpu(dst, i);
+		dst_tdx_vcpu = to_tdx(dst_vcpu);
+
+		dst_vcpu->cpu = -1;
+
+		/* Destination vCPU initialization skipped so do it here. */
+		apic_base = APIC_DEFAULT_PHYS_BASE | LAPIC_MODE_X2APIC |
+			(kvm_vcpu_is_reset_bsp(dst_vcpu) ?
+			 MSR_IA32_APICBASE_BSP : 0);
+		if (kvm_apic_set_base(dst_vcpu, apic_base, true))
+			return -EINVAL;
+
+		/* Copy lapic state. */
+		ret = kvm_apic_get_state(src_vcpu, &src_lapic_state);
+		if (ret)
+			return -EINVAL;
+
+		ret = kvm_apic_set_state(dst_vcpu, &src_lapic_state);
+		if (ret)
+			return -EINVAL;
+
+		/*
+		 * pi_desc stores state of posted interrupts for VMs which are
+		 * processed by pcpu during VM entry/runtime. For
+		 * non-confidential VMs, this storage is synchronized to vcpu
+		 * state using set_lapic_state(sync_pir_to_virr).
+		 *
+		 * For TDX VMs, KVM doesn't have access to virtual lapic page,
+		 * so in order to preserve the interrupt state, copy over
+		 * pi_desc contents to destination VM during copyless migration.
+		 */
+		dst_tdx_vcpu->vt = src_tdx_vcpu->vt;
+		for (vector = 0; vector < MAX_APIC_VECTOR; vector++) {
+			if (pi_test_pir(vector, &src_tdx_vcpu->vt.pi_desc)) {
+				__vmx_deliver_posted_interrupt(
+						dst_vcpu,
+						&dst_tdx_vcpu->vt.pi_desc,
+						vector);
+			}
+		}
+
+		/* Copy non-TDX vCPU state. */
+		memcpy(dst_vcpu->arch.regs, src_vcpu->arch.regs,
+		       NR_VCPU_REGS * sizeof(src_vcpu->arch.regs[0]));
+
+		dst_vcpu->arch.regs_avail = src_vcpu->arch.regs_avail;
+		dst_vcpu->arch.regs_dirty = src_vcpu->arch.regs_dirty;
+		dst_vcpu->arch.tsc_offset = dst_tdx->tsc_offset;
+		dst_vcpu->arch.guest_state_protected =
+			src_vcpu->arch.guest_state_protected;
+		dst_vcpu->arch.xfd_no_write_intercept =
+			src_vcpu->arch.xfd_no_write_intercept;
+		dst_vcpu->arch.tsc_offset = dst_tdx->tsc_offset;
+
+		/* Copy TD structures. */
+		dst_tdx_vcpu->vp.tdvpr_page = src_tdx_vcpu->vp.tdvpr_page;
+		dst_tdx_vcpu->vp.tdcx_pages = src_tdx_vcpu->vp.tdcx_pages;
+
+		td_vmcs_write64(dst_tdx_vcpu, POSTED_INTR_DESC_ADDR,
+				__pa(&dst_tdx_vcpu->vt.pi_desc));
+
+		/* Copy current vCPU status. */
+		dst_tdx_vcpu->ext_exit_qualification =
+			src_tdx_vcpu->ext_exit_qualification;
+		dst_tdx_vcpu->exit_gpa = src_tdx_vcpu->exit_gpa;
+		dst_tdx_vcpu->vp_enter_args = src_tdx_vcpu->vp_enter_args;
+		dst_tdx_vcpu->vp_enter_ret = src_tdx_vcpu->vp_enter_ret;
+		dst_tdx_vcpu->guest_entered = src_tdx_vcpu->guest_entered;
+		dst_tdx_vcpu->map_gpa_next = src_tdx_vcpu->map_gpa_next;
+		dst_tdx_vcpu->map_gpa_end = src_tdx_vcpu->map_gpa_end;
+
+		/* Copy mirror EPT tables. */
+		vcpu_load(dst_vcpu);
+		if (kvm_mmu_move_mirror_pages_from(dst_vcpu, src_vcpu)) {
+			vcpu_put(dst_vcpu);
+			return -EINVAL;
+		}
+		vcpu_put(dst_vcpu);
+
+		dst_vcpu->arch.mp_state = KVM_MP_STATE_RUNNABLE;
+		dst_tdx_vcpu->state = VCPU_TD_STATE_INITIALIZED;
+
+		/*
+		 * Set these source's vCPU migrated structures to NULL to avoid
+		 * freeing them during source VM shutdown.
+		 */
+		src_tdx_vcpu->vp.tdvpr_page = NULL;
+		src_tdx_vcpu->vp.tdcx_pages = NULL;
+	}
+
+	return 0;
+}
+
 static int tdx_migrate_from(struct kvm *dst, struct kvm *src)
 {
-	return -EINVAL;
+	struct kvm_tdx *src_tdx, *dst_tdx;
+	bool charged = false;
+	int ret;
+
+	src_tdx = to_kvm_tdx(src);
+	dst_tdx = to_kvm_tdx(dst);
+
+	ret = -EINVAL;
+
+	if (src_tdx->state != TD_STATE_RUNNABLE) {
+		pr_warn("Cannot migrate from a non finalized VM\n");
+		goto abort;
+	}
+
+	/* Transfer miscellaneous cgroup. */
+	dst_tdx->misc_cg = get_current_misc_cg();
+	if (dst_tdx->misc_cg != src_tdx->misc_cg) {
+		ret = misc_cg_try_charge(MISC_CG_RES_TDX, dst_tdx->misc_cg, 1);
+		if (ret)
+			goto abort_dst_cgroup;
+		charged = true;
+	}
+
+	dst_tdx->hkid = src_tdx->hkid;
+
+	/* Copy VM data. */
+	dst_tdx->attributes = src_tdx->attributes;
+	dst_tdx->xfam = src_tdx->xfam;
+	dst_tdx->tsc_offset = src_tdx->tsc_offset;
+	dst_tdx->tsc_multiplier = src_tdx->tsc_multiplier;
+	dst_tdx->nr_premapped = src_tdx->nr_premapped;
+	dst_tdx->wait_for_sept_zap = src_tdx->wait_for_sept_zap;
+	dst_tdx->kvm.arch.gfn_direct_bits = src_tdx->kvm.arch.gfn_direct_bits;
+
+	/* Copy TD structures. */
+	dst_tdx->td.tdcs_nr_pages = src_tdx->td.tdcs_nr_pages;
+	dst_tdx->td.tdcx_nr_pages = src_tdx->td.tdcx_nr_pages;
+	dst_tdx->td.tdr_page = src_tdx->td.tdr_page;
+	dst_tdx->td.tdcs_pages = src_tdx->td.tdcs_pages;
+
+	/* Copy per-vCPU state. */
+	ret = tdx_migrate_vcpus(dst, src);
+	if (ret)
+		goto late_abort;
+
+	dst->mem_attr_array.xa_head = src->mem_attr_array.xa_head;
+	src->mem_attr_array.xa_head = NULL;
+
+	dst_tdx->state = TD_STATE_RUNNABLE;
+
+	/*
+	 * Set these source's vCPU migrated structures to NULL to avoid
+	 * freeing them during source VM shutdown.
+	 */
+	src_tdx->hkid = -1;
+	src_tdx->td.tdr_page = NULL;
+	src_tdx->td.tdcs_pages = NULL;
+
+	return 0;
+
+late_abort:
+	/*
+	 * If we aborted after the state transfer already started, the src VM
+	 * is no longer valid.
+	 */
+	kvm_vm_dead(src);
+
+abort_dst_cgroup:
+	if (charged)
+		misc_cg_uncharge(MISC_CG_RES_TDX, dst_tdx->misc_cg, 1);
+	put_misc_cg(dst_tdx->misc_cg);
+	dst_tdx->misc_cg = NULL;
+abort:
+	dst_tdx->hkid = -1;
+	dst_tdx->td.tdr_page = 0;
+	return ret;
 }
 
 int tdx_vm_move_enc_context_from(struct kvm *kvm, struct kvm *src_kvm)
Signature · 1 line
2.50.0.rc1.591.g9c95f17f64-goog
#7 lore raw

RFC PATCH v2 7/10 KVM: selftests: Refactor userspace_mem_region creation out of vm_mem_add

In reply to Ryan Afranji

Refactor the creation and committing of userspace_mem_region to their own functions so that they can reused by future TDX migration functions.

---
 tools/testing/selftests/kvm/lib/kvm_util.c | 147 +++++++++++++--------
 1 file changed, 89 insertions(+), 58 deletions(-)
tools/testing/selftests/kvm/lib/kvm_util.c +89 -58
index 2b442639ee2d..3c131718b81a 100644
@@ -974,50 +974,47 @@ void vm_set_user_memory_region2(struct kvm_vm *vm, uint32_t slot, uint32_t flags
 		    errno, strerror(errno));
 }
 
-
-/* FIXME: This thing needs to be ripped apart and rewritten. */
-void vm_mem_add(struct kvm_vm *vm, enum vm_mem_backing_src_type src_type,
-		uint64_t guest_paddr, uint32_t slot, uint64_t npages,
-		uint32_t flags, int guest_memfd, uint64_t guest_memfd_offset)
+static struct userspace_mem_region *vm_mem_region_alloc(struct kvm_vm *vm,
+								uint64_t guest_paddr,
+								uint32_t slot,
+								size_t npages,
+								uint32_t flags)
 {
-	int ret;
 	struct userspace_mem_region *region;
-	size_t backing_src_pagesz = get_backing_src_pagesz(src_type);
-	size_t mem_size = npages * vm->page_size;
-	size_t alignment;
 
 	TEST_REQUIRE_SET_USER_MEMORY_REGION2();
 
 	TEST_ASSERT(vm_adjust_num_guest_pages(vm->mode, npages) == npages,
-		"Number of guest pages is not compatible with the host. "
-		"Try npages=%d", vm_adjust_num_guest_pages(vm->mode, npages));
+		    "Number of guest pages is not compatible with the host. "
+		    "Try npages=%d", vm_adjust_num_guest_pages(vm->mode, npages));
 
 	TEST_ASSERT((guest_paddr % vm->page_size) == 0, "Guest physical "
-		"address not on a page boundary.\n"
-		"  guest_paddr: 0x%lx vm->page_size: 0x%x",
-		guest_paddr, vm->page_size);
+		    "address not on a page boundary.\n"
+		    "  guest_paddr: 0x%lx vm->page_size: 0x%x",
+		    guest_paddr, vm->page_size);
 	TEST_ASSERT((((guest_paddr >> vm->page_shift) + npages) - 1)
-		<= vm->max_gfn, "Physical range beyond maximum "
-		"supported physical address,\n"
-		"  guest_paddr: 0x%lx npages: 0x%lx\n"
-		"  vm->max_gfn: 0x%lx vm->page_size: 0x%x",
-		guest_paddr, npages, vm->max_gfn, vm->page_size);
+		    <= vm->max_gfn, "Physical range beyond maximum "
+		    "supported physical address,\n"
+		    "  guest_paddr: 0x%lx npages: 0x%lx\n"
+		    "  vm->max_gfn: 0x%lx vm->page_size: 0x%x",
+		    guest_paddr, npages, vm->max_gfn, vm->page_size);
 
 	/*
 	 * Confirm a mem region with an overlapping address doesn't
 	 * already exist.
 	 */
 	region = (struct userspace_mem_region *) userspace_mem_region_find(
-		vm, guest_paddr, (guest_paddr + npages * vm->page_size) - 1);
+			vm, guest_paddr,
+			(guest_paddr + npages * vm->page_size) - 1);
 	if (region != NULL)
 		TEST_FAIL("overlapping userspace_mem_region already "
-			"exists\n"
-			"  requested guest_paddr: 0x%lx npages: 0x%lx "
-			"page_size: 0x%x\n"
-			"  existing guest_paddr: 0x%lx size: 0x%lx",
-			guest_paddr, npages, vm->page_size,
-			(uint64_t) region->region.guest_phys_addr,
-			(uint64_t) region->region.memory_size);
+			  "exists\n"
+			  "  requested guest_paddr: 0x%lx npages: 0x%lx "
+			  "page_size: 0x%x\n"
+			  "  existing guest_paddr: 0x%lx size: 0x%lx",
+			  guest_paddr, npages, vm->page_size,
+			  (uint64_t) region->region.guest_phys_addr,
+			  (uint64_t) region->region.memory_size);
 
 	/* Confirm no region with the requested slot already exists. */
 	hash_for_each_possible(vm->regions.slot_hash, region, slot_node,
@@ -1026,19 +1023,73 @@ void vm_mem_add(struct kvm_vm *vm, enum vm_mem_backing_src_type src_type,
 			continue;
 
 		TEST_FAIL("A mem region with the requested slot "
-			"already exists.\n"
-			"  requested slot: %u paddr: 0x%lx npages: 0x%lx\n"
-			"  existing slot: %u paddr: 0x%lx size: 0x%lx",
-			slot, guest_paddr, npages,
-			region->region.slot,
-			(uint64_t) region->region.guest_phys_addr,
-			(uint64_t) region->region.memory_size);
+			  "already exists.\n"
+			  "  requested slot: %u paddr: 0x%lx npages: 0x%lx\n"
+			  "  existing slot: %u paddr: 0x%lx size: 0x%lx",
+			  slot, guest_paddr, npages,
+			  region->region.slot,
+			  (uint64_t) region->region.guest_phys_addr,
+			  (uint64_t) region->region.memory_size);
 	}
 
 	/* Allocate and initialize new mem region structure. */
 	region = calloc(1, sizeof(*region));
 	TEST_ASSERT(region != NULL, "Insufficient Memory");
-	region->mmap_size = mem_size;
+
+	region->unused_phy_pages = sparsebit_alloc();
+	if (vm_arch_has_protected_memory(vm))
+		region->protected_phy_pages = sparsebit_alloc();
+	sparsebit_set_num(region->unused_phy_pages,
+			  guest_paddr >> vm->page_shift, npages);
+	region->region.slot = slot;
+	region->region.flags = flags;
+	region->region.guest_phys_addr = guest_paddr;
+	region->region.memory_size = npages * vm->page_size;
+
+	region->mmap_start = NULL;
+	region->mmap_size = 0;
+	region->host_mem = NULL;
+	region->fd = -1;
+
+	return region;
+}
+
+static void userspace_mem_region_commit(struct kvm_vm *vm,
+					struct userspace_mem_region *region)
+{
+	int ret;
+
+	region->region.userspace_addr = (uintptr_t) region->host_mem;
+	ret = __vm_ioctl(vm, KVM_SET_USER_MEMORY_REGION2, &region->region);
+	TEST_ASSERT(ret == 0, "KVM_SET_USER_MEMORY_REGION2 IOCTL failed,\n"
+		    "  rc: %i errno: %i\n"
+		    "  slot: %u flags: 0x%x\n"
+		    "  guest_phys_addr: 0x%lx size: 0x%lx guest_memfd: %d",
+		    ret, errno, region->region.slot, region->region.flags,
+		    (uint64_t) region->region.guest_phys_addr,
+		    (uint64_t) region->region.memory_size,
+		    region->region.guest_memfd);
+
+	/* Add to quick lookup data structures */
+	vm_userspace_mem_region_gpa_insert(&vm->regions.gpa_tree, region);
+	vm_userspace_mem_region_hva_insert(&vm->regions.hva_tree, region);
+	hash_add(vm->regions.slot_hash, &region->slot_node,
+		 region->region.slot);
+}
+
+/* FIXME: This thing needs to be ripped apart and rewritten. */
+void vm_mem_add(struct kvm_vm *vm, enum vm_mem_backing_src_type src_type,
+		uint64_t guest_paddr, uint32_t slot, uint64_t npages,
+		uint32_t flags, int guest_memfd, uint64_t guest_memfd_offset)
+{
+	int ret;
+	struct userspace_mem_region *region;
+	size_t backing_src_pagesz = get_backing_src_pagesz(src_type);
+	size_t mem_size = npages * vm->page_size;
+	size_t alignment;
+
+	region = vm_mem_region_alloc(vm, guest_paddr, slot, npages,
+					     flags);
 
 #ifdef __s390x__
 	/* On s390x, the host address must be aligned to 1M (due to PGSTEs) */
@@ -1058,6 +1109,8 @@ void vm_mem_add(struct kvm_vm *vm, enum vm_mem_backing_src_type src_type,
 
 	TEST_ASSERT_EQ(guest_paddr, align_up(guest_paddr, backing_src_pagesz));
 
+	region->mmap_size = mem_size;
+
 	/* Add enough memory to align up if necessary */
 	if (alignment > 1)
 		region->mmap_size += alignment;
@@ -1117,29 +1170,7 @@ void vm_mem_add(struct kvm_vm *vm, enum vm_mem_backing_src_type src_type,
 		region->region.guest_memfd = -1;
 	}
 
-	region->unused_phy_pages = sparsebit_alloc();
-	if (vm_arch_has_protected_memory(vm))
-		region->protected_phy_pages = sparsebit_alloc();
-	sparsebit_set_num(region->unused_phy_pages,
-		guest_paddr >> vm->page_shift, npages);
-	region->region.slot = slot;
-	region->region.flags = flags;
-	region->region.guest_phys_addr = guest_paddr;
-	region->region.memory_size = npages * vm->page_size;
-	region->region.userspace_addr = (uintptr_t) region->host_mem;
-	ret = __vm_ioctl(vm, KVM_SET_USER_MEMORY_REGION2, &region->region);
-	TEST_ASSERT(ret == 0, "KVM_SET_USER_MEMORY_REGION2 IOCTL failed,\n"
-		"  rc: %i errno: %i\n"
-		"  slot: %u flags: 0x%x\n"
-		"  guest_phys_addr: 0x%lx size: 0x%lx guest_memfd: %d",
-		ret, errno, slot, flags,
-		guest_paddr, (uint64_t) region->region.memory_size,
-		region->region.guest_memfd);
-
-	/* Add to quick lookup data structures */
-	vm_userspace_mem_region_gpa_insert(&vm->regions.gpa_tree, region);
-	vm_userspace_mem_region_hva_insert(&vm->regions.hva_tree, region);
-	hash_add(vm->regions.slot_hash, &region->slot_node, slot);
+	userspace_mem_region_commit(vm, region);
 
 	/* If shared memory, create an alias. */
 	if (region->fd >= 0) {
Signature · 1 line
2.50.0.rc1.591.g9c95f17f64-goog
#8 lore raw

RFC PATCH v2 8/10 KVM: selftests: TDX: Add tests for TDX in-place migration

In reply to Ryan Afranji

From: Sagi Shahar <[email protected]>

Adds selftests for TDX in-place migration.

---
 tools/testing/selftests/kvm/Makefile.kvm      |   1 +
 .../testing/selftests/kvm/include/kvm_util.h  |  20 +
 .../selftests/kvm/include/x86/tdx/tdx_util.h  |   1 +
 tools/testing/selftests/kvm/lib/kvm_util.c    |  50 ++-
 .../selftests/kvm/lib/x86/tdx/tdx_util.c      |   3 +-
 .../selftests/kvm/x86/tdx_migrate_tests.c     | 358 ++++++++++++++++++
 6 files changed, 429 insertions(+), 4 deletions(-)
 create mode 100644 tools/testing/selftests/kvm/x86/tdx_migrate_tests.c
tools/testing/selftests/kvm/Makefile.kvm +1 -0
index 1c7ea61e9031..d4c8cfb5910f 100644
@@ -155,6 +155,7 @@ TEST_GEN_PROGS_x86 += pre_fault_memory_test
 TEST_GEN_PROGS_x86 += x86/tdx_vm_test
 TEST_GEN_PROGS_x86 += x86/tdx_shared_mem_test
 TEST_GEN_PROGS_x86 += x86/tdx_upm_test
+TEST_GEN_PROGS_x86 += x86/tdx_migrate_tests
 
 # Compiled outputs used by test targets
 TEST_GEN_PROGS_EXTENDED_x86 += x86/nx_huge_pages_test
tools/testing/selftests/kvm/include/kvm_util.h +20 -0
index 267f78f3f16f..1b6489081e74 100644
@@ -110,6 +110,9 @@ struct kvm_vm {
 
 	struct kvm_binary_stats stats;
 
+	/* VM was migrated using KVM_CAP_VM_MOVE_ENC_CONTEXT_FROM */
+	bool enc_migrated;
+
 	/*
 	 * KVM region slots. These are the default memslots used by page
 	 * allocators, e.g., lib/elf uses the memslots[MEM_REGION_CODE]
@@ -673,6 +676,7 @@ static inline bool vm_arch_has_protected_memory(struct kvm_vm *vm)
 
 void vm_mem_region_set_flags(struct kvm_vm *vm, uint32_t slot, uint32_t flags);
 void vm_mem_region_move(struct kvm_vm *vm, uint32_t slot, uint64_t new_gpa);
+void vm_migrate_mem_regions(struct kvm_vm *dst_vm, struct kvm_vm *src_vm);
 void vm_mem_region_delete(struct kvm_vm *vm, uint32_t slot);
 struct kvm_vcpu *__vm_vcpu_add(struct kvm_vm *vm, uint32_t vcpu_id);
 void vm_populate_vaddr_bitmap(struct kvm_vm *vm);
@@ -1132,6 +1136,22 @@ static inline struct kvm_vcpu *vm_vcpu_add(struct kvm_vm *vm, uint32_t vcpu_id,
 	return vcpu;
 }
 
+/*
+ * Adds a vCPU with no defaults. This vcpu will be used for migration
+ *
+ * Input Args:
+ *   vm - Virtual Machine
+ *   vcpu_id - The id of the VCPU to add to the VM.
+ */
+struct kvm_vcpu *vm_arch_vcpu_add_for_migration(struct kvm_vm *vm,
+						uint32_t vcpu_id);
+
+static inline struct kvm_vcpu *vm_vcpu_add_for_migration(struct kvm_vm *vm,
+							 uint32_t vcpu_id)
+{
+	return vm_arch_vcpu_add_for_migration(vm, vcpu_id);
+}
+
 /* Re-create a vCPU after restarting a VM, e.g. for state save/restore tests. */
 struct kvm_vcpu *vm_arch_vcpu_recreate(struct kvm_vm *vm, uint32_t vcpu_id);
 
tools/testing/selftests/kvm/include/x86/tdx/tdx_util.h +1 -0
index ae39b78aa4af..9b495e621225 100644
@@ -9,6 +9,7 @@
 extern uint64_t tdx_s_bit;
 void tdx_filter_cpuid(struct kvm_vm *vm, struct kvm_cpuid2 *cpuid_data);
 void __tdx_mask_cpuid_features(struct kvm_cpuid_entry2 *entry);
+void tdx_enable_capabilities(struct kvm_vm *vm);
 
 struct kvm_vcpu *td_vcpu_add(struct kvm_vm *vm, uint32_t vcpu_id, void *guest_code);
 
tools/testing/selftests/kvm/lib/kvm_util.c +48 -2

62 lines not on this page · raw message

tools/testing/selftests/kvm/lib/x86/tdx/tdx_util.c +1 -2
index c5bee67099c5..ef03d42f58d0 100644
@@ -344,7 +344,7 @@ static void register_encrypted_memory_region(struct kvm_vm *vm,
  * TD creation/setup/finalization
  */
 
-static void tdx_enable_capabilities(struct kvm_vm *vm)
+void tdx_enable_capabilities(struct kvm_vm *vm)
 {
 	int rc;
 
@@ -574,7 +574,6 @@ void td_initialize(struct kvm_vm *vm, enum vm_mem_backing_src_type src_type,
 	uint64_t nr_pages_required;
 
 	tdx_enable_capabilities(vm);
-
 	tdx_td_init(vm, attributes);
 
 	nr_pages_required = vm_nr_pages_required(VM_MODE_DEFAULT, 1, 0);
tools/testing/selftests/kvm/x86/tdx_migrate_tests.c new file +358 -0

358 lines not on this page · raw message

Signature · 1 line
2.50.0.rc1.591.g9c95f17f64-goog
#9 lore raw

RFC PATCH v2 9/10 KVM: selftests: Add TDX support for ucalls

In reply to Ryan Afranji

From: Ackerley Tng <[email protected]>

ucalls for non-Coco VMs work by having the guest write to the rdi register, then perform an io instruction to exit to the host. The host then reads rdi using kvm_get_regs().

CPU registers can't be read using kvm_get_regs() for TDX, so TDX guests use MMIO to pass the struct ucall's hva to the host. MMIO was chosen because it is one of the simplest (hence unlikely to fail) mechanisms that support passing 8 bytes from guest to host.

A new kvm_mem_region_type, MEM_REGION_UCALL, is added so TDX VMs can set up a different memslot for the ucall_pool that is set up as shared memory.

---
 .../testing/selftests/kvm/include/kvm_util.h  |   1 +
 .../testing/selftests/kvm/include/x86/ucall.h |   4 +-
 .../testing/selftests/kvm/lib/ucall_common.c  |   2 +-
 .../selftests/kvm/lib/x86/tdx/tdx_util.c      |  40 +++++++
 tools/testing/selftests/kvm/lib/x86/ucall.c   | 108 ++++++++++++------
 5 files changed, 118 insertions(+), 37 deletions(-)
tools/testing/selftests/kvm/include/kvm_util.h +1 -0
index 1b6489081e74..8b252a668c78 100644
@@ -80,6 +80,7 @@ enum kvm_mem_region_type {
 	MEM_REGION_PT,
 	MEM_REGION_TEST_DATA,
 	MEM_REGION_TDX_BOOT_PARAMS,
+	MEM_REGION_UCALL,
 	NR_MEM_REGIONS,
 };
 
tools/testing/selftests/kvm/include/x86/ucall.h +1 -3
index d3825dcc3cd9..0494a4a21557 100644
@@ -6,8 +6,6 @@
 
 #define UCALL_EXIT_REASON       KVM_EXIT_IO
 
-static inline void ucall_arch_init(struct kvm_vm *vm, vm_paddr_t mmio_gpa)
-{
-}
+void ucall_arch_init(struct kvm_vm *vm, vm_paddr_t mmio_gpa);
 
 #endif
tools/testing/selftests/kvm/lib/ucall_common.c +1 -1
index 42151e571953..5f195d4d15dc 100644
@@ -33,7 +33,7 @@ void ucall_init(struct kvm_vm *vm, vm_paddr_t mmio_gpa)
 	int i;
 
 	vaddr = vm_vaddr_alloc_shared(vm, sizeof(*hdr), KVM_UTIL_MIN_VADDR,
-				      MEM_REGION_DATA);
+				      MEM_REGION_UCALL);
 	hdr = (struct ucall_header *)addr_gva2hva(vm, vaddr);
 	memset(hdr, 0, sizeof(*hdr));
 
tools/testing/selftests/kvm/lib/x86/tdx/tdx_util.c +40 -0
index ef03d42f58d0..a3612bf187a0 100644
@@ -11,6 +11,7 @@
 #include "tdx/td_boot.h"
 #include "tdx/tdx.h"
 #include "test_util.h"
+#include "ucall_common.h"
 
 uint64_t tdx_s_bit;
 
@@ -568,6 +569,43 @@ static void td_setup_boot_parameters(struct kvm_vm *vm, enum vm_mem_backing_src_
 	TEST_ASSERT_EQ(addr, TD_BOOT_PARAMETERS_GPA);
 }
 
+/*
+ * GPA where ucall headers/pool will be set up
+ *
+ * TD_UCALL_POOL_GPA is arbitrarily chosen to
+ *
+ * + Be within the 4GB address space
+ * + Not clash with the other memslots for boot parameters, boot code and test
+ *   code
+ */
+#define TD_UCALL_POOL_GPA 0x30000000
+/*
+ * GPA to use for ucall MMIO writes
+ *
+ * TD_UCALL_MMIO_GPA is arbitrarily chosen to
+ *
+ * + Be within the 4GB address space
+ * + Not clash with the other memslots for boot parameters, boot code and test
+ *   code
+ * + Not be configured in any memslot (unconfigured GPAs are treated as
+ *   MMIOs). For now, TDX VMs can't be used with KVM_MEM_READONLY so using
+ *   readonly memslots won't work for TDX VMs.
+ */
+#define TD_UCALL_MMIO_GPA 0x40000000
+#define TD_UCALL_MEMSLOT  4
+
+static void td_setup_ucall(struct kvm_vm *vm)
+{
+	int npages;
+
+	npages = ucall_nr_pages_required(PAGE_SIZE);
+	vm_userspace_mem_region_add(vm, VM_MEM_SRC_ANONYMOUS, TD_UCALL_POOL_GPA,
+				    TD_UCALL_MEMSLOT, npages, 0);
+	vm->memslots[MEM_REGION_UCALL] = TD_UCALL_MEMSLOT;
+
+	ucall_init(vm, TD_UCALL_MMIO_GPA);
+}
+
 void td_initialize(struct kvm_vm *vm, enum vm_mem_backing_src_type src_type,
 		   uint64_t attributes)
 {
@@ -593,6 +631,8 @@ void td_initialize(struct kvm_vm *vm, enum vm_mem_backing_src_type src_type,
 
 	td_setup_boot_code(vm, src_type);
 	td_setup_boot_parameters(vm, src_type);
+
+	td_setup_ucall(vm);
 }
 
 void td_finalize(struct kvm_vm *vm)
tools/testing/selftests/kvm/lib/x86/ucall.c +75 -33

127 lines not on this page · raw message

Signature · 1 line
2.50.0.rc1.591.g9c95f17f64-goog
#10 lore raw

RFC PATCH v2 10/10 KVM: selftests: Add irqfd/interrupts test for TDX with migration

In reply to Ryan Afranji

From: Ackerley Tng <[email protected]>

Adds a selftest to verify interrupts sent to a TDX VM before migration are successfully handled by the migrated VM.

---
 tools/testing/selftests/kvm/Makefile.kvm      |   1 +
 .../testing/selftests/kvm/include/kvm_util.h  |   4 +
 .../selftests/kvm/include/x86/tdx/tdx_util.h  |   2 +
 .../selftests/kvm/include/x86/tdx/test_util.h |   5 +
 tools/testing/selftests/kvm/lib/kvm_util.c    |  35 ++-
 .../selftests/kvm/lib/x86/tdx/tdx_util.c      |  20 ++
 .../selftests/kvm/lib/x86/tdx/test_util.c     |  17 ++
 .../kvm/x86/tdx_irqfd_migrate_test.c          | 264 ++++++++++++++++++
 .../selftests/kvm/x86/tdx_migrate_tests.c     |  21 --
 9 files changed, 343 insertions(+), 26 deletions(-)
 create mode 100644 tools/testing/selftests/kvm/x86/tdx_irqfd_migrate_test.c
tools/testing/selftests/kvm/Makefile.kvm +1 -0
index d4c8cfb5910f..4ae0d105c2a7 100644
@@ -156,6 +156,7 @@ TEST_GEN_PROGS_x86 += x86/tdx_vm_test
 TEST_GEN_PROGS_x86 += x86/tdx_shared_mem_test
 TEST_GEN_PROGS_x86 += x86/tdx_upm_test
 TEST_GEN_PROGS_x86 += x86/tdx_migrate_tests
+TEST_GEN_PROGS_x86 += x86/tdx_irqfd_migrate_test
 
 # Compiled outputs used by test targets
 TEST_GEN_PROGS_EXTENDED_x86 += x86/nx_huge_pages_test
tools/testing/selftests/kvm/include/kvm_util.h +4 -0
index 8b252a668c78..f93ac2b9b0ff 100644
@@ -80,6 +80,7 @@ enum kvm_mem_region_type {
 	MEM_REGION_PT,
 	MEM_REGION_TEST_DATA,
 	MEM_REGION_TDX_BOOT_PARAMS,
+	MEM_REGION_TDX_SHARED_DATA,
 	MEM_REGION_UCALL,
 	NR_MEM_REGIONS,
 };
@@ -958,6 +959,9 @@ int _kvm_irq_line(struct kvm_vm *vm, uint32_t irq, int level);
 struct kvm_irq_routing *kvm_gsi_routing_create(void);
 void kvm_gsi_routing_irqchip_add(struct kvm_irq_routing *routing,
 		uint32_t gsi, uint32_t pin);
+void kvm_gsi_routing_msi_add(struct kvm_irq_routing *routing, uint32_t gsi,
+			     uint32_t address_lo, uint32_t address_hi,
+			     uint32_t data);
 int _kvm_gsi_routing_write(struct kvm_vm *vm, struct kvm_irq_routing *routing);
 void kvm_gsi_routing_write(struct kvm_vm *vm, struct kvm_irq_routing *routing);
 
tools/testing/selftests/kvm/include/x86/tdx/tdx_util.h +2 -0
index 9b495e621225..4393c8649718 100644
@@ -10,6 +10,8 @@ extern uint64_t tdx_s_bit;
 void tdx_filter_cpuid(struct kvm_vm *vm, struct kvm_cpuid2 *cpuid_data);
 void __tdx_mask_cpuid_features(struct kvm_cpuid_entry2 *entry);
 void tdx_enable_capabilities(struct kvm_vm *vm);
+int __tdx_migrate_from(int dst_fd, int src_fd);
+void tdx_migrate_from(struct kvm_vm *dst_vm, struct kvm_vm *src_vm);
 
 struct kvm_vcpu *td_vcpu_add(struct kvm_vm *vm, uint32_t vcpu_id, void *guest_code);
 
tools/testing/selftests/kvm/include/x86/tdx/test_util.h +5 -0
index 3330d5a54698..0dd859974cb3 100644
@@ -130,4 +130,9 @@ uint64_t tdx_test_read_64bit(struct kvm_vcpu *vcpu, uint64_t port);
  */
 uint64_t tdx_test_read_64bit_report_from_guest(struct kvm_vcpu *vcpu);
 
+/*
+ * Enables X2APIC for TDX guests.
+ */
+void tdx_guest_x2apic_enable(void);
+
 #endif // SELFTEST_TDX_TEST_UTIL_H
tools/testing/selftests/kvm/lib/kvm_util.c +30 -5
index 9dc3c7bf0443..bbb489635064 100644
@@ -1293,10 +1293,12 @@ static void vm_migrate_mem_region(struct kvm_vm *dst_vm, struct kvm_vm *src_vm,
 				  struct userspace_mem_region *src_region)
 {
 	struct userspace_mem_region *dst_region;
-	int dst_guest_memfd;
+	int dst_guest_memfd = -1;
 
-	dst_guest_memfd =
-		vm_link_guest_memfd(dst_vm, src_region->region.guest_memfd, 0);
+	if (src_region->region.guest_memfd != -1)
+		dst_guest_memfd = vm_link_guest_memfd(dst_vm,
+						      src_region->region.guest_memfd,
+						      0);
 
 	dst_region = vm_mem_region_alloc(
 			dst_vm, src_region->region.guest_phys_addr,
@@ -1312,8 +1314,12 @@ static void vm_migrate_mem_region(struct kvm_vm *dst_vm, struct kvm_vm *src_vm,
 	src_region->host_mem = 0;
 
 	dst_region->region.guest_memfd = dst_guest_memfd;
-	dst_region->region.guest_memfd_offset =
-		src_region->region.guest_memfd_offset;
+	if (src_region->region.guest_memfd == -1) {
+		dst_region->fd = src_region->fd;
+	} else {
+		dst_region->region.guest_memfd_offset =
+			src_region->region.guest_memfd_offset;
+	}
 
 	userspace_mem_region_commit(dst_vm, dst_region);
 }
@@ -2057,6 +2063,25 @@ void kvm_gsi_routing_irqchip_add(struct kvm_irq_routing *routing,
 	routing->nr++;
 }
 
+void kvm_gsi_routing_msi_add(struct kvm_irq_routing *routing, uint32_t gsi,
+			     uint32_t address_lo, uint32_t address_hi,
+			     uint32_t data)
+{
+	int i;
+
+	assert(routing);
+	assert(routing->nr < KVM_MAX_IRQ_ROUTES);
+
+	i = routing->nr;
+	routing->entries[i].gsi = gsi;
+	routing->entries[i].type = KVM_IRQ_ROUTING_MSI;
+	routing->entries[i].flags = 0;
+	routing->entries[i].u.msi.address_lo = address_lo;
+	routing->entries[i].u.msi.address_hi = address_hi;
+	routing->entries[i].u.msi.data = data;
+	routing->nr++;
+}
+
 int _kvm_gsi_routing_write(struct kvm_vm *vm, struct kvm_irq_routing *routing)
 {
 	int ret;
tools/testing/selftests/kvm/lib/x86/tdx/tdx_util.c +20 -0
index a3612bf187a0..8216a778474a 100644
@@ -372,6 +372,26 @@ static void tdx_apply_cr4_restrictions(struct kvm_sregs *sregs)
 	sregs->cr4 &= ~(X86_CR4_VMXE | X86_CR4_SMXE);
 }
 
+int __tdx_migrate_from(int dst_fd, int src_fd)
+{
+	struct kvm_enable_cap cap = {
+		.cap = KVM_CAP_VM_MOVE_ENC_CONTEXT_FROM,
+		.args = { src_fd }
+	};
+
+	return ioctl(dst_fd, KVM_ENABLE_CAP, &cap);
+}
+
+void tdx_migrate_from(struct kvm_vm *dst_vm, struct kvm_vm *src_vm)
+{
+	int ret;
+
+	vm_migrate_mem_regions(dst_vm, src_vm);
+	ret = __tdx_migrate_from(dst_vm->fd, src_vm->fd);
+	TEST_ASSERT(!ret, "Migration failed, ret: %d, errno: %d\n", ret, errno);
+	src_vm->enc_migrated = true;
+}
+
 static void load_td_boot_code(struct kvm_vm *vm)
 {
 	void *boot_code_hva = addr_gpa2hva(vm, FOUR_GIGABYTES_GPA - TD_BOOT_CODE_SIZE);
tools/testing/selftests/kvm/lib/x86/tdx/test_util.c +17 -0
index f92ddda2d1ac..7b622ccb2433 100644
@@ -6,6 +6,7 @@
 #include <sys/wait.h>
 #include <unistd.h>
 
+#include "apic.h"
 #include "kvm_util.h"
 #include "tdx/tdcall.h"
 #include "tdx/tdx.h"
@@ -185,3 +186,19 @@ uint64_t tdx_test_read_64bit_report_from_guest(struct kvm_vcpu *vcpu)
 {
 	return tdx_test_read_64bit(vcpu, TDX_TEST_REPORT_PORT);
 }
+
+void tdx_guest_x2apic_enable(void)
+{
+	uint64_t x2apic_spiv = APIC_BASE_MSR + (APIC_SPIV >> 4);
+	uint64_t value, ret;
+
+	/*
+	 * x2apic does not have to be enabled for TDs, TDs already have x2apic
+	 * enabled, and must use x2apic. Hence, we just soft-enable APIC.
+	 */
+	ret = tdg_vp_vmcall_instruction_rdmsr(x2apic_spiv, &value);
+	GUEST_ASSERT_EQ(ret, 0);
+	ret = tdg_vp_vmcall_instruction_wrmsr(x2apic_spiv,
+					      value | APIC_SPIV_APIC_ENABLED);
+	GUEST_ASSERT_EQ(ret, 0);
+}
tools/testing/selftests/kvm/x86/tdx_irqfd_migrate_test.c new file +264 -0

264 lines not on this page · raw message

tools/testing/selftests/kvm/x86/tdx_migrate_tests.c +0 -21
index e15da2aa0437..498e42f37697 100644
@@ -10,27 +10,6 @@
 #define NR_MIGRATE_TEST_VMS 10
 #define TDX_IOEXIT_TEST_PORT 0x50
 
-static int __tdx_migrate_from(int dst_fd, int src_fd)
-{
-	struct kvm_enable_cap cap = {
-		.cap = KVM_CAP_VM_MOVE_ENC_CONTEXT_FROM,
-		.args = { src_fd }
-	};
-
-	return ioctl(dst_fd, KVM_ENABLE_CAP, &cap);
-}
-
-
-static void tdx_migrate_from(struct kvm_vm *dst_vm, struct kvm_vm *src_vm)
-{
-	int ret;
-
-	vm_migrate_mem_regions(dst_vm, src_vm);
-	ret = __tdx_migrate_from(dst_vm->fd, src_vm->fd);
-	TEST_ASSERT(!ret, "Migration failed, ret: %d, errno: %d\n", ret, errno);
-	src_vm->enc_migrated = true;
-}
-
 void guest_code(void)
 {
 	int ret;
Signature · 1 line
2.50.0.rc1.591.g9c95f17f64-goog
#11 lore raw
In reply to Ryan Afranji
Quoted 8 lines · Ryan Afranji

Hello,

This is RFC v2 for the TDX intra-host migration patch series. It addresses comments in RFC v1 [1] and is rebased onto the latest kvm/next (v6.16-rc1).

This patchset was built on top of the latest TDX selftests [2] and gmem linking [3] RFC patch series.

In case someone is feeling necromantic, don't bother reviewing this series. I've provided a pile of feedback off-list (I forgot this on-list RFC existed), and the next version should be have significant changes.

#12 lore raw
Quoted 8 lines · Sean Christopherson
Quoted 6 lines · Ryan Afranji

Hello,

This is RFC v2 for the TDX intra-host migration patch series. It addresses comments in RFC v1 [1] and is rebased onto the latest kvm/next (v6.16-rc1).

Quoted 6 lines
Quoted 2 lines

This patchset was built on top of the latest TDX selftests [2] and gmem linking [3] RFC patch series.

In case someone is feeling necromantic, don't bother reviewing this series. I've provided a pile of feedback off-list (I forgot this on-list RFC existed), and the next version should be have significant changes.

Thanks Sean for the note. We will soon refresh intrahost migration support for software-protected VMs and this series. Need to inject these features at the right place in the queue of guest_memfd and TDX features already under review.