Skip to contentSkip to conversation

KVM: TDX: Decrease TDX VM shutdown time

PATCH Superseded Adrian Hunter 3 participants 8 messages Apr 17, 2025 – Apr 22, 2025

Subsystems (5)
Documentation x86 KVM KVM x86 +1 more subsystems

v2 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

v2 cover KVM: TDX: Decrease TDX VM shutdown time

Hi

The version 1 RFC:

	https://lore.kernel.org/all/[email protected]/

listed 3 options and implemented option 2. Sean replied with code for option 1, which tested out OK, so here it is plus a commit log.

It depends upon kvm_trylock_all_vcpus(kvm) which is the assumed result of Maxim's work-in-progress here:

      https://lore.kernel.org/all/[email protected]/

Note it is assumed that kvm_trylock_all_vcpus(kvm) follows the return value semantics of mutex_trylock() i.e. 1 means locks have been successfully acquired, 0 means not succesful.

Sean Christopherson (1):
KVM: TDX: Add sub-ioctl KVM_TDX_TERMINATE_VM

 Documentation/virt/kvm/x86/intel-tdx.rst | 16 ++++++++
 arch/x86/include/uapi/asm/kvm.h          |  1 +
 arch/x86/kvm/vmx/tdx.c                   | 63 ++++++++++++++++++++++----------
 3 files changed, 61 insertions(+), 19 deletions(-)

Regards
Adrian

#1 lore raw

PATCH v2 1/1 KVM: TDX: Add sub-ioctl KVM_TDX_TERMINATE_VM

From: Sean Christopherson <[email protected]>

Add sub-ioctl KVM_TDX_TERMINATE_VM to release the HKID prior to shutdown, which enables more efficient reclaim of private memory.

Private memory is removed from MMU/TDP when guest_memfds are closed. If the HKID has not been released, the TDX VM is still in RUNNABLE state, so pages must be removed using "Dynamic Page Removal" procedure (refer TDX Module Base spec) which involves a number of steps:
Block further address translation
Exit each VCPU
Clear Secure EPT entry
Flush/write-back/invalidate relevant caches

However, when the HKID is released, the TDX VM moves to TD_TEARDOWN state where all TDX VM pages are effectively unmapped, so pages can be reclaimed directly.

Reclaiming TD Pages in TD_TEARDOWN State was seen to decrease the total reclaim time. For example:

	VCPUs	Size (GB)	Before (secs)	After (secs)
	 4	 18		  72		 24
	32	107		 517		134
	64	400		5539		467

[Adrian: wrote commit message, added KVM_TDX_TERMINATE_VM documentation,
and moved cpus_read_lock() inside kvm->lock for consistency as reported by lockdep]

---
 Documentation/virt/kvm/x86/intel-tdx.rst | 16 ++++++
 arch/x86/include/uapi/asm/kvm.h          |  1 +
 arch/x86/kvm/vmx/tdx.c                   | 63 +++++++++++++++++-------
 3 files changed, 61 insertions(+), 19 deletions(-)
Documentation/virt/kvm/x86/intel-tdx.rst +16 -0
index de41d4c01e5c..e5d4d9cf4cf2 100644
@@ -38,6 +38,7 @@ ioctl with TDX specific sub-ioctl() commands.
           KVM_TDX_INIT_MEM_REGION,
           KVM_TDX_FINALIZE_VM,
           KVM_TDX_GET_CPUID,
+          KVM_TDX_TERMINATE_VM,
 
           KVM_TDX_CMD_NR_MAX,
   };
@@ -214,6 +215,21 @@ struct kvm_cpuid2.
 	  __u32 padding[3];
   };
 
+KVM_TDX_TERMINATE_VM
+-------------------
+:Type: vm ioctl
+:Returns: 0 on success, <0 on error
+
+Release Host Key ID (HKID) to allow more efficient reclaim of private memory.
+After this, the TD is no longer in a runnable state.
+
+Using KVM_TDX_TERMINATE_VM is optional.
+
+- id: KVM_TDX_TERMINATE_VM
+- flags: must be 0
+- data: must be 0
+- hw_error: must be 0
+
 KVM TDX creation flow
 =====================
 In addition to the standard KVM flow, new TDX ioctls need to be called.  The
arch/x86/include/uapi/asm/kvm.h +1 -0
index 225a12e0d5d6..a2f973e1d75d 100644
@@ -939,6 +939,7 @@ enum kvm_tdx_cmd_id {
 	KVM_TDX_INIT_MEM_REGION,
 	KVM_TDX_FINALIZE_VM,
 	KVM_TDX_GET_CPUID,
+	KVM_TDX_TERMINATE_VM,
 
 	KVM_TDX_CMD_NR_MAX,
 };
arch/x86/kvm/vmx/tdx.c +44 -19

120 lines not on this page · raw message

Signature · 1 line
2.43.0
#2 lore raw

Re: [PATCH V2 1/1] KVM: TDX: Add sub-ioctl KVM_TDX_TERMINATE_VM

arch/x86/kvm/vmx/tdx.c static int tdx_td_finalize(struct kvm *kvm, struct kvm_tdx_cmd *cmd) · line 2801

Quoted 10 lines · Adrian Hunter

...

arch/x86/kvm/vmx/tdx.c +8 -0
+static int tdx_terminate_vm(struct kvm *kvm)
+{
+       int r = 0;
+
+       guard(mutex)(&kvm->lock);
+       cpus_read_lock();
+
+       if (!kvm_trylock_all_vcpus(kvm)) {

Does this need to be a trylock variant? Is userspace expected to keep retrying this operation indefinitely?

Quoted 13 lines
arch/x86/kvm/vmx/tdx.c +13 -0
+               r = -EBUSY;
+               goto out;
+       }
+
+       kvm_vm_dead(kvm);
+       kvm_unlock_all_vcpus(kvm);
+
+       __tdx_release_hkid(kvm, true);
+out:
+       cpus_read_unlock();
+       return r;
+}
+
#3 lore raw

Re: [PATCH V2 1/1] KVM: TDX: Add sub-ioctl KVM_TDX_TERMINATE_VM

Quoted 14 lines · Vishal Annapurve

arch/x86/kvm/vmx/tdx.c static int tdx_td_finalize(struct kvm *kvm, struct kvm_tdx_cmd *cmd) · line 2801

Quoted 10 lines · Adrian Hunter

...

arch/x86/kvm/vmx/tdx.c +8 -0
+static int tdx_terminate_vm(struct kvm *kvm)
+{
+       int r = 0;
+
+       guard(mutex)(&kvm->lock);
+       cpus_read_lock();
+
+       if (!kvm_trylock_all_vcpus(kvm)) {

Does this need to be a trylock variant? Is userspace expected to keep retrying this operation indefinitely?

Userspace is expected to not be stupid, i.e. not be doing things with vCPUs when terminating the VM. This is already rather unpleasant, I'd rather not have to think hard about what could go wrong if KVM has to wait on all vCPU mutexes.

#4 lore raw

Re: [PATCH V2 1/1] KVM: TDX: Add sub-ioctl KVM_TDX_TERMINATE_VM

Quoted 15 lines · Sean Christopherson
Quoted 14 lines · Vishal Annapurve

arch/x86/kvm/vmx/tdx.c static int tdx_td_finalize(struct kvm *kvm, struct kvm_tdx_cmd *cmd) · line 2801

Quoted 10 lines · Adrian Hunter

...

arch/x86/kvm/vmx/tdx.c +8 -0
+static int tdx_terminate_vm(struct kvm *kvm)
+{
+       int r = 0;
+
+       guard(mutex)(&kvm->lock);
+       cpus_read_lock();
+
+       if (!kvm_trylock_all_vcpus(kvm)) {

Does this need to be a trylock variant? Is userspace expected to keep retrying this operation indefinitely?

No issue was seen in testing with a QEMU hack with no retrying. Presumably if user space is not doing anything with the TDX VM at the same time, then there should not be contention.

KVM_TDX_TERMINATE_VM is optional so it is not necessary to wait indefinitely.

Quoted 3 lines

Userspace is expected to not be stupid, i.e. not be doing things with vCPUs when terminating the VM. This is already rather unpleasant, I'd rather not have to think hard about what could go wrong if KVM has to wait on all vCPU mutexes.

#5 lore raw

Re: [PATCH V2 1/1] KVM: TDX: Add sub-ioctl KVM_TDX_TERMINATE_VM

In reply to Adrian Hunter
Quoted 21 lines · Adrian Hunter

From: Sean Christopherson <[email protected]>

Add sub-ioctl KVM_TDX_TERMINATE_VM to release the HKID prior to shutdown, which enables more efficient reclaim of private memory.

Private memory is removed from MMU/TDP when guest_memfds are closed. If the HKID has not been released, the TDX VM is still in RUNNABLE state, so pages must be removed using "Dynamic Page Removal" procedure (refer TDX Module Base spec) which involves a number of steps:
Block further address translation
Exit each VCPU
Clear Secure EPT entry
Flush/write-back/invalidate relevant caches

However, when the HKID is released, the TDX VM moves to TD_TEARDOWN state where all TDX VM pages are effectively unmapped, so pages can be reclaimed directly.

Reclaiming TD Pages in TD_TEARDOWN State was seen to decrease the total reclaim time. For example:

Quoted 8 lines
	VCPUs	Size (GB)	Before (secs)	After (secs)
	 4	 18		  72		 24
	32	107		 517		134
	64	400		5539		467

[Adrian: wrote commit message, added KVM_TDX_TERMINATE_VM documentation,
and moved cpus_read_lock() inside kvm->lock for consistency as reported by lockdep]

/facepalm

I over-thought this. We've had an long-standing battle with kvm_lock vs. cpus_read_lock(), but this is kvm->lock, not kvm_lock. /sigh

arch/x86/kvm/vmx/tdx.c static int tdx_td_finalize(struct kvm *kvm, struct kvm_tdx_cmd *cmd) · line 2798

Quoted 5 lines
arch/x86/kvm/vmx/tdx.c +5 -0
+static int tdx_terminate_vm(struct kvm *kvm)
+{
+	int r = 0;
+
+	guard(mutex)(&kvm->lock);

With kvm->lock taken outside cpus_read_lock(), just handle KVM_TDX_TERMINATE_VM in the switch statement, i.e. let tdx_vm_ioctl() deal with kvm->lock.

Quoted 32 lines
arch/x86/kvm/vmx/tdx.c +19 -0
+	cpus_read_lock();
+
+	if (!kvm_trylock_all_vcpus(kvm)) {
+		r = -EBUSY;
+		goto out;
+	}
+
+	kvm_vm_dead(kvm);
+	kvm_unlock_all_vcpus(kvm);
+
+	__tdx_release_hkid(kvm, true);
+out:
+	cpus_read_unlock();
+	return r;
+}
+
 int tdx_vm_ioctl(struct kvm *kvm, void __user *argp)
 {
 	struct kvm_tdx_cmd tdx_cmd;
@@ -2805,6 +2827,9 @@ int tdx_vm_ioctl(struct kvm *kvm, void __user *argp)
 	if (tdx_cmd.hw_error)
 		return -EINVAL;
 
+	if (tdx_cmd.id == KVM_TDX_TERMINATE_VM)
+		return tdx_terminate_vm(kvm);
+
 	mutex_lock(&kvm->lock);
 
 	switch (tdx_cmd.id) {
Signature · 1 line
2.43.0
#6 lore raw

Re: [PATCH V2 1/1] KVM: TDX: Add sub-ioctl KVM_TDX_TERMINATE_VM

Quoted 36 lines · Sean Christopherson
Quoted 21 lines · Adrian Hunter

From: Sean Christopherson <[email protected]>

Add sub-ioctl KVM_TDX_TERMINATE_VM to release the HKID prior to shutdown, which enables more efficient reclaim of private memory.

Private memory is removed from MMU/TDP when guest_memfds are closed. If the HKID has not been released, the TDX VM is still in RUNNABLE state, so pages must be removed using "Dynamic Page Removal" procedure (refer TDX Module Base spec) which involves a number of steps:
Block further address translation
Exit each VCPU
Clear Secure EPT entry
Flush/write-back/invalidate relevant caches

However, when the HKID is released, the TDX VM moves to TD_TEARDOWN state where all TDX VM pages are effectively unmapped, so pages can be reclaimed directly.

Reclaiming TD Pages in TD_TEARDOWN State was seen to decrease the total reclaim time. For example:

Quoted 8 lines
	VCPUs	Size (GB)	Before (secs)	After (secs)
	 4	 18		  72		 24
	32	107		 517		134
	64	400		5539		467

[Adrian: wrote commit message, added KVM_TDX_TERMINATE_VM documentation,
and moved cpus_read_lock() inside kvm->lock for consistency as reported by lockdep]

/facepalm

I over-thought this. We've had an long-standing battle with kvm_lock vs. cpus_read_lock(), but this is kvm->lock, not kvm_lock. /sigh

Quoted 8 lines

arch/x86/kvm/vmx/tdx.c static int tdx_td_finalize(struct kvm *kvm, struct kvm_tdx_cmd *cmd) · line 2798

Quoted 5 lines
arch/x86/kvm/vmx/tdx.c +5 -0
+static int tdx_terminate_vm(struct kvm *kvm)
+{
+	int r = 0;
+
+	guard(mutex)(&kvm->lock);

With kvm->lock taken outside cpus_read_lock(), just handle KVM_TDX_TERMINATE_VM in the switch statement, i.e. let tdx_vm_ioctl() deal with kvm->lock.

Ok, also cpus_read_lock() can go back where it was in __tdx_release_hkid().

But also in __tdx_release_hkid(), there is

	if (KVM_BUG_ON(refcount_read(&kvm->users_count) && !terminate, kvm))
		return;

However, __tdx_td_init() calls tdx_mmu_release_hkid() on the error path so that is not correct.

Quoted 33 lines
Quoted 33 lines
arch/x86/kvm/vmx/tdx.c +19 -0
+	cpus_read_lock();
+
+	if (!kvm_trylock_all_vcpus(kvm)) {
+		r = -EBUSY;
+		goto out;
+	}
+
+	kvm_vm_dead(kvm);
+	kvm_unlock_all_vcpus(kvm);
+
+	__tdx_release_hkid(kvm, true);
+out:
+	cpus_read_unlock();
+	return r;
+}
+
 int tdx_vm_ioctl(struct kvm *kvm, void __user *argp)
 {
 	struct kvm_tdx_cmd tdx_cmd;
@@ -2805,6 +2827,9 @@ int tdx_vm_ioctl(struct kvm *kvm, void __user *argp)
 	if (tdx_cmd.hw_error)
 		return -EINVAL;
 
+	if (tdx_cmd.id == KVM_TDX_TERMINATE_VM)
+		return tdx_terminate_vm(kvm);
+
 	mutex_lock(&kvm->lock);
 
 	switch (tdx_cmd.id) {
Signature · 1 line
2.43.0
#7 lore raw

Re: [PATCH V2 1/1] KVM: TDX: Add sub-ioctl KVM_TDX_TERMINATE_VM

Quoted 48 lines · Adrian Hunter
Quoted 36 lines · Sean Christopherson
Quoted 21 lines · Adrian Hunter

From: Sean Christopherson <[email protected]>

Add sub-ioctl KVM_TDX_TERMINATE_VM to release the HKID prior to shutdown, which enables more efficient reclaim of private memory.

Private memory is removed from MMU/TDP when guest_memfds are closed. If the HKID has not been released, the TDX VM is still in RUNNABLE state, so pages must be removed using "Dynamic Page Removal" procedure (refer TDX Module Base spec) which involves a number of steps:
Block further address translation
Exit each VCPU
Clear Secure EPT entry
Flush/write-back/invalidate relevant caches

However, when the HKID is released, the TDX VM moves to TD_TEARDOWN state where all TDX VM pages are effectively unmapped, so pages can be reclaimed directly.

Reclaiming TD Pages in TD_TEARDOWN State was seen to decrease the total reclaim time. For example:

Quoted 8 lines
	VCPUs	Size (GB)	Before (secs)	After (secs)
	 4	 18		  72		 24
	32	107		 517		134
	64	400		5539		467

[Adrian: wrote commit message, added KVM_TDX_TERMINATE_VM documentation,
and moved cpus_read_lock() inside kvm->lock for consistency as reported by lockdep]

/facepalm

I over-thought this. We've had an long-standing battle with kvm_lock vs. cpus_read_lock(), but this is kvm->lock, not kvm_lock. /sigh

Quoted 8 lines

Review

Quoted 5 lines
(quoted diff) +5 -0
+static int tdx_terminate_vm(struct kvm *kvm)
+{
+	int r = 0;
+
+	guard(mutex)(&kvm->lock);

With kvm->lock taken outside cpus_read_lock(), just handle KVM_TDX_TERMINATE_VM in the switch statement, i.e. let tdx_vm_ioctl() deal with kvm->lock.

Ok, also cpus_read_lock() can go back where it was in __tdx_release_hkid().

Quoted 8 lines

But also in __tdx_release_hkid(), there is

	if (KVM_BUG_ON(refcount_read(&kvm->users_count) && !terminate, kvm))
		return;

However, __tdx_td_init() calls tdx_mmu_release_hkid() on the error path so that is not correct.

So it ends up like this:

Documentation/virt/kvm/x86/intel-tdx.rst +16 -0
index de41d4c01e5c..e5d4d9cf4cf2 100644
@@ -38,6 +38,7 @@ ioctl with TDX specific sub-ioctl() commands.
           KVM_TDX_INIT_MEM_REGION,
           KVM_TDX_FINALIZE_VM,
           KVM_TDX_GET_CPUID,
+          KVM_TDX_TERMINATE_VM,
 
           KVM_TDX_CMD_NR_MAX,
   };
@@ -214,6 +215,21 @@ struct kvm_cpuid2.
 	  __u32 padding[3];
   };
 
+KVM_TDX_TERMINATE_VM
+-------------------
+:Type: vm ioctl
+:Returns: 0 on success, <0 on error
+
+Release Host Key ID (HKID) to allow more efficient reclaim of private memory.
+After this, the TD is no longer in a runnable state.
+
+Using KVM_TDX_TERMINATE_VM is optional.
+
+- id: KVM_TDX_TERMINATE_VM
+- flags: must be 0
+- data: must be 0
+- hw_error: must be 0
+
 KVM TDX creation flow
 =====================
 In addition to the standard KVM flow, new TDX ioctls need to be called.  The
arch/x86/include/uapi/asm/kvm.h +1 -0
index 225a12e0d5d6..a2f973e1d75d 100644
@@ -939,6 +939,7 @@ enum kvm_tdx_cmd_id {
 	KVM_TDX_INIT_MEM_REGION,
 	KVM_TDX_FINALIZE_VM,
 	KVM_TDX_GET_CPUID,
+	KVM_TDX_TERMINATE_VM,
 
 	KVM_TDX_CMD_NR_MAX,
 };
arch/x86/kvm/vmx/tdx.c +26 -15

78 lines not on this page · raw message