KVM: Drop vm_dead, pivot on vm_bugged for -EIO
Subsystems (8)
Sean Christopherson
maintainercover KVM: Drop vm_dead, pivot on vm_bugged for -EIO
Drop vm_dead and instead reject ioctls based only on vm_bugged. Checking vm_dead (or vm_bugged) is inherently racy due as it's not protected by any locks. For vm_bugged, imperfection is a-ok as the goal is purely to limit the damage done by a kernel/hardware bug. But rejecting ioclts based on vm_dead is dangerous as it gives us a false sense of security, e.g. see the race found by syzbot in commit ecf371f8b02d ("KVM: SVM: Reject SEV{-ES} intra host migration if vCPU creation is in-flight").
This series was motivated by the last patch, a.k.a. KVM_TDX_TERMINATE_VM. I applied a slightly different version of that patch for 6.17[*], but I'm reposting it with the vm_dead changes due to Paolo's question about whether or not we should have a generic KVM_TERMINATE_VM; dropping vm_dead doesn't make much sense if we want to add KVM_TERMINATE_VM.
[*] https://lore.kernel.org/all/[email protected]
Sean Christopherson (5):
KVM: Never clear KVM_REQ_VM_DEAD from a vCPU's requests
KVM: TDX: Exit with MEMORY_FAULT on unexpected pending S-EPT Violation
KVM: Reject ioctls only if the VM is bugged, not simply marked dead
KVM: selftests: Use for-loop to handle all successful SEV migrations
KVM: TDX: Add sub-ioctl KVM_TDX_TERMINATE_VM
Documentation/virt/kvm/x86/intel-tdx.rst | 22 ++++++++- arch/arm64/kvm/arm.c | 2 +- arch/arm64/kvm/vgic/vgic-init.c | 2 +- arch/x86/include/uapi/asm/kvm.h | 7 ++- arch/x86/kvm/mmu/mmu.c | 2 +- arch/x86/kvm/vmx/tdx.c | 45 +++++++++++++------ arch/x86/kvm/vmx/tdx.h | 1 + arch/x86/kvm/x86.c | 2 +- include/linux/kvm_host.h | 11 +++-- .../selftests/kvm/x86/sev_migrate_tests.c | 34 ++++++-------- virt/kvm/kvm_main.c | 10 ++--- 11 files changed, 90 insertions(+), 48 deletions(-)
base-commit: beafd7ecf2255e8b62a42dc04f54843033db3d24
Signature · 1 line
2.50.1.552.g942d659e1b-goog
Sean Christopherson
maintainerPATCH 1/5 KVM: Never clear KVM_REQ_VM_DEAD from a vCPU's requests
Use kvm_test_request() instead of kvm_check_request() when querying KVM_REQ_VM_DEAD, i.e. don't clear KVM_REQ_VM_DEAD, as the entire purpose of KVM_REQ_VM_DEAD is to prevent the vCPU from enterring the guest ever again, even if userspace insists on redoing KVM_RUN.
Ensuring KVM_REQ_VM_DEAD is never cleared will allow relaxing KVM's rule that ioctls can't be invoked on dead VMs, to only disallow ioctls if the VM is bugged, i.e. if KVM hit a KVM_BUG_ON().
Opportunistically add compile-time assertions to guard against clearing KVM_REQ_VM_DEAD through the standard APIs.
--- arch/arm64/kvm/arm.c | 2 +- arch/x86/kvm/mmu/mmu.c | 2 +- arch/x86/kvm/vmx/tdx.c | 2 +- arch/x86/kvm/x86.c | 2 +- include/linux/kvm_host.h | 9 +++++++-- 5 files changed, 11 insertions(+), 6 deletions(-)
arch/arm64/kvm/arm.c +1 -1
@@ -1013,7 +1013,7 @@ static int kvm_vcpu_suspend(struct kvm_vcpu *vcpu)static int check_vcpu_requests (struct kvm_vcpu *vcpu) {if (kvm_request_pending (vcpu)) { -if (kvm_check_request (KVM_REQ_VM_DEAD , vcpu)) +if (kvm_test_request (KVM_REQ_VM_DEAD , vcpu))return -EIO ;if (kvm_check_request (KVM_REQ_SLEEP , vcpu))
arch/x86/kvm/mmu/mmu.c +1 -1
@@ -4915,7 +4915,7 @@ int kvm_tdp_map_page(struct kvm_vcpu *vcpu, gpa_t gpa, u64 error_code, u8 *levelif (signal_pending (current))return -EINTR ; -if (kvm_check_request (KVM_REQ_VM_DEAD , vcpu)) +if (kvm_test_request (KVM_REQ_VM_DEAD , vcpu))return -EIO ;cond_resched ();
arch/x86/kvm/vmx/tdx.c +1 -1
@@ -2010,7 +2010,7 @@ static int tdx_handle_ept_violation(struct kvm_vcpu *vcpu)if (kvm_vcpu_has_events (vcpu) ||signal_pending (current))break ; -if (kvm_check_request (KVM_REQ_VM_DEAD , vcpu)) { +if (kvm_test_request (KVM_REQ_VM_DEAD , vcpu)) { ret = -EIO ;break ; }
arch/x86/kvm/x86.c +1 -1
@@ -10649,7 +10649,7 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)bool req_immediate_exit = false;if (kvm_request_pending (vcpu)) { -if (kvm_check_request (KVM_REQ_VM_DEAD , vcpu)) { +if (kvm_test_request (KVM_REQ_VM_DEAD , vcpu)) { r = -EIO ; goto out; }
include/linux/kvm_host.h +7 -2
@@ -2261,13 +2261,18 @@ static inline bool kvm_test_request(int req, struct kvm_vcpu *vcpu)return test_bit (req &KVM_REQUEST_MASK , (void *)&vcpu->requests); } -static inline void kvm_clear_request (int req,struct kvm_vcpu *vcpu) +static __always_inline void kvm_clear_request(int req, struct kvm_vcpu *vcpu) { +BUILD_BUG_ON (req ==KVM_REQ_VM_DEAD ); +clear_bit (req &KVM_REQUEST_MASK , (void *)&vcpu->requests); } -static inline bool kvm_check_request (int req,struct kvm_vcpu *vcpu) +static __always_inline bool kvm_check_request(int req, struct kvm_vcpu *vcpu) { +/* Once a VM is dead, it needs to stay dead. */ +BUILD_BUG_ON (req ==KVM_REQ_VM_DEAD ); +if (kvm_test_request (req, vcpu)) {kvm_clear_request (req, vcpu);
Signature · 1 line
2.50.1.552.g942d659e1b-goog
Sean Christopherson
maintainerPATCH 2/5 KVM: TDX: Exit with MEMORY_FAULT on unexpected pending S-EPT Violation
In reply to Sean ChristophersonExit to userspace with -EFAULT and a valid MEMORY_FAULT exit if a vCPU hits an unexpected pending S-EPT Violation instead of marking the VM dead. While it's unlikely the VM can continue on, whether or not to terminate the VM is not KVM's decision to make.
Set memory_fault.size to zero to communicate to userspace that reported fault is "bad", and to effectively terminate the VM if userspace blindly treats the exit as a conversion attempt (KVM_SET_MEMORY_ATTRIBUTES will fail with -EINVAL if the size is zero).
Opportunistically delete the pr_warn(), which could be abused to spam the kernel log, and is largely useless outside of interact debug as it doesn't specify which VM encountered a failure.
--- arch/x86/kvm/vmx/tdx.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-)
arch/x86/kvm/vmx/tdx.c +2 -4
@@ -1937,10 +1937,8 @@ static int tdx_handle_ept_violation(struct kvm_vcpu *vcpu)if (vt_is_tdx_private_gpa (vcpu->kvm, gpa)) {if (tdx_is_sept_violation_unexpected_pending (vcpu)) { -pr_warn ("Guest access before accepting 0x%llx on vCPU %d\n" , - gpa, vcpu->vcpu_id); -kvm_vm_dead (vcpu->kvm); -return -EIO ; +kvm_prepare_memory_fault_exit (vcpu, gpa,0 , true, false, true); +return -EFAULT ; }/* * Always treat SEPT violations as write faults. Ignore the
Signature · 1 line
2.50.1.552.g942d659e1b-goog
Re: [PATCH 2/5] KVM: TDX: Exit with MEMORY_FAULT on unexpected pending S-EPT Violation
Quoted 14 lines · Sean Christopherson
arch/x86/kvm/vmx/tdx.c +0
-0@@ -1937,10 +1937,8 @@ static int tdx_handle_ept_violation(struct kvm_vcpu *vcpu)
if (vt_is_tdx_private_gpa(vcpu->kvm, gpa)) {
if (tdx_is_sept_violation_unexpected_pending(vcpu)) {arch/x86/kvm/vmx/tdx.c +2
-4-pr_warn ("Guest access before accepting 0x%llx on vCPU %d\n" , - gpa, vcpu->vcpu_id); -kvm_vm_dead (vcpu->kvm); -return -EIO ; +kvm_prepare_memory_fault_exit (vcpu, gpa,0 , true, false, true); +return -EFAULT ;
Quoted 3 lines
}
/*
* Always treat SEPT violations as write faults. Ignore the
The vm_dead was added because mirror EPT will KVM_BUG_ON() if there is an attempt to set the mirror EPT entry when it is already present. And the unaccepted memory access will trigger an EPT violation for a mirror PTE that is already set. I think this is a better solution irrespective of the vm_dead changes.
But hmm, tangentially related, but Yan do we have a similar problem with KVM_PRE_FAULT_MEMORY after we started setting pre_fault_allowed during TD finalization?
Sean Christopherson
maintainerRe: [PATCH 2/5] KVM: TDX: Exit with MEMORY_FAULT on unexpected pending S-EPT Violation
Quoted 16 lines · Rick P Edgecombe
Quoted 15 lines · Sean Christopherson
arch/x86/kvm/vmx/tdx.c +0
-0@@ -1937,10 +1937,8 @@ static int tdx_handle_ept_violation(struct kvm_vcpu *vcpu)
if (vt_is_tdx_private_gpa(vcpu->kvm, gpa)) {
if (tdx_is_sept_violation_unexpected_pending(vcpu)) {arch/x86/kvm/vmx/tdx.c +2
-4-pr_warn ("Guest access before accepting 0x%llx on vCPU %d\n" , - gpa, vcpu->vcpu_id); -kvm_vm_dead (vcpu->kvm); -return -EIO ; +kvm_prepare_memory_fault_exit (vcpu, gpa,0 , true, false, true); +return -EFAULT ;}
Quoted 8 lines
Quoted 2 lines
/*
* Always treat SEPT violations as write faults. Ignore theThe
vm_deadwas added because mirror EPT willKVM_BUG_ON()if there is an attempt to set the mirror EPT entry when it is already present. And the unaccepted memory access will trigger an EPT violation for a mirror PTE that is already set. I think this is a better solution irrespective of thevm_deadchanges.
In that case, this change will expose KVM to the KVM_BUG_ON(), because nothing prevents userspace from re-running the vCPU. Which KVM_BUG_ON() exactly gets hit?
Edgecombe, Rick P
maintainerRe: [PATCH 2/5] KVM: TDX: Exit with MEMORY_FAULT on unexpected pending S-EPT Violation
Quoted 9 lines · Sean Christopherson
Quoted 6 lines
The
vm_deadwas added because mirror EPT willKVM_BUG_ON()if there is an attempt to set the mirror EPT entry when it is already present. And the unaccepted memory access will trigger an EPT violation for a mirror PTE that is
already set. I think this is a better solution irrespective of thevm_deadchanges.In that case, this change will expose KVM to the
KVM_BUG_ON(), because nothing prevents userspace from re-running the vCPU.
If userspace runs the vCPU again then an EPT violation gets triggered again, which again gets kicked out to userspace. The new check will prevent it from getting into the fault handler, right?
Quoted 1 line
Which
KVM_BUG_ON()exactly gets hit?
Should be:
static int __must_check set_external_spte_present(struct kvm *kvm, tdp_ptep_t sptep,
gfn_t gfn, u64 old_spte, u64 new_spte, int level)
{
bool was_present = is_shadow_present_pte(old_spte);
bool is_present = is_shadow_present_pte(new_spte);
bool is_leaf = is_present && is_last_spte(new_spte, level);
kvm_pfn_t new_pfn = spte_to_pfn(new_spte);
int ret = 0;
KVM_BUG_ON(was_present, kvm);Sean Christopherson
maintainerRe: [PATCH 2/5] KVM: TDX: Exit with MEMORY_FAULT on unexpected pending S-EPT Violation
Quoted 7 lines · Rick P Edgecombe
Quoted 6 lines · Sean Christopherson
Quoted 5 lines
The
vm_deadwas added because mirror EPT willKVM_BUG_ON()if there is an attempt to set the mirror EPT entry when it is already present. And the unaccepted memory access will trigger an EPT violation for a mirror PTE that is already set. I think this is a better solution irrespective of thevm_deadchanges.
Quoted 6 lines
Quoted 2 lines
In that case, this change will expose KVM to the
KVM_BUG_ON(), because nothing prevents userspace from re-running the vCPU.If userspace runs the vCPU again then an EPT violation gets triggered again, which again gets kicked out to userspace. The new check will prevent it from getting into the fault handler, right?
Yes? But I'm confused about why you mentioned vm_dead, and why you're calling this a "new check". This effectively does two things: drops kvm_vm_dead() and switches from EOI => EFAULT. _If_ setting vm_dead was necessary, then we have a problem.
I assume by "The vm_dead was added" you really mean "forcing an exit to userspace", and that kvm_vm_dead()+EIO was a somewhat arbitrary way of forcing an exit?
Quoted 8 lines
Quoted 1 line
Which
KVM_BUG_ON()exactly gets hit?Should be:
static int
__must_checkset_external_spte_present(struct kvm *kvm,tdp_ptep_tsptep,
gfn_tgfn, u64old_spte, u64new_spte, int level)
Quoted 8 lines
{
bool was_present = is_shadow_present_pte(old_spte);
bool is_present = is_shadow_present_pte(new_spte);
bool is_leaf = is_present && is_last_spte(new_spte, level);
kvm_pfn_t new_pfn = spte_to_pfn(new_spte);
int ret = 0;
KVM_BUG_ON(was_present, kvm);Yeah, I don't see how that can be reach in this scenario.
Edgecombe, Rick P
maintainerRe: [PATCH 2/5] KVM: TDX: Exit with MEMORY_FAULT on unexpected pending S-EPT Violation
Quoted 10 lines · Sean Christopherson
Quoted 3 lines
If userspace runs the vCPU again then an EPT violation gets triggered again, which again gets kicked out to userspace. The new check will prevent it from getting into the fault handler, right?
Yes? But I'm confused about why you mentioned
vm_dead, and why you're calling this a "new check". This effectively does two things: dropskvm_vm_dead()and switches from EOI => EFAULT. _If_ settingvm_deadwas necessary, then we have a
problem.
Quoted 3 lines
I assume by "The
vm_deadwas added" you really mean "forcing an exit to userspace",
and that kvm_vm_dead()+EIO was a somewhat arbitrary way of forcing an exit?
Sorry, yes vm_dead prevents an EPT violation loop but not the KVM_BUG_ON(). The whole if clause prevents the KVM_BUG_ON(). Your patch prevents the ept violation loop in a better way.
Re: [PATCH 2/5] KVM: TDX: Exit with MEMORY_FAULT on unexpected pending S-EPT Violation
In reply to Edgecombe, Rick PQuoted 23 lines · Rick P
Quoted 9 lines · Sean Christopherson
Quoted 6 lines
The
vm_deadwas added because mirror EPT willKVM_BUG_ON()if there is an attempt to set the mirror EPT entry when it is already present. And the unaccepted memory access will trigger an EPT violation for a mirror PTE that is
already set. I think this is a better solution irrespective of thevm_deadchanges.In that case, this change will expose KVM to the
KVM_BUG_ON(), because nothing prevents userspace from re-running the vCPU.If userspace runs the vCPU again then an EPT violation gets triggered again, which again gets kicked out to userspace. The new check will prevent it from getting into the fault handler, right?
Quoted 1 line
Which
KVM_BUG_ON()exactly gets hit?Should be:
static int
__must_checkset_external_spte_present(struct kvm *kvm,tdp_ptep_tsptep,
gfn_tgfn, u64old_spte, u64new_spte, int level)
Quoted 8 lines
{
bool was_present = is_shadow_present_pte(old_spte);
bool is_present = is_shadow_present_pte(new_spte);
bool is_leaf = is_present && is_last_spte(new_spte, level);
kvm_pfn_t new_pfn = spte_to_pfn(new_spte);
int ret = 0;
KVM_BUG_ON(was_present, kvm);Yes, this KVM_BUG_ON() could be triggered in an earlier code if tdx_is_sept_violation_unexpected_pending() wasn't checked to prevent repeated EPT faults.
But the KVM_BUG_ON() is no longer reachable after the commit cc7ed3358e41 ("KVM:
x86/mmu: Always set SPTE's dirty bit if it's created as writable").
With the current code, even without putting VM to dead, the worst case is the endless invocation of EPT fault handler, returning RET_PF_SPURIOUS.
Re: [PATCH 2/5] KVM: TDX: Exit with MEMORY_FAULT on unexpected pending S-EPT Violation
In reply to Edgecombe, Rick PQuoted 25 lines · Rick P
Quoted 14 lines · Sean Christopherson
arch/x86/kvm/vmx/tdx.c +0
-0@@ -1937,10 +1937,8 @@ static int tdx_handle_ept_violation(struct kvm_vcpu *vcpu)
if (vt_is_tdx_private_gpa(vcpu->kvm, gpa)) {
if (tdx_is_sept_violation_unexpected_pending(vcpu)) {arch/x86/kvm/vmx/tdx.c +2
-4-pr_warn ("Guest access before accepting 0x%llx on vCPU %d\n" , - gpa, vcpu->vcpu_id); -kvm_vm_dead (vcpu->kvm); -return -EIO ; +kvm_prepare_memory_fault_exit (vcpu, gpa,0 , true, false, true); +return -EFAULT ;Quoted 3 lines
}
/*
* Always treat SEPT violations as write faults. Ignore theThe
vm_deadwas added because mirror EPT willKVM_BUG_ON()if there is an attempt to set the mirror EPT entry when it is already present. And the unaccepted memory access will trigger an EPT violation for a mirror PTE that is already set. I think this is a better solution irrespective of thevm_deadchanges.
Quoted 5 lines
Reviewed-by Rick Edgecombe <[email protected]>But hmm, tangentially related, but Yan do we have a similar problem with
KVM_PRE_FAULT_MEMORYafter we started settingpre_fault_allowedduring TD finalization?
Sean's commit 6385d01eec16 ("KVM: x86/mmu: Don't overwrite shadow-present MMU SPTEs when prefaulting") should have prevented repeated invocation of set_external_spte_present() with prefaulted entries.
Edgecombe, Rick P
maintainerRe: [PATCH 2/5] KVM: TDX: Exit with MEMORY_FAULT on unexpected pending S-EPT Violation
Quoted 6 lines · Yan Zhao
Quoted 3 lines
But hmm, tangentially related, but Yan do we have a similar problem with
KVM_PRE_FAULT_MEMORYafter we started settingpre_fault_allowedduring TD finalization?Sean's commit
6385d01eec16("KVM: x86/mmu: Don't overwrite shadow-present MMU SPTEs when prefaulting") should have prevented repeated invocation ofset_external_spte_present()with prefaulted entries.
Ok, great.
Re: [PATCH 2/5] KVM: TDX: Exit with MEMORY_FAULT on unexpected pending S-EPT Violation
In reply to Sean ChristophersonQuoted 9 lines · Sean Christopherson
Exit to userspace with
-EFAULTand a validMEMORY_FAULTexit if a vCPU hits an unexpected pending S-EPT Violation instead of marking the VM dead. While it's unlikely the VM can continue on, whether or not to terminate the VM is not KVM's decision to make.Set memory_fault.size to zero to communicate to userspace that reported fault is "bad", and to effectively terminate the VM if userspace blindly treats the exit as a conversion attempt (
KVM_SET_MEMORY_ATTRIBUTESwill fail with-EINVALif the size is zero).
This sets a special contract on size zero.
I had a patch internally, which introduce a new exit type:
+ /* KVM_EXIT_GUEST_ERROR */
+ struct {
+ #define KVM_GUEST_ERROR_TDX_ACCESS_PENDING_PAGE 0
+ __u32 error_type; + __u32 ndata; + __u64 data[16];
+ } guest_error;
how about it?
Quoted 26 lines
Opportunistically delete the
pr_warn(), which could be abused to spam the kernel log, and is largely useless outside of interact debug as it doesn't specify which VM encountered a failure.Signed-off-by Sean Christopherson <[email protected]>--- arch/x86/kvm/vmx/tdx.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-)arch/x86/kvm/vmx/tdx.c +2
-4@@ -1937,10 +1937,8 @@ static int tdx_handle_ept_violation(struct kvm_vcpu *vcpu)if (vt_is_tdx_private_gpa (vcpu->kvm, gpa)) {if (tdx_is_sept_violation_unexpected_pending (vcpu)) { -pr_warn ("Guest access before accepting 0x%llx on vCPU %d\n" , - gpa, vcpu->vcpu_id); -kvm_vm_dead (vcpu->kvm); -return -EIO ; +kvm_prepare_memory_fault_exit (vcpu, gpa,0 , true, false, true); +return -EFAULT ; }/* * Always treat SEPT violations as write faults. Ignore the
Re: [PATCH 2/5] KVM: TDX: Exit with MEMORY_FAULT on unexpected pending S-EPT Violation
Quoted 6 lines · Xiaoyao Li
Quoted 5 lines · Sean Christopherson
Exit to userspace with
-EFAULTand a validMEMORY_FAULTexit if a vCPU hits an unexpected pending S-EPT Violation instead of marking the VM dead. While it's unlikely the VM can continue on, whether or not to terminate the VM is not KVM's decision to make.
Quoted 6 lines
Quoted 4 lines
Set memory_fault.size to zero to communicate to userspace that reported fault is "bad", and to effectively terminate the VM if userspace blindly treats the exit as a conversion attempt (
KVM_SET_MEMORY_ATTRIBUTESwill fail with-EINVALif the size is zero).This sets a special contract on size zero.
+1.
Or would it be good to use pr_warn_once() to indicate that the VM termination reason is "Guest access before accepting" if a new exit type is not specified?
This info is straightforward and helpful in cases when a TD is accidentally killed.
Quoted 39 lines
I had a patch internally, which introduce a new exit type:
(quoted diff) +7
-0+ /* KVM_EXIT_GUEST_ERROR */ + struct { + #define KVM_GUEST_ERROR_TDX_ACCESS_PENDING_PAGE 0 + __u32 error_type; + __u32 ndata; + __u64 data[16]; + } guest_error;how about it?
Quoted 25 lines
Opportunistically delete the
pr_warn(), which could be abused to spam the kernel log, and is largely useless outside of interact debug as it doesn't specify which VM encountered a failure.Signed-off-by Sean Christopherson <[email protected]>--- arch/x86/kvm/vmx/tdx.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-)arch/x86/kvm/vmx/tdx.c +2
-4@@ -1937,10 +1937,8 @@ static int tdx_handle_ept_violation(struct kvm_vcpu *vcpu)if (vt_is_tdx_private_gpa (vcpu->kvm, gpa)) {if (tdx_is_sept_violation_unexpected_pending (vcpu)) { -pr_warn ("Guest access before accepting 0x%llx on vCPU %d\n" , - gpa, vcpu->vcpu_id); -kvm_vm_dead (vcpu->kvm); -return -EIO ; +kvm_prepare_memory_fault_exit (vcpu, gpa,0 , true, false, true); +return -EFAULT ; }/* * Always treat SEPT violations as write faults. Ignore the
Sean Christopherson
maintainerPATCH 3/5 KVM: Reject ioctls only if the VM is bugged, not simply marked dead
In reply to Sean ChristophersonRelax the protection against interacting with a buggy KVM to only reject ioctls if the VM is bugged, i.e. allow userspace to invoke ioctls if KVM deliberately terminated the VM. Drop kvm.vm_dead as there are no longer any readers, and KVM shouldn't rely on vm_dead for functional correctness. The only functional guarantees provided by kvm_vm_dead() come by way of KVM_REQ_VM_DEAD, which ensures that vCPU won't re-enter the guest.
Practically speaking, this only affects x86, which uses kvm_vm_dead() to prevent running a VM whose resources have been partially freed or has run one or more of its vCPUs into an architecturally defined state. In these cases, there is no (known) danger to KVM, the goal is purely to prevent entering the guest.
As evidenced by commit ecf371f8b02d ("KVM: SVM: Reject SEV{-ES} intra host migration if vCPU creation is in-flight"), the restriction on invoking ioctls only blocks _new_ ioctls. I.e. KVM mustn't rely on blocking ioctls for functional safety (whereas KVM_REQ_VM_DEAD is guaranteed to prevent vCPUs from entering the guest).
--- arch/arm64/kvm/vgic/vgic-init.c | 2 +- include/linux/kvm_host.h | 2 -- tools/testing/selftests/kvm/x86/sev_migrate_tests.c | 5 +---- virt/kvm/kvm_main.c | 10 +++++----- 4 files changed, 7 insertions(+), 12 deletions(-)
arch/arm64/kvm/vgic/vgic-init.c +1 -1
@@ -612,7 +612,7 @@ int kvm_vgic_map_resources(struct kvm *kvm)mutex_unlock (&kvm->arch.config_lock); out_slots:if (ret) -kvm_vm_dead (kvm); +kvm_vm_bugged (kvm);mutex_unlock (&kvm->slots_lock);
include/linux/kvm_host.h +0 -2
@@ -854,7 +854,6 @@ struct kvm {u32 dirty_ring_size;bool dirty_ring_with_bitmap;bool vm_bugged; -bool vm_dead;#ifdef CONFIG_HAVE_KVM_PM_NOTIFIER struct notifier_block pm_notifier;
@@ -894,7 +893,6 @@ struct kvm {static inline void kvm_vm_dead (struct kvm *kvm) { - kvm->vm_dead = true;kvm_make_all_cpus_request (kvm,KVM_REQ_VM_DEAD ); }
tools/testing/selftests/kvm/x86/sev_migrate_tests.c +1 -4
@@ -87,10 +87,7 @@ static void test_sev_migrate_from(bool es)sev_migrate_from (dst_vms[i], dst_vms[i -1 ]);/* Migrate the guest back to the original VM. */ - ret =__sev_migrate_from (src_vm, dst_vms[NR_MIGRATE_TEST_VMS -1 ]); -TEST_ASSERT (ret ==-1 && errno ==EIO , -"VM that was migrated from should be dead. ret %d, errno: %d" , ret, - errno); +sev_migrate_from (src_vm, dst_vms[NR_MIGRATE_TEST_VMS -1 ]);kvm_vm_free (src_vm);for (i =0 ; i <NR_MIGRATE_TEST_VMS ; ++i)
virt/kvm/kvm_main.c +5 -5
@@ -4408,7 +4408,7 @@ static long kvm_vcpu_ioctl(struct file *filp,struct kvm_fpu *fpu =NULL ;struct kvm_sregs *kvm_sregs =NULL ; -if (vcpu->kvm->mm != current->mm || vcpu->kvm->vm_dead) +if (vcpu->kvm->mm != current->mm || vcpu->kvm->vm_bugged)return -EIO ;if (unlikely (_IOC_TYPE (ioctl) !=KVMIO ))
@@ -4651,7 +4651,7 @@ static long kvm_vcpu_compat_ioctl(struct file *filp,void __user *argp =compat_ptr (arg);int r; -if (vcpu->kvm->mm != current->mm || vcpu->kvm->vm_dead) +if (vcpu->kvm->mm != current->mm || vcpu->kvm->vm_bugged)return -EIO ;switch (ioctl) {
@@ -4717,7 +4717,7 @@ static long kvm_device_ioctl(struct file *filp, unsigned int ioctl, {struct kvm_device *dev = filp->private_data; -if (dev->kvm->mm != current->mm || dev->kvm->vm_dead) +if (dev->kvm->mm != current->mm || dev->kvm->vm_bugged)return -EIO ;switch (ioctl) {
@@ -5139,7 +5139,7 @@ static long kvm_vm_ioctl(struct file *filp,void __user *argp = (void __user *)arg;int r; -if (kvm->mm != current->mm || kvm->vm_dead) +if (kvm->mm != current->mm || kvm->vm_bugged)return -EIO ;switch (ioctl) {case KVM_CREATE_VCPU :
@@ -5403,7 +5403,7 @@ static long kvm_vm_compat_ioctl(struct file *filp,struct kvm *kvm = filp->private_data;int r; -if (kvm->mm != current->mm || kvm->vm_dead) +if (kvm->mm != current->mm || kvm->vm_bugged)return -EIO ; r =kvm_arch_vm_compat_ioctl (filp, ioctl, arg);
Signature · 1 line
2.50.1.552.g942d659e1b-goog
Re: [PATCH 3/5] KVM: Reject ioctls only if the VM is bugged, not simply marked dead
Quoted 6 lines · Sean Christopherson
Relax the protection against interacting with a buggy KVM to only reject ioctls if the VM is bugged, i.e. allow userspace to invoke ioctls if KVM deliberately terminated the VM. Drop kvm.vm_dead as there are no longer any readers, and KVM shouldn't rely on
vm_deadfor functional correctness. The only functional guarantees provided bykvm_vm_dead()come by way ofKVM_REQ_VM_DEAD, which ensures that vCPU won't re-enter the guest.
If ioctls are allowed for dead VMs, would it be possible for userspace to create a new vCPU and attempt to enter a dead VM? is this something KVM should prevent?
Quoted 4 lines
Practically speaking, this only affects x86, which uses
kvm_vm_dead()to prevent running a VM whose resources have been partially freed or has run one or more of its vCPUs into an architecturally defined state. In these
^^^ undefined?
Quoted 8 lines
cases, there is no (known) danger to KVM, the goal is purely to prevent entering the guest.
As evidenced by commit
ecf371f8b02d("KVM: SVM: Reject SEV{-ES} intra host migration if vCPU creation is in-flight"), the restriction on invoking ioctls only blocks _new_ ioctls. I.e. KVM mustn't rely on blocking ioctls for functional safety (whereasKVM_REQ_VM_DEADis guaranteed to prevent vCPUs from entering the guest).
Sean Christopherson
maintainerPATCH 4/5 KVM: selftests: Use for-loop to handle all successful SEV migrations
In reply to Sean ChristophersonUse the main for-loop in the "SEV migrate from" testcase to handle all successful migrations, as there is nothing inherently unique about the original source VM beyond it needing to be created as an SEV VM.
No functional change intended.
--- .../selftests/kvm/x86/sev_migrate_tests.c | 31 +++++++++---------- 1 file changed, 14 insertions(+), 17 deletions(-)
tools/testing/selftests/kvm/x86/sev_migrate_tests.c +14 -17
@@ -14,7 +14,7 @@#include "kselftest.h" #define NR_MIGRATE_TEST_VCPUS 4 -#define NR_MIGRATE_TEST_VMS 3 +#define NR_MIGRATE_TEST_VMS 4#define NR_LOCK_TESTING_THREADS 3#define NR_LOCK_TESTING_ITERATIONS 10000
@@ -72,26 +72,23 @@ static void sev_migrate_from(struct kvm_vm *dst, struct kvm_vm *src)static void test_sev_migrate_from (bool es) { -struct kvm_vm *src_vm; -struct kvm_vm *dst_vms[NR_MIGRATE_TEST_VMS ]; -int i, ret; +struct kvm_vm *vms[NR_MIGRATE_TEST_VMS ]; +int i; - src_vm =sev_vm_create (es); -for (i =0 ; i <NR_MIGRATE_TEST_VMS ; ++i) - dst_vms[i] =aux_vm_create (true); - -/* Initial migration from the src to the first dst. */ -sev_migrate_from (dst_vms[0 ], src_vm); - -for (i =1 ; i <NR_MIGRATE_TEST_VMS ; i++) -sev_migrate_from (dst_vms[i], dst_vms[i -1 ]); + vms[0 ] =sev_vm_create (es); +for (i =1 ; i <NR_MIGRATE_TEST_VMS ; ++i) + vms[i] =aux_vm_create (true); -/* Migrate the guest back to the original VM. */ -sev_migrate_from (src_vm, dst_vms[NR_MIGRATE_TEST_VMS -1 ]); +/* +* Migrate in N times, in a chain from the initial SEV VM to each "aux" +* VM, and finally back to the original SEV VM. KVM disallows KVM_RUN +* on the source after migration, but all other ioctls should succeed. +*/ +for (i =0 ; i <NR_MIGRATE_TEST_VMS ; i++) +sev_migrate_from (vms[(i +1 ) %NR_MIGRATE_TEST_VMS ], vms[i]); -kvm_vm_free (src_vm);for (i =0 ; i <NR_MIGRATE_TEST_VMS ; ++i) -kvm_vm_free (dst_vms[i]); +kvm_vm_free (vms[i]); }struct locking_thread_input {
Signature · 1 line
2.50.1.552.g942d659e1b-goog
Sean Christopherson
maintainerPATCH 5/5 KVM: TDX: Add sub-ioctl KVM_TDX_TERMINATE_VM
In reply to Sean ChristophersonAdd sub-ioctl KVM_TDX_TERMINATE_VM to release the HKID prior to shutdown, which enables more efficient reclaim of private memory.
Private memory is removed from MMU/TDP when guest_memfds are closed. If the HKID has not been released, the TDX VM is still in the RUNNABLE state, and so pages must be removed using "Dynamic Page Removal" procedure (refer to the TDX Module Base spec) which involves a number of steps:
Block further address translation
Exit each VCPU
Clear Secure EPT entry
Flush/write-back/invalidate relevant caches
However, when the HKID is released, the TDX VM moves to TD_TEARDOWN state, where all TDX VM pages are effectively unmapped, so pages can be reclaimed directly.
Reclaiming TD Pages in TD_TEARDOWN State was seen to decrease the total reclaim time. For example:
VCPUs Size (GB) Before (secs) After (secs)
4 18 72 24
32 107 517 134
64 400 5539 467Add kvm_tdx_capabilities.supported_caps along with KVM_TDX_CAP_TERMINATE_VM to advertise support to userspace. Use a new field in kvm_tdx_capabilities instead of adding yet another generic KVM_CAP to avoid bleeding TDX details into common code (and #ifdefs), and so that userspace can query TDX capabilities in one shot. Enumerating capabilities as a mask of bits does limit supported_caps to 64 capabilities, but in the unlikely event KVM needs to enumerate more than 64 TDX capabilities, there are another 249 u64 entries reserved for future expansion.
To preserve the KVM_BUG_ON() sanity check that deals with HKID assignment, track if a TD is terminated and assert that, when an S-EPT entry is removed, either the TD has an assigned HKID or the TD was explicitly terminated.
--- Documentation/virt/kvm/x86/intel-tdx.rst | 22 +++++++++++++- arch/x86/include/uapi/asm/kvm.h | 7 ++++- arch/x86/kvm/vmx/tdx.c | 37 +++++++++++++++++++----- arch/x86/kvm/vmx/tdx.h | 1 + 4 files changed, 57 insertions(+), 10 deletions(-)
Documentation/virt/kvm/x86/intel-tdx.rst +21 -1
@@ -38,6 +38,7 @@ ioctl with TDX specific sub-ioctl() commands. KVM_TDX_INIT_MEM_REGION, KVM_TDX_FINALIZE_VM, KVM_TDX_GET_CPUID, + KVM_TDX_TERMINATE_VM, KVM_TDX_CMD_NR_MAX, };
@@ -92,7 +93,10 @@ to be configured to the TDX guest. __u64 kernel_tdvmcallinfo_1_r12; __u64 user_tdvmcallinfo_1_r12; - __u64 reserved[250]; + /* Misc capabilities enumerated via the KVM_TDX_CAP_* namespace. */ + __u64 supported_caps; + + __u64 reserved[249]; /* Configurable CPUID bits for userspace */ struct kvm_cpuid2 cpuid;
@@ -227,6 +231,22 @@ struct kvm_cpuid2. __u32 padding[3]; }; +KVM_TDX_TERMINATE_VM +-------------------- +:Capability: KVM_TDX_CAP_TERMINATE_VM +:Type: vm ioctl +:Returns: 0 on success, <0 on error + +Release Host Key ID (HKID) to allow more efficient reclaim of private memory. +After this, the TD is no longer in a runnable state. + +Using KVM_TDX_TERMINATE_VM is optional. + +- id: KVM_TDX_TERMINATE_VM +- flags: must be 0 +- data: must be 0 +- hw_error: must be 0 + KVM TDX creation flow ===================== In addition to the standard KVM flow, new TDX ioctls need to be called. The
arch/x86/include/uapi/asm/kvm.h +6 -1
@@ -940,6 +940,7 @@ enum kvm_tdx_cmd_id {KVM_TDX_INIT_MEM_REGION ,KVM_TDX_FINALIZE_VM ,KVM_TDX_GET_CPUID , +KVM_TDX_TERMINATE_VM , KVM_TDX_CMD_NR_MAX, };
@@ -962,6 +963,8 @@ struct kvm_tdx_cmd {__u64 hw_error; }; +#define KVM_TDX_CAP_TERMINATE_VM _BITULL(0) +struct kvm_tdx_capabilities {__u64 supported_attrs;__u64 supported_xfam;
@@ -971,7 +974,9 @@ struct kvm_tdx_capabilities {__u64 kernel_tdvmcallinfo_1_r12;__u64 user_tdvmcallinfo_1_r12; -__u64 reserved[250 ]; +__u64 supported_caps; + +__u64 reserved[249 ];/* Configurable CPUID bits for userspace */ struct kvm_cpuid2 cpuid;
arch/x86/kvm/vmx/tdx.c +29 -8
@@ -188,6 +188,8 @@ static int init_kvm_tdx_caps(const struct tdx_sys_info_td_conf *td_conf,if (!caps->supported_xfam)return -EIO ; + caps->supported_caps =KVM_TDX_CAP_TERMINATE_VM ; + caps->cpuid.nent = td_conf->num_cpuid_config; caps->user_tdvmcallinfo_1_r11 =
@@ -520,6 +522,7 @@ void tdx_mmu_release_hkid(struct kvm *kvm) goto out; } +write_lock (&kvm->mmu_lock); for_each_online_cpu(i) { if (packages_allocated && cpumask_test_and_set_cpu(topology_physical_package_id(i),
@@ -544,7 +547,7 @@ void tdx_mmu_release_hkid(struct kvm *kvm) }else {tdx_hkid_free (kvm_tdx); } - +write_unlock (&kvm->mmu_lock); out:mutex_unlock (&tdx_lock);cpus_read_unlock ();
@@ -1884,13 +1887,13 @@ static int tdx_sept_remove_private_spte(struct kvm *kvm, gfn_t gfn,struct page *page =pfn_to_page (pfn);int ret; -/* -* HKID is released after all private pages have been removed, and set -* before any might be populated. Warn if zapping is attempted when -* there can't be anything populated in the private EPT. -*/ -if (KVM_BUG_ON (!is_hkid_assigned (to_kvm_tdx (kvm)), kvm)) -return -EINVAL ; +if (!is_hkid_assigned (to_kvm_tdx (kvm))) { +KVM_BUG_ON (!to_kvm_tdx (kvm)->vm_terminated, kvm); + ret =tdx_reclaim_page (page); +if (!ret) +tdx_unpin (kvm, page); +return ret; + } ret =tdx_sept_zap_private_spte (kvm, gfn, level, page);if (ret <=0 )
@@ -2884,6 +2887,21 @@ static int tdx_td_finalize(struct kvm *kvm, struct kvm_tdx_cmd *cmd)return 0 ; } +static int tdx_terminate_vm (struct kvm *kvm) +{ +if (kvm_trylock_all_vcpus (kvm)) +return -EBUSY ; + +kvm_vm_dead (kvm); +to_kvm_tdx (kvm)->vm_terminated = true; + +kvm_unlock_all_vcpus (kvm); + +tdx_mmu_release_hkid (kvm); + +return 0 ; +} +int tdx_vm_ioctl (struct kvm *kvm,void __user *argp) {struct kvm_tdx_cmd tdx_cmd;
@@ -2911,6 +2929,9 @@ int tdx_vm_ioctl(struct kvm *kvm, void __user *argp)case KVM_TDX_FINALIZE_VM : r =tdx_td_finalize (kvm, &tdx_cmd);break ; +case KVM_TDX_TERMINATE_VM : + r =tdx_terminate_vm (kvm); +break ;default : r = -EINVAL ; goto out;
arch/x86/kvm/vmx/tdx.h +1 -0
@@ -45,6 +45,7 @@ struct kvm_tdx {* Set/unset is protected with kvm->mmu_lock. */ bool wait_for_sept_zap; +bool vm_terminated; };/* TDX module vCPU states */
Signature · 1 line
2.50.1.552.g942d659e1b-goog
Re: [PATCH 5/5] KVM: TDX: Add sub-ioctl KVM_TDX_TERMINATE_VM
Quoted 4 lines · Sean Christopherson
arch/x86/kvm/vmx/tdx.c +4
-0+static int tdx_terminate_vm (struct kvm *kvm) +{ +if (kvm_trylock_all_vcpus (kvm)) +return -EBUSY ;
arch/x86/kvm/vmx/tdx.c static int tdx_td_finalize(struct kvm *kvm, struct kvm_tdx_cmd *cmd) · line 2903
Quoted 10 lines
arch/x86/kvm/vmx/tdx.c +10
-0+ +kvm_vm_dead (kvm); +to_kvm_tdx (kvm)->vm_terminated = true; + +kvm_unlock_all_vcpus (kvm); + +tdx_mmu_release_hkid (kvm); + +return 0 ; +}
As I think I mentioned when removing vm_dead first came up, I think we need more checks. I spent some time going through the code and came up with what is below:
First, we need to avoid TDX VCPU sub-IOCTLs from racing with tdx_mmu_release_hkid(). But having any TDX sub-IOCTL run after KVM_TDX_TERMINATE_VM raises questions of what might happen, so it is much simpler to understand, if that is not possible. There are 3 options:
1. Require that KVM_TDX_TERMINATE_VM is valid only if kvm_tdx->state == TD_STATE_RUNNABLE. Since currently all the TDX sub-IOCTLs are for initialization, that would block the opportunity for any to run after KVM_TDX_TERMINATE_VM.
2. Check vm_terminated in tdx_vm_ioctl() and tdx_vcpu_ioctl()
3. Test KVM_REQ_VM_DEAD in tdx_vm_ioctl() and tdx_vcpu_ioctl()
[ Note cannot check is_hkid_assigned() because that is racy ]
Secondly, I suggest we avoid SEAMCALLs that will fail and result in KVM_BUG_ON() if HKID has been released.
There are 2 groups of those: MMU-related and TDVPS_ACCESSORS.
For the MMU-related, the following 2 functions should return an error immediately if vm_terminated:
tdx_sept_link_private_spt()
tdx_sept_set_private_spte()
For that not be racy, extra synchronization is needed so that vm_terminated can be reliably checked when holding mmu lock i.e.
static int tdx_terminate_vm(struct kvm *kvm)
{
if (kvm_trylock_all_vcpus(kvm))
return -EBUSY;
kvm_vm_dead(kvm);
+
+ write_lock(&kvm->mmu_lock);
to_kvm_tdx(kvm)->vm_terminated = true;+ write_unlock(&kvm->mmu_lock);
kvm_unlock_all_vcpus(kvm); tdx_mmu_release_hkid(kvm); return 0; }
Finally, there are 2 TDVPS_ACCESSORS that need avoiding:
tdx_load_mmu_pgd()
skip td_vmcs_write64() if vm_terminated
tdx_protected_apic_has_interrupt()
skip td_state_non_arch_read64() if vm_terminated
Sean Christopherson
maintainerRe: [PATCH 5/5] KVM: TDX: Add sub-ioctl KVM_TDX_TERMINATE_VM
+Chao
Quoted 70 lines · Adrian Hunter
Quoted 4 lines · Sean Christopherson
(quoted diff) +4
-0+static int tdx_terminate_vm(struct kvm *kvm) +{ + if (kvm_trylock_all_vcpus(kvm)) + return -EBUSY;Review
Quoted 10 lines
(quoted diff) +10
-0+ + kvm_vm_dead(kvm); + to_kvm_tdx(kvm)->vm_terminated = true; + + kvm_unlock_all_vcpus(kvm); + + tdx_mmu_release_hkid(kvm); + + return 0; +}As I think I mentioned when removing
vm_deadfirst came up, I think we need more checks. I spent some time going through the code and came up with what is below:First, we need to avoid TDX VCPU sub-IOCTLs from racing with
tdx_mmu_release_hkid(). But having any TDX sub-IOCTL run afterKVM_TDX_TERMINATE_VMraises questions of what might happen, so it is much simpler to understand, if that is not possible. There are 3 options:1. Require that
KVM_TDX_TERMINATE_VMis valid only ifkvm_tdx->state==TD_STATE_RUNNABLE. Since currently all the TDX sub-IOCTLs are for initialization, that would block the opportunity for any to run afterKVM_TDX_TERMINATE_VM.2. Check
vm_terminatedintdx_vm_ioctl()andtdx_vcpu_ioctl()3. Test
KVM_REQ_VM_DEADintdx_vm_ioctl()andtdx_vcpu_ioctl()[ Note cannot check
is_hkid_assigned()because that is racy ]Secondly, I suggest we avoid SEAMCALLs that will fail and result in
KVM_BUG_ON()if HKID has been released.There are 2 groups of those: MMU-related and
TDVPS_ACCESSORS.For the MMU-related, the following 2 functions should return an error immediately if
vm_terminated:
tdx_sept_link_private_spt()
tdx_sept_set_private_spte()For that not be racy, extra synchronization is needed so that
vm_terminatedcan be reliably checked when holding mmu lock i.e.static int
tdx_terminate_vm(struct kvm *kvm){ if (kvm_trylock_all_vcpus(kvm)) return -EBUSY; kvm_vm_dead(kvm);(quoted diff) +2
-0+ + write_lock(&kvm->mmu_lock);to_kvm_tdx(kvm)->vm_terminated = true;+
write_unlock(&kvm->mmu_lock);kvm_unlock_all_vcpus(kvm); tdx_mmu_release_hkid(kvm); return 0; }
Quoted 7 lines
Finally, there are 2
TDVPS_ACCESSORSthat need avoiding:
tdx_load_mmu_pgd()
skiptd_vmcs_write64()ifvm_terminated
tdx_protected_apic_has_interrupt()
skiptd_state_non_arch_read64()ifvm_terminated
Oof. And as Chao pointed out[*], removing the vm_dead check would allow creating and running vCPUs in a dead VM, which is most definitely not desirable. Squashing the vCPU creation case is easy enough if we keep vm_dead but still generally allow ioctls, and it's probably worth doing that no matter what (to plug the hole where pending vCPU creations could succeed):
virt/kvm/kvm_main.c +5 -0
@@ -4207,6 +4207,11 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id)mutex_lock (&kvm->lock); +if (kvm->vm_dead) { + r = -EIO ; + goto unlock_vcpu_destroy; + } +if (kvm_get_vcpu_by_id (kvm, id)) { r = -EEXIST ; goto unlock_vcpu_destroy;
And then to ensure vCPUs can't do anything, check KVM_REQ_VM_DEAD after acquiring vcpu->mutex.
virt/kvm/kvm_main.c +6 -0
@@ -4433,6 +4433,12 @@ static long kvm_vcpu_ioctl(struct file *filp,if (mutex_lock_killable (&vcpu->mutex))return -EINTR ; + +if (kvm_test_request (KVM_REQ_VM_DEAD , vcpu)) { + r = -EIO ; + goto out; + } +switch (ioctl) {case KVM_RUN : {struct pid *oldpid;
That should address all TDVPS paths (I hope), and I _think_ would address all MMU-related paths as well? E.g. prefault requires a vCPU.
Disallowing (most) vCPU ioctls but not all VM ioctls on vm_dead isn't great ABI (understatement), but I think we need/want the above changes even if we keep the general vm_dead restriction. And given the extremely ad hoc behavior of taking kvm->lock for VM ioctls, trying to enforce vm_dead for "all" VM ioctls seems like a fool's errand.
So I'm leaning toward keeping "KVM: Reject ioctls only if the VM is bugged, not simply marked dead" (with a different shortlog+changelog), but keeping vm_dead (and not introducing kvm_tdx.vm_terminated).
Thoughts?
[*] https://lore.kernel.org/all/aIlzeT+yFG2Tvb3%[email protected]
Re: [PATCH 5/5] KVM: TDX: Add sub-ioctl KVM_TDX_TERMINATE_VM
Quoted 135 lines · Sean Christopherson
+Chao
Quoted 70 lines · Adrian Hunter
Quoted 4 lines · Sean Christopherson
(quoted diff) +4
-0+static int tdx_terminate_vm(struct kvm *kvm) +{ + if (kvm_trylock_all_vcpus(kvm)) + return -EBUSY;Review
Quoted 10 lines
(quoted diff) +10
-0+ + kvm_vm_dead(kvm); + to_kvm_tdx(kvm)->vm_terminated = true; + + kvm_unlock_all_vcpus(kvm); + + tdx_mmu_release_hkid(kvm); + + return 0; +}As I think I mentioned when removing
vm_deadfirst came up, I think we need more checks. I spent some time going through the code and came up with what is below:First, we need to avoid TDX VCPU sub-IOCTLs from racing with
tdx_mmu_release_hkid(). But having any TDX sub-IOCTL run afterKVM_TDX_TERMINATE_VMraises questions of what might happen, so it is much simpler to understand, if that is not possible. There are 3 options:1. Require that
KVM_TDX_TERMINATE_VMis valid only ifkvm_tdx->state==TD_STATE_RUNNABLE. Since currently all the TDX sub-IOCTLs are for initialization, that would block the opportunity for any to run afterKVM_TDX_TERMINATE_VM.2. Check
vm_terminatedintdx_vm_ioctl()andtdx_vcpu_ioctl()3. Test
KVM_REQ_VM_DEADintdx_vm_ioctl()andtdx_vcpu_ioctl()[ Note cannot check
is_hkid_assigned()because that is racy ]Secondly, I suggest we avoid SEAMCALLs that will fail and result in
KVM_BUG_ON()if HKID has been released.There are 2 groups of those: MMU-related and
TDVPS_ACCESSORS.For the MMU-related, the following 2 functions should return an error immediately if
vm_terminated:
tdx_sept_link_private_spt()
tdx_sept_set_private_spte()For that not be racy, extra synchronization is needed so that
vm_terminatedcan be reliably checked when holding mmu lock i.e.static int
tdx_terminate_vm(struct kvm *kvm){ if (kvm_trylock_all_vcpus(kvm)) return -EBUSY; kvm_vm_dead(kvm);(quoted diff) +2
-0+ + write_lock(&kvm->mmu_lock);to_kvm_tdx(kvm)->vm_terminated = true;+
write_unlock(&kvm->mmu_lock);kvm_unlock_all_vcpus(kvm); tdx_mmu_release_hkid(kvm); return 0; }Quoted 7 lines
Finally, there are 2
TDVPS_ACCESSORSthat need avoiding:
tdx_load_mmu_pgd()
skiptd_vmcs_write64()ifvm_terminated
tdx_protected_apic_has_interrupt()
skiptd_state_non_arch_read64()ifvm_terminatedOof. And as Chao pointed out[*], removing the
vm_deadcheck would allow creating and running vCPUs in a dead VM, which is most definitely not desirable. Squashing the vCPU creation case is easy enough if we keepvm_deadbut still generally allow ioctls, and it's probably worth doing that no matter what (to plug the hole where pending vCPU creations could succeed):virt/kvm/kvm_main.c +5
-0@@ -4207,6 +4207,11 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id)mutex_lock (&kvm->lock); +if (kvm->vm_dead) { + r = -EIO ; + goto unlock_vcpu_destroy; + } +if (kvm_get_vcpu_by_id (kvm, id)) { r = -EEXIST ; goto unlock_vcpu_destroy;And then to ensure vCPUs can't do anything, check
KVM_REQ_VM_DEADafter acquiringvcpu->mutex.virt/kvm/kvm_main.c +6
-0@@ -4433,6 +4433,12 @@ static long kvm_vcpu_ioctl(struct file *filp,if (mutex_lock_killable (&vcpu->mutex))return -EINTR ; + +if (kvm_test_request (KVM_REQ_VM_DEAD , vcpu)) { + r = -EIO ; + goto out; + } +switch (ioctl) {case KVM_RUN : {struct pid *oldpid;That should address all TDVPS paths (I hope), and I _think_ would address all MMU-related paths as well? E.g. prefault requires a vCPU.
Disallowing (most) vCPU ioctls but not all VM ioctls on
vm_deadisn't great ABI (understatement), but I think we need/want the above changes even if we keep the generalvm_deadrestriction. And given the extremely ad hoc behavior of takingkvm->lockfor VM ioctls, trying to enforcevm_deadfor "all" VM ioctls seems like a fool's errand.
Quoted 5 lines
So I'm leaning toward keeping "KVM: Reject ioctls only if the VM is bugged, not simply marked dead" (with a different shortlog+changelog), but keeping
vm_dead(and not introducing kvm_tdx.vm_terminated).Thoughts?
That covers the cases I listed, so it is fine by me.
Re: [PATCH 5/5] KVM: TDX: Add sub-ioctl KVM_TDX_TERMINATE_VM
In reply to Sean ChristophersonQuoted 11 lines
Oof. And as Chao pointed out[*], removing the
vm_deadcheck would allow creating and running vCPUs in a dead VM, which is most definitely not desirable. Squashing the vCPU creation case is easy enough if we keepvm_deadbut still generally allow ioctls, and it's probably worth doing that no matter what (to plug the hole where pending vCPU creations could succeed):virt/kvm/kvm_main.c +0
-0
virt/kvm/kvm_main.c static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id) · line 4214
Quoted 8 lines
virt/kvm/kvm_main.c +5
-0@@ -4207,6 +4207,11 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id)mutex_lock (&kvm->lock); +if (kvm->vm_dead) { + r = -EIO ; + goto unlock_vcpu_destroy; + } +
yes. this addresses my concern.
Quoted 35 lines
virt/kvm/kvm_main.c +0
-0if (kvm_get_vcpu_by_id (kvm, id)) { r = -EEXIST ; goto unlock_vcpu_destroy;And then to ensure vCPUs can't do anything, check
KVM_REQ_VM_DEADafter acquiringvcpu->mutex.virt/kvm/kvm_main.c +6
-0@@ -4433,6 +4433,12 @@ static long kvm_vcpu_ioctl(struct file *filp,if (mutex_lock_killable (&vcpu->mutex))return -EINTR ; + +if (kvm_test_request (KVM_REQ_VM_DEAD , vcpu)) { + r = -EIO ; + goto out; + } +switch (ioctl) {case KVM_RUN : {struct pid *oldpid;That should address all TDVPS paths (I hope), and I _think_ would address all MMU-related paths as well? E.g. prefault requires a vCPU.
Disallowing (most) vCPU ioctls but not all VM ioctls on
vm_deadisn't great ABI (understatement), but I think we need/want the above changes even if we keep the generalvm_deadrestriction. And given the extremely ad hoc behavior of takingkvm->lockfor VM ioctls, trying to enforcevm_deadfor "all" VM ioctls seems like a fool's errand.
Quoted 3 lines
So I'm leaning toward keeping "KVM: Reject ioctls only if the VM is bugged, not simply marked dead" (with a different shortlog+changelog), but keeping
vm_dead(and not introducing kvm_tdx.vm_terminated).
Sounds good to me.
With kvm_tdx.vm_terminated removed, we should consider adding a comment above the is_hkid_assigned() check in tdx_sept_remove_private_spte() to clarify that !is_hkid_assigned() indicates the guest has been terminated, allowing private pages to be reclaimed directly without zapping.