Skip to contentSkip to conversation

KVM: Prevent module exit until all VMs are freed

PATCH New David Matlack 1 participant 3 messages Mar 9, 2022

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

v2 cover KVM: Prevent module exit until all VMs are freed

This series fixes a long-standing theoretical bug where the KVM module can be unloaded while there are still references to a struct kvm. This can be reproduced with a simple patch [1] to run some delayed work 10 seconds after a VM file descriptor is released.

This bug was originally fixed by Ben Gardon <[email protected]> in Google's kernel due to a race with an internal kernel daemon.

KVM's async_pf code looks susceptible to this race since its inception, but clearly no one has noticed. Upcoming changes to the TDP MMU will move zapping to asynchronous workers which is much more likely to hit this issue. Fix it now to close the gap in async_pf and prepare for the TDP MMU zapping changes.

While here I noticed some further cleanups that could be done in the async_pf code. It seems unnecessary for the async_pf code to grab a reference via kvm_get_kvm() because there's code to synchronously drain its queue of work in kvm_destroy_vm() -> ... -> kvm_clear_async_pf_completion_queue() (at least on x86). This is actually dead code because kvm_destroy_vm() will never be called while there are references to kvm.users_count (which the async_pf callbacks themselves hold). It seems we could either drop the reference grabbing in async_pf.c or drop the call to kvm_clear_async_pf_completion_queue().

These patches apply on the latest kvm/queue commit b13a3befc815 ("KVM:
selftests: Add test to populate a VM with the max possible guest mem") after reverting commit c9bdd0aa6800 ("KVM: allow struct kvm to outlive the file descriptors").

v2:
- Add comment explaining try variant [Sean]
- Additional fixes commit to clarify bug not limited to async_pf [Sean]
- Collect R-b tags.

v1: https://lore.kernel.org/kvm/[email protected]/

[1] To repro: Apply the following patch, run a KVM selftest, and then unload the KVM module within 10 seconds of the selftest finishing. The kernel will panic. With the fix applied, module unloading will fail until the final struct kvm reference is put.

include/linux/kvm_host.h +2 -0
index 9536ffa0473b..db827cf6a7a7 100644
@@ -771,6 +771,8 @@ struct kvm {
        struct notifier_block pm_notifier;
 #endif
        char stats_id[KVM_STATS_NAME_SIZE];
+
+       struct delayed_work run_after_vm_release_work;
 };
 
 #define kvm_err(fmt, ...) \
virt/kvm/kvm_main.c +13 -0
index 64eb99444688..35ae6d32dae5 100644
@@ -1258,12 +1258,25 @@ void kvm_put_kvm_no_destroy(struct kvm *kvm)
 }
 EXPORT_SYMBOL_GPL(kvm_put_kvm_no_destroy);
 
+static void run_after_vm_release(struct work_struct *work)
+{
+       struct delayed_work *dwork = to_delayed_work(work);
+       struct kvm *kvm = container_of(dwork, struct kvm, run_after_vm_release_work);
+
+       pr_info("I'm still alive!\n");
+       kvm_put_kvm(kvm);
+}
+
 static int kvm_vm_release(struct inode *inode, struct file *filp)
 {
        struct kvm *kvm = filp->private_data;
 
        kvm_irqfd_release(kvm);
 
+       kvm_get_kvm(kvm);
+       INIT_DELAYED_WORK(&kvm->run_after_vm_release_work, run_after_vm_release);
+       schedule_delayed_work(&kvm->run_after_vm_release_work, 10 * HZ);
+
        kvm_put_kvm(kvm);
        return 0;
 }

David Matlack (2):
KVM: Prevent module exit until all VMs are freed Revert "KVM: set owner of cpu and vm file operations"

 virt/kvm/kvm_main.c | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

base-commit: ce41d078aaa9cf15cbbb4a42878cc6160d76525e

Signature · 1 line
2.35.1.616.g0bdcbb4464-goog
#1 lore raw

PATCH v2 1/2 KVM: Prevent module exit until all VMs are freed

Tie the lifetime the KVM module to the lifetime of each VM via kvm.users_count. This way anything that grabs a reference to the VM via kvm_get_kvm() cannot accidentally outlive the KVM module.

Prior to this commit, the lifetime of the KVM module was tied to the lifetime of /dev/kvm file descriptors, VM file descriptors, and vCPU file descriptors by their respective file_operations "owner" field. This approach is insufficient because references grabbed via kvm_get_kvm() donot prevent closing any of the aforementioned file descriptors.

This fixes a long standing theoretical bug in KVM that at least affects async page faults. kvm_setup_async_pf() grabs a reference via kvm_get_kvm(), and drops it in an asynchronous work callback. Nothing prevents the VM file descriptor from being closed and the KVM module from being unloaded before this callback runs.

PPC and s390 also look broken beyond the Fixes commits listed below, but the below commits should be more than enough to guarantee inclusion in all stable kernels.

Fixes 3d3aab1b973b ("KVM: set owner of cpu and vm file operations")
[ This 2.6.29 commit was an incomplete attempt to fix this bug. ]
Fixes af585b921e5d ("KVM: Halt vcpu if page it tries to access is swapped out")
[ This 2.6.38 commit introduced async_pf and is definitely broken. ]
[ Based on a patch from Ben implemented for Google's kernel. ]
---
 virt/kvm/kvm_main.c | 9 +++++++++
 1 file changed, 9 insertions(+)
virt/kvm/kvm_main.c +9 -0
index 9581a24c3d17..e17f9fd847e0 100644
@@ -117,6 +117,8 @@ EXPORT_SYMBOL_GPL(kvm_debugfs_dir);
 
 static const struct file_operations stat_fops_per_vm;
 
+static struct file_operations kvm_chardev_ops;
+
 static long kvm_vcpu_ioctl(struct file *file, unsigned int ioctl,
 			   unsigned long arg);
 #ifdef CONFIG_KVM_COMPAT
@@ -1132,6 +1134,12 @@ static struct kvm *kvm_create_vm(unsigned long type)
 	preempt_notifier_inc();
 	kvm_init_pm_notifier(kvm);
 
+	/* Use the "try" variant to play nice with e.g. "rmmod --wait". */
+	if (!try_module_get(kvm_chardev_ops.owner)) {
+		r = -ENODEV;
+		goto out_err;
+	}
+
 	return kvm;
 
 out_err:
@@ -1221,6 +1229,7 @@ static void kvm_destroy_vm(struct kvm *kvm)
 	preempt_notifier_dec();
 	hardware_disable_all();
 	mmdrop(mm);
+	module_put(kvm_chardev_ops.owner);
 }
 
 void kvm_get_kvm(struct kvm *kvm)

base-commit: ce41d078aaa9cf15cbbb4a42878cc6160d76525e

Signature · 1 line
2.35.1.616.g0bdcbb4464-goog
#2 lore raw

PATCH v2 2/2 Revert "KVM: set owner of cpu and vm file operations"

In reply to David Matlack

This reverts commit 3d3aab1b973b01bd2a1aa46307e94a1380b1d802.

Now that the KVM module's lifetime is tied to kvm.users_count, there is no need to also tie it's lifetime to the lifetime of the VM and vCPU file descriptors.

---
 virt/kvm/kvm_main.c | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)
virt/kvm/kvm_main.c +2 -4
index e17f9fd847e0..dfbd9592eaba 100644
@@ -3673,7 +3673,7 @@ static int kvm_vcpu_release(struct inode *inode, struct file *filp)
 	return 0;
 }
 
-static struct file_operations kvm_vcpu_fops = {
+static const struct file_operations kvm_vcpu_fops = {
 	.release        = kvm_vcpu_release,
 	.unlocked_ioctl = kvm_vcpu_ioctl,
 	.mmap           = kvm_vcpu_mmap,
@@ -4724,7 +4724,7 @@ static long kvm_vm_compat_ioctl(struct file *filp,
 }
 #endif
 
-static struct file_operations kvm_vm_fops = {
+static const struct file_operations kvm_vm_fops = {
 	.release        = kvm_vm_release,
 	.unlocked_ioctl = kvm_vm_ioctl,
 	.llseek		= noop_llseek,
@@ -5731,8 +5731,6 @@ int kvm_init(void *opaque, unsigned vcpu_size, unsigned vcpu_align,
 		goto out_free_5;
 
 	kvm_chardev_ops.owner = module;
-	kvm_vm_fops.owner = module;
-	kvm_vcpu_fops.owner = module;
 
 	r = misc_register(&kvm_dev);
 	if (r) {
Signature · 1 line
2.35.1.616.g0bdcbb4464-goog