Skip to contentSkip to conversation

KVM: Harden against unpaired kvm_mmu_notifier_invalidate_range_end() calls

PATCH Applied Sean Christopherson 1 participant 2 messages Jan 10, 2024 – Jan 31, 2024 KVM

lore
Share & tools
lore

Sean Christopherson

author reviewer
#0 lore raw

PATCH 1/1 KVM: Harden against unpaired kvm_mmu_notifier_invalidate_range_end() calls

When handling the end of an mmu_notifier invalidation, WARN if mn_active_invalidate_count is already 0 do not decrement it further, i.e. avoid causing mn_active_invalidate_count to underflow/wrap. In the worst case scenario, effectively corrupting mn_active_invalidate_count could cause kvm_swap_active_memslots() to hang indefinitely.

end() calls are *supposed* to be paired with start(), i.e. underflow can only happen if there is a bug elsewhere in the kernel, but due to lack of lockdep assertions in the mmu_notifier helpers, it's all too easy for a bug to go unnoticed for some time, e.g. see the recently introduced PAGEMAP_SCAN ioctl().

Ideally, mmu_notifiers would incorporate lockdep assertions, but users of mmu_notifiers aren't required to hold any one specific lock, i.e. adding the necessary annotations to make lockdep aware of all locks that are mutally exclusive with mm_take_all_locks() isn't trivial.

---
 virt/kvm/kvm_main.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
virt/kvm/kvm_main.c +3 -1
index 10bfc88a69f7..8f03b56dafbd 100644
@@ -890,7 +890,9 @@ static void kvm_mmu_notifier_invalidate_range_end(struct mmu_notifier *mn,
 
 	/* Pairs with the increment in range_start(). */
 	spin_lock(&kvm->mn_invalidate_lock);
-	wake = (--kvm->mn_active_invalidate_count == 0);
+	if (!WARN_ON_ONCE(!kvm->mn_active_invalidate_count))
+		--kvm->mn_active_invalidate_count;
+	wake = !kvm->mn_active_invalidate_count;
 	spin_unlock(&kvm->mn_invalidate_lock);
 
 	/*

base-commit: 1c6d984f523f67ecfad1083bb04c55d91977bb15

Signature · 1 line
2.43.0.472.g3155946c3a-goog
author reviewer Applied
#1 lore raw
Quoted 6 lines · Sean Christopherson

When handling the end of an mmu_notifier invalidation, WARN if mn_active_invalidate_count is already 0 do not decrement it further, i.e. avoid causing mn_active_invalidate_count to underflow/wrap. In the worst case scenario, effectively corrupting mn_active_invalidate_count could cause kvm_swap_active_memslots() to hang indefinitely.

Quoted 7 lines

end() calls are *supposed* to be paired with start(), i.e. underflow can only happen if there is a bug elsewhere in the kernel, but due to lack of lockdep assertions in the mmu_notifier helpers, it's all too easy for a bug to go unnoticed for some time, e.g. see the recently introduced PAGEMAP_SCAN ioctl().

[...]

Applied to kvm-x86 generic, thanks!

[1/1] KVM: Harden against unpaired kvm_mmu_notifier_invalidate_range_end() calls

      https://github.com/kvm-x86/linux/commit/d489ec956583

--
https://github.com/kvm-x86/linux/tree/next