KVM: Harden against unpaired kvm_mmu_notifier_invalidate_range_end() calls
Sean Christopherson
reviewerPATCH 1/1 KVM: Harden against unpaired kvm_mmu_notifier_invalidate_range_end() calls
When handling the end of an mmu_notifier invalidation, WARN if mn_active_invalidate_count is already 0 do not decrement it further, i.e. avoid causing mn_active_invalidate_count to underflow/wrap. In the worst case scenario, effectively corrupting mn_active_invalidate_count could cause kvm_swap_active_memslots() to hang indefinitely.
end() calls are *supposed* to be paired with start(), i.e. underflow can only happen if there is a bug elsewhere in the kernel, but due to lack of lockdep assertions in the mmu_notifier helpers, it's all too easy for a bug to go unnoticed for some time, e.g. see the recently introduced PAGEMAP_SCAN ioctl().
Ideally, mmu_notifiers would incorporate lockdep assertions, but users of mmu_notifiers aren't required to hold any one specific lock, i.e. adding the necessary annotations to make lockdep aware of all locks that are mutally exclusive with mm_take_all_locks() isn't trivial.
--- virt/kvm/kvm_main.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
virt/kvm/kvm_main.c +3 -1
@@ -890,7 +890,9 @@ static void kvm_mmu_notifier_invalidate_range_end(struct mmu_notifier *mn,/* Pairs with the increment in range_start(). */ spin_lock (&kvm->mn_invalidate_lock); - wake = (--kvm->mn_active_invalidate_count ==0 ); +if (!WARN_ON_ONCE (!kvm->mn_active_invalidate_count)) + --kvm->mn_active_invalidate_count; + wake = !kvm->mn_active_invalidate_count;spin_unlock (&kvm->mn_invalidate_lock);/*
base-commit: 1c6d984f523f67ecfad1083bb04c55d91977bb15
Signature · 1 line
2.43.0.472.g3155946c3a-goog
Quoted 6 lines · Sean Christopherson
When handling the end of an
mmu_notifierinvalidation, WARN ifmn_active_invalidate_countis already 0 do not decrement it further, i.e. avoid causingmn_active_invalidate_countto underflow/wrap. In the worst case scenario, effectively corruptingmn_active_invalidate_countcould causekvm_swap_active_memslots()to hang indefinitely.
Quoted 7 lines
end()calls are *supposed* to be paired withstart(), i.e. underflow can only happen if there is a bug elsewhere in the kernel, but due to lack of lockdep assertions in themmu_notifierhelpers, it's all too easy for a bug to go unnoticed for some time, e.g. see the recently introducedPAGEMAP_SCANioctl().[...]
Applied to kvm-x86 generic, thanks!
[1/1] KVM: Harden against unpaired kvm_mmu_notifier_invalidate_range_end() calls
https://github.com/kvm-x86/linux/commit/d489ec956583