KVM: x86: MMU(ish) fixes for 6.8
Sean Christopherson
reviewerTwo more MMU-related fixes for 6.8. The first, and worst, fixes a data corruption bug during live migration due to KVM failing to mark a memslot dirty when emulating an atomic access. Luckily, our userspace caught the corruption during checksumming after the final pause, but I've no idea if QEMU-based VMs have such protection.
The second fixes a long-standing, but recently exposed, issue where yielding mmu_lock to vCPUs attempting to fault in memory that is _currently_ being zapped/modified can bog down the invalidation task due it constantly yielding to vCPUS (which end up doing nothing).
The following changes since commit 9895ceeb5cd61092f147f8d611e2df575879dd6f:
Merge tag 'kvmarm-fixes-6.8-2' of git://git.kernel.org/pub/scm/linux/kernel/git/kvmarm/kvmarm into HEAD (2024-02-16 12:02:38 -0500)
are available in the Git repository at:
https://github.com/kvm-x86/linux.git tags/kvm-x86-fixes-6.8-2
for you to fetch changes up to d02c357e5bfa7dfd618b7b3015624beb71f58f1f:
KVM: x86/mmu: Retry fault before acquiring mmu_lock if mapping is changing (2024-02-23 10:14:34 -0800)
----------------------------------------------------------------
KVM x86 fixes for 6.8, round 2:
- When emulating an atomic access, mark the gfn as dirty in the memslot to fix a bug where KVM could fail to mark the slot as dirty during live migration, ultimately resulting in guest data corruption due to a dirty page not being re-copied from the source to the target.
- Check for mmu_notifier invalidation events before faulting in the pfn, and before acquiring mmu_lock, to avoid unnecessary work and lock contention. Contending mmu_lock is especially problematic on preemptible kernels, as KVM may yield mmu_lock in response to the contention, which severely degrades overall performance due to vCPUs making it difficult for the task that triggered invalidation to make forward progress.
Note, due to another kernel bug, this fix isn't limited to preemtible kernels, as any kernel built with CONFIG_PREEMPT_DYNAMIC=y will yield contended rwlocks and spinlocks.
https://lore.kernel.org/all/[email protected]
----------------------------------------------------------------
Sean Christopherson (2):
KVM: x86: Mark target gfn of emulated atomic instruction as dirty
KVM: x86/mmu: Retry fault before acquiring mmu_lock if mapping is changing
arch/x86/kvm/mmu/mmu.c | 42 ++++++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/x86.c | 10 ++++++++++ include/linux/kvm_host.h | 26 ++++++++++++++++++++++++++ 3 files changed, 78 insertions(+)
Paolo Bonzini
maintainerQuoted 16 lines · Sean Christopherson
Two more MMU-related fixes for 6.8. The first, and worst, fixes a data corruption bug during live migration due to KVM failing to mark a memslot dirty when emulating an atomic access. Luckily, our userspace caught the corruption during checksumming after the final pause, but I've no idea if QEMU-based VMs have such protection.
The second fixes a long-standing, but recently exposed, issue where yielding
mmu_lockto vCPUs attempting to fault in memory that is _currently_ being zapped/modified can bog down the invalidation task due it constantly yielding to vCPUS (which end up doing nothing).The following changes since commit
9895ceeb5cd61092f147f8d611e2df575879dd6f:Merge tag 'kvmarm-fixes-6.8-2' of
git://git.kernel.org/pub/scm/linux/kernel/git/kvmarm/kvmarminto HEAD (2024-02-16 12:02:38 -0500)
Quoted 7 lines
are available in the Git repository at:
https://github.com/kvm-x86/linux.git tags/kvm-x86-fixes-6.8-2
for you to fetch changes up to
d02c357e5bfa7dfd618b7b3015624beb71f58f1f:KVM: x86/mmu: Retry fault before acquiring
mmu_lockif mapping is changing (2024-02-23 10:14:34 -0800)
Pulled, thanks.
Paolo
Quoted 31 lines
----------------------------------------------------------------
KVM x86 fixes for 6.8, round 2:- When emulating an atomic access, mark the gfn as dirty in the memslot to fix a bug where KVM could fail to mark the slot as dirty during live migration, ultimately resulting in guest data corruption due to a dirty page not being re-copied from the source to the target.
- Check for
mmu_notifierinvalidation events before faulting in the pfn, and before acquiringmmu_lock, to avoid unnecessary work and lock contention. Contendingmmu_lockis especially problematic on preemptible kernels, as KVM may yieldmmu_lockin response to the contention, which severely degrades overall performance due to vCPUs making it difficult for the task that triggered invalidation to make forward progress.Note, due to another kernel bug, this fix isn't limited to preemtible kernels, as any kernel built with
CONFIG_PREEMPT_DYNAMIC=ywill yield contended rwlocks and spinlocks.https://lore.kernel.org/all/[email protected]----------------------------------------------------------------
Sean Christopherson (2):
KVM: x86: Mark target gfn of emulated atomic instruction as dirty
KVM: x86/mmu: Retry fault before acquiringmmu_lockif mapping is changingarch/x86/kvm/mmu/mmu.c | 42 ++++++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/x86.c | 10 ++++++++++ include/linux/kvm_host.h | 26 ++++++++++++++++++++++++++ 3 files changed, 78 insertions(+)