KVM: x86/mmu: Fix an sign-extension bug with mmu_seq that hangs vCPUs
PATCH 1/1 KVM: x86/mmu: Fix an sign-extension bug with mmu_seq that hangs vCPUs
Take the vCPU's mmu_seq snapshot as an "unsigned long" instead of an "int" when checking to see if a page fault is stale, as the sequence count is stored as an "unsigned long" everywhere else in KVM. This fixes a bug where KVM will effectively hang vCPUs due to always thinking page faults are stale, which results in KVM refusing to "fix" faults.
mmu_invalidate_seq (née mmu_notifier_seq) is a sequence counter used when KVM is handling page faults to detect if userspace mappings relevant to the guest were invalidated between snapshotting the counter and acquiring mmu_lock, i.e. to ensure that the userspace mapping KVM is using to resolve the page fault is fresh. If KVM sees that the counter has changed, KVM simply resumes the guest without fixing the fault.
What _should_ happen is that the source of the mmu_notifier invalidations eventually goes away, mmu_invalidate_seq becomes stable, and KVM can once again fix guest page fault(s).
But for a long-lived VM and/or a VM that the host just doesn't particularly like, it's possible for a VM to be on the receiving end of 2 billion (with a B) mmu_notifier invalidations. When that happens, bit 31 will be set in mmu_invalidate_seq. This causes the value to be turned into a 32-bit negative value when implicitly cast to an "int" by is_page_fault_stale(), and then sign-extended into a 64-bit unsigned when the signed "int" is implicitly cast back to an "unsigned long" on the call to mmu_invalidate_retry_hva().
As a result of the casting and sign-extension, given a sequence counter of e.g. 0x8002dc25, mmu_invalidate_retry_hva() ends up doing
if (0x8002dc25 != 0xffffffff8002dc25)
and signals that the page fault is stale and needs to be retried even though the sequence counter is stable, and KVM effectively hangs any vCPU that takes a page fault (EPT violation or #NPF when TDP is enabled).
Note, upstream commit ba6e3fe25543 ("KVM: x86/mmu: Grab mmu_invalidate_seq in kvm_faultin_pfn()") unknowingly fixed the bug in v6.3 when refactoring how KVM tracks the sequence counter snapshot.
--- arch/x86/kvm/mmu/mmu.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
arch/x86/kvm/mmu/mmu.c +2 -1
@@ -4212,7 +4212,8 @@ static int kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault)* root was invalidated by a memslot update or a relevant mmu_notifier fired. */ static bool is_page_fault_stale (struct kvm_vcpu *vcpu, -struct kvm_page_fault *fault,int mmu_seq) +struct kvm_page_fault *fault, +unsigned long mmu_seq) {struct kvm_mmu_page *sp =to_shadow_page (vcpu->arch.mmu->root.hpa);
base-commit: 802aacbbffe2512dce9f8f33ad99d01cfec435de
Signature · 1 line
2.42.0.rc2.253.gd59a3bf2b4-goog
Greg Kroah-Hartman
maintainerQuoted 39 lines · Sean Christopherson
Take the vCPU's
mmu_seqsnapshot as an "unsigned long" instead of an "int" when checking to see if a page fault is stale, as the sequence count is stored as an "unsigned long" everywhere else in KVM. This fixes a bug where KVM will effectively hang vCPUs due to always thinking page faults are stale, which results in KVM refusing to "fix" faults.
mmu_invalidate_seq(néemmu_notifier_seq) is a sequence counter used when KVM is handling page faults to detect if userspace mappings relevant to the guest were invalidated between snapshotting the counter and acquiringmmu_lock, i.e. to ensure that the userspace mapping KVM is using to resolve the page fault is fresh. If KVM sees that the counter has changed, KVM simply resumes the guest without fixing the fault.What _should_ happen is that the source of the
mmu_notifierinvalidations eventually goes away,mmu_invalidate_seqbecomes stable, and KVM can once again fix guest page fault(s).But for a long-lived VM and/or a VM that the host just doesn't particularly like, it's possible for a VM to be on the receiving end of 2 billion (with a B)
mmu_notifierinvalidations. When that happens, bit 31 will be set inmmu_invalidate_seq. This causes the value to be turned into a 32-bit negative value when implicitly cast to an "int" byis_page_fault_stale(), and then sign-extended into a 64-bit unsigned when the signed "int" is implicitly cast back to an "unsigned long" on the call tommu_invalidate_retry_hva().As a result of the casting and sign-extension, given a sequence counter of e.g.
0x8002dc25,mmu_invalidate_retry_hva()ends up doingif (0x8002dc25 != 0xffffffff8002dc25)and signals that the page fault is stale and needs to be retried even though the sequence counter is stable, and KVM effectively hangs any vCPU that takes a page fault (EPT violation or #NPF when TDP is enabled).
Note, upstream commit
ba6e3fe25543("KVM: x86/mmu: Grabmmu_invalidate_seqinkvm_faultin_pfn()") unknowingly fixed the bug in v6.3 when refactoring how KVM tracks the sequence counter snapshot.
Quoted 6 lines
Reported-by Brian Rak <[email protected]>Reported-by Amaan Cheval <[email protected]>Reported-by Eric Wheeler <[email protected]>Fixes a955cad84cda ("KVM: x86/mmu: Retry page fault if root is invalidated by memslot update")Signed-off-by Sean Christopherson <[email protected]>
What is the git commit id of this change in Linus's tree?
thanks,
greg k-h
Quoted 5 lines · Greg Kroah-Hartman
Quoted 4 lines · Sean Christopherson
Note, upstream commit
ba6e3fe25543("KVM: x86/mmu: Grabmmu_invalidate_seqinkvm_faultin_pfn()") unknowingly fixed the bug in v6.3 when refactoring how KVM tracks the sequence counter snapshot.
Quoted 8 lines
Quoted 6 lines
Reported-by Brian Rak <[email protected]>Reported-by Amaan Cheval <[email protected]>Reported-by Eric Wheeler <[email protected]>Fixes a955cad84cda ("KVM: x86/mmu: Retry page fault if root is invalidated by memslot update")Signed-off-by Sean Christopherson <[email protected]>What is the git commit id of this change in Linus's tree?
There is none. Commit ba6e3fe25543 (landed in v6.3) unknowingly fixed the bug as part of a completely unrelated refactoring.
Greg Kroah-Hartman
maintainerQuoted 13 lines · Sean Christopherson
Quoted 12 lines · Greg Kroah-Hartman
Quoted 10 lines · Sean Christopherson
Note, upstream commit
ba6e3fe25543("KVM: x86/mmu: Grabmmu_invalidate_seqinkvm_faultin_pfn()") unknowingly fixed the bug in v6.3 when refactoring how KVM tracks the sequence counter snapshot.Reported-by Brian Rak <[email protected]>Reported-by Amaan Cheval <[email protected]>Reported-by Eric Wheeler <[email protected]>Fixes a955cad84cda ("KVM: x86/mmu: Retry page fault if root is invalidated by memslot update")Signed-off-by Sean Christopherson <[email protected]>
Quoted 4 lines
Quoted 1 line
What is the git commit id of this change in Linus's tree?
There is none. Commit
ba6e3fe25543(landed in v6.3) unknowingly fixed the bug as part of a completely unrelated refactoring.
Ah, missed that in the text here, thanks!
greg k-h
Greg Kroah-Hartman
maintainerQuoted 14 lines · Greg Kroah-Hartman
Quoted 13 lines · Sean Christopherson
Quoted 12 lines · Greg Kroah-Hartman
Quoted 10 lines · Sean Christopherson
Note, upstream commit
ba6e3fe25543("KVM: x86/mmu: Grabmmu_invalidate_seqinkvm_faultin_pfn()") unknowingly fixed the bug in v6.3 when refactoring how KVM tracks the sequence counter snapshot.Reported-by Brian Rak <[email protected]>Reported-by Amaan Cheval <[email protected]>Reported-by Eric Wheeler <[email protected]>Fixes a955cad84cda ("KVM: x86/mmu: Retry page fault if root is invalidated by memslot update")Signed-off-by Sean Christopherson <[email protected]>
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
What is the git commit id of this change in Linus's tree?
There is none. Commit
ba6e3fe25543(landed in v6.3) unknowingly fixed the bug as part of a completely unrelated refactoring.Ah, missed that in the text here, thanks!
Now queued up, thanks.
greg k-h