Skip to content

KVM: x86: MMU page fault clean-up

Sean Christopherson kvm lkml 1 version Applied

Status from this message: Queued, thanks!

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest16 patches 1 replyApplied

Patches in v1 Applied

#Subject and filesSizeRepliesApplied
1/16KVM: x86: Use gpa_t for cr2/gpa to fix TDP support on 32-bit KVMarch/x86/include/asm/kvm_host.h arch/x86/kvm/mmu/mmu.c arch/x86/kvm/mmu/paging_tmpl.h +5 more+94 -780 replies
2/16KVM: x86/mmu: Move definition of make_mmu_pages_available() uparch/x86/kvm/mmu/mmu.c+20 -210 replies
3/16KVM: x86/mmu: Fold nonpaging_map() into nonpaging_page_fault()arch/x86/kvm/mmu/mmu.c+49 -570 replies
4/16KVM: x86/mmu: Move nonpaging_page_fault() below try_async_pf()arch/x86/kvm/mmu/mmu.c+69 -730 replies
5/16KVM: x86/mmu: Refactor handling of cache consistency with TDParch/x86/kvm/mmu/mmu.c+14 -160 replies
6/16KVM: x86/mmu: Refactor the per-slot level calculation in mapping_level()arch/x86/kvm/mmu/mmu.c+5 -50 replies
7/16KVM: x86/mmu: Refactor handling of forced 4k pages in page faultsarch/x86/kvm/mmu/mmu.c arch/x86/kvm/mmu/paging_tmpl.h+29 -320 replies
8/16KVM: x86/mmu: Incorporate guest's page level into max level for shadow MMUarch/x86/kvm/mmu/paging_tmpl.h+8 -120 replies
9/16KVM: x86/mmu: Persist gfn_lpage_is_disallowed() to max_levelarch/x86/kvm/mmu/mmu.c+18 -100 replies
10/16KVM: x86/mmu: Rename lpage_disallowed to account_disallowed_nx_lpagearch/x86/kvm/mmu/mmu.c+2 -20 replies
11/16KVM: x86/mmu: Consolidate tdp_page_fault() and nonpaging_page_fault()arch/x86/kvm/mmu/mmu.c+61 -1050 replies
12/16KVM: x86/mmu: Move transparent_hugepage_adjust() above __direct_map()arch/x86/kvm/mmu/mmu.c+38 -380 replies
13/16KVM: x86/mmu: Move calls to thp_adjust() down a levelarch/x86/kvm/mmu/mmu.c arch/x86/kvm/mmu/paging_tmpl.h+18 -240 replies
14/16KVM: x86/mmu: Move root_hpa validity checks to top of page fault handlerarch/x86/kvm/mmu/mmu.c+4 -100 replies
15/16KVM: x86/mmu: WARN on an invalid root_hpaarch/x86/kvm/mmu/mmu.c arch/x86/kvm/mmu/paging_tmpl.h+2 -20 replies
16/16KVM: x86/mmu: WARN if root_hpa is invalid when handling a page faultarch/x86/kvm/mmu/mmu.c+1 -30 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [PATCH 00/16] KVM: x86: MMU page fault clean-up

The original purpose of this series was to call thp_adjust() from __direct_map() and FNAME(fetch) to eliminate a page refcounting quirk[*]. Before doing that, I wanted to clean up the large page handling so that the map/fetch funtions weren't being passed multiple booleans that tracked the same basic info. While trying to decipher all the the interactions, I stumbled across a handful of fun things:

- 32-bit KVM w/ TDP is completely broken with respect to 64-bit GPAs due to the page fault handlers and all related flows dropping bits 63:32 of the GPA. As a result, KVM inserts the wrong GPA and the guest hangs because it generates EPT/NPT faults until it's killed.

- The TDP and non-paging page fault flows are identical except for one-off constraints on guest page size.

- The !VALID_PAGE(root_hpa) checks in the page fault flows are bogus. They were added a few years ago to "fix" a nVMX bug and are no longer needed now that nVMX is in much better shape.

Patch 1 fixes the 32-bit KVM w/ TDP issue. More details below.

Patches 2-12 are 99% refactoring to merge TDP and non-paging page fault handling, and to do the thp_adjust() move. These are basically nops from a functional perspective. There are technically functional changes in a few patches, but they are very superficial and in theory won't be observable in normal usage.

Patches 13-16 add WARNs on the !VALID_PAGE(root_hpa) checks to make it clear that root_hpa is expected to be valid when handling page faults, e.g. for the longest time I thought KVM relied on the checks in map/fetch to correctly handle kvm_mmu_zap_all().

32-bit KVM w/ TDP:

I marked this patch for stable because it's obviously a bug fix, but I'm entirely not sure we want to backport the fix. Obviously no userspace VMM is actually exposing 64-bit GPAs to its guests, i.e. odds are this won't actually fix any real world use cases. And, the scope of the changes are likely going to make backporting a pain. But, on the other hand, if it's not backported then future bug fixes in related code are likely to conflict, and it does fix the case where a buggy guest kernel accesses a non-existent 64-bit GPA (crashes instead of hanging indefinitely).

I'm also not confident I found all the cases where KVM is truncating the GPA. AFAIK, 32-bit Qemu simply doesn't support 64-bit GPAs. To confirm the bug and verify the fix, I hacked KVM and the guest kernel to generate 64-bit GPAs when remapping MMIO, which covers a tiny fragment of KVM.

arch/x86/kvm/x86.c +1 -1
index fa46fbed60013..49a59bcb32117 100644
@@ -5737,7 +5737,7 @@ static int emulator_read_write(struct x86_emulate_ctxt *ctxt,
        vcpu->run->mmio.len = min(8u, vcpu->mmio_fragments[0].len);
        vcpu->run->mmio.is_write = vcpu->mmio_is_write = ops->write;
        vcpu->run->exit_reason = KVM_EXIT_MMIO;
-       vcpu->run->mmio.phys_addr = gpa;
+       vcpu->run->mmio.phys_addr = gpa & 0xffffffffull;
 
        return ops->read_write_exit_mmio(vcpu, gpa, val, bytes);
 }
arch/x86/mm/ioremap.c +4 -1
index b9c78f3bcd673..e22f254987bea 100644
@@ -184,7 +184,10 @@ static void __iomem *__ioremap_caller(resource_size_t phys_addr,
        if (kernel_map_sync_memtype(phys_addr, size, pcm))
                goto err_free_area;
 
-       if (ioremap_page_range(vaddr, vaddr + size, phys_addr, prot))
+       BUG_ON(!boot_cpu_data.x86_phys_bits);
+
+       if (ioremap_page_range(vaddr, vaddr + size,
+                              phys_addr | BIT_ULL(boot_cpu_data.x86_phys_bits - 1), prot))
                goto err_free_area;
 
        ret_addr = (void __iomem *) (vaddr + offset);

[*] https://lkml.kernel.org/r/[email protected]

Sean Christopherson (16):
KVM: x86: Use gpa_t for cr2/gpa to fix TDP support on 32-bit KVM
KVM: x86/mmu: Move definition of make_mmu_pages_available() up
KVM: x86/mmu: Fold nonpaging_map() into nonpaging_page_fault()
KVM: x86/mmu: Move nonpaging_page_fault() below try_async_pf()
KVM: x86/mmu: Refactor handling of cache consistency with TDP
KVM: x86/mmu: Refactor the per-slot level calculation in
mapping_level()
KVM: x86/mmu: Refactor handling of forced 4k pages in page faults
KVM: x86/mmu: Incorporate guest's page level into max level for shadow
MMU
KVM: x86/mmu: Persist gfn_lpage_is_disallowed() to max_level
KVM: x86/mmu: Rename lpage_disallowed to account_disallowed_nx_lpage
KVM: x86/mmu: Consolidate tdp_page_fault() and nonpaging_page_fault()
KVM: x86/mmu: Move transparent_hugepage_adjust() above __direct_map()
KVM: x86/mmu: Move calls to thp_adjust() down a level
KVM: x86/mmu: Move root_hpa validity checks to top of page fault
handler
KVM: x86/mmu: WARN on an invalid root_hpa
KVM: x86/mmu: WARN if root_hpa is invalid when handling a page fault

 arch/x86/include/asm/kvm_host.h |   8 +-
 arch/x86/kvm/mmu/mmu.c          | 438 ++++++++++++++------------------
 arch/x86/kvm/mmu/paging_tmpl.h  |  58 +++--
 arch/x86/kvm/mmutrace.h         |  12 +-
 arch/x86/kvm/x86.c              |  40 ++-
 arch/x86/kvm/x86.h              |   2 +-
 include/linux/kvm_host.h        |   6 +-
 virt/kvm/async_pf.c             |  10 +-
 8 files changed, 259 insertions(+), 315 deletions(-)
Signature · 1 line
2.24.0