RFC KVM: x86/mmu: Allocate memslot rmaps on demand
Get series with b4
b4 am [email protected]Versions newest first
- v1 RFC Latest5 patches 0 repliesNew
Patches in v1 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/5 | KVM: x86/mmu: Walk memslot rmaps by indexarch/ | +24 | 0 replies | |
| 2/5 | KVM: x86: Count rmaps per level in a debugfs helperarch/ | +19 | 0 replies | |
| 3/5 | KVM: x86/mmu: Allocate memslot rmaps in on-demand chunksarch/ arch/ arch/ +1 more | +153 | 0 replies | |
| 4/5 | KVM: selftests: Declare __tdp_map() in the x86 processor headertools/ | +1 | 0 replies | |
| 5/5 | KVM: selftests: Test dirty logging of sparse nested mappingstools/ tools/ | +276 | 0 replies |
Reviews trailers per patch
No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.
Cover letter [RFC PATCH 0/5] KVM: x86/mmu: Allocate memslot rmaps on demand
From: Fred Griffoul <[email protected]>
When an L1 guest runs a nested guest, KVM allocates rmaps for all of L1's memory the first time it creates a shadow root, one 8-byte head per 4 KiB page. For an L1 with 4 TiB of memory that is 8 GiB, allocated at once on the first nested VM-Enter, whether L2 maps 1% or all of L1's memory. Sean pointed at this cost on Mushahid's lpage_info patch [1].
This RFC allocates rmaps in page-sized chunks instead, when the first SPTE in a chunk's range is added. Each memslot keeps a per-level directory of chunks, 8 bytes per 2 MiB at the 4 KiB level, so the 4 TiB L1 above allocates 16 MiB on the first nested VM-Enter. A missing chunk reads as empty rmaps and slot walks skip it. When L2 maps every gfn, rmaps cost what they did before plus the directory.
Patches 1 and 2 prepare the slot walk and the debugfs rmap statistics, with no functional change. Patch 3 does the chunking. Patches 4 and 5 add a selftest in which L2 writes pages around chunk boundaries and checks the dirty log after the memslot is write-protected, including eager splitting of nested huge pages.
I would like feedback on:
- The data structure. A directory of page-sized chunks keeps lookups to one extra load. An xarray would be simpler to grow but adds a tree walk to every rmap lookup.
- Freeing chunks once they are empty. That would let the shadow MMU page limit bound rmap memory too, but lockless aging reads chunks without mmu_lock, so freeing needs an RCU grace period. I left it out of this RFC.
- Doing the same for gfn_write_track, 2 bytes per 4 KiB page, which is still allocated for every memslot on the first shadow root.
Limitations:
- Chunks are only freed with the memslot.
- Eager page splitting on dirty-log enable splits L2's huge mappings, so every 2 MiB that L2 mapped with a huge page then gets a chunk.
The series is based on kvm.git next.
[1] https://lore.kernel.org/all/[email protected]/
Fred
Fred Griffoul (5):
KVM: x86/mmu: Walk memslot rmaps by index
KVM: x86: Count rmaps per level in a debugfs helper
KVM: x86/mmu: Allocate memslot rmaps in on-demand chunks
KVM: selftests: Declare __tdp_map() in the x86 processor header
KVM: selftests: Test dirty logging of sparse nested mappings
arch/x86/include/asm/kvm_host.h | 22 +- arch/x86/kvm/debugfs.c | 46 ++- arch/x86/kvm/mmu/mmu.c | 126 +++++++- arch/x86/kvm/x86.c | 22 +- tools/testing/selftests/kvm/Makefile.kvm | 1 + .../selftests/kvm/include/x86/processor.h | 1 + .../kvm/x86/nested_dirty_log_sparse_test.c | 275 ++++++++++++++++++ 7 files changed, 460 insertions(+), 33 deletions(-) create mode 100644 tools/testing/selftests/kvm/x86/nested_dirty_log_sparse_test.c
base-commit: d21eef2084a9af87ac2a783bc7c2b175ba114e4e
Signature · 1 line
2.47.3