Skip to content

KVM: x86/mmu: Allow yielding on mmu_notifier zap

Sean Christopherson kvm lkml 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest22 patches 27 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/22KVM: selftests: Check for a potential unhandled exception iff KVM_RUN succeededtools/testing/selftests/kvm/lib/kvm_util.c+2 -10 replies
2/22KVM: selftests: Rename max_guest_memory_test to mmu_stress_testtools/testing/selftests/kvm/Makefile tools/testing/selftests/kvm/mmu_stress_test.c+1 -10 replies
3/22KVM: selftests: Only muck with SREGS on x86 in mmu_stress_testtools/testing/selftests/kvm/mmu_stress_test.c+3 -30 replies
4/22KVM: selftests: Compute number of extra pages needed in mmu_stress_testtools/testing/selftests/kvm/mmu_stress_test.c+7 -13 replies
5/22KVM: selftests: Enable mmu_stress_test on arm64tools/testing/selftests/kvm/Makefile+1 -00 replies
6/22KVM: selftests: Use vcpu_arch_put_guest() in mmu_stress_testtools/testing/selftests/kvm/mmu_stress_test.c+1 -10 replies
7/22KVM: selftests: Precisely limit the number of guest loops in mmu_stress_testtools/testing/selftests/kvm/mmu_stress_test.c+19 -60 replies
8/22KVM: selftests: Add a read-only mprotect() phase to mmu_stress_testtools/testing/selftests/kvm/mmu_stress_test.c+18 -40 replies
9/22KVM: selftests: Verify KVM correctly handles mprotect(PROT_READ)tools/testing/selftests/kvm/mmu_stress_test.c+84 -33 replies
10/22KVM: x86/mmu: Move walk_slot_rmaps() up near for_each_slot_rmap_range()arch/x86/kvm/mmu/mmu.c+54 -531 reply
11/22KVM: x86/mmu: Plumb a @can_yield parameter into __walk_slot_rmaps()arch/x86/kvm/mmu/mmu.c+8 -40 replies
12/22KVM: x86/mmu: Add a helper to walk and zap rmaps for a memslotarch/x86/kvm/mmu/mmu.c+12 -30 replies
13/22KVM: x86/mmu: Honor NEED_RESCHED when zapping rmaps and blocking is allowedarch/x86/kvm/mmu/mmu.c+6 -100 replies
14/22KVM: x86/mmu: Morph kvm_handle_gfn_range() into an aging specific helperarch/x86/kvm/mmu/mmu.c+22 -462 replies
15/22KVM: x86/mmu: Fold mmu_spte_age() into kvm_rmap_age_gfn_range()arch/x86/kvm/mmu/mmu.c+22 -280 replies
16/22KVM: x86/mmu: Add KVM_RMAP_MANY to replace open coded '1' and '1ul' literalsarch/x86/kvm/mmu/mmu.c+16 -150 replies
17/22KVM: x86/mmu: Refactor low level rmap helpers to prep for walking w/o mmu_lockarch/x86/kvm/mmu/mmu.c+50 -330 replies
18/22KVM: x86/mmu: Use KVM_PAGES_PER_HPAGE() instead of an open coded equivalentarch/x86/kvm/mmu/mmu.c+1 -10 replies
19/22KVM: x86/mmu: Add infrastructure to allow walking rmaps outside of mmu_lockarch/x86/kvm/mmu/mmu.c+71 -915 replies
20/22KVM: x86/mmu: Add support for lockless walks of rmap SPTEsarch/x86/kvm/mmu/mmu.c+63 -230 replies
21/22KVM: x86/mmu: Support rmap walks without holding mmu_lock when aging gfnsarch/x86/kvm/mmu/mmu.c+59 -42 replies
22/22***HACK*** KVM: x86: Don't take mmu_lock when aging gfnsarch/x86/kvm/svm/svm.c arch/x86/kvm/vmx/vmx.c virt/kvm/kvm_main.c+8 -30 replies

Reviews trailers per patch

1 of 22 patches has a Reviewed-by or Acked-by.

PatchSean ChristophersonJames Houghton
1/22
2/22
3/22
4/22R-b
5/22
6/22
7/22
8/22
9/22
10/22
11/22
12/22
13/22
14/22
15/22
16/22
17/22
18/22
19/22
20/22
21/22
22/22Not-signed-off-by

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v1 thread

Cover letter [PATCH 00/22] KVM: x86/mmu: Allow yielding on mmu_notifier zap

The main intent of this series is to allow yielding, i.e. cond_resched(), when unmapping memory in shadow MMUs in response to an mmu_notifier invalidation. There is zero reason not to yield, and in fact I _thought_ KVM did yield, but because of how KVM grew over the years, the unmap path got left behind.

The first half of the series is reworks max_guest_memory_test into mmu_stress_test, to give some confidence in the mmu_notifier-related changes.

Oliver and Marc, there's on patch lurking in here to enable said test on arm64. It's as well tested as I can make it (and that took much longer than anticipated because arm64 hit races in the test that x86 doesn't for whatever reason).

The middle of the series reworks x86's shadow MMU logic to use the zap flow that can yield.

The last third or so is a wee bit adventurous, and is kinda of an RFC, but well tested. It's essentially prep/post work for James' MGLRU, and allows aging SPTEs in x86's shadow MMU to run outside of mmu_lock, e.g. so that nested TDP (stage-2) MMUs can participate in MGLRU.

If everything checks out, my goal is to land the selftests and yielding changes in 6.12. The aging stuff is incomplete and meaningless without James' MGLRU, I'm posting it here purely so that folks can see the end state when the mmu_notifier invalidation paths also moves to a different API.

James, the aging stuff is quite well tested (see below). Can you try working into/on-top of your MGLRU series? And if you're feeling very kind, hammer it a bit more? :-) I haven't looked at the latest ideas and/or discussion on the MGLRU series, but I'm hoping that being able to support the shadow MMU (absent the stupid eptad=0 case) in MGLRU will allow for few shenanigans, e.g. no need to toggle flags during runtime.

As for testing, I spun up a VM and ran a compilation loop and stress in the VM, while simultaneously running a small userspace program to age the VM's memory (also in an infinite loop), using the same basic methodology as access_tracking_perf_test.c (I put almost all of guest memory into a memfd and then aged only that range of memory).

I confirmed that the locking does work, e.g. that there was (infrequent) contention, and am fairly confident that the idea pans out. E.g. I hit the BUG_ON(!is_shadow_present_pte()) using that setup, which is the only reason those patches exist :-)

Sean Christopherson (22):
KVM: selftests: Check for a potential unhandled exception iff KVM_RUN
succeeded
KVM: selftests: Rename max_guest_memory_test to mmu_stress_test
KVM: selftests: Only muck with SREGS on x86 in mmu_stress_test
KVM: selftests: Compute number of extra pages needed in
mmu_stress_test
KVM: selftests: Enable mmu_stress_test on arm64
KVM: selftests: Use vcpu_arch_put_guest() in mmu_stress_test
KVM: selftests: Precisely limit the number of guest loops in
mmu_stress_test
KVM: selftests: Add a read-only mprotect() phase to mmu_stress_test
KVM: selftests: Verify KVM correctly handles mprotect(PROT_READ)
KVM: x86/mmu: Move walk_slot_rmaps() up near
for_each_slot_rmap_range()
KVM: x86/mmu: Plumb a @can_yield parameter into __walk_slot_rmaps()
KVM: x86/mmu: Add a helper to walk and zap rmaps for a memslot
KVM: x86/mmu: Honor NEED_RESCHED when zapping rmaps and blocking is
allowed
KVM: x86/mmu: Morph kvm_handle_gfn_range() into an aging specific
helper
KVM: x86/mmu: Fold mmu_spte_age() into kvm_rmap_age_gfn_range()
KVM: x86/mmu: Add KVM_RMAP_MANY to replace open coded '1' and '1ul'
literals
KVM: x86/mmu: Refactor low level rmap helpers to prep for walking w/o
mmu_lock
KVM: x86/mmu: Use KVM_PAGES_PER_HPAGE() instead of an open coded
equivalent
KVM: x86/mmu: Add infrastructure to allow walking rmaps outside of
mmu_lock
KVM: x86/mmu: Add support for lockless walks of rmap SPTEs
KVM: x86/mmu: Support rmap walks without holding mmu_lock when aging
gfns
***HACK*** KVM: x86: Don't take mmu_lock when aging gfns

 arch/x86/kvm/mmu/mmu.c                        | 527 +++++++++++-------
 arch/x86/kvm/svm/svm.c                        |   2 +
 arch/x86/kvm/vmx/vmx.c                        |   2 +
 tools/testing/selftests/kvm/Makefile          |   3 +-
 tools/testing/selftests/kvm/lib/kvm_util.c    |   3 +-
 ..._guest_memory_test.c => mmu_stress_test.c} | 144 ++++-
 virt/kvm/kvm_main.c                           |   7 +-
 7 files changed, 482 insertions(+), 206 deletions(-)
 rename tools/testing/selftests/kvm/{max_guest_memory_test.c => mmu_stress_test.c} (65%)

base-commit: 332d2c1d713e232e163386c35a3ba0c1b90df83f

Signature · 1 line
2.46.0.76.ge559c4bf1a-goog