Skip to content

KVM: selftests: Stress save+restore and #PF (ft. nested)

Yosry Ahmed kvm lkml 5 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v5 Latest13 patches 2 repliesIn review
  2. v412 patches 8 repliesSuperseded
  3. v310 patches 22 repliesSuperseded
Earlier versions · 2
VersionPostedPatchesRepliesStatus
v210 patches6 repliesSuperseded
v18 patches3 repliesSuperseded

Changes in v5 from the cover letter

- Only set KVM_CAP_EXCEPTION_PAYLOAD when running nested [Sashiko].
- Do not intercept #PF by default for nVMX tests [Sean].
- Nits and cleanups [Sean].

Patches in v5 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/13KVM: selftests: Use __stringify() instead of custom XSTR() macrostools/testing/selftests/kvm/include/test_util.h tools/testing/selftests/kvm/x86/evmcs_smm_controls_test.c tools/testing/selftests/kvm/x86/fix_hypercall_test.c +1 more+3 -90 replies
2/13KVM: selftests: Fix RAX and RFLAGS VMCB offsets when running L2tools/testing/selftests/kvm/lib/x86/svm.c+11 -80 replies
3/13KVM: selftests: Rework GPR registers switching for SVM (and fix offsets)tools/testing/selftests/kvm/include/x86/processor.h tools/testing/selftests/kvm/lib/x86/processor.c tools/testing/selftests/kvm/lib/x86/svm.c+54 -280 replies
4/13KVM: selftests: Handle rflags save/restore for SVM in guest_regstools/testing/selftests/kvm/include/x86/processor.h tools/testing/selftests/kvm/lib/x86/svm.c+5 -50 replies
5/13KVM: selftests: Reuse GPR switching logic for nVMXtools/testing/selftests/kvm/include/x86/evmcs.h tools/testing/selftests/kvm/include/x86/vmx.h+49 -660 replies
6/13KVM: selftests: Drop HORRIFIC_L2_UCALL_CLOBBER_HACKtools/testing/selftests/kvm/lib/x86/ucall.c+2 -300 replies
7/13KVM: selftests: Add a blank line before logging assertion failurestools/testing/selftests/kvm/lib/assert.c+1 -10 replies
8/13KVM: selftests: Expose PTE masks to guests as part of an MMUtools/testing/selftests/kvm/include/x86/processor.h tools/testing/selftests/kvm/lib/x86/processor.c+13 -00 replies
9/13KVM: selftests: Do not intercept #PF by default in nVMX teststools/testing/selftests/kvm/lib/x86/vmx.c+1 -11 reply
10/13KVM: selftests: Add basic stress test for save+restore and #PF handlingtools/testing/selftests/kvm/Makefile.kvm tools/testing/selftests/kvm/include/x86/processor.h tools/testing/selftests/kvm/x86/save_restore_pf_stress_test.c+174 -00 replies
11/13KVM: selftests: Trigger save+restore randomly in the #PF stress testtools/testing/selftests/kvm/x86/save_restore_pf_stress_test.c+49 -70 replies
12/13KVM: selftests: Support running stress save+restore and #PF test in L2tools/testing/selftests/kvm/x86/save_restore_pf_stress_test.c+53 -30 replies
13/13KVM: selftests: Trigger L2->L1 exits stress save+restore and #PF testtools/testing/selftests/kvm/include/x86/processor.h tools/testing/selftests/kvm/x86/save_restore_pf_stress_test.c+45 -40 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v5 yet.

Read the v5 thread

Cover letter [PATCH v5 00/13] KVM: selftests: Stress save+restore and #PF (ft. nested)

Add a stress test for save+restore while the guest is triggering and handling #PFs, in both L1 and L2. The goal was to create a generic selftest that would catch bugs like the one fixed by commit 5c247d08bc81 ("KVM: nSVM: Use vcpu->arch.cr2 when updating vmcb12 on nested #VMEXIT"), instead of relying on high-level testing (e.g. building GCC in L2) to catch it.

The test tries to be as generic as possible by triggering #PFs in a guest and installing a proper #PF handler, while the host is continuously doing save+restore cycles. Exiting to userspace is randomly triggered by a second thread that constantly signals the vCPU thread.

Patches (1-6) are prep patches, fixing GPR and RFLAGS switching for nSVM and generalizing GPR switching to cover nVMX, which is needed for the test to run properly with nVMX. Patch 6 removes HORRIFIC_L2_UCALL_CLOBBER_HACK, as it is no longer needed. While this series does not have the "complete" fix added by commit 6783ca4105a7 ("KVM: selftests: Add a shameful hack to preserve/clobber GPRs across ucall"), it's a good step in the right direction.

Patches (7-9) add minor infrastructure improvements (assertion failure formatting and exposing MMU masks to the guest).

Patches (10-13) add the actual test. The test is first introduced as a simple (read: dummy) stress test that just explicitly syncs to userspace after each #PF handling to do save+restore, then gradually evolves to add the random signaling, nested support, and triggering L2->L1 exits. After the last patch, the test reliably reproduces the CR2 bug.

v4 -> v5:
- Only set KVM_CAP_EXCEPTION_PAYLOAD when running nested [Sashiko].
- Do not intercept #PF by default for nVMX tests [Sean].
- Nits and cleanups [Sean].

v3 -> v4:
- Switch guest_regs back to a struct and provide struct offsets as ASM constraints similar to kvm-unit-tests [Sean].
- Expose PTE masks to guests as part of a guest MMU [Sean].
- Use __stringify() from <linux/stringify.h> instead of custom STR() and XSTR() macros [Sean].
- Handle guest RFLAGS save/restore as part of guest_regs.
- Run both L0 and L2 tests sequentially by default if nested is supported, instead of requiring a '-n' command line argument [Sean].
- Minor fixups and cleanups [Sean].

v2 -> v3:
- Rebased on top of L2 stack rework in selftests.
- Fix GPR array size (i.e. NR_GUEST_REGS) [Sashiko].
- Handle evmcs_vmlaunch() and evmcs_vmresume() [Sashiko].
- Fix off-by-one assertion error in intermediate patches.
- Increase inter-signal delay from 100us to 1msec.

v1 -> v2:
- Switch guest_regs to an array, which simplifies the offsets calculation and forgoes the dependency on using OFFSET() or defining the struct offsets for assembly otherwise.
- Move page table mapping to the test (instead of a generic helper), as the helper mistakenly tried to map the entire memslot, not just page tables.
- Do not use identity mappings for page tables as it collisions with GVAs used for ELF in some cases.
- Simplify page table walking by using loops.
- Make sure the signals are ignored before creating the signaling thread [Sashiko]
- Assert that the guest actually ran and had page faults [Sashiko]
- Add a patch to fix RAX and RFLAGS offsets in run_guest() [Sashiko]
- Initialize exception_has_payload when injecting a #UD [Sashiko]
- Only check KVM_STATE_NESTED_GUEST_MODE when running in nested mode [Sashiko]

v1: https://lore.kernel.org/all/[email protected]/
v2: https://lore.kernel.org/kvm/[email protected]/
v3: https://lore.kernel.org/kvm/[email protected]/
v4: https://lore.kernel.org/kvm/[email protected]/

Yosry Ahmed (13):
KVM: selftests: Use __stringify() instead of custom XSTR() macros
KVM: selftests: Fix RAX and RFLAGS VMCB offsets when running L2
KVM: selftests: Rework GPR registers switching for SVM (and fix
offsets)
KVM: selftests: Handle rflags save/restore for SVM in guest_regs
KVM: selftests: Reuse GPR switching logic for nVMX
KVM: selftests: Drop HORRIFIC_L2_UCALL_CLOBBER_HACK
KVM: selftests: Add a blank line before logging assertion failures
KVM: selftests: Expose PTE masks to guests as part of an MMU
KVM: selftests: Do not intercept #PF by default in nVMX tests
KVM: selftests: Add basic stress test for save+restore and #PF
handling
KVM: selftests: Trigger save+restore randomly in the #PF stress test
KVM: selftests: Support running stress save+restore and #PF test in L2
KVM: selftests: Trigger L2->L1 exits stress save+restore and #PF test

 tools/testing/selftests/kvm/Makefile.kvm      |   1 +
 .../testing/selftests/kvm/include/test_util.h |   1 +
 .../testing/selftests/kvm/include/x86/evmcs.h |  46 +--
 .../selftests/kvm/include/x86/processor.h     |  52 +++-
 tools/testing/selftests/kvm/include/x86/vmx.h |  69 ++---
 tools/testing/selftests/kvm/lib/assert.c      |   2 +-
 .../testing/selftests/kvm/lib/x86/processor.c |  14 +
 tools/testing/selftests/kvm/lib/x86/svm.c     |  62 ++--
 tools/testing/selftests/kvm/lib/x86/ucall.c   |  32 +-
 tools/testing/selftests/kvm/lib/x86/vmx.c     |   2 +-
 .../kvm/x86/evmcs_smm_controls_test.c         |   5 +-
 .../selftests/kvm/x86/fix_hypercall_test.c    |   1 -
 .../kvm/x86/save_restore_pf_stress_test.c     | 288 ++++++++++++++++++
 tools/testing/selftests/kvm/x86/smm_test.c    |   5 +-
 14 files changed, 439 insertions(+), 141 deletions(-)
 create mode 100644 tools/testing/selftests/kvm/x86/save_restore_pf_stress_test.c

base-commit: 271255273d5ff348fe29d89fe4712b2f7f7907c3

Signature · 1 line
2.55.0.487.gaf234c4eb3-goog