KVM: selftests: Stress save+restore and #PF (ft. nested)
v5 cover KVM: selftests: Stress save+restore and #PF (ft. nested)
Add a stress test for save+restore while the guest is triggering and handling #PFs, in both L1 and L2. The goal was to create a generic selftest that would catch bugs like the one fixed by commit 5c247d08bc81 ("KVM: nSVM: Use vcpu->arch.cr2 when updating vmcb12 on nested #VMEXIT"), instead of relying on high-level testing (e.g. building GCC in L2) to catch it.
The test tries to be as generic as possible by triggering #PFs in a guest and installing a proper #PF handler, while the host is continuously doing save+restore cycles. Exiting to userspace is randomly triggered by a second thread that constantly signals the vCPU thread.
Patches (1-6) are prep patches, fixing GPR and RFLAGS switching for nSVM and generalizing GPR switching to cover nVMX, which is needed for the test to run properly with nVMX. Patch 6 removes HORRIFIC_L2_UCALL_CLOBBER_HACK, as it is no longer needed. While this series does not have the "complete" fix added by commit 6783ca4105a7 ("KVM: selftests: Add a shameful hack to preserve/clobber GPRs across ucall"), it's a good step in the right direction.
Patches (7-9) add minor infrastructure improvements (assertion failure formatting and exposing MMU masks to the guest).
Patches (10-13) add the actual test. The test is first introduced as a simple (read: dummy) stress test that just explicitly syncs to userspace after each #PF handling to do save+restore, then gradually evolves to add the random signaling, nested support, and triggering L2->L1 exits. After the last patch, the test reliably reproduces the CR2 bug.
v4 -> v5:
- Only set KVM_CAP_EXCEPTION_PAYLOAD when running nested [Sashiko].
- Do not intercept #PF by default for nVMX tests [Sean].
- Nits and cleanups [Sean].
v3 -> v4:
- Switch guest_regs back to a struct and provide struct offsets as ASM constraints similar to kvm-unit-tests [Sean].
- Expose PTE masks to guests as part of a guest MMU [Sean].
- Use __stringify() from <linux/stringify.h> instead of custom STR() and XSTR() macros [Sean].
- Handle guest RFLAGS save/restore as part of guest_regs.
- Run both L0 and L2 tests sequentially by default if nested is supported, instead of requiring a '-n' command line argument [Sean].
- Minor fixups and cleanups [Sean].
v2 -> v3:
- Rebased on top of L2 stack rework in selftests.
- Fix GPR array size (i.e. NR_GUEST_REGS) [Sashiko].
- Handle evmcs_vmlaunch() and evmcs_vmresume() [Sashiko].
- Fix off-by-one assertion error in intermediate patches.
- Increase inter-signal delay from 100us to 1msec.
v1 -> v2:
- Switch guest_regs to an array, which simplifies the offsets calculation and forgoes the dependency on using OFFSET() or defining the struct offsets for assembly otherwise.
- Move page table mapping to the test (instead of a generic helper), as the helper mistakenly tried to map the entire memslot, not just page tables.
- Do not use identity mappings for page tables as it collisions with GVAs used for ELF in some cases.
- Simplify page table walking by using loops.
- Make sure the signals are ignored before creating the signaling thread [Sashiko]
- Assert that the guest actually ran and had page faults [Sashiko]
- Add a patch to fix RAX and RFLAGS offsets in run_guest() [Sashiko]
- Initialize exception_has_payload when injecting a #UD [Sashiko]
- Only check KVM_STATE_NESTED_GUEST_MODE when running in nested mode [Sashiko]
v1: https://lore.kernel.org/all/[email protected]/
v2: https://lore.kernel.org/kvm/[email protected]/
v3: https://lore.kernel.org/kvm/[email protected]/
v4: https://lore.kernel.org/kvm/[email protected]/
Yosry Ahmed (13):
KVM: selftests: Use __stringify() instead of custom XSTR() macros
KVM: selftests: Fix RAX and RFLAGS VMCB offsets when running L2
KVM: selftests: Rework GPR registers switching for SVM (and fix
offsets)
KVM: selftests: Handle rflags save/restore for SVM in guest_regs
KVM: selftests: Reuse GPR switching logic for nVMX
KVM: selftests: Drop HORRIFIC_L2_UCALL_CLOBBER_HACK
KVM: selftests: Add a blank line before logging assertion failures
KVM: selftests: Expose PTE masks to guests as part of an MMU
KVM: selftests: Do not intercept #PF by default in nVMX tests
KVM: selftests: Add basic stress test for save+restore and #PF
handling
KVM: selftests: Trigger save+restore randomly in the #PF stress test
KVM: selftests: Support running stress save+restore and #PF test in L2
KVM: selftests: Trigger L2->L1 exits stress save+restore and #PF test
tools/testing/selftests/kvm/Makefile.kvm | 1 + .../testing/selftests/kvm/include/test_util.h | 1 + .../testing/selftests/kvm/include/x86/evmcs.h | 46 +-- .../selftests/kvm/include/x86/processor.h | 52 +++- tools/testing/selftests/kvm/include/x86/vmx.h | 69 ++--- tools/testing/selftests/kvm/lib/assert.c | 2 +- .../testing/selftests/kvm/lib/x86/processor.c | 14 + tools/testing/selftests/kvm/lib/x86/svm.c | 62 ++-- tools/testing/selftests/kvm/lib/x86/ucall.c | 32 +- tools/testing/selftests/kvm/lib/x86/vmx.c | 2 +- .../kvm/x86/evmcs_smm_controls_test.c | 5 +- .../selftests/kvm/x86/fix_hypercall_test.c | 1 - .../kvm/x86/save_restore_pf_stress_test.c | 288 ++++++++++++++++++ tools/testing/selftests/kvm/x86/smm_test.c | 5 +- 14 files changed, 439 insertions(+), 141 deletions(-) create mode 100644 tools/testing/selftests/kvm/x86/save_restore_pf_stress_test.c
base-commit: 271255273d5ff348fe29d89fe4712b2f7f7907c3
Signature · 1 line
2.55.0.487.gaf234c4eb3-goog
PATCH v5 1/13 KVM: selftests: Use __stringify() instead of custom XSTR() macros
Drop the custom defined XSTR() macros in KVM selftests and use __stringify() instead. Include stringify.h in test_util.h to make it available for all tests instead of including it in all the tests that need it, as more tests will start using it.
No functional change intended.
--- tools/testing/selftests/kvm/include/test_util.h | 1 + tools/testing/selftests/kvm/x86/evmcs_smm_controls_test.c | 5 +---- tools/testing/selftests/kvm/x86/fix_hypercall_test.c | 1 - tools/testing/selftests/kvm/x86/smm_test.c | 5 +---- 4 files changed, 3 insertions(+), 9 deletions(-)
tools/testing/selftests/kvm/include/test_util.h +1 -0
@@ -23,6 +23,7 @@#include <linux/mman.h> #include <linux/types.h> +#include <linux/stringify.h> #define msecs_to_usecs (msec) ((msec) * 1000ULL)
tools/testing/selftests/kvm/x86/evmcs_smm_controls_test.c +1 -4
@@ -22,9 +22,6 @@#define SYNC_PORT 0xe -#define STR (x) #x -#define XSTR (s) STR(s) -/* * SMI handler: runs in real-address mode. * Reports SMRAM_STAGE via port IO, then does RSM.
@@ -37,7 +34,7 @@ static u8 smi_handler[] = {static inline void sync_with_host (u64 phase) { - asm volatile("in $" XSTR(SYNC_PORT) ", %%al \n" + asm volatile("in $" __stringify(SYNC_PORT) ", %%al \n" : "+a" (phase)); }
tools/testing/selftests/kvm/x86/fix_hypercall_test.c +0 -1
@@ -6,7 +6,6 @@*/ #include <asm/kvm_para.h> #include <linux/kvm_para.h> -#include <linux/stringify.h> #include <stdint.h> #include "kvm_test_harness.h"
tools/testing/selftests/kvm/x86/smm_test.c +1 -4
@@ -22,9 +22,6 @@#define SMRAM_GPA 0x1000000#define SMRAM_STAGE 0xfe -#define STR (x) #x -#define XSTR (s) STR(s) -#define SYNC_PORT 0xe#define DONE 0xff
@@ -42,7 +39,7 @@ u8 smi_handler[] = {static inline void sync_with_host (u64 phase) { - asm volatile("in $" XSTR(SYNC_PORT)", %%al \n" + asm volatile("in $" __stringify(SYNC_PORT)", %%al \n" : "+a" (phase)); }
Signature · 1 line
2.55.0.487.gaf234c4eb3-goog
PATCH v5 2/13 KVM: selftests: Fix RAX and RFLAGS VMCB offsets when running L2
In reply to Yosry AhmedThe offsets used (0x170 and 0x1f8) are offsets within vmcb_save_area, not vmcb. The correct offsets should include the base of vmcb_save_area within vmcb (which is 0x400 -- so 0x570 and 0x5f8).
Instead of just correcting the offsets, use vmcb->save.rax and vmcb->save.rflags as parameters to the asm block and avoid hardcoding offsets completely. While at it, also use guest_regs.rax directly instead of assuming it's at offset 0 of guest_regs.
Note: "+m" must be used for vmcb_rax and vmcb_rflags, as caching those fields in registers would be wrong as the underlying KVM will update them in memory.
The same problem was recently fixed (differently) for kvm-unit-tests [1].
[1]https://lore.kernel.org/all/[email protected]/
--- tools/testing/selftests/kvm/lib/x86/svm.c | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-)
tools/testing/selftests/kvm/lib/x86/svm.c +11 -8
@@ -164,19 +164,22 @@ void run_guest(struct vmcb *vmcb, u64 vmcb_gpa) { asmvolatile ("vmload %[vmcb_gpa]\n\t" -"mov rflags, %%r15\n\t" // rflags -"mov %%r15, 0x170(%[vmcb])\n\t" -"mov guest_regs, %%r15\n\t" // rax -"mov %%r15, 0x1f8(%[vmcb])\n\t" +"mov rflags, %%r15\n\t" +"mov %%r15, %[vmcb_rflags]\n\t" +"mov %[guest_regs_rax], %%r15\n\t" +"mov %%r15, %[vmcb_rax]\n\t" LOAD_GPR_C "vmrun %[vmcb_gpa]\n\t" SAVE_GPR_C -"mov 0x170(%[vmcb]), %%r15\n\t" // rflags +"mov %[vmcb_rflags], %%r15\n\t" "mov %%r15, rflags\n\t" -"mov 0x1f8(%[vmcb]), %%r15\n\t" // rax -"mov %%r15, guest_regs\n\t" +"mov %[vmcb_rax], %%r15\n\t" // rax +"mov %%r15, %[guest_regs_rax]\n\t" "vmsave %[vmcb_gpa]\n\t" - : : [vmcb]"r" (vmcb), [vmcb_gpa]"a" (vmcb_gpa) + : [vmcb_rflags]"+m" (vmcb->save.rflags), + [vmcb_rax]"+m" (vmcb->save.rax), + [guest_regs_rax]"+rm" (guest_regs.rax) + : [vmcb_gpa]"a" (vmcb_gpa) :"r15" ,"memory" ); }
Signature · 1 line
2.55.0.487.gaf234c4eb3-goog
[email protected]
AI reviewRe: [PATCH v5 02/13] KVM: selftests: Fix RAX and RFLAGS VMCB offsets when running L2
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [High] The inline assembly macros LOAD_GPR_C and SAVE_GPR_C use incorrect offsets to access guest_regs, shifted by 8 bytes. This corrupts guest registers and causes an out-of-bounds access that clobbers the rflags variable. --
--- Patch [2]: [PATCH v5 02/13] KVM: selftests: Fix RAX and RFLAGS VMCB offsets when running L2 --- commit b67ef5c1f8e53b1e2adf72da7c94eb12e46b483e
Author: Yosry Ahmed <[email protected]>
KVM: selftests: Fix RAX and RFLAGS VMCB offsets when running L2
This commit corrects the offsets for rax and rflags within the VMCB save area. It uses the vmcb->save.rax and vmcb->save.rflags C variables directly as parameters in the inline assembly, avoiding hardcoded miscalculated offsets.
Quoted 7 lines
tools/testing/selftests/kvm/lib/x86/svm.c +0
-0@@ -164,19 +164,22 @@ void run_guest(struct vmcb *vmcb, u64 vmcb_gpa) { asm volatile (
[ ... ]
tools/testing/selftests/kvm/lib/x86/svm.c void run_guest(struct vmcb *vmcb, u64 vmcb_gpa) · line 173
Quoted 3 lines
tools/testing/selftests/kvm/lib/x86/svm.c +0
-0LOAD_GPR_C "vmrun %[vmcb_gpa]\n\t" SAVE_GPR_C
[Severity: High]
This is a pre-existing issue, but I noticed an out-of-bounds memory access in tools/testing/selftests/kvm/lib/x86/svm.c related to the SAVE_GPR_C macro used here in run_guest().
The macro at line 135 contains the instruction "xchg %%r15, guest_regs+0x80".
Since struct gpr64_regs is exactly 128 bytes (0x80) long, doesn't this instruction write past the end of the guest_regs structure and overwrite the adjacent rflags variable?
Could the offsets in LOAD_GPR_C and SAVE_GPR_C be shifted by 8 bytes, causing them to initialize guest registers from the wrong offsets and corrupt memory?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
PATCH v5 3/13 KVM: selftests: Rework GPR registers switching for SVM (and fix offsets)
In reply to Yosry AhmedThe assembly code defined by SAVE_GPR_C uses the wrong offsets for some registers in guest_regs. For example, the offset of RCX should be 0x08 not 0x10. Also, the last offset in the struct (R15) is 0x78, not 0x80, so the code actually saves and restore beyond the end of gpr64_regs.
Eliminate hardcoded offsets by dynamically generating offsets using offset_of() and using macros to pass the offsets to assembly as asm constraints.
To avoid register conflicts in inline assembly (since almost all GPRs are context-switched), access guest_regs via absolute symbol addressing (guest_regs + offset) rather than using a base register which could get overwritten mid-assembly.
While at it, rename SAVE_GPR_C and LOAD_GPR_C to a single macro, SVM_SWITCH_GPRS_ASM, rename gpr64_regs to guest_regs, and expose it in processor.h (in preparation for reusing it for VMX).
--- .../selftests/kvm/include/x86/processor.h | 31 +++++++++++- .../testing/selftests/kvm/lib/x86/processor.c | 2 + tools/testing/selftests/kvm/lib/x86/svm.c | 49 +++++++++---------- 3 files changed, 54 insertions(+), 28 deletions(-)
tools/testing/selftests/kvm/include/x86/processor.h +29 -2
@@ -398,8 +398,7 @@ static inline unsigned int x86_model(unsigned int eax)#define PTE_GET_PA (pte) ((pte) & PHYSICAL_PAGE_MASK)#define PTE_GET_PFN (pte) (PTE_GET_PA(pte) >> PAGE_SHIFT) -/* General Registers in 64-Bit Mode */ -struct gpr64_regs { +struct guest_regs {u64 rax;u64 rcx;u64 rdx;
@@ -418,6 +417,34 @@ struct gpr64_regs {u64 r15; }; +extern struct guest_regs guest_regs; + +#define GUEST_REG_OFFSET (name) \ + [off_##name] "i" (offsetof(struct guest_regs, name)) + +#define GUEST_REGS_OFFSETS \ + GUEST_REG_OFFSET(rax), \ + GUEST_REG_OFFSET(rcx), \ + GUEST_REG_OFFSET(rdx), \ + GUEST_REG_OFFSET(rbx), \ + GUEST_REG_OFFSET(rsp), \ + GUEST_REG_OFFSET(rbp), \ + GUEST_REG_OFFSET(rsi), \ + GUEST_REG_OFFSET(rdi), \ + GUEST_REG_OFFSET(r8), \ + GUEST_REG_OFFSET(r9), \ + GUEST_REG_OFFSET(r10), \ + GUEST_REG_OFFSET(r11), \ + GUEST_REG_OFFSET(r12), \ + GUEST_REG_OFFSET(r13), \ + GUEST_REG_OFFSET(r14), \ + GUEST_REG_OFFSET(r15) + +#define GUEST_REG (name) "guest_regs + %c[off_" #name "]" + +#define GUEST_SWITCH_GPR_ASM (name) \ + "xchg %%" #name ", " GUEST_REG(name) "\n\t" +struct desc64 {u16 limit0;u16 base0;
tools/testing/selftests/kvm/lib/x86/processor.c +2 -0
@@ -29,6 +29,8 @@ bool host_cpu_is_amd_compatible;bool is_forced_emulation_enabled;u64 guest_tsc_khz; +struct guest_regs guest_regs; +const char *ex_str (int vector) {switch (vector) {
tools/testing/selftests/kvm/lib/x86/svm.c +23 -26
77 lines not on this page · raw message
Signature · 1 line
2.55.0.487.gaf234c4eb3-goog
PATCH v5 4/13 KVM: selftests: Handle rflags save/restore for SVM in guest_regs
In reply to Yosry AhmedInstead of handling rflags separately, add it to guest_regs. No functional change intended.
--- tools/testing/selftests/kvm/include/x86/processor.h | 4 +++- tools/testing/selftests/kvm/lib/x86/svm.c | 6 ++---- 2 files changed, 5 insertions(+), 5 deletions(-)
tools/testing/selftests/kvm/include/x86/processor.h +3 -1
@@ -415,6 +415,7 @@ struct guest_regs {u64 r13;u64 r14;u64 r15; +u64 rflags; };extern struct guest_regs guest_regs;
@@ -438,7 +439,8 @@ extern struct guest_regs guest_regs;GUEST_REG_OFFSET (r12), \GUEST_REG_OFFSET (r13), \GUEST_REG_OFFSET (r14), \ - GUEST_REG_OFFSET(r15) +GUEST_REG_OFFSET (r15), \ + GUEST_REG_OFFSET(rflags)#define GUEST_REG (name) "guest_regs + %c[off_" #name "]"
tools/testing/selftests/kvm/lib/x86/svm.c +2 -4
@@ -13,8 +13,6 @@#define SEV_DEV_PATH "/dev/sev" -u64 rflags; -/* Allocate memory regions for nested SVM tests. * * Input Args:
@@ -161,7 +159,7 @@ void run_guest(struct vmcb *vmcb, u64 vmcb_gpa) { asmvolatile ("vmload %[vmcb_gpa]\n\t" -"mov rflags, %%r15\n\t" + "mov " GUEST_REG(rflags) ", %%r15\n\t""mov %%r15, %[vmcb_rflags]\n\t" "mov " GUEST_REG(rax) ", %%r15\n\t""mov %%r15, %[vmcb_rax]\n\t"
@@ -169,7 +167,7 @@ void run_guest(struct vmcb *vmcb, u64 vmcb_gpa)"vmrun %[vmcb_gpa]\n\t" SVM_SWITCH_GPRS_ASM "mov %[vmcb_rflags], %%r15\n\t" -"mov %%r15, rflags\n\t" + "mov %%r15, " GUEST_REG(rflags) "\n\t""mov %[vmcb_rax], %%r15\n\t" // rax "mov %%r15, " GUEST_REG(rax) "\n\t""vmsave %[vmcb_gpa]\n\t"
Signature · 1 line
2.55.0.487.gaf234c4eb3-goog
PATCH v5 5/13 KVM: selftests: Reuse GPR switching logic for nVMX
In reply to Yosry AhmedReuse the GPR switching logic for nVMX by defining VMX_SWITCH_GPRS_ASM, which is essentially the same as SVM_SWITCH_GPRS_ASM but also switches RAX and doesn't switch RFLAGS, replacing the push/pop of a subset of the registers.
The long clobber list of registers is no longer needed as registers are saved and restored appropriately (and not clobbered by L2).
Define VMX_SWITCH_GPRS_ASM before including evmcs.h, such that it can be used by evmcs_vmlaunch() and evmcs_vmresume().
This replaces the apparently thread-safe push/pop sequence with the global GPR switching logic used by SVM, which isn't thread-safe at all.
However this is still an improvement because:
- The VMX logic is half-baked and prompts the UCALL clobber hack as it doesn't properly save/restore everything. Reusing the GPR switching logic used by SVM allows for dropping that hack.
- Hitting a problem due to half-baked GPR save/restore logic is arguably more likely than thread-safety. Evidently, adding more involved stress tests fails on VMX with the existing push/pop sequence. OTOH, there are no known failures on SVM due to lack of thread-safety fo save/restore. Only one test currently uses more than one vCPU with nested (the memstress test).
The logical next step is to move the guest_regs to be per-vCPU, making it thread-safe for both VMX and SVM in a proper way.
--- .../testing/selftests/kvm/include/x86/evmcs.h | 46 +++++-------- tools/testing/selftests/kvm/include/x86/vmx.h | 69 +++++++++---------- 2 files changed, 49 insertions(+), 66 deletions(-)
tools/testing/selftests/kvm/include/x86/evmcs.h +16 -30
@@ -1207,30 +1207,23 @@ static inline int evmcs_vmlaunch(void) current_evmcs->hv_clean_fields =0 ; - __asm__ __volatile__("push %%rbp;" -"push %%rcx;" -"push %%rdx;" -"push %%rsi;" -"push %%rdi;" -"push $0;" + __asm__ __volatile__("push $0;" "mov %%rsp, (%[host_rsp]);" "lea 1f(%%rip), %%rax;" "mov %%rax, (%[host_rip]);" +VMX_SWITCH_GPRS_ASM "vmlaunch;" "incq (%%rsp);" -"1: pop %%rax;" -"pop %%rdi;" -"pop %%rsi;" -"pop %%rdx;" -"pop %%rcx;" -"pop %%rbp;" +"1: ;" +VMX_SWITCH_GPRS_ASM +"pop %%rax;" : [ret]"=&a" (ret) : [host_rsp]"r" ((u64)¤t_evmcs->host_rsp), [host_rip]"r" - ((u64)¤t_evmcs->host_rip) - :"memory" ,"cc" ,"rbx" ,"r8" ,"r9" ,"r10" , -"r11" ,"r12" ,"r13" ,"r14" ,"r15" ); + ((u64)¤t_evmcs->host_rip), + GUEST_REGS_OFFSETS + :"memory" ,"cc" );return ret; }
@@ -1246,30 +1239,23 @@ static inline int evmcs_vmresume(void)/* HOST_RSP */ current_evmcs->hv_clean_fields &= ~HV_VMX_ENLIGHTENED_CLEAN_FIELD_HOST_POINTER ; - __asm__ __volatile__("push %%rbp;" -"push %%rcx;" -"push %%rdx;" -"push %%rsi;" -"push %%rdi;" -"push $0;" + __asm__ __volatile__("push $0;" "mov %%rsp, (%[host_rsp]);" "lea 1f(%%rip), %%rax;" "mov %%rax, (%[host_rip]);" +VMX_SWITCH_GPRS_ASM "vmresume;" "incq (%%rsp);" -"1: pop %%rax;" -"pop %%rdi;" -"pop %%rsi;" -"pop %%rdx;" -"pop %%rcx;" -"pop %%rbp;" +"1: ;" +VMX_SWITCH_GPRS_ASM +"pop %%rax;" : [ret]"=&a" (ret) : [host_rsp]"r" ((u64)¤t_evmcs->host_rsp), [host_rip]"r" - ((u64)¤t_evmcs->host_rip) - :"memory" ,"cc" ,"rbx" ,"r8" ,"r9" ,"r10" , -"r11" ,"r12" ,"r13" ,"r14" ,"r15" ); + ((u64)¤t_evmcs->host_rip), + GUEST_REGS_OFFSETS + :"memory" ,"cc" );return ret; }
tools/testing/selftests/kvm/include/x86/vmx.h +33 -36
110 lines not on this page · raw message
Signature · 1 line
2.55.0.487.gaf234c4eb3-goog
[email protected]
AI reviewRe: [PATCH v5 05/13] KVM: selftests: Reuse GPR switching logic for nVMX
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] The newly introduced VMX_SWITCH_GPRS_ASM macro uses a shared global variable `guest_regs` for…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The newly introduced VMX_SWITCH_GPRS_ASM macro uses a shared global variable guest_regs for swapping host and guest general-purpose registers, introducing a race condition when multiple vCPUs execute nested VMX instructions concurrently. --
--- Patch [5]: [PATCH v5 05/13] KVM: selftests: Reuse GPR switching logic for nVMX --- commit a0021d28e505a4c708c955c20d96663fad9fc879
Author: Yosry Ahmed <[email protected]>
KVM: selftests: Reuse GPR switching logic for nVMX
This commit reuses the SVM GPR switching logic for nVMX by defining VMX_SWITCH_GPRS_ASM. It replaces the previous push/pop sequence with macro calls that swap a broader set of registers using GUEST_SWITCH_GPR_ASM.
tools/testing/selftests/kvm/include/x86/vmx.h struct vmx_msr_entry { · line 295
Quoted 10 lines
tools/testing/selftests/kvm/include/x86/vmx.h +3
-0@@ -290,6 +290,23 @@ struct vmx_msr_entry {u64 value; } __attribute__ ((aligned(16))); +#define VMX_SWITCH_GPRS_ASM \ + GUEST_SWITCH_GPR_ASM(rax) \ + GUEST_SWITCH_GPR_ASM(rbx) \
[Severity: Medium]
Since GUEST_SWITCH_GPR_ASM() uses the global guest_regs structure, will this cause a data race when multiple vCPUs execute nested VMX instructions concurrently?
When vmlaunch or vmresume is executed concurrently by multiple threads (such as in the memstress test), they might race to read and write to this shared variable, potentially corrupting host callee-saved registers like rbx or r12. Could this lead to host test code crashes when one vCPU restores another vCPU's parked host registers?
Quoted 2 lines
tools/testing/selftests/kvm/include/x86/vmx.h +2
-0+GUEST_SWITCH_GPR_ASM (rcx) \ +GUEST_SWITCH_GPR_ASM (rdx) \
[ ... ]
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
PATCH v5 6/13 KVM: selftests: Drop HORRIFIC_L2_UCALL_CLOBBER_HACK
In reply to Yosry AhmedNow that nVMX test codes preserves GPRs across nested VM-Exits (specifically RBP, RDX, and RDI among others), drop the ucall-specific hack to avoid clobbering these registers.
--- tools/testing/selftests/kvm/lib/x86/ucall.c | 32 ++------------------- 1 file changed, 2 insertions(+), 30 deletions(-)
tools/testing/selftests/kvm/lib/x86/ucall.c +2 -30
@@ -10,36 +10,8 @@void ucall_arch_do_ucall (gva_t uc) { -/* -* FIXME: Revert this hack (the entire commit that added it) once nVMX -* preserves L2 GPRs across a nested VM-Exit. If a ucall from L2, e.g. -* to do a GUEST_SYNC(), lands the vCPU in L1, any and all GPRs can be -* clobbered by L1. Save and restore non-volatile GPRs (clobbering RBP -* in particular is problematic) along with RDX and RDI (which are -* inputs), and clobber volatile GPRs. *sigh* -*/ -#define HORRIFIC_L2_UCALL_CLOBBER_HACK \ - "rcx", "rsi", "r8", "r9", "r10", "r11" - - asmvolatile ("push %%rbp\n\t" -"push %%r15\n\t" -"push %%r14\n\t" -"push %%r13\n\t" -"push %%r12\n\t" -"push %%rbx\n\t" -"push %%rdx\n\t" -"push %%rdi\n\t" -"in %[port], %%al\n\t" -"pop %%rdi\n\t" -"pop %%rdx\n\t" -"pop %%rbx\n\t" -"pop %%r12\n\t" -"pop %%r13\n\t" -"pop %%r14\n\t" -"pop %%r15\n\t" -"pop %%rbp\n\t" - : : [port] "d" (UCALL_PIO_PORT), "D" (uc) : "rax", "memory", - HORRIFIC_L2_UCALL_CLOBBER_HACK); + asmvolatile ("in %[port], %%al" + : : [port]"d" (UCALL_PIO_PORT ),"D" (uc) :"rax" ,"memory" ); }void *ucall_arch_get_ucall (struct kvm_vcpu *vcpu)
Signature · 1 line
2.55.0.487.gaf234c4eb3-goog
[email protected]
AI reviewRe: [PATCH v5 06/13] KVM: selftests: Drop HORRIFIC_L2_UCALL_CLOBBER_HACK
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] Data race causing register corruption in multi-vCPU nested tests due to globally shared…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] Data race causing register corruption in multi-vCPU nested tests due to globally shared guest_regs structure. --
--- Patch [6]: [PATCH v5 06/13] KVM: selftests: Drop HORRIFIC_L2_UCALL_CLOBBER_HACK --- commit 9a9a1bcc6fc0c85786cd983d4fae11f1dfafdf51
Author: Yosry Ahmed <[email protected]>
KVM: selftests: Drop HORRIFIC_L2_UCALL_CLOBBER_HACK
This commit drops the hack in ucall_arch_do_ucall(), as the nVMX test code now preserves GPRs across nested VM-Exits.
Quoted 8 lines
tools/testing/selftests/kvm/lib/x86/ucall.c +0
-0@@ -10,36 +10,8 @@void ucall_arch_do_ucall (gva_t uc) {
[ ... ]
tools/testing/selftests/kvm/lib/x86/ucall.c line 15
Quoted 7 lines
tools/testing/selftests/kvm/lib/x86/ucall.c +2
-4-"pop %%r15\n\t" -"pop %%rbp\n\t" - : : [port] "d" (UCALL_PIO_PORT), "D" (uc) : "rax", "memory", - HORRIFIC_L2_UCALL_CLOBBER_HACK); + asmvolatile ("in %[port], %%al" + : : [port]"d" (UCALL_PIO_PORT ),"D" (uc) :"rax" ,"memory" ); }
[Severity: Medium]
Does dropping this hack introduce a regression for multi-vCPU nested tests where L2 registers could get clobbered?
The nVMX code that now preserves GPRs appears to use a globally shared structure:
tools/testing/selftests/kvm/lib/x86/processor.c:
struct guest_regs guest_regs;
The VMX_SWITCH_GPRS_ASM macro uses this struct during nested VM entry and exit:
tools/testing/selftests/kvm/include/x86/vmx.h:
#define VMX_SWITCH_GPRS_ASM \ GUEST_SWITCH_GPR_ASM(rax) \ GUEST_SWITCH_GPR_ASM(rbx) \
...
Because guest_regs is not thread-local, wouldn't concurrent vCPU threads running vmlaunch or vmresume clobber each other's register state when accessing this shared struct?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=6
PATCH v5 7/13 KVM: selftests: Add a blank line before logging assertion failures
In reply to Yosry AhmedAdd a blank line for visual separation, which is especially useful for tests that use carriage returns for same-line printing, in case a failure occurs before a blank line is ever logged by the test.
--- tools/testing/selftests/kvm/lib/assert.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/selftests/kvm/lib/assert.c +1 -1
@@ -74,7 +74,7 @@ test_assert(bool exp, const char *exp_str,if (!(exp)) {va_start (ap, fmt); - fprintf(stderr, "==== Test Assertion Failure ====\n" + fprintf(stderr, "\n==== Test Assertion Failure ====\n" " %s:%u: %s\n" " pid=%d tid=%d errno=%d - %s\n", file, line, exp_str, getpid(), kvm_gettid(),
Signature · 1 line
2.55.0.487.gaf234c4eb3-goog
PATCH v5 8/13 KVM: selftests: Expose PTE masks to guests as part of an MMU
In reply to Yosry AhmedExpose a guest_mmu to the guest to allow guest code to use the PTE masks for page table manipulation. Since guest page tables are not mapped in the guest by default, zero the PGD in guest_mmu in an attempt to make it more difficult for new tests to shoot themselves in the foot and assume that page tables can be immediately used by guest code.
Ultimately, guest code can read CR3 any way, so guest_mmu.pgd doesn't add a lot of value.
--- tools/testing/selftests/kvm/include/x86/processor.h | 1 + tools/testing/selftests/kvm/lib/x86/processor.c | 12 ++++++++++++ 2 files changed, 13 insertions(+)
tools/testing/selftests/kvm/include/x86/processor.h +1 -0
@@ -24,6 +24,7 @@ extern bool host_cpu_is_amd;extern bool host_cpu_is_hygon;extern bool host_cpu_is_amd_compatible;extern u64 guest_tsc_khz; +extern struct kvm_mmu guest_mmu;#ifndef MAX_NR_CPUID_ENTRIES #define MAX_NR_CPUID_ENTRIES 100
tools/testing/selftests/kvm/lib/x86/processor.c +12 -0
@@ -28,6 +28,7 @@ bool host_cpu_is_hygon;bool host_cpu_is_amd_compatible;bool is_forced_emulation_enabled;u64 guest_tsc_khz; +struct kvm_mmu guest_mmu;struct guest_regs guest_regs;
@@ -831,6 +832,17 @@ void kvm_arch_vm_post_create(struct kvm_vm *vm, unsigned int nr_vcpus)TEST_ASSERT (r >0 ,"KVM_GET_TSC_KHZ did not provide a valid TSC frequency." ); guest_tsc_khz = r;sync_global_to_guest (vm, guest_tsc_khz); + +/* +* The guest MMU is just a placeholder to provide access to PTE masks +* (for now). The guest does not have mappings for its own page tables +* by default, so any meaningful use of guest page tables requires +* explicit setup by the test. Zero the PGD to make it obvious the guest +* page tables are not immediately usable by guest code. +*/ + guest_mmu = vm->mmu; + guest_mmu.pgd =0 ; +sync_global_to_guest (vm, guest_mmu); }void vcpu_arch_set_entry_point (struct kvm_vcpu *vcpu,void *guest_code)
Signature · 1 line
2.55.0.487.gaf234c4eb3-goog
PATCH v5 9/13 KVM: selftests: Do not intercept #PF by default in nVMX tests
In reply to Yosry Ahmedinit_vmcs_control_fields() sets PFEC_MASK and PFEC_MATCH so that they never match, which reverses the meaning of the PF_VECTOR bit in EXCEPTION_BITMAP (which is zeroed), effectively enabling #PF interception by default.
The relevant part of the SDM describes this:
When a page fault occurs, a processor consults (1) bit 14 of the exception bitmap; (2) the error code produced with the page fault [PFEC]; (3) the page-fault error-code mask field [PFEC_MASK]; and (4) the page-fault error-code match field [PFEC_MATCH]. It checks if PFEC & PFEC_MASK = PFEC_MATCH. If there is equality, the specification of bit 14 in the exception bitmap is followed (for example, a VM exit occurs if that bit is set). If there is inequality, the meaning of that bit is reversed (for example, a VM exit occurs if that bit is clear).
Clear PFEC_MATCH such that there is equality, and the #PF bit in the exception bitmap is followed.
--- tools/testing/selftests/kvm/lib/x86/vmx.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/selftests/kvm/lib/x86/vmx.c +1 -1
@@ -232,7 +232,7 @@ static inline void init_vmcs_control_fields(struct vmx_pages *vmx)vmwrite (EXCEPTION_BITMAP ,0 );vmwrite (PAGE_FAULT_ERROR_CODE_MASK ,0 ); -vmwrite (PAGE_FAULT_ERROR_CODE_MATCH ,-1 );/* Never match */ +vmwrite (PAGE_FAULT_ERROR_CODE_MATCH ,0 );vmwrite (CR3_TARGET_COUNT ,0 );vmwrite (VM_EXIT_CONTROLS ,rdmsr (MSR_IA32_VMX_EXIT_CTLS ) |VM_EXIT_HOST_ADDR_SPACE_SIZE );/* 64-bit host */
Signature · 1 line
2.55.0.487.gaf234c4eb3-goog
Re: [PATCH v5 09/13] KVM: selftests: Do not intercept #PF by default in nVMX tests
Quoted 17 lines · Yosry Ahmed
init_vmcs_control_fields()setsPFEC_MASKandPFEC_MATCHso that they never match, which reverses the meaning of thePF_VECTORbit inEXCEPTION_BITMAP(which is zeroed), effectively enabling #PF interception by default.The relevant part of the SDM describes this:
When a page fault occurs, a processor consults (1) bit 14 of the exception bitmap; (2) the error code produced with the page fault [PFEC]; (3) the page-fault error-code mask field [PFEC_MASK]; and (4) the page-fault error-code match field [PFEC_MATCH]. It checks if PFEC &PFEC_MASK=PFEC_MATCH. If there is equality, the specification of bit 14 in the exception bitmap is followed (for example, a VM exit occurs if that bit is set). If there is inequality, the meaning of that bit is reversed (for example, a VM exit occurs if that bit is clear).
Quoted 4 lines
Clear
PFEC_MATCHsuch that there is equality, and the #PF bit in the exception bitmap is followed.Signed-off-by Yosry Ahmed <[email protected]>
Oh this needs either:
or
PATCH v5 10/13 KVM: selftests: Add basic stress test for save+restore and #PF handling
In reply to Yosry AhmedAdd a basic stress test for handling #PFs in a guest while the host is doing save+restore cycles. The guest periodically accesses non-present memory causing a #PF, and the #PF handler walks the page tables and updates the PTE to be present, like a proper #PF handler.
After every access (and #PF), the guest triggers a sync and the test performs save+restore of the VM. This is not very meaningful as save+restore are performed after the access and #PF handling complete, but following changes will change that.
--- tools/testing/selftests/kvm/Makefile.kvm | 1 + .../selftests/kvm/include/x86/processor.h | 13 ++ .../kvm/x86/save_restore_pf_stress_test.c | 160 ++++++++++++++++++ 3 files changed, 174 insertions(+) create mode 100644 tools/testing/selftests/kvm/x86/save_restore_pf_stress_test.c
tools/testing/selftests/kvm/Makefile.kvm +1 -0
@@ -107,6 +107,7 @@ TEST_GEN_PROGS_x86 += x86/pmu_counters_test TEST_GEN_PROGS_x86 += x86/pmu_event_filter_test TEST_GEN_PROGS_x86 += x86/private_mem_conversions_test TEST_GEN_PROGS_x86 += x86/private_mem_kvm_exits_test +TEST_GEN_PROGS_x86 += x86/save_restore_pf_stress_test TEST_GEN_PROGS_x86 += x86/set_boot_cpu_id TEST_GEN_PROGS_x86 += x86/set_sregs_test TEST_GEN_PROGS_x86 += x86/smaller_maxphyaddr_emulation_test
tools/testing/selftests/kvm/include/x86/processor.h +13 -0
@@ -614,6 +614,14 @@ static inline void set_cr0(u64 val) __asm__ __volatile__("mov %0, %%cr0" : :"r" (val) :"memory" ); } +static inline u64 get_cr2 (void ) +{ +u64 cr2; + + __asm__ __volatile__("mov %%cr2, %[cr2]" : [cr2]"=r" (cr2)); +return cr2; +} +static inline u64 get_cr3 (void ) {u64 cr3;
@@ -909,6 +917,11 @@ static inline void write_sse_reg(int reg, const sse128_t *data) } } +static inline void invlpg (u64 addr) +{ + __asm__ __volatile__("invlpg (%0)" : :"r" (addr) :"memory" ); +} +static inline void cpu_relax (void ) { asmvolatile ("rep; nop" :::"memory" );
tools/testing/selftests/kvm/x86/save_restore_pf_stress_test.c new file +160 -0
160 lines not on this page · raw message
Signature · 1 line
2.55.0.487.gaf234c4eb3-goog
PATCH v5 11/13 KVM: selftests: Trigger save+restore randomly in the #PF stress test
In reply to Yosry AhmedInstead of an explicit GUEST_SYNC() after each access+#PF, run another thread that keeps sending SIGUSR to the vCPU thread, essentially triggering exits to userspace and save+restore on random points in guest execution. This makes the test a lot more meaningful as it opens the door to exercising race conditions between #PF handling in the guest and save+restore in the host.
The signals are ignored using SIG_IGN outside of __vcpu_run() to avoid interrupting other ioctls/sysctls performed by the test.
--- .../kvm/x86/save_restore_pf_stress_test.c | 56 ++++++++++++++++--- 1 file changed, 49 insertions(+), 7 deletions(-)
tools/testing/selftests/kvm/x86/save_restore_pf_stress_test.c +49 -7
@@ -5,6 +5,8 @@#include <errno.h> #include <sys/types.h> #include <time.h> +#include <pthread.h> +#include <signal.h> #include <unistd.h> #include "test_util.h"
@@ -76,15 +78,41 @@ static void guest_access_memory(void *arg)/* Clear the present bit again so it faults next time */ *guest_get_pte (vaddr) &= ~PTE_PRESENT_MASK (&guest_mmu);invlpg (vaddr); + } +} + +static void *sigusr_thread_fn (void *arg) +{ +pthread_t vcpu_thread = (pthread_t )arg; -GUEST_SYNC (guest_faults); +for (;;) { +pthread_testcancel (); +pthread_kill (vcpu_thread,SIGUSR1 ); +usleep (msecs_to_usecs (1 )); } +return NULL ; +} + +static void dummy_signal_handler (int signo) {} +static struct sigaction sa; + +static void vcpu_sigusr_listen (void ) +{ + sa.sa_handler = dummy_signal_handler; +sigaction (SIGUSR1 , &sa,NULL ); +} + +static void vcpu_sigusr_ignore (void ) +{ + sa.sa_handler =SIG_IGN ; +sigaction (SIGUSR1 , &sa,NULL ); }int main (int argc,char *argv[]) {struct kvm_x86_state *state;int r, i, level; +pthread_t sigusr_thread;gpa_t gpa, pgtable_gpa;struct kvm_vcpu *vcpu;struct kvm_vm *vm;
@@ -128,18 +156,29 @@ int main(int argc, char *argv[]) pgtable_gpa =PTE_GET_PA (pte); } +/* Initialize the thread sending SIGUSR and install the handler */ +vcpu_sigusr_ignore (); + r =pthread_create (&sigusr_thread,NULL , sigusr_thread_fn, + (void *)pthread_self ()); +TEST_ASSERT (!r,"pthread_create() failed: %d" , r); +for (i =1 ; i <=NR_ITERATIONS ; i++) { +/* +* Only handle SIGUSR while the vCPU is running, otherwise +* ignore it to avoid interrupting other ioctls/syscalls. +*/ +vcpu_sigusr_listen (); r =__vcpu_run (vcpu); -TEST_ASSERT (!r,"vcpu_run failed" ); -TEST_ASSERT_KVM_EXIT_REASON (vcpu,KVM_EXIT_IO ); +TEST_ASSERT (!r || errno ==EINTR ,"Expected success or SIGUSR1" ); +vcpu_sigusr_ignore (); -get_ucall (vcpu, &uc); -if (uc.cmd ==UCALL_ABORT ) { +/* The guest only exits due to a signal or failed assertion */ +if (!r) { +TEST_ASSERT_KVM_EXIT_REASON (vcpu,KVM_EXIT_IO ); +TEST_ASSERT_EQ (get_ucall (vcpu, &uc),UCALL_ABORT );REPORT_GUEST_ASSERT (uc);break ; } -TEST_ASSERT_EQ (uc.cmd,UCALL_SYNC ); -TEST_ASSERT_EQ (uc.args[1 ], i); state =vcpu_save_state (vcpu);
@@ -153,8 +192,11 @@ int main(int argc, char *argv[])pr_info ("\n" );sync_global_from_guest (vm, guest_faults); +TEST_ASSERT (guest_faults,"No guest page faults triggered" );pr_info ("Guest page faults: %lu\n" , guest_faults); +pthread_cancel (sigusr_thread); +pthread_join (sigusr_thread,NULL );kvm_vm_free (vm);return 0 ; }
Signature · 1 line
2.55.0.487.gaf234c4eb3-goog
PATCH v5 12/13 KVM: selftests: Support running stress save+restore and #PF test in L2
In reply to Yosry AhmedExtend the stress test to allow running the access+#PF code in L2 instead of L1 by adding proper L1 guest code to bootstrap L2. By default, the test runs in L2 after running in L1 if nested is supported.
--- .../kvm/x86/save_restore_pf_stress_test.c | 56 ++++++++++++++++++- 1 file changed, 53 insertions(+), 3 deletions(-)
tools/testing/selftests/kvm/x86/save_restore_pf_stress_test.c +53 -3
@@ -8,10 +8,13 @@#include <pthread.h> #include <signal.h> #include <unistd.h> +#include <getopt.h> #include "test_util.h" #include "kvm_util.h" #include "processor.h" +#include "svm_util.h" +#include "vmx.h" #define NR_ITERATIONS 500
@@ -81,6 +84,31 @@ static void guest_access_memory(void *arg) } } +static void l1_svm_code (struct svm_test_data *svm) +{ +generic_svm_setup (svm, guest_access_memory); +run_guest (svm->vmcb, svm->vmcb_gpa); +GUEST_ASSERT (false); +} + +static void l1_vmx_code (struct vmx_pages *vmx) +{ +GUEST_ASSERT (prepare_for_vmx_operation (vmx)); +GUEST_ASSERT (load_vmcs (vmx)); +prepare_vmcs (vmx, guest_access_memory); + +GUEST_ASSERT (!vmlaunch ()); +GUEST_ASSERT (false); +} + +static void l1_guest_code (void *test_data) +{ +if (this_cpu_has (X86_FEATURE_SVM )) +l1_svm_code (test_data); +else +l1_vmx_code (test_data); +} +static void *sigusr_thread_fn (void *arg) {pthread_t vcpu_thread = (pthread_t )arg;
@@ -108,7 +136,7 @@ static void vcpu_sigusr_ignore(void)sigaction (SIGUSR1 , &sa,NULL ); } -int main (int argc,char *argv[]) +static void run_test (bool nested) {struct kvm_x86_state *state;int r, i, level;
@@ -121,9 +149,17 @@ int main(int argc, char *argv[])gva_t gva;u64 pte; - vm =vm_create_with_one_vcpu (&vcpu, guest_access_memory); + vm =vm_create_with_one_vcpu (&vcpu, nested ? l1_guest_code : guest_access_memory);vm_install_exception_handler (vm,PF_VECTOR , guest_pf_handler); +if (nested) { +if (kvm_cpu_has (X86_FEATURE_SVM )) +vcpu_alloc_svm (vm, &gva); +else +vcpu_alloc_vmx (vm, &gva); +vcpu_args_set (vcpu,1 , gva); + } +/* Allocate a page and write the pattern to it */ gva =vm_alloc_page (vm); *(u64 *)addr_gva2hva (vm, gva) =PATTERN ;
@@ -193,10 +229,24 @@ int main(int argc, char *argv[])sync_global_from_guest (vm, guest_faults);TEST_ASSERT (guest_faults,"No guest page faults triggered" ); -pr_info ("Guest page faults: %lu\n" , guest_faults); +pr_info ("Guest page faults%s: %lu\n" , nested ?" (in L2)" :"" , guest_faults);pthread_cancel (sigusr_thread);pthread_join (sigusr_thread,NULL );kvm_vm_free (vm); +} + +int main (int argc,char *argv[]) +{ +pr_info ("Running save+restore stress test...\n" ); +run_test (/*nested=*/ false); + +if (!kvm_cpu_has (X86_FEATURE_SVM ) && !kvm_cpu_has (X86_FEATURE_VMX )) { +pr_info ("Nested virtualization not supported, skipping nested test\n" ); +return 0 ; + } + +pr_info ("Running save+restore stress test with a nested guest...\n" ); +run_test (/*nested=*/ true);return 0 ; }
Signature · 1 line
2.55.0.487.gaf234c4eb3-goog
PATCH v5 13/13 KVM: selftests: Trigger L2->L1 exits stress save+restore and #PF test
In reply to Yosry AhmedExtend the testing coverage in L2 by forcing a nested VM-Exit from L2 to L1 right after restore on every other iteration. Forcing a nested VM-Exit while L0 has control (e.g. without explicitly running L2 and making a hypercall) is valuable, as it often happens during live migration (e.g. L1 timer interrupt fires by the time the VM lands on the destination).
To force the nested VM-Exit inject a #UD in to the saved vCPU state, and intercept #UD from L1.
With this change, the test reliably reproduces the CR2 bug fixed by commit 5c247d08bc81 ("KVM: nSVM: Use vcpu->arch.cr2 when updating vmcb12 on nested #VMEXIT") -- at least on Milan, Genoa, and Turin CPUs.
--- .../selftests/kvm/include/x86/processor.h | 5 +++ .../kvm/x86/save_restore_pf_stress_test.c | 44 +++++++++++++++++-- 2 files changed, 45 insertions(+), 4 deletions(-)
tools/testing/selftests/kvm/include/x86/processor.h +5 -0
@@ -958,6 +958,11 @@ struct kvm_x86_state *vcpu_save_state(struct kvm_vcpu *vcpu);void vcpu_load_state (struct kvm_vcpu *vcpu,struct kvm_x86_state *state);void kvm_x86_state_cleanup (struct kvm_x86_state *state); +static inline bool kvm_x86_state_is_guest_mode (struct kvm_x86_state *state) +{ +return state->nested.size && (state->nested.flags &KVM_STATE_NESTED_GUEST_MODE ); +} +const struct kvm_msr_list *kvm_get_msr_index_list (void );const struct kvm_msr_list *kvm_get_feature_msr_index_list (void );bool kvm_msr_is_in_save_restore_list (u32 msr_index);
tools/testing/selftests/kvm/x86/save_restore_pf_stress_test.c +40 -4
83 lines not on this page · raw message
Signature · 1 line
2.55.0.487.gaf234c4eb3-goog
Sean Christopherson
maintainerQuoted 8 lines · Yosry Ahmed
Add a stress test for save+restore while the guest is triggering and handling #PFs, in both L1 and L2. The goal was to create a generic selftest that would catch bugs like the one fixed by commit
5c247d08bc81("KVM: nSVM: Usevcpu->arch.cr2when updating vmcb12 on nested #VMEXIT"), instead of relying on high-level testing (e.g. building GCC in L2) to catch it.[...]
Applied to kvm-x86 selftests, thanks!
[01/13] KVM: selftests: Use __stringify() instead of custom XSTR() macros
https://github.com/kvm-x86/linux/commit/9be54a82656c [02/13] KVM: selftests: Fix RAX and RFLAGS VMCB offsets when running L2 https://github.com/kvm-x86/linux/commit/53f4ad49d76b [03/13] KVM: selftests: Rework GPR registers switching for SVM (and fix offsets) https://github.com/kvm-x86/linux/commit/546455d9c724 [04/13] KVM: selftests: Handle rflags save/restore for SVM in guest_regs https://github.com/kvm-x86/linux/commit/9547309406d5 [05/13] KVM: selftests: Reuse GPR switching logic for nVMX https://github.com/kvm-x86/linux/commit/3d31fea7b6d1 [06/13] KVM: selftests: Drop HORRIFIC_L2_UCALL_CLOBBER_HACK https://github.com/kvm-x86/linux/commit/2bdef77c6019 [07/13] KVM: selftests: Add a blank line before logging assertion failures https://github.com/kvm-x86/linux/commit/12f4d8b06a53 [08/13] KVM: selftests: Expose PTE masks to guests as part of an MMU https://github.com/kvm-x86/linux/commit/e21d4dc0ca63 [09/13] KVM: selftests: Do not intercept #PF by default in nVMX tests https://github.com/kvm-x86/linux/commit/7317f5b143aa [10/13] KVM: selftests: Add basic stress test for save+restore and #PF handling https://github.com/kvm-x86/linux/commit/d24dde74f72f [11/13] KVM: selftests: Trigger save+restore randomly in the #PF stress test https://github.com/kvm-x86/linux/commit/d99ad3c842f0 [12/13] KVM: selftests: Support running stress save+restore and #PF test in L2 https://github.com/kvm-x86/linux/commit/1494b3d17c95 [13/13] KVM: selftests: Trigger L2->L1 exits stress save+restore and #PF test https://github.com/kvm-x86/linux/commit/583ad2052ddd