kvm: selftests: add aarch64 framework and dirty
cover kvm: selftests: add aarch64 framework and dirty
This series provides KVM selftests that test dirty log tracking on AArch64 for both 4K and 64K guest page sizes. Additionally the framework provides an easy way to test dirty log tracking with the recently posted dynamic IPA and 52bit IPA series[1].
The series breaks down into parts as follows:
01-02: generalize guest code to host userspace exit support by
introducing "ucalls" - hypercalls to userspace
03-05: prepare common code for a new architecture 06-07: add virtual memory setup support for AArch64
08: add vcpu setup support for AArch64
09: port the dirty log test to AArch64
10-11: add 64K guest page size support for the dirty log test 12-13: prepare the dirty log test to also test > 40-bit guest
physical address setups by allowing the test memory region to be placed at the top of physical memory
[1] https://www.spinics.net/lists/arm-kernel/msg676819.html
Thanks,
drew
Andrew Jones (13):
kvm: selftests: vcpu_setup: set cr4.osfxsr
kvm: selftests: introduce ucall
kvm: selftests: move arch-specific files to arch-specific locations
kvm: selftests: add cscope make target
kvm: selftests: tidy up kvm_util
kvm: selftests: add vm_phy_pages_alloc
kvm: selftests: add virt mem support for aarch64
kvm: selftests: add vcpu support for aarch64
kvm: selftests: port dirty_log_test to aarch64
kvm: selftests: introduce new VM mode for 64K pages
kvm: selftests: dirty_log_test: also test 64K pages on aarch64
kvm: selftests: stop lying to aarch64 tests about PA-bits
kvm: selftests: support high GPAs in dirty_log_test
tools/testing/selftests/kvm/.gitignore | 11 +-
tools/testing/selftests/kvm/Makefile | 36 +-
tools/testing/selftests/kvm/dirty_log_test.c | 374 +++++++++----
.../selftests/kvm/include/aarch64/processor.h | 55 ++
.../testing/selftests/kvm/include/kvm_util.h | 166 +++---
.../testing/selftests/kvm/include/sparsebit.h | 6 +-
.../testing/selftests/kvm/include/test_util.h | 6 +-
.../kvm/include/{x86.h => x86_64/processor.h} | 24 +-
.../selftests/kvm/include/{ => x86_64}/vmx.h | 6 +-
.../selftests/kvm/lib/aarch64/processor.c | 311 +++++++++++
tools/testing/selftests/kvm/lib/assert.c | 2 +-
tools/testing/selftests/kvm/lib/kvm_util.c | 499 +++++++-----------
.../selftests/kvm/lib/kvm_util_internal.h | 33 +-
tools/testing/selftests/kvm/lib/ucall.c | 144 +++++
.../kvm/lib/{x86.c => x86_64/processor.c} | 197 ++++++-
.../selftests/kvm/lib/{ => x86_64}/vmx.c | 4 +-
.../kvm/{ => x86_64}/cr4_cpuid_sync_test.c | 14 +-
.../kvm/{ => x86_64}/set_sregs_test.c | 2 +-
.../selftests/kvm/{ => x86_64}/state_test.c | 25 +-
.../kvm/{ => x86_64}/sync_regs_test.c | 2 +-
.../kvm/{ => x86_64}/vmx_tsc_adjust_test.c | 23 +-
21 files changed, 1329 insertions(+), 611 deletions(-)
create mode 100644 tools/testing/selftests/kvm/include/aarch64/processor.h
rename tools/testing/selftests/kvm/include/{x86.h => x86_64/processor.h} (98%)
rename tools/testing/selftests/kvm/include/{ => x86_64}/vmx.h (99%)
create mode 100644 tools/testing/selftests/kvm/lib/aarch64/processor.c
create mode 100644 tools/testing/selftests/kvm/lib/ucall.c
rename tools/testing/selftests/kvm/lib/{x86.c => x86_64/processor.c} (85%)
rename tools/testing/selftests/kvm/lib/{ => x86_64}/vmx.c (99%)
rename tools/testing/selftests/kvm/{ => x86_64}/cr4_cpuid_sync_test.c (91%)
rename tools/testing/selftests/kvm/{ => x86_64}/set_sregs_test.c (98%)
rename tools/testing/selftests/kvm/{ => x86_64}/state_test.c (90%)
rename tools/testing/selftests/kvm/{ => x86_64}/sync_regs_test.c (99%)
rename tools/testing/selftests/kvm/{ => x86_64}/vmx_tsc_adjust_test.c (91%)Signature · 1 line
2.17.1
PATCH 1/13 kvm: selftests: vcpu_setup: set cr4.osfxsr
Guest code may want to call functions that have variable arguments. To do so, we either need to compile with -mno-sse or enable SSE in the VCPUs. As it should be pretty safe to turn on the feature, and -mno-sse would make linking test code with standard libraries difficult, we choose the feature enabling.
--- tools/testing/selftests/kvm/lib/x86.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/selftests/kvm/lib/x86.c +1 -1
@@ -626,7 +626,7 @@ void vcpu_setup(struct kvm_vm *vm, int vcpuid, int pgd_memslot, int gdt_memslot)switch (vm->mode) {case VM_MODE_FLAT48PG : sregs.cr0 =X86_CR0_PE |X86_CR0_NE |X86_CR0_PG ; - sregs.cr4 |=X86_CR4_PAE ; + sregs.cr4 |=X86_CR4_PAE |X86_CR4_OSFXSR ; sregs.efer |= (EFER_LME |EFER_LMA |EFER_NX );kvm_seg_set_unusable (&sregs.ldt);
Signature · 1 line
2.17.1
PATCH 2/13 kvm: selftests: introduce ucall
In reply to Andrew JonesRework the guest exit to userspace code to generalize the concept into what it is, a "hypercall to userspace", and provide two implementations of it: the PortIO version currently used, but only useable by x86, and an MMIO version that other architectures (except s390) can use.
--- tools/testing/selftests/kvm/Makefile | 2 +- .../selftests/kvm/cr4_cpuid_sync_test.c | 12 +- tools/testing/selftests/kvm/dirty_log_test.c | 5 +- .../testing/selftests/kvm/include/kvm_util.h | 76 ++++----- tools/testing/selftests/kvm/lib/kvm_util.c | 14 -- tools/testing/selftests/kvm/lib/ucall.c | 144 ++++++++++++++++++ tools/testing/selftests/kvm/state_test.c | 23 +-- .../selftests/kvm/vmx_tsc_adjust_test.c | 17 +-- 8 files changed, 214 insertions(+), 79 deletions(-) create mode 100644 tools/testing/selftests/kvm/lib/ucall.c
tools/testing/selftests/kvm/Makefile +1 -1
@@ -3,7 +3,7 @@ all: top_srcdir = ../../../../ UNAME_M := $(shell uname -m) -LIBKVM = lib/assert.c lib/elf.c lib/io.c lib/kvm_util.c lib/sparsebit.c +LIBKVM = lib/assert.c lib/elf.c lib/io.c lib/kvm_util.c lib/ucall.c lib/sparsebit.c LIBKVM_x86_64 = lib/x86.c lib/vmx.c TEST_GEN_PROGS_x86_64 = set_sregs_test
tools/testing/selftests/kvm/cr4_cpuid_sync_test.c +6 -6
@@ -67,6 +67,7 @@ int main(int argc, char *argv[])struct kvm_vm *vm;struct kvm_sregs sregs;struct kvm_cpuid_entry2 *entry; +struct ucall uc;int rc; entry =kvm_get_supported_cpuid_entry (1 );
@@ -87,21 +88,20 @@ int main(int argc, char *argv[]) rc =_vcpu_run (vm,VCPU_ID );if (run->exit_reason ==KVM_EXIT_IO ) { -switch (run->io.port) { -case GUEST_PORT_SYNC : +switch (get_ucall (vm,VCPU_ID , &uc)) { +case UCALL_SYNC :/* emulate hypervisor clearing CR4.OSXSAVE */ vcpu_sregs_get (vm,VCPU_ID , &sregs); sregs.cr4 &= ~X86_CR4_OSXSAVE ;vcpu_sregs_set (vm,VCPU_ID , &sregs);break ; -case GUEST_PORT_ABORT : +case UCALL_ABORT :TEST_ASSERT (false,"Guest CR4 bit (OSXSAVE) unsynchronized with CPUID bit." );break ; -case GUEST_PORT_DONE : +case UCALL_DONE : goto done;default : -TEST_ASSERT (false,"Unknown port 0x%x." , - run->io.port); +TEST_ASSERT (false,"Unknown ucall 0x%x." , uc.cmd); } } }
tools/testing/selftests/kvm/dirty_log_test.c +2 -3
@@ -110,7 +110,7 @@ void *vcpu_worker(void *data)uint64_t loops, *guest_array, pages_count =0 ;struct kvm_vm *vm = data;struct kvm_run *run; -struct guest_args args; +struct ucall uc; run =vcpu_state (vm,VCPU_ID );
@@ -124,9 +124,8 @@ void *vcpu_worker(void *data)while (!READ_ONCE (host_quit)) {/* Let the guest to dirty these random pages */ ret =_vcpu_run (vm,VCPU_ID ); -guest_args_read (vm,VCPU_ID , &args);if (run->exit_reason ==KVM_EXIT_IO && - args.port ==GUEST_PORT_SYNC ) { +get_ucall (vm,VCPU_ID , &uc) ==UCALL_SYNC ) { pages_count +=TEST_PAGES_PER_LOOP ;generate_random_array (guest_array,TEST_PAGES_PER_LOOP ); }else {
tools/testing/selftests/kvm/include/kvm_util.h +41 -35
84 lines not on this page · raw message
tools/testing/selftests/kvm/lib/kvm_util.c +0 -14
@@ -1580,17 +1580,3 @@ void *addr_gva2hva(struct kvm_vm *vm, vm_vaddr_t gva) {return addr_gpa2hva (vm,addr_gva2gpa (vm, gva)); } - -void guest_args_read (struct kvm_vm *vm,uint32_t vcpu_id, -struct guest_args *args) -{ -struct kvm_run *run =vcpu_state (vm, vcpu_id); -struct kvm_regs regs; - -memset (®s,0 ,sizeof (regs)); -vcpu_regs_get (vm, vcpu_id, ®s); - - args->port = run->io.port; - args->arg0 = regs.rdi; - args->arg1 = regs.rsi; -}
tools/testing/selftests/kvm/lib/ucall.c new file +144 -0
144 lines not on this page · raw message
tools/testing/selftests/kvm/state_test.c +12 -11
@@ -127,6 +127,7 @@ int main(int argc, char *argv[])struct kvm_vm *vm;struct kvm_run *run;struct kvm_x86_state *state; +struct ucall uc;int stage;struct kvm_cpuid_entry2 *entry =kvm_get_supported_cpuid_entry (1 );
@@ -155,23 +156,23 @@ int main(int argc, char *argv[])memset (®s1,0 ,sizeof (regs1));vcpu_regs_get (vm,VCPU_ID , ®s1); -switch (run->io.port) { -case GUEST_PORT_ABORT : -TEST_ASSERT (false,"%s at %s:%d" , (const char *) regs1.rdi, - __FILE__, regs1.rsi); +switch (get_ucall (vm,VCPU_ID , &uc)) { +case UCALL_ABORT : +TEST_ASSERT (false,"%s at %s:%d" , (const char *)uc.args[0 ], + __FILE__, uc.args[1 ]);/* NOT REACHED */ -case GUEST_PORT_SYNC : +case UCALL_SYNC :break ; -case GUEST_PORT_DONE : +case UCALL_DONE : goto done;default : -TEST_ASSERT (false,"Unknown port 0x%x." , run->io.port); +TEST_ASSERT (false,"Unknown ucall 0x%x." , uc.cmd); } -/* PORT_SYNC is handled here. */ -TEST_ASSERT (!strcmp ((const char *)regs1.rdi,"hello" ) && - regs1.rsi == stage,"Unexpected register values vmexit #%lx, got %lx" , - stage, (ulong ) regs1.rsi); +/* UCALL_SYNC is handled here. */ +TEST_ASSERT (!strcmp ((const char *)uc.args[0 ],"hello" ) && + uc.args[1 ] == stage,"Unexpected register values vmexit #%lx, got %lx" , + stage, (ulong )uc.args[1 ]); state =vcpu_save_state (vm,VCPU_ID );kvm_vm_release (vm);
tools/testing/selftests/kvm/vmx_tsc_adjust_test.c +8 -9
@@ -146,26 +146,25 @@ int main(int argc, char *argv[])for (;;) {volatile struct kvm_run *run =vcpu_state (vm,VCPU_ID ); -struct guest_args args; +struct ucall uc;vcpu_run (vm,VCPU_ID ); -guest_args_read (vm,VCPU_ID , &args);TEST_ASSERT (run->exit_reason ==KVM_EXIT_IO ,"Got exit_reason other than KVM_EXIT_IO: %u (%s)\n" , run->exit_reason,exit_reason_str (run->exit_reason)); -switch (args.port) { -case GUEST_PORT_ABORT : -TEST_ASSERT (false,"%s" , (const char *) args.arg0); +switch (get_ucall (vm,VCPU_ID , &uc)) { +case UCALL_ABORT : +TEST_ASSERT (false,"%s" , (const char *)uc.args[0 ]);/* NOT REACHED */ -case GUEST_PORT_SYNC : -report (args.arg1); +case UCALL_SYNC : +report (uc.args[1 ]);break ; -case GUEST_PORT_DONE : +case UCALL_DONE : goto done;default : -TEST_ASSERT (false,"Unknown port 0x%x." , args.port); +TEST_ASSERT (false,"Unknown ucall 0x%x." , uc.cmd); } }
Signature · 1 line
2.17.1
PATCH 3/13 kvm: selftests: move arch-specific files to arch-specific locations
In reply to Andrew Jones---
tools/testing/selftests/kvm/.gitignore | 11 +++++-----
tools/testing/selftests/kvm/Makefile | 22 +++++++++----------
.../testing/selftests/kvm/include/kvm_util.h | 2 +-
.../testing/selftests/kvm/include/sparsebit.h | 6 ++---
.../testing/selftests/kvm/include/test_util.h | 6 ++---
.../kvm/include/{x86.h => x86_64/processor.h} | 8 +++----
.../selftests/kvm/include/{ => x86_64}/vmx.h | 6 ++---
tools/testing/selftests/kvm/lib/assert.c | 2 +-
.../selftests/kvm/lib/kvm_util_internal.h | 8 +++----
.../kvm/lib/{x86.c => x86_64/processor.c} | 6 ++---
.../selftests/kvm/lib/{ => x86_64}/vmx.c | 4 ++--
.../kvm/{ => x86_64}/cr4_cpuid_sync_test.c | 2 +-
.../kvm/{ => x86_64}/dirty_log_test.c | 0
.../kvm/{ => x86_64}/set_sregs_test.c | 2 +-
.../selftests/kvm/{ => x86_64}/state_test.c | 2 +-
.../kvm/{ => x86_64}/sync_regs_test.c | 2 +-
.../kvm/{ => x86_64}/vmx_tsc_adjust_test.c | 6 ++---
17 files changed, 48 insertions(+), 47 deletions(-)
rename tools/testing/selftests/kvm/include/{x86.h => x86_64/processor.h} (99%)
rename tools/testing/selftests/kvm/include/{ => x86_64}/vmx.h (99%)
rename tools/testing/selftests/kvm/lib/{x86.c => x86_64/processor.c} (99%)
rename tools/testing/selftests/kvm/lib/{ => x86_64}/vmx.c (99%)
rename tools/testing/selftests/kvm/{ => x86_64}/cr4_cpuid_sync_test.c (98%)
rename tools/testing/selftests/kvm/{ => x86_64}/dirty_log_test.c (100%)
rename tools/testing/selftests/kvm/{ => x86_64}/set_sregs_test.c (98%)
rename tools/testing/selftests/kvm/{ => x86_64}/state_test.c (99%)
rename tools/testing/selftests/kvm/{ => x86_64}/sync_regs_test.c (99%)
rename tools/testing/selftests/kvm/{ => x86_64}/vmx_tsc_adjust_test.c (98%)tools/testing/selftests/kvm/.gitignore +6 -5
@@ -1,5 +1,6 @@ -cr4_cpuid_sync_test -set_sregs_test -sync_regs_test -vmx_tsc_adjust_test -state_test +/x86_64/cr4_cpuid_sync_test +/x86_64/set_sregs_test +/x86_64/sync_regs_test +/x86_64/vmx_tsc_adjust_test +/x86_64/state_test +/x86_64/dirty_log_test
tools/testing/selftests/kvm/Makefile +11 -11
@@ -1,25 +1,25 @@ all: -top_srcdir = ../../../../ +top_srcdir = ../../../.. UNAME_M := $(shell uname -m) LIBKVM = lib/assert.c lib/elf.c lib/io.c lib/kvm_util.c lib/ucall.c lib/sparsebit.c -LIBKVM_x86_64 = lib/x86.c lib/vmx.c +LIBKVM_x86_64 = lib/x86_64/processor.c lib/x86_64/vmx.c -TEST_GEN_PROGS_x86_64 = set_sregs_test -TEST_GEN_PROGS_x86_64 += sync_regs_test -TEST_GEN_PROGS_x86_64 += vmx_tsc_adjust_test -TEST_GEN_PROGS_x86_64 += cr4_cpuid_sync_test -TEST_GEN_PROGS_x86_64 += state_test -TEST_GEN_PROGS_x86_64 += dirty_log_test +TEST_GEN_PROGS_x86_64 = x86_64/set_sregs_test +TEST_GEN_PROGS_x86_64 += x86_64/sync_regs_test +TEST_GEN_PROGS_x86_64 += x86_64/vmx_tsc_adjust_test +TEST_GEN_PROGS_x86_64 += x86_64/cr4_cpuid_sync_test +TEST_GEN_PROGS_x86_64 += x86_64/state_test +TEST_GEN_PROGS_x86_64 += x86_64/dirty_log_test TEST_GEN_PROGS += $(TEST_GEN_PROGS_$(UNAME_M)) LIBKVM += $(LIBKVM_$(UNAME_M)) INSTALL_HDR_PATH = $(top_srcdir)/usr -LINUX_HDR_PATH = $(INSTALL_HDR_PATH)/include/ -LINUX_TOOL_INCLUDE = $(top_srcdir)tools/include -CFLAGS += -O2 -g -std=gnu99 -I$(LINUX_TOOL_INCLUDE) -I$(LINUX_HDR_PATH) -Iinclude -I$(<D) -I.. +LINUX_HDR_PATH = $(INSTALL_HDR_PATH)/include +LINUX_TOOL_INCLUDE = $(top_srcdir)/tools/include +CFLAGS += -O2 -g -std=gnu99 -I$(LINUX_TOOL_INCLUDE) -I$(LINUX_HDR_PATH) -Iinclude -Iinclude/$(UNAME_M) -I.. LDFLAGS += -lpthread # After inclusion, $(OUTPUT) is defined and
tools/testing/selftests/kvm/include/kvm_util.h +1 -1
@@ -7,7 +7,7 @@* */ #ifndef SELFTEST_KVM_UTIL_H -#define SELFTEST_KVM_UTIL_H 1 +#define SELFTEST_KVM_UTIL_H #include "test_util.h"
tools/testing/selftests/kvm/include/sparsebit.h +3 -3
@@ -15,8 +15,8 @@* even in the case where most bits are set. */ -#ifndef _TEST_SPARSEBIT_H_ -#define _TEST_SPARSEBIT_H_ +#ifndef SELFTEST_KVM_SPARSEBIT_H +#define SELFTEST_KVM_SPARSEBIT_H #include <stdbool.h> #include <stdint.h>
@@ -72,4 +72,4 @@ void sparsebit_validate_internal(struct sparsebit *sbit); }#endif -#endif /* _TEST_SPARSEBIT_H_ */ +#endif /* SELFTEST_KVM_SPARSEBIT_H */
tools/testing/selftests/kvm/include/test_util.h +3 -3
@@ -7,8 +7,8 @@* */ -#ifndef TEST_UTIL_H -#define TEST_UTIL_H 1 +#ifndef SELFTEST_KVM_TEST_UTIL_H +#define SELFTEST_KVM_TEST_UTIL_H #include <stdlib.h> #include <stdarg.h>
@@ -41,4 +41,4 @@ void test_assert(bool exp, const char *exp_str,#a , #b, #a, (unsigned long) __a, #b, (unsigned long) __b); \ } while (0) -#endif /* TEST_UTIL_H */ +#endif /* SELFTEST_KVM_TEST_UTIL_H */
tools/testing/selftests/kvm/include/x86_64/processor.h renamed from tools/testing/selftests/kvm/include/x86.h +4 -4
@@ -1,5 +1,5 @@/* -* tools/testing/selftests/kvm/include/x86.h +* tools/testing/selftests/kvm/include/x86_64/processor.h * * Copyright (C) 2018, Google LLC. *
@@ -7,8 +7,8 @@* */ -#ifndef SELFTEST_KVM_X86_H -#define SELFTEST_KVM_X86_H +#ifndef SELFTEST_KVM_PROCESSOR_H +#define SELFTEST_KVM_PROCESSOR_H #include <assert.h> #include <stdint.h>
@@ -1044,4 +1044,4 @@ void vcpu_load_state(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_x86_state *s#define MSR_VM_IGNNE 0xc0010115#define MSR_VM_HSAVE_PA 0xc0010117 -#endif /* !SELFTEST_KVM_X86_H */ +#endif /* SELFTEST_KVM_PROCESSOR_H */
tools/testing/selftests/kvm/include/x86_64/vmx.h renamed from tools/testing/selftests/kvm/include/vmx.h +3 -3
@@ -1,5 +1,5 @@/* -* tools/testing/selftests/kvm/include/vmx.h +* tools/testing/selftests/kvm/include/x86_64/vmx.h * * Copyright (C) 2018, Google LLC. *
@@ -11,7 +11,7 @@#define SELFTEST_KVM_VMX_H #include <stdint.h> -#include "x86.h" +#include "processor.h" #define CPUID_VMX_BIT 5
@@ -549,4 +549,4 @@ struct vmx_pages *vcpu_alloc_vmx(struct kvm_vm *vm, vm_vaddr_t *p_vmx_gva);bool prepare_for_vmx_operation (struct vmx_pages *vmx);void prepare_vmcs (struct vmx_pages *vmx,void *guest_rip,void *guest_rsp); -#endif /* !SELFTEST_KVM_VMX_H */ +#endif /* SELFTEST_KVM_VMX_H */
tools/testing/selftests/kvm/lib/assert.c +1 -1
@@ -13,7 +13,7 @@#include <execinfo.h> #include <sys/syscall.h> -#include "../../kselftest.h" +#include "kselftest.h" /* Dumps the current stack trace to stderr. */ static void __attribute__((noinline))test_dump_stack (void );
tools/testing/selftests/kvm/lib/kvm_util_internal.h +4 -4
@@ -1,13 +1,13 @@/* -* tools/testing/selftests/kvm/lib/kvm_util.c +* tools/testing/selftests/kvm/lib/kvm_util_internal.h * * Copyright (C) 2018, Google LLC. * * This work is licensed under the terms of the GNU GPL, version 2. */ -#ifndef KVM_UTIL_INTERNAL_H -#define KVM_UTIL_INTERNAL_H 1 +#ifndef SELFTEST_KVM_UTIL_INTERNAL_H +#define SELFTEST_KVM_UTIL_INTERNAL_H #include "sparsebit.h"
@@ -69,4 +69,4 @@ void regs_dump(FILE *stream, struct kvm_regs *regs,void sregs_dump (FILE *stream,struct kvm_sregs *sregs,uint8_t indent); -#endif +#endif /* SELFTEST_KVM_UTIL_INTERNAL_H */
tools/testing/selftests/kvm/lib/x86_64/processor.c renamed from tools/testing/selftests/kvm/lib/x86.c +3 -3
@@ -1,5 +1,5 @@/* -* tools/testing/selftests/kvm/lib/x86.c +* tools/testing/selftests/kvm/lib/x86_64/processor.c * * Copyright (C) 2018, Google LLC. *
@@ -10,8 +10,8 @@#include "test_util.h" #include "kvm_util.h" -#include "kvm_util_internal.h" -#include "x86.h" +#include "../kvm_util_internal.h" +#include "processor.h" /* Minimum physical address used for virtual translation tables. */ #define KVM_GUEST_PAGE_TABLE_MIN_PADDR 0x180000
tools/testing/selftests/kvm/lib/x86_64/vmx.c renamed from tools/testing/selftests/kvm/lib/vmx.c +2 -2
@@ -1,5 +1,5 @@/* -* tools/testing/selftests/kvm/lib/x86.c +* tools/testing/selftests/kvm/lib/x86_64/vmx.c * * Copyright (C) 2018, Google LLC. *
@@ -10,7 +10,7 @@#include "test_util.h" #include "kvm_util.h" -#include "x86.h" +#include "processor.h" #include "vmx.h" /* Allocate memory regions for nested VMX tests.
tools/testing/selftests/kvm/x86_64/cr4_cpuid_sync_test.c renamed from tools/testing/selftests/kvm/cr4_cpuid_sync_test.c +1 -1
@@ -17,7 +17,7 @@#include "test_util.h" #include "kvm_util.h" -#include "x86.h" +#include "processor.h" #define X86_FEATURE_XSAVE (1<<26)#define X86_FEATURE_OSXSAVE (1<<27)
tools/testing/selftests/kvm/x86_64/dirty_log_test.c renamed from tools/testing/selftests/kvm/dirty_log_test.c +0 -0
tools/testing/selftests/kvm/x86_64/set_sregs_test.c renamed from tools/testing/selftests/kvm/set_sregs_test.c +1 -1
@@ -22,7 +22,7 @@#include "test_util.h" #include "kvm_util.h" -#include "x86.h" +#include "processor.h" #define VCPU_ID 5
tools/testing/selftests/kvm/x86_64/state_test.c renamed from tools/testing/selftests/kvm/state_test.c +1 -1
@@ -17,7 +17,7 @@#include "test_util.h" #include "kvm_util.h" -#include "x86.h" +#include "processor.h" #include "vmx.h" #define VCPU_ID 5
tools/testing/selftests/kvm/x86_64/sync_regs_test.c renamed from tools/testing/selftests/kvm/sync_regs_test.c +1 -1
@@ -19,7 +19,7 @@#include "test_util.h" #include "kvm_util.h" -#include "x86.h" +#include "processor.h" #define VCPU_ID 5
tools/testing/selftests/kvm/x86_64/vmx_tsc_adjust_test.c renamed from tools/testing/selftests/kvm/vmx_tsc_adjust_test.c +3 -3
@@ -1,5 +1,5 @@/* -* gtests/tests/vmx_tsc_adjust_test.c +* vmx_tsc_adjust_test * * Copyright (C) 2018, Google LLC. *
@@ -22,13 +22,13 @@#include "test_util.h" #include "kvm_util.h" -#include "x86.h" +#include "processor.h" #include "vmx.h" #include <string.h> #include <sys/ioctl.h> -#include "../kselftest.h" +#include "kselftest.h" #ifndef MSR_IA32_TSC_ADJUST #define MSR_IA32_TSC_ADJUST 0x3b
Signature · 1 line
2.17.1
PATCH 4/13 kvm: selftests: add cscope make target
In reply to Andrew Jones--- tools/testing/selftests/kvm/Makefile | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-)
tools/testing/selftests/kvm/Makefile +10 -1
@@ -28,7 +28,7 @@ include ../lib.mk STATIC_LIBS := $(OUTPUT)/libkvm.a LIBKVM_OBJ := $(patsubst %.c, $(OUTPUT)/%.o, $(LIBKVM)) -EXTRA_CLEAN += $(LIBKVM_OBJ) $(STATIC_LIBS) +EXTRA_CLEAN += $(LIBKVM_OBJ) $(STATIC_LIBS) cscope.* x := $(shell mkdir -p $(sort $(dir $(LIBKVM_OBJ)))) $(LIBKVM_OBJ): $(OUTPUT)/%.o: %.c
@@ -40,3 +40,12 @@ $(OUTPUT)/libkvm.a: $(LIBKVM_OBJ) all: $(STATIC_LIBS) $(TEST_GEN_PROGS): $(STATIC_LIBS) $(STATIC_LIBS):| khdr + +cscope: include_paths = $(LINUX_TOOL_INCLUDE) $(LINUX_HDR_PATH) include lib .. +cscope: + $(RM) cscope.* + (find $(include_paths) -name '*.h' \ + -exec realpath --relative-base=$(PWD) {} \;; \ + find . -name '*.c' \ + -exec realpath --relative-base=$(PWD) {} \;) | sort -u > cscope.files + cscope -b
Signature · 1 line
2.17.1
PATCH 5/13 kvm: selftests: tidy up kvm_util
In reply to Andrew JonesTidy up kvm-util code: code/comment formatting, remove unused code, and move x86 specific code out. We also move vcpu_dump() out of common code, because not all arches (AArch64) have KVM_GET_REGS.
--- .../testing/selftests/kvm/include/kvm_util.h | 77 ++-- .../selftests/kvm/include/x86_64/processor.h | 16 +- tools/testing/selftests/kvm/lib/kvm_util.c | 372 +++++------------- .../selftests/kvm/lib/kvm_util_internal.h | 22 +- .../selftests/kvm/lib/x86_64/processor.c | 179 +++++++++ .../selftests/kvm/x86_64/dirty_log_test.c | 1 + 6 files changed, 337 insertions(+), 330 deletions(-)
tools/testing/selftests/kvm/include/kvm_util.h +26 -51
@@ -17,12 +17,6 @@#include "sparsebit.h" -/* -* Memslots can't cover the gfn starting at this gpa otherwise vCPUs can't be -* created. Only applies to VMs using EPT. -*/ -#define KVM_DEFAULT_IDENTITY_MAP_ADDRESS 0xfffbc000ul -/* Callers of kvm_util only have an incomplete/opaque description of the * structure kvm_util is using to maintain the state of a VM.
@@ -33,11 +27,11 @@ typedef uint64_t vm_paddr_t; /* Virtual Machine (Guest) physical address */typedef uint64_t vm_vaddr_t ;/* Virtual Machine (Guest) virtual address */ /* Minimum allocated guest virtual and physical addresses */ -#define KVM_UTIL_MIN_VADDR 0x2000 +#define KVM_UTIL_MIN_VADDR 0x2000#define DEFAULT_GUEST_PHY_PAGES 512#define DEFAULT_GUEST_STACK_VADDR_MIN 0xab6000 -#define DEFAULT_STACK_PGS 5 +#define DEFAULT_STACK_PGS 5enum vm_guest_mode {VM_MODE_FLAT48PG ,
@@ -57,15 +51,15 @@ void kvm_vm_restart(struct kvm_vm *vmp, int perm);void kvm_vm_release (struct kvm_vm *vmp);void kvm_vm_get_dirty_log (struct kvm_vm *vm,int slot,void *log); -int kvm_memcmp_hva_gva (void *hva, -struct kvm_vm *vm,const vm_vaddr_t gva,size_t len); +int kvm_memcmp_hva_gva (void *hva,struct kvm_vm *vm,const vm_vaddr_t gva, +size_t len);void kvm_vm_elf_load (struct kvm_vm *vm,const char *filename, -uint32_t data_memslot,uint32_t pgd_memslot); +uint32_t data_memslot,uint32_t pgd_memslot);void vm_dump (FILE *stream,struct kvm_vm *vm,uint8_t indent); -void vcpu_dump (FILE *stream,struct kvm_vm *vm, -uint32_t vcpuid,uint8_t indent); +void vcpu_dump (FILE *stream,struct kvm_vm *vm,uint32_t vcpuid, +uint8_t indent);void vm_create_irqchip (struct kvm_vm *vm);
@@ -74,13 +68,14 @@ void vm_userspace_mem_region_add(struct kvm_vm *vm, uint64_t guest_paddr, uint32_t slot, uint64_t npages, uint32_t flags); -void vcpu_ioctl (struct kvm_vm *vm, -uint32_t vcpuid,unsigned long ioctl,void *arg); +void vcpu_ioctl (struct kvm_vm *vm,uint32_t vcpuid,unsigned long ioctl, +void *arg);void vm_ioctl (struct kvm_vm *vm,unsigned long ioctl,void *arg);void vm_mem_region_set_flags (struct kvm_vm *vm,uint32_t slot,uint32_t flags); -void vm_vcpu_add (struct kvm_vm *vm,uint32_t vcpuid,int pgd_memslot,int gdt_memslot); +void vm_vcpu_add (struct kvm_vm *vm,uint32_t vcpuid,int pgd_memslot, +int gdt_memslot);vm_vaddr_t vm_vaddr_alloc (struct kvm_vm *vm,size_t sz,vm_vaddr_t vaddr_min, -uint32_t data_memslot,uint32_t pgd_memslot); +uint32_t data_memslot,uint32_t pgd_memslot);void virt_map (struct kvm_vm *vm,uint64_t vaddr,uint64_t paddr,size_t size,uint32_t pgd_memslot);void *addr_gpa2hva (struct kvm_vm *vm,vm_paddr_t gpa);
@@ -92,53 +87,33 @@ struct kvm_run *vcpu_state(struct kvm_vm *vm, uint32_t vcpuid);void vcpu_run (struct kvm_vm *vm,uint32_t vcpuid);int _vcpu_run (struct kvm_vm *vm,uint32_t vcpuid);void vcpu_set_mp_state (struct kvm_vm *vm,uint32_t vcpuid, -struct kvm_mp_state *mp_state); -void vcpu_regs_get (struct kvm_vm *vm, -uint32_t vcpuid,struct kvm_regs *regs); -void vcpu_regs_set (struct kvm_vm *vm, -uint32_t vcpuid,struct kvm_regs *regs); +struct kvm_mp_state *mp_state); +void vcpu_regs_get (struct kvm_vm *vm,uint32_t vcpuid,struct kvm_regs *regs); +void vcpu_regs_set (struct kvm_vm *vm,uint32_t vcpuid,struct kvm_regs *regs);void vcpu_args_set (struct kvm_vm *vm,uint32_t vcpuid,unsigned int num, ...); -void vcpu_sregs_get (struct kvm_vm *vm, -uint32_t vcpuid,struct kvm_sregs *sregs); -void vcpu_sregs_set (struct kvm_vm *vm, -uint32_t vcpuid,struct kvm_sregs *sregs); -int _vcpu_sregs_set (struct kvm_vm *vm, -uint32_t vcpuid,struct kvm_sregs *sregs); +void vcpu_sregs_get (struct kvm_vm *vm,uint32_t vcpuid, +struct kvm_sregs *sregs); +void vcpu_sregs_set (struct kvm_vm *vm,uint32_t vcpuid, +struct kvm_sregs *sregs); +int _vcpu_sregs_set (struct kvm_vm *vm,uint32_t vcpuid, +struct kvm_sregs *sregs);void vcpu_events_get (struct kvm_vm *vm,uint32_t vcpuid, -struct kvm_vcpu_events *events); +struct kvm_vcpu_events *events);void vcpu_events_set (struct kvm_vm *vm,uint32_t vcpuid, -struct kvm_vcpu_events *events); +struct kvm_vcpu_events *events);const char *exit_reason_str (unsigned int exit_reason);void virt_pgd_alloc (struct kvm_vm *vm,uint32_t pgd_memslot);void virt_pg_map (struct kvm_vm *vm,uint64_t vaddr,uint64_t paddr, -uint32_t pgd_memslot); -vm_paddr_t vm_phy_page_alloc (struct kvm_vm *vm, -vm_paddr_t paddr_min,uint32_t memslot); - -struct kvm_cpuid2 *kvm_get_supported_cpuid (void ); -void vcpu_set_cpuid ( -struct kvm_vm *vm,uint32_t vcpuid,struct kvm_cpuid2 *cpuid); - -struct kvm_cpuid_entry2 * -kvm_get_supported_cpuid_index (uint32_t function,uint32_t index); - -static inline struct kvm_cpuid_entry2 * -kvm_get_supported_cpuid_entry (uint32_t function) -{ -return kvm_get_supported_cpuid_index (function,0 ); -} +uint32_t pgd_memslot); +vm_paddr_t vm_phy_page_alloc (struct kvm_vm *vm,vm_paddr_t paddr_min, +uint32_t memslot);struct kvm_vm *vm_create_default (uint32_t vcpuid,uint64_t extra_mem_size,void *guest_code);void vm_vcpu_add_default (struct kvm_vm *vm,uint32_t vcpuid,void *guest_code); -typedef void (*vmx_guest_code_t )(vm_vaddr_t vmxon_vaddr, -vm_paddr_t vmxon_paddr, -vm_vaddr_t vmcs_vaddr, -vm_paddr_t vmcs_paddr); -struct kvm_userspace_memory_region *kvm_userspace_memory_region_find (struct kvm_vm *vm,uint64_t start,uint64_t end);
tools/testing/selftests/kvm/include/x86_64/processor.h +15 -1
@@ -305,7 +305,21 @@ static inline unsigned long get_xmm(int n)struct kvm_x86_state ;struct kvm_x86_state *vcpu_save_state (struct kvm_vm *vm,uint32_t vcpuid); -void vcpu_load_state (struct kvm_vm *vm,uint32_t vcpuid,struct kvm_x86_state *state); +void vcpu_load_state (struct kvm_vm *vm,uint32_t vcpuid, +struct kvm_x86_state *state); + +struct kvm_cpuid2 *kvm_get_supported_cpuid (void ); +void vcpu_set_cpuid (struct kvm_vm *vm,uint32_t vcpuid, +struct kvm_cpuid2 *cpuid); + +struct kvm_cpuid_entry2 * +kvm_get_supported_cpuid_index (uint32_t function,uint32_t index); + +static inline struct kvm_cpuid_entry2 * +kvm_get_supported_cpuid_entry (uint32_t function) +{ +return kvm_get_supported_cpuid_index (function,0 ); +}/* * Basic CPU control in CR0
tools/testing/selftests/kvm/lib/kvm_util.c +106 -266
798 lines not on this page · raw message
tools/testing/selftests/kvm/lib/kvm_util_internal.h +10 -12
@@ -11,18 +11,19 @@#include "sparsebit.h" +#define KVM_DEV_PATH "/dev/kvm" +#ifndef BITS_PER_BYTE -#define BITS_PER_BYTE 8 +#define BITS_PER_BYTE 8#endif #ifndef BITS_PER_LONG -#define BITS_PER_LONG (BITS_PER_BYTE * sizeof(long)) +#define BITS_PER_LONG (BITS_PER_BYTE * sizeof(long))#endif #define DIV_ROUND_UP (n, d) (((n) + (d) - 1) / (d)) -#define BITS_TO_LONGS (nr) DIV_ROUND_UP(nr, BITS_PER_LONG) +#define BITS_TO_LONGS (nr) DIV_ROUND_UP(nr, BITS_PER_LONG) -/* Concrete definition of struct kvm_vm. */ struct userspace_mem_region {struct userspace_mem_region *next, *prev;struct kvm_userspace_memory_region region;
@@ -52,7 +53,6 @@ struct kvm_vm {struct userspace_mem_region *userspace_mem_region_head;struct sparsebit *vpages_valid;struct sparsebit *vpages_mapped; -bool has_irqchip;bool pgd_created;vm_paddr_t pgd;
@@ -60,13 +60,11 @@ struct kvm_vm { vm_vaddr_t tss; }; -struct vcpu *vcpu_find (struct kvm_vm *vm, -uint32_t vcpuid); -void vcpu_setup (struct kvm_vm *vm,int vcpuid,int pgd_memslot,int gdt_memslot); +struct vcpu *vcpu_find (struct kvm_vm *vm,uint32_t vcpuid); +void vcpu_setup (struct kvm_vm *vm,int vcpuid,int pgd_memslot, +int gdt_memslot);void virt_dump (FILE *stream,struct kvm_vm *vm,uint8_t indent); -void regs_dump (FILE *stream,struct kvm_regs *regs, -uint8_t indent); -void sregs_dump (FILE *stream,struct kvm_sregs *sregs, -uint8_t indent); +void regs_dump (FILE *stream,struct kvm_regs *regs,uint8_t indent); +void sregs_dump (FILE *stream,struct kvm_sregs *sregs,uint8_t indent);#endif /* SELFTEST_KVM_UTIL_INTERNAL_H */
tools/testing/selftests/kvm/lib/x86_64/processor.c +179 -0
197 lines not on this page · raw message
tools/testing/selftests/kvm/x86_64/dirty_log_test.c +1 -0
@@ -15,6 +15,7 @@#include "test_util.h" #include "kvm_util.h" +#include "processor.h" #define DEBUG printf
Signature · 1 line
2.17.1
PATCH 6/13 kvm: selftests: add vm_phy_pages_alloc
In reply to Andrew Jones--- .../testing/selftests/kvm/include/kvm_util.h | 2 + tools/testing/selftests/kvm/lib/kvm_util.c | 60 ++++++++++++------- 2 files changed, 39 insertions(+), 23 deletions(-)
tools/testing/selftests/kvm/include/kvm_util.h +2 -0
@@ -109,6 +109,8 @@ void virt_pg_map(struct kvm_vm *vm, uint64_t vaddr, uint64_t paddr, uint32_t pgd_memslot);vm_paddr_t vm_phy_page_alloc (struct kvm_vm *vm,vm_paddr_t paddr_min,uint32_t memslot); +vm_paddr_t vm_phy_pages_alloc (struct kvm_vm *vm,size_t num, +vm_paddr_t paddr_min,uint32_t memslot);struct kvm_vm *vm_create_default (uint32_t vcpuid,uint64_t extra_mem_size,void *guest_code);
tools/testing/selftests/kvm/lib/kvm_util.c +37 -23
88 lines not on this page · raw message
Signature · 1 line
2.17.1
PATCH 7/13 kvm: selftests: add virt mem support for aarch64
In reply to Andrew Jones--- .../selftests/kvm/lib/aarch64/processor.c | 216 ++++++++++++++++++ tools/testing/selftests/kvm/lib/kvm_util.c | 2 + .../selftests/kvm/lib/kvm_util_internal.h | 2 + 3 files changed, 220 insertions(+) create mode 100644 tools/testing/selftests/kvm/lib/aarch64/processor.c
tools/testing/selftests/kvm/lib/aarch64/processor.c new file +216 -0
@@ -0,0 +1,216 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* AArch64 code +* +* Copyright (C) 2018, Red Hat, Inc. +*/ + +#include "kvm_util.h" +#include "../kvm_util_internal.h" + +#define KVM_GUEST_PAGE_TABLE_MIN_PADDR 0x180000 + +static uint64_t page_align (struct kvm_vm *vm,uint64_t v) +{ +return (v + vm->page_size) & ~(vm->page_size -1 ); +} + +static uint64_t pgd_index (struct kvm_vm *vm,vm_vaddr_t gva) +{ +unsigned int shift = (vm->pgtable_levels -1 ) * (vm->page_shift -3 ) + vm->page_shift; +uint64_t mask = (1UL << (vm->va_bits - shift)) -1 ; + +return (gva >> shift) & mask; +} + +static uint64_t pud_index (struct kvm_vm *vm,vm_vaddr_t gva) +{ +unsigned int shift =2 * (vm->page_shift -3 ) + vm->page_shift; +uint64_t mask = (1UL << (vm->page_shift -3 )) -1 ; + +TEST_ASSERT (vm->pgtable_levels ==4 , +"Mode %d does not have 4 page table levels" , vm->mode); + +return (gva >> shift) & mask; +} + +static uint64_t pmd_index (struct kvm_vm *vm,vm_vaddr_t gva) +{ +unsigned int shift = (vm->page_shift -3 ) + vm->page_shift; +uint64_t mask = (1UL << (vm->page_shift -3 )) -1 ; + +TEST_ASSERT (vm->pgtable_levels >=3 , +"Mode %d does not have >= 3 page table levels" , vm->mode); + +return (gva >> shift) & mask; +} + +static uint64_t pte_index (struct kvm_vm *vm,vm_vaddr_t gva) +{ +uint64_t mask = (1UL << (vm->page_shift -3 )) -1 ; +return (gva >> vm->page_shift) & mask; +} + +static uint64_t pte_addr (struct kvm_vm *vm,uint64_t entry) +{ +uint64_t mask = ((1UL << (vm->va_bits - vm->page_shift)) -1 ) << vm->page_shift; +return entry & mask; +} + +static uint64_t ptrs_per_pgd (struct kvm_vm *vm) +{ +unsigned int shift = (vm->pgtable_levels -1 ) * (vm->page_shift -3 ) + vm->page_shift; +return 1 << (vm->va_bits - shift); +} + +static uint64_t ptrs_per_pte (struct kvm_vm *vm) +{ +return 1 << (vm->page_shift -3 ); +} + +void virt_pgd_alloc (struct kvm_vm *vm,uint32_t pgd_memslot) +{ +int rc; + +if (!vm->pgd_created) { +vm_paddr_t paddr =vm_phy_pages_alloc (vm, +page_align (vm,ptrs_per_pgd (vm) *8 ) / vm->page_size, +KVM_GUEST_PAGE_TABLE_MIN_PADDR , pgd_memslot); + vm->pgd = paddr; + vm->pgd_created = true; + } +} + +void _virt_pg_map (struct kvm_vm *vm,uint64_t vaddr,uint64_t paddr, +uint32_t pgd_memslot,uint64_t flags) +{ +uint8_t attr_idx = flags &7 ; +uint64_t *ptep; + +TEST_ASSERT ((vaddr % vm->page_size) ==0 , +"Virtual address not on page boundary,\n" +" vaddr: 0x%lx vm->page_size: 0x%x" , vaddr, vm->page_size); +TEST_ASSERT (sparsebit_is_set (vm->vpages_valid, + (vaddr >> vm->page_shift)), +"Invalid virtual address, vaddr: 0x%lx" , vaddr); +TEST_ASSERT ((paddr % vm->page_size) ==0 , +"Physical address not on page boundary,\n" +" paddr: 0x%lx vm->page_size: 0x%x" , paddr, vm->page_size); +TEST_ASSERT ((paddr >> vm->page_shift) <= vm->max_gfn, +"Physical address beyond beyond maximum supported,\n" +" paddr: 0x%lx vm->max_gfn: 0x%lx vm->page_size: 0x%x" , + paddr, vm->max_gfn, vm->page_size); + + ptep =addr_gpa2hva (vm, vm->pgd) +pgd_index (vm, vaddr) *8 ; +if (!*ptep) { + *ptep =vm_phy_page_alloc (vm,KVM_GUEST_PAGE_TABLE_MIN_PADDR , pgd_memslot); + *ptep |=3 ; + } + +switch (vm->pgtable_levels) { +case 4 : + ptep =addr_gpa2hva (vm,pte_addr (vm, *ptep)) +pud_index (vm, vaddr) *8 ; +if (!*ptep) { + *ptep =vm_phy_page_alloc (vm,KVM_GUEST_PAGE_TABLE_MIN_PADDR , pgd_memslot); + *ptep |=3 ; + } +/* fall through */ +case 3 : + ptep =addr_gpa2hva (vm,pte_addr (vm, *ptep)) +pmd_index (vm, vaddr) *8 ; +if (!*ptep) { + *ptep =vm_phy_page_alloc (vm,KVM_GUEST_PAGE_TABLE_MIN_PADDR , pgd_memslot); + *ptep |=3 ; + } +/* fall through */ +case 2 : + ptep =addr_gpa2hva (vm,pte_addr (vm, *ptep)) +pte_index (vm, vaddr) *8 ; +break ; +default : +TEST_ASSERT (false,"Page table levels must be 2, 3, or 4" ); + } + + *ptep = paddr |3 ; + *ptep |= (attr_idx <<2 ) | (1 <<10 )/* Access Flag */ ; +} + +void virt_pg_map (struct kvm_vm *vm,uint64_t vaddr,uint64_t paddr, +uint32_t pgd_memslot) +{ +uint64_t attr_idx =4 ;/* NORMAL (See DEFAULT_MAIR_EL1) */ + +_virt_pg_map (vm, vaddr, paddr, pgd_memslot, attr_idx); +} + +vm_paddr_t addr_gva2gpa (struct kvm_vm *vm,vm_vaddr_t gva) +{ +uint64_t *ptep; + +if (!vm->pgd_created) + goto unmapped_gva; + + ptep =addr_gpa2hva (vm, vm->pgd) +pgd_index (vm, gva) *8 ; +if (!ptep) + goto unmapped_gva; + +switch (vm->pgtable_levels) { +case 4 : + ptep =addr_gpa2hva (vm,pte_addr (vm, *ptep)) +pud_index (vm, gva) *8 ; +if (!ptep) + goto unmapped_gva; +/* fall through */ +case 3 : + ptep =addr_gpa2hva (vm,pte_addr (vm, *ptep)) +pmd_index (vm, gva) *8 ; +if (!ptep) + goto unmapped_gva; +/* fall through */ +case 2 : + ptep =addr_gpa2hva (vm,pte_addr (vm, *ptep)) +pte_index (vm, gva) *8 ; +if (!ptep) + goto unmapped_gva; +break ; +default : +TEST_ASSERT (false,"Page table levels must be 2, 3, or 4" ); + } + +return pte_addr (vm, *ptep) + (gva & (vm->page_size -1 )); + +unmapped_gva: +TEST_ASSERT (false,"No mapping for vm virtual address, " +"gva: 0x%lx" , gva); +} + +static void pte_dump (FILE *stream,struct kvm_vm *vm,uint8_t indent,uint64_t page,int level) +{ +#ifdef DEBUG_VM +static const char *const type[] = {"" ,"pud" ,"pmd" ,"pte" }; +uint64_t pte, *ptep; + +if (level ==4 ) +return ; + +for (pte = page; pte < page +ptrs_per_pte (vm) *8 ; pte +=8 ) { + ptep =addr_gpa2hva (vm, pte); +if (!*ptep) +continue ; +printf ("%*s%s: %lx: %lx at %p\n" , indent,"" , type[level], pte, *ptep, ptep); +pte_dump (stream, vm, indent +1 ,pte_addr (vm, *ptep), level +1 ); + } +#endif +} + +void virt_dump (FILE *stream,struct kvm_vm *vm,uint8_t indent) +{ +int level =4 - (vm->pgtable_levels -1 ); +uint64_t pgd, *ptep; + +if (!vm->pgd_created) +return ; + +for (pgd = vm->pgd; pgd < vm->pgd +ptrs_per_pgd (vm) *8 ; pgd +=8 ) { + ptep =addr_gpa2hva (vm, pgd); +if (!*ptep) +continue ; +printf ("%*spgd: %lx: %lx at %p\n" , indent,"" , pgd, *ptep, ptep); +pte_dump (stream, vm, indent +1 ,pte_addr (vm, *ptep), level); + } +}
tools/testing/selftests/kvm/lib/kvm_util.c +2 -0
@@ -106,6 +106,8 @@ struct kvm_vm *vm_create(enum vm_guest_mode mode, uint64_t phy_pages, int perm)/* Setup mode specific traits. */ switch (vm->mode) {case VM_MODE_FLAT48PG : + vm->pgtable_levels =4 ; + vm->va_bits =48 ; vm->page_size =0x1000 ; vm->page_shift =12 ;
tools/testing/selftests/kvm/lib/kvm_util_internal.h +2 -0
@@ -46,6 +46,8 @@ struct kvm_vm {int mode;int kvm_fd;int fd; +unsigned int pgtable_levels; +unsigned int va_bits;unsigned int page_size;unsigned int page_shift;uint64_t max_gfn;
Signature · 1 line
2.17.1
PATCH 8/13 kvm: selftests: add vcpu support for aarch64
In reply to Andrew JonesThis code adds VM and VCPU setup code for the VM_MODE_FLAT48PG mode. The VM_MODE_FLAT48PG isn't yet fully supportable, as it defines the guest physical address limit as 52-bits, and KVM currently only supports guests with up to 40-bit physical addresses (see KVM_PHYS_SHIFT). VM_MODE_FLAT48PG will work fine, though, as long as no >= 40-bit physical addresses are used.
--- .../selftests/kvm/include/aarch64/processor.h | 55 ++++++++++++ .../selftests/kvm/lib/aarch64/processor.c | 83 +++++++++++++++++++ 2 files changed, 138 insertions(+) create mode 100644 tools/testing/selftests/kvm/include/aarch64/processor.h
tools/testing/selftests/kvm/include/aarch64/processor.h new file +55 -0
@@ -0,0 +1,55 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* AArch64 processor specific defines +* +* Copyright (C) 2018, Red Hat, Inc. +*/ +#ifndef SELFTEST_KVM_PROCESSOR_H +#define SELFTEST_KVM_PROCESSOR_H + +#include "kvm_util.h" + + +#define ARM64_CORE_REG (x) (KVM_REG_ARM64 | KVM_REG_SIZE_U64 | \ + KVM_REG_ARM_CORE | KVM_REG_ARM_CORE_REG(x)) + +#define CPACR_EL1 3, 0, 1, 0, 2 +#define TCR_EL1 3, 0, 2, 0, 2 +#define MAIR_EL1 3, 0, 10, 2, 0 +#define TTBR0_EL1 3, 0, 2, 0, 0 +#define SCTLR_EL1 3, 0, 1, 0, 0 + +/* +* Default MAIR +* index attribute +* DEVICE_nGnRnE 0 0000:0000 +* DEVICE_nGnRE 1 0000:0100 +* DEVICE_GRE 2 0000:1100 +* NORMAL_NC 3 0100:0100 +* NORMAL 4 1111:1111 +* NORMAL_WT 5 1011:1011 +*/ +#define DEFAULT_MAIR_EL1 ((0x00ul << (0 * 8)) | \ + (0x04ul << (1 * 8)) | \ + (0x0cul << (2 * 8)) | \ + (0x44ul << (3 * 8)) | \ + (0xfful << (4 * 8)) | \ + (0xbbul << (5 * 8))) + +static inline void get_reg (struct kvm_vm *vm,uint32_t vcpuid,uint64_t id,uint64_t *addr) +{ +struct kvm_one_reg reg; + reg.id = id; + reg.addr = (uint64_t )addr; +vcpu_ioctl (vm, vcpuid,KVM_GET_ONE_REG , ®); +} + +static inline void set_reg (struct kvm_vm *vm,uint32_t vcpuid,uint64_t id,uint64_t val) +{ +struct kvm_one_reg reg; + reg.id = id; + reg.addr = (uint64_t )&val; +vcpu_ioctl (vm, vcpuid,KVM_SET_ONE_REG , ®); +} + +#endif /* SELFTEST_KVM_PROCESSOR_H */
tools/testing/selftests/kvm/lib/aarch64/processor.c +83 -0
96 lines not on this page · raw message
Signature · 1 line
2.17.1
PATCH 9/13 kvm: selftests: port dirty_log_test to aarch64
In reply to Andrew JonesWhile we're messing with the code for the port and to support guest page sizes that are less than the host page size, we also make some code formatting cleanups and apply sync_global_to_guest().
---
tools/testing/selftests/kvm/.gitignore | 2 +-
tools/testing/selftests/kvm/Makefile | 5 +-
.../kvm/{x86_64 => }/dirty_log_test.c | 163 +++++++++---------
3 files changed, 90 insertions(+), 80 deletions(-)
rename tools/testing/selftests/kvm/{x86_64 => }/dirty_log_test.c (65%)tools/testing/selftests/kvm/.gitignore +1 -1
@@ -3,4 +3,4 @@ /x86_64/sync_regs_test /x86_64/vmx_tsc_adjust_test /x86_64/state_test -/x86_64/dirty_log_test +/dirty_log_test
tools/testing/selftests/kvm/Makefile +4 -1
@@ -5,13 +5,16 @@ UNAME_M := $(shell uname -m) LIBKVM = lib/assert.c lib/elf.c lib/io.c lib/kvm_util.c lib/ucall.c lib/sparsebit.c LIBKVM_x86_64 = lib/x86_64/processor.c lib/x86_64/vmx.c +LIBKVM_aarch64 = lib/aarch64/processor.c TEST_GEN_PROGS_x86_64 = x86_64/set_sregs_test TEST_GEN_PROGS_x86_64 += x86_64/sync_regs_test TEST_GEN_PROGS_x86_64 += x86_64/vmx_tsc_adjust_test TEST_GEN_PROGS_x86_64 += x86_64/cr4_cpuid_sync_test TEST_GEN_PROGS_x86_64 += x86_64/state_test -TEST_GEN_PROGS_x86_64 += x86_64/dirty_log_test +TEST_GEN_PROGS_x86_64 += dirty_log_test + +TEST_GEN_PROGS_aarch64 += dirty_log_test TEST_GEN_PROGS += $(TEST_GEN_PROGS_$(UNAME_M)) LIBKVM += $(LIBKVM_$(UNAME_M))
tools/testing/selftests/kvm/dirty_log_test.c renamed from tools/testing/selftests/kvm/x86_64/dirty_log_test.c +85 -78
287 lines not on this page · raw message
Signature · 1 line
2.17.1
PATCH 10/13 kvm: selftests: introduce new VM mode for 64K pages
In reply to Andrew JonesRename VM_MODE_FLAT48PG to be more descriptive of its config and add a new config that has the same parameters, except with 64K pages.
--- .../testing/selftests/kvm/include/kvm_util.h | 7 ++- .../selftests/kvm/lib/aarch64/processor.c | 8 +++- tools/testing/selftests/kvm/lib/kvm_util.c | 47 ++++++++++++------- .../selftests/kvm/lib/kvm_util_internal.h | 1 + .../selftests/kvm/lib/x86_64/processor.c | 10 ++-- 5 files changed, 48 insertions(+), 25 deletions(-)
tools/testing/selftests/kvm/include/kvm_util.h +6 -1
@@ -34,9 +34,14 @@ typedef uint64_t vm_vaddr_t; /* Virtual Machine (Guest) virtual address */#define DEFAULT_STACK_PGS 5enum vm_guest_mode { -VM_MODE_FLAT48PG , +VM_MODE_P52V48_4K , +VM_MODE_P52V48_64K , +NUM_VM_MODES , }; +#define vm_guest_mode_string (m) vm_guest_mode_string[m] +extern const char *const vm_guest_mode_string[]; +enum vm_mem_backing_src_type {VM_MEM_SRC_ANONYMOUS ,VM_MEM_SRC_ANONYMOUS_THP ,
tools/testing/selftests/kvm/lib/aarch64/processor.c +6 -2
@@ -226,7 +226,7 @@ struct kvm_vm *vm_create_default(uint32_t vcpuid, uint64_t extra_mem_pages,uint64_t extra_pg_pages = (extra_mem_pages / ptrs_per_4k_pte) *2 ;struct kvm_vm *vm; - vm =vm_create (VM_MODE_FLAT48PG ,DEFAULT_GUEST_PHY_PAGES + extra_pg_pages,O_RDWR ); + vm =vm_create (VM_MODE_P52V48_4K ,DEFAULT_GUEST_PHY_PAGES + extra_pg_pages,O_RDWR );kvm_vm_elf_load (vm, program_invocation_name,0 ,0 );vm_vcpu_add_default (vm, vcpuid, guest_code);
@@ -267,10 +267,14 @@ void vcpu_setup(struct kvm_vm *vm, int vcpuid, int pgd_memslot, int gdt_memslot)get_reg (vm, vcpuid,ARM64_SYS_REG (TCR_EL1 ), &tcr_el1);switch (vm->mode) { -case VM_MODE_FLAT48PG : +case VM_MODE_P52V48_4K : tcr_el1 |=0ul <<14 ;/* TG0 = 4KB */ tcr_el1 |=6ul <<32 ;/* IPS = 52 bits */ break ; +case VM_MODE_P52V48_64K : + tcr_el1 |=1ul <<14 ;/* TG0 = 64KB */ + tcr_el1 |=6ul <<32 ;/* IPS = 52 bits */ +break ;default :TEST_ASSERT (false,"Unknown guest mode, mode: 0x%x" , vm->mode); }
tools/testing/selftests/kvm/lib/kvm_util.c +30 -17
90 lines not on this page · raw message
tools/testing/selftests/kvm/lib/kvm_util_internal.h +1 -0
@@ -47,6 +47,7 @@ struct kvm_vm {int kvm_fd;int fd;unsigned int pgtable_levels; +unsigned int pa_bits;unsigned int va_bits;unsigned int page_size;unsigned int page_shift;
tools/testing/selftests/kvm/lib/x86_64/processor.c +5 -5
@@ -231,7 +231,7 @@ void virt_pgd_alloc(struct kvm_vm *vm, uint32_t pgd_memslot) {int rc; -TEST_ASSERT (vm->mode ==VM_MODE_FLAT48PG ,"Attempt to use " +TEST_ASSERT (vm->mode ==VM_MODE_P52V48_4K ,"Attempt to use " "unknown or unsupported guest mode, mode: 0x%x" , vm->mode);/* If needed, create page map l4 table. */
@@ -264,7 +264,7 @@ void virt_pg_map(struct kvm_vm *vm, uint64_t vaddr, uint64_t paddr,uint16_t index[4 ];struct pageMapL4Entry *pml4e; -TEST_ASSERT (vm->mode ==VM_MODE_FLAT48PG ,"Attempt to use " +TEST_ASSERT (vm->mode ==VM_MODE_P52V48_4K ,"Attempt to use " "unknown or unsupported guest mode, mode: 0x%x" , vm->mode); TEST_ASSERT((vaddr % vm->page_size) == 0,
@@ -551,7 +551,7 @@ vm_paddr_t addr_gva2gpa(struct kvm_vm *vm, vm_vaddr_t gva)struct pageTableEntry *pte;void *hva; -TEST_ASSERT (vm->mode ==VM_MODE_FLAT48PG ,"Attempt to use " +TEST_ASSERT (vm->mode ==VM_MODE_P52V48_4K ,"Attempt to use " "unknown or unsupported guest mode, mode: 0x%x" , vm->mode); index[0 ] = (gva >>12 ) &0x1ffu ;
@@ -624,7 +624,7 @@ void vcpu_setup(struct kvm_vm *vm, int vcpuid, int pgd_memslot, int gdt_memslot)kvm_setup_gdt (vm, &sregs.gdt, gdt_memslot, pgd_memslot);switch (vm->mode) { -case VM_MODE_FLAT48PG : +case VM_MODE_P52V48_4K : sregs.cr0 =X86_CR0_PE |X86_CR0_NE |X86_CR0_PG ; sregs.cr4 |=X86_CR4_PAE |X86_CR4_OSFXSR ; sregs.efer |= (EFER_LME |EFER_LMA |EFER_NX );
@@ -873,7 +873,7 @@ struct kvm_vm *vm_create_default(uint32_t vcpuid, uint64_t extra_mem_pages,uint64_t extra_pg_pages = extra_mem_pages /512 *2 ;/* Create VM */ - vm =vm_create (VM_MODE_FLAT48PG , + vm =vm_create (VM_MODE_P52V48_4K ,DEFAULT_GUEST_PHY_PAGES + extra_pg_pages,O_RDWR );
Signature · 1 line
2.17.1
PATCH 11/13 kvm: selftests: dirty_log_test: also test 64K pages on aarch64
In reply to Andrew Jones--- tools/testing/selftests/kvm/dirty_log_test.c | 183 ++++++++++++++----- 1 file changed, 137 insertions(+), 46 deletions(-)
tools/testing/selftests/kvm/dirty_log_test.c +137 -46
@@ -5,6 +5,8 @@* Copyright (C) 2018, Red Hat, Inc. */ +#define _GNU_SOURCE/* for program_invocation_name */ +#include <stdio.h> #include <stdlib.h> #include <unistd.h>
@@ -30,9 +32,6 @@*/ #define TEST_MEM_OFFSET (1ul << 30)/* 1G */ -/* Size of the testing memory slot */ -#define TEST_MEM_PAGES (1ul << 18)/* 1G for 4K pages */ -/* How many pages to dirty for each guest loop */ #define TEST_PAGES_PER_LOOP 1024
@@ -50,6 +49,7 @@*/ static uint64_t host_page_size;static uint64_t guest_page_size; +static uint64_t guest_num_pages;static uint64_t random_array[TEST_PAGES_PER_LOOP ];static uint64_t iteration;
@@ -64,7 +64,7 @@ static void guest_code(void)while (true) {for (i =0 ; i <TEST_PAGES_PER_LOOP ; i++) {uint64_t addr =TEST_MEM_OFFSET ; - addr += (READ_ONCE (random_array[i]) %TEST_MEM_PAGES ) + addr += (READ_ONCE (random_array[i]) % guest_num_pages) * guest_page_size; addr &= ~(host_page_size -1 ); *(uint64_t *)addr =READ_ONCE (iteration);
@@ -141,8 +141,10 @@ static void vm_dirty_log_verify(unsigned long *bmap) {uint64_t page;uint64_t *value_ptr; +uint64_t step = host_page_size >= guest_page_size ?1 : + guest_page_size / host_page_size; -for (page =0 ; page < host_num_pages; page++) { +for (page =0 ; page < host_num_pages; page += step) { value_ptr = host_test_mem + page * host_page_size;/* If this is a special page that we were tracking... */
@@ -203,71 +205,64 @@ static void vm_dirty_log_verify(unsigned long *bmap) } } -static void help (char *name) +static struct kvm_vm *create_vm (enum vm_guest_mode mode,uint32_t vcpuid, +uint64_t extra_mem_pages,void *guest_code) { -puts ("" ); -printf ("usage: %s [-i iterations] [-I interval] [-h]\n" , name); -puts ("" ); -printf (" -i: specify iteration counts (default: %" PRIu64")\n" , -TEST_HOST_LOOP_N ); -printf (" -I: specify interval in ms (default: %" PRIu64" ms)\n" , -TEST_HOST_LOOP_INTERVAL ); -puts ("" ); -exit (0 ); +struct kvm_vm *vm; +uint64_t extra_pg_pages = extra_mem_pages /512 *2 ; + + vm =vm_create (mode,DEFAULT_GUEST_PHY_PAGES + extra_pg_pages,O_RDWR ); +kvm_vm_elf_load (vm, program_invocation_name,0 ,0 ); +#ifdef __x86_64__ +vm_create_irqchip (vm); +#endif +vm_vcpu_add_default (vm, vcpuid, guest_code); +return vm; } -int main (int argc,char *argv[]) +static void run_test (enum vm_guest_mode mode,unsigned long iterations, +unsigned long interval) { +unsigned int guest_page_shift;pthread_t vcpu_thread;struct kvm_vm *vm; -unsigned long iterations =TEST_HOST_LOOP_N ; -unsigned long interval =TEST_HOST_LOOP_INTERVAL ;unsigned long *bmap; -int opt; -while ((opt =getopt (argc, argv,"hi:I:" )) !=-1 ) { -switch (opt) { -case 'i' : - iterations =strtol (optarg,NULL ,10 ); -break ; -case 'I' : - interval =strtol (optarg,NULL ,10 ); -break ; -case 'h' : -default : -help (argv[0 ]); -break ; - } +switch (mode) { +case VM_MODE_P52V48_4K : + guest_page_shift =12 ; +break ; +case VM_MODE_P52V48_64K : + guest_page_shift =16 ; +break ; +default : +TEST_ASSERT (false,"Unknown guest mode, mode: 0x%x" , mode); } -TEST_ASSERT (iterations >2 ,"Iterations must be greater than two" ); -TEST_ASSERT (interval >0 ,"Interval must be greater than zero" ); - -DEBUG ("Test iterations: %" PRIu64", interval: %" PRIu64" (ms)\n" , - iterations, interval); - -srandom (time (0 )); +DEBUG ("Testing guest mode: %s\n" ,vm_guest_mode_string (mode)); - guest_page_size =4096 ; + guest_page_size = (1ul << guest_page_shift); +/* 1G of guest page sized pages */ + guest_num_pages = (1ul << (30 - guest_page_shift)); host_page_size =getpagesize (); - host_num_pages = (TEST_MEM_PAGES * guest_page_size) / host_page_size + - !!((TEST_MEM_PAGES * guest_page_size) % host_page_size); + host_num_pages = (guest_num_pages * guest_page_size) / host_page_size + + !!((guest_num_pages * guest_page_size) % host_page_size); bmap =bitmap_alloc (host_num_pages); host_bmap_track =bitmap_alloc (host_num_pages); - vm =vm_create_default (VCPU_ID ,TEST_MEM_PAGES , guest_code); + vm =create_vm (mode,VCPU_ID , guest_num_pages, guest_code);/* Add an extra memory slot for testing dirty logging */ vm_userspace_mem_region_add (vm,VM_MEM_SRC_ANONYMOUS ,TEST_MEM_OFFSET ,TEST_MEM_SLOT_INDEX , -TEST_MEM_PAGES , + guest_num_pages,KVM_MEM_LOG_DIRTY_PAGES );/* Do 1:1 mapping for the dirty track memory slot */ virt_map (vm,TEST_MEM_OFFSET ,TEST_MEM_OFFSET , -TEST_MEM_PAGES * guest_page_size,0 ); + guest_num_pages * guest_page_size,0 );/* Cache the HVA pointer of the region */ host_test_mem =addr_gpa2hva (vm, (vm_paddr_t )TEST_MEM_OFFSET );
@@ -279,13 +274,18 @@ int main(int argc, char *argv[])ucall_init (vm,UCALL_MMIO ,NULL );#endif -/* Tell the guest about the page sizes */ +/* Export the shared variables to the guest */ sync_global_to_guest (vm, host_page_size);sync_global_to_guest (vm, guest_page_size); +sync_global_to_guest (vm, guest_num_pages);/* Start the iterations */ iteration =1 ;sync_global_to_guest (vm, iteration); + host_quit = false; + host_dirty_count =0 ; + host_clear_count =0 ; + host_track_next_count =0 ;pthread_create (&vcpu_thread,NULL , vcpu_worker, vm);
@@ -310,6 +310,97 @@ int main(int argc, char *argv[])free (host_bmap_track);ucall_uninit (vm);kvm_vm_free (vm); +} + +static struct vm_guest_modes { +enum vm_guest_mode mode; +bool supported; +bool enabled; +} vm_guest_modes[NUM_VM_MODES ] = { + {VM_MODE_P52V48_4K ,1 ,1 , }, +#ifdef __aarch64__ + {VM_MODE_P52V48_64K ,1 ,1 , }, +#else + {VM_MODE_P52V48_64K ,0 ,0 , }, +#endif +}; + +static void help (char *name) +{ +int i; + +puts ("" ); +printf ("usage: %s [-h] [-i iterations] [-I interval] [-m mode]\n" , name); +puts ("" ); +printf (" -i: specify iteration counts (default: %" PRIu64")\n" , +TEST_HOST_LOOP_N ); +printf (" -I: specify interval in ms (default: %" PRIu64" ms)\n" , +TEST_HOST_LOOP_INTERVAL ); +printf (" -m: specify the guest mode ID to test " +"(default: test all supported modes)\n" +" This option may be used multiple times.\n" +" Guest mode IDs:\n" ); +for (i =0 ; i <NUM_VM_MODES ; ++i) { +printf (" %d: %s%s\n" , + vm_guest_modes[i].mode, +vm_guest_mode_string (vm_guest_modes[i].mode), + vm_guest_modes[i].supported ?" (supported)" :"" ); + } +puts ("" ); +exit (0 ); +} + +int main (int argc,char *argv[]) +{ +unsigned long iterations =TEST_HOST_LOOP_N ; +unsigned long interval =TEST_HOST_LOOP_INTERVAL ; +bool mode_selected = false; +unsigned int mode; +int opt, i; + +while ((opt =getopt (argc, argv,"hi:I:m:" )) !=-1 ) { +switch (opt) { +case 'i' : + iterations =strtol (optarg,NULL ,10 ); +break ; +case 'I' : + interval =strtol (optarg,NULL ,10 ); +break ; +case 'm' : +if (!mode_selected) { +for (i =0 ; i <NUM_VM_MODES ; ++i) + vm_guest_modes[i].enabled =0 ; + mode_selected = true; + } + mode =strtoul (optarg,NULL ,10 ); +TEST_ASSERT (mode <NUM_VM_MODES , +"Guest mode ID %d too big" , mode); + vm_guest_modes[mode].enabled =1 ; +break ; +case 'h' : +default : +help (argv[0 ]); +break ; + } + } + +TEST_ASSERT (iterations >2 ,"Iterations must be greater than two" ); +TEST_ASSERT (interval >0 ,"Interval must be greater than zero" ); + +DEBUG ("Test iterations: %" PRIu64", interval: %" PRIu64" (ms)\n" , + iterations, interval); + +srandom (time (0 )); + +for (i =0 ; i <NUM_VM_MODES ; ++i) { +if (!vm_guest_modes[i].enabled) +continue ; +TEST_ASSERT (vm_guest_modes[i].supported, +"Guest mode ID %d (%s) not supported." , + vm_guest_modes[i].mode, +vm_guest_mode_string (vm_guest_modes[i].mode)); +run_test (vm_guest_modes[i].mode, iterations, interval); + }return 0 ; }
Signature · 1 line
2.17.1
PATCH 12/13 kvm: selftests: stop lying to aarch64 tests about PA-bits
In reply to Andrew JonesLet's add the 40 PA-bit versions of the VM modes, that AArch64 should have been using, so we can extend the dirty log test without breaking things.
--- tools/testing/selftests/kvm/dirty_log_test.c | 13 ++++++++++--- tools/testing/selftests/kvm/include/kvm_util.h | 2 ++ .../selftests/kvm/lib/aarch64/processor.c | 8 ++++++++ tools/testing/selftests/kvm/lib/kvm_util.c | 16 ++++++++++++++++ 4 files changed, 36 insertions(+), 3 deletions(-)
tools/testing/selftests/kvm/dirty_log_test.c +10 -3
@@ -230,9 +230,11 @@ static void run_test(enum vm_guest_mode mode, unsigned long iterations,switch (mode) {case VM_MODE_P52V48_4K : +case VM_MODE_P40V48_4K : guest_page_shift =12 ;break ;case VM_MODE_P52V48_64K : +case VM_MODE_P40V48_64K : guest_page_shift =16 ;break ;default :
@@ -317,11 +319,16 @@ static struct vm_guest_modes {bool supported;bool enabled; } vm_guest_modes[NUM_VM_MODES] = { +#if defined(__x86_64__) { VM_MODE_P52V48_4K, 1, 1, }, -#ifdef __aarch64__ - { VM_MODE_P52V48_64K, 1, 1, }, -#else { VM_MODE_P52V48_64K, 0, 0, }, + { VM_MODE_P40V48_4K, 0, 0, }, + { VM_MODE_P40V48_64K, 0, 0, }, +#elif defined(__aarch64__) + { VM_MODE_P52V48_4K, 0, 0, }, + { VM_MODE_P52V48_64K, 0, 0, }, + { VM_MODE_P40V48_4K, 1, 1, }, + { VM_MODE_P40V48_64K, 1, 1, }, #endif };
tools/testing/selftests/kvm/include/kvm_util.h +2 -0
@@ -36,6 +36,8 @@ typedef uint64_t vm_vaddr_t; /* Virtual Machine (Guest) virtual address */enum vm_guest_mode {VM_MODE_P52V48_4K ,VM_MODE_P52V48_64K , +VM_MODE_P40V48_4K , +VM_MODE_P40V48_64K ,NUM_VM_MODES , };
tools/testing/selftests/kvm/lib/aarch64/processor.c +8 -0
@@ -275,6 +275,14 @@ void vcpu_setup(struct kvm_vm *vm, int vcpuid, int pgd_memslot, int gdt_memslot) tcr_el1 |=1ul <<14 ;/* TG0 = 64KB */ tcr_el1 |=6ul <<32 ;/* IPS = 52 bits */ break ; +case VM_MODE_P40V48_4K : + tcr_el1 |=0ul <<14 ;/* TG0 = 4KB */ + tcr_el1 |=2ul <<32 ;/* IPS = 40 bits */ +break ; +case VM_MODE_P40V48_64K : + tcr_el1 |=1ul <<14 ;/* TG0 = 64KB */ + tcr_el1 |=2ul <<32 ;/* IPS = 40 bits */ +break ;default :TEST_ASSERT (false,"Unknown guest mode, mode: 0x%x" , vm->mode); }
tools/testing/selftests/kvm/lib/kvm_util.c +16 -0
@@ -76,6 +76,8 @@ static void vm_open(struct kvm_vm *vm, int perm)const char *const vm_guest_mode_string[] = {"PA-bits:52, VA-bits:48, 4K pages" ,"PA-bits:52, VA-bits:48, 64K pages" , +"PA-bits:40, VA-bits:48, 4K pages" , +"PA-bits:40, VA-bits:48, 64K pages" , };/*
@@ -124,6 +126,20 @@ struct kvm_vm *vm_create(enum vm_guest_mode mode, uint64_t phy_pages, int perm) vm->page_size =0x10000 ; vm->page_shift =16 ;break ; +case VM_MODE_P40V48_4K : + vm->pgtable_levels =4 ; + vm->pa_bits =40 ; + vm->va_bits =48 ; + vm->page_size =0x1000 ; + vm->page_shift =12 ; +break ; +case VM_MODE_P40V48_64K : + vm->pgtable_levels =3 ; + vm->pa_bits =40 ; + vm->va_bits =48 ; + vm->page_size =0x10000 ; + vm->page_shift =16 ; +break ;default :TEST_ASSERT (false,"Unknown guest mode, mode: 0x%x" , mode); }
Signature · 1 line
2.17.1
PATCH 13/13 kvm: selftests: support high GPAs in dirty_log_test
In reply to Andrew Jones--- tools/testing/selftests/kvm/dirty_log_test.c | 65 +++++++++++++++----- 1 file changed, 50 insertions(+), 15 deletions(-)
tools/testing/selftests/kvm/dirty_log_test.c +50 -15
@@ -26,11 +26,8 @@/* The memory slot index to track dirty pages */ #define TEST_MEM_SLOT_INDEX 1 -/* -* GPA offset of the testing memory slot. Must be bigger than the -* default vm mem slot, which is DEFAULT_GUEST_PHY_PAGES. -*/ -#define TEST_MEM_OFFSET (1ul << 30)/* 1G */ +/* Default guest test memory offset, 1G */ +#define DEFAULT_GUEST_TEST_MEM 0x40000000/* How many pages to dirty for each guest loop */ #define TEST_PAGES_PER_LOOP 1024
@@ -53,6 +50,12 @@ static uint64_t guest_num_pages;static uint64_t random_array[TEST_PAGES_PER_LOOP ];static uint64_t iteration; +/* +* GPA offset of the testing memory slot. Must be bigger than +* DEFAULT_GUEST_PHY_PAGES. +*/ +static uint64_t guest_test_mem =DEFAULT_GUEST_TEST_MEM ; +/* * Continuously write to the first 8 bytes of a random pages within * the testing memory region.
@@ -63,7 +66,7 @@ static void guest_code(void)while (true) {for (i =0 ; i <TEST_PAGES_PER_LOOP ; i++) { -uint64_t addr =TEST_MEM_OFFSET ; +uint64_t addr = guest_test_mem; addr += (READ_ONCE (random_array[i]) % guest_num_pages) * guest_page_size; addr &= ~(host_page_size -1 );
@@ -221,20 +224,29 @@ static struct kvm_vm *create_vm(enum vm_guest_mode mode, uint32_t vcpuid, }static void run_test (enum vm_guest_mode mode,unsigned long iterations, -unsigned long interval) +unsigned long interval,bool top_offset) { -unsigned int guest_page_shift; +unsigned int guest_pa_bits, guest_page_shift;pthread_t vcpu_thread;struct kvm_vm *vm; +uint64_t max_gfn;unsigned long *bmap;switch (mode) {case VM_MODE_P52V48_4K : -case VM_MODE_P40V48_4K : + guest_pa_bits =52 ; guest_page_shift =12 ;break ;case VM_MODE_P52V48_64K : + guest_pa_bits =52 ; + guest_page_shift =16 ; +break ; +case VM_MODE_P40V48_4K : + guest_pa_bits =40 ; + guest_page_shift =12 ; +break ;case VM_MODE_P40V48_64K : + guest_pa_bits =40 ; guest_page_shift =16 ;break ;default :
@@ -243,6 +255,7 @@ static void run_test(enum vm_guest_mode mode, unsigned long iterations,DEBUG ("Testing guest mode: %s\n" ,vm_guest_mode_string (mode)); + max_gfn = (1ul << (guest_pa_bits - guest_page_shift)) -1 ; guest_page_size = (1ul << guest_page_shift);/* 1G of guest page sized pages */ guest_num_pages = (1ul << (30 - guest_page_shift));
@@ -250,6 +263,13 @@ static void run_test(enum vm_guest_mode mode, unsigned long iterations, host_num_pages = (guest_num_pages * guest_page_size) / host_page_size + !!((guest_num_pages * guest_page_size) % host_page_size); +if (top_offset) { + guest_test_mem = (max_gfn - guest_num_pages) * guest_page_size; + guest_test_mem &= ~(host_page_size -1 ); + } + +DEBUG ("guest test mem offset: 0x%lx\n" , guest_test_mem); + bmap =bitmap_alloc (host_num_pages); host_bmap_track =bitmap_alloc (host_num_pages);
@@ -257,17 +277,17 @@ static void run_test(enum vm_guest_mode mode, unsigned long iterations,/* Add an extra memory slot for testing dirty logging */ vm_userspace_mem_region_add (vm,VM_MEM_SRC_ANONYMOUS , -TEST_MEM_OFFSET , + guest_test_mem,TEST_MEM_SLOT_INDEX , guest_num_pages,KVM_MEM_LOG_DIRTY_PAGES );/* Do 1:1 mapping for the dirty track memory slot */ -virt_map (vm,TEST_MEM_OFFSET ,TEST_MEM_OFFSET , +virt_map (vm, guest_test_mem, guest_test_mem, guest_num_pages * guest_page_size,0 );/* Cache the HVA pointer of the region */ - host_test_mem =addr_gpa2hva (vm, (vm_paddr_t )TEST_MEM_OFFSET ); + host_test_mem =addr_gpa2hva (vm, (vm_paddr_t )guest_test_mem);#ifdef __x86_64__vcpu_set_cpuid (vm,VCPU_ID ,kvm_get_supported_cpuid ());
@@ -279,6 +299,7 @@ static void run_test(enum vm_guest_mode mode, unsigned long iterations,/* Export the shared variables to the guest */ sync_global_to_guest (vm, host_page_size);sync_global_to_guest (vm, guest_page_size); +sync_global_to_guest (vm, guest_test_mem);sync_global_to_guest (vm, guest_num_pages);/* Start the iterations */
@@ -337,12 +358,17 @@ static void help(char *name)int i;puts ("" ); -printf ("usage: %s [-h] [-i iterations] [-I interval] [-m mode]\n" , name); +printf ("usage: %s [-h] [-i iterations] [-I interval] " +"[-o offset] [-t] [-m mode]\n" , name);puts ("" );printf (" -i: specify iteration counts (default: %" PRIu64")\n" ,TEST_HOST_LOOP_N );printf (" -I: specify interval in ms (default: %" PRIu64" ms)\n" ,TEST_HOST_LOOP_INTERVAL ); +printf (" -o: guest test memory offset (default: 0x%lx)\n" , +DEFAULT_GUEST_TEST_MEM ); +printf (" -t: map guest test memory at the top of the allowed " +"physical address range\n" );printf (" -m: specify the guest mode ID to test " "(default: test all supported modes)\n" " This option may be used multiple times.\n"
@@ -362,10 +388,11 @@ int main(int argc, char *argv[])unsigned long iterations =TEST_HOST_LOOP_N ;unsigned long interval =TEST_HOST_LOOP_INTERVAL ;bool mode_selected = false; +bool top_offset = false;unsigned int mode;int opt, i; -while ((opt =getopt (argc, argv,"hi:I:m:" )) !=-1 ) { +while ((opt =getopt (argc, argv,"hi:I:o:tm:" )) !=-1 ) {switch (opt) {case 'i' : iterations =strtol (optarg,NULL ,10 );
@@ -373,6 +400,12 @@ int main(int argc, char *argv[])case 'I' : interval =strtol (optarg,NULL ,10 );break ; +case 'o' : + guest_test_mem =strtoull (optarg,NULL ,0 ); +break ; +case 't' : + top_offset = true; +break ;case 'm' :if (!mode_selected) {for (i =0 ; i <NUM_VM_MODES ; ++i)
@@ -393,6 +426,8 @@ int main(int argc, char *argv[])TEST_ASSERT (iterations >2 ,"Iterations must be greater than two" );TEST_ASSERT (interval >0 ,"Interval must be greater than zero" ); +TEST_ASSERT (!top_offset || guest_test_mem ==DEFAULT_GUEST_TEST_MEM , +"Cannot use both -o [offset] and -t at the same time" );DEBUG ("Test iterations: %" PRIu64", interval: %" PRIu64" (ms)\n" , iterations, interval);
@@ -406,7 +441,7 @@ int main(int argc, char *argv[])"Guest mode ID %d (%s) not supported." , vm_guest_modes[i].mode, vm_guest_mode_string(vm_guest_modes[i].mode)); -run_test (vm_guest_modes[i].mode, iterations, interval); +run_test (vm_guest_modes[i].mode, iterations, interval, top_offset); }return 0 ;
Signature · 1 line
2.17.1
Quoted 18 lines · Andrew Jones
This series provides KVM selftests that test dirty log tracking on AArch64 for both 4K and 64K guest page sizes. Additionally the framework provides an easy way to test dirty log tracking with the recently posted dynamic IPA and 52bit IPA series[1].
The series breaks down into parts as follows:
01-02: generalize guest code to host userspace exit support by
introducing "ucalls" - hypercalls to userspace
03-05: prepare common code for a new architecture 06-07: add virtual memory setup support for AArch64
08: add vcpu setup support for AArch64
09: port the dirty log test to AArch64
10-11: add 64K guest page size support for the dirty log test 12-13: prepare the dirty log test to also test > 40-bit guest
physical address setups by allowing the test memory region to be placed at the top of physical memory
Quoted 2 lines
Hi Suzuki,
Here's an [untested] add-on patch that should provide the means to test dirty logging with a 52-bit guest physical address space. Hopefully it'll be as easy as compiling and then running with
$ ./dirty_log_test -t
Thanks,
drew
Quoted 1 line
From
09ea1d724551a95ef9962d357049ef21ea8c77e8Mon Sep 17 00:00:00 2001
From: Andrew Jones <[email protected]>
Date: Wed, 19 Sep 2018 14:23:30 +0200
Subject: [PATCH] kvm: selftests: aarch64: dirty_log_test: test with 52 PA-bits
--- tools/testing/selftests/kvm/dirty_log_test.c | 24 ++++++++++++++++--- .../testing/selftests/kvm/include/kvm_util.h | 2 ++ tools/testing/selftests/kvm/lib/kvm_util.c | 17 +++++++++---- .../selftests/kvm/lib/kvm_util_internal.h | 1 + 4 files changed, 36 insertions(+), 8 deletions(-)
tools/testing/selftests/kvm/dirty_log_test.c +21 -3
@@ -82,6 +82,7 @@ static void guest_code(void)static bool host_quit;/* Points to the test VM memory region on which we track dirty logs */ +static uint8_t host_ipa_limit;static void *host_test_mem;static uint64_t host_num_pages;
@@ -209,12 +210,14 @@ static void vm_dirty_log_verify(unsigned long *bmap) }static struct kvm_vm *create_vm (enum vm_guest_mode mode,uint32_t vcpuid, -uint64_t extra_mem_pages,void *guest_code) +uint64_t extra_mem_pages,void *guest_code, +unsigned long type) {struct kvm_vm *vm;uint64_t extra_pg_pages = extra_mem_pages /512 *2 ; - vm =vm_create (mode,DEFAULT_GUEST_PHY_PAGES + extra_pg_pages,O_RDWR ); + vm =_vm_create (mode,DEFAULT_GUEST_PHY_PAGES + extra_pg_pages, +O_RDWR , type);kvm_vm_elf_load (vm, program_invocation_name,0 ,0 );#ifdef __x86_64__vm_create_irqchip (vm);
@@ -231,15 +234,22 @@ static void run_test(enum vm_guest_mode mode, unsigned long iterations,struct kvm_vm *vm;uint64_t max_gfn;unsigned long *bmap; +unsigned long type =0 ;switch (mode) {case VM_MODE_P52V48_4K : guest_pa_bits =52 ; guest_page_shift =12 ; +#ifdef __aarch64__ + type =KVM_VM_TYPE_ARM_IPA_SIZE (52 ); +#endif break ;case VM_MODE_P52V48_64K : guest_pa_bits =52 ; guest_page_shift =16 ; +#ifdef __aarch64__ + type =KVM_VM_TYPE_ARM_IPA_SIZE (52 ); +#endif break ;case VM_MODE_P40V48_4K : guest_pa_bits =40 ;
@@ -273,7 +283,7 @@ static void run_test(enum vm_guest_mode mode, unsigned long iterations, bmap =bitmap_alloc (host_num_pages); host_bmap_track =bitmap_alloc (host_num_pages); - vm =create_vm (mode,VCPU_ID , guest_num_pages, guest_code); + vm =create_vm (mode,VCPU_ID , guest_num_pages, guest_code, type);/* Add an extra memory slot for testing dirty logging */ vm_userspace_mem_region_add(vm, VM_MEM_SRC_ANONYMOUS,
@@ -392,6 +402,14 @@ int main(int argc, char *argv[])unsigned int mode;int opt, i; +#ifdef __aarch64__ + host_ipa_limit =kvm_check_cap (KVM_CAP_ARM_VM_IPA_SIZE ); +if (host_ipa_limit ==52 ) { + vm_guest_modes[VM_MODE_P52V48_4K ].supported =1 ; + vm_guest_modes[VM_MODE_P52V48_64K ].supported =1 ; + } +#endif +while ((opt =getopt (argc, argv,"hi:I:o:tm:" )) !=-1 ) {switch (opt) {case 'i' :
tools/testing/selftests/kvm/include/kvm_util.h +2 -0
@@ -53,6 +53,8 @@ enum vm_mem_backing_src_type {int kvm_check_cap (long cap);struct kvm_vm *vm_create (enum vm_guest_mode mode,uint64_t phy_pages,int perm); +struct kvm_vm *_vm_create (enum vm_guest_mode mode,uint64_t phy_pages, +int perm,unsigned long type);void kvm_vm_free (struct kvm_vm *vmp);void kvm_vm_restart (struct kvm_vm *vmp,int perm);void kvm_vm_release (struct kvm_vm *vmp);
tools/testing/selftests/kvm/lib/kvm_util.c +12 -5
@@ -62,13 +62,13 @@ int kvm_check_cap(long cap)return ret; } -static void vm_open (struct kvm_vm *vm,int perm) +static void vm_open (struct kvm_vm *vm,int perm,unsigned long type) { vm->kvm_fd =open (KVM_DEV_PATH , perm);if (vm->kvm_fd <0 )exit (KSFT_SKIP ); - vm->fd =ioctl (vm->kvm_fd,KVM_CREATE_VM ,NULL ); + vm->fd =ioctl (vm->kvm_fd,KVM_CREATE_VM , type);TEST_ASSERT (vm->fd >=0 ,"KVM_CREATE_VM ioctl failed, " "rc: %i errno: %i" , vm->fd, errno); }
@@ -101,7 +101,8 @@ _Static_assert(sizeof(vm_guest_mode_string)/sizeof(char *) == NUM_VM_MODES,* descriptor to control the created VM is created with the permissions * given by perm (e.g. O_RDWR). */ -struct kvm_vm *vm_create (enum vm_guest_mode mode,uint64_t phy_pages,int perm) +struct kvm_vm *_vm_create (enum vm_guest_mode mode,uint64_t phy_pages, +int perm,unsigned long type) {struct kvm_vm *vm;int kvm_fd;
@@ -110,7 +111,8 @@ struct kvm_vm *vm_create(enum vm_guest_mode mode, uint64_t phy_pages, int perm)TEST_ASSERT (vm !=NULL ,"Insufficent Memory" ); vm->mode = mode; -vm_open (vm, perm); + vm->type = type; +vm_open (vm, perm, type);/* Setup mode specific traits. */ switch (vm->mode) {
@@ -166,6 +168,11 @@ struct kvm_vm *vm_create(enum vm_guest_mode mode, uint64_t phy_pages, int perm)return vm; } +struct kvm_vm *vm_create (enum vm_guest_mode mode,uint64_t phy_pages,int perm) +{ +return _vm_create (mode, phy_pages, perm,0 ); +} +/* * VM Restart *
@@ -183,7 +190,7 @@ void kvm_vm_restart(struct kvm_vm *vmp, int perm) {struct userspace_mem_region *region; -vm_open (vmp, perm); +vm_open (vmp, perm, vmp->type);if (vmp->has_irqchip)vm_create_irqchip (vmp);
tools/testing/selftests/kvm/lib/kvm_util_internal.h +1 -0
@@ -44,6 +44,7 @@ struct vcpu {struct kvm_vm {int mode; +unsigned long type;int kvm_fd;int fd;unsigned int pgtable_levels;
Signature · 1 line
2.17.1
Suzuki K Poulose
reviewerHi Drew,
Quoted 22 lines · Andrew Jones
Quoted 18 lines · Andrew Jones
This series provides KVM selftests that test dirty log tracking on AArch64 for both 4K and 64K guest page sizes. Additionally the framework provides an easy way to test dirty log tracking with the recently posted dynamic IPA and 52bit IPA series[1].
The series breaks down into parts as follows:
01-02: generalize guest code to host userspace exit support by
introducing "ucalls" - hypercalls to userspace
03-05: prepare common code for a new architecture 06-07: add virtual memory setup support for AArch64
08: add vcpu setup support for AArch64
09: port the dirty log test to AArch64
10-11: add 64K guest page size support for the dirty log test 12-13: prepare the dirty log test to also test > 40-bit guest
physical address setups by allowing the test memory region to be placed at the top of physical memoryQuoted 2 lines
Quoted 7 lines
Hi Suzuki,
Here's an [untested] add-on patch that should provide the means to test dirty logging with a 52-bit guest physical address space. Hopefully it'll be as easy as compiling and then running with
$ ./dirty_log_test -t
I will give this series a spin with 52 IPA and let you know.
Thanks
Suzuki
Quoted 180 lines
Thanks,
drewFrom
09ea1d724551a95ef9962d357049ef21ea8c77e8Mon Sep 17 00:00:00 2001
From: Andrew Jones <[email protected]>
Date: Wed, 19 Sep 2018 14:23:30 +0200
Subject: [PATCH] kvm: selftests: aarch64:dirty_log_test: test with 52 PA-bitsSigned-off-by Andrew Jones <[email protected]>--- tools/testing/selftests/kvm/dirty_log_test.c | 24 ++++++++++++++++--- .../testing/selftests/kvm/include/kvm_util.h | 2 ++ tools/testing/selftests/kvm/lib/kvm_util.c | 17 +++++++++---- .../selftests/kvm/lib/kvm_util_internal.h | 1 + 4 files changed, 36 insertions(+), 8 deletions(-)tools/testing/selftests/kvm/dirty_log_test.c +21
-3@@ -82,6 +82,7 @@ static void guest_code(void)static bool host_quit;/* Points to the test VM memory region on which we track dirty logs */ +static uint8_t host_ipa_limit;static void *host_test_mem;static uint64_t host_num_pages;@@ -209,12 +210,14 @@ static void vm_dirty_log_verify(unsigned long *bmap) }static struct kvm_vm *create_vm (enum vm_guest_mode mode,uint32_t vcpuid, -uint64_t extra_mem_pages,void *guest_code) +uint64_t extra_mem_pages,void *guest_code, +unsigned long type) {struct kvm_vm *vm;uint64_t extra_pg_pages = extra_mem_pages /512 *2 ; - vm =vm_create (mode,DEFAULT_GUEST_PHY_PAGES + extra_pg_pages,O_RDWR ); + vm =_vm_create (mode,DEFAULT_GUEST_PHY_PAGES + extra_pg_pages, +O_RDWR , type);kvm_vm_elf_load (vm, program_invocation_name,0 ,0 );#ifdef __x86_64__vm_create_irqchip (vm);@@ -231,15 +234,22 @@ static void run_test(enum vm_guest_mode mode, unsigned long iterations,struct kvm_vm *vm;uint64_t max_gfn;unsigned long *bmap; +unsigned long type =0 ;switch (mode) {case VM_MODE_P52V48_4K : guest_pa_bits =52 ; guest_page_shift =12 ; +#ifdef __aarch64__ + type =KVM_VM_TYPE_ARM_IPA_SIZE (52 ); +#endif break ;case VM_MODE_P52V48_64K : guest_pa_bits =52 ; guest_page_shift =16 ; +#ifdef __aarch64__ + type =KVM_VM_TYPE_ARM_IPA_SIZE (52 ); +#endif break ;case VM_MODE_P40V48_4K : guest_pa_bits =40 ;@@ -273,7 +283,7 @@ static void run_test(enum vm_guest_mode mode, unsigned long iterations, bmap =bitmap_alloc (host_num_pages); host_bmap_track =bitmap_alloc (host_num_pages); - vm =create_vm (mode,VCPU_ID , guest_num_pages, guest_code); + vm =create_vm (mode,VCPU_ID , guest_num_pages, guest_code, type);/* Add an extra memory slot for testing dirty logging */ vm_userspace_mem_region_add(vm, VM_MEM_SRC_ANONYMOUS,@@ -392,6 +402,14 @@ int main(int argc, char *argv[])unsigned int mode;int opt, i; +#ifdef __aarch64__ + host_ipa_limit =kvm_check_cap (KVM_CAP_ARM_VM_IPA_SIZE ); +if (host_ipa_limit ==52 ) { + vm_guest_modes[VM_MODE_P52V48_4K ].supported =1 ; + vm_guest_modes[VM_MODE_P52V48_64K ].supported =1 ; + } +#endif +while ((opt =getopt (argc, argv,"hi:I:o:tm:" )) !=-1 ) {switch (opt) {case 'i' :tools/testing/selftests/kvm/include/kvm_util.h +2
-0@@ -53,6 +53,8 @@ enum vm_mem_backing_src_type {int kvm_check_cap (long cap);struct kvm_vm *vm_create (enum vm_guest_mode mode,uint64_t phy_pages,int perm); +struct kvm_vm *_vm_create (enum vm_guest_mode mode,uint64_t phy_pages, +int perm,unsigned long type);void kvm_vm_free (struct kvm_vm *vmp);void kvm_vm_restart (struct kvm_vm *vmp,int perm);void kvm_vm_release (struct kvm_vm *vmp);tools/testing/selftests/kvm/lib/kvm_util.c +12
-5@@ -62,13 +62,13 @@ int kvm_check_cap(long cap)return ret; } -static void vm_open (struct kvm_vm *vm,int perm) +static void vm_open (struct kvm_vm *vm,int perm,unsigned long type) { vm->kvm_fd =open (KVM_DEV_PATH , perm);if (vm->kvm_fd <0 )exit (KSFT_SKIP ); - vm->fd =ioctl (vm->kvm_fd,KVM_CREATE_VM ,NULL ); + vm->fd =ioctl (vm->kvm_fd,KVM_CREATE_VM , type);TEST_ASSERT (vm->fd >=0 ,"KVM_CREATE_VM ioctl failed, " "rc: %i errno: %i" , vm->fd, errno); }@@ -101,7 +101,8 @@ _Static_assert(sizeof(vm_guest_mode_string)/sizeof(char *) == NUM_VM_MODES,* descriptor to control the created VM is created with the permissions * given by perm (e.g. O_RDWR). */ -struct kvm_vm *vm_create (enum vm_guest_mode mode,uint64_t phy_pages,int perm) +struct kvm_vm *_vm_create (enum vm_guest_mode mode,uint64_t phy_pages, +int perm,unsigned long type) {struct kvm_vm *vm;int kvm_fd;@@ -110,7 +111,8 @@ struct kvm_vm *vm_create(enum vm_guest_mode mode, uint64_t phy_pages, int perm)TEST_ASSERT (vm !=NULL ,"Insufficent Memory" ); vm->mode = mode; -vm_open (vm, perm); + vm->type = type; +vm_open (vm, perm, type);/* Setup mode specific traits. */ switch (vm->mode) {@@ -166,6 +168,11 @@ struct kvm_vm *vm_create(enum vm_guest_mode mode, uint64_t phy_pages, int perm)return vm; } +struct kvm_vm *vm_create (enum vm_guest_mode mode,uint64_t phy_pages,int perm) +{ +return _vm_create (mode, phy_pages, perm,0 ); +} +/* * VM Restart * @@ -183,7 +190,7 @@ void kvm_vm_restart(struct kvm_vm *vmp, int perm) {struct userspace_mem_region *region; -vm_open (vmp, perm); +vm_open (vmp, perm, vmp->type);if (vmp->has_irqchip)vm_create_irqchip (vmp);tools/testing/selftests/kvm/lib/kvm_util_internal.h +1
-0@@ -44,6 +44,7 @@ struct vcpu {struct kvm_vm {int mode; +unsigned long type;int kvm_fd;int fd;unsigned int pgtable_levels;
Hi Drew,
Quoted 4 lines · Andrew Jones
This series provides KVM selftests that test dirty log tracking on AArch64 for both 4K and 64K guest page sizes. Additionally the framework provides an easy way to test dirty log tracking with the recently posted dynamic IPA and 52bit IPA series[1].
I was trying to parse the commit text of patch 2, and I realized that I don't understand the 'hypercall to userspace' thing at all, which probably means I have no idea how the selftests work overall.
I then spent a while reading various bits of documentation in the kernel tree, LWN, etc., only to realize that I don't understand how this test framework actually works.
Are the selftests modules, userspace programs, or code that is compiled with the kernel, and (somehow?) run from userspace. I thought the latter, partially based on your explanation at ELC, but then I don't understand how the "compile and run" make target works.
Can you help me paint the overall picture, or point me to the piece of documentation/presentation that explains the high-level picture, which I must have obviously missed somehow?
Thanks!
Christoffer
Quoted 73 lines
The series breaks down into parts as follows:
01-02: generalize guest code to host userspace exit support by
introducing "ucalls" - hypercalls to userspace
03-05: prepare common code for a new architecture 06-07: add virtual memory setup support for AArch64
08: add vcpu setup support for AArch64
09: port the dirty log test to AArch64
10-11: add 64K guest page size support for the dirty log test 12-13: prepare the dirty log test to also test > 40-bit guest
physical address setups by allowing the test memory region to be placed at the top of physical memory[1] https://www.spinics.net/lists/arm-kernel/msg676819.html
Thanks,
drewAndrew Jones (13):
kvm: selftests:vcpu_setup: set cr4.osfxsr
kvm: selftests: introduce ucall
kvm: selftests: move arch-specific files to arch-specific locations
kvm: selftests: add cscope make target
kvm: selftests: tidy upkvm_util
kvm: selftests: addvm_phy_pages_alloc
kvm: selftests: add virt mem support for aarch64
kvm: selftests: add vcpu support for aarch64
kvm: selftests: portdirty_log_testto aarch64
kvm: selftests: introduce new VM mode for 64K pages
kvm: selftests:dirty_log_test: also test 64K pages on aarch64
kvm: selftests: stop lying to aarch64 tests about PA-bits
kvm: selftests: support high GPAs indirty_log_testtools/testing/selftests/kvm/.gitignore | 11 +- tools/testing/selftests/kvm/Makefile | 36 +- tools/testing/selftests/kvm/dirty_log_test.c | 374 +++++++++---- .../selftests/kvm/include/aarch64/processor.h | 55 ++ .../testing/selftests/kvm/include/kvm_util.h | 166 +++--- .../testing/selftests/kvm/include/sparsebit.h | 6 +- .../testing/selftests/kvm/include/test_util.h | 6 +- .../kvm/include/{x86.h => x86_64/processor.h} | 24 +- .../selftests/kvm/include/{ => x86_64}/vmx.h | 6 +- .../selftests/kvm/lib/aarch64/processor.c | 311 +++++++++++ tools/testing/selftests/kvm/lib/assert.c | 2 +- tools/testing/selftests/kvm/lib/kvm_util.c | 499 +++++++----------- .../selftests/kvm/lib/kvm_util_internal.h | 33 +- tools/testing/selftests/kvm/lib/ucall.c | 144 +++++ .../kvm/lib/{x86.c => x86_64/processor.c} | 197 ++++++- .../selftests/kvm/lib/{ => x86_64}/vmx.c | 4 +- .../kvm/{ => x86_64}/cr4_cpuid_sync_test.c | 14 +- .../kvm/{ => x86_64}/set_sregs_test.c | 2 +- .../selftests/kvm/{ => x86_64}/state_test.c | 25 +- .../kvm/{ => x86_64}/sync_regs_test.c | 2 +- .../kvm/{ => x86_64}/vmx_tsc_adjust_test.c | 23 +- 21 files changed, 1329 insertions(+), 611 deletions(-) create mode 100644 tools/testing/selftests/kvm/include/aarch64/processor.h rename tools/testing/selftests/kvm/include/{x86.h => x86_64/processor.h} (98%) rename tools/testing/selftests/kvm/include/{ => x86_64}/vmx.h (99%) create mode 100644 tools/testing/selftests/kvm/lib/aarch64/processor.c create mode 100644 tools/testing/selftests/kvm/lib/ucall.c rename tools/testing/selftests/kvm/lib/{x86.c => x86_64/processor.c} (85%) rename tools/testing/selftests/kvm/lib/{ => x86_64}/vmx.c (99%) rename tools/testing/selftests/kvm/{ => x86_64}/cr4_cpuid_sync_test.c (91%) rename tools/testing/selftests/kvm/{ => x86_64}/set_sregs_test.c (98%) rename tools/testing/selftests/kvm/{ => x86_64}/state_test.c (90%) rename tools/testing/selftests/kvm/{ => x86_64}/sync_regs_test.c (99%) rename tools/testing/selftests/kvm/{ => x86_64}/vmx_tsc_adjust_test.c (91%)Signature · 1 line
2.17.1
Hi Christoffer,
Thanks for your interest in these tests. There isn't any documentation that I know of, but it's a good idea to have some. I'll write something up soon. I'll also try to answer your questions now.
Quoted 11 lines · Christoffer Dall
Hi Drew,
Quoted 4 lines · Andrew Jones
This series provides KVM selftests that test dirty log tracking on AArch64 for both 4K and 64K guest page sizes. Additionally the framework provides an easy way to test dirty log tracking with the recently posted dynamic IPA and 52bit IPA series[1].
I was trying to parse the commit text of patch 2, and I realized that I don't understand the 'hypercall to userspace' thing at all, which probably means I have no idea how the selftests work overall.
There are three parts to a kvm selftest: 1) the test code which runs in host userspace and _is_ the kvm userspace used with kvm, 2) the vcpu thread code which executes KVM_RUN for the guest code and possibly also some host userspace test code, and 3) the guest code, which is naturally run in the vcpu thread, but in guest mode.
The need for a "ucall" arises for 2's "possibly also some host userspace test code". In that case the guest code needs to invoke an exit from guest mode, not just to kvm, but all the way to kvm userspace. For AArch64, as you know, this can be done with an MMIO access. The reason patch 2 generalizes the concept is because for x86 this can and is done with a PIO access.
Quoted 9 lines
I then spent a while reading various bits of documentation in the kernel tree, LWN, etc., only to realize that I don't understand how this test framework actually works.
Are the selftests modules, userspace programs, or code that is compiled with the kernel, and (somehow?) run from userspace. I thought the latter, partially based on your explanation at ELC, but then I don't understand how the "compile and run" make target works.
The tests are standalone userspace programs which are compiled separately, but have dependencies on kernel headers. As stated above, for kvm, each selftest is a kvm userspace (including its vcpu thread code) and guest code combined. While there's a lot of complexity in the framework, particularly for memory management, and a bit for vcpu setup, most of that can be shared among tests using the kvm_util.h and test_util.h APIs, allowing a given test to only have a relatively simple main(), vcpu thread "vcpu_worker()" function, and "guest_code()" function. Guest mode code can easily share code with the kvm userspace test code (assuming the guest page tables are set up in the default way) and even data can be shared as long the accesses are done with the appropriate mappings (gva vs. hva). There's a small API to help with that as well.
Quoted 4 lines
Can you help me paint the overall picture, or point me to the piece of documentation/presentation that explains the high-level picture, which I must have obviously missed somehow?
We definitely need the documentation and, in hindsight, it looks like it would have been a good BoF topic last week too.
I think this framework has a lot of potential for KVM API testing and even for quick & dirty guest code instruction sequence tests (although instruction sequences would also fit kvm-unit-tests). I hope I can help get you and anyone else interested started.
Thanks,
drew
Quoted 79 lines
Thanks!
Christoffer
Quoted 73 lines
The series breaks down into parts as follows:
01-02: generalize guest code to host userspace exit support by
introducing "ucalls" - hypercalls to userspace
03-05: prepare common code for a new architecture 06-07: add virtual memory setup support for AArch64
08: add vcpu setup support for AArch64
09: port the dirty log test to AArch64
10-11: add 64K guest page size support for the dirty log test 12-13: prepare the dirty log test to also test > 40-bit guest
physical address setups by allowing the test memory region to be placed at the top of physical memory[1] https://www.spinics.net/lists/arm-kernel/msg676819.html
Thanks,
drewAndrew Jones (13):
kvm: selftests:vcpu_setup: set cr4.osfxsr
kvm: selftests: introduce ucall
kvm: selftests: move arch-specific files to arch-specific locations
kvm: selftests: add cscope make target
kvm: selftests: tidy upkvm_util
kvm: selftests: addvm_phy_pages_alloc
kvm: selftests: add virt mem support for aarch64
kvm: selftests: add vcpu support for aarch64
kvm: selftests: portdirty_log_testto aarch64
kvm: selftests: introduce new VM mode for 64K pages
kvm: selftests:dirty_log_test: also test 64K pages on aarch64
kvm: selftests: stop lying to aarch64 tests about PA-bits
kvm: selftests: support high GPAs indirty_log_testtools/testing/selftests/kvm/.gitignore | 11 +- tools/testing/selftests/kvm/Makefile | 36 +- tools/testing/selftests/kvm/dirty_log_test.c | 374 +++++++++---- .../selftests/kvm/include/aarch64/processor.h | 55 ++ .../testing/selftests/kvm/include/kvm_util.h | 166 +++--- .../testing/selftests/kvm/include/sparsebit.h | 6 +- .../testing/selftests/kvm/include/test_util.h | 6 +- .../kvm/include/{x86.h => x86_64/processor.h} | 24 +- .../selftests/kvm/include/{ => x86_64}/vmx.h | 6 +- .../selftests/kvm/lib/aarch64/processor.c | 311 +++++++++++ tools/testing/selftests/kvm/lib/assert.c | 2 +- tools/testing/selftests/kvm/lib/kvm_util.c | 499 +++++++----------- .../selftests/kvm/lib/kvm_util_internal.h | 33 +- tools/testing/selftests/kvm/lib/ucall.c | 144 +++++ .../kvm/lib/{x86.c => x86_64/processor.c} | 197 ++++++- .../selftests/kvm/lib/{ => x86_64}/vmx.c | 4 +- .../kvm/{ => x86_64}/cr4_cpuid_sync_test.c | 14 +- .../kvm/{ => x86_64}/set_sregs_test.c | 2 +- .../selftests/kvm/{ => x86_64}/state_test.c | 25 +- .../kvm/{ => x86_64}/sync_regs_test.c | 2 +- .../kvm/{ => x86_64}/vmx_tsc_adjust_test.c | 23 +- 21 files changed, 1329 insertions(+), 611 deletions(-) create mode 100644 tools/testing/selftests/kvm/include/aarch64/processor.h rename tools/testing/selftests/kvm/include/{x86.h => x86_64/processor.h} (98%) rename tools/testing/selftests/kvm/include/{ => x86_64}/vmx.h (99%) create mode 100644 tools/testing/selftests/kvm/lib/aarch64/processor.c create mode 100644 tools/testing/selftests/kvm/lib/ucall.c rename tools/testing/selftests/kvm/lib/{x86.c => x86_64/processor.c} (85%) rename tools/testing/selftests/kvm/lib/{ => x86_64}/vmx.c (99%) rename tools/testing/selftests/kvm/{ => x86_64}/cr4_cpuid_sync_test.c (91%) rename tools/testing/selftests/kvm/{ => x86_64}/set_sregs_test.c (98%) rename tools/testing/selftests/kvm/{ => x86_64}/state_test.c (90%) rename tools/testing/selftests/kvm/{ => x86_64}/sync_regs_test.c (99%) rename tools/testing/selftests/kvm/{ => x86_64}/vmx_tsc_adjust_test.c (91%)Signature · 1 line
2.17.1
Paolo Bonzini
maintainerQuoted 5 lines · Andrew Jones
There are three parts to a kvm selftest: 1) the test code which runs in host userspace and _is_ the kvm userspace used with kvm, 2) the vcpu thread code which executes
KVM_RUNfor the guest code and possibly also some host userspace test code, and 3) the guest code, which is naturally run in the vcpu thread, but in guest mode.
Note that the vcpu thread is a specialty of this test. Usually it's not needed and 1+2 are the same.
Paolo
Quoted 7 lines · Andrew Jones
Hi Christoffer,
Thanks for your interest in these tests. There isn't any documentation that I know of, but it's a good idea to have some. I'll write something up soon. I'll also try to answer your questions now.
That sounds great, thanks!
Quoted 19 lines · Christoffer Dall
Quoted 11 lines
Hi Drew,
Quoted 4 lines · Andrew Jones
This series provides KVM selftests that test dirty log tracking on AArch64 for both 4K and 64K guest page sizes. Additionally the framework provides an easy way to test dirty log tracking with the recently posted dynamic IPA and 52bit IPA series[1].
I was trying to parse the commit text of patch 2, and I realized that I don't understand the 'hypercall to userspace' thing at all, which probably means I have no idea how the selftests work overall.
There are three parts to a kvm selftest: 1) the test code which runs in host userspace and _is_ the kvm userspace used with kvm, 2) the vcpu thread code which executes
KVM_RUNfor the guest code and possibly also some host userspace test code, and 3) the guest code, which is naturally run in the vcpu thread, but in guest mode.
Quoted 7 lines
The need for a "ucall" arises for 2's "possibly also some host userspace test code". In that case the guest code needs to invoke an exit from guest mode, not just to kvm, but all the way to kvm userspace. For AArch64, as you know, this can be done with an MMIO access. The reason patch 2 generalizes the concept is because for x86 this can and is done with a PIO access.
So in the world of normal KVM userspace, (2) would be a thread in the same process as (1), sharing its mm. Is this a different setup somehow, why?
Quoted 10 lines
Quoted 9 lines
I then spent a while reading various bits of documentation in the kernel tree, LWN, etc., only to realize that I don't understand how this test framework actually works.
Are the selftests modules, userspace programs, or code that is compiled with the kernel, and (somehow?) run from userspace. I thought the latter, partially based on your explanation at ELC, but then I don't understand how the "compile and run" make target works.
Quoted 13 lines
The tests are standalone userspace programs which are compiled separately, but have dependencies on kernel headers. As stated above, for kvm, each selftest is a kvm userspace (including its vcpu thread code) and guest code combined. While there's a lot of complexity in the framework, particularly for memory management, and a bit for vcpu setup, most of that can be shared among tests using the
kvm_util.handtest_util.hAPIs, allowing a given test to only have a relatively simplemain(), vcpu thread "vcpu_worker()" function, and "guest_code()" function. Guest mode code can easily share code with the kvm userspace test code (assuming the guest page tables are set up in the default way) and even data can be shared as long the accesses are done with the appropriate mappings (gva vs. hva). There's a small API to help with that as well.
Sounds cool. Beware of the attributes of the mappings such that both the guest and host have mapped the memory cacheable etc., but I'm sure you've thought of that already.
Quoted 7 lines
Quoted 4 lines
Can you help me paint the overall picture, or point me to the piece of documentation/presentation that explains the high-level picture, which I must have obviously missed somehow?
We definitely need the documentation and, in hindsight, it looks like it would have been a good BoF topic last week too.
An overview of the different testing approaches would be a good KVM Forum talk for next year, IMHO. When should you use kvm-unit-tests, and when should you use kselftests, some examples, etc. Just saying ;)
Quoted 6 lines
I think this framework has a lot of potential for KVM API testing and even for quick & dirty guest code instruction sequence tests (although instruction sequences would also fit kvm-unit-tests). I hope I can help get you and anyone else interested started.
I'll have a look at this series and glance at the code some more, it would be interesting to consider if using some of this for nested virt tests makes sense.
Thanks,
Christoffer
Quoted 38 lines · Christoffer Dall
Quoted 7 lines · Andrew Jones
Hi Christoffer,
Thanks for your interest in these tests. There isn't any documentation that I know of, but it's a good idea to have some. I'll write something up soon. I'll also try to answer your questions now.
That sounds great, thanks!
Quoted 19 lines · Christoffer Dall
Quoted 11 lines
Hi Drew,
Quoted 4 lines · Andrew Jones
This series provides KVM selftests that test dirty log tracking on AArch64 for both 4K and 64K guest page sizes. Additionally the framework provides an easy way to test dirty log tracking with the recently posted dynamic IPA and 52bit IPA series[1].
I was trying to parse the commit text of patch 2, and I realized that I don't understand the 'hypercall to userspace' thing at all, which probably means I have no idea how the selftests work overall.
There are three parts to a kvm selftest: 1) the test code which runs in host userspace and _is_ the kvm userspace used with kvm, 2) the vcpu thread code which executes
KVM_RUNfor the guest code and possibly also some host userspace test code, and 3) the guest code, which is naturally run in the vcpu thread, but in guest mode.Quoted 7 lines
The need for a "ucall" arises for 2's "possibly also some host userspace test code". In that case the guest code needs to invoke an exit from guest mode, not just to kvm, but all the way to kvm userspace. For AArch64, as you know, this can be done with an MMIO access. The reason patch 2 generalizes the concept is because for x86 this can and is done with a PIO access.
Quoted 3 lines
So in the world of normal KVM userspace, (2) would be a thread in the same process as (1), sharing its mm. Is this a different setup somehow, why?
It's the same setup. Actually the only difference is what Paolo pointed out in his reply. There's no need for an independent vcpu thread to be spawned when only one vcpu thread is needed and no additional main thread is needed. I.e. the main test / kvm userspace code can call KVM_RUN itself.
Quoted 25 lines
Quoted 11 lines
Quoted 10 lines
I then spent a while reading various bits of documentation in the kernel tree, LWN, etc., only to realize that I don't understand how this test framework actually works.
Are the selftests modules, userspace programs, or code that is compiled with the kernel, and (somehow?) run from userspace. I thought the latter, partially based on your explanation at ELC, but then I don't understand how the "compile and run" make target works.
Quoted 13 lines
The tests are standalone userspace programs which are compiled separately, but have dependencies on kernel headers. As stated above, for kvm, each selftest is a kvm userspace (including its vcpu thread code) and guest code combined. While there's a lot of complexity in the framework, particularly for memory management, and a bit for vcpu setup, most of that can be shared among tests using the
kvm_util.handtest_util.hAPIs, allowing a given test to only have a relatively simplemain(), vcpu thread "vcpu_worker()" function, and "guest_code()" function. Guest mode code can easily share code with the kvm userspace test code (assuming the guest page tables are set up in the default way) and even data can be shared as long the accesses are done with the appropriate mappings (gva vs. hva). There's a small API to help with that as well.
Quoted 3 lines
Sounds cool. Beware of the attributes of the mappings such that both the guest and host have mapped the memory cacheable etc., but I'm sure you've thought of that already.
Right. If you look at virt_pg_map(), then you'll see that I have the default set to NORMAL memory. It can be overridden by calling _virt_pg_map() directly - which might be nice to do to specifically test stage1/stage2 mapping combinations.
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
Can you help me paint the overall picture, or point me to the piece of documentation/presentation that explains the high-level picture, which I must have obviously missed somehow?
Quoted 6 lines
Quoted 2 lines
We definitely need the documentation and, in hindsight, it looks like it would have been a good BoF topic last week too.
An overview of the different testing approaches would be a good KVM Forum talk for next year, IMHO. When should you use kvm-unit-tests, and when should you use kselftests, some examples, etc. Just saying ;)
:-)
Quoted 11 lines
Quoted 7 lines
I think this framework has a lot of potential for KVM API testing and even for quick & dirty guest code instruction sequence tests (although instruction sequences would also fit kvm-unit-tests). I hope I can help get you and anyone else interested started.
I'll have a look at this series and glance at the code some more, it would be interesting to consider if using some of this for nested virt tests makes sense.
Yes. x86 has many nested tests. I think the framework was originally created with that in mind.
Thanks,
drew