Skip to contentSkip to conversation

kvm: selftests: add aarch64 framework and dirty

PATCH In review Andrew Jones 4 participants 21 messages Sep 18, 2018 – Nov 1, 2018

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

cover kvm: selftests: add aarch64 framework and dirty

This series provides KVM selftests that test dirty log tracking on AArch64 for both 4K and 64K guest page sizes. Additionally the framework provides an easy way to test dirty log tracking with the recently posted dynamic IPA and 52bit IPA series[1].

The series breaks down into parts as follows:

01-02: generalize guest code to host userspace exit support by
introducing "ucalls" - hypercalls to userspace
03-05: prepare common code for a new architecture 06-07: add virtual memory setup support for AArch64
08: add vcpu setup support for AArch64
09: port the dirty log test to AArch64
10-11: add 64K guest page size support for the dirty log test 12-13: prepare the dirty log test to also test > 40-bit guest
physical address setups by allowing the test memory region to be placed at the top of physical memory

[1] https://www.spinics.net/lists/arm-kernel/msg676819.html

Thanks,
drew

Andrew Jones (13):
kvm: selftests: vcpu_setup: set cr4.osfxsr
kvm: selftests: introduce ucall
kvm: selftests: move arch-specific files to arch-specific locations
kvm: selftests: add cscope make target
kvm: selftests: tidy up kvm_util
kvm: selftests: add vm_phy_pages_alloc
kvm: selftests: add virt mem support for aarch64
kvm: selftests: add vcpu support for aarch64
kvm: selftests: port dirty_log_test to aarch64
kvm: selftests: introduce new VM mode for 64K pages
kvm: selftests: dirty_log_test: also test 64K pages on aarch64
kvm: selftests: stop lying to aarch64 tests about PA-bits
kvm: selftests: support high GPAs in dirty_log_test

 tools/testing/selftests/kvm/.gitignore        |  11 +-
 tools/testing/selftests/kvm/Makefile          |  36 +-
 tools/testing/selftests/kvm/dirty_log_test.c  | 374 +++++++++----
 .../selftests/kvm/include/aarch64/processor.h |  55 ++
 .../testing/selftests/kvm/include/kvm_util.h  | 166 +++---
 .../testing/selftests/kvm/include/sparsebit.h |   6 +-
 .../testing/selftests/kvm/include/test_util.h |   6 +-
 .../kvm/include/{x86.h => x86_64/processor.h} |  24 +-
 .../selftests/kvm/include/{ => x86_64}/vmx.h  |   6 +-
 .../selftests/kvm/lib/aarch64/processor.c     | 311 +++++++++++
 tools/testing/selftests/kvm/lib/assert.c      |   2 +-
 tools/testing/selftests/kvm/lib/kvm_util.c    | 499 +++++++-----------
 .../selftests/kvm/lib/kvm_util_internal.h     |  33 +-
 tools/testing/selftests/kvm/lib/ucall.c       | 144 +++++
 .../kvm/lib/{x86.c => x86_64/processor.c}     | 197 ++++++-
 .../selftests/kvm/lib/{ => x86_64}/vmx.c      |   4 +-
 .../kvm/{ => x86_64}/cr4_cpuid_sync_test.c    |  14 +-
 .../kvm/{ => x86_64}/set_sregs_test.c         |   2 +-
 .../selftests/kvm/{ => x86_64}/state_test.c   |  25 +-
 .../kvm/{ => x86_64}/sync_regs_test.c         |   2 +-
 .../kvm/{ => x86_64}/vmx_tsc_adjust_test.c    |  23 +-
 21 files changed, 1329 insertions(+), 611 deletions(-)
 create mode 100644 tools/testing/selftests/kvm/include/aarch64/processor.h
 rename tools/testing/selftests/kvm/include/{x86.h => x86_64/processor.h} (98%)
 rename tools/testing/selftests/kvm/include/{ => x86_64}/vmx.h (99%)
 create mode 100644 tools/testing/selftests/kvm/lib/aarch64/processor.c
 create mode 100644 tools/testing/selftests/kvm/lib/ucall.c
 rename tools/testing/selftests/kvm/lib/{x86.c => x86_64/processor.c} (85%)
 rename tools/testing/selftests/kvm/lib/{ => x86_64}/vmx.c (99%)
 rename tools/testing/selftests/kvm/{ => x86_64}/cr4_cpuid_sync_test.c (91%)
 rename tools/testing/selftests/kvm/{ => x86_64}/set_sregs_test.c (98%)
 rename tools/testing/selftests/kvm/{ => x86_64}/state_test.c (90%)
 rename tools/testing/selftests/kvm/{ => x86_64}/sync_regs_test.c (99%)
 rename tools/testing/selftests/kvm/{ => x86_64}/vmx_tsc_adjust_test.c (91%)
Signature · 1 line
2.17.1
#1 lore raw

PATCH 1/13 kvm: selftests: vcpu_setup: set cr4.osfxsr

Guest code may want to call functions that have variable arguments. To do so, we either need to compile with -mno-sse or enable SSE in the VCPUs. As it should be pretty safe to turn on the feature, and -mno-sse would make linking test code with standard libraries difficult, we choose the feature enabling.

---
 tools/testing/selftests/kvm/lib/x86.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/selftests/kvm/lib/x86.c +1 -1
index a3122f1949a8..bb6c65ebfa77 100644
@@ -626,7 +626,7 @@ void vcpu_setup(struct kvm_vm *vm, int vcpuid, int pgd_memslot, int gdt_memslot)
 	switch (vm->mode) {
 	case VM_MODE_FLAT48PG:
 		sregs.cr0 = X86_CR0_PE | X86_CR0_NE | X86_CR0_PG;
-		sregs.cr4 |= X86_CR4_PAE;
+		sregs.cr4 |= X86_CR4_PAE | X86_CR4_OSFXSR;
 		sregs.efer |= (EFER_LME | EFER_LMA | EFER_NX);
 
 		kvm_seg_set_unusable(&sregs.ldt);
Signature · 1 line
2.17.1
#2 lore raw

PATCH 2/13 kvm: selftests: introduce ucall

In reply to Andrew Jones

Rework the guest exit to userspace code to generalize the concept into what it is, a "hypercall to userspace", and provide two implementations of it: the PortIO version currently used, but only useable by x86, and an MMIO version that other architectures (except s390) can use.

---
 tools/testing/selftests/kvm/Makefile          |   2 +-
 .../selftests/kvm/cr4_cpuid_sync_test.c       |  12 +-
 tools/testing/selftests/kvm/dirty_log_test.c  |   5 +-
 .../testing/selftests/kvm/include/kvm_util.h  |  76 ++++-----
 tools/testing/selftests/kvm/lib/kvm_util.c    |  14 --
 tools/testing/selftests/kvm/lib/ucall.c       | 144 ++++++++++++++++++
 tools/testing/selftests/kvm/state_test.c      |  23 +--
 .../selftests/kvm/vmx_tsc_adjust_test.c       |  17 +--
 8 files changed, 214 insertions(+), 79 deletions(-)
 create mode 100644 tools/testing/selftests/kvm/lib/ucall.c
tools/testing/selftests/kvm/Makefile +1 -1
index 87d1a8488af8..0b254dbf6e86 100644
@@ -3,7 +3,7 @@ all:
 top_srcdir = ../../../../
 UNAME_M := $(shell uname -m)
 
-LIBKVM = lib/assert.c lib/elf.c lib/io.c lib/kvm_util.c lib/sparsebit.c
+LIBKVM = lib/assert.c lib/elf.c lib/io.c lib/kvm_util.c lib/ucall.c lib/sparsebit.c
 LIBKVM_x86_64 = lib/x86.c lib/vmx.c
 
 TEST_GEN_PROGS_x86_64 = set_sregs_test
tools/testing/selftests/kvm/cr4_cpuid_sync_test.c +6 -6
index 11ec358bf969..fd4f419fe9ab 100644
@@ -67,6 +67,7 @@ int main(int argc, char *argv[])
 	struct kvm_vm *vm;
 	struct kvm_sregs sregs;
 	struct kvm_cpuid_entry2 *entry;
+	struct ucall uc;
 	int rc;
 
 	entry = kvm_get_supported_cpuid_entry(1);
@@ -87,21 +88,20 @@ int main(int argc, char *argv[])
 		rc = _vcpu_run(vm, VCPU_ID);
 
 		if (run->exit_reason == KVM_EXIT_IO) {
-			switch (run->io.port) {
-			case GUEST_PORT_SYNC:
+			switch (get_ucall(vm, VCPU_ID, &uc)) {
+			case UCALL_SYNC:
 				/* emulate hypervisor clearing CR4.OSXSAVE */
 				vcpu_sregs_get(vm, VCPU_ID, &sregs);
 				sregs.cr4 &= ~X86_CR4_OSXSAVE;
 				vcpu_sregs_set(vm, VCPU_ID, &sregs);
 				break;
-			case GUEST_PORT_ABORT:
+			case UCALL_ABORT:
 				TEST_ASSERT(false, "Guest CR4 bit (OSXSAVE) unsynchronized with CPUID bit.");
 				break;
-			case GUEST_PORT_DONE:
+			case UCALL_DONE:
 				goto done;
 			default:
-				TEST_ASSERT(false, "Unknown port 0x%x.",
-					    run->io.port);
+				TEST_ASSERT(false, "Unknown ucall 0x%x.", uc.cmd);
 			}
 		}
 	}
tools/testing/selftests/kvm/dirty_log_test.c +2 -3
index 0c2cdc105f96..7cf3e4ae6046 100644
@@ -110,7 +110,7 @@ void *vcpu_worker(void *data)
 	uint64_t loops, *guest_array, pages_count = 0;
 	struct kvm_vm *vm = data;
 	struct kvm_run *run;
-	struct guest_args args;
+	struct ucall uc;
 
 	run = vcpu_state(vm, VCPU_ID);
 
@@ -124,9 +124,8 @@ void *vcpu_worker(void *data)
 	while (!READ_ONCE(host_quit)) {
 		/* Let the guest to dirty these random pages */
 		ret = _vcpu_run(vm, VCPU_ID);
-		guest_args_read(vm, VCPU_ID, &args);
 		if (run->exit_reason == KVM_EXIT_IO &&
-		    args.port == GUEST_PORT_SYNC) {
+		    get_ucall(vm, VCPU_ID, &uc) == UCALL_SYNC) {
 			pages_count += TEST_PAGES_PER_LOOP;
 			generate_random_array(guest_array, TEST_PAGES_PER_LOOP);
 		} else {
tools/testing/selftests/kvm/include/kvm_util.h +41 -35

84 lines not on this page · raw message

tools/testing/selftests/kvm/lib/kvm_util.c +0 -14
index e9ba389c48db..107622ce2b8c 100644
@@ -1580,17 +1580,3 @@ void *addr_gva2hva(struct kvm_vm *vm, vm_vaddr_t gva)
 {
 	return addr_gpa2hva(vm, addr_gva2gpa(vm, gva));
 }
-
-void guest_args_read(struct kvm_vm *vm, uint32_t vcpu_id,
-		     struct guest_args *args)
-{
-	struct kvm_run *run = vcpu_state(vm, vcpu_id);
-	struct kvm_regs regs;
-
-	memset(&regs, 0, sizeof(regs));
-	vcpu_regs_get(vm, vcpu_id, &regs);
-
-	args->port = run->io.port;
-	args->arg0 = regs.rdi;
-	args->arg1 = regs.rsi;
-}
tools/testing/selftests/kvm/lib/ucall.c new file +144 -0

144 lines not on this page · raw message

tools/testing/selftests/kvm/state_test.c +12 -11
index 900e3e9dfb9f..cdf82735d6e7 100644
@@ -127,6 +127,7 @@ int main(int argc, char *argv[])
 	struct kvm_vm *vm;
 	struct kvm_run *run;
 	struct kvm_x86_state *state;
+	struct ucall uc;
 	int stage;
 
 	struct kvm_cpuid_entry2 *entry = kvm_get_supported_cpuid_entry(1);
@@ -155,23 +156,23 @@ int main(int argc, char *argv[])
 
 		memset(&regs1, 0, sizeof(regs1));
 		vcpu_regs_get(vm, VCPU_ID, &regs1);
-		switch (run->io.port) {
-		case GUEST_PORT_ABORT:
-			TEST_ASSERT(false, "%s at %s:%d", (const char *) regs1.rdi,
-				    __FILE__, regs1.rsi);
+		switch (get_ucall(vm, VCPU_ID, &uc)) {
+		case UCALL_ABORT:
+			TEST_ASSERT(false, "%s at %s:%d", (const char *)uc.args[0],
+				    __FILE__, uc.args[1]);
 			/* NOT REACHED */
-		case GUEST_PORT_SYNC:
+		case UCALL_SYNC:
 			break;
-		case GUEST_PORT_DONE:
+		case UCALL_DONE:
 			goto done;
 		default:
-			TEST_ASSERT(false, "Unknown port 0x%x.", run->io.port);
+			TEST_ASSERT(false, "Unknown ucall 0x%x.", uc.cmd);
 		}
 
-		/* PORT_SYNC is handled here.  */
-		TEST_ASSERT(!strcmp((const char *)regs1.rdi, "hello") &&
-			    regs1.rsi == stage, "Unexpected register values vmexit #%lx, got %lx",
-			    stage, (ulong) regs1.rsi);
+		/* UCALL_SYNC is handled here.  */
+		TEST_ASSERT(!strcmp((const char *)uc.args[0], "hello") &&
+			    uc.args[1] == stage, "Unexpected register values vmexit #%lx, got %lx",
+			    stage, (ulong)uc.args[1]);
 
 		state = vcpu_save_state(vm, VCPU_ID);
 		kvm_vm_release(vm);
tools/testing/selftests/kvm/vmx_tsc_adjust_test.c +8 -9
index 49bcc68b0235..8d487c78796a 100644
@@ -146,26 +146,25 @@ int main(int argc, char *argv[])
 
 	for (;;) {
 		volatile struct kvm_run *run = vcpu_state(vm, VCPU_ID);
-		struct guest_args args;
+		struct ucall uc;
 
 		vcpu_run(vm, VCPU_ID);
-		guest_args_read(vm, VCPU_ID, &args);
 		TEST_ASSERT(run->exit_reason == KVM_EXIT_IO,
 			    "Got exit_reason other than KVM_EXIT_IO: %u (%s)\n",
 			    run->exit_reason,
 			    exit_reason_str(run->exit_reason));
 
-		switch (args.port) {
-		case GUEST_PORT_ABORT:
-			TEST_ASSERT(false, "%s", (const char *) args.arg0);
+		switch (get_ucall(vm, VCPU_ID, &uc)) {
+		case UCALL_ABORT:
+			TEST_ASSERT(false, "%s", (const char *)uc.args[0]);
 			/* NOT REACHED */
-		case GUEST_PORT_SYNC:
-			report(args.arg1);
+		case UCALL_SYNC:
+			report(uc.args[1]);
 			break;
-		case GUEST_PORT_DONE:
+		case UCALL_DONE:
 			goto done;
 		default:
-			TEST_ASSERT(false, "Unknown port 0x%x.", args.port);
+			TEST_ASSERT(false, "Unknown ucall 0x%x.", uc.cmd);
 		}
 	}
 
Signature · 1 line
2.17.1
#3 lore raw

PATCH 3/13 kvm: selftests: move arch-specific files to arch-specific locations

In reply to Andrew Jones
---
 tools/testing/selftests/kvm/.gitignore        | 11 +++++-----
 tools/testing/selftests/kvm/Makefile          | 22 +++++++++----------
 .../testing/selftests/kvm/include/kvm_util.h  |  2 +-
 .../testing/selftests/kvm/include/sparsebit.h |  6 ++---
 .../testing/selftests/kvm/include/test_util.h |  6 ++---
 .../kvm/include/{x86.h => x86_64/processor.h} |  8 +++----
 .../selftests/kvm/include/{ => x86_64}/vmx.h  |  6 ++---
 tools/testing/selftests/kvm/lib/assert.c      |  2 +-
 .../selftests/kvm/lib/kvm_util_internal.h     |  8 +++----
 .../kvm/lib/{x86.c => x86_64/processor.c}     |  6 ++---
 .../selftests/kvm/lib/{ => x86_64}/vmx.c      |  4 ++--
 .../kvm/{ => x86_64}/cr4_cpuid_sync_test.c    |  2 +-
 .../kvm/{ => x86_64}/dirty_log_test.c         |  0
 .../kvm/{ => x86_64}/set_sregs_test.c         |  2 +-
 .../selftests/kvm/{ => x86_64}/state_test.c   |  2 +-
 .../kvm/{ => x86_64}/sync_regs_test.c         |  2 +-
 .../kvm/{ => x86_64}/vmx_tsc_adjust_test.c    |  6 ++---
 17 files changed, 48 insertions(+), 47 deletions(-)
 rename tools/testing/selftests/kvm/include/{x86.h => x86_64/processor.h} (99%)
 rename tools/testing/selftests/kvm/include/{ => x86_64}/vmx.h (99%)
 rename tools/testing/selftests/kvm/lib/{x86.c => x86_64/processor.c} (99%)
 rename tools/testing/selftests/kvm/lib/{ => x86_64}/vmx.c (99%)
 rename tools/testing/selftests/kvm/{ => x86_64}/cr4_cpuid_sync_test.c (98%)
 rename tools/testing/selftests/kvm/{ => x86_64}/dirty_log_test.c (100%)
 rename tools/testing/selftests/kvm/{ => x86_64}/set_sregs_test.c (98%)
 rename tools/testing/selftests/kvm/{ => x86_64}/state_test.c (99%)
 rename tools/testing/selftests/kvm/{ => x86_64}/sync_regs_test.c (99%)
 rename tools/testing/selftests/kvm/{ => x86_64}/vmx_tsc_adjust_test.c (98%)
tools/testing/selftests/kvm/.gitignore +6 -5
index 4202139d81d9..9b3933fb5043 100644
@@ -1,5 +1,6 @@
-cr4_cpuid_sync_test
-set_sregs_test
-sync_regs_test
-vmx_tsc_adjust_test
-state_test
+/x86_64/cr4_cpuid_sync_test
+/x86_64/set_sregs_test
+/x86_64/sync_regs_test
+/x86_64/vmx_tsc_adjust_test
+/x86_64/state_test
+/x86_64/dirty_log_test
tools/testing/selftests/kvm/Makefile +11 -11
index 0b254dbf6e86..82d77e10b554 100644
@@ -1,25 +1,25 @@
 all:
 
-top_srcdir = ../../../../
+top_srcdir = ../../../..
 UNAME_M := $(shell uname -m)
 
 LIBKVM = lib/assert.c lib/elf.c lib/io.c lib/kvm_util.c lib/ucall.c lib/sparsebit.c
-LIBKVM_x86_64 = lib/x86.c lib/vmx.c
+LIBKVM_x86_64 = lib/x86_64/processor.c lib/x86_64/vmx.c
 
-TEST_GEN_PROGS_x86_64 = set_sregs_test
-TEST_GEN_PROGS_x86_64 += sync_regs_test
-TEST_GEN_PROGS_x86_64 += vmx_tsc_adjust_test
-TEST_GEN_PROGS_x86_64 += cr4_cpuid_sync_test
-TEST_GEN_PROGS_x86_64 += state_test
-TEST_GEN_PROGS_x86_64 += dirty_log_test
+TEST_GEN_PROGS_x86_64 = x86_64/set_sregs_test
+TEST_GEN_PROGS_x86_64 += x86_64/sync_regs_test
+TEST_GEN_PROGS_x86_64 += x86_64/vmx_tsc_adjust_test
+TEST_GEN_PROGS_x86_64 += x86_64/cr4_cpuid_sync_test
+TEST_GEN_PROGS_x86_64 += x86_64/state_test
+TEST_GEN_PROGS_x86_64 += x86_64/dirty_log_test
 
 TEST_GEN_PROGS += $(TEST_GEN_PROGS_$(UNAME_M))
 LIBKVM += $(LIBKVM_$(UNAME_M))
 
 INSTALL_HDR_PATH = $(top_srcdir)/usr
-LINUX_HDR_PATH = $(INSTALL_HDR_PATH)/include/
-LINUX_TOOL_INCLUDE = $(top_srcdir)tools/include
-CFLAGS += -O2 -g -std=gnu99 -I$(LINUX_TOOL_INCLUDE) -I$(LINUX_HDR_PATH) -Iinclude -I$(<D) -I..
+LINUX_HDR_PATH = $(INSTALL_HDR_PATH)/include
+LINUX_TOOL_INCLUDE = $(top_srcdir)/tools/include
+CFLAGS += -O2 -g -std=gnu99 -I$(LINUX_TOOL_INCLUDE) -I$(LINUX_HDR_PATH) -Iinclude -Iinclude/$(UNAME_M) -I..
 LDFLAGS += -lpthread
 
 # After inclusion, $(OUTPUT) is defined and
tools/testing/selftests/kvm/include/kvm_util.h +1 -1
index 138289f7d489..607f68ba4ceb 100644
@@ -7,7 +7,7 @@
  *
  */
 #ifndef SELFTEST_KVM_UTIL_H
-#define SELFTEST_KVM_UTIL_H 1
+#define SELFTEST_KVM_UTIL_H
 
 #include "test_util.h"
 
tools/testing/selftests/kvm/include/sparsebit.h +3 -3
index 54cfeb6568d3..31e030915c1f 100644
@@ -15,8 +15,8 @@
  * even in the case where most bits are set.
  */
 
-#ifndef _TEST_SPARSEBIT_H_
-#define _TEST_SPARSEBIT_H_
+#ifndef SELFTEST_KVM_SPARSEBIT_H
+#define SELFTEST_KVM_SPARSEBIT_H
 
 #include <stdbool.h>
 #include <stdint.h>
@@ -72,4 +72,4 @@ void sparsebit_validate_internal(struct sparsebit *sbit);
 }
 #endif
 
-#endif /* _TEST_SPARSEBIT_H_ */
+#endif /* SELFTEST_KVM_SPARSEBIT_H */
tools/testing/selftests/kvm/include/test_util.h +3 -3
index 73c3933436ec..c7dafe8bd02c 100644
@@ -7,8 +7,8 @@
  *
  */
 
-#ifndef TEST_UTIL_H
-#define TEST_UTIL_H 1
+#ifndef SELFTEST_KVM_TEST_UTIL_H
+#define SELFTEST_KVM_TEST_UTIL_H
 
 #include <stdlib.h>
 #include <stdarg.h>
@@ -41,4 +41,4 @@ void test_assert(bool exp, const char *exp_str,
 		    #a, #b, #a, (unsigned long) __a, #b, (unsigned long) __b); \
 } while (0)
 
-#endif /* TEST_UTIL_H */
+#endif /* SELFTEST_KVM_TEST_UTIL_H */
tools/testing/selftests/kvm/include/x86_64/processor.h renamed from tools/testing/selftests/kvm/include/x86.h +4 -4
similarity index 99%
rename from tools/testing/selftests/kvm/include/x86.h
rename to tools/testing/selftests/kvm/include/x86_64/processor.h
index 42c3596815b8..bb76e8db9b2b 100644
@@ -1,5 +1,5 @@
 /*
- * tools/testing/selftests/kvm/include/x86.h
+ * tools/testing/selftests/kvm/include/x86_64/processor.h
  *
  * Copyright (C) 2018, Google LLC.
  *
@@ -7,8 +7,8 @@
  *
  */
 
-#ifndef SELFTEST_KVM_X86_H
-#define SELFTEST_KVM_X86_H
+#ifndef SELFTEST_KVM_PROCESSOR_H
+#define SELFTEST_KVM_PROCESSOR_H
 
 #include <assert.h>
 #include <stdint.h>
@@ -1044,4 +1044,4 @@ void vcpu_load_state(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_x86_state *s
 #define MSR_VM_IGNNE                    0xc0010115
 #define MSR_VM_HSAVE_PA                 0xc0010117
 
-#endif /* !SELFTEST_KVM_X86_H */
+#endif /* SELFTEST_KVM_PROCESSOR_H */
tools/testing/selftests/kvm/include/x86_64/vmx.h renamed from tools/testing/selftests/kvm/include/vmx.h +3 -3
similarity index 99%
rename from tools/testing/selftests/kvm/include/vmx.h
rename to tools/testing/selftests/kvm/include/x86_64/vmx.h
index b9ffe1024d3a..12ebd836f7ef 100644
@@ -1,5 +1,5 @@
 /*
- * tools/testing/selftests/kvm/include/vmx.h
+ * tools/testing/selftests/kvm/include/x86_64/vmx.h
  *
  * Copyright (C) 2018, Google LLC.
  *
@@ -11,7 +11,7 @@
 #define SELFTEST_KVM_VMX_H
 
 #include <stdint.h>
-#include "x86.h"
+#include "processor.h"
 
 #define CPUID_VMX_BIT				5
 
@@ -549,4 +549,4 @@ struct vmx_pages *vcpu_alloc_vmx(struct kvm_vm *vm, vm_vaddr_t *p_vmx_gva);
 bool prepare_for_vmx_operation(struct vmx_pages *vmx);
 void prepare_vmcs(struct vmx_pages *vmx, void *guest_rip, void *guest_rsp);
 
-#endif /* !SELFTEST_KVM_VMX_H */
+#endif /* SELFTEST_KVM_VMX_H */
tools/testing/selftests/kvm/lib/assert.c +1 -1
index cd01144d27c8..6398efe67885 100644
@@ -13,7 +13,7 @@
 #include <execinfo.h>
 #include <sys/syscall.h>
 
-#include "../../kselftest.h"
+#include "kselftest.h"
 
 /* Dumps the current stack trace to stderr. */
 static void __attribute__((noinline)) test_dump_stack(void);
tools/testing/selftests/kvm/lib/kvm_util_internal.h +4 -4
index 542ed606b338..0278315cd930 100644
@@ -1,13 +1,13 @@
 /*
- * tools/testing/selftests/kvm/lib/kvm_util.c
+ * tools/testing/selftests/kvm/lib/kvm_util_internal.h
  *
  * Copyright (C) 2018, Google LLC.
  *
  * This work is licensed under the terms of the GNU GPL, version 2.
  */
 
-#ifndef KVM_UTIL_INTERNAL_H
-#define KVM_UTIL_INTERNAL_H 1
+#ifndef SELFTEST_KVM_UTIL_INTERNAL_H
+#define SELFTEST_KVM_UTIL_INTERNAL_H
 
 #include "sparsebit.h"
 
@@ -69,4 +69,4 @@ void regs_dump(FILE *stream, struct kvm_regs *regs,
 void sregs_dump(FILE *stream, struct kvm_sregs *sregs,
 	uint8_t indent);
 
-#endif
+#endif /* SELFTEST_KVM_UTIL_INTERNAL_H */
tools/testing/selftests/kvm/lib/x86_64/processor.c renamed from tools/testing/selftests/kvm/lib/x86.c +3 -3
similarity index 99%
rename from tools/testing/selftests/kvm/lib/x86.c
rename to tools/testing/selftests/kvm/lib/x86_64/processor.c
index bb6c65ebfa77..79c2c5c203c0 100644
@@ -1,5 +1,5 @@
 /*
- * tools/testing/selftests/kvm/lib/x86.c
+ * tools/testing/selftests/kvm/lib/x86_64/processor.c
  *
  * Copyright (C) 2018, Google LLC.
  *
@@ -10,8 +10,8 @@
 
 #include "test_util.h"
 #include "kvm_util.h"
-#include "kvm_util_internal.h"
-#include "x86.h"
+#include "../kvm_util_internal.h"
+#include "processor.h"
 
 /* Minimum physical address used for virtual translation tables. */
 #define KVM_GUEST_PAGE_TABLE_MIN_PADDR 0x180000
tools/testing/selftests/kvm/lib/x86_64/vmx.c renamed from tools/testing/selftests/kvm/lib/vmx.c +2 -2
similarity index 99%
rename from tools/testing/selftests/kvm/lib/vmx.c
rename to tools/testing/selftests/kvm/lib/x86_64/vmx.c
index b987c3c970eb..d7c401472247 100644
@@ -1,5 +1,5 @@
 /*
- * tools/testing/selftests/kvm/lib/x86.c
+ * tools/testing/selftests/kvm/lib/x86_64/vmx.c
  *
  * Copyright (C) 2018, Google LLC.
  *
@@ -10,7 +10,7 @@
 
 #include "test_util.h"
 #include "kvm_util.h"
-#include "x86.h"
+#include "processor.h"
 #include "vmx.h"
 
 /* Allocate memory regions for nested VMX tests.
tools/testing/selftests/kvm/x86_64/cr4_cpuid_sync_test.c renamed from tools/testing/selftests/kvm/cr4_cpuid_sync_test.c +1 -1
similarity index 98%
rename from tools/testing/selftests/kvm/cr4_cpuid_sync_test.c
rename to tools/testing/selftests/kvm/x86_64/cr4_cpuid_sync_test.c
index fd4f419fe9ab..d503a51fad30 100644
@@ -17,7 +17,7 @@
 #include "test_util.h"
 
 #include "kvm_util.h"
-#include "x86.h"
+#include "processor.h"
 
 #define X86_FEATURE_XSAVE	(1<<26)
 #define X86_FEATURE_OSXSAVE	(1<<27)
tools/testing/selftests/kvm/x86_64/dirty_log_test.c renamed from tools/testing/selftests/kvm/dirty_log_test.c +0 -0
similarity index 100%
rename from tools/testing/selftests/kvm/dirty_log_test.c
rename to tools/testing/selftests/kvm/x86_64/dirty_log_test.c
tools/testing/selftests/kvm/x86_64/set_sregs_test.c renamed from tools/testing/selftests/kvm/set_sregs_test.c +1 -1
similarity index 98%
rename from tools/testing/selftests/kvm/set_sregs_test.c
rename to tools/testing/selftests/kvm/x86_64/set_sregs_test.c
index 881419d5746e..35640e8e95bc 100644
@@ -22,7 +22,7 @@
 #include "test_util.h"
 
 #include "kvm_util.h"
-#include "x86.h"
+#include "processor.h"
 
 #define VCPU_ID                  5
 
tools/testing/selftests/kvm/x86_64/state_test.c renamed from tools/testing/selftests/kvm/state_test.c +1 -1
similarity index 99%
rename from tools/testing/selftests/kvm/state_test.c
rename to tools/testing/selftests/kvm/x86_64/state_test.c
index cdf82735d6e7..43df194a7c1e 100644
@@ -17,7 +17,7 @@
 #include "test_util.h"
 
 #include "kvm_util.h"
-#include "x86.h"
+#include "processor.h"
 #include "vmx.h"
 
 #define VCPU_ID		5
tools/testing/selftests/kvm/x86_64/sync_regs_test.c renamed from tools/testing/selftests/kvm/sync_regs_test.c +1 -1
similarity index 99%
rename from tools/testing/selftests/kvm/sync_regs_test.c
rename to tools/testing/selftests/kvm/x86_64/sync_regs_test.c
index 213343e5dff9..c8478ce9ea77 100644
@@ -19,7 +19,7 @@
 
 #include "test_util.h"
 #include "kvm_util.h"
-#include "x86.h"
+#include "processor.h"
 
 #define VCPU_ID 5
 
tools/testing/selftests/kvm/x86_64/vmx_tsc_adjust_test.c renamed from tools/testing/selftests/kvm/vmx_tsc_adjust_test.c +3 -3
similarity index 98%
rename from tools/testing/selftests/kvm/vmx_tsc_adjust_test.c
rename to tools/testing/selftests/kvm/x86_64/vmx_tsc_adjust_test.c
index 8d487c78796a..38a91a5f04ac 100644
@@ -1,5 +1,5 @@
 /*
- * gtests/tests/vmx_tsc_adjust_test.c
+ * vmx_tsc_adjust_test
  *
  * Copyright (C) 2018, Google LLC.
  *
@@ -22,13 +22,13 @@
 
 #include "test_util.h"
 #include "kvm_util.h"
-#include "x86.h"
+#include "processor.h"
 #include "vmx.h"
 
 #include <string.h>
 #include <sys/ioctl.h>
 
-#include "../kselftest.h"
+#include "kselftest.h"
 
 #ifndef MSR_IA32_TSC_ADJUST
 #define MSR_IA32_TSC_ADJUST 0x3b
Signature · 1 line
2.17.1
#4 lore raw

PATCH 4/13 kvm: selftests: add cscope make target

In reply to Andrew Jones
---
 tools/testing/selftests/kvm/Makefile | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)
tools/testing/selftests/kvm/Makefile +10 -1
index 82d77e10b554..d603d7865cd1 100644
@@ -28,7 +28,7 @@ include ../lib.mk
 
 STATIC_LIBS := $(OUTPUT)/libkvm.a
 LIBKVM_OBJ := $(patsubst %.c, $(OUTPUT)/%.o, $(LIBKVM))
-EXTRA_CLEAN += $(LIBKVM_OBJ) $(STATIC_LIBS)
+EXTRA_CLEAN += $(LIBKVM_OBJ) $(STATIC_LIBS) cscope.*
 
 x := $(shell mkdir -p $(sort $(dir $(LIBKVM_OBJ))))
 $(LIBKVM_OBJ): $(OUTPUT)/%.o: %.c
@@ -40,3 +40,12 @@ $(OUTPUT)/libkvm.a: $(LIBKVM_OBJ)
 all: $(STATIC_LIBS)
 $(TEST_GEN_PROGS): $(STATIC_LIBS)
 $(STATIC_LIBS):| khdr
+
+cscope: include_paths = $(LINUX_TOOL_INCLUDE) $(LINUX_HDR_PATH) include lib ..
+cscope:
+	$(RM) cscope.*
+	(find $(include_paths) -name '*.h' \
+		-exec realpath --relative-base=$(PWD) {} \;; \
+	find . -name '*.c' \
+		-exec realpath --relative-base=$(PWD) {} \;) | sort -u > cscope.files
+	cscope -b
Signature · 1 line
2.17.1
#5 lore raw

PATCH 5/13 kvm: selftests: tidy up kvm_util

In reply to Andrew Jones

Tidy up kvm-util code: code/comment formatting, remove unused code, and move x86 specific code out. We also move vcpu_dump() out of common code, because not all arches (AArch64) have KVM_GET_REGS.

---
 .../testing/selftests/kvm/include/kvm_util.h  |  77 ++--
 .../selftests/kvm/include/x86_64/processor.h  |  16 +-
 tools/testing/selftests/kvm/lib/kvm_util.c    | 372 +++++-------------
 .../selftests/kvm/lib/kvm_util_internal.h     |  22 +-
 .../selftests/kvm/lib/x86_64/processor.c      | 179 +++++++++
 .../selftests/kvm/x86_64/dirty_log_test.c     |   1 +
 6 files changed, 337 insertions(+), 330 deletions(-)
tools/testing/selftests/kvm/include/kvm_util.h +26 -51
index 607f68ba4ceb..fcde5cf48d06 100644
@@ -17,12 +17,6 @@
 
 #include "sparsebit.h"
 
-/*
- * Memslots can't cover the gfn starting at this gpa otherwise vCPUs can't be
- * created. Only applies to VMs using EPT.
- */
-#define KVM_DEFAULT_IDENTITY_MAP_ADDRESS 0xfffbc000ul
-
 
 /* Callers of kvm_util only have an incomplete/opaque description of the
  * structure kvm_util is using to maintain the state of a VM.
@@ -33,11 +27,11 @@ typedef uint64_t vm_paddr_t; /* Virtual Machine (Guest) physical address */
 typedef uint64_t vm_vaddr_t; /* Virtual Machine (Guest) virtual address */
 
 /* Minimum allocated guest virtual and physical addresses */
-#define KVM_UTIL_MIN_VADDR 0x2000
+#define KVM_UTIL_MIN_VADDR		0x2000
 
 #define DEFAULT_GUEST_PHY_PAGES		512
 #define DEFAULT_GUEST_STACK_VADDR_MIN	0xab6000
-#define DEFAULT_STACK_PGS               5
+#define DEFAULT_STACK_PGS		5
 
 enum vm_guest_mode {
 	VM_MODE_FLAT48PG,
@@ -57,15 +51,15 @@ void kvm_vm_restart(struct kvm_vm *vmp, int perm);
 void kvm_vm_release(struct kvm_vm *vmp);
 void kvm_vm_get_dirty_log(struct kvm_vm *vm, int slot, void *log);
 
-int kvm_memcmp_hva_gva(void *hva,
-	struct kvm_vm *vm, const vm_vaddr_t gva, size_t len);
+int kvm_memcmp_hva_gva(void *hva, struct kvm_vm *vm, const vm_vaddr_t gva,
+		       size_t len);
 
 void kvm_vm_elf_load(struct kvm_vm *vm, const char *filename,
-	uint32_t data_memslot, uint32_t pgd_memslot);
+		     uint32_t data_memslot, uint32_t pgd_memslot);
 
 void vm_dump(FILE *stream, struct kvm_vm *vm, uint8_t indent);
-void vcpu_dump(FILE *stream, struct kvm_vm *vm,
-	uint32_t vcpuid, uint8_t indent);
+void vcpu_dump(FILE *stream, struct kvm_vm *vm, uint32_t vcpuid,
+	       uint8_t indent);
 
 void vm_create_irqchip(struct kvm_vm *vm);
 
@@ -74,13 +68,14 @@ void vm_userspace_mem_region_add(struct kvm_vm *vm,
 	uint64_t guest_paddr, uint32_t slot, uint64_t npages,
 	uint32_t flags);
 
-void vcpu_ioctl(struct kvm_vm *vm,
-	uint32_t vcpuid, unsigned long ioctl, void *arg);
+void vcpu_ioctl(struct kvm_vm *vm, uint32_t vcpuid, unsigned long ioctl,
+		void *arg);
 void vm_ioctl(struct kvm_vm *vm, unsigned long ioctl, void *arg);
 void vm_mem_region_set_flags(struct kvm_vm *vm, uint32_t slot, uint32_t flags);
-void vm_vcpu_add(struct kvm_vm *vm, uint32_t vcpuid, int pgd_memslot, int gdt_memslot);
+void vm_vcpu_add(struct kvm_vm *vm, uint32_t vcpuid, int pgd_memslot,
+		 int gdt_memslot);
 vm_vaddr_t vm_vaddr_alloc(struct kvm_vm *vm, size_t sz, vm_vaddr_t vaddr_min,
-	uint32_t data_memslot, uint32_t pgd_memslot);
+			  uint32_t data_memslot, uint32_t pgd_memslot);
 void virt_map(struct kvm_vm *vm, uint64_t vaddr, uint64_t paddr,
 	      size_t size, uint32_t pgd_memslot);
 void *addr_gpa2hva(struct kvm_vm *vm, vm_paddr_t gpa);
@@ -92,53 +87,33 @@ struct kvm_run *vcpu_state(struct kvm_vm *vm, uint32_t vcpuid);
 void vcpu_run(struct kvm_vm *vm, uint32_t vcpuid);
 int _vcpu_run(struct kvm_vm *vm, uint32_t vcpuid);
 void vcpu_set_mp_state(struct kvm_vm *vm, uint32_t vcpuid,
-	struct kvm_mp_state *mp_state);
-void vcpu_regs_get(struct kvm_vm *vm,
-	uint32_t vcpuid, struct kvm_regs *regs);
-void vcpu_regs_set(struct kvm_vm *vm,
-	uint32_t vcpuid, struct kvm_regs *regs);
+		       struct kvm_mp_state *mp_state);
+void vcpu_regs_get(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_regs *regs);
+void vcpu_regs_set(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_regs *regs);
 void vcpu_args_set(struct kvm_vm *vm, uint32_t vcpuid, unsigned int num, ...);
-void vcpu_sregs_get(struct kvm_vm *vm,
-	uint32_t vcpuid, struct kvm_sregs *sregs);
-void vcpu_sregs_set(struct kvm_vm *vm,
-	uint32_t vcpuid, struct kvm_sregs *sregs);
-int _vcpu_sregs_set(struct kvm_vm *vm,
-	uint32_t vcpuid, struct kvm_sregs *sregs);
+void vcpu_sregs_get(struct kvm_vm *vm, uint32_t vcpuid,
+		    struct kvm_sregs *sregs);
+void vcpu_sregs_set(struct kvm_vm *vm, uint32_t vcpuid,
+		    struct kvm_sregs *sregs);
+int _vcpu_sregs_set(struct kvm_vm *vm, uint32_t vcpuid,
+		    struct kvm_sregs *sregs);
 void vcpu_events_get(struct kvm_vm *vm, uint32_t vcpuid,
-			  struct kvm_vcpu_events *events);
+		     struct kvm_vcpu_events *events);
 void vcpu_events_set(struct kvm_vm *vm, uint32_t vcpuid,
-			  struct kvm_vcpu_events *events);
+		     struct kvm_vcpu_events *events);
 
 const char *exit_reason_str(unsigned int exit_reason);
 
 void virt_pgd_alloc(struct kvm_vm *vm, uint32_t pgd_memslot);
 void virt_pg_map(struct kvm_vm *vm, uint64_t vaddr, uint64_t paddr,
-	uint32_t pgd_memslot);
-vm_paddr_t vm_phy_page_alloc(struct kvm_vm *vm,
-	vm_paddr_t paddr_min, uint32_t memslot);
-
-struct kvm_cpuid2 *kvm_get_supported_cpuid(void);
-void vcpu_set_cpuid(
-	struct kvm_vm *vm, uint32_t vcpuid, struct kvm_cpuid2 *cpuid);
-
-struct kvm_cpuid_entry2 *
-kvm_get_supported_cpuid_index(uint32_t function, uint32_t index);
-
-static inline struct kvm_cpuid_entry2 *
-kvm_get_supported_cpuid_entry(uint32_t function)
-{
-	return kvm_get_supported_cpuid_index(function, 0);
-}
+		 uint32_t pgd_memslot);
+vm_paddr_t vm_phy_page_alloc(struct kvm_vm *vm, vm_paddr_t paddr_min,
+			     uint32_t memslot);
 
 struct kvm_vm *vm_create_default(uint32_t vcpuid, uint64_t extra_mem_size,
 				 void *guest_code);
 void vm_vcpu_add_default(struct kvm_vm *vm, uint32_t vcpuid, void *guest_code);
 
-typedef void (*vmx_guest_code_t)(vm_vaddr_t vmxon_vaddr,
-				 vm_paddr_t vmxon_paddr,
-				 vm_vaddr_t vmcs_vaddr,
-				 vm_paddr_t vmcs_paddr);
-
 struct kvm_userspace_memory_region *
 kvm_userspace_memory_region_find(struct kvm_vm *vm, uint64_t start,
 				 uint64_t end);
tools/testing/selftests/kvm/include/x86_64/processor.h +15 -1
index bb76e8db9b2b..edec2aad2440 100644
@@ -305,7 +305,21 @@ static inline unsigned long get_xmm(int n)
 
 struct kvm_x86_state;
 struct kvm_x86_state *vcpu_save_state(struct kvm_vm *vm, uint32_t vcpuid);
-void vcpu_load_state(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_x86_state *state);
+void vcpu_load_state(struct kvm_vm *vm, uint32_t vcpuid,
+		     struct kvm_x86_state *state);
+
+struct kvm_cpuid2 *kvm_get_supported_cpuid(void);
+void vcpu_set_cpuid(struct kvm_vm *vm, uint32_t vcpuid,
+		    struct kvm_cpuid2 *cpuid);
+
+struct kvm_cpuid_entry2 *
+kvm_get_supported_cpuid_index(uint32_t function, uint32_t index);
+
+static inline struct kvm_cpuid_entry2 *
+kvm_get_supported_cpuid_entry(uint32_t function)
+{
+	return kvm_get_supported_cpuid_index(function, 0);
+}
 
 /*
  * Basic CPU control in CR0
tools/testing/selftests/kvm/lib/kvm_util.c +106 -266

798 lines not on this page · raw message

tools/testing/selftests/kvm/lib/kvm_util_internal.h +10 -12
index 0278315cd930..62b9dc1926dc 100644
@@ -11,18 +11,19 @@
 
 #include "sparsebit.h"
 
+#define KVM_DEV_PATH		"/dev/kvm"
+
 #ifndef BITS_PER_BYTE
-#define BITS_PER_BYTE           8
+#define BITS_PER_BYTE		8
 #endif
 
 #ifndef BITS_PER_LONG
-#define BITS_PER_LONG (BITS_PER_BYTE * sizeof(long))
+#define BITS_PER_LONG		(BITS_PER_BYTE * sizeof(long))
 #endif
 
 #define DIV_ROUND_UP(n, d)	(((n) + (d) - 1) / (d))
-#define BITS_TO_LONGS(nr)       DIV_ROUND_UP(nr, BITS_PER_LONG)
+#define BITS_TO_LONGS(nr)	DIV_ROUND_UP(nr, BITS_PER_LONG)
 
-/* Concrete definition of struct kvm_vm. */
 struct userspace_mem_region {
 	struct userspace_mem_region *next, *prev;
 	struct kvm_userspace_memory_region region;
@@ -52,7 +53,6 @@ struct kvm_vm {
 	struct userspace_mem_region *userspace_mem_region_head;
 	struct sparsebit *vpages_valid;
 	struct sparsebit *vpages_mapped;
-
 	bool has_irqchip;
 	bool pgd_created;
 	vm_paddr_t pgd;
@@ -60,13 +60,11 @@ struct kvm_vm {
 	vm_vaddr_t tss;
 };
 
-struct vcpu *vcpu_find(struct kvm_vm *vm,
-	uint32_t vcpuid);
-void vcpu_setup(struct kvm_vm *vm, int vcpuid, int pgd_memslot, int gdt_memslot);
+struct vcpu *vcpu_find(struct kvm_vm *vm, uint32_t vcpuid);
+void vcpu_setup(struct kvm_vm *vm, int vcpuid, int pgd_memslot,
+		int gdt_memslot);
 void virt_dump(FILE *stream, struct kvm_vm *vm, uint8_t indent);
-void regs_dump(FILE *stream, struct kvm_regs *regs,
-	uint8_t indent);
-void sregs_dump(FILE *stream, struct kvm_sregs *sregs,
-	uint8_t indent);
+void regs_dump(FILE *stream, struct kvm_regs *regs, uint8_t indent);
+void sregs_dump(FILE *stream, struct kvm_sregs *sregs, uint8_t indent);
 
 #endif /* SELFTEST_KVM_UTIL_INTERNAL_H */
tools/testing/selftests/kvm/lib/x86_64/processor.c +179 -0

197 lines not on this page · raw message

tools/testing/selftests/kvm/x86_64/dirty_log_test.c +1 -0
index 7cf3e4ae6046..395a6e07d37c 100644
@@ -15,6 +15,7 @@
 
 #include "test_util.h"
 #include "kvm_util.h"
+#include "processor.h"
 
 #define  DEBUG                 printf
 
Signature · 1 line
2.17.1
#6 lore raw

PATCH 6/13 kvm: selftests: add vm_phy_pages_alloc

In reply to Andrew Jones
---
 .../testing/selftests/kvm/include/kvm_util.h  |  2 +
 tools/testing/selftests/kvm/lib/kvm_util.c    | 60 ++++++++++++-------
 2 files changed, 39 insertions(+), 23 deletions(-)
tools/testing/selftests/kvm/include/kvm_util.h +2 -0
index fcde5cf48d06..752715a78a9b 100644
@@ -109,6 +109,8 @@ void virt_pg_map(struct kvm_vm *vm, uint64_t vaddr, uint64_t paddr,
 		 uint32_t pgd_memslot);
 vm_paddr_t vm_phy_page_alloc(struct kvm_vm *vm, vm_paddr_t paddr_min,
 			     uint32_t memslot);
+vm_paddr_t vm_phy_pages_alloc(struct kvm_vm *vm, size_t num,
+			      vm_paddr_t paddr_min, uint32_t memslot);
 
 struct kvm_vm *vm_create_default(uint32_t vcpuid, uint64_t extra_mem_size,
 				 void *guest_code);
tools/testing/selftests/kvm/lib/kvm_util.c +37 -23

88 lines not on this page · raw message

Signature · 1 line
2.17.1
#7 lore raw

PATCH 7/13 kvm: selftests: add virt mem support for aarch64

In reply to Andrew Jones
---
 .../selftests/kvm/lib/aarch64/processor.c     | 216 ++++++++++++++++++
 tools/testing/selftests/kvm/lib/kvm_util.c    |   2 +
 .../selftests/kvm/lib/kvm_util_internal.h     |   2 +
 3 files changed, 220 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/lib/aarch64/processor.c
tools/testing/selftests/kvm/lib/aarch64/processor.c new file +216 -0
new file mode 100644
index 000000000000..464d4d074249
@@ -0,0 +1,216 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * AArch64 code
+ *
+ * Copyright (C) 2018, Red Hat, Inc.
+ */
+
+#include "kvm_util.h"
+#include "../kvm_util_internal.h"
+
+#define KVM_GUEST_PAGE_TABLE_MIN_PADDR		0x180000
+
+static uint64_t page_align(struct kvm_vm *vm, uint64_t v)
+{
+	return (v + vm->page_size) & ~(vm->page_size - 1);
+}
+
+static uint64_t pgd_index(struct kvm_vm *vm, vm_vaddr_t gva)
+{
+	unsigned int shift = (vm->pgtable_levels - 1) * (vm->page_shift - 3) + vm->page_shift;
+	uint64_t mask = (1UL << (vm->va_bits - shift)) - 1;
+
+	return (gva >> shift) & mask;
+}
+
+static uint64_t pud_index(struct kvm_vm *vm, vm_vaddr_t gva)
+{
+	unsigned int shift = 2 * (vm->page_shift - 3) + vm->page_shift;
+	uint64_t mask = (1UL << (vm->page_shift - 3)) - 1;
+
+	TEST_ASSERT(vm->pgtable_levels == 4,
+		"Mode %d does not have 4 page table levels", vm->mode);
+
+	return (gva >> shift) & mask;
+}
+
+static uint64_t pmd_index(struct kvm_vm *vm, vm_vaddr_t gva)
+{
+	unsigned int shift = (vm->page_shift - 3) + vm->page_shift;
+	uint64_t mask = (1UL << (vm->page_shift - 3)) - 1;
+
+	TEST_ASSERT(vm->pgtable_levels >= 3,
+		"Mode %d does not have >= 3 page table levels", vm->mode);
+
+	return (gva >> shift) & mask;
+}
+
+static uint64_t pte_index(struct kvm_vm *vm, vm_vaddr_t gva)
+{
+	uint64_t mask = (1UL << (vm->page_shift - 3)) - 1;
+	return (gva >> vm->page_shift) & mask;
+}
+
+static uint64_t pte_addr(struct kvm_vm *vm, uint64_t entry)
+{
+	uint64_t mask = ((1UL << (vm->va_bits - vm->page_shift)) - 1) << vm->page_shift;
+	return entry & mask;
+}
+
+static uint64_t ptrs_per_pgd(struct kvm_vm *vm)
+{
+	unsigned int shift = (vm->pgtable_levels - 1) * (vm->page_shift - 3) + vm->page_shift;
+	return 1 << (vm->va_bits - shift);
+}
+
+static uint64_t ptrs_per_pte(struct kvm_vm *vm)
+{
+	return 1 << (vm->page_shift - 3);
+}
+
+void virt_pgd_alloc(struct kvm_vm *vm, uint32_t pgd_memslot)
+{
+	int rc;
+
+	if (!vm->pgd_created) {
+		vm_paddr_t paddr = vm_phy_pages_alloc(vm,
+			page_align(vm, ptrs_per_pgd(vm) * 8) / vm->page_size,
+			KVM_GUEST_PAGE_TABLE_MIN_PADDR, pgd_memslot);
+		vm->pgd = paddr;
+		vm->pgd_created = true;
+	}
+}
+
+void _virt_pg_map(struct kvm_vm *vm, uint64_t vaddr, uint64_t paddr,
+		  uint32_t pgd_memslot, uint64_t flags)
+{
+	uint8_t attr_idx = flags & 7;
+	uint64_t *ptep;
+
+	TEST_ASSERT((vaddr % vm->page_size) == 0,
+		"Virtual address not on page boundary,\n"
+		"  vaddr: 0x%lx vm->page_size: 0x%x", vaddr, vm->page_size);
+	TEST_ASSERT(sparsebit_is_set(vm->vpages_valid,
+		(vaddr >> vm->page_shift)),
+		"Invalid virtual address, vaddr: 0x%lx", vaddr);
+	TEST_ASSERT((paddr % vm->page_size) == 0,
+		"Physical address not on page boundary,\n"
+		"  paddr: 0x%lx vm->page_size: 0x%x", paddr, vm->page_size);
+	TEST_ASSERT((paddr >> vm->page_shift) <= vm->max_gfn,
+		"Physical address beyond beyond maximum supported,\n"
+		"  paddr: 0x%lx vm->max_gfn: 0x%lx vm->page_size: 0x%x",
+		paddr, vm->max_gfn, vm->page_size);
+
+	ptep = addr_gpa2hva(vm, vm->pgd) + pgd_index(vm, vaddr) * 8;
+	if (!*ptep) {
+		*ptep = vm_phy_page_alloc(vm, KVM_GUEST_PAGE_TABLE_MIN_PADDR, pgd_memslot);
+		*ptep |= 3;
+	}
+
+	switch (vm->pgtable_levels) {
+	case 4:
+		ptep = addr_gpa2hva(vm, pte_addr(vm, *ptep)) + pud_index(vm, vaddr) * 8;
+		if (!*ptep) {
+			*ptep = vm_phy_page_alloc(vm, KVM_GUEST_PAGE_TABLE_MIN_PADDR, pgd_memslot);
+			*ptep |= 3;
+		}
+		/* fall through */
+	case 3:
+		ptep = addr_gpa2hva(vm, pte_addr(vm, *ptep)) + pmd_index(vm, vaddr) * 8;
+		if (!*ptep) {
+			*ptep = vm_phy_page_alloc(vm, KVM_GUEST_PAGE_TABLE_MIN_PADDR, pgd_memslot);
+			*ptep |= 3;
+		}
+		/* fall through */
+	case 2:
+		ptep = addr_gpa2hva(vm, pte_addr(vm, *ptep)) + pte_index(vm, vaddr) * 8;
+		break;
+	default:
+		TEST_ASSERT(false, "Page table levels must be 2, 3, or 4");
+	}
+
+	*ptep = paddr | 3;
+	*ptep |= (attr_idx << 2) | (1 << 10) /* Access Flag */;
+}
+
+void virt_pg_map(struct kvm_vm *vm, uint64_t vaddr, uint64_t paddr,
+		 uint32_t pgd_memslot)
+{
+	uint64_t attr_idx = 4; /* NORMAL (See DEFAULT_MAIR_EL1) */
+
+	_virt_pg_map(vm, vaddr, paddr, pgd_memslot, attr_idx);
+}
+
+vm_paddr_t addr_gva2gpa(struct kvm_vm *vm, vm_vaddr_t gva)
+{
+	uint64_t *ptep;
+
+	if (!vm->pgd_created)
+		goto unmapped_gva;
+
+	ptep = addr_gpa2hva(vm, vm->pgd) + pgd_index(vm, gva) * 8;
+	if (!ptep)
+		goto unmapped_gva;
+
+	switch (vm->pgtable_levels) {
+	case 4:
+		ptep = addr_gpa2hva(vm, pte_addr(vm, *ptep)) + pud_index(vm, gva) * 8;
+		if (!ptep)
+			goto unmapped_gva;
+		/* fall through */
+	case 3:
+		ptep = addr_gpa2hva(vm, pte_addr(vm, *ptep)) + pmd_index(vm, gva) * 8;
+		if (!ptep)
+			goto unmapped_gva;
+		/* fall through */
+	case 2:
+		ptep = addr_gpa2hva(vm, pte_addr(vm, *ptep)) + pte_index(vm, gva) * 8;
+		if (!ptep)
+			goto unmapped_gva;
+		break;
+	default:
+		TEST_ASSERT(false, "Page table levels must be 2, 3, or 4");
+	}
+
+	return pte_addr(vm, *ptep) + (gva & (vm->page_size - 1));
+
+unmapped_gva:
+	TEST_ASSERT(false, "No mapping for vm virtual address, "
+		    "gva: 0x%lx", gva);
+}
+
+static void pte_dump(FILE *stream, struct kvm_vm *vm, uint8_t indent, uint64_t page, int level)
+{
+#ifdef DEBUG_VM
+	static const char * const type[] = { "", "pud", "pmd", "pte" };
+	uint64_t pte, *ptep;
+
+	if (level == 4)
+		return;
+
+	for (pte = page; pte < page + ptrs_per_pte(vm) * 8; pte += 8) {
+		ptep = addr_gpa2hva(vm, pte);
+		if (!*ptep)
+			continue;
+		printf("%*s%s: %lx: %lx at %p\n", indent, "", type[level], pte, *ptep, ptep);
+		pte_dump(stream, vm, indent + 1, pte_addr(vm, *ptep), level + 1);
+	}
+#endif
+}
+
+void virt_dump(FILE *stream, struct kvm_vm *vm, uint8_t indent)
+{
+	int level = 4 - (vm->pgtable_levels - 1);
+	uint64_t pgd, *ptep;
+
+	if (!vm->pgd_created)
+		return;
+
+	for (pgd = vm->pgd; pgd < vm->pgd + ptrs_per_pgd(vm) * 8; pgd += 8) {
+		ptep = addr_gpa2hva(vm, pgd);
+		if (!*ptep)
+			continue;
+		printf("%*spgd: %lx: %lx at %p\n", indent, "", pgd, *ptep, ptep);
+		pte_dump(stream, vm, indent + 1, pte_addr(vm, *ptep), level);
+	}
+}
tools/testing/selftests/kvm/lib/kvm_util.c +2 -0
index ea5e5ea316de..b5e9eb7360a2 100644
@@ -106,6 +106,8 @@ struct kvm_vm *vm_create(enum vm_guest_mode mode, uint64_t phy_pages, int perm)
 	/* Setup mode specific traits. */
 	switch (vm->mode) {
 	case VM_MODE_FLAT48PG:
+		vm->pgtable_levels = 4;
+		vm->va_bits = 48;
 		vm->page_size = 0x1000;
 		vm->page_shift = 12;
 
tools/testing/selftests/kvm/lib/kvm_util_internal.h +2 -0
index 62b9dc1926dc..813145f3e4b9 100644
@@ -46,6 +46,8 @@ struct kvm_vm {
 	int mode;
 	int kvm_fd;
 	int fd;
+	unsigned int pgtable_levels;
+	unsigned int va_bits;
 	unsigned int page_size;
 	unsigned int page_shift;
 	uint64_t max_gfn;
Signature · 1 line
2.17.1
#8 lore raw

PATCH 8/13 kvm: selftests: add vcpu support for aarch64

In reply to Andrew Jones

This code adds VM and VCPU setup code for the VM_MODE_FLAT48PG mode. The VM_MODE_FLAT48PG isn't yet fully supportable, as it defines the guest physical address limit as 52-bits, and KVM currently only supports guests with up to 40-bit physical addresses (see KVM_PHYS_SHIFT). VM_MODE_FLAT48PG will work fine, though, as long as no >= 40-bit physical addresses are used.

---
 .../selftests/kvm/include/aarch64/processor.h | 55 ++++++++++++
 .../selftests/kvm/lib/aarch64/processor.c     | 83 +++++++++++++++++++
 2 files changed, 138 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/include/aarch64/processor.h
tools/testing/selftests/kvm/include/aarch64/processor.h new file +55 -0
new file mode 100644
index 000000000000..9ef2ab1a0c08
@@ -0,0 +1,55 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * AArch64 processor specific defines
+ *
+ * Copyright (C) 2018, Red Hat, Inc.
+ */
+#ifndef SELFTEST_KVM_PROCESSOR_H
+#define SELFTEST_KVM_PROCESSOR_H
+
+#include "kvm_util.h"
+
+
+#define ARM64_CORE_REG(x) (KVM_REG_ARM64 | KVM_REG_SIZE_U64 | \
+			   KVM_REG_ARM_CORE | KVM_REG_ARM_CORE_REG(x))
+
+#define CPACR_EL1	3, 0,  1, 0, 2
+#define TCR_EL1		3, 0,  2, 0, 2
+#define MAIR_EL1	3, 0, 10, 2, 0
+#define TTBR0_EL1	3, 0,  2, 0, 0
+#define SCTLR_EL1	3, 0,  1, 0, 0
+
+/*
+ * Default MAIR
+ *                  index   attribute
+ * DEVICE_nGnRnE      0     0000:0000
+ * DEVICE_nGnRE       1     0000:0100
+ * DEVICE_GRE         2     0000:1100
+ * NORMAL_NC          3     0100:0100
+ * NORMAL             4     1111:1111
+ * NORMAL_WT          5     1011:1011
+ */
+#define DEFAULT_MAIR_EL1 ((0x00ul << (0 * 8)) | \
+			  (0x04ul << (1 * 8)) | \
+			  (0x0cul << (2 * 8)) | \
+			  (0x44ul << (3 * 8)) | \
+			  (0xfful << (4 * 8)) | \
+			  (0xbbul << (5 * 8)))
+
+static inline void get_reg(struct kvm_vm *vm, uint32_t vcpuid, uint64_t id, uint64_t *addr)
+{
+	struct kvm_one_reg reg;
+	reg.id = id;
+	reg.addr = (uint64_t)addr;
+	vcpu_ioctl(vm, vcpuid, KVM_GET_ONE_REG, &reg);
+}
+
+static inline void set_reg(struct kvm_vm *vm, uint32_t vcpuid, uint64_t id, uint64_t val)
+{
+	struct kvm_one_reg reg;
+	reg.id = id;
+	reg.addr = (uint64_t)&val;
+	vcpu_ioctl(vm, vcpuid, KVM_SET_ONE_REG, &reg);
+}
+
+#endif /* SELFTEST_KVM_PROCESSOR_H */
tools/testing/selftests/kvm/lib/aarch64/processor.c +83 -0

96 lines not on this page · raw message

Signature · 1 line
2.17.1
#9 lore raw

PATCH 9/13 kvm: selftests: port dirty_log_test to aarch64

In reply to Andrew Jones

While we're messing with the code for the port and to support guest page sizes that are less than the host page size, we also make some code formatting cleanups and apply sync_global_to_guest().

---
 tools/testing/selftests/kvm/.gitignore        |   2 +-
 tools/testing/selftests/kvm/Makefile          |   5 +-
 .../kvm/{x86_64 => }/dirty_log_test.c         | 163 +++++++++---------
 3 files changed, 90 insertions(+), 80 deletions(-)
 rename tools/testing/selftests/kvm/{x86_64 => }/dirty_log_test.c (65%)
tools/testing/selftests/kvm/.gitignore +1 -1
index 9b3933fb5043..c228400cba07 100644
@@ -3,4 +3,4 @@
 /x86_64/sync_regs_test
 /x86_64/vmx_tsc_adjust_test
 /x86_64/state_test
-/x86_64/dirty_log_test
+/dirty_log_test
tools/testing/selftests/kvm/Makefile +4 -1
index d603d7865cd1..3875639025fe 100644
@@ -5,13 +5,16 @@ UNAME_M := $(shell uname -m)
 
 LIBKVM = lib/assert.c lib/elf.c lib/io.c lib/kvm_util.c lib/ucall.c lib/sparsebit.c
 LIBKVM_x86_64 = lib/x86_64/processor.c lib/x86_64/vmx.c
+LIBKVM_aarch64 = lib/aarch64/processor.c
 
 TEST_GEN_PROGS_x86_64 = x86_64/set_sregs_test
 TEST_GEN_PROGS_x86_64 += x86_64/sync_regs_test
 TEST_GEN_PROGS_x86_64 += x86_64/vmx_tsc_adjust_test
 TEST_GEN_PROGS_x86_64 += x86_64/cr4_cpuid_sync_test
 TEST_GEN_PROGS_x86_64 += x86_64/state_test
-TEST_GEN_PROGS_x86_64 += x86_64/dirty_log_test
+TEST_GEN_PROGS_x86_64 += dirty_log_test
+
+TEST_GEN_PROGS_aarch64 += dirty_log_test
 
 TEST_GEN_PROGS += $(TEST_GEN_PROGS_$(UNAME_M))
 LIBKVM += $(LIBKVM_$(UNAME_M))
tools/testing/selftests/kvm/dirty_log_test.c renamed from tools/testing/selftests/kvm/x86_64/dirty_log_test.c +85 -78

287 lines not on this page · raw message

Signature · 1 line
2.17.1
#10 lore raw

PATCH 10/13 kvm: selftests: introduce new VM mode for 64K pages

In reply to Andrew Jones

Rename VM_MODE_FLAT48PG to be more descriptive of its config and add a new config that has the same parameters, except with 64K pages.

---
 .../testing/selftests/kvm/include/kvm_util.h  |  7 ++-
 .../selftests/kvm/lib/aarch64/processor.c     |  8 +++-
 tools/testing/selftests/kvm/lib/kvm_util.c    | 47 ++++++++++++-------
 .../selftests/kvm/lib/kvm_util_internal.h     |  1 +
 .../selftests/kvm/lib/x86_64/processor.c      | 10 ++--
 5 files changed, 48 insertions(+), 25 deletions(-)
tools/testing/selftests/kvm/include/kvm_util.h +6 -1
index 752715a78a9b..6e3dff9d9d94 100644
@@ -34,9 +34,14 @@ typedef uint64_t vm_vaddr_t; /* Virtual Machine (Guest) virtual address */
 #define DEFAULT_STACK_PGS		5
 
 enum vm_guest_mode {
-	VM_MODE_FLAT48PG,
+	VM_MODE_P52V48_4K,
+	VM_MODE_P52V48_64K,
+	NUM_VM_MODES,
 };
 
+#define vm_guest_mode_string(m) vm_guest_mode_string[m]
+extern const char * const vm_guest_mode_string[];
+
 enum vm_mem_backing_src_type {
 	VM_MEM_SRC_ANONYMOUS,
 	VM_MEM_SRC_ANONYMOUS_THP,
tools/testing/selftests/kvm/lib/aarch64/processor.c +6 -2
index 871fe2173679..b1dfc0d4b68e 100644
@@ -226,7 +226,7 @@ struct kvm_vm *vm_create_default(uint32_t vcpuid, uint64_t extra_mem_pages,
 	uint64_t extra_pg_pages = (extra_mem_pages / ptrs_per_4k_pte) * 2;
 	struct kvm_vm *vm;
 
-	vm = vm_create(VM_MODE_FLAT48PG, DEFAULT_GUEST_PHY_PAGES + extra_pg_pages, O_RDWR);
+	vm = vm_create(VM_MODE_P52V48_4K, DEFAULT_GUEST_PHY_PAGES + extra_pg_pages, O_RDWR);
 
 	kvm_vm_elf_load(vm, program_invocation_name, 0, 0);
 	vm_vcpu_add_default(vm, vcpuid, guest_code);
@@ -267,10 +267,14 @@ void vcpu_setup(struct kvm_vm *vm, int vcpuid, int pgd_memslot, int gdt_memslot)
 	get_reg(vm, vcpuid, ARM64_SYS_REG(TCR_EL1), &tcr_el1);
 
 	switch (vm->mode) {
-	case VM_MODE_FLAT48PG:
+	case VM_MODE_P52V48_4K:
 		tcr_el1 |= 0ul << 14; /* TG0 = 4KB */
 		tcr_el1 |= 6ul << 32; /* IPS = 52 bits */
 		break;
+	case VM_MODE_P52V48_64K:
+		tcr_el1 |= 1ul << 14; /* TG0 = 64KB */
+		tcr_el1 |= 6ul << 32; /* IPS = 52 bits */
+		break;
 	default:
 		TEST_ASSERT(false, "Unknown guest mode, mode: 0x%x", vm->mode);
 	}
tools/testing/selftests/kvm/lib/kvm_util.c +30 -17

90 lines not on this page · raw message

tools/testing/selftests/kvm/lib/kvm_util_internal.h +1 -0
index 813145f3e4b9..5e05fb98dc62 100644
@@ -47,6 +47,7 @@ struct kvm_vm {
 	int kvm_fd;
 	int fd;
 	unsigned int pgtable_levels;
+	unsigned int pa_bits;
 	unsigned int va_bits;
 	unsigned int page_size;
 	unsigned int page_shift;
tools/testing/selftests/kvm/lib/x86_64/processor.c +5 -5
index d96b5f9cc344..7092d1389dda 100644
@@ -231,7 +231,7 @@ void virt_pgd_alloc(struct kvm_vm *vm, uint32_t pgd_memslot)
 {
 	int rc;
 
-	TEST_ASSERT(vm->mode == VM_MODE_FLAT48PG, "Attempt to use "
+	TEST_ASSERT(vm->mode == VM_MODE_P52V48_4K, "Attempt to use "
 		"unknown or unsupported guest mode, mode: 0x%x", vm->mode);
 
 	/* If needed, create page map l4 table. */
@@ -264,7 +264,7 @@ void virt_pg_map(struct kvm_vm *vm, uint64_t vaddr, uint64_t paddr,
 	uint16_t index[4];
 	struct pageMapL4Entry *pml4e;
 
-	TEST_ASSERT(vm->mode == VM_MODE_FLAT48PG, "Attempt to use "
+	TEST_ASSERT(vm->mode == VM_MODE_P52V48_4K, "Attempt to use "
 		"unknown or unsupported guest mode, mode: 0x%x", vm->mode);
 
 	TEST_ASSERT((vaddr % vm->page_size) == 0,
@@ -551,7 +551,7 @@ vm_paddr_t addr_gva2gpa(struct kvm_vm *vm, vm_vaddr_t gva)
 	struct pageTableEntry *pte;
 	void *hva;
 
-	TEST_ASSERT(vm->mode == VM_MODE_FLAT48PG, "Attempt to use "
+	TEST_ASSERT(vm->mode == VM_MODE_P52V48_4K, "Attempt to use "
 		"unknown or unsupported guest mode, mode: 0x%x", vm->mode);
 
 	index[0] = (gva >> 12) & 0x1ffu;
@@ -624,7 +624,7 @@ void vcpu_setup(struct kvm_vm *vm, int vcpuid, int pgd_memslot, int gdt_memslot)
 	kvm_setup_gdt(vm, &sregs.gdt, gdt_memslot, pgd_memslot);
 
 	switch (vm->mode) {
-	case VM_MODE_FLAT48PG:
+	case VM_MODE_P52V48_4K:
 		sregs.cr0 = X86_CR0_PE | X86_CR0_NE | X86_CR0_PG;
 		sregs.cr4 |= X86_CR4_PAE | X86_CR4_OSFXSR;
 		sregs.efer |= (EFER_LME | EFER_LMA | EFER_NX);
@@ -873,7 +873,7 @@ struct kvm_vm *vm_create_default(uint32_t vcpuid, uint64_t extra_mem_pages,
 	uint64_t extra_pg_pages = extra_mem_pages / 512 * 2;
 
 	/* Create VM */
-	vm = vm_create(VM_MODE_FLAT48PG,
+	vm = vm_create(VM_MODE_P52V48_4K,
 		       DEFAULT_GUEST_PHY_PAGES + extra_pg_pages,
 		       O_RDWR);
 
Signature · 1 line
2.17.1
#11 lore raw

PATCH 11/13 kvm: selftests: dirty_log_test: also test 64K pages on aarch64

In reply to Andrew Jones
---
 tools/testing/selftests/kvm/dirty_log_test.c | 183 ++++++++++++++-----
 1 file changed, 137 insertions(+), 46 deletions(-)
tools/testing/selftests/kvm/dirty_log_test.c +137 -46
index d90f1091f687..3afc7d607a2e 100644
@@ -5,6 +5,8 @@
  * Copyright (C) 2018, Red Hat, Inc.
  */
 
+#define _GNU_SOURCE /* for program_invocation_name */
+
 #include <stdio.h>
 #include <stdlib.h>
 #include <unistd.h>
@@ -30,9 +32,6 @@
  */
 #define TEST_MEM_OFFSET			(1ul << 30) /* 1G */
 
-/* Size of the testing memory slot */
-#define TEST_MEM_PAGES			(1ul << 18) /* 1G for 4K pages */
-
 /* How many pages to dirty for each guest loop */
 #define TEST_PAGES_PER_LOOP		1024
 
@@ -50,6 +49,7 @@
  */
 static uint64_t host_page_size;
 static uint64_t guest_page_size;
+static uint64_t guest_num_pages;
 static uint64_t random_array[TEST_PAGES_PER_LOOP];
 static uint64_t iteration;
 
@@ -64,7 +64,7 @@ static void guest_code(void)
 	while (true) {
 		for (i = 0; i < TEST_PAGES_PER_LOOP; i++) {
 			uint64_t addr = TEST_MEM_OFFSET;
-			addr += (READ_ONCE(random_array[i]) % TEST_MEM_PAGES)
+			addr += (READ_ONCE(random_array[i]) % guest_num_pages)
 				* guest_page_size;
 			addr &= ~(host_page_size - 1);
 			*(uint64_t *)addr = READ_ONCE(iteration);
@@ -141,8 +141,10 @@ static void vm_dirty_log_verify(unsigned long *bmap)
 {
 	uint64_t page;
 	uint64_t *value_ptr;
+	uint64_t step = host_page_size >= guest_page_size ? 1 :
+				guest_page_size / host_page_size;
 
-	for (page = 0; page < host_num_pages; page++) {
+	for (page = 0; page < host_num_pages; page += step) {
 		value_ptr = host_test_mem + page * host_page_size;
 
 		/* If this is a special page that we were tracking... */
@@ -203,71 +205,64 @@ static void vm_dirty_log_verify(unsigned long *bmap)
 	}
 }
 
-static void help(char *name)
+static struct kvm_vm *create_vm(enum vm_guest_mode mode, uint32_t vcpuid,
+				uint64_t extra_mem_pages, void *guest_code)
 {
-	puts("");
-	printf("usage: %s [-i iterations] [-I interval] [-h]\n", name);
-	puts("");
-	printf(" -i: specify iteration counts (default: %"PRIu64")\n",
-	       TEST_HOST_LOOP_N);
-	printf(" -I: specify interval in ms (default: %"PRIu64" ms)\n",
-	       TEST_HOST_LOOP_INTERVAL);
-	puts("");
-	exit(0);
+	struct kvm_vm *vm;
+	uint64_t extra_pg_pages = extra_mem_pages / 512 * 2;
+
+	vm = vm_create(mode, DEFAULT_GUEST_PHY_PAGES + extra_pg_pages, O_RDWR);
+	kvm_vm_elf_load(vm, program_invocation_name, 0, 0);
+#ifdef __x86_64__
+	vm_create_irqchip(vm);
+#endif
+	vm_vcpu_add_default(vm, vcpuid, guest_code);
+	return vm;
 }
 
-int main(int argc, char *argv[])
+static void run_test(enum vm_guest_mode mode, unsigned long iterations,
+		     unsigned long interval)
 {
+	unsigned int guest_page_shift;
 	pthread_t vcpu_thread;
 	struct kvm_vm *vm;
-	unsigned long iterations = TEST_HOST_LOOP_N;
-	unsigned long interval = TEST_HOST_LOOP_INTERVAL;
 	unsigned long *bmap;
-	int opt;
 
-	while ((opt = getopt(argc, argv, "hi:I:")) != -1) {
-		switch (opt) {
-		case 'i':
-			iterations = strtol(optarg, NULL, 10);
-			break;
-		case 'I':
-			interval = strtol(optarg, NULL, 10);
-			break;
-		case 'h':
-		default:
-			help(argv[0]);
-			break;
-		}
+	switch (mode) {
+	case VM_MODE_P52V48_4K:
+		guest_page_shift = 12;
+		break;
+	case VM_MODE_P52V48_64K:
+		guest_page_shift = 16;
+		break;
+	default:
+		TEST_ASSERT(false, "Unknown guest mode, mode: 0x%x", mode);
 	}
 
-	TEST_ASSERT(iterations > 2, "Iterations must be greater than two");
-	TEST_ASSERT(interval > 0, "Interval must be greater than zero");
-
-	DEBUG("Test iterations: %"PRIu64", interval: %"PRIu64" (ms)\n",
-	      iterations, interval);
-
-	srandom(time(0));
+	DEBUG("Testing guest mode: %s\n", vm_guest_mode_string(mode));
 
-	guest_page_size = 4096;
+	guest_page_size = (1ul << guest_page_shift);
+	/* 1G of guest page sized pages */
+	guest_num_pages = (1ul << (30 - guest_page_shift));
 	host_page_size = getpagesize();
-	host_num_pages = (TEST_MEM_PAGES * guest_page_size) / host_page_size +
-			 !!((TEST_MEM_PAGES * guest_page_size) % host_page_size);
+	host_num_pages = (guest_num_pages * guest_page_size) / host_page_size +
+			 !!((guest_num_pages * guest_page_size) % host_page_size);
 
 	bmap = bitmap_alloc(host_num_pages);
 	host_bmap_track = bitmap_alloc(host_num_pages);
 
-	vm = vm_create_default(VCPU_ID, TEST_MEM_PAGES, guest_code);
+	vm = create_vm(mode, VCPU_ID, guest_num_pages, guest_code);
 
 	/* Add an extra memory slot for testing dirty logging */
 	vm_userspace_mem_region_add(vm, VM_MEM_SRC_ANONYMOUS,
 				    TEST_MEM_OFFSET,
 				    TEST_MEM_SLOT_INDEX,
-				    TEST_MEM_PAGES,
+				    guest_num_pages,
 				    KVM_MEM_LOG_DIRTY_PAGES);
 
 	/* Do 1:1 mapping for the dirty track memory slot */
 	virt_map(vm, TEST_MEM_OFFSET, TEST_MEM_OFFSET,
-		 TEST_MEM_PAGES * guest_page_size, 0);
+		 guest_num_pages * guest_page_size, 0);
 
 	/* Cache the HVA pointer of the region */
 	host_test_mem = addr_gpa2hva(vm, (vm_paddr_t)TEST_MEM_OFFSET);
@@ -279,13 +274,18 @@ int main(int argc, char *argv[])
 	ucall_init(vm, UCALL_MMIO, NULL);
 #endif
 
-	/* Tell the guest about the page sizes */
+	/* Export the shared variables to the guest */
 	sync_global_to_guest(vm, host_page_size);
 	sync_global_to_guest(vm, guest_page_size);
+	sync_global_to_guest(vm, guest_num_pages);
 
 	/* Start the iterations */
 	iteration = 1;
 	sync_global_to_guest(vm, iteration);
+	host_quit = false;
+	host_dirty_count = 0;
+	host_clear_count = 0;
+	host_track_next_count = 0;
 
 	pthread_create(&vcpu_thread, NULL, vcpu_worker, vm);
 
@@ -310,6 +310,97 @@ int main(int argc, char *argv[])
 	free(host_bmap_track);
 	ucall_uninit(vm);
 	kvm_vm_free(vm);
+}
+
+static struct vm_guest_modes {
+	enum vm_guest_mode mode;
+	bool supported;
+	bool enabled;
+} vm_guest_modes[NUM_VM_MODES] = {
+	{ VM_MODE_P52V48_4K,	1, 1, },
+#ifdef __aarch64__
+	{ VM_MODE_P52V48_64K,	1, 1, },
+#else
+	{ VM_MODE_P52V48_64K,	0, 0, },
+#endif
+};
+
+static void help(char *name)
+{
+	int i;
+
+	puts("");
+	printf("usage: %s [-h] [-i iterations] [-I interval] [-m mode]\n", name);
+	puts("");
+	printf(" -i: specify iteration counts (default: %"PRIu64")\n",
+	       TEST_HOST_LOOP_N);
+	printf(" -I: specify interval in ms (default: %"PRIu64" ms)\n",
+	       TEST_HOST_LOOP_INTERVAL);
+	printf(" -m: specify the guest mode ID to test "
+	       "(default: test all supported modes)\n"
+	       "     This option may be used multiple times.\n"
+	       "     Guest mode IDs:\n");
+	for (i = 0; i < NUM_VM_MODES; ++i) {
+		printf("         %d:    %s%s\n",
+		       vm_guest_modes[i].mode,
+		       vm_guest_mode_string(vm_guest_modes[i].mode),
+		       vm_guest_modes[i].supported ? " (supported)" : "");
+	}
+	puts("");
+	exit(0);
+}
+
+int main(int argc, char *argv[])
+{
+	unsigned long iterations = TEST_HOST_LOOP_N;
+	unsigned long interval = TEST_HOST_LOOP_INTERVAL;
+	bool mode_selected = false;
+	unsigned int mode;
+	int opt, i;
+
+	while ((opt = getopt(argc, argv, "hi:I:m:")) != -1) {
+		switch (opt) {
+		case 'i':
+			iterations = strtol(optarg, NULL, 10);
+			break;
+		case 'I':
+			interval = strtol(optarg, NULL, 10);
+			break;
+		case 'm':
+			if (!mode_selected) {
+				for (i = 0; i < NUM_VM_MODES; ++i)
+					vm_guest_modes[i].enabled = 0;
+				mode_selected = true;
+			}
+			mode = strtoul(optarg, NULL, 10);
+			TEST_ASSERT(mode < NUM_VM_MODES,
+				    "Guest mode ID %d too big", mode);
+			vm_guest_modes[mode].enabled = 1;
+			break;
+		case 'h':
+		default:
+			help(argv[0]);
+			break;
+		}
+	}
+
+	TEST_ASSERT(iterations > 2, "Iterations must be greater than two");
+	TEST_ASSERT(interval > 0, "Interval must be greater than zero");
+
+	DEBUG("Test iterations: %"PRIu64", interval: %"PRIu64" (ms)\n",
+	      iterations, interval);
+
+	srandom(time(0));
+
+	for (i = 0; i < NUM_VM_MODES; ++i) {
+		if (!vm_guest_modes[i].enabled)
+			continue;
+		TEST_ASSERT(vm_guest_modes[i].supported,
+			    "Guest mode ID %d (%s) not supported.",
+			    vm_guest_modes[i].mode,
+			    vm_guest_mode_string(vm_guest_modes[i].mode));
+		run_test(vm_guest_modes[i].mode, iterations, interval);
+	}
 
 	return 0;
 }
Signature · 1 line
2.17.1
#12 lore raw

PATCH 12/13 kvm: selftests: stop lying to aarch64 tests about PA-bits

In reply to Andrew Jones

Let's add the 40 PA-bit versions of the VM modes, that AArch64 should have been using, so we can extend the dirty log test without breaking things.

---
 tools/testing/selftests/kvm/dirty_log_test.c     | 13 ++++++++++---
 tools/testing/selftests/kvm/include/kvm_util.h   |  2 ++
 .../selftests/kvm/lib/aarch64/processor.c        |  8 ++++++++
 tools/testing/selftests/kvm/lib/kvm_util.c       | 16 ++++++++++++++++
 4 files changed, 36 insertions(+), 3 deletions(-)
tools/testing/selftests/kvm/dirty_log_test.c +10 -3
index 3afc7d607a2e..61396882ad4e 100644
@@ -230,9 +230,11 @@ static void run_test(enum vm_guest_mode mode, unsigned long iterations,
 
 	switch (mode) {
 	case VM_MODE_P52V48_4K:
+	case VM_MODE_P40V48_4K:
 		guest_page_shift = 12;
 		break;
 	case VM_MODE_P52V48_64K:
+	case VM_MODE_P40V48_64K:
 		guest_page_shift = 16;
 		break;
 	default:
@@ -317,11 +319,16 @@ static struct vm_guest_modes {
 	bool supported;
 	bool enabled;
 } vm_guest_modes[NUM_VM_MODES] = {
+#if defined(__x86_64__)
 	{ VM_MODE_P52V48_4K,	1, 1, },
-#ifdef __aarch64__
-	{ VM_MODE_P52V48_64K,	1, 1, },
-#else
 	{ VM_MODE_P52V48_64K,	0, 0, },
+	{ VM_MODE_P40V48_4K,	0, 0, },
+	{ VM_MODE_P40V48_64K,	0, 0, },
+#elif defined(__aarch64__)
+	{ VM_MODE_P52V48_4K,	0, 0, },
+	{ VM_MODE_P52V48_64K,	0, 0, },
+	{ VM_MODE_P40V48_4K,	1, 1, },
+	{ VM_MODE_P40V48_64K,	1, 1, },
 #endif
 };
 
tools/testing/selftests/kvm/include/kvm_util.h +2 -0
index 6e3dff9d9d94..d76431322a30 100644
@@ -36,6 +36,8 @@ typedef uint64_t vm_vaddr_t; /* Virtual Machine (Guest) virtual address */
 enum vm_guest_mode {
 	VM_MODE_P52V48_4K,
 	VM_MODE_P52V48_64K,
+	VM_MODE_P40V48_4K,
+	VM_MODE_P40V48_64K,
 	NUM_VM_MODES,
 };
 
tools/testing/selftests/kvm/lib/aarch64/processor.c +8 -0
index b1dfc0d4b68e..b6022e2f116e 100644
@@ -275,6 +275,14 @@ void vcpu_setup(struct kvm_vm *vm, int vcpuid, int pgd_memslot, int gdt_memslot)
 		tcr_el1 |= 1ul << 14; /* TG0 = 64KB */
 		tcr_el1 |= 6ul << 32; /* IPS = 52 bits */
 		break;
+	case VM_MODE_P40V48_4K:
+		tcr_el1 |= 0ul << 14; /* TG0 = 4KB */
+		tcr_el1 |= 2ul << 32; /* IPS = 40 bits */
+		break;
+	case VM_MODE_P40V48_64K:
+		tcr_el1 |= 1ul << 14; /* TG0 = 64KB */
+		tcr_el1 |= 2ul << 32; /* IPS = 40 bits */
+		break;
 	default:
 		TEST_ASSERT(false, "Unknown guest mode, mode: 0x%x", vm->mode);
 	}
tools/testing/selftests/kvm/lib/kvm_util.c +16 -0
index 4d3bb515de17..dd8c8ed087b6 100644
@@ -76,6 +76,8 @@ static void vm_open(struct kvm_vm *vm, int perm)
 const char * const vm_guest_mode_string[] = {
 	"PA-bits:52, VA-bits:48, 4K pages",
 	"PA-bits:52, VA-bits:48, 64K pages",
+	"PA-bits:40, VA-bits:48, 4K pages",
+	"PA-bits:40, VA-bits:48, 64K pages",
 };
 
 /*
@@ -124,6 +126,20 @@ struct kvm_vm *vm_create(enum vm_guest_mode mode, uint64_t phy_pages, int perm)
 		vm->page_size = 0x10000;
 		vm->page_shift = 16;
 		break;
+	case VM_MODE_P40V48_4K:
+		vm->pgtable_levels = 4;
+		vm->pa_bits = 40;
+		vm->va_bits = 48;
+		vm->page_size = 0x1000;
+		vm->page_shift = 12;
+		break;
+	case VM_MODE_P40V48_64K:
+		vm->pgtable_levels = 3;
+		vm->pa_bits = 40;
+		vm->va_bits = 48;
+		vm->page_size = 0x10000;
+		vm->page_shift = 16;
+		break;
 	default:
 		TEST_ASSERT(false, "Unknown guest mode, mode: 0x%x", mode);
 	}
Signature · 1 line
2.17.1
#13 lore raw

PATCH 13/13 kvm: selftests: support high GPAs in dirty_log_test

In reply to Andrew Jones
---
 tools/testing/selftests/kvm/dirty_log_test.c | 65 +++++++++++++++-----
 1 file changed, 50 insertions(+), 15 deletions(-)
tools/testing/selftests/kvm/dirty_log_test.c +50 -15
index 61396882ad4e..d59820cc2d39 100644
@@ -26,11 +26,8 @@
 /* The memory slot index to track dirty pages */
 #define TEST_MEM_SLOT_INDEX		1
 
-/*
- * GPA offset of the testing memory slot. Must be bigger than the
- * default vm mem slot, which is DEFAULT_GUEST_PHY_PAGES.
- */
-#define TEST_MEM_OFFSET			(1ul << 30) /* 1G */
+/* Default guest test memory offset, 1G */
+#define DEFAULT_GUEST_TEST_MEM		0x40000000
 
 /* How many pages to dirty for each guest loop */
 #define TEST_PAGES_PER_LOOP		1024
@@ -53,6 +50,12 @@ static uint64_t guest_num_pages;
 static uint64_t random_array[TEST_PAGES_PER_LOOP];
 static uint64_t iteration;
 
+/*
+ * GPA offset of the testing memory slot. Must be bigger than
+ * DEFAULT_GUEST_PHY_PAGES.
+ */
+static uint64_t guest_test_mem = DEFAULT_GUEST_TEST_MEM;
+
 /*
  * Continuously write to the first 8 bytes of a random pages within
  * the testing memory region.
@@ -63,7 +66,7 @@ static void guest_code(void)
 
 	while (true) {
 		for (i = 0; i < TEST_PAGES_PER_LOOP; i++) {
-			uint64_t addr = TEST_MEM_OFFSET;
+			uint64_t addr = guest_test_mem;
 			addr += (READ_ONCE(random_array[i]) % guest_num_pages)
 				* guest_page_size;
 			addr &= ~(host_page_size - 1);
@@ -221,20 +224,29 @@ static struct kvm_vm *create_vm(enum vm_guest_mode mode, uint32_t vcpuid,
 }
 
 static void run_test(enum vm_guest_mode mode, unsigned long iterations,
-		     unsigned long interval)
+		     unsigned long interval, bool top_offset)
 {
-	unsigned int guest_page_shift;
+	unsigned int guest_pa_bits, guest_page_shift;
 	pthread_t vcpu_thread;
 	struct kvm_vm *vm;
+	uint64_t max_gfn;
 	unsigned long *bmap;
 
 	switch (mode) {
 	case VM_MODE_P52V48_4K:
-	case VM_MODE_P40V48_4K:
+		guest_pa_bits = 52;
 		guest_page_shift = 12;
 		break;
 	case VM_MODE_P52V48_64K:
+		guest_pa_bits = 52;
+		guest_page_shift = 16;
+		break;
+	case VM_MODE_P40V48_4K:
+		guest_pa_bits = 40;
+		guest_page_shift = 12;
+		break;
 	case VM_MODE_P40V48_64K:
+		guest_pa_bits = 40;
 		guest_page_shift = 16;
 		break;
 	default:
@@ -243,6 +255,7 @@ static void run_test(enum vm_guest_mode mode, unsigned long iterations,
 
 	DEBUG("Testing guest mode: %s\n", vm_guest_mode_string(mode));
 
+	max_gfn = (1ul << (guest_pa_bits - guest_page_shift)) - 1;
 	guest_page_size = (1ul << guest_page_shift);
 	/* 1G of guest page sized pages */
 	guest_num_pages = (1ul << (30 - guest_page_shift));
@@ -250,6 +263,13 @@ static void run_test(enum vm_guest_mode mode, unsigned long iterations,
 	host_num_pages = (guest_num_pages * guest_page_size) / host_page_size +
 			 !!((guest_num_pages * guest_page_size) % host_page_size);
 
+	if (top_offset) {
+		guest_test_mem = (max_gfn - guest_num_pages) * guest_page_size;
+		guest_test_mem &= ~(host_page_size - 1);
+	}
+
+	DEBUG("guest test mem offset: 0x%lx\n", guest_test_mem);
+
 	bmap = bitmap_alloc(host_num_pages);
 	host_bmap_track = bitmap_alloc(host_num_pages);
 
@@ -257,17 +277,17 @@ static void run_test(enum vm_guest_mode mode, unsigned long iterations,
 
 	/* Add an extra memory slot for testing dirty logging */
 	vm_userspace_mem_region_add(vm, VM_MEM_SRC_ANONYMOUS,
-				    TEST_MEM_OFFSET,
+				    guest_test_mem,
 				    TEST_MEM_SLOT_INDEX,
 				    guest_num_pages,
 				    KVM_MEM_LOG_DIRTY_PAGES);
 
 	/* Do 1:1 mapping for the dirty track memory slot */
-	virt_map(vm, TEST_MEM_OFFSET, TEST_MEM_OFFSET,
+	virt_map(vm, guest_test_mem, guest_test_mem,
 		 guest_num_pages * guest_page_size, 0);
 
 	/* Cache the HVA pointer of the region */
-	host_test_mem = addr_gpa2hva(vm, (vm_paddr_t)TEST_MEM_OFFSET);
+	host_test_mem = addr_gpa2hva(vm, (vm_paddr_t)guest_test_mem);
 
 #ifdef __x86_64__
 	vcpu_set_cpuid(vm, VCPU_ID, kvm_get_supported_cpuid());
@@ -279,6 +299,7 @@ static void run_test(enum vm_guest_mode mode, unsigned long iterations,
 	/* Export the shared variables to the guest */
 	sync_global_to_guest(vm, host_page_size);
 	sync_global_to_guest(vm, guest_page_size);
+	sync_global_to_guest(vm, guest_test_mem);
 	sync_global_to_guest(vm, guest_num_pages);
 
 	/* Start the iterations */
@@ -337,12 +358,17 @@ static void help(char *name)
 	int i;
 
 	puts("");
-	printf("usage: %s [-h] [-i iterations] [-I interval] [-m mode]\n", name);
+	printf("usage: %s [-h] [-i iterations] [-I interval] "
+	       "[-o offset] [-t] [-m mode]\n", name);
 	puts("");
 	printf(" -i: specify iteration counts (default: %"PRIu64")\n",
 	       TEST_HOST_LOOP_N);
 	printf(" -I: specify interval in ms (default: %"PRIu64" ms)\n",
 	       TEST_HOST_LOOP_INTERVAL);
+	printf(" -o: guest test memory offset (default: 0x%lx)\n",
+	       DEFAULT_GUEST_TEST_MEM);
+	printf(" -t: map guest test memory at the top of the allowed "
+	       "physical address range\n");
 	printf(" -m: specify the guest mode ID to test "
 	       "(default: test all supported modes)\n"
 	       "     This option may be used multiple times.\n"
@@ -362,10 +388,11 @@ int main(int argc, char *argv[])
 	unsigned long iterations = TEST_HOST_LOOP_N;
 	unsigned long interval = TEST_HOST_LOOP_INTERVAL;
 	bool mode_selected = false;
+	bool top_offset = false;
 	unsigned int mode;
 	int opt, i;
 
-	while ((opt = getopt(argc, argv, "hi:I:m:")) != -1) {
+	while ((opt = getopt(argc, argv, "hi:I:o:tm:")) != -1) {
 		switch (opt) {
 		case 'i':
 			iterations = strtol(optarg, NULL, 10);
@@ -373,6 +400,12 @@ int main(int argc, char *argv[])
 		case 'I':
 			interval = strtol(optarg, NULL, 10);
 			break;
+		case 'o':
+			guest_test_mem = strtoull(optarg, NULL, 0);
+			break;
+		case 't':
+			top_offset = true;
+			break;
 		case 'm':
 			if (!mode_selected) {
 				for (i = 0; i < NUM_VM_MODES; ++i)
@@ -393,6 +426,8 @@ int main(int argc, char *argv[])
 
 	TEST_ASSERT(iterations > 2, "Iterations must be greater than two");
 	TEST_ASSERT(interval > 0, "Interval must be greater than zero");
+	TEST_ASSERT(!top_offset || guest_test_mem == DEFAULT_GUEST_TEST_MEM,
+		    "Cannot use both -o [offset] and -t at the same time");
 
 	DEBUG("Test iterations: %"PRIu64", interval: %"PRIu64" (ms)\n",
 	      iterations, interval);
@@ -406,7 +441,7 @@ int main(int argc, char *argv[])
 			    "Guest mode ID %d (%s) not supported.",
 			    vm_guest_modes[i].mode,
 			    vm_guest_mode_string(vm_guest_modes[i].mode));
-		run_test(vm_guest_modes[i].mode, iterations, interval);
+		run_test(vm_guest_modes[i].mode, iterations, interval, top_offset);
 	}
 
 	return 0;
Signature · 1 line
2.17.1
#14 lore raw
In reply to Andrew Jones
Quoted 18 lines · Andrew Jones

This series provides KVM selftests that test dirty log tracking on AArch64 for both 4K and 64K guest page sizes. Additionally the framework provides an easy way to test dirty log tracking with the recently posted dynamic IPA and 52bit IPA series[1].

The series breaks down into parts as follows:

01-02: generalize guest code to host userspace exit support by
introducing "ucalls" - hypercalls to userspace
03-05: prepare common code for a new architecture 06-07: add virtual memory setup support for AArch64
08: add vcpu setup support for AArch64
09: port the dirty log test to AArch64
10-11: add 64K guest page size support for the dirty log test 12-13: prepare the dirty log test to also test > 40-bit guest
physical address setups by allowing the test memory region to be placed at the top of physical memory

Quoted 2 lines

[1] https://www.spinics.net/lists/arm-kernel/msg676819.html

Hi Suzuki,

Here's an [untested] add-on patch that should provide the means to test dirty logging with a 52-bit guest physical address space. Hopefully it'll be as easy as compiling and then running with

 $ ./dirty_log_test -t

Thanks,
drew

Quoted 1 line

From 09ea1d724551a95ef9962d357049ef21ea8c77e8 Mon Sep 17 00:00:00 2001

From: Andrew Jones <[email protected]>
Date: Wed, 19 Sep 2018 14:23:30 +0200
Subject: [PATCH] kvm: selftests: aarch64: dirty_log_test: test with 52 PA-bits

---
 tools/testing/selftests/kvm/dirty_log_test.c  | 24 ++++++++++++++++---
 .../testing/selftests/kvm/include/kvm_util.h  |  2 ++
 tools/testing/selftests/kvm/lib/kvm_util.c    | 17 +++++++++----
 .../selftests/kvm/lib/kvm_util_internal.h     |  1 +
 4 files changed, 36 insertions(+), 8 deletions(-)
tools/testing/selftests/kvm/dirty_log_test.c +21 -3
index d59820cc2d39..c11c76e09766 100644
@@ -82,6 +82,7 @@ static void guest_code(void)
 static bool host_quit;
 
 /* Points to the test VM memory region on which we track dirty logs */
+static uint8_t host_ipa_limit;
 static void *host_test_mem;
 static uint64_t host_num_pages;
 
@@ -209,12 +210,14 @@ static void vm_dirty_log_verify(unsigned long *bmap)
 }
 
 static struct kvm_vm *create_vm(enum vm_guest_mode mode, uint32_t vcpuid,
-				uint64_t extra_mem_pages, void *guest_code)
+				uint64_t extra_mem_pages, void *guest_code,
+				unsigned long type)
 {
 	struct kvm_vm *vm;
 	uint64_t extra_pg_pages = extra_mem_pages / 512 * 2;
 
-	vm = vm_create(mode, DEFAULT_GUEST_PHY_PAGES + extra_pg_pages, O_RDWR);
+	vm = _vm_create(mode, DEFAULT_GUEST_PHY_PAGES + extra_pg_pages,
+			O_RDWR, type);
 	kvm_vm_elf_load(vm, program_invocation_name, 0, 0);
 #ifdef __x86_64__
 	vm_create_irqchip(vm);
@@ -231,15 +234,22 @@ static void run_test(enum vm_guest_mode mode, unsigned long iterations,
 	struct kvm_vm *vm;
 	uint64_t max_gfn;
 	unsigned long *bmap;
+	unsigned long type = 0;
 
 	switch (mode) {
 	case VM_MODE_P52V48_4K:
 		guest_pa_bits = 52;
 		guest_page_shift = 12;
+#ifdef __aarch64__
+		type = KVM_VM_TYPE_ARM_IPA_SIZE(52);
+#endif
 		break;
 	case VM_MODE_P52V48_64K:
 		guest_pa_bits = 52;
 		guest_page_shift = 16;
+#ifdef __aarch64__
+		type = KVM_VM_TYPE_ARM_IPA_SIZE(52);
+#endif
 		break;
 	case VM_MODE_P40V48_4K:
 		guest_pa_bits = 40;
@@ -273,7 +283,7 @@ static void run_test(enum vm_guest_mode mode, unsigned long iterations,
 	bmap = bitmap_alloc(host_num_pages);
 	host_bmap_track = bitmap_alloc(host_num_pages);
 
-	vm = create_vm(mode, VCPU_ID, guest_num_pages, guest_code);
+	vm = create_vm(mode, VCPU_ID, guest_num_pages, guest_code, type);
 
 	/* Add an extra memory slot for testing dirty logging */
 	vm_userspace_mem_region_add(vm, VM_MEM_SRC_ANONYMOUS,
@@ -392,6 +402,14 @@ int main(int argc, char *argv[])
 	unsigned int mode;
 	int opt, i;
 
+#ifdef __aarch64__
+	host_ipa_limit = kvm_check_cap(KVM_CAP_ARM_VM_IPA_SIZE);
+	if (host_ipa_limit == 52) {
+		vm_guest_modes[VM_MODE_P52V48_4K].supported = 1;
+		vm_guest_modes[VM_MODE_P52V48_64K].supported = 1;
+	}
+#endif
+
 	while ((opt = getopt(argc, argv, "hi:I:o:tm:")) != -1) {
 		switch (opt) {
 		case 'i':
tools/testing/selftests/kvm/include/kvm_util.h +2 -0
index d76431322a30..5202fce337e3 100644
@@ -53,6 +53,8 @@ enum vm_mem_backing_src_type {
 int kvm_check_cap(long cap);
 
 struct kvm_vm *vm_create(enum vm_guest_mode mode, uint64_t phy_pages, int perm);
+struct kvm_vm *_vm_create(enum vm_guest_mode mode, uint64_t phy_pages,
+			  int perm, unsigned long type);
 void kvm_vm_free(struct kvm_vm *vmp);
 void kvm_vm_restart(struct kvm_vm *vmp, int perm);
 void kvm_vm_release(struct kvm_vm *vmp);
tools/testing/selftests/kvm/lib/kvm_util.c +12 -5
index 542336db7b4f..e1805c9e0f39 100644
@@ -62,13 +62,13 @@ int kvm_check_cap(long cap)
 	return ret;
 }
 
-static void vm_open(struct kvm_vm *vm, int perm)
+static void vm_open(struct kvm_vm *vm, int perm, unsigned long type)
 {
 	vm->kvm_fd = open(KVM_DEV_PATH, perm);
 	if (vm->kvm_fd < 0)
 		exit(KSFT_SKIP);
 
-	vm->fd = ioctl(vm->kvm_fd, KVM_CREATE_VM, NULL);
+	vm->fd = ioctl(vm->kvm_fd, KVM_CREATE_VM, type);
 	TEST_ASSERT(vm->fd >= 0, "KVM_CREATE_VM ioctl failed, "
 		"rc: %i errno: %i", vm->fd, errno);
 }
@@ -101,7 +101,8 @@ _Static_assert(sizeof(vm_guest_mode_string)/sizeof(char *) == NUM_VM_MODES,
  * descriptor to control the created VM is created with the permissions
  * given by perm (e.g. O_RDWR).
  */
-struct kvm_vm *vm_create(enum vm_guest_mode mode, uint64_t phy_pages, int perm)
+struct kvm_vm *_vm_create(enum vm_guest_mode mode, uint64_t phy_pages,
+			  int perm, unsigned long type)
 {
 	struct kvm_vm *vm;
 	int kvm_fd;
@@ -110,7 +111,8 @@ struct kvm_vm *vm_create(enum vm_guest_mode mode, uint64_t phy_pages, int perm)
 	TEST_ASSERT(vm != NULL, "Insufficent Memory");
 
 	vm->mode = mode;
-	vm_open(vm, perm);
+	vm->type = type;
+	vm_open(vm, perm, type);
 
 	/* Setup mode specific traits. */
 	switch (vm->mode) {
@@ -166,6 +168,11 @@ struct kvm_vm *vm_create(enum vm_guest_mode mode, uint64_t phy_pages, int perm)
 	return vm;
 }
 
+struct kvm_vm *vm_create(enum vm_guest_mode mode, uint64_t phy_pages, int perm)
+{
+	return _vm_create(mode, phy_pages, perm, 0);
+}
+
 /*
  * VM Restart
  *
@@ -183,7 +190,7 @@ void kvm_vm_restart(struct kvm_vm *vmp, int perm)
 {
 	struct userspace_mem_region *region;
 
-	vm_open(vmp, perm);
+	vm_open(vmp, perm, vmp->type);
 	if (vmp->has_irqchip)
 		vm_create_irqchip(vmp);
 
tools/testing/selftests/kvm/lib/kvm_util_internal.h +1 -0
index 5e05fb98dc62..51a56102a5c9 100644
@@ -44,6 +44,7 @@ struct vcpu {
 
 struct kvm_vm {
 	int mode;
+	unsigned long type;
 	int kvm_fd;
 	int fd;
 	unsigned int pgtable_levels;
Signature · 1 line
2.17.1
#15 lore raw

Hi Drew,

Quoted 22 lines · Andrew Jones
Quoted 18 lines · Andrew Jones

This series provides KVM selftests that test dirty log tracking on AArch64 for both 4K and 64K guest page sizes. Additionally the framework provides an easy way to test dirty log tracking with the recently posted dynamic IPA and 52bit IPA series[1].

The series breaks down into parts as follows:

01-02: generalize guest code to host userspace exit support by
introducing "ucalls" - hypercalls to userspace
03-05: prepare common code for a new architecture 06-07: add virtual memory setup support for AArch64
08: add vcpu setup support for AArch64
09: port the dirty log test to AArch64
10-11: add 64K guest page size support for the dirty log test 12-13: prepare the dirty log test to also test > 40-bit guest
physical address setups by allowing the test memory region to be placed at the top of physical memory

Quoted 2 lines

[1] https://www.spinics.net/lists/arm-kernel/msg676819.html

Quoted 7 lines

Hi Suzuki,

Here's an [untested] add-on patch that should provide the means to test dirty logging with a 52-bit guest physical address space. Hopefully it'll be as easy as compiling and then running with

  $ ./dirty_log_test -t

I will give this series a spin with 52 IPA and let you know.

Thanks
Suzuki

Quoted 180 lines

Thanks,
drew

From 09ea1d724551a95ef9962d357049ef21ea8c77e8 Mon Sep 17 00:00:00 2001
From: Andrew Jones <[email protected]>
Date: Wed, 19 Sep 2018 14:23:30 +0200
Subject: [PATCH] kvm: selftests: aarch64: dirty_log_test: test with 52 PA-bits

---
  tools/testing/selftests/kvm/dirty_log_test.c  | 24 ++++++++++++++++---
  .../testing/selftests/kvm/include/kvm_util.h  |  2 ++
  tools/testing/selftests/kvm/lib/kvm_util.c    | 17 +++++++++----
  .../selftests/kvm/lib/kvm_util_internal.h     |  1 +
  4 files changed, 36 insertions(+), 8 deletions(-)
tools/testing/selftests/kvm/dirty_log_test.c +21 -3
index d59820cc2d39..c11c76e09766 100644
@@ -82,6 +82,7 @@ static void guest_code(void)
  static bool host_quit;
 
  /* Points to the test VM memory region on which we track dirty logs */
+static uint8_t host_ipa_limit;
  static void *host_test_mem;
  static uint64_t host_num_pages;
 
@@ -209,12 +210,14 @@ static void vm_dirty_log_verify(unsigned long *bmap)
  }
 
  static struct kvm_vm *create_vm(enum vm_guest_mode mode, uint32_t vcpuid,
-				uint64_t extra_mem_pages, void *guest_code)
+				uint64_t extra_mem_pages, void *guest_code,
+				unsigned long type)
  {
  	struct kvm_vm *vm;
  	uint64_t extra_pg_pages = extra_mem_pages / 512 * 2;
 
-	vm = vm_create(mode, DEFAULT_GUEST_PHY_PAGES + extra_pg_pages, O_RDWR);
+	vm = _vm_create(mode, DEFAULT_GUEST_PHY_PAGES + extra_pg_pages,
+			O_RDWR, type);
  	kvm_vm_elf_load(vm, program_invocation_name, 0, 0);
  #ifdef __x86_64__
  	vm_create_irqchip(vm);
@@ -231,15 +234,22 @@ static void run_test(enum vm_guest_mode mode, unsigned long iterations,
  	struct kvm_vm *vm;
  	uint64_t max_gfn;
  	unsigned long *bmap;
+	unsigned long type = 0;
 
  	switch (mode) {
  	case VM_MODE_P52V48_4K:
  		guest_pa_bits = 52;
  		guest_page_shift = 12;
+#ifdef __aarch64__
+		type = KVM_VM_TYPE_ARM_IPA_SIZE(52);
+#endif
  		break;
  	case VM_MODE_P52V48_64K:
  		guest_pa_bits = 52;
  		guest_page_shift = 16;
+#ifdef __aarch64__
+		type = KVM_VM_TYPE_ARM_IPA_SIZE(52);
+#endif
  		break;
  	case VM_MODE_P40V48_4K:
  		guest_pa_bits = 40;
@@ -273,7 +283,7 @@ static void run_test(enum vm_guest_mode mode, unsigned long iterations,
  	bmap = bitmap_alloc(host_num_pages);
  	host_bmap_track = bitmap_alloc(host_num_pages);
 
-	vm = create_vm(mode, VCPU_ID, guest_num_pages, guest_code);
+	vm = create_vm(mode, VCPU_ID, guest_num_pages, guest_code, type);
 
  	/* Add an extra memory slot for testing dirty logging */
  	vm_userspace_mem_region_add(vm, VM_MEM_SRC_ANONYMOUS,
@@ -392,6 +402,14 @@ int main(int argc, char *argv[])
  	unsigned int mode;
  	int opt, i;
 
+#ifdef __aarch64__
+	host_ipa_limit = kvm_check_cap(KVM_CAP_ARM_VM_IPA_SIZE);
+	if (host_ipa_limit == 52) {
+		vm_guest_modes[VM_MODE_P52V48_4K].supported = 1;
+		vm_guest_modes[VM_MODE_P52V48_64K].supported = 1;
+	}
+#endif
+
  	while ((opt = getopt(argc, argv, "hi:I:o:tm:")) != -1) {
  		switch (opt) {
  		case 'i':
tools/testing/selftests/kvm/include/kvm_util.h +2 -0
index d76431322a30..5202fce337e3 100644
@@ -53,6 +53,8 @@ enum vm_mem_backing_src_type {
  int kvm_check_cap(long cap);
 
  struct kvm_vm *vm_create(enum vm_guest_mode mode, uint64_t phy_pages, int perm);
+struct kvm_vm *_vm_create(enum vm_guest_mode mode, uint64_t phy_pages,
+			  int perm, unsigned long type);
  void kvm_vm_free(struct kvm_vm *vmp);
  void kvm_vm_restart(struct kvm_vm *vmp, int perm);
  void kvm_vm_release(struct kvm_vm *vmp);
tools/testing/selftests/kvm/lib/kvm_util.c +12 -5
index 542336db7b4f..e1805c9e0f39 100644
@@ -62,13 +62,13 @@ int kvm_check_cap(long cap)
  	return ret;
  }
 
-static void vm_open(struct kvm_vm *vm, int perm)
+static void vm_open(struct kvm_vm *vm, int perm, unsigned long type)
  {
  	vm->kvm_fd = open(KVM_DEV_PATH, perm);
  	if (vm->kvm_fd < 0)
  		exit(KSFT_SKIP);
 
-	vm->fd = ioctl(vm->kvm_fd, KVM_CREATE_VM, NULL);
+	vm->fd = ioctl(vm->kvm_fd, KVM_CREATE_VM, type);
  	TEST_ASSERT(vm->fd >= 0, "KVM_CREATE_VM ioctl failed, "
  		"rc: %i errno: %i", vm->fd, errno);
  }
@@ -101,7 +101,8 @@ _Static_assert(sizeof(vm_guest_mode_string)/sizeof(char *) == NUM_VM_MODES,
   * descriptor to control the created VM is created with the permissions
   * given by perm (e.g. O_RDWR).
   */
-struct kvm_vm *vm_create(enum vm_guest_mode mode, uint64_t phy_pages, int perm)
+struct kvm_vm *_vm_create(enum vm_guest_mode mode, uint64_t phy_pages,
+			  int perm, unsigned long type)
  {
  	struct kvm_vm *vm;
  	int kvm_fd;
@@ -110,7 +111,8 @@ struct kvm_vm *vm_create(enum vm_guest_mode mode, uint64_t phy_pages, int perm)
  	TEST_ASSERT(vm != NULL, "Insufficent Memory");
 
  	vm->mode = mode;
-	vm_open(vm, perm);
+	vm->type = type;
+	vm_open(vm, perm, type);
 
  	/* Setup mode specific traits. */
  	switch (vm->mode) {
@@ -166,6 +168,11 @@ struct kvm_vm *vm_create(enum vm_guest_mode mode, uint64_t phy_pages, int perm)
  	return vm;
  }
 
+struct kvm_vm *vm_create(enum vm_guest_mode mode, uint64_t phy_pages, int perm)
+{
+	return _vm_create(mode, phy_pages, perm, 0);
+}
+
  /*
   * VM Restart
   *
@@ -183,7 +190,7 @@ void kvm_vm_restart(struct kvm_vm *vmp, int perm)
  {
  	struct userspace_mem_region *region;
 
-	vm_open(vmp, perm);
+	vm_open(vmp, perm, vmp->type);
  	if (vmp->has_irqchip)
  		vm_create_irqchip(vmp);
 
tools/testing/selftests/kvm/lib/kvm_util_internal.h +1 -0
index 5e05fb98dc62..51a56102a5c9 100644
@@ -44,6 +44,7 @@ struct vcpu {
 
  struct kvm_vm {
  	int mode;
+	unsigned long type;
  	int kvm_fd;
  	int fd;
  	unsigned int pgtable_levels;
#16 lore raw
In reply to Andrew Jones

Hi Drew,

Quoted 4 lines · Andrew Jones

This series provides KVM selftests that test dirty log tracking on AArch64 for both 4K and 64K guest page sizes. Additionally the framework provides an easy way to test dirty log tracking with the recently posted dynamic IPA and 52bit IPA series[1].

I was trying to parse the commit text of patch 2, and I realized that I don't understand the 'hypercall to userspace' thing at all, which probably means I have no idea how the selftests work overall.

I then spent a while reading various bits of documentation in the kernel tree, LWN, etc., only to realize that I don't understand how this test framework actually works.

Are the selftests modules, userspace programs, or code that is compiled with the kernel, and (somehow?) run from userspace. I thought the latter, partially based on your explanation at ELC, but then I don't understand how the "compile and run" make target works.

Can you help me paint the overall picture, or point me to the piece of documentation/presentation that explains the high-level picture, which I must have obviously missed somehow?

Thanks!

Christoffer

Quoted 73 lines

The series breaks down into parts as follows:

01-02: generalize guest code to host userspace exit support by
introducing "ucalls" - hypercalls to userspace
03-05: prepare common code for a new architecture 06-07: add virtual memory setup support for AArch64
08: add vcpu setup support for AArch64
09: port the dirty log test to AArch64
10-11: add 64K guest page size support for the dirty log test 12-13: prepare the dirty log test to also test > 40-bit guest
physical address setups by allowing the test memory region to be placed at the top of physical memory

[1] https://www.spinics.net/lists/arm-kernel/msg676819.html

Thanks,
drew

Andrew Jones (13):
kvm: selftests: vcpu_setup: set cr4.osfxsr
kvm: selftests: introduce ucall
kvm: selftests: move arch-specific files to arch-specific locations
kvm: selftests: add cscope make target
kvm: selftests: tidy up kvm_util
kvm: selftests: add vm_phy_pages_alloc
kvm: selftests: add virt mem support for aarch64
kvm: selftests: add vcpu support for aarch64
kvm: selftests: port dirty_log_test to aarch64
kvm: selftests: introduce new VM mode for 64K pages
kvm: selftests: dirty_log_test: also test 64K pages on aarch64
kvm: selftests: stop lying to aarch64 tests about PA-bits
kvm: selftests: support high GPAs in dirty_log_test

 tools/testing/selftests/kvm/.gitignore        |  11 +-
 tools/testing/selftests/kvm/Makefile          |  36 +-
 tools/testing/selftests/kvm/dirty_log_test.c  | 374 +++++++++----
 .../selftests/kvm/include/aarch64/processor.h |  55 ++
 .../testing/selftests/kvm/include/kvm_util.h  | 166 +++---
 .../testing/selftests/kvm/include/sparsebit.h |   6 +-
 .../testing/selftests/kvm/include/test_util.h |   6 +-
 .../kvm/include/{x86.h => x86_64/processor.h} |  24 +-
 .../selftests/kvm/include/{ => x86_64}/vmx.h  |   6 +-
 .../selftests/kvm/lib/aarch64/processor.c     | 311 +++++++++++
 tools/testing/selftests/kvm/lib/assert.c      |   2 +-
 tools/testing/selftests/kvm/lib/kvm_util.c    | 499 +++++++-----------
 .../selftests/kvm/lib/kvm_util_internal.h     |  33 +-
 tools/testing/selftests/kvm/lib/ucall.c       | 144 +++++
 .../kvm/lib/{x86.c => x86_64/processor.c}     | 197 ++++++-
 .../selftests/kvm/lib/{ => x86_64}/vmx.c      |   4 +-
 .../kvm/{ => x86_64}/cr4_cpuid_sync_test.c    |  14 +-
 .../kvm/{ => x86_64}/set_sregs_test.c         |   2 +-
 .../selftests/kvm/{ => x86_64}/state_test.c   |  25 +-
 .../kvm/{ => x86_64}/sync_regs_test.c         |   2 +-
 .../kvm/{ => x86_64}/vmx_tsc_adjust_test.c    |  23 +-
 21 files changed, 1329 insertions(+), 611 deletions(-)
 create mode 100644 tools/testing/selftests/kvm/include/aarch64/processor.h
 rename tools/testing/selftests/kvm/include/{x86.h => x86_64/processor.h} (98%)
 rename tools/testing/selftests/kvm/include/{ => x86_64}/vmx.h (99%)
 create mode 100644 tools/testing/selftests/kvm/lib/aarch64/processor.c
 create mode 100644 tools/testing/selftests/kvm/lib/ucall.c
 rename tools/testing/selftests/kvm/lib/{x86.c => x86_64/processor.c} (85%)
 rename tools/testing/selftests/kvm/lib/{ => x86_64}/vmx.c (99%)
 rename tools/testing/selftests/kvm/{ => x86_64}/cr4_cpuid_sync_test.c (91%)
 rename tools/testing/selftests/kvm/{ => x86_64}/set_sregs_test.c (98%)
 rename tools/testing/selftests/kvm/{ => x86_64}/state_test.c (90%)
 rename tools/testing/selftests/kvm/{ => x86_64}/sync_regs_test.c (99%)
 rename tools/testing/selftests/kvm/{ => x86_64}/vmx_tsc_adjust_test.c (91%)
Signature · 1 line
2.17.1
#17 lore raw

Hi Christoffer,

Thanks for your interest in these tests. There isn't any documentation that I know of, but it's a good idea to have some. I'll write something up soon. I'll also try to answer your questions now.

Quoted 11 lines · Christoffer Dall

Hi Drew,

Quoted 4 lines · Andrew Jones

This series provides KVM selftests that test dirty log tracking on AArch64 for both 4K and 64K guest page sizes. Additionally the framework provides an easy way to test dirty log tracking with the recently posted dynamic IPA and 52bit IPA series[1].

I was trying to parse the commit text of patch 2, and I realized that I don't understand the 'hypercall to userspace' thing at all, which probably means I have no idea how the selftests work overall.

There are three parts to a kvm selftest: 1) the test code which runs in host userspace and _is_ the kvm userspace used with kvm, 2) the vcpu thread code which executes KVM_RUN for the guest code and possibly also some host userspace test code, and 3) the guest code, which is naturally run in the vcpu thread, but in guest mode.

The need for a "ucall" arises for 2's "possibly also some host userspace test code". In that case the guest code needs to invoke an exit from guest mode, not just to kvm, but all the way to kvm userspace. For AArch64, as you know, this can be done with an MMIO access. The reason patch 2 generalizes the concept is because for x86 this can and is done with a PIO access.

Quoted 9 lines

I then spent a while reading various bits of documentation in the kernel tree, LWN, etc., only to realize that I don't understand how this test framework actually works.

Are the selftests modules, userspace programs, or code that is compiled with the kernel, and (somehow?) run from userspace. I thought the latter, partially based on your explanation at ELC, but then I don't understand how the "compile and run" make target works.

The tests are standalone userspace programs which are compiled separately, but have dependencies on kernel headers. As stated above, for kvm, each selftest is a kvm userspace (including its vcpu thread code) and guest code combined. While there's a lot of complexity in the framework, particularly for memory management, and a bit for vcpu setup, most of that can be shared among tests using the kvm_util.h and test_util.h APIs, allowing a given test to only have a relatively simple main(), vcpu thread "vcpu_worker()" function, and "guest_code()" function. Guest mode code can easily share code with the kvm userspace test code (assuming the guest page tables are set up in the default way) and even data can be shared as long the accesses are done with the appropriate mappings (gva vs. hva). There's a small API to help with that as well.

Quoted 4 lines

Can you help me paint the overall picture, or point me to the piece of documentation/presentation that explains the high-level picture, which I must have obviously missed somehow?

We definitely need the documentation and, in hindsight, it looks like it would have been a good BoF topic last week too.

I think this framework has a lot of potential for KVM API testing and even for quick & dirty guest code instruction sequence tests (although instruction sequences would also fit kvm-unit-tests). I hope I can help get you and anyone else interested started.

Thanks,
drew

Quoted 79 lines

Thanks!

Christoffer

Quoted 73 lines

The series breaks down into parts as follows:

01-02: generalize guest code to host userspace exit support by
introducing "ucalls" - hypercalls to userspace
03-05: prepare common code for a new architecture 06-07: add virtual memory setup support for AArch64
08: add vcpu setup support for AArch64
09: port the dirty log test to AArch64
10-11: add 64K guest page size support for the dirty log test 12-13: prepare the dirty log test to also test > 40-bit guest
physical address setups by allowing the test memory region to be placed at the top of physical memory

[1] https://www.spinics.net/lists/arm-kernel/msg676819.html

Thanks,
drew

Andrew Jones (13):
kvm: selftests: vcpu_setup: set cr4.osfxsr
kvm: selftests: introduce ucall
kvm: selftests: move arch-specific files to arch-specific locations
kvm: selftests: add cscope make target
kvm: selftests: tidy up kvm_util
kvm: selftests: add vm_phy_pages_alloc
kvm: selftests: add virt mem support for aarch64
kvm: selftests: add vcpu support for aarch64
kvm: selftests: port dirty_log_test to aarch64
kvm: selftests: introduce new VM mode for 64K pages
kvm: selftests: dirty_log_test: also test 64K pages on aarch64
kvm: selftests: stop lying to aarch64 tests about PA-bits
kvm: selftests: support high GPAs in dirty_log_test

 tools/testing/selftests/kvm/.gitignore        |  11 +-
 tools/testing/selftests/kvm/Makefile          |  36 +-
 tools/testing/selftests/kvm/dirty_log_test.c  | 374 +++++++++----
 .../selftests/kvm/include/aarch64/processor.h |  55 ++
 .../testing/selftests/kvm/include/kvm_util.h  | 166 +++---
 .../testing/selftests/kvm/include/sparsebit.h |   6 +-
 .../testing/selftests/kvm/include/test_util.h |   6 +-
 .../kvm/include/{x86.h => x86_64/processor.h} |  24 +-
 .../selftests/kvm/include/{ => x86_64}/vmx.h  |   6 +-
 .../selftests/kvm/lib/aarch64/processor.c     | 311 +++++++++++
 tools/testing/selftests/kvm/lib/assert.c      |   2 +-
 tools/testing/selftests/kvm/lib/kvm_util.c    | 499 +++++++-----------
 .../selftests/kvm/lib/kvm_util_internal.h     |  33 +-
 tools/testing/selftests/kvm/lib/ucall.c       | 144 +++++
 .../kvm/lib/{x86.c => x86_64/processor.c}     | 197 ++++++-
 .../selftests/kvm/lib/{ => x86_64}/vmx.c      |   4 +-
 .../kvm/{ => x86_64}/cr4_cpuid_sync_test.c    |  14 +-
 .../kvm/{ => x86_64}/set_sregs_test.c         |   2 +-
 .../selftests/kvm/{ => x86_64}/state_test.c   |  25 +-
 .../kvm/{ => x86_64}/sync_regs_test.c         |   2 +-
 .../kvm/{ => x86_64}/vmx_tsc_adjust_test.c    |  23 +-
 21 files changed, 1329 insertions(+), 611 deletions(-)
 create mode 100644 tools/testing/selftests/kvm/include/aarch64/processor.h
 rename tools/testing/selftests/kvm/include/{x86.h => x86_64/processor.h} (98%)
 rename tools/testing/selftests/kvm/include/{ => x86_64}/vmx.h (99%)
 create mode 100644 tools/testing/selftests/kvm/lib/aarch64/processor.c
 create mode 100644 tools/testing/selftests/kvm/lib/ucall.c
 rename tools/testing/selftests/kvm/lib/{x86.c => x86_64/processor.c} (85%)
 rename tools/testing/selftests/kvm/lib/{ => x86_64}/vmx.c (99%)
 rename tools/testing/selftests/kvm/{ => x86_64}/cr4_cpuid_sync_test.c (91%)
 rename tools/testing/selftests/kvm/{ => x86_64}/set_sregs_test.c (98%)
 rename tools/testing/selftests/kvm/{ => x86_64}/state_test.c (90%)
 rename tools/testing/selftests/kvm/{ => x86_64}/sync_regs_test.c (99%)
 rename tools/testing/selftests/kvm/{ => x86_64}/vmx_tsc_adjust_test.c (91%)
Signature · 1 line
2.17.1
maintainer
#18 lore raw
Quoted 5 lines · Andrew Jones

There are three parts to a kvm selftest: 1) the test code which runs in host userspace and _is_ the kvm userspace used with kvm, 2) the vcpu thread code which executes KVM_RUN for the guest code and possibly also some host userspace test code, and 3) the guest code, which is naturally run in the vcpu thread, but in guest mode.

Note that the vcpu thread is a specialty of this test. Usually it's not needed and 1+2 are the same.

Paolo

#19 lore raw
In reply to Andrew Jones
Quoted 7 lines · Andrew Jones

Hi Christoffer,

Thanks for your interest in these tests. There isn't any documentation that I know of, but it's a good idea to have some. I'll write something up soon. I'll also try to answer your questions now.

That sounds great, thanks!

Quoted 19 lines · Christoffer Dall
Quoted 11 lines

Hi Drew,

Quoted 4 lines · Andrew Jones

This series provides KVM selftests that test dirty log tracking on AArch64 for both 4K and 64K guest page sizes. Additionally the framework provides an easy way to test dirty log tracking with the recently posted dynamic IPA and 52bit IPA series[1].

I was trying to parse the commit text of patch 2, and I realized that I don't understand the 'hypercall to userspace' thing at all, which probably means I have no idea how the selftests work overall.

There are three parts to a kvm selftest: 1) the test code which runs in host userspace and _is_ the kvm userspace used with kvm, 2) the vcpu thread code which executes KVM_RUN for the guest code and possibly also some host userspace test code, and 3) the guest code, which is naturally run in the vcpu thread, but in guest mode.

Quoted 7 lines

The need for a "ucall" arises for 2's "possibly also some host userspace test code". In that case the guest code needs to invoke an exit from guest mode, not just to kvm, but all the way to kvm userspace. For AArch64, as you know, this can be done with an MMIO access. The reason patch 2 generalizes the concept is because for x86 this can and is done with a PIO access.

So in the world of normal KVM userspace, (2) would be a thread in the same process as (1), sharing its mm. Is this a different setup somehow, why?

Quoted 10 lines
Quoted 9 lines

I then spent a while reading various bits of documentation in the kernel tree, LWN, etc., only to realize that I don't understand how this test framework actually works.

Are the selftests modules, userspace programs, or code that is compiled with the kernel, and (somehow?) run from userspace. I thought the latter, partially based on your explanation at ELC, but then I don't understand how the "compile and run" make target works.

Quoted 13 lines

The tests are standalone userspace programs which are compiled separately, but have dependencies on kernel headers. As stated above, for kvm, each selftest is a kvm userspace (including its vcpu thread code) and guest code combined. While there's a lot of complexity in the framework, particularly for memory management, and a bit for vcpu setup, most of that can be shared among tests using the kvm_util.h and test_util.h APIs, allowing a given test to only have a relatively simple main(), vcpu thread "vcpu_worker()" function, and "guest_code()" function. Guest mode code can easily share code with the kvm userspace test code (assuming the guest page tables are set up in the default way) and even data can be shared as long the accesses are done with the appropriate mappings (gva vs. hva). There's a small API to help with that as well.

Sounds cool. Beware of the attributes of the mappings such that both the guest and host have mapped the memory cacheable etc., but I'm sure you've thought of that already.

Quoted 7 lines
Quoted 4 lines

Can you help me paint the overall picture, or point me to the piece of documentation/presentation that explains the high-level picture, which I must have obviously missed somehow?

We definitely need the documentation and, in hindsight, it looks like it would have been a good BoF topic last week too.

An overview of the different testing approaches would be a good KVM Forum talk for next year, IMHO. When should you use kvm-unit-tests, and when should you use kselftests, some examples, etc. Just saying ;)

Quoted 6 lines

I think this framework has a lot of potential for KVM API testing and even for quick & dirty guest code instruction sequence tests (although instruction sequences would also fit kvm-unit-tests). I hope I can help get you and anyone else interested started.

I'll have a look at this series and glance at the code some more, it would be interesting to consider if using some of this for nested virt tests makes sense.

Thanks,

Christoffer

#20 lore raw
Quoted 38 lines · Christoffer Dall
Quoted 7 lines · Andrew Jones

Hi Christoffer,

Thanks for your interest in these tests. There isn't any documentation that I know of, but it's a good idea to have some. I'll write something up soon. I'll also try to answer your questions now.

That sounds great, thanks!

Quoted 19 lines · Christoffer Dall
Quoted 11 lines

Hi Drew,

Quoted 4 lines · Andrew Jones

This series provides KVM selftests that test dirty log tracking on AArch64 for both 4K and 64K guest page sizes. Additionally the framework provides an easy way to test dirty log tracking with the recently posted dynamic IPA and 52bit IPA series[1].

I was trying to parse the commit text of patch 2, and I realized that I don't understand the 'hypercall to userspace' thing at all, which probably means I have no idea how the selftests work overall.

There are three parts to a kvm selftest: 1) the test code which runs in host userspace and _is_ the kvm userspace used with kvm, 2) the vcpu thread code which executes KVM_RUN for the guest code and possibly also some host userspace test code, and 3) the guest code, which is naturally run in the vcpu thread, but in guest mode.

Quoted 7 lines

The need for a "ucall" arises for 2's "possibly also some host userspace test code". In that case the guest code needs to invoke an exit from guest mode, not just to kvm, but all the way to kvm userspace. For AArch64, as you know, this can be done with an MMIO access. The reason patch 2 generalizes the concept is because for x86 this can and is done with a PIO access.

Quoted 3 lines

So in the world of normal KVM userspace, (2) would be a thread in the same process as (1), sharing its mm. Is this a different setup somehow, why?

It's the same setup. Actually the only difference is what Paolo pointed out in his reply. There's no need for an independent vcpu thread to be spawned when only one vcpu thread is needed and no additional main thread is needed. I.e. the main test / kvm userspace code can call KVM_RUN itself.

Quoted 25 lines
Quoted 11 lines
Quoted 10 lines

I then spent a while reading various bits of documentation in the kernel tree, LWN, etc., only to realize that I don't understand how this test framework actually works.

Are the selftests modules, userspace programs, or code that is compiled with the kernel, and (somehow?) run from userspace. I thought the latter, partially based on your explanation at ELC, but then I don't understand how the "compile and run" make target works.

Quoted 13 lines

The tests are standalone userspace programs which are compiled separately, but have dependencies on kernel headers. As stated above, for kvm, each selftest is a kvm userspace (including its vcpu thread code) and guest code combined. While there's a lot of complexity in the framework, particularly for memory management, and a bit for vcpu setup, most of that can be shared among tests using the kvm_util.h and test_util.h APIs, allowing a given test to only have a relatively simple main(), vcpu thread "vcpu_worker()" function, and "guest_code()" function. Guest mode code can easily share code with the kvm userspace test code (assuming the guest page tables are set up in the default way) and even data can be shared as long the accesses are done with the appropriate mappings (gva vs. hva). There's a small API to help with that as well.

Quoted 3 lines

Sounds cool. Beware of the attributes of the mappings such that both the guest and host have mapped the memory cacheable etc., but I'm sure you've thought of that already.

Right. If you look at virt_pg_map(), then you'll see that I have the default set to NORMAL memory. It can be overridden by calling _virt_pg_map() directly - which might be nice to do to specifically test stage1/stage2 mapping combinations.

Quoted 6 lines
Quoted 6 lines
Quoted 5 lines

Can you help me paint the overall picture, or point me to the piece of documentation/presentation that explains the high-level picture, which I must have obviously missed somehow?

Quoted 6 lines
Quoted 2 lines

We definitely need the documentation and, in hindsight, it looks like it would have been a good BoF topic last week too.

An overview of the different testing approaches would be a good KVM Forum talk for next year, IMHO. When should you use kvm-unit-tests, and when should you use kselftests, some examples, etc. Just saying ;)

:-)

Quoted 11 lines
Quoted 7 lines

I think this framework has a lot of potential for KVM API testing and even for quick & dirty guest code instruction sequence tests (although instruction sequences would also fit kvm-unit-tests). I hope I can help get you and anyone else interested started.

I'll have a look at this series and glance at the code some more, it would be interesting to consider if using some of this for nested virt tests makes sense.

Yes. x86 has many nested tests. I think the framework was originally created with that in mind.

Thanks,
drew