API test framework
v2 cover API test framework
This patchset adds an API test framework. Rather than driving kvm from qemu, we now have a way of calling the kvm API directly and observing the results. We can switch to guest mode and back at will and see any micro effects such as the result of executing particular instructions.
A first test is added, for commit edde99ce0 ("KVM: Write protect memory after slot swap"). This would be pretty hard to test for using the current qemu-driven tests, but detects 5 million failures in about two minutes on my machine.
As an experiment, the framework is coded in C++.
Avi Kivity (14):
Makefile: add support for C++
Improve autodepend includes
v2: new
Add exception class for kernel errors (errno)
v2: new
Add try_main() for running a program under an exception handler
v2: new
Introduce a C++ wrapper for the kvm APIs
v2: throw std::exception exceptions, not ints, for libc failures
better msr list management
Add support for calling a function in guest mode
v2: use tr1::function instead of boost::function
Add sample test using the api test harness
api: add support for KVM_SET_USER_MEMORY_REGION flags field
v2: new
api: support KVM_GET_DIRTY_LOG ioctl
v2: new
api: add memory map management
v2: new
Build tests with debug information
v2: new
api: Add support for creating an identity map with a hole
v2: new
Introduce libapi.a to avoid long Makefile recipes
v2: new
Add dirty log test
v2: newMakefile | 8 +- api/api-sample.cc | 30 ++++++++ api/dirty-log.cc | 78 ++++++++++++++++++++ api/exception.cc | 33 ++++++++ api/exception.hh | 19 +++++ api/identity.cc | 95 ++++++++++++++++++++++++ api/identity.hh | 43 +++++++++++ api/kvmxx.cc | 194 +++++++++++++++++++++++++++++++++++++++++++++++++ api/kvmxx.hh | 85 +++++++++++++++++++++ api/memmap.cc | 76 +++++++++++++++++++ api/memmap.hh | 43 +++++++++++ config-x86-common.mak | 15 ++++- 12 files changed, 715 insertions(+), 4 deletions(-) create mode 100644 api/api-sample.cc create mode 100644 api/dirty-log.cc create mode 100644 api/exception.cc create mode 100644 api/exception.hh create mode 100644 api/identity.cc create mode 100644 api/identity.hh create mode 100644 api/kvmxx.cc create mode 100644 api/kvmxx.hh create mode 100644 api/memmap.cc create mode 100644 api/memmap.hh
PATCH v2 1/14 Makefile: add support for C++
--- Makefile | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-)
Makefile +3 -1
@@ -30,11 +30,13 @@ CFLAGS += -O1 CFLAGS += $(autodepend-flags) -g -fomit-frame-pointer -Wall CFLAGS += $(call cc-option, -fno-stack-protector, "") CFLAGS += $(call cc-option, -fno-stack-protector-all, "") +CFLAGS += -I. -CXXFLAGS = $(autodepend-flags) +CXXFLAGS += $(CFLAGS) autodepend-flags = -MMD -MF $(dir $*).$(notdir $*).d +LDFLAGS += $(CFLAGS) LDFLAGS += -pthread -lrt kvmtrace_objs= kvmtrace.o
Signature · 1 line
1.7.1
PATCH v2 2/14 Improve autodepend includes
In reply to Avi KivityInstead of listing all directories explicitly, include all autodepend files.
--- Makefile | 2 +- config-x86-common.mak | 1 - 2 files changed, 1 insertions(+), 2 deletions(-)
Makefile +1 -1
@@ -50,7 +50,7 @@ $(libcflat): $(cflatobjs) %.o: %.S $(CC) $(CFLAGS) -c -nostdlib -o $@ $< --include .*.d +-include .*.d */.*.d */*/.*.d install: mkdir -p $(DESTDIR)
config-x86-common.mak +0 -1
@@ -81,4 +81,3 @@ arch_clean: $(RM) $(TEST_DIR)/*.o $(TEST_DIR)/*.flat $(TEST_DIR)/*.elf \ $(TEST_DIR)/.*.d $(TEST_DIR)/lib/.*.d $(TEST_DIR)/lib/*.o --include $(TEST_DIR)/.*.d lib/.*.d lib/x86/.*.d
Signature · 1 line
1.7.1
PATCH v2 3/14 Add exception class for kernel errors (errno)
In reply to Avi Kivity--- api/exception.cc | 20 ++++++++++++++++++++ api/exception.hh | 16 ++++++++++++++++ 2 files changed, 36 insertions(+), 0 deletions(-) create mode 100644 api/exception.cc create mode 100644 api/exception.hh
api/exception.cc new file +20 -0
@@ -0,0 +1,20 @@ +#include "exception.hh" +#include <cstdio> +#include <cstring> + +errno_exception::errno_exception(int errno) + : _errno(errno) +{ +} + +int errno_exception::errno() const +{ + return _errno; +} + +const char *errno_exception::what() +{ + std::snprintf(_buf, sizeof _buf, "error: %s (%d)", + std::strerror(_errno), _errno); + return _buf; +}
api/exception.hh new file +16 -0
@@ -0,0 +1,16 @@ +#ifndef EXCEPTION_HH +#define EXCEPTION_HH + +#include <exception> + +class errno_exception : public std::exception { +public: + explicit errno_exception(int err_no); + int errno() const; + virtual const char *what(); +private: + int _errno; + char _buf[1000]; +}; + +#endif
Signature · 1 line
1.7.1
PATCH v2 4/14 Add try_main() for running a program under an exception handler
In reply to Avi Kivity--- api/exception.cc | 13 +++++++++++++ api/exception.hh | 3 +++ 2 files changed, 16 insertions(+), 0 deletions(-)
api/exception.cc +13 -0
@@ -18,3 +18,16 @@ const char *errno_exception::what() std::strerror(_errno), _errno); return _buf; } + +int try_main(int (*main)(int argc, char** argv), int argc, char** argv, + int ret_on_exception) +{ + try { + return main(argc, argv); + } catch (std::exception& e) { + std::fprintf(stderr, "exception: %s\n", e.what()); + } catch (...) { + std::fprintf(stderr, "unknown exception\n"); + } + return ret_on_exception; +}
api/exception.hh +3 -0
@@ -13,4 +13,7 @@ private: char _buf[1000]; }; +int try_main(int (*main)(int argc, char** argv), int argc, char** argv, + int ret_on_exception = 127); + #endif
Signature · 1 line
1.7.1
PATCH v2 5/14 Introduce a C++ wrapper for the kvm APIs
In reply to Avi KivityIntroduce exception-safe objects for calling system, vm, and vcpu ioctls.
--- api/kvmxx.cc | 185 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ api/kvmxx.hh | 83 ++++++++++++++++++++++++++ 2 files changed, 268 insertions(+), 0 deletions(-) create mode 100644 api/kvmxx.cc create mode 100644 api/kvmxx.hh
api/kvmxx.cc new file +185 -0
@@ -0,0 +1,185 @@ +#include "kvmxx.hh" +#include "exception.hh" +#include <fcntl.h> +#include <sys/ioctl.h> +#include <sys/mman.h> +#include <stdlib.h> +#include <memory> +#include <algorithm> + +namespace kvm { + +static long check_error(long r) +{ + if (r == -1) { + throw errno_exception(errno); + } + return r; +} + +fd::fd(int fd) + : _fd(fd) +{ +} + +fd::fd(const fd& other) + : _fd(::dup(other._fd)) +{ + check_error(_fd); +} + +fd::fd(std::string device_node, int flags) + : _fd(::open(device_node.c_str(), flags)) +{ + check_error(_fd); +} + +long fd::ioctl(unsigned nr, long arg) +{ + return check_error(::ioctl(_fd, nr, arg)); +} + +vcpu::vcpu(vm& vm, int id) + : _vm(vm), _fd(vm._fd.ioctl(KVM_CREATE_VCPU, id)), _shared(NULL) + , _mmap_size(_vm._system._fd.ioctl(KVM_GET_VCPU_MMAP_SIZE, 0)) + +{ + kvm_run *shared = static_cast<kvm_run*>(::mmap(NULL, _mmap_size, + PROT_READ | PROT_WRITE, + MAP_SHARED, + _fd.get(), 0)); + if (shared == MAP_FAILED) { + throw errno_exception(errno); + } + _shared = shared; +} + +vcpu::~vcpu() +{ + munmap(_shared, _mmap_size); +} + +void vcpu::run() +{ + _fd.ioctl(KVM_RUN, 0); +} + +kvm_regs vcpu::regs() +{ + kvm_regs regs; + _fd.ioctlp(KVM_GET_REGS, ®s); + return regs; +} + +void vcpu::set_regs(const kvm_regs& regs) +{ + _fd.ioctlp(KVM_SET_REGS, const_cast<kvm_regs*>(®s)); +} + +kvm_sregs vcpu::sregs() +{ + kvm_sregs sregs; + _fd.ioctlp(KVM_GET_SREGS, &sregs); + return sregs; +} + +void vcpu::set_sregs(const kvm_sregs& sregs) +{ + _fd.ioctlp(KVM_SET_SREGS, const_cast<kvm_sregs*>(&sregs)); +} + +class vcpu::kvm_msrs_ptr { +public: + explicit kvm_msrs_ptr(size_t nmsrs); + ~kvm_msrs_ptr() { ::free(_kvm_msrs); } + kvm_msrs* operator->() { return _kvm_msrs; } + kvm_msrs* get() { return _kvm_msrs; } +private: + kvm_msrs* _kvm_msrs; +}; + +vcpu::kvm_msrs_ptr::kvm_msrs_ptr(size_t nmsrs) + : _kvm_msrs(0) +{ + size_t size = sizeof(kvm_msrs) + sizeof(kvm_msr_entry) * nmsrs; + _kvm_msrs = static_cast<kvm_msrs*>(::malloc(size)); + if (!_kvm_msrs) { + throw std::bad_alloc(); + } +} + +std::vector<kvm_msr_entry> vcpu::msrs(std::vector<uint32_t> indices) +{ + kvm_msrs_ptr msrs(indices.size()); + msrs->nmsrs = indices.size(); + for (unsigned i = 0; i < msrs->nmsrs; ++i) { + msrs->entries[i].index = indices[i]; + } + _fd.ioctlp(KVM_GET_MSRS, msrs.get()); + return std::vector<kvm_msr_entry>(msrs->entries, + msrs->entries + msrs->nmsrs); +} + +void vcpu::set_msrs(const std::vector<kvm_msr_entry>& msrs) +{ + kvm_msrs_ptr _msrs(msrs.size()); + _msrs->nmsrs = msrs.size(); + std::copy(msrs.begin(), msrs.end(), _msrs->entries); + _fd.ioctlp(KVM_SET_MSRS, _msrs.get()); +} + +void vcpu::set_debug(uint64_t dr[8], bool enabled, bool singlestep) +{ + kvm_guest_debug gd; + + gd.control = 0; + if (enabled) { + gd.control |= KVM_GUESTDBG_ENABLE; + } + if (singlestep) { + gd.control |= KVM_GUESTDBG_SINGLESTEP; + } + for (int i = 0; i < 8; ++i) { + gd.arch.debugreg[i] = dr[i]; + } + _fd.ioctlp(KVM_SET_GUEST_DEBUG, &gd); +} + +vm::vm(system& system) + : _system(system), _fd(system._fd.ioctl(KVM_CREATE_VM, 0)) +{ +} + +void vm::set_memory_region(int slot, void *addr, uint64_t gpa, size_t len) +{ + struct kvm_userspace_memory_region umr; + + umr.slot = slot; + umr.flags = 0; + umr.guest_phys_addr = gpa; + umr.memory_size = len; + umr.userspace_addr = reinterpret_cast<uint64_t>(addr); + _fd.ioctlp(KVM_SET_USER_MEMORY_REGION, &umr); +} + +void vm::set_tss_addr(uint32_t addr) +{ + _fd.ioctl(KVM_SET_TSS_ADDR, addr); +} + +system::system(std::string device_node) + : _fd(device_node, O_RDWR) +{ +} + +bool system::check_extension(int extension) +{ + return _fd.ioctl(KVM_CHECK_EXTENSION, extension); +} + +int system::get_extension_int(int extension) +{ + return _fd.ioctl(KVM_CHECK_EXTENSION, extension); +} + +};
api/kvmxx.hh new file +83 -0
83 lines not on this page · raw message
Signature · 1 line
1.7.1
PATCH v2 6/14 Add support for calling a function in guest mode
In reply to Avi KivityThis patch provides a way to establish an "identity" guest which has a 1:1 gva->hva translation. This allows the host to switch to guest mode, call a function in the same address space, and return.
Because long mode virtual addresses are 47 bits long, and some hosts have smaller physical addresses, we target 32-bit mode only. On x86_64 the code needs to be run with 'setarch i386 -3' to limit the address space to 3GB, so the address space occupied by the local APIC is left unused.
--- api/identity.cc | 76 +++++++++++++++++++++++++++++++++++++++++++++++++ api/identity.hh | 28 ++++++++++++++++++ config-x86-common.mak | 1 + 3 files changed, 105 insertions(+), 0 deletions(-) create mode 100644 api/identity.cc create mode 100644 api/identity.hh
api/identity.cc new file +76 -0
@@ -0,0 +1,76 @@ + +#include "identity.hh" +#include <stdio.h> + +namespace identity { + +typedef unsigned long ulong; + +void setup_vm(kvm::vm& vm) +{ + vm.set_memory_region(0, NULL, 0, 3UL << 30); + vm.set_tss_addr(3UL << 30); +} + +void vcpu::setup_sregs() +{ + kvm_sregs sregs = { }; + kvm_segment dseg = { }; + dseg.base = 0; dseg.limit = -1U; dseg.type = 3; dseg.present = 1; + dseg.dpl = 3; dseg.db = 1; dseg.s = 1; dseg.l = 0; dseg.g = 1; + kvm_segment cseg = dseg; + cseg.type = 11; + + sregs.cs = cseg; asm ("mov %%cs, %0" : "=rm"(sregs.cs.selector)); + sregs.ds = dseg; asm ("mov %%ds, %0" : "=rm"(sregs.ds.selector)); + sregs.es = dseg; asm ("mov %%es, %0" : "=rm"(sregs.es.selector)); + sregs.fs = dseg; asm ("mov %%fs, %0" : "=rm"(sregs.fs.selector)); + sregs.gs = dseg; asm ("mov %%gs, %0" : "=rm"(sregs.gs.selector)); + sregs.ss = dseg; asm ("mov %%ss, %0" : "=rm"(sregs.ss.selector)); + + uint32_t gsbase; + asm ("mov %%gs:0, %0" : "=r"(gsbase)); + sregs.gs.base = gsbase; + + sregs.tr.base = reinterpret_cast<ulong>(&*_stack.begin()); + sregs.tr.type = 11; + sregs.tr.s = 0; + sregs.tr.present = 1; + + sregs.cr0 = 0x11; /* PE, ET, !PG */ + sregs.cr4 = 0; + sregs.efer = 0; + sregs.apic_base = 0xfee00000; + _vcpu.set_sregs(sregs); +} + +void vcpu::thunk(vcpu* zis) +{ + zis->_guest_func(); + asm volatile("outb %%al, %%dx" : : "a"(0), "d"(0)); +} + +void vcpu::setup_regs() +{ + kvm_regs regs = {}; + regs.rflags = 0x3202; + regs.rsp = reinterpret_cast<ulong>(&*_stack.end()); + regs.rsp &= ~15UL; + ulong* sp = reinterpret_cast<ulong *>(regs.rsp); + *--sp = reinterpret_cast<ulong>((char*)this); + *--sp = 0; + regs.rsp = reinterpret_cast<ulong>(sp); + regs.rip = reinterpret_cast<ulong>(&vcpu::thunk); + printf("rip %llx\n", regs.rip); + _vcpu.set_regs(regs); +} + +vcpu::vcpu(kvm::vcpu& vcpu, std::tr1::function<void ()> guest_func, + unsigned long stack_size) + : _vcpu(vcpu), _guest_func(guest_func), _stack(stack_size) +{ + setup_sregs(); + setup_regs(); +} + +}
api/identity.hh new file +28 -0
@@ -0,0 +1,28 @@ +#ifndef API_IDENTITY_HH +#define API_IDENTITY_HH + +#include "kvmxx.hh" +#include <tr1/functional> +#include <vector> + +namespace identity { + +void setup_vm(kvm::vm& vm); + +class vcpu { +public: + vcpu(kvm::vcpu& vcpu, std::tr1::function<void ()> guest_func, + unsigned long stack_size = 256 * 1024); +private: + static void thunk(vcpu* vcpu); + void setup_regs(); + void setup_sregs(); +private: + kvm::vcpu& _vcpu; + std::tr1::function<void ()> _guest_func; + std::vector<char> _stack; +}; + +} + +#endif
config-x86-common.mak +1 -0
@@ -81,3 +81,4 @@ arch_clean: $(RM) $(TEST_DIR)/*.o $(TEST_DIR)/*.flat $(TEST_DIR)/*.elf \ $(TEST_DIR)/.*.d $(TEST_DIR)/lib/.*.d $(TEST_DIR)/lib/*.o +api/%.o: CFLAGS += -m32
Signature · 1 line
1.7.1
PATCH v2 7/14 Add sample test using the api test harness
In reply to Avi KivityCall a function setting a global variable.
--- api/api-sample.cc | 29 +++++++++++++++++++++++++++++ config-x86-common.mak | 7 +++++++ 2 files changed, 36 insertions(+), 0 deletions(-) create mode 100644 api/api-sample.cc
api/api-sample.cc new file +29 -0
@@ -0,0 +1,29 @@ + +#include "api/kvmxx.hh" +#include "api/identity.hh" +#include "api/exception.hh" +#include "stdio.h" + +static int global = 0; + +static void set_global() +{ + global = 1; +} + +int test_main(int ac, char** av) +{ + kvm::system system; + kvm::vm vm(system); + identity::setup_vm(vm); + kvm::vcpu vcpu(vm, 0); + identity::vcpu thread(vcpu, set_global); + vcpu.run(); + printf("global %d\n", global); + return global == 1 ? 0 : 1; +} + +int main(int ac, char** av) +{ + return try_main(test_main, ac, av); +}
config-x86-common.mak +7 -0
@@ -32,6 +32,8 @@ tests-common = $(TEST_DIR)/vmexit.flat $(TEST_DIR)/tsc.flat \ $(TEST_DIR)/hypercall.flat $(TEST_DIR)/sieve.flat \ $(TEST_DIR)/kvmclock_test.flat +tests-common += api/api-sample + tests_and_config = $(TEST_DIR)/*.flat $(TEST_DIR)/unittests.cfg test_cases: $(tests-common) $(tests)
@@ -82,3 +84,8 @@ arch_clean: $(TEST_DIR)/.*.d $(TEST_DIR)/lib/.*.d $(TEST_DIR)/lib/*.o api/%.o: CFLAGS += -m32 + +api/api-sample: LDLIBS += -lstdc++ +api/api-sample: LDFLAGS += -m32 + +api/api-sample: api/api-sample.o api/kvmxx.o api/identity.o api/exception.o
Signature · 1 line
1.7.1
PATCH v2 8/14 api: add support for KVM_SET_USER_MEMORY_REGION flags field
In reply to Avi Kivity--- api/kvmxx.cc | 5 +++-- api/kvmxx.hh | 3 ++- 2 files changed, 5 insertions(+), 3 deletions(-)
api/kvmxx.cc +3 -2
@@ -150,12 +150,13 @@ vm::vm(system& system) { } -void vm::set_memory_region(int slot, void *addr, uint64_t gpa, size_t len) +void vm::set_memory_region(int slot, void *addr, uint64_t gpa, size_t len, + uint32_t flags) { struct kvm_userspace_memory_region umr; umr.slot = slot; - umr.flags = 0; + umr.flags = flags; umr.guest_phys_addr = gpa; umr.memory_size = len; umr.userspace_addr = reinterpret_cast<uint64_t>(addr);
api/kvmxx.hh +2 -1
@@ -57,7 +57,8 @@ private: class vm { public: explicit vm(system& system); - void set_memory_region(int slot, void *addr, uint64_t gpa, size_t len); + void set_memory_region(int slot, void *addr, uint64_t gpa, size_t len, + uint32_t flags = 0); void set_tss_addr(uint32_t addr); system& sys() { return _system; } private:
Signature · 1 line
1.7.1
PATCH v2 9/14 api: support KVM_GET_DIRTY_LOG ioctl
In reply to Avi Kivity--- api/kvmxx.cc | 8 ++++++++ api/kvmxx.hh | 1 + 2 files changed, 9 insertions(+), 0 deletions(-)
api/kvmxx.cc +8 -0
@@ -163,6 +163,14 @@ void vm::set_memory_region(int slot, void *addr, uint64_t gpa, size_t len, _fd.ioctlp(KVM_SET_USER_MEMORY_REGION, &umr); } +void vm::get_dirty_log(int slot, void *log) +{ + struct kvm_dirty_log kdl; + kdl.slot = slot; + kdl.dirty_bitmap = log; + _fd.ioctlp(KVM_GET_DIRTY_LOG, &kdl); +} + void vm::set_tss_addr(uint32_t addr) { _fd.ioctl(KVM_SET_TSS_ADDR, addr);
api/kvmxx.hh +1 -0
@@ -59,6 +59,7 @@ public: explicit vm(system& system); void set_memory_region(int slot, void *addr, uint64_t gpa, size_t len, uint32_t flags = 0); + void get_dirty_log(int slot, void *log); void set_tss_addr(uint32_t addr); system& sys() { return _system; } private:
Signature · 1 line
1.7.1
PATCH v2 10/14 api: add memory map management
In reply to Avi KivityAdd a class to manage the memory map and a class to represent a memory slot.
--- api/memmap.cc | 76 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++ api/memmap.hh | 43 ++++++++++++++++++++++++++++++++ 2 files changed, 119 insertions(+), 0 deletions(-) create mode 100644 api/memmap.cc create mode 100644 api/memmap.hh
api/memmap.cc new file +76 -0
@@ -0,0 +1,76 @@ + +#include "memmap.hh" + +mem_slot::mem_slot(mem_map& map, uint64_t gpa, uint64_t size, void* hva) + : _map(map) + , _slot(map._free_slots.top()) + , _gpa(gpa) + , _size(size) + , _hva(hva) + , _dirty_log_enabled(false) + , _log() +{ + map._free_slots.pop(); + update(); +} + +mem_slot::~mem_slot() +{ + _size = 0; + try { + update(); + _map._free_slots.push(_slot); + } catch (...) { + // can't do much if we can't undo slot registration - leak the slot + } +} + +void mem_slot::set_dirty_logging(bool enabled) +{ + if (_dirty_log_enabled != enabled) { + _dirty_log_enabled = enabled; + if (enabled) { + int logsize = ((_size >> 12) + bits_per_word - 1) / bits_per_word; + _log.resize(logsize); + } else { + _log.resize(0); + } + update(); + } +} + +void mem_slot::update() +{ + uint32_t flags = 0; + if (_dirty_log_enabled) { + flags |= KVM_MEM_LOG_DIRTY_PAGES; + } + _map._vm.set_memory_region(_slot, _hva, _gpa, _size, flags); +} + +bool mem_slot::dirty_logging() const +{ + return _dirty_log_enabled; +} + +void mem_slot::update_dirty_log() +{ + _map._vm.get_dirty_log(_slot, &_log[0]); +} + +bool mem_slot::is_dirty(uint64_t gpa) const +{ + uint64_t pagenr = (gpa - _gpa) >> 12; + ulong wordnr = pagenr / bits_per_word; + ulong bit = 1ULL << (pagenr % bits_per_word); + return _log[wordnr] & bit; +} + +mem_map::mem_map(kvm::vm& vm) + : _vm(vm) +{ + int nr_slots = vm.sys().get_extension_int(KVM_CAP_NR_MEMSLOTS); + for (int i = 0; i < nr_slots; ++i) { + _free_slots.push(i); + } +}
api/memmap.hh new file +43 -0
43 lines not on this page · raw message
Signature · 1 line
1.7.1
PATCH v2 11/14 Build tests with debug information
In reply to Avi Kivity--- Makefile | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
Makefile +1 -1
@@ -6,7 +6,7 @@ DESTDIR := $(PREFIX)/share/qemu/tests .PHONY: arch_clean clean #make sure env CFLAGS variable is not used -CFLAGS = +CFLAGS = -g libgcc := $(shell $(CC) --print-libgcc-file-name)
Signature · 1 line
1.7.1
PATCH v2 12/14 api: Add support for creating an identity map with a hole
In reply to Avi KivityThe hole may be used for mmio or dirty logging.
--- api/api-sample.cc | 3 ++- api/identity.cc | 23 +++++++++++++++++++++-- api/identity.hh | 17 ++++++++++++++++- config-x86-common.mak | 1 + 4 files changed, 40 insertions(+), 4 deletions(-)
api/api-sample.cc +2 -1
@@ -15,7 +15,8 @@ int test_main(int ac, char** av) { kvm::system system; kvm::vm vm(system); - identity::setup_vm(vm); + mem_map memmap(vm); + identity::vm ident_vm(vm, memmap); kvm::vcpu vcpu(vm, 0); identity::vcpu thread(vcpu, set_global); vcpu.run();
api/identity.cc +21 -2
@@ -6,9 +6,28 @@ namespace identity { typedef unsigned long ulong; -void setup_vm(kvm::vm& vm) +hole::hole() + : address(), size() { - vm.set_memory_region(0, NULL, 0, 3UL << 30); +} + +hole::hole(void* address, size_t size) + : address(address), size(size) +{ +} + +vm::vm(kvm::vm& vm, mem_map& mmap, hole h) +{ + uint64_t hole_gpa = reinterpret_cast<uint64_t>(h.address); + char* hole_hva = static_cast<char*>(h.address); + if (h.address) { + _slots.push_back(mem_slot_ptr(new mem_slot(mmap, 0, hole_gpa, NULL))); + } + uint64_t hole_end = hole_gpa + h.size; + uint64_t end = 3U << 30; + _slots.push_back(mem_slot_ptr(new mem_slot(mmap, hole_end, + end - hole_end, + hole_hva + h.size))); vm.set_tss_addr(3UL << 30); }
api/identity.hh +16 -1
@@ -2,12 +2,27 @@ #define API_IDENTITY_HH #include "kvmxx.hh" +#include "memmap.hh" #include <tr1/functional> +#include <tr1/memory> #include <vector> namespace identity { -void setup_vm(kvm::vm& vm); +struct hole { + hole(); + hole(void* address, size_t size); + void* address; + size_t size; +}; + +class vm { +public: + vm(kvm::vm& vm, mem_map& mmap, hole address_space_hole = hole()); +private: + typedef std::tr1::shared_ptr<mem_slot> mem_slot_ptr; + std::vector<mem_slot_ptr> _slots; +}; class vcpu { public:
config-x86-common.mak +1 -0
@@ -89,3 +89,4 @@ api/api-sample: LDLIBS += -lstdc++ api/api-sample: LDFLAGS += -m32 api/api-sample: api/api-sample.o api/kvmxx.o api/identity.o api/exception.o +api/api-sample: api/memmap.o
Signature · 1 line
1.7.1
PATCH v2 13/14 Introduce libapi.a to avoid long Makefile recipes
In reply to Avi Kivity--- config-x86-common.mak | 10 ++++++---- 1 files changed, 6 insertions(+), 4 deletions(-)
config-x86-common.mak +6 -4
@@ -85,8 +85,10 @@ arch_clean: api/%.o: CFLAGS += -m32 -api/api-sample: LDLIBS += -lstdc++ -api/api-sample: LDFLAGS += -m32 +api/%: LDLIBS += -lstdc++ +api/%: LDFLAGS += -m32 -api/api-sample: api/api-sample.o api/kvmxx.o api/identity.o api/exception.o -api/api-sample: api/memmap.o +api/libapi.a: api/kvmxx.o api/identity.o api/exception.o api/memmap.o + $(AR) rcs $@ $^ + +api/api-sample: api/api-sample.o api/libapi.a
Signature · 1 line
1.7.1
PATCH v2 14/14 Add dirty log test
In reply to Avi KivityThis checks the failure that was fixed by kernel commit edde99ce0529 ("KVM: Write protect memory after slot swap"). Two threads are used; a guest thread continuously updates a shared variable, which is also sampled by a host thread that also checks if dirty logging marked it as dirty.
It detects about 5 million failures with the fix reverted, and 0 failures with the fix applied.
--- api/dirty-log.cc | 78 +++++++++++++++++++++++++++++++++++++++++++++++++ config-x86-common.mak | 7 +++- 2 files changed, 83 insertions(+), 2 deletions(-) create mode 100644 api/dirty-log.cc
api/dirty-log.cc new file +78 -0
@@ -0,0 +1,78 @@ +#include "kvmxx.hh" +#include "memmap.hh" +#include "identity.hh" +#include <boost/thread/thread.hpp> +#include <stdlib.h> +#include <stdio.h> + +namespace { + +void delay_loop(unsigned n) +{ + for (unsigned i = 0; i < n; ++i) { + asm volatile("pause"); + } + } + +void write_mem(volatile bool& running, volatile int* shared_var) +{ + while (running) { + ++*shared_var; + delay_loop(1000); + } +} + +void check_dirty_log(mem_slot& slot, + volatile bool& running, + volatile int* shared_var, + int& nr_fail) +{ + uint64_t shared_var_gpa = reinterpret_cast<uint64_t>(shared_var); + slot.set_dirty_logging(true); + slot.update_dirty_log(); + for (int i = 0; i < 10000000; ++i) { + int sample1 = *shared_var; + delay_loop(600); + int sample2 = *shared_var; + slot.update_dirty_log(); + if (!slot.is_dirty(shared_var_gpa) && sample1 != sample2) { + ++nr_fail; + } + } + running = false; + slot.set_dirty_logging(false); +} + +} + +using boost::ref; +using std::tr1::bind; + +int main(int ac, char **av) +{ + kvm::system sys; + kvm::vm vm(sys); + mem_map memmap(vm); + void* logged_slot_virt; + posix_memalign(&logged_slot_virt, 4096, 4096); + int* shared_var = static_cast<int*>(logged_slot_virt); + identity::hole hole(logged_slot_virt, 4096); + identity::vm ident_vm(vm, memmap, hole); + kvm::vcpu vcpu(vm, 0); + bool running = true; + int nr_fail = 0; + mem_slot logged_slot(memmap, + reinterpret_cast<uint64_t>(logged_slot_virt), + 4096, logged_slot_virt); + boost::thread host_poll_thread(check_dirty_log, ref(logged_slot), + ref(running), + ref(shared_var), ref(nr_fail)); + identity::vcpu guest_write_thread(vcpu, + bind(write_mem, + ref(running), + ref(shared_var))); + vcpu.run(); + host_poll_thread.join(); + printf("Dirty bitmap failures: %d\n", nr_fail); + return nr_fail == 0 ? 0 : 1; +}
config-x86-common.mak +5 -2
@@ -33,6 +33,7 @@ tests-common = $(TEST_DIR)/vmexit.flat $(TEST_DIR)/tsc.flat \ $(TEST_DIR)/kvmclock_test.flat tests-common += api/api-sample +tests-common += api/dirty-log tests_and_config = $(TEST_DIR)/*.flat $(TEST_DIR)/unittests.cfg
@@ -85,10 +86,12 @@ arch_clean: api/%.o: CFLAGS += -m32 -api/%: LDLIBS += -lstdc++ +api/%: LDLIBS += -lstdc++ -lboost_thread-mt -lpthread api/%: LDFLAGS += -m32 api/libapi.a: api/kvmxx.o api/identity.o api/exception.o api/memmap.o $(AR) rcs $@ $^ -api/api-sample: api/api-sample.o api/libapi.a +api/api-sample: api/api-sample.o api/libapi.a + +api/dirty-log: api/dirty-log.o api/libapi.a
Signature · 1 line
1.7.1
Marcelo Tosatti
AppliedQuoted 56 lines · Avi Kivity
This patchset adds an API test framework. Rather than driving kvm from qemu, we now have a way of calling the kvm API directly and observing the results. We can switch to guest mode and back at will and see any micro effects such as the result of executing particular instructions.
A first test is added, for commit
edde99ce0("KVM: Write protect memory after slot swap"). This would be pretty hard to test for using the current qemu-driven tests, but detects 5 million failures in about two minutes on my machine.As an experiment, the framework is coded in C++.
Avi Kivity (14):
Makefile: add support for C++
Improve autodepend includesv2: new Add exception class for kernel errors (errno) v2: new Add try_main() for running a program under an exception handler v2: new Introduce a C++ wrapper for the kvm APIs v2: throw std::exception exceptions, not ints, for libc failures better msr list management Add support for calling a function in guest mode v2: use tr1::function instead of boost::function Add sample test using the api test harness api: add support for KVM_SET_USER_MEMORY_REGION flags field v2: new api: support KVM_GET_DIRTY_LOG ioctl v2: new api: add memory map management v2: new Build tests with debug information v2: new api: Add support for creating an identity map with a hole v2: new Introduce libapi.a to avoid long Makefile recipes v2: new Add dirty log test v2: newMakefile | 8 +- api/api-sample.cc | 30 ++++++++ api/dirty-log.cc | 78 ++++++++++++++++++++ api/exception.cc | 33 ++++++++ api/exception.hh | 19 +++++ api/identity.cc | 95 ++++++++++++++++++++++++ api/identity.hh | 43 +++++++++++ api/kvmxx.cc | 194 +++++++++++++++++++++++++++++++++++++++++++++++++ api/kvmxx.hh | 85 +++++++++++++++++++++ api/memmap.cc | 76 +++++++++++++++++++ api/memmap.hh | 43 +++++++++++ config-x86-common.mak | 15 ++++- 12 files changed, 715 insertions(+), 4 deletions(-) create mode 100644 api/api-sample.cc create mode 100644 api/dirty-log.cc
Quoted 8 lines
create mode 100644 api/exception.cc create mode 100644 api/exception.hh create mode 100644 api/identity.cc create mode 100644 api/identity.hh create mode 100644 api/kvmxx.cc create mode 100644 api/kvmxx.hh create mode 100644 api/memmap.cc create mode 100644 api/memmap.hh
Applied, thanks.