Add monitor tests to XTF
v3 cover Add monitor tests to XTF
Extend the framework to support (simple) monitor related tests.
Changes from v2
- Check the returned value from xc_translate_foreign_address.
Changes from v1:
- Refactored the monitor test (cleanup)
- Replace the "emul-unimplemented" test with a simpler mem_access test
Petre Pircalabu (4):
xtf-runner: split into logical components
xtf: Add executable test class
xtf: Add monitor test class
xtf: Add monitor mem_access test
Makefile | 6 +- build/common.mk | 22 +- build/files.mk | 3 + build/gen.mk | 25 +- build/mkinfo.py | 84 +++++-- common/report.c | 8 - docs/all-tests.dox | 4 + include/monitor/monitor.h | 136 +++++++++++ include/xtf/report.h | 8 + monitor/Makefile | 20 ++ monitor/monitor.c | 481 +++++++++++++++++++++++++++++++++++++ tests/monitor-mem-access/Makefile | 14 ++ tests/monitor-mem-access/main.c | 37 +++ tests/monitor-mem-access/monitor.c | 164 +++++++++++++ xtf-runner | 334 +++---------------------- xtf/__init__.py | 12 + xtf/domu_test.py | 179 ++++++++++++++ xtf/exceptions.py | 6 + xtf/executable_test.py | 83 +++++++ xtf/logger.py | 23 ++ xtf/monitor_test.py | 132 ++++++++++ xtf/suite.py | 100 ++++++++ xtf/test.py | 139 +++++++++++ xtf/utils.py | 17 ++ xtf/xl_domu.py | 121 ++++++++++ 25 files changed, 1826 insertions(+), 332 deletions(-) create mode 100644 include/monitor/monitor.h create mode 100644 monitor/Makefile create mode 100644 monitor/monitor.c create mode 100644 tests/monitor-mem-access/Makefile create mode 100644 tests/monitor-mem-access/main.c create mode 100644 tests/monitor-mem-access/monitor.c create mode 100644 xtf/__init__.py create mode 100644 xtf/domu_test.py create mode 100644 xtf/exceptions.py create mode 100644 xtf/executable_test.py create mode 100644 xtf/logger.py create mode 100644 xtf/monitor_test.py create mode 100644 xtf/suite.py create mode 100644 xtf/test.py create mode 100644 xtf/utils.py create mode 100644 xtf/xl_domu.py
Signature · 7 lines
2.7.4 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v3 2/4 xtf: Add executable test class
The Executable test class runs on host (dom0). The class spawns a process and searches the program output(stdio) for a specific pattern.
--- xtf/__init__.py | 2 +- xtf/executable_test.py | 83 ++++++++++++++++++++++++++++++++++++++++++++++++++ xtf/suite.py | 5 ++- 3 files changed, 88 insertions(+), 2 deletions(-) create mode 100644 xtf/executable_test.py
xtf/__init__.py +1 -1
@@ -3,7 +3,7 @@ # All test categories default_categories = set(("functional", "xsa")) -non_default_categories = set(("special", "utility", "in-development")) +non_default_categories = set(("special", "utility", "in-development", "host")) all_categories = default_categories | non_default_categories # All test environments
xtf/executable_test.py new file +83 -0
83 lines not on this page · raw message
xtf/suite.py +4 -1
@@ -75,7 +75,10 @@ def gather_all_test_info(): try: info_file = open(path.join("tests", test, "info.json")) except IOError: - continue + try: + info_file = open(path.join("tests", test, "host.json")) + except IOError: + continue # Ignore tests which have bad JSON try:
Signature · 7 lines
2.7.4 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v3 1/4 xtf-runner: split into logical components
In reply to Petre PircalabuSplit the xtf-runner script file into multiple modules in order to support multiple test types.
Features:
- 2 abstract types (TestInfo and TestInstance) to represent the test information (info.json) and, respectively to implement the test execution.
TestInfo has to implement the "all_instances" method to create the list of TestInstance objects.
TestInstance has to implement "set_up", "run", and "clean-up" methods.
- TestResult - represents an XTF test result (SUCCESS, SKIP, ERROR, FAILURE, CRASH). The values should be kept in sync with the C code from report.h
- Dynamic test class loading. Each info.json shoudl contain a "class_name" field which specifies the test info class describing the test. This value defaults to "xtf.domu_test.DomuTestInfo"
- custom test info parameters. info.json can have the "extra" field, implemented as a dictionary, which contains parameters specific for a certain test info class.
e.g. TEST-EXTRA-INFO := arg1='--address=0x80000000 --id=4' arg2=42
- logger class (print depending on the quiet field)
- DomuTestInfo/DomuTest instance. Simple test which loads a XEN DomU and checks the output for a specific pattern.
- toolstack abstraction using a wrapper class (e.g. (xtf.xl_domu.XLDomU)
--- build/gen.mk | 13 ++- build/mkinfo.py | 84 +++++++++++--- xtf-runner | 334 +++++------------------------------------------------- xtf/__init__.py | 12 ++ xtf/domu_test.py | 179 +++++++++++++++++++++++++++++ xtf/exceptions.py | 6 + xtf/logger.py | 23 ++++ xtf/suite.py | 97 ++++++++++++++++ xtf/test.py | 139 +++++++++++++++++++++++ xtf/xl_domu.py | 121 ++++++++++++++++++++ 10 files changed, 687 insertions(+), 321 deletions(-) create mode 100644 xtf/__init__.py create mode 100644 xtf/domu_test.py create mode 100644 xtf/exceptions.py create mode 100644 xtf/logger.py create mode 100644 xtf/suite.py create mode 100644 xtf/test.py create mode 100644 xtf/xl_domu.py
build/gen.mk +12 -1
@@ -27,12 +27,23 @@ else TEST-CFGS := $(foreach env,$(TEST-ENVS),test-$(env)-$(NAME).cfg) endif +CLASS ?= "xtf.domu_test.DomuTestInfo" + .PHONY: build build: $(foreach env,$(TEST-ENVS),test-$(env)-$(NAME)) $(TEST-CFGS) build: info.json +MKINFO-OPTS := -n "$(NAME)" +MKINFO-OPTS += -c "$(CLASS)" +MKINFO-OPTS += -t "$(CATEGORY)" +MKINFO-OPTS += -e "$(TEST-ENVS)" +MKINFO-OPTS += -v "$(VARY-CFG)" +ifneq (x$(TEST-EXTRA-INFO), x) +MKINFO-OPTS += -x "$(TEST-EXTRA-INFO)" +endif + info.json: $(ROOT)/build/mkinfo.py FORCE - @$(PYTHON) $< [email protected] "$(NAME)" "$(CATEGORY)" "$(TEST-ENVS)" "$(VARY-CFG)" + @$(PYTHON) $< $(MKINFO-OPTS) [email protected] @$(call move-if-changed,[email protected],$@) .PHONY: install install-each-env
build/mkinfo.py +67 -17
91 lines not on this page · raw message
xtf-runner +31 -303
394 lines not on this page · raw message
xtf/__init__.py new file +12 -0
@@ -0,0 +1,12 @@ +#!/usr/bin/env python +# -*- coding: utf-8 -*- + +# All test categories +default_categories = set(("functional", "xsa")) +non_default_categories = set(("special", "utility", "in-development")) +all_categories = default_categories | non_default_categories + +# All test environments +pv_environments = set(("pv64", "pv32pae")) +hvm_environments = set(("hvm64", "hvm32pae", "hvm32pse", "hvm32")) +all_environments = pv_environments | hvm_environments
xtf/domu_test.py new file +179 -0
179 lines not on this page · raw message
xtf/exceptions.py new file +6 -0
@@ -0,0 +1,6 @@ +#!/usr/bin/env python +# -*- coding: utf-8 -*- + +class RunnerError(Exception): + """ Errors relating to xtf-runner itself """ +
xtf/logger.py new file +23 -0
@@ -0,0 +1,23 @@ +#!/usr/bin/env python +# -*- coding: utf-8 -*- + +class Singleton(type): + """Singleton meta class""" + _instances = {} + def __call__(cls, *args, **kwargs): + if cls not in cls._instances: + cls._instances[cls] = super(Singleton, cls).__call__(*args, **kwargs) + return cls._instances[cls] + +class Logger(object): + """Logger class for XTF.""" + __metaclass__ = Singleton + + def initialize(self, opts): + """Initialize logger""" + self.quiet = opts.quiet + + def log(self, message): + """Display the message""" + if not self.quiet: + print message
xtf/suite.py new file +97 -0
97 lines not on this page · raw message
xtf/test.py new file +139 -0
139 lines not on this page · raw message
xtf/xl_domu.py new file +121 -0
121 lines not on this page · raw message
Signature · 7 lines
2.7.4 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v3 4/4 xtf: Add monitor mem_access test
In reply to Petre PircalabuThe monitor application resets the execute permisions on a specific page of the DOMU and handles the generated vm_event request.
--- docs/all-tests.dox | 1 + tests/monitor-mem-access/Makefile | 14 ++++ tests/monitor-mem-access/main.c | 37 +++++++++ tests/monitor-mem-access/monitor.c | 164 +++++++++++++++++++++++++++++++++++++ 4 files changed, 216 insertions(+) create mode 100644 tests/monitor-mem-access/Makefile create mode 100644 tests/monitor-mem-access/main.c create mode 100644 tests/monitor-mem-access/monitor.c
docs/all-tests.dox +1 -0
@@ -148,4 +148,5 @@ enable BTS. @section index-monitor Monitor +@subpage test-monitor-mem-access - memory access monitor test */
tests/monitor-mem-access/Makefile new file +14 -0
@@ -0,0 +1,14 @@ +include $(ROOT)/build/common.mk + +NAME := monitor-mem-access +CATEGORY := monitor +TEST-ENVS := hvm64 +CLASS := xtf.monitor_test.MonitorTestInfo +TEST-EXTRA-INFO := monitor_args='--address=0x\$$(nm --defined-only @@VM_PATH@@ | grep test_fn | cut -d \ -f 1)' + +obj-perenv += main.o + +obj-monitor += monitor.o + +include $(ROOT)/build/gen.mk +
tests/monitor-mem-access/main.c new file +37 -0
@@ -0,0 +1,37 @@ +/** +* @file tests/monitor-mem-access/main.c +* @ref test-monitor-mem-access +* +* @page test-monitor-mem-access monitor-mem-access +*/ + +#include <xtf.h> + +const char test_title[] ="Test monitor mem-access event" ; + +static int count; + +static void __attribute__((section (".text.protected" ))) __attribute__((noinline))test_fn (void ) +{ + count++; +} + +void test_main (void ) +{ +test_fn (); + +if ( count ) +xtf_success (NULL ); +else +xtf_failure (NULL ); +} + +/* +* Local variables: +* mode: C +* c-file-style: "BSD" +* c-basic-offset: 4 +* tab-width: 4 +* indent-tabs-mode: nil +* End: +*/
tests/monitor-mem-access/monitor.c new file +164 -0
164 lines not on this page · raw message
Signature · 7 lines
2.7.4 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v3 3/4 xtf: Add monitor test class
In reply to Petre PircalabuThis class starts alongside the domain a monitor application which opens an event channel corresponding to that domain and handles the received requests.
Use the "monitor_args" key to pass test specific arguments to the monitor application.
The arguments will be added in the test's Makefile using the TEST-EXTRA-INFO variable.
--- Makefile | 6 +- build/common.mk | 22 ++- build/files.mk | 3 + build/gen.mk | 12 ++ common/report.c | 8 - docs/all-tests.dox | 3 + include/monitor/monitor.h | 136 +++++++++++++ include/xtf/report.h | 8 + monitor/Makefile | 20 ++ monitor/monitor.c | 481 ++++++++++++++++++++++++++++++++++++++++++++++ xtf/__init__.py | 2 +- xtf/monitor_test.py | 132 +++++++++++++ xtf/utils.py | 17 ++ 13 files changed, 838 insertions(+), 12 deletions(-) create mode 100644 include/monitor/monitor.h create mode 100644 monitor/Makefile create mode 100644 monitor/monitor.c create mode 100644 xtf/monitor_test.py create mode 100644 xtf/utils.py
Makefile +4 -2
@@ -32,7 +32,9 @@ INSTALL_PROGRAM := $(INSTALL) -p OBJCOPY := $(CROSS_COMPILE)objcopy PYTHON := python -export CC CPP INSTALL INSTALL_DATA INSTALL_DIR INSTALL_PROGRAM OBJCOPY PYTHON +HOSTCC := gcc + +export CC CPP INSTALL INSTALL_DATA INSTALL_DIR INSTALL_PROGRAM OBJCOPY PYTHON HOSTCC .PHONY: all all:
@@ -51,7 +53,7 @@ install: done define all_sources - find include/ arch/ common/ tests/ -name "*.[hcsS]" + find include/ arch/ common/ tests/ monitor/ -name "*.[hcsS]" endef .PHONY: cscope
build/common.mk +21 -1
@@ -1,4 +1,4 @@ -ALL_CATEGORIES := special functional xsa utility in-development +ALL_CATEGORIES := special functional xsa utility in-development monitor ALL_ENVIRONMENTS := pv64 pv32pae hvm64 hvm32pae hvm32pse hvm32
@@ -35,11 +35,20 @@ COMMON_AFLAGS-x86_64 := -m64 COMMON_CFLAGS-x86_32 := -m32 COMMON_CFLAGS-x86_64 := -m64 +#HOSTCFLAGS := -Wall -Werror +HOSTCFLAGS := +HOSTLDFLAGS := +HOSTLDLIBS := +HOSTCFLAGS += -D__XEN_TOOLS__ -g -O3 -I$(ROOT)/include/monitor +HOSTCFLAGS += -DXC_WANT_COMPAT_DEVICEMODEL_API -DXC_WANT_COMPAT_MAP_FOREIGN_API +HOSTLDLIBS += -lxenctrl -lxenstore -lxenevtchn + defcfg-pv := $(ROOT)/config/default-pv.cfg.in defcfg-hvm := $(ROOT)/config/default-hvm.cfg.in obj-perarch := obj-perenv := +obj-monitor := include $(ROOT)/build/files.mk
@@ -90,8 +99,19 @@ DEPS-$(1) = $$(head-$(1)) \ endef +# Setup monitor rules +define MONITOR_setup +DEPS-MONITOR = \ + $$(obj-monitor:%.o=%-monitor.o) + +%-monitor.o: %.c + $$(HOSTCC) $$(HOSTCFLAGS) -c $$< -o $$@ +endef + $(foreach env,$(ALL_ENVIRONMENTS),$(eval $(call PERENV_setup,$(env)))) +$(eval $(call MONITOR_setup)) + define move-if-changed if ! cmp -s $(1) $(2); then mv -f $(1) $(2); else rm -f $(1); fi endef
build/files.mk +3 -0
@@ -54,3 +54,6 @@ $(foreach env,$(32BIT_ENVIRONMENTS),$(eval obj-$(env) += $(obj-32))) # 64bit specific objects obj-64 += $(ROOT)/arch/x86/entry_64.o $(foreach env,$(64BIT_ENVIRONMENTS),$(eval obj-$(env) += $(obj-64))) + +# Monitor common objects +obj-monitor += $(ROOT)/monitor/monitor.o
build/gen.mk +12 -0
@@ -32,6 +32,9 @@ CLASS ?= "xtf.domu_test.DomuTestInfo" .PHONY: build build: $(foreach env,$(TEST-ENVS),test-$(env)-$(NAME)) $(TEST-CFGS) build: info.json +ifeq (x$(CATEGORY),xmonitor) +build: test-monitor-$(NAME) +endif MKINFO-OPTS := -n "$(NAME)" MKINFO-OPTS += -c "$(CLASS)"
@@ -100,6 +103,15 @@ install-each-env: install-$(1) install-$(1).cfg endef $(foreach env,$(TEST-ENVS),$(eval $(call PERENV_build,$(env)))) +define MONITOR_build +test-monitor-$(NAME): $(DEPS-MONITOR) + @echo $(obj-monitor) + @echo $(DEPS-MONITOR) + $(HOSTCC) $(HOSTLDFLAGS) $(DEPS-MONITOR) $(HOSTLDLIBS) -o $$@ +endef + +$(eval $(call MONITOR_build)) + .PHONY: clean clean: find $(ROOT) \( -name "*.o" -o -name "*.d" \) -delete
common/report.c +0 -8
@@ -2,14 +2,6 @@#include <xtf/report.h> #include <xtf/hypercall.h> -enum test_status { -STATUS_RUNNING ,/**< Test not yet completed. */ -STATUS_SUCCESS ,/**< Test was successful. */ -STATUS_SKIP ,/**< Test cannot be completed. */ -STATUS_ERROR ,/**< Issue with the test itself. */ -STATUS_FAILURE ,/**< Issue with the tested matter. */ -}; -/** Current status of this test. */ static enum test_status status;
docs/all-tests.dox +3 -0
@@ -145,4 +145,7 @@ enable BTS. @subpage test-nested-svm - Nested SVM tests. @subpage test-nested-vmx - Nested VT-x tests. + + +@section index-monitor Monitor */
include/monitor/monitor.h new file +136 -0
136 lines not on this page · raw message
include/xtf/report.h +8 -0
@@ -24,6 +24,14 @@* kept. */ +enum test_status { +STATUS_RUNNING ,/**< Test not yet completed. */ +STATUS_SUCCESS ,/**< Test was successful. */ +STATUS_SKIP ,/**< Test cannot be completed. */ +STATUS_ERROR ,/**< Issue with the test itself. */ +STATUS_FAILURE ,/**< Issue with the tested matter. */ +}; +/** * Report test success. */
monitor/Makefile new file +20 -0
@@ -0,0 +1,20 @@ +.PHONY: all + +all: monitor + +HOSTCC ?= gcc + +OBJS = monitor.o + +#HOSTCFLAGS += -Wall -Werror +HOSTCFLAGS += -D__XEN_TOOLS__ -g -O0 +HOSTCFLAGS += -DXC_WANT_COMPAT_DEVICEMODEL_API -DXC_WANT_COMPAT_MAP_FOREIGN_API + +%.o : %.c + $(HOSTCC) -c $(HOSTCFLAGS) $(HOSTCPPFLAGS) $< -o $@ + +monitor: $(OBJS) + $(HOSTCC) -o $@ $^ -lxenctrl -lxenstore -lxenevtchn + +clean: + $(RM) $(OBJS) monitor
monitor/monitor.c new file +481 -0
481 lines not on this page · raw message
xtf/__init__.py +1 -1
@@ -3,7 +3,7 @@ # All test categories default_categories = set(("functional", "xsa")) -non_default_categories = set(("special", "utility", "in-development", "host")) +non_default_categories = set(("special", "utility", "in-development", "host", "monitor")) all_categories = default_categories | non_default_categories # All test environments
xtf/monitor_test.py new file +132 -0
132 lines not on this page · raw message
xtf/utils.py new file +17 -0
@@ -0,0 +1,17 @@ +#!/usr/bin/env python +# -*- coding: utf-8 -*- +""" XTF utils """ + +import threading + +class XTFAsyncCall(threading.Thread): + def __init__(self, group=None, target=None, name=None, args=(), kwargs={}): + super(XTFAsyncCall, self).__init__(group, target, name, args, kwargs) + self._return = None + def run(self): + if self._Thread__target is not None: + self._return = self._Thread__target(*self._Thread__args, + **self._Thread__kwargs) + def join(self): + threading.Thread.join(self) + return self._return
Signature · 7 lines
2.7.4 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel