Skip to contentSkip to conversation

Add monitor tests to XTF

PATCH Superseded Petre Pircalabu 2 participants 6 messages Dec 14, 2018 – Dec 17, 2018 Kbuild

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#2 lore raw

PATCH 2/4 xtf: Add executable test class

In reply to Petre Pircalabu

The Executable test class runs on host (dom0). The class spawns a process and searches the program output(stdio) for a specific pattern.

---
 xtf/__init__.py        |  2 +-
 xtf/executable_test.py | 83 ++++++++++++++++++++++++++++++++++++++++++++++++++
 xtf/suite.py           |  5 ++-
 3 files changed, 88 insertions(+), 2 deletions(-)
 create mode 100644 xtf/executable_test.py
xtf/__init__.py +1 -1
index 889c1d5..07c269a 100644
@@ -3,7 +3,7 @@
 
 # All test categories
 default_categories     = set(("functional", "xsa"))
-non_default_categories = set(("special", "utility", "in-development"))
+non_default_categories = set(("special", "utility", "in-development", "host"))
 all_categories         = default_categories | non_default_categories
 
 # All test environments
xtf/executable_test.py new file +83 -0

83 lines not on this page · raw message

xtf/suite.py +4 -1
index ad7d30f..2e0727c 100644
@@ -75,7 +75,10 @@ def gather_all_test_info():
             try:
                 info_file = open(path.join("tests", test, "info.json"))
             except IOError:
-                continue
+                try:
+                    info_file = open(path.join("tests", test, "host.json"))
+                except IOError:
+                    continue
 
             # Ignore tests which have bad JSON
             try:
Signature · 7 lines
2.7.4


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#4 lore raw

PATCH 4/4 xtf: Add emul-unimpl test

In reply to Petre Pircalabu

Add a new test to verify if XEN can correctly handle the X86EMUL_UNIMPLEMENTED event.

The XTF DomU test image just executes a instruction not implemented by the XEN X86 emulator (fstenv) and checks if the execution was successfull. This instruction will be the first one in a custom .text section.

In order to instruct XEN to try to emulate that instruction the monitor application changes the attributes of that specific page in order to inhibit execution. This will trigger a MEM_ACCESS request that will be responded by toggling the EMULATE flag.
The emulation will fail, which will trigger an EMUL_UNIMPLEMENTED request which will be handled by enabling execution on that specific page (altp2m) and singlestepping that instruction.

The test will be successfull if the instruction can be executed correctly.

---
 docs/all-tests.dox             |   2 +-
 tests/emul-unimpl/Makefile     |  15 ++
 tests/emul-unimpl/extra.cfg.in |   3 +
 tests/emul-unimpl/main.c       |  59 ++++++++
 tests/emul-unimpl/monitor.c    | 310 +++++++++++++++++++++++++++++++++++++++++
 5 files changed, 388 insertions(+), 1 deletion(-)
 create mode 100644 tests/emul-unimpl/Makefile
 create mode 100644 tests/emul-unimpl/extra.cfg.in
 create mode 100644 tests/emul-unimpl/main.c
 create mode 100644 tests/emul-unimpl/monitor.c
docs/all-tests.dox +1 -1
index 3ee552e..b7457be 100644
@@ -149,5 +149,5 @@ enable BTS.
 
 @section index-monitor Monitor
 
-@subpage test-emul_unimplemented - @Test EMUL_UNIMPLEMENTED event generation
+@subpage test-emul-unimpl - @Test EMUL_UNIMPLEMENTED event generation
 */
tests/emul-unimpl/Makefile new file +15 -0
new file mode 100644
index 0000000..5d79e42
@@ -0,0 +1,15 @@
+include $(ROOT)/build/common.mk
+
+NAME      		:= emul-unimpl
+CATEGORY  		:= monitor
+TEST-ENVS 		:= hvm64
+CLASS	  		:= xtf.monitor_test.MonitorTestInfo
+TEST-EXTRA-INFO	:= monitor_args='--address=0x\$$(nm --defined-only @@VM_PATH@@ | grep test_fn | cut -d \  -f 1)'
+
+TEST-EXTRA-CFG := extra.cfg.in
+
+obj-perenv += main.o
+
+obj-monitor += monitor.o
+
+include $(ROOT)/build/gen.mk
tests/emul-unimpl/extra.cfg.in new file +3 -0
new file mode 100644
index 0000000..e432a0c
@@ -0,0 +1,3 @@
+# Enable altp2m
+altp2m = "mixed"
+altp2mhvm = 1
tests/emul-unimpl/main.c new file +59 -0

59 lines not on this page · raw message

tests/emul-unimpl/monitor.c new file +310 -0

310 lines not on this page · raw message

Signature · 7 lines
2.7.4


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#1 lore raw

PATCH 1/4 xtf-runner: split into logical components

In reply to Petre Pircalabu

Split the xtf-runner script file into multiple modules in order to support multiple test types.

Features:
- 2 abstract types (TestInfo and TestInstance) to represent the test information (info.json) and, respectively to implement the test execution.
TestInfo has to implement the "all_instances" method to create the list of TestInstance objects.
TestInstance has to implement "set_up", "run", and "clean-up" methods.
- TestResult - represents an XTF test result (SUCCESS, SKIP, ERROR, FAILURE, CRASH). The values should be kept in sync with the C code from report.h
- Dynamic test class loading. Each info.json shoudl contain a "class_name" field which specifies the test info class describing the test. This value defaults to "xtf.domu_test.DomuTestInfo"
- custom test info parameters. info.json can have the "extra" field, implemented as a dictionary, which contains parameters specific for a certain test info class.

    e.g. TEST-EXTRA-INFO := arg1='--address=0x80000000 --id=4' arg2=42

- logger class (print depending on the quiet field)
- DomuTestInfo/DomuTest instance. Simple test which loads a XEN DomU and checks the output for a specific pattern.
- toolstack abstraction using a wrapper class (e.g. (xtf.xl_domu.XLDomU)

---
 build/gen.mk      |  13 ++-
 build/mkinfo.py   |  84 +++++++++++---
 xtf-runner        | 334 +++++-------------------------------------------------
 xtf/__init__.py   |  12 ++
 xtf/domu_test.py  | 179 +++++++++++++++++++++++++++++
 xtf/exceptions.py |   6 +
 xtf/logger.py     |  23 ++++
 xtf/suite.py      |  97 ++++++++++++++++
 xtf/test.py       | 139 +++++++++++++++++++++++
 xtf/xl_domu.py    | 121 ++++++++++++++++++++
 10 files changed, 687 insertions(+), 321 deletions(-)
 create mode 100644 xtf/__init__.py
 create mode 100644 xtf/domu_test.py
 create mode 100644 xtf/exceptions.py
 create mode 100644 xtf/logger.py
 create mode 100644 xtf/suite.py
 create mode 100644 xtf/test.py
 create mode 100644 xtf/xl_domu.py
build/gen.mk +12 -1
index 8d7a6bf..c19ca6a 100644
@@ -27,12 +27,23 @@ else
 TEST-CFGS := $(foreach env,$(TEST-ENVS),test-$(env)-$(NAME).cfg)
 endif
 
+CLASS ?= "xtf.domu_test.DomuTestInfo"
+
 .PHONY: build
 build: $(foreach env,$(TEST-ENVS),test-$(env)-$(NAME)) $(TEST-CFGS)
 build: info.json
 
+MKINFO-OPTS := -n "$(NAME)"
+MKINFO-OPTS += -c "$(CLASS)"
+MKINFO-OPTS += -t "$(CATEGORY)"
+MKINFO-OPTS += -e "$(TEST-ENVS)"
+MKINFO-OPTS += -v "$(VARY-CFG)"
+ifneq (x$(TEST-EXTRA-INFO), x)
+MKINFO-OPTS += -x "$(TEST-EXTRA-INFO)"
+endif
+
 info.json: $(ROOT)/build/mkinfo.py FORCE
-	@$(PYTHON) $< [email protected] "$(NAME)" "$(CATEGORY)" "$(TEST-ENVS)" "$(VARY-CFG)"
+	@$(PYTHON) $< $(MKINFO-OPTS) [email protected]
 	@$(call move-if-changed,[email protected],$@)
 
 .PHONY: install install-each-env
build/mkinfo.py +67 -17

91 lines not on this page · raw message

xtf-runner +31 -303

394 lines not on this page · raw message

xtf/__init__.py new file +12 -0
new file mode 100644
index 0000000..889c1d5
@@ -0,0 +1,12 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+
+# All test categories
+default_categories     = set(("functional", "xsa"))
+non_default_categories = set(("special", "utility", "in-development"))
+all_categories         = default_categories | non_default_categories
+
+# All test environments
+pv_environments        = set(("pv64", "pv32pae"))
+hvm_environments       = set(("hvm64", "hvm32pae", "hvm32pse", "hvm32"))
+all_environments       = pv_environments | hvm_environments
xtf/domu_test.py new file +179 -0

179 lines not on this page · raw message

xtf/exceptions.py new file +6 -0
new file mode 100644
index 0000000..26801a2
@@ -0,0 +1,6 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+
+class RunnerError(Exception):
+    """ Errors relating to xtf-runner itself """
+
xtf/logger.py new file +23 -0
new file mode 100644
index 0000000..ec279e5
@@ -0,0 +1,23 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+
+class Singleton(type):
+    """Singleton meta class"""
+    _instances = {}
+    def __call__(cls, *args, **kwargs):
+        if cls not in cls._instances:
+            cls._instances[cls] = super(Singleton, cls).__call__(*args, **kwargs)
+        return cls._instances[cls]
+
+class Logger(object):
+    """Logger class for XTF."""
+    __metaclass__ = Singleton
+
+    def initialize(self, opts):
+        """Initialize logger"""
+        self.quiet = opts.quiet
+
+    def log(self, message):
+        """Display the message"""
+        if not self.quiet:
+            print message
xtf/suite.py new file +97 -0

97 lines not on this page · raw message

xtf/test.py new file +139 -0

139 lines not on this page · raw message

xtf/xl_domu.py new file +121 -0

121 lines not on this page · raw message

Signature · 7 lines
2.7.4


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#3 lore raw

PATCH 3/4 xtf: Add monitor test class

In reply to Petre Pircalabu

This class starts alongside the domain a monitor application which opens an event channel corresponding to that domain and handles the received requests.
Use the "monitor_args" key to pass test specific arguments to the monitor application.
The arguments will be added in the test's Makefile using the TEST-EXTRA-INFO variable.

---
 Makefile                  |   6 +-
 build/common.mk           |  22 ++-
 build/files.mk            |   3 +
 build/gen.mk              |  12 ++
 docs/all-tests.dox        |   5 +
 include/monitor/monitor.h | 117 +++++++++++++
 monitor/Makefile          |  20 +++
 monitor/monitor.c         | 409 ++++++++++++++++++++++++++++++++++++++++++++++
 xtf/__init__.py           |   2 +-
 xtf/monitor_test.py       | 132 +++++++++++++++
 xtf/utils.py              |  17 ++
 11 files changed, 741 insertions(+), 4 deletions(-)
 create mode 100644 include/monitor/monitor.h
 create mode 100644 monitor/Makefile
 create mode 100644 monitor/monitor.c
 create mode 100644 xtf/monitor_test.py
 create mode 100644 xtf/utils.py
Makefile +4 -2
index 15a865f..db28075 100644
@@ -32,7 +32,9 @@ INSTALL_PROGRAM := $(INSTALL) -p
 OBJCOPY         := $(CROSS_COMPILE)objcopy
 PYTHON          := python
 
-export CC CPP INSTALL INSTALL_DATA INSTALL_DIR INSTALL_PROGRAM OBJCOPY PYTHON
+HOSTCC          := gcc
+
+export CC CPP INSTALL INSTALL_DATA INSTALL_DIR INSTALL_PROGRAM OBJCOPY PYTHON HOSTCC
 
 .PHONY: all
 all:
@@ -51,7 +53,7 @@ install:
 	done
 
 define all_sources
-	find include/ arch/ common/ tests/ -name "*.[hcsS]"
+	find include/ arch/ common/ tests/ monitor/ -name "*.[hcsS]"
 endef
 
 .PHONY: cscope
build/common.mk +21 -1
index b786ddf..1ec0fa4 100644
@@ -1,4 +1,4 @@
-ALL_CATEGORIES     := special functional xsa utility in-development
+ALL_CATEGORIES     := special functional xsa utility in-development monitor
 
 ALL_ENVIRONMENTS   := pv64 pv32pae hvm64 hvm32pae hvm32pse hvm32
 
@@ -35,11 +35,20 @@ COMMON_AFLAGS-x86_64 := -m64
 COMMON_CFLAGS-x86_32 := -m32
 COMMON_CFLAGS-x86_64 := -m64
 
+#HOSTCFLAGS := -Wall -Werror
+HOSTCFLAGS  :=
+HOSTLDFLAGS :=
+HOSTLDLIBS  :=
+HOSTCFLAGS  += -D__XEN_TOOLS__ -g -O3 -I$(ROOT)/include/monitor
+HOSTCFLAGS  += -DXC_WANT_COMPAT_DEVICEMODEL_API -DXC_WANT_COMPAT_MAP_FOREIGN_API
+HOSTLDLIBS  += -lxenctrl -lxenstore -lxenevtchn
+
 defcfg-pv    := $(ROOT)/config/default-pv.cfg.in
 defcfg-hvm   := $(ROOT)/config/default-hvm.cfg.in
 
 obj-perarch :=
 obj-perenv  :=
+obj-monitor :=
 include $(ROOT)/build/files.mk
 
 
@@ -90,8 +99,19 @@ DEPS-$(1) = $$(head-$(1)) \
 
 endef
 
+# Setup monitor rules
+define MONITOR_setup
+DEPS-MONITOR = \
+	$$(obj-monitor:%.o=%-monitor.o)
+
+%-monitor.o: %.c
+	$$(HOSTCC) $$(HOSTCFLAGS) -c $$< -o $$@
+endef
+
 $(foreach env,$(ALL_ENVIRONMENTS),$(eval $(call PERENV_setup,$(env))))
 
+$(eval $(call MONITOR_setup))
+
 define move-if-changed
 	if ! cmp -s $(1) $(2); then mv -f $(1) $(2); else rm -f $(1); fi
 endef
build/files.mk +3 -0
index dfa27e4..972c797 100644
@@ -54,3 +54,6 @@ $(foreach env,$(32BIT_ENVIRONMENTS),$(eval obj-$(env) += $(obj-32)))
 # 64bit specific objects
 obj-64  += $(ROOT)/arch/x86/entry_64.o
 $(foreach env,$(64BIT_ENVIRONMENTS),$(eval obj-$(env) += $(obj-64)))
+
+# Monitor common objects
+obj-monitor += $(ROOT)/monitor/monitor.o
build/gen.mk +12 -0
index c19ca6a..1e6773a 100644
@@ -32,6 +32,9 @@ CLASS ?= "xtf.domu_test.DomuTestInfo"
 .PHONY: build
 build: $(foreach env,$(TEST-ENVS),test-$(env)-$(NAME)) $(TEST-CFGS)
 build: info.json
+ifeq (x$(CATEGORY),xmonitor)
+build: test-monitor-$(NAME)
+endif
 
 MKINFO-OPTS := -n "$(NAME)"
 MKINFO-OPTS += -c "$(CLASS)"
@@ -100,6 +103,15 @@ install-each-env: install-$(1) install-$(1).cfg
 endef
 $(foreach env,$(TEST-ENVS),$(eval $(call PERENV_build,$(env))))
 
+define MONITOR_build
+test-monitor-$(NAME): $(DEPS-MONITOR)
+	@echo $(obj-monitor)
+	@echo $(DEPS-MONITOR)
+	$(HOSTCC) $(HOSTLDFLAGS) $(DEPS-MONITOR) $(HOSTLDLIBS) -o $$@
+endef
+
+$(eval $(call MONITOR_build))
+
 .PHONY: clean
 clean:
 	find $(ROOT) \( -name "*.o" -o -name "*.d" \) -delete
docs/all-tests.dox +5 -0
index 732d44c..3ee552e 100644
@@ -145,4 +145,9 @@ enable BTS.
 @subpage test-nested-svm - Nested SVM tests.
 
 @subpage test-nested-vmx - Nested VT-x tests.
+
+
+@section index-monitor Monitor
+
+@subpage test-emul_unimplemented - @Test EMUL_UNIMPLEMENTED event generation
 */
include/monitor/monitor.h new file +117 -0

117 lines not on this page · raw message

monitor/Makefile new file +20 -0
new file mode 100644
index 0000000..64d4f8a
@@ -0,0 +1,20 @@
+.PHONY: all
+
+all: monitor
+
+HOSTCC ?= gcc
+
+OBJS = monitor.o
+
+#HOSTCFLAGS += -Wall -Werror
+HOSTCFLAGS += -D__XEN_TOOLS__ -g -O0
+HOSTCFLAGS += -DXC_WANT_COMPAT_DEVICEMODEL_API -DXC_WANT_COMPAT_MAP_FOREIGN_API
+
+%.o : %.c
+	$(HOSTCC) -c $(HOSTCFLAGS) $(HOSTCPPFLAGS) $< -o $@
+
+monitor: $(OBJS)
+	$(HOSTCC) -o $@ $^ -lxenctrl -lxenstore -lxenevtchn
+
+clean:
+	$(RM) $(OBJS) monitor
monitor/monitor.c new file +409 -0

409 lines not on this page · raw message

xtf/__init__.py +1 -1
index 07c269a..e405013 100644
@@ -3,7 +3,7 @@
 
 # All test categories
 default_categories     = set(("functional", "xsa"))
-non_default_categories = set(("special", "utility", "in-development", "host"))
+non_default_categories = set(("special", "utility", "in-development", "host", "monitor"))
 all_categories         = default_categories | non_default_categories
 
 # All test environments
xtf/monitor_test.py new file +132 -0

132 lines not on this page · raw message

xtf/utils.py new file +17 -0
new file mode 100644
index 0000000..96c570b
@@ -0,0 +1,17 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+""" XTF utils """
+
+import threading
+
+class XTFAsyncCall(threading.Thread):
+    def __init__(self, group=None, target=None, name=None, args=(), kwargs={}):
+        super(XTFAsyncCall, self).__init__(group, target, name, args, kwargs)
+        self._return = None
+    def run(self):
+        if self._Thread__target is not None:
+            self._return = self._Thread__target(*self._Thread__args,
+                                                **self._Thread__kwargs)
+    def join(self):
+        threading.Thread.join(self)
+        return self._return
Signature · 7 lines
2.7.4


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#0 lore raw

cover Add monitor tests to XTF

Extend the framework to support (simple) monitor related tests.

Petre Pircalabu (4):
xtf-runner: split into logical components
xtf: Add executable test class
xtf: Add monitor test class
xtf: Add emul-unimpl test

 Makefile                       |   6 +-
 build/common.mk                |  22 ++-
 build/files.mk                 |   3 +
 build/gen.mk                   |  25 ++-
 build/mkinfo.py                |  84 +++++++--
 docs/all-tests.dox             |   5 +
 include/monitor/monitor.h      | 117 ++++++++++++
 monitor/Makefile               |  20 ++
 monitor/monitor.c              | 409 +++++++++++++++++++++++++++++++++++++++++
 tests/emul-unimpl/Makefile     |  15 ++
 tests/emul-unimpl/extra.cfg.in |   3 +
 tests/emul-unimpl/main.c       |  59 ++++++
 tests/emul-unimpl/monitor.c    | 310 +++++++++++++++++++++++++++++++
 xtf-runner                     | 334 ++++-----------------------------
 xtf/__init__.py                |  12 ++
 xtf/domu_test.py               | 179 ++++++++++++++++++
 xtf/exceptions.py              |   6 +
 xtf/executable_test.py         |  83 +++++++++
 xtf/logger.py                  |  23 +++
 xtf/monitor_test.py            | 132 +++++++++++++
 xtf/suite.py                   | 100 ++++++++++
 xtf/test.py                    | 139 ++++++++++++++
 xtf/utils.py                   |  17 ++
 xtf/xl_domu.py                 | 121 ++++++++++++
 24 files changed, 1900 insertions(+), 324 deletions(-)
 create mode 100644 include/monitor/monitor.h
 create mode 100644 monitor/Makefile
 create mode 100644 monitor/monitor.c
 create mode 100644 tests/emul-unimpl/Makefile
 create mode 100644 tests/emul-unimpl/extra.cfg.in
 create mode 100644 tests/emul-unimpl/main.c
 create mode 100644 tests/emul-unimpl/monitor.c
 create mode 100644 xtf/__init__.py
 create mode 100644 xtf/domu_test.py
 create mode 100644 xtf/exceptions.py
 create mode 100644 xtf/executable_test.py
 create mode 100644 xtf/logger.py
 create mode 100644 xtf/monitor_test.py
 create mode 100644 xtf/suite.py
 create mode 100644 xtf/test.py
 create mode 100644 xtf/utils.py
 create mode 100644 xtf/xl_domu.py
Signature · 7 lines
2.7.4


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#5 lore raw

Re: [PATCH XTF 4/4] xtf: Add emul-unimpl test

In reply to Petre Pircalabu
Quoted 8 lines
Quoted 1 line
Quoted 1 line · On 14.12.18 at 14:34, <[email protected]> wrote:

Add a new test to verify if XEN can correctly handle the X86EMUL_UNIMPLEMENTED event.

The XTF DomU test image just executes a instruction not implemented by the XEN X86 emulator (fstenv) and checks if the execution was successfull. This instruction will be the first one in a custom .text section.

May I suggest that you use an insn which is liable to remain unimplemented? FSTENV, together with {F,FX,X}{SAVE,RSTOR} are at the top of my (emulator) list of items to work on. If you pick any instruction which we _can_ reasonably implement, chances would then be that eventually your test will fail, preventing an osstest push. This would better be avoided from the beginning.

Jan

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel