Avocado-based functional tests
cover Avocado-based functional tests
Background
==========
A few months ago, Cleber started a thread[1] about the system for Avocado-based tests. He invited people to take a look at it, we got some interesting feedback.
However, I saw no feedback on the actual code itself, and I don't know what needs to be done to get this merged.
To make sure we're having a discussion about the implementation of avocado_qemu and to figure out what's still necessary to get it included, I am sending all the commits from Amador's avocado_qemu branch[2] as a RFC.
Note that I didn't review most of this code yet, and I will probably send comments and questions as replies to this RFC later.
Also, many of the patches in this series probably can be squashed together, but I decided to not do that before getting some feedback on the changes.
[1] https://www.mail-archive.com/[email protected]/msg506859.html
[2] https://github.com/apahim/qemu/commits/avocado_qemu
README
======
Below, I'm copying the raw contents of tests/avocado/README.rst, for reference:
----------------------------------------------------------------------
========================================
QEMU tests using the Avocado Framework
========================================
This directory [tests/avocado] hosts functional tests written using Avocado Testing Framework.
Installation
============
To install Avocado and the dependencies needed for these tests, run::
pip install --user avocado-framework avocado-framework-plugin-varianter-yaml-to-mux aexpect
Alternatively, follow the instructions on this link::
http://avocado-framework.readthedocs.io/en/latest/GetStartedGuide.html#installing-avocado
Overview
========
In this directory [tests/avocado], an ``avocado_qemu`` package is provided, containing the ``test`` module, which inherits from ``avocado.Test`` and provides a builtin and easy-to-use Qemu virtual machine. Here's a template that can be used as reference to start writing your own tests::
from avocado_qemu import test
class MyTest(test.QemuTest):
"""
:avocado: enable
"""
def setUp(self):
self.vm.args.extend(['-m', '512'])
self.vm.launch()
def test_01(self):
res = self.vm.qmp('human-monitor-command',
command_line='info version')
self.assertIn('v2.9.0', res['return'])
def tearDown(self):
self.vm.shutdown()To execute your test, run::
avocado run test_my_test.py
To execute all tests, run::
avocado run .
If you don't specify the Qemu binary to use, the ``avocado_qemu`` package will automatically probe it. The probe will try to use the Qemu binary from the git tree build directory, using the same architecture as the local system (if the architecture is not specified). If the Qemu binary is not available in the git tree build directory, the next try is to use the system installed Qemu binary.
You can define a number of optional parameters, providing them via YAML file using the Avocado parameters system:
- ``qemu_bin``: Use a given Qemu binary, skipping the automatic probe. Example: ``qemu_bin: /usr/libexec/qemu-kvm``.
- ``qemu_dst_bin``: Use a given Qemu binary to create the destination VM when the migration process takes place. If it's not provided, the same binary used in the source VM will be used for the destination VM.
Example: ``qemu_dst_bin: /usr/libexec/qemu-kvm-binary2``.
- ``arch``: Probe the Qemu binary from a given architecture. It has no effect if ``qemu_bin`` is specified. If not provided, the binary probe will use the system architecture. Example: ``arch: x86_64``
- ``image_path``: When a test requires (usually a bootable) image, this parameter is used to define where the image is located. When undefined it uses ``$QEMU_ROOT/bootable_image_$arch.qcow2``. The image is added to the qemu command __only__ when the test requires an image. By default ``,snapshot=on`` is used, but it can be altered by ``image_snapshot`` parameter.
- ``image_user`` and ``image_pass``: When using a ``image_path``, if you want to get the console from the Guest OS you have to define the Guest OS credentials. Example: ``image_user: avocado`` and ``image_pass: p4ssw0rd``. Both parameters have defaults to ``avocado``.
- ``machine_type``: Use this option to define a machine type for the VM.
Example: ``machine_type: pc``
- ``machine_accel``: Use this option to define a machine acceleration for the VM. Example: ``machine_accel: kvm``.
- ``machine_kvm_type``: Use this option to select the KVM type when the ``accel`` is ``kvm`` and there are more than one KVM types available.
Example: ``machine_kvm_type: PR``
Run the test with::
$ avocado run test_my_test.py -m parameters.yaml
Additionally, you can use a variants file to to set different values for each parameter. Using the YAML tag ``!mux`` Avocado will execute the tests once per combination of parameters. Example::
$ cat variants.yaml
architecture: !mux
x86_64:
arch: x86_64
i386:
arch: i386Run it the with::
$ avocado run test_my_test.py -m variants.yaml
You can use both the parameters file and the variants file in the same command line::
$ avocado run test_my_test.py -m parameters.yaml variants.yaml
Avocado will then merge the parameters from both files and create the proper variants.
See ``avocado run --help`` and ``man avocado`` for several other options, such as ``--filter-by-tags``, ``--show-job-log``, ``--failfast``, etc.
Uninstallation
==============
If you've followed the installation instructions above, you can easily uninstall Avocado. Start by listing the packages you have installed::
pip list --user
And remove any package you want with::
pip uninstall <package_name>
Amador Pahim (12):
qemu.py: Introduce _create_console() method Introduce the basic framework to run Avocado tests
avocado_qemu: Fix exception name in caller
avocado_qemu: Improve migration error message
avocado_qemu: Functional test for RHBZ#1431939
avocado_qemu: Functional test for RHBZ#1447027
avocado_qemu: Functional test for RHBZ#1436616
avocado_qemu: Clean unneeded 'pass'
avocado_qemu: Set QMP log level to INFO
avocado_qemu: Introduce the add_image() VM API
avocado_qemu: Tests fixes
avocado_qemu: Force vmimage distro
Cleber Rosa (3):
avocado_qemu: Remove duplicate PortTracker implementation
avocado_qemu: Simplify the installation instructions
avocado_qemu: Add a few VNC related tests
Lukáš Doktor (9):
avocado_qemu: Improve handle_prompts to allow login after booted vm
avocado_qemu: Be lenient towards poluted serial console
avocado_qemu: Increase the login timeout to 60s
avocado_qemu: Add " " after the default prompt regexp
avocado_qemu: Store "arch" in VM
avocado_qemu: Provide defaults for user and pass
avocado_qemu: Ignore kernel messages on get_console
avocado_qemu: Add support to request image for testing
avocado_qemu: Functional test for RHBZ1473203
scripts/qemu.py | 59 ++- tests/avocado/README.rst | 132 +++++++ tests/avocado/avocado_qemu/__init__.py | 0 tests/avocado/avocado_qemu/test.py | 418 +++++++++++++++++++++ tests/avocado/parameters.yaml | 19 + tests/avocado/test_info_memdev_host_nodes.py | 66 ++++ tests/avocado/test_nec-usb-xhci.py | 63 ++++ .../test_nec-usb-xhci.py.data/parameters.yaml | 4 + tests/avocado/test_numa_hotplug.py | 120 ++++++ tests/avocado/test_ovmf_with_240_vcpus.py | 70 ++++ .../parameters.yaml | 2 + tests/avocado/test_vnc.py | 58 +++ tests/avocado/variants.yaml | 62 +++ tests/qemu-iotests/iotests.py | 28 +- 14 files changed, 1077 insertions(+), 24 deletions(-) create mode 100644 tests/avocado/README.rst create mode 100644 tests/avocado/avocado_qemu/__init__.py create mode 100644 tests/avocado/avocado_qemu/test.py create mode 100644 tests/avocado/parameters.yaml create mode 100644 tests/avocado/test_info_memdev_host_nodes.py create mode 100644 tests/avocado/test_nec-usb-xhci.py create mode 100644 tests/avocado/test_nec-usb-xhci.py.data/parameters.yaml create mode 100644 tests/avocado/test_numa_hotplug.py create mode 100644 tests/avocado/test_ovmf_with_240_vcpus.py create mode 100644 tests/avocado/test_ovmf_with_240_vcpus.py.data/parameters.yaml create mode 100644 tests/avocado/test_vnc.py create mode 100644 tests/avocado/variants.yaml
Signature · 1 line
2.14.3
RFC 1/24 qemu.py: Introduce _create_console() method
From: Amador Pahim <[email protected]>
This patch adds the QEMUMachine._create_console() method, which returns a list with the chardev console device arguments to be used in the qemu command line.
--- scripts/qemu.py | 49 ++++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 44 insertions(+), 5 deletions(-)
scripts/qemu.py +44 -5
@@ -55,7 +55,7 @@ class QEMUMachine(object): def __init__(self, binary, args=None, wrapper=None, name=None, test_dir="/var/tmp", monitor_address=None, - socket_scm_helper=None): + socket_scm_helper=None, arch=None): ''' Initialize a QEMUMachine
@@ -91,6 +91,10 @@ class QEMUMachine(object): self._test_dir = test_dir self._temp_dir = None self._launched = False + if arch is None: + arch = binary.split('-')[-1] + self._arch = arch + self._console_address = None # just in case logging wasn't configured by the main script: logging.basicConfig()
@@ -179,6 +183,39 @@ class QEMUMachine(object): '-mon', 'chardev=mon,mode=control', '-display', 'none', '-vga', 'none'] + def _create_console(self, console_address): + for item in self._args: + for option in ['isa-serial', 'spapr-vty', 'sclpconsole']: + if option in item: + return [] + + chardev = 'socket,id=console,{address},server,nowait' + if console_address is None: + console_address = tempfile.mktemp() + chardev = chardev.format(address='path=%s' % + console_address) + elif isinstance(console_address, tuple): + chardev = chardev.format(address='host=%s,port=%s' % + (console_address[0], + console_address[1])) + else: + chardev = chardev.format(address='path=%s' % console_address) + + self._console_address = console_address + + device = '{dev_type},chardev=console' + if '86' in self._arch: + device = device.format(dev_type='isa-serial') + elif 'ppc' in self._arch: + device = device.format(dev_type='spapr-vty') + elif 's390x' in self._arch: + device = device.format(dev_type='sclpconsole') + else: + return [] + + return ['-chardev', chardev, + '-device', device] + def _pre_launch(self): self._temp_dir = tempfile.mkdtemp(dir=self._test_dir) if self._monitor_address is not None:
@@ -206,7 +243,7 @@ class QEMUMachine(object): shutil.rmtree(self._temp_dir) self._temp_dir = None - def launch(self): + def launch(self, console_address=None): """ Launch the VM and make sure we cleanup and expose the command line/output in case of exception
@@ -218,7 +255,7 @@ class QEMUMachine(object): self._iolog = None self._qemu_full_args = None try: - self._launch() + self._launch(console_address) self._launched = True except: self.shutdown()
@@ -230,12 +267,14 @@ class QEMUMachine(object): LOG.debug('Output: %r', self._iolog) raise - def _launch(self): + def _launch(self, console_address): '''Launch the VM and establish a QMP connection''' devnull = open(os.path.devnull, 'rb') self._pre_launch() + bargs = self._base_args() + bargs.extend(self._create_console(console_address)) self._qemu_full_args = (self._wrapper + [self._binary] + - self._base_args() + self._args) + bargs + self.args) self._popen = subprocess.Popen(self._qemu_full_args, stdin=devnull, stdout=self._qemu_log_file,
Signature · 1 line
2.14.3
Re: [Qemu-devel] [RFC 01/24] qemu.py: Introduce _create_console() method
Quoted 27 lines · Eduardo Habkost
From: Amador Pahim <[email protected]>
This patch adds the
QEMUMachine._create_console()method, which returns a list with the chardev console device arguments to be used in the qemu command line.Signed-off-by Amador Pahim <[email protected]>[ehabkost: reword commit message]Signed-off-by Eduardo Habkost <[email protected]>--- scripts/qemu.py | 49 ++++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 44 insertions(+), 5 deletions(-)scripts/qemu.py +1
-1@@ -55,7 +55,7 @@ class QEMUMachine(object): def __init__(self, binary, args=None, wrapper=None, name=None, test_dir="/var/tmp", monitor_address=None, - socket_scm_helper=None): + socket_scm_helper=None, arch=None): ''' Initialize a QEMUMachine
scripts/qemu.py class QEMUMachine(object): · line 95
Quoted 5 lines
scripts/qemu.py +2
-0@@ -91,6 +91,10 @@ class QEMUMachine(object): self._test_dir = test_dir self._temp_dir = None self._launched = False + if arch is None: + arch = binary.split('-')[-1]
This is not good. We don't want a test case to break only because we renamed a QEMU binary (e.g. RHEL uses /usr/libexec/qemu-kvm, and I can imagine people using qemu.py to write test scripts for it).
Quoted 6 lines
scripts/qemu.py +2
-0+ self._arch = arch + self._console_address = None # just in case logging wasn't configured by the main script: logging.basicConfig()
scripts/qemu.py class QEMUMachine(object): · line 190
Quoted 8 lines
scripts/qemu.py +5
-0@@ -179,6 +183,39 @@ class QEMUMachine(object): '-mon', 'chardev=mon,mode=control', '-display', 'none', '-vga', 'none'] + def _create_console(self, console_address): + for item in self._args: + for option in ['isa-serial', 'spapr-vty', 'sclpconsole']: + if option in item: + return []
This is very fragile. What's the goal of this chunk of code?
Quoted 12 lines
scripts/qemu.py +12
-0+ + chardev = 'socket,id=console,{address},server,nowait' + if console_address is None: + console_address = tempfile.mktemp() + chardev = chardev.format(address='path=%s' % + console_address) + elif isinstance(console_address, tuple): + chardev = chardev.format(address='host=%s,port=%s' % + (console_address[0], + console_address[1])) + else: + chardev = chardev.format(address='path=%s' % console_address)
scripts/qemu.py class QEMUMachine(object): · line 212
Quoted 10 lines
scripts/qemu.py +10
-0+ + self._console_address = console_address + + device = '{dev_type},chardev=console' + if '86' in self._arch: + device = device.format(dev_type='isa-serial') + elif 'ppc' in self._arch: + device = device.format(dev_type='spapr-vty') + elif 's390x' in self._arch: + device = device.format(dev_type='sclpconsole')
Why do we need this? Why isn't -serial enough?
Quoted 47 lines
scripts/qemu.py +12
-4+ else: + return [] + + return ['-chardev', chardev, + '-device', device] + def _pre_launch(self): self._temp_dir = tempfile.mkdtemp(dir=self._test_dir) if self._monitor_address is not None:@@ -206,7 +243,7 @@ class QEMUMachine(object): shutil.rmtree(self._temp_dir) self._temp_dir = None - def launch(self): + def launch(self, console_address=None): """ Launch the VM and make sure we cleanup and expose the command line/output in case of exception@@ -218,7 +255,7 @@ class QEMUMachine(object): self._iolog = None self._qemu_full_args = None try: - self._launch() + self._launch(console_address) self._launched = True except: self.shutdown()@@ -230,12 +267,14 @@ class QEMUMachine(object): LOG.debug('Output: %r', self._iolog) raise - def _launch(self): + def _launch(self, console_address): '''Launch the VM and establish a QMP connection''' devnull = open(os.path.devnull, 'rb') self._pre_launch() + bargs = self._base_args() + bargs.extend(self._create_console(console_address)) self._qemu_full_args = (self._wrapper + [self._binary] + - self._base_args() + self._args) + bargs + self.args) self._popen = subprocess.Popen(self._qemu_full_args, stdin=devnull, stdout=self._qemu_log_file,Signature · 1 line
2.14.3
Signature · 1 line
Eduardo
Re: [Qemu-devel] [RFC 01/24] qemu.py: Introduce _create_console() method
Quoted 19 lines · Eduardo Habkost
Quoted 18 lines · Eduardo Habkost
From: Amador Pahim <[email protected]>
This patch adds the
QEMUMachine._create_console()method, which returns a list with the chardev console device arguments to be used in the qemu command line.Signed-off-by Amador Pahim <[email protected]>[ehabkost: reword commit message]Signed-off-by Eduardo Habkost <[email protected]>--- scripts/qemu.py | 49 ++++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 44 insertions(+), 5 deletions(-)scripts/qemu.py +0
-0@@ -55,7 +55,7 @@ class QEMUMachine(object):
[...]
Quoted 15 lines
Quoted 15 lines
scripts/qemu.py +15
-0+ chardev = 'socket,id=console,{address},server,nowait' + if console_address is None: + console_address = tempfile.mktemp() + chardev = chardev.format(address='path=%s' % + console_address) + elif isinstance(console_address, tuple): + chardev = chardev.format(address='host=%s,port=%s' % + (console_address[0], + console_address[1])) + else: + chardev = chardev.format(address='path=%s' % console_address) + + self._console_address = console_address + + device = '{dev_type},chardev=console'
Quoted 8 lines
scripts/qemu.py class QEMUMachine(object): · line 212
Quoted 6 lines
scripts/qemu.py +6
-0+ if '86' in self._arch: + device = device.format(dev_type='isa-serial') + elif 'ppc' in self._arch: + device = device.format(dev_type='spapr-vty') + elif 's390x' in self._arch: + device = device.format(dev_type='sclpconsole')Why do we need this? Why isn't -serial enough?
AFAIK, at least on s390x, -serial is not implemented and you need to specify a sclpconsole device.
Thomas
Re: [Qemu-devel] [RFC 01/24] qemu.py: Introduce _create_console() method
Quoted 45 lines · Thomas Huth
Quoted 19 lines · Eduardo Habkost
Quoted 18 lines · Eduardo Habkost
From: Amador Pahim <[email protected]>
This patch adds the
QEMUMachine._create_console()method, which returns a list with the chardev console device arguments to be used in the qemu command line.Signed-off-by Amador Pahim <[email protected]>[ehabkost: reword commit message]Signed-off-by Eduardo Habkost <[email protected]>--- scripts/qemu.py | 49 ++++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 44 insertions(+), 5 deletions(-)scripts/qemu.py +0
-0@@ -55,7 +55,7 @@ class QEMUMachine(object):[...]
Quoted 15 lines
Quoted 15 lines
scripts/qemu.py +15
-0+ chardev = 'socket,id=console,{address},server,nowait' + if console_address is None: + console_address = tempfile.mktemp() + chardev = chardev.format(address='path=%s' % + console_address) + elif isinstance(console_address, tuple): + chardev = chardev.format(address='host=%s,port=%s' % + (console_address[0], + console_address[1])) + else: + chardev = chardev.format(address='path=%s' % console_address) + + self._console_address = console_address + + device = '{dev_type},chardev=console'Quoted 8 lines
scripts/qemu.py class QEMUMachine(object): · line 212
Quoted 6 lines
scripts/qemu.py +6
-0+ if '86' in self._arch: + device = device.format(dev_type='isa-serial') + elif 'ppc' in self._arch: + device = device.format(dev_type='spapr-vty') + elif 's390x' in self._arch: + device = device.format(dev_type='sclpconsole')Why do we need this? Why isn't -serial enough?
Quoted 2 lines
AFAIK, at least on s390x, -serial is not implemented and you need to specify a sclpconsole device.
It sounds like it would be useful if it did implement it. Does anybody see any drawbacks?
Signature · 1 line
Eduardo
Re: [Qemu-devel] [RFC 01/24] qemu.py: Introduce _create_console() method
In reply to Eduardo HabkostQuoted 27 lines · Eduardo Habkost
Quoted 26 lines · Eduardo Habkost
From: Amador Pahim <[email protected]>
This patch adds the
QEMUMachine._create_console()method, which returns a list with the chardev console device arguments to be used in the qemu command line.Signed-off-by Amador Pahim <[email protected]>[ehabkost: reword commit message]Signed-off-by Eduardo Habkost <[email protected]>--- scripts/qemu.py | 49 ++++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 44 insertions(+), 5 deletions(-)scripts/qemu.py +1
-1@@ -55,7 +55,7 @@ class QEMUMachine(object): def __init__(self, binary, args=None, wrapper=None, name=None, test_dir="/var/tmp", monitor_address=None, - socket_scm_helper=None): + socket_scm_helper=None, arch=None): ''' Initialize a QEMUMachine
Quoted 13 lines
scripts/qemu.py class QEMUMachine(object): · line 95
Quoted 6 lines
scripts/qemu.py +2
-0@@ -91,6 +91,10 @@ class QEMUMachine(object): self._test_dir = test_dir self._temp_dir = None self._launched = False + if arch is None: + arch = binary.split('-')[-1]This is not good. We don't want a test case to break only because we renamed a QEMU binary (e.g. RHEL uses
/usr/libexec/qemu-kvm, and I can imagine people usingqemu.pyto write test scripts for it).
Agreed. This is a simple but fragile way of getting the arch for a binary. How about getting this from the QEMU binary itself using a "query-target" QMP command? This would, for the sake of simplicity, require a separate QEMU execution, though.
But an even more important usability issue is this "automagic" behavior.
While it may be useful for higher level APIs, such as
"avocado_qemu.Test", IMO it should be explicitly enabled, and current behavior should be preserved. Only by doing something like
m = QEMUMachine("/path/to/binary", automatic_devices=True)
Devices based on the architecture (and machine type?) would be added by default. Naming is hard, so suggestions for the option name are appreciated.
Quoted 9 lines
Quoted 9 lines
scripts/qemu.py +2
-0+ self._arch = arch + self._console_address = None # just in case logging wasn't configured by the main script: logging.basicConfig()@@ -179,6 +183,39 @@ class QEMUMachine(object): '-mon', 'chardev=mon,mode=control', '-display', 'none', '-vga', 'none']
Quoted 9 lines
scripts/qemu.py class QEMUMachine(object): · line 190
Quoted 5 lines
scripts/qemu.py +5
-0+ def _create_console(self, console_address): + for item in self._args: + for option in ['isa-serial', 'spapr-vty', 'sclpconsole']: + if option in item: + return []This is very fragile. What's the goal of this chunk of code?
Agreed again.
Quoted 12 lines
Quoted 12 lines
scripts/qemu.py +12
-0+ + chardev = 'socket,id=console,{address},server,nowait' + if console_address is None: + console_address = tempfile.mktemp() + chardev = chardev.format(address='path=%s' % + console_address) + elif isinstance(console_address, tuple): + chardev = chardev.format(address='host=%s,port=%s' % + (console_address[0], + console_address[1])) + else: + chardev = chardev.format(address='path=%s' % console_address)
Quoted 8 lines
scripts/qemu.py class QEMUMachine(object): · line 210
Quoted 8 lines
scripts/qemu.py +8
-0+ + self._console_address = console_address + + device = '{dev_type},chardev=console' + if '86' in self._arch: + device = device.format(dev_type='isa-serial') + elif 'ppc' in self._arch: + device = device.format(dev_type='spapr-vty')
In my executions with both ppc and ppc64, it failed with ppc not supporting spapr-vty. I'd go with exact arch matches, and not approximations.
Quoted 2 lines
scripts/qemu.py class QEMUMachine(object): · line 212
Quoted 2 lines
scripts/qemu.py +2
-0+ elif 's390x' in self._arch: + device = device.format(dev_type='sclpconsole')
And this is fragile because s390x won't allow for more than one operator console. Exact qemu-system-s390x output is:
Multiple VT220 operator consoles are not supported SCLP event initialization failed.
Quoted 4 lines
Why do we need this? Why isn't -serial enough?
No, some arches need special handling, but this is indeed too fragile.
Quoted 13 lines
Quoted 13 lines
scripts/qemu.py +6
-0+ else: + return [] + + return ['-chardev', chardev, + '-device', device] + def _pre_launch(self): self._temp_dir = tempfile.mkdtemp(dir=self._test_dir) if self._monitor_address is not None:@@ -206,7 +243,7 @@ class QEMUMachine(object): shutil.rmtree(self._temp_dir) self._temp_dir = None
Quoted 2 lines
scripts/qemu.py class QEMUMachine(object): · line 246
Quoted 2 lines
scripts/qemu.py +1
-1- def launch(self): + def launch(self, console_address=None):
Querying the console address should be available to users, but I don't see the reason for making it a launch() argument.
Quoted 16 lines
Quoted 16 lines
scripts/qemu.py +1
-1""" Launch the VM and make sure we cleanup and expose the command line/output in case of exception@@ -218,7 +255,7 @@ class QEMUMachine(object): self._iolog = None self._qemu_full_args = None try: - self._launch() + self._launch(console_address) self._launched = True except: self.shutdown()@@ -230,12 +267,14 @@ class QEMUMachine(object): LOG.debug('Output: %r', self._iolog) raise
Quoted 10 lines
scripts/qemu.py class QEMUMachine(object): · line 277
Quoted 10 lines
scripts/qemu.py +4
-2- def _launch(self): + def _launch(self, console_address): '''Launch the VM and establish a QMP connection''' devnull = open(os.path.devnull, 'rb') self._pre_launch() + bargs = self._base_args() + bargs.extend(self._create_console(console_address)) self._qemu_full_args = (self._wrapper + [self._binary] + - self._base_args() + self._args) + bargs + self.args)
With the addition of "automagic" devices, I'd propose three internal argument categories:
1) base args (self._base_args()), which will always be necessary and
created by QEMUMachine.
2) auto args, always empty if QEMUMachine() is not given
automatic_devices=True.
3) user args (self._args)
- Cleber.
Quoted 7 lines
Quoted 6 lines
scripts/qemu.py +0
-0self._popen = subprocess.Popen(self._qemu_full_args, stdin=devnull, stdout=self._qemu_log_file,Signature · 1 line
2.14.3
RFC 2/24 Introduce the basic framework to run Avocado tests
In reply to Eduardo HabkostFrom: Amador Pahim <[email protected]>
Avocado Testing Framework can help with functional tests in qemu development process. This patch creates the basic infrastructure to use with Avocado tests. It contains:
- A README file with the initial documentation.
- The test module which inherits from avocado.Test and adds a VM object, created from scripts/qemu.py module. The QemuTest class is the test API for the actual Qemu tests to inherit from.
- A parameters yaml file with the supported keys.
- A variants yaml file with the Qemu supported architectures.
After this commit, you can expect a series of real-world tests, written using this new framework.
--- scripts/qemu.py | 12 +- tests/avocado/README.rst | 111 ++++++++++ tests/avocado/avocado_qemu/__init__.py | 0 tests/avocado/avocado_qemu/test.py | 365 +++++++++++++++++++++++++++++++++ tests/avocado/parameters.yaml | 28 +++ tests/avocado/variants.yaml | 62 ++++++ tests/qemu-iotests/iotests.py | 28 +-- 7 files changed, 586 insertions(+), 20 deletions(-) create mode 100644 tests/avocado/README.rst create mode 100644 tests/avocado/avocado_qemu/__init__.py create mode 100644 tests/avocado/avocado_qemu/test.py create mode 100644 tests/avocado/parameters.yaml create mode 100644 tests/avocado/variants.yaml
scripts/qemu.py +6 -6
@@ -81,7 +81,7 @@ class QEMUMachine(object): self._qemu_log_file = None self._popen = None self._binary = binary - self._args = list(args) # Force copy args in case we modify them + self.args = list(args) # Force copy args in case we modify them self._wrapper = wrapper self._events = [] self._iolog = None
@@ -109,8 +109,8 @@ class QEMUMachine(object): # This can be used to add an unused monitor instance. def add_monitor_telnet(self, ip, port): args = 'tcp:%s:%d,server,nowait,telnet' % (ip, port) - self._args.append('-monitor') - self._args.append(args) + self.args.append('-monitor') + self.args.append(args) def add_fd(self, fd, fdset, opaque, opts=''): '''Pass a file descriptor to the VM'''
@@ -120,8 +120,8 @@ class QEMUMachine(object): if opts: options.append(opts) - self._args.append('-add-fd') - self._args.append(','.join(options)) + self.args.append('-add-fd') + self.args.append(','.join(options)) return self def send_fd_scm(self, fd_file_path):
@@ -184,7 +184,7 @@ class QEMUMachine(object): '-display', 'none', '-vga', 'none'] def _create_console(self, console_address): - for item in self._args: + for item in self.args: for option in ['isa-serial', 'spapr-vty', 'sclpconsole']: if option in item: return []
tests/avocado/README.rst new file +111 -0
111 lines not on this page · raw message
tests/avocado/avocado_qemu/__init__.py new file +0 -0
tests/avocado/avocado_qemu/test.py new file +365 -0
365 lines not on this page · raw message
tests/avocado/parameters.yaml new file +28 -0
@@ -0,0 +1,28 @@ +# Probe the Qemu binary from a given architecture. It has no effect if +# 'qemu_bin' is specified. If not provided, the binary probe will use +# the local system architecture. +arch: null + +# Use a given Qemu binary, skipping the automatic probe. +qemu_bin: null +# Use a given Qemu binary to create the destination VM when the +# migration process is called. If it's not provided, the same binary +# used in the source VM will be used for the destination VM. +qemu_dst_bin: null + +# VMs are defined without image. If the 'image_path' is specified, it +# will be used as the VM image. The '-snapshot' option will then be used +# to avoid writing data to the image. +image_path: null +# Username used to get the console from the Guest OS. +image_user: null +# Password used to get the console from the Guest OS. +image_pass: null + +# Use this option to define a machine type for the VM. +machine_type: null +# Use this option to define a machine acceleration for the VM. +machine_accel: null +# Use this option to select the KVM type when the 'machine_accel' is set +# to 'kvm' and there are more than one KVM types available. +machine_kvm_type: null
tests/avocado/variants.yaml new file +62 -0
62 lines not on this page · raw message
tests/qemu-iotests/iotests.py +14 -14
@@ -293,18 +293,18 @@ class VM(qtest.QEMUQtestMachine): self._num_drives = 0 def add_object(self, opts): - self._args.append('-object') - self._args.append(opts) + self.args.append('-object') + self.args.append(opts) return self def add_device(self, opts): - self._args.append('-device') - self._args.append(opts) + self.args.append('-device') + self.args.append(opts) return self def add_drive_raw(self, opts): - self._args.append('-drive') - self._args.append(opts) + self.args.append('-drive') + self.args.append(opts) return self def add_drive(self, path, opts='', interface='virtio', format=imgfmt):
@@ -322,27 +322,27 @@ class VM(qtest.QEMUQtestMachine): if format == 'luks' and 'key-secret' not in opts: # default luks support - if luks_default_secret_object not in self._args: + if luks_default_secret_object not in self.args: self.add_object(luks_default_secret_object) options.append(luks_default_key_secret_opt) - self._args.append('-drive') - self._args.append(','.join(options)) + self.args.append('-drive') + self.args.append(','.join(options)) self._num_drives += 1 return self def add_blockdev(self, opts): - self._args.append('-blockdev') + self.args.append('-blockdev') if isinstance(opts, str): - self._args.append(opts) + self.args.append(opts) else: - self._args.append(','.join(opts)) + self.args.append(','.join(opts)) return self def add_incoming(self, addr): - self._args.append('-incoming') - self._args.append(addr) + self.args.append('-incoming') + self.args.append(addr) return self def pause_drive(self, drive, event=None):
Signature · 1 line
2.14.3
Re: [Qemu-devel] [RFC 02/24] Introduce the basic framework to run Avocado tests
Quoted 38 lines · Eduardo Habkost
From: Amador Pahim <[email protected]>
Avocado Testing Framework can help with functional tests in qemu development process. This patch creates the basic infrastructure to use with Avocado tests. It contains:
- A README file with the initial documentation.
- The test module which inherits from avocado.Test and adds a VM object, created fromscripts/qemu.pymodule. The QemuTest class is the test API for the actual Qemu tests to inherit from.
- A parameters yaml file with the supported keys.
- A variants yaml file with the Qemu supported architectures.After this commit, you can expect a series of real-world tests, written using this new framework.
Signed-off-by Amador Pahim <[email protected]>Signed-off-by Eduardo Habkost <[email protected]>--- scripts/qemu.py | 12 +- tests/avocado/README.rst | 111 ++++++++++ tests/avocado/avocado_qemu/__init__.py | 0 tests/avocado/avocado_qemu/test.py | 365 +++++++++++++++++++++++++++++++++ tests/avocado/parameters.yaml | 28 +++ tests/avocado/variants.yaml | 62 ++++++ tests/qemu-iotests/iotests.py | 28 +-- 7 files changed, 586 insertions(+), 20 deletions(-) create mode 100644 tests/avocado/README.rst create mode 100644 tests/avocado/avocado_qemu/__init__.py create mode 100644 tests/avocado/avocado_qemu/test.py create mode 100644 tests/avocado/parameters.yaml create mode 100644 tests/avocado/variants.yamlscripts/qemu.py +0
-0
scripts/qemu.py class QEMUMachine(object): · line 84
Quoted 5 lines
scripts/qemu.py +1
-1@@ -81,7 +81,7 @@ class QEMUMachine(object): self._qemu_log_file = None self._popen = None self._binary = binary - self._args = list(args) # Force copy args in case we modify them + self.args = list(args) # Force copy args in case we modify them
This should go to a separate patch.
Also, I would prefer a self.add_args(...) interface, instead of exposing self.args to the outside directly. I remember some past discussions about this, but I don't remember what was the conclusion.
I will review tests/avocado/* later.
Quoted 34 lines
scripts/qemu.py +5
-5self._wrapper = wrapper self._events = [] self._iolog = None@@ -109,8 +109,8 @@ class QEMUMachine(object): # This can be used to add an unused monitor instance. def add_monitor_telnet(self, ip, port): args = 'tcp:%s:%d,server,nowait,telnet' % (ip, port) - self._args.append('-monitor') - self._args.append(args) + self.args.append('-monitor') + self.args.append(args) def add_fd(self, fd, fdset, opaque, opts=''): '''Pass a file descriptor to the VM'''@@ -120,8 +120,8 @@ class QEMUMachine(object): if opts: options.append(opts) - self._args.append('-add-fd') - self._args.append(','.join(options)) + self.args.append('-add-fd') + self.args.append(','.join(options)) return self def send_fd_scm(self, fd_file_path):@@ -184,7 +184,7 @@ class QEMUMachine(object): '-display', 'none', '-vga', 'none'] def _create_console(self, console_address): - for item in self._args: + for item in self.args: for option in ['isa-serial', 'spapr-vty', 'sclpconsole']: if option in item: return []
[...]
Signature · 1 line
Eduardo
RFC 3/24 avocado_qemu: Improve handle_prompts to allow login after booted vm
In reply to Eduardo HabkostFrom: Lukáš Doktor <[email protected]>
When the VM is booted before calling "vm.get_console" this command fails to login as there is no new output in the console. Let's just press enter (which should be relatively harmless as login asks for the user again and when already logged in we might "only" execute what's already written.
--- tests/avocado/avocado_qemu/test.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
tests/avocado/avocado_qemu/test.py +2 -1
@@ -123,7 +123,8 @@ def _handle_prompts(session, username, password, prompt, timeout=10, password_prompt_count = 0 login_prompt_count = 0 last_chance = False - + # Send enter to refresh output (in case session was attached after boot) + session.sendline() output = "" while True: try:
Signature · 1 line
2.14.3
RFC 4/24 avocado_qemu: Be lenient towards poluted serial console
In reply to Eduardo HabkostFrom: Lukáš Doktor <[email protected]>
The serial console is frequently spammed kernel messages, let's not be so strict and match lines also containing other parts.
There is still possibility of failure in case kernel injects message on a new line after the expected output producing non-matching last_line. Anyway checking for any-line would be a bit too prone to false-positives so let's just go with this at this point.
--- tests/avocado/avocado_qemu/test.py | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
tests/avocado/avocado_qemu/test.py +3 -3
@@ -131,9 +131,9 @@ def _handle_prompts(session, username, password, prompt, timeout=10, match, text = session.read_until_last_line_matches( [r"[Aa]re you sure", r"[Pp]assword:\s*", # Prompt of rescue mode for Red Hat. - r"\(or (press|type) Control-D to continue\):\s*$", - r"[Gg]ive.*[Ll]ogin:\s*$", # Prompt of rescue mode for SUSE. - r"(?<![Ll]ast )[Ll]ogin:\s*$", # Don't match "Last Login:" + r"\(or (press|type) Control-D to continue\):\s*", + r"[Gg]ive.*[Ll]ogin:\s*", # Prompt of rescue mode for SUSE. + r"(?<![Ll]ast )[Ll]ogin:\s*", # Don't match "Last Login:" r"[Cc]onnection.*closed", r"[Cc]onnection.*refused", r"[Pp]lease wait", r"[Ww]arning", r"[Ee]nter.*username", r"[Ee]nter.*password", r"[Cc]onnection timed out", prompt,
Signature · 1 line
2.14.3
RFC 5/24 avocado_qemu: Increase the login timeout to 60s
In reply to Eduardo HabkostFrom: Lukáš Doktor <[email protected]>
The 10s login is quite short, let's use 60s and see whether it fits.
--- tests/avocado/avocado_qemu/test.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tests/avocado/avocado_qemu/test.py +1 -1
@@ -97,7 +97,7 @@ def _get_qemu_bin(arch): return qemu_binary -def _handle_prompts(session, username, password, prompt, timeout=10, +def _handle_prompts(session, username, password, prompt, timeout=60, debug=False): """ Connect to a remote host (guest) using SSH or Telnet or else.
Signature · 1 line
2.14.3
RFC 6/24 avocado_qemu: Add " " after the default prompt regexp
In reply to Eduardo HabkostFrom: Lukáš Doktor <[email protected]>
The prompt regexp is quite basic one (# or $), let's require at least one additional space afterwards, which is the case in most modern systems and decreases the probability of false-positives.
--- tests/avocado/avocado_qemu/test.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tests/avocado/avocado_qemu/test.py +2 -2
@@ -270,7 +270,7 @@ class _VM(qemu.QEMUMachine): self.password = password super(_VM, self).__init__(qemu_bin, name=self.name, arch=arch) - def get_console(self, console_address=None, prompt="[\#\$]"): + def get_console(self, console_address=None, prompt=r"[\#\$] "): """ :param address: Socket address, can be either a unix socket path (string) or a tuple in the form (address, port)
@@ -297,7 +297,7 @@ class _VM(qemu.QEMUMachine): console = aexpect.ShellSession(nc_cmd) try: logging.info('Console: Waiting login prompt...') - _handle_prompts(console, self.username, self.password, "[\#\$]") + _handle_prompts(console, self.username, self.password, prompt) logging.info('Console: Ready!') except: console.close()
Signature · 1 line
2.14.3
RFC 7/24 avocado_qemu: Store "arch" in VM
In reply to Eduardo HabkostFrom: Lukáš Doktor <[email protected]>
The "arch" is quite important info regarding the current VM, let's store it so tests can query for it.
--- tests/avocado/avocado_qemu/test.py | 1 + 1 file changed, 1 insertion(+)
tests/avocado/avocado_qemu/test.py +1 -0
@@ -258,6 +258,7 @@ class _VM(qemu.QEMUMachine): qemu_dst_bin=None): if arch is None: arch = os.uname()[4] + self.arch = arch self.ports = _PortTracker() self.name = "qemu-%s" % str(uuid.uuid4())[:8] if qemu_bin is None:
Signature · 1 line
2.14.3
RFC 8/24 avocado_qemu: Provide defaults for user and pass
In reply to Eduardo HabkostFrom: Lukáš Doktor <[email protected]>
For conveniency let's set default user to root and password to 123456 to avoid the need to always input those. This password is inspired by the default password of JeOS available from avocado-vt.
--- tests/avocado/avocado_qemu/test.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tests/avocado/avocado_qemu/test.py +2 -2
@@ -344,8 +344,8 @@ class QemuTest(Test): job=job, runner_queue=runner_queue) self.vm = _VM(qemu_bin=self.params.get('qemu_bin'), arch=self.params.get('arch'), - username=self.params.get('image_user'), - password=self.params.get('image_pass'), + username=self.params.get('image_user', default="root"), + password=self.params.get('image_pass', default="123456"), qemu_dst_bin=self.params.get('qemu_dst_bin')) self.vm.image = self.params.get('image_path')
Signature · 1 line
2.14.3
RFC 9/24 avocado_qemu: Ignore kernel messages on get_console
In reply to Eduardo HabkostFrom: Lukáš Doktor <[email protected]>
The get_console (and _handle_prompt) uses the last non-empty line to check what is going on, but when debug is enabled, kernel produces lots of lines spoiling the output. Let's also ignore the messages that looks like kernel debugs ([ $float] $msg).
This significantly improves the results on my machine with JeOS and enabled debug.
--- tests/avocado/avocado_qemu/test.py | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-)
tests/avocado/avocado_qemu/test.py +14 -2
@@ -26,6 +26,7 @@ extra features intended for Qemu testing. import logging import os +import re import sys import time import uuid
@@ -120,6 +121,17 @@ def _handle_prompts(session, username, password, prompt, timeout=60, :return: If connect succeed return the output text to script for further debug. """ + re_kernel_message = re.compile(r"^\[\s*\d+.\d+\] ") + + def get_last_nonempty_line(cont): + """Return last non-empty non-kernel line""" + nonempty_lines = [_ for _ in cont.splitlines() + if _.strip() and not re_kernel_message.match(_)] + if nonempty_lines: + return nonempty_lines[-1] + else: + return "" + password_prompt_count = 0 login_prompt_count = 0 last_chance = False
@@ -128,7 +140,7 @@ def _handle_prompts(session, username, password, prompt, timeout=60, output = "" while True: try: - match, text = session.read_until_last_line_matches( + match, text = session.read_until_output_matches( [r"[Aa]re you sure", r"[Pp]assword:\s*", # Prompt of rescue mode for Red Hat. r"\(or (press|type) Control-D to continue\):\s*",
@@ -137,7 +149,7 @@ def _handle_prompts(session, username, password, prompt, timeout=60, r"[Cc]onnection.*closed", r"[Cc]onnection.*refused", r"[Pp]lease wait", r"[Ww]arning", r"[Ee]nter.*username", r"[Ee]nter.*password", r"[Cc]onnection timed out", prompt, - r"Escape character is.*"], + r"Escape character is.*"], get_last_nonempty_line, timeout=timeout, internal_timeout=0.5) output += text if match == 0: # "Are you sure you want to continue connecting"
Signature · 1 line
2.14.3
RFC 10/24 avocado_qemu: Add support to request image for testing
In reply to Eduardo HabkostFrom: Lukáš Doktor <[email protected]>
Some of the tests require (usually bootable) image to be executed, let's add a helper function which uses the default params to define the image or provides useful default and explains what's going on in case the image is not available.
--- tests/avocado/README.rst | 12 ++++++----- tests/avocado/avocado_qemu/test.py | 44 ++++++++++++++++++++++++++++++++------ 2 files changed, 44 insertions(+), 12 deletions(-)
tests/avocado/README.rst +7 -5
@@ -59,14 +59,16 @@ file using the Avocado parameters system: - ``arch``: Probe the Qemu binary from a given architecture. It has no effect if ``qemu_bin`` is specified. If not provided, the binary probe will use the system architecture. Example: ``arch: x86_64`` -- ``image_path``: VMs are defined without image. If the ``image_path`` - is specified, it will be used as the VM image. The ``-snapshot`` - option will then be used to avoid writing into the image. Example: - ``image_path: /var/lib/images/fedora-25.img`` +- ``image_path``: When a test requires (usually a bootable) image, this + parameter is used to define where the image is located. When undefined + it uses ``$QEMU_ROOT/bootable_image_$arch.qcow2``. The image is added + to the qemu command __only__ when the test requires an image. By + default ``,snapshot=on`` is used, but it can be altered by + ``image_snapshot`` parameter. - ``image_user`` and ``image_pass``: When using a ``image_path``, if you want to get the console from the Guest OS you have to define the Guest OS credentials. Example: ``image_user: root`` and - ``image_pass: p4ssw0rd`` + ``image_pass: p4ssw0rd``. By default it uses ``root`` and ``123456``. - ``machine_type``: Use this option to define a machine type for the VM. Example: ``machine_type: pc`` - ``machine_accel``: Use this option to define a machine acceleration
tests/avocado/avocado_qemu/test.py +37 -7
59 lines not on this page · raw message
Signature · 1 line
2.14.3
RFC 11/24 avocado_qemu: Fix exception name in caller
In reply to Eduardo HabkostFrom: Amador Pahim <[email protected]>
There's a exception name mismatch when callin QEMUMigrationError. Fixing.
--- tests/avocado/avocado_qemu/test.py | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
tests/avocado/avocado_qemu/test.py +3 -3
@@ -328,7 +328,7 @@ class _VM(qemu.QEMUMachine): logging.info("Migration successful") return True elif 'failed' in res['return']: - raise QEMUMigrateError("Migration of %s failed" % self) + raise QEMUMigrationError("Migration of %s failed" % self) return False port = self.ports.find_free_port()
@@ -343,8 +343,8 @@ class _VM(qemu.QEMUMachine): text='Waiting for migration to complete') if mig_result is None: - raise QEMUMigrateError("Migration of %s did not complete after " - "%s s" % (self.name, timeout)) + raise QEMUMigrationError("Migration of %s did not complete after " + "%s s" % (self.name, timeout)) return newvm
Signature · 1 line
2.14.3
RFC 12/24 avocado_qemu: Improve migration error message
In reply to Eduardo HabkostFrom: Amador Pahim <[email protected]>
Expose the QMP result on migration error.
--- tests/avocado/avocado_qemu/test.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
tests/avocado/avocado_qemu/test.py +2 -1
@@ -328,7 +328,8 @@ class _VM(qemu.QEMUMachine): logging.info("Migration successful") return True elif 'failed' in res['return']: - raise QEMUMigrationError("Migration of %s failed" % self) + logging.error(res) + raise QEMUMigrationError("Migration of %s failed" % self.name) return False port = self.ports.find_free_port()
Signature · 1 line
2.14.3
RFC 13/24 avocado_qemu: Functional test for RHBZ#1431939
In reply to Eduardo HabkostFrom: Amador Pahim <[email protected]>
This issue was fixed in commit d81d857.
According to the RHBZ1431939, the issue is 'host nodes' returning '128'. It should return empty value when using default policy or 0 when using bind policy.
Test consists in inspect the result of the 'info memdev' QMP command after hot-adding memory.
--- tests/avocado/test_info_memdev_host_nodes.py | 66 ++++++++++++++++++++++++++++ 1 file changed, 66 insertions(+) create mode 100644 tests/avocado/test_info_memdev_host_nodes.py
tests/avocado/test_info_memdev_host_nodes.py new file +66 -0
@@ -0,0 +1,66 @@ +from avocado import main +from avocado_qemu import test + + +class TestInfoMemdev(test.QemuTest): + """ + + :avocado: enable + :avocado: tags=qmp,object_add,device_add,memdev + """ + + def setUp(self): + self.vm.args.extend(['-m', '4G,slots=32,maxmem=40G']) + self.vm.launch() + + def test_hotplug_memory_default_policy(self): + """ + According to the RHBZ1431939, the issue is 'host nodes' + returning '128'. It should return empty value when memory + hotplug default policy is used. + + Fixed in commit d81d857f4421d205395d55200425daa6591c28a5. + :avocado: tags=RHBZ1431939 + """ + + cmd = 'object_add memory-backend-ram,id=mem1,size=1G' + res = self.vm.qmp('human-monitor-command', command_line=cmd) + self.assertEqual('', res['return']) + + cmd = 'device_add pc-dimm,id=dimm1,memdev=mem1' + res = self.vm.qmp('human-monitor-command', command_line=cmd) + self.assertEqual('', res['return']) + + cmd = 'info memdev' + res = self.vm.qmp('human-monitor-command', command_line=cmd) + self.assertIn('policy: default\r', res['return']) + self.assertIn('host nodes: \r', res['return']) + + def test_hotplug_memory_bind_policy(self): + """ + According to the RHBZ1431939, the issue is 'host nodes' + returning '128'. It should return 0 when memory hotplug + bind policy is used. + + Fixed in commit d81d857f4421d205395d55200425daa6591c28a5. + :avocado: tags=RHBZ1431939 + """ + + cmd = 'object_add memory-backend-ram,id=mem1,host-nodes=0,size=2G,policy=bind' + res = self.vm.qmp('human-monitor-command', command_line=cmd) + self.assertEqual('', res['return']) + + cmd = 'device_add pc-dimm,id=dimm1,memdev=mem1' + res = self.vm.qmp('human-monitor-command', command_line=cmd) + self.assertEqual('', res['return']) + + cmd = 'info memdev' + res = self.vm.qmp('human-monitor-command', command_line=cmd) + self.assertIn('policy: bind\r', res['return']) + self.assertIn('host nodes: 0\r', res['return']) + + def tearDown(self): + self.vm.shutdown() + +if __name__ == "__main__": + avocado.main()
Signature · 1 line
2.14.3
Re: [Qemu-devel] [RFC 13/24] avocado_qemu: Functional test for RHBZ#1431939
tests/avocado/test_info_memdev_host_nodes.py line 28
Quoted 13 lines · Eduardo Habkost
tests/avocado/test_info_memdev_host_nodes.py new file +13
-0+ def test_hotplug_memory_default_policy(self): + """ + According to the RHBZ1431939, the issue is 'host nodes' + returning '128'. It should return empty value when memory + hotplug default policy is used. + + Fixed in commit d81d857f4421d205395d55200425daa6591c28a5. + :avocado: tags=RHBZ1431939 + """ + + cmd = 'object_add memory-backend-ram,id=mem1,size=1G' + res = self.vm.qmp('human-monitor-command', command_line=cmd) + self.assertEqual('', res['return'])
General question about QMP test coding style:
What happens if res['return'] does not exist because the QMP command failed?
I tend to use dict.get() to prevent KeyError. That way the assertEqual() will fail instead of an unhandled KeyError in the test code.
Stefan
Re: [Qemu-devel] [RFC 13/24] avocado_qemu: Functional test for RHBZ#1431939
Quoted 17 lines · Stefan Hajnoczi
tests/avocado/test_info_memdev_host_nodes.py line 28
Quoted 13 lines · Eduardo Habkost
tests/avocado/test_info_memdev_host_nodes.py new file +13
-0+ def test_hotplug_memory_default_policy(self): + """ + According to the RHBZ1431939, the issue is 'host nodes' + returning '128'. It should return empty value when memory + hotplug default policy is used. + + Fixed in commit d81d857f4421d205395d55200425daa6591c28a5. + :avocado: tags=RHBZ1431939 + """ + + cmd = 'object_add memory-backend-ram,id=mem1,size=1G' + res = self.vm.qmp('human-monitor-command', command_line=cmd) + self.assertEqual('', res['return'])General question about QMP test coding style:
Quoted 6 lines
What happens if res['return'] does not exist because the QMP command failed?
I tend to use
dict.get()to prevent KeyError. That way theassertEqual()will fail instead of an unhandled KeyError in the test code.
It looks like vm.command() would be appropriate on most cases, as it will check for errors and return res['result'] automatically.
vm.qmp() seems to be useful only if you really don't want an exception to be raised in case of QMP errors.
Maybe we should rename .qmp() to .raw_qmp() to discourage people from using it.
Signature · 1 line
Eduardo
Re: [Qemu-devel] [RFC 13/24] avocado_qemu: Functional test for RHBZ#1431939
Quoted 25 lines · Eduardo Habkost
Quoted 17 lines · Stefan Hajnoczi
tests/avocado/test_info_memdev_host_nodes.py line 28
Quoted 13 lines · Eduardo Habkost
tests/avocado/test_info_memdev_host_nodes.py new file +13
-0+ def test_hotplug_memory_default_policy(self): + """ + According to the RHBZ1431939, the issue is 'host nodes' + returning '128'. It should return empty value when memory + hotplug default policy is used. + + Fixed in commit d81d857f4421d205395d55200425daa6591c28a5. + :avocado: tags=RHBZ1431939 + """ + + cmd = 'object_add memory-backend-ram,id=mem1,size=1G' + res = self.vm.qmp('human-monitor-command', command_line=cmd) + self.assertEqual('', res['return'])General question about QMP test coding style:
Quoted 6 lines
What happens if res['return'] does not exist because the QMP command failed?
I tend to use
dict.get()to prevent KeyError. That way theassertEqual()will fail instead of an unhandled KeyError in the test code.
Quoted 8 lines
It looks like
vm.command()would be appropriate on most cases, as it will check for errors and return res['result'] automatically.
vm.qmp()seems to be useful only if you really don't want an exception to be raised in case of QMP errors.Maybe we should rename .qmp() to .raw_qmp() to discourage people from using it.
Sounds good.
Stefan
RFC 14/24 avocado_qemu: Functional test for RHBZ#1447027
In reply to Eduardo HabkostFrom: Amador Pahim <[email protected]>
Fixed in commit e85c0d1.
According to the RHBZ1447027, the issue is: guest cannot boot with 240 or above vcpus when using ovmf.
Test consists in set the VM with 240 vcpus and the OVMF device to then check whether the VM can be started.
A parameters.yaml file is provided in order to customize the OVMF files locations. Execute with:
$ avocado run test_ovmf_with_240_vcpus.py -m \
test_ovmf_with_240_vcpus.py.data/parameters.yaml--- tests/avocado/test_ovmf_with_240_vcpus.py | 70 ++++++++++++++++++++++ .../parameters.yaml | 2 + 2 files changed, 72 insertions(+) create mode 100644 tests/avocado/test_ovmf_with_240_vcpus.py create mode 100644 tests/avocado/test_ovmf_with_240_vcpus.py.data/parameters.yaml
tests/avocado/test_ovmf_with_240_vcpus.py new file +70 -0
@@ -0,0 +1,70 @@ +import os +import shutil +import sys + +from avocado import main +from avocado_qemu import test + + +class TestOvmfVcpus(test.QemuTest): + """ + Run with: + + avocado run test_ovmf_with_240_vcpus.py \ + -m test_ovmf_with_240_vcpus.py.data/parameters.yaml + + :avocado: enable + :avocado: tags=ovmf + """ + + def setUp(self): + ovmf_code_path = self.params.get('OVMF_CODE', + default='/usr/share/edk2/ovmf/OVMF_CODE.secboot.fd') + ovmf_vars_path = self.params.get('OVMF_VARS', + default='/usr/share/edk2/ovmf/OVMF_VARS.fd') + if not ovmf_code_path or not os.path.exists(ovmf_code_path): + basename = os.path.basename(__file__) + self.cancel('OVMF_CODE file not found. Set the correct ' + 'path on "%s.data/parameters.yaml" and run this test ' + 'with: "avocado run %s -m %s.data/parameters.yaml"' % + (basename, basename, basename)) + if not ovmf_vars_path or not os.path.exists(ovmf_vars_path): + basename = os.path.basename(__file__) + self.cancel('OVMF_VARS file not found. Set the correct ' + 'path on "%s.data/parameters.yaml" and run this test ' + 'with: "avocado run %s -m %s.data/parameters.yaml"' % + (basename, basename, basename)) + + ovmf_vars_tmp = os.path.join(self.workdir, + os.path.basename(ovmf_vars_path)) + if not os.path.exists(ovmf_vars_tmp): + shutil.copy(ovmf_vars_path, self.workdir) + + self.vm.args.extend(['-drive', + 'file=%s,if=pflash,format=raw,readonly=on,unit=0' % + ovmf_code_path]) + self.vm.args.extend(['-drive', + 'file=%s,if=pflash,format=raw,unit=1' % + ovmf_vars_tmp]) + + self.vm.args.extend(['-smp', '240']) + + def test_run_vm(self): + """ + According to the RHBZ1447027, the issue is: Guest cannot boot + with 240 or above vcpus when using ovmf. + Fixed in commit e85c0d14014514a2f0faeae5b4c23fab5b234de4. + + :avocado: tags=RHBZ1447027 + """ + + try: + self.vm.launch() + except Exception as details: + self.fail(details) + + def tearDown(self): + self.vm.shutdown() + +if __name__ == "__main__": + avocado.main()
tests/avocado/test_ovmf_with_240_vcpus.py.data/parameters.yaml new file +2 -0
@@ -0,0 +1,2 @@ +OVMF_CODE: /usr/share/edk2/ovmf/OVMF_CODE.secboot.fd +OVMF_VARS: /usr/share/edk2/ovmf/OVMF_VARS.fd
Signature · 1 line
2.14.3
RFC 15/24 avocado_qemu: Functional test for RHBZ#1436616
In reply to Eduardo HabkostFrom: Amador Pahim <[email protected]>
Fixed in commit 243afe858b95765b98d16a1f0dd50dca262858ad.
According to the RHBZ1436616, the issue is: usb-storage device under pci-bridge is unusable after migration.
Test consists in create a VM with an usb-storage device and check whether it's still available after a live migration.
A parameters.yaml file is provided in order to customize the guest image location and credentials. Execute with:
avocado run test_nec-usb-xhci.py \
-m test_nec-usb-xhci.py.data/parameters.yaml--- tests/avocado/test_nec-usb-xhci.py | 55 ++++++++++++++++++++++ .../test_nec-usb-xhci.py.data/parameters.yaml | 4 ++ 2 files changed, 59 insertions(+) create mode 100644 tests/avocado/test_nec-usb-xhci.py create mode 100644 tests/avocado/test_nec-usb-xhci.py.data/parameters.yaml
tests/avocado/test_nec-usb-xhci.py new file +55 -0
@@ -0,0 +1,55 @@ +import copy +import os +import tempfile + +from avocado_qemu import test +from avocado.utils import process + +class TestNecUsbXhci(test.QemuTest): + """ + Run with: + + avocado run test_nec-usb-xhci.py \ + -m test_nec-usb-xhci.py.data/parameters.yaml + + :avocado: enable + :avocado: tags=usbstorage + """ + + def setUp(self): + usbdevice = os.path.join(self.workdir, 'usb.img') + self.request_image() + process.run('dd if=/dev/zero of=%s bs=1M count=10' % usbdevice) + self.vm.args.extend(['-device', 'pci-bridge,id=bridge1,chassis_nr=1']) + self.vm.args.extend(['-device', 'nec-usb-xhci,id=xhci1,bus=bridge1,addr=0x3']) + self.vm.args.extend(['-drive', 'file=%s,format=raw,id=drive_usb,if=none' % usbdevice]) + self.vm.args.extend(['-device', 'usb-storage,drive=drive_usb,id=device_usb,bus=xhci1.0']) + self.vm.launch() + + def test_available_after_migration(self): + """ + According to the RHBZ1436616, the issue is: usb-storage device + under pci-bridge is unusable after migration. + + Fixed in commit 243afe858b95765b98d16a1f0dd50dca262858ad. + + :avocado: tags=migration,RHBZ1436616 + """ + console = self.vm.get_console() + console.sendline('fdisk -l') + result = console.read_nonblocking() + console.close() + self.assertIn('Disk /dev/sdb: 10 MiB, 10485760 bytes, 20480 sectors', + result) + + self.vm_dst = self.vm.migrate() + console = self.vm_dst.get_console() + console.sendline('fdisk -l') + result = console.read_nonblocking() + console.close() + self.assertIn('Disk /dev/sdb: 10 MiB, 10485760 bytes, 20480 sectors', + result) + + def tearDown(self): + self.vm.shutdown() + self.vm_dst.shutdown()
tests/avocado/test_nec-usb-xhci.py.data/parameters.yaml new file +4 -0
@@ -0,0 +1,4 @@ +machine_accel: kvm +image_path: /var/lib/images/fedora-25.img +image_user: root +image_pass: p4ssw0rd
Signature · 1 line
2.14.3
RFC 16/24 avocado_qemu: Functional test for RHBZ1473203
In reply to Eduardo HabkostFrom: Lukáš Doktor <[email protected]>
Adds regresion test for RHBZ1473203 which runs VM with 16 numa nodes and then verifies memory allocation is accurate before and after hotplugging memory into default and 13th node.
--- tests/avocado/test_numa_hotplug.py | 116 +++++++++++++++++++++++++++++++++++++ 1 file changed, 116 insertions(+) create mode 100644 tests/avocado/test_numa_hotplug.py
tests/avocado/test_numa_hotplug.py new file +116 -0
@@ -0,0 +1,116 @@ +import re +import time + +from avocado_qemu import test + + +class TestNumaHotplug(test.QemuTest): + """ + Verifies that "info numa" and "/sys/devices/system/node/" contains + correct values before/after inserting memory devices into default + and then into 13th numa node. + + Associated bug trackers: RHBZ1473203 + https://bugzilla.redhat.com/show_bug.cgi?id=1473203 + + Fixed in kernel commit dc421b200f91930c9c6a9586810ff8c232cf10fc. + + :avocado: enable + :avocado: tags=RHBZ1473203,requires_linux,numa,memory,ppc64le + """ + + def setUp(self): + self.request_image() + self.vm.args.extend(["-m", "4G,slots=208,maxmem=80G"]) + self.vm.args.extend(["-numa", "node"] * 16) + self.vm.launch() + + def check_mem_console(self, console, exp): + """ + Verifies that memory layout is according to exp using console/ssh + + :param console: session + :param exp: list of MemTotals per node in MB, tolerance is +-100MB + """ + out = console.cmd_output_safe("echo /sys/devices/system/node/node*") + nodes = re.findall(r"/sys/devices/system/node/node\d+", out) + self.assertEqual(len(nodes), len(exp), "Number of nodes is not " + "%s:\n%s" % (len(exp), out)) + for i in xrange(len(exp)): + out = console.cmd_output_safe("cat /sys/devices/system/node/" + "node%s/meminfo" % i) + mem = re.search(r"MemTotal:\s*(\d+) kB", out) + self.assertTrue(mem, "Failed to obtain node%s MemTotal:\n%s" + % (i, out)) + _exp = exp[i] * 1024 + mem = int(mem.group(1)) + self.assertGreater(mem, _exp - 102400, "TotalMem of node%s is not " + "%s+-51200 kb (%s)" % (i, _exp, mem)) + self.assertLess(mem, _exp + 102400, "TotalMem of node%s is not " + "%s+-51200 kb (%s)" % (i, _exp, mem)) + + def check_mem_monitor(self, monitor, exp): + """ + Verifies that memory layout is according to exp using QMP monitor + + :param console: session + :param exp: list of MemTotals per node in MB, tolerance is +-100MB + """ + ret = monitor("human-monitor-command", command_line="info numa") + out = ret["return"] + self.assertTrue(out.startswith("%s nodes" % len(exp)), "Number of " + "nodes is not %s:\n%s" % (len(exp), out)) + for i in xrange(len(exp)): + _exp = "node %s size: %s MB" % (i, exp[i]) + self.assertIn(_exp, out, "%s is not in 'info numa' output, " + "probably wrong memory size reported:\n%s" + % (_exp, out)) + + @staticmethod + def _retry_until_timeout(timeout, func, *args, **kwargs): + """ + Repeat the function until it returns anything ignoring AssertionError. + After the deadline repeate the function one more time without ignoring + timeout. + """ + end = time.time() + timeout + while time.time() < end: + try: + ret = func(*args, **kwargs) + except AssertionError: + continue + break + else: + ret = func(*args, **kwargs) + return ret + + def test_hotplug_mem_into_node(self): + console = self.vm.get_console() + exp = [256] * 16 + self.check_mem_monitor(self.vm.qmp, exp) + self.check_mem_console(console, exp) + cmd = "object_add memory-backend-ram,id=mem2,size=1G" + res = self.vm.qmp("human-monitor-command", command_line=cmd) + self.assertEqual(res["return"], "") + cmd = "device_add pc-dimm,id=dimm2,memdev=mem2" + res = self.vm.qmp("human-monitor-command", command_line=cmd) + self.assertEqual(res["return"], "") + exp = [1280] + [256] * 15 + self.check_mem_monitor(self.vm.qmp, exp) + # Wait up to 10s to propagate the changes + self._retry_until_timeout(10, self.check_mem_console, console, exp) + cmd = "object_add memory-backend-ram,id=mem8,size=1G" + res = self.vm.qmp("human-monitor-command", command_line=cmd) + self.assertEqual(res["return"], "") + cmd = "device_add pc-dimm,id=dimm8,memdev=mem8,node=13" + res = self.vm.qmp("human-monitor-command", command_line=cmd) + self.assertEqual(res["return"], "") + time.sleep(5) + exp = [1280] + [256] * 12 + [1280] + [256] * 2 + self.check_mem_monitor(self.vm.qmp, exp) + # Wait up to 10s to propagate the changes + self._retry_until_timeout(10, self.check_mem_console, console, exp) + console.close() + + def tearDown(self): + self.vm.shutdown()
Signature · 1 line
2.14.3
RFC 17/24 avocado_qemu: Remove duplicate PortTracker implementation
In reply to Eduardo HabkostFrom: Cleber Rosa <[email protected]>
The PortTracker class was introduced on Avocado's own utility libraries, starting with version 54.0, and it actually received fixes after that.
Let's avoid this duplicate implementation and rely on the standard one.
--- tests/avocado/README.rst | 2 +- tests/avocado/avocado_qemu/test.py | 45 +------------------------------------- 2 files changed, 2 insertions(+), 45 deletions(-)
tests/avocado/README.rst +1 -1
@@ -6,7 +6,7 @@ Framework. To install Avocado, follow the instructions from this link:: Tests here are written keeping the minimum amount of dependencies. To run the tests, you need the Avocado core package (`python-avocado` on Fedora, `avocado-framework` on pip). Extra dependencies should be -documented in this file. +documented in this file. The current minimum required version is 54.0. In this directory, an ``avocado_qemu`` package is provided, containing the ``test`` module, which inherits from ``avocado.Test`` and provides
tests/avocado/avocado_qemu/test.py +1 -44
63 lines not on this page · raw message
Signature · 1 line
2.14.3
RFC 18/24 avocado_qemu: Simplify the installation instructions
In reply to Eduardo HabkostFrom: Cleber Rosa <[email protected]>
Let's just give a single command line that can be reliably executed to install avocado and the dependencies we're aware for the existing tests.
The installation approach chosen is the most universal one (pip) and one that impacts the least the developer's system, being limited to the user's home directory.
--- tests/avocado/README.rst | 39 +++++++++++++++++++++++++++++---------- 1 file changed, 29 insertions(+), 10 deletions(-)
tests/avocado/README.rst +29 -10
@@ -1,12 +1,23 @@ -This directory is hosting functional tests written using Avocado Testing -Framework. To install Avocado, follow the instructions from this link:: +======================================== + QEMU tests using the Avocado Framework +======================================== + +This directory hosts functional tests written using Avocado Testing +Framework. + +Installation +============ + +To install Avocado and the dependencies needed for these tests, run:: + + pip install --user avocado-framework avocado-framework-plugin-varianter-yaml-to-mux aexpect + +Alternatively, follow the instructions on this link:: http://avocado-framework.readthedocs.io/en/latest/GetStartedGuide.html#installing-avocado -Tests here are written keeping the minimum amount of dependencies. To -run the tests, you need the Avocado core package (`python-avocado` on -Fedora, `avocado-framework` on pip). Extra dependencies should be -documented in this file. The current minimum required version is 54.0. +Overview +======== In this directory, an ``avocado_qemu`` package is provided, containing the ``test`` module, which inherits from ``avocado.Test`` and provides
@@ -77,10 +88,6 @@ file using the Avocado parameters system: ``accel`` is ``kvm`` and there are more than one KVM types available. Example: ``machine_kvm_type: PR`` -To use a parameters file, you have to install the yaml_to_mux plugin -(`python2-avocado-plugins-varianter-yaml-to-mux` on Fedora, -`avocado-framework-plugin-varianter-yaml-to-mux` on pip). - Run the test with:: $ avocado run test_my_test.py -m parameters.yaml
@@ -111,3 +118,15 @@ proper variants. See ``avocado run --help`` and ``man avocado`` for several other options, such as ``--filter-by-tags``, ``--show-job-log``, ``--failfast``, etc. + +Uninstallation +============== + +If you've followed the installation instructions above, you can easily +uninstall Avocado. Start by listing the packages you have installed:: + + pip list --user + +And remove any package you want with:: + + pip uninstall <package_name>
Signature · 1 line
2.14.3
RFC 19/24 avocado_qemu: Clean unneeded 'pass'
In reply to Eduardo HabkostFrom: Amador Pahim <[email protected]>
The docstrings are enough. Let's remove the 'pass' instructions.
--- tests/avocado/avocado_qemu/test.py | 6 ------ 1 file changed, 6 deletions(-)
tests/avocado/avocado_qemu/test.py +0 -6
@@ -49,42 +49,36 @@ class QEMULoginTimeoutError(Exception): """ If timeout expires """ - pass class QEMULoginAuthenticationError(Exception): """ If authentication fails """ - pass class QEMULoginProcessTerminatedError(Exception): """ If the client terminates during login """ - pass class QEMULoginError(Exception): """ If some other error occurs """ - pass class QEMUConsoleError(Exception): """ If some error with the console access happens """ - pass class QEMUMigrationError(Exception): """ If some error with the migration happens """ - pass def _get_qemu_bin(arch):
Signature · 1 line
2.14.3
RFC 20/24 avocado_qemu: Set QMP log level to INFO
In reply to Eduardo HabkostFrom: Amador Pahim <[email protected]>
Otherwise we will see the debug messages in our test logs.
--- tests/avocado/avocado_qemu/test.py | 1 + 1 file changed, 1 insertion(+)
tests/avocado/avocado_qemu/test.py +1 -0
@@ -235,6 +235,7 @@ class _VM(qemu.QEMUMachine): self.username = username self.password = password super(_VM, self).__init__(qemu_bin, name=self.name, arch=arch) + logging.getLogger('QMP').setLevel(logging.INFO) def get_console(self, console_address=None, prompt=r"[\#\$] "): """
Signature · 1 line
2.14.3
Re: [Qemu-devel] [RFC 20/24] avocado_qemu: Set QMP log level to INFO
Quoted 3 lines · Eduardo Habkost
From: Amador Pahim <[email protected]>
Otherwise we will see the debug messages in our test logs.
Why don't we want to see the QMP debug messages when debugging is enabled?
Quoted 22 lines
Signed-off-by Amador Pahim <[email protected]>Signed-off-by Eduardo Habkost <[email protected]>--- tests/avocado/avocado_qemu/test.py | 1 + 1 file changed, 1 insertion(+)tests/avocado/avocado_qemu/test.py +1
-0@@ -235,6 +235,7 @@ class _VM(qemu.QEMUMachine): self.username = username self.password = password super(_VM, self).__init__(qemu_bin, name=self.name, arch=arch) + logging.getLogger('QMP').setLevel(logging.INFO) def get_console(self, console_address=None, prompt=r"[\#\$] "): """Signature · 1 line
2.14.3
Signature · 1 line
Eduardo
RFC 21/24 avocado_qemu: Introduce the add_image() VM API
In reply to Eduardo HabkostFrom: Amador Pahim <[email protected]>
Uses can not add an image to the virtual machine with the option to configure the user/password using cloudinit.
--- tests/avocado/README.rst | 4 +- tests/avocado/avocado_qemu/test.py | 136 ++++++++++++++++++++++++++----------- tests/avocado/parameters.yaml | 9 --- tests/avocado/test_nec-usb-xhci.py | 14 ++-- tests/avocado/test_numa_hotplug.py | 5 +- 5 files changed, 111 insertions(+), 57 deletions(-)
tests/avocado/README.rst +2 -2
@@ -78,8 +78,8 @@ file using the Avocado parameters system: ``image_snapshot`` parameter. - ``image_user`` and ``image_pass``: When using a ``image_path``, if you want to get the console from the Guest OS you have to define the Guest - OS credentials. Example: ``image_user: root`` and - ``image_pass: p4ssw0rd``. By default it uses ``root`` and ``123456``. + OS credentials. Example: ``image_user: avocado`` and + ``image_pass: p4ssw0rd``. Both parameters have defaults to ``avocado``. - ``machine_type``: Use this option to define a machine type for the VM. Example: ``machine_type: pc`` - ``machine_accel``: Use this option to define a machine acceleration
tests/avocado/avocado_qemu/test.py +96 -40
184 lines not on this page · raw message
tests/avocado/parameters.yaml +0 -9
@@ -10,15 +10,6 @@ qemu_bin: null # used in the source VM will be used for the destination VM. qemu_dst_bin: null -# VMs are defined without image. If the 'image_path' is specified, it -# will be used as the VM image. The '-snapshot' option will then be used -# to avoid writing data to the image. -image_path: null -# Username used to get the console from the Guest OS. -image_user: null -# Password used to get the console from the Guest OS. -image_pass: null - # Use this option to define a machine type for the VM. machine_type: null # Use this option to define a machine acceleration for the VM.
tests/avocado/test_nec-usb-xhci.py +9 -5
@@ -4,6 +4,7 @@ import tempfile from avocado_qemu import test from avocado.utils import process +from avocado.utils import vmimage class TestNecUsbXhci(test.QemuTest): """
@@ -17,8 +18,10 @@ class TestNecUsbXhci(test.QemuTest): """ def setUp(self): + self.image = vmimage.get() + self.vm.add_image(self.image.path, cloudinit=True, snapshot=False) + usbdevice = os.path.join(self.workdir, 'usb.img') - self.request_image() process.run('dd if=/dev/zero of=%s bs=1M count=10' % usbdevice) self.vm.args.extend(['-device', 'pci-bridge,id=bridge1,chassis_nr=1']) self.vm.args.extend(['-device', 'nec-usb-xhci,id=xhci1,bus=bridge1,addr=0x3'])
@@ -35,17 +38,18 @@ class TestNecUsbXhci(test.QemuTest): :avocado: tags=migration,RHBZ1436616 """ + console = self.vm.get_console() - console.sendline('fdisk -l') - result = console.read_nonblocking() + console.sendline('sudo fdisk -l') + result = console.read_up_to_prompt() console.close() self.assertIn('Disk /dev/sdb: 10 MiB, 10485760 bytes, 20480 sectors', result) self.vm_dst = self.vm.migrate() console = self.vm_dst.get_console() - console.sendline('fdisk -l') - result = console.read_nonblocking() + console.sendline('sudo fdisk -l') + result = console.read_up_to_prompt() console.close() self.assertIn('Disk /dev/sdb: 10 MiB, 10485760 bytes, 20480 sectors', result)
tests/avocado/test_numa_hotplug.py +4 -1
@@ -2,6 +2,7 @@ import re import time from avocado_qemu import test +from avocado.utils import vmimage class TestNumaHotplug(test.QemuTest):
@@ -20,7 +21,9 @@ class TestNumaHotplug(test.QemuTest): """ def setUp(self): - self.request_image() + self.image = vmimage.get() + self.vm.add_image(self.image.path, cloudinit=True, snapshot=False) + self.vm.args.extend(["-m", "4G,slots=208,maxmem=80G"]) self.vm.args.extend(["-numa", "node"] * 16) self.vm.launch()
Signature · 1 line
2.14.3
RFC 22/24 avocado_qemu: Tests fixes
In reply to Eduardo HabkostFrom: Amador Pahim <[email protected]>
- Using acell=KVM when we have a guest.
- Better resilience for the cases where live-migration fails.
--- tests/avocado/test_nec-usb-xhci.py | 6 +++++- tests/avocado/test_numa_hotplug.py | 1 + 2 files changed, 6 insertions(+), 1 deletion(-)
tests/avocado/test_nec-usb-xhci.py +5 -1
@@ -18,8 +18,10 @@ class TestNecUsbXhci(test.QemuTest): """ def setUp(self): + self.vm_dst = None self.image = vmimage.get() self.vm.add_image(self.image.path, cloudinit=True, snapshot=False) + self.vm.args.extend(['-machine', 'accel=kvm']) usbdevice = os.path.join(self.workdir, 'usb.img') process.run('dd if=/dev/zero of=%s bs=1M count=10' % usbdevice)
@@ -56,4 +58,6 @@ class TestNecUsbXhci(test.QemuTest): def tearDown(self): self.vm.shutdown() - self.vm_dst.shutdown() + if self.vm_dst is not None: + self.vm_dst.shutdown() + os.remove(self.image.path)
tests/avocado/test_numa_hotplug.py +1 -0
@@ -24,6 +24,7 @@ class TestNumaHotplug(test.QemuTest): self.image = vmimage.get() self.vm.add_image(self.image.path, cloudinit=True, snapshot=False) + self.vm.args.extend(['-machine', 'accel=kvm']) self.vm.args.extend(["-m", "4G,slots=208,maxmem=80G"]) self.vm.args.extend(["-numa", "node"] * 16) self.vm.launch()
Signature · 1 line
2.14.3
RFC 23/24 avocado_qemu: Force vmimage distro
In reply to Eduardo HabkostFrom: Amador Pahim <[email protected]>
Not all distros will support cloudinit. Let's use Fedora, which is proven to work.
--- tests/avocado/test_nec-usb-xhci.py | 2 +- tests/avocado/test_numa_hotplug.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
tests/avocado/test_nec-usb-xhci.py +1 -1
@@ -19,7 +19,7 @@ class TestNecUsbXhci(test.QemuTest): def setUp(self): self.vm_dst = None - self.image = vmimage.get() + self.image = vmimage.get('Fedora') self.vm.add_image(self.image.path, cloudinit=True, snapshot=False) self.vm.args.extend(['-machine', 'accel=kvm'])
tests/avocado/test_numa_hotplug.py +1 -1
@@ -21,7 +21,7 @@ class TestNumaHotplug(test.QemuTest): """ def setUp(self): - self.image = vmimage.get() + self.image = vmimage.get('Fedora') self.vm.add_image(self.image.path, cloudinit=True, snapshot=False) self.vm.args.extend(['-machine', 'accel=kvm'])
Signature · 1 line
2.14.3
RFC 24/24 avocado_qemu: Add a few VNC related tests
In reply to Eduardo HabkostFrom: Cleber Rosa <[email protected]>
These exercise the basics of firing up QEMU with and without a VNC device, and attempting to set the password when a VNC device and password is given (or not).
--- tests/avocado/test_vnc.py | 58 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 58 insertions(+) create mode 100644 tests/avocado/test_vnc.py
tests/avocado/test_vnc.py new file +58 -0
@@ -0,0 +1,58 @@ +from avocado import main +from avocado_qemu import test + + +class Vnc(test.QemuTest): + """ + :avocado: enable + :avocado: tags=vnc,quick + """ + def test_no_vnc(self): + self.vm.args = ['-nodefaults', '-S'] + self.vm.launch() + self.assertFalse(self.vm.qmp('query-vnc')['return']['enabled']) + + def test_no_vnc_change_password(self): + self.vm.args = ['-nodefaults', '-S'] + self.vm.launch() + self.assertFalse(self.vm.qmp('query-vnc')['return']['enabled']) + set_password_response = self.vm.qmp('change', + device='vnc', + target='password', + arg='new_password') + self.assertIn('error', set_password_response) + self.assertEqual(set_password_response['error']['class'], + 'GenericError') + self.assertEqual(set_password_response['error']['desc'], + 'Could not set password') + + def test_vnc_change_password_requires_a_password(self): + self.vm.args = ['-nodefaults', '-S', '-vnc', ':0'] + self.vm.launch() + self.assertTrue(self.vm.qmp('query-vnc')['return']['enabled']) + set_password_response = self.vm.qmp('change', + device='vnc', + target='password', + arg='new_password') + self.assertIn('error', set_password_response) + self.assertEqual(set_password_response['error']['class'], + 'GenericError') + self.assertEqual(set_password_response['error']['desc'], + 'Could not set password') + + def test_vnc_change_password(self): + self.vm.args = ['-nodefaults', '-S', '-vnc', ':0,password'] + self.vm.launch() + self.assertTrue(self.vm.qmp('query-vnc')['return']['enabled']) + set_password_response = self.vm.qmp('change', + device='vnc', + target='password', + arg='new_password') + self.assertEqual(set_password_response['return'], {}) + + def tearDown(self): + self.vm.shutdown() + + +if __name__ == '__main__': + main()
Signature · 1 line
2.14.3
This series seems to have some coding style problems. See output below for more information:
Hi,
This series seems to have some coding style problems. See output below for more information:
Type: series
Message-id: [email protected]
Subject: [Qemu-devel] [RFC 00/24] Avocado-based functional tests
=== TEST SCRIPT BEGIN ===
#!/bin/bash
BASE=base
n=1
total=$(git log --oneline $BASE.. | wc -l)
failed=0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
commits="$(git log --format=%H --reverse $BASE..)" for c in $commits; do
echo "Checking PATCH $n/$total: $(git log -n 1 --format=%s $c)..."
if ! git show $c --format=email | ./scripts/checkpatch.pl --mailback -; then
failed=1
echo
fi
n=$((n+1))done
exit $failed
=== TEST SCRIPT END ===
Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
* [new tag] patchew/[email protected] -> patchew/[email protected]
Switched to a new branch 'test'ec49778cf4 avocado_qemu: Add a few VNC related tests1f46f502ab avocado_qemu: Force vmimage distro3c652684e4 avocado_qemu: Tests fixesc4e2d71898 avocado_qemu: Introduce the add_image() VM API 35b9856b7d avocado_qemu: Set QMP log level to INFO3d81ee79d0 avocado_qemu: Clean unneeded 'pass'3fea592db2 avocado_qemu: Simplify the installation instructions d1b9960835 avocado_qemu: Remove duplicate PortTracker implementation 030c3444fa avocado_qemu: Functional test for RHBZ1473203 4452f51949 avocado_qemu: Functional test for RHBZ#1436616 a791819b84 avocado_qemu: Functional test for RHBZ#1447027 1b525617c2 avocado_qemu: Functional test for RHBZ#1431939 8cb3174c7d avocado_qemu: Improve migration error message 800fdc0407 avocado_qemu: Fix exception name in caller487b76eeea avocado_qemu: Add support to request image for testing 2349952ea0 avocado_qemu: Ignore kernel messages on get_console ee1ba766ab avocado_qemu: Provide defaults for user and pass 2151fc0647 avocado_qemu: Store "arch" in VM27d897fc64 avocado_qemu: Add " " after the default prompt regexp 1b31385780 avocado_qemu: Increase the login timeout to 60s f964227931 avocado_qemu: Be lenient towards poluted serial console 903c5ccd71 avocado_qemu: Improve handle_prompts to allow login after booted vm 46e5bdc04f Introduce the basic framework to run Avocado tests 9a629a9861 qemu.py: Introduce _create_console() method
=== OUTPUT BEGIN ===
Checking PATCH 1/24: qemu.py: Introduce _create_console() method... Checking PATCH 2/24: Introduce the basic framework to run Avocado tests...
WARNING: line over 80 characters
#311: FILE: tests/avocado/avocado_qemu/test.py:118:
+ :raise QEMULoginProcessTerminatedError: If the client terminates during login
WARNING: line over 80 characters
#399: FILE: tests/avocado/avocado_qemu/test.py:206:
+ raise QEMULoginProcessTerminatedError(details.status, details.output)
total: 0 errors, 2 warnings, 661 lines checked
Your patch has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
Checking PATCH 3/24: avocado_qemu: Improve handle_prompts to allow login after booted vm... Checking PATCH 4/24: avocado_qemu: Be lenient towards poluted serial console... Checking PATCH 5/24: avocado_qemu: Increase the login timeout to 60s... Checking PATCH 6/24: avocado_qemu: Add " " after the default prompt regexp... Checking PATCH 7/24: avocado_qemu: Store "arch" in VM...
Checking PATCH 8/24: avocado_qemu: Provide defaults for user and pass... Checking PATCH 9/24: avocado_qemu: Ignore kernel messages on get_console... Checking PATCH 10/24: avocado_qemu: Add support to request image for testing... Checking PATCH 11/24: avocado_qemu: Fix exception name in caller...
Checking PATCH 12/24: avocado_qemu: Improve migration error message... Checking PATCH 13/24: avocado_qemu: Functional test for RHBZ#1431939...
WARNING: line over 80 characters
#73: FILE: tests/avocado/test_info_memdev_host_nodes.py:49:
+ cmd = 'object_add memory-backend-ram,id=mem1,host-nodes=0,size=2G,policy=bind'
total: 0 errors, 1 warnings, 66 lines checked
Your patch has style problems, please review. If any of these errors are false positives report them to the maintainer, see CHECKPATCH in MAINTAINERS.
Checking PATCH 14/24: avocado_qemu: Functional test for RHBZ#1447027...
ERROR: line over 90 characters
#51: FILE: tests/avocado/test_ovmf_with_240_vcpus.py:22:
+ default='/usr/share/edk2/ovmf/OVMF_CODE.secboot.fd')
WARNING: line over 80 characters
#53: FILE: tests/avocado/test_ovmf_with_240_vcpus.py:24:
+ default='/usr/share/edk2/ovmf/OVMF_VARS.fd')
total: 1 errors, 1 warnings, 72 lines checked
Your patch has style problems, please review. If any of these errors are false positives report them to the maintainer, see CHECKPATCH in MAINTAINERS.
Checking PATCH 15/24: avocado_qemu: Functional test for RHBZ#1436616...
WARNING: line over 80 characters
#53: FILE: tests/avocado/test_nec-usb-xhci.py:24:
+ self.vm.args.extend(['-device', 'nec-usb-xhci,id=xhci1,bus=bridge1,addr=0x3'])
ERROR: line over 90 characters
#54: FILE: tests/avocado/test_nec-usb-xhci.py:25:
+ self.vm.args.extend(['-drive', 'file=%s,format=raw,id=drive_usb,if=none' % usbdevice])
ERROR: line over 90 characters
#55: FILE: tests/avocado/test_nec-usb-xhci.py:26:
+ self.vm.args.extend(['-device', 'usb-storage,drive=drive_usb,id=device_usb,bus=xhci1.0'])
total: 2 errors, 1 warnings, 59 lines checked
Your patch has style problems, please review. If any of these errors are false positives report them to the maintainer, see CHECKPATCH in MAINTAINERS.
Checking PATCH 16/24: avocado_qemu: Functional test for RHBZ1473203... Checking PATCH 17/24: avocado_qemu: Remove duplicate PortTracker implementation... Checking PATCH 18/24: avocado_qemu: Simplify the installation instructions... Checking PATCH 19/24: avocado_qemu: Clean unneeded 'pass'...
Checking PATCH 20/24: avocado_qemu: Set QMP log level to INFO... Checking PATCH 21/24: avocado_qemu: Introduce the add_image() VM API... Checking PATCH 22/24: avocado_qemu: Tests fixes...
Checking PATCH 23/24: avocado_qemu: Force vmimage distro...
Checking PATCH 24/24: avocado_qemu: Add a few VNC related tests... === OUTPUT END ===
Test command exited with code: 1
---
Email generated automatically by Patchew [http://patchew.org/].
Please send your feedback to [email protected]