Skip to contentSkip to conversation

Acceptance tests: arm/aarch64 host and target enablers

PATCH New Cleber Rosa 1 participant 3 messages Sep 11, 2019

lore
Share & tools
lore
#0 lore raw

cover Acceptance tests: arm/aarch64 host and target enablers

The acceptance tests have been running primarily on x86_64 hosts, and some tests will fail when run under non-x86 hosts or targets.

This series addresses issues that enables running the acceptance tests under aarch64 hosts, and on other hosts using arm and aarch64 targets.

Cleber Rosa (2):
Python libs: close console sockets before shutting down the VMs Python libs: enable machine type auto selection

 python/qemu/machine.py                    | 92 ++++++++++++++++++++++-
 tests/acceptance/avocado_qemu/__init__.py |  1 +
 2 files changed, 89 insertions(+), 4 deletions(-)
Signature · 1 line
2.21.0
#2 lore raw

PATCH 2/2 Python libs: enable machine type auto selection

The arm and aarch64 targets are noteworthy because they don't have a default machine type. This introduces and opt-in option to users of the qemu.machine.QEMUMachine Python class, in which a default machine type will be set when a number of conditions are met.

The motivation for this feature is that a number of (acceptance) tests are target agnostic, and should run with all tarets (in theory while it's pending continuous test execution). These tests can be listed with:

  avocado list --filter-by-tags='-arch' --filter-by-tags-include-empty \
               --filter-by-tags-include-empty-key tests/acceptance/

Combined with the "qemu_bin" parameter that acceptance tests take, one can run those tests with arm and aarch64 targets without adding specific logic in each test. To execute those with a aarch64 QEMU binary one can run:

  avocado run --filter-by-tags='-arch' --filter-by-tags-include-empty \
              --filter-by-tags-include-empty-key \
              -p qemu_bin=aarch64-softmmu/qemu-system-aarch64 \
              tests/acceptance/

When this feature is activated, the default automatic selection of a machine type will be logged (as it will also be done in the QEMU command line that is already logged).

---
 python/qemu/machine.py                    | 81 +++++++++++++++++++++++
 tests/acceptance/avocado_qemu/__init__.py |  1 +
 2 files changed, 82 insertions(+)
python/qemu/machine.py +81 -0
index 4f533b7881..350052a30b 100644
@@ -69,6 +69,15 @@ class QEMUMachine(object):
         # vm is guaranteed to be shut down here
     """
 
+
+    #: The default machine model, used by :meth:`set_machine_auto_selection` and
+    #: :meth:`get_machine_auto_selection`
+    DEFAULT_MACHINE_MODELS = {
+        'arm': 'virt',
+        'aarch64': 'virt',
+        }
+
+
     def __init__(self, binary, args=None, wrapper=None, name=None,
                  test_dir="/var/tmp", monitor_address=None,
                  socket_scm_helper=None):
@@ -108,6 +117,7 @@ class QEMUMachine(object):
         self._temp_dir = None
         self._launched = False
         self._machine = None
+        self._machine_auto_selection = False
         self._console_set = False
         self._console_device_type = None
         self._console_address = None
@@ -286,6 +296,7 @@ class QEMUMachine(object):
 
         self._iolog = None
         self._qemu_full_args = None
+        self._perform_machine_auto_selection()
         try:
             self._launch()
             self._launched = True
@@ -533,3 +544,73 @@ class QEMUMachine(object):
                                                  socket.SOCK_STREAM)
             self._console_socket.connect(self._console_address)
         return self._console_socket
+
+    def set_machine_auto_selection(self, value):
+        """
+        Defines whether QEMUMachine will set a machine model when needed
+
+        The machine model will only be set automatically if:
+
+        1) No machine model was explicity set with :meth:`set_machine`
+        2) Target binary is known to not work properly when a machine
+           is not specified (usually producing "No machine specified,
+           and there's no default" on the command line)
+        3) A default machine model exists in :data:`DEFAULT_MACHINE_MODELS`
+        4) :func:`get_target_from_qemu_binary` is capable of returning
+           the target for the binary set on this class
+
+        If machine auto selection is enabled, it will be done so by
+        calling :meth:`set_machine` during the machine launc, that is
+        on :meth:`launch` so that a manually set machine model is not
+        discarded.
+
+        @param value: wether to enable or disable the machine auto selection
+                      feature
+        @type value: bool
+        """
+        self._machine_auto_selection = value
+
+    def get_machine_auto_selection(self):
+        """
+        Returns whether QEMUMachine will set a machine model when needed
+
+        Please refer to :meth:`set_machine_auto_selection` for a more
+        complete explanation.
+        """
+        return self._machine_auto_selection
+
+    def _perform_machine_auto_selection(self):
+        if not (self.get_machine_auto_selection() and self._machine is None):
+            return
+
+        target = self._get_target_from_qemu_binary()
+        if target is None:
+            LOG.warn('Machine auto selection was requested, but it was not '
+                     'applied as the binary "%s" could not be probed for its '
+                     'target architecture', self._binary)
+            return
+
+        machine_model = self.DEFAULT_MACHINE_MODELS.get(target, None)
+        if machine_model is None:
+            LOG.warn('Machine auto selection was requested, but it was not '
+                     'applied as there is no default machine model defined '
+                     'for target architecture "%s"', target)
+            return
+
+        LOG.debug('Setting machine model to: %s', machine_model)
+        self.set_machine(machine_model)
+
+    def _get_target_from_qemu_binary(self):
+        """
+        Returns the target for this machine's QEMU binary
+        """
+        qemu_machine = QEMUMachine(self._binary)
+        qemu_machine.add_args('-S')
+        qemu_machine.set_machine('none')
+        try:
+            qemu_machine.launch()
+            target_resp = qemu_machine.command('query-target')
+            qemu_machine.shutdown()
+            return target_resp['arch']
+        except Exception:
+            return None
tests/acceptance/avocado_qemu/__init__.py +1 -0
index bd41e0443c..025bb6a725 100644
@@ -70,6 +70,7 @@ class Test(avocado.Test):
 
     def _new_vm(self, *args):
         vm = QEMUMachine(self.qemu_bin)
+        vm.set_machine_auto_selection(True)
         if args:
             vm.add_args(*args)
         return vm
Signature · 1 line
2.21.0
#1 lore raw

PATCH 1/2 Python libs: close console sockets before shutting down the VMs

In reply to Cleber Rosa

Currently, the console socket on QEMUMachine is closed after the QMP command to gracefully exit QEMU is executed. Because of a possible deadlock (QEMU waiting for the socket to become writable) let's close the console socket earlier.

Reference: <[email protected]>
Reference: https://bugs.launchpad.net/qemu/+bug/1829779
From: Eduardo Habkost <[email protected]>

---
 python/qemu/machine.py | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)
python/qemu/machine.py +7 -4
index 128a3d1dc2..4f533b7881 100644
@@ -271,10 +271,6 @@ class QEMUMachine(object):
 
         self._qemu_log_path = None
 
-        if self._console_socket is not None:
-            self._console_socket.close()
-            self._console_socket = None
-
         if self._temp_dir is not None:
             shutil.rmtree(self._temp_dir)
             self._temp_dir = None
@@ -333,6 +329,13 @@ class QEMUMachine(object):
         """
         Terminate the VM and clean up
         """
+        # If we keep the console socket open, we may deadlock waiting
+        # for QEMU to exit, while QEMU is waiting for the socket to
+        # become writeable.
+        if self._console_socket is not None:
+            self._console_socket.close()
+            self._console_socket = None
+
         if self.is_running():
             try:
                 if not has_quit:
Signature · 1 line
2.21.0