Skip to contentSkip to conversation

Acceptance Tests: target architecture support

PATCH Superseded Cleber Rosa 11 participants 94 messages Jan 17, 2019 – Jun 10, 2019

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

cover Acceptance Tests: target architecture support

The current acceptance tests don't provide any type of architecture information that can be used to influence the selection of the QEMU binary used on them[1]. If one is running tests on a x86_64 host, the default QEMU binary will be "x86_64-softmmu/qemu-system-x86_64".

Given the nature of QEMU, some tests will be architecture agnostic, while others will be architecture dependent. The "check-qtest" and "check-qtest-TARGET" make targets exemplify that pattern.

For the acceptance tests, the same requirement exists. Tests should be allowed to influence the binary used, and when they don't, a default selection mechanism should kick in[2]. The proposed solution here requires only that an Avocado tag is set, such as:

class My(Test):

       def test_nx_cpu_flag(self):
           """
           :avocado: tags=arch:x86_64
           """
           test_code()

The value of the "arch" key, in this case, "x86_64" will be used when selecting the QEMU binary to use in the test. At the same time, if "x86_64-softmmu" is not a built target, the test will be filtered out by "make check-acceptance"[3].

Besides the convention explained above, where the binary will be selected from the "arch" tag, it's also possible to set an "arch" *parameter* that will also influence the QEMU binary selection:

  $ avocado run -p arch=aarch64 works-on-many-arches.py

Finally, it's also posible to set the "qemu_bin" parameter, which will define (instead of just influencing) the QEMU binary to be used:

 $ avocado run -p qemu_bin=qemu-bin-aarch64 test.py

As examples for the idea proposed here, a number of "boot linux console" tests have been added, for a number of different target architectures. When the build environment includes them (as it has been added to Travis CI jobs) the architecture specific tests will be automatically executed.

As mentioned previously, this patch series include ideas present in other patch series, and from different authors. I tried by best to include the information about authorship, but if I missed any, please accept my apologies and let me know.

---

[1] - The "boot_linux_console.py" contains a "x86_64" test tag, but that is informational only, in the sense that it's not consumed by the test itself, or used by "make check-acceptance" to filter out tests.

[2] - This patch series doesn't attempt to change the default selection mechanism. Possible changes in this area may include looking for any one built binary first, no matter the host architecture.

[3] - On a previous proposed version, the test class would look at the "arch" parameter given, and would cancel the test if there wasn't a match.

---

Git Info:
- URI: https://github.com/clebergnu/qemu/tree/sent/target_arch
- Remote: https://github.com/clebergnu/qemu
- Branch: sent/target_arch

Cleber Rosa (17):
scripts/qemu.py: log QEMU launch command line
Acceptance tests: show avocado test execution by default Acceptance tests: improve docstring on pick_default_qemu_bin() Acceptance tests: fix doc reference to avocado_qemu directory Acceptance tests: introduce arch parameter and attribute Acceptance tests: use "arch:" tag to filter target specific tests Acceptance tests: look for target architecture in test tags first Boot Linux Console Test: rename the x86_64 after the arch and machine Boot Linux Console Test: update the x86_64 kernel
Boot Linux Console Test: refactor the console watcher into utility
method
scripts/qemu.py: support adding a console with the default serial
device
Boot Linux Console Test: add a test for mips64el + malta Boot Linux Console Test: add a test for ppc64 + pseries Boot Linux Console Test: add a test for aarch64 + virt Boot Linux Console Test: add a test for arm + virt
Boot Linux Console Test: add a test for s390x + s390-ccw-virtio Boot Linux Console Test: add a test for alpha + clipper

Philippe Mathieu-Daudé (1):
Boot Linux Console Test: add a test for mips + malta

 .travis.yml                               |   4 +-
 docs/devel/testing.rst                    |  23 ++-
 scripts/qemu.py                           |  32 ++--
 tests/Makefile.include                    |   5 +-
 tests/acceptance/avocado_qemu/__init__.py |  22 ++-
 tests/acceptance/boot_linux_console.py    | 208 ++++++++++++++++++++--
 tests/requirements.txt                    |   2 +-
 7 files changed, 254 insertions(+), 42 deletions(-)
Signature · 1 line
2.20.1
#1 lore raw

PATCH 1/18 scripts/qemu.py: log QEMU launch command line

Even when the launch of QEMU succeeds, it's useful to have the command line recorded.

---
 scripts/qemu.py | 1 +
 1 file changed, 1 insertion(+)
scripts/qemu.py +1 -0
index 6e3b0e6771..ec3567d4e2 100644
@@ -312,6 +312,7 @@ class QEMUMachine(object):
         self._pre_launch()
         self._qemu_full_args = (self._wrapper + [self._binary] +
                                 self._base_args() + self._args)
+        LOG.debug('VM launch command: %r', ' '.join(self._qemu_full_args))
         self._popen = subprocess.Popen(self._qemu_full_args,
                                        stdin=devnull,
                                        stdout=self._qemu_log_file,
Signature · 1 line
2.20.1
Reviewed-by on v1 1/18
#2 lore raw

Re: [Qemu-devel] [PATCH 01/18] scripts/qemu.py: log QEMU launch command line

Quoted 4 lines · Cleber Rosa

Even when the launch of QEMU succeeds, it's useful to have the command line recorded.

Quoted 19 lines
---
 scripts/qemu.py | 1 +
 1 file changed, 1 insertion(+)
scripts/qemu.py +1 -0
index 6e3b0e6771..ec3567d4e2 100644
@@ -312,6 +312,7 @@ class QEMUMachine(object):
         self._pre_launch()
         self._qemu_full_args = (self._wrapper + [self._binary] +
                                 self._base_args() + self._args)
+        LOG.debug('VM launch command: %r', ' '.join(self._qemu_full_args))
         self._popen = subprocess.Popen(self._qemu_full_args,
                                        stdin=devnull,
                                        stdout=self._qemu_log_file,
Signature · 1 line
2.20.1
Signature · 3 lines
Caio Carrara
Software Engineer, Virt Team - Red Hat
[email protected]
Reviewed-by on v1 1/18
#3 lore raw

Re: [Qemu-devel] [PATCH 01/18] scripts/qemu.py: log QEMU launch command line

In reply to Cleber Rosa
Quoted 13 lines · Cleber Rosa

Even when the launch of QEMU succeeds, it's useful to have the command line recorded.

---
 scripts/qemu.py | 1 +
 1 file changed, 1 insertion(+)
scripts/qemu.py +0 -0
index 6e3b0e6771..ec3567d4e2 100644

scripts/qemu.py class QEMUMachine(object): · line 315

Quoted 4 lines
scripts/qemu.py +1 -0
@@ -312,6 +312,7 @@ class QEMUMachine(object):
         self._pre_launch()
         self._qemu_full_args = (self._wrapper + [self._binary] +
                                 self._base_args() + self._args)
+        LOG.debug('VM launch command: %r', ' '.join(self._qemu_full_args))

Useful!

Quoted 4 lines
scripts/qemu.py +0 -0
         self._popen = subprocess.Popen(self._qemu_full_args,
                                        stdin=devnull,
                                        stdout=self._qemu_log_file,
Reviewed-by on v1 1/18
#4 lore raw

Re: [Qemu-devel] [PATCH 01/18] scripts/qemu.py: log QEMU launch command line

In reply to Cleber Rosa
Quoted 13 lines · Cleber Rosa

Even when the launch of QEMU succeeds, it's useful to have the command line recorded.

---
 scripts/qemu.py | 1 +
 1 file changed, 1 insertion(+)
scripts/qemu.py +0 -0
index 6e3b0e6771..ec3567d4e2 100644

scripts/qemu.py class QEMUMachine(object): · line 318

Quoted 7 lines
scripts/qemu.py +1 -0
@@ -312,6 +312,7 @@ class QEMUMachine(object):
         self._pre_launch()
         self._qemu_full_args = (self._wrapper + [self._binary] +
                                 self._base_args() + self._args)
+        LOG.debug('VM launch command: %r', ' '.join(self._qemu_full_args))
         self._popen = subprocess.Popen(self._qemu_full_args,
                                        stdin=devnull,
                                        stdout=self._qemu_log_file,

--
Alex Bennée

#5 lore raw

PATCH 2/18 Acceptance tests: show avocado test execution by default

In reply to Cleber Rosa

The current version of the "check-acceptance" target will only show one line for execution of all tests. That's probably OK if the tests to be run are quick enough and they're always the same.

But, there's already one test alone that takes on average ~5 seconds to run, we intend to adapt the list of tests to match the user's build environment (among other choices).

Because of that, let's present the default Avocado UI by default. Users can always choose a different output by setting the AVOCADO_SHOW variable.

---
 .travis.yml            | 2 +-
 tests/Makefile.include | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
.travis.yml +1 -1
index 93fd0164a0..844d514afa 100644
@@ -188,7 +188,7 @@ matrix:
     # Acceptance (Functional) tests
     - env:
         - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu"
-        - TEST_CMD="make AVOCADO_SHOW=app check-acceptance"
+        - TEST_CMD="make check-acceptance"
       addons:
         apt:
           packages:
tests/Makefile.include +1 -1
index f403a6571d..c73298740d 100644
@@ -956,7 +956,7 @@ TESTS_RESULTS_DIR=$(BUILD_DIR)/tests/results
 # Controls the output generated by Avocado when running tests.
 # Any number of command separated loggers are accepted.  For more
 # information please refer to "avocado --help".
-AVOCADO_SHOW=none
+AVOCADO_SHOW=app
 
 PYTHON3 = $(shell $(PYTHON) -c 'import sys; print(1 if sys.version_info >= (3, 0) else 0)')
 ifeq ($(PYTHON3), 1)
Signature · 1 line
2.20.1
Reviewed-by on v1 2/18
#6 lore raw

Re: [Qemu-devel] [PATCH 02/18] Acceptance tests: show avocado test execution by default

Quoted 8 lines · Cleber Rosa

The current version of the "check-acceptance" target will only show one line for execution of all tests. That's probably OK if the tests to be run are quick enough and they're always the same.

But, there's already one test alone that takes on average ~5 seconds to run, we intend to adapt the list of tests to match the user's build environment (among other choices).

Quoted 5 lines

Because of that, let's present the default Avocado UI by default. Users can always choose a different output by setting the AVOCADO_SHOW variable.

Quoted 34 lines
---
 .travis.yml            | 2 +-
 tests/Makefile.include | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
.travis.yml +1 -1
index 93fd0164a0..844d514afa 100644
@@ -188,7 +188,7 @@ matrix:
     # Acceptance (Functional) tests
     - env:
         - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu"
-        - TEST_CMD="make AVOCADO_SHOW=app check-acceptance"
+        - TEST_CMD="make check-acceptance"
       addons:
         apt:
           packages:
tests/Makefile.include +1 -1
index f403a6571d..c73298740d 100644
@@ -956,7 +956,7 @@ TESTS_RESULTS_DIR=$(BUILD_DIR)/tests/results
 # Controls the output generated by Avocado when running tests.
 # Any number of command separated loggers are accepted.  For more
 # information please refer to "avocado --help".
-AVOCADO_SHOW=none
+AVOCADO_SHOW=app
 
 PYTHON3 = $(shell $(PYTHON) -c 'import sys; print(1 if sys.version_info >= (3, 0) else 0)')
 ifeq ($(PYTHON3), 1)
Signature · 1 line
2.20.1
Signature · 3 lines
Caio Carrara
Software Engineer, Virt Team - Red Hat
[email protected]
Reviewed-by on v1 2/18
#7 lore raw

Re: [Qemu-devel] [PATCH 02/18] Acceptance tests: show avocado test execution by default

In reply to Cleber Rosa
Quoted 8 lines · Cleber Rosa

The current version of the "check-acceptance" target will only show one line for execution of all tests. That's probably OK if the tests to be run are quick enough and they're always the same.

But, there's already one test alone that takes on average ~5 seconds to run, we intend to adapt the list of tests to match the user's build environment (among other choices).

Quoted 5 lines

Because of that, let's present the default Avocado UI by default. Users can always choose a different output by setting the AVOCADO_SHOW variable.

Quoted 32 lines
---
 .travis.yml            | 2 +-
 tests/Makefile.include | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
.travis.yml +1 -1
index 93fd0164a0..844d514afa 100644
@@ -188,7 +188,7 @@ matrix:
     # Acceptance (Functional) tests
     - env:
         - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu"
-        - TEST_CMD="make AVOCADO_SHOW=app check-acceptance"
+        - TEST_CMD="make check-acceptance"
       addons:
         apt:
           packages:
tests/Makefile.include +1 -1
index f403a6571d..c73298740d 100644
@@ -956,7 +956,7 @@ TESTS_RESULTS_DIR=$(BUILD_DIR)/tests/results
 # Controls the output generated by Avocado when running tests.
 # Any number of command separated loggers are accepted.  For more
 # information please refer to "avocado --help".
-AVOCADO_SHOW=none
+AVOCADO_SHOW=app
 
 PYTHON3 = $(shell $(PYTHON) -c 'import sys; print(1 if sys.version_info >= (3, 0) else 0)')
 ifeq ($(PYTHON3), 1)
Reviewed-by on v1 2/18
#8 lore raw

Re: [Qemu-devel] [PATCH 02/18] Acceptance tests: show avocado test execution by default

In reply to Cleber Rosa
Quoted 8 lines · Cleber Rosa

The current version of the "check-acceptance" target will only show one line for execution of all tests. That's probably OK if the tests to be run are quick enough and they're always the same.

But, there's already one test alone that takes on average ~5 seconds to run, we intend to adapt the list of tests to match the user's build environment (among other choices).

Quoted 5 lines

Because of that, let's present the default Avocado UI by default. Users can always choose a different output by setting the AVOCADO_SHOW variable.

Quoted 23 lines
---
 .travis.yml            | 2 +-
 tests/Makefile.include | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
.travis.yml +1 -1
index 93fd0164a0..844d514afa 100644
@@ -188,7 +188,7 @@ matrix:
     # Acceptance (Functional) tests
     - env:
         - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu"
-        - TEST_CMD="make AVOCADO_SHOW=app check-acceptance"
+        - TEST_CMD="make check-acceptance"
       addons:
         apt:
           packages:
tests/Makefile.include +0 -0
index f403a6571d..c73298740d 100644

tests/Makefile.include TESTS_RESULTS_DIR=$(BUILD_DIR)/tests/results · line 962

Quoted 8 lines
tests/Makefile.include +1 -1
@@ -956,7 +956,7 @@ TESTS_RESULTS_DIR=$(BUILD_DIR)/tests/results
 # Controls the output generated by Avocado when running tests.
 # Any number of command separated loggers are accepted.  For more
 # information please refer to "avocado --help".
-AVOCADO_SHOW=none
+AVOCADO_SHOW=app
 
 PYTHON3 = $(shell $(PYTHON) -c 'import sys; print(1 if sys.version_info >= (3, 0) else 0)')
 ifeq ($(PYTHON3), 1)

--
Alex Bennée

#9 lore raw

PATCH 3/18 Acceptance tests: improve docstring on pick_default_qemu_bin()

In reply to Cleber Rosa

Making it clear what is returned by this utility function.

---
 tests/acceptance/avocado_qemu/__init__.py | 4 ++++
 1 file changed, 4 insertions(+)
tests/acceptance/avocado_qemu/__init__.py +4 -0
index 1e54fd5932..d8d5b48dac 100644
@@ -27,6 +27,10 @@ def pick_default_qemu_bin():
     """
     Picks the path of a QEMU binary, starting either in the current working
     directory or in the source tree root directory.
+
+    :returns: the path to the default QEMU binary or None if one could not
+              be found
+    :rtype: str or None
     """
     arch = os.uname()[4]
     qemu_bin_relative_path = os.path.join("%s-softmmu" % arch,
Signature · 1 line
2.20.1
Reviewed-by on v1 3/18
#10 lore raw

Re: [Qemu-devel] [PATCH 03/18] Acceptance tests: improve docstring on pick_default_qemu_bin()

Quoted 4 lines · Cleber Rosa

Making it clear what is returned by this utility function.

Quoted 22 lines
---
 tests/acceptance/avocado_qemu/__init__.py | 4 ++++
 1 file changed, 4 insertions(+)
tests/acceptance/avocado_qemu/__init__.py +4 -0
index 1e54fd5932..d8d5b48dac 100644
@@ -27,6 +27,10 @@ def pick_default_qemu_bin():
     """
     Picks the path of a QEMU binary, starting either in the current working
     directory or in the source tree root directory.
+
+    :returns: the path to the default QEMU binary or None if one could not
+              be found
+    :rtype: str or None
     """
     arch = os.uname()[4]
     qemu_bin_relative_path = os.path.join("%s-softmmu" % arch,
Signature · 1 line
2.20.1
Signature · 3 lines
Caio Carrara
Software Engineer, Virt Team - Red Hat
[email protected]
#11 lore raw

PATCH 4/18 Acceptance tests: fix doc reference to avocado_qemu directory

In reply to Cleber Rosa

The "this directory" reference is misleading and confusing, it's a leftover from when this text was proposed in a README file inside the "tests/acceptance/avocado_qemu" directory.

When that text was moved to the top level docs directory, the reference was not updated.

---
 docs/devel/testing.rst | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
docs/devel/testing.rst +3 -2
index 18e2c0868a..44c9b3ae74 100644
@@ -590,8 +590,9 @@ Alternatively, follow the instructions on this link:
 Overview
 --------
 
-This directory provides the ``avocado_qemu`` Python module, containing
-the ``avocado_qemu.Test`` class.  Here's a simple usage example:
+The ``tests/acceptance/avocado_qemu`` directory provides the
+``avocado_qemu`` Python module, containing the ``avocado_qemu.Test``
+class.  Here's a simple usage example:
 
 .. code::
 
Signature · 1 line
2.20.1
Reviewed-by on v1 4/18
#12 lore raw

Re: [Qemu-devel] [PATCH 04/18] Acceptance tests: fix doc reference to avocado_qemu directory

Quoted 8 lines · Cleber Rosa

The "this directory" reference is misleading and confusing, it's a leftover from when this text was proposed in a README file inside the "tests/acceptance/avocado_qemu" directory.

When that text was moved to the top level docs directory, the reference was not updated.

Quoted 23 lines
---
 docs/devel/testing.rst | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
docs/devel/testing.rst +3 -2
index 18e2c0868a..44c9b3ae74 100644
@@ -590,8 +590,9 @@ Alternatively, follow the instructions on this link:
 Overview
 --------
 
-This directory provides the ``avocado_qemu`` Python module, containing
-the ``avocado_qemu.Test`` class.  Here's a simple usage example:
+The ``tests/acceptance/avocado_qemu`` directory provides the
+``avocado_qemu`` Python module, containing the ``avocado_qemu.Test``
+class.  Here's a simple usage example:
 
 .. code::
 
Signature · 1 line
2.20.1
Signature · 3 lines
Caio Carrara
Software Engineer, Virt Team - Red Hat
[email protected]
Reviewed-by on v1 4/18
#13 lore raw

Re: [Qemu-devel] [PATCH 04/18] Acceptance tests: fix doc reference to avocado_qemu directory

In reply to Cleber Rosa
Quoted 8 lines · Cleber Rosa

The "this directory" reference is misleading and confusing, it's a leftover from when this text was proposed in a README file inside the "tests/acceptance/avocado_qemu" directory.

When that text was moved to the top level docs directory, the reference was not updated.

Quoted 21 lines
---
 docs/devel/testing.rst | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
docs/devel/testing.rst +3 -2
index 18e2c0868a..44c9b3ae74 100644
@@ -590,8 +590,9 @@ Alternatively, follow the instructions on this link:
 Overview
 --------
 
-This directory provides the ``avocado_qemu`` Python module, containing
-the ``avocado_qemu.Test`` class.  Here's a simple usage example:
+The ``tests/acceptance/avocado_qemu`` directory provides the
+``avocado_qemu`` Python module, containing the ``avocado_qemu.Test``
+class.  Here's a simple usage example:
 
 .. code::
 
#14 lore raw

PATCH 5/18 Acceptance tests: introduce arch parameter and attribute

In reply to Cleber Rosa

It's useful to define the architecture that should be used in situations such as:
* the intended target of the QEMU binary to be used on tests
* the architecture of code to be run within the QEMU binary, such as a kernel image or a full blown guest OS image

This commit introduces both a test parameter and a test instance attribute, that will contain such a value.

Now, when the "arch" test parameter is given, it will influence the selection of the default QEMU binary, if one is not given explicitly by means of the "qemu_img" parameter.

---
 docs/devel/testing.rst                    | 18 ++++++++++++++++++
 tests/acceptance/avocado_qemu/__init__.py | 13 ++++++++++---
 2 files changed, 28 insertions(+), 3 deletions(-)
docs/devel/testing.rst +18 -0
index 44c9b3ae74..60a16c5834 100644
@@ -689,6 +689,17 @@ vm
 A QEMUMachine instance, initially configured according to the given
 ``qemu_bin`` parameter.
 
+arch
+~~~~
+
+The architecture that will be used on a number of different
+scenarios.  For instance, when a QEMU binary is not explicitly given,
+the one selected will depend on this attribute.
+
+The ``arch`` attribute will be set to the test parameter of the same
+name, and if one is not given explicitly, it will default to the
+system architecture (as given by ``uname``).
+
 qemu_bin
 ~~~~~~~~
 
@@ -711,6 +722,13 @@ like the following:
 
   PARAMS (key=qemu_bin, path=*, default=x86_64-softmmu/qemu-system-x86_64) => 'x86_64-softmmu/qemu-system-x86_64
 
+arch
+~~~~
+
+The architecture that will be used on a number of different scenarios.
+This parameter has a direct relation with the ``arch`` attribute.  If
+not given, it will default to None.
+
 qemu_bin
 ~~~~~~~~
 
tests/acceptance/avocado_qemu/__init__.py +10 -3
index d8d5b48dac..7a38851753 100644
@@ -23,16 +23,22 @@ def is_readable_executable_file(path):
     return os.path.isfile(path) and os.access(path, os.R_OK | os.X_OK)
 
 
-def pick_default_qemu_bin():
+def pick_default_qemu_bin(arch=None):
     """
     Picks the path of a QEMU binary, starting either in the current working
     directory or in the source tree root directory.
 
+    :param arch: the arch to use when looking for a QEMU binary (the target
+                 will match the arch given).  If None (the default) arch
+                 will be the current host system arch (as given by
+                 :func:`os.uname`).
+    :type arch: str
     :returns: the path to the default QEMU binary or None if one could not
               be found
     :rtype: str or None
     """
-    arch = os.uname()[4]
+    if arch is None:
+        arch = os.uname()[4]
     qemu_bin_relative_path = os.path.join("%s-softmmu" % arch,
                                           "qemu-system-%s" % arch)
     if is_readable_executable_file(qemu_bin_relative_path):
@@ -47,8 +53,9 @@ def pick_default_qemu_bin():
 class Test(avocado.Test):
     def setUp(self):
         self.vm = None
+        self.arch = self.params.get('arch')
         self.qemu_bin = self.params.get('qemu_bin',
-                                        default=pick_default_qemu_bin())
+                                        default=pick_default_qemu_bin(self.arch))
         if self.qemu_bin is None:
             self.cancel("No QEMU binary defined or found in the source tree")
         self.vm = QEMUMachine(self.qemu_bin)
Signature · 1 line
2.20.1
#15 lore raw

Re: [Qemu-devel] [PATCH 05/18] Acceptance tests: introduce arch parameter and attribute

Hi, Cleber.

Quoted 27 lines · Cleber Rosa

It's useful to define the architecture that should be used in situations such as:
* the intended target of the QEMU binary to be used on tests
* the architecture of code to be run within the QEMU binary, such as a kernel image or a full blown guest OS image

This commit introduces both a test parameter and a test instance attribute, that will contain such a value.

Now, when the "arch" test parameter is given, it will influence the selection of the default QEMU binary, if one is not given explicitly by means of the "qemu_img" parameter.

---
 docs/devel/testing.rst                    | 18 ++++++++++++++++++
 tests/acceptance/avocado_qemu/__init__.py | 13 ++++++++++---
 2 files changed, 28 insertions(+), 3 deletions(-)
docs/devel/testing.rst +0 -0
index 44c9b3ae74..60a16c5834 100644
@@ -689,6 +689,17 @@ vm
 A QEMUMachine instance, initially configured according to the given
 ``qemu_bin`` parameter.
 

docs/devel/testing.rst vm · line 701

Quoted 10 lines
docs/devel/testing.rst +10 -0
+arch
+~~~~
+
+The architecture that will be used on a number of different
+scenarios.  For instance, when a QEMU binary is not explicitly given,
+the one selected will depend on this attribute.
+
+The ``arch`` attribute will be set to the test parameter of the same
+name, and if one is not given explicitly, it will default to the
+system architecture (as given by ``uname``).

Would you think it's important to make explicit that it's the *host* system architecture? Just like you did in the docstrings of pick_default_qemu_bin().

Quoted 48 lines
docs/devel/testing.rst +8 -0
+
 qemu_bin
 ~~~~~~~~
 
@@ -711,6 +722,13 @@ like the following:
 
   PARAMS (key=qemu_bin, path=*, default=x86_64-softmmu/qemu-system-x86_64) => 'x86_64-softmmu/qemu-system-x86_64
 
+arch
+~~~~
+
+The architecture that will be used on a number of different scenarios.
+This parameter has a direct relation with the ``arch`` attribute.  If
+not given, it will default to None.
+
 qemu_bin
 ~~~~~~~~
 
tests/acceptance/avocado_qemu/__init__.py +8 -2
index d8d5b48dac..7a38851753 100644
@@ -23,16 +23,22 @@ def is_readable_executable_file(path):
     return os.path.isfile(path) and os.access(path, os.R_OK | os.X_OK)
 
 
-def pick_default_qemu_bin():
+def pick_default_qemu_bin(arch=None):
     """
     Picks the path of a QEMU binary, starting either in the current working
     directory or in the source tree root directory.
 
+    :param arch: the arch to use when looking for a QEMU binary (the target
+                 will match the arch given).  If None (the default) arch
+                 will be the current host system arch (as given by
+                 :func:`os.uname`).
+    :type arch: str
     :returns: the path to the default QEMU binary or None if one could not
               be found
     :rtype: str or None
     """
-    arch = os.uname()[4]
+    if arch is None:
+        arch = os.uname()[4]
     qemu_bin_relative_path = os.path.join("%s-softmmu" % arch,
                                           "qemu-system-%s" % arch)
     if is_readable_executable_file(qemu_bin_relative_path):

tests/acceptance/avocado_qemu/__init__.py def pick_default_qemu_bin(): · line 58

Quoted 7 lines
tests/acceptance/avocado_qemu/__init__.py +2 -1
@@ -47,8 +53,9 @@ def pick_default_qemu_bin():
 class Test(avocado.Test):
     def setUp(self):
         self.vm = None
+        self.arch = self.params.get('arch')
         self.qemu_bin = self.params.get('qemu_bin',
-                                        default=pick_default_qemu_bin())
+                                        default=pick_default_qemu_bin(self.arch))

Since it was used the named argument in function declaration I think it's more consistent also use the argument name here in the function
call: pick_default_qemu_bin(arch=self.arch)

Quoted 6 lines
tests/acceptance/avocado_qemu/__init__.py +0 -0
         if self.qemu_bin is None:
             self.cancel("No QEMU binary defined or found in the source tree")
         self.vm = QEMUMachine(self.qemu_bin)
Signature · 1 line
2.20.1

Thanks,

Signature · 3 lines
Caio Carrara
Software Engineer, Virt Team - Red Hat
[email protected]
#16 lore raw

Re: [Qemu-devel] [PATCH 05/18] Acceptance tests: introduce arch parameter and attribute

Quoted 30 lines · Caio Carrara

Hi, Cleber.

Quoted 27 lines · Cleber Rosa

It's useful to define the architecture that should be used in situations such as:
* the intended target of the QEMU binary to be used on tests
* the architecture of code to be run within the QEMU binary, such as a kernel image or a full blown guest OS image

This commit introduces both a test parameter and a test instance attribute, that will contain such a value.

Now, when the "arch" test parameter is given, it will influence the selection of the default QEMU binary, if one is not given explicitly by means of the "qemu_img" parameter.

---
 docs/devel/testing.rst                    | 18 ++++++++++++++++++
 tests/acceptance/avocado_qemu/__init__.py | 13 ++++++++++---
 2 files changed, 28 insertions(+), 3 deletions(-)
docs/devel/testing.rst +0 -0
index 44c9b3ae74..60a16c5834 100644
@@ -689,6 +689,17 @@ vm
 A QEMUMachine instance, initially configured according to the given
 ``qemu_bin`` parameter.
 
Quoted 14 lines

docs/devel/testing.rst vm · line 701

Quoted 10 lines
docs/devel/testing.rst +10 -0
+arch
+~~~~
+
+The architecture that will be used on a number of different
+scenarios.  For instance, when a QEMU binary is not explicitly given,
+the one selected will depend on this attribute.
+
+The ``arch`` attribute will be set to the test parameter of the same
+name, and if one is not given explicitly, it will default to the
+system architecture (as given by ``uname``).

Would you think it's important to make explicit that it's the *host* system architecture? Just like you did in the docstrings of pick_default_qemu_bin().

Agreed.

Quoted 33 lines
Quoted 33 lines
docs/devel/testing.rst +8 -0
+
 qemu_bin
 ~~~~~~~~
 
@@ -711,6 +722,13 @@ like the following:
 
   PARAMS (key=qemu_bin, path=*, default=x86_64-softmmu/qemu-system-x86_64) => 'x86_64-softmmu/qemu-system-x86_64
 
+arch
+~~~~
+
+The architecture that will be used on a number of different scenarios.
+This parameter has a direct relation with the ``arch`` attribute.  If
+not given, it will default to None.
+
 qemu_bin
 ~~~~~~~~
 
tests/acceptance/avocado_qemu/__init__.py +1 -1
index d8d5b48dac..7a38851753 100644
@@ -23,16 +23,22 @@ def is_readable_executable_file(path):
     return os.path.isfile(path) and os.access(path, os.R_OK | os.X_OK)
 
 
-def pick_default_qemu_bin():
+def pick_default_qemu_bin(arch=None):
     """
     Picks the path of a QEMU binary, starting either in the current working
     directory or in the source tree root directory.
 
Quoted 4 lines

tests/acceptance/avocado_qemu/__init__.py def is_readable_executable_file(path): · line 34

Quoted 4 lines
tests/acceptance/avocado_qemu/__init__.py +4 -0
+    :param arch: the arch to use when looking for a QEMU binary (the target
+                 will match the arch given).  If None (the default) arch
+                 will be the current host system arch (as given by
+                 :func:`os.uname`).

\o/ thanks :)

Quoted 15 lines
Quoted 15 lines
tests/acceptance/avocado_qemu/__init__.py +3 -1
+    :type arch: str
     :returns: the path to the default QEMU binary or None if one could not
               be found
     :rtype: str or None
     """
-    arch = os.uname()[4]
+    if arch is None:
+        arch = os.uname()[4]
     qemu_bin_relative_path = os.path.join("%s-softmmu" % arch,
                                           "qemu-system-%s" % arch)
     if is_readable_executable_file(qemu_bin_relative_path):
@@ -47,8 +53,9 @@ def pick_default_qemu_bin():
 class Test(avocado.Test):
     def setUp(self):
         self.vm = None
Quoted 8 lines

tests/acceptance/avocado_qemu/__init__.py def pick_default_qemu_bin(): · line 58

Quoted 4 lines
tests/acceptance/avocado_qemu/__init__.py +2 -1
+        self.arch = self.params.get('arch')
         self.qemu_bin = self.params.get('qemu_bin',
-                                        default=pick_default_qemu_bin())
+                                        default=pick_default_qemu_bin(self.arch))

Since it was used the named argument in function declaration I think it's more consistent also use the argument name here in the function
call: pick_default_qemu_bin(arch=self.arch)

With Caio's comments addressed:

Quoted 10 lines
Quoted 7 lines
tests/acceptance/avocado_qemu/__init__.py +0 -0
         if self.qemu_bin is None:
             self.cancel("No QEMU binary defined or found in the source tree")
         self.vm = QEMUMachine(self.qemu_bin)
Signature · 1 line
2.20.1

Thanks,

#17 lore raw

Re: [Qemu-devel] [PATCH 05/18] Acceptance tests: introduce arch parameter and attribute

In reply to Caio Carrara
Quoted 30 lines · Caio Carrara

Hi, Cleber.

Quoted 27 lines · Cleber Rosa

It's useful to define the architecture that should be used in situations such as:
* the intended target of the QEMU binary to be used on tests
* the architecture of code to be run within the QEMU binary, such as a kernel image or a full blown guest OS image

This commit introduces both a test parameter and a test instance attribute, that will contain such a value.

Now, when the "arch" test parameter is given, it will influence the selection of the default QEMU binary, if one is not given explicitly by means of the "qemu_img" parameter.

---
 docs/devel/testing.rst                    | 18 ++++++++++++++++++
 tests/acceptance/avocado_qemu/__init__.py | 13 ++++++++++---
 2 files changed, 28 insertions(+), 3 deletions(-)
docs/devel/testing.rst +0 -0
index 44c9b3ae74..60a16c5834 100644
@@ -689,6 +689,17 @@ vm
 A QEMUMachine instance, initially configured according to the given
 ``qemu_bin`` parameter.
 
Quoted 15 lines

docs/devel/testing.rst vm · line 701

Quoted 10 lines
docs/devel/testing.rst +10 -0
+arch
+~~~~
+
+The architecture that will be used on a number of different
+scenarios.  For instance, when a QEMU binary is not explicitly given,
+the one selected will depend on this attribute.
+
+The ``arch`` attribute will be set to the test parameter of the same
+name, and if one is not given explicitly, it will default to the
+system architecture (as given by ``uname``).

Would you think it's important to make explicit that it's the *host* system architecture? Just like you did in the docstrings of pick_default_qemu_bin().

Yes, you're right. Adding that.

#18 lore raw

Re: [Qemu-devel] [PATCH 05/18] Acceptance tests: introduce arch parameter and attribute

In reply to Caio Carrara
Quoted 96 lines · Caio Carrara

Hi, Cleber.

Quoted 27 lines · Cleber Rosa

It's useful to define the architecture that should be used in situations such as:
* the intended target of the QEMU binary to be used on tests
* the architecture of code to be run within the QEMU binary, such as a kernel image or a full blown guest OS image

This commit introduces both a test parameter and a test instance attribute, that will contain such a value.

Now, when the "arch" test parameter is given, it will influence the selection of the default QEMU binary, if one is not given explicitly by means of the "qemu_img" parameter.

---
 docs/devel/testing.rst                    | 18 ++++++++++++++++++
 tests/acceptance/avocado_qemu/__init__.py | 13 ++++++++++---
 2 files changed, 28 insertions(+), 3 deletions(-)
docs/devel/testing.rst +0 -0
index 44c9b3ae74..60a16c5834 100644
@@ -689,6 +689,17 @@ vm
 A QEMUMachine instance, initially configured according to the given
 ``qemu_bin`` parameter.
 

docs/devel/testing.rst vm · line 701

Quoted 10 lines
docs/devel/testing.rst +10 -0
+arch
+~~~~
+
+The architecture that will be used on a number of different
+scenarios.  For instance, when a QEMU binary is not explicitly given,
+the one selected will depend on this attribute.
+
+The ``arch`` attribute will be set to the test parameter of the same
+name, and if one is not given explicitly, it will default to the
+system architecture (as given by ``uname``).

Would you think it's important to make explicit that it's the *host* system architecture? Just like you did in the docstrings of pick_default_qemu_bin().

Quoted 51 lines
docs/devel/testing.rst +8 -0
+
 qemu_bin
 ~~~~~~~~
 
@@ -711,6 +722,13 @@ like the following:
 
   PARAMS (key=qemu_bin, path=*, default=x86_64-softmmu/qemu-system-x86_64) => 'x86_64-softmmu/qemu-system-x86_64
 
+arch
+~~~~
+
+The architecture that will be used on a number of different scenarios.
+This parameter has a direct relation with the ``arch`` attribute.  If
+not given, it will default to None.
+
 qemu_bin
 ~~~~~~~~
 
tests/acceptance/avocado_qemu/__init__.py +8 -2
index d8d5b48dac..7a38851753 100644
@@ -23,16 +23,22 @@ def is_readable_executable_file(path):
     return os.path.isfile(path) and os.access(path, os.R_OK | os.X_OK)
 
 
-def pick_default_qemu_bin():
+def pick_default_qemu_bin(arch=None):
     """
     Picks the path of a QEMU binary, starting either in the current working
     directory or in the source tree root directory.
 
+    :param arch: the arch to use when looking for a QEMU binary (the target
+                 will match the arch given).  If None (the default) arch
+                 will be the current host system arch (as given by
+                 :func:`os.uname`).
+    :type arch: str
     :returns: the path to the default QEMU binary or None if one could not
               be found
     :rtype: str or None
     """
-    arch = os.uname()[4]
+    if arch is None:
+        arch = os.uname()[4]
     qemu_bin_relative_path = os.path.join("%s-softmmu" % arch,
                                           "qemu-system-%s" % arch)
     if is_readable_executable_file(qemu_bin_relative_path):
@@ -47,8 +53,9 @@ def pick_default_qemu_bin():
 class Test(avocado.Test):
     def setUp(self):
         self.vm = None
Quoted 9 lines

tests/acceptance/avocado_qemu/__init__.py def pick_default_qemu_bin(): · line 58

Quoted 4 lines
tests/acceptance/avocado_qemu/__init__.py +2 -1
+        self.arch = self.params.get('arch')
         self.qemu_bin = self.params.get('qemu_bin',
-                                        default=pick_default_qemu_bin())
+                                        default=pick_default_qemu_bin(self.arch))

Since it was used the named argument in function declaration I think it's more consistent also use the argument name here in the function
call: pick_default_qemu_bin(arch=self.arch)

Oops, I missed this part of your review in the previous message. Anyway, I don't see a problem with either approach, so I'm fine changing it.

- Cleber.

#19 lore raw

Re: [Qemu-devel] [PATCH 05/18] Acceptance tests: introduce arch parameter and attribute

In reply to Cleber Rosa

Hi Cleber,

Quoted 27 lines · Cleber Rosa

It's useful to define the architecture that should be used in situations such as:
* the intended target of the QEMU binary to be used on tests
* the architecture of code to be run within the QEMU binary, such as a kernel image or a full blown guest OS image

This commit introduces both a test parameter and a test instance attribute, that will contain such a value.

Now, when the "arch" test parameter is given, it will influence the selection of the default QEMU binary, if one is not given explicitly by means of the "qemu_img" parameter.

---
  docs/devel/testing.rst                    | 18 ++++++++++++++++++
  tests/acceptance/avocado_qemu/__init__.py | 13 ++++++++++---
  2 files changed, 28 insertions(+), 3 deletions(-)
docs/devel/testing.rst +0 -0
index 44c9b3ae74..60a16c5834 100644
@@ -689,6 +689,17 @@ vm
  A QEMUMachine instance, initially configured according to the given
  ``qemu_bin`` parameter.
 

docs/devel/testing.rst vm · line 701

Quoted 10 lines
docs/devel/testing.rst +10 -0
+arch
+~~~~
+
+The architecture that will be used on a number of different
+scenarios.  For instance, when a QEMU binary is not explicitly given,
+the one selected will depend on this attribute.
+
+The ``arch`` attribute will be set to the test parameter of the same
+name, and if one is not given explicitly, it will default to the
+system architecture (as given by ``uname``).

Unless I'm missing something, the arch attribute will be None if I don't pass the arch parameter nor set the arch tag.

------
[root@xxx qemu]# cat tests/acceptance/arch.py
from avocado_qemu import Test

class Arch(Test):
    """
    :avocado: enable
    """
    def test(self):
        self.assertEqual(self.arch, "x86_64")
[root@xxx qemu]# ./tests/venv/bin/avocado run tests/acceptance/arch.py JOB ID     : 032aabb8ec2364944d0e29df3b61931b42f4587c
JOB LOG    : /root/avocado/job-results/job-2019-01-31T08.42-032aabb/job.log (1/1) tests/acceptance/arch.py:Arch.test: FAIL: None != 'x86_64' (0.03 s) RESULTS    : PASS 0 | ERROR 0 | FAIL 1 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB TIME   : 0.20 s
[root@xxx qemu]# ./tests/venv/bin/avocado run -p arch=x86_64 tests/acceptance/arch.py JOB ID     : 1913a73510cab4b6b118bd7e94e2746011368628
JOB LOG    : /root/avocado/job-results/job-2019-01-31T08.42-1913a73/job.log (1/1) tests/acceptance/arch.py:Arch.test: PASS (0.01 s)
RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB TIME   : 0.19 s
[root@xxx qemu]# cat tests/acceptance/arch_tag.py
from avocado_qemu import Test

class Arch(Test):
    """
    :avocado: enable
    :avocado: tags=arch:x86_64
    """
    def test(self):
        self.assertEqual(self.arch, "x86_64")
[root@xxx qemu]# ./tests/venv/bin/avocado run tests/acceptance/arch_tag.py JOB ID     : bc85be065586b8e9cf371489e1801489910e41b0
JOB LOG    : /root/avocado/job-results/job-2019-01-31T08.43-bc85be0/job.log (1/1) tests/acceptance/arch_tag.py:Arch.test: PASS (0.01 s)
RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB TIME   : 0.19 s
------

- Wainer

Quoted 58 lines
docs/devel/testing.rst +8 -0
+
  qemu_bin
  ~~~~~~~~
 
@@ -711,6 +722,13 @@ like the following:
 
    PARAMS (key=qemu_bin, path=*, default=x86_64-softmmu/qemu-system-x86_64) => 'x86_64-softmmu/qemu-system-x86_64
 
+arch
+~~~~
+
+The architecture that will be used on a number of different scenarios.
+This parameter has a direct relation with the ``arch`` attribute.  If
+not given, it will default to None.
+
  qemu_bin
  ~~~~~~~~
 
tests/acceptance/avocado_qemu/__init__.py +10 -3
index d8d5b48dac..7a38851753 100644
@@ -23,16 +23,22 @@ def is_readable_executable_file(path):
      return os.path.isfile(path) and os.access(path, os.R_OK | os.X_OK)
 
 
-def pick_default_qemu_bin():
+def pick_default_qemu_bin(arch=None):
      """
      Picks the path of a QEMU binary, starting either in the current working
      directory or in the source tree root directory.
 
+    :param arch: the arch to use when looking for a QEMU binary (the target
+                 will match the arch given).  If None (the default) arch
+                 will be the current host system arch (as given by
+                 :func:`os.uname`).
+    :type arch: str
      :returns: the path to the default QEMU binary or None if one could not
                be found
      :rtype: str or None
      """
-    arch = os.uname()[4]
+    if arch is None:
+        arch = os.uname()[4]
      qemu_bin_relative_path = os.path.join("%s-softmmu" % arch,
                                            "qemu-system-%s" % arch)
      if is_readable_executable_file(qemu_bin_relative_path):
@@ -47,8 +53,9 @@ def pick_default_qemu_bin():
  class Test(avocado.Test):
      def setUp(self):
          self.vm = None
+        self.arch = self.params.get('arch')
          self.qemu_bin = self.params.get('qemu_bin',
-                                        default=pick_default_qemu_bin())
+                                        default=pick_default_qemu_bin(self.arch))
          if self.qemu_bin is None:
              self.cancel("No QEMU binary defined or found in the source tree")
          self.vm = QEMUMachine(self.qemu_bin)
#20 lore raw

Re: [Qemu-devel] [PATCH 05/18] Acceptance tests: introduce arch parameter and attribute

Quoted 34 lines · Wainer dos Santos Moschetta

Hi Cleber,

Quoted 31 lines · Cleber Rosa

It's useful to define the architecture that should be used in situations such as:
  * the intended target of the QEMU binary to be used on tests
  * the architecture of code to be run within the QEMU binary, such as a kernel image or a full blown guest OS image

This commit introduces both a test parameter and a test instance attribute, that will contain such a value.

Now, when the "arch" test parameter is given, it will influence the selection of the default QEMU binary, if one is not given explicitly by means of the "qemu_img" parameter.

---
  docs/devel/testing.rst                    | 18 ++++++++++++++++++ tests/acceptance/avocado_qemu/__init__.py | 13 ++++++++++---

  2 files changed, 28 insertions(+), 3 deletions(-)
docs/devel/testing.rst +0 -0
index 44c9b3ae74..60a16c5834 100644
@@ -689,6 +689,17 @@ vm

  A QEMUMachine instance, initially configured according to the given ``qemu_bin`` parameter.
  +arch
+~~~~

docs/devel/testing.rst +3 -0
+
+The architecture that will be used on a number of different
+scenarios.  For instance, when a QEMU binary is not explicitly given,
Quoted 9 lines

docs/devel/testing.rst vm · line 701

Quoted 5 lines
docs/devel/testing.rst +5 -0
+the one selected will depend on this attribute.
+
+The ``arch`` attribute will be set to the test parameter of the same
+name, and if one is not given explicitly, it will default to the
+system architecture (as given by ``uname``).

Unless I'm missing something, the arch attribute will be None if I don't pass the arch parameter nor set the arch tag.

I think you're not missing anything, but I was. Thanks for catching that.

- Cleber.

Quoted 106 lines

------
[root@xxx qemu]# cat tests/acceptance/arch.py
from avocado_qemu import Test

class Arch(Test):
    """
    :avocado: enable
    """
    def test(self):
        self.assertEqual(self.arch, "x86_64")
[root@xxx qemu]# ./tests/venv/bin/avocado run tests/acceptance/arch.py JOB ID     : 032aabb8ec2364944d0e29df3b61931b42f4587c
JOB LOG    : /root/avocado/job-results/job-2019-01-31T08.42-032aabb/job.log (1/1) tests/acceptance/arch.py:Arch.test: FAIL: None != 'x86_64' (0.03 s) RESULTS    : PASS 0 | ERROR 0 | FAIL 1 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB TIME   : 0.20 s
[root@xxx qemu]# ./tests/venv/bin/avocado run -p arch=x86_64 tests/acceptance/arch.py
JOB ID     : 1913a73510cab4b6b118bd7e94e2746011368628
JOB LOG    : /root/avocado/job-results/job-2019-01-31T08.42-1913a73/job.log (1/1) tests/acceptance/arch.py:Arch.test: PASS (0.01 s) RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB TIME   : 0.19 s
[root@xxx qemu]# cat tests/acceptance/arch_tag.py
from avocado_qemu import Test

class Arch(Test):
    """
    :avocado: enable
    :avocado: tags=arch:x86_64
    """
    def test(self):
        self.assertEqual(self.arch, "x86_64")
[root@xxx qemu]# ./tests/venv/bin/avocado run tests/acceptance/arch_tag.py JOB ID     : bc85be065586b8e9cf371489e1801489910e41b0
JOB LOG    : /root/avocado/job-results/job-2019-01-31T08.43-bc85be0/job.log (1/1) tests/acceptance/arch_tag.py:Arch.test: PASS (0.01 s) RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB TIME   : 0.19 s
------

- Wainer

Quoted 59 lines

+
  qemu_bin
  ~~~~~~~~
  @@ -711,6 +722,13 @@ like the following:
      PARAMS (key=qemu_bin, path=*,
default=x86_64-softmmu/qemu-system-x86_64) =>
'x86_64-softmmu/qemu-system-x86_64
  +arch
+~~~~

docs/devel/testing.rst +5 -0
+
+The architecture that will be used on a number of different scenarios.
+This parameter has a direct relation with the ``arch`` attribute.  If
+not given, it will default to None.
+

  qemu_bin
  ~~~~~~~~
  diff --git a/tests/acceptance/avocado_qemu/__init__.py b/tests/acceptance/avocado_qemu/__init__.py index d8d5b48dac..7a38851753 100644

tests/acceptance/avocado_qemu/__init__.py +0 -0
@@ -23,16 +23,22 @@ def is_readable_executable_file(path):
       return os.path.isfile(path) and os.access(path, os.R_OK | os.X_OK)

    -def pick_default_qemu_bin():
+def pick_default_qemu_bin(arch=None):
      """
      Picks the path of a QEMU binary, starting either in the current working
      directory or in the source tree root directory.
  +    :param arch: the arch to use when looking for a QEMU binary (the target
+                 will match the arch given).  If None (the default) arch +                 will be the current host system arch (as given by +                 :func:os.uname).
+    :type arch: str
      :returns: the path to the default QEMU binary or None if one could not
                be found
      :rtype: str or None
      """
-    arch = os.uname()[4]
+    if arch is None:
+        arch = os.uname()[4]
      qemu_bin_relative_path = os.path.join("%s-softmmu" % arch, "qemu-system-%s" % arch) if is_readable_executable_file(qemu_bin_relative_path):

tests/acceptance/avocado_qemu/__init__.py +0 -0
@@ -47,8 +53,9 @@ def pick_default_qemu_bin():

  class Test(avocado.Test):
      def setUp(self):
          self.vm = None
+        self.arch = self.params.get('arch')
          self.qemu_bin = self.params.get('qemu_bin',
-                                        default=pick_default_qemu_bin()) +
default=pick_default_qemu_bin(self.arch))
          if self.qemu_bin is None:
              self.cancel("No QEMU binary defined or found in the source tree")
          self.vm = QEMUMachine(self.qemu_bin)

Signature · 4 lines
Cleber Rosa
[ Sr Software Engineer - Virtualization Team - Red Hat ]
[ Avocado Test Framework - avocado-framework.github.io ]
[  7ABB 96EB 8B46 B94D 5E0F  E9BB 657E 8D33 A5F2 09F3  ]
#21 lore raw

PATCH 6/18 Acceptance tests: use "arch:" tag to filter target specific tests

In reply to Cleber Rosa

Currently, the only test that contains some target architecture information is "boot_linux_console.py" which test contains a "x86_64" tag. But that tag is not respected in the default execution, that is, "make check-acceptance" doesn't do anything with it.

That said, even the target architecture handling currently present in the "avocado_qemu.Test" class, class is pretty limited. For instance, by default, it chooses a target based on the host architecture.

Because the original implementation of the tags feature in Avocado did not include any time of namespace or "key:val" mechanism, no tag has relation to another tag. The new implementation of the tags feature from version 67.0 onwards, allows "key:val" tags, and because of that, a test can be classified with a tag in a given key. For instance, the new proposed version of the "boot_linux_console.py" test, which downloads and attempts to run a x86_64 kernel, is now tagged as:

:avocado: tags=arch:x86_64

This means that it can be filtered (out) when no x86_64 target is available. At the same time, tests that don't have a "arch:" tag, will not be filtered out.

---
 tests/Makefile.include                 | 3 +++
 tests/acceptance/boot_linux_console.py | 2 +-
 tests/requirements.txt                 | 2 +-
 3 files changed, 5 insertions(+), 2 deletions(-)
tests/Makefile.include +3 -0
index c73298740d..af00c35ff5 100644
@@ -957,6 +957,7 @@ TESTS_RESULTS_DIR=$(BUILD_DIR)/tests/results
 # Any number of command separated loggers are accepted.  For more
 # information please refer to "avocado --help".
 AVOCADO_SHOW=app
+AVOCADO_TAGS=$(patsubst %-softmmu,-t arch:%, $(filter %-softmmu,$(TARGET_DIRS)))
 
 PYTHON3 = $(shell $(PYTHON) -c 'import sys; print(1 if sys.version_info >= (3, 0) else 0)')
 ifeq ($(PYTHON3), 1)
@@ -983,6 +984,8 @@ check-acceptance: check-venv $(TESTS_RESULTS_DIR)
 	$(call quiet-command, \
             $(TESTS_VENV_DIR)/bin/python -m avocado \
             --show=$(AVOCADO_SHOW) run --job-results-dir=$(TESTS_RESULTS_DIR) \
+            --filter-by-tags-include-empty --filter-by-tags-include-empty-key \
+            $(AVOCADO_TAGS) \
             --failfast=on $(SRC_PATH)/tests/acceptance, \
             "AVOCADO", "tests/acceptance")
 
tests/acceptance/boot_linux_console.py +1 -1
index 98324f7591..46b20bdfe2 100644
@@ -19,7 +19,7 @@ class BootLinuxConsole(Test):
     and the kernel command line is properly passed from QEMU to the kernel
 
     :avocado: enable
-    :avocado: tags=x86_64
+    :avocado: tags=arch:x86_64
     """
 
     timeout = 60
tests/requirements.txt +1 -1
index 64c6e27a94..426a9e8a9a 100644
@@ -1,4 +1,4 @@
 # Add Python module requirements, one per line, to be installed
 # in the tests/venv Python virtual environment. For more info,
 # refer to: https://pip.pypa.io/en/stable/user_guide/#id1
-avocado-framework==65.0
+avocado-framework==67.0
Signature · 1 line
2.20.1
#22 lore raw

Re: [Qemu-devel] [PATCH 06/18] Acceptance tests: use "arch:" tag to filter target specific tests

Quoted 36 lines · On Thu, 17 Jan 2019 13:56:16 -0500 Cleber Rosa <[email protected]> wrote:

Currently, the only test that contains some target architecture information is "boot_linux_console.py" which test contains a "x86_64" tag. But that tag is not respected in the default execution, that is, "make check-acceptance" doesn't do anything with it.

That said, even the target architecture handling currently present in the "avocado_qemu.Test" class, class is pretty limited. For instance, by default, it chooses a target based on the host architecture.

Because the original implementation of the tags feature in Avocado did not include any time of namespace or "key:val" mechanism, no tag has relation to another tag. The new implementation of the tags feature from version 67.0 onwards, allows "key:val" tags, and because of that, a test can be classified with a tag in a given key. For instance, the new proposed version of the "boot_linux_console.py" test, which downloads and attempts to run a x86_64 kernel, is now tagged as:

:avocado: tags=arch:x86_64

This means that it can be filtered (out) when no x86_64 target is available. At the same time, tests that don't have a "arch:" tag, will not be filtered out.

---
 tests/Makefile.include                 | 3 +++
 tests/acceptance/boot_linux_console.py | 2 +-
 tests/requirements.txt                 | 2 +-
 3 files changed, 5 insertions(+), 2 deletions(-)
tests/acceptance/boot_linux_console.py +0 -0
index 98324f7591..46b20bdfe2 100644

tests/acceptance/boot_linux_console.py class BootLinuxConsole(Test): · line 25

Quoted 8 lines
tests/acceptance/boot_linux_console.py +1 -1
@@ -19,7 +19,7 @@ class BootLinuxConsole(Test):
     and the kernel command line is properly passed from QEMU to the kernel
 
     :avocado: enable
-    :avocado: tags=x86_64
+    :avocado: tags=arch:x86_64
     """
 
     timeout = 60

You probably want to do the same change in virtio_version.py; otherwise, if I run the acceptance tests on s390x, it will run into timeouts (it looks like that test is intended to be run with x86 machines anyway.)

#23 lore raw

Re: [Qemu-devel] [PATCH 06/18] Acceptance tests: use "arch:" tag to filter target specific tests

Quoted 46 lines · Cornelia Huck
Quoted 43 lines · On Thu, 17 Jan 2019 13:56:16 -0500 Cleber Rosa <[email protected]> wrote:

Currently, the only test that contains some target architecture information is "boot_linux_console.py" which test contains a "x86_64" tag. But that tag is not respected in the default execution, that is, "make check-acceptance" doesn't do anything with it.

That said, even the target architecture handling currently present in the "avocado_qemu.Test" class, class is pretty limited. For instance, by default, it chooses a target based on the host architecture.

Because the original implementation of the tags feature in Avocado did not include any time of namespace or "key:val" mechanism, no tag has relation to another tag. The new implementation of the tags feature from version 67.0 onwards, allows "key:val" tags, and because of that, a test can be classified with a tag in a given key. For instance, the new proposed version of the "boot_linux_console.py" test, which downloads and attempts to run a x86_64 kernel, is now tagged as:

:avocado: tags=arch:x86_64

This means that it can be filtered (out) when no x86_64 target is available. At the same time, tests that don't have a "arch:" tag, will not be filtered out.

---
 tests/Makefile.include                 | 3 +++
 tests/acceptance/boot_linux_console.py | 2 +-
 tests/requirements.txt                 | 2 +-
 3 files changed, 5 insertions(+), 2 deletions(-)
tests/acceptance/boot_linux_console.py +1 -1
index 98324f7591..46b20bdfe2 100644
@@ -19,7 +19,7 @@ class BootLinuxConsole(Test):
     and the kernel command line is properly passed from QEMU to the kernel
 
     :avocado: enable
-    :avocado: tags=x86_64
+    :avocado: tags=arch:x86_64
     """
 
Quoted 7 lines

tests/acceptance/boot_linux_console.py class BootLinuxConsole(Test): · line 25

Quoted 1 line
tests/acceptance/boot_linux_console.py +0 -0
     timeout = 60

You probably want to do the same change in virtio_version.py; otherwise, if I run the acceptance tests on s390x, it will run into timeouts (it looks like that test is intended to be run with x86 machines anyway.)

Right, good catch. Also, the same applies to the "linux_initrd.py" test, merged after this version.

- Cleber.

Reviewed-by on v1 6/18
#24 lore raw

Re: [Qemu-devel] [PATCH 06/18] Acceptance tests: use "arch:" tag to filter target specific tests

In reply to Cleber Rosa
Quoted 17 lines · Cleber Rosa

Currently, the only test that contains some target architecture information is "boot_linux_console.py" which test contains a "x86_64" tag. But that tag is not respected in the default execution, that is, "make check-acceptance" doesn't do anything with it.

That said, even the target architecture handling currently present in the "avocado_qemu.Test" class, class is pretty limited. For instance, by default, it chooses a target based on the host architecture.

Because the original implementation of the tags feature in Avocado did not include any time of namespace or "key:val" mechanism, no tag has relation to another tag. The new implementation of the tags feature from version 67.0 onwards, allows "key:val" tags, and because of that, a test can be classified with a tag in a given key. For instance, the new proposed version of the "boot_linux_console.py" test, which downloads and attempts to run a x86_64 kernel, is now tagged as:

Quoted 7 lines

:avocado: tags=arch:x86_64

This means that it can be filtered (out) when no x86_64 target is available. At the same time, tests that don't have a "arch:" tag, will not be filtered out.

Quoted 6 lines
---
 tests/Makefile.include                 | 3 +++
 tests/acceptance/boot_linux_console.py | 2 +-
 tests/requirements.txt                 | 2 +-
 3 files changed, 5 insertions(+), 2 deletions(-)

{...}

Quoted 1 line
Signature · 3 lines
Caio Carrara
Software Engineer, Virt Team - Red Hat
[email protected]
#25 lore raw

PATCH 7/18 Acceptance tests: look for target architecture in test tags first

In reply to Cleber Rosa

A test can, optionally, be tagged for one or many architectures. If a test has been tagged for a single architecture, there's a high chance that the test won't run on other architectures. This changes the default order of choosing a default target architecture to use based on the 'arch' tag value first.

The precedence order is for choosing a QEMU binary to use for a test is now:

* qemu_bin parameter
* arch parameter
* arch tag value (for example, x86_64 if ":avocado: tags=arch:x86_64 is used)

This means that if one runs:

 $ avocado run -p qemu_bin=/usr/bin/qemu-system-x86_64 test.py

No arch parameter or tag will influence the selection of the QEMU target binary. If one runs:

 $ avocado run -p arch=ppc64 test.py

The target binary selection mechanism will attempt to find a binary such as "ppc64-softmmu/qemu-system-ppc64". And finally, if one runs a test that is tagged (in its docstring) with "arch:aarch64":

 $ avocado run aarch64.py

The target binary selection mechanism will attempt to find a binary such as "aarch64-softmmu/qemu-system-aarch64".

At this time, no provision is made to cancel the execution of tests if the arch parameter given (manually) does not match the test "arch" tag, but it may be a useful default behavior to be added in the future.

---
 tests/acceptance/avocado_qemu/__init__.py | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
tests/acceptance/avocado_qemu/__init__.py +6 -1
index 7a38851753..ddbdb7b926 100644
@@ -53,7 +53,12 @@ def pick_default_qemu_bin(arch=None):
 class Test(avocado.Test):
     def setUp(self):
         self.vm = None
-        self.arch = self.params.get('arch')
+        arches = self.tags.get('arch', [])
+        if len(arches) == 1:
+            arch = arches.pop()
+        else:
+            arch = None
+        self.arch = self.params.get('arch', default=arch)
         self.qemu_bin = self.params.get('qemu_bin',
                                         default=pick_default_qemu_bin(self.arch))
         if self.qemu_bin is None:
Signature · 1 line
2.20.1
Reviewed-by on v1 7/18
#26 lore raw

Re: [Qemu-devel] [PATCH 07/18] Acceptance tests: look for target architecture in test tags first

Quoted 32 lines · Cleber Rosa

A test can, optionally, be tagged for one or many architectures. If a test has been tagged for a single architecture, there's a high chance that the test won't run on other architectures. This changes the default order of choosing a default target architecture to use based on the 'arch' tag value first.

The precedence order is for choosing a QEMU binary to use for a test is now:

* qemu_bin parameter
* arch parameter
* arch tag value (for example, x86_64 if ":avocado: tags=arch:x86_64 is used)

This means that if one runs:

 $ avocado run -p qemu_bin=/usr/bin/qemu-system-x86_64 test.py

No arch parameter or tag will influence the selection of the QEMU target binary. If one runs:

 $ avocado run -p arch=ppc64 test.py

The target binary selection mechanism will attempt to find a binary such as "ppc64-softmmu/qemu-system-ppc64". And finally, if one runs a test that is tagged (in its docstring) with "arch:aarch64":

 $ avocado run aarch64.py

The target binary selection mechanism will attempt to find a binary such as "aarch64-softmmu/qemu-system-aarch64".

Quoted 6 lines

At this time, no provision is made to cancel the execution of tests if the arch parameter given (manually) does not match the test "arch" tag, but it may be a useful default behavior to be added in the future.

Quoted 25 lines
---
 tests/acceptance/avocado_qemu/__init__.py | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
tests/acceptance/avocado_qemu/__init__.py +6 -1
index 7a38851753..ddbdb7b926 100644
@@ -53,7 +53,12 @@ def pick_default_qemu_bin(arch=None):
 class Test(avocado.Test):
     def setUp(self):
         self.vm = None
-        self.arch = self.params.get('arch')
+        arches = self.tags.get('arch', [])
+        if len(arches) == 1:
+            arch = arches.pop()
+        else:
+            arch = None
+        self.arch = self.params.get('arch', default=arch)
         self.qemu_bin = self.params.get('qemu_bin',
                                         default=pick_default_qemu_bin(self.arch))
         if self.qemu_bin is None:
Signature · 1 line
2.20.1
Signature · 3 lines
Caio Carrara
Software Engineer, Virt Team - Red Hat
[email protected]
#27 lore raw

PATCH 8/18 Boot Linux Console Test: rename the x86_64 after the arch and machine

In reply to Cleber Rosa

Given that the test is specific to x86_64 and pc, and new tests are going to be added to the same class, let's rename it accordingly. Also, let's make the class documentation not architecture specific.

---
 tests/acceptance/boot_linux_console.py | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)
tests/acceptance/boot_linux_console.py +7 -4
index 46b20bdfe2..89df7f6e4f 100644
@@ -15,16 +15,19 @@ from avocado_qemu import Test
 
 class BootLinuxConsole(Test):
     """
-    Boots a x86_64 Linux kernel and checks that the console is operational
-    and the kernel command line is properly passed from QEMU to the kernel
+    Boots a Linux kernel and checks that the console is operational and the
+    kernel command line is properly passed from QEMU to the kernel
 
     :avocado: enable
-    :avocado: tags=arch:x86_64
     """
 
     timeout = 60
 
-    def test(self):
+    def test_x86_64_pc(self):
+        """
+        :avocado: tags=arch:x86_64
+        :avocado: tags=machine:pc
+        """
         kernel_url = ('https://mirrors.kernel.org/fedora/releases/28/'
                       'Everything/x86_64/os/images/pxeboot/vmlinuz')
         kernel_hash = '238e083e114c48200f80d889f7e32eeb2793e02a'
Signature · 1 line
2.20.1
Reviewed-by on v1 8/18
#28 lore raw

Re: [Qemu-devel] [PATCH 08/18] Boot Linux Console Test: rename the x86_64 after the arch and machine

Quoted 5 lines · Cleber Rosa

Given that the test is specific to x86_64 and pc, and new tests are going to be added to the same class, let's rename it accordingly. Also, let's make the class documentation not architecture specific.

Quoted 35 lines
---
 tests/acceptance/boot_linux_console.py | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)
tests/acceptance/boot_linux_console.py +7 -4
index 46b20bdfe2..89df7f6e4f 100644
@@ -15,16 +15,19 @@ from avocado_qemu import Test
 
 class BootLinuxConsole(Test):
     """
-    Boots a x86_64 Linux kernel and checks that the console is operational
-    and the kernel command line is properly passed from QEMU to the kernel
+    Boots a Linux kernel and checks that the console is operational and the
+    kernel command line is properly passed from QEMU to the kernel
 
     :avocado: enable
-    :avocado: tags=arch:x86_64
     """
 
     timeout = 60
 
-    def test(self):
+    def test_x86_64_pc(self):
+        """
+        :avocado: tags=arch:x86_64
+        :avocado: tags=machine:pc
+        """
         kernel_url = ('https://mirrors.kernel.org/fedora/releases/28/'
                       'Everything/x86_64/os/images/pxeboot/vmlinuz')
         kernel_hash = '238e083e114c48200f80d889f7e32eeb2793e02a'
Signature · 1 line
2.20.1
Signature · 3 lines
Caio Carrara
Software Engineer, Virt Team - Red Hat
[email protected]
Reviewed-by on v1 8/18
#29 lore raw

Re: [Qemu-devel] [PATCH 08/18] Boot Linux Console Test: rename the x86_64 after the arch and machine

In reply to Cleber Rosa
Quoted 28 lines · Cleber Rosa

Given that the test is specific to x86_64 and pc, and new tests are going to be added to the same class, let's rename it accordingly. Also, let's make the class documentation not architecture specific.

---
 tests/acceptance/boot_linux_console.py | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)
tests/acceptance/boot_linux_console.py +2 -3
index 46b20bdfe2..89df7f6e4f 100644
@@ -15,16 +15,19 @@ from avocado_qemu import Test
 
 class BootLinuxConsole(Test):
     """
-    Boots a x86_64 Linux kernel and checks that the console is operational
-    and the kernel command line is properly passed from QEMU to the kernel
+    Boots a Linux kernel and checks that the console is operational and the
+    kernel command line is properly passed from QEMU to the kernel
 
     :avocado: enable
-    :avocado: tags=arch:x86_64
     """
 
     timeout = 60

tests/acceptance/boot_linux_console.py from avocado_qemu import Test · line 33

Quoted 10 lines
tests/acceptance/boot_linux_console.py +5 -1
 
-    def test(self):
+    def test_x86_64_pc(self):
+        """
+        :avocado: tags=arch:x86_64
+        :avocado: tags=machine:pc
+        """
         kernel_url = ('https://mirrors.kernel.org/fedora/releases/28/'
                       'Everything/x86_64/os/images/pxeboot/vmlinuz')
         kernel_hash = '238e083e114c48200f80d889f7e32eeb2793e02a'
#30 lore raw

PATCH 9/18 Boot Linux Console Test: update the x86_64 kernel

In reply to Cleber Rosa

To the stock Fedora 29 kernel, from the Fedora 28. New tests will be added using the 29 kernel, so for consistency, let's also update it here.

---
 tests/acceptance/boot_linux_console.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
tests/acceptance/boot_linux_console.py +2 -2
index 89df7f6e4f..35b31162d4 100644
@@ -28,9 +28,9 @@ class BootLinuxConsole(Test):
         :avocado: tags=arch:x86_64
         :avocado: tags=machine:pc
         """
-        kernel_url = ('https://mirrors.kernel.org/fedora/releases/28/'
+        kernel_url = ('https://mirrors.kernel.org/fedora/releases/29/'
                       'Everything/x86_64/os/images/pxeboot/vmlinuz')
-        kernel_hash = '238e083e114c48200f80d889f7e32eeb2793e02a'
+        kernel_hash = '23bebd2680757891cf7adedb033532163a792495'
         kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
 
         self.vm.set_machine('pc')
Signature · 1 line
2.20.1
Reviewed-by on v1 9/18
#31 lore raw

Re: [Qemu-devel] [PATCH 09/18] Boot Linux Console Test: update the x86_64 kernel

Quoted 5 lines · Cleber Rosa

To the stock Fedora 29 kernel, from the Fedora 28. New tests will be added using the 29 kernel, so for consistency, let's also update it here.

Quoted 23 lines
---
 tests/acceptance/boot_linux_console.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
tests/acceptance/boot_linux_console.py +2 -2
index 89df7f6e4f..35b31162d4 100644
@@ -28,9 +28,9 @@ class BootLinuxConsole(Test):
         :avocado: tags=arch:x86_64
         :avocado: tags=machine:pc
         """
-        kernel_url = ('https://mirrors.kernel.org/fedora/releases/28/'
+        kernel_url = ('https://mirrors.kernel.org/fedora/releases/29/'
                       'Everything/x86_64/os/images/pxeboot/vmlinuz')
-        kernel_hash = '238e083e114c48200f80d889f7e32eeb2793e02a'
+        kernel_hash = '23bebd2680757891cf7adedb033532163a792495'
         kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
 
         self.vm.set_machine('pc')
Signature · 1 line
2.20.1
Signature · 3 lines
Caio Carrara
Software Engineer, Virt Team - Red Hat
[email protected]
#32 lore raw

PATCH 10/18 Boot Linux Console Test: refactor the console watcher into utility method

In reply to Cleber Rosa

This introduces a utility method that monitors the console device and looks for either a message that signals the test success or failure.

---
 tests/acceptance/boot_linux_console.py | 30 ++++++++++++++++++--------
 1 file changed, 21 insertions(+), 9 deletions(-)
tests/acceptance/boot_linux_console.py +21 -9
index 35b31162d4..278bb2be3d 100644
@@ -23,6 +23,25 @@ class BootLinuxConsole(Test):
 
     timeout = 60
 
+    def wait_for_console_pattern(self, success_message,
+                                 failure_message='Kernel panic - not syncing'):
+        """
+        Waits for messages to appear on the console, while logging the content
+
+        :param success_message: if this message appears, test succeeds
+        :param failure_message: if this message appears, test fails
+        """
+        console = self.vm.console_socket.makefile()
+        console_logger = logging.getLogger('console')
+        while True:
+            msg = console.readline()
+            console_logger.debug(msg.strip())
+            if success_message in msg:
+                break
+            if failure_message in msg:
+                fail = 'Failure message found in console: %s' % failure_message
+                self.fail(fail)
+
     def test_x86_64_pc(self):
         """
         :avocado: tags=arch:x86_64
@@ -39,12 +58,5 @@ class BootLinuxConsole(Test):
         self.vm.add_args('-kernel', kernel_path,
                          '-append', kernel_command_line)
         self.vm.launch()
-        console = self.vm.console_socket.makefile()
-        console_logger = logging.getLogger('console')
-        while True:
-            msg = console.readline()
-            console_logger.debug(msg.strip())
-            if 'Kernel command line: %s' % kernel_command_line in msg:
-                break
-            if 'Kernel panic - not syncing' in msg:
-                self.fail("Kernel panic reached")
+        console_pattern = 'Kernel command line: %s' % kernel_command_line
+        self.wait_for_console_pattern(console_pattern)
Signature · 1 line
2.20.1
Reviewed-by on v1 10/18
#33 lore raw

Re: [Qemu-devel] [PATCH 10/18] Boot Linux Console Test: refactor the console watcher into utility method

Quoted 4 lines · Cleber Rosa

This introduces a utility method that monitors the console device and looks for either a message that signals the test success or failure.

Quoted 4 lines
---
 tests/acceptance/boot_linux_console.py | 30 ++++++++++++++++++--------
 1 file changed, 21 insertions(+), 9 deletions(-)

{...}

Quoted 1 line
Signature · 3 lines
Caio Carrara
Software Engineer, Virt Team - Red Hat
[email protected]
Reviewed-by on v1 10/18
#34 lore raw

Re: [Qemu-devel] [PATCH 10/18] Boot Linux Console Test: refactor the console watcher into utility method

In reply to Cleber Rosa

Hi Cleber,

Quoted 25 lines · Cleber Rosa

This introduces a utility method that monitors the console device and looks for either a message that signals the test success or failure.

---
 tests/acceptance/boot_linux_console.py | 30 ++++++++++++++++++--------
 1 file changed, 21 insertions(+), 9 deletions(-)
tests/acceptance/boot_linux_console.py +9 -0
index 35b31162d4..278bb2be3d 100644
@@ -23,6 +23,25 @@ class BootLinuxConsole(Test):
 
     timeout = 60
 
+    def wait_for_console_pattern(self, success_message,
+                                 failure_message='Kernel panic - not syncing'):
+        """
+        Waits for messages to appear on the console, while logging the content
+
+        :param success_message: if this message appears, test succeeds
+        :param failure_message: if this message appears, test fails
+        """
+        console = self.vm.console_socket.makefile()

tests/acceptance/boot_linux_console.py class BootLinuxConsole(Test): · line 44

Quoted 10 lines
tests/acceptance/boot_linux_console.py +10 -0
+        console_logger = logging.getLogger('console')
+        while True:
+            msg = console.readline()
+            console_logger.debug(msg.strip())
+            if success_message in msg:
+                break
+            if failure_message in msg:
+                fail = 'Failure message found in console: %s' % failure_message
+                self.fail(fail)
+

This helper is more generic than the BootLinuxConsole class, can you move it out? I'd like to use it in test_uefi_ovmf_x86_64_pc and test_uefi_armvirtqemu_aarch64_virt.

Anyway this can be a follow-up patch, so regardless:

Quoted 19 lines
tests/acceptance/boot_linux_console.py +2 -9
     def test_x86_64_pc(self):
         """
         :avocado: tags=arch:x86_64
@@ -39,12 +58,5 @@ class BootLinuxConsole(Test):
         self.vm.add_args('-kernel', kernel_path,
                          '-append', kernel_command_line)
         self.vm.launch()
-        console = self.vm.console_socket.makefile()
-        console_logger = logging.getLogger('console')
-        while True:
-            msg = console.readline()
-            console_logger.debug(msg.strip())
-            if 'Kernel command line: %s' % kernel_command_line in msg:
-                break
-            if 'Kernel panic - not syncing' in msg:
-                self.fail("Kernel panic reached")
+        console_pattern = 'Kernel command line: %s' % kernel_command_line
+        self.wait_for_console_pattern(console_pattern)
#35 lore raw

Re: [Qemu-devel] [PATCH 10/18] Boot Linux Console Test: refactor the console watcher into utility method

Quoted 43 lines · Philippe Mathieu-Daudé

Hi Cleber,

Quoted 25 lines · Cleber Rosa

This introduces a utility method that monitors the console device and looks for either a message that signals the test success or failure.

---
 tests/acceptance/boot_linux_console.py | 30 ++++++++++++++++++--------
 1 file changed, 21 insertions(+), 9 deletions(-)
tests/acceptance/boot_linux_console.py +9 -0
index 35b31162d4..278bb2be3d 100644
@@ -23,6 +23,25 @@ class BootLinuxConsole(Test):
 
     timeout = 60
 
+    def wait_for_console_pattern(self, success_message,
+                                 failure_message='Kernel panic - not syncing'):
+        """
+        Waits for messages to appear on the console, while logging the content
+
+        :param success_message: if this message appears, test succeeds
+        :param failure_message: if this message appears, test fails
+        """
+        console = self.vm.console_socket.makefile()

tests/acceptance/boot_linux_console.py class BootLinuxConsole(Test): · line 44

Quoted 10 lines
tests/acceptance/boot_linux_console.py +10 -0
+        console_logger = logging.getLogger('console')
+        while True:
+            msg = console.readline()
+            console_logger.debug(msg.strip())
+            if success_message in msg:
+                break
+            if failure_message in msg:
+                fail = 'Failure message found in console: %s' % failure_message
+                self.fail(fail)
+

This helper is more generic than the BootLinuxConsole class, can you move it out? I'd like to use it in test_uefi_ovmf_x86_64_pc and test_uefi_armvirtqemu_aarch64_virt.

Quoted 3 lines

Anyway this can be a follow-up patch, so regardless:

Sure thing, I'll work on something on top of this. Tracking it here in the mean time:

https://trello.com/c/oJtL1nRJ/78-create-utility-out-of-waitforconsolepattern

- Cleber.

#36 lore raw

Re: [Qemu-devel] [PATCH 10/18] Boot Linux Console Test: refactor the console watcher into utility method

In reply to Cleber Rosa

Hi Cleber,

Quoted 14 lines · Cleber Rosa

This introduces a utility method that monitors the console device and looks for either a message that signals the test success or failure.

---
  tests/acceptance/boot_linux_console.py | 30 ++++++++++++++++++--------
  1 file changed, 21 insertions(+), 9 deletions(-)
tests/acceptance/boot_linux_console.py +0 -0
index 35b31162d4..278bb2be3d 100644
@@ -23,6 +23,25 @@ class BootLinuxConsole(Test):
 

tests/acceptance/boot_linux_console.py class BootLinuxConsole(Test): · line 33

Quoted 10 lines
tests/acceptance/boot_linux_console.py +8 -0
      timeout = 60
 
+    def wait_for_console_pattern(self, success_message,
+                                 failure_message='Kernel panic - not syncing'):
+        """
+        Waits for messages to appear on the console, while logging the content
+
+        :param success_message: if this message appears, test succeeds
+        :param failure_message: if this message appears, test fails
+        """

What's the docstring format used here? I am wondering if :type is optional.

- Wainer

Quoted 29 lines
tests/acceptance/boot_linux_console.py +13 -9
+        console = self.vm.console_socket.makefile()
+        console_logger = logging.getLogger('console')
+        while True:
+            msg = console.readline()
+            console_logger.debug(msg.strip())
+            if success_message in msg:
+                break
+            if failure_message in msg:
+                fail = 'Failure message found in console: %s' % failure_message
+                self.fail(fail)
+
      def test_x86_64_pc(self):
          """
          :avocado: tags=arch:x86_64
@@ -39,12 +58,5 @@ class BootLinuxConsole(Test):
          self.vm.add_args('-kernel', kernel_path,
                           '-append', kernel_command_line)
          self.vm.launch()
-        console = self.vm.console_socket.makefile()
-        console_logger = logging.getLogger('console')
-        while True:
-            msg = console.readline()
-            console_logger.debug(msg.strip())
-            if 'Kernel command line: %s' % kernel_command_line in msg:
-                break
-            if 'Kernel panic - not syncing' in msg:
-                self.fail("Kernel panic reached")
+        console_pattern = 'Kernel command line: %s' % kernel_command_line
+        self.wait_for_console_pattern(console_pattern)
#37 lore raw

Re: [Qemu-devel] [PATCH 10/18] Boot Linux Console Test: refactor the console watcher into utility method

Quoted 31 lines · Wainer dos Santos Moschetta

Hi Cleber,

Quoted 17 lines · Cleber Rosa

This introduces a utility method that monitors the console device and looks for either a message that signals the test success or failure.

---
  tests/acceptance/boot_linux_console.py | 30 ++++++++++++++++++--------

  1 file changed, 21 insertions(+), 9 deletions(-)
tests/acceptance/boot_linux_console.py +0 -0
index 35b31162d4..278bb2be3d 100644
@@ -23,6 +23,25 @@ class BootLinuxConsole(Test):

        timeout = 60
  +    def wait_for_console_pattern(self, success_message, +                                 failure_message='Kernel panic - not

Quoted 8 lines

syncing'):
+        """
+        Waits for messages to appear on the console, while logging the content
+
+        :param success_message: if this message appears, test succeeds +        :param failure_message: if this message appears, test fails +        """

What's the docstring format used here? I am wondering if :type is optional.

Quoted 2 lines

- Wainer

I've been using the same Sphinx-based docstring statements we've been using in Avocado, for the sake of cross reference. So yes, type is optional, and go on the same line, or on a different one with ":type <name of parame>".

But, there may be issues that we don't know as we're not building those API docs. IIRC Eduardo Habkost is going to propose a GSOC project related to building API docs.

- Cleber.

#38 lore raw

PATCH 11/18 scripts/qemu.py: support adding a console with the default serial device

In reply to Cleber Rosa

The set_console() utility function traditionally adds a device either based on the explicitly given device type, or based on the machine type, a known good type of device.

But, for a number of machine types, it may be impossible or inconvenient to add the devices my means of "-device" command line options, and then it may better to just use the "-serial" option and let QEMU itself, based on the machine type, set the device accordingly.

To achieve that, the behavior of set_console() now flags the intention to add a console device on launch(), and if no explicit device type is given, and there's no definition on CONSOLE_DEV_TYPES, the "-serial" is going to be added to the QEMU command line, instead of raising exceptions.

---
 scripts/qemu.py | 28 +++++++++++++++-------------
 1 file changed, 15 insertions(+), 13 deletions(-)
scripts/qemu.py +15 -13
index ec3567d4e2..88e1608b42 100644
@@ -121,6 +121,7 @@ class QEMUMachine(object):
         self._temp_dir = None
         self._launched = False
         self._machine = None
+        self._console_set = False
         self._console_device_type = None
         self._console_address = None
         self._console_socket = None
@@ -240,13 +241,17 @@ class QEMUMachine(object):
                 '-display', 'none', '-vga', 'none']
         if self._machine is not None:
             args.extend(['-machine', self._machine])
-        if self._console_device_type is not None:
+        if self._console_set:
             self._console_address = os.path.join(self._temp_dir,
                                                  self._name + "-console.sock")
             chardev = ('socket,id=console,path=%s,server,nowait' %
                        self._console_address)
-            device = '%s,chardev=console' % self._console_device_type
-            args.extend(['-chardev', chardev, '-device', device])
+            args.extend(['-chardev', chardev])
+            if self._console_device_type is None:
+                args.extend(['-serial', 'chardev:console'])
+            else:
+                device = '%s,chardev=console' % self._console_device_type
+                args.extend(['-device', device])
         return args
 
     def _pre_launch(self):
@@ -479,23 +484,20 @@ class QEMUMachine(object):
         machine launch time, as it depends on the temporary directory
         to be created.
 
-        @param device_type: the device type, such as "isa-serial"
+        @param device_type: the device type, such as "isa-serial".  If
+                            None is given (the default value) a "-serial
+                            chardev:console" command line argument will
+                            be used instead, resorting to the machine's
+                            default device type.
         @raises: QEMUMachineAddDeviceError if the device type is not given
                  and can not be determined.
         """
-        if device_type is None:
-            if self._machine is None:
-                raise QEMUMachineAddDeviceError("Can not add a console device:"
-                                                " QEMU instance without a "
-                                                "defined machine type")
+        self._console_set = True
+        if device_type is None and self._machine is not None:
             for regex, device in CONSOLE_DEV_TYPES.items():
                 if re.match(regex, self._machine):
                     device_type = device
                     break
-            if device_type is None:
-                raise QEMUMachineAddDeviceError("Can not add a console device:"
-                                                " no matching console device "
-                                                "type definition")
         self._console_device_type = device_type
 
     @property
Signature · 1 line
2.20.1
Reviewed-by on v1 11/18
#39 lore raw

Re: [Qemu-devel] [PATCH 11/18] scripts/qemu.py: support adding a console with the default serial device

Quoted 10 lines · Cleber Rosa

The set_console() utility function traditionally adds a device either based on the explicitly given device type, or based on the machine type, a known good type of device.

But, for a number of machine types, it may be impossible or inconvenient to add the devices my means of "-device" command line options, and then it may better to just use the "-serial" option and let QEMU itself, based on the machine type, set the device accordingly.

Quoted 7 lines

To achieve that, the behavior of set_console() now flags the intention to add a console device on launch(), and if no explicit device type is given, and there's no definition on CONSOLE_DEV_TYPES, the "-serial" is going to be added to the QEMU command line, instead of raising exceptions.

Quoted 4 lines
---
 scripts/qemu.py | 28 +++++++++++++++-------------
 1 file changed, 15 insertions(+), 13 deletions(-)

{...}

Quoted 1 line
Signature · 3 lines
Caio Carrara
Software Engineer, Virt Team - Red Hat
[email protected]
#40 lore raw

Re: [Qemu-devel] [PATCH 11/18] scripts/qemu.py: support adding a console with the default serial device

In reply to Cleber Rosa

Hi Cleber,

me again. :)

Quoted 55 lines · Cleber Rosa

The set_console() utility function traditionally adds a device either based on the explicitly given device type, or based on the machine type, a known good type of device.

But, for a number of machine types, it may be impossible or inconvenient to add the devices my means of "-device" command line options, and then it may better to just use the "-serial" option and let QEMU itself, based on the machine type, set the device accordingly.

To achieve that, the behavior of set_console() now flags the intention to add a console device on launch(), and if no explicit device type is given, and there's no definition on CONSOLE_DEV_TYPES, the "-serial" is going to be added to the QEMU command line, instead of raising exceptions.

---
  scripts/qemu.py | 28 +++++++++++++++-------------
  1 file changed, 15 insertions(+), 13 deletions(-)
scripts/qemu.py +8 -3
index ec3567d4e2..88e1608b42 100644
@@ -121,6 +121,7 @@ class QEMUMachine(object):
          self._temp_dir = None
          self._launched = False
          self._machine = None
+        self._console_set = False
          self._console_device_type = None
          self._console_address = None
          self._console_socket = None
@@ -240,13 +241,17 @@ class QEMUMachine(object):
                  '-display', 'none', '-vga', 'none']
          if self._machine is not None:
              args.extend(['-machine', self._machine])
-        if self._console_device_type is not None:
+        if self._console_set:
              self._console_address = os.path.join(self._temp_dir,
                                                   self._name + "-console.sock")
              chardev = ('socket,id=console,path=%s,server,nowait' %
                         self._console_address)
-            device = '%s,chardev=console' % self._console_device_type
-            args.extend(['-chardev', chardev, '-device', device])
+            args.extend(['-chardev', chardev])
+            if self._console_device_type is None:
+                args.extend(['-serial', 'chardev:console'])
+            else:
+                device = '%s,chardev=console' % self._console_device_type
+                args.extend(['-device', device])
          return args
 
      def _pre_launch(self):

scripts/qemu.py class QEMUMachine(object): · line 491

Quoted 9 lines
scripts/qemu.py +5 -1
@@ -479,23 +484,20 @@ class QEMUMachine(object):
          machine launch time, as it depends on the temporary directory
          to be created.
 
-        @param device_type: the device type, such as "isa-serial"
+        @param device_type: the device type, such as "isa-serial".  If
+                            None is given (the default value) a "-serial
+                            chardev:console" command line argument will
+                            be used instead, resorting to the machine's
+                            default device type.

Shouldn't you mention it will look for device type on CONSOLE_DEV_TYPES if device_type is None and machine is not None?

The description on set_console()'s docstring is out-of-sync with current implementation too.

- Wainer

Quoted 21 lines
scripts/qemu.py +2 -9
          @raises: QEMUMachineAddDeviceError if the device type is not given
                   and can not be determined.
          """
-        if device_type is None:
-            if self._machine is None:
-                raise QEMUMachineAddDeviceError("Can not add a console device:"
-                                                " QEMU instance without a "
-                                                "defined machine type")
+        self._console_set = True
+        if device_type is None and self._machine is not None:
              for regex, device in CONSOLE_DEV_TYPES.items():
                  if re.match(regex, self._machine):
                      device_type = device
                      break
-            if device_type is None:
-                raise QEMUMachineAddDeviceError("Can not add a console device:"
-                                                " no matching console device "
-                                                "type definition")
          self._console_device_type = device_type
 
      @property
#41 lore raw

Re: [Qemu-devel] [PATCH 11/18] scripts/qemu.py: support adding a console with the default serial device

Quoted 64 lines · Wainer dos Santos Moschetta

Hi Cleber,

me again. :)

Quoted 59 lines · Cleber Rosa

The set_console() utility function traditionally adds a device either based on the explicitly given device type, or based on the machine type, a known good type of device.

But, for a number of machine types, it may be impossible or inconvenient to add the devices my means of "-device" command line options, and then it may better to just use the "-serial" option and let QEMU itself, based on the machine type, set the device accordingly.

To achieve that, the behavior of set_console() now flags the intention to add a console device on launch(), and if no explicit device type is given, and there's no definition on CONSOLE_DEV_TYPES, the "-serial" is going to be added to the QEMU command line, instead of raising exceptions.

---
  scripts/qemu.py | 28 +++++++++++++++-------------

  1 file changed, 15 insertions(+), 13 deletions(-)
scripts/qemu.py +0 -0
index ec3567d4e2..88e1608b42 100644
@@ -121,6 +121,7 @@ class QEMUMachine(object):

          self._temp_dir = None
          self._launched = False
          self._machine = None
+        self._console_set = False
          self._console_device_type = None
          self._console_address = None
          self._console_socket = None

scripts/qemu.py +0 -0
@@ -240,13 +241,17 @@ class QEMUMachine(object):

                  '-display', 'none', '-vga', 'none']
          if self._machine is not None:
              args.extend(['-machine', self._machine])
-        if self._console_device_type is not None:
+        if self._console_set:
              self._console_address = os.path.join(self._temp_dir, self._name + "-console.sock")
              chardev = ('socket,id=console,path=%s,server,nowait' % self._console_address)
-            device = '%s,chardev=console' % self._console_device_type -            args.extend(['-chardev', chardev, '-device', device]) +            args.extend(['-chardev', chardev])
+            if self._console_device_type is None:
+                args.extend(['-serial', 'chardev:console']) +            else:
+                device = '%s,chardev=console' % self._console_device_type
+                args.extend(['-device', device])
          return args
        def _pre_launch(self):

scripts/qemu.py +0 -0
@@ -479,23 +484,20 @@ class QEMUMachine(object):

          machine launch time, as it depends on the temporary directory to be created.
  -        @param device_type: the device type, such as "isa-serial"

Quoted 9 lines
Quoted 5 lines

+        @param device_type: the device type, such as "isa-serial".  If +                            None is given (the default value) a "-serial +                            chardev:console" command line argument will +                            be used instead, resorting to the machine's +                            default device type.

Shouldn't you mention it will look for device type on CONSOLE_DEV_TYPES if device_type is None and machine is not None?

Yes, you're right. How about this:

...
@param device_type: the device type, such as "isa-serial". If
None is given (the default value) a "-serial chardev:console" command line argument will be used instead, resorting to the machine's default device type, if a machine type is set, and a matching entry exists on
CONSOLE_DEV_TYPES.

...

Quoted 3 lines

The description on set_console()'s docstring is out-of-sync with current implementation too.

Good catch! I'm rewriting that as:

...

This is a convenience method that will either use the provided device type, of if not given, it will use the device type set on CONSOLE_DEV_TYPES if a machine type is set, and a matching entry exists on CONSOLE_DEV_TYPES.

...

How does it sound?

Thanks!
- Cleber.

Quoted 29 lines

- Wainer

Quoted 25 lines

          @raises: QEMUMachineAddDeviceError if the device type is not given
                   and can not be determined.
          """
-        if device_type is None:
-            if self._machine is None:
-                raise QEMUMachineAddDeviceError("Can not add a console device:"
-                                                " QEMU instance without a "
-                                                "defined machine type") +        self._console_set = True
+        if device_type is None and self._machine is not None:
              for regex, device in CONSOLE_DEV_TYPES.items():
                  if re.match(regex, self._machine):
                      device_type = device
                      break
-            if device_type is None:
-                raise QEMUMachineAddDeviceError("Can not add a console device:"
-                                                " no matching console device "
-                                                "type definition") self._console_device_type = device_type
        @property

#42 lore raw

PATCH 12/18 Boot Linux Console Test: add a test for mips + malta

In reply to Cleber Rosa

From: Philippe Mathieu-Daudé <[email protected]>

Similar to the x86_64 + pc test, it boots a Linux kernel on a Malta board and verify the serial is working. Also, it relies on the serial device set by the machine itself.

If mips is a target being built, "make check-acceptance" will automatically include this test by the use of the "arch:mips" tags.

Alternatively, this test can be run using:

    $ avocado run -t arch:mips tests/acceptance
    $ avocado run -t machine:malta tests/acceptance
    $ avocado run -t endian:big tests/acceptance
---
 .travis.yml                            |  2 +-
 scripts/qemu.py                        |  1 -
 tests/acceptance/boot_linux_console.py | 20 ++++++++++++++++++++
 3 files changed, 21 insertions(+), 2 deletions(-)
.travis.yml +1 -1
index 844d514afa..49f9016e6a 100644
@@ -187,7 +187,7 @@ matrix:
 
     # Acceptance (Functional) tests
     - env:
-        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu"
+        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu"
         - TEST_CMD="make check-acceptance"
       addons:
         apt:
scripts/qemu.py +0 -1
index 88e1608b42..ef84b0f843 100644
@@ -35,7 +35,6 @@ def kvm_available(target_arch=None):
 #: Maps machine types to the preferred console device types
 CONSOLE_DEV_TYPES = {
     r'^clipper$': 'isa-serial',
-    r'^malta': 'isa-serial',
     r'^(pc.*|q35.*|isapc)$': 'isa-serial',
     r'^(40p|powernv|prep)$': 'isa-serial',
     r'^pseries.*': 'spapr-vty',
tests/acceptance/boot_linux_console.py +20 -0
index 278bb2be3d..0678ec91d2 100644
@@ -60,3 +60,23 @@ class BootLinuxConsole(Test):
         self.vm.launch()
         console_pattern = 'Kernel command line: %s' % kernel_command_line
         self.wait_for_console_pattern(console_pattern)
+
+    def test_mips_malta(self):
+        """
+        :avocado: tags=arch:mips
+        :avocado: tags=machine:malta
+        :avocado: tags=endian:big
+        """
+        kernel_url = ('http://people.debian.org/~aurel32/qemu/mips/'
+                      'vmlinux-3.2.0-4-4kc-malta')
+        kernel_hash = '592e384a4edc16dade52a6cd5c785c637bcbc9ad'
+        kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
+
+        self.vm.set_machine('malta')
+        self.vm.set_console()
+        kernel_command_line = 'console=ttyS0 printk.time=0'
+        self.vm.add_args('-kernel', kernel_path,
+                         '-append', kernel_command_line)
+        self.vm.launch()
+        console_pattern = 'Kernel command line: %s' % kernel_command_line
+        self.wait_for_console_pattern(console_pattern)
Signature · 1 line
2.20.1
Reviewed-by on v1 12/18
#43 lore raw

Re: [Qemu-devel] [PATCH 12/18] Boot Linux Console Test: add a test for mips + malta

Quoted 9 lines · Cleber Rosa

From: Philippe Mathieu-Daudé <[email protected]>

Similar to the x86_64 + pc test, it boots a Linux kernel on a Malta board and verify the serial is working. Also, it relies on the serial device set by the machine itself.

If mips is a target being built, "make check-acceptance" will automatically include this test by the use of the "arch:mips" tags.

Quoted 8 lines

Alternatively, this test can be run using:

    $ avocado run -t arch:mips tests/acceptance
    $ avocado run -t machine:malta tests/acceptance
    $ avocado run -t endian:big tests/acceptance
Quoted 6 lines
---
 .travis.yml                            |  2 +-
 scripts/qemu.py                        |  1 -
 tests/acceptance/boot_linux_console.py | 20 ++++++++++++++++++++
 3 files changed, 21 insertions(+), 2 deletions(-)

{...}

Quoted 1 line
Signature · 3 lines
Caio Carrara
Software Engineer, Virt Team - Red Hat
[email protected]
#44 lore raw

Re: [Qemu-devel] [PATCH 12/18] Boot Linux Console Test: add a test for mips + malta

In reply to Cleber Rosa

Hi Cleber,

Quoted 57 lines · Cleber Rosa

From: Philippe Mathieu-Daudé <[email protected]>

Similar to the x86_64 + pc test, it boots a Linux kernel on a Malta board and verify the serial is working. Also, it relies on the serial device set by the machine itself.

If mips is a target being built, "make check-acceptance" will automatically include this test by the use of the "arch:mips" tags.

Alternatively, this test can be run using:

    $ avocado run -t arch:mips tests/acceptance
    $ avocado run -t machine:malta tests/acceptance
    $ avocado run -t endian:big tests/acceptance
---
 .travis.yml                            |  2 +-
 scripts/qemu.py                        |  1 -
 tests/acceptance/boot_linux_console.py | 20 ++++++++++++++++++++
 3 files changed, 21 insertions(+), 2 deletions(-)
.travis.yml +1 -1
index 844d514afa..49f9016e6a 100644
@@ -187,7 +187,7 @@ matrix:
 
     # Acceptance (Functional) tests
     - env:
-        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu"
+        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu"
         - TEST_CMD="make check-acceptance"
       addons:
         apt:
scripts/qemu.py +0 -1
index 88e1608b42..ef84b0f843 100644
@@ -35,7 +35,6 @@ def kvm_available(target_arch=None):
 #: Maps machine types to the preferred console device types
 CONSOLE_DEV_TYPES = {
     r'^clipper$': 'isa-serial',
-    r'^malta': 'isa-serial',
     r'^(pc.*|q35.*|isapc)$': 'isa-serial',
     r'^(40p|powernv|prep)$': 'isa-serial',
     r'^pseries.*': 'spapr-vty',
tests/acceptance/boot_linux_console.py +1 -0
index 278bb2be3d..0678ec91d2 100644
@@ -60,3 +60,23 @@ class BootLinuxConsole(Test):
         self.vm.launch()
         console_pattern = 'Kernel command line: %s' % kernel_command_line
         self.wait_for_console_pattern(console_pattern)
+

tests/acceptance/boot_linux_console.py class BootLinuxConsole(Test): · line 73

Quoted 10 lines
tests/acceptance/boot_linux_console.py +10 -0
+    def test_mips_malta(self):
+        """
+        :avocado: tags=arch:mips
+        :avocado: tags=machine:malta
+        :avocado: tags=endian:big
+        """
+        kernel_url = ('http://people.debian.org/~aurel32/qemu/mips/'
+                      'vmlinux-3.2.0-4-4kc-malta')
+        kernel_hash = '592e384a4edc16dade52a6cd5c785c637bcbc9ad'
+        kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)

Alex was concerned about using this "not-so-easily-reproducible" binary:
https://lists.gnu.org/archive/html/qemu-devel/2018-06/msg08441.html

So I sent another patch using a reproducible one, which you reviewed:
https://lists.gnu.org/archive/html/qemu-devel/2018-10/msg02747.html

Do you mind base this series on it?

Thanks,

Phil.

Quoted 10 lines
tests/acceptance/boot_linux_console.py +9 -0
+
+        self.vm.set_machine('malta')
+        self.vm.set_console()
+        kernel_command_line = 'console=ttyS0 printk.time=0'
+        self.vm.add_args('-kernel', kernel_path,
+                         '-append', kernel_command_line)
+        self.vm.launch()
+        console_pattern = 'Kernel command line: %s' % kernel_command_line
+        self.wait_for_console_pattern(console_pattern)
#45 lore raw

Re: [Qemu-devel] [PATCH 12/18] Boot Linux Console Test: add a test for mips + malta

Quoted 65 lines · Philippe Mathieu-Daudé

Hi Cleber,

Quoted 62 lines · Cleber Rosa

From: Philippe Mathieu-Daudé <[email protected]>

Similar to the x86_64 + pc test, it boots a Linux kernel on a Malta board and verify the serial is working. Also, it relies on the serial device set by the machine itself.

If mips is a target being built, "make check-acceptance" will automatically include this test by the use of the "arch:mips" tags.

Alternatively, this test can be run using:

    $ avocado run -t arch:mips tests/acceptance
    $ avocado run -t machine:malta tests/acceptance
    $ avocado run -t endian:big tests/acceptance
---
 .travis.yml                            |  2 +-
 scripts/qemu.py                        |  1 -
 tests/acceptance/boot_linux_console.py | 20 ++++++++++++++++++++
 3 files changed, 21 insertions(+), 2 deletions(-)
.travis.yml +1 -1
index 844d514afa..49f9016e6a 100644
@@ -187,7 +187,7 @@ matrix:
 
     # Acceptance (Functional) tests
     - env:
-        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu"
+        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu"
         - TEST_CMD="make check-acceptance"
       addons:
         apt:
scripts/qemu.py +0 -1
index 88e1608b42..ef84b0f843 100644
@@ -35,7 +35,6 @@ def kvm_available(target_arch=None):
 #: Maps machine types to the preferred console device types
 CONSOLE_DEV_TYPES = {
     r'^clipper$': 'isa-serial',
-    r'^malta': 'isa-serial',
     r'^(pc.*|q35.*|isapc)$': 'isa-serial',
     r'^(40p|powernv|prep)$': 'isa-serial',
     r'^pseries.*': 'spapr-vty',
tests/acceptance/boot_linux_console.py +6 -0
index 278bb2be3d..0678ec91d2 100644
@@ -60,3 +60,23 @@ class BootLinuxConsole(Test):
         self.vm.launch()
         console_pattern = 'Kernel command line: %s' % kernel_command_line
         self.wait_for_console_pattern(console_pattern)
+
+    def test_mips_malta(self):
+        """
+        :avocado: tags=arch:mips
+        :avocado: tags=machine:malta
+        :avocado: tags=endian:big
Quoted 9 lines

tests/acceptance/boot_linux_console.py class BootLinuxConsole(Test): · line 73

Quoted 5 lines
tests/acceptance/boot_linux_console.py +5 -0
+        """
+        kernel_url = ('http://people.debian.org/~aurel32/qemu/mips/'
+                      'vmlinux-3.2.0-4-4kc-malta')
+        kernel_hash = '592e384a4edc16dade52a6cd5c785c637bcbc9ad'
+        kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)

Alex was concerned about using this "not-so-easily-reproducible" binary:
https://lists.gnu.org/archive/html/qemu-devel/2018-06/msg08441.html

I clearly missed that.

Quoted 3 lines

So I sent another patch using a reproducible one, which you reviewed:
https://lists.gnu.org/archive/html/qemu-devel/2018-10/msg02747.html

And I clearly forgot about that :)

Quoted 5 lines

Do you mind base this series on it?

Thanks,

Phil.

Of course! Thanks for pointing that out.

- Cleber.

#46 lore raw

PATCH 13/18 Boot Linux Console Test: add a test for mips64el + malta

In reply to Cleber Rosa

Similar to the x86_64 + pc test, it boots a Linux kernel on a Malta board and verify the serial is working.

If mips64el is a target being built, "make check-acceptance" will automatically include this test by the use of the "arch:mips" tags.

Alternatively, this test can be run using:

    $ avocado run -t arch:mips64el tests/acceptance
    $ avocado run -t machine:malta tests/acceptance
---
 .travis.yml                            |  2 +-
 tests/acceptance/boot_linux_console.py | 39 ++++++++++++++++++++++++++
 2 files changed, 40 insertions(+), 1 deletion(-)
.travis.yml +1 -1
index 49f9016e6a..28648f7a61 100644
@@ -187,7 +187,7 @@ matrix:
 
     # Acceptance (Functional) tests
     - env:
-        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu"
+        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu"
         - TEST_CMD="make check-acceptance"
       addons:
         apt:
tests/acceptance/boot_linux_console.py +39 -0
index 0678ec91d2..20b845fce1 100644
@@ -9,8 +9,11 @@
 # later.  See the COPYING file in the top-level directory.
 
 import logging
+import os
 
 from avocado_qemu import Test
+from avocado.utils import process
+from avocado.utils import archive
 
 
 class BootLinuxConsole(Test):
@@ -80,3 +83,39 @@ class BootLinuxConsole(Test):
         self.vm.launch()
         console_pattern = 'Kernel command line: %s' % kernel_command_line
         self.wait_for_console_pattern(console_pattern)
+
+    def test_mips64el_malta(self):
+        """
+        This test requires the ar tool to extract "data.tar.gz" from
+        the Debian package.
+
+        The kernel can be rebuilt using this Debian kernel source [1] and
+        following the instructions on [2].
+
+        [1] https://kernel-team.pages.debian.net/kernel-handbook/ch-common-tasks.html#s-common-official
+        [2] http://snapshot.debian.org/package/linux-2.6/2.6.32-48/#linux-source-2.6.32_2.6.32-48
+
+        :avocado: tags=arch:mips64el
+        :avocado: tags=machine:malta
+        """
+        deb_url = ('http://snapshot.debian.org/archive/debian/20130217T032700Z/'
+                   'pool/main/l/linux-2.6/'
+                   'linux-image-2.6.32-5-5kc-malta_2.6.32-48_mipsel.deb')
+        deb_hash = '1aaec92083bf22fda31e0d27fa8d9a388e5fc3d5'
+        deb_path = self.fetch_asset(deb_url, asset_hash=deb_hash)
+
+        cwd = os.getcwd()
+        os.chdir(self.workdir)
+        process.run("ar x %s data.tar.gz" % deb_path)
+        archive.extract("data.tar.gz", self.workdir)
+        os.chdir(cwd)
+        kernel_path = self.workdir + '/boot/vmlinux-2.6.32-5-5kc-malta'
+
+        self.vm.set_machine('malta')
+        self.vm.set_console()
+        kernel_command_line = 'console=ttyS0 printk.time=0'
+        self.vm.add_args('-kernel', kernel_path,
+                         '-append', kernel_command_line)
+        self.vm.launch()
+        console_pattern = 'Kernel command line: %s' % kernel_command_line
+        self.wait_for_console_pattern(console_pattern)
Signature · 1 line
2.20.1
Reviewed-by on v1 13/18
#47 lore raw

Re: [Qemu-devel] [PATCH 13/18] Boot Linux Console Test: add a test for mips64el + malta

Quoted 6 lines · Cleber Rosa

Similar to the x86_64 + pc test, it boots a Linux kernel on a Malta board and verify the serial is working.

If mips64el is a target being built, "make check-acceptance" will automatically include this test by the use of the "arch:mips" tags.

Quoted 7 lines

Alternatively, this test can be run using:

    $ avocado run -t arch:mips64el tests/acceptance
    $ avocado run -t machine:malta tests/acceptance
Quoted 77 lines
---
 .travis.yml                            |  2 +-
 tests/acceptance/boot_linux_console.py | 39 ++++++++++++++++++++++++++
 2 files changed, 40 insertions(+), 1 deletion(-)
.travis.yml +1 -1
index 49f9016e6a..28648f7a61 100644
@@ -187,7 +187,7 @@ matrix:
 
     # Acceptance (Functional) tests
     - env:
-        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu"
+        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu"
         - TEST_CMD="make check-acceptance"
       addons:
         apt:
tests/acceptance/boot_linux_console.py +39 -0
index 0678ec91d2..20b845fce1 100644
@@ -9,8 +9,11 @@
 # later.  See the COPYING file in the top-level directory.
 
 import logging
+import os
 
 from avocado_qemu import Test
+from avocado.utils import process
+from avocado.utils import archive
 
 
 class BootLinuxConsole(Test):
@@ -80,3 +83,39 @@ class BootLinuxConsole(Test):
         self.vm.launch()
         console_pattern = 'Kernel command line: %s' % kernel_command_line
         self.wait_for_console_pattern(console_pattern)
+
+    def test_mips64el_malta(self):
+        """
+        This test requires the ar tool to extract "data.tar.gz" from
+        the Debian package.
+
+        The kernel can be rebuilt using this Debian kernel source [1] and
+        following the instructions on [2].
+
+        [1] https://kernel-team.pages.debian.net/kernel-handbook/ch-common-tasks.html#s-common-official
+        [2] http://snapshot.debian.org/package/linux-2.6/2.6.32-48/#linux-source-2.6.32_2.6.32-48
+
+        :avocado: tags=arch:mips64el
+        :avocado: tags=machine:malta
+        """
+        deb_url = ('http://snapshot.debian.org/archive/debian/20130217T032700Z/'
+                   'pool/main/l/linux-2.6/'
+                   'linux-image-2.6.32-5-5kc-malta_2.6.32-48_mipsel.deb')
+        deb_hash = '1aaec92083bf22fda31e0d27fa8d9a388e5fc3d5'
+        deb_path = self.fetch_asset(deb_url, asset_hash=deb_hash)
+
+        cwd = os.getcwd()
+        os.chdir(self.workdir)
+        process.run("ar x %s data.tar.gz" % deb_path)
+        archive.extract("data.tar.gz", self.workdir)
+        os.chdir(cwd)
+        kernel_path = self.workdir + '/boot/vmlinux-2.6.32-5-5kc-malta'
+
+        self.vm.set_machine('malta')
+        self.vm.set_console()
+        kernel_command_line = 'console=ttyS0 printk.time=0'
+        self.vm.add_args('-kernel', kernel_path,
+                         '-append', kernel_command_line)
+        self.vm.launch()
+        console_pattern = 'Kernel command line: %s' % kernel_command_line
+        self.wait_for_console_pattern(console_pattern)
Signature · 1 line
2.20.1
Signature · 3 lines
Caio Carrara
Software Engineer, Virt Team - Red Hat
[email protected]
Reviewed-by on v1 13/18
#48 lore raw

Re: [Qemu-devel] [PATCH 13/18] Boot Linux Console Test: add a test for mips64el + malta

In reply to Cleber Rosa

Hi Cleber,

Quoted 62 lines · Cleber Rosa

Similar to the x86_64 + pc test, it boots a Linux kernel on a Malta board and verify the serial is working.

If mips64el is a target being built, "make check-acceptance" will automatically include this test by the use of the "arch:mips" tags.

Alternatively, this test can be run using:

    $ avocado run -t arch:mips64el tests/acceptance
    $ avocado run -t machine:malta tests/acceptance
---
 .travis.yml                            |  2 +-
 tests/acceptance/boot_linux_console.py | 39 ++++++++++++++++++++++++++
 2 files changed, 40 insertions(+), 1 deletion(-)
.travis.yml +1 -1
index 49f9016e6a..28648f7a61 100644
@@ -187,7 +187,7 @@ matrix:
 
     # Acceptance (Functional) tests
     - env:
-        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu"
+        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu"
         - TEST_CMD="make check-acceptance"
       addons:
         apt:
tests/acceptance/boot_linux_console.py +14 -0
index 0678ec91d2..20b845fce1 100644
@@ -9,8 +9,11 @@
 # later.  See the COPYING file in the top-level directory.
 
 import logging
+import os
 
 from avocado_qemu import Test
+from avocado.utils import process
+from avocado.utils import archive
 
 
 class BootLinuxConsole(Test):
@@ -80,3 +83,39 @@ class BootLinuxConsole(Test):
         self.vm.launch()
         console_pattern = 'Kernel command line: %s' % kernel_command_line
         self.wait_for_console_pattern(console_pattern)
+
+    def test_mips64el_malta(self):
+        """
+        This test requires the ar tool to extract "data.tar.gz" from
+        the Debian package.
+
+        The kernel can be rebuilt using this Debian kernel source [1] and
+        following the instructions on [2].
+
+        [1] https://kernel-team.pages.debian.net/kernel-handbook/ch-common-tasks.html#s-common-official
+        [2] http://snapshot.debian.org/package/linux-2.6/2.6.32-48/#linux-source-2.6.32_2.6.32-48

tests/acceptance/boot_linux_console.py class BootLinuxConsole(Test): · line 106

Quoted 10 lines
tests/acceptance/boot_linux_console.py +10 -0
+
+        :avocado: tags=arch:mips64el
+        :avocado: tags=machine:malta
+        """
+        deb_url = ('http://snapshot.debian.org/archive/debian/20130217T032700Z/'
+                   'pool/main/l/linux-2.6/'
+                   'linux-image-2.6.32-5-5kc-malta_2.6.32-48_mipsel.deb')
+        deb_hash = '1aaec92083bf22fda31e0d27fa8d9a388e5fc3d5'
+        deb_path = self.fetch_asset(deb_url, asset_hash=deb_hash)
+

Can you refactor this code ...

tests/acceptance/boot_linux_console.py class BootLinuxConsole(Test): · line 112

Quoted 6 lines
tests/acceptance/boot_linux_console.py +6 -0
+        cwd = os.getcwd()
+        os.chdir(self.workdir)
+        process.run("ar x %s data.tar.gz" % deb_path)
+        archive.extract("data.tar.gz", self.workdir)
+        os.chdir(cwd)
+        kernel_path = self.workdir + '/boot/vmlinux-2.6.32-5-5kc-malta'

... as an helper function?

Can be follow-up patch, so:

Quoted 10 lines
tests/acceptance/boot_linux_console.py +9 -0
+
+        self.vm.set_machine('malta')
+        self.vm.set_console()
+        kernel_command_line = 'console=ttyS0 printk.time=0'
+        self.vm.add_args('-kernel', kernel_path,
+                         '-append', kernel_command_line)
+        self.vm.launch()
+        console_pattern = 'Kernel command line: %s' % kernel_command_line
+        self.wait_for_console_pattern(console_pattern)
#49 lore raw

Re: [Qemu-devel] [PATCH 13/18] Boot Linux Console Test: add a test for mips64el + malta

Quoted 78 lines · Philippe Mathieu-Daudé

Hi Cleber,

Quoted 62 lines · Cleber Rosa

Similar to the x86_64 + pc test, it boots a Linux kernel on a Malta board and verify the serial is working.

If mips64el is a target being built, "make check-acceptance" will automatically include this test by the use of the "arch:mips" tags.

Alternatively, this test can be run using:

    $ avocado run -t arch:mips64el tests/acceptance
    $ avocado run -t machine:malta tests/acceptance
---
 .travis.yml                            |  2 +-
 tests/acceptance/boot_linux_console.py | 39 ++++++++++++++++++++++++++
 2 files changed, 40 insertions(+), 1 deletion(-)
.travis.yml +1 -1
index 49f9016e6a..28648f7a61 100644
@@ -187,7 +187,7 @@ matrix:
 
     # Acceptance (Functional) tests
     - env:
-        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu"
+        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu"
         - TEST_CMD="make check-acceptance"
       addons:
         apt:
tests/acceptance/boot_linux_console.py +14 -0
index 0678ec91d2..20b845fce1 100644
@@ -9,8 +9,11 @@
 # later.  See the COPYING file in the top-level directory.
 
 import logging
+import os
 
 from avocado_qemu import Test
+from avocado.utils import process
+from avocado.utils import archive
 
 
 class BootLinuxConsole(Test):
@@ -80,3 +83,39 @@ class BootLinuxConsole(Test):
         self.vm.launch()
         console_pattern = 'Kernel command line: %s' % kernel_command_line
         self.wait_for_console_pattern(console_pattern)
+
+    def test_mips64el_malta(self):
+        """
+        This test requires the ar tool to extract "data.tar.gz" from
+        the Debian package.
+
+        The kernel can be rebuilt using this Debian kernel source [1] and
+        following the instructions on [2].
+
+        [1] https://kernel-team.pages.debian.net/kernel-handbook/ch-common-tasks.html#s-common-official
+        [2] http://snapshot.debian.org/package/linux-2.6/2.6.32-48/#linux-source-2.6.32_2.6.32-48

tests/acceptance/boot_linux_console.py class BootLinuxConsole(Test): · line 106

Quoted 10 lines
tests/acceptance/boot_linux_console.py +10 -0
+
+        :avocado: tags=arch:mips64el
+        :avocado: tags=machine:malta
+        """
+        deb_url = ('http://snapshot.debian.org/archive/debian/20130217T032700Z/'
+                   'pool/main/l/linux-2.6/'
+                   'linux-image-2.6.32-5-5kc-malta_2.6.32-48_mipsel.deb')
+        deb_hash = '1aaec92083bf22fda31e0d27fa8d9a388e5fc3d5'
+        deb_path = self.fetch_asset(deb_url, asset_hash=deb_hash)
+

Can you refactor this code ...

Quoted 9 lines

tests/acceptance/boot_linux_console.py class BootLinuxConsole(Test): · line 112

Quoted 6 lines
tests/acceptance/boot_linux_console.py +6 -0
+        cwd = os.getcwd()
+        os.chdir(self.workdir)
+        process.run("ar x %s data.tar.gz" % deb_path)
+        archive.extract("data.tar.gz", self.workdir)
+        os.chdir(cwd)
+        kernel_path = self.workdir + '/boot/vmlinux-2.6.32-5-5kc-malta'

... as an helper function?

Yes. In fact, because of the kernel change in the previous patch, I've added it there.

Quoted 3 lines

Can be follow-up patch, so:

In the longer term, I expect to add deb (and RPM, etc) support to the avocado.utils.archive module:

https://trello.com/c/bQBnvkjv/1437-avocadoutilsarchive-support-rpm-deb-etc

Thanks!
- Cleber.

#50 lore raw

Re: [Qemu-devel] [PATCH 13/18] Boot Linux Console Test: add a test for mips64el + malta

Quoted 92 lines · Cleber Rosa
Quoted 78 lines · Philippe Mathieu-Daudé

Hi Cleber,

Quoted 62 lines · Cleber Rosa

Similar to the x86_64 + pc test, it boots a Linux kernel on a Malta board and verify the serial is working.

If mips64el is a target being built, "make check-acceptance" will automatically include this test by the use of the "arch:mips" tags.

Alternatively, this test can be run using:

    $ avocado run -t arch:mips64el tests/acceptance
    $ avocado run -t machine:malta tests/acceptance
---
 .travis.yml                            |  2 +-
 tests/acceptance/boot_linux_console.py | 39 ++++++++++++++++++++++++++
 2 files changed, 40 insertions(+), 1 deletion(-)
.travis.yml +1 -1
index 49f9016e6a..28648f7a61 100644
@@ -187,7 +187,7 @@ matrix:
 
     # Acceptance (Functional) tests
     - env:
-        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu"
+        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu"
         - TEST_CMD="make check-acceptance"
       addons:
         apt:
tests/acceptance/boot_linux_console.py +14 -0
index 0678ec91d2..20b845fce1 100644
@@ -9,8 +9,11 @@
 # later.  See the COPYING file in the top-level directory.
 
 import logging
+import os
 
 from avocado_qemu import Test
+from avocado.utils import process
+from avocado.utils import archive
 
 
 class BootLinuxConsole(Test):
@@ -80,3 +83,39 @@ class BootLinuxConsole(Test):
         self.vm.launch()
         console_pattern = 'Kernel command line: %s' % kernel_command_line
         self.wait_for_console_pattern(console_pattern)
+
+    def test_mips64el_malta(self):
+        """
+        This test requires the ar tool to extract "data.tar.gz" from
+        the Debian package.
+
+        The kernel can be rebuilt using this Debian kernel source [1] and
+        following the instructions on [2].
+
+        [1] https://kernel-team.pages.debian.net/kernel-handbook/ch-common-tasks.html#s-common-official
+        [2] http://snapshot.debian.org/package/linux-2.6/2.6.32-48/#linux-source-2.6.32_2.6.32-48

tests/acceptance/boot_linux_console.py class BootLinuxConsole(Test): · line 106

Quoted 10 lines
tests/acceptance/boot_linux_console.py +10 -0
+
+        :avocado: tags=arch:mips64el
+        :avocado: tags=machine:malta
+        """
+        deb_url = ('http://snapshot.debian.org/archive/debian/20130217T032700Z/'
+                   'pool/main/l/linux-2.6/'
+                   'linux-image-2.6.32-5-5kc-malta_2.6.32-48_mipsel.deb')
+        deb_hash = '1aaec92083bf22fda31e0d27fa8d9a388e5fc3d5'
+        deb_path = self.fetch_asset(deb_url, asset_hash=deb_hash)
+

Can you refactor this code ...

Quoted 9 lines

tests/acceptance/boot_linux_console.py class BootLinuxConsole(Test): · line 112

Quoted 6 lines
tests/acceptance/boot_linux_console.py +6 -0
+        cwd = os.getcwd()
+        os.chdir(self.workdir)
+        process.run("ar x %s data.tar.gz" % deb_path)
+        archive.extract("data.tar.gz", self.workdir)
+        os.chdir(cwd)
+        kernel_path = self.workdir + '/boot/vmlinux-2.6.32-5-5kc-malta'

... as an helper function?

Yes. In fact, because of the kernel change in the previous patch, I've added it there.

Quoted 8 lines
Quoted 3 lines

Can be follow-up patch, so:

In the longer term, I expect to add deb (and RPM, etc) support to the avocado.utils.archive module:

https://trello.com/c/bQBnvkjv/1437-avocadoutilsarchive-support-rpm-deb-etc

Awesome, thanks!

#51 lore raw

Re: [Qemu-devel] [PATCH 13/18] Boot Linux Console Test: add a test for mips64el + malta

In reply to Cleber Rosa
Quoted 72 lines · Cleber Rosa

Similar to the x86_64 + pc test, it boots a Linux kernel on a Malta board and verify the serial is working.

If mips64el is a target being built, "make check-acceptance" will automatically include this test by the use of the "arch:mips" tags.

Alternatively, this test can be run using:

    $ avocado run -t arch:mips64el tests/acceptance
    $ avocado run -t machine:malta tests/acceptance
---
 .travis.yml                            |  2 +-
 tests/acceptance/boot_linux_console.py | 39 ++++++++++++++++++++++++++
 2 files changed, 40 insertions(+), 1 deletion(-)
.travis.yml +1 -1
index 49f9016e6a..28648f7a61 100644
@@ -187,7 +187,7 @@ matrix:
 
     # Acceptance (Functional) tests
     - env:
-        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu"
+        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu"
         - TEST_CMD="make check-acceptance"
       addons:
         apt:
tests/acceptance/boot_linux_console.py +24 -0
index 0678ec91d2..20b845fce1 100644
@@ -9,8 +9,11 @@
 # later.  See the COPYING file in the top-level directory.
 
 import logging
+import os
 
 from avocado_qemu import Test
+from avocado.utils import process
+from avocado.utils import archive
 
 
 class BootLinuxConsole(Test):
@@ -80,3 +83,39 @@ class BootLinuxConsole(Test):
         self.vm.launch()
         console_pattern = 'Kernel command line: %s' % kernel_command_line
         self.wait_for_console_pattern(console_pattern)
+
+    def test_mips64el_malta(self):
+        """
+        This test requires the ar tool to extract "data.tar.gz" from
+        the Debian package.
+
+        The kernel can be rebuilt using this Debian kernel source [1] and
+        following the instructions on [2].
+
+        [1] https://kernel-team.pages.debian.net/kernel-handbook/ch-common-tasks.html#s-common-official
+        [2] http://snapshot.debian.org/package/linux-2.6/2.6.32-48/#linux-source-2.6.32_2.6.32-48
+
+        :avocado: tags=arch:mips64el
+        :avocado: tags=machine:malta
+        """
+        deb_url = ('http://snapshot.debian.org/archive/debian/20130217T032700Z/'
+                   'pool/main/l/linux-2.6/'
+                   'linux-image-2.6.32-5-5kc-malta_2.6.32-48_mipsel.deb')
+        deb_hash = '1aaec92083bf22fda31e0d27fa8d9a388e5fc3d5'
+        deb_path = self.fetch_asset(deb_url, asset_hash=deb_hash)
+

tests/acceptance/boot_linux_console.py class BootLinuxConsole(Test): · line 116

Quoted 10 lines
tests/acceptance/boot_linux_console.py +10 -0
+        cwd = os.getcwd()
+        os.chdir(self.workdir)
+        process.run("ar x %s data.tar.gz" % deb_path)
+        archive.extract("data.tar.gz", self.workdir)
+        os.chdir(cwd)
+        kernel_path = self.workdir + '/boot/vmlinux-2.6.32-5-5kc-malta'
+
+        self.vm.set_machine('malta')
+        self.vm.set_console()
+        kernel_command_line = 'console=ttyS0 printk.time=0'

What about having a generic KERNEL_COMMON_COMMAND_LINE='printk.time=0 ' then using
kernel_command_line = KERNEL_COMMON_COMMAND_LINE + 'console=ttyS0'
in each tests of this series?

Quoted 6 lines
tests/acceptance/boot_linux_console.py +5 -0
+        self.vm.add_args('-kernel', kernel_path,
+                         '-append', kernel_command_line)
+        self.vm.launch()
+        console_pattern = 'Kernel command line: %s' % kernel_command_line
+        self.wait_for_console_pattern(console_pattern)
#52 lore raw

Re: [Qemu-devel] [PATCH 13/18] Boot Linux Console Test: add a test for mips64el + malta

Quoted 80 lines · Philippe Mathieu-Daudé
Quoted 79 lines · Cleber Rosa

Similar to the x86_64 + pc test, it boots a Linux kernel on a Malta board and verify the serial is working.

If mips64el is a target being built, "make check-acceptance" will automatically include this test by the use of the "arch:mips" tags.

Alternatively, this test can be run using:

    $ avocado run -t arch:mips64el tests/acceptance
    $ avocado run -t machine:malta tests/acceptance
---
 .travis.yml                            |  2 +-
 tests/acceptance/boot_linux_console.py | 39 ++++++++++++++++++++++++++
 2 files changed, 40 insertions(+), 1 deletion(-)
.travis.yml +1 -1
index 49f9016e6a..28648f7a61 100644
@@ -187,7 +187,7 @@ matrix:
 
     # Acceptance (Functional) tests
     - env:
-        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu"
+        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu"
         - TEST_CMD="make check-acceptance"
       addons:
         apt:
tests/acceptance/boot_linux_console.py +31 -0
index 0678ec91d2..20b845fce1 100644
@@ -9,8 +9,11 @@
 # later.  See the COPYING file in the top-level directory.
 
 import logging
+import os
 
 from avocado_qemu import Test
+from avocado.utils import process
+from avocado.utils import archive
 
 
 class BootLinuxConsole(Test):
@@ -80,3 +83,39 @@ class BootLinuxConsole(Test):
         self.vm.launch()
         console_pattern = 'Kernel command line: %s' % kernel_command_line
         self.wait_for_console_pattern(console_pattern)
+
+    def test_mips64el_malta(self):
+        """
+        This test requires the ar tool to extract "data.tar.gz" from
+        the Debian package.
+
+        The kernel can be rebuilt using this Debian kernel source [1] and
+        following the instructions on [2].
+
+        [1] https://kernel-team.pages.debian.net/kernel-handbook/ch-common-tasks.html#s-common-official
+        [2] http://snapshot.debian.org/package/linux-2.6/2.6.32-48/#linux-source-2.6.32_2.6.32-48
+
+        :avocado: tags=arch:mips64el
+        :avocado: tags=machine:malta
+        """
+        deb_url = ('http://snapshot.debian.org/archive/debian/20130217T032700Z/'
+                   'pool/main/l/linux-2.6/'
+                   'linux-image-2.6.32-5-5kc-malta_2.6.32-48_mipsel.deb')
+        deb_hash = '1aaec92083bf22fda31e0d27fa8d9a388e5fc3d5'
+        deb_path = self.fetch_asset(deb_url, asset_hash=deb_hash)
+
+        cwd = os.getcwd()
+        os.chdir(self.workdir)
+        process.run("ar x %s data.tar.gz" % deb_path)
+        archive.extract("data.tar.gz", self.workdir)
+        os.chdir(cwd)
+        kernel_path = self.workdir + '/boot/vmlinux-2.6.32-5-5kc-malta'
+
Quoted 9 lines

tests/acceptance/boot_linux_console.py class BootLinuxConsole(Test): · line 116

Quoted 3 lines
tests/acceptance/boot_linux_console.py +3 -0
+        self.vm.set_machine('malta')
+        self.vm.set_console()
+        kernel_command_line = 'console=ttyS0 printk.time=0'

What about having a generic KERNEL_COMMON_COMMAND_LINE='printk.time=0 ' then using
kernel_command_line = KERNEL_COMMON_COMMAND_LINE + 'console=ttyS0'
in each tests of this series?

Not sure... I do like the fact that it's more uniform, but I don't like the locality aspect. Having to refer back to something defined either in the module or class level for something so small is a big con, IMO.

I'll keep it AS IS for now, and if more people feel like it's a good idea, or you feel stronger that it's a good idea, I can add it in the next version.

Thanks!
- Cleber.

Quoted 6 lines
Quoted 6 lines
tests/acceptance/boot_linux_console.py +5 -0
+        self.vm.add_args('-kernel', kernel_path,
+                         '-append', kernel_command_line)
+        self.vm.launch()
+        console_pattern = 'Kernel command line: %s' % kernel_command_line
+        self.wait_for_console_pattern(console_pattern)
#53 lore raw

Re: [Qemu-devel] [PATCH 13/18] Boot Linux Console Test: add a test for mips64el + malta

Quoted 91 lines · Cleber Rosa
Quoted 80 lines · Philippe Mathieu-Daudé
Quoted 79 lines · Cleber Rosa

Similar to the x86_64 + pc test, it boots a Linux kernel on a Malta board and verify the serial is working.

If mips64el is a target being built, "make check-acceptance" will automatically include this test by the use of the "arch:mips" tags.

Alternatively, this test can be run using:

    $ avocado run -t arch:mips64el tests/acceptance
    $ avocado run -t machine:malta tests/acceptance
---
 .travis.yml                            |  2 +-
 tests/acceptance/boot_linux_console.py | 39 ++++++++++++++++++++++++++
 2 files changed, 40 insertions(+), 1 deletion(-)
.travis.yml +1 -1
index 49f9016e6a..28648f7a61 100644
@@ -187,7 +187,7 @@ matrix:
 
     # Acceptance (Functional) tests
     - env:
-        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu"
+        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu"
         - TEST_CMD="make check-acceptance"
       addons:
         apt:
tests/acceptance/boot_linux_console.py +31 -0
index 0678ec91d2..20b845fce1 100644
@@ -9,8 +9,11 @@
 # later.  See the COPYING file in the top-level directory.
 
 import logging
+import os
 
 from avocado_qemu import Test
+from avocado.utils import process
+from avocado.utils import archive
 
 
 class BootLinuxConsole(Test):
@@ -80,3 +83,39 @@ class BootLinuxConsole(Test):
         self.vm.launch()
         console_pattern = 'Kernel command line: %s' % kernel_command_line
         self.wait_for_console_pattern(console_pattern)
+
+    def test_mips64el_malta(self):
+        """
+        This test requires the ar tool to extract "data.tar.gz" from
+        the Debian package.
+
+        The kernel can be rebuilt using this Debian kernel source [1] and
+        following the instructions on [2].
+
+        [1] https://kernel-team.pages.debian.net/kernel-handbook/ch-common-tasks.html#s-common-official
+        [2] http://snapshot.debian.org/package/linux-2.6/2.6.32-48/#linux-source-2.6.32_2.6.32-48
+
+        :avocado: tags=arch:mips64el
+        :avocado: tags=machine:malta
+        """
+        deb_url = ('http://snapshot.debian.org/archive/debian/20130217T032700Z/'
+                   'pool/main/l/linux-2.6/'
+                   'linux-image-2.6.32-5-5kc-malta_2.6.32-48_mipsel.deb')
+        deb_hash = '1aaec92083bf22fda31e0d27fa8d9a388e5fc3d5'
+        deb_path = self.fetch_asset(deb_url, asset_hash=deb_hash)
+
+        cwd = os.getcwd()
+        os.chdir(self.workdir)
+        process.run("ar x %s data.tar.gz" % deb_path)
+        archive.extract("data.tar.gz", self.workdir)
+        os.chdir(cwd)
+        kernel_path = self.workdir + '/boot/vmlinux-2.6.32-5-5kc-malta'
+
Quoted 9 lines

tests/acceptance/boot_linux_console.py class BootLinuxConsole(Test): · line 116

Quoted 3 lines
tests/acceptance/boot_linux_console.py +3 -0
+        self.vm.set_machine('malta')
+        self.vm.set_console()
+        kernel_command_line = 'console=ttyS0 printk.time=0'

What about having a generic KERNEL_COMMON_COMMAND_LINE='printk.time=0 ' then using
kernel_command_line = KERNEL_COMMON_COMMAND_LINE + 'console=ttyS0'
in each tests of this series?

Quoted 7 lines

Not sure... I do like the fact that it's more uniform, but I don't like the locality aspect. Having to refer back to something defined either in the module or class level for something so small is a big con, IMO.

I'll keep it AS IS for now, and if more people feel like it's a good idea, or you feel stronger that it's a good idea, I can add it in the next version.

I'm fine with that, however I'd really like all the Linux tests to use "printk.time=0" in their kernel cmdline, it really simplify parsing/diffing the console output.

Quoted 7 lines
Quoted 6 lines
Quoted 6 lines
tests/acceptance/boot_linux_console.py +5 -0
+        self.vm.add_args('-kernel', kernel_path,
+                         '-append', kernel_command_line)
+        self.vm.launch()
+        console_pattern = 'Kernel command line: %s' % kernel_command_line
+        self.wait_for_console_pattern(console_pattern)
#54 lore raw

Re: [Qemu-devel] [PATCH 13/18] Boot Linux Console Test: add a test for mips64el + malta

Quoted 96 lines · Philippe Mathieu-Daudé
Quoted 95 lines · Cleber Rosa
Quoted 80 lines · Philippe Mathieu-Daudé
Quoted 79 lines · Cleber Rosa

Similar to the x86_64 + pc test, it boots a Linux kernel on a Malta board and verify the serial is working.

If mips64el is a target being built, "make check-acceptance" will automatically include this test by the use of the "arch:mips" tags.

Alternatively, this test can be run using:

    $ avocado run -t arch:mips64el tests/acceptance
    $ avocado run -t machine:malta tests/acceptance
---
 .travis.yml                            |  2 +-
 tests/acceptance/boot_linux_console.py | 39 ++++++++++++++++++++++++++
 2 files changed, 40 insertions(+), 1 deletion(-)
.travis.yml +1 -1
index 49f9016e6a..28648f7a61 100644
@@ -187,7 +187,7 @@ matrix:
 
     # Acceptance (Functional) tests
     - env:
-        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu"
+        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu"
         - TEST_CMD="make check-acceptance"
       addons:
         apt:
tests/acceptance/boot_linux_console.py +31 -0
index 0678ec91d2..20b845fce1 100644
@@ -9,8 +9,11 @@
 # later.  See the COPYING file in the top-level directory.
 
 import logging
+import os
 
 from avocado_qemu import Test
+from avocado.utils import process
+from avocado.utils import archive
 
 
 class BootLinuxConsole(Test):
@@ -80,3 +83,39 @@ class BootLinuxConsole(Test):
         self.vm.launch()
         console_pattern = 'Kernel command line: %s' % kernel_command_line
         self.wait_for_console_pattern(console_pattern)
+
+    def test_mips64el_malta(self):
+        """
+        This test requires the ar tool to extract "data.tar.gz" from
+        the Debian package.
+
+        The kernel can be rebuilt using this Debian kernel source [1] and
+        following the instructions on [2].
+
+        [1] https://kernel-team.pages.debian.net/kernel-handbook/ch-common-tasks.html#s-common-official
+        [2] http://snapshot.debian.org/package/linux-2.6/2.6.32-48/#linux-source-2.6.32_2.6.32-48
+
+        :avocado: tags=arch:mips64el
+        :avocado: tags=machine:malta
+        """
+        deb_url = ('http://snapshot.debian.org/archive/debian/20130217T032700Z/'
+                   'pool/main/l/linux-2.6/'
+                   'linux-image-2.6.32-5-5kc-malta_2.6.32-48_mipsel.deb')
+        deb_hash = '1aaec92083bf22fda31e0d27fa8d9a388e5fc3d5'
+        deb_path = self.fetch_asset(deb_url, asset_hash=deb_hash)
+
+        cwd = os.getcwd()
+        os.chdir(self.workdir)
+        process.run("ar x %s data.tar.gz" % deb_path)
+        archive.extract("data.tar.gz", self.workdir)
+        os.chdir(cwd)
+        kernel_path = self.workdir + '/boot/vmlinux-2.6.32-5-5kc-malta'
+
Quoted 9 lines

tests/acceptance/boot_linux_console.py class BootLinuxConsole(Test): · line 116

Quoted 3 lines
tests/acceptance/boot_linux_console.py +3 -0
+        self.vm.set_machine('malta')
+        self.vm.set_console()
+        kernel_command_line = 'console=ttyS0 printk.time=0'

What about having a generic KERNEL_COMMON_COMMAND_LINE='printk.time=0 ' then using
kernel_command_line = KERNEL_COMMON_COMMAND_LINE + 'console=ttyS0'
in each tests of this series?

Not sure... I do like the fact that it's more uniform, but I don't like the locality aspect. Having to refer back to something defined either in the module or class level for something so small is a big con, IMO.

Quoted 8 lines
Quoted 3 lines

I'll keep it AS IS for now, and if more people feel like it's a good idea, or you feel stronger that it's a good idea, I can add it in the next version.

I'm fine with that, however I'd really like all the Linux tests to use "printk.time=0" in their kernel cmdline, it really simplify parsing/diffing the console output.

OK, I'll include that in v2.

Thanks!
- Cleber.

Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
Quoted 6 lines
tests/acceptance/boot_linux_console.py +5 -0
+        self.vm.add_args('-kernel', kernel_path,
+                         '-append', kernel_command_line)
+        self.vm.launch()
+        console_pattern = 'Kernel command line: %s' % kernel_command_line
+        self.wait_for_console_pattern(console_pattern)
#55 lore raw

Re: [Qemu-devel] [PATCH 13/18] Boot Linux Console Test: add a test for mips64el + malta

In reply to Cleber Rosa
Quoted 5 lines · Cleber Rosa

Similar to the x86_64 + pc test, it boots a Linux kernel on a Malta board and verify the serial is working.

If mips64el is a target being built, "make check-acceptance" will automatically include this test by the use of the "arch:mips" tags.

s/arch:mips/arch:mips64el

Quoted 47 lines

Alternatively, this test can be run using:

     $ avocado run -t arch:mips64el tests/acceptance
     $ avocado run -t machine:malta tests/acceptance
---
  .travis.yml                            |  2 +-
  tests/acceptance/boot_linux_console.py | 39 ++++++++++++++++++++++++++
  2 files changed, 40 insertions(+), 1 deletion(-)
.travis.yml +1 -1
index 49f9016e6a..28648f7a61 100644
@@ -187,7 +187,7 @@ matrix:
 
      # Acceptance (Functional) tests
      - env:
-        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu"
+        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu"
          - TEST_CMD="make check-acceptance"
        addons:
          apt:
tests/acceptance/boot_linux_console.py +4 -0
index 0678ec91d2..20b845fce1 100644
@@ -9,8 +9,11 @@
  # later.  See the COPYING file in the top-level directory.
 
  import logging
+import os
 
  from avocado_qemu import Test
+from avocado.utils import process
+from avocado.utils import archive
 
 
  class BootLinuxConsole(Test):
@@ -80,3 +83,39 @@ class BootLinuxConsole(Test):
          self.vm.launch()
          console_pattern = 'Kernel command line: %s' % kernel_command_line
          self.wait_for_console_pattern(console_pattern)
+

tests/acceptance/boot_linux_console.py class BootLinuxConsole(Test): · line 96

Quoted 10 lines
tests/acceptance/boot_linux_console.py +10 -0
+    def test_mips64el_malta(self):
+        """
+        This test requires the ar tool to extract "data.tar.gz" from
+        the Debian package.
+
+        The kernel can be rebuilt using this Debian kernel source [1] and
+        following the instructions on [2].
+
+        [1] https://kernel-team.pages.debian.net/kernel-handbook/ch-common-tasks.html#s-common-official
+        [2] http://snapshot.debian.org/package/linux-2.6/2.6.32-48/#linux-source-2.6.32_2.6.32-48

There references 1 and 2 are inverted.

- Wainer

Quoted 25 lines
tests/acceptance/boot_linux_console.py +25 -0
+
+        :avocado: tags=arch:mips64el
+        :avocado: tags=machine:malta
+        """
+        deb_url = ('http://snapshot.debian.org/archive/debian/20130217T032700Z/'
+                   'pool/main/l/linux-2.6/'
+                   'linux-image-2.6.32-5-5kc-malta_2.6.32-48_mipsel.deb')
+        deb_hash = '1aaec92083bf22fda31e0d27fa8d9a388e5fc3d5'
+        deb_path = self.fetch_asset(deb_url, asset_hash=deb_hash)
+
+        cwd = os.getcwd()
+        os.chdir(self.workdir)
+        process.run("ar x %s data.tar.gz" % deb_path)
+        archive.extract("data.tar.gz", self.workdir)
+        os.chdir(cwd)
+        kernel_path = self.workdir + '/boot/vmlinux-2.6.32-5-5kc-malta'
+
+        self.vm.set_machine('malta')
+        self.vm.set_console()
+        kernel_command_line = 'console=ttyS0 printk.time=0'
+        self.vm.add_args('-kernel', kernel_path,
+                         '-append', kernel_command_line)
+        self.vm.launch()
+        console_pattern = 'Kernel command line: %s' % kernel_command_line
+        self.wait_for_console_pattern(console_pattern)
#56 lore raw

Re: [Qemu-devel] [PATCH 13/18] Boot Linux Console Test: add a test for mips64el + malta

Quoted 10 lines · Wainer dos Santos Moschetta
Quoted 5 lines · Cleber Rosa

Similar to the x86_64 + pc test, it boots a Linux kernel on a Malta board and verify the serial is working.

If mips64el is a target being built, "make check-acceptance" will automatically include this test by the use of the "arch:mips" tags.

s/arch:mips/arch:mips64el

Good catch!

Quoted 58 lines
Quoted 58 lines

Alternatively, this test can be run using:

     $ avocado run -t arch:mips64el tests/acceptance
     $ avocado run -t machine:malta tests/acceptance

---
  .travis.yml                            |  2 +- tests/acceptance/boot_linux_console.py | 39 ++++++++++++++++++++++++++

  2 files changed, 40 insertions(+), 1 deletion(-)
.travis.yml +0 -0
index 49f9016e6a..28648f7a61 100644
@@ -187,7 +187,7 @@ matrix:
         # Acceptance (Functional) tests

      - env:
-        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu"
+        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu"
          - TEST_CMD="make check-acceptance" addons:
          apt:

tests/acceptance/boot_linux_console.py +0 -0
index 0678ec91d2..20b845fce1 100644
@@ -9,8 +9,11 @@
   # later.  See the COPYING file in the top-level directory.

    import logging
+import os
    from avocado_qemu import Test

tests/acceptance/boot_linux_console.py +2 -0
+from avocado.utils import process
+from avocado.utils import archive

      class BootLinuxConsole(Test):

tests/acceptance/boot_linux_console.py +0 -0
@@ -80,3 +83,39 @@ class BootLinuxConsole(Test):
           self.vm.launch()

          console_pattern = 'Kernel command line: %s' % kernel_command_line self.wait_for_console_pattern(console_pattern)

tests/acceptance/boot_linux_console.py +7 -0
+
+    def test_mips64el_malta(self):
+        """
+        This test requires the ar tool to extract "data.tar.gz" from
+        the Debian package.
+
+        The kernel can be rebuilt using this Debian kernel source [1]

and
+        following the instructions on [2].
+
+        [1]
https://kernel-team.pages.debian.net/kernel-handbook/ch-common-tasks.html#s-common-official

Quoted 6 lines
Quoted 3 lines

+        [2]
http://snapshot.debian.org/package/linux-2.6/2.6.32-48/#linux-source-2.6.32_2.6.32-48

There references 1 and 2 are inverted.

And yet another one good catch! Your proofreading is awesome!

- Cleber.

Quoted 31 lines

- Wainer

Quoted 28 lines

+
+        :avocado: tags=arch:mips64el
+        :avocado: tags=machine:malta
+        """
+        deb_url = ('http://snapshot.debian.org/archive/debian/20130217T032700Z/' +                   'pool/main/l/linux-2.6/'
+ 'linux-image-2.6.32-5-5kc-malta_2.6.32-48_mipsel.deb') +        deb_hash = '1aaec92083bf22fda31e0d27fa8d9a388e5fc3d5' +        deb_path = self.fetch_asset(deb_url, asset_hash=deb_hash)

tests/acceptance/boot_linux_console.py +15 -0
+
+        cwd = os.getcwd()
+        os.chdir(self.workdir)
+        process.run("ar x %s data.tar.gz" % deb_path)
+        archive.extract("data.tar.gz", self.workdir)
+        os.chdir(cwd)
+        kernel_path = self.workdir + '/boot/vmlinux-2.6.32-5-5kc-malta'
+
+        self.vm.set_machine('malta')
+        self.vm.set_console()
+        kernel_command_line = 'console=ttyS0 printk.time=0'
+        self.vm.add_args('-kernel', kernel_path,
+                         '-append', kernel_command_line)
+        self.vm.launch()
+        console_pattern = 'Kernel command line: %s' %

kernel_command_line
+        self.wait_for_console_pattern(console_pattern)

Signature · 4 lines
Cleber Rosa
[ Sr Software Engineer - Virtualization Team - Red Hat ]
[ Avocado Test Framework - avocado-framework.github.io ]
[  7ABB 96EB 8B46 B94D 5E0F  E9BB 657E 8D33 A5F2 09F3  ]
#57 lore raw

PATCH 14/18 Boot Linux Console Test: add a test for ppc64 + pseries

In reply to Cleber Rosa

Just like the previous tests, boots a Linux kernel on a ppc64 target using the pseries machine.

---
 .travis.yml                            |  2 +-
 scripts/qemu.py                        |  1 -
 tests/acceptance/boot_linux_console.py | 19 +++++++++++++++++++
 3 files changed, 20 insertions(+), 2 deletions(-)
.travis.yml +1 -1
index 28648f7a61..54100eea5a 100644
@@ -187,7 +187,7 @@ matrix:
 
     # Acceptance (Functional) tests
     - env:
-        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu"
+        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu"
         - TEST_CMD="make check-acceptance"
       addons:
         apt:
scripts/qemu.py +0 -1
index ef84b0f843..1531e28fc1 100644
@@ -37,7 +37,6 @@ CONSOLE_DEV_TYPES = {
     r'^clipper$': 'isa-serial',
     r'^(pc.*|q35.*|isapc)$': 'isa-serial',
     r'^(40p|powernv|prep)$': 'isa-serial',
-    r'^pseries.*': 'spapr-vty',
     r'^s390-ccw-virtio.*': 'sclpconsole',
     }
 
tests/acceptance/boot_linux_console.py +19 -0
index 20b845fce1..f3ccd23a7a 100644
@@ -119,3 +119,22 @@ class BootLinuxConsole(Test):
         self.vm.launch()
         console_pattern = 'Kernel command line: %s' % kernel_command_line
         self.wait_for_console_pattern(console_pattern)
+
+    def test_ppc64_pseries(self):
+        """
+        :avocado: tags=arch:ppc64
+        :avocado: tags=machine:pseries
+        """
+        kernel_url = ('http://mirrors.rit.edu/fedora/fedora-secondary/'
+                      'releases/29/Everything/ppc64le/os/ppc/ppc64/vmlinuz')
+        kernel_hash = '3fe04abfc852b66653b8c3c897a59a689270bc77'
+        kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
+
+        self.vm.set_machine('pseries')
+        self.vm.set_console()
+        kernel_command_line = 'console=hvc0'
+        self.vm.add_args('-kernel', kernel_path,
+                         '-append', kernel_command_line)
+        self.vm.launch()
+        console_pattern = 'Kernel command line: %s' % kernel_command_line
+        self.wait_for_console_pattern(console_pattern)
Signature · 1 line
2.20.1
Reviewed-by on v1 14/18
#58 lore raw

Re: [Qemu-devel] [PATCH 14/18] Boot Linux Console Test: add a test for ppc64 + pseries

Quoted 4 lines · Cleber Rosa

Just like the previous tests, boots a Linux kernel on a ppc64 target using the pseries machine.

Quoted 6 lines
---
 .travis.yml                            |  2 +-
 scripts/qemu.py                        |  1 -
 tests/acceptance/boot_linux_console.py | 19 +++++++++++++++++++
 3 files changed, 20 insertions(+), 2 deletions(-)

{...}

Quoted 1 line
Signature · 3 lines
Caio Carrara
Software Engineer, Virt Team - Red Hat
[email protected]
#59 lore raw

Re: [Qemu-devel] [PATCH 14/18] Boot Linux Console Test: add a test for ppc64 + pseries

In reply to Cleber Rosa
Quoted 2 lines · Cleber Rosa

Just like the previous tests, boots a Linux kernel on a ppc64 target using the pseries machine.

So running this on my rather slow SynQuacer I get:

(04/16) /home/alex/lsrc/qemu.git/tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_ppc64_pseries: INTERRUPTED: Test reported status but did not finish\nRunner error occurred: Timeout reached\nOriginal status: ERROR\n{'name': '04-/home/alex/lsrc/qemu.git/tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_ppc64_pseries', 'logdir': '/home/alex/lsrc/qemu.git/te... (60.93 s)

which I'm guessing is a timeout occurring.

I wonder if that means we should:

a) set timeouts longer for when running on TCG
or
b) split tests into TCG and KVM tests and select KVM tests on appropriate HW

The qemu.py code has (slightly flawed) logic for detecting KVM and passing --enable-kvm. Maybe we should be doing that here?

Quoted 50 lines
---
 .travis.yml                            |  2 +-
 scripts/qemu.py                        |  1 -
 tests/acceptance/boot_linux_console.py | 19 +++++++++++++++++++
 3 files changed, 20 insertions(+), 2 deletions(-)
.travis.yml +1 -1
index 28648f7a61..54100eea5a 100644
@@ -187,7 +187,7 @@ matrix:
 
     # Acceptance (Functional) tests
     - env:
-        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu"
+        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu"
         - TEST_CMD="make check-acceptance"
       addons:
         apt:
scripts/qemu.py +0 -1
index ef84b0f843..1531e28fc1 100644
@@ -37,7 +37,6 @@ CONSOLE_DEV_TYPES = {
     r'^clipper$': 'isa-serial',
     r'^(pc.*|q35.*|isapc)$': 'isa-serial',
     r'^(40p|powernv|prep)$': 'isa-serial',
-    r'^pseries.*': 'spapr-vty',
     r'^s390-ccw-virtio.*': 'sclpconsole',
     }
 
tests/acceptance/boot_linux_console.py +9 -0
index 20b845fce1..f3ccd23a7a 100644
@@ -119,3 +119,22 @@ class BootLinuxConsole(Test):
         self.vm.launch()
         console_pattern = 'Kernel command line: %s' % kernel_command_line
         self.wait_for_console_pattern(console_pattern)
+
+    def test_ppc64_pseries(self):
+        """
+        :avocado: tags=arch:ppc64
+        :avocado: tags=machine:pseries
+        """
+        kernel_url = ('http://mirrors.rit.edu/fedora/fedora-secondary/'
+                      'releases/29/Everything/ppc64le/os/ppc/ppc64/vmlinuz')
+        kernel_hash = '3fe04abfc852b66653b8c3c897a59a689270bc77'

tests/acceptance/boot_linux_console.py class BootLinuxConsole(Test): · line 140

Quoted 10 lines
tests/acceptance/boot_linux_console.py +10 -0
+        kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
+
+        self.vm.set_machine('pseries')
+        self.vm.set_console()
+        kernel_command_line = 'console=hvc0'
+        self.vm.add_args('-kernel', kernel_path,
+                         '-append', kernel_command_line)
+        self.vm.launch()
+        console_pattern = 'Kernel command line: %s' % kernel_command_line
+        self.wait_for_console_pattern(console_pattern)

--
Alex Bennée

#60 lore raw

Re: [Qemu-devel] [PATCH 14/18] Boot Linux Console Test: add a test for ppc64 + pseries

Quoted 12 lines · Alex Bennée
Quoted 2 lines · Cleber Rosa

Just like the previous tests, boots a Linux kernel on a ppc64 target using the pseries machine.

So running this on my rather slow SynQuacer I get:

(04/16) /home/alex/lsrc/qemu.git/tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_ppc64_pseries: INTERRUPTED: Test reported status but did not finish\nRunner error occurred: Timeout reached\nOriginal status: ERROR\n{'name': '04-/home/alex/lsrc/qemu.git/tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_ppc64_pseries', 'logdir': '/home/alex/lsrc/qemu.git/te... (60.93 s)

which I'm guessing is a timeout occurring.

Yes, that's what's happening. It's hard to pinpoint, and control, the sluggishness points in such a test running on a different environment. For this one execution, I do trust your assessment, and it's most certainly caused by your "slow SynQuacer", spending too much running emulation code.

But, I'd like to mention that there are other possibilities. One is that you're hitting a "asset fetcher bug" that I recently fixed in Avocado[1] (fix to be available on 68.0, to be released next Monday, Feb 4th).

Even with that bug fixed, I feel like it's unfair to test code to spend its time waiting to download a file when it's not testing *the file download itself*. Because of that, there are plans to add an (optional) job pre-processing step that will make sure the needed assets are in the cache ahead of time[2][3].

Quoted 6 lines

I wonder if that means we should:

a) set timeouts longer for when running on TCG
or
b) split tests into TCG and KVM tests and select KVM tests on appropriate HW

I wonder the same, and I believe this falls into a similar scenario we've seen with the setup of console devices in the QEMUMachine class. I've started by setting the device types defined at the framework level, and then reverted to the machine's default devices (using '-serial'), because the "default" behavior of QEMU is usually what a test writer wants when not setting something else explicitly.

Quoted 3 lines

The qemu.py code has (slightly flawed) logic for detecting KVM and passing --enable-kvm. Maybe we should be doing that here?

I'm not sure. IMO, the common question is: should we magically (at a framework level) configure tests based on probed host environment characteristics? I feel like we should attempt to minimize that for the sake of tests being more obvious and more easily reproducible.

And because of that, I'd go, *initially*, with an approach more similar to your option "b".

Having said that, we don't want to rewrite most tests just to be able to test with either KVM or TCG, if the tests are not explicitly testing KVM or TCG. At this point, using KVM or TCG is test/framework *configuration*, and in Avocado we hope to solve this by having the executed tests easily identifiable and reproducible (a test ID will contain a information about the options passed, and a replay of the job will apply the same configuration).

For now, I think the best approach is to increase the timeout, because I think it's much worse to have to deal with false negatives (longer execution times that don't really mean a failure), than having a test possibly taking some more time to finish.

And sorry for extremely the long answer!
- Cleber.

[1] - https://github.com/avocado-framework/avocado/pull/2996
[2] -
https://trello.com/c/WPd4FrIy/1479-add-support-to-specify-assets-in-test-docstring
[3] - https://trello.com/c/CKP7YS6G/1481-on-cache-check-for-asset-fetcher

#61 lore raw

Re: [Qemu-devel] [PATCH 14/18] Boot Linux Console Test: add a test for ppc64 + pseries

Quoted 24 lines · Cleber Rosa
Quoted 11 lines · Alex Bennée
Quoted 2 lines · Cleber Rosa

Just like the previous tests, boots a Linux kernel on a ppc64 target using the pseries machine.

So running this on my rather slow SynQuacer I get:

(04/16) /home/alex/lsrc/qemu.git/tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_ppc64_pseries: INTERRUPTED: Test reported status but did not finish\nRunner error occurred: Timeout reached\nOriginal status: ERROR\n{'name': '04-/home/alex/lsrc/qemu.git/tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_ppc64_pseries', 'logdir': '/home/alex/lsrc/qemu.git/te... (60.93 s)

which I'm guessing is a timeout occurring.

Yes, that's what's happening. It's hard to pinpoint, and control, the sluggishness points in such a test running on a different environment. For this one execution, I do trust your assessment, and it's most certainly caused by your "slow SynQuacer", spending too much running emulation code.

But, I'd like to mention that there are other possibilities. One is that you're hitting a "asset fetcher bug" that I recently fixed in Avocado[1] (fix to be available on 68.0, to be released next Monday, Feb 4th).

Quoted 9 lines

Even with that bug fixed, I feel like it's unfair to test code to spend its time waiting to download a file when it's not testing *the file download itself*. Because of that, there are plans to add an (optional) job pre-processing step that will make sure the needed assets are in the cache ahead of time[2][3].

Quoted 3 lines

I wonder if that means we should:

a) set timeouts longer for when running on TCG

I hit the same problem with VM tests, and suggested a poor "increase timeout" patch [1].

Then Peter sent a different patch [2] which happens to inadvertently Dictionary resolve my problem, since the longer a VM took to boot on the Cavium ThunderX I have access is 288 seconds, which is closely below the 300 seconds limit =) I understood nobody seemed to really care about testing the x86 TCG backend this way, so I didn't worry much.

[1] http://lists.nongnu.org/archive/html/qemu-devel/2018-07/msg03416.html
[2] http://lists.nongnu.org/archive/html/qemu-devel/2018-08/msg04977.html

Quoted 11 lines
Quoted 3 lines

or
b) split tests into TCG and KVM tests and select KVM tests on appropriate HW

I wonder the same, and I believe this falls into a similar scenario we've seen with the setup of console devices in the QEMUMachine class. I've started by setting the device types defined at the framework level, and then reverted to the machine's default devices (using '-serial'), because the "default" behavior of QEMU is usually what a test writer wants when not setting something else explicitly.

Quoted 8 lines
Quoted 3 lines

The qemu.py code has (slightly flawed) logic for detecting KVM and passing --enable-kvm. Maybe we should be doing that here?

I'm not sure. IMO, the common question is: should we magically (at a framework level) configure tests based on probed host environment characteristics? I feel like we should attempt to minimize that for the sake of tests being more obvious and more easily reproducible.

I agree we shouldn't randomly test different features, but rather explicitly add 2 tests (TCG/KVM), and if it is not possible to run a test, mark it as SKIPPED.

An user with KVM available would then have to run --filter-out=tcg, or build QEMU with --disable-tcg.

Quoted 24 lines

And because of that, I'd go, *initially*, with an approach more similar to your option "b".

Having said that, we don't want to rewrite most tests just to be able to test with either KVM or TCG, if the tests are not explicitly testing KVM or TCG. At this point, using KVM or TCG is test/framework *configuration*, and in Avocado we hope to solve this by having the executed tests easily identifiable and reproducible (a test ID will contain a information about the options passed, and a replay of the job will apply the same configuration).

For now, I think the best approach is to increase the timeout, because I think it's much worse to have to deal with false negatives (longer execution times that don't really mean a failure), than having a test possibly taking some more time to finish.

And sorry for extremely the long answer!
- Cleber.

[1] - https://github.com/avocado-framework/avocado/pull/2996
[2] -
https://trello.com/c/WPd4FrIy/1479-add-support-to-specify-assets-in-test-docstring
[3] - https://trello.com/c/CKP7YS6G/1481-on-cache-check-for-asset-fetcher

#62 lore raw

PATCH 15/18 Boot Linux Console Test: add a test for aarch64 + virt

In reply to Cleber Rosa

Just like the previous tests, boots a Linux kernel on a aarch64 target using the virt machine.

One special option added is the CPU type, given that the kernel selected fails to boot on the virt machine's default CPU (cortex-a15).

---
 .travis.yml                            |  2 +-
 tests/acceptance/boot_linux_console.py | 20 ++++++++++++++++++++
 2 files changed, 21 insertions(+), 1 deletion(-)
.travis.yml +1 -1
index 54100eea5a..595e8c0b6c 100644
@@ -187,7 +187,7 @@ matrix:
 
     # Acceptance (Functional) tests
     - env:
-        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu"
+        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu"
         - TEST_CMD="make check-acceptance"
       addons:
         apt:
tests/acceptance/boot_linux_console.py +20 -0
index f3ccd23a7a..107700b517 100644
@@ -138,3 +138,23 @@ class BootLinuxConsole(Test):
         self.vm.launch()
         console_pattern = 'Kernel command line: %s' % kernel_command_line
         self.wait_for_console_pattern(console_pattern)
+
+    def test_aarch64_virt(self):
+        """
+        :avocado: tags=arch:aarch64
+        :avocado: tags=machine:virt
+        """
+        kernel_url = ('https://sjc.edge.kernel.org/fedora-buffet/fedora/linux/'
+                      'releases/29/Server/aarch64/os/images/pxeboot/vmlinuz')
+        kernel_hash = '8c73e469fc6ea06a58dc83a628fc695b693b8493'
+        kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
+
+        self.vm.set_machine('virt')
+        self.vm.set_console()
+        kernel_command_line = 'console=ttyAMA0'
+        self.vm.add_args('-cpu', 'cortex-a53',
+                         '-kernel', kernel_path,
+                         '-append', kernel_command_line)
+        self.vm.launch()
+        console_pattern = 'Kernel command line: %s' % kernel_command_line
+        self.wait_for_console_pattern(console_pattern)
Signature · 1 line
2.20.1
Reviewed-by on v1 15/18
#63 lore raw

Re: [Qemu-devel] [PATCH 15/18] Boot Linux Console Test: add a test for aarch64 + virt

Quoted 7 lines · Cleber Rosa

Just like the previous tests, boots a Linux kernel on a aarch64 target using the virt machine.

One special option added is the CPU type, given that the kernel selected fails to boot on the virt machine's default CPU (cortex-a15).

Quoted 5 lines
---
 .travis.yml                            |  2 +-
 tests/acceptance/boot_linux_console.py | 20 ++++++++++++++++++++
 2 files changed, 21 insertions(+), 1 deletion(-)

{...}

Quoted 1 line
Signature · 3 lines
Caio Carrara
Software Engineer, Virt Team - Red Hat
[email protected]
#64 lore raw

Re: [Qemu-devel] [PATCH 15/18] Boot Linux Console Test: add a test for aarch64 + virt

In reply to Cleber Rosa
Quoted 30 lines · Cleber Rosa

Just like the previous tests, boots a Linux kernel on a aarch64 target using the virt machine.

One special option added is the CPU type, given that the kernel selected fails to boot on the virt machine's default CPU (cortex-a15).

---
  .travis.yml                            |  2 +-
  tests/acceptance/boot_linux_console.py | 20 ++++++++++++++++++++
  2 files changed, 21 insertions(+), 1 deletion(-)
.travis.yml +1 -1
index 54100eea5a..595e8c0b6c 100644
@@ -187,7 +187,7 @@ matrix:
 
      # Acceptance (Functional) tests
      - env:
-        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu"
+        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu"
          - TEST_CMD="make check-acceptance"
        addons:
          apt:
tests/acceptance/boot_linux_console.py +0 -0
index f3ccd23a7a..107700b517 100644

tests/acceptance/boot_linux_console.py class BootLinuxConsole(Test): · line 142

Quoted 5 lines
tests/acceptance/boot_linux_console.py +2 -0
@@ -138,3 +138,23 @@ class BootLinuxConsole(Test):
          self.vm.launch()
          console_pattern = 'Kernel command line: %s' % kernel_command_line
          self.wait_for_console_pattern(console_pattern)
+
+    def test_aarch64_virt(self):

That test case fails on my system (Fedora 29 x86_64). Avocado seems unable to kill the VM so it  reaches the timeout.

I compiled QEMU with default configuration:

$ configure --python=/usr/bin/python3 --target-list=x86_64-softmmu --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu)

Follows a snippet of the Avocado's job.log file:
----

2019-01-31 14:41:34,912 test             L0602 INFO | START 07-/root/src/qemu/tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_aarch64_virt
2019-01-31 14:41:34,912 test             L0298 DEBUG| DATA (filename=output.expected) => NOT FOUND (data sources: variant, test, file)
2019-01-31 14:41:34,913 parameters       L0146 DEBUG| PARAMS (key=arch, path=*, default=aarch64) => 'aarch64'
2019-01-31 14:41:34,913 parameters       L0146 DEBUG| PARAMS (key=qemu_bin, path=*, default=aarch64-softmmu/qemu-system-aarch64) => 'aarch64-softmmu/qemu-system-aarch64'
2019-01-31 14:41:34,915 download         L0070 INFO | Fetching https://sjc.edge.kernel.org/fedora-buffet/fedora/linux/releases/29/Server/aarch64/os/images/pxeboot/vmlinuz -> /var/lib/avocado/data/cache/by_name/vmlinuz.3upct2pr
2019-01-31 14:41:35,490 download         L0054 DEBUG| Retrieved URL "https://sjc.edge.kernel.org/fedora-buffet/fedora/linux/releases/29/Server/aarch64/os/images/pxeboot/vmlinuz": content-length 8623423, date: "Thu, 31 Jan 2019 19:41:35 GMT", last-modified: "Sun, 21 Oct 2018 00:43:09 GMT"
2019-01-31 14:41:41,765 qemu             L0317 DEBUG| VM launch command: 'aarch64-softmmu/qemu-system-aarch64 -chardev socket,id=mon,path=/var/tmp/tmpizirkcud/qemu-32609-monitor.sock -mon chardev=mon,mode=control -display none -vga none -machine virt -chardev socket,id=console,path=/var/tmp/tmpizirkcud/qemu-32609-console.sock,server,nowait -serial chardev:console -cpu cortex-a53 -kernel /var/lib/avocado/data/cache/by_name/vmlinuz -append console=ttyAMA0'
2019-01-31 14:41:41,779 qmp              L0167 DEBUG| >>> {'execute': 'qmp_capabilities'}
2019-01-31 14:41:41,931 qmp              L0175 DEBUG| <<< {'return': {}}
2019-01-31 14:41:42,830 boot_linux_conso L0041 DEBUG| [    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd034]

(...)

2019-01-31 14:41:42,833 boot_linux_conso L0041 DEBUG| [    0.000000] Policy zone: DMA32
2019-01-31 14:41:42,833 boot_linux_conso L0041 DEBUG| [    0.000000] Kernel command line: console=ttyAMA0
2019-01-31 14:41:42,833 qmp              L0167 DEBUG| >>> {'execute': 'quit'}
2019-01-31 14:44:35,636 qemu             L0357 WARNI| qemu received signal -9: aarch64-softmmu/qemu-system-aarch64 -chardev socket,id=mon,path=/var/tmp/tmpizirkcud/qemu-32609-monitor.sock -mon chardev=mon,mode=control -display none -vga none -machine virt -chardev socket,id=console,path=/var/tmp/tmpizirkcud/qemu-32609-console.sock,server,nowait -serial chardev:console -cpu cortex-a53 -kernel /var/lib/avocado/data/cache/by_name/vmlinuz -append console=ttyAMA0

(...)

2019-01-31 14:44:35,663 runner           L0253 ERROR| ERROR Test reported status but did not finish -> TestAbortError: 07-/root/src/qemu/tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_aarch64_virt.
2019-01-31 14:44:35,664 runner           L0062 ERROR| Runner error occurred: Timeout reached

----

- Wainer

Quoted 18 lines
tests/acceptance/boot_linux_console.py +18 -0
+        """
+        :avocado: tags=arch:aarch64
+        :avocado: tags=machine:virt
+        """
+        kernel_url = ('https://sjc.edge.kernel.org/fedora-buffet/fedora/linux/'
+                      'releases/29/Server/aarch64/os/images/pxeboot/vmlinuz')
+        kernel_hash = '8c73e469fc6ea06a58dc83a628fc695b693b8493'
+        kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
+
+        self.vm.set_machine('virt')
+        self.vm.set_console()
+        kernel_command_line = 'console=ttyAMA0'
+        self.vm.add_args('-cpu', 'cortex-a53',
+                         '-kernel', kernel_path,
+                         '-append', kernel_command_line)
+        self.vm.launch()
+        console_pattern = 'Kernel command line: %s' % kernel_command_line
+        self.wait_for_console_pattern(console_pattern)
#65 lore raw

Re: [Qemu-devel] [PATCH 15/18] Boot Linux Console Test: add a test for aarch64 + virt

Quoted 83 lines · Wainer dos Santos Moschetta
Quoted 34 lines · Cleber Rosa

Just like the previous tests, boots a Linux kernel on a aarch64 target using the virt machine.

One special option added is the CPU type, given that the kernel selected fails to boot on the virt machine's default CPU (cortex-a15).

---
  .travis.yml                            |  2 +- tests/acceptance/boot_linux_console.py | 20 ++++++++++++++++++++

  2 files changed, 21 insertions(+), 1 deletion(-)
.travis.yml +0 -0
index 54100eea5a..595e8c0b6c 100644
@@ -187,7 +187,7 @@ matrix:
         # Acceptance (Functional) tests

      - env:
-        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu"
+        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu"

          - TEST_CMD="make check-acceptance"
        addons:
          apt:

tests/acceptance/boot_linux_console.py +0 -0
index f3ccd23a7a..107700b517 100644
@@ -138,3 +138,23 @@ class BootLinuxConsole(Test):
           self.vm.launch()
Quoted 5 lines

          console_pattern = 'Kernel command line: %s' % kernel_command_line self.wait_for_console_pattern(console_pattern) +
+    def test_aarch64_virt(self):

That test case fails on my system (Fedora 29 x86_64). Avocado seems unable to kill the VM so it  reaches the timeout.

I compiled QEMU with default configuration:

$ configure --python=/usr/bin/python3 --target-list=x86_64-softmmu
--target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu)

Follows a snippet of the Avocado's job.log file:
----

2019-01-31 14:41:34,912 test             L0602 INFO | START

07-/root/src/qemu/tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_aarch64_virt

2019-01-31 14:41:34,912 test             L0298 DEBUG| DATA
(filename=output.expected) => NOT FOUND (data sources: variant, test, file)
2019-01-31 14:41:34,913 parameters       L0146 DEBUG| PARAMS (key=arch,
path=*, default=aarch64) => 'aarch64'
2019-01-31 14:41:34,913 parameters       L0146 DEBUG| PARAMS
(key=qemu_bin, path=*, default=aarch64-softmmu/qemu-system-aarch64) =>
'aarch64-softmmu/qemu-system-aarch64'
2019-01-31 14:41:34,915 download         L0070 INFO | Fetching
https://sjc.edge.kernel.org/fedora-buffet/fedora/linux/releases/29/Server/aarch64/os/images/pxeboot/vmlinuz
-> /var/lib/avocado/data/cache/by_name/vmlinuz.3upct2pr
2019-01-31 14:41:35,490 download         L0054 DEBUG| Retrieved URL
"https://sjc.edge.kernel.org/fedora-buffet/fedora/linux/releases/29/Server/aarch64/os/images/pxeboot/vmlinuz":
content-length 8623423, date: "Thu, 31 Jan 2019 19:41:35 GMT",
last-modified: "Sun, 21 Oct 2018 00:43:09 GMT"
2019-01-31 14:41:41,765 qemu             L0317 DEBUG| VM launch command:

'aarch64-softmmu/qemu-system-aarch64 -chardev socket,id=mon,path=/var/tmp/tmpizirkcud/qemu-32609-monitor.sock -mon chardev=mon,mode=control -display none -vga none -machine virt -chardev socket,id=console,path=/var/tmp/tmpizirkcud/qemu-32609-console.sock,server,nowait -serial chardev:console -cpu cortex-a53 -kernel /var/lib/avocado/data/cache/by_name/vmlinuz -append console=ttyAMA0'

2019-01-31 14:41:41,779 qmp              L0167 DEBUG| >>> {'execute':
'qmp_capabilities'}
2019-01-31 14:41:41,931 qmp              L0175 DEBUG| <<< {'return': {}}
2019-01-31 14:41:42,830 boot_linux_conso L0041 DEBUG| [    0.000000]

Booting Linux on physical CPU 0x0000000000 [0x410fd034]

Quoted 8 lines

(...)

2019-01-31 14:41:42,833 boot_linux_conso L0041 DEBUG| [    0.000000]
Policy zone: DMA32
2019-01-31 14:41:42,833 boot_linux_conso L0041 DEBUG| [    0.000000]
Kernel command line: console=ttyAMA0
2019-01-31 14:41:42,833 qmp              L0167 DEBUG| >>> {'execute':

'quit'}

Here, a QMP response like "<<< {'return': {}}" would be expected.

Since I can not reproduce this on my system (or on Travis-CI jobs I've sent), can you tell me on top of which commit you've applied these patches?

Thanks!
- Cleber.

Quoted 43 lines
2019-01-31 14:44:35,636 qemu             L0357 WARNI| qemu received

signal -9: aarch64-softmmu/qemu-system-aarch64 -chardev socket,id=mon,path=/var/tmp/tmpizirkcud/qemu-32609-monitor.sock -mon chardev=mon,mode=control -display none -vga none -machine virt -chardev socket,id=console,path=/var/tmp/tmpizirkcud/qemu-32609-console.sock,server,nowait -serial chardev:console -cpu cortex-a53 -kernel /var/lib/avocado/data/cache/by_name/vmlinuz -append console=ttyAMA0

(...)

2019-01-31 14:44:35,663 runner           L0253 ERROR| ERROR Test

reported status but did not finish -> TestAbortError:
07-/root/src/qemu/tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_aarch64_virt.

2019-01-31 14:44:35,664 runner           L0062 ERROR| Runner error

occurred: Timeout reached
----

- Wainer

Quoted 22 lines

+        """
+        :avocado: tags=arch:aarch64
+        :avocado: tags=machine:virt
+        """
+        kernel_url = ('https://sjc.edge.kernel.org/fedora-buffet/fedora/linux/' + 'releases/29/Server/aarch64/os/images/pxeboot/vmlinuz') +        kernel_hash = '8c73e469fc6ea06a58dc83a628fc695b693b8493' +        kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
+
+        self.vm.set_machine('virt')
+        self.vm.set_console()
+        kernel_command_line = 'console=ttyAMA0'
+        self.vm.add_args('-cpu', 'cortex-a53',
+                         '-kernel', kernel_path,
+                         '-append', kernel_command_line) +        self.vm.launch()
+        console_pattern = 'Kernel command line: %s' % kernel_command_line
+        self.wait_for_console_pattern(console_pattern)

#66 lore raw

Re: [Qemu-devel] [PATCH 15/18] Boot Linux Console Test: add a test for aarch64 + virt

Quoted 95 lines · Cleber Rosa
Quoted 83 lines · Wainer dos Santos Moschetta
Quoted 34 lines · Cleber Rosa

Just like the previous tests, boots a Linux kernel on a aarch64 target using the virt machine.

One special option added is the CPU type, given that the kernel selected fails to boot on the virt machine's default CPU (cortex-a15).

---
  .travis.yml                            |  2 +- tests/acceptance/boot_linux_console.py | 20 ++++++++++++++++++++

  2 files changed, 21 insertions(+), 1 deletion(-)
.travis.yml +0 -0
index 54100eea5a..595e8c0b6c 100644
@@ -187,7 +187,7 @@ matrix:
         # Acceptance (Functional) tests

      - env:
-        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu"
+        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu"

          - TEST_CMD="make check-acceptance"
        addons:
          apt:

tests/acceptance/boot_linux_console.py +0 -0
index f3ccd23a7a..107700b517 100644
@@ -138,3 +138,23 @@ class BootLinuxConsole(Test):
           self.vm.launch()
Quoted 5 lines

          console_pattern = 'Kernel command line: %s' % kernel_command_line self.wait_for_console_pattern(console_pattern) +
+    def test_aarch64_virt(self):

That test case fails on my system (Fedora 29 x86_64). Avocado seems unable to kill the VM so it  reaches the timeout.

I compiled QEMU with default configuration:

$ configure --python=/usr/bin/python3 --target-list=x86_64-softmmu
--target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu)

Follows a snippet of the Avocado's job.log file:
----

2019-01-31 14:41:34,912 test             L0602 INFO | START

07-/root/src/qemu/tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_aarch64_virt

2019-01-31 14:41:34,912 test             L0298 DEBUG| DATA
(filename=output.expected) => NOT FOUND (data sources: variant, test, file)
2019-01-31 14:41:34,913 parameters       L0146 DEBUG| PARAMS (key=arch,
path=*, default=aarch64) => 'aarch64'
2019-01-31 14:41:34,913 parameters       L0146 DEBUG| PARAMS
(key=qemu_bin, path=*, default=aarch64-softmmu/qemu-system-aarch64) =>
'aarch64-softmmu/qemu-system-aarch64'
2019-01-31 14:41:34,915 download         L0070 INFO | Fetching
https://sjc.edge.kernel.org/fedora-buffet/fedora/linux/releases/29/Server/aarch64/os/images/pxeboot/vmlinuz
-> /var/lib/avocado/data/cache/by_name/vmlinuz.3upct2pr
2019-01-31 14:41:35,490 download         L0054 DEBUG| Retrieved URL
"https://sjc.edge.kernel.org/fedora-buffet/fedora/linux/releases/29/Server/aarch64/os/images/pxeboot/vmlinuz":
content-length 8623423, date: "Thu, 31 Jan 2019 19:41:35 GMT",
last-modified: "Sun, 21 Oct 2018 00:43:09 GMT"
2019-01-31 14:41:41,765 qemu             L0317 DEBUG| VM launch command:

'aarch64-softmmu/qemu-system-aarch64 -chardev socket,id=mon,path=/var/tmp/tmpizirkcud/qemu-32609-monitor.sock -mon chardev=mon,mode=control -display none -vga none -machine virt -chardev socket,id=console,path=/var/tmp/tmpizirkcud/qemu-32609-console.sock,server,nowait -serial chardev:console -cpu cortex-a53 -kernel /var/lib/avocado/data/cache/by_name/vmlinuz -append console=ttyAMA0'

2019-01-31 14:41:41,779 qmp              L0167 DEBUG| >>> {'execute':
'qmp_capabilities'}
2019-01-31 14:41:41,931 qmp              L0175 DEBUG| <<< {'return': {}}
2019-01-31 14:41:42,830 boot_linux_conso L0041 DEBUG| [    0.000000]

Booting Linux on physical CPU 0x0000000000 [0x410fd034]

Quoted 8 lines

(...)

2019-01-31 14:41:42,833 boot_linux_conso L0041 DEBUG| [    0.000000]
Policy zone: DMA32
2019-01-31 14:41:42,833 boot_linux_conso L0041 DEBUG| [    0.000000]
Kernel command line: console=ttyAMA0
2019-01-31 14:41:42,833 qmp              L0167 DEBUG| >>> {'execute':

'quit'}

Quoted 5 lines

Here, a QMP response like "<<< {'return': {}}" would be expected.

Since I can not reproduce this on my system (or on Travis-CI jobs I've sent), can you tell me on top of which commit you've applied these patches?

I spoke too soon:

https://travis-ci.org/clebergnu/qemu/jobs/487121425#L3033

This looks like a recent regression, and I'm guessing it's not on the test's side. I'll try to bisect it and let you know.

Thanks,
- Cleber.

#67 lore raw

Re: [Qemu-devel] [PATCH 15/18] Boot Linux Console Test: add a test for aarch64 + virt

Quoted 104 lines · Cleber Rosa
Quoted 95 lines · Cleber Rosa
Quoted 83 lines · Wainer dos Santos Moschetta
Quoted 34 lines · Cleber Rosa

Just like the previous tests, boots a Linux kernel on a aarch64 target using the virt machine.

One special option added is the CPU type, given that the kernel selected fails to boot on the virt machine's default CPU (cortex-a15).

---
  .travis.yml                            |  2 +- tests/acceptance/boot_linux_console.py | 20 ++++++++++++++++++++

  2 files changed, 21 insertions(+), 1 deletion(-)
.travis.yml +0 -0
index 54100eea5a..595e8c0b6c 100644
@@ -187,7 +187,7 @@ matrix:
         # Acceptance (Functional) tests

      - env:
-        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu"
+        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu"

          - TEST_CMD="make check-acceptance"
        addons:
          apt:

tests/acceptance/boot_linux_console.py +0 -0
index f3ccd23a7a..107700b517 100644
@@ -138,3 +138,23 @@ class BootLinuxConsole(Test):
           self.vm.launch()
Quoted 5 lines

          console_pattern = 'Kernel command line: %s' % kernel_command_line self.wait_for_console_pattern(console_pattern) +
+    def test_aarch64_virt(self):

That test case fails on my system (Fedora 29 x86_64). Avocado seems unable to kill the VM so it  reaches the timeout.

I compiled QEMU with default configuration:

$ configure --python=/usr/bin/python3 --target-list=x86_64-softmmu
--target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu)

Follows a snippet of the Avocado's job.log file:
----

2019-01-31 14:41:34,912 test             L0602 INFO | START

07-/root/src/qemu/tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_aarch64_virt

2019-01-31 14:41:34,912 test             L0298 DEBUG| DATA
(filename=output.expected) => NOT FOUND (data sources: variant, test, file)
2019-01-31 14:41:34,913 parameters       L0146 DEBUG| PARAMS (key=arch,
path=*, default=aarch64) => 'aarch64'
2019-01-31 14:41:34,913 parameters       L0146 DEBUG| PARAMS
(key=qemu_bin, path=*, default=aarch64-softmmu/qemu-system-aarch64) =>
'aarch64-softmmu/qemu-system-aarch64'
2019-01-31 14:41:34,915 download         L0070 INFO | Fetching
https://sjc.edge.kernel.org/fedora-buffet/fedora/linux/releases/29/Server/aarch64/os/images/pxeboot/vmlinuz
-> /var/lib/avocado/data/cache/by_name/vmlinuz.3upct2pr
2019-01-31 14:41:35,490 download         L0054 DEBUG| Retrieved URL
"https://sjc.edge.kernel.org/fedora-buffet/fedora/linux/releases/29/Server/aarch64/os/images/pxeboot/vmlinuz":
content-length 8623423, date: "Thu, 31 Jan 2019 19:41:35 GMT",
last-modified: "Sun, 21 Oct 2018 00:43:09 GMT"
2019-01-31 14:41:41,765 qemu             L0317 DEBUG| VM launch command:

'aarch64-softmmu/qemu-system-aarch64 -chardev socket,id=mon,path=/var/tmp/tmpizirkcud/qemu-32609-monitor.sock -mon chardev=mon,mode=control -display none -vga none -machine virt -chardev socket,id=console,path=/var/tmp/tmpizirkcud/qemu-32609-console.sock,server,nowait -serial chardev:console -cpu cortex-a53 -kernel /var/lib/avocado/data/cache/by_name/vmlinuz -append console=ttyAMA0'

2019-01-31 14:41:41,779 qmp              L0167 DEBUG| >>> {'execute':
'qmp_capabilities'}
2019-01-31 14:41:41,931 qmp              L0175 DEBUG| <<< {'return': {}}
2019-01-31 14:41:42,830 boot_linux_conso L0041 DEBUG| [    0.000000]

Booting Linux on physical CPU 0x0000000000 [0x410fd034]

Quoted 8 lines

(...)

2019-01-31 14:41:42,833 boot_linux_conso L0041 DEBUG| [    0.000000]
Policy zone: DMA32
2019-01-31 14:41:42,833 boot_linux_conso L0041 DEBUG| [    0.000000]
Kernel command line: console=ttyAMA0
2019-01-31 14:41:42,833 qmp              L0167 DEBUG| >>> {'execute':

'quit'}

Quoted 5 lines

Here, a QMP response like "<<< {'return': {}}" would be expected.

Since I can not reproduce this on my system (or on Travis-CI jobs I've sent), can you tell me on top of which commit you've applied these patches?

Quoted 7 lines

I spoke too soon:

https://travis-ci.org/clebergnu/qemu/jobs/487121425#L3033

This looks like a recent regression, and I'm guessing it's not on the test's side. I'll try to bisect it and let you know.

On a fresh environment, I am able to get this reproduced on every 2 of runs, more or less. When I hit it, I attached GDB to it, and the backtrace shows:

Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
warning: Loadable section ".note.gnu.property" outside of ELF segments
warning: Loadable section ".note.gnu.property" outside of ELF segments __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:103
103 2: movl %edx, %eax
(gdb) bt
#0 __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:103
#1 0x00007fc6ba1a2e09 in __GI___pthread_mutex_lock (mutex=mutex@entry=0x5615a233d020 <qemu_global_mutex>) at ../nptl/pthread_mutex_lock.c:80
#2 0x00005615a1bb7593 in qemu_mutex_lock_impl (mutex=0x5615a233d020 <qemu_global_mutex>, file=0x5615a1db2d4c "util/main-loop.c", line=236) at util/qemu-thread-posix.c:66
#3 0x00005615a171125e in qemu_mutex_lock_iothread_impl (file=file@entry=0x5615a1db2d4c "util/main-loop.c", line=line@entry=236) at /home/cleber/src/qemu/cpus.c:1849
#4 0x00005615a1bb415d in os_host_main_loop_wait (timeout=<optimized out>) at util/main-loop.c:236

#5  main_loop_wait (nonblocking=<optimized out>) at util/main-loop.c:497
#6  0x00005615a18fdd39 in main_loop () at vl.c:1928
#7  0x00005615a16c9ee9 in main (argc=<optimized out>, argv=<optimized

out>, envp=<optimized out>) at vl.c:4665

Running it with taskset -c 1 prevents this issue from happening, which AFAICT, contributes even further towards this being a QEMU race condition.

I'm CC'ing Peter and Claudio (listed maintainers of aarch64), as this seems to limited to that target. Any tips on what to do here?

Thanks,
- Cleber.

Quoted 3 lines

Thanks,
- Cleber.

#68 lore raw

Re: [Qemu-devel] [PATCH 15/18] Boot Linux Console Test: add a test for aarch64 + virt

Quoted 139 lines · Cleber Rosa
Quoted 104 lines · Cleber Rosa
Quoted 95 lines · Cleber Rosa
Quoted 83 lines · Wainer dos Santos Moschetta
Quoted 34 lines · Cleber Rosa

Just like the previous tests, boots a Linux kernel on a aarch64 target using the virt machine.

One special option added is the CPU type, given that the kernel selected fails to boot on the virt machine's default CPU (cortex-a15).

---
  .travis.yml                            |  2 +- tests/acceptance/boot_linux_console.py | 20 ++++++++++++++++++++

  2 files changed, 21 insertions(+), 1 deletion(-)
.travis.yml +0 -0
index 54100eea5a..595e8c0b6c 100644
@@ -187,7 +187,7 @@ matrix:
         # Acceptance (Functional) tests

      - env:
-        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu"
+        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu"

          - TEST_CMD="make check-acceptance"
        addons:
          apt:

tests/acceptance/boot_linux_console.py +0 -0
index f3ccd23a7a..107700b517 100644
@@ -138,3 +138,23 @@ class BootLinuxConsole(Test):
           self.vm.launch()
Quoted 5 lines

          console_pattern = 'Kernel command line: %s' % kernel_command_line self.wait_for_console_pattern(console_pattern) +
+    def test_aarch64_virt(self):

That test case fails on my system (Fedora 29 x86_64). Avocado seems unable to kill the VM so it  reaches the timeout.

I compiled QEMU with default configuration:

$ configure --python=/usr/bin/python3 --target-list=x86_64-softmmu
--target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu)

Follows a snippet of the Avocado's job.log file:
----

2019-01-31 14:41:34,912 test             L0602 INFO | START

07-/root/src/qemu/tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_aarch64_virt

2019-01-31 14:41:34,912 test             L0298 DEBUG| DATA
(filename=output.expected) => NOT FOUND (data sources: variant, test, file)
2019-01-31 14:41:34,913 parameters       L0146 DEBUG| PARAMS (key=arch,
path=*, default=aarch64) => 'aarch64'
2019-01-31 14:41:34,913 parameters       L0146 DEBUG| PARAMS
(key=qemu_bin, path=*, default=aarch64-softmmu/qemu-system-aarch64) =>
'aarch64-softmmu/qemu-system-aarch64'
2019-01-31 14:41:34,915 download         L0070 INFO | Fetching
https://sjc.edge.kernel.org/fedora-buffet/fedora/linux/releases/29/Server/aarch64/os/images/pxeboot/vmlinuz
-> /var/lib/avocado/data/cache/by_name/vmlinuz.3upct2pr
2019-01-31 14:41:35,490 download         L0054 DEBUG| Retrieved URL
"https://sjc.edge.kernel.org/fedora-buffet/fedora/linux/releases/29/Server/aarch64/os/images/pxeboot/vmlinuz":
content-length 8623423, date: "Thu, 31 Jan 2019 19:41:35 GMT",
last-modified: "Sun, 21 Oct 2018 00:43:09 GMT"
2019-01-31 14:41:41,765 qemu             L0317 DEBUG| VM launch command:

'aarch64-softmmu/qemu-system-aarch64 -chardev socket,id=mon,path=/var/tmp/tmpizirkcud/qemu-32609-monitor.sock -mon chardev=mon,mode=control -display none -vga none -machine virt -chardev socket,id=console,path=/var/tmp/tmpizirkcud/qemu-32609-console.sock,server,nowait -serial chardev:console -cpu cortex-a53 -kernel /var/lib/avocado/data/cache/by_name/vmlinuz -append console=ttyAMA0'

2019-01-31 14:41:41,779 qmp              L0167 DEBUG| >>> {'execute':
'qmp_capabilities'}
2019-01-31 14:41:41,931 qmp              L0175 DEBUG| <<< {'return': {}}
2019-01-31 14:41:42,830 boot_linux_conso L0041 DEBUG| [    0.000000]

Booting Linux on physical CPU 0x0000000000 [0x410fd034]

Quoted 8 lines

(...)

2019-01-31 14:41:42,833 boot_linux_conso L0041 DEBUG| [    0.000000]
Policy zone: DMA32
2019-01-31 14:41:42,833 boot_linux_conso L0041 DEBUG| [    0.000000]
Kernel command line: console=ttyAMA0
2019-01-31 14:41:42,833 qmp              L0167 DEBUG| >>> {'execute':

'quit'}

Quoted 5 lines

Here, a QMP response like "<<< {'return': {}}" would be expected.

Since I can not reproduce this on my system (or on Travis-CI jobs I've sent), can you tell me on top of which commit you've applied these patches?

Quoted 7 lines

I spoke too soon:

https://travis-ci.org/clebergnu/qemu/jobs/487121425#L3033

This looks like a recent regression, and I'm guessing it's not on the test's side. I'll try to bisect it and let you know.

On a fresh environment, I am able to get this reproduced on every 2 of runs, more or less. When I hit it, I attached GDB to it, and the backtrace shows:

Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
warning: Loadable section ".note.gnu.property" outside of ELF segments
warning: Loadable section ".note.gnu.property" outside of ELF segments __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:103
103 2: movl %edx, %eax
(gdb) bt
#0 __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:103
#1 0x00007fc6ba1a2e09 in __GI___pthread_mutex_lock (mutex=mutex@entry=0x5615a233d020 <qemu_global_mutex>) at ../nptl/pthread_mutex_lock.c:80
#2 0x00005615a1bb7593 in qemu_mutex_lock_impl (mutex=0x5615a233d020 <qemu_global_mutex>, file=0x5615a1db2d4c "util/main-loop.c", line=236) at util/qemu-thread-posix.c:66
#3 0x00005615a171125e in qemu_mutex_lock_iothread_impl (file=file@entry=0x5615a1db2d4c "util/main-loop.c", line=line@entry=236) at /home/cleber/src/qemu/cpus.c:1849
#4 0x00005615a1bb415d in os_host_main_loop_wait (timeout=<optimized out>) at util/main-loop.c:236

Quoted 4 lines
#5  main_loop_wait (nonblocking=<optimized out>) at util/main-loop.c:497
#6  0x00005615a18fdd39 in main_loop () at vl.c:1928
#7  0x00005615a16c9ee9 in main (argc=<optimized out>, argv=<optimized

out>, envp=<optimized out>) at vl.c:4665

Tip: run "thread apply all bt" so you can get a backtrace of all threads.

Quoted 6 lines

Running it with taskset -c 1 prevents this issue from happening, which AFAICT, contributes even further towards this being a QEMU race condition.

I'm CC'ing Peter and Claudio (listed maintainers of aarch64), as this seems to limited to that target. Any tips on what to do here?

I am hitting this on Travis, too, and I finally could reproduce it locally,

The guest is still writing on the serial console, but nobody is reading the data on the other side. A VCPU thread is stuck inside the EAGAIN/nanosleep loop at qemu_chr_write_buffer(), holding the QEMU global lock.

Thread 4 (Thread 0x7f2e45fff700 (LWP 6461)):

#0  0x00007f2e4ec03500 in nanosleep () at /lib64/libpthread.so.0
#1  0x00007f2e4fb229d7 in g_usleep () at /lib64/libglib-2.0.so.0
#2  0x0000559a4e7ca4c9 in qemu_chr_write_buffer (s=s@entry=0x559a502d0ac0, buf=buf@entry=0x7f2e45ffdd90 "7", len=1, offset=offset@entry=0x7f2e45ffdd60, write_all=true) at chardev/char.c:115
#3  0x0000559a4e7ca78f in qemu_chr_write (s=0x559a502d0ac0, buf=buf@entry=0x7f2e45ffdd90 "7", len=len@entry=1, write_all=write_all@entry=true) at chardev/char.c:148
#4  0x0000559a4e7cc7e2 in qemu_chr_fe_write_all (be=be@entry=0x559a504b4c50, buf=buf@entry=0x7f2e45ffdd90 "7", len=len@entry=1) at chardev/char-fe.c:53
#5  0x0000559a4e58f320 in pl011_write (opaque=0x559a504b47d0, offset=0, value=55, size=<optimized out>) at hw/char/pl011.c:183
#6  0x0000559a4e325121 in memory_region_write_accessor (mr=0x559a504b4ae0, addr=0, value=<optimized out>, size=2, shift=<optimized out>, mask=<optimized out>, attrs=...) at /home/ehabkost/rh/proj/virt/qemu/memory.c:503
#7  0x0000559a4e322cd6 in access_with_adjusted_size (addr=addr@entry=0, value=value@entry=0x7f2e45ffded8, size=size@entry=2, access_size_min=<optimized out>, access_size_max=<optimized out>, access_fn=access_fn@entry=
    0x559a4e3250a0 <memory_region_write_accessor>, mr=0x559a504b4ae0, attrs=...) at /home/ehabkost/rh/proj/virt/qemu/memory.c:569

#8 0x0000559a4e32763f in memory_region_dispatch_write (mr=mr@entry=0x559a504b4ae0, addr=addr@entry=0, data=<optimized out>, data@entry=55, size=size@entry=2, attrs=...) at /home/ehabkost/rh/proj/virt/qemu/memory.c:1497 #9 0x0000559a4e338708 in io_writex (env=env@entry=0x559a503d5620, mmu_idx=mmu_idx@entry=1, val=val@entry=55, addr=addr@entry=18446462598867529728, retaddr=139836732143069, size=2, iotlbentry=<optimized out>, iotlbentry=<optimized out>)
at /home/ehabkost/rh/proj/virt/qemu/accel/tcg/cputlb.c:945
#10 0x0000559a4e33d203 in store_helper (big_endian=false, size=2, retaddr=<optimized out>, oi=<optimized out>, val=55, addr=18446462598867529728, env=0x559a503d5620) at /home/ehabkost/rh/proj/virt/qemu/accel/tcg/cputlb.c:1544 #11 0x0000559a4e33d203 in helper_le_stw_mmu (env=0x559a503d5620, addr=18446462598867529728, val=55, oi=<optimized out>, retaddr=139836732143069) at /home/ehabkost/rh/proj/virt/qemu/accel/tcg/cputlb.c:1636 #12 0x00007f2e46bef5dd in code_gen_buffer ()
#13 0x0000559a4e352381 in cpu_tb_exec (itb=<optimized out>, cpu=0x559a503d5620) at /home/ehabkost/rh/proj/virt/qemu/accel/tcg/cpu-exec.c:171
#14 0x0000559a4e352381 in cpu_loop_exec_tb (tb_exit=<synthetic pointer>, last_tb=<synthetic pointer>, tb=<optimized out>, cpu=0x559a503d5620) at /home/ehabkost/rh/proj/virt/qemu/accel/tcg/cpu-exec.c:618 #15 0x0000559a4e352381 in cpu_exec (cpu=cpu@entry=0x559a503cd360) at /home/ehabkost/rh/proj/virt/qemu/accel/tcg/cpu-exec.c:729
#16 0x0000559a4e30ea0f in tcg_cpu_exec (cpu=0x559a503cd360) at /home/ehabkost/rh/proj/virt/qemu/cpus.c:1434
#17 0x0000559a4e310b6b in qemu_tcg_cpu_thread_fn (arg=arg@entry=0x559a503cd360) at /home/ehabkost/rh/proj/virt/qemu/cpus.c:1743
#18 0x0000559a4e83669a in qemu_thread_start (args=<optimized out>) at util/qemu-thread-posix.c:502
#19 0x00007f2e4ebf958e in start_thread () at /lib64/libpthread.so.0
#20 0x00007f2e4eb266f3 in clone () at /lib64/libc.so.6

For reference, this is the QEMU command line:

aarch64-softmmu/qemu-system-aarch64 -chardev socket,id=mon,path=/var/tmp/tmpxnkcjvf0/qemu-6453-monitor.sock -mon chardev=mon,mode=control -display none -vga none -machine virt -chardev socket,id=console,path=/var/tmp/tmpxnkcjvf0/qemu-6453-console.sock,server,nowait -serial chardev:console -cpu cortex-a53 -kernel /home/ehabkost/rh/proj/virt/qemu/tests/venv/data/cache/by_location/e959d0e1dd72e77653e218e666198db1f3d0c213/vmlinuz -append printk.time=0 console=ttyAMA0

Signature · 1 line
Eduardo
#69 lore raw

Re: [Qemu-devel] [PATCH 15/18] Boot Linux Console Test: add a test for aarch64 + virt

Quoted 156 lines · Eduardo Habkost
Quoted 139 lines · Cleber Rosa
Quoted 104 lines · Cleber Rosa
Quoted 95 lines · Cleber Rosa
Quoted 83 lines · Wainer dos Santos Moschetta
Quoted 34 lines · Cleber Rosa

Just like the previous tests, boots a Linux kernel on a aarch64 target using the virt machine.

One special option added is the CPU type, given that the kernel selected fails to boot on the virt machine's default CPU (cortex-a15).

---
  .travis.yml                            |  2 +- tests/acceptance/boot_linux_console.py | 20 ++++++++++++++++++++

  2 files changed, 21 insertions(+), 1 deletion(-)
.travis.yml +0 -0
index 54100eea5a..595e8c0b6c 100644
@@ -187,7 +187,7 @@ matrix:
         # Acceptance (Functional) tests

      - env:
-        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu"
+        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu"

          - TEST_CMD="make check-acceptance"
        addons:
          apt:

tests/acceptance/boot_linux_console.py +0 -0
index f3ccd23a7a..107700b517 100644
@@ -138,3 +138,23 @@ class BootLinuxConsole(Test):
           self.vm.launch()
Quoted 5 lines

          console_pattern = 'Kernel command line: %s' % kernel_command_line self.wait_for_console_pattern(console_pattern) +
+    def test_aarch64_virt(self):

That test case fails on my system (Fedora 29 x86_64). Avocado seems unable to kill the VM so it  reaches the timeout.

I compiled QEMU with default configuration:

$ configure --python=/usr/bin/python3 --target-list=x86_64-softmmu
--target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu)

Follows a snippet of the Avocado's job.log file:
----

2019-01-31 14:41:34,912 test             L0602 INFO | START

07-/root/src/qemu/tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_aarch64_virt

2019-01-31 14:41:34,912 test             L0298 DEBUG| DATA
(filename=output.expected) => NOT FOUND (data sources: variant, test, file)
2019-01-31 14:41:34,913 parameters       L0146 DEBUG| PARAMS (key=arch,
path=*, default=aarch64) => 'aarch64'
2019-01-31 14:41:34,913 parameters       L0146 DEBUG| PARAMS
(key=qemu_bin, path=*, default=aarch64-softmmu/qemu-system-aarch64) =>
'aarch64-softmmu/qemu-system-aarch64'
2019-01-31 14:41:34,915 download         L0070 INFO | Fetching
https://sjc.edge.kernel.org/fedora-buffet/fedora/linux/releases/29/Server/aarch64/os/images/pxeboot/vmlinuz
-> /var/lib/avocado/data/cache/by_name/vmlinuz.3upct2pr
2019-01-31 14:41:35,490 download         L0054 DEBUG| Retrieved URL
"https://sjc.edge.kernel.org/fedora-buffet/fedora/linux/releases/29/Server/aarch64/os/images/pxeboot/vmlinuz":
content-length 8623423, date: "Thu, 31 Jan 2019 19:41:35 GMT",
last-modified: "Sun, 21 Oct 2018 00:43:09 GMT"
2019-01-31 14:41:41,765 qemu             L0317 DEBUG| VM launch command:

'aarch64-softmmu/qemu-system-aarch64 -chardev socket,id=mon,path=/var/tmp/tmpizirkcud/qemu-32609-monitor.sock -mon chardev=mon,mode=control -display none -vga none -machine virt -chardev socket,id=console,path=/var/tmp/tmpizirkcud/qemu-32609-console.sock,server,nowait -serial chardev:console -cpu cortex-a53 -kernel /var/lib/avocado/data/cache/by_name/vmlinuz -append console=ttyAMA0'

2019-01-31 14:41:41,779 qmp              L0167 DEBUG| >>> {'execute':
'qmp_capabilities'}
2019-01-31 14:41:41,931 qmp              L0175 DEBUG| <<< {'return': {}}
2019-01-31 14:41:42,830 boot_linux_conso L0041 DEBUG| [    0.000000]

Booting Linux on physical CPU 0x0000000000 [0x410fd034]

Quoted 8 lines

(...)

2019-01-31 14:41:42,833 boot_linux_conso L0041 DEBUG| [    0.000000]
Policy zone: DMA32
2019-01-31 14:41:42,833 boot_linux_conso L0041 DEBUG| [    0.000000]
Kernel command line: console=ttyAMA0
2019-01-31 14:41:42,833 qmp              L0167 DEBUG| >>> {'execute':

'quit'}

Quoted 5 lines

Here, a QMP response like "<<< {'return': {}}" would be expected.

Since I can not reproduce this on my system (or on Travis-CI jobs I've sent), can you tell me on top of which commit you've applied these patches?

Quoted 7 lines

I spoke too soon:

https://travis-ci.org/clebergnu/qemu/jobs/487121425#L3033

This looks like a recent regression, and I'm guessing it's not on the test's side. I'll try to bisect it and let you know.

On a fresh environment, I am able to get this reproduced on every 2 of runs, more or less. When I hit it, I attached GDB to it, and the backtrace shows:

Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
warning: Loadable section ".note.gnu.property" outside of ELF segments
warning: Loadable section ".note.gnu.property" outside of ELF segments __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:103
103 2: movl %edx, %eax
(gdb) bt
#0 __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:103
#1 0x00007fc6ba1a2e09 in __GI___pthread_mutex_lock (mutex=mutex@entry=0x5615a233d020 <qemu_global_mutex>) at ../nptl/pthread_mutex_lock.c:80
#2 0x00005615a1bb7593 in qemu_mutex_lock_impl (mutex=0x5615a233d020 <qemu_global_mutex>, file=0x5615a1db2d4c "util/main-loop.c", line=236) at util/qemu-thread-posix.c:66
#3 0x00005615a171125e in qemu_mutex_lock_iothread_impl (file=file@entry=0x5615a1db2d4c "util/main-loop.c", line=line@entry=236) at /home/cleber/src/qemu/cpus.c:1849
#4 0x00005615a1bb415d in os_host_main_loop_wait (timeout=<optimized out>) at util/main-loop.c:236

Quoted 4 lines
#5  main_loop_wait (nonblocking=<optimized out>) at util/main-loop.c:497
#6  0x00005615a18fdd39 in main_loop () at vl.c:1928
#7  0x00005615a16c9ee9 in main (argc=<optimized out>, argv=<optimized

out>, envp=<optimized out>) at vl.c:4665

Tip: run "thread apply all bt" so you can get a backtrace of all threads.

Quoted 6 lines

Running it with taskset -c 1 prevents this issue from happening, which AFAICT, contributes even further towards this being a QEMU race condition.

I'm CC'ing Peter and Claudio (listed maintainers of aarch64), as this seems to limited to that target. Any tips on what to do here?

Quoted 7 lines

I am hitting this on Travis, too, and I finally could reproduce it locally,

The guest is still writing on the serial console, but nobody is reading the data on the other side. A VCPU thread is stuck inside the EAGAIN/nanosleep loop at qemu_chr_write_buffer(), holding the QEMU global lock.

Experimental fix below.

---
 python/qemu/__init__.py | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)
python/qemu/__init__.py +8 -4
index 81d9657ec0..4a691f34da 100644
@@ -274,10 +274,6 @@ class QEMUMachine(object):
 
         self._qemu_log_path = None
 
-        if self._console_socket is not None:
-            self._console_socket.close()
-            self._console_socket = None
-
         if self._temp_dir is not None:
             shutil.rmtree(self._temp_dir)
             self._temp_dir = None
@@ -336,6 +332,14 @@ class QEMUMachine(object):
         """
         Terminate the VM and clean up
         """
+
+        # If we keep the console socket open, we may deadlock waiting
+        # for QEMU to exit, while QEMU is waiting for the socket to
+        # become writeable.
+        if self._console_socket is not None:
+            self._console_socket.close()
+            self._console_socket = None
+
         if self.is_running():
             try:
                 self._qmp.cmd('quit')
Signature · 4 lines
2.18.0.rc1.1.g3f1ff2140

-- 
Eduardo
#70 lore raw

Re: [Qemu-devel] [PATCH 15/18] Boot Linux Console Test: add a test for aarch64 + virt

Quoted 189 lines · Eduardo Habkost
Quoted 156 lines · Eduardo Habkost
Quoted 139 lines · Cleber Rosa
Quoted 104 lines · Cleber Rosa
Quoted 95 lines · Cleber Rosa
Quoted 83 lines · Wainer dos Santos Moschetta
Quoted 34 lines · Cleber Rosa

Just like the previous tests, boots a Linux kernel on a aarch64 target using the virt machine.

One special option added is the CPU type, given that the kernel selected fails to boot on the virt machine's default CPU (cortex-a15).

---
  .travis.yml                            |  2 +- tests/acceptance/boot_linux_console.py | 20 ++++++++++++++++++++

  2 files changed, 21 insertions(+), 1 deletion(-)
.travis.yml +0 -0
index 54100eea5a..595e8c0b6c 100644
@@ -187,7 +187,7 @@ matrix:
         # Acceptance (Functional) tests

      - env:
-        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu"
+        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu"

          - TEST_CMD="make check-acceptance"
        addons:
          apt:

tests/acceptance/boot_linux_console.py +0 -0
index f3ccd23a7a..107700b517 100644
@@ -138,3 +138,23 @@ class BootLinuxConsole(Test):
           self.vm.launch()
Quoted 5 lines

          console_pattern = 'Kernel command line: %s' % kernel_command_line self.wait_for_console_pattern(console_pattern) +
+    def test_aarch64_virt(self):

That test case fails on my system (Fedora 29 x86_64). Avocado seems unable to kill the VM so it  reaches the timeout.

I compiled QEMU with default configuration:

$ configure --python=/usr/bin/python3 --target-list=x86_64-softmmu
--target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu)

Follows a snippet of the Avocado's job.log file:
----

2019-01-31 14:41:34,912 test             L0602 INFO | START

07-/root/src/qemu/tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_aarch64_virt

2019-01-31 14:41:34,912 test             L0298 DEBUG| DATA
(filename=output.expected) => NOT FOUND (data sources: variant, test, file)
2019-01-31 14:41:34,913 parameters       L0146 DEBUG| PARAMS (key=arch,
path=*, default=aarch64) => 'aarch64'
2019-01-31 14:41:34,913 parameters       L0146 DEBUG| PARAMS
(key=qemu_bin, path=*, default=aarch64-softmmu/qemu-system-aarch64) =>
'aarch64-softmmu/qemu-system-aarch64'
2019-01-31 14:41:34,915 download         L0070 INFO | Fetching
https://sjc.edge.kernel.org/fedora-buffet/fedora/linux/releases/29/Server/aarch64/os/images/pxeboot/vmlinuz
-> /var/lib/avocado/data/cache/by_name/vmlinuz.3upct2pr
2019-01-31 14:41:35,490 download         L0054 DEBUG| Retrieved URL
"https://sjc.edge.kernel.org/fedora-buffet/fedora/linux/releases/29/Server/aarch64/os/images/pxeboot/vmlinuz":
content-length 8623423, date: "Thu, 31 Jan 2019 19:41:35 GMT",
last-modified: "Sun, 21 Oct 2018 00:43:09 GMT"
2019-01-31 14:41:41,765 qemu             L0317 DEBUG| VM launch command:

'aarch64-softmmu/qemu-system-aarch64 -chardev socket,id=mon,path=/var/tmp/tmpizirkcud/qemu-32609-monitor.sock -mon chardev=mon,mode=control -display none -vga none -machine virt -chardev socket,id=console,path=/var/tmp/tmpizirkcud/qemu-32609-console.sock,server,nowait -serial chardev:console -cpu cortex-a53 -kernel /var/lib/avocado/data/cache/by_name/vmlinuz -append console=ttyAMA0'

2019-01-31 14:41:41,779 qmp              L0167 DEBUG| >>> {'execute':
'qmp_capabilities'}
2019-01-31 14:41:41,931 qmp              L0175 DEBUG| <<< {'return': {}}
2019-01-31 14:41:42,830 boot_linux_conso L0041 DEBUG| [    0.000000]

Booting Linux on physical CPU 0x0000000000 [0x410fd034]

Quoted 8 lines

(...)

2019-01-31 14:41:42,833 boot_linux_conso L0041 DEBUG| [    0.000000]
Policy zone: DMA32
2019-01-31 14:41:42,833 boot_linux_conso L0041 DEBUG| [    0.000000]
Kernel command line: console=ttyAMA0
2019-01-31 14:41:42,833 qmp              L0167 DEBUG| >>> {'execute':

'quit'}

Quoted 5 lines

Here, a QMP response like "<<< {'return': {}}" would be expected.

Since I can not reproduce this on my system (or on Travis-CI jobs I've sent), can you tell me on top of which commit you've applied these patches?

Quoted 7 lines

I spoke too soon:

https://travis-ci.org/clebergnu/qemu/jobs/487121425#L3033

This looks like a recent regression, and I'm guessing it's not on the test's side. I'll try to bisect it and let you know.

On a fresh environment, I am able to get this reproduced on every 2 of runs, more or less. When I hit it, I attached GDB to it, and the backtrace shows:

Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
warning: Loadable section ".note.gnu.property" outside of ELF segments
warning: Loadable section ".note.gnu.property" outside of ELF segments __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:103
103 2: movl %edx, %eax
(gdb) bt
#0 __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:103
#1 0x00007fc6ba1a2e09 in __GI___pthread_mutex_lock (mutex=mutex@entry=0x5615a233d020 <qemu_global_mutex>) at ../nptl/pthread_mutex_lock.c:80
#2 0x00005615a1bb7593 in qemu_mutex_lock_impl (mutex=0x5615a233d020 <qemu_global_mutex>, file=0x5615a1db2d4c "util/main-loop.c", line=236) at util/qemu-thread-posix.c:66
#3 0x00005615a171125e in qemu_mutex_lock_iothread_impl (file=file@entry=0x5615a1db2d4c "util/main-loop.c", line=line@entry=236) at /home/cleber/src/qemu/cpus.c:1849
#4 0x00005615a1bb415d in os_host_main_loop_wait (timeout=<optimized out>) at util/main-loop.c:236

Quoted 4 lines
#5  main_loop_wait (nonblocking=<optimized out>) at util/main-loop.c:497
#6  0x00005615a18fdd39 in main_loop () at vl.c:1928
#7  0x00005615a16c9ee9 in main (argc=<optimized out>, argv=<optimized

out>, envp=<optimized out>) at vl.c:4665

Tip: run "thread apply all bt" so you can get a backtrace of all threads.

Quoted 6 lines

Running it with taskset -c 1 prevents this issue from happening, which AFAICT, contributes even further towards this being a QEMU race condition.

I'm CC'ing Peter and Claudio (listed maintainers of aarch64), as this seems to limited to that target. Any tips on what to do here?

Quoted 7 lines

I am hitting this on Travis, too, and I finally could reproduce it locally,

The guest is still writing on the serial console, but nobody is reading the data on the other side. A VCPU thread is stuck inside the EAGAIN/nanosleep loop at qemu_chr_write_buffer(), holding the QEMU global lock.

Experimental fix below.

---
 python/qemu/__init__.py | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)
python/qemu/__init__.py +0 -4
index 81d9657ec0..4a691f34da 100644
@@ -274,10 +274,6 @@ class QEMUMachine(object):
 
         self._qemu_log_path = None
 
-        if self._console_socket is not None:
-            self._console_socket.close()
-            self._console_socket = None
-
         if self._temp_dir is not None:
             shutil.rmtree(self._temp_dir)
             self._temp_dir = None
@@ -336,6 +332,14 @@ class QEMUMachine(object):
         """

python/qemu/__init__.py class QEMUMachine(object): · line 342

Quoted 10 lines
python/qemu/__init__.py +8 -0
         Terminate the VM and clean up
         """
+
+        # If we keep the console socket open, we may deadlock waiting
+        # for QEMU to exit, while QEMU is waiting for the socket to
+        # become writeable.
+        if self._console_socket is not None:
+            self._console_socket.close()
+            self._console_socket = None
+

Right, this is somewhat equivalent to the following "workaround":

https://github.com/clebergnu/qemu/commit/e1713f3b91972ad57c089f276c54db3f3fa63423

I could not tell at the moment, though, if closing (or shutting down) the console socket was the appropriate fix.

What I see is that Rick's commit pointed to by Lazlo is dated from 2016, and it says that a virtio-console fix is necessary. I'd imagine that, if a fix was ever proposed, it'd have been incorporated in the F29 kernel used in this test... and that this workaround/fix would not be the best solution.

But, I'm mostly attempting to navigate this using my senses , few hard data and even less background :).

Regards,
- Cleber.

Quoted 8 lines
python/qemu/__init__.py +0 -0
         if self.is_running():
             try:
                 self._qmp.cmd('quit')
Signature · 4 lines
2.18.0.rc1.1.g3f1ff2140

-- 
Eduardo
#71 lore raw

Re: [Qemu-devel] [PATCH 15/18] Boot Linux Console Test: add a test for aarch64 + virt

CCing Daniel, who wrote commit 6ab3fc32ea64.

Quoted 205 lines · Cleber Rosa
Quoted 189 lines · Eduardo Habkost
Quoted 156 lines · Eduardo Habkost
Quoted 139 lines · Cleber Rosa
Quoted 104 lines · Cleber Rosa
Quoted 95 lines · Cleber Rosa
Quoted 83 lines · Wainer dos Santos Moschetta
Quoted 34 lines · Cleber Rosa

Just like the previous tests, boots a Linux kernel on a aarch64 target using the virt machine.

One special option added is the CPU type, given that the kernel selected fails to boot on the virt machine's default CPU (cortex-a15).

---
  .travis.yml                            |  2 +- tests/acceptance/boot_linux_console.py | 20 ++++++++++++++++++++

  2 files changed, 21 insertions(+), 1 deletion(-)
.travis.yml +0 -0
index 54100eea5a..595e8c0b6c 100644
@@ -187,7 +187,7 @@ matrix:
         # Acceptance (Functional) tests

      - env:
-        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu"
+        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu"

          - TEST_CMD="make check-acceptance"
        addons:
          apt:

tests/acceptance/boot_linux_console.py +0 -0
index f3ccd23a7a..107700b517 100644
@@ -138,3 +138,23 @@ class BootLinuxConsole(Test):
           self.vm.launch()
Quoted 5 lines

          console_pattern = 'Kernel command line: %s' % kernel_command_line self.wait_for_console_pattern(console_pattern) +
+    def test_aarch64_virt(self):

That test case fails on my system (Fedora 29 x86_64). Avocado seems unable to kill the VM so it  reaches the timeout.

I compiled QEMU with default configuration:

$ configure --python=/usr/bin/python3 --target-list=x86_64-softmmu
--target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu)

Follows a snippet of the Avocado's job.log file:
----

2019-01-31 14:41:34,912 test             L0602 INFO | START

07-/root/src/qemu/tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_aarch64_virt

2019-01-31 14:41:34,912 test             L0298 DEBUG| DATA
(filename=output.expected) => NOT FOUND (data sources: variant, test, file)
2019-01-31 14:41:34,913 parameters       L0146 DEBUG| PARAMS (key=arch,
path=*, default=aarch64) => 'aarch64'
2019-01-31 14:41:34,913 parameters       L0146 DEBUG| PARAMS
(key=qemu_bin, path=*, default=aarch64-softmmu/qemu-system-aarch64) =>
'aarch64-softmmu/qemu-system-aarch64'
2019-01-31 14:41:34,915 download         L0070 INFO | Fetching
https://sjc.edge.kernel.org/fedora-buffet/fedora/linux/releases/29/Server/aarch64/os/images/pxeboot/vmlinuz
-> /var/lib/avocado/data/cache/by_name/vmlinuz.3upct2pr
2019-01-31 14:41:35,490 download         L0054 DEBUG| Retrieved URL
"https://sjc.edge.kernel.org/fedora-buffet/fedora/linux/releases/29/Server/aarch64/os/images/pxeboot/vmlinuz":
content-length 8623423, date: "Thu, 31 Jan 2019 19:41:35 GMT",
last-modified: "Sun, 21 Oct 2018 00:43:09 GMT"
2019-01-31 14:41:41,765 qemu             L0317 DEBUG| VM launch command:

'aarch64-softmmu/qemu-system-aarch64 -chardev socket,id=mon,path=/var/tmp/tmpizirkcud/qemu-32609-monitor.sock -mon chardev=mon,mode=control -display none -vga none -machine virt -chardev socket,id=console,path=/var/tmp/tmpizirkcud/qemu-32609-console.sock,server,nowait -serial chardev:console -cpu cortex-a53 -kernel /var/lib/avocado/data/cache/by_name/vmlinuz -append console=ttyAMA0'

2019-01-31 14:41:41,779 qmp              L0167 DEBUG| >>> {'execute':
'qmp_capabilities'}
2019-01-31 14:41:41,931 qmp              L0175 DEBUG| <<< {'return': {}}
2019-01-31 14:41:42,830 boot_linux_conso L0041 DEBUG| [    0.000000]

Booting Linux on physical CPU 0x0000000000 [0x410fd034]

Quoted 8 lines

(...)

2019-01-31 14:41:42,833 boot_linux_conso L0041 DEBUG| [    0.000000]
Policy zone: DMA32
2019-01-31 14:41:42,833 boot_linux_conso L0041 DEBUG| [    0.000000]
Kernel command line: console=ttyAMA0
2019-01-31 14:41:42,833 qmp              L0167 DEBUG| >>> {'execute':

'quit'}

Quoted 5 lines

Here, a QMP response like "<<< {'return': {}}" would be expected.

Since I can not reproduce this on my system (or on Travis-CI jobs I've sent), can you tell me on top of which commit you've applied these patches?

Quoted 7 lines

I spoke too soon:

https://travis-ci.org/clebergnu/qemu/jobs/487121425#L3033

This looks like a recent regression, and I'm guessing it's not on the test's side. I'll try to bisect it and let you know.

On a fresh environment, I am able to get this reproduced on every 2 of runs, more or less. When I hit it, I attached GDB to it, and the backtrace shows:

Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
warning: Loadable section ".note.gnu.property" outside of ELF segments
warning: Loadable section ".note.gnu.property" outside of ELF segments __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:103
103 2: movl %edx, %eax
(gdb) bt
#0 __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:103
#1 0x00007fc6ba1a2e09 in __GI___pthread_mutex_lock (mutex=mutex@entry=0x5615a233d020 <qemu_global_mutex>) at ../nptl/pthread_mutex_lock.c:80
#2 0x00005615a1bb7593 in qemu_mutex_lock_impl (mutex=0x5615a233d020 <qemu_global_mutex>, file=0x5615a1db2d4c "util/main-loop.c", line=236) at util/qemu-thread-posix.c:66
#3 0x00005615a171125e in qemu_mutex_lock_iothread_impl (file=file@entry=0x5615a1db2d4c "util/main-loop.c", line=line@entry=236) at /home/cleber/src/qemu/cpus.c:1849
#4 0x00005615a1bb415d in os_host_main_loop_wait (timeout=<optimized out>) at util/main-loop.c:236

Quoted 4 lines
#5  main_loop_wait (nonblocking=<optimized out>) at util/main-loop.c:497
#6  0x00005615a18fdd39 in main_loop () at vl.c:1928
#7  0x00005615a16c9ee9 in main (argc=<optimized out>, argv=<optimized

out>, envp=<optimized out>) at vl.c:4665

Tip: run "thread apply all bt" so you can get a backtrace of all threads.

Quoted 6 lines

Running it with taskset -c 1 prevents this issue from happening, which AFAICT, contributes even further towards this being a QEMU race condition.

I'm CC'ing Peter and Claudio (listed maintainers of aarch64), as this seems to limited to that target. Any tips on what to do here?

Quoted 7 lines

I am hitting this on Travis, too, and I finally could reproduce it locally,

The guest is still writing on the serial console, but nobody is reading the data on the other side. A VCPU thread is stuck inside the EAGAIN/nanosleep loop at qemu_chr_write_buffer(), holding the QEMU global lock.

Experimental fix below.

---
 python/qemu/__init__.py | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)
python/qemu/__init__.py +0 -4
index 81d9657ec0..4a691f34da 100644
@@ -274,10 +274,6 @@ class QEMUMachine(object):
 
         self._qemu_log_path = None
 
-        if self._console_socket is not None:
-            self._console_socket.close()
-            self._console_socket = None
-
         if self._temp_dir is not None:
             shutil.rmtree(self._temp_dir)
             self._temp_dir = None
@@ -336,6 +332,14 @@ class QEMUMachine(object):
         """

python/qemu/__init__.py class QEMUMachine(object): · line 342

Quoted 10 lines
python/qemu/__init__.py +8 -0
         Terminate the VM and clean up
         """
+
+        # If we keep the console socket open, we may deadlock waiting
+        # for QEMU to exit, while QEMU is waiting for the socket to
+        # become writeable.
+        if self._console_socket is not None:
+            self._console_socket.close()
+            self._console_socket = None
+

Right, this is somewhat equivalent to the following "workaround":

   https://github.com/clebergnu/qemu/commit/e1713f3b91972ad57c089f276c54db3f3fa63423
Quoted 8 lines

I could not tell at the moment, though, if closing (or shutting down) the console socket was the appropriate fix.

What I see is that Rick's commit pointed to by Lazlo is dated from 2016, and it says that a virtio-console fix is necessary. I'd imagine that, if a fix was ever proposed, it'd have been incorporated in the F29 kernel used in this test... and that this workaround/fix would not be the best solution.

If I understood this correctly, fixing the guest driver wouldn't help here. The commit mentioned by Laszlo (6ab3fc32ea64 "hw:
replace most use of qemu_chr_fe_write with qemu_chr_fe_write_all") fixes data loss bugs but introduces the potential for deadlocks like the one we are seeing.

Unless we replace the existing ~30 qemu_chr_fe_write_all() calls in device code, we need to make sure we are constantly reading data from the console socket.

Quoted 15 lines

But, I'm mostly attempting to navigate this using my senses , few hard data and even less background :).

Regards,
- Cleber.

Quoted 8 lines
python/qemu/__init__.py +0 -0
         if self.is_running():
             try:
                 self._qmp.cmd('quit')
Signature · 4 lines
2.18.0.rc1.1.g3f1ff2140

-- 
Eduardo
Signature · 1 line
Eduardo
#72 lore raw

Re: [Qemu-devel] [PATCH 15/18] Boot Linux Console Test: add a test for aarch64 + virt

Quoted 223 lines · Eduardo Habkost

CCing Daniel, who wrote commit 6ab3fc32ea64.

Quoted 205 lines · Cleber Rosa
Quoted 189 lines · Eduardo Habkost
Quoted 156 lines · Eduardo Habkost
Quoted 139 lines · Cleber Rosa
Quoted 104 lines · Cleber Rosa
Quoted 95 lines · Cleber Rosa
Quoted 83 lines · Wainer dos Santos Moschetta
Quoted 34 lines · Cleber Rosa

Just like the previous tests, boots a Linux kernel on a aarch64 target using the virt machine.

One special option added is the CPU type, given that the kernel selected fails to boot on the virt machine's default CPU (cortex-a15).

---
  .travis.yml                            |  2 +- tests/acceptance/boot_linux_console.py | 20 ++++++++++++++++++++

  2 files changed, 21 insertions(+), 1 deletion(-)
.travis.yml +0 -0
index 54100eea5a..595e8c0b6c 100644
@@ -187,7 +187,7 @@ matrix:
         # Acceptance (Functional) tests

      - env:
-        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu"
+        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu"

          - TEST_CMD="make check-acceptance"
        addons:
          apt:

tests/acceptance/boot_linux_console.py +0 -0
index f3ccd23a7a..107700b517 100644
@@ -138,3 +138,23 @@ class BootLinuxConsole(Test):
           self.vm.launch()
Quoted 5 lines

          console_pattern = 'Kernel command line: %s' % kernel_command_line self.wait_for_console_pattern(console_pattern) +
+    def test_aarch64_virt(self):

That test case fails on my system (Fedora 29 x86_64). Avocado seems unable to kill the VM so it  reaches the timeout.

I compiled QEMU with default configuration:

$ configure --python=/usr/bin/python3 --target-list=x86_64-softmmu
--target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu)

Follows a snippet of the Avocado's job.log file:
----

2019-01-31 14:41:34,912 test             L0602 INFO | START

07-/root/src/qemu/tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_aarch64_virt

2019-01-31 14:41:34,912 test             L0298 DEBUG| DATA
(filename=output.expected) => NOT FOUND (data sources: variant, test, file)
2019-01-31 14:41:34,913 parameters       L0146 DEBUG| PARAMS (key=arch,
path=*, default=aarch64) => 'aarch64'
2019-01-31 14:41:34,913 parameters       L0146 DEBUG| PARAMS
(key=qemu_bin, path=*, default=aarch64-softmmu/qemu-system-aarch64) =>
'aarch64-softmmu/qemu-system-aarch64'
2019-01-31 14:41:34,915 download         L0070 INFO | Fetching
https://sjc.edge.kernel.org/fedora-buffet/fedora/linux/releases/29/Server/aarch64/os/images/pxeboot/vmlinuz
-> /var/lib/avocado/data/cache/by_name/vmlinuz.3upct2pr
2019-01-31 14:41:35,490 download         L0054 DEBUG| Retrieved URL
"https://sjc.edge.kernel.org/fedora-buffet/fedora/linux/releases/29/Server/aarch64/os/images/pxeboot/vmlinuz":
content-length 8623423, date: "Thu, 31 Jan 2019 19:41:35 GMT",
last-modified: "Sun, 21 Oct 2018 00:43:09 GMT"
2019-01-31 14:41:41,765 qemu             L0317 DEBUG| VM launch command:

'aarch64-softmmu/qemu-system-aarch64 -chardev socket,id=mon,path=/var/tmp/tmpizirkcud/qemu-32609-monitor.sock -mon chardev=mon,mode=control -display none -vga none -machine virt -chardev socket,id=console,path=/var/tmp/tmpizirkcud/qemu-32609-console.sock,server,nowait -serial chardev:console -cpu cortex-a53 -kernel /var/lib/avocado/data/cache/by_name/vmlinuz -append console=ttyAMA0'

2019-01-31 14:41:41,779 qmp              L0167 DEBUG| >>> {'execute':
'qmp_capabilities'}
2019-01-31 14:41:41,931 qmp              L0175 DEBUG| <<< {'return': {}}
2019-01-31 14:41:42,830 boot_linux_conso L0041 DEBUG| [    0.000000]

Booting Linux on physical CPU 0x0000000000 [0x410fd034]

Quoted 8 lines

(...)

2019-01-31 14:41:42,833 boot_linux_conso L0041 DEBUG| [    0.000000]
Policy zone: DMA32
2019-01-31 14:41:42,833 boot_linux_conso L0041 DEBUG| [    0.000000]
Kernel command line: console=ttyAMA0
2019-01-31 14:41:42,833 qmp              L0167 DEBUG| >>> {'execute':

'quit'}

Quoted 5 lines

Here, a QMP response like "<<< {'return': {}}" would be expected.

Since I can not reproduce this on my system (or on Travis-CI jobs I've sent), can you tell me on top of which commit you've applied these patches?

Quoted 7 lines

I spoke too soon:

https://travis-ci.org/clebergnu/qemu/jobs/487121425#L3033

This looks like a recent regression, and I'm guessing it's not on the test's side. I'll try to bisect it and let you know.

On a fresh environment, I am able to get this reproduced on every 2 of runs, more or less. When I hit it, I attached GDB to it, and the backtrace shows:

Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
warning: Loadable section ".note.gnu.property" outside of ELF segments
warning: Loadable section ".note.gnu.property" outside of ELF segments __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:103
103 2: movl %edx, %eax
(gdb) bt
#0 __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:103
#1 0x00007fc6ba1a2e09 in __GI___pthread_mutex_lock (mutex=mutex@entry=0x5615a233d020 <qemu_global_mutex>) at ../nptl/pthread_mutex_lock.c:80
#2 0x00005615a1bb7593 in qemu_mutex_lock_impl (mutex=0x5615a233d020 <qemu_global_mutex>, file=0x5615a1db2d4c "util/main-loop.c", line=236) at util/qemu-thread-posix.c:66
#3 0x00005615a171125e in qemu_mutex_lock_iothread_impl (file=file@entry=0x5615a1db2d4c "util/main-loop.c", line=line@entry=236) at /home/cleber/src/qemu/cpus.c:1849
#4 0x00005615a1bb415d in os_host_main_loop_wait (timeout=<optimized out>) at util/main-loop.c:236

Quoted 4 lines
#5  main_loop_wait (nonblocking=<optimized out>) at util/main-loop.c:497
#6  0x00005615a18fdd39 in main_loop () at vl.c:1928
#7  0x00005615a16c9ee9 in main (argc=<optimized out>, argv=<optimized

out>, envp=<optimized out>) at vl.c:4665

Tip: run "thread apply all bt" so you can get a backtrace of all threads.

Quoted 6 lines

Running it with taskset -c 1 prevents this issue from happening, which AFAICT, contributes even further towards this being a QEMU race condition.

I'm CC'ing Peter and Claudio (listed maintainers of aarch64), as this seems to limited to that target. Any tips on what to do here?

Quoted 7 lines

I am hitting this on Travis, too, and I finally could reproduce it locally,

The guest is still writing on the serial console, but nobody is reading the data on the other side. A VCPU thread is stuck inside the EAGAIN/nanosleep loop at qemu_chr_write_buffer(), holding the QEMU global lock.

Experimental fix below.

---
 python/qemu/__init__.py | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)
python/qemu/__init__.py +0 -4
index 81d9657ec0..4a691f34da 100644
@@ -274,10 +274,6 @@ class QEMUMachine(object):
 
         self._qemu_log_path = None
 
-        if self._console_socket is not None:
-            self._console_socket.close()
-            self._console_socket = None
-
         if self._temp_dir is not None:
             shutil.rmtree(self._temp_dir)
             self._temp_dir = None
@@ -336,6 +332,14 @@ class QEMUMachine(object):
         """

python/qemu/__init__.py class QEMUMachine(object): · line 342

Quoted 10 lines
python/qemu/__init__.py +8 -0
         Terminate the VM and clean up
         """
+
+        # If we keep the console socket open, we may deadlock waiting
+        # for QEMU to exit, while QEMU is waiting for the socket to
+        # become writeable.
+        if self._console_socket is not None:
+            self._console_socket.close()
+            self._console_socket = None
+

Right, this is somewhat equivalent to the following "workaround":

   https://github.com/clebergnu/qemu/commit/e1713f3b91972ad57c089f276c54db3f3fa63423
Quoted 8 lines

I could not tell at the moment, though, if closing (or shutting down) the console socket was the appropriate fix.

What I see is that Rick's commit pointed to by Lazlo is dated from 2016, and it says that a virtio-console fix is necessary. I'd imagine that, if a fix was ever proposed, it'd have been incorporated in the F29 kernel used in this test... and that this workaround/fix would not be the best solution.

If I understood this correctly, fixing the guest driver wouldn't help here. The commit mentioned by Laszlo (6ab3fc32ea64 "hw:
replace most use of qemu_chr_fe_write with qemu_chr_fe_write_all") fixes data loss bugs but introduces the potential for deadlocks like the one we are seeing.

Quoted 3 lines

Unless we replace the existing ~30 qemu_chr_fe_write_all() calls in device code, we need to make sure we are constantly reading data from the console socket.

Yes, you must *always* have something reading from the chardev backend. This really sucks, but it is preferrable to letting data end up in /dev/null.

If this is being a problem for tests then consider it motivation to fix the root cause problem and make the devices properly do async I/O for chardevs. Only a handful of them properly do this right now.

Regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#73 lore raw

Re: [Qemu-devel] [PATCH 15/18] Boot Linux Console Test: add a test for aarch64 + virt

Quoted 232 lines · Daniel P. Berrangé
Quoted 223 lines · Eduardo Habkost

CCing Daniel, who wrote commit 6ab3fc32ea64.

Quoted 205 lines · Cleber Rosa
Quoted 189 lines · Eduardo Habkost
Quoted 156 lines · Eduardo Habkost
Quoted 139 lines · Cleber Rosa
Quoted 104 lines · Cleber Rosa
Quoted 95 lines · Cleber Rosa
Quoted 83 lines · Wainer dos Santos Moschetta
Quoted 34 lines · Cleber Rosa

Just like the previous tests, boots a Linux kernel on a aarch64 target using the virt machine.

One special option added is the CPU type, given that the kernel selected fails to boot on the virt machine's default CPU (cortex-a15).

---
  .travis.yml                            |  2 +- tests/acceptance/boot_linux_console.py | 20 ++++++++++++++++++++

  2 files changed, 21 insertions(+), 1 deletion(-)
.travis.yml +0 -0
index 54100eea5a..595e8c0b6c 100644
@@ -187,7 +187,7 @@ matrix:
         # Acceptance (Functional) tests

      - env:
-        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu"
+        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu"

          - TEST_CMD="make check-acceptance"
        addons:
          apt:

tests/acceptance/boot_linux_console.py +0 -0
index f3ccd23a7a..107700b517 100644
@@ -138,3 +138,23 @@ class BootLinuxConsole(Test):
           self.vm.launch()
Quoted 5 lines

          console_pattern = 'Kernel command line: %s' % kernel_command_line self.wait_for_console_pattern(console_pattern) +
+    def test_aarch64_virt(self):

That test case fails on my system (Fedora 29 x86_64). Avocado seems unable to kill the VM so it  reaches the timeout.

I compiled QEMU with default configuration:

$ configure --python=/usr/bin/python3 --target-list=x86_64-softmmu
--target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu)

Follows a snippet of the Avocado's job.log file:
----

2019-01-31 14:41:34,912 test             L0602 INFO | START

07-/root/src/qemu/tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_aarch64_virt

2019-01-31 14:41:34,912 test             L0298 DEBUG| DATA
(filename=output.expected) => NOT FOUND (data sources: variant, test, file)
2019-01-31 14:41:34,913 parameters       L0146 DEBUG| PARAMS (key=arch,
path=*, default=aarch64) => 'aarch64'
2019-01-31 14:41:34,913 parameters       L0146 DEBUG| PARAMS
(key=qemu_bin, path=*, default=aarch64-softmmu/qemu-system-aarch64) =>
'aarch64-softmmu/qemu-system-aarch64'
2019-01-31 14:41:34,915 download         L0070 INFO | Fetching
https://sjc.edge.kernel.org/fedora-buffet/fedora/linux/releases/29/Server/aarch64/os/images/pxeboot/vmlinuz
-> /var/lib/avocado/data/cache/by_name/vmlinuz.3upct2pr
2019-01-31 14:41:35,490 download         L0054 DEBUG| Retrieved URL
"https://sjc.edge.kernel.org/fedora-buffet/fedora/linux/releases/29/Server/aarch64/os/images/pxeboot/vmlinuz":
content-length 8623423, date: "Thu, 31 Jan 2019 19:41:35 GMT",
last-modified: "Sun, 21 Oct 2018 00:43:09 GMT"
2019-01-31 14:41:41,765 qemu             L0317 DEBUG| VM launch command:

'aarch64-softmmu/qemu-system-aarch64 -chardev socket,id=mon,path=/var/tmp/tmpizirkcud/qemu-32609-monitor.sock -mon chardev=mon,mode=control -display none -vga none -machine virt -chardev socket,id=console,path=/var/tmp/tmpizirkcud/qemu-32609-console.sock,server,nowait -serial chardev:console -cpu cortex-a53 -kernel /var/lib/avocado/data/cache/by_name/vmlinuz -append console=ttyAMA0'

2019-01-31 14:41:41,779 qmp              L0167 DEBUG| >>> {'execute':
'qmp_capabilities'}
2019-01-31 14:41:41,931 qmp              L0175 DEBUG| <<< {'return': {}}
2019-01-31 14:41:42,830 boot_linux_conso L0041 DEBUG| [    0.000000]

Booting Linux on physical CPU 0x0000000000 [0x410fd034]

Quoted 8 lines

(...)

2019-01-31 14:41:42,833 boot_linux_conso L0041 DEBUG| [    0.000000]
Policy zone: DMA32
2019-01-31 14:41:42,833 boot_linux_conso L0041 DEBUG| [    0.000000]
Kernel command line: console=ttyAMA0
2019-01-31 14:41:42,833 qmp              L0167 DEBUG| >>> {'execute':

'quit'}

Quoted 5 lines

Here, a QMP response like "<<< {'return': {}}" would be expected.

Since I can not reproduce this on my system (or on Travis-CI jobs I've sent), can you tell me on top of which commit you've applied these patches?

Quoted 7 lines

I spoke too soon:

https://travis-ci.org/clebergnu/qemu/jobs/487121425#L3033

This looks like a recent regression, and I'm guessing it's not on the test's side. I'll try to bisect it and let you know.

On a fresh environment, I am able to get this reproduced on every 2 of runs, more or less. When I hit it, I attached GDB to it, and the backtrace shows:

Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
warning: Loadable section ".note.gnu.property" outside of ELF segments
warning: Loadable section ".note.gnu.property" outside of ELF segments __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:103
103 2: movl %edx, %eax
(gdb) bt
#0 __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:103
#1 0x00007fc6ba1a2e09 in __GI___pthread_mutex_lock (mutex=mutex@entry=0x5615a233d020 <qemu_global_mutex>) at ../nptl/pthread_mutex_lock.c:80
#2 0x00005615a1bb7593 in qemu_mutex_lock_impl (mutex=0x5615a233d020 <qemu_global_mutex>, file=0x5615a1db2d4c "util/main-loop.c", line=236) at util/qemu-thread-posix.c:66
#3 0x00005615a171125e in qemu_mutex_lock_iothread_impl (file=file@entry=0x5615a1db2d4c "util/main-loop.c", line=line@entry=236) at /home/cleber/src/qemu/cpus.c:1849
#4 0x00005615a1bb415d in os_host_main_loop_wait (timeout=<optimized out>) at util/main-loop.c:236

Quoted 4 lines
#5  main_loop_wait (nonblocking=<optimized out>) at util/main-loop.c:497
#6  0x00005615a18fdd39 in main_loop () at vl.c:1928
#7  0x00005615a16c9ee9 in main (argc=<optimized out>, argv=<optimized

out>, envp=<optimized out>) at vl.c:4665

Tip: run "thread apply all bt" so you can get a backtrace of all threads.

Quoted 6 lines

Running it with taskset -c 1 prevents this issue from happening, which AFAICT, contributes even further towards this being a QEMU race condition.

I'm CC'ing Peter and Claudio (listed maintainers of aarch64), as this seems to limited to that target. Any tips on what to do here?

Quoted 7 lines

I am hitting this on Travis, too, and I finally could reproduce it locally,

The guest is still writing on the serial console, but nobody is reading the data on the other side. A VCPU thread is stuck inside the EAGAIN/nanosleep loop at qemu_chr_write_buffer(), holding the QEMU global lock.

Experimental fix below.

---
 python/qemu/__init__.py | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)
python/qemu/__init__.py +0 -4
index 81d9657ec0..4a691f34da 100644
@@ -274,10 +274,6 @@ class QEMUMachine(object):
 
         self._qemu_log_path = None
 
-        if self._console_socket is not None:
-            self._console_socket.close()
-            self._console_socket = None
-
         if self._temp_dir is not None:
             shutil.rmtree(self._temp_dir)
             self._temp_dir = None
@@ -336,6 +332,14 @@ class QEMUMachine(object):
         """

python/qemu/__init__.py class QEMUMachine(object): · line 342

Quoted 10 lines
python/qemu/__init__.py +8 -0
         Terminate the VM and clean up
         """
+
+        # If we keep the console socket open, we may deadlock waiting
+        # for QEMU to exit, while QEMU is waiting for the socket to
+        # become writeable.
+        if self._console_socket is not None:
+            self._console_socket.close()
+            self._console_socket = None
+

Right, this is somewhat equivalent to the following "workaround":

   https://github.com/clebergnu/qemu/commit/e1713f3b91972ad57c089f276c54db3f3fa63423
Quoted 8 lines

I could not tell at the moment, though, if closing (or shutting down) the console socket was the appropriate fix.

What I see is that Rick's commit pointed to by Lazlo is dated from 2016, and it says that a virtio-console fix is necessary. I'd imagine that, if a fix was ever proposed, it'd have been incorporated in the F29 kernel used in this test... and that this workaround/fix would not be the best solution.

If I understood this correctly, fixing the guest driver wouldn't help here. The commit mentioned by Laszlo (6ab3fc32ea64 "hw:
replace most use of qemu_chr_fe_write with qemu_chr_fe_write_all") fixes data loss bugs but introduces the potential for deadlocks like the one we are seeing.

Quoted 3 lines

Unless we replace the existing ~30 qemu_chr_fe_write_all() calls in device code, we need to make sure we are constantly reading data from the console socket.

Yes, you must *always* have something reading from the chardev backend. This really sucks, but it is preferrable to letting data end up in /dev/null.

Quoted 12 lines

If this is being a problem for tests then consider it motivation to fix the root cause problem and make the devices properly do async I/O for chardevs. Only a handful of them properly do this right now.

Regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Daniel,

Thanks for the explanation. I don't feel completely capable of tackling the root cause, but I'll give it a go anyway.

Eduardo,

Looks like we'll need this type of workaround now anyways. As seen in the reproducer I wrote, a test can cause this deadlock before it gets to the test tearDown(), and either implicitly or explicitly gets to QEMUMachine::shutdown().

I would also like to avoid exposing the user (test writer) to that kind of situation though, which seems like what you tried to do. But, maybe the most honest approach right now is to make a explicit call to read from and close the socket, paired with documentation/comment about why that is being done? This would be similar to the "XXX" comments on 6ab3fc32ea64.

Regards!
- Cleber.

#74 lore raw

Re: [Qemu-devel] [PATCH 15/18] Boot Linux Console Test: add a test for aarch64 + virt

In reply to Eduardo Habkost
Quoted 10 lines · Eduardo Habkost

Thread 4 (Thread 0x7f2e45fff700 (LWP 6461)):

#0  0x00007f2e4ec03500 in nanosleep () at /lib64/libpthread.so.0
#1  0x00007f2e4fb229d7 in g_usleep () at /lib64/libglib-2.0.so.0
#2  0x0000559a4e7ca4c9 in qemu_chr_write_buffer (s=s@entry=0x559a502d0ac0, buf=buf@entry=0x7f2e45ffdd90 "7", len=1, offset=offset@entry=0x7f2e45ffdd60, write_all=true) at chardev/char.c:115
#3  0x0000559a4e7ca78f in qemu_chr_write (s=0x559a502d0ac0, buf=buf@entry=0x7f2e45ffdd90 "7", len=len@entry=1, write_all=write_all@entry=true) at chardev/char.c:148
#4  0x0000559a4e7cc7e2 in qemu_chr_fe_write_all (be=be@entry=0x559a504b4c50, buf=buf@entry=0x7f2e45ffdd90 "7", len=len@entry=1) at chardev/char-fe.c:53
#5  0x0000559a4e58f320 in pl011_write (opaque=0x559a504b47d0, offset=0, value=55, size=<optimized out>) at hw/char/pl011.c:183
#6  0x0000559a4e325121 in memory_region_write_accessor (mr=0x559a504b4ae0, addr=0, value=<optimized out>, size=2, shift=<optimized out>, mask=<optimized out>, attrs=...) at /home/ehabkost/rh/proj/virt/qemu/memory.c:503
#7  0x0000559a4e322cd6 in access_with_adjusted_size (addr=addr@entry=0, value=value@entry=0x7f2e45ffded8, size=size@entry=2, access_size_min=<optimized out>, access_size_max=<optimized out>, access_fn=access_fn@entry=
    0x559a4e3250a0 <memory_region_write_accessor>, mr=0x559a504b4ae0, attrs=...) at /home/ehabkost/rh/proj/virt/qemu/memory.c:569
Quoted 14 lines

#8 0x0000559a4e32763f in memory_region_dispatch_write (mr=mr@entry=0x559a504b4ae0, addr=addr@entry=0, data=<optimized out>, data@entry=55, size=size@entry=2, attrs=...) at /home/ehabkost/rh/proj/virt/qemu/memory.c:1497 #9 0x0000559a4e338708 in io_writex (env=env@entry=0x559a503d5620, mmu_idx=mmu_idx@entry=1, val=val@entry=55, addr=addr@entry=18446462598867529728, retaddr=139836732143069, size=2, iotlbentry=<optimized out>, iotlbentry=<optimized out>)
at /home/ehabkost/rh/proj/virt/qemu/accel/tcg/cputlb.c:945
#10 0x0000559a4e33d203 in store_helper (big_endian=false, size=2, retaddr=<optimized out>, oi=<optimized out>, val=55, addr=18446462598867529728, env=0x559a503d5620) at /home/ehabkost/rh/proj/virt/qemu/accel/tcg/cputlb.c:1544 #11 0x0000559a4e33d203 in helper_le_stw_mmu (env=0x559a503d5620, addr=18446462598867529728, val=55, oi=<optimized out>, retaddr=139836732143069) at /home/ehabkost/rh/proj/virt/qemu/accel/tcg/cputlb.c:1636 #12 0x00007f2e46bef5dd in code_gen_buffer ()
#13 0x0000559a4e352381 in cpu_tb_exec (itb=<optimized out>, cpu=0x559a503d5620) at /home/ehabkost/rh/proj/virt/qemu/accel/tcg/cpu-exec.c:171
#14 0x0000559a4e352381 in cpu_loop_exec_tb (tb_exit=<synthetic pointer>, last_tb=<synthetic pointer>, tb=<optimized out>, cpu=0x559a503d5620) at /home/ehabkost/rh/proj/virt/qemu/accel/tcg/cpu-exec.c:618 #15 0x0000559a4e352381 in cpu_exec (cpu=cpu@entry=0x559a503cd360) at /home/ehabkost/rh/proj/virt/qemu/accel/tcg/cpu-exec.c:729
#16 0x0000559a4e30ea0f in tcg_cpu_exec (cpu=0x559a503cd360) at /home/ehabkost/rh/proj/virt/qemu/cpus.c:1434
#17 0x0000559a4e310b6b in qemu_tcg_cpu_thread_fn (arg=arg@entry=0x559a503cd360) at /home/ehabkost/rh/proj/virt/qemu/cpus.c:1743
#18 0x0000559a4e83669a in qemu_thread_start (args=<optimized out>) at util/qemu-thread-posix.c:502
#19 0x00007f2e4ebf958e in start_thread () at /lib64/libpthread.so.0
#20 0x00007f2e4eb266f3 in clone () at /lib64/libc.so.6

See also <https://bugzilla.redhat.com/show_bug.cgi?id=1661940>.

$ git show 6ab3fc32ea64 -- hw/char/pl011.c

Thanks
Laszlo

#75 lore raw

Re: [Qemu-devel] [PATCH 15/18] Boot Linux Console Test: add a test for aarch64 + virt

In reply to Eduardo Habkost
Quoted 145 lines · Eduardo Habkost
Quoted 139 lines · Cleber Rosa
Quoted 104 lines · Cleber Rosa
Quoted 95 lines · Cleber Rosa
Quoted 83 lines · Wainer dos Santos Moschetta
Quoted 34 lines · Cleber Rosa

Just like the previous tests, boots a Linux kernel on a aarch64 target using the virt machine.

One special option added is the CPU type, given that the kernel selected fails to boot on the virt machine's default CPU (cortex-a15).

---
  .travis.yml                            |  2 +- tests/acceptance/boot_linux_console.py | 20 ++++++++++++++++++++

  2 files changed, 21 insertions(+), 1 deletion(-)
.travis.yml +0 -0
index 54100eea5a..595e8c0b6c 100644
@@ -187,7 +187,7 @@ matrix:
         # Acceptance (Functional) tests

      - env:
-        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu"
+        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu"

          - TEST_CMD="make check-acceptance"
        addons:
          apt:

tests/acceptance/boot_linux_console.py +0 -0
index f3ccd23a7a..107700b517 100644
@@ -138,3 +138,23 @@ class BootLinuxConsole(Test):
           self.vm.launch()
Quoted 5 lines

          console_pattern = 'Kernel command line: %s' % kernel_command_line self.wait_for_console_pattern(console_pattern) +
+    def test_aarch64_virt(self):

That test case fails on my system (Fedora 29 x86_64). Avocado seems unable to kill the VM so it  reaches the timeout.

I compiled QEMU with default configuration:

$ configure --python=/usr/bin/python3 --target-list=x86_64-softmmu
--target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu)

Follows a snippet of the Avocado's job.log file:
----

2019-01-31 14:41:34,912 test             L0602 INFO | START

07-/root/src/qemu/tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_aarch64_virt

2019-01-31 14:41:34,912 test             L0298 DEBUG| DATA
(filename=output.expected) => NOT FOUND (data sources: variant, test, file)
2019-01-31 14:41:34,913 parameters       L0146 DEBUG| PARAMS (key=arch,
path=*, default=aarch64) => 'aarch64'
2019-01-31 14:41:34,913 parameters       L0146 DEBUG| PARAMS
(key=qemu_bin, path=*, default=aarch64-softmmu/qemu-system-aarch64) =>
'aarch64-softmmu/qemu-system-aarch64'
2019-01-31 14:41:34,915 download         L0070 INFO | Fetching
https://sjc.edge.kernel.org/fedora-buffet/fedora/linux/releases/29/Server/aarch64/os/images/pxeboot/vmlinuz
-> /var/lib/avocado/data/cache/by_name/vmlinuz.3upct2pr
2019-01-31 14:41:35,490 download         L0054 DEBUG| Retrieved URL
"https://sjc.edge.kernel.org/fedora-buffet/fedora/linux/releases/29/Server/aarch64/os/images/pxeboot/vmlinuz":
content-length 8623423, date: "Thu, 31 Jan 2019 19:41:35 GMT",
last-modified: "Sun, 21 Oct 2018 00:43:09 GMT"
2019-01-31 14:41:41,765 qemu             L0317 DEBUG| VM launch command:

'aarch64-softmmu/qemu-system-aarch64 -chardev socket,id=mon,path=/var/tmp/tmpizirkcud/qemu-32609-monitor.sock -mon chardev=mon,mode=control -display none -vga none -machine virt -chardev socket,id=console,path=/var/tmp/tmpizirkcud/qemu-32609-console.sock,server,nowait -serial chardev:console -cpu cortex-a53 -kernel /var/lib/avocado/data/cache/by_name/vmlinuz -append console=ttyAMA0'

2019-01-31 14:41:41,779 qmp              L0167 DEBUG| >>> {'execute':
'qmp_capabilities'}
2019-01-31 14:41:41,931 qmp              L0175 DEBUG| <<< {'return': {}}
2019-01-31 14:41:42,830 boot_linux_conso L0041 DEBUG| [    0.000000]

Booting Linux on physical CPU 0x0000000000 [0x410fd034]

Quoted 8 lines

(...)

2019-01-31 14:41:42,833 boot_linux_conso L0041 DEBUG| [    0.000000]
Policy zone: DMA32
2019-01-31 14:41:42,833 boot_linux_conso L0041 DEBUG| [    0.000000]
Kernel command line: console=ttyAMA0
2019-01-31 14:41:42,833 qmp              L0167 DEBUG| >>> {'execute':

'quit'}

Quoted 5 lines

Here, a QMP response like "<<< {'return': {}}" would be expected.

Since I can not reproduce this on my system (or on Travis-CI jobs I've sent), can you tell me on top of which commit you've applied these patches?

Quoted 7 lines

I spoke too soon:

https://travis-ci.org/clebergnu/qemu/jobs/487121425#L3033

This looks like a recent regression, and I'm guessing it's not on the test's side. I'll try to bisect it and let you know.

On a fresh environment, I am able to get this reproduced on every 2 of runs, more or less. When I hit it, I attached GDB to it, and the backtrace shows:

Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
warning: Loadable section ".note.gnu.property" outside of ELF segments
warning: Loadable section ".note.gnu.property" outside of ELF segments __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:103
103 2: movl %edx, %eax
(gdb) bt
#0 __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:103
#1 0x00007fc6ba1a2e09 in __GI___pthread_mutex_lock (mutex=mutex@entry=0x5615a233d020 <qemu_global_mutex>) at ../nptl/pthread_mutex_lock.c:80
#2 0x00005615a1bb7593 in qemu_mutex_lock_impl (mutex=0x5615a233d020 <qemu_global_mutex>, file=0x5615a1db2d4c "util/main-loop.c", line=236) at util/qemu-thread-posix.c:66
#3 0x00005615a171125e in qemu_mutex_lock_iothread_impl (file=file@entry=0x5615a1db2d4c "util/main-loop.c", line=line@entry=236) at /home/cleber/src/qemu/cpus.c:1849
#4 0x00005615a1bb415d in os_host_main_loop_wait (timeout=<optimized out>) at util/main-loop.c:236

Quoted 4 lines
#5  main_loop_wait (nonblocking=<optimized out>) at util/main-loop.c:497
#6  0x00005615a18fdd39 in main_loop () at vl.c:1928
#7  0x00005615a16c9ee9 in main (argc=<optimized out>, argv=<optimized

out>, envp=<optimized out>) at vl.c:4665

Quoted 4 lines

Tip: run "thread apply all bt" so you can get a backtrace of all threads.

Nice, thanks!

Quoted 7 lines
Quoted 6 lines

Running it with taskset -c 1 prevents this issue from happening, which AFAICT, contributes even further towards this being a QEMU race condition.

I'm CC'ing Peter and Claudio (listed maintainers of aarch64), as this seems to limited to that target. Any tips on what to do here?

Quoted 8 lines

I am hitting this on Travis, too, and I finally could reproduce it locally,

The guest is still writing on the serial console, but nobody is reading the data on the other side. A VCPU thread is stuck inside the EAGAIN/nanosleep loop at qemu_chr_write_buffer(), holding the QEMU global lock.

This is awesome, because it really looks like a perfect explanation for the behavior/reproducer/workaround that I described here:

https://bugs.launchpad.net/qemu/+bug/1829779

Quoted 26 lines

Thread 4 (Thread 0x7f2e45fff700 (LWP 6461)):

#0  0x00007f2e4ec03500 in nanosleep () at /lib64/libpthread.so.0
#1  0x00007f2e4fb229d7 in g_usleep () at /lib64/libglib-2.0.so.0
#2  0x0000559a4e7ca4c9 in qemu_chr_write_buffer (s=s@entry=0x559a502d0ac0, buf=buf@entry=0x7f2e45ffdd90 "7", len=1, offset=offset@entry=0x7f2e45ffdd60, write_all=true) at chardev/char.c:115
#3  0x0000559a4e7ca78f in qemu_chr_write (s=0x559a502d0ac0, buf=buf@entry=0x7f2e45ffdd90 "7", len=len@entry=1, write_all=write_all@entry=true) at chardev/char.c:148
#4  0x0000559a4e7cc7e2 in qemu_chr_fe_write_all (be=be@entry=0x559a504b4c50, buf=buf@entry=0x7f2e45ffdd90 "7", len=len@entry=1) at chardev/char-fe.c:53
#5  0x0000559a4e58f320 in pl011_write (opaque=0x559a504b47d0, offset=0, value=55, size=<optimized out>) at hw/char/pl011.c:183
#6  0x0000559a4e325121 in memory_region_write_accessor (mr=0x559a504b4ae0, addr=0, value=<optimized out>, size=2, shift=<optimized out>, mask=<optimized out>, attrs=...) at /home/ehabkost/rh/proj/virt/qemu/memory.c:503
#7  0x0000559a4e322cd6 in access_with_adjusted_size (addr=addr@entry=0, value=value@entry=0x7f2e45ffded8, size=size@entry=2, access_size_min=<optimized out>, access_size_max=<optimized out>, access_fn=access_fn@entry=
    0x559a4e3250a0 <memory_region_write_accessor>, mr=0x559a504b4ae0, attrs=...) at /home/ehabkost/rh/proj/virt/qemu/memory.c:569

#8 0x0000559a4e32763f in memory_region_dispatch_write (mr=mr@entry=0x559a504b4ae0, addr=addr@entry=0, data=<optimized out>, data@entry=55, size=size@entry=2, attrs=...) at /home/ehabkost/rh/proj/virt/qemu/memory.c:1497 #9 0x0000559a4e338708 in io_writex (env=env@entry=0x559a503d5620, mmu_idx=mmu_idx@entry=1, val=val@entry=55, addr=addr@entry=18446462598867529728, retaddr=139836732143069, size=2, iotlbentry=<optimized out>, iotlbentry=<optimized out>)
at /home/ehabkost/rh/proj/virt/qemu/accel/tcg/cputlb.c:945
#10 0x0000559a4e33d203 in store_helper (big_endian=false, size=2, retaddr=<optimized out>, oi=<optimized out>, val=55, addr=18446462598867529728, env=0x559a503d5620) at /home/ehabkost/rh/proj/virt/qemu/accel/tcg/cputlb.c:1544 #11 0x0000559a4e33d203 in helper_le_stw_mmu (env=0x559a503d5620, addr=18446462598867529728, val=55, oi=<optimized out>, retaddr=139836732143069) at /home/ehabkost/rh/proj/virt/qemu/accel/tcg/cputlb.c:1636 #12 0x00007f2e46bef5dd in code_gen_buffer ()
#13 0x0000559a4e352381 in cpu_tb_exec (itb=<optimized out>, cpu=0x559a503d5620) at /home/ehabkost/rh/proj/virt/qemu/accel/tcg/cpu-exec.c:171
#14 0x0000559a4e352381 in cpu_loop_exec_tb (tb_exit=<synthetic pointer>, last_tb=<synthetic pointer>, tb=<optimized out>, cpu=0x559a503d5620) at /home/ehabkost/rh/proj/virt/qemu/accel/tcg/cpu-exec.c:618 #15 0x0000559a4e352381 in cpu_exec (cpu=cpu@entry=0x559a503cd360) at /home/ehabkost/rh/proj/virt/qemu/accel/tcg/cpu-exec.c:729
#16 0x0000559a4e30ea0f in tcg_cpu_exec (cpu=0x559a503cd360) at /home/ehabkost/rh/proj/virt/qemu/cpus.c:1434
#17 0x0000559a4e310b6b in qemu_tcg_cpu_thread_fn (arg=arg@entry=0x559a503cd360) at /home/ehabkost/rh/proj/virt/qemu/cpus.c:1743
#18 0x0000559a4e83669a in qemu_thread_start (args=<optimized out>) at util/qemu-thread-posix.c:502
#19 0x00007f2e4ebf958e in start_thread () at /lib64/libpthread.so.0
#20 0x00007f2e4eb266f3 in clone () at /lib64/libc.so.6

Quoted 8 lines

For reference, this is the QEMU command line:

aarch64-softmmu/qemu-system-aarch64 -chardev socket,id=mon,path=/var/tmp/tmpxnkcjvf0/qemu-6453-monitor.sock -mon chardev=mon,mode=control -display none -vga none -machine virt -chardev socket,id=console,path=/var/tmp/tmpxnkcjvf0/qemu-6453-console.sock,server,nowait -serial chardev:console -cpu cortex-a53 -kernel /home/ehabkost/rh/proj/virt/qemu/tests/venv/data/cache/by_location/e959d0e1dd72e77653e218e666198db1f3d0c213/vmlinuz -append printk.time=0 console=ttyAMA0

Signature · 1 line
Eduardo

Besides the command line, I hope it's also easy to use the following reproducer:

https://github.com/clebergnu/qemu/commit/c778e28c24030c4a36548b714293b319f4bf18df

It should apply cleanly to QEMU master, and instructions on how to run it are documented in the commit message.

Thanks,
- Cleber.

#76 lore raw

PATCH 16/18 Boot Linux Console Test: add a test for arm + virt

In reply to Cleber Rosa

Just like the previous tests, boots a Linux kernel on an arm target using the virt machine.

---
 .travis.yml                            |  2 +-
 tests/acceptance/boot_linux_console.py | 19 +++++++++++++++++++
 2 files changed, 20 insertions(+), 1 deletion(-)
.travis.yml +1 -1
index 595e8c0b6c..ea7c9828ac 100644
@@ -187,7 +187,7 @@ matrix:
 
     # Acceptance (Functional) tests
     - env:
-        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu"
+        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu,arm-softmmu"
         - TEST_CMD="make check-acceptance"
       addons:
         apt:
tests/acceptance/boot_linux_console.py +19 -0
index 107700b517..00aee3114e 100644
@@ -158,3 +158,22 @@ class BootLinuxConsole(Test):
         self.vm.launch()
         console_pattern = 'Kernel command line: %s' % kernel_command_line
         self.wait_for_console_pattern(console_pattern)
+
+    def test_arm_virt(self):
+        """
+        :avocado: tags=arch:arm
+        :avocado: tags=machine:virt
+        """
+        kernel_url = ('https://sjc.edge.kernel.org/fedora-buffet/fedora/linux/'
+                      'releases/29/Server/armhfp/os/images/pxeboot/vmlinuz')
+        kernel_hash = 'e9826d741b4fb04cadba8d4824d1ed3b7fb8b4d4'
+        kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
+
+        self.vm.set_machine('virt')
+        self.vm.set_console()
+        kernel_command_line = 'console=ttyAMA0'
+        self.vm.add_args('-kernel', kernel_path,
+                         '-append', kernel_command_line)
+        self.vm.launch()
+        console_pattern = 'Kernel command line: %s' % kernel_command_line
+        self.wait_for_console_pattern(console_pattern)
Signature · 1 line
2.20.1
Reviewed-by on v1 16/18
#77 lore raw

Re: [Qemu-devel] [PATCH 16/18] Boot Linux Console Test: add a test for arm + virt

Quoted 4 lines · Cleber Rosa

Just like the previous tests, boots a Linux kernel on an arm target using the virt machine.

Quoted 48 lines
---
 .travis.yml                            |  2 +-
 tests/acceptance/boot_linux_console.py | 19 +++++++++++++++++++
 2 files changed, 20 insertions(+), 1 deletion(-)
.travis.yml +1 -1
index 595e8c0b6c..ea7c9828ac 100644
@@ -187,7 +187,7 @@ matrix:
 
     # Acceptance (Functional) tests
     - env:
-        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu"
+        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu,arm-softmmu"
         - TEST_CMD="make check-acceptance"
       addons:
         apt:
tests/acceptance/boot_linux_console.py +19 -0
index 107700b517..00aee3114e 100644
@@ -158,3 +158,22 @@ class BootLinuxConsole(Test):
         self.vm.launch()
         console_pattern = 'Kernel command line: %s' % kernel_command_line
         self.wait_for_console_pattern(console_pattern)
+
+    def test_arm_virt(self):
+        """
+        :avocado: tags=arch:arm
+        :avocado: tags=machine:virt
+        """
+        kernel_url = ('https://sjc.edge.kernel.org/fedora-buffet/fedora/linux/'
+                      'releases/29/Server/armhfp/os/images/pxeboot/vmlinuz')
+        kernel_hash = 'e9826d741b4fb04cadba8d4824d1ed3b7fb8b4d4'
+        kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
+
+        self.vm.set_machine('virt')
+        self.vm.set_console()
+        kernel_command_line = 'console=ttyAMA0'
+        self.vm.add_args('-kernel', kernel_path,
+                         '-append', kernel_command_line)
+        self.vm.launch()
+        console_pattern = 'Kernel command line: %s' % kernel_command_line
+        self.wait_for_console_pattern(console_pattern)
Signature · 1 line
2.20.1
Signature · 3 lines
Caio Carrara
Software Engineer, Virt Team - Red Hat
[email protected]
#78 lore raw

PATCH 17/18 Boot Linux Console Test: add a test for s390x + s390-ccw-virtio

In reply to Cleber Rosa

Just like the previous tests, boots a Linux kernel on a s390x target using the s390-ccw-virtio machine.

Because it's not possible to have multiple VT220 consoles, '-nodefaults' is used, so that the one set with set_console() works correctly.

---
 .travis.yml                            |  2 +-
 tests/acceptance/boot_linux_console.py | 20 ++++++++++++++++++++
 2 files changed, 21 insertions(+), 1 deletion(-)
.travis.yml +1 -1
index ea7c9828ac..0d5a4b104b 100644
@@ -187,7 +187,7 @@ matrix:
 
     # Acceptance (Functional) tests
     - env:
-        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu,arm-softmmu"
+        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu,arm-softmmu,s390x-softmmu"
         - TEST_CMD="make check-acceptance"
       addons:
         apt:
tests/acceptance/boot_linux_console.py +20 -0
index 00aee3114e..c4d5477d45 100644
@@ -177,3 +177,23 @@ class BootLinuxConsole(Test):
         self.vm.launch()
         console_pattern = 'Kernel command line: %s' % kernel_command_line
         self.wait_for_console_pattern(console_pattern)
+
+    def test_s390x_s390_ccw_virtio(self):
+        """
+        :avocado: tags=arch:s390x
+        :avocado: tags=machine:s390_ccw_virtio
+        """
+        kernel_url = ('http://mirrors.rit.edu/fedora/fedora-secondary/releases'
+                      '/29/Server/s390x/os/images/kernel.img')
+        kernel_hash = 'e8e8439103ef8053418ef062644ffd46a7919313'
+        kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
+
+        self.vm.set_machine('s390-ccw-virtio')
+        self.vm.set_console()
+        kernel_command_line = 'console=sclp0'
+        self.vm.add_args('-nodefaults',
+                         '-kernel', kernel_path,
+                         '-append', kernel_command_line)
+        self.vm.launch()
+        console_pattern = 'Kernel command line: %s' % kernel_command_line
+        self.wait_for_console_pattern(console_pattern)
Signature · 1 line
2.20.1
#79 lore raw

Re: [Qemu-devel] [PATCH 17/18] Boot Linux Console Test: add a test for s390x + s390-ccw-virtio

Quoted 44 lines · On Thu, 17 Jan 2019 13:56:27 -0500 Cleber Rosa <[email protected]> wrote:

Just like the previous tests, boots a Linux kernel on a s390x target using the s390-ccw-virtio machine.

Because it's not possible to have multiple VT220 consoles, '-nodefaults' is used, so that the one set with set_console() works correctly.

---
 .travis.yml                            |  2 +-
 tests/acceptance/boot_linux_console.py | 20 ++++++++++++++++++++
 2 files changed, 21 insertions(+), 1 deletion(-)
.travis.yml +1 -1
index ea7c9828ac..0d5a4b104b 100644
@@ -187,7 +187,7 @@ matrix:
 
     # Acceptance (Functional) tests
     - env:
-        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu,arm-softmmu"
+        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu,arm-softmmu,s390x-softmmu"
         - TEST_CMD="make check-acceptance"
       addons:
         apt:
tests/acceptance/boot_linux_console.py +10 -0
index 00aee3114e..c4d5477d45 100644
@@ -177,3 +177,23 @@ class BootLinuxConsole(Test):
         self.vm.launch()
         console_pattern = 'Kernel command line: %s' % kernel_command_line
         self.wait_for_console_pattern(console_pattern)
+
+    def test_s390x_s390_ccw_virtio(self):
+        """
+        :avocado: tags=arch:s390x
+        :avocado: tags=machine:s390_ccw_virtio
+        """
+        kernel_url = ('http://mirrors.rit.edu/fedora/fedora-secondary/releases'
+                      '/29/Server/s390x/os/images/kernel.img')
+        kernel_hash = 'e8e8439103ef8053418ef062644ffd46a7919313'
+        kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)

tests/acceptance/boot_linux_console.py class BootLinuxConsole(Test): · line 199

Quoted 10 lines
tests/acceptance/boot_linux_console.py +10 -0
+
+        self.vm.set_machine('s390-ccw-virtio')
+        self.vm.set_console()
+        kernel_command_line = 'console=sclp0'
+        self.vm.add_args('-nodefaults',
+                         '-kernel', kernel_path,
+                         '-append', kernel_command_line)
+        self.vm.launch()
+        console_pattern = 'Kernel command line: %s' % kernel_command_line
+        self.wait_for_console_pattern(console_pattern)

Cool, I had been looking at adding some of the tests I'm running here to this framework as well, and this gives me a good base to work with :)

I'll try to find some time to give it a whirl.

#80 lore raw

Re: [Qemu-devel] [PATCH 17/18] Boot Linux Console Test: add a test for s390x + s390-ccw-virtio

Quoted 6 lines · Cornelia Huck

Cool, I had been looking at adding some of the tests I'm running here to this framework as well, and this gives me a good base to work with :)

I'll try to find some time to give it a whirl.

I can't assist you in finding some time, but for anything else, please ping me :)

Cheers,
- Cleber.

Reviewed-by on v1 17/18
#81 lore raw

Re: [Qemu-devel] [PATCH 17/18] Boot Linux Console Test: add a test for s390x + s390-ccw-virtio

In reply to Cleber Rosa
Quoted 8 lines · Cleber Rosa

Just like the previous tests, boots a Linux kernel on a s390x target using the s390-ccw-virtio machine.

Because it's not possible to have multiple VT220 consoles, '-nodefaults' is used, so that the one set with set_console() works correctly.

Quoted 49 lines
---
 .travis.yml                            |  2 +-
 tests/acceptance/boot_linux_console.py | 20 ++++++++++++++++++++
 2 files changed, 21 insertions(+), 1 deletion(-)
.travis.yml +1 -1
index ea7c9828ac..0d5a4b104b 100644
@@ -187,7 +187,7 @@ matrix:
 
     # Acceptance (Functional) tests
     - env:
-        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu,arm-softmmu"
+        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu,arm-softmmu,s390x-softmmu"
         - TEST_CMD="make check-acceptance"
       addons:
         apt:
tests/acceptance/boot_linux_console.py +20 -0
index 00aee3114e..c4d5477d45 100644
@@ -177,3 +177,23 @@ class BootLinuxConsole(Test):
         self.vm.launch()
         console_pattern = 'Kernel command line: %s' % kernel_command_line
         self.wait_for_console_pattern(console_pattern)
+
+    def test_s390x_s390_ccw_virtio(self):
+        """
+        :avocado: tags=arch:s390x
+        :avocado: tags=machine:s390_ccw_virtio
+        """
+        kernel_url = ('http://mirrors.rit.edu/fedora/fedora-secondary/releases'
+                      '/29/Server/s390x/os/images/kernel.img')
+        kernel_hash = 'e8e8439103ef8053418ef062644ffd46a7919313'
+        kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
+
+        self.vm.set_machine('s390-ccw-virtio')
+        self.vm.set_console()
+        kernel_command_line = 'console=sclp0'
+        self.vm.add_args('-nodefaults',
+                         '-kernel', kernel_path,
+                         '-append', kernel_command_line)
+        self.vm.launch()
+        console_pattern = 'Kernel command line: %s' % kernel_command_line
+        self.wait_for_console_pattern(console_pattern)
Signature · 1 line
2.20.1
Signature · 3 lines
Caio Carrara
Software Engineer, Virt Team - Red Hat
[email protected]
#82 lore raw

PATCH 18/18 Boot Linux Console Test: add a test for alpha + clipper

In reply to Cleber Rosa

Similar to the x86_64 + pc test, it boots a Linux kernel on a Malta board and verify the serial is working. One extra command added to the QEMU command line is '-vga std', because the kernel used is known to crash without it.

If alpha is a target being built, "make check-acceptance" will automatically include this test by the use of the "arch:alpha" tags.

Alternatively, this test can be run using:

    $ avocado run -t arch:alpha tests/acceptance
    $ avocado run -t machine:clipper tests/acceptance
---
 .travis.yml                            |  2 +-
 scripts/qemu.py                        |  1 -
 tests/acceptance/boot_linux_console.py | 22 ++++++++++++++++++++++
 3 files changed, 23 insertions(+), 2 deletions(-)
.travis.yml +1 -1
index 0d5a4b104b..73a113af87 100644
@@ -187,7 +187,7 @@ matrix:
 
     # Acceptance (Functional) tests
     - env:
-        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu,arm-softmmu,s390x-softmmu"
+        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu,arm-softmmu,s390x-softmmu,alpha-softmmu"
         - TEST_CMD="make check-acceptance"
       addons:
         apt:
scripts/qemu.py +0 -1
index 1531e28fc1..a704da418a 100644
@@ -34,7 +34,6 @@ def kvm_available(target_arch=None):
 
 #: Maps machine types to the preferred console device types
 CONSOLE_DEV_TYPES = {
-    r'^clipper$': 'isa-serial',
     r'^(pc.*|q35.*|isapc)$': 'isa-serial',
     r'^(40p|powernv|prep)$': 'isa-serial',
     r'^s390-ccw-virtio.*': 'sclpconsole',
tests/acceptance/boot_linux_console.py +22 -0
index c4d5477d45..a8028a39d4 100644
@@ -197,3 +197,25 @@ class BootLinuxConsole(Test):
         self.vm.launch()
         console_pattern = 'Kernel command line: %s' % kernel_command_line
         self.wait_for_console_pattern(console_pattern)
+
+    def test_alpha_clipper(self):
+        """
+        :avocado: tags=arch:alpha
+        :avocado: tags=machine:clipper
+        """
+        kernel_url = ('http://archive.debian.org/debian/dists/lenny/main/'
+                      'installer-alpha/current/images/cdrom/vmlinuz')
+        kernel_hash = '3a943149335529e2ed3e74d0d787b85fb5671ba3'
+        kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
+
+        uncompressed_kernel = archive.uncompress(kernel_path, self.workdir)
+
+        self.vm.set_machine('clipper')
+        self.vm.set_console()
+        kernel_command_line = 'console=ttyS0 printk.time=0'
+        self.vm.add_args('-vga', 'std',
+                         '-kernel', uncompressed_kernel,
+                         '-append', kernel_command_line)
+        self.vm.launch()
+        console_pattern = 'Kernel command line: %s' % kernel_command_line
+        self.wait_for_console_pattern(console_pattern)
Signature · 1 line
2.20.1
Reviewed-by on v1 18/18
#83 lore raw

Re: [Qemu-devel] [PATCH 18/18] Boot Linux Console Test: add a test for alpha + clipper

Quoted 8 lines · Cleber Rosa

Similar to the x86_64 + pc test, it boots a Linux kernel on a Malta board and verify the serial is working. One extra command added to the QEMU command line is '-vga std', because the kernel used is known to crash without it.

If alpha is a target being built, "make check-acceptance" will automatically include this test by the use of the "arch:alpha" tags.

Quoted 7 lines

Alternatively, this test can be run using:

    $ avocado run -t arch:alpha tests/acceptance
    $ avocado run -t machine:clipper tests/acceptance
Quoted 64 lines
---
 .travis.yml                            |  2 +-
 scripts/qemu.py                        |  1 -
 tests/acceptance/boot_linux_console.py | 22 ++++++++++++++++++++++
 3 files changed, 23 insertions(+), 2 deletions(-)
.travis.yml +1 -1
index 0d5a4b104b..73a113af87 100644
@@ -187,7 +187,7 @@ matrix:
 
     # Acceptance (Functional) tests
     - env:
-        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu,arm-softmmu,s390x-softmmu"
+        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu,arm-softmmu,s390x-softmmu,alpha-softmmu"
         - TEST_CMD="make check-acceptance"
       addons:
         apt:
scripts/qemu.py +0 -1
index 1531e28fc1..a704da418a 100644
@@ -34,7 +34,6 @@ def kvm_available(target_arch=None):
 
 #: Maps machine types to the preferred console device types
 CONSOLE_DEV_TYPES = {
-    r'^clipper$': 'isa-serial',
     r'^(pc.*|q35.*|isapc)$': 'isa-serial',
     r'^(40p|powernv|prep)$': 'isa-serial',
     r'^s390-ccw-virtio.*': 'sclpconsole',
tests/acceptance/boot_linux_console.py +22 -0
index c4d5477d45..a8028a39d4 100644
@@ -197,3 +197,25 @@ class BootLinuxConsole(Test):
         self.vm.launch()
         console_pattern = 'Kernel command line: %s' % kernel_command_line
         self.wait_for_console_pattern(console_pattern)
+
+    def test_alpha_clipper(self):
+        """
+        :avocado: tags=arch:alpha
+        :avocado: tags=machine:clipper
+        """
+        kernel_url = ('http://archive.debian.org/debian/dists/lenny/main/'
+                      'installer-alpha/current/images/cdrom/vmlinuz')
+        kernel_hash = '3a943149335529e2ed3e74d0d787b85fb5671ba3'
+        kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
+
+        uncompressed_kernel = archive.uncompress(kernel_path, self.workdir)
+
+        self.vm.set_machine('clipper')
+        self.vm.set_console()
+        kernel_command_line = 'console=ttyS0 printk.time=0'
+        self.vm.add_args('-vga', 'std',
+                         '-kernel', uncompressed_kernel,
+                         '-append', kernel_command_line)
+        self.vm.launch()
+        console_pattern = 'Kernel command line: %s' % kernel_command_line
+        self.wait_for_console_pattern(console_pattern)
Signature · 1 line
2.20.1
Signature · 3 lines
Caio Carrara
Software Engineer, Virt Team - Red Hat
[email protected]
#84 lore raw

Re: [Qemu-devel] [PATCH 18/18] Boot Linux Console Test: add a test for alpha + clipper

In reply to Cleber Rosa
Quoted 39 lines · Cleber Rosa

Similar to the x86_64 + pc test, it boots a Linux kernel on a Malta board and verify the serial is working. One extra command added to the QEMU command line is '-vga std', because the kernel used is known to crash without it.

If alpha is a target being built, "make check-acceptance" will automatically include this test by the use of the "arch:alpha" tags.

Alternatively, this test can be run using:

    $ avocado run -t arch:alpha tests/acceptance
    $ avocado run -t machine:clipper tests/acceptance
---
 .travis.yml                            |  2 +-
 scripts/qemu.py                        |  1 -
 tests/acceptance/boot_linux_console.py | 22 ++++++++++++++++++++++
 3 files changed, 23 insertions(+), 2 deletions(-)
.travis.yml +1 -1
index 0d5a4b104b..73a113af87 100644
@@ -187,7 +187,7 @@ matrix:
 
     # Acceptance (Functional) tests
     - env:
-        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu,arm-softmmu,s390x-softmmu"
+        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu,arm-softmmu,s390x-softmmu,alpha-softmmu"
         - TEST_CMD="make check-acceptance"
       addons:
         apt:
scripts/qemu.py +0 -0
index 1531e28fc1..a704da418a 100644

scripts/qemu.py def kvm_available(target_arch=None): · line 37

Quoted 4 lines
scripts/qemu.py +0 -1
@@ -34,7 +34,6 @@ def kvm_available(target_arch=None):
 
 #: Maps machine types to the preferred console device types
 CONSOLE_DEV_TYPES = {
-    r'^clipper$': 'isa-serial',

Why this change?

Quoted 34 lines
scripts/qemu.py +0 -0
     r'^(pc.*|q35.*|isapc)$': 'isa-serial',
     r'^(40p|powernv|prep)$': 'isa-serial',
     r'^s390-ccw-virtio.*': 'sclpconsole',
tests/acceptance/boot_linux_console.py +22 -0
index c4d5477d45..a8028a39d4 100644
@@ -197,3 +197,25 @@ class BootLinuxConsole(Test):
         self.vm.launch()
         console_pattern = 'Kernel command line: %s' % kernel_command_line
         self.wait_for_console_pattern(console_pattern)
+
+    def test_alpha_clipper(self):
+        """
+        :avocado: tags=arch:alpha
+        :avocado: tags=machine:clipper
+        """
+        kernel_url = ('http://archive.debian.org/debian/dists/lenny/main/'
+                      'installer-alpha/current/images/cdrom/vmlinuz')
+        kernel_hash = '3a943149335529e2ed3e74d0d787b85fb5671ba3'
+        kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
+
+        uncompressed_kernel = archive.uncompress(kernel_path, self.workdir)
+
+        self.vm.set_machine('clipper')
+        self.vm.set_console()
+        kernel_command_line = 'console=ttyS0 printk.time=0'
+        self.vm.add_args('-vga', 'std',
+                         '-kernel', uncompressed_kernel,
+                         '-append', kernel_command_line)
+        self.vm.launch()
+        console_pattern = 'Kernel command line: %s' % kernel_command_line
+        self.wait_for_console_pattern(console_pattern)
#85 lore raw

Re: [Qemu-devel] [PATCH 18/18] Boot Linux Console Test: add a test for alpha + clipper

Quoted 41 lines · Philippe Mathieu-Daudé
Quoted 40 lines · Cleber Rosa

Similar to the x86_64 + pc test, it boots a Linux kernel on a Malta board and verify the serial is working. One extra command added to the QEMU command line is '-vga std', because the kernel used is known to crash without it.

If alpha is a target being built, "make check-acceptance" will automatically include this test by the use of the "arch:alpha" tags.

Alternatively, this test can be run using:

    $ avocado run -t arch:alpha tests/acceptance
    $ avocado run -t machine:clipper tests/acceptance
---
 .travis.yml                            |  2 +-
 scripts/qemu.py                        |  1 -
 tests/acceptance/boot_linux_console.py | 22 ++++++++++++++++++++++
 3 files changed, 23 insertions(+), 2 deletions(-)
.travis.yml +1 -1
index 0d5a4b104b..73a113af87 100644
@@ -187,7 +187,7 @@ matrix:
 
     # Acceptance (Functional) tests
     - env:
-        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu,arm-softmmu,s390x-softmmu"
+        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu,arm-softmmu,s390x-softmmu,alpha-softmmu"
         - TEST_CMD="make check-acceptance"
       addons:
         apt:
scripts/qemu.py +0 -0
index 1531e28fc1..a704da418a 100644
@@ -34,7 +34,6 @@ def kvm_available(target_arch=None):
 
Quoted 6 lines

scripts/qemu.py def kvm_available(target_arch=None): · line 37

Quoted 3 lines
scripts/qemu.py +0 -1
 #: Maps machine types to the preferred console device types
 CONSOLE_DEV_TYPES = {
-    r'^clipper$': 'isa-serial',

Why this change?

Because we've come to conclusion (I believe :) that's better, whenever possible, to let QEMU pick the default device type by machine. The discussion happened here:

https://lists.gnu.org/archive/html/qemu-devel/2018-10/msg04582.html

- Cleber.

#86 lore raw

Re: [Qemu-devel] [PATCH 18/18] Boot Linux Console Test: add a test for alpha + clipper

Quoted 49 lines · Cleber Rosa
Quoted 41 lines · Philippe Mathieu-Daudé
Quoted 40 lines · Cleber Rosa

Similar to the x86_64 + pc test, it boots a Linux kernel on a Malta board and verify the serial is working. One extra command added to the QEMU command line is '-vga std', because the kernel used is known to crash without it.

If alpha is a target being built, "make check-acceptance" will automatically include this test by the use of the "arch:alpha" tags.

Alternatively, this test can be run using:

    $ avocado run -t arch:alpha tests/acceptance
    $ avocado run -t machine:clipper tests/acceptance
---
 .travis.yml                            |  2 +-
 scripts/qemu.py                        |  1 -
 tests/acceptance/boot_linux_console.py | 22 ++++++++++++++++++++++
 3 files changed, 23 insertions(+), 2 deletions(-)
.travis.yml +1 -1
index 0d5a4b104b..73a113af87 100644
@@ -187,7 +187,7 @@ matrix:
 
     # Acceptance (Functional) tests
     - env:
-        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu,arm-softmmu,s390x-softmmu"
+        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu,arm-softmmu,s390x-softmmu,alpha-softmmu"
         - TEST_CMD="make check-acceptance"
       addons:
         apt:
scripts/qemu.py +0 -0
index 1531e28fc1..a704da418a 100644
@@ -34,7 +34,6 @@ def kvm_available(target_arch=None):
 
Quoted 6 lines

scripts/qemu.py def kvm_available(target_arch=None): · line 37

Quoted 3 lines
scripts/qemu.py +0 -1
 #: Maps machine types to the preferred console device types
 CONSOLE_DEV_TYPES = {
-    r'^clipper$': 'isa-serial',

Why this change?

Quoted 5 lines

Because we've come to conclusion (I believe :) that's better, whenever possible, to let QEMU pick the default device type by machine. The discussion happened here:

https://lists.gnu.org/archive/html/qemu-devel/2018-10/msg04582.html

I was expecting this change to be global (in a single patch previous to the target-specific patches). If you prefer to do it a target at a time I'm OK with it but you should add a comment about it in the commit IMHO.

Regards,

Phil.

#87 lore raw

Re: [Qemu-devel] [PATCH 18/18] Boot Linux Console Test: add a test for alpha + clipper

Quoted 56 lines · Philippe Mathieu-Daudé
Quoted 49 lines · Cleber Rosa
Quoted 41 lines · Philippe Mathieu-Daudé
Quoted 40 lines · Cleber Rosa

Similar to the x86_64 + pc test, it boots a Linux kernel on a Malta board and verify the serial is working. One extra command added to the QEMU command line is '-vga std', because the kernel used is known to crash without it.

If alpha is a target being built, "make check-acceptance" will automatically include this test by the use of the "arch:alpha" tags.

Alternatively, this test can be run using:

    $ avocado run -t arch:alpha tests/acceptance
    $ avocado run -t machine:clipper tests/acceptance
---
 .travis.yml                            |  2 +-
 scripts/qemu.py                        |  1 -
 tests/acceptance/boot_linux_console.py | 22 ++++++++++++++++++++++
 3 files changed, 23 insertions(+), 2 deletions(-)
.travis.yml +1 -1
index 0d5a4b104b..73a113af87 100644
@@ -187,7 +187,7 @@ matrix:
 
     # Acceptance (Functional) tests
     - env:
-        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu,arm-softmmu,s390x-softmmu"
+        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu,arm-softmmu,s390x-softmmu,alpha-softmmu"
         - TEST_CMD="make check-acceptance"
       addons:
         apt:
scripts/qemu.py +0 -0
index 1531e28fc1..a704da418a 100644
@@ -34,7 +34,6 @@ def kvm_available(target_arch=None):
 
Quoted 6 lines

scripts/qemu.py def kvm_available(target_arch=None): · line 37

Quoted 3 lines
scripts/qemu.py +0 -1
 #: Maps machine types to the preferred console device types
 CONSOLE_DEV_TYPES = {
-    r'^clipper$': 'isa-serial',

Why this change?

Quoted 5 lines

Because we've come to conclusion (I believe :) that's better, whenever possible, to let QEMU pick the default device type by machine. The discussion happened here:

https://lists.gnu.org/archive/html/qemu-devel/2018-10/msg04582.html

Quoted 8 lines

I was expecting this change to be global (in a single patch previous to the target-specific patches). If you prefer to do it a target at a time I'm OK with it but you should add a comment about it in the commit IMHO.

Regards,

Phil.

You're right, that's a better approach. I've made this change part of the set_console() changes.

Thanks!
- Cleber.

#88 lore raw
In reply to Cleber Rosa
Quoted 9 lines

From: Cleber Rosa <[email protected]>

class My(Test):

      def test_nx_cpu_flag(self):
          """
          :avocado: tags=arch:x86_64
          """
          test_code()
Quoted 4 lines

The value of the "arch" key, in this case, "x86_64" will be used when selecting the QEMU binary to use in the test. At the same time, if "x86_64-softmmu" is not a built target, the test will be filtered out by "make check-acceptance"[3].

I think, the term "arch" is a little problematic in QEMU parlance. IMHO, "target" should be used instead. ("arch" is used in Linux kernel community)

The overall structure of the "tags" should be a little different. My suggestion:

"target"
"isa" (instruction set architecture, determeines how the kernel and rootfs are built) "cpu"
"machine"

This would allow clear view what a particular acceptance test tests, and will enforce consistency and clarity in the test organization across the board.

That said, I am very excited about this series.

Regards,
Aleksandar

#89 lore raw

Hi Aleksandar,

Quoted 9 lines · Aleksandar Markovic
Quoted 8 lines

From: Cleber Rosa <[email protected]>

class My(Test):

      def test_nx_cpu_flag(self):
          """
          :avocado: tags=arch:x86_64
          """
          test_code()
Quoted 7 lines
Quoted 4 lines

The value of the "arch" key, in this case, "x86_64" will be used when selecting the QEMU binary to use in the test. At the same time, if "x86_64-softmmu" is not a built target, the test will be filtered out by "make check-acceptance"[3].

I think, the term "arch" is a little problematic in QEMU parlance. IMHO, "target" should be used instead. ("arch" is used in Linux kernel community)

Using target_arch/host_arch might be more explicit, but host_arch is not very relevant regarding Avocado (except to choose the correct TCG binary), so usually arch implies target_arch.

I mean, it is unlikely you enforce --host_arch on the command line to run tests, this using --arch instead of --target_arch looks OK to me, since host_arch can be guessed.

Quoted 6 lines

The overall structure of the "tags" should be a little different. My suggestion:

"target"
"isa" (instruction set architecture, determeines how the kernel and rootfs are built)

In QEMU, "isa" is implicit with CPU, isnt' it?

I.e. I use:

$ mips64el-softmmu/qemu-system-mips64el -M Malta -cpu mips64dspr2
Quoted 5 lines

"cpu"
"machine"

This would allow clear view what a particular acceptance test tests, and will enforce consistency and clarity in the test organization across the board.

Maybe the problem you are pointing out is not Avocado test organization but QEMU CPU organization... (which also bother me with boards able to use different SoC, with different peripherals or cpus).

What is your idea on passing arch/cpu/isa to start QEMU?

Quoted 2 lines

That said, I am very excited about this series.

Me too :)

Regards,

Phil.

#90 lore raw
Quoted 20 lines · Philippe Mathieu-Daudé

Hi Aleksandar,

Quoted 9 lines · Aleksandar Markovic
Quoted 8 lines

From: Cleber Rosa <[email protected]>

class My(Test):

      def test_nx_cpu_flag(self):
          """
          :avocado: tags=arch:x86_64
          """
          test_code()
Quoted 7 lines
Quoted 4 lines

The value of the "arch" key, in this case, "x86_64" will be used when selecting the QEMU binary to use in the test. At the same time, if "x86_64-softmmu" is not a built target, the test will be filtered out by "make check-acceptance"[3].

I think, the term "arch" is a little problematic in QEMU parlance. IMHO, "target" should be used instead. ("arch" is used in Linux kernel community)

Quoted 8 lines

Using target_arch/host_arch might be more explicit, but host_arch is not very relevant regarding Avocado (except to choose the correct TCG binary), so usually arch implies target_arch.

I mean, it is unlikely you enforce --host_arch on the command line to run tests, this using --arch instead of --target_arch looks OK to me, since host_arch can be guessed.

Naming things is hard, so this is a valid discussion. But, I have to say that I also find "arch" in this context to be descriptive enough.

Quoted 26 lines
Quoted 6 lines

The overall structure of the "tags" should be a little different. My suggestion:

"target"
"isa" (instruction set architecture, determeines how the kernel and rootfs are built)

In QEMU, "isa" is implicit with CPU, isnt' it?

I.e. I use:

$ mips64el-softmmu/qemu-system-mips64el -M Malta -cpu mips64dspr2
Quoted 5 lines

"cpu"
"machine"

This would allow clear view what a particular acceptance test tests, and will enforce consistency and clarity in the test organization across the board.

Maybe the problem you are pointing out is not Avocado test organization but QEMU CPU organization... (which also bother me with boards able to use different SoC, with different peripherals or cpus).

What is your idea on passing arch/cpu/isa to start QEMU?

Quoted 1 line
Quoted 8 lines
Quoted 1 line

That said, I am very excited about this series.

Me too :)

Regards,

Phil.

Thanks! For the encouragement, help and input!

- Cleber.

#91 lore raw
Quoted 5 lines
Quoted 2 lines

I think, the term "arch" is a little problematic in QEMU parlance. IMHO, "target" should be used instead. ("arch" is used in Linux kernel community)

Naming things is hard, so this is a valid discussion. But, I have to say that I also find "arch" in this context to be descriptive enough.

This is not a "naming" problem, but a fundamental terminology convention used in QEMU. I am truly dissapointed that you chose to disrespect it.

Aleksandar

#92 lore raw
Quoted 10 lines · Aleksandar Markovic
Quoted 5 lines
Quoted 2 lines

I think, the term "arch" is a little problematic in QEMU parlance. IMHO, "target" should be used instead. ("arch" is used in Linux kernel community)

Naming things is hard, so this is a valid discussion. But, I have to say that I also find "arch" in this context to be descriptive enough.

This is not a "naming" problem, but a fundamental terminology convention used in QEMU. I am truly dissapointed that you chose to disrespect it.

Aleksandar

Hi Aleksandar,

I'm basically listening to more experienced people on the matter, as I clearly lack the background on the "terminology conventions", and thus, I'm unable to make the best naming choices by myself.

Now, we have had participation from few people, and there doesn't seem to be consensus. So, to make it clear, I didn't *choose* to disrespect it, I chose to respect how I perceived the community input.

If this makes you truly disappointed, please ask more people to give their input, and I'll gladly steer the direction of this humble ship, I mean, patch series. :)

Regards,
- Cleber.

#93 lore raw
In reply to Cleber Rosa
Patchew URL: https://patchew.org/QEMU/[email protected]/

Patchew URL: https://patchew.org/QEMU/[email protected]/

Hi,

This series seems to have some coding style problems. See output below for more information:

Subject: [Qemu-devel] [PATCH 00/18] Acceptance Tests: target architecture support
Type: series
Message-id: [email protected]

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
Switched to a new branch 'test'
a27efa3fec Boot Linux Console Test: add a test for alpha + clipper 9c5d101f9b Boot Linux Console Test: add a test for s390x + s390-ccw-virtio 7d584982d7 Boot Linux Console Test: add a test for arm + virt 039f09b5e5 Boot Linux Console Test: add a test for aarch64 + virt a52a2dd405 Boot Linux Console Test: add a test for ppc64 + pseries 1995edd0d0 Boot Linux Console Test: add a test for mips64el + malta cbd4fb29d7 Boot Linux Console Test: add a test for mips + malta 62ca82a551 scripts/qemu.py: support adding a console with the default serial device 8f31626ddd Boot Linux Console Test: refactor the console watcher into utility method c257e982f1 Boot Linux Console Test: update the x86_64 kernel 9859ac0034 Boot Linux Console Test: rename the x86_64 after the arch and machine 4fa0bc613a Acceptance tests: look for target architecture in test tags first 6051a67e66 Acceptance tests: use "arch:" tag to filter target specific tests 8e86083ca9 Acceptance tests: introduce arch parameter and attribute 6ec8dc9d2e Acceptance tests: fix doc reference to avocado_qemu directory 5c8a9897ba Acceptance tests: improve docstring on pick_default_qemu_bin() 0e98f722c6 Acceptance tests: show avocado test execution by default 7f31c568c2 scripts/qemu.py: log QEMU launch command line

=== OUTPUT BEGIN ===
1/18 Checking commit 7f31c568c2e0 (scripts/qemu.py: log QEMU launch command line) 2/18 Checking commit 0e98f722c669 (Acceptance tests: show avocado test execution by default) 3/18 Checking commit 5c8a9897ba6a (Acceptance tests: improve docstring on pick_default_qemu_bin()) 4/18 Checking commit 6ec8dc9d2eed (Acceptance tests: fix doc reference to avocado_qemu directory) 5/18 Checking commit 8e86083ca9e8 (Acceptance tests: introduce arch parameter and attribute)
WARNING: line over 80 characters
#96: FILE: tests/acceptance/avocado_qemu/__init__.py:58:
+ default=pick_default_qemu_bin(self.arch))

total: 0 errors, 1 warnings, 64 lines checked

Patch 5/18 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
6/18 Checking commit 6051a67e66ea (Acceptance tests: use "arch:" tag to filter target specific tests) 7/18 Checking commit 4fa0bc613a2e (Acceptance tests: look for target architecture in test tags first) 8/18 Checking commit 9859ac003400 (Boot Linux Console Test: rename the x86_64 after the arch and machine) 9/18 Checking commit c257e982f129 (Boot Linux Console Test: update the x86_64 kernel)
10/18 Checking commit 8f31626ddd45 (Boot Linux Console Test: refactor the console watcher into utility method) 11/18 Checking commit 62ca82a551c2 (scripts/qemu.py: support adding a console with the default serial device) 12/18 Checking commit cbd4fb29d7c8 (Boot Linux Console Test: add a test for mips + malta) 13/18 Checking commit 1995edd0d009 (Boot Linux Console Test: add a test for mips64el + malta)
ERROR: line over 90 characters
#66: FILE: tests/acceptance/boot_linux_console.py:95:
+ [1] https://kernel-team.pages.debian.net/kernel-handbook/ch-common-tasks.html#s-common-official

ERROR: line over 90 characters
#67: FILE: tests/acceptance/boot_linux_console.py:96:
+ [2] http://snapshot.debian.org/package/linux-2.6/2.6.32-48/#linux-source-2.6.32_2.6.32-48

total: 2 errors, 0 warnings, 58 lines checked

Patch 13/18 has style problems, please review. If any of these errors are false positives report them to the maintainer, see CHECKPATCH in MAINTAINERS.

14/18 Checking commit a52a2dd405ae (Boot Linux Console Test: add a test for ppc64 + pseries) 15/18 Checking commit 039f09b5e5a4 (Boot Linux Console Test: add a test for aarch64 + virt) 16/18 Checking commit 7d584982d75c (Boot Linux Console Test: add a test for arm + virt) 17/18 Checking commit 9c5d101f9bfb (Boot Linux Console Test: add a test for s390x + s390-ccw-virtio) 18/18 Checking commit a27efa3fec59 (Boot Linux Console Test: add a test for alpha + clipper) === OUTPUT END ===

Test command exited with code: 1

The full log is available at
http://patchew.org/logs/[email protected]/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [http://patchew.org/].
Please send your feedback to [email protected]