Convert avocado tests to normal Python unittests
Subsystems (3)
v3 cover Convert avocado tests to normal Python unittests
This series is an update to Thomas' v2:
https://lists.nongnu.org/archive/html/qemu-devel/2024-07/msg05805.html
wherein Thomas suggested someone could do the asset caching updates while he is on vacation:
https://lists.nongnu.org/archive/html/qemu-devel/2024-07/msg06228.html
hence this posting what I'm calling a v3.
The Avocado v88 that we use in QEMU is already on a life support
system: It is not supported by upstream anymore, and with the latest versions of Python, it won't work anymore since it depends on the "imp" module that has been removed in Python 3.12.
There have been several attempts to update the test suite in QEMU to a newer version of Avocado, but so far no attempt has successfully been merged yet.
Additionally, the whole "make check" test suite in QEMU is using the meson test runner nowadays, so running the python-based tests via the Avocodo test runner looks and feels quite like an oddball, requiring the users to deal with the knowledge of multiple test runners in parallel (e.g. the timeout settings work completely differently).
So instead of trying to update the python-based test suite in QEMU to a newer version of Avocado, we should try to better integrate it with the meson test runner instead. Indeed most tests work quite nicely without the Avocado framework already, as you can see with this patch series - it does not convert all tests, just a subset so far, but this already proves that many tests only need small modifi- cations to work without Avocado.
Only tests that use the LinuxTest / LinuxDistro and LinuxSSHMixIn classes (e.g. based on cloud-init images or using SSH) really depend on the Avocado framework, so we'd need a solution for those if we want to continue using them. One solution might be to simply use the required functions from avocado.utils for these tests, and still run them via the meson test runner instead, but that needs some further investigation that will be done later.
Now if you want to try out these patches: Apply the patches, then recompile and then run:
make check-functional
You can also run single targets e.g. with:
make check-functional-ppc
You can also run the tests without any test runner now by setting the PYTHONPATH environment variable to the "python" folder of your source tree, and by specifying the build directory via QEMU_BUILD_ROOT (if autodetection fails) and by specifying the QEMU binary via QEMU_TEST_QEMU_BINARY. For example:
export PYTHONPATH=$HOME/qemu/python
export QEMU_TEST_QEMU_BINARY=qemu-system-x86_64
export QEMU_BUILD_ROOT=$HOME/qemu/build
~/qemu/tests/functional/test_virtio_version.py
The logs of the tests can be found in the build directory under tests/functional/<arch>/<testname> - console log and general logs will be put in separate files there.
Still to be done: Update the documentation for this new test framework.
v3:
- Split tests/functional/__init__.py into multiple files
- Introduce an 'Asset' class to handle downloading & caching
- Convert all tests to use new Asset class instead of 'fetch_asset'
- Ensure 'make check-functional' pre-caches all assets before running the tests, to avoid meson test timeouts due to slow downloads
- Fix bios bits test logging in verbose mode to not print subprocess output to stdout, since it confuses TAP parsing
v2:
- Addressed review feedback from v1
- Add pycotap as a wheel instead of trying to install it on demand when running "make check-functional" (works much better now!)
- Converted much more tests
- Lots of other small improvements here and there
RFC -> v1:
- Now using pycotap for running the tests instead of "pytest"
- Change the name from "tests/pytest" to "tests/functional" accordingly
- Make it possible to run the tests directly
- Use Python's urllib instead of wget for downloading
- Lots of makefile / meson integration improvements
- Converted more tests
- Update MAINTAINERS file accordingly
- Added a patch to run check-functional in the gitlab-CI
- ... lots of other changes I forgot about ... in fact, I changed so many things that I also did not dare to pick up the Reviewed-bys from the RFC
Daniel P. Berrangé (2):
tests/functional: add a module for handling asset download & caching tests/functional: enable pre-emptive caching of assets
Thomas Huth (22):
python: Install pycotap in our venv if necessary tests/functional: Add base classes for the upcoming pytest-based tests tests/functional: Set up logging
tests/Makefile.include: Increase the level of indentation in the help
text
tests/functional: Prepare the meson build system for the functional
tests
tests/functional: Convert simple avocado tests into standalone python
tests
tests/functional: Convert avocado tests that just need a small
adjustment
tests/functional: Convert some tests that download files via
fetch_asset()
tests/functional: Add a function for extracting files from an archive tests/functional: Convert some avocado tests that needed
avocado.utils.archive
tests/functional: Convert the s390x avocado tests into standalone
tests
tests/functional: Convert the x86_cpu_model_versions test tests/functional: Convert the microblaze avocado tests into standalone
tests
tests/functional: Convert the riscv_opensbi avocado test into a
standalone test
tests/functional: Convert the virtio_gpu avocado test into a
standalone test
tests/functional: Convert most ppc avocado tests into standalone tests tests/functional: Convert the ppc_amiga avocado test into a standalone
test
tests/functional: Convert the ppc_hv avocado test into a standalone
test
tests/functional: Convert the m68k nextcube test with tesseract tests/functional: Convert the acpi-bits test into a standalone test tests/functional: Convert the rx_gdbsim avocado test into a standalone
test
gitlab-ci: Add "check-functional" to the build tests
.gitlab-ci.d/buildtest-template.yml | 3 +-
.gitlab-ci.d/buildtest.yml | 60 +++---
MAINTAINERS | 32 +--
python/wheels/pycotap-1.3.1-py3-none-any.whl | Bin 0 -> 5119 bytes
pythondeps.toml | 1 +
tests/Makefile.include | 42 ++--
tests/avocado/machine_microblaze.py | 61 ------
tests/avocado/machine_mips_loongson3v.py | 39 ----
tests/avocado/machine_sparc64_sun4u.py | 36 ----
tests/avocado/ppc_amiga.py | 38 ----
tests/avocado/riscv_opensbi.py | 63 ------
tests/avocado/tesseract_utils.py | 46 -----
.../acpi-bits/bits-config/bits-cfg.txt | 0
.../acpi-bits/bits-tests/smbios.py2 | 0
.../acpi-bits/bits-tests/smilatency.py2 | 0
.../acpi-bits/bits-tests/testacpi.py2 | 0
.../acpi-bits/bits-tests/testcpuid.py2 | 0
tests/functional/meson.build | 180 +++++++++++++++++
tests/functional/qemu_test/__init__.py | 14 ++
tests/functional/qemu_test/asset.py | 130 ++++++++++++
tests/functional/qemu_test/cmd.py | 171 ++++++++++++++++
tests/functional/qemu_test/config.py | 36 ++++
tests/functional/qemu_test/tesseract.py | 35 ++++
tests/functional/qemu_test/testcase.py | 187 ++++++++++++++++++
tests/functional/qemu_test/utils.py | 47 +++++
.../test_acpi_bits.py} | 81 ++++----
.../test_arm_canona1100.py} | 30 +--
.../test_arm_n8x0.py} | 36 ++--
.../test_avr_mega2560.py} | 22 ++-
.../test_cpu_queries.py} | 7 +-
.../test_empty_cpu_model.py} | 7 +-
.../test_info_usernet.py} | 11 +-
.../test_loongarch64_virt.py} | 46 +++--
.../test_m68k_nextcube.py} | 29 +--
.../test_mem_addr_space.py} | 52 +----
.../functional/test_microblaze_s3adsp1800.py | 39 ++++
.../test_microblazeel_s3adsp1800.py | 42 ++++
tests/functional/test_mips64el_loongson3v.py | 39 ++++
.../test_netdev_ethtool.py} | 50 ++---
.../test_pc_cpu_hotplug_props.py} | 11 +-
.../test_ppc64_hv.py} | 48 ++---
.../test_ppc64_powernv.py} | 56 ++----
.../test_ppc64_pseries.py} | 56 ++----
.../ppc_405.py => functional/test_ppc_405.py} | 28 +--
.../test_ppc_40p.py} | 63 +++---
.../test_ppc_74xx.py} | 74 +++----
tests/functional/test_ppc_amiga.py | 42 ++++
.../test_ppc_bamboo.py} | 32 +--
.../test_ppc_mpc8544ds.py} | 28 +--
.../test_ppc_virtex_ml507.py} | 28 +--
tests/functional/test_riscv_opensbi.py | 36 ++++
.../test_rx_gdbsim.py} | 57 +++---
.../test_s390x_ccw_virtio.py} | 79 ++++----
.../test_s390x_topology.py} | 86 ++++----
tests/functional/test_sparc64_sun4u.py | 40 ++++
.../version.py => functional/test_version.py} | 13 +-
.../test_virtio_gpu.py} | 64 +++---
.../test_virtio_version.py} | 8 +-
.../test_x86_cpu_model_versions.py} | 63 ++----
tests/meson.build | 1 +
60 files changed, 1631 insertions(+), 994 deletions(-)
create mode 100644 python/wheels/pycotap-1.3.1-py3-none-any.whl
delete mode 100644 tests/avocado/machine_microblaze.py
delete mode 100644 tests/avocado/machine_mips_loongson3v.py
delete mode 100644 tests/avocado/machine_sparc64_sun4u.py
delete mode 100644 tests/avocado/ppc_amiga.py
delete mode 100644 tests/avocado/riscv_opensbi.py
delete mode 100644 tests/avocado/tesseract_utils.py
rename tests/{avocado => functional}/acpi-bits/bits-config/bits-cfg.txt (100%)
rename tests/{avocado => functional}/acpi-bits/bits-tests/smbios.py2 (100%)
rename tests/{avocado => functional}/acpi-bits/bits-tests/smilatency.py2 (100%)
rename tests/{avocado => functional}/acpi-bits/bits-tests/testacpi.py2 (100%)
rename tests/{avocado => functional}/acpi-bits/bits-tests/testcpuid.py2 (100%)
create mode 100644 tests/functional/meson.build
create mode 100644 tests/functional/qemu_test/__init__.py
create mode 100644 tests/functional/qemu_test/asset.py
create mode 100644 tests/functional/qemu_test/cmd.py
create mode 100644 tests/functional/qemu_test/config.py
create mode 100644 tests/functional/qemu_test/tesseract.py
create mode 100644 tests/functional/qemu_test/testcase.py
create mode 100644 tests/functional/qemu_test/utils.py
rename tests/{avocado/acpi-bits.py => functional/test_acpi_bits.py} (86%)
mode change 100644 => 100755
rename tests/{avocado/machine_arm_canona1100.py => functional/test_arm_canona1100.py} (52%)
mode change 100644 => 100755
rename tests/{avocado/machine_arm_n8x0.py => functional/test_arm_n8x0.py} (51%)
mode change 100644 => 100755
rename tests/{avocado/machine_avr6.py => functional/test_avr_mega2560.py} (73%)
mode change 100644 => 100755
rename tests/{avocado/cpu_queries.py => functional/test_cpu_queries.py} (89%)
mode change 100644 => 100755
rename tests/{avocado/empty_cpu_model.py => functional/test_empty_cpu_model.py} (84%)
mode change 100644 => 100755
rename tests/{avocado/info_usernet.py => functional/test_info_usernet.py} (87%)
mode change 100644 => 100755
rename tests/{avocado/machine_loongarch.py => functional/test_loongarch64_virt.py} (54%)
mode change 100644 => 100755
rename tests/{avocado/machine_m68k_nextcube.py => functional/test_m68k_nextcube.py} (76%)
mode change 100644 => 100755
rename tests/{avocado/mem-addr-space-check.py => functional/test_mem_addr_space.py} (93%)
mode change 100644 => 100755
create mode 100755 tests/functional/test_microblaze_s3adsp1800.py
create mode 100755 tests/functional/test_microblazeel_s3adsp1800.py
create mode 100755 tests/functional/test_mips64el_loongson3v.py
rename tests/{avocado/netdev-ethtool.py => functional/test_netdev_ethtool.py} (66%)
mode change 100644 => 100755
rename tests/{avocado/pc_cpu_hotplug_props.py => functional/test_pc_cpu_hotplug_props.py} (90%)
mode change 100644 => 100755
rename tests/{avocado/ppc_hv_tests.py => functional/test_ppc64_hv.py} (88%)
mode change 100644 => 100755
rename tests/{avocado/ppc_powernv.py => functional/test_ppc64_powernv.py} (71%)
mode change 100644 => 100755
rename tests/{avocado/ppc_pseries.py => functional/test_ppc64_pseries.py} (76%)
mode change 100644 => 100755
rename tests/{avocado/ppc_405.py => functional/test_ppc_405.py} (53%)
mode change 100644 => 100755
rename tests/{avocado/ppc_prep_40p.py => functional/test_ppc_40p.py} (54%)
mode change 100644 => 100755
rename tests/{avocado/ppc_74xx.py => functional/test_ppc_74xx.py} (74%)
mode change 100644 => 100755
create mode 100755 tests/functional/test_ppc_amiga.py
rename tests/{avocado/ppc_bamboo.py => functional/test_ppc_bamboo.py} (60%)
mode change 100644 => 100755
rename tests/{avocado/ppc_mpc8544ds.py => functional/test_ppc_mpc8544ds.py} (55%)
mode change 100644 => 100755
rename tests/{avocado/ppc_virtex_ml507.py => functional/test_ppc_virtex_ml507.py} (60%)
mode change 100644 => 100755
create mode 100755 tests/functional/test_riscv_opensbi.py
rename tests/{avocado/machine_rx_gdbsim.py => functional/test_rx_gdbsim.py} (54%)
mode change 100644 => 100755
rename tests/{avocado/machine_s390_ccw_virtio.py => functional/test_s390x_ccw_virtio.py} (85%)
mode change 100644 => 100755
rename tests/{avocado/s390_topology.py => functional/test_s390x_topology.py} (88%)
mode change 100644 => 100755
create mode 100755 tests/functional/test_sparc64_sun4u.py
rename tests/{avocado/version.py => functional/test_version.py} (78%)
mode change 100644 => 100755
rename tests/{avocado/virtio-gpu.py => functional/test_virtio_gpu.py} (73%)
mode change 100644 => 100755
rename tests/{avocado/virtio_version.py => functional/test_virtio_version.py} (98%)
mode change 100644 => 100755
rename tests/{avocado/x86_cpu_model_versions.py => functional/test_x86_cpu_model_versions.py} (92%)
mode change 100644 => 100755Signature · 1 line
2.45.2
PATCH v3 1/24 python: Install pycotap in our venv if necessary
From: Thomas Huth <[email protected]>
The upcoming functional tests will require pycotap for providing TAP output from the python-based tests. Since we want to be able to run some of the tests offline by default, too, let's install it along with meson in our venv if necessary (it's size is only 5 kB, so adding the wheel here should not really be a problem).
The wheel file has been obtained with:
pip download --only-binary :all: --dest . --no-cache pycotap
--- python/wheels/pycotap-1.3.1-py3-none-any.whl | Bin 0 -> 5119 bytes pythondeps.toml | 1 + 2 files changed, 1 insertion(+) create mode 100644 python/wheels/pycotap-1.3.1-py3-none-any.whl
python/wheels/pycotap-1.3.1-py3-none-any.whl new file binary +0 -0
pythondeps.toml +1 -0
@@ -20,6 +20,7 @@ [meson] # The install key should match the version in python/wheels/ meson = { accepted = ">=1.1.0", installed = "1.2.3", canary = "meson" } +pycotap = { accepted = ">=1.1.0", installed = "1.3.1" } [docs] # Please keep the installed versions in sync with docs/requirements.txt
Signature · 1 line
2.45.2
Re: [PATCH v3 01/24] python: Install pycotap in our venv if necessary
Quoted 10 lines · Daniel P. Berrangé
From: Thomas Huth <[email protected]>
The upcoming functional tests will require pycotap for providing TAP output from the python-based tests. Since we want to be able to run some of the tests offline by default, too, let's install it along with meson in our venv if necessary (it's size is only 5 kB, so adding the wheel here should not really be a problem).
The wheel file has been obtained with:
Quoted 8 lines
pip download --only-binary :all: --dest . --no-cache pycotapSigned-off-by Thomas Huth <[email protected]>--- python/wheels/pycotap-1.3.1-py3-none-any.whl | Bin 0 -> 5119 bytes pythondeps.toml | 1 + 2 files changed, 1 insertion(+)create mode 100644 python/wheels/pycotap-1.3.1-py3-none-any.whl
PATCH v3 2/24 tests/functional: Add base classes for the upcoming pytest-based tests
In reply to Daniel P. BerrangéFrom: Thomas Huth <[email protected]>
The file is mostly a copy of the tests/avocado/avocado_qemu/__init__.py file with some adjustments to get rid of the Avocado dependencies (i.e. we also have to drop the LinuxSSHMixIn and LinuxTest for now).
The emulator binary and build directory are now passed via environment variables that will be set via meson.build later.
--- tests/functional/qemu_test/__init__.py | 13 ++ tests/functional/qemu_test/cmd.py | 171 +++++++++++++++++++++++++ tests/functional/qemu_test/config.py | 36 ++++++ tests/functional/qemu_test/testcase.py | 154 ++++++++++++++++++++++ 4 files changed, 374 insertions(+) create mode 100644 tests/functional/qemu_test/__init__.py create mode 100644 tests/functional/qemu_test/cmd.py create mode 100644 tests/functional/qemu_test/config.py create mode 100644 tests/functional/qemu_test/testcase.py
tests/functional/qemu_test/__init__.py new file +13 -0
@@ -0,0 +1,13 @@ +# Test class and utilities for functional tests +# +# Copyright 2024 Red Hat, Inc. +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + + +from .config import BUILD_DIR +from .cmd import has_cmd, has_cmds, run_cmd, is_readable_executable_file, \ + interrupt_interactive_console_until_pattern, wait_for_console_pattern, \ + exec_command, exec_command_and_wait_for_pattern +from .testcase import QemuSystemTest, QemuBaseTest
tests/functional/qemu_test/cmd.py new file +171 -0
171 lines not on this page · raw message
tests/functional/qemu_test/config.py new file +36 -0
@@ -0,0 +1,36 @@ +# Test class and utilities for functional tests +# +# Copyright 2018, 2024 Red Hat, Inc. +# +# Original Author (Avocado-based tests): +# Cleber Rosa <[email protected]> +# +# Adaption for standalone version: +# Thomas Huth <[email protected]> +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + +import os +from pathlib import Path + + +def _source_dir(): + # Determine top-level directory of the QEMU sources + return Path(__file__).parent.parent.parent.parent + +def _build_dir(): + root = os.getenv('QEMU_BUILD_ROOT') + if root is not None: + return Path(root) + # Makefile.mtest only exists in build dir, so if it is available, use CWD + if os.path.exists('Makefile.mtest'): + return Path(os.getcwd()) + + root = os.path.join(_source_dir(), 'build') + if os.path.exists(root): + return Path(root) + + raise Exception("Cannot identify build dir, set QEMU_BUILD_ROOT") + +BUILD_DIR = _build_dir()
tests/functional/qemu_test/testcase.py new file +154 -0
154 lines not on this page · raw message
Signature · 1 line
2.45.2
Re: [PATCH v3 02/24] tests/functional: Add base classes for the upcoming pytest-based tests
Quoted 18 lines · Daniel P. Berrangé
From: Thomas Huth <[email protected]>
The file is mostly a copy of the
tests/avocado/avocado_qemu/__init__.pyfile with some adjustments to get rid of the Avocado dependencies (i.e. we also have to drop the LinuxSSHMixIn and LinuxTest for now).The emulator binary and build directory are now passed via environment variables that will be set via meson.build later.
Signed-off-by Thomas Huth <[email protected]>[DB: split __init__.py into multiple files]Signed-off-by Daniel P. Berrangé <[email protected]>--- tests/functional/qemu_test/__init__.py | 13 ++ tests/functional/qemu_test/cmd.py | 171 +++++++++++++++++++++++++ tests/functional/qemu_test/config.py | 36 ++++++ tests/functional/qemu_test/testcase.py | 154 ++++++++++++++++++++++ 4 files changed, 374 insertions(+)
Quoted 4 lines
create mode 100644
tests/functional/qemu_test/__init__.pycreate mode 100644tests/functional/qemu_test/cmd.pycreate mode 100644tests/functional/qemu_test/config.pycreate mode 100644tests/functional/qemu_test/testcase.py
Please squash:
-- >8 --
MAINTAINERS +1 -0
@@ -4127,6 +4137,7 @@ F: .travis.yml F: docs/devel/ci* F: scripts/ci/ F: tests/docker/ +F: tests/functional/ F: tests/vm/ F: tests/lcitool/ F: tests/avocado/tuxrun_baselines.py
---
Re: [PATCH v3 02/24] tests/functional: Add base classes for the upcoming pytest-based tests
In reply to Daniel P. BerrangéQuoted 8 lines · Daniel P. Berrangé
From: Thomas Huth <[email protected]>
The file is mostly a copy of the
tests/avocado/avocado_qemu/__init__.pyfile with some adjustments to get rid of the Avocado dependencies (i.e. we also have to drop the LinuxSSHMixIn and LinuxTest for now).The emulator binary and build directory are now passed via environment variables that will be set via meson.build later.
<snip>
Quoted 39 lines
tests/functional/qemu_test/testcase.py new file +33
-0@@ -0,0 +1,154 @@ +# Test class and utilities for functional tests +# +# Copyright 2018, 2024 Red Hat, Inc. +# +# Original Author (Avocado-based tests): +# Cleber Rosa <[email protected]> +# +# Adaption for standalone version: +# Thomas Huth <[email protected]> +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + +import logging +import os +import pycotap +import sys +import unittest +import uuid + +from qemu.machine import QEMUMachine +from qemu.utils import kvm_available, tcg_available + +from .cmd import run_cmd +from .config import BUILD_DIR + + +class QemuBaseTest(unittest.TestCase): + + qemu_bin = os.getenv('QEMU_TEST_QEMU_BINARY') + arch = None + + workdir = None
tests/functional/qemu_test/testcase.py line 43
Quoted 10 lines
tests/functional/qemu_test/testcase.py new file +10
-0+ log = logging.getLogger('qemu-test') + + def setUp(self, bin_prefix): + self.assertIsNotNone(self.qemu_bin, 'QEMU_TEST_QEMU_BINARY must be set') + self.arch = self.qemu_bin.split('-')[-1] + + self.workdir = os.path.join(BUILD_DIR, 'tests/functional', self.arch, + self.id()) + if not os.path.exists(self.workdir): + os.makedirs(self.workdir)
This is racy under --repeat:
==================================== 1/4 =====================================
test: qemu:func-quick+func-riscv64 / func-riscv64-riscv_opensbi start time: 14:16:52 duration: 0.06s result: exit status 1 command: PYTHONPATH=/home/alex/lsrc/qemu.git/python:/home/alex/lsrc/qemu.git/tests/functional QEMU_BUILD_ROOT=/home/alex/lsrc/qemu.git/builds/all QEMU_TEST_QEMU_BINARY=
/home/alex/lsrc/qemu.git/builds/all/qemu-system-riscv64 MALLOC_PERTURB_=71 QEMU_TEST_QEMU_IMG=/home/alex/lsrc/qemu.git/builds/all/qemu-img /home/alex/lsrc/qemu.git/builds/al l/pyvenv/bin/python3 /home/alex/lsrc/qemu.git/tests/functional/test_riscv_opensbi.py
----------------------------------- stdout -----------------------------------
TAP version 13
not ok 1 test_riscv_opensbi.RiscvOpenSBI.test_riscv_sifive_u
not ok 2 test_riscv_opensbi.RiscvOpenSBI.test_riscv_spike
not ok 3 test_riscv_opensbi.RiscvOpenSBI.test_riscv_virt
1..3
----------------------------------- stderr -----------------------------------
Traceback (most recent call last):
File "/home/alex/lsrc/qemu.git/tests/functional/qemu_test/testcase.py", line 85, in setUp
super().setUp('qemu-system-')
File "/home/alex/lsrc/qemu.git/tests/functional/qemu_test/testcase.py", line 45, in setUp
os.makedirs(self.workdir)
File "<frozen os>", line 225, in makedirs
FileExistsError: [Errno 17] File exists: '/home/alex/lsrc/qemu.git/builds/all/tests/functional/riscv64/test_riscv_opensbi.RiscvOpenSBI.test_riscv_sifive_u'
Traceback (most recent call last):
File "/home/alex/lsrc/qemu.git/tests/functional/qemu_test/testcase.py", line 85, in setUp
super().setUp('qemu-system-')
File "/home/alex/lsrc/qemu.git/tests/functional/qemu_test/testcase.py", line 45, in setUp
os.makedirs(self.workdir)
File "<frozen os>", line 225, in makedirs
FileExistsError: [Errno 17] File exists: '/home/alex/lsrc/qemu.git/builds/all/tests/functional/riscv64/test_riscv_opensbi.RiscvOpenSBI.test_riscv_spike'
Traceback (most recent call last):
File "/home/alex/lsrc/qemu.git/tests/functional/qemu_test/testcase.py", line 85, in setUp
super().setUp('qemu-system-')
File "/home/alex/lsrc/qemu.git/tests/functional/qemu_test/testcase.py", line 45, in setUp
os.makedirs(self.workdir)
File "<frozen os>", line 225, in makedirs
FileExistsError: [Errno 17] File exists: '/home/alex/lsrc/qemu.git/builds/all/tests/functional/riscv64/test_riscv_opensbi.RiscvOpenSBI.test_riscv_virt'
(test program exited with status code 1)
We could just:
os.makedirs(self.workdir, exist_ok = True)
<snip>
Signature · 2 lines
Alex Bennée Virtualisation Tech Lead @ Linaro
Re: [PATCH v3 02/24] tests/functional: Add base classes for the upcoming pytest-based tests
Quoted 103 lines · Alex Bennée
Quoted 8 lines · Daniel P. Berrangé
From: Thomas Huth <[email protected]>
The file is mostly a copy of the
tests/avocado/avocado_qemu/__init__.pyfile with some adjustments to get rid of the Avocado dependencies (i.e. we also have to drop the LinuxSSHMixIn and LinuxTest for now).The emulator binary and build directory are now passed via environment variables that will be set via meson.build later.
<snip>
Quoted 39 lines
tests/functional/qemu_test/testcase.py new file +33
-0@@ -0,0 +1,154 @@ +# Test class and utilities for functional tests +# +# Copyright 2018, 2024 Red Hat, Inc. +# +# Original Author (Avocado-based tests): +# Cleber Rosa <[email protected]> +# +# Adaption for standalone version: +# Thomas Huth <[email protected]> +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + +import logging +import os +import pycotap +import sys +import unittest +import uuid + +from qemu.machine import QEMUMachine +from qemu.utils import kvm_available, tcg_available + +from .cmd import run_cmd +from .config import BUILD_DIR + + +class QemuBaseTest(unittest.TestCase): + + qemu_bin = os.getenv('QEMU_TEST_QEMU_BINARY') + arch = None + + workdir = Nonetests/functional/qemu_test/testcase.py line 43
Quoted 10 lines
tests/functional/qemu_test/testcase.py new file +10
-0+ log = logging.getLogger('qemu-test') + + def setUp(self, bin_prefix): + self.assertIsNotNone(self.qemu_bin, 'QEMU_TEST_QEMU_BINARY must be set') + self.arch = self.qemu_bin.split('-')[-1] + + self.workdir = os.path.join(BUILD_DIR, 'tests/functional', self.arch, + self.id()) + if not os.path.exists(self.workdir): + os.makedirs(self.workdir)This is racy under --repeat:
==================================== 1/4 =====================================
test: qemu:func-quick+func-riscv64 / func-riscv64-riscv_opensbi start time: 14:16:52 duration: 0.06s result: exit status 1 command: PYTHONPATH=/home/alex/lsrc/qemu.git/python:/home/alex/lsrc/qemu.git/tests/functional QEMU_BUILD_ROOT=/home/alex/lsrc/qemu.git/builds/all QEMU_TEST_QEMU_BINARY=
/home/alex/lsrc/qemu.git/builds/all/qemu-system-riscv64MALLOC_PERTURB_=71 QEMU_TEST_QEMU_IMG=/home/alex/lsrc/qemu.git/builds/all/qemu-img/home/alex/lsrc/qemu.git/builds/all/pyvenv/bin/python3/home/alex/lsrc/qemu.git/tests/functional/test_riscv_opensbi.py
----------------------------------- stdout -----------------------------------
TAP version 13
not ok 1 test_riscv_opensbi.RiscvOpenSBI.test_riscv_sifive_u
not ok 2 test_riscv_opensbi.RiscvOpenSBI.test_riscv_spike
not ok 3 test_riscv_opensbi.RiscvOpenSBI.test_riscv_virt
1..3
----------------------------------- stderr -----------------------------------
Traceback (most recent call last):
File "/home/alex/lsrc/qemu.git/tests/functional/qemu_test/testcase.py", line 85, in setUp
super().setUp('qemu-system-')
File "/home/alex/lsrc/qemu.git/tests/functional/qemu_test/testcase.py", line 45, in setUp
os.makedirs(self.workdir)
File "<frozen os>", line 225, in makedirs
FileExistsError: [Errno 17] File exists: '/home/alex/lsrc/qemu.git/builds/all/tests/functional/riscv64/test_riscv_opensbi.RiscvOpenSBI.test_riscv_sifive_u'Traceback (most recent call last):
File "/home/alex/lsrc/qemu.git/tests/functional/qemu_test/testcase.py", line 85, in setUp
super().setUp('qemu-system-')
File "/home/alex/lsrc/qemu.git/tests/functional/qemu_test/testcase.py", line 45, in setUp
os.makedirs(self.workdir)
File "<frozen os>", line 225, in makedirs
FileExistsError: [Errno 17] File exists: '/home/alex/lsrc/qemu.git/builds/all/tests/functional/riscv64/test_riscv_opensbi.RiscvOpenSBI.test_riscv_spike'Traceback (most recent call last):
File "/home/alex/lsrc/qemu.git/tests/functional/qemu_test/testcase.py", line 85, in setUp
super().setUp('qemu-system-')
File "/home/alex/lsrc/qemu.git/tests/functional/qemu_test/testcase.py", line 45, in setUp
os.makedirs(self.workdir)
File "<frozen os>", line 225, in makedirs
FileExistsError: [Errno 17] File exists: '/home/alex/lsrc/qemu.git/builds/all/tests/functional/riscv64/test_riscv_opensbi.RiscvOpenSBI.test_riscv_virt'
Quoted 5 lines
(test program exited with status code 1)
We could just:
os.makedirs(self.workdir, exist_ok = True)
Yeah, that's a better approach.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [PATCH v3 02/24] tests/functional: Add base classes for the upcoming pytest-based tests
In reply to Daniel P. BerrangéQuoted 65 lines · Daniel P. Berrangé
From: Thomas Huth <[email protected]>
The file is mostly a copy of the
tests/avocado/avocado_qemu/__init__.pyfile with some adjustments to get rid of the Avocado dependencies (i.e. we also have to drop the LinuxSSHMixIn and LinuxTest for now).The emulator binary and build directory are now passed via environment variables that will be set via meson.build later.
Signed-off-by Thomas Huth <[email protected]>[DB: split __init__.py into multiple files]Signed-off-by Daniel P. Berrangé <[email protected]>--- tests/functional/qemu_test/__init__.py | 13 ++ tests/functional/qemu_test/cmd.py | 171 +++++++++++++++++++++++++ tests/functional/qemu_test/config.py | 36 ++++++ tests/functional/qemu_test/testcase.py | 154 ++++++++++++++++++++++ 4 files changed, 374 insertions(+)create mode 100644
tests/functional/qemu_test/__init__.pycreate mode 100644tests/functional/qemu_test/cmd.pycreate mode 100644tests/functional/qemu_test/config.pycreate mode 100644tests/functional/qemu_test/testcase.pytests/functional/qemu_test/testcase.py new file +35
-0@@ -0,0 +1,154 @@ +# Test class and utilities for functional tests +# +# Copyright 2018, 2024 Red Hat, Inc. +# +# Original Author (Avocado-based tests): +# Cleber Rosa <[email protected]> +# +# Adaption for standalone version: +# Thomas Huth <[email protected]> +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + +import logging +import os +import pycotap +import sys +import unittest +import uuid + +from qemu.machine import QEMUMachine +from qemu.utils import kvm_available, tcg_available + +from .cmd import run_cmd +from .config import BUILD_DIR + + +class QemuBaseTest(unittest.TestCase): + + qemu_bin = os.getenv('QEMU_TEST_QEMU_BINARY') + arch = None + + workdir = None + log = logging.getLogger('qemu-test') +
tests/functional/qemu_test/testcase.py line 45
Quoted 10 lines
tests/functional/qemu_test/testcase.py new file +10
-0+ def setUp(self, bin_prefix): + self.assertIsNotNone(self.qemu_bin, 'QEMU_TEST_QEMU_BINARY must be set') + self.arch = self.qemu_bin.split('-')[-1] + + self.workdir = os.path.join(BUILD_DIR, 'tests/functional', self.arch, + self.id()) + if not os.path.exists(self.workdir): + os.makedirs(self.workdir) + + def main():
[*]
tests/functional/qemu_test/testcase.py line 48
Quoted 3 lines
tests/functional/qemu_test/testcase.py new file +3
-0+ tr = pycotap.TAPTestRunner(message_log = pycotap.LogMode.LogToError, + test_output_log = pycotap.LogMode.LogToError) + path = os.path.basename(sys.argv[0])[:-3]
Moving the 'path' line in [*] simplifies patch #9 (no duplication).
Quoted 1 line
tests/functional/qemu_test/testcase.py new file +1
-0+ unittest.main(module = None, testRunner = tr, argv=["__dummy__", path])
PATCH v3 3/24 tests/functional: Set up logging
In reply to Daniel P. BerrangéFrom: Thomas Huth <[email protected]>
Create log files for each test separately, one file that contains the basic logging and one that contains the console output.
--- tests/functional/qemu_test/testcase.py | 27 +++++++++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-)
tests/functional/qemu_test/testcase.py +26 -1
@@ -31,7 +31,8 @@ class QemuBaseTest(unittest.TestCase): arch = None workdir = None - log = logging.getLogger('qemu-test') + log = None + logdir = None def setUp(self, bin_prefix): self.assertIsNotNone(self.qemu_bin, 'QEMU_TEST_QEMU_BINARY must be set')
@@ -42,6 +43,20 @@ def setUp(self, bin_prefix): if not os.path.exists(self.workdir): os.makedirs(self.workdir) + self.logdir = self.workdir + self.log = logging.getLogger('qemu-test') + self.log.setLevel(logging.DEBUG) + self._log_fh = logging.FileHandler(os.path.join(self.logdir, + 'base.log'), mode='w') + self._log_fh.setLevel(logging.DEBUG) + fileFormatter = logging.Formatter( + '%(asctime)s - %(levelname)s: %(message)s') + self._log_fh.setFormatter(fileFormatter) + self.log.addHandler(self._log_fh) + + def tearDown(self): + self.log.removeHandler(self._log_fh) + def main(): tr = pycotap.TAPTestRunner(message_log = pycotap.LogMode.LogToError, test_output_log = pycotap.LogMode.LogToError)
@@ -61,6 +76,15 @@ def setUp(self): super().setUp('qemu-system-') + console_log = logging.getLogger('console') + console_log.setLevel(logging.DEBUG) + self._console_log_fh = logging.FileHandler(os.path.join(self.workdir, + 'console.log'), mode='w') + self._console_log_fh.setLevel(logging.DEBUG) + fileFormatter = logging.Formatter('%(asctime)s: %(message)s') + self._console_log_fh.setFormatter(fileFormatter) + console_log.addHandler(self._console_log_fh) + def set_machine(self, machinename): # TODO: We should use QMP to get the list of available machines if not self._machinehelp:
@@ -151,4 +175,5 @@ def set_vm_arg(self, arg, value): def tearDown(self): for vm in self._vms.values(): vm.shutdown() + logging.getLogger('console').removeHandler(self._console_log_fh) super().tearDown()
Signature · 1 line
2.45.2
Re: [PATCH v3 03/24] tests/functional: Set up logging
Quoted 29 lines · Daniel P. Berrangé
From: Thomas Huth <[email protected]>
Create log files for each test separately, one file that contains the basic logging and one that contains the console output.
Reviewed-by Daniel P. Berrangé <[email protected]>Signed-off-by Thomas Huth <[email protected]>--- tests/functional/qemu_test/testcase.py | 27 +++++++++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-)tests/functional/qemu_test/testcase.py +2
-1@@ -31,7 +31,8 @@ class QemuBaseTest(unittest.TestCase): arch = None workdir = None - log = logging.getLogger('qemu-test') + log = None + logdir = None def setUp(self, bin_prefix): self.assertIsNotNone(self.qemu_bin, 'QEMU_TEST_QEMU_BINARY must be set')@@ -42,6 +43,20 @@ def setUp(self, bin_prefix): if not os.path.exists(self.workdir): os.makedirs(self.workdir)
tests/functional/qemu_test/testcase.py def setUp(self, bin_prefix): · line 55
Quoted 10 lines
tests/functional/qemu_test/testcase.py +10
-0+ self.logdir = self.workdir + self.log = logging.getLogger('qemu-test') + self.log.setLevel(logging.DEBUG) + self._log_fh = logging.FileHandler(os.path.join(self.logdir, + 'base.log'), mode='w') + self._log_fh.setLevel(logging.DEBUG) + fileFormatter = logging.Formatter( + '%(asctime)s - %(levelname)s: %(message)s') + self._log_fh.setFormatter(fileFormatter) + self.log.addHandler(self._log_fh)
This is in the wrong place, we have a unique machine directory for each run, the console.log should go in there.
Signature · 2 lines
Alex Bennée Virtualisation Tech Lead @ Linaro
Re: [PATCH v3 03/24] tests/functional: Set up logging
Quoted 31 lines · Alex Bennée
Quoted 29 lines · Daniel P. Berrangé
From: Thomas Huth <[email protected]>
Create log files for each test separately, one file that contains the basic logging and one that contains the console output.
Reviewed-by Daniel P. Berrangé <[email protected]>Signed-off-by Thomas Huth <[email protected]>--- tests/functional/qemu_test/testcase.py | 27 +++++++++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-)tests/functional/qemu_test/testcase.py +2
-1@@ -31,7 +31,8 @@ class QemuBaseTest(unittest.TestCase): arch = None workdir = None - log = logging.getLogger('qemu-test') + log = None + logdir = None def setUp(self, bin_prefix): self.assertIsNotNone(self.qemu_bin, 'QEMU_TEST_QEMU_BINARY must be set')@@ -42,6 +43,20 @@ def setUp(self, bin_prefix): if not os.path.exists(self.workdir): os.makedirs(self.workdir)
Quoted 13 lines
tests/functional/qemu_test/testcase.py def setUp(self, bin_prefix): · line 55
Quoted 10 lines
tests/functional/qemu_test/testcase.py +10
-0+ self.logdir = self.workdir + self.log = logging.getLogger('qemu-test') + self.log.setLevel(logging.DEBUG) + self._log_fh = logging.FileHandler(os.path.join(self.logdir, + 'base.log'), mode='w') + self._log_fh.setLevel(logging.DEBUG) + fileFormatter = logging.Formatter( + '%(asctime)s - %(levelname)s: %(message)s') + self._log_fh.setFormatter(fileFormatter) + self.log.addHandler(self._log_fh)This is in the wrong place, we have a unique machine directory for each run, the console.log should go in there.
self.workdir already points to the unique directory of the test (see second patch), so setting logdir = workdir should be fine, I think?
Thomas
PATCH v3 4/24 tests/Makefile.include: Increase the level of indentation in the help text
In reply to Daniel P. BerrangéFrom: Thomas Huth <[email protected]>
The next patch is going to add some entries that need more space between the command and the help text, so let's increase the indentation here first.
--- tests/Makefile.include | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-)
tests/Makefile.include +15 -15
@@ -3,28 +3,28 @@ .PHONY: check-help check-help: @echo "Regression testing targets:" - @echo " $(MAKE) check Run block, qapi-schema, unit, softfloat, qtest and decodetree tests" - @echo " $(MAKE) bench Run speed tests" + @echo " $(MAKE) check Run block, qapi-schema, unit, softfloat, qtest and decodetree tests" + @echo " $(MAKE) bench Run speed tests" @echo @echo "Individual test suites:" - @echo " $(MAKE) check-qtest-TARGET Run qtest tests for given target" - @echo " $(MAKE) check-qtest Run qtest tests" - @echo " $(MAKE) check-unit Run qobject tests" - @echo " $(MAKE) check-qapi-schema Run QAPI schema tests" - @echo " $(MAKE) check-block Run block tests" + @echo " $(MAKE) check-qtest-TARGET Run qtest tests for given target" + @echo " $(MAKE) check-qtest Run qtest tests" + @echo " $(MAKE) check-unit Run qobject tests" + @echo " $(MAKE) check-qapi-schema Run QAPI schema tests" + @echo " $(MAKE) check-block Run block tests" ifneq ($(filter $(all-check-targets), check-softfloat),) - @echo " $(MAKE) check-tcg Run TCG tests" - @echo " $(MAKE) check-softfloat Run FPU emulation tests" + @echo " $(MAKE) check-tcg Run TCG tests" + @echo " $(MAKE) check-softfloat Run FPU emulation tests" endif - @echo " $(MAKE) check-avocado Run avocado (integration) tests for currently configured targets" + @echo " $(MAKE) check-avocado Run avocado (integration) tests for currently configured targets" @echo - @echo " $(MAKE) check-report.junit.xml Generates an aggregated XML test report" - @echo " $(MAKE) check-venv Creates a Python venv for tests" - @echo " $(MAKE) check-clean Clean the tests and related data" + @echo " $(MAKE) check-report.junit.xml Generates an aggregated XML test report" + @echo " $(MAKE) check-venv Creates a Python venv for tests" + @echo " $(MAKE) check-clean Clean the tests and related data" @echo @echo "The following are useful for CI builds" - @echo " $(MAKE) check-build Build most test binaries" - @echo " $(MAKE) get-vm-images Downloads all images used by avocado tests, according to configured targets (~350 MB each, 1.5 GB max)" + @echo " $(MAKE) check-build Build most test binaries" + @echo " $(MAKE) get-vm-images Downloads all images used by avocado tests, according to configured targets (~350 MB each, 1.5 GB max)" @echo @echo @echo "The variable SPEED can be set to control the gtester speed setting."
Signature · 1 line
2.45.2
Re: [PATCH v3 04/24] tests/Makefile.include: Increase the level of indentation in the help text
Quoted 11 lines · Daniel P. Berrangé
From: Thomas Huth <[email protected]>
The next patch is going to add some entries that need more space between the command and the help text, so let's increase the indentation here first.
Reviewed-by Philippe Mathieu-Daudé <[email protected]>Signed-off-by Thomas Huth <[email protected]>--- tests/Makefile.include | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-)
Re: [PATCH v3 04/24] tests/Makefile.include: Increase the level of indentation in the help text
In reply to Daniel P. BerrangéQuoted 8 lines · Daniel P. Berrangé
From: Thomas Huth <[email protected]>
The next patch is going to add some entries that need more space between the command and the help text, so let's increase the indentation here first.
Reviewed-by Philippe Mathieu-Daudé <[email protected]>Signed-off-by Thomas Huth <[email protected]>
Signature · 2 lines
Alex Bennée Virtualisation Tech Lead @ Linaro
PATCH v3 5/24 tests/functional: Prepare the meson build system for the functional tests
In reply to Daniel P. BerrangéFrom: Thomas Huth <[email protected]>
Provide a meson.build file for the upcoming python-based functional tests, and add some wrapper glue targets to the tests/Makefile.include file. We are going to use two "speed" modes for the functional tests:
The "quick" tests can be run at any time (i.e. also during "make check"), while the "thorough" tests should only be run when running a "make check-functional" test run (since these tests might download additional assets from the internet).
The changes to the meson.build files are partly based on an earlier patch by Ani Sinha.
--- tests/Makefile.include | 11 ++++++ tests/functional/meson.build | 66 ++++++++++++++++++++++++++++++++++++ tests/meson.build | 1 + 3 files changed, 78 insertions(+) create mode 100644 tests/functional/meson.build
tests/Makefile.include +11 -0
@@ -9,6 +9,8 @@ check-help: @echo "Individual test suites:" @echo " $(MAKE) check-qtest-TARGET Run qtest tests for given target" @echo " $(MAKE) check-qtest Run qtest tests" + @echo " $(MAKE) check-functional Run python-based functional tests" + @echo " $(MAKE) check-functional-TARGET Run functional tests for a given target" @echo " $(MAKE) check-unit Run qobject tests" @echo " $(MAKE) check-qapi-schema Run QAPI schema tests" @echo " $(MAKE) check-block Run block tests"
@@ -152,6 +154,15 @@ check-acceptance-deprecated-warning: check-acceptance: check-acceptance-deprecated-warning | check-avocado +FUNCTIONAL_TARGETS=$(patsubst %-softmmu,check-functional-%, $(filter %-softmmu,$(TARGETS))) +.PHONY: $(FUNCTIONAL_TARGETS) +$(FUNCTIONAL_TARGETS): + @$(MAKE) SPEED=thorough $(subst -functional,-func,$@) + +.PHONY: check-functional +check-functional: + @$(MAKE) SPEED=thorough check-func check-func-quick + # Consolidated targets .PHONY: check check-clean get-vm-images
tests/functional/meson.build new file +66 -0
66 lines not on this page · raw message
tests/meson.build +1 -0
@@ -85,3 +85,4 @@ subdir('unit') subdir('qapi-schema') subdir('qtest') subdir('migration') +subdir('functional')
Signature · 1 line
2.45.2
Re: [PATCH v3 05/24] tests/functional: Prepare the meson build system for the functional tests
Hi,
Quoted 54 lines · Daniel P. Berrangé
From: Thomas Huth <[email protected]>
Provide a meson.build file for the upcoming python-based functional tests, and add some wrapper glue targets to the tests/Makefile.include file. We are going to use two "speed" modes for the functional tests:
The "quick" tests can be run at any time (i.e. also during "make check"), while the "thorough" tests should only be run when running a "make check-functional" test run (since these tests might download additional assets from the internet).The changes to the meson.build files are partly based on an earlier patch by Ani Sinha.
Signed-off-by Thomas Huth <[email protected]>--- tests/Makefile.include | 11 ++++++ tests/functional/meson.build | 66 ++++++++++++++++++++++++++++++++++++ tests/meson.build | 1 + 3 files changed, 78 insertions(+)create mode 100644 tests/functional/meson.build
tests/functional/meson.build new file +32
-0+foreach speed : ['quick', 'thorough'] + foreach dir : target_dirs + if not dir.endswith('-softmmu') + continue + endif + + target_base = dir.split('-')[0] + test_emulator = emulators['qemu-system-' + target_base] + + if speed == 'quick' + suites = ['func-quick', 'func-' + target_base] + target_tests = get_variable('tests_' + target_base + '_quick', []) + tests_generic + else + suites = ['func-' + speed, 'func-' + target_base + '-' + speed, speed] + target_tests = get_variable('tests_' + target_base + '_' + speed, []) + endif + + test_deps = roms + test_env = environment() + if have_tools + test_env.set('QEMU_TEST_QEMU_IMG', meson.global_build_root() / 'qemu-img') + test_deps += [qemu_img] + endif + test_env.set('QEMU_TEST_QEMU_BINARY', + meson.global_build_root() / 'qemu-system-' + target_base) + test_env.set('QEMU_BUILD_ROOT', meson.project_build_root()) + test_env.set('PYTHONPATH', meson.project_source_root() / 'python:' + + meson.current_source_dir()) + + foreach test : target_tests + test('func-@0@/@1@'.format(target_base, test), + python,
tests/functional/meson.build line 66
Quoted 10 lines
tests/functional/meson.build new file +10
-0+ depends: [test_deps, test_emulator, emulator_modules], + env: test_env, + args: [meson.current_source_dir() / 'test_' + test + '.py'], + protocol: 'tap', + timeout: test_timeouts.get(test, 60), + priority: test_timeouts.get(test, 60), + suite: suites) + endforeach + endforeach +endforeach
Can we have meson report why a test is skipped? All Python unittest @skip* decorators provide a 'reason' argument, so we already have that information.
Not much help on meson doc:
https://mesonbuild.com/Unit-tests.html#skipped-tests-and-hard-errors
If I run a test manually I see the reason in the tap output:
$ FOO=bar tests/functional/test_mips64el_loongson3v.py TAP version 13 ok 1 test_mips64el_loongson3v.MipsLoongson3v.test_pmon_serial_console # SKIP untrusted code 1..1 $
Re: [PATCH v3 05/24] tests/functional: Prepare the meson build system for the functional tests
Quoted 65 lines · Philippe Mathieu-Daudé
Hi,
Quoted 62 lines · Daniel P. Berrangé
From: Thomas Huth <[email protected]>
Provide a meson.build file for the upcoming python-based functional tests, and add some wrapper glue targets to the tests/Makefile.include file. We are going to use two "speed" modes for the functional tests:
The "quick" tests can be run at any time (i.e. also during "make check"), while the "thorough" tests should only be run when running a "make check-functional" test run (since these tests might download additional assets from the internet).The changes to the meson.build files are partly based on an earlier patch by Ani Sinha.
Signed-off-by Thomas Huth <[email protected]>---
tests/Makefile.include | 11 ++++++ tests/functional/meson.build | 66 ++++++++++++++++++++++++++++++++++++ tests/meson.build | 1 +3 files changed, 78 insertions(+)create mode 100644 tests/functional/meson.build
tests/functional/meson.build new file +22
-0+foreach speed : ['quick', 'thorough'] + foreach dir : target_dirs + if not dir.endswith('-softmmu') + continue + endif + + target_base = dir.split('-')[0] + test_emulator = emulators['qemu-system-' + target_base] + + if speed == 'quick' + suites = ['func-quick', 'func-' + target_base] + target_tests = get_variable('tests_' + target_base + '_quick', []) + tests_generic + else + suites = ['func-' + speed, 'func-' + target_base + '-' + speed, speed] + target_tests = get_variable('tests_' + target_base + '_' + speed, []) + endif + + test_deps = roms + test_env = environment() + if have_tools + test_env.set('QEMU_TEST_QEMU_IMG', meson.global_build_root() /'qemu-img')
+test_deps+= [qemu_img]
+ endif
+ test_env.set('QEMU_TEST_QEMU_BINARY',
+meson.global_build_root()/ 'qemu-system-' +target_base) +test_env.set('QEMU_BUILD_ROOT', meson.project_build_root())+ test_env.set('PYTHONPATH',meson.project_source_root()/ 'python:' + +meson.current_source_dir())
+
+ foreach test :target_tests
+ test('func-@0@/@1@'.format(target_base, test),
+ python,
+ depends: [test_deps,test_emulator,emulator_modules], + env:test_env,
+ args: [meson.current_source_dir()/ 'test_' + test + '.py'], + protocol: 'tap',
+ timeout:test_timeouts.get(test, 60),
+ priority:test_timeouts.get(test, 60),
Quoted 8 lines
Quoted 4 lines
+ suite: suites)
+ endforeach
+ endforeach
+endforeachCan we have meson report why a test is skipped? All Python unittest @skip* decorators provide a 'reason' argument, so we already have that information.
I guess you'd need to extend the meson test runner for this...
Quoted 9 lines
Not much help on meson doc:
https://mesonbuild.com/Unit-tests.html#skipped-tests-and-hard-errorsIf I run a test manually I see the reason in the tap output:
$ FOO=bar tests/functional/test_mips64el_loongson3v.py TAP version 13 ok 1 test_mips64el_loongson3v.MipsLoongson3v.test_pmon_serial_console # SKIP untrusted code 1..1
You can also see this information in the txt file in the meson-logs folder, so it is certainly not completely lost.
Thomas
PATCH v3 6/24 tests/functional: Convert simple avocado tests into standalone python tests
In reply to Daniel P. BerrangéFrom: Thomas Huth <[email protected]>
These test are rather simple and don't need any modifications apart from adjusting the "from avocado_qemu" line. To ease debugging, make the files executable and add a shebang line and Python '__main__' handling, too, so that these tests can now be run by executing them directly.
---
tests/functional/meson.build | 5 ++
.../test_cpu_queries.py} | 7 ++-
.../test_empty_cpu_model.py} | 7 ++-
.../test_mem_addr_space.py} | 52 +++----------------
.../test_pc_cpu_hotplug_props.py} | 11 ++--
.../test_virtio_version.py} | 8 +--
6 files changed, 34 insertions(+), 56 deletions(-)
rename tests/{avocado/cpu_queries.py => functional/test_cpu_queries.py} (89%)
mode change 100644 => 100755
rename tests/{avocado/empty_cpu_model.py => functional/test_empty_cpu_model.py} (84%)
mode change 100644 => 100755
rename tests/{avocado/mem-addr-space-check.py => functional/test_mem_addr_space.py} (93%)
mode change 100644 => 100755
rename tests/{avocado/pc_cpu_hotplug_props.py => functional/test_pc_cpu_hotplug_props.py} (90%)
mode change 100644 => 100755
rename tests/{avocado/virtio_version.py => functional/test_virtio_version.py} (98%)
mode change 100644 => 100755tests/functional/meson.build +5 -0
@@ -14,9 +14,14 @@ test_timeouts = { } tests_generic = [ + 'empty_cpu_model', ] tests_x86_64_quick = [ + 'cpu_queries', + 'mem_addr_space', + 'pc_cpu_hotplug_props', + 'virtio_version', ] tests_x86_64_thorough = [
tests/functional/test_cpu_queries.py renamed from tests/avocado/cpu_queries.py +6 -1
@@ -1,3 +1,5 @@ +#!/usr/bin/env python3 +# # Sanity check of query-cpu-* results # # Copyright (c) 2019 Red Hat, Inc.
@@ -8,7 +10,7 @@ # This work is licensed under the terms of the GNU GPL, version 2 or # later. See the COPYING file in the top-level directory. -from avocado_qemu import QemuSystemTest +from qemu_test import QemuSystemTest class QueryCPUModelExpansion(QemuSystemTest): """
@@ -33,3 +35,6 @@ def test(self): e = self.vm.cmd('query-cpu-model-expansion', model=model, type='full') self.assertEqual(e['model']['name'], c['name']) + +if __name__ == '__main__': + QemuSystemTest.main()
tests/functional/test_empty_cpu_model.py renamed from tests/avocado/empty_cpu_model.py +6 -1
@@ -1,3 +1,5 @@ +#!/usr/bin/env python3 +# # Check for crash when using empty -cpu option # # Copyright (c) 2019 Red Hat, Inc.
@@ -7,7 +9,7 @@ # # This work is licensed under the terms of the GNU GPL, version 2 or # later. See the COPYING file in the top-level directory. -from avocado_qemu import QemuSystemTest +from qemu_test import QemuSystemTest class EmptyCPUModel(QemuSystemTest): def test(self):
@@ -17,3 +19,6 @@ def test(self): self.vm.wait() self.assertEqual(self.vm.exitcode(), 1, "QEMU exit code should be 1") self.assertRegex(self.vm.get_log(), r'-cpu option cannot be empty') + +if __name__ == '__main__': + QemuSystemTest.main()
tests/functional/test_mem_addr_space.py renamed from tests/avocado/mem-addr-space-check.py +6 -46
154 lines not on this page · raw message
tests/functional/test_pc_cpu_hotplug_props.py renamed from tests/avocado/pc_cpu_hotplug_props.py +6 -5
@@ -1,3 +1,4 @@ +#!/usr/bin/env python3 # # Ensure CPU die-id can be omitted on -device #
@@ -20,16 +21,16 @@ # License along with this library; if not, see <http://www.gnu.org/licenses/>. # -from avocado_qemu import QemuSystemTest +from qemu_test import QemuSystemTest class OmittedCPUProps(QemuSystemTest): - """ - :avocado: tags=arch:x86_64 - :avocado: tags=cpu:qemu64 - """ + def test_no_die_id(self): self.vm.add_args('-nodefaults', '-S') self.vm.add_args('-smp', '1,sockets=2,cores=2,threads=2,maxcpus=8') self.vm.add_args('-device', 'qemu64-x86_64-cpu,socket-id=1,core-id=0,thread-id=0') self.vm.launch() self.assertEqual(len(self.vm.cmd('query-cpus-fast')), 2) + +if __name__ == '__main__': + QemuSystemTest.main()
tests/functional/test_virtio_version.py renamed from tests/avocado/virtio_version.py +5 -3
@@ -1,3 +1,4 @@ +#!/usr/bin/env python3 """ Check compatibility of virtio device types """
@@ -12,7 +13,7 @@ import os from qemu.machine import QEMUMachine -from avocado_qemu import QemuSystemTest +from qemu_test import QemuSystemTest # Virtio Device IDs: VIRTIO_NET = 1
@@ -60,8 +61,6 @@ class VirtioVersionCheck(QemuSystemTest): Check if virtio-version-specific device types result in the same device tree created by `disable-modern` and `disable-legacy`. - - :avocado: tags=arch:x86_64 """ # just in case there are failures, show larger diff:
@@ -173,3 +172,6 @@ def test_modern_only_devs(self): self.check_modern_only('virtio-mouse-pci', VIRTIO_INPUT) self.check_modern_only('virtio-tablet-pci', VIRTIO_INPUT) self.check_modern_only('virtio-keyboard-pci', VIRTIO_INPUT) + +if __name__ == '__main__': + QemuSystemTest.main()
Signature · 1 line
2.45.2
Re: [PATCH v3 06/24] tests/functional: Convert simple avocado tests into standalone python tests
Quoted 19 lines · Daniel P. Berrangé
From: Thomas Huth <[email protected]>
These test are rather simple and don't need any modifications apart from adjusting the "from
avocado_qemu" line. To ease debugging, make the files executable and add a shebang line and Python '__main__' handling, too, so that these tests can now be run by executing them directly.Reviewed-by Daniel P. Berrangé <[email protected]>Reviewed-by Philippe Mathieu-Daudé <[email protected]>Signed-off-by Thomas Huth <[email protected]>--- tests/functional/meson.build | 5 ++ .../test_cpu_queries.py} | 7 ++- .../test_empty_cpu_model.py} | 7 ++- .../test_mem_addr_space.py} | 52 +++---------------- .../test_pc_cpu_hotplug_props.py} | 11 ++-- .../test_virtio_version.py} | 8 +-- 6 files changed, 34 insertions(+), 56 deletions(-)
Quoted 10 lines
rename
tests/{avocado/cpu_queries.py=>functional/test_cpu_queries.py} (89%) mode change 100644 => 100755
renametests/{avocado/empty_cpu_model.py=>functional/test_empty_cpu_model.py} (84%) mode change 100644 => 100755
renametests/{avocado/mem-addr-space-check.py=>functional/test_mem_addr_space.py} (93%) mode change 100644 => 100755
renametests/{avocado/pc_cpu_hotplug_props.py=>functional/test_pc_cpu_hotplug_props.py} (90%) mode change 100644 => 100755
renametests/{avocado/virtio_version.py=>functional/test_virtio_version.py} (98%) mode change 100644 => 100755
Please squash:
-- >8 --
MAINTAINERS +6 -0
@@ -1830,6 +1832,9 @@ F: hw/isa/apm.c F: include/hw/isa/apm.h F: tests/unit/test-x86-topo.c F: tests/qtest/test-x86-cpuid-compat.c +F: tests/functional/test_mem_addr_space.py +F: tests/functional/test_pc_cpu_hotplug_props.py +F: tests/functional/test_x86_cpu_model_versions.py PC Chipset M: Michael S. Tsirkin <[email protected]>
@@ -1896,6 +1901,8 @@ F: include/hw/boards.h F: include/hw/core/cpu.h F: include/hw/cpu/cluster.h F: include/sysemu/numa.h +F: tests/functional/test_cpu_queries.py +F: tests/functional/test_empty_cpu_model.py F: tests/unit/test-smp-parse.c T: git https://gitlab.com/ehabkost/qemu.git machine-next
@@ -2236,6 +2244,7 @@ F: net/vhost-user.c F: include/hw/virtio/ F: docs/devel/virtio* F: docs/devel/migration/virtio.rst +F: tests/functional/test_virtio_version.py virtio-balloon M: Michael S. Tsirkin <[email protected]>
tests/functional/test_x86_cpu_model_versions.py +0 -0
@@ -217,7 +217,7 @@ def test_none_alias(self): """ Check if unversioned CPU model is an alias pointing to some
version
"""
- self.machine = 'none'
+ self.set_machine('none')
self.vm.add_args('-S')
self.vm.launch()---
PATCH v3 7/24 tests/functional: Convert avocado tests that just need a small adjustment
In reply to Daniel P. BerrangéFrom: Thomas Huth <[email protected]>
These simple tests can be converted to stand-alone tests quite easily, e.g. by just setting the machine to 'none' now manually or by adding "-cpu" command line parameters, since we don't support the corresponding avocado tags in the new python test framework.
---
tests/functional/meson.build | 6 ++
.../test_info_usernet.py} | 11 ++-
.../test_ppc_74xx.py} | 74 ++++++++-----------
.../version.py => functional/test_version.py} | 13 ++--
4 files changed, 53 insertions(+), 51 deletions(-)
rename tests/{avocado/info_usernet.py => functional/test_info_usernet.py} (87%)
mode change 100644 => 100755
rename tests/{avocado/ppc_74xx.py => functional/test_ppc_74xx.py} (74%)
mode change 100644 => 100755
rename tests/{avocado/version.py => functional/test_version.py} (78%)
mode change 100644 => 100755tests/functional/meson.build +6 -0
@@ -15,6 +15,12 @@ test_timeouts = { tests_generic = [ 'empty_cpu_model', + 'info_usernet', + 'version', +] + +tests_ppc_quick = [ + 'ppc_74xx', ] tests_x86_64_quick = [
tests/functional/test_info_usernet.py renamed from tests/avocado/info_usernet.py +7 -4
@@ -1,3 +1,5 @@ +#!/usr/bin/env python3 +# # Test for the hmp command "info usernet" # # Copyright (c) 2021 Red Hat, Inc.
@@ -8,18 +10,16 @@ # This work is licensed under the terms of the GNU GPL, version 2 or # later. See the COPYING file in the top-level directory. -from avocado_qemu import QemuSystemTest +from qemu_test import QemuSystemTest from qemu.utils import get_info_usernet_hostfwd_port class InfoUsernet(QemuSystemTest): - """ - :avocado: tags=machine:none - """ def test_hostfwd(self): self.require_netdev('user') + self.machine = 'none' self.vm.add_args('-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0-:22') self.vm.launch() res = self.vm.cmd('human-monitor-command',
@@ -31,3 +31,6 @@ def test_hostfwd(self): self.assertGreater(port, 0, ('Found a redirected port that is not greater than' ' zero')) + +if __name__ == '__main__': + QemuSystemTest.main()
tests/functional/test_ppc_74xx.py renamed from tests/avocado/ppc_74xx.py +32 -42
167 lines not on this page · raw message
tests/functional/test_version.py renamed from tests/avocado/version.py +8 -5
@@ -1,3 +1,5 @@ +#!/usr/bin/env python3 +# # Version check example test # # Copyright (c) 2018 Red Hat, Inc.
@@ -9,17 +11,18 @@ # later. See the COPYING file in the top-level directory. -from avocado_qemu import QemuSystemTest +from qemu_test import QemuSystemTest class Version(QemuSystemTest): - """ - :avocado: tags=quick - :avocado: tags=machine:none - """ + def test_qmp_human_info_version(self): + self.machine = 'none' self.vm.add_args('-nodefaults') self.vm.launch() res = self.vm.cmd('human-monitor-command', command_line='info version') self.assertRegex(res, r'^(\d+\.\d+\.\d)') + +if __name__ == '__main__': + QemuSystemTest.main()
Signature · 1 line
2.45.2
Re: [PATCH v3 07/24] tests/functional: Convert avocado tests that just need a small adjustment
Quoted 15 lines · Daniel P. Berrangé
From: Thomas Huth <[email protected]>
These simple tests can be converted to stand-alone tests quite easily, e.g. by just setting the machine to 'none' now manually or by adding "-cpu" command line parameters, since we don't support the corresponding avocado tags in the new python test framework.
Reviewed-by Daniel P. Berrangé <[email protected]>Signed-off-by Thomas Huth <[email protected]>--- tests/functional/meson.build | 6 ++ .../test_info_usernet.py} | 11 ++- .../test_ppc_74xx.py} | 74 ++++++++----------- .../version.py => functional/test_version.py} | 13 ++-- 4 files changed, 53 insertions(+), 51 deletions(-)
Quoted 6 lines
rename
tests/{avocado/info_usernet.py=>functional/test_info_usernet.py} (87%) mode change 100644 => 100755
renametests/{avocado/ppc_74xx.py=>functional/test_ppc_74xx.py} (74%) mode change 100644 => 100755
renametests/{avocado/version.py=>functional/test_version.py} (78%) mode change 100644 => 100755
Please squash:
-- >8 --
MAINTAINERS +3 -0
@@ -317,6 +317,7 @@ F: configs/devices/ppc* F: docs/system/ppc/embedded.rst F: docs/system/target-ppc.rst F: tests/tcg/ppc*/* +F: tests/functional/test_ppc_74xx.py RISC-V TCG CPUs M: Palmer Dabbelt <[email protected]>
@@ -2101,6 +2108,7 @@ S: Odd Fixes F: hw/net/ F: include/hw/net/ F: tests/qtest/virtio-net-test.c +F: tests/avocado/info_usernet.py F: docs/virtio-net-failover.rst T: git https://github.com/jasowang/qemu.git net
@@ -2969,6 +2978,7 @@ S: Supported F: include/qemu/option.h F: tests/unit/test-keyval.c F: tests/unit/test-qemu-opts.c +F: tests/functional/test_version.py F: util/keyval.c F: util/qemu-option.c
tests/functional/test_info_usernet.py +1 -1
@@ -19,7 +19,7 @@ class InfoUsernet(QemuSystemTest): def test_hostfwd(self): self.require_netdev('user') - self.machine = 'none' + self.set_machine('none') self.vm.add_args('-netdev',
'user,id=vnet,hostfwd=:127.0.0.1:0-:22')
self.vm.launch()
res = self.vm.cmd('human-monitor-command',
tests/functional/test_version.py +1 -1
@@ -17,7 +17,7 @@ class Version(QemuSystemTest): def test_qmp_human_info_version(self): - self.machine = 'none' + self.set_machine('none') self.vm.add_args('-nodefaults') self.vm.launch() res = self.vm.cmd('human-monitor-command',
---
Philippe Mathieu-Daudé
Reviewed-byRe: [PATCH v3 07/24] tests/functional: Convert avocado tests that just need a small adjustment
Quoted 23 lines · Philippe Mathieu-Daudé
Quoted 14 lines · Daniel P. Berrangé
From: Thomas Huth <[email protected]>
These simple tests can be converted to stand-alone tests quite easily, e.g. by just setting the machine to 'none' now manually or by adding "-cpu" command line parameters, since we don't support the corresponding avocado tags in the new python test framework.
Reviewed-by Daniel P. Berrangé <[email protected]>Signed-off-by Thomas Huth <[email protected]>---
tests/functional/meson.build | 6 ++ .../test_info_usernet.py} | 11 ++- .../test_ppc_74xx.py} | 74 ++++++++-----------.../version.py=>functional/test_version.py} | 13 ++--Quoted 7 lines
4 files changed, 53 insertions(+), 51 deletions(-)rename
tests/{avocado/info_usernet.py=>functional/test_info_usernet.py} (87%) mode change 100644 => 100755
renametests/{avocado/ppc_74xx.py=>functional/test_ppc_74xx.py} (74%) mode change 100644 => 100755
renametests/{avocado/version.py=>functional/test_version.py} (78%) mode change 100644 => 100755
Quoted 2 lines
PATCH v3 8/24 tests/functional: add a module for handling asset download & caching
In reply to Daniel P. BerrangéThe 'Asset' class is a simple module that declares a downloadable asset that can be cached locally. Downloads are stored in the user's home dir at ~/.cache/qemu/download, using a sha256 sum of the URL.
--- tests/functional/qemu_test/__init__.py | 1 + tests/functional/qemu_test/asset.py | 96 ++++++++++++++++++++++++++ 2 files changed, 97 insertions(+) create mode 100644 tests/functional/qemu_test/asset.py
tests/functional/qemu_test/__init__.py +1 -0
@@ -6,6 +6,7 @@ # later. See the COPYING file in the top-level directory. +from .asset import Asset from .config import BUILD_DIR from .cmd import has_cmd, has_cmds, run_cmd, is_readable_executable_file, \ interrupt_interactive_console_until_pattern, wait_for_console_pattern, \
tests/functional/qemu_test/asset.py new file +96 -0
96 lines not on this page · raw message
Signature · 1 line
2.45.2
Re: [PATCH v3 08/24] tests/functional: add a module for handling asset download & caching
Quoted 16 lines · Daniel P. Berrangé
The 'Asset' class is a simple module that declares a downloadable asset that can be cached locally. Downloads are stored in the user's home dir at ~/.cache/qemu/download, using a sha256 sum of the URL.
Signed-off-by Daniel P. Berrangé <[email protected]>--- tests/functional/qemu_test/__init__.py | 1 + tests/functional/qemu_test/asset.py | 96 ++++++++++++++++++++++++++ 2 files changed, 97 insertions(+)create mode 100644
tests/functional/qemu_test/asset.pytests/functional/qemu_test/__init__.py +0
-0
tests/functional/qemu_test/__init__.py line 9
Quoted 4 lines
tests/functional/qemu_test/__init__.py +1
-0@@ -6,6 +6,7 @@ # later. See the COPYING file in the top-level directory. +from .asset import Asset
For next patch?
Quoted 41 lines
tests/functional/qemu_test/__init__.py +0
-0from .config import BUILD_DIR from .cmd import has_cmd, has_cmds, run_cmd, is_readable_executable_file, \ interrupt_interactive_console_until_pattern, wait_for_console_pattern, \tests/functional/qemu_test/asset.py new file +32
-0@@ -0,0 +1,96 @@ +# Test utilities for fetching & caching assets +# +# Copyright 2024 Red Hat, Inc. +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + +import hashlib +import logging +import os +import subprocess +import urllib.request +from pathlib import Path +from shutil import copyfileobj + + +# Instances of this class must be declared as class level variables +# starting with a name "ASSET_". This enables the pre-caching logic +# to easily find all referenced assets and download them prior to +# execution of the tests. +class Asset: + + def __init__(self, url, hash): + self.url = url + self.hash = hash + self.cache_dir = Path(Path("~").expanduser(), + ".cache", "qemu", "download") + self.cache_file = Path(self.cache_dir, + hashlib.sha256(url.encode("utf-8")).hexdigest()) + self.log = logging.getLogger('qemu-test') + + def __repr__(self):
tests/functional/qemu_test/asset.py line 42
Quoted 10 lines
tests/functional/qemu_test/asset.py new file +10
-0+ return "Asset: url=%s hash=%s cache=%s" % ( + self.url, self.hash, self.cache_file) + + def _check(self, cache_file): + if self.hash is None: + return True + if len(self.hash) == 40: + sum_prog = 'sha1sum' + elif len(self.hash) == 64: + sum_prog = 'sha256sum'
Do we want to support these? Should we declare them deprecated and emit a warning?
tests/functional/qemu_test/asset.py line 46
Quoted 4 lines
tests/functional/qemu_test/asset.py new file +4
-0+ elif len(self.hash) == 128: + sum_prog = 'sha512sum' + else: + raise Exception("unknown hash type")
Very nice :)
Re: [PATCH v3 08/24] tests/functional: add a module for handling asset download & caching
Quoted 68 lines · Philippe Mathieu-Daudé
Quoted 16 lines · Daniel P. Berrangé
The 'Asset' class is a simple module that declares a downloadable asset that can be cached locally. Downloads are stored in the user's home dir at ~/.cache/qemu/download, using a sha256 sum of the URL.
Signed-off-by Daniel P. Berrangé <[email protected]>--- tests/functional/qemu_test/__init__.py | 1 + tests/functional/qemu_test/asset.py | 96 ++++++++++++++++++++++++++ 2 files changed, 97 insertions(+)create mode 100644
tests/functional/qemu_test/asset.pytests/functional/qemu_test/__init__.py +0
-0tests/functional/qemu_test/__init__.py line 7
Quoted 2 lines
tests/functional/qemu_test/__init__.py +1
-0@@ -6,6 +6,7 @@ # later. See the COPYING file in the top-level directory. +from .asset import AssetFor next patch?
Quoted 46 lines
tests/functional/qemu_test/__init__.py +0
-0from .config import BUILD_DIR from .cmd import has_cmd, has_cmds, run_cmd, is_readable_executable_file, \ interrupt_interactive_console_until_pattern, wait_for_console_pattern, \tests/functional/qemu_test/asset.py new file +37
-0@@ -0,0 +1,96 @@ +# Test utilities for fetching & caching assets +# +# Copyright 2024 Red Hat, Inc. +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + +import hashlib +import logging +import os +import subprocess +import urllib.request +from pathlib import Path +from shutil import copyfileobj + + +# Instances of this class must be declared as class level variables +# starting with a name "ASSET_". This enables the pre-caching logic +# to easily find all referenced assets and download them prior to +# execution of the tests. +class Asset: + + def __init__(self, url, hash): + self.url = url + self.hash = hash + self.cache_dir = Path(Path("~").expanduser(), + ".cache", "qemu", "download") + self.cache_file = Path(self.cache_dir, + hashlib.sha256(url.encode("utf-8")).hexdigest()) + self.log = logging.getLogger('qemu-test') + + def __repr__(self): + return "Asset: url=%s hash=%s cache=%s" % ( + self.url, self.hash, self.cache_file) + + def _check(self, cache_file): + if self.hash is None:
Quoted 8 lines
tests/functional/qemu_test/asset.py line 42
Quoted 5 lines
tests/functional/qemu_test/asset.py new file +5
-0+ return True + if len(self.hash) == 40: + sum_prog = 'sha1sum' + elif len(self.hash) == 64: + sum_prog = 'sha256sum'Do we want to support these? Should we declare them deprecated and emit a warning?
Oh thanks for the reminder. I wanted to standardize on sha256 ,since sha1 is broken and sha512 is overkill.
Since I've run the tests once I've got everything download and can now trivially generate the sha256 for everything we need.
Quoted 11 lines
tests/functional/qemu_test/asset.py line 46
Quoted 5 lines
tests/functional/qemu_test/asset.py new file +4
-0+ elif len(self.hash) == 128: + sum_prog = 'sha512sum' + else: + raise Exception("unknown hash type")Reviewed-by Philippe Mathieu-Daudé <[email protected]>Very nice :)
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [PATCH v3 08/24] tests/functional: add a module for handling asset download & caching
In reply to Philippe Mathieu-DaudéQuoted 12 lines · Philippe Mathieu-Daudé
Quoted 11 lines · Daniel P. Berrangé
The 'Asset' class is a simple module that declares a downloadable asset that can be cached locally. Downloads are stored in the user's home dir at ~/.cache/qemu/download, using a sha256 sum of the URL.
Signed-off-by Daniel P. Berrangé <[email protected]>--- tests/functional/qemu_test/__init__.py | 1 + tests/functional/qemu_test/asset.py | 96 ++++++++++++++++++++++++++ 2 files changed, 97 insertions(+)create mode 100644
tests/functional/qemu_test/asset.py
Quoted 9 lines
tests/functional/qemu_test/__init__.py line 7
Quoted 7 lines
tests/functional/qemu_test/__init__.py +1
-0@@ -6,6 +6,7 @@ # later. See the COPYING file in the top-level directory. +from .asset import AssetFor next patch?
No, the idea is that a test class can just do 'from qemu_test import Asset', so they don't need to be aware of the fact that we've split up the code into separate files beneath the qemu_test/ directory.
Quoted 4 lines
tests/functional/qemu_test/__init__.py line 12
Quoted 4 lines
tests/functional/qemu_test/__init__.py +0
-0from .config import BUILD_DIR from .cmd import has_cmd, has_cmds, run_cmd, is_readable_executable_file, \ interrupt_interactive_console_until_pattern, wait_for_console_pattern, \
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [PATCH v3 08/24] tests/functional: add a module for handling asset download & caching
Quoted 23 lines · Daniel P. Berrangé
Quoted 12 lines · Philippe Mathieu-Daudé
Quoted 11 lines · Daniel P. Berrangé
The 'Asset' class is a simple module that declares a downloadable asset that can be cached locally. Downloads are stored in the user's home dir at ~/.cache/qemu/download, using a sha256 sum of the URL.
Signed-off-by Daniel P. Berrangé <[email protected]>--- tests/functional/qemu_test/__init__.py | 1 + tests/functional/qemu_test/asset.py | 96 ++++++++++++++++++++++++++ 2 files changed, 97 insertions(+)create mode 100644
tests/functional/qemu_test/asset.pyQuoted 9 lines
tests/functional/qemu_test/__init__.py line 7
Quoted 7 lines
tests/functional/qemu_test/__init__.py +1
-0@@ -6,6 +6,7 @@ # later. See the COPYING file in the top-level directory. +from .asset import AssetFor next patch?
Quoted 3 lines
No, the idea is that a test class can just do 'from
qemu_testimport Asset', so they don't need to be aware of the fact that we've split up the code into separate files beneath the qemu_test/ directory.
Oh indeed I missed that, thanks for clarifying.
Re: [PATCH v3 08/24] tests/functional: add a module for handling asset download & caching
In reply to Daniel P. Berrangétests/functional/qemu_test/asset.py line 29
Quoted 7 lines · Daniel P. Berrangé
tests/functional/qemu_test/asset.py new file +7
-0+ def __init__(self, url, hash): + self.url = url + self.hash = hash + self.cache_dir = Path(Path("~").expanduser(), + ".cache", "qemu", "download") + self.cache_file = Path(self.cache_dir, + hashlib.sha256(url.encode("utf-8")).hexdigest())
Out of curiosity, why are we hashing the url from which we downloaded? Why not use the known hash of the contents?
If a url goes stale, and we adjust the url to a new host, we shouldn't need to re-download just because the url changed.
r~
Re: [PATCH v3 08/24] tests/functional: add a module for handling asset download & caching
Quoted 9 lines · Richard Henderson
tests/functional/qemu_test/asset.py line 29
Quoted 7 lines · Daniel P. Berrangé
tests/functional/qemu_test/asset.py new file +7
-0+ def __init__(self, url, hash): + self.url = url + self.hash = hash + self.cache_dir = Path(Path("~").expanduser(), + ".cache", "qemu", "download") + self.cache_file = Path(self.cache_dir, + hashlib.sha256(url.encode("utf-8")).hexdigest())
Quoted 5 lines
Out of curiosity, why are we hashing the url from which we downloaded? Why not use the known hash of the contents?
If a url goes stale, and we adjust the url to a new host, we shouldn't need to re-download just because the url changed.
Hmmm, good point, will change it.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
PATCH v3 9/24 tests/functional: enable pre-emptive caching of assets
In reply to Daniel P. BerrangéMany tests need to access assets stored on remote sites. We don't want to download these during test execution when run by meson, since this risks hitting test timeouts when data transfers are slow.
Add support for pre-emptive caching of assets by setting the env var QEMU_TEST_PRECACHE to point to a timestamp file. When this is set, instead of running the test, the assets will be downloaded and saved to the cache, then the timestamp file created.
A meson custom target is created as a dependency of each test suite to trigger the pre-emptive caching logic before the test runs.
When run in caching mode, it will locate assets by looking for class level variables with a name prefix "ASSET_", and type "Asset".
At the ninja level
ninja test --suite functional
will speculatively download any assets that are not already cached, so it is advisable to set a timeout multiplier.
QEMU_TEST_NO_DOWNLOAD=1 ninja test --suite functional
will fail the test if a required asset is not already cached
nina check-func-precache
will download and cache all assets required by the functional tests
At the make level, precaching is always done by
make check-functional
--- tests/Makefile.include | 3 ++- tests/functional/meson.build | 33 +++++++++++++++++++++++-- tests/functional/qemu_test/asset.py | 34 ++++++++++++++++++++++++++ tests/functional/qemu_test/testcase.py | 8 ++++++ 4 files changed, 75 insertions(+), 3 deletions(-)
tests/Makefile.include +2 -1
@@ -161,7 +161,8 @@ $(FUNCTIONAL_TARGETS): .PHONY: check-functional check-functional: - @$(MAKE) SPEED=thorough check-func check-func-quick + @$(MAKE) precache-functional + @QEMU_TEST_NO_DOWNLOAD=1 $(MAKE) SPEED=thorough check-func check-func-quick # Consolidated targets
tests/functional/meson.build +31 -2
@@ -33,6 +33,7 @@ tests_x86_64_quick = [ tests_x86_64_thorough = [ ] +precache_all = [] foreach speed : ['quick', 'thorough'] foreach dir : target_dirs if not dir.endswith('-softmmu')
@@ -63,11 +64,35 @@ foreach speed : ['quick', 'thorough'] meson.current_source_dir()) foreach test : target_tests - test('func-@0@/@1@'.format(target_base, test), + testname = '@0@-@1@'.format(target_base, test) + testfile = 'test_' + test + '.py' + testpath = meson.current_source_dir() / testfile + teststamp = testname + '.tstamp' + test_precache_env = environment() + test_precache_env.set('QEMU_TEST_PRECACHE', meson.current_build_dir() / teststamp) + test_precache_env.set('PYTHONPATH', meson.project_source_root() / 'python:' + + meson.current_source_dir()) + precache = custom_target('func-precache-' + testname, + output: teststamp, + command: [python, testpath], + depend_files: files(testpath), + build_by_default: false, + env: test_precache_env) + precache_all += precache + + # Ideally we would add 'precache' to 'depends' here, such that + # 'build_by_default: false' lets the pre-caching automatically + # run immediately before the test runs. In practice this is + # broken in meson, with it running the pre-caching in the normal + # compile phase https://github.com/mesonbuild/meson/issues/2518 + # If the above bug ever gets fixed, when QEMU changes the min + # meson version, add the 'depends' and remove the custom + # 'run_target' logic below & in Makefile.include + test('func-' + testname, python, depends: [test_deps, test_emulator, emulator_modules], env: test_env, - args: [meson.current_source_dir() / 'test_' + test + '.py'], + args: [testpath], protocol: 'tap', timeout: test_timeouts.get(test, 60), priority: test_timeouts.get(test, 60),
@@ -75,3 +100,7 @@ foreach speed : ['quick', 'thorough'] endforeach endforeach endforeach + +run_target('precache-functional', + depends: precache_all, + command: ['true'])
tests/functional/qemu_test/asset.py +34 -0
@@ -9,6 +9,8 @@ import logging import os import subprocess +import sys +import unittest import urllib.request from pathlib import Path from shutil import copyfileobj
@@ -61,6 +63,9 @@ def fetch(self): self.cache_file, self.url) return str(self.cache_file) + if os.environ.get("QEMU_TEST_NO_DOWNLOAD", False): + raise Exception("Asset cache is invalid and downloads disabled") + self.log.info("Downloading %s to %s...", self.url, self.cache_file) tmp_cache_file = self.cache_file.with_suffix(".download")
@@ -94,3 +99,32 @@ def fetch(self): self.log.info("Cached %s at %s" % (self.url, self.cache_file)) return str(self.cache_file) + + def precache_test(test): + log = logging.getLogger('qemu-test') + log.setLevel(logging.DEBUG) + handler = logging.StreamHandler(sys.stdout) + handler.setLevel(logging.DEBUG) + formatter = logging.Formatter( + '%(asctime)s - %(name)s - %(levelname)s - %(message)s') + handler.setFormatter(formatter) + log.addHandler(handler) + for name, asset in vars(test.__class__).items(): + if name.startswith("ASSET_") and type(asset) == Asset: + log.info("Attempting to cache '%s'" % asset) + asset.fetch() + log.removeHandler(handler) + + def precache_suite(suite): + for test in suite: + if isinstance(test, unittest.TestSuite): + Asset.precache_suite(test) + elif isinstance(test, unittest.TestCase): + Asset.precache_test(test) + + def precache_suites(path, cacheTstamp): + loader = unittest.loader.defaultTestLoader + tests = loader.loadTestsFromNames([path], None) + + with open(cacheTstamp, "w") as fh: + Asset.precache_suite(tests)
tests/functional/qemu_test/testcase.py +8 -0
@@ -21,6 +21,7 @@ from qemu.machine import QEMUMachine from qemu.utils import kvm_available, tcg_available +from .asset import Asset from .cmd import run_cmd from .config import BUILD_DIR
@@ -58,6 +59,13 @@ def tearDown(self): self.log.removeHandler(self._log_fh) def main(): + path = os.path.basename(sys.argv[0])[:-3] + + cache = os.environ.get("QEMU_TEST_PRECACHE", None) + if cache is not None: + Asset.precache_suites(path, cache) + return + tr = pycotap.TAPTestRunner(message_log = pycotap.LogMode.LogToError, test_output_log = pycotap.LogMode.LogToError) path = os.path.basename(sys.argv[0])[:-3]
Signature · 1 line
2.45.2
Re: [PATCH v3 09/24] tests/functional: enable pre-emptive caching of assets
Quoted 19 lines · Daniel P. Berrangé
Many tests need to access assets stored on remote sites. We don't want to download these during test execution when run by meson, since this risks hitting test timeouts when data transfers are slow.
Add support for pre-emptive caching of assets by setting the env var
QEMU_TEST_PRECACHEto point to a timestamp file. When this is set, instead of running the test, the assets will be downloaded and saved to the cache, then the timestamp file created.A meson custom target is created as a dependency of each test suite to trigger the pre-emptive caching logic before the test runs.
When run in caching mode, it will locate assets by looking for class level variables with a name prefix "
ASSET_", and type "Asset".At the ninja level
ninja test --suite functional
Quoted 8 lines
will speculatively download any assets that are not already cached, so it is advisable to set a timeout multiplier.
QEMU_TEST_NO_DOWNLOAD=1 ninja test --suite functionalwill fail the test if a required asset is not already cached
nina check-func-precache
ninja or make, precache-functional.
Anyway, thanks for this. I can confirm that it works, and that it takes ~50m to finish here.
r~
Re: [PATCH v3 09/24] tests/functional: enable pre-emptive caching of assets
In reply to Daniel P. BerrangéQuoted 57 lines · Daniel P. Berrangé
Many tests need to access assets stored on remote sites. We don't want to download these during test execution when run by meson, since this risks hitting test timeouts when data transfers are slow.
Add support for pre-emptive caching of assets by setting the env var
QEMU_TEST_PRECACHEto point to a timestamp file. When this is set, instead of running the test, the assets will be downloaded and saved to the cache, then the timestamp file created.A meson custom target is created as a dependency of each test suite to trigger the pre-emptive caching logic before the test runs.
When run in caching mode, it will locate assets by looking for class level variables with a name prefix "
ASSET_", and type "Asset".At the ninja level
ninja test --suite functionalwill speculatively download any assets that are not already cached, so it is advisable to set a timeout multiplier.
QEMU_TEST_NO_DOWNLOAD=1 ninja test --suite functionalwill fail the test if a required asset is not already cached
nina check-func-precache
will download and cache all assets required by the functional tests
At the make level, precaching is always done by
make check-functional
Signed-off-by Daniel P. Berrangé <[email protected]>--- tests/Makefile.include | 3 ++- tests/functional/meson.build | 33 +++++++++++++++++++++++-- tests/functional/qemu_test/asset.py | 34 ++++++++++++++++++++++++++ tests/functional/qemu_test/testcase.py | 8 ++++++ 4 files changed, 75 insertions(+), 3 deletions(-)tests/functional/qemu_test/testcase.py +1
-0@@ -21,6 +21,7 @@ from qemu.machine import QEMUMachine from qemu.utils import kvm_available, tcg_available +from .asset import Asset from .cmd import run_cmd from .config import BUILD_DIR
tests/functional/qemu_test/testcase.py def tearDown(self): · line 62
Quoted 4 lines
tests/functional/qemu_test/testcase.py +1
-0@@ -58,6 +59,13 @@ def tearDown(self): self.log.removeHandler(self._log_fh) def main(): + path = os.path.basename(sys.argv[0])[:-3]
This line should be added here in patch #2 in order to avoid ...
tests/functional/qemu_test/testcase.py def tearDown(self): · line 71
Quoted 9 lines
tests/functional/qemu_test/testcase.py +6
-0+ + cache = os.environ.get("QEMU_TEST_PRECACHE", None) + if cache is not None: + Asset.precache_suites(path, cache) + return + tr = pycotap.TAPTestRunner(message_log = pycotap.LogMode.LogToError, test_output_log = pycotap.LogMode.LogToError) path = os.path.basename(sys.argv[0])[:-3]
.... this duplication.
PATCH v3 10/24 tests/functional: Convert some tests that download files via fetch_asset()
In reply to Daniel P. BerrangéFrom: Thomas Huth <[email protected]>
Now that we've got the Asset class with pre-caching, we can convert some Avocado tests that use fetch_asset() for downloading their required files.
---
MAINTAINERS | 12 ++---
tests/avocado/machine_mips_loongson3v.py | 39 ---------------
tests/functional/meson.build | 22 ++++++++
.../test_arm_n8x0.py} | 36 ++++++-------
.../test_avr_mega2560.py} | 22 ++++----
.../test_loongarch64_virt.py} | 46 +++++++++--------
tests/functional/test_mips64el_loongson3v.py | 39 +++++++++++++++
.../test_netdev_ethtool.py} | 50 +++++++------------
.../ppc_405.py => functional/test_ppc_405.py} | 28 +++++------
9 files changed, 154 insertions(+), 140 deletions(-)
delete mode 100644 tests/avocado/machine_mips_loongson3v.py
rename tests/{avocado/machine_arm_n8x0.py => functional/test_arm_n8x0.py} (51%)
mode change 100644 => 100755
rename tests/{avocado/machine_avr6.py => functional/test_avr_mega2560.py} (73%)
mode change 100644 => 100755
rename tests/{avocado/machine_loongarch.py => functional/test_loongarch64_virt.py} (54%)
mode change 100644 => 100755
create mode 100755 tests/functional/test_mips64el_loongson3v.py
rename tests/{avocado/netdev-ethtool.py => functional/test_netdev_ethtool.py} (66%)
mode change 100644 => 100755
rename tests/{avocado/ppc_405.py => functional/test_ppc_405.py} (53%)
mode change 100644 => 100755MAINTAINERS +6 -6
@@ -222,7 +222,7 @@ S: Maintained F: docs/system/target-avr.rst F: gdb-xml/avr-cpu.xml F: target/avr/ -F: tests/avocado/machine_avr6.py +F: tests/functional/test_avr_mega2560.py CRIS TCG CPUs M: Edgar E. Iglesias <[email protected]>
@@ -265,7 +265,7 @@ M: Song Gao <[email protected]> S: Maintained F: target/loongarch/ F: tests/tcg/loongarch64/ -F: tests/avocado/machine_loongarch.py +F: tests/functional/test_loongarch64_virt.py M68K TCG CPUs M: Laurent Vivier <[email protected]>
@@ -884,7 +884,7 @@ F: include/hw/display/blizzard.h F: include/hw/input/lm832x.h F: include/hw/input/tsc2xxx.h F: include/hw/misc/cbus.h -F: tests/avocado/machine_arm_n8x0.py +F: tests/functional/test_arm_n8x0.py F: docs/system/arm/nseries.rst Palm
@@ -1393,7 +1393,7 @@ F: hw/mips/loongson3_bootp.h F: hw/mips/loongson3_virt.c F: include/hw/intc/loongson_ipi.h F: include/hw/intc/loongson_liointc.h -F: tests/avocado/machine_mips_loongson3v.py +F: tests/functional/test_mips64el_loongson3v.py Boston M: Paul Burton <[email protected]>
@@ -1419,7 +1419,7 @@ PowerPC Machines L: [email protected] S: Orphan F: hw/ppc/ppc405* -F: tests/avocado/ppc_405.py +F: tests/functional/test_ppc_405.py Bamboo L: [email protected]
@@ -2486,7 +2486,7 @@ R: Sriram Yagnaraman <[email protected]> S: Maintained F: docs/system/devices/igb.rst F: hw/net/igb* -F: tests/avocado/netdev-ethtool.py +F: tests/functional/test_netdev_ethtool.py F: tests/qtest/igb-test.c F: tests/qtest/libqos/igb.c
tests/avocado/machine_mips_loongson3v.py deleted +0 -39
@@ -1,39 +0,0 @@ -# Functional tests for the Generic Loongson-3 Platform. -# -# Copyright (c) 2021 Jiaxun Yang <[email protected]> -# -# This work is licensed under the terms of the GNU GPL, version 2 or later. -# See the COPYING file in the top-level directory. -# -# SPDX-License-Identifier: GPL-2.0-or-later - -import os -import time - -from avocado import skipUnless -from avocado_qemu import QemuSystemTest -from avocado_qemu import wait_for_console_pattern - -class MipsLoongson3v(QemuSystemTest): - timeout = 60 - - @skipUnless(os.getenv('AVOCADO_ALLOW_UNTRUSTED_CODE'), 'untrusted code') - def test_pmon_serial_console(self): - """ - :avocado: tags=arch:mips64el - :avocado: tags=endian:little - :avocado: tags=machine:loongson3-virt - :avocado: tags=cpu:Loongson-3A1000 - :avocado: tags=device:liointc - :avocado: tags=device:goldfish_rtc - """ - - pmon_hash = '7c8b45dd81ccfc55ff28f5aa267a41c3' - pmon_path = self.fetch_asset('https://github.com/loongson-community/pmon/' - 'releases/download/20210112/pmon-3avirt.bin', - asset_hash=pmon_hash, algorithm='md5') - - self.vm.set_console() - self.vm.add_args('-bios', pmon_path) - self.vm.launch() - wait_for_console_pattern(self, 'CPU GODSON3 BogoMIPS:')
tests/functional/meson.build +22 -0
@@ -11,6 +11,7 @@ endif # Timeouts for individual tests that can be slow e.g. with debugging enabled test_timeouts = { + 'netdev_ethtool' : 180, } tests_generic = [
@@ -19,10 +20,30 @@ tests_generic = [ 'version', ] +tests_arm_thorough = [ + 'arm_n8x0', +] + +tests_avr_thorough = [ + 'avr_mega2560', +] + +tests_loongarch64_thorough = [ + 'loongarch64_virt', +] + +tests_mips64el_thorough = [ + 'mips64el_loongson3v', +] + tests_ppc_quick = [ 'ppc_74xx', ] +tests_ppc_thorough = [ + 'ppc_405', +] + tests_x86_64_quick = [ 'cpu_queries', 'mem_addr_space',
@@ -31,6 +52,7 @@ tests_x86_64_quick = [ ] tests_x86_64_thorough = [ + 'netdev_ethtool', ] precache_all = []
tests/functional/test_arm_n8x0.py renamed from tests/avocado/machine_arm_n8x0.py +18 -18
@@ -1,3 +1,5 @@ +#!/usr/bin/env python3 +# # Functional test that boots a Linux kernel and checks the console # # Copyright (c) 2020 Red Hat, Inc.
@@ -10,21 +12,22 @@ import os -from avocado import skipUnless -from avocado_qemu import QemuSystemTest -from avocado_qemu import wait_for_console_pattern +from unittest import skipUnless +from qemu_test import QemuSystemTest, Asset +from qemu_test import wait_for_console_pattern class N8x0Machine(QemuSystemTest): """Boots the Linux kernel and checks that the console is operational""" timeout = 90 + ASSET_KERNEL = Asset(('http://stskeeps.subnetmask.net/meego-n8x0/' + 'meego-arm-n8x0-1.0.80.20100712.1431-' + 'vmlinuz-2.6.35~rc4-129.1-n8x0'), + 'e9d5ab8d7548923a0061b6fbf601465e479ed269') + def __do_test_n8x0(self): - kernel_url = ('http://stskeeps.subnetmask.net/meego-n8x0/' - 'meego-arm-n8x0-1.0.80.20100712.1431-' - 'vmlinuz-2.6.35~rc4-129.1-n8x0') - kernel_hash = 'e9d5ab8d7548923a0061b6fbf601465e479ed269' - kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash) + kernel_path = self.ASSET_KERNEL.fetch() self.vm.set_console(console_index=1) self.vm.add_args('-kernel', kernel_path,
@@ -32,18 +35,15 @@ def __do_test_n8x0(self): self.vm.launch() wait_for_console_pattern(self, 'TSC2005 driver initializing') - @skipUnless(os.getenv('AVOCADO_ALLOW_UNTRUSTED_CODE'), 'untrusted code') + @skipUnless(os.getenv('QEMU_TEST_ALLOW_UNTRUSTED_CODE'), 'untrusted code') def test_n800(self): - """ - :avocado: tags=arch:arm - :avocado: tags=machine:n800 - """ + self.set_machine('n800') self.__do_test_n8x0() - @skipUnless(os.getenv('AVOCADO_ALLOW_UNTRUSTED_CODE'), 'untrusted code') + @skipUnless(os.getenv('QEMU_TEST_ALLOW_UNTRUSTED_CODE'), 'untrusted code') def test_n810(self): - """ - :avocado: tags=arch:arm - :avocado: tags=machine:n810 - """ + self.set_machine('n810') self.__do_test_n8x0() + +if __name__ == '__main__': + QemuSystemTest.main()
tests/functional/test_avr_mega2560.py renamed from tests/avocado/machine_avr6.py +12 -10
@@ -1,3 +1,4 @@ +#!/usr/bin/env python3 # # QEMU AVR integration tests #
@@ -19,26 +20,24 @@ import time -from avocado_qemu import QemuSystemTest +from qemu_test import QemuSystemTest, Asset class AVR6Machine(QemuSystemTest): timeout = 5 + ASSET_ROM = Asset(('https://github.com/seharris/qemu-avr-tests' + '/raw/36c3e67b8755dcf/free-rtos/Demo' + '/AVR_ATMega2560_GCC/demo.elf'), + '7eb521f511ca8f2622e0a3c5e8dd686efbb911d4') + def test_freertos(self): - """ - :avocado: tags=arch:avr - :avocado: tags=machine:arduino-mega-2560-v3 - """ """ https://github.com/seharris/qemu-avr-tests/raw/master/free-rtos/Demo/AVR_ATMega2560_GCC/demo.elf constantly prints out 'ABCDEFGHIJKLMNOPQRSTUVWXABCDEFGHIJKLMNOPQRSTUVWX' """ - rom_url = ('https://github.com/seharris/qemu-avr-tests' - '/raw/36c3e67b8755dcf/free-rtos/Demo' - '/AVR_ATMega2560_GCC/demo.elf') - rom_hash = '7eb521f511ca8f2622e0a3c5e8dd686efbb911d4' - rom_path = self.fetch_asset(rom_url, asset_hash=rom_hash) + rom_path = self.ASSET_ROM.fetch() + self.set_machine('arduino-mega-2560-v3') self.vm.add_args('-bios', rom_path) self.vm.add_args('-nographic') self.vm.launch()
@@ -48,3 +47,6 @@ def test_freertos(self): self.assertIn('ABCDEFGHIJKLMNOPQRSTUVWXABCDEFGHIJKLMNOPQRSTUVWX', self.vm.get_log()) + +if __name__ == '__main__': + QemuSystemTest.main()
tests/functional/test_loongarch64_virt.py renamed from tests/avocado/machine_loongarch.py +25 -21
71 lines not on this page · raw message
tests/functional/test_mips64el_loongson3v.py new file +39 -0
@@ -0,0 +1,39 @@ +#!/usr/bin/env python3 +# +# Functional tests for the Generic Loongson-3 Platform. +# +# Copyright (c) 2021 Jiaxun Yang <[email protected]> +# +# This work is licensed under the terms of the GNU GPL, version 2 or later. +# See the COPYING file in the top-level directory. +# +# SPDX-License-Identifier: GPL-2.0-or-later + +import os +import time + +from unittest import skipUnless +from qemu_test import QemuSystemTest, Asset +from qemu_test import wait_for_console_pattern + +class MipsLoongson3v(QemuSystemTest): + timeout = 60 + + ASSET_PMON = Asset( + ('https://github.com/loongson-community/pmon/' + 'releases/download/20210112/pmon-3avirt.bin'), + 'fcdf6bb2cb7885a4a62f31fcb0d5e368bac7b6cea28f40c6dfa678af22fea20a') + + @skipUnless(os.getenv('QEMU_TEST_ALLOW_UNTRUSTED_CODE'), 'untrusted code') + def test_pmon_serial_console(self): + self.set_machine('loongson3-virt') + + pmon_path = self.ASSET_PMON.fetch() + + self.vm.set_console() + self.vm.add_args('-bios', pmon_path) + self.vm.launch() + wait_for_console_pattern(self, 'CPU GODSON3 BogoMIPS:') + +if __name__ == '__main__': + QemuSystemTest.main()
tests/functional/test_netdev_ethtool.py renamed from tests/avocado/netdev-ethtool.py +18 -32
89 lines not on this page · raw message
tests/functional/test_ppc_405.py renamed from tests/avocado/ppc_405.py +14 -14
@@ -1,3 +1,5 @@ +#!/usr/bin/env python3 +# # Test that the U-Boot firmware boots on ppc 405 machines and check the console # # Copyright (c) 2021 Red Hat, Inc.
@@ -5,20 +7,20 @@ # This work is licensed under the terms of the GNU GPL, version 2 or # later. See the COPYING file in the top-level directory. -from avocado.utils import archive -from avocado_qemu import QemuSystemTest -from avocado_qemu import wait_for_console_pattern -from avocado_qemu import exec_command_and_wait_for_pattern +from qemu_test import QemuSystemTest, Asset +from qemu_test import wait_for_console_pattern +from qemu_test import exec_command_and_wait_for_pattern class Ppc405Machine(QemuSystemTest): timeout = 90 + ASSET_UBOOT = Asset(('https://gitlab.com/huth/u-boot/-/raw/' + 'taihu-2021-10-09/u-boot-taihu.bin'), + '3208940e908a5edc7c03eab072c60f0dcfadc2ab') + def do_test_ppc405(self): - uboot_url = ('https://gitlab.com/huth/u-boot/-/raw/' - 'taihu-2021-10-09/u-boot-taihu.bin') - uboot_hash = ('3208940e908a5edc7c03eab072c60f0dcfadc2ab'); - file_path = self.fetch_asset(uboot_url, asset_hash=uboot_hash) + file_path = self.ASSET_UBOOT.fetch() self.vm.set_console(console_index=1) self.vm.add_args('-bios', file_path) self.vm.launch()
@@ -26,11 +28,9 @@ def do_test_ppc405(self): exec_command_and_wait_for_pattern(self, 'reset', 'AMCC PowerPC 405EP') def test_ppc_ref405ep(self): - """ - :avocado: tags=arch:ppc - :avocado: tags=machine:ref405ep - :avocado: tags=cpu:405ep - :avocado: tags=accel:tcg - """ self.require_accelerator("tcg") + self.set_machine('ref405ep') self.do_test_ppc405() + +if __name__ == '__main__': + QemuSystemTest.main()
Signature · 1 line
2.45.2
PATCH v3 11/24 tests/functional: Add a function for extracting files from an archive
In reply to Daniel P. BerrangéFrom: Thomas Huth <[email protected]>
Some Avocado-based tests use the "archive" module from avocado.utils to extract files from an archive. To be able to use these tests without Avocado, we have to provide our own function for extracting files. Fortunately, there is already the tarfile module that will provide us with this functionality, so let's just add a nice wrapper function around that.
--- tests/functional/qemu_test/utils.py | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) create mode 100644 tests/functional/qemu_test/utils.py
tests/functional/qemu_test/utils.py new file +21 -0
@@ -0,0 +1,21 @@ +# Utilities for python-based QEMU tests +# +# Copyright 2024 Red Hat, Inc. +# +# Authors: +# Thomas Huth <[email protected]> +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + +import tarfile + +def archive_extract(archive, dest_dir, member=None): + with tarfile.open(archive) as tf: + if hasattr(tarfile, 'data_filter'): + tf.extraction_filter = getattr(tarfile, 'data_filter', + (lambda member, path: member)) + if member: + tf.extract(member=member, path=dest_dir) + else: + tf.extractall(path=dest_dir)
Signature · 1 line
2.45.2
Re: [PATCH v3 11/24] tests/functional: Add a function for extracting files from an archive
Quoted 9 lines · Daniel P. Berrangé
From: Thomas Huth <[email protected]>
Some Avocado-based tests use the "archive" module from avocado.utils to extract files from an archive. To be able to use these tests without Avocado, we have to provide our own function for extracting files. Fortunately, there is already the tarfile module that will provide us with this functionality, so let's just add a nice wrapper function around that.
Quoted 7 lines
Reviewed-by Daniel P. Berrangé <[email protected]>Reviewed-by Philippe Mathieu-Daudé <[email protected]>Signed-off-by Thomas Huth <[email protected]>--- tests/functional/qemu_test/utils.py | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+)create mode 100644
tests/functional/qemu_test/utils.py
PATCH v3 12/24 tests/functional: Convert some avocado tests that needed avocado.utils.archive
In reply to Daniel P. BerrangéFrom: Thomas Huth <[email protected]>
Instead of using the "archive" module from avocado.utils, switch these tests to use the new wrapper function that is based on the "tarfile" module instead.
---
MAINTAINERS | 6 +--
tests/avocado/machine_sparc64_sun4u.py | 36 -----------------
tests/functional/meson.build | 6 +++
.../test_arm_canona1100.py} | 30 ++++++++------
.../test_ppc_bamboo.py} | 32 +++++++--------
tests/functional/test_sparc64_sun4u.py | 40 +++++++++++++++++++
6 files changed, 82 insertions(+), 68 deletions(-)
delete mode 100644 tests/avocado/machine_sparc64_sun4u.py
rename tests/{avocado/machine_arm_canona1100.py => functional/test_arm_canona1100.py} (52%)
mode change 100644 => 100755
rename tests/{avocado/ppc_bamboo.py => functional/test_ppc_bamboo.py} (60%)
mode change 100644 => 100755
create mode 100755 tests/functional/test_sparc64_sun4u.pyMAINTAINERS +3 -3
@@ -731,7 +731,7 @@ S: Odd Fixes F: include/hw/arm/digic.h F: hw/*/digic* F: include/hw/*/digic* -F: tests/avocado/machine_arm_canona1100.py +F: tests/functional/test_arm_canona1100.py F: docs/system/arm/digic.rst Goldfish RTC
@@ -1426,7 +1426,7 @@ L: [email protected] S: Orphan F: hw/ppc/ppc440_bamboo.c F: hw/pci-host/ppc4xx_pci.c -F: tests/avocado/ppc_bamboo.py +F: tests/functional/test_ppc_bamboo.py e500 L: [email protected]
@@ -1712,7 +1712,7 @@ F: include/hw/pci-host/sabre.h F: hw/pci-bridge/simba.c F: include/hw/pci-bridge/simba.h F: pc-bios/openbios-sparc64 -F: tests/avocado/machine_sparc64_sun4u.py +F: tests/functional/test_sparc64_sun4u.py Sun4v M: Artyom Tarasenko <[email protected]>
tests/avocado/machine_sparc64_sun4u.py deleted +0 -36
@@ -1,36 +0,0 @@ -# Functional test that boots a Linux kernel and checks the console -# -# Copyright (c) 2020 Red Hat, Inc. -# -# Author: -# Thomas Huth <[email protected]> -# -# This work is licensed under the terms of the GNU GPL, version 2 or -# later. See the COPYING file in the top-level directory. - -import os - -from avocado_qemu import wait_for_console_pattern -from avocado.utils import archive -from boot_linux_console import LinuxKernelTest - -class Sun4uMachine(LinuxKernelTest): - """Boots the Linux kernel and checks that the console is operational""" - - timeout = 90 - - def test_sparc64_sun4u(self): - """ - :avocado: tags=arch:sparc64 - :avocado: tags=machine:sun4u - """ - tar_url = ('https://qemu-advcal.gitlab.io' - '/qac-best-of-multiarch/download/day23.tar.xz') - tar_hash = '142db83cd974ffadc4f75c8a5cad5bcc5722c240' - file_path = self.fetch_asset(tar_url, asset_hash=tar_hash) - archive.extract(file_path, self.workdir) - self.vm.set_console() - self.vm.add_args('-kernel', self.workdir + '/day23/vmlinux', - '-append', self.KERNEL_COMMON_COMMAND_LINE) - self.vm.launch() - wait_for_console_pattern(self, 'Starting logging: OK')
tests/functional/meson.build +6 -0
@@ -21,6 +21,7 @@ tests_generic = [ ] tests_arm_thorough = [ + 'arm_canona1100', 'arm_n8x0', ]
@@ -42,6 +43,11 @@ tests_ppc_quick = [ tests_ppc_thorough = [ 'ppc_405', + 'ppc_bamboo', +] + +tests_sparc64_thorough = [ + 'sparc64_sun4u', ] tests_x86_64_quick = [
tests/functional/test_arm_canona1100.py renamed from tests/avocado/machine_arm_canona1100.py +17 -13
@@ -1,3 +1,5 @@ +#!/usr/bin/env python3 +# # Functional test that boots the canon-a1100 machine with firmware # # Copyright (c) 2020 Red Hat, Inc.
@@ -8,28 +10,30 @@ # This work is licensed under the terms of the GNU GPL, version 2 or # later. See the COPYING file in the top-level directory. -from avocado_qemu import QemuSystemTest -from avocado_qemu import wait_for_console_pattern -from avocado.utils import archive +from qemu_test import QemuSystemTest, Asset +from qemu_test import wait_for_console_pattern +from qemu_test.utils import archive_extract class CanonA1100Machine(QemuSystemTest): """Boots the barebox firmware and checks that the console is operational""" timeout = 90 + ASSET_BIOS = Asset(('https://qemu-advcal.gitlab.io' + '/qac-best-of-multiarch/download/day18.tar.xz'), + '068b5fc4242b29381acee94713509f8a876e9db6') + def test_arm_canona1100(self): - """ - :avocado: tags=arch:arm - :avocado: tags=machine:canon-a1100 - :avocado: tags=device:pflash_cfi02 - """ - tar_url = ('https://qemu-advcal.gitlab.io' - '/qac-best-of-multiarch/download/day18.tar.xz') - tar_hash = '068b5fc4242b29381acee94713509f8a876e9db6' - file_path = self.fetch_asset(tar_url, asset_hash=tar_hash) - archive.extract(file_path, self.workdir) + self.set_machine('canon-a1100') + + file_path = self.ASSET_BIOS.fetch() + archive_extract(file_path, dest_dir=self.workdir, + member="day18/barebox.canon-a1100.bin") self.vm.set_console() self.vm.add_args('-bios', self.workdir + '/day18/barebox.canon-a1100.bin') self.vm.launch() wait_for_console_pattern(self, 'running /env/bin/init') + +if __name__ == '__main__': + QemuSystemTest.main()
tests/functional/test_ppc_bamboo.py renamed from tests/avocado/ppc_bamboo.py +16 -16
@@ -1,3 +1,5 @@ +#!/usr/bin/env python3 +# # Test that Linux kernel boots on the ppc bamboo board and check the console # # Copyright (c) 2021 Red Hat
@@ -5,30 +7,25 @@ # This work is licensed under the terms of the GNU GPL, version 2 or # later. See the COPYING file in the top-level directory. -from avocado.utils import archive -from avocado_qemu import QemuSystemTest -from avocado_qemu import wait_for_console_pattern -from avocado_qemu import exec_command_and_wait_for_pattern +from qemu_test.utils import archive_extract +from qemu_test import QemuSystemTest, Asset +from qemu_test import wait_for_console_pattern +from qemu_test import exec_command_and_wait_for_pattern class BambooMachine(QemuSystemTest): timeout = 90 + ASSET_IMAGE = Asset(('http://landley.net/aboriginal/downloads/binaries/' + 'system-image-powerpc-440fp.tar.gz'), + '53e5f16414b195b82d2c70272f81c2eedb39bad9') + def test_ppc_bamboo(self): - """ - :avocado: tags=arch:ppc - :avocado: tags=machine:bamboo - :avocado: tags=cpu:440epb - :avocado: tags=device:rtl8139 - :avocado: tags=accel:tcg - """ + self.set_machine('bamboo') self.require_accelerator("tcg") self.require_netdev('user') - tar_url = ('http://landley.net/aboriginal/downloads/binaries/' - 'system-image-powerpc-440fp.tar.gz') - tar_hash = '53e5f16414b195b82d2c70272f81c2eedb39bad9' - file_path = self.fetch_asset(tar_url, asset_hash=tar_hash) - archive.extract(file_path, self.workdir) + file_path = self.ASSET_IMAGE.fetch() + archive_extract(file_path, self.workdir) self.vm.set_console() self.vm.add_args('-kernel', self.workdir + '/system-image-powerpc-440fp/linux',
@@ -40,3 +37,6 @@ def test_ppc_bamboo(self): exec_command_and_wait_for_pattern(self, 'ping 10.0.2.2', '10.0.2.2 is alive!') exec_command_and_wait_for_pattern(self, 'halt', 'System Halted') + +if __name__ == '__main__': + QemuSystemTest.main()
tests/functional/test_sparc64_sun4u.py new file +40 -0
@@ -0,0 +1,40 @@ +#!/usr/bin/env python3 +# +# Functional test that boots a Linux kernel and checks the console +# +# Copyright (c) 2020 Red Hat, Inc. +# +# Author: +# Thomas Huth <[email protected]> +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + +import os + +from qemu_test import QemuSystemTest, Asset +from qemu_test import wait_for_console_pattern +from qemu_test.utils import archive_extract + +class Sun4uMachine(QemuSystemTest): + """Boots the Linux kernel and checks that the console is operational""" + + timeout = 90 + + ASSET_IMAGE = Asset(('https://qemu-advcal.gitlab.io' + '/qac-best-of-multiarch/download/day23.tar.xz'), + '142db83cd974ffadc4f75c8a5cad5bcc5722c240') + + def test_sparc64_sun4u(self): + self.set_machine('sun4u') + file_path = self.ASSET_IMAGE.fetch() + kernel_name = 'day23/vmlinux' + archive_extract(file_path, self.workdir, kernel_name) + self.vm.set_console() + self.vm.add_args('-kernel', os.path.join(self.workdir, kernel_name), + '-append', 'printk.time=0') + self.vm.launch() + wait_for_console_pattern(self, 'Starting logging: OK') + +if __name__ == '__main__': + QemuSystemTest.main()
Signature · 1 line
2.45.2
Re: [PATCH v3 12/24] tests/functional: Convert some avocado tests that needed avocado.utils.archive
Quoted 16 lines · Daniel P. Berrangé
From: Thomas Huth <[email protected]>
Instead of using the "archive" module from avocado.utils, switch these tests to use the new wrapper function that is based on the "tarfile" module instead.
Reviewed-by Daniel P. Berrangé <[email protected]>Signed-off-by Thomas Huth <[email protected]>--- MAINTAINERS | 6 +-- tests/avocado/machine_sparc64_sun4u.py | 36 ----------------- tests/functional/meson.build | 6 +++ .../test_arm_canona1100.py} | 30 ++++++++------ .../test_ppc_bamboo.py} | 32 +++++++-------- tests/functional/test_sparc64_sun4u.py | 40 +++++++++++++++++++ 6 files changed, 82 insertions(+), 68 deletions(-)
Quoted 6 lines
delete mode 100644
tests/avocado/machine_sparc64_sun4u.py
renametests/{avocado/machine_arm_canona1100.py=>functional/test_arm_canona1100.py} (52%) mode change 100644 => 100755
renametests/{avocado/ppc_bamboo.py=>functional/test_ppc_bamboo.py} (60%) mode change 100644 => 100755
create mode 100755tests/functional/test_sparc64_sun4u.py
PATCH v3 13/24 tests/functional: Convert the s390x avocado tests into standalone tests
In reply to Daniel P. BerrangéFrom: Thomas Huth <[email protected]>
These tests use archive.lzma_uncompress() from the Avocado utils, so provide a small helper function for this, based on the standard lzma module from Python instead.
And while we're at it, replace the MD5 hashes in the topology test with proper SHA256 hashes, since MD5 should not be used anymore nowadays.
---
MAINTAINERS | 4 +-
tests/functional/meson.build | 6 ++
tests/functional/qemu_test/utils.py | 14 +++
.../test_s390x_ccw_virtio.py} | 79 ++++++++---------
.../test_s390x_topology.py} | 86 ++++++++-----------
5 files changed, 94 insertions(+), 95 deletions(-)
rename tests/{avocado/machine_s390_ccw_virtio.py => functional/test_s390x_ccw_virtio.py} (85%)
mode change 100644 => 100755
rename tests/{avocado/s390_topology.py => functional/test_s390x_topology.py} (88%)
mode change 100644 => 100755MAINTAINERS +2 -2
@@ -1739,7 +1739,7 @@ S: Supported F: hw/s390x/ F: include/hw/s390x/ F: configs/devices/s390x-softmmu/default.mak -F: tests/avocado/machine_s390_ccw_virtio.py +F: tests/functional/test_s390x_ccw_virtio.py T: git https://github.com/borntraeger/qemu.git s390-next L: [email protected]
@@ -1802,7 +1802,7 @@ F: hw/s390x/cpu-topology.c F: target/s390x/kvm/stsi-topology.c F: docs/devel/s390-cpu-topology.rst F: docs/system/s390x/cpu-topology.rst -F: tests/avocado/s390_topology.py +F: tests/functional/test_s390x_topology.py X86 Machines ------------
tests/functional/meson.build +6 -0
@@ -12,6 +12,7 @@ endif # Timeouts for individual tests that can be slow e.g. with debugging enabled test_timeouts = { 'netdev_ethtool' : 180, + 's390x_ccw_virtio' : 180, } tests_generic = [
@@ -46,6 +47,11 @@ tests_ppc_thorough = [ 'ppc_bamboo', ] +tests_s390x_thorough = [ + 's390x_ccw_virtio', + 's390x_topology', +] + tests_sparc64_thorough = [ 'sparc64_sun4u', ]
tests/functional/qemu_test/utils.py +14 -0
@@ -8,6 +8,9 @@ # This work is licensed under the terms of the GNU GPL, version 2 or # later. See the COPYING file in the top-level directory. +import lzma +import os +import shutil import tarfile def archive_extract(archive, dest_dir, member=None):
@@ -19,3 +22,14 @@ def archive_extract(archive, dest_dir, member=None): tf.extract(member=member, path=dest_dir) else: tf.extractall(path=dest_dir) + +def lzma_uncompress(xz_path, output_path): + if os.path.exists(output_path): + return + with lzma.open(xz_path, 'rb') as lzma_in: + try: + with open(output_path, 'wb') as raw_out: + shutil.copyfileobj(lzma_in, raw_out) + except: + os.remove(output_path) + raise
tests/functional/test_s390x_ccw_virtio.py renamed from tests/avocado/machine_s390_ccw_virtio.py +38 -41
113 lines not on this page · raw message
tests/functional/test_s390x_topology.py renamed from tests/avocado/s390_topology.py +34 -52
183 lines not on this page · raw message
Signature · 1 line
2.45.2
Re: [PATCH v3 13/24] tests/functional: Convert the s390x avocado tests into standalone tests
Quoted 18 lines · Daniel P. Berrangé
From: Thomas Huth <[email protected]>
These tests use
archive.lzma_uncompress()from the Avocado utils, so provide a small helper function for this, based on the standard lzma module from Python instead.And while we're at it, replace the MD5 hashes in the topology test with proper SHA256 hashes, since MD5 should not be used anymore nowadays.
Signed-off-by Thomas Huth <[email protected]>--- MAINTAINERS | 4 +- tests/functional/meson.build | 6 ++ tests/functional/qemu_test/utils.py | 14 +++ .../test_s390x_ccw_virtio.py} | 79 ++++++++--------- .../test_s390x_topology.py} | 86 ++++++++----------- 5 files changed, 94 insertions(+), 95 deletions(-)
Quoted 4 lines
rename
tests/{avocado/machine_s390_ccw_virtio.py=>functional/test_s390x_ccw_virtio.py} (85%) mode change 100644 => 100755
renametests/{avocado/s390_topology.py=>functional/test_s390x_topology.py} (88%) mode change 100644 => 100755
Conversion LGTM, so:
Now while testing I'm getting errors, but maybe pre-existing...
pyvenv/bin/meson test --no-rebuild -t 1 --setup thorough --num-processes 1 --print-errorlogs --suite func-s390x --suite func-s390x-thorough qemu:func-thorough+func-s390x-thorough+thorough / func-s390x-s390x_ccw_virtio time out (After 180.0 seconds)
1/5 qemu:func-thorough+func-s390x-thorough+thorough / func-s390x-s390x_ccw_virtio TIMEOUT 180.05s killed by signal 15 SIGTERM
▶ 2/5 test_empty_cpu_model.EmptyCPUModel.test
FAIL
2/5 qemu:func-quick+func-s390x / func-s390x-empty_cpu_model
ERROR 0.21s exit status 1
stderr:
Traceback (most recent call last):
File "/home/user/qemu/tests/functional/test_empty_cpu_model.py", line
21, in test
self.assertRegex(self.vm.get_log(), r'-cpu option cannot be empty')
AssertionError: Regex didn't match: '-cpu option cannot be empty' not found in 'Could not access KVM kernel module: Permission denied\nqemu-system-s390x: failed to initialize kvm: Permission denied\n'
(test program exited with status code 1)
▶ 4/5 test_version.Version.test_qmp_human_info_version
FAIL
Traceback (most recent call last):
File "/home/user/qemu/python/qemu/qmp/protocol.py", line 425, in_session_guard
await coro
File "/home/user/qemu/python/qemu/qmp/qmp_client.py", line 250, in_establish_session
self._greeting = await self._get_greeting()
File "/home/user/qemu/python/qemu/qmp/qmp_client.py", line 270, in_get_greeting
msg = await self._recv()
File "/home/user/qemu/python/qemu/qmp/protocol.py", line 1009, in _recv
message = await self._do_recv()
File "/home/user/qemu/python/qemu/qmp/qmp_client.py", line 402, in_do_recv
msg_bytes = await self._readline()
File "/home/user/qemu/python/qemu/qmp/protocol.py", line 977, in_readline
raise EOFError
EOFError
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/user/qemu/python/qemu/machine/machine.py", line 448, in
launch
self._launch()
File "/home/user/qemu/python/qemu/machine/machine.py", line 487, in
_launch
self._post_launch()
File "/home/user/qemu/python/qemu/machine/machine.py", line 381, in
_post_launch
self._qmp.connect()
File "/home/user/qemu/python/qemu/qmp/legacy.py", line 153, in connect
self._sync(
File "/home/user/qemu/python/qemu/qmp/legacy.py", line 102, in _sync
return self._aloop.run_until_complete( File "/usr/lib/python3.10/asyncio/base_events.py", line 649, inrun_until_complete
return future.result()
File "/usr/lib/python3.10/asyncio/tasks.py", line 408, in wait_for
return await fut
File "/home/user/qemu/python/qemu/qmp/protocol.py", line 382, in connect
await self._session_guard(
File "/home/user/qemu/python/qemu/qmp/protocol.py", line 456, in_session_guard
raise ConnectError(emsg, err) from err
qemu.qmp.protocol.ConnectError: Failed to establish session: EOFError
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/user/qemu/tests/functional/test_version.py", line 22, intest_qmp_human_info_version
self.vm.launch()
File "/home/user/qemu/python/qemu/machine/machine.py", line 461, in
launch
raise VMLaunchFailure( qemu.machine.machine.VMLaunchFailure: ConnectError: Failed to establish session: EOFError Exit code: 1 Command: qemu-system-s390x -display none -vga none -chardev
socket,id=mon,fd=5 -mon chardev=mon,mode=control -machine none -nodefaults
Output: Could not access KVM kernel module: Permission denied
qemu-system-s390x: failed to initialize kvm: Permission denied
5/5 qemu:func-thorough+func-s390x-thorough+thorough / func-s390x-s390x_topology SKIP 1.14s 0 subtests passed
Summary of Failures:
1/5 qemu:func-thorough+func-s390x-thorough+thorough / func-s390x-s390x_ccw_virtio TIMEOUT 180.05s killed by signal 15 SIGTERM
2/5 qemu:func-quick+func-s390x / func-s390x-empty_cpu_model
ERROR 0.21s exit status 1
4/5 qemu:func-quick+func-s390x / func-s390x-version
ERROR 0.25s exit status 1
I'm surprised by "Could not access KVM kernel module" because kvm_available() checks for RW access:
def kvm_available(target_arch: Optional[str] = None,
qemu_bin: Optional[str] = None) -> bool:
if not os.access("/dev/kvm", os.R_OK | os.W_OK):
return FalseMaybe missing require_accelerator() in tests?
Re: [PATCH v3 13/24] tests/functional: Convert the s390x avocado tests into standalone tests
Quoted 24 lines · Philippe Mathieu-Daudé
Quoted 18 lines · Daniel P. Berrangé
From: Thomas Huth <[email protected]>
These tests use
archive.lzma_uncompress()from the Avocado utils, so provide a small helper function for this, based on the standard lzma module from Python instead.And while we're at it, replace the MD5 hashes in the topology test with proper SHA256 hashes, since MD5 should not be used anymore nowadays.
Signed-off-by Thomas Huth <[email protected]>--- MAINTAINERS | 4 +- tests/functional/meson.build | 6 ++ tests/functional/qemu_test/utils.py | 14 +++ .../test_s390x_ccw_virtio.py} | 79 ++++++++--------- .../test_s390x_topology.py} | 86 ++++++++----------- 5 files changed, 94 insertions(+), 95 deletions(-)Quoted 4 lines
rename
tests/{avocado/machine_s390_ccw_virtio.py=>functional/test_s390x_ccw_virtio.py} (85%) mode change 100644 => 100755
renametests/{avocado/s390_topology.py=>functional/test_s390x_topology.py} (88%) mode change 100644 => 100755
Quoted 5 lines
Conversion LGTM, so:
Reviewed-by Philippe Mathieu-Daudé <[email protected]>Now while testing I'm getting errors, but maybe pre-existing...
What host arch / OS are you running tests from ?
I got through the full set of testing on Fedora x86_64, so wondering if something differs.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [PATCH v3 13/24] tests/functional: Convert the s390x avocado tests into standalone tests
Quoted 27 lines · Daniel P. Berrangé
Quoted 26 lines · Philippe Mathieu-Daudé
Quoted 18 lines · Daniel P. Berrangé
From: Thomas Huth <[email protected]>
These tests use
archive.lzma_uncompress()from the Avocado utils, so provide a small helper function for this, based on the standard lzma module from Python instead.And while we're at it, replace the MD5 hashes in the topology test with proper SHA256 hashes, since MD5 should not be used anymore nowadays.
Signed-off-by Thomas Huth <[email protected]>--- MAINTAINERS | 4 +- tests/functional/meson.build | 6 ++ tests/functional/qemu_test/utils.py | 14 +++ .../test_s390x_ccw_virtio.py} | 79 ++++++++--------- .../test_s390x_topology.py} | 86 ++++++++----------- 5 files changed, 94 insertions(+), 95 deletions(-)Quoted 4 lines
rename
tests/{avocado/machine_s390_ccw_virtio.py=>functional/test_s390x_ccw_virtio.py} (85%) mode change 100644 => 100755
renametests/{avocado/s390_topology.py=>functional/test_s390x_topology.py} (88%) mode change 100644 => 100755Conversion LGTM, so:
Quoted 8 lines
Quoted 3 lines
Reviewed-by Philippe Mathieu-Daudé <[email protected]>Now while testing I'm getting errors, but maybe pre-existing...
What host arch / OS are you running tests from ?
I got through the full set of testing on Fedora
x86_64, so wondering if something differs.
Linux s390x but I'm not in the KVM group, I need to find another place to run these (or ask Alex :P).
Re: [PATCH v3 13/24] tests/functional: Convert the s390x avocado tests into standalone tests
Quoted 32 lines · Philippe Mathieu-Daudé
Quoted 31 lines · Daniel P. Berrangé
Quoted 29 lines · Philippe Mathieu-Daudé
Quoted 17 lines · Daniel P. Berrangé
From: Thomas Huth <[email protected]>
These tests use
archive.lzma_uncompress()from the Avocado utils, so provide a small helper function for this, based on the standard lzma module from Python instead.And while we're at it, replace the MD5 hashes in the topology test with proper SHA256 hashes, since MD5 should not be used anymore nowadays.
Signed-off-by Thomas Huth <[email protected]>---
MAINTAINERS | 4 +- tests/functional/meson.build | 6 ++ tests/functional/qemu_test/utils.py | 14 +++ .../test_s390x_ccw_virtio.py} | 79 ++++++++--------- .../test_s390x_topology.py} | 86 ++++++++-----------Quoted 5 lines
5 files changed, 94 insertions(+), 95 deletions(-)rename
tests/{avocado/machine_s390_ccw_virtio.py=>functional/test_s390x_ccw_virtio.py} (85%) mode change 100644 => 100755
renametests/{avocado/s390_topology.py=>functional/test_s390x_topology.py} (88%) mode change 100644 => 100755Conversion LGTM, so:
Reviewed-by Philippe Mathieu-Daudé <[email protected]>Now while testing I'm getting errors, but maybe pre-existing...
Quoted 7 lines
Quoted 4 lines
What host arch / OS are you running tests from ?
I got through the full set of testing on Fedora
x86_64, so wondering if something differs.Linux s390x but I'm not in the KVM group, I need to find another place to run these (or ask Alex :P).
Sounds like a pre-existing problem ... I guess the test should check whether it can access /dev/kvm before trying to use it...?
Thomas
Re: [PATCH v3 13/24] tests/functional: Convert the s390x avocado tests into standalone tests
Quoted 35 lines · Thomas Huth
Quoted 34 lines · Philippe Mathieu-Daudé
Quoted 33 lines · Daniel P. Berrangé
Quoted 24 lines · Philippe Mathieu-Daudé
Quoted 17 lines · Daniel P. Berrangé
From: Thomas Huth <[email protected]>
These tests use
archive.lzma_uncompress()from the Avocado utils, so provide a small helper function for this, based on the standard lzma module from Python instead.And while we're at it, replace the MD5 hashes in the topology test with proper SHA256 hashes, since MD5 should not be used anymore nowadays.
Signed-off-by Thomas Huth <[email protected]>---
MAINTAINERS | 4 +- tests/functional/meson.build | 6 ++ tests/functional/qemu_test/utils.py | 14 +++ .../test_s390x_ccw_virtio.py} | 79 ++++++++--------- .../test_s390x_topology.py} | 86 ++++++++-----------Quoted 5 lines
5 files changed, 94 insertions(+), 95 deletions(-)rename
tests/{avocado/machine_s390_ccw_virtio.py=>functional/test_s390x_ccw_virtio.py} (85%) mode change 100644 => 100755
renametests/{avocado/s390_topology.py=>functional/test_s390x_topology.py} (88%) mode change 100644 => 100755Quoted 5 lines
Conversion LGTM, so:
Reviewed-by Philippe Mathieu-Daudé <[email protected]>Now while testing I'm getting errors, but maybe pre-existing...
What host arch / OS are you running tests from ?
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
I got through the full set of testing on Fedora
x86_64, so wondering if something differs.Linux s390x but I'm not in the KVM group, I need to find another place to run these (or ask Alex :P).
Sounds like a pre-existing problem ... I guess the test should check whether it can access
/dev/kvmbefore trying to use it...?
Ah, no, I missed your comment in your earlier patch that kvm_available() already checks this ... so the problem must be something else?
Thomas
PATCH v3 14/24 tests/functional: Convert the x86_cpu_model_versions test
In reply to Daniel P. BerrangéFrom: Thomas Huth <[email protected]>
Nothing thrilling in here, it's just a straight forward conversion.
---
tests/functional/meson.build | 1 +
.../test_x86_cpu_model_versions.py} | 63 ++++++-------------
2 files changed, 20 insertions(+), 44 deletions(-)
rename tests/{avocado/x86_cpu_model_versions.py => functional/test_x86_cpu_model_versions.py} (92%)
mode change 100644 => 100755tests/functional/meson.build +1 -0
@@ -61,6 +61,7 @@ tests_x86_64_quick = [ 'mem_addr_space', 'pc_cpu_hotplug_props', 'virtio_version', + 'x86_cpu_model_versions', ] tests_x86_64_thorough = [
tests/functional/test_x86_cpu_model_versions.py renamed from tests/avocado/x86_cpu_model_versions.py +19 -44
154 lines not on this page · raw message
Signature · 1 line
2.45.2
Re: [PATCH v3 14/24] tests/functional: Convert the x86_cpu_model_versions test
Quoted 14 lines · Daniel P. Berrangé
From: Thomas Huth <[email protected]>
Nothing thrilling in here, it's just a straight forward conversion.
Signed-off-by Thomas Huth <[email protected]>--- tests/functional/meson.build | 1 + .../test_x86_cpu_model_versions.py} | 63 ++++++------------- 2 files changed, 20 insertions(+), 44 deletions(-)rename
tests/{avocado/x86_cpu_model_versions.py=>functional/test_x86_cpu_model_versions.py} (92%) mode change 100644 => 100755
tests/functional/test_x86_cpu_model_versions.py def test_4_1_alias(self): · line 220
Quoted 7 lines
tests/functional/test_x86_cpu_model_versions.py renamed from tests/avocado/x86_cpu_model_versions.py +1
-2@@ -217,9 +216,8 @@ def test_4_1_alias(self): def test_none_alias(self): """ Check if unversioned CPU model is an alias pointing to some version - - :avocado: tags=machine:none """ + self.machine = 'none'
self.set_machine('none')
Review def test_4_1_alias(self): · line 222
Quoted 2 lines
(quoted diff) +0
-0self.vm.add_args('-S') self.vm.launch()
PATCH v3 15/24 tests/functional: Convert the microblaze avocado tests into standalone tests
In reply to Daniel P. BerrangéFrom: Thomas Huth <[email protected]>
The machine_microblaze.py file contained two tests, one for each endianess. Since we only support one QEMU target binary per file in the new functional test environment, we have to split this file up into two files now.
--- MAINTAINERS | 2 +- tests/avocado/machine_microblaze.py | 61 ------------------- tests/functional/meson.build | 8 +++ .../functional/test_microblaze_s3adsp1800.py | 39 ++++++++++++ .../test_microblazeel_s3adsp1800.py | 42 +++++++++++++ 5 files changed, 90 insertions(+), 62 deletions(-) delete mode 100644 tests/avocado/machine_microblaze.py create mode 100755 tests/functional/test_microblaze_s3adsp1800.py create mode 100755 tests/functional/test_microblazeel_s3adsp1800.py
MAINTAINERS +1 -1
@@ -1327,7 +1327,7 @@ M: Edgar E. Iglesias <[email protected]> S: Maintained F: hw/microblaze/petalogix_s3adsp1800_mmu.c F: include/hw/char/xilinx_uartlite.h -F: tests/avocado/machine_microblaze.py +F: tests/functional/test_microblaze*.py petalogix_ml605 M: Edgar E. Iglesias <[email protected]>
tests/avocado/machine_microblaze.py deleted +0 -61
61 lines not on this page · raw message
tests/functional/meson.build +8 -0
@@ -34,6 +34,14 @@ tests_loongarch64_thorough = [ 'loongarch64_virt', ] +tests_microblaze_thorough = [ + 'microblaze_s3adsp1800' +] + +tests_microblazeel_thorough = [ + 'microblazeel_s3adsp1800' +] + tests_mips64el_thorough = [ 'mips64el_loongson3v', ]
tests/functional/test_microblaze_s3adsp1800.py new file +39 -0
@@ -0,0 +1,39 @@ +#!/usr/bin/env python3 +# +# Functional test that boots a microblaze Linux kernel and checks the console +# +# Copyright (c) 2018, 2021 Red Hat, Inc. +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + +import time +from qemu_test import exec_command, exec_command_and_wait_for_pattern +from qemu_test import QemuSystemTest, Asset +from qemu_test import wait_for_console_pattern +from qemu_test.utils import archive_extract + +class MicroblazeMachine(QemuSystemTest): + + timeout = 90 + + ASSET_IMAGE = Asset(('https://qemu-advcal.gitlab.io' + '/qac-best-of-multiarch/download/day17.tar.xz'), + '08bf3e3bfb6b6c7ce1e54ab65d54e189f2caf13f') + + def test_microblaze_s3adsp1800(self): + self.set_machine('petalogix-s3adsp1800') + file_path = self.ASSET_IMAGE.fetch() + archive_extract(file_path, self.workdir) + self.vm.set_console() + self.vm.add_args('-kernel', self.workdir + '/day17/ballerina.bin') + self.vm.launch() + wait_for_console_pattern(self, 'This architecture does not have ' + 'kernel memory protection') + # Note: + # The kernel sometimes gets stuck after the "This architecture ..." + # message, that's why we don't test for a later string here. This + # needs some investigation by a microblaze wizard one day... + +if __name__ == '__main__': + QemuSystemTest.main()
tests/functional/test_microblazeel_s3adsp1800.py new file +42 -0
42 lines not on this page · raw message
Signature · 1 line
2.45.2
Re: [PATCH v3 15/24] tests/functional: Convert the microblaze avocado tests into standalone tests
Quoted 15 lines · Daniel P. Berrangé
From: Thomas Huth <[email protected]>
The
machine_microblaze.pyfile contained two tests, one for each endianess. Since we only support one QEMU target binary per file in the new functional test environment, we have to split this file up into two files now.Signed-off-by Thomas Huth <[email protected]>--- MAINTAINERS | 2 +- tests/avocado/machine_microblaze.py | 61 ------------------- tests/functional/meson.build | 8 +++ .../functional/test_microblaze_s3adsp1800.py | 39 ++++++++++++ .../test_microblazeel_s3adsp1800.py | 42 +++++++++++++ 5 files changed, 90 insertions(+), 62 deletions(-)
Quoted 3 lines
delete mode 100644
tests/avocado/machine_microblaze.pycreate mode 100755tests/functional/test_microblaze_s3adsp1800.pycreate mode 100755tests/functional/test_microblazeel_s3adsp1800.py
PATCH v3 16/24 tests/functional: Convert the riscv_opensbi avocado test into a standalone test
In reply to Daniel P. BerrangéFrom: Thomas Huth <[email protected]>
The avocado test defined test functions for both, riscv32 and riscv64. Since we can run the whole file with multiple targets in the new framework, we can now consolidate the functions so we have to only define one function per machine now.
--- tests/avocado/riscv_opensbi.py | 63 -------------------------- tests/functional/meson.build | 8 ++++ tests/functional/test_riscv_opensbi.py | 36 +++++++++++++++ 3 files changed, 44 insertions(+), 63 deletions(-) delete mode 100644 tests/avocado/riscv_opensbi.py create mode 100755 tests/functional/test_riscv_opensbi.py
tests/avocado/riscv_opensbi.py deleted +0 -63
@@ -1,63 +0,0 @@ -# OpenSBI boot test for RISC-V machines -# -# Copyright (c) 2022, Ventana Micro -# -# This work is licensed under the terms of the GNU GPL, version 2 or -# later. See the COPYING file in the top-level directory. - -from avocado_qemu import QemuSystemTest -from avocado_qemu import wait_for_console_pattern - -class RiscvOpenSBI(QemuSystemTest): - """ - :avocado: tags=accel:tcg - """ - timeout = 5 - - def boot_opensbi(self): - self.vm.set_console() - self.vm.launch() - wait_for_console_pattern(self, 'Platform Name') - wait_for_console_pattern(self, 'Boot HART MEDELEG') - - def test_riscv32_spike(self): - """ - :avocado: tags=arch:riscv32 - :avocado: tags=machine:spike - """ - self.boot_opensbi() - - def test_riscv64_spike(self): - """ - :avocado: tags=arch:riscv64 - :avocado: tags=machine:spike - """ - self.boot_opensbi() - - def test_riscv32_sifive_u(self): - """ - :avocado: tags=arch:riscv32 - :avocado: tags=machine:sifive_u - """ - self.boot_opensbi() - - def test_riscv64_sifive_u(self): - """ - :avocado: tags=arch:riscv64 - :avocado: tags=machine:sifive_u - """ - self.boot_opensbi() - - def test_riscv32_virt(self): - """ - :avocado: tags=arch:riscv32 - :avocado: tags=machine:virt - """ - self.boot_opensbi() - - def test_riscv64_virt(self): - """ - :avocado: tags=arch:riscv64 - :avocado: tags=machine:virt - """ - self.boot_opensbi()
tests/functional/meson.build +8 -0
@@ -55,6 +55,14 @@ tests_ppc_thorough = [ 'ppc_bamboo', ] +tests_riscv32_quick = [ + 'riscv_opensbi', +] + +tests_riscv64_quick = [ + 'riscv_opensbi', +] + tests_s390x_thorough = [ 's390x_ccw_virtio', 's390x_topology',
tests/functional/test_riscv_opensbi.py new file +36 -0
@@ -0,0 +1,36 @@ +#!/usr/bin/env python3 +# +# OpenSBI boot test for RISC-V machines +# +# Copyright (c) 2022, Ventana Micro +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + +from qemu_test import QemuSystemTest +from qemu_test import wait_for_console_pattern + +class RiscvOpenSBI(QemuSystemTest): + + timeout = 5 + + def boot_opensbi(self): + self.vm.set_console() + self.vm.launch() + wait_for_console_pattern(self, 'Platform Name') + wait_for_console_pattern(self, 'Boot HART MEDELEG') + + def test_riscv_spike(self): + self.set_machine('spike') + self.boot_opensbi() + + def test_riscv_sifive_u(self): + self.set_machine('sifive_u') + self.boot_opensbi() + + def test_riscv_virt(self): + self.set_machine('virt') + self.boot_opensbi() + +if __name__ == '__main__': + QemuSystemTest.main()
Signature · 1 line
2.45.2
Re: [PATCH v3 16/24] tests/functional: Convert the riscv_opensbi avocado test into a standalone test
Quoted 9 lines · Daniel P. Berrangé
From: Thomas Huth <[email protected]>
The avocado test defined test functions for both, riscv32 and riscv64. Since we can run the whole file with multiple targets in the new framework, we can now consolidate the functions so we have to only define one function per machine now.
Reviewed-by Alistair Francis <[email protected]>Signed-off-by Thomas Huth <[email protected]>
Quoted 7 lines
--- tests/avocado/riscv_opensbi.py | 63 -------------------------- tests/functional/meson.build | 8 ++++ tests/functional/test_riscv_opensbi.py | 36 +++++++++++++++ 3 files changed, 44 insertions(+), 63 deletions(-)delete mode 100644
tests/avocado/riscv_opensbi.pycreate mode 100755tests/functional/test_riscv_opensbi.py
Please squash:
-- >8 --
MAINTAINERS +1 -0
@@ -336,6 +337,7 @@ F: include/hw/riscv/ F: linux-user/host/riscv32/ F: linux-user/host/riscv64/ F: tests/tcg/riscv64/ +F: tests/functional/test_riscv_opensbi.py RISC-V XThead* extensions M: Christoph Muellner <[email protected]>
---
Re: [PATCH v3 16/24] tests/functional: Convert the riscv_opensbi avocado test into a standalone test
Quoted 26 lines · Philippe Mathieu-Daudé
Quoted 9 lines · Daniel P. Berrangé
From: Thomas Huth <[email protected]>
The avocado test defined test functions for both, riscv32 and riscv64. Since we can run the whole file with multiple targets in the new framework, we can now consolidate the functions so we have to only define one function per machine now.
Reviewed-by Alistair Francis <[email protected]>Signed-off-by Thomas Huth <[email protected]>Quoted 7 lines
---
tests/avocado/riscv_opensbi.py | 63 -------------------------- tests/functional/meson.build | 8 ++++tests/functional/test_riscv_opensbi.py| 36 +++++++++++++++3 files changed, 44 insertions(+), 63 deletions(-)delete mode 100644
tests/avocado/riscv_opensbi.pycreate mode 100755tests/functional/test_riscv_opensbi.pyPlease squash:
-- >8 --
MAINTAINERS +0
-0
MAINTAINERS F: include/hw/riscv/ · line 340
Quoted 4 lines
MAINTAINERS +1
-0@@ -336,6 +337,7 @@ F: include/hw/riscv/ F: linux-user/host/riscv32/ F: linux-user/host/riscv64/ F: tests/tcg/riscv64/ +F: tests/functional/test_riscv_opensbi.py
It seems worth taking this opportunity to add per-target subdirs of tests/functional/, so that we don't have to list every individual test in MAINTAINERS.
Surely one
F: tests/functional/riscv/
etc, is actually maintainable...
r~
Re: [PATCH v3 16/24] tests/functional: Convert the riscv_opensbi avocado test into a standalone test
Quoted 36 lines · Richard Henderson
Quoted 26 lines · Philippe Mathieu-Daudé
Quoted 9 lines · Daniel P. Berrangé
From: Thomas Huth <[email protected]>
The avocado test defined test functions for both, riscv32 and riscv64. Since we can run the whole file with multiple targets in the new framework, we can now consolidate the functions so we have to only define one function per machine now.
Reviewed-by Alistair Francis <[email protected]>Signed-off-by Thomas Huth <[email protected]>Quoted 7 lines
---
tests/avocado/riscv_opensbi.py | 63 -------------------------- tests/functional/meson.build | 8 ++++tests/functional/test_riscv_opensbi.py| 36 +++++++++++++++3 files changed, 44 insertions(+), 63 deletions(-)delete mode 100644
tests/avocado/riscv_opensbi.pycreate mode 100755tests/functional/test_riscv_opensbi.pyPlease squash:
-- >8 --
MAINTAINERS +0
-0MAINTAINERS F: include/hw/riscv/ · line 340
Quoted 4 lines
MAINTAINERS +1
-0@@ -336,6 +337,7 @@ F: include/hw/riscv/ F: linux-user/host/riscv32/ F: linux-user/host/riscv64/ F: tests/tcg/riscv64/ +F: tests/functional/test_riscv_opensbi.pyIt seems worth taking this opportunity to add per-target subdirs of tests/functional/, so that we don't have to list every individual test in
MAINTAINERS.
Quoted 5 lines
Surely one
F: tests/functional/riscv/
etc, is actually maintainable...
Either that or wildcard the filenames
F: tests/functional/test_riscv_*
but subdirs matching the target is probably a little nicer to really encourage people to follow conventions more strictly. We would also need a "tests/functional/common/" for tests which aren't target specific
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [PATCH v3 16/24] tests/functional: Convert the riscv_opensbi avocado test into a standalone test
Quoted 43 lines · Daniel P. Berrangé
Quoted 36 lines · Richard Henderson
Quoted 26 lines · Philippe Mathieu-Daudé
Quoted 9 lines · Daniel P. Berrangé
From: Thomas Huth <[email protected]>
The avocado test defined test functions for both, riscv32 and riscv64. Since we can run the whole file with multiple targets in the new framework, we can now consolidate the functions so we have to only define one function per machine now.
Reviewed-by Alistair Francis <[email protected]>Signed-off-by Thomas Huth <[email protected]>Quoted 7 lines
--- tests/avocado/riscv_opensbi.py | 63 -------------------------- tests/functional/meson.build | 8 ++++ tests/functional/test_riscv_opensbi.py | 36 +++++++++++++++ 3 files changed, 44 insertions(+), 63 deletions(-)delete mode 100644
tests/avocado/riscv_opensbi.pycreate mode 100755tests/functional/test_riscv_opensbi.pyPlease squash:
-- >8 --
MAINTAINERS +0
-0MAINTAINERS F: include/hw/riscv/ · line 340
Quoted 4 lines
MAINTAINERS +1
-0@@ -336,6 +337,7 @@ F: include/hw/riscv/ F: linux-user/host/riscv32/ F: linux-user/host/riscv64/ F: tests/tcg/riscv64/ +F: tests/functional/test_riscv_opensbi.pyIt seems worth taking this opportunity to add per-target subdirs of tests/functional/, so that we don't have to list every individual test in
MAINTAINERS.Quoted 5 lines
Surely one
F: tests/functional/riscv/
etc, is actually maintainable...
Quoted 8 lines
Either that or wildcard the filenames
F: tests/functional/test_riscv_*
but subdirs matching the target is probably a little nicer to really encourage people to follow conventions more strictly. We would also need a "tests/functional/common/" for tests which aren't target specific
Hmm I'm a bit reluctant to both suggestions. Various files are added to cover a feature which ends being tested on various archs.
Once I wanted to use Avocado tags like GitLab labels, and keep a file where developers subscribe to tags; doing the parsing and label to developer conversion in Python (or Rust), and having a ./get_maintainer.sh calling both the Perl one and anything we can add in Python/Rust.
That said, wildcard pattern is a simpler start.
PATCH v3 17/24 tests/functional: Convert the virtio_gpu avocado test into a standalone test
In reply to Daniel P. BerrangéFrom: Thomas Huth <[email protected]>
Nothing thrilling in here, it's just a straight forward conversion.
---
tests/functional/meson.build | 1 +
.../test_virtio_gpu.py} | 64 +++++++++----------
2 files changed, 30 insertions(+), 35 deletions(-)
rename tests/{avocado/virtio-gpu.py => functional/test_virtio_gpu.py} (73%)
mode change 100644 => 100755tests/functional/meson.build +1 -0
@@ -82,6 +82,7 @@ tests_x86_64_quick = [ tests_x86_64_thorough = [ 'netdev_ethtool', + 'virtio_gpu', ] precache_all = []
tests/functional/test_virtio_gpu.py renamed from tests/avocado/virtio-gpu.py +29 -35
125 lines not on this page · raw message
Signature · 1 line
2.45.2
Re: [PATCH v3 17/24] tests/functional: Convert the virtio_gpu avocado test into a standalone test
Quoted 11 lines · Daniel P. Berrangé
From: Thomas Huth <[email protected]>
Nothing thrilling in here, it's just a straight forward conversion.
Signed-off-by Thomas Huth <[email protected]>--- tests/functional/meson.build | 1 + .../test_virtio_gpu.py} | 64 +++++++++---------- 2 files changed, 30 insertions(+), 35 deletions(-)rename
tests/{avocado/virtio-gpu.py=>functional/test_virtio_gpu.py} (73%) mode change 100644 => 100755
PATCH v3 18/24 tests/functional: Convert most ppc avocado tests into standalone tests
In reply to Daniel P. BerrangéFrom: Thomas Huth <[email protected]>
Nothing thrilling in here, just straight forward conversions.
---
MAINTAINERS | 8 +--
tests/functional/meson.build | 11 ++++
.../test_ppc64_powernv.py} | 56 ++++++-----------
.../test_ppc64_pseries.py} | 56 ++++++-----------
.../test_ppc_40p.py} | 63 +++++++++----------
.../test_ppc_mpc8544ds.py} | 28 +++++----
.../test_ppc_virtex_ml507.py} | 28 +++++----
7 files changed, 109 insertions(+), 141 deletions(-)
rename tests/{avocado/ppc_powernv.py => functional/test_ppc64_powernv.py} (71%)
mode change 100644 => 100755
rename tests/{avocado/ppc_pseries.py => functional/test_ppc64_pseries.py} (76%)
mode change 100644 => 100755
rename tests/{avocado/ppc_prep_40p.py => functional/test_ppc_40p.py} (54%)
mode change 100644 => 100755
rename tests/{avocado/ppc_mpc8544ds.py => functional/test_ppc_mpc8544ds.py} (55%)
mode change 100644 => 100755
rename tests/{avocado/ppc_virtex_ml507.py => functional/test_ppc_virtex_ml507.py} (60%)
mode change 100644 => 100755MAINTAINERS +4 -4
@@ -1449,7 +1449,7 @@ L: [email protected] S: Orphan F: hw/ppc/mpc8544ds.c F: hw/ppc/mpc8544_guts.c -F: tests/avocado/ppc_mpc8544ds.py +F: tests/functional/test_ppc_mpc8544ds.py New World (mac99) M: Mark Cave-Ayland <[email protected]>
@@ -1502,7 +1502,7 @@ F: hw/dma/i82374.c F: hw/rtc/m48t59-isa.c F: include/hw/isa/pc87312.h F: include/hw/rtc/m48t59.h -F: tests/avocado/ppc_prep_40p.py +F: tests/functional/test_ppc_40p.py sPAPR (pseries) M: Nicholas Piggin <[email protected]>
@@ -1526,7 +1526,7 @@ F: tests/qtest/spapr* F: tests/qtest/libqos/*spapr* F: tests/qtest/rtas* F: tests/qtest/libqos/rtas* -F: tests/avocado/ppc_pseries.py +F: tests/functional/test_ppc64_pseries.py F: tests/avocado/ppc_hv_tests.py PowerNV (Non-Virtualized)
@@ -1558,7 +1558,7 @@ M: Edgar E. Iglesias <[email protected]> L: [email protected] S: Odd Fixes F: hw/ppc/virtex_ml507.c -F: tests/avocado/ppc_virtex_ml507.py +F: tests/functional/test_ppc_virtex_ml507.py sam460ex M: BALATON Zoltan <[email protected]>
tests/functional/meson.build +11 -0
@@ -12,6 +12,9 @@ endif # Timeouts for individual tests that can be slow e.g. with debugging enabled test_timeouts = { 'netdev_ethtool' : 180, + 'ppc_40p' : 240, + 'ppc64_powernv' : 120, + 'ppc64_pseries' : 120, 's390x_ccw_virtio' : 180, }
@@ -52,7 +55,15 @@ tests_ppc_quick = [ tests_ppc_thorough = [ 'ppc_405', + 'ppc_40p', 'ppc_bamboo', + 'ppc_mpc8544ds', + 'ppc_virtex_ml507', +] + +tests_ppc64_thorough = [ + 'ppc64_powernv', + 'ppc64_pseries', ] tests_riscv32_quick = [
tests/functional/test_ppc64_powernv.py renamed from tests/avocado/ppc_powernv.py +18 -38
105 lines not on this page · raw message
tests/functional/test_ppc64_pseries.py renamed from tests/avocado/ppc_pseries.py +18 -38
110 lines not on this page · raw message
tests/functional/test_ppc_40p.py renamed from tests/avocado/ppc_prep_40p.py +28 -35
101 lines not on this page · raw message
tests/functional/test_ppc_mpc8544ds.py renamed from tests/avocado/ppc_mpc8544ds.py +15 -13
@@ -1,3 +1,5 @@ +#!/usr/bin/env python3 +# # Test that Linux kernel boots on ppc machines and check the console # # Copyright (c) 2018, 2020 Red Hat, Inc.
@@ -5,9 +7,9 @@ # This work is licensed under the terms of the GNU GPL, version 2 or # later. See the COPYING file in the top-level directory. -from avocado.utils import archive -from avocado_qemu import QemuSystemTest -from avocado_qemu import wait_for_console_pattern +from qemu_test.utils import archive_extract +from qemu_test import QemuSystemTest, Asset +from qemu_test import wait_for_console_pattern class Mpc8544dsMachine(QemuSystemTest):
@@ -15,20 +17,20 @@ class Mpc8544dsMachine(QemuSystemTest): KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 ' panic_message = 'Kernel panic - not syncing' + ASSET_IMAGE = Asset(('https://qemu-advcal.gitlab.io' + '/qac-best-of-multiarch/download/day04.tar.xz'), + 'f46724d281a9f30fa892d458be7beb7d34dc25f9') + def test_ppc_mpc8544ds(self): - """ - :avocado: tags=arch:ppc - :avocado: tags=machine:mpc8544ds - :avocado: tags=accel:tcg - """ self.require_accelerator("tcg") - tar_url = ('https://qemu-advcal.gitlab.io' - '/qac-best-of-multiarch/download/day04.tar.xz') - tar_hash = 'f46724d281a9f30fa892d458be7beb7d34dc25f9' - file_path = self.fetch_asset(tar_url, asset_hash=tar_hash) - archive.extract(file_path, self.workdir) + self.set_machine('mpc8544ds') + file_path = self.ASSET_IMAGE.fetch() + archive_extract(file_path, self.workdir, member='creek/creek.bin') self.vm.set_console() self.vm.add_args('-kernel', self.workdir + '/creek/creek.bin') self.vm.launch() wait_for_console_pattern(self, 'QEMU advent calendar 2020', self.panic_message) + +if __name__ == '__main__': + QemuSystemTest.main()
tests/functional/test_ppc_virtex_ml507.py renamed from tests/avocado/ppc_virtex_ml507.py +15 -13
@@ -1,3 +1,5 @@ +#!/usr/bin/env python3 +# # Test that Linux kernel boots on ppc machines and check the console # # Copyright (c) 2018, 2020 Red Hat, Inc.
@@ -5,9 +7,9 @@ # This work is licensed under the terms of the GNU GPL, version 2 or # later. See the COPYING file in the top-level directory. -from avocado.utils import archive -from avocado_qemu import QemuSystemTest -from avocado_qemu import wait_for_console_pattern +from qemu_test.utils import archive_extract +from qemu_test import QemuSystemTest, Asset +from qemu_test import wait_for_console_pattern class VirtexMl507Machine(QemuSystemTest):
@@ -15,18 +17,15 @@ class VirtexMl507Machine(QemuSystemTest): KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 ' panic_message = 'Kernel panic - not syncing' + ASSET_IMAGE = Asset(('https://qemu-advcal.gitlab.io' + '/qac-best-of-multiarch/download/day08.tar.xz'), + '74c68f5af7a7b8f21c03097b298f3bb77ff52c1f') + def test_ppc_virtex_ml507(self): - """ - :avocado: tags=arch:ppc - :avocado: tags=machine:virtex-ml507 - :avocado: tags=accel:tcg - """ self.require_accelerator("tcg") - tar_url = ('https://qemu-advcal.gitlab.io' - '/qac-best-of-multiarch/download/day08.tar.xz') - tar_hash = '74c68f5af7a7b8f21c03097b298f3bb77ff52c1f' - file_path = self.fetch_asset(tar_url, asset_hash=tar_hash) - archive.extract(file_path, self.workdir) + self.set_machine('virtex-ml507') + file_path = self.ASSET_IMAGE.fetch() + archive_extract(file_path, self.workdir) self.vm.set_console() self.vm.add_args('-kernel', self.workdir + '/hippo/hippo.linux', '-dtb', self.workdir + '/hippo/virtex440-ml507.dtb',
@@ -34,3 +33,6 @@ def test_ppc_virtex_ml507(self): self.vm.launch() wait_for_console_pattern(self, 'QEMU advent calendar 2020', self.panic_message) + +if __name__ == '__main__': + QemuSystemTest.main()
Signature · 1 line
2.45.2
PATCH v3 19/24 tests/functional: Convert the ppc_amiga avocado test into a standalone test
In reply to Daniel P. BerrangéFrom: Thomas Huth <[email protected]>
Use the Python standard zipfile module instead of avocado.utils for extracting the ZIP file that we download here, and use the standard subprocess module for running the "tail" command.
--- tests/avocado/ppc_amiga.py | 38 --------------------------- tests/functional/meson.build | 1 + tests/functional/test_ppc_amiga.py | 42 ++++++++++++++++++++++++++++++ 3 files changed, 43 insertions(+), 38 deletions(-) delete mode 100644 tests/avocado/ppc_amiga.py create mode 100755 tests/functional/test_ppc_amiga.py
tests/avocado/ppc_amiga.py deleted +0 -38
@@ -1,38 +0,0 @@ -# Test AmigaNG boards -# -# Copyright (c) 2023 BALATON Zoltan -# -# This work is licensed under the terms of the GNU GPL, version 2 or -# later. See the COPYING file in the top-level directory. - -from avocado.utils import archive -from avocado.utils import process -from avocado_qemu import QemuSystemTest -from avocado_qemu import wait_for_console_pattern - -class AmigaOneMachine(QemuSystemTest): - - timeout = 90 - - def test_ppc_amigaone(self): - """ - :avocado: tags=arch:ppc - :avocado: tags=machine:amigaone - :avocado: tags=device:articia - :avocado: tags=accel:tcg - """ - self.require_accelerator("tcg") - tar_name = 'A1Firmware_Floppy_05-Mar-2005.zip' - tar_url = ('https://www.hyperion-entertainment.com/index.php/' - 'downloads?view=download&format=raw&file=25') - tar_hash = 'c52e59bc73e31d8bcc3cc2106778f7ac84f6c755' - zip_file = self.fetch_asset(tar_name, locations=tar_url, - asset_hash=tar_hash) - archive.extract(zip_file, self.workdir) - cmd = f"tail -c 524288 {self.workdir}/floppy_edition/updater.image >{self.workdir}/u-boot-amigaone.bin" - process.run(cmd, shell=True) - - self.vm.set_console() - self.vm.add_args('-bios', self.workdir + '/u-boot-amigaone.bin') - self.vm.launch() - wait_for_console_pattern(self, 'FLASH:')
tests/functional/meson.build +1 -0
@@ -56,6 +56,7 @@ tests_ppc_quick = [ tests_ppc_thorough = [ 'ppc_405', 'ppc_40p', + 'ppc_amiga', 'ppc_bamboo', 'ppc_mpc8544ds', 'ppc_virtex_ml507',
tests/functional/test_ppc_amiga.py new file +42 -0
42 lines not on this page · raw message
Signature · 1 line
2.45.2
Re: [PATCH v3 19/24] tests/functional: Convert the ppc_amiga avocado test into a standalone test
Quoted 6 lines · Daniel P. Berrangé
From: Thomas Huth <[email protected]>
Use the Python standard zipfile module instead of avocado.utils for extracting the ZIP file that we download here, and use the standard subprocess module for running the "tail" command.
Quoted 8 lines
Signed-off-by Thomas Huth <[email protected]>--- tests/avocado/ppc_amiga.py | 38 --------------------------- tests/functional/meson.build | 1 + tests/functional/test_ppc_amiga.py | 42 ++++++++++++++++++++++++++++++ 3 files changed, 43 insertions(+), 38 deletions(-)delete mode 100644
tests/avocado/ppc_amiga.pycreate mode 100755tests/functional/test_ppc_amiga.py
PATCH v3 20/24 tests/functional: Convert the ppc_hv avocado test into a standalone test
In reply to Daniel P. BerrangéFrom: Thomas Huth <[email protected]>
Note: The original Avocado test seems currently to be broken, it hangs when the guest is trying to install additional packages. So mark it as broken for now until it gets fixed.
---
.../test_ppc64_hv.py} | 48 ++++++++-----------
1 file changed, 20 insertions(+), 28 deletions(-)
rename tests/{avocado/ppc_hv_tests.py => functional/test_ppc64_hv.py} (88%)
mode change 100644 => 100755tests/functional/test_ppc64_hv.py renamed from tests/avocado/ppc_hv_tests.py +20 -28
@@ -1,3 +1,5 @@ +#!/usr/bin/env python3 +# # Tests that specifically try to exercise hypervisor features of the # target machines. powernv supports the Power hypervisor ISA, and # pseries supports the nested-HV hypervisor spec.
@@ -7,10 +9,10 @@ # This work is licensed under the terms of the GNU GPL, version 2 or # later. See the COPYING file in the top-level directory. -from avocado import skipIf, skipUnless -from avocado.utils import archive -from avocado_qemu import QemuSystemTest -from avocado_qemu import wait_for_console_pattern, exec_command +from unittest import skipIf, skipUnless +from qemu_test.utils import archive_extract +from qemu_test import QemuSystemTest, Asset +from qemu_test import wait_for_console_pattern, exec_command import os import time import subprocess
@@ -47,7 +49,7 @@ def missing_deps(): @skipIf(missing_deps(), 'dependencies (%s) not installed' % ','.join(deps)) @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test sometimes gets stuck due to console handling problem') @skipUnless(os.getenv('AVOCADO_ALLOW_LARGE_STORAGE'), 'storage limited') -@skipUnless(os.getenv('SPEED') == 'slow', 'runtime limited') +@skip('broken test - guest fails to install packages') class HypervisorTest(QemuSystemTest): timeout = 1000
@@ -55,6 +57,12 @@ class HypervisorTest(QemuSystemTest): panic_message = 'Kernel panic - not syncing' good_message = 'VFS: Cannot open root device' + # Alpine use sha256 so I recalculated this myself + ASSET_ISO = Asset( + ('https://dl-cdn.alpinelinux.org/alpine/v3.18/' + 'releases/ppc64le/alpine-standard-3.18.4-ppc64le.iso'), + 'c26b8d3e17c2f3f0fed02b4b1296589c2390e6d5548610099af75300edd7b3ff') + def extract_from_iso(self, iso, path): """ Extracts a file from an iso file into the test workdir
@@ -84,14 +92,7 @@ def extract_from_iso(self, iso, path): def setUp(self): super().setUp() - iso_url = ('https://dl-cdn.alpinelinux.org/alpine/v3.18/releases/ppc64le/alpine-standard-3.18.4-ppc64le.iso') - - # Alpine use sha256 so I recalculated this myself - iso_sha256 = 'c26b8d3e17c2f3f0fed02b4b1296589c2390e6d5548610099af75300edd7b3ff' - iso_path = self.fetch_asset(iso_url, asset_hash=iso_sha256, - algorithm = "sha256") - - self.iso_path = iso_path + self.iso_path = self.ASSET_ISO.fetch() self.vmlinuz = self.extract_from_iso(iso_path, '/boot/vmlinuz-lts') self.initramfs = self.extract_from_iso(iso_path, '/boot/initramfs-lts')
@@ -159,12 +160,8 @@ def do_test_kvm(self, hpt=False): wait_for_console_pattern(self, 'alpine:~#') def test_hv_pseries(self): - """ - :avocado: tags=arch:ppc64 - :avocado: tags=machine:pseries - :avocado: tags=accel:tcg - """ self.require_accelerator("tcg") + self.set_machine('pseries') self.vm.add_args("-accel", "tcg,thread=multi") self.vm.add_args('-device', 'nvme,serial=1234,drive=drive0') self.vm.add_args("-machine", "x-vof=on,cap-nested-hv=on")
@@ -174,12 +171,8 @@ def test_hv_pseries(self): self.do_stop_alpine() def test_hv_pseries_kvm(self): - """ - :avocado: tags=arch:ppc64 - :avocado: tags=machine:pseries - :avocado: tags=accel:kvm - """ self.require_accelerator("kvm") + self.set_machine('pseries') self.vm.add_args("-accel", "kvm") self.vm.add_args('-device', 'nvme,serial=1234,drive=drive0') self.vm.add_args("-machine", "x-vof=on,cap-nested-hv=on,cap-ccf-assist=off")
@@ -189,12 +182,8 @@ def test_hv_pseries_kvm(self): self.do_stop_alpine() def test_hv_powernv(self): - """ - :avocado: tags=arch:ppc64 - :avocado: tags=machine:powernv - :avocado: tags=accel:tcg - """ self.require_accelerator("tcg") + self.set_machine('powernv') self.vm.add_args("-accel", "tcg,thread=multi") self.vm.add_args('-device', 'nvme,bus=pcie.2,addr=0x0,serial=1234,drive=drive0', '-device', 'e1000e,netdev=net0,mac=C0:FF:EE:00:00:02,bus=pcie.0,addr=0x0',
@@ -204,3 +193,6 @@ def test_hv_powernv(self): self.do_test_kvm() self.do_test_kvm(True) self.do_stop_alpine() + +if __name__ == '__main__': + QemuSystemTest.main()
Signature · 1 line
2.45.2
Nicholas Piggin
reviewerRe: [PATCH v3 20/24] tests/functional: Convert the ppc_hv avocado test into a standalone test
Thanks for doing this.
I have a fix for this problem, it's a bug with the test harness code but was not merged yet. I'll re-send it.
Thanks,
Nick
Quoted 130 lines · Daniel P. Berrangé
From: Thomas Huth <[email protected]>
Note: The original Avocado test seems currently to be broken, it hangs when the guest is trying to install additional packages. So mark it as broken for now until it gets fixed.
Signed-off-by Thomas Huth <[email protected]>--- .../test_ppc64_hv.py} | 48 ++++++++----------- 1 file changed, 20 insertions(+), 28 deletions(-) rename tests/{avocado/ppc_hv_tests.py => functional/test_ppc64_hv.py} (88%) mode change 100644 => 100755tests/functional/test_ppc64_hv.py renamed from tests/avocado/ppc_hv_tests.py +20
-28@@ -1,3 +1,5 @@ +#!/usr/bin/env python3 +# # Tests that specifically try to exercise hypervisor features of the # target machines. powernv supports the Power hypervisor ISA, and # pseries supports the nested-HV hypervisor spec.@@ -7,10 +9,10 @@ # This work is licensed under the terms of the GNU GPL, version 2 or # later. See the COPYING file in the top-level directory. -from avocado import skipIf, skipUnless -from avocado.utils import archive -from avocado_qemu import QemuSystemTest -from avocado_qemu import wait_for_console_pattern, exec_command +from unittest import skipIf, skipUnless +from qemu_test.utils import archive_extract +from qemu_test import QemuSystemTest, Asset +from qemu_test import wait_for_console_pattern, exec_command import os import time import subprocess@@ -47,7 +49,7 @@ def missing_deps(): @skipIf(missing_deps(), 'dependencies (%s) not installed' % ','.join(deps)) @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test sometimes gets stuck due to console handling problem') @skipUnless(os.getenv('AVOCADO_ALLOW_LARGE_STORAGE'), 'storage limited') -@skipUnless(os.getenv('SPEED') == 'slow', 'runtime limited') +@skip('broken test - guest fails to install packages') class HypervisorTest(QemuSystemTest): timeout = 1000@@ -55,6 +57,12 @@ class HypervisorTest(QemuSystemTest): panic_message = 'Kernel panic - not syncing' good_message = 'VFS: Cannot open root device' + # Alpine use sha256 so I recalculated this myself + ASSET_ISO = Asset( + ('https://dl-cdn.alpinelinux.org/alpine/v3.18/' + 'releases/ppc64le/alpine-standard-3.18.4-ppc64le.iso'), + 'c26b8d3e17c2f3f0fed02b4b1296589c2390e6d5548610099af75300edd7b3ff') + def extract_from_iso(self, iso, path): """ Extracts a file from an iso file into the test workdir@@ -84,14 +92,7 @@ def extract_from_iso(self, iso, path): def setUp(self): super().setUp() - iso_url = ('https://dl-cdn.alpinelinux.org/alpine/v3.18/releases/ppc64le/alpine-standard-3.18.4-ppc64le.iso') - - # Alpine use sha256 so I recalculated this myself - iso_sha256 = 'c26b8d3e17c2f3f0fed02b4b1296589c2390e6d5548610099af75300edd7b3ff' - iso_path = self.fetch_asset(iso_url, asset_hash=iso_sha256, - algorithm = "sha256") - - self.iso_path = iso_path + self.iso_path = self.ASSET_ISO.fetch() self.vmlinuz = self.extract_from_iso(iso_path, '/boot/vmlinuz-lts') self.initramfs = self.extract_from_iso(iso_path, '/boot/initramfs-lts')@@ -159,12 +160,8 @@ def do_test_kvm(self, hpt=False): wait_for_console_pattern(self, 'alpine:~#') def test_hv_pseries(self): - """ - :avocado: tags=arch:ppc64 - :avocado: tags=machine:pseries - :avocado: tags=accel:tcg - """ self.require_accelerator("tcg") + self.set_machine('pseries') self.vm.add_args("-accel", "tcg,thread=multi") self.vm.add_args('-device', 'nvme,serial=1234,drive=drive0') self.vm.add_args("-machine", "x-vof=on,cap-nested-hv=on")@@ -174,12 +171,8 @@ def test_hv_pseries(self): self.do_stop_alpine() def test_hv_pseries_kvm(self): - """ - :avocado: tags=arch:ppc64 - :avocado: tags=machine:pseries - :avocado: tags=accel:kvm - """ self.require_accelerator("kvm") + self.set_machine('pseries') self.vm.add_args("-accel", "kvm") self.vm.add_args('-device', 'nvme,serial=1234,drive=drive0') self.vm.add_args("-machine", "x-vof=on,cap-nested-hv=on,cap-ccf-assist=off")@@ -189,12 +182,8 @@ def test_hv_pseries_kvm(self): self.do_stop_alpine() def test_hv_powernv(self): - """ - :avocado: tags=arch:ppc64 - :avocado: tags=machine:powernv - :avocado: tags=accel:tcg - """ self.require_accelerator("tcg") + self.set_machine('powernv') self.vm.add_args("-accel", "tcg,thread=multi") self.vm.add_args('-device', 'nvme,bus=pcie.2,addr=0x0,serial=1234,drive=drive0', '-device', 'e1000e,netdev=net0,mac=C0:FF:EE:00:00:02,bus=pcie.0,addr=0x0',@@ -204,3 +193,6 @@ def test_hv_powernv(self): self.do_test_kvm() self.do_test_kvm(True) self.do_stop_alpine() + +if __name__ == '__main__': + QemuSystemTest.main()
PATCH v3 21/24 tests/functional: Convert the m68k nextcube test with tesseract
In reply to Daniel P. BerrangéFrom: Thomas Huth <[email protected]>
The code that handles running of tesseract needs to be tweaked a little bit to be able to run without the functions from avocado.utils, and while we're at it, drop some legacy stuff that was still there due to Tesseract 3 support that we already dropped a while ago.
---
tests/avocado/tesseract_utils.py | 46 -------------------
tests/functional/meson.build | 4 ++
tests/functional/qemu_test/tesseract.py | 35 ++++++++++++++
.../test_m68k_nextcube.py} | 29 ++++++------
4 files changed, 55 insertions(+), 59 deletions(-)
delete mode 100644 tests/avocado/tesseract_utils.py
create mode 100644 tests/functional/qemu_test/tesseract.py
rename tests/{avocado/machine_m68k_nextcube.py => functional/test_m68k_nextcube.py} (76%)
mode change 100644 => 100755tests/avocado/tesseract_utils.py deleted +0 -46
@@ -1,46 +0,0 @@ -# ... -# -# Copyright (c) 2019 Philippe Mathieu-Daudé <[email protected]> -# -# This work is licensed under the terms of the GNU GPL, version 2 or -# later. See the COPYING file in the top-level directory. - -import re -import logging - -from avocado.utils import process -from avocado.utils.path import find_command, CmdNotFoundError - -def tesseract_available(expected_version): - try: - find_command('tesseract') - except CmdNotFoundError: - return False - res = process.run('tesseract --version') - try: - version = res.stdout_text.split()[1] - except IndexError: - version = res.stderr_text.split()[1] - return int(version.split('.')[0]) >= expected_version - - match = re.match(r'tesseract\s(\d)', res) - if match is None: - return False - # now this is guaranteed to be a digit - return int(match.groups()[0]) >= expected_version - - -def tesseract_ocr(image_path, tesseract_args='', tesseract_version=3): - console_logger = logging.getLogger('tesseract') - console_logger.debug(image_path) - if tesseract_version == 4: - tesseract_args += ' --oem 1' - proc = process.run("tesseract {} {} stdout".format(tesseract_args, - image_path)) - lines = [] - for line in proc.stdout_text.split('\n'): - sline = line.strip() - if len(sline): - console_logger.debug(sline) - lines += [sline] - return lines
tests/functional/meson.build +4 -0
@@ -37,6 +37,10 @@ tests_loongarch64_thorough = [ 'loongarch64_virt', ] +tests_m68k_thorough = [ + 'm68k_nextcube' +] + tests_microblaze_thorough = [ 'microblaze_s3adsp1800' ]
tests/functional/qemu_test/tesseract.py new file +35 -0
@@ -0,0 +1,35 @@ +# ... +# +# Copyright (c) 2019 Philippe Mathieu-Daudé <[email protected]> +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + +import re +import logging + +from . import has_cmd, run_cmd + +def tesseract_available(expected_version): + if not has_cmd('tesseract'): + return False + (stdout, stderr, ret) = run_cmd([ 'tesseract', '--version']) + if ret: + return False + version = stdout.split()[1] + return int(version.split('.')[0]) >= expected_version + +def tesseract_ocr(image_path, tesseract_args=''): + console_logger = logging.getLogger('console') + console_logger.debug(image_path) + (stdout, stderr, ret) = run_cmd(['tesseract', image_path, + 'stdout']) + if ret: + return None + lines = [] + for line in stdout.split('\n'): + sline = line.strip() + if len(sline): + console_logger.debug(sline) + lines += [sline] + return lines
tests/functional/test_m68k_nextcube.py renamed from tests/avocado/machine_m68k_nextcube.py +16 -13
@@ -1,3 +1,5 @@ +#!/usr/bin/env python3 +# # Functional test that boots a VM and run OCR on the framebuffer # # Copyright (c) 2019 Philippe Mathieu-Daudé <[email protected]>
@@ -8,10 +10,10 @@ import os import time -from avocado_qemu import QemuSystemTest -from avocado import skipUnless +from qemu_test import QemuSystemTest, Asset +from unittest import skipUnless -from tesseract_utils import tesseract_available, tesseract_ocr +from qemu_test.tesseract import tesseract_available, tesseract_ocr PIL_AVAILABLE = True try:
@@ -21,19 +23,15 @@ class NextCubeMachine(QemuSystemTest): - """ - :avocado: tags=arch:m68k - :avocado: tags=machine:next-cube - :avocado: tags=device:framebuffer - """ timeout = 15 + ASSET_ROM = Asset(('https://sourceforge.net/p/previous/code/1350/tree/' + 'trunk/src/Rev_2.5_v66.BIN?format=raw'), + 'b3534796abae238a0111299fc406a9349f7fee24') + def check_bootrom_framebuffer(self, screenshot_path): - rom_url = ('https://sourceforge.net/p/previous/code/1350/tree/' - 'trunk/src/Rev_2.5_v66.BIN?format=raw') - rom_hash = 'b3534796abae238a0111299fc406a9349f7fee24' - rom_path = self.fetch_asset(rom_url, asset_hash=rom_hash) + rom_path = self.ASSET_ROM.fetch() self.vm.add_args('-bios', rom_path) self.vm.launch()
@@ -48,6 +46,7 @@ def check_bootrom_framebuffer(self, screenshot_path): @skipUnless(PIL_AVAILABLE, 'Python PIL not installed') def test_bootrom_framebuffer_size(self): + self.set_machine('next-cube') screenshot_path = os.path.join(self.workdir, "dump.ppm") self.check_bootrom_framebuffer(screenshot_path)
@@ -60,11 +59,15 @@ def test_bootrom_framebuffer_size(self): # that it is still alpha-level software. @skipUnless(tesseract_available(4), 'tesseract OCR tool not available') def test_bootrom_framebuffer_ocr_with_tesseract(self): + self.set_machine('next-cube') screenshot_path = os.path.join(self.workdir, "dump.ppm") self.check_bootrom_framebuffer(screenshot_path) - lines = tesseract_ocr(screenshot_path, tesseract_version=4) + lines = tesseract_ocr(screenshot_path) text = '\n'.join(lines) self.assertIn('Testing the FPU', text) self.assertIn('System test failed. Error code', text) self.assertIn('Boot command', text) self.assertIn('Next>', text) + +if __name__ == '__main__': + QemuSystemTest.main()
Signature · 1 line
2.45.2
Re: [PATCH v3 21/24] tests/functional: Convert the m68k nextcube test with tesseract
Quoted 14 lines · Daniel P. Berrangé
From: Thomas Huth <[email protected]>
The code that handles running of tesseract needs to be tweaked a little bit to be able to run without the functions from avocado.utils, and while we're at it, drop some legacy stuff that was still there due to Tesseract 3 support that we already dropped a while ago.
Signed-off-by Thomas Huth <[email protected]>--- tests/avocado/tesseract_utils.py | 46 ------------------- tests/functional/meson.build | 4 ++ tests/functional/qemu_test/tesseract.py | 35 ++++++++++++++ .../test_m68k_nextcube.py} | 29 ++++++------ 4 files changed, 55 insertions(+), 59 deletions(-)
Quoted 4 lines
delete mode 100644
tests/avocado/tesseract_utils.py
create mode 100644tests/functional/qemu_test/tesseract.py
renametests/{avocado/machine_m68k_nextcube.py=>functional/test_m68k_nextcube.py} (76%) mode change 100644 => 100755
PATCH v3 22/24 tests/functional: Convert the acpi-bits test into a standalone test
In reply to Daniel P. BerrangéFrom: Thomas Huth <[email protected]>
Mostly a straight-forward conversion. Looks like we can simply drop the avocado datadrainer stuff when not using the avocado framework anymore.
---
.../acpi-bits/bits-config/bits-cfg.txt | 0
.../acpi-bits/bits-tests/smbios.py2 | 0
.../acpi-bits/bits-tests/smilatency.py2 | 0
.../acpi-bits/bits-tests/testacpi.py2 | 0
.../acpi-bits/bits-tests/testcpuid.py2 | 0
tests/functional/meson.build | 2 +
.../test_acpi_bits.py} | 81 ++++++++++---------
7 files changed, 43 insertions(+), 40 deletions(-)
rename tests/{avocado => functional}/acpi-bits/bits-config/bits-cfg.txt (100%)
rename tests/{avocado => functional}/acpi-bits/bits-tests/smbios.py2 (100%)
rename tests/{avocado => functional}/acpi-bits/bits-tests/smilatency.py2 (100%)
rename tests/{avocado => functional}/acpi-bits/bits-tests/testacpi.py2 (100%)
rename tests/{avocado => functional}/acpi-bits/bits-tests/testcpuid.py2 (100%)
rename tests/{avocado/acpi-bits.py => functional/test_acpi_bits.py} (86%)
mode change 100644 => 100755tests/functional/acpi-bits/bits-config/bits-cfg.txt renamed from tests/avocado/acpi-bits/bits-config/bits-cfg.txt +0 -0
tests/functional/acpi-bits/bits-tests/smbios.py2 renamed from tests/avocado/acpi-bits/bits-tests/smbios.py2 +0 -0
tests/functional/acpi-bits/bits-tests/smilatency.py2 renamed from tests/avocado/acpi-bits/bits-tests/smilatency.py2 +0 -0
tests/functional/acpi-bits/bits-tests/testacpi.py2 renamed from tests/avocado/acpi-bits/bits-tests/testacpi.py2 +0 -0
tests/functional/acpi-bits/bits-tests/testcpuid.py2 renamed from tests/avocado/acpi-bits/bits-tests/testcpuid.py2 +0 -0
tests/functional/meson.build +2 -0
@@ -11,6 +11,7 @@ endif # Timeouts for individual tests that can be slow e.g. with debugging enabled test_timeouts = { + 'acpi_bits' : 240, 'netdev_ethtool' : 180, 'ppc_40p' : 240, 'ppc64_powernv' : 120,
@@ -97,6 +98,7 @@ tests_x86_64_quick = [ ] tests_x86_64_thorough = [ + 'acpi_bits', 'netdev_ethtool', 'virtio_gpu', ]
tests/functional/test_acpi_bits.py renamed from tests/avocado/acpi-bits.py +41 -40
172 lines not on this page · raw message
Signature · 1 line
2.45.2
Re: [PATCH v3 22/24] tests/functional: Convert the acpi-bits test into a standalone test
Quoted 16 lines · Daniel P. Berrangé
From: Thomas Huth <[email protected]>
Mostly a straight-forward conversion. Looks like we can simply drop the avocado datadrainer stuff when not using the avocado framework anymore.
Signed-off-by Thomas Huth <[email protected]>--- .../acpi-bits/bits-config/bits-cfg.txt | 0 .../acpi-bits/bits-tests/smbios.py2 | 0 .../acpi-bits/bits-tests/smilatency.py2 | 0 .../acpi-bits/bits-tests/testacpi.py2 | 0 .../acpi-bits/bits-tests/testcpuid.py2 | 0 tests/functional/meson.build | 2 + .../test_acpi_bits.py} | 81 ++++++++++--------- 7 files changed, 43 insertions(+), 40 deletions(-)
Quoted 7 lines
rename tests/{avocado =>
functional}/acpi-bits/bits-config/bits-cfg.txt(100%) rename tests/{avocado => functional}/acpi-bits/bits-tests/smbios.py2 (100%) rename tests/{avocado => functional}/acpi-bits/bits-tests/smilatency.py2 (100%) rename tests/{avocado => functional}/acpi-bits/bits-tests/testacpi.py2 (100%) rename tests/{avocado => functional}/acpi-bits/bits-tests/testcpuid.py2 (100%) renametests/{avocado/acpi-bits.py=>functional/test_acpi_bits.py} (86%) mode change 100644 => 100755
Re: [PATCH v3 22/24] tests/functional: Convert the acpi-bits test into a standalone test
Quoted 25 lines · Philippe Mathieu-Daudé
Quoted 15 lines · Daniel P. Berrangé
From: Thomas Huth <[email protected]>
Mostly a straight-forward conversion. Looks like we can simply drop the avocado datadrainer stuff when not using the avocado framework anymore.
Signed-off-by Thomas Huth <[email protected]>---
.../acpi-bits/bits-config/bits-cfg.txt | 0 .../acpi-bits/bits-tests/smbios.py2 | 0 .../acpi-bits/bits-tests/smilatency.py2 | 0 .../acpi-bits/bits-tests/testacpi.py2 | 0 .../acpi-bits/bits-tests/testcpuid.py2 | 0 tests/functional/meson.build | 2 + .../test_acpi_bits.py} | 81 ++++++++++---------Quoted 8 lines
7 files changed, 43 insertions(+), 40 deletions(-)rename tests/{avocado =>
functional}/acpi-bits/bits-config/bits-cfg.txt(100%) rename tests/{avocado => functional}/acpi-bits/bits-tests/smbios.py2 (100%) rename tests/{avocado => functional}/acpi-bits/bits-tests/smilatency.py2 (100%) rename tests/{avocado => functional}/acpi-bits/bits-tests/testacpi.py2 (100%) rename tests/{avocado => functional}/acpi-bits/bits-tests/testcpuid.py2 (100%) renametests/{avocado/acpi-bits.py=>functional/test_acpi_bits.py} (86%) mode change 100644 => 100755
Quoted 2 lines
Reviewed-by Philippe Mathieu-Daudé <[email protected]>
(missing MAINTAINERS update)
PATCH v3 23/24 tests/functional: Convert the rx_gdbsim avocado test into a standalone test
In reply to Daniel P. BerrangéFrom: Thomas Huth <[email protected]>
Provide a "gzip_uncompress" function based on the standard "gzip" module to avoid the usage of avocado.utils here.
---
tests/functional/meson.build | 4 ++
tests/functional/qemu_test/utils.py | 12 ++++
.../test_rx_gdbsim.py} | 57 ++++++++++---------
3 files changed, 45 insertions(+), 28 deletions(-)
rename tests/{avocado/machine_rx_gdbsim.py => functional/test_rx_gdbsim.py} (54%)
mode change 100644 => 100755tests/functional/meson.build +4 -0
@@ -80,6 +80,10 @@ tests_riscv64_quick = [ 'riscv_opensbi', ] +tests_rx_thorough = [ + 'rx_gdbsim', +] + tests_s390x_thorough = [ 's390x_ccw_virtio', 's390x_topology',
tests/functional/qemu_test/utils.py +12 -0
@@ -8,6 +8,7 @@ # This work is licensed under the terms of the GNU GPL, version 2 or # later. See the COPYING file in the top-level directory. +import gzip import lzma import os import shutil
@@ -23,6 +24,17 @@ def archive_extract(archive, dest_dir, member=None): else: tf.extractall(path=dest_dir) +def gzip_uncompress(gz_path, output_path): + if os.path.exists(output_path): + return + with gzip.open(gz_path, 'rb') as gz_in: + try: + with open(output_path, 'wb') as raw_out: + shutil.copyfileobj(gz_in, raw_out) + except: + os.remove(output_path) + raise + def lzma_uncompress(xz_path, output_path): if os.path.exists(output_path): return
tests/functional/test_rx_gdbsim.py renamed from tests/avocado/machine_rx_gdbsim.py +29 -28
90 lines not on this page · raw message
Signature · 1 line
2.45.2
Re: [PATCH v3 23/24] tests/functional: Convert the rx_gdbsim avocado test into a standalone test
Quoted 5 lines · Daniel P. Berrangé
From: Thomas Huth <[email protected]>
Provide a "
gzip_uncompress" function based on the standard "gzip" module to avoid the usage of avocado.utils here.
Quoted 8 lines
Signed-off-by Thomas Huth <[email protected]>--- tests/functional/meson.build | 4 ++ tests/functional/qemu_test/utils.py | 12 ++++ .../test_rx_gdbsim.py} | 57 ++++++++++--------- 3 files changed, 45 insertions(+), 28 deletions(-)rename
tests/{avocado/machine_rx_gdbsim.py=>functional/test_rx_gdbsim.py} (54%) mode change 100644 => 100755
PATCH v3 24/24 gitlab-ci: Add "check-functional" to the build tests
In reply to Daniel P. BerrangéFrom: Thomas Huth <[email protected]>
Now that we converted many tests from the "check-avocado" test suite to the "check-functional" test suite, we should make sure that these also get tested in the CI.
--- .gitlab-ci.d/buildtest-template.yml | 3 +- .gitlab-ci.d/buildtest.yml | 60 ++++++++++++++--------------- 2 files changed, 32 insertions(+), 31 deletions(-)
.gitlab-ci.d/buildtest-template.yml +2 -1
@@ -71,7 +71,7 @@ reports: junit: build/meson-logs/testlog.junit.xml -.avocado_test_job_template: +.functional_test_job_template: extends: .common_test_job_template cache: key: "${CI_JOB_NAME}-cache"
@@ -98,6 +98,7 @@ du -chs ${CI_PROJECT_DIR}/avocado-cache ; fi - export AVOCADO_ALLOW_UNTRUSTED_CODE=1 + - export QEMU_TEST_ALLOW_UNTRUSTED_CODE=1 after_script: - cd build - du -chs ${CI_PROJECT_DIR}/avocado-cache
.gitlab-ci.d/buildtest.yml +30 -30
171 lines not on this page · raw message
Signature · 1 line
2.45.2
Re: [PATCH v3 24/24] gitlab-ci: Add "check-functional" to the build tests
Quoted 6 lines · Daniel P. Berrangé
From: Thomas Huth <[email protected]>
Now that we converted many tests from the "check-avocado" test suite to the "check-functional" test suite, we should make sure that these also get tested in the CI.
Quoted 6 lines
Reviewed-by Daniel P. Berrangé <[email protected]>Signed-off-by Thomas Huth <[email protected]>--- .gitlab-ci.d/buildtest-template.yml | 3 +- .gitlab-ci.d/buildtest.yml | 60 ++++++++++++++--------------- 2 files changed, 32 insertions(+), 31 deletions(-)
Quoted 10 lines · Daniel P. Berrangé
This series is an update to Thomas' v2:
https://lists.nongnu.org/archive/html/qemu-devel/2024-07/msg05805.htmlwherein Thomas suggested someone could do the asset caching updates while he is on vacation:
https://lists.nongnu.org/archive/html/qemu-devel/2024-07/msg06228.htmlhence this posting what I'm calling a v3.
Changes included in my PR (can be considered as v2.5):
https://lore.kernel.org/qemu-devel/[email protected]/
-- >8 --
MAINTAINERS +11 -0
@@ -317,6 +317,7 @@ F: configs/devices/ppc* F: docs/system/ppc/embedded.rst F: docs/system/target-ppc.rst F: tests/tcg/ppc*/* +F: tests/functional/test_ppc_74xx.py RISC-V TCG CPUs M: Palmer Dabbelt <[email protected]>
@@ -336,6 +337,7 @@ F: include/hw/riscv/ F: linux-user/host/riscv32/ F: linux-user/host/riscv64/ F: tests/tcg/riscv64/ +F: tests/functional/test_riscv_opensbi.py RISC-V XThead* extensions M: Christoph Muellner <[email protected]>
@@ -1830,6 +1832,9 @@ F: hw/isa/apm.c F: include/hw/isa/apm.h F: tests/unit/test-x86-topo.c F: tests/qtest/test-x86-cpuid-compat.c +F: tests/functional/test_mem_addr_space.py +F: tests/functional/test_pc_cpu_hotplug_props.py +F: tests/functional/test_x86_cpu_model_versions.py PC Chipset M: Michael S. Tsirkin <[email protected]>
@@ -1896,6 +1901,8 @@ F: include/hw/boards.h F: include/hw/core/cpu.h F: include/hw/cpu/cluster.h F: include/sysemu/numa.h +F: tests/functional/test_cpu_queries.py +F: tests/functional/test_empty_cpu_model.py F: tests/unit/test-smp-parse.c T: git https://gitlab.com/ehabkost/qemu.git machine-next
@@ -2101,6 +2108,7 @@ S: Odd Fixes F: hw/net/ F: include/hw/net/ F: tests/qtest/virtio-net-test.c +F: tests/avocado/info_usernet.py F: docs/virtio-net-failover.rst T: git https://github.com/jasowang/qemu.git net
@@ -2236,6 +2244,7 @@ F: net/vhost-user.c F: include/hw/virtio/ F: docs/devel/virtio* F: docs/devel/migration/virtio.rst +F: tests/functional/test_virtio_version.py virtio-balloon M: Michael S. Tsirkin <[email protected]>
@@ -2969,6 +2978,7 @@ S: Supported F: include/qemu/option.h F: tests/unit/test-keyval.c F: tests/unit/test-qemu-opts.c +F: tests/functional/test_version.py F: util/keyval.c F: util/qemu-option.c
@@ -4127,6 +4137,7 @@ F: .travis.yml F: docs/devel/ci* F: scripts/ci/ F: tests/docker/ +F: tests/functional/ F: tests/vm/ F: tests/lcitool/ F: tests/avocado/tuxrun_baselines.py
tests/functional/test_info_usernet.py +1 -1
@@ -19,7 +19,7 @@ class InfoUsernet(QemuSystemTest): def test_hostfwd(self): self.require_netdev('user') - self.machine = 'none' + self.set_machine('none') self.vm.add_args('-netdev',
'user,id=vnet,hostfwd=:127.0.0.1:0-:22')
self.vm.launch()
res = self.vm.cmd('human-monitor-command',
tests/functional/test_version.py +1 -1
@@ -17,7 +17,7 @@ class Version(QemuSystemTest): def test_qmp_human_info_version(self): - self.machine = 'none' + self.set_machine('none') self.vm.add_args('-nodefaults') self.vm.launch() res = self.vm.cmd('human-monitor-command',
tests/functional/test_x86_cpu_model_versions.py +0 -0
@@ -217,7 +217,7 @@ def test_none_alias(self): """ Check if unversioned CPU model is an alias pointing to some
version
"""
- self.machine = 'none'
+ self.set_machine('none')
self.vm.add_args('-S')
self.vm.launch()---