Convert remaining Avocado tests to functional
cover Convert remaining Avocado tests to functional
These patches convert the remaining Avocado tests bit by bit to the functional framework, or replace them with a test that has similar test coverage. This way we are finally able to get rid of the test/avocado folder in the end, everything is then integrated with the meson test runner. It should be possible now to run almost all tests in parallel by doing something like:
make -j$(nproc) check SPEED=thorough
Thomas Huth (15):
gitlab-ci: Remove the avocado tests from the CI pipelines tests/functional: Move the check for the parameters from avocado to
functional
tests/functional: Convert reverse_debugging tests to the functional
framework
tests/functional: Convert the i386 replay avocado test tests/avocado: Remove the LinuxKernelTest class tests/functional: Convert the 32-bit big endian Wheezy mips test tests/functional: Convert the 32-bit little endian Wheezy mips test tests/functional: Convert the 64-bit little endian Wheezy mips test tests/functional: Convert the 64-bit big endian Wheezy mips test tests/avocado: Remove the boot_linux.py tests tests/functional: Use the tuxrun kernel for the x86 replay test tests/functional: Use the tuxrun kernel for the aarch64 replay test tests/functional: Convert the SMMU test to the functional framework
gitlab-ci: Update QEMU_JOB_AVOCADO and QEMU_CI_AVOCADO_TESTING Remove the remainders of the Avocado tests
MAINTAINERS | 15 +-
docs/about/build-platforms.rst | 10 +-
docs/devel/build-system.rst | 11 +-
docs/devel/codebase.rst | 5 -
docs/devel/testing/avocado.rst | 581 ------------------
docs/devel/testing/ci-jobs.rst.inc | 19 +-
docs/devel/testing/functional.rst | 3 -
docs/devel/testing/index.rst | 1 -
docs/devel/testing/main.rst | 15 -
tests/avocado/README.rst | 10 -
configure | 2 +-
.gitlab-ci.d/base.yml | 8 +-
.gitlab-ci.d/buildtest-template.yml | 13 +-
.gitlab-ci.d/buildtest.yml | 33 +-
pythondeps.toml | 8 +-
tests/Makefile.include | 60 +-
tests/avocado/avocado_qemu/__init__.py | 424 -------------
tests/avocado/avocado_qemu/linuxtest.py | 253 --------
tests/avocado/boot_linux.py | 132 ----
tests/avocado/boot_linux_console.py | 96 ---
tests/avocado/linux_ssh_mips_malta.py | 205 ------
tests/avocado/replay_kernel.py | 110 ----
tests/avocado/replay_linux.py | 206 -------
tests/avocado/smmu.py | 139 -----
tests/functional/meson.build | 13 +-
tests/functional/qemu_test/tuxruntest.py | 9 +-
.../reverse_debugging.py | 114 +---
tests/functional/test_aarch64_replay.py | 37 +-
.../functional/test_aarch64_reverse_debug.py | 37 ++
tests/functional/test_aarch64_smmu.py | 205 ++++++
tests/functional/test_i386_replay.py | 28 +
tests/functional/test_mips64_malta.py | 35 ++
tests/functional/test_mips64el_malta.py | 22 +
tests/functional/test_mips_malta.py | 108 +++-
tests/functional/test_mipsel_malta.py | 22 +
tests/functional/test_ppc64_reverse_debug.py | 41 ++
tests/functional/test_x86_64_replay.py | 43 +-
tests/functional/test_x86_64_reverse_debug.py | 36 ++
38 files changed, 672 insertions(+), 2437 deletions(-)
delete mode 100644 docs/devel/testing/avocado.rst
delete mode 100644 tests/avocado/README.rst
delete mode 100644 tests/avocado/avocado_qemu/__init__.py
delete mode 100644 tests/avocado/avocado_qemu/linuxtest.py
delete mode 100644 tests/avocado/boot_linux.py
delete mode 100644 tests/avocado/boot_linux_console.py
delete mode 100644 tests/avocado/linux_ssh_mips_malta.py
delete mode 100644 tests/avocado/replay_kernel.py
delete mode 100644 tests/avocado/replay_linux.py
delete mode 100644 tests/avocado/smmu.py
rename tests/{avocado => functional}/reverse_debugging.py (66%)
create mode 100755 tests/functional/test_aarch64_reverse_debug.py
create mode 100755 tests/functional/test_aarch64_smmu.py
create mode 100755 tests/functional/test_i386_replay.py
create mode 100755 tests/functional/test_mips64_malta.py
create mode 100755 tests/functional/test_ppc64_reverse_debug.py
create mode 100755 tests/functional/test_x86_64_reverse_debug.pySignature · 1 line
2.49.0
PATCH 1/15 gitlab-ci: Remove the avocado tests from the CI pipelines
From: Thomas Huth <[email protected]>
We are going to move the remaining Avocado tests step by step into the functional test framework. Unfortunately, Avocado fails with an error if it cannot determine a test to run, so disable the tests here now to avoid failures in the Gitlab-CI during the next steps.
--- .gitlab-ci.d/buildtest-template.yml | 11 ---------- .gitlab-ci.d/buildtest.yml | 33 +++++++++++------------------ 2 files changed, 12 insertions(+), 32 deletions(-)
.gitlab-ci.d/buildtest-template.yml +0 -11
@@ -95,7 +95,6 @@ cache: key: "${CI_JOB_NAME}-cache" paths: - - ${CI_PROJECT_DIR}/avocado-cache - ${CI_PROJECT_DIR}/functional-cache policy: pull-push artifacts:
@@ -109,16 +108,6 @@ reports: junit: build/tests/results/latest/results.xml before_script: - - mkdir -p ~/.config/avocado - - echo "[datadir.paths]" > ~/.config/avocado/avocado.conf - - echo "cache_dirs = ['${CI_PROJECT_DIR}/avocado-cache']" - >> ~/.config/avocado/avocado.conf - - echo -e '[job.output.testlogs]\nstatuses = ["FAIL", "INTERRUPT"]' - >> ~/.config/avocado/avocado.conf - - if [ -d ${CI_PROJECT_DIR}/avocado-cache ]; then - du -chs ${CI_PROJECT_DIR}/*-cache ; - fi - - export AVOCADO_ALLOW_UNTRUSTED_CODE=1 - export QEMU_TEST_ALLOW_UNTRUSTED_CODE=1 - export QEMU_TEST_CACHE_DIR=${CI_PROJECT_DIR}/functional-cache after_script:
.gitlab-ci.d/buildtest.yml +12 -21
@@ -29,8 +29,7 @@ functional-system-alpine: artifacts: true variables: IMAGE: alpine - MAKE_CHECK_ARGS: check-avocado check-functional - AVOCADO_TAGS: arch:avr arch:loongarch64 arch:mips64 arch:mipsel + MAKE_CHECK_ARGS: check-functional build-system-ubuntu: extends:
@@ -60,8 +59,7 @@ functional-system-ubuntu: artifacts: true variables: IMAGE: ubuntu2204 - MAKE_CHECK_ARGS: check-avocado check-functional - AVOCADO_TAGS: arch:alpha arch:microblazeel arch:mips64el + MAKE_CHECK_ARGS: check-functional build-system-debian: extends:
@@ -92,8 +90,7 @@ functional-system-debian: artifacts: true variables: IMAGE: debian - MAKE_CHECK_ARGS: check-avocado check-functional - AVOCADO_TAGS: arch:arm arch:i386 arch:riscv64 arch:sh4 arch:sparc arch:xtensa + MAKE_CHECK_ARGS: check-functional crash-test-debian: extends: .native_test_job_template
@@ -155,9 +152,7 @@ functional-system-fedora: artifacts: true variables: IMAGE: fedora - MAKE_CHECK_ARGS: check-avocado check-functional - AVOCADO_TAGS: arch:microblaze arch:mips arch:xtensa arch:m68k - arch:riscv32 arch:ppc arch:sparc64 + MAKE_CHECK_ARGS: check-functional crash-test-fedora: extends: .native_test_job_template
@@ -278,9 +273,7 @@ functional-system-centos: artifacts: true variables: IMAGE: centos9 - MAKE_CHECK_ARGS: check-avocado check-functional - AVOCADO_TAGS: arch:ppc64 arch:or1k arch:s390x arch:x86_64 arch:rx - arch:sh4 + MAKE_CHECK_ARGS: check-functional build-system-opensuse: extends:
@@ -309,8 +302,7 @@ functional-system-opensuse: artifacts: true variables: IMAGE: opensuse-leap - MAKE_CHECK_ARGS: check-avocado check-functional - AVOCADO_TAGS: arch:s390x arch:x86_64 arch:aarch64 + MAKE_CHECK_ARGS: check-functional # # Flaky tests. We don't run these by default and they are allow fail
@@ -338,10 +330,9 @@ functional-system-flaky: allow_failure: true variables: IMAGE: debian - MAKE_CHECK_ARGS: check-avocado check-functional + MAKE_CHECK_ARGS: check-functional QEMU_JOB_OPTIONAL: 1 QEMU_TEST_FLAKY_TESTS: 1 - AVOCADO_TAGS: flaky # This jobs explicitly disable TCG (--disable-tcg), KVM is detected by # the configure script. The container doesn't contain Xen headers so
@@ -482,8 +473,8 @@ clang-user: # Since slirp callbacks are used in QEMU Timers, we cannot use libslirp with # CFI builds, and thus have to disable it here. # -# Split in three sets of build/check/avocado to limit the execution time of each -# job +# Split in three sets of build/check/functional to limit the execution time +# of each job build-cfi-aarch64: extends: - .native_build_job_template
@@ -520,7 +511,7 @@ functional-cfi-aarch64: artifacts: true variables: IMAGE: fedora - MAKE_CHECK_ARGS: check-avocado check-functional + MAKE_CHECK_ARGS: check-functional build-cfi-ppc64-s390x: extends:
@@ -558,7 +549,7 @@ functional-cfi-ppc64-s390x: artifacts: true variables: IMAGE: fedora - MAKE_CHECK_ARGS: check-avocado check-functional + MAKE_CHECK_ARGS: check-functional build-cfi-x86_64: extends:
@@ -592,7 +583,7 @@ functional-cfi-x86_64: artifacts: true variables: IMAGE: fedora - MAKE_CHECK_ARGS: check-avocado check-functional + MAKE_CHECK_ARGS: check-functional tsan-build: extends: .native_build_job_template
Signature · 1 line
2.49.0
Re: [PATCH 01/15] gitlab-ci: Remove the avocado tests from the CI pipelines
Quoted 8 lines · Thomas Huth
From: Thomas Huth <[email protected]>
We are going to move the remaining Avocado tests step by step into the functional test framework. Unfortunately, Avocado fails with an error if it cannot determine a test to run, so disable the tests here now to avoid failures in the Gitlab-CI during the next steps.
Quoted 5 lines
Signed-off-by Thomas Huth <[email protected]>--- .gitlab-ci.d/buildtest-template.yml | 11 ---------- .gitlab-ci.d/buildtest.yml | 33 +++++++++++------------------ 2 files changed, 12 insertions(+), 32 deletions(-)
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
PATCH 2/15 tests/functional: Move the check for the parameters from avocado to functional
In reply to Thomas HuthFrom: Thomas Huth <[email protected]>
test_x86_64_pc in tests/avocado/boot_linux_console.py only checks whether the kernel parameters have correctly been passed to the kernel in the guest by looking for them in the console output of the guest. Let's move that to the functional test framework now, but instead of doing it in a separate test, let's do it for all tuxrun tests instead, so it is done automatically for all targets that have a tuxrun test.
--- tests/avocado/boot_linux_console.py | 34 ------------------------ tests/functional/qemu_test/tuxruntest.py | 9 ++++--- 2 files changed, 5 insertions(+), 38 deletions(-)
tests/avocado/boot_linux_console.py +0 -34
@@ -9,17 +9,9 @@ # later. See the COPYING file in the top-level directory. import os -import lzma -import gzip import shutil -from avocado import skip -from avocado import skipUnless -from avocado import skipUnless from avocado_qemu import QemuSystemTest -from avocado_qemu import exec_command -from avocado_qemu import exec_command_and_wait_for_pattern -from avocado_qemu import interrupt_interactive_console_until_pattern from avocado_qemu import wait_for_console_pattern from avocado.utils import process from avocado.utils import archive
@@ -68,29 +60,3 @@ def extract_from_rpm(self, rpm, path): process.run("rpm2cpio %s | cpio -id %s" % (rpm, path), shell=True) os.chdir(cwd) return os.path.normpath(os.path.join(self.workdir, path)) - -class BootLinuxConsole(LinuxKernelTest): - """ - Boots a Linux kernel and checks that the console is operational and the - kernel command line is properly passed from QEMU to the kernel - """ - timeout = 90 - - def test_x86_64_pc(self): - """ - :avocado: tags=arch:x86_64 - :avocado: tags=machine:pc - """ - kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora' - '/linux/releases/29/Everything/x86_64/os/images/pxeboot' - '/vmlinuz') - kernel_hash = '23bebd2680757891cf7adedb033532163a792495' - kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash) - - self.vm.set_console() - kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE + 'console=ttyS0' - self.vm.add_args('-kernel', kernel_path, - '-append', kernel_command_line) - self.vm.launch() - console_pattern = 'Kernel command line: %s' % kernel_command_line - self.wait_for_console_pattern(console_pattern)
tests/functional/qemu_test/tuxruntest.py +5 -4
@@ -77,12 +77,12 @@ def prepare_run(self, kernel, disk, drive, dtb=None, console_index=0): blockdev = "driver=raw,file.driver=file," \ + f"file.filename={disk},node-name=hd0" - kcmd_line = self.KERNEL_COMMON_COMMAND_LINE - kcmd_line += f" root=/dev/{self.root}" - kcmd_line += f" console={self.console}" + self.kcmd_line = self.KERNEL_COMMON_COMMAND_LINE + self.kcmd_line += f" root=/dev/{self.root}" + self.kcmd_line += f" console={self.console}" self.vm.add_args('-kernel', kernel, - '-append', kcmd_line, + '-append', self.kcmd_line, '-blockdev', blockdev) # Sometimes we need extra devices attached
@@ -103,6 +103,7 @@ def run_tuxtest_tests(self, haltmsg): wait to exit cleanly. """ ps1='root@tuxtest:~#' + self.wait_for_console_pattern(self.kcmd_line) self.wait_for_console_pattern('tuxtest login:') exec_command_and_wait_for_pattern(self, 'root', ps1) exec_command_and_wait_for_pattern(self, 'cat /proc/interrupts', ps1)
Signature · 1 line
2.49.0
Re: [PATCH 02/15] tests/functional: Move the check for the parameters from avocado to functional
Quoted 10 lines · Thomas Huth
From: Thomas Huth <[email protected]>
test_x86_64_pcintests/avocado/boot_linux_console.pyonly checks whether the kernel parameters have correctly been passed to the kernel in the guest by looking for them in the console output of the guest. Let's move that to the functional test framework now, but instead of doing it in a separate test, let's do it for all tuxrun tests instead, so it is done automatically for all targets that have a tuxrun test.
Quoted 5 lines
Signed-off-by Thomas Huth <[email protected]>--- tests/avocado/boot_linux_console.py | 34 ------------------------ tests/functional/qemu_test/tuxruntest.py | 9 ++++--- 2 files changed, 5 insertions(+), 38 deletions(-)
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
PATCH 3/15 tests/functional: Convert reverse_debugging tests to the functional framework
In reply to Thomas HuthFrom: Thomas Huth <[email protected]>
These tests are using the gdb-related library functions from the Avocado framework which we don't have in the functional framework yet. So for the time being, keep those imports and skip the test if the Avocado framework is not installed on the host.
---
MAINTAINERS | 2 +-
tests/functional/meson.build | 4 +
.../reverse_debugging.py | 114 +++---------------
.../functional/test_aarch64_reverse_debug.py | 37 ++++++
tests/functional/test_ppc64_reverse_debug.py | 41 +++++++
tests/functional/test_x86_64_reverse_debug.py | 36 ++++++
6 files changed, 138 insertions(+), 96 deletions(-)
rename tests/{avocado => functional}/reverse_debugging.py (66%)
create mode 100755 tests/functional/test_aarch64_reverse_debug.py
create mode 100755 tests/functional/test_ppc64_reverse_debug.py
create mode 100755 tests/functional/test_x86_64_reverse_debug.pyMAINTAINERS +1 -1
@@ -3670,7 +3670,7 @@ F: docs/system/replay.rst F: stubs/replay.c F: tests/avocado/replay_kernel.py F: tests/avocado/replay_linux.py -F: tests/avocado/reverse_debugging.py +F: tests/functional/*reverse_debug*.py F: tests/functional/*replay*.py F: qapi/replay.json
tests/functional/meson.build +4 -0
@@ -13,6 +13,7 @@ endif test_timeouts = { 'aarch64_aspeed' : 600, 'aarch64_raspi4' : 480, + 'aarch64_reverse_debug' : 180, 'aarch64_rme_virt' : 1200, 'aarch64_rme_sbsaref' : 1200, 'aarch64_sbsaref_alpine' : 1200,
@@ -78,6 +79,7 @@ tests_aarch64_system_thorough = [ 'aarch64_raspi3', 'aarch64_raspi4', 'aarch64_replay', + 'aarch64_reverse_debug', 'aarch64_rme_virt', 'aarch64_rme_sbsaref', 'aarch64_sbsaref',
@@ -229,6 +231,7 @@ tests_ppc64_system_thorough = [ 'ppc64_powernv', 'ppc64_pseries', 'ppc64_replay', + 'ppc64_reverse_debug', 'ppc64_tuxrun', 'ppc64_mac99', ]
@@ -311,6 +314,7 @@ tests_x86_64_system_thorough = [ 'x86_64_hotplug_cpu', 'x86_64_kvm_xen', 'x86_64_replay', + 'x86_64_reverse_debug', 'x86_64_tuxrun', ]
tests/functional/reverse_debugging.py renamed from tests/avocado/reverse_debugging.py +19 -95
158 lines not on this page · raw message
tests/functional/test_aarch64_reverse_debug.py new file +37 -0
@@ -0,0 +1,37 @@ +#!/usr/bin/env python3 +# +# SPDX-License-Identifier: GPL-2.0-or-later +# +# Reverse debugging test +# +# Copyright (c) 2020 ISP RAS +# +# Author: +# Pavel Dovgalyuk <[email protected]> +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + +from qemu_test import Asset, skipIfMissingImports +from reverse_debugging import ReverseDebugging + + +@skipIfMissingImports('avocado') +class ReverseDebugging_AArch64(ReverseDebugging): + + REG_PC = 32 + + KERNEL_ASSET = Asset( + ('https://archives.fedoraproject.org/pub/archive/fedora/linux/' + 'releases/29/Everything/aarch64/os/images/pxeboot/vmlinuz'), + '7e1430b81c26bdd0da025eeb8fbd77b5dc961da4364af26e771bd39f379cbbf7') + + def test_aarch64_virt(self): + self.set_machine('virt') + self.cpu = 'cortex-a53' + kernel_path = self.KERNEL_ASSET.fetch() + self.reverse_debugging(args=('-kernel', kernel_path)) + + +if __name__ == '__main__': + ReverseDebugging.main()
tests/functional/test_ppc64_reverse_debug.py new file +41 -0
41 lines not on this page · raw message
tests/functional/test_x86_64_reverse_debug.py new file +36 -0
@@ -0,0 +1,36 @@ +#!/usr/bin/env python3 +# +# SPDX-License-Identifier: GPL-2.0-or-later +# +# Reverse debugging test +# +# Copyright (c) 2020 ISP RAS +# +# Author: +# Pavel Dovgalyuk <[email protected]> +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + +from qemu_test import skipIfMissingImports, skipFlakyTest +from reverse_debugging import ReverseDebugging + + +@skipIfMissingImports('avocado') +class ReverseDebugging_X86_64(ReverseDebugging): + + REG_PC = 0x10 + REG_CS = 0x12 + def get_pc(self, g): + return self.get_reg_le(g, self.REG_PC) \ + + self.get_reg_le(g, self.REG_CS) * 0x10 + + @skipFlakyTest("https://gitlab.com/qemu-project/qemu/-/issues/1992") + def test_x86_64_pc(self): + self.set_machine('pc') + # start with BIOS only + self.reverse_debugging() + + +if __name__ == '__main__': + ReverseDebugging.main()
Signature · 1 line
2.49.0
Re: [PATCH 03/15] tests/functional: Convert reverse_debugging tests to the functional framework
Quoted 50 lines · Thomas Huth
From: Thomas Huth <[email protected]>
These tests are using the gdb-related library functions from the Avocado framework which we don't have in the functional framework yet. So for the time being, keep those imports and skip the test if the Avocado framework is not installed on the host.
Signed-off-by Thomas Huth <[email protected]>--- MAINTAINERS | 2 +- tests/functional/meson.build | 4 + .../reverse_debugging.py | 114 +++--------------- .../functional/test_aarch64_reverse_debug.py | 37 ++++++ tests/functional/test_ppc64_reverse_debug.py | 41 +++++++ tests/functional/test_x86_64_reverse_debug.py | 36 ++++++ 6 files changed, 138 insertions(+), 96 deletions(-) rename tests/{avocado => functional}/reverse_debugging.py (66%) create mode 100755 tests/functional/test_aarch64_reverse_debug.py create mode 100755 tests/functional/test_ppc64_reverse_debug.py create mode 100755 tests/functional/test_x86_64_reverse_debug.pyMAINTAINERS +1
-1@@ -3670,7 +3670,7 @@ F: docs/system/replay.rst F: stubs/replay.c F: tests/avocado/replay_kernel.py F: tests/avocado/replay_linux.py -F: tests/avocado/reverse_debugging.py +F: tests/functional/*reverse_debug*.py F: tests/functional/*replay*.py F: qapi/replay.jsontests/functional/test_aarch64_reverse_debug.py new file +9
-0@@ -0,0 +1,37 @@ +#!/usr/bin/env python3 +# +# SPDX-License-Identifier: GPL-2.0-or-later +# +# Reverse debugging test +# +# Copyright (c) 2020 ISP RAS +# +# Author:
tests/functional/test_aarch64_reverse_debug.py line 19
Quoted 10 lines
tests/functional/test_aarch64_reverse_debug.py new file +10
-0+# Pavel Dovgalyuk <[email protected]> +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + +from qemu_test import Asset, skipIfMissingImports +from reverse_debugging import ReverseDebugging + + +@skipIfMissingImports('avocado')
Would it make sense to specialize this to 'avocado.utils' ?
Quoted 43 lines
tests/functional/test_aarch64_reverse_debug.py new file +18
-0+class ReverseDebugging_AArch64(ReverseDebugging): + + REG_PC = 32 + + KERNEL_ASSET = Asset( + ('https://archives.fedoraproject.org/pub/archive/fedora/linux/' + 'releases/29/Everything/aarch64/os/images/pxeboot/vmlinuz'), + '7e1430b81c26bdd0da025eeb8fbd77b5dc961da4364af26e771bd39f379cbbf7') + + def test_aarch64_virt(self): + self.set_machine('virt') + self.cpu = 'cortex-a53' + kernel_path = self.KERNEL_ASSET.fetch() + self.reverse_debugging(args=('-kernel', kernel_path)) + + +if __name__ == '__main__': + ReverseDebugging.main()tests/functional/test_x86_64_reverse_debug.py new file +18
-0@@ -0,0 +1,36 @@ +#!/usr/bin/env python3 +# +# SPDX-License-Identifier: GPL-2.0-or-later +# +# Reverse debugging test +# +# Copyright (c) 2020 ISP RAS +# +# Author: +# Pavel Dovgalyuk <[email protected]> +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + +from qemu_test import skipIfMissingImports, skipFlakyTest +from reverse_debugging import ReverseDebugging + +
tests/functional/test_x86_64_reverse_debug.py line 28
Quoted 10 lines
tests/functional/test_x86_64_reverse_debug.py new file +10
-0+@skipIfMissingImports('avocado') +class ReverseDebugging_X86_64(ReverseDebugging): + + REG_PC = 0x10 + REG_CS = 0x12 + def get_pc(self, g): + return self.get_reg_le(g, self.REG_PC) \ + + self.get_reg_le(g, self.REG_CS) * 0x10 + + @skipFlakyTest("https://gitlab.com/qemu-project/qemu/-/issues/1992")
This bug refers to ppc64. Was this a copy and paste mistake, does that bug need updating to also mention x86_64 ?
Quoted 11 lines
tests/functional/test_x86_64_reverse_debug.py new file +8
-0+ def test_x86_64_pc(self): + self.set_machine('pc') + # start with BIOS only + self.reverse_debugging() + + +if __name__ == '__main__': + ReverseDebugging.main()Signature · 1 line
2.49.0
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [PATCH 03/15] tests/functional: Convert reverse_debugging tests to the functional framework
Quoted 55 lines · Daniel P. Berrangé
Quoted 54 lines · Thomas Huth
From: Thomas Huth <[email protected]>
These tests are using the gdb-related library functions from the Avocado framework which we don't have in the functional framework yet. So for the time being, keep those imports and skip the test if the Avocado framework is not installed on the host.
Signed-off-by Thomas Huth <[email protected]>--- MAINTAINERS | 2 +- tests/functional/meson.build | 4 + .../reverse_debugging.py | 114 +++--------------- .../functional/test_aarch64_reverse_debug.py | 37 ++++++ tests/functional/test_ppc64_reverse_debug.py | 41 +++++++ tests/functional/test_x86_64_reverse_debug.py | 36 ++++++ 6 files changed, 138 insertions(+), 96 deletions(-)rename tests/{avocado =>
functional}/reverse_debugging.py(66%) create mode 100755tests/functional/test_aarch64_reverse_debug.pycreate mode 100755tests/functional/test_ppc64_reverse_debug.pycreate mode 100755tests/functional/test_x86_64_reverse_debug.pyMAINTAINERS +1
-1@@ -3670,7 +3670,7 @@ F: docs/system/replay.rst F: stubs/replay.c F: tests/avocado/replay_kernel.py F: tests/avocado/replay_linux.py -F: tests/avocado/reverse_debugging.py +F: tests/functional/*reverse_debug*.py F: tests/functional/*replay*.py F: qapi/replay.jsontests/functional/test_aarch64_reverse_debug.py new file +13
-0@@ -0,0 +1,37 @@ +#!/usr/bin/env python3 +# +# SPDX-License-Identifier: GPL-2.0-or-later +# +# Reverse debugging test +# +# Copyright (c) 2020 ISP RAS +# +# Author: +# Pavel Dovgalyuk <[email protected]> +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory.
Quoted 8 lines
tests/functional/test_aarch64_reverse_debug.py line 19
Quoted 6 lines
tests/functional/test_aarch64_reverse_debug.py new file +6
-0+ +from qemu_test import Asset, skipIfMissingImports +from reverse_debugging import ReverseDebugging + + +@skipIfMissingImports('avocado')Would it make sense to specialize this to 'avocado.utils' ?
Sure, can do!
Quoted 48 lines
Quoted 48 lines
tests/functional/test_aarch64_reverse_debug.py new file +18
-0+class ReverseDebugging_AArch64(ReverseDebugging): + + REG_PC = 32 + + KERNEL_ASSET = Asset( + ('https://archives.fedoraproject.org/pub/archive/fedora/linux/' + 'releases/29/Everything/aarch64/os/images/pxeboot/vmlinuz'), + '7e1430b81c26bdd0da025eeb8fbd77b5dc961da4364af26e771bd39f379cbbf7') + + def test_aarch64_virt(self): + self.set_machine('virt') + self.cpu = 'cortex-a53' + kernel_path = self.KERNEL_ASSET.fetch() + self.reverse_debugging(args=('-kernel', kernel_path)) + + +if __name__ == '__main__': + ReverseDebugging.main()tests/functional/test_x86_64_reverse_debug.py new file +23
-0@@ -0,0 +1,36 @@ +#!/usr/bin/env python3 +# +# SPDX-License-Identifier: GPL-2.0-or-later +# +# Reverse debugging test +# +# Copyright (c) 2020 ISP RAS +# +# Author: +# Pavel Dovgalyuk <[email protected]> +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + +from qemu_test import skipIfMissingImports, skipFlakyTest +from reverse_debugging import ReverseDebugging + + +@skipIfMissingImports('avocado') +class ReverseDebugging_X86_64(ReverseDebugging): + + REG_PC = 0x10 + REG_CS = 0x12
Quoted 8 lines
tests/functional/test_x86_64_reverse_debug.py line 28
Quoted 5 lines
tests/functional/test_x86_64_reverse_debug.py new file +5
-0+ def get_pc(self, g): + return self.get_reg_le(g, self.REG_PC) \ + + self.get_reg_le(g, self.REG_CS) * 0x10 + + @skipFlakyTest("https://gitlab.com/qemu-project/qemu/-/issues/1992")This bug refers to ppc64. Was this a copy and paste mistake, does that bug need updating to also mention
x86_64?
I'm not sure whether it's the same bug, so I opened another one for x86 here:
https://gitlab.com/qemu-project/qemu/-/issues/2922
I also noticed that it fails on aarch64 in some rare cases, and in a different way. So here's another bug for aarch64:
https://gitlab.com/qemu-project/qemu/-/issues/2921
I'll use those in the decorators in the next version of the patch.
Thomas
PATCH 4/15 tests/functional: Convert the i386 replay avocado test
In reply to Thomas HuthFrom: Thomas Huth <[email protected]>
Since this was the last test in tests/avocado/replay_kernel.py, we can remove that Avocado file now.
--- MAINTAINERS | 1 - tests/avocado/replay_kernel.py | 110 --------------------------- tests/functional/meson.build | 1 + tests/functional/test_i386_replay.py | 28 +++++++ 4 files changed, 29 insertions(+), 111 deletions(-) delete mode 100644 tests/avocado/replay_kernel.py create mode 100755 tests/functional/test_i386_replay.py
MAINTAINERS +0 -1
@@ -3668,7 +3668,6 @@ F: include/system/replay.h F: docs/devel/replay.rst F: docs/system/replay.rst F: stubs/replay.c -F: tests/avocado/replay_kernel.py F: tests/avocado/replay_linux.py F: tests/functional/*reverse_debug*.py F: tests/functional/*replay*.py
tests/avocado/replay_kernel.py deleted +0 -110
110 lines not on this page · raw message
tests/functional/meson.build +1 -0
@@ -151,6 +151,7 @@ tests_i386_system_quick = [ ] tests_i386_system_thorough = [ + 'i386_replay', 'i386_tuxrun', ]
tests/functional/test_i386_replay.py new file +28 -0
@@ -0,0 +1,28 @@ +#!/usr/bin/env python3 +# +# Replay test that boots a Linux kernel on a i386 machine +# and checks the console +# +# SPDX-License-Identifier: GPL-2.0-or-later + +from qemu_test import Asset +from replay_kernel import ReplayKernelBase + + +class I386Replay(ReplayKernelBase): + + ASSET_KERNEL = Asset( + 'https://storage.tuxboot.com/20230331/i386/bzImage', + 'a3e5b32a354729e65910f5a1ffcda7c14a6c12a55e8213fb86e277f1b76ed956') + + def test_pc(self): + self.set_machine('pc') + kernel_url = () + kernel_path = self.ASSET_KERNEL.fetch() + kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE + 'console=ttyS0' + console_pattern = 'VFS: Cannot open root device' + self.run_rr(kernel_path, kernel_command_line, console_pattern, shift=5) + + +if __name__ == '__main__': + ReplayKernelBase.main()
Signature · 1 line
2.49.0
Re: [PATCH 04/15] tests/functional: Convert the i386 replay avocado test
Quoted 6 lines · Thomas Huth
From: Thomas Huth <[email protected]>
Since this was the last test in
tests/avocado/replay_kernel.py, we can remove that Avocado file now.Signed-off-by Thomas Huth <[email protected]
Quoted 8 lines
--- MAINTAINERS | 1 - tests/avocado/replay_kernel.py | 110 --------------------------- tests/functional/meson.build | 1 + tests/functional/test_i386_replay.py | 28 +++++++ 4 files changed, 29 insertions(+), 111 deletions(-) delete mode 100644 tests/avocado/replay_kernel.py create mode 100755 tests/functional/test_i386_replay.py
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
PATCH 5/15 tests/avocado: Remove the LinuxKernelTest class
In reply to Thomas HuthFrom: Thomas Huth <[email protected]>
All tests that used this class have been converted to the functional framework, so we can remove the boot_linux_console.py file now.
--- tests/avocado/boot_linux_console.py | 62 ----------------------------- 1 file changed, 62 deletions(-) delete mode 100644 tests/avocado/boot_linux_console.py
tests/avocado/boot_linux_console.py deleted +0 -62
@@ -1,62 +0,0 @@ -# Functional test that boots a Linux kernel and checks the console -# -# Copyright (c) 2018 Red Hat, Inc. -# -# Author: -# Cleber Rosa <[email protected]> -# -# This work is licensed under the terms of the GNU GPL, version 2 or -# later. See the COPYING file in the top-level directory. - -import os -import shutil - -from avocado_qemu import QemuSystemTest -from avocado_qemu import wait_for_console_pattern -from avocado.utils import process -from avocado.utils import archive - -class LinuxKernelTest(QemuSystemTest): - KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 ' - - def wait_for_console_pattern(self, success_message, vm=None): - wait_for_console_pattern(self, success_message, - failure_message='Kernel panic - not syncing', - vm=vm) - - def extract_from_deb(self, deb, path): - """ - Extracts a file from a deb package into the test workdir - - :param deb: path to the deb archive - :param path: path within the deb archive of the file to be extracted - :returns: path of the extracted file - """ - cwd = os.getcwd() - os.chdir(self.workdir) - file_path = process.run("ar t %s" % deb).stdout_text.split()[2] - process.run("ar x %s %s" % (deb, file_path)) - archive.extract(file_path, self.workdir) - os.chdir(cwd) - # Return complete path to extracted file. Because callers to - # extract_from_deb() specify 'path' with a leading slash, it is - # necessary to use os.path.relpath() as otherwise os.path.join() - # interprets it as an absolute path and drops the self.workdir part. - return os.path.normpath(os.path.join(self.workdir, - os.path.relpath(path, '/'))) - - def extract_from_rpm(self, rpm, path): - """ - Extracts a file from an RPM package into the test workdir. - - :param rpm: path to the rpm archive - :param path: path within the rpm archive of the file to be extracted - needs to be a relative path (starting with './') because - cpio(1), which is used to extract the file, expects that. - :returns: path of the extracted file - """ - cwd = os.getcwd() - os.chdir(self.workdir) - process.run("rpm2cpio %s | cpio -id %s" % (rpm, path), shell=True) - os.chdir(cwd) - return os.path.normpath(os.path.join(self.workdir, path))
Signature · 1 line
2.49.0
Re: [PATCH 05/15] tests/avocado: Remove the LinuxKernelTest class
Quoted 10 lines · Thomas Huth
From: Thomas Huth <[email protected]>
All tests that used this class have been converted to the functional framework, so we can remove the
boot_linux_console.pyfile now.Signed-off-by Thomas Huth <[email protected]>--- tests/avocado/boot_linux_console.py | 62 ----------------------------- 1 file changed, 62 deletions(-) delete mode 100644 tests/avocado/boot_linux_console.py
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
PATCH 6/15 tests/functional: Convert the 32-bit big endian Wheezy mips test
In reply to Thomas HuthFrom: Thomas Huth <[email protected]>
The test checks some entries in /proc and the output of some commands ... we put these checks into exportable functions now so that they can be reused more easily.
Additionally the linux_ssh_mips_malta.py uses SSH to test the networking of the guest. Since we don't have a SSH module in the functional framework yet, let's use the check_http_download() function here instead.
And while we're at it, also switch the NIC to e1000 now to get some more test coverage, since the "pcnet" device is already tested in the test test_mips_malta_cpio.
--- tests/avocado/linux_ssh_mips_malta.py | 8 -- tests/functional/meson.build | 2 +- tests/functional/test_mips_malta.py | 108 +++++++++++++++++++++++++- 3 files changed, 107 insertions(+), 11 deletions(-)
tests/avocado/linux_ssh_mips_malta.py +0 -8
@@ -172,14 +172,6 @@ def check_mips_malta(self, uname_m, endianess): # Wait for VM to shut down gracefully self.vm.wait() - def test_mips_malta32eb_kernel3_2_0(self): - """ - :avocado: tags=arch:mips - :avocado: tags=endian:big - :avocado: tags=device:pcnet32 - """ - self.check_mips_malta('mips', 'be') - def test_mips_malta32el_kernel3_2_0(self): """ :avocado: tags=arch:mipsel
tests/functional/meson.build +1 -1
@@ -39,7 +39,7 @@ test_timeouts = { 'arm_tuxrun' : 240, 'arm_sx1' : 360, 'intel_iommu': 300, - 'mips_malta' : 120, + 'mips_malta' : 480, 'mipsel_replay' : 480, 'mips64el_replay' : 180, 'netdev_ethtool' : 180,
tests/functional/test_mips_malta.py +106 -2
129 lines not on this page · raw message
Signature · 1 line
2.49.0
Re: [PATCH 06/15] tests/functional: Convert the 32-bit big endian Wheezy mips test
Quoted 14 lines · Thomas Huth
From: Thomas Huth <[email protected]>
The test checks some entries in /proc and the output of some commands ... we put these checks into exportable functions now so that they can be reused more easily.
Additionally the
linux_ssh_mips_malta.pyuses SSH to test the networking of the guest. Since we don't have a SSH module in the functional framework yet, let's use thecheck_http_download()function here instead.And while we're at it, also switch the NIC to e1000 now to get some more test coverage, since the "pcnet" device is already tested in the test
test_mips_malta_cpio.
Quoted 6 lines
Signed-off-by Thomas Huth <[email protected]>--- tests/avocado/linux_ssh_mips_malta.py | 8 -- tests/functional/meson.build | 2 +- tests/functional/test_mips_malta.py | 108 +++++++++++++++++++++++++- 3 files changed, 107 insertions(+), 11 deletions(-)
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
PATCH 7/15 tests/functional: Convert the 32-bit little endian Wheezy mips test
In reply to Thomas HuthFrom: Thomas Huth <[email protected]>
Reuse the test function from the big endian test to easily convert the 32-bit little endian Wheezy mips test.
--- tests/avocado/linux_ssh_mips_malta.py | 8 -------- tests/functional/meson.build | 1 + tests/functional/test_mipsel_malta.py | 22 ++++++++++++++++++++++ 3 files changed, 23 insertions(+), 8 deletions(-)
tests/avocado/linux_ssh_mips_malta.py +0 -8
@@ -172,14 +172,6 @@ def check_mips_malta(self, uname_m, endianess): # Wait for VM to shut down gracefully self.vm.wait() - def test_mips_malta32el_kernel3_2_0(self): - """ - :avocado: tags=arch:mipsel - :avocado: tags=endian:little - :avocado: tags=device:pcnet32 - """ - self.check_mips_malta('mips', 'le') - def test_mips_malta64eb_kernel3_2_0(self): """ :avocado: tags=arch:mips64
tests/functional/meson.build +1 -0
@@ -40,6 +40,7 @@ test_timeouts = { 'arm_sx1' : 360, 'intel_iommu': 300, 'mips_malta' : 480, + 'mipsel_malta' : 420, 'mipsel_replay' : 480, 'mips64el_replay' : 180, 'netdev_ethtool' : 180,
tests/functional/test_mipsel_malta.py +22 -0
@@ -13,6 +13,8 @@ from qemu_test import interrupt_interactive_console_until_pattern from qemu_test import wait_for_console_pattern +from test_mips_malta import mips_check_wheezy + class MaltaMachineConsole(LinuxKernelTest):
@@ -57,6 +59,26 @@ def test_mips_malta32el_nanomips_16k_up(self): def test_mips_malta32el_nanomips_64k_dbg(self): self.do_test_mips_malta32el_nanomips(self.ASSET_KERNEL_64K) + ASSET_WHEEZY_KERNEL = Asset( + ('https://people.debian.org/~aurel32/qemu/mipsel/' + 'vmlinux-3.2.0-4-4kc-malta'), + 'dc8a3648305b0201ca7a5cd135fe2890067a65d93c38728022bb0e656ad2bf9a') + + ASSET_WHEEZY_DISK = Asset( + ('https://people.debian.org/~aurel32/qemu/mipsel/' + 'debian_wheezy_mipsel_standard.qcow2'), + '454f09ae39f7e6461c84727b927100d2c7813841f2a0a5dce328114887ecf914') + + def test_wheezy(self): + kernel_path = self.ASSET_WHEEZY_KERNEL.fetch() + image_path = self.ASSET_WHEEZY_DISK.fetch() + kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE + + 'console=ttyS0 root=/dev/sda1') + mips_check_wheezy(self, + kernel_path, image_path, kernel_command_line, + dl_file='/boot/initrd.img-3.2.0-4-4kc-malta', + hsum='9fc9f250ed56a74e35e704ddfd5a1c5a5625adefc5c9da91f649288d3ca000f0') + class MaltaMachineYAMON(QemuSystemTest):
Signature · 1 line
2.49.0
Re: [PATCH 07/15] tests/functional: Convert the 32-bit little endian Wheezy mips test
Quoted 11 lines · Thomas Huth
From: Thomas Huth <[email protected]>
Reuse the test function from the big endian test to easily convert the 32-bit little endian Wheezy mips test.
Signed-off-by Thomas Huth <[email protected]>--- tests/avocado/linux_ssh_mips_malta.py | 8 -------- tests/functional/meson.build | 1 + tests/functional/test_mipsel_malta.py | 22 ++++++++++++++++++++++ 3 files changed, 23 insertions(+), 8 deletions(-)
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
PATCH 8/15 tests/functional: Convert the 64-bit little endian Wheezy mips test
In reply to Thomas HuthFrom: Thomas Huth <[email protected]>
Reuse the test function from the 32-bit big endian test to easily convert the 64-bit little endian Wheezy mips test.
--- tests/avocado/linux_ssh_mips_malta.py | 8 -------- tests/functional/meson.build | 1 + tests/functional/test_mips64el_malta.py | 22 ++++++++++++++++++++++ 3 files changed, 23 insertions(+), 8 deletions(-)
tests/avocado/linux_ssh_mips_malta.py +0 -8
@@ -179,11 +179,3 @@ def test_mips_malta64eb_kernel3_2_0(self): :avocado: tags=device:pcnet32 """ self.check_mips_malta('mips64', 'be') - - def test_mips_malta64el_kernel3_2_0(self): - """ - :avocado: tags=arch:mips64el - :avocado: tags=endian:little - :avocado: tags=device:pcnet32 - """ - self.check_mips_malta('mips64', 'le')
tests/functional/meson.build +1 -0
@@ -42,6 +42,7 @@ test_timeouts = { 'mips_malta' : 480, 'mipsel_malta' : 420, 'mipsel_replay' : 480, + 'mips64el_malta' : 420, 'mips64el_replay' : 180, 'netdev_ethtool' : 180, 'ppc_40p' : 240,
tests/functional/test_mips64el_malta.py +22 -0
@@ -16,6 +16,8 @@ from qemu_test import exec_command_and_wait_for_pattern from qemu_test import skipIfMissingImports, skipFlakyTest, skipUntrustedTest +from test_mips_malta import mips_check_wheezy + class MaltaMachineConsole(LinuxKernelTest):
@@ -90,6 +92,26 @@ def test_mips64el_malta_5KEc_cpio(self): # Wait for VM to shut down gracefully self.vm.wait() + ASSET_WHEEZY_KERNEL = Asset( + ('https://people.debian.org/~aurel32/qemu/mipsel/' + 'vmlinux-3.2.0-4-5kc-malta'), + '5e8b725244c59745bb8b64f5d8f49f25fecfa549f3395fb6d19a3b9e5065b85b') + + ASSET_WHEEZY_DISK = Asset( + ('https://people.debian.org/~aurel32/qemu/mipsel/' + 'debian_wheezy_mipsel_standard.qcow2'), + '454f09ae39f7e6461c84727b927100d2c7813841f2a0a5dce328114887ecf914') + + def test_wheezy(self): + kernel_path = self.ASSET_WHEEZY_KERNEL.fetch() + image_path = self.ASSET_WHEEZY_DISK.fetch() + kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE + + 'console=ttyS0 root=/dev/sda1') + mips_check_wheezy(self, + kernel_path, image_path, kernel_command_line, cpuinfo='MIPS 20Kc', + dl_file='/boot/initrd.img-3.2.0-4-5kc-malta', + hsum='7579f8b56c1187c7c04d0dc3c0c56c7a6314c5ddd3a9bf8803ecc7cf8a3be9f8') + @skipIfMissingImports('numpy', 'cv2') class MaltaMachineFramebuffer(LinuxKernelTest):
Signature · 1 line
2.49.0
Re: [PATCH 08/15] tests/functional: Convert the 64-bit little endian Wheezy mips test
Quoted 11 lines · Thomas Huth
From: Thomas Huth <[email protected]>
Reuse the test function from the 32-bit big endian test to easily convert the 64-bit little endian Wheezy mips test.
Signed-off-by Thomas Huth <[email protected]>--- tests/avocado/linux_ssh_mips_malta.py | 8 -------- tests/functional/meson.build | 1 + tests/functional/test_mips64el_malta.py | 22 ++++++++++++++++++++++ 3 files changed, 23 insertions(+), 8 deletions(-)
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
PATCH 9/15 tests/functional: Convert the 64-bit big endian Wheezy mips test
In reply to Thomas HuthFrom: Thomas Huth <[email protected]>
Reuse the test function from the 32-bit big endian test to easily convert the 64-bit big endian Wheezy mips test.
Since this was the last test in tests/avocado/linux_ssh_mips_malta.py, we can remove this avocado file now, too.
--- MAINTAINERS | 1 - tests/avocado/linux_ssh_mips_malta.py | 181 -------------------------- tests/functional/meson.build | 2 + tests/functional/test_mips64_malta.py | 35 +++++ 4 files changed, 37 insertions(+), 182 deletions(-) delete mode 100644 tests/avocado/linux_ssh_mips_malta.py create mode 100755 tests/functional/test_mips64_malta.py
MAINTAINERS +0 -1
@@ -1354,7 +1354,6 @@ F: hw/acpi/piix4.c F: hw/mips/malta.c F: hw/pci-host/gt64120.c F: include/hw/southbridge/piix.h -F: tests/avocado/linux_ssh_mips_malta.py F: tests/functional/test_mips*_malta.py F: tests/functional/test_mips*_tuxrun.py
tests/avocado/linux_ssh_mips_malta.py deleted +0 -181
181 lines not on this page · raw message
tests/functional/meson.build +2 -0
@@ -42,6 +42,7 @@ test_timeouts = { 'mips_malta' : 480, 'mipsel_malta' : 420, 'mipsel_replay' : 480, + 'mips64_malta' : 240, 'mips64el_malta' : 420, 'mips64el_replay' : 180, 'netdev_ethtool' : 180,
@@ -191,6 +192,7 @@ tests_mipsel_system_thorough = [ ] tests_mips64_system_thorough = [ + 'mips64_malta', 'mips64_tuxrun', ]
tests/functional/test_mips64_malta.py new file +35 -0
@@ -0,0 +1,35 @@ +#!/usr/bin/env python3 +# +# Functional tests for the big-endian 64-bit MIPS Malta board +# +# SPDX-License-Identifier: GPL-2.0-or-later + +from qemu_test import LinuxKernelTest, Asset +from test_mips_malta import mips_check_wheezy + + +class MaltaMachineConsole(LinuxKernelTest): + + ASSET_WHEEZY_KERNEL = Asset( + ('https://people.debian.org/~aurel32/qemu/mips/' + 'vmlinux-3.2.0-4-5kc-malta'), + '3e4ec154db080b3f1839f04dde83120654a33e5e1716863de576c47cb94f68f6') + + ASSET_WHEEZY_DISK = Asset( + ('https://people.debian.org/~aurel32/qemu/mips/' + 'debian_wheezy_mips_standard.qcow2'), + 'de03599285b8382ad309309a6c4869f6c6c42a5cfc983342bab9ec0dfa7849a2') + + def test_wheezy(self): + kernel_path = self.ASSET_WHEEZY_KERNEL.fetch() + image_path = self.ASSET_WHEEZY_DISK.fetch() + kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE + + 'console=ttyS0 root=/dev/sda1') + mips_check_wheezy(self, + kernel_path, image_path, kernel_command_line, cpuinfo='MIPS 20Kc', + dl_file='/boot/initrd.img-3.2.0-4-5kc-malta', + hsum='d98b953bb4a41c0fc0fd8d19bbc691c08989ac52568c1d3054d92dfd890d3f06') + + +if __name__ == '__main__': + LinuxKernelTest.main()
Signature · 1 line
2.49.0
Re: [PATCH 09/15] tests/functional: Convert the 64-bit big endian Wheezy mips test
Quoted 9 lines · Thomas Huth
From: Thomas Huth <[email protected]>
Reuse the test function from the 32-bit big endian test to easily convert the 64-bit big endian Wheezy mips test.
Since this was the last test in
tests/avocado/linux_ssh_mips_malta.py, we can remove this avocado file now, too.Signed-off-by Thomas Huth <[email protected]>
Quoted 8 lines
--- MAINTAINERS | 1 - tests/avocado/linux_ssh_mips_malta.py | 181 -------------------------- tests/functional/meson.build | 2 + tests/functional/test_mips64_malta.py | 35 +++++ 4 files changed, 37 insertions(+), 182 deletions(-) delete mode 100644 tests/avocado/linux_ssh_mips_malta.py create mode 100755 tests/functional/test_mips64_malta.py
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
PATCH 10/15 tests/avocado: Remove the boot_linux.py tests
In reply to Thomas HuthFrom: Thomas Huth <[email protected]>
These tests are based on the cloudinit functions from Avocado. The cloudinit is very, very slow compared to our other tests, so most of these Avocado tests have either been disabled by default with a decorator, or have been marked to only run with KVM.
We won't include this sluggish cloudinit stuff in the functional framework, and we've already got plenty of other tests there that check pretty much the same things, so let's simply get rid of these old tests now.
--- tests/avocado/boot_linux.py | 132 ------------------------------------ 1 file changed, 132 deletions(-) delete mode 100644 tests/avocado/boot_linux.py
tests/avocado/boot_linux.py deleted +0 -132
@@ -1,132 +0,0 @@ -# Functional test that boots a complete Linux system via a cloud image -# -# Copyright (c) 2018-2020 Red Hat, Inc. -# -# Author: -# Cleber Rosa <[email protected]> -# -# This work is licensed under the terms of the GNU GPL, version 2 or -# later. See the COPYING file in the top-level directory. - -import os - -from avocado_qemu.linuxtest import LinuxTest -from avocado_qemu import BUILD_DIR - -from avocado import skipUnless - - -class BootLinuxX8664(LinuxTest): - """ - :avocado: tags=arch:x86_64 - """ - timeout = 480 - - def test_pc_i440fx_tcg(self): - """ - :avocado: tags=machine:pc - :avocado: tags=accel:tcg - """ - self.require_accelerator("tcg") - self.vm.add_args("-accel", "tcg") - self.launch_and_wait(set_up_ssh_connection=False) - - def test_pc_i440fx_kvm(self): - """ - :avocado: tags=machine:pc - :avocado: tags=accel:kvm - """ - self.require_accelerator("kvm") - self.vm.add_args("-accel", "kvm") - self.launch_and_wait(set_up_ssh_connection=False) - - def test_pc_q35_tcg(self): - """ - :avocado: tags=machine:q35 - :avocado: tags=accel:tcg - """ - self.require_accelerator("tcg") - self.vm.add_args("-accel", "tcg") - self.launch_and_wait(set_up_ssh_connection=False) - - def test_pc_q35_kvm(self): - """ - :avocado: tags=machine:q35 - :avocado: tags=accel:kvm - """ - self.require_accelerator("kvm") - self.vm.add_args("-accel", "kvm") - self.launch_and_wait(set_up_ssh_connection=False) - - -# For Aarch64 we only boot KVM tests in CI as booting the current -# Fedora OS in TCG tests is very heavyweight. There are lighter weight -# distros which we use in the machine_aarch64_virt.py tests. -class BootLinuxAarch64(LinuxTest): - """ - :avocado: tags=arch:aarch64 - :avocado: tags=machine:virt - """ - timeout = 720 - - def test_virt_kvm(self): - """ - :avocado: tags=accel:kvm - :avocado: tags=cpu:host - """ - self.require_accelerator("kvm") - self.vm.add_args("-accel", "kvm") - self.vm.add_args("-machine", "virt,gic-version=host") - self.vm.add_args('-bios', - os.path.join(BUILD_DIR, 'pc-bios', - 'edk2-aarch64-code.fd')) - self.vm.add_args('-device', 'virtio-rng-pci,rng=rng0') - self.vm.add_args('-object', 'rng-random,id=rng0,filename=/dev/urandom') - self.launch_and_wait(set_up_ssh_connection=False) - - -# See the tux_baseline.py tests for almost the same coverage in a lot -# less time. -class BootLinuxPPC64(LinuxTest): - """ - :avocado: tags=arch:ppc64 - """ - - timeout = 360 - - @skipUnless(os.getenv('SPEED') == 'slow', 'runtime limited') - def test_pseries_tcg(self): - """ - :avocado: tags=machine:pseries - :avocado: tags=accel:tcg - """ - self.require_accelerator("tcg") - self.vm.add_args("-accel", "tcg") - self.launch_and_wait(set_up_ssh_connection=False) - - def test_pseries_kvm(self): - """ - :avocado: tags=machine:pseries - :avocado: tags=accel:kvm - """ - self.require_accelerator("kvm") - self.vm.add_args("-accel", "kvm") - self.vm.add_args("-machine", "cap-ccf-assist=off") - self.launch_and_wait(set_up_ssh_connection=False) - -class BootLinuxS390X(LinuxTest): - """ - :avocado: tags=arch:s390x - """ - - timeout = 240 - - @skipUnless(os.getenv('SPEED') == 'slow', 'runtime limited') - def test_s390_ccw_virtio_tcg(self): - """ - :avocado: tags=machine:s390-ccw-virtio - :avocado: tags=accel:tcg - """ - self.require_accelerator("tcg") - self.vm.add_args("-accel", "tcg") - self.launch_and_wait(set_up_ssh_connection=False)
Signature · 1 line
2.49.0
PATCH 11/15 tests/functional: Use the tuxrun kernel for the x86 replay test
In reply to Thomas HuthFrom: Thomas Huth <[email protected]>
This way we can do a full boot in record-replay mode and should get a similar test coverage compared to the old replay test from tests/avocado/replay_linux.py. Thus remove the x86 avocado replay_linux test now.
--- tests/avocado/replay_linux.py | 46 -------------------------- tests/functional/test_x86_64_replay.py | 43 ++++++++++++++++++------ 2 files changed, 33 insertions(+), 56 deletions(-)
tests/avocado/replay_linux.py +0 -46
@@ -118,52 +118,6 @@ def run_replay_dump(self, replay_path): except subprocess.CalledProcessError: self.fail('replay-dump.py failed') -@skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might timeout') -class ReplayLinuxX8664(ReplayLinux): - """ - :avocado: tags=arch:x86_64 - :avocado: tags=accel:tcg - """ - - chksum = 'e3c1b309d9203604922d6e255c2c5d098a309c2d46215d8fc026954f3c5c27a0' - - def test_pc_i440fx(self): - """ - :avocado: tags=machine:pc - """ - self.run_rr(shift=1) - - def test_pc_q35(self): - """ - :avocado: tags=machine:q35 - """ - self.run_rr(shift=3) - -@skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might timeout') -class ReplayLinuxX8664Virtio(ReplayLinux): - """ - :avocado: tags=arch:x86_64 - :avocado: tags=virtio - :avocado: tags=accel:tcg - """ - - hdd = 'virtio-blk-pci' - cd = 'virtio-blk-pci' - bus = None - - chksum = 'e3c1b309d9203604922d6e255c2c5d098a309c2d46215d8fc026954f3c5c27a0' - - def test_pc_i440fx(self): - """ - :avocado: tags=machine:pc - """ - self.run_rr(shift=1) - - def test_pc_q35(self): - """ - :avocado: tags=machine:q35 - """ - self.run_rr(shift=3) @skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might timeout') class ReplayLinuxAarch64(ReplayLinux):
tests/functional/test_x86_64_replay.py +33 -10
63 lines not on this page · raw message
Signature · 1 line
2.49.0
Re: [PATCH 11/15] tests/functional: Use the tuxrun kernel for the x86 replay test
Quoted 7 lines · Thomas Huth
From: Thomas Huth <[email protected]>
This way we can do a full boot in record-replay mode and should get a similar test coverage compared to the old replay test from
tests/avocado/replay_linux.py. Thus remove the x86 avocadoreplay_linuxtest now.
Quoted 5 lines
Signed-off-by Thomas Huth <[email protected]>--- tests/avocado/replay_linux.py | 46 -------------------------- tests/functional/test_x86_64_replay.py | 43 ++++++++++++++++++------ 2 files changed, 33 insertions(+), 56 deletions(-)
Quoted 17 lines
tests/functional/test_x86_64_replay.py +3
-1@@ -5,30 +5,53 @@ # # SPDX-License-Identifier: GPL-2.0-or-later -from qemu_test import Asset, skipFlakyTest +from subprocess import check_call, DEVNULL + +from qemu_test import Asset, skipFlakyTest, get_qemu_img from replay_kernel import ReplayKernelBase class X86Replay(ReplayKernelBase):
tests/functional/test_x86_64_replay.py line 22
Quoted 10 lines
tests/functional/test_x86_64_replay.py +6
-3ASSET_KERNEL = Asset( - ('https://archives.fedoraproject.org/pub/archive/fedora/linux' - '/releases/29/Everything/x86_64/os/images/pxeboot/vmlinuz'), - '8f237d84712b1b411baf3af2aeaaee10b9aae8e345ec265b87ab3a39639eb143') + 'https://storage.tuxboot.com/buildroot/20241119/x86_64/bzImage', + 'f57bfc6553bcd6e0a54aab86095bf642b33b5571d14e3af1731b18c87ed5aef8') + + ASSET_ROOTFS = Asset( + 'https://storage.tuxboot.com/buildroot/20241119/x86_64/rootfs.ext4.zst', + '4b8b2a99117519c5290e1202cb36eb6c7aaba92b357b5160f5970cf5fb78a751')
As a general question, I wonder if we want to add some logic to the pre-cache job to clean up old cached files.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [PATCH 11/15] tests/functional: Use the tuxrun kernel for the x86 replay test
Quoted 33 lines · Daniel P. Berrangé
Quoted 7 lines · Thomas Huth
From: Thomas Huth <[email protected]>
This way we can do a full boot in record-replay mode and should get a similar test coverage compared to the old replay test from
tests/avocado/replay_linux.py. Thus remove the x86 avocadoreplay_linuxtest now.Quoted 5 lines
Signed-off-by Thomas Huth <[email protected]>--- tests/avocado/replay_linux.py | 46 -------------------------- tests/functional/test_x86_64_replay.py | 43 ++++++++++++++++++------ 2 files changed, 33 insertions(+), 56 deletions(-)Reviewed-by Daniel P. Berrangé <[email protected]>Quoted 17 lines
tests/functional/test_x86_64_replay.py +3
-1@@ -5,30 +5,53 @@ # # SPDX-License-Identifier: GPL-2.0-or-later -from qemu_test import Asset, skipFlakyTest +from subprocess import check_call, DEVNULL + +from qemu_test import Asset, skipFlakyTest, get_qemu_img from replay_kernel import ReplayKernelBase class X86Replay(ReplayKernelBase):
Quoted 14 lines
tests/functional/test_x86_64_replay.py line 22
Quoted 10 lines
tests/functional/test_x86_64_replay.py +6
-3ASSET_KERNEL = Asset( - ('https://archives.fedoraproject.org/pub/archive/fedora/linux' - '/releases/29/Everything/x86_64/os/images/pxeboot/vmlinuz'), - '8f237d84712b1b411baf3af2aeaaee10b9aae8e345ec265b87ab3a39639eb143') + 'https://storage.tuxboot.com/buildroot/20241119/x86_64/bzImage', + 'f57bfc6553bcd6e0a54aab86095bf642b33b5571d14e3af1731b18c87ed5aef8') + + ASSET_ROOTFS = Asset( + 'https://storage.tuxboot.com/buildroot/20241119/x86_64/rootfs.ext4.zst', + '4b8b2a99117519c5290e1202cb36eb6c7aaba92b357b5160f5970cf5fb78a751')As a general question, I wonder if we want to add some logic to the pre-cache job to clean up old cached files.
Yes, I also asked myself that question already. Being a "cache", there should also be a way to evict old files that are not used anymore.
Maybe we could update the timestamp of the assets each time they are used, and when doing "make clean", we also check the cache and delete the assets with timestamps older than 6 months or so?
Thomas
Re: [PATCH 11/15] tests/functional: Use the tuxrun kernel for the x86 replay test
Quoted 45 lines · Thomas Huth
Quoted 35 lines · Daniel P. Berrangé
Quoted 7 lines · Thomas Huth
From: Thomas Huth <[email protected]>
This way we can do a full boot in record-replay mode and should get a similar test coverage compared to the old replay test from
tests/avocado/replay_linux.py. Thus remove the x86 avocadoreplay_linuxtest now.Quoted 5 lines
Signed-off-by Thomas Huth <[email protected]>--- tests/avocado/replay_linux.py | 46 -------------------------- tests/functional/test_x86_64_replay.py | 43 ++++++++++++++++++------ 2 files changed, 33 insertions(+), 56 deletions(-)Reviewed-by Daniel P. Berrangé <[email protected]>Quoted 19 lines
tests/functional/test_x86_64_replay.py +5
-4@@ -5,30 +5,53 @@ # # SPDX-License-Identifier: GPL-2.0-or-later -from qemu_test import Asset, skipFlakyTest +from subprocess import check_call, DEVNULL + +from qemu_test import Asset, skipFlakyTest, get_qemu_img from replay_kernel import ReplayKernelBase class X86Replay(ReplayKernelBase): ASSET_KERNEL = Asset( - ('https://archives.fedoraproject.org/pub/archive/fedora/linux' - '/releases/29/Everything/x86_64/os/images/pxeboot/vmlinuz'), - '8f237d84712b1b411baf3af2aeaaee10b9aae8e345ec265b87ab3a39639eb143') + 'https://storage.tuxboot.com/buildroot/20241119/x86_64/bzImage', + 'f57bfc6553bcd6e0a54aab86095bf642b33b5571d14e3af1731b18c87ed5aef8')Quoted 8 lines
tests/functional/test_x86_64_replay.py line 22
Quoted 4 lines
tests/functional/test_x86_64_replay.py +4
-0+ + ASSET_ROOTFS = Asset( + 'https://storage.tuxboot.com/buildroot/20241119/x86_64/rootfs.ext4.zst', + '4b8b2a99117519c5290e1202cb36eb6c7aaba92b357b5160f5970cf5fb78a751')As a general question, I wonder if we want to add some logic to the pre-cache job to clean up old cached files.
Quoted 5 lines
Yes, I also asked myself that question already. Being a "cache", there should also be a way to evict old files that are not used anymore. Maybe we could update the timestamp of the assets each time they are used, and when doing "make clean", we also check the cache and delete the assets with timestamps older than 6 months or so?
Yeah, some time based check + "touch" would be needed, so we don't aggressively purge files cached by a dev's use a different branch that happens to not be checked out currently.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
PATCH 12/15 tests/functional: Use the tuxrun kernel for the aarch64 replay test
In reply to Thomas HuthFrom: Thomas Huth <[email protected]>
This way we can do a full boot in record-replay mode and should get a similar test coverage compared to the old replay test from tests/avocado/replay_linux.py.
Since the aarch64 test was the last avocado test in the tests/avocado/replay_linux.py file, we can remove this file now completely.
--- MAINTAINERS | 1 - tests/avocado/replay_linux.py | 160 ------------------------ tests/functional/test_aarch64_replay.py | 37 ++++-- 3 files changed, 29 insertions(+), 169 deletions(-) delete mode 100644 tests/avocado/replay_linux.py
MAINTAINERS +0 -1
@@ -3667,7 +3667,6 @@ F: include/system/replay.h F: docs/devel/replay.rst F: docs/system/replay.rst F: stubs/replay.c -F: tests/avocado/replay_linux.py F: tests/functional/*reverse_debug*.py F: tests/functional/*replay*.py F: qapi/replay.json
tests/avocado/replay_linux.py deleted +0 -160
160 lines not on this page · raw message
tests/functional/test_aarch64_replay.py +29 -8
@@ -5,25 +5,46 @@ # # SPDX-License-Identifier: GPL-2.0-or-later -from qemu_test import Asset +from subprocess import check_call, DEVNULL + +from qemu_test import Asset, get_qemu_img from replay_kernel import ReplayKernelBase class Aarch64Replay(ReplayKernelBase): ASSET_KERNEL = Asset( - ('https://archives.fedoraproject.org/pub/archive/fedora/linux/' - 'releases/29/Everything/aarch64/os/images/pxeboot/vmlinuz'), - '7e1430b81c26bdd0da025eeb8fbd77b5dc961da4364af26e771bd39f379cbbf7') + 'https://storage.tuxboot.com/buildroot/20241119/arm64/Image', + 'b74743c5e89e1cea0f73368d24ae0ae85c5204ff84be3b5e9610417417d2f235') + + ASSET_ROOTFS = Asset( + 'https://storage.tuxboot.com/buildroot/20241119/arm64/rootfs.ext4.zst', + 'a1acaaae2068df4648d04ff75f532aaa8c5edcd6b936122b6f0db4848a07b465') def test_aarch64_virt(self): + self.require_netdev('user') self.set_machine('virt') - self.cpu = 'cortex-a53' + self.cpu = 'cortex-a57' kernel_path = self.ASSET_KERNEL.fetch() + + raw_disk = self.uncompress(self.ASSET_ROOTFS) + disk = self.scratch_file('scratch.qcow2') + qemu_img = get_qemu_img(self) + check_call([qemu_img, 'create', '-f', 'qcow2', '-b', raw_disk, + '-F', 'raw', disk], stdout=DEVNULL, stderr=DEVNULL) + + args = ('-drive', 'file=%s,snapshot=on,id=hd0,if=none' % disk, + '-drive', 'driver=blkreplay,id=hd0-rr,if=none,image=hd0', + '-device', 'virtio-blk-device,drive=hd0-rr', + '-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0-:22', + '-device', 'virtio-net,netdev=vnet', + '-object', 'filter-replay,id=replay,netdev=vnet') + kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE + - 'console=ttyAMA0') - console_pattern = 'VFS: Cannot open root device' - self.run_rr(kernel_path, kernel_command_line, console_pattern) + 'console=ttyAMA0 root=/dev/vda') + console_pattern = 'Welcome to TuxTest' + self.run_rr(kernel_path, kernel_command_line, console_pattern, + args=args) if __name__ == '__main__':
Signature · 1 line
2.49.0
Re: [PATCH 12/15] tests/functional: Use the tuxrun kernel for the aarch64 replay test
Quoted 10 lines · Thomas Huth
From: Thomas Huth <[email protected]>
This way we can do a full boot in record-replay mode and should get a similar test coverage compared to the old replay test from
tests/avocado/replay_linux.py.Since the aarch64 test was the last avocado test in the
tests/avocado/replay_linux.pyfile, we can remove this file now completely.
Quoted 7 lines
Signed-off-by Thomas Huth <[email protected]>--- MAINTAINERS | 1 - tests/avocado/replay_linux.py | 160 ------------------------ tests/functional/test_aarch64_replay.py | 37 ++++-- 3 files changed, 29 insertions(+), 169 deletions(-) delete mode 100644 tests/avocado/replay_linux.py
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
PATCH 13/15 tests/functional: Convert the SMMU test to the functional framework
In reply to Thomas HuthFrom: Thomas Huth <[email protected]>
This test was using cloudinit and a "dnf install" command in the guest to exercise the NIC with SMMU enabled. Since we don't have the cloudinit stuff in the functional framework and we should not rely on having access to external networks (once our ASSETs have been cached), we rather boot into the initrd first, manually mount the root disk and then use the check_http_download() function from the functional framework here instead for testing whether the network works as expected.
Unfortunately, there seems to be a small race when using the files from Fedora 33: To enter the initrd shell, we have to send a "return" once. But it does not seem to work if we send it too early. Using a sleep(0.2) makes it work reliably for me, but to make it even more unlikely to trigger this situation, let's better limit the Fedora 33 tests to only run with KVM.
Finally, while we're at it, we also add some lines for testing writes to the hard disk, as we already do it in the test_intel_iommu test.
--- MAINTAINERS | 2 +- tests/avocado/smmu.py | 139 ----------------- tests/functional/meson.build | 2 + tests/functional/test_aarch64_smmu.py | 205 ++++++++++++++++++++++++++ 4 files changed, 208 insertions(+), 140 deletions(-) delete mode 100644 tests/avocado/smmu.py create mode 100755 tests/functional/test_aarch64_smmu.py
MAINTAINERS +1 -1
@@ -211,7 +211,7 @@ L: [email protected] S: Maintained F: hw/arm/smmu* F: include/hw/arm/smmu* -F: tests/avocado/smmu.py +F: tests/functional/test_aarch64_smmu.py AVR TCG CPUs M: Michael Rolnik <[email protected]>
tests/avocado/smmu.py deleted +0 -139
139 lines not on this page · raw message
tests/functional/meson.build +2 -0
@@ -18,6 +18,7 @@ test_timeouts = { 'aarch64_rme_sbsaref' : 1200, 'aarch64_sbsaref_alpine' : 1200, 'aarch64_sbsaref_freebsd' : 720, + 'aarch64_smmu' : 720, 'aarch64_tuxrun' : 240, 'aarch64_virt' : 360, 'aarch64_virt_gpu' : 480,
@@ -88,6 +89,7 @@ tests_aarch64_system_thorough = [ 'aarch64_sbsaref', 'aarch64_sbsaref_alpine', 'aarch64_sbsaref_freebsd', + 'aarch64_smmu', 'aarch64_tcg_plugins', 'aarch64_tuxrun', 'aarch64_virt',
tests/functional/test_aarch64_smmu.py new file +205 -0
205 lines not on this page · raw message
Signature · 1 line
2.49.0
PATCH 14/15 gitlab-ci: Update QEMU_JOB_AVOCADO and QEMU_CI_AVOCADO_TESTING
In reply to Thomas HuthFrom: Thomas Huth <[email protected]>
Since we don't run the Avocado jobs in the CI anymore, rename these variables to QEMU_JOB_FUNCTIONAL and QEMU_CI_FUNCTIONAL_TESTING.
Also, there was a mismatch between the documentation and the implementation of QEMU_CI_AVOCADO_TESTING: While the documentation said that you had to "Set this variable to have the tests using the Avocado framework run automatically", you indeed needed to set it to make the pipelines appear in your dashboard - but they were never run automatically in forks and had to be triggered manually. Let's improve this now: No need to hide these pipelines from the users by default anymore (the functional tests should be stable enough nowadays), and rather allow the users to run the pipelines auto- matically with this switch now instead, as was documented.
--- docs/devel/testing/ci-jobs.rst.inc | 19 +++++++++---------- .gitlab-ci.d/base.yml | 8 ++------ .gitlab-ci.d/buildtest-template.yml | 2 +- 3 files changed, 12 insertions(+), 17 deletions(-)
docs/devel/testing/ci-jobs.rst.inc +9 -10
@@ -126,10 +126,10 @@ QEMU_JOB_PUBLISH The job is for publishing content after a branch has been merged into the upstream default branch. -QEMU_JOB_AVOCADO -~~~~~~~~~~~~~~~~ +QEMU_JOB_FUNCTIONAL +~~~~~~~~~~~~~~~~~~~ -The job runs the Avocado integration test suite +The job runs the functional test suite Contributor controlled runtime variables ----------------------------------------
@@ -149,13 +149,12 @@ the jobs to be manually started from the UI Set this variable to 2 to create the pipelines and run all the jobs immediately, as was the historical behaviour -QEMU_CI_AVOCADO_TESTING -~~~~~~~~~~~~~~~~~~~~~~~ -By default, tests using the Avocado framework are not run automatically in -the pipelines (because multiple artifacts have to be downloaded, and if -these artifacts are not already cached, downloading them make the jobs -reach the timeout limit). Set this variable to have the tests using the -Avocado framework run automatically. +QEMU_CI_FUNCTIONAL_TESTING +~~~~~~~~~~~~~~~~~~~~~~~~~~ +By default, tests using the functional framework are not run automatically +in the pipelines (because multiple artifacts have to be downloaded, which +might cause a lot of network traffic). Set this variable to have the tests +using the functional framework run automatically. Other misc variables --------------------
.gitlab-ci.d/base.yml +2 -6
@@ -69,10 +69,6 @@ variables: - if: '$QEMU_CI != "1" && $QEMU_CI != "2" && $CI_PROJECT_NAMESPACE != $QEMU_CI_UPSTREAM' when: never - # Avocado jobs don't run in forks unless $QEMU_CI_AVOCADO_TESTING is set - - if: '$QEMU_JOB_AVOCADO && $QEMU_CI_AVOCADO_TESTING != "1" && $CI_PROJECT_NAMESPACE != $QEMU_CI_UPSTREAM' - when: never - ############################################################# # Stage 2: fine tune execution of jobs in specific scenarios
@@ -101,8 +97,8 @@ variables: when: manual allow_failure: true - # Avocado jobs can be manually start in forks if $QEMU_CI_AVOCADO_TESTING is unset - - if: '$QEMU_JOB_AVOCADO && $CI_PROJECT_NAMESPACE != $QEMU_CI_UPSTREAM' + # Functional jobs can be manually started in forks + - if: '$QEMU_JOB_FUNCTIONAL && $QEMU_CI_FUNCTIONAL_TESTING != "1" && $CI_PROJECT_NAMESPACE != $QEMU_CI_UPSTREAM' when: manual allow_failure: true
.gitlab-ci.d/buildtest-template.yml +1 -1
@@ -114,4 +114,4 @@ - cd build - du -chs ${CI_PROJECT_DIR}/*-cache variables: - QEMU_JOB_AVOCADO: 1 + QEMU_JOB_FUNCTIONAL: 1
Signature · 1 line
2.49.0
Re: [PATCH 14/15] gitlab-ci: Update QEMU_JOB_AVOCADO and QEMU_CI_AVOCADO_TESTING
Quoted 43 lines · Thomas Huth
From: Thomas Huth <[email protected]>
Since we don't run the Avocado jobs in the CI anymore, rename these variables to
QEMU_JOB_FUNCTIONALandQEMU_CI_FUNCTIONAL_TESTING.Also, there was a mismatch between the documentation and the implementation of
QEMU_CI_AVOCADO_TESTING: While the documentation said that you had to "Set this variable to have the tests using the Avocado framework run automatically", you indeed needed to set it to make the pipelines appear in your dashboard - but they were never run automatically in forks and had to be triggered manually. Let's improve this now: No need to hide these pipelines from the users by default anymore (the functional tests should be stable enough nowadays), and rather allow the users to run the pipelines auto- matically with this switch now instead, as was documented.Signed-off-by Thomas Huth <[email protected]>--- docs/devel/testing/ci-jobs.rst.inc | 19 +++++++++---------- .gitlab-ci.d/base.yml | 8 ++------ .gitlab-ci.d/buildtest-template.yml | 2 +- 3 files changed, 12 insertions(+), 17 deletions(-)docs/devel/testing/ci-jobs.rst.inc +3
-3@@ -126,10 +126,10 @@ QEMU_JOB_PUBLISH The job is for publishing content after a branch has been merged into the upstream default branch. -QEMU_JOB_AVOCADO -~~~~~~~~~~~~~~~~ +QEMU_JOB_FUNCTIONAL +~~~~~~~~~~~~~~~~~~~ -The job runs the Avocado integration test suite +The job runs the functional test suite Contributor controlled runtime variables ----------------------------------------@@ -149,13 +149,12 @@ the jobs to be manually started from the UI Set this variable to 2 to create the pipelines and run all
docs/devel/testing/ci-jobs.rst.inc the jobs to be manually started from the UI · line 152
Quoted 10 lines
docs/devel/testing/ci-jobs.rst.inc +1
-7the jobs immediately, as was the historical behaviour -QEMU_CI_AVOCADO_TESTING -~~~~~~~~~~~~~~~~~~~~~~~ -By default, tests using the Avocado framework are not run automatically in -the pipelines (because multiple artifacts have to be downloaded, and if -these artifacts are not already cached, downloading them make the jobs -reach the timeout limit). Set this variable to have the tests using the -Avocado framework run automatically. +QEMU_CI_FUNCTIONAL_TESTING
Both old and new terms are quite verbose, how about dropping '_TESTING' suffix ?
With, or without that rename though:
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [PATCH 14/15] gitlab-ci: Update QEMU_JOB_AVOCADO and QEMU_CI_AVOCADO_TESTING
Quoted 5 lines · Daniel P. Berrangé
Quoted 4 lines · Thomas Huth
From: Thomas Huth <[email protected]>
Since we don't run the Avocado jobs in the CI anymore, rename these variables to
QEMU_JOB_FUNCTIONALandQEMU_CI_FUNCTIONAL_TESTING.
...
Quoted 11 lines
docs/devel/testing/ci-jobs.rst.inc the jobs to be manually started from the UI · line 152
Quoted 8 lines
docs/devel/testing/ci-jobs.rst.inc +1
-7-QEMU_CI_AVOCADO_TESTING -~~~~~~~~~~~~~~~~~~~~~~~ -By default, tests using the Avocado framework are not run automatically in -the pipelines (because multiple artifacts have to be downloaded, and if -these artifacts are not already cached, downloading them make the jobs -reach the timeout limit). Set this variable to have the tests using the -Avocado framework run automatically. +QEMU_CI_FUNCTIONAL_TESTINGBoth old and new terms are quite verbose, how about dropping '
_TESTING' suffix ?
Sure, I'll drop it in v2!
Quoted 3 lines
With, or without that rename though:
Reviewed-by Daniel P. Berrangé <[email protected]>
Thanks!
Thomas
PATCH 15/15 Remove the remainders of the Avocado tests
In reply to Thomas HuthFrom: Thomas Huth <[email protected]>
Now that all Avocado tests have been converted to or been replaced by other functional tests, we can delete the remainders of the Avocado tests from the QEMU source tree.
--- MAINTAINERS | 8 +- docs/about/build-platforms.rst | 10 +- docs/devel/build-system.rst | 11 +- docs/devel/codebase.rst | 5 - docs/devel/testing/avocado.rst | 581 ------------------------ docs/devel/testing/functional.rst | 3 - docs/devel/testing/index.rst | 1 - docs/devel/testing/main.rst | 15 - tests/avocado/README.rst | 10 - configure | 2 +- pythondeps.toml | 8 +- tests/Makefile.include | 60 +-- tests/avocado/avocado_qemu/__init__.py | 424 ----------------- tests/avocado/avocado_qemu/linuxtest.py | 253 ----------- 14 files changed, 16 insertions(+), 1375 deletions(-) delete mode 100644 docs/devel/testing/avocado.rst delete mode 100644 tests/avocado/README.rst delete mode 100644 tests/avocado/avocado_qemu/__init__.py delete mode 100644 tests/avocado/avocado_qemu/linuxtest.py
MAINTAINERS +1 -7
@@ -2072,7 +2072,7 @@ S: Supported F: hw/acpi/viot.c F: hw/acpi/viot.h -ACPI/AVOCADO/BIOSBITS +ACPI/FUNCTIONAL/BIOSBITS M: Ani Sinha <[email protected]> M: Michael S. Tsirkin <[email protected]> S: Supported
@@ -4245,12 +4245,6 @@ R: Philippe Mathieu-Daudé <[email protected]> S: Maintained F: tests/tcg/Makefile.target -Integration Testing with the Avocado framework -W: https://trello.com/b/6Qi1pxVn/avocado-qemu -R: Cleber Rosa <[email protected]> -S: Odd Fixes -F: tests/avocado/ - GitLab custom runner (Works On Arm Sponsored) M: Alex Bennée <[email protected]> M: Philippe Mathieu-Daudé <[email protected]>
docs/about/build-platforms.rst +5 -5
@@ -123,11 +123,11 @@ Rust build dependencies to build QEMU. Optional build dependencies - Build components whose absence does not affect the ability to build - QEMU may not be available in distros, or may be too old for QEMU's - requirements. Many of these, such as the Avocado testing framework - or various linters, are written in Python and therefore can also - be installed using ``pip``. Cross compilers are another example + Build components whose absence does not affect the ability to build QEMU + may not be available in distros, or may be too old for our requirements. + Many of these, such as additional modules for the functional testing + framework or various linters, are written in Python and therefore can + also be installed using ``pip``. Cross compilers are another example of optional build-time dependency; in this case it is possible to download them from repositories such as EPEL, to use container-based cross compilation using ``docker`` or ``podman``, or to use pre-built
docs/devel/build-system.rst +3 -8
@@ -134,7 +134,7 @@ in how the build process runs Python code. At this stage, ``configure`` also queries the chosen Python interpreter about QEMU's build dependencies. Note that the build process does *not* -look for ``meson``, ``sphinx-build`` or ``avocado`` binaries in the PATH; +look for ``meson`` or ``sphinx-build`` binaries in the PATH; likewise, there are no options such as ``--meson`` or ``--sphinx-build``. This avoids a potential mismatch, where Meson and Sphinx binaries on the PATH might operate in a different Python environment than the one chosen
@@ -151,7 +151,7 @@ virtual environment with ``pip``, either from wheels in ``python/wheels`` or by downloading the package with PyPI. Downloading can be disabled with ``--disable-download``; and anyway, it only happens when a ``configure`` option (currently, only ``--enable-docs``) is explicitly enabled but -the dependencies are not present\ [#pip]_. +the dependencies are not present. .. [#distlib] The scripts are created based on the package's metadata, specifically the ``console_script`` entry points. This is the
@@ -164,10 +164,6 @@ the dependencies are not present\ [#pip]_. because the Python Packaging Authority provides a package ``distlib.scripts`` to perform this task. -.. [#pip] ``pip`` might also be used when running ``make check-avocado`` - if downloading is enabled, to ensure that Avocado is - available. - The required versions of the packages are stored in a configuration file ``pythondeps.toml``. The format is custom to QEMU, but it is documented at the top of the file itself and it should be easy to understand. The
@@ -497,8 +493,7 @@ number of dynamically created files listed later. ``pyvenv/bin``, and calling ``pip`` to install dependencies. ``tests/Makefile.include`` - Rules for external test harnesses. These include the TCG tests - and the Avocado-based integration tests. + Rules for external test harnesses like the TCG tests. ``tests/docker/Makefile.include`` Rules for Docker tests. Like ``tests/Makefile.include``, this file is
docs/devel/codebase.rst +0 -5
@@ -175,11 +175,6 @@ yet, so sometimes the source code is all you have. * `tests <https://gitlab.com/qemu-project/qemu/-/tree/master/tests>`_: QEMU `test <testing>` suite - - `avocado <https://gitlab.com/qemu-project/qemu/-/tree/master/tests/avocado>`_: - Functional tests booting full VM using `Avocado framework <checkavocado-ref>`. - Those tests will be transformed and moved into - `tests/functional <https://gitlab.com/qemu-project/qemu/-/tree/master/tests/functional>`_ - in the future. - `data <https://gitlab.com/qemu-project/qemu/-/tree/master/tests/data>`_: Data for various tests. - `decode <https://gitlab.com/qemu-project/qemu/-/tree/master/tests/decode>`_:
docs/devel/testing/avocado.rst deleted +0 -581
581 lines not on this page · raw message
docs/devel/testing/functional.rst +0 -3
@@ -6,9 +6,6 @@ Functional testing with Python The ``tests/functional`` directory hosts functional tests written in Python. They are usually higher level tests, and may interact with external resources and with various guest operating systems. -The functional tests have initially evolved from the Avocado tests, so there -is a lot of similarity to those tests here (see :ref:`checkavocado-ref` for -details about the Avocado tests). The tests should be written in the style of the Python `unittest`_ framework, using stdio for the TAP protocol. The folder ``tests/functional/qemu_test``
docs/devel/testing/index.rst +0 -1
@@ -10,7 +10,6 @@ testing infrastructure. main qtest functional - avocado acpi-bits ci fuzzing
docs/devel/testing/main.rst +0 -15
@@ -894,21 +894,6 @@ Python. You can run the functional tests simply by executing: See :ref:`checkfunctional-ref` for more details. -Integration tests using the Avocado Framework ---------------------------------------------- - -The ``tests/avocado`` directory hosts integration tests. They're usually -higher level tests, and may interact with external resources and with -various guest operating systems. - -You can run the avocado tests simply by executing: - -.. code:: - - make check-avocado - -See :ref:`checkavocado-ref` for more details. - .. _checktcg-ref: Testing with "make check-tcg"
tests/avocado/README.rst deleted +0 -10
@@ -1,10 +0,0 @@ -============================================= -Integration tests using the Avocado Framework -============================================= - -This directory contains integration tests. They're usually higher -level, and may interact with external resources and with various -guest operating systems. - -For more information, please refer to ``docs/devel/testing.rst``, -section "Integration tests using the Avocado Framework".
configure +1 -1
@@ -1685,7 +1685,7 @@ LINKS="$LINKS pc-bios/optionrom/Makefile" LINKS="$LINKS pc-bios/s390-ccw/Makefile" LINKS="$LINKS pc-bios/vof/Makefile" LINKS="$LINKS .gdbinit scripts" # scripts needed by relative path in .gdbinit -LINKS="$LINKS tests/avocado tests/data" +LINKS="$LINKS tests/data" LINKS="$LINKS tests/qemu-iotests/check tests/qemu-iotests/Makefile" LINKS="$LINKS python" for f in $LINKS ; do
pythondeps.toml +2 -6
@@ -27,9 +27,5 @@ pycotap = { accepted = ">=1.1.0", installed = "1.3.1" } sphinx = { accepted = ">=3.4.3", installed = "5.3.0", canary = "sphinx-build" } sphinx_rtd_theme = { accepted = ">=0.5", installed = "1.1.1" } -[avocado] -# Note that qemu.git/python/ is always implicitly installed. -# Prefer an LTS version when updating the accepted versions of -# avocado-framework, for example right now the limit is 92.x. -avocado-framework = { accepted = "(>=103.0, <104.0)", installed = "103.0", canary = "avocado" } -pycdlib = { accepted = ">=1.11.0" } +[testdeps] +qemu.qmp = { accepted = ">=0.0.3", installed = "0.0.3" }
tests/Makefile.include +4 -56
104 lines not on this page · raw message
tests/avocado/avocado_qemu/__init__.py deleted +0 -424
424 lines not on this page · raw message
tests/avocado/avocado_qemu/linuxtest.py deleted +0 -253
253 lines not on this page · raw message
Signature · 1 line
2.49.0
Re: [PATCH 15/15] Remove the remainders of the Avocado tests
Quoted 23 lines · Thomas Huth
From: Thomas Huth <[email protected]>
Now that all Avocado tests have been converted to or been replaced by other functional tests, we can delete the remainders of the Avocado tests from the QEMU source tree.
Signed-off-by Thomas Huth <[email protected]>--- MAINTAINERS | 8 +- docs/about/build-platforms.rst | 10 +- docs/devel/build-system.rst | 11 +- docs/devel/codebase.rst | 5 - docs/devel/testing/avocado.rst | 581 ------------------------ docs/devel/testing/functional.rst | 3 - docs/devel/testing/index.rst | 1 - docs/devel/testing/main.rst | 15 - tests/avocado/README.rst | 10 - configure | 2 +- pythondeps.toml | 8 +- tests/Makefile.include | 60 +-- tests/avocado/avocado_qemu/__init__.py | 424 ----------------- tests/avocado/avocado_qemu/linuxtest.py | 253 ----------- 14 files changed, 16 insertions(+), 1375 deletions(-)
Quoted 4 lines
delete mode 100644
docs/devel/testing/avocado.rstdelete mode 100644tests/avocado/README.rstdelete mode 100644tests/avocado/avocado_qemu/__init__.pydelete mode 100644tests/avocado/avocado_qemu/linuxtest.py
Thanks a lot Thomas for this huge effort of converting from Avocado to the simpler functional framework... Sincerely appreciated!
Re: [PATCH 15/15] Remove the remainders of the Avocado tests
In reply to Thomas HuthQuoted 19 lines · Thomas Huth
From: Thomas Huth <[email protected]>
Now that all Avocado tests have been converted to or been replaced by other functional tests, we can delete the remainders of the Avocado tests from the QEMU source tree.
Signed-off-by Thomas Huth <[email protected]>--- MAINTAINERS | 8 +- docs/about/build-platforms.rst | 10 +- docs/devel/build-system.rst | 11 +- docs/devel/codebase.rst | 5 - docs/devel/testing/avocado.rst | 581 ------------------------ docs/devel/testing/functional.rst | 3 - docs/devel/testing/index.rst | 1 - docs/devel/testing/main.rst | 15 - tests/avocado/README.rst | 10 - configure | 2 +- pythondeps.toml | 8 +-
Quoted 8 lines
tests/Makefile.include | 60 +-- tests/avocado/avocado_qemu/__init__.py | 424 ----------------- tests/avocado/avocado_qemu/linuxtest.py | 253 ----------- 14 files changed, 16 insertions(+), 1375 deletions(-) delete mode 100644 docs/devel/testing/avocado.rst delete mode 100644 tests/avocado/README.rst delete mode 100644 tests/avocado/avocado_qemu/__init__.py delete mode 100644 tests/avocado/avocado_qemu/linuxtest.py
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|