Skip to contentSkip to conversation

Convert remaining Avocado tests to functional

PATCH Superseded Thomas Huth 3 participants 34 messages Mar 25, 2025 – Apr 14, 2025 Arm/Amlogic Meson SoC support

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover Convert remaining Avocado tests to functional

These patches convert the remaining Avocado tests bit by bit to the functional framework, or replace them with a test that has similar test coverage. This way we are finally able to get rid of the test/avocado folder in the end, everything is then integrated with the meson test runner. It should be possible now to run almost all tests in parallel by doing something like:

make -j$(nproc) check SPEED=thorough

Thomas Huth (15):
gitlab-ci: Remove the avocado tests from the CI pipelines tests/functional: Move the check for the parameters from avocado to
functional
tests/functional: Convert reverse_debugging tests to the functional
framework
tests/functional: Convert the i386 replay avocado test tests/avocado: Remove the LinuxKernelTest class tests/functional: Convert the 32-bit big endian Wheezy mips test tests/functional: Convert the 32-bit little endian Wheezy mips test tests/functional: Convert the 64-bit little endian Wheezy mips test tests/functional: Convert the 64-bit big endian Wheezy mips test tests/avocado: Remove the boot_linux.py tests tests/functional: Use the tuxrun kernel for the x86 replay test tests/functional: Use the tuxrun kernel for the aarch64 replay test tests/functional: Convert the SMMU test to the functional framework
gitlab-ci: Update QEMU_JOB_AVOCADO and QEMU_CI_AVOCADO_TESTING Remove the remainders of the Avocado tests

 MAINTAINERS                                   |  15 +-
 docs/about/build-platforms.rst                |  10 +-
 docs/devel/build-system.rst                   |  11 +-
 docs/devel/codebase.rst                       |   5 -
 docs/devel/testing/avocado.rst                | 581 ------------------
 docs/devel/testing/ci-jobs.rst.inc            |  19 +-
 docs/devel/testing/functional.rst             |   3 -
 docs/devel/testing/index.rst                  |   1 -
 docs/devel/testing/main.rst                   |  15 -
 tests/avocado/README.rst                      |  10 -
 configure                                     |   2 +-
 .gitlab-ci.d/base.yml                         |   8 +-
 .gitlab-ci.d/buildtest-template.yml           |  13 +-
 .gitlab-ci.d/buildtest.yml                    |  33 +-
 pythondeps.toml                               |   8 +-
 tests/Makefile.include                        |  60 +-
 tests/avocado/avocado_qemu/__init__.py        | 424 -------------
 tests/avocado/avocado_qemu/linuxtest.py       | 253 --------
 tests/avocado/boot_linux.py                   | 132 ----
 tests/avocado/boot_linux_console.py           |  96 ---
 tests/avocado/linux_ssh_mips_malta.py         | 205 ------
 tests/avocado/replay_kernel.py                | 110 ----
 tests/avocado/replay_linux.py                 | 206 -------
 tests/avocado/smmu.py                         | 139 -----
 tests/functional/meson.build                  |  13 +-
 tests/functional/qemu_test/tuxruntest.py      |   9 +-
 .../reverse_debugging.py                      | 114 +---
 tests/functional/test_aarch64_replay.py       |  37 +-
 .../functional/test_aarch64_reverse_debug.py  |  37 ++
 tests/functional/test_aarch64_smmu.py         | 205 ++++++
 tests/functional/test_i386_replay.py          |  28 +
 tests/functional/test_mips64_malta.py         |  35 ++
 tests/functional/test_mips64el_malta.py       |  22 +
 tests/functional/test_mips_malta.py           | 108 +++-
 tests/functional/test_mipsel_malta.py         |  22 +
 tests/functional/test_ppc64_reverse_debug.py  |  41 ++
 tests/functional/test_x86_64_replay.py        |  43 +-
 tests/functional/test_x86_64_reverse_debug.py |  36 ++
 38 files changed, 672 insertions(+), 2437 deletions(-)
 delete mode 100644 docs/devel/testing/avocado.rst
 delete mode 100644 tests/avocado/README.rst
 delete mode 100644 tests/avocado/avocado_qemu/__init__.py
 delete mode 100644 tests/avocado/avocado_qemu/linuxtest.py
 delete mode 100644 tests/avocado/boot_linux.py
 delete mode 100644 tests/avocado/boot_linux_console.py
 delete mode 100644 tests/avocado/linux_ssh_mips_malta.py
 delete mode 100644 tests/avocado/replay_kernel.py
 delete mode 100644 tests/avocado/replay_linux.py
 delete mode 100644 tests/avocado/smmu.py
 rename tests/{avocado => functional}/reverse_debugging.py (66%)
 create mode 100755 tests/functional/test_aarch64_reverse_debug.py
 create mode 100755 tests/functional/test_aarch64_smmu.py
 create mode 100755 tests/functional/test_i386_replay.py
 create mode 100755 tests/functional/test_mips64_malta.py
 create mode 100755 tests/functional/test_ppc64_reverse_debug.py
 create mode 100755 tests/functional/test_x86_64_reverse_debug.py
Signature · 1 line
2.49.0
#1 lore raw

PATCH 1/15 gitlab-ci: Remove the avocado tests from the CI pipelines

From: Thomas Huth <[email protected]>

We are going to move the remaining Avocado tests step by step into the functional test framework. Unfortunately, Avocado fails with an error if it cannot determine a test to run, so disable the tests here now to avoid failures in the Gitlab-CI during the next steps.

---
 .gitlab-ci.d/buildtest-template.yml | 11 ----------
 .gitlab-ci.d/buildtest.yml          | 33 +++++++++++------------------
 2 files changed, 12 insertions(+), 32 deletions(-)
.gitlab-ci.d/buildtest-template.yml +0 -11
index 39da7698b09..13fa4f4a4fc 100644
@@ -95,7 +95,6 @@
   cache:
     key: "${CI_JOB_NAME}-cache"
     paths:
-      - ${CI_PROJECT_DIR}/avocado-cache
       - ${CI_PROJECT_DIR}/functional-cache
     policy: pull-push
   artifacts:
@@ -109,16 +108,6 @@
     reports:
       junit: build/tests/results/latest/results.xml
   before_script:
-    - mkdir -p ~/.config/avocado
-    - echo "[datadir.paths]" > ~/.config/avocado/avocado.conf
-    - echo "cache_dirs = ['${CI_PROJECT_DIR}/avocado-cache']"
-           >> ~/.config/avocado/avocado.conf
-    - echo -e '[job.output.testlogs]\nstatuses = ["FAIL", "INTERRUPT"]'
-           >> ~/.config/avocado/avocado.conf
-    - if [ -d ${CI_PROJECT_DIR}/avocado-cache ]; then
-        du -chs ${CI_PROJECT_DIR}/*-cache ;
-      fi
-    - export AVOCADO_ALLOW_UNTRUSTED_CODE=1
     - export QEMU_TEST_ALLOW_UNTRUSTED_CODE=1
     - export QEMU_TEST_CACHE_DIR=${CI_PROJECT_DIR}/functional-cache
   after_script:
.gitlab-ci.d/buildtest.yml +12 -21
index 00f4bfcd9f3..431bc07d8fb 100644
@@ -29,8 +29,7 @@ functional-system-alpine:
       artifacts: true
   variables:
     IMAGE: alpine
-    MAKE_CHECK_ARGS: check-avocado check-functional
-    AVOCADO_TAGS: arch:avr arch:loongarch64 arch:mips64 arch:mipsel
+    MAKE_CHECK_ARGS: check-functional
 
 build-system-ubuntu:
   extends:
@@ -60,8 +59,7 @@ functional-system-ubuntu:
       artifacts: true
   variables:
     IMAGE: ubuntu2204
-    MAKE_CHECK_ARGS: check-avocado check-functional
-    AVOCADO_TAGS: arch:alpha arch:microblazeel arch:mips64el
+    MAKE_CHECK_ARGS: check-functional
 
 build-system-debian:
   extends:
@@ -92,8 +90,7 @@ functional-system-debian:
       artifacts: true
   variables:
     IMAGE: debian
-    MAKE_CHECK_ARGS: check-avocado check-functional
-    AVOCADO_TAGS: arch:arm arch:i386 arch:riscv64 arch:sh4 arch:sparc arch:xtensa
+    MAKE_CHECK_ARGS: check-functional
 
 crash-test-debian:
   extends: .native_test_job_template
@@ -155,9 +152,7 @@ functional-system-fedora:
       artifacts: true
   variables:
     IMAGE: fedora
-    MAKE_CHECK_ARGS: check-avocado check-functional
-    AVOCADO_TAGS: arch:microblaze arch:mips arch:xtensa arch:m68k
-      arch:riscv32 arch:ppc arch:sparc64
+    MAKE_CHECK_ARGS: check-functional
 
 crash-test-fedora:
   extends: .native_test_job_template
@@ -278,9 +273,7 @@ functional-system-centos:
       artifacts: true
   variables:
     IMAGE: centos9
-    MAKE_CHECK_ARGS: check-avocado check-functional
-    AVOCADO_TAGS: arch:ppc64 arch:or1k arch:s390x arch:x86_64 arch:rx
-      arch:sh4
+    MAKE_CHECK_ARGS: check-functional
 
 build-system-opensuse:
   extends:
@@ -309,8 +302,7 @@ functional-system-opensuse:
       artifacts: true
   variables:
     IMAGE: opensuse-leap
-    MAKE_CHECK_ARGS: check-avocado check-functional
-    AVOCADO_TAGS: arch:s390x arch:x86_64 arch:aarch64
+    MAKE_CHECK_ARGS: check-functional
 
 #
 # Flaky tests. We don't run these by default and they are allow fail
@@ -338,10 +330,9 @@ functional-system-flaky:
   allow_failure: true
   variables:
     IMAGE: debian
-    MAKE_CHECK_ARGS: check-avocado check-functional
+    MAKE_CHECK_ARGS: check-functional
     QEMU_JOB_OPTIONAL: 1
     QEMU_TEST_FLAKY_TESTS: 1
-    AVOCADO_TAGS: flaky
 
 # This jobs explicitly disable TCG (--disable-tcg), KVM is detected by
 # the configure script. The container doesn't contain Xen headers so
@@ -482,8 +473,8 @@ clang-user:
 # Since slirp callbacks are used in QEMU Timers, we cannot use libslirp with
 # CFI builds, and thus have to disable it here.
 #
-# Split in three sets of build/check/avocado to limit the execution time of each
-# job
+# Split in three sets of build/check/functional to limit the execution time
+# of each job
 build-cfi-aarch64:
   extends:
     - .native_build_job_template
@@ -520,7 +511,7 @@ functional-cfi-aarch64:
       artifacts: true
   variables:
     IMAGE: fedora
-    MAKE_CHECK_ARGS: check-avocado check-functional
+    MAKE_CHECK_ARGS: check-functional
 
 build-cfi-ppc64-s390x:
   extends:
@@ -558,7 +549,7 @@ functional-cfi-ppc64-s390x:
       artifacts: true
   variables:
     IMAGE: fedora
-    MAKE_CHECK_ARGS: check-avocado check-functional
+    MAKE_CHECK_ARGS: check-functional
 
 build-cfi-x86_64:
   extends:
@@ -592,7 +583,7 @@ functional-cfi-x86_64:
       artifacts: true
   variables:
     IMAGE: fedora
-    MAKE_CHECK_ARGS: check-avocado check-functional
+    MAKE_CHECK_ARGS: check-functional
 
 tsan-build:
   extends: .native_build_job_template
Signature · 1 line
2.49.0
Reviewed-by on v1 1/15
#2 lore raw

Re: [PATCH 01/15] gitlab-ci: Remove the avocado tests from the CI pipelines

Quoted 8 lines · Thomas Huth

From: Thomas Huth <[email protected]>

We are going to move the remaining Avocado tests step by step into the functional test framework. Unfortunately, Avocado fails with an error if it cannot determine a test to run, so disable the tests here now to avoid failures in the Gitlab-CI during the next steps.

Quoted 5 lines
---
 .gitlab-ci.d/buildtest-template.yml | 11 ----------
 .gitlab-ci.d/buildtest.yml          | 33 +++++++++++------------------
 2 files changed, 12 insertions(+), 32 deletions(-)

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#3 lore raw

PATCH 2/15 tests/functional: Move the check for the parameters from avocado to functional

In reply to Thomas Huth

From: Thomas Huth <[email protected]>

test_x86_64_pc in tests/avocado/boot_linux_console.py only checks whether the kernel parameters have correctly been passed to the kernel in the guest by looking for them in the console output of the guest. Let's move that to the functional test framework now, but instead of doing it in a separate test, let's do it for all tuxrun tests instead, so it is done automatically for all targets that have a tuxrun test.

---
 tests/avocado/boot_linux_console.py      | 34 ------------------------
 tests/functional/qemu_test/tuxruntest.py |  9 ++++---
 2 files changed, 5 insertions(+), 38 deletions(-)
tests/avocado/boot_linux_console.py +0 -34
index c15f39ae1f3..cbb1e2fb506 100644
@@ -9,17 +9,9 @@
 # later.  See the COPYING file in the top-level directory.
 
 import os
-import lzma
-import gzip
 import shutil
 
-from avocado import skip
-from avocado import skipUnless
-from avocado import skipUnless
 from avocado_qemu import QemuSystemTest
-from avocado_qemu import exec_command
-from avocado_qemu import exec_command_and_wait_for_pattern
-from avocado_qemu import interrupt_interactive_console_until_pattern
 from avocado_qemu import wait_for_console_pattern
 from avocado.utils import process
 from avocado.utils import archive
@@ -68,29 +60,3 @@ def extract_from_rpm(self, rpm, path):
         process.run("rpm2cpio %s | cpio -id %s" % (rpm, path), shell=True)
         os.chdir(cwd)
         return os.path.normpath(os.path.join(self.workdir, path))
-
-class BootLinuxConsole(LinuxKernelTest):
-    """
-    Boots a Linux kernel and checks that the console is operational and the
-    kernel command line is properly passed from QEMU to the kernel
-    """
-    timeout = 90
-
-    def test_x86_64_pc(self):
-        """
-        :avocado: tags=arch:x86_64
-        :avocado: tags=machine:pc
-        """
-        kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
-                      '/linux/releases/29/Everything/x86_64/os/images/pxeboot'
-                      '/vmlinuz')
-        kernel_hash = '23bebd2680757891cf7adedb033532163a792495'
-        kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
-
-        self.vm.set_console()
-        kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE + 'console=ttyS0'
-        self.vm.add_args('-kernel', kernel_path,
-                         '-append', kernel_command_line)
-        self.vm.launch()
-        console_pattern = 'Kernel command line: %s' % kernel_command_line
-        self.wait_for_console_pattern(console_pattern)
tests/functional/qemu_test/tuxruntest.py +5 -4
index ad74156f9c5..c2bd5baaae9 100644
@@ -77,12 +77,12 @@ def prepare_run(self, kernel, disk, drive, dtb=None, console_index=0):
         blockdev = "driver=raw,file.driver=file," \
             + f"file.filename={disk},node-name=hd0"
 
-        kcmd_line = self.KERNEL_COMMON_COMMAND_LINE
-        kcmd_line += f" root=/dev/{self.root}"
-        kcmd_line += f" console={self.console}"
+        self.kcmd_line = self.KERNEL_COMMON_COMMAND_LINE
+        self.kcmd_line += f" root=/dev/{self.root}"
+        self.kcmd_line += f" console={self.console}"
 
         self.vm.add_args('-kernel', kernel,
-                         '-append', kcmd_line,
+                         '-append', self.kcmd_line,
                          '-blockdev', blockdev)
 
         # Sometimes we need extra devices attached
@@ -103,6 +103,7 @@ def run_tuxtest_tests(self, haltmsg):
         wait to exit cleanly.
         """
         ps1='root@tuxtest:~#'
+        self.wait_for_console_pattern(self.kcmd_line)
         self.wait_for_console_pattern('tuxtest login:')
         exec_command_and_wait_for_pattern(self, 'root', ps1)
         exec_command_and_wait_for_pattern(self, 'cat /proc/interrupts', ps1)
Signature · 1 line
2.49.0
Reviewed-by on v1 2/15
#4 lore raw

Re: [PATCH 02/15] tests/functional: Move the check for the parameters from avocado to functional

Quoted 10 lines · Thomas Huth

From: Thomas Huth <[email protected]>

test_x86_64_pc in tests/avocado/boot_linux_console.py only checks whether the kernel parameters have correctly been passed to the kernel in the guest by looking for them in the console output of the guest. Let's move that to the functional test framework now, but instead of doing it in a separate test, let's do it for all tuxrun tests instead, so it is done automatically for all targets that have a tuxrun test.

Quoted 5 lines
---
 tests/avocado/boot_linux_console.py      | 34 ------------------------
 tests/functional/qemu_test/tuxruntest.py |  9 ++++---
 2 files changed, 5 insertions(+), 38 deletions(-)

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#5 lore raw

PATCH 3/15 tests/functional: Convert reverse_debugging tests to the functional framework

In reply to Thomas Huth

From: Thomas Huth <[email protected]>

These tests are using the gdb-related library functions from the Avocado framework which we don't have in the functional framework yet. So for the time being, keep those imports and skip the test if the Avocado framework is not installed on the host.

---
 MAINTAINERS                                   |   2 +-
 tests/functional/meson.build                  |   4 +
 .../reverse_debugging.py                      | 114 +++---------------
 .../functional/test_aarch64_reverse_debug.py  |  37 ++++++
 tests/functional/test_ppc64_reverse_debug.py  |  41 +++++++
 tests/functional/test_x86_64_reverse_debug.py |  36 ++++++
 6 files changed, 138 insertions(+), 96 deletions(-)
 rename tests/{avocado => functional}/reverse_debugging.py (66%)
 create mode 100755 tests/functional/test_aarch64_reverse_debug.py
 create mode 100755 tests/functional/test_ppc64_reverse_debug.py
 create mode 100755 tests/functional/test_x86_64_reverse_debug.py
MAINTAINERS +1 -1
index 8f470a1c9b7..73ccf5e5176 100644
@@ -3670,7 +3670,7 @@ F: docs/system/replay.rst
 F: stubs/replay.c
 F: tests/avocado/replay_kernel.py
 F: tests/avocado/replay_linux.py
-F: tests/avocado/reverse_debugging.py
+F: tests/functional/*reverse_debug*.py
 F: tests/functional/*replay*.py
 F: qapi/replay.json
 
tests/functional/meson.build +4 -0
index 96d28289279..bcb4f50a62f 100644
@@ -13,6 +13,7 @@ endif
 test_timeouts = {
   'aarch64_aspeed' : 600,
   'aarch64_raspi4' : 480,
+  'aarch64_reverse_debug' : 180,
   'aarch64_rme_virt' : 1200,
   'aarch64_rme_sbsaref' : 1200,
   'aarch64_sbsaref_alpine' : 1200,
@@ -78,6 +79,7 @@ tests_aarch64_system_thorough = [
   'aarch64_raspi3',
   'aarch64_raspi4',
   'aarch64_replay',
+  'aarch64_reverse_debug',
   'aarch64_rme_virt',
   'aarch64_rme_sbsaref',
   'aarch64_sbsaref',
@@ -229,6 +231,7 @@ tests_ppc64_system_thorough = [
   'ppc64_powernv',
   'ppc64_pseries',
   'ppc64_replay',
+  'ppc64_reverse_debug',
   'ppc64_tuxrun',
   'ppc64_mac99',
 ]
@@ -311,6 +314,7 @@ tests_x86_64_system_thorough = [
   'x86_64_hotplug_cpu',
   'x86_64_kvm_xen',
   'x86_64_replay',
+  'x86_64_reverse_debug',
   'x86_64_tuxrun',
 ]
 
tests/functional/reverse_debugging.py renamed from tests/avocado/reverse_debugging.py +19 -95

158 lines not on this page · raw message

tests/functional/test_aarch64_reverse_debug.py new file +37 -0
new file mode 100755
index 00000000000..82925bf5908
@@ -0,0 +1,37 @@
+#!/usr/bin/env python3
+#
+# SPDX-License-Identifier: GPL-2.0-or-later
+#
+# Reverse debugging test
+#
+# Copyright (c) 2020 ISP RAS
+#
+# Author:
+#  Pavel Dovgalyuk <[email protected]>
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later.  See the COPYING file in the top-level directory.
+
+from qemu_test import Asset, skipIfMissingImports
+from reverse_debugging import ReverseDebugging
+
+
+@skipIfMissingImports('avocado')
+class ReverseDebugging_AArch64(ReverseDebugging):
+
+    REG_PC = 32
+
+    KERNEL_ASSET = Asset(
+        ('https://archives.fedoraproject.org/pub/archive/fedora/linux/'
+         'releases/29/Everything/aarch64/os/images/pxeboot/vmlinuz'),
+        '7e1430b81c26bdd0da025eeb8fbd77b5dc961da4364af26e771bd39f379cbbf7')
+
+    def test_aarch64_virt(self):
+        self.set_machine('virt')
+        self.cpu = 'cortex-a53'
+        kernel_path = self.KERNEL_ASSET.fetch()
+        self.reverse_debugging(args=('-kernel', kernel_path))
+
+
+if __name__ == '__main__':
+    ReverseDebugging.main()
tests/functional/test_ppc64_reverse_debug.py new file +41 -0

41 lines not on this page · raw message

tests/functional/test_x86_64_reverse_debug.py new file +36 -0
new file mode 100755
index 00000000000..aba31f68748
@@ -0,0 +1,36 @@
+#!/usr/bin/env python3
+#
+# SPDX-License-Identifier: GPL-2.0-or-later
+#
+# Reverse debugging test
+#
+# Copyright (c) 2020 ISP RAS
+#
+# Author:
+#  Pavel Dovgalyuk <[email protected]>
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later.  See the COPYING file in the top-level directory.
+
+from qemu_test import skipIfMissingImports, skipFlakyTest
+from reverse_debugging import ReverseDebugging
+
+
+@skipIfMissingImports('avocado')
+class ReverseDebugging_X86_64(ReverseDebugging):
+
+    REG_PC = 0x10
+    REG_CS = 0x12
+    def get_pc(self, g):
+        return self.get_reg_le(g, self.REG_PC) \
+            + self.get_reg_le(g, self.REG_CS) * 0x10
+
+    @skipFlakyTest("https://gitlab.com/qemu-project/qemu/-/issues/1992")
+    def test_x86_64_pc(self):
+        self.set_machine('pc')
+        # start with BIOS only
+        self.reverse_debugging()
+
+
+if __name__ == '__main__':
+    ReverseDebugging.main()
Signature · 1 line
2.49.0
#6 lore raw

Re: [PATCH 03/15] tests/functional: Convert reverse_debugging tests to the functional framework

Quoted 50 lines · Thomas Huth

From: Thomas Huth <[email protected]>

These tests are using the gdb-related library functions from the Avocado framework which we don't have in the functional framework yet. So for the time being, keep those imports and skip the test if the Avocado framework is not installed on the host.

---
 MAINTAINERS                                   |   2 +-
 tests/functional/meson.build                  |   4 +
 .../reverse_debugging.py                      | 114 +++---------------
 .../functional/test_aarch64_reverse_debug.py  |  37 ++++++
 tests/functional/test_ppc64_reverse_debug.py  |  41 +++++++
 tests/functional/test_x86_64_reverse_debug.py |  36 ++++++
 6 files changed, 138 insertions(+), 96 deletions(-)
 rename tests/{avocado => functional}/reverse_debugging.py (66%)
 create mode 100755 tests/functional/test_aarch64_reverse_debug.py
 create mode 100755 tests/functional/test_ppc64_reverse_debug.py
 create mode 100755 tests/functional/test_x86_64_reverse_debug.py
MAINTAINERS +1 -1
index 8f470a1c9b7..73ccf5e5176 100644
@@ -3670,7 +3670,7 @@ F: docs/system/replay.rst
 F: stubs/replay.c
 F: tests/avocado/replay_kernel.py
 F: tests/avocado/replay_linux.py
-F: tests/avocado/reverse_debugging.py
+F: tests/functional/*reverse_debug*.py
 F: tests/functional/*replay*.py
 F: qapi/replay.json
 
tests/functional/test_aarch64_reverse_debug.py new file +9 -0
new file mode 100755
index 00000000000..82925bf5908
@@ -0,0 +1,37 @@
+#!/usr/bin/env python3
+#
+# SPDX-License-Identifier: GPL-2.0-or-later
+#
+# Reverse debugging test
+#
+# Copyright (c) 2020 ISP RAS
+#
+# Author:

tests/functional/test_aarch64_reverse_debug.py line 19

Quoted 10 lines
tests/functional/test_aarch64_reverse_debug.py new file +10 -0
+#  Pavel Dovgalyuk <[email protected]>
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later.  See the COPYING file in the top-level directory.
+
+from qemu_test import Asset, skipIfMissingImports
+from reverse_debugging import ReverseDebugging
+
+
+@skipIfMissingImports('avocado')

Would it make sense to specialize this to 'avocado.utils' ?

Quoted 43 lines
tests/functional/test_aarch64_reverse_debug.py new file +18 -0
+class ReverseDebugging_AArch64(ReverseDebugging):
+
+    REG_PC = 32
+
+    KERNEL_ASSET = Asset(
+        ('https://archives.fedoraproject.org/pub/archive/fedora/linux/'
+         'releases/29/Everything/aarch64/os/images/pxeboot/vmlinuz'),
+        '7e1430b81c26bdd0da025eeb8fbd77b5dc961da4364af26e771bd39f379cbbf7')
+
+    def test_aarch64_virt(self):
+        self.set_machine('virt')
+        self.cpu = 'cortex-a53'
+        kernel_path = self.KERNEL_ASSET.fetch()
+        self.reverse_debugging(args=('-kernel', kernel_path))
+
+
+if __name__ == '__main__':
+    ReverseDebugging.main()
tests/functional/test_x86_64_reverse_debug.py new file +18 -0
new file mode 100755
index 00000000000..aba31f68748
@@ -0,0 +1,36 @@
+#!/usr/bin/env python3
+#
+# SPDX-License-Identifier: GPL-2.0-or-later
+#
+# Reverse debugging test
+#
+# Copyright (c) 2020 ISP RAS
+#
+# Author:
+#  Pavel Dovgalyuk <[email protected]>
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later.  See the COPYING file in the top-level directory.
+
+from qemu_test import skipIfMissingImports, skipFlakyTest
+from reverse_debugging import ReverseDebugging
+
+

tests/functional/test_x86_64_reverse_debug.py line 28

Quoted 10 lines
tests/functional/test_x86_64_reverse_debug.py new file +10 -0
+@skipIfMissingImports('avocado')
+class ReverseDebugging_X86_64(ReverseDebugging):
+
+    REG_PC = 0x10
+    REG_CS = 0x12
+    def get_pc(self, g):
+        return self.get_reg_le(g, self.REG_PC) \
+            + self.get_reg_le(g, self.REG_CS) * 0x10
+
+    @skipFlakyTest("https://gitlab.com/qemu-project/qemu/-/issues/1992")

This bug refers to ppc64. Was this a copy and paste mistake, does that bug need updating to also mention x86_64 ?

Quoted 11 lines
tests/functional/test_x86_64_reverse_debug.py new file +8 -0
+    def test_x86_64_pc(self):
+        self.set_machine('pc')
+        # start with BIOS only
+        self.reverse_debugging()
+
+
+if __name__ == '__main__':
+    ReverseDebugging.main()
Signature · 1 line
2.49.0

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#7 lore raw

Re: [PATCH 03/15] tests/functional: Convert reverse_debugging tests to the functional framework

Quoted 55 lines · Daniel P. Berrangé
Quoted 54 lines · Thomas Huth

From: Thomas Huth <[email protected]>

These tests are using the gdb-related library functions from the Avocado framework which we don't have in the functional framework yet. So for the time being, keep those imports and skip the test if the Avocado framework is not installed on the host.

---
  MAINTAINERS                                   |   2 +-
  tests/functional/meson.build                  |   4 +
  .../reverse_debugging.py                      | 114 +++---------------
  .../functional/test_aarch64_reverse_debug.py  |  37 ++++++
  tests/functional/test_ppc64_reverse_debug.py  |  41 +++++++
  tests/functional/test_x86_64_reverse_debug.py |  36 ++++++
  6 files changed, 138 insertions(+), 96 deletions(-)

rename tests/{avocado => functional}/reverse_debugging.py (66%) create mode 100755 tests/functional/test_aarch64_reverse_debug.py create mode 100755 tests/functional/test_ppc64_reverse_debug.py create mode 100755 tests/functional/test_x86_64_reverse_debug.py

MAINTAINERS +1 -1
index 8f470a1c9b7..73ccf5e5176 100644
@@ -3670,7 +3670,7 @@ F: docs/system/replay.rst
  F: stubs/replay.c
  F: tests/avocado/replay_kernel.py
  F: tests/avocado/replay_linux.py
-F: tests/avocado/reverse_debugging.py
+F: tests/functional/*reverse_debug*.py
  F: tests/functional/*replay*.py
  F: qapi/replay.json
 
tests/functional/test_aarch64_reverse_debug.py new file +13 -0
new file mode 100755
index 00000000000..82925bf5908
@@ -0,0 +1,37 @@
+#!/usr/bin/env python3
+#
+# SPDX-License-Identifier: GPL-2.0-or-later
+#
+# Reverse debugging test
+#
+# Copyright (c) 2020 ISP RAS
+#
+# Author:
+#  Pavel Dovgalyuk <[email protected]>
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later.  See the COPYING file in the top-level directory.
Quoted 8 lines

tests/functional/test_aarch64_reverse_debug.py line 19

Quoted 6 lines
tests/functional/test_aarch64_reverse_debug.py new file +6 -0
+
+from qemu_test import Asset, skipIfMissingImports
+from reverse_debugging import ReverseDebugging
+
+
+@skipIfMissingImports('avocado')

Would it make sense to specialize this to 'avocado.utils' ?

Sure, can do!

Quoted 48 lines
Quoted 48 lines
tests/functional/test_aarch64_reverse_debug.py new file +18 -0
+class ReverseDebugging_AArch64(ReverseDebugging):
+
+    REG_PC = 32
+
+    KERNEL_ASSET = Asset(
+        ('https://archives.fedoraproject.org/pub/archive/fedora/linux/'
+         'releases/29/Everything/aarch64/os/images/pxeboot/vmlinuz'),
+        '7e1430b81c26bdd0da025eeb8fbd77b5dc961da4364af26e771bd39f379cbbf7')
+
+    def test_aarch64_virt(self):
+        self.set_machine('virt')
+        self.cpu = 'cortex-a53'
+        kernel_path = self.KERNEL_ASSET.fetch()
+        self.reverse_debugging(args=('-kernel', kernel_path))
+
+
+if __name__ == '__main__':
+    ReverseDebugging.main()
tests/functional/test_x86_64_reverse_debug.py new file +23 -0
new file mode 100755
index 00000000000..aba31f68748
@@ -0,0 +1,36 @@
+#!/usr/bin/env python3
+#
+# SPDX-License-Identifier: GPL-2.0-or-later
+#
+# Reverse debugging test
+#
+# Copyright (c) 2020 ISP RAS
+#
+# Author:
+#  Pavel Dovgalyuk <[email protected]>
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later.  See the COPYING file in the top-level directory.
+
+from qemu_test import skipIfMissingImports, skipFlakyTest
+from reverse_debugging import ReverseDebugging
+
+
+@skipIfMissingImports('avocado')
+class ReverseDebugging_X86_64(ReverseDebugging):
+
+    REG_PC = 0x10
+    REG_CS = 0x12
Quoted 8 lines

tests/functional/test_x86_64_reverse_debug.py line 28

Quoted 5 lines
tests/functional/test_x86_64_reverse_debug.py new file +5 -0
+    def get_pc(self, g):
+        return self.get_reg_le(g, self.REG_PC) \
+            + self.get_reg_le(g, self.REG_CS) * 0x10
+
+    @skipFlakyTest("https://gitlab.com/qemu-project/qemu/-/issues/1992")

This bug refers to ppc64. Was this a copy and paste mistake, does that bug need updating to also mention x86_64 ?

I'm not sure whether it's the same bug, so I opened another one for x86 here:

https://gitlab.com/qemu-project/qemu/-/issues/2922

I also noticed that it fails on aarch64 in some rare cases, and in a different way. So here's another bug for aarch64:

https://gitlab.com/qemu-project/qemu/-/issues/2921

I'll use those in the decorators in the next version of the patch.

Thomas

#8 lore raw

PATCH 4/15 tests/functional: Convert the i386 replay avocado test

In reply to Thomas Huth

From: Thomas Huth <[email protected]>

Since this was the last test in tests/avocado/replay_kernel.py, we can remove that Avocado file now.

---
 MAINTAINERS                          |   1 -
 tests/avocado/replay_kernel.py       | 110 ---------------------------
 tests/functional/meson.build         |   1 +
 tests/functional/test_i386_replay.py |  28 +++++++
 4 files changed, 29 insertions(+), 111 deletions(-)
 delete mode 100644 tests/avocado/replay_kernel.py
 create mode 100755 tests/functional/test_i386_replay.py
MAINTAINERS +0 -1
index 73ccf5e5176..4e1d816f2b4 100644
@@ -3668,7 +3668,6 @@ F: include/system/replay.h
 F: docs/devel/replay.rst
 F: docs/system/replay.rst
 F: stubs/replay.c
-F: tests/avocado/replay_kernel.py
 F: tests/avocado/replay_linux.py
 F: tests/functional/*reverse_debug*.py
 F: tests/functional/*replay*.py
tests/avocado/replay_kernel.py deleted +0 -110

110 lines not on this page · raw message

tests/functional/meson.build +1 -0
index bcb4f50a62f..452d863f187 100644
@@ -151,6 +151,7 @@ tests_i386_system_quick = [
 ]
 
 tests_i386_system_thorough = [
+  'i386_replay',
   'i386_tuxrun',
 ]
 
tests/functional/test_i386_replay.py new file +28 -0
new file mode 100755
index 00000000000..7c4c2602da9
@@ -0,0 +1,28 @@
+#!/usr/bin/env python3
+#
+# Replay test that boots a Linux kernel on a i386 machine
+# and checks the console
+#
+# SPDX-License-Identifier: GPL-2.0-or-later
+
+from qemu_test import Asset
+from replay_kernel import ReplayKernelBase
+
+
+class I386Replay(ReplayKernelBase):
+
+    ASSET_KERNEL = Asset(
+         'https://storage.tuxboot.com/20230331/i386/bzImage',
+        'a3e5b32a354729e65910f5a1ffcda7c14a6c12a55e8213fb86e277f1b76ed956')
+
+    def test_pc(self):
+        self.set_machine('pc')
+        kernel_url = ()
+        kernel_path = self.ASSET_KERNEL.fetch()
+        kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE + 'console=ttyS0'
+        console_pattern = 'VFS: Cannot open root device'
+        self.run_rr(kernel_path, kernel_command_line, console_pattern, shift=5)
+
+
+if __name__ == '__main__':
+    ReplayKernelBase.main()
Signature · 1 line
2.49.0
Reviewed-by on v1 4/15
#9 lore raw

Re: [PATCH 04/15] tests/functional: Convert the i386 replay avocado test

Quoted 6 lines · Thomas Huth

From: Thomas Huth <[email protected]>

Since this was the last test in tests/avocado/replay_kernel.py, we can remove that Avocado file now.

Quoted 8 lines
---
 MAINTAINERS                          |   1 -
 tests/avocado/replay_kernel.py       | 110 ---------------------------
 tests/functional/meson.build         |   1 +
 tests/functional/test_i386_replay.py |  28 +++++++
 4 files changed, 29 insertions(+), 111 deletions(-)
 delete mode 100644 tests/avocado/replay_kernel.py
 create mode 100755 tests/functional/test_i386_replay.py

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#10 lore raw

PATCH 5/15 tests/avocado: Remove the LinuxKernelTest class

In reply to Thomas Huth

From: Thomas Huth <[email protected]>

All tests that used this class have been converted to the functional framework, so we can remove the boot_linux_console.py file now.

---
 tests/avocado/boot_linux_console.py | 62 -----------------------------
 1 file changed, 62 deletions(-)
 delete mode 100644 tests/avocado/boot_linux_console.py
tests/avocado/boot_linux_console.py deleted +0 -62
deleted file mode 100644
index cbb1e2fb506..00000000000
@@ -1,62 +0,0 @@
-# Functional test that boots a Linux kernel and checks the console
-#
-# Copyright (c) 2018 Red Hat, Inc.
-#
-# Author:
-#  Cleber Rosa <[email protected]>
-#
-# This work is licensed under the terms of the GNU GPL, version 2 or
-# later.  See the COPYING file in the top-level directory.
-
-import os
-import shutil
-
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import wait_for_console_pattern
-from avocado.utils import process
-from avocado.utils import archive
-
-class LinuxKernelTest(QemuSystemTest):
-    KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 '
-
-    def wait_for_console_pattern(self, success_message, vm=None):
-        wait_for_console_pattern(self, success_message,
-                                 failure_message='Kernel panic - not syncing',
-                                 vm=vm)
-
-    def extract_from_deb(self, deb, path):
-        """
-        Extracts a file from a deb package into the test workdir
-
-        :param deb: path to the deb archive
-        :param path: path within the deb archive of the file to be extracted
-        :returns: path of the extracted file
-        """
-        cwd = os.getcwd()
-        os.chdir(self.workdir)
-        file_path = process.run("ar t %s" % deb).stdout_text.split()[2]
-        process.run("ar x %s %s" % (deb, file_path))
-        archive.extract(file_path, self.workdir)
-        os.chdir(cwd)
-        # Return complete path to extracted file.  Because callers to
-        # extract_from_deb() specify 'path' with a leading slash, it is
-        # necessary to use os.path.relpath() as otherwise os.path.join()
-        # interprets it as an absolute path and drops the self.workdir part.
-        return os.path.normpath(os.path.join(self.workdir,
-                                             os.path.relpath(path, '/')))
-
-    def extract_from_rpm(self, rpm, path):
-        """
-        Extracts a file from an RPM package into the test workdir.
-
-        :param rpm: path to the rpm archive
-        :param path: path within the rpm archive of the file to be extracted
-                     needs to be a relative path (starting with './') because
-                     cpio(1), which is used to extract the file, expects that.
-        :returns: path of the extracted file
-        """
-        cwd = os.getcwd()
-        os.chdir(self.workdir)
-        process.run("rpm2cpio %s | cpio -id %s" % (rpm, path), shell=True)
-        os.chdir(cwd)
-        return os.path.normpath(os.path.join(self.workdir, path))
Signature · 1 line
2.49.0
Reviewed-by on v1 5/15
#11 lore raw

Re: [PATCH 05/15] tests/avocado: Remove the LinuxKernelTest class

Quoted 10 lines · Thomas Huth

From: Thomas Huth <[email protected]>

All tests that used this class have been converted to the functional framework, so we can remove the boot_linux_console.py file now.

---
 tests/avocado/boot_linux_console.py | 62 -----------------------------
 1 file changed, 62 deletions(-)
 delete mode 100644 tests/avocado/boot_linux_console.py

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#12 lore raw

PATCH 6/15 tests/functional: Convert the 32-bit big endian Wheezy mips test

In reply to Thomas Huth

From: Thomas Huth <[email protected]>

The test checks some entries in /proc and the output of some commands ... we put these checks into exportable functions now so that they can be reused more easily.

Additionally the linux_ssh_mips_malta.py uses SSH to test the networking of the guest. Since we don't have a SSH module in the functional framework yet, let's use the check_http_download() function here instead.

And while we're at it, also switch the NIC to e1000 now to get some more test coverage, since the "pcnet" device is already tested in the test test_mips_malta_cpio.

---
 tests/avocado/linux_ssh_mips_malta.py |   8 --
 tests/functional/meson.build          |   2 +-
 tests/functional/test_mips_malta.py   | 108 +++++++++++++++++++++++++-
 3 files changed, 107 insertions(+), 11 deletions(-)
tests/avocado/linux_ssh_mips_malta.py +0 -8
index d9bb525ad9c..73d294ad5cf 100644
@@ -172,14 +172,6 @@ def check_mips_malta(self, uname_m, endianess):
         # Wait for VM to shut down gracefully
         self.vm.wait()
 
-    def test_mips_malta32eb_kernel3_2_0(self):
-        """
-        :avocado: tags=arch:mips
-        :avocado: tags=endian:big
-        :avocado: tags=device:pcnet32
-        """
-        self.check_mips_malta('mips', 'be')
-
     def test_mips_malta32el_kernel3_2_0(self):
         """
         :avocado: tags=arch:mipsel
tests/functional/meson.build +1 -1
index 452d863f187..3f8fabd218d 100644
@@ -39,7 +39,7 @@ test_timeouts = {
   'arm_tuxrun' : 240,
   'arm_sx1' : 360,
   'intel_iommu': 300,
-  'mips_malta' : 120,
+  'mips_malta' : 480,
   'mipsel_replay' : 480,
   'mips64el_replay' : 180,
   'netdev_ethtool' : 180,
tests/functional/test_mips_malta.py +106 -2

129 lines not on this page · raw message

Signature · 1 line
2.49.0
Reviewed-by on v1 6/15
#13 lore raw

Re: [PATCH 06/15] tests/functional: Convert the 32-bit big endian Wheezy mips test

Quoted 14 lines · Thomas Huth

From: Thomas Huth <[email protected]>

The test checks some entries in /proc and the output of some commands ... we put these checks into exportable functions now so that they can be reused more easily.

Additionally the linux_ssh_mips_malta.py uses SSH to test the networking of the guest. Since we don't have a SSH module in the functional framework yet, let's use the check_http_download() function here instead.

And while we're at it, also switch the NIC to e1000 now to get some more test coverage, since the "pcnet" device is already tested in the test test_mips_malta_cpio.

Quoted 6 lines
---
 tests/avocado/linux_ssh_mips_malta.py |   8 --
 tests/functional/meson.build          |   2 +-
 tests/functional/test_mips_malta.py   | 108 +++++++++++++++++++++++++-
 3 files changed, 107 insertions(+), 11 deletions(-)

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#14 lore raw

PATCH 7/15 tests/functional: Convert the 32-bit little endian Wheezy mips test

In reply to Thomas Huth

From: Thomas Huth <[email protected]>

Reuse the test function from the big endian test to easily convert the 32-bit little endian Wheezy mips test.

---
 tests/avocado/linux_ssh_mips_malta.py |  8 --------
 tests/functional/meson.build          |  1 +
 tests/functional/test_mipsel_malta.py | 22 ++++++++++++++++++++++
 3 files changed, 23 insertions(+), 8 deletions(-)
tests/avocado/linux_ssh_mips_malta.py +0 -8
index 73d294ad5cf..c1300aec961 100644
@@ -172,14 +172,6 @@ def check_mips_malta(self, uname_m, endianess):
         # Wait for VM to shut down gracefully
         self.vm.wait()
 
-    def test_mips_malta32el_kernel3_2_0(self):
-        """
-        :avocado: tags=arch:mipsel
-        :avocado: tags=endian:little
-        :avocado: tags=device:pcnet32
-        """
-        self.check_mips_malta('mips', 'le')
-
     def test_mips_malta64eb_kernel3_2_0(self):
         """
         :avocado: tags=arch:mips64
tests/functional/meson.build +1 -0
index 3f8fabd218d..c7bd4eb4edf 100644
@@ -40,6 +40,7 @@ test_timeouts = {
   'arm_sx1' : 360,
   'intel_iommu': 300,
   'mips_malta' : 480,
+  'mipsel_malta' : 420,
   'mipsel_replay' : 480,
   'mips64el_replay' : 180,
   'netdev_ethtool' : 180,
tests/functional/test_mipsel_malta.py +22 -0
index fe9c3a172e6..9ee2884da8e 100755
@@ -13,6 +13,8 @@
 from qemu_test import interrupt_interactive_console_until_pattern
 from qemu_test import wait_for_console_pattern
 
+from test_mips_malta import mips_check_wheezy
+
 
 class MaltaMachineConsole(LinuxKernelTest):
 
@@ -57,6 +59,26 @@ def test_mips_malta32el_nanomips_16k_up(self):
     def test_mips_malta32el_nanomips_64k_dbg(self):
         self.do_test_mips_malta32el_nanomips(self.ASSET_KERNEL_64K)
 
+    ASSET_WHEEZY_KERNEL = Asset(
+        ('https://people.debian.org/~aurel32/qemu/mipsel/'
+         'vmlinux-3.2.0-4-4kc-malta'),
+        'dc8a3648305b0201ca7a5cd135fe2890067a65d93c38728022bb0e656ad2bf9a')
+
+    ASSET_WHEEZY_DISK = Asset(
+        ('https://people.debian.org/~aurel32/qemu/mipsel/'
+         'debian_wheezy_mipsel_standard.qcow2'),
+        '454f09ae39f7e6461c84727b927100d2c7813841f2a0a5dce328114887ecf914')
+
+    def test_wheezy(self):
+        kernel_path = self.ASSET_WHEEZY_KERNEL.fetch()
+        image_path = self.ASSET_WHEEZY_DISK.fetch()
+        kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE
+                               + 'console=ttyS0 root=/dev/sda1')
+        mips_check_wheezy(self,
+            kernel_path, image_path, kernel_command_line,
+            dl_file='/boot/initrd.img-3.2.0-4-4kc-malta',
+            hsum='9fc9f250ed56a74e35e704ddfd5a1c5a5625adefc5c9da91f649288d3ca000f0')
+
 
 class MaltaMachineYAMON(QemuSystemTest):
 
Signature · 1 line
2.49.0
Reviewed-by on v1 7/15
#15 lore raw

Re: [PATCH 07/15] tests/functional: Convert the 32-bit little endian Wheezy mips test

Quoted 11 lines · Thomas Huth

From: Thomas Huth <[email protected]>

Reuse the test function from the big endian test to easily convert the 32-bit little endian Wheezy mips test.

---
 tests/avocado/linux_ssh_mips_malta.py |  8 --------
 tests/functional/meson.build          |  1 +
 tests/functional/test_mipsel_malta.py | 22 ++++++++++++++++++++++
 3 files changed, 23 insertions(+), 8 deletions(-)

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#16 lore raw

PATCH 8/15 tests/functional: Convert the 64-bit little endian Wheezy mips test

In reply to Thomas Huth

From: Thomas Huth <[email protected]>

Reuse the test function from the 32-bit big endian test to easily convert the 64-bit little endian Wheezy mips test.

---
 tests/avocado/linux_ssh_mips_malta.py   |  8 --------
 tests/functional/meson.build            |  1 +
 tests/functional/test_mips64el_malta.py | 22 ++++++++++++++++++++++
 3 files changed, 23 insertions(+), 8 deletions(-)
tests/avocado/linux_ssh_mips_malta.py +0 -8
index c1300aec961..2fa5cf9a6c1 100644
@@ -179,11 +179,3 @@ def test_mips_malta64eb_kernel3_2_0(self):
         :avocado: tags=device:pcnet32
         """
         self.check_mips_malta('mips64', 'be')
-
-    def test_mips_malta64el_kernel3_2_0(self):
-        """
-        :avocado: tags=arch:mips64el
-        :avocado: tags=endian:little
-        :avocado: tags=device:pcnet32
-        """
-        self.check_mips_malta('mips64', 'le')
tests/functional/meson.build +1 -0
index c7bd4eb4edf..63876bc7c42 100644
@@ -42,6 +42,7 @@ test_timeouts = {
   'mips_malta' : 480,
   'mipsel_malta' : 420,
   'mipsel_replay' : 480,
+  'mips64el_malta' : 420,
   'mips64el_replay' : 180,
   'netdev_ethtool' : 180,
   'ppc_40p' : 240,
tests/functional/test_mips64el_malta.py +22 -0
index a8da15a26b3..dd37212f9df 100755
@@ -16,6 +16,8 @@
 from qemu_test import exec_command_and_wait_for_pattern
 from qemu_test import skipIfMissingImports, skipFlakyTest, skipUntrustedTest
 
+from test_mips_malta import mips_check_wheezy
+
 
 class MaltaMachineConsole(LinuxKernelTest):
 
@@ -90,6 +92,26 @@ def test_mips64el_malta_5KEc_cpio(self):
         # Wait for VM to shut down gracefully
         self.vm.wait()
 
+    ASSET_WHEEZY_KERNEL = Asset(
+        ('https://people.debian.org/~aurel32/qemu/mipsel/'
+         'vmlinux-3.2.0-4-5kc-malta'),
+        '5e8b725244c59745bb8b64f5d8f49f25fecfa549f3395fb6d19a3b9e5065b85b')
+
+    ASSET_WHEEZY_DISK = Asset(
+        ('https://people.debian.org/~aurel32/qemu/mipsel/'
+         'debian_wheezy_mipsel_standard.qcow2'),
+        '454f09ae39f7e6461c84727b927100d2c7813841f2a0a5dce328114887ecf914')
+
+    def test_wheezy(self):
+        kernel_path = self.ASSET_WHEEZY_KERNEL.fetch()
+        image_path = self.ASSET_WHEEZY_DISK.fetch()
+        kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE
+                               + 'console=ttyS0 root=/dev/sda1')
+        mips_check_wheezy(self,
+            kernel_path, image_path, kernel_command_line, cpuinfo='MIPS 20Kc',
+            dl_file='/boot/initrd.img-3.2.0-4-5kc-malta',
+            hsum='7579f8b56c1187c7c04d0dc3c0c56c7a6314c5ddd3a9bf8803ecc7cf8a3be9f8')
+
 
 @skipIfMissingImports('numpy', 'cv2')
 class MaltaMachineFramebuffer(LinuxKernelTest):
Signature · 1 line
2.49.0
Reviewed-by on v1 8/15
#17 lore raw

Re: [PATCH 08/15] tests/functional: Convert the 64-bit little endian Wheezy mips test

Quoted 11 lines · Thomas Huth

From: Thomas Huth <[email protected]>

Reuse the test function from the 32-bit big endian test to easily convert the 64-bit little endian Wheezy mips test.

---
 tests/avocado/linux_ssh_mips_malta.py   |  8 --------
 tests/functional/meson.build            |  1 +
 tests/functional/test_mips64el_malta.py | 22 ++++++++++++++++++++++
 3 files changed, 23 insertions(+), 8 deletions(-)

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#18 lore raw

PATCH 9/15 tests/functional: Convert the 64-bit big endian Wheezy mips test

In reply to Thomas Huth

From: Thomas Huth <[email protected]>

Reuse the test function from the 32-bit big endian test to easily convert the 64-bit big endian Wheezy mips test.

Since this was the last test in tests/avocado/linux_ssh_mips_malta.py, we can remove this avocado file now, too.

---
 MAINTAINERS                           |   1 -
 tests/avocado/linux_ssh_mips_malta.py | 181 --------------------------
 tests/functional/meson.build          |   2 +
 tests/functional/test_mips64_malta.py |  35 +++++
 4 files changed, 37 insertions(+), 182 deletions(-)
 delete mode 100644 tests/avocado/linux_ssh_mips_malta.py
 create mode 100755 tests/functional/test_mips64_malta.py
MAINTAINERS +0 -1
index 4e1d816f2b4..7b843c543ff 100644
@@ -1354,7 +1354,6 @@ F: hw/acpi/piix4.c
 F: hw/mips/malta.c
 F: hw/pci-host/gt64120.c
 F: include/hw/southbridge/piix.h
-F: tests/avocado/linux_ssh_mips_malta.py
 F: tests/functional/test_mips*_malta.py
 F: tests/functional/test_mips*_tuxrun.py
 
tests/avocado/linux_ssh_mips_malta.py deleted +0 -181

181 lines not on this page · raw message

tests/functional/meson.build +2 -0
index 63876bc7c42..b36fcab0896 100644
@@ -42,6 +42,7 @@ test_timeouts = {
   'mips_malta' : 480,
   'mipsel_malta' : 420,
   'mipsel_replay' : 480,
+  'mips64_malta' : 240,
   'mips64el_malta' : 420,
   'mips64el_replay' : 180,
   'netdev_ethtool' : 180,
@@ -191,6 +192,7 @@ tests_mipsel_system_thorough = [
 ]
 
 tests_mips64_system_thorough = [
+  'mips64_malta',
   'mips64_tuxrun',
 ]
 
tests/functional/test_mips64_malta.py new file +35 -0
new file mode 100755
index 00000000000..53c3e0c1221
@@ -0,0 +1,35 @@
+#!/usr/bin/env python3
+#
+# Functional tests for the big-endian 64-bit MIPS Malta board
+#
+# SPDX-License-Identifier: GPL-2.0-or-later
+
+from qemu_test import LinuxKernelTest, Asset
+from test_mips_malta import mips_check_wheezy
+
+
+class MaltaMachineConsole(LinuxKernelTest):
+
+    ASSET_WHEEZY_KERNEL = Asset(
+        ('https://people.debian.org/~aurel32/qemu/mips/'
+         'vmlinux-3.2.0-4-5kc-malta'),
+        '3e4ec154db080b3f1839f04dde83120654a33e5e1716863de576c47cb94f68f6')
+
+    ASSET_WHEEZY_DISK = Asset(
+        ('https://people.debian.org/~aurel32/qemu/mips/'
+         'debian_wheezy_mips_standard.qcow2'),
+        'de03599285b8382ad309309a6c4869f6c6c42a5cfc983342bab9ec0dfa7849a2')
+
+    def test_wheezy(self):
+        kernel_path = self.ASSET_WHEEZY_KERNEL.fetch()
+        image_path = self.ASSET_WHEEZY_DISK.fetch()
+        kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE
+                               + 'console=ttyS0 root=/dev/sda1')
+        mips_check_wheezy(self,
+            kernel_path, image_path, kernel_command_line, cpuinfo='MIPS 20Kc',
+            dl_file='/boot/initrd.img-3.2.0-4-5kc-malta',
+            hsum='d98b953bb4a41c0fc0fd8d19bbc691c08989ac52568c1d3054d92dfd890d3f06')
+
+
+if __name__ == '__main__':
+    LinuxKernelTest.main()
Signature · 1 line
2.49.0
Reviewed-by on v1 9/15
#19 lore raw

Re: [PATCH 09/15] tests/functional: Convert the 64-bit big endian Wheezy mips test

Quoted 9 lines · Thomas Huth

From: Thomas Huth <[email protected]>

Reuse the test function from the 32-bit big endian test to easily convert the 64-bit big endian Wheezy mips test.

Since this was the last test in tests/avocado/linux_ssh_mips_malta.py, we can remove this avocado file now, too.

Quoted 8 lines
---
 MAINTAINERS                           |   1 -
 tests/avocado/linux_ssh_mips_malta.py | 181 --------------------------
 tests/functional/meson.build          |   2 +
 tests/functional/test_mips64_malta.py |  35 +++++
 4 files changed, 37 insertions(+), 182 deletions(-)
 delete mode 100644 tests/avocado/linux_ssh_mips_malta.py
 create mode 100755 tests/functional/test_mips64_malta.py

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#20 lore raw

PATCH 10/15 tests/avocado: Remove the boot_linux.py tests

In reply to Thomas Huth

From: Thomas Huth <[email protected]>

These tests are based on the cloudinit functions from Avocado. The cloudinit is very, very slow compared to our other tests, so most of these Avocado tests have either been disabled by default with a decorator, or have been marked to only run with KVM.

We won't include this sluggish cloudinit stuff in the functional framework, and we've already got plenty of other tests there that check pretty much the same things, so let's simply get rid of these old tests now.

---
 tests/avocado/boot_linux.py | 132 ------------------------------------
 1 file changed, 132 deletions(-)
 delete mode 100644 tests/avocado/boot_linux.py
tests/avocado/boot_linux.py deleted +0 -132
deleted file mode 100644
index a029ef4ad1e..00000000000
@@ -1,132 +0,0 @@
-# Functional test that boots a complete Linux system via a cloud image
-#
-# Copyright (c) 2018-2020 Red Hat, Inc.
-#
-# Author:
-#  Cleber Rosa <[email protected]>
-#
-# This work is licensed under the terms of the GNU GPL, version 2 or
-# later.  See the COPYING file in the top-level directory.
-
-import os
-
-from avocado_qemu.linuxtest import LinuxTest
-from avocado_qemu import BUILD_DIR
-
-from avocado import skipUnless
-
-
-class BootLinuxX8664(LinuxTest):
-    """
-    :avocado: tags=arch:x86_64
-    """
-    timeout = 480
-
-    def test_pc_i440fx_tcg(self):
-        """
-        :avocado: tags=machine:pc
-        :avocado: tags=accel:tcg
-        """
-        self.require_accelerator("tcg")
-        self.vm.add_args("-accel", "tcg")
-        self.launch_and_wait(set_up_ssh_connection=False)
-
-    def test_pc_i440fx_kvm(self):
-        """
-        :avocado: tags=machine:pc
-        :avocado: tags=accel:kvm
-        """
-        self.require_accelerator("kvm")
-        self.vm.add_args("-accel", "kvm")
-        self.launch_and_wait(set_up_ssh_connection=False)
-
-    def test_pc_q35_tcg(self):
-        """
-        :avocado: tags=machine:q35
-        :avocado: tags=accel:tcg
-        """
-        self.require_accelerator("tcg")
-        self.vm.add_args("-accel", "tcg")
-        self.launch_and_wait(set_up_ssh_connection=False)
-
-    def test_pc_q35_kvm(self):
-        """
-        :avocado: tags=machine:q35
-        :avocado: tags=accel:kvm
-        """
-        self.require_accelerator("kvm")
-        self.vm.add_args("-accel", "kvm")
-        self.launch_and_wait(set_up_ssh_connection=False)
-
-
-# For Aarch64 we only boot KVM tests in CI as booting the current
-# Fedora OS in TCG tests is very heavyweight. There are lighter weight
-# distros which we use in the machine_aarch64_virt.py tests.
-class BootLinuxAarch64(LinuxTest):
-    """
-    :avocado: tags=arch:aarch64
-    :avocado: tags=machine:virt
-    """
-    timeout = 720
-
-    def test_virt_kvm(self):
-        """
-        :avocado: tags=accel:kvm
-        :avocado: tags=cpu:host
-        """
-        self.require_accelerator("kvm")
-        self.vm.add_args("-accel", "kvm")
-        self.vm.add_args("-machine", "virt,gic-version=host")
-        self.vm.add_args('-bios',
-                         os.path.join(BUILD_DIR, 'pc-bios',
-                                      'edk2-aarch64-code.fd'))
-        self.vm.add_args('-device', 'virtio-rng-pci,rng=rng0')
-        self.vm.add_args('-object', 'rng-random,id=rng0,filename=/dev/urandom')
-        self.launch_and_wait(set_up_ssh_connection=False)
-
-
-# See the tux_baseline.py tests for almost the same coverage in a lot
-# less time.
-class BootLinuxPPC64(LinuxTest):
-    """
-    :avocado: tags=arch:ppc64
-    """
-
-    timeout = 360
-
-    @skipUnless(os.getenv('SPEED') == 'slow', 'runtime limited')
-    def test_pseries_tcg(self):
-        """
-        :avocado: tags=machine:pseries
-        :avocado: tags=accel:tcg
-        """
-        self.require_accelerator("tcg")
-        self.vm.add_args("-accel", "tcg")
-        self.launch_and_wait(set_up_ssh_connection=False)
-
-    def test_pseries_kvm(self):
-        """
-        :avocado: tags=machine:pseries
-        :avocado: tags=accel:kvm
-        """
-        self.require_accelerator("kvm")
-        self.vm.add_args("-accel", "kvm")
-        self.vm.add_args("-machine", "cap-ccf-assist=off")
-        self.launch_and_wait(set_up_ssh_connection=False)
-
-class BootLinuxS390X(LinuxTest):
-    """
-    :avocado: tags=arch:s390x
-    """
-
-    timeout = 240
-
-    @skipUnless(os.getenv('SPEED') == 'slow', 'runtime limited')
-    def test_s390_ccw_virtio_tcg(self):
-        """
-        :avocado: tags=machine:s390-ccw-virtio
-        :avocado: tags=accel:tcg
-        """
-        self.require_accelerator("tcg")
-        self.vm.add_args("-accel", "tcg")
-        self.launch_and_wait(set_up_ssh_connection=False)
Signature · 1 line
2.49.0
#21 lore raw

PATCH 11/15 tests/functional: Use the tuxrun kernel for the x86 replay test

In reply to Thomas Huth

From: Thomas Huth <[email protected]>

This way we can do a full boot in record-replay mode and should get a similar test coverage compared to the old replay test from tests/avocado/replay_linux.py. Thus remove the x86 avocado replay_linux test now.

---
 tests/avocado/replay_linux.py          | 46 --------------------------
 tests/functional/test_x86_64_replay.py | 43 ++++++++++++++++++------
 2 files changed, 33 insertions(+), 56 deletions(-)
tests/avocado/replay_linux.py +0 -46
index 59169224353..6ba283d3bf6 100644
@@ -118,52 +118,6 @@ def run_replay_dump(self, replay_path):
         except subprocess.CalledProcessError:
             self.fail('replay-dump.py failed')
 
-@skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might timeout')
-class ReplayLinuxX8664(ReplayLinux):
-    """
-    :avocado: tags=arch:x86_64
-    :avocado: tags=accel:tcg
-    """
-
-    chksum = 'e3c1b309d9203604922d6e255c2c5d098a309c2d46215d8fc026954f3c5c27a0'
-
-    def test_pc_i440fx(self):
-        """
-        :avocado: tags=machine:pc
-        """
-        self.run_rr(shift=1)
-
-    def test_pc_q35(self):
-        """
-        :avocado: tags=machine:q35
-        """
-        self.run_rr(shift=3)
-
-@skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might timeout')
-class ReplayLinuxX8664Virtio(ReplayLinux):
-    """
-    :avocado: tags=arch:x86_64
-    :avocado: tags=virtio
-    :avocado: tags=accel:tcg
-    """
-
-    hdd = 'virtio-blk-pci'
-    cd = 'virtio-blk-pci'
-    bus = None
-
-    chksum = 'e3c1b309d9203604922d6e255c2c5d098a309c2d46215d8fc026954f3c5c27a0'
-
-    def test_pc_i440fx(self):
-        """
-        :avocado: tags=machine:pc
-        """
-        self.run_rr(shift=1)
-
-    def test_pc_q35(self):
-        """
-        :avocado: tags=machine:q35
-        """
-        self.run_rr(shift=3)
 
 @skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might timeout')
 class ReplayLinuxAarch64(ReplayLinux):
tests/functional/test_x86_64_replay.py +33 -10

63 lines not on this page · raw message

Signature · 1 line
2.49.0
Reviewed-by on v1 11/15
#22 lore raw

Re: [PATCH 11/15] tests/functional: Use the tuxrun kernel for the x86 replay test

Quoted 7 lines · Thomas Huth

From: Thomas Huth <[email protected]>

This way we can do a full boot in record-replay mode and should get a similar test coverage compared to the old replay test from tests/avocado/replay_linux.py. Thus remove the x86 avocado replay_linux test now.

Quoted 5 lines
---
 tests/avocado/replay_linux.py          | 46 --------------------------
 tests/functional/test_x86_64_replay.py | 43 ++++++++++++++++++------
 2 files changed, 33 insertions(+), 56 deletions(-)
Quoted 17 lines
tests/functional/test_x86_64_replay.py +3 -1
index 180f23a60c5..27287d452dc 100755
@@ -5,30 +5,53 @@
 #
 # SPDX-License-Identifier: GPL-2.0-or-later
 
-from qemu_test import Asset, skipFlakyTest
+from subprocess import check_call, DEVNULL
+
+from qemu_test import Asset, skipFlakyTest, get_qemu_img
 from replay_kernel import ReplayKernelBase
 
 
 class X86Replay(ReplayKernelBase):
 

tests/functional/test_x86_64_replay.py line 22

Quoted 10 lines
tests/functional/test_x86_64_replay.py +6 -3
     ASSET_KERNEL = Asset(
-         ('https://archives.fedoraproject.org/pub/archive/fedora/linux'
-          '/releases/29/Everything/x86_64/os/images/pxeboot/vmlinuz'),
-        '8f237d84712b1b411baf3af2aeaaee10b9aae8e345ec265b87ab3a39639eb143')
+        'https://storage.tuxboot.com/buildroot/20241119/x86_64/bzImage',
+        'f57bfc6553bcd6e0a54aab86095bf642b33b5571d14e3af1731b18c87ed5aef8')
+
+    ASSET_ROOTFS = Asset(
+        'https://storage.tuxboot.com/buildroot/20241119/x86_64/rootfs.ext4.zst',
+        '4b8b2a99117519c5290e1202cb36eb6c7aaba92b357b5160f5970cf5fb78a751')

As a general question, I wonder if we want to add some logic to the pre-cache job to clean up old cached files.

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#23 lore raw

Re: [PATCH 11/15] tests/functional: Use the tuxrun kernel for the x86 replay test

Quoted 33 lines · Daniel P. Berrangé
Quoted 7 lines · Thomas Huth

From: Thomas Huth <[email protected]>

This way we can do a full boot in record-replay mode and should get a similar test coverage compared to the old replay test from tests/avocado/replay_linux.py. Thus remove the x86 avocado replay_linux test now.

Quoted 5 lines
---
  tests/avocado/replay_linux.py          | 46 --------------------------
  tests/functional/test_x86_64_replay.py | 43 ++++++++++++++++++------
  2 files changed, 33 insertions(+), 56 deletions(-)
Quoted 17 lines
tests/functional/test_x86_64_replay.py +3 -1
index 180f23a60c5..27287d452dc 100755
@@ -5,30 +5,53 @@
  #
  # SPDX-License-Identifier: GPL-2.0-or-later
 
-from qemu_test import Asset, skipFlakyTest
+from subprocess import check_call, DEVNULL
+
+from qemu_test import Asset, skipFlakyTest, get_qemu_img
  from replay_kernel import ReplayKernelBase
 
 
  class X86Replay(ReplayKernelBase):
 
Quoted 14 lines

tests/functional/test_x86_64_replay.py line 22

Quoted 10 lines
tests/functional/test_x86_64_replay.py +6 -3
      ASSET_KERNEL = Asset(
-         ('https://archives.fedoraproject.org/pub/archive/fedora/linux'
-          '/releases/29/Everything/x86_64/os/images/pxeboot/vmlinuz'),
-        '8f237d84712b1b411baf3af2aeaaee10b9aae8e345ec265b87ab3a39639eb143')
+        'https://storage.tuxboot.com/buildroot/20241119/x86_64/bzImage',
+        'f57bfc6553bcd6e0a54aab86095bf642b33b5571d14e3af1731b18c87ed5aef8')
+
+    ASSET_ROOTFS = Asset(
+        'https://storage.tuxboot.com/buildroot/20241119/x86_64/rootfs.ext4.zst',
+        '4b8b2a99117519c5290e1202cb36eb6c7aaba92b357b5160f5970cf5fb78a751')

As a general question, I wonder if we want to add some logic to the pre-cache job to clean up old cached files.

Yes, I also asked myself that question already. Being a "cache", there should also be a way to evict old files that are not used anymore.
Maybe we could update the timestamp of the assets each time they are used, and when doing "make clean", we also check the cache and delete the assets with timestamps older than 6 months or so?

Thomas

#24 lore raw

Re: [PATCH 11/15] tests/functional: Use the tuxrun kernel for the x86 replay test

Quoted 45 lines · Thomas Huth
Quoted 35 lines · Daniel P. Berrangé
Quoted 7 lines · Thomas Huth

From: Thomas Huth <[email protected]>

This way we can do a full boot in record-replay mode and should get a similar test coverage compared to the old replay test from tests/avocado/replay_linux.py. Thus remove the x86 avocado replay_linux test now.

Quoted 5 lines
---
  tests/avocado/replay_linux.py          | 46 --------------------------
  tests/functional/test_x86_64_replay.py | 43 ++++++++++++++++++------
  2 files changed, 33 insertions(+), 56 deletions(-)
Quoted 19 lines
tests/functional/test_x86_64_replay.py +5 -4
index 180f23a60c5..27287d452dc 100755
@@ -5,30 +5,53 @@
  #
  # SPDX-License-Identifier: GPL-2.0-or-later
-from qemu_test import Asset, skipFlakyTest
+from subprocess import check_call, DEVNULL
+
+from qemu_test import Asset, skipFlakyTest, get_qemu_img
  from replay_kernel import ReplayKernelBase
  class X86Replay(ReplayKernelBase):
      ASSET_KERNEL = Asset(
-         ('https://archives.fedoraproject.org/pub/archive/fedora/linux'
-          '/releases/29/Everything/x86_64/os/images/pxeboot/vmlinuz'),
-        '8f237d84712b1b411baf3af2aeaaee10b9aae8e345ec265b87ab3a39639eb143')
+        'https://storage.tuxboot.com/buildroot/20241119/x86_64/bzImage',
+        'f57bfc6553bcd6e0a54aab86095bf642b33b5571d14e3af1731b18c87ed5aef8')
Quoted 8 lines

tests/functional/test_x86_64_replay.py line 22

Quoted 4 lines
tests/functional/test_x86_64_replay.py +4 -0
+
+    ASSET_ROOTFS = Asset(
+        'https://storage.tuxboot.com/buildroot/20241119/x86_64/rootfs.ext4.zst',
+        '4b8b2a99117519c5290e1202cb36eb6c7aaba92b357b5160f5970cf5fb78a751')

As a general question, I wonder if we want to add some logic to the pre-cache job to clean up old cached files.

Quoted 5 lines

Yes, I also asked myself that question already. Being a "cache", there should also be a way to evict old files that are not used anymore. Maybe we could update the timestamp of the assets each time they are used, and when doing "make clean", we also check the cache and delete the assets with timestamps older than 6 months or so?

Yeah, some time based check + "touch" would be needed, so we don't aggressively purge files cached by a dev's use a different branch that happens to not be checked out currently.

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#25 lore raw

PATCH 12/15 tests/functional: Use the tuxrun kernel for the aarch64 replay test

In reply to Thomas Huth

From: Thomas Huth <[email protected]>

This way we can do a full boot in record-replay mode and should get a similar test coverage compared to the old replay test from tests/avocado/replay_linux.py.

Since the aarch64 test was the last avocado test in the tests/avocado/replay_linux.py file, we can remove this file now completely.

---
 MAINTAINERS                             |   1 -
 tests/avocado/replay_linux.py           | 160 ------------------------
 tests/functional/test_aarch64_replay.py |  37 ++++--
 3 files changed, 29 insertions(+), 169 deletions(-)
 delete mode 100644 tests/avocado/replay_linux.py
MAINTAINERS +0 -1
index 7b843c543ff..acdd8941168 100644
@@ -3667,7 +3667,6 @@ F: include/system/replay.h
 F: docs/devel/replay.rst
 F: docs/system/replay.rst
 F: stubs/replay.c
-F: tests/avocado/replay_linux.py
 F: tests/functional/*reverse_debug*.py
 F: tests/functional/*replay*.py
 F: qapi/replay.json
tests/avocado/replay_linux.py deleted +0 -160

160 lines not on this page · raw message

tests/functional/test_aarch64_replay.py +29 -8
index 04cde433bcf..04dc32722c9 100755
@@ -5,25 +5,46 @@
 #
 # SPDX-License-Identifier: GPL-2.0-or-later
 
-from qemu_test import Asset
+from subprocess import check_call, DEVNULL
+
+from qemu_test import Asset, get_qemu_img
 from replay_kernel import ReplayKernelBase
 
 
 class Aarch64Replay(ReplayKernelBase):
 
     ASSET_KERNEL = Asset(
-        ('https://archives.fedoraproject.org/pub/archive/fedora/linux/'
-         'releases/29/Everything/aarch64/os/images/pxeboot/vmlinuz'),
-        '7e1430b81c26bdd0da025eeb8fbd77b5dc961da4364af26e771bd39f379cbbf7')
+        'https://storage.tuxboot.com/buildroot/20241119/arm64/Image',
+        'b74743c5e89e1cea0f73368d24ae0ae85c5204ff84be3b5e9610417417d2f235')
+
+    ASSET_ROOTFS = Asset(
+        'https://storage.tuxboot.com/buildroot/20241119/arm64/rootfs.ext4.zst',
+        'a1acaaae2068df4648d04ff75f532aaa8c5edcd6b936122b6f0db4848a07b465')
 
     def test_aarch64_virt(self):
+        self.require_netdev('user')
         self.set_machine('virt')
-        self.cpu = 'cortex-a53'
+        self.cpu = 'cortex-a57'
         kernel_path = self.ASSET_KERNEL.fetch()
+
+        raw_disk = self.uncompress(self.ASSET_ROOTFS)
+        disk = self.scratch_file('scratch.qcow2')
+        qemu_img = get_qemu_img(self)
+        check_call([qemu_img, 'create', '-f', 'qcow2', '-b', raw_disk,
+                    '-F', 'raw', disk], stdout=DEVNULL, stderr=DEVNULL)
+
+        args = ('-drive', 'file=%s,snapshot=on,id=hd0,if=none' % disk,
+                '-drive', 'driver=blkreplay,id=hd0-rr,if=none,image=hd0',
+                '-device', 'virtio-blk-device,drive=hd0-rr',
+                '-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0-:22',
+                '-device', 'virtio-net,netdev=vnet',
+                '-object', 'filter-replay,id=replay,netdev=vnet')
+
         kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
-                               'console=ttyAMA0')
-        console_pattern = 'VFS: Cannot open root device'
-        self.run_rr(kernel_path, kernel_command_line, console_pattern)
+                               'console=ttyAMA0 root=/dev/vda')
+        console_pattern = 'Welcome to TuxTest'
+        self.run_rr(kernel_path, kernel_command_line, console_pattern,
+                    args=args)
 
 
 if __name__ == '__main__':
Signature · 1 line
2.49.0
Reviewed-by on v1 12/15
#26 lore raw

Re: [PATCH 12/15] tests/functional: Use the tuxrun kernel for the aarch64 replay test

Quoted 10 lines · Thomas Huth

From: Thomas Huth <[email protected]>

This way we can do a full boot in record-replay mode and should get a similar test coverage compared to the old replay test from tests/avocado/replay_linux.py.

Since the aarch64 test was the last avocado test in the tests/avocado/replay_linux.py file, we can remove this file now completely.

Quoted 7 lines
---
 MAINTAINERS                             |   1 -
 tests/avocado/replay_linux.py           | 160 ------------------------
 tests/functional/test_aarch64_replay.py |  37 ++++--
 3 files changed, 29 insertions(+), 169 deletions(-)
 delete mode 100644 tests/avocado/replay_linux.py

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#27 lore raw

PATCH 13/15 tests/functional: Convert the SMMU test to the functional framework

In reply to Thomas Huth

From: Thomas Huth <[email protected]>

This test was using cloudinit and a "dnf install" command in the guest to exercise the NIC with SMMU enabled. Since we don't have the cloudinit stuff in the functional framework and we should not rely on having access to external networks (once our ASSETs have been cached), we rather boot into the initrd first, manually mount the root disk and then use the check_http_download() function from the functional framework here instead for testing whether the network works as expected.

Unfortunately, there seems to be a small race when using the files from Fedora 33: To enter the initrd shell, we have to send a "return" once. But it does not seem to work if we send it too early. Using a sleep(0.2) makes it work reliably for me, but to make it even more unlikely to trigger this situation, let's better limit the Fedora 33 tests to only run with KVM.

Finally, while we're at it, we also add some lines for testing writes to the hard disk, as we already do it in the test_intel_iommu test.

---
 MAINTAINERS                           |   2 +-
 tests/avocado/smmu.py                 | 139 -----------------
 tests/functional/meson.build          |   2 +
 tests/functional/test_aarch64_smmu.py | 205 ++++++++++++++++++++++++++
 4 files changed, 208 insertions(+), 140 deletions(-)
 delete mode 100644 tests/avocado/smmu.py
 create mode 100755 tests/functional/test_aarch64_smmu.py
MAINTAINERS +1 -1
index acdd8941168..9bd9cb836ce 100644
@@ -211,7 +211,7 @@ L: [email protected]
 S: Maintained
 F: hw/arm/smmu*
 F: include/hw/arm/smmu*
-F: tests/avocado/smmu.py
+F: tests/functional/test_aarch64_smmu.py
 
 AVR TCG CPUs
 M: Michael Rolnik <[email protected]>
tests/avocado/smmu.py deleted +0 -139

139 lines not on this page · raw message

tests/functional/meson.build +2 -0
index b36fcab0896..8c1d3bb4bdb 100644
@@ -18,6 +18,7 @@ test_timeouts = {
   'aarch64_rme_sbsaref' : 1200,
   'aarch64_sbsaref_alpine' : 1200,
   'aarch64_sbsaref_freebsd' : 720,
+  'aarch64_smmu' : 720,
   'aarch64_tuxrun' : 240,
   'aarch64_virt' : 360,
   'aarch64_virt_gpu' : 480,
@@ -88,6 +89,7 @@ tests_aarch64_system_thorough = [
   'aarch64_sbsaref',
   'aarch64_sbsaref_alpine',
   'aarch64_sbsaref_freebsd',
+  'aarch64_smmu',
   'aarch64_tcg_plugins',
   'aarch64_tuxrun',
   'aarch64_virt',
tests/functional/test_aarch64_smmu.py new file +205 -0

205 lines not on this page · raw message

Signature · 1 line
2.49.0
#28 lore raw

PATCH 14/15 gitlab-ci: Update QEMU_JOB_AVOCADO and QEMU_CI_AVOCADO_TESTING

In reply to Thomas Huth

From: Thomas Huth <[email protected]>

Since we don't run the Avocado jobs in the CI anymore, rename these variables to QEMU_JOB_FUNCTIONAL and QEMU_CI_FUNCTIONAL_TESTING.

Also, there was a mismatch between the documentation and the implementation of QEMU_CI_AVOCADO_TESTING: While the documentation said that you had to "Set this variable to have the tests using the Avocado framework run automatically", you indeed needed to set it to make the pipelines appear in your dashboard - but they were never run automatically in forks and had to be triggered manually. Let's improve this now: No need to hide these pipelines from the users by default anymore (the functional tests should be stable enough nowadays), and rather allow the users to run the pipelines auto- matically with this switch now instead, as was documented.

---
 docs/devel/testing/ci-jobs.rst.inc  | 19 +++++++++----------
 .gitlab-ci.d/base.yml               |  8 ++------
 .gitlab-ci.d/buildtest-template.yml |  2 +-
 3 files changed, 12 insertions(+), 17 deletions(-)
docs/devel/testing/ci-jobs.rst.inc +9 -10
index 3756bbe3554..4f4c96dd883 100644
@@ -126,10 +126,10 @@ QEMU_JOB_PUBLISH
 The job is for publishing content after a branch has been
 merged into the upstream default branch.
 
-QEMU_JOB_AVOCADO
-~~~~~~~~~~~~~~~~
+QEMU_JOB_FUNCTIONAL
+~~~~~~~~~~~~~~~~~~~
 
-The job runs the Avocado integration test suite
+The job runs the functional test suite
 
 Contributor controlled runtime variables
 ----------------------------------------
@@ -149,13 +149,12 @@ the jobs to be manually started from the UI
 Set this variable to 2 to create the pipelines and run all
 the jobs immediately, as was the historical behaviour
 
-QEMU_CI_AVOCADO_TESTING
-~~~~~~~~~~~~~~~~~~~~~~~
-By default, tests using the Avocado framework are not run automatically in
-the pipelines (because multiple artifacts have to be downloaded, and if
-these artifacts are not already cached, downloading them make the jobs
-reach the timeout limit). Set this variable to have the tests using the
-Avocado framework run automatically.
+QEMU_CI_FUNCTIONAL_TESTING
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+By default, tests using the functional framework are not run automatically
+in the pipelines (because multiple artifacts have to be downloaded, which
+might cause a lot of network traffic). Set this variable to have the tests
+using the functional framework run automatically.
 
 Other misc variables
 --------------------
.gitlab-ci.d/base.yml +2 -6
index 25b88aaa06a..172028e1731 100644
@@ -69,10 +69,6 @@ variables:
     - if: '$QEMU_CI != "1" && $QEMU_CI != "2" && $CI_PROJECT_NAMESPACE != $QEMU_CI_UPSTREAM'
       when: never
 
-    # Avocado jobs don't run in forks unless $QEMU_CI_AVOCADO_TESTING is set
-    - if: '$QEMU_JOB_AVOCADO && $QEMU_CI_AVOCADO_TESTING != "1" && $CI_PROJECT_NAMESPACE != $QEMU_CI_UPSTREAM'
-      when: never
-
 
     #############################################################
     # Stage 2: fine tune execution of jobs in specific scenarios
@@ -101,8 +97,8 @@ variables:
       when: manual
       allow_failure: true
 
-    # Avocado jobs can be manually start in forks if $QEMU_CI_AVOCADO_TESTING is unset
-    - if: '$QEMU_JOB_AVOCADO && $CI_PROJECT_NAMESPACE != $QEMU_CI_UPSTREAM'
+    # Functional jobs can be manually started in forks
+    - if: '$QEMU_JOB_FUNCTIONAL && $QEMU_CI_FUNCTIONAL_TESTING != "1" && $CI_PROJECT_NAMESPACE != $QEMU_CI_UPSTREAM'
       when: manual
       allow_failure: true
 
.gitlab-ci.d/buildtest-template.yml +1 -1
index 13fa4f4a4fc..d4f145fdb58 100644
@@ -114,4 +114,4 @@
     - cd build
     - du -chs ${CI_PROJECT_DIR}/*-cache
   variables:
-    QEMU_JOB_AVOCADO: 1
+    QEMU_JOB_FUNCTIONAL: 1
Signature · 1 line
2.49.0
Reviewed-by on v1 14/15
#29 lore raw

Re: [PATCH 14/15] gitlab-ci: Update QEMU_JOB_AVOCADO and QEMU_CI_AVOCADO_TESTING

Quoted 43 lines · Thomas Huth

From: Thomas Huth <[email protected]>

Since we don't run the Avocado jobs in the CI anymore, rename these variables to QEMU_JOB_FUNCTIONAL and QEMU_CI_FUNCTIONAL_TESTING.

Also, there was a mismatch between the documentation and the implementation of QEMU_CI_AVOCADO_TESTING: While the documentation said that you had to "Set this variable to have the tests using the Avocado framework run automatically", you indeed needed to set it to make the pipelines appear in your dashboard - but they were never run automatically in forks and had to be triggered manually. Let's improve this now: No need to hide these pipelines from the users by default anymore (the functional tests should be stable enough nowadays), and rather allow the users to run the pipelines auto- matically with this switch now instead, as was documented.

---
 docs/devel/testing/ci-jobs.rst.inc  | 19 +++++++++----------
 .gitlab-ci.d/base.yml               |  8 ++------
 .gitlab-ci.d/buildtest-template.yml |  2 +-
 3 files changed, 12 insertions(+), 17 deletions(-)
docs/devel/testing/ci-jobs.rst.inc +3 -3
index 3756bbe3554..4f4c96dd883 100644
@@ -126,10 +126,10 @@ QEMU_JOB_PUBLISH
 The job is for publishing content after a branch has been
 merged into the upstream default branch.
 
-QEMU_JOB_AVOCADO
-~~~~~~~~~~~~~~~~
+QEMU_JOB_FUNCTIONAL
+~~~~~~~~~~~~~~~~~~~
 
-The job runs the Avocado integration test suite
+The job runs the functional test suite
 
 Contributor controlled runtime variables
 ----------------------------------------
@@ -149,13 +149,12 @@ the jobs to be manually started from the UI
 Set this variable to 2 to create the pipelines and run all

docs/devel/testing/ci-jobs.rst.inc the jobs to be manually started from the UI · line 152

Quoted 10 lines
docs/devel/testing/ci-jobs.rst.inc +1 -7
 the jobs immediately, as was the historical behaviour
 
-QEMU_CI_AVOCADO_TESTING
-~~~~~~~~~~~~~~~~~~~~~~~
-By default, tests using the Avocado framework are not run automatically in
-the pipelines (because multiple artifacts have to be downloaded, and if
-these artifacts are not already cached, downloading them make the jobs
-reach the timeout limit). Set this variable to have the tests using the
-Avocado framework run automatically.
+QEMU_CI_FUNCTIONAL_TESTING

Both old and new terms are quite verbose, how about dropping '_TESTING' suffix ?

With, or without that rename though:

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#30 lore raw

Re: [PATCH 14/15] gitlab-ci: Update QEMU_JOB_AVOCADO and QEMU_CI_AVOCADO_TESTING

Quoted 5 lines · Daniel P. Berrangé
Quoted 4 lines · Thomas Huth

From: Thomas Huth <[email protected]>

Since we don't run the Avocado jobs in the CI anymore, rename these variables to QEMU_JOB_FUNCTIONAL and QEMU_CI_FUNCTIONAL_TESTING.

...

Quoted 11 lines

docs/devel/testing/ci-jobs.rst.inc the jobs to be manually started from the UI · line 152

Quoted 8 lines
docs/devel/testing/ci-jobs.rst.inc +1 -7
-QEMU_CI_AVOCADO_TESTING
-~~~~~~~~~~~~~~~~~~~~~~~
-By default, tests using the Avocado framework are not run automatically in
-the pipelines (because multiple artifacts have to be downloaded, and if
-these artifacts are not already cached, downloading them make the jobs
-reach the timeout limit). Set this variable to have the tests using the
-Avocado framework run automatically.
+QEMU_CI_FUNCTIONAL_TESTING

Both old and new terms are quite verbose, how about dropping '_TESTING' suffix ?

Sure, I'll drop it in v2!

Quoted 3 lines

With, or without that rename though:

Thanks!

Thomas

#31 lore raw

PATCH 15/15 Remove the remainders of the Avocado tests

In reply to Thomas Huth

From: Thomas Huth <[email protected]>

Now that all Avocado tests have been converted to or been replaced by other functional tests, we can delete the remainders of the Avocado tests from the QEMU source tree.

---
 MAINTAINERS                             |   8 +-
 docs/about/build-platforms.rst          |  10 +-
 docs/devel/build-system.rst             |  11 +-
 docs/devel/codebase.rst                 |   5 -
 docs/devel/testing/avocado.rst          | 581 ------------------------
 docs/devel/testing/functional.rst       |   3 -
 docs/devel/testing/index.rst            |   1 -
 docs/devel/testing/main.rst             |  15 -
 tests/avocado/README.rst                |  10 -
 configure                               |   2 +-
 pythondeps.toml                         |   8 +-
 tests/Makefile.include                  |  60 +--
 tests/avocado/avocado_qemu/__init__.py  | 424 -----------------
 tests/avocado/avocado_qemu/linuxtest.py | 253 -----------
 14 files changed, 16 insertions(+), 1375 deletions(-)
 delete mode 100644 docs/devel/testing/avocado.rst
 delete mode 100644 tests/avocado/README.rst
 delete mode 100644 tests/avocado/avocado_qemu/__init__.py
 delete mode 100644 tests/avocado/avocado_qemu/linuxtest.py
MAINTAINERS +1 -7
index 9bd9cb836ce..56408133cd6 100644
@@ -2072,7 +2072,7 @@ S: Supported
 F: hw/acpi/viot.c
 F: hw/acpi/viot.h
 
-ACPI/AVOCADO/BIOSBITS
+ACPI/FUNCTIONAL/BIOSBITS
 M: Ani Sinha <[email protected]>
 M: Michael S. Tsirkin <[email protected]>
 S: Supported
@@ -4245,12 +4245,6 @@ R: Philippe Mathieu-Daudé <[email protected]>
 S: Maintained
 F: tests/tcg/Makefile.target
 
-Integration Testing with the Avocado framework
-W: https://trello.com/b/6Qi1pxVn/avocado-qemu
-R: Cleber Rosa <[email protected]>
-S: Odd Fixes
-F: tests/avocado/
-
 GitLab custom runner (Works On Arm Sponsored)
 M: Alex Bennée <[email protected]>
 M: Philippe Mathieu-Daudé <[email protected]>
docs/about/build-platforms.rst +5 -5
index 1552b1a7044..52521552c8a 100644
@@ -123,11 +123,11 @@ Rust build dependencies
   to build QEMU.
 
 Optional build dependencies
-  Build components whose absence does not affect the ability to build
-  QEMU may not be available in distros, or may be too old for QEMU's
-  requirements.  Many of these, such as the Avocado testing framework
-  or various linters, are written in Python and therefore can also
-  be installed using ``pip``.  Cross compilers are another example
+  Build components whose absence does not affect the ability to build QEMU
+  may not be available in distros, or may be too old for our requirements.
+  Many of these, such as additional modules for the functional testing
+  framework or various linters, are written in Python and therefore can
+  also be installed using ``pip``.  Cross compilers are another example
   of optional build-time dependency; in this case it is possible to
   download them from repositories such as EPEL, to use container-based
   cross compilation using ``docker`` or ``podman``, or to use pre-built
docs/devel/build-system.rst +3 -8
index a759982f45c..258cfad3fe7 100644
@@ -134,7 +134,7 @@ in how the build process runs Python code.
 
 At this stage, ``configure`` also queries the chosen Python interpreter
 about QEMU's build dependencies.  Note that the build process does  *not*
-look for ``meson``, ``sphinx-build`` or ``avocado`` binaries in the PATH;
+look for ``meson`` or ``sphinx-build`` binaries in the PATH;
 likewise, there are no options such as ``--meson`` or ``--sphinx-build``.
 This avoids a potential mismatch, where Meson and Sphinx binaries on the
 PATH might operate in a different Python environment than the one chosen
@@ -151,7 +151,7 @@ virtual environment with ``pip``, either from wheels in ``python/wheels``
 or by downloading the package with PyPI.  Downloading can be disabled with
 ``--disable-download``; and anyway, it only happens when a ``configure``
 option (currently, only ``--enable-docs``) is explicitly enabled but
-the dependencies are not present\ [#pip]_.
+the dependencies are not present.
 
 .. [#distlib] The scripts are created based on the package's metadata,
               specifically the ``console_script`` entry points.  This is the
@@ -164,10 +164,6 @@ the dependencies are not present\ [#pip]_.
               because the Python Packaging Authority provides a package
               ``distlib.scripts`` to perform this task.
 
-.. [#pip] ``pip`` might also be used when running ``make check-avocado``
-           if downloading is enabled, to ensure that Avocado is
-           available.
-
 The required versions of the packages are stored in a configuration file
 ``pythondeps.toml``.  The format is custom to QEMU, but it is documented
 at the top of the file itself and it should be easy to understand.  The
@@ -497,8 +493,7 @@ number of dynamically created files listed later.
   ``pyvenv/bin``, and calling ``pip`` to install dependencies.
 
 ``tests/Makefile.include``
-  Rules for external test harnesses. These include the TCG tests
-  and the Avocado-based integration tests.
+  Rules for external test harnesses like the TCG tests.
 
 ``tests/docker/Makefile.include``
   Rules for Docker tests. Like ``tests/Makefile.include``, this file is
docs/devel/codebase.rst +0 -5
index 1b09953197b..1da17f40494 100644
@@ -175,11 +175,6 @@ yet, so sometimes the source code is all you have.
 * `tests <https://gitlab.com/qemu-project/qemu/-/tree/master/tests>`_:
   QEMU `test <testing>` suite
 
-  - `avocado <https://gitlab.com/qemu-project/qemu/-/tree/master/tests/avocado>`_:
-    Functional tests booting full VM using `Avocado framework <checkavocado-ref>`.
-    Those tests will be transformed and moved into
-    `tests/functional <https://gitlab.com/qemu-project/qemu/-/tree/master/tests/functional>`_
-    in the future.
   - `data <https://gitlab.com/qemu-project/qemu/-/tree/master/tests/data>`_:
     Data for various tests.
   - `decode <https://gitlab.com/qemu-project/qemu/-/tree/master/tests/decode>`_:
docs/devel/testing/avocado.rst deleted +0 -581

581 lines not on this page · raw message

docs/devel/testing/functional.rst +0 -3
index 9bc973392aa..8030cb4299f 100644
@@ -6,9 +6,6 @@ Functional testing with Python
 The ``tests/functional`` directory hosts functional tests written in
 Python. They are usually higher level tests, and may interact with
 external resources and with various guest operating systems.
-The functional tests have initially evolved from the Avocado tests, so there
-is a lot of similarity to those tests here (see :ref:`checkavocado-ref` for
-details about the Avocado tests).
 
 The tests should be written in the style of the Python `unittest`_ framework,
 using stdio for the TAP protocol. The folder ``tests/functional/qemu_test``
docs/devel/testing/index.rst +0 -1
index 1171f7db8f0..ccc2fc6cbc1 100644
@@ -10,7 +10,6 @@ testing infrastructure.
    main
    qtest
    functional
-   avocado
    acpi-bits
    ci
    fuzzing
docs/devel/testing/main.rst +0 -15
index 9869bcf0341..3c03d769539 100644
@@ -894,21 +894,6 @@ Python. You can run the functional tests simply by executing:
 
 See :ref:`checkfunctional-ref` for more details.
 
-Integration tests using the Avocado Framework
----------------------------------------------
-
-The ``tests/avocado`` directory hosts integration tests. They're usually
-higher level tests, and may interact with external resources and with
-various guest operating systems.
-
-You can run the avocado tests simply by executing:
-
-.. code::
-
-  make check-avocado
-
-See :ref:`checkavocado-ref` for more details.
-
 .. _checktcg-ref:
 
 Testing with "make check-tcg"
tests/avocado/README.rst deleted +0 -10
deleted file mode 100644
index 94488371bbe..00000000000
@@ -1,10 +0,0 @@
-=============================================
-Integration tests using the Avocado Framework
-=============================================
-
-This directory contains integration tests. They're usually higher
-level, and may interact with external resources and with various
-guest operating systems.
-
-For more information, please refer to ``docs/devel/testing.rst``,
-section "Integration tests using the Avocado Framework".
configure +1 -1
index 02f1dd2311f..000309cf610 100755
@@ -1685,7 +1685,7 @@ LINKS="$LINKS pc-bios/optionrom/Makefile"
 LINKS="$LINKS pc-bios/s390-ccw/Makefile"
 LINKS="$LINKS pc-bios/vof/Makefile"
 LINKS="$LINKS .gdbinit scripts" # scripts needed by relative path in .gdbinit
-LINKS="$LINKS tests/avocado tests/data"
+LINKS="$LINKS tests/data"
 LINKS="$LINKS tests/qemu-iotests/check tests/qemu-iotests/Makefile"
 LINKS="$LINKS python"
 for f in $LINKS ; do
pythondeps.toml +2 -6
index c03c9df81b5..7eaaa0fed10 100644
@@ -27,9 +27,5 @@ pycotap = { accepted = ">=1.1.0", installed = "1.3.1" }
 sphinx = { accepted = ">=3.4.3", installed = "5.3.0", canary = "sphinx-build" }
 sphinx_rtd_theme = { accepted = ">=0.5", installed = "1.1.1" }
 
-[avocado]
-# Note that qemu.git/python/ is always implicitly installed.
-# Prefer an LTS version when updating the accepted versions of
-# avocado-framework, for example right now the limit is 92.x.
-avocado-framework = { accepted = "(>=103.0, <104.0)", installed = "103.0", canary = "avocado" }
-pycdlib = { accepted = ">=1.11.0" }
+[testdeps]
+qemu.qmp = { accepted = ">=0.0.3", installed = "0.0.3" }
tests/Makefile.include +4 -56

104 lines not on this page · raw message

tests/avocado/avocado_qemu/__init__.py deleted +0 -424

424 lines not on this page · raw message

tests/avocado/avocado_qemu/linuxtest.py deleted +0 -253

253 lines not on this page · raw message

Signature · 1 line
2.49.0
Reviewed-by on v1 15/15
#32 lore raw

Re: [PATCH 15/15] Remove the remainders of the Avocado tests

Quoted 23 lines · Thomas Huth

From: Thomas Huth <[email protected]>

Now that all Avocado tests have been converted to or been replaced by other functional tests, we can delete the remainders of the Avocado tests from the QEMU source tree.

---
  MAINTAINERS                             |   8 +-
  docs/about/build-platforms.rst          |  10 +-
  docs/devel/build-system.rst             |  11 +-
  docs/devel/codebase.rst                 |   5 -
  docs/devel/testing/avocado.rst          | 581 ------------------------
  docs/devel/testing/functional.rst       |   3 -
  docs/devel/testing/index.rst            |   1 -
  docs/devel/testing/main.rst             |  15 -
  tests/avocado/README.rst                |  10 -
  configure                               |   2 +-
  pythondeps.toml                         |   8 +-
  tests/Makefile.include                  |  60 +--
  tests/avocado/avocado_qemu/__init__.py  | 424 -----------------
  tests/avocado/avocado_qemu/linuxtest.py | 253 -----------
  14 files changed, 16 insertions(+), 1375 deletions(-)
Quoted 4 lines

delete mode 100644 docs/devel/testing/avocado.rst delete mode 100644 tests/avocado/README.rst delete mode 100644 tests/avocado/avocado_qemu/__init__.py delete mode 100644 tests/avocado/avocado_qemu/linuxtest.py

Thanks a lot Thomas for this huge effort of converting from Avocado to the simpler functional framework... Sincerely appreciated!

Reviewed-by on v1 15/15
#33 lore raw

Re: [PATCH 15/15] Remove the remainders of the Avocado tests

In reply to Thomas Huth
Quoted 19 lines · Thomas Huth

From: Thomas Huth <[email protected]>

Now that all Avocado tests have been converted to or been replaced by other functional tests, we can delete the remainders of the Avocado tests from the QEMU source tree.

---
 MAINTAINERS                             |   8 +-
 docs/about/build-platforms.rst          |  10 +-
 docs/devel/build-system.rst             |  11 +-
 docs/devel/codebase.rst                 |   5 -
 docs/devel/testing/avocado.rst          | 581 ------------------------
 docs/devel/testing/functional.rst       |   3 -
 docs/devel/testing/index.rst            |   1 -
 docs/devel/testing/main.rst             |  15 -
 tests/avocado/README.rst                |  10 -
 configure                               |   2 +-
 pythondeps.toml                         |   8 +-
Quoted 8 lines
 tests/Makefile.include                  |  60 +--
 tests/avocado/avocado_qemu/__init__.py  | 424 -----------------
 tests/avocado/avocado_qemu/linuxtest.py | 253 -----------
 14 files changed, 16 insertions(+), 1375 deletions(-)
 delete mode 100644 docs/devel/testing/avocado.rst
 delete mode 100644 tests/avocado/README.rst
 delete mode 100644 tests/avocado/avocado_qemu/__init__.py
 delete mode 100644 tests/avocado/avocado_qemu/linuxtest.py

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|