tests/avocado: Disable the test_sbsaref_edk2_firmware by default
PATCH v2 1/1 tests/avocado: Disable the test_sbsaref_edk2_firmware by default
The test fails occasionally, see e.g.:
https://gitlab.com/thuth/qemu/-/jobs/4196177756#L489
https://gitlab.com/thuth/qemu/-/jobs/4623296271#L290
It also fails on my laptop in ca. 50% of all runs. Thus disable it by default by using the QEMU_TEST_FLAKY_TESTS environment variable to fence it (which we also already use in flaky qtests). While we're at it, also document this variable in docs/devel/testing.rst.
Signed-off-by Thomas Huth <[email protected]>
---
v2: Use QEMU_TEST_FLAKY_TESTS instead of GITLAB_CI as sugested by PMM
docs/devel/testing.rst | 5 +++++ tests/avocado/machine_aarch64_sbsaref.py | 2 +- 2 files changed, 6 insertions(+), 1 deletion(-)
docs/devel/testing.rst +5 -0
@@ -1384,6 +1384,11 @@ variable as shown on the code snippet below to skip the test: def test(self): do_something() +QEMU_TEST_FLAKY_TESTS +^^^^^^^^^^^^^^^^^^^^^ +Some tests are not working reliably and thus are disabled by default. +Set this environment variable to enable them. + Uninstalling Avocado ~~~~~~~~~~~~~~~~~~~~
tests/avocado/machine_aarch64_sbsaref.py +1 -1
@@ -8,7 +8,6 @@ import os -from avocado import skip from avocado import skipUnless from avocado.utils import archive
@@ -76,6 +75,7 @@ def fetch_firmware(self): "sbsa-ref", ) + @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is not reliable') def test_sbsaref_edk2_firmware(self): """ :avocado: tags=cpu:cortex-a57
Signature · 1 line
2.39.3