Test more "check-acceptance" tests in the gitlab-CI
cover Test more "check-acceptance" tests in the gitlab-CI
These patches enable more "check-acceptance" tests in the gitlab-CI, fixing a broken URL, setting AVOCADO_ALLOW_UNTRUSTED_CODE and adding some missing packages to the containers.
Thomas Huth (4):
tests/acceptance/ppc_prep_40p: Fix the URL to the NetBSD-4.0 archive test/acceptance: Remove the CONTINUOUS_INTEGRATION tags tests/acceptance: Enable AVOCADO_ALLOW_UNTRUSTED_CODE in the gitlab-CI test/docker/dockerfiles: Add missing packages for acceptance tests
.gitlab-ci.yml | 1 + tests/acceptance/ppc_prep_40p.py | 4 +--- tests/docker/dockerfiles/centos8.docker | 1 + tests/docker/dockerfiles/debian-amd64.docker | 3 +++ tests/docker/dockerfiles/fedora.docker | 1 + tests/docker/dockerfiles/ubuntu2004.docker | 1 + 6 files changed, 8 insertions(+), 3 deletions(-)
Signature · 1 line
2.18.2
PATCH 1/4 tests/acceptance/ppc_prep_40p: Fix the URL to the NetBSD-4.0 archive
The current URL on cdn.netbsd.org is failing - using archive.netbsd.org instead seems to be fine.
--- tests/acceptance/ppc_prep_40p.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tests/acceptance/ppc_prep_40p.py +1 -1
@@ -35,7 +35,7 @@ class IbmPrep40pMachine(Test): '7020-40p/P12H0456.IMG') bios_hash = '1775face4e6dc27f3a6ed955ef6eb331bf817f03' bios_path = self.fetch_asset(bios_url, asset_hash=bios_hash) - drive_url = ('https://cdn.netbsd.org/pub/NetBSD/NetBSD-archive/' + drive_url = ('https://archive.netbsd.org/pub/NetBSD-archive/' 'NetBSD-4.0/prep/installation/floppy/generic_com0.fs') drive_hash = 'dbcfc09912e71bd5f0d82c7c1ee43082fb596ceb' drive_path = self.fetch_asset(drive_url, asset_hash=drive_hash)
Signature · 1 line
2.18.2
Re: [PATCH 1/4] tests/acceptance/ppc_prep_40p: Fix the URL to the NetBSD-4.0 archive
Quoted 14 lines · Thomas Huth
The current URL on cdn.netbsd.org is failing - using archive.netbsd.org instead seems to be fine.
Signed-off-by Thomas Huth <[email protected]>--- tests/acceptance/ppc_prep_40p.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)tests/acceptance/ppc_prep_40p.py +0
-0
tests/acceptance/ppc_prep_40p.py class IbmPrep40pMachine(Test): · line 41
Quoted 8 lines
tests/acceptance/ppc_prep_40p.py +1
-1@@ -35,7 +35,7 @@ class IbmPrep40pMachine(Test): '7020-40p/P12H0456.IMG') bios_hash = '1775face4e6dc27f3a6ed955ef6eb331bf817f03' bios_path = self.fetch_asset(bios_url, asset_hash=bios_hash) - drive_url = ('https://cdn.netbsd.org/pub/NetBSD/NetBSD-archive/' + drive_url = ('https://archive.netbsd.org/pub/NetBSD-archive/' 'NetBSD-4.0/prep/installation/floppy/generic_com0.fs') drive_hash = 'dbcfc09912e71bd5f0d82c7c1ee43082fb596ceb' drive_path = self.fetch_asset(drive_url, asset_hash=drive_hash)
Thanks!
PATCH 2/4 test/acceptance: Remove the CONTINUOUS_INTEGRATION tags
In reply to Thomas HuthWe are not running the acceptance tests on Travis anymore, so these checks can be removed now.
--- tests/acceptance/ppc_prep_40p.py | 2 -- 1 file changed, 2 deletions(-)
tests/acceptance/ppc_prep_40p.py +0 -2
@@ -22,7 +22,6 @@ class IbmPrep40pMachine(Test): # All rights reserved. # U.S. Government Users Restricted Rights - Use, duplication or disclosure # restricted by GSA ADP Schedule Contract with IBM Corp. - @skipIf(os.getenv('CONTINUOUS_INTEGRATION'), 'Running on Travis-CI') @skipUnless(os.getenv('AVOCADO_ALLOW_UNTRUSTED_CODE'), 'untrusted code') def test_factory_firmware_and_netbsd(self): """
@@ -61,7 +60,6 @@ class IbmPrep40pMachine(Test): wait_for_console_pattern(self, '>> Memory: 192M') wait_for_console_pattern(self, '>> CPU type PowerPC,604') - @skipIf(os.getenv('CONTINUOUS_INTEGRATION'), 'Running on Travis-CI') def test_openbios_and_netbsd(self): """ :avocado: tags=arch:ppc
Signature · 1 line
2.18.2
Re: [PATCH 2/4] test/acceptance: Remove the CONTINUOUS_INTEGRATION tags
Quoted 7 lines · Thomas Huth
We are not running the acceptance tests on Travis anymore, so these checks can be removed now.
Signed-off-by Thomas Huth <[email protected]>--- tests/acceptance/ppc_prep_40p.py | 2 -- 1 file changed, 2 deletions(-)
PATCH 3/4 tests/acceptance: Enable AVOCADO_ALLOW_UNTRUSTED_CODE in the gitlab-CI
In reply to Thomas HuthThe tests are running in containers here, so it should be OK to run with AVOCADO_ALLOW_UNTRUSTED_CODE enabled in this case.
--- .gitlab-ci.yml | 1 + 1 file changed, 1 insertion(+)
.gitlab-ci.yml +1 -0
@@ -66,6 +66,7 @@ include: - if [ -d ${CI_PROJECT_DIR}/avocado-cache ]; then du -chs ${CI_PROJECT_DIR}/avocado-cache ; fi + - export AVOCADO_ALLOW_UNTRUSTED_CODE=1 after_script: - cd build - python3 -c 'import json; r = json.load(open("tests/results/latest/results.json")); [print(t["logfile"]) for t in r["tests"] if t["status"] not in ("PASS", "SKIP", "CANCEL")]' | xargs cat
Signature · 1 line
2.18.2
Re: [PATCH 3/4] tests/acceptance: Enable AVOCADO_ALLOW_UNTRUSTED_CODE in the gitlab-CI
Quoted 7 lines · Thomas Huth
The tests are running in containers here, so it should be OK to run with
AVOCADO_ALLOW_UNTRUSTED_CODEenabled in this case.Signed-off-by Thomas Huth <[email protected]>--- .gitlab-ci.yml | 1 + 1 file changed, 1 insertion(+)
Thanks :)
PATCH 4/4 test/docker/dockerfiles: Add missing packages for acceptance tests
In reply to Thomas HuthSome of the "check-acceptance" tests are still skipped in the CI since the docker images do not provide the necessary packages, e.g. the netcat binary. Add them to get more test coverage.
--- tests/docker/dockerfiles/centos8.docker | 1 + tests/docker/dockerfiles/debian-amd64.docker | 3 +++ tests/docker/dockerfiles/fedora.docker | 1 + tests/docker/dockerfiles/ubuntu2004.docker | 1 + 4 files changed, 6 insertions(+)
tests/docker/dockerfiles/centos8.docker +1 -0
@@ -18,6 +18,7 @@ ENV PACKAGES \ lzo-devel \ make \ mesa-libEGL-devel \ + nmap-ncat \ nettle-devel \ ninja-build \ perl-Test-Harness \
tests/docker/dockerfiles/debian-amd64.docker +3 -0
@@ -23,6 +23,9 @@ RUN apt update && \ libsnappy-dev \ libvte-dev \ netcat-openbsd \ + openssh-client \ + python3-numpy \ + python3-opencv \ python3-venv # virgl
tests/docker/dockerfiles/fedora.docker +1 -0
@@ -73,6 +73,7 @@ ENV PACKAGES \ mingw64-pixman \ mingw64-pkg-config \ mingw64-SDL2 \ + nmap-ncat \ ncurses-devel \ nettle-devel \ ninja-build \
tests/docker/dockerfiles/ubuntu2004.docker +1 -0
@@ -47,6 +47,7 @@ ENV PACKAGES flex bison \ libxen-dev \ libzstd-dev \ make \ + netcat-openbsd \ ninja-build \ python3-numpy \ python3-opencv \
Signature · 1 line
2.18.2
Re: [PATCH 4/4] test/docker/dockerfiles: Add missing packages for acceptance tests
Quoted 11 lines · Thomas Huth
Some of the "check-acceptance" tests are still skipped in the CI since the docker images do not provide the necessary packages, e.g. the netcat binary. Add them to get more test coverage.
Signed-off-by Thomas Huth <[email protected]>--- tests/docker/dockerfiles/centos8.docker | 1 + tests/docker/dockerfiles/debian-amd64.docker | 3 +++ tests/docker/dockerfiles/fedora.docker | 1 + tests/docker/dockerfiles/ubuntu2004.docker | 1 + 4 files changed, 6 insertions(+)
Finally...
Quoted 3 lines · Thomas Huth
These patches enable more "check-acceptance" tests in the gitlab-CI, fixing a broken URL, setting
AVOCADO_ALLOW_UNTRUSTED_CODEand adding some missing packages to the containers.
Queued to pr/261020-testing-and-misc-1, thanks.
Signature · 1 line
Alex Bennée
Quoted 7 lines · Alex Bennée
Quoted 3 lines · Thomas Huth
These patches enable more "check-acceptance" tests in the gitlab-CI, fixing a broken URL, setting
AVOCADO_ALLOW_UNTRUSTED_CODEand adding some missing packages to the containers.Queued to pr/261020-testing-and-misc-1, thanks.
Ahh I shall drop as your PR should be in first anyway.
Signature · 1 line
Alex Bennée