Skip to contentSkip to conversation

docker: Update Travis-CI image to run acceptance tests

PATCH In review Philippe Mathieu-Daudé 3 participants 10 messages Aug 18, 2019 – Feb 24, 2020

lore
Share & tools
lore
#0 lore raw

cover docker: Update Travis-CI image to run acceptance tests

This series adds a 'test-acceptance' Docker target, so we can run 'make docker-test-acceptance@travis' similarly to how the Travis-CI service runs its builds.

This is particularly useful to debug Travis-CI failures locally.

Regards,

Phil.

Philippe Mathieu-Daudé (4):
tests/docker: Use one package per line to improve readability tests/docker: update our Travis image to run acceptance tests locally tests/docker: Add test-acceptance runner
tests/docker: Use Travis-CI default environment variables

 tests/docker/dockerfiles/travis.docker | 22 ++++++++++++++++++----
 tests/docker/test-acceptance           | 21 +++++++++++++++++++++
 2 files changed, 39 insertions(+), 4 deletions(-)
 create mode 100755 tests/docker/test-acceptance
Signature · 1 line
2.20.1
#1 lore raw

PATCH 1/4 tests/docker: Use one package per line to improve readability

Use one package per line to improve readability. This also helps while reviewing patches.

---
 tests/docker/dockerfiles/travis.docker | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
tests/docker/dockerfiles/travis.docker +9 -1
index e72dc85ca7..35714664a1 100644
@@ -5,7 +5,15 @@ ENV LC_ALL en_US.UTF-8
 RUN sed -i "s/# deb-src/deb-src/" /etc/apt/sources.list
 RUN apt-get update
 RUN apt-get -y build-dep qemu
-RUN apt-get -y install device-tree-compiler python2.7 python-yaml dh-autoreconf gdb strace lsof net-tools gcovr
+RUN apt-get -y install \
+    device-tree-compiler \
+    dh-autoreconf \
+    gcovr \
+    gdb strace \
+    lsof \
+    net-tools \
+    python2.7 \
+    python-yaml
 # Travis tools require PhantomJS / Neo4j / Maven accessible
 # in their PATH (QEMU build won't access them).
 ENV PATH /usr/local/phantomjs/bin:/usr/local/phantomjs:/usr/local/neo4j-3.2.7/bin:/usr/local/maven-3.5.2/bin:/usr/local/cmake-3.9.2/bin:/usr/local/clang-5.0.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
Signature · 1 line
2.20.1
#2 lore raw

Re: [Qemu-devel] [PATCH 1/4] tests/docker: Use one package per line to improve readability

Quoted 13 lines · Philippe Mathieu-Daudé

Use one package per line to improve readability. This also helps while reviewing patches.

---
 tests/docker/dockerfiles/travis.docker | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
tests/docker/dockerfiles/travis.docker +0 -0
index e72dc85ca7..35714664a1 100644

tests/docker/dockerfiles/travis.docker ENV LC_ALL en_US.UTF-8 · line 12

Quoted 9 lines
tests/docker/dockerfiles/travis.docker +5 -1
@@ -5,7 +5,15 @@ ENV LC_ALL en_US.UTF-8
 RUN sed -i "s/# deb-src/deb-src/" /etc/apt/sources.list
 RUN apt-get update
 RUN apt-get -y build-dep qemu
-RUN apt-get -y install device-tree-compiler python2.7 python-yaml dh-autoreconf gdb strace lsof net-tools gcovr
+RUN apt-get -y install \
+    device-tree-compiler \
+    dh-autoreconf \
+    gcovr \
+    gdb strace \

Two in a single line here.

Quoted 11 lines
tests/docker/dockerfiles/travis.docker +4 -0
+    lsof \
+    net-tools \
+    python2.7 \
+    python-yaml
 # Travis tools require PhantomJS / Neo4j / Maven accessible
 # in their PATH (QEMU build won't access them).
 ENV PATH /usr/local/phantomjs/bin:/usr/local/phantomjs:/usr/local/neo4j-3.2.7/bin:/usr/local/maven-3.5.2/bin:/usr/local/cmake-3.9.2/bin:/usr/local/clang-5.0.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
Signature · 1 line
2.20.1
#3 lore raw

PATCH 2/4 tests/docker: update our Travis image to run acceptance tests locally

In reply to Philippe Mathieu-Daudé

Since commit aa983ff67c3, Travis CI runs acceptance tests using the Avocado framework. Since Avocado requires Python 3, update our Docker image to be able to run these tests locally.

---
 tests/docker/dockerfiles/travis.docker | 2 ++
 1 file changed, 2 insertions(+)
tests/docker/dockerfiles/travis.docker +2 -0
index 35714664a1..b5c5036534 100644
@@ -13,6 +13,8 @@ RUN apt-get -y install \
     lsof \
     net-tools \
     python2.7 \
+    python3-pip \
+    python3.5-venv \
     python-yaml
 # Travis tools require PhantomJS / Neo4j / Maven accessible
 # in their PATH (QEMU build won't access them).
Signature · 1 line
2.20.1
Tested-by Reviewed-by on v1 2/4
#4 lore raw

Re: [Qemu-devel] [PATCH 2/4] tests/docker: update our Travis image to run acceptance tests locally

Quoted 26 lines · Philippe Mathieu-Daudé

Since commit aa983ff67c3, Travis CI runs acceptance tests using the Avocado framework. Since Avocado requires Python 3, update our Docker image to be able to run these tests locally.

---
 tests/docker/dockerfiles/travis.docker | 2 ++
 1 file changed, 2 insertions(+)
tests/docker/dockerfiles/travis.docker +2 -0
index 35714664a1..b5c5036534 100644
@@ -13,6 +13,8 @@ RUN apt-get -y install \
     lsof \
     net-tools \
     python2.7 \
+    python3-pip \
+    python3.5-venv \
     python-yaml
 # Travis tools require PhantomJS / Neo4j / Maven accessible
 # in their PATH (QEMU build won't access them).
Signature · 1 line
2.20.1
#5 lore raw

PATCH 3/4 tests/docker: Add test-acceptance runner

In reply to Philippe Mathieu-Daudé

Add a runner script to be able to run acceptance tests within Docker images. We can now reproduce Travis CI builds locally (and debug them!).

---
 tests/docker/test-acceptance | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)
 create mode 100755 tests/docker/test-acceptance
tests/docker/test-acceptance new file +21 -0
new file mode 100755
index 0000000000..84edaa676c
@@ -0,0 +1,21 @@
+#!/bin/bash -e
+#
+# Compile default Travis-CI target and run Avocado acceptance tests
+#
+# Copyright (c) 2019 Red Hat Inc.
+#
+# Authors:
+#  Philippe Mathieu-Daudé <[email protected]>
+#
+# This work is licensed under the terms of the GNU GPL, version 2
+# or (at your option) any later version. See the COPYING file in
+# the top-level directory.
+
+. common.rc
+
+cd "$BUILD_DIR"
+
+DEF_TARGET_LIST="x86_64-softmmu,mips-softmmu,mips64el-softmmu,aarch64-softmmu,arm-softmmu,s390x-softmmu,alpha-softmmu"
+TARGET_LIST=${TARGET_LIST:-$DEF_TARGET_LIST} \
+build_qemu
+check_qemu check-acceptance
Signature · 1 line
2.20.1
#6 lore raw

Re: [Qemu-devel] [PATCH 3/4] tests/docker: Add test-acceptance runner

Quoted 41 lines · Philippe Mathieu-Daudé

Add a runner script to be able to run acceptance tests within Docker images. We can now reproduce Travis CI builds locally (and debug them!).

---
 tests/docker/test-acceptance | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)
 create mode 100755 tests/docker/test-acceptance
tests/docker/test-acceptance new file +21 -0
new file mode 100755
index 0000000000..84edaa676c
@@ -0,0 +1,21 @@
+#!/bin/bash -e
+#
+# Compile default Travis-CI target and run Avocado acceptance tests
+#
+# Copyright (c) 2019 Red Hat Inc.
+#
+# Authors:
+#  Philippe Mathieu-Daudé <[email protected]>
+#
+# This work is licensed under the terms of the GNU GPL, version 2
+# or (at your option) any later version. See the COPYING file in
+# the top-level directory.
+
+. common.rc
+
+cd "$BUILD_DIR"
+
+DEF_TARGET_LIST="x86_64-softmmu,mips-softmmu,mips64el-softmmu,aarch64-softmmu,arm-softmmu,s390x-softmmu,alpha-softmmu"
+TARGET_LIST=${TARGET_LIST:-$DEF_TARGET_LIST} \
+build_qemu
+check_qemu check-acceptance
Signature · 1 line
2.20.1

I'm currently seeing test errors when running in a container:

MKDIR /tmp/qemu-test/build/tests/results
AVOCADO tests/acceptance
JOB ID : fe56cc0b2d1adbc0b5bb5828902e113d596edccf
JOB LOG : /tmp/qemu-test/build/tests/results/job-2019-08-19T22.13-fe56cc0/job.log
(01/27) /tmp/qemu-test/src/tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_x86_64_pc: ERROR: join() argument must be str or bytes, not 'NoneType' (0.05 s)
Interrupting job (failfast).
RESULTS : PASS 0 | ERROR 1 | FAIL 0 | SKIP 26 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB TIME : 0.26 s
/tmp/qemu-test/src/tests/Makefile.include:1158: recipe for target 'check-acceptance' failed
make: *** [check-acceptance] Error 9

That being said, I'm not running it under docker, but under podman, although I'm not convinced yet that is the defining issue. I'll try to identify what's going here.

Cheers,
- Cleber.

#7 lore raw

Re: [Qemu-devel] [PATCH 3/4] tests/docker: Add test-acceptance runner

Quoted 56 lines · Cleber Rosa
Quoted 41 lines · Philippe Mathieu-Daudé

Add a runner script to be able to run acceptance tests within Docker images. We can now reproduce Travis CI builds locally (and debug them!).

---
 tests/docker/test-acceptance | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)
 create mode 100755 tests/docker/test-acceptance
tests/docker/test-acceptance new file +21 -0
new file mode 100755
index 0000000000..84edaa676c
@@ -0,0 +1,21 @@
+#!/bin/bash -e
+#
+# Compile default Travis-CI target and run Avocado acceptance tests
+#
+# Copyright (c) 2019 Red Hat Inc.
+#
+# Authors:
+#  Philippe Mathieu-Daudé <[email protected]>
+#
+# This work is licensed under the terms of the GNU GPL, version 2
+# or (at your option) any later version. See the COPYING file in
+# the top-level directory.
+
+. common.rc
+
+cd "$BUILD_DIR"
+
+DEF_TARGET_LIST="x86_64-softmmu,mips-softmmu,mips64el-softmmu,aarch64-softmmu,arm-softmmu,s390x-softmmu,alpha-softmmu"
+TARGET_LIST=${TARGET_LIST:-$DEF_TARGET_LIST} \
+build_qemu
+check_qemu check-acceptance
Signature · 1 line
2.20.1

I'm currently seeing test errors when running in a container:

MKDIR /tmp/qemu-test/build/tests/results
AVOCADO tests/acceptance
JOB ID : fe56cc0b2d1adbc0b5bb5828902e113d596edccf
JOB LOG : /tmp/qemu-test/build/tests/results/job-2019-08-19T22.13-fe56cc0/job.log
(01/27) /tmp/qemu-test/src/tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_x86_64_pc: ERROR: join() argument must be str or bytes, not 'NoneType' (0.05 s)
Interrupting job (failfast).
RESULTS : PASS 0 | ERROR 1 | FAIL 0 | SKIP 26 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB TIME : 0.26 s
/tmp/qemu-test/src/tests/Makefile.include:1158: recipe for target 'check-acceptance' failed
make: *** [check-acceptance] Error 9

Quoted 3 lines

That being said, I'm not running it under docker, but under podman, although I'm not convinced yet that is the defining issue. I'll try to identify what's going here.

Was there a conclusion to the discussion about this series?

Signature · 1 line
Alex Bennée
#8 lore raw

Re: [Qemu-devel] [PATCH 3/4] tests/docker: Add test-acceptance runner

Quoted 57 lines · Alex Bennée
Quoted 56 lines · Cleber Rosa
Quoted 41 lines · Philippe Mathieu-Daudé

Add a runner script to be able to run acceptance tests within Docker images. We can now reproduce Travis CI builds locally (and debug them!).

---
  tests/docker/test-acceptance | 21 +++++++++++++++++++++
  1 file changed, 21 insertions(+)

create mode 100755 tests/docker/test-acceptance

tests/docker/test-acceptance new file +21 -0
new file mode 100755
index 0000000000..84edaa676c
@@ -0,0 +1,21 @@
+#!/bin/bash -e
+#
+# Compile default Travis-CI target and run Avocado acceptance tests
+#
+# Copyright (c) 2019 Red Hat Inc.
+#
+# Authors:
+#  Philippe Mathieu-Daudé <[email protected]>
+#
+# This work is licensed under the terms of the GNU GPL, version 2
+# or (at your option) any later version. See the COPYING file in
+# the top-level directory.
+
+. common.rc
+
+cd "$BUILD_DIR"
+
+DEF_TARGET_LIST="x86_64-softmmu,mips-softmmu,mips64el-softmmu,aarch64-softmmu,arm-softmmu,s390x-softmmu,alpha-softmmu"
+TARGET_LIST=${TARGET_LIST:-$DEF_TARGET_LIST} \
+build_qemu
+check_qemu check-acceptance
Signature · 1 line
2.20.1

I'm currently seeing test errors when running in a container:

MKDIR /tmp/qemu-test/build/tests/results
AVOCADO tests/acceptance
JOB ID : fe56cc0b2d1adbc0b5bb5828902e113d596edccf
JOB LOG : /tmp/qemu-test/build/tests/results/job-2019-08-19T22.13-fe56cc0/job.log
(01/27) /tmp/qemu-test/src/tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_x86_64_pc: ERROR: join() argument must be str or bytes, not 'NoneType' (0.05 s)
Interrupting job (failfast).
RESULTS : PASS 0 | ERROR 1 | FAIL 0 | SKIP 26 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB TIME : 0.26 s
/tmp/qemu-test/src/tests/Makefile.include:1158: recipe for target 'check-acceptance' failed
make: *** [check-acceptance] Error 9

Quoted 5 lines
Quoted 3 lines

That being said, I'm not running it under docker, but under podman, although I'm not convinced yet that is the defining issue. I'll try to identify what's going here.

Was there a conclusion to the discussion about this series?

No sure if there was a conclusion...

tests/docker/ is listed as 'Maintained' but the listed maintainers only use Docker. If Podman is supported, then maybe we need another maintainer for Podman? We are not testing it so far.

#9 lore raw

PATCH 4/4 tests/docker: Use Travis-CI default environment variables

In reply to Philippe Mathieu-Daudé

Update to environment variables of our Docker image to closely reproduce the builds run by the Travis-CI service.

Default variables from:
https://docs.travis-ci.com/user/environment-variables/#default-environment-variables

---
 tests/docker/dockerfiles/travis.docker | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)
tests/docker/dockerfiles/travis.docker +7 -3
index b5c5036534..a0e8152317 100644
@@ -1,7 +1,11 @@
 FROM travisci/ci-sardonyx:packer-1546978056-2c98a19
-ENV DEBIAN_FRONTEND noninteractive
-ENV LANG en_US.UTF-8
-ENV LC_ALL en_US.UTF-8
+ENV CI=true \
+    TRAVIS=true \
+    CONTINUOUS_INTEGRATION=true \
+    DEBIAN_FRONTEND=noninteractive \
+    LANG=en_US.UTF-8 \
+    LC_ALL=en_US.UTF-8 \
+    TRAVIS_OS_NAME=linux
 RUN sed -i "s/# deb-src/deb-src/" /etc/apt/sources.list
 RUN apt-get update
 RUN apt-get -y build-dep qemu
Signature · 1 line
2.20.1