Skip to contentSkip to conversation

docker: misc cleanups

PATCH In review John Snow 3 participants 13 messages Sep 23, 2019 – Sep 24, 2019

lore
Share & tools
lore

John Snow

author
#0 lore raw

v3 cover docker: misc cleanups

This should include anything I've sent so far (not including the RFC for more exploratory changes) that hasn't already been included in a roundup by Alex Bennée.

Mostly, it removes unused docker files and replaces python2 with python3 in all of the container environments.

V3:
- Reorder patches to remove everything first, THEN patch for python3
- Split removal of debian-ports and debian-sid into two patches.

V2:
- Rolled in python2 --> python3 conversion
- Added travis conversion to python patch
- Remove debian8 dockerfile, which is also now unused.

John Snow (6):
docker: remove debian8-mxe definitions
docker: removed unused debian8 partial image
docker: remove 'deprecated' image definitions
docker: remove unused debian-ports
docker: remove unused debian-sid
docker: move tests from python2 to python3

 tests/docker/Makefile.include                 |  9 +++--
 tests/docker/dockerfiles/centos7.docker       |  2 +-
 tests/docker/dockerfiles/debian-ports.docker  | 36 -------------------
 tests/docker/dockerfiles/debian-sid.docker    | 35 ------------------
 .../dockerfiles/debian-xtensa-cross.docker    |  2 +-
 tests/docker/dockerfiles/debian10.docker      |  2 +-
 tests/docker/dockerfiles/debian8.docker       | 34 ------------------
 tests/docker/dockerfiles/debian9.docker       |  2 +-
 tests/docker/dockerfiles/travis.docker        |  2 +-
 tests/docker/dockerfiles/ubuntu.docker        |  2 +-
 tests/docker/dockerfiles/ubuntu1804.docker    |  2 +-
 11 files changed, 11 insertions(+), 117 deletions(-)
 delete mode 100644 tests/docker/dockerfiles/debian-ports.docker
 delete mode 100644 tests/docker/dockerfiles/debian-sid.docker
 delete mode 100644 tests/docker/dockerfiles/debian8.docker
Signature · 1 line
2.21.0
author
#1 lore raw

PATCH v3 1/6 docker: remove debian8-mxe definitions

We don't have a debian8-mxe dockerfile anymore.

---
 tests/docker/Makefile.include | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tests/docker/Makefile.include +1 -1
index d796f71b63..485de7f59d 100644
@@ -7,7 +7,7 @@ DOCKER_FILES_DIR := $(SRC_PATH)/tests/docker/dockerfiles
 DOCKER_DEPRECATED_IMAGES := debian
 # we don't run tests on intermediate images (used as base by another image)
 DOCKER_PARTIAL_IMAGES := debian debian8 debian9 debian10 debian-sid
-DOCKER_PARTIAL_IMAGES += debian8-mxe debian9-mxe debian-ports debian-bootstrap
+DOCKER_PARTIAL_IMAGES += debian9-mxe debian-ports debian-bootstrap
 DOCKER_IMAGES := $(filter-out $(DOCKER_DEPRECATED_IMAGES),$(sort $(notdir $(basename $(wildcard $(DOCKER_FILES_DIR)/*.docker)))))
 DOCKER_TARGETS := $(patsubst %,docker-image-%,$(DOCKER_IMAGES))
 # Use a global constant ccache directory to speed up repetitive builds
Signature · 1 line
2.21.0
author
#2 lore raw

PATCH v3 2/6 docker: removed unused debian8 partial image

In reply to John Snow

debian8 partial base is also not consumed by any image, so remove it. For QEMU's development cycle, we only support debian9 (stretch) and debian10 (buster).

---
 tests/docker/Makefile.include           |  2 +-
 tests/docker/dockerfiles/debian8.docker | 34 -------------------------
 2 files changed, 1 insertion(+), 35 deletions(-)
 delete mode 100644 tests/docker/dockerfiles/debian8.docker
tests/docker/Makefile.include +1 -1
index 485de7f59d..b20a810016 100644
@@ -6,7 +6,7 @@ DOCKER_SUFFIX := .docker
 DOCKER_FILES_DIR := $(SRC_PATH)/tests/docker/dockerfiles
 DOCKER_DEPRECATED_IMAGES := debian
 # we don't run tests on intermediate images (used as base by another image)
-DOCKER_PARTIAL_IMAGES := debian debian8 debian9 debian10 debian-sid
+DOCKER_PARTIAL_IMAGES := debian debian9 debian10 debian-sid
 DOCKER_PARTIAL_IMAGES += debian9-mxe debian-ports debian-bootstrap
 DOCKER_IMAGES := $(filter-out $(DOCKER_DEPRECATED_IMAGES),$(sort $(notdir $(basename $(wildcard $(DOCKER_FILES_DIR)/*.docker)))))
 DOCKER_TARGETS := $(patsubst %,docker-image-%,$(DOCKER_IMAGES))
tests/docker/dockerfiles/debian8.docker deleted +0 -34
deleted file mode 100644
index 1212a85c35..0000000000
@@ -1,34 +0,0 @@
-#
-# Docker multiarch cross-compiler target
-#
-# This docker target is builds on Debian and Emdebian's cross compiler targets
-# to build distro with a selection of cross compilers for building test binaries.
-#
-# On its own you can't build much but the docker-foo-cross targets
-# build on top of the base debian image.
-#
-FROM debian:jessie-slim
-
-MAINTAINER Philippe Mathieu-Daudé <[email protected]>
-
-# Duplicate deb line as deb-src
-RUN cat /etc/apt/sources.list | sed "s/^deb\ /deb-src /" >> /etc/apt/sources.list
-
-# Setup some basic tools we need
-RUN apt update && \
-    DEBIAN_FRONTEND=noninteractive apt-get install -yy eatmydata && \
-    DEBIAN_FRONTEND=noninteractive eatmydata \
-    apt-get install -y --no-install-recommends \
-        bison \
-        binutils-multiarch \
-        build-essential \
-        ca-certificates \
-        clang \
-        curl \
-        flex \
-        gettext \
-        git \
-        gnupg \
-        pkg-config \
-        python-minimal
-
Signature · 1 line
2.21.0
author
#3 lore raw

PATCH v3 3/6 docker: remove 'deprecated' image definitions

In reply to John Snow

There isn't a debian.dockerfile anymore,
so perform some ghost-busting.

---
 tests/docker/Makefile.include | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)
tests/docker/Makefile.include +3 -4
index b20a810016..983224ad96 100644
@@ -4,11 +4,10 @@
 
 DOCKER_SUFFIX := .docker
 DOCKER_FILES_DIR := $(SRC_PATH)/tests/docker/dockerfiles
-DOCKER_DEPRECATED_IMAGES := debian
 # we don't run tests on intermediate images (used as base by another image)
-DOCKER_PARTIAL_IMAGES := debian debian9 debian10 debian-sid
+DOCKER_PARTIAL_IMAGES := debian9 debian10 debian-sid
 DOCKER_PARTIAL_IMAGES += debian9-mxe debian-ports debian-bootstrap
-DOCKER_IMAGES := $(filter-out $(DOCKER_DEPRECATED_IMAGES),$(sort $(notdir $(basename $(wildcard $(DOCKER_FILES_DIR)/*.docker)))))
+DOCKER_IMAGES := $(sort $(notdir $(basename $(wildcard $(DOCKER_FILES_DIR)/*.docker))))
 DOCKER_TARGETS := $(patsubst %,docker-image-%,$(DOCKER_IMAGES))
 # Use a global constant ccache directory to speed up repetitive builds
 DOCKER_CCACHE_DIR := $$HOME/.cache/qemu-docker-ccache
@@ -159,7 +158,7 @@ docker-image-debian-powerpc-user-cross: docker-binfmt-image-debian-powerpc-user
 DOCKER_USER_IMAGES += debian-powerpc-user
 
 # Expand all the pre-requistes for each docker image and test combination
-$(foreach i,$(filter-out $(DOCKER_PARTIAL_IMAGES),$(DOCKER_IMAGES) $(DOCKER_DEPRECATED_IMAGES)), \
+$(foreach i,$(filter-out $(DOCKER_PARTIAL_IMAGES),$(DOCKER_IMAGES)), \
 	$(foreach t,$(DOCKER_TESTS) $(DOCKER_TOOLS), \
 		$(eval .PHONY: docker-$t@$i) \
 		$(eval docker-$t@$i: docker-image-$i docker-run-$t@$i) \
Signature · 1 line
2.21.0
Reviewed-by on v3 3/6
#4 lore raw

Re: [PATCH v3 3/6] docker: remove 'deprecated' image definitions

Quoted 28 lines · John Snow

There isn't a debian.dockerfile anymore,
so perform some ghost-busting.

---
 tests/docker/Makefile.include | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)
tests/docker/Makefile.include +2 -3
index b20a810016..983224ad96 100644
@@ -4,11 +4,10 @@
 
 DOCKER_SUFFIX := .docker
 DOCKER_FILES_DIR := $(SRC_PATH)/tests/docker/dockerfiles
-DOCKER_DEPRECATED_IMAGES := debian
 # we don't run tests on intermediate images (used as base by another image)
-DOCKER_PARTIAL_IMAGES := debian debian9 debian10 debian-sid
+DOCKER_PARTIAL_IMAGES := debian9 debian10 debian-sid
 DOCKER_PARTIAL_IMAGES += debian9-mxe debian-ports debian-bootstrap
-DOCKER_IMAGES := $(filter-out $(DOCKER_DEPRECATED_IMAGES),$(sort $(notdir $(basename $(wildcard $(DOCKER_FILES_DIR)/*.docker)))))
+DOCKER_IMAGES := $(sort $(notdir $(basename $(wildcard $(DOCKER_FILES_DIR)/*.docker))))
 DOCKER_TARGETS := $(patsubst %,docker-image-%,$(DOCKER_IMAGES))
 # Use a global constant ccache directory to speed up repetitive builds
 DOCKER_CCACHE_DIR := $$HOME/.cache/qemu-docker-ccache

tests/docker/Makefile.include docker-image-debian-powerpc-user-cross: docker-binfmt-image-debian-powerpc-user · line 164

Quoted 8 lines
tests/docker/Makefile.include +1 -1
@@ -159,7 +158,7 @@ docker-image-debian-powerpc-user-cross: docker-binfmt-image-debian-powerpc-user
 DOCKER_USER_IMAGES += debian-powerpc-user
 
 # Expand all the pre-requistes for each docker image and test combination
-$(foreach i,$(filter-out $(DOCKER_PARTIAL_IMAGES),$(DOCKER_IMAGES) $(DOCKER_DEPRECATED_IMAGES)), \
+$(foreach i,$(filter-out $(DOCKER_PARTIAL_IMAGES),$(DOCKER_IMAGES)), \
 	$(foreach t,$(DOCKER_TESTS) $(DOCKER_TOOLS), \
 		$(eval .PHONY: docker-$t@$i) \
 		$(eval docker-$t@$i: docker-image-$i docker-run-$t@$i) \
author
#5 lore raw

PATCH v3 4/6 docker: remove unused debian-ports

In reply to John Snow

debian-ports is listed as a partial image, so we cannot run tests against it. Since it isn't used by any other testable image, remove it for now as it is prone to bitrot.

---
 tests/docker/Makefile.include                |  2 +-
 tests/docker/dockerfiles/debian-ports.docker | 36 --------------------
 2 files changed, 1 insertion(+), 37 deletions(-)
 delete mode 100644 tests/docker/dockerfiles/debian-ports.docker
tests/docker/Makefile.include +1 -1
index 983224ad96..7bcd97101e 100644
@@ -6,7 +6,7 @@ DOCKER_SUFFIX := .docker
 DOCKER_FILES_DIR := $(SRC_PATH)/tests/docker/dockerfiles
 # we don't run tests on intermediate images (used as base by another image)
 DOCKER_PARTIAL_IMAGES := debian9 debian10 debian-sid
-DOCKER_PARTIAL_IMAGES += debian9-mxe debian-ports debian-bootstrap
+DOCKER_PARTIAL_IMAGES += debian9-mxe debian-bootstrap
 DOCKER_IMAGES := $(sort $(notdir $(basename $(wildcard $(DOCKER_FILES_DIR)/*.docker))))
 DOCKER_TARGETS := $(patsubst %,docker-image-%,$(DOCKER_IMAGES))
 # Use a global constant ccache directory to speed up repetitive builds
tests/docker/dockerfiles/debian-ports.docker deleted +0 -36
deleted file mode 100644
index 61bc3f2993..0000000000
@@ -1,36 +0,0 @@
-#
-# Docker multiarch cross-compiler target
-#
-# This docker target is builds on Debian Ports cross compiler targets
-# to build distro with a selection of cross compilers for building test binaries.
-#
-# On its own you can't build much but the docker-foo-cross targets
-# build on top of the base debian image.
-#
-FROM debian:unstable
-
-MAINTAINER Philippe Mathieu-Daudé <[email protected]>
-
-RUN echo "deb [arch=amd64] http://deb.debian.org/debian unstable main" > /etc/apt/sources.list
-
-# Duplicate deb line as deb-src
-RUN cat /etc/apt/sources.list | sed -ne "s/^deb\ \(\[.*\]\ \)\?\(.*\)/deb-src \2/p" >> /etc/apt/sources.list
-
-# Setup some basic tools we need
-RUN apt-get update && \
-    DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata && \
-    DEBIAN_FRONTEND=noninteractive eatmydata \
-    apt-get install -y --no-install-recommends \
-        bison \
-        build-essential \
-        ca-certificates \
-        clang \
-        debian-ports-archive-keyring \
-        flex \
-        gettext \
-        git \
-        pkg-config \
-        psmisc \
-        python \
-        texinfo \
-        $(apt-get -s build-dep qemu | egrep ^Inst | fgrep '[all]' | cut -d\  -f2)
Signature · 1 line
2.21.0
Acked-by on v3 4/6
#6 lore raw

Re: [PATCH v3 4/6] docker: remove unused debian-ports

Quoted 57 lines · John Snow

debian-ports is listed as a partial image, so we cannot run tests against it. Since it isn't used by any other testable image, remove it for now as it is prone to bitrot.

---
 tests/docker/Makefile.include                |  2 +-
 tests/docker/dockerfiles/debian-ports.docker | 36 --------------------
 2 files changed, 1 insertion(+), 37 deletions(-)
 delete mode 100644 tests/docker/dockerfiles/debian-ports.docker
tests/docker/Makefile.include +1 -1
index 983224ad96..7bcd97101e 100644
@@ -6,7 +6,7 @@ DOCKER_SUFFIX := .docker
 DOCKER_FILES_DIR := $(SRC_PATH)/tests/docker/dockerfiles
 # we don't run tests on intermediate images (used as base by another image)
 DOCKER_PARTIAL_IMAGES := debian9 debian10 debian-sid
-DOCKER_PARTIAL_IMAGES += debian9-mxe debian-ports debian-bootstrap
+DOCKER_PARTIAL_IMAGES += debian9-mxe debian-bootstrap
 DOCKER_IMAGES := $(sort $(notdir $(basename $(wildcard $(DOCKER_FILES_DIR)/*.docker))))
 DOCKER_TARGETS := $(patsubst %,docker-image-%,$(DOCKER_IMAGES))
 # Use a global constant ccache directory to speed up repetitive builds
tests/docker/dockerfiles/debian-ports.docker deleted +0 -26
deleted file mode 100644
index 61bc3f2993..0000000000
@@ -1,36 +0,0 @@
-#
-# Docker multiarch cross-compiler target
-#
-# This docker target is builds on Debian Ports cross compiler targets
-# to build distro with a selection of cross compilers for building test binaries.
-#
-# On its own you can't build much but the docker-foo-cross targets
-# build on top of the base debian image.
-#
-FROM debian:unstable
-
-MAINTAINER Philippe Mathieu-Daudé <[email protected]>
-
-RUN echo "deb [arch=amd64] http://deb.debian.org/debian unstable main" > /etc/apt/sources.list
-
-# Duplicate deb line as deb-src
-RUN cat /etc/apt/sources.list | sed -ne "s/^deb\ \(\[.*\]\ \)\?\(.*\)/deb-src \2/p" >> /etc/apt/sources.list
-
-# Setup some basic tools we need
-RUN apt-get update && \
-    DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata && \
-    DEBIAN_FRONTEND=noninteractive eatmydata \
-    apt-get install -y --no-install-recommends \
-        bison \
-        build-essential \
-        ca-certificates \

tests/docker/dockerfiles/debian-ports.docker line 36

Quoted 10 lines
tests/docker/dockerfiles/debian-ports.docker deleted +0 -10
-        clang \
-        debian-ports-archive-keyring \
-        flex \
-        gettext \
-        git \
-        pkg-config \
-        psmisc \
-        python \
-        texinfo \
-        $(apt-get -s build-dep qemu | egrep ^Inst | fgrep '[all]' | cut -d\  -f2)
author
#7 lore raw

PATCH v3 5/6 docker: remove unused debian-sid

In reply to John Snow

debian-sid is listed as a partial image, so we cannot run tests against it. Since it isn't used by any other testable image, remove it for now as it is prone to bitrot.

---
 tests/docker/Makefile.include              |  2 +-
 tests/docker/dockerfiles/debian-sid.docker | 35 ----------------------
 2 files changed, 1 insertion(+), 36 deletions(-)
 delete mode 100644 tests/docker/dockerfiles/debian-sid.docker
tests/docker/Makefile.include +1 -1
index 7bcd97101e..d7b1082b5b 100644
@@ -5,7 +5,7 @@
 DOCKER_SUFFIX := .docker
 DOCKER_FILES_DIR := $(SRC_PATH)/tests/docker/dockerfiles
 # we don't run tests on intermediate images (used as base by another image)
-DOCKER_PARTIAL_IMAGES := debian9 debian10 debian-sid
+DOCKER_PARTIAL_IMAGES := debian9 debian10
 DOCKER_PARTIAL_IMAGES += debian9-mxe debian-bootstrap
 DOCKER_IMAGES := $(sort $(notdir $(basename $(wildcard $(DOCKER_FILES_DIR)/*.docker))))
 DOCKER_TARGETS := $(patsubst %,docker-image-%,$(DOCKER_IMAGES))
tests/docker/dockerfiles/debian-sid.docker deleted +0 -35
deleted file mode 100644
index 2a1bcc33b2..0000000000
@@ -1,35 +0,0 @@
-#
-# Debian Sid Base
-#
-# Currently we can build all our guests with cross-compilers in the
-# latest Debian release (Buster). However new compilers will first
-# arrive in Sid. However Sid is a rolling distro which may be broken
-# at any particular time. To try and mitigate this we use Debian's
-# snapshot archive which provides a "stable" view of what state Sid
-# was in.
-#
-
-# This must be earlier than the snapshot date we are aiming for
-FROM debian:sid-20190812-slim
-
- # Use a snapshot known to work (see http://snapshot.debian.org/#Usage)
-ENV DEBIAN_SNAPSHOT_DATE "20190820"
-RUN sed -i "s%^deb \(https\?://\)deb.debian.org/debian/\? \(.*\)%deb [check-valid-until=no] \1snapshot.debian.org/archive/debian/${DEBIAN_SNAPSHOT_DATE} \2%" /etc/apt/sources.list
-
-# Duplicate deb line as deb-src
-RUN cat /etc/apt/sources.list | sed "s/^deb\ /deb-src /" >> /etc/apt/sources.list
-
-# Install common build utilities
-RUN apt update && \
-    DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata && \
-    DEBIAN_FRONTEND=noninteractive eatmydata \
-    apt install -y --no-install-recommends \
-        bison \
-        build-essential \
-        ca-certificates \
-        flex \
-        git \
-        pkg-config \
-        psmisc \
-        python \
-        texinfo || { echo "Failed to build - see debian-sid.docker notes"; exit 1; }
Signature · 1 line
2.21.0
Acked-by on v3 5/6
#8 lore raw

Re: [PATCH v3 5/6] docker: remove unused debian-sid

Quoted 56 lines · John Snow

debian-sid is listed as a partial image, so we cannot run tests against it. Since it isn't used by any other testable image, remove it for now as it is prone to bitrot.

---
 tests/docker/Makefile.include              |  2 +-
 tests/docker/dockerfiles/debian-sid.docker | 35 ----------------------
 2 files changed, 1 insertion(+), 36 deletions(-)
 delete mode 100644 tests/docker/dockerfiles/debian-sid.docker
tests/docker/Makefile.include +1 -1
index 7bcd97101e..d7b1082b5b 100644
@@ -5,7 +5,7 @@
 DOCKER_SUFFIX := .docker
 DOCKER_FILES_DIR := $(SRC_PATH)/tests/docker/dockerfiles
 # we don't run tests on intermediate images (used as base by another image)
-DOCKER_PARTIAL_IMAGES := debian9 debian10 debian-sid
+DOCKER_PARTIAL_IMAGES := debian9 debian10
 DOCKER_PARTIAL_IMAGES += debian9-mxe debian-bootstrap
 DOCKER_IMAGES := $(sort $(notdir $(basename $(wildcard $(DOCKER_FILES_DIR)/*.docker))))
 DOCKER_TARGETS := $(patsubst %,docker-image-%,$(DOCKER_IMAGES))
tests/docker/dockerfiles/debian-sid.docker deleted +0 -25
deleted file mode 100644
index 2a1bcc33b2..0000000000
@@ -1,35 +0,0 @@
-#
-# Debian Sid Base
-#
-# Currently we can build all our guests with cross-compilers in the
-# latest Debian release (Buster). However new compilers will first
-# arrive in Sid. However Sid is a rolling distro which may be broken
-# at any particular time. To try and mitigate this we use Debian's
-# snapshot archive which provides a "stable" view of what state Sid
-# was in.
-#
-
-# This must be earlier than the snapshot date we are aiming for
-FROM debian:sid-20190812-slim
-
- # Use a snapshot known to work (see http://snapshot.debian.org/#Usage)
-ENV DEBIAN_SNAPSHOT_DATE "20190820"
-RUN sed -i "s%^deb \(https\?://\)deb.debian.org/debian/\? \(.*\)%deb [check-valid-until=no] \1snapshot.debian.org/archive/debian/${DEBIAN_SNAPSHOT_DATE} \2%" /etc/apt/sources.list
-
-# Duplicate deb line as deb-src
-RUN cat /etc/apt/sources.list | sed "s/^deb\ /deb-src /" >> /etc/apt/sources.list
-
-# Install common build utilities
-RUN apt update && \
-    DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata && \
-    DEBIAN_FRONTEND=noninteractive eatmydata \

tests/docker/dockerfiles/debian-sid.docker line 35

Quoted 10 lines
tests/docker/dockerfiles/debian-sid.docker deleted +0 -10
-    apt install -y --no-install-recommends \
-        bison \
-        build-essential \
-        ca-certificates \
-        flex \
-        git \
-        pkg-config \
-        psmisc \
-        python \
-        texinfo || { echo "Failed to build - see debian-sid.docker notes"; exit 1; }
author
#9 lore raw
In reply to John Snow

CC ehabkost for patch 6/6.

Quoted 40 lines · John Snow

This should include anything I've sent so far (not including the RFC for more exploratory changes) that hasn't already been included in a roundup by Alex Bennée.

Mostly, it removes unused docker files and replaces python2 with python3 in all of the container environments.

V3:
- Reorder patches to remove everything first, THEN patch for python3
- Split removal of debian-ports and debian-sid into two patches.

V2:
- Rolled in python2 --> python3 conversion
- Added travis conversion to python patch
- Remove debian8 dockerfile, which is also now unused.

John Snow (6):
docker: remove debian8-mxe definitions
docker: removed unused debian8 partial image
docker: remove 'deprecated' image definitions
docker: remove unused debian-ports
docker: remove unused debian-sid
docker: move tests from python2 to python3

 tests/docker/Makefile.include                 |  9 +++--
 tests/docker/dockerfiles/centos7.docker       |  2 +-
 tests/docker/dockerfiles/debian-ports.docker  | 36 -------------------
 tests/docker/dockerfiles/debian-sid.docker    | 35 ------------------
 .../dockerfiles/debian-xtensa-cross.docker    |  2 +-
 tests/docker/dockerfiles/debian10.docker      |  2 +-
 tests/docker/dockerfiles/debian8.docker       | 34 ------------------
 tests/docker/dockerfiles/debian9.docker       |  2 +-
 tests/docker/dockerfiles/travis.docker        |  2 +-
 tests/docker/dockerfiles/ubuntu.docker        |  2 +-
 tests/docker/dockerfiles/ubuntu1804.docker    |  2 +-
 11 files changed, 11 insertions(+), 117 deletions(-)
 delete mode 100644 tests/docker/dockerfiles/debian-ports.docker
 delete mode 100644 tests/docker/dockerfiles/debian-sid.docker
 delete mode 100644 tests/docker/dockerfiles/debian8.docker
Signature · 1 line
—js
author
#10 lore raw

PATCH v3 6/6 docker: move tests from python2 to python3

In reply to John Snow

As part of the push to drop python2 support, replace any explicit python2 dependencies with python3 versions.

For centos, python2 still exists as an implicit dependency, but by adding python3 we will be able to build even if the configure script begins to require python 3.5+.

Tested with centos7, fedora, ubuntu, ubuntu1804, and debian 9 (amd64). Tested under a custom configure script that requires Python 3.5+.

the travis dockerfile is also moved to using python3, which was tested by running make docker-test-build@travis, which I hope is sufficient.

---
 tests/docker/dockerfiles/centos7.docker             | 2 +-
 tests/docker/dockerfiles/debian-xtensa-cross.docker | 2 +-
 tests/docker/dockerfiles/debian10.docker            | 2 +-
 tests/docker/dockerfiles/debian9.docker             | 2 +-
 tests/docker/dockerfiles/travis.docker              | 2 +-
 tests/docker/dockerfiles/ubuntu.docker              | 2 +-
 tests/docker/dockerfiles/ubuntu1804.docker          | 2 +-
 7 files changed, 7 insertions(+), 7 deletions(-)
tests/docker/dockerfiles/centos7.docker +1 -1
index e0b9d7dbe9..953637065c 100644
@@ -25,6 +25,7 @@ ENV PACKAGES \
     nettle-devel \
     perl-Test-Harness \
     pixman-devel \
+    python3 \
     SDL-devel \
     spice-glib-devel \
     spice-server-devel \
@@ -34,4 +35,3 @@ ENV PACKAGES \
     zlib-devel
 RUN yum install -y $PACKAGES
 RUN rpm -q $PACKAGES | sort > /packages.txt
-
tests/docker/dockerfiles/debian-xtensa-cross.docker +1 -1
index b9c2e2e531..e6f93f65ee 100644
@@ -18,7 +18,7 @@ RUN apt-get update && \
         flex \
         gettext \
         git \
-        python-minimal
+        python3-minimal
 
 ENV CPU_LIST csp dc232b dc233c
 ENV TOOLCHAIN_RELEASE 2018.02
tests/docker/dockerfiles/debian10.docker +1 -1
index 30a78813f2..dad498b52e 100644
@@ -26,7 +26,7 @@ RUN apt update && \
         git \
         pkg-config \
         psmisc \
-        python \
+        python3 \
         python3-sphinx \
         texinfo \
         $(apt-get -s build-dep qemu | egrep ^Inst | fgrep '[all]' | cut -d\  -f2)
tests/docker/dockerfiles/debian9.docker +1 -1
index b36f1d4ed8..8cbd742bb5 100644
@@ -26,7 +26,7 @@ RUN apt update && \
         git \
         pkg-config \
         psmisc \
-        python \
+        python3 \
         python3-sphinx \
         texinfo \
         $(apt-get -s build-dep qemu | egrep ^Inst | fgrep '[all]' | cut -d\  -f2)
tests/docker/dockerfiles/travis.docker +1 -1
index e72dc85ca7..ea14da29d9 100644
@@ -5,7 +5,7 @@ ENV LC_ALL en_US.UTF-8
 RUN sed -i "s/# deb-src/deb-src/" /etc/apt/sources.list
 RUN apt-get update
 RUN apt-get -y build-dep qemu
-RUN apt-get -y install device-tree-compiler python2.7 python-yaml dh-autoreconf gdb strace lsof net-tools gcovr
+RUN apt-get -y install device-tree-compiler python3 python3-yaml dh-autoreconf gdb strace lsof net-tools gcovr
 # Travis tools require PhantomJS / Neo4j / Maven accessible
 # in their PATH (QEMU build won't access them).
 ENV PATH /usr/local/phantomjs/bin:/usr/local/phantomjs:/usr/local/neo4j-3.2.7/bin:/usr/local/maven-3.5.2/bin:/usr/local/cmake-3.9.2/bin:/usr/local/clang-5.0.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
tests/docker/dockerfiles/ubuntu.docker +1 -1
index a4f601395c..f486492224 100644
@@ -60,7 +60,7 @@ ENV PACKAGES flex bison \
     libvte-2.91-dev \
     libxen-dev \
     make \
-    python-yaml \
+    python3-yaml \
     python3-sphinx \
     sparse \
     texinfo \
tests/docker/dockerfiles/ubuntu1804.docker +1 -1
index 883f9bcf31..3cc4f492c4 100644
@@ -47,7 +47,7 @@ ENV PACKAGES flex bison \
     libvte-2.91-dev \
     libxen-dev \
     make \
-    python-yaml \
+    python3-yaml \
     python3-sphinx \
     sparse \
     texinfo \
Signature · 1 line
2.21.0
Reviewed-by Eduardo Habkost Reviewed-by Reviewed-by Cleber Rosa Tested-by Cleber Rosa on v3 6/6
#11 lore raw

Re: [PATCH v3 6/6] docker: move tests from python2 to python3

Quoted 7 lines · John Snow

As part of the push to drop python2 support, replace any explicit python2 dependencies with python3 versions.

For centos, python2 still exists as an implicit dependency, but by adding python3 we will be able to build even if the configure script begins to require python 3.5+.

Quoted 7 lines

Tested with centos7, fedora, ubuntu, ubuntu1804, and debian 9 (amd64). Tested under a custom configure script that requires Python 3.5+.

the travis dockerfile is also moved to using python3, which was tested by running make docker-test-build@travis, which I hope is sufficient.

It seems this one already has:

Quoted 106 lines
---
 tests/docker/dockerfiles/centos7.docker             | 2 +-
 tests/docker/dockerfiles/debian-xtensa-cross.docker | 2 +-
 tests/docker/dockerfiles/debian10.docker            | 2 +-
 tests/docker/dockerfiles/debian9.docker             | 2 +-
 tests/docker/dockerfiles/travis.docker              | 2 +-
 tests/docker/dockerfiles/ubuntu.docker              | 2 +-
 tests/docker/dockerfiles/ubuntu1804.docker          | 2 +-
 7 files changed, 7 insertions(+), 7 deletions(-)
tests/docker/dockerfiles/centos7.docker +1 -1
index e0b9d7dbe9..953637065c 100644
@@ -25,6 +25,7 @@ ENV PACKAGES \
     nettle-devel \
     perl-Test-Harness \
     pixman-devel \
+    python3 \
     SDL-devel \
     spice-glib-devel \
     spice-server-devel \
@@ -34,4 +35,3 @@ ENV PACKAGES \
     zlib-devel
 RUN yum install -y $PACKAGES
 RUN rpm -q $PACKAGES | sort > /packages.txt
-
tests/docker/dockerfiles/debian-xtensa-cross.docker +1 -1
index b9c2e2e531..e6f93f65ee 100644
@@ -18,7 +18,7 @@ RUN apt-get update && \
         flex \
         gettext \
         git \
-        python-minimal
+        python3-minimal
 
 ENV CPU_LIST csp dc232b dc233c
 ENV TOOLCHAIN_RELEASE 2018.02
tests/docker/dockerfiles/debian10.docker +1 -1
index 30a78813f2..dad498b52e 100644
@@ -26,7 +26,7 @@ RUN apt update && \
         git \
         pkg-config \
         psmisc \
-        python \
+        python3 \
         python3-sphinx \
         texinfo \
         $(apt-get -s build-dep qemu | egrep ^Inst | fgrep '[all]' | cut -d\  -f2)
tests/docker/dockerfiles/debian9.docker +1 -1
index b36f1d4ed8..8cbd742bb5 100644
@@ -26,7 +26,7 @@ RUN apt update && \
         git \
         pkg-config \
         psmisc \
-        python \
+        python3 \
         python3-sphinx \
         texinfo \
         $(apt-get -s build-dep qemu | egrep ^Inst | fgrep '[all]' | cut -d\  -f2)
tests/docker/dockerfiles/travis.docker +1 -1
index e72dc85ca7..ea14da29d9 100644
@@ -5,7 +5,7 @@ ENV LC_ALL en_US.UTF-8
 RUN sed -i "s/# deb-src/deb-src/" /etc/apt/sources.list
 RUN apt-get update
 RUN apt-get -y build-dep qemu
-RUN apt-get -y install device-tree-compiler python2.7 python-yaml dh-autoreconf gdb strace lsof net-tools gcovr
+RUN apt-get -y install device-tree-compiler python3 python3-yaml dh-autoreconf gdb strace lsof net-tools gcovr
 # Travis tools require PhantomJS / Neo4j / Maven accessible
 # in their PATH (QEMU build won't access them).
 ENV PATH /usr/local/phantomjs/bin:/usr/local/phantomjs:/usr/local/neo4j-3.2.7/bin:/usr/local/maven-3.5.2/bin:/usr/local/cmake-3.9.2/bin:/usr/local/clang-5.0.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
tests/docker/dockerfiles/ubuntu.docker +1 -1
index a4f601395c..f486492224 100644
@@ -60,7 +60,7 @@ ENV PACKAGES flex bison \
     libvte-2.91-dev \
     libxen-dev \
     make \
-    python-yaml \
+    python3-yaml \
     python3-sphinx \
     sparse \
     texinfo \
tests/docker/dockerfiles/ubuntu1804.docker +1 -1
index 883f9bcf31..3cc4f492c4 100644
@@ -47,7 +47,7 @@ ENV PACKAGES flex bison \
     libvte-2.91-dev \
     libxen-dev \
     make \
-    python-yaml \
+    python3-yaml \
     python3-sphinx \
     sparse \
     texinfo \
#12 lore raw
In reply to John Snow
Quoted 3 lines · John Snow

This should include anything I've sent so far (not including the RFC for more exploratory changes) that hasn't already been included in a roundup by Alex Bennée.

Queued to testing/next, thanks.

Quoted 21 lines

Mostly, it removes unused docker files and replaces python2 with python3 in all of the container environments.

V3:
- Reorder patches to remove everything first, THEN patch for python3
- Split removal of debian-ports and debian-sid into two patches.

V2:
- Rolled in python2 --> python3 conversion
- Added travis conversion to python patch
- Remove debian8 dockerfile, which is also now unused.

John Snow (6):
docker: remove debian8-mxe definitions
docker: removed unused debian8 partial image
docker: remove 'deprecated' image definitions
docker: remove unused debian-ports
docker: remove unused debian-sid
docker: move tests from python2 to python3

Quoted 15 lines
 tests/docker/Makefile.include                 |  9 +++--
 tests/docker/dockerfiles/centos7.docker       |  2 +-
 tests/docker/dockerfiles/debian-ports.docker  | 36 -------------------
 tests/docker/dockerfiles/debian-sid.docker    | 35 ------------------
 .../dockerfiles/debian-xtensa-cross.docker    |  2 +-
 tests/docker/dockerfiles/debian10.docker      |  2 +-
 tests/docker/dockerfiles/debian8.docker       | 34 ------------------
 tests/docker/dockerfiles/debian9.docker       |  2 +-
 tests/docker/dockerfiles/travis.docker        |  2 +-
 tests/docker/dockerfiles/ubuntu.docker        |  2 +-
 tests/docker/dockerfiles/ubuntu1804.docker    |  2 +-
 11 files changed, 11 insertions(+), 117 deletions(-)
 delete mode 100644 tests/docker/dockerfiles/debian-ports.docker
 delete mode 100644 tests/docker/dockerfiles/debian-sid.docker
 delete mode 100644 tests/docker/dockerfiles/debian8.docker

--
Alex Bennée