docker: misc cleanups
cover docker: misc cleanups
Another handful of misc tidyups.
Note: this WILL conflict with the other cleanups already in-flight, but I didn't have a convenient branch to base against at the time; but it should be pretty trivial to resolve/respin.
John Snow (3):
docker: remove 'deprecated' image definitions
docker: remove debian8-mxe definitions
docker: remove unused debian-sid and debian-ports
tests/docker/Makefile.include | 9 +++-- tests/docker/dockerfiles/debian-ports.docker | 36 -------------------- tests/docker/dockerfiles/debian-sid.docker | 35 ------------------- 3 files changed, 4 insertions(+), 76 deletions(-) delete mode 100644 tests/docker/dockerfiles/debian-ports.docker delete mode 100644 tests/docker/dockerfiles/debian-sid.docker
Signature · 1 line
2.21.0
PATCH 2/3 docker: remove debian8-mxe definitions
We don't have a debian8-mxe dockerfile anymore.
--- tests/docker/Makefile.include | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tests/docker/Makefile.include +1 -1
@@ -6,7 +6,7 @@ DOCKER_SUFFIX := .docker DOCKER_FILES_DIR := $(SRC_PATH)/tests/docker/dockerfiles # we don't run tests on intermediate images (used as base by another image) DOCKER_PARTIAL_IMAGES := debian8 debian9 debian10 debian-sid -DEBIAN_PARTIAL_IMAGES += debian8-mxe debian-9-mxe debian-ports debian-bootstrap +DEBIAN_PARTIAL_IMAGES += debian-9-mxe debian-ports debian-bootstrap DOCKER_IMAGES := $(sort $(notdir $(basename $(wildcard $(DOCKER_FILES_DIR)/*.docker)))) DOCKER_TARGETS := $(patsubst %,docker-image-%,$(DOCKER_IMAGES)) # Use a global constant ccache directory to speed up repetitive builds
Signature · 1 line
2.21.0
PATCH 1/3 docker: remove 'deprecated' image definitions
In reply to John SnowThere isn't a debian.dockerfile anymore,
so perform some ghost-busting.
--- tests/docker/Makefile.include | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-)
tests/docker/Makefile.include +3 -4
@@ -4,11 +4,10 @@ DOCKER_SUFFIX := .docker DOCKER_FILES_DIR := $(SRC_PATH)/tests/docker/dockerfiles -DOCKER_DEPRECATED_IMAGES := debian # we don't run tests on intermediate images (used as base by another image) -DOCKER_PARTIAL_IMAGES := debian debian8 debian9 debian10 debian-sid +DOCKER_PARTIAL_IMAGES := debian8 debian9 debian10 debian-sid DEBIAN_PARTIAL_IMAGES += debian8-mxe debian-9-mxe debian-ports debian-bootstrap -DOCKER_IMAGES := $(filter-out $(DOCKER_DEPRECATED_IMAGES),$(sort $(notdir $(basename $(wildcard $(DOCKER_FILES_DIR)/*.docker))))) +DOCKER_IMAGES := $(sort $(notdir $(basename $(wildcard $(DOCKER_FILES_DIR)/*.docker)))) DOCKER_TARGETS := $(patsubst %,docker-image-%,$(DOCKER_IMAGES)) # Use a global constant ccache directory to speed up repetitive builds DOCKER_CCACHE_DIR := $$HOME/.cache/qemu-docker-ccache
@@ -158,7 +157,7 @@ docker-image-debian-powerpc-user-cross: docker-binfmt-image-debian-powerpc-user DOCKER_USER_IMAGES += debian-powerpc-user # Expand all the pre-requistes for each docker image and test combination -$(foreach i,$(filter-out $(DOCKER_PARTIAL_IMAGES),$(DOCKER_IMAGES) $(DOCKER_DEPRECATED_IMAGES)), \ +$(foreach i,$(filter-out $(DOCKER_PARTIAL_IMAGES),$(DOCKER_IMAGES)), \ $(foreach t,$(DOCKER_TESTS) $(DOCKER_TOOLS), \ $(eval .PHONY: docker-$t@$i) \ $(eval docker-$t@$i: docker-image-$i docker-run-$t@$i) \
Signature · 1 line
2.21.0
PATCH 3/3 docker: remove unused debian-sid and debian-ports
In reply to John SnowThese are listed as "partial" images, but have no user. Remove them.
--- tests/docker/Makefile.include | 4 +-- tests/docker/dockerfiles/debian-ports.docker | 36 -------------------- tests/docker/dockerfiles/debian-sid.docker | 35 ------------------- 3 files changed, 2 insertions(+), 73 deletions(-) delete mode 100644 tests/docker/dockerfiles/debian-ports.docker delete mode 100644 tests/docker/dockerfiles/debian-sid.docker
tests/docker/Makefile.include +2 -2
@@ -5,8 +5,8 @@ DOCKER_SUFFIX := .docker DOCKER_FILES_DIR := $(SRC_PATH)/tests/docker/dockerfiles # we don't run tests on intermediate images (used as base by another image) -DOCKER_PARTIAL_IMAGES := debian8 debian9 debian10 debian-sid -DEBIAN_PARTIAL_IMAGES += debian-9-mxe debian-ports debian-bootstrap +DOCKER_PARTIAL_IMAGES := debian8 debian9 debian10 +DEBIAN_PARTIAL_IMAGES += debian-9-mxe debian-bootstrap DOCKER_IMAGES := $(sort $(notdir $(basename $(wildcard $(DOCKER_FILES_DIR)/*.docker)))) DOCKER_TARGETS := $(patsubst %,docker-image-%,$(DOCKER_IMAGES)) # Use a global constant ccache directory to speed up repetitive builds
tests/docker/dockerfiles/debian-ports.docker deleted +0 -36
@@ -1,36 +0,0 @@ -# -# Docker multiarch cross-compiler target -# -# This docker target is builds on Debian Ports cross compiler targets -# to build distro with a selection of cross compilers for building test binaries. -# -# On its own you can't build much but the docker-foo-cross targets -# build on top of the base debian image. -# -FROM debian:unstable - -MAINTAINER Philippe Mathieu-Daudé <[email protected]> - -RUN echo "deb [arch=amd64] http://deb.debian.org/debian unstable main" > /etc/apt/sources.list - -# Duplicate deb line as deb-src -RUN cat /etc/apt/sources.list | sed -ne "s/^deb\ \(\[.*\]\ \)\?\(.*\)/deb-src \2/p" >> /etc/apt/sources.list - -# Setup some basic tools we need -RUN apt-get update && \ - DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata && \ - DEBIAN_FRONTEND=noninteractive eatmydata \ - apt-get install -y --no-install-recommends \ - bison \ - build-essential \ - ca-certificates \ - clang \ - debian-ports-archive-keyring \ - flex \ - gettext \ - git \ - pkg-config \ - psmisc \ - python \ - texinfo \ - $(apt-get -s build-dep qemu | egrep ^Inst | fgrep '[all]' | cut -d\ -f2)
tests/docker/dockerfiles/debian-sid.docker deleted +0 -35
@@ -1,35 +0,0 @@ -# -# Debian Sid Base -# -# Currently we can build all our guests with cross-compilers in the -# latest Debian release (Buster). However new compilers will first -# arrive in Sid. However Sid is a rolling distro which may be broken -# at any particular time. To try and mitigate this we use Debian's -# snapshot archive which provides a "stable" view of what state Sid -# was in. -# - -# This must be earlier than the snapshot date we are aiming for -FROM debian:sid-20190812-slim - - # Use a snapshot known to work (see http://snapshot.debian.org/#Usage) -ENV DEBIAN_SNAPSHOT_DATE "20190820" -RUN sed -i "s%^deb \(https\?://\)deb.debian.org/debian/\? \(.*\)%deb [check-valid-until=no] \1snapshot.debian.org/archive/debian/${DEBIAN_SNAPSHOT_DATE} \2%" /etc/apt/sources.list - -# Duplicate deb line as deb-src -RUN cat /etc/apt/sources.list | sed "s/^deb\ /deb-src /" >> /etc/apt/sources.list - -# Install common build utilities -RUN apt update && \ - DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata && \ - DEBIAN_FRONTEND=noninteractive eatmydata \ - apt install -y --no-install-recommends \ - bison \ - build-essential \ - ca-certificates \ - flex \ - git \ - pkg-config \ - psmisc \ - python \ - texinfo || { echo "Failed to build - see debian-sid.docker notes"; exit 1; }
Signature · 1 line
2.21.0
Re: [PATCH 3/3] docker: remove unused debian-sid and debian-ports
Quoted 2 lines · John Snow
These are listed as "partial" images, but have no user. Remove them.
Well, I have WiP users from them. I could restore this content when they are ready... Ports is the base of deprecated Debian archs. On the other side Sid is the base for edge development I use from time to time to test latest gcc/binutils.
I'll try to find time to raise WiP branches to PoC.
Quoted 108 lines
Signed-off-by John Snow <[email protected]>--- tests/docker/Makefile.include | 4 +-- tests/docker/dockerfiles/debian-ports.docker | 36 -------------------- tests/docker/dockerfiles/debian-sid.docker | 35 ------------------- 3 files changed, 2 insertions(+), 73 deletions(-) delete mode 100644 tests/docker/dockerfiles/debian-ports.docker delete mode 100644 tests/docker/dockerfiles/debian-sid.dockertests/docker/Makefile.include +2
-2@@ -5,8 +5,8 @@ DOCKER_SUFFIX := .docker DOCKER_FILES_DIR := $(SRC_PATH)/tests/docker/dockerfiles # we don't run tests on intermediate images (used as base by another image) -DOCKER_PARTIAL_IMAGES := debian8 debian9 debian10 debian-sid -DEBIAN_PARTIAL_IMAGES += debian-9-mxe debian-ports debian-bootstrap +DOCKER_PARTIAL_IMAGES := debian8 debian9 debian10 +DEBIAN_PARTIAL_IMAGES += debian-9-mxe debian-bootstrap DOCKER_IMAGES := $(sort $(notdir $(basename $(wildcard $(DOCKER_FILES_DIR)/*.docker)))) DOCKER_TARGETS := $(patsubst %,docker-image-%,$(DOCKER_IMAGES)) # Use a global constant ccache directory to speed up repetitive buildstests/docker/dockerfiles/debian-ports.docker deleted +0
-36@@ -1,36 +0,0 @@ -# -# Docker multiarch cross-compiler target -# -# This docker target is builds on Debian Ports cross compiler targets -# to build distro with a selection of cross compilers for building test binaries. -# -# On its own you can't build much but the docker-foo-cross targets -# build on top of the base debian image. -# -FROM debian:unstable - -MAINTAINER Philippe Mathieu-Daudé <[email protected]> - -RUN echo "deb [arch=amd64] http://deb.debian.org/debian unstable main" > /etc/apt/sources.list - -# Duplicate deb line as deb-src -RUN cat /etc/apt/sources.list | sed -ne "s/^deb\ \(\[.*\]\ \)\?\(.*\)/deb-src \2/p" >> /etc/apt/sources.list - -# Setup some basic tools we need -RUN apt-get update && \ - DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata && \ - DEBIAN_FRONTEND=noninteractive eatmydata \ - apt-get install -y --no-install-recommends \ - bison \ - build-essential \ - ca-certificates \ - clang \ - debian-ports-archive-keyring \ - flex \ - gettext \ - git \ - pkg-config \ - psmisc \ - python \ - texinfo \ - $(apt-get -s build-dep qemu | egrep ^Inst | fgrep '[all]' | cut -d\ -f2)tests/docker/dockerfiles/debian-sid.docker deleted +0
-35@@ -1,35 +0,0 @@ -# -# Debian Sid Base -# -# Currently we can build all our guests with cross-compilers in the -# latest Debian release (Buster). However new compilers will first -# arrive in Sid. However Sid is a rolling distro which may be broken -# at any particular time. To try and mitigate this we use Debian's -# snapshot archive which provides a "stable" view of what state Sid -# was in. -# - -# This must be earlier than the snapshot date we are aiming for -FROM debian:sid-20190812-slim - - # Use a snapshot known to work (see http://snapshot.debian.org/#Usage) -ENV DEBIAN_SNAPSHOT_DATE "20190820" -RUN sed -i "s%^deb \(https\?://\)deb.debian.org/debian/\? \(.*\)%deb [check-valid-until=no] \1snapshot.debian.org/archive/debian/${DEBIAN_SNAPSHOT_DATE} \2%" /etc/apt/sources.list - -# Duplicate deb line as deb-src -RUN cat /etc/apt/sources.list | sed "s/^deb\ /deb-src /" >> /etc/apt/sources.list - -# Install common build utilities -RUN apt update && \ - DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata && \ - DEBIAN_FRONTEND=noninteractive eatmydata \ - apt install -y --no-install-recommends \ - bison \ - build-essential \ - ca-certificates \ - flex \ - git \ - pkg-config \ - psmisc \ - python \ - texinfo || { echo "Failed to build - see debian-sid.docker notes"; exit 1; }
Re: [PATCH 3/3] docker: remove unused debian-sid and debian-ports
In reply to John SnowQuoted 17 lines · John Snow
Quoted 10 lines · Philippe Mathieu-Daudé
Quoted 2 lines · John Snow
These are listed as "partial" images, but have no user. Remove them.
Well, I have WiP users from them. I could restore this content when they are ready... Ports is the base of deprecated Debian archs. On the other side Sid is the base for edge development I use from time to time to test latest gcc/binutils.
I'll try to find time to raise WiP branches to PoC.I think that's the right thing to do. Right now, the docker tests directory has a lot of stale entries and unusable tests. That might be fine for the people working on it, but it makes it hard to understand and use for those of us who only occasionally traipse into the directory.
Quoted 8 lines
I'm removing all references to python2 -- but if there's no way for me to test debian-sid and debian-ports, I can't test changes I need to make to these "partial images", so they should be removed until they are consumable.
While I am sympathetic to the idea of having a library of partial images to use for future tests, they're prone to rot if there's no way to exercise them in-tree.
Don't forget some "partial" images are only used for building TCG tests
- we want to keep them. But as git is forever I can drop the sid/ports stuff for now until Phillipe has something to use them again.
Quoted 105 lines
You can always re-add them when you have a user.
--js
Quoted 100 lines
Quoted 100 lines
Signed-off-by John Snow <[email protected]>--- tests/docker/Makefile.include | 4 +-- tests/docker/dockerfiles/debian-ports.docker | 36 -------------------- tests/docker/dockerfiles/debian-sid.docker | 35 ------------------- 3 files changed, 2 insertions(+), 73 deletions(-) delete mode 100644 tests/docker/dockerfiles/debian-ports.docker delete mode 100644 tests/docker/dockerfiles/debian-sid.dockertests/docker/Makefile.include +2
-2@@ -5,8 +5,8 @@ DOCKER_SUFFIX := .docker DOCKER_FILES_DIR := $(SRC_PATH)/tests/docker/dockerfiles # we don't run tests on intermediate images (used as base by another image) -DOCKER_PARTIAL_IMAGES := debian8 debian9 debian10 debian-sid -DEBIAN_PARTIAL_IMAGES += debian-9-mxe debian-ports debian-bootstrap +DOCKER_PARTIAL_IMAGES := debian8 debian9 debian10 +DEBIAN_PARTIAL_IMAGES += debian-9-mxe debian-bootstrap DOCKER_IMAGES := $(sort $(notdir $(basename $(wildcard $(DOCKER_FILES_DIR)/*.docker)))) DOCKER_TARGETS := $(patsubst %,docker-image-%,$(DOCKER_IMAGES)) # Use a global constant ccache directory to speed up repetitive buildstests/docker/dockerfiles/debian-ports.docker deleted +0
-36@@ -1,36 +0,0 @@ -# -# Docker multiarch cross-compiler target -# -# This docker target is builds on Debian Ports cross compiler targets -# to build distro with a selection of cross compilers for building test binaries. -# -# On its own you can't build much but the docker-foo-cross targets -# build on top of the base debian image. -# -FROM debian:unstable - -MAINTAINER Philippe Mathieu-Daudé <[email protected]> - -RUN echo "deb [arch=amd64] http://deb.debian.org/debian unstable main" > /etc/apt/sources.list - -# Duplicate deb line as deb-src -RUN cat /etc/apt/sources.list | sed -ne "s/^deb\ \(\[.*\]\ \)\?\(.*\)/deb-src \2/p" >> /etc/apt/sources.list - -# Setup some basic tools we need -RUN apt-get update && \ - DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata && \ - DEBIAN_FRONTEND=noninteractive eatmydata \ - apt-get install -y --no-install-recommends \ - bison \ - build-essential \ - ca-certificates \ - clang \ - debian-ports-archive-keyring \ - flex \ - gettext \ - git \ - pkg-config \ - psmisc \ - python \ - texinfo \ - $(apt-get -s build-dep qemu | egrep ^Inst | fgrep '[all]' | cut -d\ -f2)tests/docker/dockerfiles/debian-sid.docker deleted +0
-28@@ -1,35 +0,0 @@ -# -# Debian Sid Base -# -# Currently we can build all our guests with cross-compilers in the -# latest Debian release (Buster). However new compilers will first -# arrive in Sid. However Sid is a rolling distro which may be broken -# at any particular time. To try and mitigate this we use Debian's -# snapshot archive which provides a "stable" view of what state Sid -# was in. -# - -# This must be earlier than the snapshot date we are aiming for -FROM debian:sid-20190812-slim - - # Use a snapshot known to work (see http://snapshot.debian.org/#Usage) -ENV DEBIAN_SNAPSHOT_DATE "20190820" -RUN sed -i "s%^deb \(https\?://\)deb.debian.org/debian/\? \(.*\)%deb [check-valid-until=no] \1snapshot.debian.org/archive/debian/${DEBIAN_SNAPSHOT_DATE} \2%" /etc/apt/sources.list - -# Duplicate deb line as deb-src -RUN cat /etc/apt/sources.list | sed "s/^deb\ /deb-src /" >> /etc/apt/sources.list - -# Install common build utilities -RUN apt update && \ - DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata && \ - DEBIAN_FRONTEND=noninteractive eatmydata \ - apt install -y --no-install-recommends \ - bison \ - build-essential \
Quoted 8 lines
Quoted 8 lines
tests/docker/dockerfiles/debian-sid.docker line 35
Quoted 8 lines
tests/docker/dockerfiles/debian-sid.docker deleted +0
-7- ca-certificates \ - flex \ - git \ - pkg-config \ - psmisc \ - python \ - texinfo || { echo "Failed to build - see debian-sid.docker notes"; exit 1; }
--
Alex Bennée
Re: [PATCH 3/3] docker: remove unused debian-sid and debian-ports
In reply to Philippe Mathieu-DaudéQuoted 28 lines · Philippe Mathieu-Daudé
Quoted 27 lines · Alex Bennée
Quoted 25 lines · John Snow
Quoted 10 lines · Philippe Mathieu-Daudé
Quoted 2 lines · John Snow
These are listed as "partial" images, but have no user. Remove them.
Well, I have WiP users from them. I could restore this content when they are ready... Ports is the base of deprecated Debian archs. On the other side Sid is the base for edge development I use from time to time to test latest gcc/binutils.
I'll try to find time to raise WiP branches to PoC.I think that's the right thing to do. Right now, the docker tests directory has a lot of stale entries and unusable tests. That might be fine for the people working on it, but it makes it hard to understand and use for those of us who only occasionally traipse into the directory.
I'm removing all references to python2 -- but if there's no way for me to test debian-sid and debian-ports, I can't test changes I need to make to these "partial images", so they should be removed until they are consumable.
While I am sympathetic to the idea of having a library of partial images to use for future tests, they're prone to rot if there's no way to exercise them in-tree.
Quoted 5 lines
Quoted 3 lines
Don't forget some "partial" images are only used for building TCG tests
- we want to keep them. But as git is forever I can drop the sid/ports stuff for now until Phillipe has something to use them again.For Sid I have this case:
Does it really need sid to build binutils/gcc?
Quoted 45 lines
-- >8 --
#
# Docker Renesas RX cross-compiler target
#
# This docker target builds on the debian Sid base image. #
# Copyright (c) 2019 Philippe Mathieu-Daudé
#
# SPDX-License-Identifier: GPL-2.0-or-later
#
FROM qemu:debian-sidMAINTAINER Philippe Mathieu-Daudé <[email protected]>
RUN DEBIAN_FRONTEND=noninteractive eatmydata \ apt install -y --no-install-recommends \ wget RUN wget -qO - https://ftpmirror.gnu.org/binutils/binutils-2.32.tar.xz \ | tar -C /usr/src -xJf - && \ cd /usr/src/binutils-2.32 && \ CFLAGS=-w \ ./configure --prefix=/usr --disable-nls --target=rx-unknown-linux && \ make && make install && \ rm -rf /usr/src/binutils-2.32RUN git clone -b rx-trunk http://pf.osdn.net/gitroot/y/ys/ysato/gcc.git \
/usr/src/gccRUN DEBIAN_FRONTEND=noninteractive eatmydata \
apt install -y --no-install-recommends \
libgmp-dev libmpfr-dev libmpc-devRUN cd /usr/src/gcc && \ CPPFLAGS=-w \ ./configure --target=rx-unknown-linux \ --prefix=/usr --disable-nls \ --enable-languages=c --disable-shared \ --disable-threads --with-uclibc \ --disable-libssp --disable-libquadmath \ --disable-libgomp --disable-libatomic && \ make && make install && \ rm -rf /usr/src/gcc
Quoted 3 lines
# This image isn't designed for building QEMU but building tests ENV
QEMU_CONFIGURE_OPTS--disable-system --disable-user
---
--
Alex Bennée
Re: [PATCH 3/3] docker: remove unused debian-sid and debian-ports
In reply to Alex BennéeQuoted 23 lines · Alex Bennée
Quoted 22 lines · John Snow
Quoted 10 lines · Philippe Mathieu-Daudé
Quoted 2 lines · John Snow
These are listed as "partial" images, but have no user. Remove them.
Well, I have WiP users from them. I could restore this content when they are ready... Ports is the base of deprecated Debian archs. On the other side Sid is the base for edge development I use from time to time to test latest gcc/binutils.
I'll try to find time to raise WiP branches to PoC.I think that's the right thing to do. Right now, the docker tests directory has a lot of stale entries and unusable tests. That might be fine for the people working on it, but it makes it hard to understand and use for those of us who only occasionally traipse into the directory.
I'm removing all references to python2 -- but if there's no way for me to test debian-sid and debian-ports, I can't test changes I need to make to these "partial images", so they should be removed until they are consumable.
Quoted 7 lines
Quoted 3 lines
While I am sympathetic to the idea of having a library of partial images to use for future tests, they're prone to rot if there's no way to exercise them in-tree.
Don't forget some "partial" images are only used for building TCG tests
- we want to keep them. But as git is forever I can drop the sid/ports stuff for now until Phillipe has something to use them again.
For Sid I have this case:
-- >8 --
#
# Docker Renesas RX cross-compiler target
#
# This docker target builds on the debian Sid base image. #
# Copyright (c) 2019 Philippe Mathieu-Daudé
#
# SPDX-License-Identifier: GPL-2.0-or-later
#
FROM qemu:debian-sid
MAINTAINER Philippe Mathieu-Daudé <[email protected]>
RUN DEBIAN_FRONTEND=noninteractive eatmydata \
apt install -y --no-install-recommends \
wget
RUN wget -qO - https://ftpmirror.gnu.org/binutils/binutils-2.32.tar.xz \
| tar -C /usr/src -xJf - && \
cd /usr/src/binutils-2.32 && \
CFLAGS=-w \
./configure --prefix=/usr --disable-nls
--target=rx-unknown-linux && \
make && make install && \
rm -rf /usr/src/binutils-2.32RUN git clone -b rx-trunk http://pf.osdn.net/gitroot/y/ys/ysato/gcc.git \
/usr/src/gcc
RUN DEBIAN_FRONTEND=noninteractive eatmydata \
apt install -y --no-install-recommends \
libgmp-dev libmpfr-dev libmpc-dev
RUN cd /usr/src/gcc && \
CPPFLAGS=-w \
./configure --target=rx-unknown-linux \
--prefix=/usr --disable-nls \
--enable-languages=c --disable-shared \
--disable-threads --with-uclibc \
--disable-libssp --disable-libquadmath \
--disable-libgomp --disable-libatomic && \
make && make install && \
rm -rf /usr/src/gcc# This image isn't designed for building QEMU but building tests ENV QEMU_CONFIGURE_OPTS --disable-system --disable-user
---
Re: [PATCH 3/3] docker: remove unused debian-sid and debian-ports
In reply to Philippe Mathieu-DaudéQuoted 10 lines · Philippe Mathieu-Daudé
Quoted 2 lines · John Snow
These are listed as "partial" images, but have no user. Remove them.
Well, I have WiP users from them. I could restore this content when they are ready... Ports is the base of deprecated Debian archs. On the other side Sid is the base for edge development I use from time to time to test latest gcc/binutils.
I'll try to find time to raise WiP branches to PoC.
I think that's the right thing to do. Right now, the docker tests directory has a lot of stale entries and unusable tests. That might be fine for the people working on it, but it makes it hard to understand and use for those of us who only occasionally traipse into the directory.
I'm removing all references to python2 -- but if there's no way for me to test debian-sid and debian-ports, I can't test changes I need to make to these "partial images", so they should be removed until they are consumable.
While I am sympathetic to the idea of having a library of partial images to use for future tests, they're prone to rot if there's no way to exercise them in-tree.
You can always re-add them when you have a user.
--js
Quoted 108 lines
Quoted 108 lines
Signed-off-by John Snow <[email protected]>--- tests/docker/Makefile.include | 4 +-- tests/docker/dockerfiles/debian-ports.docker | 36 -------------------- tests/docker/dockerfiles/debian-sid.docker | 35 ------------------- 3 files changed, 2 insertions(+), 73 deletions(-) delete mode 100644 tests/docker/dockerfiles/debian-ports.docker delete mode 100644 tests/docker/dockerfiles/debian-sid.dockertests/docker/Makefile.include +2
-2@@ -5,8 +5,8 @@ DOCKER_SUFFIX := .docker DOCKER_FILES_DIR := $(SRC_PATH)/tests/docker/dockerfiles # we don't run tests on intermediate images (used as base by another image) -DOCKER_PARTIAL_IMAGES := debian8 debian9 debian10 debian-sid -DEBIAN_PARTIAL_IMAGES += debian-9-mxe debian-ports debian-bootstrap +DOCKER_PARTIAL_IMAGES := debian8 debian9 debian10 +DEBIAN_PARTIAL_IMAGES += debian-9-mxe debian-bootstrap DOCKER_IMAGES := $(sort $(notdir $(basename $(wildcard $(DOCKER_FILES_DIR)/*.docker)))) DOCKER_TARGETS := $(patsubst %,docker-image-%,$(DOCKER_IMAGES)) # Use a global constant ccache directory to speed up repetitive buildstests/docker/dockerfiles/debian-ports.docker deleted +0
-36@@ -1,36 +0,0 @@ -# -# Docker multiarch cross-compiler target -# -# This docker target is builds on Debian Ports cross compiler targets -# to build distro with a selection of cross compilers for building test binaries. -# -# On its own you can't build much but the docker-foo-cross targets -# build on top of the base debian image. -# -FROM debian:unstable - -MAINTAINER Philippe Mathieu-Daudé <[email protected]> - -RUN echo "deb [arch=amd64] http://deb.debian.org/debian unstable main" > /etc/apt/sources.list - -# Duplicate deb line as deb-src -RUN cat /etc/apt/sources.list | sed -ne "s/^deb\ \(\[.*\]\ \)\?\(.*\)/deb-src \2/p" >> /etc/apt/sources.list - -# Setup some basic tools we need -RUN apt-get update && \ - DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata && \ - DEBIAN_FRONTEND=noninteractive eatmydata \ - apt-get install -y --no-install-recommends \ - bison \ - build-essential \ - ca-certificates \ - clang \ - debian-ports-archive-keyring \ - flex \ - gettext \ - git \ - pkg-config \ - psmisc \ - python \ - texinfo \ - $(apt-get -s build-dep qemu | egrep ^Inst | fgrep '[all]' | cut -d\ -f2)tests/docker/dockerfiles/debian-sid.docker deleted +0
-35@@ -1,35 +0,0 @@ -# -# Debian Sid Base -# -# Currently we can build all our guests with cross-compilers in the -# latest Debian release (Buster). However new compilers will first -# arrive in Sid. However Sid is a rolling distro which may be broken -# at any particular time. To try and mitigate this we use Debian's -# snapshot archive which provides a "stable" view of what state Sid -# was in. -# - -# This must be earlier than the snapshot date we are aiming for -FROM debian:sid-20190812-slim - - # Use a snapshot known to work (see http://snapshot.debian.org/#Usage) -ENV DEBIAN_SNAPSHOT_DATE "20190820" -RUN sed -i "s%^deb \(https\?://\)deb.debian.org/debian/\? \(.*\)%deb [check-valid-until=no] \1snapshot.debian.org/archive/debian/${DEBIAN_SNAPSHOT_DATE} \2%" /etc/apt/sources.list - -# Duplicate deb line as deb-src -RUN cat /etc/apt/sources.list | sed "s/^deb\ /deb-src /" >> /etc/apt/sources.list - -# Install common build utilities -RUN apt update && \ - DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata && \ - DEBIAN_FRONTEND=noninteractive eatmydata \ - apt install -y --no-install-recommends \ - bison \ - build-essential \ - ca-certificates \ - flex \ - git \ - pkg-config \ - psmisc \ - python \ - texinfo || { echo "Failed to build - see debian-sid.docker notes"; exit 1; }
Re: [PATCH 3/3] docker: remove unused debian-sid and debian-ports
In reply to Philippe Mathieu-DaudéQuoted 34 lines · Philippe Mathieu-Daudé
Quoted 23 lines · Alex Bennée
Quoted 22 lines · John Snow
Quoted 10 lines · Philippe Mathieu-Daudé
Quoted 2 lines · John Snow
These are listed as "partial" images, but have no user. Remove them.
Well, I have WiP users from them. I could restore this content when they are ready... Ports is the base of deprecated Debian archs. On the other side Sid is the base for edge development I use from time to time to test latest gcc/binutils.
I'll try to find time to raise WiP branches to PoC.I think that's the right thing to do. Right now, the docker tests directory has a lot of stale entries and unusable tests. That might be fine for the people working on it, but it makes it hard to understand and use for those of us who only occasionally traipse into the directory.
I'm removing all references to python2 -- but if there's no way for me to test debian-sid and debian-ports, I can't test changes I need to make to these "partial images", so they should be removed until they are consumable.
Quoted 7 lines
Quoted 3 lines
While I am sympathetic to the idea of having a library of partial images to use for future tests, they're prone to rot if there's no way to exercise them in-tree.
Don't forget some "partial" images are only used for building TCG tests
- we want to keep them. But as git is forever I can drop the sid/ports stuff for now until Phillipe has something to use them again.For Sid I have this case:
Quoted 11 lines
-- >8 --
#
# Docker Renesas RX cross-compiler target
#
# This docker target builds on the debian Sid base image. #
# Copyright (c) 2019 Philippe Mathieu-Daudé
#
# SPDX-License-Identifier: GPL-2.0-or-later
#
FROM qemu:debian-sid
Oh well nevermind it is old, so it probably now works with debian-10.
I used the Ports base for sparc32 and x32, I don't have them on my workstation I'll check during the WE.
Quoted 36 lines
MAINTAINER Philippe Mathieu-Daudé <[email protected]>
RUN DEBIAN_FRONTEND=noninteractive eatmydata \ apt install -y --no-install-recommends \ wget RUN wget -qO - https://ftpmirror.gnu.org/binutils/binutils-2.32.tar.xz \ | tar -C /usr/src -xJf - && \ cd /usr/src/binutils-2.32 && \ CFLAGS=-w \ ./configure --prefix=/usr --disable-nls --target=rx-unknown-linux && \ make && make install && \ rm -rf /usr/src/binutils-2.32RUN git clone -b rx-trunk http://pf.osdn.net/gitroot/y/ys/ysato/gcc.git \
/usr/src/gccRUN DEBIAN_FRONTEND=noninteractive eatmydata \
apt install -y --no-install-recommends \
libgmp-dev libmpfr-dev libmpc-devRUN cd /usr/src/gcc && \ CPPFLAGS=-w \ ./configure --target=rx-unknown-linux \ --prefix=/usr --disable-nls \ --enable-languages=c --disable-shared \ --disable-threads --with-uclibc \ --disable-libssp --disable-libquadmath \ --disable-libgomp --disable-libatomic && \ make && make install && \ rm -rf /usr/src/gcc# This image isn't designed for building QEMU but building tests ENV
QEMU_CONFIGURE_OPTS--disable-system --disable-user
---
Re: [PATCH 3/3] docker: remove unused debian-sid and debian-ports
In reply to Alex BennéeQuoted 25 lines · Alex Bennée
Quoted 22 lines · John Snow
Quoted 10 lines · Philippe Mathieu-Daudé
Quoted 2 lines · John Snow
These are listed as "partial" images, but have no user. Remove them.
Well, I have WiP users from them. I could restore this content when they are ready... Ports is the base of deprecated Debian archs. On the other side Sid is the base for edge development I use from time to time to test latest gcc/binutils.
I'll try to find time to raise WiP branches to PoC.I think that's the right thing to do. Right now, the docker tests directory has a lot of stale entries and unusable tests. That might be fine for the people working on it, but it makes it hard to understand and use for those of us who only occasionally traipse into the directory.
I'm removing all references to python2 -- but if there's no way for me to test debian-sid and debian-ports, I can't test changes I need to make to these "partial images", so they should be removed until they are consumable.
Quoted 8 lines
Quoted 3 lines
While I am sympathetic to the idea of having a library of partial images to use for future tests, they're prone to rot if there's no way to exercise them in-tree.
Don't forget some "partial" images are only used for building TCG tests
- we want to keep them. But as git is forever I can drop the sid/ports stuff for now until Phillipe has something to use them again.
Oh, where? Is that defined within the tests/docker/Makefile.include or in the TCG tree where I have not gone spelunking before?
--js
Re: [PATCH 3/3] docker: remove unused debian-sid and debian-ports
In reply to Philippe Mathieu-DaudéQuoted 47 lines · Philippe Mathieu-Daudé
Quoted 34 lines · Philippe Mathieu-Daudé
Quoted 23 lines · Alex Bennée
Quoted 22 lines · John Snow
Quoted 10 lines · Philippe Mathieu-Daudé
Quoted 2 lines · John Snow
These are listed as "partial" images, but have no user. Remove them.
Well, I have WiP users from them. I could restore this content when they are ready... Ports is the base of deprecated Debian archs. On the other side Sid is the base for edge development I use from time to time to test latest gcc/binutils.
I'll try to find time to raise WiP branches to PoC.I think that's the right thing to do. Right now, the docker tests directory has a lot of stale entries and unusable tests. That might be fine for the people working on it, but it makes it hard to understand and use for those of us who only occasionally traipse into the directory.
I'm removing all references to python2 -- but if there's no way for me to test debian-sid and debian-ports, I can't test changes I need to make to these "partial images", so they should be removed until they are consumable.
Quoted 7 lines
Quoted 3 lines
While I am sympathetic to the idea of having a library of partial images to use for future tests, they're prone to rot if there's no way to exercise them in-tree.
Don't forget some "partial" images are only used for building TCG tests
- we want to keep them. But as git is forever I can drop the sid/ports stuff for now until Phillipe has something to use them again.For Sid I have this case:
Quoted 11 lines
-- >8 --
#
# Docker Renesas RX cross-compiler target
#
# This docker target builds on the debian Sid base image. #
# Copyright (c) 2019 Philippe Mathieu-Daudé
#
# SPDX-License-Identifier: GPL-2.0-or-later
#
FROM qemu:debian-sid
Quoted 5 lines
Oh well nevermind it is old, so it probably now works with debian-10.
I used the Ports base for sparc32 and x32, I don't have them on my workstation I'll check during the WE.
If you want to keep docker-sid as a partial image for possible future uses, you might consider adding a simple amd64 user -- debian-sid-amd64 would suffice.
It doesn't have to do anything interesting, but it exists to give debian-sid a user.
I'd rather not waste your hard work, but I am making a plea that any images defined in the tree can be tested by running tests that use them.
--js
Re: [PATCH 3/3] docker: remove unused debian-sid and debian-ports
In reply to John SnowQuoted 30 lines · John Snow
Quoted 29 lines · Alex Bennée
Quoted 25 lines · John Snow
Quoted 10 lines · Philippe Mathieu-Daudé
Quoted 2 lines · John Snow
These are listed as "partial" images, but have no user. Remove them.
Well, I have WiP users from them. I could restore this content when they are ready... Ports is the base of deprecated Debian archs. On the other side Sid is the base for edge development I use from time to time to test latest gcc/binutils.
I'll try to find time to raise WiP branches to PoC.I think that's the right thing to do. Right now, the docker tests directory has a lot of stale entries and unusable tests. That might be fine for the people working on it, but it makes it hard to understand and use for those of us who only occasionally traipse into the directory.
I'm removing all references to python2 -- but if there's no way for me to test debian-sid and debian-ports, I can't test changes I need to make to these "partial images", so they should be removed until they are consumable.
While I am sympathetic to the idea of having a library of partial images to use for future tests, they're prone to rot if there's no way to exercise them in-tree.
Quoted 7 lines
Quoted 4 lines
Don't forget some "partial" images are only used for building TCG tests
- we want to keep them. But as git is forever I can drop the sid/ports stuff for now until Phillipe has something to use them again.Oh, where? Is that defined within the tests/docker/Makefile.include or in the TCG tree where I have not gone spelunking before?
Now they are selected by tests/tcg/configure.sh but generally they DISTRO-ARCH-cross images.
Quoted 2 lines
--js
--
Alex Bennée
Re: [PATCH 3/3] docker: remove unused debian-sid and debian-ports
Quoted 38 lines · Alex Bennée
Quoted 35 lines · John Snow
Quoted 33 lines · Alex Bennée
Quoted 17 lines · John Snow
Quoted 10 lines · Philippe Mathieu-Daudé
Quoted 2 lines · John Snow
These are listed as "partial" images, but have no user. Remove them.
Well, I have WiP users from them. I could restore this content when they are ready... Ports is the base of deprecated Debian archs. On the other side Sid is the base for edge development I use from time to time to test latest gcc/binutils.
I'll try to find time to raise WiP branches to PoC.I think that's the right thing to do. Right now, the docker tests directory has a lot of stale entries and unusable tests. That might be fine for the people working on it, but it makes it hard to understand and use for those of us who only occasionally traipse into the directory.
Quoted 8 lines
I'm removing all references to python2 -- but if there's no way for me to test debian-sid and debian-ports, I can't test changes I need to make to these "partial images", so they should be removed until they are consumable.
While I am sympathetic to the idea of having a library of partial images to use for future tests, they're prone to rot if there's no way to exercise them in-tree.
Don't forget some "partial" images are only used for building TCG tests
- we want to keep them. But as git is forever I can drop the sid/ports stuff for now until Phillipe has something to use them again.
Quoted 6 lines
Quoted 2 lines
Oh, where? Is that defined within the tests/docker/Makefile.include or in the TCG tree where I have not gone spelunking before?
Now they are selected by
tests/tcg/configure.shbut generally they DISTRO-ARCH-cross images.
I see -- "container_image" in configure.sh chooses them and consumes them there.
Looks like that feeds to Makefile.qemu which defines docker-build-guest-tests.
We call docker.py (DOCKER_SCRIPT) as:docker.py cc ... -i qemu:$(DOCKER_IMAGE) ...
okay, so this bypasses the Docker Makefile pretty directly; but it looks like it's only using images that the Makefile there doesn't count as "Partial" universally.
You *do* have a section that marks images as "PARTIAL" if they're the wrong architecture -- maybe it would be a semantic improvement to list these as simply "DISABLED" or "UNAVAILABLE" -- the implication being that they're complete images, but will not work for the current host architecture.
Thanks for the pointer!
--js
Re: [PATCH 3/3] docker: remove unused debian-sid and debian-ports
In reply to John SnowHi John,
Quoted 11 lines · John Snow
These are listed as "partial" images, but have no user. Remove them.
Signed-off-by John Snow <[email protected]>--- tests/docker/Makefile.include | 4 +-- tests/docker/dockerfiles/debian-ports.docker | 36 -------------------- tests/docker/dockerfiles/debian-sid.docker | 35 ------------------- 3 files changed, 2 insertions(+), 73 deletions(-) delete mode 100644 tests/docker/dockerfiles/debian-ports.docker delete mode 100644 tests/docker/dockerfiles/debian-sid.docker
Can you split this patch in 2, one per image? That will ease the reverts.
Once splitted feel free to add to each new patch:
Thanks,
Phil.
Re: [PATCH 3/3] docker: remove unused debian-sid and debian-ports
Quoted 15 lines · Philippe Mathieu-Daudé
Hi John,
Quoted 11 lines · John Snow
These are listed as "partial" images, but have no user. Remove them.
Signed-off-by John Snow <[email protected]>--- tests/docker/Makefile.include | 4 +-- tests/docker/dockerfiles/debian-ports.docker | 36 -------------------- tests/docker/dockerfiles/debian-sid.docker | 35 ------------------- 3 files changed, 2 insertions(+), 73 deletions(-) delete mode 100644 tests/docker/dockerfiles/debian-ports.docker delete mode 100644 tests/docker/dockerfiles/debian-sid.docker
Quoted 9 lines
Can you split this patch in 2, one per image? That will ease the reverts.
Once splitted feel free to add to each new patch:
Reviewed-by Philippe Mathieu-Daudé <[email protected]>Thanks,
Phil.
Sounds good to me, thanks!