gitlab-ci.yml: Avoid some submodules to speed up the CI a little bit
PATCH 1/1 gitlab-ci.yml: Avoid some submodules to speed up the CI a little bit
Since the meson build system rework, the configure script prefers the git submodules over the system libraries. So we are testing compilation with capstone, fdt and libslirp as a submodule all over the place, burning CPU cycles by recompiling these third party modules and wasting some network bandwidth in the CI by cloning the submodules each time. Let's stop doing that in at least a couple of jobs and use the system libraries instead.
While we're at it, also install meson in the Fedora container, since it is new enough already, so we do not need to check out the meson submodule here.
--- .gitlab-ci.yml | 6 +++++- tests/docker/dockerfiles/centos8.docker | 1 + tests/docker/dockerfiles/debian-amd64.docker | 1 + tests/docker/dockerfiles/fedora.docker | 3 +++ tests/docker/dockerfiles/ubuntu2004.docker | 1 + 5 files changed, 11 insertions(+), 1 deletion(-)
.gitlab-ci.yml +5 -1
@@ -109,6 +109,7 @@ build-system-ubuntu: <<: *native_build_job_definition variables: IMAGE: ubuntu2004 + CONFIGURE_ARGS: --enable-fdt=system --enable-slirp=system TARGETS: aarch64-softmmu alpha-softmmu cris-softmmu hppa-softmmu moxie-softmmu microblazeel-softmmu mips64el-softmmu MAKE_CHECK_ARGS: check-build
@@ -140,6 +141,7 @@ build-system-debian: <<: *native_build_job_definition variables: IMAGE: debian-amd64 + CONFIGURE_ARGS: --enable-fdt=system TARGETS: arm-softmmu avr-softmmu i386-softmmu mipsel-softmmu riscv64-softmmu sh4eb-softmmu sparc-softmmu xtensaeb-softmmu MAKE_CHECK_ARGS: check-build
@@ -185,6 +187,7 @@ build-system-fedora: variables: IMAGE: fedora CONFIGURE_ARGS: --disable-gcrypt --enable-nettle --enable-docs + --enable-fdt=system --enable-slirp=system --enable-capstone=system TARGETS: tricore-softmmu microblaze-softmmu mips-softmmu xtensa-softmmu m68k-softmmu riscv32-softmmu ppc-softmmu sparc64-softmmu MAKE_CHECK_ARGS: check-build
@@ -216,7 +219,7 @@ build-system-centos: <<: *native_build_job_definition variables: IMAGE: centos8 - CONFIGURE_ARGS: --disable-nettle --enable-gcrypt + CONFIGURE_ARGS: --disable-nettle --enable-gcrypt --enable-fdt=system TARGETS: ppc64-softmmu or1k-softmmu s390x-softmmu x86_64-softmmu rx-softmmu sh4-softmmu nios2-softmmu MAKE_CHECK_ARGS: check-build
@@ -248,6 +251,7 @@ build-system-opensuse: <<: *native_build_job_definition variables: IMAGE: opensuse-leap + CONFIGURE_ARGS: --enable-fdt=system TARGETS: s390x-softmmu x86_64-softmmu aarch64-softmmu MAKE_CHECK_ARGS: check-build artifacts:
tests/docker/dockerfiles/centos8.docker +1 -0
@@ -15,6 +15,7 @@ ENV PACKAGES \ glib2-devel \ libaio-devel \ libepoxy-devel \ + libfdt-devel \ libgcrypt-devel \ lzo-devel \ make \
tests/docker/dockerfiles/debian-amd64.docker +1 -0
@@ -21,6 +21,7 @@ RUN apt update && \ libbz2-dev \ liblzo2-dev \ libgcrypt20-dev \ + libfdt-dev \ librdmacm-dev \ libsasl2-dev \ libsnappy-dev \
tests/docker/dockerfiles/fedora.docker +3 -0
@@ -6,6 +6,7 @@ ENV PACKAGES \ brlapi-devel \ bzip2 \ bzip2-devel \ + capstone-devel \ ccache \ clang \ cyrus-sasl-devel \
@@ -37,6 +38,7 @@ ENV PACKAGES \ libpng-devel \ librbd-devel \ libseccomp-devel \ + libslirp-devel \ libssh-devel \ libubsan \ libudev-devel \
@@ -46,6 +48,7 @@ ENV PACKAGES \ llvm \ lzo-devel \ make \ + meson \ mingw32-bzip2 \ mingw32-curl \ mingw32-glib2 \
tests/docker/dockerfiles/ubuntu2004.docker +1 -0
@@ -37,6 +37,7 @@ ENV PACKAGES flex bison \ libsasl2-dev \ libsdl2-dev \ libseccomp-dev \ + libslirp-dev \ libsnappy-dev \ libspice-protocol-dev \ libspice-server-dev \
Signature · 1 line
2.27.0
Hi Thomas,
Quoted 12 lines · Thomas Huth
Since the meson build system rework, the configure script prefers the git submodules over the system libraries. So we are testing compilation with capstone, fdt and libslirp as a submodule all over the place, burning CPU cycles by recompiling these third party modules and wasting some network bandwidth in the CI by cloning the submodules each time. Let's stop doing that in at least a couple of jobs and use the system libraries instead.
While we're at it, also install meson in the Fedora container, since it is new enough already, so we do not need to check out the meson submodule here.
Quoted 8 lines
Signed-off-by Thomas Huth <[email protected]>--- .gitlab-ci.yml | 6 +++++- tests/docker/dockerfiles/centos8.docker | 1 + tests/docker/dockerfiles/debian-amd64.docker | 1 + tests/docker/dockerfiles/fedora.docker | 3 +++ tests/docker/dockerfiles/ubuntu2004.docker | 1 + 5 files changed, 11 insertions(+), 1 deletion(-)
So now which are the jobs left building from git submodule clone?
Quoted 24 lines · Philippe Mathieu-Daudé
Hi Thomas,
Quoted 12 lines · Thomas Huth
Since the meson build system rework, the configure script prefers the git submodules over the system libraries. So we are testing compilation with capstone, fdt and libslirp as a submodule all over the place, burning CPU cycles by recompiling these third party modules and wasting some network bandwidth in the CI by cloning the submodules each time. Let's stop doing that in at least a couple of jobs and use the system libraries instead.
While we're at it, also install meson in the Fedora container, since it is new enough already, so we do not need to check out the meson submodule here.
Quoted 8 lines
Signed-off-by Thomas Huth <[email protected]>--- .gitlab-ci.yml | 6 +++++- tests/docker/dockerfiles/centos8.docker | 1 + tests/docker/dockerfiles/debian-amd64.docker | 1 + tests/docker/dockerfiles/fedora.docker | 3 +++ tests/docker/dockerfiles/ubuntu2004.docker | 1 + 5 files changed, 11 insertions(+), 1 deletion(-)
Quoted 1 line
So now which are the jobs left building from git submodule clone?
All the jobs which don't have a depending test stage jobs should still use the submodules, e.g. build-oss-fuzz, build-tci, etc. Also the centos7-based jobs still use all submodules, e.g. build-crypto-old-nettle etc.
Thomas