Run cross-compilation build tests in the gitlab-CI
v2 cover Run cross-compilation build tests in the gitlab-CI
Now that we can use all our QEMU build containers in the gitlab-CI, we can also run the cross-compilation jobs there. Of course, some problems have to be fixed first, so this is taken care of in the first four patches.
The following two patches make sure that we can also enable WHPX builds with our debian-win64-cross container, so that we can compile-test this accelerator code now, too.
The last patch then finally enables the cross-compilation jobs in the CI.
v2:
- Dropped patches that are not necessary anymore
- Added the first two patches to fix problems with the new meson build system
Thomas Huth (7):
configure: Add system = 'linux' for meson when cross-compiling tests/docker: Install python3-setuptools in the debian9-mxe containers tests/Makefile: test-image-locking needs CONFIG_POSIX tests/Makefile: test-replication needs CONFIG_POSIX dockerfiles/debian-win64-cross: Download WHPX MinGW headers
configure: Allow automatic WHPX detection
gitlab-ci: Add cross-compiling build tests
.gitlab-ci.d/crossbuilds.yml | 113 ++++++++++++++++++ .gitlab-ci.yml | 1 + MAINTAINERS | 1 + configure | 4 + tests/Makefile.include | 6 +- .../dockerfiles/debian-win64-cross.docker | 9 +- tests/docker/dockerfiles/debian9-mxe.docker | 2 +- 7 files changed, 133 insertions(+), 3 deletions(-) create mode 100644 .gitlab-ci.d/crossbuilds.yml
Signature · 1 line
2.18.2
PATCH v2 1/7 configure: Add system = 'linux' for meson when cross-compiling
Meson needs the "system = xyz" line when cross-compiling. We are already adding a "system = 'windows'" for the MinGW cross-compilation case here, so let's add a "system = 'linux'" now for Linux hosts, too.
--- configure | 3 +++ 1 file changed, 3 insertions(+)
configure +3 -0
@@ -8208,6 +8208,9 @@ if test -n "$cross_prefix"; then ?:*) pre_prefix=/ ;; esac fi + if test "$linux" = "yes" ; then + echo "system = 'linux'" >> $cross + fi case "$ARCH" in i386|x86_64) echo "cpu_family = 'x86'" >> $cross
Signature · 1 line
2.18.2
Re: [PATCH v2 1/7] configure: Add system = 'linux' for meson when cross-compiling
Quoted 8 lines · Thomas Huth
Meson needs the "system = xyz" line when cross-compiling. We are already adding a "system = 'windows'" for the MinGW cross-compilation case here, so let's add a "system = 'linux'" now for Linux hosts, too.
Signed-off-by Thomas Huth <[email protected]>--- configure | 3 +++ 1 file changed, 3 insertions(+)
Regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
PATCH v2 2/7 tests/docker: Install python3-setuptools in the debian9-mxe containers
In reply to Thomas HuthThe python setuptools are a requirement for meson, so we need to install this additional package now.
--- tests/docker/dockerfiles/debian9-mxe.docker | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tests/docker/dockerfiles/debian9-mxe.docker +1 -1
@@ -8,7 +8,7 @@ FROM qemu/debian9 MAINTAINER Philippe Mathieu-Daudé <[email protected]> RUN DEBIAN_FRONTEND=noninteractive eatmydata \ - apt install -y --no-install-recommends gnupg dirmngr + apt install -y --no-install-recommends gnupg dirmngr python3-setuptools # Add the foreign architecture we want and install dependencies RUN apt-key adv --keyserver keyserver.ubuntu.com --recv-keys C6BF758A33A3A276 && \
Signature · 1 line
2.18.2
Re: [PATCH v2 2/7] tests/docker: Install python3-setuptools in the debian9-mxe containers
Quoted 7 lines · Thomas Huth
The python setuptools are a requirement for meson, so we need to install this additional package now.
Signed-off-by Thomas Huth <[email protected]>--- tests/docker/dockerfiles/debian9-mxe.docker | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
PATCH v2 3/7 tests/Makefile: test-image-locking needs CONFIG_POSIX
In reply to Thomas Huthtest-image-locking.c uses the qemu_lock_fd_test() function which is only available on POSIX-like systems.
Message-Id: <[email protected]>
--- tests/Makefile.include | 2 ++ 1 file changed, 2 insertions(+)
tests/Makefile.include +2 -0
@@ -87,7 +87,9 @@ check-unit-$(CONFIG_BLOCK) += tests/test-blockjob$(EXESUF) check-unit-$(CONFIG_BLOCK) += tests/test-blockjob-txn$(EXESUF) check-unit-$(CONFIG_BLOCK) += tests/test-block-backend$(EXESUF) check-unit-$(CONFIG_BLOCK) += tests/test-block-iothread$(EXESUF) +ifeq ($(CONFIG_POSIX),y) check-unit-$(CONFIG_BLOCK) += tests/test-image-locking$(EXESUF) +endif check-unit-y += tests/test-x86-cpuid$(EXESUF) # all code tested by test-x86-cpuid is inside topology.h ifeq ($(CONFIG_SOFTMMU),y)
Signature · 1 line
2.18.2
Re: [PATCH v2 3/7] tests/Makefile: test-image-locking needs CONFIG_POSIX
Quoted 9 lines · Thomas Huth
test-image-locking.cuses theqemu_lock_fd_test()function which is only available on POSIX-like systems.Reviewed-by John Snow <[email protected]>Message-Id: <[email protected]>
Signed-off-by Thomas Huth <[email protected]>--- tests/Makefile.include | 2 ++ 1 file changed, 2 insertions(+)
Regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
PATCH v2 4/7 tests/Makefile: test-replication needs CONFIG_POSIX
In reply to Thomas Huthtest-replication uses sigaction() and friends which are only available on POSIX-like systems.
Message-Id: <[email protected]>
--- tests/Makefile.include | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
tests/Makefile.include +3 -1
@@ -156,11 +156,13 @@ check-unit-$(CONFIG_BLOCK) += tests/test-crypto-afsplit$(EXESUF) check-unit-$(call land,$(CONFIG_BLOCK),$(CONFIG_QEMU_PRIVATE_XTS)) += tests/test-crypto-xts$(EXESUF) check-unit-$(CONFIG_BLOCK) += tests/test-crypto-block$(EXESUF) check-unit-y += tests/test-logging$(EXESUF) -check-unit-$(call land,$(CONFIG_BLOCK),$(CONFIG_REPLICATION)) += tests/test-replication$(EXESUF) check-unit-$(CONFIG_SOFTMMU) += tests/test-bufferiszero$(EXESUF) check-unit-y += tests/test-uuid$(EXESUF) check-unit-y += tests/ptimer-test$(EXESUF) check-unit-y += tests/test-qapi-util$(EXESUF) +ifeq ($(CONFIG_BLOCK)$(CONFIG_REPLICATION)$(CONFIG_POSIX),yyy) +check-unit-y += tests/test-replication$(EXESUF) +endif generated-files-y += tests/test-qapi-types.h generated-files-y += tests/include/test-qapi-types-sub-module.h
Signature · 1 line
2.18.2
Re: [PATCH v2 4/7] tests/Makefile: test-replication needs CONFIG_POSIX
Quoted 9 lines · Thomas Huth
test-replication uses
sigaction()and friends which are only available on POSIX-like systems.Reviewed-by Philippe Mathieu-Daudé <[email protected]>Message-Id: <[email protected]>
Signed-off-by Thomas Huth <[email protected]>--- tests/Makefile.include | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
Regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
PATCH v2 5/7 dockerfiles/debian-win64-cross: Download WHPX MinGW headers
In reply to Thomas HuthTo compile-test the WHPX accelerator, we need to download these system headers first (they are unfortunately not part of any released and packaged MinGW toolchain yet).
Idea taken from another patch by Stefan Weil.
Message-Id: <[email protected]>
--- tests/docker/dockerfiles/debian-win64-cross.docker | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
tests/docker/dockerfiles/debian-win64-cross.docker +8 -1
@@ -32,7 +32,14 @@ RUN apt-get update && \ mxe-$TARGET-w64-mingw32.shared-sdl2 \ mxe-$TARGET-w64-mingw32.shared-sdl2-mixer \ mxe-$TARGET-w64-mingw32.shared-sdl2-gfx \ - mxe-$TARGET-w64-mingw32.shared-zlib + mxe-$TARGET-w64-mingw32.shared-zlib \ + curl && \ + curl -s -S -o /usr/lib/mxe/usr/x86_64-w64-mingw32.shared/include/WinHvEmulation.h \ + "https://sourceforge.net/p/mingw-w64/mingw-w64/ci/master/tree/mingw-w64-headers/include/winhvemulation.h?format=raw" && \ + curl -s -S -o /usr/lib/mxe/usr/x86_64-w64-mingw32.shared/include/WinHvPlatform.h \ + "https://sourceforge.net/p/mingw-w64/mingw-w64/ci/master/tree/mingw-w64-headers/include/winhvplatform.h?format=raw" && \ + curl -s -S -o /usr/lib/mxe/usr/x86_64-w64-mingw32.shared/include/winhvplatformdefs.h \ + "https://sourceforge.net/p/mingw-w64/mingw-w64/ci/master/tree/mingw-w64-headers/include/winhvplatformdefs.h?format=raw" # Specify the cross prefix for this image (see tests/docker/common.rc) ENV QEMU_CONFIGURE_OPTS --cross-prefix=x86_64-w64-mingw32.shared-
Signature · 1 line
2.18.2
Re: [PATCH v2 5/7] dockerfiles/debian-win64-cross: Download WHPX MinGW headers
Quoted 21 lines · Thomas Huth
To compile-test the WHPX accelerator, we need to download these system headers first (they are unfortunately not part of any released and packaged MinGW toolchain yet).
Idea taken from another patch by Stefan Weil.
Message-Id: <[email protected]>
Signed-off-by Thomas Huth <[email protected]>--- tests/docker/dockerfiles/debian-win64-cross.docker | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)tests/docker/dockerfiles/debian-win64-cross.docker +0
-0@@ -32,7 +32,14 @@ RUN apt-get update && \ mxe-$TARGET-w64-mingw32.shared-sdl2 \ mxe-$TARGET-w64-mingw32.shared-sdl2-mixer \
tests/docker/dockerfiles/debian-win64-cross.docker RUN apt-get update && \ · line 42
Quoted 10 lines
tests/docker/dockerfiles/debian-win64-cross.docker +8
-1mxe-$TARGET-w64-mingw32.shared-sdl2-gfx \ - mxe-$TARGET-w64-mingw32.shared-zlib + mxe-$TARGET-w64-mingw32.shared-zlib \ + curl && \ + curl -s -S -o /usr/lib/mxe/usr/x86_64-w64-mingw32.shared/include/WinHvEmulation.h \ + "https://sourceforge.net/p/mingw-w64/mingw-w64/ci/master/tree/mingw-w64-headers/include/winhvemulation.h?format=raw" && \ + curl -s -S -o /usr/lib/mxe/usr/x86_64-w64-mingw32.shared/include/WinHvPlatform.h \ + "https://sourceforge.net/p/mingw-w64/mingw-w64/ci/master/tree/mingw-w64-headers/include/winhvplatform.h?format=raw" && \ + curl -s -S -o /usr/lib/mxe/usr/x86_64-w64-mingw32.shared/include/winhvplatformdefs.h \ + "https://sourceforge.net/p/mingw-w64/mingw-w64/ci/master/tree/mingw-w64-headers/include/winhvplatformdefs.h?format=raw"
My only concern here is whether sourceforge.net URLs are reliable enough for CI, or whether we should instead be mirroring the w64 repo on git.qemu.org ?
I guess this will be a short term issue until a new mingw release is made and arrives i nthe distros, so we can try with the curl approach for now and change if it causes problems
Regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [PATCH v2 5/7] dockerfiles/debian-win64-cross: Download WHPX MinGW headers
Quoted 37 lines · Daniel P. Berrangé
Quoted 21 lines · Thomas Huth
To compile-test the WHPX accelerator, we need to download these system headers first (they are unfortunately not part of any released and packaged MinGW toolchain yet).
Idea taken from another patch by Stefan Weil.
Message-Id: <[email protected]>
Signed-off-by Thomas Huth <[email protected]>--- tests/docker/dockerfiles/debian-win64-cross.docker | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)tests/docker/dockerfiles/debian-win64-cross.docker +0
-0@@ -32,7 +32,14 @@ RUN apt-get update && \ mxe-$TARGET-w64-mingw32.shared-sdl2 \ mxe-$TARGET-w64-mingw32.shared-sdl2-mixer \tests/docker/dockerfiles/debian-win64-cross.docker RUN apt-get update && \ · line 42
Quoted 10 lines
tests/docker/dockerfiles/debian-win64-cross.docker +8
-1mxe-$TARGET-w64-mingw32.shared-sdl2-gfx \ - mxe-$TARGET-w64-mingw32.shared-zlib + mxe-$TARGET-w64-mingw32.shared-zlib \ + curl && \ + curl -s -S -o /usr/lib/mxe/usr/x86_64-w64-mingw32.shared/include/WinHvEmulation.h \ + "https://sourceforge.net/p/mingw-w64/mingw-w64/ci/master/tree/mingw-w64-headers/include/winhvemulation.h?format=raw" && \ + curl -s -S -o /usr/lib/mxe/usr/x86_64-w64-mingw32.shared/include/WinHvPlatform.h \ + "https://sourceforge.net/p/mingw-w64/mingw-w64/ci/master/tree/mingw-w64-headers/include/winhvplatform.h?format=raw" && \ + curl -s -S -o /usr/lib/mxe/usr/x86_64-w64-mingw32.shared/include/winhvplatformdefs.h \ + "https://sourceforge.net/p/mingw-w64/mingw-w64/ci/master/tree/mingw-w64-headers/include/winhvplatformdefs.h?format=raw"My only concern here is whether sourceforge.net URLs are reliable enough for CI, or whether we should instead be mirroring the w64 repo on git.qemu.org ?
Quoted 3 lines
I guess this will be a short term issue until a new mingw release is made and arrives i nthe distros, so we can try with the curl approach for now and change if it causes problems
That's my assumption, too. And since the containers are only rebuild when something changed (as far as I understood), the download should only occur quite rarely, so there is hopefully really not a big dependency here.
Quoted 1 line
Reviewed-by Daniel P. Berrangé <[email protected]>
Thanks!
Thomas
PATCH v2 6/7 configure: Allow automatic WHPX detection
In reply to Thomas HuthThe whpx variable is currently initialized to "no" which causes the WHPX check to skip the detection unless the user specified --enable-whpx. Since the detection code should be able to figure it out correctly, let's initialized the variable to "" on MinGW-builds for proper auto-detection instead.
Message-Id: <[email protected]>
--- configure | 1 + 1 file changed, 1 insertion(+)
configure +1 -0
@@ -849,6 +849,7 @@ case $targetos in MINGW32*) mingw32="yes" hax="yes" + whpx="" vhost_user="no" audio_possible_drivers="dsound sdl" if check_include dsound.h; then
Signature · 1 line
2.18.2
Re: [PATCH v2 6/7] configure: Allow automatic WHPX detection
Quoted 6 lines · Thomas Huth
The whpx variable is currently initialized to "no" which causes the WHPX check to skip the detection unless the user specified --enable-whpx. Since the detection code should be able to figure it out correctly, let's initialized the variable to "" on MinGW-builds for proper auto-detection instead.
Quoted 7 lines
Message-Id: <[email protected]>
Reviewed-by Philippe Mathieu-Daudé <[email protected]>Reviewed-by Stefan Weil <[email protected]>Signed-off-by Thomas Huth <[email protected]>--- configure | 1 + 1 file changed, 1 insertion(+)
Regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
PATCH v2 7/7 gitlab-ci: Add cross-compiling build tests
In reply to Thomas HuthNow that we can use all our QEMU test containers in the gitlab-CI, we can easily add some jobs that test cross-compilation for various architectures. There is just only small ugliness: Since the shared runners on gitlab.com are single-threaded, we have to split each compilation job into two parts (--disable-user and --disable-system), and exclude some additional targets, to avoid that the jobs are running too long and hitting the timeout of 1 h.
--- .gitlab-ci.d/crossbuilds.yml | 113 +++++++++++++++++++++++++++++++++++ .gitlab-ci.yml | 1 + MAINTAINERS | 1 + 3 files changed, 115 insertions(+) create mode 100644 .gitlab-ci.d/crossbuilds.yml
.gitlab-ci.d/crossbuilds.yml new file +113 -0
@@ -0,0 +1,113 @@ + +.cross_system_build_job_template: &cross_system_build_job_definition + stage: build + image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest + script: + - mkdir build + - cd build + - PKG_CONFIG_PATH=$PKG_CONFIG_PATH + ../configure --enable-werror $QEMU_CONFIGURE_OPTS --disable-user + --target-list-exclude="aarch64-softmmu i386-softmmu microblaze-softmmu + mips-softmmu mipsel-softmmu mips64-softmmu ppc64-softmmu sh4-softmmu + xtensa-softmmu" + - make -j$(expr $(nproc) + 1) all check-build + +.cross_user_build_job_template: &cross_user_build_job_definition + stage: build + image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest + script: + - mkdir build + - cd build + - PKG_CONFIG_PATH=$PKG_CONFIG_PATH + ../configure --enable-werror $QEMU_CONFIGURE_OPTS --disable-system + - make -j$(expr $(nproc) + 1) all check-build + +cross-armel-system: + <<: *cross_system_build_job_definition + variables: + IMAGE: debian-armel-cross + +cross-armel-user: + <<: *cross_user_build_job_definition + variables: + IMAGE: debian-armel-cross + +cross-armhf-system: + <<: *cross_system_build_job_definition + variables: + IMAGE: debian-armhf-cross + +cross-armhf-user: + <<: *cross_user_build_job_definition + variables: + IMAGE: debian-armhf-cross + +cross-arm64-system: + <<: *cross_system_build_job_definition + variables: + IMAGE: debian-arm64-cross + +cross-arm64-user: + <<: *cross_user_build_job_definition + variables: + IMAGE: debian-arm64-cross + +cross-mips-system: + <<: *cross_system_build_job_definition + variables: + IMAGE: debian-mips-cross + +cross-mips-user: + <<: *cross_user_build_job_definition + variables: + IMAGE: debian-mips-cross + +cross-mipsel-system: + <<: *cross_system_build_job_definition + variables: + IMAGE: debian-mipsel-cross + +cross-mipsel-user: + <<: *cross_user_build_job_definition + variables: + IMAGE: debian-mipsel-cross + +cross-mips64el-system: + <<: *cross_system_build_job_definition + variables: + IMAGE: debian-mips64el-cross + +cross-mips64el-user: + <<: *cross_user_build_job_definition + variables: + IMAGE: debian-mips64el-cross + +cross-ppc64el-system: + <<: *cross_system_build_job_definition + variables: + IMAGE: debian-ppc64el-cross + +cross-ppc64el-user: + <<: *cross_user_build_job_definition + variables: + IMAGE: debian-ppc64el-cross + +cross-s390x-system: + <<: *cross_system_build_job_definition + variables: + IMAGE: debian-s390x-cross + +cross-s390x-user: + <<: *cross_user_build_job_definition + variables: + IMAGE: debian-s390x-cross + +cross-win32-system: + <<: *cross_system_build_job_definition + variables: + IMAGE: debian-win32-cross + +cross-win64-system: + <<: *cross_system_build_job_definition + variables: + IMAGE: debian-win64-cross
.gitlab-ci.yml +1 -0
@@ -18,6 +18,7 @@ include: - local: '/.gitlab-ci.d/edk2.yml' - local: '/.gitlab-ci.d/opensbi.yml' - local: '/.gitlab-ci.d/containers.yml' + - local: '/.gitlab-ci.d/crossbuilds.yml' .native_build_job_template: &native_build_job_definition stage: build
MAINTAINERS +1 -0
@@ -3067,6 +3067,7 @@ M: Alex Bennée <[email protected]> R: Wainer dos Santos Moschetta <[email protected]> S: Maintained F: .gitlab-ci.yml +F: .gitlab-ci.d/crossbuilds.yml Guest Test Compilation Support M: Alex Bennée <[email protected]>
Signature · 1 line
2.18.2
Re: [PATCH v2 7/7] gitlab-ci: Add cross-compiling build tests
Quoted 7 lines · Thomas Huth
Now that we can use all our QEMU test containers in the gitlab-CI, we can easily add some jobs that test cross-compilation for various architectures. There is just only small ugliness: Since the shared runners on gitlab.com are single-threaded, we have to split each compilation job into two parts (--disable-user and --disable-system), and exclude some additional targets, to avoid that the jobs are running too long and hitting the timeout of 1 h.
Quoted 7 lines
Signed-off-by Thomas Huth <[email protected]>--- .gitlab-ci.d/crossbuilds.yml | 113 +++++++++++++++++++++++++++++++++++ .gitlab-ci.yml | 1 + MAINTAINERS | 1 + 3 files changed, 115 insertions(+) create mode 100644 .gitlab-ci.d/crossbuilds.yml
Quoted 9 lines
.gitlab-ci.d/crossbuilds.yml new file +2
-0@@ -0,0 +1,113 @@ + +.cross_system_build_job_template: &cross_system_build_job_definition
.gitlab-ci.d/crossbuilds.yml line 12
Quoted 10 lines
.gitlab-ci.d/crossbuilds.yml new file +10
-0+ stage: build + image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest + script: + - mkdir build + - cd build + - PKG_CONFIG_PATH=$PKG_CONFIG_PATH + ../configure --enable-werror $QEMU_CONFIGURE_OPTS --disable-user + --target-list-exclude="aarch64-softmmu i386-softmmu microblaze-softmmu + mips-softmmu mipsel-softmmu mips64-softmmu ppc64-softmmu sh4-softmmu + xtensa-softmmu"
What does this leave enabled ? Would it be shorter to just say --target-list="...explicit targets we want..." ? It would be clearer to review at least.
.gitlab-ci.d/crossbuilds.yml line 13
Quoted 1 line
.gitlab-ci.d/crossbuilds.yml new file +1
-0+ - make -j$(expr $(nproc) + 1) all check-build
None the less, it is functionally fine so
Regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [PATCH v2 7/7] gitlab-ci: Add cross-compiling build tests
Quoted 33 lines · Daniel P. Berrangé
Quoted 7 lines · Thomas Huth
Now that we can use all our QEMU test containers in the gitlab-CI, we can easily add some jobs that test cross-compilation for various architectures. There is just only small ugliness: Since the shared runners on gitlab.com are single-threaded, we have to split each compilation job into two parts (--disable-user and --disable-system), and exclude some additional targets, to avoid that the jobs are running too long and hitting the timeout of 1 h.
Quoted 7 lines
Signed-off-by Thomas Huth <[email protected]>--- .gitlab-ci.d/crossbuilds.yml | 113 +++++++++++++++++++++++++++++++++++ .gitlab-ci.yml | 1 + MAINTAINERS | 1 + 3 files changed, 115 insertions(+) create mode 100644 .gitlab-ci.d/crossbuilds.ymlReviewed-by Daniel P. Berrangé <[email protected]>Quoted 15 lines
.gitlab-ci.d/crossbuilds.yml new file +8
-0@@ -0,0 +1,113 @@ + +.cross_system_build_job_template: &cross_system_build_job_definition + stage: build + image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest + script: + - mkdir build + - cd build + - PKG_CONFIG_PATH=$PKG_CONFIG_PATH
Quoted 8 lines
.gitlab-ci.d/crossbuilds.yml line 12
Quoted 4 lines
.gitlab-ci.d/crossbuilds.yml new file +4
-0+ ../configure --enable-werror $QEMU_CONFIGURE_OPTS --disable-user + --target-list-exclude="aarch64-softmmu i386-softmmu microblaze-softmmu + mips-softmmu mipsel-softmmu mips64-softmmu ppc64-softmmu sh4-softmmu + xtensa-softmmu"What does this leave enabled ? Would it be shorter to just say --target-list="...explicit targets we want..." ? It would be clearer to review at least.
I basically excluded all targets that have a second compile test coverage, e.g. i386-softmmu code is mostly also covered by x86_64-softmmu, sh4-softmmu is covered by sh4eb-softmmu etc.
The --target-list-exclude also comes in handy when new targets are added to QEMU - you then don't have to remember to add the new targets here, they are picked up automatically.
Quoted 5 lines
.gitlab-ci.d/crossbuilds.yml line 13
Quoted 1 line
.gitlab-ci.d/crossbuilds.yml new file +1
-0+ - make -j$(expr $(nproc) + 1) all check-buildNone the less, it is functionally fine so
Reviewed-by Daniel P. Berrangé <[email protected]>
Thanks!
Thomas
Build system parts
Il dom 23 ago 2020, 13:18 Thomas Huth <[email protected]> ha scritto:
Quoted 41 lines
Now that we can use all our QEMU build containers in the gitlab-CI, we can also run the cross-compilation jobs there. Of course, some problems have to be fixed first, so this is taken care of in the first four patches.
The following two patches make sure that we can also enable WHPX builds with
our debian-win64-cross container, so that we can compile-test this accelerator
code now, too.The last patch then finally enables the cross-compilation jobs in the CI.
v2:
- Dropped patches that are not necessary anymore
- Added the first two patches to fix problems with the new meson build systemThomas Huth (7):
configure: Add system = 'linux' for meson when cross-compiling tests/docker: Install python3-setuptools in the debian9-mxe containers tests/Makefile: test-image-locking needsCONFIG_POSIXtests/Makefile: test-replication needsCONFIG_POSIXdockerfiles/debian-win64-cross: Download WHPX MinGW headers
configure: Allow automatic WHPX detection
gitlab-ci: Add cross-compiling build tests.gitlab-ci.d/crossbuilds.yml | 113 ++++++++++++++++++ .gitlab-ci.yml | 1 + MAINTAINERS | 1 + configure | 4 + tests/Makefile.include | 6 +- .../dockerfiles/debian-win64-cross.docker | 9 +- tests/docker/dockerfiles/debian9-mxe.docker | 2 +- 7 files changed, 133 insertions(+), 3 deletions(-) create mode 100644 .gitlab-ci.d/crossbuilds.yml--
2.18.2
Quoted 25 lines · Thomas Huth
Now that we can use all our QEMU build containers in the gitlab-CI, we can also run the cross-compilation jobs there. Of course, some problems have to be fixed first, so this is taken care of in the first four patches.
The following two patches make sure that we can also enable WHPX builds with our debian-win64-cross container, so that we can compile-test this accelerator code now, too.
The last patch then finally enables the cross-compilation jobs in the CI.
v2:
- Dropped patches that are not necessary anymore
- Added the first two patches to fix problems with the new meson build systemThomas Huth (7):
configure: Add system = 'linux' for meson when cross-compiling tests/docker: Install python3-setuptools in the debian9-mxe containers tests/Makefile: test-image-locking needsCONFIG_POSIXtests/Makefile: test-replication needsCONFIG_POSIXdockerfiles/debian-win64-cross: Download WHPX MinGW headers
configure: Allow automatic WHPX detection
gitlab-ci: Add cross-compiling build tests
Quoted 9 lines
.gitlab-ci.d/crossbuilds.yml | 113 ++++++++++++++++++ .gitlab-ci.yml | 1 + MAINTAINERS | 1 + configure | 4 + tests/Makefile.include | 6 +- .../dockerfiles/debian-win64-cross.docker | 9 +- tests/docker/dockerfiles/debian9-mxe.docker | 2 +- 7 files changed, 133 insertions(+), 3 deletions(-) create mode 100644 .gitlab-ci.d/crossbuilds.yml
You seem to have not sent the mail for
[PATCH 8/7] delete obsolete shippable config
;-P
Regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Quoted 36 lines · Daniel P. Berrangé
Quoted 25 lines · Thomas Huth
Now that we can use all our QEMU build containers in the gitlab-CI, we can also run the cross-compilation jobs there. Of course, some problems have to be fixed first, so this is taken care of in the first four patches.
The following two patches make sure that we can also enable WHPX builds with our debian-win64-cross container, so that we can compile-test this accelerator code now, too.
The last patch then finally enables the cross-compilation jobs in the CI.
v2:
- Dropped patches that are not necessary anymore
- Added the first two patches to fix problems with the new meson build systemThomas Huth (7):
configure: Add system = 'linux' for meson when cross-compiling tests/docker: Install python3-setuptools in the debian9-mxe containers tests/Makefile: test-image-locking needsCONFIG_POSIXtests/Makefile: test-replication needsCONFIG_POSIXdockerfiles/debian-win64-cross: Download WHPX MinGW headers
configure: Allow automatic WHPX detection
gitlab-ci: Add cross-compiling build testsQuoted 9 lines
.gitlab-ci.d/crossbuilds.yml | 113 ++++++++++++++++++ .gitlab-ci.yml | 1 + MAINTAINERS | 1 + configure | 4 + tests/Makefile.include | 6 +- .../dockerfiles/debian-win64-cross.docker | 9 +- tests/docker/dockerfiles/debian9-mxe.docker | 2 +- 7 files changed, 133 insertions(+), 3 deletions(-) create mode 100644 .gitlab-ci.d/crossbuilds.yml
Quoted 5 lines
You seem to have not sent the mail for
[PATCH 8/7] delete obsolete shippable config
;-P
Yeah, I wanted to chat with Alex about that first once the gitlab stuff has been mreged...
Thomas