Gitlab-CI improvements
cover Gitlab-CI improvements
Here are three patches for some small issues that I noticed in our gitlab-CI files recently...
Thomas Huth (3):
gitlab-ci: Merge "build-disabled" with "build-without-default-features"
gitlab-ci: Remove superfluous "dnf install" statement
gitlab-ci: Fix ..._RUNNER_AVAILABLE variables and document them
.gitlab-ci.d/buildtest.yml | 99 +++++---------------------------- .gitlab-ci.d/custom-runners.yml | 12 ++-- docs/devel/ci.rst | 13 +++++ 3 files changed, 32 insertions(+), 92 deletions(-)
Signature · 1 line
2.27.0
PATCH 1/3 gitlab-ci: Merge "build-disabled" with "build-without-default-features"
Both jobs are testing more or less the same thing (building QEMU with features disabled), so we are wasting precious CI cycles here by doing this twice. Merge the jobs by using --without-default-features by default and just adding some additional --disable-... switches which are not covered by the generic switch (yet). And while we're at it, also test compilation with "--disable-fdt" (which forces us to change the list of targets in this job, though, since some targets do not work without fdt).
--- .gitlab-ci.d/buildtest.yml | 97 +++++--------------------------------- 1 file changed, 13 insertions(+), 84 deletions(-)
.gitlab-ci.d/buildtest.yml +13 -84
@@ -202,85 +202,6 @@ acceptance-system-opensuse: MAKE_CHECK_ARGS: check-acceptance -build-disabled: - extends: .native_build_job_template - needs: - job: amd64-fedora-container - variables: - IMAGE: fedora - CONFIGURE_ARGS: - --disable-attr - --disable-auth-pam - --disable-avx2 - --disable-bochs - --disable-brlapi - --disable-bzip2 - --disable-cap-ng - --disable-capstone - --disable-cloop - --disable-coroutine-pool - --disable-curl - --disable-curses - --disable-dmg - --disable-docs - --disable-gcrypt - --disable-glusterfs - --disable-gnutls - --disable-gtk - --disable-guest-agent - --disable-iconv - --disable-keyring - --disable-kvm - --disable-libiscsi - --disable-libpmem - --disable-libssh - --disable-libudev - --disable-libusb - --disable-libxml2 - --disable-linux-aio - --disable-live-block-migration - --disable-lzo - --disable-malloc-trim - --disable-mpath - --disable-nettle - --disable-numa - --disable-opengl - --disable-parallels - --disable-pie - --disable-qcow1 - --disable-qed - --disable-qom-cast-debug - --disable-rbd - --disable-rdma - --disable-replication - --disable-sdl - --disable-seccomp - --disable-slirp - --disable-smartcard - --disable-snappy - --disable-sparse - --disable-spice - --disable-strip - --disable-tpm - --disable-usb-redir - --disable-vdi - --disable-vhost-crypto - --disable-vhost-net - --disable-vhost-scsi - --disable-vhost-kernel - --disable-vhost-user - --disable-vhost-vdpa - --disable-vhost-vsock - --disable-virglrenderer - --disable-vnc - --disable-vte - --disable-vvfat - --disable-xen - --disable-zstd - TARGETS: arm-softmmu i386-softmmu ppc64-softmmu mips64-softmmu - s390x-softmmu i386-linux-user - MAKE_CHECK_ARGS: check-qtest SPEED=slow - # This jobs explicitly disable TCG (--disable-tcg), KVM is detected by # the configure script. The container doesn't contain Xen headers so # Xen accelerator is not detected / selected. As result it build the
@@ -649,12 +570,20 @@ build-without-default-devices: build-without-default-features: extends: .native_build_job_template needs: - job: amd64-debian-container + job: amd64-fedora-container variables: - IMAGE: debian-amd64 - CONFIGURE_ARGS: --without-default-features --disable-user - --target-list-exclude=arm-softmmu,i386-softmmu,mipsel-softmmu,mips64-softmmu,ppc-softmmu - MAKE_CHECK_ARGS: check-unit + IMAGE: fedora + CONFIGURE_ARGS: + --without-default-features + --disable-capstone + --disable-fdt + --disable-pie + --disable-qom-cast-debug + --disable-slirp + --disable-strip + TARGETS: avr-softmmu i386-softmmu mips64-softmmu s390x-softmmu sh4-softmmu + sparc64-softmmu hexagon-linux-user i386-linux-user s390x-linux-user + MAKE_CHECK_ARGS: check-unit check-qtest SPEED=slow build-libvhost-user: stage: build
Signature · 1 line
2.27.0
Re: [PATCH 1/3] gitlab-ci: Merge "build-disabled" with "build-without-default-features"
Quoted 10 lines · Thomas Huth
Both jobs are testing more or less the same thing (building QEMU with features disabled), so we are wasting precious CI cycles here by doing this twice. Merge the jobs by using --without-default-features by default and just adding some additional --disable-... switches which are not covered by the generic switch (yet). And while we're at it, also test compilation with "--disable-fdt" (which forces us to change the list of targets in this job, though, since some targets do not work without fdt).
Quoted 5 lines
Signed-off-by Thomas Huth <[email protected]>--- .gitlab-ci.d/buildtest.yml | 97 +++++--------------------------------- 1 file changed, 13 insertions(+), 84 deletions(-)
PATCH 2/3 gitlab-ci: Remove superfluous "dnf install" statement
In reply to Thomas HuthThe container already features meson and ninja, so there is no need to try to install it with dnf again.
--- .gitlab-ci.d/buildtest.yml | 2 -- 1 file changed, 2 deletions(-)
.gitlab-ci.d/buildtest.yml +0 -2
@@ -590,8 +590,6 @@ build-libvhost-user: image: $CI_REGISTRY_IMAGE/qemu/fedora:latest needs: job: amd64-fedora-container - before_script: - - dnf install -y meson ninja-build script: - mkdir subprojects/libvhost-user/build - cd subprojects/libvhost-user/build
Signature · 1 line
2.27.0
Re: [PATCH 2/3] gitlab-ci: Remove superfluous "dnf install" statement
Quoted 9 lines · Thomas Huth
The container already features meson and ninja, so there is no need to try to install it with dnf again.
Signed-off-by Thomas Huth <[email protected]>--- .gitlab-ci.d/buildtest.yml | 2 -- 1 file changed, 2 deletions(-)
PATCH 3/3 gitlab-ci: Fix ..._RUNNER_AVAILABLE variables and document them
In reply to Thomas HuthThe patch that recently introduced the S390X_RUNNER_AVAILABLE variable in custom-runners.yml missed that the bottom half of the file is rather about aarch64 than s390x. Thus rename the S390X_RUNNER_AVAILABLE to AARCH64_RUNNER_AVAILABLE in those jobs.
Finally mention both variables in our CI documentation, too.
--- .gitlab-ci.d/custom-runners.yml | 12 ++++++------ docs/devel/ci.rst | 13 +++++++++++++ 2 files changed, 19 insertions(+), 6 deletions(-)
.gitlab-ci.d/custom-runners.yml +6 -6
@@ -137,7 +137,7 @@ ubuntu-20.04-aarch64-all-linux-static: - aarch64 rules: - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/' - - if: "$S390X_RUNNER_AVAILABLE" + - if: "$AARCH64_RUNNER_AVAILABLE" script: # --disable-libssh is needed because of https://bugs.launchpad.net/qemu/+bug/1838763 # --disable-glusterfs is needed because there's no static version of those libs in distro supplied packages
@@ -157,7 +157,7 @@ ubuntu-20.04-aarch64-all: - aarch64 rules: - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/' - - if: "$S390X_RUNNER_AVAILABLE" + - if: "$AARCH64_RUNNER_AVAILABLE" script: - mkdir build - cd build
@@ -174,7 +174,7 @@ ubuntu-20.04-aarch64-alldbg: - aarch64 rules: - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/' - - if: "$S390X_RUNNER_AVAILABLE" + - if: "$AARCH64_RUNNER_AVAILABLE" script: - mkdir build - cd build
@@ -193,7 +193,7 @@ ubuntu-20.04-aarch64-clang: rules: - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/' when: manual - - if: "$S390X_RUNNER_AVAILABLE" + - if: "$AARCH64_RUNNER_AVAILABLE" when: manual script: - mkdir build
@@ -211,7 +211,7 @@ ubuntu-20.04-aarch64-tci: - aarch64 rules: - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/' - - if: "$S390X_RUNNER_AVAILABLE" + - if: "$AARCH64_RUNNER_AVAILABLE" script: - mkdir build - cd build
@@ -228,7 +228,7 @@ ubuntu-20.04-aarch64-notcg: rules: - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/' when: manual - - if: "$S390X_RUNNER_AVAILABLE" + - if: "$AARCH64_RUNNER_AVAILABLE" when: manual script: - mkdir build
docs/devel/ci.rst +13 -0
@@ -48,6 +48,19 @@ these artifacts are not already cached, downloading them make the jobs reach the timeout limit). Set this variable to have the tests using the Avocado framework run automatically. +AARCH64_RUNNER_AVAILABLE +~~~~~~~~~~~~~~~~~~~~~~~~ +If you've got access to an aarch64 host that can be used as a gitlab-CI +runner, you can set this variable to enable the tests that require this +kind of host. The runner should be tagged with "aarch64". + +S390X_RUNNER_AVAILABLE +~~~~~~~~~~~~~~~~~~~~~~ +If you've got access to an IBM Z host that can be used as a gitlab-CI +runner, you can set this variable to enable the tests that require this +kind of host. The runner should be tagged with "s390x". + + Jobs on Custom Runners ======================
Signature · 1 line
2.27.0
Re: [PATCH 3/3] gitlab-ci: Fix ..._RUNNER_AVAILABLE variables and document them
Quoted 4 lines · Thomas Huth
The patch that recently introduced the
S390X_RUNNER_AVAILABLEvariable in custom-runners.yml missed that the bottom half of the file is rather about aarch64 than s390x. Thus rename theS390X_RUNNER_AVAILABLEtoAARCH64_RUNNER_AVAILABLEin those jobs.
Oops.
Quoted 1 line
Finally mention both variables in our CI documentation, too.
Thanks.
Maybe "Fix ${arch}_RUNNER_AVAILABLE" as subject.
Quoted 90 lines
Fixes c5dd0f0342 ("Improve rules for the staging branch")Signed-off-by Thomas Huth <[email protected]>--- .gitlab-ci.d/custom-runners.yml | 12 ++++++------ docs/devel/ci.rst | 13 +++++++++++++ 2 files changed, 19 insertions(+), 6 deletions(-).gitlab-ci.d/custom-runners.yml +6
-6@@ -137,7 +137,7 @@ ubuntu-20.04-aarch64-all-linux-static: - aarch64 rules: - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/' - - if: "$S390X_RUNNER_AVAILABLE" + - if: "$AARCH64_RUNNER_AVAILABLE" script: # --disable-libssh is needed because of https://bugs.launchpad.net/qemu/+bug/1838763 # --disable-glusterfs is needed because there's no static version of those libs in distro supplied packages@@ -157,7 +157,7 @@ ubuntu-20.04-aarch64-all: - aarch64 rules: - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/' - - if: "$S390X_RUNNER_AVAILABLE" + - if: "$AARCH64_RUNNER_AVAILABLE" script: - mkdir build - cd build@@ -174,7 +174,7 @@ ubuntu-20.04-aarch64-alldbg: - aarch64 rules: - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/' - - if: "$S390X_RUNNER_AVAILABLE" + - if: "$AARCH64_RUNNER_AVAILABLE" script: - mkdir build - cd build@@ -193,7 +193,7 @@ ubuntu-20.04-aarch64-clang: rules: - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/' when: manual - - if: "$S390X_RUNNER_AVAILABLE" + - if: "$AARCH64_RUNNER_AVAILABLE" when: manual script: - mkdir build@@ -211,7 +211,7 @@ ubuntu-20.04-aarch64-tci: - aarch64 rules: - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/' - - if: "$S390X_RUNNER_AVAILABLE" + - if: "$AARCH64_RUNNER_AVAILABLE" script: - mkdir build - cd build@@ -228,7 +228,7 @@ ubuntu-20.04-aarch64-notcg: rules: - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/' when: manual - - if: "$S390X_RUNNER_AVAILABLE" + - if: "$AARCH64_RUNNER_AVAILABLE" when: manual script: - mkdir builddocs/devel/ci.rst +13
-0@@ -48,6 +48,19 @@ these artifacts are not already cached, downloading them make the jobs reach the timeout limit). Set this variable to have the tests using the Avocado framework run automatically. +AARCH64_RUNNER_AVAILABLE +~~~~~~~~~~~~~~~~~~~~~~~~ +If you've got access to an aarch64 host that can be used as a gitlab-CI +runner, you can set this variable to enable the tests that require this +kind of host. The runner should be tagged with "aarch64". + +S390X_RUNNER_AVAILABLE +~~~~~~~~~~~~~~~~~~~~~~ +If you've got access to an IBM Z host that can be used as a gitlab-CI +runner, you can set this variable to enable the tests that require this +kind of host. The runner should be tagged with "s390x". + + Jobs on Custom Runners ======================
Re: [PATCH 3/3] gitlab-ci: Fix ..._RUNNER_AVAILABLE variables and document them
In reply to Thomas HuthQuoted 6 lines · Thomas Huth
The patch that recently introduced the
S390X_RUNNER_AVAILABLEvariable in custom-runners.yml missed that the bottom half of the file is rather about aarch64 than s390x. Thus rename theS390X_RUNNER_AVAILABLEtoAARCH64_RUNNER_AVAILABLEin those jobs.
Quoted 9 lines
Finally mention both variables in our CI documentation, too.
Fixes c5dd0f0342 ("Improve rules for the staging branch")Signed-off-by Thomas Huth <[email protected]>--- .gitlab-ci.d/custom-runners.yml | 12 ++++++------ docs/devel/ci.rst | 13 +++++++++++++ 2 files changed, 19 insertions(+), 6 deletions(-)
Alex Bennée
AppliedQuoted 2 lines · Thomas Huth
Here are three patches for some small issues that I noticed in our gitlab-CI files recently...
Queued to for-6.1/misc-fixes-for-rc2, thanks.
Quoted 10 lines
Thomas Huth (3):
gitlab-ci: Merge "build-disabled" with "build-without-default-features"
gitlab-ci: Remove superfluous "dnf install" statement
gitlab-ci: Fix ..._RUNNER_AVAILABLE variables and document them.gitlab-ci.d/buildtest.yml | 99 +++++---------------------------- .gitlab-ci.d/custom-runners.yml | 12 ++-- docs/devel/ci.rst | 13 +++++ 3 files changed, 32 insertions(+), 92 deletions(-)
Signature · 1 line
Alex Bennée