gitlab-ci: Add accelerator-specific Linux jobs
v3 cover gitlab-ci: Add accelerator-specific Linux jobs
Since v2:
- Fixed ARM Xen job
- Renamed jobs with -$accel trailer (Thomas)
Since v1:
- Documented cross_accel_build_job template (Claudio)
- Only add new job for s390x (Thomas)
- Do not add entry to MAINTAINERS (Daniel)
- Document 'build-tcg-disabled' job is X86 + KVM
- Drop the patches with negative review feedbacks
Hi,
I was custom to use Travis-CI for testing KVM builds on s390x/ppc with the Travis-CI jobs.
During October Travis-CI became unusable for me (extremely slow, see [1]). Then my free Travis account got updated to the new "10K credit minutes allotment" [2] which I burned without reading the notification email in time (I'd burn them eventually anyway).
Today Travis-CI is pointless to me. While I could pay to run my QEMU jobs, I don't think it is fair for an Open Source project to ask its forks to pay for a service.
As we want forks to run some CI before contributing patches, and we have cross-build Docker images available for Linux hosts, I added some cross KVM/Xen build jobs to Gitlab-CI.
Cross-building doesn't have the same coverage as native building, as we can not run the tests. But this is still useful to get link failures.
Resulting pipeline:
https://gitlab.com/philmd/qemu/-/pipelines/226240415
Regards,
Phil.
[1] https://travis-ci.community/t/build-delays-for-open-source-project/10272
[2] https://blog.travis-ci.com/2020-11-02-travis-ci-new-billing
Philippe Mathieu-Daudé (5):
gitlab-ci: Document 'build-tcg-disabled' is a KVM X86 job
gitlab-ci: Replace YAML anchors by extends (cross_system_build_job)
gitlab-ci: Introduce 'cross_accel_build_job' template
gitlab-ci: Add KVM s390x cross-build jobs
gitlab-ci: Add Xen cross-build jobs
.gitlab-ci.d/crossbuilds.yml | 78 ++++++++++++++++++++++++++---------- .gitlab-ci.yml | 5 +++ 2 files changed, 62 insertions(+), 21 deletions(-)
Signature · 1 line
2.26.2
PATCH v3 1/5 gitlab-ci: Document 'build-tcg-disabled' is a KVM X86 job
Document what this job cover (build X86 targets with KVM being the single accelerator available).
--- .gitlab-ci.yml | 5 +++++ 1 file changed, 5 insertions(+)
.gitlab-ci.yml +5 -0
@@ -220,6 +220,11 @@ build-disabled: s390x-softmmu i386-linux-user MAKE_CHECK_ARGS: check-qtest SPEED=slow +# This jobs explicitly disable TCG (--disable-tcg), KVM is detected by +# the configure script. The container doesn't contain Xen headers so +# Xen accelerator is not detected / selected. As result it build the +# i386-softmmu and x86_64-softmmu with KVM being the single accelerator +# available. build-tcg-disabled: <<: *native_build_job_definition variables:
Signature · 1 line
2.26.2
Re: [PATCH v3 1/5] gitlab-ci: Document 'build-tcg-disabled' is a KVM X86 job
Hi,
Quoted 8 lines · Philippe Mathieu-Daudé
Document what this job cover (build X86 targets with KVM being the single accelerator available).
Reviewed-by Thomas Huth <[email protected]>Signed-off-by Philippe Mathieu-Daudé <[email protected]>--- .gitlab-ci.yml | 5 +++++ 1 file changed, 5 insertions(+)
Quoted 17 lines
.gitlab-ci.yml +5
-0@@ -220,6 +220,11 @@ build-disabled: s390x-softmmu i386-linux-user MAKE_CHECK_ARGS: check-qtest SPEED=slow +# This jobs explicitly disable TCG (--disable-tcg), KVM is detected by +# the configure script. The container doesn't contain Xen headers so +# Xen accelerator is not detected / selected. As result it build the +# i386-softmmu and x86_64-softmmu with KVM being the single accelerator +# available. build-tcg-disabled: <<: *native_build_job_definition variables:
Re: [PATCH v3 1/5] gitlab-ci: Document 'build-tcg-disabled' is a KVM X86 job
In reply to Philippe Mathieu-DaudéQuoted 9 lines · Philippe Mathieu-Daudé
Document what this job cover (build X86 targets with KVM being the single accelerator available).
Reviewed-by Thomas Huth <[email protected]>Signed-off-by Philippe Mathieu-Daudé <[email protected]>--- .gitlab-ci.yml | 5 +++++ 1 file changed, 5 insertions(+)
PATCH v3 2/5 gitlab-ci: Replace YAML anchors by extends (cross_system_build_job)
In reply to Philippe Mathieu-Daudé'extends' is an alternative to using YAML anchors
and is a little more flexible and readable. See:
https://docs.gitlab.com/ee/ci/yaml/#extends
More importantly it allows exploding YAML jobs.
--- .gitlab-ci.d/crossbuilds.yml | 40 ++++++++++++++++++------------------ 1 file changed, 20 insertions(+), 20 deletions(-)
.gitlab-ci.d/crossbuilds.yml +20 -20
@@ -1,5 +1,5 @@ -.cross_system_build_job_template: &cross_system_build_job_definition +.cross_system_build_job: stage: build image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest timeout: 80m
@@ -13,7 +13,7 @@ xtensa-softmmu" - make -j$(expr $(nproc) + 1) all check-build -.cross_user_build_job_template: &cross_user_build_job_definition +.cross_user_build_job: stage: build image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest script:
@@ -24,91 +24,91 @@ - make -j$(expr $(nproc) + 1) all check-build cross-armel-system: - <<: *cross_system_build_job_definition + extends: .cross_system_build_job variables: IMAGE: debian-armel-cross cross-armel-user: - <<: *cross_user_build_job_definition + extends: .cross_user_build_job variables: IMAGE: debian-armel-cross cross-armhf-system: - <<: *cross_system_build_job_definition + extends: .cross_system_build_job variables: IMAGE: debian-armhf-cross cross-armhf-user: - <<: *cross_user_build_job_definition + extends: .cross_user_build_job variables: IMAGE: debian-armhf-cross cross-arm64-system: - <<: *cross_system_build_job_definition + extends: .cross_system_build_job variables: IMAGE: debian-arm64-cross cross-arm64-user: - <<: *cross_user_build_job_definition + extends: .cross_user_build_job variables: IMAGE: debian-arm64-cross cross-mips-system: - <<: *cross_system_build_job_definition + extends: .cross_system_build_job variables: IMAGE: debian-mips-cross cross-mips-user: - <<: *cross_user_build_job_definition + extends: .cross_user_build_job variables: IMAGE: debian-mips-cross cross-mipsel-system: - <<: *cross_system_build_job_definition + extends: .cross_system_build_job variables: IMAGE: debian-mipsel-cross cross-mipsel-user: - <<: *cross_user_build_job_definition + extends: .cross_user_build_job variables: IMAGE: debian-mipsel-cross cross-mips64el-system: - <<: *cross_system_build_job_definition + extends: .cross_system_build_job variables: IMAGE: debian-mips64el-cross cross-mips64el-user: - <<: *cross_user_build_job_definition + extends: .cross_user_build_job variables: IMAGE: debian-mips64el-cross cross-ppc64el-system: - <<: *cross_system_build_job_definition + extends: .cross_system_build_job variables: IMAGE: debian-ppc64el-cross cross-ppc64el-user: - <<: *cross_user_build_job_definition + extends: .cross_user_build_job variables: IMAGE: debian-ppc64el-cross cross-s390x-system: - <<: *cross_system_build_job_definition + extends: .cross_system_build_job variables: IMAGE: debian-s390x-cross cross-s390x-user: - <<: *cross_user_build_job_definition + extends: .cross_user_build_job variables: IMAGE: debian-s390x-cross cross-win32-system: - <<: *cross_system_build_job_definition + extends: .cross_system_build_job variables: IMAGE: fedora-win32-cross cross-win64-system: - <<: *cross_system_build_job_definition + extends: .cross_system_build_job variables: IMAGE: fedora-win64-cross
Signature · 1 line
2.26.2
PATCH v3 3/5 gitlab-ci: Introduce 'cross_accel_build_job' template
In reply to Philippe Mathieu-DaudéIntroduce a job template to cross-build accelerator specific jobs (enable a specific accelerator, disabling the others).
The specific accelerator is selected by the $ACCEL environment variable (default to KVM).
Extra options such disabling other accelerators are passed via the $ACCEL_CONFIGURE_OPTS environment variable.
--- .gitlab-ci.d/crossbuilds.yml | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)
.gitlab-ci.d/crossbuilds.yml +17 -0
@@ -13,6 +13,23 @@ xtensa-softmmu" - make -j$(expr $(nproc) + 1) all check-build +# Job to cross-build specific accelerators. +# +# Set the $ACCEL variable to select the specific accelerator (default to +# KVM), and set extra options (such disabling other accelerators) via the +# $ACCEL_CONFIGURE_OPTS variable. +.cross_accel_build_job: + stage: build + image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest + timeout: 30m + script: + - mkdir build + - cd build + - PKG_CONFIG_PATH=$PKG_CONFIG_PATH + ../configure --enable-werror $QEMU_CONFIGURE_OPTS --disable-tools + --enable-${ACCEL:-kvm} $ACCEL_CONFIGURE_OPTS + - make -j$(expr $(nproc) + 1) all check-build + .cross_user_build_job: stage: build image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
Signature · 1 line
2.26.2
PATCH v3 4/5 gitlab-ci: Add KVM s390x cross-build jobs
In reply to Philippe Mathieu-DaudéCross-build s390x target with only KVM accelerator enabled.
--- .gitlab-ci.d/crossbuilds.yml | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
.gitlab-ci.d/crossbuilds.yml +6 -1
@@ -1,4 +1,3 @@ - .cross_system_build_job: stage: build image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
@@ -120,6 +119,12 @@ cross-s390x-user: variables: IMAGE: debian-s390x-cross +cross-s390x-kvm-only: + extends: .cross_accel_build_job + variables: + IMAGE: debian-s390x-cross + ACCEL_CONFIGURE_OPTS: --disable-tcg + cross-win32-system: extends: .cross_system_build_job variables:
Signature · 1 line
2.26.2
PATCH v3 5/5 gitlab-ci: Add Xen cross-build jobs
In reply to Philippe Mathieu-DaudéCross-build ARM and X86 targets with only Xen accelerator enabled.
--- .gitlab-ci.d/crossbuilds.yml | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
.gitlab-ci.d/crossbuilds.yml +14 -0
@@ -134,3 +134,17 @@ cross-win64-system: extends: .cross_system_build_job variables: IMAGE: fedora-win64-cross + +cross-amd64-xen-only: + extends: .cross_accel_build_job + variables: + IMAGE: debian-amd64-cross + ACCEL: xen + ACCEL_CONFIGURE_OPTS: --disable-tcg --disable-kvm + +cross-arm64-xen-only: + extends: .cross_accel_build_job + variables: + IMAGE: debian-arm64-cross + ACCEL: xen + ACCEL_CONFIGURE_OPTS: --disable-tcg --disable-kvm
Signature · 1 line
2.26.2
Re: [PATCH v3 5/5] gitlab-ci: Add Xen cross-build jobs
Quoted 19 lines · Philippe Mathieu-Daudé
Cross-build ARM and X86 targets with only Xen accelerator enabled.
Signed-off-by Philippe Mathieu-Daudé <[email protected]>--- .gitlab-ci.d/crossbuilds.yml | 14 ++++++++++++++ 1 file changed, 14 insertions(+).gitlab-ci.d/crossbuilds.yml +4
-0@@ -134,3 +134,17 @@ cross-win64-system: extends: .cross_system_build_job variables: IMAGE: fedora-win64-cross + +cross-amd64-xen-only: + extends: .cross_accel_build_job + variables:
.gitlab-ci.d/crossbuilds.yml cross-win64-system: · line 150
Quoted 10 lines
.gitlab-ci.d/crossbuilds.yml +10
-0+ IMAGE: debian-amd64-cross + ACCEL: xen + ACCEL_CONFIGURE_OPTS: --disable-tcg --disable-kvm + +cross-arm64-xen-only: + extends: .cross_accel_build_job + variables: + IMAGE: debian-arm64-cross + ACCEL: xen + ACCEL_CONFIGURE_OPTS: --disable-tcg --disable-kvm