Skip to contentSkip to conversation

gitlab-ci: Add accelerator-specific Linux jobs

PATCH Superseded Philippe Mathieu-Daudé 2 participants 12 messages Dec 7, 2020 KVM

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

v2 cover gitlab-ci: Add accelerator-specific Linux jobs

Since v1:
- Documented cross_accel_build_job template (Claudio)
- Only add new job for s390x (Thomas)
- Do not add entry to MAINTAINERS (Daniel)
- Document 'build-tcg-disabled' job is X86 + KVM
- Drop the patches with negative review feedbacks

Hi,

I was custom to use Travis-CI for testing KVM builds on s390x/ppc with the Travis-CI jobs.

During October Travis-CI became unusable for me (extremely slow, see [1]). Then my free Travis account got updated to the new "10K credit minutes allotment" [2] which I burned without reading the notification email in time (I'd burn them eventually anyway).

Today Travis-CI is pointless to me. While I could pay to run my QEMU jobs, I don't think it is fair for an Open Source project to ask its forks to pay for a service.

As we want forks to run some CI before contributing patches, and we have cross-build Docker images available for Linux hosts, I added some cross KVM/Xen build jobs to Gitlab-CI.

Cross-building doesn't have the same coverage as native building, as we can not run the tests. But this is still useful to get link failures.

Resulting pipeline:
https://gitlab.com/philmd/qemu/-/pipelines/225948077

Regards,

Phil.

[1] https://travis-ci.community/t/build-delays-for-open-source-project/10272
[2] https://blog.travis-ci.com/2020-11-02-travis-ci-new-billing

Philippe Mathieu-Daudé (5):
gitlab-ci: Document 'build-tcg-disabled' is a KVM X86 job
gitlab-ci: Replace YAML anchors by extends (cross_system_build_job)
gitlab-ci: Introduce 'cross_accel_build_job' template
gitlab-ci: Add KVM s390x cross-build jobs
gitlab-ci: Add Xen cross-build jobs

 .gitlab-ci.d/crossbuilds.yml | 80 ++++++++++++++++++++++++++----------
 .gitlab-ci.yml               |  5 +++
 2 files changed, 64 insertions(+), 21 deletions(-)
Signature · 1 line
2.26.2
#1 lore raw

PATCH v2 1/5 gitlab-ci: Document 'build-tcg-disabled' is a KVM X86 job

Document what this job cover (build X86 targets with KVM being the single accelerator available).

---
 .gitlab-ci.yml | 5 +++++
 1 file changed, 5 insertions(+)
.gitlab-ci.yml +5 -0
index d0173e82b16..ee31b1020fe 100644
@@ -220,6 +220,11 @@ build-disabled:
       s390x-softmmu i386-linux-user
     MAKE_CHECK_ARGS: check-qtest SPEED=slow
 
+# This jobs explicitly disable TCG (--disable-tcg), KVM is detected by
+# the configure script. The container doesn't contain Xen headers so
+# Xen accelerator is not detected / selected. As result it build the
+# i386-softmmu and x86_64-softmmu with KVM being the single accelerator
+# available.
 build-tcg-disabled:
   <<: *native_build_job_definition
   variables:
Signature · 1 line
2.26.2
Reviewed-by on v2 1/5
#2 lore raw

Re: [PATCH v2 1/5] gitlab-ci: Document 'build-tcg-disabled' is a KVM X86 job

Quoted 15 lines · Philippe Mathieu-Daudé

Document what this job cover (build X86 targets with KVM being the single accelerator available).

---
 .gitlab-ci.yml | 5 +++++
 1 file changed, 5 insertions(+)
.gitlab-ci.yml +0 -0
index d0173e82b16..ee31b1020fe 100644
@@ -220,6 +220,11 @@ build-disabled:
       s390x-softmmu i386-linux-user

.gitlab-ci.yml build-disabled: · line 230

Quoted 10 lines
.gitlab-ci.yml +5 -0
     MAKE_CHECK_ARGS: check-qtest SPEED=slow
 
+# This jobs explicitly disable TCG (--disable-tcg), KVM is detected by
+# the configure script. The container doesn't contain Xen headers so
+# Xen accelerator is not detected / selected. As result it build the
+# i386-softmmu and x86_64-softmmu with KVM being the single accelerator
+# available.
 build-tcg-disabled:
   <<: *native_build_job_definition
   variables:
#3 lore raw

PATCH v2 2/5 gitlab-ci: Replace YAML anchors by extends (cross_system_build_job)

In reply to Philippe Mathieu-Daudé

'extends' is an alternative to using YAML anchors
and is a little more flexible and readable. See:
https://docs.gitlab.com/ee/ci/yaml/#extends

More importantly it allows exploding YAML jobs.

---
 .gitlab-ci.d/crossbuilds.yml | 40 ++++++++++++++++++------------------
 1 file changed, 20 insertions(+), 20 deletions(-)
.gitlab-ci.d/crossbuilds.yml +20 -20
index 03ebfabb3fa..099949aaef3 100644
@@ -1,5 +1,5 @@
 
-.cross_system_build_job_template: &cross_system_build_job_definition
+.cross_system_build_job:
   stage: build
   image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
   timeout: 80m
@@ -13,7 +13,7 @@
           xtensa-softmmu"
     - make -j$(expr $(nproc) + 1) all check-build
 
-.cross_user_build_job_template: &cross_user_build_job_definition
+.cross_user_build_job:
   stage: build
   image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
   script:
@@ -24,91 +24,91 @@
     - make -j$(expr $(nproc) + 1) all check-build
 
 cross-armel-system:
-  <<: *cross_system_build_job_definition
+  extends: .cross_system_build_job
   variables:
     IMAGE: debian-armel-cross
 
 cross-armel-user:
-  <<: *cross_user_build_job_definition
+  extends: .cross_user_build_job
   variables:
     IMAGE: debian-armel-cross
 
 cross-armhf-system:
-  <<: *cross_system_build_job_definition
+  extends: .cross_system_build_job
   variables:
     IMAGE: debian-armhf-cross
 
 cross-armhf-user:
-  <<: *cross_user_build_job_definition
+  extends: .cross_user_build_job
   variables:
     IMAGE: debian-armhf-cross
 
 cross-arm64-system:
-  <<: *cross_system_build_job_definition
+  extends: .cross_system_build_job
   variables:
     IMAGE: debian-arm64-cross
 
 cross-arm64-user:
-  <<: *cross_user_build_job_definition
+  extends: .cross_user_build_job
   variables:
     IMAGE: debian-arm64-cross
 
 cross-mips-system:
-  <<: *cross_system_build_job_definition
+  extends: .cross_system_build_job
   variables:
     IMAGE: debian-mips-cross
 
 cross-mips-user:
-  <<: *cross_user_build_job_definition
+  extends: .cross_user_build_job
   variables:
     IMAGE: debian-mips-cross
 
 cross-mipsel-system:
-  <<: *cross_system_build_job_definition
+  extends: .cross_system_build_job
   variables:
     IMAGE: debian-mipsel-cross
 
 cross-mipsel-user:
-  <<: *cross_user_build_job_definition
+  extends: .cross_user_build_job
   variables:
     IMAGE: debian-mipsel-cross
 
 cross-mips64el-system:
-  <<: *cross_system_build_job_definition
+  extends: .cross_system_build_job
   variables:
     IMAGE: debian-mips64el-cross
 
 cross-mips64el-user:
-  <<: *cross_user_build_job_definition
+  extends: .cross_user_build_job
   variables:
     IMAGE: debian-mips64el-cross
 
 cross-ppc64el-system:
-  <<: *cross_system_build_job_definition
+  extends: .cross_system_build_job
   variables:
     IMAGE: debian-ppc64el-cross
 
 cross-ppc64el-user:
-  <<: *cross_user_build_job_definition
+  extends: .cross_user_build_job
   variables:
     IMAGE: debian-ppc64el-cross
 
 cross-s390x-system:
-  <<: *cross_system_build_job_definition
+  extends: .cross_system_build_job
   variables:
     IMAGE: debian-s390x-cross
 
 cross-s390x-user:
-  <<: *cross_user_build_job_definition
+  extends: .cross_user_build_job
   variables:
     IMAGE: debian-s390x-cross
 
 cross-win32-system:
-  <<: *cross_system_build_job_definition
+  extends: .cross_system_build_job
   variables:
     IMAGE: fedora-win32-cross
 
 cross-win64-system:
-  <<: *cross_system_build_job_definition
+  extends: .cross_system_build_job
   variables:
     IMAGE: fedora-win64-cross
Signature · 1 line
2.26.2
#4 lore raw

PATCH v2 3/5 gitlab-ci: Introduce 'cross_accel_build_job' template

In reply to Philippe Mathieu-Daudé

Introduce a job template to cross-build accelerator specific jobs (enable a specific accelerator, disabling the others).

The specific accelerator is selected by the $ACCEL environment variable (default to KVM).

Extra options such disabling other accelerators are passed via the $ACCEL_CONFIGURE_OPTS environment variable.

---
 .gitlab-ci.d/crossbuilds.yml | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
.gitlab-ci.d/crossbuilds.yml +17 -0
index 099949aaef3..d8685ade376 100644
@@ -13,6 +13,23 @@
           xtensa-softmmu"
     - make -j$(expr $(nproc) + 1) all check-build
 
+# Job to cross-build specific accelerators.
+#
+# Set the $ACCEL variable to select the specific accelerator (default to
+# KVM), and set extra options (such disabling other accelerators) via the
+# $ACCEL_CONFIGURE_OPTS variable.
+.cross_accel_build_job:
+  stage: build
+  image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
+  timeout: 30m
+  script:
+    - mkdir build
+    - cd build
+    - PKG_CONFIG_PATH=$PKG_CONFIG_PATH
+      ../configure --enable-werror $QEMU_CONFIGURE_OPTS --disable-tools
+        --enable-${ACCEL:-kvm} --target-list="$TARGETS" $ACCEL_CONFIGURE_OPTS
+    - make -j$(expr $(nproc) + 1) all check-build
+
 .cross_user_build_job:
   stage: build
   image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
Signature · 1 line
2.26.2
Reviewed-by on v2 3/5
#5 lore raw

Re: [PATCH v2 3/5] gitlab-ci: Introduce 'cross_accel_build_job' template

Quoted 32 lines · Philippe Mathieu-Daudé

Introduce a job template to cross-build accelerator specific jobs (enable a specific accelerator, disabling the others).

The specific accelerator is selected by the $ACCEL environment variable (default to KVM).

Extra options such disabling other accelerators are passed via the $ACCEL_CONFIGURE_OPTS environment variable.

---
 .gitlab-ci.d/crossbuilds.yml | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
.gitlab-ci.d/crossbuilds.yml +10 -0
index 099949aaef3..d8685ade376 100644
@@ -13,6 +13,23 @@
           xtensa-softmmu"
     - make -j$(expr $(nproc) + 1) all check-build
 
+# Job to cross-build specific accelerators.
+#
+# Set the $ACCEL variable to select the specific accelerator (default to
+# KVM), and set extra options (such disabling other accelerators) via the
+# $ACCEL_CONFIGURE_OPTS variable.
+.cross_accel_build_job:
+  stage: build
+  image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
+  timeout: 30m
+  script:

.gitlab-ci.d/crossbuilds.yml line 35

Quoted 10 lines
.gitlab-ci.d/crossbuilds.yml +7 -0
+    - mkdir build
+    - cd build
+    - PKG_CONFIG_PATH=$PKG_CONFIG_PATH
+      ../configure --enable-werror $QEMU_CONFIGURE_OPTS --disable-tools
+        --enable-${ACCEL:-kvm} --target-list="$TARGETS" $ACCEL_CONFIGURE_OPTS
+    - make -j$(expr $(nproc) + 1) all check-build
+
 .cross_user_build_job:
   stage: build
   image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest

I wonder whether we could also simply use the .cross_user_build_job - e.g. by adding a $EXTRA_CONFIGURE_OPTS variable in the "../configure ..." line so that the accel-jobs could use that for their --enable... and --disable... settings?

Anyway, I've got no strong opinion on that one, and I'm also fine if we add this new template, so:

#6 lore raw

Re: [PATCH v2 3/5] gitlab-ci: Introduce 'cross_accel_build_job' template

Quoted 40 lines · Thomas Huth
Quoted 39 lines · Philippe Mathieu-Daudé

Introduce a job template to cross-build accelerator specific jobs (enable a specific accelerator, disabling the others).

The specific accelerator is selected by the $ACCEL environment variable (default to KVM).

Extra options such disabling other accelerators are passed via the $ACCEL_CONFIGURE_OPTS environment variable.

---
 .gitlab-ci.d/crossbuilds.yml | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
.gitlab-ci.d/crossbuilds.yml +17 -0
index 099949aaef3..d8685ade376 100644
@@ -13,6 +13,23 @@
           xtensa-softmmu"
     - make -j$(expr $(nproc) + 1) all check-build
 
+# Job to cross-build specific accelerators.
+#
+# Set the $ACCEL variable to select the specific accelerator (default to
+# KVM), and set extra options (such disabling other accelerators) via the
+# $ACCEL_CONFIGURE_OPTS variable.
+.cross_accel_build_job:
+  stage: build
+  image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
+  timeout: 30m
+  script:
+    - mkdir build
+    - cd build
+    - PKG_CONFIG_PATH=$PKG_CONFIG_PATH
+      ../configure --enable-werror $QEMU_CONFIGURE_OPTS --disable-tools
+        --enable-${ACCEL:-kvm} --target-list="$TARGETS" $ACCEL_CONFIGURE_OPTS
+    - make -j$(expr $(nproc) + 1) all check-build
+
Quoted 8 lines

.gitlab-ci.d/crossbuilds.yml line 35

Quoted 3 lines
.gitlab-ci.d/crossbuilds.yml +0 -0
 .cross_user_build_job:
   stage: build
   image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest

I wonder whether we could also simply use the .cross_user_build_job - e.g. by adding a $EXTRA_CONFIGURE_OPTS variable in the "../configure ..." line so that the accel-jobs could use that for their --enable... and --disable... settings?

Well cross_user_build_job build tools (I'm not sure that desired).

Quoted 4 lines

Anyway, I've got no strong opinion on that one, and I'm also fine if we add this new template, so:

Thanks, we can improve on top.

#7 lore raw

PATCH v2 4/5 gitlab-ci: Add KVM s390x cross-build jobs

In reply to Philippe Mathieu-Daudé

Cross-build s390x target with only KVM accelerator enabled.

---
 .gitlab-ci.d/crossbuilds.yml | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
.gitlab-ci.d/crossbuilds.yml +7 -1
index d8685ade376..7a94a66b4b3 100644
@@ -1,4 +1,3 @@
-
 .cross_system_build_job:
   stage: build
   image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
@@ -120,6 +119,13 @@ cross-s390x-user:
   variables:
     IMAGE: debian-s390x-cross
 
+cross-s390x-kvm:
+  extends: .cross_accel_build_job
+  variables:
+    IMAGE: debian-s390x-cross
+    TARGETS: s390x-softmmu
+    ACCEL_CONFIGURE_OPTS: --disable-tcg
+
 cross-win32-system:
   extends: .cross_system_build_job
   variables:
Signature · 1 line
2.26.2
Reviewed-by on v2 4/5
#8 lore raw

Re: [PATCH v2 4/5] gitlab-ci: Add KVM s390x cross-build jobs

Quoted 17 lines · Philippe Mathieu-Daudé

Cross-build s390x target with only KVM accelerator enabled.

---
 .gitlab-ci.d/crossbuilds.yml | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
.gitlab-ci.d/crossbuilds.yml +0 -1
index d8685ade376..7a94a66b4b3 100644
@@ -1,4 +1,3 @@
-
 .cross_system_build_job:
   stage: build
   image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest

.gitlab-ci.d/crossbuilds.yml cross-s390x-user: · line 122

Quoted 4 lines
.gitlab-ci.d/crossbuilds.yml +1 -0
@@ -120,6 +119,13 @@ cross-s390x-user:
   variables:
     IMAGE: debian-s390x-cross
 
+cross-s390x-kvm:

I'd still prefer "-no-tcg" or maybe "-kvm-only" ... but that's just a matter of taste, so:

#9 lore raw

PATCH v2 5/5 gitlab-ci: Add Xen cross-build jobs

In reply to Philippe Mathieu-Daudé

Cross-build ARM and X86 targets with only Xen accelerator enabled.

---
 .gitlab-ci.d/crossbuilds.yml | 15 +++++++++++++++
 1 file changed, 15 insertions(+)
.gitlab-ci.d/crossbuilds.yml +15 -0
index 7a94a66b4b3..31f10f1e145 100644
@@ -135,3 +135,18 @@ cross-win64-system:
   extends: .cross_system_build_job
   variables:
     IMAGE: fedora-win64-cross
+
+cross-amd64-xen:
+  extends: .cross_accel_build_job
+  variables:
+    IMAGE: debian-amd64-cross
+    ACCEL: xen
+    TARGETS: i386-softmmu,x86_64-softmmu
+    ACCEL_CONFIGURE_OPTS: --disable-tcg --disable-kvm
+
+cross-arm64-xen:
+  extends: .cross_accel_build_job
+  variables:
+    IMAGE: debian-arm64-cross
+    ACCEL: xen
+    TARGETS: aarch64-softmmu
Signature · 1 line
2.26.2
#10 lore raw

Re: [PATCH v2 5/5] gitlab-ci: Add Xen cross-build jobs

Quoted 20 lines · Philippe Mathieu-Daudé

Cross-build ARM and X86 targets with only Xen accelerator enabled.

---
 .gitlab-ci.d/crossbuilds.yml | 15 +++++++++++++++
 1 file changed, 15 insertions(+)
.gitlab-ci.d/crossbuilds.yml +5 -0
index 7a94a66b4b3..31f10f1e145 100644
@@ -135,3 +135,18 @@ cross-win64-system:
   extends: .cross_system_build_job
   variables:
     IMAGE: fedora-win64-cross
+
+cross-amd64-xen:
+  extends: .cross_accel_build_job
+  variables:
+    IMAGE: debian-amd64-cross

.gitlab-ci.d/crossbuilds.yml cross-win64-system: · line 152

Quoted 10 lines
.gitlab-ci.d/crossbuilds.yml +10 -0
+    ACCEL: xen
+    TARGETS: i386-softmmu,x86_64-softmmu
+    ACCEL_CONFIGURE_OPTS: --disable-tcg --disable-kvm
+
+cross-arm64-xen:
+  extends: .cross_accel_build_job
+  variables:
+    IMAGE: debian-arm64-cross
+    ACCEL: xen
+    TARGETS: aarch64-softmmu

Could you please simply replace aarch64-softmmu by arm-softmmu in the target-list-exclude statement in this file instead of adding a new job for arm64? That should have the same results and will spare us one job...

Thanks,
Thomas

#11 lore raw

Re: [PATCH v2 5/5] gitlab-ci: Add Xen cross-build jobs

Quoted 26 lines · Thomas Huth
Quoted 25 lines · Philippe Mathieu-Daudé

Cross-build ARM and X86 targets with only Xen accelerator enabled.

---
 .gitlab-ci.d/crossbuilds.yml | 15 +++++++++++++++
 1 file changed, 15 insertions(+)
.gitlab-ci.d/crossbuilds.yml +10 -0
index 7a94a66b4b3..31f10f1e145 100644
@@ -135,3 +135,18 @@ cross-win64-system:
   extends: .cross_system_build_job
   variables:
     IMAGE: fedora-win64-cross
+
+cross-amd64-xen:
+  extends: .cross_accel_build_job
+  variables:
+    IMAGE: debian-amd64-cross
+    ACCEL: xen
+    TARGETS: i386-softmmu,x86_64-softmmu
+    ACCEL_CONFIGURE_OPTS: --disable-tcg --disable-kvm
+
+cross-arm64-xen:
Quoted 8 lines

.gitlab-ci.d/crossbuilds.yml cross-win64-system: · line 152

Quoted 5 lines
.gitlab-ci.d/crossbuilds.yml +5 -0
+  extends: .cross_accel_build_job
+  variables:
+    IMAGE: debian-arm64-cross
+    ACCEL: xen
+    TARGETS: aarch64-softmmu

Could you please simply replace aarch64-softmmu by arm-softmmu in the target-list-exclude statement in this file instead of adding a new job for arm64? That should have the same results and will spare us one job...

Ah, I now see my mistake, this is not the job I wanted to add, I probably messed during rebase. I'll respin with the proper xen-only config.

Quoted 4 lines

Thanks,
Thomas