gitlab-ci: Allow forks to select & restrict build jobs
v2 cover gitlab-ci: Allow forks to select & restrict build jobs
Hi,
2 months ago GitLab added time limit to their free CI offer [1]. This series provide developers with the possibility to not run all jobs. By default all jobs are started, but we can restrict by selecting a subset of them.
Since v1:
- switch from "all but skip some" to "all or select some"
I tried to use a project-wide environment variable [2] to set the default set of selected features a fork is interested in, but it didn't work out well yet.
A friendly way to use this feature is with git aliases [3]:
$ git config alias.pushci_system \
'push -o ci.variable="QEMU_BUILD=system"'
$ git config alias.pushci_debian \
'push -o ci.variable="QEMU_BUILD=debian"'Then you can run the jobs based on Debian images (only) using:
$ git pushci_debian gitlab_repo my_branch_for_debian
Or run all system-mode emulation jobs only using:
$ git pushci_system my_gitlab_repo branch_with_system_feature
Comments welcomed!
Regards,
Phil.
[1] https://about.gitlab.com/releases/2020/09/01/ci-minutes-update-free-users/
[2] https://docs.gitlab.com/ee/ci/variables/README.html#create-a-custom-variable-in-the-ui
[3] https://docs.gitlab.com/ee/user/project/push_options.html#useful-git-aliases
Supersedes: <[email protected]>
Philippe Mathieu-Daudé (11):
gitlab-ci: Drop generic cache rule
gitlab-ci: Replace YAML anchors by extends (cross_system_build_job)
gitlab-ci: Replace YAML anchors by extends (native_build_job)
gitlab-ci: Replace YAML anchors by extends (native_test_job)
gitlab-ci: Replace YAML anchors by extends (acceptance_test_job)
gitlab-ci: Rename acceptance_test_job -> integration_test_job
gitlab-ci: Extract common job definition as 'cross_common_job'
gitlab-ci: Extract common job definition as 'native_common_job'
gitlab-ci: Add rules to select cross-jobs to build
gitlab-ci: Add rules to select building/testing native jobs
gitlab-ci: Move artifacts expiry rule to common 'native_build_job'
.gitlab-ci.d/crossbuilds.yml | 77 ++++++++++++------ .gitlab-ci.yml | 148 +++++++++++++++++++---------------- 2 files changed, 136 insertions(+), 89 deletions(-)
Signature · 1 line
2.26.2
RFC PATCH v2 1/11 gitlab-ci: Drop generic cache rule
This cache rule is meant for Avocado artifacts, but affects all jobs. Moreover the 'acceptance_template' template already include a more detailled rule to cache artifacts.
--- .gitlab-ci.yml | 6 ------ 1 file changed, 6 deletions(-)
.gitlab-ci.yml +0 -6
@@ -7,12 +7,6 @@ stages: - build - test -# We assume GitLab has it's own caching set up for RPM/APT repositories so we -# just take care of avocado assets here. -cache: - paths: - - $HOME/avocado/data/cache - include: - local: '/.gitlab-ci.d/edk2.yml' - local: '/.gitlab-ci.d/opensbi.yml'
Signature · 1 line
2.26.2
RFC PATCH v2 2/11 gitlab-ci: Replace YAML anchors by extends (cross_system_build_job)
In reply to Philippe Mathieu-Daudé'extends' is an alternative to using YAML anchors
and is a little more flexible and readable. See:
https://docs.gitlab.com/ee/ci/yaml/#extends
--- .gitlab-ci.d/crossbuilds.yml | 40 ++++++++++++++++++------------------ 1 file changed, 20 insertions(+), 20 deletions(-)
.gitlab-ci.d/crossbuilds.yml +20 -20
@@ -1,5 +1,5 @@ -.cross_system_build_job_template: &cross_system_build_job_definition +.cross_system_build_job: stage: build image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest timeout: 80m
@@ -13,7 +13,7 @@ xtensa-softmmu" - make -j$(expr $(nproc) + 1) all check-build -.cross_user_build_job_template: &cross_user_build_job_definition +.cross_user_build_job: stage: build image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest script:
@@ -24,91 +24,91 @@ - make -j$(expr $(nproc) + 1) all check-build cross-armel-system: - <<: *cross_system_build_job_definition + extends: .cross_system_build_job variables: IMAGE: debian-armel-cross cross-armel-user: - <<: *cross_user_build_job_definition + extends: .cross_user_build_job variables: IMAGE: debian-armel-cross cross-armhf-system: - <<: *cross_system_build_job_definition + extends: .cross_system_build_job variables: IMAGE: debian-armhf-cross cross-armhf-user: - <<: *cross_user_build_job_definition + extends: .cross_user_build_job variables: IMAGE: debian-armhf-cross cross-arm64-system: - <<: *cross_system_build_job_definition + extends: .cross_system_build_job variables: IMAGE: debian-arm64-cross cross-arm64-user: - <<: *cross_user_build_job_definition + extends: .cross_user_build_job variables: IMAGE: debian-arm64-cross cross-mips-system: - <<: *cross_system_build_job_definition + extends: .cross_system_build_job variables: IMAGE: debian-mips-cross cross-mips-user: - <<: *cross_user_build_job_definition + extends: .cross_user_build_job variables: IMAGE: debian-mips-cross cross-mipsel-system: - <<: *cross_system_build_job_definition + extends: .cross_system_build_job variables: IMAGE: debian-mipsel-cross cross-mipsel-user: - <<: *cross_user_build_job_definition + extends: .cross_user_build_job variables: IMAGE: debian-mipsel-cross cross-mips64el-system: - <<: *cross_system_build_job_definition + extends: .cross_system_build_job variables: IMAGE: debian-mips64el-cross cross-mips64el-user: - <<: *cross_user_build_job_definition + extends: .cross_user_build_job variables: IMAGE: debian-mips64el-cross cross-ppc64el-system: - <<: *cross_system_build_job_definition + extends: .cross_system_build_job variables: IMAGE: debian-ppc64el-cross cross-ppc64el-user: - <<: *cross_user_build_job_definition + extends: .cross_user_build_job variables: IMAGE: debian-ppc64el-cross cross-s390x-system: - <<: *cross_system_build_job_definition + extends: .cross_system_build_job variables: IMAGE: debian-s390x-cross cross-s390x-user: - <<: *cross_user_build_job_definition + extends: .cross_user_build_job variables: IMAGE: debian-s390x-cross cross-win32-system: - <<: *cross_system_build_job_definition + extends: .cross_system_build_job variables: IMAGE: fedora-win32-cross cross-win64-system: - <<: *cross_system_build_job_definition + extends: .cross_system_build_job variables: IMAGE: fedora-win64-cross
Signature · 1 line
2.26.2
RFC PATCH v2 3/11 gitlab-ci: Replace YAML anchors by extends (native_build_job)
In reply to Philippe Mathieu-Daudé'extends' is an alternative to using YAML anchors
and is a little more flexible and readable. See:
https://docs.gitlab.com/ee/ci/yaml/#extends
--- .gitlab-ci.yml | 32 ++++++++++++++++---------------- 1 file changed, 16 insertions(+), 16 deletions(-)
.gitlab-ci.yml +16 -16
@@ -13,7 +13,7 @@ include: - local: '/.gitlab-ci.d/containers.yml' - local: '/.gitlab-ci.d/crossbuilds.yml' -.native_build_job_template: &native_build_job_definition +.native_build_job: stage: build image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest before_script:
@@ -68,7 +68,7 @@ include: - du -chs ${CI_PROJECT_DIR}/avocado-cache build-system-ubuntu: - <<: *native_build_job_definition + extends: .native_build_job variables: IMAGE: ubuntu2004 TARGETS: aarch64-softmmu alpha-softmmu cris-softmmu hppa-softmmu
@@ -99,7 +99,7 @@ acceptance-system-ubuntu: <<: *acceptance_definition build-system-debian: - <<: *native_build_job_definition + extends: .native_build_job variables: IMAGE: debian-amd64 TARGETS: arm-softmmu avr-softmmu i386-softmmu mipsel-softmmu
@@ -130,7 +130,7 @@ acceptance-system-debian: <<: *acceptance_definition build-system-fedora: - <<: *native_build_job_definition + extends: .native_build_job variables: IMAGE: fedora CONFIGURE_ARGS: --disable-gcrypt --enable-nettle
@@ -162,7 +162,7 @@ acceptance-system-fedora: <<: *acceptance_definition build-system-centos: - <<: *native_build_job_definition + extends: .native_build_job variables: IMAGE: centos8 CONFIGURE_ARGS: --disable-nettle --enable-gcrypt
@@ -194,7 +194,7 @@ acceptance-system-centos: <<: *acceptance_definition build-disabled: - <<: *native_build_job_definition + extends: .native_build_job variables: IMAGE: fedora CONFIGURE_ARGS: --disable-attr --disable-avx2 --disable-bochs
@@ -219,7 +219,7 @@ build-disabled: MAKE_CHECK_ARGS: check-qtest SPEED=slow build-tcg-disabled: - <<: *native_build_job_definition + extends: .native_build_job variables: IMAGE: centos8 script:
@@ -239,7 +239,7 @@ build-tcg-disabled: 260 261 262 263 264 270 272 273 277 279 build-user: - <<: *native_build_job_definition + extends: .native_build_job variables: IMAGE: debian-all-test-cross CONFIGURE_ARGS: --disable-tools --disable-system
@@ -249,7 +249,7 @@ build-user: # we skip sparc64-linux-user until it has been fixed somewhat # we skip cris-linux-user as it doesn't use the common run loop build-user-plugins: - <<: *native_build_job_definition + extends: .native_build_job variables: IMAGE: debian-all-test-cross CONFIGURE_ARGS: --disable-tools --disable-system --enable-plugins --enable-debug-tcg --target-list-exclude=sparc64-linux-user,cris-linux-user
@@ -257,7 +257,7 @@ build-user-plugins: timeout: 1h 30m build-clang: - <<: *native_build_job_definition + extends: .native_build_job variables: IMAGE: fedora CONFIGURE_ARGS: --cc=clang --cxx=clang++
@@ -267,7 +267,7 @@ build-clang: # These targets are on the way out build-deprecated: - <<: *native_build_job_definition + extends: .native_build_job variables: IMAGE: debian-all-test-cross CONFIGURE_ARGS: --disable-docs --disable-tools
@@ -292,7 +292,7 @@ check-deprecated: allow_failure: true build-oss-fuzz: - <<: *native_build_job_definition + extends: .native_build_job variables: IMAGE: fedora script:
@@ -310,7 +310,7 @@ build-oss-fuzz: - cd build-oss-fuzz && make check-qtest-i386 check-unit build-tci: - <<: *native_build_job_definition + extends: .native_build_job variables: IMAGE: fedora script:
@@ -335,7 +335,7 @@ build-tci: # These jobs test old gcrypt and nettle from RHEL7 # which had some API differences. build-crypto-old-nettle: - <<: *native_build_job_definition + extends: .native_build_job variables: IMAGE: centos7 TARGETS: x86_64-softmmu x86_64-linux-user
@@ -356,7 +356,7 @@ check-crypto-old-nettle: build-crypto-old-gcrypt: - <<: *native_build_job_definition + extends: .native_build_job variables: IMAGE: centos7 TARGETS: x86_64-softmmu x86_64-linux-user
@@ -377,7 +377,7 @@ check-crypto-old-gcrypt: build-crypto-only-gnutls: - <<: *native_build_job_definition + extends: .native_build_job variables: IMAGE: centos7 TARGETS: x86_64-softmmu x86_64-linux-user
Signature · 1 line
2.26.2
RFC PATCH v2 4/11 gitlab-ci: Replace YAML anchors by extends (native_test_job)
In reply to Philippe Mathieu-Daudé'extends' is an alternative to using YAML anchors
and is a little more flexible and readable. See:
https://docs.gitlab.com/ee/ci/yaml/#extends
--- .gitlab-ci.yml | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-)
.gitlab-ci.yml +13 -13
@@ -34,7 +34,7 @@ include: make -j"$JOBS" $MAKE_CHECK_ARGS ; fi -.native_test_job_template: &native_test_job_definition +.native_test_job: stage: test image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest script:
@@ -80,7 +80,7 @@ build-system-ubuntu: - build check-system-ubuntu: - <<: *native_test_job_definition + extends: .native_test_job needs: - job: build-system-ubuntu artifacts: true
@@ -89,7 +89,7 @@ check-system-ubuntu: MAKE_CHECK_ARGS: check acceptance-system-ubuntu: - <<: *native_test_job_definition + extends: .native_test_job needs: - job: build-system-ubuntu artifacts: true
@@ -111,7 +111,7 @@ build-system-debian: - build check-system-debian: - <<: *native_test_job_definition + extends: .native_test_job needs: - job: build-system-debian artifacts: true
@@ -120,7 +120,7 @@ check-system-debian: MAKE_CHECK_ARGS: check acceptance-system-debian: - <<: *native_test_job_definition + extends: .native_test_job needs: - job: build-system-debian artifacts: true
@@ -143,7 +143,7 @@ build-system-fedora: - build check-system-fedora: - <<: *native_test_job_definition + extends: .native_test_job needs: - job: build-system-fedora artifacts: true
@@ -152,7 +152,7 @@ check-system-fedora: MAKE_CHECK_ARGS: check acceptance-system-fedora: - <<: *native_test_job_definition + extends: .native_test_job needs: - job: build-system-fedora artifacts: true
@@ -175,7 +175,7 @@ build-system-centos: - build check-system-centos: - <<: *native_test_job_definition + extends: .native_test_job needs: - job: build-system-centos artifacts: true
@@ -184,7 +184,7 @@ check-system-centos: MAKE_CHECK_ARGS: check acceptance-system-centos: - <<: *native_test_job_definition + extends: .native_test_job needs: - job: build-system-centos artifacts: true
@@ -282,7 +282,7 @@ build-deprecated: # We split the check-tcg step as test failures are expected but we still # want to catch the build breaking. check-deprecated: - <<: *native_test_job_definition + extends: .native_test_job needs: - job: build-deprecated artifacts: true
@@ -346,7 +346,7 @@ build-crypto-old-nettle: - build check-crypto-old-nettle: - <<: *native_test_job_definition + extends: .native_test_job needs: - job: build-crypto-old-nettle artifacts: true
@@ -367,7 +367,7 @@ build-crypto-old-gcrypt: - build check-crypto-old-gcrypt: - <<: *native_test_job_definition + extends: .native_test_job needs: - job: build-crypto-old-gcrypt artifacts: true
@@ -388,7 +388,7 @@ build-crypto-only-gnutls: - build check-crypto-only-gnutls: - <<: *native_test_job_definition + extends: .native_test_job needs: - job: build-crypto-only-gnutls artifacts: true
Signature · 1 line
2.26.2
RFC PATCH v2 5/11 gitlab-ci: Replace YAML anchors by extends (acceptance_test_job)
In reply to Philippe Mathieu-Daudé'extends' is an alternative to using YAML anchors
and is a little more flexible and readable. See:
https://docs.gitlab.com/ee/ci/yaml/#extends
--- .gitlab-ci.yml | 15 ++++++--------- 1 file changed, 6 insertions(+), 9 deletions(-)
.gitlab-ci.yml +6 -9
@@ -42,7 +42,8 @@ include: - find . -type f -exec touch {} + - make $MAKE_CHECK_ARGS -.acceptance_template: &acceptance_definition +.acceptance_test_job: + extends: .native_test_job cache: key: "${CI_JOB_NAME}-cache" paths:
@@ -89,14 +90,13 @@ check-system-ubuntu: MAKE_CHECK_ARGS: check acceptance-system-ubuntu: - extends: .native_test_job + extends: .acceptance_test_job needs: - job: build-system-ubuntu artifacts: true variables: IMAGE: ubuntu2004 MAKE_CHECK_ARGS: check-acceptance - <<: *acceptance_definition build-system-debian: extends: .native_build_job
@@ -120,14 +120,13 @@ check-system-debian: MAKE_CHECK_ARGS: check acceptance-system-debian: - extends: .native_test_job + extends: .acceptance_test_job needs: - job: build-system-debian artifacts: true variables: IMAGE: debian-amd64 MAKE_CHECK_ARGS: check-acceptance - <<: *acceptance_definition build-system-fedora: extends: .native_build_job
@@ -152,14 +151,13 @@ check-system-fedora: MAKE_CHECK_ARGS: check acceptance-system-fedora: - extends: .native_test_job + extends: .acceptance_test_job needs: - job: build-system-fedora artifacts: true variables: IMAGE: fedora MAKE_CHECK_ARGS: check-acceptance - <<: *acceptance_definition build-system-centos: extends: .native_build_job
@@ -184,14 +182,13 @@ check-system-centos: MAKE_CHECK_ARGS: check acceptance-system-centos: - extends: .native_test_job + extends: .acceptance_test_job needs: - job: build-system-centos artifacts: true variables: IMAGE: centos8 MAKE_CHECK_ARGS: check-acceptance - <<: *acceptance_definition build-disabled: extends: .native_build_job
Signature · 1 line
2.26.2
RFC PATCH v2 6/11 gitlab-ci: Rename acceptance_test_job -> integration_test_job
In reply to Philippe Mathieu-Daudé--- .gitlab-ci.yml | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-)
.gitlab-ci.yml +9 -9
@@ -42,7 +42,7 @@ include: - find . -type f -exec touch {} + - make $MAKE_CHECK_ARGS -.acceptance_test_job: +.integration_test_job: extends: .native_test_job cache: key: "${CI_JOB_NAME}-cache"
@@ -89,8 +89,8 @@ check-system-ubuntu: IMAGE: ubuntu2004 MAKE_CHECK_ARGS: check -acceptance-system-ubuntu: - extends: .acceptance_test_job +integration-system-ubuntu: + extends: .integration_test_job needs: - job: build-system-ubuntu artifacts: true
@@ -119,8 +119,8 @@ check-system-debian: IMAGE: debian-amd64 MAKE_CHECK_ARGS: check -acceptance-system-debian: - extends: .acceptance_test_job +integration-system-debian: + extends: .integration_test_job needs: - job: build-system-debian artifacts: true
@@ -150,8 +150,8 @@ check-system-fedora: IMAGE: fedora MAKE_CHECK_ARGS: check -acceptance-system-fedora: - extends: .acceptance_test_job +integration-system-fedora: + extends: .integration_test_job needs: - job: build-system-fedora artifacts: true
@@ -181,8 +181,8 @@ check-system-centos: IMAGE: centos8 MAKE_CHECK_ARGS: check -acceptance-system-centos: - extends: .acceptance_test_job +integration-system-centos: + extends: .integration_test_job needs: - job: build-system-centos artifacts: true
Signature · 1 line
2.26.2
RFC PATCH v2 7/11 gitlab-ci: Extract common job definition as 'cross_common_job'
In reply to Philippe Mathieu-DaudéExtract the common definitions shared by '.cross_system_build_job' and '.cross_user_build_job' to '.cross_common_job'.
--- .gitlab-ci.d/crossbuilds.yml | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)
.gitlab-ci.d/crossbuilds.yml +5 -4
@@ -1,7 +1,9 @@ - -.cross_system_build_job: +.cross_common_job: stage: build image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest + +.cross_system_build_job: + extends: .cross_common_job timeout: 80m script: - mkdir build
@@ -14,8 +16,7 @@ - make -j$(expr $(nproc) + 1) all check-build .cross_user_build_job: - stage: build - image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest + extends: .cross_common_job script: - mkdir build - cd build
Signature · 1 line
2.26.2
RFC PATCH v2 8/11 gitlab-ci: Extract common job definition as 'native_common_job'
In reply to Philippe Mathieu-DaudéExtract the common definitions shared by '.native_build_job' and '.native_test_job' to '.native_common_job'.
--- .gitlab-ci.yml | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
.gitlab-ci.yml +6 -3
@@ -13,9 +13,12 @@ include: - local: '/.gitlab-ci.d/containers.yml' - local: '/.gitlab-ci.d/crossbuilds.yml' -.native_build_job: - stage: build +.native_common_job: image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest + +.native_build_job: + extends: .native_common_job + stage: build before_script: - JOBS=$(expr $(nproc) + 1) - sed -i s,git.qemu.org/git,gitlab.com/qemu-project, .gitmodules
@@ -35,8 +38,8 @@ include: fi .native_test_job: + extends: .native_common_job stage: test - image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest script: - cd build - find . -type f -exec touch {} +
Signature · 1 line
2.26.2
RFC PATCH v2 9/11 gitlab-ci: Add rules to select cross-jobs to build
In reply to Philippe Mathieu-DaudéAdd rules to select some crossbuild jobs.
The following tags are available to restrict the CI jobs:
- all (select all jobs, this is default)
- cross (select all cross-jobs)
- system (select all cross-system jobs)
- user (select all cross-user jobs)
- $ARCH (select an architecture: arm/mips/ppc/sparc/...)
Developers can combine tags in the QEMU_BUILD variable when pushing a branch (or tag) to repositories. Examples:
$ git push -o ci.variable="QEMU_BUILD=user" myrepo mybranch $ git push -o ci.variable="QEMU_BUILD=user,system" myrepo mybranch
References:
- https://docs.gitlab.com/ee/ci/yaml/#rulesif
- https://docs.gitlab.com/ee/user/project/push_options.html#push-options-for-gitlab-cicd
--- .gitlab-ci.d/crossbuilds.yml | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+)
.gitlab-ci.d/crossbuilds.yml +30 -0
@@ -1,6 +1,36 @@ .cross_common_job: stage: build image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest + rules: + # If the if statement is true, the job is added to the pipeline. + # We only filter for push events + - if: '$CI_PIPELINE_SOURCE != "push"' + # Build all when no variable defined, or set to "all" + - if: $QEMU_BUILD == null || $QEMU_BUILD =~ /^all$/ + # Build specific job name + - if: $QEMU_BUILD =~ /^$CI_JOB_NAME$/ + # Build set of jobs by feature + - if: $QEMU_BUILD =~ /cross/ + - if: $QEMU_BUILD =~ /system/ && $CI_JOB_NAME =~ /system/ + - if: $QEMU_BUILD =~ /user/ && $CI_JOB_NAME =~ /user/ + # Build set of jobs by arch + - if: $QEMU_BUILD =~ /aarch64/ && ($CI_JOB_NAME =~ /aarch64/ || $IMAGE =~ /aarch64/) + - if: $QEMU_BUILD =~ /alpha/ && ($CI_JOB_NAME =~ /alpha/ || $IMAGE =~ /alpha/) + - if: $QEMU_BUILD =~ /arm/ && ($CI_JOB_NAME =~ /arm/ || $IMAGE =~ /arm/) + - if: $QEMU_BUILD =~ /avr/ && ($CI_JOB_NAME =~ /avr/ || $IMAGE =~ /avr/) + - if: $QEMU_BUILD =~ /hppa/ && ($CI_JOB_NAME =~ /hppa/ || $IMAGE =~ /hppa/) + - if: $QEMU_BUILD =~ /i386/ && ($CI_JOB_NAME =~ /i386/ || $IMAGE =~ /i386/) + - if: $QEMU_BUILD =~ /lm32/ && ($CI_JOB_NAME =~ /lm32/ || $IMAGE =~ /lm32/) + - if: $QEMU_BUILD =~ /m68k/ && ($CI_JOB_NAME =~ /m68k/ || $IMAGE =~ /m68k/) + - if: $QEMU_BUILD =~ /ppc/ && ($CI_JOB_NAME =~ /ppc/ || $IMAGE =~ /ppc/) + - if: $QEMU_BUILD =~ /riscv/ && ($CI_JOB_NAME =~ /riscv/ || $IMAGE =~ /riscv/) + - if: $QEMU_BUILD =~ /s390x/ && ($CI_JOB_NAME =~ /s390x/ || $IMAGE =~ /s390x/) + - if: $QEMU_BUILD =~ /sparc/ && ($CI_JOB_NAME =~ /sparc/ || $IMAGE =~ /sparc/) + - if: $QEMU_BUILD =~ /tricore/ && ($CI_JOB_NAME =~ /tricore/ || $IMAGE =~ /tricore/) + - if: $QEMU_BUILD =~ /x86/ && ($CI_JOB_NAME =~ /x86/ || $IMAGE =~ /x86/) + - if: $QEMU_BUILD =~ /xtensa/ && ($CI_JOB_NAME =~ /xtensa/ || $IMAGE =~ /xtensa/) + # In all other cases, the job is excluded from a pipeline. + - when: never .cross_system_build_job: extends: .cross_common_job
Signature · 1 line
2.26.2
RFC PATCH v2 10/11 gitlab-ci: Add rules to select building/testing native jobs
In reply to Philippe Mathieu-DaudéAdd rules to select various build/test jobs.
The following tags are available to restrict CI jobs:
- user (user-mode jobs) - system (system-mode jobs) - centos (jobs based on CentOS distribution image) - debian (... Debian) - fedora (... Fedora) - ubuntu (... Ubuntu) - crypto (jobs testing the crypto feature) - tci (jobs testing TCI feature) - fuzz (fuzzer job)
- integration (integration tests)
- $ARCH (select an architecture: arm/mips/ppc/sparc/...)
Developers can combine tags in the QEMU_BUILD variable when pushing a branch (or tag) to repositories. Examples:
$ git push -o ci.variable="QEMU_BUILD=user" myrepo mybranch $ git push -o ci.variable="QEMU_BUILD=user,debian,crypto,fuzz" myrepo mybranch
References:
- https://docs.gitlab.com/ee/ci/yaml/#rulesif
- https://docs.gitlab.com/ee/user/project/push_options.html#push-options-for-gitlab-cicd
--- .gitlab-ci.yml | 38 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 38 insertions(+)
.gitlab-ci.yml +38 -0
@@ -15,6 +15,44 @@ include: .native_common_job: image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest + rules: + # If the if statement is true, the job is added to the pipeline. + # We only filter for push events + - if: '$CI_PIPELINE_SOURCE != "push"' + # Build all when no variable defined, or set to "all" + - if: $QEMU_BUILD == null || $QEMU_BUILD =~ /^all$/ + # Build specific job name + - if: $QEMU_BUILD == $CI_JOB_NAME + # Build jobs using particular distribution image + - if: $QEMU_BUILD =~ /centos/ && $IMAGE =~ /^centos/ + - if: $QEMU_BUILD =~ /debian/ && $IMAGE =~ /^debian/ + - if: $QEMU_BUILD =~ /fedora/ && $IMAGE =~ /^fedora/ + - if: $QEMU_BUILD =~ /ubuntu/ && $IMAGE =~ /^ubuntu/ + # Build set of jobs by feature + - if: $QEMU_BUILD =~ /system/ && ($TARGETS =~ /softmmu/ || $CONFIGURE_ARGS =~ /disable-user/) + - if: $QEMU_BUILD =~ /user/ && ($TARGETS =~ /user/ || $CONFIGURE_ARGS =~ /disable-system/) + - if: $QEMU_BUILD =~ /integration/ && ($CI_JOB_NAME =~ /^integration/ || $CI_JOB_NAME =~ /^build-system/ || $MAKE_CHECK_ARGS =~ /check-acceptance/) # integration depends on build-system + - if: $QEMU_BUILD =~ /crypto/ && $CI_JOB_NAME =~ /crypto/ + - if: $QEMU_BUILD =~ /tci/ && $CI_JOB_NAME =~ /tci$/ + - if: $QEMU_BUILD =~ /fuzz/ && $CI_JOB_NAME =~ /fuzz$/ + # Build set of jobs by arch + - if: $QEMU_BUILD =~ /aarch64/ && ($CI_JOB_NAME =~ /aarch64/ || $TARGETS =~ /aarch64/) + - if: $QEMU_BUILD =~ /alpha/ && ($CI_JOB_NAME =~ /alpha/ || $TARGETS =~ /alpha/) + - if: $QEMU_BUILD =~ /arm/ && ($CI_JOB_NAME =~ /arm/ || $TARGETS =~ /arm/) + - if: $QEMU_BUILD =~ /avr/ && ($CI_JOB_NAME =~ /avr/ || $TARGETS =~ /avr/) + - if: $QEMU_BUILD =~ /hppa/ && ($CI_JOB_NAME =~ /hppa/ || $TARGETS =~ /hppa/) + - if: $QEMU_BUILD =~ /i386/ && ($CI_JOB_NAME =~ /i386/ || $TARGETS =~ /i386/) + - if: $QEMU_BUILD =~ /lm32/ && ($CI_JOB_NAME =~ /lm32/ || $TARGETS =~ /lm32/) + - if: $QEMU_BUILD =~ /m68k/ && ($CI_JOB_NAME =~ /m68k/ || $TARGETS =~ /m68k/) + - if: $QEMU_BUILD =~ /ppc/ && ($CI_JOB_NAME =~ /ppc/ || $TARGETS =~ /ppc/) + - if: $QEMU_BUILD =~ /riscv/ && ($CI_JOB_NAME =~ /riscv/ || $TARGETS =~ /riscv/) + - if: $QEMU_BUILD =~ /s390x/ && ($CI_JOB_NAME =~ /s390x/ || $TARGETS =~ /s390x/) + - if: $QEMU_BUILD =~ /sparc/ && ($CI_JOB_NAME =~ /sparc/ || $TARGETS =~ /sparc/) + - if: $QEMU_BUILD =~ /tricore/ && ($CI_JOB_NAME =~ /tricore/ || $TARGETS =~ /tricore/) + - if: $QEMU_BUILD =~ /x86/ && ($CI_JOB_NAME =~ /x86/ || $TARGETS =~ /x86/) + - if: $QEMU_BUILD =~ /xtensa/ && ($CI_JOB_NAME =~ /xtensa/ || $TARGETS =~ /xtensa/) + # In all other cases, the job is excluded from a pipeline. + - when: never .native_build_job: extends: .native_common_job
Signature · 1 line
2.26.2
RFC PATCH v2 11/11 gitlab-ci: Move artifacts expiry rule to common 'native_build_job'
In reply to Philippe Mathieu-DaudéBuild jobs include the 'native_build_job' template. Move the 'artifacts expiry' rule there. Now all build jobs benefit from it.
--- .gitlab-ci.yml | 24 ++++-------------------- 1 file changed, 4 insertions(+), 20 deletions(-)
.gitlab-ci.yml +4 -20
@@ -74,6 +74,10 @@ include: then make -j"$JOBS" $MAKE_CHECK_ARGS ; fi + artifacts: + expire_in: 2 days + paths: + - build .native_test_job: extends: .native_common_job
@@ -116,10 +120,6 @@ build-system-ubuntu: TARGETS: aarch64-softmmu alpha-softmmu cris-softmmu hppa-softmmu moxie-softmmu microblazeel-softmmu mips64el-softmmu MAKE_CHECK_ARGS: check-build - artifacts: - expire_in: 2 days - paths: - - build check-system-ubuntu: extends: .native_test_job
@@ -146,10 +146,6 @@ build-system-debian: TARGETS: arm-softmmu avr-softmmu i386-softmmu mipsel-softmmu riscv64-softmmu sh4eb-softmmu sparc-softmmu xtensaeb-softmmu MAKE_CHECK_ARGS: check-build - artifacts: - expire_in: 2 days - paths: - - build check-system-debian: extends: .native_test_job
@@ -177,10 +173,6 @@ build-system-fedora: TARGETS: tricore-softmmu microblaze-softmmu mips-softmmu xtensa-softmmu m68k-softmmu riscv32-softmmu ppc-softmmu sparc64-softmmu MAKE_CHECK_ARGS: check-build - artifacts: - expire_in: 2 days - paths: - - build check-system-fedora: extends: .native_test_job
@@ -208,10 +200,6 @@ build-system-centos: TARGETS: ppc64-softmmu or1k-softmmu s390x-softmmu x86_64-softmmu rx-softmmu sh4-softmmu nios2-softmmu MAKE_CHECK_ARGS: check-build - artifacts: - expire_in: 2 days - paths: - - build check-system-centos: extends: .native_test_job
@@ -312,10 +300,6 @@ build-deprecated: MAKE_CHECK_ARGS: build-tcg TARGETS: ppc64abi32-linux-user tilegx-linux-user lm32-softmmu unicore32-softmmu - artifacts: - expire_in: 2 days - paths: - - build # We split the check-tcg step as test failures are expected but we still # want to catch the build breaking.
Signature · 1 line
2.26.2