gitlab-ci: Document custom CI/CD variables, fix 'when:' conditions
v3 cover gitlab-ci: Document custom CI/CD variables, fix 'when:' conditions
- Document custom CI/CD variables (thuth)
- Fix 'when:' condition (danpb)
Supersedes: <[email protected]>
Philippe Mathieu-Daudé (4):
docs: Document GitLab custom CI/CD variables
gitlab-ci: Fix 'when:' condition in acceptance_test_job_template
gitlab-ci: Fix 'when:' condition in EDK2 jobs
gitlab-ci: Extract OpenSBI job rules and fix 'when:' condition
docs/devel/ci.rst | 40 +++++++++++++++++++++++++++++ .gitlab-ci.d/buildtest-template.yml | 4 +-- .gitlab-ci.d/edk2.yml | 6 ++--- .gitlab-ci.d/opensbi.yml | 30 +++++++++++++--------- .gitlab-ci.yml | 19 ++------------ 5 files changed, 65 insertions(+), 34 deletions(-)
Signature · 1 line
2.31.1
PATCH v3 1/4 docs: Document GitLab custom CI/CD variables
We introduced the QEMU_CI_AVOCADO_TESTING variable in commit f56bf4caf ("gitlab: Run Avocado tests manually (except mainstream CI)"), but forgot to document it properly. Do it now.
--- docs/devel/ci.rst | 40 ++++++++++++++++++++++++++++++++++++++++ .gitlab-ci.yml | 19 ++----------------- 2 files changed, 42 insertions(+), 17 deletions(-)
docs/devel/ci.rst +40 -0
@@ -8,6 +8,46 @@ found at:: https://wiki.qemu.org/Testing/CI +Custom CI/CD variables +====================== + +QEMU CI pipelines can be tuned setting some CI environment variables. + +Set variable globally in the user's CI namespace +------------------------------------------------ + +Variables can be set globally in the user's CI namespace setting. + +For further information about how to set these variables, please refer to:: + + https://docs.gitlab.com/ee/ci/variables/#create-a-custom-variable-in-the-ui + +Set variable manually when pushing a branch or tag to the user's repository +--------------------------------------------------------------------------- + +Variables can be set manually when pushing a branch or tag, using +git-push command line arguments. + +Example setting the QEMU_CI_EXAMPLE_VAR variable: + +.. code:: + + git push -o ci.variable="QEMU_CI_EXAMPLE_VAR=value" myrepo mybranch + +For further information about how to set these variables, please refer to:: + + https://docs.gitlab.com/ee/user/project/push_options.html#push-options-for-gitlab-cicd + +Here is a list of the most used variables: + +QEMU_CI_AVOCADO_TESTING +~~~~~~~~~~~~~~~~~~~~~~~ +By default, tests using the Avocado framework are not run automatically in +the pipelines (because multiple artifacts have to be downloaded, and if +these artifacts are not already cached, downloading them make the jobs +reach the timeout limit). Set this variable to have the tests using the +Avocado framework run automatically. + Jobs on Custom Runners ======================
.gitlab-ci.yml +2 -17
@@ -16,24 +16,9 @@ # QEMU CI jobs are based on templates. Some templates provide # user-configurable options, modifiable via configuration variables. # -# These variables can be set globally in the user's CI namespace -# setting: -# https://docs.gitlab.com/ee/ci/variables/#create-a-custom-variable-in-the-ui -# or set manually each time a branch/tag is pushed, as a git-push -# command line argument: -# https://docs.gitlab.com/ee/user/project/push_options.html#push-options-for-gitlab-cicd +# See https://qemu-project.gitlab.io/qemu/devel/ci.html#custom-ci-cd-variables +# for more information. # -# Example setting the QEMU_CI_EXAMPLE_VAR variable: -# -# git push -o ci.variable="QEMU_CI_EXAMPLE_VAR=value" myrepo mybranch -# -# ---------------------------------------------------------------------- -# -# List of environment variables that can be use to modify the set -# of jobs selected: -# -# - QEMU_CI_AVOCADO_TESTING -# If set, tests using the Avocado framework will be run include: - local: '/.gitlab-ci.d/qemu-project.yml'
Signature · 1 line
2.31.1
Re: [PATCH-for-6.1 v3 1/4] docs: Document GitLab custom CI/CD variables
Quoted 10 lines · Philippe Mathieu-Daudé
We introduced the
QEMU_CI_AVOCADO_TESTINGvariable in commitf56bf4caf("gitlab: Run Avocado tests manually (except mainstream CI)"), but forgot to document it properly. Do it now.Suggested-by Thomas Huth <[email protected]>Signed-off-by Philippe Mathieu-Daudé <[email protected]>--- docs/devel/ci.rst | 40 ++++++++++++++++++++++++++++++++++++++++ .gitlab-ci.yml | 19 ++----------------- 2 files changed, 42 insertions(+), 17 deletions(-)
Regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
PATCH v3 2/4 gitlab-ci: Fix 'when:' condition in acceptance_test_job_template
In reply to Philippe Mathieu-DaudéJobs depending on another should not use the 'when: always' condition, because if a dependency failed we should not keep running jobs depending on it. The correct condition is 'when: on_success'.
--- .gitlab-ci.d/buildtest-template.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
.gitlab-ci.d/buildtest-template.yml +2 -2
@@ -73,9 +73,9 @@ # in its namespace setting or via git-push option, see documentation # in /.gitlab-ci.yml of this repository). - if: '$CI_PROJECT_NAMESPACE == "qemu-project"' - when: always + when: on_success - if: '$QEMU_CI_AVOCADO_TESTING' - when: always + when: on_success # Otherwise, set to manual (the jobs are created but not run). - when: manual allow_failure: true
Signature · 1 line
2.31.1
Re: [PATCH-for-6.1 v3 2/4] gitlab-ci: Fix 'when:' condition in acceptance_test_job_template
Quoted 11 lines · Philippe Mathieu-Daudé
Jobs depending on another should not use the 'when: always' condition, because if a dependency failed we should not keep running jobs depending on it. The correct condition is 'when:
on_success'.Fixes f56bf4caf71 ("gitlab: Run Avocado tests manually (except mainstream CI)")Reported-by Daniel P. Berrangé <[email protected]>Signed-off-by Philippe Mathieu-Daudé <[email protected]>--- .gitlab-ci.d/buildtest-template.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
Regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
PATCH v3 3/4 gitlab-ci: Fix 'when:' condition in EDK2 jobs
In reply to Philippe Mathieu-DaudéJobs depending on another should not use the 'when: always' condition, because if a dependency failed we should not keep running jobs depending on it. The correct condition is 'when: on_success'.
--- .gitlab-ci.d/edk2.yml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
.gitlab-ci.d/edk2.yml +3 -3
@@ -8,11 +8,11 @@ - .gitlab-ci.d/edk2/Dockerfile # or roms/edk2/ is modified (submodule updated) - roms/edk2/* - when: always + when: on_success - if: '$CI_COMMIT_REF_NAME =~ /^edk2/' # or the branch/tag starts with 'edk2' - when: always + when: on_success - if: '$CI_COMMIT_MESSAGE =~ /edk2/i' # or last commit description contains 'EDK2' - when: always + when: on_success docker-edk2: extends: .edk2_job_rules
Signature · 1 line
2.31.1
Re: [PATCH-for-6.1 v3 3/4] gitlab-ci: Fix 'when:' condition in EDK2 jobs
Quoted 11 lines · Philippe Mathieu-Daudé
Jobs depending on another should not use the 'when: always' condition, because if a dependency failed we should not keep running jobs depending on it. The correct condition is 'when:
on_success'.Fixes 71920809cea ("gitlab-ci.yml: Add jobs to build EDK2 firmware binaries")Reported-by Daniel P. Berrangé <[email protected]>Signed-off-by Philippe Mathieu-Daudé <[email protected]>--- .gitlab-ci.d/edk2.yml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
Regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
PATCH v3 4/4 gitlab-ci: Extract OpenSBI job rules and fix 'when:' condition
In reply to Philippe Mathieu-DaudéFirst, all jobs depending on 'docker-opensbi' job must use at most all the rules that triggers it. The simplest way to ensure that is to always use the same rules. Extract all the rules to a reusable section, and include this section (with the 'extends' keyword) in both 'docker-opensbi' and 'build-opensbi' jobs.
Second, jobs depending on another should not use the 'when: always' condition, because if a dependency failed we should not keep running jobs depending on it. The correct condition is 'when: on_success'.
The problems were introduced in commit c6fc0fc1a71 ("gitlab-ci.yml:
Add jobs to build OpenSBI firmware binaries"), but were revealed in commit 91e9c47e50a ("docker: OpenSBI build job depends on OpenSBI container").
This fix is similar to the one used with the EDK2 firmware job in commit ac0595cf6b3 ("gitlab-ci: Extract EDK2 job rules to reusable section").
--- .gitlab-ci.d/opensbi.yml | 30 ++++++++++++++++++------------ 1 file changed, 18 insertions(+), 12 deletions(-)
.gitlab-ci.d/opensbi.yml +18 -12
@@ -1,10 +1,23 @@ -docker-opensbi: - stage: containers - rules: # Only run this job when the Dockerfile is modified +# All jobs needing docker-opensbi must use the same rules it uses. +.opensbi_job_rules: + rules: # Only run this job when ... - changes: + # this file is modified - .gitlab-ci.d/opensbi.yml + # or the Dockerfile is modified - .gitlab-ci.d/opensbi/Dockerfile - when: always + when: on_success + - changes: # or roms/opensbi/ is modified (submodule updated) + - roms/opensbi/* + when: on_success + - if: '$CI_COMMIT_REF_NAME =~ /^opensbi/' # or the branch/tag starts with 'opensbi' + when: on_success + - if: '$CI_COMMIT_MESSAGE =~ /opensbi/i' # or last commit description contains 'OpenSBI' + when: on_success + +docker-opensbi: + extends: .opensbi_job_rules + stage: containers image: docker:19.03.1 services: - docker:19.03.1-dind
@@ -24,16 +37,9 @@ docker-opensbi: - docker push $IMAGE_TAG build-opensbi: + extends: .opensbi_job_rules stage: build needs: ['docker-opensbi'] - rules: # Only run this job when ... - - changes: # ... roms/opensbi/ is modified (submodule updated) - - roms/opensbi/* - when: always - - if: '$CI_COMMIT_REF_NAME =~ /^opensbi/' # or the branch/tag starts with 'opensbi' - when: always - - if: '$CI_COMMIT_MESSAGE =~ /opensbi/i' # or last commit description contains 'OpenSBI' - when: always artifacts: paths: # 'artifacts.zip' will contains the following files: - pc-bios/opensbi-riscv32-generic-fw_dynamic.bin
Signature · 1 line
2.31.1