gitlab: stopp edk2/opensbi jobs running in forks by default
cover gitlab: stopp edk2/opensbi jobs running in forks by default
When we introducd the QEMU_CI variable to control running of pipelines in gitlab forks, we didn't include the ekd2/opensbi jobs in the rules. This caused pipelines to be unexpectedly created in some cases. This addresses that oversight.
Daniel P. Berrangé (3):
gitlab: normalize indentation in edk2/opensbi rules
gitlab: tweak comments in edk2/opensbi jobs
gitlab: honour QEMU_CI variable in edk2/opensbi jobs
.gitlab-ci.d/edk2.yml | 133 ++++++++++++++++++++++---------------- .gitlab-ci.d/opensbi.yml | 134 +++++++++++++++++++++++---------------- 2 files changed, 158 insertions(+), 109 deletions(-)
Signature · 1 line
2.36.1
PATCH 1/3 gitlab: normalize indentation in edk2/opensbi rules
The edk2/opensbi gitlab CI config was using single space indents which is not consistent with the rest of the gitlab CI config files.
--- .gitlab-ci.d/edk2.yml | 108 +++++++++++++++++++------------------- .gitlab-ci.d/opensbi.yml | 110 +++++++++++++++++++-------------------- 2 files changed, 109 insertions(+), 109 deletions(-)
.gitlab-ci.d/edk2.yml +54 -54
@@ -1,60 +1,60 @@ # All jobs needing docker-edk2 must use the same rules it uses. .edk2_job_rules: - rules: # Only run this job when ... - - changes: - # this file is modified - - .gitlab-ci.d/edk2.yml - # or the Dockerfile is modified - - .gitlab-ci.d/edk2/Dockerfile - # or roms/edk2/ is modified (submodule updated) - - roms/edk2/* - when: on_success - - if: '$CI_COMMIT_REF_NAME =~ /^edk2/' # or the branch/tag starts with 'edk2' - when: on_success - - if: '$CI_COMMIT_MESSAGE =~ /edk2/i' # or last commit description contains 'EDK2' - when: on_success + rules: # Only run this job when ... + - changes: + # this file is modified + - .gitlab-ci.d/edk2.yml + # or the Dockerfile is modified + - .gitlab-ci.d/edk2/Dockerfile + # or roms/edk2/ is modified (submodule updated) + - roms/edk2/* + when: on_success + - if: '$CI_COMMIT_REF_NAME =~ /^edk2/' # or the branch/tag starts with 'edk2' + when: on_success + - if: '$CI_COMMIT_MESSAGE =~ /edk2/i' # or last commit description contains 'EDK2' + when: on_success docker-edk2: - extends: .edk2_job_rules - stage: containers - image: docker:19.03.1 - services: - - docker:19.03.1-dind - variables: - GIT_DEPTH: 3 - IMAGE_TAG: $CI_REGISTRY_IMAGE:edk2-cross-build - # We don't use TLS - DOCKER_HOST: tcp://docker:2375 - DOCKER_TLS_CERTDIR: "" - before_script: - - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY - script: - - docker pull $IMAGE_TAG || true - - docker build --cache-from $IMAGE_TAG --tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA - --tag $IMAGE_TAG .gitlab-ci.d/edk2 - - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA - - docker push $IMAGE_TAG + extends: .edk2_job_rules + stage: containers + image: docker:19.03.1 + services: + - docker:19.03.1-dind + variables: + GIT_DEPTH: 3 + IMAGE_TAG: $CI_REGISTRY_IMAGE:edk2-cross-build + # We don't use TLS + DOCKER_HOST: tcp://docker:2375 + DOCKER_TLS_CERTDIR: "" + before_script: + - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY + script: + - docker pull $IMAGE_TAG || true + - docker build --cache-from $IMAGE_TAG --tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA + --tag $IMAGE_TAG .gitlab-ci.d/edk2 + - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA + - docker push $IMAGE_TAG build-edk2: - extends: .edk2_job_rules - stage: build - needs: ['docker-edk2'] - artifacts: - paths: # 'artifacts.zip' will contains the following files: - - pc-bios/edk2*bz2 - - pc-bios/edk2-licenses.txt - - edk2-stdout.log - - edk2-stderr.log - image: $CI_REGISTRY_IMAGE:edk2-cross-build - variables: - GIT_DEPTH: 3 - script: # Clone the required submodules and build EDK2 - - git submodule update --init roms/edk2 - - git -C roms/edk2 submodule update --init -- - ArmPkg/Library/ArmSoftFloatLib/berkeley-softfloat-3 - BaseTools/Source/C/BrotliCompress/brotli - CryptoPkg/Library/OpensslLib/openssl - MdeModulePkg/Library/BrotliCustomDecompressLib/brotli - - export JOBS=$(($(getconf _NPROCESSORS_ONLN) + 1)) - - echo "=== Using ${JOBS} simultaneous jobs ===" - - make -j${JOBS} -C roms efi 2>&1 1>edk2-stdout.log | tee -a edk2-stderr.log >&2 + extends: .edk2_job_rules + stage: build + needs: ['docker-edk2'] + artifacts: + paths: # 'artifacts.zip' will contains the following files: + - pc-bios/edk2*bz2 + - pc-bios/edk2-licenses.txt + - edk2-stdout.log + - edk2-stderr.log + image: $CI_REGISTRY_IMAGE:edk2-cross-build + variables: + GIT_DEPTH: 3 + script: # Clone the required submodules and build EDK2 + - git submodule update --init roms/edk2 + - git -C roms/edk2 submodule update --init -- + ArmPkg/Library/ArmSoftFloatLib/berkeley-softfloat-3 + BaseTools/Source/C/BrotliCompress/brotli + CryptoPkg/Library/OpensslLib/openssl + MdeModulePkg/Library/BrotliCustomDecompressLib/brotli + - export JOBS=$(($(getconf _NPROCESSORS_ONLN) + 1)) + - echo "=== Using ${JOBS} simultaneous jobs ===" + - make -j${JOBS} -C roms efi 2>&1 1>edk2-stdout.log | tee -a edk2-stderr.log >&2
.gitlab-ci.d/opensbi.yml +55 -55
116 lines not on this page · raw message
Signature · 1 line
2.36.1
Re: [PATCH 1/3] gitlab: normalize indentation in edk2/opensbi rules
Quoted 9 lines · Daniel P. Berrangé
The edk2/opensbi gitlab CI config was using single space indents which is not consistent with the rest of the gitlab CI config files.
Signed-off-by Daniel P. Berrangé<[email protected]>--- .gitlab-ci.d/edk2.yml | 108 +++++++++++++++++++------------------- .gitlab-ci.d/opensbi.yml | 110 +++++++++++++++++++-------------------- 2 files changed, 109 insertions(+), 109 deletions(-)
r~
PATCH 2/3 gitlab: tweak comments in edk2/opensbi jobs
In reply to Daniel P. BerrangéGet rid of comments stating the obvious and re-arrange remaining comments. The opensbi split of rules for file matches is also merged into one rule.
--- .gitlab-ci.d/edk2.yml | 14 ++++++++------ .gitlab-ci.d/opensbi.yml | 15 ++++++++------- 2 files changed, 16 insertions(+), 13 deletions(-)
.gitlab-ci.d/edk2.yml +8 -6
@@ -1,17 +1,19 @@ # All jobs needing docker-edk2 must use the same rules it uses. .edk2_job_rules: - rules: # Only run this job when ... + rules: + # Run if any files affecting the build output are touched - changes: - # this file is modified - .gitlab-ci.d/edk2.yml - # or the Dockerfile is modified - .gitlab-ci.d/edk2/Dockerfile - # or roms/edk2/ is modified (submodule updated) - roms/edk2/* when: on_success - - if: '$CI_COMMIT_REF_NAME =~ /^edk2/' # or the branch/tag starts with 'edk2' + + # Run if the branch/tag starts with 'edk2' + - if: '$CI_COMMIT_REF_NAME =~ /^edk2/' when: on_success - - if: '$CI_COMMIT_MESSAGE =~ /edk2/i' # or last commit description contains 'EDK2' + + # Run if last commit msg contains 'EDK2' (case insensitive) + - if: '$CI_COMMIT_MESSAGE =~ /edk2/i' when: on_success docker-edk2:
.gitlab-ci.d/opensbi.yml +8 -7
@@ -1,18 +1,19 @@ # All jobs needing docker-opensbi must use the same rules it uses. .opensbi_job_rules: - rules: # Only run this job when ... + rules: + # Run if any files affecting the build output are touched - changes: - # this file is modified - .gitlab-ci.d/opensbi.yml - # or the Dockerfile is modified - .gitlab-ci.d/opensbi/Dockerfile - when: on_success - - changes: # or roms/opensbi/ is modified (submodule updated) - roms/opensbi/* when: on_success - - if: '$CI_COMMIT_REF_NAME =~ /^opensbi/' # or the branch/tag starts with 'opensbi' + + # Run if the branch/tag starts with 'opensbi' + - if: '$CI_COMMIT_REF_NAME =~ /^opensbi/' when: on_success - - if: '$CI_COMMIT_MESSAGE =~ /opensbi/i' # or last commit description contains 'OpenSBI' + + # Run if the last commit msg contains 'OpenSBI' (case insensitive) + - if: '$CI_COMMIT_MESSAGE =~ /opensbi/i' when: on_success docker-opensbi:
Signature · 1 line
2.36.1
Re: [PATCH 2/3] gitlab: tweak comments in edk2/opensbi jobs
Quoted 9 lines · Daniel P. Berrangé
Get rid of comments stating the obvious and re-arrange remaining comments. The opensbi split of rules for file matches is also merged into one rule.
Signed-off-by Daniel P. Berrangé<[email protected]>--- .gitlab-ci.d/edk2.yml | 14 ++++++++------ .gitlab-ci.d/opensbi.yml | 15 ++++++++------- 2 files changed, 16 insertions(+), 13 deletions(-)
r~
PATCH 3/3 gitlab: honour QEMU_CI variable in edk2/opensbi jobs
In reply to Daniel P. BerrangéTo preserve contributor CI credits we don't want jobs to run by default unless the QEMU_CI variable is set. For most jobs we can achieve this using the base template, but the edk2/opensbi jobs are a little special as they have some complex conditions we can't easily model in the base template.
We duplicate existing rules and put them under control of QEMU_CI variable, such that QEMU_CI=1 creates manual jobs and QEMU_CI=2 immediately runs jobs.
--- .gitlab-ci.d/edk2.yml | 23 +++++++++++++++++++++++ .gitlab-ci.d/opensbi.yml | 23 +++++++++++++++++++++++ 2 files changed, 46 insertions(+)
.gitlab-ci.d/edk2.yml +23 -0
@@ -1,6 +1,29 @@ # All jobs needing docker-edk2 must use the same rules it uses. .edk2_job_rules: rules: + # Forks don't get pipelines unless QEMU_CI=1 or QEMU_CI=2 is set + - if: '$QEMU_CI != "1" && $QEMU_CI != "2" && $CI_PROJECT_NAMESPACE != "qemu-project"' + when: never + + # In forks, if QEMU_CI=1 is set, then create manual job + # if any of the files affecting the build are touched + - if: '$QEMU_CI == "1" && $CI_PROJECT_NAMESPACE != "qemu-project"' + changes: + - .gitlab-ci.d/edk2.yml + - .gitlab-ci.d/edk2/Dockerfile + - roms/edk2/* + when: manual + + # In forks, if QEMU_CI=1 is set, then create manual job + # if the branch/tag starts with 'edk2' + - if: '$QEMU_CI == "1" && $CI_PROJECT_NAMESPACE != "qemu-project" && $CI_COMMIT_REF_NAME =~ /^edk2/' + when: manual + + # In forks, if QEMU_CI=1 is set, then create manual job + # if last commit msg contains 'EDK2' (case insensitive) + - if: '$QEMU_CI == "1" && $CI_PROJECT_NAMESPACE != "qemu-project" && $CI_COMMIT_MESSAGE =~ /edk2/i' + when: on_success + # Run if any files affecting the build output are touched - changes: - .gitlab-ci.d/edk2.yml
.gitlab-ci.d/opensbi.yml +23 -0
@@ -1,6 +1,29 @@ # All jobs needing docker-opensbi must use the same rules it uses. .opensbi_job_rules: rules: + # Forks don't get pipelines unless QEMU_CI=1 or QEMU_CI=2 is set + - if: '$QEMU_CI != "1" && $QEMU_CI != "2" && $CI_PROJECT_NAMESPACE != "qemu-project"' + when: never + + # In forks, if QEMU_CI=1 is set, then create manual job + # if any files affecting the build output are touched + - if: '$QEMU_CI == "1" && $CI_PROJECT_NAMESPACE != "qemu-project"' + changes: + - .gitlab-ci.d/opensbi.yml + - .gitlab-ci.d/opensbi/Dockerfile + - roms/opensbi/* + when: manual + + # In forks, if QEMU_CI=1 is set, then create manual job + # if the branch/tag starts with 'opensbi' + - if: '$QEMU_CI == "1" && $CI_PROJECT_NAMESPACE != "qemu-project" && $CI_COMMIT_REF_NAME =~ /^opensbi/' + when: manual + + # In forks, if QEMU_CI=1 is set, then create manual job + # if the last commit msg contains 'OpenSBI' (case insensitive) + - if: '$QEMU_CI == "1" && $CI_PROJECT_NAMESPACE != "qemu-project" && $CI_COMMIT_MESSAGE =~ /opensbi/i' + when: manual + # Run if any files affecting the build output are touched - changes: - .gitlab-ci.d/opensbi.yml
Signature · 1 line
2.36.1
Re: [PATCH 3/3] gitlab: honour QEMU_CI variable in edk2/opensbi jobs
.gitlab-ci.d/edk2.yml line 25
Quoted 9 lines · Daniel P. Berrangé
.gitlab-ci.d/edk2.yml +9
-0+ # In forks, if QEMU_CI=1 is set, then create manual job + # if the branch/tag starts with 'edk2' + - if: '$QEMU_CI == "1" && $CI_PROJECT_NAMESPACE != "qemu-project" && $CI_COMMIT_REF_NAME =~ /^edk2/' + when: manual + + # In forks, if QEMU_CI=1 is set, then create manual job + # if last commit msg contains 'EDK2' (case insensitive) + - if: '$QEMU_CI == "1" && $CI_PROJECT_NAMESPACE != "qemu-project" && $CI_COMMIT_MESSAGE =~ /edk2/i' + when: on_success
manual on last line?
r~
Re: [PATCH 3/3] gitlab: honour QEMU_CI variable in edk2/opensbi jobs
Quoted 11 lines · Richard Henderson
.gitlab-ci.d/edk2.yml line 25
Quoted 9 lines · Daniel P. Berrangé
.gitlab-ci.d/edk2.yml +9
-0+ # In forks, if QEMU_CI=1 is set, then create manual job + # if the branch/tag starts with 'edk2' + - if: '$QEMU_CI == "1" && $CI_PROJECT_NAMESPACE != "qemu-project" && $CI_COMMIT_REF_NAME =~ /^edk2/' + when: manual + + # In forks, if QEMU_CI=1 is set, then create manual job + # if last commit msg contains 'EDK2' (case insensitive) + - if: '$QEMU_CI == "1" && $CI_PROJECT_NAMESPACE != "qemu-project" && $CI_COMMIT_MESSAGE =~ /edk2/i' + when: on_success
Quoted 1 line
manual on last line?
/facepalm Yes, of course.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Quoted 5 lines · Daniel P. Berrangé
When we introducd the
QEMU_CIvariable to control running of pipelines in gitlab forks, we didn't include the ekd2/opensbi jobs in the rules. This caused pipelines to be unexpectedly created in some cases. This addresses that oversight.
Quoted 8 lines
Daniel P. Berrangé (3):
gitlab: normalize indentation in edk2/opensbi rules
gitlab: tweak comments in edk2/opensbi jobs
gitlab: honourQEMU_CIvariable in edk2/opensbi jobs.gitlab-ci.d/edk2.yml | 133 ++++++++++++++++++++++---------------- .gitlab-ci.d/opensbi.yml | 134 +++++++++++++++++++++++---------------- 2 files changed, 158 insertions(+), 109 deletions(-)
Thanks, queued to my testing-next branch now:
https://gitlab.com/thuth/qemu/-/commits/testing-next
(with the "on_success" changed to "manual" in the last patch)
Thomas