gitlab-ci: handle all conditional jobs in the generating script (branch yem/gitlab-ci-cond)
cover gitlab-ci: handle all conditional jobs in the generating script (branch yem/gitlab-ci-cond)
Hello All!
The grammar in gitlab-CI yaml, to specify conditions under which jobs should be spawned or not, is not very nice to handle.
Take adavantage of the fact that we are now generating the pipeline description from a script, to decide in that script exactly what jobs should be spawned.
This is both more usual, to use a classic scripting language (shell for now, but could be whatever in the future), and more versatile (we can more easilt express conditions than in the limited gitlab-CI YAML).
Here are examples of resulting pipelines, with child pipelines created under the various expected conditions:
Arbitrary branch, only run the basic tests:
git push gitlab
https://gitlab.com/ymorin/buildroot/-/pipelines/186557459
Maintenance branch, run the basic checks, and check the defconfigs:
git push gitlab HEAD:toto.no.x
https://gitlab.com/ymorin/buildroot/-/pipelines/186556538
Tag, run all the tests:
git tag FOO-YEM
git push gitlab FOO-YEM
https://gitlab.com/ymorin/buildroot/-/pipelines/186559009
Build all defconfigs:
git push gitlab HEAD:FOO-defconfigs
https://gitlab.com/ymorin/buildroot/-/pipelines/186551495
Build one defconfig:
git push gitlab HEAD:FOO-warp7_defconfig
https://gitlab.com/ymorin/buildroot/-/pipelines/186530844
Build all runtime tests:
git push gitlab HEAD:FOO-runtime-tests
https://gitlab.com/ymorin/buildroot/-/pipelines/186554657
Build one runtime test:
git push gitlab HEAD:FOO-tests.package.test_openssh
https://gitlab.com/ymorin/buildroot/-/pipelines/186530918
Notes:
- 'gitlab' is my git remote pointing to gitlab,
- some pipelines were canceled to not exhaust my job limit
- there were too many jobs in the tag, so it exceed the limits of my free account...
Regards,
Yann E. MORIN.
The following changes since commit 97d431c181268716b38cd5a63eb0e8b78b003b3b
package/mrp: new package (2020-09-06 16:38:02 +0200)
are available in the git repository at:
https://git.busybox.net/~ymorin/git/buildroot
for you to fetch changes up to c1ea91abce5dbf7068615b247ff77dd7f8ec8fbb
gitlab-ci: only check defconfigs for known branches (2020-09-06 21:59:52 +0200)
----------------------------------------------------------------
Yann E. MORIN (10):
gitlab-ci: introduce main() in generating script
gitlab-ci: share the image version with the child
gitlab-ci: handle the defconfig build conditions in script
gitlab-ci: defconfig_check is included in defconfig build
gitlab-ci: handle the run-time tests conditions in script
gitlab-ci: handle single defconfig in script
gitlab-ci: handle single runtime test in script
gitlab-ci: handle the basic tests in script
gitlab-ci: move before-script before script
gitlab-ci: only check defconfigs for known branches
support/misc/gitlab-ci.yml.in | 70 +++---------------- support/scripts/generate-gitlab-ci-yml | 118 ++++++++++++++++++++++++++++++--- 2 files changed, 117 insertions(+), 71 deletions(-)
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH 1/10 gitlab-ci: introduce main() in generating script
This script is currently very crude, but we're going to extend it, at which point it will be nicer to have functions, local variables, et al.
Introduce a main() in preparation of those future evolutions.
--- support/scripts/generate-gitlab-ci-yml | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-)
support/scripts/generate-gitlab-ci-yml +11 -7
@@ -2,16 +2,20 @@ set -e set -o pipefail -input="${1}" +main() { + local template="${1}" -cat "${input}" + cat "${template}" -( - cd configs - LC_ALL=C ls -1 *_defconfig -) \ + ( + cd configs + LC_ALL=C ls -1 *_defconfig + ) \ | sed -r -e 's/^(.+)$/\1: { extends: .defconfig }\n\1_check: { extends: .defconfig_check }/' -./support/testing/run-tests -l 2>&1 \ + ./support/testing/run-tests -l 2>&1 \ | sed -r -e '/^test_run \((.*)\).*/!d; s//\1: { extends: .runtime_test }/' \ | LC_ALL=C sort +} + +main "${@}"
Signature · 1 line
2.20.1
PATCH 1/10 gitlab-ci: introduce main() in generating script
Le 06/09/2020 ? 22:12, Yann E. MORIN a ?crit?:
Quoted 9 lines
This script is currently very crude, but we're going to extend it, at which point it will be nicer to have functions, local variables, et al.
Introduce a
main()in preparation of those future evolutions.Signed-off-by Yann E. MORIN <[email protected]>Cc Romain Naour <[email protected]>Cc Thomas Petazzoni <[email protected]>Cc Arnout Vandecappelle <[email protected]>
Best regards,
Romain
Quoted 37 lines
--- support/scripts/generate-gitlab-ci-yml | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-)support/scripts/generate-gitlab-ci-yml +11
-7@@ -2,16 +2,20 @@ set -e set -o pipefail -input="${1}" +main() { + local template="${1}" -cat "${input}" + cat "${template}" -( - cd configs - LC_ALL=C ls -1 *_defconfig -) \ + ( + cd configs + LC_ALL=C ls -1 *_defconfig + ) \ | sed -r -e 's/^(.+)$/\1: { extends: .defconfig }\n\1_check: { extends: .defconfig_check }/' -./support/testing/run-tests -l 2>&1 \ + ./support/testing/run-tests -l 2>&1 \ | sed -r -e '/^test_run \((.*)\).*/!d; s//\1: { extends: .runtime_test }/' \ | LC_ALL=C sort +} + +main "${@}"
PATCH 2/10 gitlab-ci: share the image version with the child
In reply to Yann E. MORINCurrently, the image name and version are duplicated in the main pipeline and the generated, child pipeline.
This is a condition for a future gaffe, so let's use the image from the main pipeline when generating the child one.
--- support/misc/gitlab-ci.yml.in | 5 ----- support/scripts/generate-gitlab-ci-yml | 14 +++++++++++++- 2 files changed, 13 insertions(+), 6 deletions(-)
support/misc/gitlab-ci.yml.in +0 -5
@@ -1,8 +1,3 @@ -# Configuration for Gitlab-CI. -# Builds appear on https://gitlab.com/buildroot.org/buildroot/pipelines - -image: buildroot/base:20200814.2228 - .check_base: rules: - if: '$CI_COMMIT_REF_NAME =~ /^.*-.*_defconfig$/ || $CI_COMMIT_REF_NAME =~ /^.*-tests\..*$/'
support/scripts/generate-gitlab-ci-yml +13 -1
@@ -5,7 +5,7 @@ set -o pipefail main() { local template="${1}" - cat "${template}" + preamble "${template}" ( cd configs
@@ -18,4 +18,16 @@ main() { | LC_ALL=C sort } +preamble() { + local template="${1}" + + cat - "${template}" <<-_EOF_ + # This file is generated; do not edit! + # Builds appear on https://gitlab.com/buildroot.org/buildroot/pipelines + + image: ${CI_JOB_IMAGE} + +_EOF_ +} + main "${@}"
Signature · 1 line
2.20.1
PATCH 2/10 gitlab-ci: share the image version with the child
Le 06/09/2020 ? 22:12, Yann E. MORIN a ?crit?:
Quoted 43 lines
Currently, the image name and version are duplicated in the main pipeline and the generated, child pipeline.
This is a condition for a future gaffe, so let's use the image from the main pipeline when generating the child one.
Signed-off-by Yann E. MORIN <[email protected]>Cc Romain Naour <[email protected]>Cc Thomas Petazzoni <[email protected]>Cc Arnout Vandecappelle <[email protected]>--- support/misc/gitlab-ci.yml.in | 5 ----- support/scripts/generate-gitlab-ci-yml | 14 +++++++++++++- 2 files changed, 13 insertions(+), 6 deletions(-)support/misc/gitlab-ci.yml.in +0
-5@@ -1,8 +1,3 @@ -# Configuration for Gitlab-CI. -# Builds appear on https://gitlab.com/buildroot.org/buildroot/pipelines - -image: buildroot/base:20200814.2228 - .check_base: rules: - if: '$CI_COMMIT_REF_NAME =~ /^.*-.*_defconfig$/ || $CI_COMMIT_REF_NAME =~ /^.*-tests\..*$/'support/scripts/generate-gitlab-ci-yml +1
-1@@ -5,7 +5,7 @@ set -o pipefail main() { local template="${1}" - cat "${template}" + preamble "${template}" ( cd configs@@ -18,4 +18,16 @@ main() { | LC_ALL=C sort
support/scripts/generate-gitlab-ci-yml main() { · line 28
Quoted 10 lines
support/scripts/generate-gitlab-ci-yml +8
-0} +preamble() { + local template="${1}" + + cat - "${template}" <<-_EOF_ + # This file is generated; do not edit! + # Builds appear on https://gitlab.com/buildroot.org/buildroot/pipelines + + image: ${CI_JOB_IMAGE}
Note: This script must be run inside a gitlab runner where ${CI_JOB_IMAGE}, if it used locally (for testing) this variable is not defined.
Best regards,
Romain
Quoted 6 lines
support/scripts/generate-gitlab-ci-yml +4
-0+ +_EOF_ +} + main "${@}"
PATCH 3/10 gitlab-ci: handle the defconfig build conditions in script
In reply to Yann E. MORINNote that we do not propagate the existing comment, because it is partially wrong; instead we just keep the per-condition comments.
--- support/misc/gitlab-ci.yml.in | 9 ------- support/scripts/generate-gitlab-ci-yml | 34 ++++++++++++++++++++++---- 2 files changed, 29 insertions(+), 14 deletions(-)
support/misc/gitlab-ci.yml.in +0 -9
@@ -67,15 +67,6 @@ check-package: .defconfig: extends: .defconfig_base - # Running the defconfigs for every push is too much, so limit to - # explicit triggers through the API. - rules: - # For tags, create a pipeline. - - if: '$CI_COMMIT_TAG' - # For pipeline created by using a trigger token. - - if: '$CI_PIPELINE_TRIGGERED' - # For the branch or tag name named *-defconfigs, create a pipeline. - - if: '$CI_COMMIT_REF_NAME =~ /^.*-defconfigs$/' before_script: - DEFCONFIG_NAME=${CI_JOB_NAME}
support/scripts/generate-gitlab-ci-yml +29 -5
@@ -7,11 +7,7 @@ main() { preamble "${template}" - ( - cd configs - LC_ALL=C ls -1 *_defconfig - ) \ - | sed -r -e 's/^(.+)$/\1: { extends: .defconfig }\n\1_check: { extends: .defconfig_check }/' + gen_defconfigs ./support/testing/run-tests -l 2>&1 \ | sed -r -e '/^test_run \((.*)\).*/!d; s//\1: { extends: .runtime_test }/' \
@@ -30,4 +26,32 @@ preamble() { _EOF_ } +gen_defconfigs() { + local -a defconfigs + local build_defconfigs cfg + + defconfigs=( $(cd configs; LC_ALL=C ls -1 *_defconfig) ) + + build_defconfigs=false + if [ -n "${CI_COMMIT_TAG}" ]; then + # For tags, create a pipeline. + build_defconfigs=true + fi + if [ -n "${CI_PIPELINE_TRIGGERED}" ]; then + # For pipeline created by using a trigger token. + build_defconfigs=true + fi + case "${CI_COMMIT_REF_NAME}" in + # For the branch or tag name named *-defconfigs, create a pipeline. + (*-defconfigs) build_defconfigs=true;; + esac + + for cfg in "${defconfigs[@]}"; do + printf '%s_check: { extends: .defconfig_check }\n' "${cfg}" + if ${build_defconfigs}; then + printf '%s: { extends: .defconfig }\n' "${cfg}" + fi + done +} + main "${@}"
Signature · 1 line
2.20.1
PATCH 3/10 gitlab-ci: handle the defconfig build conditions in script
Le 06/09/2020 ? 22:12, Yann E. MORIN a ?crit?:
Quoted 68 lines
Note that we do not propagate the existing comment, because it is partially wrong; instead we just keep the per-condition comments.
Signed-off-by Yann E. MORIN <[email protected]>Cc Romain Naour <[email protected]>Cc Thomas Petazzoni <[email protected]>Cc Arnout Vandecappelle <[email protected]>--- support/misc/gitlab-ci.yml.in | 9 ------- support/scripts/generate-gitlab-ci-yml | 34 ++++++++++++++++++++++---- 2 files changed, 29 insertions(+), 14 deletions(-)support/misc/gitlab-ci.yml.in +0
-9@@ -67,15 +67,6 @@ check-package: .defconfig: extends: .defconfig_base - # Running the defconfigs for every push is too much, so limit to - # explicit triggers through the API. - rules: - # For tags, create a pipeline. - - if: '$CI_COMMIT_TAG' - # For pipeline created by using a trigger token. - - if: '$CI_PIPELINE_TRIGGERED' - # For the branch or tag name named *-defconfigs, create a pipeline. - - if: '$CI_COMMIT_REF_NAME =~ /^.*-defconfigs$/' before_script: - DEFCONFIG_NAME=${CI_JOB_NAME}support/scripts/generate-gitlab-ci-yml +16
-5@@ -7,11 +7,7 @@ main() { preamble "${template}" - ( - cd configs - LC_ALL=C ls -1 *_defconfig - ) \ - | sed -r -e 's/^(.+)$/\1: { extends: .defconfig }\n\1_check: { extends: .defconfig_check }/' + gen_defconfigs ./support/testing/run-tests -l 2>&1 \ | sed -r -e '/^test_run \((.*)\).*/!d; s//\1: { extends: .runtime_test }/' \@@ -30,4 +26,32 @@ preamble() { _EOF_ } +gen_defconfigs() { + local -a defconfigs + local build_defconfigs cfg + + defconfigs=( $(cd configs; LC_ALL=C ls -1 *_defconfig) ) + + build_defconfigs=false + if [ -n "${CI_COMMIT_TAG}" ]; then + # For tags, create a pipeline. + build_defconfigs=true + fi + if [ -n "${CI_PIPELINE_TRIGGERED}" ]; then + # For pipeline created by using a trigger token. + build_defconfigs=true + fi
support/scripts/generate-gitlab-ci-yml preamble() { · line 53
Quoted 10 lines
support/scripts/generate-gitlab-ci-yml +10
-0+ case "${CI_COMMIT_REF_NAME}" in + # For the branch or tag name named *-defconfigs, create a pipeline. + (*-defconfigs) build_defconfigs=true;; + esac + + for cfg in "${defconfigs[@]}"; do + printf '%s_check: { extends: .defconfig_check }\n' "${cfg}" + if ${build_defconfigs}; then + printf '%s: { extends: .defconfig }\n' "${cfg}" + fi
Indeed since the defconfig_check was introduced, defconfig_check jobs and defconfig jobs is created in the same pipeline when a tag or a branch named *-defconfigs is pushed in gitlab...
Hopefully this is fixed by the following patch of this series.
Best regards,
Romain
Quoted 5 lines
support/scripts/generate-gitlab-ci-yml +3
-0+ done +} + main "${@}"
PATCH 4/10 gitlab-ci: defconfig_check is included in defconfig build
In reply to Yann E. MORINWhen we build the defconfigs, we already check they are correct, so there is no need to run the correctness check explicitly.
--- support/scripts/generate-gitlab-ci-yml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
support/scripts/generate-gitlab-ci-yml +2 -1
@@ -47,9 +47,10 @@ gen_defconfigs() { esac for cfg in "${defconfigs[@]}"; do - printf '%s_check: { extends: .defconfig_check }\n' "${cfg}" if ${build_defconfigs}; then printf '%s: { extends: .defconfig }\n' "${cfg}" + else + printf '%s_check: { extends: .defconfig_check }\n' "${cfg}" fi done }
Signature · 1 line
2.20.1
PATCH 4/10 gitlab-ci: defconfig_check is included in defconfig build
Le 06/09/2020 ? 22:12, Yann E. MORIN a ?crit?:
Quoted 7 lines
When we build the defconfigs, we already check they are correct, so there is no need to run the correctness check explicitly.
Signed-off-by Yann E. MORIN <[email protected]>Cc Romain Naour <[email protected]>Cc Thomas Petazzoni <[email protected]>Cc Arnout Vandecappelle <[email protected]>
Best regards,
Romain
Quoted 21 lines
--- support/scripts/generate-gitlab-ci-yml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)support/scripts/generate-gitlab-ci-yml +2
-1@@ -47,9 +47,10 @@ gen_defconfigs() { esac for cfg in "${defconfigs[@]}"; do - printf '%s_check: { extends: .defconfig_check }\n' "${cfg}" if ${build_defconfigs}; then printf '%s: { extends: .defconfig }\n' "${cfg}" + else + printf '%s_check: { extends: .defconfig_check }\n' "${cfg}" fi done }
PATCH 5/10 gitlab-ci: handle the run-time tests conditions in script
In reply to Yann E. MORINNote that we do not propagate the existing comment, because it is partially wrong; instead we just keep the per-condition comments.
--- support/misc/gitlab-ci.yml.in | 9 ------- support/scripts/generate-gitlab-ci-yml | 33 ++++++++++++++++++++++---- 2 files changed, 29 insertions(+), 13 deletions(-)
support/misc/gitlab-ci.yml.in +0 -9
@@ -96,15 +96,6 @@ one-defconfig: .runtime_test: extends: .runtime_test_base - # Running the runtime tests for every push is too much, so limit to - # explicit triggers through the API. - rules: - # For tags, create a pipeline. - - if: '$CI_COMMIT_TAG' - # For pipeline created by using a trigger token. - - if: '$CI_PIPELINE_TRIGGERED' - # For the branch or tag name named *-runtime-tests, create a pipeline. - - if: '$CI_COMMIT_REF_NAME =~ /^.*-runtime-tests$/' before_script: - TEST_CASE_NAME=${CI_JOB_NAME}
support/scripts/generate-gitlab-ci-yml +29 -4
@@ -8,10 +8,7 @@ main() { preamble "${template}" gen_defconfigs - - ./support/testing/run-tests -l 2>&1 \ - | sed -r -e '/^test_run \((.*)\).*/!d; s//\1: { extends: .runtime_test }/' \ - | LC_ALL=C sort + gen_tests } preamble() {
@@ -55,4 +52,32 @@ gen_defconfigs() { done } +gen_tests() { + local -a tests + local run_tests tst + + tests=( $(./support/testing/run-tests -l 2>&1 \ + |sed -r -e '/^test_run \((.*)\).*/!d; s//\1/'\ + |LC_ALL=C sort) + ) + + run_tests=false + if [ -n "${CI_COMMIT_TAG}" ]; then + # For tags, create a pipeline. + run_tests=true + fi + if [ -n "${CI_PIPELINE_TRIGGERED}" ]; then + # For pipeline created by using a trigger token. + run_tests=true + fi + case "${CI_COMMIT_REF_NAME}" in + # For the branch or tag name named *-runtime-tests, create a pipeline. + (*-runtime-tests) run_tests=true;; + esac + + if ${run_tests}; then + printf '%s: { extends: .runtime_test }\n' "${tests[@]}" + fi +} + main "${@}"
Signature · 1 line
2.20.1
PATCH 5/10 gitlab-ci: handle the run-time tests conditions in script
Le 06/09/2020 ? 22:12, Yann E. MORIN a ?crit?:
Quoted 49 lines
Note that we do not propagate the existing comment, because it is partially wrong; instead we just keep the per-condition comments.
Signed-off-by Yann E. MORIN <[email protected]>Cc Romain Naour <[email protected]>Cc Thomas Petazzoni <[email protected]>Cc Arnout Vandecappelle <[email protected]>--- support/misc/gitlab-ci.yml.in | 9 ------- support/scripts/generate-gitlab-ci-yml | 33 ++++++++++++++++++++++---- 2 files changed, 29 insertions(+), 13 deletions(-)support/misc/gitlab-ci.yml.in +0
-9@@ -96,15 +96,6 @@ one-defconfig: .runtime_test: extends: .runtime_test_base - # Running the runtime tests for every push is too much, so limit to - # explicit triggers through the API. - rules: - # For tags, create a pipeline. - - if: '$CI_COMMIT_TAG' - # For pipeline created by using a trigger token. - - if: '$CI_PIPELINE_TRIGGERED' - # For the branch or tag name named *-runtime-tests, create a pipeline. - - if: '$CI_COMMIT_REF_NAME =~ /^.*-runtime-tests$/' before_script: - TEST_CASE_NAME=${CI_JOB_NAME}support/scripts/generate-gitlab-ci-yml +1
-4@@ -8,10 +8,7 @@ main() { preamble "${template}" gen_defconfigs - - ./support/testing/run-tests -l 2>&1 \ - | sed -r -e '/^test_run \((.*)\).*/!d; s//\1: { extends: .runtime_test }/' \ - | LC_ALL=C sort + gen_tests } preamble() {
support/scripts/generate-gitlab-ci-yml gen_defconfigs() { · line 61
Quoted 10 lines
support/scripts/generate-gitlab-ci-yml +7
-0@@ -55,4 +52,32 @@ gen_defconfigs() { done } +gen_tests() { + local -a tests + local run_tests tst + + tests=( $(./support/testing/run-tests -l 2>&1 \ + |sed -r -e '/^test_run \((.*)\).*/!d; s//\1/'\ + |LC_ALL=C sort)
Maybe we should keep the space after the pipe | ?
Best regards,
Romain
Quoted 23 lines
support/scripts/generate-gitlab-ci-yml +21
-0+ ) + + run_tests=false + if [ -n "${CI_COMMIT_TAG}" ]; then + # For tags, create a pipeline. + run_tests=true + fi + if [ -n "${CI_PIPELINE_TRIGGERED}" ]; then + # For pipeline created by using a trigger token. + run_tests=true + fi + case "${CI_COMMIT_REF_NAME}" in + # For the branch or tag name named *-runtime-tests, create a pipeline. + (*-runtime-tests) run_tests=true;; + esac + + if ${run_tests}; then + printf '%s: { extends: .runtime_test }\n' "${tests[@]}" + fi +} + main "${@}"
PATCH 5/10 gitlab-ci: handle the run-time tests conditions in script
Romain, All,
On 2020-09-08 23:14 +0200, Romain Naour spake thusly:
Quoted 9 lines
Le 06/09/2020 ? 22:12, Yann E. MORIN a ?crit?:
Quoted 8 lines
Note that we do not propagate the existing comment, because it is partially wrong; instead we just keep the per-condition comments.
Signed-off-by Yann E. MORIN <[email protected]>Cc Romain Naour <[email protected]>Cc Thomas Petazzoni <[email protected]>Cc Arnout Vandecappelle <[email protected]>---
[--SNIP--]
Quoted 20 lines
Quoted 20 lines
support/scripts/generate-gitlab-ci-yml +1
-4@@ -8,10 +8,7 @@ main() { preamble "${template}" gen_defconfigs - - ./support/testing/run-tests -l 2>&1 \ - | sed -r -e '/^test_run \((.*)\).*/!d; s//\1: { extends: .runtime_test }/' \ - | LC_ALL=C sort + gen_tests } preamble() {@@ -55,4 +52,32 @@ gen_defconfigs() { done }
Quoted 9 lines
support/scripts/generate-gitlab-ci-yml gen_defconfigs() { · line 61
Quoted 7 lines
support/scripts/generate-gitlab-ci-yml +7
-0+gen_tests() { + local -a tests + local run_tests tst + + tests=( $(./support/testing/run-tests -l 2>&1 \ + |sed -r -e '/^test_run \((.*)\).*/!d; s//\1/'\ + |LC_ALL=C sort)Maybe we should keep the space after the pipe | ?
I've fixed that when applying, thanks.
Regards,
Yann E. MORIN.
Quoted 30 lines
Reviewed-by Romain Naour <[email protected]>Best regards,
RomainQuoted 23 lines
support/scripts/generate-gitlab-ci-yml +21
-0+ ) + + run_tests=false + if [ -n "${CI_COMMIT_TAG}" ]; then + # For tags, create a pipeline. + run_tests=true + fi + if [ -n "${CI_PIPELINE_TRIGGERED}" ]; then + # For pipeline created by using a trigger token. + run_tests=true + fi + case "${CI_COMMIT_REF_NAME}" in + # For the branch or tag name named *-runtime-tests, create a pipeline. + (*-runtime-tests) run_tests=true;; + esac + + if ${run_tests}; then + printf '%s: { extends: .runtime_test }\n' "${tests[@]}" + fi +} + main "${@}"
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH 6/10 gitlab-ci: handle single defconfig in script
In reply to Yann E. MORIN--- support/misc/gitlab-ci.yml.in | 11 ----------- support/scripts/generate-gitlab-ci-yml | 8 +++++++- 2 files changed, 7 insertions(+), 12 deletions(-)
support/misc/gitlab-ci.yml.in +0 -11
@@ -64,20 +64,9 @@ check-package: - output/build/packages-file-list.txt - output/build/*/.config - runtime-test.log - -.defconfig: - extends: .defconfig_base before_script: - DEFCONFIG_NAME=${CI_JOB_NAME} -one-defconfig: - extends: .defconfig_base - rules: - # For the branch or tag name named *-*_defconfigs, create a pipeline. - - if: '$CI_COMMIT_REF_NAME =~ /^.*-.*_defconfig$/' - before_script: - - DEFCONFIG_NAME=$(echo ${CI_COMMIT_REF_NAME} | sed -e 's,^.*-,,g') - .runtime_test_base: # Keep build directories so the rootfs can be an artifact of the job. The # runner will clean up those files for us.
support/scripts/generate-gitlab-ci-yml +7 -1
@@ -40,7 +40,13 @@ gen_defconfigs() { fi case "${CI_COMMIT_REF_NAME}" in # For the branch or tag name named *-defconfigs, create a pipeline. - (*-defconfigs) build_defconfigs=true;; + (*-defconfigs) + build_defconfigs=true + ;; + (*-*_defconfig) + defconfigs=( "${CI_COMMIT_REF_NAME##*-}" ) + build_defconfigs=true + ;; esac for cfg in "${defconfigs[@]}"; do
Signature · 1 line
2.20.1
PATCH 6/10 gitlab-ci: handle single defconfig in script
Le 06/09/2020 ? 22:12, Yann E. MORIN a ?crit?:
Quoted 4 lines
Signed-off-by Yann E. MORIN <[email protected]>Cc Romain Naour <[email protected]>Cc Thomas Petazzoni <[email protected]>Cc Arnout Vandecappelle <[email protected]>
Best regards,
Romain
Quoted 50 lines
--- support/misc/gitlab-ci.yml.in | 11 ----------- support/scripts/generate-gitlab-ci-yml | 8 +++++++- 2 files changed, 7 insertions(+), 12 deletions(-)support/misc/gitlab-ci.yml.in +0
-11@@ -64,20 +64,9 @@ check-package: - output/build/packages-file-list.txt - output/build/*/.config - runtime-test.log - -.defconfig: - extends: .defconfig_base before_script: - DEFCONFIG_NAME=${CI_JOB_NAME} -one-defconfig: - extends: .defconfig_base - rules: - # For the branch or tag name named *-*_defconfigs, create a pipeline. - - if: '$CI_COMMIT_REF_NAME =~ /^.*-.*_defconfig$/' - before_script: - - DEFCONFIG_NAME=$(echo ${CI_COMMIT_REF_NAME} | sed -e 's,^.*-,,g') - .runtime_test_base: # Keep build directories so the rootfs can be an artifact of the job. The # runner will clean up those files for us.support/scripts/generate-gitlab-ci-yml +7
-1@@ -40,7 +40,13 @@ gen_defconfigs() { fi case "${CI_COMMIT_REF_NAME}" in # For the branch or tag name named *-defconfigs, create a pipeline. - (*-defconfigs) build_defconfigs=true;; + (*-defconfigs) + build_defconfigs=true + ;; + (*-*_defconfig) + defconfigs=( "${CI_COMMIT_REF_NAME##*-}" ) + build_defconfigs=true + ;; esac for cfg in "${defconfigs[@]}"; do
PATCH 7/10 gitlab-ci: handle single runtime test in script
In reply to Yann E. MORIN--- support/misc/gitlab-ci.yml.in | 9 --------- support/scripts/generate-gitlab-ci-yml | 8 +++++++- 2 files changed, 7 insertions(+), 10 deletions(-)
support/misc/gitlab-ci.yml.in +0 -9
@@ -82,15 +82,6 @@ check-package: - test-output/*.log - test-output/*/.config - test-output/*/images/* - -.runtime_test: - extends: .runtime_test_base before_script: - TEST_CASE_NAME=${CI_JOB_NAME} -one-runtime_test: - extends: .runtime_test_base - rules: - - if: '$CI_COMMIT_REF_NAME =~ /^.*-tests\..*$/' - before_script: - - TEST_CASE_NAME=$(echo ${CI_COMMIT_REF_NAME} | sed -e 's,^.*-,,g')
support/scripts/generate-gitlab-ci-yml +7 -1
@@ -78,7 +78,13 @@ gen_tests() { fi case "${CI_COMMIT_REF_NAME}" in # For the branch or tag name named *-runtime-tests, create a pipeline. - (*-runtime-tests) run_tests=true;; + (*-runtime-tests) + run_tests=true + ;; + (*-tests.*) + tests=( "${CI_COMMIT_REF_NAME##*-}" ) + run_tests=true + ;; esac if ${run_tests}; then
Signature · 1 line
2.20.1
PATCH 7/10 gitlab-ci: handle single runtime test in script
Le 06/09/2020 ? 22:12, Yann E. MORIN a ?crit?:
Quoted 4 lines
Signed-off-by Yann E. MORIN <[email protected]>Cc Romain Naour <[email protected]>Cc Thomas Petazzoni <[email protected]>Cc Arnout Vandecappelle <[email protected]>
Best regards,
Romain
Quoted 45 lines
--- support/misc/gitlab-ci.yml.in | 9 --------- support/scripts/generate-gitlab-ci-yml | 8 +++++++- 2 files changed, 7 insertions(+), 10 deletions(-)support/misc/gitlab-ci.yml.in +0
-9@@ -82,15 +82,6 @@ check-package: - test-output/*.log - test-output/*/.config - test-output/*/images/* - -.runtime_test: - extends: .runtime_test_base before_script: - TEST_CASE_NAME=${CI_JOB_NAME} -one-runtime_test: - extends: .runtime_test_base - rules: - - if: '$CI_COMMIT_REF_NAME =~ /^.*-tests\..*$/' - before_script: - - TEST_CASE_NAME=$(echo ${CI_COMMIT_REF_NAME} | sed -e 's,^.*-,,g')support/scripts/generate-gitlab-ci-yml +7
-1@@ -78,7 +78,13 @@ gen_tests() { fi case "${CI_COMMIT_REF_NAME}" in # For the branch or tag name named *-runtime-tests, create a pipeline. - (*-runtime-tests) run_tests=true;; + (*-runtime-tests) + run_tests=true + ;; + (*-tests.*) + tests=( "${CI_COMMIT_REF_NAME##*-}" ) + run_tests=true + ;; esac if ${run_tests}; then
PATCH 8/10 gitlab-ci: handle the basic tests in script
In reply to Yann E. MORINNote that those tests were so far ignored only when requesting a single defconfig build, or a single runtime test build; everything else was trigerring thoses tests.
However, it feels more natural that they are also ignored when all defconfigs build. or all runtime tests, are explictly requested.
--- support/misc/gitlab-ci.yml.in | 15 +++------------ support/scripts/generate-gitlab-ci-yml | 21 +++++++++++++++++++-- 2 files changed, 22 insertions(+), 14 deletions(-)
support/misc/gitlab-ci.yml.in +3 -12
@@ -1,24 +1,15 @@ -.check_base: - rules: - - if: '$CI_COMMIT_REF_NAME =~ /^.*-.*_defconfig$/ || $CI_COMMIT_REF_NAME =~ /^.*-tests\..*$/' - when: never - - when: always - -check-DEVELOPERS: - extends: .check_base +.check-DEVELOPERS_base: # get-developers should print just "No action specified"; if it prints # anything else, it's a parse error. # The initial ! is removed by YAML so we need to quote it. script: - "! utils/get-developers | grep -v 'No action specified'" -check-flake8: - extends: .check_base +.check-flake8_base: script: - make check-flake8 -check-package: - extends: .check_base +.check-package_base: script: - make check-package
support/scripts/generate-gitlab-ci-yml +19 -2
@@ -7,6 +7,7 @@ main() { preamble "${template}" + gen_basics gen_defconfigs gen_tests }
@@ -23,6 +24,22 @@ preamble() { _EOF_ } +gen_basics() { + local tst + + # Skip basic tests when explicitly building defconfigs or runtime tests + case "${CI_COMMIT_REF_NAME}" in + (*-defconfigs) return;; + (*-*_defconfig) return;; + (*-runtime-tests) return;; + (*-tests.*) return;; + esac + + for tst in DEVELOPERS flake8 package; do + printf 'check-%s: { extends: .check-%s_base }\n' "${tst}" "${tst}" + done +} + gen_defconfigs() { local -a defconfigs local build_defconfigs cfg
@@ -51,7 +68,7 @@ gen_defconfigs() { for cfg in "${defconfigs[@]}"; do if ${build_defconfigs}; then - printf '%s: { extends: .defconfig }\n' "${cfg}" + printf '%s: { extends: .defconfig_base }\n' "${cfg}" else printf '%s_check: { extends: .defconfig_check }\n' "${cfg}" fi
@@ -88,7 +105,7 @@ gen_tests() { esac if ${run_tests}; then - printf '%s: { extends: .runtime_test }\n' "${tests[@]}" + printf '%s: { extends: .runtime_test_base }\n' "${tests[@]}" fi }
Signature · 1 line
2.20.1
PATCH 8/10 gitlab-ci: handle the basic tests in script
Le 06/09/2020 ? 22:12, Yann E. MORIN a ?crit?:
Quoted 6 lines
Note that those tests were so far ignored only when requesting a single defconfig build, or a single runtime test build; everything else was trigerring thoses tests.
However, it feels more natural that they are also ignored when all defconfigs build. or all runtime tests, are explictly requested.
Agree.
Best regards,
Romain
Quoted 96 lines
Signed-off-by Yann E. MORIN <[email protected]>Cc Romain Naour <[email protected]>Cc Thomas Petazzoni <[email protected]>Cc Arnout Vandecappelle <[email protected]>--- support/misc/gitlab-ci.yml.in | 15 +++------------ support/scripts/generate-gitlab-ci-yml | 21 +++++++++++++++++++-- 2 files changed, 22 insertions(+), 14 deletions(-)support/misc/gitlab-ci.yml.in +3
-12@@ -1,24 +1,15 @@ -.check_base: - rules: - - if: '$CI_COMMIT_REF_NAME =~ /^.*-.*_defconfig$/ || $CI_COMMIT_REF_NAME =~ /^.*-tests\..*$/' - when: never - - when: always - -check-DEVELOPERS: - extends: .check_base +.check-DEVELOPERS_base: # get-developers should print just "No action specified"; if it prints # anything else, it's a parse error. # The initial ! is removed by YAML so we need to quote it. script: - "! utils/get-developers | grep -v 'No action specified'" -check-flake8: - extends: .check_base +.check-flake8_base: script: - make check-flake8 -check-package: - extends: .check_base +.check-package_base: script: - make check-packagesupport/scripts/generate-gitlab-ci-yml +19
-2@@ -7,6 +7,7 @@ main() { preamble "${template}" + gen_basics gen_defconfigs gen_tests }@@ -23,6 +24,22 @@ preamble() { _EOF_ } +gen_basics() { + local tst + + # Skip basic tests when explicitly building defconfigs or runtime tests + case "${CI_COMMIT_REF_NAME}" in + (*-defconfigs) return;; + (*-*_defconfig) return;; + (*-runtime-tests) return;; + (*-tests.*) return;; + esac + + for tst in DEVELOPERS flake8 package; do + printf 'check-%s: { extends: .check-%s_base }\n' "${tst}" "${tst}" + done +} + gen_defconfigs() { local -a defconfigs local build_defconfigs cfg@@ -51,7 +68,7 @@ gen_defconfigs() { for cfg in "${defconfigs[@]}"; do if ${build_defconfigs}; then - printf '%s: { extends: .defconfig }\n' "${cfg}" + printf '%s: { extends: .defconfig_base }\n' "${cfg}" else printf '%s_check: { extends: .defconfig_check }\n' "${cfg}" fi@@ -88,7 +105,7 @@ gen_tests() { esac if ${run_tests}; then - printf '%s: { extends: .runtime_test }\n' "${tests[@]}" + printf '%s: { extends: .runtime_test_base }\n' "${tests[@]}" fi }
PATCH 9/10 gitlab-ci: move before-script before script
In reply to Yann E. MORIN--- support/misc/gitlab-ci.yml.in | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)
support/misc/gitlab-ci.yml.in +6 -6
@@ -14,6 +14,8 @@ - make check-package .defconfig_check: + before_script: + - DEFCONFIG_NAME=$(echo ${CI_JOB_NAME} | sed -e 's,_check$,,g') script: - echo "Configure Buildroot for ${DEFCONFIG_NAME}" - make ${DEFCONFIG_NAME}
@@ -23,10 +25,10 @@ expire_in: 2 weeks paths: - .config - before_script: - - DEFCONFIG_NAME=$(echo ${CI_JOB_NAME} | sed -e 's,_check$,,g') .defconfig_base: + before_script: + - DEFCONFIG_NAME=${CI_JOB_NAME} script: - echo "Configure Buildroot for ${DEFCONFIG_NAME}" - make ${DEFCONFIG_NAME}
@@ -55,10 +57,10 @@ - output/build/packages-file-list.txt - output/build/*/.config - runtime-test.log - before_script: - - DEFCONFIG_NAME=${CI_JOB_NAME} .runtime_test_base: + before_script: + - TEST_CASE_NAME=${CI_JOB_NAME} # Keep build directories so the rootfs can be an artifact of the job. The # runner will clean up those files for us. # Multiply every emulator timeout by 10 to avoid sporadic failures in
@@ -73,6 +75,4 @@ - test-output/*.log - test-output/*/.config - test-output/*/images/* - before_script: - - TEST_CASE_NAME=${CI_JOB_NAME}
Signature · 1 line
2.20.1
PATCH 9/10 gitlab-ci: move before-script before script
Le 06/09/2020 ? 22:12, Yann E. MORIN a ?crit?:
Quoted 4 lines
Signed-off-by Yann E. MORIN <[email protected]>Cc Romain Naour <[email protected]>Cc Thomas Petazzoni <[email protected]>Cc Arnout Vandecappelle <[email protected]>
Best regards,
Romain
Quoted 51 lines
--- support/misc/gitlab-ci.yml.in | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)support/misc/gitlab-ci.yml.in +6
-6@@ -14,6 +14,8 @@ - make check-package .defconfig_check: + before_script: + - DEFCONFIG_NAME=$(echo ${CI_JOB_NAME} | sed -e 's,_check$,,g') script: - echo "Configure Buildroot for ${DEFCONFIG_NAME}" - make ${DEFCONFIG_NAME}@@ -23,10 +25,10 @@ expire_in: 2 weeks paths: - .config - before_script: - - DEFCONFIG_NAME=$(echo ${CI_JOB_NAME} | sed -e 's,_check$,,g') .defconfig_base: + before_script: + - DEFCONFIG_NAME=${CI_JOB_NAME} script: - echo "Configure Buildroot for ${DEFCONFIG_NAME}" - make ${DEFCONFIG_NAME}@@ -55,10 +57,10 @@ - output/build/packages-file-list.txt - output/build/*/.config - runtime-test.log - before_script: - - DEFCONFIG_NAME=${CI_JOB_NAME} .runtime_test_base: + before_script: + - TEST_CASE_NAME=${CI_JOB_NAME} # Keep build directories so the rootfs can be an artifact of the job. The # runner will clean up those files for us. # Multiply every emulator timeout by 10 to avoid sporadic failures in@@ -73,6 +75,4 @@ - test-output/*.log - test-output/*/.config - test-output/*/images/* - before_script: - - TEST_CASE_NAME=${CI_JOB_NAME}
PATCH 10/10 gitlab-ci: only check defconfigs for known branches
In reply to Yann E. MORINCurrently, the check of defconfigs is run for all branches, even those that are pushed only to run runtime tests. This is very inconvenient.
In fact, we only want to check the defconfigs on standard branches, that is master, next, and the maintenance branches.
This will also decrease drastically the number gitlab-ci minutes used when one pushes their repo to gitlab.com, where the number of CI minutes are now going to be pretty severely restricted.
--- support/scripts/generate-gitlab-ci-yml | 29 ++++++++++++++------------ 1 file changed, 16 insertions(+), 13 deletions(-)
support/scripts/generate-gitlab-ci-yml +16 -13
@@ -42,37 +42,40 @@ gen_basics() { gen_defconfigs() { local -a defconfigs - local build_defconfigs cfg + local template cfg ext defconfigs=( $(cd configs; LC_ALL=C ls -1 *_defconfig) ) - build_defconfigs=false if [ -n "${CI_COMMIT_TAG}" ]; then # For tags, create a pipeline. - build_defconfigs=true + template=base fi if [ -n "${CI_PIPELINE_TRIGGERED}" ]; then # For pipeline created by using a trigger token. - build_defconfigs=true + template=base fi case "${CI_COMMIT_REF_NAME}" in + # For master, next, and maintenance branches, only check the defconfigs + (master|next|????.??.x) + template=check + ext=_check + ;; # For the branch or tag name named *-defconfigs, create a pipeline. (*-defconfigs) - build_defconfigs=true + template=base ;; (*-*_defconfig) defconfigs=( "${CI_COMMIT_REF_NAME##*-}" ) - build_defconfigs=true + template=base ;; esac - for cfg in "${defconfigs[@]}"; do - if ${build_defconfigs}; then - printf '%s: { extends: .defconfig_base }\n' "${cfg}" - else - printf '%s_check: { extends: .defconfig_check }\n' "${cfg}" - fi - done + if [ -n "${template}" ]; then + for cfg in "${defconfigs[@]}"; do + printf '%s%s: { extends: .defconfig_%s }\n' \ + "${cfg}" "${ext}" "${template}" + done + fi } gen_tests() {
Signature · 1 line
2.20.1
PATCH 10/10 gitlab-ci: only check defconfigs for known branches
Le 06/09/2020 ? 22:12, Yann E. MORIN a ?crit?:
Quoted 9 lines
Currently, the check of defconfigs is run for all branches, even those that are pushed only to run runtime tests. This is very inconvenient.
In fact, we only want to check the defconfigs on standard branches, that is master, next, and the maintenance branches.
This will also decrease drastically the number gitlab-ci minutes used when one pushes their repo to gitlab.com, where the number of CI minutes are now going to be pretty severely restricted.
Note, some qemu defconfig can be runtime tested. But ok it's not part of the runtime testing infrastructure.
Best regards,
Romain
Quoted 68 lines
Signed-off-by Yann E. MORIN <[email protected]>Cc Romain Naour <[email protected]>Cc Thomas Petazzoni <[email protected]>Cc Arnout Vandecappelle <[email protected]>--- support/scripts/generate-gitlab-ci-yml | 29 ++++++++++++++------------ 1 file changed, 16 insertions(+), 13 deletions(-)support/scripts/generate-gitlab-ci-yml +16
-13@@ -42,37 +42,40 @@ gen_basics() { gen_defconfigs() { local -a defconfigs - local build_defconfigs cfg + local template cfg ext defconfigs=( $(cd configs; LC_ALL=C ls -1 *_defconfig) ) - build_defconfigs=false if [ -n "${CI_COMMIT_TAG}" ]; then # For tags, create a pipeline. - build_defconfigs=true + template=base fi if [ -n "${CI_PIPELINE_TRIGGERED}" ]; then # For pipeline created by using a trigger token. - build_defconfigs=true + template=base fi case "${CI_COMMIT_REF_NAME}" in + # For master, next, and maintenance branches, only check the defconfigs + (master|next|????.??.x) + template=check + ext=_check + ;; # For the branch or tag name named *-defconfigs, create a pipeline. (*-defconfigs) - build_defconfigs=true + template=base ;; (*-*_defconfig) defconfigs=( "${CI_COMMIT_REF_NAME##*-}" ) - build_defconfigs=true + template=base ;; esac - for cfg in "${defconfigs[@]}"; do - if ${build_defconfigs}; then - printf '%s: { extends: .defconfig_base }\n' "${cfg}" - else - printf '%s_check: { extends: .defconfig_check }\n' "${cfg}" - fi - done + if [ -n "${template}" ]; then + for cfg in "${defconfigs[@]}"; do + printf '%s%s: { extends: .defconfig_%s }\n' \ + "${cfg}" "${ext}" "${template}" + done + fi } gen_tests() {
cover gitlab-ci: handle all conditional jobs in the generating script (branch yem/gitlab-ci-cond)
In reply to Yann E. MORINHello Yann,
Le 06/09/2020 ? 22:12, Yann E. MORIN a ?crit?:
Quoted 5 lines
Hello All!
The grammar in gitlab-CI yaml, to specify conditions under which jobs should be spawned or not, is not very nice to handle.
Quoted 7 lines
Take adavantage of the fact that we are now generating the pipeline description from a script, to decide in that script exactly what jobs should be spawned.
This is both more usual, to use a classic scripting language (shell for now, but could be whatever in the future), and more versatile (we can more easilt express conditions than in the limited gitlab-CI YAML).
On one hand, it's a pity to workaround the gitlab-ci Yaml and not take the opportunity to practice this specific language...
But on the other hand the language is not still stable over the time (see the only/except keyword deprecation we used until recently).
Also the generate-gitlab-ci-yml after this series doesn't seem too complex :)
Maybe we can complete the manual about defconfigs testing in gitlab. runtime-tests in gitlab is already described in the manual.
Best regards,
Romain
Quoted 75 lines
Here are examples of resulting pipelines, with child pipelines created under the various expected conditions:
Arbitrary branch, only run the basic tests:
git push gitlab
https://gitlab.com/ymorin/buildroot/-/pipelines/186557459Maintenance branch, run the basic checks, and check the defconfigs:
git push gitlab HEAD:toto.no.x
https://gitlab.com/ymorin/buildroot/-/pipelines/186556538Tag, run all the tests:
git tag FOO-YEM
git push gitlab FOO-YEM
https://gitlab.com/ymorin/buildroot/-/pipelines/186559009Build all defconfigs:
git push gitlab HEAD:FOO-defconfigs
https://gitlab.com/ymorin/buildroot/-/pipelines/186551495Build one defconfig:
git push gitlab HEAD:FOO-warp7_defconfig
https://gitlab.com/ymorin/buildroot/-/pipelines/186530844Build all runtime tests:
git push gitlab HEAD:FOO-runtime-tests
https://gitlab.com/ymorin/buildroot/-/pipelines/186554657Build one runtime test:
git push gitlab HEAD:FOO-tests.package.test_openssh
https://gitlab.com/ymorin/buildroot/-/pipelines/186530918Notes:
- 'gitlab' is my git remote pointing to gitlab,
- some pipelines were canceled to not exhaust my job limit
- there were too many jobs in the tag, so it exceed the limits of my free account...Regards,
Yann E. MORIN.The following changes since commit
97d431c181268716b38cd5a63eb0e8b78b003b3bpackage/mrp: new package (2020-09-06 16:38:02 +0200)
are available in the git repository at:
https://git.busybox.net/~ymorin/git/buildroot
for you to fetch changes up to
c1ea91abce5dbf7068615b247ff77dd7f8ec8fbbgitlab-ci: only check defconfigs for known branches (2020-09-06 21:59:52 +0200)
----------------------------------------------------------------
Yann E. MORIN (10):
gitlab-ci: introducemain()in generating script
gitlab-ci: share the image version with the child
gitlab-ci: handle the defconfig build conditions in script
gitlab-ci:defconfig_checkis included in defconfig build
gitlab-ci: handle the run-time tests conditions in script
gitlab-ci: handle single defconfig in script
gitlab-ci: handle single runtime test in script
gitlab-ci: handle the basic tests in script
gitlab-ci: move before-script before script
gitlab-ci: only check defconfigs for known branchessupport/misc/gitlab-ci.yml.in | 70 +++---------------- support/scripts/generate-gitlab-ci-yml | 118 ++++++++++++++++++++++++++++++--- 2 files changed, 117 insertions(+), 71 deletions(-)
cover gitlab-ci: handle all conditional jobs in the generating script (branch yem/gitlab-ci-cond)
Romain, All,
On 2020-09-08 22:47 +0200, Romain Naour spake thusly:
Quoted 14 lines
Le 06/09/2020 ? 22:12, Yann E. MORIN a ?crit?:
Quoted 5 lines
Hello All!
The grammar in gitlab-CI yaml, to specify conditions under which jobs should be spawned or not, is not very nice to handle.
Quoted 7 lines
Take adavantage of the fact that we are now generating the pipeline description from a script, to decide in that script exactly what jobs should be spawned.
This is both more usual, to use a classic scripting language (shell for now, but could be whatever in the future), and more versatile (we can more easilt express conditions than in the limited gitlab-CI YAML).
Quoted 8 lines
On one hand, it's a pity to workaround the gitlab-ci Yaml and not take the opportunity to practice this specific language...
But on the other hand the language is not still stable over the time (see the only/except keyword deprecation we used until recently).
Also the generate-gitlab-ci-yml after this series doesn't seem too complex :)Maybe we can complete the manual about defconfigs testing in gitlab. runtime-tests in gitlab is already described in the manual.
Thanks for the review. Series applied to master now.
Regards,
Yann E. MORIN.
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'