automation: avoid unnecessary analyses
cover automation: avoid unnecessary analyses
This series aims to reduce the analyses performed by the ECLAIR runner by avoiding some branches already excluded by other jobs and requiring analyses on people/.* to be manually triggered.
Simone Ballarin (2):
automation: avoid pipelines on specific branches automation/eclair: avoid unintentional ECLAIR analysis
.gitlab-ci.yml | 6 ++++++ automation/gitlab-ci/analyze.yaml | 14 ++++++++++---- automation/gitlab-ci/build.yaml | 6 ------ automation/gitlab-ci/test.yaml | 5 ----- 4 files changed, 16 insertions(+), 15 deletions(-)
Signature · 1 line
2.34.1
PATCH 1/2 automation: avoid pipelines on specific branches
This patch avoids the execution of pipelines in the following branches:
- master
- smoke
- coverirty-tested/.*
- stable-.*
The job-level exclusions have been removed as they are pointless with this new workspace-level exclusion.
--- .gitlab-ci.yml | 6 ++++++ 1 file changed, 6 insertions(+)
.gitlab-ci.yml +6 -0
@@ -1,3 +1,9 @@ +workflow: + rules: + - if: $CI_COMMIT_BRANCH =~ /^(master|smoke|^coverity-tested\/.*|stable-.*)$/ + when: never + - when: always + stages: - analyze - build
Signature · 1 line
2.34.1
Re: [XEN PATCH 1/2] automation: avoid pipelines on specific branches
Quoted 10 lines · Simone Ballarin
This patch avoids the execution of pipelines in the following branches:
- master
- smoke
- coverirty-tested/.*
- stable-.*The job-level exclusions have been removed as they are pointless with this new workspace-level exclusion.
Very good thanks!
One question on the commit message: what did you mean by "the job-level exclusions have been removed as they are pointless with this new workspace-level exclusion"
It doesn't look like any job-level exclusion has been removed by this patch?
Quoted 23 lines
Signed-off-by Simone Ballarin <[email protected]>--- .gitlab-ci.yml | 6 ++++++ 1 file changed, 6 insertions(+).gitlab-ci.yml +6
-0@@ -1,3 +1,9 @@ +workflow: + rules: + - if: $CI_COMMIT_BRANCH =~ /^(master|smoke|^coverity-tested\/.*|stable-.*)$/ + when: never + - when: always + stages: - analyze - buildSignature · 1 line
2.34.1
PATCH 2/2 automation/eclair: avoid unintentional ECLAIR analysis
In reply to Simone BallarinWith this patch, ECLAIR jobs will need to be manually started for "people/.*" pipelines.
This avoids occupying the runner on analyzes that might not be used by developers.
If developers want to analyze their own repositories they need to launch them from GitLab.
--- automation/gitlab-ci/analyze.yaml | 14 ++++++++++---- automation/gitlab-ci/build.yaml | 6 ------ automation/gitlab-ci/test.yaml | 5 ----- 3 files changed, 10 insertions(+), 15 deletions(-)
automation/gitlab-ci/analyze.yaml +10 -4
@@ -20,21 +20,27 @@ codequality: gl-code-quality-report.json needs: [] -eclair-x86_64: +.eclair-analysis:triggered: extends: .eclair-analysis + allow_failure: true + rules: + - if: $CI_PROJECT_PATH =~ /^xen-project\/people\/.*$/ + when: manual + - when: always + +eclair-x86_64: + extends: .eclair-analysis:triggered variables: LOGFILE: "eclair-x86_64.log" VARIANT: "X86_64" RULESET: "Set1" - allow_failure: true eclair-ARM64: - extends: .eclair-analysis + extends: .eclair-analysis:triggered variables: LOGFILE: "eclair-ARM64.log" VARIANT: "ARM64" RULESET: "Set1" - allow_failure: true .eclair-analysis:on-schedule: extends: .eclair-analysis
automation/gitlab-ci/build.yaml +0 -6
@@ -12,11 +12,6 @@ - '*/*.log' when: always needs: [] - except: - - master - - smoke - - /^coverity-tested\/.*/ - - /^stable-.*/ .gcc-tmpl: variables: &gcc
@@ -269,7 +264,6 @@ .test-jobs-artifact-common: stage: build needs: [] - except: !reference [.test-jobs-common, except] # Arm test artifacts
automation/gitlab-ci/test.yaml +0 -5
@@ -1,11 +1,6 @@ .test-jobs-common: stage: test image: registry.gitlab.com/xen-project/xen/${CONTAINER} - except: - - master - - smoke - - /^coverity-tested\/.*/ - - /^stable-.*/ .arm64-test-needs: &arm64-test-needs - alpine-3.18-arm64-rootfs-export
Signature · 1 line
2.34.1
Re: [XEN PATCH 2/2] automation/eclair: avoid unintentional ECLAIR analysis
Quoted 42 lines · Simone Ballarin
With this patch, ECLAIR jobs will need to be manually started for "people/.*" pipelines.
This avoids occupying the runner on analyzes that might not be used by developers.
If developers want to analyze their own repositories they need to launch them from GitLab.
Signed-off-by Simone Ballarin <[email protected]>--- automation/gitlab-ci/analyze.yaml | 14 ++++++++++---- automation/gitlab-ci/build.yaml | 6 ------ automation/gitlab-ci/test.yaml | 5 ----- 3 files changed, 10 insertions(+), 15 deletions(-)automation/gitlab-ci/analyze.yaml +9
-2@@ -20,21 +20,27 @@ codequality: gl-code-quality-report.json needs: [] -eclair-x86_64: +.eclair-analysis:triggered: extends: .eclair-analysis + allow_failure: true + rules: + - if: $CI_PROJECT_PATH =~ /^xen-project\/people\/.*$/ + when: manual + - when: always + +eclair-x86_64: + extends: .eclair-analysis:triggered variables: LOGFILE: "eclair-x86_64.log" VARIANT: "X86_64" RULESET: "Set1" - allow_failure: true eclair-ARM64:
automation/gitlab-ci/analyze.yaml line 46
Quoted 10 lines
automation/gitlab-ci/analyze.yaml +1
-2- extends: .eclair-analysis + extends: .eclair-analysis:triggered variables: LOGFILE: "eclair-ARM64.log" VARIANT: "ARM64" RULESET: "Set1" - allow_failure: true .eclair-analysis:on-schedule: extends: .eclair-analysis
Everything so far looks great and I am ready to Ack.
Quoted 30 lines
automation/gitlab-ci/build.yaml +0
-6@@ -12,11 +12,6 @@ - '*/*.log' when: always needs: [] - except: - - master - - smoke - - /^coverity-tested\/.*/ - - /^stable-.*/ .gcc-tmpl: variables: &gcc@@ -269,7 +264,6 @@ .test-jobs-artifact-common: stage: build needs: [] - except: !reference [.test-jobs-common, except] # Arm test artifactsautomation/gitlab-ci/test.yaml +0
-0@@ -1,11 +1,6 @@ .test-jobs-common:
automation/gitlab-ci/test.yaml line 6
Quoted 10 lines
automation/gitlab-ci/test.yaml +0
-5stage: test image: registry.gitlab.com/xen-project/xen/${CONTAINER} - except: - - master - - smoke - - /^coverity-tested\/.*/ - - /^stable-.*/ .arm64-test-needs: &arm64-test-needs - alpine-3.18-arm64-rootfs-export
These changes instead belongs to the first patch, right?
Re: [XEN PATCH 2/2] automation/eclair: avoid unintentional ECLAIR analysis
Quoted 95 lines · Stefano Stabellini
Quoted 42 lines · Simone Ballarin
With this patch, ECLAIR jobs will need to be manually started for "people/.*" pipelines.
This avoids occupying the runner on analyzes that might not be used by developers.
If developers want to analyze their own repositories they need to launch them from GitLab.
Signed-off-by Simone Ballarin <[email protected]>--- automation/gitlab-ci/analyze.yaml | 14 ++++++++++---- automation/gitlab-ci/build.yaml | 6 ------ automation/gitlab-ci/test.yaml | 5 ----- 3 files changed, 10 insertions(+), 15 deletions(-)automation/gitlab-ci/analyze.yaml +9
-2@@ -20,21 +20,27 @@ codequality: gl-code-quality-report.json needs: [] -eclair-x86_64: +.eclair-analysis:triggered: extends: .eclair-analysis + allow_failure: true + rules: + - if: $CI_PROJECT_PATH =~ /^xen-project\/people\/.*$/ + when: manual + - when: always + +eclair-x86_64: + extends: .eclair-analysis:triggered variables: LOGFILE: "eclair-x86_64.log" VARIANT: "X86_64" RULESET: "Set1" - allow_failure: true eclair-ARM64:automation/gitlab-ci/analyze.yaml line 46
Quoted 10 lines
automation/gitlab-ci/analyze.yaml +1
-2- extends: .eclair-analysis + extends: .eclair-analysis:triggered variables: LOGFILE: "eclair-ARM64.log" VARIANT: "ARM64" RULESET: "Set1" - allow_failure: true .eclair-analysis:on-schedule: extends: .eclair-analysisEverything so far looks great and I am ready to Ack.
Quoted 38 lines
automation/gitlab-ci/build.yaml +0
-6@@ -12,11 +12,6 @@ - '*/*.log' when: always needs: [] - except: - - master - - smoke - - /^coverity-tested\/.*/ - - /^stable-.*/ .gcc-tmpl: variables: &gcc@@ -269,7 +264,6 @@ .test-jobs-artifact-common: stage: build needs: [] - except: !reference [.test-jobs-common, except] # Arm test artifactsautomation/gitlab-ci/test.yaml +0
-5@@ -1,11 +1,6 @@ .test-jobs-common: stage: test image: registry.gitlab.com/xen-project/xen/${CONTAINER} - except: - - master - - smoke - - /^coverity-tested\/.*/ - - /^stable-.*/
Quoted 4 lines
automation/gitlab-ci/test.yaml line 6
Quoted 2 lines
automation/gitlab-ci/test.yaml +0
-0.arm64-test-needs: &arm64-test-needs - alpine-3.18-arm64-rootfs-exportThese changes instead belongs to the first patch, right?
Yes, sorry. Moreover, it is the answer to your question on the other patch.
Signature · 3 lines
Simone Ballarin, M.Sc. Field Application Engineer, BUGSENG (https://bugseng.com)
Re: [XEN PATCH 2/2] automation/eclair: avoid unintentional ECLAIR analysis
Quoted 99 lines · Simone Ballarin
Quoted 98 lines · Stefano Stabellini
Quoted 43 lines · Simone Ballarin
With this patch, ECLAIR jobs will need to be manually started for "people/.*" pipelines.
This avoids occupying the runner on analyzes that might not be used by developers.
If developers want to analyze their own repositories they need to launch them from GitLab.
Signed-off-by Simone Ballarin <[email protected]>--- automation/gitlab-ci/analyze.yaml | 14 ++++++++++---- automation/gitlab-ci/build.yaml | 6 ------ automation/gitlab-ci/test.yaml | 5 ----- 3 files changed, 10 insertions(+), 15 deletions(-)automation/gitlab-ci/analyze.yaml +9
-2@@ -20,21 +20,27 @@ codequality: gl-code-quality-report.json needs: [] -eclair-x86_64: +.eclair-analysis:triggered: extends: .eclair-analysis + allow_failure: true + rules: + - if: $CI_PROJECT_PATH =~ /^xen-project\/people\/.*$/ + when: manual + - when: always + +eclair-x86_64: + extends: .eclair-analysis:triggered variables: LOGFILE: "eclair-x86_64.log" VARIANT: "X86_64" RULESET: "Set1" - allow_failure: true eclair-ARM64:automation/gitlab-ci/analyze.yaml line 46
Quoted 10 lines
automation/gitlab-ci/analyze.yaml +1
-2- extends: .eclair-analysis + extends: .eclair-analysis:triggered variables: LOGFILE: "eclair-ARM64.log" VARIANT: "ARM64" RULESET: "Set1" - allow_failure: true .eclair-analysis:on-schedule: extends: .eclair-analysisEverything so far looks great and I am ready to Ack.
Quoted 40 lines
automation/gitlab-ci/build.yaml +0
-6@@ -12,11 +12,6 @@ - '*/*.log' when: always needs: [] - except: - - master - - smoke - - /^coverity-tested\/.*/ - - /^stable-.*/ .gcc-tmpl: variables: &gcc@@ -269,7 +264,6 @@ .test-jobs-artifact-common: stage: build needs: [] - except: !reference [.test-jobs-common, except] # Arm test artifactsautomation/gitlab-ci/test.yaml +0
-5@@ -1,11 +1,6 @@ .test-jobs-common: stage: test image: registry.gitlab.com/xen-project/xen/${CONTAINER} - except: - - master - - smoke - - /^coverity-tested\/.*/ - - /^stable-.*/
Quoted 7 lines
Quoted 4 lines
automation/gitlab-ci/test.yaml line 6
Quoted 2 lines
automation/gitlab-ci/test.yaml +0
-0.arm64-test-needs: &arm64-test-needs - alpine-3.18-arm64-rootfs-exportThese changes instead belongs to the first patch, right?
Yes, sorry. Moreover, it is the answer to your question on the other patch.
No problem. Please re-send. One more comment. In the first patch we have:
+workflow:
+ rules:
+ - if: $CI_COMMIT_BRANCH =~ /^(master|smoke|^coverity-tested\/.*|stable-.*)$/
+ when: never
+ - when: always
I think it is an excellent idea to skip "master" as well as the others, but maybe we could make them "manual" instead of "never" just in case someone wants to see the results on master. It might happen on special occasions.
Re: [XEN PATCH 2/2] automation/eclair: avoid unintentional ECLAIR analysis
Quoted 117 lines · Stefano Stabellini
Quoted 99 lines · Simone Ballarin
Quoted 98 lines · Stefano Stabellini
Quoted 43 lines · Simone Ballarin
With this patch, ECLAIR jobs will need to be manually started for "people/.*" pipelines.
This avoids occupying the runner on analyzes that might not be used by developers.
If developers want to analyze their own repositories they need to launch them from GitLab.
Signed-off-by Simone Ballarin <[email protected]>--- automation/gitlab-ci/analyze.yaml | 14 ++++++++++---- automation/gitlab-ci/build.yaml | 6 ------ automation/gitlab-ci/test.yaml | 5 ----- 3 files changed, 10 insertions(+), 15 deletions(-)automation/gitlab-ci/analyze.yaml +9
-2@@ -20,21 +20,27 @@ codequality: gl-code-quality-report.json needs: [] -eclair-x86_64: +.eclair-analysis:triggered: extends: .eclair-analysis + allow_failure: true + rules: + - if: $CI_PROJECT_PATH =~ /^xen-project\/people\/.*$/ + when: manual + - when: always + +eclair-x86_64: + extends: .eclair-analysis:triggered variables: LOGFILE: "eclair-x86_64.log" VARIANT: "X86_64" RULESET: "Set1" - allow_failure: true eclair-ARM64:automation/gitlab-ci/analyze.yaml line 46
Quoted 10 lines
automation/gitlab-ci/analyze.yaml +1
-2- extends: .eclair-analysis + extends: .eclair-analysis:triggered variables: LOGFILE: "eclair-ARM64.log" VARIANT: "ARM64" RULESET: "Set1" - allow_failure: true .eclair-analysis:on-schedule: extends: .eclair-analysisEverything so far looks great and I am ready to Ack.
Quoted 40 lines
automation/gitlab-ci/build.yaml +0
-6@@ -12,11 +12,6 @@ - '*/*.log' when: always needs: [] - except: - - master - - smoke - - /^coverity-tested\/.*/ - - /^stable-.*/ .gcc-tmpl: variables: &gcc@@ -269,7 +264,6 @@ .test-jobs-artifact-common: stage: build needs: [] - except: !reference [.test-jobs-common, except] # Arm test artifactsautomation/gitlab-ci/test.yaml +0
-5@@ -1,11 +1,6 @@ .test-jobs-common: stage: test image: registry.gitlab.com/xen-project/xen/${CONTAINER} - except: - - master - - smoke - - /^coverity-tested\/.*/ - - /^stable-.*/Quoted 7 lines
Quoted 4 lines
automation/gitlab-ci/test.yaml line 6
Quoted 2 lines
automation/gitlab-ci/test.yaml +0
-0.arm64-test-needs: &arm64-test-needs - alpine-3.18-arm64-rootfs-exportThese changes instead belongs to the first patch, right?
Yes, sorry. Moreover, it is the answer to your question on the other patch.
No problem. Please re-send. One more comment. In the first patch we have:
(quoted diff) +5
-0+workflow: + rules: + - if: $CI_COMMIT_BRANCH =~ /^(master|smoke|^coverity-tested\/.*|stable-.*)$/ + when: never + - when: always
Quoted 4 lines
I think it is an excellent idea to skip "master" as well as the others, but maybe we could make them "manual" instead of "never" just in case someone wants to see the results on master. It might happen on special occasions.
One more thing: today the Eclair jobs always start even if WTOKEN is not defined (hence the jobs cannot actually succeed). Would it make sense to also add a check on the existence of WTOKEN? If WTOKEN exists, start the Eclair jobs, otherwise do not ?
Re: [XEN PATCH 2/2] automation/eclair: avoid unintentional ECLAIR analysis
Quoted 119 lines · Stefano Stabellini
Quoted 118 lines · Stefano Stabellini
Quoted 99 lines · Simone Ballarin
Quoted 98 lines · Stefano Stabellini
Quoted 43 lines · Simone Ballarin
With this patch, ECLAIR jobs will need to be manually started for "people/.*" pipelines.
This avoids occupying the runner on analyzes that might not be used by developers.
If developers want to analyze their own repositories they need to launch them from GitLab.
Signed-off-by Simone Ballarin <[email protected]>--- automation/gitlab-ci/analyze.yaml | 14 ++++++++++---- automation/gitlab-ci/build.yaml | 6 ------ automation/gitlab-ci/test.yaml | 5 ----- 3 files changed, 10 insertions(+), 15 deletions(-)automation/gitlab-ci/analyze.yaml +9
-2@@ -20,21 +20,27 @@ codequality: gl-code-quality-report.json needs: [] -eclair-x86_64: +.eclair-analysis:triggered: extends: .eclair-analysis + allow_failure: true + rules: + - if: $CI_PROJECT_PATH =~ /^xen-project\/people\/.*$/ + when: manual + - when: always + +eclair-x86_64: + extends: .eclair-analysis:triggered variables: LOGFILE: "eclair-x86_64.log" VARIANT: "X86_64" RULESET: "Set1" - allow_failure: true eclair-ARM64:automation/gitlab-ci/analyze.yaml line 46
Quoted 10 lines
automation/gitlab-ci/analyze.yaml +1
-2- extends: .eclair-analysis + extends: .eclair-analysis:triggered variables: LOGFILE: "eclair-ARM64.log" VARIANT: "ARM64" RULESET: "Set1" - allow_failure: true .eclair-analysis:on-schedule: extends: .eclair-analysisEverything so far looks great and I am ready to Ack.
Quoted 40 lines
automation/gitlab-ci/build.yaml +0
-6@@ -12,11 +12,6 @@ - '*/*.log' when: always needs: [] - except: - - master - - smoke - - /^coverity-tested\/.*/ - - /^stable-.*/ .gcc-tmpl: variables: &gcc@@ -269,7 +264,6 @@ .test-jobs-artifact-common: stage: build needs: [] - except: !reference [.test-jobs-common, except] # Arm test artifactsautomation/gitlab-ci/test.yaml +0
-5@@ -1,11 +1,6 @@ .test-jobs-common: stage: test image: registry.gitlab.com/xen-project/xen/${CONTAINER} - except: - - master - - smoke - - /^coverity-tested\/.*/ - - /^stable-.*/Quoted 7 lines
Quoted 4 lines
automation/gitlab-ci/test.yaml line 6
Quoted 2 lines
automation/gitlab-ci/test.yaml +0
-0.arm64-test-needs: &arm64-test-needs - alpine-3.18-arm64-rootfs-exportThese changes instead belongs to the first patch, right?
Yes, sorry. Moreover, it is the answer to your question on the other patch.
No problem. Please re-send. One more comment. In the first patch we have:
(quoted diff) +3
-0+workflow: + rules: + - if: $CI_COMMIT_BRANCH =~/^(master|smoke|^coverity-tested\/.*|stable-.*)$/
(quoted diff) +2
-0+ when: never + - when: always
Quoted 4 lines
Quoted 4 lines
I think it is an excellent idea to skip "master" as well as the others, but maybe we could make them "manual" instead of "never" just in case someone wants to see the results on master. It might happen on special occasions.
"when:manual" cannot be used at workflow-level, but only at job-level. I've made a couple of experiments and I found several problems in doing that related to the clauses "allow_failure" and "needs".
Quoted 5 lines
One more thing: today the Eclair jobs always start even if WTOKEN is not defined (hence the jobs cannot actually succeed). Would it make sense to also add a check on the existence of WTOKEN? If WTOKEN exists, start the Eclair jobs, otherwise do not ?
No problem.
Signature · 3 lines
Simone Ballarin, M.Sc. Field Application Engineer, BUGSENG (https://bugseng.com)
Re: [XEN PATCH 2/2] automation/eclair: avoid unintentional ECLAIR analysis
In reply to Stefano StabelliniQuoted 118 lines · Stefano Stabellini
Quoted 99 lines · Simone Ballarin
Quoted 98 lines · Stefano Stabellini
Quoted 43 lines · Simone Ballarin
With this patch, ECLAIR jobs will need to be manually started for "people/.*" pipelines.
This avoids occupying the runner on analyzes that might not be used by developers.
If developers want to analyze their own repositories they need to launch them from GitLab.
Signed-off-by Simone Ballarin <[email protected]>--- automation/gitlab-ci/analyze.yaml | 14 ++++++++++---- automation/gitlab-ci/build.yaml | 6 ------ automation/gitlab-ci/test.yaml | 5 ----- 3 files changed, 10 insertions(+), 15 deletions(-)automation/gitlab-ci/analyze.yaml +9
-2@@ -20,21 +20,27 @@ codequality: gl-code-quality-report.json needs: [] -eclair-x86_64: +.eclair-analysis:triggered: extends: .eclair-analysis + allow_failure: true + rules: + - if: $CI_PROJECT_PATH =~ /^xen-project\/people\/.*$/ + when: manual + - when: always + +eclair-x86_64: + extends: .eclair-analysis:triggered variables: LOGFILE: "eclair-x86_64.log" VARIANT: "X86_64" RULESET: "Set1" - allow_failure: true eclair-ARM64:automation/gitlab-ci/analyze.yaml line 46
Quoted 10 lines
automation/gitlab-ci/analyze.yaml +1
-2- extends: .eclair-analysis + extends: .eclair-analysis:triggered variables: LOGFILE: "eclair-ARM64.log" VARIANT: "ARM64" RULESET: "Set1" - allow_failure: true .eclair-analysis:on-schedule: extends: .eclair-analysisEverything so far looks great and I am ready to Ack.
Quoted 40 lines
automation/gitlab-ci/build.yaml +0
-6@@ -12,11 +12,6 @@ - '*/*.log' when: always needs: [] - except: - - master - - smoke - - /^coverity-tested\/.*/ - - /^stable-.*/ .gcc-tmpl: variables: &gcc@@ -269,7 +264,6 @@ .test-jobs-artifact-common: stage: build needs: [] - except: !reference [.test-jobs-common, except] # Arm test artifactsautomation/gitlab-ci/test.yaml +0
-5@@ -1,11 +1,6 @@ .test-jobs-common: stage: test image: registry.gitlab.com/xen-project/xen/${CONTAINER} - except: - - master - - smoke - - /^coverity-tested\/.*/ - - /^stable-.*/Quoted 7 lines
Quoted 4 lines
automation/gitlab-ci/test.yaml line 6
Quoted 2 lines
automation/gitlab-ci/test.yaml +0
-0.arm64-test-needs: &arm64-test-needs - alpine-3.18-arm64-rootfs-exportThese changes instead belongs to the first patch, right?
Yes, sorry. Moreover, it is the answer to your question on the other patch.
No problem. Please re-send. One more comment. In the first patch we have:
(quoted diff) +3
-0+workflow: + rules: + - if: $CI_COMMIT_BRANCH =~/^(master|smoke|^coverity-tested\/.*|stable-.*)$/
(quoted diff) +2
-0+ when: never + - when: always
Quoted 4 lines
I think it is an excellent idea to skip "master" as well as the others, but maybe we could make them "manual" instead of "never" just in case someone wants to see the results on master. It might happen on special occasions.
"manual" cannot be used at workflow-level only at job-level. I've made a couple of experiments and I find several problems in doing that:
Signature · 3 lines
Simone Ballarin, M.Sc. Field Application Engineer, BUGSENG (https://bugseng.com)