Skip to contentSkip to conversation

automation: avoid unnecessary analyses

PATCH In review Simone Ballarin 2 participants 5 messages Aug 21, 2023 – Aug 22, 2023

lore
Share & tools
lore
#0 lore raw

v2 cover automation: avoid unnecessary analyses

This series aims to reduce the analyses performed by the ECLAIR runner by avoiding some branches already excluded by other jobs and requiring analyses on people/.* to be manually triggered.

---
Changes in v2:
- move some changes from 1/2 to 2/2.
- remove useless except clause in .yocto-test
- avoid ECLAIR jobs if the WTOKEN variable is not defined

Simone Ballarin (2):
automation/eclair: avoid unintentional ECLAIR analysis
automation: avoid pipelines on specific branches

 .gitlab-ci.yml                    |  6 ++++++
 automation/gitlab-ci/analyze.yaml | 22 +++++++++++++++++-----
 automation/gitlab-ci/build.yaml   | 11 -----------
 automation/gitlab-ci/test.yaml    |  5 -----
 automation/scripts/eclair         |  5 -----
 5 files changed, 23 insertions(+), 26 deletions(-)
Signature · 1 line
2.34.1
#1 lore raw

PATCH v2 1/2 automation/eclair: avoid unintentional ECLAIR analysis

With this patch, ECLAIR jobs will need to be manually started for "people/.*" pipelines and will not be triggered if the WTOKEN variable is missing.

This avoids occupying the runner on analyzes that might not be used by developers.

If developers want to analyze their own repositories they need to launch them from GitLab.

---
Changes in v2:
- avoid ECLAIR jobs if the WTOKEN variable is not defined.

---
 automation/gitlab-ci/analyze.yaml | 22 +++++++++++++++++-----
 automation/scripts/eclair         |  5 -----
 2 files changed, 17 insertions(+), 10 deletions(-)
automation/gitlab-ci/analyze.yaml +17 -5
index 4aa4abe2ee..bd9a68de31 100644
@@ -18,28 +18,40 @@
       - '*.log'
     reports:
       codequality: gl-code-quality-report.json
+  rules:
+    - if: $WTOKEN == null
+      when: never
+    - when: always
   needs: []
 
-eclair-x86_64:
+.eclair-analysis:triggered:
   extends: .eclair-analysis
+  allow_failure: true
+  rules:
+    - if: $WTOKEN && $CI_PROJECT_PATH =~ /^xen-project\/people\/.*$/
+      when: manual
+    - !reference [.eclair-analysis, rules]
+
+eclair-x86_64:
+  extends: .eclair-analysis:triggered
   variables:
     LOGFILE: "eclair-x86_64.log"
     VARIANT: "X86_64"
     RULESET: "Set1"
-  allow_failure: true
 
 eclair-ARM64:
-  extends: .eclair-analysis
+  extends: .eclair-analysis:triggered
   variables:
     LOGFILE: "eclair-ARM64.log"
     VARIANT: "ARM64"
     RULESET: "Set1"
-  allow_failure: true
 
 .eclair-analysis:on-schedule:
   extends: .eclair-analysis
   rules:
-    - if: $CI_PIPELINE_SOURCE == "schedule"
+    - if: $CI_PIPELINE_SOURCE != "schedule"
+      when: never
+    - !reference [.eclair-analysis, rules]
 
 eclair-x86_64-Set1:on-schedule:
   extends: .eclair-analysis:on-schedule
automation/scripts/eclair +0 -5
index 813a56eb6a..14e47a6f97 100755
@@ -4,11 +4,6 @@ ECLAIR_ANALYSIS_DIR=automation/eclair_analysis
 ECLAIR_DIR="${ECLAIR_ANALYSIS_DIR}/ECLAIR"
 ECLAIR_OUTPUT_DIR=$(realpath "${ECLAIR_OUTPUT_DIR}")
 
-if [ -z "${WTOKEN:-}" ]; then
-    echo "Failure: the WTOKEN variable is not defined." >&2
-    exit 1
-fi
-
 "${ECLAIR_ANALYSIS_DIR}/prepare.sh" "${VARIANT}"
 
 ex=0
Signature · 1 line
2.34.1
#3 lore raw

PATCH v2 2/2 automation: avoid pipelines on specific branches

In reply to Simone Ballarin

This patch avoids the execution of pipelines in the following branches:
- master
- smoke
- coverirty-tested/.*
- stable-.*

The job-level exclusions have been removed as they are pointless with this new workspace-level exclusion.

---
Changes in v2:
- remove useless except clause in .yocto-test.

---
 .gitlab-ci.yml                  |  6 ++++++
 automation/gitlab-ci/build.yaml | 11 -----------
 automation/gitlab-ci/test.yaml  |  5 -----
 3 files changed, 6 insertions(+), 16 deletions(-)
.gitlab-ci.yml +6 -0
index ee5430b8b7..ef4484e09a 100644
@@ -1,3 +1,9 @@
+workflow:
+  rules:
+    - if: $CI_COMMIT_BRANCH =~ /^(master|smoke|^coverity-tested\/.*|stable-.*)$/
+      when: never
+    - when: always
+
 stages:
   - analyze
   - build
automation/gitlab-ci/build.yaml +0 -11
index 1a4a5e490d..b633facff4 100644
@@ -12,11 +12,6 @@
       - '*/*.log'
     when: always
   needs: []
-  except:
-    - master
-    - smoke
-    - /^coverity-tested\/.*/
-    - /^stable-.*/
 
 .gcc-tmpl:
   variables: &gcc
@@ -214,11 +209,6 @@
 .yocto-test:
   stage: build
   image: registry.gitlab.com/xen-project/xen/${CONTAINER}
-  except:
-    - master
-    - smoke
-    - /^coverity-tested\/.*/
-    - /^stable-.*/
   script:
     - ./automation/build/yocto/build-yocto.sh -v --log-dir=./logs --xen-dir=`pwd` ${YOCTO_BOARD} ${YOCTO_OUTPUT}
   variables:
@@ -269,7 +259,6 @@
 .test-jobs-artifact-common:
   stage: build
   needs: []
-  except: !reference [.test-jobs-common, except]
 
 # Arm test artifacts
 
automation/gitlab-ci/test.yaml +0 -5
index 810631bc46..8737f367c8 100644
@@ -1,11 +1,6 @@
 .test-jobs-common:
   stage: test
   image: registry.gitlab.com/xen-project/xen/${CONTAINER}
-  except:
-    - master
-    - smoke
-    - /^coverity-tested\/.*/
-    - /^stable-.*/
 
 .arm64-test-needs: &arm64-test-needs
   - alpine-3.18-arm64-rootfs-export
Signature · 1 line
2.34.1
Reviewed-by on v2 2/2
#4 lore raw

Re: [XEN PATCH v2 2/2] automation: avoid pipelines on specific branches

Quoted 11 lines · Simone Ballarin

This patch avoids the execution of pipelines in the following branches:
- master
- smoke
- coverirty-tested/.*
- stable-.*

The job-level exclusions have been removed as they are pointless with this new workspace-level exclusion.

Quoted 78 lines

---
Changes in v2:
- remove useless except clause in .yocto-test.

---
 .gitlab-ci.yml                  |  6 ++++++
 automation/gitlab-ci/build.yaml | 11 -----------
 automation/gitlab-ci/test.yaml  |  5 -----
 3 files changed, 6 insertions(+), 16 deletions(-)
.gitlab-ci.yml +6 -0
index ee5430b8b7..ef4484e09a 100644
@@ -1,3 +1,9 @@
+workflow:
+  rules:
+    - if: $CI_COMMIT_BRANCH =~ /^(master|smoke|^coverity-tested\/.*|stable-.*)$/
+      when: never
+    - when: always
+
 stages:
   - analyze
   - build
automation/gitlab-ci/build.yaml +0 -11
index 1a4a5e490d..b633facff4 100644
@@ -12,11 +12,6 @@
       - '*/*.log'
     when: always
   needs: []
-  except:
-    - master
-    - smoke
-    - /^coverity-tested\/.*/
-    - /^stable-.*/
 
 .gcc-tmpl:
   variables: &gcc
@@ -214,11 +209,6 @@
 .yocto-test:
   stage: build
   image: registry.gitlab.com/xen-project/xen/${CONTAINER}
-  except:
-    - master
-    - smoke
-    - /^coverity-tested\/.*/
-    - /^stable-.*/
   script:
     - ./automation/build/yocto/build-yocto.sh -v --log-dir=./logs --xen-dir=`pwd` ${YOCTO_BOARD} ${YOCTO_OUTPUT}
   variables:
@@ -269,7 +259,6 @@
 .test-jobs-artifact-common:
   stage: build
   needs: []
-  except: !reference [.test-jobs-common, except]
 
 # Arm test artifacts
 
automation/gitlab-ci/test.yaml +0 -5
index 810631bc46..8737f367c8 100644
@@ -1,11 +1,6 @@
 .test-jobs-common:
   stage: test
   image: registry.gitlab.com/xen-project/xen/${CONTAINER}
-  except:
-    - master
-    - smoke
-    - /^coverity-tested\/.*/
-    - /^stable-.*/
 
 .arm64-test-needs: &arm64-test-needs
   - alpine-3.18-arm64-rootfs-export
Signature · 1 line
2.34.1
Reviewed-by on v2 1/2
#2 lore raw

Re: [XEN PATCH v2 1/2] automation/eclair: avoid unintentional ECLAIR analysis

In reply to Simone Ballarin
Quoted 11 lines · Simone Ballarin

With this patch, ECLAIR jobs will need to be manually started for "people/.*" pipelines and will not be triggered if the WTOKEN variable is missing.

This avoids occupying the runner on analyzes that might not be used by developers.

If developers want to analyze their own repositories they need to launch them from GitLab.

Great job!

Quoted 77 lines

---
Changes in v2:
- avoid ECLAIR jobs if the WTOKEN variable is not defined.

---
 automation/gitlab-ci/analyze.yaml | 22 +++++++++++++++++-----
 automation/scripts/eclair         |  5 -----
 2 files changed, 17 insertions(+), 10 deletions(-)
automation/gitlab-ci/analyze.yaml +17 -5
index 4aa4abe2ee..bd9a68de31 100644
@@ -18,28 +18,40 @@
       - '*.log'
     reports:
       codequality: gl-code-quality-report.json
+  rules:
+    - if: $WTOKEN == null
+      when: never
+    - when: always
   needs: []
 
-eclair-x86_64:
+.eclair-analysis:triggered:
   extends: .eclair-analysis
+  allow_failure: true
+  rules:
+    - if: $WTOKEN && $CI_PROJECT_PATH =~ /^xen-project\/people\/.*$/
+      when: manual
+    - !reference [.eclair-analysis, rules]
+
+eclair-x86_64:
+  extends: .eclair-analysis:triggered
   variables:
     LOGFILE: "eclair-x86_64.log"
     VARIANT: "X86_64"
     RULESET: "Set1"
-  allow_failure: true
 
 eclair-ARM64:
-  extends: .eclair-analysis
+  extends: .eclair-analysis:triggered
   variables:
     LOGFILE: "eclair-ARM64.log"
     VARIANT: "ARM64"
     RULESET: "Set1"
-  allow_failure: true
 
 .eclair-analysis:on-schedule:
   extends: .eclair-analysis
   rules:
-    - if: $CI_PIPELINE_SOURCE == "schedule"
+    - if: $CI_PIPELINE_SOURCE != "schedule"
+      when: never
+    - !reference [.eclair-analysis, rules]
 
 eclair-x86_64-Set1:on-schedule:
   extends: .eclair-analysis:on-schedule
automation/scripts/eclair +0 -5
index 813a56eb6a..14e47a6f97 100755
@@ -4,11 +4,6 @@ ECLAIR_ANALYSIS_DIR=automation/eclair_analysis
 ECLAIR_DIR="${ECLAIR_ANALYSIS_DIR}/ECLAIR"
 ECLAIR_OUTPUT_DIR=$(realpath "${ECLAIR_OUTPUT_DIR}")
 
-if [ -z "${WTOKEN:-}" ]; then
-    echo "Failure: the WTOKEN variable is not defined." >&2
-    exit 1
-fi
-
 "${ECLAIR_ANALYSIS_DIR}/prepare.sh" "${VARIANT}"
 
 ex=0
Signature · 1 line
2.34.1