Skip to contentSkip to conversation

automation/eclair: do not allow failure for triggered analyses

PATCH In review Federico Serafini 4 participants 13 messages Apr 29, 2024 – May 6, 2024

lore
Share & tools
lore
#3 lore raw

PATCH 2/3 automation: do not allow failure for triggered analyses

In reply to Federico Serafini

Do not allow_failure for triggered analyses:
introducing regressions of clean guidelines will cause a CI failure.

---
 automation/gitlab-ci/analyze.yaml | 1 -
 1 file changed, 1 deletion(-)
automation/gitlab-ci/analyze.yaml +0 -1
index 46c9d8e2e5..32bf570149 100644
@@ -26,7 +26,6 @@
 
 .eclair-analysis:triggered:
   extends: .eclair-analysis
-  allow_failure: true
   rules:
     - if: $CI_PIPELINE_SOURCE == "schedule"
       when: never
Signature · 1 line
2.34.1
#5 lore raw

PATCH 3/3 automation/eclair: make explicit there are no adopted files by default

In reply to Federico Serafini

Update ECLAIR configuration to consider no adopted files by default.

---
 automation/eclair_analysis/ECLAIR/analysis.ecl | 4 ++++
 1 file changed, 4 insertions(+)
automation/eclair_analysis/ECLAIR/analysis.ecl +4 -0
index a604582da3..66ed7f952c 100644
@@ -21,6 +21,10 @@ map_strings("scheduled-analysis",analysis_kind)
 
 -eval_file=toolchain.ecl
 -eval_file=public_APIs.ecl
+
+-doc="Initially, there are no files tagged as adopted."
+-file_tag+={adopted,"none()"}
+
 if(not(scheduled_analysis),
     eval_file("adopted.ecl")
 )
Signature · 1 line
2.34.1
#0 lore raw

cover automation/eclair: do not allow failure for triggered analyses

Patch 1/3 does some preparation work.

Patch 2/3, as the title says, removes allow_failure = true for triggered analyses.

Patch 3/3 makes explicit that initally no files are tagged as adopted, this is needed by the scheduled analysis.

Federico Serafini (3):
automation/eclair: tag Rule 7.2 as clean and temporarily remove Rules

    1.1 and 8.2

automation: do not allow failure for triggered analyses automation/eclair: make explicit there are no adopted files by default

 automation/eclair_analysis/ECLAIR/analysis.ecl | 4 ++++
 automation/eclair_analysis/ECLAIR/tagging.ecl  | 2 +-
 automation/gitlab-ci/analyze.yaml              | 1 -
 3 files changed, 5 insertions(+), 2 deletions(-)
Signature · 1 line
2.34.1
#1 lore raw

PATCH 1/3 automation/eclair: tag Rule 7.2 as clean and temporarily remove Rules 1.1 and 8.2

Update ECLAIR configuration to consider Rule 7.2 as clean.

Temporarily remove the clean tag from Rules 1.1 and 8.2:
when violations of such rules will be addressed, the clean tag will be reintroduced.

---
 automation/eclair_analysis/ECLAIR/tagging.ecl | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
automation/eclair_analysis/ECLAIR/tagging.ecl +1 -1
index d609b470eb..bdf94ed996 100644
@@ -19,7 +19,7 @@
 
 -doc_begin="Clean guidelines: new violations for these guidelines are not accepted."
 
--service_selector={clean_guidelines_common,"MC3R1.D1.1||MC3R1.D2.1||MC3R1.D4.1||MC3R1.D4.11||MC3R1.D4.14||MC3R1.R1.1||MC3R1.R11.7||MC3R1.R11.9||MC3R1.R12.5||MC3R1.R1.3||MC3R1.R1.4||MC3R1.R14.1||MC3R1.R16.7||MC3R1.R17.1||MC3R1.R17.3||MC3R1.R17.4||MC3R1.R17.5||MC3R1.R17.6||MC3R1.R20.13||MC3R1.R20.14||MC3R1.R20.4||MC3R1.R20.9||MC3R1.R21.13||MC3R1.R21.19||MC3R1.R21.21||MC3R1.R2.2||MC3R1.R22.2||MC3R1.R22.4||MC3R1.R22.5||MC3R1.R22.6||MC3R1.R2.6||MC3R1.R3.1||MC3R1.R3.2||MC3R1.R4.1||MC3R1.R4.2||MC3R1.R5.1||MC3R1.R5.2||MC3R1.R5.4||MC3R1.R5.6||MC3R1.R6.1||MC3R1.R6.2||MC3R1.R7.1||MC3R1.R7.4||MC3R1.R8.1||MC3R1.R8.10||MC3R1.R8.12||MC3R1.R8.14||MC3R1.R8.2||MC3R1.R8.5||MC3R1.R8.6||MC3R1.R8.8||MC3R1.R9.2||MC3R1.R9.3||MC3R1.R9.4||MC3R1.R9.5"
+-service_selector={clean_guidelines_common,"MC3R1.D1.1||MC3R1.D2.1||MC3R1.D4.1||MC3R1.D4.11||MC3R1.D4.14||MC3R1.R11.7||MC3R1.R11.9||MC3R1.R12.5||MC3R1.R1.3||MC3R1.R1.4||MC3R1.R14.1||MC3R1.R16.7||MC3R1.R17.1||MC3R1.R17.3||MC3R1.R17.4||MC3R1.R17.5||MC3R1.R17.6||MC3R1.R20.13||MC3R1.R20.14||MC3R1.R20.4||MC3R1.R20.9||MC3R1.R21.13||MC3R1.R21.19||MC3R1.R21.21||MC3R1.R2.2||MC3R1.R22.2||MC3R1.R22.4||MC3R1.R22.5||MC3R1.R22.6||MC3R1.R2.6||MC3R1.R3.1||MC3R1.R3.2||MC3R1.R4.1||MC3R1.R4.2||MC3R1.R5.1||MC3R1.R5.2||MC3R1.R5.4||MC3R1.R5.6||MC3R1.R6.1||MC3R1.R6.2||MC3R1.R7.1||MC3R1.R7.2||MC3R1.R7.4||MC3R1.R8.1||MC3R1.R8.10||MC3R1.R8.12||MC3R1.R8.14||MC3R1.R8.5||MC3R1.R8.6||MC3R1.R8.8||MC3R1.R9.2||MC3R1.R9.3||MC3R1.R9.4||MC3R1.R9.5"
 }
 
 -setq=target,getenv("XEN_TARGET_ARCH")
Signature · 1 line
2.34.1
Acked-by on v1 1/3
#2 lore raw

Re: [XEN PATCH 1/3] automation/eclair: tag Rule 7.2 as clean and temporarily remove Rules 1.1 and 8.2

Quoted 7 lines · Federico Serafini

Update ECLAIR configuration to consider Rule 7.2 as clean.

Temporarily remove the clean tag from Rules 1.1 and 8.2:
when violations of such rules will be addressed, the clean tag will be reintroduced.

Quoted 20 lines
---
 automation/eclair_analysis/ECLAIR/tagging.ecl | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
automation/eclair_analysis/ECLAIR/tagging.ecl +1 -1
index d609b470eb..bdf94ed996 100644
@@ -19,7 +19,7 @@
 
 -doc_begin="Clean guidelines: new violations for these guidelines are not accepted."
 
--service_selector={clean_guidelines_common,"MC3R1.D1.1||MC3R1.D2.1||MC3R1.D4.1||MC3R1.D4.11||MC3R1.D4.14||MC3R1.R1.1||MC3R1.R11.7||MC3R1.R11.9||MC3R1.R12.5||MC3R1.R1.3||MC3R1.R1.4||MC3R1.R14.1||MC3R1.R16.7||MC3R1.R17.1||MC3R1.R17.3||MC3R1.R17.4||MC3R1.R17.5||MC3R1.R17.6||MC3R1.R20.13||MC3R1.R20.14||MC3R1.R20.4||MC3R1.R20.9||MC3R1.R21.13||MC3R1.R21.19||MC3R1.R21.21||MC3R1.R2.2||MC3R1.R22.2||MC3R1.R22.4||MC3R1.R22.5||MC3R1.R22.6||MC3R1.R2.6||MC3R1.R3.1||MC3R1.R3.2||MC3R1.R4.1||MC3R1.R4.2||MC3R1.R5.1||MC3R1.R5.2||MC3R1.R5.4||MC3R1.R5.6||MC3R1.R6.1||MC3R1.R6.2||MC3R1.R7.1||MC3R1.R7.4||MC3R1.R8.1||MC3R1.R8.10||MC3R1.R8.12||MC3R1.R8.14||MC3R1.R8.2||MC3R1.R8.5||MC3R1.R8.6||MC3R1.R8.8||MC3R1.R9.2||MC3R1.R9.3||MC3R1.R9.4||MC3R1.R9.5"
+-service_selector={clean_guidelines_common,"MC3R1.D1.1||MC3R1.D2.1||MC3R1.D4.1||MC3R1.D4.11||MC3R1.D4.14||MC3R1.R11.7||MC3R1.R11.9||MC3R1.R12.5||MC3R1.R1.3||MC3R1.R1.4||MC3R1.R14.1||MC3R1.R16.7||MC3R1.R17.1||MC3R1.R17.3||MC3R1.R17.4||MC3R1.R17.5||MC3R1.R17.6||MC3R1.R20.13||MC3R1.R20.14||MC3R1.R20.4||MC3R1.R20.9||MC3R1.R21.13||MC3R1.R21.19||MC3R1.R21.21||MC3R1.R2.2||MC3R1.R22.2||MC3R1.R22.4||MC3R1.R22.5||MC3R1.R22.6||MC3R1.R2.6||MC3R1.R3.1||MC3R1.R3.2||MC3R1.R4.1||MC3R1.R4.2||MC3R1.R5.1||MC3R1.R5.2||MC3R1.R5.4||MC3R1.R5.6||MC3R1.R6.1||MC3R1.R6.2||MC3R1.R7.1||MC3R1.R7.2||MC3R1.R7.4||MC3R1.R8.1||MC3R1.R8.10||MC3R1.R8.12||MC3R1.R8.14||MC3R1.R8.5||MC3R1.R8.6||MC3R1.R8.8||MC3R1.R9.2||MC3R1.R9.3||MC3R1.R9.4||MC3R1.R9.5"
 }
 
 -setq=target,getenv("XEN_TARGET_ARCH")
Signature · 1 line
2.34.1
Acked-by on v1 2/3
#4 lore raw

Re: [XEN PATCH 2/3] automation: do not allow failure for triggered analyses

In reply to Federico Serafini
Quoted 4 lines · Federico Serafini

Do not allow_failure for triggered analyses:
introducing regressions of clean guidelines will cause a CI failure.

Quoted 19 lines
---
 automation/gitlab-ci/analyze.yaml | 1 -
 1 file changed, 1 deletion(-)
automation/gitlab-ci/analyze.yaml +0 -1
index 46c9d8e2e5..32bf570149 100644
@@ -26,7 +26,6 @@
 
 .eclair-analysis:triggered:
   extends: .eclair-analysis
-  allow_failure: true
   rules:
     - if: $CI_PIPELINE_SOURCE == "schedule"
       when: never
Signature · 1 line
2.34.1
Acked-by on v1 3/3
#6 lore raw

Re: [XEN PATCH 3/3] automation/eclair: make explicit there are no adopted files by default

In reply to Federico Serafini
Quoted 3 lines · Federico Serafini

Update ECLAIR configuration to consider no adopted files by default.

Quoted 22 lines
---
 automation/eclair_analysis/ECLAIR/analysis.ecl | 4 ++++
 1 file changed, 4 insertions(+)
automation/eclair_analysis/ECLAIR/analysis.ecl +4 -0
index a604582da3..66ed7f952c 100644
@@ -21,6 +21,10 @@ map_strings("scheduled-analysis",analysis_kind)
 
 -eval_file=toolchain.ecl
 -eval_file=public_APIs.ecl
+
+-doc="Initially, there are no files tagged as adopted."
+-file_tag+={adopted,"none()"}
+
 if(not(scheduled_analysis),
     eval_file("adopted.ecl")
 )
Signature · 1 line
2.34.1
#7 lore raw
In reply to Federico Serafini
Quoted 7 lines · Federico Serafini

Patch 1/3 does some preparation work.

Patch 2/3, as the title says, removes allow_failure = true for triggered analyses.

Patch 3/3 makes explicit that initally no files are tagged as adopted, this is needed by the scheduled analysis.

I'm afraid that something in this series is broken.

Since these patches went in, all pipelines are now getting a status of blocked rather than passed.

If I manually start the Eclair jobs, then eventually the pipeline gets to Passed.

~Andrew

#8 lore raw
Quoted 9 lines · Andrew Cooper
Quoted 7 lines · Federico Serafini

Patch 1/3 does some preparation work.

Patch 2/3, as the title says, removes allow_failure = true for triggered analyses.

Patch 3/3 makes explicit that initally no files are tagged as adopted, this is needed by the scheduled analysis.

Quoted 7 lines

I'm afraid that something in this series is broken.

Since these patches went in, all pipelines are now getting a status of blocked rather than passed.

If I manually start the Eclair jobs, then eventually the pipeline gets to Passed.

Can you provide us a link to those failures?
I am looking at gitlab xen-project/xen and xen-project/patchew and everything seems ok.

Signature · 3 lines
Federico Serafini, M.Sc.

Software Engineer, BUGSENG (http://bugseng.com)
#9 lore raw
Quoted 17 lines · Federico Serafini
Quoted 16 lines · Andrew Cooper
Quoted 9 lines · Federico Serafini

Patch 1/3 does some preparation work.

Patch 2/3, as the title says, removes allow_failure = true for triggered
analyses.

Patch 3/3 makes explicit that initally no files are tagged as adopted, this
is needed by the scheduled analysis.

I'm afraid that something in this series is broken.

Since these patches went in, all pipelines are now getting a status of blocked rather than passed.

Quoted 7 lines
Quoted 2 lines

If I manually start the Eclair jobs, then eventually the pipeline gets to Passed.

Can you provide us a link to those failures?
I am looking at gitlab xen-project/xen and xen-project/patchew and everything seems ok.

https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines/1276081658
is the first one I noticed as blocked, and I manually ran.  That ended up working.

https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines/1277724847
is still in the blocked state.  The build-each-commit failure is unrelated.

~Andrew

#10 lore raw
Quoted 26 lines · Andrew Cooper
Quoted 17 lines · Federico Serafini
Quoted 16 lines · Andrew Cooper
Quoted 9 lines · Federico Serafini

Patch 1/3 does some preparation work.

Patch 2/3, as the title says, removes allow_failure = true for triggered
analyses.

Patch 3/3 makes explicit that initally no files are tagged as adopted, this
is needed by the scheduled analysis.

I'm afraid that something in this series is broken.

Since these patches went in, all pipelines are now getting a status of blocked rather than passed.

Quoted 7 lines
Quoted 2 lines

If I manually start the Eclair jobs, then eventually the pipeline gets to Passed.

Can you provide us a link to those failures?
I am looking at gitlab xen-project/xen and xen-project/patchew and everything seems ok.

Quoted 6 lines

https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines/1276081658
is the first one I noticed as blocked, and I manually ran.  That ended up working.

https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines/1277724847
is still in the blocked state.  The build-each-commit failure is unrelated.

This is intentional and was introduced by
commit 7c1bf8661db5c00bd8c9a25015fe8678b2ff9ac6

The ECLAIR analysis under people/* need to be activated manually.

Is this causing some problems to the CI?

Signature · 3 lines
Federico Serafini, M.Sc.

Software Engineer, BUGSENG (http://bugseng.com)
#11 lore raw
Quoted 35 lines · Federico Serafini
Quoted 26 lines · Andrew Cooper
Quoted 17 lines · Federico Serafini
Quoted 16 lines · Andrew Cooper
Quoted 9 lines · Federico Serafini

Patch 1/3 does some preparation work.

Patch 2/3, as the title says, removes allow_failure = true for triggered
analyses.

Patch 3/3 makes explicit that initally no files are tagged as adopted, this
is needed by the scheduled analysis.

I'm afraid that something in this series is broken.

Since these patches went in, all pipelines are now getting a status of blocked rather than passed.

Quoted 7 lines
Quoted 2 lines

If I manually start the Eclair jobs, then eventually the pipeline gets to Passed.

Can you provide us a link to those failures?
I am looking at gitlab xen-project/xen and xen-project/patchew and everything seems ok.

Quoted 7 lines

https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines/1276081658
is the first one I noticed as blocked, and I manually ran.  That ended up working.

https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines/1277724847
is still in the blocked state.  The build-each-commit failure is unrelated.

Quoted 5 lines

This is intentional and was introduced by
commit 7c1bf8661db5c00bd8c9a25015fe8678b2ff9ac6

The ECLAIR analysis under people/* need to be activated manually.

Yes.  I know, and that matches the behaviour I saw.

Quoted 3 lines

Is this causing some problems to the CI?

Yes.

See https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines

Prior to this series, the manual actions were not used but the pipeline was overall in the Passed state.  Specifically, they ended up being skipped.

After this series, the manual actions are now blocking the pipeline, not letting it complete, and not marking it as passed.

~Andrew

#12 lore raw
Quoted 7 lines · Andrew Cooper
Quoted 6 lines · Federico Serafini
Quoted 5 lines · Andrew Cooper
Quoted 4 lines · Federico Serafini
Quoted 3 lines · Andrew Cooper
Quoted 2 lines · Federico Serafini

Patch 1/3 does some preparation work.

Hi,

Quoted 43 lines
Quoted 29 lines
Quoted 21 lines
Quoted 13 lines
Quoted 13 lines
Quoted 7 lines

Patch 2/3, as the title says, removes allow_failure = true for triggered
analyses.

Patch 3/3 makes explicit that initally no files are tagged as adopted, this
is needed by the scheduled analysis.

I'm afraid that something in this series is broken.

Since these patches went in, all pipelines are now getting a status of blocked rather than passed.

Quoted 7 lines
Quoted 2 lines

If I manually start the Eclair jobs, then eventually the pipeline gets to Passed.

Can you provide us a link to those failures?
I am looking at gitlab xen-project/xen and xen-project/patchew and everything seems ok.

Quoted 7 lines

https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines/1276081658
is the first one I noticed as blocked, and I manually ran.  That ended up working.

https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines/1277724847
is still in the blocked state.  The build-each-commit failure is unrelated.

Quoted 5 lines

This is intentional and was introduced by
commit 7c1bf8661db5c00bd8c9a25015fe8678b2ff9ac6

The ECLAIR analysis under people/* need to be activated manually.

Yes.  I know, and that matches the behaviour I saw.

Quoted 3 lines

Is this causing some problems to the CI?

Yes.

Quoted 7 lines

See https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines

Prior to this series, the manual actions were not used but the pipeline was overall in the Passed state.  Specifically, they ended up being skipped.

After this series, the manual actions are now blocking the pipeline, not letting it complete, and not marking it as passed.

one way to fix this can be to set allow_failure when in the people/* branches, which will cause the stage to be skipped and thus mark the pipeline as Passed when all other steps have completed. See https://gitlab.com/xen-project/people/bugseng/xen/-/pipelines/1278550868

Signature · 2 lines
Nicola Vetrini, BSc
Software Engineer, BUGSENG srl (https://bugseng.com)