Skip to contentSkip to conversation

CI: Always move the bisect build log back

PATCH In review Anthony PERARD 2 participants 6 messages Aug 23, 2023 – Aug 24, 2023

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 CI: Always move the bisect build log back

On failure of "build"-each-commit script, the next command that move the log back into the build directory isn't executed. Fix that by using "after_script" which is always executed even if the main "script" fails. (We would still miss the log when the jobs times out.)

---
 automation/gitlab-ci/test.yaml | 1 +
 1 file changed, 1 insertion(+)
automation/gitlab-ci/test.yaml +1 -0
index 810631bc46..5099f2e6b6 100644
@@ -140,6 +140,7 @@ build-each-commit-gcc:
     CC: gcc
   script:
     - BASE=${BASE_SHA:-${CI_COMMIT_BEFORE_SHA}} TIP=${TIP_SHA:-${CI_COMMIT_SHA}} ./automation/gitlab-ci/build-each-commit.sh 2>&1 | tee ../build-each-commit-gcc.log
+  after_script:
     - mv ../build-each-commit-gcc.log .
   artifacts:
     paths:
Signature · 1 line
Anthony PERARD
#1 lore raw
Quoted 15 lines · Anthony PERARD

On failure of "build"-each-commit script, the next command that move the log back into the build directory isn't executed. Fix that by using "after_script" which is always executed even if the main "script" fails. (We would still miss the log when the jobs times out.)

---
 automation/gitlab-ci/test.yaml | 1 +
 1 file changed, 1 insertion(+)
automation/gitlab-ci/test.yaml +0 -0
index 810631bc46..5099f2e6b6 100644

automation/gitlab-ci/test.yaml build-each-commit-gcc: · line 146

Quoted 7 lines
automation/gitlab-ci/test.yaml +1 -0
@@ -140,6 +140,7 @@ build-each-commit-gcc:
     CC: gcc
   script:
     - BASE=${BASE_SHA:-${CI_COMMIT_BEFORE_SHA}} TIP=${TIP_SHA:-${CI_COMMIT_SHA}} ./automation/gitlab-ci/build-each-commit.sh 2>&1 | tee ../build-each-commit-gcc.log
+  after_script:
     - mv ../build-each-commit-gcc.log .
   artifacts:
     paths:

Thanks for looking into this, and yeah that is dumb, but why play games with the parent directory?

$ git diff
automation/gitlab-ci/test.yaml +0 -0
index 810631bc4624..b4c2f22a1b07 100644
@@ -136,11 +136,11 @@ build-each-commit-gcc:

   extends: .test-jobs-common
   variables:
     CONTAINER: debian:stretch
+    LOGFILE: build-each-commit-gcc.log
     XEN_TARGET_ARCH: x86_64
     CC: gcc
   script:
-    - BASE=${BASE_SHA:-${CI_COMMIT_BEFORE_SHA}} TIP=${TIP_SHA:-${CI_COMMIT_SHA}} ./automation/gitlab-ci/build-each-commit.sh 2>&1 | tee ../build-each-commit-gcc.log
-    - mv ../build-each-commit-gcc.log .
+    - BASE=${BASE_SHA:-${CI_COMMIT_BEFORE_SHA}} TIP=${TIP_SHA:-${CI_COMMIT_SHA}} ./automation/gitlab-ci/build-each-commit.sh 2>&1 | tee ${LOGFILE} artifacts:
     paths:
       - '*.log'

This is prevailing style of the other tests, and also e.g. won't emit the (whole) log file if e.g. disk space fills up.

~Andrew

#2 lore raw
Quoted 11 lines · Andrew Cooper
Quoted 10 lines · Anthony PERARD

On failure of "build"-each-commit script, the next command that move the log back into the build directory isn't executed. Fix that by using "after_script" which is always executed even if the main "script" fails. (We would still miss the log when the jobs times out.)

---
 automation/gitlab-ci/test.yaml | 1 +
 1 file changed, 1 insertion(+)
Quoted 15 lines

automation/gitlab-ci/test.yaml build-each-commit-gcc: · line 146

Quoted 12 lines
automation/gitlab-ci/test.yaml +1 -0
index 810631bc46..5099f2e6b6 100644
@@ -140,6 +140,7 @@ build-each-commit-gcc:
     CC: gcc
   script:
     - BASE=${BASE_SHA:-${CI_COMMIT_BEFORE_SHA}} TIP=${TIP_SHA:-${CI_COMMIT_SHA}} ./automation/gitlab-ci/build-each-commit.sh 2>&1 | tee ../build-each-commit-gcc.log
+  after_script:
     - mv ../build-each-commit-gcc.log .
   artifacts:
     paths:

Thanks for looking into this, and yeah that is dumb, but why play games with the parent directory?

git clean -ffdx has the tendency to remove everything that's not committed, that's why. But maybe we can teach ./build-each-commit.sh to ignore that logfile.

Cheers,

Signature · 1 line
Anthony PERARD
#3 lore raw
Quoted 26 lines · Anthony PERARD
Quoted 11 lines · Andrew Cooper
Quoted 10 lines · Anthony PERARD

On failure of "build"-each-commit script, the next command that move the log back into the build directory isn't executed. Fix that by using "after_script" which is always executed even if the main "script" fails. (We would still miss the log when the jobs times out.)

---
 automation/gitlab-ci/test.yaml | 1 +
 1 file changed, 1 insertion(+)
Quoted 14 lines

automation/gitlab-ci/test.yaml build-each-commit-gcc: · line 146

Quoted 12 lines
automation/gitlab-ci/test.yaml +1 -0
index 810631bc46..5099f2e6b6 100644
@@ -140,6 +140,7 @@ build-each-commit-gcc:
     CC: gcc
   script:
     - BASE=${BASE_SHA:-${CI_COMMIT_BEFORE_SHA}} TIP=${TIP_SHA:-${CI_COMMIT_SHA}} ./automation/gitlab-ci/build-each-commit.sh 2>&1 | tee ../build-each-commit-gcc.log
+  after_script:
     - mv ../build-each-commit-gcc.log .
   artifacts:
     paths:

Thanks for looking into this, and yeah that is dumb, but why play games with the parent directory?

Quoted 3 lines

git clean -ffdx has the tendency to remove everything that's not committed, that's why. But maybe we can teach ./build-each-commit.sh to ignore that logfile.

Oh, right.  Yeah, lets not lose the log file like that.

I'd say that teaching git clean to leave the file interacted and not copying it is going to be a more robust option.

~Andrew

#4 lore raw
Quoted 27 lines · Andrew Cooper
Quoted 26 lines · Anthony PERARD
Quoted 25 lines · Andrew Cooper
Quoted 15 lines · Anthony PERARD

On failure of "build"-each-commit script, the next command that move the log back into the build directory isn't executed. Fix that by using "after_script" which is always executed even if the main "script" fails. (We would still miss the log when the jobs times out.)

---
 automation/gitlab-ci/test.yaml | 1 +
 1 file changed, 1 insertion(+)
automation/gitlab-ci/test.yaml +0 -0
index 810631bc46..5099f2e6b6 100644

automation/gitlab-ci/test.yaml build-each-commit-gcc: · line 146

Quoted 7 lines
automation/gitlab-ci/test.yaml +1 -0
@@ -140,6 +140,7 @@ build-each-commit-gcc:
     CC: gcc
   script:
     - BASE=${BASE_SHA:-${CI_COMMIT_BEFORE_SHA}} TIP=${TIP_SHA:-${CI_COMMIT_SHA}} ./automation/gitlab-ci/build-each-commit.sh 2>&1 | tee ../build-each-commit-gcc.log
+  after_script:
     - mv ../build-each-commit-gcc.log .
   artifacts:
     paths:

Thanks for looking into this, and yeah that is dumb, but why play games with the parent directory?

Quoted 8 lines
Quoted 3 lines

git clean -ffdx has the tendency to remove everything that's not committed, that's why. But maybe we can teach ./build-each-commit.sh to ignore that logfile.

Oh, right.  Yeah, lets not lose the log file like that.

I'd say that teaching git clean to leave the file interacted and not copying it is going to be a more robust option.

Yep, just tried that. But "Tree is dirty, aborted" :'(

./build-test.sh refuses to run if there's something in the git worktree.

This test is going to need more rework to be useful in the gitlab-ci.

Signature · 1 line
Anthony PERARD
#5 lore raw
Quoted 33 lines · Anthony PERARD
Quoted 32 lines · Andrew Cooper
Quoted 26 lines · Anthony PERARD
Quoted 11 lines · Andrew Cooper
Quoted 10 lines · Anthony PERARD

On failure of "build"-each-commit script, the next command that move the log back into the build directory isn't executed. Fix that by using "after_script" which is always executed even if the main "script" fails. (We would still miss the log when the jobs times out.)

---
 automation/gitlab-ci/test.yaml | 1 +
 1 file changed, 1 insertion(+)
Quoted 14 lines

automation/gitlab-ci/test.yaml build-each-commit-gcc: · line 146

Quoted 12 lines
automation/gitlab-ci/test.yaml +1 -0
index 810631bc46..5099f2e6b6 100644
@@ -140,6 +140,7 @@ build-each-commit-gcc:
     CC: gcc
   script:
     - BASE=${BASE_SHA:-${CI_COMMIT_BEFORE_SHA}} TIP=${TIP_SHA:-${CI_COMMIT_SHA}} ./automation/gitlab-ci/build-each-commit.sh 2>&1 | tee ../build-each-commit-gcc.log
+  after_script:
     - mv ../build-each-commit-gcc.log .
   artifacts:
     paths:

Thanks for looking into this, and yeah that is dumb, but why play games with the parent directory?

Quoted 3 lines

git clean -ffdx has the tendency to remove everything that's not committed, that's why. But maybe we can teach ./build-each-commit.sh to ignore that logfile.

Oh, right.  Yeah, lets not lose the log file like that.

Quoted 7 lines
Quoted 2 lines

I'd say that teaching git clean to leave the file interacted and not copying it is going to be a more robust option.

Yep, just tried that. But "Tree is dirty, aborted" :'(

./build-test.sh refuses to run if there's something in the git worktree.

This test is going to need more rework to be useful in the gitlab-ci.

Urgh fine.  Lets just go with your fix in the short term.  It's definitely better than nothing.

~Andrew