Skip to contentSkip to conversation

automation: test to build each commit in a branch

PATCH In review Wei Liu 1 participant 7 messages Feb 28, 2019 – Apr 1, 2019

lore
Share & tools
lore

Wei Liu

author
#0 lore raw

cover automation: test to build each commit in a branch

We have a requirement to make each commit buildable in xen.git. Put this into a test in Gitlab CI.

Wei Liu (5):
automation: allow build-test.sh to run in detached HEAD state
automation: set ret for potential error in build-test.sh
automation: add a script to build newly pushed commits in Gitlab CI DO NOT APPLY: temporarily drop some tests
automation: introduce a test to build each commit

 .gitlab-ci.yml                            |  4 +-
 automation/gitlab-ci/build-each-commit.sh | 16 +++++++
 automation/gitlab-ci/test.yaml            | 75 ++-----------------------------
 automation/scripts/build-test.sh          | 18 +++++---
 4 files changed, 35 insertions(+), 78 deletions(-)
 create mode 100755 automation/gitlab-ci/build-each-commit.sh
Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
author
#1 lore raw

PATCH 1/5 automation: allow build-test.sh to run in detached HEAD state

---
 automation/scripts/build-test.sh | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)
automation/scripts/build-test.sh +12 -5
index c318b65d5a..206a4f9a4a 100755
@@ -5,6 +5,9 @@
 #
 # The default rune is rather simple. To do a cross-build, please put your usual
 # build rune in a shell script and invoke it with this script.
+#
+# Set NON_SYMBOLIC_REF=1 if you want to use this script in detached HEAD state.
+# This is currently used by automated test system.
 
 if test $# -lt 2 ; then
     echo "Usage:"
@@ -25,10 +28,14 @@ fi
 BASE=$1; shift
 TIP=$1; shift
 
-ORIG_BRANCH=`git symbolic-ref -q --short HEAD`
-if test $? -ne 0; then
-    echo "Detached HEAD, aborted"
-    exit 1
+if [[ "_${NON_SYMBOLIC_REF}" != "_1" ]]; then
+    ORIG=`git symbolic-ref -q --short HEAD`
+    if test $? -ne 0; then
+        echo "Detached HEAD, aborted"
+        exit 1
+    fi
+else
+    ORIG=`git rev-parse HEAD`
 fi
 
 while read num rev; do
@@ -55,7 +62,7 @@ while read num rev; do
 done < <(git rev-list $BASE..$TIP | nl -ba | tac)
 
 echo "Restoring original HEAD"
-git checkout $ORIG_BRANCH
+git checkout $ORIG
 gco_ret=$?
 if test $gco_ret -ne 0; then
     echo "Failed to restore orignal HEAD. Check tree status before doing anything else!"
Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
author
#2 lore raw

PATCH 2/5 automation: set ret for potential error in build-test.sh

In reply to Wei Liu

git rev-list can fail if the base..tip range contains invalid commit(s). If that happens ret never gets a chance to be set.

Set ret before hand to fix the issue.

---
 automation/scripts/build-test.sh | 1 +
 1 file changed, 1 insertion(+)
automation/scripts/build-test.sh +1 -0
index 206a4f9a4a..da643adc01 100755
@@ -38,6 +38,7 @@ else
     ORIG=`git rev-parse HEAD`
 fi
 
+ret=1
 while read num rev; do
     echo "Testing $num $rev"
 
Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
author
#3 lore raw

PATCH 3/5 automation: add a script to build newly pushed commits in Gitlab CI

In reply to Wei Liu
---
 automation/gitlab-ci/build-each-commit.sh | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)
 create mode 100755 automation/gitlab-ci/build-each-commit.sh
automation/gitlab-ci/build-each-commit.sh new file +16 -0
new file mode 100755
index 0000000000..5b2e224710
@@ -0,0 +1,16 @@
+#!/bin/bash
+
+# For a newly pushed branch the BEFORE_SHA will be all 0s
+if [[ ${CI_COMMIT_BEFORE_SHA} == 0000000000000000000000000000000000000000 ]]; then
+    echo "Newly pushed branch, skipped"
+    exit 0
+fi
+
+git merge-base --is-ancestor ${CI_COMMIT_BEFORE_SHA} ${CI_COMMIT_SHA}
+if [[ $? -ne 0 ]]; then
+    echo "${CI_COMMIT_SHA} is not a descendent of ${CI_COMMIT_BEFORE_SHA}, skipped"
+    exit 0
+fi
+
+NON_SYMBOLIC_REF=1 ./automation/scripts/build-test.sh ${CI_COMMIT_BEFORE_SHA} ${CI_COMMIT_SHA} \
+       	bash -c "make -j4 distclean && ./automation/scripts/build"
Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
author
#4 lore raw

PATCH 4/5 DO NOT APPLY: temporarily drop some tests

In reply to Wei Liu

This helps shorten turn-around time.

---
 .gitlab-ci.yml                 |  4 +--
 automation/gitlab-ci/test.yaml | 68 ------------------------------------------
 2 files changed, 2 insertions(+), 70 deletions(-)
.gitlab-ci.yml +2 -2
index c8bd7519d5..72ff9136ce 100644
@@ -1,7 +1,7 @@
 stages:
-  - build
+        # - build
   - test
 
 include:
-  - 'automation/gitlab-ci/build.yaml'
+        #- 'automation/gitlab-ci/build.yaml'
   - 'automation/gitlab-ci/test.yaml'
automation/gitlab-ci/test.yaml +0 -68

74 lines not on this page · raw message

Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
author
#5 lore raw

PATCH 5/5 automation: introduce a test to build each commit

In reply to Wei Liu

This is added to the test stage so that its failure won't block other things.

---
Note: this patch will look different when it is applied to staging, but you can see the core idea here.

---
 automation/gitlab-ci/test.yaml | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
automation/gitlab-ci/test.yaml +4 -3
index 140b11b892..b9563fed1c 100644
@@ -1,14 +1,15 @@
 # Test jobs
-qemu-smoke-x86-64-gcc:
+build-each-commit-gcc:
   stage: test
   image: registry.gitlab.com/xen-project/xen/${CONTAINER}
   variables:
     CONTAINER: debian:stretch
+    XEN_TARGET_ARCH: x86_64
+    CC: gcc
   script:
-    - ./automation/scripts/qemu-smoke-x86-64.sh pv 2>&1 | tee qemu-smoke-x86-64.log
+    - ./automation/gitlab-ci/build-each-commit.sh 2>&1 | tee build-each-commit-gcc.log
   artifacts:
     paths:
-      - smoke.serial
       - '*.log'
     when: always
   tags:
Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
author
#6 lore raw
In reply to Wei Liu

Ping?

Quoted 10 lines · Wei Liu

We have a requirement to make each commit buildable in xen.git. Put this into a test in Gitlab CI.

Wei Liu (5):
automation: allow build-test.sh to run in detached HEAD state
automation: set ret for potential error in build-test.sh
automation: add a script to build newly pushed commits in Gitlab CI DO NOT APPLY: temporarily drop some tests
automation: introduce a test to build each commit

Quoted 10 lines
 .gitlab-ci.yml                            |  4 +-
 automation/gitlab-ci/build-each-commit.sh | 16 +++++++
 automation/gitlab-ci/test.yaml            | 75 ++-----------------------------
 automation/scripts/build-test.sh          | 18 +++++---
 4 files changed, 35 insertions(+), 78 deletions(-)
 create mode 100755 automation/gitlab-ci/build-each-commit.sh
Signature · 1 line
2.11.0

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel