travis: Improve log browseability
cover travis: Improve log browseability
While I was working on [1] I needed to check what dependencies were missing on Travis environment to build a kernel. The test was coded so that a build failure would cancel the test. As a result, and because all tests pass'ed (or cancel'ed) I could not see the Avocado logs (consequentely not able to check the missing packages). That was the main reason for the Patch 01 in this series. Also it seems pointless to print the log of the tests that pass.
The patch 02 is a tentative to improve the browsing of build vs test logs. Honestly I was expecting that Travis was able to put the output of steps each on toggled text boxes (see the 'before_script' section, each command output has a
toggle). But that isn't the behavior of the 'script' section. At least it prints the end of the build command ('The command "BUILD_RC=0 && make -j3 || BUILD_RC=$?" exited with 0.') which can be grep'ed.
Git:
- Tree: https://github.com/wainersm/qemu
- Brach: travis_improved_browse
CI:
- Travis (FAIL): https://travis-ci.org/wainersm/qemu/builds/630948777
QEMU keep failing with --without-default-devices
References:
[1] https://www.mail-archive.com/[email protected]/msg663432.html
Wainer dos Santos Moschetta (2):
travis.yml: avocado: Print logs of non-pass tests only travis.yml: Detach build and test steps
.travis.yml | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
Signature · 1 line
2.23.0
PATCH 1/2 travis.yml: avocado: Print logs of non-pass tests only
The acceptance tests build on Travis is configured to print the entire Avocado's job log in case any test fail. Usually one is interested on failed tests only though. So this change the Travis configuration in order to show the log of tests which status is different from 'PASS' and 'SKIP' only. Note that 'CANCEL'-ed tests will have the log printed too because it can help to debug some condition on CI environment which is not being fulfilled.
--- .travis.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
.travis.yml +2 -2
@@ -262,8 +262,8 @@ matrix: - env: - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,aarch64-softmmu,arm-softmmu,s390x-softmmu,alpha-softmmu,ppc-softmmu,ppc64-softmmu,m68k-softmmu,sparc-softmmu" - TEST_CMD="make check-acceptance" - after_failure: - - cat tests/results/latest/job.log + after_script: + - python3 -c 'import json; r = json.load(open("tests/results/latest/results.json")); [print(t["logfile"]) for t in r["tests"] if t["status"] not in ("PASS", "SKIP")]' | xargs cat addons: apt: packages:
Signature · 1 line
2.23.0
Re: [PATCH 1/2] travis.yml: avocado: Print logs of non-pass tests only
Quoted 7 lines · Wainer dos Santos Moschetta
The acceptance tests build on Travis is configured to print the entire Avocado's job log in case any test fail. Usually one is interested on failed tests only though. So this change the Travis configuration in order to show the log of tests which status is different from 'PASS' and 'SKIP' only. Note that 'CANCEL'-ed tests will have the log printed too because it can help to debug some condition on CI environment which is not being fulfilled.
Nice!
Tested with a broken test:
https://travis-ci.org/philmd/qemu/jobs/631341382
Quoted 21 lines
Signed-off-by Wainer dos Santos Moschetta <[email protected]>--- .travis.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-).travis.yml +2
-2@@ -262,8 +262,8 @@ matrix: - env: - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,aarch64-softmmu,arm-softmmu,s390x-softmmu,alpha-softmmu,ppc-softmmu,ppc64-softmmu,m68k-softmmu,sparc-softmmu" - TEST_CMD="make check-acceptance" - after_failure: - - cat tests/results/latest/job.log + after_script: + - python3 -c 'import json; r = json.load(open("tests/results/latest/results.json")); [print(t["logfile"]) for t in r["tests"] if t["status"] not in ("PASS", "SKIP")]' | xargs cat addons: apt: packages:
Re: [PATCH 1/2] travis.yml: avocado: Print logs of non-pass tests only
In reply to Wainer dos Santos MoschettaQuoted 26 lines · Wainer dos Santos Moschetta
The acceptance tests build on Travis is configured to print the entire Avocado's job log in case any test fail. Usually one is interested on failed tests only though. So this change the Travis configuration in order to show the log of tests which status is different from 'PASS' and 'SKIP' only. Note that 'CANCEL'-ed tests will have the log printed too because it can help to debug some condition on CI environment which is not being fulfilled.
Signed-off-by Wainer dos Santos Moschetta <[email protected]>--- .travis.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-).travis.yml +2
-2@@ -262,8 +262,8 @@ matrix: - env: - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,aarch64-softmmu,arm-softmmu,s390x-softmmu,alpha-softmmu,ppc-softmmu,ppc64-softmmu,m68k-softmmu,sparc-softmmu" - TEST_CMD="make check-acceptance" - after_failure: - - cat tests/results/latest/job.log + after_script: + - python3 -c 'import json; r = json.load(open("tests/results/latest/results.json"));
Quoted 2 lines
[
print(t["logfile"])for t in r["tests"] if t["status"] not in ("PASS", "SKIP")]' | xargs cat
Shame there is no scriptlet for this sort of thing packaged with avocado. Anyway I'll give it a spin and see if any failures come up.
Quoted 3 lines
addons:
apt:
packages:
Signature · 1 line
Alex Bennée
Re: [PATCH 1/2] travis.yml: avocado: Print logs of non-pass tests only
Quoted 24 lines · Alex Bennée
Quoted 22 lines · Wainer dos Santos Moschetta
The acceptance tests build on Travis is configured to print the entire Avocado's job log in case any test fail. Usually one is interested on failed tests only though. So this change the Travis configuration in order to show the log of tests which status is different from 'PASS' and 'SKIP' only. Note that 'CANCEL'-ed tests will have the log printed too because it can help to debug some condition on CI environment which is not being fulfilled.
Signed-off-by Wainer dos Santos Moschetta <[email protected]>--- .travis.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-).travis.yml +0
-1@@ -262,8 +262,8 @@ matrix: - env: - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,aarch64-softmmu,arm-softmmu,s390x-softmmu,alpha-softmmu,ppc-softmmu,ppc64-softmmu,m68k-softmmu,sparc-softmmu" - TEST_CMD="make check-acceptance" - after_failure:
Quoted 8 lines
Quoted 6 lines
.travis.yml +2
-1- - cat tests/results/latest/job.log + after_script: + - python3 -c 'import json; r = json.load(open("tests/results/latest/results.json"));[
print(t["logfile"])for t in r["tests"] if t["status"] not in ("PASS", "SKIP")]' | xargs catShame there is no scriptlet for this sort of thing packaged with avocado. Anyway I'll give it a spin and see if any failures come up.
Good point. I'm working on such as scriptlet ('plugin' on Avocado jargon) but I think we can stick with this change.
Thanks!
- Wainer
Quoted 5 lines
Quoted 4 lines
addons:
apt:
packages:
PATCH 2/2 travis.yml: Detach build and test steps
In reply to Wainer dos Santos MoschettaCurrently build and test commands are a single step in a Travis's script block. In order to see the output of the tests one needs to scroll down the log to find where the build messages ended and the limit is not clear. If they were in different steps then Travis would print the result build command, which can be easily grep'ed.
So this change is made to detach those commands
to ease the visualization of the output.
Note that all steps on the script block is executed regardless if one previous has failed. To overcome it, let's save the return code of the build then check whether succeed or failed on the test step.
--- .travis.yml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
.travis.yml +2 -1
@@ -94,7 +94,8 @@ before_script: - mkdir -p ${BUILD_DIR} && cd ${BUILD_DIR} - ${SRC_DIR}/configure ${BASE_CONFIG} ${CONFIG} || { cat config.log && exit 1; } script: - - make -j3 && travis_retry ${TEST_CMD} + - BUILD_RC=0 && make -j3 || BUILD_RC=$? + - if [ "$BUILD_RC" -eq 0 ] ; then travis_retry ${TEST_CMD} ; else $(exit $BUILD_RC); fi after_script: - if command -v ccache ; then ccache --show-stats ; fi
Signature · 1 line
2.23.0
Re: [PATCH 2/2] travis.yml: Detach build and test steps
Quoted 7 lines · Wainer dos Santos Moschetta
Currently build and test commands are a single step in a Travis's
scriptblock. In order to see the output of the tests one needs to scroll down the log to find where the build messages ended and the limit is not clear. If they were in different steps then Travis would print the result build command, which can be easily grep'ed.
Quoted 7 lines
So this change is made to detach those commands
to ease the visualization of the output.Note that all steps on the
scriptblock is executed regardless if one previous has failed. To overcome it, let's save the return code of the build then check whether succeed or failed on the test step.
Tested with a broken test:
https://travis-ci.org/philmd/qemu/jobs/631341382
Quoted 21 lines
Signed-off-by Wainer dos Santos Moschetta <[email protected]>--- .travis.yml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-).travis.yml +2
-1@@ -94,7 +94,8 @@ before_script: - mkdir -p ${BUILD_DIR} && cd ${BUILD_DIR} - ${SRC_DIR}/configure ${BASE_CONFIG} ${CONFIG} || { cat config.log && exit 1; } script: - - make -j3 && travis_retry ${TEST_CMD} + - BUILD_RC=0 && make -j3 || BUILD_RC=$? + - if [ "$BUILD_RC" -eq 0 ] ; then travis_retry ${TEST_CMD} ; else $(exit $BUILD_RC); fi after_script: - if command -v ccache ; then ccache --show-stats ; fi