test/run: Exit on error from tests
PATCH 1/1 test/run: Exit on error from tests
Currently, result isn't saved outside of the scope of run_test, so we always exit on error here. An easier way to deal with having test/run show failures is to exit on failure.
---
Automation is only as good as the humans behind it. If you forget, like I did, that you finally fixed Jenkins to email you every time, that lack of emails about my non-travis jobs failing wasn't "no problems", it was "You forgot to migrate all the Docker stuff to the new build server". So, I'm going to apply this ASAP as I would not have let in the u-boot-dm PR, had my setup been functioning as expected.
--- test/run | 18 ++++-------------- 1 file changed, 4 insertions(+), 14 deletions(-)
test/run +4 -14
@@ -1,24 +1,14 @@ #!/bin/bash -run_test() { - $@ - [ $? -ne 0 ] && result=$((result+1)) - echo "result $result" -} +# Exit on error +set +e # Run all tests that the standard sandbox build can support -run_test ./test/py/test.py --bd sandbox --build +./test/py/test.py --bd sandbox --build # Run tests which require sandbox_spl -run_test ./test/py/test.py --bd sandbox_spl --build -k \ +./test/py/test.py --bd sandbox_spl --build -k \ test/py/tests/test_ofplatdata.py # Run tests for the flat DT version of sandbox ./test/py/test.py --bd sandbox_flattree --build - -if [ $result == 0 ]; then - echo "Tests passed!" -else - echo "Tests FAILED" - exit 1 -fi
Signature · 1 line
2.7.4
PATCH 1/1 test/run: Exit on error from tests
Hi Tom,
Quoted 14 lines · Tom Rini
Currently, result isn't saved outside of the scope of
run_test, so we always exit on error here. An easier way to deal with having test/run show failures is to exit on failure.Cc Simon Glass <[email protected]>Fixes 2f52018c3cbb ("test/run: Report and return failure")Signed-off-by Tom Rini <[email protected]>---
Automation is only as good as the humans behind it. If you forget, like I did, that you finally fixed Jenkins to email you every time, that lack of emails about my non-travis jobs failing wasn't "no problems", it was "You forgot to migrate all the Docker stuff to the new build server". So, I'm going to apply this ASAP as I would not have let in the u-boot-dm PR, had my setup been functioning as expected.
Quoted 3 lines
--- test/run | 18 ++++-------------- 1 file changed, 4 insertions(+), 14 deletions(-)
I'm OK with this if it's what you want. This means that we will not run all the tests if one of the earlier one fails.
Did you see this one?
http://patchwork.ozlabs.org/patch/841453/
Regards,
Simon
PATCH 1/1 test/run: Exit on error from tests
Quoted 21 lines · Simon Glass
Hi Tom,
Quoted 14 lines · Tom Rini
Currently, result isn't saved outside of the scope of
run_test, so we always exit on error here. An easier way to deal with having test/run show failures is to exit on failure.Cc Simon Glass <[email protected]>Fixes 2f52018c3cbb ("test/run: Report and return failure")Signed-off-by Tom Rini <[email protected]>---
Automation is only as good as the humans behind it. If you forget, like I did, that you finally fixed Jenkins to email you every time, that lack of emails about my non-travis jobs failing wasn't "no problems", it was "You forgot to migrate all the Docker stuff to the new build server". So, I'm going to apply this ASAP as I would not have let in the u-boot-dm PR, had my setup been functioning as expected.Quoted 3 lines
--- test/run | 18 ++++-------------- 1 file changed, 4 insertions(+), 14 deletions(-)
Quoted 6 lines
I'm OK with this if it's what you want. This means that we will not run all the tests if one of the earlier one fails.
Did you see this one?
I'm playing a little catch-up and that's fine too. I've unearthed another bit of breakage to fix now (because the time PRs break my setup is the time I break runtime of my setup and assume it's working).
Signature · 8 lines
Tom -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: not available URL: <http://lists.denx.de/pipermail/u-boot/attachments/20171127/dd53b42d/attachment.sig>
PATCH 1/1 test/run: Exit on error from tests
Hi Tom,
Quoted 25 lines · Tom Rini
Quoted 24 lines · Simon Glass
Hi Tom,
Quoted 14 lines · Tom Rini
Currently, result isn't saved outside of the scope of
run_test, so we always exit on error here. An easier way to deal with having test/run show failures is to exit on failure.Cc Simon Glass <[email protected]>Fixes 2f52018c3cbb ("test/run: Report and return failure")Signed-off-by Tom Rini <[email protected]>---
Automation is only as good as the humans behind it. If you forget, like I did, that you finally fixed Jenkins to email you every time, that lack of emails about my non-travis jobs failing wasn't "no problems", it was "You forgot to migrate all the Docker stuff to the new build server". So, I'm going to apply this ASAP as I would not have let in the u-boot-dm PR, had my setup been functioning as expected.Quoted 3 lines
--- test/run | 18 ++++-------------- 1 file changed, 4 insertions(+), 14 deletions(-)I'm OK with this if it's what you want. This means that we will not run all the tests if one of the earlier one fails.
Quoted 7 lines
Quoted 3 lines
Did you see this one?
I'm playing a little catch-up and that's fine too. I've unearthed another bit of breakage to fix now (because the time PRs break my setup is the time I break runtime of my setup and assume it's working).
Yes, OK, sorry about the churn. As you can tell I'm trying to get it so all the tests we have run automatically both with 'make tests' and with travis-ci. Still a way to go but it is closer.
Regards,
Simon
PATCH 1/1 test/run: Exit on error from tests
Quoted 32 lines · Simon Glass
Hi Tom,
Quoted 29 lines · Tom Rini
Quoted 27 lines · Simon Glass
Hi Tom,
Quoted 14 lines · Tom Rini
Currently, result isn't saved outside of the scope of
run_test, so we always exit on error here. An easier way to deal with having test/run show failures is to exit on failure.Cc Simon Glass <[email protected]>Fixes 2f52018c3cbb ("test/run: Report and return failure")Signed-off-by Tom Rini <[email protected]>---
Automation is only as good as the humans behind it. If you forget, like I did, that you finally fixed Jenkins to email you every time, that lack of emails about my non-travis jobs failing wasn't "no problems", it was "You forgot to migrate all the Docker stuff to the new build server". So, I'm going to apply this ASAP as I would not have let in the u-boot-dm PR, had my setup been functioning as expected.Quoted 3 lines
--- test/run | 18 ++++-------------- 1 file changed, 4 insertions(+), 14 deletions(-)I'm OK with this if it's what you want. This means that we will not run all the tests if one of the earlier one fails.
Did you see this one?
Quoted 7 lines
Quoted 3 lines
I'm playing a little catch-up and that's fine too. I've unearthed another bit of breakage to fix now (because the time PRs break my setup is the time I break runtime of my setup and assume it's working).
Yes, OK, sorry about the churn. As you can tell I'm trying to get it so all the tests we have run automatically both with 'make tests' and with travis-ci. Still a way to go but it is closer.
I'm looking forward to it all being in there :) Thanks!
Signature · 8 lines
Tom -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: not available URL: <http://lists.denx.de/pipermail/u-boot/attachments/20171128/cc3e3ed9/attachment.sig>