libevent: add granularity to ptest log
PATCH v3 1/1 libevent: add granularity to ptest log
From: Trevor Gamblin <[email protected]>
The libevent ptest used to report only a global pass or a fail result. Count individual PASS, FAIL, SKIP results. The SKIP results now include tests that are disabled in the libevent code.
libevent's ptest output did not comply with the automake-style output "result: testname", and reported a FAIL status at the end of the test run if any of the libevent tests failed. This patch makes the log consistent with the automake style:
PASS: http/cancel_by_host_no_ns
PASS: http/cancel_inactive_server
PASS: http/cancel_by_host_no_ns_inactive_server
SKIP: http/cancel_by_host_server_timeout
SKIP: http/cancel_server_timeout
and provides a summary as follows:
=== Test Summary ===
TOTAL: 316
PASSED: 300
FAILED: 0
SKIPPED: 16
DURATION: 87
END: /usr/lib/libevent/ptest--- .../libevent/libevent/run-ptest | 43 ++++++++++++------- .../libevent/libevent_2.1.11.bb | 3 ++ 2 files changed, 31 insertions(+), 15 deletions(-)
meta/recipes-support/libevent/libevent/run-ptest +28 -15
@@ -1,18 +1,31 @@ #!/bin/sh -fail=0 -for test in ./test/* -do - $test - if [ $? -ne 0 ] - then - fail=1 - fi -done +# run-ptest - 'ptest' test infrastructure shell script that +# wraps the libevent test scripts +# +# Trevor Gamblin <[email protected]> +############################################################### +LIBEVENTLIB=@libdir@/libevent +LOG="${LIBEVENTLIB}/ptest/libevent_ptest_$(date +%Y%m%d-%H%M%S).log" -if [ $fail -eq 0 ] -then - echo "PASS: libevent" -else - echo "FAIL: libevent" -fi +cd ${LIBEVENTLIB}/ptest + +# Run the libevent regress test, but format the pass/fail outputs so +# that ptest can count them. Ignore the line containing "tests or +# "TESTS" as that line is the libevent total at the end of the test +# and is counted as an extra test failure, e.g. +# "2/300 TESTS FAILED. (31 skipped)" will be seen as a test failure +# rather than a total +./test/regress 2>&1| sed -e '/TESTS/d' -e '/tests/d' -e '/OK/ s/^/PASS: / ; /FAILED/ s/^/FAIL: / ; /SKIPPED/ s/^/SKIP: / ; /DISABLED/ s/^/SKIP: /' | cut -f1,2 -d ':' | tee -a ${LOG} + +passed=`grep PASS ${LOG}|wc -l` +failed=`grep FAIL ${LOG}|wc -l` +skipped=`grep -E SKIP ${LOG}|wc -l` +all=$((passed + failed + skipped)) + +( echo "=== Test Summary ===" + echo "TOTAL: ${all}" + echo "PASSED: ${passed}" + echo "FAILED: ${failed}" + echo "SKIPPED: ${skipped}" +) | tee -a ${LOG}
meta/recipes-support/libevent/libevent_2.1.11.bb +3 -0
@@ -43,4 +43,7 @@ do_install_ptest() { do install -m 0755 $file ${D}${PTEST_PATH}/test done + + # handle multilib + sed -i s:@libdir@:${libdir}:g ${D}${PTEST_PATH}/run-ptest }
Signature · 1 line
2.21.0
Quoted 89 lines · Trevor Gamblin
From: Trevor Gamblin <[email protected]>
The libevent ptest used to report only a global pass or a fail result. Count individual PASS, FAIL, SKIP results. The SKIP results now include tests that are disabled in the libevent code.
libevent's ptest output did not comply with the automake-style output "result: testname", and reported a FAIL status at the end of the test run if any of the libevent tests failed. This patch makes the log consistent with the automake style:
PASS: http/cancel_by_host_no_ns
PASS: http/cancel_inactive_server
PASS: http/cancel_by_host_no_ns_inactive_server
SKIP: http/cancel_by_host_server_timeout
SKIP: http/cancel_server_timeoutand provides a summary as follows:
=== Test Summary === TOTAL: 316 PASSED: 300 FAILED: 0 SKIPPED: 16 DURATION: 87 END: /usr/lib/libevent/ptestSigned-off-by Trevor Gamblin <[email protected]>--- .../libevent/libevent/run-ptest | 43 ++++++++++++------- .../libevent/libevent_2.1.11.bb | 3 ++ 2 files changed, 31 insertions(+), 15 deletions(-)meta/recipes-support/libevent/libevent/run-ptest +28
-15@@ -1,18 +1,31 @@ #!/bin/sh -fail=0 -for test in ./test/* -do - $test - if [ $? -ne 0 ] - then - fail=1 - fi -done +# run-ptest - 'ptest' test infrastructure shell script that +# wraps the libevent test scripts +# +# Trevor Gamblin <[email protected]> +############################################################### +LIBEVENTLIB=@libdir@/libevent +LOG="${LIBEVENTLIB}/ptest/libevent_ptest_$(date +%Y%m%d-%H%M%S).log" -if [ $fail -eq 0 ] -then - echo "PASS: libevent" -else - echo "FAIL: libevent" -fi +cd ${LIBEVENTLIB}/ptest + +# Run the libevent regress test, but format the pass/fail outputs so +# that ptest can count them. Ignore the line containing "tests or +# "TESTS" as that line is the libevent total at the end of the test +# and is counted as an extra test failure, e.g. +# "2/300 TESTS FAILED. (31 skipped)" will be seen as a test failure +# rather than a total +./test/regress 2>&1| sed -e '/TESTS/d' -e '/tests/d' -e '/OK/ s/^/PASS: / ; /FAILED/ s/^/FAIL: / ; /SKIPPED/ s/^/SKIP: / ; /DISABLED/ s/^/SKIP: /' | cut -f1,2 -d ':' | tee -a ${LOG} + +passed=`grep PASS ${LOG}|wc -l` +failed=`grep FAIL ${LOG}|wc -l` +skipped=`grep -E SKIP ${LOG}|wc -l` +all=$((passed + failed + skipped)) + +( echo "=== Test Summary ===" + echo "TOTAL: ${all}" + echo "PASSED: ${passed}" + echo "FAILED: ${failed}" + echo "SKIPPED: ${skipped}" +) | tee -a ${LOG}meta/recipes-support/libevent/libevent_2.1.11.bb +0
-0
meta/recipes-support/libevent/libevent_2.1.11.bb do_install_ptest() { · line 49
Quoted 7 lines
meta/recipes-support/libevent/libevent_2.1.11.bb +3
-0@@ -43,4 +43,7 @@ do_install_ptest() { do install -m 0755 $file ${D}${PTEST_PATH}/test done + + # handle multilib + sed -i s:@libdir@:${libdir}:g ${D}${PTEST_PATH}/run-ptest }
Noticed that the autobuilder results using v2 were detecting libevent's "2/300 TESTS FAILED. (31 skipped)" line at the end of the regress test as an extra failure. Fixed that up and reduced the ptest to just running "regress", as the other scripts in the test folder are more performance-oriented and don't provide a pass/fail result that can be used.
✗ patchtest: failure for libevent: add granularity to ptest log (rev2)
In reply to Trevor Gamblin== Series Details ==
Series: libevent: add granularity to ptest log (rev2)
Revision: 2
URL : https://patchwork.openembedded.org/series/19759/
State : failure
== Summary ==
Thank you for submitting this patch series to OpenEmbedded Core. This is an automated response. Several tests have been executed on the proposed series by patchtest resulting in the following failures:
* Issue Series does not apply on top of target branch [test_series_merge_on_head]
Suggested fix Rebase your series on top of targeted branch
Targeted branch master (currently at d41606244c)
If you believe any of these test results are incorrect, please reply to the mailing list ([email protected]) raising your concerns. Otherwise we would appreciate you correcting the issues and submitting a new version of the patchset if applicable. Please ensure you add/increment the version number when sending the new version (i.e. [PATCH] -> [PATCH v2] -> [PATCH v3] -> ...).
---
Guidelines: https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest Test suite: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe