check: add CLI option to repeat tests
PATCH 1/1 check: add CLI option to repeat tests
From: Dave Chinner <[email protected]>
Frequently when trying to reproduce a problem I want to run a set of specific tests in a loop, over and over again. I run fstests from a set of run scripts that have non-trivial overhead (e.g. patterning block devices before the runs start), so if all I want to do is run the same test 100x, using a shell loop over the entire run scripts reduces the iteration rate substantially.
Hence add an option to check to allow fstests to loop a number of times over the configured test set without stopping.
--- check | 21 +++++++++++++++++---- 1 file changed, 17 insertions(+), 4 deletions(-)
check +17 -4
@@ -26,6 +26,7 @@ subdir_xfile="" brief_test_summary=false do_report=false DUMP_OUTPUT=false +iterations=1 # This is a global variable used to pass test failure text to reporting gunk _err_msg=""
@@ -66,6 +67,7 @@ check options -n show me, do not run tests -T output timestamps -r randomize test order + -i <n> iterate the test list <n> times -d dump test output to stdout -b brief test summary -R fmt[,fmt] generate report in formats specified. Supported format: [xunit]
@@ -297,7 +299,7 @@ while [ $# -gt 0 ]; do -n) showme=true ;; -r) randomize=true ;; - + -i) iterations=$2; shift ;; -T) timestamp=true ;; -d) DUMP_OUTPUT=true ;; -b) brief_test_summary=true;;
@@ -525,7 +527,10 @@ else trap "_wrapup; exit \$status" 0 1 2 3 15 fi -for section in $HOST_OPTIONS_SECTIONS; do +function run_section() +{ + local section=$1 + OLD_FSTYP=$FSTYP OLD_TEST_FS_MOUNT_OPTS=$TEST_FS_MOUNT_OPTS get_next_config $section
@@ -540,7 +545,7 @@ for section in $HOST_OPTIONS_SECTIONS; do fi done if $skip; then - continue + return fi fi
@@ -554,7 +559,7 @@ for section in $HOST_OPTIONS_SECTIONS; do fi done if $skip; then - continue + return fi fi
@@ -700,6 +705,8 @@ for section in $HOST_OPTIONS_SECTIONS; do seqres="$REPORT_DIR/$seqnum" mkdir -p $RESULT_DIR + rm -f ${RESULT_DIR}/require_scratch* + rm -f ${RESULT_DIR}/require_test* echo -n "$seqnum" if $showme; then
@@ -858,6 +865,12 @@ for section in $HOST_OPTIONS_SECTIONS; do _test_unmount 2> /dev/null _scratch_unmount 2> /dev/null +} + +for ((iters = 0; iters < $iterations; iters++)) do + for section in $HOST_OPTIONS_SECTIONS; do + run_section $section + done done interrupt=false
Signature · 1 line
2.26.2.761.g0e0b3e54be
Quoted 11 lines · Dave Chinner
From: Dave Chinner <[email protected]>
Frequently when trying to reproduce a problem I want to run a set of specific tests in a loop, over and over again. I run fstests from a set of run scripts that have non-trivial overhead (e.g. patterning block devices before the runs start), so if all I want to do is run the same test 100x, using a shell loop over the entire run scripts reduces the iteration rate substantially.
Hence add an option to check to allow fstests to loop a number of times over the configured test set without stopping.
Hi Dave,
this is useful as I am often doing the same. However what I am really missing is the ability to end the loop when a test fails and the ability to run tests in infinite loop (or until the test fails).
It should be easy modification to make, what do you think ?
-Lukas
Quoted 90 lines
Signed-off-by Dave Chinner <[email protected]>--- check | 21 +++++++++++++++++---- 1 file changed, 17 insertions(+), 4 deletions(-)check +17
-4@@ -26,6 +26,7 @@ subdir_xfile="" brief_test_summary=false do_report=false DUMP_OUTPUT=false +iterations=1 # This is a global variable used to pass test failure text to reporting gunk _err_msg=""@@ -66,6 +67,7 @@ check options -n show me, do not run tests -T output timestamps -r randomize test order + -i <n> iterate the test list <n> times -d dump test output to stdout -b brief test summary -R fmt[,fmt] generate report in formats specified. Supported format: [xunit]@@ -297,7 +299,7 @@ while [ $# -gt 0 ]; do -n) showme=true ;; -r) randomize=true ;; - + -i) iterations=$2; shift ;; -T) timestamp=true ;; -d) DUMP_OUTPUT=true ;; -b) brief_test_summary=true;;@@ -525,7 +527,10 @@ else trap "_wrapup; exit \$status" 0 1 2 3 15 fi -for section in $HOST_OPTIONS_SECTIONS; do +function run_section() +{ + local section=$1 + OLD_FSTYP=$FSTYP OLD_TEST_FS_MOUNT_OPTS=$TEST_FS_MOUNT_OPTS get_next_config $section@@ -540,7 +545,7 @@ for section in $HOST_OPTIONS_SECTIONS; do fi done if $skip; then - continue + return fi fi@@ -554,7 +559,7 @@ for section in $HOST_OPTIONS_SECTIONS; do fi done if $skip; then - continue + return fi fi@@ -700,6 +705,8 @@ for section in $HOST_OPTIONS_SECTIONS; do seqres="$REPORT_DIR/$seqnum" mkdir -p $RESULT_DIR + rm -f ${RESULT_DIR}/require_scratch* + rm -f ${RESULT_DIR}/require_test* echo -n "$seqnum" if $showme; then@@ -858,6 +865,12 @@ for section in $HOST_OPTIONS_SECTIONS; do _test_unmount 2> /dev/null _scratch_unmount 2> /dev/null +} + +for ((iters = 0; iters < $iterations; iters++)) do + for section in $HOST_OPTIONS_SECTIONS; do + run_section $section + done done interrupt=falseSignature · 1 line
2.26.2.761.g0e0b3e54be
Quoted 10 lines · Lukas Czerner
Quoted 9 lines · Dave Chinner
From: Dave Chinner <[email protected]>
Frequently when trying to reproduce a problem I want to run a set of specific tests in a loop, over and over again. I run fstests from a set of run scripts that have non-trivial overhead (e.g. patterning block devices before the runs start), so if all I want to do is run the same test 100x, using a shell loop over the entire run scripts reduces the iteration rate substantially.
Quoted 8 lines
Quoted 2 lines
Hence add an option to check to allow fstests to loop a number of times over the configured test set without stopping.
Hi Dave,
this is useful as I am often doing the same. However what I am really missing is the ability to end the loop when a test fails and the ability to run tests in infinite loop (or until the test fails).
To run until a test set fails, I have just been using "-i 10000" because if it passes 10000 loops, it isn't going to fail.. :)
Hence I didn't see a need to complicate things with a specific "loop forever" options.
Quoted 1 line
It should be easy modification to make, what do you think ?
Feel free to add send a patch on top of this one to support that - all of the failures I've been looking at with this have been "test does not complete" failures, so it wasn't important to me.
As it is, right now I don't have time to feature creep simple modifications like this into more complex functionality - I really only sent this out because Willy mentioned on #xfs that he wished there was a way to loop a test and I said "I've got a patch on my local tree for that"....
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]