add smoketest group and try to provide template
cover add smoketest group and try to provide template
The patch [1/2] is a general patch, please review it as usual.
The patch [2/2] is RFC, by talking with Darrick, we agree to split this part from original [1/2], as RFC patch, to get separated review points. If any of you feel it's good, feel free to give it RVB, or feel free to share your review points.
More details refer to each patch commit log.
History:
https://lore.kernel.org/fstests/20230727153046.dl4palugnyidxoe7@zlang-mailbox/T/#m32ce9c4c316438074b068bd38c5e2dc6b0d53f42
https://lore.kernel.org/fstests/20230727030529.r4ivp6dmtrht5zo2@zlang-mailbox/T/#t
Thanks,
Zorro
PATCH 1/2 fstests: add smoketest group
Darrick suggests that fstests can provide a simple smoketest, by running several generic filesystem smoke testing for five minutes apiece (SOAK_DURATION="5m"). Since there are only five smoke tests, this is effectively a 20min super-quick test.
With gcov enabled, running these tests yields about ~75% coverage for iomap and ~60% for xfs; or ~50% for ext4 and ~75% for ext4; and ~45% for btrfs. Coverage was about ~65% for the pagecache.
--- doc/group-names.txt | 1 + tests/generic/475 | 2 +- tests/generic/476 | 2 +- tests/generic/521 | 2 +- tests/generic/522 | 2 +- tests/generic/642 | 2 +- 6 files changed, 6 insertions(+), 5 deletions(-)
doc/group-names.txt +1 -0
@@ -118,6 +118,7 @@ selftest tests with fixed results, used to validate testing setup send btrfs send/receive shrinkfs decreasing the size of a filesystem shutdown FS_IOC_SHUTDOWN ioctl +smoketest Simple smoke tests snapshot btrfs snapshots soak long running soak tests whose runtime can be controlled directly by setting the SOAK_DURATION variable
tests/generic/475 +1 -1
@@ -12,7 +12,7 @@ # testing efforts. # . ./common/preamble -_begin_fstest shutdown auto log metadata eio recoveryloop +_begin_fstest shutdown auto log metadata eio recoveryloop smoketest # Override the default cleanup function. _cleanup()
tests/generic/476 +1 -1
@@ -8,7 +8,7 @@ # bugs in the write path. # . ./common/preamble -_begin_fstest auto rw long_rw stress soak +_begin_fstest auto rw long_rw stress soak smoketest # Override the default cleanup function. _cleanup()
tests/generic/521 +1 -1
@@ -7,7 +7,7 @@ # Long-soak directio fsx test # . ./common/preamble -_begin_fstest soak long_rw +_begin_fstest soak long_rw smoketest # Import common functions. . ./common/filter
tests/generic/522 +1 -1
@@ -7,7 +7,7 @@ # Long-soak buffered fsx test # . ./common/preamble -_begin_fstest soak long_rw +_begin_fstest soak long_rw smoketest # Import common functions. . ./common/filter
tests/generic/642 +1 -1
@@ -8,7 +8,7 @@ # bugs in the xattr code. # . ./common/preamble -_begin_fstest auto soak attr long_rw stress +_begin_fstest auto soak attr long_rw stress smoketest _cleanup() {
Signature · 1 line
2.40.1
RFC PATCH 2/2 fstests: support test template
In reply to Zorro LangThe fstests has too many big or small testing groups, and it keeps growing. It's hard for many users to pick up test cases they need. Likes the smoketest, soak test, random-load test, integrality test and so on. So most of users might just run "quick" group, or "auto" group, or "all" directly each time. Besides the group, there're some global parameters (e.g. *_FACTOR, SOAK_DURATION, etc) too, so there're many "portfolios" to use them.
So I think fstests can provide a test template, which is bigger than group, base on group and global parameters, provide reference about how to do some kinds of tests.
Some users who are familar with fstests have their own wrappers, they do different kind of tests by fstests according their experience. They have their different testing templates, some templates might be helpful and recommended to others. So I'd like to let fstests provide a template/ directory and a "-t" option to load template. By this chance, hope more people can share their great test templates to others. We can record these templates in fstests, then anyone can use them directly or refer to them to write their wrapper.
--- check | 8 ++++++++ templates/smoketest | 16 ++++++++++++++++ 2 files changed, 24 insertions(+) create mode 100644 templates/smoketest
check +8 -0
@@ -335,6 +335,14 @@ while [ $# -gt 0 ]; do ;; -i) iterations=$2; shift ;; -I) iterations=$2; istop=true; shift ;; + -t) + source templates/$2 + if [ $? -ne 0 ];then + echo "Cannot import the templates/$2" + exit 1 + fi + shift + ;; -T) timestamp=true ;; -d) DUMP_OUTPUT=true ;; -b) brief_test_summary=true;;
templates/smoketest new file +16 -0
@@ -0,0 +1,16 @@ +##/bin/bash +# For infrequent filesystem developers who simply want to run a quick test +# of the most commonly used filesystem functionality, use this command: +# +# ./check -t smoketest <other config options> +# +# This template helps fstests to run several tests to exercise the file I/O, +# metadata, and crash recovery exercisers for four minutes apiece. This +# should complete in approximately 20 minutes. + +echo "**********************" +echo "* A Quick Smoke Test *" +echo "**********************" + +[ -z "$SOAK_DURATION" ] && SOAK_DURATION="4m" +GROUP_LIST="smoketest"
Signature · 1 line
2.40.1
Re: [PATCH 1/2] fstests: add smoketest group
In reply to Zorro LangQuoted 11 lines · Zorro Lang
Darrick suggests that fstests can provide a simple smoketest, by running several generic filesystem smoke testing for five minutes apiece (SOAK_DURATION="5m"). Since there are only five smoke tests, this is effectively a 20min super-quick test.
With gcov enabled, running these tests yields about ~75% coverage for iomap and ~60% for xfs; or ~50% for ext4 and ~75% for ext4; and ~45% for btrfs. Coverage was about ~65% for the pagecache.
Signed-off-by Darrick J. Wong <[email protected]>Signed-off-by Zorro Lang <[email protected]>
Er... this excerpt looks good to me! :)
(is that even allowed??)
--D
Quoted 89 lines
--- doc/group-names.txt | 1 + tests/generic/475 | 2 +- tests/generic/476 | 2 +- tests/generic/521 | 2 +- tests/generic/522 | 2 +- tests/generic/642 | 2 +- 6 files changed, 6 insertions(+), 5 deletions(-)doc/group-names.txt +1
-0@@ -118,6 +118,7 @@ selftest tests with fixed results, used to validate testing setup send btrfs send/receive shrinkfs decreasing the size of a filesystem shutdown FS_IOC_SHUTDOWN ioctl +smoketest Simple smoke tests snapshot btrfs snapshots soak long running soak tests whose runtime can be controlled directly by setting the SOAK_DURATION variabletests/generic/475 +1
-1@@ -12,7 +12,7 @@ # testing efforts. # . ./common/preamble -_begin_fstest shutdown auto log metadata eio recoveryloop +_begin_fstest shutdown auto log metadata eio recoveryloop smoketest # Override the default cleanup function. _cleanup()tests/generic/476 +1
-1@@ -8,7 +8,7 @@ # bugs in the write path. # . ./common/preamble -_begin_fstest auto rw long_rw stress soak +_begin_fstest auto rw long_rw stress soak smoketest # Override the default cleanup function. _cleanup()tests/generic/521 +1
-1@@ -7,7 +7,7 @@ # Long-soak directio fsx test # . ./common/preamble -_begin_fstest soak long_rw +_begin_fstest soak long_rw smoketest # Import common functions. . ./common/filtertests/generic/522 +1
-1@@ -7,7 +7,7 @@ # Long-soak buffered fsx test # . ./common/preamble -_begin_fstest soak long_rw +_begin_fstest soak long_rw smoketest # Import common functions. . ./common/filtertests/generic/642 +1
-1@@ -8,7 +8,7 @@ # bugs in the xattr code. # . ./common/preamble -_begin_fstest auto soak attr long_rw stress +_begin_fstest auto soak attr long_rw stress smoketest _cleanup() {Signature · 1 line
2.40.1