fstests: provides smoketest template
PATCH 1/1 fstests: provides smoketest template
Darrick suggests that fstests can provide a simple smoketest, by running several generic filesystem smoke testing for five minutes apiece. Since there are only five smoke tests, this is effectively a 16min super-quick test.
With gcov enabled, running these tests yields about ~75% coverage for iomap and ~60% for xfs; or ~50% for ext4 and ~75% for ext4; and ~45% for btrfs. Coverage was about ~65% for the pagecache.
To implement that, this patch add a new "-t" option to ./check, and a new directory "template" under xfstests/, then we can have smoketest template, also can have more other testing templates.
---
Hi,
This patch uses another way to achieve the smoketest requirement[1]. When I reviewed the patch from Darrick [2], I thought "smoketest" might not be the last one requirement likes that. I don't want to give each kind of tests a separated option. But we might can leave a hook for more fs-devel who have good testing templates for fstests.
Although we have test groups, but those group names are too complex for the users who not always use fstests. So I'm thinking about providing some simple templates to run fstests, these templates base on test groups and some fstests global parameters, help users to know what kind of test they can do.
Feel free to discuss, and if most of you prefer the original patch [2], I'll also think about merging the original one :)
Thanks,
Zorro
[1]
https://lore.kernel.org/fstests/20230726145441.lbzzokwigrztimyq@zlang-mailbox/T/#mabc0de98699f1b877c87caccb13809c9283c0606
[2]
https://lore.kernel.org/fstests/169033660570.3222210.3010411210438664310.stgit@frogsfrogsfrogs/T/#u
check | 8 ++++++++ doc/group-names.txt | 1 + templates/smoketest | 16 ++++++++++++++++ tests/generic/475 | 2 +- tests/generic/476 | 2 +- tests/generic/521 | 2 +- tests/generic/522 | 2 +- tests/generic/642 | 2 +- 8 files changed, 30 insertions(+), 5 deletions(-) create mode 100644 templates/smoketest
check +8 -0
@@ -335,6 +335,14 @@ while [ $# -gt 0 ]; do ;; -i) iterations=$2; shift ;; -I) iterations=$2; istop=true; shift ;; + -t) + source templates/$2 + if [ $? -ne 0 ];then + echo "Cannot import the templates/$2" + exit 1 + fi + shift + ;; -T) timestamp=true ;; -d) DUMP_OUTPUT=true ;; -b) brief_test_summary=true;;
doc/group-names.txt +1 -0
@@ -118,6 +118,7 @@ selftest tests with fixed results, used to validate testing setup send btrfs send/receive shrinkfs decreasing the size of a filesystem shutdown FS_IOC_SHUTDOWN ioctl +smoketest Simple smoke tests snapshot btrfs snapshots soak long running soak tests whose runtime can be controlled directly by setting the SOAK_DURATION variable
templates/smoketest new file +16 -0
@@ -0,0 +1,16 @@ +##/bin/bash +# For infrequent filesystem developers who simply want to run a quick test +# of the most commonly used filesystem functionality, use this command: +# +# ./check -t smoketest <other config options> +# +# This template helps fstests to run several tests to exercise the file I/O, +# metadata, and crash recovery exercisers for four minutes apiece. This +# should complete in approximately 20 minutes. + +echo "**********************" +echo "* A Quick Smoke Test *" +echo "**********************" + +[ -z "$SOAK_DURATION" ] && SOAK_DURATION="4m" +GROUP_LIST="smoketest"
tests/generic/475 +1 -1
@@ -12,7 +12,7 @@ # testing efforts. # . ./common/preamble -_begin_fstest shutdown auto log metadata eio recoveryloop +_begin_fstest shutdown auto log metadata eio recoveryloop smoketest # Override the default cleanup function. _cleanup()
tests/generic/476 +1 -1
@@ -8,7 +8,7 @@ # bugs in the write path. # . ./common/preamble -_begin_fstest auto rw long_rw stress soak +_begin_fstest auto rw long_rw stress soak smoketest # Override the default cleanup function. _cleanup()
tests/generic/521 +1 -1
@@ -7,7 +7,7 @@ # Long-soak directio fsx test # . ./common/preamble -_begin_fstest soak long_rw +_begin_fstest soak long_rw smoketest # Import common functions. . ./common/filter
tests/generic/522 +1 -1
@@ -7,7 +7,7 @@ # Long-soak buffered fsx test # . ./common/preamble -_begin_fstest soak long_rw +_begin_fstest soak long_rw smoketest # Import common functions. . ./common/filter
tests/generic/642 +1 -1
@@ -8,7 +8,7 @@ # bugs in the xattr code. # . ./common/preamble -_begin_fstest auto soak attr long_rw stress +_begin_fstest auto soak attr long_rw stress smoketest _cleanup() {
Signature · 1 line
2.40.1
Quoted 97 lines · Zorro Lang
Darrick suggests that fstests can provide a simple smoketest, by running several generic filesystem smoke testing for five minutes apiece. Since there are only five smoke tests, this is effectively a 16min super-quick test.
With gcov enabled, running these tests yields about ~75% coverage for iomap and ~60% for xfs; or ~50% for ext4 and ~75% for ext4; and ~45% for btrfs. Coverage was about ~65% for the pagecache.
To implement that, this patch add a new "-t" option to ./check, and a new directory "template" under xfstests/, then we can have smoketest template, also can have more other testing templates.
Signed-off-by Darrick J. Wong <[email protected]>Signed-off-by Zorro Lang <[email protected]>---
Hi,
This patch uses another way to achieve the smoketest requirement[1]. When I reviewed the patch from Darrick [2], I thought "smoketest" might not be the last one requirement likes that. I don't want to give each kind of tests a separated option. But we might can leave a hook for more fs-devel who have good testing templates for fstests.
Although we have test groups, but those group names are too complex for the users who not always use fstests. So I'm thinking about providing some simple templates to run fstests, these templates base on test groups and some fstests global parameters, help users to know what kind of test they can do.
Feel free to discuss, and if most of you prefer the original patch [2], I'll also think about merging the original one :)
Thanks,
Zorro[1]
https://lore.kernel.org/fstests/20230726145441.lbzzokwigrztimyq@zlang-mailbox/T/#mabc0de98699f1b877c87caccb13809c9283c0606
[2]
https://lore.kernel.org/fstests/169033660570.3222210.3010411210438664310.stgit@frogsfrogsfrogs/T/#ucheck | 8 ++++++++ doc/group-names.txt | 1 + templates/smoketest | 16 ++++++++++++++++ tests/generic/475 | 2 +- tests/generic/476 | 2 +- tests/generic/521 | 2 +- tests/generic/522 | 2 +- tests/generic/642 | 2 +- 8 files changed, 30 insertions(+), 5 deletions(-) create mode 100644 templates/smoketestcheck +8
-0@@ -335,6 +335,14 @@ while [ $# -gt 0 ]; do ;; -i) iterations=$2; shift ;; -I) iterations=$2; istop=true; shift ;; + -t) + source templates/$2 + if [ $? -ne 0 ];then + echo "Cannot import the templates/$2" + exit 1 + fi + shift + ;; -T) timestamp=true ;; -d) DUMP_OUTPUT=true ;; -b) brief_test_summary=true;;doc/group-names.txt +1
-0@@ -118,6 +118,7 @@ selftest tests with fixed results, used to validate testing setup send btrfs send/receive shrinkfs decreasing the size of a filesystem shutdown FS_IOC_SHUTDOWN ioctl +smoketest Simple smoke tests snapshot btrfs snapshots soak long running soak tests whose runtime can be controlled directly by setting the SOAK_DURATION variabletemplates/smoketest new file +6
-0@@ -0,0 +1,16 @@ +##/bin/bash +# For infrequent filesystem developers who simply want to run a quick test +# of the most commonly used filesystem functionality, use this command: +# +# ./check -t smoketest <other config options> +#
templates/smoketest line 16
Quoted 10 lines
templates/smoketest new file +10
-0+# This template helps fstests to run several tests to exercise the file I/O, +# metadata, and crash recovery exercisers for four minutes apiece. This +# should complete in approximately 20 minutes. + +echo "**********************" +echo "* A Quick Smoke Test *" +echo "**********************" + +[ -z "$SOAK_DURATION" ] && SOAK_DURATION="4m" +GROUP_LIST="smoketest"
Why not simply put
SOAK_DURATION=4m
GROUP_LIST=smoketest
in configs/smoketest.config and tell people to run
HOST_OPTIONS=configs/smoketest.config ./check
?
--D
Quoted 68 lines
tests/generic/475 +1
-1@@ -12,7 +12,7 @@ # testing efforts. # . ./common/preamble -_begin_fstest shutdown auto log metadata eio recoveryloop +_begin_fstest shutdown auto log metadata eio recoveryloop smoketest # Override the default cleanup function. _cleanup()tests/generic/476 +1
-1@@ -8,7 +8,7 @@ # bugs in the write path. # . ./common/preamble -_begin_fstest auto rw long_rw stress soak +_begin_fstest auto rw long_rw stress soak smoketest # Override the default cleanup function. _cleanup()tests/generic/521 +1
-1@@ -7,7 +7,7 @@ # Long-soak directio fsx test # . ./common/preamble -_begin_fstest soak long_rw +_begin_fstest soak long_rw smoketest # Import common functions. . ./common/filtertests/generic/522 +1
-1@@ -7,7 +7,7 @@ # Long-soak buffered fsx test # . ./common/preamble -_begin_fstest soak long_rw +_begin_fstest soak long_rw smoketest # Import common functions. . ./common/filtertests/generic/642 +1
-1@@ -8,7 +8,7 @@ # bugs in the xattr code. # . ./common/preamble -_begin_fstest auto soak attr long_rw stress +_begin_fstest auto soak attr long_rw stress smoketest _cleanup() {Signature · 1 line
2.40.1
Quoted 111 lines · Darrick J. Wong
Quoted 97 lines · Zorro Lang
Darrick suggests that fstests can provide a simple smoketest, by running several generic filesystem smoke testing for five minutes apiece. Since there are only five smoke tests, this is effectively a 16min super-quick test.
With gcov enabled, running these tests yields about ~75% coverage for iomap and ~60% for xfs; or ~50% for ext4 and ~75% for ext4; and ~45% for btrfs. Coverage was about ~65% for the pagecache.
To implement that, this patch add a new "-t" option to ./check, and a new directory "template" under xfstests/, then we can have smoketest template, also can have more other testing templates.
Signed-off-by Darrick J. Wong <[email protected]>Signed-off-by Zorro Lang <[email protected]>---
Hi,
This patch uses another way to achieve the smoketest requirement[1]. When I reviewed the patch from Darrick [2], I thought "smoketest" might not be the last one requirement likes that. I don't want to give each kind of tests a separated option. But we might can leave a hook for more fs-devel who have good testing templates for fstests.
Although we have test groups, but those group names are too complex for the users who not always use fstests. So I'm thinking about providing some simple templates to run fstests, these templates base on test groups and some fstests global parameters, help users to know what kind of test they can do.
Feel free to discuss, and if most of you prefer the original patch [2], I'll also think about merging the original one :)
Thanks,
Zorro[1]
https://lore.kernel.org/fstests/20230726145441.lbzzokwigrztimyq@zlang-mailbox/T/#mabc0de98699f1b877c87caccb13809c9283c0606
[2]
https://lore.kernel.org/fstests/169033660570.3222210.3010411210438664310.stgit@frogsfrogsfrogs/T/#ucheck | 8 ++++++++ doc/group-names.txt | 1 + templates/smoketest | 16 ++++++++++++++++ tests/generic/475 | 2 +- tests/generic/476 | 2 +- tests/generic/521 | 2 +- tests/generic/522 | 2 +- tests/generic/642 | 2 +- 8 files changed, 30 insertions(+), 5 deletions(-) create mode 100644 templates/smoketestcheck +8
-0@@ -335,6 +335,14 @@ while [ $# -gt 0 ]; do ;; -i) iterations=$2; shift ;; -I) iterations=$2; istop=true; shift ;; + -t) + source templates/$2 + if [ $? -ne 0 ];then + echo "Cannot import the templates/$2" + exit 1 + fi + shift + ;; -T) timestamp=true ;; -d) DUMP_OUTPUT=true ;; -b) brief_test_summary=true;;doc/group-names.txt +1
-0@@ -118,6 +118,7 @@ selftest tests with fixed results, used to validate testing setup send btrfs send/receive shrinkfs decreasing the size of a filesystem shutdown FS_IOC_SHUTDOWN ioctl +smoketest Simple smoke tests snapshot btrfs snapshots soak long running soak tests whose runtime can be controlled directly by setting the SOAK_DURATION variabletemplates/smoketest new file +6
-0@@ -0,0 +1,16 @@ +##/bin/bash +# For infrequent filesystem developers who simply want to run a quick test +# of the most commonly used filesystem functionality, use this command: +# +# ./check -t smoketest <other config options> +#templates/smoketest line 16
Quoted 10 lines
templates/smoketest new file +10
-0+# This template helps fstests to run several tests to exercise the file I/O, +# metadata, and crash recovery exercisers for four minutes apiece. This +# should complete in approximately 20 minutes. + +echo "**********************" +echo "* A Quick Smoke Test *" +echo "**********************" + +[ -z "$SOAK_DURATION" ] && SOAK_DURATION="4m" +GROUP_LIST="smoketest"Why not simply put
Quoted 8 lines
SOAK_DURATION=4m
GROUP_LIST=smoketestin
configs/smoketest.configand tell people to runHOST_OPTIONS=configs/smoketest.config ./check
?
Hmm... I think if replace the HOST_OPTIONS, we need to write all TEST_DEV, TEST_DIR, SCRATCH_DEV .... into the new HOST_OPTIONS to, to replace the default local.config :
# HOST_OPTIONS=templates/smoketest ./check
Warning: need to define parameters for host hp-dl380pg8-01
or set variables:
TEST_DIR TEST_DEV
We can't provide a common test template with known device names, fs type, and mkfs or mount options and so on, due to these things are prepared and specified by users. But we can recommend known test groups and some general parameters.
Thanks,
Zorro
Quoted 72 lines
--D
Quoted 68 lines
tests/generic/475 +1
-1@@ -12,7 +12,7 @@ # testing efforts. # . ./common/preamble -_begin_fstest shutdown auto log metadata eio recoveryloop +_begin_fstest shutdown auto log metadata eio recoveryloop smoketest # Override the default cleanup function. _cleanup()tests/generic/476 +1
-1@@ -8,7 +8,7 @@ # bugs in the write path. # . ./common/preamble -_begin_fstest auto rw long_rw stress soak +_begin_fstest auto rw long_rw stress soak smoketest # Override the default cleanup function. _cleanup()tests/generic/521 +1
-1@@ -7,7 +7,7 @@ # Long-soak directio fsx test # . ./common/preamble -_begin_fstest soak long_rw +_begin_fstest soak long_rw smoketest # Import common functions. . ./common/filtertests/generic/522 +1
-1@@ -7,7 +7,7 @@ # Long-soak buffered fsx test # . ./common/preamble -_begin_fstest soak long_rw +_begin_fstest soak long_rw smoketest # Import common functions. . ./common/filtertests/generic/642 +1
-1@@ -8,7 +8,7 @@ # bugs in the xattr code. # . ./common/preamble -_begin_fstest auto soak attr long_rw stress +_begin_fstest auto soak attr long_rw stress smoketest _cleanup() {Signature · 1 line
2.40.1