Skip to contentSkip to conversation

generic: add a less thorough testing mode for fsync-err program

PATCH Superseded [email protected] 2 participants 3 messages Jun 29, 2017 – Jul 17, 2017

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 generic: add a less thorough testing mode for fsync-err program

From: Jeff Layton <[email protected]>

Currently we just have this test run on a whitelist of filesystems, but it would be best to be able to run it on all of them. The problem is that a lot of filesystems basically shut down once they hit metadata errors.

Allow the fsync-err testcase to operate in two different modes. One mode just does basic testing to ensure that we get an error back on all fd's when we fsync. The other does a more thorough test to ensure that we get back 0 on subsequent fsyncs when there hasn't been any write activity.

For now, we just opt-in to the more thorough testing on certain
filesystems: xfs, ext3 and ext4 on the generic test. All other filesystems will run in simple mode.

---
 common/rc         |  9 +++++++++
 src/fsync-err.c   | 51 +++++++++++++++++++++++++++++++++------------------
 tests/generic/441 | 22 +++++++++++++++-------
 3 files changed, 57 insertions(+), 25 deletions(-)
common/rc +9 -0
index 2972f89e9527..83675364cf24 100644
@@ -1738,6 +1738,15 @@ _require_test()
     touch ${RESULT_DIR}/require_test
 }
 
+_has_logdev()
+{
+	ret=0
+	[ -z "$SCRATCH_LOGDEV" -o ! -b "$SCRATCH_LOGDEV" ] &&
+	[ "$USE_EXTERNAL" != yes ] && ret=1
+
+	return $ret
+}
+
 # this test needs a logdev
 #
 _require_logdev()
src/fsync-err.c +33 -18

88 lines not on this page · raw message

tests/generic/441 +15 -7
index 2215b64db9a7..e8244224e097 100755
@@ -45,15 +45,23 @@ _cleanup()
 . ./common/dmerror
 
 # real QA test starts here
-_supported_fs ext2 ext3 ext4 xfs
 _supported_os Linux
 _require_scratch
 
-# Generally, we want to avoid journal errors in this test. Ensure that
-# journalled fs' have a logdev.
-if [ "$FSTYP" != "ext2" ]; then
-	_require_logdev
-fi
+# Generally, we want to avoid journal errors on the extended testcase. Only
+# set the -r flag if we have a logdev
+sflag='-s'
+case $FSTYP in
+	btrfs)
+		_notrun "btrfs has a specialized test for this"
+		;;
+	ext3|ext4|xfs)
+		# Do the more thorough test if we have a logdev
+		_has_logdev && sflag=''
+		;;
+	*)
+		;;
+esac
 
 _require_dm_target error
 _require_test_program fsync-err
@@ -70,7 +78,7 @@ _require_fs_space $SCRATCH_MNT 65536
 
 testfile=$SCRATCH_MNT/fsync-err-test
 
-$here/src/fsync-err -d $here/src/dmerror $testfile
+$here/src/fsync-err $sflag -d $here/src/dmerror $testfile
 
 # success, all done
 _dmerror_load_working_table
Signature · 1 line
2.13.0
#1 lore raw
Quoted 29 lines · On Thu, Jun 29, 2017 at 09:29:47AM -0400, [email protected] wrote:

From: Jeff Layton <[email protected]>

Currently we just have this test run on a whitelist of filesystems, but it would be best to be able to run it on all of them. The problem is that a lot of filesystems basically shut down once they hit metadata errors.

Allow the fsync-err testcase to operate in two different modes. One mode just does basic testing to ensure that we get an error back on all fd's when we fsync. The other does a more thorough test to ensure that we get back 0 on subsequent fsyncs when there hasn't been any write activity.

For now, we just opt-in to the more thorough testing on certain
filesystems: xfs, ext3 and ext4 on the generic test. All other filesystems will run in simple mode.

---
 common/rc         |  9 +++++++++
 src/fsync-err.c   | 51 +++++++++++++++++++++++++++++++++------------------
 tests/generic/441 | 22 +++++++++++++++-------
 3 files changed, 57 insertions(+), 25 deletions(-)
common/rc +0 -0
index 2972f89e9527..83675364cf24 100644

common/rc _require_test() · line 1745

Quoted 8 lines
common/rc +5 -0
@@ -1738,6 +1738,15 @@ _require_test()
     touch ${RESULT_DIR}/require_test
 }
 
+_has_logdev()
+{
+	ret=0
+	[ -z "$SCRATCH_LOGDEV" -o ! -b "$SCRATCH_LOGDEV" ] &&
+	[ "$USE_EXTERNAL" != yes ] && ret=1

This doesn't seem right. _has_logdev = ($SCRATCH_LOGDEV is block device) && ($USE_EXTERNAL set to yes), so we should set ret=1 if any of these conditions is not met. But this code returns 0 (true) if I only have SCRATCH_LOGDEV set but not USE_EXTERNAL.

And once we have _has_logdev, I think we can refactor _require_logdev to use _has_logdev too.

Quoted 115 lines
common/rc +4 -0
+
+	return $ret
+}
+
 # this test needs a logdev
 #
 _require_logdev()
src/fsync-err.c +33 -18
index 5b3bdd3ada07..4b0205cf2fd4 100644
@@ -13,6 +13,7 @@
 #include <string.h>
 #include <unistd.h>
 #include <getopt.h>
+#include <stdbool.h>
 
 /*
  * btrfs has a fixed stripewidth of 64k, so we need to write enough data to
@@ -25,7 +26,7 @@
 
 static void usage()
 {
-	printf("Usage: fsync-err [ -b bufsize ] [ -n num_fds ] -d dmerror path <filename>\n");
+	printf("Usage: fsync-err [ -b bufsize ] [ -n num_fds ] [ -s ] -d dmerror path <filename>\n");
 }
 
 int main(int argc, char **argv)
@@ -35,8 +36,9 @@ int main(int argc, char **argv)
 	char *dmerror_path = NULL;
 	char *cmdbuf;
 	size_t cmdsize, bufsize = DEFAULT_BUFSIZE;
+	bool simple_mode = false;
 
-	while ((i = getopt(argc, argv, "b:d:n:")) != -1) {
+	while ((i = getopt(argc, argv, "b:d:n:s")) != -1) {
 		switch (i) {
 		case 'b':
 			bufsize = strtol(optarg, &buf, 0);
@@ -55,6 +57,15 @@ int main(int argc, char **argv)
 				return 1;
 			}
 			break;
+		case 's':
+			/*
+			 * Many filesystems will continue to throw errors after
+			 * fsync has already advanced to the current error,
+			 * due to metadata writeback failures or other
+			 * issues. Allow those fs' to opt out of more thorough
+			 * testing.
+			 */
+			simple_mode = true;
 		}
 	}
 
@@ -154,16 +165,18 @@ int main(int argc, char **argv)
 		}
 	}
 
-	for (i = 0; i < numfds; ++i) {
-		ret = fsync(fd[i]);
-		if (ret < 0) {
-			/*
-			 * We did a failed write and fsync on each fd before.
-			 * Now the error should be clear since we've not done
-			 * any writes since then.
-			 */
-			printf("Third fsync on fd[%d] failed: %m\n", i);
-			return 1;
+	if (!simple_mode) {
+		for (i = 0; i < numfds; ++i) {
+			ret = fsync(fd[i]);
+			if (ret < 0) {
+				/*
+				 * We did a failed write and fsync on each fd before.
+				 * Now the error should be clear since we've not done
+				 * any writes since then.
+				 */
+				printf("Third fsync on fd[%d] failed: %m\n", i);
+				return 1;
+			}
 		}
 	}
 
@@ -185,12 +198,14 @@ int main(int argc, char **argv)
 		return 1;
 	}
 
-	for (i = 0; i < numfds; ++i) {
-		ret = fsync(fd[i]);
-		if (ret < 0) {
-			/* The error should still be clear */
-			printf("fsync after healing device on fd[%d] failed: %m\n", i);
-			return 1;
+	if (!simple_mode) {
+		for (i = 0; i < numfds; ++i) {
+			ret = fsync(fd[i]);
+			if (ret < 0) {
+				/* The error should still be clear */
+				printf("fsync after healing device on fd[%d] failed: %m\n", i);
+				return 1;
+			}
 		}
 	}
 
tests/generic/441 +0 -1
index 2215b64db9a7..e8244224e097 100755
@@ -45,15 +45,23 @@ _cleanup()
 . ./common/dmerror
 
 # real QA test starts here
-_supported_fs ext2 ext3 ext4 xfs
 _supported_os Linux

tests/generic/441 _cleanup() · line 53

Quoted 10 lines
tests/generic/441 +3 -5
 _require_scratch
 
-# Generally, we want to avoid journal errors in this test. Ensure that
-# journalled fs' have a logdev.
-if [ "$FSTYP" != "ext2" ]; then
-	_require_logdev
-fi
+# Generally, we want to avoid journal errors on the extended testcase. Only
+# set the -r flag if we have a logdev
+sflag='-s'

This part seems confusing, comments said "set the -r flag" but '-s' is actually used.

Quoted 15 lines
tests/generic/441 +11 -0
+case $FSTYP in
+	btrfs)
+		_notrun "btrfs has a specialized test for this"
+		;;
+	ext3|ext4|xfs)
+		# Do the more thorough test if we have a logdev
+		_has_logdev && sflag=''
+		;;
+	*)
+		;;
+esac
 
 _require_dm_target error
 _require_test_program fsync-err

tests/generic/441 _require_fs_space $SCRATCH_MNT 65536 · line 81

Quoted 5 lines
tests/generic/441 +1 -1
@@ -70,7 +78,7 @@ _require_fs_space $SCRATCH_MNT 65536
 
 testfile=$SCRATCH_MNT/fsync-err-test
 
-$here/src/fsync-err -d $here/src/dmerror $testfile
+$here/src/fsync-err $sflag -d $here/src/dmerror $testfile

Better to append this command to $seqres.full too for debug purpose, so we know if we're running with or without '-s' option.

Thanks,
Eryu

Quoted 6 lines
tests/generic/441 +0 -0
 # success, all done
 _dmerror_load_working_table
Signature · 1 line
2.13.0
#2 lore raw

PATCH v2 1/1 generic: add a less thorough testing mode for fsync-err program

In reply to [email protected]

From: Jeff Layton <[email protected]>

Currently we just have this test run on a whitelist of filesystems, but it would be best to be able to run it on all of them. The problem is that a lot of filesystems basically shut down once they hit metadata errors.

Allow the fsync-err testcase to operate in two different modes. One mode just does basic testing to ensure that we get an error back on all fd's when we fsync. The other does a more thorough test to ensure that we get back 0 on subsequent fsyncs when there hasn't been any write activity.

For now, we just opt-in to the more thorough testing on certain
filesystems: xfs, ext3 and ext4 on the generic test. All other filesystems will run in simple mode.

---
 common/rc         |  9 +++++++++
 src/fsync-err.c   | 51 +++++++++++++++++++++++++++++++++------------------
 tests/generic/441 | 23 ++++++++++++++++-------
 3 files changed, 58 insertions(+), 25 deletions(-)

v2: fix _has_logdev test, append command to seqres.full, fix comments

common/rc +9 -0
index fa1314c649cf..737b773ec28c 100644
@@ -1738,6 +1738,15 @@ _require_test()
     touch ${RESULT_DIR}/require_test
 }
 
+_has_logdev()
+{
+	ret=0
+	[ -z "$SCRATCH_LOGDEV" -o ! -b "$SCRATCH_LOGDEV" ] && ret=1
+	[ "$USE_EXTERNAL" != yes ] && ret=1
+
+	return $ret
+}
+
 # this test needs a logdev
 #
 _require_logdev()
src/fsync-err.c +33 -18

88 lines not on this page · raw message

tests/generic/441 +16 -7
index 2215b64db9a7..075d87723ca1 100755
@@ -45,15 +45,23 @@ _cleanup()
 . ./common/dmerror
 
 # real QA test starts here
-_supported_fs ext2 ext3 ext4 xfs
 _supported_os Linux
 _require_scratch
 
-# Generally, we want to avoid journal errors in this test. Ensure that
-# journalled fs' have a logdev.
-if [ "$FSTYP" != "ext2" ]; then
-	_require_logdev
-fi
+# Generally, we want to avoid journal errors on the extended testcase. Only
+# unset the -s flag if we have a logdev
+sflag='-s'
+case $FSTYP in
+	btrfs)
+		_notrun "btrfs has a specialized test for this"
+		;;
+	ext3|ext4|xfs)
+		# Do the more thorough test if we have a logdev
+		_has_logdev && sflag=''
+		;;
+	*)
+		;;
+esac
 
 _require_dm_target error
 _require_test_program fsync-err
@@ -70,7 +78,8 @@ _require_fs_space $SCRATCH_MNT 65536
 
 testfile=$SCRATCH_MNT/fsync-err-test
 
-$here/src/fsync-err -d $here/src/dmerror $testfile
+echo "$here/src/fsync-err $sflag -d $here/src/dmerror $testfile" >> $seqres.full
+$here/src/fsync-err $sflag -d $here/src/dmerror $testfile
 
 # success, all done
 _dmerror_load_working_table
Signature · 1 line
2.13.0