Skip to contentSkip to conversation

create _require_metadata_journaling, and add to tests that need it

PATCH Superseded Eric Sandeen 5 participants 6 messages Feb 23, 2015 – Feb 24, 2015

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 create _require_metadata_journaling, and add to tests that need it

Many tests use dm_flakey to trigger log replay, but for filesystems that don't support metadata journaling, this causes failures when it shouldn't. (i.e. we can hardly test log replay if there is no log).

For some tests they actually sync everything we care about, and find inconsistencies elsewhere, but I erred on the side of simply not running the test in most cases.

---

common/rc +29 -0
index 7449a1d..015d2f5 100644
@@ -2340,6 +2340,35 @@ _require_norecovery()
 	_scratch_unmount
 }
 
+# Does this filesystem support metadata journaling?
+# We exclude ones here that don't; otherwise we assume
+# that it does, so the test will run, fail, and motivate
+# someone to update this test for a new filesystem.
+#
+# It's possible that TEST_DEV and SCRATCH_DEV have different
+# features (it'd be odd, but possible) so check $TEST_DEV
+# by default, but we can optionaly pass any dev we want.
+_require_metadata_journaling()
+{
+	if [ -z $1 ]; then
+		DEV=$TEST_DEV
+	else
+		DEV=$1
+	fi
+
+	case "$FSTYP" in
+	ext2|vfat|msdos)
+		_notrun "$FSTYP does not support metadata journaling"
+		;;
+	ext4)
+		# ext4 could be mkfs'd without a journal...
+		_require_dumpe2fs
+		$DUMPE2FS_PROG -h $DEV | grep has_journal || \
+			_notrun "$FSTYP on $DEV not configured with metadata journaling"
+		;;
+	esac
+}
+
 # Does fiemap support?
 _require_fiemap()
 {
tests/generic/034 +1 -0
index 4ec1db8..966b3d2 100755
@@ -53,6 +53,7 @@ _supported_os Linux
 _need_to_be_root
 _require_scratch
 _require_dm_flakey
+_require_metadata_journaling $SCRATCH_DEV
 
 rm -f $seqres.full
 
tests/generic/040 +1 -0
index 5f10f48..c841fbc 100755
@@ -62,6 +62,7 @@ _supported_os Linux
 _need_to_be_root
 _require_scratch
 _require_dm_flakey
+_require_metadata_journaling $SCRATCH_DEV
 
 rm -f $seqres.full
 
tests/generic/041 +1 -0
index 36a6f42..f38b662 100755
@@ -66,6 +66,7 @@ _supported_os Linux
 _need_to_be_root
 _require_scratch
 _require_dm_flakey
+_require_metadata_journaling $SCRATCH_DEV
 
 rm -f $seqres.full
 
tests/generic/056 +1 -0
index 9ec00e3..8bb1522 100755
@@ -55,6 +55,7 @@ _supported_os Linux
 _need_to_be_root
 _require_scratch
 _require_dm_flakey
+_require_metadata_journaling $SCRATCH_DEV
 
 rm -f $seqres.full
 
tests/generic/057 +1 -0
index 4c0ffd1..3b9f89e 100755
@@ -55,6 +55,7 @@ _supported_os Linux
 _need_to_be_root
 _require_scratch
 _require_dm_flakey
+_require_metadata_journaling $SCRATCH_DEV
 
 rm -f $seqres.full
 
tests/generic/311 +1 -0
index 85e52e8..d21b6eb 100755
@@ -56,6 +56,7 @@ _supported_os Linux
 _need_to_be_root
 _require_scratch_nocheck
 _require_dm_flakey
+_require_metadata_journaling $SCRATCH_DEV
 
 # xfs_io is not required for this test, but it's the best way to verify
 # the test system supports fallocate() for allocation
tests/generic/321 +1 -0
index 3bd6b12..c821a23 100755
@@ -45,6 +45,7 @@ _supported_os Linux
 _need_to_be_root
 _require_scratch_nocheck
 _require_dm_flakey
+_require_metadata_journaling $SCRATCH_DEV
 
 rm -f $seqres.full
 
tests/generic/322 +1 -0
index 3ec2387..4c0edf6 100755
@@ -45,6 +45,7 @@ _supported_os Linux
 _need_to_be_root
 _require_scratch_nocheck
 _require_dm_flakey
+_require_metadata_journaling $SCRATCH_DEV
 
 rm -f $seqres.full
 
#1 lore raw
Quoted 19 lines · Eric Sandeen

Many tests use dm_flakey to trigger log replay, but for filesystems that don't support metadata journaling, this causes failures when it shouldn't. (i.e. we can hardly test log replay if there is no log).

For some tests they actually sync everything we care about, and find inconsistencies elsewhere, but I erred on the side of simply not running the test in most cases.

---

common/rc +0 -0
index 7449a1d..015d2f5 100644
@@ -2340,6 +2340,35 @@ _require_norecovery()
 	_scratch_unmount

common/rc _require_norecovery() · line 2350

Quoted 10 lines
common/rc +8 -0
 }
 
+# Does this filesystem support metadata journaling?
+# We exclude ones here that don't; otherwise we assume
+# that it does, so the test will run, fail, and motivate
+# someone to update this test for a new filesystem.
+#
+# It's possible that TEST_DEV and SCRATCH_DEV have different
+# features (it'd be odd, but possible) so check $TEST_DEV
+# by default, but we can optionaly pass any dev we want.

Use all 80 columns ;)

Quoted 9 lines
common/rc +9 -0
+_require_metadata_journaling()
+{
+	if [ -z $1 ]; then
+		DEV=$TEST_DEV
+	else
+		DEV=$1
+	fi
+
+	case "$FSTYP" in

common/rc _require_norecovery() · line 2369

Quoted 10 lines
common/rc +10 -0
+	ext2|vfat|msdos)
+		_notrun "$FSTYP does not support metadata journaling"
+		;;
+	ext4)
+		# ext4 could be mkfs'd without a journal...
+		_require_dumpe2fs
+		$DUMPE2FS_PROG -h $DEV | grep has_journal || \
+			_notrun "$FSTYP on $DEV not configured with metadata journaling"
+		;;
+	esac

And the default case?

Otherwise looks fine.

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#2 lore raw

PATCH v2 1/1 create _require_metadata_journaling, and add to tests that need it

In reply to Eric Sandeen

Many tests use dm_flakey to trigger log replay, but for filesystems that don't support metadata journaling, this causes failures when it shouldn't. (i.e. we can hardly test log replay if there is no log).

For some tests they actually sync everything we care about, and find inconsistencies elsewhere, but I erred on the side of simply not running the test in most cases.

---

v2: MOAR COLUMNS. Add default case.

common/rc +32 -0
index 7449a1d..243cb93 100644
@@ -2340,6 +2340,38 @@ _require_norecovery()
 	_scratch_unmount
 }
 
+# Does this filesystem support metadata journaling?
+# We exclude ones here that don't; otherwise we assume that it does, so the
+# test will run, fail, and motivate someone to update this test for a new
+# filesystem.
+#
+# It's possible that TEST_DEV and SCRATCH_DEV have different features (it'd be
+# odd, but possible) so check $TEST_DEV by default, but we can optionall pass
+# any dev we want.
+_require_metadata_journaling()
+{
+	if [ -z $1 ]; then
+		DEV=$TEST_DEV
+	else
+		DEV=$1
+	fi
+
+	case "$FSTYP" in
+	ext2|vfat|msdos)
+		_notrun "$FSTYP does not support metadata journaling"
+		;;
+	ext4)
+		# ext4 could be mkfs'd without a journal...
+		_require_dumpe2fs
+		$DUMPE2FS_PROG -h $DEV | grep has_journal || \
+			_notrun "$FSTYP on $DEV not configured with metadata journaling"
+		;;
+	*)
+		# by default we pass; if you need to, add your fs above!
+		;;
+	esac
+}
+
 # Does fiemap support?
 _require_fiemap()
 {
tests/generic/034 +1 -0
index 4ec1db8..966b3d2 100755
@@ -53,6 +53,7 @@ _supported_os Linux
 _need_to_be_root
 _require_scratch
 _require_dm_flakey
+_require_metadata_journaling $SCRATCH_DEV
 
 rm -f $seqres.full
 
tests/generic/040 +1 -0
index 5f10f48..c841fbc 100755
@@ -62,6 +62,7 @@ _supported_os Linux
 _need_to_be_root
 _require_scratch
 _require_dm_flakey
+_require_metadata_journaling $SCRATCH_DEV
 
 rm -f $seqres.full
 
tests/generic/041 +1 -0
index 36a6f42..f38b662 100755
@@ -66,6 +66,7 @@ _supported_os Linux
 _need_to_be_root
 _require_scratch
 _require_dm_flakey
+_require_metadata_journaling $SCRATCH_DEV
 
 rm -f $seqres.full
 
tests/generic/056 +1 -0
index 9ec00e3..8bb1522 100755
@@ -55,6 +55,7 @@ _supported_os Linux
 _need_to_be_root
 _require_scratch
 _require_dm_flakey
+_require_metadata_journaling $SCRATCH_DEV
 
 rm -f $seqres.full
 
tests/generic/057 +1 -0
index 4c0ffd1..3b9f89e 100755
@@ -55,6 +55,7 @@ _supported_os Linux
 _need_to_be_root
 _require_scratch
 _require_dm_flakey
+_require_metadata_journaling $SCRATCH_DEV
 
 rm -f $seqres.full
 
tests/generic/311 +1 -0
index 85e52e8..d21b6eb 100755
@@ -56,6 +56,7 @@ _supported_os Linux
 _need_to_be_root
 _require_scratch_nocheck
 _require_dm_flakey
+_require_metadata_journaling $SCRATCH_DEV
 
 # xfs_io is not required for this test, but it's the best way to verify
 # the test system supports fallocate() for allocation
tests/generic/321 +1 -0
index 3bd6b12..c821a23 100755
@@ -45,6 +45,7 @@ _supported_os Linux
 _need_to_be_root
 _require_scratch_nocheck
 _require_dm_flakey
+_require_metadata_journaling $SCRATCH_DEV
 
 rm -f $seqres.full
 
tests/generic/322 +1 -0
index 3ec2387..4c0edf6 100755
@@ -45,6 +45,7 @@ _supported_os Linux
 _need_to_be_root
 _require_scratch_nocheck
 _require_dm_flakey
+_require_metadata_journaling $SCRATCH_DEV
 
 rm -f $seqres.full
 
#3 lore raw

PATCH v3 1/1 create _require_metadata_journaling, and add to tests that need it

Many tests use dm_flakey to trigger log replay, but for filesystems that don't support metadata journaling, this causes failures when it shouldn't. (i.e. we can hardly test log replay if there is no log, and the subsequent filesystem check will turn up errors).

For some tests they actually sync everything we care about, and find inconsistencies elsewhere, but I erred on the side of simply not running the test in most cases.

---

V2: MOAR COLUMNS. Add default case.
V3: whoops, dumpe2fs &>1 | grep -q so it doesn't break output when found

common/rc +32 -0
index 7449a1d..243cb93 100644
@@ -2340,6 +2340,38 @@ _require_norecovery()
 	_scratch_unmount
 }
 
+# Does this filesystem support metadata journaling?
+# We exclude ones here that don't; otherwise we assume that it does, so the
+# test will run, fail, and motivate someone to update this test for a new
+# filesystem.
+#
+# It's possible that TEST_DEV and SCRATCH_DEV have different features (it'd be
+# odd, but possible) so check $TEST_DEV by default, but we can optionall pass
+# any dev we want.
+_require_metadata_journaling()
+{
+	if [ -z $1 ]; then
+		DEV=$TEST_DEV
+	else
+		DEV=$1
+	fi
+
+	case "$FSTYP" in
+	ext2|vfat|msdos)
+		_notrun "$FSTYP does not support metadata journaling"
+		;;
+	ext4)
+		# ext4 could be mkfs'd without a journal...
+		_require_dumpe2fs
+		$DUMPE2FS_PROG -h $DEV 2>&1 | grep -q has_journal || \
+			_notrun "$FSTYP on $DEV not configured with metadata journaling"
+		;;
+	*)
+		# by default we pass; if you need to, add your fs above!
+		;;
+	esac
+}
+
 # Does fiemap support?
 _require_fiemap()
 {
tests/generic/034 +1 -0
index 4ec1db8..966b3d2 100755
@@ -53,6 +53,7 @@ _supported_os Linux
 _need_to_be_root
 _require_scratch
 _require_dm_flakey
+_require_metadata_journaling $SCRATCH_DEV
 
 rm -f $seqres.full
 
tests/generic/040 +1 -0
index 5f10f48..c841fbc 100755
@@ -62,6 +62,7 @@ _supported_os Linux
 _need_to_be_root
 _require_scratch
 _require_dm_flakey
+_require_metadata_journaling $SCRATCH_DEV
 
 rm -f $seqres.full
 
tests/generic/041 +1 -0
index 36a6f42..f38b662 100755
@@ -66,6 +66,7 @@ _supported_os Linux
 _need_to_be_root
 _require_scratch
 _require_dm_flakey
+_require_metadata_journaling $SCRATCH_DEV
 
 rm -f $seqres.full
 
tests/generic/056 +1 -0
index 9ec00e3..8bb1522 100755
@@ -55,6 +55,7 @@ _supported_os Linux
 _need_to_be_root
 _require_scratch
 _require_dm_flakey
+_require_metadata_journaling $SCRATCH_DEV
 
 rm -f $seqres.full
 
tests/generic/057 +1 -0
index 4c0ffd1..3b9f89e 100755
@@ -55,6 +55,7 @@ _supported_os Linux
 _need_to_be_root
 _require_scratch
 _require_dm_flakey
+_require_metadata_journaling $SCRATCH_DEV
 
 rm -f $seqres.full
 
tests/generic/311 +1 -0
index 85e52e8..d21b6eb 100755
@@ -56,6 +56,7 @@ _supported_os Linux
 _need_to_be_root
 _require_scratch_nocheck
 _require_dm_flakey
+_require_metadata_journaling $SCRATCH_DEV
 
 # xfs_io is not required for this test, but it's the best way to verify
 # the test system supports fallocate() for allocation
tests/generic/321 +1 -0
index 3bd6b12..c821a23 100755
@@ -45,6 +45,7 @@ _supported_os Linux
 _need_to_be_root
 _require_scratch_nocheck
 _require_dm_flakey
+_require_metadata_journaling $SCRATCH_DEV
 
 rm -f $seqres.full
 
tests/generic/322 +1 -0
index 3ec2387..4c0edf6 100755
@@ -45,6 +45,7 @@ _supported_os Linux
 _need_to_be_root
 _require_scratch_nocheck
 _require_dm_flakey
+_require_metadata_journaling $SCRATCH_DEV
 
 rm -f $seqres.full
 
Reviewed-by Filipe Manana Tested-by Filipe Manana on v3 1/1
#4 lore raw
Quoted 10 lines · Eric Sandeen

Many tests use dm_flakey to trigger log replay, but for filesystems that don't support metadata journaling, this causes failures when it shouldn't. (i.e. we can hardly test log replay if there is no log, and the subsequent filesystem check will turn up errors).

For some tests they actually sync everything we care about, and find inconsistencies elsewhere, but I erred on the side of simply not running the test in most cases.

Works like a charm.
Thanks very much for this Eric.

Quoted 151 lines

---

V2: MOAR COLUMNS. Add default case.
V3: whoops, dumpe2fs &>1 | grep -q so it doesn't break output when found

common/rc +32 -0
index 7449a1d..243cb93 100644
@@ -2340,6 +2340,38 @@ _require_norecovery()
        _scratch_unmount
 }
 
+# Does this filesystem support metadata journaling?
+# We exclude ones here that don't; otherwise we assume that it does, so the
+# test will run, fail, and motivate someone to update this test for a new
+# filesystem.
+#
+# It's possible that TEST_DEV and SCRATCH_DEV have different features (it'd be
+# odd, but possible) so check $TEST_DEV by default, but we can optionall pass
+# any dev we want.
+_require_metadata_journaling()
+{
+       if [ -z $1 ]; then
+               DEV=$TEST_DEV
+       else
+               DEV=$1
+       fi
+
+       case "$FSTYP" in
+       ext2|vfat|msdos)
+               _notrun "$FSTYP does not support metadata journaling"
+               ;;
+       ext4)
+               # ext4 could be mkfs'd without a journal...
+               _require_dumpe2fs
+               $DUMPE2FS_PROG -h $DEV 2>&1 | grep -q has_journal || \
+                       _notrun "$FSTYP on $DEV not configured with metadata journaling"
+               ;;
+       *)
+               # by default we pass; if you need to, add your fs above!
+               ;;
+       esac
+}
+
 # Does fiemap support?
 _require_fiemap()
 {
tests/generic/034 +1 -0
index 4ec1db8..966b3d2 100755
@@ -53,6 +53,7 @@ _supported_os Linux
 _need_to_be_root
 _require_scratch
 _require_dm_flakey
+_require_metadata_journaling $SCRATCH_DEV
 
 rm -f $seqres.full
 
tests/generic/040 +1 -0
index 5f10f48..c841fbc 100755
@@ -62,6 +62,7 @@ _supported_os Linux
 _need_to_be_root
 _require_scratch
 _require_dm_flakey
+_require_metadata_journaling $SCRATCH_DEV
 
 rm -f $seqres.full
 
tests/generic/041 +1 -0
index 36a6f42..f38b662 100755
@@ -66,6 +66,7 @@ _supported_os Linux
 _need_to_be_root
 _require_scratch
 _require_dm_flakey
+_require_metadata_journaling $SCRATCH_DEV
 
 rm -f $seqres.full
 
tests/generic/056 +1 -0
index 9ec00e3..8bb1522 100755
@@ -55,6 +55,7 @@ _supported_os Linux
 _need_to_be_root
 _require_scratch
 _require_dm_flakey
+_require_metadata_journaling $SCRATCH_DEV
 
 rm -f $seqres.full
 
tests/generic/057 +1 -0
index 4c0ffd1..3b9f89e 100755
@@ -55,6 +55,7 @@ _supported_os Linux
 _need_to_be_root
 _require_scratch
 _require_dm_flakey
+_require_metadata_journaling $SCRATCH_DEV
 
 rm -f $seqres.full
 
tests/generic/311 +1 -0
index 85e52e8..d21b6eb 100755
@@ -56,6 +56,7 @@ _supported_os Linux
 _need_to_be_root
 _require_scratch_nocheck
 _require_dm_flakey
+_require_metadata_journaling $SCRATCH_DEV
 
 # xfs_io is not required for this test, but it's the best way to verify
 # the test system supports fallocate() for allocation
tests/generic/321 +1 -0
index 3bd6b12..c821a23 100755
@@ -45,6 +45,7 @@ _supported_os Linux
 _need_to_be_root
 _require_scratch_nocheck
 _require_dm_flakey
+_require_metadata_journaling $SCRATCH_DEV
 
 rm -f $seqres.full
 
tests/generic/322 +1 -1
index 3ec2387..4c0edf6 100755
@@ -45,6 +45,7 @@ _supported_os Linux
 _need_to_be_root
 _require_scratch_nocheck
 _require_dm_flakey
+_require_metadata_journaling $SCRATCH_DEV
 
 rm -f $seqres.full
 
 
 
--

To unsubscribe from this list: send the line "unsubscribe fstests" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Signature · 5 lines
Filipe David Manana,

"Reasonable men adapt themselves to the world.
 Unreasonable men adapt the world to themselves.
 That's why all progress depends on unreasonable men."
Reviewed-by on v1 1/1
#5 lore raw
In reply to Eric Sandeen
Quoted 10 lines · Eric Sandeen

Date: Mon, 23 Feb 2015 15:50:11 -0600
From: Eric Sandeen <[email protected]>
To: [email protected], Filipe Manana <[email protected]>
Subject: [PATCH] create _require_metadata_journaling,
and add to tests that need it

Many tests use dm_flakey to trigger log replay, but for filesystems that don't support metadata journaling, this causes failures when it shouldn't. (i.e. we can hardly test log replay if there is no log).

Quoted 5 lines

For some tests they actually sync everything we care about, and find inconsistencies elsewhere, but I erred on the side of simply not running the test in most cases.

I like that. Maybe the name _require_metadata_journaling() is not ideal since we have COW and log-structured fs which does not use journal. But I do not know what would be better name. Maybe _require_metadata_consistency but that's confusing, so maybe what you have is the best :)

Thanks!
-Lukas

Quoted 149 lines

---

common/rc +29 -0
index 7449a1d..015d2f5 100644
@@ -2340,6 +2340,35 @@ _require_norecovery()
 	_scratch_unmount
 }
 
+# Does this filesystem support metadata journaling?
+# We exclude ones here that don't; otherwise we assume
+# that it does, so the test will run, fail, and motivate
+# someone to update this test for a new filesystem.
+#
+# It's possible that TEST_DEV and SCRATCH_DEV have different
+# features (it'd be odd, but possible) so check $TEST_DEV
+# by default, but we can optionaly pass any dev we want.
+_require_metadata_journaling()
+{
+	if [ -z $1 ]; then
+		DEV=$TEST_DEV
+	else
+		DEV=$1
+	fi
+
+	case "$FSTYP" in
+	ext2|vfat|msdos)
+		_notrun "$FSTYP does not support metadata journaling"
+		;;
+	ext4)
+		# ext4 could be mkfs'd without a journal...
+		_require_dumpe2fs
+		$DUMPE2FS_PROG -h $DEV | grep has_journal || \
+			_notrun "$FSTYP on $DEV not configured with metadata journaling"
+		;;
+	esac
+}
+
 # Does fiemap support?
 _require_fiemap()
 {
tests/generic/034 +1 -0
index 4ec1db8..966b3d2 100755
@@ -53,6 +53,7 @@ _supported_os Linux
 _need_to_be_root
 _require_scratch
 _require_dm_flakey
+_require_metadata_journaling $SCRATCH_DEV
 
 rm -f $seqres.full
 
tests/generic/040 +1 -0
index 5f10f48..c841fbc 100755
@@ -62,6 +62,7 @@ _supported_os Linux
 _need_to_be_root
 _require_scratch
 _require_dm_flakey
+_require_metadata_journaling $SCRATCH_DEV
 
 rm -f $seqres.full
 
tests/generic/041 +1 -0
index 36a6f42..f38b662 100755
@@ -66,6 +66,7 @@ _supported_os Linux
 _need_to_be_root
 _require_scratch
 _require_dm_flakey
+_require_metadata_journaling $SCRATCH_DEV
 
 rm -f $seqres.full
 
tests/generic/056 +1 -0
index 9ec00e3..8bb1522 100755
@@ -55,6 +55,7 @@ _supported_os Linux
 _need_to_be_root
 _require_scratch
 _require_dm_flakey
+_require_metadata_journaling $SCRATCH_DEV
 
 rm -f $seqres.full
 
tests/generic/057 +1 -0
index 4c0ffd1..3b9f89e 100755
@@ -55,6 +55,7 @@ _supported_os Linux
 _need_to_be_root
 _require_scratch
 _require_dm_flakey
+_require_metadata_journaling $SCRATCH_DEV
 
 rm -f $seqres.full
 
tests/generic/311 +1 -0
index 85e52e8..d21b6eb 100755
@@ -56,6 +56,7 @@ _supported_os Linux
 _need_to_be_root
 _require_scratch_nocheck
 _require_dm_flakey
+_require_metadata_journaling $SCRATCH_DEV
 
 # xfs_io is not required for this test, but it's the best way to verify
 # the test system supports fallocate() for allocation
tests/generic/321 +1 -0
index 3bd6b12..c821a23 100755
@@ -45,6 +45,7 @@ _supported_os Linux
 _need_to_be_root
 _require_scratch_nocheck
 _require_dm_flakey
+_require_metadata_journaling $SCRATCH_DEV
 
 rm -f $seqres.full
 
tests/generic/322 +1 -1
index 3ec2387..4c0edf6 100755
@@ -45,6 +45,7 @@ _supported_os Linux
 _need_to_be_root
 _require_scratch_nocheck
 _require_dm_flakey
+_require_metadata_journaling $SCRATCH_DEV
 
 rm -f $seqres.full
 
 
--

To unsubscribe from this list: send the line "unsubscribe fstests" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html