Skip to contentSkip to conversation

fstests: btrfs: Check snapshot creation and deletion with dm-logwrites

PATCH Superseded Qu Wenruo 1 participant 1 message Aug 19, 2019 Btrfs file system

v2 of 4 Superseded Latest is v4 →

lore
Share & tools
lore

Qu Wenruo

author
#0 lore raw

PATCH v2 1/1 fstests: btrfs: Check snapshot creation and deletion with dm-logwrites

We have generic dm-logwrites with fsstress test case (generic/482), but it doesn't cover fs specific operations like btrfs snapshot creation and deletion.

Furthermore, that test is not heavy enough to bump btrfs tree height by its short runtime.

And finally, btrfs check doesn't consider dirty log as an error, unlike ext*/xfs, that's to say we don't need to mount the fs to replay the log, but just run btrfs check on the fs is enough.

So introduce a similar test case but for btrfs only.

The test case will stress btrfs by:
- Use small nodesize to bump tree height
- Create a base tree which is already high enough
- Trim tree blocks to find possible trim bugs
- Call snapshot creation and deletion along with fsstress

Also it includes certain workaround for btrfs:
- Use no-holes feature
To avoid missing hole file extents.
Although that behavior doesn't follow the on-disk format spec, it doesn't cause data loss. And will follow the new on-disk format spec of no-holes feature, so it's better to workaround it.

And an optimization for btrfs only:
- Use replay-log --fsck/--check command
Since dm-log-writes records bios sequentially, there is no way to locate certain entry unless we iterate all entries. This is becoming a big performance penalty if we replay certain a range, check the fs, then re-execute replay-log to replay another range.

We need to records the previous entry location, or we need to re-iterate all previous entries.

Thankfully, replay-log has already address it by providing --fsck and --check command, thus we don't need to break replay-log command.

Please note, for fast storage (e.g. fast NVME or unsafe cache mode), it's recommended to use log devices larger than 15G, or we can't record the full log of just 30s fsstress run.

---
For the log devices size problem, I have submitted dm-logwrites bio flag filter support, to filter out data bios.

But that is not yet merged into kernel, thus we need a large log device for short run.

Changelog:
- Better expression/words for comment
- Add requirement for no-holes features
- Use xattr to bump up tree height
So no need for max_inline mount option
- Coding style fixes for function definition
- Add -f for rm to avoid user alias setting
- Add new workload (update time stamp and create new files) for snapshot workload
- Remove an unnecessary sync call
- Get rid of wrong 2>&1 redirection
- Add to group "snapshot" and "stress"

---
 common/config               |   1 +
 common/dmlogwrites          |  28 +++++++
 src/log-writes/replay-log.c |   2 +
 tests/btrfs/192             | 149 ++++++++++++++++++++++++++++++++++++
 tests/btrfs/192.out         |   2 +
 tests/btrfs/group           |   1 +
 6 files changed, 183 insertions(+)
 create mode 100755 tests/btrfs/192
 create mode 100644 tests/btrfs/192.out
common/config +1 -0
index bd64be62..4c86a492 100644
@@ -183,6 +183,7 @@ export LOGGER_PROG="$(type -P logger)"
 export DBENCH_PROG="$(type -P dbench)"
 export DMSETUP_PROG="$(type -P dmsetup)"
 export WIPEFS_PROG="$(type -P wipefs)"
+export BLKDISCARD_PROG="$(type -P blkdiscard)"
 export DUMP_PROG="$(type -P dump)"
 export RESTORE_PROG="$(type -P restore)"
 export LVM_PROG="$(type -P lvm)"
common/dmlogwrites +28 -0
index ae2cbc6a..3292544a 100644
@@ -175,3 +175,31 @@ _log_writes_replay_log_range()
 		>> $seqres.full 2>&1
 	[ $? -ne 0 ] && _fail "replay failed"
 }
+
+# Replay and check each fua/flush (specified by $2) point.
+#
+# Since dm-log-writes records bio sequentially, even just replaying a range
+# still needs to iterate all records before the end point.
+# When number of records grows, it will be unacceptably slow, thus we need
+# to use relay-log itself to trigger fsck, avoid unnecessary seek.
+_log_writes_fast_replay_check()
+{
+	local check_point=$1
+	local blkdev=$2
+	local fsck_command
+
+	[ -z "$check_point" -o -z "$blkdev" ] && _fail \
+	"check_point and blkdev must be specified for _log_writes_fast_replay_check"
+
+	# Check program must not treat dirty log as an error.
+	# So only btrfs can use this feature
+	if [ $FSTYP = "btrfs" ]; then
+		fsck_command="$BTRFS_UTIL_PROG check $blkdev"
+	else
+		_notrun "fsck of $FSTYP reports dirty jounal/log as error, skipping test"
+	fi
+	$here/src/log-writes/replay-log --log $LOGWRITES_DEV \
+		--replay $blkdev --check $check_point --fsck "$fsck_command" \
+		2>&1 | tail -n 128 >> $seqres.full
+	[ $? -ne 0 ] && _fail "fsck failed during replay"
+}
src/log-writes/replay-log.c +2 -0
index 829b18e2..1e1cd524 100644
@@ -1,5 +1,6 @@
 // SPDX-License-Identifier: GPL-2.0
 #include <stdio.h>
+#include <errno.h>
 #include <unistd.h>
 #include <getopt.h>
 #include <stdlib.h>
@@ -375,6 +376,7 @@ int main(int argc, char **argv)
 				fprintf(stderr, "Fsck errored out on entry "
 					"%llu\n",
 					(unsigned long long)log->cur_entry - 1);
+				ret = -EUCLEAN;
 				break;
 			}
 		}
tests/btrfs/192 new file +149 -0

149 lines not on this page · raw message

tests/btrfs/192.out new file +2 -0
new file mode 100644
index 00000000..6779aa77
@@ -0,0 +1,2 @@
+QA output created by 192
+Silence is golden
tests/btrfs/group +1 -0
index 2474d43e..cab10d19 100644
@@ -194,3 +194,4 @@
 189 auto quick send clone
 190 auto quick replay balance qgroup
 191 auto quick send dedupe
+192 auto replay snapshot stress
Signature · 1 line
2.22.0