fstests: add dm-log-writes test and supporting code
Subsystems (2)
PATCH 1/2 fstests: add dm-log-writes test and supporting code
This patch adds the supporting code for using the dm-log-writes target. The bash stuff is similar to the dmflakey code, it just gives us functions to build and tear down a dm-log-writes target. We add a new LOGWRITES_DEV variable to take in the device we will use as the log and add checks for that.
I've rigged up fsx to have an integrity check mode. Basically it works like it normally works, but when it fsync()'s it marks the log with a unique mark and dumps it's buffer to a file with the mark in the filename. I did this with a system() call simply because it was the fastest. I can link the device-mapper libraries and do it programatically if that would be preferred, but this works pretty well.
The test itself just runs 200 ops and exits, then finds all of the good buffers in the directory we provided and replays up to the mark given, mounts the file system and compares the md5sum, unmounts and fsck's to check for metadata integrity. dm-log-writes will pretend to do discard and the replay tool will replay it properly depending on the underlying device, either by writing 0's or actually calling the discard ioctl, so I've enabled discard in the test for maximum fun.
This test relies on the supporting userspace code I've written for dm-logs-writes. It can be found here
https://github.com/josefbacik/log-writes.git
Thanks,
--- README | 2 + common/config | 1 + common/dmlogwrites | 80 ++++++++++++++++++++++++++++++ common/rc | 46 ++++++++++++++++++ ltp/fsx.c | 131 ++++++++++++++++++++++++++++++++++++++++++-------- tests/generic/326 | 130 +++++++++++++++++++++++++++++++++++++++++++++++++ tests/generic/326.out | 2 + tests/generic/group | 1 + 8 files changed, 374 insertions(+), 19 deletions(-) create mode 100644 common/dmlogwrites create mode 100644 tests/generic/326 create mode 100644 tests/generic/326.out
README +2 -0
@@ -78,6 +78,8 @@ Preparing system for tests (IRIX and Linux): added to the end of fsstresss and fsx invocations, respectively, in case you wish to exclude certain operational modes from these tests. + - setenv LOGWRITES_DEV to a block device to use for power fail + testing. - or add a case to the switch in common/config assigning these variables based on the hostname of your test
common/config +1 -0
@@ -191,6 +191,7 @@ export WIPEFS_PROG="`set_prog_path wipefs`" export DUMP_PROG="`set_prog_path dump`" export RESTORE_PROG="`set_prog_path restore`" export LVM_PROG="`set_prog_path lvm`" +export REPLAYLOG_PROG="`set_prog_path replay-log`" # Generate a comparable xfsprogs version number in the form of # major * 10000 + minor * 100 + release
common/dmlogwrites new file +80 -0
80 lines not on this page · raw message
common/rc +46 -0
58 lines not on this page · raw message
ltp/fsx.c +112 -19
255 lines not on this page · raw message
tests/generic/326 new file +130 -0
130 lines not on this page · raw message
tests/generic/326.out new file +2 -0
@@ -0,0 +1,2 @@ +QA output created by 326 +Silence is golden
tests/generic/group +1 -0
@@ -192,3 +192,4 @@ 323 auto aio stress 324 auto fsr quick 325 auto quick data log +326 auto log
Signature · 1 line
1.8.3.1
PATCH 2/2 fstests: btrfs balance with dm log writes test
This test runs fsstress+balance+defrag and then replays every FUA in the log and mounts, scrubs and then fscks the fs to make sure it does the balance recovery properly. Thanks,
--- tests/btrfs/093 | 135 ++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/093.out | 1 + tests/btrfs/group | 1 + 3 files changed, 137 insertions(+) create mode 100644 tests/btrfs/093 create mode 100644 tests/btrfs/093.out
tests/btrfs/093 new file +135 -0
@@ -0,0 +1,135 @@ +#! /bin/bash +# FSQA Test No. btrfs/093 +# +# Run btrfs balance and defrag operations simultaneously with fsstress +# running in background on top of dm-log-writes. +# +#----------------------------------------------------------------------- +# Copyright (C) 2015 Facebook. All rights reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +# +#----------------------------------------------------------------------- +# + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter +. ./common/dmlogwrites + +# real QA test starts here +_supported_fs btrfs +_supported_os Linux +# we check scratch dev after each loop +_need_to_be_root +_require_scratch_nocheck +_require_dm_log_writes + +rm -f $seqres.full + +_wait_balance() +{ + while [ 1 ] + do + $BTRFS_UTIL_PROG filesystem balance status $SCRATCH_MNT \ + | grep "No balance" >> $seqres.full + [ $? -eq 0 ] && break + sleep 1 + done +} + +run_test() +{ + args=`_scale_fsstress_args -p 20 -n 100 $FSSTRESS_AVOID -d $SCRATCH_MNT/stressdir` + echo "Run fsstress $args" >>$seqres.full + $FSSTRESS_PROG $args >/dev/null 2>&1 & + fsstress_pid=$! + + echo -n "Start balance worker: " >>$seqres.full + _btrfs_stress_balance $SCRATCH_MNT >/dev/null 2>&1 & + balance_pid=$! + echo "$balance_pid" >>$seqres.full + + echo -n "Start defrag worker: " >>$seqres.full + _btrfs_stress_defrag $SCRATCH_MNT $with_compress >/dev/null 2>&1 & + defrag_pid=$! + echo "$defrag_pid" >>$seqres.full + + echo "Wait for fsstress to exit and kill all background workers" >>$seqres.full + wait $fsstress_pid + kill $balance_pid $defrag_pid + wait + # wait for the balance and defrag operations to finish + while ps aux | grep "balance start" | grep -qv grep; do + sleep 1 + done + while ps aux | grep "btrfs filesystem defrag" | grep -qv grep; do + sleep 1 + done +} + +_init_log_writes + +_log_writes_mkfs >> $seqres.full 2>&1 + +_mount_log_writes + +run_test "$t" nocompress + +_unmount_log_writes +_log_writes_remove + +# Get the number of entries in the log +NUM_ENTRIES=$($REPLAYLOG_PROG --log $LOGWRITES_DEV --num-entries) + +# Start at the first FUA after the mkfs +ENTRY=$($REPLAYLOG_PROG --log $LOGWRITES_DEV --start-mark mkfs \ + --find --next-fua) + +while [ $ENTRY -lt $NUM_ENTRIES ]; +do + echo "Replaying to $ENTRY" >> $seqres.full + $REPLAYLOG_PROG --log $LOGWRITES_DEV --replay $SCRATCH_DEV --limit \ + $ENTRY > /dev/null 2>&1 + [ $? -ne 0 ] && _fatal "replay failed" + btrfsck $SCRATCH_DEV >> $seqres.full 2>&1 || _fatal "btrfsck failed" + _scratch_mount || _fatal "mount failed" + _wait_balance + $BTRFS_UTIL_PROG scrub start -B $SCRATCH_MNT >> $seqres.full 2>&1 + [ $? -ne 0 ] && _fatal "scrub failed" + _scratch_unmount + btrfsck $SCRATCH_DEV >> $seqres.full 2>&1 || _fatal "btrfsck failed" + let ENTRY+=1 + ENTRY=$($REPLAYLOG_PROG --find --start-entry $ENTRY --log \ + $LOGWRITES_DEV --next-fua) +done + +status=0 +exit +
tests/btrfs/093.out new file +1 -0
@@ -0,0 +1 @@ +QA output created by 093
tests/btrfs/group +1 -0
@@ -94,3 +94,4 @@ 090 auto quick metadata 091 auto quick qgroup 092 auto quick send +093 auto log
Signature · 1 line
1.8.3.1