fstests: concurrent test execution
PATCH 13/40 fstests: clean up loop device instantiation
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
Lots of tests do there own special thing with loop devices rather than using _create_loop_device() and _destroy_loop_device(). This oftens means they do not clean up after themselves properly, leaving stale loop devices around that result in unmountable test or scratch devices. This is common when tests are killed by user interrupt.
Even the tests that do use _destroy_loop_device and try to clean up often do it incorrectly, leading to spurious error messages.
Some tests try to use dynamic instantiation via "mount -o loop", but then don't clean up in the correct order or hack around to find the loop device that was instantiated because the test needs to know the instantiated device name
Clean this up by converting all the tests to use _create_loop_device() and _destroy_loop_device(). In all the tests, use the variable "loop_dev" for the device consistently. In _destroy_loop_device(), test that a device name has been passed so that we don't try to clean up the same device twice (e.g. once before test exit and again from the _cleanup() function). When we destroy a loop device, unset the variable used to hold the loop device name so that we don't try to destroy it twice.
This results in much more reliable cleanup and clean exit from fstests when killed by the user.
--- common/metadump | 32 ++++++++++++-------------------- tests/generic/067 | 11 ++++++++--- tests/generic/361 | 6 +++++- tests/generic/563 | 14 +++++++++----- tests/generic/564 | 12 ++++++------ tests/generic/590 | 9 ++++++--- tests/generic/744 | 4 ++-- tests/generic/746 | 6 +++++- tests/xfs/014 | 7 +++++-- tests/xfs/049 | 35 +++++++++++++++++++++-------------- tests/xfs/073 | 24 +++++++++++++----------- tests/xfs/074 | 21 ++++++++++++--------- tests/xfs/078 | 16 +++++++++------- tests/xfs/148 | 23 ++++++++++++++--------- tests/xfs/149 | 4 ++++ tests/xfs/216 | 15 ++++++++++----- tests/xfs/217 | 22 ++++++++++++++++------ tests/xfs/250 | 14 +++++++++----- tests/xfs/259 | 13 ++++++++----- tests/xfs/513 | 44 ++++++++++++++++++++------------------------ tests/xfs/521 | 8 ++++++-- tests/xfs/528 | 10 +++++++--- tests/xfs/530 | 11 ++++++----- tests/xfs/606 | 12 ++++++++---- tests/xfs/613 | 40 ++++++++++++++-------------------------- tests/xfs/613.out | 1 - 26 files changed, 235 insertions(+), 179 deletions(-)
common/metadump +12 -20
@@ -24,17 +24,9 @@ _xfs_cleanup_verify_metadump() test -n "$XFS_METADUMP_FILE" && rm -f "$XFS_METADUMP_FILE" - if [ -n "$XFS_METADUMP_IMG" ]; then - losetup -n -a -O BACK-FILE,NAME | grep "^$XFS_METADUMP_IMG" | while read backing ldev; do - losetup -d "$ldev" - done - - # Don't call rm directly with a globbed argument here to avoid - # issues issues with variable expansions. - for img in "$XFS_METADUMP_IMG"*; do - test -e "$img" && rm -f "$img" - done - fi + [ -n "$md_data_loop_dev" ] && _destroy_loop_device $md_data_loop_dev + [ -n "$md_log_loop_dev" ] && _destroy_loop_device $md_log_loop_dev + rm -f $data_img $log_img } # Can xfs_metadump snapshot the fs metadata to a v1 metadump file?
@@ -122,25 +114,25 @@ _xfs_verify_metadump_v2() _scratch_xfs_mdrestore $metadump_file # Create loopdev for data device so we can mount the fs - data_loop=$(_create_loop_device $data_img) + md_data_loop_dev=$(_create_loop_device $data_img) # Create loopdev for log device if we recovered anything - test -s "$log_img" && log_loop=$(_create_loop_device $log_img) + test -s "$log_img" && md_log_loop_dev=$(_create_loop_device $log_img) # Mount fs, run an extra test, fsck, and unmount - SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop _scratch_mount + SCRATCH_DEV=$md_data_loop_dev SCRATCH_LOGDEV=$md_log_loop_dev _scratch_mount if [ -n "$extra_test" ]; then - SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop $extra_test + SCRATCH_DEV=$md_data_loop_dev SCRATCH_LOGDEV=$md_log_loop_dev $extra_test fi - SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop _check_xfs_scratch_fs - SCRATCH_DEV=$data_loop _scratch_unmount + SCRATCH_DEV=$md_data_loop_dev SCRATCH_LOGDEV=$md_log_loop_dev _check_xfs_scratch_fs + _unmount $md_data_loop_dev # Tear down what we created - if [ -b "$log_loop" ]; then - _destroy_loop_device $log_loop + if [ -b "$md_log_loop_dev" ]; then + _destroy_loop_device $md_log_loop_dev rm -f $log_img fi - _destroy_loop_device $data_loop + _destroy_loop_device $md_data_loop_dev rm -f $data_img }
tests/generic/067 +8 -3
@@ -37,13 +37,18 @@ mount_nonexistent_mnt() $MOUNT_PROG $SCRATCH_DEV $TEST_DIR/nosuchdir >>$seqres.full 2>&1 } -# fs driver should be able to handle mounting a free loop device gracefully -# xfs ever hung, "ec53d1d xfs: don't block on buffer read errors" fixed it +# fs driver should be able to handle mounting a free loop device gracefully xfs +# ever hung, "ec53d1d xfs: don't block on buffer read errors" fixed it +# +# Using 'losetup -f' like this is racy. We might end up mounting a real loop dev +# here, so unmount $SCRATCH_MNT (not the loop dev we might not own!) and ignore +# any error it might return. mount_free_loopdev() { echo "# mount a free loop device" >>$seqres.full loopdev=`losetup -f` - $MOUNT_PROG -t $FSTYP $loopdev $SCRATCH_MNT >>$seqres.full 2>&1 + _mount $loopdev $SCRATCH_MNT >>$seqres.full 2>&1 + _unmount $SCRATCH_MNT >> /dev/null 2>&1 } # mount with wrong fs type specified.
tests/generic/361 +5 -1
@@ -17,7 +17,7 @@ _begin_fstest auto quick _cleanup() { _unmount $fs_mnt - _destroy_loop_device $loop_dev + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev cd / rm -f $tmp.* }
@@ -54,6 +54,10 @@ $XFS_IO_PROG -fc "pwrite 0 520m" $fs_mnt/testfile >>$seqres.full 2>&1 # remount should not hang $MOUNT_PROG -o remount,ro $fs_mnt >>$seqres.full 2>&1 +_unmount $fs_mnt +_destroy_loop_device $loop_dev +unset loop_dev + # success, all done echo "Silence is golden" status=0
tests/generic/563 +9 -5
@@ -87,12 +87,12 @@ reset() # cgroup I/O accounting doesn't work on partitions. Use a loop device to rule # that out. -LOOP_DEV=$(_create_loop_device $SCRATCH_DEV) -smajor=$((0x`stat -L -c %t $LOOP_DEV`)) -sminor=$((0x`stat -L -c %T $LOOP_DEV`)) +loop_dev=$(_create_loop_device $SCRATCH_DEV) +smajor=$((0x`stat -L -c %t $loop_dev`)) +sminor=$((0x`stat -L -c %T $loop_dev`)) -_mkfs_dev $LOOP_DEV >> $seqres.full 2>&1 -_mount $LOOP_DEV $SCRATCH_MNT || _fail "mount failed" +_mkfs_dev $loop_dev >> $seqres.full 2>&1 +_mount $loop_dev $SCRATCH_MNT || _fail "mount failed" blksize=$(_get_block_size "$SCRATCH_MNT")
@@ -147,6 +147,10 @@ if [ "$drop_io_cgroup" = 1 ]; then echo "-io" > $cgdir/cgroup.subtree_control || _fail "subtree control" fi +_unmount $SCRATCH_MNT +_destroy_loop_device $loop_dev +unset loop_dev + # success, all done status=0 exit
tests/generic/564 +6 -6
@@ -19,7 +19,7 @@ _cleanup() { cd / rm -rf $tmp.* - [ -z "$loopdev" ] || _destroy_loop_device $loopdev + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev } # Import common functions.
@@ -72,11 +72,11 @@ $XFS_IO_PROG -f -c "copy_range -l 32k $testdir" $testdir/copy echo echo source/destination as blkdev returns EINVAL $XFS_IO_PROG -f -c "truncate 128k" $testdir/img >> $seqres.full 2>&1 -loopdev=`_create_loop_device $testdir/img` -$XFS_IO_PROG -c "copy_range -l 32k $testdir/file" $loopdev -$XFS_IO_PROG -f -c "copy_range -l 32k $loopdev" $testdir/copy -_destroy_loop_device $loopdev -loopdev= +loop_dev=`_create_loop_device $testdir/img` +$XFS_IO_PROG -c "copy_range -l 32k $testdir/file" $loop_dev +$XFS_IO_PROG -f -c "copy_range -l 32k $loop_dev" $testdir/copy +_destroy_loop_device $loop_dev +unset loop_dev echo echo source/destination as chardev returns EINVAL
tests/generic/590 +6 -3
@@ -15,9 +15,10 @@ _begin_fstest auto prealloc preallocrw # Override the default cleanup function. _cleanup() { + _scratch_unmount + [ -n $loop_dev ] && _destroy_loop_device $loop_dev cd / rm -f $tmp.* - test -n "$loop" && _destroy_loop_device "$loop" rm -f "$TEST_DIR/$seq" }
@@ -56,9 +57,9 @@ if [[ $FSTYP = xfs ]]; then loopsz="$((filesz + (1 << 26)))" _require_fs_space "$TEST_DIR" $((loopsz / 1024)) $XFS_IO_PROG -c "truncate $loopsz" -f "$TEST_DIR/$seq" - loop="$(_create_loop_device "$TEST_DIR/$seq")" + loop_dev="$(_create_loop_device "$TEST_DIR/$seq")" USE_EXTERNAL=yes - SCRATCH_RTDEV="$loop" + SCRATCH_RTDEV="$loop_dev" disabled_features=() # disable reflink if not supported by realtime devices
@@ -114,6 +115,8 @@ $XFS_IO_PROG -c "truncate 0" -c fsync "$SCRATCH_MNT/file" # We need to do this before the loop device gets torn down. _scratch_unmount _check_scratch_fs +_destroy_loop_device $loop_dev +unset loop_dev echo "Silence is golden" status=0
tests/generic/744 +2 -2
@@ -17,8 +17,8 @@ _cleanup() _unmount $mnt2 &> /dev/null _unmount $mnt1 &> /dev/null - [ -b "$loop_dev2" ] && losetup -d $loop_dev2 - [ -b "$loop_dev1" ] && losetup -d $loop_dev1 + [ -b "$loop_dev2" ] && _destroy_loop_device $loop_dev2 + [ -b "$loop_dev1" ] && _destroy_loop_device $loop_dev1 [ -n "$seq" ] && rm -rf $TEST_DIR/$seq }
tests/generic/746 +5 -1
@@ -39,7 +39,7 @@ esac _cleanup() { _unmount $loop_mnt &> /dev/null - _destroy_loop_device $loop_dev + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev if [ $status -eq 0 ]; then rm -rf $tmp rm $img_file
@@ -223,5 +223,9 @@ while read line; do done < $fiemap_after echo "done." +_unmount $loop_mnt +_destroy_loop_device $loop_dev +unset loop_dev + status=0 exit
tests/xfs/014 +5 -2
@@ -23,6 +23,7 @@ _cleanup() { cd / _unmount $LOOP_MNT 2>/dev/null + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev _scratch_unmount 2>/dev/null rm -f $tmp.* }
@@ -166,19 +167,21 @@ LOOP_FILE=$SCRATCH_MNT/$seq.fs LOOP_MNT=$SCRATCH_MNT/$seq.mnt $MKFS_XFS_PROG -d "file=1,name=$LOOP_FILE,size=10g" >> $seqres.full 2>&1 +loop_dev=$(_create_loop_device $LOOP_FILE) mkdir -p $LOOP_MNT -_mount -o uquota,gquota $LOOP_FILE $LOOP_MNT || \ +_mount -o uquota,gquota $loop_dev $LOOP_MNT || \ _fail "Failed to mount loop fs." _test_enospc $LOOP_MNT _test_edquot $LOOP_MNT _unmount $LOOP_MNT +_destroy_loop_device $loop_dev +unset loop_dev echo $orig_sp_time > /proc/sys/fs/xfs/speculative_prealloc_lifetime _scratch_unmount - status=0 exit
tests/xfs/049 +21 -14
@@ -12,16 +12,17 @@ _begin_fstest rw auto quick # Override the default cleanup function. _cleanup() { - cd / - umount $SCRATCH_MNT/test2 > /dev/null 2>&1 - umount $SCRATCH_MNT/test > /dev/null 2>&1 - rm -f $tmp.* + cd / + _unmount $SCRATCH_MNT/test2 > /dev/null 2>&1 + _unmount $SCRATCH_MNT/test > /dev/null 2>&1 + [ -n "$loop_dev2" ] && _destroy_loop_device $loop_dev2 + [ -n "$loop_dev1" ] && _destroy_loop_device $loop_dev1 + rm -f $tmp.* - if [ -w $seqres.full ] - then - echo "--- mounts at end (after cleanup)" >> $seqres.full - mount >> $seqres.full - fi + if [ -w $seqres.full ]; then + echo "--- mounts at end (after cleanup)" >> $seqres.full + mount >> $seqres.full + fi } # Import common functions.
@@ -64,7 +65,8 @@ mkdir $SCRATCH_MNT/test $SCRATCH_MNT/test2 >> $seqres.full 2>&1 \ || _fail "!!! failed to make mount points" _log "Mount xfs via loop" -mount -t xfs -o loop $SCRATCH_MNT/test.xfs $SCRATCH_MNT/test >> $seqres.full 2>&1 \ +loop_dev1=$(_create_loop_device $SCRATCH_MNT/test.xfs) +_mount $loop_dev1 $SCRATCH_MNT/test >> $seqres.full 2>&1 \ || _fail "!!! failed to loop mount xfs" _log "stress"
@@ -80,11 +82,12 @@ dd if=/dev/zero of=$SCRATCH_MNT/test/test.ext2 bs=1024 count=10240 >> $seqres.fu || _fail "!!! create file failed" _log "Create ext2 fs in file on looped xfs" -echo y | mkfs -t ext2 $SCRATCH_MNT/test/test.ext2 >> $seqres.full 2>&1 \ +loop_dev2=$(_create_loop_device $SCRATCH_MNT/test/test.ext2) +echo y | mkfs -t ext2 $loop_dev2 >> $seqres.full 2>&1 \ || _fail "!!! failed to mkfs ext2 on xfs" _log "Mount ext2 on xfs via loop" -mount -t ext2 -o loop $SCRATCH_MNT/test/test.ext2 $SCRATCH_MNT/test2 >> $seqres.full 2>&1 \ +_mount $loop_dev2 $SCRATCH_MNT/test2 >> $seqres.full 2>&1 \ || _fail "!!! failed to loop mount xfs" _log "stress ext2 on xfs via loop"
@@ -96,12 +99,16 @@ rm -rf $SCRATCH_MNT/test/* >> $seqres.full 2>&1 \ || _fail "!!! clean failed" _log "umount ext2 on xfs" -umount $SCRATCH_MNT/test2 >> $seqres.full 2>&1 \ +_unmount $SCRATCH_MNT/test2 >> $seqres.full 2>&1 \ || _fail "!!! umount ext2 failed" +_destroy_loop_device $loop_dev2 +unset loop_dev2 _log "umount xfs" -umount $SCRATCH_MNT/test >> $seqres.full 2>&1 \ +_unmount $SCRATCH_MNT/test >> $seqres.full 2>&1 \ || _fail "!!! umount xfs failed" +_destroy_loop_device $loop_dev1 +unset loop_dev1 echo "--- mounts at end (before cleanup)" >> $seqres.full mount >> $seqres.full
tests/xfs/073 +13 -11
@@ -23,6 +23,8 @@ _cleanup() _scratch_unmount 2>/dev/null _unmount $imgs.loop 2>/dev/null _unmount $imgs.source_dir 2>/dev/null + [ -n "$loop_dev1" ] && _destroy_loop_device $loop_dev1 + [ -n "$loop_dev2" ] && _destroy_loop_device $loop_dev2 [ -d $imgs.loop ] && rmdir $imgs.loop [ -d $imgs.source_dir ] && rm -rf $imgs.source_dir rm -f $imgs.* $tmp.* /var/tmp/xfs_copy.log.*
@@ -65,10 +67,11 @@ _verify_copy() rmdir $target_dir 2>/dev/null mkdir $target_dir - _mount -t xfs -o loop $target $target_dir 2>/dev/null + loop_dev1=$(_create_loop_device $target) + _mount $loop_dev1 $target_dir 2>/dev/null if [ $? -ne 0 ]; then echo retrying mount with nouuid option >>$seqres.full - _mount -t xfs -o loop -o nouuid $target $target_dir + _mount -o nouuid $loop_dev1 $target_dir if [ $? -ne 0 ]; then echo mount failed - evil! return
@@ -100,6 +103,8 @@ _verify_copy() echo unmounting and removing new image _unmount $source_dir _unmount $target_dir > /dev/null 2>&1 + _destroy_loop_device $loop_dev1 + unset loop_dev1 rm -f $target }
@@ -134,18 +139,15 @@ ${MKFS_XFS_PROG} -dfile,name=$imgs.source,size=100g \ rmdir $imgs.source_dir 2>/dev/null mkdir $imgs.source_dir -_mount -t xfs -o loop $imgs.source $imgs.source_dir -loop2=`mount | grep $imgs.source | grep -o -e 'loop=.*[^),]' | grep -o -e '/.*$'` +loop_dev2=$(_create_loop_device $imgs.source) +_mount $loop_dev2 $imgs.source_dir cp -a $here $imgs.source_dir -_mount -t xfs -o remount,ro $imgs.source $imgs.source_dir -$XFS_COPY_PROG $imgs.source $imgs.image | _filter_copy '#' $imgs.image '#' '#' +_mount -o remount,ro $loop_dev2 $imgs.source_dir +$XFS_COPY_PROG $loop_dev2 $imgs.image 2> /dev/null | _filter_copy '#' $imgs.image '#' '#' _verify_copy $imgs.image $imgs.source $imgs.source_dir -# HACK WARNING: -# -# We're done with the nested loop mount, now we have to clean -# up the pieces that mount is incapable of doing. -losetup -d $loop2 > /dev/null 2>&1 +_destroy_loop_device $loop_dev2 +unset loop_dev2 echo echo === copying scratch device to multiple targets
tests/xfs/074 +12 -9
@@ -26,7 +26,7 @@ _begin_fstest quick auto prealloc rw _cleanup() { cd / - _destroy_loop_device $LOOP_DEV + [ -n "$loop_dev" ] &&_destroy_loop_device $loop_dev rm -f $tmp.* $LOOP_FILE }
@@ -45,10 +45,10 @@ LOOP_FILE=$TEST_DIR/$seq.img LOOP_MNT=$TEST_DIR/$seq.mnt mkdir -p $LOOP_MNT $XFS_IO_PROG -ft -c "truncate 1t" $LOOP_FILE >> $seqres.full -LOOP_DEV=`_create_loop_device $LOOP_FILE` +loop_dev=`_create_loop_device $LOOP_FILE` -_mkfs_dev -d size=260g,agcount=2 $LOOP_DEV -_mount $LOOP_DEV $LOOP_MNT +_mkfs_dev -d size=260g,agcount=2 $loop_dev +_mount $loop_dev $LOOP_MNT BLOCK_SIZE=$(_get_file_block_size $LOOP_MNT)
@@ -59,11 +59,11 @@ $XFS_IO_PROG -ft \ -c "falloc 0 $(($BLOCK_SIZE * 2097152))" \ $LOOP_MNT/foo >> $seqres.full -umount $LOOP_MNT -_check_xfs_filesystem $LOOP_DEV none none +_unmount $LOOP_MNT +_check_xfs_filesystem $loop_dev none none -_mkfs_dev -f $LOOP_DEV -_mount $LOOP_DEV $LOOP_MNT +_mkfs_dev -f $loop_dev +_mount $loop_dev $LOOP_MNT # check we trim both ends of the extent approproiately; this will fail # on 1k block size filesystems without the correct fixes in place.
@@ -73,7 +73,10 @@ $XFS_IO_PROG -ft \ $LOOP_MNT/foo >> $seqres.full _unmount $LOOP_MNT -_check_xfs_filesystem $LOOP_DEV none none +_check_xfs_filesystem $loop_dev none none + +_destroy_loop_device $loop_dev +unset loop_dev # success, all done echo "Silence is golden"
tests/xfs/078 +9 -7
@@ -17,7 +17,7 @@ _cleanup() cd / rm -f $tmp.* _unmount $LOOP_MNT 2>/dev/null - [ -n "$LOOP_DEV" ] && _destroy_loop_device $LOOP_DEV 2>/dev/null + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev 2>/dev/null # try to keep the image file if test fails [ $status -eq 0 ] && rm -f $LOOP_IMG rmdir $LOOP_MNT
@@ -55,7 +55,7 @@ _grow_loop() agsize=$5 $XFS_IO_PROG -f -c "truncate $original" $LOOP_IMG - LOOP_DEV=`_create_loop_device $LOOP_IMG` + loop_dev=`_create_loop_device $LOOP_IMG` dparam="" if [ -n "$agsize" ]; then
@@ -67,15 +67,15 @@ _grow_loop() echo echo "*** mkfs loop file (size=$original)" - $MKFS_XFS_PROG -b size=$bsize $dparam $LOOP_DEV | \ + $MKFS_XFS_PROG -b size=$bsize $dparam $loop_dev | \ _filter_mkfs 2>/dev/null echo "*** extend loop file" - _destroy_loop_device $LOOP_DEV + _destroy_loop_device $loop_dev $XFS_IO_PROG -c "pwrite $new_size $bsize" $LOOP_IMG | _filter_io - LOOP_DEV=`_create_loop_device $LOOP_IMG` + loop_dev=`_create_loop_device $LOOP_IMG` echo "*** mount loop filesystem" - _mount -t xfs $LOOP_DEV $LOOP_MNT + _mount $loop_dev $LOOP_MNT echo "*** grow loop filesystem" $XFS_GROWFS_PROG $LOOP_MNT 2>&1 | _filter_growfs 2>&1
@@ -87,9 +87,11 @@ _grow_loop() if [ "$check" -gt "0" ] then echo "*** check" - _check_xfs_filesystem $LOOP_IMG none none + _check_xfs_filesystem $loop_dev none none fi + _destroy_loop_device $loop_dev + unset loop_dev rm -f $LOOP_IMG }
tests/xfs/148 +14 -9
@@ -15,7 +15,7 @@ _cleanup() { cd / _unmount $mntpt > /dev/null 2>&1 - _destroy_loop_device $loopdev > /dev/null 2>&1 + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev rm -r -f $tmp.* }
@@ -48,12 +48,12 @@ rm -f $imgfile $imgfile.old # even when security xattrs are present so we are always doing name matches on # lookup and not name hash compares as leaf/node forms will do. $XFS_IO_PROG -f -c 'truncate 40m' $imgfile -loopdev=$(_create_loop_device $imgfile) -MKFS_OPTIONS="-m crc=0 -i size=512" _mkfs_dev $loopdev >> $seqres.full +loop_dev=$(_create_loop_device $imgfile) +MKFS_OPTIONS="-m crc=0 -i size=512" _mkfs_dev $loop_dev >> $seqres.full # Mount image file mkdir -p $mntpt -_mount $loopdev $mntpt +_mount $loop_dev $mntpt echo "creating entries" >> $seqres.full
@@ -91,7 +91,8 @@ cat $tmp.log | _filter_test_dir # Corrupt the entries _unmount $mntpt -_destroy_loop_device $loopdev +_destroy_loop_device $loop_dev +unset loop_dev cp $imgfile $imgfile.old sed -b \ -e "s/$nullstr/too_many\x00beans/g" \
@@ -99,8 +100,9 @@ sed -b \ -i $imgfile test "$(md5sum < $imgfile)" != "$(md5sum < $imgfile.old)" || _fail "sed failed to change the image file?" -loopdev=$(_create_loop_device $imgfile) -_mount $loopdev $mntpt + +loop_dev=$(_create_loop_device $imgfile) +_mount $loop_dev $mntpt # Try to access the corrupt metadata echo "++ ACCESSING BAD METADATA" | tee -a $seqres.full
@@ -111,7 +113,7 @@ cat $tmp.log | _filter_test_dir | sed -e '/Could not list/d' echo "does scrub complain?" >> $seqres.full # Does scrub complain about this? -if _supports_xfs_scrub $mntpt $loopdev; then +if _supports_xfs_scrub $mntpt $loop_dev; then $XFS_SCRUB_PROG -n $mntpt >> $seqres.full 2>&1 res=$? test $((res & 1)) -eq 0 && \
@@ -122,11 +124,14 @@ echo "does repair complain?" >> $seqres.full # Does repair complain about this? _unmount $mntpt -$XFS_REPAIR_PROG -n $loopdev >> $seqres.full 2>&1 +$XFS_REPAIR_PROG -n $loop_dev >> $seqres.full 2>&1 res=$? test $res -eq 1 || \ echo "repair failed to report corruption ($res)" +_destroy_loop_device $loop_dev +unset loop_dev + # success, all done status=0 exit
tests/xfs/149 +4 -0
@@ -84,6 +84,10 @@ $XFS_GROWFS_PROG -D 16384 $loop_symlink > /dev/null echo "=== xfs_growfs - check device node ===" $XFS_GROWFS_PROG -D 20480 $loop_dev > /dev/null +_unmount $mntdir +_destroy_loop_device $loop_dev +unset loop_dev + # success, all done status=0 exit
tests/xfs/216 +10 -5
@@ -15,6 +15,7 @@ _begin_fstest log metadata auto quick _cleanup() { _unmount $LOOP_MNT > /dev/null 2>&1 + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev cd / rm -f $tmp.* }
@@ -24,7 +25,7 @@ _scratch_mkfs_xfs >/dev/null 2>&1 _scratch_mount _require_loop -LOOP_DEV=$SCRATCH_MNT/test_fs +LOOP_IMG=$SCRATCH_MNT/test_fs LOOP_MNT=$SCRATCH_MNT/test_fs_dir loop_mkfs_opts=
@@ -55,22 +56,26 @@ _do_mkfs() for i in $*; do echo -n "fssize=${i}g " $MKFS_XFS_PROG -f -b size=4096 -l version=2 \ - -d name=$LOOP_DEV,size=${i}g $loop_mkfs_opts |grep log - _mount -o loop -t xfs $LOOP_DEV $LOOP_MNT + -d size=${i}g $loop_mkfs_opts $loop_dev |grep log + _mount $loop_dev $LOOP_MNT echo "test write" > $LOOP_MNT/test _unmount $LOOP_MNT > /dev/null 2>&1 done } # make large holey file -$XFS_IO_PROG -f -c "truncate 256g" $LOOP_DEV +$XFS_IO_PROG -f -c "truncate 256g" $LOOP_IMG -choose_golden_output $0 $LOOP_DEV +choose_golden_output $0 $LOOP_IMG #make loopback mount dir mkdir $LOOP_MNT +loop_dev=$(_create_loop_device $LOOP_IMG) + # walk over standard sizes (up to 256GB) _do_mkfs 1 2 4 8 16 32 64 128 256 +_destroy_loop_device $loop_dev +unset loop_dev status=0 exit
tests/xfs/217 +16 -6
@@ -12,6 +12,12 @@ _begin_fstest log metadata auto # Import common functions. . ./common/filter +_cleanup() +{ + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev + cd / + rm -f $tmp.* +} _require_scratch _scratch_mkfs_xfs >/dev/null 2>&1
@@ -20,7 +26,7 @@ _scratch_mount _require_fs_space $SCRATCH_MNT 2202000 _require_loop -LOOP_DEV=$SCRATCH_MNT/test_fs +LOOP_IMG=$SCRATCH_MNT/test_fs LOOP_MNT=$SCRATCH_MNT/test_fs_dir _do_mkfs()
@@ -28,28 +34,30 @@ _do_mkfs() for i in $*; do echo -n "fssize=${i}g " $MKFS_XFS_PROG -f -b size=4096 -l version=2 \ - -d name=$LOOP_DEV,size=${i}g |grep log - _mount -o loop -t xfs $LOOP_DEV $LOOP_MNT + -d size=${i}g $loop_dev |grep log + _mount $loop_dev $LOOP_MNT echo "test write" > $LOOP_MNT/test _unmount $LOOP_MNT > /dev/null 2>&1 # punch out the previous blocks so that we keep the amount of # disk space the test requires down to a minimum. - $XFS_IO_PROG -f -c "unresvsp 0 16383g" $LOOP_DEV + $XFS_IO_PROG -f -c "unresvsp 0 16383g" $LOOP_IMG done } # make large holey file -$XFS_IO_PROG -f -c "truncate 16383g" $LOOP_DEV +$XFS_IO_PROG -f -c "truncate 16383g" $LOOP_IMG #make loopback mount dir mkdir $LOOP_MNT # test if large logs are supported -$MKFS_XFS_PROG -f -l size=256m -d name=$LOOP_DEV,size=10g > /dev/null 2>&1 +$MKFS_XFS_PROG -f -l size=256m -d name=$LOOP_IMG,size=10g > /dev/null 2>&1 if [ $? -ne 0 ]; then _notrun "large log sizes not supported by mkfs" fi +loop_dev=$(_create_loop_device $LOOP_IMG) + # # walk over "new" sizes supported by recent xfsprogs. # Note that the last test is for 16TB-1GB as 32bit platforms only support
@@ -57,5 +65,7 @@ fi # _do_mkfs 512 1024 2048 4096 8192 16383 +_destroy_loop_device $loop_dev +unset loop_dev status=0 exit
tests/xfs/250 +9 -5
@@ -14,7 +14,8 @@ _cleanup() { cd / _unmount $LOOP_MNT 2>/dev/null - rm -f $LOOP_DEV + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev + rm -f $LOOP_IMG rmdir $LOOP_MNT rm -f $tmp.* }
@@ -26,7 +27,7 @@ _require_test _require_loop _require_xfs_io_command "falloc" -LOOP_DEV=$TEST_DIR/$seq.fs +LOOP_IMG=$TEST_DIR/$seq.fs LOOP_MNT=$TEST_DIR/$seq.mnt _filter_io()
@@ -45,7 +46,7 @@ _test_loop() agsize=$2 fsize=$3 - dparam="file,name=$LOOP_DEV,size=$size" + dparam="file,name=$LOOP_IMG,size=$size" if [ -n "$agsize" ]; then dparam="$dparam,agsize=$agsize" fi
@@ -55,7 +56,8 @@ _test_loop() | _filter_mkfs 2>/dev/null echo "*** mount loop filesystem" - _mount -t xfs -o loop $LOOP_DEV $LOOP_MNT + loop_dev=$(_create_loop_device $LOOP_IMG) + mount $loop_dev $LOOP_MNT echo "*** preallocate large file" $XFS_IO_PROG -f -c "resvsp 0 $fsize" $LOOP_MNT/foo | _filter_io
@@ -64,7 +66,9 @@ _test_loop() _unmount $LOOP_MNT > /dev/null 2>&1 echo "*** check loop filesystem" - _check_xfs_filesystem $LOOP_DEV none none + _check_xfs_filesystem $loop_dev none none + _destroy_loop_device $loop_dev + unset loop_dev } _test_loop 50g 16m 40G
tests/xfs/259 +8 -5
@@ -12,7 +12,10 @@ _begin_fstest auto quick # Override the default cleanup function. _cleanup() { - rm -f "$testfile" + [ -n "$loop_dev" ] && _destroy_loop_device $testfile + rm -f "$testfile" + cd / + rm -f $tmp.* } # Import common functions.
@@ -45,13 +48,13 @@ for del in $sizes_to_check; do rm -f "$testfile" dd if=/dev/zero "of=$testfile" bs=1 count=0 seek=$ddseek \ >/dev/null 2>&1 || echo "dd failed" - lofile=$(losetup -f) - losetup $lofile "$testfile" - $MKFS_XFS_PROG -l size=32m -b size=$bs $lofile | _filter_mkfs \ + loop_dev=$(_create_loop_device $testfile) + $MKFS_XFS_PROG -l size=32m -b size=$bs $loop_dev | _filter_mkfs \ >/dev/null 2> $tmp.mkfs || echo "mkfs failed!" . $tmp.mkfs sync - losetup -d $lofile + _destroy_loop_device $loop_dev + unset loop_dev done done
tests/xfs/513 +20 -24
124 lines not on this page · raw message
tests/xfs/521 +6 -2
@@ -21,7 +21,7 @@ _cleanup() { cd / _scratch_unmount >> $seqres.full 2>&1 - test -e "$rtdev" && losetup -d $rtdev >> $seqres.full 2>&1 + [ -n "$rt_loop_dev" ] && _destroy_loop_device $rt_loop_dev rm -f $tmp.* $TEST_DIR/$seq.rtvol }
@@ -35,7 +35,7 @@ _require_no_large_scratch_dev echo "Create fake rt volume" truncate -s 400m $TEST_DIR/$seq.rtvol -rtdev=$(_create_loop_device $TEST_DIR/$seq.rtvol) +rt_loop_dev=$(_create_loop_device $TEST_DIR/$seq.rtvol) echo "Format and mount 100m rt volume" export USE_EXTERNAL=yes
@@ -69,6 +69,10 @@ cp -p $testdir/original $testdir/copy3 echo "Check filesystem" _check_scratch_fs +_scratch_unmount +_destroy_loop_device $rt_loop_dev +unset rt_loop_dev + # success, all done status=0 exit
tests/xfs/528 +7 -3
@@ -15,7 +15,7 @@ _cleanup() { cd / _scratch_unmount >> $seqres.full 2>&1 - test -e "$rtdev" && losetup -d $rtdev >> $seqres.full 2>&1 + [ -n "$rt_loop_dev" ] && _destroy_loop_device $rt_loop_dev rm -f $tmp.* $TEST_DIR/$seq.rtvol }
@@ -155,11 +155,11 @@ test_ops() { echo "Create fake rt volume" $XFS_IO_PROG -f -c "truncate 400m" $TEST_DIR/$seq.rtvol -rtdev=$(_create_loop_device $TEST_DIR/$seq.rtvol) +rt_loop_dev=$(_create_loop_device $TEST_DIR/$seq.rtvol) echo "Make sure synth rt volume works" export USE_EXTERNAL=yes -export SCRATCH_RTDEV=$rtdev +export SCRATCH_RTDEV=$rt_loop_dev _scratch_mkfs > $seqres.full _try_scratch_mount || \ _notrun "Could not mount with synthetic rt volume"
@@ -170,6 +170,10 @@ test_ops 262144 # not a power of two test_ops 327680 +_scratch_unmount +_destroy_loop_device $rt_loop_dev +unset rt_loop_dev + # success, all done status=0 exit
tests/xfs/530 +6 -5
@@ -15,7 +15,7 @@ _cleanup() { cd / _scratch_unmount >> $seqres.full 2>&1 - test -e "$rtdev" && losetup -d $rtdev >> $seqres.full 2>&1 + [ -n "$rt_loop_dev" ] && _destroy_loop_device $rt_loop_dev rm -f $tmp.* $TEST_DIR/$seq.rtvol }
@@ -52,12 +52,12 @@ fi rtdevsz=$((nr_bits * rtextsz)) truncate -s $rtdevsz $TEST_DIR/$seq.rtvol -rtdev=$(_create_loop_device $TEST_DIR/$seq.rtvol) +rt_loop_dev=$(_create_loop_device $TEST_DIR/$seq.rtvol) echo "Format and mount rt volume" export USE_EXTERNAL=yes -export SCRATCH_RTDEV=$rtdev +export SCRATCH_RTDEV=$rt_loop_dev _scratch_mkfs -d size=$((1024 * 1024 * 1024)) \ -r size=${rtextsz},extsize=${rtextsz} >> $seqres.full _try_scratch_mount || _notrun "Couldn't mount fs with synthetic rt volume"
@@ -116,8 +116,9 @@ done echo "Check filesystem" _check_scratch_fs -losetup -d $rtdev -rm -f $TEST_DIR/$seq.rtvol +_scratch_unmount +_destroy_loop_device $rt_loop_dev +unset rt_loop_dev # success, all done status=0
tests/xfs/606 +8 -4
@@ -13,10 +13,8 @@ _begin_fstest auto quick growfs _cleanup() { - local dev _unmount $LOOP_MNT 2>/dev/null - dev=$(losetup -j testfile | cut -d: -f1) - losetup -d $dev 2>/dev/null + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev rm -rf $LOOP_IMG $LOOP_MNT cd / rm -f $tmp.*
@@ -40,7 +38,9 @@ $MKFS_XFS_PROG -f $LOOP_IMG >$seqres.full # Extend by just 8K, expected to start with the last full-size AG ends of # above 1G block device. $XFS_IO_PROG -f -c "truncate 1073750016" $LOOP_IMG -_mount -oloop $LOOP_IMG $LOOP_MNT + +loop_dev=$(_create_loop_device $LOOP_IMG) +_mount $loop_dev $LOOP_MNT # A known bug shows "XFS_IOC_FSGROWFSDATA xfsctl failed: No space left on # device" at here, refer to _fixed_by_kernel_commit above $XFS_GROWFS_PROG $LOOP_MNT >$seqres.full
@@ -48,6 +48,10 @@ if [ $? -ne 0 ];then echo "xfs_growfs fails!" fi +_unmount $LOOP_MNT +_destroy_loop_device $loop_dev +unset loop_dev + echo "Silence is golden" # success, all done status=0
tests/xfs/613 +14 -26
@@ -15,14 +15,8 @@ _cleanup() cd / rm -f $tmp.* _unmount $LOOP_MNT 2>/dev/null - if [ -n "$LOOP_DEV" ];then - _destroy_loop_device $LOOP_DEV 2>/dev/null - fi - if [ -n "$LOOP_SPARE_DEV" ];then - _destroy_loop_device $LOOP_SPARE_DEV 2>/dev/null - fi + [ -n $loop_dev ] &&_destroy_loop_device $loop_dev rm -f $LOOP_IMG - rm -f $LOOP_SPARE_IMG rmdir $LOOP_MNT }
@@ -38,16 +32,11 @@ _require_loop _require_xfs_io_command "falloc" LOOP_IMG=$TEST_DIR/$seq.dev -LOOP_SPARE_IMG=$TEST_DIR/$seq.logdev LOOP_MNT=$TEST_DIR/$seq.mnt echo "** create loop device" $XFS_IO_PROG -f -c "truncate 32g" $LOOP_IMG -LOOP_DEV=`_create_loop_device $LOOP_IMG` - -echo "** create loop log device" -$XFS_IO_PROG -f -c "truncate 1g" $LOOP_SPARE_IMG -LOOP_SPARE_DEV=`_create_loop_device $LOOP_SPARE_IMG` +loop_dev=`_create_loop_device $LOOP_IMG` echo "** create loop mount point" rmdir $LOOP_MNT 2>/dev/null
@@ -56,8 +45,7 @@ mkdir -p $LOOP_MNT || _fail "cannot create loopback mount point" filter_loop() { sed -e "s,\B$LOOP_MNT,LOOP_MNT,g" \ - -e "s,\B$LOOP_DEV,LOOP_DEV,g" \ - -e "s,\B$LOOP_SPARE_DEV,LOOP_SPARE_DEV,g" + -e "s,\B$loop_dev,LOOP_DEV,g" } filter_xfs_opt()
@@ -70,22 +58,22 @@ MKFS_OPTIONS="" do_mkfs() { echo "FORMAT: $@" | filter_loop | tee -a $seqres.full - $MKFS_XFS_PROG -f $* $LOOP_DEV | _filter_mkfs >>$seqres.full 2>$tmp.mkfs + $MKFS_XFS_PROG -f $* $loop_dev | _filter_mkfs >>$seqres.full 2>$tmp.mkfs if [ "${PIPESTATUS[0]}" -ne 0 ]; then - _fail "Fails on _mkfs_dev $* $LOOP_DEV" + _fail "Fails on _mkfs_dev $* $loop_dev" fi . $tmp.mkfs } is_dev_mounted() { - findmnt --source $LOOP_DEV >/dev/null + findmnt --source $loop_dev >/dev/null return $? } get_mount_info() { - findmnt --source $LOOP_DEV -o OPTIONS -n + findmnt --source $loop_dev -o OPTIONS -n } force_unmount()
@@ -104,29 +92,29 @@ _do_test() local info # mount test - _mount $LOOP_DEV $LOOP_MNT $opts 2>>$seqres.full + _mount $loop_dev $LOOP_MNT $opts 2>>$seqres.full rc=$? if [ $rc -eq 0 ];then if [ "${mounted}" = "fail" ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: expect mount to fail, but it succeeded" return 1 fi is_dev_mounted if [ $? -ne 0 ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: fs not mounted even mount return 0" return 1 fi else if [ "${mounted}" = "pass" ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: expect mount to succeed, but it failed" return 1 fi is_dev_mounted if [ $? -eq 0 ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: fs is mounted even mount return non-zero" return 1 fi
@@ -142,13 +130,13 @@ _do_test() rc=$? if [ $rc -eq 0 ];then if [ "$found" != "true" ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: expected to find \"$key\" in mount info \"$info\"" return 1 fi else if [ "$found" != "false" ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: did not expect to find \"$key\" in \"$info\"" return 1 fi
tests/xfs/613.out +0 -1
@@ -1,6 +1,5 @@ QA output created by 613 ** create loop device -** create loop log device ** create loop mount point ** start xfs mount testing ... FORMAT: -m crc=0
Signature · 1 line
2.45.2
Re: [PATCH 13/40] fstests: clean up loop device instantiation
Quoted 237 lines · Dave Chinner
From: Dave Chinner <[email protected]>
Lots of tests do there own special thing with loop devices rather than using
_create_loop_device()and_destroy_loop_device(). This oftens means they do not clean up after themselves properly, leaving stale loop devices around that result in unmountable test or scratch devices. This is common when tests are killed by user interrupt.Even the tests that do use
_destroy_loop_deviceand try to clean up often do it incorrectly, leading to spurious error messages.Some tests try to use dynamic instantiation via "mount -o loop", but then don't clean up in the correct order or hack around to find the loop device that was instantiated because the test needs to know the instantiated device name
Clean this up by converting all the tests to use
_create_loop_device()and_destroy_loop_device(). In all the tests, use the variable "loop_dev" for the device consistently. In_destroy_loop_device(), test that a device name has been passed so that we don't try to clean up the same device twice (e.g. once before test exit and again from the_cleanup()function). When we destroy a loop device, unset the variable used to hold the loop device name so that we don't try to destroy it twice.This results in much more reliable cleanup and clean exit from fstests when killed by the user.
Signed-off-by Dave Chinner <[email protected]>--- common/metadump | 32 ++++++++++++-------------------- tests/generic/067 | 11 ++++++++--- tests/generic/361 | 6 +++++- tests/generic/563 | 14 +++++++++----- tests/generic/564 | 12 ++++++------ tests/generic/590 | 9 ++++++--- tests/generic/744 | 4 ++-- tests/generic/746 | 6 +++++- tests/xfs/014 | 7 +++++-- tests/xfs/049 | 35 +++++++++++++++++++++-------------- tests/xfs/073 | 24 +++++++++++++----------- tests/xfs/074 | 21 ++++++++++++--------- tests/xfs/078 | 16 +++++++++------- tests/xfs/148 | 23 ++++++++++++++--------- tests/xfs/149 | 4 ++++ tests/xfs/216 | 15 ++++++++++----- tests/xfs/217 | 22 ++++++++++++++++------ tests/xfs/250 | 14 +++++++++----- tests/xfs/259 | 13 ++++++++----- tests/xfs/513 | 44 ++++++++++++++++++++------------------------ tests/xfs/521 | 8 ++++++-- tests/xfs/528 | 10 +++++++--- tests/xfs/530 | 11 ++++++----- tests/xfs/606 | 12 ++++++++---- tests/xfs/613 | 40 ++++++++++++++-------------------------- tests/xfs/613.out | 1 - 26 files changed, 235 insertions(+), 179 deletions(-)common/metadump +12
-20@@ -24,17 +24,9 @@ _xfs_cleanup_verify_metadump() test -n "$XFS_METADUMP_FILE" && rm -f "$XFS_METADUMP_FILE" - if [ -n "$XFS_METADUMP_IMG" ]; then - losetup -n -a -O BACK-FILE,NAME | grep "^$XFS_METADUMP_IMG" | while read backing ldev; do - losetup -d "$ldev" - done - - # Don't call rm directly with a globbed argument here to avoid - # issues issues with variable expansions. - for img in "$XFS_METADUMP_IMG"*; do - test -e "$img" && rm -f "$img" - done - fi + [ -n "$md_data_loop_dev" ] && _destroy_loop_device $md_data_loop_dev + [ -n "$md_log_loop_dev" ] && _destroy_loop_device $md_log_loop_dev + rm -f $data_img $log_img } # Can xfs_metadump snapshot the fs metadata to a v1 metadump file?@@ -122,25 +114,25 @@ _xfs_verify_metadump_v2() _scratch_xfs_mdrestore $metadump_file # Create loopdev for data device so we can mount the fs - data_loop=$(_create_loop_device $data_img) + md_data_loop_dev=$(_create_loop_device $data_img) # Create loopdev for log device if we recovered anything - test -s "$log_img" && log_loop=$(_create_loop_device $log_img) + test -s "$log_img" && md_log_loop_dev=$(_create_loop_device $log_img) # Mount fs, run an extra test, fsck, and unmount - SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop _scratch_mount + SCRATCH_DEV=$md_data_loop_dev SCRATCH_LOGDEV=$md_log_loop_dev _scratch_mount if [ -n "$extra_test" ]; then - SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop $extra_test + SCRATCH_DEV=$md_data_loop_dev SCRATCH_LOGDEV=$md_log_loop_dev $extra_test fi - SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop _check_xfs_scratch_fs - SCRATCH_DEV=$data_loop _scratch_unmount + SCRATCH_DEV=$md_data_loop_dev SCRATCH_LOGDEV=$md_log_loop_dev _check_xfs_scratch_fs + _unmount $md_data_loop_dev # Tear down what we created - if [ -b "$log_loop" ]; then - _destroy_loop_device $log_loop + if [ -b "$md_log_loop_dev" ]; then + _destroy_loop_device $md_log_loop_dev rm -f $log_img fi - _destroy_loop_device $data_loop + _destroy_loop_device $md_data_loop_dev rm -f $data_img }tests/generic/067 +8
-3@@ -37,13 +37,18 @@ mount_nonexistent_mnt() $MOUNT_PROG $SCRATCH_DEV $TEST_DIR/nosuchdir >>$seqres.full 2>&1 } -# fs driver should be able to handle mounting a free loop device gracefully -# xfs ever hung, "ec53d1d xfs: don't block on buffer read errors" fixed it +# fs driver should be able to handle mounting a free loop device gracefully xfs +# ever hung, "ec53d1d xfs: don't block on buffer read errors" fixed it +# +# Using 'losetup -f' like this is racy. We might end up mounting a real loop dev +# here, so unmount $SCRATCH_MNT (not the loop dev we might not own!) and ignore +# any error it might return. mount_free_loopdev() { echo "# mount a free loop device" >>$seqres.full loopdev=`losetup -f` - $MOUNT_PROG -t $FSTYP $loopdev $SCRATCH_MNT >>$seqres.full 2>&1 + _mount $loopdev $SCRATCH_MNT >>$seqres.full 2>&1 + _unmount $SCRATCH_MNT >> /dev/null 2>&1 } # mount with wrong fs type specified.tests/generic/361 +5
-1@@ -17,7 +17,7 @@ _begin_fstest auto quick _cleanup() { _unmount $fs_mnt - _destroy_loop_device $loop_dev + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev cd / rm -f $tmp.* }@@ -54,6 +54,10 @@ $XFS_IO_PROG -fc "pwrite 0 520m" $fs_mnt/testfile >>$seqres.full 2>&1 # remount should not hang $MOUNT_PROG -o remount,ro $fs_mnt >>$seqres.full 2>&1 +_unmount $fs_mnt +_destroy_loop_device $loop_dev +unset loop_dev + # success, all done echo "Silence is golden" status=0tests/generic/563 +9
-5@@ -87,12 +87,12 @@ reset() # cgroup I/O accounting doesn't work on partitions. Use a loop device to rule # that out. -LOOP_DEV=$(_create_loop_device $SCRATCH_DEV) -smajor=$((0x`stat -L -c %t $LOOP_DEV`)) -sminor=$((0x`stat -L -c %T $LOOP_DEV`)) +loop_dev=$(_create_loop_device $SCRATCH_DEV) +smajor=$((0x`stat -L -c %t $loop_dev`)) +sminor=$((0x`stat -L -c %T $loop_dev`)) -_mkfs_dev $LOOP_DEV >> $seqres.full 2>&1 -_mount $LOOP_DEV $SCRATCH_MNT || _fail "mount failed" +_mkfs_dev $loop_dev >> $seqres.full 2>&1 +_mount $loop_dev $SCRATCH_MNT || _fail "mount failed" blksize=$(_get_block_size "$SCRATCH_MNT")@@ -147,6 +147,10 @@ if [ "$drop_io_cgroup" = 1 ]; then echo "-io" > $cgdir/cgroup.subtree_control || _fail "subtree control" fi +_unmount $SCRATCH_MNT +_destroy_loop_device $loop_dev +unset loop_dev + # success, all done status=0 exittests/generic/564 +6
-6@@ -19,7 +19,7 @@ _cleanup() { cd / rm -rf $tmp.* - [ -z "$loopdev" ] || _destroy_loop_device $loopdev + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev } # Import common functions.@@ -72,11 +72,11 @@ $XFS_IO_PROG -f -c "copy_range -l 32k $testdir" $testdir/copy echo echo source/destination as blkdev returns EINVAL $XFS_IO_PROG -f -c "truncate 128k" $testdir/img >> $seqres.full 2>&1 -loopdev=`_create_loop_device $testdir/img` -$XFS_IO_PROG -c "copy_range -l 32k $testdir/file" $loopdev -$XFS_IO_PROG -f -c "copy_range -l 32k $loopdev" $testdir/copy -_destroy_loop_device $loopdev -loopdev= +loop_dev=`_create_loop_device $testdir/img` +$XFS_IO_PROG -c "copy_range -l 32k $testdir/file" $loop_dev +$XFS_IO_PROG -f -c "copy_range -l 32k $loop_dev" $testdir/copy +_destroy_loop_device $loop_dev +unset loop_dev echo echo source/destination as chardev returns EINVALtests/generic/590 +0
-0
tests/generic/590 _begin_fstest auto prealloc preallocrw · line 19
Quoted 5 lines
tests/generic/590 +2
-0@@ -15,9 +15,10 @@ _begin_fstest auto prealloc preallocrw # Override the default cleanup function. _cleanup() { + _scratch_unmount + [ -n $loop_dev ] && _destroy_loop_device $loop_dev
The $loop_dev need quotation marks, or the _destroy_loop_device always be run. Same issue on xfs/513 and xfs/613. I'll add "..." to them.
Thanks,
Zorro
Quoted 1,072 lines
tests/generic/590 +4
-3cd / rm -f $tmp.* - test -n "$loop" && _destroy_loop_device "$loop" rm -f "$TEST_DIR/$seq" }@@ -56,9 +57,9 @@ if [[ $FSTYP = xfs ]]; then loopsz="$((filesz + (1 << 26)))" _require_fs_space "$TEST_DIR" $((loopsz / 1024)) $XFS_IO_PROG -c "truncate $loopsz" -f "$TEST_DIR/$seq" - loop="$(_create_loop_device "$TEST_DIR/$seq")" + loop_dev="$(_create_loop_device "$TEST_DIR/$seq")" USE_EXTERNAL=yes - SCRATCH_RTDEV="$loop" + SCRATCH_RTDEV="$loop_dev" disabled_features=() # disable reflink if not supported by realtime devices@@ -114,6 +115,8 @@ $XFS_IO_PROG -c "truncate 0" -c fsync "$SCRATCH_MNT/file" # We need to do this before the loop device gets torn down. _scratch_unmount _check_scratch_fs +_destroy_loop_device $loop_dev +unset loop_dev echo "Silence is golden" status=0tests/generic/744 +2
-2@@ -17,8 +17,8 @@ _cleanup() _unmount $mnt2 &> /dev/null _unmount $mnt1 &> /dev/null - [ -b "$loop_dev2" ] && losetup -d $loop_dev2 - [ -b "$loop_dev1" ] && losetup -d $loop_dev1 + [ -b "$loop_dev2" ] && _destroy_loop_device $loop_dev2 + [ -b "$loop_dev1" ] && _destroy_loop_device $loop_dev1 [ -n "$seq" ] && rm -rf $TEST_DIR/$seq }tests/generic/746 +5
-1@@ -39,7 +39,7 @@ esac _cleanup() { _unmount $loop_mnt &> /dev/null - _destroy_loop_device $loop_dev + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev if [ $status -eq 0 ]; then rm -rf $tmp rm $img_file@@ -223,5 +223,9 @@ while read line; do done < $fiemap_after echo "done." +_unmount $loop_mnt +_destroy_loop_device $loop_dev +unset loop_dev + status=0 exittests/xfs/014 +5
-2@@ -23,6 +23,7 @@ _cleanup() { cd / _unmount $LOOP_MNT 2>/dev/null + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev _scratch_unmount 2>/dev/null rm -f $tmp.* }@@ -166,19 +167,21 @@ LOOP_FILE=$SCRATCH_MNT/$seq.fs LOOP_MNT=$SCRATCH_MNT/$seq.mnt $MKFS_XFS_PROG -d "file=1,name=$LOOP_FILE,size=10g" >> $seqres.full 2>&1 +loop_dev=$(_create_loop_device $LOOP_FILE) mkdir -p $LOOP_MNT -_mount -o uquota,gquota $LOOP_FILE $LOOP_MNT || \ +_mount -o uquota,gquota $loop_dev $LOOP_MNT || \ _fail "Failed to mount loop fs." _test_enospc $LOOP_MNT _test_edquot $LOOP_MNT _unmount $LOOP_MNT +_destroy_loop_device $loop_dev +unset loop_dev echo $orig_sp_time > /proc/sys/fs/xfs/speculative_prealloc_lifetime _scratch_unmount - status=0 exittests/xfs/049 +21
-14@@ -12,16 +12,17 @@ _begin_fstest rw auto quick # Override the default cleanup function. _cleanup() { - cd / - umount $SCRATCH_MNT/test2 > /dev/null 2>&1 - umount $SCRATCH_MNT/test > /dev/null 2>&1 - rm -f $tmp.* + cd / + _unmount $SCRATCH_MNT/test2 > /dev/null 2>&1 + _unmount $SCRATCH_MNT/test > /dev/null 2>&1 + [ -n "$loop_dev2" ] && _destroy_loop_device $loop_dev2 + [ -n "$loop_dev1" ] && _destroy_loop_device $loop_dev1 + rm -f $tmp.* - if [ -w $seqres.full ] - then - echo "--- mounts at end (after cleanup)" >> $seqres.full - mount >> $seqres.full - fi + if [ -w $seqres.full ]; then + echo "--- mounts at end (after cleanup)" >> $seqres.full + mount >> $seqres.full + fi } # Import common functions.@@ -64,7 +65,8 @@ mkdir $SCRATCH_MNT/test $SCRATCH_MNT/test2 >> $seqres.full 2>&1 \ || _fail "!!! failed to make mount points" _log "Mount xfs via loop" -mount -t xfs -o loop $SCRATCH_MNT/test.xfs $SCRATCH_MNT/test >> $seqres.full 2>&1 \ +loop_dev1=$(_create_loop_device $SCRATCH_MNT/test.xfs) +_mount $loop_dev1 $SCRATCH_MNT/test >> $seqres.full 2>&1 \ || _fail "!!! failed to loop mount xfs" _log "stress"@@ -80,11 +82,12 @@ dd if=/dev/zero of=$SCRATCH_MNT/test/test.ext2 bs=1024 count=10240 >> $seqres.fu || _fail "!!! create file failed" _log "Create ext2 fs in file on looped xfs" -echo y | mkfs -t ext2 $SCRATCH_MNT/test/test.ext2 >> $seqres.full 2>&1 \ +loop_dev2=$(_create_loop_device $SCRATCH_MNT/test/test.ext2) +echo y | mkfs -t ext2 $loop_dev2 >> $seqres.full 2>&1 \ || _fail "!!! failed to mkfs ext2 on xfs" _log "Mount ext2 on xfs via loop" -mount -t ext2 -o loop $SCRATCH_MNT/test/test.ext2 $SCRATCH_MNT/test2 >> $seqres.full 2>&1 \ +_mount $loop_dev2 $SCRATCH_MNT/test2 >> $seqres.full 2>&1 \ || _fail "!!! failed to loop mount xfs" _log "stress ext2 on xfs via loop"@@ -96,12 +99,16 @@ rm -rf $SCRATCH_MNT/test/* >> $seqres.full 2>&1 \ || _fail "!!! clean failed" _log "umount ext2 on xfs" -umount $SCRATCH_MNT/test2 >> $seqres.full 2>&1 \ +_unmount $SCRATCH_MNT/test2 >> $seqres.full 2>&1 \ || _fail "!!! umount ext2 failed" +_destroy_loop_device $loop_dev2 +unset loop_dev2 _log "umount xfs" -umount $SCRATCH_MNT/test >> $seqres.full 2>&1 \ +_unmount $SCRATCH_MNT/test >> $seqres.full 2>&1 \ || _fail "!!! umount xfs failed" +_destroy_loop_device $loop_dev1 +unset loop_dev1 echo "--- mounts at end (before cleanup)" >> $seqres.full mount >> $seqres.fulltests/xfs/073 +13
-11@@ -23,6 +23,8 @@ _cleanup() _scratch_unmount 2>/dev/null _unmount $imgs.loop 2>/dev/null _unmount $imgs.source_dir 2>/dev/null + [ -n "$loop_dev1" ] && _destroy_loop_device $loop_dev1 + [ -n "$loop_dev2" ] && _destroy_loop_device $loop_dev2 [ -d $imgs.loop ] && rmdir $imgs.loop [ -d $imgs.source_dir ] && rm -rf $imgs.source_dir rm -f $imgs.* $tmp.* /var/tmp/xfs_copy.log.*@@ -65,10 +67,11 @@ _verify_copy() rmdir $target_dir 2>/dev/null mkdir $target_dir - _mount -t xfs -o loop $target $target_dir 2>/dev/null + loop_dev1=$(_create_loop_device $target) + _mount $loop_dev1 $target_dir 2>/dev/null if [ $? -ne 0 ]; then echo retrying mount with nouuid option >>$seqres.full - _mount -t xfs -o loop -o nouuid $target $target_dir + _mount -o nouuid $loop_dev1 $target_dir if [ $? -ne 0 ]; then echo mount failed - evil! return@@ -100,6 +103,8 @@ _verify_copy() echo unmounting and removing new image _unmount $source_dir _unmount $target_dir > /dev/null 2>&1 + _destroy_loop_device $loop_dev1 + unset loop_dev1 rm -f $target }@@ -134,18 +139,15 @@ ${MKFS_XFS_PROG} -dfile,name=$imgs.source,size=100g \ rmdir $imgs.source_dir 2>/dev/null mkdir $imgs.source_dir -_mount -t xfs -o loop $imgs.source $imgs.source_dir -loop2=`mount | grep $imgs.source | grep -o -e 'loop=.*[^),]' | grep -o -e '/.*$'` +loop_dev2=$(_create_loop_device $imgs.source) +_mount $loop_dev2 $imgs.source_dir cp -a $here $imgs.source_dir -_mount -t xfs -o remount,ro $imgs.source $imgs.source_dir -$XFS_COPY_PROG $imgs.source $imgs.image | _filter_copy '#' $imgs.image '#' '#' +_mount -o remount,ro $loop_dev2 $imgs.source_dir +$XFS_COPY_PROG $loop_dev2 $imgs.image 2> /dev/null | _filter_copy '#' $imgs.image '#' '#' _verify_copy $imgs.image $imgs.source $imgs.source_dir -# HACK WARNING: -# -# We're done with the nested loop mount, now we have to clean -# up the pieces that mount is incapable of doing. -losetup -d $loop2 > /dev/null 2>&1 +_destroy_loop_device $loop_dev2 +unset loop_dev2 echo echo === copying scratch device to multiple targetstests/xfs/074 +12
-9@@ -26,7 +26,7 @@ _begin_fstest quick auto prealloc rw _cleanup() { cd / - _destroy_loop_device $LOOP_DEV + [ -n "$loop_dev" ] &&_destroy_loop_device $loop_dev rm -f $tmp.* $LOOP_FILE }@@ -45,10 +45,10 @@ LOOP_FILE=$TEST_DIR/$seq.img LOOP_MNT=$TEST_DIR/$seq.mnt mkdir -p $LOOP_MNT $XFS_IO_PROG -ft -c "truncate 1t" $LOOP_FILE >> $seqres.full -LOOP_DEV=`_create_loop_device $LOOP_FILE` +loop_dev=`_create_loop_device $LOOP_FILE` -_mkfs_dev -d size=260g,agcount=2 $LOOP_DEV -_mount $LOOP_DEV $LOOP_MNT +_mkfs_dev -d size=260g,agcount=2 $loop_dev +_mount $loop_dev $LOOP_MNT BLOCK_SIZE=$(_get_file_block_size $LOOP_MNT)@@ -59,11 +59,11 @@ $XFS_IO_PROG -ft \ -c "falloc 0 $(($BLOCK_SIZE * 2097152))" \ $LOOP_MNT/foo >> $seqres.full -umount $LOOP_MNT -_check_xfs_filesystem $LOOP_DEV none none +_unmount $LOOP_MNT +_check_xfs_filesystem $loop_dev none none -_mkfs_dev -f $LOOP_DEV -_mount $LOOP_DEV $LOOP_MNT +_mkfs_dev -f $loop_dev +_mount $loop_dev $LOOP_MNT # check we trim both ends of the extent approproiately; this will fail # on 1k block size filesystems without the correct fixes in place.@@ -73,7 +73,10 @@ $XFS_IO_PROG -ft \ $LOOP_MNT/foo >> $seqres.full _unmount $LOOP_MNT -_check_xfs_filesystem $LOOP_DEV none none +_check_xfs_filesystem $loop_dev none none + +_destroy_loop_device $loop_dev +unset loop_dev # success, all done echo "Silence is golden"tests/xfs/078 +9
-7@@ -17,7 +17,7 @@ _cleanup() cd / rm -f $tmp.* _unmount $LOOP_MNT 2>/dev/null - [ -n "$LOOP_DEV" ] && _destroy_loop_device $LOOP_DEV 2>/dev/null + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev 2>/dev/null # try to keep the image file if test fails [ $status -eq 0 ] && rm -f $LOOP_IMG rmdir $LOOP_MNT@@ -55,7 +55,7 @@ _grow_loop() agsize=$5 $XFS_IO_PROG -f -c "truncate $original" $LOOP_IMG - LOOP_DEV=`_create_loop_device $LOOP_IMG` + loop_dev=`_create_loop_device $LOOP_IMG` dparam="" if [ -n "$agsize" ]; then@@ -67,15 +67,15 @@ _grow_loop() echo echo "*** mkfs loop file (size=$original)" - $MKFS_XFS_PROG -b size=$bsize $dparam $LOOP_DEV | \ + $MKFS_XFS_PROG -b size=$bsize $dparam $loop_dev | \ _filter_mkfs 2>/dev/null echo "*** extend loop file" - _destroy_loop_device $LOOP_DEV + _destroy_loop_device $loop_dev $XFS_IO_PROG -c "pwrite $new_size $bsize" $LOOP_IMG | _filter_io - LOOP_DEV=`_create_loop_device $LOOP_IMG` + loop_dev=`_create_loop_device $LOOP_IMG` echo "*** mount loop filesystem" - _mount -t xfs $LOOP_DEV $LOOP_MNT + _mount $loop_dev $LOOP_MNT echo "*** grow loop filesystem" $XFS_GROWFS_PROG $LOOP_MNT 2>&1 | _filter_growfs 2>&1@@ -87,9 +87,11 @@ _grow_loop() if [ "$check" -gt "0" ] then echo "*** check" - _check_xfs_filesystem $LOOP_IMG none none + _check_xfs_filesystem $loop_dev none none fi + _destroy_loop_device $loop_dev + unset loop_dev rm -f $LOOP_IMG }tests/xfs/148 +14
-9@@ -15,7 +15,7 @@ _cleanup() { cd / _unmount $mntpt > /dev/null 2>&1 - _destroy_loop_device $loopdev > /dev/null 2>&1 + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev rm -r -f $tmp.* }@@ -48,12 +48,12 @@ rm -f $imgfile $imgfile.old # even when security xattrs are present so we are always doing name matches on # lookup and not name hash compares as leaf/node forms will do. $XFS_IO_PROG -f -c 'truncate 40m' $imgfile -loopdev=$(_create_loop_device $imgfile) -MKFS_OPTIONS="-m crc=0 -i size=512" _mkfs_dev $loopdev >> $seqres.full +loop_dev=$(_create_loop_device $imgfile) +MKFS_OPTIONS="-m crc=0 -i size=512" _mkfs_dev $loop_dev >> $seqres.full # Mount image file mkdir -p $mntpt -_mount $loopdev $mntpt +_mount $loop_dev $mntpt echo "creating entries" >> $seqres.full@@ -91,7 +91,8 @@ cat $tmp.log | _filter_test_dir # Corrupt the entries _unmount $mntpt -_destroy_loop_device $loopdev +_destroy_loop_device $loop_dev +unset loop_dev cp $imgfile $imgfile.old sed -b \ -e "s/$nullstr/too_many\x00beans/g" \@@ -99,8 +100,9 @@ sed -b \ -i $imgfile test "$(md5sum < $imgfile)" != "$(md5sum < $imgfile.old)" || _fail "sed failed to change the image file?" -loopdev=$(_create_loop_device $imgfile) -_mount $loopdev $mntpt + +loop_dev=$(_create_loop_device $imgfile) +_mount $loop_dev $mntpt # Try to access the corrupt metadata echo "++ ACCESSING BAD METADATA" | tee -a $seqres.full@@ -111,7 +113,7 @@ cat $tmp.log | _filter_test_dir | sed -e '/Could not list/d' echo "does scrub complain?" >> $seqres.full # Does scrub complain about this? -if _supports_xfs_scrub $mntpt $loopdev; then +if _supports_xfs_scrub $mntpt $loop_dev; then $XFS_SCRUB_PROG -n $mntpt >> $seqres.full 2>&1 res=$? test $((res & 1)) -eq 0 && \@@ -122,11 +124,14 @@ echo "does repair complain?" >> $seqres.full # Does repair complain about this? _unmount $mntpt -$XFS_REPAIR_PROG -n $loopdev >> $seqres.full 2>&1 +$XFS_REPAIR_PROG -n $loop_dev >> $seqres.full 2>&1 res=$? test $res -eq 1 || \ echo "repair failed to report corruption ($res)" +_destroy_loop_device $loop_dev +unset loop_dev + # success, all done status=0 exittests/xfs/149 +4
-0@@ -84,6 +84,10 @@ $XFS_GROWFS_PROG -D 16384 $loop_symlink > /dev/null echo "=== xfs_growfs - check device node ===" $XFS_GROWFS_PROG -D 20480 $loop_dev > /dev/null +_unmount $mntdir +_destroy_loop_device $loop_dev +unset loop_dev + # success, all done status=0 exittests/xfs/216 +10
-5@@ -15,6 +15,7 @@ _begin_fstest log metadata auto quick _cleanup() { _unmount $LOOP_MNT > /dev/null 2>&1 + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev cd / rm -f $tmp.* }@@ -24,7 +25,7 @@ _scratch_mkfs_xfs >/dev/null 2>&1 _scratch_mount _require_loop -LOOP_DEV=$SCRATCH_MNT/test_fs +LOOP_IMG=$SCRATCH_MNT/test_fs LOOP_MNT=$SCRATCH_MNT/test_fs_dir loop_mkfs_opts=@@ -55,22 +56,26 @@ _do_mkfs() for i in $*; do echo -n "fssize=${i}g " $MKFS_XFS_PROG -f -b size=4096 -l version=2 \ - -d name=$LOOP_DEV,size=${i}g $loop_mkfs_opts |grep log - _mount -o loop -t xfs $LOOP_DEV $LOOP_MNT + -d size=${i}g $loop_mkfs_opts $loop_dev |grep log + _mount $loop_dev $LOOP_MNT echo "test write" > $LOOP_MNT/test _unmount $LOOP_MNT > /dev/null 2>&1 done } # make large holey file -$XFS_IO_PROG -f -c "truncate 256g" $LOOP_DEV +$XFS_IO_PROG -f -c "truncate 256g" $LOOP_IMG -choose_golden_output $0 $LOOP_DEV +choose_golden_output $0 $LOOP_IMG #make loopback mount dir mkdir $LOOP_MNT +loop_dev=$(_create_loop_device $LOOP_IMG) + # walk over standard sizes (up to 256GB) _do_mkfs 1 2 4 8 16 32 64 128 256 +_destroy_loop_device $loop_dev +unset loop_dev status=0 exittests/xfs/217 +16
-6@@ -12,6 +12,12 @@ _begin_fstest log metadata auto # Import common functions. . ./common/filter +_cleanup() +{ + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev + cd / + rm -f $tmp.* +} _require_scratch _scratch_mkfs_xfs >/dev/null 2>&1@@ -20,7 +26,7 @@ _scratch_mount _require_fs_space $SCRATCH_MNT 2202000 _require_loop -LOOP_DEV=$SCRATCH_MNT/test_fs +LOOP_IMG=$SCRATCH_MNT/test_fs LOOP_MNT=$SCRATCH_MNT/test_fs_dir _do_mkfs()@@ -28,28 +34,30 @@ _do_mkfs() for i in $*; do echo -n "fssize=${i}g " $MKFS_XFS_PROG -f -b size=4096 -l version=2 \ - -d name=$LOOP_DEV,size=${i}g |grep log - _mount -o loop -t xfs $LOOP_DEV $LOOP_MNT + -d size=${i}g $loop_dev |grep log + _mount $loop_dev $LOOP_MNT echo "test write" > $LOOP_MNT/test _unmount $LOOP_MNT > /dev/null 2>&1 # punch out the previous blocks so that we keep the amount of # disk space the test requires down to a minimum. - $XFS_IO_PROG -f -c "unresvsp 0 16383g" $LOOP_DEV + $XFS_IO_PROG -f -c "unresvsp 0 16383g" $LOOP_IMG done } # make large holey file -$XFS_IO_PROG -f -c "truncate 16383g" $LOOP_DEV +$XFS_IO_PROG -f -c "truncate 16383g" $LOOP_IMG #make loopback mount dir mkdir $LOOP_MNT # test if large logs are supported -$MKFS_XFS_PROG -f -l size=256m -d name=$LOOP_DEV,size=10g > /dev/null 2>&1 +$MKFS_XFS_PROG -f -l size=256m -d name=$LOOP_IMG,size=10g > /dev/null 2>&1 if [ $? -ne 0 ]; then _notrun "large log sizes not supported by mkfs" fi +loop_dev=$(_create_loop_device $LOOP_IMG) + # # walk over "new" sizes supported by recent xfsprogs. # Note that the last test is for 16TB-1GB as 32bit platforms only support@@ -57,5 +65,7 @@ fi # _do_mkfs 512 1024 2048 4096 8192 16383 +_destroy_loop_device $loop_dev +unset loop_dev status=0 exittests/xfs/250 +9
-5@@ -14,7 +14,8 @@ _cleanup() { cd / _unmount $LOOP_MNT 2>/dev/null - rm -f $LOOP_DEV + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev + rm -f $LOOP_IMG rmdir $LOOP_MNT rm -f $tmp.* }@@ -26,7 +27,7 @@ _require_test _require_loop _require_xfs_io_command "falloc" -LOOP_DEV=$TEST_DIR/$seq.fs +LOOP_IMG=$TEST_DIR/$seq.fs LOOP_MNT=$TEST_DIR/$seq.mnt _filter_io()@@ -45,7 +46,7 @@ _test_loop() agsize=$2 fsize=$3 - dparam="file,name=$LOOP_DEV,size=$size" + dparam="file,name=$LOOP_IMG,size=$size" if [ -n "$agsize" ]; then dparam="$dparam,agsize=$agsize" fi@@ -55,7 +56,8 @@ _test_loop() | _filter_mkfs 2>/dev/null echo "*** mount loop filesystem" - _mount -t xfs -o loop $LOOP_DEV $LOOP_MNT + loop_dev=$(_create_loop_device $LOOP_IMG) + mount $loop_dev $LOOP_MNT echo "*** preallocate large file" $XFS_IO_PROG -f -c "resvsp 0 $fsize" $LOOP_MNT/foo | _filter_io@@ -64,7 +66,9 @@ _test_loop() _unmount $LOOP_MNT > /dev/null 2>&1 echo "*** check loop filesystem" - _check_xfs_filesystem $LOOP_DEV none none + _check_xfs_filesystem $loop_dev none none + _destroy_loop_device $loop_dev + unset loop_dev } _test_loop 50g 16m 40Gtests/xfs/259 +8
-5@@ -12,7 +12,10 @@ _begin_fstest auto quick # Override the default cleanup function. _cleanup() { - rm -f "$testfile" + [ -n "$loop_dev" ] && _destroy_loop_device $testfile + rm -f "$testfile" + cd / + rm -f $tmp.* } # Import common functions.@@ -45,13 +48,13 @@ for del in $sizes_to_check; do rm -f "$testfile" dd if=/dev/zero "of=$testfile" bs=1 count=0 seek=$ddseek \ >/dev/null 2>&1 || echo "dd failed" - lofile=$(losetup -f) - losetup $lofile "$testfile" - $MKFS_XFS_PROG -l size=32m -b size=$bs $lofile | _filter_mkfs \ + loop_dev=$(_create_loop_device $testfile) + $MKFS_XFS_PROG -l size=32m -b size=$bs $loop_dev | _filter_mkfs \ >/dev/null 2> $tmp.mkfs || echo "mkfs failed!" . $tmp.mkfs sync - losetup -d $lofile + _destroy_loop_device $loop_dev + unset loop_dev done donetests/xfs/513 +20
-24@@ -15,12 +15,8 @@ _cleanup() cd / rm -f $tmp.* _unmount $LOOP_MNT 2>/dev/null - if [ -n "$LOOP_DEV" ];then - _destroy_loop_device $LOOP_DEV 2>/dev/null - fi - if [ -n "$LOOP_SPARE_DEV" ];then - _destroy_loop_device $LOOP_SPARE_DEV 2>/dev/null - fi + [ -n $loop_dev ] &&_destroy_loop_device $loop_dev + [ -n $loop_spare_dev ] &&_destroy_loop_device $loop_spare_dev rm -f $LOOP_IMG rm -f $LOOP_SPARE_IMG rmdir $LOOP_MNT@@ -42,11 +38,11 @@ LOOP_MNT=$TEST_DIR/$seq.mnt echo "** create loop device" $XFS_IO_PROG -f -c "truncate 32g" $LOOP_IMG -LOOP_DEV=`_create_loop_device $LOOP_IMG` +loop_dev=`_create_loop_device $LOOP_IMG` echo "** create loop log device" $XFS_IO_PROG -f -c "truncate 1g" $LOOP_SPARE_IMG -LOOP_SPARE_DEV=`_create_loop_device $LOOP_SPARE_IMG` +loop_spare_dev=`_create_loop_device $LOOP_SPARE_IMG` echo "** create loop mount point" rmdir $LOOP_MNT 2>/dev/null@@ -55,8 +51,8 @@ mkdir -p $LOOP_MNT || _fail "cannot create loopback mount point" filter_loop() { sed -e "s,\B$LOOP_MNT,LOOP_MNT,g" \ - -e "s,\B$LOOP_DEV,LOOP_DEV,g" \ - -e "s,\B$LOOP_SPARE_DEV,LOOP_SPARE_DEV,g" + -e "s,\B$loop_dev,LOOP_DEV,g" \ + -e "s,\B$loop_spare_dev,LOOP_SPARE_DEV,g" } filter_xfs_opt()@@ -69,22 +65,22 @@ MKFS_OPTIONS="" do_mkfs() { echo "FORMAT: $@" | filter_loop | tee -a $seqres.full - $MKFS_XFS_PROG -f $* $LOOP_DEV | _filter_mkfs >>$seqres.full 2>$tmp.mkfs + $MKFS_XFS_PROG -f $* $loop_dev | _filter_mkfs >>$seqres.full 2>$tmp.mkfs if [ "${PIPESTATUS[0]}" -ne 0 ]; then - _fail "Fails on _mkfs_dev $* $LOOP_DEV" + _fail "Fails on _mkfs_dev $* $loop_dev" fi . $tmp.mkfs } is_dev_mounted() { - findmnt --source $LOOP_DEV >/dev/null + findmnt --source $loop_dev >/dev/null return $? } get_mount_info() { - findmnt --source $LOOP_DEV -o OPTIONS -n + findmnt --source $loop_dev -o OPTIONS -n } force_unmount()@@ -103,29 +99,29 @@ _do_test() local info # mount test - _mount $LOOP_DEV $LOOP_MNT $opts 2>>$seqres.full + _mount $loop_dev $LOOP_MNT $opts 2>>$seqres.full rc=$? if [ $rc -eq 0 ];then if [ "${mounted}" = "fail" ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: expect mount to fail, but it succeeded" return 1 fi is_dev_mounted if [ $? -ne 0 ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: fs not mounted even mount return 0" return 1 fi else if [ "${mounted}" = "pass" ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: expect mount to succeed, but it failed" return 1 fi is_dev_mounted if [ $? -eq 0 ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: fs is mounted even mount return non-zero" return 1 fi@@ -141,13 +137,13 @@ _do_test() rc=$? if [ $rc -eq 0 ];then if [ "$found" != "true" ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: expected to find \"$key\" in mount info \"$info\"" return 1 fi else if [ "$found" != "false" ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: did not expect to find \"$key\" in \"$info\"" return 1 fi@@ -253,9 +249,9 @@ do_test "-o logbsize=512k" fail # Test logdev do_mkfs do_test "" pass "logdev" "false" -do_test "-o logdev=$LOOP_SPARE_DEV" fail -do_mkfs -l logdev=$LOOP_SPARE_DEV -do_test "-o logdev=$LOOP_SPARE_DEV" pass "logdev=$LOOP_SPARE_DEV" "true" +do_test "-o logdev=$loop_spare_dev" fail +do_mkfs -l logdev=$loop_spare_dev +do_test "-o logdev=$loop_spare_dev" pass "logdev=$loop_spare_dev" "true" do_test "" fail # Test noaligntests/xfs/521 +6
-2@@ -21,7 +21,7 @@ _cleanup() { cd / _scratch_unmount >> $seqres.full 2>&1 - test -e "$rtdev" && losetup -d $rtdev >> $seqres.full 2>&1 + [ -n "$rt_loop_dev" ] && _destroy_loop_device $rt_loop_dev rm -f $tmp.* $TEST_DIR/$seq.rtvol }@@ -35,7 +35,7 @@ _require_no_large_scratch_dev echo "Create fake rt volume" truncate -s 400m $TEST_DIR/$seq.rtvol -rtdev=$(_create_loop_device $TEST_DIR/$seq.rtvol) +rt_loop_dev=$(_create_loop_device $TEST_DIR/$seq.rtvol) echo "Format and mount 100m rt volume" export USE_EXTERNAL=yes@@ -69,6 +69,10 @@ cp -p $testdir/original $testdir/copy3 echo "Check filesystem" _check_scratch_fs +_scratch_unmount +_destroy_loop_device $rt_loop_dev +unset rt_loop_dev + # success, all done status=0 exittests/xfs/528 +7
-3@@ -15,7 +15,7 @@ _cleanup() { cd / _scratch_unmount >> $seqres.full 2>&1 - test -e "$rtdev" && losetup -d $rtdev >> $seqres.full 2>&1 + [ -n "$rt_loop_dev" ] && _destroy_loop_device $rt_loop_dev rm -f $tmp.* $TEST_DIR/$seq.rtvol }@@ -155,11 +155,11 @@ test_ops() { echo "Create fake rt volume" $XFS_IO_PROG -f -c "truncate 400m" $TEST_DIR/$seq.rtvol -rtdev=$(_create_loop_device $TEST_DIR/$seq.rtvol) +rt_loop_dev=$(_create_loop_device $TEST_DIR/$seq.rtvol) echo "Make sure synth rt volume works" export USE_EXTERNAL=yes -export SCRATCH_RTDEV=$rtdev +export SCRATCH_RTDEV=$rt_loop_dev _scratch_mkfs > $seqres.full _try_scratch_mount || \ _notrun "Could not mount with synthetic rt volume"@@ -170,6 +170,10 @@ test_ops 262144 # not a power of two test_ops 327680 +_scratch_unmount +_destroy_loop_device $rt_loop_dev +unset rt_loop_dev + # success, all done status=0 exittests/xfs/530 +6
-5@@ -15,7 +15,7 @@ _cleanup() { cd / _scratch_unmount >> $seqres.full 2>&1 - test -e "$rtdev" && losetup -d $rtdev >> $seqres.full 2>&1 + [ -n "$rt_loop_dev" ] && _destroy_loop_device $rt_loop_dev rm -f $tmp.* $TEST_DIR/$seq.rtvol }@@ -52,12 +52,12 @@ fi rtdevsz=$((nr_bits * rtextsz)) truncate -s $rtdevsz $TEST_DIR/$seq.rtvol -rtdev=$(_create_loop_device $TEST_DIR/$seq.rtvol) +rt_loop_dev=$(_create_loop_device $TEST_DIR/$seq.rtvol) echo "Format and mount rt volume" export USE_EXTERNAL=yes -export SCRATCH_RTDEV=$rtdev +export SCRATCH_RTDEV=$rt_loop_dev _scratch_mkfs -d size=$((1024 * 1024 * 1024)) \ -r size=${rtextsz},extsize=${rtextsz} >> $seqres.full _try_scratch_mount || _notrun "Couldn't mount fs with synthetic rt volume"@@ -116,8 +116,9 @@ done echo "Check filesystem" _check_scratch_fs -losetup -d $rtdev -rm -f $TEST_DIR/$seq.rtvol +_scratch_unmount +_destroy_loop_device $rt_loop_dev +unset rt_loop_dev # success, all done status=0tests/xfs/606 +8
-4@@ -13,10 +13,8 @@ _begin_fstest auto quick growfs _cleanup() { - local dev _unmount $LOOP_MNT 2>/dev/null - dev=$(losetup -j testfile | cut -d: -f1) - losetup -d $dev 2>/dev/null + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev rm -rf $LOOP_IMG $LOOP_MNT cd / rm -f $tmp.*@@ -40,7 +38,9 @@ $MKFS_XFS_PROG -f $LOOP_IMG >$seqres.full # Extend by just 8K, expected to start with the last full-size AG ends of # above 1G block device. $XFS_IO_PROG -f -c "truncate 1073750016" $LOOP_IMG -_mount -oloop $LOOP_IMG $LOOP_MNT + +loop_dev=$(_create_loop_device $LOOP_IMG) +_mount $loop_dev $LOOP_MNT # A known bug shows "XFS_IOC_FSGROWFSDATA xfsctl failed: No space left on # device" at here, refer to _fixed_by_kernel_commit above $XFS_GROWFS_PROG $LOOP_MNT >$seqres.full@@ -48,6 +48,10 @@ if [ $? -ne 0 ];then echo "xfs_growfs fails!" fi +_unmount $LOOP_MNT +_destroy_loop_device $loop_dev +unset loop_dev + echo "Silence is golden" # success, all done status=0tests/xfs/613 +14
-26@@ -15,14 +15,8 @@ _cleanup() cd / rm -f $tmp.* _unmount $LOOP_MNT 2>/dev/null - if [ -n "$LOOP_DEV" ];then - _destroy_loop_device $LOOP_DEV 2>/dev/null - fi - if [ -n "$LOOP_SPARE_DEV" ];then - _destroy_loop_device $LOOP_SPARE_DEV 2>/dev/null - fi + [ -n $loop_dev ] &&_destroy_loop_device $loop_dev rm -f $LOOP_IMG - rm -f $LOOP_SPARE_IMG rmdir $LOOP_MNT }@@ -38,16 +32,11 @@ _require_loop _require_xfs_io_command "falloc" LOOP_IMG=$TEST_DIR/$seq.dev -LOOP_SPARE_IMG=$TEST_DIR/$seq.logdev LOOP_MNT=$TEST_DIR/$seq.mnt echo "** create loop device" $XFS_IO_PROG -f -c "truncate 32g" $LOOP_IMG -LOOP_DEV=`_create_loop_device $LOOP_IMG` - -echo "** create loop log device" -$XFS_IO_PROG -f -c "truncate 1g" $LOOP_SPARE_IMG -LOOP_SPARE_DEV=`_create_loop_device $LOOP_SPARE_IMG` +loop_dev=`_create_loop_device $LOOP_IMG` echo "** create loop mount point" rmdir $LOOP_MNT 2>/dev/null@@ -56,8 +45,7 @@ mkdir -p $LOOP_MNT || _fail "cannot create loopback mount point" filter_loop() { sed -e "s,\B$LOOP_MNT,LOOP_MNT,g" \ - -e "s,\B$LOOP_DEV,LOOP_DEV,g" \ - -e "s,\B$LOOP_SPARE_DEV,LOOP_SPARE_DEV,g" + -e "s,\B$loop_dev,LOOP_DEV,g" } filter_xfs_opt()@@ -70,22 +58,22 @@ MKFS_OPTIONS="" do_mkfs() { echo "FORMAT: $@" | filter_loop | tee -a $seqres.full - $MKFS_XFS_PROG -f $* $LOOP_DEV | _filter_mkfs >>$seqres.full 2>$tmp.mkfs + $MKFS_XFS_PROG -f $* $loop_dev | _filter_mkfs >>$seqres.full 2>$tmp.mkfs if [ "${PIPESTATUS[0]}" -ne 0 ]; then - _fail "Fails on _mkfs_dev $* $LOOP_DEV" + _fail "Fails on _mkfs_dev $* $loop_dev" fi . $tmp.mkfs } is_dev_mounted() { - findmnt --source $LOOP_DEV >/dev/null + findmnt --source $loop_dev >/dev/null return $? } get_mount_info() { - findmnt --source $LOOP_DEV -o OPTIONS -n + findmnt --source $loop_dev -o OPTIONS -n } force_unmount()@@ -104,29 +92,29 @@ _do_test() local info # mount test - _mount $LOOP_DEV $LOOP_MNT $opts 2>>$seqres.full + _mount $loop_dev $LOOP_MNT $opts 2>>$seqres.full rc=$? if [ $rc -eq 0 ];then if [ "${mounted}" = "fail" ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: expect mount to fail, but it succeeded" return 1 fi is_dev_mounted if [ $? -ne 0 ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: fs not mounted even mount return 0" return 1 fi else if [ "${mounted}" = "pass" ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: expect mount to succeed, but it failed" return 1 fi is_dev_mounted if [ $? -eq 0 ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: fs is mounted even mount return non-zero" return 1 fi@@ -142,13 +130,13 @@ _do_test() rc=$? if [ $rc -eq 0 ];then if [ "$found" != "true" ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: expected to find \"$key\" in mount info \"$info\"" return 1 fi else if [ "$found" != "false" ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: did not expect to find \"$key\" in \"$info\"" return 1 fitests/xfs/613.out +0
-1@@ -1,6 +1,5 @@ QA output created by 613 ** create loop device -** create loop log device ** create loop mount point ** start xfs mount testing ... FORMAT: -m crc=0Signature · 1 line
2.45.2
Re: [PATCH 13/40] fstests: clean up loop device instantiation
In reply to Dave ChinnerQuoted 174 lines · Dave Chinner
From: Dave Chinner <[email protected]>
Lots of tests do there own special thing with loop devices rather than using
_create_loop_device()and_destroy_loop_device(). This oftens means they do not clean up after themselves properly, leaving stale loop devices around that result in unmountable test or scratch devices. This is common when tests are killed by user interrupt.Even the tests that do use
_destroy_loop_deviceand try to clean up often do it incorrectly, leading to spurious error messages.Some tests try to use dynamic instantiation via "mount -o loop", but then don't clean up in the correct order or hack around to find the loop device that was instantiated because the test needs to know the instantiated device name
Clean this up by converting all the tests to use
_create_loop_device()and_destroy_loop_device(). In all the tests, use the variable "loop_dev" for the device consistently. In_destroy_loop_device(), test that a device name has been passed so that we don't try to clean up the same device twice (e.g. once before test exit and again from the_cleanup()function). When we destroy a loop device, unset the variable used to hold the loop device name so that we don't try to destroy it twice.This results in much more reliable cleanup and clean exit from fstests when killed by the user.
Signed-off-by Dave Chinner <[email protected]>--- common/metadump | 32 ++++++++++++-------------------- tests/generic/067 | 11 ++++++++--- tests/generic/361 | 6 +++++- tests/generic/563 | 14 +++++++++----- tests/generic/564 | 12 ++++++------ tests/generic/590 | 9 ++++++--- tests/generic/744 | 4 ++-- tests/generic/746 | 6 +++++- tests/xfs/014 | 7 +++++-- tests/xfs/049 | 35 +++++++++++++++++++++-------------- tests/xfs/073 | 24 +++++++++++++----------- tests/xfs/074 | 21 ++++++++++++--------- tests/xfs/078 | 16 +++++++++------- tests/xfs/148 | 23 ++++++++++++++--------- tests/xfs/149 | 4 ++++ tests/xfs/216 | 15 ++++++++++----- tests/xfs/217 | 22 ++++++++++++++++------ tests/xfs/250 | 14 +++++++++----- tests/xfs/259 | 13 ++++++++----- tests/xfs/513 | 44 ++++++++++++++++++++------------------------ tests/xfs/521 | 8 ++++++-- tests/xfs/528 | 10 +++++++--- tests/xfs/530 | 11 ++++++----- tests/xfs/606 | 12 ++++++++---- tests/xfs/613 | 40 ++++++++++++++-------------------------- tests/xfs/613.out | 1 - 26 files changed, 235 insertions(+), 179 deletions(-)common/metadump +12
-20@@ -24,17 +24,9 @@ _xfs_cleanup_verify_metadump() test -n "$XFS_METADUMP_FILE" && rm -f "$XFS_METADUMP_FILE" - if [ -n "$XFS_METADUMP_IMG" ]; then - losetup -n -a -O BACK-FILE,NAME | grep "^$XFS_METADUMP_IMG" | while read backing ldev; do - losetup -d "$ldev" - done - - # Don't call rm directly with a globbed argument here to avoid - # issues issues with variable expansions. - for img in "$XFS_METADUMP_IMG"*; do - test -e "$img" && rm -f "$img" - done - fi + [ -n "$md_data_loop_dev" ] && _destroy_loop_device $md_data_loop_dev + [ -n "$md_log_loop_dev" ] && _destroy_loop_device $md_log_loop_dev + rm -f $data_img $log_img } # Can xfs_metadump snapshot the fs metadata to a v1 metadump file?@@ -122,25 +114,25 @@ _xfs_verify_metadump_v2() _scratch_xfs_mdrestore $metadump_file # Create loopdev for data device so we can mount the fs - data_loop=$(_create_loop_device $data_img) + md_data_loop_dev=$(_create_loop_device $data_img) # Create loopdev for log device if we recovered anything - test -s "$log_img" && log_loop=$(_create_loop_device $log_img) + test -s "$log_img" && md_log_loop_dev=$(_create_loop_device $log_img) # Mount fs, run an extra test, fsck, and unmount - SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop _scratch_mount + SCRATCH_DEV=$md_data_loop_dev SCRATCH_LOGDEV=$md_log_loop_dev _scratch_mount if [ -n "$extra_test" ]; then - SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop $extra_test + SCRATCH_DEV=$md_data_loop_dev SCRATCH_LOGDEV=$md_log_loop_dev $extra_test fi - SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop _check_xfs_scratch_fs - SCRATCH_DEV=$data_loop _scratch_unmount + SCRATCH_DEV=$md_data_loop_dev SCRATCH_LOGDEV=$md_log_loop_dev _check_xfs_scratch_fs + _unmount $md_data_loop_dev # Tear down what we created - if [ -b "$log_loop" ]; then - _destroy_loop_device $log_loop + if [ -b "$md_log_loop_dev" ]; then + _destroy_loop_device $md_log_loop_dev rm -f $log_img fi - _destroy_loop_device $data_loop + _destroy_loop_device $md_data_loop_dev rm -f $data_img }tests/generic/067 +8
-3@@ -37,13 +37,18 @@ mount_nonexistent_mnt() $MOUNT_PROG $SCRATCH_DEV $TEST_DIR/nosuchdir >>$seqres.full 2>&1 } -# fs driver should be able to handle mounting a free loop device gracefully -# xfs ever hung, "ec53d1d xfs: don't block on buffer read errors" fixed it +# fs driver should be able to handle mounting a free loop device gracefully xfs +# ever hung, "ec53d1d xfs: don't block on buffer read errors" fixed it +# +# Using 'losetup -f' like this is racy. We might end up mounting a real loop dev +# here, so unmount $SCRATCH_MNT (not the loop dev we might not own!) and ignore +# any error it might return. mount_free_loopdev() { echo "# mount a free loop device" >>$seqres.full loopdev=`losetup -f` - $MOUNT_PROG -t $FSTYP $loopdev $SCRATCH_MNT >>$seqres.full 2>&1 + _mount $loopdev $SCRATCH_MNT >>$seqres.full 2>&1 + _unmount $SCRATCH_MNT >> /dev/null 2>&1 } # mount with wrong fs type specified.tests/generic/361 +5
-1@@ -17,7 +17,7 @@ _begin_fstest auto quick _cleanup() { _unmount $fs_mnt - _destroy_loop_device $loop_dev + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev cd / rm -f $tmp.* }@@ -54,6 +54,10 @@ $XFS_IO_PROG -fc "pwrite 0 520m" $fs_mnt/testfile >>$seqres.full 2>&1 # remount should not hang $MOUNT_PROG -o remount,ro $fs_mnt >>$seqres.full 2>&1 +_unmount $fs_mnt +_destroy_loop_device $loop_dev +unset loop_dev + # success, all done echo "Silence is golden" status=0tests/generic/563 +0
-0
tests/generic/563 reset() · line 90
Quoted 7 lines
tests/generic/563 +1
-3@@ -87,12 +87,12 @@ reset() # cgroup I/O accounting doesn't work on partitions. Use a loop device to rule # that out. -LOOP_DEV=$(_create_loop_device $SCRATCH_DEV) -smajor=$((0x`stat -L -c %t $LOOP_DEV`)) -sminor=$((0x`stat -L -c %T $LOOP_DEV`)) +loop_dev=$(_create_loop_device $SCRATCH_DEV)
This test case isn't be changed entirely, there're still some LOOP_DEV, e.g.
_destroy_loop_device $LOOP_DEV > /dev/null 2>&1 ... _mount $LOOP_DEV $SCRATCH_MNT || _fail "mount failed"
I'll change them to loop_dev.
Thanks,
Zorro
Quoted 1,133 lines
tests/generic/563 +8
-2+smajor=$((0x`stat -L -c %t $loop_dev`)) +sminor=$((0x`stat -L -c %T $loop_dev`)) -_mkfs_dev $LOOP_DEV >> $seqres.full 2>&1 -_mount $LOOP_DEV $SCRATCH_MNT || _fail "mount failed" +_mkfs_dev $loop_dev >> $seqres.full 2>&1 +_mount $loop_dev $SCRATCH_MNT || _fail "mount failed" blksize=$(_get_block_size "$SCRATCH_MNT")@@ -147,6 +147,10 @@ if [ "$drop_io_cgroup" = 1 ]; then echo "-io" > $cgdir/cgroup.subtree_control || _fail "subtree control" fi +_unmount $SCRATCH_MNT +_destroy_loop_device $loop_dev +unset loop_dev + # success, all done status=0 exittests/generic/564 +6
-6@@ -19,7 +19,7 @@ _cleanup() { cd / rm -rf $tmp.* - [ -z "$loopdev" ] || _destroy_loop_device $loopdev + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev } # Import common functions.@@ -72,11 +72,11 @@ $XFS_IO_PROG -f -c "copy_range -l 32k $testdir" $testdir/copy echo echo source/destination as blkdev returns EINVAL $XFS_IO_PROG -f -c "truncate 128k" $testdir/img >> $seqres.full 2>&1 -loopdev=`_create_loop_device $testdir/img` -$XFS_IO_PROG -c "copy_range -l 32k $testdir/file" $loopdev -$XFS_IO_PROG -f -c "copy_range -l 32k $loopdev" $testdir/copy -_destroy_loop_device $loopdev -loopdev= +loop_dev=`_create_loop_device $testdir/img` +$XFS_IO_PROG -c "copy_range -l 32k $testdir/file" $loop_dev +$XFS_IO_PROG -f -c "copy_range -l 32k $loop_dev" $testdir/copy +_destroy_loop_device $loop_dev +unset loop_dev echo echo source/destination as chardev returns EINVALtests/generic/590 +6
-3@@ -15,9 +15,10 @@ _begin_fstest auto prealloc preallocrw # Override the default cleanup function. _cleanup() { + _scratch_unmount + [ -n $loop_dev ] && _destroy_loop_device $loop_dev cd / rm -f $tmp.* - test -n "$loop" && _destroy_loop_device "$loop" rm -f "$TEST_DIR/$seq" }@@ -56,9 +57,9 @@ if [[ $FSTYP = xfs ]]; then loopsz="$((filesz + (1 << 26)))" _require_fs_space "$TEST_DIR" $((loopsz / 1024)) $XFS_IO_PROG -c "truncate $loopsz" -f "$TEST_DIR/$seq" - loop="$(_create_loop_device "$TEST_DIR/$seq")" + loop_dev="$(_create_loop_device "$TEST_DIR/$seq")" USE_EXTERNAL=yes - SCRATCH_RTDEV="$loop" + SCRATCH_RTDEV="$loop_dev" disabled_features=() # disable reflink if not supported by realtime devices@@ -114,6 +115,8 @@ $XFS_IO_PROG -c "truncate 0" -c fsync "$SCRATCH_MNT/file" # We need to do this before the loop device gets torn down. _scratch_unmount _check_scratch_fs +_destroy_loop_device $loop_dev +unset loop_dev echo "Silence is golden" status=0tests/generic/744 +2
-2@@ -17,8 +17,8 @@ _cleanup() _unmount $mnt2 &> /dev/null _unmount $mnt1 &> /dev/null - [ -b "$loop_dev2" ] && losetup -d $loop_dev2 - [ -b "$loop_dev1" ] && losetup -d $loop_dev1 + [ -b "$loop_dev2" ] && _destroy_loop_device $loop_dev2 + [ -b "$loop_dev1" ] && _destroy_loop_device $loop_dev1 [ -n "$seq" ] && rm -rf $TEST_DIR/$seq }tests/generic/746 +5
-1@@ -39,7 +39,7 @@ esac _cleanup() { _unmount $loop_mnt &> /dev/null - _destroy_loop_device $loop_dev + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev if [ $status -eq 0 ]; then rm -rf $tmp rm $img_file@@ -223,5 +223,9 @@ while read line; do done < $fiemap_after echo "done." +_unmount $loop_mnt +_destroy_loop_device $loop_dev +unset loop_dev + status=0 exittests/xfs/014 +5
-2@@ -23,6 +23,7 @@ _cleanup() { cd / _unmount $LOOP_MNT 2>/dev/null + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev _scratch_unmount 2>/dev/null rm -f $tmp.* }@@ -166,19 +167,21 @@ LOOP_FILE=$SCRATCH_MNT/$seq.fs LOOP_MNT=$SCRATCH_MNT/$seq.mnt $MKFS_XFS_PROG -d "file=1,name=$LOOP_FILE,size=10g" >> $seqres.full 2>&1 +loop_dev=$(_create_loop_device $LOOP_FILE) mkdir -p $LOOP_MNT -_mount -o uquota,gquota $LOOP_FILE $LOOP_MNT || \ +_mount -o uquota,gquota $loop_dev $LOOP_MNT || \ _fail "Failed to mount loop fs." _test_enospc $LOOP_MNT _test_edquot $LOOP_MNT _unmount $LOOP_MNT +_destroy_loop_device $loop_dev +unset loop_dev echo $orig_sp_time > /proc/sys/fs/xfs/speculative_prealloc_lifetime _scratch_unmount - status=0 exittests/xfs/049 +21
-14@@ -12,16 +12,17 @@ _begin_fstest rw auto quick # Override the default cleanup function. _cleanup() { - cd / - umount $SCRATCH_MNT/test2 > /dev/null 2>&1 - umount $SCRATCH_MNT/test > /dev/null 2>&1 - rm -f $tmp.* + cd / + _unmount $SCRATCH_MNT/test2 > /dev/null 2>&1 + _unmount $SCRATCH_MNT/test > /dev/null 2>&1 + [ -n "$loop_dev2" ] && _destroy_loop_device $loop_dev2 + [ -n "$loop_dev1" ] && _destroy_loop_device $loop_dev1 + rm -f $tmp.* - if [ -w $seqres.full ] - then - echo "--- mounts at end (after cleanup)" >> $seqres.full - mount >> $seqres.full - fi + if [ -w $seqres.full ]; then + echo "--- mounts at end (after cleanup)" >> $seqres.full + mount >> $seqres.full + fi } # Import common functions.@@ -64,7 +65,8 @@ mkdir $SCRATCH_MNT/test $SCRATCH_MNT/test2 >> $seqres.full 2>&1 \ || _fail "!!! failed to make mount points" _log "Mount xfs via loop" -mount -t xfs -o loop $SCRATCH_MNT/test.xfs $SCRATCH_MNT/test >> $seqres.full 2>&1 \ +loop_dev1=$(_create_loop_device $SCRATCH_MNT/test.xfs) +_mount $loop_dev1 $SCRATCH_MNT/test >> $seqres.full 2>&1 \ || _fail "!!! failed to loop mount xfs" _log "stress"@@ -80,11 +82,12 @@ dd if=/dev/zero of=$SCRATCH_MNT/test/test.ext2 bs=1024 count=10240 >> $seqres.fu || _fail "!!! create file failed" _log "Create ext2 fs in file on looped xfs" -echo y | mkfs -t ext2 $SCRATCH_MNT/test/test.ext2 >> $seqres.full 2>&1 \ +loop_dev2=$(_create_loop_device $SCRATCH_MNT/test/test.ext2) +echo y | mkfs -t ext2 $loop_dev2 >> $seqres.full 2>&1 \ || _fail "!!! failed to mkfs ext2 on xfs" _log "Mount ext2 on xfs via loop" -mount -t ext2 -o loop $SCRATCH_MNT/test/test.ext2 $SCRATCH_MNT/test2 >> $seqres.full 2>&1 \ +_mount $loop_dev2 $SCRATCH_MNT/test2 >> $seqres.full 2>&1 \ || _fail "!!! failed to loop mount xfs" _log "stress ext2 on xfs via loop"@@ -96,12 +99,16 @@ rm -rf $SCRATCH_MNT/test/* >> $seqres.full 2>&1 \ || _fail "!!! clean failed" _log "umount ext2 on xfs" -umount $SCRATCH_MNT/test2 >> $seqres.full 2>&1 \ +_unmount $SCRATCH_MNT/test2 >> $seqres.full 2>&1 \ || _fail "!!! umount ext2 failed" +_destroy_loop_device $loop_dev2 +unset loop_dev2 _log "umount xfs" -umount $SCRATCH_MNT/test >> $seqres.full 2>&1 \ +_unmount $SCRATCH_MNT/test >> $seqres.full 2>&1 \ || _fail "!!! umount xfs failed" +_destroy_loop_device $loop_dev1 +unset loop_dev1 echo "--- mounts at end (before cleanup)" >> $seqres.full mount >> $seqres.fulltests/xfs/073 +13
-11@@ -23,6 +23,8 @@ _cleanup() _scratch_unmount 2>/dev/null _unmount $imgs.loop 2>/dev/null _unmount $imgs.source_dir 2>/dev/null + [ -n "$loop_dev1" ] && _destroy_loop_device $loop_dev1 + [ -n "$loop_dev2" ] && _destroy_loop_device $loop_dev2 [ -d $imgs.loop ] && rmdir $imgs.loop [ -d $imgs.source_dir ] && rm -rf $imgs.source_dir rm -f $imgs.* $tmp.* /var/tmp/xfs_copy.log.*@@ -65,10 +67,11 @@ _verify_copy() rmdir $target_dir 2>/dev/null mkdir $target_dir - _mount -t xfs -o loop $target $target_dir 2>/dev/null + loop_dev1=$(_create_loop_device $target) + _mount $loop_dev1 $target_dir 2>/dev/null if [ $? -ne 0 ]; then echo retrying mount with nouuid option >>$seqres.full - _mount -t xfs -o loop -o nouuid $target $target_dir + _mount -o nouuid $loop_dev1 $target_dir if [ $? -ne 0 ]; then echo mount failed - evil! return@@ -100,6 +103,8 @@ _verify_copy() echo unmounting and removing new image _unmount $source_dir _unmount $target_dir > /dev/null 2>&1 + _destroy_loop_device $loop_dev1 + unset loop_dev1 rm -f $target }@@ -134,18 +139,15 @@ ${MKFS_XFS_PROG} -dfile,name=$imgs.source,size=100g \ rmdir $imgs.source_dir 2>/dev/null mkdir $imgs.source_dir -_mount -t xfs -o loop $imgs.source $imgs.source_dir -loop2=`mount | grep $imgs.source | grep -o -e 'loop=.*[^),]' | grep -o -e '/.*$'` +loop_dev2=$(_create_loop_device $imgs.source) +_mount $loop_dev2 $imgs.source_dir cp -a $here $imgs.source_dir -_mount -t xfs -o remount,ro $imgs.source $imgs.source_dir -$XFS_COPY_PROG $imgs.source $imgs.image | _filter_copy '#' $imgs.image '#' '#' +_mount -o remount,ro $loop_dev2 $imgs.source_dir +$XFS_COPY_PROG $loop_dev2 $imgs.image 2> /dev/null | _filter_copy '#' $imgs.image '#' '#' _verify_copy $imgs.image $imgs.source $imgs.source_dir -# HACK WARNING: -# -# We're done with the nested loop mount, now we have to clean -# up the pieces that mount is incapable of doing. -losetup -d $loop2 > /dev/null 2>&1 +_destroy_loop_device $loop_dev2 +unset loop_dev2 echo echo === copying scratch device to multiple targetstests/xfs/074 +12
-9@@ -26,7 +26,7 @@ _begin_fstest quick auto prealloc rw _cleanup() { cd / - _destroy_loop_device $LOOP_DEV + [ -n "$loop_dev" ] &&_destroy_loop_device $loop_dev rm -f $tmp.* $LOOP_FILE }@@ -45,10 +45,10 @@ LOOP_FILE=$TEST_DIR/$seq.img LOOP_MNT=$TEST_DIR/$seq.mnt mkdir -p $LOOP_MNT $XFS_IO_PROG -ft -c "truncate 1t" $LOOP_FILE >> $seqres.full -LOOP_DEV=`_create_loop_device $LOOP_FILE` +loop_dev=`_create_loop_device $LOOP_FILE` -_mkfs_dev -d size=260g,agcount=2 $LOOP_DEV -_mount $LOOP_DEV $LOOP_MNT +_mkfs_dev -d size=260g,agcount=2 $loop_dev +_mount $loop_dev $LOOP_MNT BLOCK_SIZE=$(_get_file_block_size $LOOP_MNT)@@ -59,11 +59,11 @@ $XFS_IO_PROG -ft \ -c "falloc 0 $(($BLOCK_SIZE * 2097152))" \ $LOOP_MNT/foo >> $seqres.full -umount $LOOP_MNT -_check_xfs_filesystem $LOOP_DEV none none +_unmount $LOOP_MNT +_check_xfs_filesystem $loop_dev none none -_mkfs_dev -f $LOOP_DEV -_mount $LOOP_DEV $LOOP_MNT +_mkfs_dev -f $loop_dev +_mount $loop_dev $LOOP_MNT # check we trim both ends of the extent approproiately; this will fail # on 1k block size filesystems without the correct fixes in place.@@ -73,7 +73,10 @@ $XFS_IO_PROG -ft \ $LOOP_MNT/foo >> $seqres.full _unmount $LOOP_MNT -_check_xfs_filesystem $LOOP_DEV none none +_check_xfs_filesystem $loop_dev none none + +_destroy_loop_device $loop_dev +unset loop_dev # success, all done echo "Silence is golden"tests/xfs/078 +9
-7@@ -17,7 +17,7 @@ _cleanup() cd / rm -f $tmp.* _unmount $LOOP_MNT 2>/dev/null - [ -n "$LOOP_DEV" ] && _destroy_loop_device $LOOP_DEV 2>/dev/null + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev 2>/dev/null # try to keep the image file if test fails [ $status -eq 0 ] && rm -f $LOOP_IMG rmdir $LOOP_MNT@@ -55,7 +55,7 @@ _grow_loop() agsize=$5 $XFS_IO_PROG -f -c "truncate $original" $LOOP_IMG - LOOP_DEV=`_create_loop_device $LOOP_IMG` + loop_dev=`_create_loop_device $LOOP_IMG` dparam="" if [ -n "$agsize" ]; then@@ -67,15 +67,15 @@ _grow_loop() echo echo "*** mkfs loop file (size=$original)" - $MKFS_XFS_PROG -b size=$bsize $dparam $LOOP_DEV | \ + $MKFS_XFS_PROG -b size=$bsize $dparam $loop_dev | \ _filter_mkfs 2>/dev/null echo "*** extend loop file" - _destroy_loop_device $LOOP_DEV + _destroy_loop_device $loop_dev $XFS_IO_PROG -c "pwrite $new_size $bsize" $LOOP_IMG | _filter_io - LOOP_DEV=`_create_loop_device $LOOP_IMG` + loop_dev=`_create_loop_device $LOOP_IMG` echo "*** mount loop filesystem" - _mount -t xfs $LOOP_DEV $LOOP_MNT + _mount $loop_dev $LOOP_MNT echo "*** grow loop filesystem" $XFS_GROWFS_PROG $LOOP_MNT 2>&1 | _filter_growfs 2>&1@@ -87,9 +87,11 @@ _grow_loop() if [ "$check" -gt "0" ] then echo "*** check" - _check_xfs_filesystem $LOOP_IMG none none + _check_xfs_filesystem $loop_dev none none fi + _destroy_loop_device $loop_dev + unset loop_dev rm -f $LOOP_IMG }tests/xfs/148 +14
-9@@ -15,7 +15,7 @@ _cleanup() { cd / _unmount $mntpt > /dev/null 2>&1 - _destroy_loop_device $loopdev > /dev/null 2>&1 + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev rm -r -f $tmp.* }@@ -48,12 +48,12 @@ rm -f $imgfile $imgfile.old # even when security xattrs are present so we are always doing name matches on # lookup and not name hash compares as leaf/node forms will do. $XFS_IO_PROG -f -c 'truncate 40m' $imgfile -loopdev=$(_create_loop_device $imgfile) -MKFS_OPTIONS="-m crc=0 -i size=512" _mkfs_dev $loopdev >> $seqres.full +loop_dev=$(_create_loop_device $imgfile) +MKFS_OPTIONS="-m crc=0 -i size=512" _mkfs_dev $loop_dev >> $seqres.full # Mount image file mkdir -p $mntpt -_mount $loopdev $mntpt +_mount $loop_dev $mntpt echo "creating entries" >> $seqres.full@@ -91,7 +91,8 @@ cat $tmp.log | _filter_test_dir # Corrupt the entries _unmount $mntpt -_destroy_loop_device $loopdev +_destroy_loop_device $loop_dev +unset loop_dev cp $imgfile $imgfile.old sed -b \ -e "s/$nullstr/too_many\x00beans/g" \@@ -99,8 +100,9 @@ sed -b \ -i $imgfile test "$(md5sum < $imgfile)" != "$(md5sum < $imgfile.old)" || _fail "sed failed to change the image file?" -loopdev=$(_create_loop_device $imgfile) -_mount $loopdev $mntpt + +loop_dev=$(_create_loop_device $imgfile) +_mount $loop_dev $mntpt # Try to access the corrupt metadata echo "++ ACCESSING BAD METADATA" | tee -a $seqres.full@@ -111,7 +113,7 @@ cat $tmp.log | _filter_test_dir | sed -e '/Could not list/d' echo "does scrub complain?" >> $seqres.full # Does scrub complain about this? -if _supports_xfs_scrub $mntpt $loopdev; then +if _supports_xfs_scrub $mntpt $loop_dev; then $XFS_SCRUB_PROG -n $mntpt >> $seqres.full 2>&1 res=$? test $((res & 1)) -eq 0 && \@@ -122,11 +124,14 @@ echo "does repair complain?" >> $seqres.full # Does repair complain about this? _unmount $mntpt -$XFS_REPAIR_PROG -n $loopdev >> $seqres.full 2>&1 +$XFS_REPAIR_PROG -n $loop_dev >> $seqres.full 2>&1 res=$? test $res -eq 1 || \ echo "repair failed to report corruption ($res)" +_destroy_loop_device $loop_dev +unset loop_dev + # success, all done status=0 exittests/xfs/149 +4
-0@@ -84,6 +84,10 @@ $XFS_GROWFS_PROG -D 16384 $loop_symlink > /dev/null echo "=== xfs_growfs - check device node ===" $XFS_GROWFS_PROG -D 20480 $loop_dev > /dev/null +_unmount $mntdir +_destroy_loop_device $loop_dev +unset loop_dev + # success, all done status=0 exittests/xfs/216 +10
-5@@ -15,6 +15,7 @@ _begin_fstest log metadata auto quick _cleanup() { _unmount $LOOP_MNT > /dev/null 2>&1 + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev cd / rm -f $tmp.* }@@ -24,7 +25,7 @@ _scratch_mkfs_xfs >/dev/null 2>&1 _scratch_mount _require_loop -LOOP_DEV=$SCRATCH_MNT/test_fs +LOOP_IMG=$SCRATCH_MNT/test_fs LOOP_MNT=$SCRATCH_MNT/test_fs_dir loop_mkfs_opts=@@ -55,22 +56,26 @@ _do_mkfs() for i in $*; do echo -n "fssize=${i}g " $MKFS_XFS_PROG -f -b size=4096 -l version=2 \ - -d name=$LOOP_DEV,size=${i}g $loop_mkfs_opts |grep log - _mount -o loop -t xfs $LOOP_DEV $LOOP_MNT + -d size=${i}g $loop_mkfs_opts $loop_dev |grep log + _mount $loop_dev $LOOP_MNT echo "test write" > $LOOP_MNT/test _unmount $LOOP_MNT > /dev/null 2>&1 done } # make large holey file -$XFS_IO_PROG -f -c "truncate 256g" $LOOP_DEV +$XFS_IO_PROG -f -c "truncate 256g" $LOOP_IMG -choose_golden_output $0 $LOOP_DEV +choose_golden_output $0 $LOOP_IMG #make loopback mount dir mkdir $LOOP_MNT +loop_dev=$(_create_loop_device $LOOP_IMG) + # walk over standard sizes (up to 256GB) _do_mkfs 1 2 4 8 16 32 64 128 256 +_destroy_loop_device $loop_dev +unset loop_dev status=0 exittests/xfs/217 +16
-6@@ -12,6 +12,12 @@ _begin_fstest log metadata auto # Import common functions. . ./common/filter +_cleanup() +{ + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev + cd / + rm -f $tmp.* +} _require_scratch _scratch_mkfs_xfs >/dev/null 2>&1@@ -20,7 +26,7 @@ _scratch_mount _require_fs_space $SCRATCH_MNT 2202000 _require_loop -LOOP_DEV=$SCRATCH_MNT/test_fs +LOOP_IMG=$SCRATCH_MNT/test_fs LOOP_MNT=$SCRATCH_MNT/test_fs_dir _do_mkfs()@@ -28,28 +34,30 @@ _do_mkfs() for i in $*; do echo -n "fssize=${i}g " $MKFS_XFS_PROG -f -b size=4096 -l version=2 \ - -d name=$LOOP_DEV,size=${i}g |grep log - _mount -o loop -t xfs $LOOP_DEV $LOOP_MNT + -d size=${i}g $loop_dev |grep log + _mount $loop_dev $LOOP_MNT echo "test write" > $LOOP_MNT/test _unmount $LOOP_MNT > /dev/null 2>&1 # punch out the previous blocks so that we keep the amount of # disk space the test requires down to a minimum. - $XFS_IO_PROG -f -c "unresvsp 0 16383g" $LOOP_DEV + $XFS_IO_PROG -f -c "unresvsp 0 16383g" $LOOP_IMG done } # make large holey file -$XFS_IO_PROG -f -c "truncate 16383g" $LOOP_DEV +$XFS_IO_PROG -f -c "truncate 16383g" $LOOP_IMG #make loopback mount dir mkdir $LOOP_MNT # test if large logs are supported -$MKFS_XFS_PROG -f -l size=256m -d name=$LOOP_DEV,size=10g > /dev/null 2>&1 +$MKFS_XFS_PROG -f -l size=256m -d name=$LOOP_IMG,size=10g > /dev/null 2>&1 if [ $? -ne 0 ]; then _notrun "large log sizes not supported by mkfs" fi +loop_dev=$(_create_loop_device $LOOP_IMG) + # # walk over "new" sizes supported by recent xfsprogs. # Note that the last test is for 16TB-1GB as 32bit platforms only support@@ -57,5 +65,7 @@ fi # _do_mkfs 512 1024 2048 4096 8192 16383 +_destroy_loop_device $loop_dev +unset loop_dev status=0 exittests/xfs/250 +9
-5@@ -14,7 +14,8 @@ _cleanup() { cd / _unmount $LOOP_MNT 2>/dev/null - rm -f $LOOP_DEV + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev + rm -f $LOOP_IMG rmdir $LOOP_MNT rm -f $tmp.* }@@ -26,7 +27,7 @@ _require_test _require_loop _require_xfs_io_command "falloc" -LOOP_DEV=$TEST_DIR/$seq.fs +LOOP_IMG=$TEST_DIR/$seq.fs LOOP_MNT=$TEST_DIR/$seq.mnt _filter_io()@@ -45,7 +46,7 @@ _test_loop() agsize=$2 fsize=$3 - dparam="file,name=$LOOP_DEV,size=$size" + dparam="file,name=$LOOP_IMG,size=$size" if [ -n "$agsize" ]; then dparam="$dparam,agsize=$agsize" fi@@ -55,7 +56,8 @@ _test_loop() | _filter_mkfs 2>/dev/null echo "*** mount loop filesystem" - _mount -t xfs -o loop $LOOP_DEV $LOOP_MNT + loop_dev=$(_create_loop_device $LOOP_IMG) + mount $loop_dev $LOOP_MNT echo "*** preallocate large file" $XFS_IO_PROG -f -c "resvsp 0 $fsize" $LOOP_MNT/foo | _filter_io@@ -64,7 +66,9 @@ _test_loop() _unmount $LOOP_MNT > /dev/null 2>&1 echo "*** check loop filesystem" - _check_xfs_filesystem $LOOP_DEV none none + _check_xfs_filesystem $loop_dev none none + _destroy_loop_device $loop_dev + unset loop_dev } _test_loop 50g 16m 40Gtests/xfs/259 +8
-5@@ -12,7 +12,10 @@ _begin_fstest auto quick # Override the default cleanup function. _cleanup() { - rm -f "$testfile" + [ -n "$loop_dev" ] && _destroy_loop_device $testfile + rm -f "$testfile" + cd / + rm -f $tmp.* } # Import common functions.@@ -45,13 +48,13 @@ for del in $sizes_to_check; do rm -f "$testfile" dd if=/dev/zero "of=$testfile" bs=1 count=0 seek=$ddseek \ >/dev/null 2>&1 || echo "dd failed" - lofile=$(losetup -f) - losetup $lofile "$testfile" - $MKFS_XFS_PROG -l size=32m -b size=$bs $lofile | _filter_mkfs \ + loop_dev=$(_create_loop_device $testfile) + $MKFS_XFS_PROG -l size=32m -b size=$bs $loop_dev | _filter_mkfs \ >/dev/null 2> $tmp.mkfs || echo "mkfs failed!" . $tmp.mkfs sync - losetup -d $lofile + _destroy_loop_device $loop_dev + unset loop_dev done donetests/xfs/513 +20
-24@@ -15,12 +15,8 @@ _cleanup() cd / rm -f $tmp.* _unmount $LOOP_MNT 2>/dev/null - if [ -n "$LOOP_DEV" ];then - _destroy_loop_device $LOOP_DEV 2>/dev/null - fi - if [ -n "$LOOP_SPARE_DEV" ];then - _destroy_loop_device $LOOP_SPARE_DEV 2>/dev/null - fi + [ -n $loop_dev ] &&_destroy_loop_device $loop_dev + [ -n $loop_spare_dev ] &&_destroy_loop_device $loop_spare_dev rm -f $LOOP_IMG rm -f $LOOP_SPARE_IMG rmdir $LOOP_MNT@@ -42,11 +38,11 @@ LOOP_MNT=$TEST_DIR/$seq.mnt echo "** create loop device" $XFS_IO_PROG -f -c "truncate 32g" $LOOP_IMG -LOOP_DEV=`_create_loop_device $LOOP_IMG` +loop_dev=`_create_loop_device $LOOP_IMG` echo "** create loop log device" $XFS_IO_PROG -f -c "truncate 1g" $LOOP_SPARE_IMG -LOOP_SPARE_DEV=`_create_loop_device $LOOP_SPARE_IMG` +loop_spare_dev=`_create_loop_device $LOOP_SPARE_IMG` echo "** create loop mount point" rmdir $LOOP_MNT 2>/dev/null@@ -55,8 +51,8 @@ mkdir -p $LOOP_MNT || _fail "cannot create loopback mount point" filter_loop() { sed -e "s,\B$LOOP_MNT,LOOP_MNT,g" \ - -e "s,\B$LOOP_DEV,LOOP_DEV,g" \ - -e "s,\B$LOOP_SPARE_DEV,LOOP_SPARE_DEV,g" + -e "s,\B$loop_dev,LOOP_DEV,g" \ + -e "s,\B$loop_spare_dev,LOOP_SPARE_DEV,g" } filter_xfs_opt()@@ -69,22 +65,22 @@ MKFS_OPTIONS="" do_mkfs() { echo "FORMAT: $@" | filter_loop | tee -a $seqres.full - $MKFS_XFS_PROG -f $* $LOOP_DEV | _filter_mkfs >>$seqres.full 2>$tmp.mkfs + $MKFS_XFS_PROG -f $* $loop_dev | _filter_mkfs >>$seqres.full 2>$tmp.mkfs if [ "${PIPESTATUS[0]}" -ne 0 ]; then - _fail "Fails on _mkfs_dev $* $LOOP_DEV" + _fail "Fails on _mkfs_dev $* $loop_dev" fi . $tmp.mkfs } is_dev_mounted() { - findmnt --source $LOOP_DEV >/dev/null + findmnt --source $loop_dev >/dev/null return $? } get_mount_info() { - findmnt --source $LOOP_DEV -o OPTIONS -n + findmnt --source $loop_dev -o OPTIONS -n } force_unmount()@@ -103,29 +99,29 @@ _do_test() local info # mount test - _mount $LOOP_DEV $LOOP_MNT $opts 2>>$seqres.full + _mount $loop_dev $LOOP_MNT $opts 2>>$seqres.full rc=$? if [ $rc -eq 0 ];then if [ "${mounted}" = "fail" ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: expect mount to fail, but it succeeded" return 1 fi is_dev_mounted if [ $? -ne 0 ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: fs not mounted even mount return 0" return 1 fi else if [ "${mounted}" = "pass" ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: expect mount to succeed, but it failed" return 1 fi is_dev_mounted if [ $? -eq 0 ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: fs is mounted even mount return non-zero" return 1 fi@@ -141,13 +137,13 @@ _do_test() rc=$? if [ $rc -eq 0 ];then if [ "$found" != "true" ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: expected to find \"$key\" in mount info \"$info\"" return 1 fi else if [ "$found" != "false" ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: did not expect to find \"$key\" in \"$info\"" return 1 fi@@ -253,9 +249,9 @@ do_test "-o logbsize=512k" fail # Test logdev do_mkfs do_test "" pass "logdev" "false" -do_test "-o logdev=$LOOP_SPARE_DEV" fail -do_mkfs -l logdev=$LOOP_SPARE_DEV -do_test "-o logdev=$LOOP_SPARE_DEV" pass "logdev=$LOOP_SPARE_DEV" "true" +do_test "-o logdev=$loop_spare_dev" fail +do_mkfs -l logdev=$loop_spare_dev +do_test "-o logdev=$loop_spare_dev" pass "logdev=$loop_spare_dev" "true" do_test "" fail # Test noaligntests/xfs/521 +6
-2@@ -21,7 +21,7 @@ _cleanup() { cd / _scratch_unmount >> $seqres.full 2>&1 - test -e "$rtdev" && losetup -d $rtdev >> $seqres.full 2>&1 + [ -n "$rt_loop_dev" ] && _destroy_loop_device $rt_loop_dev rm -f $tmp.* $TEST_DIR/$seq.rtvol }@@ -35,7 +35,7 @@ _require_no_large_scratch_dev echo "Create fake rt volume" truncate -s 400m $TEST_DIR/$seq.rtvol -rtdev=$(_create_loop_device $TEST_DIR/$seq.rtvol) +rt_loop_dev=$(_create_loop_device $TEST_DIR/$seq.rtvol) echo "Format and mount 100m rt volume" export USE_EXTERNAL=yes@@ -69,6 +69,10 @@ cp -p $testdir/original $testdir/copy3 echo "Check filesystem" _check_scratch_fs +_scratch_unmount +_destroy_loop_device $rt_loop_dev +unset rt_loop_dev + # success, all done status=0 exittests/xfs/528 +7
-3@@ -15,7 +15,7 @@ _cleanup() { cd / _scratch_unmount >> $seqres.full 2>&1 - test -e "$rtdev" && losetup -d $rtdev >> $seqres.full 2>&1 + [ -n "$rt_loop_dev" ] && _destroy_loop_device $rt_loop_dev rm -f $tmp.* $TEST_DIR/$seq.rtvol }@@ -155,11 +155,11 @@ test_ops() { echo "Create fake rt volume" $XFS_IO_PROG -f -c "truncate 400m" $TEST_DIR/$seq.rtvol -rtdev=$(_create_loop_device $TEST_DIR/$seq.rtvol) +rt_loop_dev=$(_create_loop_device $TEST_DIR/$seq.rtvol) echo "Make sure synth rt volume works" export USE_EXTERNAL=yes -export SCRATCH_RTDEV=$rtdev +export SCRATCH_RTDEV=$rt_loop_dev _scratch_mkfs > $seqres.full _try_scratch_mount || \ _notrun "Could not mount with synthetic rt volume"@@ -170,6 +170,10 @@ test_ops 262144 # not a power of two test_ops 327680 +_scratch_unmount +_destroy_loop_device $rt_loop_dev +unset rt_loop_dev + # success, all done status=0 exittests/xfs/530 +6
-5@@ -15,7 +15,7 @@ _cleanup() { cd / _scratch_unmount >> $seqres.full 2>&1 - test -e "$rtdev" && losetup -d $rtdev >> $seqres.full 2>&1 + [ -n "$rt_loop_dev" ] && _destroy_loop_device $rt_loop_dev rm -f $tmp.* $TEST_DIR/$seq.rtvol }@@ -52,12 +52,12 @@ fi rtdevsz=$((nr_bits * rtextsz)) truncate -s $rtdevsz $TEST_DIR/$seq.rtvol -rtdev=$(_create_loop_device $TEST_DIR/$seq.rtvol) +rt_loop_dev=$(_create_loop_device $TEST_DIR/$seq.rtvol) echo "Format and mount rt volume" export USE_EXTERNAL=yes -export SCRATCH_RTDEV=$rtdev +export SCRATCH_RTDEV=$rt_loop_dev _scratch_mkfs -d size=$((1024 * 1024 * 1024)) \ -r size=${rtextsz},extsize=${rtextsz} >> $seqres.full _try_scratch_mount || _notrun "Couldn't mount fs with synthetic rt volume"@@ -116,8 +116,9 @@ done echo "Check filesystem" _check_scratch_fs -losetup -d $rtdev -rm -f $TEST_DIR/$seq.rtvol +_scratch_unmount +_destroy_loop_device $rt_loop_dev +unset rt_loop_dev # success, all done status=0tests/xfs/606 +8
-4@@ -13,10 +13,8 @@ _begin_fstest auto quick growfs _cleanup() { - local dev _unmount $LOOP_MNT 2>/dev/null - dev=$(losetup -j testfile | cut -d: -f1) - losetup -d $dev 2>/dev/null + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev rm -rf $LOOP_IMG $LOOP_MNT cd / rm -f $tmp.*@@ -40,7 +38,9 @@ $MKFS_XFS_PROG -f $LOOP_IMG >$seqres.full # Extend by just 8K, expected to start with the last full-size AG ends of # above 1G block device. $XFS_IO_PROG -f -c "truncate 1073750016" $LOOP_IMG -_mount -oloop $LOOP_IMG $LOOP_MNT + +loop_dev=$(_create_loop_device $LOOP_IMG) +_mount $loop_dev $LOOP_MNT # A known bug shows "XFS_IOC_FSGROWFSDATA xfsctl failed: No space left on # device" at here, refer to _fixed_by_kernel_commit above $XFS_GROWFS_PROG $LOOP_MNT >$seqres.full@@ -48,6 +48,10 @@ if [ $? -ne 0 ];then echo "xfs_growfs fails!" fi +_unmount $LOOP_MNT +_destroy_loop_device $loop_dev +unset loop_dev + echo "Silence is golden" # success, all done status=0tests/xfs/613 +14
-26@@ -15,14 +15,8 @@ _cleanup() cd / rm -f $tmp.* _unmount $LOOP_MNT 2>/dev/null - if [ -n "$LOOP_DEV" ];then - _destroy_loop_device $LOOP_DEV 2>/dev/null - fi - if [ -n "$LOOP_SPARE_DEV" ];then - _destroy_loop_device $LOOP_SPARE_DEV 2>/dev/null - fi + [ -n $loop_dev ] &&_destroy_loop_device $loop_dev rm -f $LOOP_IMG - rm -f $LOOP_SPARE_IMG rmdir $LOOP_MNT }@@ -38,16 +32,11 @@ _require_loop _require_xfs_io_command "falloc" LOOP_IMG=$TEST_DIR/$seq.dev -LOOP_SPARE_IMG=$TEST_DIR/$seq.logdev LOOP_MNT=$TEST_DIR/$seq.mnt echo "** create loop device" $XFS_IO_PROG -f -c "truncate 32g" $LOOP_IMG -LOOP_DEV=`_create_loop_device $LOOP_IMG` - -echo "** create loop log device" -$XFS_IO_PROG -f -c "truncate 1g" $LOOP_SPARE_IMG -LOOP_SPARE_DEV=`_create_loop_device $LOOP_SPARE_IMG` +loop_dev=`_create_loop_device $LOOP_IMG` echo "** create loop mount point" rmdir $LOOP_MNT 2>/dev/null@@ -56,8 +45,7 @@ mkdir -p $LOOP_MNT || _fail "cannot create loopback mount point" filter_loop() { sed -e "s,\B$LOOP_MNT,LOOP_MNT,g" \ - -e "s,\B$LOOP_DEV,LOOP_DEV,g" \ - -e "s,\B$LOOP_SPARE_DEV,LOOP_SPARE_DEV,g" + -e "s,\B$loop_dev,LOOP_DEV,g" } filter_xfs_opt()@@ -70,22 +58,22 @@ MKFS_OPTIONS="" do_mkfs() { echo "FORMAT: $@" | filter_loop | tee -a $seqres.full - $MKFS_XFS_PROG -f $* $LOOP_DEV | _filter_mkfs >>$seqres.full 2>$tmp.mkfs + $MKFS_XFS_PROG -f $* $loop_dev | _filter_mkfs >>$seqres.full 2>$tmp.mkfs if [ "${PIPESTATUS[0]}" -ne 0 ]; then - _fail "Fails on _mkfs_dev $* $LOOP_DEV" + _fail "Fails on _mkfs_dev $* $loop_dev" fi . $tmp.mkfs } is_dev_mounted() { - findmnt --source $LOOP_DEV >/dev/null + findmnt --source $loop_dev >/dev/null return $? } get_mount_info() { - findmnt --source $LOOP_DEV -o OPTIONS -n + findmnt --source $loop_dev -o OPTIONS -n } force_unmount()@@ -104,29 +92,29 @@ _do_test() local info # mount test - _mount $LOOP_DEV $LOOP_MNT $opts 2>>$seqres.full + _mount $loop_dev $LOOP_MNT $opts 2>>$seqres.full rc=$? if [ $rc -eq 0 ];then if [ "${mounted}" = "fail" ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: expect mount to fail, but it succeeded" return 1 fi is_dev_mounted if [ $? -ne 0 ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: fs not mounted even mount return 0" return 1 fi else if [ "${mounted}" = "pass" ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: expect mount to succeed, but it failed" return 1 fi is_dev_mounted if [ $? -eq 0 ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: fs is mounted even mount return non-zero" return 1 fi@@ -142,13 +130,13 @@ _do_test() rc=$? if [ $rc -eq 0 ];then if [ "$found" != "true" ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: expected to find \"$key\" in mount info \"$info\"" return 1 fi else if [ "$found" != "false" ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: did not expect to find \"$key\" in \"$info\"" return 1 fitests/xfs/613.out +0
-1@@ -1,6 +1,5 @@ QA output created by 613 ** create loop device -** create loop log device ** create loop mount point ** start xfs mount testing ... FORMAT: -m crc=0Signature · 1 line
2.45.2
Re: [PATCH 13/40] fstests: clean up loop device instantiation
In reply to Dave ChinnerQuoted 70 lines · Dave Chinner
From: Dave Chinner <[email protected]>
Lots of tests do there own special thing with loop devices rather than using
_create_loop_device()and_destroy_loop_device(). This oftens means they do not clean up after themselves properly, leaving stale loop devices around that result in unmountable test or scratch devices. This is common when tests are killed by user interrupt.Even the tests that do use
_destroy_loop_deviceand try to clean up often do it incorrectly, leading to spurious error messages.Some tests try to use dynamic instantiation via "mount -o loop", but then don't clean up in the correct order or hack around to find the loop device that was instantiated because the test needs to know the instantiated device name
Clean this up by converting all the tests to use
_create_loop_device()and_destroy_loop_device(). In all the tests, use the variable "loop_dev" for the device consistently. In_destroy_loop_device(), test that a device name has been passed so that we don't try to clean up the same device twice (e.g. once before test exit and again from the_cleanup()function). When we destroy a loop device, unset the variable used to hold the loop device name so that we don't try to destroy it twice.This results in much more reliable cleanup and clean exit from fstests when killed by the user.
Signed-off-by Dave Chinner <[email protected]>--- common/metadump | 32 ++++++++++++-------------------- tests/generic/067 | 11 ++++++++--- tests/generic/361 | 6 +++++- tests/generic/563 | 14 +++++++++----- tests/generic/564 | 12 ++++++------ tests/generic/590 | 9 ++++++--- tests/generic/744 | 4 ++-- tests/generic/746 | 6 +++++- tests/xfs/014 | 7 +++++-- tests/xfs/049 | 35 +++++++++++++++++++++-------------- tests/xfs/073 | 24 +++++++++++++----------- tests/xfs/074 | 21 ++++++++++++--------- tests/xfs/078 | 16 +++++++++------- tests/xfs/148 | 23 ++++++++++++++--------- tests/xfs/149 | 4 ++++ tests/xfs/216 | 15 ++++++++++----- tests/xfs/217 | 22 ++++++++++++++++------ tests/xfs/250 | 14 +++++++++----- tests/xfs/259 | 13 ++++++++----- tests/xfs/513 | 44 ++++++++++++++++++++------------------------ tests/xfs/521 | 8 ++++++-- tests/xfs/528 | 10 +++++++--- tests/xfs/530 | 11 ++++++----- tests/xfs/606 | 12 ++++++++---- tests/xfs/613 | 40 ++++++++++++++-------------------------- tests/xfs/613.out | 1 - 26 files changed, 235 insertions(+), 179 deletions(-)common/metadump +0
-3@@ -24,17 +24,9 @@ _xfs_cleanup_verify_metadump() test -n "$XFS_METADUMP_FILE" && rm -f "$XFS_METADUMP_FILE" - if [ -n "$XFS_METADUMP_IMG" ]; then - losetup -n -a -O BACK-FILE,NAME | grep "^$XFS_METADUMP_IMG" | while read backing ldev; do - losetup -d "$ldev"
common/metadump _xfs_cleanup_verify_metadump() · line 28
Quoted 10 lines
common/metadump +2
-8- done - - # Don't call rm directly with a globbed argument here to avoid - # issues issues with variable expansions. - for img in "$XFS_METADUMP_IMG"*; do - test -e "$img" && rm -f "$img" - done - fi + [ -n "$md_data_loop_dev" ] && _destroy_loop_device $md_data_loop_dev + [ -n "$md_log_loop_dev" ] && _destroy_loop_device $md_log_loop_dev
As Darrick reported that he always hit below error:
losetup: /dev/loop0: detach failed: No such device or address Cannot destroy loop device /dev/loop0
I'm wondering we should use [ -b ] not [ -n ] at here? Or output the stderr to /dev/null directly. Due to... (see below)
Quoted 16 lines
common/metadump +3
-2+ rm -f $data_img $log_img } # Can xfs_metadump snapshot the fs metadata to a v1 metadump file?@@ -122,25 +114,25 @@ _xfs_verify_metadump_v2() _scratch_xfs_mdrestore $metadump_file # Create loopdev for data device so we can mount the fs - data_loop=$(_create_loop_device $data_img) + md_data_loop_dev=$(_create_loop_device $data_img) # Create loopdev for log device if we recovered anything - test -s "$log_img" && log_loop=$(_create_loop_device $log_img) + test -s "$log_img" && md_log_loop_dev=$(_create_loop_device $log_img) # Mount fs, run an extra test, fsck, and unmount
common/metadump _xfs_verify_metadump_v2() · line 128
Quoted 10 lines
common/metadump +4
-4- SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop _scratch_mount + SCRATCH_DEV=$md_data_loop_dev SCRATCH_LOGDEV=$md_log_loop_dev _scratch_mount if [ -n "$extra_test" ]; then - SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop $extra_test + SCRATCH_DEV=$md_data_loop_dev SCRATCH_LOGDEV=$md_log_loop_dev $extra_test fi - SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop _check_xfs_scratch_fs - SCRATCH_DEV=$data_loop _scratch_unmount + SCRATCH_DEV=$md_data_loop_dev SCRATCH_LOGDEV=$md_log_loop_dev _check_xfs_scratch_fs + _unmount $md_data_loop_dev
We umount loop devices we created above at here, and ...
common/metadump _xfs_verify_metadump_v2() · line 135
Quoted 10 lines
common/metadump +3
-3# Tear down what we created - if [ -b "$log_loop" ]; then - _destroy_loop_device $log_loop + if [ -b "$md_log_loop_dev" ]; then + _destroy_loop_device $md_log_loop_dev rm -f $log_img fi - _destroy_loop_device $data_loop + _destroy_loop_device $md_data_loop_dev
we even has destroied them at here.
So that _xfs_cleanup_verify_metadump function hit "detach failed: No such device or address", due to those loop devices might have been released, but the variables are not unset.
I didn't give it a test, just did a code review. As Darrick uses this part mostly, could you please check what I said above?
Thanks,
Zorro
Quoted 1,198 lines
common/metadump +0
-0rm -f $data_img }tests/generic/067 +8
-3@@ -37,13 +37,18 @@ mount_nonexistent_mnt() $MOUNT_PROG $SCRATCH_DEV $TEST_DIR/nosuchdir >>$seqres.full 2>&1 } -# fs driver should be able to handle mounting a free loop device gracefully -# xfs ever hung, "ec53d1d xfs: don't block on buffer read errors" fixed it +# fs driver should be able to handle mounting a free loop device gracefully xfs +# ever hung, "ec53d1d xfs: don't block on buffer read errors" fixed it +# +# Using 'losetup -f' like this is racy. We might end up mounting a real loop dev +# here, so unmount $SCRATCH_MNT (not the loop dev we might not own!) and ignore +# any error it might return. mount_free_loopdev() { echo "# mount a free loop device" >>$seqres.full loopdev=`losetup -f` - $MOUNT_PROG -t $FSTYP $loopdev $SCRATCH_MNT >>$seqres.full 2>&1 + _mount $loopdev $SCRATCH_MNT >>$seqres.full 2>&1 + _unmount $SCRATCH_MNT >> /dev/null 2>&1 } # mount with wrong fs type specified.tests/generic/361 +5
-1@@ -17,7 +17,7 @@ _begin_fstest auto quick _cleanup() { _unmount $fs_mnt - _destroy_loop_device $loop_dev + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev cd / rm -f $tmp.* }@@ -54,6 +54,10 @@ $XFS_IO_PROG -fc "pwrite 0 520m" $fs_mnt/testfile >>$seqres.full 2>&1 # remount should not hang $MOUNT_PROG -o remount,ro $fs_mnt >>$seqres.full 2>&1 +_unmount $fs_mnt +_destroy_loop_device $loop_dev +unset loop_dev + # success, all done echo "Silence is golden" status=0tests/generic/563 +9
-5@@ -87,12 +87,12 @@ reset() # cgroup I/O accounting doesn't work on partitions. Use a loop device to rule # that out. -LOOP_DEV=$(_create_loop_device $SCRATCH_DEV) -smajor=$((0x`stat -L -c %t $LOOP_DEV`)) -sminor=$((0x`stat -L -c %T $LOOP_DEV`)) +loop_dev=$(_create_loop_device $SCRATCH_DEV) +smajor=$((0x`stat -L -c %t $loop_dev`)) +sminor=$((0x`stat -L -c %T $loop_dev`)) -_mkfs_dev $LOOP_DEV >> $seqres.full 2>&1 -_mount $LOOP_DEV $SCRATCH_MNT || _fail "mount failed" +_mkfs_dev $loop_dev >> $seqres.full 2>&1 +_mount $loop_dev $SCRATCH_MNT || _fail "mount failed" blksize=$(_get_block_size "$SCRATCH_MNT")@@ -147,6 +147,10 @@ if [ "$drop_io_cgroup" = 1 ]; then echo "-io" > $cgdir/cgroup.subtree_control || _fail "subtree control" fi +_unmount $SCRATCH_MNT +_destroy_loop_device $loop_dev +unset loop_dev + # success, all done status=0 exittests/generic/564 +6
-6@@ -19,7 +19,7 @@ _cleanup() { cd / rm -rf $tmp.* - [ -z "$loopdev" ] || _destroy_loop_device $loopdev + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev } # Import common functions.@@ -72,11 +72,11 @@ $XFS_IO_PROG -f -c "copy_range -l 32k $testdir" $testdir/copy echo echo source/destination as blkdev returns EINVAL $XFS_IO_PROG -f -c "truncate 128k" $testdir/img >> $seqres.full 2>&1 -loopdev=`_create_loop_device $testdir/img` -$XFS_IO_PROG -c "copy_range -l 32k $testdir/file" $loopdev -$XFS_IO_PROG -f -c "copy_range -l 32k $loopdev" $testdir/copy -_destroy_loop_device $loopdev -loopdev= +loop_dev=`_create_loop_device $testdir/img` +$XFS_IO_PROG -c "copy_range -l 32k $testdir/file" $loop_dev +$XFS_IO_PROG -f -c "copy_range -l 32k $loop_dev" $testdir/copy +_destroy_loop_device $loop_dev +unset loop_dev echo echo source/destination as chardev returns EINVALtests/generic/590 +6
-3@@ -15,9 +15,10 @@ _begin_fstest auto prealloc preallocrw # Override the default cleanup function. _cleanup() { + _scratch_unmount + [ -n $loop_dev ] && _destroy_loop_device $loop_dev cd / rm -f $tmp.* - test -n "$loop" && _destroy_loop_device "$loop" rm -f "$TEST_DIR/$seq" }@@ -56,9 +57,9 @@ if [[ $FSTYP = xfs ]]; then loopsz="$((filesz + (1 << 26)))" _require_fs_space "$TEST_DIR" $((loopsz / 1024)) $XFS_IO_PROG -c "truncate $loopsz" -f "$TEST_DIR/$seq" - loop="$(_create_loop_device "$TEST_DIR/$seq")" + loop_dev="$(_create_loop_device "$TEST_DIR/$seq")" USE_EXTERNAL=yes - SCRATCH_RTDEV="$loop" + SCRATCH_RTDEV="$loop_dev" disabled_features=() # disable reflink if not supported by realtime devices@@ -114,6 +115,8 @@ $XFS_IO_PROG -c "truncate 0" -c fsync "$SCRATCH_MNT/file" # We need to do this before the loop device gets torn down. _scratch_unmount _check_scratch_fs +_destroy_loop_device $loop_dev +unset loop_dev echo "Silence is golden" status=0tests/generic/744 +2
-2@@ -17,8 +17,8 @@ _cleanup() _unmount $mnt2 &> /dev/null _unmount $mnt1 &> /dev/null - [ -b "$loop_dev2" ] && losetup -d $loop_dev2 - [ -b "$loop_dev1" ] && losetup -d $loop_dev1 + [ -b "$loop_dev2" ] && _destroy_loop_device $loop_dev2 + [ -b "$loop_dev1" ] && _destroy_loop_device $loop_dev1 [ -n "$seq" ] && rm -rf $TEST_DIR/$seq }tests/generic/746 +5
-1@@ -39,7 +39,7 @@ esac _cleanup() { _unmount $loop_mnt &> /dev/null - _destroy_loop_device $loop_dev + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev if [ $status -eq 0 ]; then rm -rf $tmp rm $img_file@@ -223,5 +223,9 @@ while read line; do done < $fiemap_after echo "done." +_unmount $loop_mnt +_destroy_loop_device $loop_dev +unset loop_dev + status=0 exittests/xfs/014 +5
-2@@ -23,6 +23,7 @@ _cleanup() { cd / _unmount $LOOP_MNT 2>/dev/null + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev _scratch_unmount 2>/dev/null rm -f $tmp.* }@@ -166,19 +167,21 @@ LOOP_FILE=$SCRATCH_MNT/$seq.fs LOOP_MNT=$SCRATCH_MNT/$seq.mnt $MKFS_XFS_PROG -d "file=1,name=$LOOP_FILE,size=10g" >> $seqres.full 2>&1 +loop_dev=$(_create_loop_device $LOOP_FILE) mkdir -p $LOOP_MNT -_mount -o uquota,gquota $LOOP_FILE $LOOP_MNT || \ +_mount -o uquota,gquota $loop_dev $LOOP_MNT || \ _fail "Failed to mount loop fs." _test_enospc $LOOP_MNT _test_edquot $LOOP_MNT _unmount $LOOP_MNT +_destroy_loop_device $loop_dev +unset loop_dev echo $orig_sp_time > /proc/sys/fs/xfs/speculative_prealloc_lifetime _scratch_unmount - status=0 exittests/xfs/049 +21
-14@@ -12,16 +12,17 @@ _begin_fstest rw auto quick # Override the default cleanup function. _cleanup() { - cd / - umount $SCRATCH_MNT/test2 > /dev/null 2>&1 - umount $SCRATCH_MNT/test > /dev/null 2>&1 - rm -f $tmp.* + cd / + _unmount $SCRATCH_MNT/test2 > /dev/null 2>&1 + _unmount $SCRATCH_MNT/test > /dev/null 2>&1 + [ -n "$loop_dev2" ] && _destroy_loop_device $loop_dev2 + [ -n "$loop_dev1" ] && _destroy_loop_device $loop_dev1 + rm -f $tmp.* - if [ -w $seqres.full ] - then - echo "--- mounts at end (after cleanup)" >> $seqres.full - mount >> $seqres.full - fi + if [ -w $seqres.full ]; then + echo "--- mounts at end (after cleanup)" >> $seqres.full + mount >> $seqres.full + fi } # Import common functions.@@ -64,7 +65,8 @@ mkdir $SCRATCH_MNT/test $SCRATCH_MNT/test2 >> $seqres.full 2>&1 \ || _fail "!!! failed to make mount points" _log "Mount xfs via loop" -mount -t xfs -o loop $SCRATCH_MNT/test.xfs $SCRATCH_MNT/test >> $seqres.full 2>&1 \ +loop_dev1=$(_create_loop_device $SCRATCH_MNT/test.xfs) +_mount $loop_dev1 $SCRATCH_MNT/test >> $seqres.full 2>&1 \ || _fail "!!! failed to loop mount xfs" _log "stress"@@ -80,11 +82,12 @@ dd if=/dev/zero of=$SCRATCH_MNT/test/test.ext2 bs=1024 count=10240 >> $seqres.fu || _fail "!!! create file failed" _log "Create ext2 fs in file on looped xfs" -echo y | mkfs -t ext2 $SCRATCH_MNT/test/test.ext2 >> $seqres.full 2>&1 \ +loop_dev2=$(_create_loop_device $SCRATCH_MNT/test/test.ext2) +echo y | mkfs -t ext2 $loop_dev2 >> $seqres.full 2>&1 \ || _fail "!!! failed to mkfs ext2 on xfs" _log "Mount ext2 on xfs via loop" -mount -t ext2 -o loop $SCRATCH_MNT/test/test.ext2 $SCRATCH_MNT/test2 >> $seqres.full 2>&1 \ +_mount $loop_dev2 $SCRATCH_MNT/test2 >> $seqres.full 2>&1 \ || _fail "!!! failed to loop mount xfs" _log "stress ext2 on xfs via loop"@@ -96,12 +99,16 @@ rm -rf $SCRATCH_MNT/test/* >> $seqres.full 2>&1 \ || _fail "!!! clean failed" _log "umount ext2 on xfs" -umount $SCRATCH_MNT/test2 >> $seqres.full 2>&1 \ +_unmount $SCRATCH_MNT/test2 >> $seqres.full 2>&1 \ || _fail "!!! umount ext2 failed" +_destroy_loop_device $loop_dev2 +unset loop_dev2 _log "umount xfs" -umount $SCRATCH_MNT/test >> $seqres.full 2>&1 \ +_unmount $SCRATCH_MNT/test >> $seqres.full 2>&1 \ || _fail "!!! umount xfs failed" +_destroy_loop_device $loop_dev1 +unset loop_dev1 echo "--- mounts at end (before cleanup)" >> $seqres.full mount >> $seqres.fulltests/xfs/073 +13
-11@@ -23,6 +23,8 @@ _cleanup() _scratch_unmount 2>/dev/null _unmount $imgs.loop 2>/dev/null _unmount $imgs.source_dir 2>/dev/null + [ -n "$loop_dev1" ] && _destroy_loop_device $loop_dev1 + [ -n "$loop_dev2" ] && _destroy_loop_device $loop_dev2 [ -d $imgs.loop ] && rmdir $imgs.loop [ -d $imgs.source_dir ] && rm -rf $imgs.source_dir rm -f $imgs.* $tmp.* /var/tmp/xfs_copy.log.*@@ -65,10 +67,11 @@ _verify_copy() rmdir $target_dir 2>/dev/null mkdir $target_dir - _mount -t xfs -o loop $target $target_dir 2>/dev/null + loop_dev1=$(_create_loop_device $target) + _mount $loop_dev1 $target_dir 2>/dev/null if [ $? -ne 0 ]; then echo retrying mount with nouuid option >>$seqres.full - _mount -t xfs -o loop -o nouuid $target $target_dir + _mount -o nouuid $loop_dev1 $target_dir if [ $? -ne 0 ]; then echo mount failed - evil! return@@ -100,6 +103,8 @@ _verify_copy() echo unmounting and removing new image _unmount $source_dir _unmount $target_dir > /dev/null 2>&1 + _destroy_loop_device $loop_dev1 + unset loop_dev1 rm -f $target }@@ -134,18 +139,15 @@ ${MKFS_XFS_PROG} -dfile,name=$imgs.source,size=100g \ rmdir $imgs.source_dir 2>/dev/null mkdir $imgs.source_dir -_mount -t xfs -o loop $imgs.source $imgs.source_dir -loop2=`mount | grep $imgs.source | grep -o -e 'loop=.*[^),]' | grep -o -e '/.*$'` +loop_dev2=$(_create_loop_device $imgs.source) +_mount $loop_dev2 $imgs.source_dir cp -a $here $imgs.source_dir -_mount -t xfs -o remount,ro $imgs.source $imgs.source_dir -$XFS_COPY_PROG $imgs.source $imgs.image | _filter_copy '#' $imgs.image '#' '#' +_mount -o remount,ro $loop_dev2 $imgs.source_dir +$XFS_COPY_PROG $loop_dev2 $imgs.image 2> /dev/null | _filter_copy '#' $imgs.image '#' '#' _verify_copy $imgs.image $imgs.source $imgs.source_dir -# HACK WARNING: -# -# We're done with the nested loop mount, now we have to clean -# up the pieces that mount is incapable of doing. -losetup -d $loop2 > /dev/null 2>&1 +_destroy_loop_device $loop_dev2 +unset loop_dev2 echo echo === copying scratch device to multiple targetstests/xfs/074 +12
-9@@ -26,7 +26,7 @@ _begin_fstest quick auto prealloc rw _cleanup() { cd / - _destroy_loop_device $LOOP_DEV + [ -n "$loop_dev" ] &&_destroy_loop_device $loop_dev rm -f $tmp.* $LOOP_FILE }@@ -45,10 +45,10 @@ LOOP_FILE=$TEST_DIR/$seq.img LOOP_MNT=$TEST_DIR/$seq.mnt mkdir -p $LOOP_MNT $XFS_IO_PROG -ft -c "truncate 1t" $LOOP_FILE >> $seqres.full -LOOP_DEV=`_create_loop_device $LOOP_FILE` +loop_dev=`_create_loop_device $LOOP_FILE` -_mkfs_dev -d size=260g,agcount=2 $LOOP_DEV -_mount $LOOP_DEV $LOOP_MNT +_mkfs_dev -d size=260g,agcount=2 $loop_dev +_mount $loop_dev $LOOP_MNT BLOCK_SIZE=$(_get_file_block_size $LOOP_MNT)@@ -59,11 +59,11 @@ $XFS_IO_PROG -ft \ -c "falloc 0 $(($BLOCK_SIZE * 2097152))" \ $LOOP_MNT/foo >> $seqres.full -umount $LOOP_MNT -_check_xfs_filesystem $LOOP_DEV none none +_unmount $LOOP_MNT +_check_xfs_filesystem $loop_dev none none -_mkfs_dev -f $LOOP_DEV -_mount $LOOP_DEV $LOOP_MNT +_mkfs_dev -f $loop_dev +_mount $loop_dev $LOOP_MNT # check we trim both ends of the extent approproiately; this will fail # on 1k block size filesystems without the correct fixes in place.@@ -73,7 +73,10 @@ $XFS_IO_PROG -ft \ $LOOP_MNT/foo >> $seqres.full _unmount $LOOP_MNT -_check_xfs_filesystem $LOOP_DEV none none +_check_xfs_filesystem $loop_dev none none + +_destroy_loop_device $loop_dev +unset loop_dev # success, all done echo "Silence is golden"tests/xfs/078 +9
-7@@ -17,7 +17,7 @@ _cleanup() cd / rm -f $tmp.* _unmount $LOOP_MNT 2>/dev/null - [ -n "$LOOP_DEV" ] && _destroy_loop_device $LOOP_DEV 2>/dev/null + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev 2>/dev/null # try to keep the image file if test fails [ $status -eq 0 ] && rm -f $LOOP_IMG rmdir $LOOP_MNT@@ -55,7 +55,7 @@ _grow_loop() agsize=$5 $XFS_IO_PROG -f -c "truncate $original" $LOOP_IMG - LOOP_DEV=`_create_loop_device $LOOP_IMG` + loop_dev=`_create_loop_device $LOOP_IMG` dparam="" if [ -n "$agsize" ]; then@@ -67,15 +67,15 @@ _grow_loop() echo echo "*** mkfs loop file (size=$original)" - $MKFS_XFS_PROG -b size=$bsize $dparam $LOOP_DEV | \ + $MKFS_XFS_PROG -b size=$bsize $dparam $loop_dev | \ _filter_mkfs 2>/dev/null echo "*** extend loop file" - _destroy_loop_device $LOOP_DEV + _destroy_loop_device $loop_dev $XFS_IO_PROG -c "pwrite $new_size $bsize" $LOOP_IMG | _filter_io - LOOP_DEV=`_create_loop_device $LOOP_IMG` + loop_dev=`_create_loop_device $LOOP_IMG` echo "*** mount loop filesystem" - _mount -t xfs $LOOP_DEV $LOOP_MNT + _mount $loop_dev $LOOP_MNT echo "*** grow loop filesystem" $XFS_GROWFS_PROG $LOOP_MNT 2>&1 | _filter_growfs 2>&1@@ -87,9 +87,11 @@ _grow_loop() if [ "$check" -gt "0" ] then echo "*** check" - _check_xfs_filesystem $LOOP_IMG none none + _check_xfs_filesystem $loop_dev none none fi + _destroy_loop_device $loop_dev + unset loop_dev rm -f $LOOP_IMG }tests/xfs/148 +14
-9@@ -15,7 +15,7 @@ _cleanup() { cd / _unmount $mntpt > /dev/null 2>&1 - _destroy_loop_device $loopdev > /dev/null 2>&1 + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev rm -r -f $tmp.* }@@ -48,12 +48,12 @@ rm -f $imgfile $imgfile.old # even when security xattrs are present so we are always doing name matches on # lookup and not name hash compares as leaf/node forms will do. $XFS_IO_PROG -f -c 'truncate 40m' $imgfile -loopdev=$(_create_loop_device $imgfile) -MKFS_OPTIONS="-m crc=0 -i size=512" _mkfs_dev $loopdev >> $seqres.full +loop_dev=$(_create_loop_device $imgfile) +MKFS_OPTIONS="-m crc=0 -i size=512" _mkfs_dev $loop_dev >> $seqres.full # Mount image file mkdir -p $mntpt -_mount $loopdev $mntpt +_mount $loop_dev $mntpt echo "creating entries" >> $seqres.full@@ -91,7 +91,8 @@ cat $tmp.log | _filter_test_dir # Corrupt the entries _unmount $mntpt -_destroy_loop_device $loopdev +_destroy_loop_device $loop_dev +unset loop_dev cp $imgfile $imgfile.old sed -b \ -e "s/$nullstr/too_many\x00beans/g" \@@ -99,8 +100,9 @@ sed -b \ -i $imgfile test "$(md5sum < $imgfile)" != "$(md5sum < $imgfile.old)" || _fail "sed failed to change the image file?" -loopdev=$(_create_loop_device $imgfile) -_mount $loopdev $mntpt + +loop_dev=$(_create_loop_device $imgfile) +_mount $loop_dev $mntpt # Try to access the corrupt metadata echo "++ ACCESSING BAD METADATA" | tee -a $seqres.full@@ -111,7 +113,7 @@ cat $tmp.log | _filter_test_dir | sed -e '/Could not list/d' echo "does scrub complain?" >> $seqres.full # Does scrub complain about this? -if _supports_xfs_scrub $mntpt $loopdev; then +if _supports_xfs_scrub $mntpt $loop_dev; then $XFS_SCRUB_PROG -n $mntpt >> $seqres.full 2>&1 res=$? test $((res & 1)) -eq 0 && \@@ -122,11 +124,14 @@ echo "does repair complain?" >> $seqres.full # Does repair complain about this? _unmount $mntpt -$XFS_REPAIR_PROG -n $loopdev >> $seqres.full 2>&1 +$XFS_REPAIR_PROG -n $loop_dev >> $seqres.full 2>&1 res=$? test $res -eq 1 || \ echo "repair failed to report corruption ($res)" +_destroy_loop_device $loop_dev +unset loop_dev + # success, all done status=0 exittests/xfs/149 +4
-0@@ -84,6 +84,10 @@ $XFS_GROWFS_PROG -D 16384 $loop_symlink > /dev/null echo "=== xfs_growfs - check device node ===" $XFS_GROWFS_PROG -D 20480 $loop_dev > /dev/null +_unmount $mntdir +_destroy_loop_device $loop_dev +unset loop_dev + # success, all done status=0 exittests/xfs/216 +10
-5@@ -15,6 +15,7 @@ _begin_fstest log metadata auto quick _cleanup() { _unmount $LOOP_MNT > /dev/null 2>&1 + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev cd / rm -f $tmp.* }@@ -24,7 +25,7 @@ _scratch_mkfs_xfs >/dev/null 2>&1 _scratch_mount _require_loop -LOOP_DEV=$SCRATCH_MNT/test_fs +LOOP_IMG=$SCRATCH_MNT/test_fs LOOP_MNT=$SCRATCH_MNT/test_fs_dir loop_mkfs_opts=@@ -55,22 +56,26 @@ _do_mkfs() for i in $*; do echo -n "fssize=${i}g " $MKFS_XFS_PROG -f -b size=4096 -l version=2 \ - -d name=$LOOP_DEV,size=${i}g $loop_mkfs_opts |grep log - _mount -o loop -t xfs $LOOP_DEV $LOOP_MNT + -d size=${i}g $loop_mkfs_opts $loop_dev |grep log + _mount $loop_dev $LOOP_MNT echo "test write" > $LOOP_MNT/test _unmount $LOOP_MNT > /dev/null 2>&1 done } # make large holey file -$XFS_IO_PROG -f -c "truncate 256g" $LOOP_DEV +$XFS_IO_PROG -f -c "truncate 256g" $LOOP_IMG -choose_golden_output $0 $LOOP_DEV +choose_golden_output $0 $LOOP_IMG #make loopback mount dir mkdir $LOOP_MNT +loop_dev=$(_create_loop_device $LOOP_IMG) + # walk over standard sizes (up to 256GB) _do_mkfs 1 2 4 8 16 32 64 128 256 +_destroy_loop_device $loop_dev +unset loop_dev status=0 exittests/xfs/217 +16
-6@@ -12,6 +12,12 @@ _begin_fstest log metadata auto # Import common functions. . ./common/filter +_cleanup() +{ + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev + cd / + rm -f $tmp.* +} _require_scratch _scratch_mkfs_xfs >/dev/null 2>&1@@ -20,7 +26,7 @@ _scratch_mount _require_fs_space $SCRATCH_MNT 2202000 _require_loop -LOOP_DEV=$SCRATCH_MNT/test_fs +LOOP_IMG=$SCRATCH_MNT/test_fs LOOP_MNT=$SCRATCH_MNT/test_fs_dir _do_mkfs()@@ -28,28 +34,30 @@ _do_mkfs() for i in $*; do echo -n "fssize=${i}g " $MKFS_XFS_PROG -f -b size=4096 -l version=2 \ - -d name=$LOOP_DEV,size=${i}g |grep log - _mount -o loop -t xfs $LOOP_DEV $LOOP_MNT + -d size=${i}g $loop_dev |grep log + _mount $loop_dev $LOOP_MNT echo "test write" > $LOOP_MNT/test _unmount $LOOP_MNT > /dev/null 2>&1 # punch out the previous blocks so that we keep the amount of # disk space the test requires down to a minimum. - $XFS_IO_PROG -f -c "unresvsp 0 16383g" $LOOP_DEV + $XFS_IO_PROG -f -c "unresvsp 0 16383g" $LOOP_IMG done } # make large holey file -$XFS_IO_PROG -f -c "truncate 16383g" $LOOP_DEV +$XFS_IO_PROG -f -c "truncate 16383g" $LOOP_IMG #make loopback mount dir mkdir $LOOP_MNT # test if large logs are supported -$MKFS_XFS_PROG -f -l size=256m -d name=$LOOP_DEV,size=10g > /dev/null 2>&1 +$MKFS_XFS_PROG -f -l size=256m -d name=$LOOP_IMG,size=10g > /dev/null 2>&1 if [ $? -ne 0 ]; then _notrun "large log sizes not supported by mkfs" fi +loop_dev=$(_create_loop_device $LOOP_IMG) + # # walk over "new" sizes supported by recent xfsprogs. # Note that the last test is for 16TB-1GB as 32bit platforms only support@@ -57,5 +65,7 @@ fi # _do_mkfs 512 1024 2048 4096 8192 16383 +_destroy_loop_device $loop_dev +unset loop_dev status=0 exittests/xfs/250 +9
-5@@ -14,7 +14,8 @@ _cleanup() { cd / _unmount $LOOP_MNT 2>/dev/null - rm -f $LOOP_DEV + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev + rm -f $LOOP_IMG rmdir $LOOP_MNT rm -f $tmp.* }@@ -26,7 +27,7 @@ _require_test _require_loop _require_xfs_io_command "falloc" -LOOP_DEV=$TEST_DIR/$seq.fs +LOOP_IMG=$TEST_DIR/$seq.fs LOOP_MNT=$TEST_DIR/$seq.mnt _filter_io()@@ -45,7 +46,7 @@ _test_loop() agsize=$2 fsize=$3 - dparam="file,name=$LOOP_DEV,size=$size" + dparam="file,name=$LOOP_IMG,size=$size" if [ -n "$agsize" ]; then dparam="$dparam,agsize=$agsize" fi@@ -55,7 +56,8 @@ _test_loop() | _filter_mkfs 2>/dev/null echo "*** mount loop filesystem" - _mount -t xfs -o loop $LOOP_DEV $LOOP_MNT + loop_dev=$(_create_loop_device $LOOP_IMG) + mount $loop_dev $LOOP_MNT echo "*** preallocate large file" $XFS_IO_PROG -f -c "resvsp 0 $fsize" $LOOP_MNT/foo | _filter_io@@ -64,7 +66,9 @@ _test_loop() _unmount $LOOP_MNT > /dev/null 2>&1 echo "*** check loop filesystem" - _check_xfs_filesystem $LOOP_DEV none none + _check_xfs_filesystem $loop_dev none none + _destroy_loop_device $loop_dev + unset loop_dev } _test_loop 50g 16m 40Gtests/xfs/259 +8
-5@@ -12,7 +12,10 @@ _begin_fstest auto quick # Override the default cleanup function. _cleanup() { - rm -f "$testfile" + [ -n "$loop_dev" ] && _destroy_loop_device $testfile + rm -f "$testfile" + cd / + rm -f $tmp.* } # Import common functions.@@ -45,13 +48,13 @@ for del in $sizes_to_check; do rm -f "$testfile" dd if=/dev/zero "of=$testfile" bs=1 count=0 seek=$ddseek \ >/dev/null 2>&1 || echo "dd failed" - lofile=$(losetup -f) - losetup $lofile "$testfile" - $MKFS_XFS_PROG -l size=32m -b size=$bs $lofile | _filter_mkfs \ + loop_dev=$(_create_loop_device $testfile) + $MKFS_XFS_PROG -l size=32m -b size=$bs $loop_dev | _filter_mkfs \ >/dev/null 2> $tmp.mkfs || echo "mkfs failed!" . $tmp.mkfs sync - losetup -d $lofile + _destroy_loop_device $loop_dev + unset loop_dev done donetests/xfs/513 +20
-24@@ -15,12 +15,8 @@ _cleanup() cd / rm -f $tmp.* _unmount $LOOP_MNT 2>/dev/null - if [ -n "$LOOP_DEV" ];then - _destroy_loop_device $LOOP_DEV 2>/dev/null - fi - if [ -n "$LOOP_SPARE_DEV" ];then - _destroy_loop_device $LOOP_SPARE_DEV 2>/dev/null - fi + [ -n $loop_dev ] &&_destroy_loop_device $loop_dev + [ -n $loop_spare_dev ] &&_destroy_loop_device $loop_spare_dev rm -f $LOOP_IMG rm -f $LOOP_SPARE_IMG rmdir $LOOP_MNT@@ -42,11 +38,11 @@ LOOP_MNT=$TEST_DIR/$seq.mnt echo "** create loop device" $XFS_IO_PROG -f -c "truncate 32g" $LOOP_IMG -LOOP_DEV=`_create_loop_device $LOOP_IMG` +loop_dev=`_create_loop_device $LOOP_IMG` echo "** create loop log device" $XFS_IO_PROG -f -c "truncate 1g" $LOOP_SPARE_IMG -LOOP_SPARE_DEV=`_create_loop_device $LOOP_SPARE_IMG` +loop_spare_dev=`_create_loop_device $LOOP_SPARE_IMG` echo "** create loop mount point" rmdir $LOOP_MNT 2>/dev/null@@ -55,8 +51,8 @@ mkdir -p $LOOP_MNT || _fail "cannot create loopback mount point" filter_loop() { sed -e "s,\B$LOOP_MNT,LOOP_MNT,g" \ - -e "s,\B$LOOP_DEV,LOOP_DEV,g" \ - -e "s,\B$LOOP_SPARE_DEV,LOOP_SPARE_DEV,g" + -e "s,\B$loop_dev,LOOP_DEV,g" \ + -e "s,\B$loop_spare_dev,LOOP_SPARE_DEV,g" } filter_xfs_opt()@@ -69,22 +65,22 @@ MKFS_OPTIONS="" do_mkfs() { echo "FORMAT: $@" | filter_loop | tee -a $seqres.full - $MKFS_XFS_PROG -f $* $LOOP_DEV | _filter_mkfs >>$seqres.full 2>$tmp.mkfs + $MKFS_XFS_PROG -f $* $loop_dev | _filter_mkfs >>$seqres.full 2>$tmp.mkfs if [ "${PIPESTATUS[0]}" -ne 0 ]; then - _fail "Fails on _mkfs_dev $* $LOOP_DEV" + _fail "Fails on _mkfs_dev $* $loop_dev" fi . $tmp.mkfs } is_dev_mounted() { - findmnt --source $LOOP_DEV >/dev/null + findmnt --source $loop_dev >/dev/null return $? } get_mount_info() { - findmnt --source $LOOP_DEV -o OPTIONS -n + findmnt --source $loop_dev -o OPTIONS -n } force_unmount()@@ -103,29 +99,29 @@ _do_test() local info # mount test - _mount $LOOP_DEV $LOOP_MNT $opts 2>>$seqres.full + _mount $loop_dev $LOOP_MNT $opts 2>>$seqres.full rc=$? if [ $rc -eq 0 ];then if [ "${mounted}" = "fail" ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: expect mount to fail, but it succeeded" return 1 fi is_dev_mounted if [ $? -ne 0 ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: fs not mounted even mount return 0" return 1 fi else if [ "${mounted}" = "pass" ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: expect mount to succeed, but it failed" return 1 fi is_dev_mounted if [ $? -eq 0 ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: fs is mounted even mount return non-zero" return 1 fi@@ -141,13 +137,13 @@ _do_test() rc=$? if [ $rc -eq 0 ];then if [ "$found" != "true" ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: expected to find \"$key\" in mount info \"$info\"" return 1 fi else if [ "$found" != "false" ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: did not expect to find \"$key\" in \"$info\"" return 1 fi@@ -253,9 +249,9 @@ do_test "-o logbsize=512k" fail # Test logdev do_mkfs do_test "" pass "logdev" "false" -do_test "-o logdev=$LOOP_SPARE_DEV" fail -do_mkfs -l logdev=$LOOP_SPARE_DEV -do_test "-o logdev=$LOOP_SPARE_DEV" pass "logdev=$LOOP_SPARE_DEV" "true" +do_test "-o logdev=$loop_spare_dev" fail +do_mkfs -l logdev=$loop_spare_dev +do_test "-o logdev=$loop_spare_dev" pass "logdev=$loop_spare_dev" "true" do_test "" fail # Test noaligntests/xfs/521 +6
-2@@ -21,7 +21,7 @@ _cleanup() { cd / _scratch_unmount >> $seqres.full 2>&1 - test -e "$rtdev" && losetup -d $rtdev >> $seqres.full 2>&1 + [ -n "$rt_loop_dev" ] && _destroy_loop_device $rt_loop_dev rm -f $tmp.* $TEST_DIR/$seq.rtvol }@@ -35,7 +35,7 @@ _require_no_large_scratch_dev echo "Create fake rt volume" truncate -s 400m $TEST_DIR/$seq.rtvol -rtdev=$(_create_loop_device $TEST_DIR/$seq.rtvol) +rt_loop_dev=$(_create_loop_device $TEST_DIR/$seq.rtvol) echo "Format and mount 100m rt volume" export USE_EXTERNAL=yes@@ -69,6 +69,10 @@ cp -p $testdir/original $testdir/copy3 echo "Check filesystem" _check_scratch_fs +_scratch_unmount +_destroy_loop_device $rt_loop_dev +unset rt_loop_dev + # success, all done status=0 exittests/xfs/528 +7
-3@@ -15,7 +15,7 @@ _cleanup() { cd / _scratch_unmount >> $seqres.full 2>&1 - test -e "$rtdev" && losetup -d $rtdev >> $seqres.full 2>&1 + [ -n "$rt_loop_dev" ] && _destroy_loop_device $rt_loop_dev rm -f $tmp.* $TEST_DIR/$seq.rtvol }@@ -155,11 +155,11 @@ test_ops() { echo "Create fake rt volume" $XFS_IO_PROG -f -c "truncate 400m" $TEST_DIR/$seq.rtvol -rtdev=$(_create_loop_device $TEST_DIR/$seq.rtvol) +rt_loop_dev=$(_create_loop_device $TEST_DIR/$seq.rtvol) echo "Make sure synth rt volume works" export USE_EXTERNAL=yes -export SCRATCH_RTDEV=$rtdev +export SCRATCH_RTDEV=$rt_loop_dev _scratch_mkfs > $seqres.full _try_scratch_mount || \ _notrun "Could not mount with synthetic rt volume"@@ -170,6 +170,10 @@ test_ops 262144 # not a power of two test_ops 327680 +_scratch_unmount +_destroy_loop_device $rt_loop_dev +unset rt_loop_dev + # success, all done status=0 exittests/xfs/530 +6
-5@@ -15,7 +15,7 @@ _cleanup() { cd / _scratch_unmount >> $seqres.full 2>&1 - test -e "$rtdev" && losetup -d $rtdev >> $seqres.full 2>&1 + [ -n "$rt_loop_dev" ] && _destroy_loop_device $rt_loop_dev rm -f $tmp.* $TEST_DIR/$seq.rtvol }@@ -52,12 +52,12 @@ fi rtdevsz=$((nr_bits * rtextsz)) truncate -s $rtdevsz $TEST_DIR/$seq.rtvol -rtdev=$(_create_loop_device $TEST_DIR/$seq.rtvol) +rt_loop_dev=$(_create_loop_device $TEST_DIR/$seq.rtvol) echo "Format and mount rt volume" export USE_EXTERNAL=yes -export SCRATCH_RTDEV=$rtdev +export SCRATCH_RTDEV=$rt_loop_dev _scratch_mkfs -d size=$((1024 * 1024 * 1024)) \ -r size=${rtextsz},extsize=${rtextsz} >> $seqres.full _try_scratch_mount || _notrun "Couldn't mount fs with synthetic rt volume"@@ -116,8 +116,9 @@ done echo "Check filesystem" _check_scratch_fs -losetup -d $rtdev -rm -f $TEST_DIR/$seq.rtvol +_scratch_unmount +_destroy_loop_device $rt_loop_dev +unset rt_loop_dev # success, all done status=0tests/xfs/606 +8
-4@@ -13,10 +13,8 @@ _begin_fstest auto quick growfs _cleanup() { - local dev _unmount $LOOP_MNT 2>/dev/null - dev=$(losetup -j testfile | cut -d: -f1) - losetup -d $dev 2>/dev/null + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev rm -rf $LOOP_IMG $LOOP_MNT cd / rm -f $tmp.*@@ -40,7 +38,9 @@ $MKFS_XFS_PROG -f $LOOP_IMG >$seqres.full # Extend by just 8K, expected to start with the last full-size AG ends of # above 1G block device. $XFS_IO_PROG -f -c "truncate 1073750016" $LOOP_IMG -_mount -oloop $LOOP_IMG $LOOP_MNT + +loop_dev=$(_create_loop_device $LOOP_IMG) +_mount $loop_dev $LOOP_MNT # A known bug shows "XFS_IOC_FSGROWFSDATA xfsctl failed: No space left on # device" at here, refer to _fixed_by_kernel_commit above $XFS_GROWFS_PROG $LOOP_MNT >$seqres.full@@ -48,6 +48,10 @@ if [ $? -ne 0 ];then echo "xfs_growfs fails!" fi +_unmount $LOOP_MNT +_destroy_loop_device $loop_dev +unset loop_dev + echo "Silence is golden" # success, all done status=0tests/xfs/613 +14
-26@@ -15,14 +15,8 @@ _cleanup() cd / rm -f $tmp.* _unmount $LOOP_MNT 2>/dev/null - if [ -n "$LOOP_DEV" ];then - _destroy_loop_device $LOOP_DEV 2>/dev/null - fi - if [ -n "$LOOP_SPARE_DEV" ];then - _destroy_loop_device $LOOP_SPARE_DEV 2>/dev/null - fi + [ -n $loop_dev ] &&_destroy_loop_device $loop_dev rm -f $LOOP_IMG - rm -f $LOOP_SPARE_IMG rmdir $LOOP_MNT }@@ -38,16 +32,11 @@ _require_loop _require_xfs_io_command "falloc" LOOP_IMG=$TEST_DIR/$seq.dev -LOOP_SPARE_IMG=$TEST_DIR/$seq.logdev LOOP_MNT=$TEST_DIR/$seq.mnt echo "** create loop device" $XFS_IO_PROG -f -c "truncate 32g" $LOOP_IMG -LOOP_DEV=`_create_loop_device $LOOP_IMG` - -echo "** create loop log device" -$XFS_IO_PROG -f -c "truncate 1g" $LOOP_SPARE_IMG -LOOP_SPARE_DEV=`_create_loop_device $LOOP_SPARE_IMG` +loop_dev=`_create_loop_device $LOOP_IMG` echo "** create loop mount point" rmdir $LOOP_MNT 2>/dev/null@@ -56,8 +45,7 @@ mkdir -p $LOOP_MNT || _fail "cannot create loopback mount point" filter_loop() { sed -e "s,\B$LOOP_MNT,LOOP_MNT,g" \ - -e "s,\B$LOOP_DEV,LOOP_DEV,g" \ - -e "s,\B$LOOP_SPARE_DEV,LOOP_SPARE_DEV,g" + -e "s,\B$loop_dev,LOOP_DEV,g" } filter_xfs_opt()@@ -70,22 +58,22 @@ MKFS_OPTIONS="" do_mkfs() { echo "FORMAT: $@" | filter_loop | tee -a $seqres.full - $MKFS_XFS_PROG -f $* $LOOP_DEV | _filter_mkfs >>$seqres.full 2>$tmp.mkfs + $MKFS_XFS_PROG -f $* $loop_dev | _filter_mkfs >>$seqres.full 2>$tmp.mkfs if [ "${PIPESTATUS[0]}" -ne 0 ]; then - _fail "Fails on _mkfs_dev $* $LOOP_DEV" + _fail "Fails on _mkfs_dev $* $loop_dev" fi . $tmp.mkfs } is_dev_mounted() { - findmnt --source $LOOP_DEV >/dev/null + findmnt --source $loop_dev >/dev/null return $? } get_mount_info() { - findmnt --source $LOOP_DEV -o OPTIONS -n + findmnt --source $loop_dev -o OPTIONS -n } force_unmount()@@ -104,29 +92,29 @@ _do_test() local info # mount test - _mount $LOOP_DEV $LOOP_MNT $opts 2>>$seqres.full + _mount $loop_dev $LOOP_MNT $opts 2>>$seqres.full rc=$? if [ $rc -eq 0 ];then if [ "${mounted}" = "fail" ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: expect mount to fail, but it succeeded" return 1 fi is_dev_mounted if [ $? -ne 0 ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: fs not mounted even mount return 0" return 1 fi else if [ "${mounted}" = "pass" ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: expect mount to succeed, but it failed" return 1 fi is_dev_mounted if [ $? -eq 0 ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: fs is mounted even mount return non-zero" return 1 fi@@ -142,13 +130,13 @@ _do_test() rc=$? if [ $rc -eq 0 ];then if [ "$found" != "true" ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: expected to find \"$key\" in mount info \"$info\"" return 1 fi else if [ "$found" != "false" ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: did not expect to find \"$key\" in \"$info\"" return 1 fitests/xfs/613.out +0
-1@@ -1,6 +1,5 @@ QA output created by 613 ** create loop device -** create loop log device ** create loop mount point ** start xfs mount testing ... FORMAT: -m crc=0Signature · 1 line
2.45.2
Re: [PATCH 13/40] fstests: clean up loop device instantiation
Quoted 84 lines · Zorro Lang
Quoted 70 lines · Dave Chinner
From: Dave Chinner <[email protected]>
Lots of tests do there own special thing with loop devices rather than using
_create_loop_device()and_destroy_loop_device(). This oftens means they do not clean up after themselves properly, leaving stale loop devices around that result in unmountable test or scratch devices. This is common when tests are killed by user interrupt.Even the tests that do use
_destroy_loop_deviceand try to clean up often do it incorrectly, leading to spurious error messages.Some tests try to use dynamic instantiation via "mount -o loop", but then don't clean up in the correct order or hack around to find the loop device that was instantiated because the test needs to know the instantiated device name
Clean this up by converting all the tests to use
_create_loop_device()and_destroy_loop_device(). In all the tests, use the variable "loop_dev" for the device consistently. In_destroy_loop_device(), test that a device name has been passed so that we don't try to clean up the same device twice (e.g. once before test exit and again from the_cleanup()function). When we destroy a loop device, unset the variable used to hold the loop device name so that we don't try to destroy it twice.This results in much more reliable cleanup and clean exit from fstests when killed by the user.
Signed-off-by Dave Chinner <[email protected]>--- common/metadump | 32 ++++++++++++-------------------- tests/generic/067 | 11 ++++++++--- tests/generic/361 | 6 +++++- tests/generic/563 | 14 +++++++++----- tests/generic/564 | 12 ++++++------ tests/generic/590 | 9 ++++++--- tests/generic/744 | 4 ++-- tests/generic/746 | 6 +++++- tests/xfs/014 | 7 +++++-- tests/xfs/049 | 35 +++++++++++++++++++++-------------- tests/xfs/073 | 24 +++++++++++++----------- tests/xfs/074 | 21 ++++++++++++--------- tests/xfs/078 | 16 +++++++++------- tests/xfs/148 | 23 ++++++++++++++--------- tests/xfs/149 | 4 ++++ tests/xfs/216 | 15 ++++++++++----- tests/xfs/217 | 22 ++++++++++++++++------ tests/xfs/250 | 14 +++++++++----- tests/xfs/259 | 13 ++++++++----- tests/xfs/513 | 44 ++++++++++++++++++++------------------------ tests/xfs/521 | 8 ++++++-- tests/xfs/528 | 10 +++++++--- tests/xfs/530 | 11 ++++++----- tests/xfs/606 | 12 ++++++++---- tests/xfs/613 | 40 ++++++++++++++-------------------------- tests/xfs/613.out | 1 - 26 files changed, 235 insertions(+), 179 deletions(-)common/metadump +0
-3@@ -24,17 +24,9 @@ _xfs_cleanup_verify_metadump() test -n "$XFS_METADUMP_FILE" && rm -f "$XFS_METADUMP_FILE" - if [ -n "$XFS_METADUMP_IMG" ]; then - losetup -n -a -O BACK-FILE,NAME | grep "^$XFS_METADUMP_IMG" | while read backing ldev; do - losetup -d "$ldev"common/metadump _xfs_cleanup_verify_metadump() · line 28
Quoted 10 lines
common/metadump +2
-8- done - - # Don't call rm directly with a globbed argument here to avoid - # issues issues with variable expansions. - for img in "$XFS_METADUMP_IMG"*; do - test -e "$img" && rm -f "$img" - done - fi + [ -n "$md_data_loop_dev" ] && _destroy_loop_device $md_data_loop_dev + [ -n "$md_log_loop_dev" ] && _destroy_loop_device $md_log_loop_devAs Darrick reported that he always hit below error:
Quoted 7 lines
losetup:
/dev/loop0: detach failed: No such device or address Cannot destroy loop device/dev/loop0I'm wondering we should use [ -b ] not [ -n ] at here? Or output the stderr to
/dev/nulldirectly. Due to... (see below)common/metadump _xfs_cleanup_verify_metadump() · line 29
Quoted 1 line
common/metadump +1
-0+ rm -f $data_img $log_img
^^^^^^ ^^^^^^
Besides above problems, this line doesn't work either, due to the "data_img" and "log_img" are local variables in other functions. Even they're not local, think these variable names are too common to be global variable in common helper.
I'll try to change part of above new code back to old code as below [1]. Hope it can help the test work as usual, then we can merge this patch and move on. And we can improve this part in later patch.
Thanks,
Zorro
[1]
- [ -n "$md_data_loop_dev" ] && _destroy_loop_device $md_data_loop_dev - [ -n "$md_log_loop_dev" ] && _destroy_loop_device $md_log_loop_dev - rm -f $data_img $log_img + if [ -n "$XFS_METADUMP_IMG" ]; then + [ -b "$md_data_loop_dev" ] && _destroy_loop_device $md_data_loop_dev + [ -b "$md_log_loop_dev" ] && _destroy_loop_device $md_log_loop_dev + for img in "$XFS_METADUMP_IMG"*; do
+ test -e "$img" && rm -f "$img"
+ done
+ fi
Quoted 1,249 lines
Quoted 15 lines
common/metadump +2
-2} # Can xfs_metadump snapshot the fs metadata to a v1 metadump file?@@ -122,25 +114,25 @@ _xfs_verify_metadump_v2() _scratch_xfs_mdrestore $metadump_file # Create loopdev for data device so we can mount the fs - data_loop=$(_create_loop_device $data_img) + md_data_loop_dev=$(_create_loop_device $data_img) # Create loopdev for log device if we recovered anything - test -s "$log_img" && log_loop=$(_create_loop_device $log_img) + test -s "$log_img" && md_log_loop_dev=$(_create_loop_device $log_img) # Mount fs, run an extra test, fsck, and unmountcommon/metadump _xfs_verify_metadump_v2() · line 128
Quoted 10 lines
common/metadump +4
-4- SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop _scratch_mount + SCRATCH_DEV=$md_data_loop_dev SCRATCH_LOGDEV=$md_log_loop_dev _scratch_mount if [ -n "$extra_test" ]; then - SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop $extra_test + SCRATCH_DEV=$md_data_loop_dev SCRATCH_LOGDEV=$md_log_loop_dev $extra_test fi - SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop _check_xfs_scratch_fs - SCRATCH_DEV=$data_loop _scratch_unmount + SCRATCH_DEV=$md_data_loop_dev SCRATCH_LOGDEV=$md_log_loop_dev _check_xfs_scratch_fs + _unmount $md_data_loop_devWe umount loop devices we created above at here, and ...
common/metadump _xfs_verify_metadump_v2() · line 135
Quoted 10 lines
common/metadump +3
-3# Tear down what we created - if [ -b "$log_loop" ]; then - _destroy_loop_device $log_loop + if [ -b "$md_log_loop_dev" ]; then + _destroy_loop_device $md_log_loop_dev rm -f $log_img fi - _destroy_loop_device $data_loop + _destroy_loop_device $md_data_loop_devwe even has destroied them at here.
So that
_xfs_cleanup_verify_metadumpfunction hit "detach failed: No such device or address", due to those loop devices might have been released, but the variables are not unset.I didn't give it a test, just did a code review. As Darrick uses this part mostly, could you please check what I said above?
Thanks,
ZorroQuoted 1,198 lines
common/metadump +0
-0rm -f $data_img }tests/generic/067 +8
-3@@ -37,13 +37,18 @@ mount_nonexistent_mnt() $MOUNT_PROG $SCRATCH_DEV $TEST_DIR/nosuchdir >>$seqres.full 2>&1 } -# fs driver should be able to handle mounting a free loop device gracefully -# xfs ever hung, "ec53d1d xfs: don't block on buffer read errors" fixed it +# fs driver should be able to handle mounting a free loop device gracefully xfs +# ever hung, "ec53d1d xfs: don't block on buffer read errors" fixed it +# +# Using 'losetup -f' like this is racy. We might end up mounting a real loop dev +# here, so unmount $SCRATCH_MNT (not the loop dev we might not own!) and ignore +# any error it might return. mount_free_loopdev() { echo "# mount a free loop device" >>$seqres.full loopdev=`losetup -f` - $MOUNT_PROG -t $FSTYP $loopdev $SCRATCH_MNT >>$seqres.full 2>&1 + _mount $loopdev $SCRATCH_MNT >>$seqres.full 2>&1 + _unmount $SCRATCH_MNT >> /dev/null 2>&1 } # mount with wrong fs type specified.tests/generic/361 +5
-1@@ -17,7 +17,7 @@ _begin_fstest auto quick _cleanup() { _unmount $fs_mnt - _destroy_loop_device $loop_dev + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev cd / rm -f $tmp.* }@@ -54,6 +54,10 @@ $XFS_IO_PROG -fc "pwrite 0 520m" $fs_mnt/testfile >>$seqres.full 2>&1 # remount should not hang $MOUNT_PROG -o remount,ro $fs_mnt >>$seqres.full 2>&1 +_unmount $fs_mnt +_destroy_loop_device $loop_dev +unset loop_dev + # success, all done echo "Silence is golden" status=0tests/generic/563 +9
-5@@ -87,12 +87,12 @@ reset() # cgroup I/O accounting doesn't work on partitions. Use a loop device to rule # that out. -LOOP_DEV=$(_create_loop_device $SCRATCH_DEV) -smajor=$((0x`stat -L -c %t $LOOP_DEV`)) -sminor=$((0x`stat -L -c %T $LOOP_DEV`)) +loop_dev=$(_create_loop_device $SCRATCH_DEV) +smajor=$((0x`stat -L -c %t $loop_dev`)) +sminor=$((0x`stat -L -c %T $loop_dev`)) -_mkfs_dev $LOOP_DEV >> $seqres.full 2>&1 -_mount $LOOP_DEV $SCRATCH_MNT || _fail "mount failed" +_mkfs_dev $loop_dev >> $seqres.full 2>&1 +_mount $loop_dev $SCRATCH_MNT || _fail "mount failed" blksize=$(_get_block_size "$SCRATCH_MNT")@@ -147,6 +147,10 @@ if [ "$drop_io_cgroup" = 1 ]; then echo "-io" > $cgdir/cgroup.subtree_control || _fail "subtree control" fi +_unmount $SCRATCH_MNT +_destroy_loop_device $loop_dev +unset loop_dev + # success, all done status=0 exittests/generic/564 +6
-6@@ -19,7 +19,7 @@ _cleanup() { cd / rm -rf $tmp.* - [ -z "$loopdev" ] || _destroy_loop_device $loopdev + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev } # Import common functions.@@ -72,11 +72,11 @@ $XFS_IO_PROG -f -c "copy_range -l 32k $testdir" $testdir/copy echo echo source/destination as blkdev returns EINVAL $XFS_IO_PROG -f -c "truncate 128k" $testdir/img >> $seqres.full 2>&1 -loopdev=`_create_loop_device $testdir/img` -$XFS_IO_PROG -c "copy_range -l 32k $testdir/file" $loopdev -$XFS_IO_PROG -f -c "copy_range -l 32k $loopdev" $testdir/copy -_destroy_loop_device $loopdev -loopdev= +loop_dev=`_create_loop_device $testdir/img` +$XFS_IO_PROG -c "copy_range -l 32k $testdir/file" $loop_dev +$XFS_IO_PROG -f -c "copy_range -l 32k $loop_dev" $testdir/copy +_destroy_loop_device $loop_dev +unset loop_dev echo echo source/destination as chardev returns EINVALtests/generic/590 +6
-3@@ -15,9 +15,10 @@ _begin_fstest auto prealloc preallocrw # Override the default cleanup function. _cleanup() { + _scratch_unmount + [ -n $loop_dev ] && _destroy_loop_device $loop_dev cd / rm -f $tmp.* - test -n "$loop" && _destroy_loop_device "$loop" rm -f "$TEST_DIR/$seq" }@@ -56,9 +57,9 @@ if [[ $FSTYP = xfs ]]; then loopsz="$((filesz + (1 << 26)))" _require_fs_space "$TEST_DIR" $((loopsz / 1024)) $XFS_IO_PROG -c "truncate $loopsz" -f "$TEST_DIR/$seq" - loop="$(_create_loop_device "$TEST_DIR/$seq")" + loop_dev="$(_create_loop_device "$TEST_DIR/$seq")" USE_EXTERNAL=yes - SCRATCH_RTDEV="$loop" + SCRATCH_RTDEV="$loop_dev" disabled_features=() # disable reflink if not supported by realtime devices@@ -114,6 +115,8 @@ $XFS_IO_PROG -c "truncate 0" -c fsync "$SCRATCH_MNT/file" # We need to do this before the loop device gets torn down. _scratch_unmount _check_scratch_fs +_destroy_loop_device $loop_dev +unset loop_dev echo "Silence is golden" status=0tests/generic/744 +2
-2@@ -17,8 +17,8 @@ _cleanup() _unmount $mnt2 &> /dev/null _unmount $mnt1 &> /dev/null - [ -b "$loop_dev2" ] && losetup -d $loop_dev2 - [ -b "$loop_dev1" ] && losetup -d $loop_dev1 + [ -b "$loop_dev2" ] && _destroy_loop_device $loop_dev2 + [ -b "$loop_dev1" ] && _destroy_loop_device $loop_dev1 [ -n "$seq" ] && rm -rf $TEST_DIR/$seq }tests/generic/746 +5
-1@@ -39,7 +39,7 @@ esac _cleanup() { _unmount $loop_mnt &> /dev/null - _destroy_loop_device $loop_dev + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev if [ $status -eq 0 ]; then rm -rf $tmp rm $img_file@@ -223,5 +223,9 @@ while read line; do done < $fiemap_after echo "done." +_unmount $loop_mnt +_destroy_loop_device $loop_dev +unset loop_dev + status=0 exittests/xfs/014 +5
-2@@ -23,6 +23,7 @@ _cleanup() { cd / _unmount $LOOP_MNT 2>/dev/null + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev _scratch_unmount 2>/dev/null rm -f $tmp.* }@@ -166,19 +167,21 @@ LOOP_FILE=$SCRATCH_MNT/$seq.fs LOOP_MNT=$SCRATCH_MNT/$seq.mnt $MKFS_XFS_PROG -d "file=1,name=$LOOP_FILE,size=10g" >> $seqres.full 2>&1 +loop_dev=$(_create_loop_device $LOOP_FILE) mkdir -p $LOOP_MNT -_mount -o uquota,gquota $LOOP_FILE $LOOP_MNT || \ +_mount -o uquota,gquota $loop_dev $LOOP_MNT || \ _fail "Failed to mount loop fs." _test_enospc $LOOP_MNT _test_edquot $LOOP_MNT _unmount $LOOP_MNT +_destroy_loop_device $loop_dev +unset loop_dev echo $orig_sp_time > /proc/sys/fs/xfs/speculative_prealloc_lifetime _scratch_unmount - status=0 exittests/xfs/049 +21
-14@@ -12,16 +12,17 @@ _begin_fstest rw auto quick # Override the default cleanup function. _cleanup() { - cd / - umount $SCRATCH_MNT/test2 > /dev/null 2>&1 - umount $SCRATCH_MNT/test > /dev/null 2>&1 - rm -f $tmp.* + cd / + _unmount $SCRATCH_MNT/test2 > /dev/null 2>&1 + _unmount $SCRATCH_MNT/test > /dev/null 2>&1 + [ -n "$loop_dev2" ] && _destroy_loop_device $loop_dev2 + [ -n "$loop_dev1" ] && _destroy_loop_device $loop_dev1 + rm -f $tmp.* - if [ -w $seqres.full ] - then - echo "--- mounts at end (after cleanup)" >> $seqres.full - mount >> $seqres.full - fi + if [ -w $seqres.full ]; then + echo "--- mounts at end (after cleanup)" >> $seqres.full + mount >> $seqres.full + fi } # Import common functions.@@ -64,7 +65,8 @@ mkdir $SCRATCH_MNT/test $SCRATCH_MNT/test2 >> $seqres.full 2>&1 \ || _fail "!!! failed to make mount points" _log "Mount xfs via loop" -mount -t xfs -o loop $SCRATCH_MNT/test.xfs $SCRATCH_MNT/test >> $seqres.full 2>&1 \ +loop_dev1=$(_create_loop_device $SCRATCH_MNT/test.xfs) +_mount $loop_dev1 $SCRATCH_MNT/test >> $seqres.full 2>&1 \ || _fail "!!! failed to loop mount xfs" _log "stress"@@ -80,11 +82,12 @@ dd if=/dev/zero of=$SCRATCH_MNT/test/test.ext2 bs=1024 count=10240 >> $seqres.fu || _fail "!!! create file failed" _log "Create ext2 fs in file on looped xfs" -echo y | mkfs -t ext2 $SCRATCH_MNT/test/test.ext2 >> $seqres.full 2>&1 \ +loop_dev2=$(_create_loop_device $SCRATCH_MNT/test/test.ext2) +echo y | mkfs -t ext2 $loop_dev2 >> $seqres.full 2>&1 \ || _fail "!!! failed to mkfs ext2 on xfs" _log "Mount ext2 on xfs via loop" -mount -t ext2 -o loop $SCRATCH_MNT/test/test.ext2 $SCRATCH_MNT/test2 >> $seqres.full 2>&1 \ +_mount $loop_dev2 $SCRATCH_MNT/test2 >> $seqres.full 2>&1 \ || _fail "!!! failed to loop mount xfs" _log "stress ext2 on xfs via loop"@@ -96,12 +99,16 @@ rm -rf $SCRATCH_MNT/test/* >> $seqres.full 2>&1 \ || _fail "!!! clean failed" _log "umount ext2 on xfs" -umount $SCRATCH_MNT/test2 >> $seqres.full 2>&1 \ +_unmount $SCRATCH_MNT/test2 >> $seqres.full 2>&1 \ || _fail "!!! umount ext2 failed" +_destroy_loop_device $loop_dev2 +unset loop_dev2 _log "umount xfs" -umount $SCRATCH_MNT/test >> $seqres.full 2>&1 \ +_unmount $SCRATCH_MNT/test >> $seqres.full 2>&1 \ || _fail "!!! umount xfs failed" +_destroy_loop_device $loop_dev1 +unset loop_dev1 echo "--- mounts at end (before cleanup)" >> $seqres.full mount >> $seqres.fulltests/xfs/073 +13
-11@@ -23,6 +23,8 @@ _cleanup() _scratch_unmount 2>/dev/null _unmount $imgs.loop 2>/dev/null _unmount $imgs.source_dir 2>/dev/null + [ -n "$loop_dev1" ] && _destroy_loop_device $loop_dev1 + [ -n "$loop_dev2" ] && _destroy_loop_device $loop_dev2 [ -d $imgs.loop ] && rmdir $imgs.loop [ -d $imgs.source_dir ] && rm -rf $imgs.source_dir rm -f $imgs.* $tmp.* /var/tmp/xfs_copy.log.*@@ -65,10 +67,11 @@ _verify_copy() rmdir $target_dir 2>/dev/null mkdir $target_dir - _mount -t xfs -o loop $target $target_dir 2>/dev/null + loop_dev1=$(_create_loop_device $target) + _mount $loop_dev1 $target_dir 2>/dev/null if [ $? -ne 0 ]; then echo retrying mount with nouuid option >>$seqres.full - _mount -t xfs -o loop -o nouuid $target $target_dir + _mount -o nouuid $loop_dev1 $target_dir if [ $? -ne 0 ]; then echo mount failed - evil! return@@ -100,6 +103,8 @@ _verify_copy() echo unmounting and removing new image _unmount $source_dir _unmount $target_dir > /dev/null 2>&1 + _destroy_loop_device $loop_dev1 + unset loop_dev1 rm -f $target }@@ -134,18 +139,15 @@ ${MKFS_XFS_PROG} -dfile,name=$imgs.source,size=100g \ rmdir $imgs.source_dir 2>/dev/null mkdir $imgs.source_dir -_mount -t xfs -o loop $imgs.source $imgs.source_dir -loop2=`mount | grep $imgs.source | grep -o -e 'loop=.*[^),]' | grep -o -e '/.*$'` +loop_dev2=$(_create_loop_device $imgs.source) +_mount $loop_dev2 $imgs.source_dir cp -a $here $imgs.source_dir -_mount -t xfs -o remount,ro $imgs.source $imgs.source_dir -$XFS_COPY_PROG $imgs.source $imgs.image | _filter_copy '#' $imgs.image '#' '#' +_mount -o remount,ro $loop_dev2 $imgs.source_dir +$XFS_COPY_PROG $loop_dev2 $imgs.image 2> /dev/null | _filter_copy '#' $imgs.image '#' '#' _verify_copy $imgs.image $imgs.source $imgs.source_dir -# HACK WARNING: -# -# We're done with the nested loop mount, now we have to clean -# up the pieces that mount is incapable of doing. -losetup -d $loop2 > /dev/null 2>&1 +_destroy_loop_device $loop_dev2 +unset loop_dev2 echo echo === copying scratch device to multiple targetstests/xfs/074 +12
-9@@ -26,7 +26,7 @@ _begin_fstest quick auto prealloc rw _cleanup() { cd / - _destroy_loop_device $LOOP_DEV + [ -n "$loop_dev" ] &&_destroy_loop_device $loop_dev rm -f $tmp.* $LOOP_FILE }@@ -45,10 +45,10 @@ LOOP_FILE=$TEST_DIR/$seq.img LOOP_MNT=$TEST_DIR/$seq.mnt mkdir -p $LOOP_MNT $XFS_IO_PROG -ft -c "truncate 1t" $LOOP_FILE >> $seqres.full -LOOP_DEV=`_create_loop_device $LOOP_FILE` +loop_dev=`_create_loop_device $LOOP_FILE` -_mkfs_dev -d size=260g,agcount=2 $LOOP_DEV -_mount $LOOP_DEV $LOOP_MNT +_mkfs_dev -d size=260g,agcount=2 $loop_dev +_mount $loop_dev $LOOP_MNT BLOCK_SIZE=$(_get_file_block_size $LOOP_MNT)@@ -59,11 +59,11 @@ $XFS_IO_PROG -ft \ -c "falloc 0 $(($BLOCK_SIZE * 2097152))" \ $LOOP_MNT/foo >> $seqres.full -umount $LOOP_MNT -_check_xfs_filesystem $LOOP_DEV none none +_unmount $LOOP_MNT +_check_xfs_filesystem $loop_dev none none -_mkfs_dev -f $LOOP_DEV -_mount $LOOP_DEV $LOOP_MNT +_mkfs_dev -f $loop_dev +_mount $loop_dev $LOOP_MNT # check we trim both ends of the extent approproiately; this will fail # on 1k block size filesystems without the correct fixes in place.@@ -73,7 +73,10 @@ $XFS_IO_PROG -ft \ $LOOP_MNT/foo >> $seqres.full _unmount $LOOP_MNT -_check_xfs_filesystem $LOOP_DEV none none +_check_xfs_filesystem $loop_dev none none + +_destroy_loop_device $loop_dev +unset loop_dev # success, all done echo "Silence is golden"tests/xfs/078 +9
-7@@ -17,7 +17,7 @@ _cleanup() cd / rm -f $tmp.* _unmount $LOOP_MNT 2>/dev/null - [ -n "$LOOP_DEV" ] && _destroy_loop_device $LOOP_DEV 2>/dev/null + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev 2>/dev/null # try to keep the image file if test fails [ $status -eq 0 ] && rm -f $LOOP_IMG rmdir $LOOP_MNT@@ -55,7 +55,7 @@ _grow_loop() agsize=$5 $XFS_IO_PROG -f -c "truncate $original" $LOOP_IMG - LOOP_DEV=`_create_loop_device $LOOP_IMG` + loop_dev=`_create_loop_device $LOOP_IMG` dparam="" if [ -n "$agsize" ]; then@@ -67,15 +67,15 @@ _grow_loop() echo echo "*** mkfs loop file (size=$original)" - $MKFS_XFS_PROG -b size=$bsize $dparam $LOOP_DEV | \ + $MKFS_XFS_PROG -b size=$bsize $dparam $loop_dev | \ _filter_mkfs 2>/dev/null echo "*** extend loop file" - _destroy_loop_device $LOOP_DEV + _destroy_loop_device $loop_dev $XFS_IO_PROG -c "pwrite $new_size $bsize" $LOOP_IMG | _filter_io - LOOP_DEV=`_create_loop_device $LOOP_IMG` + loop_dev=`_create_loop_device $LOOP_IMG` echo "*** mount loop filesystem" - _mount -t xfs $LOOP_DEV $LOOP_MNT + _mount $loop_dev $LOOP_MNT echo "*** grow loop filesystem" $XFS_GROWFS_PROG $LOOP_MNT 2>&1 | _filter_growfs 2>&1@@ -87,9 +87,11 @@ _grow_loop() if [ "$check" -gt "0" ] then echo "*** check" - _check_xfs_filesystem $LOOP_IMG none none + _check_xfs_filesystem $loop_dev none none fi + _destroy_loop_device $loop_dev + unset loop_dev rm -f $LOOP_IMG }tests/xfs/148 +14
-9@@ -15,7 +15,7 @@ _cleanup() { cd / _unmount $mntpt > /dev/null 2>&1 - _destroy_loop_device $loopdev > /dev/null 2>&1 + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev rm -r -f $tmp.* }@@ -48,12 +48,12 @@ rm -f $imgfile $imgfile.old # even when security xattrs are present so we are always doing name matches on # lookup and not name hash compares as leaf/node forms will do. $XFS_IO_PROG -f -c 'truncate 40m' $imgfile -loopdev=$(_create_loop_device $imgfile) -MKFS_OPTIONS="-m crc=0 -i size=512" _mkfs_dev $loopdev >> $seqres.full +loop_dev=$(_create_loop_device $imgfile) +MKFS_OPTIONS="-m crc=0 -i size=512" _mkfs_dev $loop_dev >> $seqres.full # Mount image file mkdir -p $mntpt -_mount $loopdev $mntpt +_mount $loop_dev $mntpt echo "creating entries" >> $seqres.full@@ -91,7 +91,8 @@ cat $tmp.log | _filter_test_dir # Corrupt the entries _unmount $mntpt -_destroy_loop_device $loopdev +_destroy_loop_device $loop_dev +unset loop_dev cp $imgfile $imgfile.old sed -b \ -e "s/$nullstr/too_many\x00beans/g" \@@ -99,8 +100,9 @@ sed -b \ -i $imgfile test "$(md5sum < $imgfile)" != "$(md5sum < $imgfile.old)" || _fail "sed failed to change the image file?" -loopdev=$(_create_loop_device $imgfile) -_mount $loopdev $mntpt + +loop_dev=$(_create_loop_device $imgfile) +_mount $loop_dev $mntpt # Try to access the corrupt metadata echo "++ ACCESSING BAD METADATA" | tee -a $seqres.full@@ -111,7 +113,7 @@ cat $tmp.log | _filter_test_dir | sed -e '/Could not list/d' echo "does scrub complain?" >> $seqres.full # Does scrub complain about this? -if _supports_xfs_scrub $mntpt $loopdev; then +if _supports_xfs_scrub $mntpt $loop_dev; then $XFS_SCRUB_PROG -n $mntpt >> $seqres.full 2>&1 res=$? test $((res & 1)) -eq 0 && \@@ -122,11 +124,14 @@ echo "does repair complain?" >> $seqres.full # Does repair complain about this? _unmount $mntpt -$XFS_REPAIR_PROG -n $loopdev >> $seqres.full 2>&1 +$XFS_REPAIR_PROG -n $loop_dev >> $seqres.full 2>&1 res=$? test $res -eq 1 || \ echo "repair failed to report corruption ($res)" +_destroy_loop_device $loop_dev +unset loop_dev + # success, all done status=0 exittests/xfs/149 +4
-0@@ -84,6 +84,10 @@ $XFS_GROWFS_PROG -D 16384 $loop_symlink > /dev/null echo "=== xfs_growfs - check device node ===" $XFS_GROWFS_PROG -D 20480 $loop_dev > /dev/null +_unmount $mntdir +_destroy_loop_device $loop_dev +unset loop_dev + # success, all done status=0 exittests/xfs/216 +10
-5@@ -15,6 +15,7 @@ _begin_fstest log metadata auto quick _cleanup() { _unmount $LOOP_MNT > /dev/null 2>&1 + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev cd / rm -f $tmp.* }@@ -24,7 +25,7 @@ _scratch_mkfs_xfs >/dev/null 2>&1 _scratch_mount _require_loop -LOOP_DEV=$SCRATCH_MNT/test_fs +LOOP_IMG=$SCRATCH_MNT/test_fs LOOP_MNT=$SCRATCH_MNT/test_fs_dir loop_mkfs_opts=@@ -55,22 +56,26 @@ _do_mkfs() for i in $*; do echo -n "fssize=${i}g " $MKFS_XFS_PROG -f -b size=4096 -l version=2 \ - -d name=$LOOP_DEV,size=${i}g $loop_mkfs_opts |grep log - _mount -o loop -t xfs $LOOP_DEV $LOOP_MNT + -d size=${i}g $loop_mkfs_opts $loop_dev |grep log + _mount $loop_dev $LOOP_MNT echo "test write" > $LOOP_MNT/test _unmount $LOOP_MNT > /dev/null 2>&1 done } # make large holey file -$XFS_IO_PROG -f -c "truncate 256g" $LOOP_DEV +$XFS_IO_PROG -f -c "truncate 256g" $LOOP_IMG -choose_golden_output $0 $LOOP_DEV +choose_golden_output $0 $LOOP_IMG #make loopback mount dir mkdir $LOOP_MNT +loop_dev=$(_create_loop_device $LOOP_IMG) + # walk over standard sizes (up to 256GB) _do_mkfs 1 2 4 8 16 32 64 128 256 +_destroy_loop_device $loop_dev +unset loop_dev status=0 exittests/xfs/217 +16
-6@@ -12,6 +12,12 @@ _begin_fstest log metadata auto # Import common functions. . ./common/filter +_cleanup() +{ + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev + cd / + rm -f $tmp.* +} _require_scratch _scratch_mkfs_xfs >/dev/null 2>&1@@ -20,7 +26,7 @@ _scratch_mount _require_fs_space $SCRATCH_MNT 2202000 _require_loop -LOOP_DEV=$SCRATCH_MNT/test_fs +LOOP_IMG=$SCRATCH_MNT/test_fs LOOP_MNT=$SCRATCH_MNT/test_fs_dir _do_mkfs()@@ -28,28 +34,30 @@ _do_mkfs() for i in $*; do echo -n "fssize=${i}g " $MKFS_XFS_PROG -f -b size=4096 -l version=2 \ - -d name=$LOOP_DEV,size=${i}g |grep log - _mount -o loop -t xfs $LOOP_DEV $LOOP_MNT + -d size=${i}g $loop_dev |grep log + _mount $loop_dev $LOOP_MNT echo "test write" > $LOOP_MNT/test _unmount $LOOP_MNT > /dev/null 2>&1 # punch out the previous blocks so that we keep the amount of # disk space the test requires down to a minimum. - $XFS_IO_PROG -f -c "unresvsp 0 16383g" $LOOP_DEV + $XFS_IO_PROG -f -c "unresvsp 0 16383g" $LOOP_IMG done } # make large holey file -$XFS_IO_PROG -f -c "truncate 16383g" $LOOP_DEV +$XFS_IO_PROG -f -c "truncate 16383g" $LOOP_IMG #make loopback mount dir mkdir $LOOP_MNT # test if large logs are supported -$MKFS_XFS_PROG -f -l size=256m -d name=$LOOP_DEV,size=10g > /dev/null 2>&1 +$MKFS_XFS_PROG -f -l size=256m -d name=$LOOP_IMG,size=10g > /dev/null 2>&1 if [ $? -ne 0 ]; then _notrun "large log sizes not supported by mkfs" fi +loop_dev=$(_create_loop_device $LOOP_IMG) + # # walk over "new" sizes supported by recent xfsprogs. # Note that the last test is for 16TB-1GB as 32bit platforms only support@@ -57,5 +65,7 @@ fi # _do_mkfs 512 1024 2048 4096 8192 16383 +_destroy_loop_device $loop_dev +unset loop_dev status=0 exittests/xfs/250 +9
-5@@ -14,7 +14,8 @@ _cleanup() { cd / _unmount $LOOP_MNT 2>/dev/null - rm -f $LOOP_DEV + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev + rm -f $LOOP_IMG rmdir $LOOP_MNT rm -f $tmp.* }@@ -26,7 +27,7 @@ _require_test _require_loop _require_xfs_io_command "falloc" -LOOP_DEV=$TEST_DIR/$seq.fs +LOOP_IMG=$TEST_DIR/$seq.fs LOOP_MNT=$TEST_DIR/$seq.mnt _filter_io()@@ -45,7 +46,7 @@ _test_loop() agsize=$2 fsize=$3 - dparam="file,name=$LOOP_DEV,size=$size" + dparam="file,name=$LOOP_IMG,size=$size" if [ -n "$agsize" ]; then dparam="$dparam,agsize=$agsize" fi@@ -55,7 +56,8 @@ _test_loop() | _filter_mkfs 2>/dev/null echo "*** mount loop filesystem" - _mount -t xfs -o loop $LOOP_DEV $LOOP_MNT + loop_dev=$(_create_loop_device $LOOP_IMG) + mount $loop_dev $LOOP_MNT echo "*** preallocate large file" $XFS_IO_PROG -f -c "resvsp 0 $fsize" $LOOP_MNT/foo | _filter_io@@ -64,7 +66,9 @@ _test_loop() _unmount $LOOP_MNT > /dev/null 2>&1 echo "*** check loop filesystem" - _check_xfs_filesystem $LOOP_DEV none none + _check_xfs_filesystem $loop_dev none none + _destroy_loop_device $loop_dev + unset loop_dev } _test_loop 50g 16m 40Gtests/xfs/259 +8
-5@@ -12,7 +12,10 @@ _begin_fstest auto quick # Override the default cleanup function. _cleanup() { - rm -f "$testfile" + [ -n "$loop_dev" ] && _destroy_loop_device $testfile + rm -f "$testfile" + cd / + rm -f $tmp.* } # Import common functions.@@ -45,13 +48,13 @@ for del in $sizes_to_check; do rm -f "$testfile" dd if=/dev/zero "of=$testfile" bs=1 count=0 seek=$ddseek \ >/dev/null 2>&1 || echo "dd failed" - lofile=$(losetup -f) - losetup $lofile "$testfile" - $MKFS_XFS_PROG -l size=32m -b size=$bs $lofile | _filter_mkfs \ + loop_dev=$(_create_loop_device $testfile) + $MKFS_XFS_PROG -l size=32m -b size=$bs $loop_dev | _filter_mkfs \ >/dev/null 2> $tmp.mkfs || echo "mkfs failed!" . $tmp.mkfs sync - losetup -d $lofile + _destroy_loop_device $loop_dev + unset loop_dev done donetests/xfs/513 +20
-24@@ -15,12 +15,8 @@ _cleanup() cd / rm -f $tmp.* _unmount $LOOP_MNT 2>/dev/null - if [ -n "$LOOP_DEV" ];then - _destroy_loop_device $LOOP_DEV 2>/dev/null - fi - if [ -n "$LOOP_SPARE_DEV" ];then - _destroy_loop_device $LOOP_SPARE_DEV 2>/dev/null - fi + [ -n $loop_dev ] &&_destroy_loop_device $loop_dev + [ -n $loop_spare_dev ] &&_destroy_loop_device $loop_spare_dev rm -f $LOOP_IMG rm -f $LOOP_SPARE_IMG rmdir $LOOP_MNT@@ -42,11 +38,11 @@ LOOP_MNT=$TEST_DIR/$seq.mnt echo "** create loop device" $XFS_IO_PROG -f -c "truncate 32g" $LOOP_IMG -LOOP_DEV=`_create_loop_device $LOOP_IMG` +loop_dev=`_create_loop_device $LOOP_IMG` echo "** create loop log device" $XFS_IO_PROG -f -c "truncate 1g" $LOOP_SPARE_IMG -LOOP_SPARE_DEV=`_create_loop_device $LOOP_SPARE_IMG` +loop_spare_dev=`_create_loop_device $LOOP_SPARE_IMG` echo "** create loop mount point" rmdir $LOOP_MNT 2>/dev/null@@ -55,8 +51,8 @@ mkdir -p $LOOP_MNT || _fail "cannot create loopback mount point" filter_loop() { sed -e "s,\B$LOOP_MNT,LOOP_MNT,g" \ - -e "s,\B$LOOP_DEV,LOOP_DEV,g" \ - -e "s,\B$LOOP_SPARE_DEV,LOOP_SPARE_DEV,g" + -e "s,\B$loop_dev,LOOP_DEV,g" \ + -e "s,\B$loop_spare_dev,LOOP_SPARE_DEV,g" } filter_xfs_opt()@@ -69,22 +65,22 @@ MKFS_OPTIONS="" do_mkfs() { echo "FORMAT: $@" | filter_loop | tee -a $seqres.full - $MKFS_XFS_PROG -f $* $LOOP_DEV | _filter_mkfs >>$seqres.full 2>$tmp.mkfs + $MKFS_XFS_PROG -f $* $loop_dev | _filter_mkfs >>$seqres.full 2>$tmp.mkfs if [ "${PIPESTATUS[0]}" -ne 0 ]; then - _fail "Fails on _mkfs_dev $* $LOOP_DEV" + _fail "Fails on _mkfs_dev $* $loop_dev" fi . $tmp.mkfs } is_dev_mounted() { - findmnt --source $LOOP_DEV >/dev/null + findmnt --source $loop_dev >/dev/null return $? } get_mount_info() { - findmnt --source $LOOP_DEV -o OPTIONS -n + findmnt --source $loop_dev -o OPTIONS -n } force_unmount()@@ -103,29 +99,29 @@ _do_test() local info # mount test - _mount $LOOP_DEV $LOOP_MNT $opts 2>>$seqres.full + _mount $loop_dev $LOOP_MNT $opts 2>>$seqres.full rc=$? if [ $rc -eq 0 ];then if [ "${mounted}" = "fail" ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: expect mount to fail, but it succeeded" return 1 fi is_dev_mounted if [ $? -ne 0 ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: fs not mounted even mount return 0" return 1 fi else if [ "${mounted}" = "pass" ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: expect mount to succeed, but it failed" return 1 fi is_dev_mounted if [ $? -eq 0 ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: fs is mounted even mount return non-zero" return 1 fi@@ -141,13 +137,13 @@ _do_test() rc=$? if [ $rc -eq 0 ];then if [ "$found" != "true" ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: expected to find \"$key\" in mount info \"$info\"" return 1 fi else if [ "$found" != "false" ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: did not expect to find \"$key\" in \"$info\"" return 1 fi@@ -253,9 +249,9 @@ do_test "-o logbsize=512k" fail # Test logdev do_mkfs do_test "" pass "logdev" "false" -do_test "-o logdev=$LOOP_SPARE_DEV" fail -do_mkfs -l logdev=$LOOP_SPARE_DEV -do_test "-o logdev=$LOOP_SPARE_DEV" pass "logdev=$LOOP_SPARE_DEV" "true" +do_test "-o logdev=$loop_spare_dev" fail +do_mkfs -l logdev=$loop_spare_dev +do_test "-o logdev=$loop_spare_dev" pass "logdev=$loop_spare_dev" "true" do_test "" fail # Test noaligntests/xfs/521 +6
-2@@ -21,7 +21,7 @@ _cleanup() { cd / _scratch_unmount >> $seqres.full 2>&1 - test -e "$rtdev" && losetup -d $rtdev >> $seqres.full 2>&1 + [ -n "$rt_loop_dev" ] && _destroy_loop_device $rt_loop_dev rm -f $tmp.* $TEST_DIR/$seq.rtvol }@@ -35,7 +35,7 @@ _require_no_large_scratch_dev echo "Create fake rt volume" truncate -s 400m $TEST_DIR/$seq.rtvol -rtdev=$(_create_loop_device $TEST_DIR/$seq.rtvol) +rt_loop_dev=$(_create_loop_device $TEST_DIR/$seq.rtvol) echo "Format and mount 100m rt volume" export USE_EXTERNAL=yes@@ -69,6 +69,10 @@ cp -p $testdir/original $testdir/copy3 echo "Check filesystem" _check_scratch_fs +_scratch_unmount +_destroy_loop_device $rt_loop_dev +unset rt_loop_dev + # success, all done status=0 exittests/xfs/528 +7
-3@@ -15,7 +15,7 @@ _cleanup() { cd / _scratch_unmount >> $seqres.full 2>&1 - test -e "$rtdev" && losetup -d $rtdev >> $seqres.full 2>&1 + [ -n "$rt_loop_dev" ] && _destroy_loop_device $rt_loop_dev rm -f $tmp.* $TEST_DIR/$seq.rtvol }@@ -155,11 +155,11 @@ test_ops() { echo "Create fake rt volume" $XFS_IO_PROG -f -c "truncate 400m" $TEST_DIR/$seq.rtvol -rtdev=$(_create_loop_device $TEST_DIR/$seq.rtvol) +rt_loop_dev=$(_create_loop_device $TEST_DIR/$seq.rtvol) echo "Make sure synth rt volume works" export USE_EXTERNAL=yes -export SCRATCH_RTDEV=$rtdev +export SCRATCH_RTDEV=$rt_loop_dev _scratch_mkfs > $seqres.full _try_scratch_mount || \ _notrun "Could not mount with synthetic rt volume"@@ -170,6 +170,10 @@ test_ops 262144 # not a power of two test_ops 327680 +_scratch_unmount +_destroy_loop_device $rt_loop_dev +unset rt_loop_dev + # success, all done status=0 exittests/xfs/530 +6
-5@@ -15,7 +15,7 @@ _cleanup() { cd / _scratch_unmount >> $seqres.full 2>&1 - test -e "$rtdev" && losetup -d $rtdev >> $seqres.full 2>&1 + [ -n "$rt_loop_dev" ] && _destroy_loop_device $rt_loop_dev rm -f $tmp.* $TEST_DIR/$seq.rtvol }@@ -52,12 +52,12 @@ fi rtdevsz=$((nr_bits * rtextsz)) truncate -s $rtdevsz $TEST_DIR/$seq.rtvol -rtdev=$(_create_loop_device $TEST_DIR/$seq.rtvol) +rt_loop_dev=$(_create_loop_device $TEST_DIR/$seq.rtvol) echo "Format and mount rt volume" export USE_EXTERNAL=yes -export SCRATCH_RTDEV=$rtdev +export SCRATCH_RTDEV=$rt_loop_dev _scratch_mkfs -d size=$((1024 * 1024 * 1024)) \ -r size=${rtextsz},extsize=${rtextsz} >> $seqres.full _try_scratch_mount || _notrun "Couldn't mount fs with synthetic rt volume"@@ -116,8 +116,9 @@ done echo "Check filesystem" _check_scratch_fs -losetup -d $rtdev -rm -f $TEST_DIR/$seq.rtvol +_scratch_unmount +_destroy_loop_device $rt_loop_dev +unset rt_loop_dev # success, all done status=0tests/xfs/606 +8
-4@@ -13,10 +13,8 @@ _begin_fstest auto quick growfs _cleanup() { - local dev _unmount $LOOP_MNT 2>/dev/null - dev=$(losetup -j testfile | cut -d: -f1) - losetup -d $dev 2>/dev/null + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev rm -rf $LOOP_IMG $LOOP_MNT cd / rm -f $tmp.*@@ -40,7 +38,9 @@ $MKFS_XFS_PROG -f $LOOP_IMG >$seqres.full # Extend by just 8K, expected to start with the last full-size AG ends of # above 1G block device. $XFS_IO_PROG -f -c "truncate 1073750016" $LOOP_IMG -_mount -oloop $LOOP_IMG $LOOP_MNT + +loop_dev=$(_create_loop_device $LOOP_IMG) +_mount $loop_dev $LOOP_MNT # A known bug shows "XFS_IOC_FSGROWFSDATA xfsctl failed: No space left on # device" at here, refer to _fixed_by_kernel_commit above $XFS_GROWFS_PROG $LOOP_MNT >$seqres.full@@ -48,6 +48,10 @@ if [ $? -ne 0 ];then echo "xfs_growfs fails!" fi +_unmount $LOOP_MNT +_destroy_loop_device $loop_dev +unset loop_dev + echo "Silence is golden" # success, all done status=0tests/xfs/613 +14
-26@@ -15,14 +15,8 @@ _cleanup() cd / rm -f $tmp.* _unmount $LOOP_MNT 2>/dev/null - if [ -n "$LOOP_DEV" ];then - _destroy_loop_device $LOOP_DEV 2>/dev/null - fi - if [ -n "$LOOP_SPARE_DEV" ];then - _destroy_loop_device $LOOP_SPARE_DEV 2>/dev/null - fi + [ -n $loop_dev ] &&_destroy_loop_device $loop_dev rm -f $LOOP_IMG - rm -f $LOOP_SPARE_IMG rmdir $LOOP_MNT }@@ -38,16 +32,11 @@ _require_loop _require_xfs_io_command "falloc" LOOP_IMG=$TEST_DIR/$seq.dev -LOOP_SPARE_IMG=$TEST_DIR/$seq.logdev LOOP_MNT=$TEST_DIR/$seq.mnt echo "** create loop device" $XFS_IO_PROG -f -c "truncate 32g" $LOOP_IMG -LOOP_DEV=`_create_loop_device $LOOP_IMG` - -echo "** create loop log device" -$XFS_IO_PROG -f -c "truncate 1g" $LOOP_SPARE_IMG -LOOP_SPARE_DEV=`_create_loop_device $LOOP_SPARE_IMG` +loop_dev=`_create_loop_device $LOOP_IMG` echo "** create loop mount point" rmdir $LOOP_MNT 2>/dev/null@@ -56,8 +45,7 @@ mkdir -p $LOOP_MNT || _fail "cannot create loopback mount point" filter_loop() { sed -e "s,\B$LOOP_MNT,LOOP_MNT,g" \ - -e "s,\B$LOOP_DEV,LOOP_DEV,g" \ - -e "s,\B$LOOP_SPARE_DEV,LOOP_SPARE_DEV,g" + -e "s,\B$loop_dev,LOOP_DEV,g" } filter_xfs_opt()@@ -70,22 +58,22 @@ MKFS_OPTIONS="" do_mkfs() { echo "FORMAT: $@" | filter_loop | tee -a $seqres.full - $MKFS_XFS_PROG -f $* $LOOP_DEV | _filter_mkfs >>$seqres.full 2>$tmp.mkfs + $MKFS_XFS_PROG -f $* $loop_dev | _filter_mkfs >>$seqres.full 2>$tmp.mkfs if [ "${PIPESTATUS[0]}" -ne 0 ]; then - _fail "Fails on _mkfs_dev $* $LOOP_DEV" + _fail "Fails on _mkfs_dev $* $loop_dev" fi . $tmp.mkfs } is_dev_mounted() { - findmnt --source $LOOP_DEV >/dev/null + findmnt --source $loop_dev >/dev/null return $? } get_mount_info() { - findmnt --source $LOOP_DEV -o OPTIONS -n + findmnt --source $loop_dev -o OPTIONS -n } force_unmount()@@ -104,29 +92,29 @@ _do_test() local info # mount test - _mount $LOOP_DEV $LOOP_MNT $opts 2>>$seqres.full + _mount $loop_dev $LOOP_MNT $opts 2>>$seqres.full rc=$? if [ $rc -eq 0 ];then if [ "${mounted}" = "fail" ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: expect mount to fail, but it succeeded" return 1 fi is_dev_mounted if [ $? -ne 0 ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: fs not mounted even mount return 0" return 1 fi else if [ "${mounted}" = "pass" ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: expect mount to succeed, but it failed" return 1 fi is_dev_mounted if [ $? -eq 0 ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: fs is mounted even mount return non-zero" return 1 fi@@ -142,13 +130,13 @@ _do_test() rc=$? if [ $rc -eq 0 ];then if [ "$found" != "true" ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: expected to find \"$key\" in mount info \"$info\"" return 1 fi else if [ "$found" != "false" ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: did not expect to find \"$key\" in \"$info\"" return 1 fitests/xfs/613.out +0
-1@@ -1,6 +1,5 @@ QA output created by 613 ** create loop device -** create loop log device ** create loop mount point ** start xfs mount testing ... FORMAT: -m crc=0Signature · 1 line
2.45.2
Re: [PATCH 13/40] fstests: clean up loop device instantiation
Quoted 105 lines · Zorro Lang
Quoted 84 lines · Zorro Lang
Quoted 70 lines · Dave Chinner
From: Dave Chinner <[email protected]>
Lots of tests do there own special thing with loop devices rather than using
_create_loop_device()and_destroy_loop_device(). This oftens means they do not clean up after themselves properly, leaving stale loop devices around that result in unmountable test or scratch devices. This is common when tests are killed by user interrupt.Even the tests that do use
_destroy_loop_deviceand try to clean up often do it incorrectly, leading to spurious error messages.Some tests try to use dynamic instantiation via "mount -o loop", but then don't clean up in the correct order or hack around to find the loop device that was instantiated because the test needs to know the instantiated device name
Clean this up by converting all the tests to use
_create_loop_device()and_destroy_loop_device(). In all the tests, use the variable "loop_dev" for the device consistently. In_destroy_loop_device(), test that a device name has been passed so that we don't try to clean up the same device twice (e.g. once before test exit and again from the_cleanup()function). When we destroy a loop device, unset the variable used to hold the loop device name so that we don't try to destroy it twice.This results in much more reliable cleanup and clean exit from fstests when killed by the user.
Signed-off-by Dave Chinner <[email protected]>--- common/metadump | 32 ++++++++++++-------------------- tests/generic/067 | 11 ++++++++--- tests/generic/361 | 6 +++++- tests/generic/563 | 14 +++++++++----- tests/generic/564 | 12 ++++++------ tests/generic/590 | 9 ++++++--- tests/generic/744 | 4 ++-- tests/generic/746 | 6 +++++- tests/xfs/014 | 7 +++++-- tests/xfs/049 | 35 +++++++++++++++++++++-------------- tests/xfs/073 | 24 +++++++++++++----------- tests/xfs/074 | 21 ++++++++++++--------- tests/xfs/078 | 16 +++++++++------- tests/xfs/148 | 23 ++++++++++++++--------- tests/xfs/149 | 4 ++++ tests/xfs/216 | 15 ++++++++++----- tests/xfs/217 | 22 ++++++++++++++++------ tests/xfs/250 | 14 +++++++++----- tests/xfs/259 | 13 ++++++++----- tests/xfs/513 | 44 ++++++++++++++++++++------------------------ tests/xfs/521 | 8 ++++++-- tests/xfs/528 | 10 +++++++--- tests/xfs/530 | 11 ++++++----- tests/xfs/606 | 12 ++++++++---- tests/xfs/613 | 40 ++++++++++++++-------------------------- tests/xfs/613.out | 1 - 26 files changed, 235 insertions(+), 179 deletions(-)common/metadump +0
-3@@ -24,17 +24,9 @@ _xfs_cleanup_verify_metadump() test -n "$XFS_METADUMP_FILE" && rm -f "$XFS_METADUMP_FILE" - if [ -n "$XFS_METADUMP_IMG" ]; then - losetup -n -a -O BACK-FILE,NAME | grep "^$XFS_METADUMP_IMG" | while read backing ldev; do - losetup -d "$ldev"common/metadump _xfs_cleanup_verify_metadump() · line 28
Quoted 10 lines
common/metadump +2
-8- done - - # Don't call rm directly with a globbed argument here to avoid - # issues issues with variable expansions. - for img in "$XFS_METADUMP_IMG"*; do - test -e "$img" && rm -f "$img" - done - fi + [ -n "$md_data_loop_dev" ] && _destroy_loop_device $md_data_loop_dev + [ -n "$md_log_loop_dev" ] && _destroy_loop_device $md_log_loop_devAs Darrick reported that he always hit below error:
Quoted 7 lines
losetup:
/dev/loop0: detach failed: No such device or address Cannot destroy loop device/dev/loop0I'm wondering we should use [ -b ] not [ -n ] at here? Or output the stderr to
/dev/nulldirectly. Due to... (see below)common/metadump _xfs_cleanup_verify_metadump() · line 29
Quoted 1 line
common/metadump +1
-0+ rm -f $data_img $log_img^^^^^^ ^^^^^^Besides above problems, this line doesn't work either, due to the "
data_img" and "log_img" are local variables in other functions. Even they're not local, think these variable names are too common to be global variable in common helper.I'll try to change part of above new code back to old code as below [1]. Hope it can help the test work as usual, then we can merge this patch and move on. And we can improve this part in later patch.
Thanks,
Zorro[1]
Review
Quoted 6 lines
(quoted diff) +3
-3- [ -n "$md_data_loop_dev" ] && _destroy_loop_device $md_data_loop_dev - [ -n "$md_log_loop_dev" ] && _destroy_loop_device $md_log_loop_dev - rm -f $data_img $log_img + if [ -n "$XFS_METADUMP_IMG" ]; then + [ -b "$md_data_loop_dev" ] && _destroy_loop_device $md_data_loop_dev + [ -b "$md_log_loop_dev" ] && _destroy_loop_device $md_log_loop_dev
Finally I can reproduce the issue which Darrick metioned, it needs a new xfsprogs feature which hasn't been merged. Then I found The [ -b ] only solves half of the problem, due to even the /dev/loop0 is deteched, the /dev/loop0 is still there, and [ -b /dev/loop0 ] return true.
To solve this problem totally, I think there're 3 ways:
1) Check if a loop device is really detected, likes the old code did:
"losetup -n -a -O BACK-FILE,NAME | grep "^$XFS_METADUMP_IMG" | while read backing ldev"
2) unset md_data_loop_dev and md_log_loop_dev properly, after _xfs_verify_metadump_v2() deteches them. Something likes this:
@@ -134,9 +134,11 @@ _xfs_verify_metadump_v2()
# Tear down what we created
if [ -b "$md_log_loop_dev" ]; then
_destroy_loop_device $md_log_loop_dev
+ unset md_log_loop_dev
rm -f $log_img
fi
_destroy_loop_device $md_data_loop_dev
+ unset md_data_loop_dev
rm -f $data_img
3) Ignore all outputs of _destroy_loop_device, e.g. _destroy_loop_device $md_data_loop_dev >/dev/null 2>&1 _destroy_loop_device $md_log_loop_dev >/dev/null 2>&1
The 1st and 3rd methods can avoid this failure totally. But the 2nd one looks smoother, and the xfs/129 test passed with it. I'm just not sure if I missed something else :)
# ./check -s default xfs/129 SECTION -- default FSTYP -- xfs (non-debug) PLATFORM -- Linux/x86_64 dell-per750-41 6.12.0-0.rc7.58.fc42.x86_64 #1 SMP PREEMPT_DYNAMIC Mon Nov 11 15:23:45 UTC 2024 MKFS_OPTIONS -- -f /dev/sda6 MOUNT_OPTIONS -- -o context=system_u:object_r:root_t:s0 /dev/sda6 /mnt/scratch
xfs/129 9s ... 14s
Ran: xfs/129
Passed all 1 tests
If you all agree, I can change this part as the 2nd way when I merge it.
Thanks,
Zorro
Quoted 1,254 lines
(quoted diff) +4
-0+ for img in "$XFS_METADUMP_IMG"*; do + test -e "$img" && rm -f "$img" + done + fiQuoted 1,250 lines
Quoted 16 lines
common/metadump +2
-2} # Can xfs_metadump snapshot the fs metadata to a v1 metadump file?@@ -122,25 +114,25 @@ _xfs_verify_metadump_v2() _scratch_xfs_mdrestore $metadump_file # Create loopdev for data device so we can mount the fs - data_loop=$(_create_loop_device $data_img) + md_data_loop_dev=$(_create_loop_device $data_img) # Create loopdev for log device if we recovered anything - test -s "$log_img" && log_loop=$(_create_loop_device $log_img) + test -s "$log_img" && md_log_loop_dev=$(_create_loop_device $log_img) # Mount fs, run an extra test, fsck, and unmountcommon/metadump _xfs_verify_metadump_v2() · line 128
Quoted 10 lines
common/metadump +4
-4- SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop _scratch_mount + SCRATCH_DEV=$md_data_loop_dev SCRATCH_LOGDEV=$md_log_loop_dev _scratch_mount if [ -n "$extra_test" ]; then - SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop $extra_test + SCRATCH_DEV=$md_data_loop_dev SCRATCH_LOGDEV=$md_log_loop_dev $extra_test fi - SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop _check_xfs_scratch_fs - SCRATCH_DEV=$data_loop _scratch_unmount + SCRATCH_DEV=$md_data_loop_dev SCRATCH_LOGDEV=$md_log_loop_dev _check_xfs_scratch_fs + _unmount $md_data_loop_devWe umount loop devices we created above at here, and ...
common/metadump _xfs_verify_metadump_v2() · line 135
Quoted 10 lines
common/metadump +3
-3# Tear down what we created - if [ -b "$log_loop" ]; then - _destroy_loop_device $log_loop + if [ -b "$md_log_loop_dev" ]; then + _destroy_loop_device $md_log_loop_dev rm -f $log_img fi - _destroy_loop_device $data_loop + _destroy_loop_device $md_data_loop_devwe even has destroied them at here.
So that
_xfs_cleanup_verify_metadumpfunction hit "detach failed: No such device or address", due to those loop devices might have been released, but the variables are not unset.I didn't give it a test, just did a code review. As Darrick uses this part mostly, could you please check what I said above?
Thanks,
ZorroQuoted 1,198 lines
common/metadump +0
-0rm -f $data_img }tests/generic/067 +8
-3@@ -37,13 +37,18 @@ mount_nonexistent_mnt() $MOUNT_PROG $SCRATCH_DEV $TEST_DIR/nosuchdir >>$seqres.full 2>&1 } -# fs driver should be able to handle mounting a free loop device gracefully -# xfs ever hung, "ec53d1d xfs: don't block on buffer read errors" fixed it +# fs driver should be able to handle mounting a free loop device gracefully xfs +# ever hung, "ec53d1d xfs: don't block on buffer read errors" fixed it +# +# Using 'losetup -f' like this is racy. We might end up mounting a real loop dev +# here, so unmount $SCRATCH_MNT (not the loop dev we might not own!) and ignore +# any error it might return. mount_free_loopdev() { echo "# mount a free loop device" >>$seqres.full loopdev=`losetup -f` - $MOUNT_PROG -t $FSTYP $loopdev $SCRATCH_MNT >>$seqres.full 2>&1 + _mount $loopdev $SCRATCH_MNT >>$seqres.full 2>&1 + _unmount $SCRATCH_MNT >> /dev/null 2>&1 } # mount with wrong fs type specified.tests/generic/361 +5
-1@@ -17,7 +17,7 @@ _begin_fstest auto quick _cleanup() { _unmount $fs_mnt - _destroy_loop_device $loop_dev + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev cd / rm -f $tmp.* }@@ -54,6 +54,10 @@ $XFS_IO_PROG -fc "pwrite 0 520m" $fs_mnt/testfile >>$seqres.full 2>&1 # remount should not hang $MOUNT_PROG -o remount,ro $fs_mnt >>$seqres.full 2>&1 +_unmount $fs_mnt +_destroy_loop_device $loop_dev +unset loop_dev + # success, all done echo "Silence is golden" status=0tests/generic/563 +9
-5@@ -87,12 +87,12 @@ reset() # cgroup I/O accounting doesn't work on partitions. Use a loop device to rule # that out. -LOOP_DEV=$(_create_loop_device $SCRATCH_DEV) -smajor=$((0x`stat -L -c %t $LOOP_DEV`)) -sminor=$((0x`stat -L -c %T $LOOP_DEV`)) +loop_dev=$(_create_loop_device $SCRATCH_DEV) +smajor=$((0x`stat -L -c %t $loop_dev`)) +sminor=$((0x`stat -L -c %T $loop_dev`)) -_mkfs_dev $LOOP_DEV >> $seqres.full 2>&1 -_mount $LOOP_DEV $SCRATCH_MNT || _fail "mount failed" +_mkfs_dev $loop_dev >> $seqres.full 2>&1 +_mount $loop_dev $SCRATCH_MNT || _fail "mount failed" blksize=$(_get_block_size "$SCRATCH_MNT")@@ -147,6 +147,10 @@ if [ "$drop_io_cgroup" = 1 ]; then echo "-io" > $cgdir/cgroup.subtree_control || _fail "subtree control" fi +_unmount $SCRATCH_MNT +_destroy_loop_device $loop_dev +unset loop_dev + # success, all done status=0 exittests/generic/564 +6
-6@@ -19,7 +19,7 @@ _cleanup() { cd / rm -rf $tmp.* - [ -z "$loopdev" ] || _destroy_loop_device $loopdev + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev } # Import common functions.@@ -72,11 +72,11 @@ $XFS_IO_PROG -f -c "copy_range -l 32k $testdir" $testdir/copy echo echo source/destination as blkdev returns EINVAL $XFS_IO_PROG -f -c "truncate 128k" $testdir/img >> $seqres.full 2>&1 -loopdev=`_create_loop_device $testdir/img` -$XFS_IO_PROG -c "copy_range -l 32k $testdir/file" $loopdev -$XFS_IO_PROG -f -c "copy_range -l 32k $loopdev" $testdir/copy -_destroy_loop_device $loopdev -loopdev= +loop_dev=`_create_loop_device $testdir/img` +$XFS_IO_PROG -c "copy_range -l 32k $testdir/file" $loop_dev +$XFS_IO_PROG -f -c "copy_range -l 32k $loop_dev" $testdir/copy +_destroy_loop_device $loop_dev +unset loop_dev echo echo source/destination as chardev returns EINVALtests/generic/590 +6
-3@@ -15,9 +15,10 @@ _begin_fstest auto prealloc preallocrw # Override the default cleanup function. _cleanup() { + _scratch_unmount + [ -n $loop_dev ] && _destroy_loop_device $loop_dev cd / rm -f $tmp.* - test -n "$loop" && _destroy_loop_device "$loop" rm -f "$TEST_DIR/$seq" }@@ -56,9 +57,9 @@ if [[ $FSTYP = xfs ]]; then loopsz="$((filesz + (1 << 26)))" _require_fs_space "$TEST_DIR" $((loopsz / 1024)) $XFS_IO_PROG -c "truncate $loopsz" -f "$TEST_DIR/$seq" - loop="$(_create_loop_device "$TEST_DIR/$seq")" + loop_dev="$(_create_loop_device "$TEST_DIR/$seq")" USE_EXTERNAL=yes - SCRATCH_RTDEV="$loop" + SCRATCH_RTDEV="$loop_dev" disabled_features=() # disable reflink if not supported by realtime devices@@ -114,6 +115,8 @@ $XFS_IO_PROG -c "truncate 0" -c fsync "$SCRATCH_MNT/file" # We need to do this before the loop device gets torn down. _scratch_unmount _check_scratch_fs +_destroy_loop_device $loop_dev +unset loop_dev echo "Silence is golden" status=0tests/generic/744 +2
-2@@ -17,8 +17,8 @@ _cleanup() _unmount $mnt2 &> /dev/null _unmount $mnt1 &> /dev/null - [ -b "$loop_dev2" ] && losetup -d $loop_dev2 - [ -b "$loop_dev1" ] && losetup -d $loop_dev1 + [ -b "$loop_dev2" ] && _destroy_loop_device $loop_dev2 + [ -b "$loop_dev1" ] && _destroy_loop_device $loop_dev1 [ -n "$seq" ] && rm -rf $TEST_DIR/$seq }tests/generic/746 +5
-1@@ -39,7 +39,7 @@ esac _cleanup() { _unmount $loop_mnt &> /dev/null - _destroy_loop_device $loop_dev + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev if [ $status -eq 0 ]; then rm -rf $tmp rm $img_file@@ -223,5 +223,9 @@ while read line; do done < $fiemap_after echo "done." +_unmount $loop_mnt +_destroy_loop_device $loop_dev +unset loop_dev + status=0 exittests/xfs/014 +5
-2@@ -23,6 +23,7 @@ _cleanup() { cd / _unmount $LOOP_MNT 2>/dev/null + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev _scratch_unmount 2>/dev/null rm -f $tmp.* }@@ -166,19 +167,21 @@ LOOP_FILE=$SCRATCH_MNT/$seq.fs LOOP_MNT=$SCRATCH_MNT/$seq.mnt $MKFS_XFS_PROG -d "file=1,name=$LOOP_FILE,size=10g" >> $seqres.full 2>&1 +loop_dev=$(_create_loop_device $LOOP_FILE) mkdir -p $LOOP_MNT -_mount -o uquota,gquota $LOOP_FILE $LOOP_MNT || \ +_mount -o uquota,gquota $loop_dev $LOOP_MNT || \ _fail "Failed to mount loop fs." _test_enospc $LOOP_MNT _test_edquot $LOOP_MNT _unmount $LOOP_MNT +_destroy_loop_device $loop_dev +unset loop_dev echo $orig_sp_time > /proc/sys/fs/xfs/speculative_prealloc_lifetime _scratch_unmount - status=0 exittests/xfs/049 +21
-14@@ -12,16 +12,17 @@ _begin_fstest rw auto quick # Override the default cleanup function. _cleanup() { - cd / - umount $SCRATCH_MNT/test2 > /dev/null 2>&1 - umount $SCRATCH_MNT/test > /dev/null 2>&1 - rm -f $tmp.* + cd / + _unmount $SCRATCH_MNT/test2 > /dev/null 2>&1 + _unmount $SCRATCH_MNT/test > /dev/null 2>&1 + [ -n "$loop_dev2" ] && _destroy_loop_device $loop_dev2 + [ -n "$loop_dev1" ] && _destroy_loop_device $loop_dev1 + rm -f $tmp.* - if [ -w $seqres.full ] - then - echo "--- mounts at end (after cleanup)" >> $seqres.full - mount >> $seqres.full - fi + if [ -w $seqres.full ]; then + echo "--- mounts at end (after cleanup)" >> $seqres.full + mount >> $seqres.full + fi } # Import common functions.@@ -64,7 +65,8 @@ mkdir $SCRATCH_MNT/test $SCRATCH_MNT/test2 >> $seqres.full 2>&1 \ || _fail "!!! failed to make mount points" _log "Mount xfs via loop" -mount -t xfs -o loop $SCRATCH_MNT/test.xfs $SCRATCH_MNT/test >> $seqres.full 2>&1 \ +loop_dev1=$(_create_loop_device $SCRATCH_MNT/test.xfs) +_mount $loop_dev1 $SCRATCH_MNT/test >> $seqres.full 2>&1 \ || _fail "!!! failed to loop mount xfs" _log "stress"@@ -80,11 +82,12 @@ dd if=/dev/zero of=$SCRATCH_MNT/test/test.ext2 bs=1024 count=10240 >> $seqres.fu || _fail "!!! create file failed" _log "Create ext2 fs in file on looped xfs" -echo y | mkfs -t ext2 $SCRATCH_MNT/test/test.ext2 >> $seqres.full 2>&1 \ +loop_dev2=$(_create_loop_device $SCRATCH_MNT/test/test.ext2) +echo y | mkfs -t ext2 $loop_dev2 >> $seqres.full 2>&1 \ || _fail "!!! failed to mkfs ext2 on xfs" _log "Mount ext2 on xfs via loop" -mount -t ext2 -o loop $SCRATCH_MNT/test/test.ext2 $SCRATCH_MNT/test2 >> $seqres.full 2>&1 \ +_mount $loop_dev2 $SCRATCH_MNT/test2 >> $seqres.full 2>&1 \ || _fail "!!! failed to loop mount xfs" _log "stress ext2 on xfs via loop"@@ -96,12 +99,16 @@ rm -rf $SCRATCH_MNT/test/* >> $seqres.full 2>&1 \ || _fail "!!! clean failed" _log "umount ext2 on xfs" -umount $SCRATCH_MNT/test2 >> $seqres.full 2>&1 \ +_unmount $SCRATCH_MNT/test2 >> $seqres.full 2>&1 \ || _fail "!!! umount ext2 failed" +_destroy_loop_device $loop_dev2 +unset loop_dev2 _log "umount xfs" -umount $SCRATCH_MNT/test >> $seqres.full 2>&1 \ +_unmount $SCRATCH_MNT/test >> $seqres.full 2>&1 \ || _fail "!!! umount xfs failed" +_destroy_loop_device $loop_dev1 +unset loop_dev1 echo "--- mounts at end (before cleanup)" >> $seqres.full mount >> $seqres.fulltests/xfs/073 +13
-11@@ -23,6 +23,8 @@ _cleanup() _scratch_unmount 2>/dev/null _unmount $imgs.loop 2>/dev/null _unmount $imgs.source_dir 2>/dev/null + [ -n "$loop_dev1" ] && _destroy_loop_device $loop_dev1 + [ -n "$loop_dev2" ] && _destroy_loop_device $loop_dev2 [ -d $imgs.loop ] && rmdir $imgs.loop [ -d $imgs.source_dir ] && rm -rf $imgs.source_dir rm -f $imgs.* $tmp.* /var/tmp/xfs_copy.log.*@@ -65,10 +67,11 @@ _verify_copy() rmdir $target_dir 2>/dev/null mkdir $target_dir - _mount -t xfs -o loop $target $target_dir 2>/dev/null + loop_dev1=$(_create_loop_device $target) + _mount $loop_dev1 $target_dir 2>/dev/null if [ $? -ne 0 ]; then echo retrying mount with nouuid option >>$seqres.full - _mount -t xfs -o loop -o nouuid $target $target_dir + _mount -o nouuid $loop_dev1 $target_dir if [ $? -ne 0 ]; then echo mount failed - evil! return@@ -100,6 +103,8 @@ _verify_copy() echo unmounting and removing new image _unmount $source_dir _unmount $target_dir > /dev/null 2>&1 + _destroy_loop_device $loop_dev1 + unset loop_dev1 rm -f $target }@@ -134,18 +139,15 @@ ${MKFS_XFS_PROG} -dfile,name=$imgs.source,size=100g \ rmdir $imgs.source_dir 2>/dev/null mkdir $imgs.source_dir -_mount -t xfs -o loop $imgs.source $imgs.source_dir -loop2=`mount | grep $imgs.source | grep -o -e 'loop=.*[^),]' | grep -o -e '/.*$'` +loop_dev2=$(_create_loop_device $imgs.source) +_mount $loop_dev2 $imgs.source_dir cp -a $here $imgs.source_dir -_mount -t xfs -o remount,ro $imgs.source $imgs.source_dir -$XFS_COPY_PROG $imgs.source $imgs.image | _filter_copy '#' $imgs.image '#' '#' +_mount -o remount,ro $loop_dev2 $imgs.source_dir +$XFS_COPY_PROG $loop_dev2 $imgs.image 2> /dev/null | _filter_copy '#' $imgs.image '#' '#' _verify_copy $imgs.image $imgs.source $imgs.source_dir -# HACK WARNING: -# -# We're done with the nested loop mount, now we have to clean -# up the pieces that mount is incapable of doing. -losetup -d $loop2 > /dev/null 2>&1 +_destroy_loop_device $loop_dev2 +unset loop_dev2 echo echo === copying scratch device to multiple targetstests/xfs/074 +12
-9@@ -26,7 +26,7 @@ _begin_fstest quick auto prealloc rw _cleanup() { cd / - _destroy_loop_device $LOOP_DEV + [ -n "$loop_dev" ] &&_destroy_loop_device $loop_dev rm -f $tmp.* $LOOP_FILE }@@ -45,10 +45,10 @@ LOOP_FILE=$TEST_DIR/$seq.img LOOP_MNT=$TEST_DIR/$seq.mnt mkdir -p $LOOP_MNT $XFS_IO_PROG -ft -c "truncate 1t" $LOOP_FILE >> $seqres.full -LOOP_DEV=`_create_loop_device $LOOP_FILE` +loop_dev=`_create_loop_device $LOOP_FILE` -_mkfs_dev -d size=260g,agcount=2 $LOOP_DEV -_mount $LOOP_DEV $LOOP_MNT +_mkfs_dev -d size=260g,agcount=2 $loop_dev +_mount $loop_dev $LOOP_MNT BLOCK_SIZE=$(_get_file_block_size $LOOP_MNT)@@ -59,11 +59,11 @@ $XFS_IO_PROG -ft \ -c "falloc 0 $(($BLOCK_SIZE * 2097152))" \ $LOOP_MNT/foo >> $seqres.full -umount $LOOP_MNT -_check_xfs_filesystem $LOOP_DEV none none +_unmount $LOOP_MNT +_check_xfs_filesystem $loop_dev none none -_mkfs_dev -f $LOOP_DEV -_mount $LOOP_DEV $LOOP_MNT +_mkfs_dev -f $loop_dev +_mount $loop_dev $LOOP_MNT # check we trim both ends of the extent approproiately; this will fail # on 1k block size filesystems without the correct fixes in place.@@ -73,7 +73,10 @@ $XFS_IO_PROG -ft \ $LOOP_MNT/foo >> $seqres.full _unmount $LOOP_MNT -_check_xfs_filesystem $LOOP_DEV none none +_check_xfs_filesystem $loop_dev none none + +_destroy_loop_device $loop_dev +unset loop_dev # success, all done echo "Silence is golden"tests/xfs/078 +9
-7@@ -17,7 +17,7 @@ _cleanup() cd / rm -f $tmp.* _unmount $LOOP_MNT 2>/dev/null - [ -n "$LOOP_DEV" ] && _destroy_loop_device $LOOP_DEV 2>/dev/null + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev 2>/dev/null # try to keep the image file if test fails [ $status -eq 0 ] && rm -f $LOOP_IMG rmdir $LOOP_MNT@@ -55,7 +55,7 @@ _grow_loop() agsize=$5 $XFS_IO_PROG -f -c "truncate $original" $LOOP_IMG - LOOP_DEV=`_create_loop_device $LOOP_IMG` + loop_dev=`_create_loop_device $LOOP_IMG` dparam="" if [ -n "$agsize" ]; then@@ -67,15 +67,15 @@ _grow_loop() echo echo "*** mkfs loop file (size=$original)" - $MKFS_XFS_PROG -b size=$bsize $dparam $LOOP_DEV | \ + $MKFS_XFS_PROG -b size=$bsize $dparam $loop_dev | \ _filter_mkfs 2>/dev/null echo "*** extend loop file" - _destroy_loop_device $LOOP_DEV + _destroy_loop_device $loop_dev $XFS_IO_PROG -c "pwrite $new_size $bsize" $LOOP_IMG | _filter_io - LOOP_DEV=`_create_loop_device $LOOP_IMG` + loop_dev=`_create_loop_device $LOOP_IMG` echo "*** mount loop filesystem" - _mount -t xfs $LOOP_DEV $LOOP_MNT + _mount $loop_dev $LOOP_MNT echo "*** grow loop filesystem" $XFS_GROWFS_PROG $LOOP_MNT 2>&1 | _filter_growfs 2>&1@@ -87,9 +87,11 @@ _grow_loop() if [ "$check" -gt "0" ] then echo "*** check" - _check_xfs_filesystem $LOOP_IMG none none + _check_xfs_filesystem $loop_dev none none fi + _destroy_loop_device $loop_dev + unset loop_dev rm -f $LOOP_IMG }tests/xfs/148 +14
-9@@ -15,7 +15,7 @@ _cleanup() { cd / _unmount $mntpt > /dev/null 2>&1 - _destroy_loop_device $loopdev > /dev/null 2>&1 + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev rm -r -f $tmp.* }@@ -48,12 +48,12 @@ rm -f $imgfile $imgfile.old # even when security xattrs are present so we are always doing name matches on # lookup and not name hash compares as leaf/node forms will do. $XFS_IO_PROG -f -c 'truncate 40m' $imgfile -loopdev=$(_create_loop_device $imgfile) -MKFS_OPTIONS="-m crc=0 -i size=512" _mkfs_dev $loopdev >> $seqres.full +loop_dev=$(_create_loop_device $imgfile) +MKFS_OPTIONS="-m crc=0 -i size=512" _mkfs_dev $loop_dev >> $seqres.full # Mount image file mkdir -p $mntpt -_mount $loopdev $mntpt +_mount $loop_dev $mntpt echo "creating entries" >> $seqres.full@@ -91,7 +91,8 @@ cat $tmp.log | _filter_test_dir # Corrupt the entries _unmount $mntpt -_destroy_loop_device $loopdev +_destroy_loop_device $loop_dev +unset loop_dev cp $imgfile $imgfile.old sed -b \ -e "s/$nullstr/too_many\x00beans/g" \@@ -99,8 +100,9 @@ sed -b \ -i $imgfile test "$(md5sum < $imgfile)" != "$(md5sum < $imgfile.old)" || _fail "sed failed to change the image file?" -loopdev=$(_create_loop_device $imgfile) -_mount $loopdev $mntpt + +loop_dev=$(_create_loop_device $imgfile) +_mount $loop_dev $mntpt # Try to access the corrupt metadata echo "++ ACCESSING BAD METADATA" | tee -a $seqres.full@@ -111,7 +113,7 @@ cat $tmp.log | _filter_test_dir | sed -e '/Could not list/d' echo "does scrub complain?" >> $seqres.full # Does scrub complain about this? -if _supports_xfs_scrub $mntpt $loopdev; then +if _supports_xfs_scrub $mntpt $loop_dev; then $XFS_SCRUB_PROG -n $mntpt >> $seqres.full 2>&1 res=$? test $((res & 1)) -eq 0 && \@@ -122,11 +124,14 @@ echo "does repair complain?" >> $seqres.full # Does repair complain about this? _unmount $mntpt -$XFS_REPAIR_PROG -n $loopdev >> $seqres.full 2>&1 +$XFS_REPAIR_PROG -n $loop_dev >> $seqres.full 2>&1 res=$? test $res -eq 1 || \ echo "repair failed to report corruption ($res)" +_destroy_loop_device $loop_dev +unset loop_dev + # success, all done status=0 exittests/xfs/149 +4
-0@@ -84,6 +84,10 @@ $XFS_GROWFS_PROG -D 16384 $loop_symlink > /dev/null echo "=== xfs_growfs - check device node ===" $XFS_GROWFS_PROG -D 20480 $loop_dev > /dev/null +_unmount $mntdir +_destroy_loop_device $loop_dev +unset loop_dev + # success, all done status=0 exittests/xfs/216 +10
-5@@ -15,6 +15,7 @@ _begin_fstest log metadata auto quick _cleanup() { _unmount $LOOP_MNT > /dev/null 2>&1 + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev cd / rm -f $tmp.* }@@ -24,7 +25,7 @@ _scratch_mkfs_xfs >/dev/null 2>&1 _scratch_mount _require_loop -LOOP_DEV=$SCRATCH_MNT/test_fs +LOOP_IMG=$SCRATCH_MNT/test_fs LOOP_MNT=$SCRATCH_MNT/test_fs_dir loop_mkfs_opts=@@ -55,22 +56,26 @@ _do_mkfs() for i in $*; do echo -n "fssize=${i}g " $MKFS_XFS_PROG -f -b size=4096 -l version=2 \ - -d name=$LOOP_DEV,size=${i}g $loop_mkfs_opts |grep log - _mount -o loop -t xfs $LOOP_DEV $LOOP_MNT + -d size=${i}g $loop_mkfs_opts $loop_dev |grep log + _mount $loop_dev $LOOP_MNT echo "test write" > $LOOP_MNT/test _unmount $LOOP_MNT > /dev/null 2>&1 done } # make large holey file -$XFS_IO_PROG -f -c "truncate 256g" $LOOP_DEV +$XFS_IO_PROG -f -c "truncate 256g" $LOOP_IMG -choose_golden_output $0 $LOOP_DEV +choose_golden_output $0 $LOOP_IMG #make loopback mount dir mkdir $LOOP_MNT +loop_dev=$(_create_loop_device $LOOP_IMG) + # walk over standard sizes (up to 256GB) _do_mkfs 1 2 4 8 16 32 64 128 256 +_destroy_loop_device $loop_dev +unset loop_dev status=0 exittests/xfs/217 +16
-6@@ -12,6 +12,12 @@ _begin_fstest log metadata auto # Import common functions. . ./common/filter +_cleanup() +{ + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev + cd / + rm -f $tmp.* +} _require_scratch _scratch_mkfs_xfs >/dev/null 2>&1@@ -20,7 +26,7 @@ _scratch_mount _require_fs_space $SCRATCH_MNT 2202000 _require_loop -LOOP_DEV=$SCRATCH_MNT/test_fs +LOOP_IMG=$SCRATCH_MNT/test_fs LOOP_MNT=$SCRATCH_MNT/test_fs_dir _do_mkfs()@@ -28,28 +34,30 @@ _do_mkfs() for i in $*; do echo -n "fssize=${i}g " $MKFS_XFS_PROG -f -b size=4096 -l version=2 \ - -d name=$LOOP_DEV,size=${i}g |grep log - _mount -o loop -t xfs $LOOP_DEV $LOOP_MNT + -d size=${i}g $loop_dev |grep log + _mount $loop_dev $LOOP_MNT echo "test write" > $LOOP_MNT/test _unmount $LOOP_MNT > /dev/null 2>&1 # punch out the previous blocks so that we keep the amount of # disk space the test requires down to a minimum. - $XFS_IO_PROG -f -c "unresvsp 0 16383g" $LOOP_DEV + $XFS_IO_PROG -f -c "unresvsp 0 16383g" $LOOP_IMG done } # make large holey file -$XFS_IO_PROG -f -c "truncate 16383g" $LOOP_DEV +$XFS_IO_PROG -f -c "truncate 16383g" $LOOP_IMG #make loopback mount dir mkdir $LOOP_MNT # test if large logs are supported -$MKFS_XFS_PROG -f -l size=256m -d name=$LOOP_DEV,size=10g > /dev/null 2>&1 +$MKFS_XFS_PROG -f -l size=256m -d name=$LOOP_IMG,size=10g > /dev/null 2>&1 if [ $? -ne 0 ]; then _notrun "large log sizes not supported by mkfs" fi +loop_dev=$(_create_loop_device $LOOP_IMG) + # # walk over "new" sizes supported by recent xfsprogs. # Note that the last test is for 16TB-1GB as 32bit platforms only support@@ -57,5 +65,7 @@ fi # _do_mkfs 512 1024 2048 4096 8192 16383 +_destroy_loop_device $loop_dev +unset loop_dev status=0 exittests/xfs/250 +9
-5@@ -14,7 +14,8 @@ _cleanup() { cd / _unmount $LOOP_MNT 2>/dev/null - rm -f $LOOP_DEV + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev + rm -f $LOOP_IMG rmdir $LOOP_MNT rm -f $tmp.* }@@ -26,7 +27,7 @@ _require_test _require_loop _require_xfs_io_command "falloc" -LOOP_DEV=$TEST_DIR/$seq.fs +LOOP_IMG=$TEST_DIR/$seq.fs LOOP_MNT=$TEST_DIR/$seq.mnt _filter_io()@@ -45,7 +46,7 @@ _test_loop() agsize=$2 fsize=$3 - dparam="file,name=$LOOP_DEV,size=$size" + dparam="file,name=$LOOP_IMG,size=$size" if [ -n "$agsize" ]; then dparam="$dparam,agsize=$agsize" fi@@ -55,7 +56,8 @@ _test_loop() | _filter_mkfs 2>/dev/null echo "*** mount loop filesystem" - _mount -t xfs -o loop $LOOP_DEV $LOOP_MNT + loop_dev=$(_create_loop_device $LOOP_IMG) + mount $loop_dev $LOOP_MNT echo "*** preallocate large file" $XFS_IO_PROG -f -c "resvsp 0 $fsize" $LOOP_MNT/foo | _filter_io@@ -64,7 +66,9 @@ _test_loop() _unmount $LOOP_MNT > /dev/null 2>&1 echo "*** check loop filesystem" - _check_xfs_filesystem $LOOP_DEV none none + _check_xfs_filesystem $loop_dev none none + _destroy_loop_device $loop_dev + unset loop_dev } _test_loop 50g 16m 40Gtests/xfs/259 +8
-5@@ -12,7 +12,10 @@ _begin_fstest auto quick # Override the default cleanup function. _cleanup() { - rm -f "$testfile" + [ -n "$loop_dev" ] && _destroy_loop_device $testfile + rm -f "$testfile" + cd / + rm -f $tmp.* } # Import common functions.@@ -45,13 +48,13 @@ for del in $sizes_to_check; do rm -f "$testfile" dd if=/dev/zero "of=$testfile" bs=1 count=0 seek=$ddseek \ >/dev/null 2>&1 || echo "dd failed" - lofile=$(losetup -f) - losetup $lofile "$testfile" - $MKFS_XFS_PROG -l size=32m -b size=$bs $lofile | _filter_mkfs \ + loop_dev=$(_create_loop_device $testfile) + $MKFS_XFS_PROG -l size=32m -b size=$bs $loop_dev | _filter_mkfs \ >/dev/null 2> $tmp.mkfs || echo "mkfs failed!" . $tmp.mkfs sync - losetup -d $lofile + _destroy_loop_device $loop_dev + unset loop_dev done donetests/xfs/513 +20
-24@@ -15,12 +15,8 @@ _cleanup() cd / rm -f $tmp.* _unmount $LOOP_MNT 2>/dev/null - if [ -n "$LOOP_DEV" ];then - _destroy_loop_device $LOOP_DEV 2>/dev/null - fi - if [ -n "$LOOP_SPARE_DEV" ];then - _destroy_loop_device $LOOP_SPARE_DEV 2>/dev/null - fi + [ -n $loop_dev ] &&_destroy_loop_device $loop_dev + [ -n $loop_spare_dev ] &&_destroy_loop_device $loop_spare_dev rm -f $LOOP_IMG rm -f $LOOP_SPARE_IMG rmdir $LOOP_MNT@@ -42,11 +38,11 @@ LOOP_MNT=$TEST_DIR/$seq.mnt echo "** create loop device" $XFS_IO_PROG -f -c "truncate 32g" $LOOP_IMG -LOOP_DEV=`_create_loop_device $LOOP_IMG` +loop_dev=`_create_loop_device $LOOP_IMG` echo "** create loop log device" $XFS_IO_PROG -f -c "truncate 1g" $LOOP_SPARE_IMG -LOOP_SPARE_DEV=`_create_loop_device $LOOP_SPARE_IMG` +loop_spare_dev=`_create_loop_device $LOOP_SPARE_IMG` echo "** create loop mount point" rmdir $LOOP_MNT 2>/dev/null@@ -55,8 +51,8 @@ mkdir -p $LOOP_MNT || _fail "cannot create loopback mount point" filter_loop() { sed -e "s,\B$LOOP_MNT,LOOP_MNT,g" \ - -e "s,\B$LOOP_DEV,LOOP_DEV,g" \ - -e "s,\B$LOOP_SPARE_DEV,LOOP_SPARE_DEV,g" + -e "s,\B$loop_dev,LOOP_DEV,g" \ + -e "s,\B$loop_spare_dev,LOOP_SPARE_DEV,g" } filter_xfs_opt()@@ -69,22 +65,22 @@ MKFS_OPTIONS="" do_mkfs() { echo "FORMAT: $@" | filter_loop | tee -a $seqres.full - $MKFS_XFS_PROG -f $* $LOOP_DEV | _filter_mkfs >>$seqres.full 2>$tmp.mkfs + $MKFS_XFS_PROG -f $* $loop_dev | _filter_mkfs >>$seqres.full 2>$tmp.mkfs if [ "${PIPESTATUS[0]}" -ne 0 ]; then - _fail "Fails on _mkfs_dev $* $LOOP_DEV" + _fail "Fails on _mkfs_dev $* $loop_dev" fi . $tmp.mkfs } is_dev_mounted() { - findmnt --source $LOOP_DEV >/dev/null + findmnt --source $loop_dev >/dev/null return $? } get_mount_info() { - findmnt --source $LOOP_DEV -o OPTIONS -n + findmnt --source $loop_dev -o OPTIONS -n } force_unmount()@@ -103,29 +99,29 @@ _do_test() local info # mount test - _mount $LOOP_DEV $LOOP_MNT $opts 2>>$seqres.full + _mount $loop_dev $LOOP_MNT $opts 2>>$seqres.full rc=$? if [ $rc -eq 0 ];then if [ "${mounted}" = "fail" ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: expect mount to fail, but it succeeded" return 1 fi is_dev_mounted if [ $? -ne 0 ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: fs not mounted even mount return 0" return 1 fi else if [ "${mounted}" = "pass" ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: expect mount to succeed, but it failed" return 1 fi is_dev_mounted if [ $? -eq 0 ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: fs is mounted even mount return non-zero" return 1 fi@@ -141,13 +137,13 @@ _do_test() rc=$? if [ $rc -eq 0 ];then if [ "$found" != "true" ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: expected to find \"$key\" in mount info \"$info\"" return 1 fi else if [ "$found" != "false" ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: did not expect to find \"$key\" in \"$info\"" return 1 fi@@ -253,9 +249,9 @@ do_test "-o logbsize=512k" fail # Test logdev do_mkfs do_test "" pass "logdev" "false" -do_test "-o logdev=$LOOP_SPARE_DEV" fail -do_mkfs -l logdev=$LOOP_SPARE_DEV -do_test "-o logdev=$LOOP_SPARE_DEV" pass "logdev=$LOOP_SPARE_DEV" "true" +do_test "-o logdev=$loop_spare_dev" fail +do_mkfs -l logdev=$loop_spare_dev +do_test "-o logdev=$loop_spare_dev" pass "logdev=$loop_spare_dev" "true" do_test "" fail # Test noaligntests/xfs/521 +6
-2@@ -21,7 +21,7 @@ _cleanup() { cd / _scratch_unmount >> $seqres.full 2>&1 - test -e "$rtdev" && losetup -d $rtdev >> $seqres.full 2>&1 + [ -n "$rt_loop_dev" ] && _destroy_loop_device $rt_loop_dev rm -f $tmp.* $TEST_DIR/$seq.rtvol }@@ -35,7 +35,7 @@ _require_no_large_scratch_dev echo "Create fake rt volume" truncate -s 400m $TEST_DIR/$seq.rtvol -rtdev=$(_create_loop_device $TEST_DIR/$seq.rtvol) +rt_loop_dev=$(_create_loop_device $TEST_DIR/$seq.rtvol) echo "Format and mount 100m rt volume" export USE_EXTERNAL=yes@@ -69,6 +69,10 @@ cp -p $testdir/original $testdir/copy3 echo "Check filesystem" _check_scratch_fs +_scratch_unmount +_destroy_loop_device $rt_loop_dev +unset rt_loop_dev + # success, all done status=0 exittests/xfs/528 +7
-3@@ -15,7 +15,7 @@ _cleanup() { cd / _scratch_unmount >> $seqres.full 2>&1 - test -e "$rtdev" && losetup -d $rtdev >> $seqres.full 2>&1 + [ -n "$rt_loop_dev" ] && _destroy_loop_device $rt_loop_dev rm -f $tmp.* $TEST_DIR/$seq.rtvol }@@ -155,11 +155,11 @@ test_ops() { echo "Create fake rt volume" $XFS_IO_PROG -f -c "truncate 400m" $TEST_DIR/$seq.rtvol -rtdev=$(_create_loop_device $TEST_DIR/$seq.rtvol) +rt_loop_dev=$(_create_loop_device $TEST_DIR/$seq.rtvol) echo "Make sure synth rt volume works" export USE_EXTERNAL=yes -export SCRATCH_RTDEV=$rtdev +export SCRATCH_RTDEV=$rt_loop_dev _scratch_mkfs > $seqres.full _try_scratch_mount || \ _notrun "Could not mount with synthetic rt volume"@@ -170,6 +170,10 @@ test_ops 262144 # not a power of two test_ops 327680 +_scratch_unmount +_destroy_loop_device $rt_loop_dev +unset rt_loop_dev + # success, all done status=0 exittests/xfs/530 +6
-5@@ -15,7 +15,7 @@ _cleanup() { cd / _scratch_unmount >> $seqres.full 2>&1 - test -e "$rtdev" && losetup -d $rtdev >> $seqres.full 2>&1 + [ -n "$rt_loop_dev" ] && _destroy_loop_device $rt_loop_dev rm -f $tmp.* $TEST_DIR/$seq.rtvol }@@ -52,12 +52,12 @@ fi rtdevsz=$((nr_bits * rtextsz)) truncate -s $rtdevsz $TEST_DIR/$seq.rtvol -rtdev=$(_create_loop_device $TEST_DIR/$seq.rtvol) +rt_loop_dev=$(_create_loop_device $TEST_DIR/$seq.rtvol) echo "Format and mount rt volume" export USE_EXTERNAL=yes -export SCRATCH_RTDEV=$rtdev +export SCRATCH_RTDEV=$rt_loop_dev _scratch_mkfs -d size=$((1024 * 1024 * 1024)) \ -r size=${rtextsz},extsize=${rtextsz} >> $seqres.full _try_scratch_mount || _notrun "Couldn't mount fs with synthetic rt volume"@@ -116,8 +116,9 @@ done echo "Check filesystem" _check_scratch_fs -losetup -d $rtdev -rm -f $TEST_DIR/$seq.rtvol +_scratch_unmount +_destroy_loop_device $rt_loop_dev +unset rt_loop_dev # success, all done status=0tests/xfs/606 +8
-4@@ -13,10 +13,8 @@ _begin_fstest auto quick growfs _cleanup() { - local dev _unmount $LOOP_MNT 2>/dev/null - dev=$(losetup -j testfile | cut -d: -f1) - losetup -d $dev 2>/dev/null + [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev rm -rf $LOOP_IMG $LOOP_MNT cd / rm -f $tmp.*@@ -40,7 +38,9 @@ $MKFS_XFS_PROG -f $LOOP_IMG >$seqres.full # Extend by just 8K, expected to start with the last full-size AG ends of # above 1G block device. $XFS_IO_PROG -f -c "truncate 1073750016" $LOOP_IMG -_mount -oloop $LOOP_IMG $LOOP_MNT + +loop_dev=$(_create_loop_device $LOOP_IMG) +_mount $loop_dev $LOOP_MNT # A known bug shows "XFS_IOC_FSGROWFSDATA xfsctl failed: No space left on # device" at here, refer to _fixed_by_kernel_commit above $XFS_GROWFS_PROG $LOOP_MNT >$seqres.full@@ -48,6 +48,10 @@ if [ $? -ne 0 ];then echo "xfs_growfs fails!" fi +_unmount $LOOP_MNT +_destroy_loop_device $loop_dev +unset loop_dev + echo "Silence is golden" # success, all done status=0tests/xfs/613 +14
-26@@ -15,14 +15,8 @@ _cleanup() cd / rm -f $tmp.* _unmount $LOOP_MNT 2>/dev/null - if [ -n "$LOOP_DEV" ];then - _destroy_loop_device $LOOP_DEV 2>/dev/null - fi - if [ -n "$LOOP_SPARE_DEV" ];then - _destroy_loop_device $LOOP_SPARE_DEV 2>/dev/null - fi + [ -n $loop_dev ] &&_destroy_loop_device $loop_dev rm -f $LOOP_IMG - rm -f $LOOP_SPARE_IMG rmdir $LOOP_MNT }@@ -38,16 +32,11 @@ _require_loop _require_xfs_io_command "falloc" LOOP_IMG=$TEST_DIR/$seq.dev -LOOP_SPARE_IMG=$TEST_DIR/$seq.logdev LOOP_MNT=$TEST_DIR/$seq.mnt echo "** create loop device" $XFS_IO_PROG -f -c "truncate 32g" $LOOP_IMG -LOOP_DEV=`_create_loop_device $LOOP_IMG` - -echo "** create loop log device" -$XFS_IO_PROG -f -c "truncate 1g" $LOOP_SPARE_IMG -LOOP_SPARE_DEV=`_create_loop_device $LOOP_SPARE_IMG` +loop_dev=`_create_loop_device $LOOP_IMG` echo "** create loop mount point" rmdir $LOOP_MNT 2>/dev/null@@ -56,8 +45,7 @@ mkdir -p $LOOP_MNT || _fail "cannot create loopback mount point" filter_loop() { sed -e "s,\B$LOOP_MNT,LOOP_MNT,g" \ - -e "s,\B$LOOP_DEV,LOOP_DEV,g" \ - -e "s,\B$LOOP_SPARE_DEV,LOOP_SPARE_DEV,g" + -e "s,\B$loop_dev,LOOP_DEV,g" } filter_xfs_opt()@@ -70,22 +58,22 @@ MKFS_OPTIONS="" do_mkfs() { echo "FORMAT: $@" | filter_loop | tee -a $seqres.full - $MKFS_XFS_PROG -f $* $LOOP_DEV | _filter_mkfs >>$seqres.full 2>$tmp.mkfs + $MKFS_XFS_PROG -f $* $loop_dev | _filter_mkfs >>$seqres.full 2>$tmp.mkfs if [ "${PIPESTATUS[0]}" -ne 0 ]; then - _fail "Fails on _mkfs_dev $* $LOOP_DEV" + _fail "Fails on _mkfs_dev $* $loop_dev" fi . $tmp.mkfs } is_dev_mounted() { - findmnt --source $LOOP_DEV >/dev/null + findmnt --source $loop_dev >/dev/null return $? } get_mount_info() { - findmnt --source $LOOP_DEV -o OPTIONS -n + findmnt --source $loop_dev -o OPTIONS -n } force_unmount()@@ -104,29 +92,29 @@ _do_test() local info # mount test - _mount $LOOP_DEV $LOOP_MNT $opts 2>>$seqres.full + _mount $loop_dev $LOOP_MNT $opts 2>>$seqres.full rc=$? if [ $rc -eq 0 ];then if [ "${mounted}" = "fail" ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: expect mount to fail, but it succeeded" return 1 fi is_dev_mounted if [ $? -ne 0 ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: fs not mounted even mount return 0" return 1 fi else if [ "${mounted}" = "pass" ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: expect mount to succeed, but it failed" return 1 fi is_dev_mounted if [ $? -eq 0 ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: fs is mounted even mount return non-zero" return 1 fi@@ -142,13 +130,13 @@ _do_test() rc=$? if [ $rc -eq 0 ];then if [ "$found" != "true" ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: expected to find \"$key\" in mount info \"$info\"" return 1 fi else if [ "$found" != "false" ];then - echo "[FAILED]: mount $LOOP_DEV $LOOP_MNT $opts" + echo "[FAILED]: mount $loop_dev $LOOP_MNT $opts" echo "ERROR: did not expect to find \"$key\" in \"$info\"" return 1 fitests/xfs/613.out +0
-1@@ -1,6 +1,5 @@ QA output created by 613 ** create loop device -** create loop log device ** create loop mount point ** start xfs mount testing ... FORMAT: -m crc=0Signature · 1 line
2.45.2
PATCH 14/40 fstests: xfs/227 is really slow
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
The slowest test tto run on my test VMs is xfs/227:
...
xfs/227 826
...
It is doing nested iteration on created filesets that are explicitly defined, so separate the inner loop filesets and run the outer loops in parallel.
Also reduce the number of times we have to execute setfattr and xfs_io to once per created file instead of once per xattr/extent count per file.
The result is test runtime reduction of ~60%.
....
xfs/227 336
....
--- tests/xfs/227 | 57 ++++++++++++++++++++++++++++++++------------------- 1 file changed, 36 insertions(+), 21 deletions(-)
tests/xfs/227 +36 -21
@@ -41,7 +41,7 @@ fragment_freespace() # allocate inode space mkdir -p $_dir for i in `seq 0 1 1000`; do - touch $_file.$i + echo -n > $_file.$i done for i in `seq 0 63 1000`; do mv $_file.$i $_dir
@@ -68,19 +68,23 @@ fragment_freespace() create_attrs() { - for foo in `seq 0 1 $1`; do - $SETFATTR_PROG -n user.$foo -v 0xbabe $2 - done + ( echo "# file: $2" + for i in `seq 0 1 $1`; do + echo "user.$foo=\"0xbabe\"" + done ) > $tmp.$1.attrs + $SETFATTR_PROG --restore=$tmp.$1.attrs $2 } create_data() { - size=`expr \( $1 + 1 \) \* 4096` - $XFS_IO_PROG -f -c "truncate $size" $2 > /dev/null 2>&1 - for foo in `seq $1 -1 0`; do - let offset=$foo*4096 - $XFS_IO_PROG -f -c "resvsp $offset 4096" $2 > /dev/null 2>&1 + local blocks=$1 + local cmd_str + + for off in `seq $blocks -1 0`; do + cmd_str="-c \"resvsp $((off * 4096)) 4096\" $cmd_str" done + $XFS_IO_PROG -f -c "truncate $((($blocks + 1) * 4096))" \ + $cmd_str $2 >> seqres.full 2>&1 } # create the designated file with a certain number of attributes and a certain
@@ -119,6 +123,27 @@ create_target_attr_last() create_attrs $nattrs $target } +do_fsr() +{ + local n=$1 + local i=$2 + local j + + for j in `seq 5 1 20`; do + create_target_attr_first $i $j $targ.$i.$j >> $seqres.full 2>&1 + done + xfs_bmap -vp $targ.$i.* >> $seqres.full 2>&1 + FSRXFSTEST=true xfs_fsr -d -v -C $n $targ.$i.* >> $seqres.full 2>&1 + xfs_bmap -vp $targ.$i.* >> $seqres.full 2>&1 + for j in `seq 5 1 20`; do + create_target_attr_last $i $j $targ.$i.$j >> $seqres.full 2>&1 + done + xfs_bmap -vp $targ.$i.* >> $seqres.full 2>&1 + FSRXFSTEST=true xfs_fsr -d -v -C $n $targ.$i.* >> $seqres.full 2>&1 + xfs_bmap -vp $targ.$i.* >> $seqres.full 2>&1 +} + + # use a small filesystem so we can control freespace easily _scratch_mkfs_sized $((50 * 1024 * 1024)) >> $seqres.full 2>&1 _scratch_mount
@@ -154,19 +179,9 @@ targ=$SCRATCH_MNT/fsr_test_file.$$ for n in `seq 4 1 12`; do echo "*** n == $n ***" >> $seqres.full for i in `seq 5 1 15`; do - for j in `seq 5 1 20`; do - create_target_attr_first $i $j $targ.$i.$j >> $seqres.full 2>&1 - done - xfs_bmap -vp $targ.$i.* >> $seqres.full 2>&1 - FSRXFSTEST=true xfs_fsr -d -v -C $n $targ.$i.* >> $seqres.full 2>&1 - xfs_bmap -vp $targ.$i.* >> $seqres.full 2>&1 - for j in `seq 5 1 20`; do - create_target_attr_last $i $j $targ.$i.$j >> $seqres.full 2>&1 - done - xfs_bmap -vp $targ.$i.* >> $seqres.full 2>&1 - FSRXFSTEST=true xfs_fsr -d -v -C $n $targ.$i.* >> $seqres.full 2>&1 - xfs_bmap -vp $targ.$i.* >> $seqres.full 2>&1 + do_fsr $n $i & done + wait done _scratch_unmount
Signature · 1 line
2.45.2
PATCH 15/40 fstests: mark tests that are unreliable when run in parallel
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
Add a group named "unreliable_in_parallel" to mark tests that do not give reliable results when multiple tests are run in parallel. Generally this happens with tests that are reliant on caching in some way, such as generating specific file layouts using buffered IO or expecting inodes to be cached in memory. These are perturbed by other tests running sync(), generating memory pressure, dropping caches, etc.
Hence whether these tests pass or fail is wholly dependent on what tests are running at the same time, and hence randomly fail when nothing has actually gone wrong. Hence they are unreliable as regression tests when running tests in parallel, so we add them to the "unreliable_in_parallel" group and a parallel check can exclude this group.
As tests are updated to be robust against external interference, they can be removed from the unreliable_in_parallel group.
--- doc/group-names.txt | 1 + tests/generic/336 | 7 ++++++- tests/generic/561 | 8 +++++++- tests/xfs/177 | 8 ++++++-- tests/xfs/232 | 6 +++++- tests/xfs/237 | 8 +++++++- tests/xfs/243 | 7 +++++-- tests/xfs/300 | 8 ++++++-- tests/xfs/440 | 6 +++++- tests/xfs/527 | 5 ++++- tests/xfs/631 | 7 ++++++- tests/xfs/802 | 7 ++++++- 12 files changed, 64 insertions(+), 14 deletions(-)
doc/group-names.txt +1 -0
@@ -138,6 +138,7 @@ trim FITRIM ioctl udf UDF functionality tests union tests from the unionmount test suite unlink O_TMPFILE unlinked files +unreliable_in_parallel randomly fail when run in parallel with other tests unshare fallocate FALLOC_FL_UNSHARE_RANGE v2log XFS v2 log format tests verity fsverity
tests/generic/336 +6 -1
@@ -9,8 +9,13 @@ # file F2 from directory B into directory C, fsync inode F1, power fail and # remount the filesystem, file F2 exists and is located only in directory C. # + +# unreliable_in_parallel: external sync operations can change what is synced to +# the log before the flakey device drops writes. hence post-remount file +# contents can be different to what the test expects. + . ./common/preamble -_begin_fstest auto quick metadata log +_begin_fstest auto quick metadata log unreliable_in_parallel # Override the default cleanup function. _cleanup()
tests/generic/561 +7 -1
@@ -7,8 +7,14 @@ # Dedup & random I/O race test, do multi-threads fsstress and dedupe on # same directory/files # + +# unreliable_in_parallel: duperemove is buggy. It can get stuck in endless +# fiemap mapping loops, and this seems to happen a *lot* when the system is +# under heavy load. when they do this, they don't die when they are supposed to +# and so have to be manually killed to end the test. + . ./common/preamble -_begin_fstest auto stress dedupe +_begin_fstest auto stress dedupe unreliable_in_parallel # Override the default cleanup function. _cleanup()
tests/xfs/177 +6 -2
@@ -21,9 +21,13 @@ # Regrettably, there is no way to poke /only/ XFS inode reclamation directly, # so we're stuck with setting xfssyncd_centisecs to a low value and sleeping # while watching the internal inode cache counters. -# + +# unreliable_in_parallel: cache residency is affected by external drop caches +# operations. Hence counting inodes "in cache" often does not reflect what the +# test has actually done. + . ./common/preamble -_begin_fstest auto ioctl +_begin_fstest auto ioctl unreliable_in_parallel _cleanup() {
tests/xfs/232 +5 -1
@@ -12,8 +12,12 @@ # - Wait for the reclaim to run. # - Write more and see how bad fragmentation is. # + +# unreliable_in_parallel: external sync operations affect what happens while +# the test is waiting for COW expiration. + . ./common/preamble -_begin_fstest auto quick clone fiemap prealloc +_begin_fstest auto quick clone fiemap prealloc unreliable_in_parallel # Override the default cleanup function. _cleanup()
tests/xfs/237 +7 -1
@@ -6,8 +6,14 @@ # # Test AIO DIO CoW behavior when the write temporarily fails. # + +# unreliable_in_parallel: external drop caches can co-incide with the error +# table being loaded, so the test being run fails with EIO trying to load the +# inode from disk instead of whatever operation it is supposed to fail on when +# the inode is already cached in memory. + . ./common/preamble -_begin_fstest auto quick clone eio +_begin_fstest auto quick clone eio unreliable_in_parallel # Override the default cleanup function. _cleanup()
tests/xfs/243 +5 -2
@@ -15,9 +15,12 @@ # 5. delalloc # - CoW across the halfway mark, starting with the unwritten extent. # - Check that the files are now different where we say they're different. -# + +# unreliable_in_parallel: external sync can affect the layout of the files being +# created, results in unreliable detection of delalloc extents. + . ./common/preamble -_begin_fstest auto quick clone punch prealloc +_begin_fstest auto quick clone punch prealloc unreliable_in_parallel # Import common functions. . ./common/filter
tests/xfs/300 +6 -2
@@ -5,9 +5,13 @@ # FS QA Test No. 300 # # Test xfs_fsr / exchangerange management of di_forkoff w/ selinux -# + +# unreliable_in_parallel: file layout appears to be perturbed by load related +# timing issues. Not 100% sure, but the backwards write does not reliably +# fragment the source file under heavy external load + . ./common/preamble -_begin_fstest auto fsr +_begin_fstest auto fsr unreliable_in_parallel # Import common functions. . ./common/filter
tests/xfs/440 +5 -1
@@ -8,8 +8,12 @@ # a file that has CoW reservations and no dirty pages. The reservations # should shift over to the new owner, but they do not. # + +# unreliable_in_parallel: external sync(1) and/or drop caches can reclaim inodes +# and free post-eof space, resulting in lower than expected block counts. + . ./common/preamble -_begin_fstest auto quick clone quota +_begin_fstest auto quick clone quota unreliable_in_parallel # Import common functions. . ./common/reflink
tests/xfs/527 +4 -1
@@ -14,8 +14,11 @@ # xfs: fix incorrect root dquot corruption error when switching group/project # quota types +# unreliable_in_parallel: dmesg check can pick up corruptions from other tests. +# Need to filter corruption reports by short scratch dev name. + . ./common/preamble -_begin_fstest auto quick quota +_begin_fstest auto quick quota unreliable_in_parallel # Import common functions. . ./common/quota
tests/xfs/631 +6 -1
@@ -7,8 +7,13 @@ # Post-EOF preallocation defeat test for direct I/O with extent size hints. # +# unreliable_in_parallel: external cache drops can result in the extent size +# being truncated as the inode is evicted from cache between writes. This can +# increase the number of extents significantly beyond what would be expected +# from the extent size hint. + . ./common/preamble -_begin_fstest auto quick prealloc rw +_begin_fstest auto quick prealloc rw unreliable_in_parallel . ./common/filter
tests/xfs/802 +6 -1
@@ -8,8 +8,13 @@ # filesystem, and that we can read the health reports after the fact. IOWs, # this is basic testing for the systemd background services. # + +# unreliable_in_parallel: this appears to try to run scrub services on all +# mounted filesystems - that's aproblem when there are a hundred other test +# filesystems mounted running other tests... + . ./common/preamble -_begin_fstest auto scrub +_begin_fstest auto scrub unreliable_in_parallel _cleanup() {
Signature · 1 line
2.45.2
PATCH 16/40 fstests: use udevadm wait in preference to settle
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
When running lots of tests in parallel, there are lots of filesystems and block devices changing state. This generates a lot of udev events when means the udev event queue is rarely empty. Unfortunately, an empty event queue is what udev settling waits upon. Hence calling UDEV_SETTLE_PROG can mean waiting for a lot of time for other tests to stop generating udev events.
For the majority of cases, what we care about is that udev has performed device node addition or removal, not that there are no udev events pending. Recent(-ish) systemd releases support 'udevadm wait' to wait for a specific file to be created or unlinked rather than waiting for the event that does that work to be completed.
Hence we don't have to wait for the udev event queue to empty, just for the udev event that does the device node manipulation to complete.
Introduce detection of 'udevadm wait' support and a _udev_wait() wrapper function to use it if it is available. If it isn't, the use the existing UDEV_SETTLE_PROG behaviour.
--- common/config | 35 +++++++++++++++++++++++++---------- common/rc | 25 ++++++++++++++++--------- tests/btrfs/291 | 5 +++-- tests/generic/081 | 6 +++--- tests/generic/108 | 7 +++---- tests/generic/459 | 6 +++--- 6 files changed, 53 insertions(+), 31 deletions(-)
common/config +25 -10
@@ -165,7 +165,7 @@ export XFS_MDRESTORE_PROG="$(type -P xfs_mdrestore)" export XFS_ADMIN_PROG="$(type -P xfs_admin)" export XFS_GROWFS_PROG=$(type -P xfs_growfs) export XFS_SPACEMAN_PROG="$(type -P xfs_spaceman)" -export XFS_SCRUB_PROG="$(type -P xfs_scrub)" +#export XFS_SCRUB_PROG="$(type -P xfs_scrub)" export XFS_PARALLEL_REPAIR_PROG="$(type -P xfs_prepair)" export XFS_PARALLEL_REPAIR64_PROG="$(type -P xfs_prepair64)" export __XFSDUMP_PROG="$(type -P xfsdump)"
@@ -236,18 +236,30 @@ export BTRFS_MAP_LOGICAL_PROG=$(type -P btrfs-map-logical) export PARTED_PROG="$(type -P parted)" export XFS_PROPERTY_PROG="$(type -P xfs_property)" -# use 'udevadm settle' or 'udevsettle' to wait for lv to be settled. -# newer systems have udevadm command but older systems like RHEL5 don't. -# But if neither one is available, just set it to "sleep 1" to wait for lv to -# be settled -UDEV_SETTLE_PROG="$(type -P udevadm)" -if [ "$UDEV_SETTLE_PROG" == "" ]; then - # try udevsettle command +# udev wait functions. +# +# This is how we wait for udev to create or remove device nodes after running a +# device create/remove command for logical volumes (e.g. lvm or dm). +# +# We can wait for the udev queue to empty via "settling". This, however, has +# major issues when running tests in parallel - the udev queue takes a long time +# to reach empty state. Hence if we have udev > 2.51 installed we use device +# waiting instead. This waits for the device node to appear/disappear rather +# than waiting for the udev queue to empty. +# +# If none of these methods are available, fall back to a simple delay (sleep 1) +# and hope this is sufficient. +UDEVADM_PROG="$(type -P udevadm)" +if [ -z "$UDEVADM_PROG" ]; then UDEV_SETTLE_PROG="$(type -P udevsettle)" else - # udevadm is available, add 'settle' as subcommand - UDEV_SETTLE_PROG="$UDEV_SETTLE_PROG settle" + UDEV_SETTLE_PROG="$UDEVADM_PROG settle" + $UDEVADM_PROG help | grep -q "Wait for device or device symlink" + if [ $? -eq 0 ]; then + UDEV_WAIT_PROG="$UDEVADM_PROG wait" + fi fi + # neither command is available, use sleep 1 # # Udev events are sent via netlink to userspace through
@@ -258,8 +270,11 @@ fi # exist or always be 0. We check for /proc/net to see CONFIG_NET was enabled. if [[ "$UDEV_SETTLE_PROG" == "" || ! -d /proc/net ]]; then UDEV_SETTLE_PROG="sleep 1" + unset UDEV_WAIT_PROG fi export UDEV_SETTLE_PROG +export UDEVADM_PROG +export UDEV_WAIT_PROG # Set MODPROBE_PATIENT_RM_TIMEOUT_SECONDS to "forever" if you want the patient # modprobe removal to run forever trying to remove a module.
common/rc +16 -9
@@ -5191,22 +5191,29 @@ _require_label_get_max() dummy=$(_label_get_max) } +_udev_wait() +{ + local args="$*" + + if [ -z "$UDEV_WAIT_PROG" ]; then + $UDEV_SETTLE_PROG >/dev/null 2>&1 + else + $UDEV_WAIT_PROG $args + fi +} + _dmsetup_remove() { - $UDEV_SETTLE_PROG >/dev/null 2>&1 - $DMSETUP_PROG remove --retry "$@" >>$seqres.full 2>&1 - $UDEV_SETTLE_PROG >/dev/null 2>&1 + [ $# -le 0 ] && return + + $DMSETUP_PROG remove --deferred "$@" >>$seqres.full 2>&1 + _udev_wait --removed /dev/mapper/$1 } _dmsetup_create() { - # Wait for udev to settle so that the dm creation doesn't fail because - # some udev subprogram opened one of the block devices mentioned in the - # table string w/ O_EXCL. Do it again at the end so that an immediate - # device open won't also fail. - $UDEV_SETTLE_PROG >/dev/null 2>&1 $DMSETUP_PROG create "$@" >>$seqres.full 2>&1 || return 1 - $UDEV_SETTLE_PROG >/dev/null 2>&1 + _udev_wait /dev/mapper/$1 } _require_btime()
tests/btrfs/291 +3 -2
@@ -21,6 +21,7 @@ _cleanup() cd / _log_writes_cleanup &> /dev/null $LVM_PROG vgremove -f -y $vgname >>$seqres.full 2>&1 + _udev_wait --removed /dev/mapper/$vgname-$lvname losetup -d $loop_dev >>$seqres.full 2>&1 rm -f $img _restore_fsverity_signatures
@@ -106,7 +107,7 @@ snap_dev=/dev/mapper/vg_replay-$snapname $LVM_PROG vgcreate -f $vgname $loop_dev >>$seqres.full 2>&1 || _fail "failed to vgcreate $vgname" $LVM_PROG lvcreate -L "$replay_bytes"B -n $lvname $vgname -y >>$seqres.full 2>&1 || \ _fail "failed to lvcreate $lvname" -$UDEV_SETTLE_PROG >>$seqres.full 2>&1 +_udev_wait /dev/mapper/$vgname-$lvname replay_log_prog=$here/src/log-writes/replay-log num_entries=$($replay_log_prog --log $LOGWRITES_DEV --num-entries)
@@ -125,7 +126,7 @@ do $LVM_PROG lvcreate -s -L 4M -n $snapname $vgname/$lvname >>$seqres.full 2>&1 || \ _fail "Failed to create snapshot" - $UDEV_SETTLE_PROG >>$seqres.full 2>&1 + _udev_wait /dev/mapper/$vgname-$snapname orphan=$(count_item $snap_dev ORPHAN) [ $state -eq 0 ] && [ $orphan -gt 0 ] && state=1
tests/generic/081 +3 -3
@@ -38,7 +38,7 @@ _cleanup() $LVM_PROG vgremove -f $vgname >>$seqres.full 2>&1 $LVM_PROG pvremove -f $SCRATCH_DEV >>$seqres.full 2>&1 pv_ret=$? - $UDEV_SETTLE_PROG + _udev_wait --removed /dev/mapper/$vgname-$lvname test $pv_ret -eq 0 && break sleep 2 done
@@ -70,8 +70,8 @@ $LVM_PROG vgcreate -f $vgname $SCRATCH_DEV >>$seqres.full 2>&1 # We use yes pipe instead of 'lvcreate --yes' because old version of lvm # (like 2.02.95 in RHEL6) don't support --yes option yes | $LVM_PROG lvcreate -L ${lvsize}M -n $lvname $vgname >>$seqres.full 2>&1 -# wait for lvcreation to fully complete -$UDEV_SETTLE_PROG >>$seqres.full 2>&1 +_udev_wait /dev/mapper/$vgname-$lvname + # _mkfs_dev exits the test on failure, this can make sure lv is created in # above vgcreate/lvcreate steps
tests/generic/108 +3 -4
@@ -20,8 +20,8 @@ _cleanup() echo running > /sys/block/`_short_dev $SCSI_DEBUG_DEV`/device/state _unmount $SCRATCH_MNT >>$seqres.full 2>&1 $LVM_PROG vgremove -f $vgname >>$seqres.full 2>&1 - $LVM_PROG pvremove -f $SCRATCH_DEV $SCSI_DEBUG_DEV >>$seqres.full 2>&1 - $UDEV_SETTLE_PROG + pvremove -f $SCRATCH_DEV $SCSI_DEBUG_DEV >>$seqres.full 2>&1 + _udev_wait --removed /dev/mapper/$vgname-$lvname _put_scsi_debug_dev rm -f $tmp.* }
@@ -57,8 +57,7 @@ $LVM_PROG vgcreate -f $vgname $SCSI_DEBUG_DEV $SCRATCH_DEV >>$seqres.full 2>&1 # (like 2.02.95 in RHEL6) don't support --yes option yes | $LVM_PROG lvcreate -i 2 -I 4m -L ${lvsize}m -n $lvname $vgname \ >>$seqres.full 2>&1 -# wait for lv creation to fully complete -$UDEV_SETTLE_PROG >>$seqres.full 2>&1 +_udev_wait /dev/mapper/$vgname-$lvname # _mkfs_dev exits the test on failure, this makes sure test lv is created by # above vgcreate/lvcreate operations
tests/generic/459 +3 -3
@@ -31,7 +31,7 @@ _cleanup() _unmount $SCRATCH_MNT >>$seqres.full 2>&1 $LVM_PROG vgremove -ff $vgname >>$seqres.full 2>&1 $LVM_PROG pvremove -ff $SCRATCH_DEV >>$seqres.full 2>&1 - $UDEV_SETTLE_PROG + _udev_wait --removed /dev/mapper/$vgname-$lvname } # Import common functions.
@@ -88,8 +88,7 @@ $LVM_PROG lvcreate --thinpool $poolname --errorwhenfull y \ $LVM_PROG lvcreate --virtualsize $virtsize \ -T $vgname/$poolname \ -n $lvname >>$seqres.full 2>&1 - -$UDEV_SETTLE_PROG &>/dev/null +_udev_wait /dev/mapper/$vgname-$lvname _mkfs_dev /dev/mapper/$vgname-$lvname >>$seqres.full 2>&1 # Running the test over the original volume doesn't reproduce the problem
@@ -97,6 +96,7 @@ _mkfs_dev /dev/mapper/$vgname-$lvname >>$seqres.full 2>&1 # reproducible, so, create a snapshot and run the test over it. $LVM_PROG lvcreate -k n -s $vgname/$lvname \ -n $snapname >>$seqres.full 2>&1 +_udev_wait /dev/mapper/$vgname-$snapname # Catch mount failure so we don't blindly go an freeze the root filesystem # instead of lvm volume.
Signature · 1 line
2.45.2
Re: [PATCH 16/40] fstests: use udevadm wait in preference to settle
Quoted 38 lines · Dave Chinner
From: Dave Chinner <[email protected]>
When running lots of tests in parallel, there are lots of filesystems and block devices changing state. This generates a lot of udev events when means the udev event queue is rarely empty. Unfortunately, an empty event queue is what udev settling waits upon. Hence calling
UDEV_SETTLE_PROGcan mean waiting for a lot of time for other tests to stop generating udev events.For the majority of cases, what we care about is that udev has performed device node addition or removal, not that there are no udev events pending. Recent(-ish) systemd releases support 'udevadm wait' to wait for a specific file to be created or unlinked rather than waiting for the event that does that work to be completed.
Hence we don't have to wait for the udev event queue to empty, just for the udev event that does the device node manipulation to complete.
Introduce detection of 'udevadm wait' support and a
_udev_wait()wrapper function to use it if it is available. If it isn't, the use the existingUDEV_SETTLE_PROGbehaviour.Signed-off-by Dave Chinner <[email protected]>--- common/config | 35 +++++++++++++++++++++++++---------- common/rc | 25 ++++++++++++++++--------- tests/btrfs/291 | 5 +++-- tests/generic/081 | 6 +++--- tests/generic/108 | 7 +++---- tests/generic/459 | 6 +++--- 6 files changed, 53 insertions(+), 31 deletions(-)common/config +0
-0
common/config export XFS_MDRESTORE_PROG="$(type -P xfs_mdrestore)" · line 168
Quoted 5 lines
common/config +1
-1@@ -165,7 +165,7 @@ export XFS_MDRESTORE_PROG="$(type -P xfs_mdrestore)" export XFS_ADMIN_PROG="$(type -P xfs_admin)" export XFS_GROWFS_PROG=$(type -P xfs_growfs) export XFS_SPACEMAN_PROG="$(type -P xfs_spaceman)" -export XFS_SCRUB_PROG="$(type -P xfs_scrub)" +#export XFS_SCRUB_PROG="$(type -P xfs_scrub)"
If you have problems with online fsck, please report them to the mailing list.
--D
Quoted 212 lines
common/config +24
-9export XFS_PARALLEL_REPAIR_PROG="$(type -P xfs_prepair)" export XFS_PARALLEL_REPAIR64_PROG="$(type -P xfs_prepair64)" export __XFSDUMP_PROG="$(type -P xfsdump)"@@ -236,18 +236,30 @@ export BTRFS_MAP_LOGICAL_PROG=$(type -P btrfs-map-logical) export PARTED_PROG="$(type -P parted)" export XFS_PROPERTY_PROG="$(type -P xfs_property)" -# use 'udevadm settle' or 'udevsettle' to wait for lv to be settled. -# newer systems have udevadm command but older systems like RHEL5 don't. -# But if neither one is available, just set it to "sleep 1" to wait for lv to -# be settled -UDEV_SETTLE_PROG="$(type -P udevadm)" -if [ "$UDEV_SETTLE_PROG" == "" ]; then - # try udevsettle command +# udev wait functions. +# +# This is how we wait for udev to create or remove device nodes after running a +# device create/remove command for logical volumes (e.g. lvm or dm). +# +# We can wait for the udev queue to empty via "settling". This, however, has +# major issues when running tests in parallel - the udev queue takes a long time +# to reach empty state. Hence if we have udev > 2.51 installed we use device +# waiting instead. This waits for the device node to appear/disappear rather +# than waiting for the udev queue to empty. +# +# If none of these methods are available, fall back to a simple delay (sleep 1) +# and hope this is sufficient. +UDEVADM_PROG="$(type -P udevadm)" +if [ -z "$UDEVADM_PROG" ]; then UDEV_SETTLE_PROG="$(type -P udevsettle)" else - # udevadm is available, add 'settle' as subcommand - UDEV_SETTLE_PROG="$UDEV_SETTLE_PROG settle" + UDEV_SETTLE_PROG="$UDEVADM_PROG settle" + $UDEVADM_PROG help | grep -q "Wait for device or device symlink" + if [ $? -eq 0 ]; then + UDEV_WAIT_PROG="$UDEVADM_PROG wait" + fi fi + # neither command is available, use sleep 1 # # Udev events are sent via netlink to userspace through@@ -258,8 +270,11 @@ fi # exist or always be 0. We check for /proc/net to see CONFIG_NET was enabled. if [[ "$UDEV_SETTLE_PROG" == "" || ! -d /proc/net ]]; then UDEV_SETTLE_PROG="sleep 1" + unset UDEV_WAIT_PROG fi export UDEV_SETTLE_PROG +export UDEVADM_PROG +export UDEV_WAIT_PROG # Set MODPROBE_PATIENT_RM_TIMEOUT_SECONDS to "forever" if you want the patient # modprobe removal to run forever trying to remove a module.common/rc +16
-9@@ -5191,22 +5191,29 @@ _require_label_get_max() dummy=$(_label_get_max) } +_udev_wait() +{ + local args="$*" + + if [ -z "$UDEV_WAIT_PROG" ]; then + $UDEV_SETTLE_PROG >/dev/null 2>&1 + else + $UDEV_WAIT_PROG $args + fi +} + _dmsetup_remove() { - $UDEV_SETTLE_PROG >/dev/null 2>&1 - $DMSETUP_PROG remove --retry "$@" >>$seqres.full 2>&1 - $UDEV_SETTLE_PROG >/dev/null 2>&1 + [ $# -le 0 ] && return + + $DMSETUP_PROG remove --deferred "$@" >>$seqres.full 2>&1 + _udev_wait --removed /dev/mapper/$1 } _dmsetup_create() { - # Wait for udev to settle so that the dm creation doesn't fail because - # some udev subprogram opened one of the block devices mentioned in the - # table string w/ O_EXCL. Do it again at the end so that an immediate - # device open won't also fail. - $UDEV_SETTLE_PROG >/dev/null 2>&1 $DMSETUP_PROG create "$@" >>$seqres.full 2>&1 || return 1 - $UDEV_SETTLE_PROG >/dev/null 2>&1 + _udev_wait /dev/mapper/$1 } _require_btime()tests/btrfs/291 +3
-2@@ -21,6 +21,7 @@ _cleanup() cd / _log_writes_cleanup &> /dev/null $LVM_PROG vgremove -f -y $vgname >>$seqres.full 2>&1 + _udev_wait --removed /dev/mapper/$vgname-$lvname losetup -d $loop_dev >>$seqres.full 2>&1 rm -f $img _restore_fsverity_signatures@@ -106,7 +107,7 @@ snap_dev=/dev/mapper/vg_replay-$snapname $LVM_PROG vgcreate -f $vgname $loop_dev >>$seqres.full 2>&1 || _fail "failed to vgcreate $vgname" $LVM_PROG lvcreate -L "$replay_bytes"B -n $lvname $vgname -y >>$seqres.full 2>&1 || \ _fail "failed to lvcreate $lvname" -$UDEV_SETTLE_PROG >>$seqres.full 2>&1 +_udev_wait /dev/mapper/$vgname-$lvname replay_log_prog=$here/src/log-writes/replay-log num_entries=$($replay_log_prog --log $LOGWRITES_DEV --num-entries)@@ -125,7 +126,7 @@ do $LVM_PROG lvcreate -s -L 4M -n $snapname $vgname/$lvname >>$seqres.full 2>&1 || \ _fail "Failed to create snapshot" - $UDEV_SETTLE_PROG >>$seqres.full 2>&1 + _udev_wait /dev/mapper/$vgname-$snapname orphan=$(count_item $snap_dev ORPHAN) [ $state -eq 0 ] && [ $orphan -gt 0 ] && state=1tests/generic/081 +3
-3@@ -38,7 +38,7 @@ _cleanup() $LVM_PROG vgremove -f $vgname >>$seqres.full 2>&1 $LVM_PROG pvremove -f $SCRATCH_DEV >>$seqres.full 2>&1 pv_ret=$? - $UDEV_SETTLE_PROG + _udev_wait --removed /dev/mapper/$vgname-$lvname test $pv_ret -eq 0 && break sleep 2 done@@ -70,8 +70,8 @@ $LVM_PROG vgcreate -f $vgname $SCRATCH_DEV >>$seqres.full 2>&1 # We use yes pipe instead of 'lvcreate --yes' because old version of lvm # (like 2.02.95 in RHEL6) don't support --yes option yes | $LVM_PROG lvcreate -L ${lvsize}M -n $lvname $vgname >>$seqres.full 2>&1 -# wait for lvcreation to fully complete -$UDEV_SETTLE_PROG >>$seqres.full 2>&1 +_udev_wait /dev/mapper/$vgname-$lvname + # _mkfs_dev exits the test on failure, this can make sure lv is created in # above vgcreate/lvcreate stepstests/generic/108 +3
-4@@ -20,8 +20,8 @@ _cleanup() echo running > /sys/block/`_short_dev $SCSI_DEBUG_DEV`/device/state _unmount $SCRATCH_MNT >>$seqres.full 2>&1 $LVM_PROG vgremove -f $vgname >>$seqres.full 2>&1 - $LVM_PROG pvremove -f $SCRATCH_DEV $SCSI_DEBUG_DEV >>$seqres.full 2>&1 - $UDEV_SETTLE_PROG + pvremove -f $SCRATCH_DEV $SCSI_DEBUG_DEV >>$seqres.full 2>&1 + _udev_wait --removed /dev/mapper/$vgname-$lvname _put_scsi_debug_dev rm -f $tmp.* }@@ -57,8 +57,7 @@ $LVM_PROG vgcreate -f $vgname $SCSI_DEBUG_DEV $SCRATCH_DEV >>$seqres.full 2>&1 # (like 2.02.95 in RHEL6) don't support --yes option yes | $LVM_PROG lvcreate -i 2 -I 4m -L ${lvsize}m -n $lvname $vgname \ >>$seqres.full 2>&1 -# wait for lv creation to fully complete -$UDEV_SETTLE_PROG >>$seqres.full 2>&1 +_udev_wait /dev/mapper/$vgname-$lvname # _mkfs_dev exits the test on failure, this makes sure test lv is created by # above vgcreate/lvcreate operationstests/generic/459 +3
-3@@ -31,7 +31,7 @@ _cleanup() _unmount $SCRATCH_MNT >>$seqres.full 2>&1 $LVM_PROG vgremove -ff $vgname >>$seqres.full 2>&1 $LVM_PROG pvremove -ff $SCRATCH_DEV >>$seqres.full 2>&1 - $UDEV_SETTLE_PROG + _udev_wait --removed /dev/mapper/$vgname-$lvname } # Import common functions.@@ -88,8 +88,7 @@ $LVM_PROG lvcreate --thinpool $poolname --errorwhenfull y \ $LVM_PROG lvcreate --virtualsize $virtsize \ -T $vgname/$poolname \ -n $lvname >>$seqres.full 2>&1 - -$UDEV_SETTLE_PROG &>/dev/null +_udev_wait /dev/mapper/$vgname-$lvname _mkfs_dev /dev/mapper/$vgname-$lvname >>$seqres.full 2>&1 # Running the test over the original volume doesn't reproduce the problem@@ -97,6 +96,7 @@ _mkfs_dev /dev/mapper/$vgname-$lvname >>$seqres.full 2>&1 # reproducible, so, create a snapshot and run the test over it. $LVM_PROG lvcreate -k n -s $vgname/$lvname \ -n $snapname >>$seqres.full 2>&1 +_udev_wait /dev/mapper/$vgname-$snapname # Catch mount failure so we don't blindly go an freeze the root filesystem # instead of lvm volume.Signature · 1 line
2.45.2
Re: [PATCH 16/40] fstests: use udevadm wait in preference to settle
Quoted 34 lines · Darrick J. Wong
Quoted 33 lines · Dave Chinner
From: Dave Chinner <[email protected]>
When running lots of tests in parallel, there are lots of filesystems and block devices changing state. This generates a lot of udev events when means the udev event queue is rarely empty. Unfortunately, an empty event queue is what udev settling waits upon. Hence calling
UDEV_SETTLE_PROGcan mean waiting for a lot of time for other tests to stop generating udev events.For the majority of cases, what we care about is that udev has performed device node addition or removal, not that there are no udev events pending. Recent(-ish) systemd releases support 'udevadm wait' to wait for a specific file to be created or unlinked rather than waiting for the event that does that work to be completed.
Hence we don't have to wait for the udev event queue to empty, just for the udev event that does the device node manipulation to complete.
Introduce detection of 'udevadm wait' support and a
_udev_wait()wrapper function to use it if it is available. If it isn't, the use the existingUDEV_SETTLE_PROGbehaviour.Signed-off-by Dave Chinner <[email protected]>--- common/config | 35 +++++++++++++++++++++++++---------- common/rc | 25 ++++++++++++++++--------- tests/btrfs/291 | 5 +++-- tests/generic/081 | 6 +++--- tests/generic/108 | 7 +++---- tests/generic/459 | 6 +++--- 6 files changed, 53 insertions(+), 31 deletions(-)
Quoted 13 lines
common/config export XFS_MDRESTORE_PROG="$(type -P xfs_mdrestore)" · line 168
Quoted 10 lines
common/config +1
-1@@ -165,7 +165,7 @@ export XFS_MDRESTORE_PROG="$(type -P xfs_mdrestore)" export XFS_ADMIN_PROG="$(type -P xfs_admin)" export XFS_GROWFS_PROG=$(type -P xfs_growfs) export XFS_SPACEMAN_PROG="$(type -P xfs_spaceman)" -export XFS_SCRUB_PROG="$(type -P xfs_scrub)" +#export XFS_SCRUB_PROG="$(type -P xfs_scrub)"If you have problems with online fsck, please report them to the mailing list.
Thank you for finding this, Darrick, but there is absolutely no need to treat me like a n00b who doesn't know how kernel development works.
I simply forgot I did this when debugging the hundreds of busy-after-unmount failures that were occurring whilst trying to get this patchset to work.
I turned off scrub while I debugged these issues to reduce the amount of noise and variations in the end-of-test failures, and I simply never noticed that I had left this hunk in the patch.
So, nothing wrong with scrub, just a simple case of missing one line of debug changes in a patchset that touches 400+ files....
Removed.
-Dave.
Signature · 2 lines
Dave Chinner [email protected]
Re: [PATCH 16/40] fstests: use udevadm wait in preference to settle
Quoted 57 lines · Dave Chinner
Quoted 34 lines · Darrick J. Wong
Quoted 33 lines · Dave Chinner
From: Dave Chinner <[email protected]>
When running lots of tests in parallel, there are lots of filesystems and block devices changing state. This generates a lot of udev events when means the udev event queue is rarely empty. Unfortunately, an empty event queue is what udev settling waits upon. Hence calling
UDEV_SETTLE_PROGcan mean waiting for a lot of time for other tests to stop generating udev events.For the majority of cases, what we care about is that udev has performed device node addition or removal, not that there are no udev events pending. Recent(-ish) systemd releases support 'udevadm wait' to wait for a specific file to be created or unlinked rather than waiting for the event that does that work to be completed.
Hence we don't have to wait for the udev event queue to empty, just for the udev event that does the device node manipulation to complete.
Introduce detection of 'udevadm wait' support and a
_udev_wait()wrapper function to use it if it is available. If it isn't, the use the existingUDEV_SETTLE_PROGbehaviour.Signed-off-by Dave Chinner <[email protected]>--- common/config | 35 +++++++++++++++++++++++++---------- common/rc | 25 ++++++++++++++++--------- tests/btrfs/291 | 5 +++-- tests/generic/081 | 6 +++--- tests/generic/108 | 7 +++---- tests/generic/459 | 6 +++--- 6 files changed, 53 insertions(+), 31 deletions(-)Quoted 13 lines
common/config export XFS_MDRESTORE_PROG="$(type -P xfs_mdrestore)" · line 168
Quoted 10 lines
common/config +1
-1@@ -165,7 +165,7 @@ export XFS_MDRESTORE_PROG="$(type -P xfs_mdrestore)" export XFS_ADMIN_PROG="$(type -P xfs_admin)" export XFS_GROWFS_PROG=$(type -P xfs_growfs) export XFS_SPACEMAN_PROG="$(type -P xfs_spaceman)" -export XFS_SCRUB_PROG="$(type -P xfs_scrub)" +#export XFS_SCRUB_PROG="$(type -P xfs_scrub)"If you have problems with online fsck, please report them to the mailing list.
Thank you for finding this, Darrick, but there is absolutely no need to treat me like a n00b who doesn't know how kernel development works.
I simply forgot I did this when debugging the hundreds of busy-after-unmount failures that were occurring whilst trying to get this patchset to work.
Quoted 8 lines
I turned off scrub while I debugged these issues to reduce the amount of noise and variations in the end-of-test failures, and I simply never noticed that I had left this hunk in the patch.
So, nothing wrong with scrub, just a simple case of missing one line of debug changes in a patchset that touches 400+ files....
Removed.
Thanks Darrick and Dave, this's a big patchset, warm welcome more review/test from anyone :) I've prepared a "for-dave-check-parallel" branch which merged Dave's "check-parallel" branch onto fstests next release(this weekend), and fixed some conflict. I also removed above comment mark and re-pushed. Feel free to merge&update code or review&test on that, I'm testing on it too :)
Thanks,
Zorro
Quoted 6 lines
-Dave.
Signature · 2 lines
Dave Chinner [email protected]
PATCH 17/40 xfs/442: rescale load so it's not exponential
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
....
xfs/442 491
....
xfs/442 takes a long time to run because it is scaling the load by the number of processes it is going to run on twice. It scales the number of operations by the number of processes it is going to run, meaning that doubling the number of processes quadruples the runtime.
Reduce it to scale linearly by fixing the number of ops it runs per process.
--- tests/xfs/442 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tests/xfs/442 +1 -1
@@ -63,7 +63,7 @@ _scratch_mkfs_sized $((1600 * 1048576)) > $seqres.full 2>&1 _scratch_mount >> $seqres.full 2>&1 nr_cpus=$((LOAD_FACTOR * 4)) -nr_ops=$((25000 * nr_cpus * TIME_FACTOR)) +nr_ops=$((25000 * TIME_FACTOR)) _run_fsstress -w -d $SCRATCH_MNT -n $nr_ops -p $nr_cpus echo "Check quota before remount"
Signature · 1 line
2.45.2
PATCH 18/40 xfs/176: fix broken setup code
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
The test does not pass the mkfs output through the mkfs filter, so the inode size is not set up correctly. Hence it calculates the CHUNK_SIZE as 0, and it ends up getting stuck in an endless loop throwing ENOSPC errors because the offset never changes.
While there, use 'echo -n' rather than 'touch' to create zero length files much faster.
--- tests/xfs/176 | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tests/xfs/176 +2 -2
@@ -21,7 +21,7 @@ _require_scratch_xfs_shrink _require_xfs_io_command "falloc" _require_xfs_io_command "fpunch" -_scratch_mkfs "-d size=50m -m crc=1 -i sparse" | +_scratch_mkfs "-d size=50m -m crc=1 -i sparse" | \ _filter_mkfs > /dev/null 2> $tmp.mkfs . $tmp.mkfs # for isize cat $tmp.mkfs >> $seqres.full
@@ -69,7 +69,7 @@ _alloc_inodes() i=0 while [ true ]; do - touch $dir/$i 2>> $seqres.full || break + echo -n > $dir/$i >> $seqres.full 2>&1 || break i=$((i + 1)) done }
Signature · 1 line
2.45.2
PATCH 19/40 xfs/177: remove unused slab object count location checks
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
Stale code; we count XFS inodes through the sysfs stats code now so remove it.
--- tests/xfs/177 | 7 ------- 1 file changed, 7 deletions(-)
tests/xfs/177 +0 -7
@@ -58,13 +58,6 @@ dump_debug_info() { echo "round $1 baseline: $baseline_count high: $high_count fresh: $fresh_count post: $post_count end: $end_count" >> $seqres.full } -# Either of these need to be available to monitor slab usage -xfs_ino_objcount_file=/sys/kernel/slab/xfs_inode/objects -slabinfo_file=/proc/slabinfo -if [ ! -r "$xfs_ino_objcount_file" ] && [ ! -r "$slabinfo_file" ]; then - _notrun "Cannot find xfs_inode slab count?" -fi - # Background reclamation of disused xfs inodes is scheduled for ($xfssyncd / 6) # centiseconds after the first inode is tagged for reclamation. It's not great # to encode this implementation detail in a test like this, but there isn't
Signature · 1 line
2.45.2
PATCH 20/40 fstests: remove uses of killall where possible
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
there are many unnecessary uses of killall and stale checks for it's existence. Parallel check execution means killall is considered harmful, so get rid of these unneccesary uses.
--- common/config | 1 - common/fuzzy | 6 +++--- doc/requirement-checking.txt | 6 +++--- tests/btrfs/192 | 1 - tests/btrfs/212 | 2 -- tests/generic/270 | 1 - tests/generic/310 | 22 +++++++++++++++++----- tests/generic/751 | 1 - tests/overlay/058 | 12 ++++++++---- tests/xfs/011 | 1 - tests/xfs/013 | 1 - tests/xfs/051 | 1 - tests/xfs/057 | 1 - tests/xfs/070 | 9 +++++---- tests/xfs/079 | 1 - tests/xfs/141 | 1 - tests/xfs/167 | 3 --- tests/xfs/297 | 2 -- tests/xfs/442 | 1 - 19 files changed, 36 insertions(+), 37 deletions(-)
common/config +0 -1
@@ -182,7 +182,6 @@ export CHACL_PROG="$(type -P chacl)" export ATTR_PROG="$(type -P attr)" export QUOTA_PROG="$(type -P quota)" export XFS_QUOTA_PROG="$(type -P xfs_quota)" -export KILLALL_PROG="$(type -P killall)" export INDENT_PROG="$(type -P indent)" export XFS_COPY_PROG="$(type -P xfs_copy)" export FSTRIM_PROG="$(type -P fstrim)"
common/fuzzy +3 -3
@@ -1188,14 +1188,14 @@ _scratch_xfs_stress_scrub_cleanup() { # a race condition that can hang fstests. # # If the xfs_io -c freeze process is asleep waiting for a write lock on - # s_umount or sb_write when the killall signal is delivered, it will + # s_umount or sb_write when the kill signal is delivered, it will # not check for pending signals until after it has frozen the fs. If # even one thread of the stress test processes (xfs_io, fsstress, etc.) - # is waiting for read locks on sb_write when the killall signals are + # is waiting for read locks on sb_write when the kill signals are # delivered, they will block in the kernel until someone thaws the fs, # and the `wait' below will wait forever. # - # Hence we issue the killall, wait for the freezer loop to exit, thaw + # Hence we issue the kill, wait for the freezer loop to exit, thaw # the filesystem, and wait for the rest of the children. if [ -n "$__SCRUB_STRESS_FREEZE_PID" ]; then echo "Waiting for fs freezer $__SCRUB_STRESS_FREEZE_PID to exit at $(date)" >> $seqres.full
doc/requirement-checking.txt +3 -3
@@ -40,11 +40,11 @@ _require_command "$NAME_PROG" name should then be used to refer to the command when executing it. For example: - _require_command "KILLALL_PROG" killall + _require_command "$XFS_DB_PROG" "xfs_db" - to locate the killall command and then: + to locate the xfs_db command and then: - $KILLALL_PROG -q $FSSTRESS_PROG + $XFS_DB_PROG -c "sb 0" -c "print" $SCRATCH_DEV to make use of it.
tests/btrfs/192 +0 -1
@@ -27,7 +27,6 @@ _cleanup() . ./common/dmlogwrites -_require_command "$KILLALL_PROG" killall _require_command "$BLKDISCARD_PROG" blkdiscard _require_btrfs_fs_feature "no_holes" _require_btrfs_mkfs_feature "no-holes"
tests/btrfs/212 +0 -2
@@ -25,8 +25,6 @@ _cleanup() . ./common/filter _require_scratch -_require_command "$KILLALL_PROG" killall - _scratch_mkfs >> $seqres.full _scratch_mount
tests/generic/270 +0 -1
@@ -56,7 +56,6 @@ _workout() _require_quota _require_user _require_scratch -_require_command "$KILLALL_PROG" killall _require_command "$SETCAP_PROG" setcap _require_attrs security
tests/generic/310 +17 -5
@@ -29,14 +29,19 @@ _begin_fstest auto # Override the default cleanup function. _cleanup() { + if [ -n "$readdir_pid" ]; then + pkill --parent "$readdir_pid" t_readdir > /dev/null 2>&1 + kill -9 $readdir_pid > /dev/null 2>&1 + wait + fi rm -rf $TEST_DIR/tmp + rm -f $tmp.* } # Import common functions. . ./common/filter _require_test -_require_command "$KILLALL_PROG" killall dmesg -c > /dev/null
@@ -80,9 +85,12 @@ done _test_read() { - $here/src/t_readdir_1 $SEQ_DIR & + $here/src/t_readdir_1 $SEQ_DIR > /dev/null 2>&1 & + readdir_pid=$! sleep $RUN_TIME - $KILLALL_PROG t_readdir_1 + kill $readdir_pid + unset readdir_pid + wait check_kernel_bug if [ $? -ne 0 ]; then _fatal "kernel bug detected, check dmesg for more infomation."
@@ -91,9 +99,13 @@ _test_read() _test_lseek() { - $here/src/t_readdir_2 $SEQ_DIR & + $here/src/t_readdir_2 $SEQ_DIR > /dev/null 2>&1 & + readdir_pid=$! sleep $RUN_TIME - $KILLALL_PROG t_readdir_2 + kill $readdir_pid + unset readdir_pid + wait + check_kernel_bug if [ $? -ne 0 ]; then _fatal "kernel bug detected, check dmesg for more infomation."
tests/generic/751 +0 -1
@@ -80,7 +80,6 @@ fio_err=$tmp.fio.err _require_test _require_scratch _require_split_huge_pages_knob -_require_command "$KILLALL_PROG" "killall" _fixed_by_git_commit kernel 2a0774c2886d \ "XArray: set the marks correctly when splitting an entry"
tests/overlay/058 +8 -4
@@ -21,8 +21,10 @@ _begin_fstest auto quick exportfs # Override the default cleanup function. _cleanup() { - $KILLALL_PROG -9 open_by_handle >/dev/null 2>&1 - wait + if [ -n "$pids" ]; then + kill -9 $pids >/dev/null 2>&1 + wait + fi cd / rm -f $tmp.* }
@@ -36,7 +38,6 @@ _require_test_program "open_by_handle" # We need to require all features together, because nfs_export cannot # be enabled when index is disabled _require_scratch_overlay_features index nfs_export -_require_command "$KILLALL_PROG" killall # All overlay dirs are on scratch partition lower=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER
@@ -89,9 +90,11 @@ test_file_handles $SCRATCH_MNT -rnp -i $tmp.lower_file_handles # overlay dentries in cache _scratch_cycle_mount "index=on,nfs_export=on" test_file_handles $SCRATCH_MNT -rnps -i $tmp.upper_file_handles & +pids=$! # Give the above 1 second to get to sleep loop sleep 1 test_file_handles $SCRATCH_MNT -rnps -i $tmp.lower_file_handles & +pids="$pids $!" # Give the above 1 second to get to sleep loop sleep 1
@@ -105,8 +108,9 @@ test_file_handles $SCRATCH_MNT -rnp -i $tmp.upper_file_handles test_file_handles $SCRATCH_MNT -rnp -i $tmp.lower_file_handles # SIGPIPE avoids Terminated/Killed message from bash -$KILLALL_PROG -q -13 open_by_handle +kill -13 $pids > /dev/null 2>&1 wait +unset pids status=0 exit
tests/xfs/011 +0 -1
@@ -14,7 +14,6 @@ _begin_fstest auto freeze log metadata quick _require_scratch _require_freeze _require_xfs_sysfs $(_short_dev $TEST_DEV)/log -_require_command "$KILLALL_PROG" killall . ./common/filter
tests/xfs/013 +0 -1
@@ -74,7 +74,6 @@ _cleaner() _require_scratch _require_xfs_mkfs_finobt _require_xfs_finobt -_require_command "$KILLALL_PROG" killall _scratch_mkfs_xfs "-m crc=1,finobt=1 -d agcount=2" | \ _filter_mkfs 2>> $seqres.full
tests/xfs/051 +0 -1
@@ -19,7 +19,6 @@ _begin_fstest shutdown auto log metadata _require_scratch _require_dm_target flakey _require_xfs_sysfs debug/log_recovery_delay -_require_command "$KILLALL_PROG" killall echo "Silence is golden."
tests/xfs/057 +0 -1
@@ -42,7 +42,6 @@ _cleanup() _require_xfs_io_error_injection log_item_pin _require_xfs_io_error_injection log_bad_crc _require_scratch -_require_command "$KILLALL_PROG" killall echo "Silence is golden."
tests/xfs/070 +5 -4
@@ -24,10 +24,10 @@ _begin_fstest auto quick repair # Override the default cleanup function. _cleanup() { + [ -n "$repair_pid" ] && kill -9 $repair_pid > /dev/null 2>&1 + wait > /dev/null 2>&1 cd / rm -f $tmp.* - $KILLALL_PROG -9 $XFS_REPAIR_PROG > /dev/null 2>&1 - wait > /dev/null 2>&1 } # Start and monitor an xfs_repair of the scratch device. This test can induce a
@@ -45,7 +45,8 @@ _xfs_repair_noscan() repair_pid=$! # monitor progress for as long as it is running - while [ `pgrep xfs_repair` ]; do + running=`pgrep xfs_repair` + while [ -n "$running" ]; do grep "couldn't verify primary superblock" $tmp.repair \ > /dev/null 2>&1 if [ $? == 0 ]; then
@@ -58,6 +59,7 @@ _xfs_repair_noscan() fi sleep 1 + running=`pgrep xfs_repair` done wait
@@ -72,7 +74,6 @@ _xfs_repair_noscan() # Modify as appropriate. _require_scratch_nocheck -_require_command "$KILLALL_PROG" killall _scratch_mkfs | _filter_mkfs > /dev/null 2> $tmp.mkfs
tests/xfs/079 +0 -1
@@ -23,7 +23,6 @@ _begin_fstest shutdown auto log quick # Modify as appropriate. _require_scratch _require_v2log -_require_command "$KILLALL_PROG" killall echo "Silence is golden."
tests/xfs/141 +0 -1
@@ -20,7 +20,6 @@ _begin_fstest auto log metadata # Modify as appropriate. _require_xfs_io_error_injection "log_bad_crc" _require_scratch -_require_command "$KILLALL_PROG" killall echo "Silence is golden."
tests/xfs/167 +0 -3
@@ -21,10 +21,7 @@ workout() # Import common functions. . ./common/filter - -_require_command "$KILLALL_PROG" killall _require_xfs_io_command "falloc" - _require_scratch _scratch_mkfs_xfs >/dev/null 2>&1 _scratch_mount
tests/xfs/297 +0 -2
@@ -24,10 +24,8 @@ _cleanup() # Import common functions. . ./common/filter - _require_scratch _require_freeze -_require_command "$KILLALL_PROG" killall logblks=$(_scratch_find_xfs_min_logblocks -d agcount=16,su=256k,sw=12 -l su=256k) _scratch_mkfs_xfs -d agcount=16,su=256k,sw=12 -l su=256k,size=${logblks}b >/dev/null 2>&1
tests/xfs/442 +0 -1
@@ -21,7 +21,6 @@ _begin_fstest auto stress clone quota _require_scratch_reflink _require_quota -_require_command "$KILLALL_PROG" "killall" report_quota_blocks() { $XFS_QUOTA_PROG -x -c "report $1" $SCRATCH_MNT | \
Signature · 1 line
2.45.2
PATCH 21/40 generic/127: reduce runtime
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
...
generic/127 684
...
This takes a long time to run because it runs 6 individual invocations of fsx sequentially. Make them run concurrently as they can operate on separate files.
...
generic/127 168
...
--- tests/generic/127 | 67 ++++++++++++++++++++++++++----------------- tests/generic/127.out | 6 ---- 2 files changed, 41 insertions(+), 32 deletions(-)
tests/generic/127 +41 -26
@@ -16,7 +16,10 @@ _begin_fstest rw auto # Override the default cleanup function. _cleanup() { - rm -f $tmp.output + [ -n "$fsx_pids" ] && kill $fsx_pids 2> /dev/null + wait + cd / + rm -f $tmp.* } # Import common functions.
@@ -27,7 +30,6 @@ FSX_ARGS="-q -l $FSX_FILE_SIZE -o 65536 -S 191110531 -N 100000" _fsx_lite_nommap() { - echo "=== FSX Light Mode, No Memory Mapping ===" dd if=/dev/zero of=$TEST_DIR/fsx_lite_nommap bs=${FSX_FILE_SIZE} count=1 > /dev/null 2>&1 if ! ltp/fsx $FSX_ARGS -L -R -W $FSX_AVOID $TEST_DIR/fsx_lite_nommap > $tmp.output 2>&1 then
@@ -41,7 +43,6 @@ _fsx_lite_nommap() _fsx_lite_mmap() { - echo "=== FSX Light Mode, Memory Mapping ===" dd if=/dev/zero of=$TEST_DIR/fsx_lite_mmap bs=${FSX_FILE_SIZE} count=1 > /dev/null 2>&1 if ! ltp/fsx $FSX_ARGS -L $FSX_AVOID $TEST_DIR/fsx_lite_mmap > $tmp.output 2>&1 then
@@ -55,42 +56,56 @@ _fsx_lite_mmap() _fsx_std_nommap() { - echo "=== FSX Standard Mode, No Memory Mapping ===" - if ! ltp/fsx $FSX_ARGS -R -W $FSX_AVOID $TEST_DIR/fsx_std_nommap > $tmp.output 2>&1 - then - echo "ltp/fsx $FSX_ARGS -R -W fsx_std_nommap" - cat $tmp.output - return 1 - fi - cat $tmp.output - return 0 + local fname="$TEST_DIR/$1" + + if ! ltp/fsx $FSX_ARGS -R -W $FSX_AVOID $fname > $tmp.output 2>&1 + then + echo "ltp/fsx $FSX_ARGS -R -W fsx_std_nommap" + cat $tmp.output + return 1 + fi + cat $tmp.output + return 0 } _fsx_std_mmap() { - echo "=== FSX Standard Mode, Memory Mapping ===" - if ! ltp/fsx $FSX_ARGS $FSX_AVOID $TEST_DIR/fsx_std_mmap > $tmp.output 2>&1 - then - echo "ltp/fsx $FSX_ARGS fsx_std_mmap" + local fname="$TEST_DIR/$1" + + if ! ltp/fsx $FSX_ARGS $FSX_AVOID $fname > $tmp.output 2>&1 + then + echo "ltp/fsx $FSX_ARGS fsx_std_mmap" + cat $tmp.output + return 1 + fi cat $tmp.output - return 1 - fi - cat $tmp.output - return 0 + return 0 } _require_test -_fsx_lite_nommap || exit -_fsx_lite_mmap || exit +_fsx_lite_nommap & +fsx_pids=$! -_fsx_std_nommap || exit -_fsx_std_mmap || exit +_fsx_lite_mmap & +fsx_pids="$! $fsx_pids" + +_fsx_std_nommap fsx_std_nommap & +fsx_pids="$! $fsx_pids" + +_fsx_std_mmap fsx_std_mmap & +fsx_pids="$! $fsx_pids" #flush cache after write FSX_ARGS="-f $FSX_ARGS" -_fsx_std_nommap || exit -_fsx_std_mmap || exit +_fsx_std_nommap fsx_std_nommap_flush & +fsx_pids="$! $fsx_pids" + +_fsx_std_mmap fsx_std_mmap_flush & +fsx_pids="$! $fsx_pids" + +wait $fsx_pids +unset fsx_pids status=0 _cleanup
tests/generic/127.out +0 -6
@@ -1,13 +1,7 @@ QA output created by 127 -=== FSX Light Mode, No Memory Mapping === All 100000 operations completed A-OK! -=== FSX Light Mode, Memory Mapping === All 100000 operations completed A-OK! -=== FSX Standard Mode, No Memory Mapping === All 100000 operations completed A-OK! -=== FSX Standard Mode, Memory Mapping === All 100000 operations completed A-OK! -=== FSX Standard Mode, No Memory Mapping === All 100000 operations completed A-OK! -=== FSX Standard Mode, Memory Mapping === All 100000 operations completed A-OK!
Signature · 1 line
2.45.2
PATCH 22/40 quota: system project quota files need to be shared
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
Tests that treat them as exclusively owned end up tripping over other tests that do the same. Fix this by using append and filter techniques to update the files, then using different project quota ids for each test.
--- common/quota | 35 +++++++---------------------------- tests/generic/603 | 8 +++++--- tests/generic/691 | 6 ++++-- 3 files changed, 16 insertions(+), 33 deletions(-)
common/quota +7 -28
@@ -274,7 +274,7 @@ _choose_prid() _qmount() { - _scratch_unmount >/dev/null 2>&1 + _scratch_unmount _try_scratch_mount || _fail "qmount failed" # xfs doesn't need these setups and quotacheck even fails on xfs # redirect the output to $seqres.full for debug purpose and ignore results
@@ -432,39 +432,18 @@ _create_project_quota() mkdir $prjdir chmod ugo+rwx $prjdir - if [ -f /etc/projects -a ! -f $tmp.projects.bk ];then - cat /etc/projects > $tmp.projects.bk - echo >/etc/projects - fi - if [ -f /etc/projid -a ! -f $tmp.projid.bk ];then - cat /etc/projid > $tmp.projid.bk - echo >/etc/projid - fi - - cat >>/etc/projects <<EOF -$id:$prjdir -EOF - cat >>/etc/projid <<EOF -$name:$id -EOF + echo "$id:$prjdir" >> /etc/project + echo "$name:$id" >> /etc/projid $XFS_IO_PROG -r -c "chproj $id" -c "chattr +P" $prjdir } # If you've called _create_project_quota, then use this function in _cleanup _restore_project_quota() { - if [ -f $tmp.projects.bk ];then - cat $tmp.projects.bk > /etc/projects && \ - rm -f $tmp.projects.bk - else - rm -f /etc/projects - fi - if [ -f $tmp.projid.bk ];then - cat $tmp.projid.bk > /etc/projid && \ - rm -f $tmp.projid.bk - else - rm -f /etc/projid - fi + local id=$1 + + sed -i "/^$id:/d" /etc/project + sed -i "/:$id$/d" /etc/projid } # make sure this script returns success
tests/generic/603 +5 -3
@@ -10,10 +10,12 @@ . ./common/preamble _begin_fstest auto quick quota +projid=$seq + # Override the default cleanup function. _cleanup() { - _restore_project_quota + _restore_project_quota $projid cd / rm -f $tmp.* }
@@ -122,8 +124,8 @@ _require_prjquota $SCRATCH_DEV BLOCK_SIZE=$(_get_file_block_size $SCRATCH_MNT) rm -rf $SCRATCH_MNT/t mkdir $SCRATCH_MNT/t -$XFS_IO_PROG -r -c "chproj 100" -c "chattr +P" $SCRATCH_MNT/t -_create_project_quota $SCRATCH_MNT/t 100 $qa_user +$XFS_IO_PROG -r -c "chproj $projid" -c "chattr +P" $SCRATCH_MNT/t +_create_project_quota $SCRATCH_MNT/t $projid $qa_user echo "### Set up different grace timers to each type of quota" UBGRACE=12
tests/generic/691 +4 -2
@@ -16,10 +16,12 @@ . ./common/preamble _begin_fstest auto quota +projid=$seq + # Override the default cleanup function. _cleanup() { - _restore_project_quota + _restore_project_quota $projid cd / rm -r -f $tmp.* }
@@ -65,7 +67,7 @@ exercise() fi _qmount if [ "$type" = "P" ];then - _create_project_quota $SCRATCH_MNT/t 100 $qa_user + _create_project_quota $SCRATCH_MNT/t $projid $qa_user file=$SCRATCH_MNT/t/testfile fi
Signature · 1 line
2.45.2
PATCH 23/40 dmesg: reduce noise from other tests
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
dmesg records everything from every test concurrently running, so noise from other tests can cause multiple other tests to fail because they detect something from another test. Update the filter behaviour to minimise this crosstalk problem.
`
--- common/rc | 13 +++++++------ tests/generic/310 | 5 +---- tests/xfs/079 | 2 +- 3 files changed, 9 insertions(+), 11 deletions(-)
common/rc +7 -6
@@ -4591,6 +4591,7 @@ _check_dmesg_filter() grep -E -v -e "BUG: MAX_LOCKDEP_CHAIN_HLOCKS too low" \ -e "BUG: MAX_STACK_TRACE_ENTRIES too low" \ + -e "WARNING: Reset corrupted AGFL" \ $extra_filter }
@@ -4613,18 +4614,18 @@ _check_dmesg() fi rm -f ${RESULT_DIR}/check_dmesg - # default filter is a simple cat command, caller could provide a - # customized filter and pass the name through the first argument, to - # filter out intentional WARNINGs or Oopses - local filter=${1:-_check_dmesg_filter} + # we always filter dmesg for global things that should not cause + # test failures. Custom filters are then applied afterwards. + _dmesg_since_test_start | _check_dmesg_filter > $seqres.dmesg + if [ -n "$1" ]; then + cat $seqres.dmesg | $1 > $seqres.dmesg + fi - _dmesg_since_test_start | $filter >$seqres.dmesg grep -E -q -e "kernel BUG at" \ -e "WARNING:" \ -e "\bBUG:" \ -e "Oops:" \ -e "possible recursive locking detected" \ - -e "Internal error" \ -e "(INFO|ERR): suspicious RCU usage" \ -e "INFO: possible circular locking dependency detected" \ -e "general protection fault:" \
tests/generic/310 +1 -4
@@ -46,7 +46,6 @@ _require_test dmesg -c > /dev/null nr_bug=`dmesg | grep -c "kernel BUG"` -nr_error=`dmesg | grep -wc "error"` nr_null=`dmesg | grep -c "kernel NULL pointer dereference"` nr_warning=`dmesg | grep -c "^WARNING"` nr_lockdep=`dmesg | grep -c "possible recursive locking detected"`
@@ -55,20 +54,18 @@ nr_lockdep=`dmesg | grep -c "possible recursive locking detected"` check_kernel_bug() { new_bug=`dmesg | grep -c "kernel BUG"` - new_error=`dmesg | grep -wc "error"` new_null=`dmesg | grep -c "kernel NULL pointer dereference"` new_warning=`dmesg | grep -c "^WARNING"` new_lockdep=`dmesg | grep -c "possible recursive locking detected"` # no kernel bug is detected - if [ $new_bug -eq $nr_bug -a $new_error -eq $nr_error -a \ + if [ $new_bug -eq $nr_bug -a \ $new_null -eq $nr_null -a $new_warning -eq $nr_warning -a \ $new_lockdep -eq $nr_lockdep ]; then return 0 fi nr_bug=$new_bug - nr_error=$new_error nr_null=$new_null nr_warning=$new_warning nr_lockdep=$new_lockdep
tests/xfs/079 +1 -1
@@ -45,7 +45,7 @@ _try_scratch_mount "-o logbsize=64k" # The mount may not fail on v4 filesystems. Check for CRC mismatch warning # messages to detect failure in this case. -dmesg -c | grep XFS | grep CRC +_dmesg_since_test_start | grep "XFS $(_short_dev $SCRATCH_DEV)" | grep CRC # success, all done status=0
Signature · 1 line
2.45.2
PATCH 24/40 fstests: stop using /tmp directly
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
Tests should be using $tmp, not /tmp. this causes problems when multiple tests all use /tmp/foo as a temporary test state file and then step on each other.
Note that there are some tests that use /tmp to store "test stop" files for background processes. Those that have proven to be unreliable at stopping tests when interrupted by ctrl-c are also updated to track and kill background processes in the cleanup function.
--- tests/generic/162 | 4 ++-- tests/generic/163 | 4 ++-- tests/generic/164 | 4 ++-- tests/generic/165 | 4 ++-- tests/generic/166 | 6 +++--- tests/generic/167 | 4 ++-- tests/generic/168 | 7 +++++-- tests/generic/170 | 7 +++++-- tests/generic/333 | 6 +++--- tests/generic/334 | 6 +++--- tests/generic/373 | 2 +- tests/generic/374 | 2 +- tests/generic/670 | 2 +- tests/generic/671 | 2 +- tests/generic/672 | 2 +- tests/generic/722 | 15 ++++----------- tests/xfs/601 | 2 +- 17 files changed, 39 insertions(+), 40 deletions(-)
tests/generic/162 +2 -2
@@ -15,7 +15,7 @@ _register_cleanup "_cleanup" BUS _cleanup() { cd / - rm -rf $tmp.* $finished_file + rm -rf $tmp.* wait }
@@ -30,7 +30,7 @@ _scratch_mkfs > $seqres.full 2>&1 _scratch_mount >> $seqres.full 2>&1 testdir=$SCRATCH_MNT/test-$seq -finished_file=/tmp/finished +finished_file=$tmp.finished rm -rf $finished_file mkdir $testdir
tests/generic/163 +2 -2
@@ -15,7 +15,7 @@ _register_cleanup "_cleanup" BUS _cleanup() { cd / - rm -rf $tmp.* $finished_file + rm -rf $tmp.* wait }
@@ -30,7 +30,7 @@ _scratch_mkfs > $seqres.full 2>&1 _scratch_mount >> $seqres.full 2>&1 testdir=$SCRATCH_MNT/test-$seq -finished_file=/tmp/finished +finished_file=$tmp.finished rm -rf $finished_file mkdir $testdir
tests/generic/164 +2 -2
@@ -16,7 +16,7 @@ _register_cleanup "_cleanup" BUS _cleanup() { cd / - rm -rf $tmp.* $finished_file + rm -rf $tmp.* wait }
@@ -32,7 +32,7 @@ _scratch_mkfs > $seqres.full 2>&1 _scratch_mount >> $seqres.full 2>&1 testdir=$SCRATCH_MNT/test-$seq -finished_file=/tmp/finished +finished_file=$tmp.finished rm -rf $finished_file mkdir $testdir
tests/generic/165 +2 -2
@@ -16,7 +16,7 @@ _register_cleanup "_cleanup" BUS _cleanup() { cd / - rm -rf $tmp.* $finished_file + rm -rf $tmp.* wait }
@@ -33,7 +33,7 @@ _scratch_mkfs > $seqres.full 2>&1 _scratch_mount >> $seqres.full 2>&1 testdir=$SCRATCH_MNT/test-$seq -finished_file=/tmp/finished +finished_file=$tmp.finished rm -rf $finished_file mkdir $testdir
tests/generic/166 +3 -3
@@ -16,7 +16,7 @@ _register_cleanup "_cleanup" BUS _cleanup() { cd / - rm -rf $tmp.* $finished_file + rm -rf $tmp.* wait }
@@ -33,8 +33,8 @@ _scratch_mkfs > $seqres.full 2>&1 _scratch_mount >> $seqres.full 2>&1 testdir=$SCRATCH_MNT/test-$seq -finished_file=/tmp/finished -do_snapshot=/tmp/snapshot +finished_file=$tmp.finished +do_snapshot=$tmp.snapshot rm -rf $finished_file mkdir $testdir
tests/generic/167 +2 -2
@@ -16,7 +16,7 @@ _register_cleanup "_cleanup" BUS _cleanup() { cd / - rm -rf $tmp.* $finished_file + rm -rf $tmp.* wait }
@@ -32,7 +32,7 @@ _scratch_mkfs > $seqres.full 2>&1 _scratch_mount >> $seqres.full 2>&1 testdir=$SCRATCH_MNT/test-$seq -finished_file=/tmp/finished +finished_file=$tmp.finished rm -rf $finished_file mkdir $testdir
tests/generic/168 +5 -2
@@ -16,7 +16,8 @@ _register_cleanup "_cleanup" BUS _cleanup() { cd / - rm -rf $tmp.* $finished_file + rm -rf $tmp.* + [ -n "$write_pid" ] && kill -9 $write_pid > /dev/null 2>&1 wait }
@@ -31,7 +32,7 @@ _scratch_mkfs > $seqres.full 2>&1 _scratch_mount >> $seqres.full 2>&1 testdir=$SCRATCH_MNT/test-$seq -finished_file=/tmp/finished +finished_file=$tmp.finished rm -rf $finished_file mkdir $testdir
@@ -57,6 +58,7 @@ overwrite() { echo "Reflink and write the target" overwrite & +write_pid=$! seq 1 10 | while read j; do seq 0 $nr_loops | while read i; do _reflink_range $testdir/file1 $((i * blksz)) \
@@ -66,6 +68,7 @@ seq 1 10 | while read j; do done touch $finished_file wait +unset write_pid # success, all done status=0
tests/generic/170 +5 -2
@@ -16,7 +16,8 @@ _register_cleanup "_cleanup" BUS _cleanup() { cd / - rm -rf $tmp.* $finished_file + rm -rf $tmp.* + [ -n "$write_pid" ] && kill -9 $write_pid > /dev/null 2>&1 wait }
@@ -32,7 +33,7 @@ _scratch_mkfs > $seqres.full 2>&1 _scratch_mount >> $seqres.full 2>&1 testdir=$SCRATCH_MNT/test-$seq -finished_file=/tmp/finished +finished_file=$tmp.finished rm -rf $finished_file mkdir $testdir
@@ -58,6 +59,7 @@ overwrite() { echo "Reflink and dio write the target" overwrite & +write_pid=$! seq 1 10 | while read j; do seq 0 $nr_loops | while read i; do _reflink_range $testdir/file1 $((i * blksz)) \
@@ -67,6 +69,7 @@ seq 1 10 | while read j; do done touch $finished_file wait +unset write_pid # success, all done status=0
tests/generic/333 +3 -3
@@ -16,7 +16,7 @@ _register_cleanup "_cleanup" BUS _cleanup() { cd / - rm -rf $tmp.* $finished_file $abort_file + rm -rf $tmp.* wait }
@@ -33,9 +33,9 @@ _scratch_mkfs_sized $((400 * 1048576)) > $seqres.full 2>&1 _scratch_mount >> $seqres.full 2>&1 testdir=$SCRATCH_MNT/test-$seq -finished_file=/tmp/finished +finished_file=$tmp.finished rm -rf $finished_file -abort_file=/tmp/abort +abort_file=$tmp.abort rm -rf $abort_file mkdir $testdir
tests/generic/334 +3 -3
@@ -16,7 +16,7 @@ _register_cleanup "_cleanup" BUS _cleanup() { cd / - rm -rf $tmp.* $finished_file $abort_file + rm -rf $tmp.* wait }
@@ -32,9 +32,9 @@ _scratch_mkfs_sized $((400 * 1048576)) > $seqres.full 2>&1 _scratch_mount >> $seqres.full 2>&1 testdir=$SCRATCH_MNT/test-$seq -finished_file=/tmp/finished +finished_file=$tmp.finished rm -rf $finished_file -abort_file=/tmp/abort +abort_file=$tmp.abort rm -rf $abort_file mkdir $testdir
tests/generic/373 +1 -1
@@ -32,7 +32,7 @@ _scratch_mount >> $seqres.full 2>&1 testdir=$SCRATCH_MNT/test-$seq mkdir $testdir -otherdir=/tmp/m.$seq +otherdir=$tmp.m.$seq othertestdir=$otherdir/test-$seq rm -rf $otherdir mkdir $otherdir
tests/generic/374 +1 -1
@@ -31,7 +31,7 @@ _scratch_mount >> $seqres.full 2>&1 testdir=$SCRATCH_MNT/test-$seq mkdir $testdir -otherdir=/tmp/m.$seq +otherdir=$tmp.m.$seq othertestdir=$otherdir/test-$seq rm -rf $otherdir mkdir $otherdir
tests/generic/670 +1 -1
@@ -23,7 +23,7 @@ _scratch_mkfs > $seqres.full 2>&1 _scratch_mount >> $seqres.full 2>&1 testdir=$SCRATCH_MNT/test-$seq -finished_file=/tmp/finished +finished_file=$tmp.finished rm -rf $finished_file mkdir $testdir
tests/generic/671 +1 -1
@@ -23,7 +23,7 @@ _scratch_mkfs > $seqres.full 2>&1 _scratch_mount >> $seqres.full 2>&1 testdir=$SCRATCH_MNT/test-$seq -finished_file=/tmp/finished +finished_file=$tmp.finished rm -rf $finished_file mkdir $testdir
tests/generic/672 +1 -1
@@ -22,7 +22,7 @@ _scratch_mkfs > $seqres.full 2>&1 _scratch_mount >> $seqres.full 2>&1 testdir=$SCRATCH_MNT/test-$seq -finished_file=/tmp/finished +finished_file=$tmp.finished rm -rf $finished_file mkdir $testdir
tests/generic/722 +4 -11
@@ -10,13 +10,6 @@ . ./common/preamble _begin_fstest auto quick fiexchange -# Override the default cleanup function. -_cleanup() -{ - cd / - rm -r -f $tmp.* $dir -} - # Import common functions. . ./common/filter . ./common/reflink
@@ -38,8 +31,8 @@ old_a=$(md5sum $SCRATCH_MNT/a | awk '{print $1}') old_b=$(md5sum $SCRATCH_MNT/b | awk '{print $1}') echo "md5 a: $old_a md5 b: $old_b" >> $seqres.full -od -tx1 -Ad -c $SCRATCH_MNT/a > /tmp/a0 -od -tx1 -Ad -c $SCRATCH_MNT/b > /tmp/b0 +od -tx1 -Ad -c $SCRATCH_MNT/a > $tmp.a0 +od -tx1 -Ad -c $SCRATCH_MNT/b > $tmp.b0 echo swap >> $seqres.full $XFS_IO_PROG -c "exchangerange -f $SCRATCH_MNT/a" $SCRATCH_MNT/b
@@ -53,8 +46,8 @@ echo "md5 a: $new_a md5 b: $new_b" >> $seqres.full test $old_a = $new_b || echo "scratch file B doesn't match old file A" test $old_b = $new_a || echo "scratch file A doesn't match old file B" -od -tx1 -Ad -c $SCRATCH_MNT/a > /tmp/a1 -od -tx1 -Ad -c $SCRATCH_MNT/b > /tmp/b1 +od -tx1 -Ad -c $SCRATCH_MNT/a > $tmp.a1 +od -tx1 -Ad -c $SCRATCH_MNT/b > $tmp.b1 # success, all done echo Silence is golden
tests/xfs/601 +1 -1
@@ -15,7 +15,7 @@ _register_cleanup "_cleanup" BUS _cleanup() { cd / - rm -rf $tmp.* $testdir + rm -rf $tmp.* } # Import common functions.
Signature · 1 line
2.45.2
PATCH 25/40 fstests: scale some tests for high CPU count sanity
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
Several tests use lots of processes to stress the filesystem. many of them haven't really considered what this means for running the test on high CPU machines (e.g. >32p) and the potential contention and performance issues this might trigger.
Some of these tests simply need to increase the size of the journal. Some need to run on filesystems with high inherent concurrency (e.g. larger AG count). Some need more efficient/faster file creation. And so on.
This commit is a collection of those sorts of changes to improve runtimes on high CPU count machines.
--- src/aio-dio-regress/aio-last-ref-held-by-io.c | 5 ++++- tests/generic/251 | 5 ++++- tests/generic/323 | 7 +++++-- tests/generic/530 | 2 +- tests/generic/531 | 8 +++++++- tests/xfs/013 | 4 ++-- tests/xfs/076 | 6 +++--- tests/xfs/176 | 6 +++--- tests/xfs/297 | 4 +++- tests/xfs/501 | 2 +- tests/xfs/502 | 2 +- 11 files changed, 34 insertions(+), 17 deletions(-)
src/aio-dio-regress/aio-last-ref-held-by-io.c +4 -1
@@ -85,11 +85,14 @@ aio_test_thread(void *data)/* * Problems have been easier to trigger when spreading the * workload over the available CPUs. +* +* If CPU hotplug is active, this can randomly fail so dump the error +* to stderror so it can be filtered out easily by the caller. */ CPU_ZERO (&cpuset);CPU_SET (mycpu, &cpuset);if (sched_setaffinity (mytid,sizeof (cpuset), &cpuset)) { -printf ("FAILED to set thread %d to run on cpu %ld\n" , +fprintf (stderr,"FAILED to set thread %d to run on cpu %ld\n" , mytid, mycpu); }
tests/generic/251 +4 -1
@@ -175,9 +175,12 @@ nproc=20 # Copy $here to the scratch fs and make coipes of the replica. The fstests # output (and hence $seqres.full) could be in $here, so we need to snapshot # $here before computing file checksums. +# +# $here/* as the files to copy so we avoid any .git directory that might be +# much, much larger than the rest of the fstests source tree we are copying. content=$SCRATCH_MNT/orig mkdir -p $content -cp -axT $here/ $content/ +cp -ax $here/* $content/ mkdir -p $tmp
tests/generic/323 +5 -2
@@ -23,12 +23,15 @@ _require_aiodio aio-last-ref-held-by-io testfile=$TEST_DIR/aio-testfile $XFS_IO_PROG -ftc "pwrite 0 10m" $testfile | _filter_xfs_io -$AIO_TEST 0 100 $testfile +# This can emit cpu affinity setting failures that aren't considered test +# failures but cause golden image failures. Redirect the test output to +# $seqres.full so that it is captured but doesn't directly cause test failures. +$AIO_TEST 0 100 $testfile 2>> $seqres.full if [ $? -ne 0 ]; then exit $status fi -$AIO_TEST 1 100 $testfile +$AIO_TEST 1 100 $testfile 2>> $seqres.full if [ $? -ne 0 ]; then exit $status fi
tests/generic/530 +1 -1
@@ -22,7 +22,7 @@ _require_scratch_shutdown _require_metadata_journaling _require_test_program "t_open_tmpfiles" -_scratch_mkfs >> $seqres.full 2>&1 +_scratch_mkfs "-l size=256m" >> $seqres.full 2>&1 _scratch_mount # Set ULIMIT_NOFILE to min(file-max / 2, 50000 files per LOAD_FACTOR)
tests/generic/531 +7 -1
@@ -21,7 +21,13 @@ _require_scratch _require_xfs_io_command "-T" _require_test_program "t_open_tmpfiles" -_scratch_mkfs >> $seqres.full 2>&1 +# On high CPU count machines, this runs a -lot- of create and unlink +# concurrency. Set the filesytsem up to handle this. +if [ $FSTYP = "xfs" ]; then + _scratch_mkfs "-d agcount=32" >> $seqres.full 2>&1 +else + _scratch_mkfs >> $seqres.full 2>&1 +fi _scratch_mount # Try to load up all the CPUs, two threads per CPU.
tests/xfs/013 +2 -2
@@ -28,7 +28,7 @@ _create() mkdir -p $dir for i in $(seq 0 $count) do - touch $dir/$i 2>&1 | filter_enospc + echo -n > $dir/$i 2>&1 | filter_enospc done }
@@ -42,7 +42,7 @@ _rand_replace() do file=$((RANDOM % count)) rm -f $dir/$file - touch $dir/$file 2>&1 | filter_enospc + echo -n > $dir/$file 2>&1 | filter_enospc done }
tests/xfs/076 +3 -3
@@ -47,10 +47,10 @@ _alloc_inodes() dir=$1 i=0 - while [ true ]; do - touch $dir/$i 2>> $seqres.full || break + ( while [ true ]; do + echo -n > $dir/$i || break i=$((i + 1)) - done + done ) >> $seqres.full 2>&1 }
tests/xfs/176 +3 -3
@@ -68,10 +68,10 @@ _alloc_inodes() dir=$1 i=0 - while [ true ]; do - echo -n > $dir/$i >> $seqres.full 2>&1 || break + ( while [ true ]; do + echo -n > $dir/$i || break i=$((i + 1)) - done + done ) >> $seqres.full 2>&1 } # Find a sparse inode cluster after logend_agno/logend_agino.
tests/xfs/297 +3 -1
@@ -34,7 +34,9 @@ _scratch_mount STRESS_DIR="$SCRATCH_MNT/testdir" mkdir -p $STRESS_DIR -_run_fsstress_bg -d $STRESS_DIR -n 1000 -p 1000 $FSSTRESS_AVOID +# turn off sync as this can lead to near deadlock conditions due to every +# fsstress process lockstepping against freeze on large CPU count machines +_run_fsstress_bg -d $STRESS_DIR -f sync=0 -n 1000 -p 1000 $FSSTRESS_AVOID # Freeze/unfreeze file system randomly echo "Start freeze/unfreeze randomly" | tee -a $seqres.full
tests/xfs/501 +1 -1
@@ -33,7 +33,7 @@ _require_xfs_sysfs debug/log_recovery_delay _require_scratch _require_test_program "t_open_tmpfiles" -_scratch_mkfs >> $seqres.full 2>&1 +_scratch_mkfs "-l size=256m" >> $seqres.full 2>&1 _scratch_mount # Set ULIMIT_NOFILE to min(file-max / 2, 30000 files per LOAD_FACTOR)
tests/xfs/502 +1 -1
@@ -23,7 +23,7 @@ _require_xfs_io_error_injection "iunlink_fallback" _require_scratch _require_test_program "t_open_tmpfiles" -_scratch_mkfs | _filter_mkfs 2> $tmp.mkfs > /dev/null +_scratch_mkfs "-l size=256m" | _filter_mkfs 2> $tmp.mkfs > /dev/null cat $tmp.mkfs >> $seqres.full . $tmp.mkfs
Signature · 1 line
2.45.2
Re: [PATCH 25/40] fstests: scale some tests for high CPU count sanity
Quoted 55 lines · Dave Chinner
From: Dave Chinner <[email protected]>
Several tests use lots of processes to stress the filesystem. many of them haven't really considered what this means for running the test on high CPU machines (e.g. >32p) and the potential contention and performance issues this might trigger.
Some of these tests simply need to increase the size of the journal. Some need to run on filesystems with high inherent concurrency (e.g. larger AG count). Some need more efficient/faster file creation. And so on.
This commit is a collection of those sorts of changes to improve runtimes on high CPU count machines.
Signed-off-by Dave Chinner <[email protected]>--- src/aio-dio-regress/aio-last-ref-held-by-io.c | 5 ++++- tests/generic/251 | 5 ++++- tests/generic/323 | 7 +++++-- tests/generic/530 | 2 +- tests/generic/531 | 8 +++++++- tests/xfs/013 | 4 ++-- tests/xfs/076 | 6 +++--- tests/xfs/176 | 6 +++--- tests/xfs/297 | 4 +++- tests/xfs/501 | 2 +- tests/xfs/502 | 2 +- 11 files changed, 34 insertions(+), 17 deletions(-)src/aio-dio-regress/aio-last-ref-held-by-io.c +4
-1@@ -85,11 +85,14 @@ aio_test_thread(void *data)/* * Problems have been easier to trigger when spreading the * workload over the available CPUs. +* +* If CPU hotplug is active, this can randomly fail so dump the error +* to stderror so it can be filtered out easily by the caller. */ CPU_ZERO (&cpuset);CPU_SET (mycpu, &cpuset);if (sched_setaffinity (mytid,sizeof (cpuset), &cpuset)) { -printf ("FAILED to set thread %d to run on cpu %ld\n" , +fprintf (stderr,"FAILED to set thread %d to run on cpu %ld\n" , mytid, mycpu); }tests/generic/251 +0
-0
tests/generic/251 nproc=20 · line 183
Quoted 10 lines
tests/generic/251 +4
-1@@ -175,9 +175,12 @@ nproc=20 # Copy $here to the scratch fs and make coipes of the replica. The fstests # output (and hence $seqres.full) could be in $here, so we need to snapshot # $here before computing file checksums. +# +# $here/* as the files to copy so we avoid any .git directory that might be +# much, much larger than the rest of the fstests source tree we are copying. content=$SCRATCH_MNT/orig mkdir -p $content -cp -axT $here/ $content/ +cp -ax $here/* $content/
Hi Dave,
Darrick sent a patch to review this line:
https://lore.kernel.org/fstests/173258395238.4031902.16373799205312238046.stgit@frogsfrogsfrogs/T/#u
Please help to review, if you don't need this change anymore, I'll fix this conflict (by removing your above change on g/251) when I merge this patch.
Thanks,
Zorro
Quoted 160 lines
tests/generic/251 +0
-0mkdir -p $tmptests/generic/323 +5
-2@@ -23,12 +23,15 @@ _require_aiodio aio-last-ref-held-by-io testfile=$TEST_DIR/aio-testfile $XFS_IO_PROG -ftc "pwrite 0 10m" $testfile | _filter_xfs_io -$AIO_TEST 0 100 $testfile +# This can emit cpu affinity setting failures that aren't considered test +# failures but cause golden image failures. Redirect the test output to +# $seqres.full so that it is captured but doesn't directly cause test failures. +$AIO_TEST 0 100 $testfile 2>> $seqres.full if [ $? -ne 0 ]; then exit $status fi -$AIO_TEST 1 100 $testfile +$AIO_TEST 1 100 $testfile 2>> $seqres.full if [ $? -ne 0 ]; then exit $status fitests/generic/530 +1
-1@@ -22,7 +22,7 @@ _require_scratch_shutdown _require_metadata_journaling _require_test_program "t_open_tmpfiles" -_scratch_mkfs >> $seqres.full 2>&1 +_scratch_mkfs "-l size=256m" >> $seqres.full 2>&1 _scratch_mount # Set ULIMIT_NOFILE to min(file-max / 2, 50000 files per LOAD_FACTOR)tests/generic/531 +7
-1@@ -21,7 +21,13 @@ _require_scratch _require_xfs_io_command "-T" _require_test_program "t_open_tmpfiles" -_scratch_mkfs >> $seqres.full 2>&1 +# On high CPU count machines, this runs a -lot- of create and unlink +# concurrency. Set the filesytsem up to handle this. +if [ $FSTYP = "xfs" ]; then + _scratch_mkfs "-d agcount=32" >> $seqres.full 2>&1 +else + _scratch_mkfs >> $seqres.full 2>&1 +fi _scratch_mount # Try to load up all the CPUs, two threads per CPU.tests/xfs/013 +2
-2@@ -28,7 +28,7 @@ _create() mkdir -p $dir for i in $(seq 0 $count) do - touch $dir/$i 2>&1 | filter_enospc + echo -n > $dir/$i 2>&1 | filter_enospc done }@@ -42,7 +42,7 @@ _rand_replace() do file=$((RANDOM % count)) rm -f $dir/$file - touch $dir/$file 2>&1 | filter_enospc + echo -n > $dir/$file 2>&1 | filter_enospc done }tests/xfs/076 +3
-3@@ -47,10 +47,10 @@ _alloc_inodes() dir=$1 i=0 - while [ true ]; do - touch $dir/$i 2>> $seqres.full || break + ( while [ true ]; do + echo -n > $dir/$i || break i=$((i + 1)) - done + done ) >> $seqres.full 2>&1 }tests/xfs/176 +3
-3@@ -68,10 +68,10 @@ _alloc_inodes() dir=$1 i=0 - while [ true ]; do - echo -n > $dir/$i >> $seqres.full 2>&1 || break + ( while [ true ]; do + echo -n > $dir/$i || break i=$((i + 1)) - done + done ) >> $seqres.full 2>&1 } # Find a sparse inode cluster after logend_agno/logend_agino.tests/xfs/297 +3
-1@@ -34,7 +34,9 @@ _scratch_mount STRESS_DIR="$SCRATCH_MNT/testdir" mkdir -p $STRESS_DIR -_run_fsstress_bg -d $STRESS_DIR -n 1000 -p 1000 $FSSTRESS_AVOID +# turn off sync as this can lead to near deadlock conditions due to every +# fsstress process lockstepping against freeze on large CPU count machines +_run_fsstress_bg -d $STRESS_DIR -f sync=0 -n 1000 -p 1000 $FSSTRESS_AVOID # Freeze/unfreeze file system randomly echo "Start freeze/unfreeze randomly" | tee -a $seqres.fulltests/xfs/501 +1
-1@@ -33,7 +33,7 @@ _require_xfs_sysfs debug/log_recovery_delay _require_scratch _require_test_program "t_open_tmpfiles" -_scratch_mkfs >> $seqres.full 2>&1 +_scratch_mkfs "-l size=256m" >> $seqres.full 2>&1 _scratch_mount # Set ULIMIT_NOFILE to min(file-max / 2, 30000 files per LOAD_FACTOR)tests/xfs/502 +1
-1@@ -23,7 +23,7 @@ _require_xfs_io_error_injection "iunlink_fallback" _require_scratch _require_test_program "t_open_tmpfiles" -_scratch_mkfs | _filter_mkfs 2> $tmp.mkfs > /dev/null +_scratch_mkfs "-l size=256m" | _filter_mkfs 2> $tmp.mkfs > /dev/null cat $tmp.mkfs >> $seqres.full . $tmp.mkfsSignature · 1 line
2.45.2
PATCH 26/40 generic/310: cleanup killing background processes
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
Use the trick we used with fsstress of copying the binary to a test specific name so that we can simply use pkill to reliably kill the background processes this test runs. Also use SIGPIPE to avoid bash from throwing out "Killed" errors.
--- tests/generic/310 | 21 +++++++++------------ 1 file changed, 9 insertions(+), 12 deletions(-)
tests/generic/310 +9 -12
@@ -29,11 +29,8 @@ _begin_fstest auto # Override the default cleanup function. _cleanup() { - if [ -n "$readdir_pid" ]; then - pkill --parent "$readdir_pid" t_readdir > /dev/null 2>&1 - kill -9 $readdir_pid > /dev/null 2>&1 - wait - fi + pkill -9 $seq.t_readdir > /dev/null 2>&1 + wait rm -rf $TEST_DIR/tmp rm -f $tmp.* }
@@ -74,6 +71,8 @@ check_kernel_bug() RUN_TIME=$((30 * $TIME_FACTOR)) +cp $here/src/t_readdir_1 $TEST_DIR/$seq.t_readdir_1 +cp $here/src/t_readdir_2 $TEST_DIR/$seq.t_readdir_2 SEQ_DIR=$TEST_DIR/$seq mkdir -p $SEQ_DIR for n in {1..4096}; do
@@ -82,12 +81,11 @@ done _test_read() { - $here/src/t_readdir_1 $SEQ_DIR > /dev/null 2>&1 & - readdir_pid=$! + $TEST_DIR/$seq.t_readdir_1 $SEQ_DIR > /dev/null 2>&1 & sleep $RUN_TIME - kill $readdir_pid - unset readdir_pid + pkill -PIPE $seq.t_readdir_1 wait + check_kernel_bug if [ $? -ne 0 ]; then _fatal "kernel bug detected, check dmesg for more infomation."
@@ -96,11 +94,10 @@ _test_read() _test_lseek() { - $here/src/t_readdir_2 $SEQ_DIR > /dev/null 2>&1 & + $TEST_DIR/$seq.t_readdir_2 $SEQ_DIR > /dev/null 2>&1 & readdir_pid=$! sleep $RUN_TIME - kill $readdir_pid - unset readdir_pid + pkill -PIPE $seq.t_readdir_2 wait check_kernel_bug
Signature · 1 line
2.45.2
PATCH 27/40 filter: handle mount errors from CONFIG_BLK_DEV_WRITE_MOUNTED=y
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
Kernels post 6.x may have CONFIG_BLK_DEV_WRITE_MOUNTED=y which prevents mount from opening the block device on a mounted filesystem. This results in an error such as:
mount: <dev>: Can't open blockdev
which is not the error that callers of _filter_error_mount() are looking for. It is, however, a direct result of the test trying to mount an alreayd mounted filesystem, so it is reflecting the same error case. Hence this mismatch in errors should not fail the test.
Catch this mount error and convert it to the expected "already mounted" error for the tests that exercise this behaviour.
There is also a minor test change here to push mount failure information to $seqres.full in the cases where mount errors occur.
--- common/filter | 8 +++++++- tests/xfs/154 | 1 + 2 files changed, 8 insertions(+), 1 deletion(-)
common/filter +7 -1
@@ -430,6 +430,10 @@ _filter_ro_mount() { # v2.38 and later: # dmesg(1) may have more information after failed mount mount system call # +# Kernels post 6.x may have CONFIG_BLK_DEV_WRITE_MOUNTED=y which prevents mount +# from opening the block device on a mounted filesytsem: +# mount: <dev>: Can't open blockdev +# # This is also true for ESTALE error. So let's remove all the changing parts # and keep the 'prior to v2.21' format: # mount: Structure needs cleaning
@@ -437,7 +441,9 @@ _filter_ro_mount() { _filter_error_mount() { grep -v "dmesg(1) may have more information after failed mount" | \ - sed -e "s/mount:\(.*failed:\)/mount:/" | _filter_ending_dot + sed -e "s/mount:\(.*failed:\)/mount:/" \ + -e "s/mount: SCRATCH_DEV: Can't open blockdev/mount: TEST_DIR\/extra_mnt: SCRATCH_DEV already mounted or mount point busy/" |\ + _filter_ending_dot } # Similar to _filter_error_mount, filter a busy mount output.
tests/xfs/154 +1 -0
@@ -46,6 +46,7 @@ test $? -eq 137 || echo "repair should have been killed??" _check_scratch_xfs_features NEEDSREPAIR _try_scratch_mount &> $tmp.mount res=$? +cat $tmp.mount >> $seqres.full _filter_error_mount < $tmp.mount if [ $res -eq 0 ]; then echo "Should not be able to mount after needsrepair crash"
Signature · 1 line
2.45.2
PATCH 28/40 filters: add a filter that accepts EIO instead of other errors
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
Running a dm-flakey or dm-error test that loads a table that returns EIO to all IO and then running a command that is expected to fail with a specific error is racy.
If there is memory pressure at the same time that the table is loaded, cached inodes can be turfed from memory and the command then needs to read the inode it is about to act on from disk again. This results in the inode read getting EIO and failing (e.g. xfs_io will return a stat() error) rather than having the desired operation fail.
This results in spurious test failures that look like this:
generic/331 - output mismatch (see /mnt/xfs/runner-41/results-2024-11-20-10:57:31/xfs/generic/331.out.bad)
--- tests/generic/331.out 2022-12-21 15:53:25.487044098 +1100
+++ /mnt/xfs/runner-41/results-2024-11-20-10:57:31/xfs/generic/331.out.bad 2024-11-20 11:02:12.123572607 +1100
@@ -5,7 +5,8 @@
1886e67cf8783e89ce6ddc5bb09a3944 SCRATCH_MNT/test-331/file1
1886e67cf8783e89ce6ddc5bb09a3944 SCRATCH_MNT/test-331/file2
CoW and unmount
-fdatasync: Input/output error
+/mnt/xfs/runner-41/scratch/test-331/file2: Input/output error
+stat: Input/output error
Compare files
...Add a new "flakey EIO filter" that will catch -any- EIO error from the command and change it to the error we expected to see.
--- common/filter | 9 +++++++++ tests/btrfs/160 | 3 ++- tests/generic/252 | 3 ++- tests/generic/329 | 3 ++- tests/generic/331 | 5 +++-- tests/generic/484 | 3 ++- tests/generic/743 | 4 +++- tests/xfs/237 | 3 ++- tests/xfs/240 | 5 +++-- 9 files changed, 28 insertions(+), 10 deletions(-)
common/filter +9 -0
@@ -662,5 +662,14 @@ _filter_trailing_whitespace() sed -E -e "s/\s+$//" } +# Catch -any- EIO error regardless of it's source and replace it with the +# supplied error message. +_filter_flakey_EIO() +{ + local message="$*" + + sed -e "s#.*: Input\/output error#$message#" +} + # make sure this script returns success /bin/true
tests/btrfs/160 +2 -1
@@ -66,7 +66,8 @@ _dmerror_load_working_table # open again and call fsync echo "The following fsync should fail with EIO:" -$XFS_IO_PROG -c fsync $testfile +$XFS_IO_PROG -c fsync $testfile |& \ + _filter_flakey_EIO "fsync: Input/output error" echo "done" # close file
tests/generic/252 +2 -1
@@ -66,7 +66,8 @@ $XFS_IO_PROG -f -c "pwrite -S 0x63 $bufsize 1" $testdir/file2 >> $seqres.full $XFS_IO_PROG -f -c "pwrite -S 0x63 -b $bufsize 0 $filesize" $TEST_DIR/moo >> $seqres.full _scratch_sync _dmerror_load_error_table -$AIO_TEST -a $alignment -f DIRECT -b $bufsize $TEST_DIR/moo $testdir/file2 >> $seqres.full +$AIO_TEST -a $alignment -f DIRECT -b $bufsize $TEST_DIR/moo $testdir/file2 |& \ + _filter_flakey_EIO "write missed bytes expect 8388608 got 0" _dmerror_load_working_table _dmerror_unmount _dmerror_mount
tests/generic/329 +2 -1
@@ -59,7 +59,8 @@ $XFS_IO_PROG -f -c "pwrite -S 0x63 $bufsize 1" $testdir/file2 >> $seqres.full $XFS_IO_PROG -f -c "pwrite -S 0x63 -b $bufsize 0 $filesize" $TEST_DIR/moo >> $seqres.full _scratch_sync _dmerror_load_error_table -$AIO_TEST -a $alignment -f DIRECT -b $bufsize $TEST_DIR/moo $testdir/file2 >> $seqres.full +$AIO_TEST -a $alignment -f DIRECT -b $bufsize $TEST_DIR/moo $testdir/file2 |& \ + _filter_flakey_EIO "write missed bytes expect 8388608 got 0" _dmerror_load_working_table _dmerror_unmount _dmerror_mount
tests/generic/331 +3 -2
@@ -59,8 +59,9 @@ $XFS_IO_PROG -f -c "pwrite -S 0x63 $bufsize 1" $testdir/file2 >> $seqres.full $XFS_IO_PROG -f -c "pwrite -S 0x63 -b $bufsize 0 $filesize" $TEST_DIR/moo >> $seqres.full _scratch_sync _dmerror_load_error_table -$AIO_TEST -b $bufsize $TEST_DIR/moo $testdir/file2 >> $seqres.full -$XFS_IO_PROG -c "fdatasync" $testdir/file2 +$AIO_TEST -b $bufsize $TEST_DIR/moo $testdir/file2 >> $seqres.full 2>&1 +$XFS_IO_PROG -c "fdatasync" $testdir/file2 |& \ + _filter_flakey_EIO "fdatasync: Input/output error" _dmerror_load_working_table _dmerror_unmount _dmerror_mount
tests/generic/484 +2 -1
@@ -54,7 +54,8 @@ $XFS_IO_PROG -c "pwrite -W -q 0 $datalen" $testfile _dmerror_load_error_table # rewrite the data and call fdatasync -$XFS_IO_PROG -c "pwrite -w -q 0 $datalen" $testfile +$XFS_IO_PROG -c "pwrite -w -q 0 $datalen" $testfile |& \ + _filter_flakey_EIO "fdatasync: Input/output error" # heal the device error _dmerror_load_working_table
tests/generic/743 +3 -1
@@ -21,6 +21,7 @@ _cleanup() } # Import common functions. +. ./common/filter . ./common/dmerror _fixed_by_kernel_commit 631426ba1d45 \
@@ -55,7 +56,8 @@ _dmerror_mount stat "$SCRATCH_MNT/a" >> $seqres.full echo read with IO errors _dmerror_load_error_table -$TIMEOUT_PROG -s KILL 10s $XFS_IO_PROG -c "mmap -r 0 $filesz" -c "madvise -R 0 $filesz" "$SCRATCH_MNT/a" +$TIMEOUT_PROG -s KILL 10s $XFS_IO_PROG -c "mmap -r 0 $filesz" -c "madvise -R 0 $filesz" "$SCRATCH_MNT/a" |& \ + _filter_flakey_EIO "madvise: Bad address" _dmerror_load_working_table # success, all done
tests/xfs/237 +2 -1
@@ -68,7 +68,8 @@ $XFS_IO_PROG -f -c "pwrite -S 0x63 $bufsize 1" $testdir/file2 >> $seqres.full $XFS_IO_PROG -f -c "pwrite -S 0x63 -b $bufsize 0 $filesize" $TEST_DIR/moo >> $seqres.full _scratch_sync _dmerror_load_error_table -$AIO_TEST -a $alignment -f DIRECT -b $bufsize $TEST_DIR/moo $testdir/file2 >> $seqres.full +$AIO_TEST -a $alignment -f DIRECT -b $bufsize $TEST_DIR/moo $testdir/file2 |& \ + _filter_flakey_EIO "write missed bytes expect 8388608 got 0" _dmerror_load_working_table _dmerror_unmount _dmerror_mount
tests/xfs/240 +3 -2
@@ -64,8 +64,9 @@ $XFS_IO_PROG -f -c "pwrite -S 0x63 $bufsize 1" $testdir/file2 >> $seqres.full $XFS_IO_PROG -f -c "pwrite -S 0x63 -b $bufsize 0 $filesize" $TEST_DIR/moo >> $seqres.full _scratch_sync _dmerror_load_error_table -$AIO_TEST -b $bufsize $TEST_DIR/moo $testdir/file2 >> $seqres.full -$XFS_IO_PROG -c "fdatasync" $testdir/file2 +$AIO_TEST -b $bufsize $TEST_DIR/moo $testdir/file2 >> $seqres.full 2>&1 +$XFS_IO_PROG -c "fdatasync" $testdir/file2 |& \ + _filter_flakey_EIO "fdatasync: Input/output error" _dmerror_load_working_table _dmerror_unmount _dmerror_mount
Signature · 1 line
2.45.2
PATCH 29/40 generic/085: general cleanup for reliability and debugging
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
This test was quite unreliable during development of the parallel check runner. It redirects all errors to /dev/null, so there was no way to debug it when it failed.
Use common mount/unmount helpers, redirect errors to $seqres.full, make sure the cleanup code is always run at test exit and only attempt to kill processes if they are still running during cleanup.
--- tests/generic/085 | 21 ++++++++++++--------- 1 file changed, 12 insertions(+), 9 deletions(-)
tests/generic/085 +12 -9
@@ -16,18 +16,20 @@ _begin_fstest auto freeze mount # Override the default cleanup function. _cleanup() { + cleanup_dmdev cd / rm -f $tmp.* - cleanup_dmdev } cleanup_dmdev() { + if [ -n "$pid" ]; then + kill -9 $pid 2>> $seqres.full + wait $pid 2>> $seqres.full + fi # in case it's still suspended and/or mounted - $DMSETUP_PROG resume $lvdev >/dev/null 2>&1 - [ -n "$pid" ] && kill -9 $pid 2>/dev/null - wait $pid - _unmount $lvdev >/dev/null 2>&1 + $DMSETUP_PROG resume $lvdev >> $seqres.full 2>&1 + _unmount -q $SCRATCH_MNT _dmsetup_remove $node }
@@ -64,18 +66,19 @@ setup_dmdev # important, as long as they're racing with each other. So just throw away the # outputs and ignore the results. for ((i=0; i<100; i++)); do - $DMSETUP_PROG suspend $lvdev >/dev/null 2>&1 - $DMSETUP_PROG resume $lvdev >/dev/null 2>&1 + $DMSETUP_PROG suspend $lvdev >> $seqres.full 2>&1 + $DMSETUP_PROG resume $lvdev >> $seqres.full 2>&1 done & pid=$! for ((i=0; i<100; i++)); do - _mount $lvdev $SCRATCH_MNT >/dev/null 2>&1 - _unmount $lvdev >/dev/null 2>&1 + _mount $lvdev $SCRATCH_MNT >> $seqres.full 2>&1 + _unmount $lvdev >> $seqres.full 2>&1 done & pid="$pid $!" wait $pid unset pid +cleanup_dmdev status=0 exit
Signature · 1 line
2.45.2
PATCH 30/40 fstests: don't use directory stacks
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
Using bash directory stacking (pushd, popd, etc) seems to be somewhat unreliable. I've been seeing occasional random failures from both pushd and popd commands that cause the test to fail, and there does not appear to be any reason for the failures occurring.
Rather than wasting time chasing ghosts, just get rid of the directory stacking altogether.
--- tests/generic/099 | 8 +++----- tests/generic/109 | 5 +---- tests/generic/135 | 24 +++++++++++------------- tests/generic/707 | 7 ++++--- 4 files changed, 19 insertions(+), 25 deletions(-)
tests/generic/099 +3 -5
@@ -253,15 +253,13 @@ echo "" echo "=== Recursive change ACL ===" rm -fr root mkdir root -pushd root >/dev/null # create an arbitrary little tree -for i in 1 2 3 4 5 6 7 8 9 0 -do +( cd root ; for i in 1 2 3 4 5 6 7 8 9 0; do mkdir -p a/$i mkdir -p b/c$i/$i touch a/$i/mumble -done -popd >/dev/null +done ) + chown -R 12345:54321 root echo "Change #1..." _runas -u 12345 -g 54321 -- chacl -r u::rwx,g::-w-,o::--x root
tests/generic/109 +1 -4
@@ -54,10 +54,7 @@ _scratch_mount >> $seqres.full 2>&1 # Test different directory sizes to test various directory formats for f in 1 2 3 4 5 8 12 18 27 40 60 90 135 202 303 454 681 1020 1530 2295; do mkdir $SCRATCH_MNT/dir$f - pushd $SCRATCH_MNT/dir$f >/dev/null - filldir $f - renamedir - popd >/dev/null + ( cd $SCRATCH_MNT/dir$f ; filldir $f ; renamedir ) done status=0
tests/generic/135 +11 -13
@@ -21,30 +21,28 @@ _scratch_mount cd $SCRATCH_MNT # create file with async I/O -$XFS_IO_PROG -f -c 'pwrite -b 4k -S 0x12 0 4k' async_file > /dev/null +$XFS_IO_PROG -f -c 'pwrite -b 4k -S 0x12 0 4k' $SCRATCH_MNT/async_file > /dev/null # create file with sync I/O -$XFS_IO_PROG -f -s -c 'pwrite -b 4k -S 0x34 0 4k' sync_file > /dev/null +$XFS_IO_PROG -f -s -c 'pwrite -b 4k -S 0x34 0 4k' $SCRATCH_MNT/sync_file > /dev/null # create file with direct I/O -$XFS_IO_PROG -f -d -c 'pwrite -b 4k -S 0x56 0 4k' direct_file > /dev/null +$XFS_IO_PROG -f -d -c 'pwrite -b 4k -S 0x56 0 4k' $SCRATCH_MNT/direct_file > /dev/null # create file, truncate and then dirty again -$XFS_IO_PROG -f -c 'pwrite -b 4k -S 0x78 0 4k' trunc_file > /dev/null -$XFS_IO_PROG -f -c 'truncate 2k' trunc_file > /dev/null -$XFS_IO_PROG -c 'pwrite 1k 0 1k' trunc_file > /dev/null +$XFS_IO_PROG -f -c 'pwrite -b 4k -S 0x78 0 4k' $SCRATCH_MNT/trunc_file > /dev/null +$XFS_IO_PROG -f -c 'truncate 2k' $SCRATCH_MNT/trunc_file > /dev/null +$XFS_IO_PROG -c 'pwrite 1k 0 1k' $SCRATCH_MNT/trunc_file > /dev/null -pushd / > /dev/null _scratch_cycle_mount -popd > /dev/null # check file size and contents -od -Ad -x async_file -od -Ad -x sync_file -od -Ad -x direct_file -od -Ad -x trunc_file +od -Ad -x $SCRATCH_MNT/async_file +od -Ad -x $SCRATCH_MNT/sync_file +od -Ad -x $SCRATCH_MNT/direct_file +od -Ad -x $SCRATCH_MNT/trunc_file -rm -f async_file sync_file direct_file trunc_file +rm -f $SCRATCH_MNT/* status=0 exit
tests/generic/707 +4 -3
@@ -42,7 +42,7 @@ create_files() # We use slightly longer file name to make directory grow faster and # hopefully convert between various types for (( i = 0; i < $files; i++ )); do - touch somewhatlongerfilename$i + echo -n > somewhatlongerfilename$i done }
@@ -50,12 +50,13 @@ for (( i = 0; i <= $moves; i++ )); do mkdir $SCRATCH_MNT/dir$i done +start_dir=$PWD for (( l = 0; l < $loops; l++ )); do mkdir $SCRATCH_MNT/dir0/dir - pushd $SCRATCH_MNT/dir0/dir &>/dev/null + cd $SCRATCH_MNT/dir0/dir create_files & BGPID=$! - popd &>/dev/null + cd $start_dir for (( i = 0; i < $moves; i++ )); do mv $SCRATCH_MNT/dir$i/dir $SCRATCH_MNT/dir$((i+1))/dir done
Signature · 1 line
2.45.2
Re: [PATCH 30/40] fstests: don't use directory stacks
Quoted 62 lines · Dave Chinner
From: Dave Chinner <[email protected]>
Using bash directory stacking (pushd, popd, etc) seems to be somewhat unreliable. I've been seeing occasional random failures from both pushd and popd commands that cause the test to fail, and there does not appear to be any reason for the failures occurring.
Rather than wasting time chasing ghosts, just get rid of the directory stacking altogether.
Signed-off-by Dave Chinner <[email protected]>--- tests/generic/099 | 8 +++----- tests/generic/109 | 5 +---- tests/generic/135 | 24 +++++++++++------------- tests/generic/707 | 7 ++++--- 4 files changed, 19 insertions(+), 25 deletions(-)tests/generic/099 +3
-5@@ -253,15 +253,13 @@ echo "" echo "=== Recursive change ACL ===" rm -fr root mkdir root -pushd root >/dev/null # create an arbitrary little tree -for i in 1 2 3 4 5 6 7 8 9 0 -do +( cd root ; for i in 1 2 3 4 5 6 7 8 9 0; do mkdir -p a/$i mkdir -p b/c$i/$i touch a/$i/mumble -done -popd >/dev/null +done ) + chown -R 12345:54321 root echo "Change #1..." _runas -u 12345 -g 54321 -- chacl -r u::rwx,g::-w-,o::--x roottests/generic/109 +1
-4@@ -54,10 +54,7 @@ _scratch_mount >> $seqres.full 2>&1 # Test different directory sizes to test various directory formats for f in 1 2 3 4 5 8 12 18 27 40 60 90 135 202 303 454 681 1020 1530 2295; do mkdir $SCRATCH_MNT/dir$f - pushd $SCRATCH_MNT/dir$f >/dev/null - filldir $f - renamedir - popd >/dev/null + ( cd $SCRATCH_MNT/dir$f ; filldir $f ; renamedir ) done status=0tests/generic/135 +0
-0
tests/generic/135 _scratch_mount · line 21
Quoted 1 line
tests/generic/135 +0
-0@@ -21,30 +21,28 @@ _scratch_mount cd $SCRATCH_MNT
^^^^^^^^^^^^^^^
Below "_scratch_cycle_mount" fails, due to above line. I'll remove it.
Thanks,
Zorro
Quoted 73 lines
tests/generic/135 +11
-13# create file with async I/O -$XFS_IO_PROG -f -c 'pwrite -b 4k -S 0x12 0 4k' async_file > /dev/null +$XFS_IO_PROG -f -c 'pwrite -b 4k -S 0x12 0 4k' $SCRATCH_MNT/async_file > /dev/null # create file with sync I/O -$XFS_IO_PROG -f -s -c 'pwrite -b 4k -S 0x34 0 4k' sync_file > /dev/null +$XFS_IO_PROG -f -s -c 'pwrite -b 4k -S 0x34 0 4k' $SCRATCH_MNT/sync_file > /dev/null # create file with direct I/O -$XFS_IO_PROG -f -d -c 'pwrite -b 4k -S 0x56 0 4k' direct_file > /dev/null +$XFS_IO_PROG -f -d -c 'pwrite -b 4k -S 0x56 0 4k' $SCRATCH_MNT/direct_file > /dev/null # create file, truncate and then dirty again -$XFS_IO_PROG -f -c 'pwrite -b 4k -S 0x78 0 4k' trunc_file > /dev/null -$XFS_IO_PROG -f -c 'truncate 2k' trunc_file > /dev/null -$XFS_IO_PROG -c 'pwrite 1k 0 1k' trunc_file > /dev/null +$XFS_IO_PROG -f -c 'pwrite -b 4k -S 0x78 0 4k' $SCRATCH_MNT/trunc_file > /dev/null +$XFS_IO_PROG -f -c 'truncate 2k' $SCRATCH_MNT/trunc_file > /dev/null +$XFS_IO_PROG -c 'pwrite 1k 0 1k' $SCRATCH_MNT/trunc_file > /dev/null -pushd / > /dev/null _scratch_cycle_mount -popd > /dev/null # check file size and contents -od -Ad -x async_file -od -Ad -x sync_file -od -Ad -x direct_file -od -Ad -x trunc_file +od -Ad -x $SCRATCH_MNT/async_file +od -Ad -x $SCRATCH_MNT/sync_file +od -Ad -x $SCRATCH_MNT/direct_file +od -Ad -x $SCRATCH_MNT/trunc_file -rm -f async_file sync_file direct_file trunc_file +rm -f $SCRATCH_MNT/* status=0 exittests/generic/707 +4
-3@@ -42,7 +42,7 @@ create_files() # We use slightly longer file name to make directory grow faster and # hopefully convert between various types for (( i = 0; i < $files; i++ )); do - touch somewhatlongerfilename$i + echo -n > somewhatlongerfilename$i done }@@ -50,12 +50,13 @@ for (( i = 0; i <= $moves; i++ )); do mkdir $SCRATCH_MNT/dir$i done +start_dir=$PWD for (( l = 0; l < $loops; l++ )); do mkdir $SCRATCH_MNT/dir0/dir - pushd $SCRATCH_MNT/dir0/dir &>/dev/null + cd $SCRATCH_MNT/dir0/dir create_files & BGPID=$! - popd &>/dev/null + cd $start_dir for (( i = 0; i < $moves; i++ )); do mv $SCRATCH_MNT/dir$i/dir $SCRATCH_MNT/dir$((i+1))/dir doneSignature · 1 line
2.45.2
Re: [PATCH 30/40] fstests: don't use directory stacks
Quoted 66 lines · Zorro Lang
Quoted 62 lines · Dave Chinner
From: Dave Chinner <[email protected]>
Using bash directory stacking (pushd, popd, etc) seems to be somewhat unreliable. I've been seeing occasional random failures from both pushd and popd commands that cause the test to fail, and there does not appear to be any reason for the failures occurring.
Rather than wasting time chasing ghosts, just get rid of the directory stacking altogether.
Signed-off-by Dave Chinner <[email protected]>--- tests/generic/099 | 8 +++----- tests/generic/109 | 5 +---- tests/generic/135 | 24 +++++++++++------------- tests/generic/707 | 7 ++++--- 4 files changed, 19 insertions(+), 25 deletions(-)tests/generic/099 +3
-5@@ -253,15 +253,13 @@ echo "" echo "=== Recursive change ACL ===" rm -fr root mkdir root -pushd root >/dev/null # create an arbitrary little tree -for i in 1 2 3 4 5 6 7 8 9 0 -do +( cd root ; for i in 1 2 3 4 5 6 7 8 9 0; do mkdir -p a/$i mkdir -p b/c$i/$i touch a/$i/mumble -done -popd >/dev/null +done ) + chown -R 12345:54321 root echo "Change #1..." _runas -u 12345 -g 54321 -- chacl -r u::rwx,g::-w-,o::--x roottests/generic/109 +1
-4@@ -54,10 +54,7 @@ _scratch_mount >> $seqres.full 2>&1 # Test different directory sizes to test various directory formats for f in 1 2 3 4 5 8 12 18 27 40 60 90 135 202 303 454 681 1020 1530 2295; do mkdir $SCRATCH_MNT/dir$f - pushd $SCRATCH_MNT/dir$f >/dev/null - filldir $f - renamedir - popd >/dev/null + ( cd $SCRATCH_MNT/dir$f ; filldir $f ; renamedir ) done status=0tests/generic/135 +0
-0tests/generic/135 _scratch_mount · line 21
Quoted 1 line
tests/generic/135 +0
-0@@ -21,30 +21,28 @@ _scratch_mount cd $SCRATCH_MNT^^^^^^^^^^^^^^^
Quoted 1 line
Below "
_scratch_cycle_mount" fails, due to above line. I'll remove it.
I've updated my local tree with this fix and the others that you've noticed. Thanks!
-Dave.
Signature · 2 lines
Dave Chinner [email protected]
PATCH 31/40 fstests: clean up a couple of dm-flakey tests
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
Just little things I've found that should be cleaned up.
--- tests/generic/066 | 3 +++ tests/generic/321 | 4 +++- tests/xfs/051 | 7 +++++++ 3 files changed, 13 insertions(+), 1 deletion(-)
tests/generic/066 +3 -0
@@ -24,6 +24,7 @@ _begin_fstest auto quick attr metadata log _cleanup() { _cleanup_flakey + cd / rm -f $tmp.* }
@@ -88,5 +89,7 @@ _flakey_drop_and_remount echo "xattr names and values after second fsync log replay:" _getfattr --absolute-names --dump $SCRATCH_MNT/foobar | _filter_scratch +_unmount_flakey + status=0 exit
tests/generic/321 +3 -1
@@ -13,6 +13,8 @@ _begin_fstest auto quick metadata log _cleanup() { _cleanup_flakey + cd / + rm -f $tmp.* } # Import common functions.
@@ -57,7 +59,7 @@ _rename_test() $XFS_IO_PROG -c "fsync" $SCRATCH_MNT/bar $XFS_IO_PROG -c "fsync" $SCRATCH_MNT/bar/foo - _flakey_drop_and_remount\ + _flakey_drop_and_remount _ls_l $SCRATCH_MNT | tail -n +2 | awk '{ print $1, $9 }' _ls_l $SCRATCH_MNT/bar | tail -n +2 | awk '{ print $1, $9 }'
tests/xfs/051 +7 -0
@@ -11,6 +11,13 @@ . ./common/preamble _begin_fstest shutdown auto log metadata +_cleanup() +{ + _cleanup_flakey + cd / + rm -f $tmp.* +} + # Import common functions. . ./common/dmflakey
Signature · 1 line
2.45.2
PATCH 32/40 fstests: clean up termination of various tests
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
Accumulated minor fixes to improve reliablity of the termination of various tests when interrupted.
--- tests/generic/084 | 12 +++++++++--- tests/generic/464 | 10 +++++----- tests/generic/650 | 8 ++++---- tests/xfs/013 | 1 + tests/xfs/558 | 1 + 5 files changed, 20 insertions(+), 12 deletions(-)
tests/generic/084 +9 -3
@@ -12,6 +12,15 @@ . ./common/preamble _begin_fstest auto metadata quick +_cleanup() +{ + [ -n "$link_pids" ] && kill $link_pids + [ -n "$open_pid" ] && kill $open_pid + wait + cd / + rm -f $tmp.* +} + # Import common functions. . ./common/filter
@@ -57,9 +66,6 @@ done & sleep 5 kill $! >/dev/null 2>&1 -kill $open_pid $link_pids >/dev/null 2>&1 -wait $open_pid $link_pids - # all done, no oops/hang expected, _check_filesystems checks SCRATCH_DEV after test status=0 exit
tests/generic/464 +5 -5
@@ -66,22 +66,22 @@ _scratch_mount # for each operation and runs for $LOOP_TIME seconds, and check filesystem # consistency after each iteration for i in `seq 1 $LOOP_CNT`; do - rm -f $stop + touch $stop for j in `seq 1 $PROC_CNT`; do - while [ ! -e $stop ]; do + while [ -e $stop ]; do do_write done & - while [ ! -e $stop ]; do + while [ -e $stop ]; do do_append done & - while [ ! -e $stop ]; do + while [ -e $stop ]; do do_writeback done & done sleep $LOOP_TIME - touch $stop + rm -f $stop wait _scratch_unmount
tests/generic/650 +4 -4
@@ -52,8 +52,6 @@ nr_hotplug_cpus="${#hotplug_cpus[@]}" test "$nr_hotplug_cpus" -gt 0 || _notrun "CPU hotplugging not supported" stress_dir="$TEST_DIR/$seq" -rm -r -f "$stress_dir" -mkdir -p "$stress_dir" echo "Silence is golden."
@@ -79,12 +77,14 @@ fsstress_args+=(--duration="$((SOAK_DURATION / 10))") nr_ops=$((2500 * TIME_FACTOR)) fsstress_args+=(-n $nr_ops) for ((i = 0; i < 10; i++)); do - _run_fsstress_bg -w "${fsstress_args[@]}" - _wait_for_fsstress + rm -rf "$stress_dir" + mkdir -p "$stress_dir" + _run_fsstress "${fsstress_args[@]}" _test_cycle_mount done rm -f $sentinel_file +wait # success, all done status=0
tests/xfs/013 +1 -0
@@ -120,6 +120,7 @@ _kill_fsstress # clean out the competing fsstress allocations, then everything else rm -rf $SCRATCH_MNT/fsstress rm -rf $SCRATCH_MNT/dir* +wait _scratch_unmount
tests/xfs/558 +1 -0
@@ -193,6 +193,7 @@ wait_for_errortag & $XFS_IO_PROG -c "pwrite -S 0x56 $((2 * blksz)) $((2 * blksz))" \ -c 'fsync' $SCRATCH_MNT/file >> $seqres.full rm -f $sentryfile +wait cat "$tracefile" >> $seqres.full grep -q iomap_invalid "$tracefile"
Signature · 1 line
2.45.2
PATCH 33/40 vfstests: some tests require the testdir to be shared
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
This ensures that these tests will run successfully when the parallel check infrastructure makes all the scratch and test mounts private.
--- tests/generic/589 | 2 ++ tests/generic/632 | 2 ++ 2 files changed, 4 insertions(+)
tests/generic/589 +2 -0
@@ -164,7 +164,9 @@ move_test() move_run unbindable private } +_mount --make-shared $TEST_DIR move_test +_mount --make-private $TEST_DIR # success, all done status=0
tests/generic/632 +2 -0
@@ -22,8 +22,10 @@ _begin_fstest auto quick mount _require_test _require_test_program "detached_mounts_propagation" +_mount --make-shared $TEST_DIR $here/src/detached_mounts_propagation $TEST_DIR >> $seqres.full status=$? +_mount --make-private $TEST_DIR echo silence is golden exit
Signature · 1 line
2.45.2
PATCH 34/40 xfs/629: single extent files should be within tolerance
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
The test passes if we have between 2 and 40 extents (despite what the comment says!), with the target being 20. There is absolutely no reason for considering a single extent file a failure - that indicates the filesystem completely defeated the fragmentation behaviour the test was trying to cause. Hence expand the range of "test pass" tolerance to 1-41 extents.
--- tests/xfs/629 | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tests/xfs/629 +2 -2
@@ -59,8 +59,8 @@ _scratch_sync for ((n=0; n<nfiles; n++)); do count=$(_count_extents $workfile.$n) - # Acceptible extent count range is 1-40 - _within_tolerance "file.$n extent count" $count 21 19 -v + # Acceptible extent count range is 1-41 + _within_tolerance "file.$n extent count" $count 21 20 -v done status=0
Signature · 1 line
2.45.2
PATCH 35/40 xfs/076: fix broken mkfs filtering
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
The test does not do what it says on the packet - the mkfs output is not actually passed to the mkfs filter, so it doesn't know what inode size mkfs actually used. Hence CHUNK_SIZE ends up being calculated as 0, and that means it enters an endless loop because offset never decreases.
Fix it by adding the missing line continuation.
--- tests/xfs/076 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tests/xfs/076 +1 -1
@@ -68,7 +68,7 @@ _require_xfs_sparse_inodes # bitmap consuming all the free space in our small data device. unset SCRATCH_RTDEV -_scratch_mkfs "-d size=96m -m crc=1 -i sparse" | tee -a $seqres.full | +_scratch_mkfs "-d size=96m -m crc=1 -i sparse" | tee -a $seqres.full | \ _filter_mkfs > /dev/null 2> $tmp.mkfs . $tmp.mkfs # for isize
Signature · 1 line
2.45.2
PATCH 36/40 fstests: capture some failures to seqres.full
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
Whilst trying to debug test failures, I found a few places where errors needed to be redirected to $seqres.full rather than /dev/null. This is a collection of all the conversions that haven't been captured by some other bug fix patch.
Note that calling _check_filesystems() after removing the require_test/scratch files means it is a no-op, so I removed that call at the same time as capturing unmount failures after the test has failed.
--- check | 5 ++--- tests/generic/049 | 2 +- tests/generic/603 | 6 +++--- tests/xfs/495 | 2 +- 4 files changed, 7 insertions(+), 8 deletions(-)
check +2 -3
@@ -1004,14 +1004,13 @@ function run_section() if [ $sts -ne 0 ]; then _dump_err_cont "[failed, exit status $sts]" - _test_unmount 2> /dev/null - _scratch_unmount 2> /dev/null + _test_unmount 2>> $seqres.full + _scratch_unmount 2>> $seqres.full rm -f ${RESULT_DIR}/require_test* rm -f ${RESULT_DIR}/require_scratch* # Even though we failed, there may be something interesting in # dmesg which can help debugging. _check_dmesg - (_adjust_oom_score 250; _check_filesystems) tc_status="fail" else # The test apparently passed, so check for corruption
tests/generic/049 +1 -1
@@ -56,7 +56,7 @@ i=1; while [ $i -lt 1000 ] do file=$SCRATCH_MNT/$i - $XFS_IO_PROG -f -c "pwrite -b 32k -S 0xff 0 32k" $file > /dev/null + $XFS_IO_PROG -f -c "pwrite -b 32k -S 0xff 0 32k" $file >> $seqres.full if [ $? -ne 0 ] then echo error creating/writing file $file
tests/generic/603 +3 -3
@@ -77,7 +77,7 @@ test_grace() _filter_xfs_io_error | tee -a $seqres.full repquota -v -$type $SCRATCH_MNT | grep -v "^root" >>$seqres.full 2>&1 # Reset grace time here, make below grace time test more accurate - setquota -$type $qa_user -T $bgrace $igrace $SCRATCH_MNT 2>/dev/null + setquota -$type $qa_user -T $bgrace $igrace $SCRATCH_MNT >>$seqres.full 2>&1 # Now sleep enough grace time and check that softlimit got enforced sleep $((bgrace + 1)) echo "Try to write 1 one more block after grace..."
@@ -89,13 +89,13 @@ test_grace() # And now the softlimit test for inodes # First reset space limits so that we don't have problems with # space reservations on XFS - setquota -$type $qa_user 0 0 3 100 $SCRATCH_MNT + setquota -$type $qa_user 0 0 3 100 $SCRATCH_MNT echo "Create 2 more files, over the inode softlimit..." su $qa_user -c "touch $dir/file3 $dir/file4" 2>&1 >>$seqres.full | \ _filter_scratch | tee -a $seqres.full repquota -v -$type $SCRATCH_MNT | grep -v "^root" >>$seqres.full 2>&1 # Reset grace time here, make below grace time test more accurate - setquota -$type $qa_user -T $bgrace $igrace $SCRATCH_MNT 2>/dev/null + setquota -$type $qa_user -T $bgrace $igrace $SCRATCH_MNT >>$seqres.full 2>&1 # Wait and check grace time enforcement sleep $((igrace+1)) echo "Try to create one more inode after grace..."
tests/xfs/495 +1 -1
@@ -24,7 +24,7 @@ _require_populate_commands _require_xfs_db_command "fuzz" echo "Format and populate" -_scratch_populate_cached nofill > $seqres.full 2>&1 +_scratch_populate_cached nofill >> $seqres.full 2>&1 filter_nbrepair() { grep rebuilding | sed -e 's/directory inode [0-9]*/directory inode XXX/g'
Signature · 1 line
2.45.2
PATCH 37/40 fstests: always use fail-at-unmount semantics for XFS
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
Rather than require every test that tests unmount in failure conditions have to set up fail-at-unmount semantics for the underlying filesystem, use these semantics for all test and scratch device mounts.
This currently only affects XFS filesystems, and helps prevent unexpected unmount hangs in EIO tests because metadata writes are configured to try forever by default.
--- common/rc | 2 ++ 1 file changed, 2 insertions(+)
common/rc +2 -0
@@ -410,6 +410,7 @@ _try_scratch_mount() mount_ret=$? [ $mount_ret -ne 0 ] && return $mount_ret _idmapped_mount $SCRATCH_DEV $SCRATCH_MNT + _prepare_for_eio_shutdown $SCRATCH_DEV } # mount scratch device with given options and _fail if mount fails
@@ -609,6 +610,7 @@ _test_mount() mount_ret=$? [ $mount_ret -ne 0 ] && return $mount_ret _idmapped_mount $TEST_DEV $TEST_DIR + _prepare_for_eio_shutdown $TEST_DEV } _test_unmount()
Signature · 1 line
2.45.2
PATCH 38/40 generic/062: don't leave debug files in $here on failure
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
Move them to $seqres.<debug-file> so they don't pollute the source tree that fstests is running from.
--- tests/generic/062 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tests/generic/062 +1 -1
@@ -196,7 +196,7 @@ echo "*** compare before and after backups" diff $tmp.backup1 $tmp.backup2 if [ $? -ne 0 ]; then echo "urk, failed - creating $seq.backup1 and $seq.backup2" - cp $tmp.backup1 $seq.backup1 && cp $tmp.backup2 $seq.backup2 + cp $tmp.backup1 $seqres.backup1 && cp $tmp.backup2 $seqres.backup2 status=1 exit fi
Signature · 1 line
2.45.2
PATCH 39/40 fstests: quota grace periods unreliable under load
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
Starting the quota grace period doesn't necessary happen predictably when the system is under heavy load. This results in random test failures where grace period timeouts are expected.
--- tests/generic/601 | 7 ++++--- tests/xfs/050 | 5 +++++ 2 files changed, 9 insertions(+), 3 deletions(-)
tests/generic/601 +4 -3
@@ -52,9 +52,10 @@ $XFS_QUOTA_PROG -x -c "timer -u -i 100 $qa_user" $SCRATCH_MNT # raw ("since epoch") grace expiry get=`repquota -up $SCRATCH_MNT | grep "^$qa_user" | awk '{print $NF}'` -# Either the new expiry must match; or be one second after the set time, to -# deal with the seconds counter incrementing. -if [ "$get" != "$set" ] && [ "$get" -ne "$((set + 1))" ]; then +# Either the new expiry must match or be soon after the set time as there +# may be load related delays on setting the new timer. A 10s "grace" window +# should be enough for everyone. +if [ "$get" != "$set" ] && [ "$get" -gt "$((set + 10))" ]; then echo "set grace to $set but got grace $get" fi
tests/xfs/050 +5 -0
@@ -115,6 +115,11 @@ _exercise() echo echo "*** push past the soft block limit" | tee -a $seqres.full _file_as_id $SCRATCH_MNT/softie $id $type $bsize 300 + + # sync so that the data is actually allocated as soft limit timers + # are not started until block allocation is done. This means the test + # will work even if remount sliently fails due to busy mounts + _scratch_sync _qmount $XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \ -c "repquota -birnN -$type" $SCRATCH_DEV |
Signature · 1 line
2.45.2
PATCH 40/40 fstests: check-parallel
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
Runs tests in parallel runner threads. Each runner thread has it's own set of tests to run, and runs a separate instance of check to run those tests.
check-parallel sets up loop devices, mount points, results directories, etc for each instance and divides the tests up between the runner threads.
It currently hard codes the XFS and generic test lists, and then gives each check invocation an explicit list of tests to run. It also passes through exclusions so that test exclude filtering is still done by check.
This is far from ideal, but I didn't want to have to embark on a major refactoring of check to be able to run stuff in parallel. It was quite the challenge just to get all the tests and test infrastructure up to the point where they can run reliably in parallel.
Hence I've left the actual factoring of test selection and setup out of the patchset for the moment. The plan is to factor both the test setup and the test list runner loop out of check and share them between check and check-parallel, hence not requiring check-parallel to run check directly. That is future work, however.
With the current test runner setup, it is not uncommon to see >5000% cpu usage, 150-200kiops and 4-5GB/s of disk bandwidth being used when running 64 runners. This is a serious stress load as it is constantly mounting and unmounting dozens of filesystems, creating and destroying devices, dropping caches, running sync, running CPU hot plug, running page cache migration, etc.
The massive amount of IO that load generates causes qemu hosts to abort (i.e. crash) because they run out of vm map segments. Hence bumping up the max_map_count on the host like so:
echo 1048576 > /proc/sys/vm/max_map_count
is necessary.
There is no significant memory pressure to speak of from running the tests like this. I've seen a maximum of about 50GB of RAM used when running tests like this, so running on a 64p/64GB VM the additional concurrency doesn't really stress memory capacity like it does CPU and IO.
All the runners are executed in private mount namespaces. This is to prevent ephemeral mount namespace clones from taking a reference to every mounted filesystem in the machine and so causing random "device busy after unmount" failures in the tests that are running concurrently with the mount namespace setup and teardown.
A typical pstree -N mnt looks like:
$ pstree -N mnt
[4026531841]
bash
bash───pstree
[0]
sudo───sudo───check-parallel─┬─check-parallel───nsexec───check───311─┬─cut
│ └─md5sum
├─check-parallel───nsexec───check───750─┬─750───sleep
│ └─750.fsstress───4*[750.fsstress───{750.fsstress}]
├─check-parallel───nsexec───check───013───013───sed
├─check-parallel───nsexec───check───251───cp
├─check-parallel───nsexec───check───467───open_by_handle
├─check-parallel───nsexec───check───650─┬─650───sleep
│ └─650.fsstress─┬─61*[650.fsstress───{650.fsstress}]
│ └─2*[650.fsstress]
├─check-parallel───nsexec───check───707
├─check-parallel───nsexec───check───705
├─check-parallel───nsexec───check───416
├─check-parallel───nsexec───check───477───2*[open_by_handle]
├─check-parallel───nsexec───check───140───140
├─check-parallel───nsexec───check───562
├─check-parallel───nsexec───check───415───xfs_io───{xfs_io}
├─check-parallel───nsexec───check───291
├─check-parallel───nsexec───check───017
├─check-parallel───nsexec───check───016
├─check-parallel───nsexec───check───168───2*[168───168]
├─check-parallel───nsexec───check───672───2*[672───672]
├─check-parallel───nsexec───check───170─┬─170───170───170
│ └─170───170
├─check-parallel───nsexec───check───531───122*[t_open_tmpfiles]
├─check-parallel───nsexec───check───387
├─check-parallel───nsexec───check───748
├─check-parallel───nsexec───check───388─┬─388.fsstress───4*[388.fsstress───{388.fsstress}]
│ └─sleep
├─check-parallel───nsexec───check───328───328
├─check-parallel───nsexec───check───352
├─check-parallel───nsexec───check───042
├─check-parallel───nsexec───check───426───open_by_handle
├─check-parallel───nsexec───check───756───2*[open_by_handle]
├─check-parallel───nsexec───check───227
├─check-parallel───nsexec───check───208───aio-dio-invalid───2*[aio-dio-invalid]
├─check-parallel───nsexec───check───746───cp
├─check-parallel───nsexec───check───187───187
├─check-parallel───nsexec───check───027───8*[027]
├─check-parallel───nsexec───check───045───xfs_io───{xfs_io}
├─check-parallel───nsexec───check───044
├─check-parallel───nsexec───check───204
├─check-parallel───nsexec───check───186───186
├─check-parallel───nsexec───check───449
├─check-parallel───nsexec───check───231───su───fsx
├─check-parallel───nsexec───check───509
├─check-parallel───nsexec───check───127───5*[127───fsx]
├─check-parallel───nsexec───check───047
├─check-parallel───nsexec───check───043
├─check-parallel───nsexec───check───475───pkill
├─check-parallel───nsexec───check───299─┬─fio─┬─4*[fio]
│ │ ├─2*[fio───4*[{fio}]]
│ │ └─{fio}
│ └─pgrep
├─check-parallel───nsexec───check───551───aio-dio-write-v
├─check-parallel───nsexec───check───323───aio-last-ref-he───100*[{aio-last-ref-he}]
├─check-parallel───nsexec───check───648───sleep
├─check-parallel───nsexec───check───046
├─check-parallel───nsexec───check───753─┬─753.fsstress───4*[753.fsstress]
│ └─pkill
├─check-parallel───nsexec───check───507───507
├─check-parallel───nsexec───check───629─┬─3*[629───xfs_io───{xfs_io}]
│ └─5*[629]
├─check-parallel───nsexec───check───073───umount
├─check-parallel───nsexec───check───615───615
├─check-parallel───nsexec───check───176───punch-alternati
├─check-parallel───nsexec───check───294
├─check-parallel───nsexec───check───236───236
├─check-parallel───nsexec───check───165─┬─165─┬─165─┬─cut
│ │ │ └─xfs_io───{xfs_io}
│ │ └─165───grep
│ └─165
├─check-parallel───nsexec───check───259───sync
├─check-parallel───nsexec───check───442───442.fsstress───4*[442.fsstress───{442.fsstress}]
├─check-parallel───nsexec───check───558───255*[558]
├─check-parallel───nsexec───check───358───358───358
├─check-parallel───nsexec───check───169───169
└─check-parallel───nsexec───check───297─┬─297.fsstress─┬─284*[297.fsstress───{297.fsstress}]
│ └─716*[297.fsstress]
└─sleepA typical test run looks like:
$ time sudo ./check-parallel /mnt/xfs -s xfs -x dump Runner 63 Failures: xfs/170 Runner 36 Failures: xfs/050 Runner 30 Failures: xfs/273 Runner 29 Failures: generic/135 Runner 25 Failures: generic/603 Tests run: 1140 Failure count: 5
Ten slowest tests - runtime in seconds:
xfs/013 454
generic/707 414
generic/017 398
generic/387 395
generic/748 390
xfs/140 351
generic/562 351
generic/705 347
generic/251 344
xfs/016 343
Cleanup on Aisle 5?
total 0
crw-------. 1 root root 10, 236 Nov 27 09:27 control
lrwxrwxrwx. 1 root root 7 Nov 27 09:27 fast -> ../dm-0/dev/mapper/fast 1.4T 192G 1.2T 14% /mnt/xfs
real 9m29.056s user 0m0.005s sys 0m0.022s
$
Yeah, that runtime is real - under 10 minutes for a full XFS auto group test run. When running this normally (i.e. via check) on this machine, it usually takes just under 4 hours to run the same set of tests. i.e. I can run ./check-parallel roughly 25x times on this machine in the same time it takes to run ./check.
--- check | 7 +- check-parallel | 205 +++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 208 insertions(+), 4 deletions(-) create mode 100755 check-parallel
check +3 -4
@@ -33,7 +33,7 @@ exclude_tests=() _err_msg="" # start the initialisation work now -iam=check +iam=check.$$ # mkfs.xfs uses the presence of both of these variables to enable formerly # supported tiny filesystem configurations that fstests use for fuzz testing
@@ -460,7 +460,7 @@ fi _wrapup() { - seq="check" + seq="check.$$" check="$RESULT_BASE/check" $interrupt && sect_stop=`_wallclock`
@@ -552,7 +552,6 @@ _wrapup() sum_bad=`expr $sum_bad + ${#bad[*]}` _wipe_counters - rm -f /tmp/*.rawout /tmp/*.out /tmp/*.err /tmp/*.time if ! $OPTIONS_HAVE_SECTIONS; then rm -f $tmp.* fi
@@ -808,7 +807,7 @@ function run_section() init_rc - seq="check" + seq="check.$$" check="$RESULT_BASE/check" # don't leave old full output behind on a clean run
check-parallel new file +205 -0
205 lines not on this page · raw message
Signature · 1 line
2.45.2
Quoted 7 lines · Dave Chinner
Hi folks,
This patchset introduces the ability to run fstests concurrently instead of serially as the current check script does. A git branch containing this patchset can be pulled from here:
https://git.kernel.org/pub/scm/linux/kernel/git/dgc/xfstests-dev.git check-parallel
Hi Dave,
I've merged your "check-parallel" branch, and rebase on fstests' patches-in-queue branch (which is nearly the next release). I just pushed a new branch "for-dave-check-parallel" which fixed all conflicts. It'll be "next next" release, feel free to update base on that. I'll test that branch too :)
Thanks,
Zorro
Quoted 774 lines
The motivation for this is the ever growing runtime of fstests as more tests are added, along with the extremely low resource usage of individual tests. This means that a typical machine running fstests is under-utilised and a single auto group test set execution takes hours.
Yes, I know that I could scale out testing by running lots of little VMs at once, and I already do that. The problem is that I don't get back a complete auto test run result for hours. Want to check that a one-line fix has not caused any regressions? That's, at minimum, an overnight wait for the test farm to crunch through a dozen configs.
On my 64p/128GB RAM VM, 'check -g auto -s xfs -x dump' typically takes around 230-240 minutes to run. With this patchset applied, it runs the same set of tests in 8-10 minutes. I can run ~25 complete auto group test sets with check-parallel in the same time it takes check to run one.
IOWs, I can have the most common config complete a full regression test run as fast as I can turn around a new kernel with a new change to test. I have CPU, memory and IO to burn in my test machines, but what I lack is instant feedback for the change I just made. check-parallel is fast enough that it gives me pretty much instant feedback....
Most of this patchset is preparing infrastructure for concurrent test execution and fixing bugs in tests that I've found whilst getting concurrent execution working reliably. The infrastructure changes center around:
- getting rid of killall - there's nothing quite like one test killing the processes of 15 other tests at the same time...
- cleaning up background process instantiation and reaping, which is a lot more important when an interrupt needs to kill hundreds of processes instead of just a couple.
- isolating error reporting (e.g. dmesg filtering) so that one test failure doesn't trigger lots of other false failure detections
- sanitising the use of loopback devices
- avoiding the use of fixed device names - multiple tests need to use dm-flakey, dm-error, etc devices at the same time, so each test needs a unique device name
- marking tests that are unreliable when outside perturbations occur. These include tests that expect to find delalloc extents, write specific offset patterns in memory then sync them to create specific layouts, etc. e.g. If another test runs sync(1), then those write patterns no longer produce the expected output.
- taming tests that weren't designed for high CPU count machines
- replacingudevadm settlecalls because udev is -never- idle when there are tens to hundreds of block devices and filesystems being rapidly set up and torn down.
- converting sync(1) and sync(2) to syncfs(2) to avoid ihaving hundreds of concurrent superblock list traversals lock-stepping with multiple mount/unmounts every second.There are lots of little other things, but those are the main test and test infrastructure changes. Some of these are big - the fsstress execution rework touches 105 files, but it now means that every single fsstress execution in fstests is controlled by 4 helper functions:
_run_fsstress()- run fsstress synchronously
_run_fsstress_bg- run fsstress in background
_wait_for_fsstress- wait for background fsstress
_kill_fsstress- kill+wait for background fsstressThe test infrastructure also automatically handles cleanup of fsstress processes when the test is interrupted, so tests using fsstress don't need a custom
_cleanup()function just to call_kill_fsstress(). This is something that should have been done a long time ago, but now it is critical for being able to manage multiple independent concurrent fsstress invocations sanely.There are some tests that just can't be run reliably in a concurrent environment - if there is outside interference in, say, page cache flushing then the tests fail. These tests have been added to the "
unreliable_in_parallel" test group with a comment explaining why they are unreliable. The check-parallel script automatically excludes this test group.The only remaining set of tests that are somewhat flakey are the tests that exercise quotas. Quota tests randomly fail for various reasons. Sometimes they don't detect EDQUOT conditions. Sometimes repquota emits weird "device not found" errors. Sometimes grace periods don't start, sometimes they don't time out, or time out and then don't trigger EDQUOT. I don't know why these weird things are happening yet, and until I properly integrate test group selection into check-parallel I can't really isolate the quota tests to focus on them alone.
There are several patches that speed up test runtime. There were several tests that were taking 12-15 minutes to run each, and these made up >95% of the entire check-parallel runtime. In general, these tests have been made to run with more concurrency to speed them up. the result is that the longest individual test runtime has dropped to around 7 minutes, and the elapsed runtime for check-parallel has dropped to 8-10 minutes.
Hence there are 39 patches that are doing prep work on tests and infrastructure to make tests run reliabily in concurrent test contexts. The last patch is the check-parallel runner script that runs the tests concurrently.
The check-parallel script is still very rudimentary. I hard coded the tests it runs (generic+xfs auto tests) and the concurrency so that I could run explicit sets of tests with './check --exact-order <list>'. It is pointed at a mounted directory, and it creates all the infrastructure it needs to run the tests within that directory.
This enabled me to break up the tests across a set of identical runner process contexts. Each runner does:
- create runner directory
- create test and scratch image files
- create loopback devices for test and scratch devices
- sets up results directories
- execute check in it's own private mount namespace so it can't see any of the mounts that other runners are using.
- tears down loopback devices
- reports test failures.If you run with the same directory over and over again, then it reuses the same runner infrastructure and test and scratch image files. The results directories are not overwritten as they are date-stamped, hence using the same mount point automatically creates a result archive for later data mining.
A typical target directory (
/mnt/xfs) looks like:
/mnt/xfs/runner-0/.../mnt/xfs/runner-1/...
..../mnt/xfs/runner-63/...And each runner directory:
log
results-2024-11-19-11:22:25/...
results-2024-11-19-12:36:28/...
.....
results-2024-11-27-13:32:42/...
scratch/
scratch.img
test/
test.imgThe log file is the check output for that runner and should look familiar:
SECTION -- xfs FSTYP -- xfs (debug) PLATFORM -- Linux/x86_64 test1 6.12.0-dgc+ #297 SMP PREEMPT_DYNAMIC Wed Nov 27 08:13:06 AEDT 2024 MKFS_OPTIONS -- -f -m rmapbt=1 /dev/loop10
MOUNT_OPTIONS-- -o context=system_u:object_r:root_t:s0/dev/loop10/mnt/xfs/runner-10/scratchgeneric/387 309s generic/551 49s generic/289 2s xfs/274 3s generic/677 3s generic/143 5s generic/304 1s generic/390 4s generic/427 4s xfs/103 1s generic/252 3s xfs/045 1s generic/374 2s generic/002 1s generic/534 1s generic/039 1sgeneric/595 [not run] No encryption support for xfs
xfs/122 [not run] Could not compile test program (see end of /mnt/xfs/runner-10/results-2024-11-27-13:32:42/xfs/xfs/122.full)
xfs/556 [not run]xfs_scrubnot found
Ran: generic/387 generic/551 generic/289 xfs/274 generic/677 generic/143 generic/304 generic/390 generic/427 xfs/103 generic/252 xfs/045 generic/374 generic/002 generic/534 generic/039 generic/595 xfs/122 xfs/556 Not run: generic/595 xfs/122 xfs/556
Passed all 19 testsSECTION -- xfs
=========================
Ran: generic/387 generic/551 generic/289 xfs/274 generic/677 generic/143 generic/304 generic/390 generic/427 xfs/103 generic/252 xfs/045 generic/374 generic/002 generic/534 generic/039 generic/595 xfs/122 xfs/556 Not run: generic/595 xfs/122 xfs/556
Passed all 19 testsDoing something with all the log files from a run can be done with "
/mnt/xfs/*/log". e.g. grep, vi, etc.The results directory contains the same check.{full,log,time} and test results directories as per a normal check invocation, so there's little difference in checking/analysing results with a parallel execution run.
Because the tests run in private mount namespaces, it's easy to see what check-parallel is running at any point in time using 'pstree -N mnt'. Here's a run that is hung on an unmount not completing:
$ pstree -N mnt [4026531841] bash bash───pstree [0] sudo───sudo───check-parallel─┬─check-parallel───nsexec───check───311───fsync-tester ├─check-parallel───nsexec───check───467───open_by_handle ├─check-parallel───nsexec───check───338 ├─check-parallel───nsexec───check───421 ├─check-parallel───nsexec───check───441 ├─check-parallel───nsexec───check───232 ├─check-parallel───nsexec───check───477───open_by_handle ├─check-parallel───nsexec───check───420 ├─check-parallel───nsexec───check───426───open_by_handle ├─check-parallel───nsexec───check───756───open_by_handle ├─check-parallel───nsexec───check───231 ├─check-parallel───nsexec───check───475───475.fsstress───475.fsstress───{475.fsstress} ├─check-parallel───nsexec───check───388───388.fsstress───388.fsstress───{388.fsstress} ├─check-parallel───nsexec───check───259───sync ├─check-parallel───nsexec───check───622───sync ├─check-parallel───nsexec───check───318───sync ├─check-parallel───nsexec───check───753───umount ├─check-parallel───nsexec───check───086 ├─check-parallel───nsexec───check───648───648.fsstress───648.fsstress───{648.fsstress} ├─check-parallel───nsexec───check───391 ├─check-parallel───nsexec───check───315───sync └─check-parallel───nsexec───check───183───bulkstat_unlinkAll the other processes are in sync or dropping caches and stuck waiting for the superblock
s_umountlock that the unmount holds. Finding where it is stuck:$ pgrep [u]mount 1081885 $ sudo cat /proc/1081885/stack [<0>] xfs_ail_push_all_sync+0x9c/0xf0 [<0>] xfs_unmount_flush_inodes+0x41/0x70 [<0>] xfs_unmountfs+0x59/0x190 [<0>] xfs_fs_put_super+0x3b/0x90 [<0>] generic_shutdown_super+0x77/0x160 [<0>] kill_block_super+0x1b/0x40 [<0>] xfs_kill_sb+0x12/0x30 [<0>] deactivate_locked_super+0x38/0x100 [<0>] deactivate_super+0x41/0x50 [<0>] cleanup_mnt+0x9f/0x160 [<0>] __cleanup_mnt+0x12/0x20 [<0>] task_work_run+0x89/0xb0 [<0>] resume_user_mode_work+0x4f/0x60 [<0>] syscall_exit_to_user_mode+0x76/0xb0 [<0>] do_syscall_64+0x74/0x130 [<0>] entry_SYSCALL_64_after_hwframe+0x76/0x7e $Yup, known bug - a shutdown vs
XFS_ISTALEinode cluster freeing issue that leaves pinned, stale inodes in the AIL.The point I'm making here is that running tests concurrently doesn't change anything material in how you'd go about discovering and diagnosing failures. The only difference is in what that initial failure might look like. e.g. Failures that result in dmesg output will cause a huge number of tests to all fail with "check dmesg for failure output" reports. Hence there is a bit of sifting to find which test triggered the dmesg output, but failure detection still works just fine.
The check-parallel script is really only at proof-of-concept stage. It is sufficient to run tests in parallel, but that's about it. most of the work so far has gone into making the generic and XFS tests run reliably in parallel. I have not run any other tests, and haven't done full conversions of other test directories or C code in src/. e.g. the sync -> syncfs was only done for the common dir and the generic and xfs tests dirs
I do not plan to do these conversions for ext4/btrfs/overlay/etc any time soon as these conversions are mostly about improving execution time, not test correctness. Hence they are not a priority for me - the priority is further developing the concurrent execution environment. i.e. check-parallel.
To that end, I need to factor all the test selection and exclusion code out of check, and do the same with the actual test list runner loop. That way I can reuse all the existing code from within the check-parallel context rather than having to call check to do all of that work itself. I would like to get check-parallel to the point where it is mostly just s/check/check-parallel/ on the command line to move from serial to concurrent test execution.
I also want to try to integrate the config section stuff into check-parallel. This is more a case of defining what devices the config needs to create (i.e. as loop devices) rather than what devices it should be using. I think I can do this just by defining a different set of environment variables (e.g.
NEED_SCRATCHDEV,NEED_LOGDEV, etc) and triggering the loop device creation from these variables.In a way, the fact that check-parallel bootstraps it's own runtime environment almost makes it entirely zero-config. As it stands right now, you should be able to pull this patchset, create your base test directory (make sure you have at least 100GB of free disk space), run './check-parallel <
test_dir> -x dump' and it Should Just Work.At this point, the test infrastructure problems are largely solved. My focus is now on further development of the check-parallel script and integrating it tightly into the existing check infrastructure rather than open-coding test lists and configuration information. This will probably take a bit of time, so I'd like to get the bug fixes, improvements and infrastructure changes underway so I'm not left carrying a huge patchset for months....
----------------------------------------------------------------
Dave Chinner (40):
xfs/448: get rid of assert-on-failure
fstests: cleanup fsstress process management
fuzzy: don't use killall
fstests: per-test dmflakey instances
fstests: per-test dmerror instances
fstests: per-test dmhuge instances
fstests: per-test dmthin instances
fstests: per-test dmdust instances
fstests: per-test dmdelay instances
fstests: fix DM device creation/removal vs udev races
fstests: use syncfs rather than sync
fstests: clean up mount and unmount operations
fstests: clean up loop device instantiation
fstests: xfs/227 is really slow
fstests: mark tests that are unreliable when run in parallel
fstests: use udevadm wait in preference to settle xfs/442: rescale load so it's not exponential xfs/176: fix broken setup code
xfs/177: remove unused slab object count location checks
fstests: remove uses of killall where possible generic/127: reduce runtime
quota: system project quota files need to be shared
dmesg: reduce noise from other tests
fstests: stop using /tmp directly
fstests: scale some tests for high CPU count sanity generic/310: cleanup killing background processes
filter: handle mount errors fromCONFIG_BLK_DEV_WRITE_MOUNTED=y
filters: add a filter that accepts EIO instead of other errors generic/085: general cleanup for reliability and debugging
fstests: don't use directory stacks
fstests: clean up a couple of dm-flakey tests
fstests: clean up termination of various tests
vfstests: some tests require the testdir to be shared xfs/629: single extent files should be within tolerance xfs/076: fix broken mkfs filtering
fstests: capture some failures to seqres.full
fstests: always use fail-at-unmount semantics for XFS generic/062: don't leave debug files in $here on failure
fstests: quota grace periods unreliable under load
fstests: check-parallelcheck | 12 - check-parallel | 205 ++++++++++++++++++ common/btrfs | 4 common/config | 36 ++- common/dmdelay | 24 +- common/dmdust | 14 - common/dmerror | 74 +++--- common/dmflakey | 60 ++--- common/dmhugedisk | 21 + common/dmlogwrites | 4 common/dmthin | 12 - common/encrypt | 2 common/filter | 17 + common/fuzzy | 37 +-- common/log | 2 common/metadump | 32 +- common/overlay | 10 common/populate | 8 common/preamble | 1 common/quota | 37 --- common/rc | 166 +++++++++++--- common/repair | 2 common/report | 2 common/verity | 2 common/xfs | 2 doc/group-names.txt | 1 doc/requirement-checking.txt | 6 ltp/fsstress.c | 28 ++ src/aio-dio-regress/aio-last-ref-held-by-io.c | 5 src/dmerror | 6 tests/btrfs/004 | 11 tests/btrfs/007 | 3 tests/btrfs/012 | 4 tests/btrfs/028 | 6 tests/btrfs/049 | 4 tests/btrfs/057 | 4 tests/btrfs/060 | 14 - tests/btrfs/061 | 13 - tests/btrfs/062 | 13 - tests/btrfs/063 | 13 - tests/btrfs/064 | 13 - tests/btrfs/065 | 14 - tests/btrfs/066 | 14 - tests/btrfs/067 | 14 - tests/btrfs/068 | 14 - tests/btrfs/069 | 13 - tests/btrfs/070 | 13 - tests/btrfs/071 | 13 - tests/btrfs/072 | 14 - tests/btrfs/073 | 13 - tests/btrfs/074 | 13 - tests/btrfs/078 | 12 - tests/btrfs/100 | 4 tests/btrfs/101 | 4 tests/btrfs/136 | 6 tests/btrfs/160 | 3 tests/btrfs/192 | 12 - tests/btrfs/195 | 2 tests/btrfs/212 | 16 - tests/btrfs/232 | 4 tests/btrfs/252 | 5 tests/btrfs/261 | 2 tests/btrfs/284 | 4 tests/btrfs/286 | 2 tests/btrfs/291 | 5 tests/btrfs/320 | 6 tests/btrfs/332 | 4 tests/ext4/004 | 4 tests/ext4/057 | 14 - tests/ext4/058 | 3 tests/ext4/307 | 4 tests/generic/013 | 22 - tests/generic/015 | 2 tests/generic/019 | 13 - tests/generic/029 | 2 tests/generic/030 | 2 tests/generic/032 | 2 tests/generic/034 | 2 tests/generic/039 | 2 tests/generic/040 | 2 tests/generic/041 | 2 tests/generic/042 | 4 tests/generic/048 | 2 tests/generic/049 | 4 tests/generic/050 | 3 tests/generic/051 | 24 -- tests/generic/054 | 2 tests/generic/055 | 6 tests/generic/057 | 2 tests/generic/059 | 2 tests/generic/062 | 2 tests/generic/065 | 2 tests/generic/066 | 5 tests/generic/067 | 17 - tests/generic/068 | 7 tests/generic/070 | 12 - tests/generic/073 | 2 tests/generic/076 | 10 tests/generic/076.out | 1 tests/generic/081 | 8 tests/generic/083 | 13 - tests/generic/083.out | 1 tests/generic/084 | 12 - tests/generic/085 | 21 + tests/generic/090 | 4 tests/generic/092 | 2 tests/generic/098 | 4 tests/generic/099 | 8 tests/generic/101 | 2 tests/generic/104 | 2 tests/generic/106 | 2 tests/generic/107 | 2 tests/generic/108 | 9 tests/generic/109 | 5 tests/generic/117 | 4 tests/generic/127 | 67 +++-- tests/generic/127.out | 6 tests/generic/135 | 24 -- tests/generic/150 | 2 tests/generic/151 | 2 tests/generic/152 | 2 tests/generic/157 | 3 tests/generic/158 | 3 tests/generic/159 | 2 tests/generic/160 | 2 tests/generic/162 | 4 tests/generic/163 | 4 tests/generic/164 | 4 tests/generic/165 | 4 tests/generic/166 | 6 tests/generic/167 | 4 tests/generic/168 | 7 tests/generic/170 | 7 tests/generic/171 | 6 tests/generic/172 | 6 tests/generic/173 | 6 tests/generic/174 | 6 tests/generic/204 | 8 tests/generic/232 | 5 tests/generic/232.out | 1 tests/generic/247 | 2 tests/generic/250 | 2 tests/generic/251 | 5 tests/generic/252 | 5 tests/generic/265 | 2 tests/generic/266 | 2 tests/generic/267 | 2 tests/generic/268 | 2 tests/generic/269 | 8 tests/generic/270 | 10 tests/generic/271 | 2 tests/generic/272 | 2 tests/generic/273 | 2 tests/generic/274 | 6 tests/generic/275 | 6 tests/generic/276 | 2 tests/generic/278 | 2 tests/generic/279 | 2 tests/generic/281 | 2 tests/generic/282 | 2 tests/generic/283 | 2 tests/generic/306 | 2 tests/generic/310 | 24 +- tests/generic/315 | 2 tests/generic/317 | 2 tests/generic/318 | 2 tests/generic/321 | 4 tests/generic/323 | 7 tests/generic/325 | 2 tests/generic/328 | 4 tests/generic/329 | 5 tests/generic/330 | 2 tests/generic/331 | 7 tests/generic/332 | 4 tests/generic/333 | 6 tests/generic/334 | 6 tests/generic/335 | 2 tests/generic/336 | 9 tests/generic/341 | 2 tests/generic/342 | 2 tests/generic/343 | 2 tests/generic/347 | 2 tests/generic/348 | 2 tests/generic/353 | 2 tests/generic/361 | 8 tests/generic/373 | 4 tests/generic/374 | 4 tests/generic/376 | 2 tests/generic/382 | 2 tests/generic/387 | 2 tests/generic/388 | 24 -- tests/generic/390 | 11 tests/generic/391 | 2 tests/generic/395 | 2 tests/generic/409 | 11 tests/generic/410 | 11 tests/generic/411 | 11 tests/generic/416 | 2 tests/generic/422 | 4 tests/generic/425 | 2 tests/generic/441 | 2 tests/generic/459 | 8 tests/generic/461 | 17 - tests/generic/464 | 10 tests/generic/474 | 4 tests/generic/475 | 17 - tests/generic/476 | 15 - tests/generic/479 | 2 tests/generic/480 | 2 tests/generic/482 | 12 - tests/generic/483 | 2 tests/generic/484 | 3 tests/generic/489 | 2 tests/generic/502 | 2 tests/generic/505 | 2 tests/generic/506 | 2 tests/generic/507 | 2 tests/generic/508 | 2 tests/generic/510 | 2 tests/generic/520 | 2 tests/generic/526 | 2 tests/generic/527 | 2 tests/generic/530 | 2 tests/generic/531 | 8 tests/generic/535 | 2 tests/generic/546 | 2 tests/generic/547 | 5 tests/generic/556 | 2 tests/generic/560 | 7 tests/generic/561 | 25 +- tests/generic/563 | 24 +- tests/generic/564 | 12 - tests/generic/579 | 8 tests/generic/585 | 4 tests/generic/589 | 11 tests/generic/590 | 9 tests/generic/599 | 2 tests/generic/601 | 7 tests/generic/603 | 14 - tests/generic/604 | 2 tests/generic/610 | 2 tests/generic/620 | 1 tests/generic/628 | 4 tests/generic/629 | 4 tests/generic/631 | 2 tests/generic/632 | 2 tests/generic/640 | 2 tests/generic/642 | 10 tests/generic/648 | 25 -- tests/generic/650 | 19 - tests/generic/670 | 2 tests/generic/671 | 2 tests/generic/672 | 2 tests/generic/673 | 2 tests/generic/674 | 2 tests/generic/675 | 2 tests/generic/677 | 2 tests/generic/683 | 2 tests/generic/684 | 2 tests/generic/685 | 2 tests/generic/686 | 2 tests/generic/687 | 2 tests/generic/688 | 2 tests/generic/690 | 2 tests/generic/691 | 6 tests/generic/694 | 2 tests/generic/695 | 2 tests/generic/698 | 4 tests/generic/699 | 8 tests/generic/703 | 2 tests/generic/704 | 2 tests/generic/707 | 7 tests/generic/716 | 2 tests/generic/717 | 2 tests/generic/718 | 2 tests/generic/719 | 2 tests/generic/721 | 2 tests/generic/722 | 15 - tests/generic/725 | 2 tests/generic/726 | 2 tests/generic/727 | 2 tests/generic/730 | 2 tests/generic/731 | 2 tests/generic/732 | 4 tests/generic/735 | 2 tests/generic/738 | 2 tests/generic/741 | 2 tests/generic/743 | 4 tests/generic/744 | 10 tests/generic/745 | 4 tests/generic/746 | 18 - tests/generic/747 | 4 tests/generic/749 | 4 tests/generic/750 | 10 tests/generic/751 | 1 tests/generic/753 | 17 - tests/overlay/019 | 48 ++-- tests/overlay/021 | 8 tests/overlay/058 | 12 - tests/xfs/006 | 7 tests/xfs/011 | 11 tests/xfs/013 | 22 - tests/xfs/014 | 11 tests/xfs/016 | 4 tests/xfs/017 | 13 - tests/xfs/017.out | 1 tests/xfs/032 | 2 tests/xfs/049 | 39 ++- tests/xfs/050 | 5 tests/xfs/051 | 10 tests/xfs/052 | 2 tests/xfs/057 | 9 tests/xfs/070 | 9 tests/xfs/073 | 36 +-- tests/xfs/074 | 23 +- tests/xfs/076 | 8 tests/xfs/077 | 2 tests/xfs/078 | 20 - tests/xfs/079 | 17 - tests/xfs/104 | 14 - tests/xfs/110 | 2 tests/xfs/118 | 4 tests/xfs/119 | 2 tests/xfs/128 | 2 tests/xfs/133 | 2 tests/xfs/134 | 2 tests/xfs/137 | 2 tests/xfs/141 | 13 - tests/xfs/148 | 29 +- tests/xfs/149 | 8 tests/xfs/154 | 1 tests/xfs/158 | 4 tests/xfs/161 | 2 tests/xfs/167 | 17 - tests/xfs/168 | 8 tests/xfs/176 | 10 tests/xfs/177 | 15 - tests/xfs/186 | 4 tests/xfs/195 | 2 tests/xfs/201 | 4 tests/xfs/212 | 2 tests/xfs/216 | 23 +- tests/xfs/217 | 24 +- tests/xfs/227 | 59 +++-- tests/xfs/231 | 4 tests/xfs/232 | 10 tests/xfs/234 | 2 tests/xfs/236 | 2 tests/xfs/237 | 13 - tests/xfs/239 | 2 tests/xfs/240 | 7 tests/xfs/241 | 2 tests/xfs/243 | 11 tests/xfs/246 | 2 tests/xfs/250 | 19 + tests/xfs/259 | 13 - tests/xfs/264 | 4 tests/xfs/265 | 2 tests/xfs/270 | 2 tests/xfs/272 | 2 tests/xfs/274 | 2 tests/xfs/276 | 2 tests/xfs/289 | 4 tests/xfs/291 | 4 tests/xfs/297 | 12 - tests/xfs/300 | 8 tests/xfs/305 | 8 tests/xfs/309 | 2 tests/xfs/312 | 2 tests/xfs/313 | 2 tests/xfs/314 | 2 tests/xfs/315 | 4 tests/xfs/316 | 2 tests/xfs/317 | 2 tests/xfs/318 | 4 tests/xfs/319 | 2 tests/xfs/320 | 2 tests/xfs/321 | 2 tests/xfs/322 | 2 tests/xfs/323 | 2 tests/xfs/324 | 2 tests/xfs/325 | 4 tests/xfs/326 | 4 tests/xfs/327 | 2 tests/xfs/420 | 2 tests/xfs/421 | 2 tests/xfs/423 | 2 tests/xfs/438 | 2 tests/xfs/440 | 8 tests/xfs/442 | 15 - tests/xfs/448 | 6 tests/xfs/495 | 2 tests/xfs/501 | 2 tests/xfs/502 | 2 tests/xfs/507 | 2 tests/xfs/511 | 2 tests/xfs/513 | 48 +--- tests/xfs/519 | 2 tests/xfs/520 | 2 tests/xfs/521 | 8 tests/xfs/527 | 5 tests/xfs/528 | 10 tests/xfs/530 | 11 tests/xfs/538 | 4 tests/xfs/541 | 2 tests/xfs/544 | 2 tests/xfs/553 | 4 tests/xfs/558 | 7 tests/xfs/601 | 2 tests/xfs/606 | 14 - tests/xfs/607 | 6 tests/xfs/609 | 20 - tests/xfs/610 | 20 - tests/xfs/613 | 44 +-- tests/xfs/613.out | 1 tests/xfs/617 | 2 tests/xfs/629 | 6 tests/xfs/630 | 2 tests/xfs/631 | 9 tests/xfs/790 | 2 tests/xfs/791 | 2 tests/xfs/792 | 2 tests/xfs/802 | 7 423 files changed, 1827 insertions(+), 1629 deletions(-)
Quoted 9 lines · Zorro Lang
Quoted 7 lines · Dave Chinner
Hi folks,
This patchset introduces the ability to run fstests concurrently instead of serially as the current check script does. A git branch containing this patchset can be pulled from here:
https://git.kernel.org/pub/scm/linux/kernel/git/dgc/xfstests-dev.git check-parallel
Quoted 7 lines
Hi Dave,
I've merged your "check-parallel" branch, and rebase on fstests' patches-in-queue branch (which is nearly the next release). I just pushed a new branch "for-dave-check-parallel" which fixed all conflicts. It'll be "next next" release, feel free to update base on that. I'll test that branch too :)
I ran this through my test infrastructure at zorro's request. I saw a bunch of loop dev errors trickle out:
129.out.bad renamed from 129.out +3 -0
@@ -2,3 +2,6 @@ Create the original file blocks Reflink every other block Create metadump file, restore it and check restored fs +losetup: /dev/loop0: detach failed: No such device or address +Cannot destroy loop device /dev/loop0 +(see /var/tmp/fstests/xfs/129.full for details)
and I noticed the runtimes for running serially went way up. Not sure if that was because my dev tree has a bunch of metadir fixes in it or not; will run that again over the weekend with upstream tot to see if it that brings the total runtime back down.
--D
Quoted 779 lines
Thanks,
ZorroQuoted 774 lines
The motivation for this is the ever growing runtime of fstests as more tests are added, along with the extremely low resource usage of individual tests. This means that a typical machine running fstests is under-utilised and a single auto group test set execution takes hours.
Yes, I know that I could scale out testing by running lots of little VMs at once, and I already do that. The problem is that I don't get back a complete auto test run result for hours. Want to check that a one-line fix has not caused any regressions? That's, at minimum, an overnight wait for the test farm to crunch through a dozen configs.
On my 64p/128GB RAM VM, 'check -g auto -s xfs -x dump' typically takes around 230-240 minutes to run. With this patchset applied, it runs the same set of tests in 8-10 minutes. I can run ~25 complete auto group test sets with check-parallel in the same time it takes check to run one.
IOWs, I can have the most common config complete a full regression test run as fast as I can turn around a new kernel with a new change to test. I have CPU, memory and IO to burn in my test machines, but what I lack is instant feedback for the change I just made. check-parallel is fast enough that it gives me pretty much instant feedback....
Most of this patchset is preparing infrastructure for concurrent test execution and fixing bugs in tests that I've found whilst getting concurrent execution working reliably. The infrastructure changes center around:
- getting rid of killall - there's nothing quite like one test killing the processes of 15 other tests at the same time...
- cleaning up background process instantiation and reaping, which is a lot more important when an interrupt needs to kill hundreds of processes instead of just a couple.
- isolating error reporting (e.g. dmesg filtering) so that one test failure doesn't trigger lots of other false failure detections
- sanitising the use of loopback devices
- avoiding the use of fixed device names - multiple tests need to use dm-flakey, dm-error, etc devices at the same time, so each test needs a unique device name
- marking tests that are unreliable when outside perturbations occur. These include tests that expect to find delalloc extents, write specific offset patterns in memory then sync them to create specific layouts, etc. e.g. If another test runs sync(1), then those write patterns no longer produce the expected output.
- taming tests that weren't designed for high CPU count machines
- replacingudevadm settlecalls because udev is -never- idle when there are tens to hundreds of block devices and filesystems being rapidly set up and torn down.
- converting sync(1) and sync(2) to syncfs(2) to avoid ihaving hundreds of concurrent superblock list traversals lock-stepping with multiple mount/unmounts every second.There are lots of little other things, but those are the main test and test infrastructure changes. Some of these are big - the fsstress execution rework touches 105 files, but it now means that every single fsstress execution in fstests is controlled by 4 helper functions:
_run_fsstress()- run fsstress synchronously
_run_fsstress_bg- run fsstress in background
_wait_for_fsstress- wait for background fsstress
_kill_fsstress- kill+wait for background fsstressThe test infrastructure also automatically handles cleanup of fsstress processes when the test is interrupted, so tests using fsstress don't need a custom
_cleanup()function just to call_kill_fsstress(). This is something that should have been done a long time ago, but now it is critical for being able to manage multiple independent concurrent fsstress invocations sanely.There are some tests that just can't be run reliably in a concurrent environment - if there is outside interference in, say, page cache flushing then the tests fail. These tests have been added to the "
unreliable_in_parallel" test group with a comment explaining why they are unreliable. The check-parallel script automatically excludes this test group.The only remaining set of tests that are somewhat flakey are the tests that exercise quotas. Quota tests randomly fail for various reasons. Sometimes they don't detect EDQUOT conditions. Sometimes repquota emits weird "device not found" errors. Sometimes grace periods don't start, sometimes they don't time out, or time out and then don't trigger EDQUOT. I don't know why these weird things are happening yet, and until I properly integrate test group selection into check-parallel I can't really isolate the quota tests to focus on them alone.
There are several patches that speed up test runtime. There were several tests that were taking 12-15 minutes to run each, and these made up >95% of the entire check-parallel runtime. In general, these tests have been made to run with more concurrency to speed them up. the result is that the longest individual test runtime has dropped to around 7 minutes, and the elapsed runtime for check-parallel has dropped to 8-10 minutes.
Hence there are 39 patches that are doing prep work on tests and infrastructure to make tests run reliabily in concurrent test contexts. The last patch is the check-parallel runner script that runs the tests concurrently.
The check-parallel script is still very rudimentary. I hard coded the tests it runs (generic+xfs auto tests) and the concurrency so that I could run explicit sets of tests with './check --exact-order <list>'. It is pointed at a mounted directory, and it creates all the infrastructure it needs to run the tests within that directory.
This enabled me to break up the tests across a set of identical runner process contexts. Each runner does:
- create runner directory
- create test and scratch image files
- create loopback devices for test and scratch devices
- sets up results directories
- execute check in it's own private mount namespace so it can't see any of the mounts that other runners are using.
- tears down loopback devices
- reports test failures.If you run with the same directory over and over again, then it reuses the same runner infrastructure and test and scratch image files. The results directories are not overwritten as they are date-stamped, hence using the same mount point automatically creates a result archive for later data mining.
A typical target directory (
/mnt/xfs) looks like:
/mnt/xfs/runner-0/.../mnt/xfs/runner-1/...
..../mnt/xfs/runner-63/...And each runner directory:
log
results-2024-11-19-11:22:25/...
results-2024-11-19-12:36:28/...
.....
results-2024-11-27-13:32:42/...
scratch/
scratch.img
test/
test.imgThe log file is the check output for that runner and should look familiar:
SECTION -- xfs FSTYP -- xfs (debug) PLATFORM -- Linux/x86_64 test1 6.12.0-dgc+ #297 SMP PREEMPT_DYNAMIC Wed Nov 27 08:13:06 AEDT 2024 MKFS_OPTIONS -- -f -m rmapbt=1 /dev/loop10
MOUNT_OPTIONS-- -o context=system_u:object_r:root_t:s0/dev/loop10/mnt/xfs/runner-10/scratchgeneric/387 309s generic/551 49s generic/289 2s xfs/274 3s generic/677 3s generic/143 5s generic/304 1s generic/390 4s generic/427 4s xfs/103 1s generic/252 3s xfs/045 1s generic/374 2s generic/002 1s generic/534 1s generic/039 1sgeneric/595 [not run] No encryption support for xfs
xfs/122 [not run] Could not compile test program (see end of /mnt/xfs/runner-10/results-2024-11-27-13:32:42/xfs/xfs/122.full)
xfs/556 [not run]xfs_scrubnot found
Ran: generic/387 generic/551 generic/289 xfs/274 generic/677 generic/143 generic/304 generic/390 generic/427 xfs/103 generic/252 xfs/045 generic/374 generic/002 generic/534 generic/039 generic/595 xfs/122 xfs/556 Not run: generic/595 xfs/122 xfs/556
Passed all 19 testsSECTION -- xfs
=========================
Ran: generic/387 generic/551 generic/289 xfs/274 generic/677 generic/143 generic/304 generic/390 generic/427 xfs/103 generic/252 xfs/045 generic/374 generic/002 generic/534 generic/039 generic/595 xfs/122 xfs/556 Not run: generic/595 xfs/122 xfs/556
Passed all 19 testsDoing something with all the log files from a run can be done with "
/mnt/xfs/*/log". e.g. grep, vi, etc.The results directory contains the same check.{full,log,time} and test results directories as per a normal check invocation, so there's little difference in checking/analysing results with a parallel execution run.
Because the tests run in private mount namespaces, it's easy to see what check-parallel is running at any point in time using 'pstree -N mnt'. Here's a run that is hung on an unmount not completing:
$ pstree -N mnt [4026531841] bash bash───pstree [0] sudo───sudo───check-parallel─┬─check-parallel───nsexec───check───311───fsync-tester ├─check-parallel───nsexec───check───467───open_by_handle ├─check-parallel───nsexec───check───338 ├─check-parallel───nsexec───check───421 ├─check-parallel───nsexec───check───441 ├─check-parallel───nsexec───check───232 ├─check-parallel───nsexec───check───477───open_by_handle ├─check-parallel───nsexec───check───420 ├─check-parallel───nsexec───check───426───open_by_handle ├─check-parallel───nsexec───check───756───open_by_handle ├─check-parallel───nsexec───check───231 ├─check-parallel───nsexec───check───475───475.fsstress───475.fsstress───{475.fsstress} ├─check-parallel───nsexec───check───388───388.fsstress───388.fsstress───{388.fsstress} ├─check-parallel───nsexec───check───259───sync ├─check-parallel───nsexec───check───622───sync ├─check-parallel───nsexec───check───318───sync ├─check-parallel───nsexec───check───753───umount ├─check-parallel───nsexec───check───086 ├─check-parallel───nsexec───check───648───648.fsstress───648.fsstress───{648.fsstress} ├─check-parallel───nsexec───check───391 ├─check-parallel───nsexec───check───315───sync └─check-parallel───nsexec───check───183───bulkstat_unlinkAll the other processes are in sync or dropping caches and stuck waiting for the superblock
s_umountlock that the unmount holds. Finding where it is stuck:$ pgrep [u]mount 1081885 $ sudo cat /proc/1081885/stack [<0>] xfs_ail_push_all_sync+0x9c/0xf0 [<0>] xfs_unmount_flush_inodes+0x41/0x70 [<0>] xfs_unmountfs+0x59/0x190 [<0>] xfs_fs_put_super+0x3b/0x90 [<0>] generic_shutdown_super+0x77/0x160 [<0>] kill_block_super+0x1b/0x40 [<0>] xfs_kill_sb+0x12/0x30 [<0>] deactivate_locked_super+0x38/0x100 [<0>] deactivate_super+0x41/0x50 [<0>] cleanup_mnt+0x9f/0x160 [<0>] __cleanup_mnt+0x12/0x20 [<0>] task_work_run+0x89/0xb0 [<0>] resume_user_mode_work+0x4f/0x60 [<0>] syscall_exit_to_user_mode+0x76/0xb0 [<0>] do_syscall_64+0x74/0x130 [<0>] entry_SYSCALL_64_after_hwframe+0x76/0x7e $Yup, known bug - a shutdown vs
XFS_ISTALEinode cluster freeing issue that leaves pinned, stale inodes in the AIL.The point I'm making here is that running tests concurrently doesn't change anything material in how you'd go about discovering and diagnosing failures. The only difference is in what that initial failure might look like. e.g. Failures that result in dmesg output will cause a huge number of tests to all fail with "check dmesg for failure output" reports. Hence there is a bit of sifting to find which test triggered the dmesg output, but failure detection still works just fine.
The check-parallel script is really only at proof-of-concept stage. It is sufficient to run tests in parallel, but that's about it. most of the work so far has gone into making the generic and XFS tests run reliably in parallel. I have not run any other tests, and haven't done full conversions of other test directories or C code in src/. e.g. the sync -> syncfs was only done for the common dir and the generic and xfs tests dirs
I do not plan to do these conversions for ext4/btrfs/overlay/etc any time soon as these conversions are mostly about improving execution time, not test correctness. Hence they are not a priority for me - the priority is further developing the concurrent execution environment. i.e. check-parallel.
To that end, I need to factor all the test selection and exclusion code out of check, and do the same with the actual test list runner loop. That way I can reuse all the existing code from within the check-parallel context rather than having to call check to do all of that work itself. I would like to get check-parallel to the point where it is mostly just s/check/check-parallel/ on the command line to move from serial to concurrent test execution.
I also want to try to integrate the config section stuff into check-parallel. This is more a case of defining what devices the config needs to create (i.e. as loop devices) rather than what devices it should be using. I think I can do this just by defining a different set of environment variables (e.g.
NEED_SCRATCHDEV,NEED_LOGDEV, etc) and triggering the loop device creation from these variables.In a way, the fact that check-parallel bootstraps it's own runtime environment almost makes it entirely zero-config. As it stands right now, you should be able to pull this patchset, create your base test directory (make sure you have at least 100GB of free disk space), run './check-parallel <
test_dir> -x dump' and it Should Just Work.At this point, the test infrastructure problems are largely solved. My focus is now on further development of the check-parallel script and integrating it tightly into the existing check infrastructure rather than open-coding test lists and configuration information. This will probably take a bit of time, so I'd like to get the bug fixes, improvements and infrastructure changes underway so I'm not left carrying a huge patchset for months....
----------------------------------------------------------------
Dave Chinner (40):
xfs/448: get rid of assert-on-failure
fstests: cleanup fsstress process management
fuzzy: don't use killall
fstests: per-test dmflakey instances
fstests: per-test dmerror instances
fstests: per-test dmhuge instances
fstests: per-test dmthin instances
fstests: per-test dmdust instances
fstests: per-test dmdelay instances
fstests: fix DM device creation/removal vs udev races
fstests: use syncfs rather than sync
fstests: clean up mount and unmount operations
fstests: clean up loop device instantiation
fstests: xfs/227 is really slow
fstests: mark tests that are unreliable when run in parallel
fstests: use udevadm wait in preference to settle xfs/442: rescale load so it's not exponential xfs/176: fix broken setup code
xfs/177: remove unused slab object count location checks
fstests: remove uses of killall where possible generic/127: reduce runtime
quota: system project quota files need to be shared
dmesg: reduce noise from other tests
fstests: stop using /tmp directly
fstests: scale some tests for high CPU count sanity generic/310: cleanup killing background processes
filter: handle mount errors fromCONFIG_BLK_DEV_WRITE_MOUNTED=y
filters: add a filter that accepts EIO instead of other errors generic/085: general cleanup for reliability and debugging
fstests: don't use directory stacks
fstests: clean up a couple of dm-flakey tests
fstests: clean up termination of various tests
vfstests: some tests require the testdir to be shared xfs/629: single extent files should be within tolerance xfs/076: fix broken mkfs filtering
fstests: capture some failures to seqres.full
fstests: always use fail-at-unmount semantics for XFS generic/062: don't leave debug files in $here on failure
fstests: quota grace periods unreliable under load
fstests: check-parallelcheck | 12 - check-parallel | 205 ++++++++++++++++++ common/btrfs | 4 common/config | 36 ++- common/dmdelay | 24 +- common/dmdust | 14 - common/dmerror | 74 +++--- common/dmflakey | 60 ++--- common/dmhugedisk | 21 + common/dmlogwrites | 4 common/dmthin | 12 - common/encrypt | 2 common/filter | 17 + common/fuzzy | 37 +-- common/log | 2 common/metadump | 32 +- common/overlay | 10 common/populate | 8 common/preamble | 1 common/quota | 37 --- common/rc | 166 +++++++++++--- common/repair | 2 common/report | 2 common/verity | 2 common/xfs | 2 doc/group-names.txt | 1 doc/requirement-checking.txt | 6 ltp/fsstress.c | 28 ++ src/aio-dio-regress/aio-last-ref-held-by-io.c | 5 src/dmerror | 6 tests/btrfs/004 | 11 tests/btrfs/007 | 3 tests/btrfs/012 | 4 tests/btrfs/028 | 6 tests/btrfs/049 | 4 tests/btrfs/057 | 4 tests/btrfs/060 | 14 - tests/btrfs/061 | 13 - tests/btrfs/062 | 13 - tests/btrfs/063 | 13 - tests/btrfs/064 | 13 - tests/btrfs/065 | 14 - tests/btrfs/066 | 14 - tests/btrfs/067 | 14 - tests/btrfs/068 | 14 - tests/btrfs/069 | 13 - tests/btrfs/070 | 13 - tests/btrfs/071 | 13 - tests/btrfs/072 | 14 - tests/btrfs/073 | 13 - tests/btrfs/074 | 13 - tests/btrfs/078 | 12 - tests/btrfs/100 | 4 tests/btrfs/101 | 4 tests/btrfs/136 | 6 tests/btrfs/160 | 3 tests/btrfs/192 | 12 - tests/btrfs/195 | 2 tests/btrfs/212 | 16 - tests/btrfs/232 | 4 tests/btrfs/252 | 5 tests/btrfs/261 | 2 tests/btrfs/284 | 4 tests/btrfs/286 | 2 tests/btrfs/291 | 5 tests/btrfs/320 | 6 tests/btrfs/332 | 4 tests/ext4/004 | 4 tests/ext4/057 | 14 - tests/ext4/058 | 3 tests/ext4/307 | 4 tests/generic/013 | 22 - tests/generic/015 | 2 tests/generic/019 | 13 - tests/generic/029 | 2 tests/generic/030 | 2 tests/generic/032 | 2 tests/generic/034 | 2 tests/generic/039 | 2 tests/generic/040 | 2 tests/generic/041 | 2 tests/generic/042 | 4 tests/generic/048 | 2 tests/generic/049 | 4 tests/generic/050 | 3 tests/generic/051 | 24 -- tests/generic/054 | 2 tests/generic/055 | 6 tests/generic/057 | 2 tests/generic/059 | 2 tests/generic/062 | 2 tests/generic/065 | 2 tests/generic/066 | 5 tests/generic/067 | 17 - tests/generic/068 | 7 tests/generic/070 | 12 - tests/generic/073 | 2 tests/generic/076 | 10 tests/generic/076.out | 1 tests/generic/081 | 8 tests/generic/083 | 13 - tests/generic/083.out | 1 tests/generic/084 | 12 - tests/generic/085 | 21 + tests/generic/090 | 4 tests/generic/092 | 2 tests/generic/098 | 4 tests/generic/099 | 8 tests/generic/101 | 2 tests/generic/104 | 2 tests/generic/106 | 2 tests/generic/107 | 2 tests/generic/108 | 9 tests/generic/109 | 5 tests/generic/117 | 4 tests/generic/127 | 67 +++-- tests/generic/127.out | 6 tests/generic/135 | 24 -- tests/generic/150 | 2 tests/generic/151 | 2 tests/generic/152 | 2 tests/generic/157 | 3 tests/generic/158 | 3 tests/generic/159 | 2 tests/generic/160 | 2 tests/generic/162 | 4 tests/generic/163 | 4 tests/generic/164 | 4 tests/generic/165 | 4 tests/generic/166 | 6 tests/generic/167 | 4 tests/generic/168 | 7 tests/generic/170 | 7 tests/generic/171 | 6 tests/generic/172 | 6 tests/generic/173 | 6 tests/generic/174 | 6 tests/generic/204 | 8 tests/generic/232 | 5 tests/generic/232.out | 1 tests/generic/247 | 2 tests/generic/250 | 2 tests/generic/251 | 5 tests/generic/252 | 5 tests/generic/265 | 2 tests/generic/266 | 2 tests/generic/267 | 2 tests/generic/268 | 2 tests/generic/269 | 8 tests/generic/270 | 10 tests/generic/271 | 2 tests/generic/272 | 2 tests/generic/273 | 2 tests/generic/274 | 6 tests/generic/275 | 6 tests/generic/276 | 2 tests/generic/278 | 2 tests/generic/279 | 2 tests/generic/281 | 2 tests/generic/282 | 2 tests/generic/283 | 2 tests/generic/306 | 2 tests/generic/310 | 24 +- tests/generic/315 | 2 tests/generic/317 | 2 tests/generic/318 | 2 tests/generic/321 | 4 tests/generic/323 | 7 tests/generic/325 | 2 tests/generic/328 | 4 tests/generic/329 | 5 tests/generic/330 | 2 tests/generic/331 | 7 tests/generic/332 | 4 tests/generic/333 | 6 tests/generic/334 | 6 tests/generic/335 | 2 tests/generic/336 | 9 tests/generic/341 | 2 tests/generic/342 | 2 tests/generic/343 | 2 tests/generic/347 | 2 tests/generic/348 | 2 tests/generic/353 | 2 tests/generic/361 | 8 tests/generic/373 | 4 tests/generic/374 | 4 tests/generic/376 | 2 tests/generic/382 | 2 tests/generic/387 | 2 tests/generic/388 | 24 -- tests/generic/390 | 11 tests/generic/391 | 2 tests/generic/395 | 2 tests/generic/409 | 11 tests/generic/410 | 11 tests/generic/411 | 11 tests/generic/416 | 2 tests/generic/422 | 4 tests/generic/425 | 2 tests/generic/441 | 2 tests/generic/459 | 8 tests/generic/461 | 17 - tests/generic/464 | 10 tests/generic/474 | 4 tests/generic/475 | 17 - tests/generic/476 | 15 - tests/generic/479 | 2 tests/generic/480 | 2 tests/generic/482 | 12 - tests/generic/483 | 2 tests/generic/484 | 3 tests/generic/489 | 2 tests/generic/502 | 2 tests/generic/505 | 2 tests/generic/506 | 2 tests/generic/507 | 2 tests/generic/508 | 2 tests/generic/510 | 2 tests/generic/520 | 2 tests/generic/526 | 2 tests/generic/527 | 2 tests/generic/530 | 2 tests/generic/531 | 8 tests/generic/535 | 2 tests/generic/546 | 2 tests/generic/547 | 5 tests/generic/556 | 2 tests/generic/560 | 7 tests/generic/561 | 25 +- tests/generic/563 | 24 +- tests/generic/564 | 12 - tests/generic/579 | 8 tests/generic/585 | 4 tests/generic/589 | 11 tests/generic/590 | 9 tests/generic/599 | 2 tests/generic/601 | 7 tests/generic/603 | 14 - tests/generic/604 | 2 tests/generic/610 | 2 tests/generic/620 | 1 tests/generic/628 | 4 tests/generic/629 | 4 tests/generic/631 | 2 tests/generic/632 | 2 tests/generic/640 | 2 tests/generic/642 | 10 tests/generic/648 | 25 -- tests/generic/650 | 19 - tests/generic/670 | 2 tests/generic/671 | 2 tests/generic/672 | 2 tests/generic/673 | 2 tests/generic/674 | 2 tests/generic/675 | 2 tests/generic/677 | 2 tests/generic/683 | 2 tests/generic/684 | 2 tests/generic/685 | 2 tests/generic/686 | 2 tests/generic/687 | 2 tests/generic/688 | 2 tests/generic/690 | 2 tests/generic/691 | 6 tests/generic/694 | 2 tests/generic/695 | 2 tests/generic/698 | 4 tests/generic/699 | 8 tests/generic/703 | 2 tests/generic/704 | 2 tests/generic/707 | 7 tests/generic/716 | 2 tests/generic/717 | 2 tests/generic/718 | 2 tests/generic/719 | 2 tests/generic/721 | 2 tests/generic/722 | 15 - tests/generic/725 | 2 tests/generic/726 | 2 tests/generic/727 | 2 tests/generic/730 | 2 tests/generic/731 | 2 tests/generic/732 | 4 tests/generic/735 | 2 tests/generic/738 | 2 tests/generic/741 | 2 tests/generic/743 | 4 tests/generic/744 | 10 tests/generic/745 | 4 tests/generic/746 | 18 - tests/generic/747 | 4 tests/generic/749 | 4 tests/generic/750 | 10 tests/generic/751 | 1 tests/generic/753 | 17 - tests/overlay/019 | 48 ++-- tests/overlay/021 | 8 tests/overlay/058 | 12 - tests/xfs/006 | 7 tests/xfs/011 | 11 tests/xfs/013 | 22 - tests/xfs/014 | 11 tests/xfs/016 | 4 tests/xfs/017 | 13 - tests/xfs/017.out | 1 tests/xfs/032 | 2 tests/xfs/049 | 39 ++- tests/xfs/050 | 5 tests/xfs/051 | 10 tests/xfs/052 | 2 tests/xfs/057 | 9 tests/xfs/070 | 9 tests/xfs/073 | 36 +-- tests/xfs/074 | 23 +- tests/xfs/076 | 8 tests/xfs/077 | 2 tests/xfs/078 | 20 - tests/xfs/079 | 17 - tests/xfs/104 | 14 - tests/xfs/110 | 2 tests/xfs/118 | 4 tests/xfs/119 | 2 tests/xfs/128 | 2 tests/xfs/133 | 2 tests/xfs/134 | 2 tests/xfs/137 | 2 tests/xfs/141 | 13 - tests/xfs/148 | 29 +- tests/xfs/149 | 8 tests/xfs/154 | 1 tests/xfs/158 | 4 tests/xfs/161 | 2 tests/xfs/167 | 17 - tests/xfs/168 | 8 tests/xfs/176 | 10 tests/xfs/177 | 15 - tests/xfs/186 | 4 tests/xfs/195 | 2 tests/xfs/201 | 4 tests/xfs/212 | 2 tests/xfs/216 | 23 +- tests/xfs/217 | 24 +- tests/xfs/227 | 59 +++-- tests/xfs/231 | 4 tests/xfs/232 | 10 tests/xfs/234 | 2 tests/xfs/236 | 2 tests/xfs/237 | 13 - tests/xfs/239 | 2 tests/xfs/240 | 7 tests/xfs/241 | 2 tests/xfs/243 | 11 tests/xfs/246 | 2 tests/xfs/250 | 19 + tests/xfs/259 | 13 - tests/xfs/264 | 4 tests/xfs/265 | 2 tests/xfs/270 | 2 tests/xfs/272 | 2 tests/xfs/274 | 2 tests/xfs/276 | 2 tests/xfs/289 | 4 tests/xfs/291 | 4 tests/xfs/297 | 12 - tests/xfs/300 | 8 tests/xfs/305 | 8 tests/xfs/309 | 2 tests/xfs/312 | 2 tests/xfs/313 | 2 tests/xfs/314 | 2 tests/xfs/315 | 4 tests/xfs/316 | 2 tests/xfs/317 | 2 tests/xfs/318 | 4 tests/xfs/319 | 2 tests/xfs/320 | 2 tests/xfs/321 | 2 tests/xfs/322 | 2 tests/xfs/323 | 2 tests/xfs/324 | 2 tests/xfs/325 | 4 tests/xfs/326 | 4 tests/xfs/327 | 2 tests/xfs/420 | 2 tests/xfs/421 | 2 tests/xfs/423 | 2 tests/xfs/438 | 2 tests/xfs/440 | 8 tests/xfs/442 | 15 - tests/xfs/448 | 6 tests/xfs/495 | 2 tests/xfs/501 | 2 tests/xfs/502 | 2 tests/xfs/507 | 2 tests/xfs/511 | 2 tests/xfs/513 | 48 +--- tests/xfs/519 | 2 tests/xfs/520 | 2 tests/xfs/521 | 8 tests/xfs/527 | 5 tests/xfs/528 | 10 tests/xfs/530 | 11 tests/xfs/538 | 4 tests/xfs/541 | 2 tests/xfs/544 | 2 tests/xfs/553 | 4 tests/xfs/558 | 7 tests/xfs/601 | 2 tests/xfs/606 | 14 - tests/xfs/607 | 6 tests/xfs/609 | 20 - tests/xfs/610 | 20 - tests/xfs/613 | 44 +-- tests/xfs/613.out | 1 tests/xfs/617 | 2 tests/xfs/629 | 6 tests/xfs/630 | 2 tests/xfs/631 | 9 tests/xfs/790 | 2 tests/xfs/791 | 2 tests/xfs/792 | 2 tests/xfs/802 | 7 423 files changed, 1827 insertions(+), 1629 deletions(-)
Quoted 31 lines · Darrick J. Wong
Quoted 9 lines · Zorro Lang
Quoted 7 lines · Dave Chinner
Hi folks,
This patchset introduces the ability to run fstests concurrently instead of serially as the current check script does. A git branch containing this patchset can be pulled from here:
https://git.kernel.org/pub/scm/linux/kernel/git/dgc/xfstests-dev.git check-parallel
Quoted 7 lines
Hi Dave,
I've merged your "check-parallel" branch, and rebase on fstests' patches-in-queue branch (which is nearly the next release). I just pushed a new branch "for-dave-check-parallel" which fixed all conflicts. It'll be "next next" release, feel free to update base on that. I'll test that branch too :)
I ran this through my test infrastructure at zorro's request. I saw a bunch of loop dev errors trickle out:
129.out.bad renamed from 129.out +3
-0@@ -2,3 +2,6 @@ Create the original file blocks Reflink every other block Create metadump file, restore it and check restored fs +losetup: /dev/loop0: detach failed: No such device or address +Cannot destroy loop device /dev/loop0 +(see /var/tmp/fstests/xfs/129.full for details)
Quoted 4 lines
and I noticed the runtimes for running serially went way up. Not sure if that was because my dev tree has a bunch of metadir fixes in it or not; will run that again over the weekend with upstream tot to see if it that brings the total runtime back down.
Thanks Darrick! The "[PATCH] fstests: clean up loop device instantiation" does below change [1], it looks different with your original code. You trys to check [ -n "$XFS_METADUMP_IMG" ] before loop detaching, and you use a while loop to do the loop detaching. Does this change break your original test?
Thanks,
Zorro
[1]
common/metadump +3 -11
@@ -24,17 +24,9 @@ _xfs_cleanup_verify_metadump() test -n "$XFS_METADUMP_FILE" && rm -f "$XFS_METADUMP_FILE" - if [ -n "$XFS_METADUMP_IMG" ]; then - losetup -n -a -O BACK-FILE,NAME | grep "^$XFS_METADUMP_IMG" | while read backing ldev; do - losetup -d "$ldev" - done - - # Don't call rm directly with a globbed argument here to avoid - # issues issues with variable expansions. - for img in "$XFS_METADUMP_IMG"*; do - test -e "$img" && rm -f "$img" - done - fi + [ -n "$md_data_loop_dev" ] && _destroy_loop_device $md_data_loop_dev + [ -n "$md_log_loop_dev" ] && _destroy_loop_device $md_log_loop_dev + rm -f $data_img $log_img
Quoted 783 lines
--D
Quoted 779 lines
Thanks,
ZorroQuoted 774 lines
The motivation for this is the ever growing runtime of fstests as more tests are added, along with the extremely low resource usage of individual tests. This means that a typical machine running fstests is under-utilised and a single auto group test set execution takes hours.
Yes, I know that I could scale out testing by running lots of little VMs at once, and I already do that. The problem is that I don't get back a complete auto test run result for hours. Want to check that a one-line fix has not caused any regressions? That's, at minimum, an overnight wait for the test farm to crunch through a dozen configs.
On my 64p/128GB RAM VM, 'check -g auto -s xfs -x dump' typically takes around 230-240 minutes to run. With this patchset applied, it runs the same set of tests in 8-10 minutes. I can run ~25 complete auto group test sets with check-parallel in the same time it takes check to run one.
IOWs, I can have the most common config complete a full regression test run as fast as I can turn around a new kernel with a new change to test. I have CPU, memory and IO to burn in my test machines, but what I lack is instant feedback for the change I just made. check-parallel is fast enough that it gives me pretty much instant feedback....
Most of this patchset is preparing infrastructure for concurrent test execution and fixing bugs in tests that I've found whilst getting concurrent execution working reliably. The infrastructure changes center around:
- getting rid of killall - there's nothing quite like one test killing the processes of 15 other tests at the same time...
- cleaning up background process instantiation and reaping, which is a lot more important when an interrupt needs to kill hundreds of processes instead of just a couple.
- isolating error reporting (e.g. dmesg filtering) so that one test failure doesn't trigger lots of other false failure detections
- sanitising the use of loopback devices
- avoiding the use of fixed device names - multiple tests need to use dm-flakey, dm-error, etc devices at the same time, so each test needs a unique device name
- marking tests that are unreliable when outside perturbations occur. These include tests that expect to find delalloc extents, write specific offset patterns in memory then sync them to create specific layouts, etc. e.g. If another test runs sync(1), then those write patterns no longer produce the expected output.
- taming tests that weren't designed for high CPU count machines
- replacingudevadm settlecalls because udev is -never- idle when there are tens to hundreds of block devices and filesystems being rapidly set up and torn down.
- converting sync(1) and sync(2) to syncfs(2) to avoid ihaving hundreds of concurrent superblock list traversals lock-stepping with multiple mount/unmounts every second.There are lots of little other things, but those are the main test and test infrastructure changes. Some of these are big - the fsstress execution rework touches 105 files, but it now means that every single fsstress execution in fstests is controlled by 4 helper functions:
_run_fsstress()- run fsstress synchronously
_run_fsstress_bg- run fsstress in background
_wait_for_fsstress- wait for background fsstress
_kill_fsstress- kill+wait for background fsstressThe test infrastructure also automatically handles cleanup of fsstress processes when the test is interrupted, so tests using fsstress don't need a custom
_cleanup()function just to call_kill_fsstress(). This is something that should have been done a long time ago, but now it is critical for being able to manage multiple independent concurrent fsstress invocations sanely.There are some tests that just can't be run reliably in a concurrent environment - if there is outside interference in, say, page cache flushing then the tests fail. These tests have been added to the "
unreliable_in_parallel" test group with a comment explaining why they are unreliable. The check-parallel script automatically excludes this test group.The only remaining set of tests that are somewhat flakey are the tests that exercise quotas. Quota tests randomly fail for various reasons. Sometimes they don't detect EDQUOT conditions. Sometimes repquota emits weird "device not found" errors. Sometimes grace periods don't start, sometimes they don't time out, or time out and then don't trigger EDQUOT. I don't know why these weird things are happening yet, and until I properly integrate test group selection into check-parallel I can't really isolate the quota tests to focus on them alone.
There are several patches that speed up test runtime. There were several tests that were taking 12-15 minutes to run each, and these made up >95% of the entire check-parallel runtime. In general, these tests have been made to run with more concurrency to speed them up. the result is that the longest individual test runtime has dropped to around 7 minutes, and the elapsed runtime for check-parallel has dropped to 8-10 minutes.
Hence there are 39 patches that are doing prep work on tests and infrastructure to make tests run reliabily in concurrent test contexts. The last patch is the check-parallel runner script that runs the tests concurrently.
The check-parallel script is still very rudimentary. I hard coded the tests it runs (generic+xfs auto tests) and the concurrency so that I could run explicit sets of tests with './check --exact-order <list>'. It is pointed at a mounted directory, and it creates all the infrastructure it needs to run the tests within that directory.
This enabled me to break up the tests across a set of identical runner process contexts. Each runner does:
- create runner directory
- create test and scratch image files
- create loopback devices for test and scratch devices
- sets up results directories
- execute check in it's own private mount namespace so it can't see any of the mounts that other runners are using.
- tears down loopback devices
- reports test failures.If you run with the same directory over and over again, then it reuses the same runner infrastructure and test and scratch image files. The results directories are not overwritten as they are date-stamped, hence using the same mount point automatically creates a result archive for later data mining.
A typical target directory (
/mnt/xfs) looks like:
/mnt/xfs/runner-0/.../mnt/xfs/runner-1/...
..../mnt/xfs/runner-63/...And each runner directory:
log
results-2024-11-19-11:22:25/...
results-2024-11-19-12:36:28/...
.....
results-2024-11-27-13:32:42/...
scratch/
scratch.img
test/
test.imgThe log file is the check output for that runner and should look familiar:
SECTION -- xfs FSTYP -- xfs (debug) PLATFORM -- Linux/x86_64 test1 6.12.0-dgc+ #297 SMP PREEMPT_DYNAMIC Wed Nov 27 08:13:06 AEDT 2024 MKFS_OPTIONS -- -f -m rmapbt=1 /dev/loop10
MOUNT_OPTIONS-- -o context=system_u:object_r:root_t:s0/dev/loop10/mnt/xfs/runner-10/scratchgeneric/387 309s generic/551 49s generic/289 2s xfs/274 3s generic/677 3s generic/143 5s generic/304 1s generic/390 4s generic/427 4s xfs/103 1s generic/252 3s xfs/045 1s generic/374 2s generic/002 1s generic/534 1s generic/039 1sgeneric/595 [not run] No encryption support for xfs
xfs/122 [not run] Could not compile test program (see end of /mnt/xfs/runner-10/results-2024-11-27-13:32:42/xfs/xfs/122.full)
xfs/556 [not run]xfs_scrubnot found
Ran: generic/387 generic/551 generic/289 xfs/274 generic/677 generic/143 generic/304 generic/390 generic/427 xfs/103 generic/252 xfs/045 generic/374 generic/002 generic/534 generic/039 generic/595 xfs/122 xfs/556 Not run: generic/595 xfs/122 xfs/556
Passed all 19 testsSECTION -- xfs
=========================
Ran: generic/387 generic/551 generic/289 xfs/274 generic/677 generic/143 generic/304 generic/390 generic/427 xfs/103 generic/252 xfs/045 generic/374 generic/002 generic/534 generic/039 generic/595 xfs/122 xfs/556 Not run: generic/595 xfs/122 xfs/556
Passed all 19 testsDoing something with all the log files from a run can be done with "
/mnt/xfs/*/log". e.g. grep, vi, etc.The results directory contains the same check.{full,log,time} and test results directories as per a normal check invocation, so there's little difference in checking/analysing results with a parallel execution run.
Because the tests run in private mount namespaces, it's easy to see what check-parallel is running at any point in time using 'pstree -N mnt'. Here's a run that is hung on an unmount not completing:
$ pstree -N mnt [4026531841] bash bash───pstree [0] sudo───sudo───check-parallel─┬─check-parallel───nsexec───check───311───fsync-tester ├─check-parallel───nsexec───check───467───open_by_handle ├─check-parallel───nsexec───check───338 ├─check-parallel───nsexec───check───421 ├─check-parallel───nsexec───check───441 ├─check-parallel───nsexec───check───232 ├─check-parallel───nsexec───check───477───open_by_handle ├─check-parallel───nsexec───check───420 ├─check-parallel───nsexec───check───426───open_by_handle ├─check-parallel───nsexec───check───756───open_by_handle ├─check-parallel───nsexec───check───231 ├─check-parallel───nsexec───check───475───475.fsstress───475.fsstress───{475.fsstress} ├─check-parallel───nsexec───check───388───388.fsstress───388.fsstress───{388.fsstress} ├─check-parallel───nsexec───check───259───sync ├─check-parallel───nsexec───check───622───sync ├─check-parallel───nsexec───check───318───sync ├─check-parallel───nsexec───check───753───umount ├─check-parallel───nsexec───check───086 ├─check-parallel───nsexec───check───648───648.fsstress───648.fsstress───{648.fsstress} ├─check-parallel───nsexec───check───391 ├─check-parallel───nsexec───check───315───sync └─check-parallel───nsexec───check───183───bulkstat_unlinkAll the other processes are in sync or dropping caches and stuck waiting for the superblock
s_umountlock that the unmount holds. Finding where it is stuck:$ pgrep [u]mount 1081885 $ sudo cat /proc/1081885/stack [<0>] xfs_ail_push_all_sync+0x9c/0xf0 [<0>] xfs_unmount_flush_inodes+0x41/0x70 [<0>] xfs_unmountfs+0x59/0x190 [<0>] xfs_fs_put_super+0x3b/0x90 [<0>] generic_shutdown_super+0x77/0x160 [<0>] kill_block_super+0x1b/0x40 [<0>] xfs_kill_sb+0x12/0x30 [<0>] deactivate_locked_super+0x38/0x100 [<0>] deactivate_super+0x41/0x50 [<0>] cleanup_mnt+0x9f/0x160 [<0>] __cleanup_mnt+0x12/0x20 [<0>] task_work_run+0x89/0xb0 [<0>] resume_user_mode_work+0x4f/0x60 [<0>] syscall_exit_to_user_mode+0x76/0xb0 [<0>] do_syscall_64+0x74/0x130 [<0>] entry_SYSCALL_64_after_hwframe+0x76/0x7e $Yup, known bug - a shutdown vs
XFS_ISTALEinode cluster freeing issue that leaves pinned, stale inodes in the AIL.The point I'm making here is that running tests concurrently doesn't change anything material in how you'd go about discovering and diagnosing failures. The only difference is in what that initial failure might look like. e.g. Failures that result in dmesg output will cause a huge number of tests to all fail with "check dmesg for failure output" reports. Hence there is a bit of sifting to find which test triggered the dmesg output, but failure detection still works just fine.
The check-parallel script is really only at proof-of-concept stage. It is sufficient to run tests in parallel, but that's about it. most of the work so far has gone into making the generic and XFS tests run reliably in parallel. I have not run any other tests, and haven't done full conversions of other test directories or C code in src/. e.g. the sync -> syncfs was only done for the common dir and the generic and xfs tests dirs
I do not plan to do these conversions for ext4/btrfs/overlay/etc any time soon as these conversions are mostly about improving execution time, not test correctness. Hence they are not a priority for me - the priority is further developing the concurrent execution environment. i.e. check-parallel.
To that end, I need to factor all the test selection and exclusion code out of check, and do the same with the actual test list runner loop. That way I can reuse all the existing code from within the check-parallel context rather than having to call check to do all of that work itself. I would like to get check-parallel to the point where it is mostly just s/check/check-parallel/ on the command line to move from serial to concurrent test execution.
I also want to try to integrate the config section stuff into check-parallel. This is more a case of defining what devices the config needs to create (i.e. as loop devices) rather than what devices it should be using. I think I can do this just by defining a different set of environment variables (e.g.
NEED_SCRATCHDEV,NEED_LOGDEV, etc) and triggering the loop device creation from these variables.In a way, the fact that check-parallel bootstraps it's own runtime environment almost makes it entirely zero-config. As it stands right now, you should be able to pull this patchset, create your base test directory (make sure you have at least 100GB of free disk space), run './check-parallel <
test_dir> -x dump' and it Should Just Work.At this point, the test infrastructure problems are largely solved. My focus is now on further development of the check-parallel script and integrating it tightly into the existing check infrastructure rather than open-coding test lists and configuration information. This will probably take a bit of time, so I'd like to get the bug fixes, improvements and infrastructure changes underway so I'm not left carrying a huge patchset for months....
----------------------------------------------------------------
Dave Chinner (40):
xfs/448: get rid of assert-on-failure
fstests: cleanup fsstress process management
fuzzy: don't use killall
fstests: per-test dmflakey instances
fstests: per-test dmerror instances
fstests: per-test dmhuge instances
fstests: per-test dmthin instances
fstests: per-test dmdust instances
fstests: per-test dmdelay instances
fstests: fix DM device creation/removal vs udev races
fstests: use syncfs rather than sync
fstests: clean up mount and unmount operations
fstests: clean up loop device instantiation
fstests: xfs/227 is really slow
fstests: mark tests that are unreliable when run in parallel
fstests: use udevadm wait in preference to settle xfs/442: rescale load so it's not exponential xfs/176: fix broken setup code
xfs/177: remove unused slab object count location checks
fstests: remove uses of killall where possible generic/127: reduce runtime
quota: system project quota files need to be shared
dmesg: reduce noise from other tests
fstests: stop using /tmp directly
fstests: scale some tests for high CPU count sanity generic/310: cleanup killing background processes
filter: handle mount errors fromCONFIG_BLK_DEV_WRITE_MOUNTED=y
filters: add a filter that accepts EIO instead of other errors generic/085: general cleanup for reliability and debugging
fstests: don't use directory stacks
fstests: clean up a couple of dm-flakey tests
fstests: clean up termination of various tests
vfstests: some tests require the testdir to be shared xfs/629: single extent files should be within tolerance xfs/076: fix broken mkfs filtering
fstests: capture some failures to seqres.full
fstests: always use fail-at-unmount semantics for XFS generic/062: don't leave debug files in $here on failure
fstests: quota grace periods unreliable under load
fstests: check-parallelcheck | 12 - check-parallel | 205 ++++++++++++++++++ common/btrfs | 4 common/config | 36 ++- common/dmdelay | 24 +- common/dmdust | 14 - common/dmerror | 74 +++--- common/dmflakey | 60 ++--- common/dmhugedisk | 21 + common/dmlogwrites | 4 common/dmthin | 12 - common/encrypt | 2 common/filter | 17 + common/fuzzy | 37 +-- common/log | 2 common/metadump | 32 +- common/overlay | 10 common/populate | 8 common/preamble | 1 common/quota | 37 --- common/rc | 166 +++++++++++--- common/repair | 2 common/report | 2 common/verity | 2 common/xfs | 2 doc/group-names.txt | 1 doc/requirement-checking.txt | 6 ltp/fsstress.c | 28 ++ src/aio-dio-regress/aio-last-ref-held-by-io.c | 5 src/dmerror | 6 tests/btrfs/004 | 11 tests/btrfs/007 | 3 tests/btrfs/012 | 4 tests/btrfs/028 | 6 tests/btrfs/049 | 4 tests/btrfs/057 | 4 tests/btrfs/060 | 14 - tests/btrfs/061 | 13 - tests/btrfs/062 | 13 - tests/btrfs/063 | 13 - tests/btrfs/064 | 13 - tests/btrfs/065 | 14 - tests/btrfs/066 | 14 - tests/btrfs/067 | 14 - tests/btrfs/068 | 14 - tests/btrfs/069 | 13 - tests/btrfs/070 | 13 - tests/btrfs/071 | 13 - tests/btrfs/072 | 14 - tests/btrfs/073 | 13 - tests/btrfs/074 | 13 - tests/btrfs/078 | 12 - tests/btrfs/100 | 4 tests/btrfs/101 | 4 tests/btrfs/136 | 6 tests/btrfs/160 | 3 tests/btrfs/192 | 12 - tests/btrfs/195 | 2 tests/btrfs/212 | 16 - tests/btrfs/232 | 4 tests/btrfs/252 | 5 tests/btrfs/261 | 2 tests/btrfs/284 | 4 tests/btrfs/286 | 2 tests/btrfs/291 | 5 tests/btrfs/320 | 6 tests/btrfs/332 | 4 tests/ext4/004 | 4 tests/ext4/057 | 14 - tests/ext4/058 | 3 tests/ext4/307 | 4 tests/generic/013 | 22 - tests/generic/015 | 2 tests/generic/019 | 13 - tests/generic/029 | 2 tests/generic/030 | 2 tests/generic/032 | 2 tests/generic/034 | 2 tests/generic/039 | 2 tests/generic/040 | 2 tests/generic/041 | 2 tests/generic/042 | 4 tests/generic/048 | 2 tests/generic/049 | 4 tests/generic/050 | 3 tests/generic/051 | 24 -- tests/generic/054 | 2 tests/generic/055 | 6 tests/generic/057 | 2 tests/generic/059 | 2 tests/generic/062 | 2 tests/generic/065 | 2 tests/generic/066 | 5 tests/generic/067 | 17 - tests/generic/068 | 7 tests/generic/070 | 12 - tests/generic/073 | 2 tests/generic/076 | 10 tests/generic/076.out | 1 tests/generic/081 | 8 tests/generic/083 | 13 - tests/generic/083.out | 1 tests/generic/084 | 12 - tests/generic/085 | 21 + tests/generic/090 | 4 tests/generic/092 | 2 tests/generic/098 | 4 tests/generic/099 | 8 tests/generic/101 | 2 tests/generic/104 | 2 tests/generic/106 | 2 tests/generic/107 | 2 tests/generic/108 | 9 tests/generic/109 | 5 tests/generic/117 | 4 tests/generic/127 | 67 +++-- tests/generic/127.out | 6 tests/generic/135 | 24 -- tests/generic/150 | 2 tests/generic/151 | 2 tests/generic/152 | 2 tests/generic/157 | 3 tests/generic/158 | 3 tests/generic/159 | 2 tests/generic/160 | 2 tests/generic/162 | 4 tests/generic/163 | 4 tests/generic/164 | 4 tests/generic/165 | 4 tests/generic/166 | 6 tests/generic/167 | 4 tests/generic/168 | 7 tests/generic/170 | 7 tests/generic/171 | 6 tests/generic/172 | 6 tests/generic/173 | 6 tests/generic/174 | 6 tests/generic/204 | 8 tests/generic/232 | 5 tests/generic/232.out | 1 tests/generic/247 | 2 tests/generic/250 | 2 tests/generic/251 | 5 tests/generic/252 | 5 tests/generic/265 | 2 tests/generic/266 | 2 tests/generic/267 | 2 tests/generic/268 | 2 tests/generic/269 | 8 tests/generic/270 | 10 tests/generic/271 | 2 tests/generic/272 | 2 tests/generic/273 | 2 tests/generic/274 | 6 tests/generic/275 | 6 tests/generic/276 | 2 tests/generic/278 | 2 tests/generic/279 | 2 tests/generic/281 | 2 tests/generic/282 | 2 tests/generic/283 | 2 tests/generic/306 | 2 tests/generic/310 | 24 +- tests/generic/315 | 2 tests/generic/317 | 2 tests/generic/318 | 2 tests/generic/321 | 4 tests/generic/323 | 7 tests/generic/325 | 2 tests/generic/328 | 4 tests/generic/329 | 5 tests/generic/330 | 2 tests/generic/331 | 7 tests/generic/332 | 4 tests/generic/333 | 6 tests/generic/334 | 6 tests/generic/335 | 2 tests/generic/336 | 9 tests/generic/341 | 2 tests/generic/342 | 2 tests/generic/343 | 2 tests/generic/347 | 2 tests/generic/348 | 2 tests/generic/353 | 2 tests/generic/361 | 8 tests/generic/373 | 4 tests/generic/374 | 4 tests/generic/376 | 2 tests/generic/382 | 2 tests/generic/387 | 2 tests/generic/388 | 24 -- tests/generic/390 | 11 tests/generic/391 | 2 tests/generic/395 | 2 tests/generic/409 | 11 tests/generic/410 | 11 tests/generic/411 | 11 tests/generic/416 | 2 tests/generic/422 | 4 tests/generic/425 | 2 tests/generic/441 | 2 tests/generic/459 | 8 tests/generic/461 | 17 - tests/generic/464 | 10 tests/generic/474 | 4 tests/generic/475 | 17 - tests/generic/476 | 15 - tests/generic/479 | 2 tests/generic/480 | 2 tests/generic/482 | 12 - tests/generic/483 | 2 tests/generic/484 | 3 tests/generic/489 | 2 tests/generic/502 | 2 tests/generic/505 | 2 tests/generic/506 | 2 tests/generic/507 | 2 tests/generic/508 | 2 tests/generic/510 | 2 tests/generic/520 | 2 tests/generic/526 | 2 tests/generic/527 | 2 tests/generic/530 | 2 tests/generic/531 | 8 tests/generic/535 | 2 tests/generic/546 | 2 tests/generic/547 | 5 tests/generic/556 | 2 tests/generic/560 | 7 tests/generic/561 | 25 +- tests/generic/563 | 24 +- tests/generic/564 | 12 - tests/generic/579 | 8 tests/generic/585 | 4 tests/generic/589 | 11 tests/generic/590 | 9 tests/generic/599 | 2 tests/generic/601 | 7 tests/generic/603 | 14 - tests/generic/604 | 2 tests/generic/610 | 2 tests/generic/620 | 1 tests/generic/628 | 4 tests/generic/629 | 4 tests/generic/631 | 2 tests/generic/632 | 2 tests/generic/640 | 2 tests/generic/642 | 10 tests/generic/648 | 25 -- tests/generic/650 | 19 - tests/generic/670 | 2 tests/generic/671 | 2 tests/generic/672 | 2 tests/generic/673 | 2 tests/generic/674 | 2 tests/generic/675 | 2 tests/generic/677 | 2 tests/generic/683 | 2 tests/generic/684 | 2 tests/generic/685 | 2 tests/generic/686 | 2 tests/generic/687 | 2 tests/generic/688 | 2 tests/generic/690 | 2 tests/generic/691 | 6 tests/generic/694 | 2 tests/generic/695 | 2 tests/generic/698 | 4 tests/generic/699 | 8 tests/generic/703 | 2 tests/generic/704 | 2 tests/generic/707 | 7 tests/generic/716 | 2 tests/generic/717 | 2 tests/generic/718 | 2 tests/generic/719 | 2 tests/generic/721 | 2 tests/generic/722 | 15 - tests/generic/725 | 2 tests/generic/726 | 2 tests/generic/727 | 2 tests/generic/730 | 2 tests/generic/731 | 2 tests/generic/732 | 4 tests/generic/735 | 2 tests/generic/738 | 2 tests/generic/741 | 2 tests/generic/743 | 4 tests/generic/744 | 10 tests/generic/745 | 4 tests/generic/746 | 18 - tests/generic/747 | 4 tests/generic/749 | 4 tests/generic/750 | 10 tests/generic/751 | 1 tests/generic/753 | 17 - tests/overlay/019 | 48 ++-- tests/overlay/021 | 8 tests/overlay/058 | 12 - tests/xfs/006 | 7 tests/xfs/011 | 11 tests/xfs/013 | 22 - tests/xfs/014 | 11 tests/xfs/016 | 4 tests/xfs/017 | 13 - tests/xfs/017.out | 1 tests/xfs/032 | 2 tests/xfs/049 | 39 ++- tests/xfs/050 | 5 tests/xfs/051 | 10 tests/xfs/052 | 2 tests/xfs/057 | 9 tests/xfs/070 | 9 tests/xfs/073 | 36 +-- tests/xfs/074 | 23 +- tests/xfs/076 | 8 tests/xfs/077 | 2 tests/xfs/078 | 20 - tests/xfs/079 | 17 - tests/xfs/104 | 14 - tests/xfs/110 | 2 tests/xfs/118 | 4 tests/xfs/119 | 2 tests/xfs/128 | 2 tests/xfs/133 | 2 tests/xfs/134 | 2 tests/xfs/137 | 2 tests/xfs/141 | 13 - tests/xfs/148 | 29 +- tests/xfs/149 | 8 tests/xfs/154 | 1 tests/xfs/158 | 4 tests/xfs/161 | 2 tests/xfs/167 | 17 - tests/xfs/168 | 8 tests/xfs/176 | 10 tests/xfs/177 | 15 - tests/xfs/186 | 4 tests/xfs/195 | 2 tests/xfs/201 | 4 tests/xfs/212 | 2 tests/xfs/216 | 23 +- tests/xfs/217 | 24 +- tests/xfs/227 | 59 +++-- tests/xfs/231 | 4 tests/xfs/232 | 10 tests/xfs/234 | 2 tests/xfs/236 | 2 tests/xfs/237 | 13 - tests/xfs/239 | 2 tests/xfs/240 | 7 tests/xfs/241 | 2 tests/xfs/243 | 11 tests/xfs/246 | 2 tests/xfs/250 | 19 + tests/xfs/259 | 13 - tests/xfs/264 | 4 tests/xfs/265 | 2 tests/xfs/270 | 2 tests/xfs/272 | 2 tests/xfs/274 | 2 tests/xfs/276 | 2 tests/xfs/289 | 4 tests/xfs/291 | 4 tests/xfs/297 | 12 - tests/xfs/300 | 8 tests/xfs/305 | 8 tests/xfs/309 | 2 tests/xfs/312 | 2 tests/xfs/313 | 2 tests/xfs/314 | 2 tests/xfs/315 | 4 tests/xfs/316 | 2 tests/xfs/317 | 2 tests/xfs/318 | 4 tests/xfs/319 | 2 tests/xfs/320 | 2 tests/xfs/321 | 2 tests/xfs/322 | 2 tests/xfs/323 | 2 tests/xfs/324 | 2 tests/xfs/325 | 4 tests/xfs/326 | 4 tests/xfs/327 | 2 tests/xfs/420 | 2 tests/xfs/421 | 2 tests/xfs/423 | 2 tests/xfs/438 | 2 tests/xfs/440 | 8 tests/xfs/442 | 15 - tests/xfs/448 | 6 tests/xfs/495 | 2 tests/xfs/501 | 2 tests/xfs/502 | 2 tests/xfs/507 | 2 tests/xfs/511 | 2 tests/xfs/513 | 48 +--- tests/xfs/519 | 2 tests/xfs/520 | 2 tests/xfs/521 | 8 tests/xfs/527 | 5 tests/xfs/528 | 10 tests/xfs/530 | 11 tests/xfs/538 | 4 tests/xfs/541 | 2 tests/xfs/544 | 2 tests/xfs/553 | 4 tests/xfs/558 | 7 tests/xfs/601 | 2 tests/xfs/606 | 14 - tests/xfs/607 | 6 tests/xfs/609 | 20 - tests/xfs/610 | 20 - tests/xfs/613 | 44 +-- tests/xfs/613.out | 1 tests/xfs/617 | 2 tests/xfs/629 | 6 tests/xfs/630 | 2 tests/xfs/631 | 9 tests/xfs/790 | 2 tests/xfs/791 | 2 tests/xfs/792 | 2 tests/xfs/802 | 7 423 files changed, 1827 insertions(+), 1629 deletions(-)
Quoted 24 lines · Darrick J. Wong
Quoted 9 lines · Zorro Lang
Quoted 7 lines · Dave Chinner
Hi folks,
This patchset introduces the ability to run fstests concurrently instead of serially as the current check script does. A git branch containing this patchset can be pulled from here:
https://git.kernel.org/pub/scm/linux/kernel/git/dgc/xfstests-dev.git check-parallel
Quoted 7 lines
Hi Dave,
I've merged your "check-parallel" branch, and rebase on fstests' patches-in-queue branch (which is nearly the next release). I just pushed a new branch "for-dave-check-parallel" which fixed all conflicts. It'll be "next next" release, feel free to update base on that. I'll test that branch too :)
I ran this through my test infrastructure at zorro's request. I saw a bunch of loop dev errors trickle out:
129.out.bad line 7
Quoted 6 lines
129.out.bad renamed from 129.out +3
-0@@ -2,3 +2,6 @@ Create the original file blocks Reflink every other block Create metadump file, restore it and check restored fs +losetup: /dev/loop0: detach failed: No such device or address +Cannot destroy loop device /dev/loop0 +(see /var/tmp/fstests/xfs/129.full for details)
Almost certainly I missed the conversion of names in _xfs_verify_metadump_v1() from "data_loop" to "md_data_loop_dev" and such. common/metadump is liley missing "unset md_data_loop_dev" after destroying the loop devices, too.
Not sure why that isn't triggering on my setup, trivial to fix. I'll sort it out and fold it back into the original loopdev cleanup patch in the set.
Quoted 1 line
and I noticed the runtimes for running serially went way up.
Not seeing that here; I don't think any of the changes I've made should affect the runtime of a normal check test pass; the tests should take the same time to run or run faster after this patchset, even serially...
Quoted 4 lines
Not sure
if that was because my dev tree has a bunch of metadir fixes in it or not; will run that again over the weekend with upstream tot to see if it that brings the total runtime back down.
OK.
-Dave.
Signature · 2 lines
Dave Chinner [email protected]
Quoted 41 lines · Dave Chinner
Quoted 24 lines · Darrick J. Wong
Quoted 9 lines · Zorro Lang
Quoted 7 lines · Dave Chinner
Hi folks,
This patchset introduces the ability to run fstests concurrently instead of serially as the current check script does. A git branch containing this patchset can be pulled from here:
https://git.kernel.org/pub/scm/linux/kernel/git/dgc/xfstests-dev.git check-parallel
Quoted 7 lines
Hi Dave,
I've merged your "check-parallel" branch, and rebase on fstests' patches-in-queue branch (which is nearly the next release). I just pushed a new branch "for-dave-check-parallel" which fixed all conflicts. It'll be "next next" release, feel free to update base on that. I'll test that branch too :)
I ran this through my test infrastructure at zorro's request. I saw a bunch of loop dev errors trickle out:
129.out.bad line 7
Quoted 6 lines
129.out.bad renamed from 129.out +3
-0@@ -2,3 +2,6 @@ Create the original file blocks Reflink every other block Create metadump file, restore it and check restored fs +losetup: /dev/loop0: detach failed: No such device or address +Cannot destroy loop device /dev/loop0 +(see /var/tmp/fstests/xfs/129.full for details)Almost certainly I missed the conversion of names in
_xfs_verify_metadump_v1()from "data_loop" to "md_data_loop_dev" and such. common/metadump is liley missing "unsetmd_data_loop_dev" after destroying the loop devices, too.Not sure why that isn't triggering on my setup, trivial to fix. I'll sort it out and fold it back into the original loopdev cleanup patch in the set.
Quoted 6 lines
Quoted 1 line
and I noticed the runtimes for running serially went way up.
Not seeing that here; I don't think any of the changes I've made should affect the runtime of a normal check test pass; the tests should take the same time to run or run faster after this patchset, even serially...
Hi Dave,
I replied several emails for this issue:
I've tried to fix this and all other small issues on "for-dave-check-parallel" branch:
# git clone -b for-dave-check-parallel git://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git
Could you take a look at it? Currently my test passed on this branch. As my original plan, I'd like to merge for-dave-check-parallel branch onto for-next and push in this week.
So if no one has other critical issue to report, I'll push it as plan. Then we can fix later issues, and other's patches can move on.
Thanks,
Zorro
Quoted 13 lines
Quoted 5 lines
Not sure
if that was because my dev tree has a bunch of metadir fixes in it or not; will run that again over the weekend with upstream tot to see if it that brings the total runtime back down.OK.
-Dave.
Signature · 2 lines
Dave Chinner [email protected]
Quoted 55 lines · Zorro Lang
Quoted 41 lines · Dave Chinner
Quoted 24 lines · Darrick J. Wong
Quoted 9 lines · Zorro Lang
Quoted 7 lines · Dave Chinner
Hi folks,
This patchset introduces the ability to run fstests concurrently instead of serially as the current check script does. A git branch containing this patchset can be pulled from here:
https://git.kernel.org/pub/scm/linux/kernel/git/dgc/xfstests-dev.git check-parallel
Quoted 7 lines
Hi Dave,
I've merged your "check-parallel" branch, and rebase on fstests' patches-in-queue branch (which is nearly the next release). I just pushed a new branch "for-dave-check-parallel" which fixed all conflicts. It'll be "next next" release, feel free to update base on that. I'll test that branch too :)
I ran this through my test infrastructure at zorro's request. I saw a bunch of loop dev errors trickle out:
129.out.bad line 7
Quoted 6 lines
129.out.bad renamed from 129.out +3
-0@@ -2,3 +2,6 @@ Create the original file blocks Reflink every other block Create metadump file, restore it and check restored fs +losetup: /dev/loop0: detach failed: No such device or address +Cannot destroy loop device /dev/loop0 +(see /var/tmp/fstests/xfs/129.full for details)Almost certainly I missed the conversion of names in
_xfs_verify_metadump_v1()from "data_loop" to "md_data_loop_dev" and such. common/metadump is liley missing "unsetmd_data_loop_dev" after destroying the loop devices, too.Not sure why that isn't triggering on my setup, trivial to fix. I'll sort it out and fold it back into the original loopdev cleanup patch in the set.
Quoted 6 lines
Quoted 1 line
and I noticed the runtimes for running serially went way up.
Not seeing that here; I don't think any of the changes I've made should affect the runtime of a normal check test pass; the tests should take the same time to run or run faster after this patchset, even serially...
Hi Dave,
I replied several emails for this issue:
Quoted 8 lines
I've tried to fix this and all other small issues on "for-dave-check-parallel" branch:
# git clone -b for-dave-check-parallel git://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.gitCould you take a look at it? Currently my test passed on this branch. As my original plan, I'd like to merge for-dave-check-parallel branch onto for-next and push in this week.
If it fixes the reported failure, then I'm fine with that.
Unfortunately, the machine I've been developing this code on has now been dead for a week and a half (main board failure), and the vendor doesn't seem to care about the "NBD on-site" warranty SLA...
I have no idea when it'll be fixed, so if what you've done works for everyone else right now, I'll clean up the remaining things I noticed when I've got the machine back up and running and can test this stuff again...
-Dave.
Signature · 2 lines
Dave Chinner [email protected]