fstests: sector size fixes and whiteouts...
cover fstests: sector size fixes and whiteouts...
Hi folks,
The first 2 patches are fallout from a kernel change in 4.0-rc1 which changes the physical block size of a ram disk (/dev/ram0) from 512 bytes to 4kB. This bit several tests because it means mkfs.xfs generates different geometry and so has different output.
The third patch addresses another new issue - it appears that loop device tardown is much faster than it's ever been and so it triggering races with unmount trying to tear down it down. unmount is being a sore loser and so tests are failing randomly.
The final patch is a test for RENAME_WHITEOUT. I have no idea if it encodes correct behaviour because the behaviour is completely undocumented. So I've simply made a test that exercises ext4's behaviour and I'm using that to verify the new XFS code matches the same userspace visible behaviour as ext4.
-Dave.
PATCH 1/4 xfs/104: log size too small for 4k sector drives
From: Dave Chinner <[email protected]>
xfs/104, xfs/119 and xfs/291 have small fixed log sizes. A recent change to the kernel ramdisk changed it's physical sector size from 512B to 4kB, and this results in mkfs calculating a log size larger than the fixed test size and hence the tests fail.
Change the log size to a larger size that works with 4k sectors, and also increase the size of the filesystem being created so that the amount of data space in the filesystem does not change and hence does not perturb the rest of the test.
--- tests/xfs/104 | 8 ++++---- tests/xfs/104.out | 46 +++++++++++++++++++++++----------------------- tests/xfs/119 | 2 +- tests/xfs/291 | 2 +- 4 files changed, 29 insertions(+), 29 deletions(-)
tests/xfs/104 +4 -4
@@ -81,10 +81,10 @@ modsize=`expr 4 \* $incsize` # pause after this many increments [ `expr $endsize / $dbsize` -lt $dblocks ] || _notrun "Scratch device too small" nags=4 -size=`expr 120 \* 1048576` # 120 megabytes initially +size=`expr 125 \* 1048576` # 120 megabytes initially sizeb=`expr $size / $dbsize` # in data blocks echo "*** creating scratch filesystem" -_create_scratch -lsize=5m -dsize=${size} -dagcount=${nags} +_create_scratch -lsize=10m -dsize=${size} -dagcount=${nags} fillsize=`expr 110 \* 1048576` # 110 megabytes of filling echo "*** using some initial space on scratch filesystem"
@@ -95,13 +95,13 @@ _fill_scratch $fillsize # Kick off more stress threads on each iteration, grow; repeat. # while [ $size -le $endsize ]; do - echo "*** stressing a ${size} byte filesystem" + echo "*** stressing filesystem" echo "*** stressing a ${sizeb} block filesystem" >> $seqres.full _stress_scratch sleep 1 size=`expr $size + $incsize` sizeb=`expr $size / $dbsize` # in data blocks - echo "*** growing to a ${size} byte filesystem" + echo "*** growing filesystem" echo "*** growing to a ${sizeb} block filesystem" >> $seqres.full xfs_growfs -D ${sizeb} $SCRATCH_MNT \ | tee -a $seqres.full | _filter_mkfs 2>$tmp.growfs
tests/xfs/104.out +23 -23
113 lines not on this page · raw message
tests/xfs/119 +1 -1
@@ -54,7 +54,7 @@ _require_scratch # this may hang sync -export MKFS_OPTIONS="-l version=2,size=1200b,su=64k" +export MKFS_OPTIONS="-l version=2,size=2500b,su=64k" export MOUNT_OPTIONS="-o logbsize=64k" _scratch_mkfs_xfs >/dev/null
tests/xfs/291 +1 -1
@@ -46,7 +46,7 @@ _supported_os IRIX Linux # real QA test starts here rm -f $seqres.full _require_scratch -_scratch_mkfs_xfs -n size=16k -l size=5m -d size=128m >> $seqres.full 2>&1 +_scratch_mkfs_xfs -n size=16k -l size=10m -d size=133m >> $seqres.full 2>&1 _scratch_mount # First we cause very badly fragmented freespace, then
Signature · 1 line
2.0.0
PATCH 2/4 xfs: don't output mkfs sector sizes into golden output
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
The mkfs sector sizes are dependent on the underlying device in use, and so is not fixed. hence it needs to be filtered from any golden output file, otherwise tests that just differ by sector size will fail.
--- tests/xfs/096 | 5 +++-- tests/xfs/206 | 33 +++++++++++++++------------------ 2 files changed, 18 insertions(+), 20 deletions(-)
tests/xfs/096 +3 -2
@@ -63,7 +63,7 @@ _cleanup() # = sunit=0 blks # realtime =none extsz=65536 blocks=N, rtextents=N # -_mkfs_filter() +mkfs_filter() { tee -a $seqres.full | \ sed \
@@ -80,6 +80,7 @@ _mkfs_filter() -e '/ *= *mmr=[0-9][0-9]* *$/d' \ -e 's/ *mixed-case=[YN]//' \ -e 's/ *ascii-ci=[01]//' \ + -e "s/\(sunit=\)\([0-9]* blks,\)/\11 blks,/" \ -e 's/sectsz=[0-9][0-9]* *//' \ -e 's/, lazy-count.*//' \ -e '/inode-paths/d' \
@@ -145,7 +146,7 @@ do fi echo "--- mkfs=$mkfs ---" export MKFS_OPTIONS="$mkfs" - _scratch_mkfs_xfs 2>&1 | _mkfs_filter + _scratch_mkfs_xfs 2>&1 | mkfs_filter echo "" echo "" done
tests/xfs/206 +15 -18
@@ -73,34 +73,31 @@ echo "=== truncate file ===" dd if=/dev/zero of=$tmpfile bs=1 seek=19998630180864 count=1 >/dev/null 2>&1 \ || _fail "!!! failed to truncate loopback file to correct size" +mkfs_filter() +{ + sed -e "s,^meta-data=.*isize,meta-data=FILE isize,g" \ + -e "s/\(^log.*blocks=\)\([0-9]*,\)/\1XXXXX,/" \ + -e "s/, projid32bit=[0-9]//" \ + -e "s/ ftype=[0-9]//" \ + -e "s/\(sectsz\)\(=[0-9]* *\)/\1=512 /" \ + -e "s/\(sunit=\)\([0-9]* blks,\)/\10 blks,/" \ + -e "/.*crc=/d" +} + # mkfs slightly smaller than that echo "=== mkfs.xfs ===" -mkfs.xfs -f -bsize=4096 -dagsize=76288719b,size=3905982455b -llazy-count=0 $tmpfile \ - | sed -e "s,^meta-data=.*isize,meta-data=FILE isize,g" \ - -e "s/\(^log.*blocks=\)\([0-9]*,\)/\1XXXXX,/" \ - -e "s/, projid32bit=[0-9]//" \ - -e "s/ ftype=[0-9]//" \ - -e "/.*crc=/d" +mkfs.xfs -f -bsize=4096 -dagsize=76288719b,size=3905982455b \ + -llazy-count=0 $tmpfile | mkfs_filter mount -o loop $tmpfile $tmpdir || _fail "!!! failed to loopback mount" # see what happens when we growfs it echo "=== xfs_growfs ===" -xfs_growfs $tmpdir \ - | sed -e "s,^meta-data=.*isize,meta-data=FILE isize,g" \ - -e "s/\(^log.*blocks=\)\([0-9]*,\)/\1XXXXX,/" \ - -e "s/, projid32bit=[0-9]//" \ - -e "s/ ftype=[0-9]//" \ - -e "/.*crc=/d" +xfs_growfs $tmpdir | mkfs_filter # and double-check the new geometry echo "=== xfs_info ===" -xfs_info $tmpdir \ - | sed -e "s,^meta-data=.*isize,meta-data=FILE isize,g" \ - -e "s/\(^log.*blocks=\)\([0-9]*,\)/\1XXXXX,/" \ - -e "s/, projid32bit=[0-9]//" \ - -e "s/ ftype=[0-9]//" \ - -e "/.*crc=/d" +xfs_info $tmpdir | mkfs_filter # _cleanup cleans up for us
Signature · 1 line
2.0.0
PATCH 3/4 xfs/049: umount -d fails when kernel wins teardown race
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
When /etc/mtab is linked to /proc/mounts and we are using mount time created loop devices (i.e. mount -o loop), the unmount can fail with this amazingly informative error message:
umount: /mnt/scratch/test2: filesystem was unmounted, but mount(8) failed: Invalid argument
What it actually means in this case is that the kernel tore down the loop device when the last reference went away, and it did it so fast that mount was not able to find it in /etc/mtab after the unmount syscall. Hence it could not find the loop device it was supposed to tear down and has a hissy fit.
This is simple to fix: mount does not need to tear down the loop device as the kernel does it automatically. Remove the "-d" from the umount command, and the test passes again.
There's quite a few other tests that also use umount -d - fix them as well.
--- tests/xfs/049 | 8 ++++---- tests/xfs/073 | 6 +++--- tests/xfs/078 | 4 ++-- tests/xfs/216 | 2 +- tests/xfs/217 | 2 +- tests/xfs/250 | 4 ++-- 6 files changed, 13 insertions(+), 13 deletions(-)
tests/xfs/049 +4 -4
@@ -29,8 +29,8 @@ echo "QA output created by $seq" _cleanup() { cd / - umount -d $SCRATCH_MNT/test2 > /dev/null 2>&1 - umount -d $SCRATCH_MNT/test > /dev/null 2>&1 + umount $SCRATCH_MNT/test2 > /dev/null 2>&1 + umount $SCRATCH_MNT/test > /dev/null 2>&1 rm -f $tmp.* if [ -w $seqres.full ]
@@ -123,11 +123,11 @@ rm -rf $SCRATCH_MNT/test/* >> $seqres.full 2>&1 \ || _fail "!!! clean failed" _log "umount ext2 on xfs" -umount -d $SCRATCH_MNT/test2 >> $seqres.full 2>&1 \ +umount $SCRATCH_MNT/test2 >> $seqres.full 2>&1 \ || _fail "!!! umount ext2 failed" _log "umount xfs" -umount -d $SCRATCH_MNT/test >> $seqres.full 2>&1 \ +umount $SCRATCH_MNT/test >> $seqres.full 2>&1 \ || _fail "!!! umount xfs failed" echo "--- mounts at end (before cleanup)" >> $seqres.full
tests/xfs/073 +3 -3
@@ -41,9 +41,9 @@ _cleanup() { cd / umount $SCRATCH_MNT 2>/dev/null - umount -d $imgs.loop 2>/dev/null + umount $imgs.loop 2>/dev/null [ -d $imgs.loop ] && rmdir $imgs.loop - umount -d $imgs.source_dir 2>/dev/null + umount $imgs.source_dir 2>/dev/null [ -d $imgs.source_dir ] && rm -rf $imgs.source_dir rm -f $imgs.* $tmp.* /var/tmp/xfs_copy.log.* }
@@ -119,7 +119,7 @@ _verify_copy() echo unmounting and removing new image umount $source_dir - umount -d $target_dir > /dev/null 2>&1 + umount $target_dir > /dev/null 2>&1 rm -f $target }
tests/xfs/078 +2 -2
@@ -36,7 +36,7 @@ _cleanup() { cd / rm -f $tmp.* - umount -d $LOOP_MNT 2>/dev/null + umount $LOOP_MNT 2>/dev/null rmdir $LOOP_MNT }
@@ -97,7 +97,7 @@ _grow_loop() $XFS_GROWFS_PROG $LOOP_MNT 2>&1 | _filter_growfs 2>&1 echo "*** unmount" - umount -d $LOOP_MNT > /dev/null 2>&1 + umount $LOOP_MNT > /dev/null 2>&1 # Large grows takes forever to check.. if [ "$check" -gt "0" ]
tests/xfs/216 +1 -1
@@ -60,7 +60,7 @@ _do_mkfs() -d name=$LOOP_DEV,size=${i}g |grep log mount -o loop -t xfs $LOOP_DEV $LOOP_MNT echo "test write" > $LOOP_MNT/test - umount -d $LOOP_MNT > /dev/null 2>&1 + umount $LOOP_MNT > /dev/null 2>&1 done } # make large holey file
tests/xfs/217 +1 -1
@@ -62,7 +62,7 @@ _do_mkfs() -d name=$LOOP_DEV,size=${i}g |grep log mount -o loop -t xfs $LOOP_DEV $LOOP_MNT echo "test write" > $LOOP_MNT/test - umount -d $LOOP_MNT > /dev/null 2>&1 + umount $LOOP_MNT > /dev/null 2>&1 # punch out the previous blocks so that we keep the amount of # disk space the test requires down to a minimum.
tests/xfs/250 +2 -2
@@ -33,7 +33,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 _cleanup() { cd / - umount -d $LOOP_MNT 2>/dev/null + umount $LOOP_MNT 2>/dev/null rm -f $LOOP_DEV rmdir $LOOP_MNT }
@@ -84,7 +84,7 @@ _test_loop() xfs_io -f -c "resvsp 0 $fsize" $LOOP_MNT/foo | _filter_io echo "*** unmount loop filesystem" - umount -d $LOOP_MNT > /dev/null 2>&1 + umount $LOOP_MNT > /dev/null 2>&1 echo "*** check loop filesystem" _check_xfs_filesystem $LOOP_DEV none none
Signature · 1 line
2.0.0
PATCH 4/4 generic: Add rudimetary RENAME_WHITEOUT test
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
There is no API documentation for RENAME_WHITEOUT. There is no developer documentation for RENAME_WHITEOUT. There are not comments in the overlayfs or ext4 implementation of RENAME_WHITEOUT.
Hence, this test simply tries to expose basic RENAME_WHITEOUT behaviour from ext4 so we can reverse-engineer and verify bug-for-bug renameat2(RENAME_WHITEOUT) ext4 compatibility.
Note: uses generic/078 just to keep out of the way of the 6-7 other pending new tests.
--- src/renameat2.c | 4 ++-- tests/generic/078 | 66 +++++++++++++++++++++++++++++++++++++++++++++++++++ tests/generic/078.out | 51 +++++++++++++++++++++++++++++++++++++++ tests/generic/group | 1 + 4 files changed, 120 insertions(+), 2 deletions(-) create mode 100755 tests/generic/078 create mode 100644 tests/generic/078.out
src/renameat2.c +2 -2
@@ -96,9 +96,9 @@ int main(int argc, char *argv[])* Turn EEXIST into ENOTEMPTY. E.g. XFS uses EEXIST, and that * is also accepted by the standards. * -* This applies only to plain rename (flags == 0). +* This applies only to plain rename and RENAME_WHITEOUT */ -if (!flags && errno ==EEXIST ) +if (errno ==EEXIST && (!flags || (flags &RENAME_WHITEOUT ))) errno =ENOTEMPTY ;perror ("" );
tests/generic/078 new file +66 -0
66 lines not on this page · raw message
tests/generic/078.out new file +51 -0
51 lines not on this page · raw message
tests/generic/group +1 -0
@@ -69,6 +69,7 @@ 075 rw udf auto quick 076 metadata rw udf auto quick stress 077 acl attr auto enospc +078 auto quick metadata 079 acl attr ioctl metadata auto quick 083 rw auto enospc stress 088 perms auto quick
Signature · 1 line
2.0.0