xfstest 285: userspace vs. fragmented multiblock dir2
PATCH 1/1 xfstest 285: userspace vs. fragmented multiblock dir2
Current xfs_repair is unhappy with fragmented multiblock v2 directories. This test shows it ... patches to fix it soon to follow.
---
285 new file +128 -0
@@ -0,0 +1,128 @@ +#! /bin/bash +# FS QA Test No. 285 +# +# Test userspace on a fragmented multi-block dir2 filesystem +# +#----------------------------------------------------------------------- +# Copyright (c) Red Hat, Inc. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +#----------------------------------------------------------------------- +# +# creator [email protected] + +seq=`basename $0` +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter + +_supported_fs xfs +_supported_os IRIX Linux + +# real QA test starts here +rm -f $seq.full +_require_scratch +_scratch_mkfs_xfs -n size=16k -d size=128m >> $seq.full 2>&1 +_scratch_mount + +# First we cause very badly fragmented freespace, then +# make some multiblock directories in the filesystem, which +# should come out very fragmented as a result + +# Step 1: Cause badly fragmented free space +mkdir $SCRATCH_MNT/fragdir +for I in `seq 0 27200`; do + (echo data > $SCRATCH_MNT/fragdir/f$I) >> $seq.full 2>&1 +done +sync + +for I in `seq 0 2 27200`; do + rm -f $SCRATCH_MNT/fragdir/f$I >> $seq.full 2>&1 +done +sync + +# Soak up any remaining freespace +xfs_io -f -c "pwrite 0 16m" -c "fsync" $SCRATCH_MNT/space_file.large >> $seq.full 2>&1 + +# Take a look at freespace for any post-mortem on the test +_scratch_unmount +xfs_db -c freesp $SCRATCH_DEV >> $seq.full 2>&1 +_scratch_mount + +# Step 2: Make a bunch of (hopefully fragmented) multiblock +# dir2 directories + +# - FMT_LOCAL +mkdir $SCRATCH_MNT/S_IFDIR.FMT_LOCAL +touch $SCRATCH_MNT/S_IFDIR.FMT_LOCAL/localdirfile + +# - FMT_EXTENTS +mkdir $SCRATCH_MNT/S_IFDIR.FMT_EXTENTS +for I in `seq 1 100`; do + touch $SCRATCH_MNT/S_IFDIR.FMT_EXTENTS/extent_dir_file_$I +done +# With a few missing +for I in `seq 10 2 20` 100; do + rm -f $SCRATCH_MNT/S_IFDIR.FMT_EXTENTS/extent_dir_file_$I +done + +# - FMT_BTREE +mkdir $SCRATCH_MNT/S_IFDIR.FMT_BTREE +for I in `seq 1 1000`; do + touch $SCRATCH_MNT/S_IFDIR.FMT_BTREE/btree_dir_file_$I +done +# With a few missing +for I in `seq 10 2 20` 1000; do + rm -f $SCRATCH_MNT/S_IFDIR.FMT_BTREE/btree_dir_file_$I +done + +# Dave's special hack - grow freespace tree +mkdir $SCRATCH_MNT/S_IFDIR.FMT_BTREE2 +for I in `seq 1 5000`; do + touch $SCRATCH_MNT/S_IFDIR.FMT_BTREE2/btree2_dir_file_$I +done +# Remove every other (odds) +for I in `seq 1 2 5000`; do + rm -f $SCRATCH_MNT/S_IFDIR.FMT_BTREE2/btree2_dir_file_$I +done + +_scratch_unmount +# Can xfs_repair and xfs_check cope with this monster? +_scratch_xfs_repair >> $seq.full 2>&1 || _fail "xfs_repair failed" +xfs_check $SCRATCH_DEV >> $seq.full 2>&1 || _fail "xfs_check failed" + +# Yes they can! Now... +# Can xfs_metadump cope with this monster? +xfs_metadump $SCRATCH_DEV $tmp.metadump || _fail "xfs_metadump failed" +xfs_mdrestore $tmp.metadump $tmp.img || _fail "xfs_mdrestore failed" +xfs_repair $tmp.img >> $seq.full 2>&1 || _fail "xfs_repair of metadump failed" + +# Yes it can; success, all done +status=0 +exit +
group +1 -0
@@ -403,3 +403,4 @@ deprecated 282 dump ioctl auto quick 283 dump ioctl auto quick 284 auto +285 repair dir auto
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Hey Eric,
Quoted 5 lines · Eric Sandeen
Current
xfs_repairis unhappy with fragmented multiblock v2 directories. This test shows it ... patches to fix it soon to follow.Signed-off-by Eric Sandeen <[email protected]>
This failed on my system... looks like it's just PEBKAC. My partitions are too small?
# ./check 287 No patches applied KERNEL_TOPDIR -- /root/xfs BRANCH -- refs/heads/master2 DESC -- v3.4-10158-gf8f5701 PATCHES: FSTYP -- xfs (debug) PLATFORM -- Linux/x86_64 nfs10 3.5.0-rc1-0.7-default+ MKFS_OPTIONS -- -f -bsize=4096 /dev/sdb4 MOUNT_OPTIONS -- /dev/sdb4 /mnt/scratch
287 - no qualified output
Ran: 287
Failures: 287
Failed 1 of 1 tests
Regards,
Ben
P.S. Here's the contents of $seq.full.
meta-data=/dev/sdb4 isize=256 agcount=4, agsize=8192 blks
= sectsz=512 attr=2, projid32bit=0
data = bsize=4096 blocks=32768, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=16384 ascii-ci=0
log =internal log bsize=4096 blocks=1904, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0./287: line 60: /mnt/scratch/fragdir/f27196: No space left on device ./287: line 60: /mnt/scratch/fragdir/f27197: No space left on device ./287: line 60: /mnt/scratch/fragdir/f27198: No space left on device ./287: line 60: /mnt/scratch/fragdir/f27199: No space left on device ./287: line 60: /mnt/scratch/fragdir/f27200: No space left on device wrote 16777216/16777216 bytes at offset 0
16 MiB, 4096 ops; 0.0000 sec (565.311 MiB/sec and 144719.6410 ops/sec)
from to extents blocks pct
1 1 11197 11197 100.00
Phase 1 - find and verify superblock...
Phase 2 - using internal log
- zero log...
- scan filesystem freespace and inode maps...
- found root inode chunk
Phase 3 - for each AG...
- scan and clear agi unlinked lists...
- process known inodes and perform inode discovery...
- agno = 0
- agno = 1
- agno = 2
- agno = 3
- process newly discovered inodes...
Phase 4 - check for duplicate blocks...
- setting up duplicate extent list...
- check for inodes claiming duplicate blocks...
- agno = 0
- agno = 1
- agno = 2
- agno = 3
Phase 5 - rebuild AG headers and trees...
- reset superblock...
Phase 6 - check inode connectivity...
- resetting contents of realtime bitmap and summary inodes
- traversing filesystem ...
- traversal finished ...
- moving disconnected inodes to lost+found ...
Phase 7 - verify and correct link counts...
donexfs_db got a bbmap for 127248xfs_db got a bbmap for 195224xfs_db got a bbmap for 195352xfs_db got a bbmap for 195288xfs_db got a bbmap for 248080xfs_db got a bbmap for 247952xfs_db got a bbmap for 247872xfs_db got a bbmap for 247808xfs_db got a bbmap for 247744xfs_db got a bbmap for 247680xfs_db got a bbmap for 247616xfs_db got a bbmap for 247520xfs_db got a bbmap for 247456xfs_db got a bbmap for 247392xfs_db got a bbmap for 248016xfs_db got a bbmap for 248192xfs_db got a bbmap for 248256xfs_db got a bbmap for 248384xfs_db got a bbmap for 248128
Phase 1 - find and verify superblock...
Phase 2 - using internal log
- zero log...
- scan filesystem freespace and inode maps...
- found root inode chunk
Phase 3 - for each AG...
- scan and clear agi unlinked lists...
- process known inodes and perform inode discovery...
- agno = 0
- agno = 1
- agno = 2
- agno = 3
- process newly discovered inodes...
Phase 4 - check for duplicate blocks...
- setting up duplicate extent list...
- check for inodes claiming duplicate blocks...
- agno = 1
- agno = 0
- agno = 2
- agno = 3
Phase 5 - rebuild AG headers and trees...
- reset superblock...
Phase 6 - check inode connectivity...
- resetting contents of realtime bitmap and summary inodes
- traversing filesystem ...
- traversal finished ...
- moving disconnected inodes to lost+found ...
Phase 7 - verify and correct link counts...
done
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 23 lines · Ben Myers
Hey Eric,
Quoted 5 lines · Eric Sandeen
Current
xfs_repairis unhappy with fragmented multiblock v2 directories. This test shows it ... patches to fix it soon to follow.Signed-off-by Eric Sandeen <[email protected]>This failed on my system... looks like it's just PEBKAC. My partitions are too small?
# ./check 287 No patches applied KERNEL_TOPDIR -- /root/xfs BRANCH -- refs/heads/master2 DESC -- v3.4-10158-gf8f5701 PATCHES: FSTYP -- xfs (debug) PLATFORM -- Linux/x86_64 nfs10 3.5.0-rc1-0.7-default+ MKFS_OPTIONS -- -f -bsize=4096 /dev/sdb4 MOUNT_OPTIONS -- /dev/sdb4 /mnt/scratch
Quoted 1 line
287 - no qualified output
It failed because there is not 287.out file in the patch. Every test has to have a golden image to compare the result against...
Cheers,
Dave.
Signature · 7 lines
Dave Chinner [email protected] _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Quoted 24 lines · Dave Chinner
Quoted 23 lines · Ben Myers
Hey Eric,
Quoted 5 lines · Eric Sandeen
Current
xfs_repairis unhappy with fragmented multiblock v2 directories. This test shows it ... patches to fix it soon to follow.Signed-off-by Eric Sandeen <[email protected]>This failed on my system... looks like it's just PEBKAC. My partitions are too small?
# ./check 287 No patches applied KERNEL_TOPDIR -- /root/xfs BRANCH -- refs/heads/master2 DESC -- v3.4-10158-gf8f5701 PATCHES: FSTYP -- xfs (debug) PLATFORM -- Linux/x86_64 nfs10 3.5.0-rc1-0.7-default+ MKFS_OPTIONS -- -f -bsize=4096 /dev/sdb4 MOUNT_OPTIONS -- /dev/sdb4 /mnt/scratch
Quoted 5 lines
Quoted 1 line
287 - no qualified output
It failed because there is not 287.out file in the patch. Every test has to have a golden image to compare the result against...
Argh.....
Sorry.
Quoted 6 lines
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
PATCH v2 1/1 xfstest 287: userspace vs. fragmented multiblock dir2
In reply to Eric SandeenCurrent xfs_repair is unhappy with fragmented multiblock v2 directories. This test shows it ... patches to fix it soon to follow.
---
V2: Renumbered, now with even more 287.out-ness.
(hand-renamed to 287, should apply...)
((we need to merge the 2 git trees ... ))
287 new file +128 -0
@@ -0,0 +1,128 @@ +#! /bin/bash +# FS QA Test No. 287 +# +# Test xfs_repair on fragmented multi-block dir2 fs +# +#----------------------------------------------------------------------- +# Copyright (c) Red Hat, Inc. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +#----------------------------------------------------------------------- +# +# creator [email protected] + +seq=`basename $0` +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter + +_supported_fs xfs +_supported_os IRIX Linux + +# real QA test starts here +rm -f $seq.full +_require_scratch +_scratch_mkfs_xfs -n size=16k -d size=128m >> $seq.full 2>&1 +_scratch_mount + +# First we cause very badly fragmented freespace, then +# make some multiblock directories in the filesystem, which +# should come out very fragmented as a result + +# Step 1: Cause badly fragmented free space +mkdir $SCRATCH_MNT/fragdir +for I in `seq 0 27200`; do + (echo data > $SCRATCH_MNT/fragdir/f$I) >> $seq.full 2>&1 +done +sync + +for I in `seq 0 2 27200`; do + rm -f $SCRATCH_MNT/fragdir/f$I >> $seq.full 2>&1 +done +sync + +# Soak up any remaining freespace +xfs_io -f -c "pwrite 0 16m" -c "fsync" $SCRATCH_MNT/space_file.large >> $seq.full 2>&1 + +# Take a look at freespace for any post-mortem on the test +_scratch_unmount +xfs_db -c freesp $SCRATCH_DEV >> $seq.full 2>&1 +_scratch_mount + +# Step 2: Make a bunch of (hopefully fragmented) multiblock +# dir2 directories + +# - FMT_LOCAL +mkdir $SCRATCH_MNT/S_IFDIR.FMT_LOCAL +touch $SCRATCH_MNT/S_IFDIR.FMT_LOCAL/localdirfile + +# - FMT_EXTENTS +mkdir $SCRATCH_MNT/S_IFDIR.FMT_EXTENTS +for I in `seq 1 100`; do + touch $SCRATCH_MNT/S_IFDIR.FMT_EXTENTS/extent_dir_file_$I +done +# With a few missing +for I in `seq 10 2 20` 100; do + rm -f $SCRATCH_MNT/S_IFDIR.FMT_EXTENTS/extent_dir_file_$I +done + +# - FMT_BTREE +mkdir $SCRATCH_MNT/S_IFDIR.FMT_BTREE +for I in `seq 1 1000`; do + touch $SCRATCH_MNT/S_IFDIR.FMT_BTREE/btree_dir_file_$I +done +# With a few missing +for I in `seq 10 2 20` 1000; do + rm -f $SCRATCH_MNT/S_IFDIR.FMT_BTREE/btree_dir_file_$I +done + +# Dave's special hack - grow freespace tree +mkdir $SCRATCH_MNT/S_IFDIR.FMT_BTREE2 +for I in `seq 1 5000`; do + touch $SCRATCH_MNT/S_IFDIR.FMT_BTREE2/btree2_dir_file_$I +done +# Remove every other (odds) +for I in `seq 1 2 5000`; do + rm -f $SCRATCH_MNT/S_IFDIR.FMT_BTREE2/btree2_dir_file_$I +done + +_scratch_unmount +# Can xfs_repair and xfs_check cope with this monster? +_scratch_xfs_repair >> $seq.full 2>&1 || _fail "xfs_repair failed" +xfs_check $SCRATCH_DEV >> $seq.full 2>&1 || _fail "xfs_check failed" + +# Yes they can! Now... +# Can xfs_metadump cope with this monster? +xfs_metadump $SCRATCH_DEV $tmp.metadump || _fail "xfs_metadump failed" +xfs_mdrestore $tmp.metadump $tmp.img || _fail "xfs_mdrestore failed" +xfs_repair $tmp.img >> $seq.full 2>&1 || _fail "xfs_repair of metadump failed" + +# Yes it can; success, all done +status=0 +exit +
287.out new file +1 -0
@@ -0,0 +1 @@ +QA output created by 287
group +1 -0
@@ -403,3 +403,4 @@ deprecated 284 auto 285 auto rw 286 other +287 repair
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH V2] xfstest 287: userspace vs. fragmented multiblock dir2
Quoted 163 lines · Eric Sandeen
Current
xfs_repairis unhappy with fragmented multiblock v2 directories. This test shows it ... patches to fix it soon to follow.Signed-off-by Eric Sandeen<[email protected]>---
V2: Renumbered, now with even more 287.out-ness.
(hand-renamed to 287, should apply...)
((we need to merge the 2 git trees ... ))287 new file +128
-0@@ -0,0 +1,128 @@ +#! /bin/bash +# FS QA Test No. 287 +# +# Test xfs_repair on fragmented multi-block dir2 fs +# +#----------------------------------------------------------------------- +# Copyright (c) Red Hat, Inc. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +#----------------------------------------------------------------------- +# +# creator [email protected] + +seq=`basename $0` +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter + +_supported_fs xfs +_supported_os IRIX Linux + +# real QA test starts here +rm -f $seq.full +_require_scratch +_scratch_mkfs_xfs -n size=16k -d size=128m>> $seq.full 2>&1 +_scratch_mount + +# First we cause very badly fragmented freespace, then +# make some multiblock directories in the filesystem, which +# should come out very fragmented as a result + +# Step 1: Cause badly fragmented free space +mkdir $SCRATCH_MNT/fragdir +for I in `seq 0 27200`; do + (echo data> $SCRATCH_MNT/fragdir/f$I)>> $seq.full 2>&1 +done +sync + +for I in `seq 0 2 27200`; do + rm -f $SCRATCH_MNT/fragdir/f$I>> $seq.full 2>&1 +done +sync + +# Soak up any remaining freespace +xfs_io -f -c "pwrite 0 16m" -c "fsync" $SCRATCH_MNT/space_file.large>> $seq.full 2>&1 + +# Take a look at freespace for any post-mortem on the test +_scratch_unmount +xfs_db -c freesp $SCRATCH_DEV>> $seq.full 2>&1 +_scratch_mount + +# Step 2: Make a bunch of (hopefully fragmented) multiblock +# dir2 directories + +# - FMT_LOCAL +mkdir $SCRATCH_MNT/S_IFDIR.FMT_LOCAL +touch $SCRATCH_MNT/S_IFDIR.FMT_LOCAL/localdirfile + +# - FMT_EXTENTS +mkdir $SCRATCH_MNT/S_IFDIR.FMT_EXTENTS +for I in `seq 1 100`; do + touch $SCRATCH_MNT/S_IFDIR.FMT_EXTENTS/extent_dir_file_$I +done +# With a few missing +for I in `seq 10 2 20` 100; do + rm -f $SCRATCH_MNT/S_IFDIR.FMT_EXTENTS/extent_dir_file_$I +done + +# - FMT_BTREE +mkdir $SCRATCH_MNT/S_IFDIR.FMT_BTREE +for I in `seq 1 1000`; do + touch $SCRATCH_MNT/S_IFDIR.FMT_BTREE/btree_dir_file_$I +done +# With a few missing +for I in `seq 10 2 20` 1000; do + rm -f $SCRATCH_MNT/S_IFDIR.FMT_BTREE/btree_dir_file_$I +done + +# Dave's special hack - grow freespace tree +mkdir $SCRATCH_MNT/S_IFDIR.FMT_BTREE2 +for I in `seq 1 5000`; do + touch $SCRATCH_MNT/S_IFDIR.FMT_BTREE2/btree2_dir_file_$I +done +# Remove every other (odds) +for I in `seq 1 2 5000`; do + rm -f $SCRATCH_MNT/S_IFDIR.FMT_BTREE2/btree2_dir_file_$I +done + +_scratch_unmount +# Can xfs_repair and xfs_check cope with this monster? +_scratch_xfs_repair>> $seq.full 2>&1 || _fail "xfs_repair failed" +xfs_check $SCRATCH_DEV>> $seq.full 2>&1 || _fail "xfs_check failed" + +# Yes they can! Now... +# Can xfs_metadump cope with this monster? +xfs_metadump $SCRATCH_DEV $tmp.metadump || _fail "xfs_metadump failed" +xfs_mdrestore $tmp.metadump $tmp.img || _fail "xfs_mdrestore failed" +xfs_repair $tmp.img>> $seq.full 2>&1 || _fail "xfs_repair of metadump failed" + +# Yes it can; success, all done +status=0 +exit +287.out new file +1
-0@@ -0,0 +1 @@ +QA output created by 287group +1
-0@@ -403,3 +403,4 @@ deprecated 284 auto 285 auto rw 286 other +287 repair
Quoted 4 lines
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
This patch has been committed to git://oss.sgi.com/xfs/cmds/xfstests master branch, commit 2a4ed (test 291).
--Mark.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs