xfstests: fixes and new tests
cover xfstests: fixes and new tests
Hi folks,
In this series, there are a some cleanup patches related to running small TEST_DEV filesystems out of space on repeated runs. There are some tests that leave large files lying around that consume space, so when you have a 4GB TEST_DEV soem of the tests will fail because they need 3GB of free space.
There is a new test for maximum ACL count support in XFS - I've removed the ACL count subtest from shared/051 and created a new XFS specific test that uses the maximum supported number of ACLs for the given filesystem configuration rather than 25. This new test is what exposed the transaction overrun when replacing a remote attribute with an inline attribute.
There is also a patch to remove the DMAPI tests from the auto group
- almost nobody has an XFS/DMAPI enabled kernel that can run these tests, so they can add "-g dmapi" to their tests runs rather than have everyone else have to run "-x dmapi" to avoid the "notrun" noise that is generated.
Finally, there is a xfs_repair filter patch that chops out the new CRC/corruption detection messages that are emitted when verifiers detect problems. This makes xfs/030 and xfs/033 pass....
Cheers,
Dave.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
PATCH 1/6 generic/204: tweak reserve pool size
From: Dave Chinner <[email protected]>
On small block size filesystems, the reserve pool size is kept constant at 4MB. filesystems with smaller blocks use comparitively more blocks for indexing metadata (e.g. in the inode and extent btrees) and so having a higher indirect block usage. Hence we need to leave the reserve pool at 1024 block and not scale it for a constant size.
This makes the test pass on a filesystem made with MKFS_OPTIONS="-b size=1024 -m crc=1".
--- tests/generic/204 | 2 -- 1 file changed, 2 deletions(-)
tests/generic/204 +0 -2
@@ -74,7 +74,6 @@ space=97920000 # files = (97920000 / (isize + bsize)) files=$((space / (isize + dbsize))) -resv_blks=$((resv_blks * (4096 / dbsize))) echo files $files, resvblks $resv_blks >> $seqres.full _scratch_resvblks $resv_blks >> $seqres.full 2>&1
@@ -88,5 +87,4 @@ _check_scratch_fs # success, all done echo "*** done" -rm -f $seqres.full status=0
Signature · 6 lines
1.9.0 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 1/6] generic/204: tweak reserve pool size
Quoted 11 lines · Dave Chinner
From: Dave Chinner <[email protected]>
On small block size filesystems, the reserve pool size is kept constant at 4MB. filesystems with smaller blocks use comparitively more blocks for indexing metadata (e.g. in the inode and extent btrees) and so having a higher indirect block usage. Hence we need to leave the reserve pool at 1024 block and not scale it for a constant size.
This makes the test pass on a filesystem made with MKFS_OPTIONS="-b size=1024 -m crc=1".
Looks good,
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
PATCH 2/6 shared/051: remove ACL count subtest
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
Different versions of XFS support different numbers of ACLs on disk. Remove that subtest from this shared test to prevent it form causing spurious failures on filesystems that support more than 25 ACLs.
--- tests/shared/051 | 45 ----------------- tests/shared/051.out | 135 --------------------------------------------------- 2 files changed, 180 deletions(-)
tests/shared/051 +0 -45
@@ -318,51 +318,6 @@ chacl m:user1:rwx file1 chacl a::rwx file1 set +x -# we return E2BIG if hit the max acl limits on new kernel, but EINVAL -# on old kernel. So we need to filter out the error message in order -# to make the updated golden output works for both old and new kernels. -_filter_largeacl() -{ - sed -e "s/Invalid argument/Argument list too long/" -} - -#------------------------------------------------------- - -echo "" -echo "=== Test out large ACLs ===" -touch largeaclfile -XFS_ACL_MAX_ENTRIES=25 -num_aces_pre=`expr $XFS_ACL_MAX_ENTRIES - 1` -num_aces_post=`expr $XFS_ACL_MAX_ENTRIES + 1` - -acl1=`_create_n_aces $num_aces_pre` -acl2=`_create_n_aces $XFS_ACL_MAX_ENTRIES` -acl3=`_create_n_aces $num_aces_post` -acl4=`_create_n_aces 16` # Andreas G. libacl size for initial get -acl5=`_create_n_aces 17` # 1 over A.G. libacl initial size - -echo "1 below xfs acl max" -chacl $acl1 largeaclfile -getfacl --numeric largeaclfile | _filter_aces - -echo "xfs acl max" -chacl $acl2 largeaclfile -getfacl --numeric largeaclfile | _filter_aces - -echo "1 above xfs acl max" -chacl $acl3 largeaclfile 2>&1 | _filter_largeacl -getfacl --numeric largeaclfile | _filter_aces - -echo "use 16 aces" -chacl $acl4 largeaclfile -getfacl --numeric largeaclfile | _filter_aces - -echo "use 17 aces" -chacl $acl5 largeaclfile -getfacl --numeric largeaclfile | _filter_aces - -#------------------------------------------------------- - # success, all done status=0 exit
tests/shared/051.out +0 -135
138 lines not on this page · raw message
Signature · 6 lines
1.9.0 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 2/6] shared/051: remove ACL count subtest
Quoted 7 lines · Dave Chinner
From: Dave Chinner <[email protected]>
Different versions of XFS support different numbers of ACLs on disk. Remove that subtest from this shared test to prevent it form causing spurious failures on filesystems that support more than 25 ACLs.
Signed-off-by Dave Chinner <[email protected]>
Looks good,
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
PATCH 3/6 xfs: introduce new large ACL test
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
Having just removed the largeacl test from the shared ACL test, reintroduce the same test as an XFS specific test so that we can handle the different limits in supported ACL count appropriately.
--- tests/xfs/010 | 147 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/010.out | 9 ++++ tests/xfs/group | 1 + 3 files changed, 157 insertions(+) create mode 100644 tests/xfs/010 create mode 100644 tests/xfs/010.out
tests/xfs/010 new file +147 -0
@@ -0,0 +1,147 @@ +#! /bin/bash +# FS QA Test No. xfs/010 +# +# Test out ACL count limits +# +#----------------------------------------------------------------------- +# Copyright (c) 2000-2002 Silicon Graphics, Inc. All Rights Reserved. +# Copyright (c) 2014 Red hat, Inc. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +# +#----------------------------------------------------------------------- +# + +seq=`basename $0` +seqres=$RESULT_DIR/$seq + +here=`pwd` +tmp=/tmp/$$ +runas=$here/src/runas +status=1 # FAILure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter +. ./common/attr + +_cleanup() +{ + cd / + rm -f $tmp.* + [ -n "$TEST_DIR" ] && rm -rf $TEST_DIR/$seq.dir1 +} + +# real QA test starts here +_supported_fs xfs +_supported_os Linux +_require_scratch + +[ -x $runas ] || _notrun "$runas executable not found" + +rm -f $seqres.full + +_need_to_be_root +_acl_setup_ids +_require_acls + +# get dir +cd $TEST_DIR +rm -rf $seq.dir1 +mkdir $seq.dir1 +cd $seq.dir1 + +echo "QA output created by $seq" + +# we return E2BIG if hit the max acl limits on new kernel, but EINVAL +# on old kernel. So we need to filter out the error message in order +# to make the updated golden output works for both old and new kernels. +_filter_largeacl() +{ + sed -e "s/Invalid argument/Argument list too long/" +} + +# filter all the non-ace stuff from the acl output so the count is +# correct. Note that this assumes that _create_n_aces always creates rwx acls. +_filter_acls() +{ + _filter_aces | grep ':rwx' +} + +# CRC format filesystems have much larger ACL counts. The actual number is into +# the thousands, but testing that meany takes too long, so just test well past +# the old limit of 25. +get_max_acls() +{ + xfs_info $TEST_DIR | _filter_mkfs > /dev/null 2> $tmp.info + . $tmp.info + if [ $_fs_has_crcs -eq 0 ]; then + echo 25 + else + echo 5461 + fi +} + +# store the output in seqres.full, then run again an count and filter the +# output. +check_acls() +{ + _acl=$1 + _count=$2 + + chacl $_acl largeaclfile 2>&1 | _filter_largeacl + getfacl --numeric largeaclfile | _filter_aces \ + >> $seqres.full 2> /dev/null + nacls=`getfacl --numeric largeaclfile | _filter_acls | wc -l` + if [ $nacls -ne $_count ]; then + echo Wrong ACL count - $nacls != $_count + fi +} + + +echo "" +echo "=== Test out large ACLs ===" +touch largeaclfile + +XFS_ACL_MAX_ENTRIES=$(get_max_acls) +num_aces_pre=$((XFS_ACL_MAX_ENTRIES - 1)) +num_aces_post=$((XFS_ACL_MAX_ENTRIES + 1)) + +acl1=`_create_n_aces $num_aces_pre` +acl2=`_create_n_aces $XFS_ACL_MAX_ENTRIES` +acl3=`_create_n_aces $num_aces_post` +acl4=`_create_n_aces 16` # Andreas G. libacl size for initial get +acl5=`_create_n_aces 17` # 1 over A.G. libacl initial size + +echo "1 below xfs acl max" +check_acls $acl1 $num_aces_pre + +echo "xfs acl max" +check_acls $acl2 $XFS_ACL_MAX_ENTRIES + +# we expect the ACL change to fail, so the old ACLs should remain on the +# file. Hence the expected ACL count is XFS_ACL_MAX_ENTRIES, not num_aces_post. +echo "1 above xfs acl max" +check_acls $acl3 $XFS_ACL_MAX_ENTRIES + +echo "use 16 aces" +check_acls $acl4 16 + +echo "use 17 aces" +check_acls $acl5 17 + +# success, all done +status=0 +exit
tests/xfs/010.out new file +9 -0
@@ -0,0 +1,9 @@ +QA output created by 010 + +=== Test out large ACLs === +1 below xfs acl max +xfs acl max +1 above xfs acl max +chacl: cannot set access acl on "largeaclfile": Argument list too long +use 16 aces +use 17 aces
tests/xfs/group +1 -0
@@ -7,6 +7,7 @@ 007 auto quota quick 008 rw ioctl auto quick 009 rw ioctl auto prealloc quick +010 acl quick auto 012 rw auto quick 016 rw auto quick 017 mount auto quick stress
Signature · 6 lines
1.9.0 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 3/6] xfs: introduce new large ACL test
Quoted 5 lines · Dave Chinner
From: Dave Chinner <[email protected]>
Having just removed the largeacl test from the shared ACL test, reintroduce the same test as an XFS specific test so that we can handle the different limits in supported ACL count appropriately.
I'd prefer to have this as a shared test, and define a _max_acl_size function in common.acl that does a switch for the fs type. Only start with XFS for now, but if the maintainers of other filesystems care enough they could add their limits.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 3/6] xfs: introduce new large ACL test
Quoted 11 lines · Christoph Hellwig
Quoted 5 lines · Dave Chinner
From: Dave Chinner <[email protected]>
Having just removed the largeacl test from the shared ACL test, reintroduce the same test as an XFS specific test so that we can handle the different limits in supported ACL count appropriately.
I'd prefer to have this as a shared test, and define a
_max_acl_sizefunction in common.acl that does a switch for the fs type. Only start with XFS for now, but if the maintainers of other filesystems care enough they could add their limits.
Ok, I'll add it as a generic test, along with a _max_acl_size function and a _requires_max_acl_size helper....
Cheers,
Dave.
Signature · 7 lines
Dave Chinner [email protected] _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
PATCH v2 3/6 generic: introduce new large ACL test
From: Dave Chinner <[email protected]>
Having just removed the largeacl test from the shared ACL test, reintroduce the same test as an generic test so that we can handle the different limits in supported ACL count appropriately across different filesystems and different configurations within filesystem types.
Filesystems have to add support to _acl_get_max to run this test - the default behaviour right now is to throw a notrun error like this:
generic/026 14s ... [not run] ext4 does not define maximum ACL count
--- common/attr | 31 ++++++++++++ tests/generic/026 | 132 ++++++++++++++++++++++++++++++++++++++++++++++++++ tests/generic/026.out | 9 ++++ tests/generic/group | 1 + 4 files changed, 173 insertions(+)
common/attr +31 -0
@@ -21,6 +21,37 @@ #----------------------------------------------------------------------- # common extended attribute and ACL support +# filesystems that want to test maximum supported acl counts need to +# add support in here +_acl_get_max() +{ + case $FSTYP in + xfs) + # CRC format filesystems have much larger ACL counts. The actual + # number is into the thousands, but testing that meany takes too + # long, so just test well past the old limit of 25. + xfs_info $TEST_DIR | _filter_mkfs > /dev/null 2> $tmp.info + . $tmp.info + rm $tmp.info + if [ $_fs_has_crcs -eq 0 ]; then + echo 25 + else + echo 5461 + fi + ;; + *) + echo 0 + ;; + esac +} + +_require_acl_get_max() +{ + if [ $(_acl_get_max) -eq 0 ]; then + _notrun "$FSTYP does not define maximum ACL count" + fi +} + # pick three unused user/group ids, store them as $acl[1-3] # _acl_setup_ids()
tests/generic/026 new file +132 -0
132 lines not on this page · raw message
tests/generic/026.out new file +9 -0
@@ -0,0 +1,9 @@ +QA output created by 026 + +=== Test out large ACLs === +1 below acl max +acl max +1 above acl max +chacl: cannot set access acl on "largeaclfile": Argument list too long +use 16 aces +use 17 aces
tests/generic/group +1 -0
@@ -28,6 +28,7 @@ 023 auto quick 024 auto quick 025 auto quick +026 acl quick auto 053 acl repair auto quick 062 attr udf auto quick 068 other auto freeze dangerous stress
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 3/6 v2] generic: introduce new large ACL test
Quoted 8 lines · Dave Chinner
From: Dave Chinner <[email protected]>
Having just removed the largeacl test from the shared ACL test, reintroduce the same test as an generic test so that we can handle the different limits in supported ACL count appropriately across different filesystems and different configurations within filesystem types.
Quoted 7 lines
Filesystems have to add support to
_acl_get_maxto run this test - the default behaviour right now is to throw a notrun error like this:generic/026 14s ... [not run] ext4 does not define maximum ACL count
Signed-off-by Dave Chinner <[email protected]>
Looks good,
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
PATCH 4/6 generic: cleanup space after test in TESTDIR
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
A couple of tests leave behind large files or directory structures when they complete, which leads to small TEST_DEVs running out of space during other tests. Make those space hogs clean up after themselves so that random tests don't fail with ENOSPC errors.
--- tests/generic/070 | 1 + tests/xfs/292 | 2 ++ 2 files changed, 3 insertions(+)
tests/generic/070 +1 -0
@@ -34,6 +34,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 _cleanup() { cd / + rm -rf $TEST_DIR/fsstress rm -f $tmp.* }
tests/xfs/292 +2 -0
@@ -57,6 +57,8 @@ mkfs.xfs -f $fsfile 2>&1 | sed -e "s:$fsfile:FILENAME:g" | grep meta-data echo "mkfs.xfs with cmdline geometry" mkfs.xfs -f -d su=16k,sw=5 $fsfile 2>&1 | sed -e "s:$fsfile:FILENAME:g" | grep meta-data +rm -f $fsfile + # success, all done status=0 exit
Signature · 6 lines
1.9.0 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 4/6] generic: cleanup space after test in TESTDIR
Quoted 8 lines · Dave Chinner
From: Dave Chinner <[email protected]>
A couple of tests leave behind large files or directory structures when they complete, which leads to small
TEST_DEVsrunning out of space during other tests. Make those space hogs clean up after themselves so that random tests don't fail with ENOSPC errors.Signed-off-by Dave Chinner <[email protected]>
Looks good,
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
PATCH 5/6 xfs: remove dmapi tests from the auto group
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
There is no mainline kernel support for DMAPI in XFS, and so every time the xfstests auto group is run it throws a large number of "[not run] Assuming DMAPI modules are not loaded". I've noted that people are excluding the dmapi group to avoid this, so rather than inflict pain on everyone, make hose few that need to do dmapi testing include it specifically.
IOWs, remove the DMAPI tests from the auto group.
--- tests/xfs/group | 50 +++++++++++++++++++++++++------------------------- 1 file changed, 25 insertions(+), 25 deletions(-)
tests/xfs/group +25 -25
@@ -93,48 +93,48 @@ 138 metadata log auto 139 metadata log auto 140 metadata log auto -142 dmapi auto -143 dmapi auto -144 dmapi auto -145 dmapi auto -146 dmapi auto -147 dmapi auto +142 dmapi +143 dmapi +144 dmapi +145 dmapi +146 dmapi +147 dmapi 148 repair auto 149 repair auto -150 dmapi auto -151 dmapi auto -152 dmapi auto -153 dmapi auto -154 dmapi auto -155 dmapi auto -156 dmapi auto -157 dmapi auto -158 dmapi auto -159 dmapi auto -160 dmapi auto -161 dmapi auto -162 dmapi auto -163 dmapi auto +150 dmapi +151 dmapi +152 dmapi +153 dmapi +154 dmapi +155 dmapi +156 dmapi +157 dmapi +158 dmapi +159 dmapi +160 dmapi +161 dmapi +162 dmapi +163 dmapi 164 rw pattern auto prealloc quick 165 rw pattern auto prealloc quick 166 rw metadata auto quick 167 rw metadata auto stress -168 dmapi auto +168 dmapi 170 rw filestreams auto quick 171 rw filestreams 172 rw filestreams 173 rw filestreams 174 rw filestreams auto -175 dmapi auto -176 dmapi auto -177 rw other auto +175 dmapi +176 dmapi +177 dmapi 178 mkfs other auto 179 metadata rw auto 180 metadata rw auto 181 log auto quick 182 metadata rw auto 183 rw other auto quick -185 dmapi auto +185 dmapi 186 attr auto quick 187 attr auto quick 188 ci dir auto
Signature · 6 lines
1.9.0 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 5/6] xfs: remove dmapi tests from the auto group
Quoted 9 lines · Dave Chinner
From: Dave Chinner <[email protected]>
There is no mainline kernel support for DMAPI in XFS, and so every time the xfstests auto group is run it throws a large number of "[not run] Assuming DMAPI modules are not loaded". I've noted that people are excluding the dmapi group to avoid this, so rather than inflict pain on everyone, make hose few that need to do dmapi testing include it specifically.
Quoted 3 lines
IOWs, remove the DMAPI tests from the auto group.
Signed-off-by Dave Chinner <[email protected]>
Yes, we shouldn't run them by default. They also probably should move from the tests/xfs/ dir to a new tests/dmapi one as they aren't really XFS specific, they are specific to the out of tree dmapi code which at one point could also support JFS.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
PATCH 6/6 filter: xfs_repair emits new corruption messagse
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
xfs_repair now dumps metadata and CRC corruption information to the output from the verifier infrastrcuture. Filter this out so that it doesn't cause spurious test failures such as in xfs/030 and xfs/033.
--- common/repair | 3 +++ 1 file changed, 3 insertions(+)
common/repair +3 -0
@@ -80,6 +80,9 @@ s/\s+- \d+:\d\d:\d\d:.*\n//g; # crc enabled filesystem output /XFS_CORRUPTION_ERROR/ && next; /^bad uuid/ && next; +/^Metadata corruption detected/ && next; +/^Metadata CRC error detected/ && next; +/^agfl has bad CRC/ && next; print;' }
Signature · 6 lines
1.9.0 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 6/6] filter: xfs_repair emits new corruption messagse
Quoted 7 lines · Dave Chinner
From: Dave Chinner <[email protected]>
xfs_repairnow dumps metadata and CRC corruption information to the output from the verifier infrastrcuture. Filter this out so that it doesn't cause spurious test failures such as in xfs/030 and xfs/033.Signed-off-by Dave Chinner <[email protected]>
Looks good,
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs