xfstests: generic xattr enospc cleanup test
PATCH 1/1 xfstests: generic xattr enospc cleanup test
XFS had a regression where inode reclaim in the unlink codepath would not correctly tear down extended attribute forks where no xattr extents are present. Add a generic test to create this condition.
The test sets extended attributes on a series of files under ENOSPC conditions and then verifies that the files can be removed without syslog warnings or errors.
--- tests/generic/103 | 91 +++++++++++++++++++++++++++++++++++++++++++++++++++ tests/generic/103.out | 2 ++ tests/generic/group | 1 + 3 files changed, 94 insertions(+) create mode 100755 tests/generic/103 create mode 100644 tests/generic/103.out
tests/generic/103 new file +91 -0
@@ -0,0 +1,91 @@ +#! /bin/bash +# FSQA Test No. 103 +# +# Test attribute fork teardown. This test is inspired by a regression in XFS +# that resulted in problematic removal of inodes with remote attribute forks +# without attribute extents. The attribute fork condition is created by +# attempting to set larger attribute values on a filesystem that is at or near +# ENOSPC. +# +#----------------------------------------------------------------------- +# Copyright (c) 2015 Red Hat, Inc. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +# +#----------------------------------------------------------------------- +# + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 25 + +# get standard environment, filters and checks +. ./common/rc +. ./common/attr + +# real QA test starts here +_supported_fs generic +_supported_os Linux +_require_scratch +_require_attrs + +rm -f $seqres.full + +_consume_freesp() +{ + file=$1 + + # consume nearly all available space (leave ~512kB) + avail=`_get_available_space $SCRATCH_MNT` + filesizekb=$((avail / 1024 - 512)) + $XFS_IO_PROG -fc "falloc 0 ${filesizekb}k" $file +} + +_scratch_mkfs >> $seqres.full 2>&1 +_scratch_mount + +for i in $(seq 0 63); do + touch $SCRATCH_MNT/$seq.$i +done + +# Generate a large attribute value and consume the rest of the space in the +# filesystem. +$XFS_IO_PROG -fc "pwrite 0 64k" $SCRATCH_MNT/attrval > /dev/null 2>&1 +_consume_freesp $SCRATCH_MNT/spc + +# Set attributes on the test files. These should start to hit ENOSPC. +for i in $(seq 0 63); do + $SETFATTR_PROG -n user.test -v "`cat $SCRATCH_MNT/attrval`" \ + $SCRATCH_MNT/$seq.$i >> $seqres.full 2>&1 +done + +# Remove the files with attributes to test attribute fork teardown. Problems +# result in dmesg output. +rm -f $SCRATCH_MNT/$seq.* + +echo Silence is golden. + +status=0 +exit
tests/generic/103.out new file +2 -0
@@ -0,0 +1,2 @@ +QA output created by 103 +Silence is golden.
tests/generic/group +1 -0
@@ -98,6 +98,7 @@ 097 udf auto 099 udf auto 100 udf auto +103 auto enospc quick 105 acl auto quick 112 rw aio auto quick 113 rw aio auto quick
Signature · 1 line
1.9.3
Quoted 74 lines · Brian Foster
XFS had a regression where inode reclaim in the unlink codepath would not correctly tear down extended attribute forks where no xattr extents are present. Add a generic test to create this condition.
The test sets extended attributes on a series of files under ENOSPC conditions and then verifies that the files can be removed without syslog warnings or errors.
Signed-off-by Brian Foster <[email protected]>--- tests/generic/103 | 91 +++++++++++++++++++++++++++++++++++++++++++++++++++ tests/generic/103.out | 2 ++ tests/generic/group | 1 + 3 files changed, 94 insertions(+) create mode 100755 tests/generic/103 create mode 100644 tests/generic/103.outtests/generic/103 new file +51
-0@@ -0,0 +1,91 @@ +#! /bin/bash +# FSQA Test No. 103 +# +# Test attribute fork teardown. This test is inspired by a regression in XFS +# that resulted in problematic removal of inodes with remote attribute forks +# without attribute extents. The attribute fork condition is created by +# attempting to set larger attribute values on a filesystem that is at or near +# ENOSPC. +# +#----------------------------------------------------------------------- +# Copyright (c) 2015 Red Hat, Inc. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +# +#----------------------------------------------------------------------- +# + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 25 + +# get standard environment, filters and checks +. ./common/rc +. ./common/attr + +# real QA test starts here +_supported_fs generic +_supported_os Linux +_require_scratch
tests/generic/103 line 61
Quoted 10 lines
tests/generic/103 new file +10
-0+_require_attrs + +rm -f $seqres.full + +_consume_freesp() +{ + file=$1 + + # consume nearly all available space (leave ~512kB) + avail=`_get_available_space $SCRATCH_MNT`
Seems you missed "_get_available_space" function definition in common/rc? I got "command not found" error.
Thanks,
Eryu
Quoted 57 lines
tests/generic/103 new file +30
-0+ filesizekb=$((avail / 1024 - 512)) + $XFS_IO_PROG -fc "falloc 0 ${filesizekb}k" $file +} + +_scratch_mkfs >> $seqres.full 2>&1 +_scratch_mount + +for i in $(seq 0 63); do + touch $SCRATCH_MNT/$seq.$i +done + +# Generate a large attribute value and consume the rest of the space in the +# filesystem. +$XFS_IO_PROG -fc "pwrite 0 64k" $SCRATCH_MNT/attrval > /dev/null 2>&1 +_consume_freesp $SCRATCH_MNT/spc + +# Set attributes on the test files. These should start to hit ENOSPC. +for i in $(seq 0 63); do + $SETFATTR_PROG -n user.test -v "`cat $SCRATCH_MNT/attrval`" \ + $SCRATCH_MNT/$seq.$i >> $seqres.full 2>&1 +done + +# Remove the files with attributes to test attribute fork teardown. Problems +# result in dmesg output. +rm -f $SCRATCH_MNT/$seq.* + +echo Silence is golden. + +status=0 +exittests/generic/103.out new file +2
-0@@ -0,0 +1,2 @@ +QA output created by 103 +Silence is golden.tests/generic/group +1
-0@@ -98,6 +98,7 @@ 097 udf auto 099 udf auto 100 udf auto +103 auto enospc quick 105 acl auto quick 112 rw aio auto quick 113 rw aio auto quickSignature · 6 lines
1.9.3 -- To unsubscribe from this list: send the line "unsubscribe fstests" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 80 lines · Eryu Guan
Quoted 79 lines · Brian Foster
XFS had a regression where inode reclaim in the unlink codepath would not correctly tear down extended attribute forks where no xattr extents are present. Add a generic test to create this condition.
The test sets extended attributes on a series of files under ENOSPC conditions and then verifies that the files can be removed without syslog warnings or errors.
Signed-off-by Brian Foster <[email protected]>--- tests/generic/103 | 91 +++++++++++++++++++++++++++++++++++++++++++++++++++ tests/generic/103.out | 2 ++ tests/generic/group | 1 + 3 files changed, 94 insertions(+) create mode 100755 tests/generic/103 create mode 100644 tests/generic/103.outtests/generic/103 new file +56
-0@@ -0,0 +1,91 @@ +#! /bin/bash +# FSQA Test No. 103 +# +# Test attribute fork teardown. This test is inspired by a regression in XFS +# that resulted in problematic removal of inodes with remote attribute forks +# without attribute extents. The attribute fork condition is created by +# attempting to set larger attribute values on a filesystem that is at or near +# ENOSPC. +# +#----------------------------------------------------------------------- +# Copyright (c) 2015 Red Hat, Inc. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +# +#----------------------------------------------------------------------- +# + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 25 + +# get standard environment, filters and checks +. ./common/rc +. ./common/attr + +# real QA test starts here +_supported_fs generic +_supported_os Linux +_require_scratch +_require_attrs + +rm -f $seqres.full + +_consume_freesp()
Quoted 9 lines
tests/generic/103 line 61
Quoted 5 lines
tests/generic/103 new file +5
-0+{ + file=$1 + + # consume nearly all available space (leave ~512kB) + avail=`_get_available_space $SCRATCH_MNT`Seems you missed "
_get_available_space" function definition in common/rc? I got "command not found" error.
Ah, sorry... that is actually part of a previous patch. I meant to point out that this depends on the following patch due to the use of the new helper:
http://thread.gmane.org/gmane.comp.file-systems.fstests/834
Sorry for the confusion.
Brian
Quoted 59 lines
Thanks,
EryuQuoted 57 lines
tests/generic/103 new file +30
-0+ filesizekb=$((avail / 1024 - 512)) + $XFS_IO_PROG -fc "falloc 0 ${filesizekb}k" $file +} + +_scratch_mkfs >> $seqres.full 2>&1 +_scratch_mount + +for i in $(seq 0 63); do + touch $SCRATCH_MNT/$seq.$i +done + +# Generate a large attribute value and consume the rest of the space in the +# filesystem. +$XFS_IO_PROG -fc "pwrite 0 64k" $SCRATCH_MNT/attrval > /dev/null 2>&1 +_consume_freesp $SCRATCH_MNT/spc + +# Set attributes on the test files. These should start to hit ENOSPC. +for i in $(seq 0 63); do + $SETFATTR_PROG -n user.test -v "`cat $SCRATCH_MNT/attrval`" \ + $SCRATCH_MNT/$seq.$i >> $seqres.full 2>&1 +done + +# Remove the files with attributes to test attribute fork teardown. Problems +# result in dmesg output. +rm -f $SCRATCH_MNT/$seq.* + +echo Silence is golden. + +status=0 +exittests/generic/103.out new file +2
-0@@ -0,0 +1,2 @@ +QA output created by 103 +Silence is golden.tests/generic/group +1
-0@@ -98,6 +98,7 @@ 097 udf auto 099 udf auto 100 udf auto +103 auto enospc quick 105 acl auto quick 112 rw aio auto quick 113 rw aio auto quickSignature · 6 lines
1.9.3 -- To unsubscribe from this list: send the line "unsubscribe fstests" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 76 lines · Brian Foster
XFS had a regression where inode reclaim in the unlink codepath would not correctly tear down extended attribute forks where no xattr extents are present. Add a generic test to create this condition.
The test sets extended attributes on a series of files under ENOSPC conditions and then verifies that the files can be removed without syslog warnings or errors.
Signed-off-by Brian Foster <[email protected]>--- tests/generic/103 | 91 +++++++++++++++++++++++++++++++++++++++++++++++++++ tests/generic/103.out | 2 ++ tests/generic/group | 1 + 3 files changed, 94 insertions(+) create mode 100755 tests/generic/103 create mode 100644 tests/generic/103.outtests/generic/103 new file +53
-0@@ -0,0 +1,91 @@ +#! /bin/bash +# FSQA Test No. 103 +# +# Test attribute fork teardown. This test is inspired by a regression in XFS +# that resulted in problematic removal of inodes with remote attribute forks +# without attribute extents. The attribute fork condition is created by +# attempting to set larger attribute values on a filesystem that is at or near +# ENOSPC. +# +#----------------------------------------------------------------------- +# Copyright (c) 2015 Red Hat, Inc. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +# +#----------------------------------------------------------------------- +# + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 25 + +# get standard environment, filters and checks +. ./common/rc +. ./common/attr + +# real QA test starts here +_supported_fs generic +_supported_os Linux +_require_scratch +_require_attrs +
tests/generic/103 line 63
Quoted 10 lines
tests/generic/103 new file +10
-0+rm -f $seqres.full + +_consume_freesp() +{ + file=$1 + + # consume nearly all available space (leave ~512kB) + avail=`_get_available_space $SCRATCH_MNT` + filesizekb=$((avail / 1024 - 512)) + $XFS_IO_PROG -fc "falloc 0 ${filesizekb}k" $file
This fails ext2/3 which don't have fallocate(2) support, need _require_xfs_io_command "falloc".
This blocks ext2/3 from testing but extN just doesn't support 64k xattr value, we don't lose much test coverage. On the other hand, changing falloc to pwrite extends test time significantly (2s -> 30+s) and doesn't gain much benefit.
Quoted 13 lines
tests/generic/103 new file +13
-0+} + +_scratch_mkfs >> $seqres.full 2>&1 +_scratch_mount + +for i in $(seq 0 63); do + touch $SCRATCH_MNT/$seq.$i +done + +# Generate a large attribute value and consume the rest of the space in the +# filesystem. +$XFS_IO_PROG -fc "pwrite 0 64k" $SCRATCH_MNT/attrval > /dev/null 2>&1 +_consume_freesp $SCRATCH_MNT/spc
tests/generic/103 line 86
Quoted 10 lines
tests/generic/103 new file +10
-0+ +# Set attributes on the test files. These should start to hit ENOSPC. +for i in $(seq 0 63); do + $SETFATTR_PROG -n user.test -v "`cat $SCRATCH_MNT/attrval`" \ + $SCRATCH_MNT/$seq.$i >> $seqres.full 2>&1 +done + +# Remove the files with attributes to test attribute fork teardown. Problems +# result in dmesg output. +rm -f $SCRATCH_MNT/$seq.*
I see xfs corruption after the test, I think that's expected.
But I just want to point out here that _check_dmesg is unable to detect the call traces in dmesg now, as the call trace doesn't have "WARNING", "kernel BUG" etc. keywords that _check_dmesg searches for, we rely on _check_fimesystems to fail the test.
Thanks,
Eryu
Quoted 32 lines
tests/generic/103 new file +5
-0+ +echo Silence is golden. + +status=0 +exittests/generic/103.out new file +2
-0@@ -0,0 +1,2 @@ +QA output created by 103 +Silence is golden.tests/generic/group +1
-0@@ -98,6 +98,7 @@ 097 udf auto 099 udf auto 100 udf auto +103 auto enospc quick 105 acl auto quick 112 rw aio auto quick 113 rw aio auto quickSignature · 6 lines
1.9.3 -- To unsubscribe from this list: send the line "unsubscribe fstests" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 82 lines · Eryu Guan
Quoted 81 lines · Brian Foster
XFS had a regression where inode reclaim in the unlink codepath would not correctly tear down extended attribute forks where no xattr extents are present. Add a generic test to create this condition.
The test sets extended attributes on a series of files under ENOSPC conditions and then verifies that the files can be removed without syslog warnings or errors.
Signed-off-by Brian Foster <[email protected]>--- tests/generic/103 | 91 +++++++++++++++++++++++++++++++++++++++++++++++++++ tests/generic/103.out | 2 ++ tests/generic/group | 1 + 3 files changed, 94 insertions(+) create mode 100755 tests/generic/103 create mode 100644 tests/generic/103.outtests/generic/103 new file +58
-0@@ -0,0 +1,91 @@ +#! /bin/bash +# FSQA Test No. 103 +# +# Test attribute fork teardown. This test is inspired by a regression in XFS +# that resulted in problematic removal of inodes with remote attribute forks +# without attribute extents. The attribute fork condition is created by +# attempting to set larger attribute values on a filesystem that is at or near +# ENOSPC. +# +#----------------------------------------------------------------------- +# Copyright (c) 2015 Red Hat, Inc. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +# +#----------------------------------------------------------------------- +# + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 25 + +# get standard environment, filters and checks +. ./common/rc +. ./common/attr + +# real QA test starts here +_supported_fs generic +_supported_os Linux +_require_scratch +_require_attrs + +rm -f $seqres.full + +_consume_freesp() +{ + file=$1
Quoted 9 lines
tests/generic/103 line 63
Quoted 5 lines
tests/generic/103 new file +5
-0+ + # consume nearly all available space (leave ~512kB) + avail=`_get_available_space $SCRATCH_MNT` + filesizekb=$((avail / 1024 - 512)) + $XFS_IO_PROG -fc "falloc 0 ${filesizekb}k" $fileThis fails ext2/3 which don't have fallocate(2) support, need
_require_xfs_io_command"falloc".
Ok.
Quoted 5 lines
This blocks ext2/3 from testing but extN just doesn't support 64k xattr value, we don't lose much test coverage. On the other hand, changing falloc to pwrite extends test time significantly (2s -> 30+s) and doesn't gain much benefit.
Yeah, I don't think it's worth that. It runs really quickly now and the reason behind it is an XFS regression.
Quoted 17 lines
Quoted 17 lines
tests/generic/103 new file +17
-0+} + +_scratch_mkfs >> $seqres.full 2>&1 +_scratch_mount + +for i in $(seq 0 63); do + touch $SCRATCH_MNT/$seq.$i +done + +# Generate a large attribute value and consume the rest of the space in the +# filesystem. +$XFS_IO_PROG -fc "pwrite 0 64k" $SCRATCH_MNT/attrval > /dev/null 2>&1 +_consume_freesp $SCRATCH_MNT/spc + +# Set attributes on the test files. These should start to hit ENOSPC. +for i in $(seq 0 63); do + $SETFATTR_PROG -n user.test -v "`cat $SCRATCH_MNT/attrval`" \
Quoted 9 lines
tests/generic/103 line 86
Quoted 6 lines
tests/generic/103 new file +6
-0+ $SCRATCH_MNT/$seq.$i >> $seqres.full 2>&1 +done + +# Remove the files with attributes to test attribute fork teardown. Problems +# result in dmesg output. +rm -f $SCRATCH_MNT/$seq.*I see xfs corruption after the test, I think that's expected.
The problem should be fixed upstream, though 4.1 might still fail. I also need to backport the fix now that it's merged. ;)
Quoted 5 lines
But I just want to point out here that
_check_dmesgis unable to detect the call traces in dmesg now, as the call trace doesn't have "WARNING", "kernel BUG" etc. keywords that_check_dmesgsearches for, we rely on_check_fimesystemsto fail the test.
Good catch, thanks. The failure here is an XFS error as opposed to a BUG() or warning. I've updated _check_dmesg() to catch "Internal error" as well.
Brian
Quoted 34 lines
Thanks,
EryuQuoted 32 lines
tests/generic/103 new file +5
-0+ +echo Silence is golden. + +status=0 +exittests/generic/103.out new file +2
-0@@ -0,0 +1,2 @@ +QA output created by 103 +Silence is golden.tests/generic/group +1
-0@@ -98,6 +98,7 @@ 097 udf auto 099 udf auto 100 udf auto +103 auto enospc quick 105 acl auto quick 112 rw aio auto quick 113 rw aio auto quickSignature · 6 lines
1.9.3 -- To unsubscribe from this list: send the line "unsubscribe fstests" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html