xfstests: Add a group of reservation space test
Subsystems (2)
cover xfstests: Add a group of reservation space test
This patch set add a group of reservation space test. Especailly for btrfs. It includes three parts: copy workload, prealloc and write posix.
For test 264, I hope it's usefull for Josef's reserve improve work. For test 265 and 266, the current btrfs is not pass yet.
TO avoid fill the huge disk, I just resize the filesystem size which NOT all filesystems supportted. Here I just make these tests support btrfs only, but it's can ream to ext3/4 if needed.
Wu Bo (3):
xfstests 264: add a copy and reserve test
xfstests 265: add a prealloc and reserve test
xfstests 266: add a write and reserve test
264 | 158 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 264.out | 6 ++ 265 | 107 ++++++++++++++++++++++++++++++++++++++++++ 265.out | 5 ++ 266 | 104 +++++++++++++++++++++++++++++++++++++++++ 266.out | 5 ++ group | 3 + 7 files changed, 388 insertions(+), 0 deletions(-) create mode 100755 264 create mode 100644 264.out create mode 100755 265 create mode 100644 265.out create mode 100755 266 create mode 100644 266.out
Signature · 1 line
1.7.3.1
PATCH 1/3 xfstests 264: add a copy and reserve test
This test is a stress test. It creates a set of threads for coping small files into disk. I use a 2G disk for test, the ENOSPC arises usually but the disk is not full under kenerl 3.0 with intel64.
--- 264 | 158 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 264.out | 6 ++ group | 1 + 3 files changed, 165 insertions(+), 0 deletions(-) create mode 100755 264 create mode 100644 264.out
264 new file +158 -0
@@ -0,0 +1,158 @@ +#! /bin/bash +# FS QA Test No. 264 +# +# reservation test with heavy cp workload +# +#----------------------------------------------------------------------- +# Copyright (c) 2011-2012 Fujitsu, Inc. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +# +#----------------------------------------------------------------------- +# +#creator [email protected] + +seq=`basename $0` +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=0 # success is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -rf $TEST_DIR/* $tmp.* + + if [ $_resize -eq 1 ] + then + btrfs filesystem resize ${_old}B $TEST_DIR > /dev/null 2>&1 + fi + + _cleanup_testdir +} + +. ./common.rc +. ./common.filter + +threads=0 # will be set later +count=1 + +_threads_set() +{ + _cpu_num=`cat /proc/cpuinfo | grep "processor" | wc -l` + threads=$(( $_cpu_num * 50 )) + if [ $threads -gt 200 ] + then + threads=200 + fi + + echo "create $threads threads to run..." +} + +_file_create() +{ + _i=0 + + if ! mkdir $TEST_DIR/origin + then + echo "mkdir origin err" + status=1 + exit + fi + + cd $TEST_DIR/origin + + _disksize=`df --block-size=1 | grep $TEST_DEV | awk '{print $2}'` + _disksize=$(( $_disksize / 3 )) + _num=$(( $_disksize / $count / $threads / 4096 )) + _count=$count + while [ $_i -lt $_num ] + do + dd if=/dev/zero of=file_$_i bs=4096 count=$_count > /dev/null 2>&1 + _i=$(( $_i + 1 )) + done + + cd $here +} + +_porter() +{ + _suffix=$1 + + if ! mkdir $TEST_DIR/sub_$_suffix + then + echo "mkdir sub_xxx err" + status=1 + exit + fi + + cp -r $TEST_DIR/origin $TEST_DIR/sub_$_suffix + + sync +} + +_do_workload() +{ + _pid=1 + + rm -rf $TEST_DIR/* + + _threads_set + _file_create + + _threads=$threads + + while [ $_pid -lt $_threads ] + do + _porter $_pid & + + _pid=$(( $_pid + 1 )) + done + + wait +} + +# real QA test starts here +_supported_fs btrfs +_supported_os IRIX Linux + +_setup_testdir +_check_test_fs + +echo "------------------------------" +echo "start the workload" +echo "------------------------------" + +_resize=0 +_new=$(( 2 * 1024 * 1024 * 1024 )) # 2G +_old=`df --block-size=1 | grep $TEST_DEV | awk '{print $2}'` +if [ $_new -lt $_old ] +then + btrfs filesystem resize ${_new}B $TEST_DIR > /dev/null 2>&1 + if [ $? -ne 0 ] + then + echo "btrfs filesystem resize err" + status=1 + exit + fi + _resize=1 +fi + +_do_workload + +echo "done" +exit
264.out new file +6 -0
@@ -0,0 +1,6 @@ +QA output created by 264 +------------------------------ +start the workload +------------------------------ +create 100 threads to run... +done
group +1 -0
@@ -377,3 +377,4 @@ deprecated 261 auto quick quota 262 auto quick quota 263 rw auto quick +264 auto rw
Signature · 1 line
1.7.3.1
Re: [PATCH 1/3] xfstests 264: add a copy and reserve test
Quoted 3 lines · WuBo
This test is a stress test. It creates a set of threads for coping small files into disk. I use a 2G disk for test, the ENOSPC arises usually but the disk is not full under kenerl 3.0 with intel64.
It seems like you really want to use _scratch_mkfs_sized instead of the btrfs-specific resize option and make the test generic. Also we already have used up these test numbers in the xfstests-dev repository, and I will commit another batch of tests today.
Re: [PATCH 1/3] xfstests 264: add a copy and reserve test
Quoted 7 lines · Christoph Hellwig
Quoted 3 lines · WuBo
This test is a stress test. It creates a set of threads for coping small files into disk. I use a 2G disk for test, the ENOSPC arises usually but the disk is not full under kenerl 3.0 with intel64.
It seems like you really want to use
_scratch_mkfs_sizedinstead of the btrfs-specific resize option and make the test generic. Also we already
OK, thanks for _scratch_mkfs_sized, It's will make these tests generic.
Quoted 2 lines
have used up these test numbers in the xfstests-dev repository, and I will commit another batch of tests today.
I will change my test numbers in next version.
thanks,
wubo
Quoted 6 lines
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 2/3 xfstests 265: add a prealloc and reserve test
In reply to WuBoThis test is for preallocation test. If the disk is full, just with a prealloc file has some free space that prealloc early. We need to check whether the write to the free space is success or not.
--- 265 | 107 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 265.out | 5 +++ group | 1 + 3 files changed, 113 insertions(+), 0 deletions(-) create mode 100755 265 create mode 100644 265.out
265 new file +107 -0
@@ -0,0 +1,107 @@ +#! /bin/bash +# FS QA Test No. 265 +# +# preallocation test +# +#----------------------------------------------------------------------- +# Copyright (c) 2011-2012 Fujitsu, Inc. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +# +#----------------------------------------------------------------------- +# +#creator [email protected] + +seq=`basename $0` +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=0 # success is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + rm -f $TEST_DIR/* $tmp.* + + if [ $_resize -eq 1 ] + then + btrfs filesystem resize ${_old}B $TEST_DIR > /dev/null 2>&1 + fi + + _cleanup_testdir +} + +. ./common.rc +. ./common.filter + +# real QA test starts here +_supported_fs btrfs +_supported_os IRIX Linux + +_setup_testdir +_check_test_fs + +echo "------------------------------" +echo "preallocation test" +echo "------------------------------" + +_resize=0 +_new=$(( 1 * 1024 * 1024 * 1024 )) # 1G +_old=`df --block-size=1 | grep $TEST_DEV | awk '{print $2}'` +if [ $_new -lt $_old ] +then + btrfs filesystem resize ${_new}B $TEST_DIR > /dev/null 2>&1 + if [ $? -ne 0 ] + then + echo "btrfs filesystem resize err" + status=1 + exit + fi + _resize=1 +fi + +rm -rf $TEST_DIR/* +cd $TEST_DIR +dd if=/dev/zero of=test bs=4K count=1 > /dev/null 2>&1 +if [ $? -ne 0 ] +then + echo "create file err" + status=1 + exit +fi + +fallocate -n -o 4K -l 1M test > /dev/null 2>&1 +if [ $? -ne 0 ] +then + echo "fallocate file err" + status=1 + exit +fi + +dd if=/dev/zero of=tmp1 bs=1M > /dev/null 2>&1 +dd if=/dev/zero of=tmp2 bs=4K > /dev/null 2>&1 +sync + +dd if=/dev/zero of=test seek=1 bs=4K count=2 conv=notrunc > /dev/null 2>&1 +if [ $? -ne 0 ] +then + echo "fill prealloc range err" + status=1 + exit +fi + +echo "done" +exit
265.out new file +5 -0
@@ -0,0 +1,5 @@ +QA output created by 265 +------------------------------ +preallocation test +------------------------------ +done
group +1 -0
@@ -378,3 +378,4 @@ deprecated 262 auto quick quota 263 rw auto quick 264 auto rw +265 auto rw
Signature · 1 line
1.7.3.1
Re: [PATCH 2/3] xfstests 265: add a prealloc and reserve test
Hi Wu Bo,
Quoted 5 lines · WuBo
This test is for preallocation test. If the disk is full, just with a prealloc file has some free space that prealloc early. We need to check whether the write to the free space is success or not.
Signed-off-by Wu Bo <[email protected]>
This test is failing for me because I don't have fallocate installed. I suggest the test could to be changed to check for binaries it uses, possibly the version of those binaries, and then not run unless the right ones are installed. But the best I can do right now is make a note of it.
Just a heads up. ;)
Regards,
Ben
Re: [PATCH 2/3] xfstests 265: add a prealloc and reserve test
Quoted 7 lines · Ben Myers
Hi Wu Bo,
Quoted 4 lines · WuBo
This test is for preallocation test. If the disk is full, just with a prealloc file has some free space that prealloc early. We need to check whether the write to the free space is success or not.
Quoted 7 lines
Quoted 1 line
Signed-off-by Wu Bo <[email protected]>This test is failing for me because I don't have fallocate installed. I suggest the test could to be changed to check for binaries it uses, possibly the version of those binaries, and then not run unless the right ones are installed. But the best I can do right now is make a note of it.
It might be even better to just use the xfs_io falloc command as we generally expect an uptodate xfs_io for use with xfstests.
Re: [PATCH 2/3] xfstests 265: add a prealloc and reserve test
Quoted 10 lines · Christoph Hellwig
Quoted 9 lines · Ben Myers
Hi Wu Bo,
Quoted 5 lines · WuBo
This test is for preallocation test. If the disk is full, just with a prealloc file has some free space that prealloc early. We need to check whether the write to the free space is success or not.
Signed-off-by Wu Bo <[email protected]>
Quoted 8 lines
Quoted 5 lines
This test is failing for me because I don't have fallocate installed. I suggest the test could to be changed to check for binaries it uses, possibly the version of those binaries, and then not run unless the right ones are installed. But the best I can do right now is make a note of it.
It might be even better to just use the
xfs_iofalloc command as we generally expect an uptodatexfs_iofor use with xfstests.
Got it.
thanks,
wubo
Quoted 6 lines
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 3/3 xfstests 266: add a write and reserve test
In reply to WuBoThis test is for write-posix test. If writing a file when the disk is almost full, the posix wants the call to write as much as possible but not none.
quote the POSIX:
If a write() requests that more bytes be written than there is room for (for example, [XSI] [Option Start] the process' file size limit or [Option End] the physical end of a medium), only as many bytes as there is room for shall be written. For example, suppose there is space for 20 bytes more in a file before reaching a limit. A write of 512 bytes will return 20. The next write of a non-zero number of bytes would give a failure return (except as noted below).
--- 266 | 104 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 266.out | 5 +++ group | 1 + 3 files changed, 110 insertions(+), 0 deletions(-) create mode 100755 266 create mode 100644 266.out
266 new file +104 -0
@@ -0,0 +1,104 @@ +#! /bin/bash +# FS QA Test No. 266 +# +# The posix write test. when write size is larger than disk free size, +# should write as more as possible +# +#----------------------------------------------------------------------- +# Copyright (c) 2011-2012 Fujitsu, Inc. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +# +#----------------------------------------------------------------------- +# +#creator [email protected] + +seq=`basename $0` +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=0 # success is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + rm -f $TEST_DIR/* $tmp.* + + if [ $_resize -eq 1 ] + then + btrfs filesystem resize ${_old}B $TEST_DIR > /dev/null 2>&1 + fi + + _cleanup_testdir +} + +. ./common.rc +. ./common.filter + +# real QA test starts here +_supported_fs btrfs +_supported_os IRIX Linux + +_setup_testdir +_check_test_fs + +echo "------------------------------" +echo "write lack test" +echo "------------------------------" + +_resize=0 +_new=$(( 1 * 1024 * 1024 * 1024 )) # 1G +_old=`df --block-size=1 | grep $TEST_DEV | awk '{print $2}'` +if [ $_new -lt $_old ] +then + btrfs filesystem resize ${_new}B $TEST_DIR > /dev/null 2>&1 + if [ $? -ne 0 ] + then + echo "btrfs filesystem resize err" + status=1 + exit + fi + _resize=1 +fi + +rm -rf $TEST_DIR/* +cd $TEST_DIR +dd if=/dev/zero of=tmp1 bs=4K count=1 > /dev/null 2>&1 +if [ $? -ne 0 ] +then + echo "create file err" + status=1 + exit +fi + +dd if=/dev/zero of=tmp2 bs=1M > /dev/null 2>&1 +dd if=/dev/zero of=tmp3 bs=4K > /dev/null 2>&1 +sync + +rm -f tmp1 +sync + +dd if=/dev/zero of=tmp1 bs=8K count=1 > /dev/null 2>&1 +_filesize=`du tmp1 | awk '{print $1}'` +if [ $_filesize -ne 4 ] +then + echo "write file err" + status=1 + exit +fi + +echo "done" +exit
266.out new file +5 -0
@@ -0,0 +1,5 @@ +QA output created by 266 +------------------------------ +write lack test +------------------------------ +done
group +1 -0
@@ -379,3 +379,4 @@ deprecated 263 rw auto quick 264 auto rw 265 auto rw +266 auto rw
Signature · 1 line
1.7.3.1
PATCH 1/1 change fallocate to xfs_io falloc
In reply to WuBoThanks to Ben for pointing it that the fallocate maybe uninstall without checking it, and change fallocate to xfs_io as Christoph suggested.
--- 274 | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-)
274 +2 -1
@@ -46,6 +46,7 @@ _cleanup() _supported_fs generic _supported_os IRIX Linux _require_scratch +_require_xfs_io_falloc echo "------------------------------" echo "preallocation test"
@@ -67,7 +68,7 @@ then exit fi -fallocate -n -o 4K -l 1M test >/dev/null 2>&1 +$XFS_IO_PROG -F -f -c 'falloc 4096 1m' test >/dev/null if [ $? -ne 0 ] then echo "fallocate file err"
Signature · 6 lines
1.7.3.1 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH] change fallocate to xfs_io falloc
Hey Wu Bo,
Quoted 5 lines · WuBo
Thanks to Ben for pointing it that the fallocate maybe uninstall without checking it, and change fallocate to
xfs_ioas Christoph suggested.Signed-off-by Wu Bo <[email protected]>
Nice! It works great for me. ;)
Thanks,
Ben
Quoted 32 lines
--- 274 | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-)274 +2
-1@@ -46,6 +46,7 @@ _cleanup() _supported_fs generic _supported_os IRIX Linux _require_scratch +_require_xfs_io_falloc echo "------------------------------" echo "preallocation test"@@ -67,7 +68,7 @@ then exit fi -fallocate -n -o 4K -l 1M test >/dev/null 2>&1 +$XFS_IO_PROG -F -f -c 'falloc 4096 1m' test >/dev/null if [ $? -ne 0 ] then echo "fallocate file err"Signature · 6 lines
1.7.3.1 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs