snapshot, defragment and raid test cases for btrfs

Quoted 6 lines · Anand Jain

Hi,
Attached is the patch for the xfstests, which adds snapshot, defragment and volume management test cases for the btrfs (257, 258 and 259 respectively).

A patch per test is the preferred form for review.

Quoted 2 lines

This introduces a new user variable 'DISK_POOL' which should be set to disks for the raid tests.

It's used as part of the scratch device, so it'd probaby be better to call it "SCRATCH_DEV_POOL". Indeed, if you do that, you could then leave SCRATCH_DEV empty and fill it out with the first device of the pool by default when setting up the scratch device in the first place. That woul dthen get around all the mess you have where you have to do "$SCRATCH_DEV $DISK_POOL" concatenations in all the tests using the disk pool....

Quoted 38 lines

An example of usage of these tests is as below.
------------
[root@localhost xfstests]# cat local.config
TEST_DEV="/dev/sdd"
TEST_DIR=/btrfs
SCRATCH_DEV="/dev/sde"
SCRATCH_MNT=/btrfs1
DISK_POOL="/dev/sdf /dev/sdg"
[root@localhost xfstests]#
[root@localhost xfstests]# ./check 257 258 259
FSTYP -- btrfs
PLATFORM -- Linux/i686 localhost 3.0.0-rc6+
MKFS_OPTIONS -- /dev/sde
MOUNT_OPTIONS -- -o context=system_u:object_r:nfs_t:s0 /dev/sde /btrfs1
257 8s
258 3s
259 33s
Ran: 257 258 259
Passed all 3 tests
----------------
Thank you.
Anand
From 62d90cd24afc192e091d9b81fb78d7c666237e97 Mon Sep 17 00:00:00 2001
From: Anand Jain <[email protected]>
Date: Fri, 5 Aug 2011 15:36:25 +0800
Subject: [PATCH] Adding 257, 258, 259 to test snapshot, defragment and raid in btrfs
Adding test case number 257, 258 and 259 to test snapshot, defragment and raid support in btrfs. This also adds a user input variable 'DISK_POOL'.
Signed-off-by Anand Jain <[email protected]>

Quoted 14 lines

---
 257               |  198 ++++++++++++++++++++++++++++++++++++++++++
 257.out           |    8 ++
 258               |   75 ++++++++++++++++
 258.out           |    2 +
 259               |  220 +++++++++++++++++++++++++++++++++++++++++++++++
 259.out           |    9 ++
 common.rc         |   58 +++++++++++++
 group             |    3 +
 src/checksum_misc |   28 ++++++
 src/devmgt        |   25 ++++++
 src/fsmisc        |  247 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 src/randmisc      |   70 +++++++++++++++
 src/snapmisc      |   73 ++++++++++++++++

Put all of these "misc" stuff in a common.snapshot file or something similar. The src directory is for source code to binaries that need to be built for other tests...

Quoted 69 lines

257 new file +63 -0

new file mode 100755
index 0000000..2a75944
@@ -0,0 +1,198 @@
+#! /bin/bash
+# FS QA Test No. 257
+#
+# Extented btrfs snapshot test cases
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2011 Oracle  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#
+#-----------------------------------------------------------------------
+#
+# creator
[email protected]
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+
+_cleanup()
+{
+    rm -f $tmp.*
+}
+
+trap "_cleanup ; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_need_to_be_root
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+
+. ./src/checksum_misc
+. ./src/randmisc
+. ./src/fsmisc
+. ./src/snapmisc
+
+_scratch_mkfs > /dev/null 2>&1
+_scratch_mount
+
+#FS fill size
+#name="DirDepth MinFiles MaxFiles MinFileSz MaxFileSz"
+QUICK="1 4 4 4096 4096"
+SMALL="1 10 100 4096 8192"
+MEDIUM="10 30 1000 4096 8192"
+LARGE="1000 1 1000 4096 1048576"

257 line 73

Quoted 10 lines

257 new file +10 -0

+FDPROF=$QUICK
+dolog=1
+
+# compare check sum of $1 with $2
+_testa_snap_check()
+{
+	local volname
+	volname=`echo $1 | rev | cut -d"/" -f1 | rev`
+	#$here/src/checksum_misc save_checksum $1 $tmp.$volname.sum
+	save_checksum $1 $tmp.$volname.sum

Please remove dead code (from eveywhere).

257 line 76

Quoted 3 lines

257 new file +3 -0

+	#$here/src/checksum_misc verify_checksum $2 $tmp.$volname.sum
+	verify_checksum $2 $tmp.$volname.sum
+	[ $dolog = 1 ] && echo "testa_snap_check ok"

Please remove all the dolog stuff - the test requires the output to pass, so it isn't conditional.

As it is, i don't think this is the best way to verify checksums. See below as I get to the fsfill code....

257 line 89

Quoted 11 lines

257 new file +11 -0

+_testb_file_append_check()
+{
+	local volname
+
+	volname=`echo $1 | rev | cut -d"/" -f1 | rev`
+	#$here/src/checksum_misc save_checksum $1 $tmp.$volname.sum
+	save_checksum $1 $tmp.$volname.sum
+
+	# modify the snap
+	# $here/src/fsmisc modifyfs_fillblk ${2}
+	modifyfs_fillblk ${2}

modify_fs_fillblk is only used here. It doesn't need to be a generic function. just open code it here and not force people to look in a different place to find out what the test does.

Quoted 8 lines

257 new file +8 -0

+	#$here/src/fsmisc check_sum $1 $tmp.$volname.sum
+	verify_checksum $1 $tmp.$volname.sum
+	echo "testb_file_append_check ok"
+}
+
+
+# arg1: Original subvolume
+# arg2 : its snapshot

257 line 108

Quoted 10 lines

257 new file +10 -0

+_testc_file_append2_check()
+{
+	local volname
+
+	volname=`echo $1 | rev | cut -d"/" -f1 | rev`
+	#$here/src/checksum_misc save_checksum $1 $tmp.$volname.sum
+	save_checksum $1 $tmp.$volname.sum
+
+	#$here/src/fsmisc modifyfs_append $2
+	modifyfs_append $2

Ditto for modifyfs_append.

Quoted 7 lines

257 new file +7 -0

+
+	#$here/src/checksum_misc verify_checksum $1 $tmp.$volname.sum
+	verify_checksum $1 $tmp.$volname.sum
+	echo "testc_file_append2_check ok"
+}
+
+# arg1: Original subvolume

257 line 125

Quoted 10 lines

257 new file +10 -0

+# arg2 : its snapshot
+_testd_read_modify_check()
+{
+	local volname
+
+	volname=`echo $1 | rev | cut -d"/" -f1 | rev`
+	#$here/src/checksum_misc save_checksum $1 $tmp.$volname.sum
+	save_checksum $1 $tmp.$volname.sum
+	#$here/src/fsmisc modifyfs_readmodifywrite ${2}
+	modifyfs_readmodifywrite ${2}

Ditto for modifyfs_readmodifywrite.

Quoted 35 lines

257 new file +35 -0

+
+	#$here/src/checksum_misc verify_checksum $1 $tmp.$volname.sum
+	verify_checksum $1 $tmp.$volname.sum
+	echo "testd_read_modify_check ok"
+}
+
+# arg 1: depth
+# arg 2: base vol
+_teste_nested_snap_check()
+{
+	local n
+	local x
+
+	# we need two additional snapshot to compare
+	#$here/src/snapmisc create_nestedclones $(($1+2)) $2
+	create_nestedclones $(($1+2)) $2
+	#$here/src/snapmisc update_clonelist
+	update_clonelist
+
+	n=${#CLONE_LIST[*]}
+	if [ $n -lt $1 ]; then echo Clone List Error; status=1; exit; fi
+
+	dolog=0
+	for i in `seq 1 $1`
+	do
+		n=$(($n-1))
+		_testa_snap_check ${CLONE_LIST[$(($n-1))]} ${CLONE_LIST[$n]}
+	done
+	dolog=1
+	echo "teste_nested_snap_check ok"
+}
+
+_testf_delete_file_check()
+{
+	local org=$1

257 line 170

Quoted 10 lines

257 new file +10 -0

+	local org_n=`echo $org | rev | cut -d"/" -f1 | rev`
+	local afile
+
+	#$here/src/checksum_misc save_checksum $org $tmp.${org_n}.sum
+	save_checksum $org $tmp.${org_n}.sum
+
+	#afile=`$here/src/randmisc picka rfile ${2}`
+	afile=`picka rfile ${2}`
+	#$here/src/fsmisc modifyfs_delete $afile
+	modifyfs_delete $afile

Same for modifyfs_delete - all it does is run 'rm -f <file>'!

257 line 183

Quoted 13 lines

257 new file +13 -0

+
+	#$here/src/checksum_misc verify_checksum $org $tmp.${org_n}.sum
+	verify_checksum $org $tmp.${org_n}.sum
+
+	echo "testf_delete_file_check ok"
+}
+
+# real QA test starts here
+
+firstvol="$SCRATCH_MNT/sv1"
+btrfs subvolume create $firstvol | _filter_scratch
+#$here/src/fsmisc fillfs $firstvol $FDPROF
+fillfs $firstvol $FDPROF

If you are using the QUICK config, then you aren't using any of the randomness that fillfs actually provides, so why don't you use _populate_fs? From common.rc:

# Populate a filesystem with inodes for performance experiments #
# usage: populate [-v] [-n ndirs] [-f nfiles] [-d depth] [-r root] [-s size]

(yeah, I know, the function is called _populate_fs, not populate)

One of the reasons I suggest for usingthis is that I don't think randomness is a good property for a regression test - you cannot repeat the same test twice in a row, which defeats the purpose of having a regression test. See comments on the fillfs function below.

257 line 196

Quoted 13 lines

257 new file +13 -0

+SNAPNAME=0
+#$here/src/snapmisc create_snap $firstvol $SCRATCH_MNT
+create_snap $firstvol $SCRATCH_MNT
+# Single clone test
+_testa_snap_check $firstvol $SNAPNAME
+_testb_file_append_check $firstvol $SNAPNAME
+_testc_file_append2_check $firstvol $SNAPNAME
+_testd_read_modify_check $firstvol $SNAPNAME
+# nested clone test
+_teste_nested_snap_check  7 $firstvol  # as of now don't do beyond 7 - btrfs bug
+SNAPNAME=0
+create_snap $firstvol $SCRATCH_MNT
+_testf_delete_file_check $firstvol $SNAPNAME

Please separate the different tests by a line of whitespace. i.e. everywhere you've got a comment, put a line of whitespce before it.

....

[defrag test]

257 line 64

Quoted 7 lines

257 new file +7 -0

+#FS fill size
+#name="DirDepth MinFiles MaxFiles MinFileSz MaxFileSz"
+QUICK="1 4 4 4096 4096"
+SMALL="1 10 100 4096 8192"
+MEDIUM="10 30 1000 4096 8192"
+LARGE="1000 1 1000 4096 1048576"
+FDPROF=$QUICK

You only use one of these test setups, so why even have two levels of indirection and a bunch of unused configurations?

257

Quoted 5 lines

257 new file +5 -0

+
+_testa_defrag()
+{
+        btrfs filesystem defragment $1
+        if [ ! $? ]; then echo "Error: Defrag failed"; exit; fi

The normal thing to do here is:

        btrfs filesystem defragment $1 || _fail "Defrag failed with $?"

257

Quoted 5 lines

257 new file +5 -0

+        echo "testa_defrag.... ok"
+}
+
+fillfs $SCRATCH_MNT $FDPROF
+_testa_defrag $SCRATCH_MNT

You don't need a function here. The test is effectively:

echo "silence is golden"

# fillfs DirDepth MinFiles MaxFiles MinFileSz MaxFileSz
fillfs 1 4 4 4096 4096
btrfs filesystem defragment $1 || _fail "Defrag failed with $?"

That's 2 lines of code for the test, and zero indirection.

258.out line 1

Quoted 7 lines

258.out new file +1 -0

new file mode 100644
index 0000000..6ccf5cf
@@ -0,0 +1,2 @@
+QA output created by 258

+silence is golden

.....

Quoted 1 line

258.out new file +1 -0
+# btrfs vol tests

.....

258.out

Quoted 9 lines

258.out new file +9 -0

+#FS fill size
+#name="DirDepth MinFiles MaxFiles MinFileSz MaxFileSz"
+QUICK="1 4 4 4096 4096"
+SMALL="1 10 100 4096 8192"
+MEDIUM="10 30 1000 4096 8192"
+LARGE="1000 1 1000 4096 1048576"
+FDPROF=$QUICK
+
+FSDEV="$SCRATCH_DEV $DISK_POOL"

Why?

258.out

Quoted 7 lines

258.out new file +7 -0

+if_error()
+{
+        if [[ $? -ne 0 ]]; then
+                echo "$1"
+                exit
+        fi
+}

Seriously?

258.out

Quoted 5 lines

258.out new file +5 -0

+fun_error()
+{
+        echo $1
+        exit
+}

dead code

258.out

Quoted 3 lines

258.out new file +3 -0
+test1_default()
+{
+	local i

dead.

258.out

Quoted 1 line

258.out new file +1 -0
+	mk_fs $1 "$2"

Oh, no, you didn't implement your own mkfs function as well?

Please have a look at common.rc and the scratch_mkfs functions and how that supports different filesystems. Add your multiple device mkfs operations to the btrfs part of that code.

Oh, and to mount it, use 'scratch_mount' and you don't need to pass the mountpoint to every function...

[....]

258.out

Quoted 5 lines

258.out new file +5 -0

+test6_add()
+{
+	local i
+	local devs[]="( $2 )"
+	local n=${#devs[@]}

ugh.

258.out

Quoted 4 lines

258.out new file +4 -0

+
+	n=$(($n-1))
+
+	_trace_wipe "$2"

What the does this do? it appears to dd zeros over the first 400k of the devices passed to it. Why? And if it's needed for mkfs.btrfs to work successfully, then why wasn't this done in your mk_fs function? What bug in btrfs is this working around?

258.out

Quoted 7 lines

258.out new file +7 -0

+	mk_fs $1 "${devs[0]}"
+	fillfs $1 $FDPROF
+	for i in `seq 1 $n`
+	do
+		# echo i=$i ${devs[$i]} $1
+		btrfs device add ${devs[$i]} $1 2>&1 > /dev/null
+	done

You do all that nasty array stuff, when all you need to is this:

	while [ $# -gt 0 ]; do
		btrfs device add $1 $SCRATCH_MNT 2>&1 > /dev/null
		shift;
	done

(assuming you aren't passing the mount point as $1 anymore)

258.out

Quoted 8 lines

258.out new file +8 -0

+	btrfs filesystem balance $1
+	btrfs filesystem show 2>&1 | egrep devid |awk '{
+		if ( $6 == "0.00" ) {
+			print "Error: Device "$8" balance failed"
+			exit
+		}
+	}'
+	if_error "Error: test6_add... failed"

	[ $? -eq 0 ] || _fail "test6_add failed with $?"

258.out

Quoted 13 lines

258.out new file +13 -0

+	echo "test6_add ok"
+}
+
+test7_replace()
+{
+	local i
+	local x
+	local devs=( $2 )
+	local n=${#devs[@]}
+	local ds
+	local DEVHTL=""
+
+	if [ $n -lt 3 ]; then echo "Error: Need atleast 3 physical disks for the vol:test7_replace"; exit; fi

This test should not be here - it should in the main test setup section and use '_not_run "Requires at least 3 scratch devices"'. That is, not having 3 devices is not a failure, it's a reason not to run the test.

Indeed, because none of the other tests have this requirement, you shoul dput this into it's own test so that the other tests here can still be run even if this is not met.

258.out

Quoted 2 lines

258.out new file +2 -0
+	n=$(($n-1))
+	ds=${devs[@]:0:$n}

Ah, what?

I can't work out what this....

258.out

Quoted 6 lines

258.out new file +6 -0

+	_trace_wipe "$2"
+	mk_fs $1 "$ds"
+	fillfs $1 $FDPROF
+
+	#fail a disk
+	ds=${devs[@]:$(($n-1)):1}

... and this are doing even after spending 5 minutes reading the bash man page.

Seriously, if you're going to write code that looks like line noise, comment it so people who aren't bash experts can understand what your intent is here. I'll have to refer to the bash man page every time i look at this code, so I really can't tell if it's doing what you meant or whether it's broken.

Write dumb code - it's far easier to review and maintain....

Quoted 22 lines

258.out new file +22 -0

+	devmgt remove ${ds}
+
+	btrfs fi show | egrep "Some devices missing" > /dev/null 2>&1
+	if_error "Error: btrfs did not report device missing"
+	# if [ ! $? ]; then echo "Error: btrfs did not report device missing"; exit; fi
+
+	# add a new disk to btrfs
+	ds=${devs[@]:$(($n)):1}
+	btrfs device add ${ds} ${1}
+	btrfs fi balance $1
+	btrfs filesystem show 2>&1 | egrep devid |awk '{
+		if ( $6 == "0.00" ) {
+			print "Error: Device "$8" balance failed"
+			exit
+		}
+	}'
+	if_error "Error: test7_repalce... failed"
+	echo "test7_replace ok"
+
+	# cleaup. add the removed disk
+	umount $1 > /dev/null 2>&1
+	devmgt add "${DEVHTL}"

258.out

Quoted 10 lines

258.out new file +10 -0

+}
+
+function test8_remove
+{
+	local res="ok"
+	_trace_wipe "$2"
+	mk_fs $1 "$2"
+	fillfs $1 $FDPROF
+
+	dev_del=`echo ${2} | awk '{print $NF}'`

Why use a different method for calulating the number of devices to test 7? This version is far easier to understand, though I still don't know if it is correct because there's nothing that describes what the test is supposed tobe doing....

258.out

Quoted 9 lines

258.out new file +9 -0

+	btrfs device delete $dev_del $1 || if_error "Error: delete failed"
+
+	btrfs fi balance $1
+	btrfs filesystem show 2>&1 | egrep devid |awk '{
+		if ( $6 == "0.00" ) {
+			print "Error: Device "$8" balance failed"
+			exit
+		}
+	} '

	btrfs filesystem show 2>&1 | awk '/devid/ {

258.out

Quoted 3 lines

258.out new file +3 -0

+	if_error "test8_remove... failed"
+	echo "test8_remove ok"
+	umount $1 > /dev/null 2>&1

scratch_umount

258.out

Quoted 1 line

258.out new file +1 -0
+	_trace_wipe "$2"

why?

Quoted 35 lines

258.out new file +13 -0

+}
+
+test1_default $SCRATCH_MNT "$FSDEV"
+test2_raid0 $SCRATCH_MNT "$FSDEV"
+test3_raid1 $SCRATCH_MNT "$FSDEV"
+test4_raid10 $SCRATCH_MNT "$FSDEV"
+test5_single $SCRATCH_MNT "$FSDEV"
+test6_add $SCRATCH_MNT "$FSDEV"
+test7_replace $SCRATCH_MNT "$FSDEV"
+test8_remove $SCRATCH_MNT "$FSDEV"
+
+status=0
+exit

259.out new file +9 -0

new file mode 100644
index 0000000..b3bd62a
@@ -0,0 +1,9 @@
+QA output created by 259
+test1_default ok
+test2_raid0 ok
+test3_raid1 ok
+test4_raid10 ok
+test5_single ok
+test6_add ok
+test7_replace ok
+test8_remove ok

common.rc +0 -0

index cb23a02..a354f69 100644
@@ -1559,7 +1559,65 @@ _test_inode_extsz()
     echo $blocks
 }

common.rc _test_inode_extsz() · line 1570

Quoted 10 lines

common.rc +9 -0

 
+_require_disk_pool()
+{
+	local i
+	case "$FSTYP" in
+	btrfs)
+		if [ -z "$DISK_POOL" ]
+		then
+			_notrun "this test requires a valid \$DISK_POOL"
+		fi

The usual format is:

		if [ -z "$DISK_POOL" ]; then
			.....
		fi

common.rc _test_inode_extsz() · line 1577

Quoted 7 lines

common.rc +7 -0

+		if [ "`echo $DISK_POOL|wc -w`" -lt 2 ]
+		then
+			_notrun "this test needs more than 1 disk in DISK_POOL"
+		fi
+
+		for i in $DISK_POOL
+		do

Same: for i in $DISK_POOL; do

Quoted 11 lines

common.rc +11 -0

+			if [ "`_is_block_dev $i`" = "" ]
+			then
+				_notrun "this test requires valid block disk $i"
+			fi
+			if [ "`_is_block_dev $i`" = "`_is_block_dev $TEST_DEV`" ]
+			then
+				_notrun "$i is part of TEST_DEV, this test requires unique disks"
+			fi
+			if [ "`_is_block_dev $i`" = "`_is_block_dev $SCRATCH_DEV`" ]
+			then
+				_notrun "$i is part of SCRATCH_DEV, this test requires unique disks"

common.rc _test_inode_extsz() · line 1598

Quoted 10 lines

common.rc +10 -0

+			fi
+			if _mount | grep -q $i
+			then
+				if ! $UMOUNT_PROG $i
+			        then
+			            echo "failed to unmount $i - aborting"
+			            exit 1
+			        fi
+			fi
+			dd if=/dev/zero of=$i bs=4096 count=100 > /dev/null 2>&1

Umm, why is this zeroing here? Once again, if this is necessary then it should be done only when necessary by the code that needs to do it (i.e. the mkfs or device add/remove commands).

common.rc _test_inode_extsz() · line 1602

Quoted 4 lines

common.rc +4 -0

+		done
+	;;
+	esac
+}

Needs to _notrun for all FSTYPs other than btrfs.

Quoted 7 lines

common.rc +7 -0

+_trace_wipe()
+{
+	local i
+	case "$FSTYP" in
+	btrfs)
+		for i in $1
+		do

common.rc _test_inode_extsz() · line 1620

Quoted 10 lines

common.rc +10 -0

+			dd if=/dev/zero of=$i bs=4096 count=100 > /dev/null 2>&1
+			if [ $? != 0 ]
+			then
+				echo "disk $i access failed"
+				exit 1
+			fi
+		done
+	;;
+	esac
+}

This needs to be folded into whatever btrfs mkfs code that needs it.

Quoted 38 lines

common.rc +0 -0

 ################################################################################
 
 if [ "$iam" != new -a "$iam" != bench ]

group +3 -0

index 0c746c8..62eebfb 100644
@@ -370,3 +370,6 @@ deprecated
 254 auto quick
 255 auto quick prealloc
 256 auto quick
+257 auto quick
+258 auto quick
+259 auto quick

src/checksum_misc new file +18 -0

new file mode 100644
index 0000000..42488cb
@@ -0,0 +1,28 @@
+#! /bin/bash
+# To Create the sha256 sum for the arg1
+# arg1 FS to generate sha256
+# arg2 File name to save the sha256 output
+function save_checksum()
+{
+	local i=0
+	>$2
+	cd $1
+	#echo "$1"
+	for i in `find . -type f`; do sha256sum $i >> $2; done
+	cd $OLDPWD
+}
+
+
+
+# To check the sha256 for the TESTFS
+# arg1 FS to be tested

src/checksum_misc line 28

Quoted 10 lines

src/checksum_misc new file +10 -0

+# arg2 sha256 file
+function verify_checksum()
+{
+	cd $1
+	if [ ! -f $2 ]; then echo "$2 file not found"; status=1;exit; fi
+	#$SHA256SUM --status -c $2 | $GREP "FAILED"
+	sha256sum -c $2 | grep "FAILED"
+	if [ $? == 0 ]; then status=1;exit; fi
+	cd $OLDPWD
+}

This checksum stuff only appears to be used by a single test, so put it in that test.

Further, it you use files of a known contents (rather that reading from /dev/urandom), you only need to dump the checksum into the golden output and you don't need a verify step at all as the golden image check at the end of the test will do it for you. (i.e. if the checksum is different to that in the golden output, the test will fail.) See 242 and 252 for examples of this style of checksum verification.

Quoted 9 lines

src/devmgt new file +3 -0

new file mode 100644
index 0000000..d86b5c9
@@ -0,0 +1,25 @@
+#! /bin/bash
+# arg 1 remove/add
+# arg 2 /dev/sdx or return of devmgt resply

src/devmgt line 13

Quoted 10 lines

src/devmgt new file +10 -0

+function devmgt
+{
+	local x
+	local d
+
+	if [ ${1} == "remove" ]; then
+		d=`echo $2|cut -d"/" -f3`
+		x=`ls -l /sys/class/block/${d} | cut -d "/" -f12 | sed 's/:/ /g'`
+		#log_event log "Removing the disk ${d} ${x} ..."
+		echo "scsi remove-single-device ${x}" > /proc/scsi/scsi

I don't have a /proc/scsi on any of my machines, even those with SAS disks. So tests that use this are going to need a test for having support for this API and _not_run if it doesn't exist.

src/devmgt line 23

Quoted 10 lines

src/devmgt new file +10 -0

+		if [ ! $? ]; then echo "Error: Remove a disk failed"; exit 1; fi
+		DEVHTL=${x}
+		#log_event done
+		return
+	else
+		#log_event log "cleaning.. adding back the removed disk ${2} .. "
+		echo "scsi add-single-device ${2}" > /proc/scsi/scsi
+		if [ ! $? ]; then echo "Error: Remove a disk failed"; exit 1; fi
+		#log_event done
+		return

I was going to ask what device this operated on, but I see that the global DEVHTL is passed back into the add function. needs documentation that the device needs to be stored by the caller....

src/devmgt line 25

Quoted 2 lines

src/devmgt new file +2 -0
+	fi
+}

As it is, this function is only by a single test - you should move it into that test given the _notrun check requirement. And it probably shoul dbe two functions - one for remove and one for add...

Quoted 34 lines

src/fsmisc new file +28 -0

new file mode 100644
index 0000000..2ee942d
@@ -0,0 +1,247 @@
+#! /bin/bash
+# Create Dir tree and files in it.
+# arg1 basedir
+# arg2 dir depth
+# arg3 nfile_min
+# arg4 nfile_max
+# arg5 fsize_min
+# arg6 fsize_max
+
+function fillfs()
+{
+	umask 000
+	local j
+	local i
+	local DIRP
+	local FCNT
+	local FILEP
+	local SCNT
+	local BCNT
+	#log_event log "filling $1 with dir=$2 files=$3 to $4 of size=$5 to $6....."
+	DIRP=$1
+	for ((j=0; j<$2; j++))
+	do
+		DIRP=`mktemp -dq $DIRP/dir.XXXXXX`
+		FCNT=$(r_ranged $3 $4)
+		for ((i=0; i<$FCNT; i++))
+		do
+			FILEP=`mktemp -q $DIRP/file.XXXXXX`

src/fsmisc line 38

Quoted 10 lines

src/fsmisc new file +10 -0

+			SCNT=$(r_ranged $5 $6)
+			dd if=/dev/urandom of=$FILEP bs=$SCNT count=1 status=noxfer 2>/dev/null &
+			# if some kernel may not support non aligned blocks well
+			#BCNT=$(($SCNT/4096))
+			#dd if=/dev/urandom of=$FILEP bs=4096 count=$BCNT status=noxfer 2>/dev/null &
+		done
+	done
+	wait $!
+	#log_event done
+}

As mentioned previously, this give unrepeatable fill patterns because it is random. While that might be good for testing some corner cases, it's bad for reproducing problems when they occur. This is the reason when se a known seed for things that generate random operations (like fsx) so when a problem is found we can reproduce the workload -exactly-.

But given the small test set sizes (even for large), the randomness is not really buying you anything that a fixed configuration or two would not test. Hence I'd drop the randomness altogether and use known patterns and filenames instead to make the test code simpler and allows you to put the checksums directly into the golden output for simple verification.

Remember, once a test is integrated into xfstests it is no longer a standalone test, so having lots of different configurations that are never used just makes it harder to understand and maintain. Indeed, if all the randomness goes away, all you need to do is make sure _populate_fs writes patterned files rather than zeroes. e.g. uses something like:

	$XFS_IO_PROG -f -c "pwrite -S 0xa55ac33c 0 $fsize" $outfile

instead of dd if=/dev/zero....

FWIW, that also gets around the performance problem of read lots of data from /dev/urandom - I get about 10MB/s throughput from /dev/urandom, which means writing large files will take quite some time - far slower than disk performance....

Quoted 6 lines

src/fsmisc new file +6 -0

+function modifyfs_fillblk()
+{
+	local FSIZE
+	local BLKS
+	local NBLK
+	local FALLOC

src/fsmisc line 78

Quoted 10 lines

src/fsmisc new file +10 -0

+	local WS
+
+	#log_event log "filling the allocated blocks....."
+	for i in `find /$1 -type f`
+	do
+		FSIZE=`stat -t $i | cut -d" " -f2`
+		BLKS=`stat -c "%B" $i`
+		NBLK=`stat -c "%b" $i`
+		FALLOC=$(($BLKS * $NBLK))
+		WS=$(($FALLOC - $FSIZE))

WS is unused, and given that you know the files aren't sparse, why wouldn't you simple overwrite from offset 0 to $FSIZE?

src/fsmisc line 80

Quoted 2 lines

src/fsmisc new file +2 -0

+		#echo $FSIZE $BLKS $NBLK $FALLOC $WS
+		dd if=/dev/urandom of=$i obs=$FALLOC count=1 status=noxfer 2>/dev/null &

		xfs_io -f -c "pwrite -S 0xbeefbabe 0 $FSIZE" $f

Quoted 11 lines

src/fsmisc new file +11 -0

+	done
+	wait $!
+	#log_event done
+}
+
+
+# Append a random size to the files
+# arg1 : FS in question
+
+function modifyfs_append()
+{

src/fsmisc line 101

Quoted 10 lines

src/fsmisc new file +10 -0

+	local FSIZE
+	local X
+	local N
+	local i
+	#log_event log "appending the files......"
+	N=0
+	for i in `find $1 -type f`
+	do
+		if [ $N == 0 ]; then
+			N=$(($N+1))

You do this just to execute this on the first loop? Just put it outside the loop and drop N altogether

src/fsmisc line 104

Quoted 3 lines

src/fsmisc new file +3 -0

+			X=$i
+			FSIZE=`stat -t $X | cut -d" " -f2`
+			dd if=$X of=$X seek=1 bs=$FSIZE obs=$FSIZE count=1 status=noxfer 2>/dev/null &

so it doubles the size of the file.

	xfs_io -a -f "pwrite -S 0xa55ac33c" $FSIZE $FSIZE" $i

src/fsmisc line 108

Quoted 4 lines

src/fsmisc new file +4 -0

+			continue
+		fi
+		FSIZE=`stat -t $i | cut -d" " -f2`
+		dd if=$X of=$i seek=1 bs=$FSIZE obs=$FSIZE count=1 status=noxfer 2>/dev/null &

and that copies from the start of the previous file to the end of the current file. IOWs, if we are using a fixed pattern, it's the same operation as the N=0 case, without the risk that the source file is too short and we end up with a short read. It also means this ends up as:

	for f in `find $1 -type f`; do
		FSIZE=`stat -t $i | cut -d" " -f2`
		xfs_io -a -f "pwrite -S 0xa55ac33c" $FSIZE $FSIZE" $i
	done;

.....

Quoted 23 lines

src/fsmisc new file +23 -0

+# To have a clean FS to test
+# arg 1: mount point
+# arg 2: metadata Raid Type/devs
+# arg 3: data Raid Type/none
+# arg 4: devices/none
+#or
+# arg 1: mount point
+# arg 2: devices
+function mk_fs()
+{
+	local ndev
+
+	umount $1 > /dev/null 2>&1
+
+	if [ $# == 2 ]; then
+		# use default options to create the FS.
+		ndev=( $2 )
+		#echo ${ndev[@]}
+		mkfs.btrfs ${ndev[@]} > /dev/null 2>&1
+		#mkfs.btrfs ${ndev[@]} 2>&1 | _filter_scratch
+		if [ $? != 0 ]; then echo mkfs failed ; exit; fi
+	else if [ $# == 4 ]; then
+		ndev=( $4 )

src/fsmisc line 162

Quoted 10 lines

src/fsmisc new file +10 -0

+		mkfs.btrfs -m $2 -d $3 ${ndev[@]} > /dev/null 2>&1
+		#mkfs.btrfs -m $2 -d $3 ${ndev[@]} 2>&1 | _filter_scratch
+		if [ $? != 0 ]; then echo mkfs failed; exit; fi
+	else
+		echo "Coding Error 234: xfstests alias would like to hear about it"
+	fi
+	fi
+
+	mount -t btrfs ${ndev[0]} $1
+}

See my comments about folding this into btrfs specific branches of scratch_mkfs.

Quoted 9 lines

src/fsmisc new file +9 -0

+# arg1: mntpoint
+# arg2: Number of devs to use
+# arg3: devs in array
+function mk_fs_loopdev()
+{
+	local ndev
+
+	#log_event log "creating a new btrfs fs on $3...."
+	umount $1 > /dev/null 2>&1

src/fsmisc line 182

Quoted 10 lines

src/fsmisc new file +10 -0

+
+	ndev=( $3 )
+	#mkfs.btrfs -m $2 -d $3 ${ndev[@]} > ./tmp/out
+	#if [ $? != 0 ]; then echo mkfs failed ; cat ./tmp/out; exit 1; fi
+	#mount -t btrfs ${ndev[0]} $1
+
+#	echo
+#	echo $# ndev=${ndev[@]}
+	#log_event done
+}

Unused, does nothing. Please remove.

src/fsmisc line 188

Quoted 6 lines

src/fsmisc new file +6 -0

+
+
+function leaned_fs
+{
+	echo $1 $2
+}

Unused, please remove.

Quoted 22 lines

src/fsmisc new file +22 -0

+#arg 1: mntpt
+#arg 2: phy disks
+#arg 3: Number of total loop devs
+#arg 4: initial mkfs devs
+
+function fs_on_loopdev()
+{
+	local sz
+	local n=0
+	local devname
+	local devs
+
+	mk_fs $1 "$2"
+
+	sz=$((`df -k $1 | egrep "$1" | awk '{print $4}'`/$(($3+1))))
+	sz=$(($sz*1024))
+	if [ $sz -lt 2671771648 ]; then echo "Error: Need at least 2G space for this test"; exit 1; fi
+
+	for n in $(seq 1 $3)
+	do
+		devname=`mktemp -q $1/dev.XXXXXX`
+		dd if=/dev/zero of=$devname bs=8192 count=64000 status=noxfer 2>/dev/null

src/fsmisc line 221

Quoted 10 lines

src/fsmisc new file +10 -0

+		losetup -f $devname
+		if [ $? != 0 ]; then echo "Error losetup failed";exit 1;fi
+		LOOPDEV[$(($n-1))]=`losetup -a | grep $devname|cut -d":" -f1`
+		LOOPFILE[$(($n-1))]=$devname
+	done
+
+	mkdir -p $1/mnt
+	devs=${LOOPDEV[@]:0:$4}
+	mk_fs $1/mnt "$devs"
+}

Unused, please remove.

Quoted 16 lines

src/fsmisc new file +16 -0

+
+#arg 1: mntpt
+function clean_loopdev()
+{
+	local i
+
+	#log_event log "Cleaning loop devices....."
+
+	local n=${#LOOPDEV[@]}
+	if [ ! $n ]; then return; fi
+
+	n=$(($n-1))
+
+	umount ${LOOPDEV[0]}
+	if [ $? != 0 ]; then echo "Error: umount loopdev failed"; exit 1;fi
+

src/fsmisc line 247

Quoted 10 lines

src/fsmisc new file +10 -0

+	for i in `seq 0 $n`
+	do
+		losetup -d ${LOOPDEV[$i]}
+		unlink ${LOOPFILE[$i]}
+		#echo ${LOOPDEVS[$i]}
+		#ls -l $MNTPT/fd$i
+	done
+
+	#log_event done
+}

Also unused.

IOWs, this file can pretty much go away.

Quoted 14 lines

src/randmisc new file +8 -0

new file mode 100644
index 0000000..a0a065b
@@ -0,0 +1,70 @@
+#! /bin/bash
+
+# Generate Random number in a range
+# arg1 min
+# arg2 max
+
+function r_ranged()
+{

src/randmisc line 18

Quoted 10 lines

src/randmisc new file +10 -0

+	local X
+	local Y
+	local S
+	if [ $2 == 0 ]; then echo 0; return; fi
+	Y=$RANDOM
+	((X = $2 - $1 + 1))
+	((X = $Y % $X))
+	((X = $X + $1))
+	echo $X
+}

only used by the fillfs function and the picka function below. Given that fillfs isn't actually random with the proposed configuration (and is better off without the randomness), and picka is used only once, this can move to whereever the picka function goes...

Quoted 42 lines

src/randmisc new file +42 -0

+
+# Picks a rand(r)/first(f)/last(l) file or subvol
+# arg1: what
+# arg2: FS
+function picka()
+{
+	local MAX
+	local X
+	local R
+	local i
+
+	case $1 in
+	rfile)
+		MAX=`find $2 -type f | wc -l`
+		X=`r_ranged 1 $MAX`
+		R=0
+		for i in `find $2 -type f`; do
+			if [ "$X" == "$R" ]; then echo $i; return; fi
+			R=$(($R+1))
+		done
+	;;
+	rsnap)
+		MAX=`$BTRFS subvolume list $2 | wc -l`
+		X=`r_ranged 1 $MAX`
+		R=0
+		for i in `$BTRFS subvolume list $2`; do
+			if [ $X == $R ]; then echo $i; return; fi
+			R=$(($R+1))
+		done
+	;;
+	ffile)
+		for i in `find $2 -type f`; do
+			echo $i
+			break
+		done
+	;;
+	fsnap)
+		for i in `$BTRFS subvolume list $2`; do
+			echo $i;
+			break
+		done
+	;;

src/randmisc line 70

Quoted 10 lines

src/randmisc new file +10 -0

+	lfile)
+		for i in `find $2 -type f`; do
+			sleep 1
+		done
+		echo $i
+	;;
+	lsnap)
+	;;
+	esac
+}

picka is used only once, for the rfile case. Move the function into that test, remove all the unused code. can probably even just be open coded in that test...

Quoted 11 lines

src/snapmisc new file +5 -0

new file mode 100644
index 0000000..7bd059f
@@ -0,0 +1,73 @@
+#! /bin/bash
+# Create a snapshot
+# arg1 source
+# arg2 dest dir
+

src/snapmisc line 15

Quoted 10 lines

src/snapmisc new file +10 -0

+# Return snapshot name in the SNAPNAME
+function create_snap()
+{
+	local x
+	if [ ! -d $2 ]; then echo Destination dir $2 not present; fi
+	SNAPNAME=`mktemp -u $2/snap.XXXXXX`
+	btrfs subvolume snapshot $1 $SNAPNAME > /dev/null
+	if [ $? != 0 ]; then echo Error snapshot create failed; status=1;exit; fi
+	return
+}

I'm not sure this really needs a wrapper, especially as $2 is always $SCRATCHMNT and will be present. i.e, it only needs to be:

	btrfs subvolume snapshot $1 `mktemp -u $2/snap.XXXXXX` \
		> /dev/null || _fail "snapshot create failed with $?"

src/snapmisc line 25

Quoted 10 lines

src/snapmisc new file +10 -0

+
+# Destroy a snapshot
+# arg 1: snapshot to be deleted
+
+function destroy_snap()
+{
+	#log_event log "deleting snapshot $1....."
+	btrfs subvolume delete $1
+	#log_event done done
+}

That certainly doesn't need a wrapper.

Quoted 4 lines

src/snapmisc new file +4 -0

+
+# Creates n clones
+# arg 1: number of clones required
+# arg 2: soruce of which clone has to be taken

src/snapmisc line 39

Quoted 10 lines

src/snapmisc new file +10 -0

+
+function create_nclones()
+{
+	local i
+
+	for i in `seq 1 $1`
+	do
+		create_snap $2 $SCRATCH_MNT
+	done
+}

unused, please remove.

src/snapmisc line 46

Quoted 7 lines

src/snapmisc new file +7 -0

+
+function update_clonelist()
+{
+	local i
+	# Is there a way btrfs can distinguish sv and ss ?
+	local n=0
+	for i in `btrfs subvolume list $SCRATCH_MNT | rev|cut -d" " -f1|rev`

Why do you reverse the stream, cut it and then reverse it again?

src/snapmisc line 53

Quoted 7 lines

src/snapmisc new file +7 -0

+	do
+		#echo i=$i n=$n
+		CLONE_LIST[$n]="${SCRATCH_MNT}/${i}"
+                n=$((n+1))
+        done
+	#echo n=$n
+	#for i in `seq 0 $((n-1))`; do echo i=$i ${CLONE_LIST[$i]}; done

kill all the debug.

Quoted 6 lines

src/snapmisc new file +6 -0

+}
+
+# arg1: number of clones required
+# arg2: base subvol
+function create_nestedclones()
+{

src/snapmisc line 69

Quoted 10 lines

src/snapmisc new file +10 -0

+	local i
+	local x
+	local w
+	local SNAPNAME=0
+
+	x=$2
+	for i in `seq 1 $1`
+	do
+		create_snap $x $SCRATCH_MNT
+		x=$SNAPNAME

If you open code creat_snap, this global variable abuse goes away.

src/snapmisc line 73

Quoted 4 lines

src/snapmisc new file +4 -0

+		#w=`btrfs subvolume list $SCRATCH_MNT|wc -l`
+		#echo i=$i x=$x w=$w
+	done
+}

Both create_nestedclones() and update_clonelist() are use donly once, and one is called directly after the other. Just open code it in the test - much simpler and easier to understand...

Cheers,

Dave.

Signature · 2 lines

Dave Chinner
[email protected]

Anand Jain

author

2011-08-11 19:52 UTC #5 lore raw

Dave,

Thanks for the great review comments, I have accepted almost all comments and sent out a new set of patches. Kindly review further, let me know.

- Anand

Anand Jain

author

2011-08-11 19:52 UTC #6 lore raw

PATCH 1/3 Added test case 257 for btrfs extended snapshot tests

This adds the test case 257 to test the snapshot feature in:wq btrfs. This also added a new user set variable SCRATCH_DEV_POOL, which should be set to dev pool for btrfs.

Signed-off-by Anand Jain <[email protected]>

---
 257       |  192 
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 257.out   |    2 +
 common.rc |   86 +++++++++++++++++++++++++++
 group     |    1 +
 4 files changed, 281 insertions(+), 0 deletions(-)
 create mode 100755 257
 create mode 100644 257.out

257 new file +192 -0

new file mode 100755
index 0000000..9d7c8ff
@@ -0,0 +1,192 @@
+#!/bin/bash
+# FS QA Test No. 257
+#
+# Extented btrfs snapshot test cases
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2011 Oracle  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#
+#-----------------------------------------------------------------------
+#
+# creator
[email protected]
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+
+_cleanup()
+{
+    rm -f $tmp.*
+}
+
+trap "_cleanup ; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_need_to_be_root
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+
+_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs failed"
+_scratch_mount
+
+# Create and save sha256sum
+# arg1 FS to generate sha256
+# arg2 File name to save the sha256 output
+_save_checksum()
+{
+	local i=0
+	>$2
+	cd $1
+	for i in `find . -type f`; do sha256sum $i >> $2; done
+	cd $OLDPWD
+}
+
+# Verify the sha256sum for a FS
+# arg1 FS to be tested
+# arg2 sha256 file
+_verify_checksum()
+{
+	cd $1
+	[ -f $2 ] || _fail "checksum file $2 not found"
+	sha256sum -c $2 | grep "FAILED"
+	cd $OLDPWD
+}
+
+# Create a snapshot
+# arg1 dest dir
+# Return snapshot name in the SNAPNAME
+_create_snap()
+{
+	local x
+	[ -d $1 ] || _fail "Destination dir $1 not present"
+	SNAPNAME=`mktemp -u $SCRATCH_MNT/snap.XXXXXX`
+	btrfs subvolume snapshot $1 $SNAPNAME > /dev/null || _fail "snapshot create failed"
+}
+
+# Reads and writes new data but does not allocate new blocks
+# arg1 FS to be modified
+_read_modify_write()
+{
+	local i
+	local FSIZE
+	for i in `find $1 -type f`
+	do
+		FSIZE=`stat -t $i | cut -d" " -f2`
+		dd if=$i of=/dev/null obs=$FSIZE count=1 status=noxfer 2>/dev/null &
+		dd if=/dev/urandom of=$i obs=$FSIZE count=1 status=noxfer 2>/dev/null &
+	done
+	wait $!
+}
+
+# Fills the allocated blocks
+# arg1 FS in question
+_fill_blk()
+{
+	local FSIZE
+	local BLKS
+	local NBLK
+	local FALLOC
+	local WS
+
+	for i in `find /$1 -type f`
+	do
+		FSIZE=`stat -t $i | cut -d" " -f2`
+		BLKS=`stat -c "%B" $i`
+		NBLK=`stat -c "%b" $i`
+		FALLOC=$(($BLKS * $NBLK))
+		WS=$(($FALLOC - $FSIZE))
+		dd if=/dev/urandom of=$i oseek=$FSIZE obs=$WS count=1 status=noxfer 2>/dev/null &
+	done
+	wait $!
+}
+
+
+# Append a random size to the files
+# arg1 : FS in question
+_append_file()
+{
+	local FSIZE
+	local X
+	local N
+	local i
+	N=0
+	for i in `find $1 -type f`
+	do
+		if [ $N == 0 ]; then
+			X=$i
+			FSIZE=`stat -t $X | cut -d" " -f2`
+			dd if=$X of=$X seek=1 bs=$FSIZE obs=$FSIZE count=1 status=noxfer 2>/dev/null &
+			N=$(($N+1))
+			continue
+		fi
+		FSIZE=`stat -t $i | cut -d" " -f2`
+		dd if=$X of=$i seek=1 bs=$FSIZE obs=$FSIZE count=1 status=noxfer 2>/dev/null &
+		X=$i
+	done
+	wait $!
+}
+
+##################### real QA test starts here###################################
+# sv1 - is just a name nothing spl
+firstvol="$SCRATCH_MNT/sv1"
+btrfs subvolume create $firstvol > /dev/null || _fail "btrfs subvolume create $firstvol failed"
+_fillfs 1 10 100 4096 8192 $firstvol
+SNAPNAME=0
+_create_snap $firstvol
+_save_checksum $firstvol $tmp.sv1.sum
+_verify_checksum $SNAPNAME $tmp.sv1.sum
+
+#Append1 the files
+_fill_blk $SNAPNAME
+_verify_checksum $firstvol $tmp.sv1.sum
+
+#Append2 the files
+_append_file $SNAPNAME
+_verify_checksum $firstvol $tmp.sv1.sum
+
+#read modify write
+_read_modify_write $SNAPNAME
+_verify_checksum $firstvol $tmp.sv1.sum
+
+#nested snapshot test
+src_vol=$firstvol
+for i in `seq 1 7`; do
+	SNAPNAME=0
+	_create_snap $src_vol
+	_verify_checksum $SNAPNAME $tmp.sv1.sum
+	src_vol=$SNAPNAME
+done
+
+# file delete test
+SNAPNAME=0
+_create_snap $firstvol
+tname=`echo $SNAPNAME | rev | cut -d"/" -f1 | rev`
+_save_checksum $SNAPNAME $tmp.$tname.sum
+\rm -rf $firstvol/*
+_verify_checksum $SNAPNAME $tmp.$tname.sum
+
+umount $SCRATCH_DEV || _fail "unmount failed"
+
+echo "Silence is golden"
+status=0; exit

257.out new file +2 -0

new file mode 100644
index 0000000..cc3693f
@@ -0,0 +1,2 @@
+QA output created by 257
+Silence is golden

common.rc +86 -0

91 lines not on this page · raw message

################################################################################

if [ "$iam" != new -a "$iam" != bench ]

group +1 -0

index 0c746c8..84c45da 100644
@@ -370,3 +370,4 @@ deprecated
 254 auto quick
 255 auto quick prealloc
 256 auto quick
+257 auto quick

Signature · 6 lines

1.7.1

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

Anand Jain

author

2011-08-15 07:44 UTC #7 lore raw

PATCH v2 1/3 Added test case 257 for btrfs extended snapshot tests

Added extended test cases for btrfs snapshot. This adds a new user-variable SCRATCH_DEV_POOL which should be set to a disk set.

Signed-off-by Anand Jain <[email protected]>

---
 257       |  192 
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 257.out   |    2 +
 common.rc |   86 +++++++++++++++++++++++++++
 group     |    1 +
 4 files changed, 281 insertions(+), 0 deletions(-)
 create mode 100755 257
 create mode 100644 257.out

257 new file +192 -0

new file mode 100755
index 0000000..9d7c8ff
@@ -0,0 +1,192 @@
+#!/bin/bash
+# FS QA Test No. 257
+#
+# Extented btrfs snapshot test cases
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2011 Oracle  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#
+#-----------------------------------------------------------------------
+#
+# creator
[email protected]
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+
+_cleanup()
+{
+    rm -f $tmp.*
+}
+
+trap "_cleanup ; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_need_to_be_root
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+
+_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs failed"
+_scratch_mount
+
+# Create and save sha256sum
+# arg1 FS to generate sha256
+# arg2 File name to save the sha256 output
+_save_checksum()
+{
+	local i=0
+	>$2
+	cd $1
+	for i in `find . -type f`; do sha256sum $i >> $2; done
+	cd $OLDPWD
+}
+
+# Verify the sha256sum for a FS
+# arg1 FS to be tested
+# arg2 sha256 file
+_verify_checksum()
+{
+	cd $1
+	[ -f $2 ] || _fail "checksum file $2 not found"
+	sha256sum -c $2 | grep "FAILED"
+	cd $OLDPWD
+}
+
+# Create a snapshot
+# arg1 dest dir
+# Return snapshot name in the SNAPNAME
+_create_snap()
+{
+	local x
+	[ -d $1 ] || _fail "Destination dir $1 not present"
+	SNAPNAME=`mktemp -u $SCRATCH_MNT/snap.XXXXXX`
+	btrfs subvolume snapshot $1 $SNAPNAME > /dev/null || _fail "snapshot create failed"
+}
+
+# Reads and writes new data but does not allocate new blocks
+# arg1 FS to be modified
+_read_modify_write()
+{
+	local i
+	local FSIZE
+	for i in `find $1 -type f`
+	do
+		FSIZE=`stat -t $i | cut -d" " -f2`
+		dd if=$i of=/dev/null obs=$FSIZE count=1 status=noxfer 2>/dev/null &
+		dd if=/dev/urandom of=$i obs=$FSIZE count=1 status=noxfer 2>/dev/null &
+	done
+	wait $!
+}
+
+# Fills the allocated blocks
+# arg1 FS in question
+_fill_blk()
+{
+	local FSIZE
+	local BLKS
+	local NBLK
+	local FALLOC
+	local WS
+
+	for i in `find /$1 -type f`
+	do
+		FSIZE=`stat -t $i | cut -d" " -f2`
+		BLKS=`stat -c "%B" $i`
+		NBLK=`stat -c "%b" $i`
+		FALLOC=$(($BLKS * $NBLK))
+		WS=$(($FALLOC - $FSIZE))
+		dd if=/dev/urandom of=$i oseek=$FSIZE obs=$WS count=1 status=noxfer 2>/dev/null &
+	done
+	wait $!
+}
+
+
+# Append a random size to the files
+# arg1 : FS in question
+_append_file()
+{
+	local FSIZE
+	local X
+	local N
+	local i
+	N=0
+	for i in `find $1 -type f`
+	do
+		if [ $N == 0 ]; then
+			X=$i
+			FSIZE=`stat -t $X | cut -d" " -f2`
+			dd if=$X of=$X seek=1 bs=$FSIZE obs=$FSIZE count=1 status=noxfer 2>/dev/null &
+			N=$(($N+1))
+			continue
+		fi
+		FSIZE=`stat -t $i | cut -d" " -f2`
+		dd if=$X of=$i seek=1 bs=$FSIZE obs=$FSIZE count=1 status=noxfer 2>/dev/null &
+		X=$i
+	done
+	wait $!
+}
+
+##################### real QA test starts here###################################
+# sv1 - is just a name nothing spl
+firstvol="$SCRATCH_MNT/sv1"
+btrfs subvolume create $firstvol > /dev/null || _fail "btrfs subvolume create $firstvol failed"
+_fillfs 1 10 100 4096 8192 $firstvol
+SNAPNAME=0
+_create_snap $firstvol
+_save_checksum $firstvol $tmp.sv1.sum
+_verify_checksum $SNAPNAME $tmp.sv1.sum
+
+#Append1 the files
+_fill_blk $SNAPNAME
+_verify_checksum $firstvol $tmp.sv1.sum
+
+#Append2 the files
+_append_file $SNAPNAME
+_verify_checksum $firstvol $tmp.sv1.sum
+
+#read modify write
+_read_modify_write $SNAPNAME
+_verify_checksum $firstvol $tmp.sv1.sum
+
+#nested snapshot test
+src_vol=$firstvol
+for i in `seq 1 7`; do
+	SNAPNAME=0
+	_create_snap $src_vol
+	_verify_checksum $SNAPNAME $tmp.sv1.sum
+	src_vol=$SNAPNAME
+done
+
+# file delete test
+SNAPNAME=0
+_create_snap $firstvol
+tname=`echo $SNAPNAME | rev | cut -d"/" -f1 | rev`
+_save_checksum $SNAPNAME $tmp.$tname.sum
+\rm -rf $firstvol/*
+_verify_checksum $SNAPNAME $tmp.$tname.sum
+
+umount $SCRATCH_DEV || _fail "unmount failed"
+
+echo "Silence is golden"
+status=0; exit

257.out new file +2 -0

new file mode 100644
index 0000000..cc3693f
@@ -0,0 +1,2 @@
+QA output created by 257
+Silence is golden

common.rc +86 -0

91 lines not on this page · raw message

################################################################################

if [ "$iam" != new -a "$iam" != bench ]

group +1 -0

index 0c746c8..84c45da 100644
@@ -370,3 +370,4 @@ deprecated
 254 auto quick
 255 auto quick prealloc
 256 auto quick
+257 auto quick

Signature · 1 line

1.7.1

Christoph Hellwig

2011-09-02 08:45 UTC #8 lore raw

Re: [PATCH 1/3] Added test case 257 for btrfs extended snapshot tests

Dave, Alex:

any objections to these patches? I not I'm going to put them in this weekend.

Quoted 343 lines · Anand Jain

This adds the test case 257 to test the snapshot feature in:wq btrfs. This also added a new user set variable SCRATCH_DEV_POOL, which should be set to dev pool for btrfs.

Signed-off-by Anand Jain <[email protected]>

---
 257       |  192
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 257.out   |    2 +
 common.rc |   86 +++++++++++++++++++++++++++
 group     |    1 +
 4 files changed, 281 insertions(+), 0 deletions(-)
 create mode 100755 257
 create mode 100644 257.out

257 new file +84 -0

new file mode 100755
index 0000000..9d7c8ff
@@ -0,0 +1,192 @@
+#!/bin/bash
+# FS QA Test No. 257
+#
+# Extented btrfs snapshot test cases
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2011 Oracle  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#
+#-----------------------------------------------------------------------
+#
+# creator
[email protected]
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+
+_cleanup()
+{
+    rm -f $tmp.*
+}
+
+trap "_cleanup ; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_need_to_be_root
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+
+_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs failed"
+_scratch_mount
+
+# Create and save sha256sum
+# arg1 FS to generate sha256
+# arg2 File name to save the sha256 output
+_save_checksum()
+{
+	local i=0
+	>$2
+	cd $1
+	for i in `find . -type f`; do sha256sum $i >> $2; done
+	cd $OLDPWD
+}
+
+# Verify the sha256sum for a FS
+# arg1 FS to be tested
+# arg2 sha256 file
+_verify_checksum()
+{
+	cd $1
+	[ -f $2 ] || _fail "checksum file $2 not found"
+	sha256sum -c $2 | grep "FAILED"
+	cd $OLDPWD
+}
+
+# Create a snapshot
+# arg1 dest dir
+# Return snapshot name in the SNAPNAME
+_create_snap()
+{
+	local x
+	[ -d $1 ] || _fail "Destination dir $1 not present"
+	SNAPNAME=`mktemp -u $SCRATCH_MNT/snap.XXXXXX`
+	btrfs subvolume snapshot $1 $SNAPNAME > /dev/null || _fail

"snapshot create failed"

257 new file +35 -0

+}
+
+# Reads and writes new data but does not allocate new blocks
+# arg1 FS to be modified
+_read_modify_write()
+{
+	local i
+	local FSIZE
+	for i in `find $1 -type f`
+	do
+		FSIZE=`stat -t $i | cut -d" " -f2`
+		dd if=$i of=/dev/null obs=$FSIZE count=1 status=noxfer 2>/dev/null &
+		dd if=/dev/urandom of=$i obs=$FSIZE count=1 status=noxfer 2>/dev/null &
+	done
+	wait $!
+}
+
+# Fills the allocated blocks
+# arg1 FS in question
+_fill_blk()
+{
+	local FSIZE
+	local BLKS
+	local NBLK
+	local FALLOC
+	local WS
+
+	for i in `find /$1 -type f`
+	do
+		FSIZE=`stat -t $i | cut -d" " -f2`
+		BLKS=`stat -c "%B" $i`
+		NBLK=`stat -c "%b" $i`
+		FALLOC=$(($BLKS * $NBLK))
+		WS=$(($FALLOC - $FSIZE))
+		dd if=/dev/urandom of=$i oseek=$FSIZE obs=$WS count=1

status=noxfer 2>/dev/null &

257 new file +20 -0

+	done
+	wait $!
+}
+
+
+# Append a random size to the files
+# arg1 : FS in question
+_append_file()
+{
+	local FSIZE
+	local X
+	local N
+	local i
+	N=0
+	for i in `find $1 -type f`
+	do
+		if [ $N == 0 ]; then
+			X=$i
+			FSIZE=`stat -t $X | cut -d" " -f2`
+			dd if=$X of=$X seek=1 bs=$FSIZE obs=$FSIZE count=1 status=noxfer

2>/dev/null &

257 new file +5 -0

+			N=$(($N+1))
+			continue
+		fi
+		FSIZE=`stat -t $i | cut -d" " -f2`
+		dd if=$X of=$i seek=1 bs=$FSIZE obs=$FSIZE count=1 status=noxfer

2>/dev/null &

257 new file +6 -0

+		X=$i
+	done
+	wait $!
+}
+
+##################### real QA test starts

here###################################

257 new file +3 -0

+# sv1 - is just a name nothing spl
+firstvol="$SCRATCH_MNT/sv1"
+btrfs subvolume create $firstvol > /dev/null || _fail "btrfs

subvolume create $firstvol failed"

257 new file +39 -0

+_fillfs 1 10 100 4096 8192 $firstvol
+SNAPNAME=0
+_create_snap $firstvol
+_save_checksum $firstvol $tmp.sv1.sum
+_verify_checksum $SNAPNAME $tmp.sv1.sum
+
+#Append1 the files
+_fill_blk $SNAPNAME
+_verify_checksum $firstvol $tmp.sv1.sum
+
+#Append2 the files
+_append_file $SNAPNAME
+_verify_checksum $firstvol $tmp.sv1.sum
+
+#read modify write
+_read_modify_write $SNAPNAME
+_verify_checksum $firstvol $tmp.sv1.sum
+
+#nested snapshot test
+src_vol=$firstvol
+for i in `seq 1 7`; do
+	SNAPNAME=0
+	_create_snap $src_vol
+	_verify_checksum $SNAPNAME $tmp.sv1.sum
+	src_vol=$SNAPNAME
+done
+
+# file delete test
+SNAPNAME=0
+_create_snap $firstvol
+tname=`echo $SNAPNAME | rev | cut -d"/" -f1 | rev`
+_save_checksum $SNAPNAME $tmp.$tname.sum
+\rm -rf $firstvol/*
+_verify_checksum $SNAPNAME $tmp.$tname.sum
+
+umount $SCRATCH_DEV || _fail "unmount failed"
+
+echo "Silence is golden"
+status=0; exit

257.out new file +2 -0

new file mode 100644
index 0000000..cc3693f
@@ -0,0 +1,2 @@
+QA output created by 257
+Silence is golden

common.rc +40 -0

index cb23a02..db7c2dd 100644
@@ -1559,7 +1559,93 @@ _test_inode_extsz()
     echo $blocks
 }
 
+# Generate Random number in a range
+# arg1 min, arg2 max
+_rand_range()
+{
+	local X
+	local Y
+	if [ $2 == 0 ]; then echo 0; return; fi
+	RANDOM=1
+	Y=$RANDOM
+	((X = $2 - $1 + 1))
+	((X = $Y % $X))
+	((X = $X + $1))
+	echo $X
+}
+
+# Create Dir tree and files in it.
+# arg1 basedir
+# arg2 dir depth
+# arg3 nfile_min
+# arg4 nfile_max
+# arg5 fsize_min
+# arg6 fsize_max
+_fillfs()
+{
+	umask 000
+	local j
+	local i
+	local DIRP
+	local FCNT
+	local FILEP
+	local SCNT
+	local BCNT
+	DIRP=$6
+	for ((j=0; j<$1; j++)); do
+		DIRP=`mktemp -dq $DIRP/dir.XXXXXX`
+		FCNT=$(_rand_range $2 $3)
+		for ((i=0; i<$FCNT; i++)); do
+			FILEP=`mktemp -q $DIRP/file.XXXXXX`
+			SCNT=$(_rand_range $4 $5)
+			dd if=/dev/urandom of=$FILEP bs=$SCNT count=1 status=noxfer

2>/dev/null &

common.rc +46 -0

+		done
+	done
+	wait $!
+}
+
+# scratch_dev_pool should contain the disks pool for the btrfs raid
+_require_scratch_dev_pool()
+{
+	local i
+	case "$FSTYP" in
+	btrfs)
+		if [ -z "$SCRATCH_DEV_POOL" ]
+		then
+			_notrun "this test requires a valid \$SCRATCH_DEV_POOL"
+		fi
+		if [ "`echo $SCRATCH_DEV_POOL|wc -w`" -lt 2 ]
+		then
+			_notrun "this test needs 2 or more disks in SCRATCH_DEV_POOL"
+		fi
 
+		for i in $SCRATCH_DEV_POOL
+		do
+			if [ "`_is_block_dev $i`" = "" ]
+			then
+				_notrun "this test requires valid block disk $i"
+			fi
+			if [ "`_is_block_dev $i`" = "`_is_block_dev $TEST_DEV`" ]
+			then
+				_notrun "$i is part of TEST_DEV, this test requires unique disks"
+			fi
+			if [ "`_is_block_dev $i`" = "`_is_block_dev $SCRATCH_DEV`" ]
+			then
+				_notrun "$i is part of SCRATCH_DEV, this test requires unique disks"
+			fi
+			if _mount | grep -q $i
+			then
+				if ! $UMOUNT_PROG $i
+			        then
+			            echo "failed to unmount $i - aborting"
+			            exit 1
+			        fi
+			fi
+			dd if=/dev/zero of=$i bs=4096 count=100 > /dev/null 2>&1
+		done
+	;;
+	esac
+}

################################################################################

if [ "$iam" != new -a "$iam" != bench ]

group +1 -0

index 0c746c8..84c45da 100644
@@ -370,3 +370,4 @@ deprecated
 254 auto quick
 255 auto quick prealloc
 256 auto quick
+257 auto quick

Signature · 6 lines

1.7.1

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

---end quoted text---

Anand Jain

author

2011-08-11 19:53 UTC #9 lore raw

PATCH 2/3 Added test case 258 for btrfs defragmentation

Signed-off-by Anand Jain <[email protected]>

---
 258     |   60 
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 258.out |    2 ++
 group   |    1 +
 3 files changed, 63 insertions(+), 0 deletions(-)
 create mode 100755 258
 create mode 100644 258.out

258 new file +60 -0

new file mode 100755
index 0000000..798bbaf
@@ -0,0 +1,60 @@
+#! /bin/bash
+# FS QA Test No. 258
+#
+# btrfs defragmentation tests
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2011 Oracle.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+# creator
[email protected]
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+    cd /
+    rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_need_to_be_root
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+
+_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs failed"
+_scratch_mount
+
+_fillfs 1 10 100 4096 8192 $SCRATCH_MNT
+
+btrfs filesystem defragment $SCRATCH_MNT
+[ $? = 20 ] || _fail "Defragment failed"
+
+umount $SCRATCH_MNT || _fail "umount failed"
+
+echo "Silence is golden"
+status=0; exit

258.out new file +2 -0

new file mode 100644
index 0000000..9d47016
@@ -0,0 +1,2 @@
+QA output created by 258
+Silence is golden

group +1 -0

index 84c45da..739f806 100644
@@ -371,3 +371,4 @@ deprecated
 255 auto quick prealloc
 256 auto quick
 257 auto quick
+258 auto quick

Signature · 6 lines

1.7.1

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

Anand Jain

author

2011-08-11 19:53 UTC #10 lore raw

PATCH 3/3 Added test case 259 for the btrfs raid features

Added test case 259 for the btrfs raid features. SCRATCH_DEV_POOL must be set to 2 or more disks.

Signed-off-by Anand Jain <[email protected]>

---
 259       |  186 
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 259.out   |    2 +
 common.rc |    6 ++
 group     |    1 +
 4 files changed, 195 insertions(+), 0 deletions(-)
 create mode 100755 259
 create mode 100644 259.out

259 new file +186 -0

new file mode 100755
index 0000000..b4ba403
@@ -0,0 +1,186 @@
+#! /bin/bash
+# FS QA Test No. 259
+#
+# btrfs vol tests
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2011 Oracle.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+# creator
[email protected]
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+    cd /
+    rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_need_to_be_root
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+_require_scratch_dev_pool
+_require_proc_scsi
+
+# arg 1 remove/add
+# arg 2 /dev/sdx or return of devmgt resply
+_devmgt()
+{
+	local x
+	local d
+
+	if [ $1 == "remove" ]; then
+		d=`echo $2|cut -d"/" -f3`
+		x=`ls -l /sys/class/block/${d} | cut -d "/" -f12 | sed 's/:/ /g'`
+		echo "scsi remove-single-device ${x}" > /proc/scsi/scsi || _fail "Remove disk failed"
+		DEVHTL=${x}
+	else
+		echo "scsi add-single-device ${2}" > /proc/scsi/scsi || _fail "Add disk failed"
+	fi
+}
+
+# Test cases related to raid in btrfs
+_test_raid0()
+{
+	export MKFS_OPTIONS="-m raid0 -d raid0"
+	_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs failed"
+	_scratch_mount
+	_fillfs 1 10 100 4096 8192 $SCRATCH_MNT
+	umount $SCRATCH_MNT
+}
+
+_test_raid1()
+{
+	export MKFS_OPTIONS="-m raid1 -d raid1"
+	_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs failed"
+	_scratch_mount
+	_fillfs 1 10 100 4096 8192 $SCRATCH_MNT
+	umount $SCRATCH_MNT
+}
+
+_test_raid10()
+{
+	export MKFS_OPTIONS="-m raid10 -d raid10"
+	_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs failed"
+	_scratch_mount
+	_fillfs 1 10 100 4096 8192 $SCRATCH_MNT
+	umount $SCRATCH_MNT
+}
+
+_test_single()
+{
+	export MKFS_OPTIONS="-m single -d single"
+	_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs failed"
+	_scratch_mount
+	_fillfs 1 10 100 4096 8192 $SCRATCH_MNT
+	umount $SCRATCH_MNT
+}
+
+_test_add()
+{
+	local i
+	local devs[]="( $SCRATCH_DEV_POOL )"
+	local n=${#devs[@]}
+
+	n=$(($n-1))
+
+	export MKFS_OPTIONS=""
+	_scratch_mkfs > /dev/null 2>&1 || _fail "mkfs failed"
+	_scratch_mount
+	_fillfs 1 10 100 4096 8192 $SCRATCH_MNT
+	for i in `seq 1 $n`
+	do
+		btrfs device add ${devs[$i]} $SCRATCH_MNT > /dev/null 2>&1 || _fail "device add failed"
+	done
+	btrfs filesystem balance $SCRATCH_MNT || _fail "balance failed"
+	#btrfs filesystem show $SCRATCH_DEV 2>&1 | egrep devid |awk '{
+	#	if ( $6 == "0.00" ) { exit 1 }
+	#}' || _fail "test6_add... failed"
+	umount $SCRATCH_MNT
+}
+
+_test_replace()
+{
+	local i
+	local x
+	local devs=( $SCRATCH_DEV_POOL )
+	local n=${#devs[@]}
+	local ds
+	local DEVHTL=""
+
+	# exclude the last disk in the disk pool
+	n=$(($n-1))
+	ds=${devs[@]:0:$n}
+
+	export MKFS_OPTIONS=""
+	_scratch_mkfs "$ds" > /dev/null 2>&1 || _fail "mkfs failed"
+	_scratch_mount
+	_fillfs 1 10 100 4096 8192 $SCRATCH_MNT
+
+	#pick the 2nd last disk
+	ds=${devs[@]:$(($n-1)):1}
+
+	#fail disk
+	_devmgt remove ${ds}
+
+	btrfs fi show $SCRATCH_DEV | grep "Some devices missing" > /dev/null || _fail \
+							"btrfs did not report device missing"
+
+	# add a new disk to btrfs
+	ds=${devs[@]:$(($n)):1}
+	btrfs device add ${ds} $SCRATCH_MNT > /dev/null 2>&1 || _fail "dev add failed"
+
+	# cleaup. add the removed disk
+	umount $SCRATCH_MNT
+	_devmgt add "${DEVHTL}"
+}
+
+_test_remove()
+{
+	_scratch_mkfs "$SCRATCH_DEV_POOL" > /dev/null 2>&1 || _fail "mkfs failed"
+	_scratch_mount
+	_fillfs 1 10 100 4096 8192 $SCRATCH_MNT
+
+	# pick last dev in the list
+	dev_del=`echo ${SCRATCH_DEV_POOL} | awk '{print $NF}'`
+	btrfs device delete $dev_del $SCRATCH_MNT || _fail "btrfs device delete failed"
+	btrfs fi show $SCRATCH_DEV 2>&1 | grep $dev_del > /dev/null && _fail "btrfs still shows the deleted dev"
+	umount $SCRATCH_MNT
+}
+
+_test_raid0
+_test_raid1
+_test_raid10
+_test_single
+_test_add
+_test_replace
+_test_remove
+
+echo "Silence is golden"
+status=0; exit

259.out new file +2 -0

new file mode 100644
index 0000000..bfbd2de
@@ -0,0 +1,2 @@
+QA output created by 259
+Silence is golden

common.rc +6 -0

index db7c2dd..aa45a7e 100644
@@ -1646,6 +1646,12 @@ _require_scratch_dev_pool()
 	;;
 	esac
 }
+
+# we need this to test removing a dev from the system
+_require_proc_scsi()
+{
+	[ -e /proc/scsi/scsi ]  || _notrun "/proc/scsi/scsi is not present"
+}

################################################################################

if [ "$iam" != new -a "$iam" != bench ]

group +1 -0

index 739f806..62eebfb 100644
@@ -372,3 +372,4 @@ deprecated
 256 auto quick
 257 auto quick
 258 auto quick
+259 auto quick

Signature · 1 line

1.7.1

Christoph Hellwig

2011-09-02 08:49 UTC #11 lore raw

Re: [PATCH 3/3] Added test case 259 for the btrfs raid features

Quoted 2 lines · Anand Jain

Added test case 259 for the btrfs raid features. SCRATCH_DEV_POOL must be set to 2 or more disks.

Any chance you can document how SCRATCH_DEV_POOL is supposed to be used in the README file? An addition patch is fine, no need to update
the existing ones. Also is there a chance you could allow setting
only SCRATCH_DEV_POOL for btrfs, and derive SCRATCH_DEV for that as an additional step?

Quoted 16 lines

259 new file +11 -0

+# arg 1 remove/add
+# arg 2 /dev/sdx or return of devmgt resply
+_devmgt()
+{
+	local x
+	local d
+
+	if [ $1 == "remove" ]; then
+		d=`echo $2|cut -d"/" -f3`
+		x=`ls -l /sys/class/block/${d} | cut -d "/" -f12 | sed 's/:/ /g'`
+		echo "scsi remove-single-device ${x}" > /proc/scsi/scsi || _fail

"Remove disk failed"

259 new file +3 -0

+		DEVHTL=${x}
+	else
+		echo "scsi add-single-device ${2}" > /proc/scsi/scsi || _fail

"Add disk failed"

Review

Quoted 2 lines

(quoted diff) +2 -0
+	fi
+}

Please use the sysfs interface instead of the deprecated /proc/scsi/scsi interface. I would also suggest to split this routine into two for removing and adding, and move them to the common helper library, so it could be used for other tests.

common.rc _require_scratch_dev_pool() · line 1654

Quoted 5 lines

common.rc +5 -0

+# we need this to test removing a dev from the system
+_require_proc_scsi()
+{
+	[ -e /proc/scsi/scsi ]  || _notrun "/proc/scsi/scsi is not present"
+}

The _require need really is that the device you want to work on is a SCSI device.

Anand Jain

author

2011-10-03 08:56 UTC #12 lore raw

Re: [PATCH 3/3] Added test case 259 for the btrfs raid features

Christoph,

apologies for the delay I was on a long leave.

Quoted 3 lines

Also is there a chance you could allow setting
only SCRATCH_DEV_POOL for btrfs, and derive SCRATCH_DEV for that as an additional step?

There is some challenge to set SCRATCH_DEV based on FSTYP since SCRATCH_DEV is used even before FSTYP is set.

common.config uses SCRATCH_DEV and we set FSTYP in common. check calls these two script files in the following logic.
--------------------------

 # we need common.config
 if ! . ./common.config
 then
    echo "$iam: failed to source common.config"
    exit 1
 fi

 # we need common
 . ./common
 ---------------------------

to avoid major changes what we could do is to find-out FSTYP on our own in the file common.config and set SCRATCH_DEV based on SCRATCH_DEV_POOL, but thats a bit ugly way. I am open to any suggestions. thanks.

Quoted 4 lines

Please use the sysfs interface instead of the deprecated /proc/scsi/scsi interface. I would also suggest to split this routine into two for removing and adding, and move them to the common helper library, so it could be used for other tests.

yes. will get something like below code. in the common

-----
remove:
echo 1 > /sys/class/scsi_device/${host}:${channel}:${id}:${lun}/device/delete

add:
echo "$channel $id $lun" > /sys/class/scsi_host/host${host}/scan
-----

Quoted 8 lines

common.rc _require_scratch_dev_pool() · line 1654
Quoted 5 lines
common.rc +5 -0
+# we need this to test removing a dev from the system
+_require_proc_scsi()
+{
+	[ -e /proc/scsi/scsi ]  || _notrun "/proc/scsi/scsi is not present"
+}
The _require need really is that the device you want to work on is a SCSI device.

will get this.

Thanks -Anand

Anand Jain

author

2011-10-10 09:55 UTC #13 lore raw

PATCH 1/1 Changes to received review comments

Now SCRATCH_DEV is derived from SCRATCH_DEV_POOL. Moved code to delete and scan block device to common.rc and updated README.

Signed-off-by Anand Jain <[email protected]>

---
 259           |   29 ++++++++---------------------
 README        |    9 +++++++--
 common.config |   14 ++++++++++++++
 common.rc     |   34 ++++++++++++++++++++++++++++++++--
 4 files changed, 61 insertions(+), 25 deletions(-)

259 +7 ~~-20~~

index 522191d..fe90147 100755
@@ -46,24 +46,7 @@ _supported_fs btrfs
 _supported_os Linux
 _require_scratch
 _require_scratch_dev_pool
-_require_proc_scsi
-
-# arg 1 remove/add
-# arg 2 /dev/sdx or return of devmgt resply
-_devmgt()
-{
-	local x
-	local d
-
-	if [ $1 == "remove" ]; then
-		d=`echo $2|cut -d"/" -f3`
-		x=`ls -l /sys/class/block/${d} | cut -d "/" -f12 | sed 's/:/ /g'`
-		echo "scsi remove-single-device ${x}" > /proc/scsi/scsi || _fail "Remove disk failed"
-		DEVHTL=${x}
-	else
-		echo "scsi add-single-device ${2}" > /proc/scsi/scsi || _fail "Add disk failed"
-	fi
-}
+_require_deletable_scratch_dev_pool
 
 # Test cases related to raid in btrfs
 _test_raid0()

@@ -128,10 +111,10 @@ _test_add()
 _test_replace()
 {
 	local i
-	local x
 	local devs=( $SCRATCH_DEV_POOL )
 	local n=${#devs[@]}
 	local ds
+	local d
 	local DEVHTL=""
 
 	# exclude the last disk in the disk pool

@@ -146,8 +129,12 @@ _test_replace()
 	#pick the 2nd last disk
 	ds=${devs[@]:$(($n-1)):1}
 
+	# retrive the HTL for this scsi disk
+	d=`echo $ds|cut -d"/" -f3`
+	DEVHTL=`ls -l /sys/class/block/${d} | rev | cut -d "/" -f 3 | rev`
+
 	#fail disk
-	_devmgt remove ${ds}
+	_devmgt_remove ${DEVHTL}
 
 	btrfs fi show $SCRATCH_DEV | grep "Some devices missing" > /dev/null

|| _fail \
"btrfs did not report device missing"
@@ -162,7 +149,7 @@ _test_replace()

 	# cleaup. add the removed disk
 	umount $SCRATCH_MNT
-	_devmgt add "${DEVHTL}"
+	_devmgt_add "${DEVHTL}"
 }

_test_remove()

README +6 -1

index 5367be6..407888a 100644
@@ -34,14 +34,19 @@ Preparing system for tests (IRIX and Linux):
             - leave empty and expect this partition to be clobbered
               by some tests.  If this is not provided, many tests will
               not be run.
-
+
         (these must be two DIFFERENT partitions)
+
+    - for btrfs only: some tests would need 3 or more independent SCRATCH disks,
+      which should be setenv SCRATCH_DEV_POOL instead of SCRATCH_DEV
+
 
     - setup your environment
         - setenv TEST_DEV "device containing TEST PARTITION"
         - setenv TEST_DIR "mount point of TEST PARTITION"
        	- optionally:
              - setenv SCRATCH_DEV "device containing SCRATCH PARTITION"
+             - setenv SCRATCH_DEV_POOL "pool of SCRATCH disks for testing btrfs"
              - setenv SCRATCH_MNT "mount point for SCRATCH PARTITION"
              - setenv TAPE_DEV "tape device for testing xfsdump"
              - setenv RMT_TAPE_DEV "remote tape device for testing

xfsdump"
@@ -63,7 +68,7 @@ Preparing system for tests (IRIX and Linux):
tape which can be overwritten.

- make sure $TEST_DEV is a mounted XFS partition
- - make sure that $SCRATCH_DEV contains nothing useful
+ - make sure that $SCRATCH_DEV or $SCRATCH_DEV_POOL contains nothing useful

Running tests:

common.config +14 -0

index 3642139..7ee255e 100644
@@ -228,6 +228,20 @@ if [ ! -d "$TEST_DIR" ]; then
     exit 1
 fi
 
+# a btrfs tester will set only SCRATCH_DEV_POOL, we will put first of its dev
+# to SCRATCH_DEV and rest to SCRATCH_DEV_POOL to maintain the backward compatibility
+if [ "$HOSTOS" == "Linux" ]; then
+    FSTYP_tmp=`blkid -c /dev/null -s TYPE -o value $TEST_DEV`
+else
+    FSTYP_tmp=xfs
+fi
+if [ "$FSTYP_tmp" == "btrfs" ]; then
+    if [ ! -z "$SCRATCH_DEV_POOL" ]; then
+	SCRATCH_DEV=`echo $SCRATCH_DEV_POOL | cut -d" " -f 1`
+	SCRATCH_DEV_POOL=`echo $SCRATCH_DEV_POOL | cut -d" " -f 2-`
+    fi
+fi
+	
 echo $SCRATCH_DEV | grep -q ":" > /dev/null 2>&1
 if [ ! -z "$SCRATCH_DEV" -a ! -b "$SCRATCH_DEV" -a "$?" != "0" ]; then
     echo "common.config: Error: \$SCRATCH_DEV ($SCRATCH_DEV) is not a

block device or a NFS filesystem"

common.rc +32 -2

index 02dde11..a614090 100644
@@ -1609,9 +1609,19 @@ _require_scratch_dev_pool()
 	esac
 }
 
-_require_proc_scsi()
+# We will check if the device is virtual (eg: loop device) since it does not
+# have the delete entry-point. Otherwise SCSI and USB devices are fine.
+_require_deletable_scratch_dev_pool()
 {
-	[ -e /proc/scsi/scsi ]  || _notrun "/proc/scsi/scsi is not present"
+	local i
+	local x
+	for i in $SCRATCH_DEV_POOL; do
+		x=`echo $i | cut -d"/" -f 3`
+		ls -l /sys/class/block/${x} | grep -q "virtual"
+		if [ $? == "0" ]; then
+			_notrun "$i is a virtual device which is not deletable"
+		fi
+	done
 }
 
 # Generate Random number in a range

@@ -1659,6 +1669,26 @@ _fillfs()
 	wait $!
 }
 
+# arg 1 is dev to remove and is output of the below eg.
+# ls -l /sys/class/block/sdd | rev | cut -d "/" -f 3 | rev
+_devmgt_remove()
+{
+	echo 1 > /sys/class/scsi_device/${1}/device/delete || _fail "Remove disk failed"
+}
+
+# arg 1 is dev to add and is output of the below eg.
+# ls -l /sys/class/block/sdd | rev | cut -d "/" -f 3 | rev
+_devmgt_add()
+{
+	local h
+	local tdl
+	# arg 1 will be in h:t:d:l format now in the h and "t d l" format
+	h=`echo ${1} | cut -d":" -f 1`
+	tdl=`echo ${1} | cut -d":" -f 2-|sed 's/:/ /g'`
+
+	echo ${tdl} >  /sys/class/scsi_host/host${h}/scan || _fail "Add disk failed"
+}
+

################################################################################
if [ "$iam" != new -a "$iam" != bench ]
then

Signature · 6 lines

1.7.1

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

Christoph Hellwig

2011-10-10 11:21 UTC #14 lore raw

Re: [PATCH] Changes to received review comments

Quoted 3 lines · Anand Jain

Now SCRATCH_DEV is derived from SCRATCH_DEV_POOL. Moved code to delete and scan block device to common.rc and updated README.

Can you resend the whole patch? Note that xfstests is up to test 262
in the meantime, so it will also need to be renumbered.

Anand Jain

author

2011-10-11 11:22 UTC #15 lore raw

Re: [PATCH] Changes to received review comments

Christoph,

Email following this will have patches, kindly find them.

Thanks, Anand

Quoted 7 lines · Christoph Hellwig

Quoted 3 lines · Anand Jain
Now SCRATCH_DEV is derived from SCRATCH_DEV_POOL. Moved code to delete and scan block device to common.rc and updated README.
Can you resend the whole patch? Note that xfstests is up to test 262
in the meantime, so it will also need to be renumbered.

Anand Jain

author

2011-10-11 11:23 UTC #16 lore raw

PATCH 1/3 263: Functional test case for the btrfs snapshot

Create snapshots in various ways, modify the data around the block and file boundaries and verify the data integrity.

Signed-off-by Anand Jain <[email protected]>

---
 263           |  192 
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 263.out       |    2 +
 README        |    7 ++-
 common.config |   14 ++++
 common.rc     |  122 ++++++++++++++++++++++++++++++++++++
 group         |    1 +
 6 files changed, 337 insertions(+), 1 deletions(-)
 create mode 100755 263
 create mode 100644 263.out

263 new file +192 -0

new file mode 100755
index 0000000..f26a677
@@ -0,0 +1,192 @@
+#!/bin/bash
+# FS QA Test No. 263
+#
+# Extented btrfs snapshot test cases
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2011 Oracle  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#
+#-----------------------------------------------------------------------
+#
+# creator
[email protected]
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+
+_cleanup()
+{
+    rm -f $tmp.*
+}
+
+trap "_cleanup ; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_need_to_be_root
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+
+_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs failed"
+_scratch_mount
+
+# Create and save sha256sum
+# arg1 FS to generate sha256
+# arg2 File name to save the sha256 output
+_save_checksum()
+{
+	local i=0
+	>$2
+	cd $1
+	for i in `find . -type f`; do sha256sum $i >> $2; done
+	cd $OLDPWD
+}
+
+# Verify the sha256sum for a FS
+# arg1 FS to be tested
+# arg2 sha256 file
+_verify_checksum()
+{
+	cd $1
+	[ -f $2 ] || _fail "checksum file $2 not found"
+	sha256sum -c $2 | grep "FAILED"
+	cd $OLDPWD
+}
+
+# Create a snapshot
+# arg1 dest dir
+# Return snapshot name in the SNAPNAME
+_create_snap()
+{
+	local x
+	[ -d $1 ] || _fail "Destination dir $1 not present"
+	SNAPNAME=`mktemp -u $SCRATCH_MNT/snap.XXXXXX`
+	btrfs subvolume snapshot $1 $SNAPNAME > /dev/null || _fail "snapshot create failed"
+}
+
+# Reads and writes new data but does not allocate new blocks
+# arg1 FS to be modified
+_read_modify_write()
+{
+	local i
+	local FSIZE
+	for i in `find $1 -type f`
+	do
+		FSIZE=`stat -t $i | cut -d" " -f2`
+		dd if=$i of=/dev/null obs=$FSIZE count=1 status=noxfer 2>/dev/null &
+		dd if=/dev/urandom of=$i obs=$FSIZE count=1 status=noxfer 2>/dev/null &
+	done
+	wait $!
+}
+
+# Fills the allocated blocks
+# arg1 FS in question
+_fill_blk()
+{
+	local FSIZE
+	local BLKS
+	local NBLK
+	local FALLOC
+	local WS
+
+	for i in `find /$1 -type f`
+	do
+		FSIZE=`stat -t $i | cut -d" " -f2`
+		BLKS=`stat -c "%B" $i`
+		NBLK=`stat -c "%b" $i`
+		FALLOC=$(($BLKS * $NBLK))
+		WS=$(($FALLOC - $FSIZE))
+		dd if=/dev/urandom of=$i oseek=$FSIZE obs=$WS count=1 status=noxfer 2>/dev/null &
+	done
+	wait $!
+}
+
+
+# Append a random size to the files
+# arg1 : FS in question
+_append_file()
+{
+	local FSIZE
+	local X
+	local N
+	local i
+	N=0
+	for i in `find $1 -type f`
+	do
+		if [ $N == 0 ]; then
+			X=$i
+			FSIZE=`stat -t $X | cut -d" " -f2`
+			dd if=$X of=$X seek=1 bs=$FSIZE obs=$FSIZE count=1 status=noxfer 2>/dev/null &
+			N=$(($N+1))
+			continue
+		fi
+		FSIZE=`stat -t $i | cut -d" " -f2`
+		dd if=$X of=$i seek=1 bs=$FSIZE obs=$FSIZE count=1 status=noxfer 2>/dev/null &
+		X=$i
+	done
+	wait $!
+}
+
+##################### real QA test starts here###################################
+# sv1 - is just a name nothing spl
+firstvol="$SCRATCH_MNT/sv1"
+btrfs subvolume create $firstvol > /dev/null || _fail "btrfs subvolume create $firstvol failed"
+_fillfs 1 10 100 4096 8192 $firstvol
+SNAPNAME=0
+_create_snap $firstvol
+_save_checksum $firstvol $tmp.sv1.sum
+_verify_checksum $SNAPNAME $tmp.sv1.sum
+
+#Append1 the files
+_fill_blk $SNAPNAME
+_verify_checksum $firstvol $tmp.sv1.sum
+
+#Append2 the files
+_append_file $SNAPNAME
+_verify_checksum $firstvol $tmp.sv1.sum
+
+#read modify write
+_read_modify_write $SNAPNAME
+_verify_checksum $firstvol $tmp.sv1.sum
+
+#nested snapshot test
+src_vol=$firstvol
+for i in `seq 1 7`; do
+	SNAPNAME=0
+	_create_snap $src_vol
+	_verify_checksum $SNAPNAME $tmp.sv1.sum
+	src_vol=$SNAPNAME
+done
+
+# file delete test
+SNAPNAME=0
+_create_snap $firstvol
+tname=`echo $SNAPNAME | rev | cut -d"/" -f1 | rev`
+_save_checksum $SNAPNAME $tmp.$tname.sum
+\rm -rf $firstvol/*
+_verify_checksum $SNAPNAME $tmp.$tname.sum
+
+umount $SCRATCH_DEV || _fail "unmount failed"
+
+echo "Silence is golden"
+status=0; exit

263.out new file +2 -0

new file mode 100644
index 0000000..7a26e35
@@ -0,0 +1,2 @@
+QA output created by 263
+Silence is golden

README +5 -0

index 5367be6..7c135c7 100644
@@ -36,12 +36,17 @@ Preparing system for tests (IRIX and Linux):
               not be run.
 
         (these must be two DIFFERENT partitions)
+
+    - for btrfs only: some tests would need 3 or more independent SCRATCH disks,
+      which should be setenv SCRATCH_DEV_POOL instead of SCRATCH_DEV
+
 
     - setup your environment
         - setenv TEST_DEV "device containing TEST PARTITION"
         - setenv TEST_DIR "mount point of TEST PARTITION"
        	- optionally:
              - setenv SCRATCH_DEV "device containing SCRATCH PARTITION"
+             - setenv SCRATCH_DEV_POOL "pool of SCRATCH disks for testing btrfs"
              - setenv SCRATCH_MNT "mount point for SCRATCH PARTITION"
              - setenv TAPE_DEV "tape device for testing xfsdump"
              - setenv RMT_TAPE_DEV "remote tape device for testing

xfsdump"
@@ -63,7 +68,7 @@ Preparing system for tests (IRIX and Linux):
tape which can be overwritten.

- make sure $TEST_DEV is a mounted XFS partition
- - make sure that $SCRATCH_DEV contains nothing useful
+ - make sure that $SCRATCH_DEV or $SCRATCH_DEV_POOL contains nothing useful

Running tests:

common.config +14 -0

index e94624e..4212fc8 100644
@@ -229,6 +229,20 @@ if [ ! -d "$TEST_DIR" ]; then
     exit 1
 fi
 
+# a btrfs tester will set only SCRATCH_DEV_POOL, we will put first of its dev
+# to SCRATCH_DEV and rest to SCRATCH_DEV_POOL to maintain the backward compatibility
+if [ "$HOSTOS" == "Linux" ]; then
+    FSTYP_tmp=`blkid -c /dev/null -s TYPE -o value $TEST_DEV`
+else
+    FSTYP_tmp=xfs
+fi
+if [ "$FSTYP_tmp" == "btrfs" ]; then
+    if [ ! -z "$SCRATCH_DEV_POOL" ]; then
+	SCRATCH_DEV=`echo $SCRATCH_DEV_POOL | cut -d" " -f 1`
+	SCRATCH_DEV_POOL=`echo $SCRATCH_DEV_POOL | cut -d" " -f 2-`
+    fi
+fi
+	
 echo $SCRATCH_DEV | grep -q ":" > /dev/null 2>&1
 if [ ! -z "$SCRATCH_DEV" -a ! -b "$SCRATCH_DEV" -a "$?" != "0" ]; then
     echo "common.config: Error: \$SCRATCH_DEV ($SCRATCH_DEV) is not a

block device or a NFS filesystem"

common.rc +122 -0

127 lines not on this page · raw message

################################################################################

group +1 -0

index 17466a1..d86624f 100644
@@ -376,3 +376,4 @@ deprecated
 260 auto quick trim
 261 auto quick quota
 262 auto quick quota
+263 auto quick

Signature · 1 line

1.7.1

David Sterba

2011-10-11 11:38 UTC #17 lore raw

Re: [PATCH 1/3] 263: Functional test case for the btrfs snapshot

please please fix your mail client not to wrap long lines ...

Quoted 148 lines · Anand Jain

Create snapshots in various ways, modify the data around the block and file boundaries and verify the data integrity.

Signed-off-by Anand Jain <[email protected]>

---
 263           |  192 
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 263.out       |    2 +
 README        |    7 ++-
 common.config |   14 ++++
 common.rc     |  122 ++++++++++++++++++++++++++++++++++++
 group         |    1 +
 6 files changed, 337 insertions(+), 1 deletions(-)
 create mode 100755 263
 create mode 100644 263.out

263 new file +84 -0

new file mode 100755
index 0000000..f26a677
@@ -0,0 +1,192 @@
+#!/bin/bash
+# FS QA Test No. 263
+#
+# Extented btrfs snapshot test cases
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2011 Oracle  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#
+#-----------------------------------------------------------------------
+#
+# creator
[email protected]
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+
+_cleanup()
+{
+    rm -f $tmp.*
+}
+
+trap "_cleanup ; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_need_to_be_root
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+
+_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs failed"
+_scratch_mount
+
+# Create and save sha256sum
+# arg1 FS to generate sha256
+# arg2 File name to save the sha256 output
+_save_checksum()
+{
+	local i=0
+	>$2
+	cd $1
+	for i in `find . -type f`; do sha256sum $i >> $2; done
+	cd $OLDPWD
+}
+
+# Verify the sha256sum for a FS
+# arg1 FS to be tested
+# arg2 sha256 file
+_verify_checksum()
+{
+	cd $1
+	[ -f $2 ] || _fail "checksum file $2 not found"
+	sha256sum -c $2 | grep "FAILED"
+	cd $OLDPWD
+}
+
+# Create a snapshot
+# arg1 dest dir
+# Return snapshot name in the SNAPNAME
+_create_snap()
+{
+	local x
+	[ -d $1 ] || _fail "Destination dir $1 not present"
+	SNAPNAME=`mktemp -u $SCRATCH_MNT/snap.XXXXXX`
+	btrfs subvolume snapshot $1 $SNAPNAME > /dev/null || _fail "snapshot

create failed"

263 new file +12 -0

+}
+
+# Reads and writes new data but does not allocate new blocks
+# arg1 FS to be modified
+_read_modify_write()
+{
+	local i
+	local FSIZE
+	for i in `find $1 -type f`
+	do
+		FSIZE=`stat -t $i | cut -d" " -f2`
+		dd if=$i of=/dev/null obs=$FSIZE count=1 status=noxfer

2>/dev/null &

263 new file +1 -0

+		dd if=/dev/urandom of=$i obs=$FSIZE count=1 status=noxfer

2>/dev/null &

263 new file +22 -0

+	done
+	wait $!
+}
+
+# Fills the allocated blocks
+# arg1 FS in question
+_fill_blk()
+{
+	local FSIZE
+	local BLKS
+	local NBLK
+	local FALLOC
+	local WS
+
+	for i in `find /$1 -type f`
+	do
+		FSIZE=`stat -t $i | cut -d" " -f2`
+		BLKS=`stat -c "%B" $i`
+		NBLK=`stat -c "%b" $i`
+		FALLOC=$(($BLKS * $NBLK))
+		WS=$(($FALLOC - $FSIZE))
+		dd if=/dev/urandom of=$i oseek=$FSIZE obs=$WS count=1

status=noxfer 2>/dev/null &

...

Quoted 312 lines

263 new file +20 -0

+	done
+	wait $!
+}
+
+
+# Append a random size to the files
+# arg1 : FS in question
+_append_file()
+{
+	local FSIZE
+	local X
+	local N
+	local i
+	N=0
+	for i in `find $1 -type f`
+	do
+		if [ $N == 0 ]; then
+			X=$i
+			FSIZE=`stat -t $X | cut -d" " -f2`
+			dd if=$X of=$X seek=1 bs=$FSIZE obs=$FSIZE count=1

status=noxfer 2>/dev/null &

263 new file +5 -0

+			N=$(($N+1))
+			continue
+		fi
+		FSIZE=`stat -t $i | cut -d" " -f2`
+		dd if=$X of=$i seek=1 bs=$FSIZE obs=$FSIZE count=1

status=noxfer 2>/dev/null &

263 new file +6 -0

+		X=$i
+	done
+	wait $!
+}
+
+##################### real QA test starts

here###################################

263 new file +3 -0

+# sv1 - is just a name nothing spl
+firstvol="$SCRATCH_MNT/sv1"
+btrfs subvolume create $firstvol > /dev/null || _fail "btrfs subvolume

create $firstvol failed"

263 new file +39 -0

+_fillfs 1 10 100 4096 8192 $firstvol
+SNAPNAME=0
+_create_snap $firstvol
+_save_checksum $firstvol $tmp.sv1.sum
+_verify_checksum $SNAPNAME $tmp.sv1.sum
+
+#Append1 the files
+_fill_blk $SNAPNAME
+_verify_checksum $firstvol $tmp.sv1.sum
+
+#Append2 the files
+_append_file $SNAPNAME
+_verify_checksum $firstvol $tmp.sv1.sum
+
+#read modify write
+_read_modify_write $SNAPNAME
+_verify_checksum $firstvol $tmp.sv1.sum
+
+#nested snapshot test
+src_vol=$firstvol
+for i in `seq 1 7`; do
+	SNAPNAME=0
+	_create_snap $src_vol
+	_verify_checksum $SNAPNAME $tmp.sv1.sum
+	src_vol=$SNAPNAME
+done
+
+# file delete test
+SNAPNAME=0
+_create_snap $firstvol
+tname=`echo $SNAPNAME | rev | cut -d"/" -f1 | rev`
+_save_checksum $SNAPNAME $tmp.$tname.sum
+\rm -rf $firstvol/*
+_verify_checksum $SNAPNAME $tmp.$tname.sum
+
+umount $SCRATCH_DEV || _fail "unmount failed"
+
+echo "Silence is golden"
+status=0; exit

263.out new file +2 -0

new file mode 100644
index 0000000..7a26e35
@@ -0,0 +1,2 @@
+QA output created by 263
+Silence is golden

README +5 -0

index 5367be6..7c135c7 100644
@@ -36,12 +36,17 @@ Preparing system for tests (IRIX and Linux):
               not be run.
 
         (these must be two DIFFERENT partitions)
+
+    - for btrfs only: some tests would need 3 or more independent 
 SCRATCH disks,
+      which should be setenv SCRATCH_DEV_POOL instead of SCRATCH_DEV
+
 
     - setup your environment
         - setenv TEST_DEV "device containing TEST PARTITION"
         - setenv TEST_DIR "mount point of TEST PARTITION"
        	- optionally:
              - setenv SCRATCH_DEV "device containing SCRATCH PARTITION"
+             - setenv SCRATCH_DEV_POOL "pool of SCRATCH disks for

testing btrfs"
- setenv SCRATCH_MNT "mount point for SCRATCH PARTITION"
- setenv TAPE_DEV "tape device for testing xfsdump"
- setenv RMT_TAPE_DEV "remote tape device for testing
xfsdump"

README +1 -1

@@ -63,7 +68,7 @@ Preparing system for tests (IRIX and Linux):
       tape which can be overwritten.
 
     - make sure $TEST_DEV is a mounted XFS partition
-    - make sure that $SCRATCH_DEV contains nothing useful
+    - make sure that $SCRATCH_DEV or $SCRATCH_DEV_POOL contains nothing

useful

Running tests:

common.config +1 -0

index e94624e..4212fc8 100644
@@ -229,6 +229,20 @@ if [ ! -d "$TEST_DIR" ]; then
     exit 1
 fi
 
+# a btrfs tester will set only SCRATCH_DEV_POOL, we will put first of

its dev
+# to SCRATCH_DEV and rest to SCRATCH_DEV_POOL to maintain the backward compatibility

common.config +12 -0

+if [ "$HOSTOS" == "Linux" ]; then
+    FSTYP_tmp=`blkid -c /dev/null -s TYPE -o value $TEST_DEV`
+else
+    FSTYP_tmp=xfs
+fi
+if [ "$FSTYP_tmp" == "btrfs" ]; then
+    if [ ! -z "$SCRATCH_DEV_POOL" ]; then
+	SCRATCH_DEV=`echo $SCRATCH_DEV_POOL | cut -d" " -f 1`
+	SCRATCH_DEV_POOL=`echo $SCRATCH_DEV_POOL | cut -d" " -f 2-`
+    fi
+fi
+	
 echo $SCRATCH_DEV | grep -q ":" > /dev/null 2>&1
 if [ ! -z "$SCRATCH_DEV" -a ! -b "$SCRATCH_DEV" -a "$?" != "0" ]; then
     echo "common.config: Error: \$SCRATCH_DEV ($SCRATCH_DEV) is not a

block device or a NFS filesystem"

common.rc +9 -0

index e948169..861f1f8 100644
@@ -1591,6 +1591,128 @@ _test_inode_extsz()
     echo $blocks
 }
 
+# scratch_dev_pool should contain the disks pool for the btrfs raid
+_require_scratch_dev_pool()
+{
+	local i
+	case "$FSTYP" in
+	btrfs)
+		if [ -z "$SCRATCH_DEV_POOL" ]
+		then
+			_notrun "this test requires a valid

\$SCRATCH_DEV_POOL"

common.rc +4 -0

+		fi
+		if [ "`echo $SCRATCH_DEV_POOL|wc -w`" -lt 2 ]
+		then
+			_notrun "this test needs 2 or more disks in

SCRATCH_DEV_POOL"

common.rc +7 -0

+		fi
+
+		for i in $SCRATCH_DEV_POOL
+		do
+			if [ "`_is_block_dev $i`" = "" ]
+			then
+				_notrun "this test requires valid block disk

$i"

common.rc +2 -0

+			fi
+			if [ "`_is_block_dev $i`" = "`_is_block_dev

$TEST_DEV`" ]

common.rc +2 -0

+			then
+				_notrun "$i is part of TEST_DEV, this test

requires unique disks"

common.rc +2 -0

+			fi
+			if [ "`_is_block_dev $i`" = "`_is_block_dev

$SCRATCH_DEV`" ]

common.rc +2 -0

+			then
+				_notrun "$i is part of SCRATCH_DEV, this

test requires unique disks"

common.rc +10 -0

+			fi
+			if _mount | grep -q $i
+			then
+				if ! $UMOUNT_PROG $i
+			        then
+			            echo "failed to unmount $i - aborting"
+			            exit 1
+			        fi
+			fi
+			dd if=/dev/zero of=$i bs=4096 count=100 > /dev/null

2>&1

common.rc +6 -0

+		done
+	;;
+	esac
+}
+
+# We will check if the device is virtual (eg: loop device) since it

does not

common.rc +10 -0

+# have the delete entry-point. Otherwise SCSI and USB devices are fine.
+_require_deletable_scratch_dev_pool()
+{
+	local i
+	local x
+	for i in $SCRATCH_DEV_POOL; do
+		x=`echo $i | cut -d"/" -f 3`
+		ls -l /sys/class/block/${x} | grep -q "virtual"
+		if [ $? == "0" ]; then
+			_notrun "$i is a virtual device which is not

deletable"

common.rc +44 -0

+		fi
+	done
+}
+
+# Generate Random number in a range
+# arg1 min, arg2 max
+_rand_range()
+{
+	local X
+	local Y
+	if [ $2 == 0 ]; then echo 0; return; fi
+	RANDOM=1
+	Y=$RANDOM
+	((X = $2 - $1 + 1))
+	((X = $Y % $X))
+	((X = $X + $1))
+	echo $X
+}
+
+# Create Dir tree and files in it.
+# arg1 basedir
+# arg2 dir depth
+# arg3 nfile_min
+# arg4 nfile_max
+# arg5 fsize_min
+# arg6 fsize_max
+_fillfs()
+{
+	umask 000
+	local j
+	local i
+	local DIRP
+	local FCNT
+	local FILEP
+	local SCNT
+	local BCNT
+	DIRP=$6
+	for ((j=0; j<$1; j++)); do
+		DIRP=`mktemp -dq $DIRP/dir.XXXXXX`
+		FCNT=$(_rand_range $2 $3)
+		for ((i=0; i<$FCNT; i++)); do
+			FILEP=`mktemp -q $DIRP/file.XXXXXX`
+			SCNT=$(_rand_range $4 $5)
+			dd if=/dev/urandom of=$FILEP bs=$SCNT count=1

status=noxfer 2>/dev/null &

common.rc +10 -0

+		done
+	done
+	wait $!
+}
+
+# arg 1 is dev to remove and is output of the below eg.
+# ls -l /sys/class/block/sdd | rev | cut -d "/" -f 3 | rev
+_devmgt_remove()
+{
+	echo 1 > /sys/class/scsi_device/${1}/device/delete || _fail "Remove

disk failed"

common.rc +13 -0

+}
+
+# arg 1 is dev to add and is output of the below eg.
+# ls -l /sys/class/block/sdd | rev | cut -d "/" -f 3 | rev
+_devmgt_add()
+{
+	local h
+	local tdl
+	# arg 1 will be in h:t:d:l format now in the h and "t d l" format
+	h=`echo ${1} | cut -d":" -f 1`
+	tdl=`echo ${1} | cut -d":" -f 2-|sed 's/:/ /g'`
+
+	echo ${tdl} >  /sys/class/scsi_host/host${h}/scan || _fail "Add disk

failed"
+}

################################################################################

group +1 -0

index 17466a1..d86624f 100644
@@ -376,3 +376,4 @@ deprecated
 260 auto quick trim
 261 auto quick quota
 262 auto quick quota
+263 auto quick

Signature · 7 lines

1.7.1


--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Christoph Hellwig

2011-10-11 11:42 UTC #18 lore raw

Re: [PATCH 1/3] 263: Functional test case for the btrfs snapshot

Quoted 1 line · David Sterba

please please fix your mail client not to wrap long lines ...

And while we're at it please fix yours to not full quote :)

David Sterba

2011-10-11 11:47 UTC #19 lore raw

Re: [PATCH 1/3] 263: Functional test case for the btrfs snapshot

Quoted 4 lines · Christoph Hellwig

Quoted 1 line · David Sterba
please please fix your mail client not to wrap long lines ...
And while we're at it please fix yours to not full quote :)

It was on purpose, I have marked some of those long lines.

david

Anand Jain

author

2011-10-11 11:24 UTC #20 lore raw

PATCH 2/3 264: Functional test case for the btrfs de-fragmentation

To verify the btrfs de-fragmentation does not fail

Signed-off-by Anand Jain <[email protected]>

---
 264     |   60 
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 264.out |    2 ++
 group   |    1 +
 3 files changed, 63 insertions(+), 0 deletions(-)
 create mode 100755 264
 create mode 100644 264.out

264 new file +60 -0

new file mode 100755
index 0000000..61efeb8
@@ -0,0 +1,60 @@
+#! /bin/bash
+# FS QA Test No. 264
+#
+# btrfs defragmentation tests
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2011 Oracle.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+# creator
[email protected]
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+    cd /
+    rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_need_to_be_root
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+
+_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs failed"
+_scratch_mount
+
+_fillfs 1 10 100 4096 8192 $SCRATCH_MNT
+
+btrfs filesystem defragment $SCRATCH_MNT
+[ $? = 20 ] || _fail "Defragment failed"
+
+umount $SCRATCH_MNT || _fail "umount failed"
+
+echo "Silence is golden"
+status=0; exit

264.out new file +2 -0

new file mode 100644
index 0000000..eca790b
@@ -0,0 +1,2 @@
+QA output created by 264
+Silence is golden

group +1 -0

index d86624f..b5863be 100644
@@ -377,3 +377,4 @@ deprecated
 261 auto quick quota
 262 auto quick quota
 263 auto quick
+264 auto quick

Signature · 1 line

1.7.1

Anand Jain

author

2011-10-11 11:25 UTC #21 lore raw

PATCH 3/3 265: Functional test case for the btrfs raid operations

This will verify the various raid features in btrfs and device replacement functionality

Signed-off-by Anand Jain <[email protected]>

---
 265     |  187 
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 265.out |    2 +
 group   |    1 +
 3 files changed, 190 insertions(+), 0 deletions(-)
 create mode 100755 265
 create mode 100644 265.out

265 new file +187 -0

new file mode 100755
index 0000000..4e35f6b
@@ -0,0 +1,187 @@
+#! /bin/bash
+# FS QA Test No. 265
+#
+# btrfs vol tests
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2011 Oracle.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+# creator
[email protected]
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+dev_removed=0
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+    cd /
+    rm -f $tmp.*
+    if [ $dev_removed == 1 ]; then
+	umount $SCRATCH_MNT
+        _devmgt_add "${DEVHTL}"
+    fi
+}
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_need_to_be_root
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+_require_scratch_dev_pool
+_require_deletable_scratch_dev_pool
+
+# Test cases related to raid in btrfs
+_test_raid0()
+{
+	export MKFS_OPTIONS="-m raid0 -d raid0"
+	_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs failed"
+	_scratch_mount
+	_fillfs 1 10 100 4096 8192 $SCRATCH_MNT
+	umount $SCRATCH_MNT
+}
+
+_test_raid1()
+{
+	export MKFS_OPTIONS="-m raid1 -d raid1"
+	_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs failed"
+	_scratch_mount
+	_fillfs 1 10 100 4096 8192 $SCRATCH_MNT
+	umount $SCRATCH_MNT
+}
+
+_test_raid10()
+{
+	export MKFS_OPTIONS="-m raid10 -d raid10"
+	_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs failed"
+	_scratch_mount
+	_fillfs 1 10 100 4096 8192 $SCRATCH_MNT
+	umount $SCRATCH_MNT
+}
+
+_test_single()
+{
+	export MKFS_OPTIONS="-m single -d single"
+	_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs failed"
+	_scratch_mount
+	_fillfs 1 10 100 4096 8192 $SCRATCH_MNT
+	umount $SCRATCH_MNT
+}
+
+_test_add()
+{
+	local i
+	local devs[]="( $SCRATCH_DEV_POOL )"
+	local n=${#devs[@]}
+
+	n=$(($n-1))
+
+	export MKFS_OPTIONS=""
+	_scratch_mkfs > /dev/null 2>&1 || _fail "mkfs failed"
+	_scratch_mount
+	_fillfs 1 10 100 4096 8192 $SCRATCH_MNT
+	for i in `seq 1 $n`
+	do
+		btrfs device add ${devs[$i]} $SCRATCH_MNT > /dev/null 2>&1 || _fail "device add failed"
+	done
+	btrfs filesystem balance $SCRATCH_MNT || _fail "balance failed"
+	#btrfs filesystem show $SCRATCH_DEV 2>&1 | egrep devid |awk '{
+	#	if ( $6 == "0.00" ) { exit 1 }
+	#}' || _fail "test6_add... failed"
+	umount $SCRATCH_MNT
+}
+
+_test_replace()
+{
+	local i
+	local devs=( $SCRATCH_DEV_POOL )
+	local n=${#devs[@]}
+	local ds
+	local d
+	local DEVHTL=""
+
+	# exclude the last disk in the disk pool
+	n=$(($n-1))
+	ds=${devs[@]:0:$n}
+
+	export MKFS_OPTIONS="-m raid1 -d raid1"
+	_scratch_mkfs "$ds" > /dev/null 2>&1 || _fail "tr: mkfs failed"
+	_scratch_mount
+	_fillfs 1 10 100 4096 8192 $SCRATCH_MNT
+
+	#pick the 2nd last disk
+	ds=${devs[@]:$(($n-1)):1}
+
+	# retrive the HTL for this scsi disk
+	d=`echo $ds|cut -d"/" -f3`
+	DEVHTL=`ls -l /sys/class/block/${d} | rev | cut -d "/" -f 3 | rev`
+
+	#fail disk
+	_devmgt_remove ${DEVHTL}
+	dev_removed=1
+
+	btrfs fi show $SCRATCH_DEV | grep "Some devices missing" > /dev/null || _fail \
+							"btrfs did not report device missing"
+
+	# add a new disk to btrfs
+	ds=${devs[@]:$(($n)):1}
+	btrfs device add ${ds} $SCRATCH_MNT > /dev/null 2>&1 || _fail "dev add failed"
+	# in some system balance fails if there is no delay (a bug)
+	# putting sleep 10 to work around as of now
+	# sleep 10
+	btrfs fi balance $SCRATCH_MNT || _fail "dev balance failed"
+	#btrfs filesystem show | egrep devid |awk '{
+	#	if ( $6 == "0.00" ) { exit 1 }
+	#}' || _fail "btrfs balance failed"
+
+	# cleaup. add the removed disk
+	umount $SCRATCH_MNT
+	_devmgt_add "${DEVHTL}"
+	dev_removed=0
+}
+
+_test_remove()
+{
+	_scratch_mkfs "$SCRATCH_DEV_POOL" > /dev/null 2>&1 || _fail "mkfs failed"
+	_scratch_mount
+	_fillfs 1 10 100 4096 8192 $SCRATCH_MNT
+
+	# pick last dev in the list
+	dev_del=`echo ${SCRATCH_DEV_POOL} | awk '{print $NF}'`
+	btrfs device delete $dev_del $SCRATCH_MNT || _fail "btrfs device delete failed"
+	btrfs fi show $SCRATCH_DEV 2>&1 | grep $dev_del > /dev/null && _fail "btrfs still shows the deleted dev"
+	umount $SCRATCH_MNT
+}
+
+_test_raid0
+_test_raid1
+_test_raid10
+_test_single
+_test_add
+_test_replace
+_test_remove
+
+echo "Silence is golden"
+status=0; exit

265.out new file +2 -0

new file mode 100644
index 0000000..9fa4eb0
@@ -0,0 +1,2 @@
+QA output created by 265
+Silence is golden

group +1 -0

index b5863be..b991b24 100644
@@ -378,3 +378,4 @@ deprecated
 262 auto quick quota
 263 auto quick
 264 auto quick
+265 auto

Signature · 1 line

1.7.1

Anand Jain

author

2011-10-12 04:50 UTC #22 lore raw

cover xfstest patch

Anand Jain (3):
263: Functional test case for the btrfs snapshot 264: Functional test case for the btrfs de-fragmentation 265: Functional test case for the btrfs raid operations

 263           |  192 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 263.out       |    2 +
 264           |   60 ++++++++++++++++++
 264.out       |    2 +
 265           |  187 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 265.out       |    2 +
 README        |    7 ++-
 common.config |   14 ++++
 common.rc     |  122 ++++++++++++++++++++++++++++++++++++
 group         |    3 +
 10 files changed, 590 insertions(+), 1 deletions(-)
 create mode 100755 263
 create mode 100644 263.out
 create mode 100755 264
 create mode 100644 264.out
 create mode 100755 265
 create mode 100644 265.out

Signature · 6 lines

1.7.7

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

Anand Jain

author

2011-10-12 04:50 UTC #23 lore raw

PATCH 1/3 263: Functional test case for the btrfs snapshot

Create snapshots in various ways, modify the data around the block and file boundaries and verify the data integrity.

Signed-off-by Anand Jain <[email protected]>

---
 263           |  192 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 263.out       |    2 +
 README        |    7 ++-
 common.config |   14 ++++
 common.rc     |  122 ++++++++++++++++++++++++++++++++++++
 group         |    1 +
 6 files changed, 337 insertions(+), 1 deletions(-)
 create mode 100755 263
 create mode 100644 263.out

263 new file +192 -0

new file mode 100755
index 0000000..f26a677
@@ -0,0 +1,192 @@
+#!/bin/bash
+# FS QA Test No. 263
+#
+# Extented btrfs snapshot test cases
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2011 Oracle  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#
+#-----------------------------------------------------------------------
+#
+# creator
[email protected]
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+
+_cleanup()
+{
+    rm -f $tmp.*
+}
+
+trap "_cleanup ; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_need_to_be_root
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+
+_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs failed"
+_scratch_mount
+
+# Create and save sha256sum
+# arg1 FS to generate sha256
+# arg2 File name to save the sha256 output
+_save_checksum()
+{
+	local i=0
+	>$2
+	cd $1
+	for i in `find . -type f`; do sha256sum $i >> $2; done
+	cd $OLDPWD
+}
+
+# Verify the sha256sum for a FS
+# arg1 FS to be tested
+# arg2 sha256 file
+_verify_checksum()
+{
+	cd $1
+	[ -f $2 ] || _fail "checksum file $2 not found"
+	sha256sum -c $2 | grep "FAILED"
+	cd $OLDPWD
+}
+
+# Create a snapshot
+# arg1 dest dir
+# Return snapshot name in the SNAPNAME
+_create_snap()
+{
+	local x
+	[ -d $1 ] || _fail "Destination dir $1 not present"
+	SNAPNAME=`mktemp -u $SCRATCH_MNT/snap.XXXXXX`
+	btrfs subvolume snapshot $1 $SNAPNAME > /dev/null || _fail "snapshot create failed"
+}
+
+# Reads and writes new data but does not allocate new blocks
+# arg1 FS to be modified
+_read_modify_write()
+{
+	local i
+	local FSIZE
+	for i in `find $1 -type f`
+	do
+		FSIZE=`stat -t $i | cut -d" " -f2`
+		dd if=$i of=/dev/null obs=$FSIZE count=1 status=noxfer 2>/dev/null &
+		dd if=/dev/urandom of=$i obs=$FSIZE count=1 status=noxfer 2>/dev/null &
+	done
+	wait $!
+}
+
+# Fills the allocated blocks
+# arg1 FS in question
+_fill_blk()
+{
+	local FSIZE
+	local BLKS
+	local NBLK
+	local FALLOC
+	local WS
+
+	for i in `find /$1 -type f`
+	do
+		FSIZE=`stat -t $i | cut -d" " -f2`
+		BLKS=`stat -c "%B" $i`
+		NBLK=`stat -c "%b" $i`
+		FALLOC=$(($BLKS * $NBLK))
+		WS=$(($FALLOC - $FSIZE))
+		dd if=/dev/urandom of=$i oseek=$FSIZE obs=$WS count=1 status=noxfer 2>/dev/null &
+	done
+	wait $!
+}
+
+
+# Append a random size to the files
+# arg1 : FS in question
+_append_file()
+{
+	local FSIZE
+	local X
+	local N
+	local i
+	N=0
+	for i in `find $1 -type f`
+	do
+		if [ $N == 0 ]; then
+			X=$i
+			FSIZE=`stat -t $X | cut -d" " -f2`
+			dd if=$X of=$X seek=1 bs=$FSIZE obs=$FSIZE count=1 status=noxfer 2>/dev/null &
+			N=$(($N+1))
+			continue
+		fi
+		FSIZE=`stat -t $i | cut -d" " -f2`
+		dd if=$X of=$i seek=1 bs=$FSIZE obs=$FSIZE count=1 status=noxfer 2>/dev/null &
+		X=$i
+	done
+	wait $!
+}
+
+##################### real QA test starts here###################################
+# sv1 - is just a name nothing spl
+firstvol="$SCRATCH_MNT/sv1"
+btrfs subvolume create $firstvol > /dev/null || _fail "btrfs subvolume create $firstvol failed"
+_fillfs 1 10 100 4096 8192 $firstvol
+SNAPNAME=0
+_create_snap $firstvol
+_save_checksum $firstvol $tmp.sv1.sum
+_verify_checksum $SNAPNAME $tmp.sv1.sum
+
+#Append1 the files
+_fill_blk $SNAPNAME
+_verify_checksum $firstvol $tmp.sv1.sum
+
+#Append2 the files
+_append_file $SNAPNAME
+_verify_checksum $firstvol $tmp.sv1.sum
+
+#read modify write
+_read_modify_write $SNAPNAME
+_verify_checksum $firstvol $tmp.sv1.sum
+
+#nested snapshot test
+src_vol=$firstvol
+for i in `seq 1 7`; do
+	SNAPNAME=0
+	_create_snap $src_vol
+	_verify_checksum $SNAPNAME $tmp.sv1.sum
+	src_vol=$SNAPNAME
+done
+
+# file delete test
+SNAPNAME=0
+_create_snap $firstvol
+tname=`echo $SNAPNAME | rev | cut -d"/" -f1 | rev`
+_save_checksum $SNAPNAME $tmp.$tname.sum
+\rm -rf $firstvol/*
+_verify_checksum $SNAPNAME $tmp.$tname.sum
+
+umount $SCRATCH_DEV || _fail "unmount failed"
+
+echo "Silence is golden"
+status=0; exit

263.out new file +2 -0

new file mode 100644
index 0000000..7a26e35
@@ -0,0 +1,2 @@
+QA output created by 263
+Silence is golden

README +6 -1

index 5367be6..7c135c7 100644
@@ -36,12 +36,17 @@ Preparing system for tests (IRIX and Linux):
               not be run.
               
         (these must be two DIFFERENT partitions)
+
+    - for btrfs only: some tests would need 3 or more independent SCRATCH disks,
+      which should be setenv SCRATCH_DEV_POOL instead of SCRATCH_DEV
+              
               
     - setup your environment
         - setenv TEST_DEV "device containing TEST PARTITION"
         - setenv TEST_DIR "mount point of TEST PARTITION"   
        	- optionally:
              - setenv SCRATCH_DEV "device containing SCRATCH PARTITION"
+             - setenv SCRATCH_DEV_POOL "pool of SCRATCH disks for testing btrfs"
              - setenv SCRATCH_MNT "mount point for SCRATCH PARTITION"
              - setenv TAPE_DEV "tape device for testing xfsdump"
              - setenv RMT_TAPE_DEV "remote tape device for testing xfsdump"

@@ -63,7 +68,7 @@ Preparing system for tests (IRIX and Linux):
       tape which can be overwritten.
           
     - make sure $TEST_DEV is a mounted XFS partition
-    - make sure that $SCRATCH_DEV contains nothing useful
+    - make sure that $SCRATCH_DEV or $SCRATCH_DEV_POOL contains nothing useful
     
 Running tests:

common.config +14 -0

index e94624e..4212fc8 100644
@@ -229,6 +229,20 @@ if [ ! -d "$TEST_DIR" ]; then
     exit 1
 fi
 
+# a btrfs tester will set only SCRATCH_DEV_POOL, we will put first of its dev
+# to SCRATCH_DEV and rest to SCRATCH_DEV_POOL to maintain the backward compatibility
+if [ "$HOSTOS" == "Linux" ]; then
+    FSTYP_tmp=`blkid -c /dev/null -s TYPE -o value $TEST_DEV`
+else
+    FSTYP_tmp=xfs
+fi
+if [ "$FSTYP_tmp" == "btrfs" ]; then
+    if [ ! -z "$SCRATCH_DEV_POOL" ]; then
+	SCRATCH_DEV=`echo $SCRATCH_DEV_POOL | cut -d" " -f 1`
+	SCRATCH_DEV_POOL=`echo $SCRATCH_DEV_POOL | cut -d" " -f 2-`
+    fi
+fi
+	
 echo $SCRATCH_DEV | grep -q ":" > /dev/null 2>&1
 if [ ! -z "$SCRATCH_DEV" -a ! -b "$SCRATCH_DEV" -a "$?" != "0" ]; then
     echo "common.config: Error: \$SCRATCH_DEV ($SCRATCH_DEV) is not a block device or a NFS filesystem"

common.rc +122 -0

128 lines not on this page · raw message

group +1 -0

index 17466a1..d86624f 100644
@@ -376,3 +376,4 @@ deprecated
 260 auto quick trim
 261 auto quick quota
 262 auto quick quota
+263 auto quick

Signature · 1 line

1.7.7

Dave Chinner

2011-10-13 00:56 UTC #24 lore raw

Re: [PATCH 1/3] 263: Functional test case for the btrfs snapshot

Quoted 2 lines · Anand Jain

Create snapshots in various ways, modify the data around the block and file boundaries and verify the data integrity.

....

Quoted 20 lines

common.rc +20 -0

+
+# Create Dir tree and files in it.
+# arg1 basedir
+# arg2 dir depth
+# arg3 nfile_min
+# arg4 nfile_max
+# arg5 fsize_min
+# arg6 fsize_max
+_fillfs()
+{
+	umask 000
+	local j
+	local i
+	local DIRP
+	local FCNT
+	local FILEP
+	local SCNT
+	local BCNT
+	DIRP=$6
+	for ((j=0; j<$1; j++)); do

common.rc _test_inode_extsz() · line 1695

Quoted 10 lines

common.rc +10 -0

+		DIRP=`mktemp -dq $DIRP/dir.XXXXXX`
+		FCNT=$(_rand_range $2 $3)
+		for ((i=0; i<$FCNT; i++)); do
+			FILEP=`mktemp -q $DIRP/file.XXXXXX`
+			SCNT=$(_rand_range $4 $5)
+			dd if=/dev/urandom of=$FILEP bs=$SCNT count=1 status=noxfer 2>/dev/null &
+		done
+	done
+	wait $!
+}

Please, no, not -another- new and just a bit different "fill filesystem" function.

There's already been one added recently in test 256 which I commented at the time could use _populate_fs, and I said the same thing in my original review of this series, too.

So please modify the _populate_fs function to do what you need.

Cheers,

Dave.

Signature · 2 lines

Dave Chinner
[email protected]

Anand Jain

author

2011-10-18 06:25 UTC #25 lore raw

cover xfstests patches

Looks like there are no further comments. Changes per Dave pointing out to reuse _populate_fs. Thanks for the review.

Anand Jain (3):
264: Functional test case for the btrfs snapshot 265: Functional test case for the btrfs de-fragmentation 266: Functional test case for the btrfs raid operations

 264           |  192 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 264.out       |    2 +
 265           |   60 ++++++++++++++++++
 265.out       |    2 +
 266           |  187 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 266.out       |    2 +
 README        |    7 ++-
 common.config |   14 ++++
 common.rc     |   95 +++++++++++++++++++++++++++-
 group         |    3 +
 10 files changed, 559 insertions(+), 5 deletions(-)
 create mode 100755 264
 create mode 100644 264.out
 create mode 100755 265
 create mode 100644 265.out
 create mode 100755 266
 create mode 100644 266.out

Signature · 1 line

1.7.7

Anand Jain

author

2011-10-18 06:26 UTC #26 lore raw

PATCH 1/3 264: Functional test case for the btrfs snapshot

Create snapshots in various ways, modify the data around the block and file boundaries and verify the data integrity.

Signed-off-by Anand Jain <[email protected]>

---
 264           |  192 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 264.out       |    2 +
 README        |    7 ++-
 common.config |   14 ++++
 common.rc     |   95 +++++++++++++++++++++++++++-
 group         |    1 +
 6 files changed, 306 insertions(+), 5 deletions(-)
 create mode 100755 264
 create mode 100644 264.out

264 new file +192 -0

new file mode 100755
index 0000000..cd7aae2
@@ -0,0 +1,192 @@
+#!/bin/bash
+# FS QA Test No. 263
+#
+# Extented btrfs snapshot test cases
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2011 Oracle  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#
+#-----------------------------------------------------------------------
+#
+# creator
[email protected]
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+
+_cleanup()
+{
+    rm -f $tmp.*
+}
+
+trap "_cleanup ; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_need_to_be_root
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+
+_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs failed"
+_scratch_mount
+
+# Create and save sha256sum
+# arg1 FS to generate sha256
+# arg2 File name to save the sha256 output
+_save_checksum()
+{
+	local i=0
+	>$2
+	cd $1
+	for i in `find . -type f`; do sha256sum $i >> $2; done
+	cd $OLDPWD
+}
+
+# Verify the sha256sum for a FS
+# arg1 FS to be tested
+# arg2 sha256 file
+_verify_checksum()
+{
+	cd $1
+	[ -f $2 ] || _fail "checksum file $2 not found"
+	sha256sum -c $2 | grep "FAILED"
+	cd $OLDPWD
+}
+
+# Create a snapshot
+# arg1 dest dir
+# Return snapshot name in the SNAPNAME
+_create_snap()
+{
+	local x
+	[ -d $1 ] || _fail "Destination dir $1 not present"
+	SNAPNAME=`mktemp -u $SCRATCH_MNT/snap.XXXXXX`
+	btrfs subvolume snapshot $1 $SNAPNAME > /dev/null || _fail "snapshot create failed"
+}
+
+# Reads and writes new data but does not allocate new blocks
+# arg1 FS to be modified
+_read_modify_write()
+{
+	local i
+	local FSIZE
+	for i in `find $1 -type f`
+	do
+		FSIZE=`stat -t $i | cut -d" " -f2`
+		dd if=$i of=/dev/null obs=$FSIZE count=1 status=noxfer 2>/dev/null &
+		dd if=/dev/urandom of=$i obs=$FSIZE count=1 status=noxfer 2>/dev/null &
+	done
+	wait $!
+}
+
+# Fills the allocated blocks
+# arg1 FS in question
+_fill_blk()
+{
+	local FSIZE
+	local BLKS
+	local NBLK
+	local FALLOC
+	local WS
+
+	for i in `find /$1 -type f`
+	do
+		FSIZE=`stat -t $i | cut -d" " -f2`
+		BLKS=`stat -c "%B" $i`
+		NBLK=`stat -c "%b" $i`
+		FALLOC=$(($BLKS * $NBLK))
+		WS=$(($FALLOC - $FSIZE))
+		dd if=/dev/urandom of=$i oseek=$FSIZE obs=$WS count=1 status=noxfer 2>/dev/null &
+	done
+	wait $!
+}
+
+
+# Append a random size to the files
+# arg1 : FS in question
+_append_file()
+{
+	local FSIZE
+	local X
+	local N
+	local i
+	N=0
+	for i in `find $1 -type f`
+	do
+		if [ $N == 0 ]; then
+			X=$i
+			FSIZE=`stat -t $X | cut -d" " -f2`
+			dd if=$X of=$X seek=1 bs=$FSIZE obs=$FSIZE count=1 status=noxfer 2>/dev/null &
+			N=$(($N+1))
+			continue
+		fi
+		FSIZE=`stat -t $i | cut -d" " -f2`
+		dd if=$X of=$i seek=1 bs=$FSIZE obs=$FSIZE count=1 status=noxfer 2>/dev/null &
+		X=$i
+	done
+	wait $!
+}
+
+##################### real QA test starts here###################################
+# sv1 - is just a name nothing spl
+firstvol="$SCRATCH_MNT/sv1"
+btrfs subvolume create $firstvol > /dev/null || _fail "btrfs subvolume create $firstvol failed"
+_populate_fs -n 1 -f 20 -d 10 -r $firstvol -s 10 -x
+SNAPNAME=0
+_create_snap $firstvol
+_save_checksum $firstvol $tmp.sv1.sum
+_verify_checksum $SNAPNAME $tmp.sv1.sum
+
+#Append1 the files
+_fill_blk $SNAPNAME
+_verify_checksum $firstvol $tmp.sv1.sum
+
+#Append2 the files
+_append_file $SNAPNAME
+_verify_checksum $firstvol $tmp.sv1.sum
+
+#read modify write
+_read_modify_write $SNAPNAME
+_verify_checksum $firstvol $tmp.sv1.sum
+
+#nested snapshot test
+src_vol=$firstvol
+for i in `seq 1 7`; do
+	SNAPNAME=0
+	_create_snap $src_vol
+	_verify_checksum $SNAPNAME $tmp.sv1.sum
+	src_vol=$SNAPNAME
+done
+
+# file delete test
+SNAPNAME=0
+_create_snap $firstvol
+tname=`echo $SNAPNAME | rev | cut -d"/" -f1 | rev`
+_save_checksum $SNAPNAME $tmp.$tname.sum
+\rm -rf $firstvol/*
+_verify_checksum $SNAPNAME $tmp.$tname.sum
+
+umount $SCRATCH_DEV || _fail "unmount failed"
+
+echo "Silence is golden"
+status=0; exit

264.out new file +2 -0

new file mode 100644
index 0000000..7a26e35
@@ -0,0 +1,2 @@
+QA output created by 263
+Silence is golden

README +6 -1

index 5367be6..7c135c7 100644
@@ -36,12 +36,17 @@ Preparing system for tests (IRIX and Linux):
               not be run.
               
         (these must be two DIFFERENT partitions)
+
+    - for btrfs only: some tests would need 3 or more independent SCRATCH disks,
+      which should be setenv SCRATCH_DEV_POOL instead of SCRATCH_DEV
+              
               
     - setup your environment
         - setenv TEST_DEV "device containing TEST PARTITION"
         - setenv TEST_DIR "mount point of TEST PARTITION"   
        	- optionally:
              - setenv SCRATCH_DEV "device containing SCRATCH PARTITION"
+             - setenv SCRATCH_DEV_POOL "pool of SCRATCH disks for testing btrfs"
              - setenv SCRATCH_MNT "mount point for SCRATCH PARTITION"
              - setenv TAPE_DEV "tape device for testing xfsdump"
              - setenv RMT_TAPE_DEV "remote tape device for testing xfsdump"

@@ -63,7 +68,7 @@ Preparing system for tests (IRIX and Linux):
       tape which can be overwritten.
           
     - make sure $TEST_DEV is a mounted XFS partition
-    - make sure that $SCRATCH_DEV contains nothing useful
+    - make sure that $SCRATCH_DEV or $SCRATCH_DEV_POOL contains nothing useful
     
 Running tests:

common.config +14 -0

index e94624e..4212fc8 100644
@@ -229,6 +229,20 @@ if [ ! -d "$TEST_DIR" ]; then
     exit 1
 fi
 
+# a btrfs tester will set only SCRATCH_DEV_POOL, we will put first of its dev
+# to SCRATCH_DEV and rest to SCRATCH_DEV_POOL to maintain the backward compatibility
+if [ "$HOSTOS" == "Linux" ]; then
+    FSTYP_tmp=`blkid -c /dev/null -s TYPE -o value $TEST_DEV`
+else
+    FSTYP_tmp=xfs
+fi
+if [ "$FSTYP_tmp" == "btrfs" ]; then
+    if [ ! -z "$SCRATCH_DEV_POOL" ]; then
+	SCRATCH_DEV=`echo $SCRATCH_DEV_POOL | cut -d" " -f 1`
+	SCRATCH_DEV_POOL=`echo $SCRATCH_DEV_POOL | cut -d" " -f 2-`
+    fi
+fi
+	
 echo $SCRATCH_DEV | grep -q ":" > /dev/null 2>&1
 if [ ! -z "$SCRATCH_DEV" -a ! -b "$SCRATCH_DEV" -a "$?" != "0" ]; then
     echo "common.config: Error: \$SCRATCH_DEV ($SCRATCH_DEV) is not a block device or a NFS filesystem"

common.rc +91 -4

132 lines not on this page · raw message

group +1 -0

index 2a8970c..cfbae8c 100644
@@ -377,3 +377,4 @@ deprecated
 261 auto quick quota
 262 auto quick quota
 263 rw auto quick
+264 auto quick

Signature · 1 line

1.7.7

Christoph Hellwig

2011-10-19 09:42 UTC #27 lore raw

Re: [PATCH 1/3] 264: Functional test case for the btrfs snapshot

Quoted 2 lines · Anand Jain

Create snapshots in various ways, modify the data around the block and file boundaries and verify the data integrity.

The test itselt looks good enough, but I have some comments on the pool infrastructure changes. I also think they should probably be a separate preparatory patch, or at least documented in the changelog as well.

Quoted 8 lines

index 5367be6..7c135c7 100644

README +1 -0

@@ -36,12 +36,17 @@ Preparing system for tests (IRIX and Linux):
               not be run.
 
         (these must be two DIFFERENT partitions)
+

README Preparing system for tests (IRIX and Linux): · line 49

Quoted 10 lines

README +4 -0

+    - for btrfs only: some tests would need 3 or more independent SCRATCH disks,
+      which should be setenv SCRATCH_DEV_POOL instead of SCRATCH_DEV
+              
 
     - setup your environment
         - setenv TEST_DEV "device containing TEST PARTITION"
         - setenv TEST_DIR "mount point of TEST PARTITION"   
        	- optionally:
              - setenv SCRATCH_DEV "device containing SCRATCH PARTITION"
+             - setenv SCRATCH_DEV_POOL "pool of SCRATCH disks for testing btrfs"

How does one find out what the pool name is? You'll also need to document how to create the pool from disks.

README if [ ! -d "$TEST_DIR" ]; then · line 238

Quoted 11 lines

README +7 -0

@@ -229,6 +229,20 @@ if [ ! -d "$TEST_DIR" ]; then
     exit 1
 fi
 
+# a btrfs tester will set only SCRATCH_DEV_POOL, we will put first of its dev
+# to SCRATCH_DEV and rest to SCRATCH_DEV_POOL to maintain the backward compatibility
+if [ "$HOSTOS" == "Linux" ]; then
+    FSTYP_tmp=`blkid -c /dev/null -s TYPE -o value $TEST_DEV`
+else
+    FSTYP_tmp=xfs
+fi

Why do we need a second FSTYP detection? If the existing one isn't early enough make sure it's done early enough instead of duplicating it.

common.rc _nfiles() · line 1505

Quoted 12 lines

common.rc +5 -1

@@ -1498,7 +1498,11 @@ _nfiles()
                 file=f$f
                 echo > $file
                 if [ $size -gt 0 ]; then
-                    dd if=/dev/zero of=$file bs=1024 count=$size
+		    if [ $randomdata == false ]; then
+	                dd if=/dev/zero of=$file bs=1024 count=$size 2>&1 | _filter_dd
+		    else
+                    	dd if=/dev/urandom of=$file bs=1024 count=$size 2>&1 | _filter_dd
+		    fi

I'd rather see the randomdata flag passed down explicitly to _descend and _nfiles rather than setting a magic environment variable.

common.rc _nfiles() · line 1519

Quoted 10 lines

common.rc +5 -1

@@ -1508,7 +1512,11 @@ _nfiles()
 _descend()
 {
         dirname=$1; depth=$2
-        mkdir $dirname  || die "mkdir $dirname failed"
+	if [ -d $dirname ]; then
+		dirname=`mktemp -dq $dirname/dir.XXXXXX`
+	else
+	        mkdir $dirname  || die "mkdir $dirname failed"
+	fi

Why would the directory here already exist? This at least needs very good documentation. Also the indentation seems off compared to the surrounding code.

common.rc _populate_fs() · line 1562

Quoted 5 lines

common.rc +1 -0

@@ -1550,6 +1559,7 @@ _populate_fs()
         s)      size=$OPTARG;;
         v)      verbose=true;;
         r)      root=$OPTARG;;
+	x)	randomdata=true;;

indendation is off again.

common.rc _test_inode_extsz() · line 1611

Quoted 8 lines

common.rc +8 -0

+# scratch_dev_pool should contain the disks pool for the btrfs raid
+_require_scratch_dev_pool()
+{
+	local i
+	case "$FSTYP" in
+	btrfs)
+		if [ -z "$SCRATCH_DEV_POOL" ]
+		then

For new code I'd generally prefer the more readable

		if [ ... ]; then

although the above form unfortunately still is fairly common in xfsprogs.

Quoted 24 lines

common.rc +24 -0

+# We will check if the device is virtual (eg: loop device) since it does not
+# have the delete entry-point. Otherwise SCSI and USB devices are fine. 
+_require_deletable_scratch_dev_pool()
+{
+	local i
+	local x
+	for i in $SCRATCH_DEV_POOL; do
+		x=`echo $i | cut -d"/" -f 3`
+		ls -l /sys/class/block/${x} | grep -q "virtual" 
+		if [ $? == "0" ]; then
+			_notrun "$i is a virtual device which is not deletable"
+		fi
+	done
+}
+
+# arg 1 is dev to remove and is output of the below eg.
+# ls -l /sys/class/block/sdd | rev | cut -d "/" -f 3 | rev
+_devmgt_remove()
+{
+	echo 1 > /sys/class/scsi_device/${1}/device/delete || _fail "Remove disk failed"
+}
+
+# arg 1 is dev to add and is output of the below eg.
+# ls -l /sys/class/block/sdd | rev | cut -d "/" -f 3 | rev

common.rc _test_inode_extsz() · line 1680

Quoted 10 lines

common.rc +10 -0

+_devmgt_add()
+{
+	local h
+	local tdl
+	# arg 1 will be in h:t:d:l format now in the h and "t d l" format
+	h=`echo ${1} | cut -d":" -f 1`
+	tdl=`echo ${1} | cut -d":" -f 2-|sed 's/:/ /g'`
+
+	echo ${tdl} >  /sys/class/scsi_host/host${h}/scan || _fail "Add disk failed"
+}

This code looks a bit fragile to me, but I think we can fix it on the go if we encouter issues.

group deprecated · line 380

Quoted 9 lines

group +1 -0

index 2a8970c..cfbae8c 100644
@@ -377,3 +377,4 @@ deprecated
 261 auto quick quota
 262 auto quick quota
 263 rw auto quick
+264 auto quick

It might be worth to add pool or snaphot groups if you add more tests like this.

Anand Jain

author

2011-10-20 15:31 UTC #28 lore raw

Re: [PATCH 1/3] 264: Functional test case for the btrfs snapshot

comments in line.

Quoted 21 lines · Christoph Hellwig

Quoted 2 lines · Anand Jain
Create snapshots in various ways, modify the data around the block and file boundaries and verify the data integrity.
The test itselt looks good enough, but I have some comments on the pool infrastructure changes. I also think they should probably be a separate preparatory patch, or at least documented in the changelog as well.
Quoted 12 lines
index 5367be6..7c135c7 100644
README +4 -0
@@ -36,12 +36,17 @@ Preparing system for tests (IRIX and Linux):
               not be run.
 
         (these must be two DIFFERENT partitions)
+
+    - for btrfs only: some tests would need 3 or more independent SCRATCH disks,
+      which should be setenv SCRATCH_DEV_POOL instead of SCRATCH_DEV
+              
 

Quoted 10 lines

README Preparing system for tests (IRIX and Linux): · line 49

Quoted 6 lines

README +1 -0

     - setup your environment
         - setenv TEST_DEV "device containing TEST PARTITION"
         - setenv TEST_DIR "mount point of TEST PARTITION"   
        	- optionally:
              - setenv SCRATCH_DEV "device containing SCRATCH PARTITION"
+             - setenv SCRATCH_DEV_POOL "pool of SCRATCH disks for testing btrfs"

How does one find out what the pool name is? You'll also need to document how to create the pool from disks.

agreed.

Quoted 4 lines

README +0 -0

@@ -229,6 +229,20 @@ if [ ! -d "$TEST_DIR" ]; then
     exit 1
 fi

Quoted 11 lines

README if [ ! -d "$TEST_DIR" ]; then · line 238
Quoted 7 lines
README +7 -0
+# a btrfs tester will set only SCRATCH_DEV_POOL, we will put first of its dev
+# to SCRATCH_DEV and rest to SCRATCH_DEV_POOL to maintain the backward compatibility
+if [ "$HOSTOS" == "Linux" ]; then
+    FSTYP_tmp=`blkid -c /dev/null -s TYPE -o value $TEST_DEV`
+else
+    FSTYP_tmp=xfs
+fi
Why do we need a second FSTYP detection? If the existing one isn't early enough make sure it's done early enough instead of duplicating it.

looks like its ok not to have FSTYP checked here, it will follow the following logic..

btrfs FS OR any FS
SCRATCH_DEV_POOL is unset and SCRATCH_DEV is set
. test-case with _require_scratch_dev_pool will not run . test-case without _require_scratch_dev_pool will run

SCRATCH_DEV_POOL is set and SCRATCH_DEV is unset
. test-case with _require_scratch_dev_pool
- runs only if FSTYP=btrfs
. test-case without _require_scratch_dev_pool will run using first
dev in the SCRATCH_DEV_POOL as a SCRATCH_DEV
- if FSTYP=btrfs it includes SCRATCH_DEV_POOL disks to the FS
- if FSTYP=non-btrfs SCRATCH_DEV_POOL is ignored

SCRATCH_DEV_POOL is set and SCRATCH_DEV is set
. reports error in the config

SCRATCH_DEV_POOL is unset and SCRATCH_DEV is unset
. no change

Quoted 7 lines

common.rc +0 -1

@@ -1498,7 +1498,11 @@ _nfiles()
                 file=f$f
                 echo > $file
                 if [ $size -gt 0 ]; then
-                    dd if=/dev/zero of=$file bs=1024 count=$size

Quoted 8 lines

common.rc _nfiles() · line 1505
Quoted 5 lines
common.rc +5 -0
+		    if [ $randomdata == false ]; then
+	                dd if=/dev/zero of=$file bs=1024 count=$size 2>&1 | _filter_dd
+		    else
+                    	dd if=/dev/urandom of=$file bs=1024 count=$size 2>&1 | _filter_dd
+		    fi
I'd rather see the randomdata flag passed down explicitly to _descend and _nfiles rather than setting a magic environment variable.

makes sense. added.

Quoted 6 lines

common.rc +1 -1

@@ -1508,7 +1512,11 @@ _nfiles()
 _descend()
 {
         dirname=$1; depth=$2
-        mkdir $dirname  || die "mkdir $dirname failed"
+	if [ -d $dirname ]; then

Quoted 8 lines

common.rc _nfiles() · line 1519
Quoted 4 lines
common.rc +4 -0
+		dirname=`mktemp -dq $dirname/dir.XXXXXX`
+	else
+	        mkdir $dirname  || die "mkdir $dirname failed"
+	fi
Why would the directory here already exist? This at least needs very good documentation. Also the indentation seems off compared to the surrounding code.

Hmm. Changing this back to the original design - where calling function has to ensure a new directory name which does not exists.

Quoted 7 lines

common.rc _populate_fs() · line 1562

Quoted 5 lines

common.rc +1 -0

@@ -1550,6 +1559,7 @@ _populate_fs()
         s)      size=$OPTARG;;
         v)      verbose=true;;
         r)      root=$OPTARG;;
+	x)	randomdata=true;;

indendation is off again.

oh! thks for pointing.

Quoted 11 lines

common.rc _test_inode_extsz() · line 1611
Quoted 8 lines
common.rc +8 -0
+# scratch_dev_pool should contain the disks pool for the btrfs raid
+_require_scratch_dev_pool()
+{
+	local i
+	case "$FSTYP" in
+	btrfs)
+		if [ -z "$SCRATCH_DEV_POOL" ]
+		then
For new code I'd generally prefer the more readable

Quoted 4 lines

		if [ ... ]; then
although the above form unfortunately still is fairly common in xfsprogs.

got it.

Quoted 29 lines

common.rc +29 -0

+# We will check if the device is virtual (eg: loop device) since it does not
+# have the delete entry-point. Otherwise SCSI and USB devices are fine. 
+_require_deletable_scratch_dev_pool()
+{
+	local i
+	local x
+	for i in $SCRATCH_DEV_POOL; do
+		x=`echo $i | cut -d"/" -f 3`
+		ls -l /sys/class/block/${x} | grep -q "virtual" 
+		if [ $? == "0" ]; then
+			_notrun "$i is a virtual device which is not deletable"
+		fi
+	done
+}
+
+# arg 1 is dev to remove and is output of the below eg.
+# ls -l /sys/class/block/sdd | rev | cut -d "/" -f 3 | rev
+_devmgt_remove()
+{
+	echo 1 > /sys/class/scsi_device/${1}/device/delete || _fail "Remove disk failed"
+}
+
+# arg 1 is dev to add and is output of the below eg.
+# ls -l /sys/class/block/sdd | rev | cut -d "/" -f 3 | rev
+_devmgt_add()
+{
+	local h
+	local tdl
+	# arg 1 will be in h:t:d:l format now in the h and "t d l" format

Quoted 8 lines

common.rc _test_inode_extsz() · line 1680
Quoted 5 lines
common.rc +5 -0
+	h=`echo ${1} | cut -d":" -f 1`
+	tdl=`echo ${1} | cut -d":" -f 2-|sed 's/:/ /g'`
+
+	echo ${tdl} >  /sys/class/scsi_host/host${h}/scan || _fail "Add disk failed"
+}
This code looks a bit fragile to me, but I think we can fix it on the go if we encouter issues.

ok.

Quoted 4 lines

Quoted 4 lines
group +0 -0
index 2a8970c..cfbae8c 100644

Quoted 8 lines

group deprecated · line 380
Quoted 5 lines
group +1 -0
@@ -377,3 +377,4 @@ deprecated
 261 auto quick quota
 262 auto quick quota
 263 rw auto quick
+264 auto quick
It might be worth to add pool or snaphot groups if you add more tests like this.

this can be considered when appropriate I think, as of now there is one test case in each of them.
Thanks

Anand

Anand Jain

author

2011-10-18 06:26 UTC #29 lore raw

PATCH 2/3 265: Functional test case for the btrfs de-fragmentation

To verify the btrfs de-fragmentation does not fail.

Signed-off-by Anand Jain <[email protected]>

---
 265     |   60 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 265.out |    2 ++
 group   |    1 +
 3 files changed, 63 insertions(+), 0 deletions(-)
 create mode 100755 265
 create mode 100644 265.out

265 new file +60 -0

new file mode 100755
index 0000000..81ec69d
@@ -0,0 +1,60 @@
+#! /bin/bash
+# FS QA Test No. 265
+#
+# btrfs defragmentation tests
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2011 Oracle.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+# creator
[email protected]
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+    cd /
+    rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_need_to_be_root
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+
+_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs failed"
+_scratch_mount
+
+_populate_fs -n 1 -f 20 -d 10 -r $SCRATCH_MNT -s 10
+
+btrfs filesystem defragment $SCRATCH_MNT
+[ $? = 20 ] || _fail "Defragment failed"
+
+umount $SCRATCH_MNT || _fail "umount failed"
+
+echo "Silence is golden"
+status=0; exit

265.out new file +2 -0

new file mode 100644
index 0000000..9fa4eb0
@@ -0,0 +1,2 @@
+QA output created by 265
+Silence is golden

group +1 -0

index cfbae8c..6bac8b5 100644
@@ -378,3 +378,4 @@ deprecated
 262 auto quick quota
 263 rw auto quick
 264 auto quick
+265 auto quick

Signature · 1 line

1.7.7

Christoph Hellwig

2011-10-19 09:43 UTC #30 lore raw

Re: [PATCH 2/3] 265: Functional test case for the btrfs de-fragmentation

Quoted 1 line · Anand Jain

To verify the btrfs de-fragmentation does not fail.

Any reason you can't simply fold btrfs defragmentation testing into the existing common defragmentation test (218)?

Anand Jain

author

2011-10-20 15:32 UTC #31 lore raw

Re: [PATCH 2/3] 265: Functional test case for the btrfs de-fragmentation

Quoted 5 lines · Christoph Hellwig

Quoted 1 line · Anand Jain
To verify the btrfs de-fragmentation does not fail.
Any reason you can't simply fold btrfs defragmentation testing into the existing common defragmentation test (218)?

oh! 218 ! let me defer defrag for now. will get the other test-cases going. thanks
anand

Anand Jain

author

2011-10-18 06:26 UTC #32 lore raw

PATCH 3/3 266: Functional test case for the btrfs raid operations

This will verify the various raid features in btrfs and device replacement functionality.

Signed-off-by Anand Jain <[email protected]>

---
 266     |  187 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 266.out |    2 +
 group   |    1 +
 3 files changed, 190 insertions(+), 0 deletions(-)
 create mode 100755 266
 create mode 100644 266.out

266 new file +187 -0

new file mode 100755
index 0000000..3e70630
@@ -0,0 +1,187 @@
+#! /bin/bash
+# FS QA Test No. 266
+#
+# btrfs vol tests
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2011 Oracle.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+# creator
[email protected]
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+dev_removed=0
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+    cd /
+    rm -f $tmp.*
+    if [ $dev_removed == 1 ]; then
+	umount $SCRATCH_MNT
+        _devmgt_add "${DEVHTL}"
+    fi
+}
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_need_to_be_root
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+_require_scratch_dev_pool
+_require_deletable_scratch_dev_pool
+
+# Test cases related to raid in btrfs
+_test_raid0()
+{
+	export MKFS_OPTIONS="-m raid0 -d raid0"
+	_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs failed"
+	_scratch_mount
+	_populate_fs -n 1 -f 20 -d 10 -r $SCRATCH_MNT -s 10
+	umount $SCRATCH_MNT
+}
+
+_test_raid1()
+{
+	export MKFS_OPTIONS="-m raid1 -d raid1"
+	_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs failed"
+	_scratch_mount
+	_populate_fs -n 1 -f 20 -d 10 -r $SCRATCH_MNT -s 10
+	umount $SCRATCH_MNT
+}
+
+_test_raid10()
+{
+	export MKFS_OPTIONS="-m raid10 -d raid10"
+	_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs failed"
+	_scratch_mount
+	_populate_fs -n 1 -f 20 -d 10 -r $SCRATCH_MNT -s 10
+	umount $SCRATCH_MNT
+}
+
+_test_single()
+{
+	export MKFS_OPTIONS="-m single -d single"
+	_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs failed"
+	_scratch_mount
+	_populate_fs -n 1 -f 20 -d 10 -r $SCRATCH_MNT -s 10
+	umount $SCRATCH_MNT
+}
+
+_test_add()
+{
+	local i
+	local devs[]="( $SCRATCH_DEV_POOL )"
+	local n=${#devs[@]}
+
+	n=$(($n-1))
+
+	export MKFS_OPTIONS=""
+	_scratch_mkfs > /dev/null 2>&1 || _fail "mkfs failed"
+	_scratch_mount
+	_populate_fs -n 1 -f 20 -d 10 -r $SCRATCH_MNT -s 10
+	for i in `seq 1 $n`
+	do
+		btrfs device add ${devs[$i]} $SCRATCH_MNT > /dev/null 2>&1 || _fail "device add failed"
+	done
+	btrfs filesystem balance $SCRATCH_MNT || _fail "balance failed"
+	#btrfs filesystem show $SCRATCH_DEV 2>&1 | egrep devid |awk '{
+	#	if ( $6 == "0.00" ) { exit 1 }
+	#}' || _fail "test6_add... failed"
+	umount $SCRATCH_MNT
+}
+
+_test_replace()
+{
+	local i
+	local devs=( $SCRATCH_DEV_POOL )
+	local n=${#devs[@]}
+	local ds
+	local d
+	local DEVHTL=""
+
+	# exclude the last disk in the disk pool
+	n=$(($n-1))
+	ds=${devs[@]:0:$n}
+
+	export MKFS_OPTIONS="-m raid1 -d raid1"
+	_scratch_mkfs "$ds" > /dev/null 2>&1 || _fail "tr: mkfs failed"
+	_scratch_mount
+	_populate_fs -n 1 -f 20 -d 10 -r $SCRATCH_MNT -s 10
+
+	#pick the 2nd last disk 
+	ds=${devs[@]:$(($n-1)):1}
+
+	# retrive the HTL for this scsi disk
+	d=`echo $ds|cut -d"/" -f3`
+	DEVHTL=`ls -l /sys/class/block/${d} | rev | cut -d "/" -f 3 | rev`
+
+	#fail disk
+	_devmgt_remove ${DEVHTL}
+	dev_removed=1
+
+	btrfs fi show $SCRATCH_DEV | grep "Some devices missing" > /dev/null || _fail \
+							"btrfs did not report device missing"
+
+	# add a new disk to btrfs
+	ds=${devs[@]:$(($n)):1}
+	btrfs device add ${ds} $SCRATCH_MNT > /dev/null 2>&1 || _fail "dev add failed"
+	# in some system balance fails if there is no delay (a bug)
+	# putting sleep 10 to work around as of now
+	# sleep 10
+	btrfs fi balance $SCRATCH_MNT || _fail "dev balance failed"
+	#btrfs filesystem show | egrep devid |awk '{
+	#	if ( $6 == "0.00" ) { exit 1 }
+	#}' || _fail "btrfs balance failed"
+
+	# cleaup. add the removed disk
+	umount $SCRATCH_MNT
+	_devmgt_add "${DEVHTL}"
+	dev_removed=0
+}
+
+_test_remove()
+{
+	_scratch_mkfs "$SCRATCH_DEV_POOL" > /dev/null 2>&1 || _fail "mkfs failed"
+	_scratch_mount
+	_populate_fs -n 1 -f 20 -d 10 -r $SCRATCH_MNT -s 10
+
+	# pick last dev in the list
+	dev_del=`echo ${SCRATCH_DEV_POOL} | awk '{print $NF}'`
+	btrfs device delete $dev_del $SCRATCH_MNT || _fail "btrfs device delete failed"
+	btrfs fi show $SCRATCH_DEV 2>&1 | grep $dev_del > /dev/null && _fail "btrfs still shows the deleted dev"
+	umount $SCRATCH_MNT
+}
+
+_test_raid0
+_test_raid1
+_test_raid10
+_test_single
+_test_add
+_test_replace
+_test_remove
+
+echo "Silence is golden"
+status=0; exit

266.out new file +2 -0

new file mode 100644
index 0000000..54ba60f
@@ -0,0 +1,2 @@
+QA output created by 266
+Silence is golden

group +1 -0

index 6bac8b5..817d245 100644
@@ -379,3 +379,4 @@ deprecated
 263 rw auto quick
 264 auto quick
 265 auto quick
+266 auto

Signature · 1 line

1.7.7

Christoph Hellwig

2011-10-19 09:45 UTC #33 lore raw

Re: [PATCH 3/3] 266: Functional test case for the btrfs raid operations

266 line 42

Quoted 4 lines

266 new file +4 -0

+    if [ $dev_removed == 1 ]; then
+	umount $SCRATCH_MNT
+        _devmgt_add "${DEVHTL}"
+    fi

messy indendation.

266 line 112

Quoted 4 lines

266 new file +4 -0

+	btrfs filesystem balance $SCRATCH_MNT || _fail "balance failed"
+	#btrfs filesystem show $SCRATCH_DEV 2>&1 | egrep devid |awk '{
+	#	if ( $6 == "0.00" ) { exit 1 }
+	#}' || _fail "test6_add... failed"

what is the point of the commented out code?

otherwise looks good to me.

Anand Jain

author

2011-10-20 15:32 UTC #34 lore raw

Re: [PATCH 3/3] 266: Functional test case for the btrfs raid operations

Quoted 6 lines · Christoph Hellwig

266 line 42
Quoted 4 lines
266 new file +4 -0
+    if [ $dev_removed == 1 ]; then
+	umount $SCRATCH_MNT
+        _devmgt_add "${DEVHTL}"
+    fi
messy indendation.

got it. thanks.

Quoted 8 lines

266 line 112
Quoted 4 lines
266 new file +4 -0
+	btrfs filesystem balance $SCRATCH_MNT || _fail "balance failed"
+	#btrfs filesystem show $SCRATCH_DEV 2>&1 | egrep devid |awk '{
+	#	if ( $6 == "0.00" ) { exit 1 }
+	#}' || _fail "test6_add... failed"
what is the point of the commented out code?
otherwise looks good to me.

yes that has to be removed.

Anand Jain

author

2011-10-20 15:37 UTC #35 lore raw

cover xfstests enhancement and bug fix

Anand Jain (5):
updating to fill files with random data
Added SCRATCH_DEV_POOL to specify multiple disks for the btrfs RAID 264: Functional test case for the btrfs snapshot
265: Functional test case for the btrfs raid operations _populate_fs should use OPTIND when getopts is used

 264           |  193 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 264.out       |    2 +
 265           |  187 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 265.out       |    2 +
 README        |   16 ++++-
 common.config |   11 +++
 common.rc     |   96 ++++++++++++++++++++++++++--
 group         |    2 +
 8 files changed, 498 insertions(+), 11 deletions(-)
 create mode 100755 264
 create mode 100644 264.out
 create mode 100755 265
 create mode 100644 265.out

Signature · 1 line

1.7.7

Anand Jain

author

2011-10-20 15:38 UTC #36 lore raw

PATCH 1/5 fill files with random data

snapshot data integrity test-case needs filesystem with random data.

Signed-off-by Anand Jain <[email protected]>

---
 common.rc |   21 ++++++++++++++-------
 1 files changed, 14 insertions(+), 7 deletions(-)

common.rc +14 -7

index e948169..37379a5 100644
@@ -1490,6 +1490,7 @@ _die()
         exit 1
 }
 
+#takes files, randomdata
 _nfiles()
 {
         f=0

@@ -1498,20 +1499,24 @@ _nfiles()
                 file=f$f
                 echo > $file
                 if [ $size -gt 0 ]; then
-                    dd if=/dev/zero of=$file bs=1024 count=$size
+                    if [ $2 == false ]; then
+                        dd if=/dev/zero of=$file bs=1024 count=$size 2>&1 | _filter_dd
+                    else
+                        dd if=/dev/urandom of=$file bs=1024 count=$size 2>&1 | _filter_dd
+                    fi
                 fi
 		let f=$f+1
         done
 }
 
-# takes dirname, depth
+# takes dirname, depth, randomdata
 _descend()
 {
-        dirname=$1; depth=$2
+        dirname=$1; depth=$2; randomdata=$3
         mkdir $dirname  || die "mkdir $dirname failed"
         cd $dirname
 
-        _nfiles $files           # files for this dir
+        _nfiles $files $randomdata          # files for this dir and data type
 
         [ $depth -eq 0 ] && return
 	let deep=$depth-1 # go 1 down

@@ -1529,7 +1534,7 @@ _descend()
 
 # Populate a filesystem with inodes for performance experiments
 #
-# usage: populate [-v] [-n ndirs] [-f nfiles] [-d depth] [-r root] [-s size]
+# usage: populate [-v] [-n ndirs] [-f nfiles] [-d depth] [-r root] [-s size] [-x]
 #
 _populate_fs()
 {

@@ -1540,8 +1545,9 @@ _populate_fs()
     depth=2         # depth of tree from root to leaves
     verbose=false
     root=root       # path of initial root of directory tree
+    randomdata=false # -x data type urandom or zero
 
-    while getopts "d:f:n:r:s:v" c
+    while getopts "d:f:n:r:s:v:x" c
     do
         case $c in
         d)      depth=$OPTARG;;

@@ -1550,10 +1556,11 @@ _populate_fs()
         s)      size=$OPTARG;;
         v)      verbose=true;;
         r)      root=$OPTARG;;
+        x)      randomdata=true;;
         esac
     done
 
-    _descend $root $depth
+    _descend $root $depth $randomdata
     wait
 
     cd $here

Signature · 1 line

1.7.7

Christoph Hellwig

Reviewed-by on v1 1/5

2011-10-25 11:35 UTC #37 lore raw

Re: [PATCH 1/5] fill files with random data

Looks good,

Anand Jain

author

2011-10-20 15:38 UTC #38 lore raw

PATCH 2/5 Added SCRATCH_DEV_POOL to specify multiple disks for the btrfs RAID

SCRATCH_DEV takes single disk as the scratch place for testing. New SCRATCH_DEV_POOL can used to specify multiple disks for the scratch btrfs filesystem.

Using SCRATCH_DEV and or SCRATCH_DEV_POOL will follow the following logic.
btrfs FS OR any FS
SCRATCH_DEV_POOL is unset and SCRATCH_DEV is set
. test-case with _require_scratch_dev_pool will not run
. test-case without _require_scratch_dev_pool will run

SCRATCH_DEV_POOL is set and SCRATCH_DEV is set
. reports error in the config

SCRATCH_DEV_POOL is unset and SCRATCH_DEV is unset
. no change

Signed-off-by Anand Jain <[email protected]>

---
 README        |   16 +++++++++---
 common.config |   11 ++++++++
 common.rc     |   76 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 3 files changed, 98 insertions(+), 5 deletions(-)

README +12 -4

index 5367be6..0f52f02 100644
@@ -34,14 +34,22 @@ Preparing system for tests (IRIX and Linux):
             - leave empty and expect this partition to be clobbered
               by some tests.  If this is not provided, many tests will
               not be run.
-              
-        (these must be two DIFFERENT partitions)
+              (SCRATCH and TEST must be two DIFFERENT partitions)
+              OR
+        - for btrfs only: some btrfs test cases will need 3 or more independent
+              SCRATCH disks which should be set using SCRATCH_DEV_POOL (for eg:
+              SCRATCH_DEV_POOL="/dev/sda /dev/sdb /dev/sdc") with which
+              SCRATCH_DEV should be unused by the tester, and for the legacy
+              support SCRATCH_DEV will be set to the first disk of the
+              SCRATCH_DEV_POOL by xfstests script.
               
     - setup your environment
         - setenv TEST_DEV "device containing TEST PARTITION"
         - setenv TEST_DIR "mount point of TEST PARTITION"   
        	- optionally:
-             - setenv SCRATCH_DEV "device containing SCRATCH PARTITION"
+             - setenv SCRATCH_DEV "device containing SCRATCH PARTITION" OR
+               (btrfs only) setenv SCRATCH_DEV_POOL "to 3 or more SCRATCH disks for
+               testing btrfs raid concepts"
              - setenv SCRATCH_MNT "mount point for SCRATCH PARTITION"
              - setenv TAPE_DEV "tape device for testing xfsdump"
              - setenv RMT_TAPE_DEV "remote tape device for testing xfsdump"

@@ -63,7 +71,7 @@ Preparing system for tests (IRIX and Linux):
       tape which can be overwritten.
           
     - make sure $TEST_DEV is a mounted XFS partition
-    - make sure that $SCRATCH_DEV contains nothing useful
+    - make sure that $SCRATCH_DEV or $SCRATCH_DEV_POOL contains nothing useful
     
 Running tests:

common.config +11 -0

index e94624e..1df2bbd 100644
@@ -229,6 +229,17 @@ if [ ! -d "$TEST_DIR" ]; then
     exit 1
 fi
 
+# a btrfs tester will set only SCRATCH_DEV_POOL, we will put first of its dev
+# to SCRATCH_DEV and rest to SCRATCH_DEV_POOL to maintain the backward compatibility
+if [ ! -z "$SCRATCH_DEV_POOL" ]; then
+    if [ ! -z "$SCRATCH_DEV" ]; then
+        echo "common.config: Error: \$SCRATCH_DEV should be unset when \$SCRATCH_DEV_POOL is set"
+        exit 1
+    fi
+    SCRATCH_DEV=`echo $SCRATCH_DEV_POOL | awk '{print $1}'`
+    SCRATCH_DEV_POOL=`echo $SCRATCH_DEV_POOL | awk '{ for (i = 2; i <= NF; i++) print $i}'`
+fi
+
 echo $SCRATCH_DEV | grep -q ":" > /dev/null 2>&1
 if [ ! -z "$SCRATCH_DEV" -a ! -b "$SCRATCH_DEV" -a "$?" != "0" ]; then
     echo "common.config: Error: \$SCRATCH_DEV ($SCRATCH_DEV) is not a block device or a NFS filesystem"

common.rc +75 -1

88 lines not on this page · raw message

Signature · 1 line

1.7.7

Christoph Hellwig

Reviewed-by on v1 2/5

2011-10-25 11:36 UTC #39 lore raw

Re: [PATCH 2/5] Added SCRATCH_DEV_POOL to specify multiple disks for the btrfs RAID

Looks good,

Anand Jain

author

2011-10-20 15:38 UTC #40 lore raw

PATCH 3/5 264: Functional test case for the btrfs snapshot

Create snapshots in various ways, modify the data around the block and file boundaries and verify the data integrity.

Signed-off-by Anand Jain <[email protected]>

---
 264     |  193 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 264.out |    2 +
 group   |    1 +
 3 files changed, 196 insertions(+), 0 deletions(-)
 create mode 100755 264
 create mode 100644 264.out

264 new file +193 -0

new file mode 100755
index 0000000..d655d1e
@@ -0,0 +1,193 @@
+#!/bin/bash
+# FS QA Test No. 264
+#
+# Extented btrfs snapshot test cases
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2011 Oracle  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#
+#-----------------------------------------------------------------------
+#
+# creator
[email protected]
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+
+_cleanup()
+{
+    rm -f $tmp.*
+}
+
+trap "_cleanup ; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_need_to_be_root
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+
+_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs failed"
+_scratch_mount
+
+# Create and save sha256sum
+# arg1 FS to generate sha256
+# arg2 File name to save the sha256 output
+_save_checksum()
+{
+	local i=0
+	>$2
+	cd $1
+	for i in `find . -type f`; do sha256sum $i >> $2; done
+	cd $OLDPWD
+}
+
+# Verify the sha256sum for a FS
+# arg1 FS to be tested
+# arg2 sha256 file
+_verify_checksum()
+{
+	cd $1
+	[ -f $2 ] || _fail "checksum file $2 not found"
+	sha256sum -c $2 | grep "FAILED"
+	cd $OLDPWD
+}
+
+# Create a snapshot
+# arg1 dest dir
+# Return snapshot name in the SNAPNAME
+_create_snap()
+{
+	local x
+	[ -d $1 ] || _fail "Destination dir $1 not present"
+	SNAPNAME=`mktemp -u $SCRATCH_MNT/snap.XXXXXX`
+	btrfs subvolume snapshot $1 $SNAPNAME > /dev/null || _fail "snapshot create failed"
+}
+
+# Reads and writes new data but does not allocate new blocks
+# arg1 FS to be modified
+_read_modify_write()
+{
+	local i
+	local FSIZE
+	for i in `find $1 -type f`
+	do
+		FSIZE=`stat -t $i | cut -d" " -f2`
+		dd if=$i of=/dev/null obs=$FSIZE count=1 status=noxfer 2>/dev/null &
+		dd if=/dev/urandom of=$i obs=$FSIZE count=1 status=noxfer 2>/dev/null &
+	done
+	wait $!
+}
+
+# Fills the allocated blocks
+# arg1 FS in question
+_fill_blk()
+{
+	local FSIZE
+	local BLKS
+	local NBLK
+	local FALLOC
+	local WS
+
+	for i in `find /$1 -type f`
+	do
+		FSIZE=`stat -t $i | cut -d" " -f2`
+		BLKS=`stat -c "%B" $i`
+		NBLK=`stat -c "%b" $i`
+		FALLOC=$(($BLKS * $NBLK))
+		WS=$(($FALLOC - $FSIZE))
+		dd if=/dev/urandom of=$i oseek=$FSIZE obs=$WS count=1 status=noxfer 2>/dev/null &
+	done
+	wait $!
+}
+
+
+# Append a random size to the files
+# arg1 : FS in question
+_append_file()
+{
+	local FSIZE
+	local X
+	local N
+	local i
+	N=0
+	for i in `find $1 -type f`
+	do
+		if [ $N == 0 ]; then
+			X=$i
+			FSIZE=`stat -t $X | cut -d" " -f2`
+			dd if=$X of=$X seek=1 bs=$FSIZE obs=$FSIZE count=1 status=noxfer 2>/dev/null &
+			N=$(($N+1))
+			continue
+		fi
+		FSIZE=`stat -t $i | cut -d" " -f2`
+		dd if=$X of=$i seek=1 bs=$FSIZE obs=$FSIZE count=1 status=noxfer 2>/dev/null &
+		X=$i
+	done
+	wait $!
+}
+
+##################### real QA test starts here###################################
+# sv1 - is just a name nothing spl
+firstvol="$SCRATCH_MNT/sv1"
+btrfs subvolume create $firstvol > /dev/null || _fail "btrfs subvolume create $firstvol failed"
+dirp=`mktemp -duq $firstvol/dir.XXXXXX`
+_populate_fs -n 1 -f 20 -d 10 -r $dirp -s 10 -x
+SNAPNAME=0
+_create_snap $firstvol
+_save_checksum $firstvol $tmp.sv1.sum
+_verify_checksum $SNAPNAME $tmp.sv1.sum
+
+#Append1 the files
+_fill_blk $SNAPNAME
+_verify_checksum $firstvol $tmp.sv1.sum
+
+#Append2 the files
+_append_file $SNAPNAME
+_verify_checksum $firstvol $tmp.sv1.sum
+
+#read modify write
+_read_modify_write $SNAPNAME
+_verify_checksum $firstvol $tmp.sv1.sum
+
+#nested snapshot test
+src_vol=$firstvol
+for i in `seq 1 7`; do
+	SNAPNAME=0
+	_create_snap $src_vol
+	_verify_checksum $SNAPNAME $tmp.sv1.sum
+	src_vol=$SNAPNAME
+done
+
+# file delete test
+SNAPNAME=0
+_create_snap $firstvol
+tname=`echo $SNAPNAME | rev | cut -d"/" -f1 | rev`
+_save_checksum $SNAPNAME $tmp.$tname.sum
+\rm -rf $firstvol/*
+_verify_checksum $SNAPNAME $tmp.$tname.sum
+
+umount $SCRATCH_DEV || _fail "unmount failed"
+
+echo "Silence is golden"
+status=0; exit

264.out new file +2 -0

new file mode 100644
index 0000000..eca790b
@@ -0,0 +1,2 @@
+QA output created by 264
+Silence is golden

group +1 -0

index 2a8970c..d2abe42 100644
@@ -377,3 +377,4 @@ deprecated
 261 auto quick quota
 262 auto quick quota
 263 rw auto quick
+264 auto

Signature · 1 line

1.7.7

Christoph Hellwig

Reviewed-by on v1 3/5

2011-10-25 11:36 UTC #41 lore raw

Re: [PATCH 3/5] 264: Functional test case for the btrfs snapshot

Looks good,

Anand Jain

author

2011-10-20 15:38 UTC #42 lore raw

PATCH 4/5 265: Functional test case for the btrfs raid operations

This will verify the various raid features in btrfs and device replacement functionality.

Signed-off-by Anand Jain <[email protected]>

---
 265     |  187 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 265.out |    2 +
 group   |    1 +
 3 files changed, 190 insertions(+), 0 deletions(-)
 create mode 100755 265
 create mode 100644 265.out

265 new file +187 -0

new file mode 100755
index 0000000..10f214f
@@ -0,0 +1,187 @@
+#! /bin/bash
+# FS QA Test No. 265
+#
+# btrfs vol tests
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2011 Oracle.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+# creator
[email protected]
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+dev_removed=0
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+    cd /
+    rm -f $tmp.*
+    if [ $dev_removed == 1 ]; then
+        umount $SCRATCH_MNT
+        _devmgt_add "${DEVHTL}"
+    fi
+}
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_need_to_be_root
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+_require_scratch_dev_pool
+_require_deletable_scratch_dev_pool
+
+# Test cases related to raid in btrfs
+_test_raid0()
+{
+	export MKFS_OPTIONS="-m raid0 -d raid0"
+	_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs failed"
+	_scratch_mount
+	dirp=`mktemp -duq $SCRATCH_MNT/dir.XXXXXX`
+	_populate_fs -n 1 -f 20 -d 10 -r $dirp -s 10
+	umount $SCRATCH_MNT
+}
+
+_test_raid1()
+{
+	export MKFS_OPTIONS="-m raid1 -d raid1"
+	_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs failed"
+	_scratch_mount
+	dirp=`mktemp -duq $SCRATCH_MNT/dir.XXXXXX`
+	_populate_fs -n 1 -f 20 -d 10 -r $dirp -s 10
+	umount $SCRATCH_MNT
+}
+
+_test_raid10()
+{
+	export MKFS_OPTIONS="-m raid10 -d raid10"
+	_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs failed"
+	_scratch_mount
+	dirp=`mktemp -duq $SCRATCH_MNT/dir.XXXXXX`
+	_populate_fs -n 1 -f 20 -d 10 -r $dirp -s 10
+	umount $SCRATCH_MNT
+}
+
+_test_single()
+{
+	export MKFS_OPTIONS="-m single -d single"
+	_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs failed"
+	_scratch_mount
+	dirp=`mktemp -duq $SCRATCH_MNT/dir.XXXXXX`
+	_populate_fs -n 1 -f 20 -d 10 -r $dirp -s 10
+	umount $SCRATCH_MNT
+}
+
+_test_add()
+{
+	local i
+	local devs[]="( $SCRATCH_DEV_POOL )"
+	local n=${#devs[@]}
+
+	n=$(($n-1))
+
+	export MKFS_OPTIONS=""
+	_scratch_mkfs > /dev/null 2>&1 || _fail "mkfs failed"
+	_scratch_mount
+	dirp=`mktemp -duq $SCRATCH_MNT/dir.XXXXXX`
+	_populate_fs -n 1 -f 20 -d 10 -r $dirp -s 10
+	for i in `seq 1 $n`; do
+		btrfs device add ${devs[$i]} $SCRATCH_MNT > /dev/null 2>&1 || _fail "device add failed"
+	done
+	btrfs filesystem balance $SCRATCH_MNT || _fail "balance failed"
+	umount $SCRATCH_MNT
+}
+
+_test_replace()
+{
+	local i
+	local devs=( $SCRATCH_DEV_POOL )
+	local n=${#devs[@]}
+	local ds
+	local d
+	local DEVHTL=""
+
+	# exclude the last disk in the disk pool
+	n=$(($n-1))
+	ds=${devs[@]:0:$n}
+
+	export MKFS_OPTIONS="-m raid1 -d raid1"
+	_scratch_mkfs "$ds" > /dev/null 2>&1 || _fail "tr: mkfs failed"
+	_scratch_mount
+	dirp=`mktemp -duq $SCRATCH_MNT/dir.XXXXXX`
+	_populate_fs -n 1 -f 20 -d 10 -r $dirp -s 10
+
+	#pick the 2nd last disk 
+	ds=${devs[@]:$(($n-1)):1}
+
+	# retrive the HTL for this scsi disk
+	d=`echo $ds|cut -d"/" -f3`
+	DEVHTL=`ls -l /sys/class/block/${d} | rev | cut -d "/" -f 3 | rev`
+
+	#fail disk
+	_devmgt_remove ${DEVHTL}
+	dev_removed=1
+
+	btrfs fi show $SCRATCH_DEV | grep "Some devices missing" > /dev/null || _fail \
+							"btrfs did not report device missing"
+
+	# add a new disk to btrfs
+	ds=${devs[@]:$(($n)):1}
+	btrfs device add ${ds} $SCRATCH_MNT > /dev/null 2>&1 || _fail "dev add failed"
+	# in some system balance fails if there is no delay (a bug)
+	# putting sleep 10 to work around as of now
+	# sleep 10
+	btrfs fi balance $SCRATCH_MNT || _fail "dev balance failed"
+
+	# cleaup. add the removed disk
+	umount $SCRATCH_MNT
+	_devmgt_add "${DEVHTL}"
+	dev_removed=0
+}
+
+_test_remove()
+{
+	_scratch_mkfs "$SCRATCH_DEV_POOL" > /dev/null 2>&1 || _fail "mkfs failed"
+	_scratch_mount
+	dirp=`mktemp -duq $SCRATCH_MNT/dir.XXXXXX`
+	_populate_fs -n 1 -f 20 -d 10 -r $dirp -s 10
+
+	# pick last dev in the list
+	dev_del=`echo ${SCRATCH_DEV_POOL} | awk '{print $NF}'`
+	btrfs device delete $dev_del $SCRATCH_MNT || _fail "btrfs device delete failed"
+	btrfs fi show $SCRATCH_DEV 2>&1 | grep $dev_del > /dev/null && _fail "btrfs still shows the deleted dev"
+	umount $SCRATCH_MNT
+}
+
+_test_raid0
+_test_raid1
+_test_raid10
+_test_single
+_test_add
+_test_replace
+_test_remove
+
+echo "Silence is golden"
+status=0; exit

265.out new file +2 -0

new file mode 100644
index 0000000..9fa4eb0
@@ -0,0 +1,2 @@
+QA output created by 265
+Silence is golden

group +1 -0

index d2abe42..94ddef6 100644
@@ -378,3 +378,4 @@ deprecated
 262 auto quick quota
 263 rw auto quick
 264 auto
+265 auto

Signature · 6 lines

1.7.7

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

Christoph Hellwig

Reviewed-by on v1 4/5

2011-10-25 11:37 UTC #43 lore raw

Re: [PATCH 4/5] 265: Functional test case for the btrfs raid operations

Looks good,

Anand Jain

author

2011-10-20 15:38 UTC #44 lore raw

PATCH 5/5 _populate_fs should use OPTIND when getopts is used

Signed-off-by Anand Jain <[email protected]>

---
 common.rc |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

common.rc +1 -0

index e3c4e67..cab0b64 100644
@@ -1547,6 +1547,7 @@ _populate_fs()
     root=root       # path of initial root of directory tree
     randomdata=false # -x data type urandom or zero
 
+    OPTIND=1
     while getopts "d:f:n:r:s:v:x" c
     do
         case $c in

Signature · 6 lines

1.7.7

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

Christoph Hellwig

Reviewed-by on v1 5/5

2011-10-25 11:37 UTC #45 lore raw

Re: [PATCH 5/5] _populate_fs should use OPTIND when getopts is used

Looks good,

Anand Jain

author

2011-10-12 04:50 UTC #46 lore raw

PATCH 2/3 264: Functional test case for the btrfs de-fragmentation

To verify the btrfs de-fragmentation does not fail

Signed-off-by Anand Jain <[email protected]>

---
 264     |   60 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 264.out |    2 ++
 group   |    1 +
 3 files changed, 63 insertions(+), 0 deletions(-)
 create mode 100755 264
 create mode 100644 264.out

264 new file +60 -0

new file mode 100755
index 0000000..61efeb8
@@ -0,0 +1,60 @@
+#! /bin/bash
+# FS QA Test No. 264
+#
+# btrfs defragmentation tests
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2011 Oracle.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+# creator
[email protected]
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+    cd /
+    rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_need_to_be_root
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+
+_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs failed"
+_scratch_mount
+
+_fillfs 1 10 100 4096 8192 $SCRATCH_MNT
+
+btrfs filesystem defragment $SCRATCH_MNT
+[ $? = 20 ] || _fail "Defragment failed"
+
+umount $SCRATCH_MNT || _fail "umount failed"
+
+echo "Silence is golden"
+status=0; exit

264.out new file +2 -0

new file mode 100644
index 0000000..eca790b
@@ -0,0 +1,2 @@
+QA output created by 264
+Silence is golden

group +1 -0

index d86624f..b5863be 100644
@@ -377,3 +377,4 @@ deprecated
 261 auto quick quota
 262 auto quick quota
 263 auto quick
+264 auto quick

Signature · 1 line

1.7.7

Anand Jain

author

2011-10-12 04:50 UTC #47 lore raw

PATCH 3/3 265: Functional test case for the btrfs raid operations