xfstests: enable many tests to run on ext2/3/4

PATCH In review Eric Sandeen 2 participants 4 messages May 21, 2009 – May 25, 2009

lore

Share & tools

lore

Eric Sandeen

author

2009-05-21 20:14 UTC #0 lore raw

PATCH 1/1 xfstests: enable many tests to run on ext2/3/4

This gets many of the tests running on ext4 (and should on ext2/ext3 as well).

A few things could still be cleaned up; rather than -xfs, -ext3, -ext4 etc it'd be better to do --fstype xfs or whatnot, and lose as much of the special-casing as possible. But this gets it going, so maybe worth merging now as a first step.

This creates a new supported_fs "generic" type which should run on any generic filesystem (well, assuming it supports acls & xattrs, along with general posix-ness).

It also adds a "generic" group, so you can do:

./check -ext4 -g generic

and that way it'll avoid any fs-specific tests. They wouldn't fail anyway, but it gets noisy to see all the "notrun" tests.

Long term there's probably a better way to do this; maybe just skip all output when there's a fs type mismatch, not sure. Right now fs compat info is in the test itself, so putting "generic" into the group file might kind of be a sort of layering violation, whaddya think? (OTOH "udf" was there already; maybe non-generic tests -should- have fs types in the group file and then you could do:

./check -ext4 -g generic -g ext4

or that last part could be automatic...)

There are a few more tests which could almost work for ext4 with a bit more work:

020 runs into different attribute limits I think 051 ditto but for acls
075 uses the -x preallocation call for xfs 062 gets a different recursive walk of files 164 uses xfs_io to do file mapping
192 is doing something odd I haven't sorted out yet

Signed-off-by Eric Sandeen <[email protected]>

---

 001       |    2 +-
 002       |    2 +-
 005       |    2 +-
 006       |    2 +-
 007       |    2 +-
 010       |    2 +-
 011       |    2 +-
 013       |    2 +-
 014       |    2 +-
 069       |    4 ++--
 070       |    2 +-
 074       |    2 +-
 076       |    2 +-
 093       |    2 +-
 097       |    2 +-
 099       |    2 +-
 100       |    2 +-
 105       |    2 +-
 123       |    2 +-
 124       |    2 +-
 125       |    2 +-
 128       |    2 +-
 131       |    2 +-
 184       |    2 +-
 193       |    2 +-
 common    |   18 ++++++++++++++++++
 common.rc |   17 +++++++++++++----
 group     |   50 +++++++++++++++++++++++++-------------------------
 new       |    2 +-
 29 files changed, 83 insertions(+), 56 deletions(-)

001 +1 -1

index 2ff2c05..2dddffb 100755
@@ -33,7 +33,7 @@ done_cleanup=false
 trap "_cleanup; rm -f $tmp.*; exit \$status" 0 1 2 3 15
 
 # real QA test starts here
-_supported_fs xfs udf nfs
+_supported_fs generic
 _supported_os IRIX Linux
 
 verbose=true

002 +1 -1

index 13c6be3..c2d2b5a 100755
@@ -30,7 +30,7 @@ _cleanup()
 }
 
 # real QA test starts here
-_supported_fs xfs udf nfs
+_supported_fs generic
 _supported_os IRIX Linux
 
 _setup_testdir

005 +1 -1

index d21dcb5..644f63d 100755
@@ -53,7 +53,7 @@ _touch()
 . ./common.filter
 
 # real QA test starts here
-_supported_fs xfs udf nfs
+_supported_fs generic
 
 # IRIX UDF does not support symlinks
 if [ $FSTYP == 'udf' ]; then

006 +1 -1

index 4da277f..49b5949 100755
@@ -40,7 +40,7 @@ _count()
 . ./common.filter
 
 # real QA test starts here
-_supported_fs xfs udf nfs
+_supported_fs generic
 _supported_os IRIX Linux
 
 _setup_testdir

007 +1 -1

index 290f716..7a97d71 100755
@@ -34,7 +34,7 @@ _cleanup()
 . ./common.filter
 
 # real QA test starts here
-_supported_fs xfs udf nfs
+_supported_fs generic
 _supported_os IRIX Linux
 
 _setup_testdir

010 +1 -1

index c51ecb2..f1bc885 100755
@@ -42,7 +42,7 @@ _filter_dbtest()
 [ -x $here/src/dbtest ] || _notrun "dbtest was not built for this platform"
 
 # real QA test starts here
-_supported_fs xfs udf nfs
+_supported_fs generic
 _supported_os IRIX Linux
 
 _setup_testdir

011 +1 -1

index 655229b..e1a3d7d 100755
@@ -32,7 +32,7 @@ _cleanup()
 . ./common.filter
 
 # real QA test starts here
-_supported_fs xfs udf nfs
+_supported_fs generic
 _supported_os IRIX Linux
 
 _setup_testdir

013 +1 -1

index 1c99e8f..e417d5a 100755
@@ -78,7 +78,7 @@ _do_test()
 
 
 # real QA test starts here
-_supported_fs xfs udf
+_supported_fs generic
 _supported_os IRIX Linux
 
 _setup_testdir

014 +1 -1

index acc3696..f5b64d0 100755
@@ -30,7 +30,7 @@ _cleanup()
 . ./common.rc
 . ./common.filter
 
-_supported_fs xfs udf nfs
+_supported_fs generic
 _supported_os IRIX Linux
 
 _setup_testdir

069 +2 -2

index 11efd98..8da6a68 100755
@@ -23,7 +23,7 @@ trap "rm -rf $tmp.*; exit \$status" 0 1 2 3 15
 . ./common.filter
 
 # real QA test starts here
-_supported_fs xfs udf nfs
+_supported_fs generic
 _supported_os IRIX Linux
 
 _require_scratch

@@ -32,7 +32,7 @@ rm -f $seq.full
 umount $SCRATCH_DEV >/dev/null 2>&1
 
 echo "*** mkfs"
-_scratch_mkfs >/dev/null || _fail "mkfs failed"
+_scratch_mkfs &>/dev/null || _fail "mkfs failed"
 
 echo "*** mount FS"
 _scratch_mount >/dev/null || _fail "mount failed"

070 +1 -1

index 65e158a..9302500 100755
@@ -30,7 +30,7 @@ _cleanup()
 . ./common.filter
 
 # real QA test starts here
-_supported_fs xfs udf nfs
+_supported_fs generic
 _supported_os IRIX Linux
 
 _setup_testdir

074 +1 -1

index d5e0364..0bd0807 100755
@@ -97,7 +97,7 @@ _process_args()
 # real QA test starts here
 rm -f $here/$seq.full
 
-_supported_fs xfs udf nfs
+_supported_fs generic
 _supported_os IRIX Linux
 
 #

076 +1 -1

index 975de2e..c0f6675 100755
@@ -39,7 +39,7 @@ trap "_cleanup; rm -f $tmp.*; exit \$status" 0 1 2 3 15
 . ./common.filter
 
 # real QA test starts here
-_supported_fs xfs udf
+_supported_fs generic
 _supported_os IRIX Linux
 
 _require_scratch

093 +1 -1

index c1243e2..1139788 100755
@@ -43,7 +43,7 @@ _filefilter()
 }
 
 # real QA test starts here
-_supported_fs xfs udf
+_supported_fs generic
 _supported_os IRIX
 
 [ -x $runas ] || _notrun "$runas executable not found"

097 +1 -1

index b7fe1f2..9c38d18 100755
@@ -58,7 +58,7 @@ else
 fi
 
 # real QA test starts here
-_supported_fs udf xfs
+_supported_fs generic
 _supported_os IRIX
 
 _require_scratch

099 +1 -1

index 1c5daeb..6b824dd 100755
@@ -62,7 +62,7 @@ rm -f $seq.full
 
 #-------------------------------------------------------
 # real QA test starts here
-_supported_fs xfs udf
+_supported_fs generic
 _supported_os IRIX
 
 _acl_setup_ids

100 +1 -1

index aa6c4ca..57e9bb6 100755
@@ -29,7 +29,7 @@ _cleanup()
 }
  
 # real QA test starts here
-_supported_fs xfs udf nfs
+_supported_fs generic
 _supported_os IRIX Linux
 
 # Use _populate_fs() in common.rc to create a directory structure.

105 +1 -1

index cd201dd..f154422 100755
@@ -32,7 +32,7 @@ _cleanup()
 . ./common.attr
 
 # Modify as appropriate.
-_supported_fs xfs udf
+_supported_fs generic
 _supported_os IRIX Linux
 
 # real QA test starts here

123 +1 -1

index 63cd48f..ba7c454 100755
@@ -57,7 +57,7 @@ _user_do()
 . ./common.filter
 
 # real QA test starts here
-_supported_fs xfs udf nfs
+_supported_fs generic
 _supported_os Linux IRIX
 
 _require_user

124 +1 -1

index 29af340..5adef48 100755
@@ -33,7 +33,7 @@ _cleanup()
 . ./common.filter
 
 # real QA test starts here
-_supported_fs xfs udf nfs
+_supported_fs generic
 _supported_os Linux
 
 _setup_testdir

125 +1 -1

index ff538ef..f432034 100755
@@ -28,7 +28,7 @@ _cleanup()
 . ./common.filter
 
 # real QA test starts here
-_supported_fs xfs udf nfs
+_supported_fs generic
 _supported_os Linux
 
 _require_user

128 +1 -1

index 24bdac7..227d05c 100755
@@ -30,7 +30,7 @@ _cleanup()
 . ./common.filter
 
 # real QA test starts here
-_supported_fs xfs udf nfs
+_supported_fs generic
 _supported_os Linux
 
 _setup_testdir

131 +1 -1

index 78627f5..67adb81 100755
@@ -28,7 +28,7 @@ _cleanup()
 . ./common.filter
 
 # real QA test starts here
-_supported_fs xfs udf nfs
+_supported_fs generic
 _supported_os Linux
 
 _setup_testdir

184 +1 -1

index 30839cb..408e645 100755
@@ -30,7 +30,7 @@ _cleanup()
 . ./common.filter
 
 # real QA test starts here
-_supported_fs xfs udf nfs
+_supported_fs generic
 _supported_os IRIX Linux
 
 _setup_testdir

193 +1 -1

index 54c51a2..743c70f 100755
@@ -50,7 +50,7 @@ _cleanup_files()
 . ./common.filter
 
 # real QA test starts here
-_supported_fs xfs nfs udf
+_supported_fs generic
 _supported_os Linux
 
 _require_user

common +18 -0

index 92ec9e2..ea31375 100644
@@ -105,6 +105,9 @@ check options
     -xfs                test XFS (default)
     -udf                test UDF
     -nfs                test NFS
+    -ext2		test ext2
+    -ext3		test ext3
+    -ext4		test ext4
     -l			line mode diff
     -xdiff		graphical mode diff
     -udiff		show unified diff (default)

@@ -132,6 +135,21 @@ testlist options
 	    xpand=false
 	    ;;
 
+	-ext2)	# -ext2 ... set FSTYP to ext2
+	    FSTYP=ext2
+	    xpand=false
+	    ;;
+
+	-ext3)	# -ext3 ... set FSTYP to ext3
+	    FSTYP=ext3
+	    xpand=false
+	    ;;
+
+	-ext4)	# -ext4 ... set FSTYP to ext4
+	    FSTYP=ext4
+	    xpand=false
+	    ;;
+
 	-nfs)	# -nfs ... set FSTYP to nfs
 	    FSTYP=nfs
 	    xpand=false

common.rc +13 -4

index 317e009..06304c8 100644
@@ -49,6 +49,10 @@ _mount_opts()
     nfs)
 	export MOUNT_OPTIONS=$NFS_MOUNT_OPTIONS
 	;;
+    ext2|ext3|ext4)
+	# acls & xattrs aren't turned on by default on ext$FOO
+	export MOUNT_OPTIONS="-o acl,user_xattr $EXT_MOUNT_OPTIONS"
+	;;
     *)
 	;;
     esac

@@ -546,7 +550,7 @@ _supported_fs()
 {
     for f
     do
-	if [ "$f" = "$FSTYP" ]
+	if [ "$f" = "$FSTYP" -o "$f" = "generic" ]
 	then
 	    return
 	fi

@@ -585,7 +589,7 @@ _require_scratch()
 		     _notrun "this test requires a valid \$SCRATCH_DEV"
 		 fi
 		 ;;
-	nfs*|ext2|ext3|reiserfs)
+	nfs*|ext2|ext3|ext4|reiserfs)
 		 echo $SCRATCH_DEV | grep -q ":" > /dev/null 2>&1
 		 if [ ! -z "$SCRATCH_DEV" -a ! -b "$SCRATCH_DEV" -a "$?" != "0" ]
 		 then

@@ -988,6 +992,10 @@ _check_testdir()
     esac
 }
 
+_setup_generic_testdir()
+{
+    testdir=$TEST_DIR
+}
 
 _setup_xfs_testdir()
 {

@@ -1067,7 +1075,7 @@ _setup_testdir()
 	_setup_nfs_scratchdir
 	;;
     *)
-	_fail "\$FSTYP is not xfs, udf or nfs"
+	_setup_generic_testdir
 	;;
     esac
 }

@@ -1088,7 +1096,8 @@ _cleanup_testdir()
 	[ -n "$testdir" ] && $UMOUNT_PROG $testdir
 	;;
     *)
-	_fail "\$FSTYP is not xfs, udf or nfs"
+	# do nothing, testdir is $TEST_DIR
+	:
 	;;
     esac
 }

group +25 ~~-25~~

107 lines not on this page · raw message

new +1 -1

index c924ffe..ac61221 100755
@@ -99,7 +99,7 @@ _cleanup()
 # real QA test starts here
 
 # Modify as appropriate.
-_supported_fs xfs udf nfs
+_supported_fs generic
 _supported_os IRIX Linux
 
 # if error

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

Christoph Hellwig

2009-05-24 14:39 UTC #1 lore raw

Wow, that's a nice start. The only important thing missing is checking the filesystems after each test run for the non-xfs case.

Maybe we should put this in in stages? The _supported_fs generic thing is a nice cleanup already for the existing xfs/nfs/udf setup and should go in ASAP.

The _scratch_mkfs output fix in 069 could also be a separate patch.

The _setup_generic_testdir should be generalized to match XFS for the default case and just set testdir in _setup_testdir instead of another function. Also the comment there should be updated.

Same for _cleanup_testdir.

Btw, the way udf and nfs are currently handled look not very nice to me. We should not set up the test device by default for any filesystem but rather have a -setup or similar option to set it up if needed.

In common I would indeed prefer a new fstype option, but we might aswell put the current version in as-is. Especially if we could tie up a really generic fstype= that wouldn't require listing the filesystems if they don't require special mount options or similar.

The only thing preventing that is as far as I can see the current difference in _require_scratch for xfs and udf vs the rest. Which looks really weird to me, need to investigate what's going on.

As for the generic group I must say I don't like it very much, the filtering of notrun (maybe only notrun because of the filesystem type mismatch) sounds much better to me.

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

Eric Sandeen

author

2009-05-24 16:38 UTC #2 lore raw

Quoted 2 lines · Christoph Hellwig

Wow, that's a nice start. The only important thing missing is checking the filesystems after each test run for the non-xfs case.

Yep, that was in the back of my mind. Probably needs more abstraction to make that cleaner.

Quoted 3 lines

Maybe we should put this in in stages? The _supported_fs generic thing is a nice cleanup already for the existing xfs/nfs/udf setup and should go in ASAP.

sure, I can break it up.

Quoted 11 lines

The _scratch_mkfs output fix in 069 could also be a separate patch.
The _setup_generic_testdir should be generalized to match XFS for the default case and just set testdir in _setup_testdir instead of another function. Also the comment there should be updated.
Same for _cleanup_testdir.
Btw, the way udf and nfs are currently handled look not very nice to me. We should not set up the test device by default for any filesystem but rather have a -setup or similar option to set it up if needed.

many of them likely fail, too, some of the acl & attr tests have some assumptions about xfs limits.

Quoted 4 lines

In common I would indeed prefer a new fstype option, but we might aswell put the current version in as-is. Especially if we could tie up a really generic fstype= that wouldn't require listing the filesystems if they don't require special mount options or similar.

Ok. I'd even thought that maybe by default, w/o options, it should just run as whatever $TEST_DEV is formatted to (though that's trickier for nfs I guess)

Quoted 3 lines

The only thing preventing that is as far as I can see the current difference in _require_scratch for xfs and udf vs the rest. Which looks really weird to me, need to investigate what's going on.

I think this is because even for udf etc, it still expects $TEST_DIR to be xfs, so swizzles around test & scratch. yeah, I agree that's messy.

Quoted 3 lines

As for the generic group I must say I don't like it very much, the filtering of notrun (maybe only notrun because of the filesystem type mismatch) sounds much better to me.

yeah, after I ran it a bit more I think I tend to agree....

I'll work on breaking this up a bit and tidying up some of the loose ends, since the basic approach seems sane to more than one person now :)

Thanks,
-Eric

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

Eric Sandeen

author

2009-05-25 15:31 UTC #3 lore raw

In reply to Eric Sandeen

Quoted 12 lines · Eric Sandeen

This gets many of the tests running on ext4 (and should on ext2/ext3 as well).
A few things could still be cleaned up; rather than -xfs, -ext3, -ext4 etc it'd be better to do --fstype xfs or whatnot, and lose as much of the special-casing as possible. But this gets it going, so maybe worth merging now as a first step.
This creates a new supported_fs "generic" type which should run on any generic filesystem (well, assuming it supports acls & xattrs, along with general posix-ness).

Quoted 6 lines

It also adds a "generic" group, so you can do:
./check -ext4 -g generic
and that way it'll avoid any fs-specific tests. They wouldn't fail anyway, but it gets noisy to see all the "notrun" tests.

For the uninitiated :) This applies on top of:

http://git.kernel.org/?p=fs/xfs/xfstests-dev.git

Apply my patch & build it with make... you'll need xfsprogs, xfsprogs-devel, and xfsprogs-qa-devel (or make install-dev, make install-qa from the xfsprogs tarball)* A few other standard devel packages like e2fsprogs-devel, acl-devel, attr-devel, libaio-devel are needed too.

edit a local.config file in the top level dir:

TEST_DEV=/dev/sdb1       # device containing TEST PARTITION
TEST_DIR=/mnt/test       # mount point of TEST PARTITION
SCRATCH_DEV=/dev/sdb3    # device containing SCRATCH PARTITION

SCRATCH_MNT=/mnt/scratch # mount point for SCRATCH PARTITION

mkfs.ext4 the $TEST_DEV

./check -ext4 -g generic

-Eric

*another TODO is probably to make ./configure cope with lack of at least the xfsprogs-qa-devel headers. There's still some work to make this cleaner/easier but this at least gets things going.

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs