fstests: various fixups uncovered by the always_cow mode

PATCH In review Christoph Hellwig 2 participants 9 messages Nov 10, 2018 – Nov 20, 2018

lore

Share & tools

lore

Christoph Hellwig

author

2018-11-10 21:36 UTC #0 lore raw

Mostly just being stricter in checking for preallocation support in various places.

Christoph Hellwig

author

2018-11-10 21:36 UTC #1 lore raw

PATCH 1/5 generic: move prealloc-enabled fsx to separate tests

Currently generic/075 and generic/112 have two extra fsx passes each that exercise fsx with preallocation, which are only enabled for XFS.

These tests can also be run with other file systems, given that the XFS prealloc ioctls are implemented in generic code since the addition of the fallocate system call. This also means a version of XFS that does not support preallocation (e.g. because it always writes out of place) can skip the prealloc tests while still completing the normal fsx tests just fine.

Signed-off-by Christoph Hellwig <[email protected]>

---
 tests/generic/075     |   7 +--
 tests/generic/075.out |  10 +--
 tests/generic/112     |  13 +---
 tests/generic/112.out |  10 +--
 tests/generic/601     | 136 +++++++++++++++++++++++++++++++++++++++++
 tests/generic/601.out |  10 +++
 tests/generic/602     | 137 ++++++++++++++++++++++++++++++++++++++++++
 tests/generic/602.out |  10 +++
 tests/generic/group   |   2 +
 9 files changed, 299 insertions(+), 36 deletions(-)
 create mode 100755 tests/generic/601
 create mode 100644 tests/generic/601.out
 create mode 100755 tests/generic/602
 create mode 100644 tests/generic/602.out

tests/generic/075 +1 -6

index 2b957891..424ec485 100755
@@ -48,14 +48,11 @@ _do_test()
     echo "fsx.$_n : $_filter_param"
     echo "-----------------------------------------------"
 
-    if [ "$FSTYP" != "xfs" ]
-    then
 	if [ "$_n" = "1" -o "$_n" = "3" ]
 	then
 	    # HACK: only xfs handles preallocation (-x) so just skip this test
 	    return
 	fi
-    fi
 
     # This cd and use of -P gets full debug on $here (not TEST_DEV)
     cd $out

@@ -138,9 +135,7 @@ _check_test_fs
 # -x: pre-allocate file space, exercising unwritten extents
 
 _do_test 0 "-d -N $numops1 -S 0"
-_do_test 1 "-d -N $numops1 -S 0 -x"
-_do_test 2 "-d -N $numops2 -l $filelen -S 0"
-_do_test 3 "-d -N $numops2 -l $filelen -S 0 -x"
+_do_test 1 "-d -N $numops2 -l $filelen -S 0"
 
 rm -f $seq.*.fsx{good,log}
 exit 0

tests/generic/075.out +1 -9

index 1f81cdf0..3ad4ae95 100644
@@ -6,13 +6,5 @@ fsx.0 : -d -N numops -S 0
 -----------------------------------------------
 
 -----------------------------------------------
-fsx.1 : -d -N numops -S 0 -x
------------------------------------------------
-
------------------------------------------------
-fsx.2 : -d -N numops -l filelen -S 0
------------------------------------------------
-
------------------------------------------------
-fsx.3 : -d -N numops -l filelen -S 0 -x
+fsx.1 : -d -N numops -l filelen -S 0
 -----------------------------------------------

tests/generic/112 +1 ~~-12~~

index 1879e7b5..c9c5e194 100755
@@ -48,15 +48,6 @@ _do_test()
     echo "fsx.$_n : $_filter_param"
     echo "-----------------------------------------------"
 
-    if [ "$FSTYP" != "xfs" ]
-    then
-        if [ "$_n" = "1" -o "$_n" = "3" ]
-        then
-            # HACK: only xfs handles preallocation (-x) so just skip this test
-            return
-        fi
-    fi
-
     # This cd and use of -P gets full debug on $here (not TEST_DEV)
     cd $out
     if ! $here/ltp/fsx $_param -P $here $FSX_AVOID $seq.$_n &>/dev/null

@@ -139,9 +130,7 @@ _check_test_fs
 # -A: use the AIO system calls
 
 _do_test 0 "-A -d -N $numops1 -S 0"
-_do_test 1 "-A -d -N $numops1 -S 0 -x"
-_do_test 2 "-A -d -N $numops2 -l $filelen -S 0"
-_do_test 3 "-A -d -N $numops2 -l $filelen -S 0 -x"
+_do_test 1 "-A -d -N $numops2 -l $filelen -S 0"
 
 rm -f $seq.*.fsx{good,log}
 exit 0

tests/generic/112.out +1 -9

index 7d1709cf..488ec682 100644
@@ -6,13 +6,5 @@ fsx.0 : -A -d -N numops -S 0
 -----------------------------------------------
 
 -----------------------------------------------
-fsx.1 : -A -d -N numops -S 0 -x
------------------------------------------------
-
------------------------------------------------
-fsx.2 : -A -d -N numops -l filelen -S 0
------------------------------------------------
-
------------------------------------------------
-fsx.3 : -A -d -N numops -l filelen -S 0 -x
+fsx.1 : -A -d -N numops -l filelen -S 0
 -----------------------------------------------

tests/generic/601 new file +136 -0

136 lines not on this page · raw message

tests/generic/601.out new file +10 -0

new file mode 100644
index 00000000..d1346da2
@@ -0,0 +1,10 @@
+QA output created by 601
+brevity is wit...
+
+-----------------------------------------------
+fsx.0 : -d -N numops -S 0 -x
+-----------------------------------------------
+
+-----------------------------------------------
+fsx.1 : -d -N numops -l filelen -S 0 -x
+-----------------------------------------------

tests/generic/602 new file +137 -0

137 lines not on this page · raw message

tests/generic/602.out new file +10 -0

new file mode 100644
index 00000000..bf771f04
@@ -0,0 +1,10 @@
+QA output created by 602
+brevity is wit...
+
+-----------------------------------------------
+fsx.0 : -A -d -N numops -S 0 -x
+-----------------------------------------------
+
+-----------------------------------------------
+fsx.1 : -A -d -N numops -l filelen -S 0 -x
+-----------------------------------------------

tests/generic/group +2 -0

index 54d71d55..b01e1cb0 100644
@@ -519,3 +519,5 @@
 514 auto quick clone
 515 auto quick clone
 516 auto quick dedupe clone
+601 auto rw
+602 auto aio rw

Signature · 1 line

2.19.1

Eryu Guan

2018-11-19 00:05 UTC #2 lore raw

Re: [PATCH 1/5] generic: move prealloc-enabled fsx to separate tests

Quoted 33 lines · Christoph Hellwig

Currently generic/075 and generic/112 have two extra fsx passes each that exercise fsx with preallocation, which are only enabled for XFS.
These tests can also be run with other file systems, given that the XFS prealloc ioctls are implemented in generic code since the addition of the fallocate system call. This also means a version of XFS that does not support preallocation (e.g. because it always writes out of place) can skip the prealloc tests while still completing the normal fsx tests just fine.
Signed-off-by Christoph Hellwig <[email protected]>
---
 tests/generic/075     |   7 +--
 tests/generic/075.out |  10 +--
 tests/generic/112     |  13 +---
 tests/generic/112.out |  10 +--
 tests/generic/601     | 136 +++++++++++++++++++++++++++++++++++++++++
 tests/generic/601.out |  10 +++
 tests/generic/602     | 137 ++++++++++++++++++++++++++++++++++++++++++
 tests/generic/602.out |  10 +++
 tests/generic/group   |   2 +
 9 files changed, 299 insertions(+), 36 deletions(-)
 create mode 100755 tests/generic/601
 create mode 100644 tests/generic/601.out
 create mode 100755 tests/generic/602
 create mode 100644 tests/generic/602.out
tests/generic/075 +0 -0
index 2b957891..424ec485 100755
@@ -48,14 +48,11 @@ _do_test()
     echo "fsx.$_n : $_filter_param"

tests/generic/075 _do_test() · line 58

Quoted 10 lines

tests/generic/075 +0 -3

     echo "-----------------------------------------------"
 
-    if [ "$FSTYP" != "xfs" ]
-    then
 	if [ "$_n" = "1" -o "$_n" = "3" ]
 	then
 	    # HACK: only xfs handles preallocation (-x) so just skip this test
 	    return
 	fi
-    fi

Looks like the whole hunk should be removed here.

Quoted 89 lines

tests/generic/075 +1 -3

     # This cd and use of -P gets full debug on $here (not TEST_DEV)
     cd $out

@@ -138,9 +135,7 @@ _check_test_fs
 # -x: pre-allocate file space, exercising unwritten extents
 
 _do_test 0 "-d -N $numops1 -S 0"
-_do_test 1 "-d -N $numops1 -S 0 -x"
-_do_test 2 "-d -N $numops2 -l $filelen -S 0"
-_do_test 3 "-d -N $numops2 -l $filelen -S 0 -x"
+_do_test 1 "-d -N $numops2 -l $filelen -S 0"
 
 rm -f $seq.*.fsx{good,log}
 exit 0

tests/generic/075.out +1 -9

index 1f81cdf0..3ad4ae95 100644
@@ -6,13 +6,5 @@ fsx.0 : -d -N numops -S 0
 -----------------------------------------------
 
 -----------------------------------------------
-fsx.1 : -d -N numops -S 0 -x
------------------------------------------------
-
------------------------------------------------
-fsx.2 : -d -N numops -l filelen -S 0
------------------------------------------------
-
------------------------------------------------
-fsx.3 : -d -N numops -l filelen -S 0 -x
+fsx.1 : -d -N numops -l filelen -S 0
 -----------------------------------------------

tests/generic/112 +1 ~~-12~~

index 1879e7b5..c9c5e194 100755
@@ -48,15 +48,6 @@ _do_test()
     echo "fsx.$_n : $_filter_param"
     echo "-----------------------------------------------"
 
-    if [ "$FSTYP" != "xfs" ]
-    then
-        if [ "$_n" = "1" -o "$_n" = "3" ]
-        then
-            # HACK: only xfs handles preallocation (-x) so just skip this test
-            return
-        fi
-    fi
-
     # This cd and use of -P gets full debug on $here (not TEST_DEV)
     cd $out
     if ! $here/ltp/fsx $_param -P $here $FSX_AVOID $seq.$_n &>/dev/null

@@ -139,9 +130,7 @@ _check_test_fs
 # -A: use the AIO system calls
 
 _do_test 0 "-A -d -N $numops1 -S 0"
-_do_test 1 "-A -d -N $numops1 -S 0 -x"
-_do_test 2 "-A -d -N $numops2 -l $filelen -S 0"
-_do_test 3 "-A -d -N $numops2 -l $filelen -S 0 -x"
+_do_test 1 "-A -d -N $numops2 -l $filelen -S 0"
 
 rm -f $seq.*.fsx{good,log}
 exit 0

tests/generic/112.out +1 -9

index 7d1709cf..488ec682 100644
@@ -6,13 +6,5 @@ fsx.0 : -A -d -N numops -S 0
 -----------------------------------------------
 
 -----------------------------------------------
-fsx.1 : -A -d -N numops -S 0 -x
------------------------------------------------
-
------------------------------------------------
-fsx.2 : -A -d -N numops -l filelen -S 0
------------------------------------------------
-
------------------------------------------------
-fsx.3 : -A -d -N numops -l filelen -S 0 -x
+fsx.1 : -A -d -N numops -l filelen -S 0
 -----------------------------------------------

tests/generic/601 new file +0 -0

new file mode 100755
index 00000000..3fb785a1

tests/generic/601 line 3

Quoted 3 lines

tests/generic/601 new file +3 -0

@@ -0,0 +1,136 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2000-2003 Silicon Graphics, Inc.  All Rights Reserved.

Hmm, I think it's better to write proper Copyright information for new tests, also it's better to follow new code style instead of copying the old test. Some examples inline..

Quoted 9 lines

tests/generic/601 new file +9 -0

+#
+# FS QA Test No. 601
+#
+# fsx (non-AIO variant, with preallocation)
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+

tests/generic/601 line 22

Quoted 10 lines

tests/generic/601 new file +10 -0

+here=`pwd`
+tmp=/tmp/$$
+status=0	# success is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+    cd /
+    rm -rf $TEST_DIR/fsx.* $tmp.*
+}

Use tab as indention. Please ./new to generate new test template, so all has been taken care of.

tests/generic/601 line 28

Quoted 6 lines

tests/generic/601 new file +6 -0

+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+_do_test()

No leading underscore for local functions. But I'm wondering if we could just use "run_fsx" from common/rc

tests/generic/601 line 31

Quoted 3 lines

tests/generic/601 new file +3 -0
+{
+    _n="$1"
+    _param="$2"

Similarly, local variables without "_". And declare local variables as "local".

Quoted 58 lines

tests/generic/601 new file +58 -0

+
+    out=$TEST_DIR/fsx
+    rm -rf $out
+    if ! mkdir $out
+    then
+	echo "    failed to mkdir $out"
+	status=1
+	exit
+    fi
+
+    _filter_param=`echo "$_param" | sed\
+	-e 's/-N [0-9][0-9]*/-N numops/' \
+	-e 's/-l [0-9][0-9]*/-l filelen/'`
+
+    echo ""
+    echo "-----------------------------------------------"
+    echo "fsx.$_n : $_filter_param"
+    echo "-----------------------------------------------"
+
+    # This cd and use of -P gets full debug on $here (not TEST_DEV)
+    cd $out
+    if ! $here/ltp/fsx $_param -P $here $seq.$_n $FSX_AVOID &>/dev/null
+    then
+	echo "    fsx ($_param) failed, $? - compare $seqres.$_n.{good,bad,fsxlog}"
+	mv $out/$seq.$_n $seqres.$_n.full
+	mv $here/$seq.$_n.fsxlog $seqres.$_n.fsxlog
+	od -xAx $seqres.$_n.full > $seqres.$_n.bad
+	od -xAx $here/$seq.$_n.fsxgood > $seqres.$_n.good
+	rm -f $here/$seq.$_n.fsxgood
+	status=1
+	exit
+    fi
+
+    cd $here
+    _check_test_fs
+}
+
+_usage()
+{
+    echo "$0: [-l filelen] [-n numops1] [-N numops2]"
+}
+
+_process_args()
+{
+    OPTIND=1
+    while getopts "l:n:N:?" c $@
+    do
+        case $c
+        in
+            l)
+                filelen=$OPTARG
+		param_type="$param_type, overidde -l"
+                ;;
+            N)
+                numops2=$OPTARG
+		param_type="$param_type, overidde -N"
+                ;;
+            n)

tests/generic/601 line 99

Quoted 10 lines

tests/generic/601 new file +10 -0

+                numops1=$OPTARG
+		param_type="$param_type, overidde -n"
+                ;;
+	    ?)
+		_usage
+		exit	
+		;;
+	esac
+    done
+}

Above two functions doesn't seem that useful in new tests, I think we can just remove them.

Quoted 10 lines

tests/generic/601 new file +10 -0

+
+# real QA test starts here
+_supported_fs generic
+_supported_os Linux
+_require_test
+_require_xfs_io_command "falloc"
+
+size10=`expr 10 \* 1024 \* 1024`	# 10 megabytes
+filelen=$size10
+numops1=1000

tests/generic/601 line 119

Quoted 10 lines

tests/generic/601 new file +10 -0

+numops2=10000
+
+# can override the params here
+_process_args "$@"
+
+echo "Params are for $param_type" >>$seqres.full
+echo "Params: n = $numops1 N = $numops2 l = $filelen" >>$seqres.full
+
+rm -f $seqres.full
+echo "brevity is wit..."

"Silence is golden"

tests/generic/601 line 121

Quoted 2 lines

tests/generic/601 new file +2 -0
+
+_check_test_fs

We do fsck after each test, no need to check fs before test.

And generic/602 has similar problems as well.

Thanks,
Eryu

Quoted 203 lines

tests/generic/601 new file +15 -0

+
+# Options:
+# -d: debug output for all operations
+# -l flen: the upper bound on file size (default 262144)
+# -o oplen: the upper bound on operation size
+# -N numops: total # operations to do (default infinity)
+# -P: save .fsxlog and .fsxgood files in dirpath (default ./)
+# -S seed: for random # generator (default 1) 0 gets timestamp
+# -x: pre-allocate file space, exercising unwritten extents
+
+_do_test 0 "-d -N $numops1 -S 0 -x"
+_do_test 1 "-d -N $numops2 -l $filelen -S 0 -x"
+
+rm -f $seq.*.fsx{good,log}
+exit 0

tests/generic/601.out new file +10 -0

new file mode 100644
index 00000000..d1346da2
@@ -0,0 +1,10 @@
+QA output created by 601
+brevity is wit...
+
+-----------------------------------------------
+fsx.0 : -d -N numops -S 0 -x
+-----------------------------------------------
+
+-----------------------------------------------
+fsx.1 : -d -N numops -l filelen -S 0 -x
+-----------------------------------------------

tests/generic/602 new file +137 -0

new file mode 100755
index 00000000..ec853eb1
@@ -0,0 +1,137 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2005 Silicon Graphics, Inc.  All Rights Reserved.
+#
+# FS QA Test No. 602
+#
+# fsx (AIO variant, with preallocation, based on 601)
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=0	# success is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+    cd /
+    rm -rf $TEST_DIR/fsx.* $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+_do_test()
+{
+    _n="$1"
+    _param="$2"
+
+    out=$TEST_DIR/fsx
+    rm -rf $out
+    if ! mkdir $out
+    then
+	echo "    failed to mkdir $out"
+	status=1
+	exit
+    fi
+
+    _filter_param=`echo "$_param" | sed\
+	-e 's/-N [0-9][0-9]*/-N numops/' \
+	-e 's/-l [0-9][0-9]*/-l filelen/'`
+
+    echo ""
+    echo "-----------------------------------------------"
+    echo "fsx.$_n : $_filter_param"
+    echo "-----------------------------------------------"
+
+    # This cd and use of -P gets full debug on $here (not TEST_DEV)
+    cd $out
+    if ! $here/ltp/fsx $_param -P $here $FSX_AVOID $seq.$_n &>/dev/null
+    then
+	echo "    fsx ($_param) returned $? - see $seq.$_n.full"
+	mv $here/$seq.$_n.fsxlog $seqres.$_n.full
+	status=1
+	exit
+    fi
+
+    cd $here
+    _check_test_fs
+}
+
+_usage()
+{
+    echo "$0: [-l filelen] [-n numops1] [-N numops2]"
+}
+
+_process_args()
+{
+    OPTIND=1
+    while getopts "l:n:N:?" c $@
+    do
+        case $c
+        in
+            l)
+                filelen=$OPTARG
+		param_type="$param_type, overidde -l"
+                ;;
+            N)
+                numops2=$OPTARG
+		param_type="$param_type, overidde -N"
+                ;;
+            n)
+                numops1=$OPTARG
+		param_type="$param_type, overidde -n"
+                ;;
+	    ?)
+		_usage
+		exit	
+		;;
+	esac
+    done
+}
+
+# real QA test starts here
+_supported_fs generic
+_supported_os Linux
+_require_test
+_require_aio
+_require_xfs_io_command "falloc"
+
+[ -x $here/ltp/aio-stress ] || \
+	_notrun "fsx not built with AIO for this platform"
+
+size10=`expr 10 \* 1024 \* 1024`	# 10 megabytes
+filelen=$size10
+numops1=1000
+numops2=10000
+
+# can override the params here
+_process_args "$@"
+
+echo "Params are for $param_type" >>$seqres.full
+echo "Params: n = $numops1 N = $numops2 l = $filelen" >>$seqres.full
+
+rm -f $seqres.full
+echo "brevity is wit..."
+
+_check_test_fs
+
+# Options:
+# -d: debug output for all operations
+# -l flen: the upper bound on file size (default 262144)
+# -o oplen: the upper bound on operation size
+# -N numops: total # operations to do (default infinity)
+# -P: save .fsxlog and .fsxgood files in dirpath (default ./)
+# -S seed: for random # generator (default 1) 0 gets timestamp
+# -x: pre-allocate file space, exercising unwritten extents
+# -A: use the AIO system calls
+
+_do_test 0 "-A -d -N $numops1 -S 0 -x"
+_do_test 1 "-A -d -N $numops2 -l $filelen -S 0 -x"
+
+rm -f $seq.*.fsx{good,log}
+exit 0

tests/generic/602.out new file +10 -0

new file mode 100644
index 00000000..bf771f04
@@ -0,0 +1,10 @@
+QA output created by 602
+brevity is wit...
+
+-----------------------------------------------
+fsx.0 : -A -d -N numops -S 0 -x
+-----------------------------------------------
+
+-----------------------------------------------
+fsx.1 : -A -d -N numops -l filelen -S 0 -x
+-----------------------------------------------

tests/generic/group +2 -0

index 54d71d55..b01e1cb0 100644
@@ -519,3 +519,5 @@
 514 auto quick clone
 515 auto quick clone
 516 auto quick dedupe clone
+601 auto rw
+602 auto aio rw

Signature · 1 line

2.19.1

Christoph Hellwig

author

2018-11-20 00:44 UTC #3 lore raw

Re: [PATCH 1/5] generic: move prealloc-enabled fsx to separate tests

Quoted 6 lines · Eryu Guan

tests/generic/601 line 3
Quoted 3 lines
tests/generic/601 new file +3 -0
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2000-2003 Silicon Graphics, Inc.  All Rights Reserved.
Hmm, I think it's better to write proper Copyright information for new tests,

Quoted 8 lines

tests/generic/601 line 22
Quoted 5 lines
tests/generic/601 new file +5 -0
+_cleanup()
+{
+    cd /
+    rm -rf $TEST_DIR/fsx.* $tmp.*
+}
Use tab as indention. Please ./new to generate new test template, so all has been taken care of.

I can replace spaces with tabs, but I don't want to regenerate the whole tests. In the it should stick to what we originally tested as much as possible.

I'll look into the other changes.

Christoph Hellwig

author

2018-11-10 21:36 UTC #4 lore raw

PATCH 2/5 generic: don't assume preallocation is supported when reflinks are supported

In reply to Christoph Hellwig

Various reflink tests currently use fallocate to preallocate space without first checking that preallocations are supported. Add explicit checks.

Signed-off-by Christoph Hellwig <[email protected]>

---
 tests/generic/176 | 1 +
 tests/generic/372 | 1 +
 tests/generic/414 | 1 +
 tests/generic/415 | 1 +
 tests/generic/447 | 1 +
 tests/generic/515 | 1 +
 6 files changed, 6 insertions(+)

tests/generic/176 +1 -0

index e0c0f225..a084578a 100755
@@ -31,6 +31,7 @@ _cleanup()
 _supported_os Linux
 _require_scratch_reflink
 _require_cp_reflink
+_require_xfs_io_command "fpunch"
 _require_test_program "punch-alternating"
 
 rm -f "$seqres.full"

tests/generic/372 +1 -0

index 44914453..21cf283b 100755
@@ -31,6 +31,7 @@ _cleanup()
 _supported_os Linux
 _supported_fs generic
 _require_scratch_reflink
+_require_xfs_io_command "falloc"
 _require_xfs_io_command "fiemap"
 _require_scratch_explicit_shared_extents

tests/generic/414 +1 -0

index 7c76d580..f9f26744 100755
@@ -32,6 +32,7 @@ _cleanup()
 _supported_os Linux
 _supported_fs generic
 _require_scratch_reflink
+_require_xfs_io_command "falloc"
 _require_xfs_io_command "fiemap"
 
 echo "Format and mount"

tests/generic/415 +1 -0

index 14fbc232..0b7554ff 100755
@@ -34,6 +34,7 @@ _supported_fs generic
 _supported_os Linux
 _require_scratch_reflink
 _require_cp_reflink
+_require_xfs_io_command "fpunch"
 
 _scratch_mkfs >> $seqres.full 2>&1 || _fail "mkfs failed"
 _scratch_mount || _fail "mount failed"

tests/generic/447 +1 -0

index 3775e49f..1e7cb3c8 100755
@@ -32,6 +32,7 @@ _supported_os Linux
 _require_scratch_reflink
 _require_cp_reflink
 _require_test_program "punch-alternating"
+_require_xfs_io_command "fpunch"
 
 rm -f "$seqres.full"

tests/generic/515 +1 -0

index 1ae9fbfe..f9e0b3be 100755
@@ -31,6 +31,7 @@ _cleanup()
 _supported_fs generic
 _supported_os Linux
 _require_scratch_reflink
+_require_xfs_io_command "falloc"
 
 rm -f $seqres.full

Signature · 1 line

2.19.1

Christoph Hellwig

author

2018-11-10 21:36 UTC #5 lore raw

PATCH 3/5 xfs: don't assume preallocation is always supported on XFS

In reply to Christoph Hellwig

The always_cow mode can't usefull preallocate space gіven that it always has to write out of place, and thus will reject falloc or ioctl calls to preallocate space in a file. Add explicit checks for preallocation support in various XFS-specific tests to support this.

---
 tests/xfs/009 | 1 +
 tests/xfs/072 | 3 +--
 tests/xfs/084 | 1 +
 tests/xfs/104 | 1 +
 tests/xfs/128 | 1 +
 tests/xfs/164 | 1 +
 tests/xfs/165 | 1 +
 tests/xfs/166 | 2 ++
 tests/xfs/167 | 1 +
 tests/xfs/231 | 1 +
 tests/xfs/232 | 1 +
 tests/xfs/242 | 1 +
 tests/xfs/250 | 1 +
 tests/xfs/280 | 1 +
 tests/xfs/294 | 2 ++
 tests/xfs/312 | 1 +
 tests/xfs/313 | 1 +
 tests/xfs/316 | 1 +
 tests/xfs/324 | 1 +
 tests/xfs/326 | 1 +
 tests/xfs/328 | 1 +
 tests/xfs/330 | 1 +
 tests/xfs/444 | 1 +
 tests/xfs/445 | 1 +
 24 files changed, 26 insertions(+), 2 deletions(-)

tests/xfs/009 +1 -0

index 7f95156c..9385d3b9 100755
@@ -48,6 +48,7 @@ _supported_fs xfs
 _supported_os Linux
 
 _require_scratch
+_require_xfs_io_command "falloc"
 
 _filesize()
 {

tests/xfs/072 +1 -2

index 735260b7..f86e4003 100755
@@ -30,9 +30,8 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fs xfs
 _supported_os Linux
 
-[ -n "$XFS_IO_PROG" ] || _notrun "xfs_io executable not found"
-
 _require_scratch
+_require_xfs_io_command "falloc"
 
 rm -f $seqres.full
 _scratch_unmount >/dev/null 2>&1

tests/xfs/084 +1 -0

index 8e9aff3c..fcdfc185 100755
@@ -35,6 +35,7 @@ pgsize=`$here/src/feature -s`
 # real QA test starts here
 _supported_fs xfs
 _supported_os Linux
+_require_xfs_io_command "falloc"
 _require_test
 
 echo

tests/xfs/104 +1 -0

index 218e2b60..bc38f969 100755
@@ -55,6 +55,7 @@ _stress_scratch()
 # real QA test starts here
 _supported_fs xfs
 _require_scratch
+_require_xfs_io_command "falloc"
 
 rm -f $seqres.full
 _scratch_mkfs_xfs | tee -a $seqres.full | _filter_mkfs 2>$tmp.mkfs

tests/xfs/128 +1 -0

index 40ea0f04..f1b4fb4f 100755
@@ -32,6 +32,7 @@ _supported_fs xfs
 _require_test_lsattr
 _require_scratch_reflink
 _require_cp_reflink
+_require_xfs_io_command "falloc"
 
 echo "Format and mount"
 _scratch_mkfs > $seqres.full 2>&1

tests/xfs/164 +1 -0

index fdd9293d..b0ec6df5 100755
@@ -52,6 +52,7 @@ _filter_bmap()
 _supported_fs xfs
 _supported_os Linux
 _require_test
+_require_xfs_io_command "falloc"
 
 testfile=$TEST_DIR/file.$seq
 rm -f $seqres.full

tests/xfs/165 +1 -0

index 4f6738d4..990d054c 100755
@@ -49,6 +49,7 @@ _filter_bmap()
 _supported_fs xfs
 _supported_os Linux
 _require_test
+_require_xfs_io_command "falloc"
 
 # io tests 
 testfile=$TEST_DIR/file.$seq

tests/xfs/166 +2 -0

index 9814e4a3..8dc7e080 100755
@@ -60,6 +60,8 @@ _supported_fs xfs
 _supported_os Linux
 
 _require_scratch
+_require_xfs_io_command "falloc"
+
 _scratch_mkfs_xfs >/dev/null 2>&1
 _scratch_mount

tests/xfs/167 +1 -0

index 40b45e86..015002d4 100755
@@ -41,6 +41,7 @@ _supported_fs xfs
 _supported_os Linux
 
 _require_command "$KILLALL_PROG" killall
+_require_xfs_io_command "falloc"
 
 _require_scratch
 _scratch_mkfs_xfs >/dev/null 2>&1

tests/xfs/231 +1 -0

index 865c0331..12fdc490 100755
@@ -38,6 +38,7 @@ _supported_fs xfs
 _require_scratch_reflink
 _require_cp_reflink
 _require_xfs_io_command "cowextsize"
+_require_xfs_io_command "falloc"
 _require_xfs_io_command "fiemap"
 
 old_cow_lifetime=$(cat /proc/sys/fs/xfs/speculative_cow_prealloc_lifetime)

tests/xfs/232 +1 -0

index 3a88cc33..e768fa27 100755
@@ -39,6 +39,7 @@ _supported_fs xfs
 _require_xfs_io_command "cowextsize"
 _require_scratch_reflink
 _require_cp_reflink
+_require_xfs_io_command "falloc"
 _require_xfs_io_command "fiemap"
 
 old_cow_lifetime=$(cat /proc/sys/fs/xfs/speculative_cow_prealloc_lifetime)

tests/xfs/242 +1 -0

index 9ae1aebd..5ce2c6f0 100755
@@ -31,6 +31,7 @@ _supported_fs xfs
 _supported_os Linux
 _require_test
 
+_require_xfs_io_command "falloc"
 _require_xfs_io_command "zero"
 
 testfile=$TEST_DIR/242.$$

tests/xfs/250 +1 -0

index f8b800e4..1f4d818e 100755
@@ -32,6 +32,7 @@ _supported_fs xfs
 _supported_os Linux
 _require_test
 _require_loop
+_require_xfs_io_command "falloc"
 
 LOOP_DEV=$TEST_DIR/$seq.fs
 LOOP_MNT=$TEST_DIR/$seq.mnt

tests/xfs/280 +1 -0

index 172eee21..76287c85 100755
@@ -31,6 +31,7 @@ _supported_os Linux
 _supported_fs xfs
 _require_scratch_reflink
 _require_xfs_io_command "bmap"
+_require_xfs_io_command "falloc"
 
 echo "Format and mount"
 _scratch_mkfs > $seqres.full 2>&1

tests/xfs/294 +2 -0

index 4c919432..bce4d07b 100755
@@ -42,6 +42,8 @@ _supported_fs xfs
 _supported_os Linux
 _require_scratch
 _require_test_program "punch-alternating"
+_require_xfs_io_command "falloc"
+_require_xfs_io_command "fpunch"
 
 # We want to mkfs with a very specific geometry
 MKFS_OPTIONS=""

tests/xfs/312 +1 -0

index cdec9233..fa05754e 100755
@@ -34,6 +34,7 @@ _supported_os Linux
 _supported_fs xfs
 _require_cp_reflink
 _require_scratch_reflink
+_require_xfs_io_command "fpunch"
 _require_xfs_io_error_injection "bmap_finish_one"
 
 rm -f $seqres.full

tests/xfs/313 +1 -0

index 47ef7ec8..97c24f0a 100755
@@ -34,6 +34,7 @@ _supported_os Linux
 _supported_fs xfs
 _require_cp_reflink
 _require_scratch_reflink
+_require_xfs_io_command "fpunch"
 _require_xfs_io_error_injection "refcount_finish_one"
 
 rm -f $seqres.full

tests/xfs/316 +1 -0

index d0cd1e2f..063f0e43 100755
@@ -35,6 +35,7 @@ _supported_fs xfs
 _require_cp_reflink
 _require_scratch_reflink
 _require_error_injection
+_require_xfs_io_command "fpunch"
 _require_xfs_io_error_injection "refcount_continue_update"
 
 rm -f $seqres.full

tests/xfs/324 +1 -0

index a1be3318..e8ed2fc5 100755
@@ -35,6 +35,7 @@ _supported_fs xfs
 _require_cp_reflink
 _require_scratch_reflink
 _require_error_injection
+_require_xfs_io_command "fpunch"
 _require_xfs_io_error_injection "refcount_continue_update"
 
 rm -f $seqres.full

tests/xfs/326 +1 -0

index b6e01862..bb7e5590 100755
@@ -37,6 +37,7 @@ _supported_fs xfs
 _require_cp_reflink
 _require_scratch_reflink
 _require_xfs_io_command "cowextsize"
+_require_xfs_io_command "fpunch"
 _require_xfs_io_error_injection "refcount_finish_one"
 
 rm -f $seqres.full

tests/xfs/328 +1 -0

index ab8af12f..55fe3ec9 100755
@@ -33,6 +33,7 @@ _supported_fs xfs
 _require_scratch_reflink
 _require_cp_reflink
 _require_test_program "punch-alternating"
+_require_xfs_io_command "falloc" # used in FSR
 _require_command "$XFS_FSR_PROG" "xfs_fsr"
 
 rm -f "$seqres.full"

tests/xfs/330 +1 -0

index 3682525d..66a9b6e2 100755
@@ -33,6 +33,7 @@ _supported_os Linux
 _supported_fs xfs
 _require_scratch_reflink
 _require_cp_reflink
+_require_xfs_io_command "falloc" # used in FSR
 _require_command "$XFS_FSR_PROG" "xfs_fsr"
 _require_quota
 _require_nobody

tests/xfs/444 +1 -0

index f1c0ed03..7a33b9e9 100755
@@ -38,6 +38,7 @@ _supported_os Linux
 _require_check_dmesg
 _require_scratch
 _require_test_program "punch-alternating"
+_require_xfs_io_command "falloc"
 _require_xfs_db_write_array
 
 # This is only a v5 filesystem problem

tests/xfs/445 +1 -0

index 2d205a40..800d5d28 100755
@@ -53,6 +53,7 @@ drop_caches()
 _supported_fs generic
 _supported_os Linux
 _require_scratch_size $((2*1024*1024)) # kb
+_require_xfs_io_command "falloc"
 
 # check for filestreams
 _check_filestreams_support || _notrun "filestreams not available"

Signature · 1 line

2.19.1

Christoph Hellwig

author

2018-11-10 21:36 UTC #6 lore raw

PATCH 4/5 xfs: xfs_fsr requires working preallocation support

In reply to Christoph Hellwig

Check that the file system actually supports preallocation for defrag tests that end up calling xfs_fsr, as they can't be supported in always_cow mode.

Signed-off-by Christoph Hellwig <[email protected]>

---
 common/defrag | 1 +
 1 file changed, 1 insertion(+)

common/defrag +1 -0

index df249dfd..79485f0d 100644
@@ -8,6 +8,7 @@ _require_defrag()
 {
     case "$FSTYP" in
     xfs)
+        _require_xfs_io_command "falloc"
         DEFRAG_PROG="$XFS_FSR_PROG"
 	;;
     ext4|ext4dev)

Signature · 1 line

2.19.1

Christoph Hellwig

author

2018-11-10 21:36 UTC #7 lore raw

PATCH 5/5 xfs/420: only check the extent layout after syncing

In reply to Christoph Hellwig

This tests validates the correct extent layout for some hairy reflink related issues. But until we called sync or fsync we have no gurantee of any data fork layout, as only writeback moves the extents from the COW for to the data fork.

Without this we'll see an error if we use COW fork speculative preallocations for non-overwrites, which is useful to reduce fragmentation.

Signed-off-by Christoph Hellwig <[email protected]>

---
 tests/xfs/420 | 2 ++
 1 file changed, 2 insertions(+)

tests/xfs/420 +2 -0

index a083a12b..aea95c7b 100755
@@ -93,6 +93,8 @@ $XFS_IO_PROG -c "pwrite -S 0x63 $((blksz * 3)) $blksz" $testdir/file2 >> $seqres
 $XFS_IO_PROG -c "pwrite -S 0x63 0 $blksz" $testdir/file3 >> $seqres.full
 $XFS_IO_PROG -c "pwrite -S 0x63 $((blksz * 3)) $blksz" $testdir/file3 >> $seqres.full
 
+sync
+
 $XFS_IO_PROG -c "bmap -ev" -c "bmap -cv" $testdir/file1 >> $seqres.full 2>&1
 $XFS_IO_PROG -c "bmap -ev" -c "bmap -cv" $testdir/file2 >> $seqres.full 2>&1
 $XFS_IO_PROG -c "bmap -ev" -c "bmap -cv" $testdir/file3 >> $seqres.full 2>&1

Signature · 1 line

2.19.1

Eryu Guan

2018-11-19 00:09 UTC #8 lore raw

Re: [PATCH 5/5] xfs/420: only check the extent layout after syncing

Quoted 19 lines · Christoph Hellwig

This tests validates the correct extent layout for some hairy reflink related issues. But until we called sync or fsync we have no gurantee of any data fork layout, as only writeback moves the extents from the COW for to the data fork.
Without this we'll see an error if we use COW fork speculative preallocations for non-overwrites, which is useful to reduce fragmentation.
Signed-off-by Christoph Hellwig <[email protected]>
---
 tests/xfs/420 | 2 ++
 1 file changed, 2 insertions(+)
tests/xfs/420 +0 -0
index a083a12b..aea95c7b 100755

tests/xfs/420 $XFS_IO_PROG -c "pwrite -S 0x63 $((blksz * 3)) $blksz" $testdir/file2 >> $seqres · line 97

Quoted 5 lines

tests/xfs/420 +2 -0

@@ -93,6 +93,8 @@ $XFS_IO_PROG -c "pwrite -S 0x63 $((blksz * 3)) $blksz" $testdir/file2 >> $seqres
 $XFS_IO_PROG -c "pwrite -S 0x63 0 $blksz" $testdir/file3 >> $seqres.full
 $XFS_IO_PROG -c "pwrite -S 0x63 $((blksz * 3)) $blksz" $testdir/file3 >> $seqres.full
 
+sync
+

There's another round of the same checks after

echo "sync filesystem" | tee -a $seqres.full

I think we could just remove the checks before this sync, otherwise we end up doing the same checks twice.

Thanks,
Eryu

P.S.

Patch 2/3/4 in this patchset look good to me, I've applied them, so there's no need to resend them when resending patch 1 and 5.

Quoted 6 lines

tests/xfs/420 +0 -0

 $XFS_IO_PROG -c "bmap -ev" -c "bmap -cv" $testdir/file1 >> $seqres.full 2>&1
 $XFS_IO_PROG -c "bmap -ev" -c "bmap -cv" $testdir/file2 >> $seqres.full 2>&1
 $XFS_IO_PROG -c "bmap -ev" -c "bmap -cv" $testdir/file3 >> $seqres.full 2>&1

Signature · 1 line

2.19.1