Skip to contentSkip to conversation

fstests: new way to run overlay tests

PATCH In review Amir Goldstein 2 participants 12 messages Feb 28, 2017 – Mar 1, 2017

lore
Share & tools
lore
#0 lore raw

v5 cover fstests: new way to run overlay tests

Hi Eryu,

I rebased my branch to your master and made some minor adaptations to Xiong's new tests. Also reordered the patches:

- patches 1-6 are generic fixes and re-factoring - if you wish, you can merge them before the overlay config changes.

- patches 7-10 implement the new overlay config. patches 7 and 10 include adaptations of Xiong's new tests to new config.

This is the 5th revision of new overlayfs config.
The main motivation of this work is to help catch overlayfs bugs related to leaking objects in underlying (base) fs.

With this change, all you have to do to run overlay tests if you already have a local.config setup to test a local file system is:
./check -overlay

See README.overlay for some examples of config files to use with -overlay.

The legacy overlayfs configuration, where TEST_DEV is a directory still works, but it should be deprecated.

Tested ./check -overlay -g quick with both legacy overlay configuration and the new base fs configuration.

Eryu wished to keep these changes soaking on the list for while, so for those who can help with testing, you can get pull the branch for testing from my github tree [1].

Thanks,
Amir.

[1] https://github.com/amir73il/xfstests/tree/ovl_base_fs

v5:
- Rebased to weekly update
- Adapted new overlay tests by Xiong to new config
- Reordered patches "fixes and re-factoring first"

v4:
- Address review comments from Eryu
- Fix sanity checks for already mounted scratch base fs
- Fix handling of SCRATCH_DEV_POOL
- Fix problems when run in kvm-xfstests
- Add README.overlay with example config files
- Remove documentation for overlay config sections

v3:
- Mount cycle base test fs
- Fix bugs in non overlay specific sanity checks
- Run -overlay test with existing config file of base fs
- Run overlay tests per base fs by adding overlay config sections

v2:
- Test and scratch base dirs each have thier own base fs
- Support mount cycles of base fs for scratch tests

v1:
- Both test and scratch base dirs on a single base fs

Amir Goldstein (10):
fstests: sanity check that test partitions are not mounted elsewhere
fstests: use _test_mount() consistently
fstests: canonicalize mount points on every config section
fstests: fix test and scratch filters for overlapping DEV/MNT paths
fstests: allow overlay SCRATCH_DEV to be a base fs mount point generic/064: access SCRATCH_MNT after _scratch_mount
overlay: rename OVERLAY_LOWER/UPPER/WORK_DIR
overlay: configure TEST/SCRATCH vars to base fs
overlay: mount/unmount base fs before/after running tests
overlay: use OVL_BASE_SCRATCH_MNT instead of SCRATCH_DEV

 README                |  16 ++--
 README.overlay        |  46 +++++++++++
 check                 |  18 ++---
 common/config         | 160 ++++++++++++++++++++++++++++++++++-----
 common/filter         |  33 +++++++-
 common/rc             | 206 ++++++++++++++++++++++++++++++++++++--------------
 tests/generic/064     |  10 ++-
 tests/overlay/001     |   7 +-
 tests/overlay/002     |   2 +-
 tests/overlay/003     |   5 +-
 tests/overlay/004     |   7 +-
 tests/overlay/005     |  30 ++++----
 tests/overlay/006     |  10 +--
 tests/overlay/008     |   8 +-
 tests/overlay/009     |   2 +-
 tests/overlay/010     |  10 +--
 tests/overlay/011     |   6 +-
 tests/overlay/012     |   4 +-
 tests/overlay/013     |   4 +-
 tests/overlay/014     |  19 ++---
 tests/overlay/015     |   2 +-
 tests/overlay/016     |   2 +-
 tests/overlay/017     |   2 +-
 tests/overlay/018     |   2 +-
 tests/overlay/019     |   2 +-
 tests/overlay/020     |   2 +-
 tests/overlay/021     |   6 +-
 tests/overlay/022     |   2 +-
 tests/overlay/023     |   4 +-
 tests/overlay/023.out |   2 +-
 tests/overlay/024     |   2 +-
 tests/overlay/027     |   2 +-
 tests/overlay/028     |   2 +-
 tests/overlay/029     |   4 +-
 34 files changed, 466 insertions(+), 173 deletions(-)
 create mode 100644 README.overlay
Signature · 1 line
2.7.4
#1 lore raw

PATCH v5 1/10 fstests: sanity check that test partitions are not mounted elsewhere

When $TEST_DEV is mounted at a different location then $TEST_DIR, _require_test() aborts the test with an error:
TEST_DEV=/dev/sda5 is mounted but not on TEST_DIR=/mnt/test

There are several problems with current sanity check:
1. the output of the error is mixed into out.bad and hard to see
2. the test partition is unmounted at the end of the test regardless of the fact that it not pass the sanity that we have exclusivity
3. scratch partition has a similar sanity check in _require_scratch(), but we may not get to it, because $SCRATCH_DEV is unmounted prior to running the tests (which could unmount another mount point).

To solve all these problems, introduce a helper _check_mounted_on(). It checks if a device is mounted on a given mount point and optionally checks the mounted fs type.

The sanity checks in _require_scratch() and _require_test() are converted to use the helper and gain the check for correct fs type.

The helper is used in init_rc() to sanity check both test and scratch partitions, before tests are run and before $SCRATCH_DEV is unmounted.

---
 common/rc | 90 +++++++++++++++++++++++++++++++++++++++------------------------
 1 file changed, 56 insertions(+), 34 deletions(-)
common/rc +56 -34
index 8f23334..dab3cb2 100644
@@ -1354,6 +1354,43 @@ _supported_os()
     _notrun "not suitable for this OS: $HOSTOS"
 }
 
+# check if a FS on a device is mounted
+# if so, verify that it is mounted on mount point
+# if fstype is given as argument, verify that it is also
+# mounted with correct fs type
+#
+_check_mounted_on()
+{
+	local devname=$1
+	local dev=$2
+	local mntname=$3
+	local mnt=$4
+	local type=$5
+
+	# Note that we use -F here so grep doesn't try to interpret an NFS over
+	# IPv6 server as a regular expression
+	local mount_rec=`_mount | grep -F "$dev"`
+	[ -n "$mount_rec" ] || return 1 # 1 = not mounted
+
+	# if it's mounted, make sure its on $mnt
+	if ! (echo $mount_rec | grep -q "$mnt")
+	then
+		echo "$devname=$dev is mounted but not on $mntname=$mnt - aborting"
+		echo "Already mounted result:"
+		echo $mount_rec
+		return 2 # 2 = mounted on wrong mnt
+	fi
+
+	if [ -n "$type" -a "`_fs_type $dev`" != "$type" ]
+	then
+		echo "$devname=$dev is mounted but not a type $type filesystem"
+		# raw $DF_PROG cannot handle NFS/CIFS/overlay correctly
+		_df_device $dev
+		return 3 # 3 = mounted as wrong type
+	fi
+	return 0 # 0 = mounted as expected
+}
+
 # this test needs a scratch partition - check we're ok & unmount it
 # No post-test check of the device is required. e.g. the test intentionally
 # finishes the test with the filesystem in a corrupt state
@@ -1408,21 +1445,12 @@ _require_scratch_nocheck()
 		 ;;
     esac
 
-    # mounted?
-    # Note that we use -F here so grep doesn't try to interpret an NFS over
-    # IPv6 server as a regular expression.
-    mount_rec=`_mount | grep -F $SCRATCH_DEV`
-    if [ "$mount_rec" ]
+    _check_mounted_on SCRATCH_DEV $SCRATCH_DEV SCRATCH_MNT $SCRATCH_MNT
+    local err=$?
+    [ $err -le 1 ] || exit 1
+    if [ $err -eq 0 ]
     then
-        # if it's mounted, make sure its on $SCRATCH_MNT
-        if ! echo $mount_rec | grep -q $SCRATCH_MNT
-        then
-            echo "\$SCRATCH_DEV=$SCRATCH_DEV is mounted but not on \$SCRATCH_MNT=$SCRATCH_MNT - aborting"
-            echo "Already mounted result:"
-            echo $mount_rec
-            exit 1
-        fi
-        # and then unmount it
+        # if it's mounted, unmount it
         if ! _scratch_unmount
         then
             echo "failed to unmount $SCRATCH_DEV"
@@ -1493,21 +1521,11 @@ _require_test()
 		 ;;
     esac
 
-    # mounted?
-    # Note that we use -F here so grep doesn't try to interpret an NFS over
-    # IPv6 server as a regular expression.
-    mount_rec=`_mount | grep -F $TEST_DEV`
-    if [ "$mount_rec" ]
+    _check_mounted_on TEST_DEV $TEST_DEV TEST_DIR $TEST_DIR
+    local err=$?
+    [ $err -le 1 ] || exit 1
+    if [ $err -ne 0 ]
     then
-        # if it's mounted, make sure its on $TEST_DIR
-        if ! echo $mount_rec | grep -q $TEST_DIR
-        then
-            echo "\$TEST_DEV=$TEST_DEV is mounted but not on \$TEST_DIR=$TEST_DIR - aborting"
-            echo "Already mounted result:"
-            echo $mount_rec
-            exit 1
-        fi
-    else
 	out=`_mount_or_remount_rw "$MOUNT_OPTIONS" $TEST_DEV $TEST_DIR`
 	if [ $? -ne 1 ]; then
 		echo $out
@@ -3132,13 +3150,17 @@ init_rc()
 		fi
 	fi
 
-	if [ "`_fs_type $TEST_DEV`" != "$FSTYP" ]
-	then
-		echo "common/rc: Error: \$TEST_DEV ($TEST_DEV) is not a MOUNTED $FSTYP filesystem"
-		# raw $DF_PROG cannot handle NFS/CIFS/overlay correctly
-		_df_device $TEST_DEV
-		exit 1
+	# Sanity check that TEST partition is not mounted at another mount point
+	# or as another fs type
+	_check_mounted_on TEST_DEV $TEST_DEV TEST_DIR $TEST_DIR $FSTYP || exit 1
+	if [ -n "$SCRATCH_DEV" ]; then
+		# Sanity check that SCRATCH partition is not mounted at another
+		# mount point, because it is about to be unmounted and formatted.
+		# Another fs type for scratch is fine (bye bye old fs type).
+		_check_mounted_on SCRATCH_DEV $SCRATCH_DEV SCRATCH_MNT $SCRATCH_MNT
+		[ $? -le 1 ] || exit 1
 	fi
+
 	# Figure out if we need to add -F ("foreign", deprecated) option to xfs_io
 	$XFS_IO_PROG -c stat $TEST_DIR 2>&1 | grep -q "is not on an XFS filesystem" && \
 		export XFS_IO_PROG="$XFS_IO_PROG -F"
Signature · 1 line
2.7.4
#2 lore raw

PATCH v5 2/10 fstests: use _test_mount() consistently

In reply to Amir Goldstein

On start of every test run and on every test, in init_rc() helper, the test partition is verified to be mounted, or is mounted by the helper _test_mount().

_test_mount() uses mount options $TEST_FS_MOUNT_OPTS and not $MOUNT_OPTIONS like _scratch_mount() does.

_test_cycle_mount(), which is called by some tests uses the _test_mount() helper as well.

Contrary to those cases, in _require_test() helper, if test partition is not mounted, the helper _mount_or_remount_rw() is called to mount the test partition with $MOUNT_OPTIONS. Although this should never happen, because of the test in init_rc(), this case is inconsistent with the rest of the code, so it has been changed to use _test_mount() as it should.

When running tests with a multi section configuration, and either FSTYP or MOUNT_OPTIONS change between sections, the helper _test_unmount() is called to unmount the old test mount and then _mount_or_remount_rw() is called to mount it again with new FSTYP and/or MOUNT_OPTIONS.
This is again inconsistent with the rest of the code, so was changed to use _test_mount() and instead of checking if MOUNT_OPTIONS have changed between sections, we check if TEST_FS_MOUNT_OPTS were changed between sections. Otherwise, we can leave the test partition mounted.

This change is needed to support overlay base fs mount and for multi section config files which include overlay FSTYP.

---
 check     | 16 ++++++++--------
 common/rc |  6 +++---
 2 files changed, 11 insertions(+), 11 deletions(-)
check +8 -8
index 5d7f75c..fbd12e9 100755
@@ -474,7 +474,7 @@ fi
 
 for section in $HOST_OPTIONS_SECTIONS; do
 	OLD_FSTYP=$FSTYP
-	OLD_MOUNT_OPTIONS=$MOUNT_OPTIONS
+	OLD_TEST_FS_MOUNT_OPTS=$TEST_FS_MOUNT_OPTS
 	get_next_config $section
 
 	# Do we need to run only some sections ?
@@ -527,18 +527,18 @@ for section in $HOST_OPTIONS_SECTIONS; do
 			status=1
 			exit
 		fi
-		out=`_mount_or_remount_rw "$MOUNT_OPTIONS" $TEST_DEV $TEST_DIR`
-		if [ $? -ne 1 ]; then
-			echo $out
+		if ! _test_mount
+		then
+			echo "check: failed to mount $TEST_DEV on $TEST_DIR"
 			status=1
 			exit
 		fi
 		_prepare_test_list
-	elif [ "$OLD_MOUNT_OPTIONS" != "$MOUNT_OPTIONS" ]; then
+	elif [ "$OLD_TEST_FS_MOUNT_OPTS" != "$TEST_FS_MOUNT_OPTS" ]; then
 		_test_unmount 2> /dev/null
-		out=`_mount_or_remount_rw "$MOUNT_OPTIONS" $TEST_DEV $TEST_DIR`
-		if [ $? -ne 1 ]; then
-			echo $out
+		if ! _test_mount
+		then
+			echo "check: failed to mount $TEST_DEV on $TEST_DIR"
 			status=1
 			exit
 		fi
common/rc +3 -3
index dab3cb2..4fa211a 100644
@@ -1526,9 +1526,9 @@ _require_test()
     [ $err -le 1 ] || exit 1
     if [ $err -ne 0 ]
     then
-	out=`_mount_or_remount_rw "$MOUNT_OPTIONS" $TEST_DEV $TEST_DIR`
-	if [ $? -ne 1 ]; then
-		echo $out
+	if ! _test_mount
+	then
+		echo "!!! failed to mount $TEST_DEV on $TEST_DIR"
 		exit 1
 	fi
     fi
Signature · 1 line
2.7.4
#3 lore raw

PATCH v5 3/10 fstests: canonicalize mount points on every config section

In reply to Amir Goldstein

Create helper _canonicalize_mountpoint() to check and canonicalize a mount point path.
Use helper to canonicalize TEST_DIR and SCRATCH_MNT after parse of every config section.

This is needed for overlay base fs mount.

---
 common/config | 34 +++++++++++++++++++++-------------
 1 file changed, 21 insertions(+), 13 deletions(-)
common/config +21 -13
index 3ef401b..c402576 100644
@@ -455,6 +455,20 @@ _check_device()
 	fi
 }
 
+# check and return a canonical mount point path
+_canonicalize_mountpoint()
+{
+	local name=$1
+	local dir=$2
+
+	if [ ! -d "$dir" ]; then
+		_fatal "common/config: $name ($dir) is not a directory"
+	fi
+
+	# this follows symlinks and removes all trailing "/"s
+	readlink -e "$dir"
+}
+
 # Parse config section options. This function will parse all the configuration
 # within a single section which name is passed as an argument. For section
 # name format see comments in get_config_sections().
@@ -543,10 +557,7 @@ get_next_config() {
 	fi
 
 	_check_device TEST_DEV required $TEST_DEV
-	if [ ! -d "$TEST_DIR" ]; then
-		echo "common/config: Error: \$TEST_DIR ($TEST_DIR) is not a directory"
-		exit 1
-	fi
+	export TEST_DIR=`_canonicalize_mountpoint TEST_DIR $TEST_DIR`
 
 	# a btrfs tester will set only SCRATCH_DEV_POOL, we will put first of its dev
 	# to SCRATCH_DEV and rest to SCRATCH_DEV_POOL to maintain the backward compatibility
@@ -561,10 +572,7 @@ get_next_config() {
 	fi
 
 	_check_device SCRATCH_DEV optional $SCRATCH_DEV
-	if [ ! -z "$SCRATCH_MNT" -a ! -d "$SCRATCH_MNT" ]; then
-		echo "common/config: Error: \$SCRATCH_MNT ($SCRATCH_MNT) is not a directory"
-		exit 1
-	fi
+	export SCRATCH_MNT=`_canonicalize_mountpoint SCRATCH_MNT $SCRATCH_MNT`
 
 	if [ -n "$USE_EXTERNAL" ]; then
 		_check_device TEST_RTDEV optional $TEST_RTDEV
@@ -591,12 +599,12 @@ if [ -z "$CONFIG_INCLUDED" ]; then
 	[ -z "$TEST_FS_MOUNT_OPTS" ] && _test_mount_opts
 	[ -z "$MKFS_OPTIONS" ] && _mkfs_opts
 	[ -z "$FSCK_OPTIONS" ] && _fsck_opts
+else
+	# canonicalize the mount points
+	# this follows symlinks and removes all trailing "/"s
+	export TEST_DIR=`_canonicalize_mountpoint TEST_DIR $TEST_DIR`
+	export SCRATCH_MNT=`_canonicalize_mountpoint SCRATCH_MNT $SCRATCH_MNT`
 fi
 
-# canonicalize the mount points
-# this follows symlinks and removes all trailing "/"s
-export TEST_DIR=`readlink -e "$TEST_DIR"`
-export SCRATCH_MNT=`readlink -e "$SCRATCH_MNT"`
-
 # make sure this script returns success
 /bin/true
Signature · 1 line
2.7.4
#4 lore raw

PATCH v5 4/10 fstests: fix test and scratch filters for overlapping DEV/MNT paths

In reply to Amir Goldstein

When configuring overlay base fs, TEST_DEV/DIR and SCRATCH_DEV/MNT are derived from the base fs mount points, where *_DEV are the path of the base fs mount point and TEST_DIR/SCRATCH_MNT are a directory under the base fs mount point.

This means that the overlay DEV paths are prefixes of the overlay mount points.
Fix the test and sctach filters to check if TEST_DEV/SCRATCH_DEV is a substring of TEST_DIR/SCRATCH_MNT and try and match the longer string first.

---
 common/filter | 26 ++++++++++++++++++++++----
 1 file changed, 22 insertions(+), 4 deletions(-)
common/filter +22 -4
index 4328159..010f080 100644
@@ -280,14 +280,32 @@ _filter_xfs_io_pages_modified()
 
 _filter_test_dir()
 {
-	sed -e "s,$TEST_DEV,TEST_DEV,g" -e "s,$TEST_DIR,TEST_DIR,g"
+	if ( echo $TEST_DIR | grep -q $TEST_DEV ); then
+		# TEST_DEV is substr of TEST_DIR (e.g. /mnt and /mnt/ovl-mnt)
+		# substitute TEST_DIR first
+		sed -e "s,$TEST_DIR,TEST_DIR,g" \
+		    -e "s,$TEST_DEV,TEST_DEV,g"
+	else
+		# TEST_DIR maybe a substr of TEST_DIR (e.g. /vdc and /dev/vdc)
+		# substitute TEST_DEV first
+		sed -e "s,$TEST_DEV,TEST_DEV,g" \
+		    -e "s,$TEST_DIR,TEST_DIR,g"
+	fi
 }
 
 _filter_scratch()
 {
-	sed -e "s,$SCRATCH_DEV,SCRATCH_DEV,g" \
-	    -e "s,$SCRATCH_MNT,SCRATCH_MNT,g" \
-	    -e "/.use_space/d"
+	if ( echo $SCRATCH_MNT | grep -q $SCRATCH_DEV ); then
+		# SCRATCH_DEV is substr of SCRATCH_MNT
+		sed -e "s,$SCRATCH_MNT,SCRATCH_MNT,g" \
+		    -e "s,$SCRATCH_DEV,SCRATCH_DEV,g" \
+		    -e "/.use_space/d"
+	else
+		# SCRATCH_MNT maybe a substr of SCRATCH_DEV
+		sed -e "s,$SCRATCH_DEV,SCRATCH_DEV,g" \
+		    -e "s,$SCRATCH_MNT,SCRATCH_MNT,g" \
+		    -e "/.use_space/d"
+	fi
 }
 
 # Turn any device in the scratch pool into SCRATCH_DEV
Signature · 1 line
2.7.4
#5 lore raw

PATCH v5 5/10 fstests: allow overlay SCRATCH_DEV to be a base fs mount point

In reply to Amir Goldstein

When configure SCRATCH_DEV to a mount point (and not to a directory therein) then user will get a false positive error in scratch tests:

$SCRATCH_DEV=/mnt/base/scratch is mounted but not on $SCRATCH_MNT=/mnt/scratch Already mounted result:
/dev/sda6 on /mnt/base/scratch type xfs (rw,relatime,attr2,inode64,noquota)

This is due to the wrong grep -F $SCRATCH_DEV which matches the mount point instead of the device in that mount.
Fix _check_mounted_on() to grep the pattern "$dev on " and "$dev on $mnt" instead of just grepping for "$dev" and "$mnt" without the " on " anchor.

---
 common/rc | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
common/rc +5 -3
index 4fa211a..fb88ffa 100644
@@ -1368,12 +1368,14 @@ _check_mounted_on()
 	local type=$5
 
 	# Note that we use -F here so grep doesn't try to interpret an NFS over
-	# IPv6 server as a regular expression
-	local mount_rec=`_mount | grep -F "$dev"`
+	# IPv6 server as a regular expression.  Because of that, we cannot use
+	# ^$dev so we use "$dev on " to avoid matching $dev to mount point field
+	# for overlay case, where $dev is a directory.
+	local mount_rec=`_mount | grep -F "$dev on "`
 	[ -n "$mount_rec" ] || return 1 # 1 = not mounted
 
 	# if it's mounted, make sure its on $mnt
-	if ! (echo $mount_rec | grep -q "$mnt")
+	if ! (echo $mount_rec | grep -q "$dev on $mnt")
 	then
 		echo "$devname=$dev is mounted but not on $mntname=$mnt - aborting"
 		echo "Already mounted result:"
Signature · 1 line
2.7.4
#6 lore raw

PATCH v5 6/10 generic/064: access SCRATCH_MNT after _scratch_mount

In reply to Amir Goldstein

Test is calling _get_block_size $SCRATCH_MNT before _scratch_mount. This results in block size of the wrong fs and a failure with overlay base fs setup.

---
 tests/generic/064 | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)
tests/generic/064 +6 -4
index 8cd31b3..156485e 100755
@@ -46,14 +46,16 @@ _require_scratch
 _require_xfs_io_command "fiemap"
 _require_xfs_io_command "finsert"
 _require_xfs_io_command "fcollapse"
-src=$SCRATCH_MNT/testfile
-dest=$SCRATCH_MNT/testfile.dest
-BLOCKS=100
-BSIZE=`_get_block_size $SCRATCH_MNT`
+
 rm -f $seqres.full
 
 _scratch_mkfs >> $seqres.full 2>&1 || _fail "mkfs failed"
 _scratch_mount || _fail "mount failed"
+
+src=$SCRATCH_MNT/testfile
+dest=$SCRATCH_MNT/testfile.dest
+BLOCKS=100
+BSIZE=`_get_block_size $SCRATCH_MNT`
 length=$(($BLOCKS * $BSIZE))
 
 # Write file
Signature · 1 line
2.7.4
#7 lore raw

PATCH v5 7/10 overlay: rename OVERLAY_LOWER/UPPER/WORK_DIR

In reply to Amir Goldstein

As preparation step for configuring test/scratch base fs for overlay tests:

- Rename OVERLAY_LOWER/UPPER/WORK_DIR => OVL_LOWER/UPPER/WORK because we want to use OVL_ prefix for all base fs vars

- Prepend "ovl-" prefix to lower/upper/work path values to isolate the overlay test dirs when running on a base fs that is also used to run non overlay tests

- Make those vars values non-configurable, because lower/upper/work directory names are an internal test detail which should not concern the user and because we wish to simplify and document the overlay tests setup

- Add helper _filter_ovl_dirs and use it to filter output of test overlay/023 which matches the constant string "work"

---
 common/config         |  8 +++++---
 common/filter         |  7 +++++++
 common/rc             | 19 +++++++++++++------
 tests/overlay/001     |  2 +-
 tests/overlay/002     |  2 +-
 tests/overlay/003     |  2 +-
 tests/overlay/004     |  4 ++--
 tests/overlay/006     | 10 +++++-----
 tests/overlay/008     |  8 ++++----
 tests/overlay/009     |  2 +-
 tests/overlay/010     |  8 ++++----
 tests/overlay/011     |  4 ++--
 tests/overlay/012     |  4 ++--
 tests/overlay/013     |  4 ++--
 tests/overlay/015     |  2 +-
 tests/overlay/016     |  2 +-
 tests/overlay/017     |  2 +-
 tests/overlay/018     |  2 +-
 tests/overlay/019     |  2 +-
 tests/overlay/020     |  2 +-
 tests/overlay/021     |  2 +-
 tests/overlay/022     |  2 +-
 tests/overlay/023     |  4 ++--
 tests/overlay/023.out |  2 +-
 tests/overlay/024     |  2 +-
 tests/overlay/027     |  2 +-
 tests/overlay/028     |  2 +-
 tests/overlay/029     |  4 ++--
 28 files changed, 66 insertions(+), 50 deletions(-)
common/config +5 -3
index c402576..d09cac1 100644
@@ -77,9 +77,11 @@ export XFS_MKFS_OPTIONS=${XFS_MKFS_OPTIONS:=-bsize=4096}
 export TIME_FACTOR=${TIME_FACTOR:=1}
 export LOAD_FACTOR=${LOAD_FACTOR:=1}
 export DEBUGFS_MNT=${DEBUGFS_MNT:="/sys/kernel/debug"}
-export OVERLAY_UPPER_DIR=${OVERLAY_UPPER_DIR:="upper"}
-export OVERLAY_LOWER_DIR=${OVERLAY_LOWER_DIR:="lower"}
-export OVERLAY_WORK_DIR=${OVERLAY_WORK_DIR:="work"}
+
+# some constants for overlayfs setup
+export OVL_UPPER="ovl-upper"
+export OVL_LOWER="ovl-lower"
+export OVL_WORK="ovl-work"
 
 export PWD=`pwd`
 #export DEBUG=${DEBUG:=...} # arbitrary CFLAGS really.
common/filter +7 -0
index 010f080..1ceb346 100644
@@ -424,5 +424,12 @@ _filter_lostfound()
 	sed -e '/^lost+found$/d'
 }
 
+_filter_ovl_dirs()
+{
+	sed -e "s,$OVL_LOWER,OVL_LOWER,g" \
+	    -e "s,$OVL_UPPER,OVL_UPPER,g" \
+	    -e "s,$OVL_WORK,OVL_WORK,g"
+}
+
 # make sure this script returns success
 /bin/true
common/rc +13 -6
index fb88ffa..cca046b 100644
@@ -277,7 +277,7 @@ _common_dev_mount_options()
 
 _overlay_basic_mount_options()
 {
-	echo "-o lowerdir=$1/$OVERLAY_LOWER_DIR,upperdir=$1/$OVERLAY_UPPER_DIR,workdir=$1/$OVERLAY_WORK_DIR"
+	echo "-o lowerdir=$1/$OVL_LOWER,upperdir=$1/$OVL_UPPER,workdir=$1/$OVL_WORK"
 }
 
 _overlay_mount_options()
@@ -336,6 +336,15 @@ _overlay_mount_dirs()
 		    -o workdir=$workdir $*
 }
 
+_overlay_mkdirs()
+{
+	local dir=$1
+
+	mkdir -p $dir/$OVL_UPPER
+	mkdir -p $dir/$OVL_LOWER
+	mkdir -p $dir/$OVL_WORK
+}
+
 # Given a dir, set up 3 subdirectories and mount on the given mnt.
 # The dir is used as the mount device so it can be seen from df or mount
 _overlay_mount()
@@ -346,12 +355,10 @@ _overlay_mount()
 
 	_supports_filetype $dir || _notrun "upper fs needs to support d_type"
 
-	mkdir -p $dir/$OVERLAY_UPPER_DIR
-	mkdir -p $dir/$OVERLAY_LOWER_DIR
-	mkdir -p $dir/$OVERLAY_WORK_DIR
+	_overlay_mkdirs $dir
 
-	_overlay_mount_dirs $dir/$OVERLAY_LOWER_DIR $dir/$OVERLAY_UPPER_DIR \
-			    $dir/$OVERLAY_WORK_DIR $OVERLAY_MOUNT_OPTIONS \
+	_overlay_mount_dirs $dir/$OVL_LOWER $dir/$OVL_UPPER \
+			    $dir/$OVL_WORK $OVERLAY_MOUNT_OPTIONS \
 			    $SELINUX_MOUNT_OPTIONS $* $dir $mnt
 }
 
tests/overlay/001 +1 -1
index 7be9ae5..099ddd5 100755
@@ -60,7 +60,7 @@ _scratch_mkfs
 _require_fs_space $SCRATCH_DEV $((4*1024*1024*2 + 8))
 
 # Create test files with different sizes in lower dir
-lowerdir=$SCRATCH_DEV/$OVERLAY_LOWER_DIR
+lowerdir=$SCRATCH_DEV/$OVL_LOWER
 mkdir -p $lowerdir
 touch $lowerdir/zero_size
 $XFS_IO_PROG -fc "pwrite 0 4k" $lowerdir/less_than_4g >>$seqres.full
tests/overlay/002 +1 -1
index ec7874e..eaf9a91 100755
@@ -59,7 +59,7 @@ _require_scratch
 _scratch_mkfs
 
 # Create our test file.
-lowerdir=$SCRATCH_DEV/$OVERLAY_LOWER_DIR
+lowerdir=$SCRATCH_DEV/$OVL_LOWER
 mkdir -p $lowerdir
 touch $lowerdir/foobar
 
tests/overlay/003 +1 -1
index fbc242a..5e610c8 100755
@@ -60,7 +60,7 @@ _require_scratch
 _scratch_mkfs
 
 # Create test files dirs in lower dir
-lowerdir=${SCRATCH_DEV}/${OVERLAY_LOWER_DIR}
+lowerdir=${SCRATCH_DEV}/${OVL_LOWER}
 mkdir -p $lowerdir
 
 touch ${lowerdir}/test_file
tests/overlay/004 +2 -2
index 4075094..8da5170 100755
@@ -53,8 +53,8 @@ _require_user
 _scratch_mkfs
 
 # Create test file in lower dir
-lowerdir=${SCRATCH_DEV}/${OVERLAY_LOWER_DIR}
-upperdir=${SCRATCH_DEV}/${OVERLAY_UPPER_DIR}
+lowerdir=${SCRATCH_DEV}/${OVL_LOWER}
+upperdir=${SCRATCH_DEV}/${OVL_UPPER}
 mkdir -p $lowerdir
 touch ${lowerdir}/attr_file1
 chmod 000 ${lowerdir}/attr_file1
tests/overlay/006 +5 -5
index 31f11ef..55fc2dd 100755
@@ -57,15 +57,15 @@ echo "Silence is golden"
 _scratch_mkfs
 
 # Create test file/dir before mount
-mkdir $SCRATCH_DEV/$OVERLAY_LOWER_DIR
-mkdir $SCRATCH_DEV/$OVERLAY_UPPER_DIR
-touch $SCRATCH_DEV/$OVERLAY_LOWER_DIR/lowertestfile
-mkdir $SCRATCH_DEV/$OVERLAY_UPPER_DIR/uppertestdir
+mkdir -p $SCRATCH_DEV/$OVL_LOWER
+mkdir -p $SCRATCH_DEV/$OVL_UPPER
+touch $SCRATCH_DEV/$OVL_LOWER/lowertestfile
+mkdir $SCRATCH_DEV/$OVL_UPPER/uppertestdir
 
 _scratch_mount
 
 # rename lowertestfile to uppertestdir, this triggers copyup and creates
-# whiteout in $OVERLAY_UPPER_DIR
+# whiteout in $OVL_UPPER
 mv $SCRATCH_MNT/lowertestfile $SCRATCH_MNT/uppertestdir
 # the lowertestfile can be removed
 rm $SCRATCH_MNT/uppertestdir/lowertestfile
tests/overlay/008 +4 -4
index cb8667c..b81cff8 100755
@@ -56,14 +56,14 @@ _require_user
 _scratch_mkfs
 
 # Create test file on lower dir, and chown to fsgqa user
-lowerdir=$SCRATCH_DEV/$OVERLAY_LOWER_DIR
-mkdir $lowerdir
+lowerdir=$SCRATCH_DEV/$OVL_LOWER
+mkdir -p $lowerdir
 touch $lowerdir/testfile
 chown fsgqa:fsgqa $lowerdir/testfile
 
 # chown upperdir to fsgqa user, so new file/dir can be created by the user
-upperdir=$SCRATCH_DEV/$OVERLAY_UPPER_DIR
-mkdir $upperdir
+upperdir=$SCRATCH_DEV/$OVL_UPPER
+mkdir -p $upperdir
 chown fsgqa:fsgqa $upperdir
 
 _scratch_mount
tests/overlay/009 +1 -1
index de94ca4..bdac6b7 100755
@@ -54,7 +54,7 @@ _require_scratch
 _scratch_mkfs
 
 # Create test file in lowerdir
-lowerdir=$SCRATCH_DEV/$OVERLAY_LOWER_DIR
+lowerdir=$SCRATCH_DEV/$OVL_LOWER
 mkdir -p $lowerdir
 touch $lowerdir/testfile
 
tests/overlay/010 +4 -4
index a302d74..5882db9 100755
@@ -55,10 +55,10 @@ _scratch_mkfs
 
 # Need two lower dirs in this test, and we mount overlay ourselves,
 # create upper and workdir as well
-lowerdir1=$SCRATCH_DEV/$OVERLAY_LOWER_DIR.1
-lowerdir2=$SCRATCH_DEV/$OVERLAY_LOWER_DIR.2
-upperdir=$SCRATCH_DEV/$OVERLAY_UPPER_DIR
-workdir=$SCRATCH_DEV/$OVERLAY_WORK_DIR
+lowerdir1=$SCRATCH_DEV/$OVL_LOWER.1
+lowerdir2=$SCRATCH_DEV/$OVL_LOWER.2
+upperdir=$SCRATCH_DEV/$OVL_UPPER
+workdir=$SCRATCH_DEV/$OVL_WORK
 mkdir -p $lowerdir1 $lowerdir2 $upperdir $workdir
 
 # One lowerdir contains test dir and test files, the other contains whiteout
tests/overlay/011 +2 -2
index 4b697b8..1d6c44a 100755
@@ -58,8 +58,8 @@ _require_attrs
 _scratch_mkfs
 
 # Create test dir on upper and make it opaque by setting proper xattr
-lowerdir=$SCRATCH_DEV/$OVERLAY_LOWER_DIR
-upperdir=$SCRATCH_DEV/$OVERLAY_UPPER_DIR
+lowerdir=$SCRATCH_DEV/$OVL_LOWER
+upperdir=$SCRATCH_DEV/$OVL_UPPER
 mkdir -p $lowerdir/testdir
 mkdir -p $upperdir/testdir
 $SETFATTR_PROG -n "trusted.overlay.opaque" -v "y" $upperdir/testdir
tests/overlay/012 +2 -2
index cfe16f2..1c4f7c6 100755
@@ -55,8 +55,8 @@ _require_scratch
 # remove all files from previous runs
 _scratch_mkfs
 
-lowerdir=$SCRATCH_DEV/$OVERLAY_LOWER_DIR
-upperdir=$SCRATCH_DEV/$OVERLAY_UPPER_DIR
+lowerdir=$SCRATCH_DEV/$OVL_LOWER
+upperdir=$SCRATCH_DEV/$OVL_UPPER
 mkdir -p $lowerdir/test
 
 _scratch_mount
tests/overlay/013 +2 -2
index e99e10a..536e981 100755
@@ -54,8 +54,8 @@ _require_test_program "t_truncate_self"
 _scratch_mkfs
 
 # copy test program to lower and upper dir
-lowerdir=$SCRATCH_DEV/$OVERLAY_LOWER_DIR
-upperdir=$SCRATCH_DEV/$OVERLAY_UPPER_DIR
+lowerdir=$SCRATCH_DEV/$OVL_LOWER
+upperdir=$SCRATCH_DEV/$OVL_UPPER
 mkdir -p $lowerdir
 mkdir -p $upperdir
 cp $here/src/t_truncate_self $lowerdir/test_lower
tests/overlay/015 +1 -1
index c39caed..0e09e0c 100755
@@ -57,7 +57,7 @@ _scratch_mkfs
 umask 022
 
 # Create test dir in lower dir and set sgid bit
-lowerdir=$SCRATCH_DEV/$OVERLAY_LOWER_DIR
+lowerdir=$SCRATCH_DEV/$OVL_LOWER
 mkdir -p $lowerdir/dir
 chown $qa_user:$qa_group $lowerdir/dir
 chmod 2775 $lowerdir/dir
tests/overlay/016 +1 -1
index cffcde7..c678ea4 100755
@@ -57,7 +57,7 @@ rm -f $seqres.full
 _scratch_mkfs >>$seqres.full 2>&1
 
 # Create our test files.
-lowerdir=$SCRATCH_DEV/$OVERLAY_LOWER_DIR
+lowerdir=$SCRATCH_DEV/$OVL_LOWER
 mkdir -p $lowerdir
 echo "This is old news" > $lowerdir/foo
 echo "This is old news" > $lowerdir/bar
tests/overlay/017 +1 -1
index 5330de2..6b12722 100755
@@ -60,7 +60,7 @@ _scratch_mkfs >>$seqres.full 2>&1
 # Not dealing with hardlinks here, because there is more to test
 # then stable inode number.
 # Hardlinks will get a test of their own.
-lowerdir=$SCRATCH_DEV/$OVERLAY_LOWER_DIR
+lowerdir=$SCRATCH_DEV/$OVL_LOWER
 mkdir -p $lowerdir
 mkdir $lowerdir/dir
 touch $lowerdir/file
tests/overlay/018 +1 -1
index df631fc..37bf11c 100755
@@ -55,7 +55,7 @@ rm -f $seqres.full
 _scratch_mkfs >>$seqres.full 2>&1
 
 # Create 2 hardlinked files in lower
-lowerdir=$SCRATCH_DEV/$OVERLAY_LOWER_DIR
+lowerdir=$SCRATCH_DEV/$OVL_LOWER
 mkdir -p $lowerdir
 echo "patient zero" >> $lowerdir/foo
 ln $lowerdir/foo $lowerdir/bar
tests/overlay/019 +1 -1
index 41ce63b..9177acb 100755
@@ -51,7 +51,7 @@ _require_scratch
 # Remove all files from previous tests
 _scratch_mkfs
 
-lowerdir=$SCRATCH_DEV/$OVERLAY_LOWER_DIR
+lowerdir=$SCRATCH_DEV/$OVL_LOWER
 mkdir -p $lowerdir
 
 _scratch_mount
tests/overlay/020 +1 -1
index 4afd40a..3cd0f05 100755
@@ -55,7 +55,7 @@ _require_scratch
 # Remove all files from previous tests
 _scratch_mkfs
 
-lowerdir=$SCRATCH_DEV/$OVERLAY_LOWER_DIR
+lowerdir=$SCRATCH_DEV/$OVL_LOWER
 mkdir -p $lowerdir/dir
 
 _scratch_mount
tests/overlay/021 +1 -1
index c79bb01..c6d68aa 100755
@@ -60,7 +60,7 @@ _scratch_mkfs
 # conservative and reserve space for 16 data copy ups per directory.
 _require_fs_space $SCRATCH_DEV $((16*(16+32+64+128)*1024))
 
-lowerdir=$SCRATCH_DEV/$OVERLAY_LOWER_DIR
+lowerdir=$SCRATCH_DEV/$OVL_LOWER
 mkdir -p $lowerdir
 
 testdir=arena
tests/overlay/022 +1 -1
index 817ee63..46034ea 100755
@@ -59,7 +59,7 @@ _require_scratch
 # Remove all files from previous tests
 _scratch_mkfs
 
-upperdir=$SCRATCH_DEV/$OVERLAY_UPPER_DIR
+upperdir=$SCRATCH_DEV/$OVL_UPPER
 mkdir -p $upperdir/upper
 mkdir -p $upperdir/work
 # mount overlay with dirs in upper
tests/overlay/023 +2 -2
index ab49e18..c855d16 100755
@@ -63,14 +63,14 @@ _require_scratch
 _scratch_mkfs
 
 # setting acls before mount
-wkdir=$SCRATCH_DEV/$OVERLAY_WORK_DIR
+wkdir=$SCRATCH_DEV/$OVL_WORK
 mkdir -p $wkdir
 setfacl -d -m o::rwx $wkdir
 
 _scratch_mount
 
 # getting acls, ACL set on workdir should be cleaned
-getfacl -p $wkdir/work 2>&1 | _filter_scratch
+getfacl -p $wkdir/work 2>&1 | _filter_scratch | _filter_ovl_dirs
 
 # success, all done
 status=0
tests/overlay/023.out +1 -1
index a1a3c37..4a243c5 100644
@@ -1,5 +1,5 @@
 QA output created by 023
-# file: SCRATCH_DEV/work/work
+# file: SCRATCH_DEV/OVL_WORK/work
 # owner: root
 # group: root
 user::---
tests/overlay/024 +1 -1
index 2dbef12..25195f7 100755
@@ -61,7 +61,7 @@ _require_scratch
 _scratch_mkfs
 
 # making workdir
-wkdir=$SCRATCH_DEV/$OVERLAY_WORK_DIR
+wkdir=$SCRATCH_DEV/$OVL_WORK
 mkdir -p $wkdir/work/foo
 
 _scratch_mount
tests/overlay/027 +1 -1
index 4bb0329..1c77956 100755
@@ -63,7 +63,7 @@ _require_chattr
 _scratch_mkfs
 
 # Preparing immutable file
-upperdir=$SCRATCH_DEV/$OVERLAY_UPPER_DIR
+upperdir=$SCRATCH_DEV/$OVL_UPPER
 mkdir -p $upperdir
 touch $upperdir/foo
 $CHATTR_PROG +i $upperdir/foo
tests/overlay/028 +1 -1
index 6045e09..ad610e2 100755
@@ -61,7 +61,7 @@ _require_command "$FLOCK_PROG" flock
 # Remove all files from previous tests
 _scratch_mkfs
 
-lowerdir=$SCRATCH_DEV/$OVERLAY_LOWER_DIR
+lowerdir=$SCRATCH_DEV/$OVL_LOWER
 mkdir -p $lowerdir
 touch $lowerdir/foo
 
tests/overlay/029 +2 -2
index 902a84c..8dce6b3 100755
@@ -65,8 +65,8 @@ _require_scratch
 _scratch_mkfs
 
 # Preparing files
-upperdir=$SCRATCH_DEV/$OVERLAY_UPPER_DIR
-lowerdir=$SCRATCH_DEV/$OVERLAY_LOWER_DIR
+upperdir=$SCRATCH_DEV/$OVL_UPPER
+lowerdir=$SCRATCH_DEV/$OVL_LOWER
 mkdir -p $upperdir/up
 echo foo > $upperdir/up/foo
 mkdir -p $lowerdir/low
Signature · 1 line
2.7.4
#8 lore raw

PATCH v5 8/10 overlay: configure TEST/SCRATCH vars to base fs

In reply to Amir Goldstein

Instead of setting the vars TEST/SCRATCH_DEV to overlay base dirs, allow setting them to block devices to configure the base fs partition, where overlay dirs will be created.

For example, the following config file can be used to run tests on xfs test/scratch partitions:

TEST_DEV=/dev/sda5
TEST_DIR=/mnt/test
SCRATCH_DEV=/dev/sda6
SCRATCH_MNT=/mnt/scratch
FSTYP=xfs

Using the same config file, but executing './check -overlay' will use the same partitions as base fs for overlayfs directories and set TEST_DIR/SCRATCH_MNT values to overlay mount points, i.e.:
/mnt/test/ovl-mnt and /mnt/scratch/ovl-mnt.

The base fs should be pre-formatted and mounted when starting the test. An upcoming change is going to support mount/umount of base fs.

The new vars OVL_BASE_SCRATCH_MNT/TEST_DIR are set to point at the overlayfs base dirs in either legacy or new config method. Tests should always use these vars and not the legacy SCRATCH/TEST_DEV vars when referring to overlay base dir.

---
 README         |  16 ++++----
 README.overlay |  18 +++++++++
 check          |   2 +-
 common/config  | 124 +++++++++++++++++++++++++++++++++++++++++++++++++++++----
 common/rc      |  28 ++++++++-----
 5 files changed, 162 insertions(+), 26 deletions(-)
 create mode 100644 README.overlay
README +9 -7
index bb42985..842e835 100644
@@ -117,16 +117,18 @@ Running tests:
     - By default the tests suite will run xfs tests:
     - ./check '*/001' '*/002' '*/003'
     - ./check '*/06?'
-    - You can explicitly specify NFS/CIFS/UDF, otherwise the filesystem type will
-      be autodetected from $TEST_DEV:
-      ./check -nfs [test(s)]
     - Groups of tests maybe ran by: ./check -g [group(s)]
       See the 'group' file for details on groups
-    - for udf tests: ./check -udf [test(s)]
-      Running all the udf tests: ./check -udf -g udf
-    - for running nfs tests: ./check -nfs [test(s)]
-    - for running cifs/smb3 tests: ./check -cifs [test(s)]
     - To randomize test order: ./check -r [test(s)]
+    - You can explicitly specify NFS/CIFS/UDF/OVERLAY, otherwise
+      the filesystem type will be autodetected from $TEST_DEV:
+        - for running nfs tests: ./check -nfs [test(s)]
+        - for running cifs/smb3 tests: ./check -cifs [test(s)]
+        - for udf tests: ./check -udf [test(s)]
+          Running all the udf tests: ./check -udf -g udf
+        - for overlay tests: ./check -overlay [test(s)]
+          The TEST and SCRATCH partitions should be pre-formatted
+          with another base fs, where the overlay dirs will be created
 
     
     The check script tests the return value of each script, and
README.overlay new file +18 -0
new file mode 100644
index 0000000..647f9ea
@@ -0,0 +1,18 @@
+
+To run xfstest on overlayfs, configure the variables of TEST and SCRATCH
+partitions to be used as the "base fs" and run './check -overlay'.
+
+For example, the following config file can be used to run tests on
+xfs test/scratch partitions:
+
+ TEST_DEV=/dev/sda5
+ TEST_DIR=/mnt/test
+ SCRATCH_DEV=/dev/sda6
+ SCRATCH_MNT=/mnt/scratch
+ FSTYP=xfs
+
+Using the same config file, but executing './check -overlay' will
+use the same partitions as base fs for overlayfs directories
+and set TEST_DIR/SCRATCH_MNT values to overlay mount points, i.e.:
+/mnt/test/ovl-mnt and /mnt/scratch/ovl-mnt, for the context of
+individual tests.
check +1 -1
index fbd12e9..66a349a 100755
@@ -258,7 +258,7 @@ while [ $# -gt 0 ]; do
 
 	-nfs)		FSTYP=nfs ;;
 	-cifs)		FSTYP=cifs ;;
-	-overlay)	FSTYP=overlay ;;
+	-overlay)	FSTYP=overlay; export OVERLAY=true ;;
 	-tmpfs)		FSTYP=tmpfs ;;
 
 	-g)	group=$2 ; shift ;
common/config +117 -7

173 lines not on this page · raw message

common/rc +17 -11
index cca046b..fa4ca5e 100644
@@ -292,7 +292,7 @@ _scratch_mount_options()
 	_scratch_options mount
 
 	if [ "$FSTYP" == "overlay" ]; then
-		echo `_overlay_mount_options $SCRATCH_DEV`
+		echo `_overlay_mount_options $OVL_BASE_SCRATCH_MNT`
 		return 0
 	fi
 	echo `_common_dev_mount_options $*` $SCRATCH_OPTIONS \
@@ -343,9 +343,10 @@ _overlay_mkdirs()
 	mkdir -p $dir/$OVL_UPPER
 	mkdir -p $dir/$OVL_LOWER
 	mkdir -p $dir/$OVL_WORK
+	mkdir -p $dir/$OVL_MNT
 }
 
-# Given a dir, set up 3 subdirectories and mount on the given mnt.
+# Given a base fs dir, set up overlay directories and mount on the given mnt.
 # The dir is used as the mount device so it can be seen from df or mount
 _overlay_mount()
 {
@@ -364,12 +365,12 @@ _overlay_mount()
 
 _overlay_test_mount()
 {
-	_overlay_mount $TEST_DEV $TEST_DIR $*
+	_overlay_mount $OVL_BASE_TEST_DIR $TEST_DIR $*
 }
 
 _overlay_scratch_mount()
 {
-	_overlay_mount $SCRATCH_DEV $SCRATCH_MNT $*
+	_overlay_mount $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT $*
 }
 
 _overlay_test_unmount()
@@ -684,10 +685,12 @@ _scratch_cleanup_files()
 {
 	case $FSTYP in
 	overlay)
-		# $SCRATCH_DEV is a valid directory in overlay case
-		rm -rf $SCRATCH_DEV/*
+		# Avoid rm -rf /* if we messed up
+		[ -n "$OVL_BASE_SCRATCH_MNT" ] || return 1
+		rm -rf $OVL_BASE_SCRATCH_MNT/*
 		;;
 	*)
+		[ -n "$SCRATCH_MNT" ] || return 1
 		_scratch_mount
 		rm -rf $SCRATCH_MNT/*
 		_scratch_unmount
@@ -1425,10 +1428,13 @@ _require_scratch_nocheck()
 		fi
 		;;
 	overlay)
-		if [ -z "$SCRATCH_DEV" -o ! -d "$SCRATCH_DEV" ]; then
-			_notrun "this test requires a valid \$SCRATCH_DEV as ovl base dir"
+		if [ -z "$OVL_BASE_SCRATCH_MNT" -o ! -d "$OVL_BASE_SCRATCH_MNT" ]; then
+			_notrun "this test requires a valid \$OVL_BASE_SCRATCH_MNT as ovl base dir"
 		fi
-		if [ ! -d "$SCRATCH_MNT" ]; then
+		# if $SCRATCH_MNT is derived from $OVL_BASE_SCRATCH_MNT then
+		# don't check $SCRATCH_MNT dir here because base fs may not be mounted
+		# and we will create the mount point anyway on _overlay_mount
+		if [ "$SCRATCH_MNT" != "$OVL_BASE_SCRATCH_MNT/$OVL_MNT" -a ! -d "$SCRATCH_MNT" ]; then
 			_notrun "this test requires a valid \$SCRATCH_MNT"
 		fi
 		;;
@@ -1501,8 +1507,8 @@ _require_test()
 		fi
 		;;
 	overlay)
-		if [ -z "$TEST_DEV" -o ! -d "$TEST_DEV" ]; then
-			_notrun "this test requires a valid \$TEST_DEV as ovl base dir"
+		if [ -z "$OVL_BASE_TEST_DIR" -o ! -d "$OVL_BASE_TEST_DIR" ]; then
+			_notrun "this test requires a valid \$TEST_DIR as ovl base dir"
 		fi
 		if [ ! -d "$TEST_DIR" ]; then
 			_notrun "this test requires a valid \$TEST_DIR"
Signature · 1 line
2.7.4
#9 lore raw

PATCH v5 9/10 overlay: mount/unmount base fs before/after running tests

In reply to Amir Goldstein

When TEST/SCRATCH_DEV are configured to the base fs block device, use this information to mount base fs before running tests, unmount it after running tests and cycle on _test_cycle_mount along with the overlay mounts.

This helps catching overlayfs bugs related to leaking objects in underlying (base) fs.

To preserve expected tests behavior, the semantics are:
- _scratch_mkfs mounts the base fs, cleans all files, creates lower/upper dirs and keeps base fs mounted
- _scratch_mount mounts base fs (if needed) and mounts overlay
- _scratch_unmount unmounts overlay and base fs

Tests that use _scratch_unmount to unmount a custom overlay mount and expect to have access to overlay base dir, were fixed to use explicit umount $SCRATCH_MNT instead.

The overlay test itself, does not support formatting the base fs, so config options like MKFS_OPTIONS and FSCK_OPTIONS are ignored on 'check -overlay'.
The config option MOUNT_OPTIONS is used to mount the base scratch fs. The config option TEST_FS_MOUNT_OPTS is used to mount the base test fs. The config option OVERLAY_MOUNT_OPTIONS is used to mount both test and scratch overlay mounts.

---
 README.overlay    | 28 +++++++++++++++++++++++
 common/rc         | 67 +++++++++++++++++++++++++++++++++++++++++++++++++++----
 tests/overlay/003 |  3 ++-
 tests/overlay/004 |  3 ++-
 tests/overlay/014 |  5 +++--
 5 files changed, 98 insertions(+), 8 deletions(-)
README.overlay +28 -0
index 647f9ea..dfb8234 100644
@@ -16,3 +16,31 @@ use the same partitions as base fs for overlayfs directories
 and set TEST_DIR/SCRATCH_MNT values to overlay mount points, i.e.:
 /mnt/test/ovl-mnt and /mnt/scratch/ovl-mnt, for the context of
 individual tests.
+
+'./check -overlay' does not support mkfs and fsck on the base fs, so
+the base fs should be pre-formatted before starting the -overlay run.
+An easy way to accomplish this is by running './check <some test>' once,
+before running './check -overlay'.
+
+Because of the lack of mkfs support, multi-section config files are only
+partly supported with './check -overlay'. Only multi-section files that
+do not change FSTYP and MKFS_OPTIONS can be safely used with -overlay.
+
+For example, the following multi-section config file can be used to
+run overlay tests on the same base fs, but with different mount options:
+
+ [xfs]
+ TEST_DEV=/dev/sda5
+ TEST_DIR=/mnt/test
+ SCRATCH_DEV=/dev/sda6
+ SCRATCH_MNT=/mnt/scratch
+ FSTYP=xfs
+
+ [xfs_pquota]
+ MOUNT_OPTIONS="-o pquota"
+ TEST_FS_MOUNT_OPTS="-o noatime"
+ OVERLAY_MOUNT_OPTIONS="-o redirect_dir=off"
+
+In the example above, MOUNT_OPTIONS will be used to mount the base scratch fs,
+TEST_FS_MOUNT_OPTS will be used to mount the base test fs and
+OVERLAY_MOUNT_OPTIONS will be used to mount both test and scratch overlays.
common/rc +63 -4

101 lines not on this page · raw message

tests/overlay/003 +2 -1
index 5e610c8..7625273 100755
@@ -89,7 +89,8 @@ rm -rf ${SCRATCH_MNT}/*
 # nothing should be listed
 ls ${SCRATCH_MNT}/
 
-_scratch_unmount
+# unmount overlayfs but not base fs
+$UMOUNT_PROG $SCRATCH_MNT
 
 rm -rf $lowerdir
 echo "Silence is golden"
tests/overlay/004 +2 -1
index 8da5170..72b0ab3 100755
@@ -85,7 +85,8 @@ _user_do "chmod g+t ${SCRATCH_MNT}/attr_file2 > /dev/null 2>&1"
 _user_do "chmod u-X ${SCRATCH_MNT}/attr_file2 > /dev/null 2>&1"
 stat -c %a ${SCRATCH_MNT}/attr_file2
 
-_scratch_unmount
+# unmount overlayfs but not base fs
+$UMOUNT_PROG $SCRATCH_MNT
 
 # check mode bits of the file that has been copied up, and
 # the file that should not have been copied up.
tests/overlay/014 +3 -2
index 36d7077..c95a892 100755
@@ -73,7 +73,8 @@ mkdir -p $lowerdir1/testdir/d
 _overlay_mount_dirs $lowerdir1 $lowerdir2 $workdir $SCRATCH_DEV $SCRATCH_MNT
 rm -rf $SCRATCH_MNT/testdir
 mkdir -p $SCRATCH_MNT/testdir/visibledir
-_scratch_unmount
+# unmount overlayfs but not base fs
+$UMOUNT_PROG $SCRATCH_MNT
 
 # mount overlay again, with lowerdir1 and lowerdir2 as multiple lowerdirs,
 # and create a new file in testdir, triggers copyup from lowerdir,
@@ -84,7 +85,7 @@ touch $SCRATCH_MNT/testdir/visiblefile
 
 # umount and mount overlay again, buggy kernel treats the copied-up dir as
 # opaque, visibledir is not seen in merged dir.
-_scratch_unmount
+$UMOUNT_PROG $SCRATCH_MNT
 _overlay_mount_dirs "$lowerdir2:$lowerdir1" $upperdir $workdir \
 		    $SCRATCH_DEV $SCRATCH_MNT
 ls $SCRATCH_MNT/testdir
Signature · 1 line
2.7.4
#10 lore raw

PATCH v5 10/10 overlay: use OVL_BASE_SCRATCH_MNT instead of SCRATCH_DEV

In reply to Amir Goldstein

Use the new var OVL_BASE_SCRATCH_MNT to refer to overlay base dir instead of the legacy SCRATCH_DEV var.

In overlay/011, when using MOUNT_PROG directly, provide OVL_BASE_SCRATCH_MNT as device argument instead of 'none'. This fix is required to support mount of base fs.

---
 tests/overlay/001 |  7 +++----
 tests/overlay/002 |  2 +-
 tests/overlay/003 |  2 +-
 tests/overlay/004 |  4 ++--
 tests/overlay/005 | 30 +++++++++++++++---------------
 tests/overlay/006 |  8 ++++----
 tests/overlay/008 |  4 ++--
 tests/overlay/009 |  2 +-
 tests/overlay/010 | 10 +++++-----
 tests/overlay/011 |  6 +++---
 tests/overlay/012 |  4 ++--
 tests/overlay/013 |  4 ++--
 tests/overlay/014 | 14 +++++++-------
 tests/overlay/015 |  2 +-
 tests/overlay/016 |  2 +-
 tests/overlay/017 |  2 +-
 tests/overlay/018 |  2 +-
 tests/overlay/019 |  2 +-
 tests/overlay/020 |  2 +-
 tests/overlay/021 |  6 +++---
 tests/overlay/022 |  2 +-
 tests/overlay/023 |  2 +-
 tests/overlay/024 |  2 +-
 tests/overlay/027 |  2 +-
 tests/overlay/028 |  2 +-
 tests/overlay/029 |  4 ++--
 26 files changed, 64 insertions(+), 65 deletions(-)
tests/overlay/001 +3 -4
index 099ddd5..c0abbfb 100755
@@ -55,12 +55,11 @@ _require_scratch
 _scratch_mkfs
 
 # overlay copy_up doesn't deal with sparse file well, holes will be filled by
-# zeros, so at least (4G + 4G + 8k) free space is needed on $SCRATCH_DEV,
-# where $SCRATCH_DEV is actually a test dir used in overlay testing
-_require_fs_space $SCRATCH_DEV $((4*1024*1024*2 + 8))
+# zeros, so at least (4G + 4G + 8k) free space is needed on scratch base fs
+_require_fs_space $OVL_BASE_SCRATCH_MNT $((4*1024*1024*2 + 8))
 
 # Create test files with different sizes in lower dir
-lowerdir=$SCRATCH_DEV/$OVL_LOWER
+lowerdir=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER
 mkdir -p $lowerdir
 touch $lowerdir/zero_size
 $XFS_IO_PROG -fc "pwrite 0 4k" $lowerdir/less_than_4g >>$seqres.full
tests/overlay/002 +1 -1
index eaf9a91..ffdec18 100755
@@ -59,7 +59,7 @@ _require_scratch
 _scratch_mkfs
 
 # Create our test file.
-lowerdir=$SCRATCH_DEV/$OVL_LOWER
+lowerdir=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER
 mkdir -p $lowerdir
 touch $lowerdir/foobar
 
tests/overlay/003 +1 -1
index 7625273..f980edb 100755
@@ -60,7 +60,7 @@ _require_scratch
 _scratch_mkfs
 
 # Create test files dirs in lower dir
-lowerdir=${SCRATCH_DEV}/${OVL_LOWER}
+lowerdir=${OVL_BASE_SCRATCH_MNT}/${OVL_LOWER}
 mkdir -p $lowerdir
 
 touch ${lowerdir}/test_file
tests/overlay/004 +2 -2
index 72b0ab3..611847a 100755
@@ -53,8 +53,8 @@ _require_user
 _scratch_mkfs
 
 # Create test file in lower dir
-lowerdir=${SCRATCH_DEV}/${OVL_LOWER}
-upperdir=${SCRATCH_DEV}/${OVL_UPPER}
+lowerdir=${OVL_BASE_SCRATCH_MNT}/${OVL_LOWER}
+upperdir=${OVL_BASE_SCRATCH_MNT}/${OVL_UPPER}
 mkdir -p $lowerdir
 touch ${lowerdir}/attr_file1
 chmod 000 ${lowerdir}/attr_file1
tests/overlay/005 +15 -15
index baadb69..f885fdc 100755
@@ -61,31 +61,31 @@ _require_loop
 _scratch_mkfs
 
 # setup loop images fs for overlayfs
-lower_img=${SCRATCH_DEV}/${seq}.$$.img
+lower_img=${OVL_BASE_SCRATCH_MNT}/${seq}.$$.img
 $XFS_IO_PROG -f -c "truncate 120m" $lower_img >>$seqres.full 2>&1
 fs_loop_dev=`_create_loop_device $lower_img`
 $MKFS_XFS_PROG -f -n ftype=1 $fs_loop_dev >>$seqres.full 2>&1
 
 # only 20m for upper dir
-upper_img=${SCRATCH_DEV}/$$.${seq}.img
+upper_img=${OVL_BASE_SCRATCH_MNT}/$$.${seq}.img
 $XFS_IO_PROG -f -c "truncate 20m" $upper_img >>$seqres.full 2>&1
 upper_loop_dev=`_create_loop_device $upper_img`
 $MKFS_XFS_PROG -f -n ftype=1 $upper_loop_dev >>$seqres.full 2>&1
 
 # mount underlying xfs
-mkdir -p ${SCRATCH_DEV}/lowermnt
-mkdir -p ${SCRATCH_DEV}/uppermnt
-$MOUNT_PROG $fs_loop_dev ${SCRATCH_DEV}/lowermnt
-$MOUNT_PROG $upper_loop_dev ${SCRATCH_DEV}/uppermnt
+mkdir -p ${OVL_BASE_SCRATCH_MNT}/lowermnt
+mkdir -p ${OVL_BASE_SCRATCH_MNT}/uppermnt
+$MOUNT_PROG $fs_loop_dev ${OVL_BASE_SCRATCH_MNT}/lowermnt
+$MOUNT_PROG $upper_loop_dev ${OVL_BASE_SCRATCH_MNT}/uppermnt
 
 # prepare dirs
-mkdir -p ${SCRATCH_DEV}/lowermnt/lower
-mkdir -p ${SCRATCH_DEV}/uppermnt/upper
-mkdir -p ${SCRATCH_DEV}/uppermnt/work
+mkdir -p ${OVL_BASE_SCRATCH_MNT}/lowermnt/lower
+mkdir -p ${OVL_BASE_SCRATCH_MNT}/uppermnt/upper
+mkdir -p ${OVL_BASE_SCRATCH_MNT}/uppermnt/work
 
-lowerd=${SCRATCH_DEV}/lowermnt/lower
-upperd=${SCRATCH_DEV}/uppermnt/upper
-workd=${SCRATCH_DEV}/uppermnt/work
+lowerd=${OVL_BASE_SCRATCH_MNT}/lowermnt/lower
+upperd=${OVL_BASE_SCRATCH_MNT}/uppermnt/upper
+workd=${OVL_BASE_SCRATCH_MNT}/uppermnt/work
 
 # Create test file in lower dir, with too big a size for
 # upper dir to copy up.
@@ -93,7 +93,7 @@ $XFS_IO_PROG -f -c "truncate 48m" ${lowerd}/test_file \
 	>>$seqres.full 2>&1
 
 # mount new overlayfs
-_overlay_mount_dirs $lowerd $upperd $workd $SCRATCH_DEV $SCRATCH_MNT
+_overlay_mount_dirs $lowerd $upperd $workd $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT
 
 # the open call triggers copy-up and it will fail ENOSPC
 $XFS_IO_PROG -f -c "o" ${SCRATCH_MNT}/test_file \
@@ -103,8 +103,8 @@ $XFS_IO_PROG -f -c "o" ${SCRATCH_MNT}/test_file \
 $UMOUNT_PROG $SCRATCH_MNT
 
 # unmount undelying xfs, this tiggers panic if memleak happens
-$UMOUNT_PROG ${SCRATCH_DEV}/uppermnt
-$UMOUNT_PROG ${SCRATCH_DEV}/lowermnt
+$UMOUNT_PROG ${OVL_BASE_SCRATCH_MNT}/uppermnt
+$UMOUNT_PROG ${OVL_BASE_SCRATCH_MNT}/lowermnt
 
 # success, all done
 echo "Silence is golden"
tests/overlay/006 +4 -4
index 55fc2dd..815657f 100755
@@ -57,10 +57,10 @@ echo "Silence is golden"
 _scratch_mkfs
 
 # Create test file/dir before mount
-mkdir -p $SCRATCH_DEV/$OVL_LOWER
-mkdir -p $SCRATCH_DEV/$OVL_UPPER
-touch $SCRATCH_DEV/$OVL_LOWER/lowertestfile
-mkdir $SCRATCH_DEV/$OVL_UPPER/uppertestdir
+mkdir -p $OVL_BASE_SCRATCH_MNT/$OVL_LOWER
+mkdir -p $OVL_BASE_SCRATCH_MNT/$OVL_UPPER
+touch $OVL_BASE_SCRATCH_MNT/$OVL_LOWER/lowertestfile
+mkdir $OVL_BASE_SCRATCH_MNT/$OVL_UPPER/uppertestdir
 
 _scratch_mount
 
tests/overlay/008 +2 -2
index b81cff8..d5bc1df 100755
@@ -56,13 +56,13 @@ _require_user
 _scratch_mkfs
 
 # Create test file on lower dir, and chown to fsgqa user
-lowerdir=$SCRATCH_DEV/$OVL_LOWER
+lowerdir=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER
 mkdir -p $lowerdir
 touch $lowerdir/testfile
 chown fsgqa:fsgqa $lowerdir/testfile
 
 # chown upperdir to fsgqa user, so new file/dir can be created by the user
-upperdir=$SCRATCH_DEV/$OVL_UPPER
+upperdir=$OVL_BASE_SCRATCH_MNT/$OVL_UPPER
 mkdir -p $upperdir
 chown fsgqa:fsgqa $upperdir
 
tests/overlay/009 +1 -1
index bdac6b7..f367887 100755
@@ -54,7 +54,7 @@ _require_scratch
 _scratch_mkfs
 
 # Create test file in lowerdir
-lowerdir=$SCRATCH_DEV/$OVL_LOWER
+lowerdir=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER
 mkdir -p $lowerdir
 touch $lowerdir/testfile
 
tests/overlay/010 +5 -5
index 5882db9..dee99cf 100755
@@ -55,10 +55,10 @@ _scratch_mkfs
 
 # Need two lower dirs in this test, and we mount overlay ourselves,
 # create upper and workdir as well
-lowerdir1=$SCRATCH_DEV/$OVL_LOWER.1
-lowerdir2=$SCRATCH_DEV/$OVL_LOWER.2
-upperdir=$SCRATCH_DEV/$OVL_UPPER
-workdir=$SCRATCH_DEV/$OVL_WORK
+lowerdir1=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER.1
+lowerdir2=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER.2
+upperdir=$OVL_BASE_SCRATCH_MNT/$OVL_UPPER
+workdir=$OVL_BASE_SCRATCH_MNT/$OVL_WORK
 mkdir -p $lowerdir1 $lowerdir2 $upperdir $workdir
 
 # One lowerdir contains test dir and test files, the other contains whiteout
@@ -68,7 +68,7 @@ mknod $lowerdir2/testdir/a c 0 0
 
 # Mount overlayfs and remove testdir, which led to kernel crash
 _overlay_mount_dirs "$lowerdir2:$lowerdir1" $upperdir $workdir \
-		    $SCRATCH_DEV $SCRATCH_MNT
+		    $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT
 rm -rf $SCRATCH_MNT/testdir
 
 # success, all done
tests/overlay/011 +3 -3
index 1d6c44a..87ac17b 100755
@@ -58,8 +58,8 @@ _require_attrs
 _scratch_mkfs
 
 # Create test dir on upper and make it opaque by setting proper xattr
-lowerdir=$SCRATCH_DEV/$OVL_LOWER
-upperdir=$SCRATCH_DEV/$OVL_UPPER
+lowerdir=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER
+upperdir=$OVL_BASE_SCRATCH_MNT/$OVL_UPPER
 mkdir -p $lowerdir/testdir
 mkdir -p $upperdir/testdir
 $SETFATTR_PROG -n "trusted.overlay.opaque" -v "y" $upperdir/testdir
@@ -68,7 +68,7 @@ $SETFATTR_PROG -n "trusted.overlay.opaque" -v "y" $upperdir/testdir
 # $upperdir overlaid on top of $lowerdir, so that "trusted.overlay.opaque"
 # xattr should be honored and should not be listed
 # mount readonly, because there's no upper and workdir
-$MOUNT_PROG -t overlay -o ro -o lowerdir=$upperdir:$lowerdir none $SCRATCH_MNT
+$MOUNT_PROG -t overlay -o ro -o lowerdir=$upperdir:$lowerdir $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT
 
 # Dump trusted.overlay xattr, we should not see the "opaque" xattr
 $GETFATTR_PROG -d -m overlay $SCRATCH_MNT/testdir
tests/overlay/012 +2 -2
index 1c4f7c6..9cf1526 100755
@@ -55,8 +55,8 @@ _require_scratch
 # remove all files from previous runs
 _scratch_mkfs
 
-lowerdir=$SCRATCH_DEV/$OVL_LOWER
-upperdir=$SCRATCH_DEV/$OVL_UPPER
+lowerdir=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER
+upperdir=$OVL_BASE_SCRATCH_MNT/$OVL_UPPER
 mkdir -p $lowerdir/test
 
 _scratch_mount
tests/overlay/013 +2 -2
index 536e981..de0fc26 100755
@@ -54,8 +54,8 @@ _require_test_program "t_truncate_self"
 _scratch_mkfs
 
 # copy test program to lower and upper dir
-lowerdir=$SCRATCH_DEV/$OVL_LOWER
-upperdir=$SCRATCH_DEV/$OVL_UPPER
+lowerdir=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER
+upperdir=$OVL_BASE_SCRATCH_MNT/$OVL_UPPER
 mkdir -p $lowerdir
 mkdir -p $upperdir
 cp $here/src/t_truncate_self $lowerdir/test_lower
tests/overlay/014 +7 -7
index c95a892..653b7d6 100755
@@ -61,16 +61,16 @@ _require_scratch
 _scratch_mkfs
 
 # Create multiple lowerdirs and upperdir, workdir, and testdir on lowerdir
-lowerdir1=$SCRATCH_DEV/lower1
-lowerdir2=$SCRATCH_DEV/lower2
-upperdir=$SCRATCH_DEV/upper
-workdir=$SCRATCH_DEV/workdir
+lowerdir1=$OVL_BASE_SCRATCH_MNT/lower1
+lowerdir2=$OVL_BASE_SCRATCH_MNT/lower2
+upperdir=$OVL_BASE_SCRATCH_MNT/upper
+workdir=$OVL_BASE_SCRATCH_MNT/workdir
 mkdir -p $lowerdir1 $lowerdir2 $upperdir $workdir
 mkdir -p $lowerdir1/testdir/d
 
 # mount overlay with $lowerdir2 as upperdir, and remove & recreate testdir,
 # make testdir on $lowerdir2 opaque
-_overlay_mount_dirs $lowerdir1 $lowerdir2 $workdir $SCRATCH_DEV $SCRATCH_MNT
+_overlay_mount_dirs $lowerdir1 $lowerdir2 $workdir $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT
 rm -rf $SCRATCH_MNT/testdir
 mkdir -p $SCRATCH_MNT/testdir/visibledir
 # unmount overlayfs but not base fs
@@ -80,14 +80,14 @@ $UMOUNT_PROG $SCRATCH_MNT
 # and create a new file in testdir, triggers copyup from lowerdir,
 # copyup should not copy overlayfs private xattr
 _overlay_mount_dirs "$lowerdir2:$lowerdir1" $upperdir $workdir \
-		    $SCRATCH_DEV $SCRATCH_MNT
+		    $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT
 touch $SCRATCH_MNT/testdir/visiblefile
 
 # umount and mount overlay again, buggy kernel treats the copied-up dir as
 # opaque, visibledir is not seen in merged dir.
 $UMOUNT_PROG $SCRATCH_MNT
 _overlay_mount_dirs "$lowerdir2:$lowerdir1" $upperdir $workdir \
-		    $SCRATCH_DEV $SCRATCH_MNT
+		    $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT
 ls $SCRATCH_MNT/testdir
 
 # success, all done
tests/overlay/015 +1 -1
index 0e09e0c..4894097 100755
@@ -57,7 +57,7 @@ _scratch_mkfs
 umask 022
 
 # Create test dir in lower dir and set sgid bit
-lowerdir=$SCRATCH_DEV/$OVL_LOWER
+lowerdir=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER
 mkdir -p $lowerdir/dir
 chown $qa_user:$qa_group $lowerdir/dir
 chmod 2775 $lowerdir/dir
tests/overlay/016 +1 -1
index c678ea4..86a7d28 100755
@@ -57,7 +57,7 @@ rm -f $seqres.full
 _scratch_mkfs >>$seqres.full 2>&1
 
 # Create our test files.
-lowerdir=$SCRATCH_DEV/$OVL_LOWER
+lowerdir=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER
 mkdir -p $lowerdir
 echo "This is old news" > $lowerdir/foo
 echo "This is old news" > $lowerdir/bar
tests/overlay/017 +1 -1
index 6b12722..5ca3040 100755
@@ -60,7 +60,7 @@ _scratch_mkfs >>$seqres.full 2>&1
 # Not dealing with hardlinks here, because there is more to test
 # then stable inode number.
 # Hardlinks will get a test of their own.
-lowerdir=$SCRATCH_DEV/$OVL_LOWER
+lowerdir=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER
 mkdir -p $lowerdir
 mkdir $lowerdir/dir
 touch $lowerdir/file
tests/overlay/018 +1 -1
index 37bf11c..7e47732 100755
@@ -55,7 +55,7 @@ rm -f $seqres.full
 _scratch_mkfs >>$seqres.full 2>&1
 
 # Create 2 hardlinked files in lower
-lowerdir=$SCRATCH_DEV/$OVL_LOWER
+lowerdir=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER
 mkdir -p $lowerdir
 echo "patient zero" >> $lowerdir/foo
 ln $lowerdir/foo $lowerdir/bar
tests/overlay/019 +1 -1
index 9177acb..3e2bc4e 100755
@@ -51,7 +51,7 @@ _require_scratch
 # Remove all files from previous tests
 _scratch_mkfs
 
-lowerdir=$SCRATCH_DEV/$OVL_LOWER
+lowerdir=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER
 mkdir -p $lowerdir
 
 _scratch_mount
tests/overlay/020 +1 -1
index 3cd0f05..de3816c 100755
@@ -55,7 +55,7 @@ _require_scratch
 # Remove all files from previous tests
 _scratch_mkfs
 
-lowerdir=$SCRATCH_DEV/$OVL_LOWER
+lowerdir=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER
 mkdir -p $lowerdir/dir
 
 _scratch_mount
tests/overlay/021 +3 -3
index c6d68aa..1347f52 100755
@@ -54,13 +54,13 @@ _scratch_mkfs
 
 # overlay copy_up doesn't deal with sparse file well, holes will be filled by
 # zeros, so for the worst case of hitting all the copy up bomb files, we need
-# (64*(16+32+64+128)M) free space on $SCRATCH_DEV.
+# (64*(16+32+64+128)M) free space on $OVL_BASE_SCRATCH_MNT.
 # However, triggering more than a total of 16 copy up bombs would require
 # really fast data copy (clone up doesn't take up space at all), so let's be
 # conservative and reserve space for 16 data copy ups per directory.
-_require_fs_space $SCRATCH_DEV $((16*(16+32+64+128)*1024))
+_require_fs_space $OVL_BASE_SCRATCH_MNT $((16*(16+32+64+128)*1024))
 
-lowerdir=$SCRATCH_DEV/$OVL_LOWER
+lowerdir=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER
 mkdir -p $lowerdir
 
 testdir=arena
tests/overlay/022 +1 -1
index 46034ea..ef4d73f 100755
@@ -59,7 +59,7 @@ _require_scratch
 # Remove all files from previous tests
 _scratch_mkfs
 
-upperdir=$SCRATCH_DEV/$OVL_UPPER
+upperdir=$OVL_BASE_SCRATCH_MNT/$OVL_UPPER
 mkdir -p $upperdir/upper
 mkdir -p $upperdir/work
 # mount overlay with dirs in upper
tests/overlay/023 +1 -1
index c855d16..8da3c20 100755
@@ -63,7 +63,7 @@ _require_scratch
 _scratch_mkfs
 
 # setting acls before mount
-wkdir=$SCRATCH_DEV/$OVL_WORK
+wkdir=$OVL_BASE_SCRATCH_MNT/$OVL_WORK
 mkdir -p $wkdir
 setfacl -d -m o::rwx $wkdir
 
tests/overlay/024 +1 -1
index 25195f7..75bd067 100755
@@ -61,7 +61,7 @@ _require_scratch
 _scratch_mkfs
 
 # making workdir
-wkdir=$SCRATCH_DEV/$OVL_WORK
+wkdir=$OVL_BASE_SCRATCH_MNT/$OVL_WORK
 mkdir -p $wkdir/work/foo
 
 _scratch_mount
tests/overlay/027 +1 -1
index 1c77956..10111b7 100755
@@ -63,7 +63,7 @@ _require_chattr
 _scratch_mkfs
 
 # Preparing immutable file
-upperdir=$SCRATCH_DEV/$OVL_UPPER
+upperdir=$OVL_BASE_SCRATCH_MNT/$OVL_UPPER
 mkdir -p $upperdir
 touch $upperdir/foo
 $CHATTR_PROG +i $upperdir/foo
tests/overlay/028 +1 -1
index ad610e2..4b8a3a1 100755
@@ -61,7 +61,7 @@ _require_command "$FLOCK_PROG" flock
 # Remove all files from previous tests
 _scratch_mkfs
 
-lowerdir=$SCRATCH_DEV/$OVL_LOWER
+lowerdir=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER
 mkdir -p $lowerdir
 touch $lowerdir/foo
 
tests/overlay/029 +2 -2
index 8dce6b3..2ac674f 100755
@@ -65,8 +65,8 @@ _require_scratch
 _scratch_mkfs
 
 # Preparing files
-upperdir=$SCRATCH_DEV/$OVL_UPPER
-lowerdir=$SCRATCH_DEV/$OVL_LOWER
+upperdir=$OVL_BASE_SCRATCH_MNT/$OVL_UPPER
+lowerdir=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER
 mkdir -p $upperdir/up
 echo foo > $upperdir/up/foo
 mkdir -p $lowerdir/low
Signature · 1 line
2.7.4
#11 lore raw
In reply to Amir Goldstein
Quoted 4 lines · Amir Goldstein

Hi Eryu,

I rebased my branch to your master and made some minor adaptations to Xiong's new tests. Also reordered the patches:

Thanks for the resending! I was about to ask for it :)

Quoted 20 lines

- patches 1-6 are generic fixes and re-factoring - if you wish, you can merge them before the overlay config changes.

- patches 7-10 implement the new overlay config. patches 7 and 10 include adaptations of Xiong's new tests to new config.

This is the 5th revision of new overlayfs config.
The main motivation of this work is to help catch overlayfs bugs related to leaking objects in underlying (base) fs.

With this change, all you have to do to run overlay tests if you already have a local.config setup to test a local file system is:
./check -overlay

See README.overlay for some examples of config files to use with -overlay.

The legacy overlayfs configuration, where TEST_DEV is a directory still works, but it should be deprecated.

Quoted 6 lines

Tested ./check -overlay -g quick with both legacy overlay configuration and the new base fs configuration.

Eryu wished to keep these changes soaking on the list for while, so for those who can help with testing, you can get pull the branch for testing from my github tree [1].

I was planning to pull them in via this week's update, if there's no objections.

Thanks again for the work!

Eryu