overlay: add fsck.overlay basic tests
v2 cover overlay: add fsck.overlay basic tests
Hi Eryu:
This is the test cases for the fsck.overlay[1] to test orphan whiteout, invalid/duplicate reiderct dirs, and the missing impure xattr. These works only when fsck.overlay exists. (These patchset split form origin patchset[2])
This revision have already handled all comments form Amir, please review.
Thanks,
Yi.
[1] https://github.com/hisilicon/overlayfs-progs
[2] https://www.spinics.net/lists/linux-unionfs/msg03988.html
Changes since v1:
- improve _overlay_fsck_dirs helper to accept more than one options.
zhangyi (F) (4):
overlay: add filesystem check helper
overlay: add fsck.overlay whiteout test
overlay: add fsck.overlay redirect directory test
overlay: add fsck.overlay impure xattr test
common/config | 1 + common/overlay | 14 +++ tests/overlay/201 | 232 +++++++++++++++++++++++++++++++++++++++++++ tests/overlay/201.out | 10 ++ tests/overlay/202 | 269 ++++++++++++++++++++++++++++++++++++++++++++++++++ tests/overlay/202.out | 11 +++ tests/overlay/203 | 143 +++++++++++++++++++++++++++ tests/overlay/203.out | 4 + tests/overlay/group | 3 + 9 files changed, 687 insertions(+) create mode 100755 tests/overlay/201 create mode 100644 tests/overlay/201.out create mode 100755 tests/overlay/202 create mode 100644 tests/overlay/202.out create mode 100755 tests/overlay/203 create mode 100644 tests/overlay/203.out
Signature · 1 line
2.5.0
PATCH v2 1/4 overlay: add filesystem check helper
Add filesystem check helper for the upcoming fsck.overlay utility. This helper invoke fsck.overlay to check and optionally repair underlying directories of overlay filesystem.
--- common/config | 1 + common/overlay | 14 ++++++++++++++ 2 files changed, 15 insertions(+)
common/config +1 -0
@@ -236,6 +236,7 @@ case "$HOSTOS" in export MKFS_REISER4_PROG="`set_prog_path mkfs.reiser4`" export E2FSCK_PROG="`set_prog_path e2fsck`" export TUNE2FS_PROG="`set_prog_path tune2fs`" + export FSCK_OVERLAY_PROG="`set_prog_path fsck.overlay`" ;; esac
common/overlay +14 -0
@@ -151,3 +151,17 @@ _require_scratch_overlay_feature() _notrun "${FSTYP} feature '${feature}' cannot be enabled on ${SCRATCH_DEV}" _scratch_unmount } + +# Helper function to check underlying dirs of overlay filesystem +_overlay_fsck_dirs() +{ + local lowerdir=$1 + local upperdir=$2 + local workdir=$3 + shift 3 + + [[ ! -x "$FSCK_OVERLAY_PROG" ]] && return 0 + + $FSCK_OVERLAY_PROG -o lowerdir=$lowerdir -o upperdir=$upperdir \ + -o workdir=$workdir $* +}
Signature · 1 line
2.5.0
PATCH v2 2/4 overlay: add fsck.overlay whiteout test
In reply to zhangyi (F)Add fsck.overlay test case to test it how to deal with orphan/valid whiteouts in underlying directories of overlayfs.
--- tests/overlay/201 | 232 ++++++++++++++++++++++++++++++++++++++++++++++++++ tests/overlay/201.out | 10 +++ tests/overlay/group | 1 + 3 files changed, 243 insertions(+) create mode 100755 tests/overlay/201 create mode 100644 tests/overlay/201.out
tests/overlay/201 new file +232 -0
@@ -0,0 +1,232 @@ +#! /bin/bash +# FS QA Test 201 +# +# Test fsck.overlay how to deal with whiteouts in overlayfs. +# +#----------------------------------------------------------------------- +# Copyright (c) 2017 Huawei. All Rights Reserved. +# Author: zhangyi (F) <[email protected]> +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +#----------------------------------------------------------------------- +# + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter + +# remove previous $seqres.full before test +rm -f $seqres.full + +# real QA test starts here +_supported_fs overlay +_supported_os Linux +_require_scratch_nocheck +_require_command "$FSCK_OVERLAY_PROG" fsck.overlay + +# remove all files from previous tests +_scratch_mkfs + +OVL_REDIRECT_XATTR="trusted.overlay.redirect" +OVL_OPAQUE_XATTR="trusted.overlay.opaque" +OVL_OPAQUE_XATTR_VAL="y" + +# Check whiteout +check_whiteout() +{ + for arg in $* + do + local ttype=`stat -c "%F:%t,%T" $arg` + + [[ $ttype == "character special file:0,0" ]] || \ + echo "Valid whiteout removed incorrectly" + done +} + +# Create a whiteout +make_whiteout() +{ + for arg in $* + do + mknod $arg c 0 0 + done +} + +# Create an opaque directory +make_opaque_dir() +{ + local target=$1 + + mkdir -p $target + $SETFATTR_PROG -n $OVL_OPAQUE_XATTR -v $OVL_OPAQUE_XATTR_VAL $target +} + +# Create a redirect directory +make_redirect_dir() +{ + local target=$1 + local value=$2 + + mkdir -p $target + $SETFATTR_PROG -n $OVL_REDIRECT_XATTR -v $value $target +} + +# Create test directories +lowerdir=$OVL_BASE_SCRATCH_MNT/lower +lowerdir2=$OVL_BASE_SCRATCH_MNT/lower2 +upperdir=$OVL_BASE_SCRATCH_MNT/upper +workdir=$OVL_BASE_SCRATCH_MNT/workdir + +make_test_dirs() +{ + rm -rf $lowerdir $lowerdir2 $upperdir $workdir + mkdir -p $lowerdir $lowerdir2 $upperdir $workdir +} + +# Test orphan whiteout in lower and upper layer, should remove +echo "+ Orphan whiteout" +make_test_dirs +make_whiteout $lowerdir/foo $upperdir/{foo,bar} + +_overlay_fsck_dirs $lowerdir $upperdir $workdir -p >> $seqres.full 2>&1 || \ + _fail "fsck should not fail" +ls $lowerdir +ls $upperdir + +# Test valid whiteout covering lower target, should not remove +echo "+ Valid whiteout" +make_test_dirs +touch $lowerdir2/{foo,bar} +make_whiteout $upperdir/foo $lowerdir/bar + +_overlay_fsck_dirs "$lowerdir:$lowerdir2" $upperdir $workdir -p >> \ + $seqres.full 2>&1 || _fail "fsck should not fail" +check_whiteout $upperdir/foo $lowerdir/bar + +# Test orphan whiteout in opaque directory, should remove +echo "+ Orphan whiteout(2)" +make_test_dirs +mkdir $lowerdir/testdir +touch $lowerdir/testdir/foo +make_opaque_dir $upperdir/testdir +make_whiteout $upperdir/testdir/foo + +_overlay_fsck_dirs $lowerdir $upperdir $workdir -p >> $seqres.full 2>&1 || \ + _fail "fsck should not fail" +ls $upperdir/testdir + +# Test orphan whiteout whose parent path is not an merged directory, +# should remove +echo "+ Orphan whiteout(3)" +make_test_dirs +mkdir $lowerdir2/{testdir1,testdir2,testdir3} +touch $lowerdir2/{testdir1/foo,testdir2/foo,testdir3/foo} +mkdir $upperdir/{testdir1,testdir2,testdir3,testdir4} +touch $lowerdir/testdir1 +make_whiteout $lowerdir/testdir2 +make_opaque_dir $lowerdir/testdir3 +make_whiteout $upperdir/{testdir1/foo,/testdir2/foo,testdir3/foo,testdir4/foo} + +_overlay_fsck_dirs "$lowerdir:$lowerdir2" $upperdir $workdir -p >> \ + $seqres.full 2>&1 || _fail "fsck should not fail" +ls $upperdir/testdir1 +ls $upperdir/testdir2 +ls $upperdir/testdir3 +ls $upperdir/testdir4 + +# Test orphan whiteout in redirect directory, should remove +echo "+ Orphan whiteout(4)" +make_test_dirs +mkdir $lowerdir/{testdir,origin} +touch $lowerdir/testdir/foo +make_redirect_dir $upperdir/testdir "origin" +make_whiteout $upperdir/testdir/foo + +_overlay_fsck_dirs $lowerdir $upperdir $workdir -p >> $seqres.full 2>&1 || \ + _fail "fsck should not fail" +ls $upperdir/testdir + +# Test valid whiteout in redirect directory cover file in lower +# redirect origin directory, should not remove +echo "+ Valid whiteout(2)" +make_test_dirs +mkdir $lowerdir/origin +touch $lowerdir/origin/foo +make_redirect_dir $upperdir/testdir "origin" +make_whiteout $upperdir/testdir/foo + +_overlay_fsck_dirs $lowerdir $upperdir $workdir -p >> $seqres.full 2>&1 || \ + _fail "fsck should not fail" +check_whiteout $upperdir/testdir/foo + +# Test valid whiteout covering lower target whose parent directory +# merge with a redirect directory in the middle layer, should not remove. +echo "+ Valid whiteout(3)" +make_test_dirs +mkdir -p $lowerdir2/origin/subdir +touch $lowerdir2/origin/subdir/foo +make_redirect_dir $lowerdir/testdir "origin" +mkdir -p $upperdir/testdir/subdir +make_whiteout $upperdir/testdir/subdir/foo + +_overlay_fsck_dirs "$lowerdir:$lowerdir2" $upperdir $workdir -p \ + >> $seqres.full 2>&1 || _fail "fsck should not fail" +check_whiteout $upperdir/testdir/subdir/foo + +# Test invalid whiteout in opaque subdirectory in a redirect directory, +# should remove +echo "+ Orphan whiteout(5)" +make_test_dirs +mkdir -p $lowerdir/origin/subdir +touch $lowerdir/origin/subdir/foo +make_redirect_dir $upperdir/testdir "origin" +make_opaque_dir $upperdir/testdir/subdir +make_whiteout $upperdir/testdir/subdir/foo + +_overlay_fsck_dirs $lowerdir $upperdir $workdir -p >> $seqres.full 2>&1 || \ + _fail "fsck should not fail" +ls $upperdir/testdir/subdir + +# Test valid whiteout in reidrect subdirectory in a opaque directory +# covering lower target, should not remove +echo "+ Valid whiteout(4)" +make_test_dirs +mkdir $lowerdir/origin +touch $lowerdir/origin/foo +make_opaque_dir $upperdir/testdir +make_redirect_dir $upperdir/testdir/subdir "/origin" +make_whiteout $upperdir/testdir/subdir/foo + +_overlay_fsck_dirs $lowerdir $upperdir $workdir -p >> $seqres.full 2>&1 || \ + _fail "fsck should not fail" +check_whiteout $upperdir/testdir/subdir/foo + +# success, all done +status=0 +exit
tests/overlay/201.out new file +10 -0
@@ -0,0 +1,10 @@ +QA output created by 201 ++ Orphan whiteout ++ Valid whiteout ++ Orphan whiteout(2) ++ Orphan whiteout(3) ++ Orphan whiteout(4) ++ Valid whiteout(2) ++ Valid whiteout(3) ++ Orphan whiteout(5) ++ Valid whiteout(4)
tests/overlay/group +1 -0
@@ -49,3 +49,4 @@ 044 auto quick copyup hardlink nonsamefs 047 auto quick copyup hardlink 048 auto quick copyup hardlink +201 auto quick fsck
Signature · 1 line
2.5.0
Re: [xfstests PATCH v2 2/4] overlay: add fsck.overlay whiteout test
Quoted 5 lines · zhangyi (F)
Add fsck.overlay test case to test it how to deal with orphan/valid whiteouts in underlying directories of overlayfs.
Signed-off-by zhangyi (F) <[email protected]>Reviewed-by Amir Goldstein <[email protected]>
So sorry for the late review! All four patches look fine to me overall, just some minor issues inline.
Quoted 14 lines
--- tests/overlay/201 | 232 ++++++++++++++++++++++++++++++++++++++++++++++++++ tests/overlay/201.out | 10 +++ tests/overlay/group | 1 + 3 files changed, 243 insertions(+) create mode 100755 tests/overlay/201 create mode 100644 tests/overlay/201.outtests/overlay/201 new file +0
-0
tests/overlay/201 line 7
Quoted 7 lines
tests/overlay/201 new file +7
-0@@ -0,0 +1,232 @@ +#! /bin/bash +# FS QA Test 201 +# +# Test fsck.overlay how to deal with whiteouts in overlayfs. +# +#----------------------------------------------------------------------- +# Copyright (c) 2017 Huawei. All Rights Reserved.
^^^^ 2018? Other tests also have 2017.
Quoted 34 lines
tests/overlay/201 new file +34
-0+# Author: zhangyi (F) <[email protected]> +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +#----------------------------------------------------------------------- +# + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc
tests/overlay/201 line 51
Quoted 10 lines
tests/overlay/201 new file +10
-0+. ./common/filter + +# remove previous $seqres.full before test +rm -f $seqres.full + +# real QA test starts here +_supported_fs overlay +_supported_os Linux +_require_scratch_nocheck +_require_command "$FSCK_OVERLAY_PROG" fsck.overlay
I noticed that you sourced common/attr and added _require_attrs in patch 3/4, but not 2/4 and 4/4, maybe we need it in all three tests.
tests/overlay/201 line 57
Quoted 6 lines
tests/overlay/201 new file +6
-0+ +# remove all files from previous tests +_scratch_mkfs + +OVL_REDIRECT_XATTR="trusted.overlay.redirect" +OVL_OPAQUE_XATTR="trusted.overlay.opaque"
I think we can put these common ovl xattr keys (along with OVL_IMPURE_XATTR)to common/overlay
tests/overlay/201 line 64
Quoted 7 lines
tests/overlay/201 new file +7
-0+OVL_OPAQUE_XATTR_VAL="y" + +# Check whiteout +check_whiteout() +{ + for arg in $* + do
We prefer the format in fstests as
for arg in $*; do
...
done
tests/overlay/201 line 67
Quoted 3 lines
tests/overlay/201 new file +3
-0+ local ttype=`stat -c "%F:%t,%T" $arg` + + [[ $ttype == "character special file:0,0" ]] || \
I'd quote $ttype (and other check_* functions in other tests) in case stat returned nothing, then it runs into a syntax error
[[ == "character special file:0,0" ]] # bash syntax error
and with $ttype quoted
[[ "" == "character special file:0,0" ]]
it's a valid test.
Quoted 41 lines
tests/overlay/201 new file +41
-0+ echo "Valid whiteout removed incorrectly" + done +} + +# Create a whiteout +make_whiteout() +{ + for arg in $* + do + mknod $arg c 0 0 + done +} + +# Create an opaque directory +make_opaque_dir() +{ + local target=$1 + + mkdir -p $target + $SETFATTR_PROG -n $OVL_OPAQUE_XATTR -v $OVL_OPAQUE_XATTR_VAL $target +} + +# Create a redirect directory +make_redirect_dir() +{ + local target=$1 + local value=$2 + + mkdir -p $target + $SETFATTR_PROG -n $OVL_REDIRECT_XATTR -v $value $target +} + +# Create test directories +lowerdir=$OVL_BASE_SCRATCH_MNT/lower +lowerdir2=$OVL_BASE_SCRATCH_MNT/lower2 +upperdir=$OVL_BASE_SCRATCH_MNT/upper +workdir=$OVL_BASE_SCRATCH_MNT/workdir + +make_test_dirs() +{ + rm -rf $lowerdir $lowerdir2 $upperdir $workdir
tests/overlay/201 line 118
Quoted 10 lines
tests/overlay/201 new file +10
-0+ mkdir -p $lowerdir $lowerdir2 $upperdir $workdir +} + +# Test orphan whiteout in lower and upper layer, should remove +echo "+ Orphan whiteout" +make_test_dirs +make_whiteout $lowerdir/foo $upperdir/{foo,bar} + +_overlay_fsck_dirs $lowerdir $upperdir $workdir -p >> $seqres.full 2>&1 || \ + _fail "fsck should not fail"
No need to _fail, we could let test to continue, so just echo. The same is true for all other tests.
Quoted 135 lines
tests/overlay/201 new file +114
-0+ls $lowerdir +ls $upperdir + +# Test valid whiteout covering lower target, should not remove +echo "+ Valid whiteout" +make_test_dirs +touch $lowerdir2/{foo,bar} +make_whiteout $upperdir/foo $lowerdir/bar + +_overlay_fsck_dirs "$lowerdir:$lowerdir2" $upperdir $workdir -p >> \ + $seqres.full 2>&1 || _fail "fsck should not fail" +check_whiteout $upperdir/foo $lowerdir/bar + +# Test orphan whiteout in opaque directory, should remove +echo "+ Orphan whiteout(2)" +make_test_dirs +mkdir $lowerdir/testdir +touch $lowerdir/testdir/foo +make_opaque_dir $upperdir/testdir +make_whiteout $upperdir/testdir/foo + +_overlay_fsck_dirs $lowerdir $upperdir $workdir -p >> $seqres.full 2>&1 || \ + _fail "fsck should not fail" +ls $upperdir/testdir + +# Test orphan whiteout whose parent path is not an merged directory, +# should remove +echo "+ Orphan whiteout(3)" +make_test_dirs +mkdir $lowerdir2/{testdir1,testdir2,testdir3} +touch $lowerdir2/{testdir1/foo,testdir2/foo,testdir3/foo} +mkdir $upperdir/{testdir1,testdir2,testdir3,testdir4} +touch $lowerdir/testdir1 +make_whiteout $lowerdir/testdir2 +make_opaque_dir $lowerdir/testdir3 +make_whiteout $upperdir/{testdir1/foo,/testdir2/foo,testdir3/foo,testdir4/foo} + +_overlay_fsck_dirs "$lowerdir:$lowerdir2" $upperdir $workdir -p >> \ + $seqres.full 2>&1 || _fail "fsck should not fail" +ls $upperdir/testdir1 +ls $upperdir/testdir2 +ls $upperdir/testdir3 +ls $upperdir/testdir4 + +# Test orphan whiteout in redirect directory, should remove +echo "+ Orphan whiteout(4)" +make_test_dirs +mkdir $lowerdir/{testdir,origin} +touch $lowerdir/testdir/foo +make_redirect_dir $upperdir/testdir "origin" +make_whiteout $upperdir/testdir/foo + +_overlay_fsck_dirs $lowerdir $upperdir $workdir -p >> $seqres.full 2>&1 || \ + _fail "fsck should not fail" +ls $upperdir/testdir + +# Test valid whiteout in redirect directory cover file in lower +# redirect origin directory, should not remove +echo "+ Valid whiteout(2)" +make_test_dirs +mkdir $lowerdir/origin +touch $lowerdir/origin/foo +make_redirect_dir $upperdir/testdir "origin" +make_whiteout $upperdir/testdir/foo + +_overlay_fsck_dirs $lowerdir $upperdir $workdir -p >> $seqres.full 2>&1 || \ + _fail "fsck should not fail" +check_whiteout $upperdir/testdir/foo + +# Test valid whiteout covering lower target whose parent directory +# merge with a redirect directory in the middle layer, should not remove. +echo "+ Valid whiteout(3)" +make_test_dirs +mkdir -p $lowerdir2/origin/subdir +touch $lowerdir2/origin/subdir/foo +make_redirect_dir $lowerdir/testdir "origin" +mkdir -p $upperdir/testdir/subdir +make_whiteout $upperdir/testdir/subdir/foo + +_overlay_fsck_dirs "$lowerdir:$lowerdir2" $upperdir $workdir -p \ + >> $seqres.full 2>&1 || _fail "fsck should not fail" +check_whiteout $upperdir/testdir/subdir/foo + +# Test invalid whiteout in opaque subdirectory in a redirect directory, +# should remove +echo "+ Orphan whiteout(5)" +make_test_dirs +mkdir -p $lowerdir/origin/subdir +touch $lowerdir/origin/subdir/foo +make_redirect_dir $upperdir/testdir "origin" +make_opaque_dir $upperdir/testdir/subdir +make_whiteout $upperdir/testdir/subdir/foo + +_overlay_fsck_dirs $lowerdir $upperdir $workdir -p >> $seqres.full 2>&1 || \ + _fail "fsck should not fail" +ls $upperdir/testdir/subdir + +# Test valid whiteout in reidrect subdirectory in a opaque directory +# covering lower target, should not remove +echo "+ Valid whiteout(4)" +make_test_dirs +mkdir $lowerdir/origin +touch $lowerdir/origin/foo +make_opaque_dir $upperdir/testdir +make_redirect_dir $upperdir/testdir/subdir "/origin" +make_whiteout $upperdir/testdir/subdir/foo + +_overlay_fsck_dirs $lowerdir $upperdir $workdir -p >> $seqres.full 2>&1 || \ + _fail "fsck should not fail" +check_whiteout $upperdir/testdir/subdir/foo + +# success, all done +status=0 +exittests/overlay/201.out new file +10
-0@@ -0,0 +1,10 @@ +QA output created by 201 ++ Orphan whiteout ++ Valid whiteout ++ Orphan whiteout(2) ++ Orphan whiteout(3) ++ Orphan whiteout(4) ++ Valid whiteout(2) ++ Valid whiteout(3) ++ Orphan whiteout(5) ++ Valid whiteout(4)tests/overlay/group +0
-0
tests/overlay/group line 52
Quoted 4 lines
tests/overlay/group +1
-0@@ -49,3 +49,4 @@ 044 auto quick copyup hardlink nonsamefs 047 auto quick copyup hardlink 048 auto quick copyup hardlink +201 auto quick fsck
It'd be great if you could help rebase the patchset against latest master in next version. Thanks a lot!
All the comments here apply to the other two tests.
Thanks,
Eryu
Re: [xfstests PATCH v2 2/4] overlay: add fsck.overlay whiteout test
On 2018/2/6 12:22, Eryu Guan Wrote:
Quoted 10 lines · zhangyi (F)
Quoted 5 lines
Add fsck.overlay test case to test it how to deal with orphan/valid whiteouts in underlying directories of overlayfs.
Signed-off-by zhangyi (F) <[email protected]>Reviewed-by Amir Goldstein <[email protected]>So sorry for the late review! All four patches look fine to me overall, just some minor issues inline.
Thanks for your comments, I will handle them soon.
Thanks,
Yi.
PATCH v2 3/4 overlay: add fsck.overlay redirect directory test
In reply to zhangyi (F)Add fsck.overlay test case to test it how to deal with invalid/valid/ duplicate redirect xattr in underlying directories of overlayfs.
--- tests/overlay/202 | 269 ++++++++++++++++++++++++++++++++++++++++++++++++++ tests/overlay/202.out | 11 +++ tests/overlay/group | 1 + 3 files changed, 281 insertions(+) create mode 100755 tests/overlay/202 create mode 100644 tests/overlay/202.out
tests/overlay/202 new file +269 -0
@@ -0,0 +1,269 @@ +#! /bin/bash +# FS QA Test 202 +# +# Test fsck.overlay how to deal with redirect xattr in overlayfs. +# +#----------------------------------------------------------------------- +# Copyright (c) 2017 Huawei. All Rights Reserved. +# Author: zhangyi (F) <[email protected]> +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +#----------------------------------------------------------------------- +# + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter +. ./common/attr + +# remove previous $seqres.full before test +rm -f $seqres.full + +# real QA test starts here +_supported_fs overlay +_supported_os Linux +_require_scratch_nocheck +_require_attrs +_require_command "$FSCK_OVERLAY_PROG" fsck.overlay + +# remove all files from previous tests +_scratch_mkfs + +OVL_REDIRECT_XATTR="trusted.overlay.redirect" +OVL_OPAQUE_XATTR="trusted.overlay.opaque" +OVL_OPAQUE_XATTR_VAL="y" + +# Create a redirect directory +make_redirect_dir() +{ + local target=$1 + local value=$2 + + mkdir -p $target + $SETFATTR_PROG -n $OVL_REDIRECT_XATTR -v $value $target +} + +# Check redirect xattr +check_redirect() +{ + local target=$1 + local expect=$2 + + value=$($GETFATTR_PROG --absolute-names --only-values -n \ + $OVL_REDIRECT_XATTR $target) + + [[ $value == $expect ]] || echo "Redirect xattr incorrect" +} + +check_no_redirect() +{ + local target=$1 + + value=$($GETFATTR_PROG --absolute-names -d -m \ + $OVL_REDIRECT_XATTR $target) + + [[ -z "$value" ]] || echo "Redirect xattr not empty" +} + +# Check opaque xattr +check_opaque() +{ + local target=$1 + + value=$($GETFATTR_PROG --absolute-names --only-values -n \ + $OVL_OPAQUE_XATTR $target) + + [[ $value == $OVL_OPAQUE_XATTR_VAL ]] || echo "Opaque xattr incorrect" +} + +# Create a whiteout +make_whiteout() +{ + for arg in $* + do + mknod $arg c 0 0 + done +} + +# Check whiteout +check_whiteout() +{ + for arg in $* + do + local ttype=`stat -c "%F:%t,%T" $arg` + + [[ $ttype == "character special file:0,0" ]] || \ + echo "Valid whiteout removed incorrectly" + done +} + +# Create test directories +lowerdir=$OVL_BASE_SCRATCH_MNT/lower +lowerdir2=$OVL_BASE_SCRATCH_MNT/lower2 +upperdir=$OVL_BASE_SCRATCH_MNT/upper +workdir=$OVL_BASE_SCRATCH_MNT/workdir + +make_test_dirs() +{ + rm -rf $lowerdir $lowerdir2 $upperdir $workdir + mkdir -p $lowerdir $lowerdir2 $upperdir $workdir +} + +# Test invalid redirect xattr point to a nonexistent origin, should remove +echo "+ Invalid redirect" +make_test_dirs +make_redirect_dir $upperdir/testdir "invalid" + +_overlay_fsck_dirs $lowerdir $upperdir $workdir -p >> $seqres.full 2>&1 || \ + _fail "fsck should not fail" +check_no_redirect $upperdir/testdir + +# Test invalid redirect xattr point to a file origin, should remove +echo "+ Invalid redirect(2)" +make_test_dirs +touch $lowerdir/origin +make_redirect_dir $upperdir/testdir "origin" + +_overlay_fsck_dirs $lowerdir $upperdir $workdir -p >> $seqres.full 2>&1 || \ + _fail "fsck should not fail" +check_no_redirect $upperdir/testdir + +# Test valid redirect xattr point to a directory origin in the same directory, +# should not remove +echo "+ Valid redirect" +make_test_dirs +mkdir $lowerdir/origin +make_whiteout $upperdir/origin +make_redirect_dir $upperdir/testdir "origin" + +_overlay_fsck_dirs $lowerdir $upperdir $workdir -p >> $seqres.full 2>&1 || \ + _fail "fsck should not fail" +check_redirect $upperdir/testdir "origin" + +# Test valid redirect xattr point to a directory origin in different directories +# should not remove +echo "+ Valid redirect(2)" +make_test_dirs +mkdir $lowerdir/origin +make_whiteout $upperdir/origin +make_redirect_dir $upperdir/testdir1/testdir2 "/origin" + +_overlay_fsck_dirs $lowerdir $upperdir $workdir -p >> $seqres.full 2>&1 || \ + _fail "fsck should not fail" +check_redirect $upperdir/testdir1/testdir2 "/origin" + +# Test valid redirect xattr but missing whiteout to cover lower target, +# should fix whiteout +echo "+ Missing whiteout" +make_test_dirs +mkdir $lowerdir/origin +make_redirect_dir $upperdir/testdir "origin" + +_overlay_fsck_dirs $lowerdir $upperdir $workdir -p >> $seqres.full 2>&1 || \ + _fail "fsck should not fail" +check_redirect $upperdir/testdir "origin" +check_whiteout $upperdir/origin + +# Test valid redirect xattrs exchanged by rename, should not remove +echo "+ Valid redirect(3)" +make_test_dirs +mkdir $lowerdir/{testdir1,testdir2} +make_redirect_dir $upperdir/testdir1 "testdir2" +make_redirect_dir $upperdir/testdir2 "testdir1" + +_overlay_fsck_dirs $lowerdir $upperdir $workdir -p >> $seqres.full 2>&1 || \ + _fail "fsck should not fail" +check_redirect $upperdir/testdir1 "testdir2" +check_redirect $upperdir/testdir2 "testdir1" + +# Test invalid redirect xattr with lower same name directory exists, +# should remove invalid redirect xattr and set opaque in yes mode +echo "+ Invalid redirect(3)" +make_test_dirs +mkdir $lowerdir/testdir +make_redirect_dir $upperdir/testdir "invalid" + +# Question get yes answer: Should set opaque dir ? +_overlay_fsck_dirs $lowerdir $upperdir $workdir -y >> $seqres.full 2>&1 || \ + _fail "fsck should not fail" +check_no_redirect $upperdir/testdir +check_opaque $upperdir/testdir + +# Test duplicate redirect xattrs point to one origin, should fail in +# auto mode, and should remove either of the duplicates in yes mode +echo "+ Duplicate redirect" +make_test_dirs +mkdir $lowerdir2/origin +make_redirect_dir $lowerdir/testdir1 "origin" +make_redirect_dir $lowerdir/testdir2 "origin" +make_redirect_dir $upperdir/testdir3 "origin" + +_overlay_fsck_dirs "$lowerdir:$lowerdir2" $upperdir $workdir -p >> \ + $seqres.full 2>&1 && _fail "fsck should fail" + +# Question get yes answer: Duplicate redirect directory, remove xattr ? +_overlay_fsck_dirs "$lowerdir:$lowerdir2" $upperdir $workdir -y >> \ + $seqres.full 2>&1 || _fail "fsck should not fail" +redirect_1=`check_redirect $lowerdir/testdir1 "origin" 2>/dev/null` +redirect_2=`check_redirect $lowerdir/testdir2 "origin" 2>/dev/null` +[[ $redirect_1 == $redirect_2 ]] && echo "Redirect xattr incorrect" +check_no_redirect $upperdir/testdir3 + +# Test duplicate redirect xattr duplicate with merge directory, should +# fail in auto mode, and should remove the redirect xattr in yes mode +echo "+ Duplicate redirect(2)" +make_test_dirs +mkdir $lowerdir/origin $upperdir/origin +make_redirect_dir $upperdir/testdir "origin" + +_overlay_fsck_dirs $lowerdir $upperdir $workdir -p >> $seqres.full 2>&1 && \ + _fail "fsck should fail" + +# Question get yes answer: Duplicate redirect directory, remove xattr ? +_overlay_fsck_dirs $lowerdir $upperdir $workdir -y >> $seqres.full 2>&1 || \ + _fail "fsck should not fail" +check_no_redirect $upperdir/testdir + +# Test duplicate redirect xattr with lower same name directory exists, +# should remove the duplicate redirect xattr and set opaque in yes mode +echo "+ Duplicate redirect(3)" +make_test_dirs +mkdir $lowerdir/{origin,testdir} $upperdir/origin +make_redirect_dir $upperdir/testdir "invalid" + +# Question one get yes answer: Duplicate redirect directory, remove xattr? +# Question two get yes answer: Should set opaque dir ? +_overlay_fsck_dirs $lowerdir $upperdir $workdir -y >> $seqres.full 2>&1 || \ + _fail "fsck should not fail" +check_no_redirect $upperdir/testdir +check_opaque $upperdir/testdir + +# success, all done +status=0 +exit
tests/overlay/202.out new file +11 -0
@@ -0,0 +1,11 @@ +QA output created by 202 ++ Invalid redirect ++ Invalid redirect(2) ++ Valid redirect ++ Valid redirect(2) ++ Missing whiteout ++ Valid redirect(3) ++ Invalid redirect(3) ++ Duplicate redirect ++ Duplicate redirect(2) ++ Duplicate redirect(3)
tests/overlay/group +1 -0
@@ -50,3 +50,4 @@ 047 auto quick copyup hardlink 048 auto quick copyup hardlink 201 auto quick fsck +202 auto quick fsck
Signature · 1 line
2.5.0
PATCH v2 4/4 overlay: add fsck.overlay impure xattr test
In reply to zhangyi (F)Add fsck.overlay test case to test it how to deal with impure xattr in underlying directories of overlayfs.
--- tests/overlay/203 | 143 ++++++++++++++++++++++++++++++++++++++++++++++++++ tests/overlay/203.out | 4 ++ tests/overlay/group | 1 + 3 files changed, 148 insertions(+) create mode 100755 tests/overlay/203 create mode 100644 tests/overlay/203.out
tests/overlay/203 new file +143 -0
@@ -0,0 +1,143 @@ +#! /bin/bash +# FS QA Test 203 +# +# Test fsck.overlay how to deal with impure xattr in overlayfs. +# +#----------------------------------------------------------------------- +# Copyright (c) 2017 Huawei. All Rights Reserved. +# Author: zhangyi (F) <[email protected]> +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +#----------------------------------------------------------------------- +# + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter + +# remove previous $seqres.full before test +rm -f $seqres.full + +# real QA test starts here +_supported_fs overlay +_supported_os Linux +_require_scratch_nocheck +_require_command "$FSCK_OVERLAY_PROG" fsck.overlay + +# remove all files from previous tests +_scratch_mkfs + +OVL_REDIRECT_XATTR="trusted.overlay.redirect" +OVL_IMPURE_XATTR="trusted.overlay.impure" +OVL_IMPURE_XATTR_VAL="y" + +# Create a redirect directory +make_redirect_dir() +{ + local target=$1 + local value=$2 + + mkdir -p $target + $SETFATTR_PROG -n $OVL_REDIRECT_XATTR -v $value $target +} + +# Remove impure xattr +remove_impure() +{ + local target=$1 + + $SETFATTR_PROG -x $OVL_IMPURE_XATTR $target +} + +# Check impure xattr +check_impure() +{ + local target=$1 + + value=$($GETFATTR_PROG --absolute-names --only-values -n \ + $OVL_IMPURE_XATTR $target) + + [[ $value == $OVL_IMPURE_XATTR_VAL ]] || echo "Missing impure xattr" +} + +# Create test directories +lowerdir=$OVL_BASE_SCRATCH_MNT/lower +lowerdir2=$OVL_BASE_SCRATCH_MNT/lower2 +upperdir=$OVL_BASE_SCRATCH_MNT/upper +workdir=$OVL_BASE_SCRATCH_MNT/workdir + +make_test_dirs() +{ + rm -rf $lowerdir $lowerdir2 $upperdir $workdir + mkdir -p $lowerdir $lowerdir2 $upperdir $workdir +} + +# Test missing impure xattr in directory which has origin targets, should fix +echo "+ Missing impure" +make_test_dirs +mkdir $lowerdir/{testdir1,testdir2} +mkdir $upperdir/{testdir1,testdir2} +touch $lowerdir/testdir1/foo +mkdir $lowerdir/testdir2/subdir +_overlay_scratch_mount_dirs $lowerdir $upperdir $workdir +touch $SCRATCH_MNT/testdir1/foo +touch $SCRATCH_MNT/testdir2/subdir +$UMOUNT_PROG $SCRATCH_MNT +remove_impure $upperdir/testdir1 +remove_impure $upperdir/testdir2 + +_overlay_fsck_dirs $lowerdir $upperdir $workdir -p >> $seqres.full 2>&1 || \ + _fail "fsck should not fail" +check_impure $upperdir/testdir1 +check_impure $upperdir/testdir2 + +# Test missing impure xattr in directory which has redirect directories, +# should fix +echo "+ Missing impure(2)" +make_test_dirs +mkdir $lowerdir/origin +make_redirect_dir $upperdir/testdir/subdir "/origin" + +_overlay_fsck_dirs $lowerdir $upperdir $workdir -p >> $seqres.full 2>&1 || \ + _fail "fsck should not fail" +check_impure $upperdir/testdir + +# Test missing impure xattr in directory which has merge directories, +# should fix +echo "+ Missing impure(3)" +make_test_dirs +mkdir $lowerdir/testdir $upperdir/testdir + +_overlay_fsck_dirs $lowerdir $upperdir $workdir -p >> $seqres.full 2>&1 || \ + _fail "fsck should not fail" +check_impure $upperdir + +# success, all done +status=0 +exit
tests/overlay/203.out new file +4 -0
@@ -0,0 +1,4 @@ +QA output created by 203 ++ Missing impure ++ Missing impure(2) ++ Missing impure(3)
tests/overlay/group +1 -0
@@ -51,3 +51,4 @@ 048 auto quick copyup hardlink 201 auto quick fsck 202 auto quick fsck +203 auto quick fsck
Signature · 1 line
2.5.0
Hi Yi,
Quoted 8 lines · zhangyi (F)
Hi Eryu:
This is the test cases for the fsck.overlay[1] to test orphan whiteout, invalid/duplicate reiderct dirs, and the missing impure xattr. These works only when fsck.overlay exists. (These patchset split form origin patchset[2])
This revision have already handled all comments form Amir, please review.
Thanks for all the revisions and reviews from Amir! I'm sorry that I'm a bit absent from previous reviews, I kind of relying on Amir's reviews..
I'll try to go through all the fsck.overlay backgrounds and tests as quickly as I can.
Thanks,
Eryu