Overlayfs tests for 6.7-rc1
v2 cover Overlayfs tests for 6.7-rc1
Zorro,
This update contains 3 new overlayfs tests for new features added in v6.7-rc1.
overlay/084, written by Alexander, tests the new nested xattrs feature. overlay/{085,086} test the new lowerdir+,datadir+ mount options.
overlay/086 was partly forked from overlay/083, but overlay/083 is not sensitive to libmount version, because the escaped commas test is not related to any specific mount option, so it wasn't copied over.
All the new tests do not run on older kernels.
Thanks,
Amir.
Changed since v1:
- Helper _require_scratch_overlay_xattr_escapes() already added by "overlay/026: Fix test expectation for newer kernels"
Amir Goldstein (4):
overlay: Add tests for nesting private xattrs
overlay: prepare for new lowerdir+,datadir+ tests
overlay: test data-only lowerdirs with datadir+ mount option
overlay: test parsing of lowerdir+,datadir+ mount options
common/overlay | 15 ++ tests/overlay/079 | 36 +++-- tests/overlay/084 | 169 +++++++++++++++++++++ tests/overlay/084.out | 61 ++++++++ tests/overlay/085 | 332 ++++++++++++++++++++++++++++++++++++++++++ tests/overlay/085.out | 42 ++++++ tests/overlay/086 | 81 +++++++++++ tests/overlay/086.out | 2 + 8 files changed, 723 insertions(+), 15 deletions(-) create mode 100755 tests/overlay/084 create mode 100644 tests/overlay/084.out create mode 100755 tests/overlay/085 create mode 100644 tests/overlay/085.out create mode 100755 tests/overlay/086 create mode 100644 tests/overlay/086.out
Signature · 1 line
2.34.1
PATCH v2 1/4 overlay: Add tests for nesting private xattrs
If overlayfs xattr escaping is supported, ensure:
* We can create "overlay.*" xattrs on a file in the overlayfs
* We can create an xwhiteout file in the overlayfs
We check for nesting support by trying to getattr an "overlay.*" xattr in an overlayfs mount, which will return ENOSUPP in older kernels.
--- tests/overlay/084 | 169 ++++++++++++++++++++++++++++++++++++++++++ tests/overlay/084.out | 61 +++++++++++++++ 2 files changed, 230 insertions(+) create mode 100755 tests/overlay/084 create mode 100644 tests/overlay/084.out
tests/overlay/084 new file +169 -0
@@ -0,0 +1,169 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2023 Red Hat, Inc. All Rights Reserved. +# Copyright (C) 2023 CTERA Networks. All Rights Reserved. +# +# FS QA Test No. 084 +# +# Test advanded nesting functionallity +# +. ./common/preamble +_begin_fstest auto quick nested + +# Override the default cleanup function. +_cleanup() +{ + cd / + # Unmount nested mounts if things fail + $UMOUNT_PROG $OVL_BASE_SCRATCH_MNT/nested 2>/dev/null + rm -rf $tmp +} + +# Import common functions. +. ./common/filter +. ./common/attr + +# real QA test starts here +_supported_fs overlay +# We use non-default scratch underlying overlay dirs, we need to check +# them explicity after test. +_require_scratch_nocheck +_require_scratch_overlay_xattr_escapes + +# remove all files from previous tests +_scratch_mkfs + +lowerdir=$OVL_BASE_SCRATCH_MNT/lower +middir=$OVL_BASE_SCRATCH_MNT/mid +upperdir=$OVL_BASE_SCRATCH_MNT/upper +workdir=$OVL_BASE_SCRATCH_MNT/workdir +nesteddir=$OVL_BASE_SCRATCH_MNT/nested + +umount_overlay() +{ + $UMOUNT_PROG $SCRATCH_MNT +} + +test_escape() +{ + local prefix=$1 + + echo -e "\n== Check xattr escape $prefix ==" + + local extra_options="" + if [ "$prefix" == "user" ]; then + extra_options="-o userxattr" + fi + + _scratch_mkfs + mkdir -p $lowerdir $middir $upperdir $workdir $nesteddir + + _overlay_scratch_mount_dirs $lowerdir $middir $workdir $extra_options + + mkdir -p $SCRATCH_MNT/layer1/dir/ $SCRATCH_MNT/layer2/dir + + touch $SCRATCH_MNT/layer1/dir/file + + # Make layer2/dir an opaque file + # Only one of these will be escaped, but both should succeed + setfattr -n user.overlay.opaque -v "y" $SCRATCH_MNT/layer2/dir + setfattr -n trusted.overlay.opaque -v "y" $SCRATCH_MNT/layer2/dir + + getfattr -m "overlay\\." --absolute-names -d $SCRATCH_MNT/layer2/dir | _filter_scratch + + umount_overlay + + getfattr -m "overlay\\." --absolute-names -d $middir/layer2/dir | _filter_scratch + + # Remount as lower and try again + _overlay_scratch_mount_dirs $middir:$lowerdir $upperdir $workdir $extra_options + + getfattr -m "overlay\\." --absolute-names -d $SCRATCH_MNT/layer2/dir | _filter_scratch + + # Recursively mount and ensure the opaque dir is working with both trusted and user xattrs + echo "nested xattr mount with trusted.overlay" + _overlay_mount_dirs $SCRATCH_MNT/layer2:$SCRATCH_MNT/layer1 - - overlayfs $nesteddir + stat $nesteddir/dir/file 2>&1 | _filter_scratch + $UMOUNT_PROG $nesteddir + + echo "nested xattr mount with user.overlay" + _overlay_mount_dirs $SCRATCH_MNT/layer2:$SCRATCH_MNT/layer1 - - -o userxattr overlayfs $nesteddir + stat $nesteddir/dir/file 2>&1 | _filter_scratch + $UMOUNT_PROG $nesteddir + + # Also ensure propagate the escaped xattr when we copy-up layer2/dir + echo "copy-up of escaped xattrs" + touch $SCRATCH_MNT/layer2/dir/other_file + getfattr -m "$prefix.overlay\\.overlay" --absolute-names -d $upperdir/layer2/dir | _filter_scratch + + umount_overlay +} + +test_escape trusted +test_escape user + +do_test_xwhiteout() +{ + local prefix=$1 + local basedir=$2 + + local extra_options="" + if [ "$prefix" == "user" ]; then + extra_options="-o userxattr" + fi + + mkdir -p $basedir/lower $basedir/upper $basedir/work + touch $basedir/lower/regular $basedir/lower/hidden $basedir/upper/hidden + setfattr -n $prefix.overlay.whiteouts -v "y" $basedir/upper + setfattr -n $prefix.overlay.whiteout -v "y" $basedir/upper/hidden + + # Test the hidden is invisible + _overlay_scratch_mount_dirs $basedir/upper:$basedir/lower - - $extra_options + ls $SCRATCH_MNT + stat $SCRATCH_MNT/hidden 2>&1 | _filter_scratch + umount_overlay +} + +# Validate that xwhiteouts work like whiteouts +test_xwhiteout() +{ + local prefix=$1 + + echo -e "\n== Check xwhiteout $prefix ==" + + _scratch_mkfs + + do_test_xwhiteout $prefix $OVL_BASE_SCRATCH_MNT +} + +test_xwhiteout trusted +test_xwhiteout user + +# Validate that (escaped) xwhiteouts work inside a nested overlayfs mount +test_escaped_xwhiteout() +{ + local prefix=$1 + + echo -e "\n== Check escaped xwhiteout $prefix ==" + + local extra_options="" + if [ "$prefix" == "user" ]; then + extra_options="-o userxattr" + fi + + _scratch_mkfs + mkdir -p $lowerdir $upperdir $workdir $nesteddir + + _overlay_mount_dirs $lowerdir $upperdir $workdir $extra_options overlayfs $nesteddir + + do_test_xwhiteout $prefix $nesteddir + + $UMOUNT_PROG $nesteddir +} + +test_escaped_xwhiteout trusted +test_escaped_xwhiteout user + +# success, all done +status=0 +exit
tests/overlay/084.out new file +61 -0
61 lines not on this page · raw message
Signature · 1 line
2.34.1
Re: [PATCH v2 1/4] overlay: Add tests for nesting private xattrs
Quoted 10 lines · Amir Goldstein
If overlayfs xattr escaping is supported, ensure:
* We can create "overlay.*" xattrs on a file in the overlayfs
* We can create an xwhiteout file in the overlayfsWe check for nesting support by trying to getattr an "overlay.*" xattr in an overlayfs mount, which will return ENOSUPP in older kernels.
Signed-off-by Alexander Larsson <[email protected]>Signed-off-by Amir Goldstein <[email protected]>---
Hi Amir,
This test passed with below kernel configuration at first:
CONFIG_OVERLAY_FS=m
# CONFIG_OVERLAY_FS_REDIRECT_DIR is not set CONFIG_OVERLAY_FS_REDIRECT_ALWAYS_FOLLOW=y
# CONFIG_OVERLAY_FS_INDEX is not set
# CONFIG_OVERLAY_FS_XINO_AUTO is not set
# CONFIG_OVERLAY_FS_METACOPY is not set
But then I found it fails if I enabled below configurations:
CONFIG_OVERLAY_FS_REDIRECT_DIR=y CONFIG_OVERLAY_FS_INDEX=y CONFIG_OVERLAY_FS_XINO_AUTO=y CONFIG_OVERLAY_FS_METACOPY=y
Without these configures, this test passed. But with them, it fails as [1]. The underlying fs is xfs (with default mkfs options), there're not specific MOUNT_OPTIONS and MKFS_OPTIONS to use.
I'll delay merging this patchset temporarily, please check.
Thanks,
Zorro
[1]
QA output created by 084
== Check xattr escape trusted ==
# file: SCRATCH_MNT/layer2/dir
trusted.overlay.opaque="y"
user.overlay.opaque="y"
# file: SCRATCH_DEV/mid/layer2/dir
trusted.overlay.overlay.opaque="y"
user.overlay.opaque="y"
mount: /mnt/fstests/SCRATCH_DIR/ovl-mnt: mount(2) system call failed: Stale file handle.
getfattr: /mnt/fstests/SCRATCH_DIR/ovl-mnt/layer2/dir: No such file or directory nested xattr mount with trusted.overlay
mount: /mnt/fstests/SCRATCH_DIR/nested: special device overlayfs does not exist.
stat: cannot statx 'SCRATCH_DEV/nested/dir/file': No such file or directory
umount: /mnt/fstests/SCRATCH_DIR/nested: not mounted.
nested xattr mount with user.overlay
mount: /mnt/fstests/SCRATCH_DIR/nested: special device overlayfs does not exist.
stat: cannot statx 'SCRATCH_DEV/nested/dir/file': No such file or directory
umount: /mnt/fstests/SCRATCH_DIR/nested: not mounted.
copy-up of escaped xattrs
touch: cannot touch '/mnt/fstests/SCRATCH_DIR/ovl-mnt/layer2/dir/other_file': No such file or directory
getfattr: /mnt/fstests/SCRATCH_DIR/upper/layer2/dir: No such file or directory
umount: /mnt/fstests/SCRATCH_DIR/ovl-mnt: not mounted.
== Check xattr escape user ==
# file: SCRATCH_MNT/layer2/dir
trusted.overlay.opaque="y"
user.overlay.opaque="y"
# file: SCRATCH_DEV/mid/layer2/dir
trusted.overlay.opaque="y"
user.overlay.overlay.opaque="y"
mount: /mnt/fstests/SCRATCH_DIR/ovl-mnt: mount(2) system call failed: Stale file handle.
getfattr: /mnt/fstests/SCRATCH_DIR/ovl-mnt/layer2/dir: No such file or directory nested xattr mount with trusted.overlay
mount: /mnt/fstests/SCRATCH_DIR/nested: special device overlayfs does not exist.
stat: cannot statx 'SCRATCH_DEV/nested/dir/file': No such file or directory
umount: /mnt/fstests/SCRATCH_DIR/nested: not mounted.
nested xattr mount with user.overlay
mount: /mnt/fstests/SCRATCH_DIR/nested: special device overlayfs does not exist.
stat: cannot statx 'SCRATCH_DEV/nested/dir/file': No such file or directory
umount: /mnt/fstests/SCRATCH_DIR/nested: not mounted.
copy-up of escaped xattrs
touch: cannot touch '/mnt/fstests/SCRATCH_DIR/ovl-mnt/layer2/dir/other_file': No such file or directory
getfattr: /mnt/fstests/SCRATCH_DIR/upper/layer2/dir: No such file or directory
umount: /mnt/fstests/SCRATCH_DIR/ovl-mnt: not mounted.
== Check xwhiteout trusted ==
regular
stat: cannot statx 'SCRATCH_MNT/hidden': No such file or directory
== Check xwhiteout user ==
regular
stat: cannot statx 'SCRATCH_MNT/hidden': No such file or directory
== Check escaped xwhiteout trusted ==
regular
stat: cannot statx 'SCRATCH_MNT/hidden': No such file or directory
== Check escaped xwhiteout user ==
regular
stat: cannot statx 'SCRATCH_MNT/hidden': No such file or directory
Quoted 251 lines
tests/overlay/084 | 169 ++++++++++++++++++++++++++++++++++++++++++ tests/overlay/084.out | 61 +++++++++++++++ 2 files changed, 230 insertions(+) create mode 100755 tests/overlay/084 create mode 100644 tests/overlay/084.outtests/overlay/084 new file +169
-0@@ -0,0 +1,169 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2023 Red Hat, Inc. All Rights Reserved. +# Copyright (C) 2023 CTERA Networks. All Rights Reserved. +# +# FS QA Test No. 084 +# +# Test advanded nesting functionallity +# +. ./common/preamble +_begin_fstest auto quick nested + +# Override the default cleanup function. +_cleanup() +{ + cd / + # Unmount nested mounts if things fail + $UMOUNT_PROG $OVL_BASE_SCRATCH_MNT/nested 2>/dev/null + rm -rf $tmp +} + +# Import common functions. +. ./common/filter +. ./common/attr + +# real QA test starts here +_supported_fs overlay +# We use non-default scratch underlying overlay dirs, we need to check +# them explicity after test. +_require_scratch_nocheck +_require_scratch_overlay_xattr_escapes + +# remove all files from previous tests +_scratch_mkfs + +lowerdir=$OVL_BASE_SCRATCH_MNT/lower +middir=$OVL_BASE_SCRATCH_MNT/mid +upperdir=$OVL_BASE_SCRATCH_MNT/upper +workdir=$OVL_BASE_SCRATCH_MNT/workdir +nesteddir=$OVL_BASE_SCRATCH_MNT/nested + +umount_overlay() +{ + $UMOUNT_PROG $SCRATCH_MNT +} + +test_escape() +{ + local prefix=$1 + + echo -e "\n== Check xattr escape $prefix ==" + + local extra_options="" + if [ "$prefix" == "user" ]; then + extra_options="-o userxattr" + fi + + _scratch_mkfs + mkdir -p $lowerdir $middir $upperdir $workdir $nesteddir + + _overlay_scratch_mount_dirs $lowerdir $middir $workdir $extra_options + + mkdir -p $SCRATCH_MNT/layer1/dir/ $SCRATCH_MNT/layer2/dir + + touch $SCRATCH_MNT/layer1/dir/file + + # Make layer2/dir an opaque file + # Only one of these will be escaped, but both should succeed + setfattr -n user.overlay.opaque -v "y" $SCRATCH_MNT/layer2/dir + setfattr -n trusted.overlay.opaque -v "y" $SCRATCH_MNT/layer2/dir + + getfattr -m "overlay\\." --absolute-names -d $SCRATCH_MNT/layer2/dir | _filter_scratch + + umount_overlay + + getfattr -m "overlay\\." --absolute-names -d $middir/layer2/dir | _filter_scratch + + # Remount as lower and try again + _overlay_scratch_mount_dirs $middir:$lowerdir $upperdir $workdir $extra_options + + getfattr -m "overlay\\." --absolute-names -d $SCRATCH_MNT/layer2/dir | _filter_scratch + + # Recursively mount and ensure the opaque dir is working with both trusted and user xattrs + echo "nested xattr mount with trusted.overlay" + _overlay_mount_dirs $SCRATCH_MNT/layer2:$SCRATCH_MNT/layer1 - - overlayfs $nesteddir + stat $nesteddir/dir/file 2>&1 | _filter_scratch + $UMOUNT_PROG $nesteddir + + echo "nested xattr mount with user.overlay" + _overlay_mount_dirs $SCRATCH_MNT/layer2:$SCRATCH_MNT/layer1 - - -o userxattr overlayfs $nesteddir + stat $nesteddir/dir/file 2>&1 | _filter_scratch + $UMOUNT_PROG $nesteddir + + # Also ensure propagate the escaped xattr when we copy-up layer2/dir + echo "copy-up of escaped xattrs" + touch $SCRATCH_MNT/layer2/dir/other_file + getfattr -m "$prefix.overlay\\.overlay" --absolute-names -d $upperdir/layer2/dir | _filter_scratch + + umount_overlay +} + +test_escape trusted +test_escape user + +do_test_xwhiteout() +{ + local prefix=$1 + local basedir=$2 + + local extra_options="" + if [ "$prefix" == "user" ]; then + extra_options="-o userxattr" + fi + + mkdir -p $basedir/lower $basedir/upper $basedir/work + touch $basedir/lower/regular $basedir/lower/hidden $basedir/upper/hidden + setfattr -n $prefix.overlay.whiteouts -v "y" $basedir/upper + setfattr -n $prefix.overlay.whiteout -v "y" $basedir/upper/hidden + + # Test the hidden is invisible + _overlay_scratch_mount_dirs $basedir/upper:$basedir/lower - - $extra_options + ls $SCRATCH_MNT + stat $SCRATCH_MNT/hidden 2>&1 | _filter_scratch + umount_overlay +} + +# Validate that xwhiteouts work like whiteouts +test_xwhiteout() +{ + local prefix=$1 + + echo -e "\n== Check xwhiteout $prefix ==" + + _scratch_mkfs + + do_test_xwhiteout $prefix $OVL_BASE_SCRATCH_MNT +} + +test_xwhiteout trusted +test_xwhiteout user + +# Validate that (escaped) xwhiteouts work inside a nested overlayfs mount +test_escaped_xwhiteout() +{ + local prefix=$1 + + echo -e "\n== Check escaped xwhiteout $prefix ==" + + local extra_options="" + if [ "$prefix" == "user" ]; then + extra_options="-o userxattr" + fi + + _scratch_mkfs + mkdir -p $lowerdir $upperdir $workdir $nesteddir + + _overlay_mount_dirs $lowerdir $upperdir $workdir $extra_options overlayfs $nesteddir + + do_test_xwhiteout $prefix $nesteddir + + $UMOUNT_PROG $nesteddir +} + +test_escaped_xwhiteout trusted +test_escaped_xwhiteout user + +# success, all done +status=0 +exittests/overlay/084.out new file +61
-0@@ -0,0 +1,61 @@ +QA output created by 084 + +== Check xattr escape trusted == +# file: SCRATCH_MNT/layer2/dir +trusted.overlay.opaque="y" +user.overlay.opaque="y" + +# file: SCRATCH_DEV/mid/layer2/dir +trusted.overlay.overlay.opaque="y" +user.overlay.opaque="y" + +# file: SCRATCH_MNT/layer2/dir +trusted.overlay.opaque="y" +user.overlay.opaque="y" + +nested xattr mount with trusted.overlay +stat: cannot statx 'SCRATCH_DEV/nested/dir/file': No such file or directory +nested xattr mount with user.overlay +stat: cannot statx 'SCRATCH_DEV/nested/dir/file': No such file or directory +copy-up of escaped xattrs +# file: SCRATCH_DEV/upper/layer2/dir +trusted.overlay.overlay.opaque="y" + + +== Check xattr escape user == +# file: SCRATCH_MNT/layer2/dir +trusted.overlay.opaque="y" +user.overlay.opaque="y" + +# file: SCRATCH_DEV/mid/layer2/dir +trusted.overlay.opaque="y" +user.overlay.overlay.opaque="y" + +# file: SCRATCH_MNT/layer2/dir +trusted.overlay.opaque="y" +user.overlay.opaque="y" + +nested xattr mount with trusted.overlay +stat: cannot statx 'SCRATCH_DEV/nested/dir/file': No such file or directory +nested xattr mount with user.overlay +stat: cannot statx 'SCRATCH_DEV/nested/dir/file': No such file or directory +copy-up of escaped xattrs +# file: SCRATCH_DEV/upper/layer2/dir +user.overlay.overlay.opaque="y" + + +== Check xwhiteout trusted == +regular +stat: cannot statx 'SCRATCH_MNT/hidden': No such file or directory + +== Check xwhiteout user == +regular +stat: cannot statx 'SCRATCH_MNT/hidden': No such file or directory + +== Check escaped xwhiteout trusted == +regular +stat: cannot statx 'SCRATCH_MNT/hidden': No such file or directory + +== Check escaped xwhiteout user == +regular +stat: cannot statx 'SCRATCH_MNT/hidden': No such file or directorySignature · 1 line
2.34.1
Re: [PATCH v2 1/4] overlay: Add tests for nesting private xattrs
Quoted 29 lines · Zorro Lang
Quoted 10 lines · Amir Goldstein
If overlayfs xattr escaping is supported, ensure:
* We can create "overlay.*" xattrs on a file in the overlayfs
* We can create an xwhiteout file in the overlayfsWe check for nesting support by trying to getattr an "overlay.*" xattr in an overlayfs mount, which will return ENOSUPP in older kernels.
Signed-off-by Alexander Larsson <[email protected]>Signed-off-by Amir Goldstein <[email protected]>---
Hi Amir,
This test passed with below kernel configuration at first:
CONFIG_OVERLAY_FS=m
#CONFIG_OVERLAY_FS_REDIRECT_DIRis not setCONFIG_OVERLAY_FS_REDIRECT_ALWAYS_FOLLOW=y
#CONFIG_OVERLAY_FS_INDEXis not set
#CONFIG_OVERLAY_FS_XINO_AUTOis not set
#CONFIG_OVERLAY_FS_METACOPYis not setBut then I found it fails if I enabled below configurations:
CONFIG_OVERLAY_FS_REDIRECT_DIR=yCONFIG_OVERLAY_FS_INDEX=yCONFIG_OVERLAY_FS_XINO_AUTO=yCONFIG_OVERLAY_FS_METACOPY=y
Quoted 6 lines
Without these configures, this test passed. But with them, it fails as [1]. The underlying fs is xfs (with default mkfs options), there're not specific
MOUNT_OPTIONSandMKFS_OPTIONSto use.I'll delay merging this patchset temporarily, please check.
good spotting!
Here is a fix if you want to fix and test it in your tree:
tests/overlay/084 +6 -4
@@ -50,9 +50,10 @@ test_escape() echo -e "\n== Check xattr escape $prefix ==" - local extra_options="" + # index feature would require nfs_export on $nesteddir mount + local extra_options="-o index=off" if [ "$prefix" == "user" ]; then - extra_options="-o userxattr" + extra_options+=",userxattr" fi _scratch_mkfs
@@ -146,9 +147,10 @@ test_escaped_xwhiteout() echo -e "\n== Check escaped xwhiteout $prefix ==" - local extra_options="" + # index feature would require nfs_export on $nesteddir mount + local extra_options="-o index=off" if [ "$prefix" == "user" ]; then - extra_options="-o userxattr" + extra_options+=",userxattr" fi _scratch_mkfs
Thanks,
Amir.
Re: [PATCH v2 1/4] overlay: Add tests for nesting private xattrs
Quoted 59 lines · Amir Goldstein
Quoted 29 lines · Zorro Lang
Quoted 10 lines · Amir Goldstein
If overlayfs xattr escaping is supported, ensure:
* We can create "overlay.*" xattrs on a file in the overlayfs
* We can create an xwhiteout file in the overlayfsWe check for nesting support by trying to getattr an "overlay.*" xattr in an overlayfs mount, which will return ENOSUPP in older kernels.
Signed-off-by Alexander Larsson <[email protected]>Signed-off-by Amir Goldstein <[email protected]>---
Hi Amir,
This test passed with below kernel configuration at first:
CONFIG_OVERLAY_FS=m
#CONFIG_OVERLAY_FS_REDIRECT_DIRis not setCONFIG_OVERLAY_FS_REDIRECT_ALWAYS_FOLLOW=y
#CONFIG_OVERLAY_FS_INDEXis not set
#CONFIG_OVERLAY_FS_XINO_AUTOis not set
#CONFIG_OVERLAY_FS_METACOPYis not setBut then I found it fails if I enabled below configurations:
CONFIG_OVERLAY_FS_REDIRECT_DIR=yCONFIG_OVERLAY_FS_INDEX=yCONFIG_OVERLAY_FS_XINO_AUTO=yCONFIG_OVERLAY_FS_METACOPY=yQuoted 6 lines
Without these configures, this test passed. But with them, it fails as [1]. The underlying fs is xfs (with default mkfs options), there're not specific
MOUNT_OPTIONSandMKFS_OPTIONSto use.I'll delay merging this patchset temporarily, please check.
good spotting!
Here is a fix if you want to fix and test it in your tree:
tests/overlay/084 +3
-2@@ -50,9 +50,10 @@ test_escape() echo -e "\n== Check xattr escape $prefix ==" - local extra_options="" + # index feature would require nfs_export on $nesteddir mount + local extra_options="-o index=off" if [ "$prefix" == "user" ]; then - extra_options="-o userxattr" + extra_options+=",userxattr" fi _scratch_mkfs
tests/overlay/084 test_escaped_xwhiteout() · line 153
Quoted 9 lines
tests/overlay/084 +3
-2@@ -146,9 +147,10 @@ test_escaped_xwhiteout() echo -e "\n== Check escaped xwhiteout $prefix ==" - local extra_options="" + # index feature would require nfs_export on $nesteddir mount + local extra_options="-o index=off" if [ "$prefix" == "user" ]; then - extra_options="-o userxattr" + extra_options+=",userxattr"
It works, so it's about the CONFIG_OVERLAY_FS_INDEX=y. I've releated fstests v2023.12.10 version, this patchset will be in next release. Will send a new version with this change?
Thanks,
Zorro
Quoted 8 lines
tests/overlay/084 +0
-0fi _scratch_mkfsThanks,
Amir.
Re: [PATCH v2 1/4] overlay: Add tests for nesting private xattrs
Quoted 64 lines · Zorro Lang
Quoted 62 lines · Amir Goldstein
Quoted 29 lines · Zorro Lang
Quoted 10 lines · Amir Goldstein
If overlayfs xattr escaping is supported, ensure:
* We can create "overlay.*" xattrs on a file in the overlayfs
* We can create an xwhiteout file in the overlayfsWe check for nesting support by trying to getattr an "overlay.*" xattr in an overlayfs mount, which will return ENOSUPP in older kernels.
Signed-off-by Alexander Larsson <[email protected]>Signed-off-by Amir Goldstein <[email protected]>---
Hi Amir,
This test passed with below kernel configuration at first:
CONFIG_OVERLAY_FS=m
#CONFIG_OVERLAY_FS_REDIRECT_DIRis not setCONFIG_OVERLAY_FS_REDIRECT_ALWAYS_FOLLOW=y
#CONFIG_OVERLAY_FS_INDEXis not set
#CONFIG_OVERLAY_FS_XINO_AUTOis not set
#CONFIG_OVERLAY_FS_METACOPYis not setBut then I found it fails if I enabled below configurations:
CONFIG_OVERLAY_FS_REDIRECT_DIR=yCONFIG_OVERLAY_FS_INDEX=yCONFIG_OVERLAY_FS_XINO_AUTO=yCONFIG_OVERLAY_FS_METACOPY=yQuoted 6 lines
Without these configures, this test passed. But with them, it fails as [1]. The underlying fs is xfs (with default mkfs options), there're not specific
MOUNT_OPTIONSandMKFS_OPTIONSto use.I'll delay merging this patchset temporarily, please check.
good spotting!
Here is a fix if you want to fix and test it in your tree:
tests/overlay/084 +3
-2@@ -50,9 +50,10 @@ test_escape() echo -e "\n== Check xattr escape $prefix ==" - local extra_options="" + # index feature would require nfs_export on $nesteddir mount + local extra_options="-o index=off" if [ "$prefix" == "user" ]; then - extra_options="-o userxattr" + extra_options+=",userxattr" fi _scratch_mkfs@@ -146,9 +147,10 @@ test_escaped_xwhiteout() echo -e "\n== Check escaped xwhiteout $prefix =="
Quoted 8 lines
tests/overlay/084 test_escape() · line 56
Quoted 6 lines
tests/overlay/084 +3
-2- local extra_options="" + # index feature would require nfs_export on $nesteddir mount + local extra_options="-o index=off" if [ "$prefix" == "user" ]; then - extra_options="-o userxattr" + extra_options+=",userxattr"It works, so it's about the
CONFIG_OVERLAY_FS_INDEX=y.
Yes.
the nested overlayfs setup requires that either the inner overlayfs has nfs_export enabled, as is done in tests overlay/068,069,070,071
or that the outer overlayfs has index disabled. The latter is easier for this test, because there is no need for the index feature in these test cases.
Quoted 4 lines
I've released fstests
v2023.12.10 version, this patchset will be in next release. Will send a new version with this change?
Ok, I will send a new version of test 084.
Thanks,
Amir.
PATCH v2 2/4 overlay: prepare for new lowerdir+,datadir+ tests
In reply to Amir GoldsteinIn preparation to forking tests for new lowerdir+,datadir+ mount options, prepare a helper to test kernel support and pass datadirs into mount helpers in overlay/079 test.
--- common/overlay | 15 +++++++++++++++ tests/overlay/079 | 36 +++++++++++++++++++++--------------- 2 files changed, 36 insertions(+), 15 deletions(-)
common/overlay +15 -0
@@ -247,6 +247,21 @@ _require_scratch_overlay_lowerdata_layers() _scratch_unmount } +# Check kernel support for lowerdir+=<lowerdir>,datadir+=<lowerdatadir> format +_require_scratch_overlay_lowerdir_add_layers() +{ + local lowerdir="$OVL_BASE_SCRATCH_MNT/$OVL_UPPER" + local datadir="$OVL_BASE_SCRATCH_MNT/$OVL_LOWER" + + _scratch_mkfs > /dev/null 2>&1 + $MOUNT_PROG -t overlay $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT \ + -o"lowerdir+=$lowerdir,datadir+=$datadir" \ + -o"redirect_dir=follow,metacopy=on" > /dev/null 2>&1 || \ + _notrun "overlay lowerdir+,datadir+ not supported on ${SCRATCH_DEV}" + + _scratch_unmount +} + # Helper function to check underlying dirs of overlay filesystem _overlay_fsck_dirs() {
tests/overlay/079 +21 -15
@@ -139,16 +139,21 @@ check_file_size_contents() mount_overlay() { - local _lowerdir=$1 + local _lowerdir=$1 _datadir2=$2 _datadir=$3 - _overlay_scratch_mount_dirs "$_lowerdir" $upperdir $workdir -o redirect_dir=on,index=on,metacopy=on + $MOUNT_PROG -t overlay $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT \ + -o"lowerdir=$_lowerdir::$_datadir2::$_datadir" \ + -o"upperdir=$upperdir,workdir=$workdir" \ + -o redirect_dir=on,metacopy=on } mount_ro_overlay() { - local _lowerdir=$1 + local _lowerdir=$1 _datadir2=$2 _datadir=$3 - _overlay_scratch_mount_dirs "$_lowerdir" "-" "-" -o ro,redirect_dir=follow,metacopy=on + $MOUNT_PROG -t overlay $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT \ + -o"lowerdir=$_lowerdir::$_datadir2::$_datadir" \ + -o redirect_dir=follow,metacopy=on } umount_overlay()
@@ -160,14 +165,14 @@ test_no_access() { local _target=$1 - mount_ro_overlay "$lowerdir::$datadir2::$datadir" + mount_ro_overlay "$lowerdir" "$datadir2" "$datadir" stat $SCRATCH_MNT/$_target >> $seqres.full 2>&1 || \ echo "No access to lowerdata layer $_target" echo "Unmount and Mount rw" umount_overlay - mount_overlay "$lowerdir::$datadir2::$datadir" + mount_overlay "$lowerdir" "$datadir2" "$datadir" stat $SCRATCH_MNT/$_target >> $seqres.full 2>&1 || \ echo "No access to lowerdata layer $_target" umount_overlay
@@ -175,11 +180,12 @@ test_no_access() test_common() { - local _lowerdirs=$1 _target=$2 _size=$3 _blocks=$4 _data="$5" - local _redirect=$6 + local _lowerdir=$1 _datadir2=$2 _datadir=$3 + local _target=$4 _size=$5 _blocks=$6 _data="$7" + local _redirect=$8 echo "Mount ro" - mount_ro_overlay $_lowerdirs + mount_ro_overlay $_lowerdir $_datadir2 $_datadir # Check redirect xattr to lowerdata [ -n "$_redirect" ] && check_redirect $lowerdir/$_target "$_redirect"
@@ -191,7 +197,7 @@ test_common() # Do a mount cycle and check size and contents again. echo "Unmount and Mount rw" umount_overlay - mount_overlay $_lowerdirs + mount_overlay $_lowerdir $_datadir2 $_datadir echo "check properties of metadata copied up file $_target" check_file_size_contents $SCRATCH_MNT/$_target $_size "$_data" check_file_blocks $SCRATCH_MNT/$_target $_blocks
@@ -203,7 +209,7 @@ test_common() check_file_size_contents $upperdir/$_target $_size "" # Trigger data copy up and check absence of metacopy xattr. - mount_overlay $_lowerdirs + mount_overlay $_lowerdir $_datadir2 $_datadir $XFS_IO_PROG -c "falloc 0 1" $SCRATCH_MNT/$_target >> $seqres.full echo "check properties of data copied up file $_target" check_file_size_contents $SCRATCH_MNT/$_target $_size "$_data"
@@ -216,7 +222,7 @@ test_lazy() { local _target=$1 - mount_overlay "$lowerdir::$datadir2::$datadir" + mount_overlay "$lowerdir" "$datadir2" "$datadir" # Metadata should be valid check_file_size $SCRATCH_MNT/$_target $datasize
@@ -305,12 +311,12 @@ test_no_access "$sharedname" echo -e "\n== Check follow to lowerdata layer with absolute redirect ==" prepare_midlayer "/subdir/$dataname" -test_common "$lowerdir::$datadir2::$datadir" "$dataname" $datasize $datablocks \ +test_common "$lowerdir" "$datadir2" "$datadir" "$dataname" $datasize $datablocks \ "$datacontent" "/subdir/$dataname" -test_common "$lowerdir::$datadir2::$datadir" "$dataname2" $datasize $datablocks \ +test_common "$lowerdir" "$datadir2" "$datadir" "$dataname2" $datasize $datablocks \ "$datacontent2" "/subdir/$dataname.2" # Shared file should be picked from upper datadir -test_common "$lowerdir::$datadir2::$datadir" "$sharedname" $datasize $datablocks \ +test_common "$lowerdir" "$datadir2" "$datadir" "$sharedname" $datasize $datablocks \ "$datacontent2" "/subdir/$dataname.shared" echo -e "\n== Check lazy follow to lowerdata layer =="
Signature · 1 line
2.34.1
Re: [PATCH v2 2/4] overlay: prepare for new lowerdir+,datadir+ tests
Quoted 19 lines · Amir Goldstein
In preparation to forking tests for new lowerdir+,datadir+ mount options, prepare a helper to test kernel support and pass datadirs into mount helpers in overlay/079 test.
Signed-off-by Amir Goldstein <[email protected]>--- common/overlay | 15 +++++++++++++++ tests/overlay/079 | 36 +++++++++++++++++++++--------------- 2 files changed, 36 insertions(+), 15 deletions(-)common/overlay +1
-0@@ -247,6 +247,21 @@ _require_scratch_overlay_lowerdata_layers() _scratch_unmount } +# Check kernel support for lowerdir+=<lowerdir>,datadir+=<lowerdatadir> format
common/overlay _require_scratch_overlay_lowerdata_layers() · line 260
Quoted 10 lines
common/overlay +10
-0+_require_scratch_overlay_lowerdir_add_layers() +{ + local lowerdir="$OVL_BASE_SCRATCH_MNT/$OVL_UPPER" + local datadir="$OVL_BASE_SCRATCH_MNT/$OVL_LOWER" + + _scratch_mkfs > /dev/null 2>&1 + $MOUNT_PROG -t overlay $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT \ + -o"lowerdir+=$lowerdir,datadir+=$datadir" \ + -o"redirect_dir=follow,metacopy=on" > /dev/null 2>&1 || \ + _notrun "overlay lowerdir+,datadir+ not supported on ${SCRATCH_DEV}"
Hi Amir,
I found overlay cases don't use helpers in common/overlay recently, always use raw $MOUNT_PROG directly (not only in this patchset). Although overlay supports new mount format, can we improve the mount helpers in common/overlay to support that? It would be to good to use common helpers to do common operation.
Anyway, that can be changed in another patch, if it takes too much time or you don't want to do it at here. What do you think?
Thanks,
Zorro
Quoted 116 lines
common/overlay +4
-0+ + _scratch_unmount +} + # Helper function to check underlying dirs of overlay filesystem _overlay_fsck_dirs() {tests/overlay/079 +21
-15@@ -139,16 +139,21 @@ check_file_size_contents() mount_overlay() { - local _lowerdir=$1 + local _lowerdir=$1 _datadir2=$2 _datadir=$3 - _overlay_scratch_mount_dirs "$_lowerdir" $upperdir $workdir -o redirect_dir=on,index=on,metacopy=on + $MOUNT_PROG -t overlay $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT \ + -o"lowerdir=$_lowerdir::$_datadir2::$_datadir" \ + -o"upperdir=$upperdir,workdir=$workdir" \ + -o redirect_dir=on,metacopy=on } mount_ro_overlay() { - local _lowerdir=$1 + local _lowerdir=$1 _datadir2=$2 _datadir=$3 - _overlay_scratch_mount_dirs "$_lowerdir" "-" "-" -o ro,redirect_dir=follow,metacopy=on + $MOUNT_PROG -t overlay $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT \ + -o"lowerdir=$_lowerdir::$_datadir2::$_datadir" \ + -o redirect_dir=follow,metacopy=on } umount_overlay()@@ -160,14 +165,14 @@ test_no_access() { local _target=$1 - mount_ro_overlay "$lowerdir::$datadir2::$datadir" + mount_ro_overlay "$lowerdir" "$datadir2" "$datadir" stat $SCRATCH_MNT/$_target >> $seqres.full 2>&1 || \ echo "No access to lowerdata layer $_target" echo "Unmount and Mount rw" umount_overlay - mount_overlay "$lowerdir::$datadir2::$datadir" + mount_overlay "$lowerdir" "$datadir2" "$datadir" stat $SCRATCH_MNT/$_target >> $seqres.full 2>&1 || \ echo "No access to lowerdata layer $_target" umount_overlay@@ -175,11 +180,12 @@ test_no_access() test_common() { - local _lowerdirs=$1 _target=$2 _size=$3 _blocks=$4 _data="$5" - local _redirect=$6 + local _lowerdir=$1 _datadir2=$2 _datadir=$3 + local _target=$4 _size=$5 _blocks=$6 _data="$7" + local _redirect=$8 echo "Mount ro" - mount_ro_overlay $_lowerdirs + mount_ro_overlay $_lowerdir $_datadir2 $_datadir # Check redirect xattr to lowerdata [ -n "$_redirect" ] && check_redirect $lowerdir/$_target "$_redirect"@@ -191,7 +197,7 @@ test_common() # Do a mount cycle and check size and contents again. echo "Unmount and Mount rw" umount_overlay - mount_overlay $_lowerdirs + mount_overlay $_lowerdir $_datadir2 $_datadir echo "check properties of metadata copied up file $_target" check_file_size_contents $SCRATCH_MNT/$_target $_size "$_data" check_file_blocks $SCRATCH_MNT/$_target $_blocks@@ -203,7 +209,7 @@ test_common() check_file_size_contents $upperdir/$_target $_size "" # Trigger data copy up and check absence of metacopy xattr. - mount_overlay $_lowerdirs + mount_overlay $_lowerdir $_datadir2 $_datadir $XFS_IO_PROG -c "falloc 0 1" $SCRATCH_MNT/$_target >> $seqres.full echo "check properties of data copied up file $_target" check_file_size_contents $SCRATCH_MNT/$_target $_size "$_data"@@ -216,7 +222,7 @@ test_lazy() { local _target=$1 - mount_overlay "$lowerdir::$datadir2::$datadir" + mount_overlay "$lowerdir" "$datadir2" "$datadir" # Metadata should be valid check_file_size $SCRATCH_MNT/$_target $datasize@@ -305,12 +311,12 @@ test_no_access "$sharedname" echo -e "\n== Check follow to lowerdata layer with absolute redirect ==" prepare_midlayer "/subdir/$dataname" -test_common "$lowerdir::$datadir2::$datadir" "$dataname" $datasize $datablocks \ +test_common "$lowerdir" "$datadir2" "$datadir" "$dataname" $datasize $datablocks \ "$datacontent" "/subdir/$dataname" -test_common "$lowerdir::$datadir2::$datadir" "$dataname2" $datasize $datablocks \ +test_common "$lowerdir" "$datadir2" "$datadir" "$dataname2" $datasize $datablocks \ "$datacontent2" "/subdir/$dataname.2" # Shared file should be picked from upper datadir -test_common "$lowerdir::$datadir2::$datadir" "$sharedname" $datasize $datablocks \ +test_common "$lowerdir" "$datadir2" "$datadir" "$sharedname" $datasize $datablocks \ "$datacontent2" "/subdir/$dataname.shared" echo -e "\n== Check lazy follow to lowerdata layer =="Signature · 1 line
2.34.1
Re: [PATCH v2 2/4] overlay: prepare for new lowerdir+,datadir+ tests
Quoted 34 lines · Zorro Lang
Quoted 19 lines · Amir Goldstein
In preparation to forking tests for new lowerdir+,datadir+ mount options, prepare a helper to test kernel support and pass datadirs into mount helpers in overlay/079 test.
Signed-off-by Amir Goldstein <[email protected]>--- common/overlay | 15 +++++++++++++++ tests/overlay/079 | 36 +++++++++++++++++++++--------------- 2 files changed, 36 insertions(+), 15 deletions(-)common/overlay +1
-0@@ -247,6 +247,21 @@ _require_scratch_overlay_lowerdata_layers() _scratch_unmount } +# Check kernel support for lowerdir+=<lowerdir>,datadir+=<lowerdatadir> formatcommon/overlay _require_scratch_overlay_lowerdata_layers() · line 260
Quoted 10 lines
common/overlay +10
-0+_require_scratch_overlay_lowerdir_add_layers() +{ + local lowerdir="$OVL_BASE_SCRATCH_MNT/$OVL_UPPER" + local datadir="$OVL_BASE_SCRATCH_MNT/$OVL_LOWER" + + _scratch_mkfs > /dev/null 2>&1 + $MOUNT_PROG -t overlay $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT \ + -o"lowerdir+=$lowerdir,datadir+=$datadir" \ + -o"redirect_dir=follow,metacopy=on" > /dev/null 2>&1 || \ + _notrun "overlay lowerdir+,datadir+ not supported on ${SCRATCH_DEV}"Hi Amir,
Quoted 8 lines
I found overlay cases don't use helpers in common/overlay recently, always use raw $MOUNT_PROG directly (not only in this patchset). Although overlay supports new mount format, can we improve the mount helpers in common/overlay to support that? It would be to good to use common helpers to do common operation.
Anyway, that can be changed in another patch, if it takes too much time or you don't want to do it at here. What do you think?
I agree. I wouldn't improve the existing helpers to support the new lowerdir+,datadir+ options as positional argument like in _overlay_scratch_mount_dirs(), but there is an opportunity to reduce dedupe of this common line with a helper:
# Mount with mnt/dev of scratch mount and custom mount options _overlay_scratch_mount_opts()
{
$MOUNT_PROG -t overlay $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT $*
}
I will work on this cleanup and post a patch when I get to it. No need to block this series for the cleanup.
Thanks,
Amir.
Re: [PATCH v2 2/4] overlay: prepare for new lowerdir+,datadir+ tests
Quoted 49 lines · Amir Goldstein
Quoted 34 lines · Zorro Lang
Quoted 19 lines · Amir Goldstein
In preparation to forking tests for new lowerdir+,datadir+ mount options, prepare a helper to test kernel support and pass datadirs into mount helpers in overlay/079 test.
Signed-off-by Amir Goldstein <[email protected]>--- common/overlay | 15 +++++++++++++++ tests/overlay/079 | 36 +++++++++++++++++++++--------------- 2 files changed, 36 insertions(+), 15 deletions(-)common/overlay +1
-0@@ -247,6 +247,21 @@ _require_scratch_overlay_lowerdata_layers() _scratch_unmount } +# Check kernel support for lowerdir+=<lowerdir>,datadir+=<lowerdatadir> formatcommon/overlay _require_scratch_overlay_lowerdata_layers() · line 260
Quoted 10 lines
common/overlay +10
-0+_require_scratch_overlay_lowerdir_add_layers() +{ + local lowerdir="$OVL_BASE_SCRATCH_MNT/$OVL_UPPER" + local datadir="$OVL_BASE_SCRATCH_MNT/$OVL_LOWER" + + _scratch_mkfs > /dev/null 2>&1 + $MOUNT_PROG -t overlay $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT \ + -o"lowerdir+=$lowerdir,datadir+=$datadir" \ + -o"redirect_dir=follow,metacopy=on" > /dev/null 2>&1 || \ + _notrun "overlay lowerdir+,datadir+ not supported on ${SCRATCH_DEV}"Hi Amir,
Quoted 8 lines
I found overlay cases don't use helpers in common/overlay recently, always use raw $MOUNT_PROG directly (not only in this patchset). Although overlay supports new mount format, can we improve the mount helpers in common/overlay to support that? It would be to good to use common helpers to do common operation.
Anyway, that can be changed in another patch, if it takes too much time or you don't want to do it at here. What do you think?
I agree. I wouldn't improve the existing helpers to support the new lowerdir+,datadir+ options as positional argument like in
_overlay_scratch_mount_dirs(), but there is an opportunity to reduce dedupe of this common line with a helper:
Quoted 8 lines
# Mount with mnt/dev of scratch mount and custom mount options
_overlay_scratch_mount_opts()
{
$MOUNT_PROG -t overlay $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT $*
}I will work on this cleanup and post a patch when I get to it. No need to block this series for the cleanup.
Agree, thanks for doing this!
Quoted 4 lines
Thanks,
Amir.
PATCH v2 3/4 overlay: test data-only lowerdirs with datadir+ mount option
In reply to Amir GoldsteinFork test overlay/079 to use the new lowerdir+,datadir+ mount options.
--- tests/overlay/085 | 332 ++++++++++++++++++++++++++++++++++++++++++ tests/overlay/085.out | 42 ++++++ 2 files changed, 374 insertions(+) create mode 100755 tests/overlay/085 create mode 100644 tests/overlay/085.out
tests/overlay/085 new file +332 -0
@@ -0,0 +1,332 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2018 Red Hat, Inc. All Rights Reserved. +# Copyright (C) 2023 CTERA Networks. All Rights Reserved. +# +# FS QA Test No. 085 +# +# Test data-only layers functionality. +# This is a variant of test overlay/079 with lowerdir+,datadir+ mount options +# +. ./common/preamble +_begin_fstest auto quick metacopy redirect prealloc + +# Import common functions. +. ./common/filter +. ./common/attr + +# real QA test starts here +_supported_fs overlay +# We use non-default scratch underlying overlay dirs, we need to check +# them explicity after test. +_require_scratch_nocheck +_require_scratch_overlay_features redirect_dir metacopy +_require_scratch_overlay_lowerdir_add_layers +_require_xfs_io_command "falloc" + +# remove all files from previous tests +_scratch_mkfs + +# File size on lower +dataname="datafile" +sharedname="shared" +datacontent="data" +dataname2="datafile2" +datacontent2="data2" +datasize="4096" + +# Number of blocks allocated by filesystem on lower. Will be queried later. +datarblocks="" +datarblocksize="" +estimated_datablocks="" + +udirname="pureupper" +ufile="upperfile" + +# Check metacopy xattr +check_metacopy() +{ + local target=$1 exist=$2 + local out_f target_f + local msg + + out_f=$(_getfattr --absolute-names --only-values -n \ + $OVL_XATTR_METACOPY $target 2>&1 | _filter_scratch) + + if [ "$exist" == "y" ];then + [ "$out_f" == "" ] && return + echo "Metacopy xattr does not exist on ${target}. stdout=$out_f" + return + fi + + if [ "$out_f" == "" ];then + echo "Metacopy xattr exists on ${target} unexpectedly." + return + fi + + target_f=`echo $target | _filter_scratch` + msg="$target_f: trusted.overlay.metacopy: No such attribute" + + [ "$out_f" == "$msg" ] && return + + echo "Error while checking xattr on ${target}. stdout=$out" +} + +# Check redirect xattr +check_redirect() +{ + local target=$1 + local expect=$2 + + value=$(_getfattr --absolute-names --only-values -n \ + $OVL_XATTR_REDIRECT $target) + + [[ "$value" == "$expect" ]] || echo "Redirect xattr incorrect. Expected=\"$expect\", actual=\"$value\"" +} + +# Check size +check_file_size() +{ + local target=$1 expected_size=$2 actual_size + + actual_size=$(_get_filesize $target) + + [ "$actual_size" == "$expected_size" ] || echo "Expected file size $expected_size but actual size is $actual_size" +} + +check_file_blocks() +{ + local target=$1 expected_blocks=$2 nr_blocks + + nr_blocks=$(stat -c "%b" $target) + + [ "$nr_blocks" == "$expected_blocks" ] || echo "Expected $expected_blocks blocks but actual number of blocks is ${nr_blocks}." +} + +check_file_contents() +{ + local target=$1 expected=$2 + local actual target_f + + target_f=`echo $target | _filter_scratch` + + read actual<$target + + [ "$actual" == "$expected" ] || echo "Expected file $target_f contents to be \"$expected\" but actual contents are \"$actual\"" +} + +check_no_file_contents() +{ + local target=$1 + local actual target_f out_f + + target_f=`echo $target | _filter_scratch` + out_f=`cat $target 2>&1 | _filter_scratch` + msg="cat: $target_f: No such file or directory" + + [ "$out_f" == "$msg" ] && return + + echo "$target_f unexpectedly has content" +} + + +check_file_size_contents() +{ + local target=$1 expected_size=$2 expected_content=$3 + + check_file_size $target $expected_size + check_file_contents $target $expected_content +} + +mount_overlay() +{ + local _lowerdir=$1 _datadir2=$2 _datadir=$3 + + $MOUNT_PROG -t overlay $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT \ + -o"lowerdir+=$_lowerdir,datadir+=$_datadir2,datadir+=$_datadir" \ + -o"upperdir=$upperdir,workdir=$workdir" \ + -o redirect_dir=on,metacopy=on +} + +mount_ro_overlay() +{ + local _lowerdir=$1 _datadir2=$2 _datadir=$3 + + $MOUNT_PROG -t overlay $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT \ + -o"lowerdir+=$_lowerdir,datadir+=$_datadir2,datadir+=$_datadir" \ + -o redirect_dir=follow,metacopy=on +} + +umount_overlay() +{ + $UMOUNT_PROG $SCRATCH_MNT +} + +test_no_access() +{ + local _target=$1 + + mount_ro_overlay "$lowerdir" "$datadir2" "$datadir" + + stat $SCRATCH_MNT/$_target >> $seqres.full 2>&1 || \ + echo "No access to lowerdata layer $_target" + + echo "Unmount and Mount rw" + umount_overlay + mount_overlay "$lowerdir" "$datadir2" "$datadir" + stat $SCRATCH_MNT/$_target >> $seqres.full 2>&1 || \ + echo "No access to lowerdata layer $_target" + umount_overlay +} + +test_common() +{ + local _lowerdir=$1 _datadir2=$2 _datadir=$3 + local _target=$4 _size=$5 _blocks=$6 _data="$7" + local _redirect=$8 + + echo "Mount ro" + mount_ro_overlay $_lowerdir $_datadir2 $_datadir + + # Check redirect xattr to lowerdata + [ -n "$_redirect" ] && check_redirect $lowerdir/$_target "$_redirect" + + echo "check properties of metadata copied up file $_target" + check_file_size_contents $SCRATCH_MNT/$_target $_size "$_data" + check_file_blocks $SCRATCH_MNT/$_target $_blocks + + # Do a mount cycle and check size and contents again. + echo "Unmount and Mount rw" + umount_overlay + mount_overlay $_lowerdir $_datadir2 $_datadir + echo "check properties of metadata copied up file $_target" + check_file_size_contents $SCRATCH_MNT/$_target $_size "$_data" + check_file_blocks $SCRATCH_MNT/$_target $_blocks + + # Trigger metadata copy up and check existence of metacopy xattr. + chmod 400 $SCRATCH_MNT/$_target + umount_overlay + check_metacopy $upperdir/$_target "y" + check_file_size_contents $upperdir/$_target $_size "" + + # Trigger data copy up and check absence of metacopy xattr. + mount_overlay $_lowerdir $_datadir2 $_datadir + $XFS_IO_PROG -c "falloc 0 1" $SCRATCH_MNT/$_target >> $seqres.full + echo "check properties of data copied up file $_target" + check_file_size_contents $SCRATCH_MNT/$_target $_size "$_data" + umount_overlay + check_metacopy $upperdir/$_target "n" + check_file_size_contents $upperdir/$_target $_size "$_data" +} + +test_lazy() +{ + local _target=$1 + + mount_overlay "$lowerdir" "$datadir2" "$datadir" + + # Metadata should be valid + check_file_size $SCRATCH_MNT/$_target $datasize + check_file_blocks $SCRATCH_MNT/$_target $estimated_datablocks + + # But have no content + check_no_file_contents $SCRATCH_MNT/$_target + + umount_overlay +} + +create_basic_files() +{ + _scratch_mkfs + mkdir -p $datadir/subdir $datadir2/subdir $lowerdir $lowerdir2 $upperdir $workdir $workdir2 + mkdir -p $upperdir/$udirname + echo "$datacontent" > $datadir/$dataname + chmod 600 $datadir/$dataname + echo "$datacontent2" > $datadir2/$dataname2 + chmod 600 $datadir2/$dataname2 + + echo "$datacontent" > $datadir/$sharedname + echo "$datacontent2" > $datadir2/$sharedname + chmod 600 $datadir/$sharedname $datadir2/$sharedname + + # Create files of size datasize. + for f in $datadir/$dataname $datadir2/$dataname2 $datadir/$sharedname $datadir2/$sharedname; do + $XFS_IO_PROG -c "falloc 0 $datasize" $f + $XFS_IO_PROG -c "fsync" $f + done + + # Query number of block + datablocks=$(stat -c "%b" $datadir/$dataname) + + # For lazy lookup file the block count is estimated based on size and block size + datablocksize=$(stat -c "%B" $datadir/$dataname) + estimated_datablocks=$(( ($datasize + $datablocksize - 1)/$datablocksize )) +} + +prepare_midlayer() +{ + local _redirect=$1 + + _scratch_mkfs + create_basic_files + if [ -n "$_redirect" ]; then + mv "$datadir/$dataname" "$datadir/$_redirect" + mv "$datadir2/$dataname2" "$datadir2/$_redirect.2" + mv "$datadir/$sharedname" "$datadir/$_redirect.shared" + mv "$datadir2/$sharedname" "$datadir2/$_redirect.shared" + fi + # Create midlayer + _overlay_scratch_mount_dirs $datadir2:$datadir $lowerdir $workdir2 -o redirect_dir=on,index=on,metacopy=on + # Trigger a metacopy with or without redirect + if [ -n "$_redirect" ]; then + mv "$SCRATCH_MNT/$_redirect" "$SCRATCH_MNT/$dataname" + mv "$SCRATCH_MNT/$_redirect.2" "$SCRATCH_MNT/$dataname2" + mv "$SCRATCH_MNT/$_redirect.shared" "$SCRATCH_MNT/$sharedname" + else + chmod 400 $SCRATCH_MNT/$dataname + chmod 400 $SCRATCH_MNT/$dataname2 + chmod 400 $SCRATCH_MNT/$sharedname + fi + umount_overlay +} + +# Create test directories +datadir=$OVL_BASE_SCRATCH_MNT/data +datadir2=$OVL_BASE_SCRATCH_MNT/data2 +lowerdir=$OVL_BASE_SCRATCH_MNT/lower +upperdir=$OVL_BASE_SCRATCH_MNT/upper +workdir=$OVL_BASE_SCRATCH_MNT/workdir +workdir2=$OVL_BASE_SCRATCH_MNT/workdir2 + +echo -e "\n== Check no follow to lowerdata layer without redirect ==" +prepare_midlayer +test_no_access "$dataname" +test_no_access "$dataname2" +test_no_access "$sharedname" + +echo -e "\n== Check no follow to lowerdata layer with relative redirect ==" +prepare_midlayer "$dataname.renamed" +test_no_access "$dataname" +test_no_access "$dataname2" +test_no_access "$sharedname" + +echo -e "\n== Check follow to lowerdata layer with absolute redirect ==" +prepare_midlayer "/subdir/$dataname" +test_common "$lowerdir" "$datadir2" "$datadir" "$dataname" $datasize $datablocks \ + "$datacontent" "/subdir/$dataname" +test_common "$lowerdir" "$datadir2" "$datadir" "$dataname2" $datasize $datablocks \ + "$datacontent2" "/subdir/$dataname.2" +# Shared file should be picked from upper datadir +test_common "$lowerdir" "$datadir2" "$datadir" "$sharedname" $datasize $datablocks \ + "$datacontent2" "/subdir/$dataname.shared" + +echo -e "\n== Check lazy follow to lowerdata layer ==" + +prepare_midlayer "/subdir/$dataname" +rm $datadir/subdir/$dataname +test_lazy $dataname + + +# success, all done +status=0 +exit
tests/overlay/085.out new file +42 -0
42 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH v2 4/4 overlay: test parsing of lowerdir+,datadir+ mount options
In reply to Amir GoldsteinFork test overlay/083 to test parsing of lowerdir+,datadir+ mount options.
--- tests/overlay/086 | 81 +++++++++++++++++++++++++++++++++++++++++++ tests/overlay/086.out | 2 ++ 2 files changed, 83 insertions(+) create mode 100755 tests/overlay/086 create mode 100644 tests/overlay/086.out
tests/overlay/086 new file +81 -0
@@ -0,0 +1,81 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2023 CTERA Networks. All Rights Reserved. +# +# FS QA Test 086 +# +# Test lowerdir+,datadir+ mount option restrictions. +# + +. ./common/preamble +_begin_fstest auto quick mount + +# Import common functions. +. ./common/filter + +# real QA test starts here +_supported_fs overlay + +# _overlay_check_* helpers do not handle special chars well +_require_scratch_nocheck +_require_scratch_overlay_lowerdir_add_layers + +# Remove all files from previous tests +_scratch_mkfs + +# Create lowerdirs with special characters +lowerdir_spaces="$OVL_BASE_SCRATCH_MNT/lower1 with spaces" +lowerdir_colons="$OVL_BASE_SCRATCH_MNT/lower2:with::colons" +lowerdir_colons_esc="$OVL_BASE_SCRATCH_MNT/lower2\:with\:\:colons" +lowerdir=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER +upperdir=$OVL_BASE_SCRATCH_MNT/$OVL_UPPER +workdir=$OVL_BASE_SCRATCH_MNT/$OVL_WORK +mkdir -p "$lowerdir_spaces" "$lowerdir_colons" + +# _overlay_mount_* helpers do not handle lowerdir+,datadir+, so execute mount directly. + +# check illegal combinations and order of lowerdir,lowerdir+,datadir+ +$MOUNT_PROG -t overlay none $SCRATCH_MNT \ + -o"lowerdir=$lowerdir,lowerdir+=$lowerdir_colons" \ + 2>> $seqres.full && \ + echo "ERROR: invalid combination of lowerdir and lowerdir+ mount options" + +$UMOUNT_PROG $SCRATCH_MNT 2>/dev/null + +$MOUNT_PROG -t overlay none $SCRATCH_MNT \ + -o"lowerdir=$lowerdir,datadir+=$lowerdir_colons" \ + -o redirect_dir=follow,metacopy=on 2>> $seqres.full && \ + echo "ERROR: invalid combination of lowerdir and datadir+ mount options" + +$UMOUNT_PROG $SCRATCH_MNT 2>/dev/null + +$MOUNT_PROG -t overlay none $SCRATCH_MNT \ + -o"datadir+=$lowerdir,lowerdir+=$lowerdir_colons" \ + -o redirect_dir=follow,metacopy=on 2>> $seqres.full && \ + echo "ERROR: invalid order of lowerdir+ and datadir+ mount options" + +$UMOUNT_PROG $SCRATCH_MNT 2>/dev/null + +# mount is expected to fail with escaped colons. +$MOUNT_PROG -t overlay none $SCRATCH_MNT \ + -o"lowerdir+=$lowerdir_colons_esc" \ + 2>> $seqres.full && \ + echo "ERROR: incorrect parsing of escaped colons in lowerdir+ mount option" + +$UMOUNT_PROG $SCRATCH_MNT 2>/dev/null + +# mount is expected to succeed without escaped colons. +$MOUNT_PROG -t overlay ovl_esc_test $SCRATCH_MNT \ + -o"lowerdir+=$lowerdir_colons,datadir+=$lowerdir_spaces" \ + -o redirect_dir=follow,metacopy=on \ + 2>&1 | tee -a $seqres.full + +# if spaces are not escaped when showing mount options, +# mount command will not show the word 'spaces' after the spaces +$MOUNT_PROG -t overlay | grep ovl_esc_test | tee -a $seqres.full | \ + grep -q 'datadir+'.*spaces || \ + echo "ERROR: escaped spaces truncated from datadir+ mount option" + +echo "Silence is golden" +status=0 +exit
tests/overlay/086.out new file +2 -0
@@ -0,0 +1,2 @@ +QA output created by 086 +Silence is golden
Signature · 1 line
2.34.1
Quoted 12 lines · Amir Goldstein
Zorro,
This update contains 3 new overlayfs tests for new features added in v6.7-rc1.
overlay/084, written by Alexander, tests the new nested xattrs feature. overlay/{085,086} test the new lowerdir+,datadir+ mount options.
overlay/086 was partly forked from overlay/083, but overlay/083 is not sensitive to libmount version, because the escaped commas test is not related to any specific mount option, so it wasn't copied over.
Quoted 8 lines
All the new tests do not run on older kernels.
Thanks,
Amir.Changed since v1:
- Helper_require_scratch_overlay_xattr_escapes()already added by "overlay/026: Fix test expectation for newer kernels"
This version looks good to me, let's have this feature test coverage at first.
Quoted 26 lines
Amir Goldstein (4):
overlay: Add tests for nesting private xattrs
overlay: prepare for new lowerdir+,datadir+ tests
overlay: test data-only lowerdirs with datadir+ mount option
overlay: test parsing of lowerdir+,datadir+ mount optionscommon/overlay | 15 ++ tests/overlay/079 | 36 +++-- tests/overlay/084 | 169 +++++++++++++++++++++ tests/overlay/084.out | 61 ++++++++ tests/overlay/085 | 332 ++++++++++++++++++++++++++++++++++++++++++ tests/overlay/085.out | 42 ++++++ tests/overlay/086 | 81 +++++++++++ tests/overlay/086.out | 2 + 8 files changed, 723 insertions(+), 15 deletions(-) create mode 100755 tests/overlay/084 create mode 100644 tests/overlay/084.out create mode 100755 tests/overlay/085 create mode 100644 tests/overlay/085.out create mode 100755 tests/overlay/086 create mode 100644 tests/overlay/086.outSignature · 1 line
2.34.1