overlay/060: add test cases of follow to lowerdata
A message that is not in the archive (sent off-list or not yet received): [email protected]
PATCH 1/4 overlay: add helper for mounting rdonly overlay
From: Amir Goldstein <[email protected]>
Allow passing empty upperdir to _overlay_mount_dirs().
--- common/overlay | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
common/overlay +6 -2
@@ -17,15 +17,19 @@ if [ -n "$OVL_BASE_FSTYP" ];then fi # helper function to do the actual overlayfs mount operation +# accepts "-" as upperdir for non-upper overlayfs _overlay_mount_dirs() { local lowerdir=$1 local upperdir=$2 local workdir=$3 shift 3 + local diropts="-olowerdir=$lowerdir" - $MOUNT_PROG -t overlay -o lowerdir=$lowerdir -o upperdir=$upperdir \ - -o workdir=$workdir `_common_dev_mount_options $*` + [ -n "$upperdir" ] && [ "$upperdir" != "-" ] && \ + diropts+=",upperdir=$upperdir,workdir=$workdir" + + $MOUNT_PROG -t overlay $diropts `_common_dev_mount_options $*` } # Mount with same options/mnt/dev of scratch mount, but optionally
Signature · 1 line
2.40.1
Re: [PATCH 1/4] overlay: add helper for mounting rdonly overlay
Quoted 7 lines · On Thu, Jul 6, 2023 at 12:51 PM <[email protected]> wrote:
From: Amir Goldstein <[email protected]>
Allow passing empty upperdir to
_overlay_mount_dirs().Signed-off-by Amir Goldstein <[email protected]>---
Forgot to say - you need to sign off on patches that you submit on behalf of someone else, so please reply with SOB on those patches.
Quoted 32 lines
common/overlay | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)common/overlay +6
-3@@ -17,15 +17,19 @@ if [ -n "$OVL_BASE_FSTYP" ];then fi # helper function to do the actual overlayfs mount operation +# accepts "-" as upperdir for non-upper overlayfs _overlay_mount_dirs() { local lowerdir=$1 local upperdir=$2 local workdir=$3 shift 3 + local diropts="-olowerdir=$lowerdir" - $MOUNT_PROG -t overlay -o lowerdir=$lowerdir -o upperdir=$upperdir \ - -o workdir=$workdir `_common_dev_mount_options $*` + [ -n "$upperdir" ] && [ "$upperdir" != "-" ] && \ + diropts+=",upperdir=$upperdir,workdir=$workdir" + + $MOUNT_PROG -t overlay $diropts `_common_dev_mount_options $*` } # Mount with same options/mnt/dev of scratch mount, but optionally --2.40.1
Re: [PATCH 1/4] overlay: add helper for mounting rdonly overlay
In reply to [email protected]Quoted 6 lines · On Thu, Jul 6, 2023 at 11:50 AM <[email protected]> wrote:
From: Amir Goldstein <[email protected]>
Allow passing empty upperdir to
_overlay_mount_dirs().Signed-off-by Amir Goldstein <[email protected]>
Quoted 33 lines
--- common/overlay | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)common/overlay +6
-3@@ -17,15 +17,19 @@ if [ -n "$OVL_BASE_FSTYP" ];then fi # helper function to do the actual overlayfs mount operation +# accepts "-" as upperdir for non-upper overlayfs _overlay_mount_dirs() { local lowerdir=$1 local upperdir=$2 local workdir=$3 shift 3 + local diropts="-olowerdir=$lowerdir" - $MOUNT_PROG -t overlay -o lowerdir=$lowerdir -o upperdir=$upperdir \ - -o workdir=$workdir `_common_dev_mount_options $*` + [ -n "$upperdir" ] && [ "$upperdir" != "-" ] && \ + diropts+=",upperdir=$upperdir,workdir=$workdir" + + $MOUNT_PROG -t overlay $diropts `_common_dev_mount_options $*` } # Mount with same options/mnt/dev of scratch mount, but optionally --2.40.1
Signature · 3 lines
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Alexander Larsson Red Hat, Inc
[email protected] [email protected]PATCH 2/4 overlay/060: add test cases of follow to lowerdata
In reply to a message not in the archiveFrom: Amir Goldstein <[email protected]>
Add test coverage for following metacopy from lower layer to lower data with absolute, relative and no redirect.
--- tests/overlay/060 | 43 ++++++++++++++++++++++++++++++++++++++----- tests/overlay/060.out | 18 ++++++++++++++++++ 2 files changed, 56 insertions(+), 5 deletions(-)
tests/overlay/060 +38 -5
@@ -7,7 +7,7 @@ # Test metadata only copy up functionality. # . ./common/preamble -_begin_fstest auto quick metacopy prealloc +_begin_fstest auto quick metacopy redirect prealloc # Import common functions. . ./common/filter
@@ -123,6 +123,13 @@ mount_overlay() _overlay_scratch_mount_dirs "$_lowerdir" $upperdir $workdir -o redirect_dir=on,index=on,metacopy=on } +mount_ro_overlay() +{ + local _lowerdir=$1 + + _overlay_scratch_mount_dirs "$_lowerdir" "-" "-" -o ro,redirect_dir=follow,metacopy=on +} + umount_overlay() { $UMOUNT_PROG $SCRATCH_MNT
@@ -146,7 +153,8 @@ test_common() check_file_size_contents $SCRATCH_MNT/$_target $_size "$_data" check_file_blocks $SCRATCH_MNT/$_target $_blocks - # Make sure copied up file is a metacopy file. + # Trigger metadata copy up and check existence of metacopy xattr. + chmod 400 $SCRATCH_MNT/$_target umount_overlay check_metacopy $upperdir/$_target "y" check_file_size_contents $upperdir/$_target $_size ""
@@ -165,7 +173,7 @@ test_common() create_basic_files() { _scratch_mkfs - mkdir -p $lowerdir $lowerdir2 $upperdir $workdir $workdir2 + mkdir -p $lowerdir/subdir $lowerdir2 $upperdir $workdir $workdir2 mkdir -p $upperdir/$udirname echo "$lowerdata" > $lowerdir/$lowername chmod 600 $lowerdir/$lowername
@@ -184,12 +192,19 @@ create_lower_link() prepare_midlayer() { + local _redirect=$1 + _scratch_mkfs create_basic_files + [ -n "$_redirect" ] && mv "$lowerdir/$lowername" "$lowerdir/$_redirect" # Create midlayer _overlay_scratch_mount_dirs $lowerdir $lowerdir2 $workdir2 -o redirect_dir=on,index=on,metacopy=on - # Trigger a metacopy - chmod 400 $SCRATCH_MNT/$lowername + # Trigger a metacopy with or without redirect + if [ -n "$_redirect" ]; then + mv "$SCRATCH_MNT/$_redirect" "$SCRATCH_MNT/$lowername" + else + chmod 400 $SCRATCH_MNT/$lowername + fi umount_overlay }
@@ -254,6 +269,24 @@ mount_overlay $lowerdir mv $SCRATCH_MNT/$lowerlink $SCRATCH_MNT/$ufile test_common $lowerdir $ufile $lowersize $lowerblocks "$lowerdata" "/$lowerlink" +echo -e "\n== Check follow to lowerdata without redirect ==" +prepare_midlayer +mount_ro_overlay "$lowerdir2:$lowerdir" +test_common "$lowerdir2:$lowerdir" $lowername $lowersize $lowerblocks \ + "$lowerdata" + +echo -e "\n== Check follow to lowerdata with relative redirect ==" +prepare_midlayer "$lowername.renamed" +mount_ro_overlay "$lowerdir2:$lowerdir" +test_common "$lowerdir2:$lowerdir" "$lowername" $lowersize $lowerblocks \ + "$lowerdata" + +echo -e "\n== Check follow to lowerdata with absolute redirect ==" +prepare_midlayer "/subdir/$lowername" +mount_ro_overlay "$lowerdir2:$lowerdir" +test_common "$lowerdir2:$lowerdir" "$lowername" $lowersize $lowerblocks \ + "$lowerdata" + # success, all done status=0 exit
tests/overlay/060.out +18 -0
@@ -40,3 +40,21 @@ check properties of metadata copied up file Unmount and Mount again check properties of metadata copied up file check properties of data copied up file + +== Check follow to lowerdata without redirect == +check properties of metadata copied up file +Unmount and Mount again +check properties of metadata copied up file +check properties of data copied up file + +== Check follow to lowerdata with relative redirect == +check properties of metadata copied up file +Unmount and Mount again +check properties of metadata copied up file +check properties of data copied up file + +== Check follow to lowerdata with absolute redirect == +check properties of metadata copied up file +Unmount and Mount again +check properties of metadata copied up file +check properties of data copied up file
Signature · 1 line
2.40.1
Quoted 7 lines · On Thu, Jul 6, 2023 at 11:50 AM <[email protected]> wrote:
From: Amir Goldstein <[email protected]>
Add test coverage for following metacopy from lower layer to lower data with absolute, relative and no redirect.
Signed-off-by Amir Goldstein <[email protected]>
Quoted 127 lines
--- tests/overlay/060 | 43 ++++++++++++++++++++++++++++++++++++++----- tests/overlay/060.out | 18 ++++++++++++++++++ 2 files changed, 56 insertions(+), 5 deletions(-)tests/overlay/060 +38
-5@@ -7,7 +7,7 @@ # Test metadata only copy up functionality. # . ./common/preamble -_begin_fstest auto quick metacopy prealloc +_begin_fstest auto quick metacopy redirect prealloc # Import common functions. . ./common/filter@@ -123,6 +123,13 @@ mount_overlay() _overlay_scratch_mount_dirs "$_lowerdir" $upperdir $workdir -o redirect_dir=on,index=on,metacopy=on } +mount_ro_overlay() +{ + local _lowerdir=$1 + + _overlay_scratch_mount_dirs "$_lowerdir" "-" "-" -o ro,redirect_dir=follow,metacopy=on +} + umount_overlay() { $UMOUNT_PROG $SCRATCH_MNT@@ -146,7 +153,8 @@ test_common() check_file_size_contents $SCRATCH_MNT/$_target $_size "$_data" check_file_blocks $SCRATCH_MNT/$_target $_blocks - # Make sure copied up file is a metacopy file. + # Trigger metadata copy up and check existence of metacopy xattr. + chmod 400 $SCRATCH_MNT/$_target umount_overlay check_metacopy $upperdir/$_target "y" check_file_size_contents $upperdir/$_target $_size ""@@ -165,7 +173,7 @@ test_common() create_basic_files() { _scratch_mkfs - mkdir -p $lowerdir $lowerdir2 $upperdir $workdir $workdir2 + mkdir -p $lowerdir/subdir $lowerdir2 $upperdir $workdir $workdir2 mkdir -p $upperdir/$udirname echo "$lowerdata" > $lowerdir/$lowername chmod 600 $lowerdir/$lowername@@ -184,12 +192,19 @@ create_lower_link() prepare_midlayer() { + local _redirect=$1 + _scratch_mkfs create_basic_files + [ -n "$_redirect" ] && mv "$lowerdir/$lowername" "$lowerdir/$_redirect" # Create midlayer _overlay_scratch_mount_dirs $lowerdir $lowerdir2 $workdir2 -o redirect_dir=on,index=on,metacopy=on - # Trigger a metacopy - chmod 400 $SCRATCH_MNT/$lowername + # Trigger a metacopy with or without redirect + if [ -n "$_redirect" ]; then + mv "$SCRATCH_MNT/$_redirect" "$SCRATCH_MNT/$lowername" + else + chmod 400 $SCRATCH_MNT/$lowername + fi umount_overlay }@@ -254,6 +269,24 @@ mount_overlay $lowerdir mv $SCRATCH_MNT/$lowerlink $SCRATCH_MNT/$ufile test_common $lowerdir $ufile $lowersize $lowerblocks "$lowerdata" "/$lowerlink" +echo -e "\n== Check follow to lowerdata without redirect ==" +prepare_midlayer +mount_ro_overlay "$lowerdir2:$lowerdir" +test_common "$lowerdir2:$lowerdir" $lowername $lowersize $lowerblocks \ + "$lowerdata" + +echo -e "\n== Check follow to lowerdata with relative redirect ==" +prepare_midlayer "$lowername.renamed" +mount_ro_overlay "$lowerdir2:$lowerdir" +test_common "$lowerdir2:$lowerdir" "$lowername" $lowersize $lowerblocks \ + "$lowerdata" + +echo -e "\n== Check follow to lowerdata with absolute redirect ==" +prepare_midlayer "/subdir/$lowername" +mount_ro_overlay "$lowerdir2:$lowerdir" +test_common "$lowerdir2:$lowerdir" "$lowername" $lowersize $lowerblocks \ + "$lowerdata" + # success, all done status=0 exittests/overlay/060.out +18
-1@@ -40,3 +40,21 @@ check properties of metadata copied up file Unmount and Mount again check properties of metadata copied up file check properties of data copied up file + +== Check follow to lowerdata without redirect == +check properties of metadata copied up file +Unmount and Mount again +check properties of metadata copied up file +check properties of data copied up file + +== Check follow to lowerdata with relative redirect == +check properties of metadata copied up file +Unmount and Mount again +check properties of metadata copied up file +check properties of data copied up file + +== Check follow to lowerdata with absolute redirect == +check properties of metadata copied up file +Unmount and Mount again +check properties of metadata copied up file +check properties of data copied up file --2.40.1
Signature · 3 lines
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Alexander Larsson Red Hat, Inc
[email protected] [email protected]PATCH 3/4 overlay: Add test for follow of lowerdata in data-only layers
In reply to a message not in the archiveFrom: Amir Goldstein <[email protected]>
Add test coverage for following metacopy from lower layer to data-only lower layers.
Data-only lower layers are configured using the syntax:
lowerdir=<lowerdir1>:<lowerdir2>::<lowerdata1>::<lowerdata2>.
Test that lowerdata files can be followed only by absolute redirect from lower layer.
Test that with two lowerdata dirs, we can lookup individual lowerdata files in both, and that a shared file is resolved from the uppermost lowerdata dir.
There is also test case for lazy-data lookups, where we remove the lowerdata file and validate that we get metadata from the metacopy file, but open fails.
--- common/overlay | 13 ++ tests/overlay/079 | 325 ++++++++++++++++++++++++++++++++++++++++++ tests/overlay/079.out | 42 ++++++ 3 files changed, 380 insertions(+) create mode 100755 tests/overlay/079 create mode 100644 tests/overlay/079.out
common/overlay +13 -0
@@ -201,6 +201,19 @@ _require_scratch_overlay_features() _scratch_unmount } +# Check kernel support for <lowerdirs>::<lowerdatadir> format +_require_scratch_overlay_lowerdata_layers() +{ + local lowerdirs="$OVL_BASE_SCRATCH_MNT/$OVL_UPPER::$OVL_BASE_SCRATCH_MNT/$OVL_LOWER" + + _scratch_mkfs > /dev/null 2>&1 + _overlay_scratch_mount_dirs "$lowerdirs" "-" "-" \ + -o ro,redirect_dir=follow,metacopy=on > /dev/null 2>&1 || \ + _notrun "overlay data-only layers not supported on ${SCRATCH_DEV}" + + _scratch_unmount +} + # Helper function to check underlying dirs of overlay filesystem _overlay_fsck_dirs() {
tests/overlay/079 new file +325 -0
325 lines not on this page · raw message
tests/overlay/079.out new file +42 -0
42 lines not on this page · raw message
Signature · 1 line
2.40.1
PATCH 4/4 overlay: Add test coverage for fs-verity support
In reply to a message not in the archiveFrom: Alexander Larsson <[email protected]>
This tests that the right xattrs are set during copy-up, and that we properly fail on missing of erronous fs-verity digests when validating.
We also ensure that verity=require fails if a metacopy has not fs-verity, and doesn't do a meta-coopy-up if the base file lacks verity.
--- common/overlay | 14 ++ common/verity | 10 +- tests/overlay/080 | 326 ++++++++++++++++++++++++++++++++++++++++++ tests/overlay/080.out | 7 + 4 files changed, 355 insertions(+), 2 deletions(-) create mode 100755 tests/overlay/080 create mode 100644 tests/overlay/080.out
common/overlay +14 -0
@@ -201,6 +201,20 @@ _require_scratch_overlay_features() _scratch_unmount } +_require_scratch_overlay_verity() +{ + local lowerdirs="$OVL_BASE_SCRATCH_MNT/$OVL_UPPER:$OVL_BASE_SCRATCH_MNT/$OVL_LOWER" + + _require_scratch_verity "$OVL_BASE_FSTYP" "$OVL_BASE_SCRATCH_MNT" + + _scratch_mkfs > /dev/null 2>&1 + _overlay_scratch_mount_dirs "$lowerdirs" "-" "-" \ + -o ro,redirect_dir=follow,metacopy=on,verity=on > /dev/null 2>&1 || \ + _notrun "overlay verity not supported on ${SCRATCH_DEV}" + + _scratch_unmount +} + # Check kernel support for <lowerdirs>::<lowerdatadir> format _require_scratch_overlay_lowerdata_layers() {
common/verity +8 -2
@@ -38,9 +38,12 @@ _require_scratch_verity() "or mkfs options are not compatible with verity" fi + local fstyp=${1:-$FSTYP} + local scratch_mnt=${2:-$SCRATCH_MNT} + # The filesystem may be aware of fs-verity but have it disabled by # CONFIG_FS_VERITY=n. Detect support via sysfs. - if [ ! -e /sys/fs/$FSTYP/features/verity ]; then + if [ ! -e /sys/fs/$fstyp/features/verity ]; then _notrun "kernel $FSTYP isn't configured with verity support" fi
@@ -68,7 +71,7 @@ _require_scratch_verity() # The filesystem may have fs-verity enabled but not actually usable by # default. E.g., ext4 only supports verity on extent-based files, so it # doesn't work on ext3-style filesystems. So, try actually using it. - if ! _fsv_can_enable $SCRATCH_MNT/tmpfile; then + if ! _fsv_can_enable $scratch_mnt/tmpfile; then _notrun "$FSTYP verity isn't usable by default with these mkfs options" fi
@@ -201,6 +204,9 @@ _scratch_mkfs_verity() btrfs) _scratch_mkfs ;; + overlay) + _scratch_mkfs # This relies on the scratch fs supporting verity + ;; *) _notrun "No verity support for $FSTYP" ;;
tests/overlay/080 new file +326 -0
326 lines not on this page · raw message
tests/overlay/080.out new file +7 -0
@@ -0,0 +1,7 @@ +QA output created by 080 + +== Check fsverity validation == + +== Check fsverity require == + +== Check fsverity copy-up ==
Signature · 1 line
2.40.1
Re: [PATCH 4/4] overlay: Add test coverage for fs-verity support
common/verity _require_scratch_verity() · line 42
Quoted 2 lines · On Thu, Jul 06, 2023 at 11:51:01AM +0200, [email protected] wrote:
common/verity +2
-0+ local fstyp=${1:-$FSTYP} + local scratch_mnt=${2:-$SCRATCH_MNT}
Some code after this still uses $FSTYP and $SCRATCH_MNT. Is that a bug?
- Eric
Re: [PATCH 4/4] overlay: Add test coverage for fs-verity support
Quoted 6 lines · Eric Biggers
common/verity _require_scratch_verity() · line 42
Quoted 2 lines · On Thu, Jul 06, 2023 at 11:51:01AM +0200, [email protected] wrote:
common/verity +2
-0+ local fstyp=${1:-$FSTYP} + local scratch_mnt=${2:-$SCRATCH_MNT}Some code after this still uses $FSTYP and $SCRATCH_MNT. Is that a bug?
Yes. Will fix.
Signature · 3 lines
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Alexander Larsson Red Hat, Inc
[email protected] [email protected]