overlayfs: Test data-only upperdirs and fs-verity
v2 cover overlayfs: Test data-only upperdirs and fs-verity
From: Alexander Larsson <[email protected]>
This adds support for testing the new data-only upperdir feature which is currently in master and will be in 6.5-rc1. It also adds tests for the fs-verity validation feature which is queued in overlayfs-next for 6.6.
All new tests check for the required features before running, so having it in early will be nice for testers of linux-next.
Changes since v1:
* Consistently use $fstyp and $scratch_mnt in _require_scratch_verity (Pointed out by Eric Biggers)
* Added Signed-off-by to patches from Amir
Alexander Larsson (1):
overlay: Add test coverage for fs-verity support
Amir Goldstein (3):
overlay: add helper for mounting rdonly overlay overlay/060: add test cases of follow to lowerdata
overlay: Add test for follow of lowerdata in data-only layers
common/overlay | 35 ++++- common/verity | 16 ++- tests/overlay/060 | 43 +++++- tests/overlay/060.out | 18 +++ tests/overlay/079 | 325 +++++++++++++++++++++++++++++++++++++++++ tests/overlay/079.out | 42 ++++++ tests/overlay/080 | 326 ++++++++++++++++++++++++++++++++++++++++++ tests/overlay/080.out | 7 + 8 files changed, 800 insertions(+), 12 deletions(-) create mode 100755 tests/overlay/079 create mode 100644 tests/overlay/079.out create mode 100755 tests/overlay/080 create mode 100644 tests/overlay/080.out
Signature · 1 line
2.40.1
PATCH v2 1/4 overlay: add helper for mounting rdonly overlay
From: Amir Goldstein <[email protected]>
Allow passing empty upperdir to _overlay_mount_dirs().
--- common/overlay | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
common/overlay +6 -2
@@ -17,15 +17,19 @@ if [ -n "$OVL_BASE_FSTYP" ];then fi # helper function to do the actual overlayfs mount operation +# accepts "-" as upperdir for non-upper overlayfs _overlay_mount_dirs() { local lowerdir=$1 local upperdir=$2 local workdir=$3 shift 3 + local diropts="-olowerdir=$lowerdir" - $MOUNT_PROG -t overlay -o lowerdir=$lowerdir -o upperdir=$upperdir \ - -o workdir=$workdir `_common_dev_mount_options $*` + [ -n "$upperdir" ] && [ "$upperdir" != "-" ] && \ + diropts+=",upperdir=$upperdir,workdir=$workdir" + + $MOUNT_PROG -t overlay $diropts `_common_dev_mount_options $*` } # Mount with same options/mnt/dev of scratch mount, but optionally
Signature · 1 line
2.40.1
PATCH v2 2/4 overlay/060: add test cases of follow to lowerdata
In reply to [email protected]From: Amir Goldstein <[email protected]>
Add test coverage for following metacopy from lower layer to lower data with absolute, relative and no redirect.
--- tests/overlay/060 | 43 ++++++++++++++++++++++++++++++++++++++----- tests/overlay/060.out | 18 ++++++++++++++++++ 2 files changed, 56 insertions(+), 5 deletions(-)
tests/overlay/060 +38 -5
@@ -7,7 +7,7 @@ # Test metadata only copy up functionality. # . ./common/preamble -_begin_fstest auto quick metacopy prealloc +_begin_fstest auto quick metacopy redirect prealloc # Import common functions. . ./common/filter
@@ -123,6 +123,13 @@ mount_overlay() _overlay_scratch_mount_dirs "$_lowerdir" $upperdir $workdir -o redirect_dir=on,index=on,metacopy=on } +mount_ro_overlay() +{ + local _lowerdir=$1 + + _overlay_scratch_mount_dirs "$_lowerdir" "-" "-" -o ro,redirect_dir=follow,metacopy=on +} + umount_overlay() { $UMOUNT_PROG $SCRATCH_MNT
@@ -146,7 +153,8 @@ test_common() check_file_size_contents $SCRATCH_MNT/$_target $_size "$_data" check_file_blocks $SCRATCH_MNT/$_target $_blocks - # Make sure copied up file is a metacopy file. + # Trigger metadata copy up and check existence of metacopy xattr. + chmod 400 $SCRATCH_MNT/$_target umount_overlay check_metacopy $upperdir/$_target "y" check_file_size_contents $upperdir/$_target $_size ""
@@ -165,7 +173,7 @@ test_common() create_basic_files() { _scratch_mkfs - mkdir -p $lowerdir $lowerdir2 $upperdir $workdir $workdir2 + mkdir -p $lowerdir/subdir $lowerdir2 $upperdir $workdir $workdir2 mkdir -p $upperdir/$udirname echo "$lowerdata" > $lowerdir/$lowername chmod 600 $lowerdir/$lowername
@@ -184,12 +192,19 @@ create_lower_link() prepare_midlayer() { + local _redirect=$1 + _scratch_mkfs create_basic_files + [ -n "$_redirect" ] && mv "$lowerdir/$lowername" "$lowerdir/$_redirect" # Create midlayer _overlay_scratch_mount_dirs $lowerdir $lowerdir2 $workdir2 -o redirect_dir=on,index=on,metacopy=on - # Trigger a metacopy - chmod 400 $SCRATCH_MNT/$lowername + # Trigger a metacopy with or without redirect + if [ -n "$_redirect" ]; then + mv "$SCRATCH_MNT/$_redirect" "$SCRATCH_MNT/$lowername" + else + chmod 400 $SCRATCH_MNT/$lowername + fi umount_overlay }
@@ -254,6 +269,24 @@ mount_overlay $lowerdir mv $SCRATCH_MNT/$lowerlink $SCRATCH_MNT/$ufile test_common $lowerdir $ufile $lowersize $lowerblocks "$lowerdata" "/$lowerlink" +echo -e "\n== Check follow to lowerdata without redirect ==" +prepare_midlayer +mount_ro_overlay "$lowerdir2:$lowerdir" +test_common "$lowerdir2:$lowerdir" $lowername $lowersize $lowerblocks \ + "$lowerdata" + +echo -e "\n== Check follow to lowerdata with relative redirect ==" +prepare_midlayer "$lowername.renamed" +mount_ro_overlay "$lowerdir2:$lowerdir" +test_common "$lowerdir2:$lowerdir" "$lowername" $lowersize $lowerblocks \ + "$lowerdata" + +echo -e "\n== Check follow to lowerdata with absolute redirect ==" +prepare_midlayer "/subdir/$lowername" +mount_ro_overlay "$lowerdir2:$lowerdir" +test_common "$lowerdir2:$lowerdir" "$lowername" $lowersize $lowerblocks \ + "$lowerdata" + # success, all done status=0 exit
tests/overlay/060.out +18 -0
@@ -40,3 +40,21 @@ check properties of metadata copied up file Unmount and Mount again check properties of metadata copied up file check properties of data copied up file + +== Check follow to lowerdata without redirect == +check properties of metadata copied up file +Unmount and Mount again +check properties of metadata copied up file +check properties of data copied up file + +== Check follow to lowerdata with relative redirect == +check properties of metadata copied up file +Unmount and Mount again +check properties of metadata copied up file +check properties of data copied up file + +== Check follow to lowerdata with absolute redirect == +check properties of metadata copied up file +Unmount and Mount again +check properties of metadata copied up file +check properties of data copied up file
Signature · 1 line
2.40.1
PATCH v2 3/4 overlay: Add test for follow of lowerdata in data-only layers
In reply to [email protected]From: Amir Goldstein <[email protected]>
Add test coverage for following metacopy from lower layer to data-only lower layers.
Data-only lower layers are configured using the syntax:
lowerdir=<lowerdir1>:<lowerdir2>::<lowerdata1>::<lowerdata2>.
Test that lowerdata files can be followed only by absolute redirect from lower layer.
Test that with two lowerdata dirs, we can lookup individual lowerdata files in both, and that a shared file is resolved from the uppermost lowerdata dir.
There is also test case for lazy-data lookups, where we remove the lowerdata file and validate that we get metadata from the metacopy file, but open fails.
--- common/overlay | 13 ++ tests/overlay/079 | 325 ++++++++++++++++++++++++++++++++++++++++++ tests/overlay/079.out | 42 ++++++ 3 files changed, 380 insertions(+) create mode 100755 tests/overlay/079 create mode 100644 tests/overlay/079.out
common/overlay +13 -0
@@ -201,6 +201,19 @@ _require_scratch_overlay_features() _scratch_unmount } +# Check kernel support for <lowerdirs>::<lowerdatadir> format +_require_scratch_overlay_lowerdata_layers() +{ + local lowerdirs="$OVL_BASE_SCRATCH_MNT/$OVL_UPPER::$OVL_BASE_SCRATCH_MNT/$OVL_LOWER" + + _scratch_mkfs > /dev/null 2>&1 + _overlay_scratch_mount_dirs "$lowerdirs" "-" "-" \ + -o ro,redirect_dir=follow,metacopy=on > /dev/null 2>&1 || \ + _notrun "overlay data-only layers not supported on ${SCRATCH_DEV}" + + _scratch_unmount +} + # Helper function to check underlying dirs of overlay filesystem _overlay_fsck_dirs() {
tests/overlay/079 new file +325 -0
325 lines not on this page · raw message
tests/overlay/079.out new file +42 -0
42 lines not on this page · raw message
Signature · 1 line
2.40.1
PATCH v2 4/4 overlay: Add test coverage for fs-verity support
In reply to [email protected]From: Alexander Larsson <[email protected]>
This tests that the right xattrs are set during copy-up, and that we properly fail on missing of erronous fs-verity digests when validating.
We also ensure that verity=require fails if a metacopy has not fs-verity, and doesn't do a meta-coopy-up if the base file lacks verity.
--- common/overlay | 14 ++ common/verity | 16 ++- tests/overlay/080 | 326 ++++++++++++++++++++++++++++++++++++++++++ tests/overlay/080.out | 7 + 4 files changed, 358 insertions(+), 5 deletions(-) create mode 100755 tests/overlay/080 create mode 100644 tests/overlay/080.out
common/overlay +14 -0
@@ -201,6 +201,20 @@ _require_scratch_overlay_features() _scratch_unmount } +_require_scratch_overlay_verity() +{ + local lowerdirs="$OVL_BASE_SCRATCH_MNT/$OVL_UPPER:$OVL_BASE_SCRATCH_MNT/$OVL_LOWER" + + _require_scratch_verity "$OVL_BASE_FSTYP" "$OVL_BASE_SCRATCH_MNT" + + _scratch_mkfs > /dev/null 2>&1 + _overlay_scratch_mount_dirs "$lowerdirs" "-" "-" \ + -o ro,redirect_dir=follow,metacopy=on,verity=on > /dev/null 2>&1 || \ + _notrun "overlay verity not supported on ${SCRATCH_DEV}" + + _scratch_unmount +} + # Check kernel support for <lowerdirs>::<lowerdatadir> format _require_scratch_overlay_lowerdata_layers() {
common/verity +11 -5
@@ -38,10 +38,13 @@ _require_scratch_verity() "or mkfs options are not compatible with verity" fi + local fstyp=${1:-$FSTYP} + local scratch_mnt=${2:-$SCRATCH_MNT} + # The filesystem may be aware of fs-verity but have it disabled by # CONFIG_FS_VERITY=n. Detect support via sysfs. - if [ ! -e /sys/fs/$FSTYP/features/verity ]; then - _notrun "kernel $FSTYP isn't configured with verity support" + if [ ! -e /sys/fs/$fstyp/features/verity ]; then + _notrun "kernel $fstyp isn't configured with verity support" fi # Select a default Merkle tree block size for when tests don't
@@ -57,7 +60,7 @@ _require_scratch_verity() # # Therefore, we default to merkle_tree_block_size == min(fs_block_size, # page_size). That maximizes the chance of verity actually working. - local fs_block_size=$(_get_block_size $SCRATCH_MNT) + local fs_block_size=$(_get_block_size $scratch_mnt) local page_size=$(get_page_size) if (( fs_block_size <= page_size )); then FSV_BLOCK_SIZE=$fs_block_size
@@ -68,8 +71,8 @@ _require_scratch_verity() # The filesystem may have fs-verity enabled but not actually usable by # default. E.g., ext4 only supports verity on extent-based files, so it # doesn't work on ext3-style filesystems. So, try actually using it. - if ! _fsv_can_enable $SCRATCH_MNT/tmpfile; then - _notrun "$FSTYP verity isn't usable by default with these mkfs options" + if ! _fsv_can_enable $scratch_mnt/tmpfile; then + _notrun "$fstyp verity isn't usable by default with these mkfs options" fi _scratch_unmount
@@ -201,6 +204,9 @@ _scratch_mkfs_verity() btrfs) _scratch_mkfs ;; + overlay) + _scratch_mkfs # This relies on the scratch fs supporting verity + ;; *) _notrun "No verity support for $FSTYP" ;;
tests/overlay/080 new file +326 -0
326 lines not on this page · raw message
tests/overlay/080.out new file +7 -0
@@ -0,0 +1,7 @@ +QA output created by 080 + +== Check fsverity validation == + +== Check fsverity require == + +== Check fsverity copy-up ==
Signature · 1 line
2.40.1
Quoted 14 lines · On Mon, Jul 10, 2023 at 11:07:09AM +0200, [email protected] wrote:
From: Alexander Larsson <[email protected]>
This adds support for testing the new data-only upperdir feature which is currently in master and will be in 6.5-rc1. It also adds tests for the fs-verity validation feature which is queued in overlayfs-next for 6.6.
All new tests check for the required features before running, so having it in early will be nice for testers of linux-next.
Changes since v1:
* Consistently use $fstyp and $scratch_mnt in_require_scratch_verity(Pointed out by Eric Biggers)
* Added Signed-off-by to patches from Amir
Quoted 7 lines
Alexander Larsson (1):
overlay: Add test coverage for fs-verity supportAmir Goldstein (3):
overlay: add helper for mounting rdonly overlay overlay/060: add test cases of follow to lowerdata
overlay: Add test for follow of lowerdata in data-only layers
This patchset looks good to me, and I didn't regression from it. So if there's not more review points from overlayfs side, I'll merge this patchset in next fstests release.
Thanks,
Zorro
Quoted 18 lines
common/overlay | 35 ++++- common/verity | 16 ++- tests/overlay/060 | 43 +++++- tests/overlay/060.out | 18 +++ tests/overlay/079 | 325 +++++++++++++++++++++++++++++++++++++++++ tests/overlay/079.out | 42 ++++++ tests/overlay/080 | 326 ++++++++++++++++++++++++++++++++++++++++++ tests/overlay/080.out | 7 + 8 files changed, 800 insertions(+), 12 deletions(-) create mode 100755 tests/overlay/079 create mode 100644 tests/overlay/079.out create mode 100755 tests/overlay/080 create mode 100644 tests/overlay/080.outSignature · 1 line
2.40.1
Quoted 19 lines · Zorro Lang
Quoted 17 lines · On Mon, Jul 10, 2023 at 11:07:09AM +0200, [email protected] wrote:
From: Alexander Larsson <[email protected]>
This adds support for testing the new data-only upperdir feature which is currently in master and will be in 6.5-rc1. It also adds tests for the fs-verity validation feature which is queued in overlayfs-next for 6.6.
All new tests check for the required features before running, so having it in early will be nice for testers of linux-next.
Changes since v1:
* Consistently use $fstyp and $scratch_mnt in_require_scratch_verity(Pointed out by Eric Biggers)
* Added Signed-off-by to patches from AmirAlexander Larsson (1):
overlay: Add test coverage for fs-verity support
Quoted 9 lines
Quoted 4 lines
Amir Goldstein (3):
overlay: add helper for mounting rdonly overlay overlay/060: add test cases of follow to lowerdata
overlay: Add test for follow of lowerdata in data-only layersThis patchset looks good to me, and I didn't regression from it. So if there's not more review points from overlayfs side, I'll merge this patchset in next fstests release.
That would be great.
Thanks,
Amir.