fstests overlay fixes for v2025.05.25
v2 cover fstests overlay fixes for v2025.05.25
Zorro,
My first batch of fixes [1] was addressing overlayfs test failures after upgrade of test machine to a newer distro (trixie) with libmount v1.41.
After a closer look, I found some tests that regressed into _notrun, which is harder to notice at first glance.
Tests generic/{633,697,697} were regressed in v2025.03.17. Test generic/699 always had a problem AFAICT, but I only just tested it with custom MOUNT_OPTIONS.
Lastly, following the review discussion between Miklos and Karel Zak, I changed the workaround to the regressions observed after upgrade to newer libmount.
Thanks,
Amir.
Changes since v1 [1]:
- Change workaround from LIBMOUNT_FORCE_MOUNT2 to --options-mode=ignore
- Add two commits to fix tests that were not running by mistake
[1] https://lore.kernel.org/fstests/[email protected]/
Amir Goldstein (6):
overlay: workaround libmount failure to remount,ro
overlay: fix regression in _repair_overlay_scratch_fs generic/604: do not run with overlayfs generic/623: do not run with overlayfs
generic: remove incorrect _require_idmapped_mounts checks generic/699: fix failure with MOUNT_OPTIONS
common/overlay | 7 ++++++- common/rc | 10 +++++++++- tests/generic/330 | 2 +- tests/generic/604 | 11 ++++++----- tests/generic/623 | 1 + tests/generic/633 | 1 - tests/generic/696 | 1 - tests/generic/697 | 1 - tests/generic/699 | 17 ++++++++++------- tests/overlay/035 | 2 +- 10 files changed, 34 insertions(+), 19 deletions(-)
Signature · 1 line
2.34.1
PATCH v2 1/6 overlay: workaround libmount failure to remount,ro
libmount >= v1.39 calls several unneeded fsconfig() calls to reconfigure lowerdir/upperdir when user requests only -o remount,ro.
Those calls fail because overlayfs does not allow making any config changes with new mount api, besides MS_RDONLY.
We workaround this problem with --options-mode ignore.
---
Changes since v1 [1]:
- Change workaround from LIBMOUNT_FORCE_MOUNT2 to --options-mode=ignore
[1] https://lore.kernel.org/fstests/[email protected]/
common/overlay | 5 ++++- tests/overlay/035 | 2 +- 2 files changed, 5 insertions(+), 2 deletions(-)
common/overlay +4 -1
@@ -127,7 +127,10 @@ _overlay_base_scratch_mount() _overlay_scratch_mount() { if echo "$*" | grep -q remount; then - $MOUNT_PROG $SCRATCH_MNT $* + # By default, libmount merges remount options with old mount options. + # overlayfs does not support re-configuring the same mount options. + # We workaround this problem with --options-mode ignore. + $MOUNT_PROG $SCRATCH_MNT --options-mode ignore $* return fi
tests/overlay/035 +1 -1
@@ -42,7 +42,7 @@ mkdir -p $lowerdir1 $lowerdir2 $upperdir $workdir # Verify that overlay is mounted read-only and that it cannot be remounted rw. _overlay_scratch_mount_opts -o"lowerdir=$lowerdir2:$lowerdir1" touch $SCRATCH_MNT/foo 2>&1 | _filter_scratch -$MOUNT_PROG -o remount,rw $SCRATCH_MNT 2>&1 | _filter_ro_mount +_scratch_remount rw 2>&1 | _filter_ro_mount $UMOUNT_PROG $SCRATCH_MNT # Make workdir immutable to prevent workdir re-create on mount
Signature · 1 line
2.34.1
Re: [PATCH v2 1/6] overlay: workaround libmount failure to remount,ro
Quoted 13 lines · Amir Goldstein
libmount >= v1.39 calls several unneeded
fsconfig()calls to reconfigure lowerdir/upperdir when user requests only -o remount,ro.Those calls fail because overlayfs does not allow making any config changes with new mount api, besides
MS_RDONLY.We workaround this problem with --options-mode ignore.
Quoted 6 lines
---
Changes since v1 [1]:
- Change workaround fromLIBMOUNT_FORCE_MOUNT2to --options-mode=ignore
I'm not sure if I understand clearly. Does overlay list are fixing this issue on kernel side, then providing a workaround to fstests to avoid the issue be triggered too?
Thanks,
Zorro
Quoted 37 lines
common/overlay | 5 ++++- tests/overlay/035 | 2 +- 2 files changed, 5 insertions(+), 2 deletions(-)common/overlay +4
-1@@ -127,7 +127,10 @@ _overlay_base_scratch_mount() _overlay_scratch_mount() { if echo "$*" | grep -q remount; then - $MOUNT_PROG $SCRATCH_MNT $* + # By default, libmount merges remount options with old mount options. + # overlayfs does not support re-configuring the same mount options. + # We workaround this problem with --options-mode ignore. + $MOUNT_PROG $SCRATCH_MNT --options-mode ignore $* return fitests/overlay/035 +1
-1@@ -42,7 +42,7 @@ mkdir -p $lowerdir1 $lowerdir2 $upperdir $workdir # Verify that overlay is mounted read-only and that it cannot be remounted rw. _overlay_scratch_mount_opts -o"lowerdir=$lowerdir2:$lowerdir1" touch $SCRATCH_MNT/foo 2>&1 | _filter_scratch -$MOUNT_PROG -o remount,rw $SCRATCH_MNT 2>&1 | _filter_ro_mount +_scratch_remount rw 2>&1 | _filter_ro_mount $UMOUNT_PROG $SCRATCH_MNT # Make workdir immutable to prevent workdir re-create on mountSignature · 1 line
2.34.1
Re: [PATCH v2 1/6] overlay: workaround libmount failure to remount,ro
Quoted 17 lines · Zorro Lang
Quoted 15 lines · Amir Goldstein
libmount >= v1.39 calls several unneeded
fsconfig()calls to reconfigure lowerdir/upperdir when user requests only -o remount,ro.Those calls fail because overlayfs does not allow making any config changes with new mount api, besides
MS_RDONLY.We workaround this problem with --options-mode ignore.
Reported-by André Almeida <[email protected]>Suggested-by Karel Zak <[email protected]>Link https://lore.kernel.org/fstests/CAJfpegtJ3SDKmC80B4AfWiC3JmtWdW2+78fRZVtsuhe-wSRPvg@mail.gmail.com/ open hereSigned-off-by Amir Goldstein <[email protected]>---
Quoted 9 lines
Quoted 4 lines
Changes since v1 [1]:
- Change workaround fromLIBMOUNT_FORCE_MOUNT2to --options-mode=ignoreI'm not sure if I understand clearly. Does overlay list are fixing this issue on kernel side, then providing a workaround to fstests to avoid the issue be triggered too?
Noone agreed to fix it on the kernel side.
At least not yet.
Thanks,
Amir.
Re: [PATCH v2 1/6] overlay: workaround libmount failure to remount,ro
Quoted 23 lines · Amir Goldstein
Quoted 22 lines · Zorro Lang
Quoted 19 lines · Amir Goldstein
libmount >= v1.39 calls several unneeded
fsconfig()calls to reconfigure lowerdir/upperdir when user requests only -o remount,ro.Those calls fail because overlayfs does not allow making any config changes with new mount api, besides
MS_RDONLY.We workaround this problem with --options-mode ignore.
Reported-by André Almeida <[email protected]>Suggested-by Karel Zak <[email protected]>Link https://lore.kernel.org/fstests/CAJfpegtJ3SDKmC80B4AfWiC3JmtWdW2+78fRZVtsuhe-wSRPvg@mail.gmail.com/ open hereSigned-off-by Amir Goldstein <[email protected]>---
Changes since v1 [1]:
- Change workaround fromLIBMOUNT_FORCE_MOUNT2to --options-mode=ignore
Quoted 7 lines
Quoted 4 lines
I'm not sure if I understand clearly. Does overlay list are fixing this issue on kernel side, then providing a workaround to fstests to avoid the issue be triggered too?
Noone agreed to fix it on the kernel side.
At least not yet.
If so, I have two questions:)
1) Will overlay fix it on kernel or mount util side?
2) Do you plan to keep this workaround until the issue be fixed in one day? Then revert this workaround?
Thanks,
Zorro
Quoted 4 lines
Thanks,
Amir.
Re: [PATCH v2 1/6] overlay: workaround libmount failure to remount,ro
Quoted 30 lines · Zorro Lang
Quoted 28 lines · Amir Goldstein
Quoted 26 lines · Zorro Lang
Quoted 13 lines · Amir Goldstein
libmount >= v1.39 calls several unneeded
fsconfig()calls to reconfigure lowerdir/upperdir when user requests only -o remount,ro.Those calls fail because overlayfs does not allow making any config changes with new mount api, besides
MS_RDONLY.We workaround this problem with --options-mode ignore.
Quoted 6 lines
---
Changes since v1 [1]:
- Change workaround fromLIBMOUNT_FORCE_MOUNT2to --options-mode=ignoreI'm not sure if I understand clearly. Does overlay list are fixing this issue on kernel side, then providing a workaround to fstests to avoid the issue be triggered too?
Quoted 5 lines
Quoted 2 lines
Noone agreed to fix it on the kernel side.
At least not yet.If so, I have two questions:)
1) Will overlay fix it on kernel or mount util side?
This is not known at this time.
Quoted 2 lines
2) Do you plan to keep this workaround until the issue be fixed in one day? Then revert this workaround?
Maybe, but keep in mind that the workaround is simply telling the library what we want to do.
We want to remount overlay ro and nothing else and that is exactly what "--options-mode ignore" tells the library to do.
I could have just as well written a test helper src/remount_rdonly.c and not have to deal with the question of which libmount version the test machine is using.
Note that the tests in question are not intended to test the remount,ro functionality itself, they are intended to test the behavior of fs in some scenarios involving a rdonly mount.
I do not want to lose important test coverage of these scenarios because of regressions in the kernel/libmount API.
We can add a new test that ONLY tests remount,ro and let that test fail on overlayfs to keep us reminded of the real regresion that needs to be fixed, but the "workaround" or as I prefer to call it "using the right tool for the test case" has to stay for those other tests.
Thanks,
Amir.
Re: [PATCH v2 1/6] overlay: workaround libmount failure to remount,ro
Quoted 31 lines · Amir Goldstein
Quoted 30 lines · Zorro Lang
Quoted 28 lines · Amir Goldstein
Quoted 26 lines · Zorro Lang
Quoted 13 lines · Amir Goldstein
libmount >= v1.39 calls several unneeded
fsconfig()calls to reconfigure lowerdir/upperdir when user requests only -o remount,ro.Those calls fail because overlayfs does not allow making any config changes with new mount api, besides
MS_RDONLY.We workaround this problem with --options-mode ignore.
Quoted 6 lines
---
Changes since v1 [1]:
- Change workaround fromLIBMOUNT_FORCE_MOUNT2to --options-mode=ignoreI'm not sure if I understand clearly. Does overlay list are fixing this issue on kernel side, then providing a workaround to fstests to avoid the issue be triggered too?
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
Noone agreed to fix it on the kernel side.
At least not yet.If so, I have two questions:)
1) Will overlay fix it on kernel or mount util side?This is not known at this time.
Oh, I thought it's getting fix :-D
Quoted 18 lines
Quoted 3 lines
2) Do you plan to keep this workaround until the issue be fixed in one day? Then revert this workaround?
Maybe, but keep in mind that the workaround is simply telling the library what we want to do.
We want to remount overlay ro and nothing else and that is exactly what "--options-mode ignore" tells the library to do.
I could have just as well written a test helper
src/remount_rdonly.cand not have to deal with the question of which libmount version the test machine is using.Note that the tests in question are not intended to test the remount,ro functionality itself, they are intended to test the behavior of fs in some scenarios involving a rdonly mount.
Quoted 7 lines
I do not want to lose important test coverage of these scenarios because of regressions in the kernel/libmount API.
We can add a new test that ONLY tests remount,ro and let that test fail on overlayfs to keep us reminded of the real regresion that needs to be fixed, but the "workaround" or as I prefer to call it "using the right tool for the test case" has to stay for those other tests.
OK, I just tried to figure out if "hide this error output on new mount APIs" is what overlay list wants. If overlay list (or vfs) acks this patch, and will track this issue. I'm good to merge this workaround for testing :)
Thanks,
Zorro
Quoted 4 lines
Thanks,
Amir.
Re: [PATCH v2 1/6] overlay: workaround libmount failure to remount,ro
Quoted 64 lines · Zorro Lang
Quoted 31 lines · Amir Goldstein
Quoted 30 lines · Zorro Lang
Quoted 28 lines · Amir Goldstein
Quoted 26 lines · Zorro Lang
Quoted 13 lines · Amir Goldstein
libmount >= v1.39 calls several unneeded
fsconfig()calls to reconfigure lowerdir/upperdir when user requests only -o remount,ro.Those calls fail because overlayfs does not allow making any config changes with new mount api, besides
MS_RDONLY.We workaround this problem with --options-mode ignore.
Quoted 6 lines
---
Changes since v1 [1]:
- Change workaround fromLIBMOUNT_FORCE_MOUNT2to --options-mode=ignoreI'm not sure if I understand clearly. Does overlay list are fixing this issue on kernel side, then providing a workaround to fstests to avoid the issue be triggered too?
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
Noone agreed to fix it on the kernel side.
At least not yet.If so, I have two questions:)
1) Will overlay fix it on kernel or mount util side?This is not known at this time.
Oh, I thought it's getting fix :-D
Quoted 21 lines
Quoted 3 lines
2) Do you plan to keep this workaround until the issue be fixed in one day? Then revert this workaround?
Maybe, but keep in mind that the workaround is simply telling the library what we want to do.
We want to remount overlay ro and nothing else and that is exactly what "--options-mode ignore" tells the library to do.
I could have just as well written a test helper
src/remount_rdonly.cand not have to deal with the question of which libmount version the test machine is using.Note that the tests in question are not intended to test the remount,ro functionality itself, they are intended to test the behavior of fs in some scenarios involving a rdonly mount.
I do not want to lose important test coverage of these scenarios because of regressions in the kernel/libmount API.
Quoted 9 lines
Quoted 4 lines
We can add a new test that ONLY tests remount,ro and let that test fail on overlayfs to keep us reminded of the real regresion that needs to be fixed, but the "workaround" or as I prefer to call it "using the right tool for the test case" has to stay for those other tests.
OK, I just tried to figure out if "hide this error output on new mount APIs" is what overlay list wants. If overlay list (or vfs) acks this patch, and will track this issue. I'm good to merge this workaround for testing :)
This workaround in v2 was suggested by libmount maintainer and approved by overlayfs maintainer:
"> So, you do not need LIBMOUNT_FORCE_MOUNT2= workaround, use
Quoted 1 line
"--options-mode ignore" or source and target ;-)
Yeah, that's definitely a better workaround.
I wouldn't call it a fix, since "mount -oremount,ro /overlay" still doesn't work the way it is supposed to, and the thought of adding code to the kernel to work around the current libmount behavior makes me go bleah."
Thanks,
Amir.
Zorro Lang
Reviewed-byRe: [PATCH v2 1/6] overlay: workaround libmount failure to remount,ro
Quoted 79 lines · Amir Goldstein
Quoted 64 lines · Zorro Lang
Quoted 31 lines · Amir Goldstein
Quoted 30 lines · Zorro Lang
Quoted 28 lines · Amir Goldstein
Quoted 26 lines · Zorro Lang
Quoted 13 lines · Amir Goldstein
libmount >= v1.39 calls several unneeded
fsconfig()calls to reconfigure lowerdir/upperdir when user requests only -o remount,ro.Those calls fail because overlayfs does not allow making any config changes with new mount api, besides
MS_RDONLY.We workaround this problem with --options-mode ignore.
Quoted 6 lines
---
Changes since v1 [1]:
- Change workaround fromLIBMOUNT_FORCE_MOUNT2to --options-mode=ignoreI'm not sure if I understand clearly. Does overlay list are fixing this issue on kernel side, then providing a workaround to fstests to avoid the issue be triggered too?
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
Noone agreed to fix it on the kernel side.
At least not yet.If so, I have two questions:)
1) Will overlay fix it on kernel or mount util side?This is not known at this time.
Oh, I thought it's getting fix :-D
Quoted 21 lines
Quoted 3 lines
2) Do you plan to keep this workaround until the issue be fixed in one day? Then revert this workaround?
Maybe, but keep in mind that the workaround is simply telling the library what we want to do.
We want to remount overlay ro and nothing else and that is exactly what "--options-mode ignore" tells the library to do.
I could have just as well written a test helper
src/remount_rdonly.cand not have to deal with the question of which libmount version the test machine is using.Note that the tests in question are not intended to test the remount,ro functionality itself, they are intended to test the behavior of fs in some scenarios involving a rdonly mount.
I do not want to lose important test coverage of these scenarios because of regressions in the kernel/libmount API.
Quoted 9 lines
Quoted 4 lines
We can add a new test that ONLY tests remount,ro and let that test fail on overlayfs to keep us reminded of the real regresion that needs to be fixed, but the "workaround" or as I prefer to call it "using the right tool for the test case" has to stay for those other tests.
OK, I just tried to figure out if "hide this error output on new mount APIs" is what overlay list wants. If overlay list (or vfs) acks this patch, and will track this issue. I'm good to merge this workaround for testing :)
This workaround in v2 was suggested by libmount maintainer and approved by overlayfs maintainer:
"> So, you do not need LIBMOUNT_FORCE_MOUNT2= workaround, use
Quoted 8 lines
Quoted 1 line
"--options-mode ignore" or source and target ;-)
Yeah, that's definitely a better workaround.
I wouldn't call it a fix, since "mount -oremount,ro /overlay" still doesn't work the way it is supposed to, and the thought of adding code to the kernel to work around the current libmount behavior makes me go bleah."
OK, just to make sure overlay/vfs folks know why these failures gone :)
With this patch, I've merged 5 patches of this patchset. Now only 4/6 still need more reviewing (I've provided my suggestion about that). If patch 4/6 can't catch up the release of this week, don't worry, I'll push these 5 patches at first, feel free to send patch 4/6 singly later :)
Thanks,
Zorro
Quoted 6 lines
Thanks,
Amir.
Re: [PATCH v2 1/6] overlay: workaround libmount failure to remount,ro
In reply to Amir GoldsteinHi Amir,
Quoted 27 lines · Amir Goldstein
Quoted 26 lines · Zorro Lang
Quoted 25 lines · Amir Goldstein
Quoted 24 lines · Zorro Lang
Quoted 13 lines · Amir Goldstein
libmount >= v1.39 calls several unneeded
fsconfig()calls to reconfigure lowerdir/upperdir when user requests only -o remount,ro.Those calls fail because overlayfs does not allow making any config changes with new mount api, besides
MS_RDONLY.We workaround this problem with --options-mode ignore.
Quoted 6 lines
---
Changes since v1 [1]:
- Change workaround fromLIBMOUNT_FORCE_MOUNT2to --options-mode=ignoreI'm not sure if I understand clearly. Does overlay list are fixing this issue on kernel side, then providing a workaround to fstests to avoid the issue be triggered too?
Quoted 5 lines
Quoted 4 lines
Quoted 2 lines
Noone agreed to fix it on the kernel side.
At least not yet.If so, I have two questions:)
1) Will overlay fix it on kernel or mount util side?This is not known at this time.
Do you know how calling fsconfig() in a "redundant" way behaves in other filesystems? Do they allow to call fsconfig() calls that doesn't change the state, like a noop?
Re: [PATCH v2 1/6] overlay: workaround libmount failure to remount,ro
Quoted 33 lines · André Almeida
Hi Amir,
Quoted 29 lines · Amir Goldstein
Quoted 28 lines · Zorro Lang
Quoted 27 lines · Amir Goldstein
Quoted 16 lines · Zorro Lang
Quoted 15 lines · Amir Goldstein
libmount >= v1.39 calls several unneeded
fsconfig()calls to reconfigure lowerdir/upperdir when user requests only -o remount,ro.Those calls fail because overlayfs does not allow making any config changes with new mount api, besides
MS_RDONLY.We workaround this problem with --options-mode ignore.
Reported-by André Almeida <[email protected]>Suggested-by Karel Zak <[email protected]>Link https://lore.kernel.org/fstests/CAJfpegtJ3SDKmC80B4AfWiC3JmtWdW2+78fRZVtsuhe-wSRPvg@mail.gmail.com/ open hereSigned-off-by Amir Goldstein <[email protected]>---
Quoted 8 lines
Quoted 4 lines
Changes since v1 [1]:
- Change workaround fromLIBMOUNT_FORCE_MOUNT2to --options-mode=ignoreI'm not sure if I understand clearly. Does overlay list are fixing this issue on kernel side, then providing a workaround to fstests to avoid the issue be triggered too?
Noone agreed to fix it on the kernel side.
At least not yet.
Quoted 8 lines
Quoted 3 lines
Quoted 2 lines
If so, I have two questions:)
1) Will overlay fix it on kernel or mount util side?This is not known at this time.
Do you know how calling
fsconfig()in a "redundant" way behaves in other filesystems? Do they allow to callfsconfig()calls that doesn't change the state, like a noop?
I don't know. didn't do a survey.
"We did this until now"
Is not a good enough reason to keep doing the same workarounds in the kernel IMO.
Thanks,
Amir.
PATCH v2 2/6 overlay: fix regression in _repair_overlay_scratch_fs
In reply to Amir Goldstein_repair_overlay_scratch_fs assumed that the base fs is mounted. This was a wrong assumption to make, and that was exposed by commit 4c6bc456 ("fstests: clean up mount and unmount operations") that converted open coded umount in generic/332 to _scratch_unmount.
After this change, there errors were observed when running generic/332 if fsck.overlay is installed:
Check for damage
+fsck.overlay:[Error]: Faile to resolve upperdir:/vdf/ovl-upper:
No such file or directory
+fsck.overlay failed, err=8
+umount: /vdf: not mounted.
Fix this by making sure that base fs is mounted before running the layers check and fix test generic/330 to conform with the umount conversion patch.
--- common/overlay | 2 ++ tests/generic/330 | 2 +- 2 files changed, 3 insertions(+), 1 deletion(-)
common/overlay +2 -0
@@ -434,6 +434,8 @@ _check_overlay_scratch_fs() _repair_overlay_scratch_fs() { + # Base fs needs to be mounted for overlayfs check + _overlay_base_scratch_mount _overlay_fsck_dirs $OVL_BASE_SCRATCH_MNT/$OVL_LOWER \ $OVL_BASE_SCRATCH_MNT/$OVL_UPPER \ $OVL_BASE_SCRATCH_MNT/$OVL_WORK -y
tests/generic/330 +1 -1
@@ -61,7 +61,7 @@ md5sum $testdir/file1 | _filter_scratch md5sum $testdir/file2 | _filter_scratch echo "Check for damage" -umount $SCRATCH_MNT +_scratch_unmount _repair_scratch_fs >> $seqres.full # success, all done
Signature · 1 line
2.34.1
Re: [PATCH v2 2/6] overlay: fix regression in _repair_overlay_scratch_fs
Quoted 14 lines · Amir Goldstein
_repair_overlay_scratch_fsassumed that the base fs is mounted. This was a wrong assumption to make, and that was exposed by commit4c6bc456("fstests: clean up mount and unmount operations") that converted open coded umount in generic/332 to_scratch_unmount.After this change, there errors were observed when running generic/332 if fsck.overlay is installed:
Check for damage
+fsck.overlay:[Error]: Faile to resolve upperdir:/vdf/ovl-upper:
No such file or directory
+fsck.overlay failed, err=8
+umount: /vdf: not mounted.
Quoted 8 lines
Fix this by making sure that base fs is mounted before running the layers check and fix test generic/330 to conform with the umount conversion patch.
Fixes 4c6bc456 ("fstests: clean up mount and unmount operations")Tested-by André Almeida <[email protected]>Signed-off-by Amir Goldstein <[email protected]>---
Thanks for this regression fix from overlay.
Quoted 33 lines
common/overlay | 2 ++ tests/generic/330 | 2 +- 2 files changed, 3 insertions(+), 1 deletion(-)common/overlay +2
-0@@ -434,6 +434,8 @@ _check_overlay_scratch_fs() _repair_overlay_scratch_fs() { + # Base fs needs to be mounted for overlayfs check + _overlay_base_scratch_mount _overlay_fsck_dirs $OVL_BASE_SCRATCH_MNT/$OVL_LOWER \ $OVL_BASE_SCRATCH_MNT/$OVL_UPPER \ $OVL_BASE_SCRATCH_MNT/$OVL_WORK -ytests/generic/330 +1
-1@@ -61,7 +61,7 @@ md5sum $testdir/file1 | _filter_scratch md5sum $testdir/file2 | _filter_scratch echo "Check for damage" -umount $SCRATCH_MNT +_scratch_unmount _repair_scratch_fs >> $seqres.full # success, all doneSignature · 1 line
2.34.1
PATCH v2 3/6 generic/604: do not run with overlayfs
In reply to Amir GoldsteinOverlayfs does not allow mounting over again with the same layers until umount is fully completed, so is not appropriate for this test which tries to mount in parallel to umount.
This is manifested as the test failure below when overlayfs strict mount checks are enabled by enabling the index feature:
$ echo Y > /sys/module/overlay/parameters/index
...
+mount: /vdf/ovl-mnt: /vdf already mounted or mount point busy.
+ dmesg(1) may have more information after failed mount system call.
+mount /vdf /vdf/ovl-mnt failedOpt-out of this test with overlayfs and remove the hacks that were placed by commit 06cee932 ("generic/604: Fix for overlayfs") to make the test pass with overlayfs in the first place.
--- tests/generic/604 | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-)
tests/generic/604 +6 -5
@@ -13,6 +13,9 @@ _begin_fstest auto quick mount # Import common functions. . ./common/filter +# Overlayfs does not allow mounting over again with the same layers +# until umount is fully completed, so is not appropriate for this test. +_exclude_fs overlay # Modify as appropriate. _require_scratch
@@ -22,11 +25,9 @@ _scratch_mount for i in $(seq 0 500); do $XFS_IO_PROG -f -c "pwrite 0 4K" $SCRATCH_MNT/$i >/dev/null done -# For overlayfs, avoid unmounting the base fs after _scratch_mount tries to -# mount the base fs. Delay the mount attempt by a small amount in the hope -# that the mount() call will try to lock s_umount /after/ umount has already -# taken it. -_unmount $SCRATCH_MNT & +# Delay the mount attempt by a small amount in the hope that the mount() call +# will try to lock s_umount /after/ umount has already taken it. +_scratch_unmount & sleep 0.01s ; _scratch_mount wait
Signature · 1 line
2.34.1
Re: [PATCH v2 3/6] generic/604: do not run with overlayfs
Quoted 13 lines · Amir Goldstein
Overlayfs does not allow mounting over again with the same layers until umount is fully completed, so is not appropriate for this test which tries to mount in parallel to umount.
This is manifested as the test failure below when overlayfs strict mount checks are enabled by enabling the index feature:
$ echo Y > /sys/module/overlay/parameters/index ... +mount: /vdf/ovl-mnt: /vdf already mounted or mount point busy. + dmesg(1) may have more information after failed mount system call. +mount /vdf /vdf/ovl-mnt failed
Quoted 7 lines
Opt-out of this test with overlayfs and remove the hacks that were placed by commit
06cee932("generic/604: Fix for overlayfs") to make the test pass with overlayfs in the first place.Tested-by André Almeida <[email protected]>Signed-off-by Amir Goldstein <[email protected]>---
Good to me,
Quoted 35 lines
tests/generic/604 | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-)tests/generic/604 +6
-5@@ -13,6 +13,9 @@ _begin_fstest auto quick mount # Import common functions. . ./common/filter +# Overlayfs does not allow mounting over again with the same layers +# until umount is fully completed, so is not appropriate for this test. +_exclude_fs overlay # Modify as appropriate. _require_scratch@@ -22,11 +25,9 @@ _scratch_mount for i in $(seq 0 500); do $XFS_IO_PROG -f -c "pwrite 0 4K" $SCRATCH_MNT/$i >/dev/null done -# For overlayfs, avoid unmounting the base fs after _scratch_mount tries to -# mount the base fs. Delay the mount attempt by a small amount in the hope -# that the mount() call will try to lock s_umount /after/ umount has already -# taken it. -_unmount $SCRATCH_MNT & +# Delay the mount attempt by a small amount in the hope that the mount() call +# will try to lock s_umount /after/ umount has already taken it. +_scratch_unmount & sleep 0.01s ; _scratch_mount waitSignature · 1 line
2.34.1
PATCH v2 4/6 generic/623: do not run with overlayfs
In reply to Amir GoldsteinThis test performs shutdown via xfs_io -c shutdown.
Overlayfs tests can use _scratch_shutdown, but they cannot use "-c shutdown" xfs_io command without jumping through hoops, so by default we do not support it.
Add this condition to _require_xfs_io_command and add the require statement to test generic/623 so it wont run with overlayfs.
--- common/rc | 8 ++++++++ tests/generic/623 | 1 + 2 files changed, 9 insertions(+)
common/rc +8 -0
@@ -3033,6 +3033,14 @@ _require_xfs_io_command() touch $testfile testio=`$XFS_IO_PROG -c "syncfs" $testfile 2>&1` ;; + "shutdown") + if [ $FSTYP = "overlay" ]; then + # Overlayfs tests can use _scratch_shutdown, but they + # cannot use "-c shutdown" xfs_io command without jumping + # through hoops, so by default we do not support it. + _notrun "xfs_io $command not supported on $FSTYP" + fi + ;; *) testio=`$XFS_IO_PROG -c "help $command" 2>&1` esac
tests/generic/623 +1 -0
@@ -16,6 +16,7 @@ _begin_fstest auto quick shutdown mmap _require_scratch_nocheck _require_scratch_shutdown +_require_xfs_io_command "shutdown" _scratch_mkfs &>> $seqres.full _scratch_mount
Signature · 1 line
2.34.1
Re: [PATCH v2 4/6] generic/623: do not run with overlayfs
Quoted 24 lines · Amir Goldstein
This test performs shutdown via
xfs_io-c shutdown.Overlayfs tests can use
_scratch_shutdown, but they cannot use "-c shutdown"xfs_iocommand without jumping through hoops, so by default we do not support it.Add this condition to
_require_xfs_io_commandand add the require statement to test generic/623 so it wont run with overlayfs.Reported-by André Almeida <[email protected]>Tested-by André Almeida <[email protected]>Signed-off-by Amir Goldstein <[email protected]>--- common/rc | 8 ++++++++ tests/generic/623 | 1 + 2 files changed, 9 insertions(+)common/rc +0
-0@@ -3033,6 +3033,14 @@ _require_xfs_io_command() touch $testfile
common/rc _require_xfs_io_command() · line 3043
Quoted 10 lines
common/rc +8
-0testio=`$XFS_IO_PROG -c "syncfs" $testfile 2>&1` ;; + "shutdown") + if [ $FSTYP = "overlay" ]; then + # Overlayfs tests can use _scratch_shutdown, but they + # cannot use "-c shutdown" xfs_io command without jumping + # through hoops, so by default we do not support it. + _notrun "xfs_io $command not supported on $FSTYP" + fi + ;;
Hmm... I'm not sure this's a good way.
For example, overlay/087 does xfs_io shutdown too, generally it should calls _require_xfs_io_command "shutdown" although it doesn't. If someone overlay test case hope to test as o/087 does, and it calls _require_xfs_io_command "shutdown", then it'll be _notrun.
If g/623 is not suitable for overlay, how about skip it for overlay clearly, by _exclude_fs overlay ?
Thanks,
Zorro
Quoted 18 lines
common/rc +0
-0*) testio=`$XFS_IO_PROG -c "help $command" 2>&1` esactests/generic/623 +1
-0@@ -16,6 +16,7 @@ _begin_fstest auto quick shutdown mmap _require_scratch_nocheck _require_scratch_shutdown +_require_xfs_io_command "shutdown" _scratch_mkfs &>> $seqres.full _scratch_mountSignature · 1 line
2.34.1
Re: [PATCH v2 4/6] generic/623: do not run with overlayfs
Quoted 31 lines · Zorro Lang
Quoted 29 lines · Amir Goldstein
This test performs shutdown via
xfs_io-c shutdown.Overlayfs tests can use
_scratch_shutdown, but they cannot use "-c shutdown"xfs_iocommand without jumping through hoops, so by default we do not support it.Add this condition to
_require_xfs_io_commandand add the require statement to test generic/623 so it wont run with overlayfs.Reported-by André Almeida <[email protected]>Tested-by André Almeida <[email protected]>Signed-off-by Amir Goldstein <[email protected]>--- common/rc | 8 ++++++++ tests/generic/623 | 1 + 2 files changed, 9 insertions(+)common/rc +3
-0@@ -3033,6 +3033,14 @@ _require_xfs_io_command() touch $testfile testio=`$XFS_IO_PROG -c "syncfs" $testfile 2>&1` ;; + "shutdown") + if [ $FSTYP = "overlay" ]; then + # Overlayfs tests can use _scratch_shutdown, but they
Quoted 8 lines
common/rc _require_xfs_io_command() · line 3043
Quoted 5 lines
common/rc +5
-0+ # cannot use "-c shutdown" xfs_io command without jumping + # through hoops, so by default we do not support it. + _notrun "xfs_io $command not supported on $FSTYP" + fi + ;;Hmm... I'm not sure this's a good way.
For example, overlay/087 doesxfs_ioshutdown too,
Yes it does but look at the effort needed to do that properly:
$XFS_IO_PROG -x -c "open $(_scratch_shutdown_handle)" -c 'shutdown -f
' -c close -c syncfs $SCRATCH_MNT | \
grep -vF '[00'Quoted 4 lines
generally it should calls
_require_xfs_io_command"shutdown" although it doesn't. If someone overlay test case hope to test as o/087 does, and it calls_require_xfs_io_command"shutdown", then it'll be_notrun.
If someone knows enough to perform the dance above with _scratch_shutdown_handle then that someone should know enough not to call _require_xfs_io_command "shutdown".
OTOH, if someone doesn't know then default is to not run.
Quoted 4 lines
If g/623 is not suitable for overlay, how about skip it for overlay clearly, by
_exclude_fs overlay?
I do not personally mind doing this _exclude_fs overlay, but it is usually prefered to require what the test needs.
Whatever you prefer is fine by me.
Thanks,
Amir.
Re: [PATCH v2 4/6] generic/623: do not run with overlayfs
Quoted 52 lines · Amir Goldstein
Quoted 31 lines · Zorro Lang
Quoted 29 lines · Amir Goldstein
This test performs shutdown via
xfs_io-c shutdown.Overlayfs tests can use
_scratch_shutdown, but they cannot use "-c shutdown"xfs_iocommand without jumping through hoops, so by default we do not support it.Add this condition to
_require_xfs_io_commandand add the require statement to test generic/623 so it wont run with overlayfs.Reported-by André Almeida <[email protected]>Tested-by André Almeida <[email protected]>Signed-off-by Amir Goldstein <[email protected]>--- common/rc | 8 ++++++++ tests/generic/623 | 1 + 2 files changed, 9 insertions(+)common/rc +3
-0@@ -3033,6 +3033,14 @@ _require_xfs_io_command() touch $testfile testio=`$XFS_IO_PROG -c "syncfs" $testfile 2>&1` ;; + "shutdown") + if [ $FSTYP = "overlay" ]; then + # Overlayfs tests can use _scratch_shutdown, but theyQuoted 8 lines
common/rc _require_xfs_io_command() · line 3043
Quoted 5 lines
common/rc +5
-0+ # cannot use "-c shutdown" xfs_io command without jumping + # through hoops, so by default we do not support it. + _notrun "xfs_io $command not supported on $FSTYP" + fi + ;;Hmm... I'm not sure this's a good way.
For example, overlay/087 doesxfs_ioshutdown too,Yes it does but look at the effort needed to do that properly:
$XFS_IO_PROG -x -c "open $(_scratch_shutdown_handle)" -c 'shutdown -f
' -c close -c syncfs $SCRATCH_MNT | \ grep -vF '[00'Quoted 4 lines
generally it should calls
_require_xfs_io_command"shutdown" although it doesn't. If someone overlay test case hope to test as o/087 does, and it calls_require_xfs_io_command"shutdown", then it'll be_notrun.
Quoted 4 lines
If someone knows enough to perform the dance above with
_scratch_shutdown_handlethen that someone should know enough not to call_require_xfs_io_command"shutdown".
OTOH, if someone doesn't know then default is to not run.
Sure, I can understand that, just this logic is a bit *obscure* :) It sounds like:
"If an overlay test case wants to do xfs_io shutdown, it shouldn't call _require_xfs_io_command "shutdown". Or call that to skip a shutdown test on overlay :)"
And the expected result of _require_xfs_io_command "shutdown" will be totally opposite with _require_scratch_shutdown on overlay, that might be confused. Can we have a clearer way to deal with that?
Quoted 10 lines
Quoted 5 lines
If g/623 is not suitable for overlay, how about skip it for overlay clearly, by
_exclude_fs overlay?I do not personally mind doing this
_exclude_fsoverlay, but it is usually prefered to require what the test needs.Whatever you prefer is fine by me.
I don't perfer "_exclude_fs overlay", just before we have a clear way to deal with overlay shutdown, this might be simple and forthright :)
Thanks,
Zorro
Quoted 4 lines
Thanks,
Amir.
Re: [PATCH v2 4/6] generic/623: do not run with overlayfs
Quoted 59 lines · Zorro Lang
Quoted 52 lines · Amir Goldstein
Quoted 31 lines · Zorro Lang
Quoted 29 lines · Amir Goldstein
This test performs shutdown via
xfs_io-c shutdown.Overlayfs tests can use
_scratch_shutdown, but they cannot use "-c shutdown"xfs_iocommand without jumping through hoops, so by default we do not support it.Add this condition to
_require_xfs_io_commandand add the require statement to test generic/623 so it wont run with overlayfs.Reported-by André Almeida <[email protected]>Tested-by André Almeida <[email protected]>Signed-off-by Amir Goldstein <[email protected]>--- common/rc | 8 ++++++++ tests/generic/623 | 1 + 2 files changed, 9 insertions(+)common/rc +3
-0@@ -3033,6 +3033,14 @@ _require_xfs_io_command() touch $testfile testio=`$XFS_IO_PROG -c "syncfs" $testfile 2>&1` ;; + "shutdown") + if [ $FSTYP = "overlay" ]; then + # Overlayfs tests can use _scratch_shutdown, but theyQuoted 8 lines
common/rc _require_xfs_io_command() · line 3043
Quoted 5 lines
common/rc +5
-0+ # cannot use "-c shutdown" xfs_io command without jumping + # through hoops, so by default we do not support it. + _notrun "xfs_io $command not supported on $FSTYP" + fi + ;;Hmm... I'm not sure this's a good way.
For example, overlay/087 doesxfs_ioshutdown too,Yes it does but look at the effort needed to do that properly:
$XFS_IO_PROG -x -c "open $(_scratch_shutdown_handle)" -c 'shutdown -f
' -c close -c syncfs $SCRATCH_MNT | \ grep -vF '[00'Quoted 4 lines
generally it should calls
_require_xfs_io_command"shutdown" although it doesn't. If someone overlay test case hope to test as o/087 does, and it calls_require_xfs_io_command"shutdown", then it'll be_notrun.Quoted 4 lines
If someone knows enough to perform the dance above with
_scratch_shutdown_handlethen that someone should know enough not to call_require_xfs_io_command"shutdown".
OTOH, if someone doesn't know then default is to not run.
Quoted 9 lines
Sure, I can understand that, just this logic is a bit *obscure* :) It sounds like:
"If an overlay test case wants to doxfs_ioshutdown, it shouldn't call_require_xfs_io_command"shutdown". Or call that to skip a shutdown test on overlay :)"And the expected result of
_require_xfs_io_command"shutdown" will be totally opposite with_require_scratch_shutdownon overlay, that might be confused. Can we have a clearer way to deal with that?
I don't really understand the confusion.
_require_xfs_io_command "shutdown"
Like any other _require statement
requires support for what this test does -
meaning that a test does xfs_io -c shutdown, just like test generic/623 does
and _require_scratch_shutdown implies that the test does _scratch_shutdown
FSTYP overlay happens to be able to do _scratch_shutdown but not able to do xfs_io -c shutdown $SCRATCH_MNT
The different _require statements simply reflect reality as it is.
We can solve the confused about o/087 not having _require_xfs_io_command "shutdown"
by moving the special hand crafted xfs_io command in o/087 to a helper _scratch_shutdown_and_syncfs to hide those internal implementation details from test writers.
See attached patch.
Thanks,
Amir.
From 3b4d6ff3ce909fa4b385a8d075bbac1f32b82cb8 Mon Sep 17 00:00:00 2001
From: Amir Goldstein <[email protected]>
Date: Fri, 6 Jun 2025 09:11:17 +0200
Subject: [PATCH] fstests: add helper _scratch_shutdown_and_syncfs
Test xfs/546 has to chain syncfs after shutdown and cannot use the _scratch_shitdown helper, because after shutdown a fd cannot be opened to execute syncfs on.
The xfs_io command of chaining syncfs after shutdown is rather more complex to execute in the derived overlayfs test overlay/087.
Add a helper to abstract this complexity from test writers. Add a _require statement to match.
--- common/rc | 27 +++++++++++++++++++++++++++ tests/overlay/087 | 13 +++---------- tests/xfs/546 | 5 ++--- 3 files changed, 32 insertions(+), 13 deletions(-)
common/rc +27 -0
@@ -595,6 +595,27 @@ _scratch_shutdown_handle() fi } +_scratch_shutdown_and_syncfs() +{ + if [ $FSTYP = "overlay" ]; then + # In lagacy overlay usage, it may specify directory as + # SCRATCH_DEV, in this case OVL_BASE_SCRATCH_DEV + # will be null, so check OVL_BASE_SCRATCH_DEV before + # running shutdown to avoid shutting down base fs accidently. + if [ -z $OVL_BASE_SCRATCH_DEV ]; then + _fail "_scratch_shutdown: call _require_scratch_shutdown first in test" + fi + # This command is complicated a bit because in the case of overlayfs the + # syncfs fd needs to be opened before shutdown and it is different from the + # shutdown fd, so we cannot use the _scratch_shutdown() helper. + # Filter out xfs_io output of active fds. + $XFS_IO_PROG -x -c "open $(_scratch_shutdown_handle)" -c 'shutdown -f ' \ + -c close -c syncfs $SCRATCH_MNT | grep -vF '[00' + else + $XFS_IO_PROG -x -c 'shutdown -f ' -c syncfs $SCRATCH_MNT + fi +} + _move_mount() { local mnt=$1
@@ -4110,6 +4131,12 @@ _require_scratch_shutdown() _scratch_unmount } +_require_scratch_shutdown_and_syncfs() +{ + _require_xfs_io_command syncfs + _require_scratch_shutdown +} + _check_s_dax() { local target=$1
tests/overlay/087 +3 -10
@@ -32,9 +32,8 @@ _begin_fstest auto quick mount shutdown # Modify as appropriate. -_require_xfs_io_command syncfs _require_scratch_nocheck -_require_scratch_shutdown +_require_scratch_shutdown_and_syncfs [ "$OVL_BASE_FSTYP" == "xfs" ] || \ _notrun "base fs $OVL_BASE_FSTYP has unknown behavior with syncfs after shutdown"
@@ -43,19 +42,13 @@ _require_scratch_shutdown # bother checking the filesystem afterwards since we never wrote anything. echo "=== syncfs after shutdown" _scratch_mount -# This command is complicated a bit because in the case of overlayfs the -# syncfs fd needs to be opened before shutdown and it is different from the -# shutdown fd, so we cannot use the _scratch_shutdown() helper. -# Filter out xfs_io output of active fds. -$XFS_IO_PROG -x -c "open $(_scratch_shutdown_handle)" -c 'shutdown -f ' -c close -c syncfs $SCRATCH_MNT | \ - grep -vF '[00' +_scratch_shutdown_and_syncfs # Now repeat the same test with a volatile overlayfs mount and expect no error _scratch_unmount echo "=== syncfs after shutdown (volatile)" _scratch_mount -o volatile -$XFS_IO_PROG -x -c "open $(_scratch_shutdown_handle)" -c 'shutdown -f ' -c close -c syncfs $SCRATCH_MNT | \ - grep -vF '[00' +_scratch_shutdown_and_syncfs # success, all done status=0
tests/xfs/546 +2 -3
@@ -27,14 +27,13 @@ _begin_fstest auto quick shutdown # Modify as appropriate. -_require_xfs_io_command syncfs _require_scratch_nocheck -_require_scratch_shutdown +_require_scratch_shutdown_and_syncfs # Reuse the fs formatted when we checked for the shutdown ioctl, and don't # bother checking the filesystem afterwards since we never wrote anything. _scratch_mount -$XFS_IO_PROG -x -c 'shutdown -f ' -c syncfs $SCRATCH_MNT +_scratch_shutdown_and_syncfs # success, all done status=0
Signature · 1 line
2.34.1
Re: [PATCH v2 4/6] generic/623: do not run with overlayfs
Quoted 84 lines · Amir Goldstein
Quoted 59 lines · Zorro Lang
Quoted 52 lines · Amir Goldstein
Quoted 31 lines · Zorro Lang
Quoted 29 lines · Amir Goldstein
This test performs shutdown via
xfs_io-c shutdown.Overlayfs tests can use
_scratch_shutdown, but they cannot use "-c shutdown"xfs_iocommand without jumping through hoops, so by default we do not support it.Add this condition to
_require_xfs_io_commandand add the require statement to test generic/623 so it wont run with overlayfs.Reported-by André Almeida <[email protected]>Tested-by André Almeida <[email protected]>Signed-off-by Amir Goldstein <[email protected]>--- common/rc | 8 ++++++++ tests/generic/623 | 1 + 2 files changed, 9 insertions(+)common/rc +3
-0@@ -3033,6 +3033,14 @@ _require_xfs_io_command() touch $testfile testio=`$XFS_IO_PROG -c "syncfs" $testfile 2>&1` ;; + "shutdown") + if [ $FSTYP = "overlay" ]; then + # Overlayfs tests can use _scratch_shutdown, but theyQuoted 8 lines
common/rc _require_xfs_io_command() · line 3043
Quoted 5 lines
common/rc +5
-0+ # cannot use "-c shutdown" xfs_io command without jumping + # through hoops, so by default we do not support it. + _notrun "xfs_io $command not supported on $FSTYP" + fi + ;;Hmm... I'm not sure this's a good way.
For example, overlay/087 doesxfs_ioshutdown too,Yes it does but look at the effort needed to do that properly:
$XFS_IO_PROG -x -c "open $(_scratch_shutdown_handle)" -c 'shutdown -f
' -c close -c syncfs $SCRATCH_MNT | \ grep -vF '[00'Quoted 4 lines
generally it should calls
_require_xfs_io_command"shutdown" although it doesn't. If someone overlay test case hope to test as o/087 does, and it calls_require_xfs_io_command"shutdown", then it'll be_notrun.Quoted 4 lines
If someone knows enough to perform the dance above with
_scratch_shutdown_handlethen that someone should know enough not to call_require_xfs_io_command"shutdown".
OTOH, if someone doesn't know then default is to not run.Quoted 9 lines
Sure, I can understand that, just this logic is a bit *obscure* :) It sounds like:
"If an overlay test case wants to doxfs_ioshutdown, it shouldn't call_require_xfs_io_command"shutdown". Or call that to skip a shutdown test on overlay :)"And the expected result of
_require_xfs_io_command"shutdown" will be totally opposite with_require_scratch_shutdownon overlay, that might be confused. Can we have a clearer way to deal with that?I don't really understand the confusion.
_require_xfs_io_command"shutdown"Like any other
_requirestatement
requires support for what this test does -
meaning that a test doesxfs_io-c shutdown, just like test generic/623 doesand
_require_scratch_shutdownimplies that the test does_scratch_shutdownFSTYP overlay happens to be able to do
_scratch_shutdownbut not able to doxfs_io-c shutdown $SCRATCH_MNT
Quoted 8 lines
The different
_requirestatements simply reflect reality as it is.We can solve the confused about o/087 not having
_require_xfs_io_command"shutdown"
by moving the special hand craftedxfs_iocommand in o/087 to a helper_scratch_shutdown_and_syncfsto hide those internal implementation details from test writers.
See attached patch.
Hmm... give me a moment to order my thoughts step by step :)
There're only 2 cases tend to do xfs_io shutdown on overlay currently (others are xfs specific test cases):
$ grep -rsn shutdown tests/|grep -- "-c" tests/generic/623:29:$XFS_IO_PROG -x -c "mmap 0 4k" -c "mwrite 0 4k" -c shutdown -c fsync \ tests/overlay/087:50:$XFS_IO_PROG -x -c "open $(_scratch_shutdown_handle)" -c 'shutdown -f ' -c close -c syncfs $SCRATCH_MNT | \ tests/overlay/087:57:$XFS_IO_PROG -x -c "open $(_scratch_shutdown_handle)" -c 'shutdown -f ' -c close -c syncfs $SCRATCH_MNT | \ ...
others shutdown cases nearly all use *_scratch_shutdown* with *_require_scratch_shutdown*, these two functions are consistent in code logic. And no one calls "_require_xfs_io_command shutdown" currently.
So g/623 and o/087 are specifal, actually they call _require_scratch_shutdown too, that makes sense for o/087. Now only g/623 doesn't make sense. Now we need to help it to make sense.
I think the key is in _require_scratch_shutdown function [1], how about add an argument to clearly tell it we need to check shutdown "only on the top layer $SCRATCH_MNT" or "try the lowest layer $BASE_SCRATCH_MNT if there is".
For example:
common/rc +4 -2
@@ -4075,15 +4075,17 @@ _require_exportfs() _require_open_by_handle } -# Does shutdown work on this fs? +# Does shutdown work on this [lower|top] layer fs? _require_scratch_shutdown() { + local layer="${1:-lower}" + [ -x $here/src/godown ] || _notrun "src/godown executable not found" _scratch_mkfs > /dev/null 2>&1 || _notrun "_scratch_mkfs failed on $SCRATCH_DEV" _scratch_mount - if [ $FSTYP = "overlay" ]; then + if [ $FSTYP = "overlay" -a "$level" = "lower" ]; then if [ -z $OVL_BASE_SCRATCH_DEV ]; then # In lagacy overlay usage, it may specify directory as # SCRATCH_DEV, in this case OVL_BASE_SCRATCH_DEV
tests/generic/623 +1 -1
@@ -15,7 +15,7 @@ _begin_fstest auto quick shutdown mmap "xfs: restore shutdown check in mapped write fault path" _require_scratch_nocheck -_require_scratch_shutdown +_require_scratch_shutdown top _scratch_mkfs &>> $seqres.full _scratch_mount
Thanks,
Zorro
[1]_require_scratch_shutdown()
{
[ -x $here/src/godown ] || _notrun "src/godown executable not found"
_scratch_mkfs > /dev/null 2>&1 || _notrun "_scratch_mkfs failed on $SCRATCH_DEV" _scratch_mount
if [ $FSTYP = "overlay" ]; then
if [ -z $OVL_BASE_SCRATCH_DEV ]; then
# In lagacy overlay usage, it may specify directory as
# SCRATCH_DEV, in this case OVL_BASE_SCRATCH_DEV
# will be null, so check OVL_BASE_SCRATCH_DEV before
# running shutdown to avoid shutting down base fs accidently. _notrun "This test requires a valid $OVL_BASE_SCRATCH_DEV as ovl base fs"
else
$here/src/godown -f $OVL_BASE_SCRATCH_MNT 2>&1 \
|| _notrun "Underlying filesystem does not support shutdown"
fi
else
$here/src/godown -f $SCRATCH_MNT 2>&1 \
|| _notrun "$FSTYP does not support shutdown"
fi
_scratch_unmount
}
Quoted 3 lines
Thanks,
Amir.
Re: [PATCH v2 4/6] generic/623: do not run with overlayfs
Quoted 149 lines · Zorro Lang
Quoted 84 lines · Amir Goldstein
Quoted 59 lines · Zorro Lang
Quoted 52 lines · Amir Goldstein
Quoted 31 lines · Zorro Lang
Quoted 29 lines · Amir Goldstein
This test performs shutdown via
xfs_io-c shutdown.Overlayfs tests can use
_scratch_shutdown, but they cannot use "-c shutdown"xfs_iocommand without jumping through hoops, so by default we do not support it.Add this condition to
_require_xfs_io_commandand add the require statement to test generic/623 so it wont run with overlayfs.Reported-by André Almeida <[email protected]>Tested-by André Almeida <[email protected]>Signed-off-by Amir Goldstein <[email protected]>--- common/rc | 8 ++++++++ tests/generic/623 | 1 + 2 files changed, 9 insertions(+)common/rc +3
-0@@ -3033,6 +3033,14 @@ _require_xfs_io_command() touch $testfile testio=`$XFS_IO_PROG -c "syncfs" $testfile 2>&1` ;; + "shutdown") + if [ $FSTYP = "overlay" ]; then + # Overlayfs tests can use _scratch_shutdown, but theyQuoted 8 lines
common/rc _require_xfs_io_command() · line 3043
Quoted 5 lines
common/rc +5
-0+ # cannot use "-c shutdown" xfs_io command without jumping + # through hoops, so by default we do not support it. + _notrun "xfs_io $command not supported on $FSTYP" + fi + ;;Hmm... I'm not sure this's a good way.
For example, overlay/087 doesxfs_ioshutdown too,Yes it does but look at the effort needed to do that properly:
$XFS_IO_PROG -x -c "open $(_scratch_shutdown_handle)" -c 'shutdown -f
' -c close -c syncfs $SCRATCH_MNT | \ grep -vF '[00'Quoted 4 lines
generally it should calls
_require_xfs_io_command"shutdown" although it doesn't. If someone overlay test case hope to test as o/087 does, and it calls_require_xfs_io_command"shutdown", then it'll be_notrun.Quoted 4 lines
If someone knows enough to perform the dance above with
_scratch_shutdown_handlethen that someone should know enough not to call_require_xfs_io_command"shutdown".
OTOH, if someone doesn't know then default is to not run.Quoted 9 lines
Sure, I can understand that, just this logic is a bit *obscure* :) It sounds like:
"If an overlay test case wants to doxfs_ioshutdown, it shouldn't call_require_xfs_io_command"shutdown". Or call that to skip a shutdown test on overlay :)"And the expected result of
_require_xfs_io_command"shutdown" will be totally opposite with_require_scratch_shutdownon overlay, that might be confused. Can we have a clearer way to deal with that?I don't really understand the confusion.
_require_xfs_io_command"shutdown"Like any other
_requirestatement
requires support for what this test does -
meaning that a test doesxfs_io-c shutdown, just like test generic/623 doesand
_require_scratch_shutdownimplies that the test does_scratch_shutdownFSTYP overlay happens to be able to do
_scratch_shutdownbut not able to doxfs_io-c shutdown $SCRATCH_MNTQuoted 8 lines
The different
_requirestatements simply reflect reality as it is.We can solve the confused about o/087 not having
_require_xfs_io_command"shutdown"
by moving the special hand craftedxfs_iocommand in o/087 to a helper_scratch_shutdown_and_syncfsto hide those internal implementation details from test writers.
See attached patch.Hmm... give me a moment to order my thoughts step by step :)
There're only 2 cases tend to do
xfs_ioshutdown on overlay currently (others are xfs specific test cases):$ grep -rsn shutdown tests/|grep -- "-c" tests/generic/623:29:$XFS_IO_PROG -x -c "mmap 0 4k" -c "mwrite 0 4k" -c shutdown -c fsync \ tests/overlay/087:50:$XFS_IO_PROG -x -c "open $(_scratch_shutdown_handle)" -c 'shutdown -f ' -c close -c syncfs $SCRATCH_MNT | \ tests/overlay/087:57:$XFS_IO_PROG -x -c "open $(_scratch_shutdown_handle)" -c 'shutdown -f ' -c close -c syncfs $SCRATCH_MNT | \ ...others shutdown cases nearly all use *_scratch_shutdown* with *_require_scratch_shutdown*, these two functions are consistent in code logic. And no one calls "
_require_xfs_io_commandshutdown" currently.So g/623 and o/087 are specifal, actually they call
_require_scratch_shutdowntoo, that makes sense for o/087. Now only g/623 doesn't make sense. Now we need to help it to make sense.I think the key is in
_require_scratch_shutdownfunction [1], how about add an argument to clearly tell it we need to check shutdown "only on the top layer $SCRATCH_MNT" or "try the lowest layer $BASE_SCRATCH_MNT if there is".For example:
common/rc +4
-2@@ -4075,15 +4075,17 @@ _require_exportfs() _require_open_by_handle } -# Does shutdown work on this fs? +# Does shutdown work on this [lower|top] layer fs? _require_scratch_shutdown() { + local layer="${1:-lower}" + [ -x $here/src/godown ] || _notrun "src/godown executable not found" _scratch_mkfs > /dev/null 2>&1 || _notrun "_scratch_mkfs failed on $SCRATCH_DEV" _scratch_mount - if [ $FSTYP = "overlay" ]; then + if [ $FSTYP = "overlay" -a "$level" = "lower" ]; then if [ -z $OVL_BASE_SCRATCH_DEV ]; then # In lagacy overlay usage, it may specify directory as # SCRATCH_DEV, in this case OVL_BASE_SCRATCH_DEVtests/generic/623 +0
-0
tests/generic/623 _begin_fstest auto quick shutdown mmap · line 18
Quoted 5 lines
tests/generic/623 +1
-1@@ -15,7 +15,7 @@ _begin_fstest auto quick shutdown mmap "xfs: restore shutdown check in mapped write fault path" _require_scratch_nocheck -_require_scratch_shutdown +_require_scratch_shutdown top
Sorry I find this utterly confusing.
Think of all the 95% of fstests developers that do not care about overlayfs what does this top mean to them and why should they use it for tests that do xfs_io -c shutdown and not for tests that do _scratch_shutdown?
The test author and reviewers should be able to look at the tests and easily derive what the test requirements should be according to simple rules. For example:
1. A test that calls _scrash_shutdown needs to _require_scratch_shutdown
2. A test that calls _scratch_shutdown_and_syncfs needs to _require_scratch_shutdown_and_syncfs
3. A test that calls xfs_io -c shutdown needs to _require_xfs_io_shutdown
I completely understand why you do not like my hack of _require_xfs_io_command "shutdown"
Would you approve if it was an explicit _require_xfs_io_shutdown helper?
# Requirements for tests that call xfs_io -c shutdown instead of using the # _scratch_shutdown helper_require_xfs_io_shutdown()
{
if [ $FSTYP = "overlay" ]; then
# Overlayfs tests can use _scratch_shutdown, but they # cannot use "xfs_io -c shutdown" command
without jumping
# through hoops, so by default we do not support it.
_notrun "xfs_io -c shutdown not supported on $FSTYP"
fi
_require_xfs_io_command "shutdown"
_require_scratch_shutdown
}Thanks,
Amir
Re: [PATCH v2 4/6] generic/623: do not run with overlayfs
Quoted 171 lines · Amir Goldstein
Quoted 149 lines · Zorro Lang
Quoted 84 lines · Amir Goldstein
Quoted 59 lines · Zorro Lang
Quoted 52 lines · Amir Goldstein
Quoted 31 lines · Zorro Lang
Quoted 29 lines · Amir Goldstein
This test performs shutdown via
xfs_io-c shutdown.Overlayfs tests can use
_scratch_shutdown, but they cannot use "-c shutdown"xfs_iocommand without jumping through hoops, so by default we do not support it.Add this condition to
_require_xfs_io_commandand add the require statement to test generic/623 so it wont run with overlayfs.Reported-by André Almeida <[email protected]>Tested-by André Almeida <[email protected]>Signed-off-by Amir Goldstein <[email protected]>--- common/rc | 8 ++++++++ tests/generic/623 | 1 + 2 files changed, 9 insertions(+)common/rc +3
-0@@ -3033,6 +3033,14 @@ _require_xfs_io_command() touch $testfile testio=`$XFS_IO_PROG -c "syncfs" $testfile 2>&1` ;; + "shutdown") + if [ $FSTYP = "overlay" ]; then + # Overlayfs tests can use _scratch_shutdown, but theyQuoted 8 lines
common/rc _require_xfs_io_command() · line 3043
Quoted 5 lines
common/rc +5
-0+ # cannot use "-c shutdown" xfs_io command without jumping + # through hoops, so by default we do not support it. + _notrun "xfs_io $command not supported on $FSTYP" + fi + ;;Hmm... I'm not sure this's a good way.
For example, overlay/087 doesxfs_ioshutdown too,Yes it does but look at the effort needed to do that properly:
$XFS_IO_PROG -x -c "open $(_scratch_shutdown_handle)" -c 'shutdown -f
' -c close -c syncfs $SCRATCH_MNT | \ grep -vF '[00'Quoted 4 lines
generally it should calls
_require_xfs_io_command"shutdown" although it doesn't. If someone overlay test case hope to test as o/087 does, and it calls_require_xfs_io_command"shutdown", then it'll be_notrun.Quoted 4 lines
If someone knows enough to perform the dance above with
_scratch_shutdown_handlethen that someone should know enough not to call_require_xfs_io_command"shutdown".
OTOH, if someone doesn't know then default is to not run.Quoted 9 lines
Sure, I can understand that, just this logic is a bit *obscure* :) It sounds like:
"If an overlay test case wants to doxfs_ioshutdown, it shouldn't call_require_xfs_io_command"shutdown". Or call that to skip a shutdown test on overlay :)"And the expected result of
_require_xfs_io_command"shutdown" will be totally opposite with_require_scratch_shutdownon overlay, that might be confused. Can we have a clearer way to deal with that?I don't really understand the confusion.
_require_xfs_io_command"shutdown"Like any other
_requirestatement
requires support for what this test does -
meaning that a test doesxfs_io-c shutdown, just like test generic/623 doesand
_require_scratch_shutdownimplies that the test does_scratch_shutdownFSTYP overlay happens to be able to do
_scratch_shutdownbut not able to doxfs_io-c shutdown $SCRATCH_MNTQuoted 8 lines
The different
_requirestatements simply reflect reality as it is.We can solve the confused about o/087 not having
_require_xfs_io_command"shutdown"
by moving the special hand craftedxfs_iocommand in o/087 to a helper_scratch_shutdown_and_syncfsto hide those internal implementation details from test writers.
See attached patch.Hmm... give me a moment to order my thoughts step by step :)
There're only 2 cases tend to do
xfs_ioshutdown on overlay currently (others are xfs specific test cases):$ grep -rsn shutdown tests/|grep -- "-c" tests/generic/623:29:$XFS_IO_PROG -x -c "mmap 0 4k" -c "mwrite 0 4k" -c shutdown -c fsync \ tests/overlay/087:50:$XFS_IO_PROG -x -c "open $(_scratch_shutdown_handle)" -c 'shutdown -f ' -c close -c syncfs $SCRATCH_MNT | \ tests/overlay/087:57:$XFS_IO_PROG -x -c "open $(_scratch_shutdown_handle)" -c 'shutdown -f ' -c close -c syncfs $SCRATCH_MNT | \ ...others shutdown cases nearly all use *_scratch_shutdown* with *_require_scratch_shutdown*, these two functions are consistent in code logic. And no one calls "
_require_xfs_io_commandshutdown" currently.So g/623 and o/087 are specifal, actually they call
_require_scratch_shutdowntoo, that makes sense for o/087. Now only g/623 doesn't make sense. Now we need to help it to make sense.I think the key is in
_require_scratch_shutdownfunction [1], how about add an argument to clearly tell it we need to check shutdown "only on the top layer $SCRATCH_MNT" or "try the lowest layer $BASE_SCRATCH_MNT if there is".For example:
common/rc +4
-2@@ -4075,15 +4075,17 @@ _require_exportfs() _require_open_by_handle } -# Does shutdown work on this fs? +# Does shutdown work on this [lower|top] layer fs? _require_scratch_shutdown() { + local layer="${1:-lower}" + [ -x $here/src/godown ] || _notrun "src/godown executable not found" _scratch_mkfs > /dev/null 2>&1 || _notrun "_scratch_mkfs failed on $SCRATCH_DEV" _scratch_mount - if [ $FSTYP = "overlay" ]; then + if [ $FSTYP = "overlay" -a "$level" = "lower" ]; then if [ -z $OVL_BASE_SCRATCH_DEV ]; then # In lagacy overlay usage, it may specify directory as # SCRATCH_DEV, in this case OVL_BASE_SCRATCH_DEVtests/generic/623 +0
-0tests/generic/623 _begin_fstest auto quick shutdown mmap · line 18
Quoted 5 lines
tests/generic/623 +1
-1@@ -15,7 +15,7 @@ _begin_fstest auto quick shutdown mmap "xfs: restore shutdown check in mapped write fault path" _require_scratch_nocheck -_require_scratch_shutdown +_require_scratch_shutdown topSorry I find this utterly confusing.
Think of all the 95% of fstests developers that do not care about overlayfs what does this top mean to them and why should they use it for tests that do
xfs_io-c shutdown and not for tests that do_scratch_shutdown?The test author and reviewers should be able to look at the tests and easily derive what the test requirements should be according to simple rules. For example:
1. A test that calls
_scrash_shutdownneeds to_require_scratch_shutdown
2. A test that calls_scratch_shutdown_and_syncfsneeds to_require_scratch_shutdown_and_syncfs
3. A test that callsxfs_io-c shutdown needs to_require_xfs_io_shutdown
Quoted 7 lines
I completely understand why you do not like my hack of
_require_xfs_io_command"shutdown"Would you approve if it was an explicit
_require_xfs_io_shutdownhelper?# Requirements for tests that call
xfs_io-c shutdown instead of using the #_scratch_shutdownhelper
OK, but you might metion that it's better not be used if _scratch_shutdown_handle is called for xfs_io, as we hope the lower layer fs supports shutdown at that time:)
Actually I'm wondering if we should help xfstests to support BASE_FSTYP and FSTYP for more upper layer fs, likes nfs, cifs, and so on. If so, overlay will not be the only one fs who uses BASE_FSTYP and BASE_SCRATCH_DEV things, then we need to differentiate if a feature (e.g. shutdown) is needed by upper layer fs or underlying fs in a case.
...
BTW, a question which isn't belong to this patch:) There're also some failures from those xfstests overlay cases which run unionmount-testsuite (can't rememember all, maybe o/102~109, o/144~117). The error (diff) output are similar as [1]. Is there a fix for that too? Or I missed the fix?
Thanks,
Zorro
[1]
103.out.bad renamed from dev/fd/63 +15 -0
@@ -1,2 +1,17 @@ QA output created by 103 +mount: /mnt/fstests/SCRATCH_DIR/union/m: wrong fs type, bad option, bad superblock on overlay, missing codepage or helper program, or other error. + dmesg(1) may have more information after failed mount system call. +Traceback (most recent call last): + File "/mnt/tests/gitlab.cee.redhat.com/kernel-qe/kernel/-/archive/master/kernel-master.tar.bz2/filesystems/xfstests/unionmount-testsuite/./run", line 362, in <module> + func(ctx) + File "/mnt/tests/gitlab.cee.redhat.com/kernel-qe/kernel/-/archive/master/kernel-master.tar.bz2/filesystems/xfstests/unionmount-testsuite/tests/rename-file.py", line 71, in subtest_5 + ctx.rename(f, f2) + File "/mnt/tests/gitlab.cee.redhat.com/kernel-qe/kernel/-/archive/master/kernel-master.tar.bz2/filesystems/xfstests/unionmount-testsuite/context.py", line 1254, in rename + remount_union(self, rotate_upper=True) + File "/mnt/tests/gitlab.cee.redhat.com/kernel-qe/kernel/-/archive/master/kernel-master.tar.bz2/filesystems/xfstests/unionmount-testsuite/remount_union.py", line 35, in remount_union + system(cmd) + File "/mnt/tests/gitlab.cee.redhat.com/kernel-qe/kernel/-/archive/master/kernel-master.tar.bz2/filesystems/xfstests/unionmount-testsuite/tool_box.py", line 25, in system + raise RuntimeError("Command failed: " + command) +RuntimeError: Command failed: mount -t overlay overlay /mnt/fstests/SCRATCH_DIR/union/m -orw,xino=on -olowerdir=/mnt/fstests/SCRATCH_DIR/union/6/u:/mnt/fstests/SCRATCH_DIR/union/5/u:/mnt/fstests/SCRATCH_DIR/union/4/u:/mnt/fstests/SCRATCH_DIR/union/3/u:/mnt/fstests/SCRATCH_DIR/union/2/u:/mnt/fstests/SCRATCH_DIR/union/1/u:/mnt/fstests/SCRATCH_DIR/union/0/u:/mnt/fstests/SCRATCH_DIR/union/l,upperdir=/mnt/fstests/SCRATCH_DIR/union/7/u,workdir=/mnt/fstests/SCRATCH_DIR/union/7/w +unionmount testsuite failed! see /var/lib/xfstests/results//overlay/103.full for details. Silence is golden
Quoted 19 lines
_require_xfs_io_shutdown()
{
if [ $FSTYP = "overlay" ]; then
# Overlayfs tests can use_scratch_shutdown, but they # cannot use "xfs_io-c shutdown" command
without jumping# through hoops, so by default we do not support it. _notrun "xfs_io -c shutdown not supported on $FSTYP" fi _require_xfs_io_command "shutdown" _require_scratch_shutdown }Thanks,
Amir
Re: [PATCH v2 4/6] generic/623: do not run with overlayfs
Quoted 176 lines · Zorro Lang
Quoted 174 lines · Amir Goldstein
Quoted 149 lines · Zorro Lang
Quoted 84 lines · Amir Goldstein
Quoted 59 lines · Zorro Lang
Quoted 52 lines · Amir Goldstein
Quoted 31 lines · Zorro Lang
Quoted 29 lines · Amir Goldstein
This test performs shutdown via
xfs_io-c shutdown.Overlayfs tests can use
_scratch_shutdown, but they cannot use "-c shutdown"xfs_iocommand without jumping through hoops, so by default we do not support it.Add this condition to
_require_xfs_io_commandand add the require statement to test generic/623 so it wont run with overlayfs.Reported-by André Almeida <[email protected]>Tested-by André Almeida <[email protected]>Signed-off-by Amir Goldstein <[email protected]>--- common/rc | 8 ++++++++ tests/generic/623 | 1 + 2 files changed, 9 insertions(+)common/rc +3
-0@@ -3033,6 +3033,14 @@ _require_xfs_io_command() touch $testfile testio=`$XFS_IO_PROG -c "syncfs" $testfile 2>&1` ;; + "shutdown") + if [ $FSTYP = "overlay" ]; then + # Overlayfs tests can use _scratch_shutdown, but theyQuoted 8 lines
common/rc _require_xfs_io_command() · line 3043
Quoted 5 lines
common/rc +5
-0+ # cannot use "-c shutdown" xfs_io command without jumping + # through hoops, so by default we do not support it. + _notrun "xfs_io $command not supported on $FSTYP" + fi + ;;Hmm... I'm not sure this's a good way.
For example, overlay/087 doesxfs_ioshutdown too,Yes it does but look at the effort needed to do that properly:
$XFS_IO_PROG -x -c "open $(_scratch_shutdown_handle)" -c 'shutdown -f
' -c close -c syncfs $SCRATCH_MNT | \ grep -vF '[00'Quoted 4 lines
generally it should calls
_require_xfs_io_command"shutdown" although it doesn't. If someone overlay test case hope to test as o/087 does, and it calls_require_xfs_io_command"shutdown", then it'll be_notrun.Quoted 4 lines
If someone knows enough to perform the dance above with
_scratch_shutdown_handlethen that someone should know enough not to call_require_xfs_io_command"shutdown".
OTOH, if someone doesn't know then default is to not run.Quoted 9 lines
Sure, I can understand that, just this logic is a bit *obscure* :) It sounds like:
"If an overlay test case wants to doxfs_ioshutdown, it shouldn't call_require_xfs_io_command"shutdown". Or call that to skip a shutdown test on overlay :)"And the expected result of
_require_xfs_io_command"shutdown" will be totally opposite with_require_scratch_shutdownon overlay, that might be confused. Can we have a clearer way to deal with that?I don't really understand the confusion.
_require_xfs_io_command"shutdown"Like any other
_requirestatement
requires support for what this test does -
meaning that a test doesxfs_io-c shutdown, just like test generic/623 doesand
_require_scratch_shutdownimplies that the test does_scratch_shutdownFSTYP overlay happens to be able to do
_scratch_shutdownbut not able to doxfs_io-c shutdown $SCRATCH_MNTQuoted 8 lines
The different
_requirestatements simply reflect reality as it is.We can solve the confused about o/087 not having
_require_xfs_io_command"shutdown"
by moving the special hand craftedxfs_iocommand in o/087 to a helper_scratch_shutdown_and_syncfsto hide those internal implementation details from test writers.
See attached patch.Hmm... give me a moment to order my thoughts step by step :)
There're only 2 cases tend to do
xfs_ioshutdown on overlay currently (others are xfs specific test cases):$ grep -rsn shutdown tests/|grep -- "-c" tests/generic/623:29:$XFS_IO_PROG -x -c "mmap 0 4k" -c "mwrite 0 4k" -c shutdown -c fsync \ tests/overlay/087:50:$XFS_IO_PROG -x -c "open $(_scratch_shutdown_handle)" -c 'shutdown -f ' -c close -c syncfs $SCRATCH_MNT | \ tests/overlay/087:57:$XFS_IO_PROG -x -c "open $(_scratch_shutdown_handle)" -c 'shutdown -f ' -c close -c syncfs $SCRATCH_MNT | \ ...others shutdown cases nearly all use *_scratch_shutdown* with *_require_scratch_shutdown*, these two functions are consistent in code logic. And no one calls "
_require_xfs_io_commandshutdown" currently.So g/623 and o/087 are specifal, actually they call
_require_scratch_shutdowntoo, that makes sense for o/087. Now only g/623 doesn't make sense. Now we need to help it to make sense.I think the key is in
_require_scratch_shutdownfunction [1], how about add an argument to clearly tell it we need to check shutdown "only on the top layer $SCRATCH_MNT" or "try the lowest layer $BASE_SCRATCH_MNT if there is".For example:
common/rc +4
-2@@ -4075,15 +4075,17 @@ _require_exportfs() _require_open_by_handle } -# Does shutdown work on this fs? +# Does shutdown work on this [lower|top] layer fs? _require_scratch_shutdown() { + local layer="${1:-lower}" + [ -x $here/src/godown ] || _notrun "src/godown executable not found" _scratch_mkfs > /dev/null 2>&1 || _notrun "_scratch_mkfs failed on $SCRATCH_DEV" _scratch_mount - if [ $FSTYP = "overlay" ]; then + if [ $FSTYP = "overlay" -a "$level" = "lower" ]; then if [ -z $OVL_BASE_SCRATCH_DEV ]; then # In lagacy overlay usage, it may specify directory as # SCRATCH_DEV, in this case OVL_BASE_SCRATCH_DEVtests/generic/623 +0
-0tests/generic/623 _begin_fstest auto quick shutdown mmap · line 18
Quoted 5 lines
tests/generic/623 +1
-1@@ -15,7 +15,7 @@ _begin_fstest auto quick shutdown mmap "xfs: restore shutdown check in mapped write fault path" _require_scratch_nocheck -_require_scratch_shutdown +_require_scratch_shutdown topSorry I find this utterly confusing.
Think of all the 95% of fstests developers that do not care about overlayfs what does this top mean to them and why should they use it for tests that do
xfs_io-c shutdown and not for tests that do_scratch_shutdown?The test author and reviewers should be able to look at the tests and easily derive what the test requirements should be according to simple rules. For example:
1. A test that calls
_scrash_shutdownneeds to_require_scratch_shutdown
2. A test that calls_scratch_shutdown_and_syncfsneeds to_require_scratch_shutdown_and_syncfs
3. A test that callsxfs_io-c shutdown needs to_require_xfs_io_shutdownI completely understand why you do not like my hack of
_require_xfs_io_command"shutdown"
Quoted 8 lines
Quoted 4 lines
Would you approve if it was an explicit
_require_xfs_io_shutdownhelper?# Requirements for tests that call
xfs_io-c shutdown instead of using the #_scratch_shutdownhelperOK, but you might metion that it's better not be used if
_scratch_shutdown_handleis called forxfs_io, as we hope the lower layer fs supports shutdown at that time:)
Sure I'll add that
Quoted 2 lines
Actually I'm wondering if we should help xfstests to support
BASE_FSTYPand FSTYP for more upper layer fs, likes nfs, cifs, and so on.
I think that could be very useful, but will require cifs/nfs to implement more complicated _mount/umount/remount helpers like overlay.
Quoted 5 lines
If so, overlay will not be
the only one fs who usesBASE_FSTYPandBASE_SCRATCH_DEVthings, then we need to differentiate if a feature (e.g. shutdown) is needed by upper layer fs or underlying fs in a case.
First of all, terminology. Many get this wrong.
In overlayfs, the "upper" and "lower" refer to the different underlying layers. In fstests BASE_SCRATCH_DEV is always the same for both OVL_UPPER and OVL_LOWER layers.
Referring to the BASE_SCRATCH_DEV as "underlying" or "base" fs is unambiguous in all cases of overlay/nfs/cifs.
I do not have a good terminology to offer for referring to the "fs under test" be it overlay/cifs/nfs. You are welcome to offer your suggestions.
Quoted 5 lines
...
BTW, a question which isn't belong to this patch:) There're also some failures from those xfstests overlay cases which run unionmount-testsuite (can't rememember all, maybe o/102~109, o/144~117). The error (diff) output are similar as [1]. Is there a fix for that too? Or I missed the fix?
I do not get these errors.
I suppose you are using the latest unionmount-testsuite, although it hasn't been changed for a while anyway.
103.out.bad 2025-06-07 05:16:59.445716549 -0400 line 3
Quoted 11 lines
Thanks,
Zorro[1]
103.out.bad 2025-06-07 05:16:59.445716549 -0400 renamed from dev/fd/63 2025-06-07 05:16:59.489929312 -0400 +2
-0@@ -1,2 +1,17 @@ QA output created by 103 +mount: /mnt/fstests/SCRATCH_DIR/union/m: wrong fs type, bad option, bad superblock on overlay, missing codepage or helper program, or other error. + dmesg(1) may have more information after failed mount system call.
Reason for failure to mount is likely to be found in dmesg. Please try to see if it is there.
103.out.bad 2025-06-07 05:16:59.445716549 -0400 line 15
Quoted 12 lines
103.out.bad 2025-06-07 05:16:59.445716549 -0400 renamed from dev/fd/63 2025-06-07 05:16:59.489929312 -0400 +12
-0+Traceback (most recent call last): + File "/mnt/tests/gitlab.cee.redhat.com/kernel-qe/kernel/-/archive/master/kernel-master.tar.bz2/filesystems/xfstests/unionmount-testsuite/./run", line 362, in <module> + func(ctx) + File "/mnt/tests/gitlab.cee.redhat.com/kernel-qe/kernel/-/archive/master/kernel-master.tar.bz2/filesystems/xfstests/unionmount-testsuite/tests/rename-file.py", line 71, in subtest_5 + ctx.rename(f, f2) + File "/mnt/tests/gitlab.cee.redhat.com/kernel-qe/kernel/-/archive/master/kernel-master.tar.bz2/filesystems/xfstests/unionmount-testsuite/context.py", line 1254, in rename + remount_union(self, rotate_upper=True) + File "/mnt/tests/gitlab.cee.redhat.com/kernel-qe/kernel/-/archive/master/kernel-master.tar.bz2/filesystems/xfstests/unionmount-testsuite/remount_union.py", line 35, in remount_union + system(cmd) + File "/mnt/tests/gitlab.cee.redhat.com/kernel-qe/kernel/-/archive/master/kernel-master.tar.bz2/filesystems/xfstests/unionmount-testsuite/tool_box.py", line 25, in system + raise RuntimeError("Command failed: " + command) +RuntimeError: Command failed: mount -t overlay overlay /mnt/fstests/SCRATCH_DIR/union/m -orw,xino=on -olowerdir=/mnt/fstests/SCRATCH_DIR/union/6/u:/mnt/fstests/SCRATCH_DIR/union/5/u:/mnt/fstests/SCRATCH_DIR/union/4/u:/mnt/fstests/SCRATCH_DIR/union/3/u:/mnt/fstests/SCRATCH_DIR/union/2/u:/mnt/fstests/SCRATCH_DIR/union/1/u:/mnt/fstests/SCRATCH_DIR/union/0/u:/mnt/fstests/SCRATCH_DIR/union/l,upperdir=/mnt/fstests/SCRATCH_DIR/union/7/u,workdir=/mnt/fstests/SCRATCH_DIR/union/7/w
Nothing looks obviously wrong in the mount command above. Was this a regression for you?
with kernel upgrade? libmount upgrade?
Thanks,
Amir.
Re: [PATCH v2 4/6] generic/623: do not run with overlayfs
Quoted 17 lines
Quoted 2 lines
Actually I'm wondering if we should help xfstests to support
BASE_FSTYPand FSTYP for more upper layer fs, likes nfs, cifs, and so on.I think that could be very useful, but will require cifs/nfs to implement more complicated _mount/umount/remount helpers like overlay.
Quoted 5 lines
If so, overlay will not be
the only one fs who usesBASE_FSTYPandBASE_SCRATCH_DEVthings, then we need to differentiate if a feature (e.g. shutdown) is needed by upper layer fs or underlying fs in a case.First of all, terminology. Many get this wrong.
In overlayfs, the "upper" and "lower" refer to the different underlying layers. In fstestsBASE_SCRATCH_DEVis always the same for bothOVL_UPPERandOVL_LOWERlayers.
Quoted 6 lines
Referring to the
BASE_SCRATCH_DEVas "underlying" or "base" fs is unambiguous in all cases of overlay/nfs/cifs.I do not have a good terminology to offer for referring to the "fs under test" be it overlay/cifs/nfs. You are welcome to offer your suggestions.
Sorry, I forgot to answer the question.
So far, with overlayfs there was no need for anything other than _require_scratch_shutdown and _scratch_shutdown and _scratch_remount for generic tests that do shutdown and remount to test consistency after a crash.
overlay tests that are aware of the underlying layers and perform operations explicitly on underlying layers are not generic tests they are overlay/* tests, so the generic helpers are not for them.
Bottom line is that I do not think that _require_scratch_shutdown should refer to fs under test or base fs, until I see a generic test case that suggests otherwise.
What we could do is a re-factoring:
_require_shutdown()
{
local dev=$1
local mnt=$2
...
}
_require_scratch_shutdown()
{
_require_shutdown $SCRATCH_DEV $SCRATCH_MNT
}
Then overlay,cifs,nfs/* tests can call _require_shutdown on base fs if they want to, but first, let's see a test case that needs it.
Thanks,
Amir.
Re: [PATCH v2 4/6] generic/623: do not run with overlayfs
Quoted 48 lines · Amir Goldstein
Quoted 17 lines
Quoted 2 lines
Actually I'm wondering if we should help xfstests to support
BASE_FSTYPand FSTYP for more upper layer fs, likes nfs, cifs, and so on.I think that could be very useful, but will require cifs/nfs to implement more complicated _mount/umount/remount helpers like overlay.
Quoted 5 lines
If so, overlay will not be
the only one fs who usesBASE_FSTYPandBASE_SCRATCH_DEVthings, then we need to differentiate if a feature (e.g. shutdown) is needed by upper layer fs or underlying fs in a case.First of all, terminology. Many get this wrong.
In overlayfs, the "upper" and "lower" refer to the different underlying layers. In fstestsBASE_SCRATCH_DEVis always the same for bothOVL_UPPERandOVL_LOWERlayers.Quoted 6 lines
Referring to the
BASE_SCRATCH_DEVas "underlying" or "base" fs is unambiguous in all cases of overlay/nfs/cifs.I do not have a good terminology to offer for referring to the "fs under test" be it overlay/cifs/nfs. You are welcome to offer your suggestions.
Sorry, I forgot to answer the question.
So far, with overlayfs there was no need for anything other than
_require_scratch_shutdownand_scratch_shutdownand_scratch_remountfor generic tests that do shutdown and remount to test consistency after a crash.overlay tests that are aware of the underlying layers and perform operations explicitly on underlying layers are not generic tests they are overlay/* tests, so the generic helpers are not for them.
Bottom line is that I do not think that
_require_scratch_shutdownshould refer to fs under test or base fs, until I see a generic test case that suggests otherwise.What we could do is a re-factoring:
_require_shutdown()
{
local dev=$1
local mnt=$2
...
}
Quoted 7 lines
_require_scratch_shutdown()
{
_require_shutdown$SCRATCH_DEV $SCRATCH_MNT
}Then overlay,cifs,nfs/* tests can call
_require_shutdownon base fs if they want to, but first, let's see a test case that needs it.
Actually I thought about the _require_shutdown when I reviewed this patch. Something likes:
g/623: _require_shutdown $SCRATCH_MNT ...
o/087: _require_shutdown $BASE_SCRATCH_MNT ...
But I dropped this idea for two reasons:
1) _require_shutdown needs to do real "shutdown" on a directory, a flexible argument to be "shutdown" is dangerous. If someone would like to check "shutdown" is supported on a $mnt, but if some exceptions cause the $mnt isn't mounted the expected fs, then the "/" might be down.
2) shutdown checking need to shutdown the $mnt and remount it. It's not only about $mnt and $dev, but also about mount options. _require_scratch_shutdown can use _scratch_mount, but how to deal with a flexible $mnt?
The 1) problem might can be fixed by checking the $mnt is a mountpoint, or return. The 2) problem is difficult to me, except we let _require_shutdown accept the mount options list as the 3rd argument. I think that looks ugly for a _require_ function :)
If you have good idea, it can replace this patch 4/6 :)
Thanks,
Zorro
Quoted 4 lines
Thanks,
Amir.
Re: [PATCH v2 4/6] generic/623: do not run with overlayfs
Quoted 73 lines · Zorro Lang
Quoted 48 lines · Amir Goldstein
Quoted 17 lines
Quoted 2 lines
Actually I'm wondering if we should help xfstests to support
BASE_FSTYPand FSTYP for more upper layer fs, likes nfs, cifs, and so on.I think that could be very useful, but will require cifs/nfs to implement more complicated _mount/umount/remount helpers like overlay.
Quoted 5 lines
If so, overlay will not be
the only one fs who usesBASE_FSTYPandBASE_SCRATCH_DEVthings, then we need to differentiate if a feature (e.g. shutdown) is needed by upper layer fs or underlying fs in a case.First of all, terminology. Many get this wrong.
In overlayfs, the "upper" and "lower" refer to the different underlying layers. In fstestsBASE_SCRATCH_DEVis always the same for bothOVL_UPPERandOVL_LOWERlayers.Quoted 6 lines
Referring to the
BASE_SCRATCH_DEVas "underlying" or "base" fs is unambiguous in all cases of overlay/nfs/cifs.I do not have a good terminology to offer for referring to the "fs under test" be it overlay/cifs/nfs. You are welcome to offer your suggestions.
Sorry, I forgot to answer the question.
So far, with overlayfs there was no need for anything other than
_require_scratch_shutdownand_scratch_shutdownand_scratch_remountfor generic tests that do shutdown and remount to test consistency after a crash.overlay tests that are aware of the underlying layers and perform operations explicitly on underlying layers are not generic tests they are overlay/* tests, so the generic helpers are not for them.
Bottom line is that I do not think that
_require_scratch_shutdownshould refer to fs under test or base fs, until I see a generic test case that suggests otherwise.What we could do is a re-factoring:
_require_shutdown()
{
local dev=$1
local mnt=$2
...
}Quoted 7 lines
_require_scratch_shutdown()
{
_require_shutdown$SCRATCH_DEV $SCRATCH_MNT
}Then overlay,cifs,nfs/* tests can call
_require_shutdownon base fs if they want to, but first, let's see a test case that needs it.Actually I thought about the
_require_shutdownwhen I reviewed this patch. Something likes:g/623:
_require_shutdown$SCRATCH_MNT ...
o/087:_require_shutdown$BASE_SCRATCH_MNT ...But I dropped this idea for two reasons:
1)_require_shutdownneeds to do real "shutdown" on a directory, a flexible argument to be "shutdown" is dangerous. If someone would like to check "shutdown" is supported on a $mnt, but if some exceptions cause the $mnt isn't mounted the expected fs, then the "/" might be down.
2) shutdown checking need to shutdown the $mnt and remount it. It's not only about $mnt and $dev, but also about mount options._require_scratch_shutdowncan use_scratch_mount, but how to deal with a flexible $mnt?
Quoted 4 lines
The 1) problem might can be fixed by checking the $mnt is a mountpoint, or return. The 2) problem is difficult to me, except we let
_require_shutdownaccept the mount options list as the 3rd argument. I think that looks ugly for a _require_ function :)
I don't like it either
Quoted 2 lines
If you have good idea, it can replace this patch 4/6 :)
Already sent a replacement patch per our earlier discussion
https://lore.kernel.org/fstests/[email protected]/
Thanks,
Amir.
Re: [PATCH v2 4/6] generic/623: do not run with overlayfs
In reply to Amir GoldsteinQuoted 236 lines · Amir Goldstein
Quoted 176 lines · Zorro Lang
Quoted 174 lines · Amir Goldstein
Quoted 149 lines · Zorro Lang
Quoted 84 lines · Amir Goldstein
Quoted 59 lines · Zorro Lang
Quoted 52 lines · Amir Goldstein
Quoted 31 lines · Zorro Lang
Quoted 29 lines · Amir Goldstein
This test performs shutdown via
xfs_io-c shutdown.Overlayfs tests can use
_scratch_shutdown, but they cannot use "-c shutdown"xfs_iocommand without jumping through hoops, so by default we do not support it.Add this condition to
_require_xfs_io_commandand add the require statement to test generic/623 so it wont run with overlayfs.Reported-by André Almeida <[email protected]>Tested-by André Almeida <[email protected]>Signed-off-by Amir Goldstein <[email protected]>--- common/rc | 8 ++++++++ tests/generic/623 | 1 + 2 files changed, 9 insertions(+)common/rc +3
-0@@ -3033,6 +3033,14 @@ _require_xfs_io_command() touch $testfile testio=`$XFS_IO_PROG -c "syncfs" $testfile 2>&1` ;; + "shutdown") + if [ $FSTYP = "overlay" ]; then + # Overlayfs tests can use _scratch_shutdown, but theyQuoted 8 lines
common/rc _require_xfs_io_command() · line 3043
Quoted 5 lines
common/rc +5
-0+ # cannot use "-c shutdown" xfs_io command without jumping + # through hoops, so by default we do not support it. + _notrun "xfs_io $command not supported on $FSTYP" + fi + ;;Hmm... I'm not sure this's a good way.
For example, overlay/087 doesxfs_ioshutdown too,Yes it does but look at the effort needed to do that properly:
$XFS_IO_PROG -x -c "open $(_scratch_shutdown_handle)" -c 'shutdown -f
' -c close -c syncfs $SCRATCH_MNT | \ grep -vF '[00'Quoted 4 lines
generally it should calls
_require_xfs_io_command"shutdown" although it doesn't. If someone overlay test case hope to test as o/087 does, and it calls_require_xfs_io_command"shutdown", then it'll be_notrun.Quoted 4 lines
If someone knows enough to perform the dance above with
_scratch_shutdown_handlethen that someone should know enough not to call_require_xfs_io_command"shutdown".
OTOH, if someone doesn't know then default is to not run.Quoted 9 lines
Sure, I can understand that, just this logic is a bit *obscure* :) It sounds like:
"If an overlay test case wants to doxfs_ioshutdown, it shouldn't call_require_xfs_io_command"shutdown". Or call that to skip a shutdown test on overlay :)"And the expected result of
_require_xfs_io_command"shutdown" will be totally opposite with_require_scratch_shutdownon overlay, that might be confused. Can we have a clearer way to deal with that?I don't really understand the confusion.
_require_xfs_io_command"shutdown"Like any other
_requirestatement
requires support for what this test does -
meaning that a test doesxfs_io-c shutdown, just like test generic/623 doesand
_require_scratch_shutdownimplies that the test does_scratch_shutdownFSTYP overlay happens to be able to do
_scratch_shutdownbut not able to doxfs_io-c shutdown $SCRATCH_MNTQuoted 8 lines
The different
_requirestatements simply reflect reality as it is.We can solve the confused about o/087 not having
_require_xfs_io_command"shutdown"
by moving the special hand craftedxfs_iocommand in o/087 to a helper_scratch_shutdown_and_syncfsto hide those internal implementation details from test writers.
See attached patch.Hmm... give me a moment to order my thoughts step by step :)
There're only 2 cases tend to do
xfs_ioshutdown on overlay currently (others are xfs specific test cases):$ grep -rsn shutdown tests/|grep -- "-c" tests/generic/623:29:$XFS_IO_PROG -x -c "mmap 0 4k" -c "mwrite 0 4k" -c shutdown -c fsync \ tests/overlay/087:50:$XFS_IO_PROG -x -c "open $(_scratch_shutdown_handle)" -c 'shutdown -f ' -c close -c syncfs $SCRATCH_MNT | \ tests/overlay/087:57:$XFS_IO_PROG -x -c "open $(_scratch_shutdown_handle)" -c 'shutdown -f ' -c close -c syncfs $SCRATCH_MNT | \ ...others shutdown cases nearly all use *_scratch_shutdown* with *_require_scratch_shutdown*, these two functions are consistent in code logic. And no one calls "
_require_xfs_io_commandshutdown" currently.So g/623 and o/087 are specifal, actually they call
_require_scratch_shutdowntoo, that makes sense for o/087. Now only g/623 doesn't make sense. Now we need to help it to make sense.I think the key is in
_require_scratch_shutdownfunction [1], how about add an argument to clearly tell it we need to check shutdown "only on the top layer $SCRATCH_MNT" or "try the lowest layer $BASE_SCRATCH_MNT if there is".For example:
common/rc +4
-2@@ -4075,15 +4075,17 @@ _require_exportfs() _require_open_by_handle } -# Does shutdown work on this fs? +# Does shutdown work on this [lower|top] layer fs? _require_scratch_shutdown() { + local layer="${1:-lower}" + [ -x $here/src/godown ] || _notrun "src/godown executable not found" _scratch_mkfs > /dev/null 2>&1 || _notrun "_scratch_mkfs failed on $SCRATCH_DEV" _scratch_mount - if [ $FSTYP = "overlay" ]; then + if [ $FSTYP = "overlay" -a "$level" = "lower" ]; then if [ -z $OVL_BASE_SCRATCH_DEV ]; then # In lagacy overlay usage, it may specify directory as # SCRATCH_DEV, in this case OVL_BASE_SCRATCH_DEVtests/generic/623 +0
-0tests/generic/623 _begin_fstest auto quick shutdown mmap · line 18
Quoted 5 lines
tests/generic/623 +1
-1@@ -15,7 +15,7 @@ _begin_fstest auto quick shutdown mmap "xfs: restore shutdown check in mapped write fault path" _require_scratch_nocheck -_require_scratch_shutdown +_require_scratch_shutdown topSorry I find this utterly confusing.
Think of all the 95% of fstests developers that do not care about overlayfs what does this top mean to them and why should they use it for tests that do
xfs_io-c shutdown and not for tests that do_scratch_shutdown?The test author and reviewers should be able to look at the tests and easily derive what the test requirements should be according to simple rules. For example:
1. A test that calls
_scrash_shutdownneeds to_require_scratch_shutdown
2. A test that calls_scratch_shutdown_and_syncfsneeds to_require_scratch_shutdown_and_syncfs
3. A test that callsxfs_io-c shutdown needs to_require_xfs_io_shutdownI completely understand why you do not like my hack of
_require_xfs_io_command"shutdown"Quoted 8 lines
Quoted 4 lines
Would you approve if it was an explicit
_require_xfs_io_shutdownhelper?# Requirements for tests that call
xfs_io-c shutdown instead of using the #_scratch_shutdownhelperOK, but you might metion that it's better not be used if
_scratch_shutdown_handleis called forxfs_io, as we hope the lower layer fs supports shutdown at that time:)Sure I'll add that
Quoted 2 lines
Actually I'm wondering if we should help xfstests to support
BASE_FSTYPand FSTYP for more upper layer fs, likes nfs, cifs, and so on.I think that could be very useful, but will require cifs/nfs to implement more complicated _mount/umount/remount helpers like overlay.
Quoted 5 lines
If so, overlay will not be
the only one fs who usesBASE_FSTYPandBASE_SCRATCH_DEVthings, then we need to differentiate if a feature (e.g. shutdown) is needed by upper layer fs or underlying fs in a case.First of all, terminology. Many get this wrong.
In overlayfs, the "upper" and "lower" refer to the different underlying layers. In fstestsBASE_SCRATCH_DEVis always the same for bothOVL_UPPERandOVL_LOWERlayers.Referring to the
BASE_SCRATCH_DEVas "underlying" or "base" fs is unambiguous in all cases of overlay/nfs/cifs.I do not have a good terminology to offer for referring to the "fs under test" be it overlay/cifs/nfs. You are welcome to offer your suggestions.
Quoted 5 lines
...
BTW, a question which isn't belong to this patch:) There're also some failures from those xfstests overlay cases which run unionmount-testsuite (can't rememember all, maybe o/102~109, o/144~117). The error (diff) output are similar as [1]. Is there a fix for that too? Or I missed the fix?I do not get these errors.
I suppose you are using the latest unionmount-testsuite, although it hasn't been changed for a while anyway.103.out.bad 2025-06-07 05:16:59.445716549 -0400 line 3
Quoted 11 lines
Thanks,
Zorro[1]
103.out.bad 2025-06-07 05:16:59.445716549 -0400 renamed from dev/fd/63 2025-06-07 05:16:59.489929312 -0400 +2
-0@@ -1,2 +1,17 @@ QA output created by 103 +mount: /mnt/fstests/SCRATCH_DIR/union/m: wrong fs type, bad option, bad superblock on overlay, missing codepage or helper program, or other error. + dmesg(1) may have more information after failed mount system call.Reason for failure to mount is likely to be found in dmesg. Please try to see if it is there.
Quoted 16 lines
103.out.bad 2025-06-07 05:16:59.445716549 -0400 line 15
Quoted 12 lines
103.out.bad 2025-06-07 05:16:59.445716549 -0400 renamed from dev/fd/63 2025-06-07 05:16:59.489929312 -0400 +12
-0+Traceback (most recent call last): + File "/mnt/tests/gitlab.cee.redhat.com/kernel-qe/kernel/-/archive/master/kernel-master.tar.bz2/filesystems/xfstests/unionmount-testsuite/./run", line 362, in <module> + func(ctx) + File "/mnt/tests/gitlab.cee.redhat.com/kernel-qe/kernel/-/archive/master/kernel-master.tar.bz2/filesystems/xfstests/unionmount-testsuite/tests/rename-file.py", line 71, in subtest_5 + ctx.rename(f, f2) + File "/mnt/tests/gitlab.cee.redhat.com/kernel-qe/kernel/-/archive/master/kernel-master.tar.bz2/filesystems/xfstests/unionmount-testsuite/context.py", line 1254, in rename + remount_union(self, rotate_upper=True) + File "/mnt/tests/gitlab.cee.redhat.com/kernel-qe/kernel/-/archive/master/kernel-master.tar.bz2/filesystems/xfstests/unionmount-testsuite/remount_union.py", line 35, in remount_union + system(cmd) + File "/mnt/tests/gitlab.cee.redhat.com/kernel-qe/kernel/-/archive/master/kernel-master.tar.bz2/filesystems/xfstests/unionmount-testsuite/tool_box.py", line 25, in system + raise RuntimeError("Command failed: " + command) +RuntimeError: Command failed: mount -t overlay overlay /mnt/fstests/SCRATCH_DIR/union/m -orw,xino=on -olowerdir=/mnt/fstests/SCRATCH_DIR/union/6/u:/mnt/fstests/SCRATCH_DIR/union/5/u:/mnt/fstests/SCRATCH_DIR/union/4/u:/mnt/fstests/SCRATCH_DIR/union/3/u:/mnt/fstests/SCRATCH_DIR/union/2/u:/mnt/fstests/SCRATCH_DIR/union/1/u:/mnt/fstests/SCRATCH_DIR/union/0/u:/mnt/fstests/SCRATCH_DIR/union/l,upperdir=/mnt/fstests/SCRATCH_DIR/union/7/u,workdir=/mnt/fstests/SCRATCH_DIR/union/7/wNothing looks obviously wrong in the mount command above. Was this a regression for you?
with kernel upgrade? libmount upgrade?
Thanks Amir, I'll check more and send another email to talk about this failure :)
Quoted 4 lines
Thanks,
Amir.
PATCH v2 5/6 generic: remove incorrect _require_idmapped_mounts checks
In reply to Amir Goldsteincommit f5661920 ("generic: add missed _require_idmapped_mounts check") wrongly adds _require_idmapped_mounts to tests that do not require idmapped mounts support.
The added _require_idmapped_mounts in test generic/633 goes against commit d8dee122 ("idmapped-mounts: always run generic vfs tests") that intentionally removed this requirement from the generic tests.
The added _require_idmapped_mounts in tests generic/69{6,7} causes those tests not to run with overlayfs, which does not support idmapped mounts. However, those tests are regression tests to kernel commit 1639a49ccdce ("fs: move S_ISGID stripping into the vfs_*() helpers") which is documented as also solving a correction issue with overlayfs, so removing this test converage is very much undesired.
Remove the incorrectly added _require_idmapped_mounts checks. Also fix the log in _require_idmapped_mounts to say that "idmapped mounts not support by $FSTYP", which is what the helper checks instead of "vfstests not support by $FSTYP" which is incorrect.
--- common/rc | 2 +- tests/generic/633 | 1 - tests/generic/696 | 1 - tests/generic/697 | 1 - 4 files changed, 1 insertion(+), 4 deletions(-)
common/rc +1 -1
@@ -2639,7 +2639,7 @@ _require_idmapped_mounts() --fstype "$FSTYP" if [ $? -ne 0 ]; then - _notrun "vfstest not support by $FSTYP" + _notrun "idmapped mounts not support by $FSTYP" fi }
tests/generic/633 +0 -1
@@ -12,7 +12,6 @@ _begin_fstest auto quick atime attr cap idmapped io_uring mount perms rw unlink # Import common functions. . ./common/filter -_require_idmapped_mounts _require_test echo "Silence is golden"
tests/generic/696 +0 -1
@@ -17,7 +17,6 @@ _begin_fstest auto quick cap idmapped mount perms rw unlink # Import common functions. . ./common/filter -_require_idmapped_mounts _require_test _require_scratch _fixed_by_kernel_commit ac6800e279a2 \
tests/generic/697 +0 -1
@@ -17,7 +17,6 @@ _begin_fstest auto quick cap acl idmapped mount perms rw unlink . ./common/filter . ./common/attr -_require_idmapped_mounts _require_test _require_acls _fixed_by_kernel_commit 1639a49ccdce \
Signature · 1 line
2.34.1
Re: [PATCH v2 5/6] generic: remove incorrect _require_idmapped_mounts checks
Quoted 20 lines · Amir Goldstein
commit
f5661920("generic: add missed_require_idmapped_mountscheck") wrongly adds_require_idmapped_mountsto tests that do not require idmapped mounts support.The added
_require_idmapped_mountsin test generic/633 goes against commitd8dee122("idmapped-mounts: always run generic vfs tests") that intentionally removed this requirement from the generic tests.The added
_require_idmapped_mountsin tests generic/69{6,7} causes those tests not to run with overlayfs, which does not support idmapped mounts. However, those tests are regression tests to kernel commit1639a49ccdce("fs: moveS_ISGIDstripping into the vfs_*() helpers") which is documented as also solving a correction issue with overlayfs, so removing this test converage is very much undesired.Remove the incorrectly added
_require_idmapped_mountschecks. Also fix the log in_require_idmapped_mountsto say that "idmapped mounts not support by $FSTYP", which is what the helper checks instead of "vfstests not support by $FSTYP" which is incorrect.
Quoted 6 lines
Cc Yang Xu <[email protected]>Cc Anthony Iliopoulos <[email protected]>Cc David Disseldorp <[email protected]>Fixes commit f5661920 ("generic: add missed _require_idmapped_mounts check")Signed-off-by Amir Goldstein <[email protected]>---
Is this one changed anything from this ?
https://lore.kernel.org/fstests/[email protected]/
Due to above link has been reviewed by Christian Brauner, do you want to add his RVB to this version?
Thanks,
Zorro
Quoted 58 lines
common/rc | 2 +- tests/generic/633 | 1 - tests/generic/696 | 1 - tests/generic/697 | 1 - 4 files changed, 1 insertion(+), 4 deletions(-)common/rc +1
-1@@ -2639,7 +2639,7 @@ _require_idmapped_mounts() --fstype "$FSTYP" if [ $? -ne 0 ]; then - _notrun "vfstest not support by $FSTYP" + _notrun "idmapped mounts not support by $FSTYP" fi }tests/generic/633 +0
-1@@ -12,7 +12,6 @@ _begin_fstest auto quick atime attr cap idmapped io_uring mount perms rw unlink # Import common functions. . ./common/filter -_require_idmapped_mounts _require_test echo "Silence is golden"tests/generic/696 +0
-1@@ -17,7 +17,6 @@ _begin_fstest auto quick cap idmapped mount perms rw unlink # Import common functions. . ./common/filter -_require_idmapped_mounts _require_test _require_scratch _fixed_by_kernel_commit ac6800e279a2 \tests/generic/697 +0
-1@@ -17,7 +17,6 @@ _begin_fstest auto quick cap acl idmapped mount perms rw unlink . ./common/filter . ./common/attr -_require_idmapped_mounts _require_test _require_acls _fixed_by_kernel_commit 1639a49ccdce \Signature · 1 line
2.34.1
Re: [PATCH v2 5/6] generic: remove incorrect _require_idmapped_mounts checks
Quoted 29 lines · Zorro Lang
Quoted 20 lines · Amir Goldstein
commit
f5661920("generic: add missed_require_idmapped_mountscheck") wrongly adds_require_idmapped_mountsto tests that do not require idmapped mounts support.The added
_require_idmapped_mountsin test generic/633 goes against commitd8dee122("idmapped-mounts: always run generic vfs tests") that intentionally removed this requirement from the generic tests.The added
_require_idmapped_mountsin tests generic/69{6,7} causes those tests not to run with overlayfs, which does not support idmapped mounts. However, those tests are regression tests to kernel commit1639a49ccdce("fs: moveS_ISGIDstripping into the vfs_*() helpers") which is documented as also solving a correction issue with overlayfs, so removing this test converage is very much undesired.Remove the incorrectly added
_require_idmapped_mountschecks. Also fix the log in_require_idmapped_mountsto say that "idmapped mounts not support by $FSTYP", which is what the helper checks instead of "vfstests not support by $FSTYP" which is incorrect.Quoted 6 lines
Cc Yang Xu <[email protected]>Cc Anthony Iliopoulos <[email protected]>Cc David Disseldorp <[email protected]>Fixes commit f5661920 ("generic: add missed _require_idmapped_mounts check")Signed-off-by Amir Goldstein <[email protected]>---
Quoted 2 lines
Is this one changed anything from this ?
https://lore.kernel.org/fstests/[email protected]/
There is no change. I just forgot to add the RVB.
Quoted 4 lines
Due to above link has been reviewed by Christian Brauner, do you want to add his RVB to this version?
Yes please.
Thanks,
Amir.
PATCH v2 6/6 generic/699: fix failure with MOUNT_OPTIONS
In reply to Amir Goldsteingeneric/699 uses overalyfs helper _overlay_mount_dirs, which is meant to be used by overlayfs tests, where MOUNT_OPTIONS refer to overalyfs mount options.
Using this helper from a generic test when FSTYP is not overlay is causing undesired results. For example, when MOUNT_OPTIONS is defined and includes a mount option not supported by overalyfs (e.g. 'acl'), the test is notrun because of:
mount: /vdc/ovl-merge: fsconfig() failed: overlay: Unknown parameter 'acl'.
There is no other generic test that includes the common/overlay helpers and uses them, so remove this practice from generic/699 as well.
--- tests/generic/699 | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-)
tests/generic/699 +10 -7
@@ -8,7 +8,6 @@ # mounts specifically. # . ./common/preamble -. ./common/overlay _begin_fstest auto quick perms attr idmapped mount # Override the default cleanup function.
@@ -96,20 +95,24 @@ reset_ownership() stat -c '%u:%g' $path } +setup_overlayfs() +{ + mkdir -p $upper $work $merge + _mount -t overlay -o lowerdir=$lower,upperdir=$upper,workdir=$work \ + overlay $merge $* +} + # Prepare overlayfs with metacopy turned off. setup_overlayfs_idmapped_lower_metacopy_off() { - mkdir -p $upper $work $merge - _overlay_mount_dirs $lower $upper $work \ - overlay $merge -ometacopy=off || \ - _notrun "overlayfs doesn't support idmappped layers" + setup_overlayfs -ometacopy=off || \ + _notrun "overlayfs doesn't support idmappped layers" } # Prepare overlayfs with metacopy turned on. setup_overlayfs_idmapped_lower_metacopy_on() { - mkdir -p $upper $work $merge - _overlay_mount_dirs $lower $upper $work overlay $merge -ometacopy=on + setup_overlayfs -ometacopy=on } reset_overlayfs()
Signature · 1 line
2.34.1
Re: [PATCH v2 6/6] generic/699: fix failure with MOUNT_OPTIONS
Quoted 9 lines · Amir Goldstein
generic/699 uses overalyfs helper
_overlay_mount_dirs, which is meant to be used by overlayfs tests, whereMOUNT_OPTIONSrefer to overalyfs mount options.Using this helper from a generic test when FSTYP is not overlay is causing undesired results. For example, when
MOUNT_OPTIONSis defined and includes a mount option not supported by overalyfs (e.g. 'acl'), the test is notrun because of:
Quoted 7 lines
mount: /vdc/ovl-merge:
fsconfig()failed: overlay: Unknown parameter 'acl'.There is no other generic test that includes the common/overlay helpers and uses them, so remove this practice from generic/699 as well.
Signed-off-by Amir Goldstein <[email protected]>---
Good to me,
Quoted 49 lines
tests/generic/699 | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-)tests/generic/699 +10
-7@@ -8,7 +8,6 @@ # mounts specifically. # . ./common/preamble -. ./common/overlay _begin_fstest auto quick perms attr idmapped mount # Override the default cleanup function.@@ -96,20 +95,24 @@ reset_ownership() stat -c '%u:%g' $path } +setup_overlayfs() +{ + mkdir -p $upper $work $merge + _mount -t overlay -o lowerdir=$lower,upperdir=$upper,workdir=$work \ + overlay $merge $* +} + # Prepare overlayfs with metacopy turned off. setup_overlayfs_idmapped_lower_metacopy_off() { - mkdir -p $upper $work $merge - _overlay_mount_dirs $lower $upper $work \ - overlay $merge -ometacopy=off || \ - _notrun "overlayfs doesn't support idmappped layers" + setup_overlayfs -ometacopy=off || \ + _notrun "overlayfs doesn't support idmappped layers" } # Prepare overlayfs with metacopy turned on. setup_overlayfs_idmapped_lower_metacopy_on() { - mkdir -p $upper $work $merge - _overlay_mount_dirs $lower $upper $work overlay $merge -ometacopy=on + setup_overlayfs -ometacopy=on } reset_overlayfs()Signature · 1 line
2.34.1