ovl: skip stale entries in merge dir cache iteration
Amir Goldstein
maintainerPATCH 1/1 ovl: skip stale entries in merge dir cache iteration
On the first getdents call, ovl_iterate() populates the readdir cache with a list of entries, but for upper entries with origin lower inode, p->ino remains zero.
Following getdents calls traverse the readdir cache list and call ovl_cache_update_ino() for entries with zero p->ino to lookup the entry in the overlay and return d_ino that is consistent with st_ino.
If the upper file was unlinked between the first getdents call and the getdents call that lists the file entry, ovl_cache_update_ino() will not find the entry and fall back to setting d_ino to the upper real st_ino, which is inconsistent with how this object was presented to users.
Instead of listing a stale entry with inconsistent d_ino, simply skip the stale entry, which is better for users.
--- fs/overlayfs/readdir.c | 5 +++++ 1 file changed, 5 insertions(+)
fs/overlayfs/readdir.c +5 -0
@@ -481,6 +481,8 @@ static int ovl_cache_update_ino(struct path *path, struct ovl_cache_entry *p) } this =lookup_one_len (p->name, dir, p->len);if (IS_ERR_OR_NULL (this) || !this->d_inode) { +/* Mark a stale entry */ + p->is_whiteout = true;if (IS_ERR (this)) { err =PTR_ERR (this); this =NULL ;
@@ -776,6 +778,9 @@ static int ovl_iterate(struct file *file, struct dir_context *ctx)if (err) goto out; } + } +/* ovl_cache_update_ino() sets is_whiteout on stale entry */ +if (!p->is_whiteout) {if (!dir_emit (ctx, p->name, p->len, p->ino, p->type))break ; }
Signature · 1 line
2.25.1
Amir Goldstein
maintainerQuoted 9 lines · Amir Goldstein
On the first getdents call,
ovl_iterate()populates the readdir cache with a list of entries, but for upper entries with origin lower inode,p->inoremains zero.Following getdents calls traverse the readdir cache list and call
ovl_cache_update_ino()for entries with zerop->inoto lookup the entry in the overlay and returnd_inothat is consistent withst_ino.
Quoted 8 lines
If the upper file was unlinked between the first getdents call and the getdents call that lists the file entry,
ovl_cache_update_ino()will not find the entry and fall back to settingd_inoto the upper realst_ino, which is inconsistent with how this object was presented to users.Instead of listing a stale entry with inconsistent
d_ino, simply skip the stale entry, which is better for users.
Miklos,
I forgot to follow up on this patch.
Upstream xfstest overlay/077 is failing without this patch. Please add:
Thanks,
Amir.
Quoted 31 lines
Signed-off-by Amir Goldstein <[email protected]>--- fs/overlayfs/readdir.c | 5 +++++ 1 file changed, 5 insertions(+)fs/overlayfs/readdir.c +5
-1@@ -481,6 +481,8 @@ static int ovl_cache_update_ino(struct path *path, struct ovl_cache_entry *p) } this =lookup_one_len (p->name, dir, p->len);if (IS_ERR_OR_NULL (this) || !this->d_inode) { +/* Mark a stale entry */ + p->is_whiteout = true;if (IS_ERR (this)) { err =PTR_ERR (this); this =NULL ;@@ -776,6 +778,9 @@ static int ovl_iterate(struct file *file, struct dir_context *ctx)if (err) goto out; } + } +/* ovl_cache_update_ino() sets is_whiteout on stale entry */ +if (!p->is_whiteout) {if (!dir_emit (ctx, p->name, p->len, p->ino, p->type))break ; } --2.25.1
Miklos Szeredi
maintainerQuoted 16 lines · Amir Goldstein
Quoted 14 lines · Amir Goldstein
On the first getdents call,
ovl_iterate()populates the readdir cache with a list of entries, but for upper entries with origin lower inode,p->inoremains zero.Following getdents calls traverse the readdir cache list and call
ovl_cache_update_ino()for entries with zerop->inoto lookup the entry in the overlay and returnd_inothat is consistent withst_ino.If the upper file was unlinked between the first getdents call and the getdents call that lists the file entry,
ovl_cache_update_ino()will not find the entry and fall back to settingd_inoto the upper realst_ino, which is inconsistent with how this object was presented to users.
Quoted 8 lines
Quoted 3 lines
Instead of listing a stale entry with inconsistent
d_ino, simply skip the stale entry, which is better for users.Miklos,
I forgot to follow up on this patch.
Upstream xfstest overlay/077 is failing without this patch.
Can't reproduce (on ext4/xfs and "-oxino=on").
Is there some trick?
Thanks,
Miklos
Amir Goldstein
maintainerQuoted 22 lines · Miklos Szeredi
Quoted 20 lines · Amir Goldstein
Quoted 17 lines · Amir Goldstein
On the first getdents call,
ovl_iterate()populates the readdir cache with a list of entries, but for upper entries with origin lower inode,p->inoremains zero.Following getdents calls traverse the readdir cache list and call
ovl_cache_update_ino()for entries with zerop->inoto lookup the entry in the overlay and returnd_inothat is consistent withst_ino.If the upper file was unlinked between the first getdents call and the getdents call that lists the file entry,
ovl_cache_update_ino()will not find the entry and fall back to settingd_inoto the upper realst_ino, which is inconsistent with how this object was presented to users.Instead of listing a stale entry with inconsistent
d_ino, simply skip the stale entry, which is better for users.
Quoted 8 lines
Quoted 4 lines
Miklos,
I forgot to follow up on this patch.
Upstream xfstest overlay/077 is failing without this patch.Can't reproduce (on ext4/xfs and "-oxino=on").
Is there some trick?
Not sure. overlay/077 fails for me on v5.14.0-rc2 on ext4/xfs.
QA output created by 077
+entry m100 has inconsistent d_ino (234 != 232) +entry f100 has inconsistent d_ino (335 != 16777542)
Silence is golden
Maybe you need to build src/t_dir_offset2?
Thanks,
Amir.
Miklos Szeredi
maintainerQuoted 33 lines · Amir Goldstein
Quoted 22 lines · Miklos Szeredi
Quoted 20 lines · Amir Goldstein
Quoted 17 lines · Amir Goldstein
On the first getdents call,
ovl_iterate()populates the readdir cache with a list of entries, but for upper entries with origin lower inode,p->inoremains zero.Following getdents calls traverse the readdir cache list and call
ovl_cache_update_ino()for entries with zerop->inoto lookup the entry in the overlay and returnd_inothat is consistent withst_ino.If the upper file was unlinked between the first getdents call and the getdents call that lists the file entry,
ovl_cache_update_ino()will not find the entry and fall back to settingd_inoto the upper realst_ino, which is inconsistent with how this object was presented to users.Instead of listing a stale entry with inconsistent
d_ino, simply skip the stale entry, which is better for users.Quoted 8 lines
Quoted 4 lines
Miklos,
I forgot to follow up on this patch.
Upstream xfstest overlay/077 is failing without this patch.Can't reproduce (on ext4/xfs and "-oxino=on").
Is there some trick?
Quoted 8 lines
Not sure. overlay/077 fails for me on v5.14.0-rc2 on ext4/xfs.
QA output created by 077
+entry m100 has inconsistentd_ino(234 != 232) +entry f100 has inconsistentd_ino(335 != 16777542)
Silence is goldenMaybe you need to build src/t_dir_offset2?
root@kvm:/opt/xfstests-dev# git log -1 --pretty=%h 10f6b231 root@kvm:/opt/xfstests-dev# cat local.config export TEST_DEV=/dev/vdb1 export TEST_DIR=/test export SCRATCH_DEV=/dev/vdb2 export SCRATCH_MNT=/scratch export FSTYP=ext4 export OVERLAY_MOUNT_OPTIONS="-o xino=on" root@kvm:/opt/xfstests-dev# make src/t_dir_offset2 make: 'src/t_dir_offset2' is up to date. root@kvm:/opt/xfstests-dev# ./check -overlay overlay/077 FSTYP -- overlay PLATFORM -- Linux/x86_64 kvm 5.14.0-rc2 #276 SMP Tue Jul 20 05:54:44 CEST 2021 MKFS_OPTIONS -- /scratch MOUNT_OPTIONS -- -o xino=on /scratch /scratch/ovl-mnt
overlay/077 1s ... 1s
Ran: overlay/077
Passed all 1 tests
root@kvm:/opt/xfstests-dev# cat results/overlay/077.full
Create file in pure upper dir:
getdents at offset 0 returned 192 bytes
created entry p0
entry p0 found as expected
Remove file in pure upper dir:
getdents at offset 0 returned 192 bytes
unlinked entry p100
entry p100 not found as expected
Create file in impure upper dir:
getdents at offset 0 returned 192 bytes
created entry o0
entry o0 found as expected
Remove file in impure upper dir:
getdents at offset 0 returned 192 bytes
unlinked entry o100
entry o100 not found as expected
Create file in merge dir:
getdents at offset 0 returned 192 bytes
created entry m0
entry m0 found as expected
Remove file in merge dir:
getdents at offset 0 returned 192 bytes
unlinked entry m100
entry m100 not found as expected
Create file in former merge dir:
getdents at offset 0 returned 192 bytes
created entry f0
entry f0 found as expected
Remove file in former merge dir:
getdents at offset 0 returned 192 bytes
unlinked entry f100
entry f100 not found as expected
Ideas for further debugging why this test isn't failing for v4.12-rc2?
Thanks,
Miklos
Miklos Szeredi
maintainerQuoted 103 lines · Miklos Szeredi
Quoted 33 lines · Amir Goldstein
Quoted 22 lines · Miklos Szeredi
Quoted 20 lines · Amir Goldstein
Quoted 17 lines · Amir Goldstein
On the first getdents call,
ovl_iterate()populates the readdir cache with a list of entries, but for upper entries with origin lower inode,p->inoremains zero.Following getdents calls traverse the readdir cache list and call
ovl_cache_update_ino()for entries with zerop->inoto lookup the entry in the overlay and returnd_inothat is consistent withst_ino.If the upper file was unlinked between the first getdents call and the getdents call that lists the file entry,
ovl_cache_update_ino()will not find the entry and fall back to settingd_inoto the upper realst_ino, which is inconsistent with how this object was presented to users.Instead of listing a stale entry with inconsistent
d_ino, simply skip the stale entry, which is better for users.Quoted 8 lines
Quoted 4 lines
Miklos,
I forgot to follow up on this patch.
Upstream xfstest overlay/077 is failing without this patch.Can't reproduce (on ext4/xfs and "-oxino=on").
Is there some trick?
Quoted 8 lines
Not sure. overlay/077 fails for me on v5.14.0-rc2 on ext4/xfs.
QA output created by 077
+entry m100 has inconsistentd_ino(234 != 232) +entry f100 has inconsistentd_ino(335 != 16777542)
Silence is goldenMaybe you need to build src/t_dir_offset2?
root@kvm:/opt/xfstests-dev# git log -1 --pretty=%h 10f6b231 root@kvm:/opt/xfstests-dev# cat local.config export TEST_DEV=/dev/vdb1 export TEST_DIR=/test export SCRATCH_DEV=/dev/vdb2 export SCRATCH_MNT=/scratch export FSTYP=ext4 export OVERLAY_MOUNT_OPTIONS="-o xino=on" root@kvm:/opt/xfstests-dev# make src/t_dir_offset2 make: 'src/t_dir_offset2' is up to date. root@kvm:/opt/xfstests-dev# ./check -overlay overlay/077 FSTYP -- overlay PLATFORM -- Linux/x86_64 kvm 5.14.0-rc2 #276 SMP Tue Jul 20 05:54:44 CEST 2021 MKFS_OPTIONS -- /scratch MOUNT_OPTIONS -- -o xino=on /scratch /scratch/ovl-mntoverlay/077 1s ... 1s
Ran: overlay/077
Passed all 1 testsroot@kvm:/opt/xfstests-dev# cat results/overlay/077.fullCreate file in pure upper dir:
getdents at offset 0 returned 192 bytes
created entry p0
entry p0 found as expectedRemove file in pure upper dir:
getdents at offset 0 returned 192 bytes
unlinked entry p100
entry p100 not found as expectedCreate file in impure upper dir:
getdents at offset 0 returned 192 bytes
created entry o0
entry o0 found as expectedRemove file in impure upper dir:
getdents at offset 0 returned 192 bytes
unlinked entry o100
entry o100 not found as expectedCreate file in merge dir:
getdents at offset 0 returned 192 bytes
created entry m0
entry m0 found as expectedRemove file in merge dir:
getdents at offset 0 returned 192 bytes
unlinked entry m100
entry m100 not found as expectedCreate file in former merge dir:
getdents at offset 0 returned 192 bytes
created entry f0
entry f0 found as expected
Quoted 6 lines
Remove file in former merge dir:
getdents at offset 0 returned 192 bytes
unlinked entry f100
entry f100 not found as expectedIdeas for further debugging why this test isn't failing for v4.12-rc2?
v5.14-rc2, that is (got the -rc2 part right the first time, though).
Thanks,
Miklos
Amir Goldstein
maintainerQuoted 105 lines · Miklos Szeredi
Quoted 103 lines · Miklos Szeredi
Quoted 33 lines · Amir Goldstein
Quoted 22 lines · Miklos Szeredi
Quoted 20 lines · Amir Goldstein
Quoted 17 lines · Amir Goldstein
On the first getdents call,
ovl_iterate()populates the readdir cache with a list of entries, but for upper entries with origin lower inode,p->inoremains zero.Following getdents calls traverse the readdir cache list and call
ovl_cache_update_ino()for entries with zerop->inoto lookup the entry in the overlay and returnd_inothat is consistent withst_ino.If the upper file was unlinked between the first getdents call and the getdents call that lists the file entry,
ovl_cache_update_ino()will not find the entry and fall back to settingd_inoto the upper realst_ino, which is inconsistent with how this object was presented to users.Instead of listing a stale entry with inconsistent
d_ino, simply skip the stale entry, which is better for users.Quoted 8 lines
Quoted 4 lines
Miklos,
I forgot to follow up on this patch.
Upstream xfstest overlay/077 is failing without this patch.Can't reproduce (on ext4/xfs and "-oxino=on").
Is there some trick?
Quoted 8 lines
Not sure. overlay/077 fails for me on v5.14.0-rc2 on ext4/xfs.
QA output created by 077
+entry m100 has inconsistentd_ino(234 != 232) +entry f100 has inconsistentd_ino(335 != 16777542)
Silence is goldenMaybe you need to build src/t_dir_offset2?
root@kvm:/opt/xfstests-dev# git log -1 --pretty=%h 10f6b231 root@kvm:/opt/xfstests-dev# cat local.config export TEST_DEV=/dev/vdb1 export TEST_DIR=/test export SCRATCH_DEV=/dev/vdb2 export SCRATCH_MNT=/scratch export FSTYP=ext4 export OVERLAY_MOUNT_OPTIONS="-o xino=on" root@kvm:/opt/xfstests-dev# make src/t_dir_offset2 make: 'src/t_dir_offset2' is up to date. root@kvm:/opt/xfstests-dev# ./check -overlay overlay/077 FSTYP -- overlay PLATFORM -- Linux/x86_64 kvm 5.14.0-rc2 #276 SMP Tue Jul 20 05:54:44 CEST 2021 MKFS_OPTIONS -- /scratch MOUNT_OPTIONS -- -o xino=on /scratch /scratch/ovl-mntoverlay/077 1s ... 1s
Ran: overlay/077
Passed all 1 testsroot@kvm:/opt/xfstests-dev# cat results/overlay/077.fullCreate file in pure upper dir:
getdents at offset 0 returned 192 bytes
created entry p0
entry p0 found as expectedRemove file in pure upper dir:
getdents at offset 0 returned 192 bytes
unlinked entry p100
entry p100 not found as expectedCreate file in impure upper dir:
getdents at offset 0 returned 192 bytes
created entry o0
entry o0 found as expectedRemove file in impure upper dir:
getdents at offset 0 returned 192 bytes
unlinked entry o100
entry o100 not found as expectedCreate file in merge dir:
getdents at offset 0 returned 192 bytes
created entry m0
entry m0 found as expectedRemove file in merge dir:
getdents at offset 0 returned 192 bytes
unlinked entry m100
entry m100 not found as expectedCreate file in former merge dir:
getdents at offset 0 returned 192 bytes
created entry f0
entry f0 found as expected
Quoted 7 lines
Quoted 6 lines
Remove file in former merge dir:
getdents at offset 0 returned 192 bytes
unlinked entry f100
entry f100 not found as expectedIdeas for further debugging why this test isn't failing for v4.12-rc2?
It's not you, it's me ;-)
The failure was lost during cleanup of t_dir_offset2 patches for submission and it is I who was running an older version of t_dir_offset2. Let me figure this out and get back to you with a working test.
Thanks,
Amir.
Amir Goldstein
maintainerQuoted 115 lines · Amir Goldstein
Quoted 105 lines · Miklos Szeredi
Quoted 103 lines · Miklos Szeredi
Quoted 33 lines · Amir Goldstein
Quoted 22 lines · Miklos Szeredi
Quoted 20 lines · Amir Goldstein
Quoted 17 lines · Amir Goldstein
On the first getdents call,
ovl_iterate()populates the readdir cache with a list of entries, but for upper entries with origin lower inode,p->inoremains zero.Following getdents calls traverse the readdir cache list and call
ovl_cache_update_ino()for entries with zerop->inoto lookup the entry in the overlay and returnd_inothat is consistent withst_ino.If the upper file was unlinked between the first getdents call and the getdents call that lists the file entry,
ovl_cache_update_ino()will not find the entry and fall back to settingd_inoto the upper realst_ino, which is inconsistent with how this object was presented to users.Instead of listing a stale entry with inconsistent
d_ino, simply skip the stale entry, which is better for users.Quoted 8 lines
Quoted 4 lines
Miklos,
I forgot to follow up on this patch.
Upstream xfstest overlay/077 is failing without this patch.Can't reproduce (on ext4/xfs and "-oxino=on").
Is there some trick?
Quoted 8 lines
Not sure. overlay/077 fails for me on v5.14.0-rc2 on ext4/xfs.
QA output created by 077
+entry m100 has inconsistentd_ino(234 != 232) +entry f100 has inconsistentd_ino(335 != 16777542)
Silence is goldenMaybe you need to build src/t_dir_offset2?
root@kvm:/opt/xfstests-dev# git log -1 --pretty=%h 10f6b231 root@kvm:/opt/xfstests-dev# cat local.config export TEST_DEV=/dev/vdb1 export TEST_DIR=/test export SCRATCH_DEV=/dev/vdb2 export SCRATCH_MNT=/scratch export FSTYP=ext4 export OVERLAY_MOUNT_OPTIONS="-o xino=on" root@kvm:/opt/xfstests-dev# make src/t_dir_offset2 make: 'src/t_dir_offset2' is up to date. root@kvm:/opt/xfstests-dev# ./check -overlay overlay/077 FSTYP -- overlay PLATFORM -- Linux/x86_64 kvm 5.14.0-rc2 #276 SMP Tue Jul 20 05:54:44 CEST 2021 MKFS_OPTIONS -- /scratch MOUNT_OPTIONS -- -o xino=on /scratch /scratch/ovl-mntoverlay/077 1s ... 1s
Ran: overlay/077
Passed all 1 testsroot@kvm:/opt/xfstests-dev# cat results/overlay/077.fullCreate file in pure upper dir:
getdents at offset 0 returned 192 bytes
created entry p0
entry p0 found as expectedRemove file in pure upper dir:
getdents at offset 0 returned 192 bytes
unlinked entry p100
entry p100 not found as expectedCreate file in impure upper dir:
getdents at offset 0 returned 192 bytes
created entry o0
entry o0 found as expectedRemove file in impure upper dir:
getdents at offset 0 returned 192 bytes
unlinked entry o100
entry o100 not found as expectedCreate file in merge dir:
getdents at offset 0 returned 192 bytes
created entry m0
entry m0 found as expectedRemove file in merge dir:
getdents at offset 0 returned 192 bytes
unlinked entry m100
entry m100 not found as expectedCreate file in former merge dir:
getdents at offset 0 returned 192 bytes
created entry f0
entry f0 found as expectedQuoted 7 lines
Quoted 6 lines
Remove file in former merge dir:
getdents at offset 0 returned 192 bytes
unlinked entry f100
entry f100 not found as expectedIdeas for further debugging why this test isn't failing for v4.12-rc2?
Quoted 7 lines
It's not you, it's me ;-)
The failure was lost during cleanup of
t_dir_offset2patches for submission and it is I who was running an older version oft_dir_offset2. Let me figure this out and get back to you with a working test.
How about you try again with a version of the test that actually has the check and the error print that I reported.... ;-)
Thanks,
Amir.
From 250615da6f01e99c055b7fc3bd9a2c1deac644e0 Mon Sep 17 00:00:00 2001
From: Amir Goldstein <[email protected]>
Date: Tue, 20 Jul 2021 17:45:43 +0300
Subject: [PATCH] src/t_dir_offset2: Check for inconsistent d_ino/st_ino
After unlink of a directory entry, that entry may still apear in getdents results of an already open directory fd, but it should return a d_ino value that is consistent with the already observed st_ino of that entry.
This is a regression test for kernel commit .... ("ovl: skip stale entries in merge dir cache iteration")
--- src/t_dir_offset2.c | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-)
src/t_dir_offset2.c +12 -6
@@ -44,6 +44,7 @@ int main(int argc, char *argv[])char buf[BUF_SIZE ];int nread, bufsize =BUF_SIZE ;struct linux_dirent64 *d; +struct stat st = {};int bpos, total, i;off_t lret;int retval =EXIT_SUCCESS ;
@@ -81,9 +82,9 @@ int main(int argc, char *argv[]) }if (filename) { - exists = !faccessat (fd, filename,F_OK ,AT_SYMLINK_NOFOLLOW ); + exists = !fstatat (fd, filename, &st,AT_SYMLINK_NOFOLLOW );if (!exists && errno !=ENOENT ) { -perror ("faccessat" ); +perror ("fstatat" );exit (EXIT_FAILURE ); } }
@@ -139,9 +140,6 @@ int main(int argc, char *argv[])continue ; } -if (nread ==0 ) -break ; -for (bpos =0 ; bpos < nread; total++) { d = (struct linux_dirent64 *) (buf + bpos);
@@ -165,8 +163,16 @@ int main(int argc, char *argv[])printf ("entry #%d: %s (d_ino=%lld, d_off=%lld)\n" , i, d->d_name, (long long int )d->d_ino, (long long int )d->d_off); -if (!strcmp (filename, d->d_name)) +if (!strcmp (filename, d->d_name)) { found =1 ; +if (st.st_ino && d->d_ino != st.st_ino) { +fprintf (stderr,"entry %s has inconsistent d_ino (%lld != %lld)\n" , + filename, + (long long int )d->d_ino, + (long long int )st.st_ino); + } + } + } bpos += d->d_reclen; }
Signature · 1 line
2.32.0
Amir Goldstein
maintainerQuoted 119 lines · Amir Goldstein
Quoted 117 lines · Amir Goldstein
Quoted 105 lines · Miklos Szeredi
Quoted 103 lines · Miklos Szeredi
Quoted 33 lines · Amir Goldstein
Quoted 22 lines · Miklos Szeredi
Quoted 20 lines · Amir Goldstein
Quoted 17 lines · Amir Goldstein
On the first getdents call,
ovl_iterate()populates the readdir cache with a list of entries, but for upper entries with origin lower inode,p->inoremains zero.Following getdents calls traverse the readdir cache list and call
ovl_cache_update_ino()for entries with zerop->inoto lookup the entry in the overlay and returnd_inothat is consistent withst_ino.If the upper file was unlinked between the first getdents call and the getdents call that lists the file entry,
ovl_cache_update_ino()will not find the entry and fall back to settingd_inoto the upper realst_ino, which is inconsistent with how this object was presented to users.Instead of listing a stale entry with inconsistent
d_ino, simply skip the stale entry, which is better for users.Quoted 8 lines
Quoted 4 lines
Miklos,
I forgot to follow up on this patch.
Upstream xfstest overlay/077 is failing without this patch.Can't reproduce (on ext4/xfs and "-oxino=on").
Is there some trick?
Quoted 8 lines
Not sure. overlay/077 fails for me on v5.14.0-rc2 on ext4/xfs.
QA output created by 077
+entry m100 has inconsistentd_ino(234 != 232) +entry f100 has inconsistentd_ino(335 != 16777542)
Silence is goldenMaybe you need to build src/t_dir_offset2?
root@kvm:/opt/xfstests-dev# git log -1 --pretty=%h 10f6b231 root@kvm:/opt/xfstests-dev# cat local.config export TEST_DEV=/dev/vdb1 export TEST_DIR=/test export SCRATCH_DEV=/dev/vdb2 export SCRATCH_MNT=/scratch export FSTYP=ext4 export OVERLAY_MOUNT_OPTIONS="-o xino=on" root@kvm:/opt/xfstests-dev# make src/t_dir_offset2 make: 'src/t_dir_offset2' is up to date. root@kvm:/opt/xfstests-dev# ./check -overlay overlay/077 FSTYP -- overlay PLATFORM -- Linux/x86_64 kvm 5.14.0-rc2 #276 SMP Tue Jul 20 05:54:44 CEST 2021 MKFS_OPTIONS -- /scratch MOUNT_OPTIONS -- -o xino=on /scratch /scratch/ovl-mntoverlay/077 1s ... 1s
Ran: overlay/077
Passed all 1 testsroot@kvm:/opt/xfstests-dev# cat results/overlay/077.fullCreate file in pure upper dir:
getdents at offset 0 returned 192 bytes
created entry p0
entry p0 found as expectedRemove file in pure upper dir:
getdents at offset 0 returned 192 bytes
unlinked entry p100
entry p100 not found as expectedCreate file in impure upper dir:
getdents at offset 0 returned 192 bytes
created entry o0
entry o0 found as expectedRemove file in impure upper dir:
getdents at offset 0 returned 192 bytes
unlinked entry o100
entry o100 not found as expectedCreate file in merge dir:
getdents at offset 0 returned 192 bytes
created entry m0
entry m0 found as expectedRemove file in merge dir:
getdents at offset 0 returned 192 bytes
unlinked entry m100
entry m100 not found as expectedCreate file in former merge dir:
getdents at offset 0 returned 192 bytes
created entry f0
entry f0 found as expectedQuoted 7 lines
Quoted 6 lines
Remove file in former merge dir:
getdents at offset 0 returned 192 bytes
unlinked entry f100
entry f100 not found as expectedIdeas for further debugging why this test isn't failing for v4.12-rc2?
It's not you, it's me ;-)
Quoted 9 lines
Quoted 5 lines
The failure was lost during cleanup of
t_dir_offset2patches for submission and it is I who was running an older version oft_dir_offset2. Let me figure this out and get back to you with a working test.How about you try again with a version of the test that actually has the check and the error print that I reported.... ;-)
P.S.1:
xino=on is futile for this test as it uses all layers on same fs
P.S.2:
The attached patch to t_dir_offset2 only reproduces the first inconsistency I reported:
entry m100 has inconsistent d_ino (234 != 232)
The second inconsistency requires another small patch to overlay/077 (attached). I will post both those patches once the kernel fix is in overlayfs-next.
Thanks,
Amir.
From e64a7b0c1ce697745c0bdb3ef8f43f0764510ef8 Mon Sep 17 00:00:00 2001
From: Amir Goldstein <[email protected]>
Date: Tue, 20 Jul 2021 19:11:03 +0300
Subject: [PATCH] overlay/077: test inconsistent d_ino in former merge dir
For testing of inconsistent d_ino/st_ino we need to unlink an entry whose st_ino is not that of the upper inode.
In the former merge dir setup we unlink all the files in the lower dir after copyup, so they all use st_ino of the upper inode.
Let the unlinked file f100 reside in a lower path that is not being unlinked so it will have the st_ino of the lower inode.
--- tests/overlay/077 | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
tests/overlay/077 +3 -2
@@ -61,8 +61,8 @@ mkdir -p $lowerdir/merge $lowerdir/former $upperdir/pure $upperdir/impure create_files $lowerdir/merge m # Files to be moved into impure upper dir create_files $lowerdir o -# File to be copied up to make former merge dir impure -touch $lowerdir/former/f100 +# File to be moved into former merge dir to make it impure +touch $lowerdir/f100 _scratch_mount
@@ -72,6 +72,7 @@ create_files $SCRATCH_MNT/former f touch $SCRATCH_MNT/merge/m100 # Move copied up files so readdir will need to lookup origin d_ino mv $SCRATCH_MNT/o* $SCRATCH_MNT/impure/ +mv $SCRATCH_MNT/f100 $SCRATCH_MNT/former/ # Remove the lower directory and mount overlay again to create # a "former merge dir"
Signature · 1 line
2.32.0
Miklos Szeredi
maintainerQuoted 134 lines · Amir Goldstein
Quoted 119 lines · Amir Goldstein
Quoted 117 lines · Amir Goldstein
Quoted 105 lines · Miklos Szeredi
Quoted 103 lines · Miklos Szeredi
Quoted 33 lines · Amir Goldstein
Quoted 22 lines · Miklos Szeredi
Quoted 20 lines · Amir Goldstein
Quoted 17 lines · Amir Goldstein
On the first getdents call,
ovl_iterate()populates the readdir cache with a list of entries, but for upper entries with origin lower inode,p->inoremains zero.Following getdents calls traverse the readdir cache list and call
ovl_cache_update_ino()for entries with zerop->inoto lookup the entry in the overlay and returnd_inothat is consistent withst_ino.If the upper file was unlinked between the first getdents call and the getdents call that lists the file entry,
ovl_cache_update_ino()will not find the entry and fall back to settingd_inoto the upper realst_ino, which is inconsistent with how this object was presented to users.Instead of listing a stale entry with inconsistent
d_ino, simply skip the stale entry, which is better for users.Quoted 8 lines
Quoted 4 lines
Miklos,
I forgot to follow up on this patch.
Upstream xfstest overlay/077 is failing without this patch.Can't reproduce (on ext4/xfs and "-oxino=on").
Is there some trick?
Quoted 8 lines
Not sure. overlay/077 fails for me on v5.14.0-rc2 on ext4/xfs.
QA output created by 077
+entry m100 has inconsistentd_ino(234 != 232) +entry f100 has inconsistentd_ino(335 != 16777542)
Silence is goldenMaybe you need to build src/t_dir_offset2?
root@kvm:/opt/xfstests-dev# git log -1 --pretty=%h 10f6b231 root@kvm:/opt/xfstests-dev# cat local.config export TEST_DEV=/dev/vdb1 export TEST_DIR=/test export SCRATCH_DEV=/dev/vdb2 export SCRATCH_MNT=/scratch export FSTYP=ext4 export OVERLAY_MOUNT_OPTIONS="-o xino=on" root@kvm:/opt/xfstests-dev# make src/t_dir_offset2 make: 'src/t_dir_offset2' is up to date. root@kvm:/opt/xfstests-dev# ./check -overlay overlay/077 FSTYP -- overlay PLATFORM -- Linux/x86_64 kvm 5.14.0-rc2 #276 SMP Tue Jul 20 05:54:44 CEST 2021 MKFS_OPTIONS -- /scratch MOUNT_OPTIONS -- -o xino=on /scratch /scratch/ovl-mntoverlay/077 1s ... 1s
Ran: overlay/077
Passed all 1 testsroot@kvm:/opt/xfstests-dev# cat results/overlay/077.fullCreate file in pure upper dir:
getdents at offset 0 returned 192 bytes
created entry p0
entry p0 found as expectedRemove file in pure upper dir:
getdents at offset 0 returned 192 bytes
unlinked entry p100
entry p100 not found as expectedCreate file in impure upper dir:
getdents at offset 0 returned 192 bytes
created entry o0
entry o0 found as expectedRemove file in impure upper dir:
getdents at offset 0 returned 192 bytes
unlinked entry o100
entry o100 not found as expectedCreate file in merge dir:
getdents at offset 0 returned 192 bytes
created entry m0
entry m0 found as expectedRemove file in merge dir:
getdents at offset 0 returned 192 bytes
unlinked entry m100
entry m100 not found as expectedCreate file in former merge dir:
getdents at offset 0 returned 192 bytes
created entry f0
entry f0 found as expectedQuoted 7 lines
Quoted 6 lines
Remove file in former merge dir:
getdents at offset 0 returned 192 bytes
unlinked entry f100
entry f100 not found as expectedIdeas for further debugging why this test isn't failing for v4.12-rc2?
It's not you, it's me ;-)
Quoted 9 lines
Quoted 5 lines
The failure was lost during cleanup of
t_dir_offset2patches for submission and it is I who was running an older version oft_dir_offset2. Let me figure this out and get back to you with a working test.How about you try again with a version of the test that actually has the check and the error print that I reported.... ;-)
P.S.1:
xino=on is futile for this test as it uses all layers on same fs
Quoted 8 lines
P.S.2:
The attached patch tot_dir_offset2only reproduces the first inconsistency I reported:
entry m100 has inconsistentd_ino(234 != 232)The second inconsistency requires another small patch to overlay/077 (attached). I will post both those patches once the kernel fix is in overlayfs-next.
Thanks.
Verified and pushed.
Miklos