ovl: relax lookup error on mismatch origin ftype
Amir Goldstein
maintainerPATCH 1/1 ovl: relax lookup error on mismatch origin ftype
We get occasional reports of lookup errors due to mismatched origin ftype from users that re-format a lower squashfs image.
Commit 13c6ad0f45fd ("ovl: document lower modification caveats") tries to discourage the practice of re-formating lower layers and describes the expected behavior as undefined.
Commit b0e0f69731cd ("ovl: restrict lower null uuid for "xino=auto"") limits the configurations in which origin file handles are followed.
In addition to these measures, change the behavior in case of detecting a mismatch origin ftype in lookup to issue a warning, not follow origin, but not fail the lookup operation either.
That should make overall more users happy without any big consequences.
---
Miklos,
I am getting tired of dealing with lower squashfs related reports. How about this?
It passes the xfstests quick tests and no, I do not have a reproducer for origin mismatch, so will wait for Byron to test the patch.
Thanks,
Amir.
fs/overlayfs/namei.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
fs/overlayfs/namei.c +2 -2
@@ -392,7 +392,7 @@ int ovl_check_origin_fh(struct ovl_fs *ofs, struct ovl_fh *fh, bool connected, upperdentry,d_inode (upperdentry)->i_mode &S_IFMT , d_inode(origin)->i_mode & S_IFMT);dput (origin); -return -EIO ; +return -EINVAL ; } static int ovl_check_origin(struct ovl_fs *ofs, struct dentry *upperdentry,
@@ -408,7 +408,7 @@ static int ovl_check_origin(struct ovl_fs *ofs, struct dentry *upperdentry,kfree (fh);if (err) { -if (err == -ESTALE ) +if (err == -ESTALE || err == -EINVAL )return 0 ;return err; }
Signature · 1 line
2.25.1
A message that is not in the archive (sent off-list or not yet received): [email protected]
Amir Goldstein
maintainerQuoted 11 lines · www
Hi Amir,
Thank you very much for your help. I have another question to clarify.
Quoted 4 lines
Quoted 1 line
3. If we upgrade overlayfs separately, we are not very good at verifying that we have solved this problem, because the recurrence probability of this problem is very low. So I want to ask, how can we quickly reproduce this problem?
Re-creating a lower squashfs after files have been copied to upper should reproduce the problem quite often.
Quoted 8 lines
Does the re-creating lower squashfs here mean that remount squashfs?
I've tested dozens of times in the remount way, but I haven't found this problem again?
My test steps are:
1). umount lower squashfs;
2). modify the file in upper dir;
3). mount lower squanshfs;
4). restart service(it will re-parse the modified file) or reboot the system and the problem didn't happen.
No. That's not what I mean by re-creating lower fs.
What I mean is that overlayfs is the file is question is in the squashfs image and has been copied up.
I don't know where the squashfs image you are using comes from but I am guessing you have replaced it with a new squashfs image. In the new squashfs image, files have different inode numbers.
I reckon this behavior is common for OpenWRT where the system image is being upgraded.
Quoted 12 lines
Quoted 10 lines · Amir Goldstein
We get occasional reports of lookup errors due to mismatched origin ftype from users that re-format a lower squashfs image.
Commit
13c6ad0f45fd("ovl: document lower modification caveats") tries to discourage the practice of re-formating lower layers and describes the expected behavior as undefined.Commit
b0e0f69731cd("ovl: restrict lower null uuid for "xino=auto"") limits the configurations in which origin file handles are followed.
Quoted 8 lines
Quoted 6 lines
In addition to these measures, change the behavior in case of detecting a mismatch origin ftype in lookup to issue a warning, not follow origin, but not fail the lookup operation either.
That should make overall more users happy without any big consequences.
We can't do that. When overlayfs reports an error, there may be some very important files that cannot be correctly parsed, resulting in some important services can not be started smoothly, which directly affects the use of the whole system.
Well, that is the purpose of my patch -
When overlayfs reports the warning it will NOT prevent reading the file, so it should NOT make the whole system unusable.
Thanks,
Amir.
A message that is not in the archive (sent off-list or not yet received): [email protected]
Amir Goldstein
maintainerQuoted 38 lines · www
Hi Amir,
Quoted 25 lines · Amir Goldstein
Quoted 11 lines · www
Hi Amir,
Thank you very much for your help. I have another question to clarify.
Quoted 4 lines
Quoted 1 line
3. If we upgrade overlayfs separately, we are not very good at verifying that we have solved this problem, because the recurrence probability of this problem is very low. So I want to ask, how can we quickly reproduce this problem?
Re-creating a lower squashfs after files have been copied to upper should reproduce the problem quite often.
Quoted 8 lines
Does the re-creating lower squashfs here mean that remount squashfs?
I've tested dozens of times in the remount way, but I haven't found this problem again?
My test steps are:
1). umount lower squashfs;
2). modify the file in upper dir;
3). mount lower squanshfs;
4). restart service(it will re-parse the modified file) or reboot the system and the problem didn't happen.No. That's not what I mean by re-creating lower fs.
What I mean is that overlayfs is the file is question is in the squashfs image and has been copied up.Quoted 7 lines
I don't know where the squashfs image you are using comes from but I am guessing you have replaced it with a new squashfs image. In the new squashfs image, files have different inode numbers.
I reckon this behavior is common for OpenWRT where the system image is being upgraded.
Quoted 11 lines
https://github.com/openbmc/openbmc/blob/master/meta-phosphor/recipes-phosphor/initrdscripts/files/obmc-init.sh (init file)
...
line 376: mount/run/image-rofs/run/initramfs/ro-t squanshfs -o ro,loop (line 358, set copy-base-filesystem-to-ram=y)
...
line 416: mkdir -p $upper/var/log --- new add line, not in the file
line 417: mount -t overlayfs -o lowerdir=run/initramfs/ro,upperdir=run/initramfs/rw/cow,workdir=run/initramfs/rw/work cow /root
...
I would like to ask if the newly added line(line 416) or "set set copy-base-filesystem-to-ram=y" causes this problem?(This folder is created and the value is set every time the system starts, but the probability of problems is not high) There are no other changes.
I don't know.
I am not sure what wasn't clear in my answer.
The lowerfs image, namely /run/image-rofs in your script can be downloaded from tftp or from the web. I have no idea what actually happens in your setup.
Every time you run this init script this lower fs image may change, while the upper f2fs rwfs does not change.
When that happens, it MAY cause the reported issue.
Thanks,
Amir.
A message that is not in the archive (sent off-list or not yet received): [email protected]
Amir Goldstein
maintainerQuoted 69 lines · www
Hi Amir,
Quoted 51 lines · Amir Goldstein
Quoted 38 lines · www
Hi Amir,
Quoted 25 lines · Amir Goldstein
Quoted 11 lines · www
Hi Amir,
Thank you very much for your help. I have another question to clarify.
Quoted 4 lines
Quoted 1 line
3. If we upgrade overlayfs separately, we are not very good at verifying that we have solved this problem, because the recurrence probability of this problem is very low. So I want to ask, how can we quickly reproduce this problem?
Re-creating a lower squashfs after files have been copied to upper should reproduce the problem quite often.
Quoted 8 lines
Does the re-creating lower squashfs here mean that remount squashfs?
I've tested dozens of times in the remount way, but I haven't found this problem again?
My test steps are:
1). umount lower squashfs;
2). modify the file in upper dir;
3). mount lower squanshfs;
4). restart service(it will re-parse the modified file) or reboot the system and the problem didn't happen.No. That's not what I mean by re-creating lower fs.
What I mean is that overlayfs is the file is question is in the squashfs image and has been copied up.Quoted 7 lines
I don't know where the squashfs image you are using comes from but I am guessing you have replaced it with a new squashfs image. In the new squashfs image, files have different inode numbers.
I reckon this behavior is common for OpenWRT where the system image is being upgraded.
Quoted 11 lines
https://github.com/openbmc/openbmc/blob/master/meta-phosphor/recipes-phosphor/initrdscripts/files/obmc-init.sh (init file)
...
line 376: mount/run/image-rofs/run/initramfs/ro-t squanshfs -o ro,loop (line 358, set copy-base-filesystem-to-ram=y)
...
line 416: mkdir -p $upper/var/log --- new add line, not in the file
line 417: mount -t overlayfs -o lowerdir=run/initramfs/ro,upperdir=run/initramfs/rw/cow,workdir=run/initramfs/rw/work cow /root
...
I would like to ask if the newly added line(line 416) or "set set copy-base-filesystem-to-ram=y" causes this problem?(This folder is created and the value is set every time the system starts, but the probability of problems is not high) There are no other changes.Quoted 7 lines
I don't know.
I am not sure what wasn't clear in my answer.
The lowerfs image, namely
/run/image-rofsin your script can be downloaded from tftp or from the web. I have no idea what actually happens in your setup.the
/run/image-rofsimage is copied form flash.
line 359: test ! -e/run/image-rofs&& ! cp $rodev/run/image-rofs-- the $rodev is mtd4
Quoted 7 lines
Quoted 4 lines
Every time you run this init script this lower fs image may change, while the upper f2fs rwfs does not change.
When that happens, it MAY cause the reported issue.the
/run/image-rofsis squashfs, readonly, cannot be modified.
Do you mean that when I upgrade the new firmware, rofs (squashfs) has been updated (rofs has been modified), is it easy to have this problem?
YES!
Thanks,
Amir.
Miklos Szeredi
maintainerQuoted 21 lines · Amir Goldstein
We get occasional reports of lookup errors due to mismatched origin ftype from users that re-format a lower squashfs image.
Commit
13c6ad0f45fd("ovl: document lower modification caveats") tries to discourage the practice of re-formating lower layers and describes the expected behavior as undefined.Commit
b0e0f69731cd("ovl: restrict lower null uuid for "xino=auto"") limits the configurations in which origin file handles are followed.In addition to these measures, change the behavior in case of detecting a mismatch origin ftype in lookup to issue a warning, not follow origin, but not fail the lookup operation either.
That should make overall more users happy without any big consequences.
Link https://lore.kernel.org/linux-unionfs/CAOQ4uxgPq9E9xxwU2CDyHy-_yCZZeymg+3n+-6AqkGGE1YtwvQ@mail.gmail.com/ open hereSigned-off-by Amir Goldstein <[email protected]>---
Quoted 7 lines
Miklos,
I am getting tired of dealing with lower squashfs related reports. How about this?
It passes the xfstests quick tests and no, I do not have a reproducer for origin mismatch, so will wait for Byron to test the patch.
Pushed a simplified variant that just changes theovl_check_origin_fh() return value from -EIO to -ESTALE. Do you see
a problem with this?
Thanks,
Miklos
Quoted 32 lines
Thanks,
Amir.fs/overlayfs/namei.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)fs/overlayfs/namei.c +2
-3@@ -392,7 +392,7 @@ int ovl_check_origin_fh(struct ovl_fs *ofs, struct ovl_fh *fh, bool connected, upperdentry,d_inode (upperdentry)->i_mode &S_IFMT , d_inode(origin)->i_mode & S_IFMT);dput (origin); -return -EIO ; +return -EINVAL ; } static int ovl_check_origin(struct ovl_fs *ofs, struct dentry *upperdentry,@@ -408,7 +408,7 @@ static int ovl_check_origin(struct ovl_fs *ofs, struct dentry *upperdentry,kfree (fh);if (err) { -if (err == -ESTALE ) +if (err == -ESTALE || err == -EINVAL )return 0 ;return err; } --2.25.1
Amir Goldstein
maintainerQuoted 28 lines · Miklos Szeredi
Quoted 26 lines · Amir Goldstein
We get occasional reports of lookup errors due to mismatched origin ftype from users that re-format a lower squashfs image.
Commit
13c6ad0f45fd("ovl: document lower modification caveats") tries to discourage the practice of re-formating lower layers and describes the expected behavior as undefined.Commit
b0e0f69731cd("ovl: restrict lower null uuid for "xino=auto"") limits the configurations in which origin file handles are followed.In addition to these measures, change the behavior in case of detecting a mismatch origin ftype in lookup to issue a warning, not follow origin, but not fail the lookup operation either.
That should make overall more users happy without any big consequences.
Link https://lore.kernel.org/linux-unionfs/CAOQ4uxgPq9E9xxwU2CDyHy-_yCZZeymg+3n+-6AqkGGE1YtwvQ@mail.gmail.com/ open hereSigned-off-by Amir Goldstein <[email protected]>---
Miklos,
I am getting tired of dealing with lower squashfs related reports. How about this?
Quoted 7 lines
Quoted 2 lines
It passes the xfstests quick tests and no, I do not have a reproducer for origin mismatch, so will wait for Byron to test the patch.
Pushed a simplified variant that just changes the
ovl_check_origin_fh()return value from-EIOto-ESTALE. Do you see
a problem with this?
The only difference is if ovl_fh_to_dentry() also emits a warning, but since ovl_check_origin_fh() already has a warning I think that your simplified version is fine.
Thanks,
Amir.
Miklos Szeredi
maintainerQuoted 33 lines · Amir Goldstein
Quoted 31 lines · Miklos Szeredi
Quoted 28 lines · Amir Goldstein
We get occasional reports of lookup errors due to mismatched origin ftype from users that re-format a lower squashfs image.
Commit
13c6ad0f45fd("ovl: document lower modification caveats") tries to discourage the practice of re-formating lower layers and describes the expected behavior as undefined.Commit
b0e0f69731cd("ovl: restrict lower null uuid for "xino=auto"") limits the configurations in which origin file handles are followed.In addition to these measures, change the behavior in case of detecting a mismatch origin ftype in lookup to issue a warning, not follow origin, but not fail the lookup operation either.
That should make overall more users happy without any big consequences.
Link https://lore.kernel.org/linux-unionfs/CAOQ4uxgPq9E9xxwU2CDyHy-_yCZZeymg+3n+-6AqkGGE1YtwvQ@mail.gmail.com/ open hereSigned-off-by Amir Goldstein <[email protected]>---
Miklos,
I am getting tired of dealing with lower squashfs related reports. How about this?
It passes the xfstests quick tests and no, I do not have a reproducer for origin mismatch, so will wait for Byron to test the patch.
Quoted 8 lines
Quoted 4 lines
Pushed a simplified variant that just changes the
ovl_check_origin_fh()return value from-EIOto-ESTALE. Do you see
a problem with this?The only difference is if
ovl_fh_to_dentry()also emits a warning, but sinceovl_check_origin_fh()already has a warning I think that your simplified version is fine.
Yeah, you are right, new version could have one less warning in the ->fh_to_dentry() case, but I agree that that should not be a problem.
Thanks,
Miklos
Amir Goldstein
maintainerQuoted 40 lines · Miklos Szeredi
Quoted 38 lines · Amir Goldstein
Quoted 35 lines · Miklos Szeredi
Quoted 21 lines · Amir Goldstein
We get occasional reports of lookup errors due to mismatched origin ftype from users that re-format a lower squashfs image.
Commit
13c6ad0f45fd("ovl: document lower modification caveats") tries to discourage the practice of re-formating lower layers and describes the expected behavior as undefined.Commit
b0e0f69731cd("ovl: restrict lower null uuid for "xino=auto"") limits the configurations in which origin file handles are followed.In addition to these measures, change the behavior in case of detecting a mismatch origin ftype in lookup to issue a warning, not follow origin, but not fail the lookup operation either.
That should make overall more users happy without any big consequences.
Link https://lore.kernel.org/linux-unionfs/CAOQ4uxgPq9E9xxwU2CDyHy-_yCZZeymg+3n+-6AqkGGE1YtwvQ@mail.gmail.com/ open hereSigned-off-by Amir Goldstein <[email protected]>---
Quoted 7 lines
Miklos,
I am getting tired of dealing with lower squashfs related reports. How about this?
It passes the xfstests quick tests and no, I do not have a reproducer for origin mismatch, so will wait for Byron to test the patch.
Pushed a simplified variant that just changes the
ovl_check_origin_fh()return value from-EIOto-ESTALE. Do you see
a problem with this?
Quoted 7 lines
Quoted 3 lines
The only difference is if
ovl_fh_to_dentry()also emits a warning, but sinceovl_check_origin_fh()already has a warning I think that your simplified version is fine.Yeah, you are right, new version could have one less warning in the ->fh_to_dentry() case, but I agree that that should not be a problem.
Technically, there is also a change in vfs api in the error returned to exportfs_decode_fh_raw(), but both callers seem to convert EINVAL to ESTALE anyway...
Thanks,
Amir.