overlayfs v8 problems with whiteout links
Subsystems (2)
Hello,
Now working on a persistent Live that uses overlayfs v8 and tmpfs xattrs, all works near well but sometimes whiteout fails,
I think maybe xattrs create and delete is not synchronized with the operations create and delete for whiteout links.
these are commands from a Debian package build,
showing a command "rm -rf" that cannot remove the files because a whiteout link was not removed successfully.
dpkg-deb: building package nvidia-kernel-2.6.38-4.jpp.1-lnet-amd64' in /usr/src/nvidia-kernel-2.6.38-4.jpp.1-lnet- amd64_270.41.06-1.lnet1+2.6.38-16_amd64.deb'.dh_prep
rm: cannot remove `debian/nvidia-kernel-2.6.38-4.jpp.1-lnet-amd64/DEBIAN':
Directory not emptydh_prep: rm -rf debian/nvidia-kernel-2.6.38-4.jpp.1-lnet-amd64/ returned exit code 1
the directory listing shows the whiteout, but it must be hidden,
# ls -R1l nvidia-kernel-2.6.38-4.jpp.1-lnet-amd64 nvidia-kernel-2.6.38-4.jpp.1-lnet-amd64: total 4 drwxr-xr-x 2 root root 4096 2011-04-22 10:19 DEBIAN
nvidia-kernel-2.6.38-4.jpp.1-lnet-amd64/DEBIAN:
total 0
lrwxrwxrwx 1 root root 18 2011-04-22 10:19 control -> (overlay-whiteout)
Thanks,
Jordi Pujol
Live never ending Tale
GNU/Linux Live forever!
http://livenet.selfip.com
A Divendres 22 Abril 2011 10:43:43, Jordi Pujol va escriure:
Quoted 3 lines
Now working on a persistent Live that uses overlayfs v8 and tmpfs xattrs, all works near well but sometimes whiteout fails,
Hello,
I am trying to reproduce this, repeat the same process and analyze the situation of evarlayfs at every step,
Thanks,
Jordi Pujol
Live never ending Tale
GNU/Linux Live forever!
http://livenet.selfip.com
A Dissabte 23 Abril 2011 14:43:39, Jordi Pujol va escriure:
Quoted 2 lines
I am trying to reproduce this, repeat the same process and analyze the situation of evarlayfs at every step,
this problem is reproducible, occurs a lot of times, here are some logs for various processes that gave this error:
to the log I have added directory listings for upperdir (/lnet/cow) and rootdir ( / )
, listed previously to the error and after it also,
http://livenet.selfip.com/ftp/debian/overlayfs/nvidia-module-build.log
http://livenet.selfip.com/ftp/debian/overlayfs/nvidia-packages-build.log
http://livenet.selfip.com/ftp/debian/overlayfs/nvidia-packages-build-1.log
Thanks,
Jordi Pujol
Live never ending Tale
GNU/Linux Live forever!
http://livenet.selfip.com
Hello,
I have solved this checking if the lower dentry exists when calling ovl_whiteout in ovl_do_remove and ovl_rename,
also added in ovl_whiteout a check for existance of lower dentry and a debug message, as it was already planned as a TODO task in the source code.
by now it works, even the debug message not appears,
Thanks,
Jordi Pujol
Live never ending Tale
GNU/Linux Live forever!
http://livenet.selfip.com
fs/overlayfs/dir.c +12 -3
@@ -33,7 +33,14 @@ static int ovl_whiteout(struct dentry *uconst struct cred *old_cred;struct cred *override_cred; -/* FIXME: recheck lower dentry to see if whiteout is really needed */ + err =0 ; +/* recheck lower dentry to see if whiteout is really needed */ +/* dont consider this an error */ +if (!ovl_dentry_lower (dentry)) { +printk (KERN_DEBUG "overlayfs: DEBUG - no lower dentry to whiteout '%s'\n" , + dentry->d_name.name); +return 0 ; + } err = -ENOMEM ; override_cred =prepare_creds ();
@@ -376,7 +383,8 @@ static int ovl_do_remove(struct dentry *ovl_dentry_version_inc (dentry->d_parent); } -if (type !=OVL_PATH_UPPER ||ovl_dentry_is_opaque (dentry)) +if ((type !=OVL_PATH_UPPER ||ovl_dentry_is_opaque (dentry)) + &&ovl_dentry_lower (dentry)) err =ovl_whiteout (upperdir, dentry);/*
@@ -554,7 +562,8 @@ static int ovl_rename(struct inode *oldd goto out_dput; } -if (old_type !=OVL_PATH_UPPER || old_opaque) +if ((old_type !=OVL_PATH_UPPER || old_opaque) + &&ovl_dentry_lower (old)) err =ovl_whiteout (old_upperdir, old);if (is_dir && old_opaque && !new_opaque)ovl_remove_opaque (olddentry);
Miklos Szeredi
maintainerQuoted 22 lines · Jordi Pujol
Hello,
I have solved this checking if the lower dentry exists when calling
ovl_whiteoutinovl_do_removeandovl_rename,
also added inovl_whiteouta check for existance of lower dentry and a debug message, as it was already planned as a TODO task in the source code.by now it works, even the debug message not appears,
Thanks,
Jordi Pujol
Live never ending Tale
GNU/Linux Live forever!
http://livenet.selfip.comfs/overlayfs/dir.c +0
-0@@ -33,7 +33,14 @@ static int ovl_whiteout(struct dentry *uconst struct cred *old_cred;struct cred *override_cred;
fs/overlayfs/dir.c static int ovl_whiteout(struct dentry *u · line 43
Quoted 10 lines
fs/overlayfs/dir.c +8
-1-/* FIXME: recheck lower dentry to see if whiteout is really needed */ + err =0 ; +/* recheck lower dentry to see if whiteout is really needed */ +/* dont consider this an error */ +if (!ovl_dentry_lower (dentry)) { +printk (KERN_DEBUG "overlayfs: DEBUG - no lower dentry to whiteout '%s'\n" , + dentry->d_name.name); +return 0 ; + }
Okay, thanks for the patch.
It doesn't look like the correct fix, however. Even though ovl_dentry_lower() returns NULL, the lower file should normally exist.
Please look a the ovl_do_lookup() logic: if oe->opaque is set to true, then oe->lowerdentry will be set to NULL, even when the lower dentry does actually exist.
So your patch might cover up the problem that you've been observing (because it won't create the whiteouts in some cases), but it will introduce another bug.
Thanks,
Miklos
A Dimarts 26 Abril 2011 17:50:54, Miklos Szeredi va escriure:
Quoted 2 lines
It doesn't look like the correct fix, however. Even though
ovl_dentry_lower()returns NULL, the lower file should normally exist.
it solves the problem, but we should investigate why,
Quoted 9 lines
Please look a the
ovl_do_lookup()logic: ifoe->opaqueis set to true, thenoe->lowerdentrywill be set to NULL, even when the lower dentry does actually exist.So your patch might cover up the problem that you've been observing (because it won't create the whiteouts in some cases), but it will introduce another bug.
Agree, the working line in this patch is checking for ovl_dentry_lower in ovl_do_remove, the other lines are part of the test; should not be used.
another patch is attached,
Thanks,
Jordi Pujol
Live never ending Tale
GNU/Linux Live forever!
http://livenet.selfip.com
fs/overlayfs/dir.c +2 -1
@@ -376,7 +383,8 @@ static int ovl_do_remove(struct dentry *ovl_dentry_version_inc (dentry->d_parent); } -if (type !=OVL_PATH_UPPER ||ovl_dentry_is_opaque (dentry)) +if ((type !=OVL_PATH_UPPER ||ovl_dentry_is_opaque (dentry)) + &&ovl_dentry_lower (dentry)) err =ovl_whiteout (upperdir, dentry);/*
Miklos Szeredi
maintainerQuoted 10 lines · Jordi Pujol
A Dimarts 26 Abril 2011 17:50:54, Miklos Szeredi va escriure:
Quoted 2 lines
It doesn't look like the correct fix, however. Even though
ovl_dentry_lower()returns NULL, the lower file should normally exist.it solves the problem, but we should investigate why,
Quoted 5 lines
Please look a the
ovl_do_lookup()logic: ifoe->opaqueis set to true, thenoe->lowerdentrywill be set to NULL, even when the lower dentry does actually exist.
Quoted 8 lines
Quoted 4 lines
So your patch might cover up the problem that you've been observing (because it won't create the whiteouts in some cases), but it will introduce another bug.
Agree, the working line in this patch is checking for
ovl_dentry_lowerinovl_do_remove, the other lines are part of the test; should not be used.another patch is attached,
I think I found the real cause of the problem, see attached patch. Can you please test?
Thanks,
Miklos
commit 999204befc34af97bea495dad0ad1d8dfc5a0fe1
Author: Miklos Szeredi <[email protected]> Date: Wed Apr 27 13:23:27 2011 +0200
ovl: fix remove after rename
If the destination of the rename didn't exist on either the upper or the lower tree then the renamed object was erronously set to opaque.
This caused the object to be replaced by a whiteout on unlink/rmdir. This in turn could result in the whiteout "showing through", since whiteouts are not handled in directories existing on the upper layer only, causing failure to remove an empty directory, for example.
fs/overlayfs/dir.c +4 -4
@@ -461,6 +461,7 @@ static int ovl_rename(struct inode *olddir, struct dentry *old, {int err;enum ovl_path_type old_type; +enum ovl_path_type new_type;struct dentry *old_upperdir;struct dentry *new_upperdir;struct dentry *olddentry;
@@ -476,8 +477,6 @@ static int ovl_rename(struct inode *olddir, struct dentry *old,return -EXDEV ;if (new->d_inode) { -enum ovl_path_type new_type; - new_type =ovl_path_type (new);if (new_type ==OVL_PATH_LOWER && old_type ==OVL_PATH_LOWER ) {
@@ -497,6 +496,8 @@ static int ovl_rename(struct inode *olddir, struct dentry *old,if (err)return err; } + }else { + new_type =OVL_PATH_UPPER ; } err =ovl_copy_up (old);
@@ -534,8 +535,7 @@ static int ovl_rename(struct inode *olddir, struct dentry *old, goto out_dput; old_opaque =ovl_dentry_is_opaque (old); - new_opaque =ovl_dentry_is_opaque (new) || -ovl_path_type (new) !=OVL_PATH_UPPER ; + new_opaque =ovl_dentry_is_opaque (new) || new_type !=OVL_PATH_UPPER ;if (is_dir && !old_opaque && new_opaque) { err =ovl_set_opaque (olddentry);
A Dimecres 27 Abril 2011 13:41:19, Miklos Szeredi va escriure:
Quoted 2 lines
I think I found the real cause of the problem, see attached patch. Can you please test?
OK, I will, now it is starting to compile,
Thanks,
Jordi Pujol
Live never ending Tale
GNU/Linux Live forever!
http://livenet.selfip.com
A Dimecres 27 Abril 2011 15:30:35, Jordi Pujol va escriure:
Quoted 6 lines
A Dimecres 27 Abril 2011 13:41:19, Miklos Szeredi va escriure:
Quoted 2 lines
I think I found the real cause of the problem, see attached patch. Can you please test?
OK, I will, now it is starting to compile,
Success, it works,
I have tried it building huge packages in /usr/src directory, Actual custom kernel Debian source is:
http://livenet.selfip.com/ftp/debian/linux-2.6/linux-2.6.38-4.jpp.6-
lnet_2.6.38-21.tar.bz2
Thanks,
Jordi Pujol
Live never ending Tale
GNU/Linux Live forever!
http://livenet.selfip.com
Miklos Szeredi
maintainerQuoted 8 lines · Jordi Pujol
A Dimecres 27 Abril 2011 15:30:35, Jordi Pujol va escriure:
Quoted 6 lines
A Dimecres 27 Abril 2011 13:41:19, Miklos Szeredi va escriure:
Quoted 2 lines
I think I found the real cause of the problem, see attached patch. Can you please test?
OK, I will, now it is starting to compile,
Quoted 7 lines
Success, it works,
I have tried it building huge packages in
/usr/srcdirectory, Actual custom kernel Debian source is:http://livenet.selfip.com/ftp/debian/linux-2.6/linux-2.6.38-4.jpp.6-
lnet_2.6.38-21.tar.bz2
Thanks for testing!
Miklos
Miklos Szeredi
maintainerQuoted 7 lines · Jordi Pujol
A Dissabte 23 Abril 2011 14:43:39, Jordi Pujol va escriure:
Quoted 2 lines
I am trying to reproduce this, repeat the same process and analyze the situation of evarlayfs at every step,
this problem is reproducible, occurs a lot of times, here are some logs for various processes that gave this error:
Quoted 7 lines
to the log I have added directory listings for upperdir (/lnet/cow) and rootdir ( / )
, listed previously to the error and after it also,http://livenet.selfip.com/ftp/debian/overlayfs/nvidia-module-build.log
http://livenet.selfip.com/ftp/debian/overlayfs/nvidia-packages-build.log
http://livenet.selfip.com/ftp/debian/overlayfs/nvidia-packages-build-1.log
Can you please list the relevant lower directory as well?
Thanks,
Miklos
A Dimarts 26 Abril 2011 17:34:48, Miklos Szeredi va escriure:
Quoted 1 line
Can you please list the relevant lower directory as well?
the lower directory arrives to /usr/src
lower /usr/src is empty
Thanks,
Jordi Pujol
Live never ending Tale
GNU/Linux Live forever!
http://livenet.selfip.com