VFS based Union Mount
Subsystems (14)
cover VFS based Union Mount
Hi,
Here is an attempt towards vfs based union mount implementation. Union mount provides the filesytem namespace unification feature. Unlike the traditional mounts which hide the contents of the mount point, the union mount presents the merged view of the mount point and the mounted filesytem.
These patches were originally developed for 2.6.11 by Jan Blunck and lately we have been working together in taking this forward. The current patchset applies against 2.6.21-rc6-mm1.
The code is in a highly experimental stage at the moment and the intention of posting this now is to get some initial feedback about the design and the future directions about how this should be taken forward.
You can find more details about union mount in the documentation included in the patchset.
Kindly review and let us know your comments.
Regards,
Bharata.
RFC PATCH 1/15 Add union mount documentation
From: Bharata B Rao <[email protected]>
Subject: Add union mount documentation.
This is an attempt to document some of the implementation details and issues of union mount.
--- Documentation/union-mounts.txt | 489 +++++++++++++++++++++++++++++++++++++++++ 1 files changed, 489 insertions(+)
Documentation/union-mounts.txt new file +489 -0
@@ -0,0 +1,489 @@ +VFS BASED UNION MOUNT +===================== + +1. Overview +2. Union stack +3. Lookup +4. Readdir +5. Copyup +6. Whiteout + 6.1. Creation and deletion + 6.2. Whiteout filetype support + 6.3. Directory renaming +7. Usage +8. State of the code +9. Extracted mail comments + +1. Overview +----------- +Union mount allows mounting of two or more filesystems transparently on +a single mount point. The contents(files or directories) of all the +filesystems become visible at the mount point after a union mount. If +there are files of same name in multiple layers, only the topmost files remain +visible in a union mount. However (currently) common named directories are +again union-ed to present a unified view at the subdir level. + +In this approach of unioning filesystems, the layering information of +different components of the union mount are maintained at the VFS layer. +Hence we call this a VFS based union mount. + +2. Union stack +-------------- +Union stack reflects the stacking of two or more filesystems of the +union mount. The stacking or the layering information is maintained +as part of dentry structures of the mountpoint and mount root. + +The union stack information in the dentry structure looks like this: + +struct dentry { + ... + +#ifdef CONFIG_UNION_MOUNT + struct dentry *d_overlaid; /* overlaid directory */ + struct dentry *d_topmost; /* topmost directory */ + struct union_info *d_union; /* union stack info */ +#endif + ... +}; + +struct union_info { + struct mutex u_mutex; + atomic_t u_count; +}; + +There is one union_info shared by all dentries which are part of +a union and u_count member holds the number of references to the union +stack. When this reaches zero, the union stack ceases to exist and +the union_info is freed. + +Union stack is essentially a singly linked list of dentries of the union +with d_topmost as the head of the list and d_overlaid points +to the next member of the stack. The walking of union stack is guarded by +the u_mutex member. + +dget() references every dentry of the overlaid union stack to make sure +that no dentry of the stack is discarded from memory while others are +still in use. Since walking of union stack is protected by a mutex, +dget() can now sleep. + +dput() also walks the union stack and releases references to all the +dentries that are part of the union. If a dentry's reference count +in a union stack reaches zero, it implies that the dentries above it +in the stack must also be unused and the union stack can be safely +destroyed at this point. + +Since dget() can sleep with union mount, it becomes necessary to +fix many callers of dget() to release and re-acquire any spinlocks +they are holding until they acquire the union lock(mutex). + +3. Lookup +--------- +With union mount, it becomes necessary to lookup pathnames not only +in the topmost filesystem but also in the underlying filesystems. + +In case of looking up a filename, the lookup routines as a rule return +the match from the topmost layer. However if the file is not found +in the topmost layer, the lookup routines have been modified to +find the file in the underlying filesystems of the union stack. + +When looking up a directory under a union mount point, the lookup +code has been modified to build a union stack (if necessary). + +When looking up a name in a union directory, it is necessary to +guarantee that the returned union stack remains valid. Hence +concurrent lookups are prevented by obtaining the mutex lock during +lookups. + +4. Readdir +---------- +The core functionality of union mount, viz., the merged view of +multiple directories is provided by the readdir()/getdents() routines. +This is achieved by reading the contents of every directory of the union +stack and by merging the result. + +The directory entries are read starting from the top layer and they +are maintained in a cache. Subsequently when the entries from the bottom layers +of the union stack are read they are checked for duplicates (in the cache) +before being passed out to the user space. There can be multiple calls +to readdir/getdents routines for reading the entries of a single directory. +But union directory cache is not maintained across these calls. Instead +for every call, the previously read entries are re-read into the cache +and newly read entries are compared against these for duplicates before +being they are returned to user space. We are aware that this is not +the most ideal solution for merging the directory entries. This approach +involves setting up the cache for every getdents() call, re-reading some +of the entries again into the cache and destroying the cache at the end +of getdents() call. And this happens for every getdents() call. + +But there is an even bigger problem. Since readdir() on the union directory +returns contents of all the underlying directories, it is possible +that the file position exceeds the inode size of the first directory. +Therefore the file position is rearranged to select the correct directory +in the union stack. This is done by subtracting the inode size if the +file position exceeds it and selecting the next member of the union stack next. + +This works well with filesystems like ext2/3 that use flat file directories. +The directory entry offsets are arranged linear and are always smaller than +the inode size of the directory. Modern filesystems have implemented +directories differently and just return special cookies as directory entry +offsets which are unrelated to the position in the directory or the inode +size. So the current approach of directory merging is working only for +file systems like ext2 and ext3. + +5. Copyup +--------- +In this implementation of union mount, only the files residing in +the topmost layer are writable. With this restriction, when a file residing +in a bottom layer is opened for writing, it is copied up to the topmost layer +and the write is allowed there. The copyup is done by first creating the +file in the topmost layer and then copying the contents of the file. + +If it becomes necessary to create a directory structure in the top layer +while copying up a file, then it is done so. + +Every time a file is opened for writing, we have introduced a check to +see if this file belongs to a union and if so resides in the bottom +layer of the union stack. Only then the copyup operation is performed. +VFS routines are used directly to create the file in the topmost layer. +However to copy the contents of the file from within the kernel splice +routines are used. + +6. Whiteout +----------- +A whiteout file is a placeholder for a file that does not exist from a +logical point of view. VFS returns -ENOENT for any reference to whiteouts. + +Typically whiteouts are created in the topmost layer when a file in +the lower layer is deleted. The whiteout essentially masks out the file +in the lower layer. + +6.1 Creation and deletion + +With union mount, a top layer whiteout is created in the following scenarios: +- A file/directory which resides only the bottom layer is removed. +- A file/directory which resides in both the layers are removed. + +The VFS calls like unlink(), rename() and rmdir() have been modified to create +a whiteout automatically when the above situation occurs. + +A whiteout is automatically deleted whenever a new file or directory +with a corresponding name is created. This happens in calls like +create(), mknod(), symlink(), link() and mkdir(). + +There is a special case in mkdir(). When a whiteout is replaced by a +directory, it is marked opaque (by using new S_OPAQUE inode flag). +And lookup wouldn't descend down to lower directories if a directory +is marked opaque. This is needed in the following scenario: + +# rm -rf dir/ +# mkdir dir + +The newly created dir/ has to be marked opaque, otherwise the contents +of union stack would become visible again. And it is not expected to +find a non-empty directory immediately after it's creation. + +6.2. Whiteout filetype support + +Creation or deletion of whiteouts is a persistent operation and hence it +needs support from the underlying filesystem. + +Linux already defines DT_WHT(include/linux/fs.h) for whiteout directory +entry (file)type. In addition we need to define the whiteout filetype +for which we make use of an unused bit in the filetype bitmask and +define S_IFWHT (include/linux/stat.h). + +Filesystems which support the whiteout filetype should set the FS_WHT +flag (include/linux/fs.h) on .fs_type in their file_system_type structure. + +Additionally they have to implement the whiteout inode operation. + +int (*whiteout)(struct inode *dir, struct dentry *dentry); + +where 'dentry' is the negative dentry to be masked out under the parent 'dir'. + +In the current implementation, there is an inode for every whiteout in the +filesystem. But since a whiteout doesn't have any usable attribute apart +from it's name(name of the whiteout file is stored as directory entry +in the parent directory), it is an ideal candidate for being replaced by +a singleton object. We have plans to explore this option at a later point +in time. + +In ext2 and ext3 filesystems, whiteout is introduced as an incompatible +feature and only readonly mounts are allowed without whiteout support. +tune2fs(8) from e2fsprogs has been modified to add whiteout support to +ext2/3. + +6.3. Directory renaming +<TODO> + +7. Usage +-------- +The way to union mount filesystems on two devices /dev/sda1 and /dev/sda2, +on a mountpoint union/ is like this: + +- Mount the first filesystem normally and this becomes the lower layer +of the union stack. +# mount /dev/sda1 union/ + +- Mount the second filesystem as a union on top of first +# mount --union /dev/sda2 union/ + +The mount(8) command from util-linux needs to be modified to make it +interpret the --union option. + +After this the union/ will have the merged contents of /dev/sda1 +and /dev/sda2. + +8. State of the code +-------------------- +The entire code is in highly experimental stage at present. + +These are a number of (un)known issues/shortcomings: + +- Unstable, might crash any time. Hasn't undergone any decent levels + of testing. +- We are touching some fastpaths in the lookup code and introducing the + latency of obtaining a mutex in dget() (only for union mount cases). + We haven't yet benchmarked this to check the (adverse) effects. +- Known to union mount correctly only two filesystems. Not tried with more. +- Unioning of subdirectories within a union mount is working, but is buggy. +- Whiteout support in ext3 is not thoroughly analyzed/tested for correctness. +- The side effects of union mount changes on other subsystems + (eg cpuset, aio, dnotify, inotify etc which are touched by union + mount changes) haven't been tested yet. +- bind/move vs union mount not yet handled. +- Readdir has issues as noted above. +- Some lockdep warnings need to be addressed still. +- In general some code cleanliness issues are yet to be handled. + +9. Extracted mail comments +-------------------------- + +These are some of the extracts from an old linux-fsdevel post. + +---- +Andries Brouwer wrote: +> +> On "union mounts". +> We must first have a theory on what "union mount" means. +> Union is a commutative operator, but here there is no symmetry +> at all, so "union" is a misnomer. There is an order. +> +> One might consider partial orders, so that one obtains a tree of mounts, +> but I do not know any applications, and there is the problem of naming. +> So, for simplicity, maybe there is a linear order. +> +> Things happen in the top one. All others are read-only. +> + +Yes, that is correct. This is naturally since the stacking of vfsmount objects +has been like this before. + +---- + +Alexander Viro wrote: +> +> > Does not same thing apply also for common subdirectories? +> +> Not. union-mount != unionfs, it does not descend into subdirectories. +> There is no way in hell to do that and permit sharing the union-mount +> components between several mountpoints. unionfs is very different animal +> and there the main point is that you are getting real, honest +> copy-on-write, i.e. if you have foo/bar/baz on underlying filesystem than +> any attempt to access foo will create a shadowing directory in the upper +> layer, any attempt to access foo/bar will do the same for foo/bar and +> attempt to write into the foo/bar/baz will lead to copying the thing into +> the upper layer and changing it there. _Very_ useful when you have a +> read-only fs and want to run make on it, for one thing - everything +> new/modified gets into the covering layer, along with the accessed part of +> directory tree. Very nice, but completely different - there are things +> impossible for one and doable on another. +> + +---- + +Werner Almesberger wrote: +> +> Hmm, now I'm throughly confused :-( What is the "union" in here then ? +> Is it that a lookup for a top-level component searches all file system +> in that list, or does it simply mean that all the file systems are +> internally linked to the same place, but only one of them is truly +> visible ? +> +> E.g., given +> +> # mount /dev/a /mnt +> # mkdir -p /mnt/foo/blah /mnt/bar +> # umount /dev/a +> # mount /dev/b /mnt +> # mkdir -p /mnt/foo/zulu /mnt/baz +> # mount -o union /dev/a /mnt +> +> # cd /mnt/foo/blah works ? +> # cd /mnt/foo/zulu works too ? (no, I guess) +> # cd /mnt/baz works ? +> # cd /mnt/bar works too ? +> # cd /mnt; touch file works ? on which device is the file created ? +> # cd /mnt/foo; touch file works ? +> # cd /mnt/foo/blah; touch file works ? +> # cd /mnt/foo/zulu; touch file works too ? (no, I guess) +> + +# cd /mnt/foo/blah works ! +# cd /mnt/foo/zulu works ! +# cd /mnt/baz works ! +# cd /mnt/bar works ! +# cd /mnt; touch file file created on /dev/a +# cd /mnt/foo; touch file file created on /dev/a +# cd /mnt/foo/blah; touch file file created on /dev/a +# cd /mnt/foo/zulu; touch file zulu copied to /dev/a and file created on it + +---- + +Alexander Viro wrote: +> +> A) suppose we have a bunch of filesystems union-mounted on /foo/bar. We do +> chdir("/foo/bar"), what should become busy? Variants: +> mountpoint, first element, last element, all of them. +> B) after the action in (A) we add another filesystem to the set. Again, what +> should happen to the busy/not busy status of the components? +> C) we start with the normal mount and union-mount something else. +> Question: what is the desired result (almost definitely the set of old +> and new mounted stuff) and who should become busy? +> D) In the cases above, what do we want to get from stat(2)? +> E) What do we want to do if we do normal mount atop of the union-mount? +> Variants: try to replace, return -EBUSY. Doing replace (i.e. if +> everything can be umounted - do it and mount the new fs in place of the +> union) is attractive - we probably might treat the normal mount same way, +> which kills the "I've clicked in my point'n'drool krapplication ten times +> and it mounted CD ten times, waaaaaah" bug reports. +> Disadvantage: may need small fixes to mount(8) (basically, "if we already +> have mtab entry for this mountpoint and mount succeeds - discard the old +> one"). +> + +I don't understand the union mount as a set of mounts because we also need a +strict order to remove duplicate filenames from the directory +listing. Therefore after union mounting a filesystem the mount-points +filesystem is busy. A chdir() to the mount-point makes the last mounted +filesystem busy since a lookup returns the root directory of the topmost +filesystem. + +---- + +Alexander Viro wrote: +> > +> > > A) suppose we have a bunch of filesystems union-mounted on +> > > /foo/bar. We do chdir("/foo/bar"), what should become busy? Variants: +> > > mountpoint, first element, last element, all of them. +> > +> > I believe that all of them. Or, we can make alternative and mark +> > none of them busy (together with Tigran yet-to-write force unmount) - +> > if there is reason why cwd should make filesystem busy at all... +> +> Ouch. "All" means that we can't, e.g expire elements of union. +> + + +---- + +Andries Brouwer wrote: +> +> > A) suppose we have a bunch of filesystems union-mounted on +> > /foo/bar. We do chdir("/foo/bar"), what should become busy? Variants: +> > mountpoint, first element, last element, all of them. +> +> Last element. +> +> > B) after the action in (A) we add another filesystem to the set. +> > Again, what should happen to the busy/not busy status of the components? +> +> Previous top one has now become busy. All other were busy already. +> +> > C) we start with the normal mount and union-mount something else. +> > Question: what is the desired result (almost definitely the set of old and +> > new mounted stuff) and who should become busy? +> +> First element now is busy. +> +> > D) In the cases above, what do we want to get from stat(2)? +> +> stat(2) on this directory looks at the top one +> +> > E) What do we want to do if we do normal mount atop of the +> > union-mount? Variants: try to replace, +> +> No. Very strange semantics for a mount. +> +> > return -EBUSY. +> +> Yes, quite reasonable. But I would prefer the third: just succeed. +> We have a file hierarchy, and do a mount - well, we already know what that +> means, and we just do it. +> +> [I would prefer to return -EBUSY only when the same filesystem was already +> mounted (in the same way) on the same mount point.] +> + + +---- + +Neil Brown wrote: +> +> A "mount" is an ordered list (pile) of directories. +> One of these elements is the "mountpoint", and it is particularly +> distiguished because ".." from the "mount" goes through ".." of the +> "mountpoint". ".." of all other directories is not accessable. +> +> Each directory in the pile has two flags (well, three if you count +> IS_MOUNTPOINT): +> +> IS_WRITABLE: You can create things in here. +> IS_VISIBLE: You can see inside this. +> +> Thus, a traditional mount has two directories in the pile. +> The bottom one IS_MOUNTPOINT +> The top one IS_WRITABLE|IS_VISIBLE +> +> With mount -o union, you can set what ever flags you like, though +> having IS_WRITABLE and not IS_VISIBLE would be a problem. +> However you can only have one IS_MOUNTPOINT directory. +> +> Now the rules: +> +> 1/ on "lookup", you do a lookup in each IS_VISIBLE directory from the +> top down until you find a match or you hit the bottom. +> +> 2/ If you decide to create something (*) then it goes in the uppermost +> IS_WRITABLE directory. +> +> 3/ "stat" (of ".") sees the IS_MOUNTPOINT directory if it IS_VISIBLE, +> otherwise the lowest IS_VISIBLE directory. +> Possibly n_links could be fiddled, but I don't know how important +> that is. +> +> 4/ The "mount" keeps only the IS_MOUNTPOINT directory busy. +> +> 5/ An open or cd to the mount makes the directory which "stat" sees +> busy. +> +> 6/ A mount is not allowed if it would change 'the directory which +> "stat" sees', and that directory is "busy". +> +> (*) It is unclear to me when creation should be allowed. +> If I say "mkdir fred", and fred does not exist in or above the +> uppermost IS_WRITABLE directory, but does exist is a lower +> IS_VISIBLE directory, should the create succeed or fail? +> Would that same be true for +> open("fred", O_CREAT) which is "create if it doesn't exist" +> or open("fred", O_CREAT|O_EXCL) which is "create and it mustn't exist". +> + +For the complete thread refer to: +http://marc.theaimsgroup.com/?l=linux-fsdevel&m=96035682927821&w=2 + +--- +- Bharata B Rao <[email protected]> +- Jan Blunck <[email protected]> + +April 2007
RFC PATCH 2/15 Add a new mount flag (MNT_UNION) for union mount
In reply to Bharata B RaoFrom: Jan Blunck <[email protected]>
Subject: Add a new mount flag (MNT_UNION) for union mount.
Introduce MNT_UNION, MS_UNION and FS_WHT flags. There are the necessary flags for doing
mount /dev/hda3 /mnt -o union
You need additional patches for util-linux for that to work.
--- fs/namespace.c | 13 ++++++++++++- include/linux/fs.h | 2 ++ include/linux/mount.h | 1 + 3 files changed, 15 insertions(+), 1 deletion(-)
fs/namespace.c +12 -1
@@ -376,6 +376,7 @@ static int show_vfsmnt(struct seq_file * { MNT_NOATIME, ",noatime" }, { MNT_NODIRATIME, ",nodiratime" }, { MNT_RELATIME, ",relatime" }, + { MNT_UNION, ",union" }, { 0, NULL } };struct proc_fs_info *fs_infop;
@@ -1127,6 +1128,14 @@ int do_add_mount(struct vfsmount *newmntif (S_ISLNK (newmnt->mnt_root->d_inode->i_mode)) goto unlock; +/* Unions couldn't be writable if the filesystem +* doesn't know about whiteouts */ + err = -ENOTSUPP ; +if ((mnt_flags &MNT_UNION ) && + !(newmnt->mnt_sb->s_flags &MS_RDONLY ) && + !(newmnt->mnt_sb->s_type->fs_flags &FS_WHT )) + goto unlock; + newmnt->mnt_flags = mnt_flags;if ((err =graft_tree (newmnt, nd))) goto unlock;
@@ -1430,9 +1439,11 @@ long do_mount(char *dev_name, char *dir_ mnt_flags |=MNT_NODIRATIME ;if (flags &MS_RELATIME ) mnt_flags |=MNT_RELATIME ; +if (flags &MS_UNION ) + mnt_flags |=MNT_UNION ; flags &= ~(MS_NOSUID |MS_NOEXEC |MS_NODEV |MS_ACTIVE | -MS_NOATIME |MS_NODIRATIME |MS_RELATIME ); +MS_NOATIME |MS_NODIRATIME |MS_RELATIME |MS_UNION );/* ... and get the mountpoint */ retval =path_lookup (dir_name,LOOKUP_FOLLOW , &nd);
include/linux/fs.h +2 -0
@@ -93,6 +93,7 @@ extern int dir_notify_enable;#define FS_REQUIRES_DEV 1#define FS_BINARY_MOUNTDATA 2#define FS_HAS_SUBTYPE 4 +#define FS_WHT 8#define FS_REVAL_DOT 16384/* Check the paths ".", ".." for staleness */ #define FS_RENAME_DOES_D_MOVE 32768/* FS will handle d_move() * during rename() internally.
@@ -109,6 +110,7 @@ extern int dir_notify_enable;#define MS_REMOUNT 32/* Alter flags of a mounted FS */ #define MS_MANDLOCK 64/* Allow mandatory locks on an FS */ #define MS_DIRSYNC 128/* Directory modifications are synchronous */ +#define MS_UNION 256/* Union mount */ #define MS_NOATIME 1024/* Do not update access times. */ #define MS_NODIRATIME 2048/* Do not update directory access times */ #define MS_BIND 4096
include/linux/mount.h +1 -0
@@ -34,6 +34,7 @@ struct mnt_namespace;#define MNT_SHARED 0x1000/* if the vfsmount is a shared mount */ #define MNT_UNBINDABLE 0x2000/* if the vfsmount is a unbindable mount */ #define MNT_PNODE_MASK 0x3000/* propogation flag mask */ +#define MNT_UNION 0x4000/* if the vfsmount is a union mount */ struct vfsmount {struct list_head mnt_hash;
RFC PATCH 3/15 Add the whiteout file type
In reply to Bharata B RaoFrom: Jan Blunck <[email protected]>
Subject: Add the whiteout file type
A white-out stops the VFS from further lookups of the white-outs name and returns -ENOENT. This is the same behaviour as if the filename isn't found. This can be used in combination with union mounts to virtually delete (white-out) files by creating a file with this file type.
--- include/linux/stat.h | 2 ++ 1 files changed, 2 insertions(+)
include/linux/stat.h +2 -0
@@ -10,6 +10,7 @@#if defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ <2 )#define S_IFMT 00170000 +#define S_IFWHT 0160000/* whiteout */ #define S_IFSOCK 0140000#define S_IFLNK 0120000#define S_IFREG 0100000
@@ -28,6 +29,7 @@#define S_ISBLK (m) (((m) & S_IFMT) == S_IFBLK)#define S_ISFIFO (m) (((m) & S_IFMT) == S_IFIFO)#define S_ISSOCK (m) (((m) & S_IFMT) == S_IFSOCK) +#define S_ISWHT (m) (((m) & S_IFMT) == S_IFWHT)#define S_IRWXU 00700#define S_IRUSR 00400
RFC PATCH 4/15 Add config options for union mount
In reply to Bharata B RaoFrom: Jan Blunck <[email protected]>
Subject: Add config options for union mount
Introduces two new config options for union mount:
CONFIG_UNION_MOUNT - Enables union mount CONFIG_UNION_MOUNT_DEBUG - Enables debugging support for union mount.
Also adds debugging routines.
FIXME: this needs some work. printk'ing isn't the right method for getting good debugging output.
--- fs/Kconfig | 16 +++++++++ include/linux/union_debug.h | 76 ++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 92 insertions(+)
fs/Kconfig +16 -0
@@ -551,6 +551,22 @@ config INOTIFY_USER If unsure, say Y. +config UNION_MOUNT + bool "Union mount support (EXPERIMENTAL)" + depends on EXPERIMENTAL + ---help--- + If you say Y here, you will be able to mount file systems as + union mount stacks. This is a VFS based implementation and + should work with all file systems. If unsure, say N. + +config UNION_MOUNT_DEBUG + bool "Union mount debugging output" + depends on UNION_MOUNT + ---help--- + If you say Y here, the union mount debugging code will be + compiled in. You have activate the appropriate UNION_MOUNT_DEBUG + flags in <file:include/linux/union.h>, too. + config QUOTA bool "Quota support" help
include/linux/union_debug.h new file +76 -0
76 lines not on this page · raw message
RFC PATCH 5/15 Introduce union stack
In reply to Bharata B RaoFrom: Jan Blunck <[email protected]>
Subject: Introduce union stack.
Adds union stack infrastructure to the dentry structure and provides locking routines to walk the union stack.
--- fs/Makefile | 2 fs/dcache.c | 5 fs/union.c | 53 +++++++++ include/linux/dcache.h | 11 + include/linux/dcache_union.h | 243 +++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 314 insertions(+)
fs/Makefile +2 -0
@@ -49,6 +49,8 @@ obj-$(CONFIG_FS_POSIX_ACL) += posix_acl. obj-$(CONFIG_NFS_COMMON) += nfs_common/ obj-$(CONFIG_GENERIC_ACL) += generic_acl.o +obj-$(CONFIG_UNION_MOUNT) += union.o + obj-$(CONFIG_QUOTA) += dquot.o obj-$(CONFIG_QFMT_V1) += quota_v1.o obj-$(CONFIG_QFMT_V2) += quota_v2.o
fs/dcache.c +5 -0
@@ -936,6 +936,11 @@ struct dentry *d_alloc(struct dentry * p#ifdef CONFIG_PROFILING dentry->d_cookie =NULL ;#endif +#ifdef CONFIG_UNION_MOUNT + dentry->d_overlaid =NULL ; + dentry->d_topmost =NULL ; + dentry->d_union =NULL ; +#endif INIT_HLIST_NODE (&dentry->d_hash);INIT_LIST_HEAD (&dentry->d_lru);INIT_LIST_HEAD (&dentry->d_subdirs);
fs/union.c new file +53 -0
53 lines not on this page · raw message
include/linux/dcache.h +11 -0
@@ -93,6 +93,12 @@ struct dentry {struct dentry *d_parent;/* parent directory */ struct qstr d_name; +#ifdef CONFIG_UNION_MOUNT +struct dentry *d_overlaid;/* overlaid directory */ +struct dentry *d_topmost;/* topmost directory */ +struct union_info *d_union;/* union directory info */ +#endif +struct list_head d_lru;/* LRU list */ /* * d_child and d_rcu can share memory
@@ -325,6 +331,11 @@ static inline struct dentry *dget(structreturn dentry; } +/* +* Reference counting for union mounts +*/ +#include <linux/dcache_union.h> +extern struct dentry *dget_locked (struct dentry *);/**
include/linux/dcache_union.h new file +243 -0
243 lines not on this page · raw message
Re: [RFC][PATCH 5/15] Introduce union stack
Quoted 72 lines · Bharata B Rao
From: Jan Blunck <[email protected]>
Subject: Introduce union stack.Adds union stack infrastructure to the dentry structure and provides locking routines to walk the union stack.
Signed-off-by Jan Blunck <[email protected]>Signed-off-by Bharata B Rao <[email protected]>--- fs/Makefile | 2 fs/dcache.c | 5 fs/union.c | 53 +++++++++ include/linux/dcache.h | 11 + include/linux/dcache_union.h | 243 +++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 314 insertions(+)fs/Makefile +2
-0@@ -49,6 +49,8 @@ obj-$(CONFIG_FS_POSIX_ACL) += posix_acl. obj-$(CONFIG_NFS_COMMON) += nfs_common/ obj-$(CONFIG_GENERIC_ACL) += generic_acl.o +obj-$(CONFIG_UNION_MOUNT) += union.o + obj-$(CONFIG_QUOTA) += dquot.o obj-$(CONFIG_QFMT_V1) += quota_v1.o obj-$(CONFIG_QFMT_V2) += quota_v2.ofs/dcache.c +5
-0@@ -936,6 +936,11 @@ struct dentry *d_alloc(struct dentry * p#ifdef CONFIG_PROFILING dentry->d_cookie =NULL ;#endif +#ifdef CONFIG_UNION_MOUNT + dentry->d_overlaid =NULL ; + dentry->d_topmost =NULL ; + dentry->d_union =NULL ; +#endif INIT_HLIST_NODE (&dentry->d_hash);INIT_LIST_HEAD (&dentry->d_lru);INIT_LIST_HEAD (&dentry->d_subdirs);fs/union.c new file +28
-0@@ -0,0 +1,53 @@ +/* +* VFS based union mount for Linux +* +* Copyright ? 2004-2007 IBM Corporation +* Author(s): Jan Blunck ([email protected]) +* +* This program is free software; you can redistribute it and/or modify it +* under the terms of the GNU General Public License as published by the Free +* Software Foundation; either version 2 of the License, or (at your option) +* any later version. +*/ + +#include <linux/fs.h> + +struct union_info *union_alloc (void ) +{ +struct union_info *info; + + info =kmalloc (sizeof (*info),GFP_ATOMIC ); +if (!info) +return NULL ; + +mutex_init (&info->u_mutex); +mutex_lock (&info->u_mutex); +atomic_set (&info->u_count,1 ); +UM_DEBUG_LOCK ("allocate union %p\n" , info); +return info; +}
fs/union.c line 38
Quoted 10 lines
fs/union.c new file +10
-0+ +struct union_info *union_get (struct union_info *info) +{ +BUG_ON (!info); +BUG_ON (!atomic_read (&info->u_count)); +atomic_inc (&info->u_count); +UM_DEBUG_LOCK ("get union %p (count=%d)\n" , info, +atomic_read (&info->u_count)); +return info; +}
The locking here needs to be laid out. It looks like union_get() needs to be called under union_lock(), while union_get2() (horrible name) grabs that lock itself, and returns with the lock held?
Similarly union_put clearly needs to be called under union_lock(), so that should be commented here.
Quoted 165 lines
fs/union.c new file +14
-0+void union_put (struct union_info *info) +{ +BUG_ON (!info); +UM_DEBUG_LOCK ("put union %p (count=%d)\n" , info, +atomic_read (&info->u_count)); +atomic_dec (&info->u_count); + +if (!atomic_read (&info->u_count)) { +UM_DEBUG_LOCK ("free union %p\n" , info); +kfree (info); + } + +return ; +}include/linux/dcache.h +11
-0@@ -93,6 +93,12 @@ struct dentry {struct dentry *d_parent;/* parent directory */ struct qstr d_name; +#ifdef CONFIG_UNION_MOUNT +struct dentry *d_overlaid;/* overlaid directory */ +struct dentry *d_topmost;/* topmost directory */ +struct union_info *d_union;/* union directory info */ +#endif +struct list_head d_lru;/* LRU list */ /* * d_child and d_rcu can share memory @@ -325,6 +331,11 @@ static inline struct dentry *dget(structreturn dentry; } +/* +* Reference counting for union mounts +*/ +#include <linux/dcache_union.h> +extern struct dentry *dget_locked (struct dentry *);/** include/linux/dcache_union.h new file +121
-0@@ -0,0 +1,243 @@ +/* +* VFS based union mount for Linux +* +* Copyright ? 2004-2007 IBM Corporation +* Author(s): Jan Blunck ([email protected]) +* +* This program is free software; you can redistribute it and/or modify it +* under the terms of the GNU General Public License as published by the Free +* Software Foundation; either version 2 of the License, or (at your option) +* any later version. +* +*/ +#ifndef __LINUX_DCACHE_UNION_H +#define __LINUX_DCACHE_UNION_H +#ifdef __KERNEL__ + +#include <linux/union_debug.h> +#include <linux/fs_struct.h> +#include <asm/atomic.h> +#include <asm/semaphore.h> + +#ifdef CONFIG_UNION_MOUNT + +/* + * This is the union info object, that describes general information about this + * union directory + * + * u_mutex protects the union stack against modification. You can reach it + * through the d_union field in struct dentry. Hold it when you are walking + * or modifing the union stack ! + * + * NOTE: Read the remark for union_trylock() below! + */ +struct union_info { + atomic_t u_count; + struct mutex u_mutex; +}; + +/* allocate/de-allocate */ +extern struct union_info *union_alloc(void); +extern struct union_info *union_get(struct union_info *); +extern void union_put(struct union_info *); + +/* + * These are the functions for locking a dentrys union. When one + * want to acquire a denties union lock, use: + * + * - union_lock() when you can sleep, + * - union_lock_spinlock() when you are holding a spinlock (that + * you CAN savely give up and reacquire again) + * - union_lock_readlock() when you are holding a readlock (that + * you CAN savely give up and reacquire again) + * + * Otherwise get the union lock early before you enter your + * "no sleeping here" code. + */ +static inline void __union_lock(struct union_info *uinfo) +{ + BUG_ON(!atomic_read(&uinfo->u_count)); + mutex_lock(&uinfo->u_mutex); +} + +static inline void union_lock(struct dentry *dentry) +{ + if (unlikely(dentry && dentry->d_union)) { + struct union_info *ui = dentry->d_union; + + UM_DEBUG_LOCK("\"%s\" locking %p (count=%d)\n", + dentry->d_name.name, ui, + atomic_read(&ui->u_count)); + __union_lock(dentry->d_union); + } +} + +static inline void __union_unlock(struct union_info *uinfo) +{ + BUG_ON(!atomic_read(&uinfo->u_count)); + mutex_unlock(&uinfo->u_mutex); +} + +static inline void union_unlock(struct dentry *dentry) +{ + if (unlikely(dentry && dentry->d_union)) { + struct union_info *ui = dentry->d_union; + + UM_DEBUG_LOCK("\"%s\" unlocking %p (count=%d)\n", + dentry->d_name.name, ui, + atomic_read(&ui->u_count)); + __union_unlock(dentry->d_union); + } +} + +/* + * Two helpers for namespace.c + * + * FIXME: clean this up to get it right + */ +static inline struct union_info *union_alloc2(struct dentry * dentry) +{ + struct union_info *uinfo; + + spin_lock(&dentry->d_lock); + if (!dentry->d_union) { + dentry->d_union = union_alloc(); + uinfo = union_get(dentry->d_union); + spin_unlock(&dentry->d_lock); + } else { + uinfo = union_get(dentry->d_union); + spin_unlock(&dentry->d_lock); + union_lock(dentry); + } + + return uinfo; +} + +static inline struct union_info *union_get2(struct dentry * dentry) +{ + struct union_info *uinfo; + + union_lock(dentry); + uinfo = union_get(dentry->d_union);
include/linux/dcache_union.h line 131
Quoted 10 lines
include/linux/dcache_union.h new file +10
-0+return uinfo; +} + +static inline void union_release (struct union_info *uinfo) +{ +if (!uinfo) +return ; + +mutex_unlock (&uinfo->u_mutex); +union_put (uinfo);
Is it safe to do this - releasing the lock before doing the put?
Quoted 116 lines
include/linux/dcache_union.h new file +112
-1+} + +/* +* Immediately return ZERO if the lock is contended, NON-ZERO if it's acquired. +*/ +static inline int union_trylock (struct dentry *dentry) +{ +int locked =1 ; + +if (unlikely (dentry && dentry->d_union)) { +UM_DEBUG_LOCK ("\"%s\" try locking %p (count=%d)\n" , + dentry->d_name.name, dentry->d_union, +atomic_read (&dentry->d_union->u_count)); +BUG_ON (!atomic_read (&dentry->d_union->u_count)); + locked =mutex_trylock (&dentry->d_union->u_mutex); +UM_DEBUG_LOCK ("\"%s\" trylock %p %s\n" , dentry->d_name.name, + dentry->d_union, + locked ?"succeeded" :"failed" ); + } +return (locked ?1 :0 ); +} + +/* +* The following functions are locking helpers to guarantee the locking order +* in some situations. +*/ + +static inline void union_lock_spinlock (struct dentry *dentry,spinlock_t *lock) +{ +while (!union_trylock (dentry)) { +spin_unlock (lock); +cpu_relax (); +spin_lock (lock); + } +} + +static inline void union_lock_readlock (struct dentry *dentry,rwlock_t *lock) +{ +while (!union_trylock (dentry)) { +read_unlock (lock); +cpu_relax (); +read_lock (lock); + } +} + +/* +* This is a *I can't get no sleep* helper which is called when we try +* to access the struct fs_struct *fs field of a struct task_struct. +* +* Yes, this is possibly starving but we have to change root, altroot +* or pwd in the frequency of this while loop. Don't think that this +* happens really often ;) +* +* This is called while holding the rwlock_t fs->lock +* +* TODO: Unlocking side of union_lock_fs() needs 3 union_unlock()s. +* May be introduce union_unlock_fs(). +* +* FIXME: This routine is used when the caller wants to dget one or +* more of fs->[root, altroot, pwd]. When the caller doesn't want to +* dget _all_ of these, it is strictly not necessary to get union_locks +* on all of these. Check. +*/ +static inline void union_lock_fs (struct fs_struct *fs) +{ +int locked; + +while (fs) { + locked =union_trylock (fs->root); +if (!locked) + goto loop1; + locked =union_trylock (fs->altroot); +if (!locked) + goto loop2; + locked =union_trylock (fs->pwd); +if (!locked) + goto loop3; +break ; + loop3: +union_unlock (fs->altroot); + loop2: +union_unlock (fs->root); + loop1: +read_unlock (&fs->lock); +UM_DEBUG_LOCK ("Failed to get all semaphores in fs_struct!\n" ); +cpu_relax (); +read_lock (&fs->lock); +continue ; + } +BUG_ON (!fs); +return ; +} + +#define IS_UNION (dentry) ((dentry)->d_overlaid || (dentry)->d_topmost || \ + (dentry)->d_overlaid) + +#else /* CONFIG_UNION_MOUNT */ + +#define union_lock(dentry) do { /* empty */ } while (0) +#define union_trylock (dentry) ({ (1); }) +#define union_unlock(dentry) do { /* empty */ } while (0) +#define union_lock_spinlock(dentry, lock) do { /* empty */ } while (0) +#define union_lock_readlock(dentry, lock) do { /* empty */ } while (0) +#define union_lock_fs(fs) do { /* empty */ } while (0) +#define IS_UNION (dentry) ({ (0); }) +#define union_alloc2 (x) ({ BUG(); (0); }) +#define union_get2 (x) ({ BUG(); (0); }) +#define union_release (x) do { BUG(); } while (0) + +#endif /* CONFIG_UNION_MOUNT */ +#endif /* __KERNEL__ */ +#endif /* __LINUX_DCACHE_UNION_H */ -To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC][PATCH 5/15] Introduce union stack
Quoted 88 lines · Serge E. Hallyn
Quoted 72 lines · Bharata B Rao
From: Jan Blunck <[email protected]>
Subject: Introduce union stack.Adds union stack infrastructure to the dentry structure and provides locking routines to walk the union stack.
Signed-off-by Jan Blunck <[email protected]>Signed-off-by Bharata B Rao <[email protected]>--- fs/Makefile | 2 fs/dcache.c | 5 fs/union.c | 53 +++++++++ include/linux/dcache.h | 11 + include/linux/dcache_union.h | 243 +++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 314 insertions(+)fs/Makefile +2
-0@@ -49,6 +49,8 @@ obj-$(CONFIG_FS_POSIX_ACL) += posix_acl. obj-$(CONFIG_NFS_COMMON) += nfs_common/ obj-$(CONFIG_GENERIC_ACL) += generic_acl.o +obj-$(CONFIG_UNION_MOUNT) += union.o + obj-$(CONFIG_QUOTA) += dquot.o obj-$(CONFIG_QFMT_V1) += quota_v1.o obj-$(CONFIG_QFMT_V2) += quota_v2.ofs/dcache.c +5
-0@@ -936,6 +936,11 @@ struct dentry *d_alloc(struct dentry * p#ifdef CONFIG_PROFILING dentry->d_cookie =NULL ;#endif +#ifdef CONFIG_UNION_MOUNT + dentry->d_overlaid =NULL ; + dentry->d_topmost =NULL ; + dentry->d_union =NULL ; +#endif INIT_HLIST_NODE (&dentry->d_hash);INIT_LIST_HEAD (&dentry->d_lru);INIT_LIST_HEAD (&dentry->d_subdirs);fs/union.c new file +28
-0@@ -0,0 +1,53 @@ +/* +* VFS based union mount for Linux +* +* Copyright ? 2004-2007 IBM Corporation +* Author(s): Jan Blunck ([email protected]) +* +* This program is free software; you can redistribute it and/or modify it +* under the terms of the GNU General Public License as published by the Free +* Software Foundation; either version 2 of the License, or (at your option) +* any later version. +*/ + +#include <linux/fs.h> + +struct union_info *union_alloc (void ) +{ +struct union_info *info; + + info =kmalloc (sizeof (*info),GFP_ATOMIC ); +if (!info) +return NULL ; + +mutex_init (&info->u_mutex); +mutex_lock (&info->u_mutex); +atomic_set (&info->u_count,1 ); +UM_DEBUG_LOCK ("allocate union %p\n" , info); +return info; +}fs/union.c line 38
Quoted 10 lines
fs/union.c new file +10
-0+ +struct union_info *union_get (struct union_info *info) +{ +BUG_ON (!info); +BUG_ON (!atomic_read (&info->u_count)); +atomic_inc (&info->u_count); +UM_DEBUG_LOCK ("get union %p (count=%d)\n" , info, +atomic_read (&info->u_count)); +return info; +}The locking here needs to be laid out. It looks like
union_get()needs to be called underunion_lock(), whileunion_get2()(horrible name) grabs that lock itself, and returns with the lock held?
Quoted 2 lines
Similarly
union_putclearly needs to be called underunion_lock(), so that should be commented here.
Agreed. This whole thing needs a fresh look and we need to establish some rules and consistency here. After you pointed out this, even union_alloc2() is looking suspect to me. Same goes for union_release() also. Thanks for pointing this out.
Regards,
Bharata.
RFC PATCH 6/15 Union-mount dentry reference counting
In reply to Bharata B RaoFrom: Jan Blunck <[email protected]>
Subject: Union-mount dentry reference counting
dget is modified to walk the union stack taking reference on every dentry that is part of the union stack. This is necessary to ensure that parts of union stack don't go away from under us. Since dget() takes a mutex for walking the stack, dget can now sleep.
dput also walks the union stack and releases references to all the dentries that are part of the union.
Since dget() can now sleep, make sure that dget() doesn't go to sleep with any spinlocks held while it tries to get the mutex.
--- fs/dcache.c | 35 ++++-- fs/dnotify.c | 5 fs/inotify.c | 8 + fs/namei.c | 42 +++++-- fs/namespace.c | 12 +- fs/proc/base.c | 17 ++ fs/union.c | 249 +++++++++++++++++++++++++++++++++++++++++++ include/linux/dcache.h | 64 ++++++++++- include/linux/dcache_union.h | 65 +++++++++++ kernel/auditsc.c | 4 kernel/cpuset.c | 4 kernel/fork.c | 10 + net/unix/af_unix.c | 5 13 files changed, 484 insertions(+), 36 deletions(-)
fs/dcache.c +28 -7
@@ -171,8 +171,7 @@ static struct dentry *d_kill(struct dent* * no dcache lock, please. */ - -void dput (struct dentry *dentry) +void __dput_single (struct dentry *dentry) {if (!dentry)return ;
@@ -190,6 +189,13 @@ repeat:return ; } +if (!__dput_single_destroy_union (dentry)) { +atomic_inc (&dentry->d_count); +spin_unlock (&dentry->d_lock); +spin_unlock (&dcache_lock); + goto repeat; + } +/* * AV: ->d_delete() is _NOT_ allowed to block now. */
@@ -285,6 +291,15 @@ int d_invalidate(struct dentry * dentry)static inline struct dentry *__dget_locked (struct dentry *dentry) { +/* +* TODO: We come here with dcache_lock held and can't +* afford to sleep now to acquire the union_lock. We should +* change all the callers to acquire union_lock first using +* the union_lock_spinlock() helper. Until that is done, +* BUG() here. +*/ +BUG_ON (IS_UNION (dentry)); +atomic_inc (&dentry->d_count);if (!list_empty (&dentry->d_lru)) { dentry_stat.nr_unused--;
@@ -392,7 +407,7 @@ static void prune_one_dentry(struct dent__d_drop (dentry); dentry =d_kill (dentry);if (!prune_parents) { -dput (dentry); +__dput_single (dentry);spin_lock (&dcache_lock);return ; }
@@ -947,7 +962,7 @@ struct dentry *d_alloc(struct dentry * pINIT_LIST_HEAD (&dentry->d_alias);if (parent) { - dentry->d_parent =dget (parent); + dentry->d_parent =__dget_single (parent); dentry->d_sb = parent->d_sb; }else {INIT_LIST_HEAD (&dentry->d_u.d_child);
@@ -1908,8 +1923,10 @@ char *d_path(struct dentry *dentry, strureturn dentry->d_op->d_dname (dentry, buf, buflen);read_lock (¤t->fs->lock); +union_lock_readlock (current->fs->root, ¤t->fs->lock); rootmnt =mntget (current->fs->rootmnt); - root =dget (current->fs->root); + root =__dget (current->fs->root); +union_unlock (current->fs->root);read_unlock (¤t->fs->lock); res =__d_path (dentry, vfsmnt, root, rootmnt, buf, buflen,0 );dput (root);
@@ -1967,10 +1984,14 @@ asmlinkage long sys_getcwd(char __user *return -ENOMEM ;read_lock (¤t->fs->lock); +union_lock_fs (current->fs); pwdmnt =mntget (current->fs->pwdmnt); - pwd =dget (current->fs->pwd); + pwd =__dget (current->fs->pwd); rootmnt =mntget (current->fs->rootmnt); - root =dget (current->fs->root); + root =__dget (current->fs->root); +union_unlock (current->fs->pwd); +union_unlock (current->fs->altroot); +union_unlock (current->fs->root);read_unlock (¤t->fs->lock); cwd =__d_path (pwd, pwdmnt, root, rootmnt, page,PAGE_SIZE ,1 );
fs/dnotify.c +4 -1
@@ -161,13 +161,16 @@ void dnotify_parent(struct dentry *dentrreturn ;spin_lock (&dentry->d_lock); +union_lock_spinlock (dentry->d_parent, &dentry->d_lock); parent = dentry->d_parent;if (parent->d_inode->i_dnotify_mask & event) { -dget (parent); +__dget (parent); +union_unlock (parent);spin_unlock (&dentry->d_lock);__inode_dir_notify (parent->d_inode, event);dput (parent); }else { +union_unlock (parent);spin_unlock (&dentry->d_lock); } }
fs/inotify.c +6 -2
@@ -325,17 +325,21 @@ void inotify_dentry_parent_queue_event(sreturn ;spin_lock (&dentry->d_lock); +union_lock_spinlock (dentry->d_parent, &dentry->d_lock); parent = dentry->d_parent; inode = parent->d_inode;if (inotify_inode_watched (inode)) { -dget (parent); +__dget (parent); +union_unlock (parent);spin_unlock (&dentry->d_lock);inotify_inode_queue_event (inode, mask, cookie, name, dentry->d_inode);dput (parent); - }else + }else { +union_unlock (parent);spin_unlock (&dentry->d_lock); + } }EXPORT_SYMBOL_GPL (inotify_dentry_parent_queue_event);
fs/namei.c +33 -9
111 lines not on this page · raw message
fs/namespace.c +9 -3
@@ -1610,12 +1610,14 @@ void set_fs_root(struct fs_struct *fs, s {struct dentry *old_root;struct vfsmount *old_rootmnt; +union_lock (dentry);write_lock (&fs->lock); old_root = fs->root; old_rootmnt = fs->rootmnt; fs->rootmnt =mntget (mnt); - fs->root =dget (dentry); + fs->root =__dget (dentry);write_unlock (&fs->lock); +union_unlock (dentry);if (old_root) {dput (old_root);mntput (old_rootmnt);
@@ -1632,12 +1634,14 @@ void set_fs_pwd(struct fs_struct *fs, ststruct dentry *old_pwd;struct vfsmount *old_pwdmnt; +union_lock (dentry);write_lock (&fs->lock); old_pwd = fs->pwd; old_pwdmnt = fs->pwdmnt; fs->pwdmnt =mntget (mnt); - fs->pwd =dget (dentry); + fs->pwd =__dget (dentry);write_unlock (&fs->lock); +union_unlock (dentry);if (old_pwd) {dput (old_pwd);
@@ -1726,8 +1730,10 @@ asmlinkage long sys_pivot_root(const cha }read_lock (¤t->fs->lock); +union_lock_readlock (current->fs->root, ¤t->fs->lock); user_nd.mnt =mntget (current->fs->rootmnt); - user_nd.dentry =dget (current->fs->root); + user_nd.dentry =__dget (current->fs->root); +union_unlock (current->fs->root);read_unlock (¤t->fs->lock);down_write (&namespace_sem);mutex_lock (&old_nd.dentry->d_inode->i_mutex);
fs/proc/base.c +14 -3
@@ -203,8 +203,10 @@ static int proc_cwd_link(struct inode *i }if (fs) {read_lock (&fs->lock); +union_lock_readlock (fs->pwd, &fs->lock); *mnt =mntget (fs->pwdmnt); - *dentry =dget (fs->pwd); + *dentry =__dget (fs->pwd); +union_unlock (fs->pwd);read_unlock (&fs->lock); result =0 ;put_fs_struct (fs);
@@ -224,8 +226,10 @@ static int proc_root_link(struct inode * }if (fs) {read_lock (&fs->lock); +union_lock_readlock (fs->root, &fs->lock); *mnt =mntget (fs->rootmnt); - *dentry =dget (fs->root); + *dentry =__dget (fs->root); +union_unlock (fs->root);read_unlock (&fs->lock); result =0 ;put_fs_struct (fs);
@@ -1252,19 +1256,26 @@ static int proc_fd_info(struct inode *in* We are not taking a ref to the file structure, so we must * hold ->file_lock. */ +repeat:spin_lock (&files->file_lock); file =fcheck_files (files, fd);if (file) { +if (!union_trylock (file->f_path.dentry)) { +spin_unlock (&files->file_lock); +cpu_relax (); + goto repeat; + }if (mnt) *mnt =mntget (file->f_path.mnt);if (dentry) - *dentry =dget (file->f_path.dentry); + *dentry =__dget (file->f_path.dentry);if (info)snprintf (info,PROC_FDINFO_MAX ,"pos:\t%lli\n" "flags:\t0%o\n" , (long long ) file->f_pos, file->f_flags); +union_unlock (file->f_path.dentry);spin_unlock (&files->file_lock);put_files_struct (files);return 0 ;
fs/union.c +249 -0
258 lines not on this page · raw message
include/linux/dcache.h +60 -4
93 lines not on this page · raw message
include/linux/dcache_union.h +64 -1
104 lines not on this page · raw message
kernel/auditsc.c +3 -1
@@ -1229,8 +1229,10 @@ void __audit_getname(const char *name) ++context->name_count;if (!context->pwd) {read_lock (¤t->fs->lock); - context->pwd =dget (current->fs->pwd); +union_lock_readlock (current->fs->pwd, ¤t->fs->lock); + context->pwd =__dget (current->fs->pwd); context->pwdmnt =mntget (current->fs->pwdmnt); +union_unlock (current->fs->pwd);read_unlock (¤t->fs->lock); }
kernel/cpuset.c +3 -1
@@ -1894,9 +1894,11 @@ static int cpuset_rmdir(struct inode *unmutex_lock (&callback_mutex);set_bit (CS_REMOVED , &cs->flags);list_del (&cs->sibling);/* delete my sibling from parent->children */ +union_lock (cs->dentry);spin_lock (&cs->dentry->d_lock); - d =dget (cs->dentry); + d =__dget (cs->dentry); cs->dentry =NULL ; +union_unlock (d);spin_unlock (&d->d_lock);cpuset_d_remove_dir (d);dput (d);
kernel/fork.c +7 -3
@@ -581,17 +581,21 @@ static inline struct fs_struct *__copy_frwlock_init (&fs->lock); fs->umask = old->umask;read_lock (&old->lock); +union_lock_fs (old); fs->rootmnt =mntget (old->rootmnt); - fs->root =dget (old->root); + fs->root =__dget (old->root); fs->pwdmnt =mntget (old->pwdmnt); - fs->pwd =dget (old->pwd); + fs->pwd =__dget (old->pwd);if (old->altroot) { fs->altrootmnt =mntget (old->altrootmnt); - fs->altroot =dget (old->altroot); + fs->altroot =__dget (old->altroot); }else { fs->altrootmnt =NULL ; fs->altroot =NULL ; } +union_unlock (old->pwd); +union_unlock (old->altroot); +union_unlock (old->root);read_unlock (&old->lock); }return fs;
net/unix/af_unix.c +4 -1
@@ -1082,8 +1082,11 @@ restart: newu->addr = otheru->addr; }if (otheru->dentry) { - newu->dentry =dget (otheru->dentry); +/* Is this safe here? I don't know ... */ +union_lock_spinlock (otheru->dentry, &otheru->lock); + newu->dentry =__dget (otheru->dentry); newu->mnt =mntget (otheru->mnt); +union_unlock (otheru->dentry); }/* Set credentials */
RFC PATCH 7/15 Union-mount mounting
In reply to Bharata B RaoFrom: Jan Blunck <[email protected]>
Subject: Union-mount mounting
Adds union mount support to mount() and umount() system calls. Sets up the union stack during mount and destroys it during unmount.
TODO: bind and move mounts aren't yet supported with union mounts.
--- fs/namespace.c | 79 +++++++++++++++++++++++++++++++++++++++++++++----- fs/union.c | 65 +++++++++++++++++++++++++++++++++++++++++ include/linux/fs.h | 3 + include/linux/union.h | 33 ++++++++++++++++++++ 4 files changed, 173 insertions(+), 7 deletions(-)
fs/namespace.c +72 -7
@@ -166,7 +166,7 @@ void mnt_set_mountpoint(struct vfsmount struct vfsmount *child_mnt) { child_mnt->mnt_parent =mntget (mnt); - child_mnt->mnt_mountpoint =dget (dentry); + child_mnt->mnt_mountpoint =__dget (dentry); dentry->d_mounted++; }
@@ -234,6 +234,10 @@ static struct vfsmount *clone_mnt(structstruct vfsmount *mnt =alloc_vfsmnt (old->mnt_devname);if (mnt) { +/* +* As of now, cloning of union mounted mnt isn't permitted. +*/ +BUG_ON (mnt->mnt_flags &MNT_UNION ); mnt->mnt_flags = old->mnt_flags;atomic_inc (&sb->s_active); mnt->mnt_sb = sb;
@@ -522,16 +526,20 @@ void release_mounts(struct list_head *he mnt = list_entry(head->next, struct vfsmount, mnt_hash);list_del_init (&mnt->mnt_hash);if (mnt->mnt_parent != mnt) { -struct dentry *dentry; -struct vfsmount *m; +struct path old_nd;spin_lock (&vfsmount_lock); - dentry = mnt->mnt_mountpoint; - m = mnt->mnt_parent; + old_nd.dentry = mnt->mnt_mountpoint; + old_nd.mnt = mnt->mnt_parent; mnt->mnt_mountpoint = mnt->mnt_root; mnt->mnt_parent = mnt; +detach_mnt_union (mnt, &old_nd);spin_unlock (&vfsmount_lock); -dput (dentry); -mntput (m); +if (mnt->mnt_flags &MNT_UNION ) { +UM_DEBUG ("shrink the mountpoint's dcache\n" ); +shrink_dcache_sb (old_nd.dentry->d_sb); + } +__dput (old_nd.dentry); +mntput (old_nd.mnt); }mntput (mnt); }
@@ -564,6 +572,7 @@ static int do_umount(struct vfsmount *mnstruct super_block *sb = mnt->mnt_sb;int retval;LIST_HEAD (umount_list); +struct union_info *uinfo =NULL ; retval =security_sb_umount (mnt, flags);if (retval)
@@ -628,6 +637,9 @@ static int do_umount(struct vfsmount *mn }down_write (&namespace_sem); +/* get the union lock which is released after release_mounts() */ +if (mnt->mnt_flags &MNT_UNION ) + uinfo =union_get2 (mnt->mnt_root);spin_lock (&vfsmount_lock); event++;
@@ -642,6 +654,8 @@ static int do_umount(struct vfsmount *mnsecurity_sb_umount_busy (mnt);up_write (&namespace_sem);release_mounts (&umount_list); +if (uinfo) +union_release (uinfo);return retval; }
@@ -857,6 +871,7 @@ static int attach_recursive_mnt(struct vtouch_mnt_namespace (current->nsproxy->mnt_ns); }else {mnt_set_mountpoint (dest_mnt, dest_dentry, source_mnt); +attach_mnt_union (source_mnt, nd);commit_tree (source_mnt); }
@@ -871,6 +886,8 @@ static int attach_recursive_mnt(struct vstatic int graft_tree (struct vfsmount *mnt,struct nameidata *nd) {int err; +struct union_info *uinfo =NULL ; +if (mnt->mnt_sb->s_flags &MS_NOUSER )return -EINVAL ;
@@ -878,6 +895,9 @@ static int graft_tree(struct vfsmount *m S_ISDIR(mnt->mnt_root->d_inode->i_mode))return -ENOTDIR ; +if (mnt->mnt_flags &MNT_UNION ) + uinfo =union_alloc2 (nd->dentry); + err = -ENOENT ;mutex_lock (&nd->dentry->d_inode->i_mutex);if (IS_DEADDIR (nd->dentry->d_inode))
@@ -894,6 +914,8 @@ out_unlock:mutex_unlock (&nd->dentry->d_inode->i_mutex);if (!err)security_sb_post_addmount (mnt, nd); +if (uinfo) +union_release (uinfo);return err; }
@@ -909,6 +931,12 @@ static int do_change_type(struct nameidaif (nd->dentry != nd->mnt->mnt_root)return -EINVAL ; +/* +* Don't change the type of union mounts +*/ +if (nd->mnt->mnt_flags &MNT_UNION ) +return -EINVAL ; +down_write (&namespace_sem);spin_lock (&vfsmount_lock);for (m = mnt; m; m = (recurse ?next_mnt (m, mnt) :NULL ))
@@ -934,6 +962,15 @@ static int do_loopback(struct nameidataif (err)return err; +/* +* bind mounting to or from union mounts is not supported +*/ + err = -EINVAL ; +if (nd->mnt->mnt_flags &MNT_UNION ) + goto out_unlocked; +if (old_nd.mnt->mnt_flags &MNT_UNION ) + goto out_unlocked; +down_write (&namespace_sem); err = -EINVAL ;if (IS_MNT_UNBINDABLE (old_nd.mnt))
@@ -962,6 +999,7 @@ static int do_loopback(struct nameidata out:up_write (&namespace_sem); +out_unlocked:path_release (&old_nd);return err; }
@@ -1019,6 +1057,15 @@ static int do_move_mount(struct nameidatif (err)return err; +/* +* moving to or from a union mount is not supported +*/ + err = -EINVAL ; +if (nd->mnt->mnt_flags &MNT_UNION ) + goto exit; +if (old_nd.mnt->mnt_flags &MNT_UNION ) + goto exit; +down_write (&namespace_sem);while (d_mountpoint (nd->dentry) &&follow_down (&nd->mnt, &nd->dentry)) ;
@@ -1074,6 +1121,7 @@ out:up_write (&namespace_sem);if (!err)path_release (&parent_nd); +exit:path_release (&old_nd);return err; }
@@ -1098,6 +1146,9 @@ static int do_new_mount(struct nameidataif (IS_ERR (mnt))return PTR_ERR (mnt); +UM_DEBUG ("dentry=%s, device=%s\n" , nd->dentry->d_name.name, + mnt->mnt_devname); +return do_add_mount (mnt, nd, mnt_flags,NULL ); }
@@ -1128,6 +1179,12 @@ int do_add_mount(struct vfsmount *newmntif (S_ISLNK (newmnt->mnt_root->d_inode->i_mode)) goto unlock; +/* Unions couldn't include shared mounts */ + err = -EINVAL ; +if ((mnt_flags &MNT_UNION ) && +IS_MNT_SHARED (nd->mnt)) + goto unlock; +/* Unions couldn't be writable if the filesystem * doesn't know about whiteouts */ err = -ENOTSUPP ;
@@ -1146,6 +1203,14 @@ int do_add_mount(struct vfsmount *newmntlist_add_tail (&newmnt->mnt_expire, fslist);spin_unlock (&vfsmount_lock); } + +UM_DEBUG ("mntpoint->d_count=%d/%p\n" , +atomic_read (&nd->dentry->d_count), + &nd->dentry->d_count); +UM_DEBUG ("mntroot->d_count=%d/%p\n" , +atomic_read (&newmnt->mnt_root->d_count), + &newmnt->mnt_root->d_count); +up_write (&namespace_sem);return 0 ;
fs/union.c +65 -0
68 lines not on this page · raw message
include/linux/fs.h +3 -0
@@ -1976,6 +1976,9 @@ static inline ino_t parent_ino(struct de/* kernel/fork.c */ extern int unshare_files (void ); +/* fs/union.c */ +#include <linux/union.h> +/* Transaction based IO helpers */ /*
include/linux/union.h new file +33 -0
@@ -0,0 +1,33 @@ +/* +* VFS based union mount for Linux +* +* Copyright © 2004-2007 IBM Corporation +* Author(s): Jan Blunck ([email protected]) +* +* This program is free software; you can redistribute it and/or modify it +* under the terms of the GNU General Public License as published by the Free +* Software Foundation; either version 2 of the License, or (at your option) +* any later version. +* +*/ +#ifndef __LINUX_UNION_H +#define __LINUX_UNION_H +#ifdef __KERNEL__ + +#ifdef CONFIG_UNION_MOUNT + +#include <linux/fs_struct.h> + +/* namespace stuff used at mount time */ +extern void attach_mnt_union (struct vfsmount *,struct nameidata *); +extern void detach_mnt_union (struct vfsmount *,struct path *); + +#else /* CONFIG_UNION_MOUNT */ + +#define attach_mnt_union(mnt,nd) do { /* empty */ } while (0) +#define detach_mnt_union(mnt,nd) do { /* empty */ } while (0) + +#endif /* CONFIG_UNION_MOUNT */ + +#endif /* __KERNEL __ */ +#endif /* __LINUX_UNION_H */
RFC PATCH 8/15 Union-mount lookup
In reply to Bharata B RaoFrom: Jan Blunck <[email protected]>
Subject: Union-mount lookup
Modifies the vfs lookup routines to work with union mounted directories.
The existing lookup routines generally lookup for a pathname only in the topmost or given directory. The changed versions of the lookup routines search for the pathname in the entire union mounted stack. Also they have been modified to setup the union stack during lookup from dcache cache and from real_lookup().
--- fs/dcache.c | 16 + fs/namei.c | 76 +++++- fs/namespace.c | 35 ++ fs/union.c | 597 +++++++++++++++++++++++++++++++++++++++++++++++++ include/linux/dcache.h | 17 + include/linux/namei.h | 4 include/linux/union.h | 27 ++ 7 files changed, 761 insertions(+), 11 deletions(-)
fs/dcache.c +15 -1
@@ -1278,7 +1278,7 @@ struct dentry * d_lookup(struct dentry *return dentry; } -struct dentry *__d_lookup (struct dentry * parent,struct qstr * name) +struct dentry *__d_lookup_single (struct dentry *parent,struct qstr *name) {unsigned int len = name->len;unsigned int hash = name->hash;
@@ -1363,6 +1363,20 @@ out:return dentry; } +struct dentry *d_lookup_single (struct dentry *parent,struct qstr *name) +{ +struct dentry *dentry; +unsigned long seq; + +do { + seq =read_seqbegin (&rename_lock); + dentry =__d_lookup_single (parent, name); +if (dentry) +break ; + }while (read_seqretry (&rename_lock, seq)); +return dentry; +} +/** * d_validate - verify dentry provided from insecure source * @dentry: The dentry alleged to be valid child of @dparent
fs/namei.c +67 -9
163 lines not on this page · raw message
fs/namespace.c +35 -0
@@ -130,6 +130,41 @@ struct vfsmount *lookup_mnt(struct vfsmoreturn child_mnt; } +/* +* find_mnt - find a vfsmount struct +* @dentry: a dentry +* +* This searches the namespace for a given dentries +* vfsmount struct. This is used by union-mount. +*/ +struct vfsmount *find_mnt (struct dentry *dentry) +{ +struct list_head *tmp; +struct vfsmount *p, *mnt =NULL ; + +down_read (&namespace_sem); +spin_lock (&vfsmount_lock); +if (list_empty (¤t->nsproxy->mnt_ns->list)) { +spin_unlock (&vfsmount_lock); +up_read (&namespace_sem); +return NULL ; + } +list_for_each (tmp, ¤t->nsproxy->mnt_ns->list) { + p = list_entry(tmp, struct vfsmount, mnt_list); +if (dentry->d_sb == p->mnt_sb) { + mnt =mntget (p); +break ; + } + } +spin_unlock (&vfsmount_lock); +up_read (&namespace_sem); + +BUG_ON (!mnt); +// UM_DEBUG_UID("found %s/%p in %s\n", dentry->d_name.name, +// dentry->d_inode, mnt->mnt_devname); +return mnt; +} +static inline int check_mnt (struct vfsmount *mnt) {return mnt->mnt_ns == current->nsproxy->mnt_ns;
fs/union.c +597 -0
600 lines not on this page · raw message
include/linux/dcache.h +16 -1
@@ -294,9 +294,23 @@ extern void d_move(struct dentry *, stru/* appendix may either be NULL or be used for transname suffixes */ extern struct dentry *d_lookup (struct dentry *,struct qstr *); -extern struct dentry *__d_lookup (struct dentry *,struct qstr *); +extern struct dentry *d_lookup_single (struct dentry *,struct qstr *); +extern struct dentry *__d_lookup_single (struct dentry *,struct qstr *);extern struct dentry *d_hash_and_lookup (struct dentry *,struct qstr *); +#ifdef CONFIG_UNION_MOUNT +extern struct dentry *__d_lookup_union (struct dentry *,struct qstr *); +#endif + +static inline struct dentry *__d_lookup (struct dentry *parent,struct qstr *name) +{ +#ifdef CONFIG_UNION_MOUNT +return __d_lookup_union (parent, name); +#else +return __d_lookup_single (parent, name); +#endif +} +/* validate "insecure" dentry pointer */ extern int d_validate (struct dentry *,struct dentry *);
@@ -426,6 +440,7 @@ static inline int d_mountpoint(struct deextern struct vfsmount *lookup_mnt (struct vfsmount *,struct dentry *);extern struct vfsmount *__lookup_mnt (struct vfsmount *,struct dentry *,int );extern struct dentry *lookup_create (struct nameidata *nd,int is_dir); +extern struct vfsmount *find_mnt (struct dentry *);extern int sysctl_vfs_cache_pressure;
include/linux/namei.h +4 -0
@@ -19,6 +19,7 @@ struct nameidata {struct vfsmount *mnt;struct qstr last;unsigned int flags; +unsigned int um_flags;int last_type;unsigned depth;char *saved_names[MAX_NESTED_LINKS +1 ];
@@ -39,6 +40,9 @@ struct path {*/ enum {LAST_NORM ,LAST_ROOT ,LAST_DOT ,LAST_DOTDOT ,LAST_BIND }; +#define LAST_UNION 0x01 +#define LAST_LOWLEVEL 0x02 +/* * The bitmask for a lookup event: * - follow links at the end
include/linux/union.h +27 -0
@@ -17,17 +17,44 @@#ifdef CONFIG_UNION_MOUNT #include <linux/fs_struct.h> +#include <linux/dcache_union.h> /* namespace stuff used at mount time */ extern void attach_mnt_union (struct vfsmount *,struct nameidata *);extern void detach_mnt_union (struct vfsmount *,struct path *); +/* lookup stuff */ +extern int follow_union_mount (struct vfsmount **,struct dentry **); +extern struct dentry *real_lookup_union (struct dentry *,struct qstr *, +struct nameidata *); +extern struct dentry *__lookup_hash_union (struct qstr *,struct dentry *, +struct nameidata *); +#else /* CONFIG_UNION_MOUNT */ #define attach_mnt_union(mnt,nd) do { /* empty */ } while (0) #define detach_mnt_union(mnt,nd) do { /* empty */ } while (0) +#define follow_union_mount(x,y) do { /* empty */ } while (0)#endif /* CONFIG_UNION_MOUNT */ +static inline struct dentry *real_lookup (struct dentry *parent,struct qstr *name,struct nameidata *nd) +{ +#ifdef CONFIG_UNION_MOUNT +return real_lookup_union (parent, name, nd); +#else +return real_lookup_single (parent, name, nd); +#endif +} + +static inline struct dentry *__lookup_hash (struct qstr *name,struct dentry *base,struct nameidata *nd) +{ +#ifdef CONFIG_UNION_MOUNT +return __lookup_hash_union (name, base, nd); +#else +return __lookup_hash_single (name, base, nd); +#endif +} +#endif /* __KERNEL __ */ #endif /* __LINUX_UNION_H */
RFC PATCH 9/15 Simple union-mount readdir
In reply to Bharata B RaoFrom: Jan Blunck <[email protected]>
Subject: Simple union-mount readdir
This is a very simple union mount readdir implementation. It modifies the readdir routine to merge the entries of union mounted directories and eliminate duplicates while walking the union stack.
FIXME: This patch needs to be reworked! At the moment this only works for ext2/3 and tmpfs. All kind of index directories that return d_off > i_size don't work with this.
The directory entries are read starting from the top layer and they are maintained in a cache. Subsequently when the entries from the bottom layers of the union stack are read they are checked for duplicates (in the cache) before being passed out to the user space. There can be multiple calls to readdir/getdents routines for reading the entries of a single directory. But union directory cache is not maitained across these calls. Instead for every call, the previously read entries are re-read into the cache and newly read entires are compared against these for duplicates before being they are returned to user space.
--- fs/aio.c | 7 fs/file_table.c | 14 + fs/readdir.c | 2 fs/union.c | 393 ++++++++++++++++++++++++++++++++++++++++++++++++++ include/linux/union.h | 17 ++ 5 files changed, 426 insertions(+), 7 deletions(-)
fs/aio.c +7 -0
@@ -21,6 +21,7 @@#include <linux/sched.h> #include <linux/fs.h> +#include <linux/mount.h> #include <linux/file.h> #include <linux/mm.h> #include <linux/mman.h>
@@ -487,6 +488,12 @@ static void aio_fput_routine(struct work/* Complete the fput */ __fput (req->ki_filp); +/* +* __fput no longer releases the dentry and vfsmnt, thanks to +* to union mount. Hence do this manually. +*/ +dput (req->ki_filp->f_path.dentry); +mntput (req->ki_filp->f_path.mnt);/* Link the iocb into the context's free list */ spin_lock_irq (&ctx->ctx_lock);
fs/file_table.c +8 -6
@@ -141,8 +141,14 @@ EXPORT_SYMBOL(get_empty_filp); void fastcall fput(struct file *file) { -if (atomic_dec_and_test (&file->f_count)) +struct dentry *dentry = file->f_path.dentry; +struct vfsmount *mnt = file->f_path.mnt; + +if (atomic_dec_and_test (&file->f_count)) {__fput (file); +dput (dentry); +mntput (mnt); + } }EXPORT_SYMBOL (fput);
@@ -152,9 +158,7 @@ EXPORT_SYMBOL(fput);*/ void fastcall __fput(struct file *file) { -struct dentry *dentry = file->f_path.dentry; -struct vfsmount *mnt = file->f_path.mnt; -struct inode *inode = dentry->d_inode; +struct inode *inode = file->f_path.dentry->d_inode;might_sleep ();
@@ -180,8 +184,6 @@ void fastcall __fput(struct file *file) file->f_path.dentry =NULL ; file->f_path.mnt =NULL ;file_free (file); -dput (dentry); -mntput (mnt); } struct file fastcall *fget(unsigned int fd)
fs/readdir.c +1 -1
@@ -33,7 +33,7 @@ int vfs_readdir(struct file *file, filldmutex_lock (&inode->i_mutex); res = -ENOENT ;if (!IS_DEADDIR (inode)) { - res = file->f_op->readdir (file, buf, filler); + res =do_readdir (file, buf, filler);file_accessed (file); }mutex_unlock (&inode->i_mutex);
fs/union.c +393 -0
402 lines not on this page · raw message
include/linux/union.h +17 -0
@@ -29,6 +29,7 @@ extern struct dentry * real_lookup_union struct nameidata *);extern struct dentry *__lookup_hash_union (struct qstr *,struct dentry *,struct nameidata *); +extern int readdir_union (struct file *,void *,filldir_t );#else /* CONFIG_UNION_MOUNT */
@@ -56,5 +57,21 @@ static inline struct dentry * __lookup_h#endif } +static inline int do_readdir (struct file *file,void *buf,filldir_t filler) +{ +#ifdef CONFIG_UNION_MOUNT +int res; + +if (file->f_dentry->d_overlaid) { +union_lock (file->f_dentry); + res =readdir_union (file, buf, filler); +union_unlock (file->f_dentry); + } else +#endif + res = file->f_op->readdir (file, buf, filler); + +return res; +} +#endif /* __KERNEL __ */ #endif /* __LINUX_UNION_H */
RFC PATCH 10/15 In-kernel file copy between union mounted filesystems
In reply to Bharata B RaoFrom: Jan Blunck <[email protected]>
Subject: In-kernel file copy between union mounted filesystems
This patch introduces in-kernel file copy between union mounted filesystems. When a file is opened for writing but resides on a lower (thus read-only) layer of the union stack it is copied to the topmost union layer first.
This patch uses the do_splice_direct() for doing the in-kernel file copy.
--- fs/namei.c | 46 +++++ fs/union.c | 384 ++++++++++++++++++++++++++++++++++++++++++++++++++ include/linux/namei.h | 2 include/linux/union.h | 14 + 4 files changed, 445 insertions(+), 1 deletion(-)
fs/namei.c +45 -1
@@ -830,8 +830,17 @@ done: path->mnt = mnt; path->dentry = dentry; -if (nd->dentry->d_sb != dentry->d_sb) +/* +* This should be checked after the following of unions. +* Otherwise we might run into trouble creating directories +* on mountpoints. :( +* But maybe we shouldn't set the LAST_LOWLEVEL flag here +* at all ... */ +if (nd->dentry->d_sb != dentry->d_sb) { path->mnt =find_mnt (dentry); +UM_DEBUG_UID ("Setting LAST_LOWLEVEL for %s\n" , name->name); + nd->um_flags |=LAST_LOWLEVEL ; + }__follow_mount (path);follow_union_mount (&path->mnt, &path->dentry);
@@ -950,6 +959,14 @@ static fastcall int __link_path_walk(conif (err)break ; +if ((nd->flags &LOOKUP_TOPMOST ) && + (nd->um_flags &LAST_LOWLEVEL )) { + err =union_create_topdir (nd,&next.dentry,&next.mnt); +if (err) + goto out_dput; + nd->um_flags &= ~LAST_LOWLEVEL ; + } + err = -ENOENT ; inode = next.dentry->d_inode;if (!inode)
@@ -1005,6 +1022,15 @@ last_component: err =do_lookup (nd, &this, &next);if (err)break ; + +if ((nd->flags &LOOKUP_TOPMOST ) && + (nd->um_flags &LAST_LOWLEVEL )) { + err =union_create_topdir (nd,&next.dentry,&next.mnt); +if (err) + goto out_dput; + nd->um_flags &= ~LAST_LOWLEVEL ; + } + inode = next.dentry->d_inode;if ((lookup_flags &LOOKUP_FOLLOW ) && inode && inode->i_op && inode->i_op->follow_link) {
@@ -1177,6 +1203,7 @@ static int fastcall do_path_lookup(int d nd->last_type =LAST_ROOT ;/* if there are only slashes... */ nd->flags = flags; + nd->um_flags =0 ; nd->depth =0 ;if (*name=='/' ) {
@@ -1721,9 +1748,18 @@ int open_namei(int dfd, const char *path nd, flag);if (error)return error; +/* test for WRONLY and RDWR - flag's special lower bits */ +if (flag &0x2 ) { +UM_DEBUG_UID ("\"%s\" opened for writing\n" , pathname); + error =union_copyup (nd, flag); +if (error) +return error; + } goto ok; } +UM_DEBUG_UID ("open called with O_CREATE\n" ); +/* * Create - we need to know the parent. */
@@ -1740,6 +1776,8 @@ int open_namei(int dfd, const char *pathif (nd->last_type !=LAST_NORM || nd->last.name[nd->last.len]) goto exit; +UM_DEBUG_UID ("do_last now\n" ); + dir = nd->dentry; nd->flags &= ~LOOKUP_PARENT ;mutex_lock (&dir->d_inode->i_mutex);
@@ -1793,6 +1831,12 @@ do_last: error = -EISDIR ;if (path.dentry->d_inode &&S_ISDIR (path.dentry->d_inode->i_mode)) goto exit; + +if (flag &0x2 ) { + error =union_copyup (nd, flag); +if (error) + goto exit; + } ok: error =may_open (nd, acc_mode, flag);if (error)
fs/union.c +384 -0
399 lines not on this page · raw message
include/linux/namei.h +2 -0
@@ -58,6 +58,8 @@ enum {LAST_NORM, LAST_ROOT, LAST_DOT, LA#define LOOKUP_PARENT 16#define LOOKUP_NOALT 32#define LOOKUP_REVAL 64 +#define LOOKUP_TOPMOST 128 +#define LOOKUP_WHT 256/* * Intent data */
include/linux/union.h +14 -0
@@ -31,11 +31,25 @@ extern struct dentry * __lookup_hash_uni struct nameidata *);extern int readdir_union (struct file *,void *,filldir_t ); +/* copy-up support */ +extern struct dentry *union_create_topmost (struct nameidata *,struct dentry *); +extern int union_create_topdir (struct nameidata *,struct dentry **,struct vfsmount **); +extern int union_is_member (struct dentry *,struct vfsmount *); +extern int union_copy_file (struct dentry *,struct vfsmount *,struct dentry *,struct vfsmount *); +extern int union_copyup (struct nameidata *,int ); +extern int union_relookup_topmost (struct nameidata *,int ); +#else /* CONFIG_UNION_MOUNT */ #define attach_mnt_union(mnt,nd) do { /* empty */ } while (0) #define detach_mnt_union(mnt,nd) do { /* empty */ } while (0) #define follow_union_mount(x,y) do { /* empty */ } while (0) +#define union_create_topmost (x,y) ({ BUG(); ERR_PTR(-EINVAL); }) +#define union_create_topdir (x,y,z) ({ (0); }) +#define union_is_member (x,y) ({ (0); }) +#define union_copy_file (dentry1,mnt1,dentry2,mnt2) ({ (0); }) +#define union_copyup (x,y) ({ (0); }) +#define union_relookup_topmost (x,y) ({ (0); })#endif /* CONFIG_UNION_MOUNT */
RFC PATCH 11/15 VFS whiteout handling
In reply to Bharata B RaoFrom: Jan Blunck <[email protected]>
Subject: VFS whiteout handling
Introduce white-out handling in the VFS.
--- fs/inode.c | 17 + fs/namei.c | 476 ++++++++++++++++++++++++++++++++++++++++++++++++-- fs/readdir.c | 10 + fs/union.c | 104 ++++++++++ include/linux/fs.h | 4 include/linux/union.h | 6 6 files changed, 605 insertions(+), 12 deletions(-)
fs/inode.c +17 -0
@@ -1418,6 +1418,21 @@ void __init inode_init(unsigned long memINIT_HLIST_HEAD (&inode_hashtable[loop]); } +/* +* Dummy default file-operations: +* Never open a whiteout. This is always a bug. +*/ +static int whiteout_no_open (struct inode *irrelevant,struct file *dontcare) +{ +printk ("Attemp to open a whiteout!\n" ); +WARN_ON (1 ); +return -ENXIO ; +} + +static struct file_operations def_wht_fops = { + .open = whiteout_no_open, +}; +void init_special_inode (struct inode *inode,umode_t mode,dev_t rdev) { inode->i_mode = mode;
@@ -1431,6 +1446,8 @@ void init_special_inode(struct inode *in inode->i_fop = &def_fifo_fops; else if (S_ISSOCK(mode)) inode->i_fop = &bad_sock_fops; + else if (S_ISWHT(mode)) + inode->i_fop = &def_wht_fops; else printk(KERN_DEBUG "init_special_inode: bogus i_mode (%o)\n", mode);
fs/namei.c +464 -12
612 lines not on this page · raw message
fs/readdir.c +10 -0
@@ -148,6 +148,11 @@ static int filldir(void * __buf, const cunsigned long d_ino;int reclen =ALIGN (NAME_OFFSET (dirent) + namlen +2 ,sizeof (long )); +#ifdef CONFIG_UNION_MOUNT +if (d_type ==DT_WHT ) +return 0 ; +#endif /* CONFIG_UNION_MOUNT */ + buf->error = -EINVAL ;/* only used if we fail.. */ if (reclen > buf->count)return -EINVAL ;
@@ -233,6 +238,11 @@ static int filldir64(void * __buf, conststruct getdents_callback64 * buf = (struct getdents_callback64 *) __buf;int reclen =ALIGN (NAME_OFFSET (dirent) + namlen +1 ,sizeof (u64)); +#ifdef CONFIG_UNION_MOUNT +if (d_type ==DT_WHT ) +return 0 ; +#endif /* CONFIG_UNION_MOUNT */ + buf->error = -EINVAL ;/* only used if we fail.. */ if (reclen > buf->count)return -EINVAL ;
fs/union.c +104 -0
137 lines not on this page · raw message
include/linux/fs.h +4 -0
@@ -151,6 +151,7 @@ extern int dir_notify_enable;#define S_NOCMTIME 128/* Do not update file c/mtime */ #define S_SWAPFILE 256/* Do not truncate: swapon got its bmaps */ #define S_PRIVATE 512/* Inode is fs-internal */ +#define S_OPAQUE 1024/* Directory is opaque */ /* * Note that nosuid etc flags are inode-specific: setting some file-system
@@ -184,6 +185,7 @@ extern int dir_notify_enable;#define IS_NOCMTIME (inode) ((inode)->i_flags & S_NOCMTIME)#define IS_SWAPFILE (inode) ((inode)->i_flags & S_SWAPFILE)#define IS_PRIVATE (inode) ((inode)->i_flags & S_PRIVATE) +#define IS_OPAQUE (inode) ((inode)->i_flags & S_OPAQUE)/* the read-only stuff doesn't really belong here, but any other place is probably as bad and I don't want to create yet another include file. */
@@ -1043,6 +1045,7 @@ extern int vfs_link(struct dentry *, strextern int vfs_rmdir (struct inode *,struct dentry *);extern int vfs_unlink (struct inode *,struct dentry *);extern int vfs_rename (struct inode *,struct dentry *,struct inode *,struct dentry *); +extern int vfs_whiteout (struct inode *,struct dentry *);/* * VFS dentry helper functions.
@@ -1168,6 +1171,7 @@ struct inode_operations {int (*mkdir) (struct inode *,struct dentry *,int );int (*rmdir) (struct inode *,struct dentry *);int (*mknod) (struct inode *,struct dentry *,int ,dev_t ); +int (*whiteout) (struct inode *,struct dentry *);int (*rename) (struct inode *,struct dentry *,struct inode *,struct dentry *);int (*readlink) (struct dentry *,char __user *,int );
include/linux/union.h +6 -0
@@ -39,6 +39,10 @@ extern int union_copy_file(struct dentryextern int union_copyup (struct nameidata *,int );extern int union_relookup_topmost (struct nameidata *,int ); +/* vfs whiteout support */ +extern int union_dir_is_empty (struct dentry *); +extern int present_in_lower (struct dentry *,struct nameidata *); +#else /* CONFIG_UNION_MOUNT */ #define attach_mnt_union(mnt,nd) do { /* empty */ } while (0)
@@ -50,6 +54,8 @@ extern int union_relookup_topmost(struct#define union_copy_file (dentry1,mnt1,dentry2,mnt2) ({ (0); })#define union_copyup (x,y) ({ (0); })#define union_relookup_topmost (x,y) ({ (0); }) +#define union_dir_is_empty (x) ({ (1); }) +#define present_in_lower (x, y) ({ (0); })#endif /* CONFIG_UNION_MOUNT */
RFC PATCH 12/15 ext2 whiteout support
In reply to Bharata B RaoFrom: Jan Blunck <[email protected]>
Subject: ext2 whiteout support
Introduce whiteout support to ext2.
--- fs/ext2/dir.c | 2 ++ fs/ext2/namei.c | 17 +++++++++++++++++ fs/ext2/super.c | 11 ++++++++++- include/linux/ext2_fs.h | 4 ++++ 4 files changed, 33 insertions(+), 1 deletion(-)
fs/ext2/dir.c +2 -0
@@ -218,6 +218,7 @@ static unsigned char ext2_filetype_table [EXT2_FT_FIFO] = DT_FIFO, [EXT2_FT_SOCK] = DT_SOCK, [EXT2_FT_SYMLINK] = DT_LNK, + [EXT2_FT_WHT] = DT_WHT, };#define S_SHIFT 12
@@ -229,6 +230,7 @@ static unsigned char ext2_type_by_mode[S [S_IFIFO >> S_SHIFT] = EXT2_FT_FIFO, [S_IFSOCK >> S_SHIFT] = EXT2_FT_SOCK, [S_IFLNK >> S_SHIFT] = EXT2_FT_SYMLINK, + [S_IFWHT >> S_SHIFT] = EXT2_FT_WHT, };static inline void ext2_set_de_type (ext2_dirent *de,struct inode *inode)
fs/ext2/namei.c +17 -0
@@ -288,6 +288,22 @@ static int ext2_rmdir (struct inode * direturn err; } +static int ext2_whiteout (struct inode *dir,struct dentry *dentry) +{ +struct inode *inode; +int err; + + inode =ext2_new_inode (dir,S_IFWHT |S_IRUGO ); + err =PTR_ERR (inode); +if (IS_ERR (inode)) + goto out; + +mark_inode_dirty (inode); + err =ext2_add_nondir (dentry, inode); +out: +return err; +} +static int ext2_rename (struct inode * old_dir,struct dentry * old_dentry,struct inode * new_dir,struct dentry * new_dentry ) {
@@ -382,6 +398,7 @@ const struct inode_operations ext2_dir_i .mkdir = ext2_mkdir, .rmdir = ext2_rmdir, .mknod = ext2_mknod, + .whiteout = ext2_whiteout, .rename = ext2_rename, #ifdef CONFIG_EXT2_FS_XATTR .setxattr = generic_setxattr,
fs/ext2/super.c +10 -1
@@ -755,6 +755,15 @@ static int ext2_fill_super(struct super_ext2_xip_verify_sb (sb);/* see if bdev supports xip, unset EXT2_MOUNT_XIP if not */ +if ((sb->s_flags &MS_UNION ) && !(sb->s_flags &MS_RDONLY )) { +if (!EXT2_HAS_INCOMPAT_FEATURE (sb, +EXT2_FEATURE_INCOMPAT_WHITEOUT )) { + sb->s_flags |=MS_RDONLY ; +ext2_warning (sb, __FUNCTION__, +"no whiteout support, mounting filesystem read-only" ); + } + } + if (le32_to_cpu(es->s_rev_level) == EXT2_GOOD_OLD_REV && (EXT2_HAS_COMPAT_FEATURE(sb, ~0U) || EXT2_HAS_RO_COMPAT_FEATURE(sb, ~0U) ||
@@ -1293,7 +1302,7 @@ static struct file_system_type ext2_fs_t .name = "ext2", .get_sb = ext2_get_sb, .kill_sb = kill_block_super, - .fs_flags = FS_REQUIRES_DEV, + .fs_flags = FS_REQUIRES_DEV | FS_WHT, };static int __init init_ext2_fs (void )
include/linux/ext2_fs.h +4 -0
@@ -61,6 +61,7 @@#define EXT2_ROOT_INO 2/* Root inode */ #define EXT2_BOOT_LOADER_INO 5/* Boot loader inode */ #define EXT2_UNDEL_DIR_INO 6/* Undelete directory inode */ +#define EXT2_WHT_INO 7/* Whiteout inode */ /* First non-reserved inode for old ext2 filesystems */ #define EXT2_GOOD_OLD_FIRST_INO 11
@@ -479,10 +480,12 @@ struct ext2_super_block {#define EXT3_FEATURE_INCOMPAT_RECOVER 0x0004#define EXT3_FEATURE_INCOMPAT_JOURNAL_DEV 0x0008#define EXT2_FEATURE_INCOMPAT_META_BG 0x0010 +#define EXT2_FEATURE_INCOMPAT_WHITEOUT 0x0020#define EXT2_FEATURE_INCOMPAT_ANY 0xffffffff#define EXT2_FEATURE_COMPAT_SUPP EXT2_FEATURE_COMPAT_EXT_ATTR#define EXT2_FEATURE_INCOMPAT_SUPP (EXT2_FEATURE_INCOMPAT_FILETYPE| \ + EXT2_FEATURE_INCOMPAT_WHITEOUT| \ EXT2_FEATURE_INCOMPAT_META_BG)#define EXT2_FEATURE_RO_COMPAT_SUPP (EXT2_FEATURE_RO_COMPAT_SPARSE_SUPER| \ EXT2_FEATURE_RO_COMPAT_LARGE_FILE| \
@@ -549,6 +552,7 @@ enum {EXT2_FT_FIFO ,EXT2_FT_SOCK ,EXT2_FT_SYMLINK , +EXT2_FT_WHT , EXT2_FT_MAX };
RFC PATCH 13/15 ext3 whiteout support
In reply to Bharata B RaoFrom: Bharata B Rao <[email protected]>
Subject: ext3 whiteout support
Introduce whiteout support for ext3.
Signed-off-bg: Jan Blunck <[email protected]>
--- fs/ext3/dir.c | 2 - fs/ext3/namei.c | 65 ++++++++++++++++++++++++++++++++++++++++++++---- fs/ext3/super.c | 11 +++++++- include/linux/ext3_fs.h | 5 ++- 4 files changed, 75 insertions(+), 8 deletions(-)
fs/ext3/dir.c +1 -1
@@ -29,7 +29,7 @@#include <linux/rbtree.h> static unsigned char ext3_filetype_table[] = { -DT_UNKNOWN ,DT_REG ,DT_DIR ,DT_CHR ,DT_BLK ,DT_FIFO ,DT_SOCK ,DT_LNK +DT_UNKNOWN ,DT_REG ,DT_DIR ,DT_CHR ,DT_BLK ,DT_FIFO ,DT_SOCK ,DT_LNK ,DT_WHT };static int ext3_readdir (struct file *,void *,filldir_t );
fs/ext3/namei.c +60 -5
107 lines not on this page · raw message
fs/ext3/super.c +10 -1
@@ -1493,6 +1493,15 @@ static int ext3_fill_super (struct super sb->s_flags = (sb->s_flags & ~MS_POSIXACL ) | ((sbi->s_mount_opt &EXT3_MOUNT_POSIX_ACL ) ?MS_POSIXACL :0 ); +if ((sb->s_flags &MS_UNION ) && !(sb->s_flags &MS_RDONLY )) { +if (!EXT3_HAS_INCOMPAT_FEATURE (sb, +EXT3_FEATURE_INCOMPAT_WHITEOUT )) { + sb->s_flags |=MS_RDONLY ; +ext3_warning (sb, __FUNCTION__, +"no whiteout support, mounting filesystem read-only" ); + } + } + if (le32_to_cpu(es->s_rev_level) == EXT3_GOOD_OLD_REV && (EXT3_HAS_COMPAT_FEATURE(sb, ~0U) || EXT3_HAS_RO_COMPAT_FEATURE(sb, ~0U) ||
@@ -2749,7 +2758,7 @@ static struct file_system_type ext3_fs_t .name = "ext3", .get_sb = ext3_get_sb, .kill_sb = kill_block_super, - .fs_flags = FS_REQUIRES_DEV, + .fs_flags = FS_REQUIRES_DEV | FS_WHT, };static int __init init_ext3_fs (void )
include/linux/ext3_fs.h +4 -1
@@ -63,6 +63,7 @@#define EXT3_UNDEL_DIR_INO 6/* Undelete directory inode */ #define EXT3_RESIZE_INO 7/* Reserved group descriptors inode */ #define EXT3_JOURNAL_INO 8/* Journal inode */ +#define EXT3_WHT_INO 9/* Whiteout inode */ /* First non-reserved inode for old ext3 filesystems */ #define EXT3_GOOD_OLD_FIRST_INO 11
@@ -582,6 +583,7 @@ static inline int ext3_valid_inum(struct#define EXT3_FEATURE_INCOMPAT_RECOVER 0x0004/* Needs recovery */ #define EXT3_FEATURE_INCOMPAT_JOURNAL_DEV 0x0008/* Journal device */ #define EXT3_FEATURE_INCOMPAT_META_BG 0x0010 +#define EXT3_FEATURE_INCOMPAT_WHITEOUT 0x0020#define EXT3_FEATURE_COMPAT_SUPP EXT2_FEATURE_COMPAT_EXT_ATTR#define EXT3_FEATURE_INCOMPAT_SUPP (EXT3_FEATURE_INCOMPAT_FILETYPE| \
@@ -648,8 +650,9 @@ struct ext3_dir_entry_2 {#define EXT3_FT_FIFO 5#define EXT3_FT_SOCK 6#define EXT3_FT_SYMLINK 7 +#define EXT3_FT_WHT 8 -#define EXT3_FT_MAX 8 +#define EXT3_FT_MAX 9/* * EXT3_DIR_PAD defines the directory entries boundaries
RFC PATCH 14/15 tmpfs whiteout support
In reply to Bharata B RaoFrom: Jan Blunck <[email protected]>
Subject: tmpfs whiteout support
Introduce whiteout support to tmpfs.
--- mm/shmem.c | 9 ++++++++- 1 files changed, 8 insertions(+), 1 deletion(-)
mm/shmem.c +8 -1
@@ -74,7 +74,7 @@#define LATENCY_LIMIT 64/* Pretend that each entry is of this size in directory's i_size */ -#define BOGO_DIRENT_SIZE 20 +#define BOGO_DIRENT_SIZE 1/* Flag allocation requirements to shmem_getpage and shmem_swp_alloc */ enum sgp_type {
@@ -1772,6 +1772,11 @@ static int shmem_create(struct inode *direturn shmem_mknod (dir, dentry, mode |S_IFREG ,0 ); } +static int shmem_whiteout (struct inode *dir,struct dentry *dentry) +{ +return shmem_mknod (dir, dentry,S_IRUGO |S_IWUGO |S_IFWHT ,0 ); +} +/* * Link a file.. */
@@ -2400,6 +2405,7 @@ static const struct inode_operations shm .rmdir = shmem_rmdir, .mknod = shmem_mknod, .rename = shmem_rename, + .whiteout = shmem_whiteout,#endif #ifdef CONFIG_TMPFS_POSIX_ACL .setattr = shmem_notify_change,
@@ -2454,6 +2460,7 @@ static struct file_system_type tmpfs_fs_ .name = "tmpfs", .get_sb = shmem_get_sb, .kill_sb = kill_litter_super, + .fs_flags = FS_WHT, };static struct vfsmount *shm_mnt;
RFC PATCH 15/15 Union-mount changes for NFS
In reply to Bharata B RaoFrom: Jan Blunck <[email protected]>
Subject: Union-mount changes for NFS
Changes necessary to mount a NFS volume into a union.
--- fs/nfs/dir.c | 41 +++++++++++++++++++++++++++-------------- fs/nfs/inode.c | 4 ++-- fs/nfs/unlink.c | 4 ++-- 3 files changed, 31 insertions(+), 18 deletions(-)
fs/nfs/dir.c +27 -14
@@ -409,7 +409,7 @@ int nfs_do_filldir(nfs_readdir_descripto/* Get a dentry if we have one */ if (dentry !=NULL ) -dput (dentry); +__dput_single (dentry); dentry =nfs_readdir_lookup (desc);/* Use readdirplus info */
@@ -435,7 +435,7 @@ int nfs_do_filldir(nfs_readdir_descripto }dir_page_release (desc);if (dentry !=NULL ) -dput (dentry); +__dput_single (dentry);dfprintk (DIRCACHE ,"NFS: nfs_do_filldir() filling ended @ cookie %Lu; returning = %d\n" , (unsigned long long )*desc->dir_cookie, res);return res;
@@ -721,6 +721,16 @@ int nfs_neg_need_reval(struct inode *dirreturn !nfs_check_verifier (dir, dentry); } +static struct dentry *__dget_parent_single (struct dentry *dentry) +{ +struct dentry *ret; + +spin_lock (&dentry->d_lock); + ret =__dget_single (dentry->d_parent); +spin_unlock (&dentry->d_lock); +return ret; +} +/* * This is called every time the dcache has a lookup hit, * and we should check whether we can really trust that
@@ -742,7 +752,7 @@ static int nfs_lookup_revalidate(structstruct nfs_fattr fattr;unsigned long verifier; - parent =dget_parent (dentry); + parent =__dget_parent_single (dentry);lock_kernel (); dir = parent->d_inode;nfs_inc_stats (dir,NFSIOS_DENTRYREVALIDATE );
@@ -788,7 +798,7 @@ static int nfs_lookup_revalidate(structnfs_refresh_verifier (dentry, verifier); out_valid:unlock_kernel (); -dput (parent); +__dput_single (parent);dfprintk (LOOKUPCACHE ,"NFS: %s(%s/%s) is valid\n" , __FUNCTION__, dentry->d_parent->d_name.name, dentry->d_name.name);
@@ -807,7 +817,7 @@ out_zap_parent: }d_drop (dentry);unlock_kernel (); -dput (parent); +__dput_single (parent);dfprintk (LOOKUPCACHE ,"NFS: %s(%s/%s) is invalid\n" , __FUNCTION__, dentry->d_parent->d_name.name, dentry->d_name.name);
@@ -1057,7 +1067,7 @@ static int nfs_open_revalidate(struct deunsigned long verifier;int openflags, ret =0 ; - parent =dget_parent (dentry); + parent =__dget_parent_single (dentry); dir = parent->d_inode;if (!is_atomic_open (dir, nd)) goto no_open;
@@ -1088,18 +1098,21 @@ static int nfs_open_revalidate(struct denfs_refresh_verifier (dentry, verifier);unlock_kernel (); out: -dput (parent); +__dput_single (parent);if (!ret)d_drop (dentry);return ret; no_open: -dput (parent); +__dput_single (parent);if (inode !=NULL &&nfs_have_delegation (inode,FMODE_READ ))return 1 ; return nfs_lookup_revalidate(dentry, nd); }#endif /* CONFIG_NFSV4 */ +/* For union mount we need this: +* - lookup the complete union if we found one +* - don't return lower layers dentries ... */ static struct dentry *nfs_readdir_lookup (nfs_readdir_descriptor_t *desc) {struct dentry *parent = desc->file->f_path.dentry;
@@ -1115,14 +1128,14 @@ static struct dentry *nfs_readdir_lookupswitch (name.len) {case 2 :if (name.name[0 ] =='.' && name.name[1 ] =='.' ) -return dget_parent (parent); +return __dget_parent_single (parent);break ;case 1 :if (name.name[0 ] =='.' ) -return dget (parent); +return __dget_single (parent); } name.hash =full_name_hash (name.name, name.len); - dentry =d_lookup (parent, &name); + dentry =d_lookup_single (parent, &name);if (dentry !=NULL ) {/* Is this a positive dentry that matches the readdir info? */ if (dentry->d_inode != NULL &&
@@ -1136,7 +1149,7 @@ static struct dentry *nfs_readdir_lookup }/* No, so d_drop to allow one to be created */ d_drop (dentry); -dput (dentry); +__dput_single (dentry); }if (!desc->plus || !(entry->fattr->valid &NFS_ATTR_FATTR ))return NULL ;
@@ -1147,13 +1160,13 @@ static struct dentry *nfs_readdir_lookup dentry->d_op =NFS_PROTO (dir)->dentry_ops; inode =nfs_fhget (dentry->d_sb, entry->fh, entry->fattr);if (IS_ERR (inode)) { -dput (dentry); +__dput_single (dentry);return NULL ; } alias =d_materialise_unique (dentry, inode);if (alias !=NULL ) { -dput (dentry); +__dput_single (dentry);if (IS_ERR (alias))return NULL ; dentry = alias;
fs/nfs/inode.c +2 -2
@@ -460,7 +460,7 @@ static struct nfs_open_context *alloc_nf ctx =kmalloc (sizeof (*ctx),GFP_KERNEL );if (ctx !=NULL ) {atomic_set (&ctx->count,1 ); - ctx->dentry =dget (dentry); + ctx->dentry =__dget_single (dentry); ctx->vfsmnt =mntget (mnt); ctx->cred =get_rpccred (cred); ctx->state =NULL ;
@@ -491,7 +491,7 @@ void put_nfs_open_context(struct nfs_openfs4_close_state (ctx->state, ctx->mode);if (ctx->cred !=NULL )put_rpccred (ctx->cred); -dput (ctx->dentry); +__dput_single (ctx->dentry);mntput (ctx->vfsmnt);kfree (ctx); }
fs/nfs/unlink.c +2 -2
@@ -129,7 +129,7 @@ static void nfs_async_unlink_done(structreturn ;put_rpccred (data->cred); data->cred =NULL ; -dput (dir); +__dput_single (dir); }/**
@@ -172,7 +172,7 @@ nfs_async_unlink(struct dentry *dentry) status =PTR_ERR (data->cred); goto out_free; } - data->dir =dget (dir); + data->dir =__dget_single (dir); data->dentry = dentry; data->next = nfs_deletes;
Quoted 7 lines · Bharata B Rao
Hi,
Here is an attempt towards vfs based union mount implementation. Union mount provides the filesytem namespace unification feature. Unlike the traditional mounts which hide the contents of the mount point, the union mount presents the merged view of the mount point and the mounted filesytem.
does this approach allow one to add directories to the union and have it behave normally. namely when imagine one has the situation
dir-b
dir-a/ (contains file foo)
if one unions this and deletes foo, that will create a whiteout entry in dir-a
now, what happens if one does
dir-c
dir-b (now contains whiteout, from previous union). dir-a (contains file foo)
will one see foo or not. i.e. are whiteouts only looked for in the topmost dir, or in every dir?
Quoted 12 lines · Shaya Potter
Quoted 7 lines · Bharata B Rao
Hi,
Here is an attempt towards vfs based union mount implementation. Union mount provides the filesytem namespace unification feature. Unlike the traditional mounts which hide the contents of the mount point, the union mount presents the merged view of the mount point and the mounted filesytem.
does this approach allow one to add directories to the union and have it behave normally. namely when imagine one has the situation
Quoted 6 lines
dir-b
dir-a/ (contains file foo)if one unions this and deletes foo, that will create a whiteout entry in dir-a
(I guess you mean to say that this creates a whiteout in dir-b)
Quoted 9 lines
now, what happens if one does
dir-c
dir-b (now contains whiteout, from previous union). dir-a (contains file foo)will one see foo or not. i.e. are whiteouts only looked for in the topmost dir, or in every dir?
No. foo is not visible. While looking for a file in a union mounted directory, the lookup starts from the topmost directory and proceeds downwards if the file isn't present the top layers. If a whiteout is found in any of the top layers, the lookup is abondoned and -ENOENT is removed. Thus until a whiteout exists in any upper layer for a corresponding file in the lower layer, the lower layer file remains hidden until the whiteout is removed.
However in the case of dir-c containing foo, the foo(from dir-c) will become visible after union mounting dir-c on top of dir-b and dir-a.
Regards,
Bharata.
Quoted 10 lines · Bharata B Rao
No. foo is not visible. While looking for a file in a union mounted directory, the lookup starts from the topmost directory and proceeds downwards if the file isn't present the top layers. If a whiteout is found in any of the top layers, the lookup is abondoned and
-ENOENTis removed. Thus until a whiteout exists in any upper layer for a corresponding file in the lower layer, the lower layer file remains hidden until the whiteout is removed.However in the case of dir-c containing foo, the foo(from dir-c) will become visible after union mounting dir-c on top of dir-b and dir-a.
ok, so the major limitation of this approach is that the top most layer has to either be, ext2, ext3 or tmpfs (in patch), and most likely not NFS (assumption is that NFS has no conception of the whiteout type of file). One thing the unionfs people are doing w/ their ODF approach, is within the ODF fs, they have a special inode that is the "whiteout" inode, and when they create a whiteout, they just create a hardlink from the dentry they want to whiteout to the "whiteout inode". could that be a worthwhile approach instead of the whiteout file type? (i.e. many file systems support the concept of a hard link).
I ask, because using union in a diskless environment. Imagine pxe booting a kernel/initramfs and then using union to create a real root fs
(shared lower layer, private rw upper layer, ala live cds). Which
brings up a different point, with unionfs, one can pivot_root into it, can one do the same for these "union mounts"? Don't know enough about the VFS to know if this should "just work" or might be a problem.
Quoted 10 lines · Shaya Potter
Quoted 8 lines · Bharata B Rao
No. foo is not visible. While looking for a file in a union mounted directory, the lookup starts from the topmost directory and proceeds downwards if the file isn't present the top layers. If a whiteout is found in any of the top layers, the lookup is abondoned and
-ENOENTis removed. Thus until a whiteout exists in any upper layer for a corresponding file in the lower layer, the lower layer file remains hidden until the whiteout is removed.
Quoted 8 lines
Quoted 3 lines
However in the case of dir-c containing foo, the foo(from dir-c) will become
visible after union mounting dir-c on top of dir-b and dir-a.ok, so the major limitation of this approach is that the top most layer has to either be, ext2, ext3 or tmpfs (in patch), and most likely not NFS (assumption is that NFS has no conception of the whiteout type of file).
I haven't played with union mounts with NFS. Hence would let Jan answer this. However note that union mount provides a writable union only if the filesystem supports the notion of whiteouts.
Quoted 6 lines
One thing the unionfs people are doing w/ their ODF approach, is within the ODF fs, they have a special inode that is the "whiteout" inode, and when they create a whiteout, they just create a hardlink from the dentry they want to whiteout to the "whiteout inode". could that be a worthwhile approach instead of the whiteout file type? (i.e. many file systems support the concept of a hard link).
We we thinking something on similar lines as noted in our documentation. Right now we maintain one inode for every whiteout. We were planning to have a single whiteout inode and have all whiteout dentries point to this. But here again we were thinking of having every filesystem support this whiteout inode type.
Anyway I will have a look at ODF from unionfs to see how this is done.
Quoted 7 lines
I ask, because using union in a diskless environment. Imagine pxe booting a kernel/initramfs and then using union to create a real root fs
(shared lower layer, private rw upper layer, ala live cds). Which
brings up a different point, with unionfs, one canpivot_rootinto it, can one do the same for these "union mounts"? Don't know enough about the VFS to know if this should "just work" or might be a problem.
I would assume that it should 'just work'. But right now it is not working. Our code is not yet ready to correctly work with move mounts. Since pivot_root has semantics similar to move mounts, pivot_root is also not working. Also chroot to a union mount point is also not working atm. We will be working to get all these right.
Regards,
Bharata.