Fallthrus as full-length symlinks?
Fallthrus were invented as a placeholders for readdir() on a union-mounted directory - basically, to use the top-level file system's readdir() cookie mechanism. Fallthrus are persistent directory entries and are implemented by the underlying file system - such as ext2 or tmpfs - in whatever way it sees fit. We've implemented them for ext2 in two ways: as a regular directory entry with a magic inode number, and as a regular directory entry with a special file type.
Recently, David Woodhouse suggested implementing fallthrus as full-length symlinks with a special flag. The interesting thing about this idea is that it could theoretically let us rename a file from the low level file system to another place in the low-level file system without copying the contents of the file up. Basically, we can arbitrarily swizzle the namespace of the low-level by maintaining a set of symlinks above.
Is this useful? Is it implementable?
Background reading:
http://valerieaurora.org/union/
-VAL
Quoted 8 lines · Valerie Aurora
Fallthrus were invented as a placeholders for
readdir()on a union-mounted directory - basically, to use the top-level file system'sreaddir()cookie mechanism. Fallthrus are persistent directory entries and are implemented by the underlying file system - such as ext2 or tmpfs - in whatever way it sees fit. We've implemented them for ext2 in two ways: as a regular directory entry with a magic inode number, and as a regular directory entry with a special file type.
Other than a possible improvement to ->rename, what's wrong with the idea of a special dirent flag? I kinda liked that idea: it's simple and requires only a small amount of change to lower file systems. Any idea in which you have to record the whiteouts using an actual file or inode is more cumbersome.
Quoted 2 lines
Recently, David Woodhouse suggested implementing fallthrus as full-length symlinks with a special flag.
Where does this "special flag" go? Is it persistent? Is it new? Would that mean having to change lower file systems to teach them about this flag?
Is there a way of doing it w/o having to change lower f/s code at all? That'll be a major advantage if possible.
Quoted 6 lines
The interesting thing about
this idea is that it could theoretically let us rename a file from the low level file system to another place in the low-level file system without copying the contents of the file up. Basically, we can arbitrarily swizzle the namespace of the low-level by maintaining a set of symlinks above.
So now you're proposing to allow something like multiple writeable branches, in that you allow something other than the topmost branch to be modified. Moreover, it appears that what you're proposing will need to modify two or more branches, right?
Maybe I don't understand what this symlink idea is all about. But I do know (and have documented it in a TOS article), that ->rename is the most difficult op to implement in a unioning system.
Getting multiple writeable branches to work reliably in Unionfs had been a major challenge. In particular, for rename. We've first tried to get rename to be optimal: to rename within the branch a file was in already, or to minimize modifications across too many layers, to reduce copyup. But we found it to be hard to do reliably, esp. when you take into account whiteouts and opaque directories; even worse when you consider multiple concurrent renames which could touch a subset of your layers.
See, some unioning operations have to proceed from top to bottom (e.g., lookups), while others have to proceed from some middle layer going upwards (e.g., copyups). This is a recipe for deadlocks and other races, b/c there's no clear ordering of operations. After trying this rename optimization for several years in unionfs, we gave up on it and just went with a simpler policy: either you can rename within the same branch, or copyup the destination into its new name (and optionally whiteout the source).
I'm not saying it can't be done, but I'm concerned that these symlinks may add too much complexity to the code. One would need to think very carefully about every operation that could affect these symlinks, and how they might interact with concurrent f/s operations, as well as ops that traverse the stack in different directions.
Also, aren't you worried about the symlinks cluttering the lower namespace and consuming inodes and data blocks? (That has been one of the criticisms of Unionfs.)
Quoted 1 line
Is this useful? Is it implementable?
It's Deja Vu all over again. :-)
Quoted 5 lines
Background reading:
http://valerieaurora.org/union/
-VAL
Erez.
Quoted 11 lines · Erez Zadok
Quoted 6 lines
The interesting thing about
this idea is that it could theoretically let us rename a file from the low level file system to another place in the low-level file system without copying the contents of the file up. Basically, we can arbitrarily swizzle the namespace of the low-level by maintaining a set of symlinks above.So now you're proposing to allow something like multiple writeable branches, in that you allow something other than the topmost branch to be modified. Moreover, it appears that what you're proposing will need to modify two or more branches, right?
I think you misunderstood. If the idea is that you can do a 'mv file_on_lower directory_on_higher/', you would just change the magic symlink on the higher branch, without writing to the lower branch, but you are able to work as if you can effectively do operations that would otherwise require write access on the lower branch.
Quoted 1 line
Quoted 1 line
Is this useful? Is it implementable?
I think it sounds very useful and simple, but I may also have missed something. One issue might be that with a lot of file systems stacked on top of each other, you could run out of stack space recursing through the symlinks, but that's something that may already be the case, or that can be worked around in other ways.
Another idea that I first had when reading the suggestion was to use a symlink to self (ln -s x x) as the encoding for a fallthrough. It does not allow renames like what you really describe, but it has another advantage in that it does not require extensions to the upper file system layout while not conflicting with any use case I can see.
Arnd <><
Quoted 13 lines · Arnd Bergmann
Quoted 11 lines · Erez Zadok
Quoted 6 lines
The interesting thing about
this idea is that it could theoretically let us rename a file from the low level file system to another place in the low-level file system without copying the contents of the file up. Basically, we can arbitrarily swizzle the namespace of the low-level by maintaining a set of symlinks above.So now you're proposing to allow something like multiple writeable branches, in that you allow something other than the topmost branch to be modified. Moreover, it appears that what you're proposing will need to modify two or more branches, right?
Quoted 6 lines
I think you misunderstood. If the idea is that you can do a 'mv
file_on_lowerdirectory_on_higher/', you would just change the magic symlink on the higher branch, without writing to the lower branch, but you are able to work as if you can effectively do operations that would otherwise require write access on the lower branch.
Arnd is right - only the top layer is modified. Right now, renaming a file on the lower layer requires copying up the contents of the file to the top layer, creating the new link to it, and creating a whiteout over the old name. This approach would instead leave the contents of the file on the lower layer, create a new link to it which contains the full path of the target in the lower layer, and create a whiteout over the old name. All of the modifications happen on the top layer still.
Quoted 7 lines
Quoted 1 line
Quoted 1 line
Is this useful? Is it implementable?
I think it sounds very useful and simple, but I may also have missed something. One issue might be that with a lot of file systems stacked on top of each other, you could run out of stack space recursing through the symlinks, but that's something that may already be the case, or that can be worked around in other ways.
There would be only one layer of these symlinks possible.
Quoted 5 lines
Another idea that I first had when reading the suggestion was to use a symlink to self (ln -s x x) as the encoding for a fallthrough. It does not allow renames like what you really describe, but it has another advantage in that it does not require extensions to the upper file system layout while not conflicting with any use case I can see.
That's certainly worth trying!
-VAL
Quoted 10 lines · Erez Zadok
Quoted 8 lines · Valerie Aurora
Fallthrus were invented as a placeholders for
readdir()on a union-mounted directory - basically, to use the top-level file system'sreaddir()cookie mechanism. Fallthrus are persistent directory entries and are implemented by the underlying file system - such as ext2 or tmpfs - in whatever way it sees fit. We've implemented them for ext2 in two ways: as a regular directory entry with a magic inode number, and as a regular directory entry with a special file type.
Quoted 5 lines
Other than a possible improvement to ->rename, what's wrong with the idea of a special dirent flag? I kinda liked that idea: it's simple and requires only a small amount of change to lower file systems. Any idea in which you have to record the whiteouts using an actual file or inode is more cumbersome.
You have it right - the major advantage is a possible simplification to rename().
Quoted 8 lines
Quoted 2 lines
Recently, David Woodhouse suggested implementing fallthrus as full-length symlinks with a special flag.
Where does this "special flag" go? Is it persistent? Is it new? Would that mean having to change lower file systems to teach them about this flag?
Is there a way of doing it w/o having to change lower f/s code at all? That'll be a major advantage if possible.
I can't think of a way to do it without using up namespace - but perhaps there is some part of the symlink target namespace that has no valid meaning that we could use instead, like Arnd's self symlink.
-VAL
Quoted 4 lines · Valerie Aurora
I can't think of a way to do it without using up namespace - but perhaps there is some part of the symlink target namespace that has no valid meaning that we could use instead, like Arnd's self symlink.
I think it's impossible to declare that, since the symlink namespace is fair game for userspace do use as it sees fit.
Emacs and Mozilla lockfiles being but one example...
[dwmw2@macbook ~]$ ls -l .#foo.html .mozilla/firefox/b8v9tyu0.default/lock
lrwxrwxrwx. 1 dwmw2 dwmw2 44 2009-11-17 19:16 .#foo.html -> [email protected]:1258450900 lrwxrwxrwx. 1 dwmw2 dwmw2 19 2009-11-17 09:51 .mozilla/firefox/b8v9tyu0.default/lock -> 90.155.92.212:+3756
I wouldn't want to bet that no userspace app has ever come up with the idea of using a symlink-to-self as a marker for something. I don't think it's good practice for us to be trying to play games like that.
Signature · 2 lines
David Woodhouse Open Source Technology Centre [email protected] Intel Corporation
Quoted 11 lines · David Woodhouse
Quoted 4 lines · Valerie Aurora
I can't think of a way to do it without using up namespace - but perhaps there is some part of the symlink target namespace that has no valid meaning that we could use instead, like Arnd's self symlink.
I think it's impossible to declare that, since the symlink namespace is fair game for userspace do use as it sees fit.
Emacs and Mozilla lockfiles being but one example...
Quoted 7 lines
[dwmw2@macbook ~]$ ls -l .#foo.html .mozilla/firefox/b8v9tyu0.default/lock
lrwxrwxrwx. 1 dwmw2 dwmw2 44 2009-11-17 19:16 .#foo.html -> [email protected]:1258450900 lrwxrwxrwx. 1 dwmw2 dwmw2 19 2009-11-17 09:51 .mozilla/firefox/b8v9tyu0.default/lock -> 90.155.92.212:+3756I wouldn't want to bet that no userspace app has ever come up with the idea of using a symlink-to-self as a marker for something. I don't think it's good practice for us to be trying to play games like that.
That's my feeling too. I don't see anyway to cleanly implement fallthrus (or whiteouts) without explicit support from the file system on the writable layer. Fortunately it doesn't take much support.
-VAL
In message <20091117194300.GD17822@shell>, Valerie Aurora writes:
[...]
Quoted 5 lines
That's my feeling too. I don't see anyway to cleanly implement fallthrus (or whiteouts) without explicit support from the file system on the writable layer. Fortunately it doesn't take much support.
-VAL
So, if understood you right, these symlinks are an optimization to simplify and improve the performance of rename()s in r-o layers. Ok.
And, people prefer to avoid self-referencing symlinks so as to prevent breaking some userland code that might depend on it.
But, this does *not* then eliminate the need to have whiteouts supported in every major f/s that can serve as the writeable layer, right? You still need those.
BTW, we might try to figure out a way to use these symlinks to optimize any copyup that's not strictly necessary. A rename() doesn't change the file's data, hence this symlink idea is suitable. But also, there are other meta-data changes to a file which don't affect its data (chmod, chown, chgrp, etc.), for which a symlink would be suitable. This would require that we could easily change the meta-data of the symlink itself, and return *that* metadata in the upper inode, while using the lower file's data for read().
Erez.
Quoted 11 lines · Erez Zadok
In message <20091117194300.GD17822@shell>, Valerie Aurora writes:
[...]Quoted 5 lines
That's my feeling too. I don't see anyway to cleanly implement fallthrus (or whiteouts) without explicit support from the file system on the writable layer. Fortunately it doesn't take much support.
-VAL
So, if understood you right, these symlinks are an optimization to simplify and improve the performance of rename()s in r-o layers. Ok.
Quoted 6 lines
And, people prefer to avoid self-referencing symlinks so as to prevent breaking some userland code that might depend on it.
But, this does *not* then eliminate the need to have whiteouts supported in every major f/s that can serve as the writeable layer, right? You still need those.
Yes. The tradeoff is namespace pollution vs. some new code, and the consensus is pretty firmly against namespace pollution.
Quoted 8 lines
BTW, we might try to figure out a way to use these symlinks to optimize any copyup that's not strictly necessary. A
rename()doesn't change the file's data, hence this symlink idea is suitable. But also, there are other meta-data changes to a file which don't affect its data (chmod, chown, chgrp, etc.), for which a symlink would be suitable. This would require that we could easily change the meta-data of the symlink itself, and return *that* metadata in the upper inode, while using the lower file's data forread().
I like this idea. Copying up the file's data in chown(), etc. is an enormous pain and hard to work into the existing code path. It might be possible to do with this with the directory entry-based approach as well.
-VAL
Quoted 9 lines · Valerie Aurora
Quoted 8 lines
BTW, we might try to figure out a way to use these symlinks to optimize any copyup that's not strictly necessary. A
rename()doesn't change the file's data, hence this symlink idea is suitable. But also, there are other meta-data changes to a file which don't affect its data (chmod, chown, chgrp, etc.), for which a symlink would be suitable. This would require that we could easily change the meta-data of the symlink itself, and return that metadata in the upper inode, while using the lower file's data forread().
Quoted 4 lines
I like this idea. Copying up the file's data in
chown(), etc. is an enormous pain and hard to work into the existing code path. It might be possible to do with this with the directory entry-based approach as well.
I guess we can even support strict atime updates with that, which would be even more painful to do with copyup because they happen more frequently than other inode changes. AFAIK the consensus for other union mount implementations was always that strictatime cannot be sanely done, or not done persistantly.
Arnd <><
Quoted 15 lines · Arnd Bergmann
Quoted 9 lines · Valerie Aurora
Quoted 8 lines
BTW, we might try to figure out a way to use these symlinks to optimize any copyup that's not strictly necessary. A
rename()doesn't change the file's data, hence this symlink idea is suitable. But also, there are other meta-data changes to a file which don't affect its data (chmod, chown, chgrp, etc.), for which a symlink would be suitable. This would require that we could easily change the meta-data of the symlink itself, and return that metadata in the upper inode, while using the lower file's data forread().Quoted 4 lines
I like this idea. Copying up the file's data in
chown(), etc. is an enormous pain and hard to work into the existing code path. It might be possible to do with this with the directory entry-based approach as well.
Quoted 5 lines
I guess we can even support strict atime updates with that, which would be even more painful to do with copyup because they happen more frequently than other inode changes. AFAIK the consensus for other union mount implementations was always that strictatime cannot be sanely done, or not done persistantly.
Okay, this seems really worthwhile to try then.
It seems like we can, as before, create a per-fs DT_FALLTHRU file type (since we're out of bits for the VFS-level file type). Then, instead of reusing the file systems' "normal" directory entry code, we reuse the symlink code. As an example, with the current code, ext2_fallthru_dentry() is a lot of copy-n-paste from ext2_add_link(); in the new version it would look a lot like or call ext2_symlink().
-VAL
Quoted 7 lines · Valerie Aurora
Quoted 3 lines · Erez Zadok
But, this does *not* then eliminate the need to have whiteouts supported in every major f/s that can serve as the writeable layer, right? You still need those.
Yes. The tradeoff is namespace pollution vs. some new code, and the consensus is pretty firmly against namespace pollution.
You could also use the xattr namespace (trusted.*) to hide that pollution. Lots of filesystems already support xattrs, so this might be easier still than adding new fs features.
Thanks,
Miklos
Valerie Aurora:
Quoted 17 lines
Fallthrus were invented as a placeholders for
readdir()on a union-mounted directory - basically, to use the top-level file system'sreaddir()cookie mechanism. Fallthrus are persistent directory entries and are implemented by the underlying file system - such as ext2 or tmpfs - in whatever way it sees fit. We've implemented them for ext2 in two ways: as a regular directory entry with a magic inode number, and as a regular directory entry with a special file type.Recently, David Woodhouse suggested implementing fallthrus as full-length symlinks with a special flag. The interesting thing about this idea is that it could theoretically let us rename a file from the low level file system to another place in the low-level file system without copying the contents of the file up. Basically, we can arbitrarily swizzle the namespace of the low-level by maintaining a set of symlinks above.
Quoted 1 line
Is this useful? Is it implementable?
I think the idea of fallthru entry is good, even if it is implemented as a special symlink.
How do you think about the file paths in /proc/pid/maps and /proc/pid/fd?
They refer the file paths, and some apps depend upon these path. I remember that the package manager in debian didn't work when the path is wrong. (But I don't know whether it is true still).
Will FS have to support such case?
J. R. Okajima
Quoted 29 lines · On Wed, Nov 18, 2009 at 02:47:15PM +0900, [email protected] wrote:
Valerie Aurora:
Quoted 17 lines
Fallthrus were invented as a placeholders for
readdir()on a union-mounted directory - basically, to use the top-level file system'sreaddir()cookie mechanism. Fallthrus are persistent directory entries and are implemented by the underlying file system - such as ext2 or tmpfs - in whatever way it sees fit. We've implemented them for ext2 in two ways: as a regular directory entry with a magic inode number, and as a regular directory entry with a special file type.Recently, David Woodhouse suggested implementing fallthrus as full-length symlinks with a special flag. The interesting thing about this idea is that it could theoretically let us rename a file from the low level file system to another place in the low-level file system without copying the contents of the file up. Basically, we can arbitrarily swizzle the namespace of the low-level by maintaining a set of symlinks above.
Quoted 1 line
Is this useful? Is it implementable?
I think the idea of fallthru entry is good, even if it is implemented as a special symlink.
How do you think about the file paths in/proc/pid/mapsand/proc/pid/fd?
They refer the file paths, and some apps depend upon these path. I remember that the package manager in debian didn't work when the path is wrong. (But I don't know whether it is true still).
Quoted 1 line
Will FS have to support such case?
My current thinking is that an overlay shouldn't be mounted over /proc or other special file systems. I think that a writable layer should only be mounted over exactly one file system, no submounts allowed.
-VAL
Valerie Aurora:
Quoted 1 line · On Wed, Nov 18, 2009 at 02:47:15PM +0900, [email protected] wrote:
:::
Quoted 8 lines
Quoted 8 lines
I think the idea of fallthru entry is good, even if it is implemented as a special symlink.
How do you think about the file paths in/proc/pid/mapsand/proc/pid/fd?
They refer the file paths, and some apps depend upon these path. I remember that the package manager in debian didn't work when the path is wrong. (But I don't know whether it is true still).
Quoted 5 lines
Quoted 1 line
Will FS have to support such case?
My current thinking is that an overlay shouldn't be mounted over /proc or other special file systems. I think that a writable layer should only be mounted over exactly one file system, no submounts allowed.
No, that is not what I mean.
When a fallthru entry is implemented as a kind of a symlink, what will they look like via /proc/pid/{fd,maps,cwd,exe}?
For example,
$ ln -s /bin/sleep /tmp/awake $ /tmp/awake 100 & [1] 32524 $ ls -l /proc/32524/exe lrwxrwxrwx 1 jro jro 0 Nov 25 11:25 exe -> /bin/sleep*
It shows the target path (real filename) even if I execute it as a symlink name.
When I invoke /UnionMount/fallthru/a.out, and if /proc/pid/exe shows something like /real/filesystem/a.out, then some app (debian start-stop-deamon(8) or something) may not work, I am afraid.
Sorry if my poor English made you misunderstood. But I hope I could make my question clear now.
J. R. Okajima
Quoted 4 lines · On Wed, 2009-11-25 at 11:36 +0900, [email protected] wrote:
No, that is not what I mean.
When a fallthru entry is implemented as a kind of a symlink, what will they look like via/proc/pid/{fd,maps,cwd,exe}?
In that respect, and many others, it would behave more like a cross-device hardlink than a symlink.
In fact, I was originally thinking that it would have something like an nfsfh for the target inode, rather than a device (or fsid?) and pathname.
Signature · 2 lines
David Woodhouse Open Source Technology Centre [email protected] Intel Corporation