Skip to contentSkip to conversation

VFS: FS CoW using redirection

RFC Al Boldi 5 participants 10 messages Jul 8, 2006 – Aug 28, 2006 VFS

lore
Share & tools
lore

Al Boldi

author
#0 lore raw

Copy on Write is a neat way to quickly achieve a semi-clustered system, by mounting any shared FS read-only and redirecting writes to some perClient FS.

Would this redirection be easy to implement into the VFS?

Thanks!

--
Al

#1 lore raw
Quoted 7 lines · Al Boldi

Copy on Write is a neat way to quickly achieve a semi-clustered system, by mounting any shared FS read-only and redirecting writes to some perClient FS.

Would this redirection be easy to implement into the VFS?

There are a variety of solutions that have been proposed or are available to do this sort of thing. You may want to start by looking at unionfs and mapfs. There are also folks looking at doing this from the block layer (look at the dm-userspace + cowd as well as evms and lvm snapshots).

-eric

author
#2 lore raw
Quoted 5 lines · Eric Van Hensbergen
Quoted 4 lines · Al Boldi

Copy on Write is a neat way to quickly achieve a semi-clustered system, by mounting any shared FS read-only and redirecting writes to some perClient FS.

Quoted 7 lines
Quoted 1 line

Would this redirection be easy to implement into the VFS?

There are a variety of solutions that have been proposed or are available to do this sort of thing. You may want to start by looking at unionfs and mapfs. There are also folks looking at doing this from the block layer (look at the dm-userspace + cowd as well as evms and lvm snapshots).

The idea here, is to do things completely transparent while being as unintrusive as possible, by relying on the current FS infrastructure, w/o introducing another VFS.

Consider something simple like this:

VFS - anyFS1 (r/w) used normally, unless ENotFound, then redirect read to
\ anyFS2, or CoW from anyFS2 to anyFS1.
anyFS2 (r/o) used normally.

i.e: Does the VFS provide hooks for perMount Handlers?

Thanks!

--
Al

author
#3 lore raw
Quoted 18 lines · Al Boldi
Quoted 5 lines · Eric Van Hensbergen
Quoted 4 lines · Al Boldi

Copy on Write is a neat way to quickly achieve a semi-clustered system, by mounting any shared FS read-only and redirecting writes to some perClient FS.

Quoted 7 lines
Quoted 1 line

Would this redirection be easy to implement into the VFS?

There are a variety of solutions that have been proposed or are available to do this sort of thing. You may want to start by looking at unionfs and mapfs. There are also folks looking at doing this from the block layer (look at the dm-userspace + cowd as well as evms and lvm snapshots).

The idea here, is to do things completely transparent while being as unintrusive as possible, by relying on the current FS infrastructure, w/o introducing another VFS.

Quoted 7 lines

Consider something simple like this:

VFS - anyFS1 (r/w) used normally, unless ENotFound, then redirect read to
\ anyFS2, or CoW from anyFS2 to anyFS1.
anyFS2 (r/o) used normally.

i.e: Does the VFS provide hooks for perMount Handlers?

Silence is probably an indication that the VFS does not provide hooks.

The question then:

1. Would it be difficult to introduce perMount Handler hooks into the VFS?

2. Would it be easier to replace the current VFS with one that offers a more flexible/open configuration?

3. Would these changes be welcome, or is the VFS considered to be a forbidden zone?

Thanks!

--
Al

#4 lore raw
In reply to Al Boldi
Quoted 6 lines · Al Boldi

Consider something simple like this:

VFS - anyFS1 (r/w) used normally, unless ENotFound, then redirect read to
\ anyFS2, or CoW from anyFS2 to anyFS1.
anyFS2 (r/o) used normally.

That concept is known as union mount. Jan Blunck did some patches in that direction, you might be able to find them in the archives. If not, just send him a mail.

Jörn

Signature · 6 lines
...one more straw can't possibly matter...
-- Kirby Bakken
-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#5 lore raw
Quoted 11 lines · Jörn Engel
Quoted 6 lines · Al Boldi

Consider something simple like this:

VFS - anyFS1 (r/w) used normally, unless ENotFound, then redirect read to
\ anyFS2, or CoW from anyFS2 to anyFS1.
anyFS2 (r/o) used normally.

That concept is known as union mount. Jan Blunck did some patches in that direction, you might be able to find them in the archives. If not, just send him a mail.

Or you can give Unionfs a try: http://www.unionfs.org

Josef "Jeff" Sipek.

Signature · 5 lines
The box said "Windows XP or better required". So I installed Linux.
-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
author
#6 lore raw
Quoted 11 lines · Josef Sipek
Quoted 10 lines · Jِrn Engel
Quoted 5 lines · Al Boldi

Consider something simple like this:

VFS - anyFS1 (r/w) used normally, unless ENotFound, then redirect read
to \ anyFS2, or CoW from anyFS2 to anyFS1.
anyFS2 (r/o) used normally.

That concept is known as union mount. Jan Blunck did some patches in that direction, you might be able to find them in the archives. If not, just send him a mail.

Thanks for the pointer!

So what was the rejecting theme?

Quoted 1 line

Or you can give Unionfs a try: http://www.unionfs.org

UnionFS is great, but it incurs additional overhead, as it lives below the real VFS. What could be really great, is to move some basic functionality abstractions from UnionFS into VFS proper.

Thanks!

--
Al

-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#7 lore raw
Quoted 2 lines · Al Boldi

So what was the rejecting theme?

I don't believe there was one. Jan simply didn't push much, so noone was forced to resist him. And noone else needed union mount enough to push Jan.

Quoted 5 lines
Quoted 1 line

Or you can give Unionfs a try: http://www.unionfs.org

UnionFS is great, but it incurs additional overhead, as it lives below the real VFS. What could be really great, is to move some basic functionality abstractions from UnionFS into VFS proper.

Welcome to Jan's work. :)

If you want to make this vision happen, one of the missing pieces is a method for copyup, an in-kernel copying routine. Unionfs needs is just the same as Jan's patches do and in the past Linus didn't like my approach of using sendfile for it. You could take a stab at the splice code and see how that can be used for copyup.

Jörn

Signature · 8 lines
Write programs that do one thing and do it well. Write programs to work
together. Write programs to handle text streams, because that is a
universal interface.
-- Doug MacIlroy
-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#8 lore raw
Quoted 1 line · Jörn Engel

On Sat, 26 August 2006 22:05:21 +0300, Al Boldi wrote:

...

Quoted 11 lines
Quoted 5 lines
Quoted 1 line

Or you can give Unionfs a try: http://www.unionfs.org

UnionFS is great, but it incurs additional overhead, as it lives below the real VFS. What could be really great, is to move some basic functionality abstractions from UnionFS into VFS proper.

If you want to make this vision happen, one of the missing pieces is a method for copyup, an in-kernel copying routine. Unionfs needs is just the same as Jan's patches do and in the past Linus didn't like my approach of using sendfile for it. You could take a stab at the splice code and see how that can be used for copyup.

The thing with union mounts/unionfs is that some of the functionality makes sense to have in a file system while other parts make sense to have in the VFS - the way I see it, namespace related bits should be in VFS while persistent state should be done on the file system level.

Josef "Jeff" Sipek.

Signature · 6 lines
If I have trouble installing Linux, something is wrong. Very wrong.
		- Linus Torvalds
-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#9 lore raw
In reply to Al Boldi
Quoted 5 lines · Al Boldi

Copy on Write is a neat way to quickly achieve a semi-clustered system, by mounting any shared FS read-only and redirecting writes to some perClient FS.

Would this redirection be easy to implement into the VFS?

I've used unionfs that way.