private mounts

Comments are appreciated. If there are no vetoes agains the patch, I think it's suitable for -mm.

Quoted 2 lines

Comments are appreciated. If there are no vetoes agains the patch, I think it's suitable for -mm.

Vetoed. Having suid application with different pathname resolution than that of parent just because it is suid is not acceptable. I'm sorry, but breaking hell knows how many existing applications is not an option.

This simple patch adds support for private (or invisible) mounts. The rationale is to allow mounts to be private for a user but still in the global namespace.

Quoted 3 lines

This simple patch adds support for private (or invisible) mounts. The rationale is to allow mounts to be private for a user but still in the global namespace.

As mentioned in the last -fsdevel thread a few times the idea of per-user mounts is fundamentally flawed. Crossing a namespace boundary must be explicit - using clone or a new unshare() syscall.

Also mentioned in that thread quite a few times is the fact the the clone() and unshare() modell does not solve people's requirements.

Quoted 2 lines

Also mentioned in that thread quite a few times is the fact the the clone() and unshare() modell does not solve people's requirements.

Could we please get of references to requirements without a rationale? There's quite enough of that from Carrion-Grade Linux crowd, TYVM.

I believe the point is:

1. Person is logged from client Y to server X, and mounts something on $HOME/mnt/private (that's on X).

2. On client Y, person does "scp X:mnt/private/secrets.txt ." and wants it to work.

The second operation is a separate login to the first.

Quoted 9 lines

I believe the point is:

1. Person is logged from client Y to server X, and mounts something on $HOME/mnt/private (that's on X).

2. On client Y, person does "scp X:mnt/private/secrets.txt ." and wants it to work.

The second operation is a separate login to the first.

Solution?

Quoted 8 lines

Quoted 8 lines

I believe the point is:

1. Person is logged from client Y to server X, and mounts something on $HOME/mnt/private (that's on X).

2. On client Y, person does "scp X:mnt/private/secrets.txt ." and wants it to work.

Quoted 3 lines

Quoted 1 line

The second operation is a separate login to the first.

Solution?

... is the same as for the same question with "set of mounts" replaced with "environment variables".

No. You can't set "mount environment" in scp.

Otherwise your analogy is nice, but misses a few points. The usage of mounts that we are talking about is much more dynamic than usage of environment variables.

Quoted 1 line

No. You can't set "mount environment" in scp.

Of course you can. It does execute the obvious set of rc files.

Quoted 3 lines

Otherwise your analogy is nice, but misses a few points. The usage of mounts that we are talking about is much more dynamic than usage of environment variables.

What the hell are you smoking and just how are you using shell?

Quoted 4 lines · Miklos Szeredi

Quoted 1 line

No. You can't set "mount environment" in scp.

Of course you can. It does execute the obvious set of rc files.

Don't think so. ftp server and sftp server sure as hell don't.

Quoted 4 lines · Miklos Szeredi

Quoted 1 line

No. You can't set "mount environment" in scp.

Of course you can. It does execute the obvious set of rc files.

Don't think so. ftp server and sftp server sure as hell don't.

Quoted 5 lines

Quoted 3 lines

Otherwise your analogy is nice, but misses a few points. The usage of mounts that we are talking about is much more dynamic than usage of environment variables.

What the hell are you smoking and just how are you using shell?

Maybe differently from you :). It's not that often that I have to tweak environment variables. They are usually set by scripts.

However if you write me a script that reads my mind as to which server I want to mount with sshfs at which time, I give you all my respect.

Quoted 1 line

Don't think so. ftp server and sftp server sure as hell don't.

Sftp sure *DOES*. It is invoked by shell, which is not run as login one, but even non-login shell sources an rc file.

Quoted 2 lines

However if you write me a script that reads my mind as to which server I want to mount with sshfs at which time, I give you all my respect.

I can't write a script that reads your mind. But I sure can write a script that finds out what you mounted in the other shells (with help of a little wrapper around the mount command).

Quoted 6 lines

Quoted 2 lines

However if you write me a script that reads my mind as to which server I want to mount with sshfs at which time, I give you all my respect.

I can't write a script that reads your mind. But I sure can write a script that finds out what you mounted in the other shells (with help of a little wrapper around the mount command).

How do you bind mount it from a different namespace? You _do_ need bind mount, since a new mount might require password input, etc...

Quoted 3 lines

I can't write a script that reads your mind. But I sure can write a script that finds out what you mounted in the other shells (with help of a little wrapper around the mount command).

How do you bind mount it from a different namespace? You _do_ need bind mount, since a new mount might require password input, etc...

Quoted 4 lines

Quoted 3 lines

I can't write a script that reads your mind. But I sure can write a script that finds out what you mounted in the other shells (with help of a little wrapper around the mount command).

Quoted 2 lines

How do you bind mount it from a different namespace? You _do_ need bind mount, since a new mount might require password input, etc...

Not nessecarily. The filesystem gets called into ->get_sb for every mount, and can then decided whether to return an existing superblock instance or setup a new one. If the credentials for the new mount match an old one it can just reuse it. (e.g. for block based filesystem it will always reuse right now)

Quoted 8 lines

Quoted 2 lines

How do you bind mount it from a different namespace? You _do_ need bind mount, since a new mount might require password input, etc...

Not nessecarily. The filesystem gets called into ->get_sb for every mount, and can then decided whether to return an existing superblock instance or setup a new one. If the credentials for the new mount match an old one it can just reuse it. (e.g. for block based filesystem it will always reuse right now)

And if the credentials are checked in userspace (sshfs)?

Well, if you can find a way to tell the userspace FUSE daemon to know that the mount is being done by the same user as the existing mount, you don't need (or want) to check the credentials - you want the FUSE daemon to tell the kernel code which superblock to reuse.

Quoted 4 lines

Well, if you can find a way to tell the userspace FUSE daemon to know that the mount is being done by the same user as the existing mount, you don't need (or want) to check the credentials - you want the FUSE daemon to tell the kernel code which superblock to reuse.

It sounds very _very_ complicated compared to just using bind mounts.

And maybe the user _does_ want a new connection to the same server (for whatever reason). Why should we _force_ a sharing of superblocks?

Actually, in terms of complexity, it's not much different from using bind mounts.

Quoted 2 lines

Actually, in terms of complexity, it's not much different from using bind mounts.

As has been suggested by Pavel, bind mounting foreign namespaces could just be done with a new bind_fd(fd, path) syscall and file descriptor passing with SCM_RIGHTS.

That sounds to me orders of magnitude less complex (on the kernel side at least) than sb sharing.

But you don't need a new system call to bind an fd.

"mount --bind /proc/self/fd/N mount_point" works, try it.

Quoted 3 lines

But you don't need a new system call to bind an fd.

"mount --bind /proc/self/fd/N mount_point" works, try it.

Ahh, yes :)

Still proc_check_root() has to be relaxed, to allow dereferencing link under a different namespace. Maybe the check should be skipped for capable(CAP_SYS_ADMIN) or similar.

What do people think about that?

Quoted 3 lines

Quoted 1 line

"mount --bind /proc/self/fd/N mount_point" works, try it.

What do people think about that?

To me it looks like an atrocious hack that works only because of the way the implementation is done and not really by design.

From fs/namespace.c:do_loopback, the function which does bind mounts:

Quoted 8 lines · Olivier Galibert

Quoted 3 lines

Quoted 1 line

"mount --bind /proc/self/fd/N mount_point" works, try it.

What do people think about that?

To me it looks like an atrocious hack that works only because of the way the implementation is done and not really by design.

From fs/namespace.c:do_loopback, the function which does bind mounts:

	if (check_mnt(nd->mnt) && (!recurse || check_mnt(old_nd.mnt))) {

check_mnt() verifies that a mountpoint is in the same namespace as the current process. recurse is set for --rbind mounts, but not --bind mounts.

Notice how old_nd.mnt is explicitly _not_ checked for being in the current namespace when doing --bind?

That says to me that Al thought about this case, and coded for it...

(I'm still not clear why the check_mnt() calls are needed at all, though).

-- Jamie
-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Quoted 3 lines

But you don't need a new system call to bind an fd.

"mount --bind /proc/self/fd/N mount_point" works, try it.

Ahh, yes :)

Still proc_check_root() has to be relaxed, to allow dereferencing link under a different namespace.

Maybe the check should be skipped for
capable(CAP_SYS_ADMIN) or similar.

Not necessary.

Why not have the FUSE daemon keep open a file descriptor for the directory it's mounted on, and have it sent that to new would-be mounters of the same directory using a unix domain socket (rather as Pavel suggested)?

No. The check is to prevent processes in chroot jails from accessing directories outside their jail. Even CAP_SYS_ADMIN processes must be forbidden from doing that.

But proc_check_root is unnecessarily strict, in that it prevents a process from traversing into a "child" namespace.

IMHO, a better security restriction anyway would be for processes in chroot jails to not be able to see processes outside the jail in /proc
- only processes inside the jail should be visible. I think everyone agrees that would be best.

If that were implemented, then proc_check_root would be redundant and could be removed entirely.

But proc_check_root is unnecessarily strict, in that it prevents a process from traversing into a "child" namespace.

IMHO, a better security restriction anyway would be for processes in chroot jails to not be able to see processes outside the jail in /proc
- only processes inside the jail should be visible. I think everyone agrees that would be best.

As someone pointed out, CAP_SYS_ADMIN processes can already escape the chroot jail with CLONE_NEWNS. (fd=open("."); clone(CLONE_NEWNS); [child:] fchdir(fd); chdir(".."))

Quoted 2 lines

Quoted 2 lines

How about fixing fchdir, so it checks whether you gone outside the tree under current->fs->rootmnt? Should be fairly easy to do.

Quoted 7 lines

Quoted 6 lines

Quoted 3 lines

I can't write a script that reads your mind. But I sure can write a script that finds out what you mounted in the other shells (with help of a little wrapper around the mount command).

How do you bind mount it from a different namespace? You _do_ need bind mount, since a new mount might require password input, etc...

Quoted 5 lines

Not nessecarily. The filesystem gets called into ->get_sb for every mount, and can then decided whether to return an existing superblock instance or setup a new one. If the credentials for the new mount match an old one it can just reuse it. (e.g. for block based filesystem it will always reuse right now)

And if the credentials are checked in userspace (sshfs)?

Quoted 15 lines

Quoted 14 lines

Quoted 8 lines

Quoted 4 lines

I can't write a script that reads your mind. But I sure can write a script that finds out what you mounted in the other shells (with help of a little wrapper around the mount command).

How do you bind mount it from a different namespace? You _do_ need bind mount, since a new mount might require password input, etc...

Not nessecarily. The filesystem gets called into ->get_sb for every mount, and can then decided whether to return an existing superblock instance or setup a new one. If the credentials for the new mount match an old one it can just reuse it. (e.g. for block based filesystem it will always reuse right now)

Quoted 1 line

And if the credentials are checked in userspace (sshfs)?

The it needs to call to userspace in ->get_sb..

Quoted 1 line · Miklos Szeredi

No. You can't set "mount environment" in scp.

Of course you can. It does execute the obvious set of rc files.

Quoted 9 lines · Al Viro

Quoted 3 lines · Miklos Szeredi

No. You can't set "mount environment" in scp.

Of course you can. It does execute the obvious set of rc files.

It doesn't work for the specified use-scenario. The reason is that there is no command or system call that can be executed from those rc files to join an existing namespace.

He wants to do this:

1. From client, login to server and do a usermount on $HOME/private.

2. From client, login to server and read the files previously mounted.

-- Jamie

Quoted 4 lines · Al Viro

Quoted 1 line · Miklos Szeredi

No. You can't set "mount environment" in scp.

Of course you can. It does execute the obvious set of rc files.

It doesn't work for the specified use-scenario. The reason is that there is no command or system call that can be executed from those rc files to join an existing namespace.

He wants to do this:

1. From client, login to server and do a usermount on $HOME/private.

2. From client, login to server and read the files previously mounted.

He wants to do this:

1. From client, login to server and do a usermount on $HOME/private.

2. From client, login to server and read the files previously mounted.

Of course you can. It does execute the obvious set of rc files.

Is there a possibility for a process to change its namespace to another existing one? That would be needed to have a per-user namespace you go to from rc files or pam.

Quoted 1 line · Al Viro

Of course you can. It does execute the obvious set of rc files.

Is there a possibility for a process to change its namespace to another existing one? That would be needed to have a per-user namespace you go to from rc files or pam.

Quoted 1 line

No. You can't set "mount environment" in scp.

Of course you can. It does execute the obvious set of rc files.

Apparently, Windows Longhorn will include something called "transactional NTFS". It's explained pretty well in

http://blogs.msdn.com/because_we_can/

Basically, a process can create a fs transaction, and all fs changes made between start of the transaction and commit are atomical - meaning nothing is visible until commit, and if commit fails, everything is rolled back.

Sound useful... Although there are no service pack installs that could fail in Linux, the same thing could be useful in rpm, yum, almost anything.

What do you think?

I think I've wanted something like that for _years_ in unix.

It's an old, old idea, and I've often wondered why we haven't implemented it.

Quoted 5 lines · Jamie Lokier

I think I've wanted something like that for _years_ in unix.

It's an old, old idea, and I've often wondered why we haven't implemented it.

I thought it is possible to rather easily to implement this on top of non-transactional FS (albeit I didn't try) and there is no need to overcomplicate an FS. Just implement a specialized user-space library and utilize it.

Artem B. Bityuckiy wrote:

No. A transaction means that _all_ processes will see the whole transaction or not.

It does _not_ mean that only a subset of programs, which happen to link with a particular user-space library, will see it or not.

For example, you can use transactions for distro package management: a whole update of a package would be a single transaction, so that at no time does any program see an inconsistent set of files. See why _every_ process in the system must have the same view?

[ If you meant that you can implement it with a user-space library that every process in the system links to, that's true. But it would rather misses the point of having filesystems in the kernel at all :) ]

Hmm, so the whole point to implement transactions in the kernel space is to do the transactions in a way that nobody can see any intermediate inconsistent state ?

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

"Jamie" == Jamie Lokier <[email protected]> writes:

Jamie> No. A transaction means that _all_ processes will see the Jamie> whole transaction or not.
This is really hard. How do you handle the case where process X starts a transaction modifies files a, b & c, but process Y has file b open for writing, and never lets it go? Or the file gets unlinked?

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

"Jamie" == Jamie Lokier <[email protected]> writes:

Jamie> No. A transaction means that _all_ processes will see the Jamie> whole transaction or not.

This is really hard. How do you handle the case where process X starts a transaction modifies files a, b & c, but process Y has file b open for writing, and never lets it go? Or the file gets unlinked?

Jamie> For example, you can use transactions for distro package Jamie> management: a whole update of a package would be a single Jamie> transaction, so that at no time does any program see an Jamie> inconsistent set of files. See why _every_ process in the Jamie> system must have the same view?

What about programs that are already open and running?

It might be doable in some sense, but I can see that details are really hard to get right. Esp without breaking existing Unix semantics.

ty den 26.04.2005 Klokka 11:01 (-0400) skreiv John Stoffel:

Quoted 8 lines

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

"Jamie" == Jamie Lokier <[email protected]> writes:

Jamie> No. A transaction means that _all_ processes will see the Jamie> whole transaction or not.

This is really hard. How do you handle the case where process X starts a transaction modifies files a, b & c, but process Y has file b open for writing, and never lets it go? Or the file gets unlinked?

That is why implementing it as a form of lock makes sense.

Quoted 11 lines

Jamie> For example, you can use transactions for distro package Jamie> management: a whole update of a package would be a single Jamie> transaction, so that at no time does any program see an Jamie> inconsistent set of files. See why _every_ process in the Jamie> system must have the same view?

What about programs that are already open and running?

It might be doable in some sense, but I can see that details are really hard to get right. Esp without breaking existing Unix semantics.

Wrong.

Cheers,
Trond
--
Trond Myklebust <[email protected]>

-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

I was wondering if it was also important to commit the changes to persistent storage once the transaction has been completed or is it just important that all processes see the changes atomically. I mean, is it also important that before any of the processes see the result of the transaction, or before the 'transacting' process knows the transaction is complete, the changes must be flushed to persistent storage? That would be all the more reason to implement it in kernel space...

I was wondering if it was also important to commit the changes to persistent storage once the transaction has been completed or is it just important that all processes see the changes atomically. I mean, is it also important that before any of the processes see the result of the transaction, or before the 'transacting' process knows the transaction is complete, the changes must be flushed to persistent storage? That would be all the more reason to implement it in kernel space...

Quoted 6 lines

Jamie> No. A transaction means that _all_ processes will see the Jamie> whole transaction or not.

This is really hard. How do you handle the case where process X starts a transaction modifies files a, b & c, but process Y has file b open for writing, and never lets it go? Or the file gets unlinked?

That is why implementing it as a form of lock makes sense.

The problem with making them exclusive locks is that you halt the system for the duration of the transaction. If it's a big transaction such as updating 1000 files for a package update, that blocks a lot of programs for a long time, and it's not necessary.

Quoted 4 lines · Jamie Lokier

The problem with making them exclusive locks is that you halt the system for the duration of the transaction. If it's a big transaction such as updating 1000 files for a package update, that blocks a lot of programs for a long time, and it's not necessary.

Surely we'll anyway block others if we have a kernel-level transaction support? What is the difference in which layer to block?

No. Why would you block? You can have transactions without blocking other processes.

When updating, say, the core-utils package (which contains cat), there's no reason why a program which executes "cat" should have to block during the update. It can simply execute the old one until the new one is committed at the end of the update.

It's analogous to RCU for protecting kernel data structures without blocking readers.

Quoted 8 lines · Jamie Lokier

No. Why would you block? You can have transactions without blocking other processes.

When updating, say, the core-utils package (which contains cat), there's no reason why a program which executes "cat" should have to block during the update. It can simply execute the old one until the new one is committed at the end of the update.

Quoted 3 lines

It's analogous to RCU for protecting kernel data structures without blocking readers.

Hmm, can't we implement a user-space locking system which admits of readers during transactions? I gues we can.

The problem with implementing in userland, as was already said in the thread, is, that if some process does not use the library, it can completely mess it up. It is only safe in kernel.

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

"Jamie" == Jamie Lokier <[email protected]> writes:

Jamie> No. A transaction means that _all_ processes will see the Jamie> whole transaction or not.

This is really hard. How do you handle the case where process X starts a transaction modifies files a, b & c, but process Y has file b open for writing, and never lets it go? Or the file gets unlinked?

What about programs that are already open and running?

It might be doable in some sense, but I can see that details are really hard to get right. Esp without breaking existing Unix semantics.

It's even harder without kernel support! :)

Quoted 1 line

It's even harder without kernel support! :)

This seems to implement something in userspace which might be interesting:
http://users.auriga.wearlab.de/~alb/libjio/

Quoted 8 lines · John Stoffel

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

"Jamie" == Jamie Lokier <[email protected]> writes:

Jamie> No. A transaction means that _all_ processes will see the Jamie> whole transaction or not.

This is really hard. How do you handle the case where process X starts a transaction modifies files a, b & c, but process Y has file b open for writing, and never lets it go? Or the file gets unlinked?

Then it starts to depend on what kind of transactions you want to implement.

You can say that a transaction isn't allowed when a process has one of the files opened for writing. Or you can say a transaction is equivalent to calling all of the I/O system calls at once. You can also decide if you want the reads and directory lookups performed in the transactions to become prerequisites for the transaction completing (so it's aborted if another process writes to those file regions or changes the directory structure in a way which breaks a prerequisite), or if you want those to lock the things which are read for the duration of the transaction, or even just ignore reads for transaction purposes. Or, you can say that transactions are limited to just directory structure, and not file contents (that's good enough for package management), or you can say they're limited to just file contents (that's good enough for databases and text file edits).

Etc, etc, quite a lot of semantic choices.

Quoted 5 lines

What about programs that are already open and running?

It might be doable in some sense, but I can see that details are really hard to get right. Esp without breaking existing Unix semantics.

It's even harder without kernel support! :)

Quoted 27 lines · Jamie Lokier

Quoted 8 lines · John Stoffel

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

"Jamie" == Jamie Lokier <[email protected]> writes:

Jamie> No. A transaction means that _all_ processes will see the Jamie> whole transaction or not.

This is really hard. How do you handle the case where process X starts a transaction modifies files a, b & c, but process Y has file b open for writing, and never lets it go? Or the file gets unlinked?

Then it starts to depend on what kind of transactions you want to implement.

You can say that a transaction isn't allowed when a process has one of the files opened for writing. Or you can say a transaction is equivalent to calling all of the I/O system calls at once. You can also decide if you want the reads and directory lookups performed in the transactions to become prerequisites for the transaction completing (so it's aborted if another process writes to those file regions or changes the directory structure in a way which breaks a prerequisite), or if you want those to lock the things which are read for the duration of the transaction, or even just ignore reads for transaction purposes. Or, you can say that transactions are limited to just directory structure, and not file contents (that's good enough for package management), or you can say they're limited to just file contents (that's good enough for databases and text file edits).

Quoted 1 line

Etc, etc, quite a lot of semantic choices.

How do we specify which calls belong to a transaction? By some kind of extra file handle?

I'd think having global per-process transaction is not the best way. So I think we should have some kind of transaction handle (probably in the file handle space) and a way to say that a syscall is done within a transaction. To avoid duplicating all syscalls, we could have set_active_transaction() operation.

Quoted 8 lines

How do we specify which calls belong to a transaction? By some kind of extra file handle?

I'd think having global per-process transaction is not the best way. So I think we should have some kind of transaction handle (probably in the file handle space) and a way to say that a syscall is done within a transaction. To avoid duplicating all syscalls, we could have set_active_transaction() operation.

That's more or less what NTFS does. See the example at
http://blogs.msdn.com/because_we_can/

If we have transactions, then I'd like to be able to do this from a shell:

I'd also like to write inside a single C program:

It's useful, and there is no good reason to disallow that.

Nonetheless, there's a need for some kind of transaction handles. A file descriptor representing a transaction seems like a natural fit.

Quoted 4 lines · Jamie Lokier

I think I've wanted something like that for _years_ in unix.

It's an old, old idea, and I've often wondered why we haven't implemented it.

I thought it is possible to rather easily to implement this on top of non-transactional FS (albeit I didn't try) and there is no need to overcomplicate an FS. Just implement a specialized user-space library and utilize it.

Atomicity is difficult, because you have lots of caches each with their own bits of state (e.g., the inode/dentry caches). Assuming your transaction is committed that isn't so much of a problem, but once you have on rollback you need to undo any changes to those caches.

Quoted 4 lines · Charles P. Wright

Atomicity is difficult, because you have lots of caches each with their own bits of state (e.g., the inode/dentry caches). Assuming your transaction is committed that isn't so much of a problem, but once you have on rollback you need to undo any changes to those caches.

I guess if you do synchronization before unlocking all is OK. Roll-back means deleting partially written things and restore old things, then run fsyncs. Whys this may be not enough?

Atomicity is difficult, because you have lots of caches each with their own bits of state (e.g., the inode/dentry caches). Assuming your transaction is committed that isn't so much of a problem, but once you have on rollback you need to undo any changes to those caches.

Isolation (this is the property that says that concurrent transactions should be the same as if there was a serial execution) is also tricky to get right. A transaction can touch any number of objects, and user- applications may not respect any lock ordering --- which means you will have deadlocks, and you must detect and resolve them (probably by aborting one of the transactions).

Quoted 4 lines · Jamie Lokier

I think I've wanted something like that for _years_ in unix. It's an old, old idea, and I've often wondered why we haven't implemented it.

I thought it is possible to rather easily to implement this on top of non-transactional FS (albeit I didn't try) and there is no need to overcomplicate an FS. Just implement a specialized user-space library and utilize it.

Apparently, Windows Longhorn will include something called "transactional NTFS". It's explained pretty well in

http://blogs.msdn.com/because_we_can/

Basically, a process can create a fs transaction, and all fs changes made between start of the transaction and commit are atomical - meaning nothing is visible until commit, and if commit fails, everything is rolled back.

Sound useful... Although there are no service pack installs that could fail in Linux, the same thing could be useful in rpm, yum, almost anything.

What do you think?

Apparently, Windows Longhorn will include something called "transactional NTFS". It's explained pretty well in

http://blogs.msdn.com/because_we_can/

Basically, a process can create a fs transaction, and all fs changes made between start of the transaction and commit are atomical - meaning nothing is visible until commit, and if commit fails, everything is rolled back.

Sound useful... Although there are no service pack installs that could fail in Linux, the same thing could be useful in rpm, yum, almost anything.

What do you think?

Quoted 8 lines

Quoted 8 lines

I believe the point is:

1. Person is logged from client Y to server X, and mounts something on $HOME/mnt/private (that's on X).

2. On client Y, person does "scp X:mnt/private/secrets.txt ." and wants it to work.

Quoted 3 lines

Quoted 1 line

The second operation is a separate login to the first.

Solution?

... is the same as for the same question with "set of mounts" replaced with "environment variables".

Quoted 8 lines · Al Viro

Quoted 8 lines

Quoted 8 lines

I believe the point is:

1. Person is logged from client Y to server X, and mounts something on $HOME/mnt/private (that's on X).

2. On client Y, person does "scp X:mnt/private/secrets.txt ." and wants it to work.

Quoted 6 lines

Quoted 3 lines

Quoted 1 line

The second operation is a separate login to the first.

Solution?

... is the same as for the same question with "set of mounts" replaced with "environment variables".

Not quite.

After changing environment variables in .profile, you can copy them to other shells using ". ~/.profile".

There is no analogous mechanism to copy namespaces.

I agree with you that Miklos' patch is not the right way to do it.

Much better is the proposal to make namespaces first-class objects, that can be switched to. Then users can choose to have themselves a namespace containing their private mounts, if they want it, with login/libpam or even a program run from .profile switching into it.

While users can be allowed to create their own namespaces which affect the path traversal of their _own_ directories, it's important that the existence of such namespaces cannot affect path traversal of other directories such as /etc, or /autofs/whatever - and that creation of namespaces by a user cannot prevent the unmounting of a non-user filesystem either.

The way to do that is shared subtrees, or something along those lines.

Here is one possible implementation:

As far as I can tell, namespaces are equivalent to predicates attached to every mount - the predicate being "this mount intercepts path traversal at this point if current namespace == X".

It makes sense, when users can create namespaces for themselves, that the predicate be changed to "this mount valid if [list of current namespace and all parent namespaces] contains X". Parent namespace means the namespace from which a CLONE_NS namespace inherits.

Then it would be safe (i.e. secure) to allow ordinary users to use CLONE_NS for the purpose of establishing private namespace(s), within which they can mount things on directories they own. But those users would continue to see mounts & unmounts done by the system in other directories such as /mnt and /autofs. Effectively this confines the new namespace to only affecting directories owned by the user.

That would work properly with suid programs, properly with autofs and also manual system-wide administration, and it is general enough that it doesn't force any particular policy. Also, it would be usable for partial sharing of resources in virtual server and chroot scenarios. What's not to like? :)

-- Jamie
-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Quoted 5 lines

Much better is the proposal to make namespaces first-class objects, that can be switched to. Then users can choose to have themselves a namespace containing their private mounts, if they want it, with login/libpam or even a program run from .profile switching into it.

Quoted 8 lines

While users can be allowed to create their own namespaces which affect the path traversal of their _own_ directories, it's important that the existence of such namespaces cannot affect path traversal of other directories such as /etc, or /autofs/whatever - and that creation of namespaces by a user cannot prevent the unmounting of a non-user filesystem either.

The way to do that is shared subtrees, or something along those lines.

Right. Adding to it. To begin with the system namespace has all its entire tree shared. So when a new namespace is cloned, the new namespace can see any new mount/unmount/binds done in the system namespace as well. (System namespace is the first initial namespace created by default).

Any private mounts done by the user in his private-namespace will first make that part of the tree private first and then will continue with the mount. Otherwise the private mount will end up showing in the system namespace(since it is shared).

Quoted 3 lines

... is the same as for the same question with "set of mounts" replaced with "environment variables".

Not quite.

After changing environment variables in .profile, you can copy them to other shells using ". ~/.profile".

There is no analogous mechanism to copy namespaces.

I agree with you that Miklos' patch is not the right way to do it.

Much better is the proposal to make namespaces first-class objects, that can be switched to. Then users can choose to have themselves a namespace containing their private mounts, if they want it, with login/libpam or even a program run from .profile switching into it.

While users can be allowed to create their own namespaces which affect the path traversal of their _own_ directories, it's important that the existence of such namespaces cannot affect path traversal of other directories such as /etc, or /autofs/whatever - and that creation of namespaces by a user cannot prevent the unmounting of a non-user filesystem either.

The way to do that is shared subtrees, or something along those lines.

Quoted 4 lines

Quoted 3 lines

... is the same as for the same question with "set of mounts" replaced with "environment variables".

Quoted 8 lines

Not quite.

After changing environment variables in .profile, you can copy them to other shells using ". ~/.profile".

There is no analogous mechanism to copy namespaces.

I agree with you that Miklos' patch is not the right way to do it.

I'm not sure that it is either. But, see bellow...

Quoted 4 lines

Much better is the proposal to make namespaces first-class objects, that can be switched to. Then users can choose to have themselves a namespace containing their private mounts, if they want it, with login/libpam or even a program run from .profile switching into it.

It would be good if it could be done just in libpam. But that would require every libpam user to call into it after the fork() or whatever, so unshare() and join_namespace() don't mess up the server running environment.

If not, then it would mean modifying numerous programs, having these modifications integrated, then having distributions pick up the changes, etc. I would imagine quite a long cycle for this to be acutally useful.

Quoted 7 lines

While users can be allowed to create their own namespaces which affect the path traversal of their _own_ directories, it's important that the existence of such namespaces cannot affect path traversal of other directories such as /etc, or /autofs/whatever - and that creation of namespaces by a user cannot prevent the unmounting of a non-user filesystem either.

Quoted 1 line

The way to do that is shared subtrees, or something along those lines.

Yes, but we would be achieving essentially the same as my patch, just with more complexity. And my patch achieves what FUSE does in 2 lines of code, namely hide the mount from other users by returning -EACCESS in case fsuid does not mach the mount owner.

I aggree that your solution is more flexible, but it's also hugely more complex. If somebody want's to implement it, fine. But don't expect me to do it, unless some company hires my for fs development (hint, hint ;)

Thanks,
Miklos
-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

I have not yet sure how invisible mount can be used to solve the FUSE problem.

Again my understanding of the basic requirement of FUSE is:

1. A user being able to setup his own VFS-mount environment which is only visible to the user.
2. The same user being able to see exactly the same VFS-mount environment from any login session.

Quoted 4 lines

Much better is the proposal to make namespaces first-class objects, that can be switched to. Then users can choose to have themselves a namespace containing their private mounts, if they want it, with login/libpam or even a program run from .profile switching into it.

It would be good if it could be done just in libpam. But that would require every libpam user to call into it after the fork() or whatever, so unshare() and join_namespace() don't mess up the server running environment.

If not, then it would mean modifying numerous programs, having these modifications integrated, then having distributions pick up the changes, etc. I would imagine quite a long cycle for this to be acutally useful.

They do. The *HAVE* to do! The 'session' stage modifies the environment, so it must be done after the fork. So if it, in addition to environment, modifies namespace, it won't make a difference.

Quoted 8 lines

Quoted 8 lines

Quoted 8 lines

I believe the point is:

1. Person is logged from client Y to server X, and mounts something on $HOME/mnt/private (that's on X).

2. On client Y, person does "scp X:mnt/private/secrets.txt ." and wants it to work.

Quoted 6 lines

Quoted 3 lines

Quoted 1 line

The second operation is a separate login to the first.

Solution?

... is the same as for the same question with "set of mounts" replaced with "environment variables".

Not quite.

After changing environment variables in .profile, you can copy them to other shells using ". ~/.profile".

There is no analogous mechanism to copy namespaces.

Quoted 5 lines

Quoted 2 lines

... is the same as for the same question with "set of mounts" replaced with "environment variables".

Not quite.

Quoted 4 lines

After changing environment variables in .profile, you can copy them to other shells using ". ~/.profile".

There is no analogous mechanism to copy namespaces.

Actually, after you add right mount xyzzy /foo lines into .profile, you can just . ~/.profile ;-).

Quoted 10 lines

Quoted 9 lines

Quoted 2 lines

... is the same as for the same question with "set of mounts" replaced with "environment variables".

Not quite.

After changing environment variables in .profile, you can copy them to other shells using ". ~/.profile".

There is no analogous mechanism to copy namespaces.

Quoted 2 lines

Actually, after you add right mount xyzzy /foo lines into .profile, you can just . ~/.profile ;-).

Is there a mount command that can do that? We're talking about private mounts - invisible to other namespaces, which includes the other shells.

If there was a /proc/NNN/namespace, that would do the trick :)

Quoted 8 lines

Quoted 2 lines

Actually, after you add right mount xyzzy /foo lines into .profile, you can just . ~/.profile ;-).

Is there a mount command that can do that? We're talking about private mounts - invisible to other namespaces, which includes the other shells.

If there was a /proc/NNN/namespace, that would do the trick :)

Sounds like the solution, then. I do not think Al Viro is going to kill you for /proc/NNN/namespace...

Looking closer, I think we already have it.

It's called /proc/NNN/root.

Does chroot into /proc/NNN/root cause the chroot'ing process to adopt the namespace of NNN? Looking at the code, I think it does.

So no new system calls are needed. A daemon to hand out per-user namespaces (or any other policy) can be written using existing kernels, and those namespaces can be joined using chroot.

That's the theory anyway. It's always possible I misread the code (as I don't use namespaces and don't have tools handy to try them).

Quoted 2 lines

Does chroot into /proc/NNN/root cause the chroot'ing process to adopt the namespace of NNN? Looking at the code, I think it does.

I've been thinking about this a bit more...would you even need chroot? (wouldn't exposing chroot functionality to a user incur additional security risk? I guess it would be okay as long as you were only chrooting to one of your other process' roots?)

It's called /proc/NNN/root.

So no new system calls are needed. A daemon to hand out per-user namespaces (or any other policy) can be written using existing kernels, and those namespaces can be joined using chroot.

That's the theory anyway. It's always possible I misread the code (as I don't use namespaces and don't have tools handy to try them).

Quoted 9 lines

It's called /proc/NNN/root.

So no new system calls are needed. A daemon to hand out per-user namespaces (or any other policy) can be written using existing kernels, and those namespaces can be joined using chroot.

That's the theory anyway. It's always possible I misread the code (as I don't use namespaces and don't have tools handy to try them).

Should have checked myself before posting my previous reply -- but this doesn't seem to work. /proc/NNN/root is represented as a symlink, but when you CLONE_NS and then try to look at another one of your process' /proc/NNN/root the link doesn't seem to have a target and you get permission denied on all accesses.

I haven't looked at the underlying procfs code, but adapting procfs for this sort of purpose feels wrong.

Quoted 10 lines · Eric Van Hensbergen

Quoted 9 lines

It's called /proc/NNN/root.

So no new system calls are needed. A daemon to hand out per-user namespaces (or any other policy) can be written using existing kernels, and those namespaces can be joined using chroot.

That's the theory anyway. It's always possible I misread the code (as I don't use namespaces and don't have tools handy to try them).

Quoted 5 lines

Should have checked myself before posting my previous reply -- but this doesn't seem to work. /proc/NNN/root is represented as a symlink, but when you CLONE_NS and then try to look at another one of your process' /proc/NNN/root the link doesn't seem to have a target and you get permission denied on all accesses.

I've looked at the code. Look in fs/proc/base.c (Linux 2.6.10), proc_root_link().

I don't see anything there to prevent you from traversing to the mounts in the other namespace.

So why is it failing? Any idea?

Quoted 7 lines

I've looked at the code. Look in fs/proc/base.c (Linux 2.6.10), proc_root_link().

I don't see anything there to prevent you from traversing to the mounts in the other namespace.

So why is it failing? Any idea?

Since you are traversing a symlink, you will be traversing the symlink in the context of traversing process's namespace.

If process 'x' is traversing /proc/y/root , the lookup for the root dentry will happen in the context of process x's namespace, and not process y's namespace. Hence process 'x' wont really get into the namespace of the process y.

Quoted 8 lines · Ram

Quoted 7 lines

I've looked at the code. Look in fs/proc/base.c (Linux 2.6.10), proc_root_link().

I don't see anything there to prevent you from traversing to the mounts in the other namespace.

So why is it failing? Any idea?

Quoted 7 lines

Since you are traversing a symlink, you will be traversing the symlink in the context of traversing process's namespace.

If process 'x' is traversing /proc/y/root , the lookup for the root dentry will happen in the context of process x's namespace, and not process y's namespace. Hence process 'x' wont really get into the namespace of the process y.

Lookups don't happen in the context of a namespace.

They happen in the context of a vfsmnt. And the switch to a new vfsmnt is done by matching against (dentry,parent-vfsmnt) pairs. current->namespace is only checked for mount & unmount operations, not for path lookups.

Which means proc_root_link, when it switches to the vfsmnt at the root of the other process, should traverse into the tree of vfsmnts which make up the other namespace.

-- Jamie

Quoted 4 lines

Which means proc_root_link, when it switches to the vfsmnt at the root of the other process, should traverse into the tree of vfsmnts which make up the other namespace.

Yes. But proc_check_root() in proc_pid_follow_link() is failing the traversal, because it is expecting the root vfsmount of the traversed process to belong to the vfsmount tree of the traversing process. In other words its expecting them to be both in the same namespace.

The permissions get denied by this code in proc_check_root():

Removing the check makes chroot enter the tree under the other process's namespace. However it does not actually change the namespace, hence mount/umount won't work.

So joinig a namespace does need a new syscall unfortunately.

Quoted 10 lines · Pavel Machek

Quoted 9 lines

Quoted 2 lines

... is the same as for the same question with "set of mounts" replaced with "environment variables".

Not quite.

After changing environment variables in .profile, you can copy them to other shells using ". ~/.profile".

There is no analogous mechanism to copy namespaces.

Quoted 2 lines

Actually, after you add right mount xyzzy /foo lines into .profile, you can just . ~/.profile ;-).

Is there a mount command that can do that? We're talking about private mounts - invisible to other namespaces, which includes the other shells.

If there was a /proc/NNN/namespace, that would do the trick :)

Quoted 18 lines · Jamie Lokier

Quoted 5 lines · Pavel Machek

Quoted 5 lines

Quoted 2 lines

... is the same as for the same question with "set of mounts" replaced with "environment variables".

Not quite.

Quoted 7 lines

Quoted 4 lines

After changing environment variables in .profile, you can copy them to other shells using ". ~/.profile".

There is no analogous mechanism to copy namespaces.

Actually, after you add right mount xyzzy /foo lines into .profile, you can just . ~/.profile ;-).

Is there a mount command that can do that? We're talking about private mounts - invisible to other namespaces, which includes the other shells.

Quoted 1 line

If there was a /proc/NNN/namespace, that would do the trick :)

I don't think you need a /proc/NNN/namespace, /proc/NNN/mounts already contains a mount table. It's pretty trivial to write a small shellscript to parse that, compare with the current namespace and do all mount/umounts to make them fit the other processes namespace. Real problem here are filesystems that don't implement ->show_options or do so only partially so that some options are lost.

-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Quoted 2 lines

Could we please get of references to requirements without a rationale? There's quite enough of that from Carrion-Grade Linux crowd, TYVM.

The rationale has been explained in that thread. E.g. this quote from Jamie Lokier in an answer to you:

Quoted 6 lines

I believe the point is:

1. Person is logged from client Y to server X, and mounts something on $HOME/mnt/private (that's on X).

Quoted 4 lines

2. On client Y, person does "scp X:mnt/private/secrets.txt ." and wants it to work.

The second operation is a separate login to the first.

Solution?

Quoted 5 lines

Quoted 5 lines

I believe the point is:

1. Person is logged from client Y to server X, and mounts something on $HOME/mnt/private (that's on X).

Quoted 6 lines

Quoted 4 lines

2. On client Y, person does "scp X:mnt/private/secrets.txt ." and wants it to work.

The second operation is a separate login to the first.

Solution?

just restart your shell. Same way you do that after adjusting $PATH.