relinquish_fs() syscall

relinquish_fs() is a replacement for the chroot("/var/empty") technique currently used for privilege-separated daemons such as recent OpenSSH. Rather than using a directory on the normal filesystem it places the process in an alternate namespace. This namespace has one filesystem of type "nullfs" -- it's empty and immutable.

This has several benefits:

* Considerably safer against root users in cage

Normal chroot's are trivial for privileged users to break out of - these tasks don't work in the namespace. You can't create a directory to do the "chroot foo; cd ../../..; chroot ." trick. You can't create device nodes or mount /proc anywhere (I added an extra check to do_mount() that even prevents a mount on top of '/')

* Can be used by non-root users!

Normally chroot is restricted to root (to prevent the escape mentioned above and also to avoid confusing setuid programs about their environment) Obviously setuid binaries are not an issue here since execve() will be impossible after relinquish_fs() is called.

This is a big deal for privilege separation; currently it's hard to implement except in a daemon that starts its life as root. Now the same techniques can be used by any process.

Imagine, for example, a jpeg decoder that after opening its input and output files called relinquish_fs(). Now if the decoder has a flaw and is fed an exploit it is not able to do much of anything. And although adding privsep is hard work this technique can, in theory, be extended to anything that processes untrusted data from the network (html parsers, ssh client, etc)

Of course, even after chroot() or relinquish_fs() an exploited process can still do bad things (chew up CPU, fork bomb, implement a rogue TCP proxy service, etc) but at least with this technique it makes it hard for an exploit to do lasting damage to the OS.

I reused the i386 syscall number from the (apparently now gone) set_altroot syscall for now.

Also, this is the first time I've dug into the VFS internals; let me know if I'm doing something terribly wrong.

Patch is versus 2.6.10-rc2-bk9

-Mitch

diff -ur linux-2.6.10-rc2-bk9-VIRGIN/arch/i386/kernel/entry.S linux-2.6.10-rc2-bk9/arch/i386/kernel/entry.S
@@ -860,7 +860,7 @@
 	.long sys_mq_getsetattr
 	.long sys_ni_syscall		/* reserved for kexec */
 	.long sys_waitid
-	.long sys_ni_syscall		/* 285 */ /* available */
+	.long sys_relinquish_fs		/* 285 */
 	.long sys_add_key
 	.long sys_request_key
 	.long sys_keyctl

diff -ur linux-2.6.10-rc2-bk9-VIRGIN/include/asm-i386/unistd.h linux-2.6.10-rc2-bk9/include/asm-i386/unistd.h
@@ -290,7 +290,7 @@
 #define __NR_mq_getsetattr	(__NR_mq_open+5)
 #define __NR_sys_kexec_load	283
 #define __NR_waitid		284
-/* #define __NR_sys_setaltroot	285 */
+#define __NR_sys_relinquish_fs	285
 #define __NR_add_key		286
 #define __NR_request_key	287
 #define __NR_keyctl		288

diff -ur linux-2.6.10-rc2-bk9-VIRGIN/include/linux/syscalls.h linux-2.6.10-rc2-bk9/include/linux/syscalls.h
@@ -506,4 +506,6 @@
 asmlinkage long sys_keyctl(int cmd, unsigned long arg2, unsigned long arg3,
 			   unsigned long arg4, unsigned long arg5);
 
+asmlinkage long sys_relinquish_fs(void);
+
 #endif

Pardon. Its equally ineffectual. It might take someone a week longer to write the exploit but an hour after that its no different.

A priviledged user can ioperm/iopl their way out.

That doesn't do name lookup, character set translation, or time (and a few other things).

Imagine a jpeg decoder using an SELinux policy.

OK, good point, at least on i386/x86_64. So before jailing itself a task will have to take CAP_SYS_RAWIO out of its permitted set. That shouldn't be too bad of a restriction for most programs to live with.

OK, could you please describe other attacks against it then?

Alan - have you looked at privsep implementation in, say, opensshd. The way privsep works is that you have two processes communicating over a UNIX domain socket. One then jails itself and handles all the hairy untrusted data. The unjailed process handles requests from inside as needed. So if the program needs to do DNS lookups then your privsep protocol would include a primitive for doing that.

Some information on openssh privsep, including the USENIX paper:
http://www.citi.umich.edu/u/provos/ssh/privsep.html

Frankly, if a program is touching lots of untrusted data AND it also needs lots of external stuff then it's a perfect candidate for privsep.

SELinux is great but it's a pretty orthoginal issue. There's no reason the two can't coexist.

-Mitch

With CAP_SYS_RAWIO I can ask the IDE controller to DMA into the kernel as one example. Without it should be sane. However if you take away all the capabilities then you don't need the other changes because mount works just fine.

Yes I realise that but you also need to realise that glibc has a lot of supporting baggage it likes to find if its going to function sanely. Yes you can deal with it but SELinux can deal with it better.

I don't see it as orthogonal except for the "done by user" aspect. "foo" isnt allowed to open files or chdir is SELinux policy at work (or for that matter the rather nice bitmap filtered syscall stuff Andrea was talking about some time back).

BTW: you also have to deal with fchdir() and potentially AF_UNIX sockets.

Can you really do that on normal file descriptors? Weird. I'd have thought you'd need to open /dev/hd* to do that.

Oh, and you also need to revoke CAP_SYS_PTRACE to prevent a compromised process from taking over a process inside the jail

True; for the root-user case you can build an equivalent jail as long as you
* remove CAP_SYS_RAWIO
* remove CAP_SYS_ADMIN to prevent mount/umount
* make the root somehow immutable (make mountpoint r/o; ext2 attribute; whatever)

I probably shouldn't have even mentioned the jail-root case in my description; my only point is that relinquish_fs() is at least as good as the currently used chroot("/var/empty") in every way. It's also stronger in some ways. Obviously a rogue process running with a full CAP_SYS_* set can do all kinds of bad things even inside a jail (up to and including a reboot)

The real point of the patch is to add a method giving this same power to unprivileged processes, functionality which is currently lacking but useful. The existing namespace support makes it almost trivial to provide.

And how is this different then people using chroot() currently? The people doing privsep designs understand the limits libc and can work within them.

Yes and that's the point. relinquish_fs() is a tool for defensive programming. SELinux is a tool for sysadmins and distribution vendors to enforce policy. Ideally they would both be used -- defense in depth is a good thing.

Is AF_UNIX in a separate namespace? My understanding (from reading unix_find_other()) is that unless you can create a UNIX socket in your filesystem you're going to have trouble creating new UNIX sockets. I think some *NIXes worked differently but linux seems sane in this regard. So you'd have to trick your privsep-peer outside the jail to send you a directory fd over a unix-domain socket you already had open before you jailed yourself.

-Mitch

inb/outb after iopl.

iirc there are anonymous unix sockets...

That was already discussed earlier in the thread.

Ah, I see now -- the sun_path[0]=='\0' code. I'll have to take a look at that; probably just need to add a check to prevent jailed processes from using those sockets (since they're supposed to be in a "null" namespace) Will investigate later this week.

It looks like this is also a weakness in code that currently uses chroot("/var/empty") It's not the end of the world since it still requires a cooperating unjailed process on the same host as the jailed one to pass in a fd which is quite an obstacle in most scenarios. Still, it's something that should be protected against.

-Mitch

You would probably want a "private" AF_UNIX namespace too. The fact its a single namespace for "anonymous" AF_UNIX and the \0 trick is used is really legacy unix compatibility. Having multiple such namespaces is certainly
doable. It's the same problem as the shared memory, semaphore and message
queue objects have because they fall out of the filesystem namespace. Posix
has fixed these but very few apps use the new forms.

Also you need to look at fchdir(). If I accidentally pass you a file handle to a directory (or maybe to a file in reiser4 world ?) you can fchdir() out of the chroot.

Alan

Looking back at the patched together set of man pages, old printouts, stale books, tricks picked up from other people's code, and occasional groveling through the kernel and library sources I've used over the years to figure out how to program Linux:

Is there a book out there that actually says how we're SUPPOSED to be doing all this stuff? Which APIs are actually recommended for use under 2.6 and later? (Does O'reilley publish an up to date version of the POSIX API, with examples?)

Rob

So someone finds a way to break into the jailed process. This is used to feed some hairy exploit to the unjailed process that expects "clean" data from the jailed process. Same as before, only now they need a two-stage exploit.

You can jail a process doing a "dangerous" job, but you can't really jail a "hairy" data stream. Not if data is expected to emerge from the jail someday.

Helge Hafting

OK, so by your logic firewalls are useless (since you can just exploit the firewall and then the host) Oh and you might as well run all your daemons as root (since an attacker can just use some other exploit to gain root later)

The entire point of a privsep design is that the unjailed process treats all data from the jailed process as untrusted. If the attacker gains full control of the jailed process it should still be a challenge to trick the unjailed process into doing something harmful.

Exactly! Now in order for the attacker to win they need to find a programming error in the jailed process AND the unjailed process. This is how defense in depth works.

That's a bogus argument - the data exiting the jail can come out in a non-hairy format that is less error-prone to process. You do all the complicated crypto/compression/parsing/etc inside the jail where a programming errors cause less damage.

Think about a client communicating with a random network hosts over an SSL connection - you can move the SSL engine into the jailed process. Now if your SSL's ASN.1 parser has a buffer overflow and you connect to a malicious server it can't immediately compromise your account.

-Mitch