Skip to contentSkip to conversation

O_NOLINK for open()

DISCUSSION Brent Casavant 6 participants 12 messages Sep 12, 2007 – Sep 14, 2007

lore
Share & tools
lore
#0 lore raw

Bear with me, I present the problem I'm trying to solve first, and then propose O_NOLINK as a solution. Responses to either my shared memory problem or the O_NOLINK idea would be most appreciated.

I've run into a rather unusual set of circumstances calling for use of shared memory, but haven't found a bulletproof solution which can be used under Linux.

I have a need to share memory between two unrelated processes, but without the ability for any other process to attach to it, for security reasons (e.g. encryption keys and plaintext will be exchanged via this shared memory area). These unrelated processes will in all likelihood not be owned by the same user (i.e. one of them is a daemon).

System V shmem is right out because the IPC key is publicly visible and there is no combination of permissions which will allow sharing the segment with just one other process (or at least just one other user). To my knowledge Linux's implementation doesn't provide ACLs for SysV shmem. SGI's proposed XPMEM suffers from the same problems for my purposes.

I had a mistaken notion that multiple mmap's of /dev/zero using a common file descriptor (which could be passed between the processes via an AF_UNIX socket) would result in shared memory, but apparently my understanding of /dev/zero mmap's was subtlely wrong, so it appears this won't work.

I could mmap a temporary tmpfs file (tmpfs so that if there is a machine crash no sensitive data persists) which is created with permissions of 0, immediately unlink it, and pass the file descriptor through an AF_UNIX socket. This does open up a very small window of vulnerability if another process is able to chmod the file and open it before the unlink.

However, it occurs to me that this problem goes away if there were a method create a file in an unlinked state to begin with. However there does not appear to be any such mechanism in Linux's open() interface. A bit of Googling indicates that Hurd has an O_NOLINK flag which seems to accomplish what I'd need, but Linux doesn't implement such a flag. There was some discussion of this in various lkml threads in the past, but none that went anywhere. Perhaps the best an explaining why other mechanisms (i.e. directories with particular permissions aren't a solution) is:

	http://marc.info/?l=linux-kernel&m=93032806224160&w=2

Of course it is reasonable to take the stance that if root or the daemon's user are malicious, all bets are off anyway.

Fully understanding this, it was suggested to me that I could fire this problem off to lkml to see if anyone has a solution I haven't already dismissed, or as a trial balloon for considering adding O_NOLINK to open().

Thoughts?

Brent

Signature · 3 lines
Brent Casavant                          All music is folk music.  I ain't
[email protected]                        never heard a horse sing a song.
Silicon Graphics, Inc.                    -- Louis Armstrong
#1 lore raw
Quoted 8 lines · Brent Casavant

I could mmap a temporary tmpfs file (tmpfs so that if there is a machine crash no sensitive data persists) which is created with permissions of 0, immediately unlink it, and pass the file descriptor through an AF_UNIX socket. This does open up a very small window of vulnerability if another process is able to chmod the file and open it before the unlink.

To avoid this window, typically one creates a temporary directory first, with 0700 permissions. Make sure you verify that you actually created the directory, and watch out for symlink attacks. Then you create the file in that directory.

This doesn't prevent another process owned by the same user (or root) from attaching, but such a process can ptrace you or touch yoour /proc spaec just as well, so you're screwed anyway (modulo SELinux-type policies.)

Quoted 10 lines

However, it occurs to me that this problem goes away if there were a method create a file in an unlinked state to begin with. However there does not appear to be any such mechanism in Linux's open() interface. A bit of Googling indicates that Hurd has an O_NOLINK flag which seems to accomplish what I'd need, but Linux doesn't implement such a flag. There was some discussion of this in various lkml threads in the past, but none that went anywhere. Perhaps the best an explaining why other mechanisms (i.e. directories with particular permissions aren't a solution) is:

Quoted 2 lines
	http://marc.info/?l=linux-kernel&m=93032806224160&w=2

This link talks about file flags handling. I don't see the relevance to this problem at all. However, this is a very long thread, so if there is anything specific that you want to point to, then please elucidate.

Quoted 2 lines

Of course it is reasonable to take the stance that if root or the daemon's user are malicious, all bets are off anyway.

Yup, see above.

-hpa

#2 lore raw
Quoted 7 lines · H. Peter Anvin
Quoted 1 line · Brent Casavant
	http://marc.info/?l=linux-kernel&m=93032806224160&w=2

This link talks about file flags handling. I don't see the relevance to this problem at all. However, this is a very long thread, so if there is anything specific that you want to point to, then please elucidate.

Oops, my mistake -- I pasted the wrong URL. I meant this thread, this post in particular:

	http://marc.info/?l=linux-kernel&m=88937224115435&w=2

Still, O_NOLINK would seem to be a valuable addition, and greatly simplify secure temporary file creation.

Brent

Signature · 3 lines
Brent Casavant                          All music is folk music.  I ain't
[email protected]                        never heard a horse sing a song.
Silicon Graphics, Inc.                    -- Louis Armstrong
#3 lore raw
Quoted 9 lines · Brent Casavant
Quoted 6 lines · H. Peter Anvin
Quoted 1 line · Brent Casavant
	http://marc.info/?l=linux-kernel&m=93032806224160&w=2

This link talks about file flags handling. I don't see the relevance to this problem at all. However, this is a very long thread, so if there is anything specific that you want to point to, then please elucidate.

Quoted 8 lines

Oops, my mistake -- I pasted the wrong URL. I meant this thread, this post in particular:

	http://marc.info/?l=linux-kernel&m=88937224115435&w=2

Still, O_NOLINK would seem to be a valuable addition, and greatly simplify secure temporary file creation.

Avoiding -- or at least detecting -- symlink racing with mkdir() is relatively simple: run mkdir(), make sure you don't get EEXIST or something like that, lstat() the resulting path -- it should be a directory with all the right modes and ownerships.

I believe -- but I'm not certain -- that mkdtemp() in glibc will do all this for you. If not, I would consider that a glibc bug.

-hpa

#4 lore raw
In reply to Brent Casavant
Quoted 6 lines · Brent Casavant

I could mmap a temporary tmpfs file (tmpfs so that if there is a machine crash no sensitive data persists) which is created with permissions of 0, immediately unlink it, and pass the file descriptor through an AF_UNIX socket. This does open up a very small window of vulnerability if another process is able to chmod the file and open it before the unlink.

Only the owner can chmod a file, so why is that a vulnerability?

Andreas.

Signature · 4 lines
Andreas Schwab, SuSE Labs, [email protected]
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."
#5 lore raw
Quoted 10 lines · Andreas Schwab
Quoted 6 lines · Brent Casavant

I could mmap a temporary tmpfs file (tmpfs so that if there is a machine crash no sensitive data persists) which is created with permissions of 0, immediately unlink it, and pass the file descriptor through an AF_UNIX socket. This does open up a very small window of vulnerability if another process is able to chmod the file and open it before the unlink.

Only the owner can chmod a file, so why is that a vulnerability?

In this particular case because the user may not normally have direct access to some of the data to be contained in that file.

Decryption keys in a key management system, in particular. If the keys are passed over secure network links such that they only ever exist in system RAM, and are not reachable via the filesystem, these keys can be protected from disclosure to the user (short of /proc/#/mem type of tricks). However, if there is even a brief window when the user can gain access to the file, these keys are at risk of disclosure.

The problem can be addressed, in this case, by having the daemon half of the design create these files, however it would provide a bit more flexibility if the client side was also capable of creating them. It's not a make-or-break problem, by any means, but does somewhat motivate an O_NOLINK flag for open().

Brent

P.S. By the way, there doesn't seem to be a way to remove /proc/#/mem
files. That might be an additional nicety -- programs worried about being snooped could unlink their own entry. /dev/mem and /dev/kmem can simply be removed by the sysadmin of such a system. If all of that were done you'd have to resort to attacking crash dumps, core dumps, or via something like kdb to extract "hidden" data.

Signature · 3 lines
Brent Casavant                          All music is folk music.  I ain't
[email protected]                        never heard a horse sing a song.
Silicon Graphics, Inc.                    -- Louis Armstrong
#6 lore raw
Quoted 6 lines · Brent Casavant

P.S. By the way, there doesn't seem to be a way to remove /proc/#/mem
files. That might be an additional nicety -- programs worried about being snooped could unlink their own entry. /dev/mem and /dev/kmem can simply be removed by the sysadmin of such a system. If all of that were done you'd have to resort to attacking crash dumps, core dumps, or via something like kdb to extract "hidden" data.

Give me a break. And learn about ptrace(2). This "unlinking" bullshit buys you zero additional security, both for /proc/*/mem and for /dev/mem (see mknod(2)).

#7 lore raw
Quoted 9 lines · Al Viro
Quoted 6 lines · Brent Casavant

P.S. By the way, there doesn't seem to be a way to remove /proc/#/mem
files. That might be an additional nicety -- programs worried about being snooped could unlink their own entry. /dev/mem and /dev/kmem can simply be removed by the sysadmin of such a system. If all of that were done you'd have to resort to attacking crash dumps, core dumps, or via something like kdb to extract "hidden" data.

Quoted 3 lines

Give me a break. And learn about ptrace(2). This "unlinking" bullshit buys you zero additional security, both for /proc/*/mem and for /dev/mem (see mknod(2)).

Yes, I fully understand that mknod can recreate the nodes -- however only the superuser can do so, and if the superuser is attacking a process all bets are off anyway. OK, so /dev/*mem isn't to worry about, since it's already owned by root. Still, /proc/#/mem is owned by the user, not root, leaving it potentially open to inspection by third party processes.

I'm thinking out loud. Sorry to cause any grief.

My (limited) understanding of ptrace is that a parent-child relationship is needed between the tracing process and the traced process (at least that's what I gather from the man page). This does give cause for concern, and I might have to see what can be done to alleviate this concern. I fully realize that making this design completely unassilable is a fools errand, but closing off as many attack vectors as possible seems prudent.

Signature · 3 lines
Brent Casavant                          All music is folk music.  I ain't
[email protected]                        never heard a horse sing a song.
Silicon Graphics, Inc.                    -- Louis Armstrong
#8 lore raw
Quoted 6 lines · Brent Casavant
Quoted 3 lines · Al Viro

Give me a break. And learn about ptrace(2). This "unlinking" bullshit buys you zero additional security, both for /proc/*/mem and for /dev/mem (see mknod(2)).

Quoted 7 lines

My (limited) understanding of ptrace is that a parent-child relationship is needed between the tracing process and the traced process (at least that's what I gather from the man page). This does give cause for concern, and I might have to see what can be done to alleviate this concern. I fully realize that making this design completely unassilable is a fools errand, but closing off as many attack vectors as possible seems prudent.

Hmm. The solution would appear to be as simple as making the target program set-user-id. As long as as the attacker isn't the superuser (or has CAP_SYS_PTRACE) we should be OK.

Thanks for the heads-up,
Brent

Signature · 3 lines
Brent Casavant                          All music is folk music.  I ain't
[email protected]                        never heard a horse sing a song.
Silicon Graphics, Inc.                    -- Louis Armstrong
#9 lore raw
In reply to Brent Casavant
Quoted 7 lines · Brent Casavant

My (limited) understanding of ptrace is that a parent-child relationship is needed between the tracing process and the traced process (at least that's what I gather from the man page). This does give cause for concern, and I might have to see what can be done to alleviate this concern. I fully realize that making this design completely unassilable is a fools errand, but closing off as many attack vectors as possible seems prudent.

No relationship needed:

strace -p <pid>

MfG
Goswin

#10 lore raw
In reply to Brent Casavant
Quoted 6 lines · Brent Casavant

System V shmem is right out because the IPC key is publicly visible and there is no combination of permissions which will allow sharing the segment with just one other process (or at least just one other user). To my knowledge Linux's implementation doesn't provide ACLs for SysV shmem. SGI's proposed XPMEM suffers from the same problems for my purposes.

SYSV shared memory has the concept of separate creator and owner ID's, so you can share the shmem segment between exactly two users. Just use IPC_SET and set shm_perm.uid to the user ID of the peer process.

Quoted 6 lines

I could mmap a temporary tmpfs file (tmpfs so that if there is a machine crash no sensitive data persists) which is created with permissions of 0, immediately unlink it, and pass the file descriptor through an AF_UNIX socket. This does open up a very small window of vulnerability if another process is able to chmod the file and open it before the unlink.

I think your worries about permissions has been cleared by the other posts, but there is still a problem: the client may call ftruncate() on the file descriptor, and then your daemon will get a nice SIGBUS when it tries to access the shared memory. Handling that gracefully may not be trivial esp. if your daemon is multi-threaded. SYSV shmem is _much_ nicer when you want shared memory between unrelated/untrusted processes.

Gabor

Signature · 4 lines
     ---------------------------------------------------------
     MTA SZTAKI Computer and Automation Research Institute
                Hungarian Academy of Sciences
     ---------------------------------------------------------
#11 lore raw
Quoted 9 lines · Gabor Gombas
Quoted 6 lines · Brent Casavant

System V shmem is right out because the IPC key is publicly visible and there is no combination of permissions which will allow sharing the segment with just one other process (or at least just one other user). To my knowledge Linux's implementation doesn't provide ACLs for SysV shmem. SGI's proposed XPMEM suffers from the same problems for my purposes.

Quoted 3 lines

SYSV shared memory has the concept of separate creator and owner ID's, so you can share the shmem segment between exactly two users. Just use IPC_SET and set shm_perm.uid to the user ID of the peer process.

Hmm. This will work as long as the peer process is running setuid to it's own unique user. Excellent idea! Since I need to make the program setuid to avoid non-priveleged ptrace attacks, this is a terrific solution.

Quoted 6 lines

I think your worries about permissions has been cleared by the other posts, but there is still a problem: the client may call ftruncate() on the file descriptor, and then your daemon will get a nice SIGBUS when it tries to access the shared memory. Handling that gracefully may not be trivial esp. if your daemon is multi-threaded. SYSV shmem is _much_ nicer when you want shared memory between unrelated/untrusted processes.

I'm actually not so concerned about the client -- that code will be trusted as well. The problem I'm trying to solve is preventing any non-priveleged code except the server and client from gaining access to their shared memory area. With the feedback I've received from this thread I think a solid design is emerging, some of which will need to be solved by system configuration by the sysadmin.

Thanks,
Brent

Signature · 3 lines
Brent Casavant                          All music is folk music.  I ain't
[email protected]                        never heard a horse sing a song.
Silicon Graphics, Inc.                    -- Louis Armstrong