shmgetfd idea

In working with ashmem and looking briefly at kdbus' memfd ideas, there's a commonality that both basically act as a method to provide applications with unlinked tmpfs/shmem fds.

In the Android case, its important to have this interface to atomically provide these unlinked tmpfs fds, because they'd like to avoid having tmpfs mounts that are writable by applications (since that creates a potential DOS on the system by applications writing random files that persist after the process has been killed). It also provides better life-cycle management for resources, since as the fds never have named links in the filesystem, their resources are automatically cleaned up when the last process with the fd dies, and there's no potential races between create and unlink with processes being terminated, which avoids the need for cleanup management.

I won't speak for the kdbus use, but my understanding is memfds address similar needs along with being something to connect with other features.

So one idea was maybe we need a new interface. Something like:

int shmgetfd(char* name, size_t size, int shmflg);

Basically this would be very similar to shmget, but would return a file descriptor which could be mapped and passed to other processes to map. Basically very similar to the in-kernel shmem_file_setup() interface.

(Thanks to Akashi-san for initially pointing out the similarity to shmget.)

Of course, shmgetfd on its own wouldn't address the quota issue right away, but it would be fairly easy have a limit for the total number of bytes a process could generate, or some other limiting mechanism.

The probably more major drawback here is that both ashmem and memfd tack on additional features that can be done to the fds.

In ashmems' case it allows for changing the segment's name, and unpinning regions which can then be lazily discarded by the kernel.

For memfd, the extra feature is sealing, which prevents modification of the file when its shared.

In ashmem's case, both vma-naming and volatile ranges are trying to address how the needed features would be generically applied to tmpfs fds (as well as potentially wider uses as well) - so with something like shmgetfd it would provide all the functionality needed. I am not aware of any current plans for memfd's sealing to be similarly worked into a generic concept - the code hasn't even been submitted, so this is too early - but in any case, its important to note none of these plans for generic functionality have been merged or even received with much interest, so I do understand how a proposal for a new interface that only solves half of the needed infrastructure may not be particularly welcome.

So while I do understand the difficulty of trying to create more generic interfaces rather then just creating a new chardev/ioctl interface to a more limited subset of functionality, I do think its worth exploring if we can find a way to share infrastructure at some level (even if its just due-diligence to prove if the more limited scope chardev/ioctl interfaces are widely agreed to be better).

Anyway, I just wanted to submit this sketched out idea as food for thought to see if there was any objection or interest (I've got a draft patch I'll send out once I get a chance to test it). So let me know if you have any feedback or comments.

Quoted 4 lines · John Stultz

Anyway, I just wanted to submit this sketched out idea as food for thought to see if there was any objection or interest (I've got a draft patch I'll send out once I get a chance to test it). So let me know if you have any feedback or comments.

The reason "kdbus-memfd" exists is primarily the sealing.

It would be nice if we can generalize the whole memfd logic, but the shmem allocation facility alone, without the sealing function cannot replace kdbus-memfd.

We would need secure sealing right from the start for the kdbus use case; other than that, there are no specific requirements from the kdbus side.

In the Android case, its important to have this interface to atomically provide these unlinked tmpfs fds, because they'd like to avoid having tmpfs mounts that are writable by applications (since that creates a potential DOS on the system by applications writing random files that persist after the process has been killed). It also provides better life-cycle management for resources, since as the fds never have named links in the filesystem, their resources are automatically cleaned up when the last process with the fd dies, and there's no potential races between create and unlink with processes being terminated, which avoids the need for cleanup management.

Quoted 11 lines · John Stultz

In the Android case, its important to have this interface to atomically provide these unlinked tmpfs fds, because they'd like to avoid having tmpfs mounts that are writable by applications (since that creates a potential DOS on the system by applications writing random files that persist after the process has been killed). It also provides better life-cycle management for resources, since as the fds never have named links in the filesystem, their resources are automatically cleaned up when the last process with the fd dies, and there's no potential races between create and unlink with processes being terminated, which avoids the need for cleanup management.

What about if tmpfs could be restricted to only only O_TMPFILE open()s?
This pretty much amounts to an option to prevent tmpfs from creating
new directory entries.

Thanks for reminding me about O_TMPFILE.. I have it on my list to look into how it could be used.

As for the O_TMPFILE only tmpfs option, it seems maybe a little clunky to me, but possible. If others think this would be preferred over a new syscall, I'll dig in deeper.

Quoted 7 lines · John Stultz

Thanks for reminding me about O_TMPFILE.. I have it on my list to look into how it could be used.

As for the O_TMPFILE only tmpfs option, it seems maybe a little clunky to me, but possible. If others think this would be preferred over a new syscall, I'll dig in deeper.

What is clunky about it? It reuses an existing interface and still points to the specific tmpfs instance that should be populated.

Quoted 10 lines · H. Peter Anvin

Quoted 7 lines · John Stultz

Thanks for reminding me about O_TMPFILE.. I have it on my list to look into how it could be used.

As for the O_TMPFILE only tmpfs option, it seems maybe a little clunky to me, but possible. If others think this would be preferred over a new syscall, I'll dig in deeper.

What is clunky about it? It reuses an existing interface and still points to the specific tmpfs instance that should be populated.

It would require new mount point convention that userland would have to standardize. To me (and admittedly its a taste thing), a new O_TMPFILE-only tmpfs mount point seems to be to be a bigger interface change from an application writers perspective then a new syscall.

But maybe I'm misunderstanding your suggestion?

Quoted 11 lines · John Stultz

Quoted 10 lines · H. Peter Anvin

Quoted 7 lines · John Stultz

Thanks for reminding me about O_TMPFILE.. I have it on my list to look into how it could be used.

As for the O_TMPFILE only tmpfs option, it seems maybe a little clunky to me, but possible. If others think this would be preferred over a new syscall, I'll dig in deeper.

What is clunky about it? It reuses an existing interface and still points to the specific tmpfs instance that should be populated.

Quoted 6 lines

It would require new mount point convention that userland would have to standardize. To me (and admittedly its a taste thing), a new O_TMPFILE-only tmpfs mount point seems to be to be a bigger interface change from an application writers perspective then a new syscall.

But maybe I'm misunderstanding your suggestion?

General purpose Linux has /dev/shm/ for that already, which will not go away anytime soon..

Quoted 2 lines

General purpose Linux has /dev/shm/ for that already, which will not go away anytime soon..

Right, though making /dev/shm/ O_TMPFILE only would likely break things, no?

Quoted 3 lines · John Stultz

Quoted 2 lines

General purpose Linux has /dev/shm/ for that already, which will not go away anytime soon..

Right, though making /dev/shm/ O_TMPFILE only would likely break things, no?

If it isn't, then you already have a writable tmpfs, which is what you said you didn't want.

Quoted 6 lines · H. Peter Anvin

Quoted 3 lines · John Stultz

Quoted 2 lines

General purpose Linux has /dev/shm/ for that already, which will not go away anytime soon..

Right, though making /dev/shm/ O_TMPFILE only would likely break things, no?

If it isn't, then you already have a writable tmpfs, which is what you said you didn't want.

Well, rather then finding a solution exclusively for Android, I'm trying to find an approach that would work more generically.

While classic Linux systems do have writable /dev/shm/, which we *have* to preserve, it seem to me that classic linux systems may some day want to deal with the issues with writable tmpfs that Android has intentionally avoided.

For examples of grumblings on these issues see:
https://bugzilla.redhat.com/show_bug.cgi?id=693253 (and its dup)

Requiring a binary on/off flag for /dev/shm makes it so you have to choose if you are a classic or new-style (android-like) system. By avoiding re-using existing convention via providing a new syscall (or alternatively with your approach, a new yet to be standardized mount point convention), it would allow best practices to be updated, and allow for a slow deprecation of the writable /dev/shm, possibly by limiting permissions to /dev/shm to only legacy applications, etc.

But yes, alternatively classic systems may be able to get around the issues via tmpfs quotas and convincing applications to use O_TMPFILE there. But to me this seems less ideal then the Android approach, where the lifecycle of the tmpfs fds more limited and clear.

And my main point being: Both Android's ashmem and kdbus' memfds are both utilizing these semantics (though maybe they aren't as important/intentional for kdbus?),

so it seems like some generic method
(which would work in both environments) would generally useful.

Again, I really do appreciate your feedback here, and I don't mean to be panning your idea (I'm quite willing to look further into it if others think its the right way)! I just want to explain my point of view and motivations a bit better.

Quoted 5 lines

But yes, alternatively classic systems may be able to get around the issues via tmpfs quotas and convincing applications to use O_TMPFILE there. But to me this seems less ideal then the Android approach, where the lifecycle of the tmpfs fds more limited and clear.

Tmpfs supports no quota, it's all a huge hole and unsafe in that regard on every system today. But ashmem and kdbus, as they are today, are not better.

Quoted 6 lines · Kay Sievers

Quoted 5 lines

But yes, alternatively classic systems may be able to get around the issues via tmpfs quotas and convincing applications to use O_TMPFILE there. But to me this seems less ideal then the Android approach, where the lifecycle of the tmpfs fds more limited and clear.

Quoted 3 lines

Tmpfs supports no quota, it's all a huge hole and unsafe in that regard on every system today. But ashmem and kdbus, as they are today, are not better.

We can fix that aspect in tmpfs. Creating new file objcts outside of filesystems really doesn't make things any better, since our toolbox around this stuff largely revolves around filesystems.

What would be the interface for additional functionality like sealing/unsealing that thing, that no operation can destruct its content as long as there is more than a single owner? That would be a new syscall or fcntl() with specific shmemfd options?

We also need to solve the problem that the inode does not show up in /proc/$PID/fd/, so that nothing can create a new file for it which we don't catch with the "single owner" logic. Or we could determine the "single owner" state from the inode itself?

Quoted 11 lines · Kay Sievers

What would be the interface for additional functionality like sealing/unsealing that thing, that no operation can destruct its content as long as there is more than a single owner? That would be a new syscall or fcntl() with specific shmemfd options?

We also need to solve the problem that the inode does not show up in /proc/$PID/fd/, so that nothing can create a new file for it which we don't catch with the "single owner" logic. Or we could determine the "single owner" state from the inode itself?

If the "single owner" is determined by the file structure (e.g. via a fcntl as opposed to a ioctl), then presumably we would simply deny an attempt to open the inode and create a new file structure for it.

On Linux, /proc/$PID/fd is an open as opposed to a dup (as much as I personally don't like those semantics, they are well set in stone at this point) so it satisfies your requirements.

Quoted 8 lines

If the "single owner" is determined by the file structure (e.g. via a fcntl as opposed to a ioctl), then presumably we would simply deny an attempt to open the inode and create a new file structure for it.

On Linux, /proc/$PID/fd is an open as opposed to a dup (as much as I personally don't like those semantics, they are well set in stone at this point) so it satisfies your requirements.

If that all could be made working, for the kdbus case we would be fine with requiring *any* tmpfs mount, create a new memfd from there with O_TMPFILE, and use new fcntl() definitios to protect/seal/unseal and identify that fd.

For the more restricted cases like Android that tmpfs mount could get a mount option to not allow the creation of any non-unlinked file, I guess.

Quoted 14 lines · Kay Sievers

Quoted 8 lines

If the "single owner" is determined by the file structure (e.g. via a fcntl as opposed to a ioctl), then presumably we would simply deny an attempt to open the inode and create a new file structure for it.

On Linux, /proc/$PID/fd is an open as opposed to a dup (as much as I personally don't like those semantics, they are well set in stone at this point) so it satisfies your requirements.

If that all could be made working, for the kdbus case we would be fine with requiring *any* tmpfs mount, create a new memfd from there with O_TMPFILE, and use new fcntl() definitios to protect/seal/unseal and identify that fd.

Quoted 4 lines

For the more restricted cases like Android that tmpfs mount could get a mount option to not allow the creation of any non-unlinked file, I guess.

Right, that would be the idea.

Quoted 4 lines · John Stultz

But yes, alternatively classic systems may be able to get around the issues via tmpfs quotas and convincing applications to use O_TMPFILE there. But to me this seems less ideal then the Android approach, where the lifecycle of the tmpfs fds more limited and clear.

Tmpfs supports no quota, it's all a huge hole and unsafe in that regard on every system today. But ashmem and kdbus, as they are today, are not better.

Quoted 17 lines · Kay Sievers

Quoted 16 lines · John Stultz

Quoted 10 lines · H. Peter Anvin

Quoted 7 lines · John Stultz

Thanks for reminding me about O_TMPFILE.. I have it on my list to look into how it could be used.

As for the O_TMPFILE only tmpfs option, it seems maybe a little clunky to me, but possible. If others think this would be preferred over a new syscall, I'll dig in deeper.

What is clunky about it? It reuses an existing interface and still points to the specific tmpfs instance that should be populated.

It would require new mount point convention that userland would have to standardize. To me (and admittedly its a taste thing), a new O_TMPFILE-only tmpfs mount point seems to be to be a bigger interface change from an application writers perspective then a new syscall.

Quoted 3 lines

Quoted 1 line

But maybe I'm misunderstanding your suggestion?

General purpose Linux has /dev/shm/ for that already, which will not go away anytime soon..

Right, though making /dev/shm/ O_TMPFILE only would likely break things, no?

In working with ashmem and looking briefly at kdbus' memfd ideas, there's a commonality that both basically act as a method to provide applications with unlinked tmpfs/shmem fds.

Quoted 3 lines · John Stultz

In working with ashmem and looking briefly at kdbus' memfd ideas, there's a commonality that both basically act as a method to provide applications with unlinked tmpfs/shmem fds.

Just use O_TMPFILE on a tmpfs file and you're done.

Quoted 5 lines · Christoph Hellwig

Quoted 3 lines · John Stultz

In working with ashmem and looking briefly at kdbus' memfd ideas, there's a commonality that both basically act as a method to provide applications with unlinked tmpfs/shmem fds.

Just use O_TMPFILE on a tmpfs file and you're done.

Ashmem and kdbus can name the deleted files, which is useful for debugging and tools to show the associated name for the file descriptor. They also show up in /proc/$PID/maps/ and possibly in /proc/$PID/fd/.

O_TMPFILE always creates files with just the name "/". Unless that is changed we wouldn't want switch over to O_TMPFILE, because we would lose that nice feature.

Quoted 11 lines · Kay Sievers

Quoted 5 lines · Christoph Hellwig

Quoted 3 lines · John Stultz

In working with ashmem and looking briefly at kdbus' memfd ideas, there's a commonality that both basically act as a method to provide applications with unlinked tmpfs/shmem fds.

Just use O_TMPFILE on a tmpfs file and you're done.

Ashmem and kdbus can name the deleted files, which is useful for debugging and tools to show the associated name for the file descriptor. They also show up in /proc/$PID/maps/ and possibly in /proc/$PID/fd/.

Quoted 3 lines

O_TMPFILE always creates files with just the name "/". Unless that is changed we wouldn't want switch over to O_TMPFILE, because we would lose that nice feature.

Not sure, but would Colin's vma-naming patch (or something like it) help address this?
https://lkml.org/lkml/2013/10/30/518

Ashmem and kdbus can name the deleted files, which is useful for debugging and tools to show the associated name for the file descriptor. They also show up in /proc/$PID/maps/ and possibly in /proc/$PID/fd/.

O_TMPFILE always creates files with just the name "/". Unless that is changed we wouldn't want switch over to O_TMPFILE, because we would lose that nice feature.

Is there are way to "fix" O_TMPFILE to accept the name of the file to be created, instead of insisting to take only the leading directory as the argument?