RFC: allow recording and passing of open file descriptors
Hello folks,
here's an attempt to make it to record open file descriptors (struct file*) and make them available to other processes via file system. The semantics are similar to dup() or passing via Unix socket in regard that the receiving process will get reference to the same struct file instance into its fdtable. But the big difference here is we're doing it directly via file system (ie. retrieving is done by a simple open()). It works pretty much like like Plan9's /srv file system:
http://man.cat-v.org/plan_9/3/srv
To archive that, the first patch introduces the concept of "file boxing", which means an open file operation can put a reference to another file into the struct file, which will then be returned to the caller, instead of the newly created one. The reason for doing it this strange way is that the new struct file instance is allocated and prepared very early, before we're calling into the actual file operation - refactoring this so that the open() file op directly returns a struct file* pointer would be a massively intrusive change, that I just don't dare to do here.
The second patch introduces a new file system "srvfs" that works like Plan9's /srv file systems.
Another use case for the first patch could be direct fd passing in FUSE, like Peng Tao and Alessio Balsini are currently working on, via other means.
https://www.spinics.net/lists/linux-fsdevel/msg196163.html
I believe the patch 1 should be pretty straightforward and not doing any harm. (it's not even compiled in unless explicitly enabled by something else)
have run,
--mtx
---
Enrico Weigelt, metux IT consult
Free software and Linux embedded engineering
[email protected] -- +49-151-27565287
RFC PATCH 1/2 fs: allow filesystems to directly pass an existing struct file
In some scenarios, file systems might want to pass an already opened struct file instance on an open() call, instead of opening a new one.
This allows similar techniques like the already well known file descriptor passing via Unix domain sockets, but now also for plain open() calls.
--- fs/Kconfig | 3 +++ fs/internal.h | 6 ++++++ fs/namei.c | 2 +- fs/open.c | 42 +++++++++++++++++++++++++++++++++++++++++- include/linux/fs.h | 9 +++++++++ 5 files changed, 60 insertions(+), 2 deletions(-)
fs/Kconfig +3 -0
@@ -357,4 +357,7 @@ source "fs/unicode/Kconfig" config IO_WQ bool +config FS_BOXED_FILE + bool + endmenu
fs/internal.h +6 -0
@@ -142,6 +142,12 @@ int do_fchownat(int dfd, const char __user *filename, uid_t user, gid_t group,int chown_common (const struct path *path,uid_t user,gid_t group);extern int vfs_open (const struct path *,struct file *); +#ifdef CONFIG_FS_BOXED_FILE +extern struct file *unbox_file (struct file *); +#else +static inline struct file *unbox_file (struct file *f) {return f; } +#endif +/* * inode.c */
fs/namei.c +1 -1
@@ -3496,7 +3496,7 @@ static struct file *path_openat(struct nameidata *nd, }if (likely (!error)) {if (likely (file->f_mode &FMODE_OPENED )) -return file; +return unbox_file (file);WARN_ON (1 ); error = -EINVAL ; }
fs/open.c +41 -1
54 lines not on this page · raw message
include/linux/fs.h +9 -0
@@ -955,6 +955,15 @@ struct file {struct address_space *f_mapping;errseq_t f_wb_err;errseq_t f_sb_err;/* for syncfs */ + +#ifdef CONFIG_FS_BOXED_FILE +/* Only for file systems that wanna pass an *existing* file to the +caller of open() instead of the newly created one. This has similar +semantics like passing an fd via unix socket, but instead via some +open() call. +*/ +struct file *boxed_file; +#endif }__randomize_layout __attribute__((aligned(4))); /* lest something weird decides that 2 is OK */
Signature · 1 line
2.20.1
RFC PATCH 2/2 fs: srvfs: new pseudo-fs for publishing opened fd's in the filesystem
In reply to Enrico Weigelt, metux IT consultThis is an implementation of Plan9's /srv filesystem, which offers storing already opened file descriptors into the fs: a process can store a open fd by just creating an empty file and writing its fdnum into it. The associated file struct will then be stored and next time that file is opened, the caller gets back exactly that opened file, quite like it was dup()ed or passed via some unix socket.
The rationale behind this: programs can prepare file descriptors (e.g. dial network connections, set up devices, etc) and leave them for other programs to pick up. This also works when opening or preparing the file needs special privileges that the consumer does not have.
Another interesting use case is long running servers that want to temporarily store their open files for a restart (possibly starting even restarting with a different binary, e.g. after upgrade).
The semantics of this file system are quite simple:
* creating files produces empty entries that can be assigned to an active file descriptor (struct file) by writing the numerical fd value into it. (the corresponding file will be looked up in the writing process' fdtable)
* before that, no other operations (except unlink) are supported
* once an entry is assigned with file descriptor, subsequent open() calls will retrieve exactly the assigned file descriptor (but most likely some other numerical fd value) - additional O_* flags have no effect on that file descriptor
* the underlying file remains open, until references, including the srvfs entry are removed.
* the srvfs entry can be removed via unlink()
* multiple mounts of the file system are separated, they do not share entries
--- MAINTAINERS | 5 ++ fs/Kconfig | 1 + fs/Makefile | 1 + fs/srvfs/Kconfig | 9 ++++ fs/srvfs/Makefile | 7 +++ fs/srvfs/file.c | 115 ++++++++++++++++++++++++++++++++++++++++++ fs/srvfs/fileref.c | 50 ++++++++++++++++++ fs/srvfs/root.c | 30 +++++++++++ fs/srvfs/srvfs.h | 39 ++++++++++++++ fs/srvfs/super.c | 123 +++++++++++++++++++++++++++++++++++++++++++++ 10 files changed, 380 insertions(+) create mode 100644 fs/srvfs/Kconfig create mode 100644 fs/srvfs/Makefile create mode 100644 fs/srvfs/file.c create mode 100644 fs/srvfs/fileref.c create mode 100644 fs/srvfs/root.c create mode 100644 fs/srvfs/srvfs.h create mode 100644 fs/srvfs/super.c
MAINTAINERS +5 -0
@@ -239,6 +239,11 @@ F: include/trace/events/9p.h F: include/uapi/linux/virtio_9p.h F: net/9p/ +SRV FILE SYSTEM: +M: Enrico Weigelt, metux IT consult <[email protected]> +S: Maintained +F: fs/srvfs + A8293 MEDIA DRIVER M: Antti Palosaari <[email protected]> L: [email protected]
fs/Kconfig +1 -0
@@ -248,6 +248,7 @@ config ARCH_HAS_GIGANTIC_PAGE source "fs/configfs/Kconfig" source "fs/efivarfs/Kconfig" +source "fs/srvfs/Kconfig" endmenu
fs/Makefile +1 -0
@@ -136,3 +136,4 @@ obj-$(CONFIG_EFIVAR_FS) += efivarfs/ obj-$(CONFIG_EROFS_FS) += erofs/ obj-$(CONFIG_VBOXSF_FS) += vboxsf/ obj-$(CONFIG_ZONEFS_FS) += zonefs/ +obj-$(CONFIG_SRV_FS) += srvfs/
fs/srvfs/Kconfig new file +9 -0
@@ -0,0 +1,9 @@ +config SRV_FS + tristate "Plan9-like /srv filesystem" + select FS_BOXED_FILE + help + If you don't know what this is about, say N. + + To compile this as a module, choose M here: the module will be called + srv-fs. Note that the file system of your root partition (the one + containing the directory /) cannot be compiled as a module.
fs/srvfs/Makefile new file +7 -0
@@ -0,0 +1,7 @@ +obj-$(CONFIG_SRV_FS) := srvfs.o + +srvfs-objs := \ + file.o \ + super.o \ + root.o \ + fileref.o
fs/srvfs/file.c new file +115 -0
115 lines not on this page · raw message
fs/srvfs/fileref.c new file +50 -0
50 lines not on this page · raw message
fs/srvfs/root.c new file +30 -0
@@ -0,0 +1,30 @@ + +#include "srvfs.h" + +#include <linux/fs.h> + +static int srvfs_dir_unlink (struct inode *inode,struct dentry *dentry) +{ +struct srvfs_fileref *fileref = dentry->d_inode->i_private; + +if (fileref ==NULL ) +return -EFAULT ; + +d_delete (dentry); +dput (dentry); + +return 0 ; +} + +static int srvfs_dir_create (struct user_namespace *mnt_userns, +struct inode *inode,struct dentry *dentry, +umode_t mode,bool excl) +{ +return srvfs_insert_file (inode->i_sb,dget (dentry)); +} + +const struct inode_operations srvfs_rootdir_inode_operations = { + .lookup = simple_lookup, + .unlink = srvfs_dir_unlink, + .create = srvfs_dir_create, +};
fs/srvfs/srvfs.h new file +39 -0
@@ -0,0 +1,39 @@ +#ifndef __LINUX_FS_SRVFS_H +#define __LINUX_FS_SRVFS_H + +#define pr_fmt (fmt) KBUILD_MODNAME ": " fmt + +#include <asm/atomic.h> +#include <linux/fs.h> +#include <linux/kref.h> + +#define SRVFS_MAGIC 0x29980123 + +struct srvfs_fileref { +atomic_t counter; +int mode; +struct file *file; +struct kref refcount; +struct file_operations f_ops; +}; + +struct srvfs_sb { +atomic_t inode_counter; +}; + +/* root.c */ +extern const struct inode_operations srvfs_rootdir_inode_operations; + +/* fileref.c */ +struct srvfs_fileref *srvfs_fileref_new (void ); +struct srvfs_fileref *srvfs_fileref_get (struct srvfs_fileref * fileref); +void srvfs_fileref_put (struct srvfs_fileref * fileref); +void srvfs_fileref_set (struct srvfs_fileref * fileref,struct file * newfile); + +/* super.c */ +int srvfs_inode_id (struct super_block *sb); + +/* file.c */ +int srvfs_insert_file (struct super_block *sb,struct dentry *dentry); + +#endif /* __LINUX_FS_SRVFS_H */
fs/srvfs/super.c new file +123 -0
123 lines not on this page · raw message
Signature · 1 line
2.20.1
kernel test robot
build botRe: [RFC PATCH 2/2] fs: srvfs: new pseudo-fs for publishing opened fd's in the filesystem
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch]
Hi "Enrico,
[FYI, it's a private test report for your RFC patch.]
[auto build test WARNING on linux/master]
[also build test WARNING on kees/for-next/pstore linus/master v5.13 next-20210629] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Enrico-Weigelt-metux-IT-consult/fs-allow-filesystems-to-directly-pass-an-existing-struct-file/20210629-234355 base: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git c54b245d011855ea91c5beff07f1db74143ce614 config: arc-allyesconfig (attached as .config) compiler: arceb-elf-gcc (GCC) 9.3.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/b27203409c71484eb143ea4bb70e7634c9c68e5e
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Enrico-Weigelt-metux-IT-consult/fs-allow-filesystems-to-directly-pass-an-existing-struct-file/20210629-234355 git checkout b27203409c71484eb143ea4bb70e7634c9c68e5e
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=arc
If you fix the issue, kindly add following tag as appropriate
All warnings (new ones prefixed by >>):
>> fs/srvfs/super.c:96:16: warning: no previous prototype for 'srvfs_mount' [-Wmissing-prototypes]
96 | struct dentry *srvfs_mount(struct file_system_type *fs_type,
| ^~~~~~~~~~~
--
>> fs/srvfs/fileref.c:25:6: warning: no previous prototype for 'srvfs_fileref_destroy' [-Wmissing-prototypes] 25 | void srvfs_fileref_destroy(struct kref *ref)
| ^~~~~~~~~~~~~~~~~~~~~
vim +/srvfs_mount +96 fs/srvfs/super.c
95
> 96 struct dentry *srvfs_mount(struct file_system_type *fs_type,
97 int flags, const char *dev_name, void *data)
98 {
99 return mount_nodev(fs_type, flags, data, srvfs_fill_super); 100 }
101
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org