Support userspace-selected fds
Subsystems (7)
v5 cover Support userspace-selected fds
5.8 material, not intended for 5.7. Now includes a patch for man-pages, attached to this cover letter.
Inspired by the X protocol's handling of XIDs, allow userspace to select the file descriptor opened by a call like openat2, so that it can use the resulting file descriptor in subsequent system calls without waiting for the response to the initial openat2 syscall.
The first patch is independent of the other two; it allows reserving file descriptors below a certain minimum for userspace-selected fd allocation only.
The second patch implements userspace-selected fd allocation for openat2, introducing a new O_SPECIFIC_FD flag and an fd field in struct open_how. In io_uring, this allows sequences like openat2/read/close without waiting for the openat2 to complete. Multiple such sequences can overlap, as long as each uses a distinct file descriptor.
The third patch adds userspace-selected fd allocation to pipe2 as well. I did this partly as a demonstration of how simple it is to wire up O_SPECIFIC_FD support for any fd-allocating system call, and partly in the hopes that this may make it more useful to wire up io_uring support for pipe2 in the future.
v5:
Rename padding field to __padding.
Add tests for non-zero __padding.
Include patch for man-pages.
v4:
Changed fd field to __u32.
Expanded and consolidated checks that return -EINVAL for invalid arguments. Simplified and commented build_open_how.
Add documentation comment for fd field.
Add kselftests.
Thanks to Aleksa Sarai for feedback.
v3:
This new version has an API to atomically increase the minimum fd and return the previous minimum, rather than just getting and setting the minimum; this makes it easier to allocate a range. (A library that might initialize after the program has already opened other file descriptors may need to check for existing open fds in the range after reserving it, and reserve more fds if needed; this can be done entirely in userspace, and we can't really do anything simpler in the kernel due to limitations on file-descriptor semantics, so this patch series avoids introducing any extra complexity in the kernel.)
This new version also supports a __get_specific_unused_fd_flags call which accepts the limit for RLIMIT_NOFILE as an argument, analogous to __get_unused_fd_flags, since io_uring needs that to correctly handle RLIMIT_NOFILE.
Thanks to Jens Axboe for review and feedback.
v2:
Version 2 was a version incorporated into a larger patch series from Jens Axboe on io_uring.
Josh Triplett (3):
fs: Support setting a minimum fd for "lowest available fd" allocation
fs: openat2: Extend open_how to allow userspace-selected fds
fs: pipe2: Support O_SPECIFIC_FD
fs/fcntl.c | 2 +- fs/file.c | 62 +++++++++++++++++-- fs/io_uring.c | 3 +- fs/open.c | 8 ++- fs/pipe.c | 16 +++-- include/linux/fcntl.h | 5 +- include/linux/fdtable.h | 1 + include/linux/file.h | 4 ++ include/uapi/asm-generic/fcntl.h | 4 ++ include/uapi/linux/openat2.h | 3 + include/uapi/linux/prctl.h | 3 + kernel/sys.c | 5 ++ tools/testing/selftests/openat2/helpers.c | 2 +- tools/testing/selftests/openat2/helpers.h | 21 +++++-- .../testing/selftests/openat2/openat2_test.c | 35 ++++++++++- 15 files changed, 150 insertions(+), 24 deletions(-)
Signature · 1 line
2.26.2
From e3f2c8b75008de46cc51802f5915833298216370 Mon Sep 17 00:00:00 2001
Message-Id: <e3f2c8b75008de46cc51802f5915833298216370.1587531331.git.josh@joshtriplett.org>
From: Josh Triplett <[email protected]>
Date: Tue, 21 Apr 2020 21:54:25 -0700
Subject: [PATCH] openat2.2, pipe.2, prctl.2: Document O_SPECIFIC_FD and
PR_INCREASE_MIN_FD
--- man2/openat2.2 | 83 ++++++++++++++++++++++++++++++++++++++++++++++++-- man2/pipe.2 | 42 +++++++++++++++++++++++++ man2/prctl.2 | 12 ++++++++ 3 files changed, 134 insertions(+), 3 deletions(-)
man2/openat2.2 +80 -3
@@ -103,9 +103,11 @@ This argument is a pointer to a structure of the following form: .in +4n .EX struct open_how { - u64 flags; /* O_* flags */ - u64 mode; /* Mode for O_{CREAT,TMPFILE} */ - u64 resolve; /* RESOLVE_* flags */ + u64 flags; /* O_* flags */ + u64 mode; /* Mode for O_{CREAT,TMPFILE} */ + u64 resolve; /* RESOLVE_* flags */ + u32 fd; /* File descriptor for O_SPECIFIC_FD */ + u32 __padding; /* Must be zeroed */ /* ... */ }; .EE
@@ -147,6 +149,12 @@ argument, .BR openat2 () returns an error if unknown or conflicting flags are specified in .IR how.flags . +.IP +The flag +.B O_SPECIFIC_FD +is only valid for openat2, as it requires the +.I fd +field. .TP .I mode This field specifies the
@@ -390,6 +398,48 @@ so that a pathname component (now) contains a bind mount. If any bits other than those listed above are set in .IR how.resolve , an error is returned. +.TP +.I fd +If +.I flags +contains +.BR O_SPECIFIC_FD , +.BR openat2 () +will allocate and return that specific file descriptor, rather than using the +lowest available file descriptor. +If +.I fd +contains \-1, +.BR openat2 () +will return the lowest available file descriptor just as if +.B O_SPECIFIC_FD +had not been specified. +.IP +The caller may wish to use the +.BR prctl (2) +option +.B PR_INCREASE_MIN_FD +to reserve a range of file descriptors for such usage. +.IP +If the specified file descriptor is already in use, +.BR openat2 () +will return \-1 and set +.I errno +to +.BR EBUSY . +.IP +If +.I flags +does not contain +.BR O_SPECIFIC_FD , +.I fd +must be zero. +.TP +.I __padding +This value must be zero, and must not be referenced by name. +Future versions of the +.I open_how +structure may give a new name and semantic to this field. .SH RETURN VALUE On success, a new file descriptor is returned. On error, \-1 is returned, and
@@ -421,6 +471,26 @@ The caller may choose to retry the .BR openat2 () call. .TP +.B EBUSY +.I how.flags +contains +.B O_SPECIFIC_FD +and the file descriptor specified in +.I fd +was in use. +.TP +.B EMFILE +.I how.flags +contains +.B O_SPECIFIC_FD +and +.I how.fd +exceeds the maximum file descriptor allowed for the process +(see the description of +.BR RLIMIT_NOFILE +in +.BR getrlimit (2)). +.TP .B EINVAL An unknown flag or invalid value was specified in .IR how .
@@ -435,6 +505,13 @@ or .BR O_TMPFILE . .TP .B EINVAL +.I how.fd +is non-zero, but +.I how.flags +does not contain +.BR O_SPECIFIC_FD . +.TP +.B EINVAL .I size was smaller than any known version of .IR "struct open_how" .
man2/pipe.2 +42 -0
60 lines not on this page · raw message
man2/prctl.2 +12 -0
@@ -531,6 +531,18 @@ All unused .BR prctl () arguments must be zero. .TP +.BR PR_INCREASE_MIN_FD " (since Linux 5.8)" +Increase the minimum file descriptor that the kernel will automatically +allocate when the process opens a new file descriptor, by +.IR arg2 . +Return (as the function result) the current minimum. +A process may pass 0 as +.I arg2 +to return the current minimum without changing it. +The remaining unused +.BR prctl () +arguments must be zero for future compatibility. +.TP .BR PR_SET_MM " (since Linux 3.3)" .\" commit 028ee4be34a09a6d48bdf30ab991ae933a7bc036 Modify certain kernel memory map descriptor fields
Signature · 1 line
2.26.2
PATCH v5 1/3 fs: Support setting a minimum fd for "lowest available fd" allocation
Some applications want to prevent the usual "lowest available fd" allocation from allocating certain file descriptors. For instance, they may want to prevent allocation of a closed fd 0, 1, or 2 other than via dup2/dup3, or reserve some low file descriptors for other purposes.
Add a prctl to increase the minimum fd and return the previous minimum.
System calls that allocate a specific file descriptor, such as dup2/dup3, ignore this minimum.
exec resets the minimum fd, to prevent one program from interfering with another program's expectations about fd allocation.
Test program:
#include <err.h>
#include <fcntl.h>
#include <stdio.h>
#include <sys/prctl.h>
int main(int argc, char *argv[])
{
if (prctl(PR_INCREASE_MIN_FD, 100, 0, 0, 0) < 0)
err(1, "prctl");
int fd = open("/dev/null", O_RDONLY);
if (fd < 0)
err(1, "open");
printf("%d\n", fd); // prints 100
return 0;
}--- fs/file.c | 23 +++++++++++++++++------ include/linux/fdtable.h | 1 + include/linux/file.h | 1 + include/uapi/linux/prctl.h | 3 +++ kernel/sys.c | 5 +++++ 5 files changed, 27 insertions(+), 6 deletions(-)
fs/file.c +17 -6
@@ -286,7 +286,6 @@ struct files_struct *dup_fd(struct files_struct *oldf, int *errorp)spin_lock_init (&newf->file_lock); newf->resize_in_progress = false;init_waitqueue_head (&newf->resize_wait); - newf->next_fd =0 ; new_fdt = &newf->fdtab; new_fdt->max_fds =NR_OPEN_DEFAULT ; new_fdt->close_on_exec = newf->close_on_exec_init;
@@ -295,6 +294,7 @@ struct files_struct *dup_fd(struct files_struct *oldf, int *errorp) new_fdt->fd = &newf->fd_array[0 ];spin_lock (&oldf->file_lock); + newf->next_fd = newf->min_fd = oldf->min_fd; old_fdt =files_fdtable (oldf); open_files =count_open_files (old_fdt);
@@ -487,9 +487,7 @@ int __alloc_fd(struct files_struct *files,spin_lock (&files->file_lock); repeat: fdt =files_fdtable (files); - fd = start; -if (fd < files->next_fd) - fd = files->next_fd; + fd =max3 (start, files->min_fd, files->next_fd);if (fd < fdt->max_fds) fd =find_next_fd (fdt, fd);
@@ -514,7 +512,7 @@ int __alloc_fd(struct files_struct *files, goto repeat;if (start <= files->next_fd) - files->next_fd = fd +1 ; + files->next_fd =max (fd +1 , files->min_fd);__set_open_fd (fd, fdt);if (flags &O_CLOEXEC )
@@ -555,7 +553,7 @@ static void __put_unused_fd(struct files_struct *files, unsigned int fd) {struct fdtable *fdt =files_fdtable (files);__clear_open_fd (fd, fdt); -if (fd < files->next_fd) +if (fd < files->next_fd && fd >= files->min_fd) files->next_fd = fd; }
@@ -684,6 +682,7 @@ void do_close_on_exec(struct files_struct *files)/* exec unshares first */ spin_lock (&files->file_lock); + files->min_fd =0 ;for (i =0 ; ; i++) {unsigned long set;unsigned fd = i *BITS_PER_LONG ;
@@ -865,6 +864,18 @@ bool get_close_on_exec(unsigned int fd)return res; } +unsigned int increase_min_fd (unsigned int num) +{ +struct files_struct *files = current->files; +unsigned int old_min_fd; + +spin_lock (&files->file_lock); + old_min_fd = files->min_fd; + files->min_fd += num; +spin_unlock (&files->file_lock); +return old_min_fd; +} + static int do_dup2(struct files_struct *files, struct file *file, unsigned fd, unsigned flags) __releases(&files->file_lock)
include/linux/fdtable.h +1 -0
@@ -60,6 +60,7 @@ struct files_struct {*/ spinlock_t file_lock____cacheline_aligned_in_smp ;unsigned int next_fd; +unsigned int min_fd;/* min for "lowest available fd" allocation */ unsigned long close_on_exec_init[1 ];unsigned long open_fds_init[1 ];unsigned long full_fds_bits_init[1 ];
include/linux/file.h +1 -0
@@ -88,6 +88,7 @@ extern bool get_close_on_exec(unsigned int fd);extern int __get_unused_fd_flags (unsigned flags,unsigned long nofile);extern int get_unused_fd_flags (unsigned flags);extern void put_unused_fd (unsigned int fd); +extern unsigned int increase_min_fd (unsigned int num);extern void fd_install (unsigned int fd,struct file *file);
include/uapi/linux/prctl.h +3 -0
@@ -238,4 +238,7 @@ struct prctl_mm_map {#define PR_SET_IO_FLUSHER 57#define PR_GET_IO_FLUSHER 58 +/* Increase minimum file descriptor for "lowest available fd" allocation */ +#define PR_INCREASE_MIN_FD 59 +#endif /* _LINUX_PRCTL_H */
kernel/sys.c +5 -0
@@ -2514,6 +2514,11 @@ SYSCALL_DEFINE5(prctl, int, option, unsigned long, arg2, unsigned long, arg3, error = (current->flags &PR_IO_FLUSHER ) ==PR_IO_FLUSHER ;break ; +case PR_INCREASE_MIN_FD : +if (arg3 || arg4 || arg5) +return -EINVAL ; + error =increase_min_fd ((unsigned int )arg2); +break ;default : error = -EINVAL ;break ;
Signature · 1 line
2.26.2
Re: [PATCH v5 1/3] fs: Support setting a minimum fd for "lowest available fd" allocation
[CC += linux-api]
Quoted 172 lines · Josh Triplett
Some applications want to prevent the usual "lowest available fd" allocation from allocating certain file descriptors. For instance, they may want to prevent allocation of a closed fd 0, 1, or 2 other than via dup2/dup3, or reserve some low file descriptors for other purposes.
Add a prctl to increase the minimum fd and return the previous minimum.
System calls that allocate a specific file descriptor, such as dup2/dup3, ignore this minimum.
exec resets the minimum fd, to prevent one program from interfering with another program's expectations about fd allocation.
Test program:
#include <err.h> #include <fcntl.h> #include <stdio.h> #include <sys/prctl.h> int main(int argc, char *argv[]) { if (prctl(PR_INCREASE_MIN_FD, 100, 0, 0, 0) < 0) err(1, "prctl"); int fd = open("/dev/null", O_RDONLY); if (fd < 0) err(1, "open"); printf("%d\n", fd); // prints 100 return 0; }Signed-off-by Josh Triplett <[email protected]>--- fs/file.c | 23 +++++++++++++++++------ include/linux/fdtable.h | 1 + include/linux/file.h | 1 + include/uapi/linux/prctl.h | 3 +++ kernel/sys.c | 5 +++++ 5 files changed, 27 insertions(+), 6 deletions(-)fs/file.c +17
-6@@ -286,7 +286,6 @@ struct files_struct *dup_fd(struct files_struct *oldf, int *errorp)spin_lock_init (&newf->file_lock); newf->resize_in_progress = false;init_waitqueue_head (&newf->resize_wait); - newf->next_fd =0 ; new_fdt = &newf->fdtab; new_fdt->max_fds =NR_OPEN_DEFAULT ; new_fdt->close_on_exec = newf->close_on_exec_init;@@ -295,6 +294,7 @@ struct files_struct *dup_fd(struct files_struct *oldf, int *errorp) new_fdt->fd = &newf->fd_array[0 ];spin_lock (&oldf->file_lock); + newf->next_fd = newf->min_fd = oldf->min_fd; old_fdt =files_fdtable (oldf); open_files =count_open_files (old_fdt);@@ -487,9 +487,7 @@ int __alloc_fd(struct files_struct *files,spin_lock (&files->file_lock); repeat: fdt =files_fdtable (files); - fd = start; -if (fd < files->next_fd) - fd = files->next_fd; + fd =max3 (start, files->min_fd, files->next_fd);if (fd < fdt->max_fds) fd =find_next_fd (fdt, fd);@@ -514,7 +512,7 @@ int __alloc_fd(struct files_struct *files, goto repeat;if (start <= files->next_fd) - files->next_fd = fd +1 ; + files->next_fd =max (fd +1 , files->min_fd);__set_open_fd (fd, fdt);if (flags &O_CLOEXEC )@@ -555,7 +553,7 @@ static void __put_unused_fd(struct files_struct *files, unsigned int fd) {struct fdtable *fdt =files_fdtable (files);__clear_open_fd (fd, fdt); -if (fd < files->next_fd) +if (fd < files->next_fd && fd >= files->min_fd) files->next_fd = fd; }@@ -684,6 +682,7 @@ void do_close_on_exec(struct files_struct *files)/* exec unshares first */ spin_lock (&files->file_lock); + files->min_fd =0 ;for (i =0 ; ; i++) {unsigned long set;unsigned fd = i *BITS_PER_LONG ;@@ -865,6 +864,18 @@ bool get_close_on_exec(unsigned int fd)return res; } +unsigned int increase_min_fd (unsigned int num) +{ +struct files_struct *files = current->files; +unsigned int old_min_fd; + +spin_lock (&files->file_lock); + old_min_fd = files->min_fd; + files->min_fd += num; +spin_unlock (&files->file_lock); +return old_min_fd; +} + static int do_dup2(struct files_struct *files, struct file *file, unsigned fd, unsigned flags) __releases(&files->file_lock)include/linux/fdtable.h +1
-0@@ -60,6 +60,7 @@ struct files_struct {*/ spinlock_t file_lock____cacheline_aligned_in_smp ;unsigned int next_fd; +unsigned int min_fd;/* min for "lowest available fd" allocation */ unsigned long close_on_exec_init[1 ];unsigned long open_fds_init[1 ];unsigned long full_fds_bits_init[1 ];include/linux/file.h +1
-0@@ -88,6 +88,7 @@ extern bool get_close_on_exec(unsigned int fd);extern int __get_unused_fd_flags (unsigned flags,unsigned long nofile);extern int get_unused_fd_flags (unsigned flags);extern void put_unused_fd (unsigned int fd); +extern unsigned int increase_min_fd (unsigned int num);extern void fd_install (unsigned int fd,struct file *file);include/uapi/linux/prctl.h +3
-0@@ -238,4 +238,7 @@ struct prctl_mm_map {#define PR_SET_IO_FLUSHER 57#define PR_GET_IO_FLUSHER 58 +/* Increase minimum file descriptor for "lowest available fd" allocation */ +#define PR_INCREASE_MIN_FD 59 +#endif /* _LINUX_PRCTL_H */ kernel/sys.c +5
-1@@ -2514,6 +2514,11 @@ SYSCALL_DEFINE5(prctl, int, option, unsigned long, arg2, unsigned long, arg3, error = (current->flags &PR_IO_FLUSHER ) ==PR_IO_FLUSHER ;break ; +case PR_INCREASE_MIN_FD : +if (arg3 || arg4 || arg5) +return -EINVAL ; + error =increase_min_fd ((unsigned int )arg2); +break ;default : error = -EINVAL ;break ; --2.26.2
Signature · 3 lines
Michael Kerrisk Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/ Linux/UNIX System Programming Training: http://man7.org/training/
Re: [PATCH v5 1/3] fs: Support setting a minimum fd for "lowest available fd" allocation
In reply to Josh TriplettQuoted 5 lines · Josh Triplett
Some applications want to prevent the usual "lowest available fd" allocation from allocating certain file descriptors. For instance, they may want to prevent allocation of a closed fd 0, 1, or 2 other than via dup2/dup3, or reserve some low file descriptors for other purposes.
Quoted 7 lines
Add a prctl to increase the minimum fd and return the previous minimum.
System calls that allocate a specific file descriptor, such as dup2/dup3, ignore this minimum.
exec resets the minimum fd, to prevent one program from interfering with another program's expectations about fd allocation.
Please make this aspect properly documented in "Effect on process attributes" section of execve(2) manual page.
[...]
fs/file.c bool get_close_on_exec(unsigned int fd) · line 877
Quoted 11 lines
fs/file.c +11
-0+unsigned int increase_min_fd (unsigned int num) +{ +struct files_struct *files = current->files; +unsigned int old_min_fd; + +spin_lock (&files->file_lock); + old_min_fd = files->min_fd; + files->min_fd += num; +spin_unlock (&files->file_lock); +return old_min_fd; +}
If it's "increase", there should be an overflow check. Otherwise it's "assign" rather than "increase".
Signature · 1 line
ldv
Re: [PATCH v5 1/3] fs: Support setting a minimum fd for "lowest available fd" allocation
Quoted 8 lines · Dmitry V. Levin
Quoted 7 lines · Josh Triplett
Some applications want to prevent the usual "lowest available fd" allocation from allocating certain file descriptors. For instance, they may want to prevent allocation of a closed fd 0, 1, or 2 other than via dup2/dup3, or reserve some low file descriptors for other purposes.
Add a prctl to increase the minimum fd and return the previous minimum.
Quoted 8 lines
Quoted 5 lines
System calls that allocate a specific file descriptor, such as dup2/dup3, ignore this minimum.
exec resets the minimum fd, to prevent one program from interfering with another program's expectations about fd allocation.
Please make this aspect properly documented in "Effect on process attributes" section of execve(2) manual page.
Done. I'll include updated manpage patches in v6.
Quoted 6 lines
Quoted 6 lines
fs/file.c +6
-0+unsigned int increase_min_fd (unsigned int num) +{ +struct files_struct *files = current->files; +unsigned int old_min_fd; + +spin_lock (&files->file_lock);
Quoted 8 lines
fs/file.c bool get_close_on_exec(unsigned int fd) · line 877
Quoted 5 lines
fs/file.c +5
-0+ old_min_fd = files->min_fd; + files->min_fd += num; +spin_unlock (&files->file_lock); +return old_min_fd; +}If it's "increase", there should be an overflow check. Otherwise it's "assign" rather than "increase".
I'll add a check in v6, to make sure that the value cannot overflow into the errno range. (Note that this is not security-sensitive, it's just providing a footgun-resistant interface. It should absolutely check, though.)
- Josh Triplett
Arnd Bergmann
maintainerRe: [PATCH v5 1/3] fs: Support setting a minimum fd for "lowest available fd" allocation
In reply to Josh TriplettQuoted 6 lines · Josh Triplett
Some applications want to prevent the usual "lowest available fd" allocation from allocating certain file descriptors. For instance, they may want to prevent allocation of a closed fd 0, 1, or 2 other than via dup2/dup3, or reserve some low file descriptors for other purposes.
Quoted 7 lines
Add a prctl to increase the minimum fd and return the previous minimum.
System calls that allocate a specific file descriptor, such as dup2/dup3, ignore this minimum.
exec resets the minimum fd, to prevent one program from interfering with another program's expectations about fd allocation.
Have you considered making this a separate system call rather than a part of prctl()?
At the moment, there are certain classes of things controlled by prctl, e.g. capabilities, floating point handling and timer behavior, but nothing that relates to file descriptors as such, so it's not an obvious decision.
Another option would be prlimit(), as it already controls the maximum file descriptor number with RLIMIT_NOFILE, and adding a minimum there would let you set min/max atomically.
Arnd
PATCH v5 2/3 fs: openat2: Extend open_how to allow userspace-selected fds
In reply to Josh TriplettInspired by the X protocol's handling of XIDs, allow userspace to select the file descriptor opened by openat2, so that it can use the resulting file descriptor in subsequent system calls without waiting for the response to openat2.
In io_uring, this allows sequences like openat2/read/close without waiting for the openat2 to complete. Multiple such sequences can overlap, as long as each uses a distinct file descriptor.
Add a new O_SPECIFIC_FD open flag to enable this behavior, only accepted by openat2 for now (ignored by open/openat like all unknown flags). Add an fd field to struct open_how (along with appropriate padding, and verify that the padding is 0 to allow replacing the padding with a field in the future).
The file table has a corresponding new function get_specific_unused_fd_flags, which gets the specified file descriptor if O_SPECIFIC_FD is set (and the fd isn't -1); otherwise it falls back to get_unused_fd_flags, to simplify callers.
The specified file descriptor must not already be open; if it is, get_specific_unused_fd_flags will fail with -EBUSY. This helps catch userspace errors.
When O_SPECIFIC_FD is set, and fd is not -1, openat2 will use the specified file descriptor rather than finding the lowest available one.
Test program:
#include <err.h>
#include <fcntl.h>
#include <stdio.h>
#include <unistd.h>
int main(void)
{
struct open_how how = {
.flags = O_RDONLY | O_SPECIFIC_FD,
.fd = 42
};
int fd = openat2(AT_FDCWD, "/dev/null", &how, sizeof(how));
if (fd < 0)
err(1, "openat2");
printf("fd=%d\n", fd); // prints fd=42
return 0;
}--- fs/fcntl.c | 2 +- fs/file.c | 39 +++++++++++++++++++ fs/io_uring.c | 3 +- fs/open.c | 8 +++- include/linux/fcntl.h | 5 ++- include/linux/file.h | 3 ++ include/uapi/asm-generic/fcntl.h | 4 ++ include/uapi/linux/openat2.h | 3 ++ tools/testing/selftests/openat2/helpers.c | 2 +- tools/testing/selftests/openat2/helpers.h | 21 +++++++--- .../testing/selftests/openat2/openat2_test.c | 35 ++++++++++++++++- 11 files changed, 111 insertions(+), 14 deletions(-)
fs/fcntl.c +1 -1
@@ -1033,7 +1033,7 @@ static int __init fcntl_init(void)* Exceptions: O_NONBLOCK is a two bit define on parisc; O_NDELAY * is defined as O_NONBLOCK on some platforms and not on others. */ -BUILD_BUG_ON (21 -1 /* for O_RDONLY being 0 */ != +BUILD_BUG_ON (22 -1 /* for O_RDONLY being 0 */ !=HWEIGHT32 ( (VALID_OPEN_FLAGS & ~(O_NONBLOCK |O_NDELAY )) | __FMODE_EXEC | __FMODE_NONOTIFY));
fs/file.c +39 -0
@@ -567,6 +567,45 @@ void put_unused_fd(unsigned int fd)EXPORT_SYMBOL (put_unused_fd); +int __get_specific_unused_fd_flags (unsigned int fd,unsigned int flags, +unsigned long nofile) +{ +int ret; +struct fdtable *fdt; +struct files_struct *files = current->files; + +if (!(flags &O_SPECIFIC_FD ) || fd ==UINT_MAX ) +return __get_unused_fd_flags (flags, nofile); + +if (fd >= nofile) +return -EBADF ; + +spin_lock (&files->file_lock); + ret =expand_files (files, fd); +if (unlikely (ret <0 )) + goto out_unlock; + fdt =files_fdtable (files); +if (fdt->fd[fd]) { + ret = -EBUSY ; + goto out_unlock; + } +__set_open_fd (fd, fdt); +if (flags &O_CLOEXEC ) +__set_close_on_exec (fd, fdt); +else +__clear_close_on_exec (fd, fdt); + ret = fd; + +out_unlock: +spin_unlock (&files->file_lock); +return ret; +} + +int get_specific_unused_fd_flags (unsigned int fd,unsigned int flags) +{ +return __get_specific_unused_fd_flags (fd, flags,rlimit (RLIMIT_NOFILE )); +} +/* * Install a file pointer in the fd array. *
fs/io_uring.c +2 -1
@@ -2997,7 +2997,8 @@ static int io_openat2(struct io_kiocb *req, bool force_nonblock)if (ret) goto err; - ret =__get_unused_fd_flags (req->open.how.flags, req->open.nofile); + ret =__get_specific_unused_fd_flags (req->open.how.fd, + req->open.how.flags, req->open.nofile);if (ret <0 ) goto err;
fs/open.c +7 -1
@@ -962,6 +962,8 @@ inline struct open_how build_open_how(int flags, umode_t mode) .mode = mode & S_IALLUGO, }; +/* O_SPECIFIC_FD doesn't work in open calls that use build_open_how. */ + how.flags &= ~O_SPECIFIC_FD ;/* O_PATH beats everything else. */ if (how.flags &O_PATH ) how.flags &=O_PATH_FLAGS ;
@@ -989,6 +991,10 @@ inline int build_open_flags(const struct open_how *how, struct open_flags *op)return -EINVAL ;if (how->resolve & ~VALID_RESOLVE_FLAGS )return -EINVAL ; +if (how->__padding !=0 ) +return -EINVAL ; +if (!(flags &O_SPECIFIC_FD ) && how->fd !=0 ) +return -EINVAL ;/* Deal with the mode. */ if (WILL_CREATE (flags)) {
@@ -1143,7 +1149,7 @@ static long do_sys_openat2(int dfd, const char __user *filename,if (IS_ERR (tmp))return PTR_ERR (tmp); - fd =get_unused_fd_flags (how->flags); + fd =get_specific_unused_fd_flags (how->fd, how->flags);if (fd >=0 ) {struct file *f =do_filp_open (dfd, tmp, &op);if (IS_ERR (f)) {
include/linux/fcntl.h +3 -2
@@ -10,7 +10,7 @@ (O_RDONLY |O_WRONLY |O_RDWR |O_CREAT |O_EXCL |O_NOCTTY |O_TRUNC | \O_APPEND |O_NDELAY |O_NONBLOCK |O_NDELAY | __O_SYNC |O_DSYNC | \FASYNC |O_DIRECT |O_LARGEFILE |O_DIRECTORY |O_NOFOLLOW | \ -O_NOATIME |O_CLOEXEC |O_PATH | __O_TMPFILE) +O_NOATIME |O_CLOEXEC |O_PATH | __O_TMPFILE |O_SPECIFIC_FD ) /* List of all valid flags for the how->upgrade_mask argument: */#define VALID_UPGRADE_FLAGS \
@@ -23,7 +23,8 @@/* List of all open_how "versions". */ #define OPEN_HOW_SIZE_VER0 24/* sizeof first published struct */ -#define OPEN_HOW_SIZE_LATEST OPEN_HOW_SIZE_VER0 +#define OPEN_HOW_SIZE_VER1 32/* added fd and pad */ +#define OPEN_HOW_SIZE_LATEST OPEN_HOW_SIZE_VER1#ifndef force_o_largefile#define force_o_largefile () (!IS_ENABLED(CONFIG_ARCH_32BIT_OFF_T))
include/linux/file.h +3 -0
@@ -87,6 +87,9 @@ extern void set_close_on_exec(unsigned int fd, int flag);extern bool get_close_on_exec (unsigned int fd);extern int __get_unused_fd_flags (unsigned flags,unsigned long nofile);extern int get_unused_fd_flags (unsigned flags); +extern int __get_specific_unused_fd_flags (unsigned int fd,unsigned int flags, +unsigned long nofile); +extern int get_specific_unused_fd_flags (unsigned int fd,unsigned int flags);extern void put_unused_fd (unsigned int fd);extern unsigned int increase_min_fd (unsigned int num);
include/uapi/asm-generic/fcntl.h +4 -0
@@ -89,6 +89,10 @@#define __O_TMPFILE 020000000#endif +#ifndef O_SPECIFIC_FD +#define O_SPECIFIC_FD 01000000000/* open as specified fd */ +#endif +/* a horrid kludge trying to make sure that this will fail on old kernels */ #define O_TMPFILE (__O_TMPFILE | O_DIRECTORY)#define O_TMPFILE_MASK (__O_TMPFILE | O_DIRECTORY | O_CREAT)
include/uapi/linux/openat2.h +3 -0
@@ -15,11 +15,14 @@* @flags: O_* flags. * @mode: O_CREAT/O_TMPFILE file mode. * @resolve: RESOLVE_* flags. +* @fd: Specific file descriptor to use, for O_SPECIFIC_FD. */ struct open_how {__u64 flags;__u64 mode;__u64 resolve; +__u32 fd; +__u32 __padding;/* Must be zeroed */ };/* how->resolve flags for openat2(2). */
tools/testing/selftests/openat2/helpers.c +1 -1
@@ -98,7 +98,7 @@ void __attribute__((constructor)) init(void)struct open_how how = {};int fd; -BUILD_BUG_ON (sizeof (struct open_how ) !=OPEN_HOW_SIZE_VER0 ); +BUILD_BUG_ON (sizeof (struct open_how ) !=OPEN_HOW_SIZE_VER1 );/* Check openat2(2) support. */ fd =sys_openat2 (AT_FDCWD ,"." , &how);
tools/testing/selftests/openat2/helpers.h +15 -6
@@ -24,28 +24,37 @@#endif /* SYS_openat2 */ /* -* Arguments for how openat2(2) should open the target path. If @resolve is -* zero, then openat2(2) operates very similarly to openat(2). +* Arguments for how openat2(2) should open the target path. If only @flags and +* @mode are non-zero, then openat2(2) operates very similarly to openat(2). * -* However, unlike openat(2), unknown bits in @flags result in -EINVAL rather -* than being silently ignored. @mode must be zero unless one of {O_CREAT, -* O_TMPFILE} are set. +* However, unlike openat(2), unknown or invalid bits in @flags result in +* -EINVAL rather than being silently ignored. @mode must be zero unless one of +* {O_CREAT, O_TMPFILE} are set. * * @flags: O_* flags. * @mode: O_CREAT/O_TMPFILE file mode. * @resolve: RESOLVE_* flags. +* @fd: Specific file descriptor to use, for O_SPECIFIC_FD. */ struct open_how {__u64 flags;__u64 mode;__u64 resolve; +__u32 fd; +__u32 __padding;/* Must be zeroed */ }; +/* List of all open_how "versions". */ #define OPEN_HOW_SIZE_VER0 24/* sizeof first published struct */ -#define OPEN_HOW_SIZE_LATEST OPEN_HOW_SIZE_VER0 +#define OPEN_HOW_SIZE_VER1 32/* added fd and pad */ +#define OPEN_HOW_SIZE_LATEST OPEN_HOW_SIZE_VER1bool needs_openat2 (const struct open_how *how); +#ifndef O_SPECIFIC_FD +#define O_SPECIFIC_FD 01000000000 +#endif +#ifndef RESOLVE_IN_ROOT /* how->resolve flags for openat2(2). */ #define RESOLVE_NO_XDEV 0x01/* Block mount-point crossings
tools/testing/selftests/openat2/openat2_test.c +33 -2
@@ -40,7 +40,7 @@ struct struct_test {int err; }; -#define NUM_OPENAT2_STRUCT_TESTS 7 +#define NUM_OPENAT2_STRUCT_TESTS 8#define NUM_OPENAT2_STRUCT_VARIATIONS 13void test_openat2_struct (void )
@@ -52,6 +52,9 @@ void test_openat2_struct(void) { .name = "normal struct", .arg.inner.flags = O_RDONLY, .size = sizeof(struct open_how) }, + { .name = "v0 struct", + .arg.inner.flags = O_RDONLY, + .size = OPEN_HOW_SIZE_VER0 }, /* Bigger struct, with zeroed out end. */ { .name = "bigger struct (zeroed out)", .arg.inner.flags = O_RDONLY,
@@ -155,7 +158,7 @@ struct flag_test {int err; }; -#define NUM_OPENAT2_FLAG_TESTS 23 +#define NUM_OPENAT2_FLAG_TESTS 31void test_openat2_flags (void ) {
@@ -223,6 +226,30 @@ void test_openat2_flags(void) { .name = "invalid how.resolve and O_PATH", .how.flags = O_PATH, .how.resolve = 0x1337, .err = -EINVAL }, + + /* O_SPECIFIC_FD tests */ + { .name = "O_SPECIFIC_FD", + .how.flags = O_RDONLY | O_SPECIFIC_FD, .how.fd = 42 }, + { .name = "O_SPECIFIC_FD if fd exists", + .how.flags = O_RDONLY | O_SPECIFIC_FD, .how.fd = 2, + .err = -EBUSY }, + { .name = "O_SPECIFIC_FD with fd -1", + .how.flags = O_RDONLY | O_SPECIFIC_FD, .how.fd = -1 }, + { .name = "fd without O_SPECIFIC_FD", + .how.flags = O_RDONLY, .how.fd = 42, + .err = -EINVAL }, + { .name = "fd -1 without O_SPECIFIC_FD", + .how.flags = O_RDONLY, .how.fd = -1, + .err = -EINVAL }, + { .name = "existing fd without O_SPECIFIC_FD", + .how.flags = O_RDONLY, .how.fd = 2, + .err = -EINVAL }, + { .name = "Non-zero padding with O_SPECIFIC_FD", + .how.flags = O_RDONLY | O_SPECIFIC_FD, .how.fd = 42, + .how.__padding = 42, .err = -EINVAL }, + { .name = "Non-zero padding without O_SPECIFIC_FD", + .how.flags = O_RDONLY, + .how.__padding = 42, .err = -EINVAL }, };BUILD_BUG_ON (ARRAY_LEN (tests) !=NUM_OPENAT2_FLAG_TESTS );
@@ -268,6 +295,10 @@ void test_openat2_flags(void)if (!(test->how.flags &O_LARGEFILE )) fdflags &= ~O_LARGEFILE ; failed |= (fdflags != test->how.flags); + +if (test->how.flags &O_SPECIFIC_FD + && test->how.fd !=-1 ) + failed |= (fd != test->how.fd); }if (failed) {
Signature · 1 line
2.26.2
Re: [PATCH v5 2/3] fs: openat2: Extend open_how to allow userspace-selected fds
[CC += linux-api]
Quoted 382 lines · Josh Triplett
Inspired by the X protocol's handling of XIDs, allow userspace to select the file descriptor opened by openat2, so that it can use the resulting file descriptor in subsequent system calls without waiting for the response to openat2.
In
io_uring, this allows sequences like openat2/read/close without waiting for the openat2 to complete. Multiple such sequences can overlap, as long as each uses a distinct file descriptor.Add a new
O_SPECIFIC_FDopen flag to enable this behavior, only accepted by openat2 for now (ignored by open/openat like all unknown flags). Add an fd field to structopen_how(along with appropriate padding, and verify that the padding is 0 to allow replacing the padding with a field in the future).The file table has a corresponding new function
get_specific_unused_fd_flags, which gets the specified file descriptor ifO_SPECIFIC_FDis set (and the fd isn't -1); otherwise it falls back toget_unused_fd_flags, to simplify callers.The specified file descriptor must not already be open; if it is,
get_specific_unused_fd_flagswill fail with-EBUSY. This helps catch userspace errors.When
O_SPECIFIC_FDis set, and fd is not -1, openat2 will use the specified file descriptor rather than finding the lowest available one.Test program:
#include <err.h> #include <fcntl.h> #include <stdio.h> #include <unistd.h> int main(void) { struct open_how how = { .flags = O_RDONLY | O_SPECIFIC_FD, .fd = 42 }; int fd = openat2(AT_FDCWD, "/dev/null", &how, sizeof(how)); if (fd < 0) err(1, "openat2"); printf("fd=%d\n", fd); // prints fd=42 return 0; }Signed-off-by Josh Triplett <[email protected]>--- fs/fcntl.c | 2 +- fs/file.c | 39 +++++++++++++++++++ fs/io_uring.c | 3 +- fs/open.c | 8 +++- include/linux/fcntl.h | 5 ++- include/linux/file.h | 3 ++ include/uapi/asm-generic/fcntl.h | 4 ++ include/uapi/linux/openat2.h | 3 ++ tools/testing/selftests/openat2/helpers.c | 2 +- tools/testing/selftests/openat2/helpers.h | 21 +++++++--- .../testing/selftests/openat2/openat2_test.c | 35 ++++++++++++++++- 11 files changed, 111 insertions(+), 14 deletions(-)fs/fcntl.c +1
-1@@ -1033,7 +1033,7 @@ static int __init fcntl_init(void)* Exceptions: O_NONBLOCK is a two bit define on parisc; O_NDELAY * is defined as O_NONBLOCK on some platforms and not on others. */ -BUILD_BUG_ON (21 -1 /* for O_RDONLY being 0 */ != +BUILD_BUG_ON (22 -1 /* for O_RDONLY being 0 */ !=HWEIGHT32 ( (VALID_OPEN_FLAGS & ~(O_NONBLOCK |O_NDELAY )) | __FMODE_EXEC | __FMODE_NONOTIFY));fs/file.c +39
-0@@ -567,6 +567,45 @@ void put_unused_fd(unsigned int fd)EXPORT_SYMBOL (put_unused_fd); +int __get_specific_unused_fd_flags (unsigned int fd,unsigned int flags, +unsigned long nofile) +{ +int ret; +struct fdtable *fdt; +struct files_struct *files = current->files; + +if (!(flags &O_SPECIFIC_FD ) || fd ==UINT_MAX ) +return __get_unused_fd_flags (flags, nofile); + +if (fd >= nofile) +return -EBADF ; + +spin_lock (&files->file_lock); + ret =expand_files (files, fd); +if (unlikely (ret <0 )) + goto out_unlock; + fdt =files_fdtable (files); +if (fdt->fd[fd]) { + ret = -EBUSY ; + goto out_unlock; + } +__set_open_fd (fd, fdt); +if (flags &O_CLOEXEC ) +__set_close_on_exec (fd, fdt); +else +__clear_close_on_exec (fd, fdt); + ret = fd; + +out_unlock: +spin_unlock (&files->file_lock); +return ret; +} + +int get_specific_unused_fd_flags (unsigned int fd,unsigned int flags) +{ +return __get_specific_unused_fd_flags (fd, flags,rlimit (RLIMIT_NOFILE )); +} +/* * Install a file pointer in the fd array. * fs/io_uring.c +2
-1@@ -2997,7 +2997,8 @@ static int io_openat2(struct io_kiocb *req, bool force_nonblock)if (ret) goto err; - ret =__get_unused_fd_flags (req->open.how.flags, req->open.nofile); + ret =__get_specific_unused_fd_flags (req->open.how.fd, + req->open.how.flags, req->open.nofile);if (ret <0 ) goto err;fs/open.c +7
-1@@ -962,6 +962,8 @@ inline struct open_how build_open_how(int flags, umode_t mode) .mode = mode & S_IALLUGO, }; +/* O_SPECIFIC_FD doesn't work in open calls that use build_open_how. */ + how.flags &= ~O_SPECIFIC_FD ;/* O_PATH beats everything else. */ if (how.flags &O_PATH ) how.flags &=O_PATH_FLAGS ;@@ -989,6 +991,10 @@ inline int build_open_flags(const struct open_how *how, struct open_flags *op)return -EINVAL ;if (how->resolve & ~VALID_RESOLVE_FLAGS )return -EINVAL ; +if (how->__padding !=0 ) +return -EINVAL ; +if (!(flags &O_SPECIFIC_FD ) && how->fd !=0 ) +return -EINVAL ;/* Deal with the mode. */ if (WILL_CREATE (flags)) {@@ -1143,7 +1149,7 @@ static long do_sys_openat2(int dfd, const char __user *filename,if (IS_ERR (tmp))return PTR_ERR (tmp); - fd =get_unused_fd_flags (how->flags); + fd =get_specific_unused_fd_flags (how->fd, how->flags);if (fd >=0 ) {struct file *f =do_filp_open (dfd, tmp, &op);if (IS_ERR (f)) {include/linux/fcntl.h +3
-2@@ -10,7 +10,7 @@ (O_RDONLY |O_WRONLY |O_RDWR |O_CREAT |O_EXCL |O_NOCTTY |O_TRUNC | \O_APPEND |O_NDELAY |O_NONBLOCK |O_NDELAY | __O_SYNC |O_DSYNC | \FASYNC |O_DIRECT |O_LARGEFILE |O_DIRECTORY |O_NOFOLLOW | \ -O_NOATIME |O_CLOEXEC |O_PATH | __O_TMPFILE) +O_NOATIME |O_CLOEXEC |O_PATH | __O_TMPFILE |O_SPECIFIC_FD ) /* List of all valid flags for the how->upgrade_mask argument: */#define VALID_UPGRADE_FLAGS \@@ -23,7 +23,8 @@/* List of all open_how "versions". */ #define OPEN_HOW_SIZE_VER0 24/* sizeof first published struct */ -#define OPEN_HOW_SIZE_LATEST OPEN_HOW_SIZE_VER0 +#define OPEN_HOW_SIZE_VER1 32/* added fd and pad */ +#define OPEN_HOW_SIZE_LATEST OPEN_HOW_SIZE_VER1#ifndef force_o_largefile#define force_o_largefile () (!IS_ENABLED(CONFIG_ARCH_32BIT_OFF_T))include/linux/file.h +3
-0@@ -87,6 +87,9 @@ extern void set_close_on_exec(unsigned int fd, int flag);extern bool get_close_on_exec (unsigned int fd);extern int __get_unused_fd_flags (unsigned flags,unsigned long nofile);extern int get_unused_fd_flags (unsigned flags); +extern int __get_specific_unused_fd_flags (unsigned int fd,unsigned int flags, +unsigned long nofile); +extern int get_specific_unused_fd_flags (unsigned int fd,unsigned int flags);extern void put_unused_fd (unsigned int fd);extern unsigned int increase_min_fd (unsigned int num);include/uapi/asm-generic/fcntl.h +4
-0@@ -89,6 +89,10 @@#define __O_TMPFILE 020000000#endif +#ifndef O_SPECIFIC_FD +#define O_SPECIFIC_FD 01000000000/* open as specified fd */ +#endif +/* a horrid kludge trying to make sure that this will fail on old kernels */ #define O_TMPFILE (__O_TMPFILE | O_DIRECTORY)#define O_TMPFILE_MASK (__O_TMPFILE | O_DIRECTORY | O_CREAT)include/uapi/linux/openat2.h +3
-0@@ -15,11 +15,14 @@* @flags: O_* flags. * @mode: O_CREAT/O_TMPFILE file mode. * @resolve: RESOLVE_* flags. +* @fd: Specific file descriptor to use, for O_SPECIFIC_FD. */ struct open_how {__u64 flags;__u64 mode;__u64 resolve; +__u32 fd; +__u32 __padding;/* Must be zeroed */ };/* how->resolve flags for openat2(2). */ tools/testing/selftests/openat2/helpers.c +1
-1@@ -98,7 +98,7 @@ void __attribute__((constructor)) init(void)struct open_how how = {};int fd; -BUILD_BUG_ON (sizeof (struct open_how ) !=OPEN_HOW_SIZE_VER0 ); +BUILD_BUG_ON (sizeof (struct open_how ) !=OPEN_HOW_SIZE_VER1 );/* Check openat2(2) support. */ fd =sys_openat2 (AT_FDCWD ,"." , &how);tools/testing/selftests/openat2/helpers.h +15
-6@@ -24,28 +24,37 @@#endif /* SYS_openat2 */ /* -* Arguments for how openat2(2) should open the target path. If @resolve is -* zero, then openat2(2) operates very similarly to openat(2). +* Arguments for how openat2(2) should open the target path. If only @flags and +* @mode are non-zero, then openat2(2) operates very similarly to openat(2). * -* However, unlike openat(2), unknown bits in @flags result in -EINVAL rather -* than being silently ignored. @mode must be zero unless one of {O_CREAT, -* O_TMPFILE} are set. +* However, unlike openat(2), unknown or invalid bits in @flags result in +* -EINVAL rather than being silently ignored. @mode must be zero unless one of +* {O_CREAT, O_TMPFILE} are set. * * @flags: O_* flags. * @mode: O_CREAT/O_TMPFILE file mode. * @resolve: RESOLVE_* flags. +* @fd: Specific file descriptor to use, for O_SPECIFIC_FD. */ struct open_how {__u64 flags;__u64 mode;__u64 resolve; +__u32 fd; +__u32 __padding;/* Must be zeroed */ }; +/* List of all open_how "versions". */ #define OPEN_HOW_SIZE_VER0 24/* sizeof first published struct */ -#define OPEN_HOW_SIZE_LATEST OPEN_HOW_SIZE_VER0 +#define OPEN_HOW_SIZE_VER1 32/* added fd and pad */ +#define OPEN_HOW_SIZE_LATEST OPEN_HOW_SIZE_VER1bool needs_openat2 (const struct open_how *how); +#ifndef O_SPECIFIC_FD +#define O_SPECIFIC_FD 01000000000 +#endif +#ifndef RESOLVE_IN_ROOT /* how->resolve flags for openat2(2). */ #define RESOLVE_NO_XDEV 0x01/* Block mount-point crossings tools/testing/selftests/openat2/openat2_test.c +33
-3@@ -40,7 +40,7 @@ struct struct_test {int err; }; -#define NUM_OPENAT2_STRUCT_TESTS 7 +#define NUM_OPENAT2_STRUCT_TESTS 8#define NUM_OPENAT2_STRUCT_VARIATIONS 13void test_openat2_struct (void )@@ -52,6 +52,9 @@ void test_openat2_struct(void) { .name = "normal struct", .arg.inner.flags = O_RDONLY, .size = sizeof(struct open_how) }, + { .name = "v0 struct", + .arg.inner.flags = O_RDONLY, + .size = OPEN_HOW_SIZE_VER0 }, /* Bigger struct, with zeroed out end. */ { .name = "bigger struct (zeroed out)", .arg.inner.flags = O_RDONLY,@@ -155,7 +158,7 @@ struct flag_test {int err; }; -#define NUM_OPENAT2_FLAG_TESTS 23 +#define NUM_OPENAT2_FLAG_TESTS 31void test_openat2_flags (void ) {@@ -223,6 +226,30 @@ void test_openat2_flags(void) { .name = "invalid how.resolve and O_PATH", .how.flags = O_PATH, .how.resolve = 0x1337, .err = -EINVAL }, + + /* O_SPECIFIC_FD tests */ + { .name = "O_SPECIFIC_FD", + .how.flags = O_RDONLY | O_SPECIFIC_FD, .how.fd = 42 }, + { .name = "O_SPECIFIC_FD if fd exists", + .how.flags = O_RDONLY | O_SPECIFIC_FD, .how.fd = 2, + .err = -EBUSY }, + { .name = "O_SPECIFIC_FD with fd -1", + .how.flags = O_RDONLY | O_SPECIFIC_FD, .how.fd = -1 }, + { .name = "fd without O_SPECIFIC_FD", + .how.flags = O_RDONLY, .how.fd = 42, + .err = -EINVAL }, + { .name = "fd -1 without O_SPECIFIC_FD", + .how.flags = O_RDONLY, .how.fd = -1, + .err = -EINVAL }, + { .name = "existing fd without O_SPECIFIC_FD", + .how.flags = O_RDONLY, .how.fd = 2, + .err = -EINVAL }, + { .name = "Non-zero padding with O_SPECIFIC_FD", + .how.flags = O_RDONLY | O_SPECIFIC_FD, .how.fd = 42, + .how.__padding = 42, .err = -EINVAL }, + { .name = "Non-zero padding without O_SPECIFIC_FD", + .how.flags = O_RDONLY, + .how.__padding = 42, .err = -EINVAL }, };BUILD_BUG_ON (ARRAY_LEN (tests) !=NUM_OPENAT2_FLAG_TESTS );@@ -268,6 +295,10 @@ void test_openat2_flags(void)if (!(test->how.flags &O_LARGEFILE )) fdflags &= ~O_LARGEFILE ; failed |= (fdflags != test->how.flags); + +if (test->how.flags &O_SPECIFIC_FD + && test->how.fd !=-1 ) + failed |= (fd != test->how.fd); }if (failed) { --2.26.2
Signature · 3 lines
Michael Kerrisk Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/ Linux/UNIX System Programming Training: http://man7.org/training/
Re: [PATCH v5 2/3] fs: openat2: Extend open_how to allow userspace-selected fds
Quoted 13 lines · Michael Kerrisk (man-pages)
[CC += linux-api]
Quoted 9 lines · Josh Triplett
Inspired by the X protocol's handling of XIDs, allow userspace to select the file descriptor opened by openat2, so that it can use the resulting file descriptor in subsequent system calls without waiting for the response to openat2.
In
io_uring, this allows sequences like openat2/read/close without waiting for the openat2 to complete. Multiple such sequences can overlap, as long as each uses a distinct file descriptor.
If this is primarily an io_uring feature, then why burden the normal openat2 API with this?
Add this flag to the io_uring API, by all means.
This would also allow Implementing a private fd table for io_uring. I.e. add a flag interpreted by file ops (IORING_PRIVATE_FD), including openat2 and freely use the private fd space without having to worry about interactions with other parts of the system.
Thanks,
Miklos
Re: [PATCH v5 2/3] fs: openat2: Extend open_how to allow userspace-selected fds
Quoted 12 lines · Miklos Szeredi
Quoted 10 lines · Michael Kerrisk (man-pages)
[CC += linux-api]
Quoted 6 lines · Josh Triplett
Inspired by the X protocol's handling of XIDs, allow userspace to select the file descriptor opened by openat2, so that it can use the resulting file descriptor in subsequent system calls without waiting for the response to openat2.
Quoted 6 lines
Quoted 3 lines
Quoted 3 lines
In
io_uring, this allows sequences like openat2/read/close without waiting for the openat2 to complete. Multiple such sequences can overlap, as long as each uses a distinct file descriptor.If this is primarily an
io_uringfeature, then why burden the normal openat2 API with this?
This feature was inspired by io_uring; it isn't exclusively of value with io_uring. (And io_uring doesn't normally change the semantics of syscalls.)
Quoted 4 lines
This would also allow Implementing a private fd table for
io_uring. I.e. add a flag interpreted by file ops (IORING_PRIVATE_FD), including openat2 and freely use the private fd space without having to worry about interactions with other parts of the system.
I definitely don't want to add a special kind of file descriptor that doesn't work in normal syscalls taking file descriptors. A file descriptor allocated via O_SPECIFIC_FD is an entirely normal file descriptor, and works anywhere a file descriptor normally works.
Re: [PATCH v5 2/3] fs: openat2: Extend open_how to allow userspace-selected fds
Quoted 18 lines · Josh Triplett
Quoted 16 lines · Miklos Szeredi
Quoted 13 lines · Michael Kerrisk (man-pages)
[CC += linux-api]
Quoted 9 lines · Josh Triplett
Inspired by the X protocol's handling of XIDs, allow userspace to select the file descriptor opened by openat2, so that it can use the resulting file descriptor in subsequent system calls without waiting for the response to openat2.
In
io_uring, this allows sequences like openat2/read/close without waiting for the openat2 to complete. Multiple such sequences can overlap, as long as each uses a distinct file descriptor.
Quoted 6 lines
Quoted 2 lines
If this is primarily an
io_uringfeature, then why burden the normal openat2 API with this?This feature was inspired by
io_uring; it isn't exclusively of value withio_uring. (Andio_uringdoesn't normally change the semantics of syscalls.)
What's the use case of O_SPECIFIC_FD beyond io_uring?
Quoted 10 lines
Quoted 5 lines
This would also allow Implementing a private fd table for
io_uring. I.e. add a flag interpreted by file ops (IORING_PRIVATE_FD), including openat2 and freely use the private fd space without having to worry about interactions with other parts of the system.I definitely don't want to add a special kind of file descriptor that doesn't work in normal syscalls taking file descriptors. A file descriptor allocated via
O_SPECIFIC_FDis an entirely normal file descriptor, and works anywhere a file descriptor normally works.
What's the use case of allocating a file descriptor within io_uring and using it outside of io_uring?
Thanks,
Miklos
Re: [PATCH v5 2/3] fs: openat2: Extend open_how to allow userspace-selected fds
Quoted 18 lines · Miklos Szeredi
Quoted 17 lines · Josh Triplett
Quoted 16 lines · Miklos Szeredi
Quoted 13 lines · Michael Kerrisk (man-pages)
[CC += linux-api]
Quoted 9 lines · Josh Triplett
Inspired by the X protocol's handling of XIDs, allow userspace to select the file descriptor opened by openat2, so that it can use the resulting file descriptor in subsequent system calls without waiting for the response to openat2.
In
io_uring, this allows sequences like openat2/read/close without waiting for the openat2 to complete. Multiple such sequences can overlap, as long as each uses a distinct file descriptor.
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
If this is primarily an
io_uringfeature, then why burden the normal openat2 API with this?This feature was inspired by
io_uring; it isn't exclusively of value withio_uring. (Andio_uringdoesn't normally change the semantics of syscalls.)What's the use case of
O_SPECIFIC_FDbeyondio_uring?
Avoiding a call to dup2 and close, if you need something as a specific file descriptor, such as when setting up to exec something, or when debugging a program.
I don't expect it to be as widely used as with io_uring, but I also don't want io_uring versions of syscalls to diverge from the underlying syscalls, and this would be a heavy divergence.
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
This would also allow Implementing a private fd table for
io_uring. I.e. add a flag interpreted by file ops (IORING_PRIVATE_FD), including openat2 and freely use the private fd space without having to worry about interactions with other parts of the system.
Quoted 7 lines
Quoted 4 lines
I definitely don't want to add a special kind of file descriptor that doesn't work in normal syscalls taking file descriptors. A file descriptor allocated via
O_SPECIFIC_FDis an entirely normal file descriptor, and works anywhere a file descriptor normally works.What's the use case of allocating a file descriptor within
io_uringand using it outside ofio_uring?
Calling a syscall not provided via io_uring. Calling a library that doesn't use io_uring. Passing the file descriptor via UNIX socket to another program. Passing the file descriptor via exec to another program. Userspace is modular, and file descriptors are widely used.
Re: [PATCH v5 2/3] fs: openat2: Extend open_how to allow userspace-selected fds
Quoted 29 lines · Josh Triplett
Quoted 18 lines · Miklos Szeredi
Quoted 17 lines · Josh Triplett
Quoted 16 lines · Miklos Szeredi
Quoted 13 lines · Michael Kerrisk (man-pages)
[CC += linux-api]
Quoted 9 lines · Josh Triplett
Inspired by the X protocol's handling of XIDs, allow userspace to select the file descriptor opened by openat2, so that it can use the resulting file descriptor in subsequent system calls without waiting for the response to openat2.
In
io_uring, this allows sequences like openat2/read/close without waiting for the openat2 to complete. Multiple such sequences can overlap, as long as each uses a distinct file descriptor.Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
If this is primarily an
io_uringfeature, then why burden the normal openat2 API with this?This feature was inspired by
io_uring; it isn't exclusively of value withio_uring. (Andio_uringdoesn't normally change the semantics of syscalls.)What's the use case of
O_SPECIFIC_FDbeyondio_uring?
Quoted 7 lines
Avoiding a call to dup2 and close, if you need something as a specific file descriptor, such as when setting up to exec something, or when debugging a program.
I don't expect it to be as widely used as with
io_uring, but I also don't wantio_uringversions of syscalls to diverge from the underlying syscalls, and this would be a heavy divergence.
What are the plans for those syscalls that don't easily lend themselves to this modification (such as accept(2))? Do we want to introduce another variant of these? Is that really worth it? If not, we are faced with the same divergence.
Compared to that, having a common flag for file ops to enable the use of fixed and private file descriptors is a clean and well contained interface.
Quoted 10 lines
Quoted 10 lines
Quoted 9 lines
Quoted 4 lines
This would also allow Implementing a private fd table for
io_uring. I.e. add a flag interpreted by file ops (IORING_PRIVATE_FD), including openat2 and freely use the private fd space without having to worry about interactions with other parts of the system.I definitely don't want to add a special kind of file descriptor that doesn't work in normal syscalls taking file descriptors. A file descriptor allocated via
O_SPECIFIC_FDis an entirely normal file descriptor, and works anywhere a file descriptor normally works.
Quoted 7 lines
Quoted 2 lines
What's the use case of allocating a file descriptor within
io_uringand using it outside ofio_uring?Calling a syscall not provided via
io_uring. Calling a library that doesn't useio_uring. Passing the file descriptor via UNIX socket to another program. Passing the file descriptor via exec to another program. Userspace is modular, and file descriptors are widely used.
I mean, you could open the file descriptor outside of io_uring in such cases, no? The point of O_SPECIFIC_FD is to be able to perform short sequences of open/dosomething/close without having to block and having to issue separate syscalls. If you're going to issue separate syscalls anyway, then I see no point in doing the open within io_uring. Or?
Thanks,
Miklos
Re: [PATCH v5 2/3] fs: openat2: Extend open_how to allow userspace-selected fds
Quoted 34 lines · Miklos Szeredi
Quoted 33 lines · Josh Triplett
Quoted 18 lines · Miklos Szeredi
Quoted 17 lines · Josh Triplett
Quoted 16 lines · Miklos Szeredi
Quoted 13 lines · Michael Kerrisk (man-pages)
[CC += linux-api]
Quoted 9 lines · Josh Triplett
Inspired by the X protocol's handling of XIDs, allow userspace to select the file descriptor opened by openat2, so that it can use the resulting file descriptor in subsequent system calls without waiting for the response to openat2.
In
io_uring, this allows sequences like openat2/read/close without waiting for the openat2 to complete. Multiple such sequences can overlap, as long as each uses a distinct file descriptor.Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
If this is primarily an
io_uringfeature, then why burden the normal openat2 API with this?This feature was inspired by
io_uring; it isn't exclusively of value withio_uring. (Andio_uringdoesn't normally change the semantics of syscalls.)What's the use case of
O_SPECIFIC_FDbeyondio_uring?Avoiding a call to dup2 and close, if you need something as a specific file descriptor, such as when setting up to exec something, or when debugging a program.
Quoted 6 lines
Quoted 3 lines
I don't expect it to be as widely used as with
io_uring, but I also don't wantio_uringversions of syscalls to diverge from the underlying syscalls, and this would be a heavy divergence.What are the plans for those syscalls that don't easily lend themselves to this modification (such as accept(2))?
accept4 has a flags argument with more flags available, so it'd be entirely possible to cleanly extend it further without introducing a new version. The same goes for other fd-producing syscalls that still have flag space available.
This may or may not provide sufficient motivation on its own to introduce a new syscall variant of a syscall that isn't currently extensible.
Quoted 3 lines
Compared to that, having a common flag for file ops to enable the use of fixed and private file descriptors is a clean and well contained interface.
"private" is not a desirable property here. See below. Even if the userspace-specified fd mechanism were to become something only accessible via io_uring (which I'd prefer to avoid), that's not a reason to avoid generating real file descriptors that work anywhere a file descriptor works.
Quoted 13 lines
Quoted 13 lines
Quoted 12 lines
Quoted 9 lines
Quoted 4 lines
This would also allow Implementing a private fd table for
io_uring. I.e. add a flag interpreted by file ops (IORING_PRIVATE_FD), including openat2 and freely use the private fd space without having to worry about interactions with other parts of the system.I definitely don't want to add a special kind of file descriptor that doesn't work in normal syscalls taking file descriptors. A file descriptor allocated via
O_SPECIFIC_FDis an entirely normal file descriptor, and works anywhere a file descriptor normally works.What's the use case of allocating a file descriptor within
io_uringand using it outside ofio_uring?
Quoted 7 lines
Quoted 4 lines
Calling a syscall not provided via
io_uring. Calling a library that doesn't useio_uring. Passing the file descriptor via UNIX socket to another program. Passing the file descriptor via exec to another program. Userspace is modular, and file descriptors are widely used.I mean, you could open the file descriptor outside of
io_uringin such cases, no?
I would prefer to not introduce that limitation in the first place, and instead open normal file descriptors.
Quoted 3 lines
The point of
O_SPECIFIC_FDis to be able to perform short sequences of open/dosomething/close without having to block and having to issue separate syscalls.
"close" is not a required component. It's entirely possible to use io_uring to open a file descriptor, do various things with it, and then leave it open for subsequent usage via either other io_uring chains or standalone syscalls.
Quoted 3 lines
If you're going to issue separate
syscalls anyway, then I see no point in doing the open withinio_uring. Or?
io_uring is not an all-or-nothing proposition. There's value in using io_uring for some operations without converting an entire program (and every library it might potentially use on a file descriptor) entirely to io_uring. Userspace is modular, and file descriptors are a common element used by many different bits of userspace.
Re: [PATCH v5 2/3] fs: openat2: Extend open_how to allow userspace-selected fds
Quoted 6 lines · Josh Triplett
Quoted 2 lines
What are the plans for those syscalls that don't easily lend themselves to this modification (such as accept(2))?
accept4 has a flags argument with more flags available, so it'd be entirely possible to cleanly extend it further without introducing a new version.
Variable argument syscalls, you are thinking?
Quoted 6 lines
Quoted 2 lines
I mean, you could open the file descriptor outside of
io_uringin such cases, no?I would prefer to not introduce that limitation in the first place, and instead open normal file descriptors.
Quoted 8 lines
Quoted 3 lines
The point of
O_SPECIFIC_FDis to be able to perform short sequences of open/dosomething/close without having to block and having to issue separate syscalls."close" is not a required component. It's entirely possible to use
io_uringto open a file descriptor, do various things with it, and then leave it open for subsequent usage via either otherio_uringchains or standalone syscalls.
If this use case arraises, we could add an op to dup/move a private descriptor to a public one. io_uring can return values, right?
Still not convinced...
Thanks,
Miklos
Re: [PATCH v5 2/3] fs: openat2: Extend open_how to allow userspace-selected fds
Quoted 12 lines · Miklos Szeredi
Quoted 11 lines
I would prefer to not introduce that limitation in the first place, and instead open normal file descriptors.
Quoted 3 lines
The point of
O_SPECIFIC_FDis to be able to perform short sequences of open/dosomething/close without having to block and having to issue separate syscalls."close" is not a required component. It's entirely possible to use
io_uringto open a file descriptor, do various things with it, and then leave it open for subsequent usage via either otherio_uringchains or standalone syscalls.
Quoted 4 lines
If this use case arraises, we could add an op to dup/move a private descriptor to a public one.
io_uringcan return values, right?Still not convinced...
Oh, and we haven't even touched on the biggest advantage of a private fd table: not having to dirty a cacheline on fdget/fdput due to the possibility of concurrent close() in a MT application.
I believe this is a sticking point in some big enterprise apps and it may even be a driving force for io_uring.
Thanks,
Miklos
Re: [PATCH v5 2/3] fs: openat2: Extend open_how to allow userspace-selected fds
Quoted 18 lines · Miklos Szeredi
Quoted 15 lines · Miklos Szeredi
Quoted 11 lines
I would prefer to not introduce that limitation in the first place, and instead open normal file descriptors.
Quoted 3 lines
The point of
O_SPECIFIC_FDis to be able to perform short sequences of open/dosomething/close without having to block and having to issue separate syscalls."close" is not a required component. It's entirely possible to use
io_uringto open a file descriptor, do various things with it, and then leave it open for subsequent usage via either otherio_uringchains or standalone syscalls.If this use case arraises, we could add an op to dup/move a private descriptor to a public one.
io_uringcan return values, right?
Quoted 8 lines
Quoted 1 line
Still not convinced...
Oh, and we haven't even touched on the biggest advantage of a private fd table: not having to dirty a cacheline on fdget/fdput due to the possibility of concurrent
close()in a MT application.I believe this is a sticking point in some big enterprise apps and it may even be a driving force for
io_uring.
https://lwn.net/Articles/787473/
And an interesting (very old) article referenced from above, that gives yet a new angle on fd allocation issues:
https://lwn.net/Articles/236843/
A private fd space would be perfect for libraries such as glibc.
Thanks,
Miklos
Re: [PATCH v5 2/3] fs: openat2: Extend open_how to allow userspace-selected fds
Quoted 29 lines · Miklos Szeredi
Quoted 18 lines · Miklos Szeredi
Quoted 15 lines · Miklos Szeredi
Quoted 11 lines
I would prefer to not introduce that limitation in the first place, and instead open normal file descriptors.
Quoted 3 lines
The point of
O_SPECIFIC_FDis to be able to perform short sequences of open/dosomething/close without having to block and having to issue separate syscalls."close" is not a required component. It's entirely possible to use
io_uringto open a file descriptor, do various things with it, and then leave it open for subsequent usage via either otherio_uringchains or standalone syscalls.If this use case arraises, we could add an op to dup/move a private descriptor to a public one.
io_uringcan return values, right?Quoted 8 lines
Quoted 1 line
Still not convinced...
Oh, and we haven't even touched on the biggest advantage of a private fd table: not having to dirty a cacheline on fdget/fdput due to the possibility of concurrent
close()in a MT application.I believe this is a sticking point in some big enterprise apps and it may even be a driving force for
io_uring.
Quoted 8 lines
https://lwn.net/Articles/787473/
And an interesting (very old) article referenced from above, that gives yet a new angle on fd allocation issues:
https://lwn.net/Articles/236843/
A private fd space would be perfect for libraries such as glibc.
Ah, io_uring already implements a fixed private fd table via io_uring_register(IORING_REGISTER_FILES,...), we just need a way to wire up open, socket, accept, etc. to fill a slot in that table instead of, or in addition to allocating a slot in the fd_table.
Thanks,
Miklos
Re: [PATCH v5 2/3] fs: openat2: Extend open_how to allow userspace-selected fds
In reply to Miklos SzerediQuoted 9 lines · Miklos Szeredi
Quoted 6 lines · Josh Triplett
Quoted 2 lines
What are the plans for those syscalls that don't easily lend themselves to this modification (such as accept(2))?
accept4 has a flags argument with more flags available, so it'd be entirely possible to cleanly extend it further without introducing a new version.
Variable argument syscalls, you are thinking?
That or repurposing an existing pointer-sized argument as an open_how-style struct, yes. But in any case, I'm not proposing that; I'm proposing changes to the existing highly extensible openat2 syscall.
Quoted 10 lines
Quoted 10 lines
Quoted 2 lines
I mean, you could open the file descriptor outside of
io_uringin such cases, no?I would prefer to not introduce that limitation in the first place, and instead open normal file descriptors.
Quoted 3 lines
The point of
O_SPECIFIC_FDis to be able to perform short sequences of open/dosomething/close without having to block and having to issue separate syscalls.
Quoted 6 lines
Quoted 4 lines
"close" is not a required component. It's entirely possible to use
io_uringto open a file descriptor, do various things with it, and then leave it open for subsequent usage via either otherio_uringchains or standalone syscalls.If this use case arraises,
This wasn't a hypothetical "someone might want this". I'm stating that this is a requirement I'm seeking to meet with this patch series, and one I intend to use. The primary use case is interoperability with other code using file descriptors and not using io_uring.
PATCH v5 3/3 fs: pipe2: Support O_SPECIFIC_FD
In reply to Josh TriplettThis allows the caller of pipe2 to specify one or both file descriptors rather than having them automatically use the lowest available file descriptor. The caller can specify either file descriptor as -1 to allow that file descriptor to use the lowest available.
--- fs/pipe.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-)
fs/pipe.c +12 -4
@@ -936,19 +936,19 @@ static int __do_pipe_flags(int *fd, struct file **files, int flags)int error;int fdw, fdr; -if (flags & ~(O_CLOEXEC |O_NONBLOCK |O_DIRECT )) +if (flags & ~(O_CLOEXEC |O_NONBLOCK |O_DIRECT |O_SPECIFIC_FD ))return -EINVAL ; error =create_pipe_files (files, flags);if (error)return error; - error =get_unused_fd_flags (flags); + error =get_specific_unused_fd_flags (fd[0 ], flags);if (error <0 ) goto err_read_pipe; fdr = error; - error =get_unused_fd_flags (flags); + error =get_specific_unused_fd_flags (fd[1 ], flags);if (error <0 ) goto err_fdr; fdw = error;
@@ -969,7 +969,11 @@ static int __do_pipe_flags(int *fd, struct file **files, int flags)int do_pipe_flags (int *fd,int flags) {struct file *files[2 ]; -int error =__do_pipe_flags (fd, files, flags); +int error; + +if (flags &O_SPECIFIC_FD ) +return -EINVAL ; + error =__do_pipe_flags (fd, files, flags);if (!error) {fd_install (fd[0 ], files[0 ]);fd_install (fd[1 ], files[1 ]);
@@ -987,6 +991,10 @@ static int do_pipe2(int __user *fildes, int flags)int fd[2 ];int error; +if (flags &O_SPECIFIC_FD ) +if (copy_from_user (fd, fildes,sizeof (fd))) +return -EFAULT ; + error =__do_pipe_flags (fd, files, flags);if (!error) {if (unlikely (copy_to_user (fildes, fd,sizeof (fd)))) {
Signature · 1 line
2.26.2
Re: [PATCH v5 3/3] fs: pipe2: Support O_SPECIFIC_FD
[CC += linux-api]
Quoted 65 lines · Josh Triplett
This allows the caller of pipe2 to specify one or both file descriptors rather than having them automatically use the lowest available file descriptor. The caller can specify either file descriptor as -1 to allow that file descriptor to use the lowest available.
Signed-off-by Josh Triplett <[email protected]>--- fs/pipe.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-)fs/pipe.c +12
-5@@ -936,19 +936,19 @@ static int __do_pipe_flags(int *fd, struct file **files, int flags)int error;int fdw, fdr; -if (flags & ~(O_CLOEXEC |O_NONBLOCK |O_DIRECT )) +if (flags & ~(O_CLOEXEC |O_NONBLOCK |O_DIRECT |O_SPECIFIC_FD ))return -EINVAL ; error =create_pipe_files (files, flags);if (error)return error; - error =get_unused_fd_flags (flags); + error =get_specific_unused_fd_flags (fd[0 ], flags);if (error <0 ) goto err_read_pipe; fdr = error; - error =get_unused_fd_flags (flags); + error =get_specific_unused_fd_flags (fd[1 ], flags);if (error <0 ) goto err_fdr; fdw = error;@@ -969,7 +969,11 @@ static int __do_pipe_flags(int *fd, struct file **files, int flags)int do_pipe_flags (int *fd,int flags) {struct file *files[2 ]; -int error =__do_pipe_flags (fd, files, flags); +int error; + +if (flags &O_SPECIFIC_FD ) +return -EINVAL ; + error =__do_pipe_flags (fd, files, flags);if (!error) {fd_install (fd[0 ], files[0 ]);fd_install (fd[1 ], files[1 ]);@@ -987,6 +991,10 @@ static int do_pipe2(int __user *fildes, int flags)int fd[2 ];int error; +if (flags &O_SPECIFIC_FD ) +if (copy_from_user (fd, fildes,sizeof (fd))) +return -EFAULT ; + error =__do_pipe_flags (fd, files, flags);if (!error) {if (unlikely (copy_to_user (fildes, fd,sizeof (fd)))) { --2.26.2
Signature · 3 lines
Michael Kerrisk Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/ Linux/UNIX System Programming Training: http://man7.org/training/
Re: [PATCH v5 3/3] fs: pipe2: Support O_SPECIFIC_FD
In reply to Josh Triplett* Josh Triplett:
Quoted 51 lines
This allows the caller of pipe2 to specify one or both file descriptors rather than having them automatically use the lowest available file descriptor. The caller can specify either file descriptor as -1 to allow that file descriptor to use the lowest available.
Signed-off-by Josh Triplett <[email protected]>--- fs/pipe.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-)fs/pipe.c +8
-4@@ -936,19 +936,19 @@ static int __do_pipe_flags(int *fd, struct file **files, int flags)int error;int fdw, fdr; -if (flags & ~(O_CLOEXEC |O_NONBLOCK |O_DIRECT )) +if (flags & ~(O_CLOEXEC |O_NONBLOCK |O_DIRECT |O_SPECIFIC_FD ))return -EINVAL ; error =create_pipe_files (files, flags);if (error)return error; - error =get_unused_fd_flags (flags); + error =get_specific_unused_fd_flags (fd[0 ], flags);if (error <0 ) goto err_read_pipe; fdr = error; - error =get_unused_fd_flags (flags); + error =get_specific_unused_fd_flags (fd[1 ], flags);if (error <0 ) goto err_fdr; fdw = error;@@ -969,7 +969,11 @@ static int __do_pipe_flags(int *fd, struct file **files, int flags)int do_pipe_flags (int *fd,int flags) {struct file *files[2 ]; -int error =__do_pipe_flags (fd, files, flags); +int error; + +if (flags &O_SPECIFIC_FD ) +return -EINVAL ; + error =__do_pipe_flags (fd, files, flags);if (!error) {fd_install (fd[0 ], files[0 ]);fd_install (fd[1 ], files[1 ]);
fs/pipe.c static int do_pipe2(int __user *fildes, int flags) · line 1000
Quoted 10 lines
fs/pipe.c +4
-0@@ -987,6 +991,10 @@ static int do_pipe2(int __user *fildes, int flags)int fd[2 ];int error; +if (flags &O_SPECIFIC_FD ) +if (copy_from_user (fd, fildes,sizeof (fd))) +return -EFAULT ; + error =__do_pipe_flags (fd, files, flags);if (!error) {if (unlikely (copy_to_user (fildes, fd,sizeof (fd)))) {
Mark, I think this will need (or at least benefit from) some valgrind changes.
Re: [PATCH v5 3/3] fs: pipe2: Support O_SPECIFIC_FD
Quoted 55 lines · Florian Weimer
* Josh Triplett:
Quoted 54 lines
This allows the caller of pipe2 to specify one or both file descriptors rather than having them automatically use the lowest available file descriptor. The caller can specify either file descriptor as -1 to allow that file descriptor to use the lowest available.
Signed-off-by Josh Triplett <[email protected]>--- fs/pipe.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-)fs/pipe.c +8
-4@@ -936,19 +936,19 @@ static int __do_pipe_flags(int *fd, struct file **files, int flags)int error;int fdw, fdr; -if (flags & ~(O_CLOEXEC |O_NONBLOCK |O_DIRECT )) +if (flags & ~(O_CLOEXEC |O_NONBLOCK |O_DIRECT |O_SPECIFIC_FD ))return -EINVAL ; error =create_pipe_files (files, flags);if (error)return error; - error =get_unused_fd_flags (flags); + error =get_specific_unused_fd_flags (fd[0 ], flags);if (error <0 ) goto err_read_pipe; fdr = error; - error =get_unused_fd_flags (flags); + error =get_specific_unused_fd_flags (fd[1 ], flags);if (error <0 ) goto err_fdr; fdw = error;@@ -969,7 +969,11 @@ static int __do_pipe_flags(int *fd, struct file **files, int flags)int do_pipe_flags (int *fd,int flags) {struct file *files[2 ]; -int error =__do_pipe_flags (fd, files, flags); +int error; + +if (flags &O_SPECIFIC_FD ) +return -EINVAL ; + error =__do_pipe_flags (fd, files, flags);if (!error) {fd_install (fd[0 ], files[0 ]);fd_install (fd[1 ], files[1 ]);@@ -987,6 +991,10 @@ static int do_pipe2(int __user *fildes, int flags)int fd[2 ];int error;
Quoted 10 lines
fs/pipe.c static int do_pipe2(int __user *fildes, int flags) · line 1000
Quoted 7 lines
fs/pipe.c +4
-0+if (flags &O_SPECIFIC_FD ) +if (copy_from_user (fd, fildes,sizeof (fd))) +return -EFAULT ; + error =__do_pipe_flags (fd, files, flags);if (!error) {if (unlikely (copy_to_user (fildes, fd,sizeof (fd)))) {Mark, I think this will need (or at least benefit from) some valgrind changes.
Yes, this makes pipe2 read the memory of its first argument from userspace, if and only if its second argument contains the O_SPECIFIC_FD flag.
- Josh Triplett
[CC += linux-api]
Quoted 90 lines · Josh Triplett
5.8 material, not intended for 5.7. Now includes a patch for man-pages, attached to this cover letter.
Inspired by the X protocol's handling of XIDs, allow userspace to select the file descriptor opened by a call like openat2, so that it can use the resulting file descriptor in subsequent system calls without waiting for the response to the initial openat2 syscall.
The first patch is independent of the other two; it allows reserving file descriptors below a certain minimum for userspace-selected fd allocation only.
The second patch implements userspace-selected fd allocation for openat2, introducing a new
O_SPECIFIC_FDflag and an fd field in structopen_how. Inio_uring, this allows sequences like openat2/read/close without waiting for the openat2 to complete. Multiple such sequences can overlap, as long as each uses a distinct file descriptor.The third patch adds userspace-selected fd allocation to pipe2 as well. I did this partly as a demonstration of how simple it is to wire up
O_SPECIFIC_FDsupport for any fd-allocating system call, and partly in the hopes that this may make it more useful to wire upio_uringsupport for pipe2 in the future.v5:
Rename padding field to
__padding.
Add tests for non-zero__padding.
Include patch for man-pages.v4:
Changed fd field to
__u32.
Expanded and consolidated checks that return-EINVALfor invalid arguments. Simplified and commentedbuild_open_how.
Add documentation comment for fd field.
Add kselftests.Thanks to Aleksa Sarai for feedback.
v3:
This new version has an API to atomically increase the minimum fd and return the previous minimum, rather than just getting and setting the minimum; this makes it easier to allocate a range. (A library that might initialize after the program has already opened other file descriptors may need to check for existing open fds in the range after reserving it, and reserve more fds if needed; this can be done entirely in userspace, and we can't really do anything simpler in the kernel due to limitations on file-descriptor semantics, so this patch series avoids introducing any extra complexity in the kernel.)
This new version also supports a
__get_specific_unused_fd_flagscall which accepts the limit forRLIMIT_NOFILEas an argument, analogous to__get_unused_fd_flags, sinceio_uringneeds that to correctly handleRLIMIT_NOFILE.Thanks to Jens Axboe for review and feedback.
v2:
Version 2 was a version incorporated into a larger patch series from Jens Axboe on
io_uring.Josh Triplett (3):
fs: Support setting a minimum fd for "lowest available fd" allocation
fs: openat2: Extendopen_howto allow userspace-selected fds
fs: pipe2: SupportO_SPECIFIC_FDfs/fcntl.c | 2 +- fs/file.c | 62 +++++++++++++++++-- fs/io_uring.c | 3 +- fs/open.c | 8 ++- fs/pipe.c | 16 +++-- include/linux/fcntl.h | 5 +- include/linux/fdtable.h | 1 + include/linux/file.h | 4 ++ include/uapi/asm-generic/fcntl.h | 4 ++ include/uapi/linux/openat2.h | 3 + include/uapi/linux/prctl.h | 3 + kernel/sys.c | 5 ++ tools/testing/selftests/openat2/helpers.c | 2 +- tools/testing/selftests/openat2/helpers.h | 21 +++++-- .../testing/selftests/openat2/openat2_test.c | 35 ++++++++++- 15 files changed, 150 insertions(+), 24 deletions(-)--
2.26.2
Signature · 3 lines
Michael Kerrisk Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/ Linux/UNIX System Programming Training: http://man7.org/training/