vfs: support statx(..., NULL, AT_EMPTY_PATH, ...)
PATCH v3 1/1 vfs: support statx(..., NULL, AT_EMPTY_PATH, ...)
The newly used helper also checks for empty ("") paths.
NULL paths with any flag value other than AT_EMPTY_PATH go the usual route and end up with -EFAULT to retain compatibility (Rust is abusing calls of the sort to detect availability of statx).
This avoids path lookup code, lockref management, memory allocation and in case of NULL path userspace memory access (which can be quite expensive with SMAP on x86_64).
Benchmarked with statx(..., AT_EMPTY_PATH, ...) running on Sapphire Rapids, with the "" path for the first two cases and NULL for the last one.
Results in ops/s:
stock: 4231237 pre-check: 5944063 (+40%) NULL path: 6601619 (+11%/+56%)
---
Diffed against fs-next and assumes c050122bdbb4 ("fs: new helper vfs_empty_path()") from vfs.empty.path is already applied.
WARNING: io_uring remains untested (modulo compilation). I presume Jens has a handy way of making sure things still work.
While the io_uring part can be added at a later date, I'm trying to avoid a scenario where someone has code which works with the NULL path and breaks when moving to io_uring. I am not going to argue about it , worst case changes to io_uring can be trivially dropped and someone(tm) can add their own variant whenever they see fit.
v3:
- also support AT_STATX* flags for NULL and ""
v2:
- support glibc passing AT_NO_AUTOMOUNT | AT_EMPTY_PATH
- tidy up some commentary
- drop the fdget_raw CLASS addition as it is already present in newer trees
fs/internal.h | 2 + fs/stat.c | 107 +++++++++++++++++++++++++++++++++++------------ io_uring/statx.c | 22 ++++++---- 3 files changed, 96 insertions(+), 35 deletions(-)
fs/internal.h +2 -0
@@ -247,6 +247,8 @@ extern const struct dentry_operations ns_dentry_operations;int getname_statx_lookup_flags (int flags);int do_statx (int dfd,struct filename *filename,unsigned int flags,unsigned int mask,struct statx __user *buffer); +int do_statx_fd (int fd,unsigned int flags,unsigned int mask, +struct statx __user *buffer);/* * fs/splice.c:
fs/stat.c +80 -27
146 lines not on this page · raw message
io_uring/statx.c +14 -8
@@ -23,6 +23,7 @@ struct io_statx {int io_statx_prep (struct io_kiocb *req,const struct io_uring_sqe *sqe) { struct io_statx *sx = io_kiocb_to_cmd(req, struct io_statx); +struct filename *filename;const char __user *path;if (sqe->buf_index || sqe->splice_fd_in)
@@ -36,14 +37,16 @@ int io_statx_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe) sx->buffer =u64_to_user_ptr (READ_ONCE (sqe->addr2)); sx->flags =READ_ONCE (sqe->statx_flags); - sx->filename =getname_flags (path, -getname_statx_lookup_flags (sx->flags)); - -if (IS_ERR (sx->filename)) { -int ret =PTR_ERR (sx->filename); - +if ((sx->flags & (AT_EMPTY_PATH |AT_STATX_SYNC_TYPE )) == + (AT_EMPTY_PATH |AT_STATX_SYNC_TYPE ) && +vfs_empty_path (sx->dfd, path)) { sx->filename =NULL ; -return ret; + }else { + filename =getname_flags (path, +getname_statx_lookup_flags (sx->flags)); +if (IS_ERR (filename)) +return PTR_ERR (filename); + sx->filename = filename; } req->flags |=REQ_F_NEED_CLEANUP ;
@@ -58,7 +61,10 @@ int io_statx(struct io_kiocb *req, unsigned int issue_flags)WARN_ON_ONCE (issue_flags &IO_URING_F_NONBLOCK ); - ret =do_statx (sx->dfd, sx->filename, sx->flags, sx->mask, sx->buffer); +if (sx->filename ==NULL ) + ret =do_statx_fd (sx->dfd, sx->flags, sx->mask, sx->buffer); +else + ret =do_statx (sx->dfd, sx->filename, sx->flags, sx->mask, sx->buffer);io_req_set_res (req, ret,0 );return IOU_OK ; }
Signature · 1 line
2.43.0
io_uring/statx.c int io_statx_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe) · line 46
Quoted 5 lines · Mateusz Guzik
io_uring/statx.c +3
-0+if ((sx->flags & (AT_EMPTY_PATH |AT_STATX_SYNC_TYPE )) == + (AT_EMPTY_PATH | AT_STATX_SYNC_TYPE) && + vfs_empty_path(sx->dfd, path)) {
sx->filename= NULL;io_uring/statx.c +0
-1-return ret;
AT_STATX_SYNC_TYPE == AT_STATX_FORCE_SYNC | AT_STATX_DONT_SYNC but AT_STATX_FORCE_SYNC and AT_STATX_DONT_SYNC obviously contradicts with each other. Thus valid uses of statx won't satisfy this condition.
And I guess the condition here should be same as the condition in SYSCALL_DEFINE5(statx) or am I wrong?
Signature · 2 lines
Xi Ruoyao <[email protected]> School of Aerospace Science and Technology, Xidian University
Quoted 8 lines · Xi Ruoyao
io_uring/statx.c int io_statx_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe) · line 46
Quoted 5 lines · Mateusz Guzik
io_uring/statx.c +3
-1+if ((sx->flags & (AT_EMPTY_PATH |AT_STATX_SYNC_TYPE )) == + (AT_EMPTY_PATH |AT_STATX_SYNC_TYPE ) && +vfs_empty_path (sx->dfd, path)) { sx->filename =NULL ; -return ret;
Quoted 4 lines
AT_STATX_SYNC_TYPE==AT_STATX_FORCE_SYNC|AT_STATX_DONT_SYNCbutAT_STATX_FORCE_SYNCandAT_STATX_DONT_SYNCobviously contradicts with each other. Thus valid uses of statx won't satisfy this condition.
I don't know wtf I was thinking, this is indeed bogus.
Quoted 3 lines
And I guess the condition here should be same as the condition in
SYSCALL_DEFINE5(statx)or am I wrong?
That I disagree with. The AUTOMOUNT thing is a glibc-local problem for fstatat. Unless if you mean the if should be of similar sort modulo the flag. :)
I am going to fix this up and write a io_uring testcase, then submit a v4. Maybe today or tomorrow.
Quoted 3 lines
--
Xi Ruoyao <[email protected]>
School of Aerospace Science and Technology, Xidian University
Signature · 1 line
Mateusz Guzik <mjguzik gmail.com>
Christian Brauner
maintainerQuoted 20 lines · Mateusz Guzik
Quoted 8 lines · Xi Ruoyao
io_uring/statx.c int io_statx_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe) · line 46
Quoted 5 lines · Mateusz Guzik
io_uring/statx.c +3
-1+if ((sx->flags & (AT_EMPTY_PATH |AT_STATX_SYNC_TYPE )) == + (AT_EMPTY_PATH |AT_STATX_SYNC_TYPE ) && +vfs_empty_path (sx->dfd, path)) { sx->filename =NULL ; -return ret;Quoted 4 lines
AT_STATX_SYNC_TYPE==AT_STATX_FORCE_SYNC|AT_STATX_DONT_SYNCbutAT_STATX_FORCE_SYNCandAT_STATX_DONT_SYNCobviously contradicts with each other. Thus valid uses of statx won't satisfy this condition.I don't know wtf I was thinking, this is indeed bogus.
Quoted 3 lines
And I guess the condition here should be same as the condition in
SYSCALL_DEFINE5(statx)or am I wrong?
Quoted 6 lines
That I disagree with. The AUTOMOUNT thing is a glibc-local problem for fstatat. Unless if you mean the if should be of similar sort modulo the flag. :)
I am going to fix this up and write a
io_uringtestcase, then submit a v4. Maybe today or tomorrow.
Fwiw, I had already dropped the io_uring bits when I pushed to #vfs.mount. So just put that as a separate patch on top.
Jens Axboe
maintainerQuoted 2 lines · Mateusz Guzik
I am going to fix this up and write a
io_uringtestcase, then submit a v4. Maybe today or tomorrow.
Sounds good, on both incremental and test case. I can just stage it separately in a branch with Christian's branch pulled in.
Thanks!
Signature · 1 line
Jens Axboe
Quoted 19 lines · Mateusz Guzik
The newly used helper also checks for empty ("") paths.
NULL paths with any flag value other than
AT_EMPTY_PATHgo the usual route and end up with-EFAULTto retain compatibility (Rust is abusing calls of the sort to detect availability of statx).This avoids path lookup code, lockref management, memory allocation and in case of NULL path userspace memory access (which can be quite expensive with SMAP on
x86_64).Benchmarked with
statx(..., AT_EMPTY_PATH, ...)running on Sapphire Rapids, with the "" path for the first two cases and NULL for the last one.Results in ops/s:
stock: 4231237
pre-check: 5944063 (+40%)
NULL path: 6601619 (+11%/+56%)
Quoted 8 lines
Signed-off-by Mateusz Guzik <[email protected]>---
Diffed against fs-next and assumes
c050122bdbb4("fs: new helpervfs_empty_path()") from vfs.empty.path is already applied.WARNING:
io_uringremains untested (modulo compilation). I presume Jens has a handy way of making sure things still work.
For non-io_uring part:
On LoongArch the time usage of 10000000 calls:
baseline
Glibc fstat: ./a.out 0.44s user 3.56s system 99% cpu 4.013 total bare statx: ./a.out 0.39s user 3.54s system 99% cpu 3.927 total
patched
Glibc fstat: ./a.out 0.49s user 1.34s system 99% cpu 1.841 total bare statx: ./a.out 0.42s user 1.32s system 99% cpu 1.748 total statx NULL: ./a.out 0.44s user 1.29s system 99% cpu 1.734 total
Signature · 2 lines
Xi Ruoyao <[email protected]> School of Aerospace Science and Technology, Xidian University
Quoted 11 lines · Mateusz Guzik
The newly used helper also checks for empty ("") paths.
NULL paths with any flag value other than
AT_EMPTY_PATHgo the usual route and end up with-EFAULTto retain compatibility (Rust is abusing calls of the sort to detect availability of statx).This avoids path lookup code, lockref management, memory allocation and in case of NULL path userspace memory access (which can be quite expensive with SMAP on
x86_64).[...]
Applied to the vfs.misc branch of the vfs/vfs.git tree. Patches in the vfs.misc branch should appear in linux-next soon.
Please report any outstanding bugs that were missed during review in a new review to the original patch series allowing us to drop it.
It's encouraged to provide Acked-bys and Reviewed-bys even though the patch has now been applied. If possible patch trailers will be updated.
Note that commit hashes shown below are subject to change due to rebase, trailer updates or similar. If in doubt, please check the listed branch.
tree: https://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs.git branch: vfs.misc
[1/1] vfs: support statx(..., NULL, AT_EMPTY_PATH, ...)
https://git.kernel.org/vfs/vfs/c/33b321ac3a51
Quoted 15 lines · Mateusz Guzik
The newly used helper also checks for empty ("") paths.
NULL paths with any flag value other than
AT_EMPTY_PATHgo the usual route and end up with-EFAULTto retain compatibility (Rust is abusing calls of the sort to detect availability of statx).This avoids path lookup code, lockref management, memory allocation and
in case of NULL path userspace memory access (which can be quite expensive with SMAP onx86_64).Benchmarked with
statx(..., AT_EMPTY_PATH, ...)running on Sapphire Rapids, with the "" path for the first two cases and NULL for the last one.
Quoted 6 lines
Results in ops/s:
stock: 4231237
pre-check: 5944063 (+40%)
NULL path: 6601619 (+11%/+56%)Signed-off-by Mateusz Guzik <[email protected]>
Hi Mateusz and Christian,
There's a special case, AT_FDCWD + NULL + AT_EMPTY_PATH, still resulting EFAULT, while AT_FDCWD + "" + AT_EMPTY_PATH is OK (returning the stat of current directory).
I know allowing NULL with AT_FDCWD won't produce any performance gain, but it seems the difference would make the document of the API more nasty.
So is it acceptable to make the kernel "hide" this difference, i.e. accept AT_FDCWD + NULL + AT_EMPTY_PATH as-is AT_FDCWD + "" + AT_EMPTY_PATH?
Signature · 2 lines
Xi Ruoyao <[email protected]> School of Aerospace Science and Technology, Xidian University
Quoted 4 lines · Xi Ruoyao
There's a special case,
AT_FDCWD+ NULL +AT_EMPTY_PATH, still resulting EFAULT, whileAT_FDCWD+ "" +AT_EMPTY_PATHis OK (returning the stat of current directory).
Quoted 8 lines
I know allowing NULL with
AT_FDCWDwon't produce any performance gain, but it seems the difference would make the document of the API more nasty.So is it acceptable to make the kernel "hide" this difference, i.e. accept
AT_FDCWD+ NULL +AT_EMPTY_PATHas-isAT_FDCWD+ "" +AT_EMPTY_PATH?
huh, that indeed makes sense to add. kind of weird this was not sorted out at the time, but i'm not going to pointer a finger at myself :) so ACK from me as far as the idea goes
I presume you can do the honors? :)
Signature · 1 line
Mateusz Guzik <mjguzik gmail.com>
Quoted 14 lines · Mateusz Guzik
Quoted 4 lines · Xi Ruoyao
There's a special case,
AT_FDCWD+ NULL +AT_EMPTY_PATH, still resulting EFAULT, whileAT_FDCWD+ "" +AT_EMPTY_PATHis OK (returning the stat of current directory).Quoted 8 lines
I know allowing NULL with
AT_FDCWDwon't produce any performance gain, but it seems the difference would make the document of the API more nasty.So is it acceptable to make the kernel "hide" this difference, i.e. accept
AT_FDCWD+ NULL +AT_EMPTY_PATHas-isAT_FDCWD+ "" +AT_EMPTY_PATH?
Quoted 5 lines
huh, that indeed makes sense to add. kind of weird this was not sorted out at the time, but i'm not going to pointer a finger at myself :) so ACK from me as far as the idea goes
I presume you can do the honors? :)
Should I use a Fixes: tag in the commit then? (I.e. should it be backported to 6.11?)
Signature · 2 lines
Xi Ruoyao <[email protected]> School of Aerospace Science and Technology, Xidian University