io_uring: add support for IORING_OP_GETDENTS
Quoted 5 lines
IORING_OP_GETDENTSbehaves much like getdents64(2) and takes the same arguments, but with a small twist: it takes an additional offset argument, and reading from the specified directory starts at the given offset.
Quoted 7 lines
For the first
IORING_OP_GETDENTScall on a directory, the offset parameter can be set to zero, and for subsequent calls, it can be set to the ->d_off field of the last structlinux_dirent64returned by the previousIORING_OP_GETDENTScall.Internally, if necessary,
IORING_OP_GETDENTSwillvfs_llseek()to the right directory position before callingvfs_getdents().
For my envisioned use case it would be important to support reading from the current file position when offset == -1 (IORING_FEAT_RW_CUR_POS). Among other things, this would let me fully read a directory with fewer round-trips. Normally after the first getdents(), another one must be issued to distinguish between EOF and a short first read. It would be nice to do both calls with linked SQEs so I could immediately know that I've reached the end of the directory.
Quoted 3 lines
IORING_OP_GETDENTSmay or may not update the specified directory's file offset, and the file offset should not be relied upon having any particular value during or after anIORING_OP_GETDENTScall.
Obviously for the above to work, we'd have to get rid of this limitation. Is that possible?
Hello!
Quoted 10 lines · Tavian Barnes
Quoted 10 lines
IORING_OP_GETDENTSbehaves much like getdents64(2) and takes the same arguments, but with a small twist: it takes an additional offset argument, and reading from the specified directory starts at the given offset.For the first
IORING_OP_GETDENTScall on a directory, the offset parameter can be set to zero, and for subsequent calls, it can be set to the ->d_off field of the last structlinux_dirent64returned by the previousIORING_OP_GETDENTScall.
Quoted 5 lines
Quoted 2 lines
Internally, if necessary,
IORING_OP_GETDENTSwillvfs_llseek()to the right directory position before callingvfs_getdents().For my envisioned use case it would be important to support reading from the current file position when offset == -1 (
IORING_FEAT_RW_CUR_POS).
Thank you for the feedback! This is easy enough to do:
fs/io_uring.c +1 -1
@@ -4349,7 +4349,7 @@ static int io_getdents(struct io_kiocb *req, unsigned int issue_flags)/* for vfs_llseek and to serialize ->iterate_shared() on this file */ mutex_lock (&req->file->f_pos_lock); -if (req->file->f_pos != getdents->pos) { +if (getdents->pos !=-1 && req->file->f_pos != getdents->pos) {loff_t res =vfs_llseek (req->file, getdents->pos,SEEK_SET );if (res <0 ) ret = res;
(A corresponding patch to the liburing testsuite is below.)
Quoted 5 lines
Among other things, this would let me fully read a directory with fewer round-trips. Normally after the first
getdents(), another one must be issued to distinguish between EOF and a short first read. It would be nice to do both calls with linked SQEs so I could immediately know that I've reached the end of the directory.
Does that work with this patch?
Quoted 6 lines
Quoted 3 lines
IORING_OP_GETDENTSmay or may not update the specified directory's file offset, and the file offset should not be relied upon having any particular value during or after anIORING_OP_GETDENTScall.Obviously for the above to work, we'd have to get rid of this limitation. Is that possible?
The idea behind wording it this way was that this would allow for further in-kernel parallelization of getdents calls in the future (by not serializing on the current-file-position-mutex), but I can see how this behavior conflicts with your use case.
Also, given how the current implementation _does_ update the directory's file offset, some userspace code will surely end up depending on that behavior, which would make it hard to change in the future.
Also, allowing unserialized in-kernel getdents calls on the same file descriptor would require a bunch more VFS work (including auditing all filesystems), and, to be completely honest, I am unlikely to be doing that work.
All in all, maybe we should just state that IORING_OP_GETDENTS is a seek (if needed) plus a getdents combo that does together update the directory's file offset, and be done with it.
A corresponding incremental patch to the testsuite to test the "off == -1 means current file position" case:
test/getdents.c +42 -0
54 lines not on this page · raw message