nfs: support FMODE_NOWAIT on O_DIRECT reads
Subsystems (3)
cover nfs: support FMODE_NOWAIT on O_DIRECT reads
I had noticed that io_uring always punts O_DIRECT NFS reads to a background thread since the file does not advertise FMODE_NOWAIT.
I am not very familiar with the NFS codebase, but looking around suggests a simple change to nfs_start_io_direct is all that is required to properly support this functionality. On the request issue side, it seems everything in NFS is actually run in the background (post this lock change), and the completion codepaths all look to have no similar locking semantics.
I unfortunately do not have the means to test the performance improvement, since even without this change my local network is the bottleneck here.
However I do suspect that there are people that would want this fix ([1]). Applying a similar patch on that GitHub issue did give performance gains.
To convince myself this works at all I did trace io_uring events through with and without the patch.
Using a test app ([2]) to issue O_DIRECT io_uring reads calls io_uring_queue_async_work without this patch, while with it the call is skipped and the completion is queued into io_uring directly from nfs_direct_read_completion.
Patch 1 here adds an unused nfs_start_io_direct_nowait which patch 2 uses in order to safely advertise FMODE_NOWAIT.
[1]: https://github.com/axboe/liburing/issues/1499
[2]: https://github.com/DylanZA/liburing/commit/264c06f1939dfd6b6bc4c967ada5960c4f4f2db3
Dylan Yudaken (2):
nfs: add nowait version of nfs_start_io_direct
nfs: expose FMODE_NOWAIT for O_DIRECT read files
fs/nfs/direct.c | 5 ++++- fs/nfs/file.c | 13 ++++++++++++- fs/nfs/internal.h | 1 + fs/nfs/io.c | 38 ++++++++++++++++++++++++++++++++++++-- 4 files changed, 53 insertions(+), 4 deletions(-)
base-commit: 670b77dfebe7257adc0defbc48a4c43cfdf6c8f6
Signature · 1 line
2.50.1
PATCH 1/2 nfs: add nowait version of nfs_start_io_direct
nfs_start_io_direct might block on existing operations to the same inode. In order to support NOWAIT O_DIRECT reads, add a non-blocking version of this nfs_start_io_direct that just returns -EAGAIN if locks could not be taken.
--- fs/nfs/internal.h | 1 + fs/nfs/io.c | 38 ++++++++++++++++++++++++++++++++++++-- 2 files changed, 37 insertions(+), 2 deletions(-)
fs/nfs/internal.h +1 -0
@@ -532,6 +532,7 @@ extern void nfs_end_io_read(struct inode *inode);extern __must_check int nfs_start_io_write (struct inode *inode);extern void nfs_end_io_write (struct inode *inode);extern __must_check int nfs_start_io_direct (struct inode *inode); +extern __must_check int nfs_start_io_direct_nowait (struct inode *inode);extern void nfs_end_io_direct (struct inode *inode);static inline bool nfs_file_io_is_buffered (struct nfs_inode *nfsi)
fs/nfs/io.c +36 -2
@@ -101,12 +101,15 @@ nfs_end_io_write(struct inode *inode)EXPORT_SYMBOL_GPL (nfs_end_io_write);/* Call with exclusively locked inode->i_rwsem */ -static void nfs_block_buffered (struct nfs_inode *nfsi,struct inode *inode) +static int nfs_block_buffered (struct nfs_inode *nfsi,struct inode *inode,bool nowait) {if (!test_bit (NFS_INO_ODIRECT , &nfsi->flags)) { +if (nowait && !mapping_empty (inode->i_mapping)) +return 1 ;set_bit (NFS_INO_ODIRECT , &nfsi->flags);nfs_sync_mapping (inode->i_mapping); } +return 0 ; }/**
@@ -143,12 +146,43 @@ nfs_start_io_direct(struct inode *inode) err =down_write_killable (&inode->i_rwsem);if (err)return err; -nfs_block_buffered (nfsi, inode); +nfs_block_buffered (nfsi, inode, false);downgrade_write (&inode->i_rwsem);return 0 ; } +/** +* nfs_start_io_direct_nowait - non-blocking variant of nfs_start_io_direct() +* @inode: file inode +* +* Try to declare that a direct I/O operation is about to start without +* blocking. +* Ensure all buffered I/O is blocked. +* If this could not be done without blocking then returns -EAGAIN. +*/ +int +nfs_start_io_direct_nowait (struct inode *inode) +{ +struct nfs_inode *nfsi =NFS_I (inode); + +if (!down_read_trylock (&inode->i_rwsem)) +return -EAGAIN ; +if (test_bit (NFS_INO_ODIRECT , &nfsi->flags)) +return 0 ; +up_read (&inode->i_rwsem); + +/* Slow path: try to flip NFS_INO_ODIRECT without blocking. */ +if (!down_write_trylock (&inode->i_rwsem)) +return -EAGAIN ; +if (nfs_block_buffered (nfsi, inode, true)) { +up_write (&inode->i_rwsem); +return -EAGAIN ; + } +downgrade_write (&inode->i_rwsem); +return 0 ; +} +/** * nfs_end_io_direct - declare that the direct i/o operation is done * @inode: file inode
Signature · 1 line
2.50.1
PATCH 2/2 nfs: expose FMODE_NOWAIT for O_DIRECT read files
In reply to Dylan YudakenO_DIRECT reads already (mostly) handle async requests, with the exception of locking the inode for direct.
Handle async requests properly by using nfs_start_io_direct_nowait, and then expose FMODE_NOWAIT since it's now properly supported.
--- fs/nfs/direct.c | 5 ++++- fs/nfs/file.c | 13 ++++++++++++- 2 files changed, 16 insertions(+), 2 deletions(-)
fs/nfs/direct.c +4 -1
@@ -473,7 +473,10 @@ ssize_t nfs_file_direct_read(struct kiocb *iocb, struct iov_iter *iter, dreq->flags =NFS_ODIRECT_SHOULD_DIRTY ;if (!swap) { - result =nfs_start_io_direct (inode); +if (iocb->ki_flags &IOCB_NOWAIT ) + result =nfs_start_io_direct_nowait (inode); +else + result =nfs_start_io_direct (inode);if (result) {/* release the reference that would usually be * consumed by nfs_direct_read_schedule_iovec()
fs/nfs/file.c +12 -1
@@ -72,8 +72,13 @@ nfs_file_open(struct inode *inode, struct file *filp)return res; res =nfs_open (inode, filp); -if (res ==0 ) +if (res ==0 ) { filp->f_mode |=FMODE_CAN_ODIRECT ; +/* flag NOWAIT on read-only O_DIRECT files only */ +if ((filp->f_flags &O_DIRECT ) && + !(filp->f_mode &FMODE_WRITE )) + filp->f_mode |=FMODE_NOWAIT ; + }return res; }
@@ -705,6 +710,12 @@ ssize_t nfs_file_write(struct kiocb *iocb, struct iov_iter *from)trace_nfs_file_write (iocb, from); +/* +* FMODE_NOWAIT is not set for writable files +*/ +if (WARN_ON_ONCE (iocb->ki_flags &IOCB_NOWAIT )) +return -EAGAIN ; + result =nfs_key_timeout_notify (file, inode);if (result)return result;
Signature · 1 line
2.50.1