io_uring/rw: fix missing NOWAIT check for O_DIRECT start write
Jens Axboe
maintainerPATCH 1/1 io_uring/rw: fix missing NOWAIT check for O_DIRECT start write
When io_uring starts a write, it'll call kiocb_start_write() to bump the super block rwsem, preventing any freezes from happening while that write is in-flight. The freeze side will grab that rwsem for writing, excluding any new writers from happening and waiting for existing writes to finish. But io_uring unconditionally uses kiocb_start_write(), which will block if someone is currently attempting to freeze the mount point. This causes a deadlock where freeze is waiting for previous writes to complete, but the previous writes cannot complete, as the task that is supposed to complete them is blocked waiting on starting a new write. This results in the following stuck trace showing that dependency with the write blocked starting a new write:
task:fio state:D stack:0 pid:886 tgid:886 ppid:876
Call trace:
__switch_to+0x1d8/0x348
__schedule+0x8e8/0x2248
schedule+0x110/0x3f0
percpu_rwsem_wait+0x1e8/0x3f8
__percpu_down_read+0xe8/0x500
io_write+0xbb8/0xff8
io_issue_sqe+0x10c/0x1020
io_submit_sqes+0x614/0x2110
__arm64_sys_io_uring_enter+0x524/0x1038
invoke_syscall+0x74/0x268
el0_svc_common.constprop.0+0x160/0x238
do_el0_svc+0x44/0x60
el0_svc+0x44/0xb0
el0t_64_sync_handler+0x118/0x128
el0t_64_sync+0x168/0x170
INFO: task fsfreeze:7364 blocked for more than 15 seconds.
Not tainted 6.12.0-rc5-00063-g76aaf945701c #7963with the attempting freezer stuck trying to grab the rwsem:
task:fsfreeze state:D stack:0 pid:7364 tgid:7364 ppid:995
Call trace: __switch_to+0x1d8/0x348 __schedule+0x8e8/0x2248 schedule+0x110/0x3f0 percpu_down_write+0x2b0/0x680 freeze_super+0x248/0x8a8 do_vfs_ioctl+0x149c/0x1b18 __arm64_sys_ioctl+0xd0/0x1a0 invoke_syscall+0x74/0x268 el0_svc_common.constprop.0+0x160/0x238 do_el0_svc+0x44/0x60 el0_svc+0x44/0xb0 el0t_64_sync_handler+0x118/0x128 el0t_64_sync+0x168/0x170
Fix this by having the io_uring side honor IOCB_NOWAIT, and only attempt a blocking grab of the super block rwsem if it isn't set. For normal issue where IOCB_NOWAIT would always be set, this returns -EAGAIN which will have io_uring core issue a blocking attempt of the write. That will in turn also get completions run, ensuring forward progress.
---
io_uring/rw.c +21 -2
@@ -1014,6 +1014,25 @@ int io_read_mshot(struct io_kiocb *req, unsigned int issue_flags)return IOU_OK ; } +static bool io_kiocb_start_write (struct io_kiocb *req,struct kiocb *kiocb) +{ +struct inode *inode; +bool ret; + +if (!(req->flags &REQ_F_ISREG )) +return true; +if (!(kiocb->ki_flags &IOCB_NOWAIT )) { +kiocb_start_write (kiocb); +return true; + } + + inode =file_inode (kiocb->ki_filp); + ret =sb_start_write_trylock (inode->i_sb); +if (ret) +__sb_writers_release (inode->i_sb,SB_FREEZE_WRITE ); +return ret; +} +int io_write (struct io_kiocb *req,unsigned int issue_flags) {bool force_nonblock = issue_flags &IO_URING_F_NONBLOCK ;
@@ -1051,8 +1070,8 @@ int io_write(struct io_kiocb *req, unsigned int issue_flags)if (unlikely (ret))return ret; -if (req->flags &REQ_F_ISREG ) -kiocb_start_write (kiocb); +if (unlikely (!io_kiocb_start_write (req, kiocb))) +return -EAGAIN ; kiocb->ki_flags |=IOCB_WRITE ;if (likely (req->file->f_op->write_iter))
Signature · 1 line
Jens Axboe