io_uring: remove PF_EXITING checking in io_poll_rewait()
Jens Axboe
maintainerPATCH 1/1 io_uring: remove PF_EXITING checking in io_poll_rewait()
We have two checks of task->flags & PF_EXITING left:
1) In io_req_task_submit(), which is called in task_work and hence always in the context of the original task. That means that req->task == current, and hence checking ->flags is totally fine.
2) In io_poll_rewait(), where we need to stop re-arming poll to prevent it interfering with cancelation. Here, req->task is not necessarily current, and hence the check is racy. Use the ctx refs state instead to check if we need to cancel this request or not.
---
fs/io_uring.c +6 -1
@@ -2114,6 +2114,7 @@ static void io_req_task_submit(struct io_kiocb *req)/* ctx stays valid until unlock, even if we drop all ours ctx->refs */ mutex_lock (&ctx->uring_lock); +/* req->task == current here, checking PF_EXITING is safe */ if (likely (!(req->task->flags &PF_EXITING )))__io_queue_sqe (req);else
@@ -4895,7 +4896,11 @@ static bool io_poll_rewait(struct io_kiocb *req, struct io_poll_iocb *poll) {struct io_ring_ctx *ctx = req->ctx; -if (unlikely (req->task->flags &PF_EXITING )) +/* +* Pairs with spin_unlock() in percpu_ref_kill() +*/ +smp_rmb (); +if (unlikely (percpu_ref_is_dying (&ctx->refs)))WRITE_ONCE (poll->canceled, true);if (!req->result && !READ_ONCE (poll->canceled)) {
Signature · 1 line
Jens Axboe
Quoted 10 lines · Jens Axboe
We have two checks of
task->flags&PF_EXITINGleft:1) In
io_req_task_submit(), which is called intask_workand hence always in the context of the original task. That means thatreq->task== current, and hence checking ->flags is totally fine.2) In
io_poll_rewait(), where we need to stop re-arming poll to prevent it interfering with cancelation. Here,req->taskis not necessarily current, and hence the check is racy. Use the ctx refs state instead to check if we need to cancel this request or not.
Hi Jens,
I saw cases that io_req_task_submit() and io_poll_rewait() in one function, why one is safe and the other one not? btw, it seems both two executes in task_work context..and task_work_add() may fail and then work goes to system_wq, is that case safe?
Quoted 31 lines
Signed-off-by Jens Axboe <[email protected]>---
fs/io_uring.c +6
-1@@ -2114,6 +2114,7 @@ static void io_req_task_submit(struct io_kiocb *req)/* ctx stays valid until unlock, even if we drop all ours ctx->refs */ mutex_lock (&ctx->uring_lock); +/* req->task == current here, checking PF_EXITING is safe */ if (likely (!(req->task->flags &PF_EXITING )))__io_queue_sqe (req);else @@ -4895,7 +4896,11 @@ static bool io_poll_rewait(struct io_kiocb *req, struct io_poll_iocb *poll) {struct io_ring_ctx *ctx = req->ctx; -if (unlikely (req->task->flags &PF_EXITING )) +/* +* Pairs with spin_unlock() in percpu_ref_kill() +*/ +smp_rmb (); +if (unlikely (percpu_ref_is_dying (&ctx->refs)))WRITE_ONCE (poll->canceled, true);if (!req->result && !READ_ONCE (poll->canceled)) {
Jens Axboe
maintainerQuoted 11 lines · Hao Xu
Quoted 10 lines · Jens Axboe
We have two checks of
task->flags&PF_EXITINGleft:1) In
io_req_task_submit(), which is called intask_workand hence always in the context of the original task. That means thatreq->task== current, and hence checking ->flags is totally fine.2) In
io_poll_rewait(), where we need to stop re-arming poll to prevent it interfering with cancelation. Here,req->taskis not necessarily current, and hence the check is racy. Use the ctx refs state instead to check if we need to cancel this request or not.
Quoted 5 lines
Hi Jens,
I saw cases thatio_req_task_submit()andio_poll_rewait()in one function, why one is safe and the other one not? btw, it seems both two executes intask_workcontext..andtask_work_add()may fail and then work goes tosystem_wq, is that case safe?
io_req_task_submit() is guaranteed to be run in the task that is req->task, io_poll_rewait() is not. The latter can get called from eg the poll waitqueue handling, which is not run from the task in question.
Signature · 1 line
Jens Axboe
Jens Axboe
maintainerQuoted 18 lines · Jens Axboe
Quoted 11 lines · Hao Xu
Quoted 10 lines · Jens Axboe
We have two checks of
task->flags&PF_EXITINGleft:1) In
io_req_task_submit(), which is called intask_workand hence always in the context of the original task. That means thatreq->task== current, and hence checking ->flags is totally fine.2) In
io_poll_rewait(), where we need to stop re-arming poll to prevent it interfering with cancelation. Here,req->taskis not necessarily current, and hence the check is racy. Use the ctx refs state instead to check if we need to cancel this request or not.Quoted 5 lines
Hi Jens,
I saw cases thatio_req_task_submit()andio_poll_rewait()in one function, why one is safe and the other one not? btw, it seems both two executes intask_workcontext..andtask_work_add()may fail and then work goes tosystem_wq, is that case safe?
Quoted 3 lines
io_req_task_submit()is guaranteed to be run in the task that isreq->task,io_poll_rewait()is not. The latter can get called from eg the poll waitqueue handling, which is not run from the task in question.
Pavel nudged me, and in the 5.15 branch we actually only do run rewait from the task itself. So this patch isn't needed, we can ignore it! Might just augment it with a comment, like it was done for submit.
Signature · 1 line
Jens Axboe
Quoted 19 lines · Jens Axboe
Quoted 18 lines · Jens Axboe
Quoted 16 lines · Hao Xu
Quoted 10 lines · Jens Axboe
We have two checks of
task->flags&PF_EXITINGleft:1) In
io_req_task_submit(), which is called intask_workand hence always in the context of the original task. That means thatreq->task== current, and hence checking ->flags is totally fine.2) In
io_poll_rewait(), where we need to stop re-arming poll to prevent it interfering with cancelation. Here,req->taskis not necessarily current, and hence the check is racy. Use the ctx refs state instead to check if we need to cancel this request or not.Hi Jens,
I saw cases thatio_req_task_submit()andio_poll_rewait()in one function, why one is safe and the other one not? btw, it seems both two executes intask_workcontext..andtask_work_add()may fail and then work goes tosystem_wq, is that case safe?
Quoted 7 lines
Quoted 3 lines
io_req_task_submit()is guaranteed to be run in the task that isreq->task,io_poll_rewait()is not. The latter can get called from eg the poll waitqueue handling, which is not run from the task in question.Pavel nudged me, and in the 5.15 branch we actually only do run rewait from the task itself. So this patch isn't needed, we can ignore it! Might just augment it with a comment, like it was done for submit.
saw this after my second email. cool!
Quoted 1 line
Quoted 12 lines · Jens Axboe
Quoted 11 lines · Hao Xu
Quoted 10 lines · Jens Axboe
We have two checks of
task->flags&PF_EXITINGleft:1) In
io_req_task_submit(), which is called intask_workand hence always in the context of the original task. That means thatreq->task== current, and hence checking ->flags is totally fine.2) In
io_poll_rewait(), where we need to stop re-arming poll to prevent it interfering with cancelation. Here,req->taskis not necessarily current, and hence the check is racy. Use the ctx refs state instead to check if we need to cancel this request or not.
Quoted 5 lines
Quoted 5 lines
Hi Jens,
I saw cases thatio_req_task_submit()andio_poll_rewait()in one function, why one is safe and the other one not? btw, it seems both two executes intask_workcontext..andtask_work_add()may fail and then work goes tosystem_wq, is that case safe?
I've got answer for the second question..
Quoted 4 lines
io_req_task_submit()is guaranteed to be run in the task that isreq->task,io_poll_rewait()is not. The latter can get called from eg the poll waitqueue handling, which is not run from the task in question.
I only found io_poll_rewait() call in io_async_task_func() and io_poll_task_func(), both are in task_work
Quoted 1 line
Jens Axboe
maintainerQuoted 18 lines · Hao Xu
Quoted 12 lines · Jens Axboe
Quoted 11 lines · Hao Xu
Quoted 10 lines · Jens Axboe
We have two checks of
task->flags&PF_EXITINGleft:1) In
io_req_task_submit(), which is called intask_workand hence always in the context of the original task. That means thatreq->task== current, and hence checking ->flags is totally fine.2) In
io_poll_rewait(), where we need to stop re-arming poll to prevent it interfering with cancelation. Here,req->taskis not necessarily current, and hence the check is racy. Use the ctx refs state instead to check if we need to cancel this request or not.Quoted 5 lines
Quoted 5 lines
Hi Jens,
I saw cases thatio_req_task_submit()andio_poll_rewait()in one function, why one is safe and the other one not? btw, it seems both two executes intask_workcontext..andtask_work_add()may fail and then work goes tosystem_wq, is that case safe?
Quoted 7 lines
I've got answer for the second question..
Quoted 4 lines
io_req_task_submit()is guaranteed to be run in the task that isreq->task,io_poll_rewait()is not. The latter can get called from eg the poll waitqueue handling, which is not run from the task in question.I only found
io_poll_rewait()call inio_async_task_func()andio_poll_task_func(), both are intask_work
Yeah see followup, my information was outdated, we only do rewait from the right context at this point. Hence the PF_EXITING check is actually fine.
Signature · 1 line
Jens Axboe