Skip to contentSkip to conversation

io_uring: remove PF_EXITING checking in io_poll_rewait()

PATCH In review Jens Axboe 2 participants 7 messages Aug 19, 2021

Subsystems (2)
lore
Share & tools
lore

Jens Axboe

author maintainer
#0 lore raw

PATCH 1/1 io_uring: remove PF_EXITING checking in io_poll_rewait()

We have two checks of task->flags & PF_EXITING left:

1) In io_req_task_submit(), which is called in task_work and hence always in the context of the original task. That means that req->task == current, and hence checking ->flags is totally fine.

2) In io_poll_rewait(), where we need to stop re-arming poll to prevent it interfering with cancelation. Here, req->task is not necessarily current, and hence the check is racy. Use the ctx refs state instead to check if we need to cancel this request or not.

---

fs/io_uring.c +6 -1
index 30edc329d803..ffce959c2370 100644
@@ -2114,6 +2114,7 @@ static void io_req_task_submit(struct io_kiocb *req)
 
 	/* ctx stays valid until unlock, even if we drop all ours ctx->refs */
 	mutex_lock(&ctx->uring_lock);
+	/* req->task == current here, checking PF_EXITING is safe */
 	if (likely(!(req->task->flags & PF_EXITING)))
 		__io_queue_sqe(req);
 	else
@@ -4895,7 +4896,11 @@ static bool io_poll_rewait(struct io_kiocb *req, struct io_poll_iocb *poll)
 {
 	struct io_ring_ctx *ctx = req->ctx;
 
-	if (unlikely(req->task->flags & PF_EXITING))
+	/*
+	 * Pairs with spin_unlock() in percpu_ref_kill()
+	 */
+	smp_rmb();
+	if (unlikely(percpu_ref_is_dying(&ctx->refs)))
 		WRITE_ONCE(poll->canceled, true);
 
 	if (!req->result && !READ_ONCE(poll->canceled)) {
Signature · 1 line
Jens Axboe
#1 lore raw
Quoted 10 lines · Jens Axboe

We have two checks of task->flags & PF_EXITING left:

1) In io_req_task_submit(), which is called in task_work and hence always in the context of the original task. That means that req->task == current, and hence checking ->flags is totally fine.

2) In io_poll_rewait(), where we need to stop re-arming poll to prevent it interfering with cancelation. Here, req->task is not necessarily current, and hence the check is racy. Use the ctx refs state instead to check if we need to cancel this request or not.

Hi Jens,
I saw cases that io_req_task_submit() and io_poll_rewait() in one function, why one is safe and the other one not? btw, it seems both two executes in task_work context..and task_work_add() may fail and then work goes to system_wq, is that case safe?

Quoted 31 lines

---

fs/io_uring.c +6 -1
index 30edc329d803..ffce959c2370 100644
@@ -2114,6 +2114,7 @@ static void io_req_task_submit(struct io_kiocb *req)
 
  	/* ctx stays valid until unlock, even if we drop all ours ctx->refs */
  	mutex_lock(&ctx->uring_lock);
+	/* req->task == current here, checking PF_EXITING is safe */
  	if (likely(!(req->task->flags & PF_EXITING)))
  		__io_queue_sqe(req);
  	else
@@ -4895,7 +4896,11 @@ static bool io_poll_rewait(struct io_kiocb *req, struct io_poll_iocb *poll)
  {
  	struct io_ring_ctx *ctx = req->ctx;
 
-	if (unlikely(req->task->flags & PF_EXITING))
+	/*
+	 * Pairs with spin_unlock() in percpu_ref_kill()
+	 */
+	smp_rmb();
+	if (unlikely(percpu_ref_is_dying(&ctx->refs)))
  		WRITE_ONCE(poll->canceled, true);
 
  	if (!req->result && !READ_ONCE(poll->canceled)) {
author maintainer
#2 lore raw
Quoted 11 lines · Hao Xu
Quoted 10 lines · Jens Axboe

We have two checks of task->flags & PF_EXITING left:

1) In io_req_task_submit(), which is called in task_work and hence always in the context of the original task. That means that req->task == current, and hence checking ->flags is totally fine.

2) In io_poll_rewait(), where we need to stop re-arming poll to prevent it interfering with cancelation. Here, req->task is not necessarily current, and hence the check is racy. Use the ctx refs state instead to check if we need to cancel this request or not.

Quoted 5 lines

Hi Jens,
I saw cases that io_req_task_submit() and io_poll_rewait() in one function, why one is safe and the other one not? btw, it seems both two executes in task_work context..and task_work_add() may fail and then work goes to system_wq, is that case safe?

io_req_task_submit() is guaranteed to be run in the task that is req->task, io_poll_rewait() is not. The latter can get called from eg the poll waitqueue handling, which is not run from the task in question.

Signature · 1 line
Jens Axboe
author maintainer
#3 lore raw
Quoted 18 lines · Jens Axboe
Quoted 11 lines · Hao Xu
Quoted 10 lines · Jens Axboe

We have two checks of task->flags & PF_EXITING left:

1) In io_req_task_submit(), which is called in task_work and hence always in the context of the original task. That means that req->task == current, and hence checking ->flags is totally fine.

2) In io_poll_rewait(), where we need to stop re-arming poll to prevent it interfering with cancelation. Here, req->task is not necessarily current, and hence the check is racy. Use the ctx refs state instead to check if we need to cancel this request or not.

Quoted 5 lines

Hi Jens,
I saw cases that io_req_task_submit() and io_poll_rewait() in one function, why one is safe and the other one not? btw, it seems both two executes in task_work context..and task_work_add() may fail and then work goes to system_wq, is that case safe?

Quoted 3 lines

io_req_task_submit() is guaranteed to be run in the task that is req->task, io_poll_rewait() is not. The latter can get called from eg the poll waitqueue handling, which is not run from the task in question.

Pavel nudged me, and in the 5.15 branch we actually only do run rewait from the task itself. So this patch isn't needed, we can ignore it! Might just augment it with a comment, like it was done for submit.

Signature · 1 line
Jens Axboe
#4 lore raw
Quoted 19 lines · Jens Axboe
Quoted 18 lines · Jens Axboe
Quoted 16 lines · Hao Xu
Quoted 10 lines · Jens Axboe

We have two checks of task->flags & PF_EXITING left:

1) In io_req_task_submit(), which is called in task_work and hence always in the context of the original task. That means that req->task == current, and hence checking ->flags is totally fine.

2) In io_poll_rewait(), where we need to stop re-arming poll to prevent it interfering with cancelation. Here, req->task is not necessarily current, and hence the check is racy. Use the ctx refs state instead to check if we need to cancel this request or not.

Hi Jens,
I saw cases that io_req_task_submit() and io_poll_rewait() in one function, why one is safe and the other one not? btw, it seems both two executes in task_work context..and task_work_add() may fail and then work goes to system_wq, is that case safe?

Quoted 7 lines
Quoted 3 lines

io_req_task_submit() is guaranteed to be run in the task that is req->task, io_poll_rewait() is not. The latter can get called from eg the poll waitqueue handling, which is not run from the task in question.

Pavel nudged me, and in the 5.15 branch we actually only do run rewait from the task itself. So this patch isn't needed, we can ignore it! Might just augment it with a comment, like it was done for submit.

saw this after my second email. cool!

Quoted 1 line
#5 lore raw
In reply to Jens Axboe
Quoted 12 lines · Jens Axboe
Quoted 11 lines · Hao Xu
Quoted 10 lines · Jens Axboe

We have two checks of task->flags & PF_EXITING left:

1) In io_req_task_submit(), which is called in task_work and hence always in the context of the original task. That means that req->task == current, and hence checking ->flags is totally fine.

2) In io_poll_rewait(), where we need to stop re-arming poll to prevent it interfering with cancelation. Here, req->task is not necessarily current, and hence the check is racy. Use the ctx refs state instead to check if we need to cancel this request or not.

Quoted 5 lines
Quoted 5 lines

Hi Jens,
I saw cases that io_req_task_submit() and io_poll_rewait() in one function, why one is safe and the other one not? btw, it seems both two executes in task_work context..and task_work_add() may fail and then work goes to system_wq, is that case safe?

I've got answer for the second question..

Quoted 4 lines

io_req_task_submit() is guaranteed to be run in the task that is req->task, io_poll_rewait() is not. The latter can get called from eg the poll waitqueue handling, which is not run from the task in question.

I only found io_poll_rewait() call in io_async_task_func() and io_poll_task_func(), both are in task_work

Quoted 1 line
author maintainer
#6 lore raw
Quoted 18 lines · Hao Xu
Quoted 12 lines · Jens Axboe
Quoted 11 lines · Hao Xu
Quoted 10 lines · Jens Axboe

We have two checks of task->flags & PF_EXITING left:

1) In io_req_task_submit(), which is called in task_work and hence always in the context of the original task. That means that req->task == current, and hence checking ->flags is totally fine.

2) In io_poll_rewait(), where we need to stop re-arming poll to prevent it interfering with cancelation. Here, req->task is not necessarily current, and hence the check is racy. Use the ctx refs state instead to check if we need to cancel this request or not.

Quoted 5 lines
Quoted 5 lines

Hi Jens,
I saw cases that io_req_task_submit() and io_poll_rewait() in one function, why one is safe and the other one not? btw, it seems both two executes in task_work context..and task_work_add() may fail and then work goes to system_wq, is that case safe?

Quoted 7 lines

I've got answer for the second question..

Quoted 4 lines

io_req_task_submit() is guaranteed to be run in the task that is req->task, io_poll_rewait() is not. The latter can get called from eg the poll waitqueue handling, which is not run from the task in question.

I only found io_poll_rewait() call in io_async_task_func() and io_poll_task_func(), both are in task_work

Yeah see followup, my information was outdated, we only do rewait from the right context at this point. Hence the PF_EXITING check is actually fine.

Signature · 1 line
Jens Axboe