io_uring: don't touch sqd->thread off tw add
PATCH 1/1 io_uring: don't touch sqd->thread off tw add
With IORING_SETUP_SQPOLL all requests are created by the SQPOLL task, which means that req->task should always match sqd->thread. Since accesses to sqd->thread should be separately protected, use req->task in io_req_normal_work_add() instead.
Note, in the eyes of io_req_normal_work_add(), the SQPOLL task struct is always pinned and alive, and sqd->thread can either be the task or NULL. It's only problematic if the compiler decides to reload the value after the null check, which is not so likely.
--- io_uring/io_uring.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-)
io_uring/io_uring.c +1 -4
@@ -1225,10 +1225,7 @@ static void io_req_normal_work_add(struct io_kiocb *req)/* SQPOLL doesn't need the task_work added, it'll run it itself */ if (ctx->flags &IORING_SETUP_SQPOLL ) { -struct io_sq_data *sqd = ctx->sq_data; - -if (sqd->thread) -__set_notify_signal (sqd->thread); +__set_notify_signal (tctx->task);return ; }
Signature · 1 line
2.47.1
Quoted 9 lines · Pavel Begunkov
With
IORING_SETUP_SQPOLLall requests are created by the SQPOLL task, which means thatreq->taskshould always matchsqd->thread. Since accesses tosqd->threadshould be separately protected, usereq->taskinio_req_normal_work_add()instead.Note, in the eyes of
io_req_normal_work_add(), the SQPOLL task struct is always pinned and alive, andsqd->threadcan either be the task or NULL. It's only problematic if the compiler decides to reload the value after the null check, which is not so likely.
We don't have much time to drag it on, let's fix it up so it hopefully gets into 6.13
Quoted 25 lines
Cc Bui Quang Minh <[email protected]>Reported-by lizetao <[email protected]>Fixes 78f9b61bd8e54 ("io_uring: wake SQPOLL task when task_work is added to an empty queue")Signed-off-by Pavel Begunkov <[email protected]>--- io_uring/io_uring.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-)io_uring/io_uring.c +1
-4@@ -1225,10 +1225,7 @@ static void io_req_normal_work_add(struct io_kiocb *req)/* SQPOLL doesn't need the task_work added, it'll run it itself */ if (ctx->flags &IORING_SETUP_SQPOLL ) { -struct io_sq_data *sqd = ctx->sq_data; - -if (sqd->thread) -__set_notify_signal (sqd->thread); +__set_notify_signal (tctx->task);return ; }
Signature · 1 line
Pavel Begunkov
Quoted 11 lines · Pavel Begunkov
With
IORING_SETUP_SQPOLLall requests are created by the SQPOLL task, which means thatreq->taskshould always matchsqd->thread. Since accesses tosqd->threadshould be separately protected, usereq->taskinio_req_normal_work_add()instead.Note, in the eyes of
io_req_normal_work_add(), the SQPOLL task struct is always pinned and alive, andsqd->threadcan either be the task or NULL. It's only problematic if the compiler decides to reload the value after the null check, which is not so likely.[...]
Applied, thanks!
[1/1] io_uring: don't touch sqd->thread off tw add
commit: bd2703b42decebdcddf76e277ba76b4c4a142d73
Best regards,
Signature · 1 line
Jens Axboe