ring buffer fixes
PATCH 2/2 io_uring: recycle kbuf recycle on tw requeue
In reply to Pavel BegunkovWhen we queue a request via tw for execution it's not going to be executed immediately, so when io_queue_async() hits IO_APOLL_READY and queues a tw but doesn't try to recycle/consume the buffer some other request may try to use the the buffer.
--- io_uring/io_uring.c | 1 + 1 file changed, 1 insertion(+)
io_uring/io_uring.c +1 -0
@@ -1728,6 +1728,7 @@ static void io_queue_async(struct io_kiocb *req, int ret)switch (io_arm_poll_handler (req,0 )) {case IO_APOLL_READY : +io_kbuf_recycle (req,0 );io_req_task_queue (req);break ;case IO_APOLL_ABORTED :
Signature · 1 line
2.37.2
cover ring buffer fixes
Pavel Begunkov (2):
io_uring/kbuf: fix not advancing READV kbuf ring
io_uring: recycle kbuf recycle on tw requeue
io_uring/io_uring.c | 1 + io_uring/kbuf.h | 8 ++++++-- 2 files changed, 7 insertions(+), 2 deletions(-)
Signature · 1 line
2.37.2
PATCH 1/2 io_uring/kbuf: fix not advancing READV kbuf ring
When we don't recycle a selected ring buffer we should advance the head of the ring, so don't just skip io_kbuf_recycle() for IORING_OP_READV but adjust the ring.
--- io_uring/kbuf.h | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
io_uring/kbuf.h +6 -2
@@ -91,9 +91,13 @@ static inline void io_kbuf_recycle(struct io_kiocb *req, unsigned issue_flags)* buffer data. However if that buffer is recycled the original request * data stored in addr is lost. Therefore forbid recycling for now. */ -if (req->opcode ==IORING_OP_READV ) +if (req->opcode ==IORING_OP_READV ) { +if ((req->flags &REQ_F_BUFFER_RING ) && req->buf_list) { + req->buf_list->head++; + req->buf_list =NULL ; + }return ; - + }if (req->flags &REQ_F_BUFFER_SELECTED )io_kbuf_recycle_legacy (req, issue_flags);if (req->flags &REQ_F_BUFFER_RING )
Signature · 1 line
2.37.2
Re: [PATCH 1/2] io_uring/kbuf: fix not advancing READV kbuf ring
Quoted 26 lines · Pavel Begunkov
When we don't recycle a selected ring buffer we should advance the head
of the ring, so don't just skipio_kbuf_recycle()forIORING_OP_READVbut adjust the ring.Fixes 934447a603b22 ("io_uring: do not recycle buffer in READV")Signed-off-by Pavel Begunkov <[email protected]>---
io_uring/kbuf.h | 8 ++++++--1 file changed, 6 insertions(+), 2 deletions(-)io_uring/kbuf.h +0
-0@@ -91,9 +91,13 @@ static inline void io_kbuf_recycle(struct io_kiocb *req, unsigned issue_flags) * buffer data. However if that buffer is recycled theoriginal request
* data stored in addr is lost. Therefore forbid recycling for now.
*/
- if (req->opcode==IORING_OP_READV)
+ if (req->opcode==IORING_OP_READV) {
+ if ((req->flags&REQ_F_BUFFER_RING) && req-Quoted 1 line
buf_list) {
Quoted 9 lines
+ req->buf_list->head++;
+ req->buf_list = NULL;
+ }
return;
-
+ }
if (req->flags&REQ_F_BUFFER_SELECTED)io_kbuf_recycle_legacy(req, issue_flags); if (req->flags&REQ_F_BUFFER_RING)
I do not know a good way to test this, but it reads to me likeit is correct. It should probably be applied before the patch I sent earlier today to remove this code so that backporting is easier.
Quoted 9 lines · Pavel Begunkov
Pavel Begunkov (2):
io_uring/kbuf: fix not advancing READV kbuf ring
io_uring: recycle kbuf recycle on tw requeue
io_uring/io_uring.c| 1 +io_uring/kbuf.h | 8 ++++++-- 2 files changed, 7 insertions(+), 2 deletions(-)[...]
Applied, thanks!
[1/2] io_uring/kbuf: fix not advancing READV kbuf ring
commit: df6d3422d3eed27afa23df092b3ce147c558d1a8
[2/2] io_uring: recycle kbuf recycle on tw requeue
commit: 336d28a8f38013a069f2d46e73aaa1880ef17a47Best regards,
Signature · 1 line
Jens Axboe