Skip to contentSkip to conversation

io_uring/uring_cmd: skip inline completion cleanup if unlocked

PATCH In review Jens Axboe 2 participants 3 messages May 6 io_uring

lore
Share & tools
lore

Jens Axboe

author maintainer
#0 lore raw

PATCH 1/1 io_uring/uring_cmd: skip inline completion cleanup if unlocked

If the call path to __io_uring_cmd_done() is not locked, then we cannot recycle the uring_cmd to our allocation cache. Check for that and skip it, and let the normal locked completion flushing do the cleanup.

---

This effectively defeats proper cache recyling for uring_cmd opcodes, with the fix it's working fine again.

io_uring/uring_cmd.c +4 -1
index 42be1be5b132..35e2aa8b9446 100644
@@ -166,7 +166,9 @@ void __io_uring_cmd_done(struct io_uring_cmd *ioucmd, s32 ret, u64 res2,
 			req->cqe.flags |= IORING_CQE_F_32;
 		io_req_set_cqe32_extra(req, res2, 0);
 	}
-	io_req_uring_cleanup(req, issue_flags);
+	/* defer cleanup if not locked, otherwise cache recyling is skipped */
+	if (!(issue_flags & IO_URING_F_UNLOCKED))
+		io_req_uring_cleanup(req, issue_flags);
 	if (req->flags & REQ_F_IOPOLL) {
 		/* order with io_iopoll_req_issued() checking ->iopoll_complete */
 		smp_store_release(&req->iopoll_completed, 1);
@@ -211,6 +213,7 @@ int io_uring_cmd_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe)
 	ac = io_uring_alloc_async_data(&req->ctx->cmd_cache, req);
 	if (!ac)
 		return -ENOMEM;
+	req->flags |= REQ_F_NEED_CLEANUP;
 	ioucmd->sqe = sqe;
 	return 0;
 }
Signature · 1 line
Jens Axboe
#1 lore raw
Quoted 18 lines · Jens Axboe

If the call path to __io_uring_cmd_done() is not locked, then we cannot recycle the uring_cmd to our allocation cache. Check for that and skip it, and let the normal locked completion flushing do the cleanup.

---

This effectively defeats proper cache recyling for uring_cmd opcodes, with the fix it's working fine again.

io_uring/uring_cmd.c +0 -0
index 42be1be5b132..35e2aa8b9446 100644

io_uring/uring_cmd.c void __io_uring_cmd_done(struct io_uring_cmd *ioucmd, s32 ret, u64 res2, · line 169

Quoted 5 lines
io_uring/uring_cmd.c +1 -1
@@ -166,7 +166,9 @@ void __io_uring_cmd_done(struct io_uring_cmd *ioucmd, s32 ret, u64 res2,
                        req->cqe.flags |= IORING_CQE_F_32;
                io_req_set_cqe32_extra(req, res2, 0);
        }
-       io_req_uring_cleanup(req, issue_flags);
+       /* defer cleanup if not locked, otherwise cache recyling is skipped */

"recycling"?

io_uring/uring_cmd.c void __io_uring_cmd_done(struct io_uring_cmd *ioucmd, s32 ret, u64 res2, · line 171

Quoted 2 lines
io_uring/uring_cmd.c +2 -0
+       if (!(issue_flags & IO_URING_F_UNLOCKED))
+               io_req_uring_cleanup(req, issue_flags);

Doesn't io_req_uring_cleanup() already check this?

Best,
Caleb

Quoted 16 lines
io_uring/uring_cmd.c +1 -1
        if (req->flags & REQ_F_IOPOLL) {
                /* order with io_iopoll_req_issued() checking ->iopoll_complete */
                smp_store_release(&req->iopoll_completed, 1);
@@ -211,6 +213,7 @@ int io_uring_cmd_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe)
        ac = io_uring_alloc_async_data(&req->ctx->cmd_cache, req);
        if (!ac)
                return -ENOMEM;
+       req->flags |= REQ_F_NEED_CLEANUP;
        ioucmd->sqe = sqe;
        return 0;
 }
 
--

Jens Axboe

author maintainer
#2 lore raw
Quoted 27 lines · Caleb Sander Mateos
Quoted 18 lines · On Wed, May 6, 2026 at 4:04?AM Jens Axboe <[email protected]> wrote:

If the call path to __io_uring_cmd_done() is not locked, then we cannot recycle the uring_cmd to our allocation cache. Check for that and skip it, and let the normal locked completion flushing do the cleanup.

---

This effectively defeats proper cache recyling for uring_cmd opcodes, with the fix it's working fine again.

io_uring/uring_cmd.c +0 -0
index 42be1be5b132..35e2aa8b9446 100644

io_uring/uring_cmd.c void __io_uring_cmd_done(struct io_uring_cmd *ioucmd, s32 ret, u64 res2, · line 169

Quoted 5 lines
io_uring/uring_cmd.c +1 -1
@@ -166,7 +166,9 @@ void __io_uring_cmd_done(struct io_uring_cmd *ioucmd, s32 ret, u64 res2,
                        req->cqe.flags |= IORING_CQE_F_32;
                io_req_set_cqe32_extra(req, res2, 0);
        }
-       io_req_uring_cleanup(req, issue_flags);
+       /* defer cleanup if not locked, otherwise cache recyling is skipped */

"recycling"?

Quoted 4 lines

io_uring/uring_cmd.c void __io_uring_cmd_done(struct io_uring_cmd *ioucmd, s32 ret, u64 res2, · line 171

Quoted 2 lines
io_uring/uring_cmd.c +2 -0
+       if (!(issue_flags & IO_URING_F_UNLOCKED))
+               io_req_uring_cleanup(req, issue_flags);

Doesn't io_req_uring_cleanup() already check this?

True it does, I think all we need is:

Quoted 8 lines

io_uring/uring_cmd.c int io_uring_cmd_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe) · line 219

Quoted 8 lines
io_uring/uring_cmd.c +1 -0
@@ -211,6 +213,7 @@ int io_uring_cmd_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe)
        ac = io_uring_alloc_async_data(&req->ctx->cmd_cache, req);
        if (!ac)
                return -ENOMEM;
+       req->flags |= REQ_F_NEED_CLEANUP;
        ioucmd->sqe = sqe;
        return 0;
 }

to ensure it's called later too. I'll update it, thanks!

Signature · 1 line
Jens Axboe