io_uring/uring_cmd: skip inline completion cleanup if unlocked
Jens Axboe
maintainerPATCH 1/1 io_uring/uring_cmd: skip inline completion cleanup if unlocked
If the call path to __io_uring_cmd_done() is not locked, then we cannot recycle the uring_cmd to our allocation cache. Check for that and skip it, and let the normal locked completion flushing do the cleanup.
---
This effectively defeats proper cache recyling for uring_cmd opcodes, with the fix it's working fine again.
io_uring/uring_cmd.c +4 -1
@@ -166,7 +166,9 @@ void __io_uring_cmd_done(struct io_uring_cmd *ioucmd, s32 ret, u64 res2, req->cqe.flags |=IORING_CQE_F_32 ;io_req_set_cqe32_extra (req, res2,0 ); } -io_req_uring_cleanup (req, issue_flags); +/* defer cleanup if not locked, otherwise cache recyling is skipped */ +if (!(issue_flags &IO_URING_F_UNLOCKED )) +io_req_uring_cleanup (req, issue_flags);if (req->flags &REQ_F_IOPOLL ) {/* order with io_iopoll_req_issued() checking ->iopoll_complete */ smp_store_release (&req->iopoll_completed,1 );
@@ -211,6 +213,7 @@ int io_uring_cmd_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe) ac =io_uring_alloc_async_data (&req->ctx->cmd_cache, req);if (!ac)return -ENOMEM ; + req->flags |=REQ_F_NEED_CLEANUP ; ioucmd->sqe = sqe;return 0 ; }
Signature · 1 line
Jens Axboe
Quoted 18 lines · Jens Axboe
If the call path to
__io_uring_cmd_done()is not locked, then we cannot recycle theuring_cmdto our allocation cache. Check for that and skip it, and let the normal locked completion flushing do the cleanup.Signed-off-by Jens Axboe <[email protected]>---
This effectively defeats proper cache recyling for
uring_cmdopcodes, with the fix it's working fine again.io_uring/uring_cmd.c +0
-0
io_uring/uring_cmd.c void __io_uring_cmd_done(struct io_uring_cmd *ioucmd, s32 ret, u64 res2, · line 169
Quoted 5 lines
io_uring/uring_cmd.c +1
-1@@ -166,7 +166,9 @@ void __io_uring_cmd_done(struct io_uring_cmd *ioucmd, s32 ret, u64 res2, req->cqe.flags |=IORING_CQE_F_32 ;io_req_set_cqe32_extra (req, res2,0 ); } -io_req_uring_cleanup (req, issue_flags); +/* defer cleanup if not locked, otherwise cache recyling is skipped */
"recycling"?
io_uring/uring_cmd.c void __io_uring_cmd_done(struct io_uring_cmd *ioucmd, s32 ret, u64 res2, · line 171
Quoted 2 lines
io_uring/uring_cmd.c +2
-0+if (!(issue_flags &IO_URING_F_UNLOCKED )) +io_req_uring_cleanup (req, issue_flags);
Doesn't io_req_uring_cleanup() already check this?
Best,
Caleb
Quoted 16 lines
io_uring/uring_cmd.c +1
-1if (req->flags &REQ_F_IOPOLL ) {/* order with io_iopoll_req_issued() checking ->iopoll_complete */ smp_store_release (&req->iopoll_completed,1 );@@ -211,6 +213,7 @@ int io_uring_cmd_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe) ac =io_uring_alloc_async_data (&req->ctx->cmd_cache, req);if (!ac)return -ENOMEM ; + req->flags |=REQ_F_NEED_CLEANUP ; ioucmd->sqe = sqe;return 0 ; } --Jens Axboe
Jens Axboe
maintainerQuoted 27 lines · Caleb Sander Mateos
Quoted 18 lines · On Wed, May 6, 2026 at 4:04?AM Jens Axboe <[email protected]> wrote:
If the call path to
__io_uring_cmd_done()is not locked, then we cannot recycle theuring_cmdto our allocation cache. Check for that and skip it, and let the normal locked completion flushing do the cleanup.Signed-off-by Jens Axboe <[email protected]>---
This effectively defeats proper cache recyling for
uring_cmdopcodes, with the fix it's working fine again.io_uring/uring_cmd.c +0
-0io_uring/uring_cmd.c void __io_uring_cmd_done(struct io_uring_cmd *ioucmd, s32 ret, u64 res2, · line 169
Quoted 5 lines
io_uring/uring_cmd.c +1
-1@@ -166,7 +166,9 @@ void __io_uring_cmd_done(struct io_uring_cmd *ioucmd, s32 ret, u64 res2, req->cqe.flags |=IORING_CQE_F_32 ;io_req_set_cqe32_extra (req, res2,0 ); } -io_req_uring_cleanup (req, issue_flags); +/* defer cleanup if not locked, otherwise cache recyling is skipped */ "recycling"?
Quoted 4 lines
io_uring/uring_cmd.c void __io_uring_cmd_done(struct io_uring_cmd *ioucmd, s32 ret, u64 res2, · line 171
Quoted 2 lines
io_uring/uring_cmd.c +2
-0+if (!(issue_flags &IO_URING_F_UNLOCKED )) +io_req_uring_cleanup (req, issue_flags);Doesn't
io_req_uring_cleanup()already check this?
True it does, I think all we need is:
Quoted 8 lines
io_uring/uring_cmd.c int io_uring_cmd_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe) · line 219
Quoted 8 lines
io_uring/uring_cmd.c +1
-0@@ -211,6 +213,7 @@ int io_uring_cmd_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe) ac =io_uring_alloc_async_data (&req->ctx->cmd_cache, req);if (!ac)return -ENOMEM ; + req->flags |=REQ_F_NEED_CLEANUP ; ioucmd->sqe = sqe;return 0 ; }
to ensure it's called later too. I'll update it, thanks!
Signature · 1 line
Jens Axboe