Add support for mixed sized CQEs
Subsystems (5)
Jens Axboe
maintainerv2 cover Add support for mixed sized CQEs
Hi,
Currently io_uring supports two modes for CQEs:
1) The standard mode, where 16b CQEs are used
2) Setting IORING_SETUP_CQE32, which makes all CQEs posted 32b
Certain features need to pass more information back than just a single 32-bit res field, and hence mandate the use of CQE32 to be able to work. Examples of that include passthrough or other uses of ->uring_cmd() like socket option getting and setting, including timestamps.
This patchset adds support for IORING_SETUP_CQE_MIXED, which allows posting both 16b and 32b CQEs on the same CQ ring. The idea here is that we need not waste twice the space for CQ rings, or use twice the space per CQE posted, if only some of the CQEs posted require the use of 32b CQEs. On a ring setup in CQE mixed mode, 32b posted CQEs will have IORING_CQE_F_32 set in cqe->flags to tell the application (or liburing) about this fact.
This is mostly trivial to support, with the corner case being attempting to post a 32b CQE when the ring is a single 16b CQE away from wrapping. As CQEs must be contigious in memory, that's simply not possible. The solution taken by this patchset is to add a special CQE type, which has IORING_CQE_F_SKIP set. This is a pad/nop CQE, which should simply be ignored, as it carries no information and serves no other purpose than to re-align the posted CQEs for ring wrap.
If used with liburing, then both the 32b vs 16b postings and the skip are transparent.
liburing support and a few basic test cases can be found here:
https://git.kernel.org/pub/scm/linux/kernel/git/axboe/liburing.git/log/?h=cqe-mixed
including man page updates for the newly added setup and CQE flags, and the patches posted here can also be found at:
https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git/log/?h=io_uring-cqe-mix
Patch 1 is just a prep patch, and patch 2 adds the cqe flags so that the core can be adapted before support is actually there. Patches 3 and 4 are exactly that, and patch 5 finally adds support for the mixed mode. Patch 6 adds support for NOP testing of this, and patches 7/8 allow IORING_SETUP_CQE_MIXED for uring_cmd/zcrx which previously required IORING_SETUP_CQE32 to work.
Documentation/networking/iou-zcrx.rst | 2 +- include/linux/io_uring_types.h | 6 --- include/trace/events/io_uring.h | 4 +- include/uapi/linux/io_uring.h | 17 ++++++ io_uring/cmd_net.c | 3 +- io_uring/fdinfo.c | 22 ++++---- io_uring/io_uring.c | 78 +++++++++++++++++++++------ io_uring/io_uring.h | 49 ++++++++++++----- io_uring/nop.c | 17 +++++- io_uring/register.c | 3 +- io_uring/uring_cmd.c | 2 +- io_uring/zcrx.c | 5 +- 12 files changed, 152 insertions(+), 56 deletions(-)
Changes since v1:
- Various little cleanups
- Rebase on for-6.18/io_uring
Signature · 1 line
Jens Axboe
Jens Axboe
maintainerPATCH v2 1/8 io_uring: remove io_ctx_cqe32() helper
It's pretty pointless and only used for the tracing helper, get rid of it.
--- include/linux/io_uring_types.h | 6 ------ include/trace/events/io_uring.h | 4 ++-- 2 files changed, 2 insertions(+), 8 deletions(-)
include/linux/io_uring_types.h +0 -6
@@ -740,10 +740,4 @@ struct io_overflow_cqe {struct list_head list; struct io_uring_cqe cqe; }; - -static inline bool io_ctx_cqe32 (struct io_ring_ctx *ctx) -{ -return ctx->flags &IORING_SETUP_CQE32 ; -} -#endif
include/trace/events/io_uring.h +2 -2
@@ -340,8 +340,8 @@ TP_PROTO(struct io_ring_ctx *ctx, void *req, struct io_uring_cqe *cqe), __entry->user_data = cqe->user_data; __entry->res = cqe->res; __entry->cflags = cqe->flags; - __entry->extra1 =io_ctx_cqe32 (ctx) ? cqe->big_cqe[0 ] :0 ; - __entry->extra2 =io_ctx_cqe32 (ctx) ? cqe->big_cqe[1 ] :0 ; + __entry->extra1 = ctx->flags &IORING_SETUP_CQE32 ? cqe->big_cqe[0 ] :0 ; + __entry->extra2 = ctx->flags &IORING_SETUP_CQE32 ? cqe->big_cqe[1 ] :0 ; ), TP_printk("ring %p, req %p, user_data 0x%llx, result %d, cflags 0x%x "
Signature · 1 line
2.50.1
Jens Axboe
maintainerPATCH v2 2/8 io_uring: add UAPI definitions for mixed CQE postings
In reply to Jens AxboeThis adds the CQE flags related to supporting a mixed CQ ring mode, where both normal (16b) and big (32b) CQEs may be posted.
No functional changes in this patch.
--- include/uapi/linux/io_uring.h | 10 ++++++++++ 1 file changed, 10 insertions(+)
include/uapi/linux/io_uring.h +10 -0
@@ -491,12 +491,22 @@ struct io_uring_cqe {* other provided buffer type, all completions with a * buffer passed back is automatically returned to the * application. +* IORING_CQE_F_SKIP If set, then the application/liburing must ignore this +* CQE. It's only purpose is to fill a gap in the ring, +* if a large CQE is attempted posted when the ring has +* just a single small CQE worth of space left before +* wrapping. +* IORING_CQE_F_32 If set, this is a 32b/big-cqe posting. Use with rings +* setup in a mixed CQE mode, where both 16b and 32b +* CQEs may be posted to the CQ ring. */ #define IORING_CQE_F_BUFFER (1U << 0)#define IORING_CQE_F_MORE (1U << 1)#define IORING_CQE_F_SOCK_NONEMPTY (1U << 2)#define IORING_CQE_F_NOTIF (1U << 3)#define IORING_CQE_F_BUF_MORE (1U << 4) +#define IORING_CQE_F_SKIP (1U << 5) +#define IORING_CQE_F_32 (1U << 15)#define IORING_CQE_BUFFER_SHIFT 16
Signature · 1 line
2.50.1
Jens Axboe
maintainerPATCH v2 3/8 io_uring/fdinfo: handle mixed sized CQEs
In reply to Jens AxboeEnsure that the CQ ring iteration handles differently sized CQEs, not just a fixed 16b or 32b size per ring. These CQEs aren't possible just yet, but prepare the fdinfo CQ ring dumping for handling them.
--- io_uring/fdinfo.c | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-)
io_uring/fdinfo.c +12 -10
@@ -65,15 +65,12 @@ static void __io_uring_show_fdinfo(struct io_ring_ctx *ctx, struct seq_file *m)unsigned int sq_tail =READ_ONCE (r->sq.tail);unsigned int cq_head =READ_ONCE (r->cq.head);unsigned int cq_tail =READ_ONCE (r->cq.tail); -unsigned int cq_shift =0 ;unsigned int sq_shift =0 ; -unsigned int sq_entries, cq_entries; +unsigned int sq_entries;int sq_pid =-1 , sq_cpu =-1 ;u64 sq_total_time =0 , sq_work_time =0 ;unsigned int i; -if (ctx->flags &IORING_SETUP_CQE32 ) - cq_shift =1 ;if (ctx->flags &IORING_SETUP_SQE128 ) sq_shift =1 ;
@@ -125,18 +122,23 @@ static void __io_uring_show_fdinfo(struct io_ring_ctx *ctx, struct seq_file *m)seq_printf (m,"\n" ); }seq_printf (m,"CQEs:\t%u\n" , cq_tail - cq_head); - cq_entries =min (cq_tail - cq_head, ctx->cq_entries); -for (i =0 ; i < cq_entries; i++) { -unsigned int entry = i + cq_head; -struct io_uring_cqe *cqe = &r->cqes[(entry & cq_mask) << cq_shift]; +while (cq_head < cq_tail) { +struct io_uring_cqe *cqe; +bool cqe32 = false; + cqe = &r->cqes[(cq_head & cq_mask)]; +if (cqe->flags &IORING_CQE_F_32 || ctx->flags &IORING_SETUP_CQE32 ) + cqe32 = true;seq_printf (m,"%5u: user_data:%llu, res:%d, flag:%x" , - entry & cq_mask, cqe->user_data, cqe->res, + cq_head & cq_mask, cqe->user_data, cqe->res, cqe->flags); -if (cq_shift) +if (cqe32)seq_printf (m,", extra1:%llu, extra2:%llu\n" , cqe->big_cqe[0 ], cqe->big_cqe[1 ]);seq_printf (m,"\n" ); + cq_head++; +if (cqe32) + cq_head++; }if (ctx->flags &IORING_SETUP_SQPOLL ) {
Signature · 1 line
2.50.1
Jens Axboe
maintainerPATCH v2 4/8 io_uring/trace: support completion tracing of mixed 32b CQEs
In reply to Jens AxboeCheck for IORING_CQE_F_32 as well, not just if the ring was setup with IORING_SETUP_CQE32 to only support big CQEs.
--- include/trace/events/io_uring.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
include/trace/events/io_uring.h +2 -2
@@ -340,8 +340,8 @@ TP_PROTO(struct io_ring_ctx *ctx, void *req, struct io_uring_cqe *cqe), __entry->user_data = cqe->user_data; __entry->res = cqe->res; __entry->cflags = cqe->flags; - __entry->extra1 = ctx->flags &IORING_SETUP_CQE32 ? cqe->big_cqe[0 ] :0 ; - __entry->extra2 = ctx->flags &IORING_SETUP_CQE32 ? cqe->big_cqe[1 ] :0 ; + __entry->extra1 = ctx->flags &IORING_SETUP_CQE32 || cqe->flags &IORING_CQE_F_32 ? cqe->big_cqe[0 ] :0 ; + __entry->extra2 = ctx->flags &IORING_SETUP_CQE32 || cqe->flags &IORING_CQE_F_32 ? cqe->big_cqe[1 ] :0 ; ), TP_printk("ring %p, req %p, user_data 0x%llx, result %d, cflags 0x%x "
Signature · 1 line
2.50.1
Jens Axboe
maintainerPATCH v2 5/8 io_uring: add support for IORING_SETUP_CQE_MIXED
In reply to Jens AxboeNormal rings support 16b CQEs for posting completions, while certain features require the ring to be configured with IORING_SETUP_CQE32, as they need to convey more information per completion. This, in turn, makes ALL the CQEs be 32b in size. This is somewhat wasteful and inefficient, particularly when only certain CQEs need to be of the bigger variant.
This adds support for setting up a ring with mixed CQE sizes, using IORING_SETUP_CQE_MIXED. When setup in this mode, CQEs posted to the ring may be either 16b or 32b in size. If a CQE is 32b in size, then IORING_CQE_F_32 is set in the CQE flags to indicate that this is the case. If this flag isn't set, the CQE is the normal 16b variant.
CQEs on these types of mixed rings may also have IORING_CQE_F_SKIP set. This can happen if the ring is one (small) CQE entry away from wrapping, and an attempt is made to post a 32b CQE. As CQEs must be contigious in the CQ ring, a 32b CQE cannot wrap the ring. For this case, a single dummy CQE is posted with the SKIP flag set. The application should simply ignore those.
--- include/uapi/linux/io_uring.h | 6 +++ io_uring/io_uring.c | 78 ++++++++++++++++++++++++++++------- io_uring/io_uring.h | 49 +++++++++++++++------- io_uring/register.c | 3 +- 4 files changed, 105 insertions(+), 31 deletions(-)
include/uapi/linux/io_uring.h +6 -0
@@ -225,6 +225,12 @@ enum io_uring_sqe_flags_bit {/* Use hybrid poll in iopoll process */ #define IORING_SETUP_HYBRID_IOPOLL (1U << 17) +/* +* Allow both 16b and 32b CQEs. If a 32b CQE is posted, it will have +* IORING_CQE_F_32 set in cqe->flags. +*/ +#define IORING_SETUP_CQE_MIXED (1U << 18) +enum io_uring_op {IORING_OP_NOP ,IORING_OP_READV ,
io_uring/io_uring.c +62 -16
165 lines not on this page · raw message
io_uring/io_uring.h +35 -14
98 lines not on this page · raw message
io_uring/register.c +2 -1
@@ -396,7 +396,8 @@ static void io_register_free_rings(struct io_ring_ctx *ctx,#define RESIZE_FLAGS (IORING_SETUP_CQSIZE | IORING_SETUP_CLAMP)#define COPY_FLAGS (IORING_SETUP_NO_SQARRAY | IORING_SETUP_SQE128 | \ - IORING_SETUP_CQE32 | IORING_SETUP_NO_MMAP) + IORING_SETUP_CQE32 | IORING_SETUP_NO_MMAP | \ + IORING_SETUP_CQE_MIXED)static int io_register_resize_rings (struct io_ring_ctx *ctx,void __user *arg) {
Signature · 1 line
2.50.1
Jens Axboe
maintainerPATCH v2 6/8 io_uring/nop: add support for IORING_SETUP_CQE_MIXED
In reply to Jens AxboeThis adds support for setting IORING_NOP_CQE32 as a flag for a NOP command, in which case a 32b CQE will be posted rather than a regular one. This is the default if the ring has been setup with IORING_SETUP_CQE32. If the ring has been setup with IORING_SETUP_CQE_MIXED, then 16b CQEs will be posted without this flag set, and 32b CQEs if this flag is set. For the latter case, sqe->off is what will be posted as cqe->big_cqe[0] and sqe->addr is what will be posted as cqe->big_cqe[1].
--- include/uapi/linux/io_uring.h | 1 + io_uring/nop.c | 17 +++++++++++++++-- 2 files changed, 16 insertions(+), 2 deletions(-)
include/uapi/linux/io_uring.h +1 -0
@@ -464,6 +464,7 @@ enum io_uring_msg_ring_flags {#define IORING_NOP_FIXED_FILE (1U << 2)#define IORING_NOP_FIXED_BUFFER (1U << 3)#define IORING_NOP_TW (1U << 4) +#define IORING_NOP_CQE32 (1U << 5)/* * IO completion data structure (Completion Queue Entry)
io_uring/nop.c +15 -2
@@ -17,11 +17,13 @@ struct io_nop {int result;int fd;unsigned int flags; +__u64 extra1; +__u64 extra2; };#define NOP_FLAGS (IORING_NOP_INJECT_RESULT | IORING_NOP_FIXED_FILE | \ IORING_NOP_FIXED_BUFFER | IORING_NOP_FILE | \ - IORING_NOP_TW) + IORING_NOP_TW | IORING_NOP_CQE32)int io_nop_prep (struct io_kiocb *req,const struct io_uring_sqe *sqe) {
@@ -41,6 +43,14 @@ int io_nop_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe) nop->fd =-1 ;if (nop->flags &IORING_NOP_FIXED_BUFFER ) req->buf_index =READ_ONCE (sqe->buf_index); +if (nop->flags &IORING_NOP_CQE32 ) { +struct io_ring_ctx *ctx = req->ctx; + +if (!(ctx->flags & (IORING_SETUP_CQE32 |IORING_SETUP_CQE_MIXED ))) +return -EINVAL ; + nop->extra1 =READ_ONCE (sqe->off); + nop->extra2 =READ_ONCE (sqe->addr); + }return 0 ; }
@@ -68,7 +78,10 @@ int io_nop(struct io_kiocb *req, unsigned int issue_flags) done:if (ret <0 )req_set_fail (req); -io_req_set_res (req, nop->result,0 ); +if (nop->flags &IORING_NOP_CQE32 ) +io_req_set_res32 (req, nop->result,0 , nop->extra1, nop->extra2); +else +io_req_set_res (req, nop->result,0 );if (nop->flags &IORING_NOP_TW ) { req->io_task_work.func = io_req_task_complete;io_req_task_work_add (req);
Signature · 1 line
2.50.1
Jens Axboe
maintainerPATCH v2 7/8 io_uring/uring_cmd: add support for IORING_SETUP_CQE_MIXED
In reply to Jens AxboeCertain users of uring_cmd currently require fixed 32b CQE support, which is propagated through IO_URING_F_CQE32. Allow IORING_SETUP_CQE_MIXED to cover that case as well, so not all CQEs posted need to be 32b in size.
--- io_uring/cmd_net.c | 3 ++- io_uring/uring_cmd.c | 2 +- 2 files changed, 3 insertions(+), 2 deletions(-)
io_uring/cmd_net.c +2 -1
@@ -4,6 +4,7 @@#include <net/sock.h> #include "uring_cmd.h" +#include "io_uring.h" static inline int io_uring_cmd_getsockopt(struct socket *sock, struct io_uring_cmd *cmd,
@@ -73,7 +74,7 @@ static bool io_process_timestamp_skb(struct io_uring_cmd *cmd, struct sock *sk, cqe->user_data =0 ; cqe->res = tskey; - cqe->flags =IORING_CQE_F_MORE ; + cqe->flags =IORING_CQE_F_MORE |ctx_cqe32_flags (cmd_to_io_kiocb (cmd)->ctx); cqe->flags |= tstype <<IORING_TIMESTAMP_TYPE_SHIFT ;if (ret ==SOF_TIMESTAMPING_TX_HARDWARE ) cqe->flags |=IORING_CQE_F_TSTAMP_HW ;
io_uring/uring_cmd.c +1 -1
@@ -248,7 +248,7 @@ int io_uring_cmd(struct io_kiocb *req, unsigned int issue_flags)if (ctx->flags &IORING_SETUP_SQE128 ) issue_flags |=IO_URING_F_SQE128 ; -if (ctx->flags &IORING_SETUP_CQE32 ) +if (ctx->flags & (IORING_SETUP_CQE32 |IORING_SETUP_CQE_MIXED )) issue_flags |=IO_URING_F_CQE32 ;if (io_is_compat (ctx)) issue_flags |=IO_URING_F_COMPAT ;
Signature · 1 line
2.50.1
Jens Axboe
maintainerPATCH v2 8/8 io_uring/zcrx: add support for IORING_SETUP_CQE_MIXED
In reply to Jens Axboezcrx currently requires the ring to be set up with fixed 32b CQEs, allow it to use IORING_SETUP_CQE_MIXED as well.
--- Documentation/networking/iou-zcrx.rst | 2 +- io_uring/zcrx.c | 5 +++-- 2 files changed, 4 insertions(+), 3 deletions(-)
Documentation/networking/iou-zcrx.rst +1 -1
@@ -75,7 +75,7 @@ Create an io_uring instance with the following required setup flags:: IORING_SETUP_SINGLE_ISSUER IORING_SETUP_DEFER_TASKRUN - IORING_SETUP_CQE32 + IORING_SETUP_CQE32 or IORING_SETUP_CQE_MIXED Create memory area ------------------
io_uring/zcrx.c +3 -2
@@ -554,8 +554,9 @@ int io_register_zcrx_ifq(struct io_ring_ctx *ctx,return -EPERM ;/* mandatory io_uring features for zc rx */ -if (!(ctx->flags &IORING_SETUP_DEFER_TASKRUN && - ctx->flags &IORING_SETUP_CQE32 )) +if (!(ctx->flags &IORING_SETUP_DEFER_TASKRUN )) +return -EINVAL ; +if (!(ctx->flags & (IORING_SETUP_CQE32 |IORING_SETUP_CQE_MIXED )))return -EINVAL ;if (copy_from_user (®, arg,sizeof (reg)))return -EFAULT ;
Signature · 1 line
2.50.1
Quoted 13 lines · Jens Axboe
Hi,
Currently
io_uringsupports two modes for CQEs:1) The standard mode, where 16b CQEs are used
2) SettingIORING_SETUP_CQE32, which makes all CQEs posted 32bCertain features need to pass more information back than just a single 32-bit res field, and hence mandate the use of CQE32 to be able to work. Examples of that include passthrough or other uses of ->uring_cmd() like socket option getting and setting, including timestamps.
Quoted 7 lines
This patchset adds support for
IORING_SETUP_CQE_MIXED, which allows posting both 16b and 32b CQEs on the same CQ ring. The idea here is that we need not waste twice the space for CQ rings, or use twice the space per CQE posted, if only some of the CQEs posted require the use of 32b CQEs. On a ring setup in CQE mixed mode, 32b posted CQEs will haveIORING_CQE_F_32set incqe->flagsto tell the application (or liburing) about this fact.
This makes a lot of sense. Have you considered something analogous for SQEs? Requiring all SQEs to be 128 bytes when an io_uring is used for a mix of 64-byte and 128-byte SQEs also wastes memory, probably even more since SQEs are 4x larger than CQEs.
Best,
Caleb
Quoted 50 lines
This is mostly trivial to support, with the corner case being attempting to post a 32b CQE when the ring is a single 16b CQE away from wrapping. As CQEs must be contigious in memory, that's simply not possible. The solution taken by this patchset is to add a special CQE type, which has
IORING_CQE_F_SKIPset. This is a pad/nop CQE, which should simply be ignored, as it carries no information and serves no other purpose than to re-align the posted CQEs for ring wrap.If used with liburing, then both the 32b vs 16b postings and the skip are transparent.
liburing support and a few basic test cases can be found here:
https://git.kernel.org/pub/scm/linux/kernel/git/axboe/liburing.git/log/?h=cqe-mixed
including man page updates for the newly added setup and CQE flags, and the patches posted here can also be found at:
https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git/log/?h=io_uring-cqe-mix
Patch 1 is just a prep patch, and patch 2 adds the cqe flags so that the core can be adapted before support is actually there. Patches 3 and 4 are exactly that, and patch 5 finally adds support for the mixed mode. Patch 6 adds support for NOP testing of this, and patches 7/8 allow
IORING_SETUP_CQE_MIXEDfor uring_cmd/zcrx which previously requiredIORING_SETUP_CQE32to work.Documentation/networking/iou-zcrx.rst | 2 +- include/linux/io_uring_types.h | 6 --- include/trace/events/io_uring.h | 4 +- include/uapi/linux/io_uring.h | 17 ++++++ io_uring/cmd_net.c | 3 +- io_uring/fdinfo.c | 22 ++++---- io_uring/io_uring.c | 78 +++++++++++++++++++++------ io_uring/io_uring.h | 49 ++++++++++++----- io_uring/nop.c | 17 +++++- io_uring/register.c | 3 +- io_uring/uring_cmd.c | 2 +- io_uring/zcrx.c | 5 +- 12 files changed, 152 insertions(+), 56 deletions(-)Changes since v1:
- Various little cleanups
- Rebase on for-6.18/io_uring--
Jens Axboe
Jens Axboe
maintainerQuoted 22 lines · Caleb Sander Mateos
Quoted 13 lines · On Thu, Aug 21, 2025 at 7:28?AM Jens Axboe <[email protected]> wrote:
Hi,
Currently
io_uringsupports two modes for CQEs:1) The standard mode, where 16b CQEs are used
2) SettingIORING_SETUP_CQE32, which makes all CQEs posted 32bCertain features need to pass more information back than just a single 32-bit res field, and hence mandate the use of CQE32 to be able to work. Examples of that include passthrough or other uses of ->uring_cmd() like socket option getting and setting, including timestamps.
Quoted 7 lines
This patchset adds support for
IORING_SETUP_CQE_MIXED, which allows posting both 16b and 32b CQEs on the same CQ ring. The idea here is that we need not waste twice the space for CQ rings, or use twice the space per CQE posted, if only some of the CQEs posted require the use of 32b CQEs. On a ring setup in CQE mixed mode, 32b posted CQEs will haveIORING_CQE_F_32set incqe->flagsto tell the application (or liburing) about this fact.
Quoted 4 lines
This makes a lot of sense. Have you considered something analogous for SQEs? Requiring all SQEs to be 128 bytes when an
io_uringis used for a mix of 64-byte and 128-byte SQEs also wastes memory, probably even more since SQEs are 4x larger than CQEs.
Adding Keith, as he and I literally just talked about that. My answer was that the case is a bit different in that 32b CQEs can be useful in cases that are predominately 16b in the first place. For example, networking workload doing send/recv/etc and the occassional get/setsockopt kind of thing. Or maybe a mix of normal recv and zero copy rx.
For the SQE case, I think it's a bit different. At least the cases I know of, it's mostly 100% 64b SQEs or 128b SQEs. I'm certainly willing to be told otherwise! Because that is kind of the key question that needs answering before even thinking about doing that kind of work.
But yes, it could be supported, and Keith (kind of) signed himself up to do that. One oddity I see on that side is that while with CQE32 the kernel can manage the potential wrap-around gap, for SQEs that's obviously on the application to do. That could just be a NOP or something like that, but you do need something to fill/skip that space. I guess that could be as simple as having an opcode that is simply "skip me", so on the kernel side it'd be easy as it'd just drop it on the floor. You still need to app side to fill one, however, and then deal with "oops SQ ring is now full" too.
Probably won't be too bad at all, however.
Signature · 1 line
Jens Axboe
Quoted 5 lines · Jens Axboe
For the SQE case, I think it's a bit different. At least the cases I know of, it's mostly 100% 64b SQEs or 128b SQEs. I'm certainly willing to be told otherwise! Because that is kind of the key question that needs answering before even thinking about doing that kind of work.
The main use case I can think of is if an application allocates one ring for uring_cmd with the 128b SQEs, and then a separate ring for normal file and network stuff. Mixed SQE's would allow that application to have just one ring without being wasteful, but I'm just not sure if the separate rings is undesirable enough to make the effort worth it.
Quoted 9 lines
But yes, it could be supported, and Keith (kind of) signed himself up to do that. One oddity I see on that side is that while with CQE32 the kernel can manage the potential wrap-around gap, for SQEs that's obviously on the application to do. That could just be a NOP or something like that, but you do need something to fill/skip that space. I guess that could be as simple as having an opcode that is simply "skip me", so on the kernel side it'd be easy as it'd just drop it on the floor. You still need to app side to fill one, however, and then deal with "oops SQ ring is now full" too.
Yep, I think it's doable, and your implementation for mixed CQEs provides a great reference. I trust we can get liburing using it correctly, but would be afraid for anyone not using the library.
Jens Axboe
maintainerQuoted 7 lines · Keith Busch
Quoted 5 lines · Jens Axboe
For the SQE case, I think it's a bit different. At least the cases I know of, it's mostly 100% 64b SQEs or 128b SQEs. I'm certainly willing to be told otherwise! Because that is kind of the key question that needs answering before even thinking about doing that kind of work.
Quoted 5 lines
The main use case I can think of is if an application allocates one ring for
uring_cmdwith the 128b SQEs, and then a separate ring for normal file and network stuff. Mixed SQE's would allow that application to have just one ring without being wasteful, but I'm just not sure if the separate rings is undesirable enough to make the effort worth it.
Indeed! And like Caleb mentioned, their use case already does this in fact, just passthrough with housekeeping buffer commands.
Quoted 10 lines
Quoted 9 lines
But yes, it could be supported, and Keith (kind of) signed himself up to do that. One oddity I see on that side is that while with CQE32 the kernel can manage the potential wrap-around gap, for SQEs that's obviously on the application to do. That could just be a NOP or something like that, but you do need something to fill/skip that space. I guess that could be as simple as having an opcode that is simply "skip me", so on the kernel side it'd be easy as it'd just drop it on the floor. You still need to app side to fill one, however, and then deal with "oops SQ ring is now full" too.
Quoted 3 lines
Yep, I think it's doable, and your implementation for mixed CQEs provides a great reference. I trust we can get liburing using it correctly, but would be afraid for anyone not using the library.
That's the same with the mixed CQEs though - as you can see from the liburing changes, it's really not hard to support. If you're using the raw interface, well then things are already a bit more complicated for you. Not too worried about that use case.
Signature · 1 line
Jens Axboe
Quoted 36 lines · Jens Axboe
Quoted 22 lines · Caleb Sander Mateos
Quoted 13 lines · On Thu, Aug 21, 2025 at 7:28?AM Jens Axboe <[email protected]> wrote:
Hi,
Currently
io_uringsupports two modes for CQEs:1) The standard mode, where 16b CQEs are used
2) SettingIORING_SETUP_CQE32, which makes all CQEs posted 32bCertain features need to pass more information back than just a single 32-bit res field, and hence mandate the use of CQE32 to be able to work. Examples of that include passthrough or other uses of ->uring_cmd() like socket option getting and setting, including timestamps.
Quoted 7 lines
This patchset adds support for
IORING_SETUP_CQE_MIXED, which allows posting both 16b and 32b CQEs on the same CQ ring. The idea here is that we need not waste twice the space for CQ rings, or use twice the space per CQE posted, if only some of the CQEs posted require the use of 32b CQEs. On a ring setup in CQE mixed mode, 32b posted CQEs will haveIORING_CQE_F_32set incqe->flagsto tell the application (or liburing) about this fact.Quoted 4 lines
This makes a lot of sense. Have you considered something analogous for SQEs? Requiring all SQEs to be 128 bytes when an
io_uringis used for a mix of 64-byte and 128-byte SQEs also wastes memory, probably even more since SQEs are 4x larger than CQEs.Adding Keith, as he and I literally just talked about that. My answer was that the case is a bit different in that 32b CQEs can be useful in cases that are predominately 16b in the first place. For example, networking workload doing send/recv/etc and the occassional get/setsockopt kind of thing. Or maybe a mix of normal recv and zero copy rx.
Quoted 4 lines
For the SQE case, I think it's a bit different. At least the cases I know of, it's mostly 100% 64b SQEs or 128b SQEs. I'm certainly willing to be told otherwise! Because that is kind of the key question that needs answering before even thinking about doing that kind of work.
We certainly have a use case that mixes the two on the same io_uring:
ublk commit/buffer register/unregister commands (64 byte SQEs) and NVMe passthru commands (128 byte SQEs). I could also imagine an application issuing both normal read/write commands and NVMe passthru commands. But you're probably right that this isn't a super common use case.
Quoted 10 lines
But yes, it could be supported, and Keith (kind of) signed himself up to do that. One oddity I see on that side is that while with CQE32 the kernel can manage the potential wrap-around gap, for SQEs that's obviously on the application to do. That could just be a NOP or something like that, but you do need something to fill/skip that space. I guess that could be as simple as having an opcode that is simply "skip me", so on the kernel side it'd be easy as it'd just drop it on the floor. You still need to app side to fill one, however, and then deal with "oops SQ ring is now full" too.
Sure, of course userspace would need to handle a misaligned big SQE at the end of the SQ analogously to mixed CQE sizes. I assume liburing should be able to do that mostly transparently, that logic could all be encapsulated by io_uring_get_sqe().
Best,
Caleb
Quoted 5 lines
Probably won't be too bad at all, however.
--
Jens Axboe
Jens Axboe
maintainerQuoted 42 lines · Caleb Sander Mateos
Quoted 36 lines · On Thu, Aug 21, 2025 at 10:12?AM Jens Axboe <[email protected]> wrote:
Quoted 22 lines · Caleb Sander Mateos
Quoted 13 lines · On Thu, Aug 21, 2025 at 7:28?AM Jens Axboe <[email protected]> wrote:
Hi,
Currently
io_uringsupports two modes for CQEs:1) The standard mode, where 16b CQEs are used
2) SettingIORING_SETUP_CQE32, which makes all CQEs posted 32bCertain features need to pass more information back than just a single 32-bit res field, and hence mandate the use of CQE32 to be able to work. Examples of that include passthrough or other uses of ->uring_cmd() like socket option getting and setting, including timestamps.
Quoted 7 lines
This patchset adds support for
IORING_SETUP_CQE_MIXED, which allows posting both 16b and 32b CQEs on the same CQ ring. The idea here is that we need not waste twice the space for CQ rings, or use twice the space per CQE posted, if only some of the CQEs posted require the use of 32b CQEs. On a ring setup in CQE mixed mode, 32b posted CQEs will haveIORING_CQE_F_32set incqe->flagsto tell the application (or liburing) about this fact.Quoted 4 lines
This makes a lot of sense. Have you considered something analogous for SQEs? Requiring all SQEs to be 128 bytes when an
io_uringis used for a mix of 64-byte and 128-byte SQEs also wastes memory, probably even more since SQEs are 4x larger than CQEs.Adding Keith, as he and I literally just talked about that. My answer was that the case is a bit different in that 32b CQEs can be useful in cases that are predominately 16b in the first place. For example, networking workload doing send/recv/etc and the occassional get/setsockopt kind of thing. Or maybe a mix of normal recv and zero copy rx.
Quoted 4 lines
For the SQE case, I think it's a bit different. At least the cases I know of, it's mostly 100% 64b SQEs or 128b SQEs. I'm certainly willing to be told otherwise! Because that is kind of the key question that needs answering before even thinking about doing that kind of work.
Quoted 6 lines
We certainly have a use case that mixes the two on the same
io_uring:
ublk commit/buffer register/unregister commands (64 byte SQEs) and NVMe passthru commands (128 byte SQEs). I could also imagine an application issuing both normal read/write commands and NVMe passthru commands. But you're probably right that this isn't a super common use case.
Yes that's a good point, and that would roughly be 50/50 in terms of 64b vs 128b SQEs?
And yes, I can imagine other uses cases too, but I'm also finding a hard time justifying those as likely. On the other hand, people do the weirdest things...
Quoted 10 lines
Quoted 9 lines
But yes, it could be supported, and Keith (kind of) signed himself up to do that. One oddity I see on that side is that while with CQE32 the kernel can manage the potential wrap-around gap, for SQEs that's obviously on the application to do. That could just be a NOP or something like that, but you do need something to fill/skip that space. I guess that could be as simple as having an opcode that is simply "skip me", so on the kernel side it'd be easy as it'd just drop it on the floor. You still need to app side to fill one, however, and then deal with "oops SQ ring is now full" too.
Quoted 4 lines
Sure, of course userspace would need to handle a misaligned big SQE at the end of the SQ analogously to mixed CQE sizes. I assume liburing should be able to do that mostly transparently, that logic could all be encapsulated by
io_uring_get_sqe().
Yep I think so, we'd need a new helper to return the kind of SQE you want, and it'd just need to get a 64b one and mark it with the SKIP opcode first if being asked for a 128b one and we're one off from wrapping around.
Signature · 1 line
Jens Axboe
Quoted 51 lines · Jens Axboe
Quoted 42 lines · Caleb Sander Mateos
Quoted 36 lines · On Thu, Aug 21, 2025 at 10:12?AM Jens Axboe <[email protected]> wrote:
Quoted 22 lines · Caleb Sander Mateos
Quoted 13 lines · On Thu, Aug 21, 2025 at 7:28?AM Jens Axboe <[email protected]> wrote:
Hi,
Currently
io_uringsupports two modes for CQEs:1) The standard mode, where 16b CQEs are used
2) SettingIORING_SETUP_CQE32, which makes all CQEs posted 32bCertain features need to pass more information back than just a single 32-bit res field, and hence mandate the use of CQE32 to be able to work. Examples of that include passthrough or other uses of ->uring_cmd() like socket option getting and setting, including timestamps.
Quoted 7 lines
This patchset adds support for
IORING_SETUP_CQE_MIXED, which allows posting both 16b and 32b CQEs on the same CQ ring. The idea here is that we need not waste twice the space for CQ rings, or use twice the space per CQE posted, if only some of the CQEs posted require the use of 32b CQEs. On a ring setup in CQE mixed mode, 32b posted CQEs will haveIORING_CQE_F_32set incqe->flagsto tell the application (or liburing) about this fact.Quoted 4 lines
This makes a lot of sense. Have you considered something analogous for SQEs? Requiring all SQEs to be 128 bytes when an
io_uringis used for a mix of 64-byte and 128-byte SQEs also wastes memory, probably even more since SQEs are 4x larger than CQEs.Adding Keith, as he and I literally just talked about that. My answer was that the case is a bit different in that 32b CQEs can be useful in cases that are predominately 16b in the first place. For example, networking workload doing send/recv/etc and the occassional get/setsockopt kind of thing. Or maybe a mix of normal recv and zero copy rx.
Quoted 4 lines
For the SQE case, I think it's a bit different. At least the cases I know of, it's mostly 100% 64b SQEs or 128b SQEs. I'm certainly willing to be told otherwise! Because that is kind of the key question that needs answering before even thinking about doing that kind of work.
Quoted 6 lines
We certainly have a use case that mixes the two on the same
io_uring:
ublk commit/buffer register/unregister commands (64 byte SQEs) and NVMe passthru commands (128 byte SQEs). I could also imagine an application issuing both normal read/write commands and NVMe passthru commands. But you're probably right that this isn't a super common use case.
Quoted 2 lines
Yes that's a good point, and that would roughly be 50/50 in terms of 64b vs 128b SQEs?
For our application, the ratio between 64 and 128 bytes SQEs depends on the ublk workload. Small ublk I/Os are translated 1-1 into NVMe passthru I/Os, so there can be as many as 3 64-byte ublk SQEs (register buffer, unregister buffer, and commit) for each 128-byte NVMe passthru SQE. Larger I/Os are sharded into more NVMe passthru commands, so there are relatively more 128-byte SQEs. And some workloads can't use ublk zero-copy (since the data needs to go through a RAID computation), in which case the only 64-byte SQE is the ublk commit.
Best,
Caleb
Quoted 27 lines
And yes, I can imagine other uses cases too, but I'm also finding a hard time justifying those as likely. On the other hand, people do the weirdest things...
Quoted 10 lines
Quoted 9 lines
But yes, it could be supported, and Keith (kind of) signed himself up to do that. One oddity I see on that side is that while with CQE32 the kernel can manage the potential wrap-around gap, for SQEs that's obviously on the application to do. That could just be a NOP or something like that, but you do need something to fill/skip that space. I guess that could be as simple as having an opcode that is simply "skip me", so on the kernel side it'd be easy as it'd just drop it on the floor. You still need to app side to fill one, however, and then deal with "oops SQ ring is now full" too.
Quoted 4 lines
Sure, of course userspace would need to handle a misaligned big SQE at the end of the SQ analogously to mixed CQE sizes. I assume liburing should be able to do that mostly transparently, that logic could all be encapsulated by
io_uring_get_sqe().Yep I think so, we'd need a new helper to return the kind of SQE you want, and it'd just need to get a 64b one and mark it with the SKIP opcode first if being asked for a 128b one and we're one off from wrapping around.
--
Jens Axboe