Skip to contentSkip to conversation

Introduce non circular SQ

PATCH Superseded Pavel Begunkov 3 participants 13 messages Oct 14, 2025 – Oct 16, 2025 io_uring

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

cover Introduce non circular SQ

Add a feature that makes the kernel to ignore SQ head/tail and always start fetching SQ entries from index 0, which helps to keep caches hot. See Patch 2 for more details.

liburing support:
https://github.com/isilence/liburing.git sq-rewind

Tested by forcing liburing to enable the flag for compatible setups.

Pavel Begunkov (2):
io_uring: check for user passing 0 nr_submit
io_uring: introduce non-circular SQ

 include/uapi/linux/io_uring.h |  6 ++++++
 io_uring/io_uring.c           | 34 +++++++++++++++++++++++++---------
 io_uring/io_uring.h           |  3 ++-
 3 files changed, 33 insertions(+), 10 deletions(-)
Signature · 1 line
2.49.0
#1 lore raw

PATCH 1/2 io_uring: check for user passing 0 nr_submit

io_submit_sqes() shouldn't be stepping into its main loop when there is nothing to submit, i.e. nr=0. Fix 0 submission queue entries checks, which should follow after all user input truncations.

---
 io_uring/io_uring.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
io_uring/io_uring.c +3 -2
index 820ef0527666..ee04ab9bf968 100644
@@ -2422,10 +2422,11 @@ int io_submit_sqes(struct io_ring_ctx *ctx, unsigned int nr)
 	unsigned int left;
 	int ret;
 
+	entries = min(nr, entries);
 	if (unlikely(!entries))
 		return 0;
-	/* make sure SQ entry isn't read before tail */
-	ret = left = min(nr, entries);
+
+	ret = left = entries;
 	io_get_task_refs(left);
 	io_submit_state_start(&ctx->submit_state, left);
 
Signature · 1 line
2.49.0
#2 lore raw

PATCH 2/2 io_uring: introduce non-circular SQ

In reply to Pavel Begunkov

Outside of SQPOLL, normally SQ entries are consumed by the time the submission syscall returns. For those cases we don't need a circular buffer and the head/tail tracking, instead the kernel can assume that entries always start from the beginning of the SQ at index 0. This patch introduces a setup flag doing exactly that.

This method is simpler in general, needs fewer operations, doesn't require looking up heads and tails, however, the main goal here is to keep caches hot. The userspace might overprovision SQ, and in the normal way we'd be touching all the cache lines, but with this feature it reuses first entries and keeps them hot. This simplicity will also be quite handy for bpf-io_uring.

To use the feature the user should set the IORING_SETUP_SQ_REWIND flag, and have a compatible liburing/userspace. The flag is restricted to IORING_SETUP_NO_SQARRAY rings and is not compatible with IORING_SETUP_SQPOLL.

Note, it uses relaxed ring synchronisation as the userspace doing the syscall is naturally in sync, and setups that share a SQ should be rolling their own intra process/thread synchronisation.

---
 include/uapi/linux/io_uring.h |  6 ++++++
 io_uring/io_uring.c           | 29 ++++++++++++++++++++++-------
 io_uring/io_uring.h           |  3 ++-
 3 files changed, 30 insertions(+), 8 deletions(-)
include/uapi/linux/io_uring.h +6 -0
index a0cc1cc0dd01..d1c654a7fa9a 100644
@@ -231,6 +231,12 @@ enum io_uring_sqe_flags_bit {
  */
 #define IORING_SETUP_CQE_MIXED		(1U << 18)
 
+/*
+ * SQEs always start at index 0 in the submission ring instead of using a
+ * wrap around indexing.
+ */
+#define IORING_SETUP_SQ_REWIND		(1U << 19)
+
 enum io_uring_op {
 	IORING_OP_NOP,
 	IORING_OP_READV,
io_uring/io_uring.c +22 -7
index ee04ab9bf968..e8af963d3233 100644
@@ -2367,12 +2367,16 @@ static void io_commit_sqring(struct io_ring_ctx *ctx)
 {
 	struct io_rings *rings = ctx->rings;
 
-	/*
-	 * Ensure any loads from the SQEs are done at this point,
-	 * since once we write the new head, the application could
-	 * write new data to them.
-	 */
-	smp_store_release(&rings->sq.head, ctx->cached_sq_head);
+	if (ctx->flags & IORING_SETUP_SQ_REWIND) {
+		ctx->cached_sq_head = 0;
+	} else {
+		/*
+		 * Ensure any loads from the SQEs are done at this point,
+		 * since once we write the new head, the application could
+		 * write new data to them.
+		 */
+		smp_store_release(&rings->sq.head, ctx->cached_sq_head);
+	}
 }
 
 /*
@@ -2418,10 +2422,15 @@ static bool io_get_sqe(struct io_ring_ctx *ctx, const struct io_uring_sqe **sqe)
 int io_submit_sqes(struct io_ring_ctx *ctx, unsigned int nr)
 	__must_hold(&ctx->uring_lock)
 {
-	unsigned int entries = io_sqring_entries(ctx);
+	unsigned int entries;
 	unsigned int left;
 	int ret;
 
+	if (ctx->flags & IORING_SETUP_SQ_REWIND)
+		entries = ctx->sq_entries;
+	else
+		entries = io_sqring_entries(ctx);
+
 	entries = min(nr, entries);
 	if (unlikely(!entries))
 		return 0;
@@ -3678,6 +3687,12 @@ static int io_uring_sanitise_params(struct io_uring_params *p)
 {
 	unsigned flags = p->flags;
 
+	if (flags & IORING_SETUP_SQ_REWIND) {
+		if ((flags & IORING_SETUP_SQPOLL) ||
+		    !(flags & IORING_SETUP_NO_SQARRAY))
+		return -EINVAL;
+	}
+
 	/* There is no way to mmap rings without a real fd */
 	if ((flags & IORING_SETUP_REGISTERED_FD_ONLY) &&
 	    !(flags & IORING_SETUP_NO_MMAP))
io_uring/io_uring.h +2 -1
index 46d9141d772a..b998ed57dd93 100644
@@ -54,7 +54,8 @@
 			IORING_SETUP_REGISTERED_FD_ONLY |\
 			IORING_SETUP_NO_SQARRAY |\
 			IORING_SETUP_HYBRID_IOPOLL |\
-			IORING_SETUP_CQE_MIXED)
+			IORING_SETUP_CQE_MIXED |\
+			IORING_SETUP_SQ_REWIND)
 
 #define IORING_ENTER_FLAGS (IORING_ENTER_GETEVENTS |\
 			IORING_ENTER_SQ_WAKEUP |\
Signature · 1 line
2.49.0
maintainer
#9 lore raw
In reply to Pavel Begunkov
Quoted 9 lines · Pavel Begunkov

Add a feature that makes the kernel to ignore SQ head/tail and always start fetching SQ entries from index 0, which helps to keep caches hot. See Patch 2 for more details.

liburing support:
https://github.com/isilence/liburing.git sq-rewind

Tested by forcing liburing to enable the flag for compatible setups.

Quoted 8 lines

Pavel Begunkov (2):
io_uring: check for user passing 0 nr_submit
io_uring: introduce non-circular SQ

 include/uapi/linux/io_uring.h |  6 ++++++
 io_uring/io_uring.c           | 34 +++++++++++++++++++++++++---------
 io_uring/io_uring.h           |  3 ++-
 3 files changed, 33 insertions(+), 10 deletions(-)

I like the concept of this, makes a lot of sense. No need to keep churning through the entire SQ ring, when apps mostly submit a few requests at the time. Will help cut down on cacheline usage.

Curious, do you have any numbers on this for any kind of workload?

Signature · 1 line
Jens Axboe
#10 lore raw
Quoted 19 lines · Jens Axboe
Quoted 9 lines · Pavel Begunkov

Add a feature that makes the kernel to ignore SQ head/tail and always start fetching SQ entries from index 0, which helps to keep caches hot. See Patch 2 for more details.

liburing support:
https://github.com/isilence/liburing.git sq-rewind

Tested by forcing liburing to enable the flag for compatible setups.

Quoted 8 lines

Pavel Begunkov (2):
io_uring: check for user passing 0 nr_submit
io_uring: introduce non-circular SQ

  include/uapi/linux/io_uring.h |  6 ++++++
  io_uring/io_uring.c           | 34 +++++++++++++++++++++++++---------
  io_uring/io_uring.h           |  3 ++-
  3 files changed, 33 insertions(+), 10 deletions(-)
Quoted 5 lines

I like the concept of this, makes a lot of sense. No need to keep churning through the entire SQ ring, when apps mostly submit a few requests at the time. Will help cut down on cacheline usage.

Curious, do you have any numbers on this for any kind of workload?

No, very likely it's a micro optimisation in the grand picture and would be hard to measure for anything sensible / realistic. It shouldn't be too difficult to come up with a test with a bunch of pinned tasks putting a memory pressure, but that would be as useful as it sounds.

Signature · 1 line
Pavel Begunkov
#11 lore raw
Quoted 24 lines · Pavel Begunkov
Quoted 23 lines · Jens Axboe
Quoted 9 lines · Pavel Begunkov

Add a feature that makes the kernel to ignore SQ head/tail and always start fetching SQ entries from index 0, which helps to keep caches hot. See Patch 2 for more details.

liburing support:
https://github.com/isilence/liburing.git sq-rewind

Tested by forcing liburing to enable the flag for compatible setups.

Quoted 8 lines

Pavel Begunkov (2):
   io_uring: check for user passing 0 nr_submit
   io_uring: introduce non-circular SQ

  include/uapi/linux/io_uring.h |  6 ++++++ io_uring/io_uring.c           | 34 +++++++++++++++++++++++++--------- io_uring/io_uring.h           |  3 ++-

  3 files changed, 33 insertions(+), 10 deletions(-)

I like the concept of this, makes a lot of sense. No need to keep churning through the entire SQ ring, when apps mostly submit a few requests at the time. Will help cut down on cacheline usage.

Quoted 7 lines
Quoted 1 line

Curious, do you have any numbers on this for any kind of workload?

No, very likely it's a micro optimisation in the grand picture and would be hard to measure for anything sensible / realistic. It shouldn't be too difficult to come up with a test with a bunch of pinned tasks putting a memory pressure, but that would be as useful as it sounds.

Simplicity was not the last priority either. E.g. userspace can replace all io_uring_get_sqe() with wrapping the SQ pointer into some array_view at the beginning and work with that.

Signature · 1 line
Pavel Begunkov
maintainer
#12 lore raw
Quoted 33 lines · Pavel Begunkov
Quoted 24 lines · Pavel Begunkov
Quoted 23 lines · Jens Axboe
Quoted 9 lines · Pavel Begunkov

Add a feature that makes the kernel to ignore SQ head/tail and always start fetching SQ entries from index 0, which helps to keep caches hot. See Patch 2 for more details.

liburing support:
https://github.com/isilence/liburing.git sq-rewind

Tested by forcing liburing to enable the flag for compatible setups.

Quoted 8 lines

Pavel Begunkov (2):
   io_uring: check for user passing 0 nr_submit
   io_uring: introduce non-circular SQ

  include/uapi/linux/io_uring.h |  6 ++++++ io_uring/io_uring.c           | 34 +++++++++++++++++++++++++--------- io_uring/io_uring.h           |  3 ++-

  3 files changed, 33 insertions(+), 10 deletions(-)

I like the concept of this, makes a lot of sense. No need to keep churning through the entire SQ ring, when apps mostly submit a few requests at the time. Will help cut down on cacheline usage.

Quoted 7 lines
Quoted 1 line

Curious, do you have any numbers on this for any kind of workload?

No, very likely it's a micro optimisation in the grand picture and would be hard to measure for anything sensible / realistic. It shouldn't be too difficult to come up with a test with a bunch of pinned tasks putting a memory pressure, but that would be as useful as it sounds.

Quoted 3 lines

Simplicity was not the last priority either. E.g. userspace can replace all io_uring_get_sqe() with wrapping the SQ pointer into some array_view at the beginning and work with that.

Yeah agree, it's nice both ways.

Signature · 1 line
Jens Axboe
maintainer
#3 lore raw

Re: [PATCH 2/2] io_uring: introduce non-circular SQ

In reply to Pavel Begunkov

include/uapi/linux/io_uring.h enum io_uring_sqe_flags_bit { · line 238

Quoted 13 lines · Pavel Begunkov
include/uapi/linux/io_uring.h +5 -0
index a0cc1cc0dd01..d1c654a7fa9a 100644
@@ -231,6 +231,12 @@ enum io_uring_sqe_flags_bit {
  */
 #define IORING_SETUP_CQE_MIXED		(1U << 18)
 
+/*
+ * SQEs always start at index 0 in the submission ring instead of using a
+ * wrap around indexing.
+ */
+#define IORING_SETUP_SQ_REWIND		(1U << 19)

No real comments on this patch, very straightforward. I do think this comment should be expanded on considerably, mostly just by adding a comparison to how it's different from the normal non-rewind way of tracking which SQEs to submit. I can do it too, or send a v2.

Signature · 1 line
Jens Axboe
#5 lore raw

Re: [PATCH 2/2] io_uring: introduce non-circular SQ

In reply to Pavel Begunkov
Quoted 35 lines · Pavel Begunkov

Outside of SQPOLL, normally SQ entries are consumed by the time the submission syscall returns. For those cases we don't need a circular buffer and the head/tail tracking, instead the kernel can assume that entries always start from the beginning of the SQ at index 0. This patch introduces a setup flag doing exactly that.

This method is simpler in general, needs fewer operations, doesn't require looking up heads and tails, however, the main goal here is to keep caches hot. The userspace might overprovision SQ, and in the normal way we'd be touching all the cache lines, but with this feature it reuses first entries and keeps them hot. This simplicity will also be quite handy for bpf-io_uring.

To use the feature the user should set the IORING_SETUP_SQ_REWIND flag, and have a compatible liburing/userspace. The flag is restricted to IORING_SETUP_NO_SQARRAY rings and is not compatible with IORING_SETUP_SQPOLL.

Note, it uses relaxed ring synchronisation as the userspace doing the syscall is naturally in sync, and setups that share a SQ should be rolling their own intra process/thread synchronisation.

---
 include/uapi/linux/io_uring.h |  6 ++++++
 io_uring/io_uring.c           | 29 ++++++++++++++++++++++-------
 io_uring/io_uring.h           |  3 ++-
 3 files changed, 30 insertions(+), 8 deletions(-)
include/uapi/linux/io_uring.h +0 -0
index a0cc1cc0dd01..d1c654a7fa9a 100644

include/uapi/linux/io_uring.h enum io_uring_sqe_flags_bit { · line 238

Quoted 8 lines
include/uapi/linux/io_uring.h +5 -0
@@ -231,6 +231,12 @@ enum io_uring_sqe_flags_bit {
  */
 #define IORING_SETUP_CQE_MIXED         (1U << 18)
 
+/*
+ * SQEs always start at index 0 in the submission ring instead of using a
+ * wrap around indexing.
+ */
+#define IORING_SETUP_SQ_REWIND         (1U << 19)

Keith's mixed-SQE-size patch series is already planning to use this
flag: https://lore.kernel.org/io-uring/[email protected]/

Quoted 10 lines
include/uapi/linux/io_uring.h +1 -0
+
 enum io_uring_op {
        IORING_OP_NOP,
        IORING_OP_READV,
io_uring/io_uring.c +0 -0
index ee04ab9bf968..e8af963d3233 100644
@@ -2367,12 +2367,16 @@ static void io_commit_sqring(struct io_ring_ctx *ctx)
 {

io_uring/io_uring.c static void io_commit_sqring(struct io_ring_ctx *ctx) · line 2371

Quoted 10 lines
io_uring/io_uring.c +2 -6
        struct io_rings *rings = ctx->rings;
 
-       /*
-        * Ensure any loads from the SQEs are done at this point,
-        * since once we write the new head, the application could
-        * write new data to them.
-        */
-       smp_store_release(&rings->sq.head, ctx->cached_sq_head);
+       if (ctx->flags & IORING_SETUP_SQ_REWIND) {
+               ctx->cached_sq_head = 0;

The only awkward thing about this interface seems to be if io_submit_sqes() aborts early without submitting all the requested SQEs. Does userspace then need to memmove() the remaining SQEs to the start of the ring? It's certainly an unlikely case but something userspace has to handle because io_alloc_req() can fail for reasons outside its control. Seems like it might simplify the userspace side if cached_sq_head wasn't rewound if not all SQEs were consumed.

Quoted 29 lines
io_uring/io_uring.c +14 -1
+       } else {
+               /*
+                * Ensure any loads from the SQEs are done at this point,
+                * since once we write the new head, the application could
+                * write new data to them.
+                */
+               smp_store_release(&rings->sq.head, ctx->cached_sq_head);
+       }
 }
 
 /*
@@ -2418,10 +2422,15 @@ static bool io_get_sqe(struct io_ring_ctx *ctx, const struct io_uring_sqe **sqe)
 int io_submit_sqes(struct io_ring_ctx *ctx, unsigned int nr)
        __must_hold(&ctx->uring_lock)
 {
-       unsigned int entries = io_sqring_entries(ctx);
+       unsigned int entries;
        unsigned int left;
        int ret;
 
+       if (ctx->flags & IORING_SETUP_SQ_REWIND)
+               entries = ctx->sq_entries;
+       else
+               entries = io_sqring_entries(ctx);
+
        entries = min(nr, entries);
        if (unlikely(!entries))
                return 0;

io_uring/io_uring.c static int io_uring_sanitise_params(struct io_uring_params *p) · line 3692

Quoted 6 lines
io_uring/io_uring.c +3 -0
@@ -3678,6 +3687,12 @@ static int io_uring_sanitise_params(struct io_uring_params *p)
 {
        unsigned flags = p->flags;
 
+       if (flags & IORING_SETUP_SQ_REWIND) {
+               if ((flags & IORING_SETUP_SQPOLL) ||
+                   !(flags & IORING_SETUP_NO_SQARRAY))

Is there a reason IORING_SETUP_NO_SQARRAY is required? It seems like the implementation would work just fine with the SQ indirection ring; the rewind would just apply to the indirection ring instead of the SQE array. The cache hit rate benefit would probably be smaller since many more SQ indirection entries fit in a single cache line, but I don't see a reason to explicitly forbid it.

Best,
Caleb

Quoted 24 lines
io_uring/io_uring.c +3 -0
+               return -EINVAL;
+       }
+
        /* There is no way to mmap rings without a real fd */
        if ((flags & IORING_SETUP_REGISTERED_FD_ONLY) &&
            !(flags & IORING_SETUP_NO_MMAP))
io_uring/io_uring.h +2 -2
index 46d9141d772a..b998ed57dd93 100644
@@ -54,7 +54,8 @@
                        IORING_SETUP_REGISTERED_FD_ONLY |\
                        IORING_SETUP_NO_SQARRAY |\
                        IORING_SETUP_HYBRID_IOPOLL |\
-                       IORING_SETUP_CQE_MIXED)
+                       IORING_SETUP_CQE_MIXED |\
+                       IORING_SETUP_SQ_REWIND)
 
 #define IORING_ENTER_FLAGS (IORING_ENTER_GETEVENTS |\
                        IORING_ENTER_SQ_WAKEUP |\
--

2.49.0

#4 lore raw

Re: [PATCH 2/2] io_uring: introduce non-circular SQ

In reply to Jens Axboe
Quoted 9 lines · Jens Axboe
Quoted 8 lines · Pavel Begunkov
include/uapi/linux/io_uring.h +0 -0
index a0cc1cc0dd01..d1c654a7fa9a 100644
@@ -231,6 +231,12 @@ enum io_uring_sqe_flags_bit {
   */
  #define IORING_SETUP_CQE_MIXED		(1U << 18)
 
Quoted 10 lines

include/uapi/linux/io_uring.h enum io_uring_sqe_flags_bit { · line 238

Quoted 5 lines
include/uapi/linux/io_uring.h +5 -0
+/*
+ * SQEs always start at index 0 in the submission ring instead of using a
+ * wrap around indexing.
+ */
+#define IORING_SETUP_SQ_REWIND		(1U << 19)

No real comments on this patch, very straightforward. I do think this comment should be expanded on considerably, mostly just by adding a comparison to how it's different from the normal non-rewind way of tracking which SQEs to submit. I can do it too, or send a v2.

Please do if that's fine with you. Not entirely sure how you want to put it into words.

Signature · 1 line
Pavel Begunkov
#6 lore raw

Re: [PATCH 2/2] io_uring: introduce non-circular SQ

In reply to Caleb Sander Mateos
Quoted 1 line · Caleb Sander Mateos

On Tue, Oct 14, 2025 at 3:57 AM Pavel Begunkov <[email protected]> wrote:

...>> + * SQEs always start at index 0 in the submission ring instead of using a

Quoted 6 lines
Quoted 3 lines

+ * wrap around indexing.
+ */
+#define IORING_SETUP_SQ_REWIND (1U << 19)

Keith's mixed-SQE-size patch series is already planning to use this
flag: https://lore.kernel.org/io-uring/[email protected]/

I'll rebase it as ususual if that gets merged before.

Quoted 9 lines

io_uring/io_uring.c static void io_commit_sqring(struct io_ring_ctx *ctx) · line 2371

Quoted 8 lines
io_uring/io_uring.c +2 -6
-       /*
-        * Ensure any loads from the SQEs are done at this point,
-        * since once we write the new head, the application could
-        * write new data to them.
-        */
-       smp_store_release(&rings->sq.head, ctx->cached_sq_head);
+       if (ctx->flags & IORING_SETUP_SQ_REWIND) {
+               ctx->cached_sq_head = 0;
Quoted 7 lines

The only awkward thing about this interface seems to be if io_submit_sqes() aborts early without submitting all the requested SQEs. Does userspace then need to memmove() the remaining SQEs to the start of the ring? It's certainly an unlikely case but something userspace has to handle because io_alloc_req() can fail for reasons outside its control. Seems like it might simplify the userspace side if cached_sq_head wasn't rewound if not all SQEs were consumed.

This kind of special rules is what usually makes interfaces a pain to work with. What if you want to abort all un-submitted requests instead? You can empty the queue, but then the next syscall will still start from the middle. Or what if the application wants to queue more requests before resubmitting previous ones? There are reasons b/c the kernel will need to handle it in a less elegant way than it potentially can otherwise. memmove sounds appropriate.

Quoted 4 lines
Quoted 4 lines
io_uring/io_uring.c +0 -0
@@ -3678,6 +3687,12 @@ static int io_uring_sanitise_params(struct io_uring_params *p)
  {
         unsigned flags = p->flags;
 
Quoted 10 lines

io_uring/io_uring.c static int io_uring_sanitise_params(struct io_uring_params *p) · line 3692

Quoted 3 lines
io_uring/io_uring.c +3 -0
+       if (flags & IORING_SETUP_SQ_REWIND) {
+               if ((flags & IORING_SETUP_SQPOLL) ||
+                   !(flags & IORING_SETUP_NO_SQARRAY))

Is there a reason IORING_SETUP_NO_SQARRAY is required? It seems like the implementation would work just fine with the SQ indirection ring; the rewind would just apply to the indirection ring instead of the SQE array. The cache hit rate benefit would probably be smaller since many more SQ indirection entries fit in a single cache line, but I don't see a reason to explicitly forbid it.

B/c I don't care about sqarray setups, they are on the way out for soft deprecation with liburing defaulting to NO_SQARRAY, and once you try to optimise the kernel IORING_SETUP_SQ_REWIND handling it might turn out that !NO_SQARRAY is in the way... or not, but you can always allow it later while limiting it would break uapi. In short, it's weighting chances of (micro) optimisations in the future vs supporting a case which is unlikely going to be used.

Signature · 1 line
Pavel Begunkov
#7 lore raw

Re: [PATCH 2/2] io_uring: introduce non-circular SQ

Quoted 28 lines · Pavel Begunkov
Quoted 1 line · Caleb Sander Mateos

On Tue, Oct 14, 2025 at 3:57 AM Pavel Begunkov <[email protected]> wrote:

...>> + * SQEs always start at index 0 in the submission ring instead of using a

Quoted 6 lines
Quoted 3 lines

+ * wrap around indexing.
+ */
+#define IORING_SETUP_SQ_REWIND (1U << 19)

Keith's mixed-SQE-size patch series is already planning to use this
flag: https://lore.kernel.org/io-uring/[email protected]/

I'll rebase it as ususual if that gets merged before.

Quoted 9 lines

io_uring/io_uring.c static void io_commit_sqring(struct io_ring_ctx *ctx) · line 2371

Quoted 8 lines
io_uring/io_uring.c +2 -6
-       /*
-        * Ensure any loads from the SQEs are done at this point,
-        * since once we write the new head, the application could
-        * write new data to them.
-        */
-       smp_store_release(&rings->sq.head, ctx->cached_sq_head);
+       if (ctx->flags & IORING_SETUP_SQ_REWIND) {
+               ctx->cached_sq_head = 0;
Quoted 7 lines

The only awkward thing about this interface seems to be if io_submit_sqes() aborts early without submitting all the requested SQEs. Does userspace then need to memmove() the remaining SQEs to the start of the ring? It's certainly an unlikely case but something userspace has to handle because io_alloc_req() can fail for reasons outside its control. Seems like it might simplify the userspace side if cached_sq_head wasn't rewound if not all SQEs were consumed.

Quoted 7 lines

This kind of special rules is what usually makes interfaces a pain to work with. What if you want to abort all un-submitted requests instead? You can empty the queue, but then the next syscall will still start from the middle. Or what if the application wants to queue more requests before resubmitting previous ones? There are reasons b/c the kernel will need to handle it in a less elegant way than it potentially can otherwise. memmove sounds appropriate.

Maybe most convenient would be a way for userspace to pass both a head and a nr/tail value to the syscall instead of assuming the head is always 0. But it's probably difficult to modify the existing syscall interface without an indirection to the head value, which seems to be a main point of this series. So always resetting to 0 and requiring userspace to memmove() the remaining SQEs in the rare case that io_uring_enter() doesn't consume all of them seems like a reasonable approach.

Quoted 16 lines
Quoted 5 lines
Quoted 5 lines
io_uring/io_uring.c +0 -0
@@ -3678,6 +3687,12 @@ static int io_uring_sanitise_params(struct io_uring_params *p)
  {
         unsigned flags = p->flags;
 
Quoted 10 lines

io_uring/io_uring.c static int io_uring_sanitise_params(struct io_uring_params *p) · line 3692

Quoted 3 lines
io_uring/io_uring.c +3 -0
+       if (flags & IORING_SETUP_SQ_REWIND) {
+               if ((flags & IORING_SETUP_SQPOLL) ||
+                   !(flags & IORING_SETUP_NO_SQARRAY))

Is there a reason IORING_SETUP_NO_SQARRAY is required? It seems like the implementation would work just fine with the SQ indirection ring; the rewind would just apply to the indirection ring instead of the SQE array. The cache hit rate benefit would probably be smaller since many more SQ indirection entries fit in a single cache line, but I don't see a reason to explicitly forbid it.

Quoted 7 lines

B/c I don't care about sqarray setups, they are on the way out for soft deprecation with liburing defaulting to NO_SQARRAY, and once you try to optimise the kernel IORING_SETUP_SQ_REWIND handling it might turn out that !NO_SQARRAY is in the way... or not, but you can always allow it later while limiting it would break uapi. In short, it's weighting chances of (micro) optimisations in the future vs supporting a case which is unlikely going to be used.

Fair point, tradeoffs either way.

Best,
Caleb

Quoted 4 lines

--
Pavel Begunkov

#8 lore raw

Re: [PATCH 2/2] io_uring: introduce non-circular SQ

Quoted 37 lines · Caleb Sander Mateos
Quoted 28 lines · Pavel Begunkov
Quoted 1 line · Caleb Sander Mateos

On Tue, Oct 14, 2025 at 3:57 AM Pavel Begunkov <[email protected]> wrote:

...>> + * SQEs always start at index 0 in the submission ring instead of using a

Quoted 6 lines
Quoted 3 lines

+ * wrap around indexing.
+ */
+#define IORING_SETUP_SQ_REWIND (1U << 19)

Keith's mixed-SQE-size patch series is already planning to use this
flag: https://lore.kernel.org/io-uring/[email protected]/

I'll rebase it as ususual if that gets merged before.

Quoted 9 lines

io_uring/io_uring.c static void io_commit_sqring(struct io_ring_ctx *ctx) · line 2371

Quoted 8 lines
io_uring/io_uring.c +2 -6
-       /*
-        * Ensure any loads from the SQEs are done at this point,
-        * since once we write the new head, the application could
-        * write new data to them.
-        */
-       smp_store_release(&rings->sq.head, ctx->cached_sq_head);
+       if (ctx->flags & IORING_SETUP_SQ_REWIND) {
+               ctx->cached_sq_head = 0;
Quoted 7 lines

The only awkward thing about this interface seems to be if io_submit_sqes() aborts early without submitting all the requested SQEs. Does userspace then need to memmove() the remaining SQEs to the start of the ring? It's certainly an unlikely case but something userspace has to handle because io_alloc_req() can fail for reasons outside its control. Seems like it might simplify the userspace side if cached_sq_head wasn't rewound if not all SQEs were consumed.

Quoted 7 lines

This kind of special rules is what usually makes interfaces a pain to work with. What if you want to abort all un-submitted requests instead? You can empty the queue, but then the next syscall will still start from the middle. Or what if the application wants to queue more requests before resubmitting previous ones? There are reasons b/c the kernel will need to handle it in a less elegant way than it potentially can otherwise. memmove sounds appropriate.

Quoted 3 lines

Maybe most convenient would be a way for userspace to pass both a head and a nr/tail value to the syscall instead of assuming the head is always 0. But it's probably difficult to modify the existing syscall

It feels fine from the API perspective, but you still need head/tail fetching and care, additional index sanitisation (Spectre), and either handling wrap around or extra border checks. All minor points, but the index handling will likely be more annoying than just doing a memmove.

Quoted 5 lines

interface without an indirection to the head value, which seems to be a main point of this series. So always resetting to 0 and requiring userspace to memmove() the remaining SQEs in the rare case that io_uring_enter() doesn't consume all of them seems like a reasonable approach.

If that's what the user wants to do as there are other ways it could be handled.

Signature · 1 line
Pavel Begunkov