Introduce non circular SQ
cover Introduce non circular SQ
Add a feature that makes the kernel to ignore SQ head/tail and always start fetching SQ entries from index 0, which helps to keep caches hot. See Patch 2 for more details.
liburing support:
https://github.com/isilence/liburing.git sq-rewind
Tested by forcing liburing to enable the flag for compatible setups.
Pavel Begunkov (2):
io_uring: check for user passing 0 nr_submit
io_uring: introduce non-circular SQ
include/uapi/linux/io_uring.h | 6 ++++++ io_uring/io_uring.c | 34 +++++++++++++++++++++++++--------- io_uring/io_uring.h | 3 ++- 3 files changed, 33 insertions(+), 10 deletions(-)
Signature · 1 line
2.49.0
PATCH 1/2 io_uring: check for user passing 0 nr_submit
io_submit_sqes() shouldn't be stepping into its main loop when there is nothing to submit, i.e. nr=0. Fix 0 submission queue entries checks, which should follow after all user input truncations.
--- io_uring/io_uring.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
io_uring/io_uring.c +3 -2
@@ -2422,10 +2422,11 @@ int io_submit_sqes(struct io_ring_ctx *ctx, unsigned int nr)unsigned int left;int ret; + entries =min (nr, entries);if (unlikely (!entries))return 0 ; -/* make sure SQ entry isn't read before tail */ - ret = left =min (nr, entries); + + ret = left = entries;io_get_task_refs (left);io_submit_state_start (&ctx->submit_state, left);
Signature · 1 line
2.49.0
PATCH 2/2 io_uring: introduce non-circular SQ
In reply to Pavel BegunkovOutside of SQPOLL, normally SQ entries are consumed by the time the submission syscall returns. For those cases we don't need a circular buffer and the head/tail tracking, instead the kernel can assume that entries always start from the beginning of the SQ at index 0. This patch introduces a setup flag doing exactly that.
This method is simpler in general, needs fewer operations, doesn't require looking up heads and tails, however, the main goal here is to keep caches hot. The userspace might overprovision SQ, and in the normal way we'd be touching all the cache lines, but with this feature it reuses first entries and keeps them hot. This simplicity will also be quite handy for bpf-io_uring.
To use the feature the user should set the IORING_SETUP_SQ_REWIND flag, and have a compatible liburing/userspace. The flag is restricted to IORING_SETUP_NO_SQARRAY rings and is not compatible with IORING_SETUP_SQPOLL.
Note, it uses relaxed ring synchronisation as the userspace doing the syscall is naturally in sync, and setups that share a SQ should be rolling their own intra process/thread synchronisation.
--- include/uapi/linux/io_uring.h | 6 ++++++ io_uring/io_uring.c | 29 ++++++++++++++++++++++------- io_uring/io_uring.h | 3 ++- 3 files changed, 30 insertions(+), 8 deletions(-)
include/uapi/linux/io_uring.h +6 -0
@@ -231,6 +231,12 @@ enum io_uring_sqe_flags_bit {*/ #define IORING_SETUP_CQE_MIXED (1U << 18) +/* +* SQEs always start at index 0 in the submission ring instead of using a +* wrap around indexing. +*/ +#define IORING_SETUP_SQ_REWIND (1U << 19) +enum io_uring_op {IORING_OP_NOP ,IORING_OP_READV ,
io_uring/io_uring.c +22 -7
@@ -2367,12 +2367,16 @@ static void io_commit_sqring(struct io_ring_ctx *ctx) {struct io_rings *rings = ctx->rings; -/* -* Ensure any loads from the SQEs are done at this point, -* since once we write the new head, the application could -* write new data to them. -*/ -smp_store_release (&rings->sq.head, ctx->cached_sq_head); +if (ctx->flags &IORING_SETUP_SQ_REWIND ) { + ctx->cached_sq_head =0 ; + }else { +/* +* Ensure any loads from the SQEs are done at this point, +* since once we write the new head, the application could +* write new data to them. +*/ +smp_store_release (&rings->sq.head, ctx->cached_sq_head); + } }/*
@@ -2418,10 +2422,15 @@ static bool io_get_sqe(struct io_ring_ctx *ctx, const struct io_uring_sqe **sqe)int io_submit_sqes (struct io_ring_ctx *ctx,unsigned int nr) __must_hold(&ctx->uring_lock) { -unsigned int entries =io_sqring_entries (ctx); +unsigned int entries;unsigned int left;int ret; +if (ctx->flags &IORING_SETUP_SQ_REWIND ) + entries = ctx->sq_entries; +else + entries =io_sqring_entries (ctx); + entries =min (nr, entries);if (unlikely (!entries))return 0 ;
@@ -3678,6 +3687,12 @@ static int io_uring_sanitise_params(struct io_uring_params *p) {unsigned flags = p->flags; +if (flags &IORING_SETUP_SQ_REWIND ) { +if ((flags &IORING_SETUP_SQPOLL ) || + !(flags &IORING_SETUP_NO_SQARRAY )) +return -EINVAL ; + } +/* There is no way to mmap rings without a real fd */ if ((flags &IORING_SETUP_REGISTERED_FD_ONLY ) && !(flags &IORING_SETUP_NO_MMAP ))
io_uring/io_uring.h +2 -1
@@ -54,7 +54,8 @@ IORING_SETUP_REGISTERED_FD_ONLY |\ IORING_SETUP_NO_SQARRAY |\ IORING_SETUP_HYBRID_IOPOLL |\ - IORING_SETUP_CQE_MIXED) + IORING_SETUP_CQE_MIXED |\ + IORING_SETUP_SQ_REWIND)#define IORING_ENTER_FLAGS (IORING_ENTER_GETEVENTS |\ IORING_ENTER_SQ_WAKEUP |\
Signature · 1 line
2.49.0
Jens Axboe
maintainerQuoted 9 lines · Pavel Begunkov
Add a feature that makes the kernel to ignore SQ head/tail and always start fetching SQ entries from index 0, which helps to keep caches hot. See Patch 2 for more details.
liburing support:
https://github.com/isilence/liburing.git sq-rewindTested by forcing liburing to enable the flag for compatible setups.
Quoted 8 lines
Pavel Begunkov (2):
io_uring: check for user passing 0nr_submit
io_uring: introduce non-circular SQinclude/uapi/linux/io_uring.h | 6 ++++++ io_uring/io_uring.c | 34 +++++++++++++++++++++++++--------- io_uring/io_uring.h | 3 ++- 3 files changed, 33 insertions(+), 10 deletions(-)
I like the concept of this, makes a lot of sense. No need to keep churning through the entire SQ ring, when apps mostly submit a few requests at the time. Will help cut down on cacheline usage.
Curious, do you have any numbers on this for any kind of workload?
Signature · 1 line
Jens Axboe
Quoted 19 lines · Jens Axboe
Quoted 9 lines · Pavel Begunkov
Add a feature that makes the kernel to ignore SQ head/tail and always start fetching SQ entries from index 0, which helps to keep caches hot. See Patch 2 for more details.
liburing support:
https://github.com/isilence/liburing.git sq-rewindTested by forcing liburing to enable the flag for compatible setups.
Quoted 8 lines
Pavel Begunkov (2):
io_uring: check for user passing 0nr_submit
io_uring: introduce non-circular SQinclude/uapi/linux/io_uring.h | 6 ++++++ io_uring/io_uring.c | 34 +++++++++++++++++++++++++--------- io_uring/io_uring.h | 3 ++- 3 files changed, 33 insertions(+), 10 deletions(-)
Quoted 5 lines
I like the concept of this, makes a lot of sense. No need to keep churning through the entire SQ ring, when apps mostly submit a few requests at the time. Will help cut down on cacheline usage.
Curious, do you have any numbers on this for any kind of workload?
No, very likely it's a micro optimisation in the grand picture and would be hard to measure for anything sensible / realistic. It shouldn't be too difficult to come up with a test with a bunch of pinned tasks putting a memory pressure, but that would be as useful as it sounds.
Signature · 1 line
Pavel Begunkov
Quoted 24 lines · Pavel Begunkov
Quoted 23 lines · Jens Axboe
Quoted 9 lines · Pavel Begunkov
Add a feature that makes the kernel to ignore SQ head/tail and always start fetching SQ entries from index 0, which helps to keep caches hot. See Patch 2 for more details.
liburing support:
https://github.com/isilence/liburing.git sq-rewindTested by forcing liburing to enable the flag for compatible setups.
Quoted 8 lines
Pavel Begunkov (2):
io_uring: check for user passing 0nr_submit
io_uring: introduce non-circular SQ
include/uapi/linux/io_uring.h| 6 ++++++ io_uring/io_uring.c | 34 +++++++++++++++++++++++++--------- io_uring/io_uring.h | 3 ++-3 files changed, 33 insertions(+), 10 deletions(-)I like the concept of this, makes a lot of sense. No need to keep churning through the entire SQ ring, when apps mostly submit a few requests at the time. Will help cut down on cacheline usage.
Quoted 7 lines
Quoted 1 line
Curious, do you have any numbers on this for any kind of workload?
No, very likely it's a micro optimisation in the grand picture and would be hard to measure for anything sensible / realistic. It shouldn't be too difficult to come up with a test with a bunch of pinned tasks putting a memory pressure, but that would be as useful as it sounds.
Simplicity was not the last priority either. E.g. userspace can replace all io_uring_get_sqe() with wrapping the SQ pointer into some array_view at the beginning and work with that.
Signature · 1 line
Pavel Begunkov
Jens Axboe
maintainerQuoted 33 lines · Pavel Begunkov
Quoted 24 lines · Pavel Begunkov
Quoted 23 lines · Jens Axboe
Quoted 9 lines · Pavel Begunkov
Add a feature that makes the kernel to ignore SQ head/tail and always start fetching SQ entries from index 0, which helps to keep caches hot. See Patch 2 for more details.
liburing support:
https://github.com/isilence/liburing.git sq-rewindTested by forcing liburing to enable the flag for compatible setups.
Quoted 8 lines
Pavel Begunkov (2):
io_uring: check for user passing 0nr_submit
io_uring: introduce non-circular SQ
include/uapi/linux/io_uring.h| 6 ++++++ io_uring/io_uring.c | 34 +++++++++++++++++++++++++--------- io_uring/io_uring.h | 3 ++-3 files changed, 33 insertions(+), 10 deletions(-)I like the concept of this, makes a lot of sense. No need to keep churning through the entire SQ ring, when apps mostly submit a few requests at the time. Will help cut down on cacheline usage.
Quoted 7 lines
Quoted 1 line
Curious, do you have any numbers on this for any kind of workload?
No, very likely it's a micro optimisation in the grand picture and would be hard to measure for anything sensible / realistic. It shouldn't be too difficult to come up with a test with a bunch of pinned tasks putting a memory pressure, but that would be as useful as it sounds.
Quoted 3 lines
Simplicity was not the last priority either. E.g. userspace can replace all
io_uring_get_sqe()with wrapping the SQ pointer into somearray_viewat the beginning and work with that.
Yeah agree, it's nice both ways.
Signature · 1 line
Jens Axboe
Jens Axboe
maintainerRe: [PATCH 2/2] io_uring: introduce non-circular SQ
In reply to Pavel Begunkovinclude/uapi/linux/io_uring.h enum io_uring_sqe_flags_bit { · line 238
Quoted 13 lines · Pavel Begunkov
include/uapi/linux/io_uring.h +5
-0@@ -231,6 +231,12 @@ enum io_uring_sqe_flags_bit {*/ #define IORING_SETUP_CQE_MIXED (1U << 18) +/* +* SQEs always start at index 0 in the submission ring instead of using a +* wrap around indexing. +*/ +#define IORING_SETUP_SQ_REWIND (1U << 19)
No real comments on this patch, very straightforward. I do think this comment should be expanded on considerably, mostly just by adding a comparison to how it's different from the normal non-rewind way of tracking which SQEs to submit. I can do it too, or send a v2.
Signature · 1 line
Jens Axboe
Re: [PATCH 2/2] io_uring: introduce non-circular SQ
In reply to Pavel BegunkovQuoted 35 lines · Pavel Begunkov
Outside of SQPOLL, normally SQ entries are consumed by the time the submission syscall returns. For those cases we don't need a circular buffer and the head/tail tracking, instead the kernel can assume that entries always start from the beginning of the SQ at index 0. This patch introduces a setup flag doing exactly that.
This method is simpler in general, needs fewer operations, doesn't require looking up heads and tails, however, the main goal here is to keep caches hot. The userspace might overprovision SQ, and in the normal way we'd be touching all the cache lines, but with this feature it reuses first entries and keeps them hot. This simplicity will also be quite handy for bpf-io_uring.
To use the feature the user should set the
IORING_SETUP_SQ_REWINDflag, and have a compatible liburing/userspace. The flag is restricted toIORING_SETUP_NO_SQARRAYrings and is not compatible withIORING_SETUP_SQPOLL.Note, it uses relaxed ring synchronisation as the userspace doing the syscall is naturally in sync, and setups that share a SQ should be rolling their own intra process/thread synchronisation.
Signed-off-by Pavel Begunkov <[email protected]>--- include/uapi/linux/io_uring.h | 6 ++++++ io_uring/io_uring.c | 29 ++++++++++++++++++++++------- io_uring/io_uring.h | 3 ++- 3 files changed, 30 insertions(+), 8 deletions(-)include/uapi/linux/io_uring.h +0
-0
include/uapi/linux/io_uring.h enum io_uring_sqe_flags_bit { · line 238
Quoted 8 lines
include/uapi/linux/io_uring.h +5
-0@@ -231,6 +231,12 @@ enum io_uring_sqe_flags_bit {*/ #define IORING_SETUP_CQE_MIXED (1U << 18) +/* +* SQEs always start at index 0 in the submission ring instead of using a +* wrap around indexing. +*/ +#define IORING_SETUP_SQ_REWIND (1U << 19)
Keith's mixed-SQE-size patch series is already planning to use this
flag: https://lore.kernel.org/io-uring/[email protected]/
Quoted 10 lines
include/uapi/linux/io_uring.h +1
-0+enum io_uring_op {IORING_OP_NOP ,IORING_OP_READV ,io_uring/io_uring.c +0
-0@@ -2367,12 +2367,16 @@ static void io_commit_sqring(struct io_ring_ctx *ctx) {
io_uring/io_uring.c static void io_commit_sqring(struct io_ring_ctx *ctx) · line 2371
Quoted 10 lines
io_uring/io_uring.c +2
-6struct io_rings *rings = ctx->rings; -/* -* Ensure any loads from the SQEs are done at this point, -* since once we write the new head, the application could -* write new data to them. -*/ -smp_store_release (&rings->sq.head, ctx->cached_sq_head); +if (ctx->flags &IORING_SETUP_SQ_REWIND ) { + ctx->cached_sq_head =0 ;
The only awkward thing about this interface seems to be if io_submit_sqes() aborts early without submitting all the requested SQEs. Does userspace then need to memmove() the remaining SQEs to the start of the ring? It's certainly an unlikely case but something userspace has to handle because io_alloc_req() can fail for reasons outside its control. Seems like it might simplify the userspace side if cached_sq_head wasn't rewound if not all SQEs were consumed.
Quoted 29 lines
io_uring/io_uring.c +14
-1+ }else { +/* +* Ensure any loads from the SQEs are done at this point, +* since once we write the new head, the application could +* write new data to them. +*/ +smp_store_release (&rings->sq.head, ctx->cached_sq_head); + } }/* @@ -2418,10 +2422,15 @@ static bool io_get_sqe(struct io_ring_ctx *ctx, const struct io_uring_sqe **sqe)int io_submit_sqes (struct io_ring_ctx *ctx,unsigned int nr) __must_hold(&ctx->uring_lock) { -unsigned int entries =io_sqring_entries (ctx); +unsigned int entries;unsigned int left;int ret; +if (ctx->flags &IORING_SETUP_SQ_REWIND ) + entries = ctx->sq_entries; +else + entries =io_sqring_entries (ctx); + entries =min (nr, entries);if (unlikely (!entries))return 0 ;
io_uring/io_uring.c static int io_uring_sanitise_params(struct io_uring_params *p) · line 3692
Quoted 6 lines
io_uring/io_uring.c +3
-0@@ -3678,6 +3687,12 @@ static int io_uring_sanitise_params(struct io_uring_params *p) {unsigned flags = p->flags; +if (flags &IORING_SETUP_SQ_REWIND ) { +if ((flags &IORING_SETUP_SQPOLL ) || + !(flags &IORING_SETUP_NO_SQARRAY ))
Is there a reason IORING_SETUP_NO_SQARRAY is required? It seems like the implementation would work just fine with the SQ indirection ring; the rewind would just apply to the indirection ring instead of the SQE array. The cache hit rate benefit would probably be smaller since many more SQ indirection entries fit in a single cache line, but I don't see a reason to explicitly forbid it.
Best,
Caleb
Quoted 24 lines
io_uring/io_uring.c +3
-0+return -EINVAL ; + } +/* There is no way to mmap rings without a real fd */ if ((flags &IORING_SETUP_REGISTERED_FD_ONLY ) && !(flags &IORING_SETUP_NO_MMAP ))io_uring/io_uring.h +2
-2@@ -54,7 +54,8 @@ IORING_SETUP_REGISTERED_FD_ONLY |\ IORING_SETUP_NO_SQARRAY |\ IORING_SETUP_HYBRID_IOPOLL |\ - IORING_SETUP_CQE_MIXED) + IORING_SETUP_CQE_MIXED |\ + IORING_SETUP_SQ_REWIND)#define IORING_ENTER_FLAGS (IORING_ENTER_GETEVENTS |\ IORING_ENTER_SQ_WAKEUP |\ --2.49.0
Re: [PATCH 2/2] io_uring: introduce non-circular SQ
In reply to Jens AxboeQuoted 9 lines · Jens Axboe
Quoted 8 lines · Pavel Begunkov
include/uapi/linux/io_uring.h +0
-0@@ -231,6 +231,12 @@ enum io_uring_sqe_flags_bit {*/ #define IORING_SETUP_CQE_MIXED (1U << 18)
Quoted 10 lines
include/uapi/linux/io_uring.h enum io_uring_sqe_flags_bit { · line 238
Quoted 5 lines
include/uapi/linux/io_uring.h +5
-0+/* +* SQEs always start at index 0 in the submission ring instead of using a +* wrap around indexing. +*/ +#define IORING_SETUP_SQ_REWIND (1U << 19)No real comments on this patch, very straightforward. I do think this comment should be expanded on considerably, mostly just by adding a comparison to how it's different from the normal non-rewind way of tracking which SQEs to submit. I can do it too, or send a v2.
Please do if that's fine with you. Not entirely sure how you want to put it into words.
Signature · 1 line
Pavel Begunkov
Re: [PATCH 2/2] io_uring: introduce non-circular SQ
In reply to Caleb Sander MateosQuoted 1 line · Caleb Sander Mateos
On Tue, Oct 14, 2025 at 3:57 AM Pavel Begunkov <[email protected]> wrote:
...>> + * SQEs always start at index 0 in the submission ring instead of using a
Quoted 6 lines
Quoted 3 lines
+ * wrap around indexing.
+ */
+#defineIORING_SETUP_SQ_REWIND(1U << 19)Keith's mixed-SQE-size patch series is already planning to use this
flag: https://lore.kernel.org/io-uring/[email protected]/
I'll rebase it as ususual if that gets merged before.
Quoted 9 lines
io_uring/io_uring.c static void io_commit_sqring(struct io_ring_ctx *ctx) · line 2371
Quoted 8 lines
io_uring/io_uring.c +2
-6-/* -* Ensure any loads from the SQEs are done at this point, -* since once we write the new head, the application could -* write new data to them. -*/ -smp_store_release (&rings->sq.head, ctx->cached_sq_head); +if (ctx->flags &IORING_SETUP_SQ_REWIND ) { + ctx->cached_sq_head =0 ;
Quoted 7 lines
The only awkward thing about this interface seems to be if
io_submit_sqes()aborts early without submitting all the requested SQEs. Does userspace then need tomemmove()the remaining SQEs to the start of the ring? It's certainly an unlikely case but something userspace has to handle becauseio_alloc_req()can fail for reasons outside its control. Seems like it might simplify the userspace side ifcached_sq_headwasn't rewound if not all SQEs were consumed.
This kind of special rules is what usually makes interfaces a pain to work with. What if you want to abort all un-submitted requests instead? You can empty the queue, but then the next syscall will still start from the middle. Or what if the application wants to queue more requests before resubmitting previous ones? There are reasons b/c the kernel will need to handle it in a less elegant way than it potentially can otherwise. memmove sounds appropriate.
Quoted 4 lines
Quoted 4 lines
io_uring/io_uring.c +0
-0@@ -3678,6 +3687,12 @@ static int io_uring_sanitise_params(struct io_uring_params *p) {unsigned flags = p->flags;
Quoted 10 lines
io_uring/io_uring.c static int io_uring_sanitise_params(struct io_uring_params *p) · line 3692
Quoted 3 lines
io_uring/io_uring.c +3
-0+if (flags &IORING_SETUP_SQ_REWIND ) { +if ((flags &IORING_SETUP_SQPOLL ) || + !(flags &IORING_SETUP_NO_SQARRAY ))Is there a reason
IORING_SETUP_NO_SQARRAYis required? It seems like the implementation would work just fine with the SQ indirection ring; the rewind would just apply to the indirection ring instead of the SQE array. The cache hit rate benefit would probably be smaller since many more SQ indirection entries fit in a single cache line, but I don't see a reason to explicitly forbid it.
B/c I don't care about sqarray setups, they are on the way out for soft deprecation with liburing defaulting to NO_SQARRAY, and once you try to optimise the kernel IORING_SETUP_SQ_REWIND handling it might turn out that !NO_SQARRAY is in the way... or not, but you can always allow it later while limiting it would break uapi. In short, it's weighting chances of (micro) optimisations in the future vs supporting a case which is unlikely going to be used.
Signature · 1 line
Pavel Begunkov
Re: [PATCH 2/2] io_uring: introduce non-circular SQ
Quoted 28 lines · Pavel Begunkov
Quoted 1 line · Caleb Sander Mateos
On Tue, Oct 14, 2025 at 3:57 AM Pavel Begunkov <[email protected]> wrote:
...>> + * SQEs always start at index 0 in the submission ring instead of using a
Quoted 6 lines
Quoted 3 lines
+ * wrap around indexing.
+ */
+#defineIORING_SETUP_SQ_REWIND(1U << 19)Keith's mixed-SQE-size patch series is already planning to use this
flag: https://lore.kernel.org/io-uring/[email protected]/I'll rebase it as ususual if that gets merged before.
Quoted 9 lines
io_uring/io_uring.c static void io_commit_sqring(struct io_ring_ctx *ctx) · line 2371
Quoted 8 lines
io_uring/io_uring.c +2
-6-/* -* Ensure any loads from the SQEs are done at this point, -* since once we write the new head, the application could -* write new data to them. -*/ -smp_store_release (&rings->sq.head, ctx->cached_sq_head); +if (ctx->flags &IORING_SETUP_SQ_REWIND ) { + ctx->cached_sq_head =0 ;Quoted 7 lines
The only awkward thing about this interface seems to be if
io_submit_sqes()aborts early without submitting all the requested SQEs. Does userspace then need tomemmove()the remaining SQEs to the start of the ring? It's certainly an unlikely case but something userspace has to handle becauseio_alloc_req()can fail for reasons outside its control. Seems like it might simplify the userspace side ifcached_sq_headwasn't rewound if not all SQEs were consumed.
Quoted 7 lines
This kind of special rules is what usually makes interfaces a pain to work with. What if you want to abort all un-submitted requests instead? You can empty the queue, but then the next syscall will still start from the middle. Or what if the application wants to queue more requests before resubmitting previous ones? There are reasons b/c the kernel will need to handle it in a less elegant way than it potentially can otherwise. memmove sounds appropriate.
Maybe most convenient would be a way for userspace to pass both a head and a nr/tail value to the syscall instead of assuming the head is always 0. But it's probably difficult to modify the existing syscall interface without an indirection to the head value, which seems to be a main point of this series. So always resetting to 0 and requiring userspace to memmove() the remaining SQEs in the rare case that io_uring_enter() doesn't consume all of them seems like a reasonable approach.
Quoted 16 lines
Quoted 5 lines
Quoted 5 lines
io_uring/io_uring.c +0
-0@@ -3678,6 +3687,12 @@ static int io_uring_sanitise_params(struct io_uring_params *p) {unsigned flags = p->flags;Quoted 10 lines
io_uring/io_uring.c static int io_uring_sanitise_params(struct io_uring_params *p) · line 3692
Quoted 3 lines
io_uring/io_uring.c +3
-0+if (flags &IORING_SETUP_SQ_REWIND ) { +if ((flags &IORING_SETUP_SQPOLL ) || + !(flags &IORING_SETUP_NO_SQARRAY ))Is there a reason
IORING_SETUP_NO_SQARRAYis required? It seems like the implementation would work just fine with the SQ indirection ring; the rewind would just apply to the indirection ring instead of the SQE array. The cache hit rate benefit would probably be smaller since many more SQ indirection entries fit in a single cache line, but I don't see a reason to explicitly forbid it.
Quoted 7 lines
B/c I don't care about sqarray setups, they are on the way out for soft deprecation with liburing defaulting to
NO_SQARRAY, and once you try to optimise the kernelIORING_SETUP_SQ_REWINDhandling it might turn out that !NO_SQARRAY is in the way... or not, but you can always allow it later while limiting it would break uapi. In short, it's weighting chances of (micro) optimisations in the future vs supporting a case which is unlikely going to be used.
Fair point, tradeoffs either way.
Best,
Caleb
Quoted 4 lines
--
Pavel Begunkov
Re: [PATCH 2/2] io_uring: introduce non-circular SQ
Quoted 37 lines · Caleb Sander Mateos
Quoted 28 lines · Pavel Begunkov
Quoted 1 line · Caleb Sander Mateos
On Tue, Oct 14, 2025 at 3:57 AM Pavel Begunkov <[email protected]> wrote:
...>> + * SQEs always start at index 0 in the submission ring instead of using a
Quoted 6 lines
Quoted 3 lines
+ * wrap around indexing.
+ */
+#defineIORING_SETUP_SQ_REWIND(1U << 19)Keith's mixed-SQE-size patch series is already planning to use this
flag: https://lore.kernel.org/io-uring/[email protected]/I'll rebase it as ususual if that gets merged before.
Quoted 9 lines
io_uring/io_uring.c static void io_commit_sqring(struct io_ring_ctx *ctx) · line 2371
Quoted 8 lines
io_uring/io_uring.c +2
-6-/* -* Ensure any loads from the SQEs are done at this point, -* since once we write the new head, the application could -* write new data to them. -*/ -smp_store_release (&rings->sq.head, ctx->cached_sq_head); +if (ctx->flags &IORING_SETUP_SQ_REWIND ) { + ctx->cached_sq_head =0 ;Quoted 7 lines
The only awkward thing about this interface seems to be if
io_submit_sqes()aborts early without submitting all the requested SQEs. Does userspace then need tomemmove()the remaining SQEs to the start of the ring? It's certainly an unlikely case but something userspace has to handle becauseio_alloc_req()can fail for reasons outside its control. Seems like it might simplify the userspace side ifcached_sq_headwasn't rewound if not all SQEs were consumed.Quoted 7 lines
This kind of special rules is what usually makes interfaces a pain to work with. What if you want to abort all un-submitted requests instead? You can empty the queue, but then the next syscall will still start from the middle. Or what if the application wants to queue more requests before resubmitting previous ones? There are reasons b/c the kernel will need to handle it in a less elegant way than it potentially can otherwise. memmove sounds appropriate.
Quoted 3 lines
Maybe most convenient would be a way for userspace to pass both a head and a nr/tail value to the syscall instead of assuming the head is always 0. But it's probably difficult to modify the existing syscall
It feels fine from the API perspective, but you still need head/tail fetching and care, additional index sanitisation (Spectre), and either handling wrap around or extra border checks. All minor points, but the index handling will likely be more annoying than just doing a memmove.
Quoted 5 lines
interface without an indirection to the head value, which seems to be a main point of this series. So always resetting to 0 and requiring userspace to
memmove()the remaining SQEs in the rare case thatio_uring_enter()doesn't consume all of them seems like a reasonable approach.
If that's what the user wants to do as there are other ways it could be handled.
Signature · 1 line
Pavel Begunkov