Add support for IORING_SETUP_SQ_REWIND
cover Add support for IORING_SETUP_SQ_REWIND
Add liburing support and tests for IORING_SETUP_SQ_REWIND.
Pavel Begunkov (2):
src/queue: Add support for non circular SQ
tests: add SETUP_SQ_REWIND tests
src/include/liburing.h | 5 ++++- src/include/liburing/io_uring.h | 12 ++++++++++++ src/queue.c | 5 +++++ test/test.h | 2 ++ 4 files changed, 23 insertions(+), 1 deletion(-)
Signature · 1 line
2.52.0
PATCH 1/2 src/queue: Add support for non circular SQ
With IORING_SETUP_SQ_REWIND, the kernel doesn't use the SQ headers and always submits SQEs with indices in a range [0, nr_submit), where nr_submit is the syscall argument. Add liburing support for that. The head is now kept to be 0, and tail effectively tracks the number of entries to submit, which matches the normal formula to_submit = (tail - head) mod N
--- src/include/liburing.h | 5 ++++- src/include/liburing/io_uring.h | 12 ++++++++++++ src/queue.c | 5 +++++ 3 files changed, 21 insertions(+), 1 deletion(-)
src/include/liburing.h +4 -1
@@ -1918,9 +1918,12 @@ IOURINGINLINE struct io_uring_sqe *_io_uring_get_sqe(struct io_uring *ring) LIBURING_NOEXCEPT {struct io_uring_sq *sq = &ring->sq; -unsigned head =io_uring_load_sq_head (ring), tail = sq->sqe_tail; +unsigned head =0 , tail = sq->sqe_tail;struct io_uring_sqe *sqe; +if (!(ring->flags &IORING_SETUP_SQ_REWIND )) + head =io_uring_load_sq_head (ring); +if (tail - head >= sq->ring_entries)return NULL ;
src/include/liburing/io_uring.h +12 -0
@@ -233,6 +233,18 @@ enum io_uring_sqe_flags_bit {*/ #define IORING_SETUP_SQE_MIXED (1U << 19) +/* +* When set, io_uring ignores SQ head and tail and fetches SQEs to submit +* starting from index 0 instead from the index stored in the head pointer. +* IOW, the user should place all SQE at the beginning of the SQ memory +* before issuing a submission syscall. +* +* It requires IORING_SETUP_NO_SQARRAY and is incompatible with +* IORING_SETUP_SQPOLL. The user must also never change the SQ head and tail +* values and keep it set to 0. Any other value is undefined behaviour. +*/ +#define IORING_SETUP_SQ_REWIND (1U << 20) +enum io_uring_op {IORING_OP_NOP ,IORING_OP_READV ,
src/queue.c +5 -0
@@ -205,6 +205,11 @@ static unsigned __io_uring_flush_sq(struct io_uring *ring)struct io_uring_sq *sq = &ring->sq;unsigned tail = sq->sqe_tail; +if (ring->flags &IORING_SETUP_SQ_REWIND ) { + sq->sqe_tail =0 ; +return tail; + } +if (sq->sqe_head != tail) { sq->sqe_head = tail;/*
Signature · 1 line
2.52.0
PATCH 2/2 tests: add SETUP_SQ_REWIND tests
In reply to Pavel BegunkovAdd a couple of IORING_SETUP_SQ_REWIND configurations to the nop test.
--- test/test.h | 2 ++ 1 file changed, 2 insertions(+)
test/test.h +2 -0
@@ -20,6 +20,8 @@ static io_uring_test_config io_uring_test_configs[] = { { IORING_SETUP_SQE128, "large SQE"}, { IORING_SETUP_CQE32, "large CQE"}, { IORING_SETUP_SQE128 | IORING_SETUP_CQE32, "large SQE/CQE" }, + { IORING_SETUP_SQ_REWIND, "rewind SQ"}, + { IORING_SETUP_SQ_REWIND | IORING_SETUP_SQE128, "large rewind SQ"}, };#define FOR_ALL_TEST_CONFIGS \
Signature · 1 line
2.52.0
Quoted 6 lines · Pavel Begunkov
Add liburing support and tests for
IORING_SETUP_SQ_REWIND.Pavel Begunkov (2):
src/queue: Add support for non circular SQ
tests: addSETUP_SQ_REWINDtests
Quoted 7 lines
src/include/liburing.h| 5 ++++-src/include/liburing/io_uring.h | 12 ++++++++++++ src/queue.c | 5 +++++ test/test.h | 2 ++ 4 files changed, 23 insertions(+), 1 deletion(-)[...]
Applied, thanks!
[1/2] src/queue: Add support for non circular SQ
commit: c22129cf0b8c936eb478d920ef84e53d89c6a5cc
[2/2] tests: add SETUP_SQ_REWIND tests
commit: 346c063d16bda52f02d00feb744aafe35b4002a9Best regards,
Signature · 1 line
Jens Axboe
Jens Axboe
maintainerQuoted 14 lines · Jens Axboe
Quoted 12 lines · Pavel Begunkov
Add liburing support and tests for
IORING_SETUP_SQ_REWIND.Pavel Begunkov (2):
src/queue: Add support for non circular SQ
tests: addSETUP_SQ_REWINDtests
src/include/liburing.h| 5 ++++-src/include/liburing/io_uring.h | 12 ++++++++++++ src/queue.c | 5 +++++ test/test.h | 2 ++ 4 files changed, 23 insertions(+), 1 deletion(-)
Quoted 8 lines
Quoted 1 line
[...]
Applied, thanks!
[1/2] src/queue: Add support for non circular SQ
commit: c22129cf0b8c936eb478d920ef84e53d89c6a5cc [2/2] tests: add SETUP_SQ_REWIND tests commit: 346c063d16bda52f02d00feb744aafe35b4002a9
Hmm I do think you're missing some spots though, no?
src/include/liburing.h +10 -2
@@ -1702,8 +1702,13 @@ IOURINGINLINE unsigned io_uring_load_sq_head(const struct io_uring *ring)IOURINGINLINE unsigned io_uring_sq_ready (const struct io_uring *ring)LIBURING_NOEXCEPT { +unsigned head =0 ; + +if (!(ring->flags &IORING_SETUP_SQ_REWIND )) + head =io_uring_load_sq_head (ring); +/* always use real head, to avoid losing sync for short submit */ -return ring->sq.sqe_tail -io_uring_load_sq_head (ring); +return ring->sq.sqe_tail - head; }/*
@@ -2063,7 +2068,7 @@ IOURINGINLINE struct io_uring_sqe *io_uring_get_sqe128(struct io_uring *ring) LIBURING_NOEXCEPT {struct io_uring_sq *sq = &ring->sq; -unsigned head =io_uring_load_sq_head (ring), tail = sq->sqe_tail; +unsigned head =0 , tail = sq->sqe_tail;struct io_uring_sqe *sqe;if (ring->flags &IORING_SETUP_SQE128 )
@@ -2071,6 +2076,9 @@ IOURINGINLINE struct io_uring_sqe *io_uring_get_sqe128(struct io_uring *ring)if (!(ring->flags &IORING_SETUP_SQE_MIXED ))return NULL ; +if (!(ring->flags &IORING_SETUP_SQ_REWIND )) + head =io_uring_load_sq_head (ring); +if (((tail +1 ) & sq->ring_mask) ==0 ) {if ((tail +2 ) - head >= sq->ring_entries)return NULL ;
Signature · 1 line
Jens Axboe
Quoted 31 lines · Jens Axboe
Quoted 14 lines · Jens Axboe
Quoted 12 lines · Pavel Begunkov
Add liburing support and tests for
IORING_SETUP_SQ_REWIND.Pavel Begunkov (2):
src/queue: Add support for non circular SQ
tests: addSETUP_SQ_REWINDtests
src/include/liburing.h| 5 ++++-src/include/liburing/io_uring.h | 12 ++++++++++++ src/queue.c | 5 +++++ test/test.h | 2 ++ 4 files changed, 23 insertions(+), 1 deletion(-)Quoted 8 lines
Quoted 1 line
[...]
Applied, thanks!
[1/2] src/queue: Add support for non circular SQ
commit: c22129cf0b8c936eb478d920ef84e53d89c6a5cc [2/2] tests: add SETUP_SQ_REWIND tests commit: 346c063d16bda52f02d00feb744aafe35b4002a9Hmm I do think you're missing some spots though, no?
src/include/liburing.h +0
-0
src/include/liburing.h IOURINGINLINE unsigned io_uring_load_sq_head(const struct io_uring *ring) · line 1708
Quoted 7 lines
src/include/liburing.h +4
-0@@ -1702,8 +1702,13 @@ IOURINGINLINE unsigned io_uring_load_sq_head(const struct io_uring *ring)IOURINGINLINE unsigned io_uring_sq_ready (const struct io_uring *ring)LIBURING_NOEXCEPT { +unsigned head =0 ; + +if (!(ring->flags &IORING_SETUP_SQ_REWIND )) + head =io_uring_load_sq_head (ring);
The head should already be zero. Actually, sounds like the get_sqe hunk from the patch is not needed either.
Signature · 1 line
Pavel Begunkov
Jens Axboe
maintainerQuoted 27 lines · Pavel Begunkov
Quoted 26 lines · Jens Axboe
Quoted 14 lines · Jens Axboe
Quoted 12 lines · Pavel Begunkov
Add liburing support and tests for
IORING_SETUP_SQ_REWIND.Pavel Begunkov (2):
src/queue: Add support for non circular SQ
tests: addSETUP_SQ_REWINDtests
src/include/liburing.h| 5 ++++-src/include/liburing/io_uring.h | 12 ++++++++++++ src/queue.c | 5 +++++ test/test.h | 2 ++ 4 files changed, 23 insertions(+), 1 deletion(-)Quoted 8 lines
Quoted 1 line
[...]
Applied, thanks!
[1/2] src/queue: Add support for non circular SQ
commit: c22129cf0b8c936eb478d920ef84e53d89c6a5cc [2/2] tests: add SETUP_SQ_REWIND tests commit: 346c063d16bda52f02d00feb744aafe35b4002a9Hmm I do think you're missing some spots though, no?
Quoted 15 lines
src/include/liburing.h IOURINGINLINE unsigned io_uring_load_sq_head(const struct io_uring *ring) · line 1708
Quoted 12 lines
src/include/liburing.h +4
-0@@ -1702,8 +1702,13 @@ IOURINGINLINE unsigned io_uring_load_sq_head(const struct io_uring *ring)IOURINGINLINE unsigned io_uring_sq_ready (const struct io_uring *ring)LIBURING_NOEXCEPT { +unsigned head =0 ; + +if (!(ring->flags &IORING_SETUP_SQ_REWIND )) + head =io_uring_load_sq_head (ring);The head should already be zero. Actually, sounds like the
get_sqehunk from the patch is not needed either.
Yeah agree, I think they are both false alarms. Might warrant a comment though. Or maybe we just fold it into io_uring_load_sq_head()?
Signature · 1 line
Jens Axboe
Quoted 44 lines · Jens Axboe
Quoted 27 lines · Pavel Begunkov
Quoted 26 lines · Jens Axboe
Quoted 14 lines · Jens Axboe
Quoted 12 lines · Pavel Begunkov
Add liburing support and tests for
IORING_SETUP_SQ_REWIND.Pavel Begunkov (2):
src/queue: Add support for non circular SQ
tests: addSETUP_SQ_REWINDtestssrc/include/liburing.h | 5 ++++- src/include/liburing/io_uring.h | 12 ++++++++++++ src/queue.c | 5 +++++ test/test.h | 2 ++ 4 files changed, 23 insertions(+), 1 deletion(-)Quoted 8 lines
Quoted 1 line
[...]
Applied, thanks!
[1/2] src/queue: Add support for non circular SQ
commit: c22129cf0b8c936eb478d920ef84e53d89c6a5cc [2/2] tests: add SETUP_SQ_REWIND tests commit: 346c063d16bda52f02d00feb744aafe35b4002a9Hmm I do think you're missing some spots though, no?
Quoted 15 lines
src/include/liburing.h IOURINGINLINE unsigned io_uring_load_sq_head(const struct io_uring *ring) · line 1708
Quoted 12 lines
src/include/liburing.h +4
-0@@ -1702,8 +1702,13 @@ IOURINGINLINE unsigned io_uring_load_sq_head(const struct io_uring *ring)IOURINGINLINE unsigned io_uring_sq_ready (const struct io_uring *ring)LIBURING_NOEXCEPT { +unsigned head =0 ; + +if (!(ring->flags &IORING_SETUP_SQ_REWIND )) + head =io_uring_load_sq_head (ring);The head should already be zero. Actually, sounds like the
get_sqehunk from the patch is not needed either.
Quoted 2 lines
Yeah agree, I think they are both false alarms. Might warrant a comment though. Or maybe we just fold it into
io_uring_load_sq_head()?
What I'm implying is that the
+ if (!(ring->flags & IORING_SETUP_SQ_REWIND))
+ head = io_uring_load_sq_head(ring);
+
change from the original patch for normal 64B _io_uring_get_sqe() doesn't seem to be necessary. I need to take a look, but that's a good thing since the function is somewhat frequently called and inlined.
That would leave __io_uring_flush_sq() to be the only place checking the flag, so maybe comments would better to be put there.
Signature · 1 line
Pavel Begunkov
Jens Axboe
maintainerQuoted 54 lines · Pavel Begunkov
Quoted 44 lines · Jens Axboe
Quoted 27 lines · Pavel Begunkov
Quoted 26 lines · Jens Axboe
Quoted 14 lines · Jens Axboe
Quoted 12 lines · Pavel Begunkov
Add liburing support and tests for
IORING_SETUP_SQ_REWIND.Pavel Begunkov (2):
src/queue: Add support for non circular SQ
tests: addSETUP_SQ_REWINDtestssrc/include/liburing.h | 5 ++++- src/include/liburing/io_uring.h | 12 ++++++++++++ src/queue.c | 5 +++++ test/test.h | 2 ++ 4 files changed, 23 insertions(+), 1 deletion(-)Quoted 8 lines
Quoted 1 line
[...]
Applied, thanks!
[1/2] src/queue: Add support for non circular SQ
commit: c22129cf0b8c936eb478d920ef84e53d89c6a5cc [2/2] tests: add SETUP_SQ_REWIND tests commit: 346c063d16bda52f02d00feb744aafe35b4002a9Hmm I do think you're missing some spots though, no?
Quoted 15 lines
src/include/liburing.h IOURINGINLINE unsigned io_uring_load_sq_head(const struct io_uring *ring) · line 1708
Quoted 12 lines
src/include/liburing.h +4
-0@@ -1702,8 +1702,13 @@ IOURINGINLINE unsigned io_uring_load_sq_head(const struct io_uring *ring)IOURINGINLINE unsigned io_uring_sq_ready (const struct io_uring *ring)LIBURING_NOEXCEPT { +unsigned head =0 ; + +if (!(ring->flags &IORING_SETUP_SQ_REWIND )) + head =io_uring_load_sq_head (ring);The head should already be zero. Actually, sounds like the
get_sqehunk from the patch is not needed either.Quoted 2 lines
Yeah agree, I think they are both false alarms. Might warrant a comment though. Or maybe we just fold it into
io_uring_load_sq_head()?What I'm implying is that the
src/include/liburing.h +3
-0+if (!(ring->flags &IORING_SETUP_SQ_REWIND )) + head =io_uring_load_sq_head (ring); +
Quoted 8 lines
change from the original patch for normal 64B
_io_uring_get_sqe()doesn't seem to be necessary. I need to take a look, but that's a good thing since the function is somewhat frequently called and inlined.That would leave
__io_uring_flush_sq()to be the only place checking the flag, so maybe comments would better to be put there.
If you have time, would you mind checking the current repo and sending a patch? Looks like I messed up a bit and committed some of these bits with:
commit 5d1c94f754f3bca156d84b4ebcaf2f211dcd09f2
Author: Jens Axboe <[email protected]> Date: Thu Jan 22 17:11:53 2026 -0700
test/sqe-mixed-noop: add SQ_REWIND testing
inadvertently as I was doing some testing.
Signature · 1 line
Jens Axboe
Quoted 1 line · Jens Axboe
On 1/24/26 3:44 AM, Pavel Begunkov wrote:
...
Quoted 17 lines
Quoted 17 lines
Quoted 5 lines
Quoted 2 lines
The head should already be zero. Actually, sounds like the
get_sqehunk from the patch is not needed either.Yeah agree, I think they are both false alarms. Might warrant a comment though. Or maybe we just fold it into
io_uring_load_sq_head()?What I'm implying is that the
src/include/liburing.h +3
-0+if (!(ring->flags &IORING_SETUP_SQ_REWIND )) + head =io_uring_load_sq_head (ring); +change from the original patch for normal 64B
_io_uring_get_sqe()doesn't seem to be necessary. I need to take a look, but that's a good thing since the function is somewhat frequently called and inlined.
Quoted 7 lines
Quoted 3 lines
That would leave
__io_uring_flush_sq()to be the only place checking the flag, so maybe comments would better to be put there.If you have time, would you mind checking the current repo and sending a patch? Looks like I messed up a bit and committed some of these bits with:
I'll take a look a bit later, but removing should be fine unless liburing overwrites the sq head.
Signature · 1 line
Pavel Begunkov