Skip to contentSkip to conversation

Add support for IORING_SETUP_SQ_REWIND

PATCH Applied Pavel Begunkov 2 participants 10 messages Jan 21 – Jan 24 io_uring

lore
Share & tools
lore
#0 lore raw

cover Add support for IORING_SETUP_SQ_REWIND

Add liburing support and tests for IORING_SETUP_SQ_REWIND.

Pavel Begunkov (2):
src/queue: Add support for non circular SQ
tests: add SETUP_SQ_REWIND tests

 src/include/liburing.h          |  5 ++++-
 src/include/liburing/io_uring.h | 12 ++++++++++++
 src/queue.c                     |  5 +++++
 test/test.h                     |  2 ++
 4 files changed, 23 insertions(+), 1 deletion(-)
Signature · 1 line
2.52.0
#1 lore raw

PATCH 1/2 src/queue: Add support for non circular SQ

With IORING_SETUP_SQ_REWIND, the kernel doesn't use the SQ headers and always submits SQEs with indices in a range [0, nr_submit), where nr_submit is the syscall argument. Add liburing support for that. The head is now kept to be 0, and tail effectively tracks the number of entries to submit, which matches the normal formula to_submit = (tail - head) mod N

---
 src/include/liburing.h          |  5 ++++-
 src/include/liburing/io_uring.h | 12 ++++++++++++
 src/queue.c                     |  5 +++++
 3 files changed, 21 insertions(+), 1 deletion(-)
src/include/liburing.h +4 -1
index 8a882a08..4758c955 100644
@@ -1918,9 +1918,12 @@ IOURINGINLINE struct io_uring_sqe *_io_uring_get_sqe(struct io_uring *ring)
 	LIBURING_NOEXCEPT
 {
 	struct io_uring_sq *sq = &ring->sq;
-	unsigned head = io_uring_load_sq_head(ring), tail = sq->sqe_tail;
+	unsigned head = 0, tail = sq->sqe_tail;
 	struct io_uring_sqe *sqe;
 
+	if (!(ring->flags & IORING_SETUP_SQ_REWIND))
+		head = io_uring_load_sq_head(ring);
+
 	if (tail - head >= sq->ring_entries)
 		return NULL;
 
src/include/liburing/io_uring.h +12 -0
index 8e8b8e6a..2696b43d 100644
@@ -233,6 +233,18 @@ enum io_uring_sqe_flags_bit {
  */
 #define IORING_SETUP_SQE_MIXED		(1U << 19)
 
+/*
+ * When set, io_uring ignores SQ head and tail and fetches SQEs to submit
+ * starting from index 0 instead from the index stored in the head pointer.
+ * IOW, the user should place all SQE at the beginning of the SQ memory
+ * before issuing a submission syscall.
+ *
+ * It requires IORING_SETUP_NO_SQARRAY and is incompatible with
+ * IORING_SETUP_SQPOLL. The user must also never change the SQ head and tail
+ * values and keep it set to 0. Any other value is undefined behaviour.
+ */
+#define IORING_SETUP_SQ_REWIND		(1U << 20)
+
 enum io_uring_op {
 	IORING_OP_NOP,
 	IORING_OP_READV,
src/queue.c +5 -0
index c8ada7e8..00995ace 100644
@@ -205,6 +205,11 @@ static unsigned __io_uring_flush_sq(struct io_uring *ring)
 	struct io_uring_sq *sq = &ring->sq;
 	unsigned tail = sq->sqe_tail;
 
+	if (ring->flags & IORING_SETUP_SQ_REWIND) {
+		sq->sqe_tail = 0;
+		return tail;
+	}
+
 	if (sq->sqe_head != tail) {
 		sq->sqe_head = tail;
 		/*
Signature · 1 line
2.52.0
#2 lore raw

PATCH 2/2 tests: add SETUP_SQ_REWIND tests

In reply to Pavel Begunkov

Add a couple of IORING_SETUP_SQ_REWIND configurations to the nop test.

---
 test/test.h | 2 ++
 1 file changed, 2 insertions(+)
test/test.h +2 -0
index e99a8d20..f6baf61d 100644
@@ -20,6 +20,8 @@ static io_uring_test_config io_uring_test_configs[] = {
 	{ IORING_SETUP_SQE128, 				"large SQE"},
 	{ IORING_SETUP_CQE32, 				"large CQE"},
 	{ IORING_SETUP_SQE128 | IORING_SETUP_CQE32, 	"large SQE/CQE" },
+	{ IORING_SETUP_SQ_REWIND,			"rewind SQ"},
+	{ IORING_SETUP_SQ_REWIND | IORING_SETUP_SQE128,	"large rewind SQ"},
 };
 
 #define FOR_ALL_TEST_CONFIGS							\
Signature · 1 line
2.52.0
maintainer Applied
#3 lore raw
In reply to Pavel Begunkov
Quoted 6 lines · Pavel Begunkov

Add liburing support and tests for IORING_SETUP_SQ_REWIND.

Pavel Begunkov (2):
src/queue: Add support for non circular SQ
tests: add SETUP_SQ_REWIND tests

Quoted 7 lines

src/include/liburing.h | 5 ++++-

 src/include/liburing/io_uring.h | 12 ++++++++++++
 src/queue.c                     |  5 +++++
 test/test.h                     |  2 ++
 4 files changed, 23 insertions(+), 1 deletion(-)

[...]

Applied, thanks!

[1/2] src/queue: Add support for non circular SQ

      commit: c22129cf0b8c936eb478d920ef84e53d89c6a5cc
[2/2] tests: add SETUP_SQ_REWIND tests
      commit: 346c063d16bda52f02d00feb744aafe35b4002a9

Best regards,

Signature · 1 line
Jens Axboe
maintainer
#4 lore raw
Quoted 14 lines · Jens Axboe
Quoted 12 lines · Pavel Begunkov

Add liburing support and tests for IORING_SETUP_SQ_REWIND.

Pavel Begunkov (2):
src/queue: Add support for non circular SQ
tests: add SETUP_SQ_REWIND tests

src/include/liburing.h | 5 ++++-

 src/include/liburing/io_uring.h | 12 ++++++++++++
 src/queue.c                     |  5 +++++
 test/test.h                     |  2 ++
 4 files changed, 23 insertions(+), 1 deletion(-)
Quoted 8 lines
Quoted 1 line

[...]

Applied, thanks!

[1/2] src/queue: Add support for non circular SQ

      commit: c22129cf0b8c936eb478d920ef84e53d89c6a5cc
[2/2] tests: add SETUP_SQ_REWIND tests
      commit: 346c063d16bda52f02d00feb744aafe35b4002a9

Hmm I do think you're missing some spots though, no?

src/include/liburing.h +10 -2
index 987b28aaf99e..016be1e80ef2 100644
@@ -1702,8 +1702,13 @@ IOURINGINLINE unsigned io_uring_load_sq_head(const struct io_uring *ring)
 IOURINGINLINE unsigned io_uring_sq_ready(const struct io_uring *ring)
 	LIBURING_NOEXCEPT
 {
+	unsigned head = 0;
+
+	if (!(ring->flags & IORING_SETUP_SQ_REWIND))
+		head = io_uring_load_sq_head(ring);
+
 	/* always use real head, to avoid losing sync for short submit */
-	return ring->sq.sqe_tail - io_uring_load_sq_head(ring);
+	return ring->sq.sqe_tail - head;
 }
 
 /*
@@ -2063,7 +2068,7 @@ IOURINGINLINE struct io_uring_sqe *io_uring_get_sqe128(struct io_uring *ring)
 	LIBURING_NOEXCEPT
 {
 	struct io_uring_sq *sq = &ring->sq;
-	unsigned head = io_uring_load_sq_head(ring), tail = sq->sqe_tail;
+	unsigned head = 0, tail = sq->sqe_tail;
 	struct io_uring_sqe *sqe;
 
 	if (ring->flags & IORING_SETUP_SQE128)
@@ -2071,6 +2076,9 @@ IOURINGINLINE struct io_uring_sqe *io_uring_get_sqe128(struct io_uring *ring)
 	if (!(ring->flags & IORING_SETUP_SQE_MIXED))
 		return NULL;
 
+	if (!(ring->flags & IORING_SETUP_SQ_REWIND))
+		head = io_uring_load_sq_head(ring);
+
 	if (((tail + 1) & sq->ring_mask) == 0) {
 		if ((tail + 2) - head >= sq->ring_entries)
 			return NULL;
Signature · 1 line
Jens Axboe
#5 lore raw
Quoted 31 lines · Jens Axboe
Quoted 14 lines · Jens Axboe
Quoted 12 lines · Pavel Begunkov

Add liburing support and tests for IORING_SETUP_SQ_REWIND.

Pavel Begunkov (2):
src/queue: Add support for non circular SQ
tests: add SETUP_SQ_REWIND tests

src/include/liburing.h | 5 ++++-

  src/include/liburing/io_uring.h | 12 ++++++++++++
  src/queue.c                     |  5 +++++
  test/test.h                     |  2 ++
  4 files changed, 23 insertions(+), 1 deletion(-)
Quoted 8 lines
Quoted 1 line

[...]

Applied, thanks!

[1/2] src/queue: Add support for non circular SQ

       commit: c22129cf0b8c936eb478d920ef84e53d89c6a5cc
[2/2] tests: add SETUP_SQ_REWIND tests
       commit: 346c063d16bda52f02d00feb744aafe35b4002a9

Hmm I do think you're missing some spots though, no?

src/include/liburing.h +0 -0
index 987b28aaf99e..016be1e80ef2 100644

src/include/liburing.h IOURINGINLINE unsigned io_uring_load_sq_head(const struct io_uring *ring) · line 1708

Quoted 7 lines
src/include/liburing.h +4 -0
@@ -1702,8 +1702,13 @@ IOURINGINLINE unsigned io_uring_load_sq_head(const struct io_uring *ring)
  IOURINGINLINE unsigned io_uring_sq_ready(const struct io_uring *ring)
  	LIBURING_NOEXCEPT
  {
+	unsigned head = 0;
+
+	if (!(ring->flags & IORING_SETUP_SQ_REWIND))
+		head = io_uring_load_sq_head(ring);

The head should already be zero. Actually, sounds like the get_sqe hunk from the patch is not needed either.

Signature · 1 line
Pavel Begunkov
maintainer
#6 lore raw
Quoted 27 lines · Pavel Begunkov
Quoted 26 lines · Jens Axboe
Quoted 14 lines · Jens Axboe
Quoted 12 lines · Pavel Begunkov

Add liburing support and tests for IORING_SETUP_SQ_REWIND.

Pavel Begunkov (2):
src/queue: Add support for non circular SQ
tests: add SETUP_SQ_REWIND tests

src/include/liburing.h | 5 ++++-

  src/include/liburing/io_uring.h | 12 ++++++++++++
  src/queue.c                     |  5 +++++
  test/test.h                     |  2 ++
  4 files changed, 23 insertions(+), 1 deletion(-)
Quoted 8 lines
Quoted 1 line

[...]

Applied, thanks!

[1/2] src/queue: Add support for non circular SQ

       commit: c22129cf0b8c936eb478d920ef84e53d89c6a5cc
[2/2] tests: add SETUP_SQ_REWIND tests
       commit: 346c063d16bda52f02d00feb744aafe35b4002a9

Hmm I do think you're missing some spots though, no?

Quoted 15 lines

src/include/liburing.h IOURINGINLINE unsigned io_uring_load_sq_head(const struct io_uring *ring) · line 1708

Quoted 12 lines
src/include/liburing.h +4 -0
index 987b28aaf99e..016be1e80ef2 100644
@@ -1702,8 +1702,13 @@ IOURINGINLINE unsigned io_uring_load_sq_head(const struct io_uring *ring)
  IOURINGINLINE unsigned io_uring_sq_ready(const struct io_uring *ring)
      LIBURING_NOEXCEPT
  {
+    unsigned head = 0;
+
+    if (!(ring->flags & IORING_SETUP_SQ_REWIND))
+        head = io_uring_load_sq_head(ring);

The head should already be zero. Actually, sounds like the get_sqe hunk from the patch is not needed either.

Yeah agree, I think they are both false alarms. Might warrant a comment though. Or maybe we just fold it into io_uring_load_sq_head()?

Signature · 1 line
Jens Axboe
#7 lore raw
Quoted 44 lines · Jens Axboe
Quoted 27 lines · Pavel Begunkov
Quoted 26 lines · Jens Axboe
Quoted 14 lines · Jens Axboe
Quoted 12 lines · Pavel Begunkov

Add liburing support and tests for IORING_SETUP_SQ_REWIND.

Pavel Begunkov (2):
src/queue: Add support for non circular SQ
tests: add SETUP_SQ_REWIND tests

src/include/liburing.h          |  5 ++++-
   src/include/liburing/io_uring.h | 12 ++++++++++++
   src/queue.c                     |  5 +++++
   test/test.h                     |  2 ++
   4 files changed, 23 insertions(+), 1 deletion(-)
Quoted 8 lines
Quoted 1 line

[...]

Applied, thanks!

[1/2] src/queue: Add support for non circular SQ

        commit: c22129cf0b8c936eb478d920ef84e53d89c6a5cc
[2/2] tests: add SETUP_SQ_REWIND tests
        commit: 346c063d16bda52f02d00feb744aafe35b4002a9

Hmm I do think you're missing some spots though, no?

Quoted 15 lines

src/include/liburing.h IOURINGINLINE unsigned io_uring_load_sq_head(const struct io_uring *ring) · line 1708

Quoted 12 lines
src/include/liburing.h +4 -0
index 987b28aaf99e..016be1e80ef2 100644
@@ -1702,8 +1702,13 @@ IOURINGINLINE unsigned io_uring_load_sq_head(const struct io_uring *ring)
   IOURINGINLINE unsigned io_uring_sq_ready(const struct io_uring *ring)
       LIBURING_NOEXCEPT
   {
+    unsigned head = 0;
+
+    if (!(ring->flags & IORING_SETUP_SQ_REWIND))
+        head = io_uring_load_sq_head(ring);

The head should already be zero. Actually, sounds like the get_sqe hunk from the patch is not needed either.

Quoted 2 lines

Yeah agree, I think they are both false alarms. Might warrant a comment though. Or maybe we just fold it into io_uring_load_sq_head()?

What I'm implying is that the

+ if (!(ring->flags & IORING_SETUP_SQ_REWIND))
+ head = io_uring_load_sq_head(ring);
+

change from the original patch for normal 64B _io_uring_get_sqe() doesn't seem to be necessary. I need to take a look, but that's a good thing since the function is somewhat frequently called and inlined.

That would leave __io_uring_flush_sq() to be the only place checking the flag, so maybe comments would better to be put there.

Signature · 1 line
Pavel Begunkov
maintainer
#8 lore raw
Quoted 54 lines · Pavel Begunkov
Quoted 44 lines · Jens Axboe
Quoted 27 lines · Pavel Begunkov
Quoted 26 lines · Jens Axboe
Quoted 14 lines · Jens Axboe
Quoted 12 lines · Pavel Begunkov

Add liburing support and tests for IORING_SETUP_SQ_REWIND.

Pavel Begunkov (2):
src/queue: Add support for non circular SQ
tests: add SETUP_SQ_REWIND tests

src/include/liburing.h          |  5 ++++-
   src/include/liburing/io_uring.h | 12 ++++++++++++
   src/queue.c                     |  5 +++++
   test/test.h                     |  2 ++
   4 files changed, 23 insertions(+), 1 deletion(-)
Quoted 8 lines
Quoted 1 line

[...]

Applied, thanks!

[1/2] src/queue: Add support for non circular SQ

        commit: c22129cf0b8c936eb478d920ef84e53d89c6a5cc
[2/2] tests: add SETUP_SQ_REWIND tests
        commit: 346c063d16bda52f02d00feb744aafe35b4002a9

Hmm I do think you're missing some spots though, no?

Quoted 15 lines

src/include/liburing.h IOURINGINLINE unsigned io_uring_load_sq_head(const struct io_uring *ring) · line 1708

Quoted 12 lines
src/include/liburing.h +4 -0
index 987b28aaf99e..016be1e80ef2 100644
@@ -1702,8 +1702,13 @@ IOURINGINLINE unsigned io_uring_load_sq_head(const struct io_uring *ring)
   IOURINGINLINE unsigned io_uring_sq_ready(const struct io_uring *ring)
       LIBURING_NOEXCEPT
   {
+    unsigned head = 0;
+
+    if (!(ring->flags & IORING_SETUP_SQ_REWIND))
+        head = io_uring_load_sq_head(ring);

The head should already be zero. Actually, sounds like the get_sqe hunk from the patch is not needed either.

Quoted 2 lines

Yeah agree, I think they are both false alarms. Might warrant a comment though. Or maybe we just fold it into io_uring_load_sq_head()?

What I'm implying is that the

src/include/liburing.h +3 -0
+    if (!(ring->flags & IORING_SETUP_SQ_REWIND))
+        head = io_uring_load_sq_head(ring);
+
Quoted 8 lines

change from the original patch for normal 64B _io_uring_get_sqe() doesn't seem to be necessary. I need to take a look, but that's a good thing since the function is somewhat frequently called and inlined.

That would leave __io_uring_flush_sq() to be the only place checking the flag, so maybe comments would better to be put there.

If you have time, would you mind checking the current repo and sending a patch? Looks like I messed up a bit and committed some of these bits with:

commit 5d1c94f754f3bca156d84b4ebcaf2f211dcd09f2

Author: Jens Axboe <[email protected]>
Date:   Thu Jan 22 17:11:53 2026 -0700

test/sqe-mixed-noop: add SQ_REWIND testing

inadvertently as I was doing some testing.

Signature · 1 line
Jens Axboe
#9 lore raw
Quoted 1 line · Jens Axboe

On 1/24/26 3:44 AM, Pavel Begunkov wrote:

...

Quoted 17 lines
Quoted 17 lines
Quoted 5 lines
Quoted 2 lines

The head should already be zero. Actually, sounds like the get_sqe hunk from the patch is not needed either.

Yeah agree, I think they are both false alarms. Might warrant a comment though. Or maybe we just fold it into io_uring_load_sq_head()?

What I'm implying is that the

src/include/liburing.h +3 -0
+    if (!(ring->flags & IORING_SETUP_SQ_REWIND))
+        head = io_uring_load_sq_head(ring);
+

change from the original patch for normal 64B _io_uring_get_sqe() doesn't seem to be necessary. I need to take a look, but that's a good thing since the function is somewhat frequently called and inlined.

Quoted 7 lines
Quoted 3 lines

That would leave __io_uring_flush_sq() to be the only place checking the flag, so maybe comments would better to be put there.

If you have time, would you mind checking the current repo and sending a patch? Looks like I messed up a bit and committed some of these bits with:

I'll take a look a bit later, but removing should be fine unless liburing overwrites the sq head.

Signature · 1 line
Pavel Begunkov