Skip to contentSkip to conversation

fix _io_uring_get_cqe()

PATCH In review Pavel Begunkov 2 participants 5 messages Feb 7, 2021 – Feb 8, 2021 io_uring

lore
Share & tools
lore
#0 lore raw

cover fix _io_uring_get_cqe()

address live-locking of _io_uring_get_cqe() reported by Victor.

Pavel Begunkov (3):
src/queue: don't wait for less than expected src/queue: clean _io_uring_get_cqe() err handling src/queue: don't loop when don't enter

 src/queue.c | 28 +++++++++++++++-------------
 1 file changed, 15 insertions(+), 13 deletions(-)
Signature · 1 line
2.24.0
#1 lore raw

PATCH 1/3 src/queue: don't wait for less than expected

__io_uring_peek_cqe() doesn't consume cqe it returns, no need to decrease wait_nr because we check against the number in CQ as well as the kernel do. One exception for that behaviour is IOPOLL, but that kernel will return if there is anything in CQ, so will work just fine.

---
 src/queue.c | 2 --
 1 file changed, 2 deletions(-)
src/queue.c +0 -2
index 94f791e..dd1df2a 100644
@@ -106,8 +106,6 @@ static int _io_uring_get_cqe(struct io_uring *ring, struct io_uring_cqe **cqe_pt
 			}
 			cq_overflow_flush = true;
 		}
-		if (data->wait_nr && cqe)
-			data->wait_nr--;
 		if (data->wait_nr || cq_overflow_flush)
 			flags = IORING_ENTER_GETEVENTS | data->get_flags;
 		if (data->submit)
Signature · 1 line
2.24.0
#2 lore raw

PATCH 2/3 src/queue: clean _io_uring_get_cqe() err handling

In reply to Pavel Begunkov

Doesn't change behaviour, just a little cleanup.

---
 src/queue.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)
src/queue.c +6 -5
index dd1df2a..be461c6 100644
@@ -117,8 +117,11 @@ static int _io_uring_get_cqe(struct io_uring *ring, struct io_uring_cqe **cqe_pt
 					data->sz);
 		if (ret < 0) {
 			err = -errno;
-		} else if (ret == (int)data->submit) {
-			data->submit = 0;
+			break;
+		}
+
+		data->submit -= ret;
+		if (ret == (int)data->submit) {
 			/*
 			 * When SETUP_IOPOLL is set, __sys_io_uring enter()
 			 * must be called to reap new completions but the call
@@ -127,12 +130,10 @@ static int _io_uring_get_cqe(struct io_uring *ring, struct io_uring_cqe **cqe_pt
 			 */
 			if (!(ring->flags & IORING_SETUP_IOPOLL))
 				data->wait_nr = 0;
-		} else {
-			data->submit -= ret;
 		}
 		if (cqe)
 			break;
-	} while (!err);
+	} while (1);
 
 	*cqe_ptr = cqe;
 	return err;
Signature · 1 line
2.24.0
#3 lore raw

PATCH 3/3 src/queue: don't loop when don't enter

In reply to Pavel Begunkov

_io_uring_get_cqe() can live-lock in some cases, always return if we're not going to do __sys_io_uring_enter().

---
 src/queue.c | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)
src/queue.c +9 -6
index be461c6..8c394dd 100644
@@ -89,12 +89,13 @@ static int _io_uring_get_cqe(struct io_uring *ring, struct io_uring_cqe **cqe_pt
 {
 	struct io_uring_cqe *cqe = NULL;
 	const int to_wait = data->wait_nr;
-	int ret = 0, err;
+	int err;
 
 	do {
 		bool cq_overflow_flush = false;
 		unsigned flags = 0;
 		unsigned nr_available;
+		int ret;
 
 		err = __io_uring_peek_cqe(ring, &cqe, &nr_available);
 		if (err)
@@ -110,11 +111,13 @@ static int _io_uring_get_cqe(struct io_uring *ring, struct io_uring_cqe **cqe_pt
 			flags = IORING_ENTER_GETEVENTS | data->get_flags;
 		if (data->submit)
 			sq_ring_needs_enter(ring, &flags);
-		if (data->wait_nr > nr_available || data->submit ||
-		    cq_overflow_flush)
-			ret = __sys_io_uring_enter2(ring->ring_fd, data->submit,
-					data->wait_nr, flags, data->arg,
-					data->sz);
+		if (data->wait_nr <= nr_available && !data->submit &&
+		    !cq_overflow_flush)
+			break;
+
+		ret = __sys_io_uring_enter2(ring->ring_fd, data->submit,
+				data->wait_nr, flags, data->arg,
+				data->sz);
 		if (ret < 0) {
 			err = -errno;
 			break;
Signature · 1 line
2.24.0
maintainer
#4 lore raw
In reply to Pavel Begunkov
Quoted 10 lines · Pavel Begunkov

address live-locking of _io_uring_get_cqe() reported by Victor.

Pavel Begunkov (3):
src/queue: don't wait for less than expected src/queue: clean _io_uring_get_cqe() err handling src/queue: don't loop when don't enter

 src/queue.c | 28 +++++++++++++++-------------
 1 file changed, 15 insertions(+), 13 deletions(-)

Nice, thanks Pavel! Applied.

Signature · 1 line
Jens Axboe