Skip to contentSkip to conversation

zc changes

PATCH Applied Pavel Begunkov 2 participants 6 messages Sep 21, 2022 io_uring

lore
Share & tools
lore
#0 lore raw

cover zc changes

We're decoupling notifications generation from whether the request failed or not. Adjust the tests, so nobody is confused reading it, and put a note in man.

Also, 4/4 adds a test for just sent zc sendmsg.

Pavel Begunkov (4):
examples: fix sendzc notif handling
test: fix zc tests
man: note about notification generation
tests: add sendmsg_zc tests

 examples/send-zerocopy.c        |  17 +++---
 man/io_uring_enter.2            |   5 +-
 src/include/liburing/io_uring.h |   1 +
 test/send-zerocopy.c            | 101 ++++++++++++++++++++++++--------
 4 files changed, 88 insertions(+), 36 deletions(-)
Signature · 1 line
2.37.2
#1 lore raw

PATCH 1/4 examples: fix sendzc notif handling

---
 examples/send-zerocopy.c | 17 ++++++++---------
 1 file changed, 8 insertions(+), 9 deletions(-)
examples/send-zerocopy.c +8 -9
index 4ed0f67..7f5f2b1 100644
@@ -190,8 +190,6 @@ static void do_tx(int domain, int type, int protocol)
 				sqe->flags |= IOSQE_FIXED_FILE;
 			}
 		}
-		if (cfg_zc)
-			compl_cqes += cfg_nr_reqs;
 
 		ret = io_uring_submit(&ring);
 		if (ret != cfg_nr_reqs)
@@ -205,19 +203,20 @@ static void do_tx(int domain, int type, int protocol)
 					error(1, -EINVAL, "F_MORE notif");
 				compl_cqes--;
 				i--;
-			} else if (cqe->res >= 0) {
-				if (!(cqe->flags & IORING_CQE_F_MORE) && cfg_zc)
-					error(1, -EINVAL, "no F_MORE");
+				io_uring_cqe_seen(&ring, cqe);
+				continue;
+			}
+			if (cqe->flags & IORING_CQE_F_MORE)
+				compl_cqes++;
+
+			if (cqe->res >= 0) {
 				packets++;
 				bytes += cqe->res;
-			} else if (cqe->res == -EAGAIN) {
-				if (cfg_zc)
-					compl_cqes--;
 			} else if (cqe->res == -ECONNREFUSED || cqe->res == -EPIPE ||
 				   cqe->res == -ECONNRESET) {
 				fprintf(stderr, "Connection failure");
 				goto out_fail;
-			} else {
+			} else if (cqe->res != -EAGAIN) {
 				error(1, cqe->res, "send failed");
 			}
 			io_uring_cqe_seen(&ring, cqe);
Signature · 1 line
2.37.2
#2 lore raw

PATCH 2/4 test: fix zc tests

In reply to Pavel Begunkov

Decouple result from F_NOTIF/F_MORE, even failed requests may produce a notification.

---
 test/send-zerocopy.c | 16 +++++++++++-----
 1 file changed, 11 insertions(+), 5 deletions(-)
test/send-zerocopy.c +11 -5
index 39c5c5d..e34e0c1 100644
@@ -117,7 +117,7 @@ static int test_send_faults(struct io_uring *ring, int sock_tx, int sock_rx)
 	int msg_flags = 0;
 	unsigned zc_flags = 0;
 	int payload_size = 100;
-	int ret, i;
+	int ret, i, nr_cqes = 2;
 
 	sqe = io_uring_get_sqe(ring);
 	io_uring_prep_send_zc(sqe, sock_tx, (void *)1UL, payload_size,
@@ -125,7 +125,7 @@ static int test_send_faults(struct io_uring *ring, int sock_tx, int sock_rx)
 	sqe->user_data = 1;
 
 	sqe = io_uring_get_sqe(ring);
-	io_uring_prep_send_zc(sqe, sock_tx, (void *)1UL, payload_size,
+	io_uring_prep_send_zc(sqe, sock_tx, tx_buffer, payload_size,
 			      msg_flags, zc_flags);
 	sqe->user_data = 2;
 	io_uring_prep_send_set_addr(sqe, (const struct sockaddr *)1UL,
@@ -134,12 +134,18 @@ static int test_send_faults(struct io_uring *ring, int sock_tx, int sock_rx)
 	ret = io_uring_submit(ring);
 	assert(ret == 2);
 
-	for (i = 0; i < 2; i++) {
+	for (i = 0; i < nr_cqes; i++) {
 		ret = io_uring_wait_cqe(ring, &cqe);
 		assert(!ret);
 		assert(cqe->user_data <= 2);
-		assert(cqe->res == -EFAULT);
-		assert(!(cqe->flags & IORING_CQE_F_MORE));
+
+		if (cqe->flags & IORING_CQE_F_NOTIF) {
+			assert(ret > 0);
+		} else {
+			assert(cqe->res == -EFAULT);
+			if (cqe->flags & IORING_CQE_F_MORE)
+				nr_cqes++;
+		}
 		io_uring_cqe_seen(ring, cqe);
 	}
 	assert(check_cq_empty(ring));
Signature · 1 line
2.37.2
#3 lore raw

PATCH 3/4 man: note about notification generation

In reply to Pavel Begunkov

Add a small note to clarify expectations on when to expect a notification.

---
 man/io_uring_enter.2 | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
man/io_uring_enter.2 +4 -1
index d87f254..e0ce051 100644
@@ -1091,7 +1091,10 @@ long time, e.g. waiting for a TCP ACK, and having a separate cqe for request
 completions allows userspace to push more data without extra delays. Note,
 notifications are only responsible for controlling the lifetime of the buffers,
 and as such don't mean anything about whether the data has atually been sent
-out or received by the other end.
+out or received by the other end. Even errored requests may generate a
+notification, and the user must check for
+.B IORING_CQE_F_MORE
+rather than relying on the result.
 
 .I fd
 must be set to the socket file descriptor,
Signature · 1 line
2.37.2
#4 lore raw

PATCH 4/4 tests: add sendmsg_zc tests

In reply to Pavel Begunkov
---
 src/include/liburing/io_uring.h |  1 +
 test/send-zerocopy.c            | 85 +++++++++++++++++++++++++--------
 2 files changed, 65 insertions(+), 21 deletions(-)
src/include/liburing/io_uring.h +1 -0
index 972b179..92f29d9 100644
@@ -213,6 +213,7 @@ enum io_uring_op {
 	IORING_OP_SOCKET,
 	IORING_OP_URING_CMD,
 	IORING_OP_SEND_ZC,
+	IORING_OP_SENDMSG_ZC,
 
 	/* this goes last, obviously */
 	IORING_OP_LAST,
test/send-zerocopy.c +64 -21

161 lines not on this page · raw message

Signature · 1 line
2.37.2
maintainer Applied
#5 lore raw
In reply to Pavel Begunkov
Quoted 6 lines · Pavel Begunkov

We're decoupling notifications generation from whether the request failed or not. Adjust the tests, so nobody is confused reading it, and put a note in man.

Also, 4/4 adds a test for just sent zc sendmsg.

Quoted 7 lines

Pavel Begunkov (4):
examples: fix sendzc notif handling
test: fix zc tests
man: note about notification generation
tests: add sendmsg_zc tests

[...]

Applied, thanks!

[1/4] examples: fix sendzc notif handling

      commit: a3a35cf05c6ed670f0f14c3181c10683d22d98da
[2/4] test: fix zc tests
      commit: 5127b05e5acf530020518d198401f09c32e09f9c
[3/4] man: note about notification generation
      commit: ccde465295cd07d0a4b25ae892de957d50424b50
[4/4] tests: add sendmsg_zc tests
      commit: dec0b1db70e8fadf9666289fa00c1ef508a1e8a2

Best regards,

Signature · 1 line
Jens Axboe