Skip to contentSkip to conversation

io_uring/net: support registered buffer for plain send and recv

PATCH Superseded Ming Lei 2 participants 5 messages Jun 1 – Jun 8 io_uring

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Ming Lei

author
#0 lore raw

cover io_uring/net: support registered buffer for plain send and recv

From: Ming Lei <[email protected]>

Hi,

This series wires IORING_RECVSEND_FIXED_BUF into the plain IORING_OP_SEND and IORING_OP_RECV paths. So far the flag has only been honoured on the SEND_ZC path, even though the import wiring is already present for plain send and completely absent for recv.

Motivation: targets such as ublk's NBD backend want to push/pull I/O data directly to/from an io_uring registered buffer over a plain send/recv on a TCP socket, without the SEND_ZC notification machinery.

The flag is accepted at prep time for the non-vectorized IORING_OP_SEND / IORING_OP_RECV opcodes only, and is mutually exclusive with buffer select, bundles and (for recv) multishot. The registered buffer is imported lazily at issue time via io_import_reg_buf() (mirroring the existing send path), and the resulting bvec iter persists in async_data so MSG_WAITALL partial send/recv retries resume at the right offset.

Patch 1 is the kernel change.
Patch 2 adds a liburing test and is meant for the liburing tree. It covers send-fixed/recv-fixed/both-fixed roundtrips (with non-zero offsets into distinct registered buffers), a large MSG_WAITALL transfer that exercises the persisted bvec iter across partial retries, and the negative validation cases (sendmsg/bundle/recv-multishot rejected with -EINVAL, bad buf_index returning -EFAULT).

Ming Lei (2):
io_uring/net: support registered buffer for plain send and recv
test: add fixed-buf-send-recv for registered buffer send/recv

 io_uring/net.c             |  46 ++++++-
 test/Makefile              |   1 +
 test/fixed-buf-send-recv.c | 300 +++++++++++++++++++++++++++++++++++++
 3 files changed, 345 insertions(+), 2 deletions(-)

--
2.54.0

author
#1 lore raw

PATCH 1/2 io_uring/net: support registered buffer for plain send and recv

From: Ming Lei <[email protected]>

So far IORING_RECVSEND_FIXED_BUF is only honoured on the SEND_ZC path, even though the import wiring is already present for plain send and completely absent for recv. Targets such as ublk's NBD backend want to push/pull I/O data directly to/from an io_uring registered buffer over a plain send/recv on a TCP socket, without the SEND_ZC notification machinery.

Wire IORING_RECVSEND_FIXED_BUF into the plain IORING_OP_SEND and IORING_OP_RECV paths:

- Accept the flag in SENDMSG_FLAGS / RECVMSG_FLAGS and, at prep time, restrict it to the non-vectorized IORING_OP_SEND / IORING_OP_RECV opcodes. It is mutually exclusive with buffer select, bundles and (for recv) multishot, and records sqe->buf_index.

- For recv, set REQ_F_IMPORT_BUFFER in setup so the registered buffer is imported lazily at issue time, mirroring the send path.

- In io_send()/io_recv(), import the registered buffer via io_import_reg_buf() (ITER_SOURCE for send, ITER_DEST for recv) and clear REQ_F_IMPORT_BUFFER. The resulting bvec iter persists in async_data, so MSG_WAITALL partial send/recv retries resume at the right offset.

---
 io_uring/net.c | 46 ++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 44 insertions(+), 2 deletions(-)
io_uring/net.c +44 -2
index f01f1d25e930..9c42c3dbccd7 100644
@@ -418,7 +418,8 @@ static int io_sendmsg_setup(struct io_kiocb *req, const struct io_uring_sqe *sqe
 	return io_net_import_vec(req, kmsg, msg.msg_iov, msg.msg_iovlen, ITER_SOURCE);
 }
 
-#define SENDMSG_FLAGS (IORING_RECVSEND_POLL_FIRST | IORING_RECVSEND_BUNDLE | IORING_SEND_VECTORIZED)
+#define SENDMSG_FLAGS (IORING_RECVSEND_POLL_FIRST | IORING_RECVSEND_BUNDLE | \
+			IORING_SEND_VECTORIZED | IORING_RECVSEND_FIXED_BUF)
 
 int io_sendmsg_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe)
 {
@@ -431,6 +432,14 @@ int io_sendmsg_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe)
 	sr->flags = READ_ONCE(sqe->ioprio);
 	if (sr->flags & ~SENDMSG_FLAGS)
 		return -EINVAL;
+	if (sr->flags & IORING_RECVSEND_FIXED_BUF) {
+		/* registered buffer send only supported for plain IORING_OP_SEND */
+		if (req->opcode != IORING_OP_SEND ||
+		    (sr->flags & IORING_RECVSEND_BUNDLE) ||
+		    (req->flags & REQ_F_BUFFER_SELECT))
+			return -EINVAL;
+		req->buf_index = READ_ONCE(sqe->buf_index);
+	}
 	sr->msg_flags = READ_ONCE(sqe->msg_flags) | MSG_NOSIGNAL;
 	if (sr->msg_flags & MSG_DONTWAIT)
 		req->flags |= REQ_F_NOWAIT;
@@ -662,6 +671,15 @@ int io_send(struct io_kiocb *req, unsigned int issue_flags)
 	    (sr->flags & IORING_RECVSEND_POLL_FIRST))
 		return -EAGAIN;
 
+	if (req->flags & REQ_F_IMPORT_BUFFER) {
+		ret = io_import_reg_buf(req, &kmsg->msg.msg_iter,
+					(u64)(uintptr_t)sr->buf, sr->len,
+					ITER_SOURCE, issue_flags);
+		if (unlikely(ret))
+			return ret;
+		req->flags &= ~REQ_F_IMPORT_BUFFER;
+	}
+
 	flags = sr->msg_flags;
 	if (issue_flags & IO_URING_F_NONBLOCK)
 		flags |= MSG_DONTWAIT;
@@ -777,6 +795,10 @@ static int io_recvmsg_prep_setup(struct io_kiocb *req)
 
 		if (req->flags & REQ_F_BUFFER_SELECT)
 			return 0;
+		if (sr->flags & IORING_RECVSEND_FIXED_BUF) {
+			req->flags |= REQ_F_IMPORT_BUFFER;
+			return 0;
+		}
 		return import_ubuf(ITER_DEST, sr->buf, sr->len,
 				   &kmsg->msg.msg_iter);
 	}
@@ -785,7 +807,7 @@ static int io_recvmsg_prep_setup(struct io_kiocb *req)
 }
 
 #define RECVMSG_FLAGS (IORING_RECVSEND_POLL_FIRST | IORING_RECV_MULTISHOT | \
-			IORING_RECVSEND_BUNDLE)
+			IORING_RECVSEND_BUNDLE | IORING_RECVSEND_FIXED_BUF)
 
 int io_recvmsg_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe)
 {
@@ -803,6 +825,14 @@ int io_recvmsg_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe)
 	sr->flags = READ_ONCE(sqe->ioprio);
 	if (sr->flags & ~RECVMSG_FLAGS)
 		return -EINVAL;
+	if (sr->flags & IORING_RECVSEND_FIXED_BUF) {
+		/* registered buffer recv only for plain IORING_OP_RECV */
+		if (req->opcode != IORING_OP_RECV ||
+		    (sr->flags & (IORING_RECV_MULTISHOT | IORING_RECVSEND_BUNDLE)) ||
+		    (req->flags & REQ_F_BUFFER_SELECT))
+			return -EINVAL;
+		req->buf_index = READ_ONCE(sqe->buf_index);
+	}
 	sr->msg_flags = READ_ONCE(sqe->msg_flags);
 	if (sr->msg_flags & MSG_DONTWAIT)
 		req->flags |= REQ_F_NOWAIT;
@@ -1199,6 +1229,18 @@ int io_recv(struct io_kiocb *req, unsigned int issue_flags)
 	if (force_nonblock)
 		flags |= MSG_DONTWAIT;
 
+	if (req->flags & REQ_F_IMPORT_BUFFER) {
+		ret = io_import_reg_buf(req, &kmsg->msg.msg_iter,
+					(u64)(uintptr_t)sr->buf, sr->len,
+					ITER_DEST, issue_flags);
+		if (unlikely(ret)) {
+			kmsg->msg.msg_inq = -1;
+			sel.buf_list = NULL;
+			goto out_free;
+		}
+		req->flags &= ~REQ_F_IMPORT_BUFFER;
+	}
+
 retry_multishot:
 	sel.buf_list = NULL;
 	if (io_do_buffer_select(req)) {
Signature · 1 line
2.54.0
maintainer
#2 lore raw
Quoted 6 lines · Ming Lei
io_uring/net.c +0 -0
index f01f1d25e930..9c42c3dbccd7 100644
@@ -431,6 +432,14 @@ int io_sendmsg_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe)
 	sr->flags = READ_ONCE(sqe->ioprio);

io_uring/net.c int io_sendmsg_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe) · line 442

Quoted 10 lines
io_uring/net.c +8 -0
 	if (sr->flags & ~SENDMSG_FLAGS)
 		return -EINVAL;
+	if (sr->flags & IORING_RECVSEND_FIXED_BUF) {
+		/* registered buffer send only supported for plain IORING_OP_SEND */
+		if (req->opcode != IORING_OP_SEND ||
+		    (sr->flags & IORING_RECVSEND_BUNDLE) ||
+		    (req->flags & REQ_F_BUFFER_SELECT))
+			return -EINVAL;
+		req->buf_index = READ_ONCE(sqe->buf_index);
+	}

I think this should either reject IORING_SEND_VECTORIZED, or if there's a use case for it, ensure that it actually works.

Outside of that, change seems straight forward to me.

Signature · 1 line
Jens Axboe
author
#3 lore raw

PATCH 2/2 liburing test: add fixed-buf-send-recv for registered buffer send/recv

In reply to Ming Lei

From: Ming Lei <[email protected]>

Exercise IORING_RECVSEND_FIXED_BUF on plain IORING_OP_SEND and
IORING_OP_RECV: send-fixed, recv-fixed and both-fixed roundtrips with non-zero offsets into distinct registered buffers, a large MSG_WAITALL transfer to cover the persisted bvec iter across partial retries, plus negative cases (sendmsg/bundle/recv-multishot rejected with -EINVAL and a bad buf_index returning -EFAULT).

---
 test/Makefile              |   1 +
 test/fixed-buf-send-recv.c | 300 +++++++++++++++++++++++++++++++++++++
 2 files changed, 301 insertions(+)
 create mode 100644 test/fixed-buf-send-recv.c
test/Makefile +1 -0
index d1cd2470..effb3bae 100644
@@ -132,6 +132,7 @@ test_srcs := \
 	file-verify.c \
 	fixed-buf-iter.c \
 	fixed-buf-merge.c \
+	fixed-buf-send-recv.c \
 	fixed-hugepage.c \
 	fixed-link.c \
 	fixed-reuse.c \
test/fixed-buf-send-recv.c new file +300 -0

300 lines not on this page · raw message

Signature · 1 line
2.54.0