Skip to content

Support for provided buffers for send

Jens Axboe io-uring 4 versions – New

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v4 Latest9 patches 0 repliesNew
  2. v38 patches 25 repliesSuperseded
  3. v23 patches 0 repliesSuperseded
Earlier versions · 1
VersionPostedPatchesRepliesStatus
v12 patches0 repliesSuperseded

Changes in v4 from the cover letter

- Drop MSG_MORE patch, separate thing anyway. Moved the flags hunk into the respective send/sendmsg patches, where they actually belonged.
- Rename IORING_FEAT_SEND_BUFS to IORING_FEAT_SEND_BUF_SELECT
- Enable MSG_WAITALL for send multishot. If set, then we retry via poll, if not set, we terminate the multishot sequence. This also fixes send multishot with short send in general.
- Add other networking related patch for recv/recvmsg multishot, managing IORING_CQE_F_MORE better rather than always needing to hit -ENOBUFS to terminate.

Patches in v4 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/9net: remove {revc,send}msg_copy_msghdr() from exportsinclude/linux/socket.h net/socket.c tools/perf/trace/beauty/include/linux/socket.h+7 -210 replies
2/9io_uring/net: add provided buffer support for IORING_OP_SENDinclude/uapi/linux/io_uring.h io_uring/io_uring.c io_uring/net.c +1 more+20 -40 replies
3/9io_uring/net: add provided buffer support for IORING_OP_SENDMSGio_uring/net.c io_uring/opdef.c+15 -10 replies
4/9io_uring/kbuf: flag request if buffer pool is empty after buffer pickinclude/linux/io_uring_types.h io_uring/kbuf.c+11 -20 replies
5/9io_uring/net: avoid redundant -ENOBUFS on recv multishot retryio_uring/net.c+6 -00 replies
6/9io_uring/net: move recv/recvmsg flags out of retry loopio_uring/net.c+8 -70 replies
7/9io_uring/net: add generic multishot retry helperio_uring/net.c+11 -100 replies
8/9io_uring/net: support multishot for sendinclude/uapi/linux/io_uring.h io_uring/net.c+86 -120 replies
9/9io_uring/net: support multishot for sendmsgio_uring/net.c+33 -140 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v4 yet.

Read the v4 thread

Cover letter [PATCHSET v4 0/9] Support for provided buffers for send

Hi,

We never supported provided buffers for sends, because it didn't seem to make a lot of sense. But it actually does make a lot of sense! If an app is receiving data, doing something with it, and then sending either the same or another buffer out based on that, then if we use provided buffers for sends we can guarantee that the sends are serialized. This is because provided buffer rings are FIFO ordered, as it's a ring buffer, and hence it doesn't really matter if you have more than one send inflight.

This provides a nice efficiency win, but more importantly, it reduces the complexity in the application as it no longer needs to track a potential backlog of sends. The app just sets up a send based buffer ring, exactly like it does for incoming data. And that's it, no more dealing with serialized sends.

In some testing with proxy [1], in basic shuffling of packets I see a 68% improvement with this over manually dealing with serializing sends. That's a pretty big win on top of making the app simpler. Using multishot further brings a nice improvement on top, about 10% extra on top.

You can also find the patches here:

https://git.kernel.dk/cgit/linux/log/?h=io_uring-send-queue

[1] https://git.kernel.dk/cgit/liburing/tree/examples/proxy.c

Changes since v3:

- Drop MSG_MORE patch, separate thing anyway. Moved the flags hunk into the respective send/sendmsg patches, where they actually belonged.
- Rename IORING_FEAT_SEND_BUFS to IORING_FEAT_SEND_BUF_SELECT
- Enable MSG_WAITALL for send multishot. If set, then we retry via poll, if not set, we terminate the multishot sequence. This also fixes send multishot with short send in general.
- Add other networking related patch for recv/recvmsg multishot, managing IORING_CQE_F_MORE better rather than always needing to hit -ENOBUFS to terminate.

Signature · 1 line
Jens Axboe