Skip to content

RFC zero-copy RX for io_uring

Jonathan Lemon io-uring 3 versions – New

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v3 RFC Latest15 patches 0 repliesNew
  2. v2 RFC13 patches 2 repliesSuperseded
  3. v1 RFC9 patches 2 repliesSuperseded

Changelog from the cover letter

v1: initial version
v2: Remove separate PROVIDE_REGION opcode, fold this functionality
into REGISTER_IFQ. Remove page_pool hooks, as it appears the page pool is currently incompatible with user-mapped memory. Add io_zctap_buffers and network driver API.
v3: Change freelist so it holds a zctap buffer index, instead of
a pointer. Add caching mechanism for better performance. Add notify mechanism which informs the app of fillq buffers removed. Clean up some refcount issues.

Patches in v3 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/15io_uring: add zctap ifq definitioninclude/linux/io_uring_types.h+11 -00 replies
2/15netdevice: add SETUP_ZCTAP to the netdev_bpf structureinclude/linux/netdevice.h+6 -00 replies
3/15io_uring: add register ifq opcodeinclude/uapi/linux/io_uring.h io_uring/Makefile io_uring/io_uring.c +2 more+165 -10 replies
4/15io_uring: create a zctap region for a mapped bufferio_uring/zctap.c io_uring/zctap.h+64 -10 replies
5/15io_uring: mark pages in ifq region with zctap information.io_uring/zctap.c+56 -50 replies
6/15io_uring: Provide driver API for zctap packet buffers.include/linux/io_uring.h io_uring/zctap.c+70 -00 replies
7/15io_uring: Allocate zctap device buffers and dma map them.io_uring/zctap.c+63 -150 replies
8/15io_uring: Add zctap buffer get/put functions and refcounting.io_uring/kbuf.c io_uring/kbuf.h io_uring/zctap.c+141 -10 replies
9/15skbuff: Introduce SKBFL_FIXED_FRAG and skb_fixed()include/linux/skbuff.h+9 -10 replies
10/15io_uring: Allocate a uarg for use by the ifq RXio_uring/zctap.c+56 -80 replies
11/15io_uring: Define the zctap iov[] returned to the user.include/uapi/linux/io_uring.h+8 -00 replies
12/15io_uring: add OP_RECV_ZC command.include/uapi/linux/io_uring.h io_uring/net.c io_uring/opdef.c +2 more+497 -00 replies
13/15io_uring: Make remove_ifq_region a delayed work callinclude/linux/io_uring_types.h io_uring/zctap.c+59 -190 replies
14/15io_uring: Add a buffer caching mechanism for zctap.io_uring/zctap.c+99 -290 replies
15/15io_uring: Notify the application as the fillq is drained.io_uring/zctap.c+40 -00 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v3 yet.

Read the v3 thread

Cover letter [RFC PATCH v3 00/15] zero-copy RX for io_uring

This series is a RFC for io_uring/zctap. This is an evolution of the earlier zctap work, re-targeted to use io_uring as the userspace API. The current code is intended to provide a zero-copy RX path for upper-level networking protocols (aka TCP and UDP). The current draft focuses on host-provided memory (not GPU memory).

This RFC contains the upper-level core code required for operation, with the intent of soliciting feedback on the general API. This does not contain the network driver side changes required for complete operation. Also please note that as an RFC, there are some things which are incomplete or in need of rework.

The intent is to use a network driver which provides header/data splitting, so the frame header (which is processed by the networking stack) does not reside in user memory.

The code is successfully receiving a zero-copy TCP stream from a remote sender. An RFC, the intent is to solicit feedback on the API and overall design. The current code will also work with system pages, copying the data out to the application - this is intended as a fallback/testing path.

There is an liburing fork: https://github.com/jlemon/liburing/tree/zctap

Which contains an examples/io_uring-net test application exercising these features. A sample run:

  # ./io_uring-net -i eth1 -q 20 -p 9999 -r 3000
   copy bytes: 1938872
     ZC bytes: 996683008
  Total bytes: 998621880, nsec:1025219375
         Rate: 7.79 Gb/s

If no queue is specified, then non-zc mode is used:

  # ./io_uring-net -p 9999
   copy bytes: 998621880
     ZC bytes: 0
  Total bytes: 998621880, nsec:1051515726
         Rate: 7.60 Gb/s

High level description:

The application allocates a frame backing store, and provides this to the kernel for use. An interface queue is requested from the networking device, and incoming frames are deposited into the provided memory region.

Responsibility for correctly steering incoming frames to the queue is outside the scope of this work - it is assumed that the user has set steering rules up separately.

Incoming frames are sent up the stack as skb's and eventually land in the application's socket receive queue. This differs from AF_XDP, which receives raw frames directly to userspace, without protocol processing.

The RECV_ZC opcode then returns an iov[] style vector which points to the data in userspace memory. When the application has completed processing of the data, the buffer is returned back to the kernel through a fill ring for reuse.

Changelog:
v1: initial version
v2: Remove separate PROVIDE_REGION opcode, fold this functionality
into REGISTER_IFQ. Remove page_pool hooks, as it appears the page pool is currently incompatible with user-mapped memory. Add io_zctap_buffers and network driver API.
v3: Change freelist so it holds a zctap buffer index, instead of
a pointer. Add caching mechanism for better performance. Add notify mechanism which informs the app of fillq buffers removed. Clean up some refcount issues.

Jonathan Lemon (15):
io_uring: add zctap ifq definition
netdevice: add SETUP_ZCTAP to the netdev_bpf structure
io_uring: add register ifq opcode
io_uring: create a zctap region for a mapped buffer
io_uring: mark pages in ifq region with zctap information.
io_uring: Provide driver API for zctap packet buffers.
io_uring: Allocate zctap device buffers and dma map them.
io_uring: Add zctap buffer get/put functions and refcounting.
skbuff: Introduce SKBFL_FIXED_FRAG and skb_fixed()
io_uring: Allocate a uarg for use by the ifq RX
io_uring: Define the zctap iov[] returned to the user.
io_uring: add OP_RECV_ZC command.
io_uring: Make remove_ifq_region a delayed work call
io_uring: Add a buffer caching mechanism for zctap.
io_uring: Notify the application as the fillq is drained.

 include/linux/io_uring.h       |  47 ++
 include/linux/io_uring_types.h |  12 +
 include/linux/netdevice.h      |   6 +
 include/linux/skbuff.h         |  10 +-
 include/uapi/linux/io_uring.h  |  24 +
 io_uring/Makefile              |   3 +-
 io_uring/io_uring.c            |   8 +
 io_uring/kbuf.c                |  13 +
 io_uring/kbuf.h                |   2 +
 io_uring/net.c                 | 121 ++++
 io_uring/opdef.c               |  15 +
 io_uring/zctap.c               | 976 +++++++++++++++++++++++++++++++++
 io_uring/zctap.h               |  31 ++
 13 files changed, 1266 insertions(+), 2 deletions(-)
 create mode 100644 io_uring/zctap.c
 create mode 100644 io_uring/zctap.h
Signature · 1 line
2.30.2