Skip to content

Add support for providers with large rx buffer

Pavel Begunkov bpf io-uring linux-doc linux-kselftest linux-rdma lkml netdev 3 versions – Applied

Status from this message: This series was applied to netdev/net-next.git (main)

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v9 Latest9 patches 2 repliesApplied 9/9
  2. v89 patches 9 repliesSuperseded
  3. v79 patches 9 repliesSuperseded 2 of 9 applied

Patches in v9 All 9 applied

#Subject and filesSizeRepliesApplied
1/9net: memzero mp params when closing a queuenet/core/netdev_rx_queue.c+4 -60 repliesApplied
2/9net: reduce indent of struct netdev_queue_mgmt_ops membersinclude/net/netdev_queues.h+14 -140 repliesApplied
3/9net: add bare bone queue configsdrivers/net/ethernet/broadcom/bnxt/bnxt.c drivers/net/ethernet/google/gve/gve_main.c drivers/net/ethernet/mellanox/mlx5/core/en_main.c +6 more+66 -160 repliesApplied
4/9net: pass queue rx page size from memory providerinclude/net/netdev_queues.h include/net/page_pool/types.h net/core/netdev_rx_queue.c+20 -00 repliesApplied
5/9eth: bnxt: store rx buffer size per queuedrivers/net/ethernet/broadcom/bnxt/bnxt.c drivers/net/ethernet/broadcom/bnxt/bnxt.h drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c +1 more+38 -270 repliesApplied
6/9eth: bnxt: adjust the fill level of agg queues with larger buffersdrivers/net/ethernet/broadcom/bnxt/bnxt.c+21 -40 repliesApplied
7/9eth: bnxt: support qcfg provided rx page sizedrivers/net/ethernet/broadcom/bnxt/bnxt.c drivers/net/ethernet/broadcom/bnxt/bnxt.h+36 -10 repliesApplied
8/9selftests: iou-zcrx: test large chunk sizestools/testing/selftests/drivers/net/hw/iou-zcrx.c tools/testing/selftests/drivers/net/hw/iou-zcrx.py+99 -120 repliesApplied
9/9io_uring/zcrx: document area chunking parameterDocumentation/networking/iou-zcrx.rst+20 -00 repliesApplied

Reviews trailers per patch

1 of 9 patches has a Reviewed-by or Acked-by.

PatchMina Almasry
1/9
2/9R-b
3/9
4/9
5/9
6/9
7/9
8/9
9/9

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v9 thread

Cover letter [PATCH net-next v9 0/9] Add support for providers with large rx buffer

Note: it's net/ only bits and doesn't include changes, which shoulf be merged separately and are posted separately. The full branch for convenience is at [1], and the patch is here:

https://lore.kernel.org/io-uring/7486ab32e99be1f614b3ef8d0e9bc77015b173f7.1764265323.git.asml.silence@gmail.com

Many modern NICs support configurable receive buffer lengths, and zcrx and memory providers can use buffers larger than 4K to improve performance. When paired with hw-gro larger rx buffer sizes can drastically reduce the number of buffers traversing the stack and save a lot of processing time. It also allows to give to users larger contiguous chunks of data. The idea was first floated around by Saeed during netdev conf 2024 and was asked about by a few folks.

Single stream benchmarks showed up to ~30% CPU util improvement. E.g. comparison for 4K vs 32K buffers using a 200Gbit NIC:

packets=23987040 (MB=2745098), rps=199559 (MB/s=22837)

CPU    %usr   %nice    %sys %iowait    %irq   %soft   %idle
  0    1.53    0.00   27.78    2.72    1.31   66.45    0.22
packets=24078368 (MB=2755550), rps=200319 (MB/s=22924)
CPU    %usr   %nice    %sys %iowait    %irq   %soft   %idle
  0    0.69    0.00    8.26   31.65    1.83   57.00    0.57

This series adds net infrastructure for memory providers configuring the size and implements it for bnxt. It's an opt-in feature for drivers, they should advertise support for the parameter in the qops and must check if the hardware supports the given size. It's limited to memory providers as it drastically simplifies implementation. It doesn't affect the fast path zcrx uAPI, and the user exposed parameter is defined in zcrx terms, which allows it to be flexible and adjusted in the future.

A liburing example can be found at [2]

full branch:
[1] https://github.com/isilence/linux.git zcrx/large-buffers-v8 Liburing example:
[2] https://github.com/isilence/liburing.git zcrx/rx-buf-len

---

The following changes since commit 0f61b1860cc3f52aef9036d7235ed1f017632193:

Linux 6.19-rc5 (2026-01-11 17:03:14 -1000)

are available in the Git repository at:

https://github.com/isilence/linux.git tags/net-queue-rx-buf-len-v9

for you to fetch changes up to d1de61db1536727c1cad049c09decff22e8b6dd7:

io_uring/zcrx: document area chunking parameter (2026-01-14 02:13:37 +0000)

v9: - correct nits from Paolo

v8: - Add stripped down qcfg
- Retain the page size across resets for bnxt

v7: - Add xa_destroy
- Rebase

v6: - Update docs and add a selftest

v5: https://lore.kernel.org/netdev/[email protected]/
- Remove all unnecessary bits like configuration via netlink, and multi-stage queue configuration.

v4: https://lore.kernel.org/all/[email protected]/
- Update fbnic qops
- Propagate max buf len for hns3
- Use configured buf size in __bnxt_alloc_rx_netmem
- Minor stylistic changes
v3: https://lore.kernel.org/all/[email protected]/
- Rebased, excluded zcrx specific patches
- Set agg_size_fac to 1 on warning
v2: https://lore.kernel.org/all/[email protected]/
- Add MAX_PAGE_ORDER check on pp init
- Applied comments rewording
- Adjust pp.max_len based on order
- Patch up mlx5 queue callbacks after rebase
- Minor ->queue_mgmt_ops refactoring
- Rebased to account for both fill level and agg_size_fac
- Pass providers buf length in struct pp_memory_provider_params and apply it in __netdev_queue_confi().
- Use ->supported_ring_params to validate drivers support of set qcfg parameters.

Jakub Kicinski (2):
net: reduce indent of struct netdev_queue_mgmt_ops members
eth: bnxt: adjust the fill level of agg queues with larger buffers

Pavel Begunkov (7):
net: memzero mp params when closing a queue
net: add bare bone queue configs
net: pass queue rx page size from memory provider
eth: bnxt: store rx buffer size per queue
eth: bnxt: support qcfg provided rx page size
selftests: iou-zcrx: test large chunk sizes io_uring/zcrx: document area chunking parameter

 Documentation/networking/iou-zcrx.rst         |  20 +++
 drivers/net/ethernet/broadcom/bnxt/bnxt.c     | 123 +++++++++++++-----
 drivers/net/ethernet/broadcom/bnxt/bnxt.h     |   2 +
 drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c |   6 +-
 drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.h |   2 +-
 drivers/net/ethernet/google/gve/gve_main.c    |   9 +-
 .../net/ethernet/mellanox/mlx5/core/en_main.c |  10 +-
 drivers/net/ethernet/meta/fbnic/fbnic_txrx.c  |   8 +-
 drivers/net/netdevsim/netdev.c                |   7 +-
 include/net/netdev_queues.h                   |  47 +++++--
 include/net/netdev_rx_queue.h                 |   2 +
 include/net/page_pool/types.h                 |   1 +
 net/core/dev.c                                |  17 +++
 net/core/netdev_rx_queue.c                    |  31 +++--
 .../selftests/drivers/net/hw/iou-zcrx.c       |  72 ++++++++--
 .../selftests/drivers/net/hw/iou-zcrx.py      |  39 ++++++
 16 files changed, 317 insertions(+), 79 deletions(-)
Signature · 1 line
2.52.0