Skip to content

net/mlx5e: Add support for devmem and io_uring TCP zero-copy

Mark Bloch bpf linux-rdma lkml netdev 4 versions – Applied

Status from this message: This series was applied to netdev/net-next.git (main)

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v6 Latest12 patches 8 repliesApplied 12/12
  2. v512 patches 1 replySuperseded
  3. v411 patches 11 repliesSuperseded
Earlier versions · 1
VersionPostedPatchesRepliesStatus
v312 patches7 repliesSuperseded

Patches in v6 All 12 applied

#Subject and filesSizeRepliesApplied
1/12net: Allow const args for of page_to_netmem()include/net/netmem.h+1 -10 repliesApplied
2/12net: Add skb_can_coalesce for netmeminclude/linux/skbuff.h+9 -30 repliesApplied
3/12page_pool: Add page_pool_dev_alloc_netmems helperinclude/net/page_pool/helpers.h+7 -00 repliesApplied
4/12net/mlx5e: SHAMPO: Reorganize mlx5_rq_shampo_allocdrivers/net/ethernet/mellanox/mlx5/core/en.h drivers/net/ethernet/mellanox/mlx5/core/en_main.c+66 -700 repliesApplied
5/12net/mlx5e: SHAMPO: Remove redundant paramsdrivers/net/ethernet/mellanox/mlx5/core/en.h drivers/net/ethernet/mellanox/mlx5/core/en/params.c drivers/net/ethernet/mellanox/mlx5/core/en_main.c+20 -240 repliesApplied
6/12net/mlx5e: SHAMPO: Improve hw gro capability checkingdrivers/net/ethernet/mellanox/mlx5/core/en_main.c+7 -60 repliesApplied
7/12net/mlx5e: SHAMPO: Separate pool for headersdrivers/net/ethernet/mellanox/mlx5/core/en.h drivers/net/ethernet/mellanox/mlx5/core/en_main.c drivers/net/ethernet/mellanox/mlx5/core/en_rx.c+69 -190 repliesApplied
8/12net/mlx5e: Convert over to netmemdrivers/net/ethernet/mellanox/mlx5/core/en.h drivers/net/ethernet/mellanox/mlx5/core/en_rx.c+63 -440 repliesApplied
9/12net/mlx5e: Add support for UNREADABLE netmem page poolsdrivers/net/ethernet/mellanox/mlx5/core/en_main.c+8 -10 repliesApplied
10/12net/mlx5e: Implement queue mgmt ops and single channel swapdrivers/net/ethernet/mellanox/mlx5/core/en_main.c+98 -02 repliesApplied
11/12net/mlx5e: Support ethtool tcp-data-split settingsdrivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c drivers/net/ethernet/mellanox/mlx5/core/en_main.c+36 -50 repliesApplied
12/12net/mlx5e: Add TX support for netmemsdrivers/net/ethernet/mellanox/mlx5/core/en/txrx.h drivers/net/ethernet/mellanox/mlx5/core/en_main.c+4 -16 repliesApplied

Reviews trailers per patch

All 12 patches have a Reviewed-by or Acked-by.

PatchMina AlmasryCosmin RatiuTariq ToukanDragos Tatulea
1/12R-b R-b
2/12R-b R-b
3/12R-b
4/12R-b R-b
5/12R-b
6/12R-b
7/12R-b
8/12R-b R-b R-b
9/12R-b R-b R-b
10/12A-b R-b R-b
11/12R-b R-b
12/12R-b R-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 4 reviewers.

Read the v6 thread

Cover letter [PATCH net-next v6 00/12] net/mlx5e: Add support for devmem and io_uring TCP zero-copy

This series adds support for zerocopy rx TCP with devmem and io_uring for ConnectX7 NICs and above. For performance reasons and simplicity HW-GRO will also be turned on when header-data split mode is on.

Performance
===========

Test setup:

* CPU: Intel(R) Xeon(R) Platinum 8380 CPU @ 2.30GHz (single NUMA)
* NIC: ConnectX7
* Benchmarking tool: kperf [0]
* Single TCP flow
* Test duration: 60s

With application thread and interrupts pinned to the *same* core:

|------+-----------+----------|
| MTU  | epoll     | io_uring |
|------+-----------+----------|
| 1500 | 61.6 Gbps | 114 Gbps |
| 4096 | 69.3 Gbps | 151 Gbps |
| 9000 | 67.8 Gbps | 187 Gbps |
|------+-----------+----------|

The CPU usage for io_uring is 95%.

Reproduction steps for io_uring:

server --no-daemon -a 2001:db8::1 --no-memcmp --iou --iou_sendzc \
	--iou_zcrx --iou_dev_name eth2 --iou_zcrx_queue_id 2

server --no-daemon -a 2001:db8::2 --no-memcmp --iou --iou_sendzc

client --src 2001:db8::2 --dst 2001:db8::1 \
	--msg-zerocopy -t 60 --cpu-min=2 --cpu-max=2

Patch overview:
================

First, a netmem API for skb_can_coalesce is added to the core to be able to do skb fragment coalescing on netmems.

The next patches introduce some cleanups in the internal SHAMPO code and improvements to hw gro capability checks in FW.

A separate page_pool is introduced for headers, to be used only when the rxq has a memory provider.

Then the driver is converted to use the netmem API and to allow support for unreadable netmem page pool.

The queue management ops are implemented.

Finally, the tcp-data-split ring parameter is exposed.

Changelog
=========
Changes from v5 [5]:
- Added a TODO in mlx5e_queue_stop regarding queue restart.
- Added Reviewed-by tag.

Changes from v4 [4]:
- Addressed silly return before goto.
- Removed extraneous '\n' and used NL_SET_ERR_MSG_MOD.
- Removed unnecessary netmem_is_net_iov() check.
- Added comment for non HDS packets being dropped when unreadable memory is used.
- Added page_pool_dev_alloc_netmems() helper.
- Added Reviewed-by tags.

Changes from v3 [3]:
- Dropped ethtool stats for hd_page_pool.

Changes from v2 [2]:
- Added support for netmem TX.
- Changed skb_can_coalesce_netmem() based on Mina's suggestion.
- Reworked tcp_data_split setting to no longer change HW-GRO in wanted_features.
- Switched to a single page pool when rxq has no memory providers.

Changes from v1 [1]:
- Added support for skb_can_coalesce_netmem().
- Avoid netmem_to_page() casts in the driver.
- Fixed code to abide 80 char limit with some exceptions to avoid code churn.

References
==========
[0] kperf: git://git.kernel.dk/kperf.git
[1] v1: https://lore.kernel.org/all/[email protected]/
[2] v2: https://lore.kernel.org/all/[email protected]/
[3] v3: https://lore.kernel.org/netdev/[email protected]/
[4] v4: https://lore.kernel.org/all/[email protected]/
[5] v5: https://lore.kernel.org/netdev/[email protected]/

Dragos Tatulea (4):
net: Allow const args for of page_to_netmem()
net: Add skb_can_coalesce for netmem
page_pool: Add page_pool_dev_alloc_netmems helper net/mlx5e: Add TX support for netmems

Saeed Mahameed (8):
net/mlx5e: SHAMPO: Reorganize mlx5_rq_shampo_alloc net/mlx5e: SHAMPO: Remove redundant params net/mlx5e: SHAMPO: Improve hw gro capability checking net/mlx5e: SHAMPO: Separate pool for headers net/mlx5e: Convert over to netmem
net/mlx5e: Add support for UNREADABLE netmem page pools net/mlx5e: Implement queue mgmt ops and single channel swap net/mlx5e: Support ethtool tcp-data-split settings

 drivers/net/ethernet/mellanox/mlx5/core/en.h  |  11 +-
 .../ethernet/mellanox/mlx5/core/en/params.c   |  36 ++-
 .../net/ethernet/mellanox/mlx5/core/en/txrx.h |   3 +-
 .../ethernet/mellanox/mlx5/core/en_ethtool.c  |  33 +-
 .../net/ethernet/mellanox/mlx5/core/en_main.c | 306 +++++++++++++-----
 .../net/ethernet/mellanox/mlx5/core/en_rx.c   | 138 ++++----
 include/linux/skbuff.h                        |  12 +-
 include/net/netmem.h                          |   2 +-
 include/net/page_pool/helpers.h               |   7 +
 9 files changed, 381 insertions(+), 167 deletions(-)

base-commit: 8909f5f4ecd551c2299b28e05254b77424c8c7dc

Signature · 1 line
2.34.1