Skip to contentSkip to conversation

net/mlx5e: Add support for devmem and io_uring TCP zero-copy

PATCH Superseded Mark Bloch 4 participants 23 messages Jun 10, 2025 – Jun 12, 2025

Subsystems (5)

v4 of 6 Superseded Latest is v6 →

lore
Share & tools
lore

Mark Bloch

author maintainer
#0 lore raw

v4 cover net/mlx5e: Add support for devmem and io_uring TCP zero-copy

This series adds support for zerocopy rx TCP with devmem and io_uring for ConnectX7 NICs and above. For performance reasons and simplicity HW-GRO will also be turned on when header-data split mode is on.

Performance
===========

Test setup:

* CPU: Intel(R) Xeon(R) Platinum 8380 CPU @ 2.30GHz (single NUMA)
* NIC: ConnectX7
* Benchmarking tool: kperf [0]
* Single TCP flow
* Test duration: 60s

With application thread and interrupts pinned to the *same* core:

|------+-----------+----------|
| MTU  | epoll     | io_uring |
|------+-----------+----------|
| 1500 | 61.6 Gbps | 114 Gbps |
| 4096 | 69.3 Gbps | 151 Gbps |
| 9000 | 67.8 Gbps | 187 Gbps |
|------+-----------+----------|

The CPU usage for io_uring is 95%.

Reproduction steps for io_uring:

server --no-daemon -a 2001:db8::1 --no-memcmp --iou --iou_sendzc \
	--iou_zcrx --iou_dev_name eth2 --iou_zcrx_queue_id 2

server --no-daemon -a 2001:db8::2 --no-memcmp --iou --iou_sendzc

client --src 2001:db8::2 --dst 2001:db8::1 \
	--msg-zerocopy -t 60 --cpu-min=2 --cpu-max=2

Patch overview:
================

First, a netmem API for skb_can_coalesce is added to the core to be able to do skb fragment coalescing on netmems.

The next patches introduce some cleanups in the internal SHAMPO code and improvements to hw gro capability checks in FW.

A separate page_pool is introduced for headers, to be used only when the rxq has a memory provider.

Then the driver is converted to use the netmem API and to allow support for unreadable netmem page pool.

The queue management ops are implemented.

Finally, the tcp-data-split ring parameter is exposed.

Changelog
=========

Changes from v3 [3]:
- Dropped ethtool stats for hd_page_pool.

Changes from v2 [2]:
- Added support for netmem TX.
- Changed skb_can_coalesce_netmem() based on Mina's suggestion.
- Reworked tcp_data_split setting to no longer change HW-GRO in wanted_features.
- Switched to a single page pool when rxq has no memory providers.

Changes from v1 [1]:
- Added support for skb_can_coalesce_netmem().
- Avoid netmem_to_page() casts in the driver.
- Fixed code to abide 80 char limit with some exceptions to avoid code churn.

References
==========
[0] kperf: git://git.kernel.dk/kperf.git
[1] v1: https://lore.kernel.org/all/[email protected]/
[2] v2: https://lore.kernel.org/all/[email protected]/
[3] v3: https://lore.kernel.org/netdev/[email protected]/

Dragos Tatulea (3):
net: Allow const args for of page_to_netmem()
net: Add skb_can_coalesce for netmem
net/mlx5e: Add TX support for netmems

Saeed Mahameed (8):
net/mlx5e: SHAMPO: Reorganize mlx5_rq_shampo_alloc net/mlx5e: SHAMPO: Remove redundant params net/mlx5e: SHAMPO: Improve hw gro capability checking net/mlx5e: SHAMPO: Separate pool for headers net/mlx5e: Convert over to netmem
net/mlx5e: Add support for UNREADABLE netmem page pools net/mlx5e: Implement queue mgmt ops and single channel swap net/mlx5e: Support ethtool tcp-data-split settings

 drivers/net/ethernet/mellanox/mlx5/core/en.h  |  11 +-
 .../ethernet/mellanox/mlx5/core/en/params.c   |  36 ++-
 .../net/ethernet/mellanox/mlx5/core/en/txrx.h |   3 +-
 .../ethernet/mellanox/mlx5/core/en_ethtool.c  |  33 +-
 .../net/ethernet/mellanox/mlx5/core/en_main.c | 305 +++++++++++++-----
 .../net/ethernet/mellanox/mlx5/core/en_rx.c   | 136 ++++----
 include/linux/skbuff.h                        |  12 +-
 include/net/netmem.h                          |   2 +-
 8 files changed, 371 insertions(+), 167 deletions(-)

base-commit: 2c7e4a2663a1ab5a740c59c31991579b6b865a26

Signature · 1 line
2.34.1
author maintainer
#1 lore raw

PATCH v4 1/11 net: Allow const args for of page_to_netmem()

From: Dragos Tatulea <[email protected]>

This allows calling page_to_netmem() with a const page * argument.

---
 include/net/netmem.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
include/net/netmem.h +1 -1
index 386164fb9c18..caf2e773c915 100644
@@ -143,7 +143,7 @@ static inline netmem_ref net_iov_to_netmem(struct net_iov *niov)
 	return (__force netmem_ref)((unsigned long)niov | NET_IOV);
 }
 
-static inline netmem_ref page_to_netmem(struct page *page)
+static inline netmem_ref page_to_netmem(const struct page *page)
 {
 	return (__force netmem_ref)page;
 }
Signature · 1 line
2.34.1
Reviewed-by on v4 1/11
#2 lore raw

Re: [PATCH net-next v4 01/11] net: Allow const args for of page_to_netmem()

Quoted 8 lines · Mark Bloch

From: Dragos Tatulea <[email protected]>

This allows calling page_to_netmem() with a const page * argument.

This is slightly better, it returns a const netmem_ref if const struct page:

https://lore.kernel.org/netdev/[email protected]/

It's probably too much of a hassle to block your series until Byungchul's change to this helper goes in for you, so this seems fine to me in the interim.

Signature · 2 lines
Thanks,
Mina
#3 lore raw

Re: [PATCH net-next v4 01/11] net: Allow const args for of page_to_netmem()y

Quoted 6 lines · Mina Almasry
Quoted 5 lines · Mark Bloch

From: Dragos Tatulea <[email protected]>

This allows calling page_to_netmem() with a const page * argument.

Quoted 8 lines
Quoted 3 lines

This is slightly better, it returns a const netmem_ref if const struct page:

https://lore.kernel.org/netdev/[email protected]/

Agree. Saw it too late though...

Quoted 6 lines

It's probably too much of a hassle to block your series until Byungchul's change to this helper goes in for you, so this seems fine to me in the interim.

Thanks for the review!

Dragos

author maintainer
#4 lore raw

PATCH v4 2/11 net: Add skb_can_coalesce for netmem

In reply to Mark Bloch

From: Dragos Tatulea <[email protected]>

Allow drivers that have moved over to netmem to do fragment coalescing.

---
 include/linux/skbuff.h | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)
include/linux/skbuff.h +9 -3
index 5520524c93bf..9508968cb300 100644
@@ -3873,20 +3873,26 @@ static inline int __must_check skb_put_padto(struct sk_buff *skb, unsigned int l
 bool csum_and_copy_from_iter_full(void *addr, size_t bytes, __wsum *csum, struct iov_iter *i)
 	__must_check;
 
-static inline bool skb_can_coalesce(struct sk_buff *skb, int i,
-				    const struct page *page, int off)
+static inline bool skb_can_coalesce_netmem(struct sk_buff *skb, int i,
+					   netmem_ref netmem, int off)
 {
 	if (skb_zcopy(skb))
 		return false;
 	if (i) {
 		const skb_frag_t *frag = &skb_shinfo(skb)->frags[i - 1];
 
-		return page == skb_frag_page(frag) &&
+		return netmem == skb_frag_netmem(frag) &&
 		       off == skb_frag_off(frag) + skb_frag_size(frag);
 	}
 	return false;
 }
 
+static inline bool skb_can_coalesce(struct sk_buff *skb, int i,
+				    const struct page *page, int off)
+{
+	return skb_can_coalesce_netmem(skb, i, page_to_netmem(page), off);
+}
+
 static inline int __skb_linearize(struct sk_buff *skb)
 {
 	return __pskb_pull_tail(skb, skb->data_len) ? 0 : -ENOMEM;
Signature · 1 line
2.34.1
Reviewed-by on v4 2/11
#5 lore raw

Re: [PATCH net-next v4 02/11] net: Add skb_can_coalesce for netmem

Quoted 9 lines · Mark Bloch

From: Dragos Tatulea <[email protected]>

Allow drivers that have moved over to netmem to do fragment coalescing.

Signature · 2 lines
Thanks,
Mina
author maintainer
#6 lore raw

PATCH v4 3/11 net/mlx5e: SHAMPO: Reorganize mlx5_rq_shampo_alloc

In reply to Mark Bloch

From: Saeed Mahameed <[email protected]>

Drop redundant SHAMPO structure alloc/free functions.

Gather together function calls pertaining to header split info, pass header per WQE (hd_per_wqe) as parameter to those function to avoid use before initialization future mistakes.

Allocate HW GRO related info outside of the header related info scope.

---
 drivers/net/ethernet/mellanox/mlx5/core/en.h  |   1 -
 .../net/ethernet/mellanox/mlx5/core/en_main.c | 135 +++++++++---------
 2 files changed, 66 insertions(+), 70 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en.h +0 -1
index 5b0d03b3efe8..211ea429ea89 100644
@@ -638,7 +638,6 @@ struct mlx5e_shampo_hd {
 	struct mlx5e_frag_page *pages;
 	u32 hd_per_wq;
 	u16 hd_per_wqe;
-	u16 pages_per_wq;
 	unsigned long *bitmap;
 	u16 pi;
 	u16 ci;
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +66 -69

187 lines not on this page · raw message

Signature · 1 line
2.34.1
author maintainer
#7 lore raw

PATCH v4 4/11 net/mlx5e: SHAMPO: Remove redundant params

In reply to Mark Bloch

From: Saeed Mahameed <[email protected]>

Two SHAMPO params are static and always the same, remove them from the global mlx5e_params struct.

---
 drivers/net/ethernet/mellanox/mlx5/core/en.h  |  4 ---
 .../ethernet/mellanox/mlx5/core/en/params.c   | 36 ++++++++++---------
 .../net/ethernet/mellanox/mlx5/core/en_main.c |  4 ---
 3 files changed, 20 insertions(+), 24 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en.h +0 -4
index 211ea429ea89..581eef34f512 100644
@@ -278,10 +278,6 @@ enum packet_merge {
 struct mlx5e_packet_merge_param {
 	enum packet_merge type;
 	u32 timeout;
-	struct {
-		u8 match_criteria_type;
-		u8 alignment_granularity;
-	} shampo;
 };
 
 struct mlx5e_params {
drivers/net/ethernet/mellanox/mlx5/core/en/params.c +20 -16
index 58ec5e44aa7a..fc945bce933a 100644
@@ -901,6 +901,7 @@ int mlx5e_build_rq_param(struct mlx5_core_dev *mdev,
 {
 	void *rqc = param->rqc;
 	void *wq = MLX5_ADDR_OF(rqc, rqc, wq);
+	u32 lro_timeout;
 	int ndsegs = 1;
 	int err;
 
@@ -926,22 +927,25 @@ int mlx5e_build_rq_param(struct mlx5_core_dev *mdev,
 		MLX5_SET(wq, wq, log_wqe_stride_size,
 			 log_wqe_stride_size - MLX5_MPWQE_LOG_STRIDE_SZ_BASE);
 		MLX5_SET(wq, wq, log_wq_sz, mlx5e_mpwqe_get_log_rq_size(mdev, params, xsk));
-		if (params->packet_merge.type == MLX5E_PACKET_MERGE_SHAMPO) {
-			MLX5_SET(wq, wq, shampo_enable, true);
-			MLX5_SET(wq, wq, log_reservation_size,
-				 mlx5e_shampo_get_log_rsrv_size(mdev, params));
-			MLX5_SET(wq, wq,
-				 log_max_num_of_packets_per_reservation,
-				 mlx5e_shampo_get_log_pkt_per_rsrv(mdev, params));
-			MLX5_SET(wq, wq, log_headers_entry_size,
-				 mlx5e_shampo_get_log_hd_entry_size(mdev, params));
-			MLX5_SET(rqc, rqc, reservation_timeout,
-				 mlx5e_choose_lro_timeout(mdev, MLX5E_DEFAULT_SHAMPO_TIMEOUT));
-			MLX5_SET(rqc, rqc, shampo_match_criteria_type,
-				 params->packet_merge.shampo.match_criteria_type);
-			MLX5_SET(rqc, rqc, shampo_no_match_alignment_granularity,
-				 params->packet_merge.shampo.alignment_granularity);
-		}
+		if (params->packet_merge.type != MLX5E_PACKET_MERGE_SHAMPO)
+			break;
+
+		MLX5_SET(wq, wq, shampo_enable, true);
+		MLX5_SET(wq, wq, log_reservation_size,
+			 mlx5e_shampo_get_log_rsrv_size(mdev, params));
+		MLX5_SET(wq, wq,
+			 log_max_num_of_packets_per_reservation,
+			 mlx5e_shampo_get_log_pkt_per_rsrv(mdev, params));
+		MLX5_SET(wq, wq, log_headers_entry_size,
+			 mlx5e_shampo_get_log_hd_entry_size(mdev, params));
+		lro_timeout =
+			mlx5e_choose_lro_timeout(mdev,
+						 MLX5E_DEFAULT_SHAMPO_TIMEOUT);
+		MLX5_SET(rqc, rqc, reservation_timeout, lro_timeout);
+		MLX5_SET(rqc, rqc, shampo_match_criteria_type,
+			 MLX5_RQC_SHAMPO_MATCH_CRITERIA_TYPE_EXTENDED);
+		MLX5_SET(rqc, rqc, shampo_no_match_alignment_granularity,
+			 MLX5_RQC_SHAMPO_NO_MATCH_ALIGNMENT_GRANULARITY_STRIDE);
 		break;
 	}
 	default: /* MLX5_WQ_TYPE_CYCLIC */
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +0 -4
index 3d11c9f87171..e1e44533b744 100644
@@ -4040,10 +4040,6 @@ static int set_feature_hw_gro(struct net_device *netdev, bool enable)
 
 	if (enable) {
 		new_params.packet_merge.type = MLX5E_PACKET_MERGE_SHAMPO;
-		new_params.packet_merge.shampo.match_criteria_type =
-			MLX5_RQC_SHAMPO_MATCH_CRITERIA_TYPE_EXTENDED;
-		new_params.packet_merge.shampo.alignment_granularity =
-			MLX5_RQC_SHAMPO_NO_MATCH_ALIGNMENT_GRANULARITY_STRIDE;
 	} else if (new_params.packet_merge.type == MLX5E_PACKET_MERGE_SHAMPO) {
 		new_params.packet_merge.type = MLX5E_PACKET_MERGE_NONE;
 	} else {
Signature · 1 line
2.34.1
author maintainer
#8 lore raw

PATCH v4 5/11 net/mlx5e: SHAMPO: Improve hw gro capability checking

In reply to Mark Bloch

From: Saeed Mahameed <[email protected]>

Add missing HW capabilities, declare the feature in netdev->vlan_features, similar to other features in mlx5e_build_nic_netdev. No functional change here as all by default disabled features are explicitly disabled at the bottom of the function.

---
 drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +7 -6
index e1e44533b744..a81d354af7c8 100644
@@ -78,7 +78,8 @@
 
 static bool mlx5e_hw_gro_supported(struct mlx5_core_dev *mdev)
 {
-	if (!MLX5_CAP_GEN(mdev, shampo))
+	if (!MLX5_CAP_GEN(mdev, shampo) ||
+	    !MLX5_CAP_SHAMPO(mdev, shampo_header_split_data_merge))
 		return false;
 
 	/* Our HW-GRO implementation relies on "KSM Mkey" for
@@ -5499,17 +5500,17 @@ static void mlx5e_build_nic_netdev(struct net_device *netdev)
 						   MLX5E_MPWRQ_UMR_MODE_ALIGNED))
 		netdev->vlan_features    |= NETIF_F_LRO;
 
+	if (mlx5e_hw_gro_supported(mdev) &&
+	    mlx5e_check_fragmented_striding_rq_cap(mdev, PAGE_SHIFT,
+						   MLX5E_MPWRQ_UMR_MODE_ALIGNED))
+		netdev->vlan_features |= NETIF_F_GRO_HW;
+
 	netdev->hw_features       = netdev->vlan_features;
 	netdev->hw_features      |= NETIF_F_HW_VLAN_CTAG_TX;
 	netdev->hw_features      |= NETIF_F_HW_VLAN_CTAG_RX;
 	netdev->hw_features      |= NETIF_F_HW_VLAN_CTAG_FILTER;
 	netdev->hw_features      |= NETIF_F_HW_VLAN_STAG_TX;
 
-	if (mlx5e_hw_gro_supported(mdev) &&
-	    mlx5e_check_fragmented_striding_rq_cap(mdev, PAGE_SHIFT,
-						   MLX5E_MPWRQ_UMR_MODE_ALIGNED))
-		netdev->hw_features    |= NETIF_F_GRO_HW;
-
 	if (mlx5e_tunnel_any_tx_proto_supported(mdev)) {
 		netdev->hw_enc_features |= NETIF_F_HW_CSUM;
 		netdev->hw_enc_features |= NETIF_F_TSO;
Signature · 1 line
2.34.1
author maintainer
#9 lore raw

PATCH v4 6/11 net/mlx5e: SHAMPO: Separate pool for headers

In reply to Mark Bloch

From: Saeed Mahameed <[email protected]>

Allow allocating a separate page pool for headers when SHAMPO is on. This will be useful for adding support to zc page pool, which has to be different from the headers page pool.
For now, the pools are the same.

---
 drivers/net/ethernet/mellanox/mlx5/core/en.h  |  4 ++
 .../net/ethernet/mellanox/mlx5/core/en_main.c | 43 ++++++++++++++++++-
 .../net/ethernet/mellanox/mlx5/core/en_rx.c   | 41 ++++++++++--------
 3 files changed, 69 insertions(+), 19 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en.h +4 -0
index 581eef34f512..c329de1d4f0a 100644
@@ -716,7 +716,11 @@ struct mlx5e_rq {
 	struct bpf_prog __rcu *xdp_prog;
 	struct mlx5e_xdpsq    *xdpsq;
 	DECLARE_BITMAP(flags, 8);
+
+	/* page pools */
 	struct page_pool      *page_pool;
+	struct page_pool      *hd_page_pool;
+
 	struct mlx5e_xdp_buff mxbuf;
 
 	/* AF_XDP zero-copy */
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +41 -2

85 lines not on this page · raw message

drivers/net/ethernet/mellanox/mlx5/core/en_rx.c +24 -17

143 lines not on this page · raw message

Signature · 1 line
2.34.1
author maintainer
#10 lore raw

PATCH v4 7/11 net/mlx5e: Convert over to netmem

In reply to Mark Bloch

From: Saeed Mahameed <[email protected]>

mlx5e_page_frag holds the physical page itself, to naturally support zc page pools, remove physical page reference from mlx5 and replace it with netmem_ref, to avoid internal handling in mlx5 for net_iov backed pages.

SHAMPO can issue packets that are not split into header and data. These packets will be dropped if the data part resides in a net_iov as the driver can't read into this area.

No performance degradation observed.

---
 drivers/net/ethernet/mellanox/mlx5/core/en.h  |   2 +-
 .../net/ethernet/mellanox/mlx5/core/en_rx.c   | 103 ++++++++++--------
 2 files changed, 61 insertions(+), 44 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en.h +1 -1
index c329de1d4f0a..65a73913b9a2 100644
@@ -553,7 +553,7 @@ struct mlx5e_icosq {
 } ____cacheline_aligned_in_smp;
 
 struct mlx5e_frag_page {
-	struct page *page;
+	netmem_ref netmem;
 	u16 frags;
 };
 
drivers/net/ethernet/mellanox/mlx5/core/en_rx.c +60 -43

253 lines not on this page · raw message

Signature · 1 line
2.34.1
Reviewed-by on v4 7/11
#11 lore raw

Re: [PATCH net-next v4 07/11] net/mlx5e: Convert over to netmem

Quoted 43 lines · Mark Bloch

From: Saeed Mahameed <[email protected]>

mlx5e_page_frag holds the physical page itself, to naturally support zc page pools, remove physical page reference from mlx5 and replace it with netmem_ref, to avoid internal handling in mlx5 for net_iov backed pages.

SHAMPO can issue packets that are not split into header and data. These packets will be dropped if the data part resides in a net_iov as the driver can't read into this area.

No performance degradation observed.

---
 drivers/net/ethernet/mellanox/mlx5/core/en.h  |   2 +-
 .../net/ethernet/mellanox/mlx5/core/en_rx.c   | 103 ++++++++++--------
 2 files changed, 61 insertions(+), 44 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en.h +1 -1
index c329de1d4f0a..65a73913b9a2 100644
@@ -553,7 +553,7 @@ struct mlx5e_icosq {
 } ____cacheline_aligned_in_smp;
 
 struct mlx5e_frag_page {
-       struct page *page;
+       netmem_ref netmem;
        u16 frags;
 };
 
drivers/net/ethernet/mellanox/mlx5/core/en_rx.c +0 -0
index e34ef53ebd0e..75e753adedef 100644
@@ -273,33 +273,33 @@ static inline u32 mlx5e_decompress_cqes_start(struct mlx5e_rq *rq,
 

drivers/net/ethernet/mellanox/mlx5/core/en_rx.c static inline u32 mlx5e_decompress_cqes_start(struct mlx5e_rq *rq, · line 281

Quoted 10 lines
drivers/net/ethernet/mellanox/mlx5/core/en_rx.c +3 -2
 #define MLX5E_PAGECNT_BIAS_MAX (PAGE_SIZE / 64)
 
-static int mlx5e_page_alloc_fragmented(struct page_pool *pool,
+static int mlx5e_page_alloc_fragmented(struct page_pool *pp,
                                       struct mlx5e_frag_page *frag_page)
 {
-       struct page *page;
+       netmem_ref netmem = page_pool_alloc_netmems(pp,
+                                                   GFP_ATOMIC | __GFP_NOWARN);
 

I would prefer if you add page_pool_dev_alloc_netmems helper for all drivers to use rather than specify the GFP params inline here.

Quoted 54 lines
drivers/net/ethernet/mellanox/mlx5/core/en_rx.c +10 -10
-       page = page_pool_dev_alloc_pages(pool);
-       if (unlikely(!page))
+       if (unlikely(!netmem))
                return -ENOMEM;
 
-       page_pool_fragment_page(page, MLX5E_PAGECNT_BIAS_MAX);
+       page_pool_fragment_netmem(netmem, MLX5E_PAGECNT_BIAS_MAX);
 
        *frag_page = (struct mlx5e_frag_page) {
-               .page   = page,
+               .netmem = netmem,
                .frags  = 0,
        };
 
        return 0;
 }
 
-static void mlx5e_page_release_fragmented(struct page_pool *pool,
+static void mlx5e_page_release_fragmented(struct page_pool *pp,
                                          struct mlx5e_frag_page *frag_page)
 {
        u16 drain_count = MLX5E_PAGECNT_BIAS_MAX - frag_page->frags;
-       struct page *page = frag_page->page;
+       netmem_ref netmem = frag_page->netmem;
 
-       if (page_pool_unref_page(page, drain_count) == 0)
-               page_pool_put_unrefed_page(pool, page, -1, true);
+       if (page_pool_unref_netmem(netmem, drain_count) == 0)
+               page_pool_put_unrefed_netmem(pp, netmem, -1, true);
 }
 
 static inline int mlx5e_get_rx_frag(struct mlx5e_rq *rq,
@@ -359,7 +359,7 @@ static int mlx5e_alloc_rx_wqe(struct mlx5e_rq *rq, struct mlx5e_rx_wqe_cyc *wqe,
                frag->flags &= ~BIT(MLX5E_WQE_FRAG_SKIP_RELEASE);
 
                headroom = i == 0 ? rq->buff.headroom : 0;
-               addr = page_pool_get_dma_addr(frag->frag_page->page);
+               addr = page_pool_get_dma_addr_netmem(frag->frag_page->netmem);
                wqe->data[i].addr = cpu_to_be64(addr + frag->offset + headroom);
        }
 
@@ -500,9 +500,10 @@ mlx5e_add_skb_shared_info_frag(struct mlx5e_rq *rq, struct skb_shared_info *sinf
                               struct xdp_buff *xdp, struct mlx5e_frag_page *frag_page,
                               u32 frag_offset, u32 len)
 {
+       netmem_ref netmem = frag_page->netmem;
        skb_frag_t *frag;
 
-       dma_addr_t addr = page_pool_get_dma_addr(frag_page->page);
+       dma_addr_t addr = page_pool_get_dma_addr_netmem(netmem);
 
        dma_sync_single_for_cpu(rq->pdev, addr + frag_offset, len, rq->buff.map_dir);
        if (!xdp_buff_has_frags(xdp)) {

drivers/net/ethernet/mellanox/mlx5/core/en_rx.c mlx5e_add_skb_shared_info_frag(struct mlx5e_rq *rq, struct skb_shared_info *sinf · line 521

Quoted 8 lines
drivers/net/ethernet/mellanox/mlx5/core/en_rx.c +2 -2
@@ -515,9 +516,9 @@ mlx5e_add_skb_shared_info_frag(struct mlx5e_rq *rq, struct skb_shared_info *sinf
        }
 
        frag = &sinfo->frags[sinfo->nr_frags++];
-       skb_frag_fill_page_desc(frag, frag_page->page, frag_offset, len);
+       skb_frag_fill_netmem_desc(frag, netmem, frag_offset, len);
 
-       if (page_is_pfmemalloc(frag_page->page))
+       if (!netmem_is_net_iov(netmem) && netmem_is_pfmemalloc(netmem))

Unnecessary netmem_is_net_iov check.

Quoted 176 lines
drivers/net/ethernet/mellanox/mlx5/core/en_rx.c +33 -26
                xdp_buff_set_frag_pfmemalloc(xdp);
        sinfo->xdp_frags_size += len;
 }
@@ -528,27 +529,29 @@ mlx5e_add_skb_frag(struct mlx5e_rq *rq, struct sk_buff *skb,
                   u32 frag_offset, u32 len,
                   unsigned int truesize)
 {
-       dma_addr_t addr = page_pool_get_dma_addr(frag_page->page);
+       dma_addr_t addr = page_pool_get_dma_addr_netmem(frag_page->netmem);
        u8 next_frag = skb_shinfo(skb)->nr_frags;
+       netmem_ref netmem = frag_page->netmem;
 
        dma_sync_single_for_cpu(rq->pdev, addr + frag_offset, len,
                                rq->buff.map_dir);
 
-       if (skb_can_coalesce(skb, next_frag, frag_page->page, frag_offset)) {
+       if (skb_can_coalesce_netmem(skb, next_frag, netmem, frag_offset)) {
                skb_coalesce_rx_frag(skb, next_frag - 1, len, truesize);
-       } else {
-               frag_page->frags++;
-               skb_add_rx_frag(skb, next_frag, frag_page->page,
-                               frag_offset, len, truesize);
+               return;
        }
+
+       frag_page->frags++;
+       skb_add_rx_frag_netmem(skb, next_frag, netmem,
+                              frag_offset, len, truesize);
 }
 
 static inline void
 mlx5e_copy_skb_header(struct mlx5e_rq *rq, struct sk_buff *skb,
-                     struct page *page, dma_addr_t addr,
+                     netmem_ref netmem, dma_addr_t addr,
                      int offset_from, int dma_offset, u32 headlen)
 {
-       const void *from = page_address(page) + offset_from;
+       const void *from = netmem_address(netmem) + offset_from;
        /* Aligning len to sizeof(long) optimizes memcpy performance */
        unsigned int len = ALIGN(headlen, sizeof(long));
 
@@ -685,7 +688,7 @@ static int mlx5e_build_shampo_hd_umr(struct mlx5e_rq *rq,
                if (unlikely(err))
                        goto err_unmap;
 
-               addr = page_pool_get_dma_addr(frag_page->page);
+               addr = page_pool_get_dma_addr_netmem(frag_page->netmem);
 
                for (int j = 0; j < MLX5E_SHAMPO_WQ_HEADER_PER_PAGE; j++) {
                        header_offset = mlx5e_shampo_hd_offset(index++);
@@ -796,7 +799,8 @@ static int mlx5e_alloc_rx_mpwqe(struct mlx5e_rq *rq, u16 ix)
                err = mlx5e_page_alloc_fragmented(rq->page_pool, frag_page);
                if (unlikely(err))
                        goto err_unmap;
-               addr = page_pool_get_dma_addr(frag_page->page);
+
+               addr = page_pool_get_dma_addr_netmem(frag_page->netmem);
                umr_wqe->inline_mtts[i] = (struct mlx5_mtt) {
                        .ptag = cpu_to_be64(addr | MLX5_EN_WR),
                };
@@ -1216,7 +1220,7 @@ static void *mlx5e_shampo_get_packet_hd(struct mlx5e_rq *rq, u16 header_index)
        struct mlx5e_frag_page *frag_page = mlx5e_shampo_hd_to_frag_page(rq, header_index);
        u16 head_offset = mlx5e_shampo_hd_offset(header_index) + rq->buff.headroom;
 
-       return page_address(frag_page->page) + head_offset;
+       return netmem_address(frag_page->netmem) + head_offset;
 }
 
 static void mlx5e_shampo_update_ipv4_udp_hdr(struct mlx5e_rq *rq, struct iphdr *ipv4)
@@ -1677,11 +1681,11 @@ mlx5e_skb_from_cqe_linear(struct mlx5e_rq *rq, struct mlx5e_wqe_frag_info *wi,
        dma_addr_t addr;
        u32 frag_size;
 
-       va             = page_address(frag_page->page) + wi->offset;
+       va             = netmem_address(frag_page->netmem) + wi->offset;
        data           = va + rx_headroom;
        frag_size      = MLX5_SKB_FRAG_SZ(rx_headroom + cqe_bcnt);
 
-       addr = page_pool_get_dma_addr(frag_page->page);
+       addr = page_pool_get_dma_addr_netmem(frag_page->netmem);
        dma_sync_single_range_for_cpu(rq->pdev, addr, wi->offset,
                                      frag_size, rq->buff.map_dir);
        net_prefetch(data);
@@ -1731,10 +1735,10 @@ mlx5e_skb_from_cqe_nonlinear(struct mlx5e_rq *rq, struct mlx5e_wqe_frag_info *wi
 
        frag_page = wi->frag_page;
 
-       va = page_address(frag_page->page) + wi->offset;
+       va = netmem_address(frag_page->netmem) + wi->offset;
        frag_consumed_bytes = min_t(u32, frag_info->frag_size, cqe_bcnt);
 
-       addr = page_pool_get_dma_addr(frag_page->page);
+       addr = page_pool_get_dma_addr_netmem(frag_page->netmem);
        dma_sync_single_range_for_cpu(rq->pdev, addr, wi->offset,
                                      rq->buff.frame0_sz, rq->buff.map_dir);
        net_prefetchw(va); /* xdp_frame data area */
@@ -2007,13 +2011,14 @@ mlx5e_skb_from_cqe_mpwrq_nonlinear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *w
 
        if (prog) {
                /* area for bpf_xdp_[store|load]_bytes */
-               net_prefetchw(page_address(frag_page->page) + frag_offset);
+               net_prefetchw(netmem_address(frag_page->netmem) + frag_offset);
                if (unlikely(mlx5e_page_alloc_fragmented(rq->page_pool,
                                                         &wi->linear_page))) {
                        rq->stats->buff_alloc_err++;
                        return NULL;
                }
-               va = page_address(wi->linear_page.page);
+
+               va = netmem_address(wi->linear_page.netmem);
                net_prefetchw(va); /* xdp_frame data area */
                linear_hr = XDP_PACKET_HEADROOM;
                linear_data_len = 0;
@@ -2124,8 +2129,8 @@ mlx5e_skb_from_cqe_mpwrq_nonlinear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *w
                        while (++pagep < frag_page);
                }
                /* copy header */
-               addr = page_pool_get_dma_addr(head_page->page);
-               mlx5e_copy_skb_header(rq, skb, head_page->page, addr,
+               addr = page_pool_get_dma_addr_netmem(head_page->netmem);
+               mlx5e_copy_skb_header(rq, skb, head_page->netmem, addr,
                                      head_offset, head_offset, headlen);
                /* skb linear part was allocated with headlen and aligned to long */
                skb->tail += headlen;
@@ -2155,11 +2160,11 @@ mlx5e_skb_from_cqe_mpwrq_linear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi,
                return NULL;
        }
 
-       va             = page_address(frag_page->page) + head_offset;
+       va             = netmem_address(frag_page->netmem) + head_offset;
        data           = va + rx_headroom;
        frag_size      = MLX5_SKB_FRAG_SZ(rx_headroom + cqe_bcnt);
 
-       addr = page_pool_get_dma_addr(frag_page->page);
+       addr = page_pool_get_dma_addr_netmem(frag_page->netmem);
        dma_sync_single_range_for_cpu(rq->pdev, addr, head_offset,
                                      frag_size, rq->buff.map_dir);
        net_prefetch(data);
@@ -2198,16 +2203,19 @@ mlx5e_skb_from_cqe_shampo(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi,
                          struct mlx5_cqe64 *cqe, u16 header_index)
 {
        struct mlx5e_frag_page *frag_page = mlx5e_shampo_hd_to_frag_page(rq, header_index);
-       dma_addr_t page_dma_addr = page_pool_get_dma_addr(frag_page->page);
        u16 head_offset = mlx5e_shampo_hd_offset(header_index);
-       dma_addr_t dma_addr = page_dma_addr + head_offset;
        u16 head_size = cqe->shampo.header_size;
        u16 rx_headroom = rq->buff.headroom;
        struct sk_buff *skb = NULL;
+       dma_addr_t page_dma_addr;
+       dma_addr_t dma_addr;
        void *hdr, *data;
        u32 frag_size;
 
-       hdr             = page_address(frag_page->page) + head_offset;
+       page_dma_addr = page_pool_get_dma_addr_netmem(frag_page->netmem);
+       dma_addr = page_dma_addr + head_offset;
+
+       hdr             = netmem_address(frag_page->netmem) + head_offset;
        data            = hdr + rx_headroom;
        frag_size       = MLX5_SKB_FRAG_SZ(rx_headroom + head_size);
 
@@ -2232,7 +2240,7 @@ mlx5e_skb_from_cqe_shampo(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi,
                }
 
                net_prefetchw(skb->data);
-               mlx5e_copy_skb_header(rq, skb, frag_page->page, dma_addr,
+               mlx5e_copy_skb_header(rq, skb, frag_page->netmem, dma_addr,
                                      head_offset + rx_headroom,
                                      rx_headroom, head_size);
                /* skb linear part was allocated with headlen and aligned to long */
@@ -2326,11 +2334,20 @@ static void (struct mlx5e_rq *rq, struct mlx5_cq
        }
 
        if (!*skb) {
-               if (likely(head_size))
+               if (likely(head_size)) {

drivers/net/ethernet/mellanox/mlx5/core/en_rx.c static void (struct mlx5e_rq *rq, struct mlx5_cq · line 2344

Quoted 10 lines
drivers/net/ethernet/mellanox/mlx5/core/en_rx.c +6 -3
                        *skb = mlx5e_skb_from_cqe_shampo(rq, wi, cqe, header_index);
-               else
-                       *skb = mlx5e_skb_from_cqe_mpwrq_nonlinear(rq, wi, cqe, cqe_bcnt,
-                                                                 data_offset, page_idx);
+               } else {
+                       struct mlx5e_frag_page *frag_page;
+
+                       frag_page = &wi->alloc_units.frag_pages[page_idx];
+                       if (unlikely(netmem_is_net_iov(frag_page->netmem)))
+                               goto free_hd_entry;

If I understand correctly, when the code reaches here, head_size == 0, which means the packet was not split. And if netmem_is_net_iov == true, then the entire packet (including the header) has landed in unreadable memory so now you are going to drop the packet, right?

If my understanding is correct, this is all subtle enough that it maybe warrants a comment. Also you may want driver stats to see how often this happens (seems like a headersplit failure causing packet drops).

Mostly nits, so,

--
Thanks,
Mina

#12 lore raw

Re: [PATCH net-next v4 07/11] net/mlx5e: Convert over to netmem

Quoted 46 lines · Mina Almasry
Quoted 45 lines · Mark Bloch

From: Saeed Mahameed <[email protected]>

mlx5e_page_frag holds the physical page itself, to naturally support zc page pools, remove physical page reference from mlx5 and replace it with netmem_ref, to avoid internal handling in mlx5 for net_iov backed pages.

SHAMPO can issue packets that are not split into header and data. These packets will be dropped if the data part resides in a net_iov as the driver can't read into this area.

No performance degradation observed.

---
 drivers/net/ethernet/mellanox/mlx5/core/en.h  |   2 +-
 .../net/ethernet/mellanox/mlx5/core/en_rx.c   | 103 ++++++++++--------
 2 files changed, 61 insertions(+), 44 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en.h +1 -1
index c329de1d4f0a..65a73913b9a2 100644
@@ -553,7 +553,7 @@ struct mlx5e_icosq {
 } ____cacheline_aligned_in_smp;
 
 struct mlx5e_frag_page {
-       struct page *page;
+       netmem_ref netmem;
        u16 frags;
 };
 
drivers/net/ethernet/mellanox/mlx5/core/en_rx.c +0 -0
index e34ef53ebd0e..75e753adedef 100644
@@ -273,33 +273,33 @@ static inline u32 mlx5e_decompress_cqes_start(struct mlx5e_rq *rq,
 
 #define MLX5E_PAGECNT_BIAS_MAX (PAGE_SIZE / 64)
 
Quoted 12 lines

drivers/net/ethernet/mellanox/mlx5/core/en_rx.c static inline u32 mlx5e_decompress_cqes_start(struct mlx5e_rq *rq, · line 281

Quoted 8 lines
drivers/net/ethernet/mellanox/mlx5/core/en_rx.c +3 -2
-static int mlx5e_page_alloc_fragmented(struct page_pool *pool,
+static int mlx5e_page_alloc_fragmented(struct page_pool *pp,
                                       struct mlx5e_frag_page *frag_page)
 {
-       struct page *page;
+       netmem_ref netmem = page_pool_alloc_netmems(pp,
+                                                   GFP_ATOMIC | __GFP_NOWARN);
 

I would prefer if you add page_pool_dev_alloc_netmems helper for all drivers to use rather than specify the GFP params inline here.

Ack. Will do.

Quoted 56 lines
Quoted 56 lines
drivers/net/ethernet/mellanox/mlx5/core/en_rx.c +10 -10
-       page = page_pool_dev_alloc_pages(pool);
-       if (unlikely(!page))
+       if (unlikely(!netmem))
                return -ENOMEM;
 
-       page_pool_fragment_page(page, MLX5E_PAGECNT_BIAS_MAX);
+       page_pool_fragment_netmem(netmem, MLX5E_PAGECNT_BIAS_MAX);
 
        *frag_page = (struct mlx5e_frag_page) {
-               .page   = page,
+               .netmem = netmem,
                .frags  = 0,
        };
 
        return 0;
 }
 
-static void mlx5e_page_release_fragmented(struct page_pool *pool,
+static void mlx5e_page_release_fragmented(struct page_pool *pp,
                                          struct mlx5e_frag_page *frag_page)
 {
        u16 drain_count = MLX5E_PAGECNT_BIAS_MAX - frag_page->frags;
-       struct page *page = frag_page->page;
+       netmem_ref netmem = frag_page->netmem;
 
-       if (page_pool_unref_page(page, drain_count) == 0)
-               page_pool_put_unrefed_page(pool, page, -1, true);
+       if (page_pool_unref_netmem(netmem, drain_count) == 0)
+               page_pool_put_unrefed_netmem(pp, netmem, -1, true);
 }
 
 static inline int mlx5e_get_rx_frag(struct mlx5e_rq *rq,
@@ -359,7 +359,7 @@ static int mlx5e_alloc_rx_wqe(struct mlx5e_rq *rq, struct mlx5e_rx_wqe_cyc *wqe,
                frag->flags &= ~BIT(MLX5E_WQE_FRAG_SKIP_RELEASE);
 
                headroom = i == 0 ? rq->buff.headroom : 0;
-               addr = page_pool_get_dma_addr(frag->frag_page->page);
+               addr = page_pool_get_dma_addr_netmem(frag->frag_page->netmem);
                wqe->data[i].addr = cpu_to_be64(addr + frag->offset + headroom);
        }
 
@@ -500,9 +500,10 @@ mlx5e_add_skb_shared_info_frag(struct mlx5e_rq *rq, struct skb_shared_info *sinf
                               struct xdp_buff *xdp, struct mlx5e_frag_page *frag_page,
                               u32 frag_offset, u32 len)
 {
+       netmem_ref netmem = frag_page->netmem;
        skb_frag_t *frag;
 
-       dma_addr_t addr = page_pool_get_dma_addr(frag_page->page);
+       dma_addr_t addr = page_pool_get_dma_addr_netmem(netmem);
 
        dma_sync_single_for_cpu(rq->pdev, addr + frag_offset, len, rq->buff.map_dir);
        if (!xdp_buff_has_frags(xdp)) {
@@ -515,9 +516,9 @@ mlx5e_add_skb_shared_info_frag(struct mlx5e_rq *rq, struct skb_shared_info *sinf
        }
 
Quoted 9 lines

drivers/net/ethernet/mellanox/mlx5/core/en_rx.c mlx5e_add_skb_shared_info_frag(struct mlx5e_rq *rq, struct skb_shared_info *sinf · line 521

Quoted 6 lines
drivers/net/ethernet/mellanox/mlx5/core/en_rx.c +2 -2
        frag = &sinfo->frags[sinfo->nr_frags++];
-       skb_frag_fill_page_desc(frag, frag_page->page, frag_offset, len);
+       skb_frag_fill_netmem_desc(frag, netmem, frag_offset, len);
 
-       if (page_is_pfmemalloc(frag_page->page))
+       if (!netmem_is_net_iov(netmem) && netmem_is_pfmemalloc(netmem))

Unnecessary netmem_is_net_iov check.

Ah nice! Will do.

Quoted 183 lines
Quoted 183 lines
drivers/net/ethernet/mellanox/mlx5/core/en_rx.c +36 -29
                xdp_buff_set_frag_pfmemalloc(xdp);
        sinfo->xdp_frags_size += len;
 }
@@ -528,27 +529,29 @@ mlx5e_add_skb_frag(struct mlx5e_rq *rq, struct sk_buff *skb,
                   u32 frag_offset, u32 len,
                   unsigned int truesize)
 {
-       dma_addr_t addr = page_pool_get_dma_addr(frag_page->page);
+       dma_addr_t addr = page_pool_get_dma_addr_netmem(frag_page->netmem);
        u8 next_frag = skb_shinfo(skb)->nr_frags;
+       netmem_ref netmem = frag_page->netmem;
 
        dma_sync_single_for_cpu(rq->pdev, addr + frag_offset, len,
                                rq->buff.map_dir);
 
-       if (skb_can_coalesce(skb, next_frag, frag_page->page, frag_offset)) {
+       if (skb_can_coalesce_netmem(skb, next_frag, netmem, frag_offset)) {
                skb_coalesce_rx_frag(skb, next_frag - 1, len, truesize);
-       } else {
-               frag_page->frags++;
-               skb_add_rx_frag(skb, next_frag, frag_page->page,
-                               frag_offset, len, truesize);
+               return;
        }
+
+       frag_page->frags++;
+       skb_add_rx_frag_netmem(skb, next_frag, netmem,
+                              frag_offset, len, truesize);
 }
 
 static inline void
 mlx5e_copy_skb_header(struct mlx5e_rq *rq, struct sk_buff *skb,
-                     struct page *page, dma_addr_t addr,
+                     netmem_ref netmem, dma_addr_t addr,
                      int offset_from, int dma_offset, u32 headlen)
 {
-       const void *from = page_address(page) + offset_from;
+       const void *from = netmem_address(netmem) + offset_from;
        /* Aligning len to sizeof(long) optimizes memcpy performance */
        unsigned int len = ALIGN(headlen, sizeof(long));
 
@@ -685,7 +688,7 @@ static int mlx5e_build_shampo_hd_umr(struct mlx5e_rq *rq,
                if (unlikely(err))
                        goto err_unmap;
 
-               addr = page_pool_get_dma_addr(frag_page->page);
+               addr = page_pool_get_dma_addr_netmem(frag_page->netmem);
 
                for (int j = 0; j < MLX5E_SHAMPO_WQ_HEADER_PER_PAGE; j++) {
                        header_offset = mlx5e_shampo_hd_offset(index++);
@@ -796,7 +799,8 @@ static int mlx5e_alloc_rx_mpwqe(struct mlx5e_rq *rq, u16 ix)
                err = mlx5e_page_alloc_fragmented(rq->page_pool, frag_page);
                if (unlikely(err))
                        goto err_unmap;
-               addr = page_pool_get_dma_addr(frag_page->page);
+
+               addr = page_pool_get_dma_addr_netmem(frag_page->netmem);
                umr_wqe->inline_mtts[i] = (struct mlx5_mtt) {
                        .ptag = cpu_to_be64(addr | MLX5_EN_WR),
                };
@@ -1216,7 +1220,7 @@ static void *mlx5e_shampo_get_packet_hd(struct mlx5e_rq *rq, u16 header_index)
        struct mlx5e_frag_page *frag_page = mlx5e_shampo_hd_to_frag_page(rq, header_index);
        u16 head_offset = mlx5e_shampo_hd_offset(header_index) + rq->buff.headroom;
 
-       return page_address(frag_page->page) + head_offset;
+       return netmem_address(frag_page->netmem) + head_offset;
 }
 
 static void mlx5e_shampo_update_ipv4_udp_hdr(struct mlx5e_rq *rq, struct iphdr *ipv4)
@@ -1677,11 +1681,11 @@ mlx5e_skb_from_cqe_linear(struct mlx5e_rq *rq, struct mlx5e_wqe_frag_info *wi,
        dma_addr_t addr;
        u32 frag_size;
 
-       va             = page_address(frag_page->page) + wi->offset;
+       va             = netmem_address(frag_page->netmem) + wi->offset;
        data           = va + rx_headroom;
        frag_size      = MLX5_SKB_FRAG_SZ(rx_headroom + cqe_bcnt);
 
-       addr = page_pool_get_dma_addr(frag_page->page);
+       addr = page_pool_get_dma_addr_netmem(frag_page->netmem);
        dma_sync_single_range_for_cpu(rq->pdev, addr, wi->offset,
                                      frag_size, rq->buff.map_dir);
        net_prefetch(data);
@@ -1731,10 +1735,10 @@ mlx5e_skb_from_cqe_nonlinear(struct mlx5e_rq *rq, struct mlx5e_wqe_frag_info *wi
 
        frag_page = wi->frag_page;
 
-       va = page_address(frag_page->page) + wi->offset;
+       va = netmem_address(frag_page->netmem) + wi->offset;
        frag_consumed_bytes = min_t(u32, frag_info->frag_size, cqe_bcnt);
 
-       addr = page_pool_get_dma_addr(frag_page->page);
+       addr = page_pool_get_dma_addr_netmem(frag_page->netmem);
        dma_sync_single_range_for_cpu(rq->pdev, addr, wi->offset,
                                      rq->buff.frame0_sz, rq->buff.map_dir);
        net_prefetchw(va); /* xdp_frame data area */
@@ -2007,13 +2011,14 @@ mlx5e_skb_from_cqe_mpwrq_nonlinear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *w
 
        if (prog) {
                /* area for bpf_xdp_[store|load]_bytes */
-               net_prefetchw(page_address(frag_page->page) + frag_offset);
+               net_prefetchw(netmem_address(frag_page->netmem) + frag_offset);
                if (unlikely(mlx5e_page_alloc_fragmented(rq->page_pool,
                                                         &wi->linear_page))) {
                        rq->stats->buff_alloc_err++;
                        return NULL;
                }
-               va = page_address(wi->linear_page.page);
+
+               va = netmem_address(wi->linear_page.netmem);
                net_prefetchw(va); /* xdp_frame data area */
                linear_hr = XDP_PACKET_HEADROOM;
                linear_data_len = 0;
@@ -2124,8 +2129,8 @@ mlx5e_skb_from_cqe_mpwrq_nonlinear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *w
                        while (++pagep < frag_page);
                }
                /* copy header */
-               addr = page_pool_get_dma_addr(head_page->page);
-               mlx5e_copy_skb_header(rq, skb, head_page->page, addr,
+               addr = page_pool_get_dma_addr_netmem(head_page->netmem);
+               mlx5e_copy_skb_header(rq, skb, head_page->netmem, addr,
                                      head_offset, head_offset, headlen);
                /* skb linear part was allocated with headlen and aligned to long */
                skb->tail += headlen;
@@ -2155,11 +2160,11 @@ mlx5e_skb_from_cqe_mpwrq_linear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi,
                return NULL;
        }
 
-       va             = page_address(frag_page->page) + head_offset;
+       va             = netmem_address(frag_page->netmem) + head_offset;
        data           = va + rx_headroom;
        frag_size      = MLX5_SKB_FRAG_SZ(rx_headroom + cqe_bcnt);
 
-       addr = page_pool_get_dma_addr(frag_page->page);
+       addr = page_pool_get_dma_addr_netmem(frag_page->netmem);
        dma_sync_single_range_for_cpu(rq->pdev, addr, head_offset,
                                      frag_size, rq->buff.map_dir);
        net_prefetch(data);
@@ -2198,16 +2203,19 @@ mlx5e_skb_from_cqe_shampo(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi,
                          struct mlx5_cqe64 *cqe, u16 header_index)
 {
        struct mlx5e_frag_page *frag_page = mlx5e_shampo_hd_to_frag_page(rq, header_index);
-       dma_addr_t page_dma_addr = page_pool_get_dma_addr(frag_page->page);
        u16 head_offset = mlx5e_shampo_hd_offset(header_index);
-       dma_addr_t dma_addr = page_dma_addr + head_offset;
        u16 head_size = cqe->shampo.header_size;
        u16 rx_headroom = rq->buff.headroom;
        struct sk_buff *skb = NULL;
+       dma_addr_t page_dma_addr;
+       dma_addr_t dma_addr;
        void *hdr, *data;
        u32 frag_size;
 
-       hdr             = page_address(frag_page->page) + head_offset;
+       page_dma_addr = page_pool_get_dma_addr_netmem(frag_page->netmem);
+       dma_addr = page_dma_addr + head_offset;
+
+       hdr             = netmem_address(frag_page->netmem) + head_offset;
        data            = hdr + rx_headroom;
        frag_size       = MLX5_SKB_FRAG_SZ(rx_headroom + head_size);
 
@@ -2232,7 +2240,7 @@ mlx5e_skb_from_cqe_shampo(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi,
                }
 
                net_prefetchw(skb->data);
-               mlx5e_copy_skb_header(rq, skb, frag_page->page, dma_addr,
+               mlx5e_copy_skb_header(rq, skb, frag_page->netmem, dma_addr,
                                      head_offset + rx_headroom,
                                      rx_headroom, head_size);
                /* skb linear part was allocated with headlen and aligned to long */
@@ -2326,11 +2334,20 @@ static void (struct mlx5e_rq *rq, struct mlx5_cq
        }
 
        if (!*skb) {
-               if (likely(head_size))
+               if (likely(head_size)) {
                        *skb = mlx5e_skb_from_cqe_shampo(rq, wi, cqe, header_index);
-               else
-                       *skb = mlx5e_skb_from_cqe_mpwrq_nonlinear(rq, wi, cqe, cqe_bcnt,
-                                                                 data_offset, page_idx);
+               } else {
+                       struct mlx5e_frag_page *frag_page;
+
Quoted 9 lines

drivers/net/ethernet/mellanox/mlx5/core/en_rx.c static void mlx5e_handle_rx_cqe_mpwrq_shampo(struct mlx5e_rq *rq, struct mlx5_cq · line 2344

Quoted 3 lines
drivers/net/ethernet/mellanox/mlx5/core/en_rx.c +3 -0
+                       frag_page = &wi->alloc_units.frag_pages[page_idx];
+                       if (unlikely(netmem_is_net_iov(frag_page->netmem)))
+                               goto free_hd_entry;

If I understand correctly, when the code reaches here, head_size == 0, which means the packet was not split. And if netmem_is_net_iov == true, then the entire packet (including the header) has landed in unreadable memory so now you are going to drop the packet, right?

Yes.

Quoted 5 lines

If my understanding is correct, this is all subtle enough that it maybe warrants a comment. Also you may want driver stats to see how often this happens (seems like a headersplit failure causing packet drops).

I will add a comment. A counter is also useful, especially to detect when the steering was not configured properly and unexpected packets are landing on this queue. But I'd prefer we add it separately. Don't want to rush it.

Quoted 4 lines

Mostly nits, so,

Thanks for the review!

Thanks,
Dragos

author maintainer
#13 lore raw

PATCH v4 8/11 net/mlx5e: Add support for UNREADABLE netmem page pools

In reply to Mark Bloch

From: Saeed Mahameed <[email protected]>

On netdev_rx_queue_restart, a special type of page pool maybe expected.

In this patch declare support for UNREADABLE netmem iov pages in the pool params only when header data split shampo RQ mode is enabled, also set the queue index in the page pool params struct.

Shampo mode requirement: Without header split rx needs to peek at the data, we can't do UNREADABLE_NETMEM.

The patch also enables the use of a separate page pool for headers when a memory provider is installed for the queue, otherwise the same common page pool continues to be used.

---
 drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +8 -1
index 5e649705e35f..a51e204bd364 100644
@@ -749,7 +749,9 @@ static void mlx5e_rq_shampo_hd_info_free(struct mlx5e_rq *rq)
 
 static bool mlx5_rq_needs_separate_hd_pool(struct mlx5e_rq *rq)
 {
-	return false;
+	struct netdev_rx_queue *rxq = __netif_get_rx_queue(rq->netdev, rq->ix);
+
+	return !!rxq->mp_params.mp_ops;
 }
 
 static int mlx5_rq_shampo_alloc(struct mlx5_core_dev *mdev,
@@ -964,6 +966,11 @@ static int mlx5e_alloc_rq(struct mlx5e_params *params,
 		pp_params.netdev    = rq->netdev;
 		pp_params.dma_dir   = rq->buff.map_dir;
 		pp_params.max_len   = PAGE_SIZE;
+		pp_params.queue_idx = rq->ix;
+
+		/* Shampo header data split allow for unreadable netmem */
+		if (test_bit(MLX5E_RQ_STATE_SHAMPO, &rq->state))
+			pp_params.flags |= PP_FLAG_ALLOW_UNREADABLE_NETMEM;
 
 		/* page_pool can be used even when there is no rq->xdp_prog,
 		 * given page_pool does not handle DMA mapping there is no
Signature · 1 line
2.34.1
Reviewed-by on v4 8/11
#14 lore raw

Re: [PATCH net-next v4 08/11] net/mlx5e: Add support for UNREADABLE netmem page pools

Quoted 30 lines · Mark Bloch

From: Saeed Mahameed <[email protected]>

On netdev_rx_queue_restart, a special type of page pool maybe expected.

In this patch declare support for UNREADABLE netmem iov pages in the pool params only when header data split shampo RQ mode is enabled, also set the queue index in the page pool params struct.

Shampo mode requirement: Without header split rx needs to peek at the data, we can't do UNREADABLE_NETMEM.

The patch also enables the use of a separate page pool for headers when a memory provider is installed for the queue, otherwise the same common page pool continues to be used.

---
 drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +0 -0
index 5e649705e35f..a51e204bd364 100644

drivers/net/ethernet/mellanox/mlx5/core/en_main.c static void mlx5e_rq_shampo_hd_info_free(struct mlx5e_rq *rq) · line 754

Quoted 7 lines
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +3 -1
@@ -749,7 +749,9 @@ static void mlx5e_rq_shampo_hd_info_free(struct mlx5e_rq *rq)
 
 static bool mlx5_rq_needs_separate_hd_pool(struct mlx5e_rq *rq)
 {
-       return false;
+       struct netdev_rx_queue *rxq = __netif_get_rx_queue(rq->netdev, rq->ix);
+
+       return !!rxq->mp_params.mp_ops;

This is kinda assuming that all future memory providers will return unreadable memory, which is not a restriction I have in mind... in theory there is nothing wrong with memory providers that feed readable pages. Technically the right thing to do here is to define a new helper page_pool_is_readable() and have the mp report to the pp if it's all readable or not.

But all this sounds like a huge hassle for an unnecessary amount of future proofing, so I guess this is fine.

Signature · 2 lines
Thanks,
Mina
#15 lore raw

Re: [PATCH net-next v4 08/11] net/mlx5e: Add support for UNREADABLE netmem page pools

Quoted 32 lines · Mina Almasry
Quoted 31 lines · Mark Bloch

From: Saeed Mahameed <[email protected]>

On netdev_rx_queue_restart, a special type of page pool maybe expected.

In this patch declare support for UNREADABLE netmem iov pages in the pool params only when header data split shampo RQ mode is enabled, also set the queue index in the page pool params struct.

Shampo mode requirement: Without header split rx needs to peek at the data, we can't do UNREADABLE_NETMEM.

The patch also enables the use of a separate page pool for headers when a memory provider is installed for the queue, otherwise the same common page pool continues to be used.

---
 drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +0 -0
index 5e649705e35f..a51e204bd364 100644
@@ -749,7 +749,9 @@ static void mlx5e_rq_shampo_hd_info_free(struct mlx5e_rq *rq)
 
Quoted 14 lines

drivers/net/ethernet/mellanox/mlx5/core/en_main.c static void mlx5e_rq_shampo_hd_info_free(struct mlx5e_rq *rq) · line 754

Quoted 6 lines
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +3 -1
 static bool mlx5_rq_needs_separate_hd_pool(struct mlx5e_rq *rq)
 {
-       return false;
+       struct netdev_rx_queue *rxq = __netif_get_rx_queue(rq->netdev, rq->ix);
+
+       return !!rxq->mp_params.mp_ops;

This is kinda assuming that all future memory providers will return unreadable memory, which is not a restriction I have in mind... in theory there is nothing wrong with memory providers that feed readable pages. Technically the right thing to do here is to define a new helper page_pool_is_readable() and have the mp report to the pp if it's all readable or not.

The API is already there: page_pool_is_unreadable(). But it uses the same logic...

However, having a pp level API is a bit limiting: as Cosmin pointed out, mlx5 can't use it because it needs to know in advance if this page_pool is for unreadable memory to correctly size the data page_pool (with or without headers).

Quoted 5 lines

But all this sounds like a huge hassle for an unnecessary amount of future proofing, so I guess this is fine.

Thanks!
Dragos

#16 lore raw

Re: [PATCH net-next v4 08/11] net/mlx5e: Add support for UNREADABLE netmem page pools

Quoted 48 lines · Dragos Tatulea
Quoted 32 lines · Mina Almasry
Quoted 31 lines · Mark Bloch

From: Saeed Mahameed <[email protected]>

On netdev_rx_queue_restart, a special type of page pool maybe expected.

In this patch declare support for UNREADABLE netmem iov pages in the pool params only when header data split shampo RQ mode is enabled, also set the queue index in the page pool params struct.

Shampo mode requirement: Without header split rx needs to peek at the data, we can't do UNREADABLE_NETMEM.

The patch also enables the use of a separate page pool for headers when a memory provider is installed for the queue, otherwise the same common page pool continues to be used.

---
 drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +0 -0
index 5e649705e35f..a51e204bd364 100644
@@ -749,7 +749,9 @@ static void mlx5e_rq_shampo_hd_info_free(struct mlx5e_rq *rq)
 
Quoted 14 lines

drivers/net/ethernet/mellanox/mlx5/core/en_main.c static void mlx5e_rq_shampo_hd_info_free(struct mlx5e_rq *rq) · line 754

Quoted 6 lines
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +3 -1
 static bool mlx5_rq_needs_separate_hd_pool(struct mlx5e_rq *rq)
 {
-       return false;
+       struct netdev_rx_queue *rxq = __netif_get_rx_queue(rq->netdev, rq->ix);
+
+       return !!rxq->mp_params.mp_ops;

This is kinda assuming that all future memory providers will return unreadable memory, which is not a restriction I have in mind... in theory there is nothing wrong with memory providers that feed readable pages. Technically the right thing to do here is to define a new helper page_pool_is_readable() and have the mp report to the pp if it's all readable or not.

Quoted 3 lines

The API is already there: page_pool_is_unreadable(). But it uses the same logic...

Ugh, I was evidently not paying attention when that was added. I guess everyone thinks memory provider == unreadable memory. I think it's more a coincidence that the first 2 memory providers give unreadable memory. Whatever I guess; it's good enough for now :D

Quoted 5 lines

However, having a pp level API is a bit limiting: as Cosmin pointed out, mlx5 can't use it because it needs to know in advance if this page_pool is for unreadable memory to correctly size the data page_pool (with or without headers).

Yeah, in that case mlx5 would do something like:

return !rxq->mp_params.mp_ops->is_readable();

If we decided that mp's could report if they're readable or not. For now I guess assuming all mps are unreadable is fine.

Signature · 2 lines
Thanks,
Mina
author maintainer
#17 lore raw

PATCH v4 9/11 net/mlx5e: Implement queue mgmt ops and single channel swap

In reply to Mark Bloch

From: Saeed Mahameed <[email protected]>

The bulk of the work is done in mlx5e_queue_mem_alloc, where we allocate and create the new channel resources, similar to mlx5e_safe_switch_params, but here we do it for a single channel using existing params, sort of a clone channel.
To swap the old channel with the new one, we deactivate and close the old channel then replace it with the new one, since the swap procedure doesn't fail in mlx5, we do it all in one place (mlx5e_queue_start).

---
 .../net/ethernet/mellanox/mlx5/core/en_main.c | 97 +++++++++++++++++++
 1 file changed, 97 insertions(+)
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +97 -0
index a51e204bd364..90687392545c 100644
@@ -5494,6 +5494,102 @@ static const struct netdev_stat_ops mlx5e_stat_ops = {
 	.get_base_stats      = mlx5e_get_base_stats,
 };
 
+struct mlx5_qmgmt_data {
+	struct mlx5e_channel *c;
+	struct mlx5e_channel_param cparam;
+};
+
+static int mlx5e_queue_mem_alloc(struct net_device *dev, void *newq,
+				 int queue_index)
+{
+	struct mlx5_qmgmt_data *new = (struct mlx5_qmgmt_data *)newq;
+	struct mlx5e_priv *priv = netdev_priv(dev);
+	struct mlx5e_channels *chs = &priv->channels;
+	struct mlx5e_params params = chs->params;
+	struct mlx5_core_dev *mdev;
+	int err;
+
+	mutex_lock(&priv->state_lock);
+	if (!test_bit(MLX5E_STATE_OPENED, &priv->state)) {
+		err = -ENODEV;
+		goto unlock;
+	}
+
+	if (queue_index >= chs->num) {
+		err = -ERANGE;
+		goto unlock;
+	}
+
+	if (MLX5E_GET_PFLAG(&chs->params, MLX5E_PFLAG_TX_PORT_TS) ||
+	    chs->params.ptp_rx   ||
+	    chs->params.xdp_prog ||
+	    priv->htb) {
+		netdev_err(priv->netdev,
+			   "Cloning channels with Port/rx PTP, XDP or HTB is not supported\n");
+		err = -EOPNOTSUPP;
+		goto unlock;
+	}
+
+	mdev = mlx5_sd_ch_ix_get_dev(priv->mdev, queue_index);
+	err = mlx5e_build_channel_param(mdev, &params, &new->cparam);
+	if (err) {
+		return err;
+		goto unlock;
+	}
+
+	err = mlx5e_open_channel(priv, queue_index, &params, NULL, &new->c);
+unlock:
+	mutex_unlock(&priv->state_lock);
+	return err;
+}
+
+static void mlx5e_queue_mem_free(struct net_device *dev, void *mem)
+{
+	struct mlx5_qmgmt_data *data = (struct mlx5_qmgmt_data *)mem;
+
+	/* not supposed to happen since mlx5e_queue_start never fails
+	 * but this is how this should be implemented just in case
+	 */
+	if (data->c)
+		mlx5e_close_channel(data->c);
+}
+
+static int mlx5e_queue_stop(struct net_device *dev, void *oldq, int queue_index)
+{
+	/* mlx5e_queue_start does not fail, we stop the old queue there */
+	return 0;
+}
+
+static int mlx5e_queue_start(struct net_device *dev, void *newq,
+			     int queue_index)
+{
+	struct mlx5_qmgmt_data *new = (struct mlx5_qmgmt_data *)newq;
+	struct mlx5e_priv *priv = netdev_priv(dev);
+	struct mlx5e_channel *old;
+
+	mutex_lock(&priv->state_lock);
+
+	/* stop and close the old */
+	old = priv->channels.c[queue_index];
+	mlx5e_deactivate_priv_channels(priv);
+	/* close old before activating new, to avoid napi conflict */
+	mlx5e_close_channel(old);
+
+	/* start the new */
+	priv->channels.c[queue_index] = new->c;
+	mlx5e_activate_priv_channels(priv);
+	mutex_unlock(&priv->state_lock);
+	return 0;
+}
+
+static const struct netdev_queue_mgmt_ops mlx5e_queue_mgmt_ops = {
+	.ndo_queue_mem_size	=	sizeof(struct mlx5_qmgmt_data),
+	.ndo_queue_mem_alloc	=	mlx5e_queue_mem_alloc,
+	.ndo_queue_mem_free	=	mlx5e_queue_mem_free,
+	.ndo_queue_start	=	mlx5e_queue_start,
+	.ndo_queue_stop		=	mlx5e_queue_stop,
+};
+
 static void mlx5e_build_nic_netdev(struct net_device *netdev)
 {
 	struct mlx5e_priv *priv = netdev_priv(netdev);
@@ -5504,6 +5600,7 @@ static void mlx5e_build_nic_netdev(struct net_device *netdev)
 	SET_NETDEV_DEV(netdev, mdev->device);
 
 	netdev->netdev_ops = &mlx5e_netdev_ops;
+	netdev->queue_mgmt_ops = &mlx5e_queue_mgmt_ops;
 	netdev->xdp_metadata_ops = &mlx5e_xdp_metadata_ops;
 	netdev->xsk_tx_metadata_ops = &mlx5e_xsk_tx_metadata_ops;
 	netdev->request_ops_lock = true;
Signature · 1 line
2.34.1
maintainer
#18 lore raw

Re: [PATCH net-next v4 09/11] net/mlx5e: Implement queue mgmt ops and single channel swap

drivers/net/ethernet/mellanox/mlx5/core/en_main.c static const struct netdev_stat_ops mlx5e_stat_ops = { · line 5537

Quoted 5 lines · On Tue, 10 Jun 2025 18:09:48 +0300 Mark Bloch wrote:
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +5 -0
+	mdev = mlx5_sd_ch_ix_get_dev(priv->mdev, queue_index);
+	err = mlx5e_build_channel_param(mdev, &params, &new->cparam);
+	if (err) {
+		return err;
+		goto unlock;

missed this nugget yesterday, sorry..

Quoted 1 line
(quoted diff) +1 -0
+	}
author maintainer
#19 lore raw

PATCH v4 10/11 net/mlx5e: Support ethtool tcp-data-split settings

In reply to Mark Bloch

From: Saeed Mahameed <[email protected]>

In mlx5, tcp header-data split requires HW GRO to be on.

Enabling it fails when HW GRO is off.
mlx5e_fix_features now keeps HW GRO on when tcp data split is enabled. Finally, when tcp data split is disabled, features are updated to maybe remove the forced HW GRO.

---
 .../ethernet/mellanox/mlx5/core/en_ethtool.c  | 33 ++++++++++++++++---
 .../net/ethernet/mellanox/mlx5/core/en_main.c |  8 +++++
 2 files changed, 36 insertions(+), 5 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c +28 -5
index ea078c9f5d15..21005cc4eb06 100644
@@ -32,6 +32,7 @@
 
 #include <linux/dim.h>
 #include <linux/ethtool_netlink.h>
+#include <net/netdev_queues.h>
 
 #include "en.h"
 #include "en/channels.h"
@@ -366,11 +367,6 @@ void mlx5e_ethtool_get_ringparam(struct mlx5e_priv *priv,
 	param->tx_max_pending = 1 << MLX5E_PARAMS_MAXIMUM_LOG_SQ_SIZE;
 	param->rx_pending     = 1 << priv->channels.params.log_rq_mtu_frames;
 	param->tx_pending     = 1 << priv->channels.params.log_sq_size;
-
-	kernel_param->tcp_data_split =
-		(priv->channels.params.packet_merge.type == MLX5E_PACKET_MERGE_SHAMPO) ?
-		ETHTOOL_TCP_DATA_SPLIT_ENABLED :
-		ETHTOOL_TCP_DATA_SPLIT_DISABLED;
 }
 
 static void mlx5e_get_ringparam(struct net_device *dev,
@@ -383,6 +379,27 @@ static void mlx5e_get_ringparam(struct net_device *dev,
 	mlx5e_ethtool_get_ringparam(priv, param, kernel_param);
 }
 
+static bool mlx5e_ethtool_set_tcp_data_split(struct mlx5e_priv *priv,
+					     u8 tcp_data_split,
+					     struct netlink_ext_ack *extack)
+{
+	struct net_device *dev = priv->netdev;
+
+	if (tcp_data_split == ETHTOOL_TCP_DATA_SPLIT_ENABLED &&
+	    !(dev->features & NETIF_F_GRO_HW)) {
+		NL_SET_ERR_MSG(extack,
+			       "TCP-data-split is not supported when GRO HW is disabled\n");
+		return false; /* GRO HW is not enabled */
+	}
+
+	/* Might need to disable HW-GRO if it was kept on due to hds. */
+	if (tcp_data_split == ETHTOOL_TCP_DATA_SPLIT_DISABLED &&
+	    dev->cfg->hds_config == ETHTOOL_TCP_DATA_SPLIT_ENABLED)
+		netdev_update_features(priv->netdev);
+
+	return true;
+}
+
 int mlx5e_ethtool_set_ringparam(struct mlx5e_priv *priv,
 				struct ethtool_ringparam *param,
 				struct netlink_ext_ack *extack)
@@ -441,6 +458,11 @@ static int mlx5e_set_ringparam(struct net_device *dev,
 {
 	struct mlx5e_priv *priv = netdev_priv(dev);
 
+	if (!mlx5e_ethtool_set_tcp_data_split(priv,
+					      kernel_param->tcp_data_split,
+					      extack))
+		return -EINVAL;
+
 	return mlx5e_ethtool_set_ringparam(priv, param, extack);
 }
 
@@ -2625,6 +2647,7 @@ const struct ethtool_ops mlx5e_ethtool_ops = {
 				     ETHTOOL_COALESCE_USE_ADAPTIVE |
 				     ETHTOOL_COALESCE_USE_CQE,
 	.supported_input_xfrm = RXH_XFRM_SYM_OR_XOR,
+	.supported_ring_params = ETHTOOL_RING_USE_TCP_DATA_SPLIT,
 	.get_drvinfo       = mlx5e_get_drvinfo,
 	.get_link          = ethtool_op_get_link,
 	.get_link_ext_state  = mlx5e_get_link_ext_state,
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +8 -0
index 90687392545c..026bd479c6dd 100644
@@ -4413,6 +4413,7 @@ static netdev_features_t mlx5e_fix_uplink_rep_features(struct net_device *netdev
 static netdev_features_t mlx5e_fix_features(struct net_device *netdev,
 					    netdev_features_t features)
 {
+	struct netdev_config *cfg = netdev->cfg_pending;
 	struct mlx5e_priv *priv = netdev_priv(netdev);
 	struct mlx5e_vlan_table *vlan;
 	struct mlx5e_params *params;
@@ -4479,6 +4480,13 @@ static netdev_features_t mlx5e_fix_features(struct net_device *netdev,
 		}
 	}
 
+	/* The header-data split ring param requires HW GRO to stay enabled. */
+	if (cfg && cfg->hds_config == ETHTOOL_TCP_DATA_SPLIT_ENABLED &&
+	    !(features & NETIF_F_GRO_HW)) {
+		netdev_warn(netdev, "Keeping HW-GRO enabled, TCP header-data split depends on it\n");
+		features |= NETIF_F_GRO_HW;
+	}
+
 	if (mlx5e_is_uplink_rep(priv)) {
 		features = mlx5e_fix_uplink_rep_features(netdev, features);
 		netdev->netns_immutable = true;
Signature · 1 line
2.34.1
maintainer
#20 lore raw

Re: [PATCH net-next v4 10/11] net/mlx5e: Support ethtool tcp-data-split settings

drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c static void mlx5e_get_ringparam(struct net_device *dev, · line 391

Quoted 2 lines · On Tue, 10 Jun 2025 18:09:49 +0300 Mark Bloch wrote:
drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c +2 -0
+		NL_SET_ERR_MSG(extack,
+			       "TCP-data-split is not supported when GRO HW is disabled\n");

unnecessary trailing new line, you probably want to use NL_SET_ERR_MSG_MOD(), too

Signature · 1 line
pw-bot: cr
author maintainer
#21 lore raw

PATCH v4 11/11 net/mlx5e: Add TX support for netmems

In reply to Mark Bloch

From: Dragos Tatulea <[email protected]>

Declare netmem TX support in netdev.

As required, use the netmem aware dma unmapping APIs for unmapping netmems in tx completion path.

---
 drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h | 3 ++-
 drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 2 ++
 2 files changed, 4 insertions(+), 1 deletion(-)
drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h +2 -1
index e837c21d3d21..6501252359b0 100644
@@ -362,7 +362,8 @@ mlx5e_tx_dma_unmap(struct device *pdev, struct mlx5e_sq_dma *dma)
 		dma_unmap_single(pdev, dma->addr, dma->size, DMA_TO_DEVICE);
 		break;
 	case MLX5E_DMA_MAP_PAGE:
-		dma_unmap_page(pdev, dma->addr, dma->size, DMA_TO_DEVICE);
+		netmem_dma_unmap_page_attrs(pdev, dma->addr, dma->size,
+					    DMA_TO_DEVICE, 0);
 		break;
 	default:
 		WARN_ONCE(true, "mlx5e_tx_dma_unmap unknown DMA type!\n");
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +2 -0
index 026bd479c6dd..1f2973e82733 100644
@@ -5740,6 +5740,8 @@ static void mlx5e_build_nic_netdev(struct net_device *netdev)
 
 	netdev->priv_flags       |= IFF_UNICAST_FLT;
 
+	netdev->netmem_tx = true;
+
 	netif_set_tso_max_size(netdev, GSO_MAX_SIZE);
 	mlx5e_set_xdp_feature(netdev);
 	mlx5e_set_netdev_dev_addr(netdev);
Signature · 1 line
2.34.1
Reviewed-by on v4 11/11
#22 lore raw

Re: [PATCH net-next v4 11/11] net/mlx5e: Add TX support for netmems

Quoted 11 lines · Mark Bloch

From: Dragos Tatulea <[email protected]>

Declare netmem TX support in netdev.

As required, use the netmem aware dma unmapping APIs for unmapping netmems in tx completion path.

Signature · 2 lines
Thanks,
Mina