Skip to contentSkip to conversation

net/mlx5: fix external buffer pool registration for Rx queue

PATCH In review Viacheslav Ovsiienko 3 participants 3 messages Feb 12, 2021 – Feb 21, 2021 Network drivers

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 net/mlx5: fix external buffer pool registration for Rx queue

On Rx queue creation the mlx5 PMD registers the data buffers of the specified pools for DMA operations. It scans the mem_list of the pools and creates the MRs (DMA related NIC objects) for the chunks found. If the pool is created with rte_pktmbuf_pool_create_extbuf() and refers to the external attached buffers (whose are in the area of application responsibility and it should explicitly register the data buffer memory for DMA with rte_dev_dma_map() call) the chunks contain the mbuf structures only, w/o any built-in data buffers. Hence, DMA with mlx5 NIC never happens to this area and there is no need to create MRs for these ones.

The extra not needed MRs were created for the pools with external buffers causing MR cache load and performance was slightly affected. The patch checks the mbuf pool type and skips MR creation for the pools with external buffers.

Fixes bdb8e5b1ea7b ("net/mlx5: allow allocated mbuf with external buffer")
---
 drivers/net/mlx5/mlx5_mr.c | 11 +++++++++++
 1 file changed, 11 insertions(+)
drivers/net/mlx5/mlx5_mr.c +11 -0
index 8b20ee3f83..da4e91fc24 100644
@@ -535,7 +535,18 @@ mlx5_mr_update_mp(struct rte_eth_dev *dev, struct mlx5_mr_ctrl *mr_ctrl,
 		.mr_ctrl = mr_ctrl,
 		.ret = 0,
 	};
+	uint32_t flags = rte_pktmbuf_priv_flags(mp);
 
+	if (flags & RTE_PKTMBUF_POOL_F_PINNED_EXT_BUF) {
+		/*
+		 * The pinned external buffer should be registered for DMA
+		 * operations by application. The mem_list of the pool contains
+		 * the list of chunks with mbuf structures w/o built-in data
+		 * buffers and DMA actually does not happen there, no need
+		 * to create MR for these chunks.
+		 */
+		return 0;
+	}
 	DRV_LOG(DEBUG, "Port %u Rx queue registering mp %s "
 		       "having %u chunks.", dev->data->port_id,
 		       mp->name, mp->nb_mem_chunks);
Signature · 1 line
2.18.1
Acked-by on v1 1/1
#1 lore raw

From: Viacheslav Ovsiienko

Quoted 16 lines

On Rx queue creation the mlx5 PMD registers the data buffers of the specified pools for DMA operations. It scans the mem_list of the pools and creates the MRs (DMA related NIC objects) for the chunks found.
If the pool is created with rte_pktmbuf_pool_create_extbuf() and refers to the external attached buffers (whose are in the area of application responsibility and it should explicitly register the data buffer memory for DMA with rte_dev_dma_map() call) the chunks contain the mbuf structures only, w/o any built-in data buffers.
Hence, DMA with mlx5 NIC never happens to this area and there is no need to create MRs for these ones.

The extra not needed MRs were created for the pools with external buffers causing MR cache load and performance was slightly affected.
The patch checks the mbuf pool type and skips MR creation for the pools with external buffers.

Quoted 4 lines
Fixes bdb8e5b1ea7b ("net/mlx5: allow allocated mbuf with external buffer")

Good catch!

Quoted 3 lines
---
 drivers/net/mlx5/mlx5_mr.c | 11 +++++++++++
 1 file changed, 11 insertions(+)
#2 lore raw
In reply to Viacheslav Ovsiienko

Hi,

Quoted 24 lines

-----Original Message-----
From: Viacheslav Ovsiienko <[email protected]>
Sent: Friday, February 12, 2021 1:07 PM
To: [email protected]
Cc: Raslan Darawsheh <[email protected]>; Matan Azrad <[email protected]>; [email protected]
Subject: [PATCH] net/mlx5: fix external buffer pool registration for Rx queue

On Rx queue creation the mlx5 PMD registers the data buffers of the specified pools for DMA operations. It scans the mem_list of the pools and creates the MRs (DMA related NIC objects) for the chunks found. If the pool is created with rte_pktmbuf_pool_create_extbuf() and refers to the external attached buffers (whose are in the area of application responsibility and it should explicitly register the data buffer memory for DMA with rte_dev_dma_map() call) the chunks contain the mbuf structures only, w/o any built-in data buffers. Hence, DMA with mlx5 NIC never happens to this area and there is no need to create MRs for these ones.

The extra not needed MRs were created for the pools with external buffers causing MR cache load and performance was slightly affected. The patch checks the mbuf pool type and skips MR creation for the pools with external buffers.

Quoted 5 lines
Fixes bdb8e5b1ea7b ("net/mlx5: allow allocated mbuf with external buffer")

---

Patch applied to next-net-mlx,

Kindest regards,
Raslan Darawsheh