net/mlx5: fix external buffer pool registration for Rx queue
PATCH 1/1 net/mlx5: fix external buffer pool registration for Rx queue
On Rx queue creation the mlx5 PMD registers the data buffers of the specified pools for DMA operations. It scans the mem_list of the pools and creates the MRs (DMA related NIC objects) for the chunks found. If the pool is created with rte_pktmbuf_pool_create_extbuf() and refers to the external attached buffers (whose are in the area of application responsibility and it should explicitly register the data buffer memory for DMA with rte_dev_dma_map() call) the chunks contain the mbuf structures only, w/o any built-in data buffers. Hence, DMA with mlx5 NIC never happens to this area and there is no need to create MRs for these ones.
The extra not needed MRs were created for the pools with external buffers causing MR cache load and performance was slightly affected. The patch checks the mbuf pool type and skips MR creation for the pools with external buffers.
--- drivers/net/mlx5/mlx5_mr.c | 11 +++++++++++ 1 file changed, 11 insertions(+)
drivers/net/mlx5/mlx5_mr.c +11 -0
@@ -535,7 +535,18 @@ mlx5_mr_update_mp(struct rte_eth_dev *dev, struct mlx5_mr_ctrl *mr_ctrl, .mr_ctrl = mr_ctrl, .ret = 0, }; +uint32_t flags =rte_pktmbuf_priv_flags (mp); +if (flags &RTE_PKTMBUF_POOL_F_PINNED_EXT_BUF ) { +/* +* The pinned external buffer should be registered for DMA +* operations by application. The mem_list of the pool contains +* the list of chunks with mbuf structures w/o built-in data +* buffers and DMA actually does not happen there, no need +* to create MR for these chunks. +*/ +return 0 ; + }DRV_LOG (DEBUG ,"Port %u Rx queue registering mp %s " "having %u chunks." , dev->data->port_id, mp->name, mp->nb_mem_chunks);
Signature · 1 line
2.18.1
From: Viacheslav Ovsiienko
Quoted 16 lines
On Rx queue creation the mlx5 PMD registers the data buffers of the specified pools for DMA operations. It scans the
mem_listof the pools and creates the MRs (DMA related NIC objects) for the chunks found.
If the pool is created withrte_pktmbuf_pool_create_extbuf()and refers to the external attached buffers (whose are in the area of application responsibility and it should explicitly register the data buffer memory for DMA withrte_dev_dma_map()call) the chunks contain the mbuf structures only, w/o any built-in data buffers.
Hence, DMA with mlx5 NIC never happens to this area and there is no need to create MRs for these ones.The extra not needed MRs were created for the pools with external buffers causing MR cache load and performance was slightly affected.
The patch checks the mbuf pool type and skips MR creation for the pools with external buffers.
Quoted 4 lines
Fixes bdb8e5b1ea7b ("net/mlx5: allow allocated mbuf with external buffer")Signed-off-by Viacheslav Ovsiienko <[email protected]>
Good catch!
Quoted 3 lines
--- drivers/net/mlx5/mlx5_mr.c | 11 +++++++++++ 1 file changed, 11 insertions(+)
Hi,
Quoted 24 lines
-----Original Message-----
From: Viacheslav Ovsiienko <[email protected]>
Sent: Friday, February 12, 2021 1:07 PM
To: [email protected]
Cc: Raslan Darawsheh <[email protected]>; Matan Azrad <[email protected]>; [email protected]
Subject: [PATCH] net/mlx5: fix external buffer pool registration for Rx queueOn Rx queue creation the mlx5 PMD registers the data buffers of the specified pools for DMA operations. It scans the
mem_listof the pools and creates the MRs (DMA related NIC objects) for the chunks found. If the pool is created withrte_pktmbuf_pool_create_extbuf()and refers to the external attached buffers (whose are in the area of application responsibility and it should explicitly register the data buffer memory for DMA withrte_dev_dma_map()call) the chunks contain the mbuf structures only, w/o any built-in data buffers. Hence, DMA with mlx5 NIC never happens to this area and there is no need to create MRs for these ones.The extra not needed MRs were created for the pools with external buffers causing MR cache load and performance was slightly affected. The patch checks the mbuf pool type and skips MR creation for the pools with external buffers.
Quoted 5 lines
Fixes bdb8e5b1ea7b ("net/mlx5: allow allocated mbuf with external buffer")Signed-off-by Viacheslav Ovsiienko <[email protected]>---
Patch applied to next-net-mlx,
Kindest regards,
Raslan Darawsheh