Skip to contentSkip to conversation

performance optimized for hns3 PMD

PATCH Superseded Min Hu (Connor) 3 participants 14 messages Nov 11, 2021 – Nov 16, 2021 Network drivers

lore
Share & tools
lore

Min Hu (Connor)

author
#0 lore raw

cover performance optimized for hns3 PMD

This patch set contains two ways to optimized Tx performance.

Chengwen Feng (2):
net/hns3: optimized Tx performance by mbuf fast free net/hns3: optimized Tx performance

 drivers/net/hns3/hns3_rxtx.c     | 129 ++++++++++++++++---------------
 drivers/net/hns3/hns3_rxtx.h     |   2 +
 drivers/net/hns3/hns3_rxtx_vec.h |   9 +++
 3 files changed, 76 insertions(+), 64 deletions(-)
Signature · 1 line
2.33.0
author
#1 lore raw

PATCH 1/2 net/hns3: optimized Tx performance by mbuf fast free

From: Chengwen Feng <[email protected]>

Currently the vector and simple xmit algorithm don't support multi_segs, so if Tx offload support MBUF_FAST_FREE, driver could invoke rte_mempool_put_bulk() to free Tx mbufs in this situation.

In the testpmd single core MAC forwarding scenario, the performance is improved by 8% at 64B on Kunpeng920 platform.

Cc [email protected]
Signed-off-by Chengwen Feng <[email protected]>
Signed-off-by Min Hu (Connor) <[email protected]>
---
 drivers/net/hns3/hns3_rxtx.c     | 11 +++++++++++
 drivers/net/hns3/hns3_rxtx.h     |  2 ++
 drivers/net/hns3/hns3_rxtx_vec.h |  9 +++++++++
 3 files changed, 22 insertions(+)
drivers/net/hns3/hns3_rxtx.c +11 -0
index d26e262335..78227a139f 100644
@@ -3059,6 +3059,8 @@ hns3_tx_queue_setup(struct rte_eth_dev *dev, uint16_t idx, uint16_t nb_desc,
 	txq->min_tx_pkt_len = hw->min_tx_pkt_len;
 	txq->tso_mode = hw->tso_mode;
 	txq->udp_cksum_mode = hw->udp_cksum_mode;
+	txq->mbuf_fast_free_en = !!(dev->data->dev_conf.txmode.offloads &
+				    DEV_TX_OFFLOAD_MBUF_FAST_FREE);
 	memset(&txq->basic_stats, 0, sizeof(struct hns3_tx_basic_stats));
 	memset(&txq->dfx_stats, 0, sizeof(struct hns3_tx_dfx_stats));
 

@@ -3991,6 +3993,14 @@ hns3_tx_free_buffer_simple(struct hns3_tx_queue *txq)
 
 		tx_entry = &txq->sw_ring[txq->next_to_clean];
 
+		if (txq->mbuf_fast_free_en) {
+			rte_mempool_put_bulk(tx_entry->mbuf->pool,
+					(void **)tx_entry, txq->tx_rs_thresh);
+			for (i = 0; i < txq->tx_rs_thresh; i++)
+				tx_entry[i].mbuf = NULL;
+			goto update_field;
+		}
+
 		for (i = 0; i < txq->tx_rs_thresh; i++)
 			rte_prefetch0((tx_entry + i)->mbuf);
 		for (i = 0; i < txq->tx_rs_thresh; i++, tx_entry++) {

@@ -3998,6 +4008,7 @@ hns3_tx_free_buffer_simple(struct hns3_tx_queue *txq)
 			tx_entry->mbuf = NULL;
 		}
 
+update_field:
 		txq->next_to_clean = (tx_next_clean + 1) % txq->nb_tx_desc;
 		txq->tx_bd_ready += txq->tx_rs_thresh;
 	}
drivers/net/hns3/hns3_rxtx.h +2 -0
index 63bafc68b6..df731856ef 100644
@@ -495,6 +495,8 @@ struct hns3_tx_queue {
 	 * this point.
 	 */
 	uint16_t pvid_sw_shift_en:1;
+	/* check whether the mbuf fast free offload is enabled */
+	uint16_t mbuf_fast_free_en:1;
 
 	/*
 	 * For better performance in tx datapath, releasing mbuf in batches is
drivers/net/hns3/hns3_rxtx_vec.h +9 -0
index 67c75e44ef..4985a7cae8 100644
@@ -18,6 +18,14 @@ hns3_tx_bulk_free_buffers(struct hns3_tx_queue *txq)
 	int i;
 
 	tx_entry = &txq->sw_ring[txq->next_to_clean];
+	if (txq->mbuf_fast_free_en) {
+		rte_mempool_put_bulk(tx_entry->mbuf->pool, (void **)tx_entry,
+				     txq->tx_rs_thresh);
+		for (i = 0; i < txq->tx_rs_thresh; i++)
+			tx_entry[i].mbuf = NULL;
+		goto update_field;
+	}
+
 	for (i = 0; i < txq->tx_rs_thresh; i++, tx_entry++) {
 		m = rte_pktmbuf_prefree_seg(tx_entry->mbuf);
 		tx_entry->mbuf = NULL;

@@ -36,6 +44,7 @@ hns3_tx_bulk_free_buffers(struct hns3_tx_queue *txq)
 	if (nb_free)
 		rte_mempool_put_bulk(free[0]->pool, (void **)free, nb_free);
 
+update_field:
 	/* Update numbers of available descriptor due to buffer freed */
 	txq->tx_bd_ready += txq->tx_rs_thresh;
 	txq->next_to_clean += txq->tx_rs_thresh;
Signature · 1 line
2.33.0
#2 lore raw

Re: [PATCH 1/2] net/hns3: optimized Tx performance by mbuf fast free

Quoted 9 lines · Min Hu (Connor)

From: Chengwen Feng <[email protected]>

Currently the vector and simple xmit algorithm don't support multi_segs, so if Tx offload support MBUF_FAST_FREE, driver could invoke rte_mempool_put_bulk() to free Tx mbufs in this situation.

In the testpmd single core MAC forwarding scenario, the performance is improved by 8% at 64B on Kunpeng920 platform.

'RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE' seems already announced in 'tx_offload_capa',
was it wrong?

Quoted 10 lines
Cc [email protected]
Signed-off-by Chengwen Feng <[email protected]>
Signed-off-by Min Hu (Connor) <[email protected]>
---
  drivers/net/hns3/hns3_rxtx.c     | 11 +++++++++++
  drivers/net/hns3/hns3_rxtx.h     |  2 ++
  drivers/net/hns3/hns3_rxtx_vec.h |  9 +++++++++
  3 files changed, 22 insertions(+)

Can you please update 'doc/guides/nics/features/hns3.ini' to announce "Free Tx mbuf on demand" feature.

drivers/net/hns3/hns3_rxtx.c hns3_tx_queue_setup(struct rte_eth_dev *dev, uint16_t idx, uint16_t nb_desc, · line 3063

Quoted 10 lines
drivers/net/hns3/hns3_rxtx.c +2 -0
index d26e262335..78227a139f 100644
@@ -3059,6 +3059,8 @@ hns3_tx_queue_setup(struct rte_eth_dev *dev, uint16_t idx, uint16_t nb_desc,
  	txq->min_tx_pkt_len = hw->min_tx_pkt_len;
  	txq->tso_mode = hw->tso_mode;
  	txq->udp_cksum_mode = hw->udp_cksum_mode;
+	txq->mbuf_fast_free_en = !!(dev->data->dev_conf.txmode.offloads &
+				    DEV_TX_OFFLOAD_MBUF_FAST_FREE);

Can you please use updated macro name, 'RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE'?

author
#3 lore raw

Re: [PATCH 1/2] net/hns3: optimized Tx performance by mbuf fast free

Hi, Ferruh,
fixed in v2, thanks.

Quoted 42 lines · Ferruh Yigit
Quoted 9 lines · Min Hu (Connor)

From: Chengwen Feng <[email protected]>

Currently the vector and simple xmit algorithm don't support multi_segs, so if Tx offload support MBUF_FAST_FREE, driver could invoke rte_mempool_put_bulk() to free Tx mbufs in this situation.

In the testpmd single core MAC forwarding scenario, the performance is improved by 8% at 64B on Kunpeng920 platform.

'RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE' seems already announced in 'tx_offload_capa',
 was it wrong?

Quoted 10 lines
Cc [email protected]
Signed-off-by Chengwen Feng <[email protected]>
Signed-off-by Min Hu (Connor) <[email protected]>

---
  drivers/net/hns3/hns3_rxtx.c     | 11 +++++++++++ drivers/net/hns3/hns3_rxtx.h     |  2 ++ drivers/net/hns3/hns3_rxtx_vec.h |  9 +++++++++

  3 files changed, 22 insertions(+)

Can you please update 'doc/guides/nics/features/hns3.ini' to announce "Free Tx mbuf on demand" feature.

drivers/net/hns3/hns3_rxtx.c hns3_tx_queue_setup(struct rte_eth_dev *dev, · line 3064

Quoted 11 lines
drivers/net/hns3/hns3_rxtx.c +2 -0
index d26e262335..78227a139f 100644
@@ -3059,6 +3059,8 @@ hns3_tx_queue_setup(struct rte_eth_dev *dev, 
 uint16_t idx, uint16_t nb_desc,
       txq->min_tx_pkt_len = hw->min_tx_pkt_len;
       txq->tso_mode = hw->tso_mode;
       txq->udp_cksum_mode = hw->udp_cksum_mode;
+    txq->mbuf_fast_free_en = !!(dev->data->dev_conf.txmode.offloads &
+                    DEV_TX_OFFLOAD_MBUF_FAST_FREE);

Can you please use updated macro name, 'RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE'? .

author
#4 lore raw

PATCH 2/2 net/hns3: optimized Tx performance

In reply to Min Hu (Connor)

From: Chengwen Feng <[email protected]>

The PMD should check whether the descriptor is done by hardware before free the corresponding mbuf. Currently the common xmit algorithm will free mbuf every time when it's invoked. Because hardware may not have finished sending, this may lead to many invalid queries which are whether the descriptors are done.

This patch uses tx_free_thresh to control whether invoke free mbuf, and free tx_rs_thresh mbufs each time.

This patch also modifies the implementation of PMD's tx_done_cleanup because the mbuf free algorithm changed.

In the testpmd single core MAC forwarding scenario, the performance is improved by 10% at 64B on Kunpeng920 platform.

Cc [email protected]
Signed-off-by Chengwen Feng <[email protected]>
Signed-off-by Min Hu (Connor) <[email protected]>
---
 drivers/net/hns3/hns3_rxtx.c | 118 ++++++++++++++++-------------------
 1 file changed, 54 insertions(+), 64 deletions(-)
drivers/net/hns3/hns3_rxtx.c +54 -64
index 78227a139f..114b2f397f 100644
@@ -3077,40 +3077,51 @@ hns3_tx_queue_setup(struct rte_eth_dev *dev, uint16_t idx, uint16_t nb_desc,
 	return 0;
 }
 
-static void
+static int
 hns3_tx_free_useless_buffer(struct hns3_tx_queue *txq)
 {
 	uint16_t tx_next_clean = txq->next_to_clean;
-	uint16_t tx_next_use   = txq->next_to_use;
-	uint16_t tx_bd_ready   = txq->tx_bd_ready;
-	uint16_t tx_bd_max     = txq->nb_tx_desc;
-	struct hns3_entry *tx_bak_pkt = &txq->sw_ring[tx_next_clean];
+	uint16_t tx_next_use = txq->next_to_use;
+	struct hns3_entry *tx_entry = &txq->sw_ring[tx_next_clean];
 	struct hns3_desc *desc = &txq->tx_ring[tx_next_clean];
-	struct rte_mbuf *mbuf;
+	int i;
 
-	while ((!(desc->tx.tp_fe_sc_vld_ra_ri &
-		rte_cpu_to_le_16(BIT(HNS3_TXD_VLD_B)))) &&
-		tx_next_use != tx_next_clean) {
-		mbuf = tx_bak_pkt->mbuf;
-		if (mbuf) {
-			rte_pktmbuf_free_seg(mbuf);
-			tx_bak_pkt->mbuf = NULL;
-		}
+	if (tx_next_use >= tx_next_clean &&
+	    tx_next_use < tx_next_clean + txq->tx_rs_thresh)
+		return -1;
 
-		desc++;
-		tx_bak_pkt++;
-		tx_next_clean++;
-		tx_bd_ready++;
-
-		if (tx_next_clean >= tx_bd_max) {
-			tx_next_clean = 0;
-			desc = txq->tx_ring;
-			tx_bak_pkt = txq->sw_ring;
-		}
+	/*
+	 * All mbufs can be released only when the VLD bits of all
+	 * descriptors in a batch are cleared.
+	 */
+	for (i = 0; i < txq->tx_rs_thresh; i++) {
+		if (desc[i].tx.tp_fe_sc_vld_ra_ri &
+			rte_le_to_cpu_16(BIT(HNS3_TXD_VLD_B)))
+			return -1;
 	}
 
-	txq->next_to_clean = tx_next_clean;
-	txq->tx_bd_ready   = tx_bd_ready;
+	for (i = 0; i < txq->tx_rs_thresh; i++) {
+		rte_pktmbuf_free_seg(tx_entry[i].mbuf);
+		tx_entry[i].mbuf = NULL;
+	}
+
+	/* Update numbers of available descriptor due to buffer freed */
+	txq->tx_bd_ready += txq->tx_rs_thresh;
+	txq->next_to_clean += txq->tx_rs_thresh;
+	if (txq->next_to_clean >= txq->nb_tx_desc)
+		txq->next_to_clean = 0;
+
+	return 0;
+}
+
+static inline int
+hns3_tx_free_required_buffer(struct hns3_tx_queue *txq, uint16_t required_bds)
+{
+	while (required_bds > txq->tx_bd_ready) {
+		if (hns3_tx_free_useless_buffer(txq) != 0)
+			return -1;
+	}
+	return 0;
 }
 
 int

@@ -4147,8 +4158,8 @@ hns3_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 	uint16_t nb_tx;
 	uint16_t i;
 
-	/* free useless buffer */
-	hns3_tx_free_useless_buffer(txq);
+	if (txq->tx_bd_ready < txq->tx_free_thresh)
+		(void)hns3_tx_free_useless_buffer(txq);
 
 	tx_next_use   = txq->next_to_use;
 	tx_bd_max     = txq->nb_tx_desc;

@@ -4163,11 +4174,14 @@ hns3_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 		nb_buf = tx_pkt->nb_segs;
 
 		if (nb_buf > txq->tx_bd_ready) {
-			txq->dfx_stats.queue_full_cnt++;
-			if (nb_tx == 0)
-				return 0;
-
-			goto end_of_tx;
+			/* Try to release the required MBUF, but avoid releasing
+			 * all MBUFs, otherwise, the MBUFs will be released for
+			 * a long time and may cause jitter.
+			 */
+			if (hns3_tx_free_required_buffer(txq, nb_buf) != 0) {
+				txq->dfx_stats.queue_full_cnt++;
+				goto end_of_tx;
+			}
 		}
 
 		/*

@@ -4577,46 +4591,22 @@ hns3_dev_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id)
 static int
 hns3_tx_done_cleanup_full(struct hns3_tx_queue *txq, uint32_t free_cnt)
 {
-	uint16_t next_to_clean = txq->next_to_clean;
-	uint16_t next_to_use   = txq->next_to_use;
-	uint16_t tx_bd_ready   = txq->tx_bd_ready;
-	struct hns3_entry *tx_pkt = &txq->sw_ring[next_to_clean];
-	struct hns3_desc *desc = &txq->tx_ring[next_to_clean];
+	uint16_t round_free_cnt;
 	uint32_t idx;
 
 	if (free_cnt == 0 || free_cnt > txq->nb_tx_desc)
 		free_cnt = txq->nb_tx_desc;
 
-	for (idx = 0; idx < free_cnt; idx++) {
-		if (next_to_clean == next_to_use)
-			break;
+	if (txq->tx_rs_thresh == 0)
+		return 0;
 
-		if (desc->tx.tp_fe_sc_vld_ra_ri &
-		    rte_cpu_to_le_16(BIT(HNS3_TXD_VLD_B)))
+	round_free_cnt = roundup(free_cnt, txq->tx_rs_thresh);
+	for (idx = 0; idx < round_free_cnt; idx += txq->tx_rs_thresh) {
+		if (hns3_tx_free_useless_buffer(txq) != 0)
 			break;
-
-		if (tx_pkt->mbuf != NULL) {
-			rte_pktmbuf_free_seg(tx_pkt->mbuf);
-			tx_pkt->mbuf = NULL;
-		}
-
-		next_to_clean++;
-		tx_bd_ready++;
-		tx_pkt++;
-		desc++;
-		if (next_to_clean == txq->nb_tx_desc) {
-			tx_pkt = txq->sw_ring;
-			desc = txq->tx_ring;
-			next_to_clean = 0;
-		}
-	}
-
-	if (idx > 0) {
-		txq->next_to_clean = next_to_clean;
-		txq->tx_bd_ready = tx_bd_ready;
 	}
 
-	return (int)idx;
+	return RTE_MIN(idx, free_cnt);
 }
 
 int
Signature · 1 line
2.33.0
#5 lore raw

Re: [PATCH 2/2] net/hns3: optimized Tx performance

Quoted 8 lines · Min Hu (Connor)

From: Chengwen Feng<[email protected]>

The PMD should check whether the descriptor is done by hardware before free the corresponding mbuf. Currently the common xmit algorithm will free mbuf every time when it's invoked. Because hardware may not have finished sending, this may lead to many invalid queries which are whether the descriptors are done.

Hi Connor, Chengwen,

Since there will be a new version, can you please reword above paragraph?

Quoted 13 lines

This patch uses tx_free_thresh to control whether invoke free mbuf, and free tx_rs_thresh mbufs each time.

This patch also modifies the implementation of PMD's tx_done_cleanup because the mbuf free algorithm changed.

In the testpmd single core MAC forwarding scenario, the performance is improved by 10% at 64B on Kunpeng920 platform.

Cc [email protected]
Signed-off-by Chengwen Feng<[email protected]>
Signed-off-by Min Hu (Connor)<[email protected]>
author
#6 lore raw

v2 cover performance optimized for hns3 PMD

In reply to Min Hu (Connor)

This patch set contains two ways to optimized Tx performance.

Chengwen Feng (2):
net/hns3: optimized Tx performance by mbuf fast free net/hns3: optimized Tx performance

 doc/guides/nics/features/hns3.ini |   1 +
 drivers/net/hns3/hns3_rxtx.c      | 129 +++++++++++++++---------------
 drivers/net/hns3/hns3_rxtx.h      |   2 +
 drivers/net/hns3/hns3_rxtx_vec.h  |   9 +++
 4 files changed, 77 insertions(+), 64 deletions(-)

---
v2:
* document hns3.ini and fix 'TE_ETH_TX_OFFLOAD_MBUF_FAST_FREE'

Signature · 1 line
2.33.0
author
#7 lore raw

PATCH v2 1/2 net/hns3: optimized Tx performance by mbuf fast free

From: Chengwen Feng <[email protected]>

Currently the vector and simple xmit algorithm don't support multi_segs, so if Tx offload support MBUF_FAST_FREE, driver could invoke rte_mempool_put_bulk() to free Tx mbufs in this situation.

In the testpmd single core MAC forwarding scenario, the performance is improved by 8% at 64B on Kunpeng920 platform.

Cc [email protected]
Signed-off-by Chengwen Feng <[email protected]>
---
 doc/guides/nics/features/hns3.ini |  1 +
 drivers/net/hns3/hns3_rxtx.c      | 11 +++++++++++
 drivers/net/hns3/hns3_rxtx.h      |  2 ++
 drivers/net/hns3/hns3_rxtx_vec.h  |  9 +++++++++
 4 files changed, 23 insertions(+)
doc/guides/nics/features/hns3.ini +1 -0
index c3464c8396..405b94f05c 100644
@@ -12,6 +12,7 @@ Queue start/stop     = Y
 Runtime Rx queue setup = Y
 Runtime Tx queue setup = Y
 Burst mode info      = Y
+Fast mbuf free       = Y
 Free Tx mbuf on demand = Y
 MTU update           = Y
 Scattered Rx         = Y
drivers/net/hns3/hns3_rxtx.c +11 -0
index d26e262335..f0a57611ec 100644
@@ -3059,6 +3059,8 @@ hns3_tx_queue_setup(struct rte_eth_dev *dev, uint16_t idx, uint16_t nb_desc,
 	txq->min_tx_pkt_len = hw->min_tx_pkt_len;
 	txq->tso_mode = hw->tso_mode;
 	txq->udp_cksum_mode = hw->udp_cksum_mode;
+	txq->mbuf_fast_free_en = !!(dev->data->dev_conf.txmode.offloads &
+				    RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE);
 	memset(&txq->basic_stats, 0, sizeof(struct hns3_tx_basic_stats));
 	memset(&txq->dfx_stats, 0, sizeof(struct hns3_tx_dfx_stats));
 

@@ -3991,6 +3993,14 @@ hns3_tx_free_buffer_simple(struct hns3_tx_queue *txq)
 
 		tx_entry = &txq->sw_ring[txq->next_to_clean];
 
+		if (txq->mbuf_fast_free_en) {
+			rte_mempool_put_bulk(tx_entry->mbuf->pool,
+					(void **)tx_entry, txq->tx_rs_thresh);
+			for (i = 0; i < txq->tx_rs_thresh; i++)
+				tx_entry[i].mbuf = NULL;
+			goto update_field;
+		}
+
 		for (i = 0; i < txq->tx_rs_thresh; i++)
 			rte_prefetch0((tx_entry + i)->mbuf);
 		for (i = 0; i < txq->tx_rs_thresh; i++, tx_entry++) {

@@ -3998,6 +4008,7 @@ hns3_tx_free_buffer_simple(struct hns3_tx_queue *txq)
 			tx_entry->mbuf = NULL;
 		}
 
+update_field:
 		txq->next_to_clean = (tx_next_clean + 1) % txq->nb_tx_desc;
 		txq->tx_bd_ready += txq->tx_rs_thresh;
 	}
drivers/net/hns3/hns3_rxtx.h +2 -0
index 63bafc68b6..df731856ef 100644
@@ -495,6 +495,8 @@ struct hns3_tx_queue {
 	 * this point.
 	 */
 	uint16_t pvid_sw_shift_en:1;
+	/* check whether the mbuf fast free offload is enabled */
+	uint16_t mbuf_fast_free_en:1;
 
 	/*
 	 * For better performance in tx datapath, releasing mbuf in batches is
drivers/net/hns3/hns3_rxtx_vec.h +9 -0
index 67c75e44ef..4985a7cae8 100644
@@ -18,6 +18,14 @@ hns3_tx_bulk_free_buffers(struct hns3_tx_queue *txq)
 	int i;
 
 	tx_entry = &txq->sw_ring[txq->next_to_clean];
+	if (txq->mbuf_fast_free_en) {
+		rte_mempool_put_bulk(tx_entry->mbuf->pool, (void **)tx_entry,
+				     txq->tx_rs_thresh);
+		for (i = 0; i < txq->tx_rs_thresh; i++)
+			tx_entry[i].mbuf = NULL;
+		goto update_field;
+	}
+
 	for (i = 0; i < txq->tx_rs_thresh; i++, tx_entry++) {
 		m = rte_pktmbuf_prefree_seg(tx_entry->mbuf);
 		tx_entry->mbuf = NULL;

@@ -36,6 +44,7 @@ hns3_tx_bulk_free_buffers(struct hns3_tx_queue *txq)
 	if (nb_free)
 		rte_mempool_put_bulk(free[0]->pool, (void **)free, nb_free);
 
+update_field:
 	/* Update numbers of available descriptor due to buffer freed */
 	txq->tx_bd_ready += txq->tx_rs_thresh;
 	txq->next_to_clean += txq->tx_rs_thresh;
Signature · 1 line
2.33.0
author
#8 lore raw

PATCH v2 2/2 net/hns3: optimized Tx performance

In reply to Min Hu (Connor)

From: Chengwen Feng <[email protected]>

This patch uses tx_free_thresh to control mbufs free when the common xmit algorithm is used.

This patch also modifies the implementation of PMD's tx_done_cleanup because the mbuf free algorithm changed.

In the testpmd single core MAC forwarding scenario, the performance is improved by 10% at 64B on Kunpeng920 platform.

Cc [email protected]
Signed-off-by Chengwen Feng <[email protected]>
---
 drivers/net/hns3/hns3_rxtx.c | 118 ++++++++++++++++-------------------
 1 file changed, 54 insertions(+), 64 deletions(-)
drivers/net/hns3/hns3_rxtx.c +54 -64
index f0a57611ec..40cc4e9c1a 100644
@@ -3077,40 +3077,51 @@ hns3_tx_queue_setup(struct rte_eth_dev *dev, uint16_t idx, uint16_t nb_desc,
 	return 0;
 }
 
-static void
+static int
 hns3_tx_free_useless_buffer(struct hns3_tx_queue *txq)
 {
 	uint16_t tx_next_clean = txq->next_to_clean;
-	uint16_t tx_next_use   = txq->next_to_use;
-	uint16_t tx_bd_ready   = txq->tx_bd_ready;
-	uint16_t tx_bd_max     = txq->nb_tx_desc;
-	struct hns3_entry *tx_bak_pkt = &txq->sw_ring[tx_next_clean];
+	uint16_t tx_next_use = txq->next_to_use;
+	struct hns3_entry *tx_entry = &txq->sw_ring[tx_next_clean];
 	struct hns3_desc *desc = &txq->tx_ring[tx_next_clean];
-	struct rte_mbuf *mbuf;
+	int i;
 
-	while ((!(desc->tx.tp_fe_sc_vld_ra_ri &
-		rte_cpu_to_le_16(BIT(HNS3_TXD_VLD_B)))) &&
-		tx_next_use != tx_next_clean) {
-		mbuf = tx_bak_pkt->mbuf;
-		if (mbuf) {
-			rte_pktmbuf_free_seg(mbuf);
-			tx_bak_pkt->mbuf = NULL;
-		}
+	if (tx_next_use >= tx_next_clean &&
+	    tx_next_use < tx_next_clean + txq->tx_rs_thresh)
+		return -1;
 
-		desc++;
-		tx_bak_pkt++;
-		tx_next_clean++;
-		tx_bd_ready++;
-
-		if (tx_next_clean >= tx_bd_max) {
-			tx_next_clean = 0;
-			desc = txq->tx_ring;
-			tx_bak_pkt = txq->sw_ring;
-		}
+	/*
+	 * All mbufs can be released only when the VLD bits of all
+	 * descriptors in a batch are cleared.
+	 */
+	for (i = 0; i < txq->tx_rs_thresh; i++) {
+		if (desc[i].tx.tp_fe_sc_vld_ra_ri &
+			rte_le_to_cpu_16(BIT(HNS3_TXD_VLD_B)))
+			return -1;
 	}
 
-	txq->next_to_clean = tx_next_clean;
-	txq->tx_bd_ready   = tx_bd_ready;
+	for (i = 0; i < txq->tx_rs_thresh; i++) {
+		rte_pktmbuf_free_seg(tx_entry[i].mbuf);
+		tx_entry[i].mbuf = NULL;
+	}
+
+	/* Update numbers of available descriptor due to buffer freed */
+	txq->tx_bd_ready += txq->tx_rs_thresh;
+	txq->next_to_clean += txq->tx_rs_thresh;
+	if (txq->next_to_clean >= txq->nb_tx_desc)
+		txq->next_to_clean = 0;
+
+	return 0;
+}
+
+static inline int
+hns3_tx_free_required_buffer(struct hns3_tx_queue *txq, uint16_t required_bds)
+{
+	while (required_bds > txq->tx_bd_ready) {
+		if (hns3_tx_free_useless_buffer(txq) != 0)
+			return -1;
+	}
+	return 0;
 }
 
 int

@@ -4147,8 +4158,8 @@ hns3_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 	uint16_t nb_tx;
 	uint16_t i;
 
-	/* free useless buffer */
-	hns3_tx_free_useless_buffer(txq);
+	if (txq->tx_bd_ready < txq->tx_free_thresh)
+		(void)hns3_tx_free_useless_buffer(txq);
 
 	tx_next_use   = txq->next_to_use;
 	tx_bd_max     = txq->nb_tx_desc;

@@ -4163,11 +4174,14 @@ hns3_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 		nb_buf = tx_pkt->nb_segs;
 
 		if (nb_buf > txq->tx_bd_ready) {
-			txq->dfx_stats.queue_full_cnt++;
-			if (nb_tx == 0)
-				return 0;
-
-			goto end_of_tx;
+			/* Try to release the required MBUF, but avoid releasing
+			 * all MBUFs, otherwise, the MBUFs will be released for
+			 * a long time and may cause jitter.
+			 */
+			if (hns3_tx_free_required_buffer(txq, nb_buf) != 0) {
+				txq->dfx_stats.queue_full_cnt++;
+				goto end_of_tx;
+			}
 		}
 
 		/*

@@ -4577,46 +4591,22 @@ hns3_dev_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id)
 static int
 hns3_tx_done_cleanup_full(struct hns3_tx_queue *txq, uint32_t free_cnt)
 {
-	uint16_t next_to_clean = txq->next_to_clean;
-	uint16_t next_to_use   = txq->next_to_use;
-	uint16_t tx_bd_ready   = txq->tx_bd_ready;
-	struct hns3_entry *tx_pkt = &txq->sw_ring[next_to_clean];
-	struct hns3_desc *desc = &txq->tx_ring[next_to_clean];
+	uint16_t round_free_cnt;
 	uint32_t idx;
 
 	if (free_cnt == 0 || free_cnt > txq->nb_tx_desc)
 		free_cnt = txq->nb_tx_desc;
 
-	for (idx = 0; idx < free_cnt; idx++) {
-		if (next_to_clean == next_to_use)
-			break;
+	if (txq->tx_rs_thresh == 0)
+		return 0;
 
-		if (desc->tx.tp_fe_sc_vld_ra_ri &
-		    rte_cpu_to_le_16(BIT(HNS3_TXD_VLD_B)))
+	round_free_cnt = roundup(free_cnt, txq->tx_rs_thresh);
+	for (idx = 0; idx < round_free_cnt; idx += txq->tx_rs_thresh) {
+		if (hns3_tx_free_useless_buffer(txq) != 0)
 			break;
-
-		if (tx_pkt->mbuf != NULL) {
-			rte_pktmbuf_free_seg(tx_pkt->mbuf);
-			tx_pkt->mbuf = NULL;
-		}
-
-		next_to_clean++;
-		tx_bd_ready++;
-		tx_pkt++;
-		desc++;
-		if (next_to_clean == txq->nb_tx_desc) {
-			tx_pkt = txq->sw_ring;
-			desc = txq->tx_ring;
-			next_to_clean = 0;
-		}
-	}
-
-	if (idx > 0) {
-		txq->next_to_clean = next_to_clean;
-		txq->tx_bd_ready = tx_bd_ready;
 	}
 
-	return (int)idx;
+	return RTE_MIN(idx, free_cnt);
 }
 
 int
Signature · 1 line
2.33.0
#9 lore raw
In reply to Min Hu (Connor)
Quoted 6 lines · Min Hu (Connor)

This patch set contains two ways to optimized Tx performance.

Chengwen Feng (2):
net/hns3: optimized Tx performance by mbuf fast free net/hns3: optimized Tx performance

Quoted 9 lines
  doc/guides/nics/features/hns3.ini |   1 +
  drivers/net/hns3/hns3_rxtx.c      | 129 +++++++++++++++---------------
  drivers/net/hns3/hns3_rxtx.h      |   2 +
  drivers/net/hns3/hns3_rxtx_vec.h  |   9 +++
  4 files changed, 77 insertions(+), 64 deletions(-)

---
v2:
* document hns3.ini and fix 'TE_ETH_TX_OFFLOAD_MBUF_FAST_FREE'

Is Connor's sign off in the v1 dropped intentionally, or forgotten?

#10 lore raw

May be forgotten, I send v2 to him this morning, and maybe forgot adding the sign off. But the patch set is OK because we both check them.

发件人：Ferruh Yigit <[email protected]> 收件人：humin (Q) <[email protected]>;dev <[email protected]> 抄 送：thomas <[email protected]>
时 间：2021-11-16 22:37:23
主 题：Re: [PATCH v2 0/2] performance optimized for hns3 PMD

Quoted 8 lines · Min Hu (Connor)

This patch set contains two ways to optimized Tx performance.

Chengwen Feng (2):
net/hns3: optimized Tx performance by mbuf fast free net/hns3: optimized Tx performance

  doc/guides/nics/features/hns3.ini |   1 +
  drivers/net/hns3/hns3_rxtx.c      | 129 +++++++++++++++

________________________________

Quoted 7 lines
  drivers/net/hns3/hns3_rxtx.h      |   2 +
  drivers/net/hns3/hns3_rxtx_vec.h |   9 +++
  4 files changed, 77 insertions(+), 64 deletions(-)

---
v2:
* document hns3.ini and fix 'TE_ETH_TX_OFFLOAD_MBUF_FAST_FREE'

Is Connor's sign off in the v1 dropped intentionally, or forgotten?

author
#11 lore raw

Hi, ferruh,
I forgot it，please added it for me when you merged them.thank you.

________________________________

胡敏 Hu Min
Mobile: +86-13528728164<tel:+86-13528728164>
Email: [email protected]<mailto:[email protected]>
发件人：Fengchengwen <[email protected]>
收件人：Ferruh Yigit <[email protected]>;humin (Q) <[email protected]>;dev <[email protected]> 抄 送：thomas <[email protected]>
时 间：2021-11-16 23:04:08
主 题：RE: [PATCH v2 0/2] performance optimized for hns3 PMD

May be forgotten, I send v2 to him this morning, and maybe forgot adding the sign off. But the patch set is OK because we both check them.

发件人：Ferruh Yigit <[email protected]> 收件人：humin (Q) <[email protected]>;dev <[email protected]> 抄 送：thomas <[email protected]>
时 间：2021-11-16 22:37:23
主 题：Re: [PATCH v2 0/2] performance optimized for hns3 PMD

Quoted 8 lines · Min Hu (Connor)

This patch set contains two ways to optimized Tx performance.

Chengwen Feng (2):
net/hns3: optimized Tx performance by mbuf fast free net/hns3: optimized Tx performance

  doc/guides/nics/features/hns3.ini |   1 +
  drivers/net/hns3/hns3_rxtx.c      | 129 +++++++++++++++

________________________________

Quoted 7 lines
  drivers/net/hns3/hns3_rxtx.h      |   2 +
  drivers/net/hns3/hns3_rxtx_vec.h |   9 +++
  4 files changed, 77 insertions(+), 64 deletions(-)

---
v2:
* document hns3.ini and fix 'TE_ETH_TX_OFFLOAD_MBUF_FAST_FREE'

Is Connor's sign off in the v1 dropped intentionally, or forgotten?

#12 lore raw
In reply to Ferruh Yigit
Quoted 3 lines · humin (Q)

Hi, ferruh,
I forgot it，please added it for me when you merged them.thank you.

ack, adding it back in next-net.

Quoted 43 lines

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

胡敏 Hu Min
Mobile: +86-13528728164 <tel:+86-13528728164>
Email: [email protected] <mailto:[email protected]>
*发件人：*Fengchengwen <[email protected]>
*收件人：*Ferruh Yigit <[email protected]>;humin (Q) <[email protected]>;dev <[email protected]> *抄 送：*thomas <[email protected]>
*时 间：*2021-11-16 23:04:08
*主 题：*RE: [PATCH v2 0/2] performance optimized for hns3 PMD

May be forgotten, I send v2 to him this morning, and maybe forgot adding the sign off. But the patch set is OK because we both check them.

*发件人：*Ferruh Yigit <[email protected]> *收件人：*humin (Q) <[email protected]>;dev <[email protected]> *抄 送：*thomas <[email protected]>
*时 间：*2021-11-16 22:37:23
*主 题：*Re: [PATCH v2 0/2] performance optimized for hns3 PMD

Quoted 8 lines · Min Hu (Connor)

This patch set contains two ways to optimized Tx performance.

Chengwen Feng (2):
   net/hns3: optimized Tx performance by mbuf fast free net/hns3: optimized Tx performance

  doc/guides/nics/features/hns3.ini |   1 + drivers/net/hns3/hns3_rxtx.c      | 129 +++++++++++++++

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Quoted 7 lines

  drivers/net/hns3/hns3_rxtx.h      |   2 +
  drivers/net/hns3/hns3_rxtx_vec.h |   9 +++

  4 files changed, 77 insertions(+), 64 deletions(-)

---
v2:
* document hns3.ini and fix 'TE_ETH_TX_OFFLOAD_MBUF_FAST_FREE'

Is Connor's sign off in the v1 dropped intentionally, or forgotten?

#13 lore raw
In reply to Min Hu (Connor)
Quoted 6 lines · Min Hu (Connor)

This patch set contains two ways to optimized Tx performance.

Chengwen Feng (2):
net/hns3: optimized Tx performance by mbuf fast free net/hns3: optimized Tx performance

Quoted 9 lines
  doc/guides/nics/features/hns3.ini |   1 +
  drivers/net/hns3/hns3_rxtx.c      | 129 +++++++++++++++---------------
  drivers/net/hns3/hns3_rxtx.h      |   2 +
  drivers/net/hns3/hns3_rxtx_vec.h  |   9 +++
  4 files changed, 77 insertions(+), 64 deletions(-)

---
v2:
* document hns3.ini and fix 'TE_ETH_TX_OFFLOAD_MBUF_FAST_FREE'

Series applied to dpdk-next-net/main, thanks.