Skip to contentSkip to conversation

Fix i40e/iavf queue reconfig and restarting

PATCH Superseded Bruce Richardson 3 participants 18 messages Aug 30, 2023 – Sep 5, 2023 Network drivers

lore
Share & tools
lore
#0 lore raw

cover Fix i40e/iavf queue reconfig and restarting

Fixes for a number of issues found when looking at stopping and restarting individual queues, sometimes with a reconfig in the middle. For i40e, it already claims to support runtime reconfig, while for iavf this support is not claimed, but appears to work ok once the bugs in the cleanup code are fixed, therefore the last patch enables it, by setting the appropriate capability flag.

Bruce Richardson (4):
net/i40e: fix buffer leak on Rx reconfiguration net/iavf: fix buffer leak on Tx queue stop net/iavf: fix restart of Rx queue on reconfigure net/iavf: add support for runtime queue reconfiguration

 doc/guides/nics/features/iavf.ini       |  2 ++
 drivers/net/i40e/i40e_rxtx.c            |  6 ++++++
 drivers/net/i40e/i40e_rxtx_vec_common.h |  1 +
 drivers/net/iavf/iavf_ethdev.c          |  4 +++-
 drivers/net/iavf/iavf_rxtx.c            |  6 ++++++
 drivers/net/iavf/iavf_rxtx_vec_avx512.c | 17 ++++++++---------
 drivers/net/iavf/iavf_rxtx_vec_common.h | 11 +++++------
 7 files changed, 31 insertions(+), 16 deletions(-)

--
2.39.2

#1 lore raw

PATCH 1/4 net/i40e: fix buffer leak on Rx reconfiguration

When reconfiguring a single queue on a device, the mbuf initializer value was not getting set, and remained at zero. This lead to mbuf leaks as the refcount was incorrect (0), so on free it wrapped around to UINT16_MAX. When setting up the mbuf initializer, also ensure that the queue is explicitly marked as using a vector function by setting the "rx_using_sse" flag.

Fixes a3c83a2527e1 ("net/i40e: enable runtime queue setup")
---
 drivers/net/i40e/i40e_rxtx.c            | 6 ++++++
 drivers/net/i40e/i40e_rxtx_vec_common.h | 1 +
 2 files changed, 7 insertions(+)
drivers/net/i40e/i40e_rxtx.c +6 -0
index b4f65b58fa..d96bbbb677 100644
@@ -1918,6 +1918,12 @@ i40e_dev_rx_queue_setup_runtime(struct rte_eth_dev *dev,
 		if (use_def_burst_func)
 			ad->rx_bulk_alloc_allowed = false;
 		i40e_set_rx_function(dev);
+
+		if (ad->rx_vec_allowed && i40e_rxq_vec_setup(rxq)) {
+			PMD_DRV_LOG(ERR, "Failed vector rx setup.");
+			return -EINVAL;
+		}
+
 		return 0;
 	} else if (ad->rx_vec_allowed && !rte_is_power_of_2(rxq->nb_rx_desc)) {
 		PMD_DRV_LOG(ERR, "Vector mode is allowed, but descriptor"
drivers/net/i40e/i40e_rxtx_vec_common.h +1 -0
index fe1a6ec75e..8b745630e4 100644
@@ -201,6 +201,7 @@ i40e_rxq_vec_setup_default(struct i40e_rx_queue *rxq)
 	rte_compiler_barrier();
 	p = (uintptr_t)&mb_def.rearm_data;
 	rxq->mbuf_initializer = *(uint64_t *)p;
+	rxq->rx_using_sse = 1;
 	return 0;
 }
 
Signature · 1 line
2.39.2
#2 lore raw

PATCH 2/4 net/iavf: fix buffer leak on Tx queue stop

In reply to Bruce Richardson

When a queue is stopped all buffers are meant to released from it, and then a new set of buffers reallocated on start. For the iavf code when using vector Tx, some buffers were left in the ring, and so those buffers were leaked. The buffers were missed as the release code only handled one side of a wrap-around case in the ring.

Fix the issue by doing a single iteration of the ring freeing all buffers in it, handling wraparound through a simple post-increment check.

Fixes 319c421f3890 ("net/avf: enable SSE Rx Tx")
Fixes 9ab9514c150e ("net/iavf: enable AVX512 for Tx")
---
 drivers/net/iavf/iavf_rxtx_vec_avx512.c | 17 ++++++++---------
 drivers/net/iavf/iavf_rxtx_vec_common.h | 11 +++++------
 2 files changed, 13 insertions(+), 15 deletions(-)
drivers/net/iavf/iavf_rxtx_vec_avx512.c +8 -9
index 3e66df5341..48337d5e03 100644
@@ -2460,20 +2460,19 @@ iavf_tx_queue_release_mbufs_avx512(struct iavf_tx_queue *txq)
 {
 	unsigned int i;
 	const uint16_t max_desc = (uint16_t)(txq->nb_tx_desc - 1);
+	const uint16_t end_desc = txq->tx_tail >> txq->use_ctx; /* next empty slot */
+	const uint16_t wrap_point = txq->nb_tx_desc >> txq->use_ctx;  /* end of SW ring */
 	struct iavf_tx_vec_entry *swr = (void *)txq->sw_ring;
 
 	if (!txq->sw_ring || txq->nb_free == max_desc)
 		return;
 
-	i = (txq->next_dd >> txq->use_ctx) + 1 -
-			(txq->rs_thresh >> txq->use_ctx);
-
-	if (txq->tx_tail < i) {
-		for (; i < (unsigned int)(txq->nb_tx_desc >> txq->use_ctx); i++) {
-			rte_pktmbuf_free_seg(swr[i].mbuf);
-			swr[i].mbuf = NULL;
-		}
-		i = 0;
+	i = (txq->next_dd - txq->rs_thresh + 1) >> txq->use_ctx;
+	while (i != end_desc) {
+		rte_pktmbuf_free_seg(swr[i].mbuf);
+		swr[i].mbuf = NULL;
+		if (++i == wrap_point)
+			i = 0;
 	}
 }
 
drivers/net/iavf/iavf_rxtx_vec_common.h +5 -6
index ddb13ce8c3..1fac957fe1 100644
@@ -186,12 +186,11 @@ _iavf_tx_queue_release_mbufs_vec(struct iavf_tx_queue *txq)
 		return;
 
 	i = txq->next_dd - txq->rs_thresh + 1;
-	if (txq->tx_tail < i) {
-		for (; i < txq->nb_tx_desc; i++) {
-			rte_pktmbuf_free_seg(txq->sw_ring[i].mbuf);
-			txq->sw_ring[i].mbuf = NULL;
-		}
-		i = 0;
+	while (i != txq->tx_tail) {
+		rte_pktmbuf_free_seg(txq->sw_ring[i].mbuf);
+		txq->sw_ring[i].mbuf = NULL;
+		if (++i == txq->nb_tx_desc)
+			i = 0;
 	}
 }
 
Signature · 1 line
2.39.2
#3 lore raw

PATCH 3/4 net/iavf: fix restart of Rx queue on reconfigure

In reply to Bruce Richardson

After reconfiguring an RX queue the mbuf_initializer value was not being correctly set. Fix this by calling the appropriate function if vector processing is enabled. This mirrors the behaviour by the i40e driver.

Fixes 319c421f3890 ("net/avf: enable SSE Rx Tx")
---
 drivers/net/iavf/iavf_rxtx.c | 6 ++++++
 1 file changed, 6 insertions(+)
drivers/net/iavf/iavf_rxtx.c +6 -0
index f7df4665d1..347f686831 100644
@@ -755,6 +755,12 @@ iavf_dev_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
 	if (check_rx_vec_allow(rxq) == false)
 		ad->rx_vec_allowed = false;
 
+	/* check vector conflict */
+	if (ad->rx_vec_allowed && iavf_rxq_vec_setup(rxq)) {
+		PMD_DRV_LOG(ERR, "Failed vector rx setup.");
+		return -EINVAL;
+	}
+
 	return 0;
 }
 
Signature · 1 line
2.39.2
#4 lore raw

PATCH 4/4 net/iavf: add support for runtime queue reconfiguration

In reply to Bruce Richardson

Unlike the i40e driver, the iavf driver does not advertise support for runtime reconfiguration, meaning that application using the same hardware may get different behaviour when using a PF vs a VF. On testing with a 40G NIC, the only blocker to reconfiguring an RX queue on the fly is the fact that this support is not advertised by the driver.

Add support for runtime reconfig by reporting it in the device info flags.

---
 doc/guides/nics/features/iavf.ini | 2 ++
 drivers/net/iavf/iavf_ethdev.c    | 4 +++-
 2 files changed, 5 insertions(+), 1 deletion(-)
doc/guides/nics/features/iavf.ini +2 -0
index b72cd98484..db4f92ce71 100644
@@ -11,6 +11,8 @@ Speed capabilities   = Y
 Link status          = Y
 Rx interrupt         = Y
 Queue start/stop     = Y
+Runtime Rx queue setup = Y
+Runtime Tx queue setup = Y
 Power mgmt address monitor = Y
 MTU update           = Y
 Scattered Rx         = Y
drivers/net/iavf/iavf_ethdev.c +3 -1
index f2fc5a5621..22fbd7d6b2 100644
@@ -1127,7 +1127,9 @@ iavf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
 	dev_info->reta_size = vf->vf_res->rss_lut_size;
 	dev_info->flow_type_rss_offloads = IAVF_RSS_OFFLOAD_ALL;
 	dev_info->max_mac_addrs = IAVF_NUM_MACADDR_MAX;
-	dev_info->dev_capa &= ~RTE_ETH_DEV_CAPA_FLOW_RULE_KEEP;
+	dev_info->dev_capa =
+		RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
+		RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
 	dev_info->rx_offload_capa =
 		RTE_ETH_RX_OFFLOAD_VLAN_STRIP |
 		RTE_ETH_RX_OFFLOAD_QINQ_STRIP |
Signature · 1 line
2.39.2
#5 lore raw

v2 cover Fix i40e/iavf queue reconfig and restarting

In reply to Bruce Richardson

Fixes for a number of issues found when looking at stopping and restarting individual queues, sometimes with a reconfig in the middle. For i40e, it already claims to support runtime reconfig, while for iavf this support is not claimed, but appears to work ok once the bugs in the cleanup code are fixed, therefore the last patch enables it, by setting the appropriate capability flag.

---
V2: Fix build issues with non-x86, non-arm architectures reported
by the CI.

Bruce Richardson (4):
net/i40e: fix buffer leak on Rx reconfiguration net/iavf: fix buffer leak on Tx queue stop net/iavf: fix restart of Rx queue on reconfigure net/iavf: add support for runtime queue reconfiguration

 doc/guides/nics/features/iavf.ini       |  2 ++
 drivers/net/i40e/i40e_rxtx.c            |  6 ++++++
 drivers/net/i40e/i40e_rxtx_vec_common.h |  1 +
 drivers/net/iavf/iavf_ethdev.c          |  4 +++-
 drivers/net/iavf/iavf_rxtx.c            |  7 +++++++
 drivers/net/iavf/iavf_rxtx_vec_avx512.c | 17 ++++++++---------
 drivers/net/iavf/iavf_rxtx_vec_common.h | 11 +++++------
 7 files changed, 32 insertions(+), 16 deletions(-)

--
2.39.2

#6 lore raw

PATCH v2 1/4 net/i40e: fix buffer leak on Rx reconfiguration

When reconfiguring a single queue on a device, the mbuf initializer value was not getting set, and remained at zero. This lead to mbuf leaks as the refcount was incorrect (0), so on free it wrapped around to UINT16_MAX. When setting up the mbuf initializer, also ensure that the queue is explicitly marked as using a vector function by setting the "rx_using_sse" flag.

Fixes a3c83a2527e1 ("net/i40e: enable runtime queue setup")
---
 drivers/net/i40e/i40e_rxtx.c            | 6 ++++++
 drivers/net/i40e/i40e_rxtx_vec_common.h | 1 +
 2 files changed, 7 insertions(+)
drivers/net/i40e/i40e_rxtx.c +6 -0
index b4f65b58fa..d96bbbb677 100644
@@ -1918,6 +1918,12 @@ i40e_dev_rx_queue_setup_runtime(struct rte_eth_dev *dev,
 		if (use_def_burst_func)
 			ad->rx_bulk_alloc_allowed = false;
 		i40e_set_rx_function(dev);
+
+		if (ad->rx_vec_allowed && i40e_rxq_vec_setup(rxq)) {
+			PMD_DRV_LOG(ERR, "Failed vector rx setup.");
+			return -EINVAL;
+		}
+
 		return 0;
 	} else if (ad->rx_vec_allowed && !rte_is_power_of_2(rxq->nb_rx_desc)) {
 		PMD_DRV_LOG(ERR, "Vector mode is allowed, but descriptor"
drivers/net/i40e/i40e_rxtx_vec_common.h +1 -0
index fe1a6ec75e..8b745630e4 100644
@@ -201,6 +201,7 @@ i40e_rxq_vec_setup_default(struct i40e_rx_queue *rxq)
 	rte_compiler_barrier();
 	p = (uintptr_t)&mb_def.rearm_data;
 	rxq->mbuf_initializer = *(uint64_t *)p;
+	rxq->rx_using_sse = 1;
 	return 0;
 }
 
Signature · 1 line
2.39.2
Acked-by on v2 1/4 Applied
#7 lore raw

RE: [PATCH v2 1/4] net/i40e: fix buffer leak on Rx reconfiguration

Quoted 52 lines

-----Original Message-----
From: Richardson, Bruce <[email protected]>
Sent: Thursday, August 31, 2023 8:34 PM
To: [email protected]
Cc: Richardson, Bruce <[email protected]>; Zhang, Qi Z <[email protected]>; [email protected]
Subject: [PATCH v2 1/4] net/i40e: fix buffer leak on Rx reconfiguration

When reconfiguring a single queue on a device, the mbuf initializer value was not getting set, and remained at zero. This lead to mbuf leaks as the refcount was incorrect (0), so on free it wrapped around to UINT16_MAX. When setting up the mbuf initializer, also ensure that the queue is explicitly marked as using a vector function by setting the "rx_using_sse" flag.

Fixes a3c83a2527e1 ("net/i40e: enable runtime queue setup")
---
 drivers/net/i40e/i40e_rxtx.c            | 6 ++++++
 drivers/net/i40e/i40e_rxtx_vec_common.h | 1 +
 2 files changed, 7 insertions(+)
drivers/net/i40e/i40e_rxtx.c index renamed from drivers/net/i40e/i40e_rxtx.c +0 -0

b4f65b58fa..d96bbbb677 100644

drivers/net/i40e/i40e_rxtx.c +6 -0
@@ -1918,6 +1918,12 @@ i40e_dev_rx_queue_setup_runtime(struct
 rte_eth_dev *dev,
 		if (use_def_burst_func)
 			ad->rx_bulk_alloc_allowed = false;
 		i40e_set_rx_function(dev);
+
+		if (ad->rx_vec_allowed && i40e_rxq_vec_setup(rxq)) {
+			PMD_DRV_LOG(ERR, "Failed vector rx setup.");
+			return -EINVAL;
+		}
+
 		return 0;
 	} else if (ad->rx_vec_allowed && !rte_is_power_of_2(rxq-
Quoted 1 line

nb_rx_desc)) {

PMD_DRV_LOG(ERR, "Vector mode is allowed, but
descriptor"

drivers/net/i40e/i40e_rxtx_vec_common.h +0 -0
index fe1a6ec75e..8b745630e4 100644

drivers/net/i40e/i40e_rxtx_vec_common.h i40e_rxq_vec_setup_default(struct i40e_rx_queue · line 205

Quoted 5 lines
drivers/net/i40e/i40e_rxtx_vec_common.h +1 -0
@@ -201,6 +201,7 @@ i40e_rxq_vec_setup_default(struct i40e_rx_queue
 *rxq)
 	rte_compiler_barrier();
 	p = (uintptr_t)&mb_def.rearm_data;
 	rxq->mbuf_initializer = *(uint64_t *)p;
+	rxq->rx_using_sse = 1;

I saw some further code clean can be done by leveraging this change.

Quoted 5 lines
drivers/net/i40e/i40e_rxtx_vec_common.h +0 -1
 	return 0;
 }
 
--

2.39.2

Applied to dpdk-next-net-intel.

Thanks
Qi

#8 lore raw

PATCH v2 2/4 net/iavf: fix buffer leak on Tx queue stop

In reply to Bruce Richardson

When a queue is stopped all buffers are meant to released from it, and then a new set of buffers reallocated on start. For the iavf code when using vector Tx, some buffers were left in the ring, and so those buffers were leaked. The buffers were missed as the release code only handled one side of a wrap-around case in the ring.

Fix the issue by doing a single iteration of the ring freeing all buffers in it, handling wraparound through a simple post-increment check.

Fixes 319c421f3890 ("net/avf: enable SSE Rx Tx")
Fixes 9ab9514c150e ("net/iavf: enable AVX512 for Tx")
---
 drivers/net/iavf/iavf_rxtx_vec_avx512.c | 17 ++++++++---------
 drivers/net/iavf/iavf_rxtx_vec_common.h | 11 +++++------
 2 files changed, 13 insertions(+), 15 deletions(-)
drivers/net/iavf/iavf_rxtx_vec_avx512.c +8 -9
index 3e66df5341..48337d5e03 100644
@@ -2460,20 +2460,19 @@ iavf_tx_queue_release_mbufs_avx512(struct iavf_tx_queue *txq)
 {
 	unsigned int i;
 	const uint16_t max_desc = (uint16_t)(txq->nb_tx_desc - 1);
+	const uint16_t end_desc = txq->tx_tail >> txq->use_ctx; /* next empty slot */
+	const uint16_t wrap_point = txq->nb_tx_desc >> txq->use_ctx;  /* end of SW ring */
 	struct iavf_tx_vec_entry *swr = (void *)txq->sw_ring;
 
 	if (!txq->sw_ring || txq->nb_free == max_desc)
 		return;
 
-	i = (txq->next_dd >> txq->use_ctx) + 1 -
-			(txq->rs_thresh >> txq->use_ctx);
-
-	if (txq->tx_tail < i) {
-		for (; i < (unsigned int)(txq->nb_tx_desc >> txq->use_ctx); i++) {
-			rte_pktmbuf_free_seg(swr[i].mbuf);
-			swr[i].mbuf = NULL;
-		}
-		i = 0;
+	i = (txq->next_dd - txq->rs_thresh + 1) >> txq->use_ctx;
+	while (i != end_desc) {
+		rte_pktmbuf_free_seg(swr[i].mbuf);
+		swr[i].mbuf = NULL;
+		if (++i == wrap_point)
+			i = 0;
 	}
 }
 
drivers/net/iavf/iavf_rxtx_vec_common.h +5 -6
index ddb13ce8c3..1fac957fe1 100644
@@ -186,12 +186,11 @@ _iavf_tx_queue_release_mbufs_vec(struct iavf_tx_queue *txq)
 		return;
 
 	i = txq->next_dd - txq->rs_thresh + 1;
-	if (txq->tx_tail < i) {
-		for (; i < txq->nb_tx_desc; i++) {
-			rte_pktmbuf_free_seg(txq->sw_ring[i].mbuf);
-			txq->sw_ring[i].mbuf = NULL;
-		}
-		i = 0;
+	while (i != txq->tx_tail) {
+		rte_pktmbuf_free_seg(txq->sw_ring[i].mbuf);
+		txq->sw_ring[i].mbuf = NULL;
+		if (++i == txq->nb_tx_desc)
+			i = 0;
 	}
 }
 
Signature · 1 line
2.39.2
Acked-by on v2 2/4
#9 lore raw

RE: [PATCH v2 2/4] net/iavf: fix buffer leak on Tx queue stop

Quoted 20 lines

-----Original Message-----
From: Richardson, Bruce <[email protected]>
Sent: Thursday, August 31, 2023 8:34 PM
To: [email protected]
Cc: Richardson, Bruce <[email protected]>; Lu, Wenzhuo <[email protected]>; Wu, Jingjing <[email protected]>; [email protected]
Subject: [PATCH v2 2/4] net/iavf: fix buffer leak on Tx queue stop

When a queue is stopped all buffers are meant to released from it, and then a new set of buffers reallocated on start. For the iavf code when using vector Tx, some buffers were left in the ring, and so those buffers were leaked. The buffers were missed as the release code only handled one side of a wrap-around case in the ring.

Fix the issue by doing a single iteration of the ring freeing all buffers in it, handling wraparound through a simple post-increment check.

Quoted 7 lines
Fixes 319c421f3890 ("net/avf: enable SSE Rx Tx")
Fixes 9ab9514c150e ("net/iavf: enable AVX512 for Tx")
Applied
#10 lore raw

RE: [PATCH v2 2/4] net/iavf: fix buffer leak on Tx queue stop

Quoted 29 lines

-----Original Message-----
From: Lu, Wenzhuo <[email protected]>
Sent: Monday, September 4, 2023 10:18 AM
To: Richardson, Bruce <[email protected]>; [email protected]
Cc: Wu, Jingjing <[email protected]>; [email protected]
Subject: RE: [PATCH v2 2/4] net/iavf: fix buffer leak on Tx queue stop

Quoted 18 lines

-----Original Message-----
From: Richardson, Bruce <[email protected]>
Sent: Thursday, August 31, 2023 8:34 PM
To: [email protected]
Cc: Richardson, Bruce <[email protected]>; Lu, Wenzhuo <[email protected]>; Wu, Jingjing <[email protected]>; [email protected]
Subject: [PATCH v2 2/4] net/iavf: fix buffer leak on Tx queue stop

When a queue is stopped all buffers are meant to released from it, and then a new set of buffers reallocated on start. For the iavf code when using vector Tx, some buffers were left in the ring, and so those buffers were leaked. The buffers were missed as the release code only handled one side of a wrap-around case in the ring.

Fix the issue by doing a single iteration of the ring freeing all buffers in it, handling wraparound through a simple post-increment check.

Quoted 8 lines
Quoted 7 lines
Fixes 319c421f3890 ("net/avf: enable SSE Rx Tx")
Fixes 9ab9514c150e ("net/iavf: enable AVX512 for Tx")

Applied to dpdk-next-net-intel.

Thanks
Qi

#11 lore raw

PATCH v2 3/4 net/iavf: fix restart of Rx queue on reconfigure

In reply to Bruce Richardson

After reconfiguring an RX queue the mbuf_initializer value was not being correctly set. Fix this by calling the appropriate function if vector processing is enabled. This mirrors the behaviour by the i40e driver.

Fixes 319c421f3890 ("net/avf: enable SSE Rx Tx")
---
 drivers/net/iavf/iavf_rxtx.c | 7 +++++++
 1 file changed, 7 insertions(+)
drivers/net/iavf/iavf_rxtx.c +7 -0
index f7df4665d1..797cdda4b2 100644
@@ -755,6 +755,13 @@ iavf_dev_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
 	if (check_rx_vec_allow(rxq) == false)
 		ad->rx_vec_allowed = false;
 
+#if defined RTE_ARCH_X86 || defined RTE_ARCH_ARM
+	/* check vector conflict */
+	if (ad->rx_vec_allowed && iavf_rxq_vec_setup(rxq)) {
+		PMD_DRV_LOG(ERR, "Failed vector rx setup.");
+		return -EINVAL;
+	}
+#endif
 	return 0;
 }
 
Signature · 1 line
2.39.2
#12 lore raw

RE: [PATCH v2 3/4] net/iavf: fix restart of Rx queue on reconfigure

Quoted 29 lines

-----Original Message-----
From: Bruce Richardson <[email protected]>
Sent: Thursday, August 31, 2023 8:34 PM
To: [email protected]
Cc: Richardson, Bruce <[email protected]>; Wu, Jingjing <[email protected]>; [email protected]
Subject: [PATCH v2 3/4] net/iavf: fix restart of Rx queue on reconfigure

After reconfiguring an RX queue the mbuf_initializer value was not being correctly set. Fix this by calling the appropriate function if vector processing is enabled. This mirrors the behaviour by the i40e driver.

Fixes 319c421f3890 ("net/avf: enable SSE Rx Tx")
---
 drivers/net/iavf/iavf_rxtx.c | 7 +++++++
 1 file changed, 7 insertions(+)
drivers/net/iavf/iavf_rxtx.c index renamed from drivers/net/iavf/iavf_rxtx.c +0 -0

f7df4665d1..797cdda4b2 100644

drivers/net/iavf/iavf_rxtx.c +0 -0
@@ -755,6 +755,13 @@ iavf_dev_rx_queue_setup(struct rte_eth_dev *dev,
 uint16_t queue_idx,

drivers/net/iavf/iavf_rxtx.c iavf_dev_rx_queue_setup(struct rte_eth_dev *dev, · line 765

Quoted 10 lines
drivers/net/iavf/iavf_rxtx.c +7 -0
 	if (check_rx_vec_allow(rxq) == false)
 		ad->rx_vec_allowed = false;
 
+#if defined RTE_ARCH_X86 || defined RTE_ARCH_ARM
+	/* check vector conflict */
+	if (ad->rx_vec_allowed && iavf_rxq_vec_setup(rxq)) {
+		PMD_DRV_LOG(ERR, "Failed vector rx setup.");
+		return -EINVAL;
+	}
+#endif

Bruce:

May I know more details about how to reproduce this issue?
As the iavf PMD does not support RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP (i40e does)
Which indicates before we call rte_eth_rx_queue_setup, the device must be stopped (the flag will be checked in rte_eth_rx_queue_setup)

So if we do below steps

rte_eth_dev_stop ..
rte_eth_rx_queue_setup
rte_eth_dev_start

the iavf_rxq_vec_setup should be invoked in rte_eth_dev_start -> iavf_set_rx_function

anything I missed?

Thanks
Qi

Quoted 5 lines
drivers/net/iavf/iavf_rxtx.c +0 -1
 	return 0;
 }
 
--

2.39.2

#13 lore raw

RE: [PATCH v2 3/4] net/iavf: fix restart of Rx queue on reconfigure

Quoted 54 lines

-----Original Message-----
From: Zhang, Qi Z
Sent: Monday, September 4, 2023 9:15 AM
To: Bruce Richardson <[email protected]>; [email protected]
Cc: Richardson, Bruce <[email protected]>; Wu, Jingjing <[email protected]>; [email protected]
Subject: RE: [PATCH v2 3/4] net/iavf: fix restart of Rx queue on reconfigure

Quoted 29 lines

-----Original Message-----
From: Bruce Richardson <[email protected]>
Sent: Thursday, August 31, 2023 8:34 PM
To: [email protected]
Cc: Richardson, Bruce <[email protected]>; Wu, Jingjing <[email protected]>; [email protected]
Subject: [PATCH v2 3/4] net/iavf: fix restart of Rx queue on reconfigure

After reconfiguring an RX queue the mbuf_initializer value was not being correctly set. Fix this by calling the appropriate function if vector processing is enabled. This mirrors the behaviour by the i40e driver.

Fixes 319c421f3890 ("net/avf: enable SSE Rx Tx")
---
 drivers/net/iavf/iavf_rxtx.c | 7 +++++++
 1 file changed, 7 insertions(+)
(quoted diff) +0 -0

b/drivers/net/iavf/iavf_rxtx.c index
f7df4665d1..797cdda4b2 100644

drivers/net/iavf/iavf_rxtx.c +0 -0
@@ -755,6 +755,13 @@ iavf_dev_rx_queue_setup(struct rte_eth_dev *dev,
 uint16_t queue_idx,

drivers/net/iavf/iavf_rxtx.c iavf_dev_rx_queue_setup(struct rte_eth_dev *dev, · line 765

Quoted 10 lines
drivers/net/iavf/iavf_rxtx.c +7 -0
 	if (check_rx_vec_allow(rxq) == false)
 		ad->rx_vec_allowed = false;
 
+#if defined RTE_ARCH_X86 || defined RTE_ARCH_ARM
+	/* check vector conflict */
+	if (ad->rx_vec_allowed && iavf_rxq_vec_setup(rxq)) {
+		PMD_DRV_LOG(ERR, "Failed vector rx setup.");
+		return -EINVAL;
+	}
+#endif

Bruce:

Quoted 3 lines

May I know more details about how to reproduce this issue? As the iavf PMD does not support
RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP (i40e does)

OK, not sure if the patch 4/4 answered my question 😊

should I squash patch 3, 4 into one? , for my understanding patch 3 doesn't appear to be a bug fix unless we announce RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP.

Quoted 22 lines

Which indicates before we call rte_eth_rx_queue_setup, the device
must be stopped (the flag will be checked in rte_eth_rx_queue_setup)

	So if we do below steps

	rte_eth_dev_stop ..
	rte_eth_rx_queue_setup
	rte_eth_dev_start

	the iavf_rxq_vec_setup should be invoked in rte_eth_dev_start ->

iavf_set_rx_function

anything I missed?

Thanks
Qi

Quoted 5 lines
drivers/net/iavf/iavf_rxtx.c +0 -1
 	return 0;
 }
 
--

2.39.2

#14 lore raw

Re: [PATCH v2 3/4] net/iavf: fix restart of Rx queue on reconfigure

Quoted 54 lines · Qi Z
Quoted 54 lines

-----Original Message-----
From: Zhang, Qi Z
Sent: Monday, September 4, 2023 9:15 AM
To: Bruce Richardson <[email protected]>; [email protected]
Cc: Richardson, Bruce <[email protected]>; Wu, Jingjing <[email protected]>; [email protected]
Subject: RE: [PATCH v2 3/4] net/iavf: fix restart of Rx queue on reconfigure

Quoted 29 lines

-----Original Message-----
From: Bruce Richardson <[email protected]>
Sent: Thursday, August 31, 2023 8:34 PM
To: [email protected]
Cc: Richardson, Bruce <[email protected]>; Wu, Jingjing <[email protected]>; [email protected]
Subject: [PATCH v2 3/4] net/iavf: fix restart of Rx queue on reconfigure

After reconfiguring an RX queue the mbuf_initializer value was not being correctly set. Fix this by calling the appropriate function if vector processing is enabled. This mirrors the behaviour by the i40e driver.

Fixes 319c421f3890 ("net/avf: enable SSE Rx Tx")
---
 drivers/net/iavf/iavf_rxtx.c | 7 +++++++
 1 file changed, 7 insertions(+)
(quoted diff) +0 -0

b/drivers/net/iavf/iavf_rxtx.c index
f7df4665d1..797cdda4b2 100644

drivers/net/iavf/iavf_rxtx.c +0 -0
@@ -755,6 +755,13 @@ iavf_dev_rx_queue_setup(struct rte_eth_dev *dev,
 uint16_t queue_idx,

drivers/net/iavf/iavf_rxtx.c iavf_dev_rx_queue_setup(struct rte_eth_dev *dev, · line 765

Quoted 10 lines
drivers/net/iavf/iavf_rxtx.c +7 -0
 	if (check_rx_vec_allow(rxq) == false)
 		ad->rx_vec_allowed = false;
 
+#if defined RTE_ARCH_X86 || defined RTE_ARCH_ARM
+	/* check vector conflict */
+	if (ad->rx_vec_allowed && iavf_rxq_vec_setup(rxq)) {
+		PMD_DRV_LOG(ERR, "Failed vector rx setup.");
+		return -EINVAL;
+	}
+#endif

Bruce:

Quoted 8 lines
Quoted 3 lines

May I know more details about how to reproduce this issue? As the iavf PMD does not support
RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP (i40e does)

OK, not sure if the patch 4/4 answered my question 😊

should I squash patch 3, 4 into one? , for my understanding patch 3 doesn't appear to be a bug fix unless we announce RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP.

You may have a point. I was experimenting with queue reconfiguration which is where I hit this issue.
However, even without queue reconfig support, the device still needs to support queue-stop followed by queue-start, I think, and there may still be an issue there - I'll have to check.

/Bruce

#15 lore raw

RE: [PATCH v2 3/4] net/iavf: fix restart of Rx queue on reconfigure

Quoted 73 lines

-----Original Message-----
From: Richardson, Bruce <[email protected]>
Sent: Monday, September 4, 2023 3:54 PM
To: Zhang, Qi Z <[email protected]>
Cc: [email protected]; Wu, Jingjing <[email protected]>; [email protected]
Subject: Re: [PATCH v2 3/4] net/iavf: fix restart of Rx queue on reconfigure

Quoted 22 lines · Qi Z
Quoted 22 lines

-----Original Message-----
From: Zhang, Qi Z
Sent: Monday, September 4, 2023 9:15 AM
To: Bruce Richardson <[email protected]>; [email protected]
Cc: Richardson, Bruce <[email protected]>; Wu, Jingjing <[email protected]>; [email protected]
Subject: RE: [PATCH v2 3/4] net/iavf: fix restart of Rx queue on reconfigure

Quoted 9 lines

-----Original Message-----
From: Bruce Richardson <[email protected]>
Sent: Thursday, August 31, 2023 8:34 PM
To: [email protected]
Cc: Richardson, Bruce <[email protected]>; Wu, Jingjing <[email protected]>; [email protected]
Subject: [PATCH v2 3/4] net/iavf: fix restart of Rx queue on reconfigure

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

After reconfiguring an RX queue the mbuf_initializer value was not being correctly set. Fix this by calling the appropriate function if vector processing is enabled. This mirrors the behaviour by the i40e

driver.

Quoted 30 lines
Quoted 30 lines
Quoted 17 lines
Fixes 319c421f3890 ("net/avf: enable SSE Rx Tx")
---
 drivers/net/iavf/iavf_rxtx.c | 7 +++++++
 1 file changed, 7 insertions(+)
(quoted diff) +0 -0

b/drivers/net/iavf/iavf_rxtx.c index
f7df4665d1..797cdda4b2 100644

drivers/net/iavf/iavf_rxtx.c +0 -0
@@ -755,6 +755,13 @@ iavf_dev_rx_queue_setup(struct rte_eth_dev
 *dev, uint16_t queue_idx,

drivers/net/iavf/iavf_rxtx.c iavf_dev_rx_queue_setup(struct rte_eth_dev · line 765

Quoted 10 lines
drivers/net/iavf/iavf_rxtx.c +7 -0
 	if (check_rx_vec_allow(rxq) == false)
 		ad->rx_vec_allowed = false;
 
+#if defined RTE_ARCH_X86 || defined RTE_ARCH_ARM
+	/* check vector conflict */
+	if (ad->rx_vec_allowed && iavf_rxq_vec_setup(rxq)) {
+		PMD_DRV_LOG(ERR, "Failed vector rx setup.");
+		return -EINVAL;
+	}
+#endif

Bruce:

Quoted 7 lines
Quoted 3 lines

May I know more details about how to reproduce this issue? As the iavf PMD does not support
RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP (i40e does)

OK, not sure if the patch 4/4 answered my question 😊

should I squash patch 3, 4 into one? , for my understanding patch 3 doesn't

Quoted 8 lines

appear to be a bug fix unless we announce
RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP.

Quoted 1 line

You may have a point. I was experimenting with queue reconfiguration which is where I hit this issue.
However, even without queue reconfig support, the device still needs to support queue-stop followed by queue-start, I think, and there may still be an issue there - I'll have to check.

Yes, queue start / stop should be supported.
Btw, I didn't see mbuf_initializer has been reset during queue stop / start, it might be a different issue.

Quoted 1 line

/Bruce

#16 lore raw

PATCH v2 4/4 net/iavf: add support for runtime queue reconfiguration

In reply to Bruce Richardson

Unlike the i40e driver, the iavf driver does not advertise support for runtime reconfiguration, meaning that application using the same hardware may get different behaviour when using a PF vs a VF. On testing with a 40G NIC, the only blocker to reconfiguring an RX queue on the fly is the fact that this support is not advertised by the driver.

Add support for runtime reconfig by reporting it in the device info flags.

---
 doc/guides/nics/features/iavf.ini | 2 ++
 drivers/net/iavf/iavf_ethdev.c    | 4 +++-
 2 files changed, 5 insertions(+), 1 deletion(-)
doc/guides/nics/features/iavf.ini +2 -0
index b72cd98484..db4f92ce71 100644
@@ -11,6 +11,8 @@ Speed capabilities   = Y
 Link status          = Y
 Rx interrupt         = Y
 Queue start/stop     = Y
+Runtime Rx queue setup = Y
+Runtime Tx queue setup = Y
 Power mgmt address monitor = Y
 MTU update           = Y
 Scattered Rx         = Y
drivers/net/iavf/iavf_ethdev.c +3 -1
index f2fc5a5621..22fbd7d6b2 100644
@@ -1127,7 +1127,9 @@ iavf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
 	dev_info->reta_size = vf->vf_res->rss_lut_size;
 	dev_info->flow_type_rss_offloads = IAVF_RSS_OFFLOAD_ALL;
 	dev_info->max_mac_addrs = IAVF_NUM_MACADDR_MAX;
-	dev_info->dev_capa &= ~RTE_ETH_DEV_CAPA_FLOW_RULE_KEEP;
+	dev_info->dev_capa =
+		RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
+		RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
 	dev_info->rx_offload_capa =
 		RTE_ETH_RX_OFFLOAD_VLAN_STRIP |
 		RTE_ETH_RX_OFFLOAD_QINQ_STRIP |
Signature · 1 line
2.39.2
Acked-by on v2 4/4
#17 lore raw

RE: [PATCH v2 4/4] net/iavf: add support for runtime queue reconfiguration

Quoted 53 lines

-----Original Message-----
From: Bruce Richardson <[email protected]>
Sent: Thursday, August 31, 2023 8:34 PM
To: [email protected]
Cc: Richardson, Bruce <[email protected]>
Subject: [PATCH v2 4/4] net/iavf: add support for runtime queue reconfiguration

Unlike the i40e driver, the iavf driver does not advertise support for runtime reconfiguration, meaning that application using the same hardware may get different behaviour when using a PF vs a VF. On testing with a 40G NIC, the only blocker to reconfiguring an RX queue on the fly is the fact that this support is not advertised by the driver.

Add support for runtime reconfig by reporting it in the device info flags.

---
 doc/guides/nics/features/iavf.ini | 2 ++
 drivers/net/iavf/iavf_ethdev.c    | 4 +++-
 2 files changed, 5 insertions(+), 1 deletion(-)
doc/guides/nics/features/iavf.ini +2 -0
index b72cd98484..db4f92ce71 100644
@@ -11,6 +11,8 @@ Speed capabilities   = Y
 Link status          = Y
 Rx interrupt         = Y
 Queue start/stop     = Y
+Runtime Rx queue setup = Y
+Runtime Tx queue setup = Y
 Power mgmt address monitor = Y
 MTU update           = Y
 Scattered Rx         = Y
drivers/net/iavf/iavf_ethdev.c +3 -1
index f2fc5a5621..22fbd7d6b2 100644
@@ -1127,7 +1127,9 @@ iavf_dev_info_get(struct rte_eth_dev *dev, struct
 rte_eth_dev_info *dev_info)
 	dev_info->reta_size = vf->vf_res->rss_lut_size;
 	dev_info->flow_type_rss_offloads = IAVF_RSS_OFFLOAD_ALL;
 	dev_info->max_mac_addrs = IAVF_NUM_MACADDR_MAX;
-	dev_info->dev_capa &= ~RTE_ETH_DEV_CAPA_FLOW_RULE_KEEP;
+	dev_info->dev_capa =
+		RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
+		RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
 	dev_info->rx_offload_capa =
 		RTE_ETH_RX_OFFLOAD_VLAN_STRIP |
Quoted 3 lines

RTE_ETH_RX_OFFLOAD_QINQ_STRIP |
--
2.39.2

Squashed with Patch 3/4, applied to dpdk-next-net-intel.

Thanks
Qi