Fix i40e/iavf queue reconfig and restarting
cover Fix i40e/iavf queue reconfig and restarting
Fixes for a number of issues found when looking at stopping and restarting individual queues, sometimes with a reconfig in the middle. For i40e, it already claims to support runtime reconfig, while for iavf this support is not claimed, but appears to work ok once the bugs in the cleanup code are fixed, therefore the last patch enables it, by setting the appropriate capability flag.
Bruce Richardson (4):
net/i40e: fix buffer leak on Rx reconfiguration net/iavf: fix buffer leak on Tx queue stop net/iavf: fix restart of Rx queue on reconfigure net/iavf: add support for runtime queue reconfiguration
doc/guides/nics/features/iavf.ini | 2 ++ drivers/net/i40e/i40e_rxtx.c | 6 ++++++ drivers/net/i40e/i40e_rxtx_vec_common.h | 1 + drivers/net/iavf/iavf_ethdev.c | 4 +++- drivers/net/iavf/iavf_rxtx.c | 6 ++++++ drivers/net/iavf/iavf_rxtx_vec_avx512.c | 17 ++++++++--------- drivers/net/iavf/iavf_rxtx_vec_common.h | 11 +++++------ 7 files changed, 31 insertions(+), 16 deletions(-)
--
2.39.2
PATCH 1/4 net/i40e: fix buffer leak on Rx reconfiguration
When reconfiguring a single queue on a device, the mbuf initializer value was not getting set, and remained at zero. This lead to mbuf leaks as the refcount was incorrect (0), so on free it wrapped around to UINT16_MAX. When setting up the mbuf initializer, also ensure that the queue is explicitly marked as using a vector function by setting the "rx_using_sse" flag.
--- drivers/net/i40e/i40e_rxtx.c | 6 ++++++ drivers/net/i40e/i40e_rxtx_vec_common.h | 1 + 2 files changed, 7 insertions(+)
drivers/net/i40e/i40e_rxtx.c +6 -0
@@ -1918,6 +1918,12 @@ i40e_dev_rx_queue_setup_runtime(struct rte_eth_dev *dev,if (use_def_burst_func) ad->rx_bulk_alloc_allowed = false;i40e_set_rx_function (dev); + +if (ad->rx_vec_allowed &&i40e_rxq_vec_setup (rxq)) { +PMD_DRV_LOG (ERR ,"Failed vector rx setup." ); +return -EINVAL ; + } +return 0 ; }else if (ad->rx_vec_allowed && !rte_is_power_of_2 (rxq->nb_rx_desc)) { PMD_DRV_LOG(ERR, "Vector mode is allowed, but descriptor"
drivers/net/i40e/i40e_rxtx_vec_common.h +1 -0
@@ -201,6 +201,7 @@ i40e_rxq_vec_setup_default(struct i40e_rx_queue *rxq)rte_compiler_barrier (); p = (uintptr_t )&mb_def.rearm_data; rxq->mbuf_initializer = *(uint64_t *)p; + rxq->rx_using_sse =1 ;return 0 ; }
Signature · 1 line
2.39.2
PATCH 2/4 net/iavf: fix buffer leak on Tx queue stop
In reply to Bruce RichardsonWhen a queue is stopped all buffers are meant to released from it, and then a new set of buffers reallocated on start. For the iavf code when using vector Tx, some buffers were left in the ring, and so those buffers were leaked. The buffers were missed as the release code only handled one side of a wrap-around case in the ring.
Fix the issue by doing a single iteration of the ring freeing all buffers in it, handling wraparound through a simple post-increment check.
--- drivers/net/iavf/iavf_rxtx_vec_avx512.c | 17 ++++++++--------- drivers/net/iavf/iavf_rxtx_vec_common.h | 11 +++++------ 2 files changed, 13 insertions(+), 15 deletions(-)
drivers/net/iavf/iavf_rxtx_vec_avx512.c +8 -9
@@ -2460,20 +2460,19 @@ iavf_tx_queue_release_mbufs_avx512(struct iavf_tx_queue *txq) {unsigned int i;const uint16_t max_desc = (uint16_t )(txq->nb_tx_desc -1 ); +const uint16_t end_desc = txq->tx_tail >> txq->use_ctx;/* next empty slot */ +const uint16_t wrap_point = txq->nb_tx_desc >> txq->use_ctx;/* end of SW ring */ struct iavf_tx_vec_entry *swr = (void *)txq->sw_ring;if (!txq->sw_ring || txq->nb_free == max_desc)return ; - i = (txq->next_dd >> txq->use_ctx) +1 - - (txq->rs_thresh >> txq->use_ctx); - -if (txq->tx_tail < i) { -for (; i < (unsigned int )(txq->nb_tx_desc >> txq->use_ctx); i++) { -rte_pktmbuf_free_seg (swr[i].mbuf); - swr[i].mbuf =NULL ; - } - i =0 ; + i = (txq->next_dd - txq->rs_thresh +1 ) >> txq->use_ctx; +while (i != end_desc) { +rte_pktmbuf_free_seg (swr[i].mbuf); + swr[i].mbuf =NULL ; +if (++i == wrap_point) + i =0 ; } }
drivers/net/iavf/iavf_rxtx_vec_common.h +5 -6
@@ -186,12 +186,11 @@ _iavf_tx_queue_release_mbufs_vec(struct iavf_tx_queue *txq)return ; i = txq->next_dd - txq->rs_thresh +1 ; -if (txq->tx_tail < i) { -for (; i < txq->nb_tx_desc; i++) { -rte_pktmbuf_free_seg (txq->sw_ring[i].mbuf); - txq->sw_ring[i].mbuf =NULL ; - } - i =0 ; +while (i != txq->tx_tail) { +rte_pktmbuf_free_seg (txq->sw_ring[i].mbuf); + txq->sw_ring[i].mbuf =NULL ; +if (++i == txq->nb_tx_desc) + i =0 ; } }
Signature · 1 line
2.39.2
PATCH 3/4 net/iavf: fix restart of Rx queue on reconfigure
In reply to Bruce RichardsonAfter reconfiguring an RX queue the mbuf_initializer value was not being correctly set. Fix this by calling the appropriate function if vector processing is enabled. This mirrors the behaviour by the i40e driver.
--- drivers/net/iavf/iavf_rxtx.c | 6 ++++++ 1 file changed, 6 insertions(+)
drivers/net/iavf/iavf_rxtx.c +6 -0
@@ -755,6 +755,12 @@ iavf_dev_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,if (check_rx_vec_allow (rxq) == false) ad->rx_vec_allowed = false; +/* check vector conflict */ +if (ad->rx_vec_allowed &&iavf_rxq_vec_setup (rxq)) { +PMD_DRV_LOG (ERR ,"Failed vector rx setup." ); +return -EINVAL ; + } +return 0 ; }
Signature · 1 line
2.39.2
PATCH 4/4 net/iavf: add support for runtime queue reconfiguration
In reply to Bruce RichardsonUnlike the i40e driver, the iavf driver does not advertise support for runtime reconfiguration, meaning that application using the same hardware may get different behaviour when using a PF vs a VF. On testing with a 40G NIC, the only blocker to reconfiguring an RX queue on the fly is the fact that this support is not advertised by the driver.
Add support for runtime reconfig by reporting it in the device info flags.
--- doc/guides/nics/features/iavf.ini | 2 ++ drivers/net/iavf/iavf_ethdev.c | 4 +++- 2 files changed, 5 insertions(+), 1 deletion(-)
doc/guides/nics/features/iavf.ini +2 -0
@@ -11,6 +11,8 @@ Speed capabilities = Y Link status = Y Rx interrupt = Y Queue start/stop = Y +Runtime Rx queue setup = Y +Runtime Tx queue setup = Y Power mgmt address monitor = Y MTU update = Y Scattered Rx = Y
drivers/net/iavf/iavf_ethdev.c +3 -1
@@ -1127,7 +1127,9 @@ iavf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) dev_info->reta_size = vf->vf_res->rss_lut_size; dev_info->flow_type_rss_offloads =IAVF_RSS_OFFLOAD_ALL ; dev_info->max_mac_addrs =IAVF_NUM_MACADDR_MAX ; - dev_info->dev_capa &= ~RTE_ETH_DEV_CAPA_FLOW_RULE_KEEP ; + dev_info->dev_capa = +RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP | +RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP ; dev_info->rx_offload_capa =RTE_ETH_RX_OFFLOAD_VLAN_STRIP | RTE_ETH_RX_OFFLOAD_QINQ_STRIP |
Signature · 1 line
2.39.2
v2 cover Fix i40e/iavf queue reconfig and restarting
In reply to Bruce RichardsonFixes for a number of issues found when looking at stopping and restarting individual queues, sometimes with a reconfig in the middle. For i40e, it already claims to support runtime reconfig, while for iavf this support is not claimed, but appears to work ok once the bugs in the cleanup code are fixed, therefore the last patch enables it, by setting the appropriate capability flag.
---
V2: Fix build issues with non-x86, non-arm architectures reported
by the CI.
Bruce Richardson (4):
net/i40e: fix buffer leak on Rx reconfiguration net/iavf: fix buffer leak on Tx queue stop net/iavf: fix restart of Rx queue on reconfigure net/iavf: add support for runtime queue reconfiguration
doc/guides/nics/features/iavf.ini | 2 ++ drivers/net/i40e/i40e_rxtx.c | 6 ++++++ drivers/net/i40e/i40e_rxtx_vec_common.h | 1 + drivers/net/iavf/iavf_ethdev.c | 4 +++- drivers/net/iavf/iavf_rxtx.c | 7 +++++++ drivers/net/iavf/iavf_rxtx_vec_avx512.c | 17 ++++++++--------- drivers/net/iavf/iavf_rxtx_vec_common.h | 11 +++++------ 7 files changed, 32 insertions(+), 16 deletions(-)
--
2.39.2
PATCH v2 1/4 net/i40e: fix buffer leak on Rx reconfiguration
When reconfiguring a single queue on a device, the mbuf initializer value was not getting set, and remained at zero. This lead to mbuf leaks as the refcount was incorrect (0), so on free it wrapped around to UINT16_MAX. When setting up the mbuf initializer, also ensure that the queue is explicitly marked as using a vector function by setting the "rx_using_sse" flag.
--- drivers/net/i40e/i40e_rxtx.c | 6 ++++++ drivers/net/i40e/i40e_rxtx_vec_common.h | 1 + 2 files changed, 7 insertions(+)
drivers/net/i40e/i40e_rxtx.c +6 -0
@@ -1918,6 +1918,12 @@ i40e_dev_rx_queue_setup_runtime(struct rte_eth_dev *dev,if (use_def_burst_func) ad->rx_bulk_alloc_allowed = false;i40e_set_rx_function (dev); + +if (ad->rx_vec_allowed &&i40e_rxq_vec_setup (rxq)) { +PMD_DRV_LOG (ERR ,"Failed vector rx setup." ); +return -EINVAL ; + } +return 0 ; }else if (ad->rx_vec_allowed && !rte_is_power_of_2 (rxq->nb_rx_desc)) { PMD_DRV_LOG(ERR, "Vector mode is allowed, but descriptor"
drivers/net/i40e/i40e_rxtx_vec_common.h +1 -0
@@ -201,6 +201,7 @@ i40e_rxq_vec_setup_default(struct i40e_rx_queue *rxq)rte_compiler_barrier (); p = (uintptr_t )&mb_def.rearm_data; rxq->mbuf_initializer = *(uint64_t *)p; + rxq->rx_using_sse =1 ;return 0 ; }
Signature · 1 line
2.39.2
RE: [PATCH v2 1/4] net/i40e: fix buffer leak on Rx reconfiguration
Quoted 52 lines
-----Original Message-----
From: Richardson, Bruce <[email protected]>
Sent: Thursday, August 31, 2023 8:34 PM
To: [email protected]
Cc: Richardson, Bruce <[email protected]>; Zhang, Qi Z <[email protected]>; [email protected]
Subject: [PATCH v2 1/4] net/i40e: fix buffer leak on Rx reconfigurationWhen reconfiguring a single queue on a device, the mbuf initializer value was not getting set, and remained at zero. This lead to mbuf leaks as the refcount was incorrect (0), so on free it wrapped around to
UINT16_MAX. When setting up the mbuf initializer, also ensure that the queue is explicitly marked as using a vector function by setting the "rx_using_sse" flag.Signed-off-by Bruce Richardson <[email protected]>--- drivers/net/i40e/i40e_rxtx.c | 6 ++++++ drivers/net/i40e/i40e_rxtx_vec_common.h | 1 + 2 files changed, 7 insertions(+)drivers/net/i40e/i40e_rxtx.c index renamed from drivers/net/i40e/i40e_rxtx.c +0
-0
b4f65b58fa..d96bbbb677100644drivers/net/i40e/i40e_rxtx.c +6
-0@@ -1918,6 +1918,12 @@ i40e_dev_rx_queue_setup_runtime(structrte_eth_dev *dev, if (use_def_burst_func) ad->rx_bulk_alloc_allowed = false;i40e_set_rx_function (dev); + +if (ad->rx_vec_allowed &&i40e_rxq_vec_setup (rxq)) { +PMD_DRV_LOG (ERR ,"Failed vector rx setup." ); +return -EINVAL ; + } +return 0 ; } else if (ad->rx_vec_allowed && !rte_is_power_of_2(rxq-Quoted 1 line
nb_rx_desc)) {PMD_DRV_LOG(ERR, "Vector mode is allowed, but
descriptor"drivers/net/i40e/i40e_rxtx_vec_common.h +0
-0
drivers/net/i40e/i40e_rxtx_vec_common.h i40e_rxq_vec_setup_default(struct i40e_rx_queue · line 205
Quoted 5 lines
drivers/net/i40e/i40e_rxtx_vec_common.h +1
-0@@ -201,6 +201,7 @@ i40e_rxq_vec_setup_default(struct i40e_rx_queue *rxq)rte_compiler_barrier (); p = (uintptr_t )&mb_def.rearm_data; rxq->mbuf_initializer = *(uint64_t *)p; + rxq->rx_using_sse =1 ;
I saw some further code clean can be done by leveraging this change.
Quoted 5 lines
drivers/net/i40e/i40e_rxtx_vec_common.h +0
-1return 0 ; } --2.39.2
Applied to dpdk-next-net-intel.
Thanks
Qi
PATCH v2 2/4 net/iavf: fix buffer leak on Tx queue stop
In reply to Bruce RichardsonWhen a queue is stopped all buffers are meant to released from it, and then a new set of buffers reallocated on start. For the iavf code when using vector Tx, some buffers were left in the ring, and so those buffers were leaked. The buffers were missed as the release code only handled one side of a wrap-around case in the ring.
Fix the issue by doing a single iteration of the ring freeing all buffers in it, handling wraparound through a simple post-increment check.
--- drivers/net/iavf/iavf_rxtx_vec_avx512.c | 17 ++++++++--------- drivers/net/iavf/iavf_rxtx_vec_common.h | 11 +++++------ 2 files changed, 13 insertions(+), 15 deletions(-)
drivers/net/iavf/iavf_rxtx_vec_avx512.c +8 -9
@@ -2460,20 +2460,19 @@ iavf_tx_queue_release_mbufs_avx512(struct iavf_tx_queue *txq) {unsigned int i;const uint16_t max_desc = (uint16_t )(txq->nb_tx_desc -1 ); +const uint16_t end_desc = txq->tx_tail >> txq->use_ctx;/* next empty slot */ +const uint16_t wrap_point = txq->nb_tx_desc >> txq->use_ctx;/* end of SW ring */ struct iavf_tx_vec_entry *swr = (void *)txq->sw_ring;if (!txq->sw_ring || txq->nb_free == max_desc)return ; - i = (txq->next_dd >> txq->use_ctx) +1 - - (txq->rs_thresh >> txq->use_ctx); - -if (txq->tx_tail < i) { -for (; i < (unsigned int )(txq->nb_tx_desc >> txq->use_ctx); i++) { -rte_pktmbuf_free_seg (swr[i].mbuf); - swr[i].mbuf =NULL ; - } - i =0 ; + i = (txq->next_dd - txq->rs_thresh +1 ) >> txq->use_ctx; +while (i != end_desc) { +rte_pktmbuf_free_seg (swr[i].mbuf); + swr[i].mbuf =NULL ; +if (++i == wrap_point) + i =0 ; } }
drivers/net/iavf/iavf_rxtx_vec_common.h +5 -6
@@ -186,12 +186,11 @@ _iavf_tx_queue_release_mbufs_vec(struct iavf_tx_queue *txq)return ; i = txq->next_dd - txq->rs_thresh +1 ; -if (txq->tx_tail < i) { -for (; i < txq->nb_tx_desc; i++) { -rte_pktmbuf_free_seg (txq->sw_ring[i].mbuf); - txq->sw_ring[i].mbuf =NULL ; - } - i =0 ; +while (i != txq->tx_tail) { +rte_pktmbuf_free_seg (txq->sw_ring[i].mbuf); + txq->sw_ring[i].mbuf =NULL ; +if (++i == txq->nb_tx_desc) + i =0 ; } }
Signature · 1 line
2.39.2
RE: [PATCH v2 2/4] net/iavf: fix buffer leak on Tx queue stop
Quoted 20 lines
-----Original Message-----
From: Richardson, Bruce <[email protected]>
Sent: Thursday, August 31, 2023 8:34 PM
To: [email protected]
Cc: Richardson, Bruce <[email protected]>; Lu, Wenzhuo <[email protected]>; Wu, Jingjing <[email protected]>; [email protected]
Subject: [PATCH v2 2/4] net/iavf: fix buffer leak on Tx queue stopWhen a queue is stopped all buffers are meant to released from it, and then a new set of buffers reallocated on start. For the iavf code when using vector Tx, some buffers were left in the ring, and so those buffers were leaked. The buffers were missed as the release code only handled one side of a wrap-around case in the ring.
Fix the issue by doing a single iteration of the ring freeing all buffers in it, handling wraparound through a simple post-increment check.
Quoted 7 lines
Fixes 319c421f3890 ("net/avf: enable SSE Rx Tx")Fixes 9ab9514c150e ("net/iavf: enable AVX512 for Tx")Signed-off-by Bruce Richardson <[email protected]>
Zhang, Qi Z
AppliedRE: [PATCH v2 2/4] net/iavf: fix buffer leak on Tx queue stop
Quoted 29 lines
-----Original Message-----
From: Lu, Wenzhuo <[email protected]>
Sent: Monday, September 4, 2023 10:18 AM
To: Richardson, Bruce <[email protected]>; [email protected]
Cc: Wu, Jingjing <[email protected]>; [email protected]
Subject: RE: [PATCH v2 2/4] net/iavf: fix buffer leak on Tx queue stopQuoted 18 lines
-----Original Message-----
From: Richardson, Bruce <[email protected]>
Sent: Thursday, August 31, 2023 8:34 PM
To: [email protected]
Cc: Richardson, Bruce <[email protected]>; Lu, Wenzhuo <[email protected]>; Wu, Jingjing <[email protected]>; [email protected]
Subject: [PATCH v2 2/4] net/iavf: fix buffer leak on Tx queue stopWhen a queue is stopped all buffers are meant to released from it, and then a new set of buffers reallocated on start. For the iavf code when using vector Tx, some buffers were left in the ring, and so those buffers were leaked. The buffers were missed as the release code only handled one side of a wrap-around case in the ring.
Fix the issue by doing a single iteration of the ring freeing all buffers in it, handling wraparound through a simple post-increment check.
Quoted 8 lines
Quoted 7 lines
Fixes 319c421f3890 ("net/avf: enable SSE Rx Tx")Fixes 9ab9514c150e ("net/iavf: enable AVX512 for Tx")Signed-off-by Bruce Richardson <[email protected]>Acked-by Wenzhuo Lu <[email protected]>
Applied to dpdk-next-net-intel.
Thanks
Qi
PATCH v2 3/4 net/iavf: fix restart of Rx queue on reconfigure
In reply to Bruce RichardsonAfter reconfiguring an RX queue the mbuf_initializer value was not being correctly set. Fix this by calling the appropriate function if vector processing is enabled. This mirrors the behaviour by the i40e driver.
--- drivers/net/iavf/iavf_rxtx.c | 7 +++++++ 1 file changed, 7 insertions(+)
drivers/net/iavf/iavf_rxtx.c +7 -0
@@ -755,6 +755,13 @@ iavf_dev_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,if (check_rx_vec_allow (rxq) == false) ad->rx_vec_allowed = false; +#if definedRTE_ARCH_X86 || definedRTE_ARCH_ARM +/* check vector conflict */ +if (ad->rx_vec_allowed &&iavf_rxq_vec_setup (rxq)) { +PMD_DRV_LOG (ERR ,"Failed vector rx setup." ); +return -EINVAL ; + } +#endif return 0 ; }
Signature · 1 line
2.39.2
RE: [PATCH v2 3/4] net/iavf: fix restart of Rx queue on reconfigure
Quoted 29 lines
-----Original Message-----
From: Bruce Richardson <[email protected]>
Sent: Thursday, August 31, 2023 8:34 PM
To: [email protected]
Cc: Richardson, Bruce <[email protected]>; Wu, Jingjing <[email protected]>; [email protected]
Subject: [PATCH v2 3/4] net/iavf: fix restart of Rx queue on reconfigureAfter reconfiguring an RX queue the
mbuf_initializervalue was not being correctly set. Fix this by calling the appropriate function if vector processing is enabled. This mirrors the behaviour by the i40e driver.Signed-off-by Bruce Richardson <[email protected]>--- drivers/net/iavf/iavf_rxtx.c | 7 +++++++ 1 file changed, 7 insertions(+)drivers/net/iavf/iavf_rxtx.c index renamed from drivers/net/iavf/iavf_rxtx.c +0
-0
f7df4665d1..797cdda4b2100644drivers/net/iavf/iavf_rxtx.c +0
-0@@ -755,6 +755,13 @@ iavf_dev_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
drivers/net/iavf/iavf_rxtx.c iavf_dev_rx_queue_setup(struct rte_eth_dev *dev, · line 765
Quoted 10 lines
drivers/net/iavf/iavf_rxtx.c +7
-0if (check_rx_vec_allow (rxq) == false) ad->rx_vec_allowed = false; +#if definedRTE_ARCH_X86 || definedRTE_ARCH_ARM +/* check vector conflict */ +if (ad->rx_vec_allowed &&iavf_rxq_vec_setup (rxq)) { +PMD_DRV_LOG (ERR ,"Failed vector rx setup." ); +return -EINVAL ; + } +#endif
Bruce:
May I know more details about how to reproduce this issue?
As the iavf PMD does not support RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP (i40e does)
Which indicates before we call rte_eth_rx_queue_setup, the device must be stopped (the flag will be checked in rte_eth_rx_queue_setup)
So if we do below steps
rte_eth_dev_stop ..
rte_eth_rx_queue_setup
rte_eth_dev_start
the iavf_rxq_vec_setup should be invoked in rte_eth_dev_start -> iavf_set_rx_function
anything I missed?
Thanks
Qi
Quoted 5 lines
drivers/net/iavf/iavf_rxtx.c +0
-1return 0 ; } --2.39.2
RE: [PATCH v2 3/4] net/iavf: fix restart of Rx queue on reconfigure
Quoted 54 lines
-----Original Message-----
From: Zhang, Qi Z
Sent: Monday, September 4, 2023 9:15 AM
To: Bruce Richardson <[email protected]>; [email protected]
Cc: Richardson, Bruce <[email protected]>; Wu, Jingjing <[email protected]>; [email protected]
Subject: RE: [PATCH v2 3/4] net/iavf: fix restart of Rx queue on reconfigureQuoted 29 lines
-----Original Message-----
From: Bruce Richardson <[email protected]>
Sent: Thursday, August 31, 2023 8:34 PM
To: [email protected]
Cc: Richardson, Bruce <[email protected]>; Wu, Jingjing <[email protected]>; [email protected]
Subject: [PATCH v2 3/4] net/iavf: fix restart of Rx queue on reconfigureAfter reconfiguring an RX queue the
mbuf_initializervalue was not being correctly set. Fix this by calling the appropriate function if vector processing is enabled. This mirrors the behaviour by the i40e driver.Signed-off-by Bruce Richardson <[email protected]>--- drivers/net/iavf/iavf_rxtx.c | 7 +++++++ 1 file changed, 7 insertions(+)(quoted diff) +0
-0
b/drivers/net/iavf/iavf_rxtx.cindexf7df4665d1..797cdda4b2100644drivers/net/iavf/iavf_rxtx.c +0
-0@@ -755,6 +755,13 @@ iavf_dev_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,drivers/net/iavf/iavf_rxtx.c iavf_dev_rx_queue_setup(struct rte_eth_dev *dev, · line 765
Quoted 10 lines
drivers/net/iavf/iavf_rxtx.c +7
-0if (check_rx_vec_allow (rxq) == false) ad->rx_vec_allowed = false; +#if definedRTE_ARCH_X86 || definedRTE_ARCH_ARM +/* check vector conflict */ +if (ad->rx_vec_allowed &&iavf_rxq_vec_setup (rxq)) { +PMD_DRV_LOG (ERR ,"Failed vector rx setup." ); +return -EINVAL ; + } +#endif Bruce:
Quoted 3 lines
May I know more details about how to reproduce this issue? As the iavf PMD does not support
RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP(i40e does)
OK, not sure if the patch 4/4 answered my question 😊
should I squash patch 3, 4 into one? , for my understanding patch 3 doesn't appear to be a bug fix unless we announce RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP.
Quoted 22 lines
Which indicates before we call
rte_eth_rx_queue_setup, the device
must be stopped (the flag will be checked inrte_eth_rx_queue_setup)So if we do below steps rte_eth_dev_stop .. rte_eth_rx_queue_setup rte_eth_dev_start the iavf_rxq_vec_setup should be invoked in rte_eth_dev_start ->
iavf_set_rx_functionanything I missed?
Thanks
QiQuoted 5 lines
drivers/net/iavf/iavf_rxtx.c +0
-1return 0 ; } --2.39.2
Re: [PATCH v2 3/4] net/iavf: fix restart of Rx queue on reconfigure
Quoted 54 lines · Qi Z
Quoted 54 lines
-----Original Message-----
From: Zhang, Qi Z
Sent: Monday, September 4, 2023 9:15 AM
To: Bruce Richardson <[email protected]>; [email protected]
Cc: Richardson, Bruce <[email protected]>; Wu, Jingjing <[email protected]>; [email protected]
Subject: RE: [PATCH v2 3/4] net/iavf: fix restart of Rx queue on reconfigureQuoted 29 lines
-----Original Message-----
From: Bruce Richardson <[email protected]>
Sent: Thursday, August 31, 2023 8:34 PM
To: [email protected]
Cc: Richardson, Bruce <[email protected]>; Wu, Jingjing <[email protected]>; [email protected]
Subject: [PATCH v2 3/4] net/iavf: fix restart of Rx queue on reconfigureAfter reconfiguring an RX queue the
mbuf_initializervalue was not being correctly set. Fix this by calling the appropriate function if vector processing is enabled. This mirrors the behaviour by the i40e driver.Signed-off-by Bruce Richardson <[email protected]>--- drivers/net/iavf/iavf_rxtx.c | 7 +++++++ 1 file changed, 7 insertions(+)(quoted diff) +0
-0
b/drivers/net/iavf/iavf_rxtx.cindexf7df4665d1..797cdda4b2100644drivers/net/iavf/iavf_rxtx.c +0
-0@@ -755,6 +755,13 @@ iavf_dev_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,drivers/net/iavf/iavf_rxtx.c iavf_dev_rx_queue_setup(struct rte_eth_dev *dev, · line 765
Quoted 10 lines
drivers/net/iavf/iavf_rxtx.c +7
-0if (check_rx_vec_allow (rxq) == false) ad->rx_vec_allowed = false; +#if definedRTE_ARCH_X86 || definedRTE_ARCH_ARM +/* check vector conflict */ +if (ad->rx_vec_allowed &&iavf_rxq_vec_setup (rxq)) { +PMD_DRV_LOG (ERR ,"Failed vector rx setup." ); +return -EINVAL ; + } +#endif Bruce:
Quoted 8 lines
Quoted 3 lines
May I know more details about how to reproduce this issue? As the iavf PMD does not support
RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP(i40e does)OK, not sure if the patch 4/4 answered my question 😊
should I squash patch 3, 4 into one? , for my understanding patch 3 doesn't appear to be a bug fix unless we announce
RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP.
You may have a point. I was experimenting with queue reconfiguration which is where I hit this issue.
However, even without queue reconfig support, the device still needs to support queue-stop followed by queue-start, I think, and there may still be an issue there - I'll have to check.
/Bruce
RE: [PATCH v2 3/4] net/iavf: fix restart of Rx queue on reconfigure
Quoted 73 lines
-----Original Message-----
From: Richardson, Bruce <[email protected]>
Sent: Monday, September 4, 2023 3:54 PM
To: Zhang, Qi Z <[email protected]>
Cc: [email protected]; Wu, Jingjing <[email protected]>; [email protected]
Subject: Re: [PATCH v2 3/4] net/iavf: fix restart of Rx queue on reconfigureQuoted 22 lines · Qi Z
Quoted 22 lines
-----Original Message-----
From: Zhang, Qi Z
Sent: Monday, September 4, 2023 9:15 AM
To: Bruce Richardson <[email protected]>; [email protected]
Cc: Richardson, Bruce <[email protected]>; Wu, Jingjing <[email protected]>; [email protected]
Subject: RE: [PATCH v2 3/4] net/iavf: fix restart of Rx queue on reconfigureQuoted 9 lines
-----Original Message-----
From: Bruce Richardson <[email protected]>
Sent: Thursday, August 31, 2023 8:34 PM
To: [email protected]
Cc: Richardson, Bruce <[email protected]>; Wu, Jingjing <[email protected]>; [email protected]
Subject: [PATCH v2 3/4] net/iavf: fix restart of Rx queue on reconfigureQuoted 3 lines
Quoted 3 lines
Quoted 3 lines
After reconfiguring an RX queue the
mbuf_initializervalue was not being correctly set. Fix this by calling the appropriate function if vector processing is enabled. This mirrors the behaviour by the i40edriver.
Quoted 30 lines
Quoted 30 lines
Quoted 17 lines
Signed-off-by Bruce Richardson <[email protected]>--- drivers/net/iavf/iavf_rxtx.c | 7 +++++++ 1 file changed, 7 insertions(+)(quoted diff) +0
-0
b/drivers/net/iavf/iavf_rxtx.cindexf7df4665d1..797cdda4b2100644drivers/net/iavf/iavf_rxtx.c +0
-0@@ -755,6 +755,13 @@ iavf_dev_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,drivers/net/iavf/iavf_rxtx.c iavf_dev_rx_queue_setup(struct rte_eth_dev · line 765
Quoted 10 lines
drivers/net/iavf/iavf_rxtx.c +7
-0if (check_rx_vec_allow (rxq) == false) ad->rx_vec_allowed = false; +#if definedRTE_ARCH_X86 || definedRTE_ARCH_ARM +/* check vector conflict */ +if (ad->rx_vec_allowed &&iavf_rxq_vec_setup (rxq)) { +PMD_DRV_LOG (ERR ,"Failed vector rx setup." ); +return -EINVAL ; + } +#endif Bruce:
Quoted 7 lines
Quoted 3 lines
May I know more details about how to reproduce this issue? As the iavf PMD does not support
RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP(i40e does)OK, not sure if the patch 4/4 answered my question 😊
should I squash patch 3, 4 into one? , for my understanding patch 3 doesn't
Quoted 8 lines
appear to be a bug fix unless we announce
RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP.Quoted 1 line
You may have a point. I was experimenting with queue reconfiguration which is where I hit this issue.
However, even without queue reconfig support, the device still needs to support queue-stop followed by queue-start, I think, and there may still be an issue there - I'll have to check.
Yes, queue start / stop should be supported.
Btw, I didn't see mbuf_initializer has been reset during queue stop / start, it might be a different issue.
Quoted 1 line
/Bruce
PATCH v2 4/4 net/iavf: add support for runtime queue reconfiguration
In reply to Bruce RichardsonUnlike the i40e driver, the iavf driver does not advertise support for runtime reconfiguration, meaning that application using the same hardware may get different behaviour when using a PF vs a VF. On testing with a 40G NIC, the only blocker to reconfiguring an RX queue on the fly is the fact that this support is not advertised by the driver.
Add support for runtime reconfig by reporting it in the device info flags.
--- doc/guides/nics/features/iavf.ini | 2 ++ drivers/net/iavf/iavf_ethdev.c | 4 +++- 2 files changed, 5 insertions(+), 1 deletion(-)
doc/guides/nics/features/iavf.ini +2 -0
@@ -11,6 +11,8 @@ Speed capabilities = Y Link status = Y Rx interrupt = Y Queue start/stop = Y +Runtime Rx queue setup = Y +Runtime Tx queue setup = Y Power mgmt address monitor = Y MTU update = Y Scattered Rx = Y
drivers/net/iavf/iavf_ethdev.c +3 -1
@@ -1127,7 +1127,9 @@ iavf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) dev_info->reta_size = vf->vf_res->rss_lut_size; dev_info->flow_type_rss_offloads =IAVF_RSS_OFFLOAD_ALL ; dev_info->max_mac_addrs =IAVF_NUM_MACADDR_MAX ; - dev_info->dev_capa &= ~RTE_ETH_DEV_CAPA_FLOW_RULE_KEEP ; + dev_info->dev_capa = +RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP | +RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP ; dev_info->rx_offload_capa =RTE_ETH_RX_OFFLOAD_VLAN_STRIP | RTE_ETH_RX_OFFLOAD_QINQ_STRIP |
Signature · 1 line
2.39.2
RE: [PATCH v2 4/4] net/iavf: add support for runtime queue reconfiguration
Quoted 53 lines
-----Original Message-----
From: Bruce Richardson <[email protected]>
Sent: Thursday, August 31, 2023 8:34 PM
To: [email protected]
Cc: Richardson, Bruce <[email protected]>
Subject: [PATCH v2 4/4] net/iavf: add support for runtime queue reconfigurationUnlike the i40e driver, the iavf driver does not advertise support for runtime reconfiguration, meaning that application using the same hardware may get different behaviour when using a PF vs a VF. On testing with a 40G NIC, the only blocker to reconfiguring an RX queue on the fly is the fact that this support is not advertised by the driver.
Add support for runtime reconfig by reporting it in the device info flags.
Signed-off-by Bruce Richardson <[email protected]>--- doc/guides/nics/features/iavf.ini | 2 ++ drivers/net/iavf/iavf_ethdev.c | 4 +++- 2 files changed, 5 insertions(+), 1 deletion(-)doc/guides/nics/features/iavf.ini +2
-0@@ -11,6 +11,8 @@ Speed capabilities = Y Link status = Y Rx interrupt = Y Queue start/stop = Y +Runtime Rx queue setup = Y +Runtime Tx queue setup = Y Power mgmt address monitor = Y MTU update = Y Scattered Rx = Ydrivers/net/iavf/iavf_ethdev.c +3
-1@@ -1127,7 +1127,9 @@ iavf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) dev_info->reta_size = vf->vf_res->rss_lut_size; dev_info->flow_type_rss_offloads =IAVF_RSS_OFFLOAD_ALL ; dev_info->max_mac_addrs =IAVF_NUM_MACADDR_MAX ; - dev_info->dev_capa &= ~RTE_ETH_DEV_CAPA_FLOW_RULE_KEEP ; + dev_info->dev_capa = +RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP | +RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP ; dev_info->rx_offload_capa = RTE_ETH_RX_OFFLOAD_VLAN_STRIP |
Quoted 3 lines
RTE_ETH_RX_OFFLOAD_QINQ_STRIP|
--
2.39.2
Squashed with Patch 3/4, applied to dpdk-next-net-intel.
Thanks
Qi