Direct re-arming of buffers on receive side
Subsystems (2)
cover Direct re-arming of buffers on receive side
Currently, the transmit side frees the buffers into the lcore cache and the receive side allocates buffers from the lcore cache. The transmit side typically frees 32 buffers resulting in 32*8=256B of stores to lcore cache. The receive side allocates 32 buffers and stores them in the receive side software ring, resulting in 32*8=256B of stores and 256B of load from the lcore cache.
This patch proposes a mechanism to avoid freeing to/allocating from the lcore cache. i.e. the receive side will free the buffers from transmit side directly into it's software ring. This will avoid the 256B of loads and stores introduced by the lcore cache. It also frees up the cache lines used by the lcore cache.
However, this solution poses several constraints:
1)The receive queue needs to know which transmit queue it should take the buffers from. The application logic decides which transmit port to use to send out the packets. In many use cases the NIC might have a single port ([1], [2], [3]), in which case a given transmit queue is always mapped to a single receive queue (1:1 Rx queue: Tx queue). This is easy to configure.
If the NIC has 2 ports (there are several references), then we will have 1:2 (RX queue: TX queue) mapping which is still easy to configure. However, if this is generalized to 'N' ports, the configuration can be long. More over the PMD would have to scan a list of transmit queues to pull the buffers from.
2)The other factor that needs to be considered is 'run-to-completion' vs 'pipeline' models. In the run-to-completion model, the receive side and the transmit side are running on the same lcore serially. In the pipeline model. The receive side and transmit side might be running on different lcores in parallel. This requires locking. This is not supported at this point.
3)Tx and Rx buffers must be from the same mempool. And we also must ensure Tx buffer free number is equal to Rx buffer free number:
(txq->tx_rs_thresh == RTE_I40E_RXQ_REARM_THRESH)
Thus, 'tx_next_dd' can be updated correctly in direct-rearm mode. This is due to tx_next_dd is a variable to compute tx sw-ring free location. Its value will be one more round than the position where next time free starts.
Current status in this RFC:
1)An API is added to allow for mapping a TX queue to a RX queue.
Currently it supports 1:1 mapping.
2)The i40e driver is changed to do the direct re-arm of the receive
side.
3)L3fwd application is modified to do the direct rearm mapping automatically without user config. This follows the rules that the thread can map TX queue to a RX queue based on the first received package destination port.
Testing status:
1.The testing results for L3fwd are as follows:
-------------------------------------------------------------------
enabled direct rearm
-------------------------------------------------------------------
Arm:
N1SDP(neon path):
without fast-free mode with fast-free mode
+14.1% +7.0%
Ampere Altra(neon path):
without fast-free mode with fast-free mode
+17.1 +14.0%
X86:
Dell-8268(limit frequency):
sse path:
without fast-free mode with fast-free mode
+6.96% +2.02% avx2 path: without fast-free mode with fast-free mode +9.04% +7.75% avx512 path: without fast-free mode with fast-free mode +5.43% +1.57%
-------------------------------------------------------------------
This patch can not affect base performance of normal mode. Furthermore, the reason for that limiting the CPU frequency is that dell-8268 can encounter i40e NIC bottleneck with maximum frequency.
2.The testing results for VPP-L3fwd are as follows:
-------------------------------------------------------------------
Arm:
N1SDP(neon path):
with direct re-arm mode enabled
+7.0%
-------------------------------------------------------------------
For Ampere Altra and X86,VPP-L3fwd test has not been done.
Reference:
[1] https://store.nvidia.com/en-us/networking/store/product/MCX623105AN-CDAT/NVIDIAMCX623105ANCDATConnectX6DxENAdapterCard100GbECryptoDisabled/
[2] https://www.intel.com/content/www/us/en/products/sku/192561/intel-ethernet-network-adapter-e810cqda1/specifications.html
[3] https://www.broadcom.com/products/ethernet-connectivity/network-adapters/100gb-nic-ocp/n1100g
Feifei Wang (5):
net/i40e: remove redundant Dtype initialization
net/i40e: enable direct rearm mode
ethdev: add API for direct rearm mode
net/i40e: add direct rearm mode internal API
examples/l3fwd: enable direct rearm mode
drivers/net/i40e/i40e_ethdev.c | 34 +++ drivers/net/i40e/i40e_rxtx.c | 4 - drivers/net/i40e/i40e_rxtx.h | 4 + drivers/net/i40e/i40e_rxtx_common_avx.h | 269 ++++++++++++++++++++++++ drivers/net/i40e/i40e_rxtx_vec_avx2.c | 14 +- drivers/net/i40e/i40e_rxtx_vec_avx512.c | 249 +++++++++++++++++++++- drivers/net/i40e/i40e_rxtx_vec_neon.c | 141 ++++++++++++- drivers/net/i40e/i40e_rxtx_vec_sse.c | 170 ++++++++++++++- examples/l3fwd/l3fwd_lpm.c | 16 +- lib/ethdev/ethdev_driver.h | 15 ++ lib/ethdev/rte_ethdev.c | 14 ++ lib/ethdev/rte_ethdev.h | 31 +++ lib/ethdev/version.map | 1 + 13 files changed, 949 insertions(+), 13 deletions(-)
Signature · 1 line
2.25.1
PATCH 1/5 net/i40e: remove redundant Dtype initialization
The Dtype field is set to 0xf by the NIC to indicate DMA completion, only after the CPU requests to be informed by setting the RS bit. Hence, it is not required to set Dtype to 0xf during initialization.
Not setting the Dtype field to 0xf helps to know that a given descriptor is not sent to the NIC yet after initialization.
--- drivers/net/i40e/i40e_rxtx.c | 4 ---- 1 file changed, 4 deletions(-)
drivers/net/i40e/i40e_rxtx.c +0 -4
@@ -2767,10 +2767,6 @@ i40e_reset_tx_queue(struct i40e_tx_queue *txq) prev = (uint16_t )(txq->nb_tx_desc -1 );for (i =0 ; i < txq->nb_tx_desc; i++) { -volatile struct i40e_tx_desc *txd = &txq->tx_ring[i]; - - txd->cmd_type_offset_bsz = -rte_cpu_to_le_64 (I40E_TX_DESC_DTYPE_DESC_DONE ); txe[i].mbuf =NULL ; txe[i].last_id = i; txe[prev].next_id = i;
Signature · 1 line
2.25.1
PATCH 2/5 net/i40e: enable direct rearm mode
In reply to Feifei WangFor i40e driver, enable direct re-arm mode. This patch supports the case of mapping Rx/Tx queues from the same single lcore.
--- drivers/net/i40e/i40e_rxtx.h | 4 + drivers/net/i40e/i40e_rxtx_common_avx.h | 269 ++++++++++++++++++++++++ drivers/net/i40e/i40e_rxtx_vec_avx2.c | 14 +- drivers/net/i40e/i40e_rxtx_vec_avx512.c | 249 +++++++++++++++++++++- drivers/net/i40e/i40e_rxtx_vec_neon.c | 141 ++++++++++++- drivers/net/i40e/i40e_rxtx_vec_sse.c | 170 ++++++++++++++- 6 files changed, 839 insertions(+), 8 deletions(-)
drivers/net/i40e/i40e_rxtx.h +4 -0
@@ -102,6 +102,8 @@ struct i40e_rx_queue {uint16_t rxrearm_nb;/**< number of remaining to be re-armed */ uint16_t rxrearm_start;/**< the idx we start the re-arming from */ +uint16_t direct_rxrearm_port;/** device TX port ID for direct re-arm mode */ +uint16_t direct_rxrearm_queue;/** TX queue index for direct re-arm mode */ uint64_t mbuf_initializer;/**< value to init mbufs */ uint16_t port_id;/**< device port ID */
@@ -121,6 +123,8 @@ struct i40e_rx_queue {uint16_t rx_using_sse;/**<flag indicate the usage of vPMD for rx */ uint8_t dcb_tc;/**< Traffic class of rx queue */ uint64_t offloads;/**< Rx offload flags of RTE_ETH_RX_OFFLOAD_* */ +/**< 0 if direct re-arm mode disabled, 1 when enabled */ +bool direct_rxrearm_enable;const struct rte_memzone *mz; };
drivers/net/i40e/i40e_rxtx_common_avx.h +269 -0
275 lines not on this page · raw message
drivers/net/i40e/i40e_rxtx_vec_avx2.c +12 -2
@@ -25,6 +25,12 @@ i40e_rxq_rearm(struct i40e_rx_queue *rxq)return i40e_rxq_rearm_common (rxq, false); } +static __rte_always_inline void +i40e_rxq_direct_rearm (struct i40e_rx_queue *rxq) +{ +return i40e_rxq_direct_rearm_common (rxq, false); +} +#ifndef RTE_LIBRTE_I40E_16BYTE_RX_DESC /* Handles 32B descriptor FDIR ID processing: * rxdp: receive descriptor ring, required to load 2nd 16B half of each desc
@@ -128,8 +134,12 @@ _recv_raw_pkts_vec_avx2(struct i40e_rx_queue *rxq, struct rte_mbuf **rx_pkts,/* See if we need to rearm the RX queue - gives the prefetch a bit * of time to act */ -if (rxq->rxrearm_nb >RTE_I40E_RXQ_REARM_THRESH ) -i40e_rxq_rearm (rxq); +if (rxq->rxrearm_nb >RTE_I40E_RXQ_REARM_THRESH ) { +if (rxq->direct_rxrearm_enable) +i40e_rxq_direct_rearm (rxq); +else +i40e_rxq_rearm (rxq); + }/* Before we start moving massive data around, check to see if * there is actually a packet available
drivers/net/i40e/i40e_rxtx_vec_avx512.c +247 -2
267 lines not on this page · raw message
drivers/net/i40e/i40e_rxtx_vec_neon.c +139 -2
153 lines not on this page · raw message
drivers/net/i40e/i40e_rxtx_vec_sse.c +168 -2
182 lines not on this page · raw message
Signature · 1 line
2.25.1
Re: [PATCH v1 2/5] net/i40e: enable direct rearm mode
Quoted 73 lines
For i40e driver, enable direct re-arm mode. This patch supports the case of mapping Rx/Tx queues from the same single lcore.
Suggested-by Honnappa Nagarahalli <[email protected]>Signed-off-by Feifei Wang <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>Reviewed-by Honnappa Nagarahalli <[email protected]>--- drivers/net/i40e/i40e_rxtx.h | 4 + drivers/net/i40e/i40e_rxtx_common_avx.h | 269 ++++++++++++++++++++++++ drivers/net/i40e/i40e_rxtx_vec_avx2.c | 14 +- drivers/net/i40e/i40e_rxtx_vec_avx512.c | 249 +++++++++++++++++++++- drivers/net/i40e/i40e_rxtx_vec_neon.c | 141 ++++++++++++- drivers/net/i40e/i40e_rxtx_vec_sse.c | 170 ++++++++++++++- 6 files changed, 839 insertions(+), 8 deletions(-)drivers/net/i40e/i40e_rxtx.h +4
-0@@ -102,6 +102,8 @@ struct i40e_rx_queue {uint16_t rxrearm_nb;/**< number of remaining to be re-armed */ uint16_t rxrearm_start;/**< the idx we start the re-arming from */ +uint16_t direct_rxrearm_port;/** device TX port ID for direct re-arm mode */ +uint16_t direct_rxrearm_queue;/** TX queue index for direct re-arm mode */ uint64_t mbuf_initializer;/**< value to init mbufs */ uint16_t port_id;/**< device port ID */ @@ -121,6 +123,8 @@ struct i40e_rx_queue {uint16_t rx_using_sse;/**<flag indicate the usage of vPMD for rx */ uint8_t dcb_tc;/**< Traffic class of rx queue */ uint64_t offloads;/**< Rx offload flags of RTE_ETH_RX_OFFLOAD_* */ +/**< 0 if direct re-arm mode disabled, 1 when enabled */ +bool direct_rxrearm_enable;const struct rte_memzone *mz; };drivers/net/i40e/i40e_rxtx_common_avx.h +26
-0@@ -209,6 +209,275 @@ i40e_rxq_rearm_common(struct i40e_rx_queue *rxq, __rte_unused bool avx512)/* Update the tail pointer on the NIC */ I40E_PCI_REG_WC_WRITE (rxq->qrx_tail, rx_id); } + +static __rte_always_inline void +i40e_rxq_direct_rearm_common(struct i40e_rx_queue *rxq, __rte_unused bool avx512) +{ +struct rte_eth_dev *dev; + struct i40e_tx_queue *txq;rivers/net/i40e/i40e_rxtx_common_avx.h +volatile union i40e_rx_desc *rxdp; +struct i40e_tx_entry *txep; +struct i40e_rx_entry *rxep; +struct rte_mbuf *m[RTE_I40E_RXQ_REARM_THRESH ]; +uint16_t tx_port_id, tx_queue_id; +uint16_t rx_id; +uint16_t i, n; +uint16_t nb_rearm =0 ; + + rxdp = rxq->rx_ring + rxq->rxrearm_start; + rxep = &rxq->sw_ring[rxq->rxrearm_start]; + + tx_port_id = rxq->direct_rxrearm_port; + tx_queue_id = rxq->direct_rxrearm_queue; + dev = &rte_eth_devices[tx_port_id]; + txq = dev->data->tx_queues[tx_queue_id]; + +/* check Rx queue is able to take in the whole +* batch of free mbufs from Tx queue +*/
drivers/net/i40e/i40e_rxtx_common_avx.h i40e_rxq_rearm_common(struct i40e_rx_queue *rxq, __rte_unused bool avx512) · line 247
Quoted 10 lines
drivers/net/i40e/i40e_rxtx_common_avx.h +10
-0+if (rxq->rxrearm_nb > txq->tx_rs_thresh) { +/* check DD bits on threshold descriptor */ +if ((txq->tx_ring[txq->tx_next_dd].cmd_type_offset_bsz & +rte_cpu_to_le_64 (I40E_TXD_QW1_DTYPE_MASK )) != +rte_cpu_to_le_64 (I40E_TX_DESC_DTYPE_DESC_DONE )) { + goto mempool_bulk; + } + +if (txq->tx_rs_thresh !=RTE_I40E_RXQ_REARM_THRESH ) + goto mempool_bulk;
I think all these checks (is this mode can be enabled) should be done at config phase, not at data-path.
drivers/net/i40e/i40e_rxtx_common_avx.h i40e_rxq_rearm_common(struct i40e_rx_queue *rxq, __rte_unused bool avx512) · line 254
Quoted 7 lines
drivers/net/i40e/i40e_rxtx_common_avx.h +7
-0+ + n = txq->tx_rs_thresh; + +/* first buffer to free from S/W ring is at index +* tx_next_dd - (tx_rs_thresh-1) +*/ + txep = &txq->sw_ring[txq->tx_next_dd - (n -1 )];
It really looks bad that RX function acesses and modifies TXQ data directly. Would be much better to hide TXD checking/manipulation into a separate TXQ function (txq_mbuf() or so) that RX path can invoke.
Quoted 10 lines
drivers/net/i40e/i40e_rxtx_common_avx.h +10
-0+ +if (txq->offloads &RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE ) { +/* directly put mbufs from Tx to Rx, +* and initialize the mbufs in vector +*/ +for (i =0 ; i < n; i++) + rxep[i].mbuf = txep[i].mbuf; + }else { +for (i =0 ; i < n; i++) { + m[i] =rte_pktmbuf_prefree_seg (txep[i].mbuf);
drivers/net/i40e/i40e_rxtx_common_avx.h i40e_rxq_rearm_common(struct i40e_rx_queue *rxq, __rte_unused bool avx512) · line 274
Quoted 10 lines
drivers/net/i40e/i40e_rxtx_common_avx.h +10
-0+/* ensure each Tx freed buffer is valid */ +if (m[i] !=NULL ) + nb_rearm++; + } + +if (nb_rearm != n) { + txq->nb_tx_free = (uint16_t )(txq->nb_tx_free + txq->tx_rs_thresh); + txq->tx_next_dd = (uint16_t )(txq->tx_next_dd + txq->tx_rs_thresh); +if (txq->tx_next_dd >= txq->nb_tx_desc) + txq->tx_next_dd = (uint16_t )(txq->tx_rs_thresh -1 );
So if nb_rearm != 0 what would happen with mbufs collected in m[]? Are you just dropping/forgetting them?
Quoted 5 lines
drivers/net/i40e/i40e_rxtx_common_avx.h +4
-0+ + goto mempool_bulk; + }else { +for (i =0 ; i < n; i++)
drivers/net/i40e/i40e_rxtx_common_avx.h i40e_rxq_rearm_common(struct i40e_rx_queue *rxq, __rte_unused bool avx512) · line 288
Quoted 10 lines
drivers/net/i40e/i40e_rxtx_common_avx.h +10
-0+ rxep[i].mbuf = m[i]; + } + } + +/* update counters for Tx */ + txq->nb_tx_free = (uint16_t )(txq->nb_tx_free + txq->tx_rs_thresh); + txq->tx_next_dd = (uint16_t )(txq->tx_next_dd + txq->tx_rs_thresh); +if (txq->tx_next_dd >= txq->nb_tx_desc) + txq->tx_next_dd = (uint16_t )(txq->tx_rs_thresh -1 ); + }else {
853 lines not shown · raw message
Part of this message is not shown here. View the raw message.
PATCH 3/5 ethdev: add API for direct rearm mode
In reply to Feifei WangAdd API for enabling direct rearm mode and for mapping RX and TX queues. Currently, the API supports 1:1(txq : rxq) mapping.
--- lib/ethdev/ethdev_driver.h | 15 +++++++++++++++ lib/ethdev/rte_ethdev.c | 14 ++++++++++++++ lib/ethdev/rte_ethdev.h | 31 +++++++++++++++++++++++++++++++ lib/ethdev/version.map | 1 + 4 files changed, 61 insertions(+)
lib/ethdev/ethdev_driver.h +15 -0
@@ -485,6 +485,16 @@ typedef int (*eth_rx_enable_intr_t)(struct rte_eth_dev *dev,typedef int (*eth_rx_disable_intr_t )(struct rte_eth_dev *dev,uint16_t rx_queue_id); +/** @internal Enable direct rearm of a receive queue of an Ethernet device. */ +typedef int (*eth_rx_direct_rearm_enable_t )(struct rte_eth_dev *dev, +uint16_t queue_id); + +/**< @internal map Rx/Tx queue of direct rearm mode */ +typedef int (*eth_rx_direct_rearm_map_t )(struct rte_eth_dev *dev, +uint16_t rx_queue_id, +uint16_t tx_port_id, +uint16_t tx_queue_id); +/** @internal Release memory resources allocated by given Rx/Tx queue. */ typedef void (*eth_queue_release_t )(struct rte_eth_dev *dev,uint16_t queue_id);
@@ -1152,6 +1162,11 @@ struct eth_dev_ops {/** Disable Rx queue interrupt */ eth_rx_disable_intr_t rx_queue_intr_disable; +/** Enable Rx queue direct rearm mode */ +eth_rx_direct_rearm_enable_t rx_queue_direct_rearm_enable; +/** Map Rx/Tx queue for direct rearm mode */ +eth_rx_direct_rearm_map_t rx_queue_direct_rearm_map; +eth_tx_queue_setup_t tx_queue_setup;/**< Set up device Tx queue */ eth_queue_release_t tx_queue_release;/**< Release Tx queue */ eth_tx_done_cleanup_t tx_done_cleanup;/**< Free Tx ring mbufs */
lib/ethdev/rte_ethdev.c +14 -0
@@ -2139,6 +2139,20 @@ rte_eth_tx_hairpin_queue_setup(uint16_t port_id, uint16_t tx_queue_id,return eth_err (port_id, ret); } +int +rte_eth_direct_rxrearm_map (uint16_t rx_port_id,uint16_t rx_queue_id, +uint16_t tx_port_id,uint16_t tx_queue_id) +{ +struct rte_eth_dev *dev; + + dev = &rte_eth_devices[rx_port_id]; + (*dev->dev_ops->rx_queue_direct_rearm_enable)(dev, rx_queue_id); + (*dev->dev_ops->rx_queue_direct_rearm_map)(dev, rx_queue_id, + tx_port_id, tx_queue_id); + +return 0 ; +} +int rte_eth_hairpin_bind (uint16_t tx_port,uint16_t rx_port) {
lib/ethdev/rte_ethdev.h +31 -0
@@ -5190,6 +5190,37 @@ __rte_experimentalint rte_eth_dev_hairpin_capability_get (uint16_t port_id,struct rte_eth_hairpin_cap *cap); +/** +* @warning +* @b EXPERIMENTAL: this API may change, or be removed, without prior notice +* +* Enable direct re-arm mode. In this mode the RX queue will be re-armed using +* buffers that have completed transmission on the transmit side. +* +* @note +* It is assumed that the buffers have completed transmission belong to the +* mempool used at the receive side, and have refcnt = 1. +* +* @param rx_port_id +* Port identifying the receive side. +* @param rx_queue_id +* The index of the receive queue identifying the receive side. +* The value must be in the range [0, nb_rx_queue - 1] previously supplied +* to rte_eth_dev_configure(). +* @param tx_port_id +* Port identifying the transmit side. +* @param tx_queue_id +* The index of the transmit queue identifying the transmit side. +* The value must be in the range [0, nb_tx_queue - 1] previously supplied +* to rte_eth_dev_configure(). +* +* @return +* - (0) if successful. +*/ +__rte_experimental +int rte_eth_direct_rxrearm_map(uint16_t rx_port_id, uint16_t rx_queue_id, +uint16_t tx_port_id,uint16_t tx_queue_id); +/** * @warning * @b EXPERIMENTAL: this structure may change without prior notice.
lib/ethdev/version.map +1 -0
@@ -279,6 +279,7 @@ EXPERIMENTAL { rte_flow_async_action_handle_create; rte_flow_async_action_handle_destroy; rte_flow_async_action_handle_update; + rte_eth_direct_rxrearm_map; }; INTERNAL {
Signature · 1 line
2.25.1
RE: [PATCH v1 3/5] ethdev: add API for direct rearm mode
Quoted 42 lines
From: Feifei Wang [mailto:[email protected]]
Sent: Wednesday, 20 April 2022 10.17Add API for enabling direct rearm mode and for mapping RX and TX queues. Currently, the API supports 1:1(txq : rxq) mapping.
Suggested-by Honnappa Nagarahalli <[email protected]>Signed-off-by Feifei Wang <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>Reviewed-by Honnappa Nagarahalli <[email protected]>--- lib/ethdev/ethdev_driver.h | 15 +++++++++++++++ lib/ethdev/rte_ethdev.c | 14 ++++++++++++++ lib/ethdev/rte_ethdev.h | 31 +++++++++++++++++++++++++++++++ lib/ethdev/version.map | 1 + 4 files changed, 61 insertions(+)lib/ethdev/ethdev_driver.h +1
-0@@ -485,6 +485,16 @@ typedef int (*eth_rx_enable_intr_t)(structrte_eth_dev *dev, typedef int (*eth_rx_disable_intr_t)(struct rte_eth_dev *dev,uint16_t rx_queue_id); +/** @internal Enable direct rearm of a receive queue of an Ethernet device. */
lib/ethdev/ethdev_driver.h +9
-0+typedef int (*eth_rx_direct_rearm_enable_t )(struct rte_eth_dev *dev, +uint16_t queue_id); + +/**< @internal map Rx/Tx queue of direct rearm mode */ +typedef int (*eth_rx_direct_rearm_map_t )(struct rte_eth_dev *dev, +uint16_t rx_queue_id, +uint16_t tx_port_id, +uint16_t tx_queue_id); +/** @internal Release memory resources allocated by given Rx/Tx queue. */
typedef void (*eth_queue_release_t)(structrte_eth_dev*dev,lib/ethdev/ethdev_driver.h +0
-0uint16_t queue_id);
lib/ethdev/ethdev_driver.h struct eth_dev_ops { · line 1166
Quoted 5 lines
lib/ethdev/ethdev_driver.h +2
-0@@ -1152,6 +1162,11 @@ struct eth_dev_ops {/** Disable Rx queue interrupt */ eth_rx_disable_intr_t rx_queue_intr_disable; +/** Enable Rx queue direct rearm mode */ +eth_rx_direct_rearm_enable_t rx_queue_direct_rearm_enable;
A disable function seems to be missing.
Quoted 18 lines
lib/ethdev/ethdev_driver.h +3
-0+/** Map Rx/Tx queue for direct rearm mode */ +eth_rx_direct_rearm_map_t rx_queue_direct_rearm_map; +eth_tx_queue_setup_t tx_queue_setup;/**< Set up device Tx queue */
eth_queue_release_ttx_queue_release; /**< Release Tx
queue */
eth_tx_done_cleanup_ttx_done_cleanup;/**< Free Tx ring
mbufs */lib/ethdev/rte_ethdev.c +0
-0@@ -2139,6 +2139,20 @@ rte_eth_tx_hairpin_queue_setup(uint16_t port_id,uint16_t tx_queue_id, return eth_err(port_id, ret); }
lib/ethdev/rte_ethdev.c rte_eth_tx_hairpin_queue_setup(uint16_t port_id, · line 2152
Quoted 10 lines
lib/ethdev/rte_ethdev.c +10
-0+int +rte_eth_direct_rxrearm_map (uint16_t rx_port_id,uint16_t rx_queue_id, +uint16_t tx_port_id,uint16_t tx_queue_id) +{ +struct rte_eth_dev *dev; + + dev = &rte_eth_devices[rx_port_id]; + (*dev->dev_ops->rx_queue_direct_rearm_enable)(dev, rx_queue_id); + (*dev->dev_ops->rx_queue_direct_rearm_map)(dev, rx_queue_id, + tx_port_id, tx_queue_id);
Here you enable the mapping before you configure it. It could cause the driver to use an uninitialized map, if it processes packets between these two function calls.
Error handling is missing. Not all drivers support this feature, and the parameters should be validated.
Regarding driver support, the driver should also expose a capability flag to the application, similar to the RTE_ETH_DEV_CAPA_RXQ_SHARE or RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE flags. The documentation for this flag could include the description of all the restrictions to using it.
Quoted 50 lines
lib/ethdev/rte_ethdev.c +4
-0+ +return 0 ; +} +int rte_eth_hairpin_bind (uint16_t tx_port,uint16_t rx_port){
lib/ethdev/rte_ethdev.h +3
-0@@ -5190,6 +5190,37 @@ __rte_experimentalint rte_eth_dev_hairpin_capability_get (uint16_t port_id,struct rte_eth_hairpin_cap *cap); +/** +* @warning +* @b EXPERIMENTAL: this API may change, or be removed, without prior notice
+ *
+ * Enable direct re-arm mode. In this mode the RX queue will be re- armed using
+ * buffers that have completed transmission on the transmit side.
+ *
+ * @note
+ * It is assumed that the buffers have completed transmission belong
to the
+ * mempool used at the receive side, and have refcnt = 1.
+ *
+ * @paramrx_port_id
+ * Port identifying the receive side.
+ * @paramrx_queue_id
+ * The index of the receive queue identifying the receive side.
+ * The value must be in the range [0,nb_rx_queue- 1] previously
supplied
+ * torte_eth_dev_configure().
+ * @paramtx_port_id
+ * Port identifying the transmit side.
+ * @paramtx_queue_id
+ * The index of the transmit queue identifying the transmit side.
+ * The value must be in the range [0,nb_tx_queue- 1] previously
supplied
+ * torte_eth_dev_configure().
+ *
+ * @return
+ * - (0) if successful.
+ */lib/ethdev/rte_ethdev.h +2
-0+__rte_experimental +int rte_eth_direct_rxrearm_map(uint16_t rx_port_id, uint16_t
rx_queue_id,
lib/ethdev/rte_ethdev.h __rte_experimental · line 5223
Quoted 2 lines
lib/ethdev/rte_ethdev.h +2
-0+ uint16_t tx_port_id, uint16_t tx_queue_id); +
I agree with the parameters to your proposed API here. Since the relevant use case only needs 1:1 mapping, exposing an API function to take some sort of array with N:M mappings would be premature, and probably not ever come into play anyway.
How do you remove, disable and/or change a mapping?
Quoted 18 lines
/**
* @warning
* @b EXPERIMENTAL: this structure may change without prior notice.lib/ethdev/version.map +1
-1@@ -279,6 +279,7 @@ EXPERIMENTAL { rte_flow_async_action_handle_create; rte_flow_async_action_handle_destroy; rte_flow_async_action_handle_update; + rte_eth_direct_rxrearm_map; }; INTERNAL { --2.25.1
回复: [PATCH v1 3/5] ethdev: add API for direct rearm mode
Quoted 57 lines
-----邮件原件-----
发件人: Morten Brørup <[email protected]>
发送时间: Wednesday, April 20, 2022 5:59 PM
收件人: Feifei Wang <[email protected]>; [email protected]; Ferruh Yigit <[email protected]>; Andrew Rybchenko <[email protected]>; Ray Kinsella <[email protected]> 抄送: [email protected]; nd <[email protected]>; Honnappa Nagarahalli <[email protected]>; Ruifeng Wang <[email protected]> 主题: RE: [PATCH v1 3/5] ethdev: add API for direct rearm modeQuoted 45 lines
From: Feifei Wang [mailto:[email protected]]
Sent: Wednesday, 20 April 2022 10.17Add API for enabling direct rearm mode and for mapping RX and TX queues. Currently, the API supports 1:1(txq : rxq) mapping.
Suggested-by Honnappa Nagarahalli <[email protected]>Signed-off-by Feifei Wang <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>Reviewed-by Honnappa Nagarahalli <[email protected]>--- lib/ethdev/ethdev_driver.h | 15 +++++++++++++++ lib/ethdev/rte_ethdev.c | 14 ++++++++++++++ lib/ethdev/rte_ethdev.h | 31 +++++++++++++++++++++++++++++++ lib/ethdev/version.map | 1 + 4 files changed, 61 insertions(+)lib/ethdev/ethdev_driver.h +0
-0@@ -485,6 +485,16 @@ typedef int (*eth_rx_enable_intr_t)(struct
rte_eth_dev*dev, typedef int (*eth_rx_disable_intr_t)(structrte_eth_dev*dev,lib/ethdev/ethdev_driver.h +1
-0uint16_t rx_queue_id); +/** @internal Enable direct rearm of a receive queue of an Ethernet device. */
lib/ethdev/ethdev_driver.h +9
-0+typedef int (*eth_rx_direct_rearm_enable_t )(struct rte_eth_dev *dev, +uint16_t queue_id); + +/**< @internal map Rx/Tx queue of direct rearm mode */ typedef int +(*eth_rx_direct_rearm_map_t )(struct rte_eth_dev *dev, +uint16_t rx_queue_id, +uint16_t tx_port_id, +uint16_t tx_queue_id); +/** @internal Release memory resources allocated by given Rx/Tx queue. */
typedef void (*eth_queue_release_t)(structrte_eth_dev*dev,lib/ethdev/ethdev_driver.h +0
-0uint16_t queue_id);@@ -1152,6 +1162,11 @@ struct eth_dev_ops {/** Disable Rx queue interrupt */ eth_rx_disable_intr_t rx_queue_intr_disable;
Quoted 4 lines
lib/ethdev/ethdev_driver.h struct eth_dev_ops { · line 1166
Quoted 2 lines
lib/ethdev/ethdev_driver.h +2
-0+/** Enable Rx queue direct rearm mode */ +eth_rx_direct_rearm_enable_t rx_queue_direct_rearm_enable;A disable function seems to be missing.
[Feifei] I will try to use offload bits to enable direct-rearm mode, thus this enable function will be removed and disable function will be unnecessary.
Quoted 27 lines
Quoted 16 lines
lib/ethdev/ethdev_driver.h +3
-0+/** Map Rx/Tx queue for direct rearm mode */ +eth_rx_direct_rearm_map_t rx_queue_direct_rearm_map; +eth_tx_queue_setup_t tx_queue_setup;/**< Set up device Tx queue */
eth_queue_release_ttx_queue_release; /**< Release Tx
queue */
eth_tx_done_cleanup_ttx_done_cleanup;/**< Free Tx ring
mbufs */lib/ethdev/rte_ethdev.c index renamed from lib/ethdev/rte_ethdev.c +0
-0
29a3d80466..8e6f0284f4100644lib/ethdev/rte_ethdev.c +0
-0@@ -2139,6 +2139,20 @@ rte_eth_tx_hairpin_queue_setup(uint16_t port_id, uint16_t tx_queue_id,lib/ethdev/rte_ethdev.c rte_eth_tx_hairpin_queue_setup(uint16_t · line 2149
Quoted 10 lines
lib/ethdev/rte_ethdev.c +7
-0return eth_err (port_id, ret); } +int +rte_eth_direct_rxrearm_map (uint16_t rx_port_id,uint16_t rx_queue_id, +uint16_t tx_port_id,uint16_t tx_queue_id) { +struct rte_eth_dev *dev; + + dev = &rte_eth_devices[rx_port_id]; + (*dev->dev_ops->rx_queue_direct_rearm_enable)(dev,
rx_queue_id);
Quoted 6 lines
lib/ethdev/rte_ethdev.c rte_eth_tx_hairpin_queue_setup(uint16_t port_id, uint16_t tx_queue_id, · line 2151
Quoted 2 lines
lib/ethdev/rte_ethdev.c +2
-0+ (*dev->dev_ops->rx_queue_direct_rearm_map)(dev, rx_queue_id, + tx_port_id, tx_queue_id);Here you enable the mapping before you configure it. It could cause the driver to use an uninitialized map, if it processes packets between these two function calls.
[Feifei] I agree with this and will change the code.
Quoted 3 lines
Error handling is missing. Not all drivers support this feature, and the parameters should be validated.
[Feifei] You are right, I think after we use 'rxq->offload' bits, we can use some 'offload bits API' to check if driver can support this.
For the parameters, I will add some check.
Quoted 5 lines
Regarding driver support, the driver should also expose a capability flag to the application, similar to the
RTE_ETH_DEV_CAPA_RXQ_SHAREorRTE_ETH_TX_OFFLOAD_MBUF_FAST_FREEflags. The documentation for this flag could include the description of all the restrictions to using it.
[Feifei] I will do like this by 'rxq->offload' bits, and add description to the documentation.
Quoted 58 lines
Quoted 50 lines
lib/ethdev/rte_ethdev.c +4
-1+ +return 0 ; +} +int rte_eth_hairpin_bind (uint16_t tx_port,uint16_t rx_port) { diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h index
04cff8ee10..4a431fcbed100644lib/ethdev/rte_ethdev.h +3
-0@@ -5190,6 +5190,37 @@ __rte_experimental int rte_eth_dev_hairpin_capability_get(uint16_t port_id, struct rte_eth_hairpin_cap *cap); +/** +* @warning +* @b EXPERIMENTAL: this API may change, or be removed, without prior notice
+ *
+ * Enable direct re-arm mode. In this mode the RX queue will be re- armed using
+ * buffers that have completed transmission on the transmit side.
+ *
+ * @note
+ * It is assumed that the buffers have completed transmission belong
to the
+ * mempool used at the receive side, and have refcnt = 1.
+ *
+ * @paramrx_port_id
+ * Port identifying the receive side.
+ * @paramrx_queue_id
+ * The index of the receive queue identifying the receive side.
+ * The value must be in the range [0,nb_rx_queue- 1] previously
supplied
+ * torte_eth_dev_configure().
+ * @paramtx_port_id
+ * Port identifying the transmit side.
+ * @paramtx_queue_id
+ * The index of the transmit queue identifying the transmit side.
+ * The value must be in the range [0,nb_tx_queue- 1] previously
supplied
+ * torte_eth_dev_configure().
+ *
+ * @return
+ * - (0) if successful.
+ */lib/ethdev/rte_ethdev.h +2
-0+__rte_experimental +int rte_eth_direct_rxrearm_map(uint16_t rx_port_id, uint16_t
rx_queue_id,lib/ethdev/rte_ethdev.h __rte_experimental · line 5223
Quoted 2 lines
lib/ethdev/rte_ethdev.h +2
-0+ uint16_t tx_port_id, uint16_t tx_queue_id); +I agree with the parameters to your proposed API here. Since the relevant use case only needs 1:1 mapping, exposing an API function to take some sort of array with N:M mappings would be premature, and probably not ever come into play anyway.
Quoted 1 line
How do you remove, disable and/or change a mapping?
[Feifei] It is not recommended that users change the map in the process of sending and receiving packets, which may bring some error risks. If user want to change mapping, he needs to stop the device and call
' rte_eth_direct_rxrearm_map ' API to rewrite the mapping.
Furthermore, for 'rxq->offload', user needs to set it before dev starts. If user want to change it, dev needs to be restarted.
Quoted 19 lines
Quoted 19 lines
/**
* @warning
* @b EXPERIMENTAL: this structure may change without prior notice.lib/ethdev/version.map index renamed from lib/ethdev/version.map +0
-0
20391ab29e..68d664498c100644lib/ethdev/version.map +1
-1@@ -279,6 +279,7 @@ EXPERIMENTAL { rte_flow_async_action_handle_create; rte_flow_async_action_handle_destroy; rte_flow_async_action_handle_update; + rte_eth_direct_rxrearm_map; }; INTERNAL { --2.25.1
Re: [PATCH v1 3/5] ethdev: add API for direct rearm mode
In reply to Feifei WangQuoted 52 lines · Feifei Wang
Add API for enabling direct rearm mode and for mapping RX and TX queues. Currently, the API supports 1:1(txq : rxq) mapping.
Suggested-by Honnappa Nagarahalli <[email protected]>Signed-off-by Feifei Wang <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>Reviewed-by Honnappa Nagarahalli <[email protected]>--- lib/ethdev/ethdev_driver.h | 15 +++++++++++++++ lib/ethdev/rte_ethdev.c | 14 ++++++++++++++ lib/ethdev/rte_ethdev.h | 31 +++++++++++++++++++++++++++++++ lib/ethdev/version.map | 1 + 4 files changed, 61 insertions(+)lib/ethdev/ethdev_driver.h +15
-0@@ -485,6 +485,16 @@ typedef int (*eth_rx_enable_intr_t)(struct rte_eth_dev *dev,typedef int (*eth_rx_disable_intr_t )(struct rte_eth_dev *dev,uint16_t rx_queue_id); +/** @internal Enable direct rearm of a receive queue of an Ethernet device. */ +typedef int (*eth_rx_direct_rearm_enable_t )(struct rte_eth_dev *dev, +uint16_t queue_id); + +/**< @internal map Rx/Tx queue of direct rearm mode */ +typedef int (*eth_rx_direct_rearm_map_t )(struct rte_eth_dev *dev, +uint16_t rx_queue_id, +uint16_t tx_port_id, +uint16_t tx_queue_id); +/** @internal Release memory resources allocated by given Rx/Tx queue. */ typedef void (*eth_queue_release_t )(struct rte_eth_dev *dev,uint16_t queue_id);@@ -1152,6 +1162,11 @@ struct eth_dev_ops {/** Disable Rx queue interrupt */ eth_rx_disable_intr_t rx_queue_intr_disable; +/** Enable Rx queue direct rearm mode */ +eth_rx_direct_rearm_enable_t rx_queue_direct_rearm_enable; +/** Map Rx/Tx queue for direct rearm mode */ +eth_rx_direct_rearm_map_t rx_queue_direct_rearm_map; +eth_tx_queue_setup_t tx_queue_setup;/**< Set up device Tx queue */ eth_queue_release_t tx_queue_release;/**< Release Tx queue */ eth_tx_done_cleanup_t tx_done_cleanup;/**< Free Tx ring mbufs */ lib/ethdev/rte_ethdev.c +0
-0
lib/ethdev/rte_ethdev.c rte_eth_tx_hairpin_queue_setup(uint16_t port_id, uint16_t tx_queue_id, · line 2148
Quoted 10 lines
lib/ethdev/rte_ethdev.c +7
-0@@ -2139,6 +2139,20 @@ rte_eth_tx_hairpin_queue_setup(uint16_t port_id, uint16_t tx_queue_id,return eth_err (port_id, ret); } +int +rte_eth_direct_rxrearm_map (uint16_t rx_port_id,uint16_t rx_queue_id, +uint16_t tx_port_id,uint16_t tx_queue_id) +{ +struct rte_eth_dev *dev; + + dev = &rte_eth_devices[rx_port_id];
I think it is rather control path. So:
We need standard checks that rx_port_id is valid. tx_port_id must be checked as well.rx_queue_id and tx_queue_id must be checked to be in the rate.
lib/ethdev/rte_ethdev.c rte_eth_tx_hairpin_queue_setup(uint16_t port_id, uint16_t tx_queue_id, · line 2151
Quoted 3 lines
lib/ethdev/rte_ethdev.c +3
-0+ (*dev->dev_ops->rx_queue_direct_rearm_enable)(dev, rx_queue_id); + (*dev->dev_ops->rx_queue_direct_rearm_map)(dev, rx_queue_id, + tx_port_id, tx_queue_id);
We must check that function pointers are not NULL as usual. Return values must be checked.
Isn't is safe to setup map and than enable. Otherwise we definitely need disable.
Also, what should happen on Tx port unplug? How to continue if we still have Rx port up and running?
Quoted 15 lines
lib/ethdev/rte_ethdev.c +4
-0+ +return 0 ; +} +int rte_eth_hairpin_bind (uint16_t tx_port,uint16_t rx_port) {lib/ethdev/rte_ethdev.h +0
-0@@ -5190,6 +5190,37 @@ __rte_experimentalint rte_eth_dev_hairpin_capability_get (uint16_t port_id,struct rte_eth_hairpin_cap *cap);
lib/ethdev/rte_ethdev.h __rte_experimental · line 5202
Quoted 10 lines
lib/ethdev/rte_ethdev.h +10
-0+/** +* @warning +* @b EXPERIMENTAL: this API may change, or be removed, without prior notice +* +* Enable direct re-arm mode. In this mode the RX queue will be re-armed using +* buffers that have completed transmission on the transmit side. +* +* @note +* It is assumed that the buffers have completed transmission belong to the +* mempool used at the receive side, and have refcnt = 1.
I think it is possible to avoid such limitations, but implementation will be less optimized - more checks.
lib/ethdev/rte_ethdev.h __rte_experimental · line 5211
Quoted 9 lines
lib/ethdev/rte_ethdev.h +9
-0+* +* @param rx_port_id +* Port identifying the receive side. +* @param rx_queue_id +* The index of the receive queue identifying the receive side. +* The value must be in the range [0, nb_rx_queue - 1] previously supplied +* to rte_eth_dev_configure(). +* @param tx_port_id +* Port identifying the transmit side.
I guess there is an assumption that Rx and Tx ports are serviced by the same driver. If so and if it is an API limitation, ethdev layer must check it.
Quoted 27 lines
lib/ethdev/rte_ethdev.h +12
-0+* @param tx_queue_id +* The index of the transmit queue identifying the transmit side. +* The value must be in the range [0, nb_tx_queue - 1] previously supplied +* to rte_eth_dev_configure(). +* +* @return +* - (0) if successful. +*/ +__rte_experimental +int rte_eth_direct_rxrearm_map(uint16_t rx_port_id, uint16_t rx_queue_id, +uint16_t tx_port_id,uint16_t tx_queue_id); +/** * @warning * @b EXPERIMENTAL: this structure may change without prior notice. lib/ethdev/version.map +1
-0@@ -279,6 +279,7 @@ EXPERIMENTAL { rte_flow_async_action_handle_create; rte_flow_async_action_handle_destroy; rte_flow_async_action_handle_update; + rte_eth_direct_rxrearm_map; }; INTERNAL {
回复: [PATCH v1 3/5] ethdev: add API for direct rearm mode
Quoted 74 lines
-----邮件原件-----
发件人: Andrew Rybchenko <[email protected]> 发送时间: Wednesday, April 20, 2022 6:41 PM
收件人: Feifei Wang <[email protected]>; [email protected]; Ferruh Yigit <[email protected]>; Ray Kinsella <[email protected]> 抄送: [email protected]; nd <[email protected]>; Honnappa Nagarahalli <[email protected]>; Ruifeng Wang <[email protected]> 主题: Re: [PATCH v1 3/5] ethdev: add API for direct rearm modelib/ethdev/ethdev_driver.h typedef int (*eth_rx_enable_intr_t)(struct
Quoted 19 lines · Feifei Wang
Add API for enabling direct rearm mode and for mapping RX and TX queues. Currently, the API supports 1:1(txq : rxq) mapping.
Suggested-by Honnappa Nagarahalli <[email protected]>Signed-off-by Feifei Wang <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>Reviewed-by Honnappa Nagarahalli <[email protected]>--- lib/ethdev/ethdev_driver.h | 15 +++++++++++++++ lib/ethdev/rte_ethdev.c | 14 ++++++++++++++ lib/ethdev/rte_ethdev.h | 31 +++++++++++++++++++++++++++++++ lib/ethdev/version.map | 1 + 4 files changed, 61 insertions(+)lib/ethdev/ethdev_driver.h +0
-0@@ -485,6 +485,16 @@ typedef int (*eth_rx_enable_intr_t)(struct
rte_eth_dev*dev,lib/ethdev/ethdev_driver.h typedef int (*eth_rx_enable_intr_t)(struct rte_eth_dev *dev,
Quoted 5 lines
lib/ethdev/ethdev_driver.h +2
-0typedef int (*eth_rx_disable_intr_t )(struct rte_eth_dev *dev,uint16_t rx_queue_id); +/** @internal Enable direct rearm of a receive queue of an Ethernet +device. */ typedef int (*eth_rx_direct_rearm_enable_t)(struct
rte_eth_dev*dev,lib/ethdev/ethdev_driver.h typedef int (*eth_rx_enable_intr_t)(struct rte_eth_dev *dev,
Quoted 9 lines
lib/ethdev/ethdev_driver.h +8
-0+ uint16_t queue_id); + +/**< @internal map Rx/Tx queue of direct rearm mode */ typedef int +(*eth_rx_direct_rearm_map_t )(struct rte_eth_dev *dev, +uint16_t rx_queue_id, +uint16_t tx_port_id, +uint16_t tx_queue_id); +/** @internal Release memory resources allocated by given Rx/Tx queue. */
lib/ethdev/ethdev_driver.h struct eth_dev_ops { · line 1170
Quoted 12 lines
lib/ethdev/ethdev_driver.h +5
-0typedef void (*eth_queue_release_t )(struct rte_eth_dev *dev,uint16_t queue_id);@@ -1152,6 +1162,11 @@ struct eth_dev_ops {/** Disable Rx queue interrupt */ eth_rx_disable_intr_t rx_queue_intr_disable; +/** Enable Rx queue direct rearm mode */ +eth_rx_direct_rearm_enable_t rx_queue_direct_rearm_enable; +/** Map Rx/Tx queue for direct rearm mode */ +eth_rx_direct_rearm_map_t rx_queue_direct_rearm_map; +eth_tx_queue_setup_t tx_queue_setup;/**< Set up device Tx queue */
lib/ethdev/ethdev_driver.h struct eth_dev_ops { · line 1171
Quoted 1 line
lib/ethdev/ethdev_driver.h +0
-0eth_queue_release_t tx_queue_release;/**< Release Tx queue */
lib/ethdev/ethdev_driver.h struct eth_dev_ops { · line 1172
Quoted 1 line
lib/ethdev/ethdev_driver.h +0
-0eth_tx_done_cleanup_t tx_done_cleanup;/**< Free Tx ring mbufs */
lib/ethdev/rte_ethdev.c rte_eth_tx_hairpin_queue_setup(uint16_t
Quoted 5 lines
lib/ethdev/rte_ethdev.c index renamed from lib/ethdev/rte_ethdev.c +0
-0
29a3d80466..8e6f0284f4100644lib/ethdev/rte_ethdev.c +0
-0@@ -2139,6 +2139,20 @@ rte_eth_tx_hairpin_queue_setup(uint16_t
port_id,uint16_ttx_queue_id,Quoted 3 lines
lib/ethdev/rte_ethdev.c +0
-0return eth_err (port_id, ret); }
Quoted 11 lines
lib/ethdev/rte_ethdev.c rte_eth_tx_hairpin_queue_setup(uint16_t port_id, uint16_t tx_queue_id, · line 2148
Quoted 6 lines
lib/ethdev/rte_ethdev.c +6
-0+int +rte_eth_direct_rxrearm_map (uint16_t rx_port_id,uint16_t rx_queue_id, +uint16_t tx_port_id,uint16_t tx_queue_id) { +struct rte_eth_dev *dev; + + dev = &rte_eth_devices[rx_port_id];I think it is rather control path. So:
We need standard checks thatrx_port_idis valid.tx_port_idmust be checked as well.rx_queue_idandtx_queue_idmust be checked to be in the rate.
[Feifei] You are right, I will add check for these.
Quoted 8 lines
lib/ethdev/rte_ethdev.c rte_eth_tx_hairpin_queue_setup(uint16_t · line 2148
Quoted 2 lines
lib/ethdev/rte_ethdev.c +1
-0+ (*dev->dev_ops->rx_queue_direct_rearm_enable)(dev,
rx_queue_id);lib/ethdev/rte_ethdev.c rte_eth_tx_hairpin_queue_setup(uint16_t port_id, uint16_t tx_queue_id, · line 2151
Quoted 2 lines
lib/ethdev/rte_ethdev.c +2
-0+ (*dev->dev_ops->rx_queue_direct_rearm_map)(dev, rx_queue_id, + tx_port_id, tx_queue_id);We must check that function pointers are not NULL as usual. Return values must be checked.
[Feifei] I agree with this, The check for pointer and return value will be added
Quoted 2 lines
Isn't is safe to setup map and than enable.
Otherwise we definitely need disable.
[Feifei] I will change code that map first and then set 'rxq->offload' to enable direct-rearm mode.
Quoted 2 lines
Also, what should happen on Tx port unplug? How to continue if we still have Rx port up and running?
[Feifei] For direct rearm mode, if Tx port unplug, it means there is no buffer from Tx. And then, Rx will put buffer from mempool as usual for rearm.
Quoted 28 lines
Quoted 17 lines
lib/ethdev/rte_ethdev.c +4
-0+ +return 0 ; +} +int rte_eth_hairpin_bind (uint16_t tx_port,uint16_t rx_port) {lib/ethdev/rte_ethdev.h index renamed from lib/ethdev/rte_ethdev.h +0
-0
04cff8ee10..4a431fcbed100644lib/ethdev/rte_ethdev.h +1
-0@@ -5190,6 +5190,37 @@ __rte_experimentalint rte_eth_dev_hairpin_capability_get (uint16_t port_id,struct rte_eth_hairpin_cap *cap); +/** lib/ethdev/rte_ethdev.h __rte_experimental · line 5203
Quoted 10 lines
lib/ethdev/rte_ethdev.h +10
-0+* @warning +* @b EXPERIMENTAL: this API may change, or be removed, without prior +notice +* +* Enable direct re-arm mode. In this mode the RX queue will be +re-armed using +* buffers that have completed transmission on the transmit side. +* +* @note +* It is assumed that the buffers have completed transmission belong to the
Quoted 4 lines
lib/ethdev/rte_ethdev.h __rte_experimental · line 5202
Quoted 1 line
lib/ethdev/rte_ethdev.h +1
-0+* mempool used at the receive side, and have refcnt = 1. I think it is possible to avoid such limitations, but implementation will be less optimized - more checks.
[Feifei] For the first limitation: Rx and Tx buffers should be from the same mempool. If we want to check this, we will add a check for each packet in the data-plane, this will significantly reduce performance. So, it is better to note this for users rather than adding check code.
For the second limitation: refcnt = 1. We have now add code to support 'refcnt = 1' in direct-rearm mode, so this note can be removed in the next version.
Quoted 7 lines
lib/ethdev/rte_ethdev.h __rte_experimental
Quoted 7 lines
lib/ethdev/rte_ethdev.h +6
-0+* +* @param rx_port_id +* Port identifying the receive side. +* @param rx_queue_id +* The index of the receive queue identifying the receive side. +* The value must be in the range [0, nb_rx_queue - 1] previously
Quoted 7 lines
supplied
lib/ethdev/rte_ethdev.h __rte_experimental · line 5211
Quoted 3 lines
lib/ethdev/rte_ethdev.h +3
-0+* to rte_eth_dev_configure(). +* @param tx_port_id +* Port identifying the transmit side. I guess there is an assumption that Rx and Tx ports are serviced by the same driver. If so and if it is an API limitation, ethdev layer must check it.
[Feifei] I agree with this. For the check that Rx and Tx port should be the same driver, I will add check for this.
Quoted 30 lines
lib/ethdev/rte_ethdev.h __rte_experimental
Quoted 4 lines
lib/ethdev/rte_ethdev.h +3
-0+* @param tx_queue_id +* The index of the transmit queue identifying the transmit side. +* The value must be in the range [0, nb_tx_queue - 1] previously supplied
lib/ethdev/rte_ethdev.h __rte_experimental
Quoted 7 lines
lib/ethdev/rte_ethdev.h +7
-0+* to rte_eth_dev_configure(). +* +* @return +* - (0) if successful. +*/ +__rte_experimental +int rte_eth_direct_rxrearm_map(uint16_t rx_port_id, uint16_t
rx_queue_id,Quoted 17 lines
lib/ethdev/rte_ethdev.h +2
-0+ uint16_t tx_port_id, uint16_t tx_queue_id); +/** * @warning * @b EXPERIMENTAL: this structure may change without prior notice. lib/ethdev/version.map index renamed from lib/ethdev/version.map +0
-0
20391ab29e..68d664498c100644lib/ethdev/version.map +1
-0@@ -279,6 +279,7 @@ EXPERIMENTAL { rte_flow_async_action_handle_create; rte_flow_async_action_handle_destroy; rte_flow_async_action_handle_update; + rte_eth_direct_rxrearm_map; }; INTERNAL {
RE: [PATCH v1 3/5] ethdev: add API for direct rearm mode
<snip>
Quoted 89 lines
Quoted 64 lines · Feifei Wang
lib/ethdev/ethdev_driver.h typedef int (*eth_rx_enable_intr_t)(struct
Quoted 19 lines
Add API for enabling direct rearm mode and for mapping RX and TX queues. Currently, the API supports 1:1(txq : rxq) mapping.
Suggested-by Honnappa Nagarahalli <[email protected]>Signed-off-by Feifei Wang <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>Reviewed-by Honnappa Nagarahalli <[email protected]>--- lib/ethdev/ethdev_driver.h | 15 +++++++++++++++ lib/ethdev/rte_ethdev.c | 14 ++++++++++++++ lib/ethdev/rte_ethdev.h | 31 +++++++++++++++++++++++++++++++ lib/ethdev/version.map | 1 + 4 files changed, 61 insertions(+)lib/ethdev/ethdev_driver.h +0
-0@@ -485,6 +485,16 @@ typedef int (*eth_rx_enable_intr_t)(struct
rte_eth_dev*dev,lib/ethdev/ethdev_driver.h typedef int (*eth_rx_enable_intr_t)(struct rte_eth_dev *dev,
Quoted 5 lines
lib/ethdev/ethdev_driver.h +2
-0typedef int (*eth_rx_disable_intr_t )(struct rte_eth_dev *dev,uint16_t rx_queue_id); +/** @internal Enable direct rearm of a receive queue of an Ethernet +device. */ typedef int (*eth_rx_direct_rearm_enable_t)(struct
rte_eth_dev*dev,lib/ethdev/ethdev_driver.h typedef int (*eth_rx_enable_intr_t)(struct rte_eth_dev *dev,
Quoted 9 lines
lib/ethdev/ethdev_driver.h +8
-0+ uint16_t queue_id); + +/**< @internal map Rx/Tx queue of direct rearm mode */ typedef int +(*eth_rx_direct_rearm_map_t )(struct rte_eth_dev *dev, +uint16_t rx_queue_id, +uint16_t tx_port_id, +uint16_t tx_queue_id); +/** @internal Release memory resources allocated by given Rx/Tx queue. */
lib/ethdev/ethdev_driver.h struct eth_dev_ops { · line 1170
Quoted 12 lines
lib/ethdev/ethdev_driver.h +5
-0typedef void (*eth_queue_release_t )(struct rte_eth_dev *dev,uint16_t queue_id);@@ -1152,6 +1162,11 @@ struct eth_dev_ops {/** Disable Rx queue interrupt */ eth_rx_disable_intr_t rx_queue_intr_disable; +/** Enable Rx queue direct rearm mode */ +eth_rx_direct_rearm_enable_t rx_queue_direct_rearm_enable; +/** Map Rx/Tx queue for direct rearm mode */ +eth_rx_direct_rearm_map_t rx_queue_direct_rearm_map; +eth_tx_queue_setup_t tx_queue_setup;/**< Set up device Tx queue */
lib/ethdev/ethdev_driver.h struct eth_dev_ops { · line 1171
Quoted 1 line
lib/ethdev/ethdev_driver.h +0
-0eth_queue_release_t tx_queue_release;/**< Release Tx queue */
lib/ethdev/ethdev_driver.h struct eth_dev_ops { · line 1172
Quoted 1 line
lib/ethdev/ethdev_driver.h +0
-0eth_tx_done_cleanup_t tx_done_cleanup;/**< Free Tx ring mbufs */
lib/ethdev/rte_ethdev.c rte_eth_tx_hairpin_queue_setup(uint16_t
Quoted 5 lines
lib/ethdev/rte_ethdev.c index renamed from lib/ethdev/rte_ethdev.c +0
-0
29a3d80466..8e6f0284f4100644lib/ethdev/rte_ethdev.c +0
-0@@ -2139,6 +2139,20 @@ rte_eth_tx_hairpin_queue_setup(uint16_t
port_id,uint16_ttx_queue_id,Quoted 3 lines
lib/ethdev/rte_ethdev.c +0
-0return eth_err (port_id, ret); }Quoted 11 lines
lib/ethdev/rte_ethdev.c rte_eth_tx_hairpin_queue_setup(uint16_t port_id, uint16_t tx_queue_id, · line 2148
Quoted 6 lines
lib/ethdev/rte_ethdev.c +6
-0+int +rte_eth_direct_rxrearm_map (uint16_t rx_port_id,uint16_t rx_queue_id, +uint16_t tx_port_id,uint16_t tx_queue_id) { +struct rte_eth_dev *dev; + + dev = &rte_eth_devices[rx_port_id];I think it is rather control path. So:
We need standard checks thatrx_port_idis valid.tx_port_idmust be checked as well.rx_queue_idandtx_queue_idmust be checked to be in the rate.[Feifei] You are right, I will add check for these.
Quoted 8 lines
lib/ethdev/rte_ethdev.c rte_eth_tx_hairpin_queue_setup(uint16_t · line 2148
Quoted 2 lines
lib/ethdev/rte_ethdev.c +1
-0+ (*dev->dev_ops->rx_queue_direct_rearm_enable)(dev,
rx_queue_id);lib/ethdev/rte_ethdev.c rte_eth_tx_hairpin_queue_setup(uint16_t port_id, uint16_t tx_queue_id, · line 2151
Quoted 2 lines
lib/ethdev/rte_ethdev.c +2
-0+ (*dev->dev_ops->rx_queue_direct_rearm_map)(dev, rx_queue_id, + tx_port_id, tx_queue_id);We must check that function pointers are not NULL as usual. Return values must be checked.
[Feifei] I agree with this, The check for pointer and return value will be added
Quoted 2 lines
Isn't is safe to setup map and than enable.
Otherwise we definitely need disable.
Quoted 8 lines
[Feifei] I will change code that map first and then set '
rxq->offload' to enable direct-rearm mode.Quoted 2 lines
Also, what should happen on Tx port unplug? How to continue if we still have Rx port up and running?
[Feifei] For direct rearm mode, if Tx port unplug, it means there is no buffer from Tx.
And then, Rx will put buffer from mempool as usual for rearm.
Andrew, when you say 'TX port unplug', do you mean the 'rte_eth_dev_tx_queue_stop' is called? Is calling 'rte_eth_dev_tx_queue_stop' allowed when the device is running?
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
lib/ethdev/rte_ethdev.c +4
-0+ +return 0 ; +} +
<snip>
Re: [PATCH v1 3/5] ethdev: add API for direct rearm mode
Quoted 22 lines · Honnappa Nagarahalli
Quoted 22 lines
Quoted 10 lines · Feifei Wang
Review
Quoted 8 lines
Add API for enabling direct rearm mode and for mapping RX and TX queues. Currently, the API supports 1:1(txq : rxq) mapping.
Suggested-by Honnappa Nagarahalli <[email protected]>Signed-off-by Feifei Wang <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>Reviewed-by Honnappa Nagarahalli <[email protected]>(quoted diff) +1
-0+ (*dev->dev_ops->rx_queue_direct_rearm_enable)(dev,
rx_queue_id);Quoted 5 lines
lib/ethdev/rte_ethdev.c rte_eth_tx_hairpin_queue_setup(uint16_t port_id, uint16_t tx_queue_id, · line 2151
Quoted 2 lines
lib/ethdev/rte_ethdev.c +2
-0+ (*dev->dev_ops->rx_queue_direct_rearm_map)(dev, rx_queue_id, + tx_port_id, tx_queue_id);We must check that function pointers are not NULL as usual. Return values must be checked.
[Feifei] I agree with this, The check for pointer and return value will be added
Quoted 2 lines
Isn't is safe to setup map and than enable.
Otherwise we definitely need disable.[Feifei] I will change code that map first and then set '
rxq->offload' to enable direct-rearm mode.
Quoted 6 lines
Quoted 5 lines
Quoted 2 lines
Also, what should happen on Tx port unplug? How to continue if we still have Rx port up and running?
[Feifei] For direct rearm mode, if Tx port unplug, it means there is no buffer from Tx.
And then, Rx will put buffer from mempool as usual for rearm.Andrew, when you say 'TX port unplug', do you mean the '
rte_eth_dev_tx_queue_stop' is called? Is calling 'rte_eth_dev_tx_queue_stop' allowed when the device is running?
I think that deferred start and presence of rte_eth_dev_tx_queue_stop() implies the possibility to stop Tx queue. But, yes, application should care about conditions to have no traffic running to Tx queue.
Anyway, I was talking about hot unplug of the entire device used as Tx port in above config.
Re: [PATCH v1 3/5] ethdev: add API for direct rearm mode
In reply to Feifei WangQuoted 14 lines · Feifei Wang
Add API for enabling direct rearm mode and for mapping RX and TX queues. Currently, the API supports 1:1(txq : rxq) mapping.
Suggested-by Honnappa Nagarahalli <[email protected]>Signed-off-by Feifei Wang <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>Reviewed-by Honnappa Nagarahalli <[email protected]>---
+ *
+ * @return
+ * - (0) if successful.
+ */
lib/ethdev/rte_ethdev.h __rte_experimental · line 5222
Quoted 3 lines
lib/ethdev/rte_ethdev.h +3
-0+__rte_experimental +int rte_eth_direct_rxrearm_map(uint16_t rx_port_id, uint16_t rx_queue_id, +uint16_t tx_port_id,uint16_t tx_queue_id);
Won't existing rte_eth_hairpin_* APIs work to achieve the same?
回复: [PATCH v1 3/5] ethdev: add API for direct rearm mode
Quoted 32 lines
-----邮件原件-----
发件人: Jerin Jacob <[email protected]>
发送时间: Wednesday, April 20, 2022 6:50 PM
收件人: Feifei Wang <[email protected]>
抄送: [email protected]; Ferruh Yigit <[email protected]>; Andrew Rybchenko <[email protected]>; Ray Kinsella <[email protected]>; dpdk-dev <[email protected]>; nd <[email protected]>; Honnappa Nagarahalli <[email protected]>; Ruifeng Wang <[email protected]>
主题: Re: [PATCH v1 3/5] ethdev: add API for direct rearm modeQuoted 14 lines · Feifei Wang
Add API for enabling direct rearm mode and for mapping RX and TX queues. Currently, the API supports 1:1(txq : rxq) mapping.
Suggested-by Honnappa Nagarahalli <[email protected]>Signed-off-by Feifei Wang <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>Reviewed-by Honnappa Nagarahalli <[email protected]>---
+ *
+ * @return
+ * - (0) if successful.
+ */Review
Quoted 2 lines
(quoted diff) +2
-0+__rte_experimental +int rte_eth_direct_rxrearm_map(uint16_t rx_port_id, uint16_t
rx_queue_id,
Quoted 4 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+ uint16_t tx_port_id, uint16_t +tx_queue_id);Won't existing rte_eth_hairpin_* APIs work to achieve the same?
[Feifei] Thanks for the comment. Look at the hairpin feature which is enabled in MLX5 driver.
I think the most important difference is that hairpin just re-directs the packet from the Rx queue to Tx queue in the same port, and Rx/Tx queue just can record the peer queue id. For direct rearm, it can map Rx queue to the Tx queue which are from different ports. And this needs Rx queue records paired port id and queue id.
Furthermore, hairpin needs to set up new hairpin queue and then it can bind Rx queue to Tx queue. and direct-rearm just can use normal queue to map. This is due to direct rearm needs used buffers and it doesn't care about packet.
Re: [PATCH v1 3/5] ethdev: add API for direct rearm mode
In reply to Feifei WangQuoted 100 lines · On Wed, 20 Apr 2022 16:16:48 +0800 Feifei Wang <[email protected]> wrote:
Add API for enabling direct rearm mode and for mapping RX and TX queues. Currently, the API supports 1:1(txq : rxq) mapping.
Suggested-by Honnappa Nagarahalli <[email protected]>Signed-off-by Feifei Wang <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>Reviewed-by Honnappa Nagarahalli <[email protected]>--- lib/ethdev/ethdev_driver.h | 15 +++++++++++++++ lib/ethdev/rte_ethdev.c | 14 ++++++++++++++ lib/ethdev/rte_ethdev.h | 31 +++++++++++++++++++++++++++++++ lib/ethdev/version.map | 1 + 4 files changed, 61 insertions(+)lib/ethdev/ethdev_driver.h +15
-0@@ -485,6 +485,16 @@ typedef int (*eth_rx_enable_intr_t)(struct rte_eth_dev *dev,typedef int (*eth_rx_disable_intr_t )(struct rte_eth_dev *dev,uint16_t rx_queue_id); +/** @internal Enable direct rearm of a receive queue of an Ethernet device. */ +typedef int (*eth_rx_direct_rearm_enable_t )(struct rte_eth_dev *dev, +uint16_t queue_id); + +/**< @internal map Rx/Tx queue of direct rearm mode */ +typedef int (*eth_rx_direct_rearm_map_t )(struct rte_eth_dev *dev, +uint16_t rx_queue_id, +uint16_t tx_port_id, +uint16_t tx_queue_id); +/** @internal Release memory resources allocated by given Rx/Tx queue. */ typedef void (*eth_queue_release_t )(struct rte_eth_dev *dev,uint16_t queue_id);@@ -1152,6 +1162,11 @@ struct eth_dev_ops {/** Disable Rx queue interrupt */ eth_rx_disable_intr_t rx_queue_intr_disable; +/** Enable Rx queue direct rearm mode */ +eth_rx_direct_rearm_enable_t rx_queue_direct_rearm_enable; +/** Map Rx/Tx queue for direct rearm mode */ +eth_rx_direct_rearm_map_t rx_queue_direct_rearm_map; +eth_tx_queue_setup_t tx_queue_setup;/**< Set up device Tx queue */ eth_queue_release_t tx_queue_release;/**< Release Tx queue */ eth_tx_done_cleanup_t tx_done_cleanup;/**< Free Tx ring mbufs */ lib/ethdev/rte_ethdev.c +14
-0@@ -2139,6 +2139,20 @@ rte_eth_tx_hairpin_queue_setup(uint16_t port_id, uint16_t tx_queue_id,return eth_err (port_id, ret); } +int +rte_eth_direct_rxrearm_map (uint16_t rx_port_id,uint16_t rx_queue_id, +uint16_t tx_port_id,uint16_t tx_queue_id) +{ +struct rte_eth_dev *dev; + + dev = &rte_eth_devices[rx_port_id]; + (*dev->dev_ops->rx_queue_direct_rearm_enable)(dev, rx_queue_id); + (*dev->dev_ops->rx_queue_direct_rearm_map)(dev, rx_queue_id, + tx_port_id, tx_queue_id); + +return 0 ; +} +int rte_eth_hairpin_bind (uint16_t tx_port,uint16_t rx_port) {lib/ethdev/rte_ethdev.h +20
-0@@ -5190,6 +5190,37 @@ __rte_experimentalint rte_eth_dev_hairpin_capability_get (uint16_t port_id,struct rte_eth_hairpin_cap *cap); +/** +* @warning +* @b EXPERIMENTAL: this API may change, or be removed, without prior notice +* +* Enable direct re-arm mode. In this mode the RX queue will be re-armed using +* buffers that have completed transmission on the transmit side. +* +* @note +* It is assumed that the buffers have completed transmission belong to the +* mempool used at the receive side, and have refcnt = 1. +* +* @param rx_port_id +* Port identifying the receive side. +* @param rx_queue_id +* The index of the receive queue identifying the receive side. +* The value must be in the range [0, nb_rx_queue - 1] previously supplied +* to rte_eth_dev_configure(). +* @param tx_port_id +* Port identifying the transmit side. +* @param tx_queue_id
lib/ethdev/rte_ethdev.h __rte_experimental · line 5222
Quoted 10 lines
lib/ethdev/rte_ethdev.h +10
-0+* The index of the transmit queue identifying the transmit side. +* The value must be in the range [0, nb_tx_queue - 1] previously supplied +* to rte_eth_dev_configure(). +* +* @return +* - (0) if successful. +*/ +__rte_experimental +int rte_eth_direct_rxrearm_map(uint16_t rx_port_id, uint16_t rx_queue_id, +uint16_t tx_port_id,uint16_t tx_queue_id);
Just looking at this.
Why is this done via API call and not a flag as part of the receive config? All the other offload and configuration happens via dev config. Doing it this way doesn't follow the existing ethdev model.
回复: [PATCH v1 3/5] ethdev: add API for direct rearm mode
Quoted 136 lines
-----邮件原件-----
发件人: Stephen Hemminger <[email protected]>
发送时间: Thursday, April 21, 2022 10:58 PM
收件人: Feifei Wang <[email protected]>
抄送: [email protected]; Ferruh Yigit <[email protected]>; Andrew Rybchenko <[email protected]>; Ray Kinsella <[email protected]>; [email protected]; nd <[email protected]>; Honnappa Nagarahalli <[email protected]>; Ruifeng Wang <[email protected]>
主题: Re: [PATCH v1 3/5] ethdev: add API for direct rearm modeQuoted 19 lines · On Wed, 20 Apr 2022 16:16:48 +0800 Feifei Wang <[email protected]> wrote:
Add API for enabling direct rearm mode and for mapping RX and TX queues. Currently, the API supports 1:1(txq : rxq) mapping.
Suggested-by Honnappa Nagarahalli <[email protected]>Signed-off-by Feifei Wang <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>Reviewed-by Honnappa Nagarahalli <[email protected]>--- lib/ethdev/ethdev_driver.h | 15 +++++++++++++++ lib/ethdev/rte_ethdev.c | 14 ++++++++++++++ lib/ethdev/rte_ethdev.h | 31 +++++++++++++++++++++++++++++++ lib/ethdev/version.map | 1 + 4 files changed, 61 insertions(+)lib/ethdev/ethdev_driver.h +0
-0@@ -485,6 +485,16 @@ typedef int (*eth_rx_enable_intr_t)(structQuoted 1 line
rte_eth_dev*dev, typedef int (*eth_rx_disable_intr_t)(structrte_eth_dev*dev,
lib/ethdev/ethdev_driver.h typedef int (*eth_rx_enable_intr_t)(struct rte_eth_dev *dev,
Quoted 4 lines
lib/ethdev/ethdev_driver.h +2
-0uint16_t rx_queue_id); +/** @internal Enable direct rearm of a receive queue of an Ethernet +device. */ typedef int (*eth_rx_direct_rearm_enable_t)(struct
rte_eth_dev*dev,Quoted 12 lines
lib/ethdev/ethdev_driver.h +8
-0+ uint16_t queue_id); + +/**< @internal map Rx/Tx queue of direct rearm mode */ typedef int +(*eth_rx_direct_rearm_map_t )(struct rte_eth_dev *dev, +uint16_t rx_queue_id, +uint16_t tx_port_id, +uint16_t tx_queue_id); +/** @internal Release memory resources allocated by given Rx/Tx queue. */ typedef void (*eth_queue_release_t)(struct
rte_eth_dev*dev,lib/ethdev/ethdev_driver.h +0
-0uint16_t queue_id);lib/ethdev/ethdev_driver.h struct eth_dev_ops { · line 1170
Quoted 9 lines
lib/ethdev/ethdev_driver.h +5
-0@@ -1152,6 +1162,11 @@ struct eth_dev_ops {/** Disable Rx queue interrupt */ eth_rx_disable_intr_t rx_queue_intr_disable; +/** Enable Rx queue direct rearm mode */ +eth_rx_direct_rearm_enable_t rx_queue_direct_rearm_enable; +/** Map Rx/Tx queue for direct rearm mode */ +eth_rx_direct_rearm_map_t rx_queue_direct_rearm_map; +eth_tx_queue_setup_t tx_queue_setup;/**< Set up device Tx queue */
lib/ethdev/ethdev_driver.h struct eth_dev_ops { · line 1171
Quoted 1 line
lib/ethdev/ethdev_driver.h +0
-0eth_queue_release_t tx_queue_release;/**< Release Tx queue */
lib/ethdev/ethdev_driver.h struct eth_dev_ops { · line 1172
Quoted 1 line
lib/ethdev/ethdev_driver.h +0
-0eth_tx_done_cleanup_t tx_done_cleanup;/**< Free Tx ring mbufs */
lib/ethdev/rte_ethdev.c rte_eth_tx_hairpin_queue_setup(uint16_t
Quoted 5 lines
lib/ethdev/rte_ethdev.c index renamed from lib/ethdev/rte_ethdev.c +0
-0
29a3d80466..8e6f0284f4100644lib/ethdev/rte_ethdev.c +0
-0@@ -2139,6 +2139,20 @@ rte_eth_tx_hairpin_queue_setup(uint16_t
port_id,uint16_ttx_queue_id,lib/ethdev/rte_ethdev.c rte_eth_tx_hairpin_queue_setup(uint16_t port_id, uint16_t tx_queue_id,
Quoted 10 lines
lib/ethdev/rte_ethdev.c +7
-0return eth_err (port_id, ret); } +int +rte_eth_direct_rxrearm_map (uint16_t rx_port_id,uint16_t rx_queue_id, +uint16_t tx_port_id,uint16_t tx_queue_id) { +struct rte_eth_dev *dev; + + dev = &rte_eth_devices[rx_port_id]; + (*dev->dev_ops->rx_queue_direct_rearm_enable)(dev,
rx_queue_id);Quoted 17 lines
lib/ethdev/rte_ethdev.c +6
-1+ (*dev->dev_ops->rx_queue_direct_rearm_map)(dev, rx_queue_id, + tx_port_id, tx_queue_id); + +return 0 ; +} +int rte_eth_hairpin_bind (uint16_t tx_port,uint16_t rx_port) { diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h index
04cff8ee10..4a431fcbed100644lib/ethdev/rte_ethdev.h +1
-0@@ -5190,6 +5190,37 @@ __rte_experimental int rte_eth_dev_hairpin_capability_get(uint16_t port_id, struct rte_eth_hairpin_cap *cap); +/** lib/ethdev/rte_ethdev.h __rte_experimental int · line 5203
Quoted 10 lines
lib/ethdev/rte_ethdev.h +10
-0+* @warning +* @b EXPERIMENTAL: this API may change, or be removed, without prior +notice +* +* Enable direct re-arm mode. In this mode the RX queue will be +re-armed using +* buffers that have completed transmission on the transmit side. +* +* @note +* It is assumed that the buffers have completed transmission belong to the
lib/ethdev/rte_ethdev.h __rte_experimental
Quoted 7 lines
lib/ethdev/rte_ethdev.h +7
-0+* mempool used at the receive side, and have refcnt = 1. +* +* @param rx_port_id +* Port identifying the receive side. +* @param rx_queue_id +* The index of the receive queue identifying the receive side. +* The value must be in the range [0, nb_rx_queue - 1] previously supplied
lib/ethdev/rte_ethdev.h __rte_experimental
Quoted 6 lines
lib/ethdev/rte_ethdev.h +6
-0+* to rte_eth_dev_configure(). +* @param tx_port_id +* Port identifying the transmit side. +* @param tx_queue_id +* The index of the transmit queue identifying the transmit side. +* The value must be in the range [0, nb_tx_queue - 1] previously supplied
lib/ethdev/rte_ethdev.h __rte_experimental
Quoted 7 lines
lib/ethdev/rte_ethdev.h +7
-0+* to rte_eth_dev_configure(). +* +* @return +* - (0) if successful. +*/ +__rte_experimental +int rte_eth_direct_rxrearm_map(uint16_t rx_port_id, uint16_t
rx_queue_id,
Quoted 7 lines
lib/ethdev/rte_ethdev.h __rte_experimental · line 5222
Quoted 1 line
lib/ethdev/rte_ethdev.h +1
-0+ uint16_t tx_port_id, uint16_t tx_queue_id);Just looking at this.
Why is this done via API call and not a flag as part of the receive config? All the other offload and configuration happens via dev config. Doing it this way doesn't follow the existing ethdev model.
[Feifei] Agree with this. I will remove direct-rearm enable function and
use "rxq->offload" bit to enable it.
For rte_eth_direct_rxrearm_map, I think it is necessary for users to call it to map Rx/Tx queue.
PATCH 4/5 net/i40e: add direct rearm mode internal API
In reply to Feifei WangFor direct rearm mode, add two internal functions.
One is to enable direct rearm mode in Rx queue.
The other is to map Tx queue with Rx queue to make Rx queue take buffers from the specific Tx queue.
--- drivers/net/i40e/i40e_ethdev.c | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+)
drivers/net/i40e/i40e_ethdev.c +34 -0
@@ -369,6 +369,13 @@ static int i40e_dev_rx_queue_intr_enable(struct rte_eth_dev *dev,static int i40e_dev_rx_queue_intr_disable (struct rte_eth_dev *dev,uint16_t queue_id); +static int i40e_dev_rx_queue_direct_rearm_enable (struct rte_eth_dev *dev, +uint16_t queue_id); +static int i40e_dev_rx_queue_direct_rearm_map (struct rte_eth_dev *dev, +uint16_t rx_queue_id, +uint16_t tx_port_id, +uint16_t tx_queue_id); +static int i40e_get_regs (struct rte_eth_dev *dev,struct rte_dev_reg_info *regs);
@@ -477,6 +484,8 @@ static const struct eth_dev_ops i40e_eth_dev_ops = { .rx_queue_setup = i40e_dev_rx_queue_setup, .rx_queue_intr_enable = i40e_dev_rx_queue_intr_enable, .rx_queue_intr_disable = i40e_dev_rx_queue_intr_disable, + .rx_queue_direct_rearm_enable = i40e_dev_rx_queue_direct_rearm_enable, + .rx_queue_direct_rearm_map = i40e_dev_rx_queue_direct_rearm_map, .rx_queue_release = i40e_dev_rx_queue_release, .tx_queue_setup = i40e_dev_tx_queue_setup, .tx_queue_release = i40e_dev_tx_queue_release,
@@ -11108,6 +11117,31 @@ i40e_dev_rx_queue_intr_disable(struct rte_eth_dev *dev, uint16_t queue_id)return 0 ; } +static int i40e_dev_rx_queue_direct_rearm_enable (struct rte_eth_dev *dev, +uint16_t queue_id) +{ +struct i40e_rx_queue *rxq; + + rxq = dev->data->rx_queues[queue_id]; + rxq->direct_rxrearm_enable =1 ; + +return 0 ; +} + +static int i40e_dev_rx_queue_direct_rearm_map (struct rte_eth_dev *dev, +uint16_t rx_queue_id,uint16_t tx_port_id, +uint16_t tx_queue_id) +{ +struct i40e_rx_queue *rxq; + + rxq = dev->data->rx_queues[rx_queue_id]; + + rxq->direct_rxrearm_port = tx_port_id; + rxq->direct_rxrearm_queue = tx_queue_id; + +return 0 ; +} +/** * This function is used to check if the register is valid. * Below is the valid registers list for X722 only:
Signature · 1 line
2.25.1
Re: [PATCH v1 4/5] net/i40e: add direct rearm mode internal API
20/04/2022 09:16, Feifei Wang пишет:
Quoted 57 lines
For direct rearm mode, add two internal functions.
One is to enable direct rearm mode in Rx queue.
The other is to map Tx queue with Rx queue to make Rx queue take buffers from the specific Tx queue.
Suggested-by Honnappa Nagarahalli <[email protected]>Signed-off-by Feifei Wang <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>Reviewed-by Honnappa Nagarahalli <[email protected]>--- drivers/net/i40e/i40e_ethdev.c | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+)drivers/net/i40e/i40e_ethdev.c +20
-0@@ -369,6 +369,13 @@ static int i40e_dev_rx_queue_intr_enable(struct rte_eth_dev *dev,static int i40e_dev_rx_queue_intr_disable (struct rte_eth_dev *dev,uint16_t queue_id); +static int i40e_dev_rx_queue_direct_rearm_enable (struct rte_eth_dev *dev, +uint16_t queue_id); +static int i40e_dev_rx_queue_direct_rearm_map (struct rte_eth_dev *dev, +uint16_t rx_queue_id, +uint16_t tx_port_id, +uint16_t tx_queue_id); +static int i40e_get_regs (struct rte_eth_dev *dev,struct rte_dev_reg_info *regs);@@ -477,6 +484,8 @@ static const struct eth_dev_ops i40e_eth_dev_ops = { .rx_queue_setup = i40e_dev_rx_queue_setup, .rx_queue_intr_enable = i40e_dev_rx_queue_intr_enable, .rx_queue_intr_disable = i40e_dev_rx_queue_intr_disable, + .rx_queue_direct_rearm_enable = i40e_dev_rx_queue_direct_rearm_enable, + .rx_queue_direct_rearm_map = i40e_dev_rx_queue_direct_rearm_map, .rx_queue_release = i40e_dev_rx_queue_release, .tx_queue_setup = i40e_dev_tx_queue_setup, .tx_queue_release = i40e_dev_tx_queue_release,@@ -11108,6 +11117,31 @@ i40e_dev_rx_queue_intr_disable(struct rte_eth_dev *dev, uint16_t queue_id)return 0 ; } +static int i40e_dev_rx_queue_direct_rearm_enable (struct rte_eth_dev *dev, +uint16_t queue_id) +{ +struct i40e_rx_queue *rxq; + + rxq = dev->data->rx_queues[queue_id]; + rxq->direct_rxrearm_enable =1 ; + +return 0 ; +} +
drivers/net/i40e/i40e_ethdev.c i40e_dev_rx_queue_intr_disable(struct rte_eth_dev *dev, uint16_t queue_id) · line 11140
Quoted 10 lines
drivers/net/i40e/i40e_ethdev.c +10
-0+static int i40e_dev_rx_queue_direct_rearm_map (struct rte_eth_dev *dev, +uint16_t rx_queue_id,uint16_t tx_port_id, +uint16_t tx_queue_id) +{ +struct i40e_rx_queue *rxq; + + rxq = dev->data->rx_queues[rx_queue_id]; + + rxq->direct_rxrearm_port = tx_port_id; + rxq->direct_rxrearm_queue = tx_queue_id;
I don't think this function should not enable that mode blindly. Instead, it needs to check first that all pre-conditions are met (tx/rx threshold values are equal, etc.).
Quoted 7 lines
drivers/net/i40e/i40e_ethdev.c +4
-0+ +return 0 ; +} +/** * This function is used to check if the register is valid. * Below is the valid registers list for X722 only:
PATCH 5/5 examples/l3fwd: enable direct rearm mode
In reply to Feifei WangEnable direct rearm mode. The mapping is decided in the data plane based on the first packet received.
--- examples/l3fwd/l3fwd_lpm.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-)
examples/l3fwd/l3fwd_lpm.c +15 -1
@@ -147,7 +147,7 @@ lpm_main_loop(__rte_unused void *dummy)unsigned lcore_id;uint64_t prev_tsc, diff_tsc, cur_tsc;int i, nb_rx; -uint16_t portid; +uint16_t portid, tx_portid;uint8_t queueid;struct lcore_conf *qconf; const uint64_t drain_tsc = (rte_get_tsc_hz() + US_PER_S - 1) /
@@ -158,6 +158,8 @@ lpm_main_loop(__rte_unused void *dummy)const uint16_t n_rx_q = qconf->n_rx_queue;const uint16_t n_tx_p = qconf->n_tx_port; +int direct_rearm_map[n_rx_q]; +if (n_rx_q ==0 ) {RTE_LOG (INFO ,L3FWD ,"lcore %u has nothing to do\n" , lcore_id);return 0 ;
@@ -169,6 +171,7 @@ lpm_main_loop(__rte_unused void *dummy) portid = qconf->rx_queue_list[i].port_id; queueid = qconf->rx_queue_list[i].queue_id; + direct_rearm_map[i] =0 ;RTE_LOG (INFO ,L3FWD ," -- lcoreid=%u portid=%u rxqueueid=%hhu\n" , lcore_id, portid, queueid);
@@ -209,6 +212,17 @@ lpm_main_loop(__rte_unused void *dummy)if (nb_rx ==0 )continue ; +/* Determine the direct rearm mapping based on the first +* packet received on the rx queue +*/ +if (direct_rearm_map[i] ==0 ) { + tx_portid =lpm_get_dst_port (qconf, pkts_burst[0 ], + portid); +rte_eth_direct_rxrearm_map (portid, queueid, + tx_portid, queueid); + direct_rearm_map[i] =1 ; + } +#if definedRTE_ARCH_X86 || defined __ARM_NEON \ || definedRTE_ARCH_PPC_64 l3fwd_lpm_send_packets(nb_rx, pkts_burst,
Signature · 1 line
2.25.1
RE: [PATCH v1 5/5] examples/l3fwd: enable direct rearm mode
Quoted 6 lines
From: Feifei Wang [mailto:[email protected]]
Sent: Wednesday, 20 April 2022 10.17Enable direct rearm mode. The mapping is decided in the data plane based
on the first packet received.
I usually don't care much about l3fwd, but putting configuration changes in the fast path is just wrong!
Also, l3fwd is often used for benchmarking, and this small piece of code in the fast path will affect benchmark results (although only very little).
Please move it out of the fast path.
RE: [PATCH v1 5/5] examples/l3fwd: enable direct rearm mode
<snip>
Quoted 9 lines
Quoted 6 lines
From: Feifei Wang [mailto:[email protected]]
Sent: Wednesday, 20 April 2022 10.17Enable direct rearm mode. The mapping is decided in the data plane based on the first packet received.
I usually don't care much about l3fwd, but putting configuration changes in the fast path is just wrong!
I would say it depends. In this case the cycles consumed by the API are very less and configuration data is very small and is already in the cache as PMD has accessed the same data structure.
If the configuration needs more cycles than a typical (depending on the application) data plane packet processing needs or brings in enormous amount of data in to the cache, it should not be done on the data plane.
Quoted 3 lines
Also, l3fwd is often used for benchmarking, and this small piece of code in the fast path will affect benchmark results (although only very little).
We do not see any impact on the performance numbers. The reason for putting in the data plane was it covers wider use case in this L3fwd application. If the app were to be simple, the configuration could be done from the control plane. Unfortunately, the performance of L3fwd application matters.
Quoted 2 lines
Please move it out of the fast path.
RE: [PATCH v1 5/5] examples/l3fwd: enable direct rearm mode
Quoted 12 lines
From: Honnappa Nagarahalli [mailto:[email protected]]
Sent: Thursday, 21 April 2022 04.35Quoted 8 lines
Quoted 6 lines
From: Feifei Wang [mailto:[email protected]]
Sent: Wednesday, 20 April 2022 10.17Enable direct rearm mode. The mapping is decided in the data plane based on the first packet received.
I usually don't care much about l3fwd, but putting configuration
changes in the
Quoted 1 line
fast path is just wrong!
Quoted 9 lines
I would say it depends. In this case the cycles consumed by the API are very less and configuration data is very small and is already in the cache as PMD has accessed the same data structure.
If the configuration needs more cycles than a typical (depending on the application) data plane packet processing needs or brings in enormous amount of data in to the cache, it should not be done on the data plane.
As a matter of principle, configuration changes should be done outside the fast path.
If we allow an exception for this feature, it will set a bad precedent about where to put configuration code.
Quoted 10 lines
Quoted 2 lines
Also, l3fwd is often used for benchmarking, and this small piece of
code in the
Quoted 1 line
fast path will affect benchmark results (although only very little).
We do not see any impact on the performance numbers. The reason for putting in the data plane was it covers wider use case in this L3fwd application. If the app were to be simple, the configuration could be done from the control plane. Unfortunately, the performance of L3fwd application matters.
Let's proceed down that path for the sake of discussion... Then the fast path is missing runtime verification that all preconditions for using remapping are present at any time.
Quoted 2 lines
Quoted 2 lines
Please move it out of the fast path.
BTW, this patch does not call the rte_eth_direct_rxrearm_enable() to enable the feature.
And finally, this feature should be disabled by default, and only enabled by a command line parameter or similar. Otherwise, future l3fwd NIC performance reports will provide misleading performance results, if the feature is utilized. Application developers, when comparing NIC performance results, don't care about the performance for this unique use case; they care about the performance for the generic use case.
RE: [PATCH v1 5/5] examples/l3fwd: enable direct rearm mode
(apologies for the late response, this one slipped my mind)
Appreciate if others could weigh their opinions.
<snip>
Quoted 24 lines
Quoted 17 lines
From: Honnappa Nagarahalli [mailto:[email protected]]
Sent: Thursday, 21 April 2022 04.35Quoted 8 lines
Quoted 6 lines
From: Feifei Wang [mailto:[email protected]]
Sent: Wednesday, 20 April 2022 10.17Enable direct rearm mode. The mapping is decided in the data plane based on the first packet received.
I usually don't care much about l3fwd, but putting configuration
changes in the
Quoted 1 line
fast path is just wrong!
I would say it depends. In this case the cycles consumed by the API are very less and configuration data is very small and is already in the cache as PMD has accessed the same data structure.
Quoted 6 lines
If the configuration needs more cycles than a typical (depending on the
application) data plane packet processing needs or brings in enormous amount of data in to the cache, it should not be done on the data plane.
Quoted 5 lines
As a matter of principle, configuration changes should be done outside the fast path.
If we allow an exception for this feature, it will set a bad precedent about where to put configuration code.
I think there are other examples though not exactly the same. For ex: the seqlock, we cannot have a scheduled out writer while holding the lock. But, it was mentioned that this can be over come easily by running the writer on an isolated core (which to me breaks some principles).
Quoted 12 lines
Quoted 11 lines
Quoted 3 lines
Also, l3fwd is often used for benchmarking, and this small piece of
code in the
Quoted 1 line
fast path will affect benchmark results (although only very little).
We do not see any impact on the performance numbers. The reason for putting in the data plane was it covers wider use case in this L3fwd application. If the app were to be simple, the configuration could be done from the control plane. Unfortunately, the performance of L3fwd application matters.
Quoted 3 lines
Let's proceed down that path for the sake of discussion... Then the fast path is missing runtime verification that all preconditions for using remapping are present at any time.
Agree, few checks (ensuring that TX and RX buffers are from the same pool, ensuring tx_rs_thresh is same as RX rearm threshold) are missing. We will add these, it is possible to add these checks outside the packet processing loop.
Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
Please move it out of the fast path.
BTW, this patch does not call the
rte_eth_direct_rxrearm_enable()to enable the feature.
Quoted 7 lines
And finally, this feature should be disabled by default, and only enabled by a command line parameter or similar. Otherwise, future l3fwd NIC performance reports will provide misleading performance results, if the feature is utilized. Application developers, when comparing NIC performance results, don't care about the performance for this unique use case; they care about the performance for the generic use case.
I think this feature is similar to fast free feature (RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE) as you have mentioned in the other thread. It should be handled similar to how fast free feature is handled.
RE: [PATCH v1 5/5] examples/l3fwd: enable direct rearm mode
Quoted 39 lines
From: Honnappa Nagarahalli [mailto:[email protected]]
Sent: Wednesday, 11 May 2022 00.02(apologies for the late response, this one slipped my mind)
Appreciate if others could weigh their opinions.
<snip>
Quoted 8 lines
Quoted 8 lines
From: Honnappa Nagarahalli [mailto:[email protected]]
Sent: Thursday, 21 April 2022 04.35Quoted 5 lines
Quoted 5 lines
From: Feifei Wang [mailto:[email protected]]
Sent: Wednesday, 20 April 2022 10.17Enable direct rearm mode. The mapping is decided in the data
plane
Quoted 7 lines
Quoted 7 lines
Quoted 3 lines
Quoted 1 line
based on the first packet received.
I usually don't care much about l3fwd, but putting configuration
changes in the
Quoted 1 line
fast path is just wrong!
I would say it depends. In this case the cycles consumed by the API are very less and configuration data is very small and is already
in
Quoted 5 lines
Quoted 5 lines
the cache as PMD has accessed the same data structure.
If the configuration needs more cycles than a typical (depending on the
application) data plane packet processing needs or brings inenormous
Quoted 5 lines
Quoted 3 lines
amount of data in to the cache, it should not be done on the data plane.
As a matter of principle, configuration changes should be done
outside the fast
Quoted 2 lines
path.
Quoted 8 lines
Quoted 1 line
If we allow an exception for this feature, it will set a bad
precedent about
Quoted 1 line
where to put configuration code.
I think there are other examples though not exactly the same. For ex:
the seqlock, we cannot have a scheduled out writer while holding the lock. But, it was mentioned that this can be over come easily by running the writer on an isolated core (which to me breaks some principles).
Referring to a bad example (which breaks some principles) does not change my opinion. ;-)
Quoted 43 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Also, l3fwd is often used for benchmarking, and this small piece
of
Quoted 2 lines
Quoted 2 lines
code in the
Quoted 1 line
fast path will affect benchmark results (although only very
little).
Quoted 2 lines
Quoted 2 lines
We do not see any impact on the performance numbers. The reason for putting in the data plane was it covers wider use case in this
L3fwd
Quoted 1 line
Quoted 1 line
application. If the app were to be simple, the configuration could
be
Quoted 1 line
Quoted 1 line
done from the control plane. Unfortunately, the performance of
L3fwd
Quoted 4 lines
Quoted 2 lines
application matters.
Let's proceed down that path for the sake of discussion... Then the
fast path is
Quoted 1 line
missing runtime verification that all preconditions for using
remapping are
Quoted 1 line
present at any time.
Agree, few checks (ensuring that TX and RX buffers are from the same pool, ensuring
tx_rs_threshis same as RX rearm threshold) are missing. We will add these, it is possible to add these checks outside the packet processing loop.Quoted 5 lines
Quoted 3 lines
Quoted 3 lines
Please move it out of the fast path.
BTW, this patch does not call the
rte_eth_direct_rxrearm_enable()toenable
Quoted 3 lines
the feature.
And finally, this feature should be disabled by default, and only
enabled by a
Quoted 1 line
command line parameter or similar. Otherwise, future l3fwd NIC
performance
Quoted 1 line
reports will provide misleading performance results, if the feature
is utilized.
Quoted 1 line
Application developers, when comparing NIC performance results, don't
Quoted 8 lines
care
Quoted 3 lines
about the performance for this unique use case; they care about the performance for the generic use case.
I think this feature is similar to fast free feature (
RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE) as you have mentioned in the other thread. It should be handled similar to how fast free feature is handled.
I agree with this comparison.
Quickly skimming l3fwd/main.c reveals that RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE is used without checking preconditions, and thus might be buggy. E.g. what happens when the NICs support RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE, and l3fwd is run with the "per-port-pool" command line option? Obviously, the "direct rearm" patch should not be punished because of bugs in similar features ("fast free"). But it is not a valid reason to allow similar bugs. You mentioned above that precondition checking will be added, so pardon me for ranting a bit here. ;-)
Furthermore, if using l3fwd for NIC performance reports, I find the results misleading if application specific optimizations are used without mentioning it in the report. This applies to both "fast free" and "direct rearm" optimizations - they only work in specific application scenarios, and thus the l3fwd NIC performance test should be run without these optimization, or at least mention that the report only covers these specific applications. Which is why I prefer that such optimizations must be explicitly enabled through a command line parameter, and not used in testing for official NIC performance reports. Taking one step back, the real problem here is that an *example* application is used for NIC performance testing, and this is the main reason for my performance related objections. I should probably object to using l3fwd for NIC performance testing instead.
I don't feel strongly about l3fwd, so I will not object to the l3fwd patch. Just providing some feedback. :-)
Re: [PATCH v1 5/5] examples/l3fwd: enable direct rearm mode
In reply to Feifei Wang20/04/2022 09:16, Feifei Wang пишет:
Quoted 46 lines
Enable direct rearm mode. The mapping is decided in the data plane based on the first packet received.
Suggested-by Honnappa Nagarahalli <[email protected]>Signed-off-by Feifei Wang <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>Reviewed-by Honnappa Nagarahalli <[email protected]>--- examples/l3fwd/l3fwd_lpm.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-)examples/l3fwd/l3fwd_lpm.c +5
-1@@ -147,7 +147,7 @@ lpm_main_loop(__rte_unused void *dummy)unsigned lcore_id;uint64_t prev_tsc, diff_tsc, cur_tsc;int i, nb_rx; -uint16_t portid; +uint16_t portid, tx_portid;uint8_t queueid;struct lcore_conf *qconf; const uint64_t drain_tsc = (rte_get_tsc_hz() + US_PER_S - 1) /@@ -158,6 +158,8 @@ lpm_main_loop(__rte_unused void *dummy)const uint16_t n_rx_q = qconf->n_rx_queue;const uint16_t n_tx_p = qconf->n_tx_port; +int direct_rearm_map[n_rx_q]; +if (n_rx_q ==0 ) {RTE_LOG (INFO ,L3FWD ,"lcore %u has nothing to do\n" , lcore_id);return 0 ;@@ -169,6 +171,7 @@ lpm_main_loop(__rte_unused void *dummy) portid = qconf->rx_queue_list[i].port_id; queueid = qconf->rx_queue_list[i].queue_id; + direct_rearm_map[i] =0 ;RTE_LOG (INFO ,L3FWD ," -- lcoreid=%u portid=%u rxqueueid=%hhu\n" , lcore_id, portid, queueid);@@ -209,6 +212,17 @@ lpm_main_loop(__rte_unused void *dummy)if (nb_rx ==0 )continue ; +/* Determine the direct rearm mapping based on the first
examples/l3fwd/l3fwd_lpm.c lpm_main_loop(__rte_unused void *dummy) · line 225
Quoted 10 lines
examples/l3fwd/l3fwd_lpm.c +10
-0+* packet received on the rx queue +*/ +if (direct_rearm_map[i] ==0 ) { + tx_portid =lpm_get_dst_port (qconf, pkts_burst[0 ], + portid); +rte_eth_direct_rxrearm_map (portid, queueid, + tx_portid, queueid); + direct_rearm_map[i] =1 ; + } +
That just doesn't look right to me: why to make decision based on the first packet? What would happen if second and all other packets have to be routed
to different ports?
In fact, this direct-rearm mode seems suitable only for hard-coded
one to one mapped forwarding (examples/l2fwd, testpmd).
For l3fwd it can be used safely only when we have one port in use. Also I think it should be selected at init-time and
it shouldn't be on by default.
To summarize, my opinion:
special cmd-line parameter to enable it.
allowable only when we run l3fwd over one port.
Quoted 3 lines
examples/l3fwd/l3fwd_lpm.c +0
-0#if definedRTE_ARCH_X86 || defined __ARM_NEON \ || definedRTE_ARCH_PPC_64 l3fwd_lpm_send_packets(nb_rx, pkts_burst,
Re: [PATCH v1 5/5] examples/l3fwd: enable direct rearm mode
25/05/2022 01:24, Honnappa Nagarahalli пишет:
Quoted 77 lines
From: Konstantin Ananyev <[email protected]>
20/04/2022 09:16, Feifei Wang пишет:
Quoted 47 lines
Quoted 47 lines
Enable direct rearm mode. The mapping is decided in the data plane based on the first packet received.
Suggested-by Honnappa Nagarahalli <[email protected]>Signed-off-by Feifei Wang <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>Reviewed-by Honnappa Nagarahalli <[email protected]>---
examples/l3fwd/l3fwd_lpm.c| 16 +++++++++++++++-1 file changed, 15 insertions(+), 1 deletion(-)examples/l3fwd/l3fwd_lpm.c +1
-1@@ -147,7 +147,7 @@ lpm_main_loop(__rte_unused void *dummy) unsigned lcore_id; uint64_t prev_tsc, diff_tsc, cur_tsc; int i, nb_rx; - uint16_t portid; + uint16_t portid, tx_portid; uint8_t queueid; struct lcore_conf *qconf;const
uint64_tdrain_tsc= (rte_get_tsc_hz()+US_PER_S- 1) /examples/l3fwd/l3fwd_lpm.c +8
-0@@ -158,6 +158,8 @@ lpm_main_loop(__rte_unused void *dummy) const uint16_t n_rx_q = qconf->n_rx_queue; const uint16_t n_tx_p = qconf->n_tx_port; + int direct_rearm_map[n_rx_q]; + if (n_rx_q == 0) { RTE_LOG(INFO, L3FWD, "lcore %u has nothing to do\n", lcore_id); return 0;@@ -169,6 +171,7 @@ lpm_main_loop(__rte_unused void *dummy) portid = qconf->rx_queue_list[i].port_id; queueid = qconf->rx_queue_list[i].queue_id; + direct_rearm_map[i] = 0; RTE_LOG(INFO, L3FWD, " -- lcoreid=%u portid=%u rxqueueid=%hhu\n", lcore_id, portid, queueid);@@ -209,6 +212,17 @@ lpm_main_loop(__rte_unused void *dummy) if (nb_rx == 0) continue; + /* Determine the direct rearm mapping based on the first + * packet received on the rx queue + */ + if (direct_rearm_map[i] == 0) { + tx_portid = lpm_get_dst_port(qconf, pkts_burst[0],Quoted 8 lines
examples/l3fwd/l3fwd_lpm.c lpm_main_loop(__rte_unused void *dummy) · line 225
Quoted 6 lines
examples/l3fwd/l3fwd_lpm.c +6
-0+ portid); + rte_eth_direct_rxrearm_map(portid, queueid, + tx_portid, queueid); + direct_rearm_map[i] = 1; + } +That just doesn't look right to me: why to make decision based on the first packet?
The TX queue depends on the incoming packet. So, this method covers more scenarios than doing it in the control plane where the outgoing queue is not known.
Quoted 2 lines
What would happen if second and all other packets have to be routed to different ports?
This is an example application and it should be fine to make this assumption. More over, it does not cause any problems if packets change in between. When the packets change back, the feature works again.
Quoted 3 lines
In fact, this direct-rearm mode seems suitable only for hard-coded one to one mapped forwarding (examples/l2fwd, testpmd). For l3fwd it can be used safely only when we have one port in use.
Can you elaborate more on the safety issue when more than one port is used?
Quoted 4 lines
Also I think it should be selected at init-time and it shouldn't be on by default.
To summarize, my opinion:
special cmd-line parameter to enable it.
Quoted 6 lines
Can you please elaborate why a command line parameter is required? Other similar features like
RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREEare enabled without a command line parameter. IMO, this is how it should ber. Essentially we are trying to measure how different PMDs perform, the ones that have implemented performance improvement features would show better performance (i.e. the PMDs implementing the features should not be penalized by asking for additional user input).
From my perspective, main purpose of l3fwd application is to demonstrate DPDK ability to do packet routing based on input packet contents. Making guesses about packet contents is a change in expected behavior. For some cases it might improve performance, for many others - will most likely cause performance drop.
I think that performance drop as default behavior
(running the same parameters as before) should not be allowed. Plus you did not provided ability to switch off that behavior, if undesired.
About comparison with RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE default enablement - I don't think it is correct. Within l3fwd app we can safely guarantee that all RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE pre-requirements are met:
in each TX queue all mbufs will belong to the same mempool and their refcnt will always equal to one.
Here you are making guesses about contents of input packets, without any guarantee that you guess will always be valid.
BTW, what's wrong with using l2fwd to demonstrate that feature? Seems like a natural choice to me.
Quoted 6 lines
Quoted 6 lines
allowable only when we run l3fwd over one port.
Quoted 3 lines
examples/l3fwd/l3fwd_lpm.c +0
-0#if defined RTE_ARCH_X86 || defined __ARM_NEON \|| defined
RTE_ARCH_PPC_64l3fwd_lpm_send_packets(nb_rx,pkts_burst,
RE: [PATCH v1 5/5] examples/l3fwd: enable direct rearm mode
<snip>
Quoted 95 lines
25/05/2022 01:24, Honnappa Nagarahalli пишет:
Quoted 74 lines
From: Konstantin Ananyev <[email protected]>
20/04/2022 09:16, Feifei Wang пишет:
Quoted 50 lines
Quoted 50 lines
Enable direct rearm mode. The mapping is decided in the data plane based on the first packet received.
Suggested-by Honnappa Nagarahalli <[email protected]>Signed-off-by Feifei Wang <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>Reviewed-by Honnappa Nagarahalli <[email protected]>---
examples/l3fwd/l3fwd_lpm.c| 16 +++++++++++++++-1 file changed, 15 insertions(+), 1 deletion(-)examples/l3fwd/l3fwd_lpm.c +1
-1@@ -147,7 +147,7 @@ lpm_main_loop(__rte_unused void *dummy) unsigned lcore_id; uint64_t prev_tsc, diff_tsc, cur_tsc; int i, nb_rx; - uint16_t portid; + uint16_t portid, tx_portid; uint8_t queueid; struct lcore_conf *qconf;const
uint64_tdrain_tsc= (rte_get_tsc_hz()+US_PER_S- 1) /examples/l3fwd/l3fwd_lpm.c +3
-0@@ -158,6 +158,8 @@ lpm_main_loop(__rte_unused void *dummy) const uint16_t n_rx_q = qconf->n_rx_queue; const uint16_t n_tx_p = qconf->n_tx_port; + int direct_rearm_map[n_rx_q]; + if (n_rx_q == 0) { RTE_LOG(INFO, L3FWD, "lcore %u has nothing to do\n", lcore_id); return 0;@@ -169,6 +171,7 @@ lpm_main_loop(__rte_unused void *dummy) portid = qconf->rx_queue_list[i].port_id; queueid = qconf->rx_queue_list[i].queue_id; + direct_rearm_map[i] = 0; RTE_LOG(INFO, L3FWD, " -- lcoreid=%u portid=%u rxqueueid=%hhu\n",
lcore_id, portid, queueid); @@ -209,6 +212,17 @@lpm_main_loop(__rte_unused void *dummy)
if (nb_rx== 0)
continue;
+ /* Determine the direct rearm mapping based on the +first
+ * packet received on the rx queue
+ */
+ if (direct_rearm_map[i] == 0) {
+tx_portid=lpm_get_dst_port(qconf, pkts_burst[0], + portid);Quoted 8 lines
Quoted 5 lines
+ rte_eth_direct_rxrearm_map(portid, queueid, +
tx_portid, queueid); + direct_rearm_map[i] = 1;
+ }
+That just doesn't look right to me: why to make decision based on the first packet?
The TX queue depends on the incoming packet. So, this method covers more scenarios than doing it in the control plane where the outgoing queue is not known.
Quoted 2 lines
What would happen if second and all other packets have to be routed to different ports?
This is an example application and it should be fine to make this assumption.
More over, it does not cause any problems if packets change in between. When
the packets change back, the feature works again.Quoted 4 lines
Quoted 3 lines
In fact, this direct-rearm mode seems suitable only for hard-coded one to one mapped forwarding (examples/l2fwd, testpmd). For l3fwd it can be used safely only when we have one port in use.
Can you elaborate more on the safety issue when more than one port is
used?
Quoted 10 lines
Quoted 5 lines
Also I think it should be selected at init-time and it shouldn't be on by default.
To summarize, my opinion:
special cmd-line parameter to enable it.Can you please elaborate why a command line parameter is required? Other similar features like
RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREEare enabled without a command line parameter. IMO, this is how it should ber. Essentially we are trying to measure how different PMDs perform, the ones that have implemented performance improvement featureswould
Quoted 2 lines
show better performance (i.e. the PMDs implementing the features should not be penalized by asking for additional user input).
Quoted 8 lines
From my perspective, main purpose of l3fwd application is to demonstrate
DPDK ability to do packet routing based on input packet contents. Making guesses about packet contents is a change in expected behavior. For some cases it might improve performance, for many others - will most likely cause performance drop.
I think that performance drop as default behavior (running the same parameters as before) should not be allowed.
Plus you did not provided ability to switch off that behavior, if undesired.
There is no drop in L3fwd performance due to this patch.
Quoted 9 lines
About comparison with
RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREEdefault enablement - I don't think it is correct.
Within l3fwd app we can safely guarantee that allRTE_ETH_TX_OFFLOAD_MBUF_FAST_FREEpre-requirements are met:
in each TX queue all mbufs will belong to the same mempool and their refcnt will always equal to one.
Here you are making guesses about contents of input packets, without any guarantee that you guess will always be valid.
This is not a guess. The code understands the incoming traffic and configures accordingly. So, it should be correct. Since it is a sample application, we do not expect the traffic to be complex. If it is complex, the performance will be the same as before or better.
Quoted 3 lines
BTW, what's wrong with using l2fwd to demonstrate that feature? Seems like a natural choice to me.
The performance of L3fwd application in DPDK has become a industry standard, hence we need to showcase the performance in L3fwd application.
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
allowable only when we run l3fwd over one port.
Quoted 3 lines
#if defined
RTE_ARCH_X86|| defined__ARM_NEON\
|| definedRTE_ARCH_PPC_64l3fwd_lpm_send_packets(nb_rx,pkts_burst,
Re: [PATCH v1 5/5] examples/l3fwd: enable direct rearm mode
Quoted 104 lines · Honnappa Nagarahalli
<snip>
Quoted 95 lines
25/05/2022 01:24, Honnappa Nagarahalli пишет:
Quoted 74 lines
From: Konstantin Ananyev <[email protected]>
20/04/2022 09:16, Feifei Wang пишет:
Quoted 50 lines
Quoted 50 lines
Enable direct rearm mode. The mapping is decided in the data plane based on the first packet received.
Suggested-by Honnappa Nagarahalli <[email protected]>Signed-off-by Feifei Wang <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>Reviewed-by Honnappa Nagarahalli <[email protected]>--- examples/l3fwd/l3fwd_lpm.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-)examples/l3fwd/l3fwd_lpm.c +4
-1@@ -147,7 +147,7 @@ lpm_main_loop(__rte_unused void *dummy) unsigned lcore_id; uint64_t prev_tsc, diff_tsc, cur_tsc; int i, nb_rx; - uint16_t portid; + uint16_t portid, tx_portid; uint8_t queueid; struct lcore_conf *qconf; const uint64_t drain_tsc = (rte_get_tsc_hz() + US_PER_S - 1) /@@ -158,6 +158,8 @@ lpm_main_loop(__rte_unused void *dummy) const uint16_t n_rx_q = qconf->n_rx_queue; const uint16_t n_tx_p = qconf->n_tx_port; + int direct_rearm_map[n_rx_q]; + if (n_rx_q == 0) { RTE_LOG(INFO, L3FWD, "lcore %u has nothing to do\n", lcore_id); return 0;@@ -169,6 +171,7 @@ lpm_main_loop(__rte_unused void *dummy) portid = qconf->rx_queue_list[i].port_id; queueid = qconf->rx_queue_list[i].queue_id; + direct_rearm_map[i] = 0; RTE_LOG(INFO, L3FWD, " -- lcoreid=%u portid=%u rxqueueid=%hhu\n", lcore_id, portid, queueid); @@ -209,6 +212,17 @@ lpm_main_loop(__rte_unused void *dummy)if (
nb_rx== 0)
continue;
+ /* Determine the direct rearm mapping based on the +first
+ * packet received on the rx queue
+ */
+ if (direct_rearm_map[i] == 0) {
+tx_portid=lpm_get_dst_port(qconf, pkts_burst[0], + portid);Quoted 8 lines
Quoted 5 lines
+ rte_eth_direct_rxrearm_map(portid, queueid, +
tx_portid, queueid); + direct_rearm_map[i] = 1;
+ }
+That just doesn't look right to me: why to make decision based on the first packet?
The TX queue depends on the incoming packet. So, this method covers more scenarios than doing it in the control plane where the outgoing queue is not known.
Quoted 2 lines
What would happen if second and all other packets have to be routed to different ports?
This is an example application and it should be fine to make this assumption.
More over, it does not cause any problems if packets change in between. When
the packets change back, the feature works again.Quoted 4 lines
Quoted 3 lines
In fact, this direct-rearm mode seems suitable only for hard-coded one to one mapped forwarding (examples/l2fwd, testpmd). For l3fwd it can be used safely only when we have one port in use.
Can you elaborate more on the safety issue when more than one port is
used?
Quoted 10 lines
Quoted 5 lines
Also I think it should be selected at init-time and it shouldn't be on by default.
To summarize, my opinion:
special cmd-line parameter to enable it.Can you please elaborate why a command line parameter is required? Other similar features like
RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREEare enabled without a command line parameter. IMO, this is how it should ber. Essentially we are trying to measure how different PMDs perform, the ones that have implemented performance improvement featureswould
Quoted 2 lines
show better performance (i.e. the PMDs implementing the features should not be penalized by asking for additional user input).
Quoted 8 lines
From my perspective, main purpose of l3fwd application is to demonstrate
DPDK ability to do packet routing based on input packet contents. Making guesses about packet contents is a change in expected behavior. For some cases it might improve performance, for many others - will most likely cause performance drop.
I think that performance drop as default behavior (running the same parameters as before) should not be allowed.
Plus you did not provided ability to switch off that behavior, if undesired.
Quoted 1 line
There is no drop in L3fwd performance due to this patch.
It is a questionable statement. Sorry. The patch adds code on data path. So, I'm almost confident that it is possible to find a case when performance slightly drops. May be it is always minor and acceptable.
Quoted 24 lines
Quoted 9 lines
About comparison with
RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREEdefault enablement - I don't think it is correct.
Within l3fwd app we can safely guarantee that allRTE_ETH_TX_OFFLOAD_MBUF_FAST_FREEpre-requirements are met:
in each TX queue all mbufs will belong to the same mempool and their refcnt will always equal to one.
Here you are making guesses about contents of input packets, without any guarantee that you guess will always be valid.This is not a guess. The code understands the incoming traffic and configures accordingly. So, it should be correct. Since it is a sample application, we do not expect the traffic to be complex. If it is complex, the performance will be the same as before or better.
Quoted 3 lines
BTW, what's wrong with using l2fwd to demonstrate that feature? Seems like a natural choice to me.
The performance of L3fwd application in DPDK has become a industry standard, hence we need to showcase the performance in L3fwd application.
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
allowable only when we run l3fwd over one port.
Quoted 3 lines
#if defined
RTE_ARCH_X86|| defined__ARM_NEON\
|| definedRTE_ARCH_PPC_64l3fwd_lpm_send_packets(nb_rx,pkts_burst,
Re: [PATCH v1 5/5] examples/l3fwd: enable direct rearm mode
In reply to Honnappa Nagarahalli31/05/2022 18:14, Honnappa Nagarahalli пишет:
Quoted 104 lines
<snip>
Quoted 95 lines
25/05/2022 01:24, Honnappa Nagarahalli пишет:
Quoted 74 lines
From: Konstantin Ananyev <[email protected]>
20/04/2022 09:16, Feifei Wang пишет:
Quoted 50 lines
Quoted 50 lines
Enable direct rearm mode. The mapping is decided in the data plane based on the first packet received.
Suggested-by Honnappa Nagarahalli <[email protected]>Signed-off-by Feifei Wang <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>Reviewed-by Honnappa Nagarahalli <[email protected]>--- examples/l3fwd/l3fwd_lpm.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-)examples/l3fwd/l3fwd_lpm.c +4
-1@@ -147,7 +147,7 @@ lpm_main_loop(__rte_unused void *dummy) unsigned lcore_id; uint64_t prev_tsc, diff_tsc, cur_tsc; int i, nb_rx; - uint16_t portid; + uint16_t portid, tx_portid; uint8_t queueid; struct lcore_conf *qconf; const uint64_t drain_tsc = (rte_get_tsc_hz() + US_PER_S - 1) /@@ -158,6 +158,8 @@ lpm_main_loop(__rte_unused void *dummy) const uint16_t n_rx_q = qconf->n_rx_queue; const uint16_t n_tx_p = qconf->n_tx_port; + int direct_rearm_map[n_rx_q]; + if (n_rx_q == 0) { RTE_LOG(INFO, L3FWD, "lcore %u has nothing to do\n", lcore_id); return 0;@@ -169,6 +171,7 @@ lpm_main_loop(__rte_unused void *dummy) portid = qconf->rx_queue_list[i].port_id; queueid = qconf->rx_queue_list[i].queue_id; + direct_rearm_map[i] = 0; RTE_LOG(INFO, L3FWD, " -- lcoreid=%u portid=%u rxqueueid=%hhu\n", lcore_id, portid, queueid); @@ -209,6 +212,17 @@ lpm_main_loop(__rte_unused void *dummy)if (
nb_rx== 0)
continue;
+ /* Determine the direct rearm mapping based on the +first
+ * packet received on the rx queue
+ */
+ if (direct_rearm_map[i] == 0) {
+tx_portid=lpm_get_dst_port(qconf, pkts_burst[0], + portid);Quoted 8 lines
Quoted 5 lines
+ rte_eth_direct_rxrearm_map(portid, queueid, +
tx_portid, queueid); + direct_rearm_map[i] = 1;
+ }
+That just doesn't look right to me: why to make decision based on the first packet?
The TX queue depends on the incoming packet. So, this method covers more scenarios than doing it in the control plane where the outgoing queue is not known.
Quoted 2 lines
What would happen if second and all other packets have to be routed to different ports?
This is an example application and it should be fine to make this assumption.
More over, it does not cause any problems if packets change in between. When
the packets change back, the feature works again.Quoted 4 lines
Quoted 3 lines
In fact, this direct-rearm mode seems suitable only for hard-coded one to one mapped forwarding (examples/l2fwd, testpmd). For l3fwd it can be used safely only when we have one port in use.
Can you elaborate more on the safety issue when more than one port is
used?
Quoted 10 lines
Quoted 5 lines
Also I think it should be selected at init-time and it shouldn't be on by default.
To summarize, my opinion:
special cmd-line parameter to enable it.Can you please elaborate why a command line parameter is required? Other similar features like
RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREEare enabled without a command line parameter. IMO, this is how it should ber. Essentially we are trying to measure how different PMDs perform, the ones that have implemented performance improvement featureswould
Quoted 2 lines
show better performance (i.e. the PMDs implementing the features should not be penalized by asking for additional user input).
Quoted 8 lines
From my perspective, main purpose of l3fwd application is to demonstrate
DPDK ability to do packet routing based on input packet contents. Making guesses about packet contents is a change in expected behavior. For some cases it might improve performance, for many others - will most likely cause performance drop.
I think that performance drop as default behavior (running the same parameters as before) should not be allowed.
Plus you did not provided ability to switch off that behavior, if undesired.
Quoted 1 line
There is no drop in L3fwd performance due to this patch.
Hmm..
Are you saying even when your guess is wrong, and you constantly hitting slow-path (check tx_queue first - failure, then allocate from mempool) you didn't observe any performance drop?
There is more work to do, and if workload is cpu-bound,
my guess - it should be noticeable.
Also, from previous experience, quite often even after
tiny changes in rx/tx code-path some slowdown was reported. Usually that happened on some low-end ARM cpus (Marvell, NXP).
Quoted 10 lines
Quoted 9 lines
About comparison with
RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREEdefault enablement - I don't think it is correct.
Within l3fwd app we can safely guarantee that allRTE_ETH_TX_OFFLOAD_MBUF_FAST_FREEpre-requirements are met:
in each TX queue all mbufs will belong to the same mempool and their refcnt will always equal to one.
Here you are making guesses about contents of input packets, without any guarantee that you guess will always be valid.This is not a guess. The code understands the incoming traffic and configures accordingly. So, it should be correct.
No, it is not.
Your code makes guess about all incoming traffic
based on just one (first) packet.
Since it is a sample application, we do not expect the traffic to be complex. If it is complex, the performance will be the same as before or better.
I am not talking about anything 'complex' here. Doing dynamic routing based on packet contents is a a basic l3fwd functionality.
Quoted 14 lines
Quoted 4 lines
BTW, what's wrong with using l2fwd to demonstrate that feature? Seems like a natural choice to me.
The performance of L3fwd application in DPDK has become a industry standard, hence we need to showcase the performance in L3fwd application.
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
allowable only when we run l3fwd over one port.
Quoted 3 lines
#if defined
RTE_ARCH_X86|| defined__ARM_NEON\
|| definedRTE_ARCH_PPC_64l3fwd_lpm_send_packets(nb_rx,pkts_burst,
RE: [PATCH v1 5/5] examples/l3fwd: enable direct rearm mode
(apologies for being late on replying, catching up after vacation)
<snip>
Quoted 109 lines
Quoted 9 lines
Quoted 9 lines
25/05/2022 01:24, Honnappa Nagarahalli пишет:
Quoted 7 lines
From: Konstantin Ananyev <[email protected]>
20/04/2022 09:16, Feifei Wang пишет:
Quoted 4 lines
Quoted 4 lines
Enable direct rearm mode. The mapping is decided in the data plane based on the first packet received.
Suggested-by Honnappa NagarahalliQuoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Signed-off-by Feifei Wang <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>Reviewed-by Honnappa Nagarahalli <[email protected]>--- examples/l3fwd/l3fwd_lpm.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-)Quoted 14 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
(quoted diff) +0
-0
b/examples/l3fwd/l3fwd_lpm.cindexbec22c44cd..38ffdf4636100644examples/l3fwd/l3fwd_lpm.c +0
-0@@ -147,7 +147,7 @@ lpm_main_loop(__rte_unused void *dummy) unsigned lcore_id;Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
examples/l3fwd/l3fwd_lpm.c +1
-1uint64_t prev_tsc, diff_tsc, cur_tsc; int i, nb_rx; - uint16_t portid; + uint16_t portid, tx_portid; uint8_t queueid; struct lcore_conf *qconf; const uint64_t drain_tsc = (rte_get_tsc_hz() + US_PER_S -1) / @@ -158,6 +158,8 @@ lpm_main_loop(__rte_unused void
*dummy)
Quoted 64 lines
Quoted 64 lines
Quoted 47 lines
Quoted 26 lines
Quoted 26 lines
const
uint16_tn_rx_q=qconf->n_rx_queue; constuint16_tn_tx_p=qconf->n_tx_port;
+ int direct_rearm_map[n_rx_q];
+
if (n_rx_q== 0) {
RTE_LOG(INFO, L3FWD, "lcore %u has nothing to do\n",lcore_id);
return 0;examples/l3fwd/l3fwd_lpm.c +1
-0@@ -169,6 +171,7 @@ lpm_main_loop(__rte_unused void *dummy) portid = qconf->rx_queue_list[i].port_id; queueid = qconf->rx_queue_list[i].queue_id; + direct_rearm_map[i] = 0; RTE_LOG(INFO, L3FWD, " -- lcoreid=%u portid=%u rxqueueid=%hhu\n", lcore_id, portid, queueid); @@ -209,6 +212,17 @@ lpm_main_loop(__rte_unused void *dummy)if (
nb_rx== 0)
continue;
+ /* Determine the direct rearm mapping based on theexamples/l3fwd/l3fwd_lpm.c +7
-0+first +* packet received on the rx queue +*/ + if (direct_rearm_map[i] == 0) { + tx_portid = lpm_get_dst_port(qconf, +pkts_burst[0 ], + portid);Quoted 8 lines
Quoted 5 lines
+ rte_eth_direct_rxrearm_map(portid, queueid, +
tx_portid, queueid); + direct_rearm_map[i] = 1;
+ }
+That just doesn't look right to me: why to make decision based on the first packet?
The TX queue depends on the incoming packet. So, this method covers more scenarios than doing it in the control plane where the outgoing queue is not known.
Quoted 2 lines
What would happen if second and all other packets have to be routed to different ports?
This is an example application and it should be fine to make this assumption.
More over, it does not cause any problems if packets change in between. When
the packets change back, the feature works again.Quoted 5 lines
Quoted 3 lines
In fact, this direct-rearm mode seems suitable only for hard-coded one to one mapped forwarding (examples/l2fwd, testpmd). For l3fwd it can be used safely only when we have one port in use.
Can you elaborate more on the safety issue when more than one port is
used?
Quoted 11 lines
Quoted 5 lines
Also I think it should be selected at init-time and it shouldn't be on by default.
To summarize, my opinion:
special cmd-line parameter to enable it.Can you please elaborate why a command line parameter is required? Other similar features like
RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREEare enabled without a command line parameter. IMO, this is how it should ber. Essentially we are trying to measure how different PMDs perform, the ones that have implemented performance improvement featuresQuoted 6 lines
Quoted 6 lines
would
Quoted 2 lines
show better performance (i.e. the PMDs implementing the features should not be penalized by asking for additional user input).
From my perspective, main purpose of l3fwd application is to
demonstrate DPDK ability to do packet routing based on input packetcontents.
Quoted 6 lines
Quoted 6 lines
Making guesses about packet contents is a change in expected behavior. For some cases it might improve performance, for many others - will most likely cause performance drop.
I think that performance drop as default behavior (running the same parameters as before) should not be allowed.
Plus you did not provided ability to switch off that behavior, if undesired.
Quoted 8 lines
Quoted 1 line
There is no drop in L3fwd performance due to this patch.
Hmm..
Are you saying even when your guess is wrong, and you constantly hitting slow- path (checktx_queuefirst - failure, then allocate from mempool) you didn't observe any performance drop?
There is more work to do, and if workload is cpu-bound, my guess - it should be noticeable.
Well, you do not have to trust our words. The platform used to test is mentioned in the cover letter. Appreciate if you could test on your platform and provide the results.
Quoted 43 lines
Also, from previous experience, quite often even after tiny changes in rx/tx code-path some slowdown was reported.
Usually that happened on some low-end ARM cpus (Marvell, NXP).Quoted 10 lines
Quoted 9 lines
About comparison with
RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREEdefault enablement - I don't think it is correct.
Within l3fwd app we can safely guarantee that allRTE_ETH_TX_OFFLOAD_MBUF_FAST_FREEpre-requirements are met:
in each TX queue all mbufs will belong to the same mempool and their refcnt will always equal to one.
Here you are making guesses about contents of input packets, without any guarantee that you guess will always be valid.This is not a guess. The code understands the incoming traffic and configures
accordingly. So, it should be correct.
No, it is not.
Your code makes guess about all incoming traffic based on just one (first) packet.Since it is a sample application, we do not expect the traffic to be complex. If it is complex, the performance will be the same as before or better.
I am not talking about anything 'complex' here.
Doing dynamic routing based on packet contents is a a basic l3fwd functionality.Quoted 5 lines
Quoted 4 lines
BTW, what's wrong with using l2fwd to demonstrate that feature? Seems like a natural choice to me.
The performance of L3fwd application in DPDK has become a industry
standard, hence we need to showcase the performance in L3fwd application.
Quoted 9 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
allowable only when we run l3fwd over one port.
Quoted 3 lines
#if defined
RTE_ARCH_X86|| defined__ARM_NEON\
|| definedRTE_ARCH_PPC_64l3fwd_lpm_send_packets(nb_rx,pkts_burst,
Quoted 23 lines
Currently, the transmit side frees the buffers into the lcore cache and the receive side allocates buffers from the lcore cache. The transmit side typically frees 32 buffers resulting in 32*8=256B of stores to lcore cache. The receive side allocates 32 buffers and stores them in the receive side software ring, resulting in 32*8=256B of stores and 256B of load from the lcore cache.
This patch proposes a mechanism to avoid freeing to/allocating from the lcore cache. i.e. the receive side will free the buffers from transmit side directly into it's software ring. This will avoid the 256B of loads and stores introduced by the lcore cache. It also frees up the cache lines used by the lcore cache.
However, this solution poses several constraints:
1)The receive queue needs to know which transmit queue it should take the buffers from. The application logic decides which transmit port to use to send out the packets. In many use cases the NIC might have a single port ([1], [2], [3]), in which case a given transmit queue is always mapped to a single receive queue (1:1 Rx queue: Tx queue). This is easy to configure.
Quoted 5 lines
If the NIC has 2 ports (there are several references), then we will have 1:2 (RX queue: TX queue) mapping which is still easy to configure. However, if this is generalized to 'N' ports, the configuration can be long. More over the PMD would have to scan a list of transmit queues to pull the buffers from.
Just to re-iterate some generic concerns about this proposal:
- We effectively link RX and TX queues - when this feature is enabled, user can't stop TX queue without stopping linked RX queue first. Right now user is free to start/stop any queues at his will. If that feature will allow to link queues from different ports, then even ports will become dependent and user will have to pay extra care when managing such ports.
- very limited usage scenario - it will have a positive effect only when we have a fixed forwarding mapping: all (or nearly all) packets from the RX queue are forwarded into the same TX queue.
Wonder did you had a chance to consider mempool-cache ZC API,
similar to one we have for the ring?
It would allow us on TX free path to avoid copying mbufs to
temporary array on the stack.
Instead we can put them straight from TX SW ring to the mempool cache.
That should save extra store/load for mbuf and might help to achieve some performance gain without by-passing mempool. It probably wouldn't be as fast as what you proposing,
but might be fast enough to consider as alternative.
Again, it would be a generic one, so we can avoid all
these implications and limitations.
Quoted 92 lines
2)The other factor that needs to be considered is 'run-to-completion' vs 'pipeline' models. In the run-to-completion model, the receive side and the transmit side are running on the same lcore serially. In the pipeline model. The receive side and transmit side might be running on different lcores in parallel. This requires locking. This is not supported at this point.
3)Tx and Rx buffers must be from the same mempool. And we also must ensure Tx buffer free number is equal to Rx buffer free number:
(txq->tx_rs_thresh==RTE_I40E_RXQ_REARM_THRESH)
Thus, 'tx_next_dd' can be updated correctly in direct-rearm mode. This is due totx_next_ddis a variable to compute tx sw-ring free location. Its value will be one more round than the position where next time free starts.Current status in this RFC:
1)An API is added to allow for mapping a TX queue to a RX queue.
Currently it supports 1:1 mapping.
2)The i40e driver is changed to do the direct re-arm of the receive
side.
3)L3fwd application is modified to do the direct rearm mapping automatically without user config. This follows the rules that the thread can map TX queue to a RX queue based on the first received package destination port.Testing status:
1.The testing results for L3fwd are as follows:
-------------------------------------------------------------------
enabled direct rearm
-------------------------------------------------------------------
Arm:
N1SDP(neon path):
without fast-free mode with fast-free mode+14.1% +7.0%Ampere Altra(neon path):
without fast-free mode with fast-free mode+17.1 +14.0%X86:
Dell-8268(limit frequency):
sse path:
without fast-free mode with fast-free mode+6.96% +2.02% avx2 path: without fast-free mode with fast-free mode +9.04% +7.75% avx512 path: without fast-free mode with fast-free mode +5.43% +1.57%-------------------------------------------------------------------
This patch can not affect base performance of normal mode. Furthermore, the reason for that limiting the CPU frequency is that dell-8268 can encounter i40e NIC bottleneck with maximum frequency.2.The testing results for VPP-L3fwd are as follows:
-------------------------------------------------------------------
Arm:
N1SDP(neon path):
with direct re-arm mode enabled+7.0%-------------------------------------------------------------------
For Ampere Altra and X86,VPP-L3fwd test has not been done.Reference:
[1] https://store.nvidia.com/en-us/networking/store/product/MCX623105AN-CDAT/NVIDIAMCX623105ANCDATConnectX6DxENAdapterCard100GbECryptoDisabled/
[2] https://www.intel.com/content/www/us/en/products/sku/192561/intel-ethernet-network-adapter-e810cqda1/specifications.html
[3] https://www.broadcom.com/products/ethernet-connectivity/network-adapters/100gb-nic-ocp/n1100gFeifei Wang (5):
net/i40e: remove redundant Dtype initialization net/i40e: enable direct rearm mode
ethdev: add API for direct rearm mode net/i40e: add direct rearm mode internal API examples/l3fwd: enable direct rearm modedrivers/net/i40e/i40e_ethdev.c | 34 +++ drivers/net/i40e/i40e_rxtx.c | 4 - drivers/net/i40e/i40e_rxtx.h | 4 + drivers/net/i40e/i40e_rxtx_common_avx.h | 269 ++++++++++++++++++++++++ drivers/net/i40e/i40e_rxtx_vec_avx2.c | 14 +- drivers/net/i40e/i40e_rxtx_vec_avx512.c | 249 +++++++++++++++++++++- drivers/net/i40e/i40e_rxtx_vec_neon.c | 141 ++++++++++++- drivers/net/i40e/i40e_rxtx_vec_sse.c | 170 ++++++++++++++- examples/l3fwd/l3fwd_lpm.c | 16 +- lib/ethdev/ethdev_driver.h | 15 ++ lib/ethdev/rte_ethdev.c | 14 ++ lib/ethdev/rte_ethdev.h | 31 +++ lib/ethdev/version.map | 1 + 13 files changed, 949 insertions(+), 13 deletions(-)
A message that is not in the archive (sent off-list or not yet received): [email protected]
16/05/2022 07:10, Feifei Wang пишет:
Quoted 37 lines
Quoted 29 lines
Quoted 23 lines
Currently, the transmit side frees the buffers into the lcore cache and the receive side allocates buffers from the lcore cache. The transmit side typically frees 32 buffers resulting in 32*8=256B of stores to lcore cache. The receive side allocates 32 buffers and stores them in the receive side software ring, resulting in 32*8=256B of stores and 256B of load from the lcore cache.
This patch proposes a mechanism to avoid freeing to/allocating from the lcore cache. i.e. the receive side will free the buffers from transmit side directly into it's software ring. This will avoid the 256B of loads and stores introduced by the lcore cache. It also frees up the cache lines used by the lcore cache.
However, this solution poses several constraints:
1)The receive queue needs to know which transmit queue it should take the buffers from. The application logic decides which transmit port to use to send out the packets. In many use cases the NIC might have a single port ([1], [2], [3]), in which case a given transmit queue is always mapped to a single receive queue (1:1 Rx queue: Tx queue). This is easy to configure.
Quoted 5 lines
If the NIC has 2 ports (there are several references), then we will have 1:2 (RX queue: TX queue) mapping which is still easy to configure. However, if this is generalized to 'N' ports, the configuration can be long. More over the PMD would have to scan a list of transmit queues to pull the buffers from.
Quoted 7 lines
Just to re-iterate some generic concerns about this proposal:
- We effectively link RX and TX queues - when this feature is enabled, user can't stop TX queue without stopping linked RX queue first. Right now user is free to start/stop any queues at his will. If that feature will allow to link queues from different ports, then even ports will become dependent and user will have to pay extra care when managing such ports.
Quoted 4 lines
[Feifei] When direct rearm enabled, there are two path for thread to choose. If there are enough Tx freed buffers, Rx can put buffers from Tx. Otherwise, Rx will put buffers from mempool as usual. Thus, users do not need to pay much attention managing ports.
What I am talking about: right now different port or different queues of
the same port can be treated as independent entities:
in general user is free to start/stop (and even reconfigure in some cases) one entity without need to stop other entity. I.E user can stop and re-configure TX queue while keep receiving packets
from RX queue.
With direct re-arm enabled, I think it wouldn't be possible any more:
before stopping/reconfiguring TX queue user would have make sure that
corresponding RX queue wouldn't be used by datapath.
Quoted 7 lines
Quoted 4 lines
- very limited usage scenario - it will have a positive effect only when we have a fixed forwarding mapping: all (or nearly all) packets from the RX queue are forwarded into the same TX queue.
[Feifei] Although the usage scenario is limited, this usage scenario has a wide range of applications, such as NIC with one port.
yes, there are NICs with one port, but no guarantee there wouldn't be several such NICs within the system.
Quoted 16 lines
Furtrhermore, I think this is a tradeoff between performance and flexibility. Our goal is to achieve best performance, this means we need to give up some flexibility decisively. For example of '
FAST_FREEMode', it deletes most of the buffer check (refcnt > 1, external buffer, chain buffer), chooses a shorest path, and then achieve significant performance improvement.Quoted 10 lines
Wonder did you had a chance to consider mempool-cache ZC API,
similar to one we have for the ring?
It would allow us on TX free path to avoid copying mbufs to
temporary array on the stack.
Instead we can put them straight from TX SW ring to the mempool cache.
That should save extra store/load for mbuf and might help to achieve some performance gain without by-passing mempool. It probably wouldn't be as fast as what you proposing,
but might be fast enough to consider as alternative.
Again, it would be a generic one, so we can avoid all
these implications and limitations.
Quoted 3 lines
[Feifei] I think this is a good try. However, the most important thing is that if we can bypass the mempool decisively to pursue the significant performance gains.
I understand the intention, and I personally think this is wrong
and dangerous attitude.
We have mempool abstraction in place for very good reason.
So we need to try to improve mempool performance (and API if necessary) at first place, not to avoid it and break our own rules and recommendations.
Quoted 5 lines
For ZC, there maybe a problem for it in i40e. The reason for that put Tx buffers into temporary is that
i40e_tx_entryincludes buffer pointer and index. Thus we cannot put TxSW_ringentry into mempool directly, we need to firstlt extract mbuf pointer. Finally, though we use ZC, we still can't avoid using a temporary stack to extract Tx buffer pointers.
When talking about ZC API for mempool cache I meant something like:
void ** mempool_cache_put_zc_start(struct rte_mempool_cache *mc, uint32_t *nb_elem, uint32_t flags);
void mempool_cache_put_zc_finish(struct rte_mempool_cache *mc, uint32_t nb_elem);
i.e. _start_ will return user a pointer inside mp-cache where to put free elems and max number of slots that can be safely filled. _finish_ will update mc->len.
As an example:
/* expect to free N mbufs */uint32_t n = N;
void **p = mempool_cache_put_zc_start(mc, &n, ...);
/* free up to n elems */
for (i = 0; i != n; i++) {
/* get next free mbuf from somewhere */
mb = extract_and_prefree_mbuf(...);
/* no more free mbufs for now */
if (mb == NULL)
break;
p[i] = mb;
}
/* finalize ZC put, with _i_ freed elems */mempool_cache_put_zc_finish(mc, i);
That way, I think we can overcome the issue with i40e_tx_entry you mentioned above. Plus it might be useful in other similar places.
Another alternative is obviously to split i40e_tx_entry into two structs (one for mbuf, second for its metadata) and have a separate array for each of them. Though with that approach we need to make sure no perf drops will be introduced, plus probably more code changes will be required.
Quoted 26 lines
From: Konstantin Ananyev [mailto:[email protected]]
Sent: Tuesday, 24 May 2022 03.26Quoted 3 lines
Furtrhermore, I think this is a tradeoff between performance and flexibility.
Our goal is to achieve best performance, this means we need to giveup some
Quoted 1 line
flexibility decisively. For example of '
FAST_FREEMode', it deletesmost
Quoted 1 line
of the buffer check (refcnt > 1, external buffer, chain buffer),
chooses a
Quoted 6 lines
shorest path, and then achieve significant performance improvement.
Quoted 5 lines
Wonder did you had a chance to consider mempool-cache ZC API, similar to one we have for the ring?
It would allow us on TX free path to avoid copying mbufs to temporary array on the stack.
Instead we can put them straight from TX SW ring to the mempoolcache.
Quoted 7 lines
Quoted 6 lines
That should save extra store/load for mbuf and might help to achieve some performance gain without by-passing mempool.
It probably wouldn't be as fast as what you proposing, but might be fast enough to consider as alternative. Again, it would be a generic one, so we can avoid all these implications and limitations.
Quoted 8 lines
Quoted 1 line
[Feifei] I think this is a good try. However, the most important
thing
Quoted 2 lines
is that if we can bypass the mempool decisively to pursue the significant performance gains.
I understand the intention, and I personally think this is wrong and dangerous attitude.
We have mempool abstraction in place for very good reason.
Yes, but the abstraction is being violated grossly elsewhere, and mempool code is copy-pasted elsewhere too.
A good example of the current situation is [1]. The cache multiplier (a definition private to the mempool library) is required for some copy-pasted code, and the solution is to expose the private definition and make it part of the public API.
The game of abstraction has already been lost. Performance won. :-(
Since we allow bypassing the mbuf/mempool library for other features, it should be allowed for this feature too.
I would even say: Why are the drivers using the mempool library, and not the mbuf library, when freeing and allocating mbufs? This behavior bypasses all the debug assertions in the mbuf library.
As you can probably see, I'm certainly not happy about the abstraction violations in DPDK. But they have been allowed for similar features, so they should be allowed here too.
Quoted 43 lines
So we need to try to improve mempool performance (and API if necessary) at first place, not to avoid it and break our own rules and recommendations.
Quoted 1 line
For ZC, there maybe a problem for it in i40e. The reason for that put
Tx
Quoted 2 lines
buffers
into temporary is thati40e_tx_entryincludes buffer pointer andindex.
Quoted 2 lines
Thus we cannot put Tx
SW_ringentry into mempool directly, we need to firstlt extract mbuf pointer. Finally, though we use ZC, we stillcan't
Quoted 2 lines
avoid
using a temporary stack to extract Tx buffer pointers.When talking about ZC API for mempool cache I meant something like:
void **mempool_cache_put_zc_start(struct rte_mempool_cache *mc, uint32_t *nb_elem, uint32_t flags);
voidmempool_cache_put_zc_finish(struct rte_mempool_cache *mc, uint32_t nb_elem);
i.e. _start_ will return user a pointer inside mp-cache where to put free elems and max number of slots that can be safely filled. _finish_ will updatemc->len.
As an example:/* expect to free N mbufs */
uint32_tn = N;
void **p =mempool_cache_put_zc_start(mc, &n, ...);/* free up to n elems */
for (i = 0; i != n; i++) {/* get next free mbuf from somewhere */ mb = extract_and_prefree_mbuf(...); /* no more free mbufs for now */ if (mb == NULL) break; p[i] = mb; }
Quoted 5 lines
/* finalize ZC put, with _i_ freed elems */
mempool_cache_put_zc_finish(mc, i);That way, I think we can overcome the issue with
i40e_tx_entryyou mentioned above. Plus it might be useful in other similar places.
Great example. This would fit perfectly into the i40e driver, if it didn't already implement the exact same by accessing the mempool cache structure directly. :-(
BTW: I tried patching the mempool library to fix some asymmetry bugs [2], but couldn't get any ACKs for it. It seems to me that the community is too risk averse to dare to modify such a core library.
[2] http://inbox.dpdk.org/dev/[email protected]/
However, adding your mempool ZC feature is not a modification, but an addition, so it should be able to gather support.
<snip>
Quoted 50 lines
[[email protected] appears similar to someone who previously sent you email, but may not be that person. Learn why this could be a risk at https://aka.ms/LearnAboutSenderIdentification.]
16/05/2022 07:10, Feifei Wang пишет:
Quoted 38 lines
Quoted 30 lines
Quoted 23 lines
Currently, the transmit side frees the buffers into the lcore cache and the receive side allocates buffers from the lcore cache. The transmit side typically frees 32 buffers resulting in 32*8=256B of stores to lcore cache. The receive side allocates 32 buffers and stores them in the receive side software ring, resulting in 32*8=256B of stores and 256B of load from the lcore cache.
This patch proposes a mechanism to avoid freeing to/allocating from the lcore cache. i.e. the receive side will free the buffers from transmit side directly into it's software ring. This will avoid the 256B of loads and stores introduced by the lcore cache. It also frees up the cache lines used by the lcore cache.
However, this solution poses several constraints:
1)The receive queue needs to know which transmit queue it should take the buffers from. The application logic decides which transmit port to use to send out the packets. In many use cases the NIC might have a single port ([1], [2], [3]), in which case a given transmit queue is always mapped to a single receive queue (1:1 Rx queue: Tx queue). This is easy to configure.
Quoted 6 lines
If the NIC has 2 ports (there are several references), then we will have
1:2 (RX queue: TX queue) mapping which is still easy to configure. However, if this is generalized to 'N' ports, the configuration can be long. More over the PMD would have to scan a list of transmit queues to pull the buffers from.Quoted 7 lines
Just to re-iterate some generic concerns about this proposal:
- We effectively link RX and TX queues - when this feature is enabled, user can't stop TX queue without stopping linked RX queue first. Right now user is free to start/stop any queues at his will. If that feature will allow to link queues from different ports, then even ports will become dependent and user will have to pay extra care when managing such ports.Quoted 5 lines
[Feifei] When direct rearm enabled, there are two path for thread to choose. If there are enough Tx freed buffers, Rx can put buffers from Tx.
Otherwise, Rx will put buffers from mempool as usual. Thus, users do not need to pay much attention managing ports.
Quoted 9 lines
What I am talking about: right now different port or different queues of the same port can be treated as independent entities:
in general user is free to start/stop (and even reconfigure in some cases) one entity without need to stop other entity.
I.E user can stop and re-configure TX queue while keep receiving packets from RX queue.
With direct re-arm enabled, I think it wouldn't be possible any more:
before stopping/reconfiguring TX queue user would have make sure that corresponding RX queue wouldn't be used by datapath.
I am trying to understand the problem better. For the TX queue to be stopped, the user must have blocked the data plane from accessing the TX queue. Like Feifei says, the RX side has the normal packet allocation path still available. Also this sounds like a corner case to me, we can handle this through checks in the queue_stop API.
Quoted 11 lines
Quoted 8 lines
Quoted 5 lines
- very limited usage scenario - it will have a positive effect only when we have a fixed forwarding mapping: all (or nearly all) packets from the RX queue are forwarded into the same TX queue.
[Feifei] Although the usage scenario is limited, this usage scenario has a wide range of applications, such as NIC with one port.
yes, there are NICs with one port, but no guarantee there wouldn't be several such NICs within the system.
What I see in my interactions is, a single NIC/DPU is under utilized for a 2 socket system. Some are adding more sockets to the system to better utilize the DPU. The NIC bandwidth continues to grow significantly. I do not think there will be a multi-DPU per server scenario.
Quoted 24 lines
Quoted 7 lines
Furtrhermore, I think this is a tradeoff between performance and flexibility.
Our goal is to achieve best performance, this means we need to give up some flexibility decisively. For example of 'FAST_FREEMode', it deletes most of the buffer check (refcnt > 1, external buffer, chain buffer), chooses a shorest path, and then achieve significant performanceimprovement.
Quoted 12 lines
Quoted 11 lines
Wonder did you had a chance to consider mempool-cache ZC API, similar to one we have for the ring?
It would allow us on TX free path to avoid copying mbufs to temporary array on the stack.
Instead we can put them straight from TX SW ring to the mempool cache. That should save extra store/load for mbuf and might help to achieve some performance gain without by-passing mempool.
It probably wouldn't be as fast as what you proposing, but might be fast enough to consider as alternative.
Again, it would be a generic one, so we can avoid all these implications and limitations.Quoted 3 lines
[Feifei] I think this is a good try. However, the most important thing is that if we can bypass the mempool decisively to pursue the significant performance gains.
Quoted 5 lines
I understand the intention, and I personally think this is wrong and dangerous attitude.
We have mempool abstraction in place for very good reason.
So we need to try to improve mempool performance (and API if necessary) at first place, not to avoid it and break our own rules and recommendations.
The abstraction can be thought of at a higher level. i.e. the driver manages the buffer allocation/free and is hidden from the application. The application does not need to be aware of how these changes are implemented.
Quoted 38 lines
Quoted 8 lines
For ZC, there maybe a problem for it in i40e. The reason for that put Tx buffers into temporary is that
i40e_tx_entryincludes buffer pointer and index.
Thus we cannot put TxSW_ringentry into mempool directly, we need to firstlt extract mbuf pointer. Finally, though we use ZC, we still can't avoid using a temporary stack to extract Tx buffer pointers.When talking about ZC API for mempool cache I meant something like:
void **mempool_cache_put_zc_start(struct rte_mempool_cache *mc, uint32_t *nb_elem, uint32_t flags); voidmempool_cache_put_zc_finish(struct rte_mempool_cache *mc, uint32_t nb_elem); i.e. _start_ will return user a pointer inside mp-cache where to put free elems and max number of slots that can be safely filled.
_finish_ will updatemc->len.
As an example:/* expect to free N mbufs */
uint32_tn = N;
void **p =mempool_cache_put_zc_start(mc, &n, ...);/* free up to n elems */
for (i = 0; i != n; i++) {/* get next free mbuf from somewhere */ mb = extract_and_prefree_mbuf(...); /* no more free mbufs for now */ if (mb == NULL) break; p[i] = mb; }/* finalize ZC put, with _i_ freed elems */
mempool_cache_put_zc_finish(mc, i);
Quoted 8 lines
That way, I think we can overcome the issue with
i40e_tx_entryyou mentioned above. Plus it might be useful in other similar places.Another alternative is obviously to split
i40e_tx_entryinto two structs (one for mbuf, second for its metadata) and have a separate array for each of them.
Though with that approach we need to make sure no perf drops will be introduced, plus probably more code changes will be required.
Commit '5171b4ee6b6" already does this (in a different way), but just for AVX512. Unfortunately, it does not record any performance improvements. We could port this to Arm NEON and look at the performance.
Quoted 4 lines
24/05/2022 21:14, Honnappa Nagarahalli пишет:
Quoted 61 lines
<snip>
Quoted 50 lines
[[email protected] appears similar to someone who previously sent you email, but may not be that person. Learn why this could be a risk at https://aka.ms/LearnAboutSenderIdentification.]
16/05/2022 07:10, Feifei Wang пишет:
Quoted 38 lines
Quoted 30 lines
Quoted 23 lines
Currently, the transmit side frees the buffers into the lcore cache and the receive side allocates buffers from the lcore cache. The transmit side typically frees 32 buffers resulting in 32*8=256B of stores to lcore cache. The receive side allocates 32 buffers and stores them in the receive side software ring, resulting in 32*8=256B of stores and 256B of load from the lcore cache.
This patch proposes a mechanism to avoid freeing to/allocating from the lcore cache. i.e. the receive side will free the buffers from transmit side directly into it's software ring. This will avoid the 256B of loads and stores introduced by the lcore cache. It also frees up the cache lines used by the lcore cache.
However, this solution poses several constraints:
1)The receive queue needs to know which transmit queue it should take the buffers from. The application logic decides which transmit port to use to send out the packets. In many use cases the NIC might have a single port ([1], [2], [3]), in which case a given transmit queue is always mapped to a single receive queue (1:1 Rx queue: Tx queue). This is easy to configure.
Quoted 6 lines
If the NIC has 2 ports (there are several references), then we will have
1:2 (RX queue: TX queue) mapping which is still easy to configure. However, if this is generalized to 'N' ports, the configuration can be long. More over the PMD would have to scan a list of transmit queues to pull the buffers from.Quoted 7 lines
Just to re-iterate some generic concerns about this proposal:
- We effectively link RX and TX queues - when this feature is enabled, user can't stop TX queue without stopping linked RX queue first. Right now user is free to start/stop any queues at his will. If that feature will allow to link queues from different ports, then even ports will become dependent and user will have to pay extra care when managing such ports.Quoted 5 lines
[Feifei] When direct rearm enabled, there are two path for thread to choose. If there are enough Tx freed buffers, Rx can put buffers from Tx.
Otherwise, Rx will put buffers from mempool as usual. Thus, users do not need to pay much attention managing ports.Quoted 9 lines
What I am talking about: right now different port or different queues of the same port can be treated as independent entities:
in general user is free to start/stop (and even reconfigure in some cases) one entity without need to stop other entity.
I.E user can stop and re-configure TX queue while keep receiving packets from RX queue.
With direct re-arm enabled, I think it wouldn't be possible any more:
before stopping/reconfiguring TX queue user would have make sure that corresponding RX queue wouldn't be used by datapath.
Quoted 1 line
I am trying to understand the problem better. For the TX queue to be stopped, the user must have blocked the data plane from accessing the TX queue.
Surely it is user responsibility tnot to call tx_burst()
for stopped/released queue.
The problem is that while TX for that queue is stopped,
RX for related queue still can continue.
So rx_burst() will try to read/modify TX queue data,
that might be already freed, or simultaneously modified by control path.
Again, it all can be mitigated by carefully re-designing and modifying control and data-path inside user app -
by doing extra checks and synchronizations, etc.
But from practical point - I presume most of users simply would avoid using this feature due all potential problems it might cause.
Quoted 2 lines
Like Feifei says, the RX side has the normal packet allocation path still available. Also this sounds like a corner case to me, we can handle this through checks in the
queue_stopAPI.
Depends.
if it would be allowed to link queues only from the same port, then yes, extra checks for queue-stop might be enough. As right now DPDK doesn't allow user to change number of queues without dev_stop() first.
Though if it would be allowed to link queues from different ports, then situation will be much worse.
Right now ports are totally independent entities (except some special cases like link-bonding, etc.).
As one port can keep doing RX/TX, second one can be stopped, re-confgured, even detached, and newly attached device might re-use same port number.
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
- very limited usage scenario - it will have a positive effect only when we have a fixed forwarding mapping: all (or nearly all) packets from the RX queue are forwarded into the same TX queue.
Quoted 6 lines
Quoted 5 lines
Quoted 2 lines
[Feifei] Although the usage scenario is limited, this usage scenario has a wide range of applications, such as NIC with one port.
yes, there are NICs with one port, but no guarantee there wouldn't be several such NICs within the system.
What I see in my interactions is, a single NIC/DPU is under utilized for a 2 socket system. Some are adding more sockets to the system to better utilize the DPU. The NIC bandwidth continues to grow significantly. I do not think there will be a multi-DPU per server scenario.
Interesting... from my experience it is visa-versa:
in many cases 200Gb/s is not that much these days
to saturate modern 2 socket x86 server.
Though I suppose a lot depends on particular HW and actual workload.
Quoted 79 lines
Quoted 25 lines
Quoted 8 lines
Furtrhermore, I think this is a tradeoff between performance and flexibility.
Our goal is to achieve best performance, this means we need to give up some flexibility decisively. For example of 'FAST_FREEMode', it deletes most of the buffer check (refcnt > 1, external buffer, chain buffer), chooses a shorest path, and then achieve significant performanceimprovement.
Quoted 12 lines
Quoted 11 lines
Wonder did you had a chance to consider mempool-cache ZC API, similar to one we have for the ring?
It would allow us on TX free path to avoid copying mbufs to temporary array on the stack.
Instead we can put them straight from TX SW ring to the mempool cache. That should save extra store/load for mbuf and might help to achieve some performance gain without by-passing mempool.
It probably wouldn't be as fast as what you proposing, but might be fast enough to consider as alternative.
Again, it would be a generic one, so we can avoid all these implications and limitations.Quoted 3 lines
[Feifei] I think this is a good try. However, the most important thing is that if we can bypass the mempool decisively to pursue the significant performance gains.
Quoted 5 lines
I understand the intention, and I personally think this is wrong and dangerous attitude.
We have mempool abstraction in place for very good reason.
So we need to try to improve mempool performance (and API if necessary) at first place, not to avoid it and break our own rules and recommendations.The abstraction can be thought of at a higher level. i.e. the driver manages the buffer allocation/free and is hidden from the application. The application does not need to be aware of how these changes are implemented.
Quoted 38 lines
Quoted 8 lines
For ZC, there maybe a problem for it in i40e. The reason for that put Tx buffers into temporary is that
i40e_tx_entryincludes buffer pointer and index.
Thus we cannot put TxSW_ringentry into mempool directly, we need to firstlt extract mbuf pointer. Finally, though we use ZC, we still can't avoid using a temporary stack to extract Tx buffer pointers.When talking about ZC API for mempool cache I meant something like:
void **mempool_cache_put_zc_start(struct rte_mempool_cache *mc, uint32_t *nb_elem, uint32_t flags); voidmempool_cache_put_zc_finish(struct rte_mempool_cache *mc, uint32_t nb_elem); i.e. _start_ will return user a pointer inside mp-cache where to put free elems and max number of slots that can be safely filled.
_finish_ will updatemc->len.
As an example:/* expect to free N mbufs */
uint32_tn = N;
void **p =mempool_cache_put_zc_start(mc, &n, ...);/* free up to n elems */
for (i = 0; i != n; i++) {/* get next free mbuf from somewhere */ mb = extract_and_prefree_mbuf(...); /* no more free mbufs for now */ if (mb == NULL) break; p[i] = mb; }/* finalize ZC put, with _i_ freed elems */
mempool_cache_put_zc_finish(mc, i);Quoted 8 lines
That way, I think we can overcome the issue with
i40e_tx_entryyou mentioned above. Plus it might be useful in other similar places.Another alternative is obviously to split
i40e_tx_entryinto two structs (one for mbuf, second for its metadata) and have a separate array for each of them.
Though with that approach we need to make sure no perf drops will be introduced, plus probably more code changes will be required.Commit '
5171b4ee6b6" already does this (in a different way), but just for AVX512. Unfortunately, it does not record any performance improvements. We could port this to Arm NEON and look at the performance.
<snip>
Quoted 60 lines
Quoted 60 lines
Quoted 60 lines
[[email protected] appears similar to someone who previously sent you email, but may not be that person. Learn why this could be a risk at https://aka.ms/LearnAboutSenderIdentification.]
16/05/2022 07:10, Feifei Wang пишет:
Quoted 38 lines
Quoted 30 lines
Quoted 23 lines
Currently, the transmit side frees the buffers into the lcore cache and the receive side allocates buffers from the lcore cache. The transmit side typically frees 32 buffers resulting in 32*8=256B of stores to lcore cache. The receive side allocates 32 buffers and stores them in the receive side software ring, resulting in 32*8=256B of stores and 256B of load from the lcore cache.
This patch proposes a mechanism to avoid freeing to/allocating from the lcore cache. i.e. the receive side will free the buffers from transmit side directly into it's software ring. This will avoid the 256B of loads and stores introduced by the lcore cache. It also frees up the cache lines used by the lcore cache.
However, this solution poses several constraints:
1)The receive queue needs to know which transmit queue it should take the buffers from. The application logic decides which transmit port to use to send out the packets. In many use cases the NIC might have a single port ([1], [2], [3]), in which case a given transmit queue is always mapped to a single receive queue (1:1 Rx queue: Tx queue). This is easy to configure.
Quoted 6 lines
If the NIC has 2 ports (there are several references), then we will have
1:2 (RX queue: TX queue) mapping which is still easy to configure. However, if this is generalized to 'N' ports, the configuration can be long. More over the PMD would have to scan a list of transmit queues to pull the buffers from.Quoted 7 lines
Just to re-iterate some generic concerns about this proposal:
- We effectively link RX and TX queues - when this feature is enabled, user can't stop TX queue without stopping linked RX queue first. Right now user is free to start/stop any queues at his will. If that feature will allow to link queues from different ports, then even ports will become dependent and user will have to pay extra care when managing such ports.Quoted 5 lines
[Feifei] When direct rearm enabled, there are two path for thread to choose. If there are enough Tx freed buffers, Rx can put buffers from Tx.
Otherwise, Rx will put buffers from mempool as usual. Thus, users do not need to pay much attention managing ports.What I am talking about: right now different port or different queues of the same port can be treated as independent entities:
in general user is free to start/stop (and even reconfigure in some cases) one entity without need to stop other entity.
I.E user can stop and re-configure TX queue while keep receiving packets from RX queue.
With direct re-arm enabled, I think it wouldn't be possible any more:
before stopping/reconfiguring TX queue user would have make sure that corresponding RX queue wouldn't be used by datapath.
Quoted 8 lines
Quoted 1 line
I am trying to understand the problem better. For the TX queue to be stopped,
the user must have blocked the data plane from accessing the TX queue.
Surely it is user responsibility tnot to call
tx_burst()for stopped/released queue. The problem is that while TX for that queue is stopped, RX for related queue still can continue.
Sorx_burst()will try to read/modify TX queue data, that might be already freed, or simultaneously modified by control path.
Understood, agree on the issue
Quoted 5 lines
Again, it all can be mitigated by carefully re-designing and modifying control and data-path inside user app - by doing extra checks and synchronizations, etc. But from practical point - I presume most of users simply would avoid using this feature due all potential problems it might cause.
That is subjective, it all depends on the performance improvements users see in their application. IMO, the performance improvement seen with this patch is worth few changes.
Quoted 5 lines
Quoted 3 lines
Like Feifei says, the RX side has the normal packet allocation path still available. Also this sounds like a corner case to me, we can handle this through checks in
the
queue_stopAPI.
Quoted 11 lines
Depends.
if it would be allowed to link queues only from the same port, then yes, extra checks for queue-stop might be enough.
As right now DPDK doesn't allow user to change number of queues withoutdev_stop()first.
Though if it would be allowed to link queues from different ports, then situation will be much worse.
Right now ports are totally independent entities (except some special cases like link-bonding, etc.).
As one port can keep doing RX/TX, second one can be stopped, re-confgured, even detached, and newly attached device might re-use same port number.
I see this as a similar restriction to the one discussed above. Do you see any issues if we enforce this with checks?
Quoted 110 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines
- very limited usage scenario - it will have a positive effect only when we have a fixed forwarding mapping: all (or nearly all) packets from the RX queue are forwarded into the same TX queue.
Quoted 6 lines
Quoted 5 lines
Quoted 2 lines
[Feifei] Although the usage scenario is limited, this usage scenario has a wide range of applications, such as NIC with one port.
yes, there are NICs with one port, but no guarantee there wouldn't be several such NICs within the system.
What I see in my interactions is, a single NIC/DPU is under utilized for a 2
socket system. Some are adding more sockets to the system to better utilize the DPU. The NIC bandwidth continues to grow significantly. I do not think there will be a multi-DPU per server scenario.
Interesting... from my experience it is visa-versa:
in many cases 200Gb/s is not that much these days to saturate modern 2 socket x86 server.
Though I suppose a lot depends on particular HW and actual workload.Quoted 26 lines
Quoted 26 lines
Quoted 9 lines
Furtrhermore, I think this is a tradeoff between performance and flexibility.
Our goal is to achieve best performance, this means we need to give up some flexibility decisively. For example of 'FAST_FREEMode', it deletes most of the buffer check (refcnt > 1, external buffer, chain buffer), chooses a shorest path, and then achieve significant performanceimprovement.
Quoted 15 lines
Quoted 11 lines
Wonder did you had a chance to consider mempool-cache ZC API, similar to one we have for the ring?
It would allow us on TX free path to avoid copying mbufs to temporary array on the stack.
Instead we can put them straight from TX SW ring to the mempool cache. That should save extra store/load for mbuf and might help to achieve some performance gain without by-passing mempool. It probably wouldn't be as fast as what you proposing, but might be fast enough to consider as alternative.
Again, it would be a generic one, so we can avoid all these implications and limitations.[Feifei] I think this is a good try. However, the most important thing is that if we can bypass the mempool decisively to pursue the significant performance gains.
Quoted 5 lines
Quoted 5 lines
I understand the intention, and I personally think this is wrong and dangerous attitude.
We have mempool abstraction in place for very good reason. So we need to try to improve mempool performance (and API if necessary) at first place, not to avoid it and break our own rules andrecommendations.
Quoted 1 line
The abstraction can be thought of at a higher level. i.e. the driver manages the
buffer allocation/free and is hidden from the application. The application does not need to be aware of how these changes are implemented.
Quoted 43 lines
Quoted 43 lines
Quoted 9 lines
For ZC, there maybe a problem for it in i40e. The reason for that put Tx buffers into temporary is that
i40e_tx_entryincludes buffer pointer and index.
Thus we cannot put TxSW_ringentry into mempool directly, we need to firstlt extract mbuf pointer. Finally, though we use ZC, we still can't avoid using a temporary stack to extract Tx buffer pointers.When talking about ZC API for mempool cache I meant something like:
void **mempool_cache_put_zc_start(struct rte_mempool_cache *mc, uint32_t *nb_elem, uint32_t flags); void mempool_cache_put_zc_finish(structrte_mempool_cache*mc,uint32_tnb_elem); i.e. _start_ will return user a pointer inside mp-cache where to put free elems and max number of slots that can be safely filled.
_finish_ will updatemc->len.
As an example:/* expect to free N mbufs */
uint32_tn = N;
void **p =mempool_cache_put_zc_start(mc, &n, ...);/* free up to n elems */
for (i = 0; i != n; i++) {/* get next free mbuf from somewhere */ mb = extract_and_prefree_mbuf(...); /* no more free mbufs for now */ if (mb == NULL) break; p[i] = mb; }/* finalize ZC put, with _i_ freed elems */
mempool_cache_put_zc_finish(mc, i);That way, I think we can overcome the issue with
i40e_tx_entryyou mentioned above. Plus it might be useful in other similar places.Quoted 6 lines
Quoted 5 lines
Another alternative is obviously to split
i40e_tx_entryinto two structs (one for mbuf, second for its metadata) and have a separate array for each of them.
Though with that approach we need to make sure no perf drops will be introduced, plus probably more code changes will be required.Commit '
5171b4ee6b6" already does this (in a different way), but just forAVX512. Unfortunately, it does not record any performance improvements. We could port this to Arm NEON and look at the performance.
Quoted 73 lines
<snip>
Quoted 60 lines
Quoted 60 lines
Quoted 60 lines
[[email protected] appears similar to someone who previously sent you email, but may not be that person. Learn why this could be a risk at https://aka.ms/LearnAboutSenderIdentification.]
16/05/2022 07:10, Feifei Wang пишет:
Quoted 38 lines
Quoted 30 lines
Quoted 23 lines
Currently, the transmit side frees the buffers into the lcore cache and the receive side allocates buffers from the lcore cache. The transmit side typically frees 32 buffers resulting in 32*8=256B of stores to lcore cache. The receive side allocates 32 buffers and stores them in the receive side software ring, resulting in 32*8=256B of stores and 256B of load from the lcore cache.
This patch proposes a mechanism to avoid freeing to/allocating from the lcore cache. i.e. the receive side will free the buffers from transmit side directly into it's software ring. This will avoid the 256B of loads and stores introduced by the lcore cache. It also frees up the cache lines used by the lcore cache.
However, this solution poses several constraints:
1)The receive queue needs to know which transmit queue it should take the buffers from. The application logic decides which transmit port to use to send out the packets. In many use cases the NIC might have a single port ([1], [2], [3]), in which case a given transmit queue is always mapped to a single receive queue (1:1 Rx queue: Tx queue). This is easy to configure.
Quoted 6 lines
If the NIC has 2 ports (there are several references), then we will have
1:2 (RX queue: TX queue) mapping which is still easy to configure. However, if this is generalized to 'N' ports, the configuration can be long. More over the PMD would have to scan a list of transmit queues to pull the buffers from.Quoted 7 lines
Just to re-iterate some generic concerns about this proposal:
- We effectively link RX and TX queues - when this feature is enabled, user can't stop TX queue without stopping linked RX queue first. Right now user is free to start/stop any queues at his will. If that feature will allow to link queues from different ports, then even ports will become dependent and user will have to pay extra care when managing such ports.Quoted 5 lines
[Feifei] When direct rearm enabled, there are two path for thread to choose. If there are enough Tx freed buffers, Rx can put buffers from Tx.
Otherwise, Rx will put buffers from mempool as usual. Thus, users do not need to pay much attention managing ports.What I am talking about: right now different port or different queues of the same port can be treated as independent entities:
in general user is free to start/stop (and even reconfigure in some cases) one entity without need to stop other entity.
I.E user can stop and re-configure TX queue while keep receiving packets from RX queue.
With direct re-arm enabled, I think it wouldn't be possible any more:
before stopping/reconfiguring TX queue user would have make sure that corresponding RX queue wouldn't be used by datapath.Quoted 8 lines
Quoted 1 line
I am trying to understand the problem better. For the TX queue to be stopped,
the user must have blocked the data plane from accessing the TX queue.
Surely it is user responsibility tnot to call
tx_burst()for stopped/released queue. The problem is that while TX for that queue is stopped, RX for related queue still can continue.
Sorx_burst()will try to read/modify TX queue data, that might be already freed, or simultaneously modified by control path.Understood, agree on the issue
Quoted 1 line
Quoted 6 lines
Quoted 4 lines
Again, it all can be mitigated by carefully re-designing and modifying control and data-path inside user app - by doing extra checks and synchronizations, etc. But from practical point - I presume most of users simply would avoid using this feature due all potential problems it might cause.
That is subjective, it all depends on the performance improvements users see in their application. IMO, the performance improvement seen with this patch is worth few changes.
Yes, it is subjective till some extent, though my feeling that it might end-up being sort of synthetic improvement used only by some show-case benchmarks.
From my perspective, it would be much more plausible, if we can introduce some sort of generic improvement, that doesn't impose all these extra constraints and implications. Like one, discussed below in that thread with ZC mempool approach.
Quoted 16 lines
Quoted 5 lines
Quoted 3 lines
Like Feifei says, the RX side has the normal packet allocation path still available. Also this sounds like a corner case to me, we can handle this through checks in
the
queue_stopAPI.Quoted 11 lines
Depends.
if it would be allowed to link queues only from the same port, then yes, extra checks for queue-stop might be enough.
As right now DPDK doesn't allow user to change number of queues withoutdev_stop()first.
Though if it would be allowed to link queues from different ports, then situation will be much worse.
Right now ports are totally independent entities (except some special cases like link-bonding, etc.).
As one port can keep doing RX/TX, second one can be stopped, re-confgured, even detached, and newly attached device might re-use same port number.
Quoted 1 line
I see this as a similar restriction to the one discussed above.
Yes, they are similar in principal, though I think that the case with queues from different port would make things much more complex.
Quoted 1 line
Do you see any issues if we enforce this with checks?
Hard to tell straightway, a lot will depend how smart such implementation would be.
Usually DPDK tends not to avoid heavy synchronizations within its data-path functions.
Quoted 111 lines
Quoted 110 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines
- very limited usage scenario - it will have a positive effect only when we have a fixed forwarding mapping: all (or nearly all) packets from the RX queue are forwarded into the same TX queue.
Quoted 6 lines
Quoted 5 lines
Quoted 2 lines
[Feifei] Although the usage scenario is limited, this usage scenario has a wide range of applications, such as NIC with one port.
yes, there are NICs with one port, but no guarantee there wouldn't be several such NICs within the system.
What I see in my interactions is, a single NIC/DPU is under utilized for a 2
socket system. Some are adding more sockets to the system to better utilize the DPU. The NIC bandwidth continues to grow significantly. I do not think there will be a multi-DPU per server scenario.
Interesting... from my experience it is visa-versa:
in many cases 200Gb/s is not that much these days to saturate modern 2 socket x86 server.
Though I suppose a lot depends on particular HW and actual workload.Quoted 26 lines
Quoted 26 lines
Quoted 9 lines
Furtrhermore, I think this is a tradeoff between performance and flexibility.
Our goal is to achieve best performance, this means we need to give up some flexibility decisively. For example of 'FAST_FREEMode', it deletes most of the buffer check (refcnt > 1, external buffer, chain buffer), chooses a shorest path, and then achieve significant performanceimprovement.
Quoted 15 lines
Quoted 11 lines
Wonder did you had a chance to consider mempool-cache ZC API, similar to one we have for the ring?
It would allow us on TX free path to avoid copying mbufs to temporary array on the stack.
Instead we can put them straight from TX SW ring to the mempool cache. That should save extra store/load for mbuf and might help to achieve some performance gain without by-passing mempool. It probably wouldn't be as fast as what you proposing, but might be fast enough to consider as alternative.
Again, it would be a generic one, so we can avoid all these implications and limitations.[Feifei] I think this is a good try. However, the most important thing is that if we can bypass the mempool decisively to pursue the significant performance gains.
Quoted 5 lines
Quoted 5 lines
I understand the intention, and I personally think this is wrong and dangerous attitude.
We have mempool abstraction in place for very good reason. So we need to try to improve mempool performance (and API if necessary) at first place, not to avoid it and break our own rules andrecommendations.
Quoted 1 line
The abstraction can be thought of at a higher level. i.e. the driver manages the
buffer allocation/free and is hidden from the application. The application does not need to be aware of how these changes are implemented.
Quoted 43 lines
Quoted 43 lines
Quoted 9 lines
For ZC, there maybe a problem for it in i40e. The reason for that put Tx buffers into temporary is that
i40e_tx_entryincludes buffer pointer and index.
Thus we cannot put TxSW_ringentry into mempool directly, we need to firstlt extract mbuf pointer. Finally, though we use ZC, we still can't avoid using a temporary stack to extract Tx buffer pointers.When talking about ZC API for mempool cache I meant something like:
void **mempool_cache_put_zc_start(struct rte_mempool_cache *mc, uint32_t *nb_elem, uint32_t flags); void mempool_cache_put_zc_finish(structrte_mempool_cache*mc,uint32_tnb_elem); i.e. _start_ will return user a pointer inside mp-cache where to put free elems and max number of slots that can be safely filled.
_finish_ will updatemc->len.
As an example:/* expect to free N mbufs */
uint32_tn = N;
void **p =mempool_cache_put_zc_start(mc, &n, ...);/* free up to n elems */
for (i = 0; i != n; i++) {/* get next free mbuf from somewhere */ mb = extract_and_prefree_mbuf(...); /* no more free mbufs for now */ if (mb == NULL) break; p[i] = mb; }/* finalize ZC put, with _i_ freed elems */
mempool_cache_put_zc_finish(mc, i);That way, I think we can overcome the issue with
i40e_tx_entryyou mentioned above. Plus it might be useful in other similar places.Quoted 6 lines
Quoted 5 lines
Another alternative is obviously to split
i40e_tx_entryinto two structs (one for mbuf, second for its metadata) and have a separate array for each of them.
Though with that approach we need to make sure no perf drops will be introduced, plus probably more code changes will be required.Commit '
5171b4ee6b6" already does this (in a different way), but just forAVX512. Unfortunately, it does not record any performance improvements. We could port this to Arm NEON and look at the performance.
Quoted 107 lines
From: Konstantin Ananyev [mailto:[email protected]]
Sent: Saturday, 4 June 2022 01.32Quoted 5 lines
<snip>
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
[[email protected] appears similar to someone who previously sent you email, but may not be that person. Learn why
this
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
could be a risk at https://aka.ms/LearnAboutSenderIdentification.]
16/05/2022 07:10, Feifei Wang пишет:
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Currently, the transmit side frees the buffers into the lcore cache and the receive side allocates buffers from the lcore
cache.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
The transmit side typically frees 32 buffers resulting in 32*8=256B of stores to lcore cache. The receive side allocates
32
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
buffers and stores them in the receive side software ring, resulting in 32*8=256B of stores and 256B of load from the
lcore cache.
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
This patch proposes a mechanism to avoid freeing to/allocating from the lcore cache. i.e. the receive side will free the
buffers
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
from transmit side directly into it's software ring. This will avoid the 256B of loads and stores introduced by the lcore
cache.
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
It also frees up the cache lines used by the lcore cache.
However, this solution poses several constraints:
1)The receive queue needs to know which transmit queue it
should
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
take the buffers from. The application logic decides which transmit port to use to send out the packets. In many use cases the NIC might have a single port ([1], [2], [3]), in which case
a
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
given transmit queue is always mapped to a single receive queue (1:1 Rx queue: Tx queue). This is easy to configure.
If the NIC has 2 ports (there are several references), then we will have
1:2 (RX queue: TX queue) mapping which is still easy toconfigure.
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 3 lines
However, if this is generalized to 'N' ports, the configuration can be long. More over the PMD would have to scan a list of transmit queues to pull the buffers from.
Just to re-iterate some generic concerns about this proposal:
- We effectively link RX and TX queues - when this featureis enabled,
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
user can't stop TX queue without stopping linked RX queue
first.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Right now user is free to start/stop any queues at his
will.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
If that feature will allow to link queues from different
ports,
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
then even ports will become dependent and user will have
to pay extra
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
care when managing such ports.
[Feifei] When direct rearm enabled, there are two path for thread
to
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
choose. If there are enough Tx freed buffers, Rx can put buffers from Tx.
Otherwise, Rx will put buffers from mempool as usual. Thus, usersdo
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
not need to pay much attention managing ports.
What I am talking about: right now different port or different
queues
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
of the same port can be treated as independent entities:
in general user is free to start/stop (and even reconfigure insome
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
cases) one entity without need to stop other entity. I.E user can stop and re-configure TX queue while keep receiving packets from RX queue.
With direct re-arm enabled, I think it wouldn't be possible anymore:
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
before stopping/reconfiguring TX queue user would have make sure
that
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 1 line
corresponding RX queue wouldn't be used by datapath.
I am trying to understand the problem better. For the TX queue to
be stopped,
Quoted 1 line
Quoted 1 line
the user must have blocked the data plane from accessing the TX
queue.
Quoted 2 lines
Quoted 2 lines
Surely it is user responsibility tnot to call
tx_burst()forstopped/released queue.
Quoted 1 line
Quoted 1 line
The problem is that while TX for that queue is stopped, RX for
related queue still
Quoted 2 lines
Quoted 2 lines
can continue.
Sorx_burst()will try to read/modify TX queue data, that might bealready freed,
Quoted 5 lines
Quoted 1 line
or simultaneously modified by control path.
Understood, agree on the issue
Quoted 2 lines
Again, it all can be mitigated by carefully re-designing and
modifying control and
Quoted 1 line
Quoted 1 line
data-path inside user app - by doing extra checks and
synchronizations, etc.
Quoted 1 line
Quoted 1 line
But from practical point - I presume most of users simply would
avoid using this
Quoted 1 line
Quoted 1 line
feature due all potential problems it might cause.
Quoted 8 lines
Quoted 1 line
That is subjective, it all depends on the performance improvements
users see in their application.
Quoted 1 line
IMO, the performance improvement seen with this patch is worth few
changes.
Yes, it is subjective till some extent, though my feeling that it might end-up being sort of synthetic improvement used only by some show-case benchmarks.
I believe that one specific important use case has already been mentioned, so I don't think this is a benchmark only feature.
Quoted 5 lines
From my perspective, it would be much more plausible,
if we can introduce some sort of generic improvement, that doesn't impose all these extra constraints and implications. Like one, discussed below in that thread with ZC mempool approach.
Considering this feature from a high level perspective, I agree with Konstantin's concerns, so I'll also support his views.
If this patch is supposed to be a generic feature, please add support for it in all NIC PMDs, not just one. (Regardless if the feature is defined as 1:1 mapping or N:M mapping.) It is purely software, so it should be available for all PMDs, not just your favorite hardware! Consider the "fast mbuf free" feature, which is pure software; why is that feature not implemented in all PMDs?
A secondary point I'm making here is that this specific feature will lead to an enormous amount of copy-paste code, instead of a generic library function easily available for all PMDs.
Quoted 30 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Like Feifei says, the RX side has the normal packet allocation path
still available.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Also this sounds like a corner case to me, we can handle this
through checks in
Quoted 4 lines
Quoted 4 lines
the
queue_stopAPI.Depends.
if it would be allowed to link queues only from the same port, thenyes, extra
Quoted 2 lines
Quoted 2 lines
checks for queue-stop might be enough.
As right now DPDK doesn't allow user to change number of queueswithout
Quoted 2 lines
Quoted 2 lines
dev_stop()first.
Though if it would be allowed to link queues from different ports,then situation
Quoted 2 lines
Quoted 2 lines
will be much worse.
Right now ports are totally independent entities (except somespecial cases like
Quoted 2 lines
Quoted 2 lines
link-bonding, etc.).
As one port can keep doing RX/TX, second one can be stopped, re-confgured,
Quoted 1 line
Quoted 1 line
even detached, and newly attached device might re-use same port
number.
Quoted 1 line
I see this as a similar restriction to the one discussed above.
Yes, they are similar in principal, though I think that the case with queues from different port would make things much more complex.
Quoted 6 lines
Quoted 1 line
Do you see any issues if we enforce this with checks?
Hard to tell straightway, a lot will depend how smart such implementation would be.
Usually DPDK tends not to avoid heavy synchronizations within its data-path functions.
Certainly! Implementing more and more of such features in the PMDs will lead to longer and longer data plane code paths in the PMDs. It is the "salami method", where each small piece makes no performance difference, but they all add up, and eventually the sum of them does impact the performance of the general use case negatively.
Quoted 141 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
- very limited usage scenario - it will have a positive effect
only
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
when we have a fixed forwarding mapping: all (or nearly
all) packets
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
from the RX queue are forwarded into the same TX queue.
[Feifei] Although the usage scenario is limited, this usage
scenario
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
has a wide range of applications, such as NIC with one port.
yes, there are NICs with one port, but no guarantee there wouldn't
be
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 1 line
several such NICs within the system.
What I see in my interactions is, a single NIC/DPU is under
utilized for a 2
Quoted 1 line
Quoted 1 line
socket system. Some are adding more sockets to the system to better
utilize the
Quoted 1 line
Quoted 1 line
DPU. The NIC bandwidth continues to grow significantly. I do not
think there will
Quoted 5 lines
Quoted 5 lines
be a multi-DPU per server scenario.
Interesting... from my experience it is visa-versa:
in many cases 200Gb/s is not that much these days to saturate modern2 socket
Quoted 8 lines
Quoted 8 lines
x86 server.
Though I suppose a lot depends on particular HW and actual workload.Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Furtrhermore, I think this is a tradeoff between performance and flexibility.
Our goal is to achieve best performance, this means we need togive
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
up some flexibility decisively. For example of '
FAST_FREEMode',it
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
deletes most of the buffer check (refcnt > 1, external buffer,
chain
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
buffer), chooses a shorest path, and then achieve significant performance
improvement.
Quoted 5 lines
Quoted 5 lines
Wonder did you had a chance to consider mempool-cache ZC API, similar to one we have for the ring?
It would allow us on TX free path to avoid copying mbufs to temporary array on the stack.
Instead we can put them straight from TX SW ring to the mempoolcache.
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
That should save extra store/load for mbuf and might help to achieve some performance gain without by-passing mempool. It probably wouldn't be as fast as what you proposing, but might
be
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 3 lines
fast enough to consider as alternative.
Again, it would be a generic one, so we can avoid all these implications and limitations.[Feifei] I think this is a good try. However, the most important thing is that if we can bypass the mempool decisively to pursue
the
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
significant performance gains.
I understand the intention, and I personally think this is wrong
and
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
dangerous attitude.
We have mempool abstraction in place for very good reason. So we need to try to improve mempool performance (and API if necessary) at first place, not to avoid it and break our own rulesand
Quoted 2 lines
Quoted 2 lines
recommendations.
Quoted 1 line
The abstraction can be thought of at a higher level. i.e. the
driver manages the
Quoted 1 line
Quoted 1 line
buffer allocation/free and is hidden from the application. The
application does
Quoted 6 lines
Quoted 6 lines
not need to be aware of how these changes are implemented.
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
For ZC, there maybe a problem for it in i40e. The reason for that put Tx buffers into temporary is that
i40e_tx_entryincludesbuffer
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
pointer and index.
Thus we cannot put TxSW_ringentry into mempool directly, weneed
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
to firstlt extract mbuf pointer. Finally, though we use ZC, we
still
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
can't avoid using a temporary stack to extract Tx buffer
pointers.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
When talking about ZC API for mempool cache I meant something
like:
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
void **
mempool_cache_put_zc_start(struct rte_mempool_cache *mc, uint32_t *nb_elem, uint32_t flags); void mempool_cache_put_zc_finish(structrte_mempool_cache*mc,uint32_tnb_elem); i.e. _start_ will return user a pointer inside mp-cache where to put free elems and maxnumber
Quoted 21 lines
Quoted 21 lines
Quoted 21 lines
Quoted 21 lines
of slots that can be safely filled.
_finish_ will updatemc->len.
As an example:/* expect to free N mbufs */
uint32_tn = N;
void **p =mempool_cache_put_zc_start(mc, &n, ...);/* free up to n elems */
for (i = 0; i != n; i++) {/* get next free mbuf from somewhere */ mb = extract_and_prefree_mbuf(...); /* no more free mbufs for now */ if (mb == NULL) break; p[i] = mb; }Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
/* finalize ZC put, with _i_ freed elems */
mempool_cache_put_zc_finish(mc, i);That way, I think we can overcome the issue with
i40e_tx_entryyou mentioned above. Plus it might be useful in other similar places.Another alternative is obviously to split
i40e_tx_entryinto two structs (one for mbuf, second for its metadata) and have aseparate
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
array for each of them.
Though with that approach we need to make sure no perf drops willbe
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 1 line
introduced, plus probably more code changes will be required.
Commit '
5171b4ee6b6" already does this (in a different way), butjust for
Quoted 1 line
Quoted 1 line
AVX512. Unfortunately, it does not record any performance
improvements. We
Quoted 2 lines
Quoted 1 line
could port this to Arm NEON and look at the performance.
<snip>
Quoted 111 lines
Quoted 111 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
[[email protected] appears similar to someone who previously sent you email, but may not be that person. Learn why
this
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
could be a risk at
https://aka.ms/LearnAboutSenderIdentification.]16/05/2022 07:10, Feifei Wang пишет:
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Currently, the transmit side frees the buffers into the lcore cache and the receive side allocates buffers from the lcore
cache.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
The transmit side typically frees 32 buffers resulting in 32*8=256B of stores to lcore cache. The receive side allocates
32
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
buffers and stores them in the receive side software ring, resulting in 32*8=256B of stores and 256B of load from the
lcore cache.
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
This patch proposes a mechanism to avoid freeing to/allocating from the lcore cache. i.e. the receive side will free the
buffers
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
from transmit side directly into it's software ring. This will avoid the 256B of loads and stores introduced by the lcore
cache.
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
It also frees up the cache lines used by the lcore cache.
However, this solution poses several constraints:
1)The receive queue needs to know which transmit queue it
should
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
take the buffers from. The application logic decides which transmit port to use to send out the packets. In many use cases the NIC might have a single port ([1], [2], [3]), in which case
a
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
given transmit queue is always mapped to a single receive queue
(1:1 Rx queue: Tx queue). This is easy to configure.If the NIC has 2 ports (there are several references), then we will have
1:2 (RX queue: TX queue) mapping which is still easy toconfigure.
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 3 lines
However, if this is generalized to 'N' ports, the configuration can be long. More over the PMD would have to scan a list of transmit queues to pull the buffers from.
Just to re-iterate some generic concerns about this proposal:
- We effectively link RX and TX queues - when this featureis enabled,
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
user can't stop TX queue without stopping linked RX queue
first.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Right now user is free to start/stop any queues at his
will.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
If that feature will allow to link queues from different
ports,
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
then even ports will become dependent and user will have
to pay extra
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
care when managing such ports.
[Feifei] When direct rearm enabled, there are two path for thread
to
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
choose. If there are enough Tx freed buffers, Rx can put buffers from Tx.
Otherwise, Rx will put buffers from mempool as usual. Thus, usersdo
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
not need to pay much attention managing ports.
What I am talking about: right now different port or different
queues
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
of the same port can be treated as independent entities:
in general user is free to start/stop (and even reconfigure insome
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
cases) one entity without need to stop other entity. I.E user can stop and re-configure TX queue while keep receiving packets from RX queue.
With direct re-arm enabled, I think it wouldn't be possible anymore:
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
before stopping/reconfiguring TX queue user would have make sure
that
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 1 line
corresponding RX queue wouldn't be used by datapath.
I am trying to understand the problem better. For the TX queue to
be stopped,
Quoted 1 line
Quoted 1 line
the user must have blocked the data plane from accessing the TX
queue.
Quoted 2 lines
Quoted 2 lines
Surely it is user responsibility tnot to call
tx_burst()forstopped/released queue.
Quoted 1 line
Quoted 1 line
The problem is that while TX for that queue is stopped, RX for
related queue still
Quoted 2 lines
Quoted 2 lines
can continue.
Sorx_burst()will try to read/modify TX queue data, that might bealready freed,
Quoted 5 lines
Quoted 1 line
or simultaneously modified by control path.
Understood, agree on the issue
Quoted 2 lines
Again, it all can be mitigated by carefully re-designing and
modifying control and
Quoted 1 line
Quoted 1 line
data-path inside user app - by doing extra checks and
synchronizations, etc.
Quoted 1 line
Quoted 1 line
But from practical point - I presume most of users simply would
avoid using this
Quoted 2 lines
Quoted 1 line
feature due all potential problems it might cause.
That is subjective, it all depends on the performance improvements
users see in their application.
Quoted 1 line
IMO, the performance improvement seen with this patch is worth few
Quoted 8 lines
Quoted 5 lines
changes.
Yes, it is subjective till some extent, though my feeling that it might end-up being sort of synthetic improvement used only by some show-case benchmarks.
I believe that one specific important use case has already been mentioned, so I don't think this is a benchmark only feature.
+1
Quoted 9 lines
Quoted 6 lines
From my perspective, it would be much more plausible, if we can
introduce some sort of generic improvement, that doesn't impose all these extra constraints and implications.
Like one, discussed below in that thread with ZC mempool approach.Considering this feature from a high level perspective, I agree with Konstantin's concerns, so I'll also support his views.
We did hack the ZC mempool approach [1], level of improvement is pretty small compared with this patch.
[1] http://patches.dpdk.org/project/dpdk/patch/[email protected]/
Quoted 6 lines
If this patch is supposed to be a generic feature, please add support for it in all NIC PMDs, not just one. (Regardless if the feature is defined as 1:1 mapping or N:M mapping.) It is purely software, so it should be available for all PMDs, not just your favorite hardware! Consider the "fast mbuf free" feature, which is pure software; why is that feature not implemented in all PMDs?
Agree, it is good to have it supported in all the drivers. We do not have a favorite hardware, just picked a PMD which we are more familiar with. We do plan to implement in other prominent PMDs.
Quoted 4 lines
A secondary point I'm making here is that this specific feature will lead to an enormous amount of copy-paste code, instead of a generic library function easily available for all PMDs.
Are you talking about the i40e driver code in specific? If yes, agree we should avoid copy-paste and we will look to reduce that.
Quoted 39 lines
Quoted 32 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Like Feifei says, the RX side has the normal packet allocation path
still available.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Also this sounds like a corner case to me, we can handle this
through checks in
Quoted 4 lines
Quoted 4 lines
the
queue_stopAPI.Depends.
if it would be allowed to link queues only from the same port, thenyes, extra
Quoted 2 lines
Quoted 2 lines
checks for queue-stop might be enough.
As right now DPDK doesn't allow user to change number of queueswithout
Quoted 2 lines
Quoted 2 lines
dev_stop()first.
Though if it would be allowed to link queues from different ports,then situation
Quoted 2 lines
Quoted 2 lines
will be much worse.
Right now ports are totally independent entities (except somespecial cases like
Quoted 2 lines
Quoted 2 lines
link-bonding, etc.).
As one port can keep doing RX/TX, second one can be stopped, re-confgured,
Quoted 1 line
Quoted 1 line
even detached, and newly attached device might re-use same port
number.
Quoted 1 line
I see this as a similar restriction to the one discussed above.
Yes, they are similar in principal, though I think that the case with queues from different port would make things much more complex.
Quoted 6 lines
Quoted 1 line
Do you see any issues if we enforce this with checks?
Hard to tell straightway, a lot will depend how smart such implementation would be.
Usually DPDK tends not to avoid heavy synchronizations within its data-path functions.
Quoted 5 lines
Certainly! Implementing more and more of such features in the PMDs will lead to longer and longer data plane code paths in the PMDs. It is the "salami method", where each small piece makes no performance difference, but they all add up, and eventually the sum of them does impact the performance of the general use case negatively.
It would be good to have a test running in UNH that shows the performance trend.
Quoted 148 lines
Quoted 148 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
- very limited usage scenario - it will have a positive effect
only
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
when we have a fixed forwarding mapping: all (or nearly
all) packets
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
from the RX queue are forwarded into the same TX queue.
[Feifei] Although the usage scenario is limited, this usage
scenario
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
has a wide range of applications, such as NIC with one port.
yes, there are NICs with one port, but no guarantee there wouldn't
be
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 1 line
several such NICs within the system.
What I see in my interactions is, a single NIC/DPU is under
utilized for a 2
Quoted 1 line
Quoted 1 line
socket system. Some are adding more sockets to the system to better
utilize the
Quoted 1 line
Quoted 1 line
DPU. The NIC bandwidth continues to grow significantly. I do not
think there will
Quoted 6 lines
Quoted 6 lines
be a multi-DPU per server scenario.
Interesting... from my experience it is visa-versa:
in many cases 200Gb/s is not that much these days to saturate modern2 socket
Quoted 8 lines
Quoted 8 lines
x86 server.
Though I suppose a lot depends on particular HW and actual workload.Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Furtrhermore, I think this is a tradeoff between performance and flexibility.
Our goal is to achieve best performance, this means we need togive
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
up some flexibility decisively. For example of '
FAST_FREEMode',it
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
deletes most of the buffer check (refcnt > 1, external buffer,
chain
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
buffer), chooses a shorest path, and then achieve significant performance
improvement.
Quoted 5 lines
Quoted 5 lines
Wonder did you had a chance to consider mempool-cache ZC API, similar to one we have for the ring?
It would allow us on TX free path to avoid copying mbufs to temporary array on the stack.
Instead we can put them straight from TX SW ring to the mempoolcache.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
That should save extra store/load for mbuf and might help to achieve some performance gain without by-passing mempool. It probably wouldn't be as fast as what you proposing, but might
be
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 3 lines
fast enough to consider as alternative.
Again, it would be a generic one, so we can avoid all these implications and limitations.[Feifei] I think this is a good try. However, the most important thing is that if we can bypass the mempool decisively to pursue
the
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
significant performance gains.
I understand the intention, and I personally think this is wrong
and
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
dangerous attitude.
We have mempool abstraction in place for very good reason. So we need to try to improve mempool performance (and API if necessary) at first place, not to avoid it and break our own rulesand
Quoted 2 lines
Quoted 2 lines
recommendations.
Quoted 1 line
The abstraction can be thought of at a higher level. i.e. the
driver manages the
Quoted 1 line
Quoted 1 line
buffer allocation/free and is hidden from the application. The
application does
Quoted 7 lines
Quoted 7 lines
not need to be aware of how these changes are implemented.
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
For ZC, there maybe a problem for it in i40e. The reason for that put Tx buffers into temporary is that
i40e_tx_entryincludesbuffer
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
pointer and index.
Thus we cannot put TxSW_ringentry into mempool directly, weneed
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
to firstlt extract mbuf pointer. Finally, though we use ZC, we
still
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
can't avoid using a temporary stack to extract Tx buffer
pointers.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
When talking about ZC API for mempool cache I meant something
like:
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
void **
mempool_cache_put_zc_start(struct rte_mempool_cache *mc, uint32_t *nb_elem, uint32_t flags); void mempool_cache_put_zc_finish(structrte_mempool_cache*mc,uint32_tnb_elem); i.e. _start_ will return user a pointer inside mp-cache where to put free elems and maxnumber
Quoted 21 lines
Quoted 21 lines
Quoted 21 lines
Quoted 21 lines
of slots that can be safely filled.
_finish_ will updatemc->len.
As an example:/* expect to free N mbufs */
uint32_tn = N;
void **p =mempool_cache_put_zc_start(mc, &n, ...);/* free up to n elems */
for (i = 0; i != n; i++) {/* get next free mbuf from somewhere */ mb = extract_and_prefree_mbuf(...); /* no more free mbufs for now */ if (mb == NULL) break; p[i] = mb; }Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
/* finalize ZC put, with _i_ freed elems */
mempool_cache_put_zc_finish(mc, i);That way, I think we can overcome the issue with
i40e_tx_entryyou mentioned above. Plus it might be useful in other similar places.Another alternative is obviously to split
i40e_tx_entryinto two structs (one for mbuf, second for its metadata) and have aseparate
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
array for each of them.
Though with that approach we need to make sure no perf drops willbe
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 1 line
introduced, plus probably more code changes will be required.
Commit '
5171b4ee6b6" already does this (in a different way), butjust for
Quoted 1 line
Quoted 1 line
AVX512. Unfortunately, it does not record any performance
improvements. We
Quoted 2 lines
Quoted 1 line
could port this to Arm NEON and look at the performance.
Quoted 169 lines
From: Honnappa Nagarahalli [mailto:[email protected]]
Sent: Wednesday, 29 June 2022 23.58<snip>
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
16/05/2022 07:10, Feifei Wang пишет:
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Currently, the transmit side frees the buffers into the
lcore
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
cache and the receive side allocates buffers from the lcore
cache.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
The transmit side typically frees 32 buffers resulting in 32*8=256B of stores to lcore cache. The receive side
allocates
Quoted 6 lines
Quoted 6 lines
32
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
buffers and stores them in the receive side software ring, resulting in 32*8=256B of stores and 256B of load from the
lcore cache.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
This patch proposes a mechanism to avoid freeing
to/allocating
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
from the lcore cache. i.e. the receive side will free the
buffers
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
from transmit side directly into it's software ring. This
will
Quoted 12 lines
Quoted 12 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
avoid the 256B of loads and stores introduced by the lcore
cache.
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
It also frees up the cache lines used by the lcore cache.
However, this solution poses several constraints:
1)The receive queue needs to know which transmit queue it
should
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
take the buffers from. The application logic decides which transmit port to use to send out the packets. In many use cases the NIC might have a single port ([1], [2], [3]), in which case
Quoted 6 lines
Quoted 6 lines
a
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
given transmit queue is always mapped to a single receive queue
(1:1 Rx queue: Tx queue). This is easy to configure.If the NIC has 2 ports (there are several references), then
we
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
will have
1:2 (RX queue: TX queue) mapping which is still easy toconfigure.
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
However, if this is generalized to 'N' ports, the configuration can be long. More over the PMD would have to scan a list of transmit queues to pull the buffers from.
Just to re-iterate some generic concerns about this
proposal:
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
- We effectively link RX and TX queues - when this
feature
Quoted 2 lines
Quoted 2 lines
is enabled,
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
user can't stop TX queue without stopping linked RX
queue
Quoted 4 lines
Quoted 4 lines
first.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Right now user is free to start/stop any queues at his
will.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
If that feature will allow to link queues from
different
Quoted 2 lines
Quoted 2 lines
ports,
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
then even ports will become dependent and user will
have
Quoted 7 lines
Quoted 7 lines
to pay extra
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
care when managing such ports.
[Feifei] When direct rearm enabled, there are two path for thread
to
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
choose. If there are enough Tx freed buffers, Rx can put
buffers
Quoted 6 lines
Quoted 6 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
from Tx.
Otherwise, Rx will put buffers from mempool as usual. Thus, usersdo
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 1 line
not need to pay much attention managing ports.
Quoted 7 lines
Quoted 7 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
What I am talking about: right now different port or different
queues
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
of the same port can be treated as independent entities:
in general user is free to start/stop (and even reconfigure insome
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
cases) one entity without need to stop other entity. I.E user can stop and re-configure TX queue while keep
receiving
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
packets from RX queue.
With direct re-arm enabled, I think it wouldn't be possibleany
Quoted 2 lines
Quoted 2 lines
more:
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
before stopping/reconfiguring TX queue user would have make
sure
Quoted 3 lines
Quoted 3 lines
that
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 1 line
corresponding RX queue wouldn't be used by datapath.
I am trying to understand the problem better. For the TX queue
to
Quoted 10 lines
Quoted 10 lines
be stopped,
Quoted 1 line
Quoted 1 line
the user must have blocked the data plane from accessing the TX
queue.
Quoted 2 lines
Quoted 2 lines
Surely it is user responsibility tnot to call
tx_burst()forstopped/released queue.
Quoted 1 line
Quoted 1 line
The problem is that while TX for that queue is stopped, RX for
related queue still
Quoted 2 lines
Quoted 2 lines
can continue.
Sorx_burst()will try to read/modify TX queue data, that mightbe
Quoted 5 lines
Quoted 5 lines
already freed,
Quoted 4 lines
Quoted 1 line
or simultaneously modified by control path.
Understood, agree on the issue
Quoted 1 line
Quoted 8 lines
Quoted 8 lines
Quoted 1 line
Quoted 1 line
Again, it all can be mitigated by carefully re-designing and
modifying control and
Quoted 1 line
Quoted 1 line
data-path inside user app - by doing extra checks and
synchronizations, etc.
Quoted 1 line
Quoted 1 line
But from practical point - I presume most of users simply would
avoid using this
Quoted 2 lines
Quoted 1 line
feature due all potential problems it might cause.
That is subjective, it all depends on the performance
improvements
Quoted 2 lines
Quoted 2 lines
users see in their application.
Quoted 1 line
IMO, the performance improvement seen with this patch is worth
few
Quoted 7 lines
Quoted 5 lines
changes.
Yes, it is subjective till some extent, though my feeling that it might end-up being sort of synthetic improvement used only by some show-case benchmarks.
I believe that one specific important use case has already been
mentioned, so I
Quoted 1 line
don't think this is a benchmark only feature.
+1
Quoted 8 lines
Quoted 6 lines
From my perspective, it would be much more plausible, if we can
introduce some sort of generic improvement, that doesn't impose all these extra constraints and implications.
Like one, discussed below in that thread with ZC mempool approach.Considering this feature from a high level perspective, I agree with
Konstantin's
Quoted 1 line
concerns, so I'll also support his views.
We did hack the ZC mempool approach [1], level of improvement is pretty small compared with this patch.
[1] http://patches.dpdk.org/project/dpdk/patch/20220613055136.1949784-
[email protected]/Quoted 2 lines
If this patch is supposed to be a generic feature, please add support
for it in all
Quoted 1 line
NIC PMDs, not just one. (Regardless if the feature is defined as 1:1
mapping or
Quoted 1 line
N:M mapping.) It is purely software, so it should be available for
all PMDs, not
Quoted 1 line
just your favorite hardware! Consider the "fast mbuf free" feature,
which is
Quoted 1 line
pure software; why is that feature not implemented in all PMDs?
Agree, it is good to have it supported in all the drivers. We do not have a favorite hardware, just picked a PMD which we are more familiar with. We do plan to implement in other prominent PMDs.
Quoted 1 line
Quoted 7 lines
Quoted 1 line
A secondary point I'm making here is that this specific feature will
lead to an
Quoted 1 line
enormous amount of copy-paste code, instead of a generic library
function
Quoted 1 line
easily available for all PMDs.
Are you talking about the i40e driver code in specific? If yes, agree we should avoid copy-paste and we will look to reduce that.
Yes, I am talking about the code that needs to be copied into all prominent PMDs. Perhaps you can move the majority of it into a common directory, if not in a generic library, so the modification per PMD becomes smaller. (I see the same copy-paste issue with the "fast mbuf free" feature, if to be supported by other than the i40e PMD.)
Please note that I do not expect you to implement this feature in other PMDs than you need. I was trying to make the point that implementing a software feature in a PMD requires copy-pasting to other PMDs, which can require a big effort; while implementing it in a library and calling the library from the PMDs require a smaller effort per PMD. I intentionally phrased it somewhat provokingly, and was lucky not to offend anyone. :-)
Quoted 47 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Like Feifei says, the RX side has the normal packet allocation path
Quoted 7 lines
Quoted 7 lines
still available.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Also this sounds like a corner case to me, we can handle this
through checks in
Quoted 4 lines
Quoted 4 lines
the
queue_stopAPI.Depends.
if it would be allowed to link queues only from the same port,then
Quoted 6 lines
Quoted 6 lines
yes, extra
Quoted 2 lines
Quoted 2 lines
checks for queue-stop might be enough.
As right now DPDK doesn't allow user to change number of queueswithout
Quoted 2 lines
Quoted 2 lines
dev_stop()first.
Though if it would be allowed to link queues from differentports,
Quoted 4 lines
Quoted 4 lines
then situation
Quoted 2 lines
Quoted 2 lines
will be much worse.
Right now ports are totally independent entities (except somespecial cases like
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
link-bonding, etc.).
As one port can keep doing RX/TX, second one can be stopped, re-confgured,
Quoted 1 line
Quoted 1 line
even detached, and newly attached device might re-use same port
number.
Quoted 1 line
I see this as a similar restriction to the one discussed above.
Yes, they are similar in principal, though I think that the case
with
Quoted 10 lines
Quoted 8 lines
queues from different port would make things much more complex.
Quoted 1 line
Do you see any issues if we enforce this with checks?
Hard to tell straightway, a lot will depend how smart such implementation would be.
Usually DPDK tends not to avoid heavy synchronizations within its data-path functions.Certainly! Implementing more and more of such features in the PMDs
will lead
Quoted 1 line
to longer and longer data plane code paths in the PMDs. It is the
Quoted 8 lines
"salami
Quoted 1 line
method", where each small piece makes no performance difference, but
they all
Quoted 1 line
add up, and eventually the sum of them does impact the performance of
the
Quoted 1 line
general use case negatively.
It would be good to have a test running in UNH that shows the performance trend.
+1
Quoted 168 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
- very limited usage scenario - it will have a positive
effect
Quoted 9 lines
Quoted 9 lines
only
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
when we have a fixed forwarding mapping: all (or nearly
all) packets
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
from the RX queue are forwarded into the same TX queue.
[Feifei] Although the usage scenario is limited, this usage
scenario
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 1 line
has a wide range of applications, such as NIC with one port.
Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
yes, there are NICs with one port, but no guarantee there wouldn't
be
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 1 line
several such NICs within the system.
What I see in my interactions is, a single NIC/DPU is under
utilized for a 2
Quoted 1 line
Quoted 1 line
socket system. Some are adding more sockets to the system to
better
Quoted 6 lines
Quoted 6 lines
utilize the
Quoted 1 line
Quoted 1 line
DPU. The NIC bandwidth continues to grow significantly. I do not
think there will
Quoted 3 lines
Quoted 3 lines
be a multi-DPU per server scenario.
Quoted 6 lines
Quoted 6 lines
Quoted 3 lines
Quoted 3 lines
Interesting... from my experience it is visa-versa:
in many cases 200Gb/s is not that much these days to saturate modern2 socket
Quoted 2 lines
Quoted 2 lines
x86 server.
Though I suppose a lot depends on particular HW and actualworkload.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Furtrhermore, I think this is a tradeoff between performance
and
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
flexibility.
Our goal is to achieve best performance, this means we needto
Quoted 2 lines
Quoted 2 lines
give
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
up some flexibility decisively. For example of '
FAST_FREEMode',
Quoted 2 lines
Quoted 2 lines
it
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
deletes most of the buffer check (refcnt > 1, external
buffer,
Quoted 5 lines
Quoted 5 lines
chain
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
buffer), chooses a shorest path, and then achieve significant performance
improvement.
Quoted 1 line
Quoted 1 line
Wonder did you had a chance to consider mempool-cache ZC
API,
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
similar to one we have for the ring?
It would allow us on TX free path to avoid copying mbufs to temporary array on the stack.
Instead we can put them straight from TX SW ring to themempool
Quoted 11 lines
Quoted 11 lines
cache.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
That should save extra store/load for mbuf and might help to achieve some performance gain without by-passing mempool. It probably wouldn't be as fast as what you proposing, but might
be
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
fast enough to consider as alternative.
Again, it would be a generic one, so we can avoid all these implications and limitations.[Feifei] I think this is a good try. However, the most
important
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
thing is that if we can bypass the mempool decisively to
pursue
Quoted 4 lines
Quoted 4 lines
the
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
significant performance gains.
I understand the intention, and I personally think this is
wrong
Quoted 16 lines
Quoted 16 lines
and
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
dangerous attitude.
We have mempool abstraction in place for very good reason. So we need to try to improve mempool performance (and API if necessary) at first place, not to avoid it and break our own rulesand
Quoted 2 lines
Quoted 2 lines
recommendations.
Quoted 1 line
The abstraction can be thought of at a higher level. i.e. the
driver manages the
Quoted 1 line
Quoted 1 line
buffer allocation/free and is hidden from the application. The
application does
Quoted 4 lines
Quoted 4 lines
not need to be aware of how these changes are implemented.
Quoted 3 lines
Quoted 3 lines
Quoted 8 lines
Quoted 8 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
For ZC, there maybe a problem for it in i40e. The reason for that put Tx buffers into temporary is that
i40e_tx_entryincludesbuffer
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
pointer and index.
Thus we cannot put TxSW_ringentry into mempool directly, weneed
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
to firstlt extract mbuf pointer. Finally, though we use ZC,
we
Quoted 7 lines
Quoted 7 lines
still
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
can't avoid using a temporary stack to extract Tx buffer
pointers.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
When talking about ZC API for mempool cache I meant something
like:
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
void ** mempool_cache_put_zc_start(struct
rte_mempool_cache*mc,
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
uint32_t*nb_elem,uint32_tflags); voidmempool_cache_put_zc_finish(struct rte_mempool_cache *mc, uint32_t nb_elem); i.e. _start_ will return user a pointer inside mp-cache where to put free elemsand
Quoted 20 lines
Quoted 20 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
max
number
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
of slots that can be safely filled.
_finish_ will updatemc->len.
As an example:/* expect to free N mbufs */
uint32_tn = N;
void **p =mempool_cache_put_zc_start(mc, &n, ...);/* free up to n elems */
for (i = 0; i != n; i++) {/* get next free mbuf from somewhere */ mb = extract_and_prefree_mbuf(...); /* no more free mbufs for now */ if (mb == NULL) break;Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
p[i] = mb; }/* finalize ZC put, with _i_ freed elems */
mempool_cache_put_zc_finish(mc, i);That way, I think we can overcome the issue with
i40e_tx_entryyou mentioned above. Plus it might be useful in other similarplaces.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Another alternative is obviously to split
i40e_tx_entryintotwo
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
structs (one for mbuf, second for its metadata) and have a
separate
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
array for each of them.
Though with that approach we need to make sure no perf dropswill
Quoted 3 lines
Quoted 3 lines
be
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 1 line
introduced, plus probably more code changes will be required.
Commit '
5171b4ee6b6" already does this (in a different way),but
Quoted 6 lines
Quoted 6 lines
just for
Quoted 1 line
Quoted 1 line
AVX512. Unfortunately, it does not record any performance
improvements. We
Quoted 2 lines
Quoted 1 line
could port this to Arm NEON and look at the performance.
<snip>
Quoted 175 lines
Quoted 175 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
16/05/2022 07:10, Feifei Wang пишет:
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Currently, the transmit side frees the buffers into the
lcore
Quoted 5 lines
Quoted 5 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
cache and the receive side allocates buffers from the lcore
cache.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
The transmit side typically frees 32 buffers resulting in 32*8=256B of stores to lcore cache. The receive side
allocates
Quoted 6 lines
Quoted 6 lines
32
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
buffers and stores them in the receive side software ring, resulting in 32*8=256B of stores and 256B of load from the
lcore cache.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
This patch proposes a mechanism to avoid freeing
to/allocating
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
from the lcore cache. i.e. the receive side will free the
buffers
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
from transmit side directly into it's software ring. This
will
Quoted 12 lines
Quoted 12 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
avoid the 256B of loads and stores introduced by the lcore
cache.
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
It also frees up the cache lines used by the lcore cache.
However, this solution poses several constraints:
1)The receive queue needs to know which transmit queue it
should
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
take the buffers from. The application logic decides which transmit port to use to send out the packets. In many use cases the NIC might have a single port ([1], [2], [3]), in which case
Quoted 7 lines
Quoted 7 lines
a
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
given transmit queue is always mapped to a single receive queue
(1:1 Rx queue: Tx queue). This is easy to configure.If the NIC has 2 ports (there are several references), then
we
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
will have
1:2 (RX queue: TX queue) mapping which is still easy toconfigure.
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
However, if this is generalized to 'N' ports, the configuration can be long. More over the PMD would have to scan a list of transmit queues to pull the buffers from.
Just to re-iterate some generic concerns about this
proposal:
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
- We effectively link RX and TX queues - when this
feature
Quoted 2 lines
Quoted 2 lines
is enabled,
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
user can't stop TX queue without stopping linked RX
queue
Quoted 5 lines
Quoted 5 lines
first.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Right now user is free to start/stop any queues at
hiswill.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
If that feature will allow to link queues from
different
Quoted 2 lines
Quoted 2 lines
ports,
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
then even ports will become dependent and user will
have
Quoted 7 lines
Quoted 7 lines
to pay extra
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
care when managing such ports.
[Feifei] When direct rearm enabled, there are two path for thread
to
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
choose. If there are enough Tx freed buffers, Rx can put
buffers
Quoted 8 lines
Quoted 8 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
from Tx.
Otherwise, Rx will put buffers from mempool as usual. Thus, usersdo
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
not need to pay much attention managing ports.
What I am talking about: right now different port or different
Quoted 7 lines
Quoted 7 lines
queues
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
of the same port can be treated as independent entities:
in general user is free to start/stop (and even reconfigure insome
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
cases) one entity without need to stop other entity. I.E user can stop and re-configure TX queue while keep
receiving
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
packets from RX queue.
With direct re-arm enabled, I think it wouldn't be possibleany
Quoted 2 lines
Quoted 2 lines
more:
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
before stopping/reconfiguring TX queue user would have make
sure
Quoted 3 lines
Quoted 3 lines
that
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 1 line
corresponding RX queue wouldn't be used by datapath.
I am trying to understand the problem better. For the TX queue
to
Quoted 10 lines
Quoted 10 lines
be stopped,
Quoted 1 line
Quoted 1 line
the user must have blocked the data plane from accessing the TX
queue.
Quoted 2 lines
Quoted 2 lines
Surely it is user responsibility tnot to call
tx_burst()forstopped/released queue.
Quoted 1 line
Quoted 1 line
The problem is that while TX for that queue is stopped, RX for
related queue still
Quoted 2 lines
Quoted 2 lines
can continue.
Sorx_burst()will try to read/modify TX queue data, that mightbe
Quoted 5 lines
Quoted 5 lines
already freed,
Quoted 4 lines
Quoted 1 line
or simultaneously modified by control path.
Understood, agree on the issue
Quoted 1 line
Quoted 8 lines
Quoted 8 lines
Quoted 1 line
Quoted 1 line
Again, it all can be mitigated by carefully re-designing and
modifying control and
Quoted 1 line
Quoted 1 line
data-path inside user app - by doing extra checks and
synchronizations, etc.
Quoted 1 line
Quoted 1 line
But from practical point - I presume most of users simply would
avoid using this
Quoted 2 lines
Quoted 1 line
feature due all potential problems it might cause.
That is subjective, it all depends on the performance
improvements
Quoted 2 lines
Quoted 2 lines
users see in their application.
Quoted 1 line
IMO, the performance improvement seen with this patch is worth
few
Quoted 7 lines
Quoted 5 lines
changes.
Yes, it is subjective till some extent, though my feeling that it might end-up being sort of synthetic improvement used only by some show-case benchmarks.
I believe that one specific important use case has already been
mentioned, so I
Quoted 1 line
don't think this is a benchmark only feature.
+1
Quoted 8 lines
Quoted 6 lines
From my perspective, it would be much more plausible, if we can
introduce some sort of generic improvement, that doesn't impose all these extra constraints and implications.
Like one, discussed below in that thread with ZC mempool approach.Considering this feature from a high level perspective, I agree with
Konstantin's
Quoted 1 line
concerns, so I'll also support his views.
We did hack the ZC mempool approach [1], level of improvement is pretty small compared with this patch.
[1] http://patches.dpdk.org/project/dpdk/patch/20220613055136.1949784-
[email protected]/Quoted 3 lines
If this patch is supposed to be a generic feature, please add support
for it in all
Quoted 1 line
NIC PMDs, not just one. (Regardless if the feature is defined as 1:1
mapping or
Quoted 1 line
N:M mapping.) It is purely software, so it should be available for
all PMDs, not
Quoted 1 line
just your favorite hardware! Consider the "fast mbuf free" feature,
which is
Quoted 1 line
pure software; why is that feature not implemented in all PMDs?
Agree, it is good to have it supported in all the drivers. We do not have a favorite hardware, just picked a PMD which we are more familiar with. We do plan to implement in other prominent PMDs.
Quoted 2 lines
A secondary point I'm making here is that this specific feature will
lead to an
Quoted 1 line
enormous amount of copy-paste code, instead of a generic library
function
Quoted 1 line
easily available for all PMDs.
Quoted 8 lines
Quoted 2 lines
Are you talking about the i40e driver code in specific? If yes, agree we should avoid copy-paste and we will look to reduce that.
Yes, I am talking about the code that needs to be copied into all prominent PMDs. Perhaps you can move the majority of it into a common directory, if not in a generic library, so the modification per PMD becomes smaller. (I see the same copy-paste issue with the "fast mbuf free" feature, if to be supported by other than the i40e PMD.)
The current abstraction does not allow for common code at this (lower) level across all the PMDs. If we look at "fast free", it is accessing the device private structure for the list of buffers to free. If it needs to be common code, this needs to be lifted up along with other dependent configuration thresholds etc.
Quoted 239 lines
Please note that I do not expect you to implement this feature in other PMDs than you need. I was trying to make the point that implementing a software feature in a PMD requires copy-pasting to other PMDs, which can require a big effort; while implementing it in a library and calling the library from the PMDs require a smaller effort per PMD. I intentionally phrased it somewhat provokingly, and was lucky not to offend anyone. :-)
Quoted 49 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Like Feifei says, the RX side has the normal packet allocation path
Quoted 7 lines
Quoted 7 lines
still available.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Also this sounds like a corner case to me, we can handle this
through checks in
Quoted 4 lines
Quoted 4 lines
the
queue_stopAPI.Depends.
if it would be allowed to link queues only from the same port,then
Quoted 6 lines
Quoted 6 lines
yes, extra
Quoted 2 lines
Quoted 2 lines
checks for queue-stop might be enough.
As right now DPDK doesn't allow user to change number of queueswithout
Quoted 2 lines
Quoted 2 lines
dev_stop()first.
Though if it would be allowed to link queues from differentports,
Quoted 7 lines
Quoted 7 lines
then situation
Quoted 2 lines
Quoted 2 lines
will be much worse.
Right now ports are totally independent entities (except somespecial cases like
Quoted 3 lines
Quoted 3 lines
link-bonding, etc.).
As one port can keep doing RX/TX, second one can be stopped, re-Quoted 6 lines
Quoted 6 lines
confgured,
Quoted 1 line
Quoted 1 line
even detached, and newly attached device might re-use same port
number.
Quoted 1 line
I see this as a similar restriction to the one discussed above.
Yes, they are similar in principal, though I think that the case
with
Quoted 10 lines
Quoted 8 lines
queues from different port would make things much more complex.
Quoted 1 line
Do you see any issues if we enforce this with checks?
Hard to tell straightway, a lot will depend how smart such implementation would be.
Usually DPDK tends not to avoid heavy synchronizations within its data-path functions.Certainly! Implementing more and more of such features in the PMDs
will lead
Quoted 1 line
to longer and longer data plane code paths in the PMDs. It is the
"salami
Quoted 8 lines
Quoted 1 line
method", where each small piece makes no performance difference, but
they all
Quoted 2 lines
add up, and eventually the sum of them does impact the performance of
the
Quoted 1 line
general use case negatively.
It would be good to have a test running in UNH that shows the performance trend.
+1
Quoted 172 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
- very limited usage scenario - it will have a positive
effect
Quoted 10 lines
Quoted 10 lines
only
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
when we have a fixed forwarding mapping: all (or
nearlyall) packets
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
from the RX queue are forwarded into the same TX queue.
[Feifei] Although the usage scenario is limited, this usage
scenario
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 1 line
has a wide range of applications, such as NIC with one port.
Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
yes, there are NICs with one port, but no guarantee there wouldn't
be
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 1 line
several such NICs within the system.
What I see in my interactions is, a single NIC/DPU is under
utilized for a 2
Quoted 1 line
Quoted 1 line
socket system. Some are adding more sockets to the system to
better
Quoted 7 lines
Quoted 7 lines
utilize the
Quoted 2 lines
Quoted 2 lines
DPU. The NIC bandwidth continues to grow significantly. I do not
think there will
Quoted 3 lines
Quoted 3 lines
be a multi-DPU per server scenario.
Quoted 6 lines
Quoted 6 lines
Quoted 3 lines
Quoted 3 lines
Interesting... from my experience it is visa-versa:
in many cases 200Gb/s is not that much these days to saturate modern2 socket
Quoted 2 lines
Quoted 2 lines
x86 server.
Though I suppose a lot depends on particular HW and actualworkload.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Furtrhermore, I think this is a tradeoff between performance
and
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
flexibility.
Our goal is to achieve best performance, this means we needto
Quoted 2 lines
Quoted 2 lines
give
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
up some flexibility decisively. For example of '
FAST_FREEMode',
Quoted 2 lines
Quoted 2 lines
it
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
deletes most of the buffer check (refcnt > 1, external
buffer,
Quoted 5 lines
Quoted 5 lines
chain
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
buffer), chooses a shorest path, and then achieve
significant performanceimprovement.
Quoted 1 line
Quoted 1 line
Wonder did you had a chance to consider mempool-cache ZC
API,
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
similar to one we have for the ring?
It would allow us on TX free path to avoid copying mbufs to temporary array on the stack.
Instead we can put them straight from TX SW ring to themempool
Quoted 11 lines
Quoted 11 lines
cache.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
That should save extra store/load for mbuf and might help to achieve some performance gain without by-passing mempool. It probably wouldn't be as fast as what you proposing, but might
be
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
fast enough to consider as alternative.
Again, it would be a generic one, so we can avoid all these implications and limitations.[Feifei] I think this is a good try. However, the most
important
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
thing is that if we can bypass the mempool decisively to
pursue
Quoted 4 lines
Quoted 4 lines
the
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
significant performance gains.
I understand the intention, and I personally think this is
wrong
Quoted 19 lines
Quoted 19 lines
and
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
dangerous attitude.
We have mempool abstraction in place for very good reason. So we need to try to improve mempool performance (and API if necessary) at first place, not to avoid it and break our own rulesand
Quoted 2 lines
Quoted 2 lines
recommendations.
Quoted 1 line
The abstraction can be thought of at a higher level. i.e. the
driver manages the
Quoted 1 line
Quoted 1 line
buffer allocation/free and is hidden from the application. The
application does
Quoted 7 lines
Quoted 7 lines
not need to be aware of how these changes are implemented.
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
For ZC, there maybe a problem for it in i40e. The reason for that put Tx buffers into temporary is that
i40e_tx_entryincludesQuoted 6 lines
Quoted 6 lines
buffer
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
pointer and index.
Thus we cannot put TxSW_ringentry into mempool directly, weneed
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
to firstlt extract mbuf pointer. Finally, though we use ZC,
we
Quoted 7 lines
Quoted 7 lines
still
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
can't avoid using a temporary stack to extract Tx buffer
pointers.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
When talking about ZC API for mempool cache I meant something
like:
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
void ** mempool_cache_put_zc_start(struct
rte_mempool_cache*mc,
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
uint32_t*nb_elem,uint32_tflags); voidmempool_cache_put_zc_finish(struct rte_mempool_cache *mc, uint32_t nb_elem); i.e. _start_ will return user a pointer inside mp-cache where to put free elemsand
Quoted 23 lines
Quoted 23 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
max
number
Quoted 21 lines
Quoted 21 lines
Quoted 21 lines
Quoted 21 lines
of slots that can be safely filled.
_finish_ will updatemc->len.
As an example:/* expect to free N mbufs */
uint32_tn = N;
void **p =mempool_cache_put_zc_start(mc, &n, ...);/* free up to n elems */
for (i = 0; i != n; i++) {/* get next free mbuf from somewhere */ mb = extract_and_prefree_mbuf(...); /* no more free mbufs for now */ if (mb == NULL) break; p[i] = mb; }Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
/* finalize ZC put, with _i_ freed elems */
mempool_cache_put_zc_finish(mc, i);That way, I think we can overcome the issue with
i40e_tx_entryyou mentioned above. Plus it might be useful in other similarplaces.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Another alternative is obviously to split
i40e_tx_entryintotwo
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
structs (one for mbuf, second for its metadata) and have a
separate
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
array for each of them.
Though with that approach we need to make sure no perf dropswill
Quoted 3 lines
Quoted 3 lines
be
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 1 line
introduced, plus probably more code changes will be required.
Commit '
5171b4ee6b6" already does this (in a different way),but
Quoted 6 lines
Quoted 6 lines
just for
Quoted 1 line
Quoted 1 line
AVX512. Unfortunately, it does not record any performance
improvements. We
Quoted 2 lines
Quoted 1 line
could port this to Arm NEON and look at the performance.
Quoted 229 lines
From: Honnappa Nagarahalli [mailto:[email protected]]
Sent: Friday, 1 July 2022 21.31<snip>
Quoted 9 lines
Quoted 9 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
16/05/2022 07:10, Feifei Wang пишет:
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Currently, the transmit side frees the buffers into the
lcore
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
cache and the receive side allocates buffers from the lcore
cache.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
The transmit side typically frees 32 buffers resulting
in
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
32*8=256B of stores to lcore cache. The receive side
allocates
Quoted 2 lines
Quoted 2 lines
32
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
buffers and stores them in the receive side software
ring,
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
resulting in 32*8=256B of stores and 256B of load from
the
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
Quoted 3 lines
lcore cache.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
This patch proposes a mechanism to avoid freeing
to/allocating
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
from the lcore cache. i.e. the receive side will free
the
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
buffers
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
from transmit side directly into it's software ring.
This
Quoted 2 lines
Quoted 2 lines
will
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
avoid the 256B of loads and stores introduced by the
lcore
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
cache.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
It also frees up the cache lines used by the lcore
cache.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
However, this solution poses several constraints:
1)The receive queue needs to know which transmit queue
it
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
should
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
take the buffers from. The application logic decides
which
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
transmit port to use to send out the packets. In many
use
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
cases the NIC might have a single port ([1], [2], [3]),
in
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
which case
a
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
given transmit queue is always mapped to a single
receive
Quoted 11 lines
Quoted 11 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
queue
(1:1 Rx queue: Tx queue). This is easy to configure.If the NIC has 2 ports (there are several references), then
we
Quoted 5 lines
Quoted 5 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
will have
1:2 (RX queue: TX queue) mapping which is still easy toconfigure.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
However, if this is generalized to 'N' ports, the configuration can be long. More over the PMD would have
to
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
scan a list of transmit queues to pull the buffers
from.
Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Just to re-iterate some generic concerns about this
proposal:
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
- We effectively link RX and TX queues - when this
feature
Quoted 2 lines
Quoted 2 lines
is enabled,
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
user can't stop TX queue without stopping linked
RX
Quoted 9 lines
Quoted 9 lines
queue
Quoted 5 lines
Quoted 5 lines
first.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Right now user is free to start/stop any queues at
hiswill.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
If that feature will allow to link queues from
different
Quoted 2 lines
Quoted 2 lines
ports,
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
then even ports will become dependent and user
will
Quoted 5 lines
Quoted 5 lines
have
Quoted 4 lines
Quoted 4 lines
to pay extra
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
care when managing such ports.
[Feifei] When direct rearm enabled, there are two path
for
Quoted 6 lines
Quoted 6 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
thread
to
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
choose. If there are enough Tx freed buffers, Rx can put
buffers
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
from Tx.
Otherwise, Rx will put buffers from mempool as usual.Thus,
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
users
do
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
not need to pay much attention managing ports.
What I am talking about: right now different port or different
queues
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
of the same port can be treated as independent entities:
in general user is free to start/stop (and evenreconfigure
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
in
some
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
cases) one entity without need to stop other entity. I.E user can stop and re-configure TX queue while keep
receiving
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
packets from RX queue.
With direct re-arm enabled, I think it wouldn't bepossible
Quoted 3 lines
Quoted 3 lines
any
Quoted 2 lines
Quoted 2 lines
more:
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
before stopping/reconfiguring TX queue user would have
make
Quoted 4 lines
Quoted 4 lines
sure
Quoted 3 lines
Quoted 3 lines
that
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 1 line
corresponding RX queue wouldn't be used by datapath.
I am trying to understand the problem better. For the TX
queue
Quoted 3 lines
Quoted 3 lines
to
Quoted 2 lines
Quoted 2 lines
be stopped,
Quoted 1 line
Quoted 1 line
the user must have blocked the data plane from accessing the
TX
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
queue.
Quoted 2 lines
Quoted 2 lines
Surely it is user responsibility tnot to call
tx_burst()forstopped/released queue.
Quoted 1 line
Quoted 1 line
The problem is that while TX for that queue is stopped, RX
for
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
related queue still
Quoted 2 lines
Quoted 2 lines
can continue.
Sorx_burst()will try to read/modify TX queue data, thatmight
Quoted 11 lines
Quoted 11 lines
be
Quoted 10 lines
Quoted 10 lines
already freed,
Quoted 5 lines
Quoted 1 line
or simultaneously modified by control path.
Understood, agree on the issue
Quoted 2 lines
Again, it all can be mitigated by carefully re-designing and
modifying control and
Quoted 1 line
Quoted 1 line
data-path inside user app - by doing extra checks and
synchronizations, etc.
Quoted 1 line
Quoted 1 line
But from practical point - I presume most of users simply
would
Quoted 6 lines
Quoted 6 lines
Quoted 3 lines
Quoted 3 lines
avoid using this
Quoted 2 lines
Quoted 1 line
feature due all potential problems it might cause.
That is subjective, it all depends on the performance
improvements
Quoted 2 lines
Quoted 2 lines
users see in their application.
Quoted 1 line
IMO, the performance improvement seen with this patch is
worth
Quoted 4 lines
Quoted 4 lines
few
Quoted 3 lines
Quoted 3 lines
changes.
Yes, it is subjective till some extent, though my feeling that
it
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
might end-up being sort of synthetic improvement used only by
some
Quoted 9 lines
Quoted 9 lines
Quoted 3 lines
Quoted 1 line
show-case benchmarks.
I believe that one specific important use case has already been
mentioned, so I
Quoted 1 line
don't think this is a benchmark only feature.
+1
Quoted 2 lines
Quoted 2 lines
From my perspective, it would be much more plausible, if we
can
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
introduce some sort of generic improvement, that doesn't impose all these extra constraints and implications.
Like one, discussed below in that thread with ZC mempoolapproach.
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Considering this feature from a high level perspective, I agree
with
Quoted 6 lines
Quoted 6 lines
Konstantin's
Quoted 1 line
concerns, so I'll also support his views.
We did hack the ZC mempool approach [1], level of improvement is pretty small compared with this patch.
[1]
http://patches.dpdk.org/project/dpdk/patch/20220613055136.1949784-
Quoted 7 lines
Quoted 7 lines
Quoted 3 lines
If this patch is supposed to be a generic feature, please add support
for it in all
Quoted 1 line
NIC PMDs, not just one. (Regardless if the feature is defined as
1:1
Quoted 2 lines
Quoted 2 lines
mapping or
Quoted 1 line
N:M mapping.) It is purely software, so it should be available
for
Quoted 2 lines
Quoted 2 lines
all PMDs, not
Quoted 1 line
just your favorite hardware! Consider the "fast mbuf free"
feature,
Quoted 3 lines
Quoted 3 lines
which is
Quoted 1 line
pure software; why is that feature not implemented in all PMDs?
Agree, it is good to have it supported in all the drivers. We do
not
Quoted 1 line
Quoted 1 line
have a favorite hardware, just picked a PMD which we are more
familiar
Quoted 4 lines
Quoted 4 lines
with. We do plan to implement in other prominent PMDs.
Quoted 2 lines
A secondary point I'm making here is that this specific feature
will
Quoted 5 lines
Quoted 5 lines
lead to an
Quoted 1 line
enormous amount of copy-paste code, instead of a generic library
function
Quoted 1 line
easily available for all PMDs.
Are you talking about the i40e driver code in specific? If yes,
agree
Quoted 3 lines
Quoted 1 line
we should avoid copy-paste and we will look to reduce that.
Yes, I am talking about the code that needs to be copied into all
prominent
Quoted 1 line
PMDs. Perhaps you can move the majority of it into a common
directory, if not
Quoted 1 line
in a generic library, so the modification per PMD becomes smaller. (I
see the
Quoted 8 lines
Quoted 1 line
same copy-paste issue with the "fast mbuf free" feature, if to be
supported by
Quoted 1 line
other than the i40e PMD.)
The current abstraction does not allow for common code at this (lower) level across all the PMDs. If we look at "fast free", it is accessing the device private structure for the list of buffers to free. If it needs to be common code, this needs to be lifted up along with other dependent configuration thresholds etc.
Exactly. The "direct re-arm" feature has some of the same design properties as "fast free": They are both purely software features, and they are both embedded deeply in the PMD code.
I don't know if it is possible, but perhaps we could redesign the private buffer structures of the PMDs, so the first part of it is common across PMDs, and the following part is truly private. The common part would obviously hold the mbuf pointer. That way, a common library could manipulate the public part.
Or perhaps some other means could be provided for a common library to manipulate the common parts of a private structure, e.g. a common fast_free function could take a few parameters to work on private buffer structures: void * buffer_array, size_t element_size, size_t mbuf_offset_in_element, unsigned int num_elements.
Quoted 12 lines
Quoted 3 lines
Please note that I do not expect you to implement this feature in
other PMDs
Quoted 1 line
than you need. I was trying to make the point that implementing a
software
Quoted 1 line
feature in a PMD requires copy-pasting to other PMDs, which can
require a big
Quoted 1 line
effort; while implementing it in a library and calling the library
from the PMDs
Quoted 2 lines
require a smaller effort per PMD. I intentionally phrased it somewhat provokingly, and was lucky not to offend anyone. :-)
Unless we find a better solution, copy-paste code across PMDs seems to be the only way to implement such features. And I agree that they should not be blocked due to "code complexity" or "copy-paste" arguments, if they are impossible to implement in a more "correct" way.
The DPDK community should accept such contributions to the common code, and be grateful that they don't just go into private forks of DPDK!
Quoted 147 lines
-----邮件原件-----
发件人: Konstantin Ananyev <[email protected]> 发送时间: Tuesday, May 24, 2022 9:26 AM
收件人: Feifei Wang <[email protected]>
抄送: nd <[email protected]>; [email protected]; Ruifeng Wang <[email protected]>; Honnappa Nagarahalli <[email protected]>
主题: Re: [PATCH v1 0/5] Direct re-arming of buffers on receive side[[email protected] appears similar to someone who previously sent you email, but may not be that person. Learn why this could be a risk at
https://aka.ms/LearnAboutSenderIdentification.]16/05/2022 07:10, Feifei Wang пишет:
Quoted 38 lines
Quoted 30 lines
Quoted 23 lines
Currently, the transmit side frees the buffers into the lcore cache and the receive side allocates buffers from the lcore cache. The transmit side typically frees 32 buffers resulting in 32*8=256B of stores to lcore cache. The receive side allocates 32 buffers and stores them in the receive side software ring, resulting in 32*8=256B of stores and 256B of load from the lcore cache.
This patch proposes a mechanism to avoid freeing to/allocating from the lcore cache. i.e. the receive side will free the buffers from transmit side directly into it's software ring. This will avoid the 256B of loads and stores introduced by the lcore cache. It also frees up the cache lines used by the lcore cache.
However, this solution poses several constraints:
1)The receive queue needs to know which transmit queue it should take the buffers from. The application logic decides which transmit port to use to send out the packets. In many use cases the NIC might have a single port ([1], [2], [3]), in which case a given transmit queue is always mapped to a single receive queue (1:1 Rx queue: Tx queue). This is easy to configure.
Quoted 6 lines
If the NIC has 2 ports (there are several references), then we will have
1:2 (RX queue: TX queue) mapping which is still easy to configure. However, if this is generalized to 'N' ports, the configuration can be long. More over the PMD would have to scan a list of transmit queues to pull the buffers from.Quoted 7 lines
Just to re-iterate some generic concerns about this proposal:
- We effectively link RX and TX queues - when this feature is enabled, user can't stop TX queue without stopping linked RX queue first. Right now user is free to start/stop any queues at his will. If that feature will allow to link queues from different ports, then even ports will become dependent and user will have to pay extra care when managing such ports.Quoted 5 lines
[Feifei] When direct rearm enabled, there are two path for thread to choose. If there are enough Tx freed buffers, Rx can put buffers from Tx.
Otherwise, Rx will put buffers from mempool as usual. Thus, users do not need to pay much attention managing ports.What I am talking about: right now different port or different queues of the same port can be treated as independent entities:
in general user is free to start/stop (and even reconfigure in some cases) one entity without need to stop other entity.
I.E user can stop and re-configure TX queue while keep receiving packets from RX queue.
With direct re-arm enabled, I think it wouldn't be possible any more:
before stopping/reconfiguring TX queue user would have make sure that corresponding RX queue wouldn't be used by datapath.Quoted 7 lines
Quoted 4 lines
- very limited usage scenario - it will have a positive effect only when we have a fixed forwarding mapping: all (or nearly all) packets from the RX queue are forwarded into the same TX queue.
[Feifei] Although the usage scenario is limited, this usage scenario has a wide range of applications, such as NIC with one port.
yes, there are NICs with one port, but no guarantee there wouldn't be several such NICs within the system.
Quoted 6 lines
Furtrhermore, I think this is a tradeoff between performance and flexibility.
Our goal is to achieve best performance, this means we need to give up some flexibility decisively. For example of 'FAST_FREEMode', it deletes most of the buffer check (refcnt > 1, external buffer, chain buffer), chooses a shorest path, and then achieve significant performanceimprovement.
Quoted 12 lines
Quoted 11 lines
Wonder did you had a chance to consider mempool-cache ZC API, similar to one we have for the ring?
It would allow us on TX free path to avoid copying mbufs to temporary array on the stack.
Instead we can put them straight from TX SW ring to the mempool cache. That should save extra store/load for mbuf and might help to achieve some performance gain without by-passing mempool.
It probably wouldn't be as fast as what you proposing, but might be fast enough to consider as alternative.
Again, it would be a generic one, so we can avoid all these implications and limitations.Quoted 3 lines
[Feifei] I think this is a good try. However, the most important thing is that if we can bypass the mempool decisively to pursue the significant performance gains.
I understand the intention, and I personally think this is wrong and dangerous attitude.
We have mempool abstraction in place for very good reason.
So we need to try to improve mempool performance (and API if necessary) at first place, not to avoid it and break our own rules and recommendations.Quoted 6 lines
For ZC, there maybe a problem for it in i40e. The reason for that put Tx buffers into temporary is that
i40e_tx_entryincludes buffer pointer and index.
Thus we cannot put TxSW_ringentry into mempool directly, we need to firstlt extract mbuf pointer. Finally, though we use ZC, we still can't avoid using a temporary stack to extract Tx buffer pointers.When talking about ZC API for mempool cache I meant something like:
void **mempool_cache_put_zc_start(struct rte_mempool_cache *mc, uint32_t *nb_elem, uint32_t flags); voidmempool_cache_put_zc_finish(struct rte_mempool_cache *mc, uint32_t nb_elem); i.e. _start_ will return user a pointer inside mp-cache where to put free elems and max number of slots that can be safely filled. _finish_ will updatemc->len.
As an example:/* expect to free N mbufs */
uint32_tn = N;
void **p =mempool_cache_put_zc_start(mc, &n, ...);/* free up to n elems */
for (i = 0; i != n; i++) {/* get next free mbuf from somewhere */ mb = extract_and_prefree_mbuf(...); /* no more free mbufs for now */ if (mb == NULL) break; p[i] = mb; }/* finalize ZC put, with _i_ freed elems */
mempool_cache_put_zc_finish(mc, i);
Quoted 7 lines
That way, I think we can overcome the issue with
i40e_tx_entryyou mentioned above. Plus it might be useful in other similar places.Another alternative is obviously to split
i40e_tx_entryinto two structs (one for mbuf, second for its metadata) and have a separate array for each of them. Though with that approach we need to make sure no perf drops will be introduced, plus probably more code changes will be required.
[Feifei] I just uploaded a RFC patch to the community:
http://patches.dpdk.org/project/dpdk/patch/[email protected]/
I do not use ZC api, and refer to i40e avx512 path, directly put mempool cache out of API. You can take a look when you are free. Thanks very much.
Quoted 4 lines
v3 cover Direct re-arming of buffers on receive side
In reply to Feifei WangCurrently, the transmit side frees the buffers into the lcore cache and the receive side allocates buffers from the lcore cache. The transmit side typically frees 32 buffers resulting in 32*8=256B of stores to lcore cache. The receive side allocates 32 buffers and stores them in the receive side software ring, resulting in 32*8=256B of stores and 256B of load from the lcore cache.
This patch proposes a mechanism to avoid freeing to/allocating from the lcore cache. i.e. the receive side will free the buffers from transmit side directly into its software ring. This will avoid the 256B of loads and stores introduced by the lcore cache. It also frees up the cache lines used by the lcore cache.
However, this solution poses several constraints:
1)The receive queue needs to know which transmit queue it should take the buffers from. The application logic decides which transmit port to use to send out the packets. In many use cases the NIC might have a single port ([1], [2], [3]), in which case a given transmit queue is always mapped to a single receive queue (1:1 Rx queue: Tx queue). This is easy to configure.
If the NIC has 2 ports (there are several references), then we will have 1:2 (RX queue: TX queue) mapping which is still easy to configure. However, if this is generalized to 'N' ports, the configuration can be long. More over the PMD would have to scan a list of transmit queues to pull the buffers from.
2)The other factor that needs to be considered is 'run-to-completion' vs 'pipeline' models. In the run-to-completion model, the receive side and the transmit side are running on the same lcore serially. In the pipeline model. The receive side and transmit side might be running on different lcores in parallel. This requires locking. This is not supported at this point.
3)Tx and Rx buffers must be from the same mempool. And we also must ensure Tx buffer free number is equal to Rx buffer free number. Thus, 'tx_next_dd' can be updated correctly in direct-rearm mode. This is due to tx_next_dd is a variable to compute tx sw-ring free location. Its value will be one more round than the position where next time free starts.
Current status in this patch:
1)Two APIs are added for users to enable direct-rearm mode:
In control plane, users can call 'rte_eth_rx_queue_rearm_data_get' to get Rx sw_ring pointer and its rxq_info.
(This avoid Tx load Rx data directly);
In data plane, users can call 'rte_eth_dev_direct_rearm' to rearm Rx buffers and free Tx buffers at the same time. Specifically, in this API, there are two separated API for Rx and Tx.
For Tx, 'rte_eth_tx_fill_sw_ring' can fill a given sw_ring by Tx freed buffers. For Rx, 'rte_eth_rx_flush_descriptor' can flush its descriptors based on the rearm buffers.
Thus, this can separate Rx and Tx operation, and user can even re-arm RX queue not from the same driver's TX queue, but from different sources too.
-----------------------------------------------------------------------
control plane:
rte_eth_rx_queue_rearm_data_get(*rxq_rearm_data);
data plane:
loop {
rte_eth_dev_direct_rearm(*rxq_rearm_data){
rte_eth_tx_fill_sw_ring{
for (i = 0; i <= 32; i++) {
sw_ring.mbuf[i] = tx.mbuf[i];
}
}
rte_eth_rx_flush_descriptor{
for (i = 0; i <= 32; i++) {
flush descs[i];
}
}
}
rte_eth_rx_burst;
rte_eth_tx_burst; }
-----------------------------------------------------------------------
2)The i40e driver is changed to do the direct re-arm of the receive
side.
3)The ixgbe driver is changed to do the direct re-arm of the receive
side.
Testing status:
(1) dpdk l3fwd test with multiple drivers:
port 0: 82599 NIC port 1: XL710 NIC
-------------------------------------------------------------
Without fast free With fast free
Thunderx2: +9.44% +7.14%
-------------------------------------------------------------
(2) dpdk l3fwd test with same driver:
port 0 && 1: XL710 NIC
-------------------------------------------------------------
*Direct rearm with exposing rx_sw_ring:
Without fast free With fast free
Ampere altra: +14.98% +15.77%
n1sdp: +6.47% +0.52%
-------------------------------------------------------------
(3) VPP test with same driver:
port 0 && 1: XL710 NIC
-------------------------------------------------------------
*Direct rearm with exposing rx_sw_ring:
Ampere altra: +4.59%
n1sdp: +5.4%
-------------------------------------------------------------
Reference:
[1] https://store.nvidia.com/en-us/networking/store/product/MCX623105AN-CDAT/NVIDIAMCX623105ANCDATConnectX6DxENAdapterCard100GbECryptoDisabled/
[2] https://www.intel.com/content/www/us/en/products/sku/192561/intel-ethernet-network-adapter-e810cqda1/specifications.html
[3] https://www.broadcom.com/products/ethernet-connectivity/network-adapters/100gb-nic-ocp/n1100g
V2:
1. Use data-plane API to enable direct-rearm (Konstantin, Honnappa)
2. Add 'txq_data_get' API to get txq info for Rx (Konstantin)
3. Use input parameter to enable direct rearm in l3fwd (Konstantin)
4. Add condition detection for direct rearm API (Morten, Andrew Rybchenko)
V3:
1. Seperate Rx and Tx operation with two APIs in direct-rearm (Konstantin)
2. Delete L3fwd change for direct rearm (Jerin)
3. enable direct rearm in ixgbe driver in Arm
Feifei Wang (3):
ethdev: enable direct rearm with separate API net/i40e: enable direct rearm with separate API net/ixgbe: enable direct rearm with separate API
drivers/net/i40e/i40e_ethdev.c | 1 + drivers/net/i40e/i40e_ethdev.h | 2 + drivers/net/i40e/i40e_rxtx.c | 19 +++ drivers/net/i40e/i40e_rxtx.h | 4 + drivers/net/i40e/i40e_rxtx_vec_common.h | 54 +++++++ drivers/net/i40e/i40e_rxtx_vec_neon.c | 42 ++++++ drivers/net/ixgbe/ixgbe_ethdev.c | 1 + drivers/net/ixgbe/ixgbe_ethdev.h | 3 + drivers/net/ixgbe/ixgbe_rxtx.c | 19 +++ drivers/net/ixgbe/ixgbe_rxtx.h | 4 + drivers/net/ixgbe/ixgbe_rxtx_vec_common.h | 48 ++++++ drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c | 52 +++++++ lib/ethdev/ethdev_driver.h | 10 ++ lib/ethdev/ethdev_private.c | 2 + lib/ethdev/rte_ethdev.c | 52 +++++++ lib/ethdev/rte_ethdev.h | 174 ++++++++++++++++++++++ lib/ethdev/rte_ethdev_core.h | 11 ++ lib/ethdev/version.map | 6 + 18 files changed, 504 insertions(+)
Signature · 1 line
2.25.1
PATCH v3 1/3 ethdev: enable direct rearm with separate API
Add 'tx_fill_sw_ring' and 'rx_flush_descriptor' API into direct rearm mode for separate Rx and Tx Operation. And this can support different multiple sources in direct rearm mode. For examples, Rx driver is ixgbe, and Tx driver is i40e.
--- lib/ethdev/ethdev_driver.h | 10 ++ lib/ethdev/ethdev_private.c | 2 + lib/ethdev/rte_ethdev.c | 52 +++++++++++ lib/ethdev/rte_ethdev.h | 174 +++++++++++++++++++++++++++++++++++ lib/ethdev/rte_ethdev_core.h | 11 +++ lib/ethdev/version.map | 6 ++ 6 files changed, 255 insertions(+)
lib/ethdev/ethdev_driver.h +10 -0
@@ -59,6 +59,10 @@ struct rte_eth_dev {eth_rx_descriptor_status_t rx_descriptor_status;/** Check the status of a Tx descriptor */ eth_tx_descriptor_status_t tx_descriptor_status; +/** Fill Rx sw-ring with Tx buffers in direct rearm mode */ +eth_tx_fill_sw_ring_t tx_fill_sw_ring; +/** Flush Rx descriptor in direct rearm mode */ +eth_rx_flush_descriptor_t rx_flush_descriptor;/** * Device data that is shared between primary and secondary processes
@@ -504,6 +508,10 @@ typedef void (*eth_rxq_info_get_t)(struct rte_eth_dev *dev,typedef void (*eth_txq_info_get_t )(struct rte_eth_dev *dev,uint16_t tx_queue_id,struct rte_eth_txq_info *qinfo); +/**< @internal Get rearm data for a receive queue of an Ethernet device. */ +typedef void (*eth_rxq_rearm_data_get_t )(struct rte_eth_dev *dev, +uint16_t tx_queue_id,struct rte_eth_rxq_rearm_data *rxq_rearm_data); +typedef int (*eth_burst_mode_get_t )(struct rte_eth_dev *dev,uint16_t queue_id,struct rte_eth_burst_mode *mode);
@@ -1215,6 +1223,8 @@ struct eth_dev_ops {eth_rxq_info_get_t rxq_info_get;/** Retrieve Tx queue information */ eth_txq_info_get_t txq_info_get; +/** Get Rx queue rearm data */ +eth_rxq_rearm_data_get_t rxq_rearm_data_get;eth_burst_mode_get_t rx_burst_mode_get;/**< Get Rx burst mode */ eth_burst_mode_get_t tx_burst_mode_get;/**< Get Tx burst mode */ eth_fw_version_get_t fw_version_get;/**< Get firmware version */
lib/ethdev/ethdev_private.c +2 -0
@@ -276,6 +276,8 @@ eth_dev_fp_ops_setup(struct rte_eth_fp_ops *fpo, fpo->rx_queue_count = dev->rx_queue_count; fpo->rx_descriptor_status = dev->rx_descriptor_status; fpo->tx_descriptor_status = dev->tx_descriptor_status; + fpo->tx_fill_sw_ring = dev->tx_fill_sw_ring; + fpo->rx_flush_descriptor = dev->rx_flush_descriptor; fpo->rxq.data = dev->data->rx_queues; fpo->rxq.clbk = (void **)(uintptr_t )dev->post_rx_burst_cbs;
lib/ethdev/rte_ethdev.c +52 -0
64 lines not on this page · raw message
lib/ethdev/rte_ethdev.h +174 -0
198 lines not on this page · raw message
lib/ethdev/rte_ethdev_core.h +11 -0
@@ -56,6 +56,13 @@ typedef int (*eth_rx_descriptor_status_t)(void *rxq, uint16_t offset);/** @internal Check the status of a Tx descriptor */ typedef int (*eth_tx_descriptor_status_t )(void *txq,uint16_t offset); +/** @internal Fill Rx sw-ring with Tx buffers in direct rearm mode */ +typedef int (*eth_tx_fill_sw_ring_t )(void *txq, +struct rte_eth_rxq_rearm_data *rxq_rearm_data); + +/** @internal Flush Rx descriptor in direct rearm mode */ +typedef int (*eth_rx_flush_descriptor_t )(void *rxq,uint16_t nb_rearm); +/** * @internal * Structure used to hold opaque pointers to internal ethdev Rx/Tx
@@ -90,6 +97,8 @@ struct rte_eth_fp_ops {eth_rx_queue_count_t rx_queue_count;/** Check the status of a Rx descriptor. */ eth_rx_descriptor_status_t rx_descriptor_status; +/** Flush Rx descriptor in direct rearm mode */ +eth_rx_flush_descriptor_t rx_flush_descriptor;/** Rx queues data. */ struct rte_ethdev_qdata rxq;uintptr_t reserved1[3 ];
@@ -106,6 +115,8 @@ struct rte_eth_fp_ops {eth_tx_prep_t tx_pkt_prepare;/** Check the status of a Tx descriptor. */ eth_tx_descriptor_status_t tx_descriptor_status; +/** Fill Rx sw-ring with Tx buffers in direct rearm mode */ +eth_tx_fill_sw_ring_t tx_fill_sw_ring;/** Tx queues data. */ struct rte_ethdev_qdata txq;uintptr_t reserved2[3 ];
lib/ethdev/version.map +6 -0
@@ -298,6 +298,12 @@ EXPERIMENTAL { rte_flow_get_q_aged_flows; rte_mtr_meter_policy_get; rte_mtr_meter_profile_get; + + # added in 23.03 + rte_eth_dev_direct_rearm; + rte_eth_rx_flush_descriptor; + rte_eth_rx_queue_rearm_data_get; + rte_eth_tx_fill_sw_ring; }; INTERNAL {
Signature · 1 line
2.25.1
RE: [PATCH v3 1/3] ethdev: enable direct rearm with separate API
Quoted 9 lines
From: Feifei Wang [mailto:[email protected]]
Sent: Wednesday, 4 January 2023 08.31Add '
tx_fill_sw_ring' and 'rx_flush_descriptor' API into direct rearm mode for separate Rx and Tx Operation. And this can support different multiple sources in direct rearm mode. For examples, Rx driver is ixgbe,
and Tx driver is i40e.
Quoted 6 lines
Suggested-by Honnappa Nagarahalli <[email protected]>Suggested-by Ruifeng Wang <[email protected]>Signed-off-by Feifei Wang <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>Reviewed-by Honnappa Nagarahalli <[email protected]>---
This feature looks very promising for performance. I am pleased to see progress on it.
Please confirm that the fast path functions are still thread safe, i.e. one EAL thread may be calling rte_eth_rx_burst() while another EAL thread is calling rte_eth_tx_burst().
A few more comments below.
Quoted 13 lines
lib/ethdev/ethdev_driver.h | 10 ++ lib/ethdev/ethdev_private.c | 2 + lib/ethdev/rte_ethdev.c | 52 +++++++++++ lib/ethdev/rte_ethdev.h | 174 +++++++++++++++++++++++++++++++++++ lib/ethdev/rte_ethdev_core.h | 11 +++ lib/ethdev/version.map | 6 ++ 6 files changed, 255 insertions(+)lib/ethdev/ethdev_driver.h +0
-0
lib/ethdev/ethdev_driver.h struct rte_eth_dev { · line 63
Quoted 5 lines
lib/ethdev/ethdev_driver.h +2
-0@@ -59,6 +59,10 @@ struct rte_eth_dev {eth_rx_descriptor_status_t rx_descriptor_status;/** Check the status of a Tx descriptor */ eth_tx_descriptor_status_t tx_descriptor_status; +/** Fill Rx sw-ring with Tx buffers in direct rearm mode */ +eth_tx_fill_sw_ring_t tx_fill_sw_ring;
What is "Rx sw-ring"? Please confirm that this is not an Intel PMD specific term and/or implementation detail, e.g. by providing a conceptual implementation for a non-Intel PMD, e.g. mlx5.
Please note: I do not request the ability to rearm between drivers from different vendors, I only request that the public ethdev API uses generic terms and concepts, so any NIC vendor can implement the direct-rearm functions in their PMDs.
lib/ethdev/ethdev_driver.h struct rte_eth_dev { · line 65
Quoted 2 lines
lib/ethdev/ethdev_driver.h +2
-0+/** Flush Rx descriptor in direct rearm mode */ +eth_rx_flush_descriptor_t rx_flush_descriptor;
descriptor -> descriptors. There are more than one. Both in comment and function name.
[...]
Quoted 14 lines
lib/ethdev/rte_ethdev.h +4
-0@@ -1818,6 +1818,17 @@ struct rte_eth_txq_info {uint8_t queue_state;/**< one of RTE_ETH_QUEUE_STATE_*. */ } __rte_cache_min_aligned; +/** +* @internal +* Structure used to hold pointers to internal ethdev Rx rearm data. +* The main purpose is to load Rx queue rearm data in direct rearm mode.
+ */
lib/ethdev/rte_ethdev.h struct rte_eth_txq_info { · line 1830
Quoted 5 lines
lib/ethdev/rte_ethdev.h +5
-0+struct rte_eth_rxq_rearm_data { +void *rx_sw_ring; +uint16_t *rearm_start; +uint16_t *rearm_nb; +} __rte_cache_min_aligned;
Please add descriptions to the fields in this structure.
回复: [PATCH v3 1/3] ethdev: enable direct rearm with separate API
Hi, Morten
Quoted 26 lines
-----邮件原件-----
发件人: Morten Brørup <[email protected]>
发送时间: Wednesday, January 4, 2023 4:22 PM
收件人: Feifei Wang <[email protected]>; [email protected]; Ferruh Yigit <[email protected]>; Andrew Rybchenko <[email protected]>
抄送: [email protected]; [email protected]; nd <[email protected]>; Honnappa Nagarahalli <[email protected]>; Ruifeng Wang <[email protected]>
主题: RE: [PATCH v3 1/3] ethdev: enable direct rearm with separate APIQuoted 8 lines
From: Feifei Wang [mailto:[email protected]]
Sent: Wednesday, 4 January 2023 08.31Add '
tx_fill_sw_ring' and 'rx_flush_descriptor' API into direct rearm mode for separate Rx and Tx Operation. And this can support different multiple sources in direct rearm mode. For examples, Rx driver is ixgbe, and Tx driver is i40e.Quoted 6 lines
Suggested-by Honnappa Nagarahalli <[email protected]>Suggested-by Ruifeng Wang <[email protected]>Signed-off-by Feifei Wang <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>Reviewed-by Honnappa Nagarahalli <[email protected]>---
Quoted 3 lines
This feature looks very promising for performance. I am pleased to see progress on it.
Thanks very much for your reviewing.
Quoted 4 lines
Please confirm that the fast path functions are still thread safe, i.e. one EAL thread may be calling
rte_eth_rx_burst()while another EAL thread is callingrte_eth_tx_burst().
For the multiple threads safe, like we say in cover letter, current direct-rearm support Rx and Tx in the same thread. If we consider multiple threads like 'pipeline model', there need to add 'lock' in the data path which can decrease the performance.
Thus, the first step we do is try to enable direct-rearm in the single thread, and then we will consider to enable direct rearm in multiple threads and improve the performance.
Quoted 11 lines
A few more comments below.
Quoted 4 lines
lib/ethdev/ethdev_driver.h | 10 ++ lib/ethdev/ethdev_private.c | 2 + lib/ethdev/rte_ethdev.c | 52 +++++++++++ lib/ethdev/rte_ethdev.h | 174+++++++++++++++++++++++++++++++++++
Quoted 4 lines
lib/ethdev/rte_ethdev_core.h | 11 +++ lib/ethdev/version.map | 6 ++ 6 files changed, 255 insertions(+)
Quoted 14 lines
lib/ethdev/ethdev_driver.h struct rte_eth_dev { · line 63
Quoted 10 lines
lib/ethdev/ethdev_driver.h +2
-0@@ -59,6 +59,10 @@ struct rte_eth_dev {eth_rx_descriptor_status_t rx_descriptor_status;/** Check the status of a Tx descriptor */ eth_tx_descriptor_status_t tx_descriptor_status; +/** Fill Rx sw-ring with Tx buffers in direct rearm mode */ +eth_tx_fill_sw_ring_t tx_fill_sw_ring;What is "Rx sw-ring"? Please confirm that this is not an Intel PMD specific term and/or implementation detail, e.g. by providing a conceptual implementation for a non-Intel PMD, e.g. mlx5.
Rx sw_ring is used to store mbufs in intel PMD. This is the same as 'rxq->elts' in mlx5. Agree with that we need to providing a conceptual implementation for all PMDs.
Quoted 5 lines
Please note: I do not request the ability to rearm between drivers from different vendors, I only request that the public ethdev API uses generic terms and concepts, so any NIC vendor can implement the direct-rearm functions in their PMDs.
Agree with this. This is also our consideration.
In the future plan, we plan to implement this function in different PMDs.
Quoted 6 lines
lib/ethdev/ethdev_driver.h struct rte_eth_dev { · line 65
Quoted 3 lines
lib/ethdev/ethdev_driver.h +2
-0+/** Flush Rx descriptor in direct rearm mode */ +eth_rx_flush_descriptor_t rx_flush_descriptor;descriptor -> descriptors. There are more than one. Both in comment and function name.
Agree.
Quoted 11 lines
[...]
Quoted 8 lines
lib/ethdev/rte_ethdev.h index renamed from lib/ethdev/rte_ethdev.h +0
-0
c129ca1eaf..381c3d535f100644lib/ethdev/rte_ethdev.h +0
-0@@ -1818,6 +1818,17 @@ struct rte_eth_txq_info {uint8_t queue_state;/**< one of RTE_ETH_QUEUE_STATE_*. */ } __rte_cache_min_aligned;
Quoted 13 lines
lib/ethdev/rte_ethdev.h struct rte_eth_txq_info { · line 1830
Quoted 11 lines
lib/ethdev/rte_ethdev.h +4
-0+/** +* @internal +* Structure used to hold pointers to internal ethdev Rx rearm data. +* The main purpose is to load Rx queue rearm data in direct rearm mode.
+ */lib/ethdev/rte_ethdev.h +5
-0+struct rte_eth_rxq_rearm_data { +void *rx_sw_ring; +uint16_t *rearm_start; +uint16_t *rearm_nb; +} __rte_cache_min_aligned;Please add descriptions to the fields in this structure.
Agree.
RE: [PATCH v3 1/3] ethdev: enable direct rearm with separate API
Quoted 37 lines
From: Feifei Wang [mailto:[email protected]]
Sent: Wednesday, 4 January 2023 09.51Hi, Morten
Quoted 7 lines
发件人: Morten Brørup <[email protected]>
发送时间: Wednesday, January 4, 2023 4:22 PMQuoted 4 lines
From: Feifei Wang [mailto:[email protected]]
Sent: Wednesday, 4 January 2023 08.31Add '
tx_fill_sw_ring' and 'rx_flush_descriptor' API into directrearm
Quoted 1 line
Quoted 1 line
mode for separate Rx and Tx Operation. And this can support
different
Quoted 11 lines
Quoted 9 lines
multiple sources in direct rearm mode. For examples, Rx driver is ixgbe, and Tx driver is i40e.
Suggested-by Honnappa Nagarahalli <[email protected]>Suggested-by Ruifeng Wang <[email protected]>Signed-off-by Feifei Wang <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>Reviewed-by Honnappa Nagarahalli <[email protected]>---
This feature looks very promising for performance. I am pleased to
see
Quoted 2 lines
progress on it.
Thanks very much for your reviewing.
Quoted 1 line
Please confirm that the fast path functions are still thread safe,
i.e. one EAL
Quoted 1 line
thread may be calling
rte_eth_rx_burst()while another EAL thread iscalling
Quoted 2 lines
rte_eth_tx_burst().
Quoted 8 lines
For the multiple threads safe, like we say in cover letter, current direct-rearm support
Rx and Tx in the same thread. If we consider multiple threads like 'pipeline model', there
need to add 'lock' in the data path which can decrease the performance. Thus, the first step we do is try to enable direct-rearm in the single thread, and then we will consider
to enable direct rearm in multiple threads and improve the performance.
OK, doing it in steps is a good idea for a feature like this - makes it easier to understand and review.
When proceeding to add support for the "pipeline model", perhaps the lockless principles from the rte_ring can be used in this feature too.
From a high level perspective, I'm somewhat worried that releasing a "work-in-progress" version of this feature in some DPDK version will cause API/ABI breakage discussions when progressing to the next steps of the implementation to make the feature more complete. Not only support for thread safety across simultaneous RX and TX, but also support for multiple mbuf pools per RX queue [1]. Marking the functions experimental should alleviate such discussions, but there is a risk of pushback to not break the API/ABI anyway.
[1]: https://elixir.bootlin.com/dpdk/v22.11.1/source/lib/ethdev/rte_ethdev.h#L1105
[...]
Quoted 10 lines
Quoted 10 lines
lib/ethdev/ethdev_driver.h struct rte_eth_dev { · line 63
Quoted 8 lines
lib/ethdev/ethdev_driver.h +2
-0@@ -59,6 +59,10 @@ struct rte_eth_dev {eth_rx_descriptor_status_t rx_descriptor_status;/** Check the status of a Tx descriptor */ eth_tx_descriptor_status_t tx_descriptor_status; +/** Fill Rx sw-ring with Tx buffers in direct rearm mode */ +eth_tx_fill_sw_ring_t tx_fill_sw_ring;What is "Rx sw-ring"? Please confirm that this is not an Intel PMD
Quoted 6 lines
specific
Quoted 2 lines
term and/or implementation detail, e.g. by providing a conceptual implementation for a non-Intel PMD, e.g. mlx5.
Rx
sw_ringis used to store mbufs in intel PMD. This is the same as 'rxq->elts'
in mlx5.
Sounds good.
Then all we need is consensus on a generic name for this, unless "Rx sw-ring" already is the generic name. (I'm not a PMD developer, so I might be completely off track here.) Naming is often debatable, so I'll stop talking about it now - I only wanted to highlight that we should avoid vendor-specific terms in public APIs intended to be implemented by multiple vendors. On the other hand... if no other vendors raise their voices before merging into the DPDK main repository, they forfeit their right to complain about it. ;-)
Quoted 2 lines
Agree with that we need to providing a conceptual
implementation for all PMDs.
My main point is that we should ensure that the feature is not too tightly coupled with the way Intel PMDs implement mbuf handling. Providing a conceptual implementation for a non-Intel PMD is one way of checking this.
The actual implementation in other PMDs could be left up to the various NIC vendors.
回复: [PATCH v3 1/3] ethdev: enable direct rearm with separate API
Sorry for my delayed reply.
Quoted 71 lines
-----邮件原件-----
发件人: Morten Brørup <[email protected]>
发送时间: Wednesday, January 4, 2023 6:11 PM
收件人: Feifei Wang <[email protected]>; [email protected]; Ferruh Yigit <[email protected]>; Andrew Rybchenko <[email protected]>
抄送: [email protected]; [email protected]; nd <[email protected]>; Honnappa Nagarahalli <[email protected]>; Ruifeng Wang <[email protected]>; nd <[email protected]>
主题: RE: [PATCH v3 1/3] ethdev: enable direct rearm with separate APIQuoted 37 lines
From: Feifei Wang [mailto:[email protected]]
Sent: Wednesday, 4 January 2023 09.51Hi, Morten
Quoted 7 lines
发件人: Morten Brørup <[email protected]>
发送时间: Wednesday, January 4, 2023 4:22 PMQuoted 4 lines
From: Feifei Wang [mailto:[email protected]]
Sent: Wednesday, 4 January 2023 08.31Add '
tx_fill_sw_ring' and 'rx_flush_descriptor' API into directrearm
Quoted 1 line
Quoted 1 line
mode for separate Rx and Tx Operation. And this can support
different
Quoted 11 lines
Quoted 9 lines
multiple sources in direct rearm mode. For examples, Rx driver is ixgbe, and Tx driver is i40e.
Suggested-by Honnappa Nagarahalli <[email protected]>Suggested-by Ruifeng Wang <[email protected]>Signed-off-by Feifei Wang <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>Reviewed-by Honnappa Nagarahalli <[email protected]>---
This feature looks very promising for performance. I am pleased to
see
Quoted 2 lines
progress on it.
Thanks very much for your reviewing.
Quoted 1 line
Please confirm that the fast path functions are still thread safe,
i.e. one EAL
Quoted 1 line
thread may be calling
rte_eth_rx_burst()while another EAL thread iscalling
Quoted 2 lines
rte_eth_tx_burst().Quoted 7 lines
For the multiple threads safe, like we say in cover letter, current direct-rearm support Rx and Tx in the same thread. If we consider multiple threads like 'pipeline model', there need to add 'lock' in the data path which can decrease the performance.
Thus, the first step we do is try to enable direct-rearm in the single thread, and then we will consider to enable direct rearm in multiple threads and improve the performance.OK, doing it in steps is a good idea for a feature like this - makes it easier to understand and review.
When proceeding to add support for the "pipeline model", perhaps the lockless principles from the
rte_ringcan be used in this feature too.From a high level perspective, I'm somewhat worried that releasing a "work- in-progress" version of this feature in some DPDK version will cause API/ABI breakage discussions when progressing to the next steps of the implementation to make the feature more complete. Not only support for thread safety across simultaneous RX and TX, but also support for multiple mbuf pools per RX queue [1]. Marking the functions experimental should alleviate such discussions, but there is a risk of pushback to not break the API/ABI anyway.
Quoted 4 lines
[1]:
https://elixir.bootlin.com/dpdk/v22.11.1/source/lib/ethdev/rte_ethdev.h#L1
105
[Feifei] I think the subsequent upgrade does not significantly damage the stability of the API we currently define.
For thread safety across simultaneous RX and TX, in the future, the lockless operation change will happen in the pmd layer, such as CAS load/store for rxq queue index of pmd. Thus, this can not affect the stability of the upper API.
For multiple mbuf pools per RX queue, direct-rearm just put Tx buffers into Rx buffers, and it do not care which mempool the buffer coming from.
From different mempool buffers eventually freed into their respective sources in the no FAST_FREE path.
I think this is a mistake in cover letter. Previous direct-rearm can just support FAST_FREE so it constraint that buffer should be from the same mempool. Now, the latest version can support no_FAST_FREE path, but we forget to make change in cover letter.
Quoted 32 lines
[...]
Quoted 10 lines
Quoted 10 lines
lib/ethdev/ethdev_driver.h struct rte_eth_dev { · line 63
Quoted 8 lines
lib/ethdev/ethdev_driver.h +2
-0@@ -59,6 +59,10 @@ struct rte_eth_dev {eth_rx_descriptor_status_t rx_descriptor_status;/** Check the status of a Tx descriptor */ eth_tx_descriptor_status_t tx_descriptor_status; +/** Fill Rx sw-ring with Tx buffers in direct rearm mode */ +eth_tx_fill_sw_ring_t tx_fill_sw_ring;What is "Rx sw-ring"? Please confirm that this is not an Intel PMD
Quoted 6 lines
specific
Quoted 2 lines
term and/or implementation detail, e.g. by providing a conceptual implementation for a non-Intel PMD, e.g. mlx5.
Rx
sw_ringis used to store mbufs in intel PMD. This is the same as 'rxq->elts'
in mlx5.Sounds good.
Then all we need is consensus on a generic name for this, unless "Rx sw-ring" already is the generic name. (I'm not a PMD developer, so I might be completely off track here.) Naming is often debatable, so I'll stop talking about it now - I only wanted to highlight that we should avoid vendor- specific terms in public APIs intended to be implemented by multiple vendors. On the other hand... if no other vendors raise their voices before merging into the DPDK main repository, they forfeit their right to complain about it. ;-)
Quoted 2 lines
Agree with that we need to providing a conceptual implementation for all PMDs.
Quoted 6 lines
My main point is that we should ensure that the feature is not too tightly coupled with the way Intel PMDs implement mbuf handling. Providing a conceptual implementation for a non-Intel PMD is one way of checking this.
The actual implementation in other PMDs could be left up to the various NIC vendors.
Yes. And we will rename our API to make it suitable for all vendors:rte_eth_direct_rearm -> rte_eth_buf_cycle (upper API for direct rearm)rte_eth_tx_fill_sw_ring -> rte_eth_tx_buf_stash (Tx queue fill Rx ring buffer )rte_eth_rx_flush_descriptor -> rte_eth_rx_descriptors_refill (Rx queue flush its descriptors)
rte_eth_rxq_rearm_data {
void *rx_sw_ring;
uint16_t *rearm_start;
uint16_t *rearm_nb;
}->
struct *rxq_recycle_info {
rte_mbuf **buf_ring;
uint16_t *offset = (uint16 *)(&rq-<ci);
uint16_t *end;
uint16_t ring_size; }
Re: [PATCH v3 1/3] ethdev: enable direct rearm with separate API
In reply to Morten BrørupQuoted 9 lines · Morten Brørup
Quoted 9 lines
From: Feifei Wang [mailto:[email protected]]
Sent: Wednesday, 4 January 2023 08.31Add '
tx_fill_sw_ring' and 'rx_flush_descriptor' API into direct rearm mode for separate Rx and Tx Operation. And this can support different multiple sources in direct rearm mode. For examples, Rx driver is ixgbe,
and Tx driver is i40e.
Quoted 9 lines
Quoted 6 lines
Suggested-by Honnappa Nagarahalli <[email protected]>Suggested-by Ruifeng Wang <[email protected]>Signed-off-by Feifei Wang <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>Reviewed-by Honnappa Nagarahalli <[email protected]>---
This feature looks very promising for performance. I am pleased to see progress on it.
Hi Morten,
Yes it brings some performance, but not to generic use case, only to specific and constraint use case.
And changes are relatively invasive comparing the usecase it supports, like it adds new two inline datapath functions and a new dev_ops.
I am worried the unnecessary complexity and possible regressions in the fundamental and simple parts of the project, with a good intention to gain a few percentage performance in a specific usecase, can hurt the project.
I can see this is compared to MBUF_FAST_FREE feature, but MBUF_FAST_FREE is just an offload benefiting from existing offload infrastructure, which requires very small update and logically change in application and simple to implement in the drivers. So, they are not same from complexity perspective.
Briefly, I am not comfortable with this change, I would like to see an explicit approval and code review from techboard to proceed.
Quoted 56 lines
Please confirm that the fast path functions are still thread safe, i.e. one EAL thread may be calling
rte_eth_rx_burst()while another EAL thread is callingrte_eth_tx_burst().A few more comments below.
Quoted 13 lines
lib/ethdev/ethdev_driver.h | 10 ++ lib/ethdev/ethdev_private.c | 2 + lib/ethdev/rte_ethdev.c | 52 +++++++++++ lib/ethdev/rte_ethdev.h | 174 +++++++++++++++++++++++++++++++++++ lib/ethdev/rte_ethdev_core.h | 11 +++ lib/ethdev/version.map | 6 ++ 6 files changed, 255 insertions(+)lib/ethdev/ethdev_driver.h +0
-0lib/ethdev/ethdev_driver.h struct rte_eth_dev { · line 63
Quoted 5 lines
lib/ethdev/ethdev_driver.h +2
-0@@ -59,6 +59,10 @@ struct rte_eth_dev {eth_rx_descriptor_status_t rx_descriptor_status;/** Check the status of a Tx descriptor */ eth_tx_descriptor_status_t tx_descriptor_status; +/** Fill Rx sw-ring with Tx buffers in direct rearm mode */ +eth_tx_fill_sw_ring_t tx_fill_sw_ring;What is "Rx sw-ring"? Please confirm that this is not an Intel PMD specific term and/or implementation detail, e.g. by providing a conceptual implementation for a non-Intel PMD, e.g. mlx5.
Please note: I do not request the ability to rearm between drivers from different vendors, I only request that the public ethdev API uses generic terms and concepts, so any NIC vendor can implement the direct-rearm functions in their PMDs.
lib/ethdev/ethdev_driver.h struct rte_eth_dev { · line 65
Quoted 2 lines
lib/ethdev/ethdev_driver.h +2
-0+/** Flush Rx descriptor in direct rearm mode */ +eth_rx_flush_descriptor_t rx_flush_descriptor;descriptor -> descriptors. There are more than one. Both in comment and function name.
[...]
Quoted 14 lines
lib/ethdev/rte_ethdev.h +4
-0@@ -1818,6 +1818,17 @@ struct rte_eth_txq_info {uint8_t queue_state;/**< one of RTE_ETH_QUEUE_STATE_*. */ } __rte_cache_min_aligned; +/** +* @internal +* Structure used to hold pointers to internal ethdev Rx rearm data. +* The main purpose is to load Rx queue rearm data in direct rearm mode.
+ */lib/ethdev/rte_ethdev.h struct rte_eth_txq_info { · line 1830
Quoted 5 lines
lib/ethdev/rte_ethdev.h +5
-0+struct rte_eth_rxq_rearm_data { +void *rx_sw_ring; +uint16_t *rearm_start; +uint16_t *rearm_nb; +} __rte_cache_min_aligned;Please add descriptions to the fields in this structure.
RE: [PATCH v3 1/3] ethdev: enable direct rearm with separate API
Quoted 20 lines
From: Ferruh Yigit [mailto:[email protected]]
Sent: Monday, 6 March 2023 13.49Quoted 16 lines · Morten Brørup
Quoted 15 lines
From: Feifei Wang [mailto:[email protected]]
Sent: Wednesday, 4 January 2023 08.31Add '
tx_fill_sw_ring' and 'rx_flush_descriptor' API into direct rearm mode for separate Rx and Tx Operation. And this can support different multiple sources in direct rearm mode. For examples, Rx driver is ixgbe,
and Tx driver is i40e.Suggested-by Honnappa Nagarahalli <[email protected]>Suggested-by Ruifeng Wang <[email protected]>Signed-off-by Feifei Wang <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>Reviewed-by Honnappa Nagarahalli <[email protected]>---
Quoted 8 lines
Quoted 1 line
This feature looks very promising for performance. I am pleased to see
progress on it.
Quoted 1 line
Hi Morten,
Yes it brings some performance, but not to generic use case, only to specific and constraint use case.
I got the impression that the supported use case is a prominent and important use case.
This is the primary argument for considering such a complex non-generic feature.
Quoted 10 lines
And changes are relatively invasive comparing the usecase it supports, like it adds new two inline datapath functions and a new
dev_ops.I am worried the unnecessary complexity and possible regressions in the fundamental and simple parts of the project, with a good intention to gain a few percentage performance in a specific usecase, can hurt the project.
Quoted 8 lines
I can see this is compared to
MBUF_FAST_FREEfeature, butMBUF_FAST_FREEis just an offload benefiting from existing offload infrastructure, which requires very small update and logically change in application and simple to implement in the drivers. So, they are not same from complexity perspective.Briefly, I am not comfortable with this change, I would like to see an explicit approval and code review from techboard to proceed.
I agree that the complexity is very high, and thus requires extra consideration. Your suggested techboard review and approval process seems like a good solution.
And the performance benefit of direct rearm should be compared to the performance using the new zero-copy mempool API.
-Morten
回复: [PATCH v3 1/3] ethdev: enable direct rearm with separate API
Hi, Morten, Ferruh
Thanks very much for your reviewing.
Whatever the worries about direct rearm or comments to improve direct rearm, these urge us to learn more and think more. I think it is beneficial and a good achievement for this exploration.
I will update the latest version for techboard code review. During this time, I need some time to do performance test for the latest version.
So it should not catch up with this week's meeting and will be done before the techboard meeting in two weeks.
Best Regards
Feifei
Quoted 73 lines
-----邮件原件-----
发件人: Morten Brørup <[email protected]>
发送时间: Monday, March 6, 2023 9:26 PM
收件人: Ferruh Yigit <[email protected]>; Feifei Wang <[email protected]>; [email protected]; Andrew Rybchenko <[email protected]>; [email protected]
抄送: [email protected]; [email protected]; nd <[email protected]>; Honnappa Nagarahalli <[email protected]>; Ruifeng Wang <[email protected]>
主题: RE: [PATCH v3 1/3] ethdev: enable direct rearm with separate APIQuoted 21 lines
From: Ferruh Yigit [mailto:[email protected]]
Sent: Monday, 6 March 2023 13.49Quoted 15 lines · Morten Brørup
Quoted 8 lines
From: Feifei Wang [mailto:[email protected]]
Sent: Wednesday, 4 January 2023 08.31Add '
tx_fill_sw_ring' and 'rx_flush_descriptor' API into direct rearm mode for separate Rx and Tx Operation. And this can support different multiple sources in direct rearm mode. For examples, Rx driver is ixgbe, and Tx driver is i40e.Quoted 6 lines
Suggested-by Honnappa Nagarahalli <[email protected]>Suggested-by Ruifeng Wang <[email protected]>Signed-off-by Feifei Wang <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>Reviewed-by Honnappa Nagarahalli <[email protected]>---
Quoted 2 lines
This feature looks very promising for performance. I am pleased to see
Quoted 7 lines
progress on it.
Quoted 1 line
Hi Morten,
Yes it brings some performance, but not to generic use case, only to specific and constraint use case.
I got the impression that the supported use case is a prominent and important use case.
This is the primary argument for considering such a complex non-generic feature.
Quoted 10 lines
And changes are relatively invasive comparing the usecase it supports, like it adds new two inline datapath functions and a new
dev_ops.I am worried the unnecessary complexity and possible regressions in the fundamental and simple parts of the project, with a good intention to gain a few percentage performance in a specific usecase, can hurt the project.
Quoted 8 lines
I can see this is compared to
MBUF_FAST_FREEfeature, butMBUF_FAST_FREEis just an offload benefiting from existing offload infrastructure, which requires very small update and logically change in application and simple to implement in the drivers. So, they are not same from complexity perspective.Briefly, I am not comfortable with this change, I would like to see an explicit approval and code review from techboard to proceed.
I agree that the complexity is very high, and thus requires extra consideration. Your suggested techboard review and approval process seems like a good solution.
And the performance benefit of direct rearm should be compared to the performance using the new zero-copy mempool API.
-Morten
Re: [PATCH v3 1/3] ethdev: enable direct rearm with separate API
In reply to Morten BrørupQuoted 24 lines · Morten Brørup
Quoted 24 lines
From: Ferruh Yigit [mailto:[email protected]]
Sent: Monday, 6 March 2023 13.49Quoted 9 lines · Morten Brørup
Quoted 9 lines
From: Feifei Wang [mailto:[email protected]]
Sent: Wednesday, 4 January 2023 08.31Add '
tx_fill_sw_ring' and 'rx_flush_descriptor' API into direct rearm mode for separate Rx and Tx Operation. And this can support different multiple sources in direct rearm mode. For examples, Rx driver is ixgbe,
and Tx driver is i40e.Quoted 8 lines
Quoted 6 lines
Suggested-by Honnappa Nagarahalli <[email protected]>Suggested-by Ruifeng Wang <[email protected]>Signed-off-by Feifei Wang <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>Reviewed-by Honnappa Nagarahalli <[email protected]>---
This feature looks very promising for performance. I am pleased to see
progress on it.
Quoted 1 line
Quoted 7 lines
Quoted 4 lines
Hi Morten,
Yes it brings some performance, but not to generic use case, only to specific and constraint use case.
I got the impression that the supported use case is a prominent and important use case.
Can you please give real life samples for this use case, other than just showing better performance number in the test bench? This helps to understand the reasoning better.
Quoted 27 lines
This is the primary argument for considering such a complex non-generic feature.
Quoted 10 lines
And changes are relatively invasive comparing the usecase it supports, like it adds new two inline datapath functions and a new
dev_ops.I am worried the unnecessary complexity and possible regressions in the fundamental and simple parts of the project, with a good intention to gain a few percentage performance in a specific usecase, can hurt the project.
Quoted 8 lines
I can see this is compared to
MBUF_FAST_FREEfeature, butMBUF_FAST_FREEis just an offload benefiting from existing offload infrastructure, which requires very small update and logically change in application and simple to implement in the drivers. So, they are not same from complexity perspective.Briefly, I am not comfortable with this change, I would like to see an explicit approval and code review from techboard to proceed.
I agree that the complexity is very high, and thus requires extra consideration. Your suggested techboard review and approval process seems like a good solution.
And the performance benefit of direct rearm should be compared to the performance using the new zero-copy mempool API.
-Morten
RE: [PATCH v3 1/3] ethdev: enable direct rearm with separate API
<snip>
Quoted 31 lines · Morten Brørup
Quoted 29 lines
Quoted 28 lines
From: Ferruh Yigit [mailto:[email protected]]
Sent: Monday, 6 March 2023 13.49Quoted 15 lines · Morten Brørup
Quoted 8 lines
From: Feifei Wang [mailto:[email protected]]
Sent: Wednesday, 4 January 2023 08.31Add '
tx_fill_sw_ring' and 'rx_flush_descriptor' API into direct rearm mode for separate Rx and Tx Operation. And this can support different multiple sources in direct rearm mode. For examples, Rx driver is ixgbe, and Tx driver is i40e.Quoted 6 lines
Suggested-by Honnappa Nagarahalli <[email protected]>Suggested-by Ruifeng Wang <[email protected]>Signed-off-by Feifei Wang <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>Reviewed-by Honnappa Nagarahalli <[email protected]>---
Quoted 2 lines
This feature looks very promising for performance. I am pleased to see
progress on it.
Quoted 1 line
Hi Morten,
Yes it brings some performance, but not to generic use case, only to specific and constraint use case.
Quoted 7 lines
Quoted 1 line
I got the impression that the supported use case is a prominent and important
use case.
Quoted 1 line
Can you please give real life samples for this use case, other than just showing better performance number in the test bench? This helps to understand the reasoning better.
The very first patch started off with a constrained but prominent use case. Though, DPU based PCIe cards running DPDK applications with 1 or max 2 ports being used in tons of data centers is not a secret anymore and not a small use case that can be ignored.
However, the design of the patch has changed significantly from then. Now the solution can be applied to any generic use case that uses run-to-completion model of DPDK. i.e. the mapping of the RX and TX ports can be done dynamically in the data plane threads. There is no need of static configuration from control plane.
On the test bench, we need to make up our mind. When we see improvements, we say it is just a test bench. On other occasions when the test bench does not show any improvements (but improvements are shown by other metrics), we say the test bench does not show any improvements.
Quoted 3 lines
Quoted 2 lines
This is the primary argument for considering such a complex non-generic
feature.
I am not sure what is the complexity here, can you please elaborate?
I see other patches/designs (ex: proactive error recovery) which are way more complex to understand and comprehend.
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
And changes are relatively invasive comparing the usecase it supports, like it adds new two inline datapath functions and a new
dev_ops.I am worried the unnecessary complexity and possible regressions in the fundamental and simple parts of the project, with a good intention to gain a few percentage performance in a specific usecase, can hurt the project.
I agree that we are touching some fundamental parts of the project. But, we also need to realize that those fundamental parts were not developed on architectures that have joined the project way later. Similarly, the use cases have evolved significantly from the original intended use cases. We cannot hold on to those fundamental designs if they affect the performance on other architectures while addressing prominent new use cases. Please note that this patch does not break any existing features or affect their performance in any negative way. The generic and originally intended use cases can benefit from this feature.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
I can see this is compared to
MBUF_FAST_FREEfeature, butMBUF_FAST_FREEis just an offload benefiting from existing offload infrastructure, which requires very small update and logically change in application and simple to implement in the drivers. So, they are not same from complexity perspective.
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
Briefly, I am not comfortable with this change, I would like to see an explicit approval and code review from techboard to proceed.
I agree that the complexity is very high, and thus requires extra consideration.
Your suggested techboard review and approval process seems like a good solution.
We can add to the agenda for the next Techboard meeting.
Quoted 6 lines
Quoted 2 lines
And the performance benefit of direct rearm should be compared to the
performance using the new zero-copy mempool API.
Quoted 3 lines
-Morten
RE: [PATCH v3 1/3] ethdev: enable direct rearm with separate API
Quoted 36 lines
Quoted 36 lines · Morten Brørup
Quoted 24 lines
Quoted 24 lines
From: Ferruh Yigit [mailto:[email protected]]
Sent: Monday, 6 March 2023 13.49Quoted 15 lines · Morten Brørup
Quoted 8 lines
From: Feifei Wang [mailto:[email protected]]
Sent: Wednesday, 4 January 2023 08.31Add '
tx_fill_sw_ring' and 'rx_flush_descriptor' API into direct rearm mode for separate Rx and Tx Operation. And this can support different multiple sources in direct rearm mode. For examples, Rx driver is ixgbe, and Tx driver is i40e.Quoted 6 lines
Suggested-by Honnappa Nagarahalli <[email protected]>Suggested-by Ruifeng Wang <[email protected]>Signed-off-by Feifei Wang <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>Reviewed-by Honnappa Nagarahalli <[email protected]>---
Quoted 2 lines
This feature looks very promising for performance. I am pleased to see
progress on it.
Quoted 1 line
Quoted 6 lines
Quoted 4 lines
Hi Morten,
Yes it brings some performance, but not to generic use case, only to specific and constraint use case.
I got the impression that the supported use case is a prominent and important
use case.
Quoted 1 line
Quoted 8 lines
Quoted 3 lines
Can you please give real life samples for this use case, other than just showing better performance number in the test bench? This helps to understand the reasoning better.
The very first patch started off with a constrained but prominent use case. Though, DPU based PCIe cards running DPDK applications with 1 or max 2 ports being used in tons of data centers is not a secret anymore and not a small use case that can be ignored. However, the design of the patch has changed significantly from then. Now the solution can be applied to any generic use case that uses run-to-completion model of DPDK. i.e. the mapping of the RX and TX ports can be done dynamically in the data plane threads. There is no need of static configuration from control plane.
+1 to this statement.
I think the authors did a good job to make it generic enough, so it can be used for many different cases, plus AFAIU it doesn't introduce new implicit restrictions to the user.
Again, this feature is totally optional, so users are free to ignore it.
Personally I do not see any good reason why we shouldn't accept this feature into DPDK. Off-course with more code reviews, extra testing, docs updates, etc..
Quoted 49 lines
On the test bench, we need to make up our mind. When we see improvements, we say it is just a test bench. On other occasions when the test bench does not show any improvements (but improvements are shown by other metrics), we say the test bench does not show any improvements.
Quoted 3 lines
Quoted 2 lines
This is the primary argument for considering such a complex non-generic
feature.
I am not sure what is the complexity here, can you please elaborate?
I see other patches/designs (ex: proactive error recovery) which are way more complex to understand and comprehend.Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
And changes are relatively invasive comparing the usecase it supports, like it adds new two inline datapath functions and a new
dev_ops.I am worried the unnecessary complexity and possible regressions in the fundamental and simple parts of the project, with a good intention to gain a few percentage performance in a specific usecase, can hurt the project.
I agree that we are touching some fundamental parts of the project. But, we also need to realize that those fundamental parts were not developed on architectures that have joined the project way later. Similarly, the use cases have evolved significantly from the original intended use cases. We cannot hold on to those fundamental designs if they affect the performance on other architectures while addressing prominent new use cases.
Please note that this patch does not break any existing features or affect their performance in any negative way. The generic and originally intended use cases can benefit from this feature.Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
I can see this is compared to
MBUF_FAST_FREEfeature, butMBUF_FAST_FREEis just an offload benefiting from existing offload infrastructure, which requires very small update and logically change in application and simple to implement in the drivers. So, they are not same from complexity perspective.Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
Briefly, I am not comfortable with this change, I would like to see an explicit approval and code review from techboard to proceed.
I agree that the complexity is very high, and thus requires extra consideration.
Your suggested techboard review and approval process seems like a good solution.
We can add to the agenda for the next Techboard meeting.
Quoted 6 lines
Quoted 2 lines
And the performance benefit of direct rearm should be compared to the
performance using the new zero-copy mempool API.
Quoted 3 lines
-Morten
Re: [PATCH v3 1/3] ethdev: enable direct rearm with separate API
In reply to Honnappa NagarahalliQuoted 43 lines · Honnappa Nagarahalli
<snip>
Quoted 31 lines · Morten Brørup
Quoted 29 lines
Quoted 28 lines
From: Ferruh Yigit [mailto:[email protected]]
Sent: Monday, 6 March 2023 13.49Quoted 15 lines · Morten Brørup
Quoted 8 lines
From: Feifei Wang [mailto:[email protected]]
Sent: Wednesday, 4 January 2023 08.31Add '
tx_fill_sw_ring' and 'rx_flush_descriptor' API into direct rearm mode for separate Rx and Tx Operation. And this can support different multiple sources in direct rearm mode. For examples, Rx driver is ixgbe, and Tx driver is i40e.Quoted 6 lines
Suggested-by Honnappa Nagarahalli <[email protected]>Suggested-by Ruifeng Wang <[email protected]>Signed-off-by Feifei Wang <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>Reviewed-by Honnappa Nagarahalli <[email protected]>---
Quoted 2 lines
This feature looks very promising for performance. I am pleased to see
progress on it.
Quoted 1 line
Hi Morten,
Yes it brings some performance, but not to generic use case, only to specific and constraint use case.
Quoted 7 lines
Quoted 1 line
I got the impression that the supported use case is a prominent and important
use case.
Quoted 1 line
Can you please give real life samples for this use case, other than just showing better performance number in the test bench? This helps to understand the reasoning better.
The very first patch started off with a constrained but prominent use case. Though, DPU based PCIe cards running DPDK applications with 1 or max 2 ports being used in tons of data centers is not a secret anymore and not a small use case that can be ignored.
However, the design of the patch has changed significantly from then. Now the solution can be applied to any generic use case that uses run-to-completion model of DPDK. i.e. the mapping of the RX and TX ports can be done dynamically in the data plane threads. There is no need of static configuration from control plane.
Quoted 6 lines
On the test bench, we need to make up our mind. When we see improvements, we say it is just a test bench. On other occasions when the test bench does not show any improvements (but improvements are shown by other metrics), we say the test bench does not show any improvements.
Quoted 3 lines
Quoted 2 lines
This is the primary argument for considering such a complex non-generic
feature.
I am not sure what is the complexity here, can you please elaborate?
I am considering from user perspective.
OK, DPDK is already low level, but ethdev has only a handful of datapath APIs (6 of them), and main ones are easy to comprehend:rte_eth_rx_burst(port_id, queue_id, rx_pkts, nb_pkts); rte_eth_tx_burst(port_id, queue_id, tx_pkts, nb_pkts);
They (magically) Rx/Tx buffers, easy to grasp.
Maybe rte_eth_tx_prepare() is a little less obvious (why/when to use it), but still I believe simple.
Whoever looks to these APIs can figure out how to use in the application.
The other three is related to the descriptors and I am not sure about their use-case, I assume they are mostly good for debugging.
But now we are adding new datapath APIs:rte_eth_tx_fill_sw_ring(port_id, queue_id, rxq_rearm_data); rte_eth_rx_flush_descriptor(port_id, queue_id, nb_rearm);
When you talk about SW ring and re-arming descriptors I believe you will loose most of the users already, driver developers will know what it is, you will know what that is, but people who are not close to the Ethernet HW won't.
And these APIs will be very visible, not like one of many control plane dev_ops. So this can confuse users who are not familiar with details.
Usage of these APIs comes with restrictions, it is possible that at some percentage of users will miss these restrictions or miss-understand them and will have issues.
Or many may be intimidated by them and stay away from using these APIs, leaving them as a burden to maintain, to test, to fix. That is why I think a real life usecase is needed, in that case at least we will know some consumers will fix or let us know when they get broken.
It may be possible to hide details under driver and user only set an offload flag, similar to FAST_FREE, but in that case feature will loose flexibility and it will be even more specific, perhaps making it less useful.
Quoted 36 lines
I see other patches/designs (ex: proactive error recovery) which are way more complex to understand and comprehend.
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
And changes are relatively invasive comparing the usecase it supports, like it adds new two inline datapath functions and a new
dev_ops.I am worried the unnecessary complexity and possible regressions in the fundamental and simple parts of the project, with a good intention to gain a few percentage performance in a specific usecase, can hurt the project.
I agree that we are touching some fundamental parts of the project. But, we also need to realize that those fundamental parts were not developed on architectures that have joined the project way later. Similarly, the use cases have evolved significantly from the original intended use cases. We cannot hold on to those fundamental designs if they affect the performance on other architectures while addressing prominent new use cases. Please note that this patch does not break any existing features or affect their performance in any negative way. The generic and originally intended use cases can benefit from this feature.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
I can see this is compared to
MBUF_FAST_FREEfeature, butMBUF_FAST_FREEis just an offload benefiting from existing offload infrastructure, which requires very small update and logically change in application and simple to implement in the drivers. So, they are not same from complexity perspective.Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
Briefly, I am not comfortable with this change, I would like to see an explicit approval and code review from techboard to proceed.
I agree that the complexity is very high, and thus requires extra consideration.
Your suggested techboard review and approval process seems like a good solution.
We can add to the agenda for the next Techboard meeting.
Quoted 6 lines
Quoted 2 lines
And the performance benefit of direct rearm should be compared to the
performance using the new zero-copy mempool API.
Quoted 3 lines
-Morten
RE: [PATCH v3 1/3] ethdev: enable direct rearm with separate API
Quoted 73 lines
-----Original Message-----
From: Ferruh Yigit <[email protected]>
Sent: Tuesday, March 7, 2023 2:41 PM
To: Honnappa Nagarahalli <[email protected]>; Morten Brørup <[email protected]>; Feifei Wang <[email protected]>; [email protected]; Andrew Rybchenko <[email protected]>; [email protected]
Cc: [email protected]; [email protected]; nd <[email protected]>; Ruifeng Wang <[email protected]>
Subject: Re: [PATCH v3 1/3] ethdev: enable direct rearm with separate APIQuoted 11 lines · Honnappa Nagarahalli
<snip>
Quoted 9 lines · Morten Brørup
Quoted 7 lines
Quoted 7 lines
From: Ferruh Yigit [mailto:[email protected]]
Sent: Monday, 6 March 2023 13.49Quoted 3 lines · Morten Brørup
Quoted 3 lines
From: Feifei Wang [mailto:[email protected]]
Sent: Wednesday, 4 January 2023 08.31Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Add '
tx_fill_sw_ring' and 'rx_flush_descriptor' API into direct rearm mode for separate Rx and Tx Operation. And this can support different multiple sources in direct rearm mode. For examples, Rx driver is ixgbe, and Tx driver is i40e.Suggested-by Honnappa NagarahalliQuoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Suggested-by Ruifeng Wang <[email protected]>Signed-off-by Feifei Wang <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>Reviewed-by Honnappa NagarahalliQuoted 14 lines
Quoted 14 lines
Quoted 14 lines
Quoted 11 lines
Quoted 4 lines
Quoted 1 line
---
This feature looks very promising for performance. I am pleased to see
progress on it.
Quoted 1 line
Hi Morten,
Yes it brings some performance, but not to generic use case, only to specific and constraint use case.
I got the impression that the supported use case is a prominent and important
Quoted 7 lines
Quoted 6 lines
use case.
Quoted 1 line
Can you please give real life samples for this use case, other than just showing better performance number in the test bench? This helps to understand the reasoning better.
The very first patch started off with a constrained but prominent use case.
Though, DPU based PCIe cards running DPDK applications with 1 or max 2 ports being used in tons of data centers is not a secret anymore and not a small use case that can be ignored.
Quoted 1 line
However, the design of the patch has changed significantly from then. Now
the solution can be applied to any generic use case that uses run-to-completion model of DPDK. i.e. the mapping of the RX and TX ports can be done dynamically in the data plane threads. There is no need of static configuration from control plane.
Quoted 2 lines
On the test bench, we need to make up our mind. When we see
improvements, we say it is just a test bench. On other occasions when the test bench does not show any improvements (but improvements are shown by other metrics), we say the test bench does not show any improvements.
Quoted 2 lines
Quoted 2 lines
Quoted 6 lines
Quoted 4 lines
Quoted 3 lines
Quoted 2 lines
This is the primary argument for considering such a complex non-generic
feature.
I am not sure what is the complexity here, can you please elaborate?
I am considering from user perspective.
Thanks for clarifying Ferruh.
Quoted 7 lines
OK, DPDK is already low level, but ethdev has only a handful of datapath APIs (6 of them), and main ones are easy to comprehend:
rte_eth_rx_burst(port_id, queue_id, rx_pkts, nb_pkts);rte_eth_tx_burst(port_id, queue_id, tx_pkts, nb_pkts);They (magically) Rx/Tx buffers, easy to grasp.
I think the pktmbuf pool part is missed here. The user needs to create a pktmbuf pool by calling rte_pktmbuf_pool_create and has to pass the cache_size parameter. This requires the user to understand what is a cache, why it is required and how it affects the performance.
There are further complexities associated with pktmbuf pool - creating a pool with external pinned memory, creating a pool with ops name etc. So, practically, the user needs to be aware of more details than just the RX and TX functions.
Quoted 10 lines
Maybe
rte_eth_tx_prepare()is a little less obvious (why/when to use it), but still I believe simple.Whoever looks to these APIs can figure out how to use in the application.
The other three is related to the descriptors and I am not sure about their use- case, I assume they are mostly good for debugging.
Quoted 7 lines
But now we are adding new datapath APIs:
rte_eth_tx_fill_sw_ring(port_id, queue_id, rxq_rearm_data);rte_eth_rx_flush_descriptor(port_id, queue_id, nb_rearm);When you talk about SW ring and re-arming descriptors I believe you will loose most of the users already, driver developers will know what it is, you will know what that is, but people who are not close to the Ethernet HW won't.
Agree, the names could be better. I personally do not want to separate out these two APIs as I do not think a use case (receiving and transmitting pkts across NICs of different types) exists to keep them separate. But, we did this based on feedback and to maintain a cleaner separation between RX and TX path. We will try to propose new names for these.
Quoted 7 lines
And these APIs will be very visible, not like one of many control plane
dev_ops. So this can confuse users who are not familiar with details.Usage of these APIs comes with restrictions, it is possible that at some percentage of users will miss these restrictions or miss-understand them and will have issues.
Agreed, there are several features already with restrictions.
Quoted 9 lines
Or many may be intimidated by them and stay away from using these APIs, leaving them as a burden to maintain, to test, to fix. That is why I think a real life usecase is needed, in that case at least we will know some consumers will fix or let us know when they get broken.
It may be possible to hide details under driver and user only set an offload flag, similar to
FAST_FREE, but in that case feature will loose flexibility and it will be even more specific, perhaps making it less useful.
Agree.
Quoted 49 lines
Quoted 3 lines
I see other patches/designs (ex: proactive error recovery) which are way more
complex to understand and comprehend.
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
And changes are relatively invasive comparing the usecase it supports, like it adds new two inline datapath functions and a new
dev_ops.Quoted 6 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
I am worried the unnecessary complexity and possible regressions in the fundamental and simple parts of the project, with a good intention to gain a few percentage performance in a specific usecase, can hurt the project.
I agree that we are touching some fundamental parts of the project. But, we
also need to realize that those fundamental parts were not developed on architectures that have joined the project way later. Similarly, the use cases have evolved significantly from the original intended use cases. We cannot hold on to those fundamental designs if they affect the performance on other architectures while addressing prominent new use cases.
Quoted 1 line
Please note that this patch does not break any existing features or affect their
performance in any negative way. The generic and originally intended use cases can benefit from this feature.
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
I can see this is compared to
MBUF_FAST_FREEfeature, butMBUF_FAST_FREEis just an offload benefiting from existing offload infrastructure, which requires very small update and logically change in application and simple to implement in the drivers. So, they are not same from complexity perspective.Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
Briefly, I am not comfortable with this change, I would like to see an explicit approval and code review from techboard to proceed.
I agree that the complexity is very high, and thus requires extra
consideration.
Quoted 11 lines
Quoted 2 lines
Your suggested techboard review and approval process seems like a good solution.
We can add to the agenda for the next Techboard meeting.
Quoted 7 lines
Quoted 3 lines
And the performance benefit of direct rearm should be compared to the
performance using the new zero-copy mempool API.
Quoted 3 lines
-Morten
Re: [PATCH v3 1/3] ethdev: enable direct rearm with separate API
In reply to Feifei WangHi Feifei,
Quoted 4 lines
Add '
tx_fill_sw_ring' and 'rx_flush_descriptor' API into direct rearm mode for separate Rx and Tx Operation. And this can support different multiple sources in direct rearm mode. For examples, Rx driver is ixgbe, and Tx driver is i40e.
Thanks for your effort and thanks for taking comments provided into consideration. That approach looks much better then previous ones.
Few nits below.
Konstantin
Quoted 137 lines
Suggested-by Honnappa Nagarahalli <[email protected]>Suggested-by Ruifeng Wang <[email protected]>Signed-off-by Feifei Wang <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>Reviewed-by Honnappa Nagarahalli <[email protected]>--- lib/ethdev/ethdev_driver.h | 10 ++ lib/ethdev/ethdev_private.c | 2 + lib/ethdev/rte_ethdev.c | 52 +++++++++++ lib/ethdev/rte_ethdev.h | 174 +++++++++++++++++++++++++++++++++++ lib/ethdev/rte_ethdev_core.h | 11 +++ lib/ethdev/version.map | 6 ++ 6 files changed, 255 insertions(+)lib/ethdev/ethdev_driver.h +10
-0@@ -59,6 +59,10 @@ struct rte_eth_dev {eth_rx_descriptor_status_t rx_descriptor_status;/** Check the status of a Tx descriptor */ eth_tx_descriptor_status_t tx_descriptor_status; +/** Fill Rx sw-ring with Tx buffers in direct rearm mode */ +eth_tx_fill_sw_ring_t tx_fill_sw_ring; +/** Flush Rx descriptor in direct rearm mode */ +eth_rx_flush_descriptor_t rx_flush_descriptor;/** * Device data that is shared between primary and secondary processes @@ -504,6 +508,10 @@ typedef void (*eth_rxq_info_get_t)(struct rte_eth_dev *dev,typedef void (*eth_txq_info_get_t )(struct rte_eth_dev *dev,uint16_t tx_queue_id,struct rte_eth_txq_info *qinfo); +/**< @internal Get rearm data for a receive queue of an Ethernet device. */ +typedef void (*eth_rxq_rearm_data_get_t )(struct rte_eth_dev *dev, +uint16_t tx_queue_id,struct rte_eth_rxq_rearm_data *rxq_rearm_data); +typedef int (*eth_burst_mode_get_t )(struct rte_eth_dev *dev,uint16_t queue_id,struct rte_eth_burst_mode *mode);@@ -1215,6 +1223,8 @@ struct eth_dev_ops {eth_rxq_info_get_t rxq_info_get;/** Retrieve Tx queue information */ eth_txq_info_get_t txq_info_get; +/** Get Rx queue rearm data */ +eth_rxq_rearm_data_get_t rxq_rearm_data_get;eth_burst_mode_get_t rx_burst_mode_get;/**< Get Rx burst mode */ eth_burst_mode_get_t tx_burst_mode_get;/**< Get Tx burst mode */ eth_fw_version_get_t fw_version_get;/**< Get firmware version */ lib/ethdev/ethdev_private.c +2
-0@@ -276,6 +276,8 @@ eth_dev_fp_ops_setup(struct rte_eth_fp_ops *fpo, fpo->rx_queue_count = dev->rx_queue_count; fpo->rx_descriptor_status = dev->rx_descriptor_status; fpo->tx_descriptor_status = dev->tx_descriptor_status; + fpo->tx_fill_sw_ring = dev->tx_fill_sw_ring; + fpo->rx_flush_descriptor = dev->rx_flush_descriptor; fpo->rxq.data = dev->data->rx_queues; fpo->rxq.clbk = (void **)(uintptr_t )dev->post_rx_burst_cbs;lib/ethdev/rte_ethdev.c +52
-0@@ -3282,6 +3282,21 @@ rte_eth_dev_set_rx_queue_stats_mapping(uint16_t port_id, uint16_t rx_queue_id, stat_idx, STAT_QMAP_RX)); } +int +rte_eth_dev_direct_rearm (uint16_t rx_port_id,uint16_t rx_queue_id, +uint16_t tx_port_id,uint16_t tx_rx_queue_id, +struct rte_eth_rxq_rearm_data *rxq_rearm_data) +{ +int nb_rearm =0 ; + + nb_rearm =rte_eth_tx_fill_sw_ring (tx_port_id, tx_rx_queue_id, rxq_rearm_data); + +if (nb_rearm >0 ) +return rte_eth_rx_flush_descriptor (rx_port_id, rx_queue_id, nb_rearm); + +return 0 ; +} +int rte_eth_dev_fw_version_get (uint16_t port_id,char *fw_version,size_t fw_size) {@@ -5323,6 +5338,43 @@ rte_eth_tx_queue_info_get(uint16_t port_id, uint16_t queue_id,return 0 ; } +int +rte_eth_rx_queue_rearm_data_get (uint16_t port_id,uint16_t queue_id, +struct rte_eth_rxq_rearm_data *rxq_rearm_data) +{ +struct rte_eth_dev *dev; + +RTE_ETH_VALID_PORTID_OR_ERR_RET (port_id, -ENODEV ); + dev = &rte_eth_devices[port_id]; + +if (queue_id >= dev->data->nb_rx_queues) { +RTE_ETHDEV_LOG (ERR ,"Invalid Rx queue_id=%u\n" , queue_id); +return -EINVAL ; + } + +if (rxq_rearm_data ==NULL ) { +RTE_ETHDEV_LOG (ERR ,"Cannot get ethdev port %u Rx queue %u rearm data to NULL\n" , + port_id, queue_id); +return -EINVAL ; + } + +if (dev->data->rx_queues ==NULL || + dev->data->rx_queues[queue_id] ==NULL ) { +RTE_ETHDEV_LOG (ERR , +"Rx queue %" PRIu16" of device with port_id=%" + PRIu16" has not been setup\n" , + queue_id, port_id); +return -EINVAL ; + } + +if (*dev->dev_ops->rxq_rearm_data_get ==NULL ) +return -ENOTSUP ; + + dev->dev_ops->rxq_rearm_data_get (dev, queue_id, rxq_rearm_data); + +return 0 ; +} +int rte_eth_rx_burst_mode_get (uint16_t port_id,uint16_t queue_id,struct rte_eth_burst_mode *mode)lib/ethdev/rte_ethdev.h +0
-0
lib/ethdev/rte_ethdev.h struct rte_eth_txq_info { · line 1825
Quoted 8 lines
lib/ethdev/rte_ethdev.h +5
-0@@ -1818,6 +1818,17 @@ struct rte_eth_txq_info {uint8_t queue_state;/**< one of RTE_ETH_QUEUE_STATE_*. */ } __rte_cache_min_aligned; +/** +* @internal +* Structure used to hold pointers to internal ethdev Rx rearm data. +* The main purpose is to load Rx queue rearm data in direct rearm mode. +*/
I think this structure owes a lot more expalantion. What each fields suppose to do and what are the constraints, etc.
In general, more doc will be needed to explain that feature.
lib/ethdev/rte_ethdev.h struct rte_eth_txq_info { · line 1827
Quoted 2 lines
lib/ethdev/rte_ethdev.h +2
-0+struct rte_eth_rxq_rearm_data { +void *rx_sw_ring;
That's misleading, we always suppose to store mbufs ptrs here, so why not be direct:
struct rte_mbuf **rx_sw_ring;
lib/ethdev/rte_ethdev.h struct rte_eth_txq_info { · line 1829
Quoted 2 lines
lib/ethdev/rte_ethdev.h +2
-0+uint16_t *rearm_start; +uint16_t *rearm_nb;
I know that for Intel NICs uint16_t is sufficient, wonder would it always be for other vendors?
Another thing to consider the case when ring position wrapping? Again I know that it is not required for Intel NICs, but would it be sufficient for API that supposed to be general?
Quoted 26 lines
lib/ethdev/rte_ethdev.h +19
-0+} __rte_cache_min_aligned; +/* Generic Burst mode flag definition, values can be ORed. */ /** @@ -3184,6 +3195,34 @@ int rte_eth_dev_set_rx_queue_stats_mapping(uint16_t port_id,uint16_t rx_queue_id, uint8_t stat_idx); +/** +* @warning +* @b EXPERIMENTAL: this API may change, or be removed, without prior notice +* +* Directly put Tx freed buffers into Rx sw-ring and flush desc. +* +* @param rx_port_id +* Port identifying the receive side. +* @param rx_queue_id +* The index of the receive queue identifying the receive side. +* The value must be in the range [0, nb_rx_queue - 1] previously supplied +* to rte_eth_dev_configure(). +* @param tx_port_id +* Port identifying the transmit side. +* @param tx_queue_id +* The index of the transmit queue identifying the transmit side. +* The value must be in the range [0, nb_tx_queue - 1] previously supplied
lib/ethdev/rte_ethdev.h int rte_eth_dev_set_rx_queue_stats_mapping(uint16_t port_id, · line 3224
Quoted 10 lines
lib/ethdev/rte_ethdev.h +10
-0+* to rte_eth_dev_configure(). +* @param rxq_rearm_data +* A pointer to a structure of type *rte_eth_txq_rearm_data* to be filled. +* @return +* - 0: Success +*/ +__rte_experimental +int rte_eth_dev_direct_rearm(uint16_t rx_port_id, uint16_t rx_queue_id, +uint16_t tx_port_id,uint16_t tx_queue_id, +struct rte_eth_rxq_rearm_data *rxq_rearm_data);
I think we need one more parameter for that function 'uint16_t offset' or so. So _rearm_ will start to populate rx_sw_ring from *rearm_start + offset position. That way we can support populating from different sources. Or should 'offset' be part of truct rte_eth_rxq_rearm_data?
Quoted 206 lines
lib/ethdev/rte_ethdev.h +136
-0+/** * Retrieve the Ethernet address of an Ethernet device. * @@ -4782,6 +4821,27 @@ int rte_eth_rx_queue_info_get(uint16_t port_id, uint16_t queue_id,int rte_eth_tx_queue_info_get (uint16_t port_id,uint16_t queue_id,struct rte_eth_txq_info *qinfo); +/** +* Get rearm data about given ports's Rx queue. +* +* @param port_id +* The port identifier of the Ethernet device. +* @param queue_id +* The Rx queue on the Ethernet device for which rearm data +* will be got. +* @param rxq_rearm_data +* A pointer to a structure of type *rte_eth_txq_rearm_data* to be filled. +* +* @return +* - 0: Success +* - -ENODEV: If *port_id* is invalid. +* - -ENOTSUP: routine is not supported by the device PMD. +* - -EINVAL: The queue_id is out of range. +*/ +__rte_experimental +int rte_eth_rx_queue_rearm_data_get(uint16_t port_id, uint16_t queue_id, +struct rte_eth_rxq_rearm_data *rxq_rearm_data); +/** * Retrieve information about the Rx packet burst mode. * @@ -6103,6 +6163,120 @@ static inline int rte_eth_tx_descriptor_status(uint16_t port_id,return (*p->tx_descriptor_status)(qd, offset); } +/** +* @warning +* @b EXPERIMENTAL: this API may change, or be removed, without prior notice +* +* Fill Rx sw-ring with Tx buffers in direct rearm mode. +* +* @param port_id +* The port identifier of the Ethernet device. +* @param queue_id +* The index of the transmit queue. +* The value must be in the range [0, nb_tx_queue - 1] previously supplied +* to rte_eth_dev_configure(). +* @param rxq_rearm_data +* A pointer to a structure of type *rte_eth_rxq_rearm_data* to be filled with +* the rearm data of a receive queue. +* @return +* - The number buffers correct to be filled in the Rx sw-ring. +* - (-EINVAL) bad port or queue. +* - (-ENODEV) bad port. +* - (-ENOTSUP) if the device does not support this function. +* +*/ +__rte_experimental +static inline int rte_eth_tx_fill_sw_ring(uint16_t port_id, +uint16_t queue_id,struct rte_eth_rxq_rearm_data *rxq_rearm_data) +{ +struct rte_eth_fp_ops *p; +void *qd; + +#ifdef RTE_ETHDEV_DEBUG_TX +if (port_id >=RTE_MAX_ETHPORTS || + queue_id >=RTE_MAX_QUEUES_PER_PORT ) { +RTE_ETHDEV_LOG (ERR , +"Invalid port_id=%u or queue_id=%u\n" , + port_id, queue_id); +return -EINVAL ; + } +#endif + + p = &rte_eth_fp_ops[port_id]; + qd = p->txq.data[queue_id]; + +#ifdef RTE_ETHDEV_DEBUG_TX +RTE_ETH_VALID_PORTID_OR_ERR_RET (port_id,0 ); + +if (qd ==NULL ) { +RTE_ETHDEV_LOG (ERR ,"Invalid Tx queue_id=%u for port_id=%u\n" , + queue_id, port_id); +return -ENODEV ; + } +#endif + +if (p->tx_fill_sw_ring ==NULL ) +return -ENOTSUP ; + +return p->tx_fill_sw_ring (qd, rxq_rearm_data); +} + +/** +* @warning +* @b EXPERIMENTAL: this API may change, or be removed, without prior notice +* +* Flush Rx descriptor in direct rearm mode. +* +* @param port_id +* The port identifier of the Ethernet device. +* @param queue_id +* The index of the receive queue. +* The value must be in the range [0, nb_rx_queue - 1] previously supplied +* to rte_eth_dev_configure(). +*@param nb_rearm +* The number of Rx sw-ring buffers need to be flushed. +* @return +* - (0) if successful. +* - (-EINVAL) bad port or queue. +* - (-ENODEV) bad port. +* - (-ENOTSUP) if the device does not support this function. +*/ +__rte_experimental +static inline int rte_eth_rx_flush_descriptor(uint16_t port_id, +uint16_t queue_id,uint16_t nb_rearm) +{ +struct rte_eth_fp_ops *p; +void *qd; + +#ifdef RTE_ETHDEV_DEBUG_RX +if (port_id >=RTE_MAX_ETHPORTS || + queue_id >=RTE_MAX_QUEUES_PER_PORT ) { +RTE_ETHDEV_LOG (ERR , +"Invalid port_id=%u or queue_id=%u\n" , + port_id, queue_id); +return -EINVAL ; + } +#endif + + p = &rte_eth_fp_ops[port_id]; + qd = p->rxq.data[queue_id]; + +#ifdef RTE_ETHDEV_DEBUG_RX +RTE_ETH_VALID_PORTID_OR_ERR_RET (port_id,0 ); + +if (qd ==NULL ) { +RTE_ETHDEV_LOG (ERR ,"Invalid Rx queue_id=%u for port_id=%u\n" , + queue_id, port_id); +return -ENODEV ; + } +#endif + +if (p->rx_flush_descriptor ==NULL ) +return -ENOTSUP ; + +return p->rx_flush_descriptor (qd, nb_rearm); +} +/** * @internal * Helper routine for rte_eth_tx_burst(). lib/ethdev/rte_ethdev_core.h +11
-0@@ -56,6 +56,13 @@ typedef int (*eth_rx_descriptor_status_t)(void *rxq, uint16_t offset);/** @internal Check the status of a Tx descriptor */ typedef int (*eth_tx_descriptor_status_t )(void *txq,uint16_t offset); +/** @internal Fill Rx sw-ring with Tx buffers in direct rearm mode */ +typedef int (*eth_tx_fill_sw_ring_t )(void *txq, +struct rte_eth_rxq_rearm_data *rxq_rearm_data); + +/** @internal Flush Rx descriptor in direct rearm mode */ +typedef int (*eth_rx_flush_descriptor_t )(void *rxq,uint16_t nb_rearm); +/** * @internal * Structure used to hold opaque pointers to internal ethdev Rx/Tx @@ -90,6 +97,8 @@ struct rte_eth_fp_ops {eth_rx_queue_count_t rx_queue_count;/** Check the status of a Rx descriptor. */ eth_rx_descriptor_status_t rx_descriptor_status; +/** Flush Rx descriptor in direct rearm mode */ +eth_rx_flush_descriptor_t rx_flush_descriptor;/** Rx queues data. */ struct rte_ethdev_qdata rxq;uintptr_t reserved1[3 ];@@ -106,6 +115,8 @@ struct rte_eth_fp_ops {eth_tx_prep_t tx_pkt_prepare;/** Check the status of a Tx descriptor. */ eth_tx_descriptor_status_t tx_descriptor_status; +/** Fill Rx sw-ring with Tx buffers in direct rearm mode */ +eth_tx_fill_sw_ring_t tx_fill_sw_ring;/** Tx queues data. */ struct rte_ethdev_qdata txq;uintptr_t reserved2[3 ];lib/ethdev/version.map +6
-0@@ -298,6 +298,12 @@ EXPERIMENTAL { rte_flow_get_q_aged_flows; rte_mtr_meter_policy_get; rte_mtr_meter_profile_get; + + # added in 23.03 + rte_eth_dev_direct_rearm; + rte_eth_rx_flush_descriptor; + rte_eth_rx_queue_rearm_data_get; + rte_eth_tx_fill_sw_ring; }; INTERNAL {
回复: [PATCH v3 1/3] ethdev: enable direct rearm with separate API
Hi, Konstantin
Thanks for your reviewing and sorry for my delayed response. For your comments, we put forward several improvement plans below.
Best Regards
Feifei
Quoted 188 lines
-----邮件原件-----
发件人: Konstantin Ananyev <[email protected]> 发送时间: Thursday, February 2, 2023 10:33 PM
收件人: Feifei Wang <[email protected]>; [email protected]; Ferruh Yigit <[email protected]>; Andrew Rybchenko <[email protected]>
抄送: [email protected]; nd <[email protected]>; Honnappa Nagarahalli <[email protected]>; Ruifeng Wang <[email protected]>
主题: Re: [PATCH v3 1/3] ethdev: enable direct rearm with separate APIHi Feifei,
Quoted 4 lines
Add '
tx_fill_sw_ring' and 'rx_flush_descriptor' API into direct rearm mode for separate Rx and Tx Operation. And this can support different multiple sources in direct rearm mode. For examples, Rx driver is ixgbe, and Tx driver is i40e.Thanks for your effort and thanks for taking comments provided into consideration.
That approach looks much better then previous ones.
Few nits below.
KonstantinQuoted 10 lines
Suggested-by Honnappa Nagarahalli <[email protected]>Suggested-by Ruifeng Wang <[email protected]>Signed-off-by Feifei Wang <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>Reviewed-by Honnappa Nagarahalli <[email protected]>--- lib/ethdev/ethdev_driver.h | 10 ++ lib/ethdev/ethdev_private.c | 2 + lib/ethdev/rte_ethdev.c | 52 +++++++++++ lib/ethdev/rte_ethdev.h | 174+++++++++++++++++++++++++++++++++++
Quoted 19 lines
lib/ethdev/rte_ethdev_core.h | 11 +++ lib/ethdev/version.map | 6 ++ 6 files changed, 255 insertions(+)lib/ethdev/ethdev_driver.h +4
-0@@ -59,6 +59,10 @@ struct rte_eth_dev {eth_rx_descriptor_status_t rx_descriptor_status;/** Check the status of a Tx descriptor */ eth_tx_descriptor_status_t tx_descriptor_status; +/** Fill Rx sw-ring with Tx buffers in direct rearm mode */ +eth_tx_fill_sw_ring_t tx_fill_sw_ring; +/** Flush Rx descriptor in direct rearm mode */ +eth_rx_flush_descriptor_t rx_flush_descriptor;/** * Device data that is shared between primary and secondary Quoted 1 line
processes @@ -504,6 +508,10 @@ typedef void
(*eth_rxq_info_get_t)(struct
rte_eth_dev*dev,lib/ethdev/ethdev_driver.h
Quoted 5 lines
typedef void (*eth_txq_info_get_t)(struct
rte_eth_dev*dev,
uint16_ttx_queue_id, structrte_eth_txq_info*qinfo);lib/ethdev/ethdev_driver.h +2
-0+/**< @internal Get rearm data for a receive queue of an Ethernet +device. */ typedef void (*eth_rxq_rearm_data_get_t)(struct rte_eth_dev*dev,
lib/ethdev/ethdev_driver.h struct eth_dev_ops { · line 1228
Quoted 13 lines
lib/ethdev/ethdev_driver.h +5
-0+ uint16_t tx_queue_id, struct rte_eth_rxq_rearm_data +*rxq_rearm_data); +typedef int (*eth_burst_mode_get_t )(struct rte_eth_dev *dev,uint16_t queue_id,struct rte_eth_burst_mode *mode);@@ -1215,6 +1223,8 @@ struct eth_dev_ops {eth_rxq_info_get_t rxq_info_get;/** Retrieve Tx queue information */ eth_txq_info_get_t txq_info_get; +/** Get Rx queue rearm data */ +eth_rxq_rearm_data_get_t rxq_rearm_data_get;eth_burst_mode_get_t rx_burst_mode_get;/**< Get Rx burst mode */
lib/ethdev/ethdev_driver.h struct eth_dev_ops { · line 1229
Quoted 1 line
lib/ethdev/ethdev_driver.h +0
-0eth_burst_mode_get_t tx_burst_mode_get;/**< Get Tx burst mode */
lib/ethdev/ethdev_driver.h struct eth_dev_ops { · line 1230
Quoted 1 line
lib/ethdev/ethdev_driver.h +0
-0eth_fw_version_get_t fw_version_get;/**< Get firmware version */
lib/ethdev/rte_ethdev.c
Quoted 18 lines
lib/ethdev/ethdev_private.c +2
-0@@ -276,6 +276,8 @@ eth_dev_fp_ops_setup(struct rte_eth_fp_ops *fpo, fpo->rx_queue_count = dev->rx_queue_count; fpo->rx_descriptor_status = dev->rx_descriptor_status; fpo->tx_descriptor_status = dev->tx_descriptor_status; + fpo->tx_fill_sw_ring = dev->tx_fill_sw_ring; + fpo->rx_flush_descriptor = dev->rx_flush_descriptor; fpo->rxq.data = dev->data->rx_queues; fpo->rxq.clbk = (void **)(uintptr_t )dev->post_rx_burst_cbs;lib/ethdev/rte_ethdev.c index renamed from lib/ethdev/rte_ethdev.c +0
-0
5d5e18db1e..2af5cb42fe100644lib/ethdev/rte_ethdev.c +0
-0@@ -3282,6 +3282,21 @@rte_eth_dev_set_rx_queue_stats_mapping(uint16_t
port_id,uint16_trx_queue_id,Quoted 11 lines
lib/ethdev/rte_ethdev.c +8
-0stat_idx, STAT_QMAP_RX)); } +int +rte_eth_dev_direct_rearm (uint16_t rx_port_id,uint16_t rx_queue_id, +uint16_t tx_port_id,uint16_t tx_rx_queue_id, +struct rte_eth_rxq_rearm_data *rxq_rearm_data) { +int nb_rearm =0 ; + + nb_rearm =rte_eth_tx_fill_sw_ring (tx_port_id, tx_rx_queue_id, +rxq_rearm_data);lib/ethdev/rte_ethdev.c rte_eth_dev_set_rx_queue_stats_mapping(uint16_t port_id, uint16_t rx_queue_id,
Quoted 10 lines
lib/ethdev/rte_ethdev.c +8
-0+ +if (nb_rearm >0 ) +return rte_eth_rx_flush_descriptor (rx_port_id, rx_queue_id, +nb_rearm); + +return 0 ; +} +int rte_eth_dev_fw_version_get(uint16_t port_id, char *fw_version, size_t
fw_size)lib/ethdev/rte_ethdev.c rte_eth_tx_queue_info_get(uint16_t port_id,
Quoted 2 lines
lib/ethdev/rte_ethdev.c +0
-0{@@ -5323,6 +5338,43 @@ rte_eth_tx_queue_info_get(uint16_t port_id,
uint16_tqueue_id,lib/ethdev/rte_ethdev.c rte_eth_tx_queue_info_get(uint16_t port_id, uint16_t queue_id,
Quoted 13 lines
lib/ethdev/rte_ethdev.c +10
-0return 0 ; } +int +rte_eth_rx_queue_rearm_data_get (uint16_t port_id,uint16_t queue_id, +struct rte_eth_rxq_rearm_data *rxq_rearm_data) { +struct rte_eth_dev *dev; + +RTE_ETH_VALID_PORTID_OR_ERR_RET (port_id, -ENODEV ); + dev = &rte_eth_devices[port_id]; + +if (queue_id >= dev->data->nb_rx_queues) { + RTE_ETHDEV_LOG(ERR, "Invalid Rx queue_id=%u\n",
queue_id);lib/ethdev/rte_ethdev.c rte_eth_tx_queue_info_get(uint16_t port_id, uint16_t queue_id,
Quoted 5 lines
lib/ethdev/rte_ethdev.c +5
-0+return -EINVAL ; + } + +if (rxq_rearm_data ==NULL ) { + RTE_ETHDEV_LOG(ERR, "Cannot get ethdev port %u Rxqueue %u rearm data to NULL\n",
Quoted 7 lines
lib/ethdev/rte_ethdev.c +7
-0+ port_id, queue_id); +return -EINVAL ; + } + +if (dev->data->rx_queues ==NULL || + dev->data->rx_queues[queue_id] ==NULL ) { + RTE_ETHDEV_LOG(ERR,lib/ethdev/rte_ethdev.c rte_eth_tx_queue_info_get(uint16_t port_id, uint16_t queue_id,
Quoted 10 lines
lib/ethdev/rte_ethdev.c +10
-0+"Rx queue %" PRIu16" of device with port_id=%" + PRIu16" has not been setup\n" , + queue_id, port_id); +return -EINVAL ; + } + +if (*dev->dev_ops->rxq_rearm_data_get ==NULL ) +return -ENOTSUP ; + + dev->dev_ops->rxq_rearm_data_get(dev, queue_id,
rxq_rearm_data);Quoted 12 lines
lib/ethdev/rte_ethdev.c +4
-0+ +return 0 ; +} + int rte_eth_rx_burst_mode_get(uint16_t port_id, uint16_t queue_id, struct rte_eth_burst_mode *mode) diff --git
a/lib/ethdev/rte_ethdev.hb/lib/ethdev/rte_ethdev.hindexc129ca1eaf..381c3d535f100644lib/ethdev/rte_ethdev.h struct rte_eth_txq_info { · line 1825
Quoted 8 lines
lib/ethdev/rte_ethdev.h +5
-0@@ -1818,6 +1818,17 @@ struct rte_eth_txq_info {uint8_t queue_state;/**< one of RTE_ETH_QUEUE_STATE_*. */ } __rte_cache_min_aligned; +/** +* @internal +* Structure used to hold pointers to internal ethdev Rx rearm data. +* The main purpose is to load Rx queue rearm data in direct rearm mode. +*/ I think this structure owes a lot more expalantion. What each fields suppose to do and what are the constraints, etc.
Quoted 1 line
In general, more doc will be needed to explain that feature.
You are right. We will add more explanation for it and also update the doc To explain the feature.
Quoted 7 lines
lib/ethdev/rte_ethdev.h struct rte_eth_txq_info { · line 1827
Quoted 2 lines
lib/ethdev/rte_ethdev.h +2
-0+struct rte_eth_rxq_rearm_data { +void *rx_sw_ring;That's misleading, we always suppose to store mbufs ptrs here, so why not be direct:
structrte_mbuf**rx_sw_ring;
Agree.
Quoted 9 lines
lib/ethdev/rte_ethdev.h struct rte_eth_txq_info { · line 1829
Quoted 2 lines
lib/ethdev/rte_ethdev.h +2
-0+uint16_t *rearm_start; +uint16_t *rearm_nb;I know that for Intel NICs
uint16_tis sufficient, wonder would it always be for other vendors?
Another thing to consider the case when ring position wrapping? Again I know that it is not required for Intel NICs, but would it be sufficient for API that supposed to be general?
For this, we re-define this structure:
rte_eth_rxq_rearm_data {
void *rx_sw_ring;
uint16_t *rearm_start;
uint16_t *rearm_nb;
}
->
struct *rxq_recycle_info {
rte_mbuf **buf_ring;
uint16_t *offset = (uint16 *)(&rq->ci);
uint16_t *end;
uint16_t ring_size; }
For the new structure, *offset is a pointer for rearm-start index of Rx buffer ring (consumer index). *end is a pointer for rearm-end index Of Rx buffer ring (producer index).
1. we look up different pmds, some pmds using 'uint_16t' as index size like intel PMD, some pmds using 'uint32_t' as index size like MLX5 or thunderx PMD.
For pmd using 'uint32_t', rearm starts at 'buf_ring[offset & (ring_size -1)]', and 'uint16_t' is enough for ring size.
2. Good question. In general path, there is a constraint that 'nb_rearm < ring_size - rq->ci', This can ensure no ring wrapping in rearm. Thus in direct-rearm, we will refer to this to solve ring wrapping.
Quoted 40 lines
lib/ethdev/rte_ethdev.h int
Quoted 7 lines
lib/ethdev/rte_ethdev.h +2
-0+} __rte_cache_min_aligned; +/* Generic Burst mode flag definition, values can be ORed. */ /** @@ -3184,6 +3195,34 @@ intrte_eth_dev_set_rx_queue_stats_mapping(uint16_t
port_id,Quoted 5 lines
lib/ethdev/rte_ethdev.h +2
-0uint16_t rx_queue_id, uint8_t stat_idx); +/** +* @warning lib/ethdev/rte_ethdev.h int rte_eth_dev_set_rx_queue_stats_mapping(uint16_t port_id,
Quoted 10 lines
lib/ethdev/rte_ethdev.h +10
-0+ * @b EXPERIMENTAL: this API may change, or be removed, without prior +notice + * + * Directly put Tx freed buffers into Rx sw-ring and flush desc. + * + * @param rx_port_id + * Port identifying the receive side. + * @param rx_queue_id + * The index of the receive queue identifying the receive side. + * The value must be in the range [0, nb_rx_queue - 1] previouslysupplied
lib/ethdev/rte_ethdev.h int rte_eth_dev_set_rx_queue_stats_mapping(uint16_t port_id,
Quoted 6 lines
lib/ethdev/rte_ethdev.h +6
-0+* to rte_eth_dev_configure(). +* @param tx_port_id +* Port identifying the transmit side. +* @param tx_queue_id +* The index of the transmit queue identifying the transmit side. +* The value must be in the range [0, nb_tx_queue - 1] previously supplied
Quoted 9 lines
lib/ethdev/rte_ethdev.h +9
-0+* to rte_eth_dev_configure(). +* @param rxq_rearm_data +* A pointer to a structure of type *rte_eth_txq_rearm_data* to be filled. +* @return +* - 0: Success +*/ +__rte_experimental +int rte_eth_dev_direct_rearm(uint16_t rx_port_id, uint16_t rx_queue_id, + uint16_t tx_port_id, uint16_t tx_queue_id,
Quoted 9 lines
lib/ethdev/rte_ethdev.h int · line 3225
Quoted 1 line
lib/ethdev/rte_ethdev.h +1
-0+ struct rte_eth_rxq_rearm_data *rxq_rearm_data);I think we need one more parameter for that function '
uint16_toffset' or so.
So _rearm_ will start to populaterx_sw_ringfrom*rearm_start+ offset position. That way we can support populating from different sources. Or should 'offset' be part of tructrte_eth_rxq_rearm_data?
Agree, please see above, we will do the change in the structure.
Quoted 215 lines
lib/ethdev/rte_ethdev.h int rte_eth_rx_queue_info_get(uint16_t
Quoted 5 lines
lib/ethdev/rte_ethdev.h +1
-0+/** * Retrieve the Ethernet address of an Ethernet device. * @@ -4782,6 +4821,27 @@ int rte_eth_rx_queue_info_get(uint16_t
port_id,uint16_tqueue_id,Quoted 12 lines
lib/ethdev/rte_ethdev.h +9
-0int rte_eth_tx_queue_info_get (uint16_t port_id,uint16_t queue_id,struct rte_eth_txq_info *qinfo); +/** +* Get rearm data about given ports's Rx queue. +* +* @param port_id +* The port identifier of the Ethernet device. +* @param queue_id +* The Rx queue on the Ethernet device for which rearm data +* will be got. +* @param rxq_rearm_data lib/ethdev/rte_ethdev.h int rte_eth_rx_queue_info_get(uint16_t port_id, uint16_t queue_id,
Quoted 10 lines
lib/ethdev/rte_ethdev.h +10
-0+* A pointer to a structure of type *rte_eth_txq_rearm_data* to be filled. +* +* @return +* - 0: Success +* - -ENODEV: If *port_id* is invalid. +* - -ENOTSUP: routine is not supported by the device PMD. +* - -EINVAL: The queue_id is out of range. +*/ +__rte_experimental +int rte_eth_rx_queue_rearm_data_get(uint16_t port_id, uint16_t
queue_id,lib/ethdev/rte_ethdev.h static inline int
Quoted 6 lines
lib/ethdev/rte_ethdev.h +2
-0+ struct rte_eth_rxq_rearm_data *rxq_rearm_data); +/** * Retrieve information about the Rx packet burst mode. * @@ -6103,6 +6163,120 @@ static inline intrte_eth_tx_descriptor_status(uint16_t
port_id,Quoted 5 lines
lib/ethdev/rte_ethdev.h +2
-0return (*p->tx_descriptor_status)(qd, offset); } +/** +* @warning lib/ethdev/rte_ethdev.h static inline int rte_eth_tx_descriptor_status(uint16_t port_id,
Quoted 10 lines
lib/ethdev/rte_ethdev.h +10
-0+ * @b EXPERIMENTAL: this API may change, or be removed, without prior +notice + * + * Fill Rx sw-ring with Tx buffers in direct rearm mode. + * + * @param port_id + * The port identifier of the Ethernet device. + * @param queue_id + * The index of the transmit queue. + * The value must be in the range [0, nb_tx_queue - 1] previouslysupplied
lib/ethdev/rte_ethdev.h int rte_eth_dev_set_rx_queue_stats_mapping(uint16_t port_id,
Quoted 3 lines
lib/ethdev/rte_ethdev.h +3
-0+* to rte_eth_dev_configure(). +* @param rxq_rearm_data +* A pointer to a structure of type *rte_eth_rxq_rearm_data* to be filled with
lib/ethdev/rte_ethdev.h static inline int rte_eth_tx_descriptor_status(uint16_t port_id, · line 6190
Quoted 11 lines
lib/ethdev/rte_ethdev.h +11
-0+* the rearm data of a receive queue. +* @return +* - The number buffers correct to be filled in the Rx sw-ring. +* - (-EINVAL) bad port or queue. +* - (-ENODEV) bad port. +* - (-ENOTSUP) if the device does not support this function. +* +*/ +__rte_experimental +static inline int rte_eth_tx_fill_sw_ring(uint16_t port_id, +uint16_t queue_id,struct rte_eth_rxq_rearm_data *rxq_rearm_data){
Quoted 11 lines
lib/ethdev/rte_ethdev.h +11
-0+struct rte_eth_fp_ops *p; +void *qd; + +#ifdef RTE_ETHDEV_DEBUG_TX +if (port_id >=RTE_MAX_ETHPORTS || + queue_id >=RTE_MAX_QUEUES_PER_PORT ) { +RTE_ETHDEV_LOG (ERR , +"Invalid port_id=%u or queue_id=%u\n" , + port_id, queue_id); +return -EINVAL ; + }lib/ethdev/rte_ethdev.h static inline int rte_eth_tx_descriptor_status(uint16_t port_id,
Quoted 10 lines
lib/ethdev/rte_ethdev.h +10
-0+#endif + + p = &rte_eth_fp_ops[port_id]; + qd = p->txq.data[queue_id]; + +#ifdef RTE_ETHDEV_DEBUG_TX +RTE_ETH_VALID_PORTID_OR_ERR_RET (port_id,0 ); + +if (qd ==NULL ) { + RTE_ETHDEV_LOG(ERR, "Invalid Tx queue_id=%u forport_id=%u\n",
Quoted 12 lines
lib/ethdev/rte_ethdev.h +12
-0+ queue_id, port_id); + return -ENODEV; + } +#endif + +if (p->tx_fill_sw_ring ==NULL ) +return -ENOTSUP ; + +return p->tx_fill_sw_ring (qd, rxq_rearm_data); } + +/** +* @warning lib/ethdev/rte_ethdev.h static inline int rte_eth_tx_descriptor_status(uint16_t port_id,
Quoted 10 lines
lib/ethdev/rte_ethdev.h +10
-0+ * @b EXPERIMENTAL: this API may change, or be removed, without prior +notice + * + * Flush Rx descriptor in direct rearm mode. + * + * @param port_id + * The port identifier of the Ethernet device. + * @param queue_id + * The index of the receive queue. + * The value must be in the range [0, nb_rx_queue - 1] previouslysupplied
Quoted 23 lines
lib/ethdev/rte_ethdev.h +23
-0+* to rte_eth_dev_configure(). +*@param nb_rearm +* The number of Rx sw-ring buffers need to be flushed. +* @return +* - (0) if successful. +* - (-EINVAL) bad port or queue. +* - (-ENODEV) bad port. +* - (-ENOTSUP) if the device does not support this function. +*/ +__rte_experimental +static inline int rte_eth_rx_flush_descriptor(uint16_t port_id, +uint16_t queue_id,uint16_t nb_rearm) { +struct rte_eth_fp_ops *p; +void *qd; + +#ifdef RTE_ETHDEV_DEBUG_RX +if (port_id >=RTE_MAX_ETHPORTS || + queue_id >=RTE_MAX_QUEUES_PER_PORT ) { +RTE_ETHDEV_LOG (ERR , +"Invalid port_id=%u or queue_id=%u\n" , + port_id, queue_id); +return -EINVAL ; + }lib/ethdev/rte_ethdev.h static inline int rte_eth_tx_descriptor_status(uint16_t port_id,
Quoted 10 lines
lib/ethdev/rte_ethdev.h +10
-0+#endif + + p = &rte_eth_fp_ops[port_id]; + qd = p->rxq.data[queue_id]; + +#ifdef RTE_ETHDEV_DEBUG_RX +RTE_ETH_VALID_PORTID_OR_ERR_RET (port_id,0 ); + +if (qd ==NULL ) { + RTE_ETHDEV_LOG(ERR, "Invalid Rx queue_id=%u forport_id=%u\n",
lib/ethdev/rte_ethdev_core.h typedef int (*eth_rx_descriptor_status_t)(void *rxq,
Quoted 18 lines
lib/ethdev/rte_ethdev.h +10
-0+ queue_id, port_id); + return -ENODEV; + } +#endif + +if (p->rx_flush_descriptor ==NULL ) +return -ENOTSUP ; + +return p->rx_flush_descriptor (qd, nb_rearm); } +/** * @internal * Helper routine for rte_eth_tx_burst(). (quoted diff) +0
-0
b/lib/ethdev/rte_ethdev_core.hindexdcf8adab92..5ecb57f6f0100644lib/ethdev/rte_ethdev_core.h +0
-0@@ -56,6 +56,13 @@ typedef int (*eth_rx_descriptor_status_t)(void *rxq,
uint16_toffset);Quoted 49 lines
lib/ethdev/rte_ethdev_core.h +7
-0/** @internal Check the status of a Tx descriptor */ typedef int (*eth_tx_descriptor_status_t )(void *txq,uint16_t offset); +/** @internal Fill Rx sw-ring with Tx buffers in direct rearm mode */ +typedef int (*eth_tx_fill_sw_ring_t )(void *txq, +struct rte_eth_rxq_rearm_data *rxq_rearm_data); + +/** @internal Flush Rx descriptor in direct rearm mode */ typedef int +(*eth_rx_flush_descriptor_t )(void *rxq,uint16_t nb_rearm); +/** * @internal * Structure used to hold opaque pointers to internal ethdev Rx/Tx
lib/ethdev/rte_ethdev_core.h +4
-0@@ -90,6 +97,8 @@ struct rte_eth_fp_ops {eth_rx_queue_count_t rx_queue_count;/** Check the status of a Rx descriptor. */ eth_rx_descriptor_status_t rx_descriptor_status; +/** Flush Rx descriptor in direct rearm mode */ +eth_rx_flush_descriptor_t rx_flush_descriptor;/** Rx queues data. */ struct rte_ethdev_qdata rxq;uintptr_t reserved1[3 ];@@ -106,6 +115,8 @@ struct rte_eth_fp_ops {eth_tx_prep_t tx_pkt_prepare;/** Check the status of a Tx descriptor. */ eth_tx_descriptor_status_t tx_descriptor_status; +/** Fill Rx sw-ring with Tx buffers in direct rearm mode */ +eth_tx_fill_sw_ring_t tx_fill_sw_ring;/** Tx queues data. */ struct rte_ethdev_qdata txq;uintptr_t reserved2[3 ];lib/ethdev/version.map index renamed from lib/ethdev/version.map +0
-0
17201fbe0f..f39f02a69b100644lib/ethdev/version.map +6
-0@@ -298,6 +298,12 @@ EXPERIMENTAL { rte_flow_get_q_aged_flows; rte_mtr_meter_policy_get; rte_mtr_meter_profile_get; + + # added in 23.03 + rte_eth_dev_direct_rearm; + rte_eth_rx_flush_descriptor; + rte_eth_rx_queue_rearm_data_get; + rte_eth_tx_fill_sw_ring; }; INTERNAL {
RE: [PATCH v3 1/3] ethdev: enable direct rearm with separate API
Hi Feifei ,
Quoted 27 lines
Quoted 9 lines
lib/ethdev/rte_ethdev.h struct rte_eth_txq_info { · line 1829
Quoted 2 lines
lib/ethdev/rte_ethdev.h +2
-0+uint16_t *rearm_start; +uint16_t *rearm_nb;I know that for Intel NICs
uint16_tis sufficient, wonder would it always be for other vendors?
Another thing to consider the case when ring position wrapping? Again I know that it is not required for Intel NICs, but would it be sufficient for API that supposed to be general?For this, we re-define this structure:
rte_eth_rxq_rearm_data { void *rx_sw_ring; uint16_t *rearm_start; uint16_t *rearm_nb; } -> struct *rxq_recycle_info { rte_mbuf **buf_ring; uint16_t *offset = (uint16 *)(&rq->ci); uint16_t *end; uint16_t ring_size;}
For the new structure, *offset is a pointer for rearm-start index of Rx buffer ring (consumer index). *end is a pointer for rearm-end index Of Rx buffer ring (producer index).
Quoted 4 lines
1. we look up different pmds, some pmds using '
uint_16t' as index size like intel PMD, some pmds using 'uint32_t' as index size like MLX5 or thunderx PMD.
For pmd using 'uint32_t', rearm starts at 'buf_ring[offset & (ring_size-1)]', and 'uint16_t' is enough for ring size.
Sounds like a smart idea to me.
Quoted 4 lines
2. Good question. In general path, there is a constraint that '
nb_rearm<ring_size-rq->ci', This can ensure no ring wrapping in rearm. Thus in direct-rearm, we will refer to this to solve ring wrapping.
Should work, I think...
Just need not to forget to document it :)
回复: [PATCH v3 1/3] ethdev: enable direct rearm with separate API
Quoted 56 lines
-----邮件原件-----
发件人: Konstantin Ananyev <[email protected]> 发送时间: Tuesday, February 28, 2023 3:32 AM
收件人: Feifei Wang <[email protected]>; Konstantin Ananyev <[email protected]>; [email protected]; Ferruh Yigit <[email protected]>; Andrew Rybchenko <[email protected]>
抄送: [email protected]; nd <[email protected]>; Honnappa Nagarahalli <[email protected]>; Ruifeng Wang <[email protected]>; nd <[email protected]>
主题: RE: [PATCH v3 1/3] ethdev: enable direct rearm with separate APIHi Feifei ,
Quoted 22 lines
Quoted 9 lines
lib/ethdev/rte_ethdev.h struct rte_eth_txq_info { · line 1829
Quoted 2 lines
lib/ethdev/rte_ethdev.h +2
-0+uint16_t *rearm_start; +uint16_t *rearm_nb;I know that for Intel NICs
uint16_tis sufficient, wonder would it always be for other vendors?
Another thing to consider the case when ring position wrapping? Again I know that it is not required for Intel NICs, but would it be sufficient for API that supposed to be general?For this, we re-define this structure:
rte_eth_rxq_rearm_data { void *rx_sw_ring; uint16_t *rearm_start; uint16_t *rearm_nb; } -> struct *rxq_recycle_info { rte_mbuf **buf_ring; uint16_t *offset = (uint16 *)(&rq->ci); uint16_t *end; uint16_t ring_size;Quoted 7 lines
}
For the new structure, *offset is a pointer for rearm-start index of Rx buffer ring (consumer index). *end is a pointer for rearm-end index Of Rx buffer ring (producer index).1. we look up different pmds, some pmds using '
uint_16t' as index size like intel PMD, some pmds using 'uint32_t' as index size like MLX5 orthunderx PMD.
Quoted 1 line
For pmd using '
uint32_t', rearm starts at 'buf_ring[offset & (ring_size-1)]',and '
uint16_t'Quoted 1 line
is enough for ring size.
Sounds like a smart idea to me.
Quoted 1 line
Quoted 6 lines
Quoted 3 lines
2. Good question. In general path, there is a constraint that '
nb_rearm<ring_size-rq->ci', This can ensure no ring wrapping in rearm. Thus in direct-rearm, we will refer to this to solve ring wrapping.Should work, I think...
Just need not to forget to document it :)
Agree, we need to doc this.
RE: [PATCH v3 1/3] ethdev: enable direct rearm with separate API
In reply to Konstantin AnanyevHi Feifei,
Quoted 35 lines
From: Konstantin Ananyev [mailto:[email protected]]
Sent: Monday, 27 February 2023 20.32Hi Feifei ,
Quoted 4 lines
Quoted 4 lines
lib/ethdev/rte_ethdev.h struct rte_eth_txq_info { · line 1829
Quoted 2 lines
lib/ethdev/rte_ethdev.h +2
-0+uint16_t *rearm_start; +uint16_t *rearm_nb;I know that for Intel NICs
uint16_tis sufficient, wonder would it alwaysbe
Quoted 3 lines
Quoted 3 lines
for other vendors?
Another thing to consider the case when ring position wrapping? Again I know that it is not required for Intel NICs, but would it besufficient
Quoted 20 lines
Quoted 2 lines
for API that supposed to be general?
For this, we re-define this structure:
rte_eth_rxq_rearm_data { void *rx_sw_ring; uint16_t *rearm_start; uint16_t *rearm_nb; } -> struct *rxq_recycle_info { rte_mbuf **buf_ring; uint16_t *offset = (uint16 *)(&rq->ci); uint16_t *end; uint16_t ring_size;}
For the new structure, *offset is a pointer for rearm-start index of Rx buffer ring (consumer index). *end is a pointer for rearm-end index Of Rx buffer ring (producer index).
Quoted 8 lines
Quoted 1 line
1. we look up different pmds, some pmds using '
uint_16t' as index size likeintel PMD,
Quoted 2 lines
some pmds using '
uint32_t' as index size like MLX5 or thunderx PMD. For pmd using 'uint32_t', rearm starts at 'buf_ring[offset & (ring_size-1)]', and '
uint16_t'Quoted 1 line
is enough for ring size.
Sounds like a smart idea to me.
When configuring an Ethernet device queue, the nb_rx/tx_desc parameter to rte_eth_rx/tx_queue_setup() is uint16_t, so I agree that uint16_t should suffice here too.
I had the following thought, but am not sure. So please take this comment for consideration only:
I think the "& (ring_size -1)" is superfluous, unless a PMD allows its index pointer to exceed the ring size, and performs the same "& (ring_size -1)" when using the index pointer to access its ring.
And if a PMD uses the index pointer like that (i.e. exceeding the ring size), you would need the same wrap protection for a 16 bit index pointer.
Quoted 11 lines
Quoted 4 lines
2. Good question. In general path, there is a constraint that '
nb_rearm<
ring_size-rq->ci',Quoted 1 line
This can ensure no ring wrapping in rearm. Thus in direct-rearm, we will
refer to this to
Quoted 1 line
solve ring wrapping.
Should work, I think...
Just need not to forget to document it :)
It is this constraint (the guarantee that there is no ring wrapping in a rearm burst) that makes me think that the "& (ring_size -1)" is superfluous.
回复: [PATCH v3 1/3] ethdev: enable direct rearm with separate API
Hi, Morten
Quoted 85 lines
-----邮件原件-----
发件人: Morten Brørup <[email protected]>
发送时间: Tuesday, February 28, 2023 4:09 PM
收件人: Konstantin Ananyev <[email protected]>; Feifei Wang <[email protected]>; Konstantin Ananyev <[email protected]>; [email protected]; Ferruh Yigit <[email protected]>; Andrew Rybchenko <[email protected]>
抄送: [email protected]; nd <[email protected]>; Honnappa Nagarahalli <[email protected]>; Ruifeng Wang <[email protected]>; nd <[email protected]>
主题: RE: [PATCH v3 1/3] ethdev: enable direct rearm with separate APIHi Feifei,
Quoted 39 lines
From: Konstantin Ananyev [mailto:[email protected]]
Sent: Monday, 27 February 2023 20.32Hi Feifei ,
Quoted 5 lines
Quoted 5 lines
lib/ethdev/rte_ethdev.h struct rte_eth_txq_info { · line 1829
Quoted 2 lines
lib/ethdev/rte_ethdev.h +2
-0+uint16_t *rearm_start; +uint16_t *rearm_nb;I know that for Intel NICs
uint16_tis sufficient, wonder would it alwaysbe
Quoted 4 lines
Quoted 4 lines
for other vendors?
Another thing to consider the case when ring position wrapping? Again I know that it is not required for Intel NICs, but would it besufficient
Quoted 15 lines
Quoted 2 lines
for API that supposed to be general?
For this, we re-define this structure:
rte_eth_rxq_rearm_data { void *rx_sw_ring; uint16_t *rearm_start; uint16_t *rearm_nb; } -> struct *rxq_recycle_info { rte_mbuf **buf_ring; uint16_t *offset = (uint16 *)(&rq->ci); uint16_t *end; uint16_t ring_size;Quoted 7 lines
}
For the new structure, *offset is a pointer for rearm-start index of Rx buffer ring (consumer index). *end is a pointer for rearm-end index Of Rx buffer ring (producer index).1. we look up different pmds, some pmds using '
uint_16t' as index size likeQuoted 8 lines
intel PMD,
Quoted 3 lines
some pmds using '
uint32_t' as index size like MLX5 or thunderx PMD. For pmd using 'uint32_t', rearm starts at 'buf_ring[offset & (ring_size-1)]', and '
uint16_t'Quoted 1 line
is enough for ring size.
Sounds like a smart idea to me.
When configuring an Ethernet device queue, the nb_rx/tx_desc parameter to rte_eth_rx/tx_queue_setup() is
uint16_t, so I agree thatuint16_tshould suffice here too.I had the following thought, but am not sure. So please take this comment for consideration only:
I think the "& (
ring_size-1)" is superfluous, unless a PMD allows its index pointer to exceed the ring size, and performs the same "& (ring_size-1)" when using the index pointer to access its ring.And if a PMD uses the index pointer like that (i.e. exceeding the ring size), you would need the same wrap protection for a 16 bit index pointer.
Quoted 8 lines
Quoted 5 lines
2. Good question. In general path, there is a constraint that '
nb_rearm<
ring_size-rq->ci',Quoted 2 lines
This can ensure no ring wrapping in rearm. Thus in direct-rearm, we will
Quoted 8 lines
Quoted 5 lines
refer to this to
Quoted 1 line
solve ring wrapping.
Should work, I think...
Just need not to forget to document it :)It is this constraint (the guarantee that there is no ring wrapping in a rearm burst) that makes me think that the "& (
ring_size-1)" is superfluous.
Actually there is some misleading here, I can explain about this.
(ring_size -1 ) is for some pmds whose index can exceed the ring_size, such as MLX5:uint16_t *offset = &mlx5_rxq->rq_ci, rq_ci is an index which can exceed the ring_size.
PATCH v3 2/3 net/i40e: enable direct rearm with separate API
In reply to Feifei WangAdd internal API to separate direct rearm operations between Rx and Tx.
--- drivers/net/i40e/i40e_ethdev.c | 1 + drivers/net/i40e/i40e_ethdev.h | 2 + drivers/net/i40e/i40e_rxtx.c | 19 +++++++++ drivers/net/i40e/i40e_rxtx.h | 4 ++ drivers/net/i40e/i40e_rxtx_vec_common.h | 54 +++++++++++++++++++++++++ drivers/net/i40e/i40e_rxtx_vec_neon.c | 42 +++++++++++++++++++ 6 files changed, 122 insertions(+)
drivers/net/i40e/i40e_ethdev.c +1 -0
@@ -497,6 +497,7 @@ static const struct eth_dev_ops i40e_eth_dev_ops = { .flow_ops_get = i40e_dev_flow_ops_get, .rxq_info_get = i40e_rxq_info_get, .txq_info_get = i40e_txq_info_get, + .rxq_rearm_data_get = i40e_rxq_rearm_data_get, .rx_burst_mode_get = i40e_rx_burst_mode_get, .tx_burst_mode_get = i40e_tx_burst_mode_get, .timesync_enable = i40e_timesync_enable,
drivers/net/i40e/i40e_ethdev.h +2 -0
@@ -1352,6 +1352,8 @@ void i40e_rxq_info_get(struct rte_eth_dev *dev, uint16_t queue_id, struct rte_eth_rxq_info *qinfo);void i40e_txq_info_get (struct rte_eth_dev *dev,uint16_t queue_id,struct rte_eth_txq_info *qinfo); +void i40e_rxq_rearm_data_get (struct rte_eth_dev *dev,uint16_t queue_id, +struct rte_eth_rxq_rearm_data *rxq_rearm_data);int i40e_rx_burst_mode_get (struct rte_eth_dev *dev,uint16_t queue_id,struct rte_eth_burst_mode *mode); int i40e_tx_burst_mode_get(struct rte_eth_dev *dev, uint16_t queue_id,
drivers/net/i40e/i40e_rxtx.c +19 -0
@@ -3197,6 +3197,19 @@ i40e_txq_info_get(struct rte_eth_dev *dev, uint16_t queue_id, qinfo->conf.offloads = txq->offloads; } +void +i40e_rxq_rearm_data_get (struct rte_eth_dev *dev,uint16_t queue_id, +struct rte_eth_rxq_rearm_data *rxq_rearm_data) +{ +struct i40e_rx_queue *rxq; + + rxq = dev->data->rx_queues[queue_id]; + + rxq_rearm_data->rx_sw_ring = rxq->sw_ring; + rxq_rearm_data->rearm_start = &rxq->rxrearm_start; + rxq_rearm_data->rearm_nb = &rxq->rxrearm_nb; +} +#ifdef RTE_ARCH_X86 static inline bool get_avx_supported (bool request_avx512)
@@ -3321,6 +3334,9 @@ i40e_set_rx_function(struct rte_eth_dev *dev)PMD_INIT_LOG (DEBUG ,"Using Vector Rx (port %d)." , dev->data->port_id); dev->rx_pkt_burst = i40e_recv_pkts_vec; +#ifdef RTE_ARCH_ARM64 + dev->rx_flush_descriptor = i40e_rx_flush_descriptor_vec; +#endif }#endif /* RTE_ARCH_X86 */ }else if (!dev->data->scattered_rx && ad->rx_bulk_alloc_allowed) {
@@ -3484,6 +3500,9 @@ i40e_set_tx_function(struct rte_eth_dev *dev)PMD_INIT_LOG (DEBUG ,"Using Vector Tx (port %d)." , dev->data->port_id); dev->tx_pkt_burst = i40e_xmit_pkts_vec; +#ifdef RTE_ARCH_ARM64 + dev->tx_fill_sw_ring = i40e_tx_fill_sw_ring; +#endif #endif /* RTE_ARCH_X86 */ }else {PMD_INIT_LOG (DEBUG ,"Simple tx finally be used." );
drivers/net/i40e/i40e_rxtx.h +4 -0
@@ -233,6 +233,10 @@ uint32_t i40e_dev_rx_queue_count(void *rx_queue);int i40e_dev_rx_descriptor_status (void *rx_queue,uint16_t offset);int i40e_dev_tx_descriptor_status (void *tx_queue,uint16_t offset); +int i40e_tx_fill_sw_ring (void *tx_queue, +struct rte_eth_rxq_rearm_data *rxq_rearm_data); +int i40e_rx_flush_descriptor_vec (void *rx_queue,uint16_t nb_rearm); +uint16_t i40e_recv_pkts_vec (void *rx_queue,struct rte_mbuf **rx_pkts,uint16_t nb_pkts); uint16_t i40e_recv_scattered_pkts_vec(void *rx_queue,
drivers/net/i40e/i40e_rxtx_vec_common.h +54 -0
60 lines not on this page · raw message
drivers/net/i40e/i40e_rxtx_vec_neon.c +42 -0
48 lines not on this page · raw message
Signature · 1 line
2.25.1
Re: [PATCH v3 2/3] net/i40e: enable direct rearm with separate API
04/01/2023 07:30, Feifei Wang пишет:
Quoted 76 lines
Add internal API to separate direct rearm operations between Rx and Tx.
Suggested-by Honnappa Nagarahalli <[email protected]>Signed-off-by Feifei Wang <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>Reviewed-by Honnappa Nagarahalli <[email protected]>--- drivers/net/i40e/i40e_ethdev.c | 1 + drivers/net/i40e/i40e_ethdev.h | 2 + drivers/net/i40e/i40e_rxtx.c | 19 +++++++++ drivers/net/i40e/i40e_rxtx.h | 4 ++ drivers/net/i40e/i40e_rxtx_vec_common.h | 54 +++++++++++++++++++++++++ drivers/net/i40e/i40e_rxtx_vec_neon.c | 42 +++++++++++++++++++ 6 files changed, 122 insertions(+)drivers/net/i40e/i40e_ethdev.c +1
-0@@ -497,6 +497,7 @@ static const struct eth_dev_ops i40e_eth_dev_ops = { .flow_ops_get = i40e_dev_flow_ops_get, .rxq_info_get = i40e_rxq_info_get, .txq_info_get = i40e_txq_info_get, + .rxq_rearm_data_get = i40e_rxq_rearm_data_get, .rx_burst_mode_get = i40e_rx_burst_mode_get, .tx_burst_mode_get = i40e_tx_burst_mode_get, .timesync_enable = i40e_timesync_enable,drivers/net/i40e/i40e_ethdev.h +2
-0@@ -1352,6 +1352,8 @@ void i40e_rxq_info_get(struct rte_eth_dev *dev, uint16_t queue_id, struct rte_eth_rxq_info *qinfo);void i40e_txq_info_get (struct rte_eth_dev *dev,uint16_t queue_id,struct rte_eth_txq_info *qinfo); +void i40e_rxq_rearm_data_get (struct rte_eth_dev *dev,uint16_t queue_id, +struct rte_eth_rxq_rearm_data *rxq_rearm_data);int i40e_rx_burst_mode_get (struct rte_eth_dev *dev,uint16_t queue_id,struct rte_eth_burst_mode *mode); int i40e_tx_burst_mode_get(struct rte_eth_dev *dev, uint16_t queue_id,drivers/net/i40e/i40e_rxtx.c +16
-0@@ -3197,6 +3197,19 @@ i40e_txq_info_get(struct rte_eth_dev *dev, uint16_t queue_id, qinfo->conf.offloads = txq->offloads; } +void +i40e_rxq_rearm_data_get (struct rte_eth_dev *dev,uint16_t queue_id, +struct rte_eth_rxq_rearm_data *rxq_rearm_data) +{ +struct i40e_rx_queue *rxq; + + rxq = dev->data->rx_queues[queue_id]; + + rxq_rearm_data->rx_sw_ring = rxq->sw_ring; + rxq_rearm_data->rearm_start = &rxq->rxrearm_start; + rxq_rearm_data->rearm_nb = &rxq->rxrearm_nb; +} +#ifdef RTE_ARCH_X86 static inline bool get_avx_supported (bool request_avx512)@@ -3321,6 +3334,9 @@ i40e_set_rx_function(struct rte_eth_dev *dev)PMD_INIT_LOG (DEBUG ,"Using Vector Rx (port %d)." , dev->data->port_id); dev->rx_pkt_burst = i40e_recv_pkts_vec; +#ifdef RTE_ARCH_ARM64 + dev->rx_flush_descriptor = i40e_rx_flush_descriptor_vec; +#endif }#endif /* RTE_ARCH_X86 */ }else if (!dev->data->scattered_rx && ad->rx_bulk_alloc_allowed) {
drivers/net/i40e/i40e_rxtx.c i40e_set_tx_function(struct rte_eth_dev *dev) · line 3505
Quoted 6 lines
drivers/net/i40e/i40e_rxtx.c +3
-0@@ -3484,6 +3500,9 @@ i40e_set_tx_function(struct rte_eth_dev *dev)PMD_INIT_LOG (DEBUG ,"Using Vector Tx (port %d)." , dev->data->port_id); dev->tx_pkt_burst = i40e_xmit_pkts_vec; +#ifdef RTE_ARCH_ARM64 + dev->tx_fill_sw_ring = i40e_tx_fill_sw_ring; +#endif
As I can see tx_fill_sw_ring() is non ARM specific, any reason to guard it with #ifdef ARM? Actually same ask for rx_flush_descriptor() - can we have generic version too?
Quoted 136 lines
drivers/net/i40e/i40e_rxtx.c +0
-0#endif /* RTE_ARCH_X86 */ }else {PMD_INIT_LOG (DEBUG ,"Simple tx finally be used." );drivers/net/i40e/i40e_rxtx.h +4
-0@@ -233,6 +233,10 @@ uint32_t i40e_dev_rx_queue_count(void *rx_queue);int i40e_dev_rx_descriptor_status (void *rx_queue,uint16_t offset);int i40e_dev_tx_descriptor_status (void *tx_queue,uint16_t offset); +int i40e_tx_fill_sw_ring (void *tx_queue, +struct rte_eth_rxq_rearm_data *rxq_rearm_data); +int i40e_rx_flush_descriptor_vec (void *rx_queue,uint16_t nb_rearm); +uint16_t i40e_recv_pkts_vec (void *rx_queue,struct rte_mbuf **rx_pkts,uint16_t nb_pkts); uint16_t i40e_recv_scattered_pkts_vec(void *rx_queue,drivers/net/i40e/i40e_rxtx_vec_common.h +54
-0@@ -146,6 +146,60 @@ i40e_tx_free_bufs(struct i40e_tx_queue *txq)return txq->tx_rs_thresh; } +int +i40e_tx_fill_sw_ring (void *tx_queue, +struct rte_eth_rxq_rearm_data *rxq_rearm_data) +{ +struct i40e_tx_queue *txq = tx_queue; +struct i40e_tx_entry *txep; +void **rxep; +struct rte_mbuf *m; +int i, n; +int nb_rearm =0 ; + +if (*rxq_rearm_data->rearm_nb < txq->tx_rs_thresh || + txq->nb_tx_free > txq->tx_free_thresh) +return 0 ; + +/* check DD bits on threshold descriptor */ +if ((txq->tx_ring[txq->tx_next_dd].cmd_type_offset_bsz & +rte_cpu_to_le_64 (I40E_TXD_QW1_DTYPE_MASK )) != +rte_cpu_to_le_64 (I40E_TX_DESC_DTYPE_DESC_DONE )) +return 0 ; + + n = txq->tx_rs_thresh; + +/* first buffer to free from S/W ring is at index +* tx_next_dd - (tx_rs_thresh-1) +*/ + txep = &txq->sw_ring[txq->tx_next_dd - (n -1 )]; + rxep = rxq_rearm_data->rx_sw_ring; + rxep += *rxq_rearm_data->rearm_start; + +if (txq->offloads &RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE ) { +/* directly put mbufs from Tx to Rx */ +for (i =0 ; i < n; i++, rxep++, txep++) + *rxep = txep[0 ].mbuf; + }else { +for (i =0 ; i < n; i++, rxep++) { + m =rte_pktmbuf_prefree_seg (txep[i].mbuf); +if (m !=NULL ) { + *rxep = m; + nb_rearm++; + } + } + n = nb_rearm; + } + +/* update counters for Tx */ + txq->nb_tx_free = (uint16_t )(txq->nb_tx_free + txq->tx_rs_thresh); + txq->tx_next_dd = (uint16_t )(txq->tx_next_dd + txq->tx_rs_thresh); +if (txq->tx_next_dd >= txq->nb_tx_desc) + txq->tx_next_dd = (uint16_t )(txq->tx_rs_thresh -1 ); + +return n; +} + static __rte_always_inline voidtx_backlog_entry (struct i40e_tx_entry *txep,struct rte_mbuf **tx_pkts,uint16_t nb_pkts)drivers/net/i40e/i40e_rxtx_vec_neon.c +42
-0@@ -739,6 +739,48 @@ i40e_xmit_fixed_burst_vec(void *__rte_restrict tx_queue,return nb_pkts; } +int +i40e_rx_flush_descriptor_vec (void *rx_queue,uint16_t nb_rearm) +{ +struct i40e_rx_queue *rxq = rx_queue; +struct i40e_rx_entry *rxep; +volatile union i40e_rx_desc *rxdp; +uint16_t rx_id; +uint64x2_t dma_addr; +uint64_t paddr; +uint16_t i; + + rxdp = rxq->rx_ring + rxq->rxrearm_start; + rxep = &rxq->sw_ring[rxq->rxrearm_start]; + +for (i =0 ; i < nb_rearm; i++) { +/* Initialize rxdp descs */ + paddr = (rxep[i].mbuf)->buf_iova +RTE_PKTMBUF_HEADROOM ; + dma_addr =vdupq_n_u64 (paddr); +/* flush desc with pa dma_addr */ +vst1q_u64 ((uint64_t *)&rxdp++->read, dma_addr); + } + +/* Update the descriptor initializer index */ + rxq->rxrearm_start += nb_rearm; + rx_id = rxq->rxrearm_start -1 ; + +if (unlikely (rxq->rxrearm_start >= rxq->nb_rx_desc)) { + rxq->rxrearm_start = rxq->rxrearm_start - rxq->nb_rx_desc; +if (!rxq->rxrearm_start) + rx_id = rxq->nb_rx_desc -1 ; +else + rx_id = rxq->rxrearm_start -1 ; + } + rxq->rxrearm_nb -= nb_rearm; + +rte_io_wmb (); +/* Update the tail pointer on the NIC */ +I40E_PCI_REG_WRITE_RELAXED (rxq->qrx_tail, rx_id); + +return 0 ; +} + void __rte_coldi40e_rx_queue_release_mbufs_vec (struct i40e_rx_queue *rxq) {
回复: [PATCH v3 2/3] net/i40e: enable direct rearm with separate API
Quoted 88 lines
-----邮件原件-----
发件人: Konstantin Ananyev <[email protected]> 发送时间: Thursday, February 2, 2023 10:38 PM
收件人: Feifei Wang <[email protected]>; Yuying Zhang <[email protected]>; Beilei Xing <[email protected]>; Ruifeng Wang <[email protected]>
抄送: [email protected]; nd <[email protected]>; Honnappa Nagarahalli <[email protected]>
主题: Re: [PATCH v3 2/3] net/i40e: enable direct rearm with separate API04/01/2023 07:30, Feifei Wang пишет:
Quoted 7 lines
Add internal API to separate direct rearm operations between Rx and Tx.
Suggested-by Honnappa Nagarahalli <[email protected]>Signed-off-by Feifei Wang <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>Reviewed-by Honnappa Nagarahalli <[email protected]>Quoted 6 lines
--- drivers/net/i40e/i40e_ethdev.c | 1 + drivers/net/i40e/i40e_ethdev.h | 2 + drivers/net/i40e/i40e_rxtx.c | 19 +++++++++ drivers/net/i40e/i40e_rxtx.h | 4 ++ drivers/net/i40e/i40e_rxtx_vec_common.h | 54+++++++++++++++++++++++++
drivers/net/i40e/i40e_ethdev.c static const struct eth_dev_ops i40e_eth_dev_ops
Quoted 8 lines
drivers/net/i40e/i40e_rxtx_vec_neon.c | 42 +++++++++++++++++++ 6 files changed, 122 insertions(+)(quoted diff) +0
-0
b/drivers/net/i40e/i40e_ethdev.cindex7726a89d99..29c1ce2470100644drivers/net/i40e/i40e_ethdev.c +0
-0@@ -497,6 +497,7 @@ static const struct eth_dev_ops i40e_eth_dev_ops= {
drivers/net/i40e/i40e_ethdev.h void i40e_rxq_info_get(struct rte_eth_dev *dev,
Quoted 12 lines
drivers/net/i40e/i40e_ethdev.c +1
-0.flow_ops_get = i40e_dev_flow_ops_get, .rxq_info_get = i40e_rxq_info_get, .txq_info_get = i40e_txq_info_get, + .rxq_rearm_data_get = i40e_rxq_rearm_data_get, .rx_burst_mode_get = i40e_rx_burst_mode_get, .tx_burst_mode_get = i40e_tx_burst_mode_get, .timesync_enable = i40e_timesync_enable,(quoted diff) +0
-0
b/drivers/net/i40e/i40e_ethdev.hindexfe943a45ff..6a6a2a6d3c100644drivers/net/i40e/i40e_ethdev.h +0
-0@@ -1352,6 +1352,8 @@ void i40e_rxq_info_get(struct rte_eth_dev *dev,
uint16_tqueue_id,drivers/net/i40e/i40e_ethdev.h void i40e_rxq_info_get(struct rte_eth_dev *dev, uint16_t queue_id,
Quoted 4 lines
drivers/net/i40e/i40e_ethdev.h +1
-0struct rte_eth_rxq_info *qinfo);void i40e_txq_info_get (struct rte_eth_dev *dev,uint16_t queue_id,struct rte_eth_txq_info *qinfo); +void i40e_rxq_rearm_data_get(struct rte_eth_dev *dev, uint16_t
queue_id,drivers/net/i40e/i40e_rxtx.c i40e_txq_info_get(struct rte_eth_dev *dev,
Quoted 9 lines
drivers/net/i40e/i40e_ethdev.h +1
-0+ struct rte_eth_rxq_rearm_data *rxq_rearm_data);int i40e_rx_burst_mode_get (struct rte_eth_dev *dev,uint16_t queue_id,struct rte_eth_burst_mode *mode); int i40e_tx_burst_mode_get(struct rte_eth_dev *dev, uint16_t
queue_id, diff --gita/drivers/net/i40e/i40e_rxtx.cb/drivers/net/i40e/i40e_rxtx.cindex788ffb51c2..d8d801acaf100644drivers/net/i40e/i40e_rxtx.c +0
-0@@ -3197,6 +3197,19 @@ i40e_txq_info_get(struct rte_eth_dev *dev,
uint16_tqueue_id,Quoted 21 lines
drivers/net/i40e/i40e_rxtx.c +11
-0qinfo->conf.offloads = txq->offloads; } +void +i40e_rxq_rearm_data_get (struct rte_eth_dev *dev,uint16_t queue_id, +struct rte_eth_rxq_rearm_data *rxq_rearm_data) { +struct i40e_rx_queue *rxq; + + rxq = dev->data->rx_queues[queue_id]; + + rxq_rearm_data->rx_sw_ring = rxq->sw_ring; + rxq_rearm_data->rearm_start = &rxq->rxrearm_start; + rxq_rearm_data->rearm_nb = &rxq->rxrearm_nb; } +#ifdef RTE_ARCH_X86 static inline bool get_avx_supported(bool request_avx512) @@ -3321,6 +3334,9 @@
i40e_set_rx_function(struct rte_eth_dev *dev)
PMD_INIT_LOG(DEBUG, "Using Vector Rx (port %d).",
dev->data->port_id);
dev->rx_pkt_burst=i40e_recv_pkts_vec;drivers/net/i40e/i40e_rxtx.c
Quoted 2 lines
drivers/net/i40e/i40e_rxtx.c +2
-0+#ifdef RTE_ARCH_ARM64 + dev->rx_flush_descriptor =
i40e_rx_flush_descriptor_vec; #endif
Quoted 13 lines
drivers/net/i40e/i40e_rxtx.c i40e_set_tx_function(struct rte_eth_dev *dev) · line 3504
Quoted 9 lines
} #endif /* RTE_ARCH_X86 */ } else if (!dev->data->scattered_rx && ad->rx_bulk_alloc_allowed) {drivers/net/i40e/i40e_rxtx.c +2
-0@@ -3484,6 +3500,9 @@ i40e_set_tx_function(struct rte_eth_dev *dev)PMD_INIT_LOG (DEBUG ,"Using Vector Tx (port %d)." , dev->data->port_id); dev->tx_pkt_burst = i40e_xmit_pkts_vec; +#ifdef RTE_ARCH_ARM64 + dev->tx_fill_sw_ring = i40e_tx_fill_sw_ring;#endif As I can see
tx_fill_sw_ring()is non ARM specific, any reason to guard it with #ifdef ARM?
Actually same ask forrx_flush_descriptor()- can we have generic version too?
Here we consider direct-rearm not enable in other architecture. Agree with that we need to have generic version to avoid this, I will update in the next version.
Quoted 142 lines
drivers/net/i40e/i40e_rxtx.c i40e_set_tx_function(struct rte_eth_dev *dev) · line 3508
Quoted 4 lines
drivers/net/i40e/i40e_rxtx.c +0
-0#endif /* RTE_ARCH_X86 */ }else {PMD_INIT_LOG (DEBUG ,"Simple tx finally be used." );diff --git
drivers/net/i40e/i40e_rxtx.h uint32_t i40e_dev_rx_queue_count(void
Quoted 5 lines
a/drivers/net/i40e/i40e_rxtx.hb/drivers/net/i40e/i40e_rxtx.hindex5e6eecc501..8a29bd89df100644drivers/net/i40e/i40e_rxtx.h +0
-0@@ -233,6 +233,10 @@ uint32_t i40e_dev_rx_queue_count(void
*rx_queue);Quoted 27 lines
drivers/net/i40e/i40e_rxtx.h +4
-0int i40e_dev_rx_descriptor_status (void *rx_queue,uint16_t offset);int i40e_dev_tx_descriptor_status (void *tx_queue,uint16_t offset); +int i40e_tx_fill_sw_ring (void *tx_queue, +struct rte_eth_rxq_rearm_data *rxq_rearm_data);int +i40e_rx_flush_descriptor_vec (void *rx_queue,uint16_t nb_rearm); +uint16_t i40e_recv_pkts_vec (void *rx_queue,struct rte_mbuf **rx_pkts,uint16_t nb_pkts); uint16_t i40e_recv_scattered_pkts_vec(void *rx_queue, diff --git
a/drivers/net/i40e/i40e_rxtx_vec_common.hb/drivers/net/i40e/i40e_rxtx_vec_common.h
indexfe1a6ec75e..eb96301a43100644drivers/net/i40e/i40e_rxtx_vec_common.h +8
-0@@ -146,6 +146,60 @@ i40e_tx_free_bufs(struct i40e_tx_queue *txq)return txq->tx_rs_thresh; } +int +i40e_tx_fill_sw_ring (void *tx_queue, +struct rte_eth_rxq_rearm_data *rxq_rearm_data) { +struct i40e_tx_queue *txq = tx_queue; +struct i40e_tx_entry *txep; +void **rxep; +struct rte_mbuf *m; +int i, n;drivers/net/i40e/i40e_rxtx_vec_common.h i40e_tx_free_bufs(struct i40e_tx_queue *txq) · line 166
Quoted 10 lines
drivers/net/i40e/i40e_rxtx_vec_common.h +10
-0+int nb_rearm =0 ; + +if (*rxq_rearm_data->rearm_nb < txq->tx_rs_thresh || + txq->nb_tx_free > txq->tx_free_thresh) +return 0 ; + +/* check DD bits on threshold descriptor */ + if ((txq->tx_ring[txq->tx_next_dd].cmd_type_offset_bsz & + rte_cpu_to_le_64(I40E_TXD_QW1_DTYPE_MASK)) != +rte_cpu_to_le_64(I40E_TX_DESC_DTYPE_DESC_DONE))drivers/net/i40e/i40e_rxtx_vec_neon.c i40e_xmit_fixed_burst_vec(void *__rte_restrict
Quoted 44 lines
drivers/net/i40e/i40e_rxtx_vec_common.h +35
-0+return 0 ; + + n = txq->tx_rs_thresh; + +/* first buffer to free from S/W ring is at index +* tx_next_dd - (tx_rs_thresh-1) +*/ + txep = &txq->sw_ring[txq->tx_next_dd - (n -1 )]; + rxep = rxq_rearm_data->rx_sw_ring; + rxep += *rxq_rearm_data->rearm_start; + +if (txq->offloads &RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE ) { +/* directly put mbufs from Tx to Rx */ +for (i =0 ; i < n; i++, rxep++, txep++) + *rxep = txep[0 ].mbuf; + }else { +for (i =0 ; i < n; i++, rxep++) { + m =rte_pktmbuf_prefree_seg (txep[i].mbuf); +if (m !=NULL ) { + *rxep = m; + nb_rearm++; + } + } + n = nb_rearm; + } + +/* update counters for Tx */ + txq->nb_tx_free = (uint16_t )(txq->nb_tx_free + txq->tx_rs_thresh); + txq->tx_next_dd = (uint16_t )(txq->tx_next_dd + txq->tx_rs_thresh); +if (txq->tx_next_dd >= txq->nb_tx_desc) + txq->tx_next_dd = (uint16_t )(txq->tx_rs_thresh -1 ); + +return n; +} + static __rte_always_inline void tx_backlog_entry(struct i40e_tx_entry *txep, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) diff --git
a/drivers/net/i40e/i40e_rxtx_vec_neon.cb/drivers/net/i40e/i40e_rxtx_vec_neon.c
index12e6f1cbcb..1509d3223b100644drivers/net/i40e/i40e_rxtx_vec_neon.c +0
-0@@ -739,6 +739,48 @@ i40e_xmit_fixed_burst_vec(void *__rte_restrict
tx_queue,Quoted 9 lines
drivers/net/i40e/i40e_rxtx_vec_neon.c +6
-0return nb_pkts; } +int +i40e_rx_flush_descriptor_vec (void *rx_queue,uint16_t nb_rearm) { +struct i40e_rx_queue *rxq = rx_queue; +struct i40e_rx_entry *rxep; +volatile union i40e_rx_desc *rxdp; +uint16_t rx_id;drivers/net/i40e/i40e_rxtx_vec_neon.c i40e_xmit_fixed_burst_vec(void *__rte_restrict tx_queue,
Quoted 10 lines
drivers/net/i40e/i40e_rxtx_vec_neon.c +10
-0+uint64x2_t dma_addr; +uint64_t paddr; +uint16_t i; + + rxdp = rxq->rx_ring + rxq->rxrearm_start; + rxep = &rxq->sw_ring[rxq->rxrearm_start]; + +for (i =0 ; i < nb_rearm; i++) { +/* Initialize rxdp descs */ + paddr = (rxep[i].mbuf)->buf_iova +
RTE_PKTMBUF_HEADROOM;Quoted 28 lines
drivers/net/i40e/i40e_rxtx_vec_neon.c +25
-0+ dma_addr =vdupq_n_u64 (paddr); +/* flush desc with pa dma_addr */ +vst1q_u64 ((uint64_t *)&rxdp++->read, dma_addr); + } + +/* Update the descriptor initializer index */ + rxq->rxrearm_start += nb_rearm; + rx_id = rxq->rxrearm_start -1 ; + +if (unlikely (rxq->rxrearm_start >= rxq->nb_rx_desc)) { + rxq->rxrearm_start = rxq->rxrearm_start - rxq->nb_rx_desc; +if (!rxq->rxrearm_start) + rx_id = rxq->nb_rx_desc -1 ; +else + rx_id = rxq->rxrearm_start -1 ; + } + rxq->rxrearm_nb -= nb_rearm; + +rte_io_wmb (); +/* Update the tail pointer on the NIC */ +I40E_PCI_REG_WRITE_RELAXED (rxq->qrx_tail, rx_id); + +return 0 ; +} + void __rte_coldi40e_rx_queue_release_mbufs_vec (struct i40e_rx_queue *rxq) {
RE: [PATCH v3 2/3] net/i40e: enable direct rearm with separate API
Quoted 31 lines
Quoted 31 lines
Quoted 10 lines
drivers/net/i40e/i40e_rxtx_vec_common.h +8
-0+int +i40e_tx_fill_sw_ring (void *tx_queue, +struct rte_eth_rxq_rearm_data *rxq_rearm_data) { +struct i40e_tx_queue *txq = tx_queue; +struct i40e_tx_entry *txep; +void **rxep; +struct rte_mbuf *m; +int i, n;drivers/net/i40e/i40e_rxtx_vec_common.h i40e_tx_free_bufs(struct i40e_tx_queue *txq) · line 169
Quoted 10 lines
drivers/net/i40e/i40e_rxtx_vec_common.h +10
-0+int nb_rearm =0 ; + +if (*rxq_rearm_data->rearm_nb < txq->tx_rs_thresh || + txq->nb_tx_free > txq->tx_free_thresh) +return 0 ; + +/* check DD bits on threshold descriptor */ + if ((txq->tx_ring[txq->tx_next_dd].cmd_type_offset_bsz & + rte_cpu_to_le_64(I40E_TXD_QW1_DTYPE_MASK)) != +rte_cpu_to_le_64(I40E_TX_DESC_DTYPE_DESC_DONE))Quoted 10 lines
drivers/net/i40e/i40e_rxtx_vec_common.h +10
-0+return 0 ; + + n = txq->tx_rs_thresh; + +/* first buffer to free from S/W ring is at index +* tx_next_dd - (tx_rs_thresh-1) +*/ + txep = &txq->sw_ring[txq->tx_next_dd - (n -1 )]; + rxep = rxq_rearm_data->rx_sw_ring; + rxep += *rxq_rearm_data->rearm_start;
Quoted 8 lines
Quoted 8 lines
drivers/net/i40e/i40e_rxtx_vec_common.h i40e_tx_free_bufs(struct i40e_tx_queue *txq) · line 185
Quoted 8 lines
drivers/net/i40e/i40e_rxtx_vec_common.h +8
-0+ +if (txq->offloads &RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE ) { +/* directly put mbufs from Tx to Rx */ +for (i =0 ; i < n; i++, rxep++, txep++) + *rxep = txep[0 ].mbuf; + }else { +for (i =0 ; i < n; i++, rxep++) { + m =rte_pktmbuf_prefree_seg (txep[i].mbuf);
One thing I forgot to ask:
What would happen if this mbuf belongs to different mempool (not one that we specify at rx_queue_setup())?
Do we need to check it here?
Or would it be upper layer constraint?
Or...?
Quoted 17 lines
Quoted 17 lines
Quoted 17 lines
drivers/net/i40e/i40e_rxtx_vec_common.h +17
-0+if (m !=NULL ) { + *rxep = m; + nb_rearm++; + } + } + n = nb_rearm; + } + +/* update counters for Tx */ + txq->nb_tx_free = (uint16_t )(txq->nb_tx_free + txq->tx_rs_thresh); + txq->tx_next_dd = (uint16_t )(txq->tx_next_dd + txq->tx_rs_thresh); +if (txq->tx_next_dd >= txq->nb_tx_desc) + txq->tx_next_dd = (uint16_t )(txq->tx_rs_thresh -1 ); + +return n; +} +
回复: [PATCH v3 2/3] net/i40e: enable direct rearm with separate API
Quoted 51 lines
-----邮件原件-----
发件人: Konstantin Ananyev <[email protected]> 发送时间: Tuesday, February 28, 2023 3:36 AM
收件人: Feifei Wang <[email protected]>; Konstantin Ananyev <[email protected]>; Yuying Zhang <[email protected]>; Beilei Xing <[email protected]>; Ruifeng Wang <[email protected]>
抄送: [email protected]; nd <[email protected]>; Honnappa Nagarahalli <[email protected]>; nd <[email protected]>
主题: RE: [PATCH v3 2/3] net/i40e: enable direct rearm with separate APIQuoted 36 lines
Quoted 36 lines
Quoted 8 lines
drivers/net/i40e/i40e_rxtx_vec_common.h +8
-0+int +i40e_tx_fill_sw_ring (void *tx_queue, +struct rte_eth_rxq_rearm_data *rxq_rearm_data) { +struct i40e_tx_queue *txq = tx_queue; +struct i40e_tx_entry *txep; +void **rxep; +struct rte_mbuf *m; +int i, n;drivers/net/i40e/i40e_rxtx_vec_common.h i40e_tx_free_bufs(struct i40e_tx_queue *txq) · line 169
Quoted 10 lines
drivers/net/i40e/i40e_rxtx_vec_common.h +10
-0+int nb_rearm =0 ; + +if (*rxq_rearm_data->rearm_nb < txq->tx_rs_thresh || + txq->nb_tx_free > txq->tx_free_thresh) +return 0 ; + +/* check DD bits on threshold descriptor */ + if ((txq->tx_ring[txq->tx_next_dd].cmd_type_offset_bsz & + rte_cpu_to_le_64(I40E_TXD_QW1_DTYPE_MASK)) != +rte_cpu_to_le_64(I40E_TX_DESC_DTYPE_DESC_DONE))Quoted 17 lines
drivers/net/i40e/i40e_rxtx_vec_common.h +17
-0+return 0 ; + + n = txq->tx_rs_thresh; + +/* first buffer to free from S/W ring is at index +* tx_next_dd - (tx_rs_thresh-1) +*/ + txep = &txq->sw_ring[txq->tx_next_dd - (n -1 )]; + rxep = rxq_rearm_data->rx_sw_ring; + rxep += *rxq_rearm_data->rearm_start; + +if (txq->offloads &RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE ) { +/* directly put mbufs from Tx to Rx */ +for (i =0 ; i < n; i++, rxep++, txep++) + *rxep = txep[0 ].mbuf; + }else { +for (i =0 ; i < n; i++, rxep++) {
Quoted 9 lines
Quoted 1 line
Quoted 1 line
drivers/net/i40e/i40e_rxtx_vec_common.h i40e_tx_free_bufs(struct i40e_tx_queue *txq) · line 185
Quoted 1 line
drivers/net/i40e/i40e_rxtx_vec_common.h +1
-0+ m =rte_pktmbuf_prefree_seg (txep[i].mbuf);One thing I forgot to ask:
What would happen if this mbuf belongs to different mempool (not one that we specify atrx_queue_setup())?
Do we need to check it here?
Or would it be upper layer constraint?
Or...?
First, 'different mempool' is valid for no FAST_FREE path in tx_free_buffers.
If buffers belong to different mempool, we can have an example here:
Buffer 1 from mempool 1, its recycle path is:
-----------------------------------------------------------------------------------------
1. queue_setup: rearm from mempool 1 into Rx sw-ring
2. rte_eth_Rx_burst: used by user app (Rx)
3. rte_eth_Tx_burst: mount on Tx sw-ring
4. rte_eth_direct_rearm: free into Rx sw-ring:
or
tx_free_buffers: free into mempool 1 (no fast_free path)
-----------------------------------------------------------------------------------------
Buffer 2 from mempool 2, its recycle path is:
-----------------------------------------------------------------------------------------
1. queue_setup: rearm from mempool 2 into Rx sw-ring
2. rte_eth_Rx_burst: used by user app (Rx)
3. rte_eth_Tx_burst: mount on Tx sw-ring
4. rte_eth_direct_rearm: free into Rx sw-ring
or
tx_free_buffers: free into mempool 2 (no fast_free_path)
-----------------------------------------------------------------------------------------
Thus, buffers from Tx different mempools are the same for Rx. The difference point is that they will be freed into different mempool if the thread uses generic free buffers. I think this cannot affect direct-rearm mode, and we do not need to check this.
Quoted 17 lines
Quoted 17 lines
Quoted 17 lines
Quoted 17 lines
drivers/net/i40e/i40e_rxtx_vec_common.h +17
-0+if (m !=NULL ) { + *rxep = m; + nb_rearm++; + } + } + n = nb_rearm; + } + +/* update counters for Tx */ + txq->nb_tx_free = (uint16_t )(txq->nb_tx_free + txq->tx_rs_thresh); + txq->tx_next_dd = (uint16_t )(txq->tx_next_dd + txq->tx_rs_thresh); +if (txq->tx_next_dd >= txq->nb_tx_desc) + txq->tx_next_dd = (uint16_t )(txq->tx_rs_thresh -1 ); + +return n; +} +
RE: [PATCH v3 2/3] net/i40e: enable direct rearm with separate API
Quoted 84 lines
Quoted 51 lines
-----邮件原件-----
发件人: Konstantin Ananyev <[email protected]> 发送时间: Tuesday, February 28, 2023 3:36 AM
收件人: Feifei Wang <[email protected]>; Konstantin Ananyev <[email protected]>; Yuying Zhang <[email protected]>; Beilei Xing <[email protected]>; Ruifeng Wang <[email protected]>
抄送: [email protected]; nd <[email protected]>; Honnappa Nagarahalli <[email protected]>; nd <[email protected]>
主题: RE: [PATCH v3 2/3] net/i40e: enable direct rearm with separate APIQuoted 36 lines
Quoted 36 lines
Quoted 8 lines
drivers/net/i40e/i40e_rxtx_vec_common.h +8
-0+int +i40e_tx_fill_sw_ring (void *tx_queue, +struct rte_eth_rxq_rearm_data *rxq_rearm_data) { +struct i40e_tx_queue *txq = tx_queue; +struct i40e_tx_entry *txep; +void **rxep; +struct rte_mbuf *m; +int i, n;drivers/net/i40e/i40e_rxtx_vec_common.h i40e_tx_free_bufs(struct i40e_tx_queue *txq) · line 169
Quoted 10 lines
drivers/net/i40e/i40e_rxtx_vec_common.h +10
-0+int nb_rearm =0 ; + +if (*rxq_rearm_data->rearm_nb < txq->tx_rs_thresh || + txq->nb_tx_free > txq->tx_free_thresh) +return 0 ; + +/* check DD bits on threshold descriptor */ + if ((txq->tx_ring[txq->tx_next_dd].cmd_type_offset_bsz & + rte_cpu_to_le_64(I40E_TXD_QW1_DTYPE_MASK)) != +rte_cpu_to_le_64(I40E_TX_DESC_DTYPE_DESC_DONE))Quoted 17 lines
drivers/net/i40e/i40e_rxtx_vec_common.h +17
-0+return 0 ; + + n = txq->tx_rs_thresh; + +/* first buffer to free from S/W ring is at index +* tx_next_dd - (tx_rs_thresh-1) +*/ + txep = &txq->sw_ring[txq->tx_next_dd - (n -1 )]; + rxep = rxq_rearm_data->rx_sw_ring; + rxep += *rxq_rearm_data->rearm_start; + +if (txq->offloads &RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE ) { +/* directly put mbufs from Tx to Rx */ +for (i =0 ; i < n; i++, rxep++, txep++) + *rxep = txep[0 ].mbuf; + }else { +for (i =0 ; i < n; i++, rxep++) {Quoted 9 lines
Quoted 1 line
Quoted 1 line
drivers/net/i40e/i40e_rxtx_vec_common.h i40e_tx_free_bufs(struct i40e_tx_queue *txq) · line 185
Quoted 1 line
drivers/net/i40e/i40e_rxtx_vec_common.h +1
-0+ m =rte_pktmbuf_prefree_seg (txep[i].mbuf);One thing I forgot to ask:
What would happen if this mbuf belongs to different mempool (not one that we specify atrx_queue_setup())?
Do we need to check it here?
Or would it be upper layer constraint?
Or...?First, 'different mempool' is valid for no
FAST_FREEpath intx_free_buffers.If buffers belong to different mempool, we can have an example here:
Buffer 1 from mempool 1, its recycle path is:
-----------------------------------------------------------------------------------------
1.queue_setup: rearm from mempool 1 into Rx sw-ring
2.rte_eth_Rx_burst: used by user app (Rx)
3.rte_eth_Tx_burst: mount on Tx sw-ring
4.rte_eth_direct_rearm: free into Rx sw-ring:
or
tx_free_buffers: free into mempool 1 (nofast_freepath)
-----------------------------------------------------------------------------------------Buffer 2 from mempool 2, its recycle path is:
-----------------------------------------------------------------------------------------
1.queue_setup: rearm from mempool 2 into Rx sw-ring
2.rte_eth_Rx_burst: used by user app (Rx)
3.rte_eth_Tx_burst: mount on Tx sw-ring
4.rte_eth_direct_rearm: free into Rx sw-ring
or
tx_free_buffers: free into mempool 2 (nofast_free_path)
-----------------------------------------------------------------------------------------
Quoted 3 lines
Thus, buffers from Tx different mempools are the same for Rx. The difference point is that they will be freed into different mempool if the thread uses generic free buffers. I think this cannot affect direct-rearm mode, and we do not need to check this.
I understand that it should work even with multiple mempools.
What I am trying to say - user may not want to use mbufs from particular mempool for RX (while it is still ok to use it for TX).
Let say user can have a separate mempool with small data-buffers (less then normal MTU) to send some 'special' paclets, or even use this memppol with small buffers for zero-copy updating of packet L2/L3 headers, etc.
Or it could be some 'special' user provided mempool.
That's why I wonder should we allow only mbufs from mempool that is assigned to that RX queue.
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
drivers/net/i40e/i40e_rxtx_vec_common.h +17
-0+if (m !=NULL ) { + *rxep = m; + nb_rearm++; + } + } + n = nb_rearm; + } + +/* update counters for Tx */ + txq->nb_tx_free = (uint16_t )(txq->nb_tx_free + txq->tx_rs_thresh); + txq->tx_next_dd = (uint16_t )(txq->tx_next_dd + txq->tx_rs_thresh); +if (txq->tx_next_dd >= txq->nb_tx_desc) + txq->tx_next_dd = (uint16_t )(txq->tx_rs_thresh -1 ); + +return n; +} +
回复: [PATCH v3 2/3] net/i40e: enable direct rearm with separate API
Quoted 108 lines
-----邮件原件-----
发件人: Konstantin Ananyev <[email protected]> 发送时间: Tuesday, March 7, 2023 7:01 PM
收件人: Feifei Wang <[email protected]>; Konstantin Ananyev <[email protected]>; Yuying Zhang <[email protected]>; Beilei Xing <[email protected]>; Ruifeng Wang <[email protected]>
抄送: [email protected]; nd <[email protected]>; Honnappa Nagarahalli <[email protected]>; nd <[email protected]>; nd <[email protected]> 主题: RE: [PATCH v3 2/3] net/i40e: enable direct rearm with separate APIQuoted 23 lines
Quoted 23 lines
-----邮件原件-----
发件人: Konstantin Ananyev <[email protected]> 发送时间: Tuesday, February 28, 2023 3:36 AM
收件人: Feifei Wang <[email protected]>; Konstantin Ananyev <[email protected]>; Yuying Zhang <[email protected]>; Beilei Xing <[email protected]>; Ruifeng Wang <[email protected]>
抄送: [email protected]; nd <[email protected]>; Honnappa Nagarahalli <[email protected]>; nd <[email protected]>
主题: RE: [PATCH v3 2/3] net/i40e: enable direct rearm with separate APIQuoted 9 lines
Quoted 9 lines
Quoted 9 lines
drivers/net/i40e/i40e_rxtx_vec_common.h +9
-0+int +i40e_tx_fill_sw_ring (void *tx_queue, +struct rte_eth_rxq_rearm_data *rxq_rearm_data) { +struct i40e_tx_queue *txq = tx_queue; +struct i40e_tx_entry *txep; +void **rxep; +struct rte_mbuf *m; +int i, n; +int nb_rearm =0 ;Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
drivers/net/i40e/i40e_rxtx_vec_common.h i40e_tx_free_bufs(struct i40e_tx_queue *txq) · line 169
Quoted 8 lines
drivers/net/i40e/i40e_rxtx_vec_common.h +8
-0+ +if (*rxq_rearm_data->rearm_nb < txq->tx_rs_thresh || + txq->nb_tx_free > txq->tx_free_thresh) +return 0 ; + +/* check DD bits on threshold descriptor */ + if ((txq->tx_ring[txq->tx_next_dd].cmd_type_offset_bsz & +rte_cpu_to_le_64(I40E_TXD_QW1_DTYPE_MASK)) !=Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
Quoted 1 line
+
rte_cpu_to_le_64(I40E_TX_DESC_DTYPE_DESC_DONE))drivers/net/i40e/i40e_rxtx_vec_common.h i40e_tx_free_bufs(struct i40e_tx_queue *txq)
Quoted 12 lines
drivers/net/i40e/i40e_rxtx_vec_common.h +12
-0+return 0 ; + + n = txq->tx_rs_thresh; + +/* first buffer to free from S/W ring is at index +* tx_next_dd - (tx_rs_thresh-1) +*/ + txep = &txq->sw_ring[txq->tx_next_dd - (n -1 )]; + rxep = rxq_rearm_data->rx_sw_ring; + rxep += *rxq_rearm_data->rearm_start; + + if (txq->offloads &
RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE) {Quoted 40 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
drivers/net/i40e/i40e_rxtx_vec_common.h +5
-0+/* directly put mbufs from Tx to Rx */ +for (i =0 ; i < n; i++, rxep++, txep++) + *rxep = txep[0 ].mbuf; + }else { +for (i =0 ; i < n; i++, rxep++) {Quoted 9 lines
Quoted 1 line
Quoted 1 line
drivers/net/i40e/i40e_rxtx_vec_common.h i40e_tx_free_bufs(struct i40e_tx_queue *txq) · line 185
Quoted 1 line
drivers/net/i40e/i40e_rxtx_vec_common.h +1
-0+ m =rte_pktmbuf_prefree_seg (txep[i].mbuf);One thing I forgot to ask:
What would happen if this mbuf belongs to different mempool (not one that we specify atrx_queue_setup())?
Do we need to check it here?
Or would it be upper layer constraint?
Or...?First, 'different mempool' is valid for no
FAST_FREEpath intx_free_buffers.If buffers belong to different mempool, we can have an example here:
Buffer 1 from mempool 1, its recycle path is:
----------------------------------------------------------------------
------------------- 1.queue_setup: rearm from mempool 1 into Rx sw-ring 2.rte_eth_Rx_burst: used by user app (Rx) 3.rte_eth_Tx_burst: mount on Tx sw-ring 4.rte_eth_direct_rearm: free into Rx sw-ring:
or
tx_free_buffers: free into mempool 1 (nofast_freepath)
----------------------------------------------------------------------
-------------------Buffer 2 from mempool 2, its recycle path is:
----------------------------------------------------------------------
------------------- 1.queue_setup: rearm from mempool 2 into Rx sw-ring 2.rte_eth_Rx_burst: used by user app (Rx) 3.rte_eth_Tx_burst: mount on Tx sw-ring 4.rte_eth_direct_rearm: free into Rx sw-ring
or
tx_free_buffers: free into mempool 2 (nofast_free_path)
----------------------------------------------------------------------
-------------------Quoted 2 lines
Thus, buffers from Tx different mempools are the same for Rx. The difference point is that they will be freed into different mempool if the
thread uses generic free buffers.
Quoted 1 line
I think this cannot affect direct-rearm mode, and we do not need to check
this.
Quoted 9 lines
I understand that it should work even with multiple mempools. What I am trying to say - user may not want to use mbufs from particular mempool for RX (while it is still ok to use it for TX).
Let say user can have a separate mempool with small data-buffers (less then normal MTU) to send some 'special' paclets, or even use this memppol with small buffers for zero-copy updating of packet L2/L3 headers, etc. Or it could be some 'special' user provided mempool.
That's why I wonder should we allow only mbufs from mempool that is assigned to that RX queue.
Sorry for my misleading. If I understand correctly this time, you means a special mempool. Maybe its buffer size is very small and this Tx buffer is generated from control plane.
However, if we recycle this Tx buffer into Rx buffer ring, there maybe some error due to its size is so small.
Thus we can only allow general buffers which is valid for Rx buffer ring. Furthermore, this should be user's responsibility to ensure the Tx recycling buffers should be valid. If we check this in the data plane, it will cost a lot of CPU cycles. At last, what we can do is to add constraint in the notes to remind users.
Quoted 21 lines
Quoted 21 lines
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
drivers/net/i40e/i40e_rxtx_vec_common.h i40e_tx_free_bufs(struct i40e_tx_queue *txq)
Quoted 12 lines
drivers/net/i40e/i40e_rxtx_vec_common.h +10
-0+if (m !=NULL ) { + *rxep = m; + nb_rearm++; + } + } + n = nb_rearm; + } + +/* update counters for Tx */ + txq->nb_tx_free = (uint16_t)(txq->nb_tx_free + txq-
tx_rs_thresh);Quoted 1 line
Quoted 1 line
Quoted 1 line
Review
Quoted 1 line
(quoted diff) +1
-0+ txq->tx_next_dd = (uint16_t)(txq->tx_next_dd + txq-
tx_rs_thresh);Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
drivers/net/i40e/i40e_rxtx_vec_common.h +6
-0+if (txq->tx_next_dd >= txq->nb_tx_desc) + txq->tx_next_dd = (uint16_t )(txq->tx_rs_thresh -1 ); + +return n; +} +
Re: 回复: [PATCH v3 2/3] net/i40e: enable direct rearm with separate API
Quoted 93 lines
Quoted 80 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
(quoted diff) +9
-0+int +i40e_tx_fill_sw_ring(void *tx_queue, + struct rte_eth_rxq_rearm_data *rxq_rearm_data) { + struct i40e_tx_queue *txq = tx_queue; + struct i40e_tx_entry *txep; + void **rxep; + struct rte_mbuf *m; + int i, n; + int nb_rearm = 0;Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Review
Quoted 8 lines
(quoted diff) +8
-0+ + if (*rxq_rearm_data->rearm_nb < txq->tx_rs_thresh || + txq->nb_tx_free > txq->tx_free_thresh) + return 0; + + /* check DD bits on threshold descriptor */ + if ((txq->tx_ring[txq->tx_next_dd].cmd_type_offset_bsz & +rte_cpu_to_le_64(I40E_TXD_QW1_DTYPE_MASK)) !=Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
Quoted 1 line
+
rte_cpu_to_le_64(I40E_TX_DESC_DTYPE_DESC_DONE))Review
Quoted 12 lines
(quoted diff) +12
-0+ return 0; + + n = txq->tx_rs_thresh; + + /* first buffer to free from S/W ring is at index + * tx_next_dd - (tx_rs_thresh-1) + */ + txep = &txq->sw_ring[txq->tx_next_dd - (n - 1)]; + rxep = rxq_rearm_data->rx_sw_ring; + rxep += *rxq_rearm_data->rearm_start; + + if (txq->offloads &
RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE) {Quoted 40 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
(quoted diff) +5
-0+ /* directly put mbufs from Tx to Rx */ + for (i = 0; i < n; i++, rxep++, txep++) + *rxep = txep[0].mbuf; + } else { + for (i = 0; i < n; i++, rxep++) {Quoted 9 lines
Quoted 1 line
Quoted 1 line
Review
Quoted 1 line
(quoted diff) +1
-0+ m = rte_pktmbuf_prefree_seg(txep[i].mbuf);One thing I forgot to ask:
What would happen if this mbuf belongs to different mempool (not one that we specify atrx_queue_setup())?
Do we need to check it here?
Or would it be upper layer constraint?
Or...?First, 'different mempool' is valid for no
FAST_FREEpath intx_free_buffers.If buffers belong to different mempool, we can have an example here:
Buffer 1 from mempool 1, its recycle path is:
----------------------------------------------------------------------
------------------- 1.queue_setup: rearm from mempool 1 into Rx sw-ring 2.rte_eth_Rx_burst: used by user app (Rx) 3.rte_eth_Tx_burst: mount on Tx sw-ring 4.rte_eth_direct_rearm: free into Rx sw-ring:
or
tx_free_buffers: free into mempool 1 (nofast_freepath)
----------------------------------------------------------------------
-------------------Buffer 2 from mempool 2, its recycle path is:
----------------------------------------------------------------------
------------------- 1.queue_setup: rearm from mempool 2 into Rx sw-ring 2.rte_eth_Rx_burst: used by user app (Rx) 3.rte_eth_Tx_burst: mount on Tx sw-ring 4.rte_eth_direct_rearm: free into Rx sw-ring
or
tx_free_buffers: free into mempool 2 (nofast_free_path)
----------------------------------------------------------------------
-------------------Quoted 2 lines
Thus, buffers from Tx different mempools are the same for Rx. The difference point is that they will be freed into different mempool if the
thread uses generic free buffers.
Quoted 1 line
I think this cannot affect direct-rearm mode, and we do not need to check
this.
Quoted 9 lines
I understand that it should work even with multiple mempools. What I am trying to say - user may not want to use mbufs from particular mempool for RX (while it is still ok to use it for TX).
Let say user can have a separate mempool with small data-buffers (less then normal MTU) to send some 'special' paclets, or even use this memppol with small buffers for zero-copy updating of packet L2/L3 headers, etc. Or it could be some 'special' user provided mempool.
That's why I wonder should we allow only mbufs from mempool that is assigned to that RX queue.Sorry for my misleading. If I understand correctly this time, you means a special mempool. Maybe its buffer size is very small and this Tx buffer is generated from control plane.
Quoted 6 lines
However, if we recycle this Tx buffer into Rx buffer ring, there maybe some error due to its size is so small.
Thus we can only allow general buffers which is valid for Rx buffer ring. Furthermore, this should be user's responsibility to ensure the Tx recycling buffers should be valid. If we check this in the data plane, it will cost a lot of CPU cycles. At last, what we can do is to add constraint in the notes to remind users.
As I thought: in theory we can add 'struct rte_mempool *mp' into rte_eth_rxq_rearm_data.
And then:
if (mbuf->pool == rxq_rearm_data->mp)
/* put mbuf into rearm buffer */
else
/* free mbuf */
For the 'proper' config (when txq contains mbufs from expected mempool) the overhead will be minimal.
In other case it might be higher, but still would work and no need for extra limitations.
Quoted 21 lines
Quoted 21 lines
Quoted 21 lines
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
Review
Quoted 12 lines
(quoted diff) +10
-0+ if (m != NULL) { + *rxep = m; + nb_rearm++; + } + } + n = nb_rearm; + } + + /* update counters for Tx */ + txq->nb_tx_free = (uint16_t)(txq->nb_tx_free + txq-
tx_rs_thresh);Quoted 1 line
Quoted 1 line
Quoted 1 line
Review
Quoted 1 line
(quoted diff) +1
-0+ txq->tx_next_dd = (uint16_t)(txq->tx_next_dd + txq-
tx_rs_thresh);Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
(quoted diff) +6
-0+ if (txq->tx_next_dd >= txq->nb_tx_desc) + txq->tx_next_dd = (uint16_t)(txq->tx_rs_thresh - 1); + + return n; +} +
RE: 回复: [PATCH v3 2/3] net/i40e: enable direct rearm with separate API
Quoted 125 lines
-----Original Message-----
From: Konstantin Ananyev <[email protected]>
Sent: Monday, March 20, 2023 12:11 AM
To: Feifei Wang <[email protected]>; Konstantin Ananyev <[email protected]>; Yuying Zhang <[email protected]>; Beilei Xing <[email protected]>; Ruifeng Wang <[email protected]>
Cc: [email protected]; nd <[email protected]>; Honnappa Nagarahalli <[email protected]>
Subject: Re: 回复: [PATCH v3 2/3] net/i40e: enable direct rearm with separate APIQuoted 48 lines
Quoted 48 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
(quoted diff) +9
-0+int +i40e_tx_fill_sw_ring(void *tx_queue, + struct rte_eth_rxq_rearm_data *rxq_rearm_data) { + struct i40e_tx_queue *txq = tx_queue; + struct i40e_tx_entry *txep; + void **rxep; + struct rte_mbuf *m; + int i, n; + int nb_rearm = 0;Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Review
Quoted 8 lines
(quoted diff) +8
-0+ + if (*rxq_rearm_data->rearm_nb < txq->tx_rs_thresh || + txq->nb_tx_free > txq->tx_free_thresh) + return 0; + + /* check DD bits on threshold descriptor */ + if ((txq->tx_ring[txq->tx_next_dd].cmd_type_offset_bsz & +rte_cpu_to_le_64(I40E_TXD_QW1_DTYPE_MASK)) !=Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
Quoted 1 line
+
rte_cpu_to_le_64(I40E_TX_DESC_DTYPE_DESC_DONE))Review
Quoted 12 lines
(quoted diff) +12
-0+ return 0; + + n = txq->tx_rs_thresh; + + /* first buffer to free from S/W ring is at index + * tx_next_dd - (tx_rs_thresh-1) + */ + txep = &txq->sw_ring[txq->tx_next_dd - (n - 1)]; + rxep = rxq_rearm_data->rx_sw_ring; + rxep += *rxq_rearm_data->rearm_start; + + if (txq->offloads &
RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE) {Quoted 15 lines
Quoted 14 lines
Quoted 6 lines
Quoted 6 lines
Review
Quoted 6 lines
(quoted diff) +6
-0+ /* directly put mbufs from Tx to Rx */ + for (i = 0; i < n; i++, rxep++, txep++) + *rxep = txep[0].mbuf; + } else { + for (i = 0; i < n; i++, rxep++) { + m = rte_pktmbuf_prefree_seg(txep[i].mbuf);One thing I forgot to ask:
What would happen if this mbuf belongs to different mempool (not one that we specify atrx_queue_setup())?
Do we need to check it here?
Or would it be upper layer constraint?
Or...?Quoted 1 line
Quoted 1 line
Quoted 1 line
First, 'different mempool' is valid for no
FAST_FREEpath in
tx_free_buffers.Quoted 34 lines
Quoted 34 lines
Quoted 28 lines
If buffers belong to different mempool, we can have an example here:
Buffer 1 from mempool 1, its recycle path is:
--------------------------------------------------------------------
--
------------------- 1.queue_setup: rearm from mempool 1 into Rx sw-ring 2.rte_eth_Rx_burst: used by user app (Rx) 3.rte_eth_Tx_burst: mount on Tx sw-ring 4.rte_eth_direct_rearm: free into Rx sw-ring:
or
tx_free_buffers: free into mempool 1 (nofast_freepath)
--------------------------------------------------------------------
--
-------------------Buffer 2 from mempool 2, its recycle path is:
--------------------------------------------------------------------
--
------------------- 1.queue_setup: rearm from mempool 2 into Rx sw-ring 2.rte_eth_Rx_burst: used by user app (Rx) 3.rte_eth_Tx_burst: mount on Tx sw-ring 4.rte_eth_direct_rearm: free into Rx sw-ring
or
tx_free_buffers: free into mempool 2 (nofast_free_path)
--------------------------------------------------------------------
--
-------------------Quoted 3 lines
Thus, buffers from Tx different mempools are the same for Rx. The difference point is that they will be freed into different mempool if the
thread uses generic free buffers.
Quoted 2 lines
I think this cannot affect direct-rearm mode, and we do not need to check
Quoted 8 lines
Quoted 8 lines
this.
I understand that it should work even with multiple mempools. What I am trying to say - user may not want to use mbufs from particular mempool for RX (while it is still ok to use it for TX). Let say user can have a separate mempool with small data-buffers (less then normal MTU) to send some 'special' paclets, or even use this memppol with small buffers for zero-copy updating of packet L2/L3
headers, etc.
Quoted 6 lines
Quoted 3 lines
Or it could be some 'special' user provided mempool. That's why I wonder should we allow only mbufs from mempool that is assigned to that RX queue.
Sorry for my misleading. If I understand correctly this time, you means a special mempool. Maybe its buffer size is very small and this Tx
buffer is generated from control plane.
Quoted 7 lines
However, if we recycle this Tx buffer into Rx buffer ring, there maybe some error due to its size is so small.
Thus we can only allow general buffers which is valid for Rx buffer ring. Furthermore, this should be user's responsibility to ensure the Tx recycling buffers should be valid. If we check this in the data plane, it will
cost a lot of CPU cycles. At last, what we can do is to add constraint in the notes to remind users.
Quoted 11 lines
As I thought: in theory we can add 'struct
rte_mempool*mp'
intorte_eth_rxq_rearm_data.
And then:
if (mbuf->pool==rxq_rearm_data->mp)
/* put mbuf into rearm buffer */
else
/* free mbuf */
For the 'proper' config (when txq contains mbufs from expected mempool) the overhead will be minimal.
In other case it might be higher, but still would work and no need for extra limitations.
It's a good idea. And try to test performance with this change, there is currently no performance degradation. Thus, I add this check in the latest version.
Quoted 23 lines
Quoted 23 lines
Quoted 23 lines
Quoted 23 lines
Quoted 4 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Review
Quoted 10 lines
(quoted diff) +10
-0+ if (m != NULL) { + *rxep = m; + nb_rearm++; + } + } + n = nb_rearm; + } + + /* update counters for Tx */ + txq->nb_tx_free = (uint16_t)(txq->nb_tx_free + txq-
tx_rs_thresh);Quoted 1 line
Quoted 1 line
Quoted 1 line
Review
Quoted 1 line
(quoted diff) +1
-0+ txq->tx_next_dd = (uint16_t)(txq->tx_next_dd + txq-
tx_rs_thresh);Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
(quoted diff) +6
-0+ if (txq->tx_next_dd >= txq->nb_tx_desc) + txq->tx_next_dd = (uint16_t)(txq->tx_rs_thresh - 1); + + return n; +} +
PATCH v3 3/3 net/ixgbe: enable direct rearm with separate API
In reply to Feifei WangAdd internal API to separate direct rearm operations between Rx and Tx.
--- drivers/net/ixgbe/ixgbe_ethdev.c | 1 + drivers/net/ixgbe/ixgbe_ethdev.h | 3 ++ drivers/net/ixgbe/ixgbe_rxtx.c | 19 +++++++++ drivers/net/ixgbe/ixgbe_rxtx.h | 4 ++ drivers/net/ixgbe/ixgbe_rxtx_vec_common.h | 48 +++++++++++++++++++++ drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c | 52 +++++++++++++++++++++++ 6 files changed, 127 insertions(+)
drivers/net/ixgbe/ixgbe_ethdev.c +1 -0
@@ -542,6 +542,7 @@ static const struct eth_dev_ops ixgbe_eth_dev_ops = { .set_mc_addr_list = ixgbe_dev_set_mc_addr_list, .rxq_info_get = ixgbe_rxq_info_get, .txq_info_get = ixgbe_txq_info_get, + .rxq_rearm_data_get = ixgbe_rxq_rearm_data_get, .timesync_enable = ixgbe_timesync_enable, .timesync_disable = ixgbe_timesync_disable, .timesync_read_rx_timestamp = ixgbe_timesync_read_rx_timestamp,
drivers/net/ixgbe/ixgbe_ethdev.h +3 -0
@@ -625,6 +625,9 @@ void ixgbe_rxq_info_get(struct rte_eth_dev *dev, uint16_t queue_id,void ixgbe_txq_info_get (struct rte_eth_dev *dev,uint16_t queue_id,struct rte_eth_txq_info *qinfo); +void ixgbe_rxq_rearm_data_get (struct rte_eth_dev *dev,uint16_t queue_id, +struct rte_eth_rxq_rearm_data *rxq_rearm_data); +int ixgbevf_dev_rx_init (struct rte_eth_dev *dev);void ixgbevf_dev_tx_init (struct rte_eth_dev *dev);
drivers/net/ixgbe/ixgbe_rxtx.c +19 -0
@@ -2559,6 +2559,9 @@ ixgbe_set_tx_function(struct rte_eth_dev *dev, struct ixgbe_tx_queue *txq) ixgbe_txq_vec_setup(txq) == 0)) {PMD_INIT_LOG (DEBUG ,"Vector tx enabled." ); dev->tx_pkt_burst = ixgbe_xmit_pkts_vec; +#ifdef RTE_ARCH_ARM64 + dev->tx_fill_sw_ring = ixgbe_tx_fill_sw_ring; +#endif } else dev->tx_pkt_burst = ixgbe_xmit_pkts_simple; }else {
@@ -4853,6 +4856,9 @@ ixgbe_set_rx_function(struct rte_eth_dev *dev) dev->data->port_id); dev->rx_pkt_burst = ixgbe_recv_pkts_vec; +#ifdef RTE_ARCH_ARM64 + dev->rx_flush_descriptor = ixgbe_rx_flush_descriptor_vec; +#endif }else if (adapter->rx_bulk_alloc_allowed) {PMD_INIT_LOG (DEBUG ,"Rx Burst Bulk Alloc Preconditions are " "satisfied. Rx Burst Bulk Alloc function "
@@ -5623,6 +5629,19 @@ ixgbe_txq_info_get(struct rte_eth_dev *dev, uint16_t queue_id, qinfo->conf.tx_deferred_start = txq->tx_deferred_start; } +void +ixgbe_rxq_rearm_data_get (struct rte_eth_dev *dev,uint16_t queue_id, +struct rte_eth_rxq_rearm_data *rxq_rearm_data) +{ +struct ixgbe_rx_queue *rxq; + + rxq = dev->data->rx_queues[queue_id]; + + rxq_rearm_data->rx_sw_ring = rxq->sw_ring; + rxq_rearm_data->rearm_start = &rxq->rxrearm_start; + rxq_rearm_data->rearm_nb = &rxq->rxrearm_nb; +} +/* * [VF] Initializes Receive Unit. */
drivers/net/ixgbe/ixgbe_rxtx.h +4 -0
@@ -295,6 +295,10 @@ int ixgbe_dev_tx_done_cleanup(void *tx_queue, uint32_t free_cnt);extern const uint32_t ptype_table[IXGBE_PACKET_TYPE_MAX ];extern const uint32_t ptype_table_tn[IXGBE_PACKET_TYPE_TN_MAX ]; +int ixgbe_tx_fill_sw_ring (void *tx_queue, +struct rte_eth_rxq_rearm_data *rxq_rearm_data); +int ixgbe_rx_flush_descriptor_vec (void *rx_queue,uint16_t nb_rearm); +uint16_t ixgbe_xmit_fixed_burst_vec (void *tx_queue,struct rte_mbuf **tx_pkts,uint16_t nb_pkts);int ixgbe_txq_vec_setup (struct ixgbe_tx_queue *txq);
drivers/net/ixgbe/ixgbe_rxtx_vec_common.h +48 -0
54 lines not on this page · raw message
drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c +52 -0
58 lines not on this page · raw message
Signature · 1 line
2.25.1
+ping konstantin,
Would you please give some comments for this patch series? Thanks very much.
Best Regards
Feifei
Hi Feifei,
Quoted 4 lines
+ping konstantin,
Would you please give some comments for this patch series? Thanks very much.
Sure, will have a look in next few days.
Apologies for the delay.
That's all right. Thanks very much for your attention~
Quoted 16 lines
-----邮件原件-----
发件人: Konstantin Ananyev <[email protected]> 发送时间: Wednesday, February 1, 2023 9:11 AM
收件人: Feifei Wang <[email protected]>
抄送: [email protected]; nd <[email protected]>
主题: Re: 回复: [PATCH v3 0/3] Direct re-arming of buffers on receive sideHi Feifei,
Quoted 4 lines
+ping konstantin,
Would you please give some comments for this patch series? Thanks very much.
Sure, will have a look in next few days.
Apologies for the delay.
Quoted 10 lines
From: Feifei Wang [mailto:[email protected]]
Sent: Wednesday, 4 January 2023 08.31Currently, the transmit side frees the buffers into the lcore cache and the receive side allocates buffers from the lcore cache. The transmit side typically frees 32 buffers resulting in 32*8=256B of stores to lcore cache. The receive side allocates 32 buffers and stores them in the receive side software ring, resulting in 32*8=256B of stores and 256B of load from the lcore cache.
Quoted 5 lines
This patch proposes a mechanism to avoid freeing to/allocating from the lcore cache. i.e. the receive side will free the buffers from transmit side directly into its software ring. This will avoid the 256B of loads and stores introduced by the lcore cache. It also frees up the cache lines used by the lcore cache.
I am starting to wonder if we have been adding unnecessary feature creep in order to make this feature too generic.
Could you please describe some of the most important high-volume use cases from real life? It would help setting the scope correctly.
Quoted 146 lines
However, this solution poses several constraints:
1)The receive queue needs to know which transmit queue it should take the buffers from. The application logic decides which transmit port to use to send out the packets. In many use cases the NIC might have a single port ([1], [2], [3]), in which case a given transmit queue is always mapped to a single receive queue (1:1 Rx queue: Tx queue). This is easy to configure.
If the NIC has 2 ports (there are several references), then we will have 1:2 (RX queue: TX queue) mapping which is still easy to configure. However, if this is generalized to 'N' ports, the configuration can be long. More over the PMD would have to scan a list of transmit queues to pull the buffers from.
2)The other factor that needs to be considered is 'run-to-completion' vs 'pipeline' models. In the run-to-completion model, the receive side and the transmit side are running on the same lcore serially. In the pipeline model. The receive side and transmit side might be running on different lcores in parallel. This requires locking. This is not supported at this point.
3)Tx and Rx buffers must be from the same mempool. And we also must ensure Tx buffer free number is equal to Rx buffer free number. Thus, '
tx_next_dd' can be updated correctly in direct-rearm mode. This is due totx_next_ddis a variable to compute tx sw-ring free location. Its value will be one more round than the position where next time free starts.Current status in this patch:
1)Two APIs are added for users to enable direct-rearm mode:
In control plane, users can call 'rte_eth_rx_queue_rearm_data_get' to get Rxsw_ringpointer and itsrxq_info.
(This avoid Tx load Rx data directly);In data plane, users can call '
rte_eth_dev_direct_rearm' to rearm Rx buffers and free Tx buffers at the same time. Specifically, in this API, there are two separated API for Rx and Tx.
For Tx, 'rte_eth_tx_fill_sw_ring' can fill a givensw_ringby Tx freed
buffers.
For Rx, 'rte_eth_rx_flush_descriptor' can flush its descriptors based on the rearm buffers.
Thus, this can separate Rx and Tx operation, and user can even re-arm RX queue not from the same driver's TX queue, but from different sources too.
-----------------------------------------------------------------------
control plane:
rte_eth_rx_queue_rearm_data_get(*rxq_rearm_data);
data plane:
loop {
rte_eth_dev_direct_rearm(*rxq_rearm_data){rte_eth_tx_fill_sw_ring{ for (i = 0; i <= 32; i++) { sw_ring.mbuf[i] = tx.mbuf[i]; } } rte_eth_rx_flush_descriptor{ for (i = 0; i <= 32; i++) { flush descs[i]; } } } rte_eth_rx_burst; rte_eth_tx_burst;}
-----------------------------------------------------------------------
2)The i40e driver is changed to do the direct re-arm of the receive
side.
3)The ixgbe driver is changed to do the direct re-arm of the receive
side.Testing status:
(1) dpdk l3fwd test with multiple drivers:port 0: 82599 NIC port 1: XL710 NIC-------------------------------------------------------------
Without fast free With fast free
Thunderx2: +9.44% +7.14%
-------------------------------------------------------------(2) dpdk l3fwd test with same driver:
port 0 && 1: XL710 NIC-------------------------------------------------------------
*Direct rearm with exposingrx_sw_ring:
Without fast free With fast free
Ampere altra: +14.98% +15.77%
n1sdp: +6.47% +0.52%
-------------------------------------------------------------(3) VPP test with same driver:
port 0 && 1: XL710 NIC-------------------------------------------------------------
*Direct rearm with exposingrx_sw_ring:
Ampere altra: +4.59%
n1sdp: +5.4%
-------------------------------------------------------------Reference:
[1] https://store.nvidia.com/en-us/networking/store/product/MCX623105AN-
CDAT/NVIDIAMCX623105ANCDATConnectX6DxENAdapterCard100GbECryptoDisabled/
[2] https://www.intel.com/content/www/us/en/products/sku/192561/intel-
ethernet-network-adapter-e810cqda1/specifications.html
[3] https://www.broadcom.com/products/ethernet-connectivity/network-
adapters/100gb-nic-ocp/n1100gV2:
1. Use data-plane API to enable direct-rearm (Konstantin, Honnappa)
2. Add 'txq_data_get' API to get txq info for Rx (Konstantin)
3. Use input parameter to enable direct rearm in l3fwd (Konstantin)
4. Add condition detection for direct rearm API (Morten, Andrew Rybchenko)V3:
1. Seperate Rx and Tx operation with two APIs in direct-rearm (Konstantin)
2. Delete L3fwd change for direct rearm (Jerin)
3. enable direct rearm in ixgbe driver in ArmFeifei Wang (3):
ethdev: enable direct rearm with separate API net/i40e: enable direct rearm with separate API net/ixgbe: enable direct rearm with separate APIdrivers/net/i40e/i40e_ethdev.c | 1 + drivers/net/i40e/i40e_ethdev.h | 2 + drivers/net/i40e/i40e_rxtx.c | 19 +++ drivers/net/i40e/i40e_rxtx.h | 4 + drivers/net/i40e/i40e_rxtx_vec_common.h | 54 +++++++ drivers/net/i40e/i40e_rxtx_vec_neon.c | 42 ++++++ drivers/net/ixgbe/ixgbe_ethdev.c | 1 + drivers/net/ixgbe/ixgbe_ethdev.h | 3 + drivers/net/ixgbe/ixgbe_rxtx.c | 19 +++ drivers/net/ixgbe/ixgbe_rxtx.h | 4 + drivers/net/ixgbe/ixgbe_rxtx_vec_common.h | 48 ++++++ drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c | 52 +++++++ lib/ethdev/ethdev_driver.h | 10 ++ lib/ethdev/ethdev_private.c | 2 + lib/ethdev/rte_ethdev.c | 52 +++++++ lib/ethdev/rte_ethdev.h | 174 ++++++++++++++++++++++ lib/ethdev/rte_ethdev_core.h | 11 ++ lib/ethdev/version.map | 6 + 18 files changed, 504 insertions(+)--
2.25.1
Quoted 22 lines
-----Original Message-----
From: Morten Brørup <[email protected]>
Sent: Wednesday, March 22, 2023 7:57 AM
To: Feifei Wang <[email protected]>
Cc: [email protected]; [email protected]; nd <[email protected]>; [email protected]; Yuying Zhang <[email protected]>; Beilei Xing <[email protected]>; Ruifeng Wang <[email protected]>; Honnappa Nagarahalli <[email protected]>
Subject: RE: [PATCH v3 0/3] Direct re-arming of buffers on receive sideQuoted 10 lines
From: Feifei Wang [mailto:[email protected]]
Sent: Wednesday, 4 January 2023 08.31Currently, the transmit side frees the buffers into the lcore cache and the receive side allocates buffers from the lcore cache. The transmit side typically frees 32 buffers resulting in 32*8=256B of stores to lcore cache. The receive side allocates 32 buffers and stores them in the receive side software ring, resulting in 32*8=256B of stores and 256B of load from the lcore cache.
Quoted 8 lines
Quoted 5 lines
This patch proposes a mechanism to avoid freeing to/allocating from the lcore cache. i.e. the receive side will free the buffers from transmit side directly into its software ring. This will avoid the 256B of loads and stores introduced by the lcore cache. It also frees up the cache lines used by the lcore cache.
I am starting to wonder if we have been adding unnecessary feature creep in order to make this feature too generic.
Can you please elaborate on the feature creep you are thinking? The features have been the same since the first implementation, but it is made more generic.
Quoted 3 lines
Could you please describe some of the most important high-volume use cases from real life? It would help setting the scope correctly.
The use cases have been discussed several times already.
Quoted 152 lines
Quoted 101 lines
However, this solution poses several constraints:
1)The receive queue needs to know which transmit queue it should take the buffers from. The application logic decides which transmit port to use to send out the packets. In many use cases the NIC might have a single port ([1], [2], [3]), in which case a given transmit queue is always mapped to a single receive queue (1:1 Rx queue: Tx queue). This is easy to configure.
If the NIC has 2 ports (there are several references), then we will have
1:2 (RX queue: TX queue) mapping which is still easy to configure. However, if this is generalized to 'N' ports, the configuration can be long. More over the PMD would have to scan a list of transmit queues to pull the buffers from.2)The other factor that needs to be considered is 'run-to-completion' vs 'pipeline' models. In the run-to-completion model, the receive side and the transmit side are running on the same lcore serially. In the pipeline model. The receive side and transmit side might be running on different lcores in parallel. This requires locking. This is not supported at this point.
3)Tx and Rx buffers must be from the same mempool. And we also must ensure Tx buffer free number is equal to Rx buffer free number. Thus, '
tx_next_dd' can be updated correctly in direct-rearm mode. This is due totx_next_ddis a variable to compute tx sw-ring free location. Its value will be one more round than the position where next time free starts.Current status in this patch:
1)Two APIs are added for users to enable direct-rearm mode:
In control plane, users can call 'rte_eth_rx_queue_rearm_data_get' to get Rxsw_ringpointer and itsrxq_info.
(This avoid Tx load Rx data directly);In data plane, users can call '
rte_eth_dev_direct_rearm' to rearm Rx buffers and free Tx buffers at the same time. Specifically, in this API, there are two separated API for Rx and Tx.
For Tx, 'rte_eth_tx_fill_sw_ring' can fill a givensw_ringby Tx
freed buffers.
For Rx, 'rte_eth_rx_flush_descriptor' can flush its descriptors based on the rearm buffers.
Thus, this can separate Rx and Tx operation, and user can even re-arm RX queue not from the same driver's TX queue, but from different sources too.
-----------------------------------------------------------------------
control plane:
rte_eth_rx_queue_rearm_data_get(*rxq_rearm_data);
data plane:
loop {
rte_eth_dev_direct_rearm(*rxq_rearm_data){rte_eth_tx_fill_sw_ring{ for (i = 0; i <= 32; i++) { sw_ring.mbuf[i] = tx.mbuf[i]; } } rte_eth_rx_flush_descriptor{ for (i = 0; i <= 32; i++) { flush descs[i]; } } } rte_eth_rx_burst; rte_eth_tx_burst;}
----------------------------------------------------------------------
- 2)The i40e driver is changed to do the direct re-arm of the receive side.
3)The ixgbe driver is changed to do the direct re-arm of the receive
side.Testing status:
(1) dpdk l3fwd test with multiple drivers:port 0: 82599 NIC port 1: XL710 NIC-------------------------------------------------------------
Without fast free With fast free
Thunderx2: +9.44% +7.14%
-------------------------------------------------------------(2) dpdk l3fwd test with same driver:
port 0 && 1: XL710 NIC-------------------------------------------------------------
*Direct rearm with exposingrx_sw_ring:
Without fast free With fast free
Ampere altra: +14.98% +15.77%
n1sdp: +6.47% +0.52%
-------------------------------------------------------------(3) VPP test with same driver:
port 0 && 1: XL710 NIC-------------------------------------------------------------
*Direct rearm with exposingrx_sw_ring:
Ampere altra: +4.59%
n1sdp: +5.4%
-------------------------------------------------------------Quoted 4 lines
Reference:
[1]
https://store.nvidia.com/en-us/networking/store/product/MCX623105AN-CDAT/NVIDIAMCX623105ANCDATConnectX6DxENAdapterCard100GbECrypto Disabled
Quoted 45 lines
/ [2]
https://www.intel.com/content/www/us/en/products/sku/192561/intel-
ethernet-network-adapter-e810cqda1/specifications.html
[3] https://www.broadcom.com/products/ethernet-connectivity/network-
adapters/100gb-nic-ocp/n1100gV2:
1. Use data-plane API to enable direct-rearm (Konstantin, Honnappa) 2. Add 'txq_data_get' API to get txq info for Rx (Konstantin) 3. Use input parameter to enable direct rearm in l3fwd (Konstantin) 4. Add condition detection for direct rearm API (Morten, Andrew Rybchenko)V3:
1. Seperate Rx and Tx operation with two APIs in direct-rearm (Konstantin) 2. Delete L3fwd change for direct rearm (Jerin) 3. enable direct rearm in ixgbe driver in ArmFeifei Wang (3):
ethdev: enable direct rearm with separate API net/i40e: enable direct rearm with separate API net/ixgbe: enable direct rearm with separate APIdrivers/net/i40e/i40e_ethdev.c | 1 + drivers/net/i40e/i40e_ethdev.h | 2 + drivers/net/i40e/i40e_rxtx.c | 19 +++ drivers/net/i40e/i40e_rxtx.h | 4 + drivers/net/i40e/i40e_rxtx_vec_common.h | 54 +++++++ drivers/net/i40e/i40e_rxtx_vec_neon.c | 42 ++++++ drivers/net/ixgbe/ixgbe_ethdev.c | 1 + drivers/net/ixgbe/ixgbe_ethdev.h | 3 + drivers/net/ixgbe/ixgbe_rxtx.c | 19 +++ drivers/net/ixgbe/ixgbe_rxtx.h | 4 + drivers/net/ixgbe/ixgbe_rxtx_vec_common.h | 48 ++++++ drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c | 52 +++++++ lib/ethdev/ethdev_driver.h | 10 ++ lib/ethdev/ethdev_private.c | 2 + lib/ethdev/rte_ethdev.c | 52 +++++++ lib/ethdev/rte_ethdev.h | 174 ++++++++++++++++++++++ lib/ethdev/rte_ethdev_core.h | 11 ++ lib/ethdev/version.map | 6 + 18 files changed, 504 insertions(+)--
2.25.1
Quoted 22 lines
From: Honnappa Nagarahalli [mailto:[email protected]]
Sent: Wednesday, 22 March 2023 14.42Quoted 19 lines
From: Morten Brørup <[email protected]>
Sent: Wednesday, March 22, 2023 7:57 AMQuoted 15 lines
From: Feifei Wang [mailto:[email protected]]
Sent: Wednesday, 4 January 2023 08.31Currently, the transmit side frees the buffers into the lcore cache and the receive side allocates buffers from the lcore cache. The transmit side typically frees 32 buffers resulting in 32*8=256B of stores to lcore cache. The receive side allocates 32 buffers and stores them in the receive side software ring, resulting in 32*8=256B of stores and 256B of load from the lcore cache.
This patch proposes a mechanism to avoid freeing to/allocating from the lcore cache. i.e. the receive side will free the buffers from transmit side directly into its software ring. This will avoid the 256B of loads and stores introduced by the lcore cache. It also frees up the cache lines used by the lcore cache.
Quoted 5 lines
Quoted 2 lines
I am starting to wonder if we have been adding unnecessary feature creep in order to make this feature too generic.
Can you please elaborate on the feature creep you are thinking? The features have been the same since the first implementation, but it is made more generic.
Maybe not "features" as such; but the API has evolved, and perhaps we could simplify both the API and the implementation if we narrowed the scope.
I'm not saying that what we have is bad or too complex; I'm only asking to consider if there are opportunities to take a step back and simplify some things.
Quoted 5 lines
Quoted 4 lines
Could you please describe some of the most important high-volume use cases from real life? It would help setting the scope correctly.
The use cases have been discussed several times already.
Yes, but they should be mentioned in the patch cover letter - and later on in the documentation.
It will help limiting the scope while developing this feature. And it will make it easier for application developers to relate to the feature and determine if it is relevant for their application.
Quoted 154 lines
Quoted 153 lines
Quoted 102 lines
However, this solution poses several constraints:
1)The receive queue needs to know which transmit queue it should take the buffers from. The application logic decides which transmit port to use to send out the packets. In many use cases the NIC might have a single port ([1], [2], [3]), in which case a given transmit queue is always mapped to a single receive queue (1:1 Rx queue: Tx queue). This is easy to configure.
If the NIC has 2 ports (there are several references), then we will have
1:2 (RX queue: TX queue) mapping which is still easy to configure. However, if this is generalized to 'N' ports, the configuration can be long. More over the PMD would have to scan a list of transmit queues to pull the buffers from.2)The other factor that needs to be considered is 'run-to-completion' vs 'pipeline' models. In the run-to-completion model, the receive side and the transmit side are running on the same lcore serially. In the pipeline model. The receive side and transmit side might be running on different lcores in parallel. This requires locking. This is not supported at this point.
3)Tx and Rx buffers must be from the same mempool. And we also must ensure Tx buffer free number is equal to Rx buffer free number. Thus, '
tx_next_dd' can be updated correctly in direct-rearm mode. This is due totx_next_ddis a variable to compute tx sw-ring free location. Its value will be one more round than the position where next time free starts.Current status in this patch:
1)Two APIs are added for users to enable direct-rearm mode:
In control plane, users can call 'rte_eth_rx_queue_rearm_data_get' to get Rxsw_ringpointer and itsrxq_info.
(This avoid Tx load Rx data directly);In data plane, users can call '
rte_eth_dev_direct_rearm' to rearm Rx buffers and free Tx buffers at the same time. Specifically, in this API, there are two separated API for Rx and Tx.
For Tx, 'rte_eth_tx_fill_sw_ring' can fill a givensw_ringby Tx
freed buffers.
For Rx, 'rte_eth_rx_flush_descriptor' can flush its descriptors based on the rearm buffers.
Thus, this can separate Rx and Tx operation, and user can even re-arm RX queue not from the same driver's TX queue, but from different sources too.
-----------------------------------------------------------------------
control plane:
rte_eth_rx_queue_rearm_data_get(*rxq_rearm_data);
data plane:
loop {
rte_eth_dev_direct_rearm(*rxq_rearm_data){rte_eth_tx_fill_sw_ring{ for (i = 0; i <= 32; i++) { sw_ring.mbuf[i] = tx.mbuf[i]; } } rte_eth_rx_flush_descriptor{ for (i = 0; i <= 32; i++) { flush descs[i]; } } } rte_eth_rx_burst; rte_eth_tx_burst;}
----------------------------------------------------------------------
- 2)The i40e driver is changed to do the direct re-arm of the receive side.
3)The ixgbe driver is changed to do the direct re-arm of the receive
side.Testing status:
(1) dpdk l3fwd test with multiple drivers:port 0: 82599 NIC port 1: XL710 NIC-------------------------------------------------------------
Without fast free With fast free
Thunderx2: +9.44% +7.14%
-------------------------------------------------------------(2) dpdk l3fwd test with same driver:
port 0 && 1: XL710 NIC-------------------------------------------------------------
*Direct rearm with exposingrx_sw_ring:
Without fast free With fast free
Ampere altra: +14.98% +15.77%
n1sdp: +6.47% +0.52%
-------------------------------------------------------------(3) VPP test with same driver:
port 0 && 1: XL710 NIC-------------------------------------------------------------
*Direct rearm with exposingrx_sw_ring:
Ampere altra: +4.59%
n1sdp: +5.4%
-------------------------------------------------------------Quoted 4 lines
Reference:
[1]
https://store.nvidia.com/en-us/networking/store/product/MCX623105AN-CDAT/NVIDIAMCX623105ANCDATConnectX6DxENAdapterCard100GbECrypto Disabled
Quoted 45 lines
/ [2]
https://www.intel.com/content/www/us/en/products/sku/192561/intel-
ethernet-network-adapter-e810cqda1/specifications.html
[3] https://www.broadcom.com/products/ethernet-connectivity/network-
adapters/100gb-nic-ocp/n1100gV2:
1. Use data-plane API to enable direct-rearm (Konstantin, Honnappa) 2. Add 'txq_data_get' API to get txq info for Rx (Konstantin) 3. Use input parameter to enable direct rearm in l3fwd (Konstantin) 4. Add condition detection for direct rearm API (Morten, Andrew Rybchenko)V3:
1. Seperate Rx and Tx operation with two APIs in direct-rearm (Konstantin) 2. Delete L3fwd change for direct rearm (Jerin) 3. enable direct rearm in ixgbe driver in ArmFeifei Wang (3):
ethdev: enable direct rearm with separate API net/i40e: enable direct rearm with separate API net/ixgbe: enable direct rearm with separate APIdrivers/net/i40e/i40e_ethdev.c | 1 + drivers/net/i40e/i40e_ethdev.h | 2 + drivers/net/i40e/i40e_rxtx.c | 19 +++ drivers/net/i40e/i40e_rxtx.h | 4 + drivers/net/i40e/i40e_rxtx_vec_common.h | 54 +++++++ drivers/net/i40e/i40e_rxtx_vec_neon.c | 42 ++++++ drivers/net/ixgbe/ixgbe_ethdev.c | 1 + drivers/net/ixgbe/ixgbe_ethdev.h | 3 + drivers/net/ixgbe/ixgbe_rxtx.c | 19 +++ drivers/net/ixgbe/ixgbe_rxtx.h | 4 + drivers/net/ixgbe/ixgbe_rxtx_vec_common.h | 48 ++++++ drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c | 52 +++++++ lib/ethdev/ethdev_driver.h | 10 ++ lib/ethdev/ethdev_private.c | 2 + lib/ethdev/rte_ethdev.c | 52 +++++++ lib/ethdev/rte_ethdev.h | 174 ++++++++++++++++++++++ lib/ethdev/rte_ethdev_core.h | 11 ++ lib/ethdev/version.map | 6 + 18 files changed, 504 insertions(+)--
2.25.1
v8 cover Recycle mbufs from Tx queue into Rx queue
In reply to Feifei WangCurrently, the transmit side frees the buffers into the lcore cache and the receive side allocates buffers from the lcore cache. The transmit side typically frees 32 buffers resulting in 32*8=256B of stores to lcore cache. The receive side allocates 32 buffers and stores them in the receive side software ring, resulting in 32*8=256B of stores and 256B of load from the lcore cache.
This patch proposes a mechanism to avoid freeing to/allocating from the lcore cache. i.e. the receive side will free the buffers from transmit side directly into its software ring. This will avoid the 256B of loads and stores introduced by the lcore cache. It also frees up the cache lines used by the lcore cache. And we can call this mode as mbufs recycle mode.
In the latest version, mbufs recycle mode is packaged as a separate API.
This allows for the users to change rxq/txq pairing in real time in data plane, according to the analysis of the packet flow by the application, for example:
-----------------------------------------------------------------------
Step 1: upper application analyse the flow direction
Step 2: recycle_rxq_info = rte_eth_recycle_rx_queue_info_get(rx_portid, rx_queueid) Step 3: rte_eth_recycle_mbufs(rx_portid, rx_queueid, tx_portid, tx_queueid, recycle_rxq_info); Step 4: rte_eth_rx_burst(rx_portid,rx_queueid);
Step 5: rte_eth_tx_burst(tx_portid,tx_queueid);
-----------------------------------------------------------------------
Above can support user to change rxq/txq pairing at run-time and user does not need to know the direction of flow in advance. This can effectively expand mbufs recycle mode's use scenarios.
Furthermore, mbufs recycle mode is no longer limited to the same pmd,
it can support moving mbufs between different vendor pmds, even can put the mbufs anywhere into your Rx mbuf ring as long as the address of the mbuf ring can be provided. In the latest version, we enable mbufs recycle mode in i40e pmd and ixgbe pmd, and also try to use i40e driver in Rx, ixgbe driver in Tx, and then achieve 7-9% performance improvement by mbufs recycle mode.
Difference between mbuf recycle, ZC API used in mempool and general path
For general path:
Rx: 32 pkts memcpy from mempool cache to rx_sw_ring
Tx: 32 pkts memcpy from tx_sw_ring to temporary variable + 32 pkts memcpy from temporary variable to mempool cache
For ZC API used in mempool:
Rx: 32 pkts memcpy from mempool cache to rx_sw_ring
Tx: 32 pkts memcpy from tx_sw_ring to zero-copy mempool cache
Refer link: http://patches.dpdk.org/project/dpdk/patch/[email protected]/
For mbufs recycle:
Rx/Tx: 32 pkts memcpy from tx_sw_ring to rx_sw_ring
Thus we can see in the one loop, compared to general path, mbufs recycle mode reduces 32+32=64 pkts memcpy; Compared to ZC API used in mempool, we can see mbufs recycle mode reduce 32 pkts memcpy in each loop.
So, mbufs recycle has its own benefits.
Testing status:
(1) dpdk l3fwd test with multiple drivers:
port 0: 82599 NIC port 1: XL710 NIC
-------------------------------------------------------------
Without fast free With fast free
Thunderx2: +7.53% +13.54%
-------------------------------------------------------------
(2) dpdk l3fwd test with same driver:
port 0 && 1: XL710 NIC
-------------------------------------------------------------
Without fast free With fast free
Ampere altra: +12.61% +11.42%
n1sdp: +8.30% +3.85%
x86-sse: +8.43% +3.72%
-------------------------------------------------------------
(3) Performance comparison with ZC_mempool used
port 0 && 1: XL710 NIC
with fast free
-------------------------------------------------------------
With recycle buffer With zc_mempool
Ampere altra: 11.42% 3.54%
-------------------------------------------------------------
Furthermore, we add recycle_mbuf engine in testpmd. Due to XL710 NIC has I/O bottleneck in testpmd in ampere altra, we can not see throughput change compared with I/O fwd engine. However, using record cmd in testpmd:
'$set record-burst-stats on'
we can see the ratio of 'Rx/Tx burst size of 32' is reduced. This indicate mbufs recycle can save CPU cycles.
V2:
1. Use data-plane API to enable direct-rearm (Konstantin, Honnappa)
2. Add 'txq_data_get' API to get txq info for Rx (Konstantin)
3. Use input parameter to enable direct rearm in l3fwd (Konstantin)
4. Add condition detection for direct rearm API (Morten, Andrew Rybchenko)
V3:
1. Seperate Rx and Tx operation with two APIs in direct-rearm (Konstantin)
2. Delete L3fwd change for direct rearm (Jerin)
3. enable direct rearm in ixgbe driver in Arm
v4:
1. Rename direct-rearm as buffer recycle. Based on this, function name and variable name are changed to let this mode more general for all drivers. (Konstantin, Morten)
2. Add ring wrapping check (Konstantin)
v5:
1. some change for ethdev API (Morten)
2. add support for avx2, sse, altivec path
v6:
1. fix ixgbe build issue in ppc
2. remove 'recycle_tx_mbufs_reuse' and 'recycle_rx_descriptors_refill' API wrapper (Tech Board meeting)
3. add recycle_mbufs engine in testpmd (Tech Board meeting)
4. add namespace in the functions related to mbufs recycle(Ferruh)
v7:
1. move 'rxq/txq data' pointers to the beginning of eth_dev structure, in order to keep them in the same cache line as rx/tx_burst function pointers (Morten)
2. add the extra description for 'rte_eth_recycle_mbufs' to show it can support feeding 1 Rx queue from 2 Tx queues in the same thread (Konstantin)
3. For i40e/ixgbe driver, make the previous copied buffers as invalid if there are Tx buffers refcnt > 1 or from unexpected mempool (Konstantin)
4. add check for the return value of 'rte_eth_recycle_rx_queue_info_get' in testpmd fwd engine (Morten)
v8:
1. add arm/x86 build option to fix ixgbe build issue in ppc
Feifei Wang (4):
ethdev: add API for mbufs recycle mode
net/i40e: implement mbufs recycle mode
net/ixgbe: implement mbufs recycle mode
app/testpmd: add recycle mbufs engine
app/test-pmd/meson.build | 1 + app/test-pmd/recycle_mbufs.c | 58 ++++++ app/test-pmd/testpmd.c | 1 + app/test-pmd/testpmd.h | 3 + doc/guides/rel_notes/release_23_11.rst | 15 ++ doc/guides/testpmd_app_ug/run_app.rst | 1 + doc/guides/testpmd_app_ug/testpmd_funcs.rst | 5 +- drivers/net/i40e/i40e_ethdev.c | 1 + drivers/net/i40e/i40e_ethdev.h | 2 + .../net/i40e/i40e_recycle_mbufs_vec_common.c | 147 ++++++++++++++ drivers/net/i40e/i40e_rxtx.c | 32 ++++ drivers/net/i40e/i40e_rxtx.h | 4 + drivers/net/i40e/meson.build | 1 + drivers/net/ixgbe/ixgbe_ethdev.c | 1 + drivers/net/ixgbe/ixgbe_ethdev.h | 3 + .../ixgbe/ixgbe_recycle_mbufs_vec_common.c | 143 ++++++++++++++ drivers/net/ixgbe/ixgbe_rxtx.c | 37 +++- drivers/net/ixgbe/ixgbe_rxtx.h | 4 + drivers/net/ixgbe/meson.build | 3 + lib/ethdev/ethdev_driver.h | 10 + lib/ethdev/ethdev_private.c | 2 + lib/ethdev/rte_ethdev.c | 31 +++ lib/ethdev/rte_ethdev.h | 181 ++++++++++++++++++ lib/ethdev/rte_ethdev_core.h | 23 ++- lib/ethdev/version.map | 4 + 25 files changed, 704 insertions(+), 9 deletions(-) create mode 100644 app/test-pmd/recycle_mbufs.c create mode 100644 drivers/net/i40e/i40e_recycle_mbufs_vec_common.c create mode 100644 drivers/net/ixgbe/ixgbe_recycle_mbufs_vec_common.c
Signature · 1 line
2.25.1
PATCH v8 1/4 ethdev: add API for mbufs recycle mode
Add 'rte_eth_recycle_rx_queue_info_get' and 'rte_eth_recycle_mbufs' APIs to recycle used mbufs from a transmit queue of an Ethernet device, and move these mbufs into a mbuf ring for a receive queue of an Ethernet device. This can bypass mempool 'put/get' operations hence saving CPU cycles.
For each recycling mbufs, the rte_eth_recycle_mbufs() function performs the following operations:
- Copy used *rte_mbuf* buffer pointers from Tx mbuf ring into Rx mbuf ring.
- Replenish the Rx descriptors with the recycling *rte_mbuf* mbufs freed from the Tx mbuf ring.
--- doc/guides/rel_notes/release_23_11.rst | 15 ++ lib/ethdev/ethdev_driver.h | 10 ++ lib/ethdev/ethdev_private.c | 2 + lib/ethdev/rte_ethdev.c | 31 +++++ lib/ethdev/rte_ethdev.h | 181 +++++++++++++++++++++++++ lib/ethdev/rte_ethdev_core.h | 23 +++- lib/ethdev/version.map | 4 + 7 files changed, 260 insertions(+), 6 deletions(-)
doc/guides/rel_notes/release_23_11.rst +15 -0
@@ -55,6 +55,13 @@ New Features Also, make sure to start the actual text at the margin. ======================================================= +* **Add mbufs recycling support. ** + + Added ``rte_eth_recycle_rx_queue_info_get`` and ``rte_eth_recycle_mbufs`` + APIs which allow the user to copy used mbufs from the Tx mbuf ring + into the Rx mbuf ring. This feature supports the case that the Rx Ethernet + device is different from the Tx Ethernet device with respective driver + callback functions in ``rte_eth_recycle_mbufs``. Removed Items -------------
@@ -100,6 +107,14 @@ ABI Changes Also, make sure to start the actual text at the margin. ======================================================= +* ethdev: Added ``recycle_tx_mbufs_reuse`` and ``recycle_rx_descriptors_refill`` + fields to ``rte_eth_dev`` structure. + +* ethdev: Structure ``rte_eth_fp_ops`` was affected to add + ``recycle_tx_mbufs_reuse`` and ``recycle_rx_descriptors_refill`` + fields, to move ``rxq`` and ``txq`` fields, to change the size of + ``reserved1`` and ``reserved2`` fields. + Known Issues ------------
lib/ethdev/ethdev_driver.h +10 -0
@@ -58,6 +58,10 @@ struct rte_eth_dev {eth_rx_descriptor_status_t rx_descriptor_status;/** Check the status of a Tx descriptor */ eth_tx_descriptor_status_t tx_descriptor_status; +/** Pointer to PMD transmit mbufs reuse function */ +eth_recycle_tx_mbufs_reuse_t recycle_tx_mbufs_reuse; +/** Pointer to PMD receive descriptors refill function */ +eth_recycle_rx_descriptors_refill_t recycle_rx_descriptors_refill;/** * Device data that is shared between primary and secondary processes
@@ -507,6 +511,10 @@ typedef void (*eth_rxq_info_get_t)(struct rte_eth_dev *dev,typedef void (*eth_txq_info_get_t )(struct rte_eth_dev *dev,uint16_t tx_queue_id,struct rte_eth_txq_info *qinfo); +typedef void (*eth_recycle_rxq_info_get_t )(struct rte_eth_dev *dev, +uint16_t rx_queue_id, +struct rte_eth_recycle_rxq_info *recycle_rxq_info); +typedef int (*eth_burst_mode_get_t )(struct rte_eth_dev *dev,uint16_t queue_id,struct rte_eth_burst_mode *mode);
@@ -1250,6 +1258,8 @@ struct eth_dev_ops {eth_rxq_info_get_t rxq_info_get;/** Retrieve Tx queue information */ eth_txq_info_get_t txq_info_get; +/** Retrieve mbufs recycle Rx queue information */ +eth_recycle_rxq_info_get_t recycle_rxq_info_get;eth_burst_mode_get_t rx_burst_mode_get;/**< Get Rx burst mode */ eth_burst_mode_get_t tx_burst_mode_get;/**< Get Tx burst mode */ eth_fw_version_get_t fw_version_get;/**< Get firmware version */
lib/ethdev/ethdev_private.c +2 -0
@@ -277,6 +277,8 @@ eth_dev_fp_ops_setup(struct rte_eth_fp_ops *fpo, fpo->rx_queue_count = dev->rx_queue_count; fpo->rx_descriptor_status = dev->rx_descriptor_status; fpo->tx_descriptor_status = dev->tx_descriptor_status; + fpo->recycle_tx_mbufs_reuse = dev->recycle_tx_mbufs_reuse; + fpo->recycle_rx_descriptors_refill = dev->recycle_rx_descriptors_refill; fpo->rxq.data = dev->data->rx_queues; fpo->rxq.clbk = (void **)(uintptr_t )dev->post_rx_burst_cbs;
lib/ethdev/rte_ethdev.c +31 -0
@@ -5876,6 +5876,37 @@ rte_eth_tx_queue_info_get(uint16_t port_id, uint16_t queue_id,return 0 ; } +int +rte_eth_recycle_rx_queue_info_get (uint16_t port_id,uint16_t queue_id, +struct rte_eth_recycle_rxq_info *recycle_rxq_info) +{ +struct rte_eth_dev *dev; + +RTE_ETH_VALID_PORTID_OR_ERR_RET (port_id, -ENODEV ); + dev = &rte_eth_devices[port_id]; + +if (queue_id >= dev->data->nb_rx_queues) { +RTE_ETHDEV_LOG (ERR ,"Invalid Rx queue_id=%u\n" , queue_id); +return -EINVAL ; + } + +if (dev->data->rx_queues ==NULL || + dev->data->rx_queues[queue_id] ==NULL ) { +RTE_ETHDEV_LOG (ERR , +"Rx queue %" PRIu16" of device with port_id=%" + PRIu16" has not been setup\n" , + queue_id, port_id); +return -EINVAL ; + } + +if (*dev->dev_ops->recycle_rxq_info_get ==NULL ) +return -ENOTSUP ; + + dev->dev_ops->recycle_rxq_info_get (dev, queue_id, recycle_rxq_info); + +return 0 ; +} +int rte_eth_rx_burst_mode_get (uint16_t port_id,uint16_t queue_id,struct rte_eth_burst_mode *mode)
lib/ethdev/rte_ethdev.h +181 -0
199 lines not on this page · raw message
lib/ethdev/rte_ethdev_core.h +17 -6
@@ -55,6 +55,13 @@ typedef int (*eth_rx_descriptor_status_t)(void *rxq, uint16_t offset);/** @internal Check the status of a Tx descriptor */ typedef int (*eth_tx_descriptor_status_t )(void *txq,uint16_t offset); +/** @internal Copy used mbufs from Tx mbuf ring into Rx mbuf ring */ +typedef uint16_t (*eth_recycle_tx_mbufs_reuse_t )(void *txq, +struct rte_eth_recycle_rxq_info *recycle_rxq_info); + +/** @internal Refill Rx descriptors with the recycling mbufs */ +typedef void (*eth_recycle_rx_descriptors_refill_t )(void *rxq,uint16_t nb); +/** * @internal * Structure used to hold opaque pointers to internal ethdev Rx/Tx
@@ -83,15 +90,17 @@ struct rte_eth_fp_ops {* Rx fast-path functions and related data. * 64-bit systems: occupies first 64B line */ +/** Rx queues data. */ +struct rte_ethdev_qdata rxq;/** PMD receive function. */ eth_rx_burst_t rx_pkt_burst;/** Get the number of used Rx descriptors. */ eth_rx_queue_count_t rx_queue_count;/** Check the status of a Rx descriptor. */ eth_rx_descriptor_status_t rx_descriptor_status; -/** Rx queues data. */ -struct rte_ethdev_qdata rxq; -uintptr_t reserved1[3 ]; +/** Refill Rx descriptors with the recycling mbufs. */ +eth_recycle_rx_descriptors_refill_t recycle_rx_descriptors_refill; +uintptr_t reserved1[2 ];/**@}*/ /**@{*/
@@ -99,15 +108,17 @@ struct rte_eth_fp_ops {* Tx fast-path functions and related data. * 64-bit systems: occupies second 64B line */ +/** Tx queues data. */ +struct rte_ethdev_qdata txq;/** PMD transmit function. */ eth_tx_burst_t tx_pkt_burst;/** PMD transmit prepare function. */ eth_tx_prep_t tx_pkt_prepare;/** Check the status of a Tx descriptor. */ eth_tx_descriptor_status_t tx_descriptor_status; -/** Tx queues data. */ -struct rte_ethdev_qdata txq; -uintptr_t reserved2[3 ]; +/** Copy used mbufs from Tx mbuf ring into Rx. */ +eth_recycle_tx_mbufs_reuse_t recycle_tx_mbufs_reuse; +uintptr_t reserved2[2 ];/**@}*/ } __rte_cache_aligned;
lib/ethdev/version.map +4 -0
@@ -312,6 +312,10 @@ EXPERIMENTAL { rte_flow_async_action_list_handle_query_update; rte_flow_async_actions_update; rte_flow_restore_info_dynflag; + + # added in 23.11 + rte_eth_recycle_mbufs; + rte_eth_recycle_rx_queue_info_get; }; INTERNAL {
Signature · 1 line
2.25.1
PATCH v8 2/4 net/i40e: implement mbufs recycle mode
In reply to Feifei WangDefine specific function implementation for i40e driver. Currently, mbufs recycle mode can support 128bit vector path and avx2 path. And can be enabled both in fast free and no fast free mode.
--- drivers/net/i40e/i40e_ethdev.c | 1 + drivers/net/i40e/i40e_ethdev.h | 2 + .../net/i40e/i40e_recycle_mbufs_vec_common.c | 147 ++++++++++++++++++ drivers/net/i40e/i40e_rxtx.c | 32 ++++ drivers/net/i40e/i40e_rxtx.h | 4 + drivers/net/i40e/meson.build | 1 + 6 files changed, 187 insertions(+) create mode 100644 drivers/net/i40e/i40e_recycle_mbufs_vec_common.c
drivers/net/i40e/i40e_ethdev.c +1 -0
@@ -496,6 +496,7 @@ static const struct eth_dev_ops i40e_eth_dev_ops = { .flow_ops_get = i40e_dev_flow_ops_get, .rxq_info_get = i40e_rxq_info_get, .txq_info_get = i40e_txq_info_get, + .recycle_rxq_info_get = i40e_recycle_rxq_info_get, .rx_burst_mode_get = i40e_rx_burst_mode_get, .tx_burst_mode_get = i40e_tx_burst_mode_get, .timesync_enable = i40e_timesync_enable,
drivers/net/i40e/i40e_ethdev.h +2 -0
@@ -1355,6 +1355,8 @@ void i40e_rxq_info_get(struct rte_eth_dev *dev, uint16_t queue_id, struct rte_eth_rxq_info *qinfo);void i40e_txq_info_get (struct rte_eth_dev *dev,uint16_t queue_id,struct rte_eth_txq_info *qinfo); +void i40e_recycle_rxq_info_get (struct rte_eth_dev *dev,uint16_t queue_id, +struct rte_eth_recycle_rxq_info *recycle_rxq_info);int i40e_rx_burst_mode_get (struct rte_eth_dev *dev,uint16_t queue_id,struct rte_eth_burst_mode *mode); int i40e_tx_burst_mode_get(struct rte_eth_dev *dev, uint16_t queue_id,
drivers/net/i40e/i40e_recycle_mbufs_vec_common.c new file +147 -0
147 lines not on this page · raw message
drivers/net/i40e/i40e_rxtx.c +32 -0
@@ -3199,6 +3199,30 @@ i40e_txq_info_get(struct rte_eth_dev *dev, uint16_t queue_id, qinfo->conf.offloads = txq->offloads; } +void +i40e_recycle_rxq_info_get (struct rte_eth_dev *dev,uint16_t queue_id, +struct rte_eth_recycle_rxq_info *recycle_rxq_info) +{ +struct i40e_rx_queue *rxq; +struct i40e_adapter *ad = +I40E_DEV_PRIVATE_TO_ADAPTER (dev->data->dev_private); + + rxq = dev->data->rx_queues[queue_id]; + + recycle_rxq_info->mbuf_ring = (void *)rxq->sw_ring; + recycle_rxq_info->mp = rxq->mp; + recycle_rxq_info->mbuf_ring_size = rxq->nb_rx_desc; + recycle_rxq_info->receive_tail = &rxq->rx_tail; + +if (ad->rx_vec_allowed) { + recycle_rxq_info->refill_requirement =RTE_I40E_RXQ_REARM_THRESH ; + recycle_rxq_info->refill_head = &rxq->rxrearm_start; + }else { + recycle_rxq_info->refill_requirement = rxq->rx_free_thresh; + recycle_rxq_info->refill_head = &rxq->rx_free_trigger; + } +} +#ifdef RTE_ARCH_X86 static inline bool get_avx_supported (bool request_avx512)
@@ -3293,6 +3317,8 @@ i40e_set_rx_function(struct rte_eth_dev *dev) dev->rx_pkt_burst = ad->rx_use_avx2 ? i40e_recv_scattered_pkts_vec_avx2 : i40e_recv_scattered_pkts_vec; + dev->recycle_rx_descriptors_refill = + i40e_recycle_rx_descriptors_refill_vec; } }else {if (ad->rx_use_avx512) {
@@ -3311,9 +3337,12 @@ i40e_set_rx_function(struct rte_eth_dev *dev) dev->rx_pkt_burst = ad->rx_use_avx2 ? i40e_recv_pkts_vec_avx2 : i40e_recv_pkts_vec; + dev->recycle_rx_descriptors_refill = + i40e_recycle_rx_descriptors_refill_vec; } }#else /* RTE_ARCH_X86 */ + dev->recycle_rx_descriptors_refill = i40e_recycle_rx_descriptors_refill_vec;if (dev->data->scattered_rx) { PMD_INIT_LOG(DEBUG, "Using Vector Scattered Rx (port %d).",
@@ -3481,15 +3510,18 @@ i40e_set_tx_function(struct rte_eth_dev *dev) dev->tx_pkt_burst = ad->tx_use_avx2 ? i40e_xmit_pkts_vec_avx2 : i40e_xmit_pkts_vec; + dev->recycle_tx_mbufs_reuse = i40e_recycle_tx_mbufs_reuse_vec; }#else /* RTE_ARCH_X86 */ PMD_INIT_LOG (DEBUG ,"Using Vector Tx (port %d)." , dev->data->port_id); dev->tx_pkt_burst = i40e_xmit_pkts_vec; + dev->recycle_tx_mbufs_reuse = i40e_recycle_tx_mbufs_reuse_vec;#endif /* RTE_ARCH_X86 */ }else {PMD_INIT_LOG (DEBUG ,"Simple tx finally be used." ); dev->tx_pkt_burst = i40e_xmit_pkts_simple; + dev->recycle_tx_mbufs_reuse = i40e_recycle_tx_mbufs_reuse_vec; } dev->tx_pkt_prepare = i40e_simple_prep_pkts; }else {
drivers/net/i40e/i40e_rxtx.h +4 -0
@@ -236,6 +236,10 @@ uint32_t i40e_dev_rx_queue_count(void *rx_queue);int i40e_dev_rx_descriptor_status (void *rx_queue,uint16_t offset);int i40e_dev_tx_descriptor_status (void *tx_queue,uint16_t offset); +uint16_t i40e_recycle_tx_mbufs_reuse_vec (void *tx_queue, +struct rte_eth_recycle_rxq_info *recycle_rxq_info); +void i40e_recycle_rx_descriptors_refill_vec (void *rx_queue,uint16_t nb_mbufs); +uint16_t i40e_recv_pkts_vec (void *rx_queue,struct rte_mbuf **rx_pkts,uint16_t nb_pkts); uint16_t i40e_recv_scattered_pkts_vec(void *rx_queue,
drivers/net/i40e/meson.build +1 -0
@@ -34,6 +34,7 @@ sources = files( 'i40e_tm.c', 'i40e_hash.c', 'i40e_vf_representor.c', + 'i40e_recycle_mbufs_vec_common.c', 'rte_pmd_i40e.c', )
Signature · 1 line
2.25.1
PATCH v8 3/4 net/ixgbe: implement mbufs recycle mode
In reply to Feifei WangDefine specific function implementation for ixgbe driver. Currently, recycle buffer mode can support 128bit vector path. And can be enabled both in fast free and no fast free mode.
--- drivers/net/ixgbe/ixgbe_ethdev.c | 1 + drivers/net/ixgbe/ixgbe_ethdev.h | 3 + .../ixgbe/ixgbe_recycle_mbufs_vec_common.c | 143 ++++++++++++++++++ drivers/net/ixgbe/ixgbe_rxtx.c | 37 ++++- drivers/net/ixgbe/ixgbe_rxtx.h | 4 + drivers/net/ixgbe/meson.build | 3 + 6 files changed, 189 insertions(+), 2 deletions(-) create mode 100644 drivers/net/ixgbe/ixgbe_recycle_mbufs_vec_common.c
drivers/net/ixgbe/ixgbe_ethdev.c +1 -0
@@ -543,6 +543,7 @@ static const struct eth_dev_ops ixgbe_eth_dev_ops = { .set_mc_addr_list = ixgbe_dev_set_mc_addr_list, .rxq_info_get = ixgbe_rxq_info_get, .txq_info_get = ixgbe_txq_info_get, + .recycle_rxq_info_get = ixgbe_recycle_rxq_info_get, .timesync_enable = ixgbe_timesync_enable, .timesync_disable = ixgbe_timesync_disable, .timesync_read_rx_timestamp = ixgbe_timesync_read_rx_timestamp,
drivers/net/ixgbe/ixgbe_ethdev.h +3 -0
@@ -626,6 +626,9 @@ void ixgbe_rxq_info_get(struct rte_eth_dev *dev, uint16_t queue_id,void ixgbe_txq_info_get (struct rte_eth_dev *dev,uint16_t queue_id,struct rte_eth_txq_info *qinfo); +void ixgbe_recycle_rxq_info_get (struct rte_eth_dev *dev,uint16_t queue_id, +struct rte_eth_recycle_rxq_info *recycle_rxq_info); +int ixgbevf_dev_rx_init (struct rte_eth_dev *dev);void ixgbevf_dev_tx_init (struct rte_eth_dev *dev);
drivers/net/ixgbe/ixgbe_recycle_mbufs_vec_common.c new file +143 -0
143 lines not on this page · raw message
drivers/net/ixgbe/ixgbe_rxtx.c +35 -2
@@ -2552,6 +2552,9 @@ ixgbe_set_tx_function(struct rte_eth_dev *dev, struct ixgbe_tx_queue *txq) (rte_eal_process_type() != RTE_PROC_PRIMARY || ixgbe_txq_vec_setup(txq) == 0)) {PMD_INIT_LOG (DEBUG ,"Vector tx enabled." ); +#if defined(RTE_ARCH_X86 ) || defined(RTE_ARCH_ARM ) + dev->recycle_tx_mbufs_reuse = ixgbe_recycle_tx_mbufs_reuse_vec; +#endif dev->tx_pkt_burst = ixgbe_xmit_pkts_vec; } else dev->tx_pkt_burst = ixgbe_xmit_pkts_simple;
@@ -4890,7 +4893,10 @@ ixgbe_set_rx_function(struct rte_eth_dev *dev)PMD_INIT_LOG (DEBUG ,"Using Vector Scattered Rx " "callback (port=%d)." , dev->data->port_id); - +#if defined(RTE_ARCH_X86 ) || defined(RTE_ARCH_ARM ) + dev->recycle_rx_descriptors_refill = + ixgbe_recycle_rx_descriptors_refill_vec; +#endif dev->rx_pkt_burst = ixgbe_recv_scattered_pkts_vec; }else if (adapter->rx_bulk_alloc_allowed) { PMD_INIT_LOG(DEBUG, "Using a Scattered with bulk "
@@ -4919,7 +4925,9 @@ ixgbe_set_rx_function(struct rte_eth_dev *dev)"burst size no less than %d (port=%d)." ,RTE_IXGBE_DESCS_PER_LOOP , dev->data->port_id); - +#if defined(RTE_ARCH_X86 ) || defined(RTE_ARCH_ARM ) + dev->recycle_rx_descriptors_refill = ixgbe_recycle_rx_descriptors_refill_vec; +#endif dev->rx_pkt_burst = ixgbe_recv_pkts_vec; }else if (adapter->rx_bulk_alloc_allowed) { PMD_INIT_LOG(DEBUG, "Rx Burst Bulk Alloc Preconditions are "
@@ -5691,6 +5699,31 @@ ixgbe_txq_info_get(struct rte_eth_dev *dev, uint16_t queue_id, qinfo->conf.tx_deferred_start = txq->tx_deferred_start; } +void +ixgbe_recycle_rxq_info_get (struct rte_eth_dev *dev,uint16_t queue_id, +struct rte_eth_recycle_rxq_info *recycle_rxq_info) +{ +struct ixgbe_rx_queue *rxq; +struct ixgbe_adapter *adapter = dev->data->dev_private; + + rxq = dev->data->rx_queues[queue_id]; + + recycle_rxq_info->mbuf_ring = (void *)rxq->sw_ring; + recycle_rxq_info->mp = rxq->mb_pool; + recycle_rxq_info->mbuf_ring_size = rxq->nb_rx_desc; + recycle_rxq_info->receive_tail = &rxq->rx_tail; + +if (adapter->rx_vec_allowed) { +#if defined(RTE_ARCH_X86 ) || defined(RTE_ARCH_ARM ) + recycle_rxq_info->refill_requirement =RTE_IXGBE_RXQ_REARM_THRESH ; + recycle_rxq_info->refill_head = &rxq->rxrearm_start; +#endif + }else { + recycle_rxq_info->refill_requirement = rxq->rx_free_thresh; + recycle_rxq_info->refill_head = &rxq->rx_free_trigger; + } +} +/* * [VF] Initializes Receive Unit. */
drivers/net/ixgbe/ixgbe_rxtx.h +4 -0
@@ -295,6 +295,10 @@ int ixgbe_dev_tx_done_cleanup(void *tx_queue, uint32_t free_cnt);extern const uint32_t ptype_table[IXGBE_PACKET_TYPE_MAX ];extern const uint32_t ptype_table_tn[IXGBE_PACKET_TYPE_TN_MAX ]; +uint16_t ixgbe_recycle_tx_mbufs_reuse_vec (void *tx_queue, +struct rte_eth_recycle_rxq_info *recycle_rxq_info); +void ixgbe_recycle_rx_descriptors_refill_vec (void *rx_queue,uint16_t nb_mbufs); +uint16_t ixgbe_xmit_fixed_burst_vec (void *tx_queue,struct rte_mbuf **tx_pkts,uint16_t nb_pkts);int ixgbe_txq_vec_setup (struct ixgbe_tx_queue *txq);
drivers/net/ixgbe/meson.build +3 -0
@@ -17,6 +17,7 @@ sources = files( 'ixgbe_rxtx.c', 'ixgbe_tm.c', 'ixgbe_vf_representor.c', + 'ixgbe_recycle_mbufs_vec_common.c', 'rte_pmd_ixgbe.c', )
@@ -26,11 +27,13 @@ deps += ['hash', 'security'] if arch_subdir == 'x86' sources += files('ixgbe_rxtx_vec_sse.c') + sources += files('ixgbe_recycle_mbufs_vec_common.c') if is_windows and cc.get_id() != 'clang' cflags += ['-fno-asynchronous-unwind-tables'] endif elif arch_subdir == 'arm' sources += files('ixgbe_rxtx_vec_neon.c') + sources += files('ixgbe_recycle_mbufs_vec_common.c') endif includes += include_directories('base')
Signature · 1 line
2.25.1
PATCH v8 4/4 app/testpmd: add recycle mbufs engine
In reply to Feifei WangAdd recycle mbufs engine for testpmd. This engine forward pkts with I/O forward mode. But enable mbufs recycle feature to recycle used txq mbufs for rxq mbuf ring, which can bypass mempool path and save CPU cycles.
--- app/test-pmd/meson.build | 1 + app/test-pmd/recycle_mbufs.c | 58 +++++++++++++++++++++ app/test-pmd/testpmd.c | 1 + app/test-pmd/testpmd.h | 3 ++ doc/guides/testpmd_app_ug/run_app.rst | 1 + doc/guides/testpmd_app_ug/testpmd_funcs.rst | 5 +- 6 files changed, 68 insertions(+), 1 deletion(-) create mode 100644 app/test-pmd/recycle_mbufs.c
app/test-pmd/meson.build +1 -0
@@ -22,6 +22,7 @@ sources = files( 'macswap.c', 'noisy_vnf.c', 'parameters.c', + 'recycle_mbufs.c', 'rxonly.c', 'shared_rxq_fwd.c', 'testpmd.c',
app/test-pmd/recycle_mbufs.c new file +58 -0
58 lines not on this page · raw message
app/test-pmd/testpmd.c +1 -0
@@ -199,6 +199,7 @@ struct fwd_engine * fwd_engines[] = { &icmp_echo_engine, &noisy_vnf_engine, &five_tuple_swap_fwd_engine, + &recycle_mbufs_engine, #ifdef RTE_LIBRTE_IEEE1588 &ieee1588_fwd_engine, #endif
app/test-pmd/testpmd.h +3 -0
@@ -188,6 +188,8 @@ struct fwd_stream {struct pkt_burst_stats rx_burst_stats;struct pkt_burst_stats tx_burst_stats;struct fwd_lcore *lcore;/**< Lcore being scheduled. */ +/**< Rx queue information for recycling mbufs */ +struct rte_eth_recycle_rxq_info recycle_rxq_info; };/**
@@ -449,6 +451,7 @@ extern struct fwd_engine csum_fwd_engine;extern struct fwd_engine icmp_echo_engine;extern struct fwd_engine noisy_vnf_engine;extern struct fwd_engine five_tuple_swap_fwd_engine; +extern struct fwd_engine recycle_mbufs_engine;#ifdef RTE_LIBRTE_IEEE1588 extern struct fwd_engine ieee1588_fwd_engine;#endif
doc/guides/testpmd_app_ug/run_app.rst +1 -0
@@ -232,6 +232,7 @@ The command line options are: noisy 5tswap shared-rxq + recycle_mbufs * ``--rss-ip``
doc/guides/testpmd_app_ug/testpmd_funcs.rst +4 -1
@@ -318,7 +318,7 @@ set fwd Set the packet forwarding mode:: testpmd> set fwd (io|mac|macswap|flowgen| \ - rxonly|txonly|csum|icmpecho|noisy|5tswap|shared-rxq) (""|retry) + rxonly|txonly|csum|icmpecho|noisy|5tswap|shared-rxq|recycle_mbufs) (""|retry) ``retry`` can be specified for forwarding engines except ``rx_only``.
@@ -364,6 +364,9 @@ The available information categories are: * ``shared-rxq``: Receive only for shared Rx queue. Resolve packet source port from mbuf and update stream statistics accordingly. +* ``recycle_mbufs``: Recycle Tx queue used mbufs for Rx queue mbuf ring. + This mode uses fast path mbuf recycle feature and forwards packets in I/O mode. + Example:: testpmd> set fwd rxonly
Signature · 1 line
2.25.1
v9 cover Recycle mbufs from Tx queue into Rx queue
In reply to Feifei WangCurrently, the transmit side frees the buffers into the lcore cache and the receive side allocates buffers from the lcore cache. The transmit side typically frees 32 buffers resulting in 32*8=256B of stores to lcore cache. The receive side allocates 32 buffers and stores them in the receive side software ring, resulting in 32*8=256B of stores and 256B of load from the lcore cache.
This patch proposes a mechanism to avoid freeing to/allocating from the lcore cache. i.e. the receive side will free the buffers from transmit side directly into its software ring. This will avoid the 256B of loads and stores introduced by the lcore cache. It also frees up the cache lines used by the lcore cache. And we can call this mode as mbufs recycle mode.
In the latest version, mbufs recycle mode is packaged as a separate API.
This allows for the users to change rxq/txq pairing in real time in data plane, according to the analysis of the packet flow by the application, for example:
-----------------------------------------------------------------------
Step 1: upper application analyse the flow direction
Step 2: recycle_rxq_info = rte_eth_recycle_rx_queue_info_get(rx_portid, rx_queueid) Step 3: rte_eth_recycle_mbufs(rx_portid, rx_queueid, tx_portid, tx_queueid, recycle_rxq_info); Step 4: rte_eth_rx_burst(rx_portid,rx_queueid);
Step 5: rte_eth_tx_burst(tx_portid,tx_queueid);
-----------------------------------------------------------------------
Above can support user to change rxq/txq pairing at run-time and user does not need to know the direction of flow in advance. This can effectively expand mbufs recycle mode's use scenarios.
Furthermore, mbufs recycle mode is no longer limited to the same pmd,
it can support moving mbufs between different vendor pmds, even can put the mbufs anywhere into your Rx mbuf ring as long as the address of the mbuf ring can be provided. In the latest version, we enable mbufs recycle mode in i40e pmd and ixgbe pmd, and also try to use i40e driver in Rx, ixgbe driver in Tx, and then achieve 7-9% performance improvement by mbufs recycle mode.
Difference between mbuf recycle, ZC API used in mempool and general path
For general path:
Rx: 32 pkts memcpy from mempool cache to rx_sw_ring
Tx: 32 pkts memcpy from tx_sw_ring to temporary variable + 32 pkts memcpy from temporary variable to mempool cache
For ZC API used in mempool:
Rx: 32 pkts memcpy from mempool cache to rx_sw_ring
Tx: 32 pkts memcpy from tx_sw_ring to zero-copy mempool cache
Refer link: http://patches.dpdk.org/project/dpdk/patch/[email protected]/
For mbufs recycle:
Rx/Tx: 32 pkts memcpy from tx_sw_ring to rx_sw_ring
Thus we can see in the one loop, compared to general path, mbufs recycle mode reduces 32+32=64 pkts memcpy; Compared to ZC API used in mempool, we can see mbufs recycle mode reduce 32 pkts memcpy in each loop.
So, mbufs recycle has its own benefits.
Testing status:
(1) dpdk l3fwd test with multiple drivers:
port 0: 82599 NIC port 1: XL710 NIC
-------------------------------------------------------------
Without fast free With fast free
Thunderx2: +7.53% +13.54%
-------------------------------------------------------------
(2) dpdk l3fwd test with same driver:
port 0 && 1: XL710 NIC
-------------------------------------------------------------
Without fast free With fast free
Ampere altra: +12.61% +11.42%
n1sdp: +8.30% +3.85%
x86-sse: +8.43% +3.72%
-------------------------------------------------------------
(3) Performance comparison with ZC_mempool used
port 0 && 1: XL710 NIC
with fast free
-------------------------------------------------------------
With recycle buffer With zc_mempool
Ampere altra: 11.42% 3.54%
-------------------------------------------------------------
Furthermore, we add recycle_mbuf engine in testpmd. Due to XL710 NIC has I/O bottleneck in testpmd in ampere altra, we can not see throughput change compared with I/O fwd engine. However, using record cmd in testpmd:
'$set record-burst-stats on'
we can see the ratio of 'Rx/Tx burst size of 32' is reduced. This indicate mbufs recycle can save CPU cycles.
V2:
1. Use data-plane API to enable direct-rearm (Konstantin, Honnappa)
2. Add 'txq_data_get' API to get txq info for Rx (Konstantin)
3. Use input parameter to enable direct rearm in l3fwd (Konstantin)
4. Add condition detection for direct rearm API (Morten, Andrew Rybchenko)
V3:
1. Seperate Rx and Tx operation with two APIs in direct-rearm (Konstantin)
2. Delete L3fwd change for direct rearm (Jerin)
3. enable direct rearm in ixgbe driver in Arm
v4:
1. Rename direct-rearm as buffer recycle. Based on this, function name and variable name are changed to let this mode more general for all drivers. (Konstantin, Morten)
2. Add ring wrapping check (Konstantin)
v5:
1. some change for ethdev API (Morten)
2. add support for avx2, sse, altivec path
v6:
1. fix ixgbe build issue in ppc
2. remove 'recycle_tx_mbufs_reuse' and 'recycle_rx_descriptors_refill' API wrapper (Tech Board meeting)
3. add recycle_mbufs engine in testpmd (Tech Board meeting)
4. add namespace in the functions related to mbufs recycle(Ferruh)
v7:
1. move 'rxq/txq data' pointers to the beginning of eth_dev structure, in order to keep them in the same cache line as rx/tx_burst function pointers (Morten)
2. add the extra description for 'rte_eth_recycle_mbufs' to show it can support feeding 1 Rx queue from 2 Tx queues in the same thread (Konstantin)
3. For i40e/ixgbe driver, make the previous copied buffers as invalid if there are Tx buffers refcnt > 1 or from unexpected mempool (Konstantin)
4. add check for the return value of 'rte_eth_recycle_rx_queue_info_get' in testpmd fwd engine (Morten)
v8:
1. add arm/x86 build option to fix ixgbe build issue in ppc
v9:
1. delete duplicate file name for ixgbe
Feifei Wang (4):
ethdev: add API for mbufs recycle mode
net/i40e: implement mbufs recycle mode
net/ixgbe: implement mbufs recycle mode
app/testpmd: add recycle mbufs engine
app/test-pmd/meson.build | 1 + app/test-pmd/recycle_mbufs.c | 58 ++++++ app/test-pmd/testpmd.c | 1 + app/test-pmd/testpmd.h | 3 + doc/guides/rel_notes/release_23_11.rst | 15 ++ doc/guides/testpmd_app_ug/run_app.rst | 1 + doc/guides/testpmd_app_ug/testpmd_funcs.rst | 5 +- drivers/net/i40e/i40e_ethdev.c | 1 + drivers/net/i40e/i40e_ethdev.h | 2 + .../net/i40e/i40e_recycle_mbufs_vec_common.c | 147 ++++++++++++++ drivers/net/i40e/i40e_rxtx.c | 32 ++++ drivers/net/i40e/i40e_rxtx.h | 4 + drivers/net/i40e/meson.build | 1 + drivers/net/ixgbe/ixgbe_ethdev.c | 1 + drivers/net/ixgbe/ixgbe_ethdev.h | 3 + .../ixgbe/ixgbe_recycle_mbufs_vec_common.c | 143 ++++++++++++++ drivers/net/ixgbe/ixgbe_rxtx.c | 37 +++- drivers/net/ixgbe/ixgbe_rxtx.h | 4 + drivers/net/ixgbe/meson.build | 2 + lib/ethdev/ethdev_driver.h | 10 + lib/ethdev/ethdev_private.c | 2 + lib/ethdev/rte_ethdev.c | 31 +++ lib/ethdev/rte_ethdev.h | 181 ++++++++++++++++++ lib/ethdev/rte_ethdev_core.h | 23 ++- lib/ethdev/version.map | 4 + 25 files changed, 703 insertions(+), 9 deletions(-) create mode 100644 app/test-pmd/recycle_mbufs.c create mode 100644 drivers/net/i40e/i40e_recycle_mbufs_vec_common.c create mode 100644 drivers/net/ixgbe/ixgbe_recycle_mbufs_vec_common.c
Signature · 1 line
2.25.1
PATCH v9 1/4 ethdev: add API for mbufs recycle mode
Add 'rte_eth_recycle_rx_queue_info_get' and 'rte_eth_recycle_mbufs' APIs to recycle used mbufs from a transmit queue of an Ethernet device, and move these mbufs into a mbuf ring for a receive queue of an Ethernet device. This can bypass mempool 'put/get' operations hence saving CPU cycles.
For each recycling mbufs, the rte_eth_recycle_mbufs() function performs the following operations:
- Copy used *rte_mbuf* buffer pointers from Tx mbuf ring into Rx mbuf ring.
- Replenish the Rx descriptors with the recycling *rte_mbuf* mbufs freed from the Tx mbuf ring.
--- doc/guides/rel_notes/release_23_11.rst | 15 ++ lib/ethdev/ethdev_driver.h | 10 ++ lib/ethdev/ethdev_private.c | 2 + lib/ethdev/rte_ethdev.c | 31 +++++ lib/ethdev/rte_ethdev.h | 181 +++++++++++++++++++++++++ lib/ethdev/rte_ethdev_core.h | 23 +++- lib/ethdev/version.map | 4 + 7 files changed, 260 insertions(+), 6 deletions(-)
doc/guides/rel_notes/release_23_11.rst +15 -0
@@ -55,6 +55,13 @@ New Features Also, make sure to start the actual text at the margin. ======================================================= +* **Add mbufs recycling support. ** + + Added ``rte_eth_recycle_rx_queue_info_get`` and ``rte_eth_recycle_mbufs`` + APIs which allow the user to copy used mbufs from the Tx mbuf ring + into the Rx mbuf ring. This feature supports the case that the Rx Ethernet + device is different from the Tx Ethernet device with respective driver + callback functions in ``rte_eth_recycle_mbufs``. Removed Items -------------
@@ -100,6 +107,14 @@ ABI Changes Also, make sure to start the actual text at the margin. ======================================================= +* ethdev: Added ``recycle_tx_mbufs_reuse`` and ``recycle_rx_descriptors_refill`` + fields to ``rte_eth_dev`` structure. + +* ethdev: Structure ``rte_eth_fp_ops`` was affected to add + ``recycle_tx_mbufs_reuse`` and ``recycle_rx_descriptors_refill`` + fields, to move ``rxq`` and ``txq`` fields, to change the size of + ``reserved1`` and ``reserved2`` fields. + Known Issues ------------
lib/ethdev/ethdev_driver.h +10 -0
@@ -58,6 +58,10 @@ struct rte_eth_dev {eth_rx_descriptor_status_t rx_descriptor_status;/** Check the status of a Tx descriptor */ eth_tx_descriptor_status_t tx_descriptor_status; +/** Pointer to PMD transmit mbufs reuse function */ +eth_recycle_tx_mbufs_reuse_t recycle_tx_mbufs_reuse; +/** Pointer to PMD receive descriptors refill function */ +eth_recycle_rx_descriptors_refill_t recycle_rx_descriptors_refill;/** * Device data that is shared between primary and secondary processes
@@ -507,6 +511,10 @@ typedef void (*eth_rxq_info_get_t)(struct rte_eth_dev *dev,typedef void (*eth_txq_info_get_t )(struct rte_eth_dev *dev,uint16_t tx_queue_id,struct rte_eth_txq_info *qinfo); +typedef void (*eth_recycle_rxq_info_get_t )(struct rte_eth_dev *dev, +uint16_t rx_queue_id, +struct rte_eth_recycle_rxq_info *recycle_rxq_info); +typedef int (*eth_burst_mode_get_t )(struct rte_eth_dev *dev,uint16_t queue_id,struct rte_eth_burst_mode *mode);
@@ -1250,6 +1258,8 @@ struct eth_dev_ops {eth_rxq_info_get_t rxq_info_get;/** Retrieve Tx queue information */ eth_txq_info_get_t txq_info_get; +/** Retrieve mbufs recycle Rx queue information */ +eth_recycle_rxq_info_get_t recycle_rxq_info_get;eth_burst_mode_get_t rx_burst_mode_get;/**< Get Rx burst mode */ eth_burst_mode_get_t tx_burst_mode_get;/**< Get Tx burst mode */ eth_fw_version_get_t fw_version_get;/**< Get firmware version */
lib/ethdev/ethdev_private.c +2 -0
@@ -277,6 +277,8 @@ eth_dev_fp_ops_setup(struct rte_eth_fp_ops *fpo, fpo->rx_queue_count = dev->rx_queue_count; fpo->rx_descriptor_status = dev->rx_descriptor_status; fpo->tx_descriptor_status = dev->tx_descriptor_status; + fpo->recycle_tx_mbufs_reuse = dev->recycle_tx_mbufs_reuse; + fpo->recycle_rx_descriptors_refill = dev->recycle_rx_descriptors_refill; fpo->rxq.data = dev->data->rx_queues; fpo->rxq.clbk = (void **)(uintptr_t )dev->post_rx_burst_cbs;
lib/ethdev/rte_ethdev.c +31 -0
@@ -5876,6 +5876,37 @@ rte_eth_tx_queue_info_get(uint16_t port_id, uint16_t queue_id,return 0 ; } +int +rte_eth_recycle_rx_queue_info_get (uint16_t port_id,uint16_t queue_id, +struct rte_eth_recycle_rxq_info *recycle_rxq_info) +{ +struct rte_eth_dev *dev; + +RTE_ETH_VALID_PORTID_OR_ERR_RET (port_id, -ENODEV ); + dev = &rte_eth_devices[port_id]; + +if (queue_id >= dev->data->nb_rx_queues) { +RTE_ETHDEV_LOG (ERR ,"Invalid Rx queue_id=%u\n" , queue_id); +return -EINVAL ; + } + +if (dev->data->rx_queues ==NULL || + dev->data->rx_queues[queue_id] ==NULL ) { +RTE_ETHDEV_LOG (ERR , +"Rx queue %" PRIu16" of device with port_id=%" + PRIu16" has not been setup\n" , + queue_id, port_id); +return -EINVAL ; + } + +if (*dev->dev_ops->recycle_rxq_info_get ==NULL ) +return -ENOTSUP ; + + dev->dev_ops->recycle_rxq_info_get (dev, queue_id, recycle_rxq_info); + +return 0 ; +} +int rte_eth_rx_burst_mode_get (uint16_t port_id,uint16_t queue_id,struct rte_eth_burst_mode *mode)
lib/ethdev/rte_ethdev.h +181 -0
199 lines not on this page · raw message
lib/ethdev/rte_ethdev_core.h +17 -6
@@ -55,6 +55,13 @@ typedef int (*eth_rx_descriptor_status_t)(void *rxq, uint16_t offset);/** @internal Check the status of a Tx descriptor */ typedef int (*eth_tx_descriptor_status_t )(void *txq,uint16_t offset); +/** @internal Copy used mbufs from Tx mbuf ring into Rx mbuf ring */ +typedef uint16_t (*eth_recycle_tx_mbufs_reuse_t )(void *txq, +struct rte_eth_recycle_rxq_info *recycle_rxq_info); + +/** @internal Refill Rx descriptors with the recycling mbufs */ +typedef void (*eth_recycle_rx_descriptors_refill_t )(void *rxq,uint16_t nb); +/** * @internal * Structure used to hold opaque pointers to internal ethdev Rx/Tx
@@ -83,15 +90,17 @@ struct rte_eth_fp_ops {* Rx fast-path functions and related data. * 64-bit systems: occupies first 64B line */ +/** Rx queues data. */ +struct rte_ethdev_qdata rxq;/** PMD receive function. */ eth_rx_burst_t rx_pkt_burst;/** Get the number of used Rx descriptors. */ eth_rx_queue_count_t rx_queue_count;/** Check the status of a Rx descriptor. */ eth_rx_descriptor_status_t rx_descriptor_status; -/** Rx queues data. */ -struct rte_ethdev_qdata rxq; -uintptr_t reserved1[3 ]; +/** Refill Rx descriptors with the recycling mbufs. */ +eth_recycle_rx_descriptors_refill_t recycle_rx_descriptors_refill; +uintptr_t reserved1[2 ];/**@}*/ /**@{*/
@@ -99,15 +108,17 @@ struct rte_eth_fp_ops {* Tx fast-path functions and related data. * 64-bit systems: occupies second 64B line */ +/** Tx queues data. */ +struct rte_ethdev_qdata txq;/** PMD transmit function. */ eth_tx_burst_t tx_pkt_burst;/** PMD transmit prepare function. */ eth_tx_prep_t tx_pkt_prepare;/** Check the status of a Tx descriptor. */ eth_tx_descriptor_status_t tx_descriptor_status; -/** Tx queues data. */ -struct rte_ethdev_qdata txq; -uintptr_t reserved2[3 ]; +/** Copy used mbufs from Tx mbuf ring into Rx. */ +eth_recycle_tx_mbufs_reuse_t recycle_tx_mbufs_reuse; +uintptr_t reserved2[2 ];/**@}*/ } __rte_cache_aligned;
lib/ethdev/version.map +4 -0
@@ -312,6 +312,10 @@ EXPERIMENTAL { rte_flow_async_action_list_handle_query_update; rte_flow_async_actions_update; rte_flow_restore_info_dynflag; + + # added in 23.11 + rte_eth_recycle_mbufs; + rte_eth_recycle_rx_queue_info_get; }; INTERNAL {
Signature · 1 line
2.25.1