net/mlx5: fix shared Rx queue limitations
PATCH 1/1 net/mlx5: fix shared Rx queue limitations
Affected patch introduced a new Rx burst function to mlx5 PMD for handling out of order completions which are possible on shared Rx queues.
This burst function is selected if and only if at least one of the Rx queues on the device is configured as shared.
This burst function contains only non-vectorized code and is not compatible with multi packet Rx (MPRQ). Although the function selection worked correctly, if either vectorized Rx datapath was supported or MPRQ was enabled, the relevant resources were still initialized. For example, additional fake mbufs were set up for vectorized Rx leading to memory leaks.
This patch fixes that by adding additional checks for shared Rx queues to vectorized and MPRQ support checks.
Shared Rx queue limitations and Rx burst functions info is also amended in mlx5 PMD documentation.
--- doc/guides/nics/mlx5.rst | 41 +++++++++++++++++++------------- drivers/net/mlx5/mlx5_rx.h | 36 +++++++++++++++------------- drivers/net/mlx5/mlx5_rxtx_vec.c | 2 ++ 3 files changed, 46 insertions(+), 33 deletions(-)
doc/guides/nics/mlx5.rst +25 -16
@@ -956,27 +956,32 @@ The Rx function is selected based on multiple parameters: - :ref:`multi-packet Rx queues (MPRQ) <mlx5_mprq_params>` - :ref:`vectorized Rx datapath <mlx5_rx_vec_param>` -This parameter may also have an impact on the behavior: +These configurations may also have an impact on the behavior: - :ref:`packet descriptor (CQE) compression <mlx5_cqe_comp_param>` +- :ref:`shared Rx queue <mlx5_shared_rx>` .. table:: Rx burst functions - +-------------------+------------------------+---------+-----------------+------+-------+ - || Function Name || Parameters to Enable || Scatter|| Error Recovery || CQE || Large| - | | | | || comp|| MTU | - +===================+========================+=========+=================+======+=======+ - | rx_burst | rx_vec_en=0 | Yes | Yes | Yes | Yes | - +-------------------+------------------------+---------+-----------------+------+-------+ - | rx_burst_vec | rx_vec_en=1 (default) | No | if CQE comp off | Yes | No | - +-------------------+------------------------+---------+-----------------+------+-------+ - | rx_burst_mprq || mprq_en=1 | No | Yes | Yes | Yes | - | || RxQs >= rxqs_min_mprq | | | | | - +-------------------+------------------------+---------+-----------------+------+-------+ - | rx_burst_mprq_vec || rx_vec_en=1 (default) | No | if CQE comp off | Yes | Yes | - | || mprq_en=1 | | | | | - | || RxQs >= rxqs_min_mprq | | | | | - +-------------------+------------------------+---------+-----------------+------+-------+ + +-------------------+------------------------+---------+-----------------+------+-------+---------+ + || Function Name || Parameters to Enable || Scatter|| Error Recovery || CQE || Large|| Shared | + | | | | || comp|| MTU | RXQ | + +===================+========================+=========+=================+======+=======+=========+ + | rx_burst | rx_vec_en=0 | Yes | Yes | Yes | Yes | No | + +-------------------+------------------------+---------+-----------------+------+-------+---------+ + | rx_burst_vec | rx_vec_en=1 (default) | No | if CQE comp off | Yes | No | No | + +-------------------+------------------------+---------+-----------------+------+-------+---------+ + | rx_burst_mprq || mprq_en=1 | No | Yes | Yes | Yes | No | + | || RxQs >= rxqs_min_mprq | | | | | | + +-------------------+------------------------+---------+-----------------+------+-------+---------+ + | rx_burst_mprq_vec || rx_vec_en=1 (default) | No | if CQE comp off | Yes | Yes | No | + | || mprq_en=1 | | | | | | + | || RxQs >= rxqs_min_mprq | | | | | | + +-------------------+------------------------+---------+-----------------+------+-------+---------+ + | rx_burst (OoO) | at least one Rx queue | Yes | Yes | Yes | Yes | Yes | + | | on the device | | | | | | + | | is shared | | | | | | + +-------------------+------------------------+---------+-----------------+------+-------+---------+ Rx/Tx Tuning
@@ -1829,6 +1834,10 @@ Shared Rx Queue Limitations ^^^^^^^^^^^ +#. Shared Rx queue is not compatible with both vectorized Rx datapath and multi-packet Rx queues. + Any configuration related to these features passed through device arguments + will be ignored. + #. Counters of received packets and bytes of devices in the same share group are same. #. Counters of received packets and bytes of queues in the same group and queue ID are same.
drivers/net/mlx5/mlx5_rx.h +19 -17
@@ -602,6 +602,23 @@ mprq_buf_to_pkt(struct mlx5_rxq_data *rxq, struct rte_mbuf *pkt, uint32_t len,return MLX5_RXQ_CODE_EXIT ; } +/** +* Check whether Shared RQ is enabled for the device. +* +* @param dev +* Pointer to Ethernet device. +* +* @return +* 0 if disabled, otherwise enabled. +*/ +static __rte_always_inline int +mlx5_shared_rq_enabled (struct rte_eth_dev *dev) +{ +struct mlx5_priv *priv = dev->data->dev_private; + +return !LIST_EMPTY (&priv->sh->shared_rxqs); +} +/** * Check whether Multi-Packet RQ can be enabled for the device. *
@@ -616,6 +633,8 @@ mlx5_check_mprq_support(struct rte_eth_dev *dev) {struct mlx5_priv *priv = dev->data->dev_private; +if (mlx5_shared_rq_enabled (dev)) +return -ENOTSUP ;if (priv->config.mprq.enabled && priv->rxqs_n >= priv->config.mprq.min_rxqs_num)return 1 ;
@@ -671,23 +690,6 @@ mlx5_mprq_enabled(struct rte_eth_dev *dev)return n == n_ibv; } -/** -* Check whether Shared RQ is enabled for the device. -* -* @param dev -* Pointer to Ethernet device. -* -* @return -* 0 if disabled, otherwise enabled. -*/ -static __rte_always_inline int -mlx5_shared_rq_enabled (struct rte_eth_dev *dev) -{ -struct mlx5_priv *priv = dev->data->dev_private; - -return !LIST_EMPTY (&priv->sh->shared_rxqs); -} -/** * Check whether given RxQ is external. *
drivers/net/mlx5/mlx5_rxtx_vec.c +2 -0
@@ -606,6 +606,8 @@ mlx5_rxq_check_vec_support(struct mlx5_rxq_data *rxq)if (!RXQ_PORT (ctrl)->config.rx_vec_en || rxq->sges_n !=0 )return -ENOTSUP ; +if (mlx5_shared_rq_enabled (RXQ_DEV (ctrl))) +return -ENOTSUP ;if (rxq->lro)return -ENOTSUP ;return 1 ;
Signature · 1 line
2.47.3
Hi,
Quoted 18 lines · Dariusz Sosnowski
Affected patch introduced a new Rx burst function to mlx5 PMD for handling out of order completions which are possible on shared Rx queues.
This burst function is selected if and only if at least one of the Rx queues on the device is configured as shared.This burst function contains only non-vectorized code and is not compatible with multi packet Rx (MPRQ). Although the function selection worked correctly, if either vectorized Rx datapath was supported or MPRQ was enabled, the relevant resources were still initialized. For example, additional fake mbufs were set up for vectorized Rx leading to memory leaks.
This patch fixes that by adding additional checks for shared Rx queues to vectorized and MPRQ support checks.
Shared Rx queue limitations and Rx burst functions info is also amended in mlx5 PMD documentation.
Quoted 6 lines
Fixes 5f9223611f35 ("net/mlx5: fix out-of-order completions in ordinary Rx burst")Signed-off-by Dariusz Sosnowski <[email protected]>
Patch applied to next-net-mlx,
Kindest regards
Raslan Darawsheh