Skip to contentSkip to conversation

net/mlx5: fix shared Rx queue limitations

PATCH In review Dariusz Sosnowski 2 participants 2 messages Feb 18 – Feb 24 Network drivers

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 net/mlx5: fix shared Rx queue limitations

Affected patch introduced a new Rx burst function to mlx5 PMD for handling out of order completions which are possible on shared Rx queues.
This burst function is selected if and only if at least one of the Rx queues on the device is configured as shared.

This burst function contains only non-vectorized code and is not compatible with multi packet Rx (MPRQ). Although the function selection worked correctly, if either vectorized Rx datapath was supported or MPRQ was enabled, the relevant resources were still initialized. For example, additional fake mbufs were set up for vectorized Rx leading to memory leaks.

This patch fixes that by adding additional checks for shared Rx queues to vectorized and MPRQ support checks.
Shared Rx queue limitations and Rx burst functions info is also amended in mlx5 PMD documentation.

Fixes 5f9223611f35 ("net/mlx5: fix out-of-order completions in ordinary Rx burst")
---
 doc/guides/nics/mlx5.rst         | 41 +++++++++++++++++++-------------
 drivers/net/mlx5/mlx5_rx.h       | 36 +++++++++++++++-------------
 drivers/net/mlx5/mlx5_rxtx_vec.c |  2 ++
 3 files changed, 46 insertions(+), 33 deletions(-)
doc/guides/nics/mlx5.rst +25 -16
index 2529c2f4c8..a1a428dd69 100644
@@ -956,27 +956,32 @@ The Rx function is selected based on multiple parameters:
 - :ref:`multi-packet Rx queues (MPRQ) <mlx5_mprq_params>`
 - :ref:`vectorized Rx datapath <mlx5_rx_vec_param>`
 
-This parameter may also have an impact on the behavior:
+These configurations may also have an impact on the behavior:
 
 - :ref:`packet descriptor (CQE) compression <mlx5_cqe_comp_param>`
+- :ref:`shared Rx queue <mlx5_shared_rx>`
 
 .. table:: Rx burst functions
 
-   +-------------------+------------------------+---------+-----------------+------+-------+
-   || Function Name    || Parameters to Enable  || Scatter|| Error Recovery || CQE || Large|
-   |                   |                        |         |                 || comp|| MTU  |
-   +===================+========================+=========+=================+======+=======+
-   | rx_burst          | rx_vec_en=0            |   Yes   | Yes             |  Yes |  Yes  |
-   +-------------------+------------------------+---------+-----------------+------+-------+
-   | rx_burst_vec      | rx_vec_en=1 (default)  |   No    | if CQE comp off |  Yes |  No   |
-   +-------------------+------------------------+---------+-----------------+------+-------+
-   | rx_burst_mprq     || mprq_en=1             |   No    | Yes             |  Yes |  Yes  |
-   |                   || RxQs >= rxqs_min_mprq |         |                 |      |       |
-   +-------------------+------------------------+---------+-----------------+------+-------+
-   | rx_burst_mprq_vec || rx_vec_en=1 (default) |   No    | if CQE comp off |  Yes |  Yes  |
-   |                   || mprq_en=1             |         |                 |      |       |
-   |                   || RxQs >= rxqs_min_mprq |         |                 |      |       |
-   +-------------------+------------------------+---------+-----------------+------+-------+
+   +-------------------+------------------------+---------+-----------------+------+-------+---------+
+   || Function Name    || Parameters to Enable  || Scatter|| Error Recovery || CQE || Large|| Shared |
+   |                   |                        |         |                 || comp|| MTU  |  RXQ    |
+   +===================+========================+=========+=================+======+=======+=========+
+   | rx_burst          | rx_vec_en=0            |   Yes   | Yes             |  Yes |  Yes  | No      |
+   +-------------------+------------------------+---------+-----------------+------+-------+---------+
+   | rx_burst_vec      | rx_vec_en=1 (default)  |   No    | if CQE comp off |  Yes |  No   | No      |
+   +-------------------+------------------------+---------+-----------------+------+-------+---------+
+   | rx_burst_mprq     || mprq_en=1             |   No    | Yes             |  Yes |  Yes  | No      |
+   |                   || RxQs >= rxqs_min_mprq |         |                 |      |       |         |
+   +-------------------+------------------------+---------+-----------------+------+-------+---------+
+   | rx_burst_mprq_vec || rx_vec_en=1 (default) |   No    | if CQE comp off |  Yes |  Yes  | No      |
+   |                   || mprq_en=1             |         |                 |      |       |         |
+   |                   || RxQs >= rxqs_min_mprq |         |                 |      |       |         |
+   +-------------------+------------------------+---------+-----------------+------+-------+---------+
+   | rx_burst (OoO)    | at least one Rx queue  |   Yes   | Yes             |  Yes |  Yes  | Yes     |
+   |                   | on the device          |         |                 |      |       |         |
+   |                   | is shared              |         |                 |      |       |         |
+   +-------------------+------------------------+---------+-----------------+------+-------+---------+
 
 
 Rx/Tx Tuning
@@ -1829,6 +1834,10 @@ Shared Rx Queue
 Limitations
 ^^^^^^^^^^^
 
+#. Shared Rx queue is not compatible with both vectorized Rx datapath and multi-packet Rx queues.
+   Any configuration related to these features passed through device arguments
+   will be ignored.
+
 #. Counters of received packets and bytes of devices in the same share group are same.
 
 #. Counters of received packets and bytes of queues in the same group and queue ID are same.
drivers/net/mlx5/mlx5_rx.h +19 -17
index 86636d598f..dffab3955b 100644
@@ -602,6 +602,23 @@ mprq_buf_to_pkt(struct mlx5_rxq_data *rxq, struct rte_mbuf *pkt, uint32_t len,
 	return MLX5_RXQ_CODE_EXIT;
 }
 
+/**
+ * Check whether Shared RQ is enabled for the device.
+ *
+ * @param dev
+ *   Pointer to Ethernet device.
+ *
+ * @return
+ *   0 if disabled, otherwise enabled.
+ */
+static __rte_always_inline int
+mlx5_shared_rq_enabled(struct rte_eth_dev *dev)
+{
+	struct mlx5_priv *priv = dev->data->dev_private;
+
+	return !LIST_EMPTY(&priv->sh->shared_rxqs);
+}
+
 /**
  * Check whether Multi-Packet RQ can be enabled for the device.
  *
@@ -616,6 +633,8 @@ mlx5_check_mprq_support(struct rte_eth_dev *dev)
 {
 	struct mlx5_priv *priv = dev->data->dev_private;
 
+	if (mlx5_shared_rq_enabled(dev))
+		return -ENOTSUP;
 	if (priv->config.mprq.enabled &&
 	    priv->rxqs_n >= priv->config.mprq.min_rxqs_num)
 		return 1;
@@ -671,23 +690,6 @@ mlx5_mprq_enabled(struct rte_eth_dev *dev)
 	return n == n_ibv;
 }
 
-/**
- * Check whether Shared RQ is enabled for the device.
- *
- * @param dev
- *   Pointer to Ethernet device.
- *
- * @return
- *   0 if disabled, otherwise enabled.
- */
-static __rte_always_inline int
-mlx5_shared_rq_enabled(struct rte_eth_dev *dev)
-{
-	struct mlx5_priv *priv = dev->data->dev_private;
-
-	return !LIST_EMPTY(&priv->sh->shared_rxqs);
-}
-
 /**
  * Check whether given RxQ is external.
  *
drivers/net/mlx5/mlx5_rxtx_vec.c +2 -0
index 1b701801c5..79e92811d5 100644
@@ -606,6 +606,8 @@ mlx5_rxq_check_vec_support(struct mlx5_rxq_data *rxq)
 
 	if (!RXQ_PORT(ctrl)->config.rx_vec_en || rxq->sges_n != 0)
 		return -ENOTSUP;
+	if (mlx5_shared_rq_enabled(RXQ_DEV(ctrl)))
+		return -ENOTSUP;
 	if (rxq->lro)
 		return -ENOTSUP;
 	return 1;
Signature · 1 line
2.47.3
#1 lore raw

Hi,

Quoted 18 lines · Dariusz Sosnowski

Affected patch introduced a new Rx burst function to mlx5 PMD for handling out of order completions which are possible on shared Rx queues.
This burst function is selected if and only if at least one of the Rx queues on the device is configured as shared.

This burst function contains only non-vectorized code and is not compatible with multi packet Rx (MPRQ). Although the function selection worked correctly, if either vectorized Rx datapath was supported or MPRQ was enabled, the relevant resources were still initialized. For example, additional fake mbufs were set up for vectorized Rx leading to memory leaks.

This patch fixes that by adding additional checks for shared Rx queues to vectorized and MPRQ support checks.
Shared Rx queue limitations and Rx burst functions info is also amended in mlx5 PMD documentation.

Quoted 6 lines
Fixes 5f9223611f35 ("net/mlx5: fix out-of-order completions in ordinary Rx burst")

Patch applied to next-net-mlx,

Kindest regards
Raslan Darawsheh