Skip to contentSkip to conversation

net/mlx5: fix Tx queues creation type check for scheduling

PATCH In review Viacheslav Ovsiienko 3 participants 3 messages Jul 29, 2021 Network drivers

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 net/mlx5: fix Tx queues creation type check for scheduling

The send scheduling on timestamp offload requires the Send Queue (SQ) shares its User Access Region (UAR) with the pacing Clock Queue. The SQ can be created by mlx5 PMD either with DevX or with Verbs. If the SQ is being created with DevX the dedicated UAR can be specified and all the SQs share the single UAR. Once SQ is being created with Verbs the SQ's UAR is allocated by the rdma-core library internally on its own and there are no UAR sharing. This caused hardware errors on WAIT WQEs and overall send scheduling malfunction.

If SQs are going to be created with Verbs and the send scheduling offload is explicitly requested via tx_pp devarg the device probing is rejected as device configuration can't satisfy the requirements.

Fixes 3ec73abeed52 ("net/mlx5/linux: fix Tx queue operations decision")
Fixes 8f848f32fc24 ("net/mlx5: introduce send scheduling devargs")
---
 drivers/net/mlx5/linux/mlx5_os.c | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)
drivers/net/mlx5/linux/mlx5_os.c +18 -0
index aa5210fa45..e3c949ffc8 100644
@@ -1769,6 +1769,24 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
 	} else {
 		priv->obj_ops = ibv_obj_ops;
 	}
+	if (config->tx_pp &&
+	    (priv->config.dv_esw_en ||
+	     priv->obj_ops.txq_obj_new != mlx5_os_txq_obj_new)) {
+		/*
+		 * HAVE_MLX5DV_DEVX_UAR_OFFSET is required to support
+		 * packet pacing and already checked above. Hence, we should
+		 * only make sure the SQs will be created with DevX, not with
+		 * Verbs. Verbs allocates the SQ UAR on its own and it can't
+		 * be shared with Clock Queue UAR as it required for the
+		 * Tx scheduling feature.
+		 */
+		DRV_LOG(ERR, "Verbs SQs, UAR can't be shared"
+			     " as required for packet pacing");
+			err = ENODEV;
+			goto error;
+		err = ENODEV;
+		goto error;
+	}
 	priv->drop_queue.hrxq = mlx5_drop_action_create(eth_dev);
 	if (!priv->drop_queue.hrxq)
 		goto error;
Signature · 1 line
2.18.1
#1 lore raw

Hi,

Quoted 23 lines

-----Original Message-----
From: Slava Ovsiienko <[email protected]>
Sent: Thursday, July 29, 2021 3:27 PM
To: [email protected]
Cc: Raslan Darawsheh <[email protected]>; Matan Azrad <[email protected]>; [email protected]
Subject: [PATCH] net/mlx5: fix Tx queues creation type check for scheduling

The send scheduling on timestamp offload requires the Send Queue (SQ) shares its User Access Region (UAR) with the pacing Clock Queue. The SQ can be created by mlx5 PMD either with DevX or with Verbs. If the SQ is being created with DevX the dedicated UAR can be specified and all the SQs share the single UAR. Once SQ is being created with Verbs the SQ's UAR is allocated by the rdma-core library internally on its own and there are no UAR sharing. This caused hardware errors on WAIT WQEs and overall send scheduling malfunction.

If SQs are going to be created with Verbs and the send scheduling offload is explicitly requested via tx_pp devarg the device probing is rejected as device configuration can't satisfy the requirements.

Quoted 5 lines
Fixes 3ec73abeed52 ("net/mlx5/linux: fix Tx queue operations decision")
Fixes 8f848f32fc24 ("net/mlx5: introduce send scheduling devargs")

Patch applied to next-net-mlx,

Kindest regards,
Raslan Darawsheh

#2 lore raw
In reply to Viacheslav Ovsiienko

29/07/2021 14:26, Viacheslav Ovsiienko:

drivers/net/mlx5/linux/mlx5_os.c mlx5_dev_spawn(struct rte_device *dpdk_dev, · line 1784

Quoted 13 lines
drivers/net/mlx5/linux/mlx5_os.c +13 -0
+	if (config->tx_pp &&
+	    (priv->config.dv_esw_en ||
+	     priv->obj_ops.txq_obj_new != mlx5_os_txq_obj_new)) {
+		/*
+		 * HAVE_MLX5DV_DEVX_UAR_OFFSET is required to support
+		 * packet pacing and already checked above. Hence, we should
+		 * only make sure the SQs will be created with DevX, not with
+		 * Verbs. Verbs allocates the SQ UAR on its own and it can't
+		 * be shared with Clock Queue UAR as it required for the
+		 * Tx scheduling feature.
+		 */
+		DRV_LOG(ERR, "Verbs SQs, UAR can't be shared"
+			     " as required for packet pacing");

Don't split logs.

drivers/net/mlx5/linux/mlx5_os.c mlx5_dev_spawn(struct rte_device *dpdk_dev, · line 1788

Quoted 4 lines
drivers/net/mlx5/linux/mlx5_os.c +4 -0
+			err = ENODEV;
+			goto error;
+		err = ENODEV;
+		goto error;

I assume only the last 2 lines should be kept.