net/mlx5: fix Tx queues creation type check for scheduling
PATCH 1/1 net/mlx5: fix Tx queues creation type check for scheduling
The send scheduling on timestamp offload requires the Send Queue (SQ) shares its User Access Region (UAR) with the pacing Clock Queue. The SQ can be created by mlx5 PMD either with DevX or with Verbs. If the SQ is being created with DevX the dedicated UAR can be specified and all the SQs share the single UAR. Once SQ is being created with Verbs the SQ's UAR is allocated by the rdma-core library internally on its own and there are no UAR sharing. This caused hardware errors on WAIT WQEs and overall send scheduling malfunction.
If SQs are going to be created with Verbs and the send scheduling offload is explicitly requested via tx_pp devarg the device probing is rejected as device configuration can't satisfy the requirements.
--- drivers/net/mlx5/linux/mlx5_os.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+)
drivers/net/mlx5/linux/mlx5_os.c +18 -0
@@ -1769,6 +1769,24 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, }else { priv->obj_ops = ibv_obj_ops; } +if (config->tx_pp && + (priv->config.dv_esw_en || + priv->obj_ops.txq_obj_new != mlx5_os_txq_obj_new)) { +/* +* HAVE_MLX5DV_DEVX_UAR_OFFSET is required to support +* packet pacing and already checked above. Hence, we should +* only make sure the SQs will be created with DevX, not with +* Verbs. Verbs allocates the SQ UAR on its own and it can't +* be shared with Clock Queue UAR as it required for the +* Tx scheduling feature. +*/ +DRV_LOG (ERR ,"Verbs SQs, UAR can't be shared" +" as required for packet pacing" ); + err =ENODEV ; + goto error; + err =ENODEV ; + goto error; + } priv->drop_queue.hrxq =mlx5_drop_action_create (eth_dev);if (!priv->drop_queue.hrxq) goto error;
Signature · 1 line
2.18.1
Hi,
Quoted 23 lines
-----Original Message-----
From: Slava Ovsiienko <[email protected]>
Sent: Thursday, July 29, 2021 3:27 PM
To: [email protected]
Cc: Raslan Darawsheh <[email protected]>; Matan Azrad <[email protected]>; [email protected]
Subject: [PATCH] net/mlx5: fix Tx queues creation type check for schedulingThe send scheduling on timestamp offload requires the Send Queue (SQ) shares its User Access Region (UAR) with the pacing Clock Queue. The SQ can be created by mlx5 PMD either with DevX or with Verbs. If the SQ is being created with DevX the dedicated UAR can be specified and all the SQs share the single UAR. Once SQ is being created with Verbs the SQ's UAR is allocated by the rdma-core library internally on its own and there are no UAR sharing. This caused hardware errors on WAIT WQEs and overall send scheduling malfunction.
If SQs are going to be created with Verbs and the send scheduling offload is explicitly requested via
tx_ppdevarg the device probing is rejected as device configuration can't satisfy the requirements.
Quoted 5 lines
Fixes 3ec73abeed52 ("net/mlx5/linux: fix Tx queue operations decision")Fixes 8f848f32fc24 ("net/mlx5: introduce send scheduling devargs")Signed-off-by Viacheslav Ovsiienko <[email protected]>
Patch applied to next-net-mlx,
Kindest regards,
Raslan Darawsheh
29/07/2021 14:26, Viacheslav Ovsiienko:
drivers/net/mlx5/linux/mlx5_os.c mlx5_dev_spawn(struct rte_device *dpdk_dev, · line 1784
Quoted 13 lines
drivers/net/mlx5/linux/mlx5_os.c +13
-0+if (config->tx_pp && + (priv->config.dv_esw_en || + priv->obj_ops.txq_obj_new != mlx5_os_txq_obj_new)) { +/* +* HAVE_MLX5DV_DEVX_UAR_OFFSET is required to support +* packet pacing and already checked above. Hence, we should +* only make sure the SQs will be created with DevX, not with +* Verbs. Verbs allocates the SQ UAR on its own and it can't +* be shared with Clock Queue UAR as it required for the +* Tx scheduling feature. +*/ +DRV_LOG (ERR ,"Verbs SQs, UAR can't be shared" +" as required for packet pacing" );
Don't split logs.
drivers/net/mlx5/linux/mlx5_os.c mlx5_dev_spawn(struct rte_device *dpdk_dev, · line 1788
Quoted 4 lines
drivers/net/mlx5/linux/mlx5_os.c +4
-0+ err =ENODEV ; + goto error; + err =ENODEV ; + goto error;
I assume only the last 2 lines should be kept.