Skip to contentSkip to conversation

net/mlx5/hws: fix send queue drain on FW WQE destroy

PATCH In review Maayan Kashani 4 participants 4 messages Apr 27, 2025 – May 13, 2025 Network drivers

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 net/mlx5/hws: fix send queue drain on FW WQE destroy

Queue sync operation was skipped on rule destroy. Unlike on fw wqe rule create in which both fence and notify_hw are set to true, on destroy fence was set to false causing previous queue operation to be stuck in the queue forever. Example:
rule_a - HW rule, rule_b - FW WQE rule.
Sequence:
rule_a destroy, burst=1 (HW rule put to queue but no DB) rule_b destroy, burst=0 (FW WQE rule cmd but no queue sync)
Outcome:
rule_a is stuck forever in the queue - no completion.

Fixes 338aaf911665 ("net/mlx5/hws: add send FW match STE using gen WQE")
---
 drivers/net/mlx5/hws/mlx5dr_send.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/mlx5/hws/mlx5dr_send.c +1 -1
index e121c7f7ed5..d01fc7ef2ca 100644
@@ -339,7 +339,7 @@ void mlx5dr_send_stes_fw(struct mlx5dr_send_engine *queue,
 	pdn = ctx->pd_num;
 
 	/* Writing through FW can't HW fence, therefore we drain the queue */
-	if (send_attr->fence)
+	if (send_attr->fence || send_attr->notify_hw)
 		mlx5dr_send_queue_action(ctx,
 					 queue_id,
 					 MLX5DR_SEND_QUEUE_ACTION_DRAIN_SYNC);
Signature · 1 line
2.21.0
#1 lore raw

Recheck-request: iol-intel-Performance

Looks like this patch was affected by an infra failure - putting in a retest request for the CI testing system.

Quoted 42 lines · Maayan Kashani

Queue sync operation was skipped on rule destroy. Unlike on fw wqe rule create in which both fence and notify_hw are set to true, on destroy fence was set to false causing previous queue operation to be stuck in the queue forever. Example:
rule_a - HW rule, rule_b - FW WQE rule.
Sequence:
rule_a destroy, burst=1 (HW rule put to queue but no DB) rule_b destroy, burst=0 (FW WQE rule cmd but no queue sync)
Outcome:
rule_a is stuck forever in the queue - no completion.

Fixes 338aaf911665 ("net/mlx5/hws: add send FW match STE using gen WQE")
---
 drivers/net/mlx5/hws/mlx5dr_send.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/mlx5/hws/mlx5dr_send.c +1 -1
index e121c7f7ed5..d01fc7ef2ca 100644
@@ -339,7 +339,7 @@ void mlx5dr_send_stes_fw(struct mlx5dr_send_engine
 *queue,
        pdn = ctx->pd_num;
 
        /* Writing through FW can't HW fence, therefore we drain the queue
 */
-       if (send_attr->fence)
+       if (send_attr->fence || send_attr->notify_hw)
                mlx5dr_send_queue_action(ctx,

queue_id,

MLX5DR_SEND_QUEUE_ACTION_DRAIN_SYNC);
--
2.21.0

Acked-by on v1 1/1
#2 lore raw
In reply to Maayan Kashani

Hi,

Quoted 50 lines

-----Original Message-----
From: Maayan Kashani <[email protected]>
Sent: Sunday, April 27, 2025 7:28 PM
To: [email protected]
Cc: Maayan Kashani <[email protected]>; Dariusz Sosnowski <[email protected]>; Raslan Darawsheh <[email protected]>; [email protected]; Alex Vesker <[email protected]>; Slava Ovsiienko <[email protected]>; Bing Zhao <[email protected]>; Ori Kam <[email protected]>; Suanming Mou <[email protected]>; Matan Azrad <[email protected]>
Subject: [PATCH] net/mlx5/hws: fix send queue drain on FW WQE destroy

Queue sync operation was skipped on rule destroy.
Unlike on fw wqe rule create in which both fence and notify_hw are set to true, on destroy fence was set to false causing previous queue operation to be stuck in the queue forever.
Example:
rule_a - HW rule, rule_b - FW WQE rule.
Sequence:
rule_a destroy, burst=1 (HW rule put to queue but no DB) rule_b destroy, burst=0 (FW WQE rule cmd but no queue sync)
Outcome:
rule_a is stuck forever in the queue - no completion.

Fixes 338aaf911665 ("net/mlx5/hws: add send FW match STE using gen WQE")
---
 drivers/net/mlx5/hws/mlx5dr_send.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/mlx5/hws/mlx5dr_send.c +1 -1
index e121c7f7ed5..d01fc7ef2ca 100644
@@ -339,7 +339,7 @@ void mlx5dr_send_stes_fw(struct mlx5dr_send_engine
 *queue,
 	pdn = ctx->pd_num;
 
 	/* Writing through FW can't HW fence, therefore we drain the queue
 */
-	if (send_attr->fence)
+	if (send_attr->fence || send_attr->notify_hw)
 		mlx5dr_send_queue_action(ctx,
drivers/net/mlx5/hws/mlx5dr_send.c +0 -1
 					 queue_id,
 					 MLX5DR_SEND_QUEUE_ACTION_DRAIN_SYNC);
--
Quoted 1 line

2.21.0

Thanks

#3 lore raw
In reply to Maayan Kashani

Hi,

Quoted 12 lines · Maayan Kashani

Queue sync operation was skipped on rule destroy. Unlike on fw wqe rule create in which both fence and notify_hw are set to true, on destroy fence was set to false causing previous queue operation to be stuck in the queue forever. Example:
rule_a - HW rule, rule_b - FW WQE rule.
Sequence:
rule_a destroy, burst=1 (HW rule put to queue but no DB) rule_b destroy, burst=0 (FW WQE rule cmd but no queue sync)
Outcome:
rule_a is stuck forever in the queue - no completion.

Quoted 6 lines
Fixes 338aaf911665 ("net/mlx5/hws: add send FW match STE using gen WQE")

---

Patch applied to next-net-mlx,

Signature · 2 lines
Kindest regards
Raslan Darawsheh