Skip to contentSkip to conversation

net/mlx5: fix data access race condition for shared Rx queue

PATCH In review Jiawei Wang 2 participants 2 messages Jul 5, 2024 – Jul 18, 2024 Network drivers

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 net/mlx5: fix data access race condition for shared Rx queue

The rxq_data resources were shared for shared Rx queue with the same group and queue ID.
The cq_ci:24 of rxq_data was unalignment with other fields in the one 32-bit data, like the dynf_meta and delay_drop.

32bit: xxxx xxxI IIII IIII IIII IIII IIII IIIx
^ .... .... .... .... ...^
| cq_ci |

The issue is that while the control thread updates the dynf_meta:1 or delay_drop:1 value during port start, another data thread updates the cq_ci at the same time, it causes the bytes race condition with different thread, and cq_ci value may be overwritten and updated the abnormal value into HW CQ DB.

This patch separates the cq_ci from the configuration data spaces, and adds checking for delay_drop and dynf_meta if shared Rx queue if started.

Fixes 02a6195cbe ("net/mlx5: support enhanced CQE compression in Rx burst")
---
 drivers/net/mlx5/mlx5_devx.c |  3 ++-
 drivers/net/mlx5/mlx5_flow.c | 24 +++++++++++++-----------
 drivers/net/mlx5/mlx5_rx.h   |  4 ++--
 3 files changed, 17 insertions(+), 14 deletions(-)
drivers/net/mlx5/mlx5_devx.c +2 -1
index 7db271acb4..8ebe784000 100644
@@ -684,7 +684,8 @@ mlx5_rxq_devx_obj_new(struct mlx5_rxq_priv *rxq)
 		DRV_LOG(ERR, "Failed to create CQ.");
 		goto error;
 	}
-	rxq_data->delay_drop = priv->config.std_delay_drop;
+	if (!rxq_data->shared || !rxq_ctrl->started)
+		rxq_data->delay_drop = priv->config.std_delay_drop;
 	/* Create RQ using DevX API. */
 	ret = mlx5_rxq_create_devx_rq_resources(rxq);
 	if (ret) {
drivers/net/mlx5/mlx5_flow.c +13 -11
index 5b3f2b9119..72fb3a55ba 100644
@@ -1853,18 +1853,20 @@ mlx5_flow_rxq_dynf_set(struct rte_eth_dev *dev)
 		if (rxq == NULL || rxq->ctrl == NULL)
 			continue;
 		data = &rxq->ctrl->rxq;
-		if (!rte_flow_dynf_metadata_avail()) {
-			data->dynf_meta = 0;
-			data->flow_meta_mask = 0;
-			data->flow_meta_offset = -1;
-			data->flow_meta_port_mask = 0;
-		} else {
-			data->dynf_meta = 1;
-			data->flow_meta_mask = rte_flow_dynf_metadata_mask;
-			data->flow_meta_offset = rte_flow_dynf_metadata_offs;
-			data->flow_meta_port_mask = priv->sh->dv_meta_mask;
+		if (!data->shared || !rxq->ctrl->started) {
+			if (!rte_flow_dynf_metadata_avail()) {
+				data->dynf_meta = 0;
+				data->flow_meta_mask = 0;
+				data->flow_meta_offset = -1;
+				data->flow_meta_port_mask = 0;
+			} else {
+				data->dynf_meta = 1;
+				data->flow_meta_mask = rte_flow_dynf_metadata_mask;
+				data->flow_meta_offset = rte_flow_dynf_metadata_offs;
+				data->flow_meta_port_mask = priv->sh->dv_meta_mask;
+			}
+			data->mark_flag = mark_flag;
 		}
-		data->mark_flag = mark_flag;
 	}
 }
 
drivers/net/mlx5/mlx5_rx.h +2 -2
index 1485556d89..7d144921ab 100644
@@ -101,14 +101,14 @@ struct __rte_cache_aligned mlx5_rxq_data {
 	unsigned int shared:1; /* Shared RXQ. */
 	unsigned int delay_drop:1; /* Enable delay drop. */
 	unsigned int cqe_comp_layout:1; /* CQE Compression Layout*/
-	unsigned int cq_ci:24;
+	uint16_t port_id;
 	volatile uint32_t *rq_db;
 	volatile uint32_t *cq_db;
-	uint16_t port_id;
 	uint32_t elts_ci;
 	uint32_t rq_ci;
 	uint16_t consumed_strd; /* Number of consumed strides in WQE. */
 	uint32_t rq_pi;
+	uint32_t cq_ci:24;
 	uint16_t rq_repl_thresh; /* Threshold for buffer replenishment. */
 	uint32_t byte_mask;
 	union {
Signature · 1 line
2.18.1
Acked-by Bing Zhao Acked-by Viacheslav Ovsiienko on v1 1/1
#1 lore raw

Hi,

From: Jiawei(Jonny) Wang <[email protected]>
Sent: Friday, July 5, 2024 4:05 PM
To: Bing Zhao; Slava Ovsiienko; Dariusz Sosnowski; Ori Kam; Suanming Mou; Matan Azrad; Alexander Kozyrev
Cc: [email protected]; Raslan Darawsheh; [email protected]
Subject: [PATCH] net/mlx5: fix data access race condition for shared Rx queue

The rxq_data resources were shared for shared Rx queue with the same group and queue ID.
The cq_ci:24 of rxq_data was unalignment with other fields in the one 32-bit data, like the dynf_meta and delay_drop.

32bit: xxxx xxxI IIII IIII IIII IIII IIII IIIx
^ .... .... .... .... ...^
| cq_ci |

The issue is that while the control thread updates the dynf_meta:1 or delay_drop:1 value during port start, another data thread updates the cq_ci at the same time, it causes the bytes race condition with different thread, and cq_ci value may be overwritten and updated the abnormal value into HW CQ DB.

This patch separates the cq_ci from the configuration data spaces, and adds checking for delay_drop and dynf_meta if shared Rx queue if started.

Fixes 02a6195cbe ("net/mlx5: support enhanced CQE compression in Rx burst")

Patch applied to next-net-mlx,

Kindest regards,
Raslan Darawsheh