Skip to contentSkip to conversation

net/mlx5: E-Switch and validation fixes

PATCH In review Dariusz Sosnowski 2 participants 5 messages Jul 18, 2024 – Jul 21, 2024 Network drivers

lore
Share & tools
lore
#0 lore raw

cover net/mlx5: E-Switch and validation fixes

Patch 1 - Fixes a bug with fdb_def_rule_en device argument, used to control default E-Switch flow rules created by mlx5 PMD.

Patches 2-3 - Fixes for flow rule validation in async flow API, which were found during testing.

Dariusz Sosnowski (3):
net/mlx5: fix disabling E-Switch default flow rules net/mlx5: fix action configuration validation net/mlx5: fix RSS and queue action validation

 drivers/net/mlx5/mlx5_flow_hw.c | 182 +++++++++++++++++---------------
 drivers/net/mlx5/mlx5_trigger.c |   4 +-
 drivers/net/mlx5/mlx5_txq.c     |  13 ++-
 3 files changed, 107 insertions(+), 92 deletions(-)

--
2.39.2

#1 lore raw

PATCH 1/3 net/mlx5: fix disabling E-Switch default flow rules

fdb_def_rule_en devarg controls whether mlx5 PMD creates default E-Switch flow rules for:

- Transferring traffic from wire, VFs and SFs to group 1 (default jump).
- Providing default behavior for application traffic (default SQ miss flow rules).

With these flow rules, applications effectively create transfer flow rules in group 1 and higher (application group is translated to one higher) allowing for faster insertion on all groups and providing ability to forward to VF, SF and wire on any group.

By default, these rules are created (fdb_def_rule_en == 1).

When these default flow rules are disabled (fdb_def_rule_en == 0) with HW Steering flow engine (dv_flow_en == 2) only creation of default jump rules was disabled. Also, necessary template table and pattern/actions templates were created as well,
but they were never used.
SQ miss flow rules were still created.
This is a bug, because with fdb_def_rule_en == 0, application should not expect any default E-Switch flow rules.

This patch fixes that by disabling all default E-Switch flow rules creation and disabling creating templates for these flow rules, when fdb_def_rule_en == 0.
If an application needs to run with these flow rules disabled, and requires flow rules providing SQ miss flow rules functionality, then application must explicitly create similar flow rules.

Fixes 1939eb6f660c ("net/mlx5: support flow port action with HWS")
---
 drivers/net/mlx5/mlx5_flow_hw.c | 142 ++++++++++++++++++--------------
 drivers/net/mlx5/mlx5_trigger.c |   4 +-
 drivers/net/mlx5/mlx5_txq.c     |  13 ++-
 3 files changed, 91 insertions(+), 68 deletions(-)
drivers/net/mlx5/mlx5_flow_hw.c +78 -64
index e7d8c251a0..fe7df7305f 100644
@@ -10580,6 +10580,7 @@ flow_hw_create_ctrl_tables(struct rte_eth_dev *dev, struct rte_flow_error *error
 	struct mlx5_flow_hw_ctrl_fdb *hw_ctrl_fdb;
 	uint32_t xmeta = priv->sh->config.dv_xmeta_en;
 	uint32_t repr_matching = priv->sh->config.repr_matching;
+	uint32_t fdb_def_rule = priv->sh->config.fdb_def_rule;
 
 	MLX5_ASSERT(priv->hw_ctrl_fdb == NULL);
 	hw_ctrl_fdb = mlx5_malloc(MLX5_MEM_ZERO, sizeof(*hw_ctrl_fdb), 0, SOCKET_ID_ANY);
@@ -10590,70 +10591,79 @@ flow_hw_create_ctrl_tables(struct rte_eth_dev *dev, struct rte_flow_error *error
 		goto err;
 	}
 	priv->hw_ctrl_fdb = hw_ctrl_fdb;
-	/* Create templates and table for default SQ miss flow rules - root table. */
-	hw_ctrl_fdb->esw_mgr_items_tmpl = flow_hw_create_ctrl_esw_mgr_pattern_template(dev, error);
-	if (!hw_ctrl_fdb->esw_mgr_items_tmpl) {
-		DRV_LOG(ERR, "port %u failed to create E-Switch Manager item"
-			" template for control flows", dev->data->port_id);
-		goto err;
-	}
-	hw_ctrl_fdb->regc_jump_actions_tmpl = flow_hw_create_ctrl_regc_jump_actions_template
-			(dev, error);
-	if (!hw_ctrl_fdb->regc_jump_actions_tmpl) {
-		DRV_LOG(ERR, "port %u failed to create REG_C set and jump action template"
-			" for control flows", dev->data->port_id);
-		goto err;
-	}
-	hw_ctrl_fdb->hw_esw_sq_miss_root_tbl = flow_hw_create_ctrl_sq_miss_root_table
-			(dev, hw_ctrl_fdb->esw_mgr_items_tmpl, hw_ctrl_fdb->regc_jump_actions_tmpl,
-			 error);
-	if (!hw_ctrl_fdb->hw_esw_sq_miss_root_tbl) {
-		DRV_LOG(ERR, "port %u failed to create table for default sq miss (root table)"
-			" for control flows", dev->data->port_id);
-		goto err;
-	}
-	/* Create templates and table for default SQ miss flow rules - non-root table. */
-	hw_ctrl_fdb->regc_sq_items_tmpl = flow_hw_create_ctrl_regc_sq_pattern_template(dev, error);
-	if (!hw_ctrl_fdb->regc_sq_items_tmpl) {
-		DRV_LOG(ERR, "port %u failed to create SQ item template for"
-			" control flows", dev->data->port_id);
-		goto err;
-	}
-	hw_ctrl_fdb->port_actions_tmpl = flow_hw_create_ctrl_port_actions_template(dev, error);
-	if (!hw_ctrl_fdb->port_actions_tmpl) {
-		DRV_LOG(ERR, "port %u failed to create port action template"
-			" for control flows", dev->data->port_id);
-		goto err;
-	}
-	hw_ctrl_fdb->hw_esw_sq_miss_tbl = flow_hw_create_ctrl_sq_miss_table
-			(dev, hw_ctrl_fdb->regc_sq_items_tmpl, hw_ctrl_fdb->port_actions_tmpl,
-			 error);
-	if (!hw_ctrl_fdb->hw_esw_sq_miss_tbl) {
-		DRV_LOG(ERR, "port %u failed to create table for default sq miss (non-root table)"
-			" for control flows", dev->data->port_id);
-		goto err;
-	}
-	/* Create templates and table for default FDB jump flow rules. */
-	hw_ctrl_fdb->port_items_tmpl = flow_hw_create_ctrl_port_pattern_template(dev, error);
-	if (!hw_ctrl_fdb->port_items_tmpl) {
-		DRV_LOG(ERR, "port %u failed to create SQ item template for"
-			" control flows", dev->data->port_id);
-		goto err;
-	}
-	hw_ctrl_fdb->jump_one_actions_tmpl = flow_hw_create_ctrl_jump_actions_template
-			(dev, MLX5_HW_LOWEST_USABLE_GROUP, error);
-	if (!hw_ctrl_fdb->jump_one_actions_tmpl) {
-		DRV_LOG(ERR, "port %u failed to create jump action template"
-			" for control flows", dev->data->port_id);
-		goto err;
-	}
-	hw_ctrl_fdb->hw_esw_zero_tbl = flow_hw_create_ctrl_jump_table
-			(dev, hw_ctrl_fdb->port_items_tmpl, hw_ctrl_fdb->jump_one_actions_tmpl,
-			 error);
-	if (!hw_ctrl_fdb->hw_esw_zero_tbl) {
-		DRV_LOG(ERR, "port %u failed to create table for default jump to group 1"
-			" for control flows", dev->data->port_id);
-		goto err;
+	if (fdb_def_rule) {
+		/* Create templates and table for default SQ miss flow rules - root table. */
+		hw_ctrl_fdb->esw_mgr_items_tmpl =
+				flow_hw_create_ctrl_esw_mgr_pattern_template(dev, error);
+		if (!hw_ctrl_fdb->esw_mgr_items_tmpl) {
+			DRV_LOG(ERR, "port %u failed to create E-Switch Manager item"
+				" template for control flows", dev->data->port_id);
+			goto err;
+		}
+		hw_ctrl_fdb->regc_jump_actions_tmpl =
+				flow_hw_create_ctrl_regc_jump_actions_template(dev, error);
+		if (!hw_ctrl_fdb->regc_jump_actions_tmpl) {
+			DRV_LOG(ERR, "port %u failed to create REG_C set and jump action template"
+				" for control flows", dev->data->port_id);
+			goto err;
+		}
+		hw_ctrl_fdb->hw_esw_sq_miss_root_tbl =
+				flow_hw_create_ctrl_sq_miss_root_table
+					(dev, hw_ctrl_fdb->esw_mgr_items_tmpl,
+					 hw_ctrl_fdb->regc_jump_actions_tmpl, error);
+		if (!hw_ctrl_fdb->hw_esw_sq_miss_root_tbl) {
+			DRV_LOG(ERR, "port %u failed to create table for default sq miss (root table)"
+				" for control flows", dev->data->port_id);
+			goto err;
+		}
+		/* Create templates and table for default SQ miss flow rules - non-root table. */
+		hw_ctrl_fdb->regc_sq_items_tmpl =
+				flow_hw_create_ctrl_regc_sq_pattern_template(dev, error);
+		if (!hw_ctrl_fdb->regc_sq_items_tmpl) {
+			DRV_LOG(ERR, "port %u failed to create SQ item template for"
+				" control flows", dev->data->port_id);
+			goto err;
+		}
+		hw_ctrl_fdb->port_actions_tmpl =
+				flow_hw_create_ctrl_port_actions_template(dev, error);
+		if (!hw_ctrl_fdb->port_actions_tmpl) {
+			DRV_LOG(ERR, "port %u failed to create port action template"
+				" for control flows", dev->data->port_id);
+			goto err;
+		}
+		hw_ctrl_fdb->hw_esw_sq_miss_tbl =
+				flow_hw_create_ctrl_sq_miss_table
+					(dev, hw_ctrl_fdb->regc_sq_items_tmpl,
+					 hw_ctrl_fdb->port_actions_tmpl, error);
+		if (!hw_ctrl_fdb->hw_esw_sq_miss_tbl) {
+			DRV_LOG(ERR, "port %u failed to create table for default sq miss (non-root table)"
+				" for control flows", dev->data->port_id);
+			goto err;
+		}
+		/* Create templates and table for default FDB jump flow rules. */
+		hw_ctrl_fdb->port_items_tmpl =
+				flow_hw_create_ctrl_port_pattern_template(dev, error);
+		if (!hw_ctrl_fdb->port_items_tmpl) {
+			DRV_LOG(ERR, "port %u failed to create SQ item template for"
+				" control flows", dev->data->port_id);
+			goto err;
+		}
+		hw_ctrl_fdb->jump_one_actions_tmpl =
+				flow_hw_create_ctrl_jump_actions_template
+					(dev, MLX5_HW_LOWEST_USABLE_GROUP, error);
+		if (!hw_ctrl_fdb->jump_one_actions_tmpl) {
+			DRV_LOG(ERR, "port %u failed to create jump action template"
+				" for control flows", dev->data->port_id);
+			goto err;
+		}
+		hw_ctrl_fdb->hw_esw_zero_tbl = flow_hw_create_ctrl_jump_table
+				(dev, hw_ctrl_fdb->port_items_tmpl,
+				 hw_ctrl_fdb->jump_one_actions_tmpl, error);
+		if (!hw_ctrl_fdb->hw_esw_zero_tbl) {
+			DRV_LOG(ERR, "port %u failed to create table for default jump to group 1"
+				" for control flows", dev->data->port_id);
+			goto err;
+		}
 	}
 	/* Create templates and table for default Tx metadata copy flow rule. */
 	if (!repr_matching && xmeta == MLX5_XMETA_MODE_META32_HWS) {
@@ -15383,6 +15393,8 @@ mlx5_flow_hw_esw_destroy_sq_miss_flow(struct rte_eth_dev *dev, uint32_t sqn)
 	}
 	proxy_dev = &rte_eth_devices[proxy_port_id];
 	proxy_priv = proxy_dev->data->dev_private;
+	/* FDB default flow rules must be enabled. */
+	MLX5_ASSERT(proxy_priv->sh->config.fdb_def_rule);
 	if (!proxy_priv->dr_ctx)
 		return 0;
 	if (!proxy_priv->hw_ctrl_fdb ||
@@ -15447,6 +15459,8 @@ mlx5_flow_hw_esw_create_default_jump_flow(struct rte_eth_dev *dev)
 	}
 	proxy_dev = &rte_eth_devices[proxy_port_id];
 	proxy_priv = proxy_dev->data->dev_private;
+	/* FDB default flow rules must be enabled. */
+	MLX5_ASSERT(proxy_priv->sh->config.fdb_def_rule);
 	if (!proxy_priv->dr_ctx) {
 		DRV_LOG(DEBUG, "Transfer proxy port (port %u) of port %u must be configured "
 			       "for HWS to create default FDB jump rule. Default rule will "
drivers/net/mlx5/mlx5_trigger.c +3 -1
index 6fa7c01cd0..a65a460731 100644
@@ -1504,7 +1504,9 @@ mlx5_traffic_enable_hws(struct rte_eth_dev *dev)
 		if (!txq)
 			continue;
 		queue = mlx5_txq_get_sqn(txq);
-		if ((priv->representor || priv->master) && config->dv_esw_en) {
+		if ((priv->representor || priv->master) &&
+		    config->dv_esw_en &&
+		    config->fdb_def_rule) {
 			if (mlx5_flow_hw_esw_create_sq_miss_flow(dev, queue, false)) {
 				mlx5_txq_release(dev, i);
 				goto error;
drivers/net/mlx5/mlx5_txq.c +10 -3
index 8eb1ae1f03..f05534e168 100644
@@ -1363,11 +1363,18 @@ rte_pmd_mlx5_external_sq_enable(uint16_t port_id, uint32_t sq_num)
 	}
 #ifdef HAVE_MLX5_HWS_SUPPORT
 	if (priv->sh->config.dv_flow_en == 2) {
-		if (mlx5_flow_hw_esw_create_sq_miss_flow(dev, sq_num, true))
-			return -rte_errno;
+		bool sq_miss_created = false;
+
+		if (priv->sh->config.fdb_def_rule) {
+			if (mlx5_flow_hw_esw_create_sq_miss_flow(dev, sq_num, true))
+				return -rte_errno;
+			sq_miss_created = true;
+		}
+
 		if (priv->sh->config.repr_matching &&
 		    mlx5_flow_hw_tx_repr_matching_flow(dev, sq_num, true)) {
-			mlx5_flow_hw_esw_destroy_sq_miss_flow(dev, sq_num);
+			if (sq_miss_created)
+				mlx5_flow_hw_esw_destroy_sq_miss_flow(dev, sq_num);
 			return -rte_errno;
 		}
 		return 0;
Signature · 1 line
2.39.2
#2 lore raw

PATCH 2/3 net/mlx5: fix action configuration validation

In reply to Dariusz Sosnowski

Checking if action configuration is required should be checked based on action type recorded in the actions template, not on user action.

Also, adds a missing internal RSS action type to configuration check skip list.

Fixes 57c7b94301ee ("net/mlx5: add async flow operation validation")
---
 drivers/net/mlx5/mlx5_flow_hw.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/net/mlx5/mlx5_flow_hw.c +2 -1
index fe7df7305f..39d1cd96d4 100644
@@ -16388,10 +16388,11 @@ flow_hw_validate_rule_actions(struct rte_eth_dev *dev,
 		user_action = &actions[act_data->action_src];
 
 		/* Skip actions which do not require conf. */
-		switch ((int)user_action->type) {
+		switch ((int)act_data->type) {
 		case RTE_FLOW_ACTION_TYPE_COUNT:
 		case MLX5_RTE_FLOW_ACTION_TYPE_COUNT:
 		case MLX5_RTE_FLOW_ACTION_TYPE_METER_MARK:
+		case MLX5_RTE_FLOW_ACTION_TYPE_RSS:
 			continue;
 		default:
 			break;
Signature · 1 line
2.39.2
#3 lore raw

PATCH 3/3 net/mlx5: fix RSS and queue action validation

In reply to Dariusz Sosnowski

mlx5 PMD supports configuration where
Rx queues managed by DPDK are not set up.
Externally allocated RQs can be used by mapping them to some DPDK Rx queue indexes using rte_pmd_mlx5_external_rx_queue_id_map() API. In this case, mlx5 PMD will allow creating flow rules which reference such external RQ.

HWS validation of RSS and QUEUE unmasked flow actions in actions templates worked by constructing a "mock" action which was then checked. This procedure incorrectly assumed that queue index 0 can be used as "always valid queue", which is not the case in scenario mentioned above, because queue 0 was not set up

This patch fixes that by removing "mock" actions, since there's no real data available for validation. RSS and QUEUE validation in unmasked action case only checks flow attributes.

Fixes d6dc072aeb12 ("net/mlx5: validate flow actions in table creation")
---
 drivers/net/mlx5/mlx5_flow_hw.c | 37 +++++++++++++--------------------
 1 file changed, 14 insertions(+), 23 deletions(-)
drivers/net/mlx5/mlx5_flow_hw.c +14 -23
index 39d1cd96d4..d243b59b71 100644
@@ -6806,8 +6806,6 @@ mlx5_hw_validate_action_mark(struct rte_eth_dev *dev,
 					      &attr, error);
 }
 
-#define MLX5_FLOW_DEFAULT_INGRESS_QUEUE 0
-
 static int
 mlx5_hw_validate_action_queue(struct rte_eth_dev *dev,
 			      const struct rte_flow_action *template_action,
@@ -6817,22 +6815,22 @@ mlx5_hw_validate_action_queue(struct rte_eth_dev *dev,
 			      struct rte_flow_error *error)
 {
 	const struct rte_flow_action_queue *queue_mask = template_mask->conf;
-	const struct rte_flow_action *action =
-		queue_mask && queue_mask->index ? template_action :
-		&(const struct rte_flow_action) {
-		.type = RTE_FLOW_ACTION_TYPE_QUEUE,
-		.conf = &(const struct rte_flow_action_queue) {
-			.index = MLX5_FLOW_DEFAULT_INGRESS_QUEUE
-		}
-	};
 	const struct rte_flow_attr attr = {
 		.ingress = template_attr->ingress,
 		.egress = template_attr->egress,
 		.transfer = template_attr->transfer
 	};
+	bool masked = queue_mask != NULL && queue_mask->index;
 
-	return mlx5_flow_validate_action_queue(action, action_flags,
-					       dev, &attr, error);
+	if (template_attr->egress || template_attr->transfer)
+		return rte_flow_error_set(error, EINVAL,
+					  RTE_FLOW_ERROR_TYPE_ATTR, NULL,
+					  "QUEUE action supported for ingress only");
+	if (masked)
+		return mlx5_flow_validate_action_queue(template_action, action_flags, dev,
+						       &attr, error);
+	else
+		return 0;
 }
 
 static int
@@ -6844,22 +6842,15 @@ mlx5_hw_validate_action_rss(struct rte_eth_dev *dev,
 			      struct rte_flow_error *error)
 {
 	const struct rte_flow_action_rss *mask = template_mask->conf;
-	const struct rte_flow_action *action = mask ? template_action :
-		&(const struct rte_flow_action) {
-		.type = RTE_FLOW_ACTION_TYPE_RSS,
-		.conf = &(const struct rte_flow_action_rss) {
-			.queue_num = 1,
-			.queue = (uint16_t [1]) {
-				MLX5_FLOW_DEFAULT_INGRESS_QUEUE
-			}
-		}
-	};
 
 	if (template_attr->egress || template_attr->transfer)
 		return rte_flow_error_set(error, EINVAL,
 					  RTE_FLOW_ERROR_TYPE_ATTR, NULL,
 					  "RSS action supported for ingress only");
-	return mlx5_validate_action_rss(dev, action, error);
+	if (mask != NULL)
+		return mlx5_validate_action_rss(dev, template_action, error);
+	else
+		return 0;
 }
 
 static int
Signature · 1 line
2.39.2
#4 lore raw
In reply to Dariusz Sosnowski

Hi,

From: Dariusz Sosnowski <[email protected]>
Sent: Thursday, July 18, 2024 12:57 PM
To: Slava Ovsiienko; Bing Zhao; Ori Kam; Suanming Mou; Matan Azrad
Cc: [email protected]; Raslan Darawsheh
Subject: [PATCH 0/3] net/mlx5: E-Switch and validation fixes

Patch 1 - Fixes a bug with fdb_def_rule_en device argument, used to control default E-Switch flow rules created by mlx5 PMD.

Patches 2-3 - Fixes for flow rule validation in async flow API, which were found during testing.

Dariusz Sosnowski (3):
net/mlx5: fix disabling E-Switch default flow rules net/mlx5: fix action configuration validation net/mlx5: fix RSS and queue action validation

 drivers/net/mlx5/mlx5_flow_hw.c | 182 +++++++++++++++++---------------
 drivers/net/mlx5/mlx5_trigger.c |   4 +-
 drivers/net/mlx5/mlx5_txq.c     |  13 ++-
 3 files changed, 107 insertions(+), 92 deletions(-)

series applied to next-net-mlx,

Kindest regards,
Raslan Darawsheh