Skip to contentSkip to conversation

net/mlx5: fix representor matching

PATCH In review Dariusz Sosnowski 4 participants 7 messages Feb 25, 2023 – Mar 19, 2023 Network drivers

lore
Share & tools
lore
#0 lore raw

cover net/mlx5: fix representor matching

This patch series addresses a few issues with representor matching mode available in mlx5 PMD:

- First patch fixes an issue with internal flow group index translation occurring when both extended metadata mode and representor matching was enabled.
- Second patch fixes an issue with isolated mode occurring when representor matching was disabled.

Dariusz Sosnowski (2):
net/mlx5: fix egress group translation in HWS net/mlx5: fix isolated mode when repr matching is disabled

 doc/guides/nics/mlx5.rst         |  3 +++
 drivers/net/mlx5/linux/mlx5_os.c | 16 ++++++++++++++++
 drivers/net/mlx5/mlx5_flow.c     |  4 ++++
 drivers/net/mlx5/mlx5_flow_hw.c  | 14 +++++++++-----
 4 files changed, 32 insertions(+), 5 deletions(-)
Signature · 1 line
2.25.1
#1 lore raw

PATCH 1/2 net/mlx5: fix egress group translation in HWS

With HW Steering enabled creating egress template tables and egress flow rules on E-Switch setups is allowed. To enable it, PMD creates a set of default egress flow rules responsible for:

- Storing representor ID (vport tag is used) in HW register. This is used for traffic source identification.
- Copying software metadata to proper HW register to allow preserving metadata across domains.

Structure of these flow rules and whether they are inserted depend on the device configuration.
There are the following cases:

1. repr_matching=1 and dv_xmeta_en=4
- An egress flow rule in group 0 is created for each Tx queue;
- Flow rule matching SQ number - fills unused REG_C_0 bits with vport tag, copies REG_A to REG_C_1 and jumps to group 1.
2. repr_matching=1 and dv_xmeta_en=0
- An egress flow rule in group 0 is created for each Tx queue;
- Flow rule matching SQ number - fills unused REG_C_0 bits with vport tag and jumps to group 1.
3. repr_matching=0 and dv_xmeta_en=4
- A single egress flow rule in group 0 is created;
- Flow rule matches all E-Switch manager TX traffic, copies REG_A to REG_C and jumps to group 1.
4. repr_matching=0 and dv_xmeta_en=0 - no default flow rules are added.

When default egress flow rules are required, they are inserted in group 0 and this group is reserved for PMD purposes. User created template tables must be created in higher groups. As a result, on template table creation PMD is translating the provided group (incrementing it in that case).

Before this patch, a condition used to check if translation of egress flow group is needed was incorrect. It did not allow translation if both representor matching AND extended metadata mode were enabled.

This patch fixes this condition - translation is allowed if and only if representor matching OR extended metadata mode is enabled.

Fixes 483181f7b6dd ("net/mlx5: support device control of representor matching")
---
 drivers/net/mlx5/mlx5_flow_hw.c | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)
drivers/net/mlx5/mlx5_flow_hw.c +9 -5
index a9c7045a3e..d3d86fe24d 100644
@@ -3260,14 +3260,18 @@ flow_hw_translate_group(struct rte_eth_dev *dev,
 						  "group index not supported");
 		*table_group = group + 1;
 	} else if (config->dv_esw_en &&
-		   !(config->repr_matching && config->dv_xmeta_en == MLX5_XMETA_MODE_META32_HWS) &&
+		   (config->repr_matching || config->dv_xmeta_en == MLX5_XMETA_MODE_META32_HWS) &&
 		   cfg->external &&
 		   flow_attr->egress) {
 		/*
-		 * On E-Switch setups, egress group translation is not done if and only if
-		 * representor matching is disabled and legacy metadata mode is selected.
-		 * In all other cases, egree group 0 is reserved for representor tagging flows
-		 * and metadata copy flows.
+		 * On E-Switch setups, default egress flow rules are inserted to allow
+		 * representor matching and/or preserving metadata across steering domains.
+		 * These flow rules are inserted in group 0 and this group is reserved by PMD
+		 * for these purposes.
+		 *
+		 * As a result, if representor matching or extended metadata mode is enabled,
+		 * group provided by the user must be incremented to avoid inserting flow rules
+		 * in group 0.
 		 */
 		if (group > MLX5_HW_MAX_EGRESS_GROUP)
 			return rte_flow_error_set(error, EINVAL,
Signature · 1 line
2.25.1
Acked-by on v1 1/2
#2 lore raw

RE: [PATCH 1/2] net/mlx5: fix egress group translation in HWS

Quoted 50 lines

-----Original Message-----
From: Dariusz Sosnowski <[email protected]>
Sent: Sunday, February 26, 2023 4:18 AM
To: Matan Azrad <[email protected]>; Slava Ovsiienko <[email protected]>
Cc: [email protected]; [email protected]; Ori Kam <[email protected]>
Subject: [PATCH 1/2] net/mlx5: fix egress group translation in HWS

With HW Steering enabled creating egress template tables and egress flow rules on E-Switch setups is allowed.
To enable it, PMD creates a set of default egress flow rules responsible for:

- Storing representor ID (vport tag is used) in HW register. This is used for traffic source identification.
- Copying software metadata to proper HW register to allow preserving metadata across domains.

Structure of these flow rules and whether they are inserted depend on the device configuration.
There are the following cases:

1. repr_matching=1 and dv_xmeta_en=4
- An egress flow rule in group 0 is created for each Tx queue;
- Flow rule matching SQ number - fills unused REG_C_0 bits with vport tag, copies REG_A to REG_C_1 and jumps to group 1.
2. repr_matching=1 and dv_xmeta_en=0
- An egress flow rule in group 0 is created for each Tx queue;
- Flow rule matching SQ number - fills unused REG_C_0 bits with vport tag and jumps to group 1.
3. repr_matching=0 and dv_xmeta_en=4
- A single egress flow rule in group 0 is created;
- Flow rule matches all E-Switch manager TX traffic, copies REG_A to REG_C and jumps to group 1.
4. repr_matching=0 and dv_xmeta_en=0 - no default flow rules are added.

When default egress flow rules are required, they are inserted in group 0 and this group is reserved for PMD purposes.
User created template tables must be created in higher groups.
As a result, on template table creation PMD is translating the provided group (incrementing it in that case).

Before this patch, a condition used to check if translation of egress flow group is needed was incorrect. It did not allow translation if both representor matching AND extended metadata mode were enabled.

This patch fixes this condition - translation is allowed if and only if representor matching OR extended metadata mode is enabled.

Quoted 6 lines
Fixes 483181f7b6dd ("net/mlx5: support device control of representor

matching")
Cc: [email protected]

#3 lore raw

PATCH 2/2 net/mlx5: fix isolated mode when repr matching is disabled

In reply to Dariusz Sosnowski

In HW steering mode, when running on an E-Switch setup, mlx5 PMD provides an ability to enable or disable representor matching (through repr_matching_en device argument). If representor matching is enabled, any ingress or egress flow rule, created on any port representor will match traffic related to that specific port.
If it is disabled, flow rule created on one of the ports, will match traffic related to all ports.

As a result, when representor matching is disabled, PMD cannot correctly create control flow rules for receiving default traffic according to port configuration. Since each port representor in the same switch domain, can have different port configuration and flow rules do not differentiate between ports, these flow rules cannot be correctly applied.
In that case, each port works in de facto isolated mode.

This patch makes sure that if representor matching is disabled, port is forced into isolated mode. Disabling flow isolated is forbidden.

Fixes 483181f7b6dd ("net/mlx5: support device control of representor matching")
---
 doc/guides/nics/mlx5.rst         |  3 +++
 drivers/net/mlx5/linux/mlx5_os.c | 16 ++++++++++++++++
 drivers/net/mlx5/mlx5_flow.c     |  4 ++++
 3 files changed, 23 insertions(+)
doc/guides/nics/mlx5.rst +3 -0
index 6510e74fb9..350fb0287e 100644
@@ -1137,6 +1137,9 @@ for an additional list of options shared with other mlx5 drivers.
   - 0. If representor matching is disabled, then there will be no implicit
     item added. As a result, ingress flow rules will match traffic
     coming to any port, not only the port on which flow rule is created.
+    Because of that, default flow rules for ingress traffic cannot be created
+    and port starts in isolated mode by default. Port cannot be switched back
+    to non-isolated mode.
 
   - 1. If representor matching is enabled (default setting),
     then each ingress pattern template has an implicit REPRESENTED_PORT
drivers/net/mlx5/linux/mlx5_os.c +16 -0
index a71474c90a..2cce0a7f0f 100644
@@ -1613,6 +1613,22 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
 			err = EINVAL;
 			goto error;
 		}
+		/*
+		 * If representor matching is disabled, PMD cannot create default flow rules
+		 * to receive traffic for all ports, since implicit source port match is not added.
+		 * Isolated mode is forced.
+		 */
+		if (priv->sh->config.dv_esw_en && !priv->sh->config.repr_matching) {
+			err = mlx5_flow_isolate(eth_dev, 1, NULL);
+			if (err < 0) {
+				err = -err;
+				goto error;
+			}
+			DRV_LOG(WARNING, "port %u ingress traffic is restricted to defined "
+					 "flow rules (isolated mode) since representor "
+					 "matching is disabled",
+				eth_dev->data->port_id);
+		}
 		return eth_dev;
 #else
 		DRV_LOG(ERR, "DV support is missing for HWS.");
drivers/net/mlx5/mlx5_flow.c +4 -0
index a6a426caf7..7ee30bdd1f 100644
@@ -8077,6 +8077,10 @@ mlx5_flow_isolate(struct rte_eth_dev *dev,
 				   "port must be stopped first");
 		return -rte_errno;
 	}
+	if (!enable && !priv->sh->config.repr_matching)
+		return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
+					  "isolated mode cannot be disabled when "
+					  "representor matching is disabled");
 	priv->isolated = !!enable;
 	if (enable)
 		dev->dev_ops = &mlx5_dev_ops_isolate;
Signature · 1 line
2.25.1
Acked-by on v1 2/2
#4 lore raw

RE: [PATCH 2/2] net/mlx5: fix isolated mode when repr matching is disabled

Quoted 28 lines

-----Original Message-----
From: Dariusz Sosnowski <[email protected]>
Sent: Sunday, February 26, 2023 4:18 AM
To: Matan Azrad <[email protected]>; Slava Ovsiienko <[email protected]>
Cc: [email protected]; [email protected]; Ori Kam <[email protected]>
Subject: [PATCH 2/2] net/mlx5: fix isolated mode when repr matching is disabled

In HW steering mode, when running on an E-Switch setup,
mlx5 PMD provides an ability to enable or disable representor matching (through repr_matching_en device argument).
If representor matching is enabled, any ingress or egress flow rule, created on any port representor will match traffic related to that specific port. If it is disabled, flow rule created on one of the ports, will match traffic related to all ports.

As a result, when representor matching is disabled, PMD cannot correctly create control flow rules for receiving default traffic according to port configuration. Since each port representor in the same switch domain, can have different port configuration and flow rules do not differentiate between ports, these flow rules cannot be correctly applied.
In that case, each port works in de facto isolated mode.

This patch makes sure that if representor matching is disabled, port is forced into isolated mode. Disabling flow isolated is forbidden.

Quoted 6 lines
Fixes 483181f7b6dd ("net/mlx5: support device control of representor

matching")
Cc: [email protected]

Thanks.

#5 lore raw
In reply to Dariusz Sosnowski

From: Dariusz Sosnowski

Quoted 13 lines

This patch series addresses a few issues with representor matching mode available in mlx5 PMD:

- First patch fixes an issue with internal flow group index translation occurring when both extended metadata mode and representor matching was enabled.
- Second patch fixes an issue with isolated mode occurring when representor matching was disabled.

Dariusz Sosnowski (2):
net/mlx5: fix egress group translation in HWS net/mlx5: fix isolated mode when repr matching is disabled

Quoted 6 lines
 doc/guides/nics/mlx5.rst         |  3 +++
 drivers/net/mlx5/linux/mlx5_os.c | 16 ++++++++++++++++
 drivers/net/mlx5/mlx5_flow.c     |  4 ++++
 drivers/net/mlx5/mlx5_flow_hw.c  | 14 +++++++++-----
 4 files changed, 32 insertions(+), 5 deletions(-)
#6 lore raw
In reply to Dariusz Sosnowski

Hi,

Quoted 21 lines

-----Original Message-----
From: Dariusz Sosnowski <[email protected]>
Sent: Saturday, February 25, 2023 10:18 PM
To: Matan Azrad <[email protected]>; Slava Ovsiienko <[email protected]>
Cc: [email protected]; [email protected]; Ori Kam <[email protected]>
Subject: [PATCH 0/2] net/mlx5: fix representor matching

This patch series addresses a few issues with representor matching mode available in mlx5 PMD:

- First patch fixes an issue with internal flow group index translation occurring when both extended metadata mode and representor matching was enabled.
- Second patch fixes an issue with isolated mode occurring when representor matching was disabled.

Dariusz Sosnowski (2):
net/mlx5: fix egress group translation in HWS net/mlx5: fix isolated mode when repr matching is disabled

Quoted 8 lines
 doc/guides/nics/mlx5.rst         |  3 +++
 drivers/net/mlx5/linux/mlx5_os.c | 16 ++++++++++++++++
 drivers/net/mlx5/mlx5_flow.c     |  4 ++++
 drivers/net/mlx5/mlx5_flow_hw.c  | 14 +++++++++-----
 4 files changed, 32 insertions(+), 5 deletions(-)

--
2.25.1

Series applied to next-net-mlx,

Kindest regards,
Raslan Darawsheh