Skip to contentSkip to conversation

net/mlx5/hws: fix matcher clean up for FDB tables

PATCH In review Dariusz Sosnowski 3 participants 5 messages Nov 9, 2022 Network drivers

lore
Share & tools
lore
#0 lore raw

cover net/mlx5/hws: fix matcher clean up for FDB tables

Before these patches, if an application was configured to run with HW Steering and E-Switch enabled, on EAL cleanup the assertion in mlx5_dev_hw_global_release() was triggered - PD release was unsuccessful.

Root cause of this issue was linked to an inability to destroy RTC objects used internally in mlx5, in HW Steering implementation.
PMD was unable to destroy RTC objects, because of dangling references to those objects. More specifically, if all matchers connected to a single flow table were created, this flow table was still referencing RTC objects when theye were being destroyed.

This patch series fixes that behavior.
Matcher uninitilization is updated to remove the references to RTC objects from flow table object if the last matcher related to the flow table was destroyed.

Erez Shitrit (2):
net/mlx5/hws: fix order of destroying default tables net/mlx5/hws: fix disconnecting matcher

 drivers/net/mlx5/hws/mlx5dr_matcher.c | 35 +++++++++++++++++++++++++++
 drivers/net/mlx5/hws/mlx5dr_table.c   |  2 +-
 2 files changed, 36 insertions(+), 1 deletion(-)
Signature · 1 line
2.25.1
#1 lore raw

PATCH 1/2 net/mlx5/hws: fix order of destroying default tables

From: Erez Shitrit <[email protected]>

This patch fixes the order dereferencing default FDB miss table and destroying the flow table object. Flow table should be destroyed before the dereference.

Fixes 394cc7ba4033 ("net/mlx5/hws: add table object")
---
 drivers/net/mlx5/hws/mlx5dr_table.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/mlx5/hws/mlx5dr_table.c +1 -1
index d3f77e4780..33052abce8 100644
@@ -131,8 +131,8 @@ mlx5dr_table_create_default_ft(struct mlx5dr_table *tbl)
 void mlx5dr_table_destroy_default_ft(struct mlx5dr_table *tbl,
 				     struct mlx5dr_devx_obj *ft_obj)
 {
-	mlx5dr_table_down_default_fdb_miss_tbl(tbl);
 	mlx5dr_cmd_destroy_obj(ft_obj);
+	mlx5dr_table_down_default_fdb_miss_tbl(tbl);
 }
 
 static int mlx5dr_table_init(struct mlx5dr_table *tbl)
Signature · 1 line
2.25.1
#2 lore raw

PATCH 2/2 net/mlx5/hws: fix disconnecting matcher

In reply to Dariusz Sosnowski

From: Erez Shitrit <[email protected]>

This patch fixes the matcher disconnection handling, by removing the RTC references from flow table if the currently removed matcher was the last one for the given table. As a result RTC in this matcher can be correctly freed, since there are no dangling references to the RTC.

Fixes c467608215b2 ("net/mlx5/hws: add matcher object")
---
 drivers/net/mlx5/hws/mlx5dr_matcher.c | 35 +++++++++++++++++++++++++++
 1 file changed, 35 insertions(+)
drivers/net/mlx5/hws/mlx5dr_matcher.c +35 -0
index 67adfeec6c..2e444c1179 100644
@@ -36,6 +36,30 @@ static void mlx5dr_matcher_destroy_end_ft(struct mlx5dr_matcher *matcher)
 	mlx5dr_table_destroy_default_ft(matcher->tbl, matcher->end_ft);
 }
 
+static int mlx5dr_matcher_free_rtc_pointing(uint32_t fw_ft_type,
+					    enum mlx5dr_table_type type,
+					    struct mlx5dr_devx_obj *devx_obj)
+{
+	struct mlx5dr_cmd_ft_modify_attr ft_attr = {0};
+	int ret;
+
+	if (type != MLX5DR_TABLE_TYPE_FDB)
+		return 0;
+
+	ft_attr.modify_fs = MLX5_IFC_MODIFY_FLOW_TABLE_RTC_ID;
+	ft_attr.type = fw_ft_type;
+	ft_attr.rtc_id_0 = 0;
+	ft_attr.rtc_id_1 = 0;
+
+	ret = mlx5dr_cmd_flow_table_modify(devx_obj, &ft_attr);
+	if (ret) {
+		DR_LOG(ERR, "Failed to disconnect previous RTC");
+		return ret;
+	}
+
+	return 0;
+}
+
 static int mlx5dr_matcher_connect(struct mlx5dr_matcher *matcher)
 {
 	struct mlx5dr_cmd_ft_modify_attr ft_attr = {0};
@@ -148,6 +172,17 @@ static int mlx5dr_matcher_disconnect(struct mlx5dr_matcher *matcher)
 
 	LIST_REMOVE(matcher, next);
 
+	if (!next) {
+		/* ft no longer points to any RTC, drop refcount */
+		ret = mlx5dr_matcher_free_rtc_pointing(tbl->fw_ft_type,
+						       tbl->type,
+						       prev_ft);
+		if (ret) {
+			DR_LOG(ERR, "Failed to reset last RTC refcount");
+			return ret;
+		}
+	}
+
 	return 0;
 }
 
Signature · 1 line
2.25.1
#3 lore raw
In reply to Dariusz Sosnowski

From: Dariusz Sosnowski

Quoted 16 lines

Before these patches, if an application was configured to run with HW Steering and E-Switch enabled, on EAL cleanup the assertion in mlx5_dev_hw_global_release() was triggered - PD release was unsuccessful.

Root cause of this issue was linked to an inability to destroy RTC objects used internally in mlx5, in HW Steering implementation.
PMD was unable to destroy RTC objects, because of dangling references to those objects. More specifically, if all matchers connected to a single flow table were created, this flow table was still referencing RTC objects when theye were being destroyed.

This patch series fixes that behavior.
Matcher uninitilization is updated to remove the references to RTC objects from flow table object if the last matcher related to the flow table was destroyed.

Quoted 8 lines

Erez Shitrit (2):
net/mlx5/hws: fix order of destroying default tables net/mlx5/hws: fix disconnecting matcher

 drivers/net/mlx5/hws/mlx5dr_matcher.c | 35
+++++++++++++++++++++++++++
 drivers/net/mlx5/hws/mlx5dr_table.c   |  2 +-
 2 files changed, 36 insertions(+), 1 deletion(-)
Quoted 2 lines

--
2.25.1

#4 lore raw
In reply to Dariusz Sosnowski

Hi,

Quoted 29 lines

-----Original Message-----
From: Dariusz Sosnowski <[email protected]>
Sent: Wednesday, November 9, 2022 11:54 AM
To: Matan Azrad <[email protected]>; Slava Ovsiienko <[email protected]>; Alex Vesker <[email protected]>; Erez Shitrit <[email protected]>
Cc: [email protected]; Raslan Darawsheh <[email protected]>
Subject: [PATCH 0/2] net/mlx5/hws: fix matcher clean up for FDB tables

Before these patches, if an application was configured to run with HW Steering and E-Switch enabled, on EAL cleanup the assertion in mlx5_dev_hw_global_release() was triggered - PD release was unsuccessful.

Root cause of this issue was linked to an inability to destroy RTC objects used internally in mlx5, in HW Steering implementation.
PMD was unable to destroy RTC objects, because of dangling references to those objects. More specifically, if all matchers connected to a single flow table were created, this flow table was still referencing RTC objects when theye were being destroyed.

This patch series fixes that behavior.
Matcher uninitilization is updated to remove the references to RTC objects from flow table object if the last matcher related to the flow table was destroyed.

Erez Shitrit (2):
net/mlx5/hws: fix order of destroying default tables net/mlx5/hws: fix disconnecting matcher

Quoted 7 lines
 drivers/net/mlx5/hws/mlx5dr_matcher.c | 35
+++++++++++++++++++++++++++
 drivers/net/mlx5/hws/mlx5dr_table.c   |  2 +-
 2 files changed, 36 insertions(+), 1 deletion(-)

--
2.25.1

series applied to next-net-mlx,

Kindest regards,
Raslan Darawsheh