Skip to contentSkip to conversation

Mellanox, mlx5 updates 2020-03-13

GIT PULL In review Saeed Mahameed 4 participants 22 messages Mar 14, 2020 – Mar 15, 2020

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

Hi Dave,

This series adds misc updates to mlx5 driver
For more information please see tag log below.

Please pull and let me know if there is any problem.

Thanks,
Saeed.

---
The following changes since commit 1d343579312311aa9875b34d5a921f5e2ec69f0a:

Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net (2020-03-12 22:34:48 -0700)

are available in the Git repository at:

git://git.kernel.org/pub/scm/linux/kernel/git/saeed/linux.git tags/mlx5-updates-2020-03-13

for you to fetch changes up to bc1a02884a33f9d49cda0c77dc8eccebd6c5c0e5:

net/mlx5: DR, Remove unneeded functions deceleration (2020-03-13 16:26:28 -0700)

----------------------------------------------------------------
mlx5-updates-2020-03-13

Misc update to mlx5 core and E-Switch driver:

1) Blue-Field, Update VF vports config when num of VFs changed

From Bodon, Various misc cleanups and refactoring
for vport enabling/disabling routines to allow them to be called dynamically and not only on E-Switch load.

This will allow ECPF (ConnectX BlueField Smartnic) support for dynamic num vf changes and dynamic vport creation and configuration as introduced in "Update VF vports config when num of VFs changed" patch.

2) From Parav and Mark, trivial clean-ups.

3) Software steering support for flow table id as destination and a clean-up patch to remove unnecessary function stubs, from Alex.

----------------------------------------------------------------
Alex Vesker (2):
net/mlx5: DR, Add support for flow table id destination action net/mlx5: DR, Remove unneeded functions deceleration

Bodong Wang (8):
net/mlx5: E-Switch, Remove redundant check of eswitch manager cap net/mlx5: E-Switch, Hold mutex when querying drop counter in legacy mode net/mlx5: E-Switch, Remove redundant warning when QoS enable failed net/mlx5: E-Switch, Prepare for vport enable/disable refactor net/mlx5: E-switch, Make vport setup/cleanup sequence symmetric net/mlx5: E-Switch, Introduce per vport configuration for eswitch modes net/mlx5: E-Switch, Update VF vports config when num of VFs changed net/mlx5: E-Switch, Refactor unload all reps per rep type

Mark Bloch (1):
net/mlx5: Accept flow rules without match

Parav Pandit (3):
net/mlx5: E-switch, Annotate termtbl_mutex mutex destroy net/mlx5: E-switch, Annotate esw state_lock mutex destroy net/mlx5: Avoid deriving mlx5_core_dev second time

 drivers/net/ethernet/mellanox/mlx5/core/en_arfs.c  |  15 +-
 drivers/net/ethernet/mellanox/mlx5/core/eswitch.c  | 245 ++++++++++++---------
 drivers/net/ethernet/mellanox/mlx5/core/eswitch.h  |  11 +
 .../ethernet/mellanox/mlx5/core/eswitch_offloads.c | 237 ++++++--------------
 .../mellanox/mlx5/core/eswitch_offloads_chains.c   |   3 +-
 .../mellanox/mlx5/core/eswitch_offloads_termtbl.c  |   3 +-
 drivers/net/ethernet/mellanox/mlx5/core/fs_core.c  |   4 +
 .../mellanox/mlx5/core/steering/dr_action.c        |  18 ++
 .../ethernet/mellanox/mlx5/core/steering/fs_dr.c   |  12 +
 .../ethernet/mellanox/mlx5/core/steering/mlx5dr.h  | 104 +--------
 10 files changed, 266 insertions(+), 386 deletions(-)
#1 lore raw

PATCH 1/14 net/mlx5: E-Switch, Remove redundant check of eswitch manager cap

From: Bodong Wang <[email protected]>

esw_vport_create_legacy_acl_tables bails out immediately for eswitch manager, hence remove all the check of esw manager cap after.

---
 drivers/net/ethernet/mellanox/mlx5/core/eswitch.c | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/eswitch.c +2 -4
index 25640864c375..b123089866e2 100644
@@ -1707,8 +1707,7 @@ static int esw_vport_create_legacy_acl_tables(struct mlx5_eswitch *esw,
 	if (mlx5_esw_is_manager_vport(esw, vport->vport))
 		return 0;
 
-	if (!mlx5_esw_is_manager_vport(esw, vport->vport) &&
-	    MLX5_CAP_ESW_INGRESS_ACL(esw->dev, flow_counter)) {
+	if (MLX5_CAP_ESW_INGRESS_ACL(esw->dev, flow_counter)) {
 		vport->ingress.legacy.drop_counter = mlx5_fc_create(esw->dev, false);
 		if (IS_ERR(vport->ingress.legacy.drop_counter)) {
 			esw_warn(esw->dev,
@@ -1722,8 +1721,7 @@ static int esw_vport_create_legacy_acl_tables(struct mlx5_eswitch *esw,
 	if (ret)
 		goto ingress_err;
 
-	if (!mlx5_esw_is_manager_vport(esw, vport->vport) &&
-	    MLX5_CAP_ESW_EGRESS_ACL(esw->dev, flow_counter)) {
+	if (MLX5_CAP_ESW_EGRESS_ACL(esw->dev, flow_counter)) {
 		vport->egress.legacy.drop_counter = mlx5_fc_create(esw->dev, false);
 		if (IS_ERR(vport->egress.legacy.drop_counter)) {
 			esw_warn(esw->dev,
Signature · 1 line
2.24.1
#2 lore raw

PATCH 2/14 net/mlx5: E-Switch, Hold mutex when querying drop counter in legacy mode

In reply to Saeed Mahameed

From: Bodong Wang <[email protected]>

Consider scenario below, CPU 1 is at risk to query already destroyed drop counters. Need to apply the same state mutex when disabling vport.

+-------------------------------+-------------------------------------+
| CPU 0                         | CPU 1                               |
+-------------------------------+-------------------------------------+
| mlx5_device_disable_sriov     | mlx5e_get_vf_stats                  |
| mlx5_eswitch_disable          | mlx5_eswitch_get_vport_stats        |
| esw_disable_vport             | mlx5_eswitch_query_vport_drop_stats |
| mlx5_fc_destroy(drop_counter) | mlx5_fc_query(drop_counter)         |
+-------------------------------+-------------------------------------+
Fixes b8a0dbe3a90b ("net/mlx5e: E-switch, Add steering drop counters")
---
 drivers/net/ethernet/mellanox/mlx5/core/eswitch.c | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/eswitch.c +10 -4
index b123089866e2..b4b93d2322a9 100644
@@ -2620,9 +2620,13 @@ static int mlx5_eswitch_query_vport_drop_stats(struct mlx5_core_dev *dev,
 	u64 bytes = 0;
 	int err = 0;
 
-	if (!vport->enabled || esw->mode != MLX5_ESWITCH_LEGACY)
+	if (esw->mode != MLX5_ESWITCH_LEGACY)
 		return 0;
 
+	mutex_lock(&esw->state_lock);
+	if (!vport->enabled)
+		goto unlock;
+
 	if (vport->egress.legacy.drop_counter)
 		mlx5_fc_query(dev, vport->egress.legacy.drop_counter,
 			      &stats->rx_dropped, &bytes);
@@ -2633,20 +2637,22 @@ static int mlx5_eswitch_query_vport_drop_stats(struct mlx5_core_dev *dev,
 
 	if (!MLX5_CAP_GEN(dev, receive_discard_vport_down) &&
 	    !MLX5_CAP_GEN(dev, transmit_discard_vport_down))
-		return 0;
+		goto unlock;
 
 	err = mlx5_query_vport_down_stats(dev, vport->vport, 1,
 					  &rx_discard_vport_down,
 					  &tx_discard_vport_down);
 	if (err)
-		return err;
+		goto unlock;
 
 	if (MLX5_CAP_GEN(dev, receive_discard_vport_down))
 		stats->rx_dropped += rx_discard_vport_down;
 	if (MLX5_CAP_GEN(dev, transmit_discard_vport_down))
 		stats->tx_dropped += tx_discard_vport_down;
 
-	return 0;
+unlock:
+	mutex_unlock(&esw->state_lock);
+	return err;
 }
 
 int mlx5_eswitch_get_vport_stats(struct mlx5_eswitch *esw,
Signature · 1 line
2.24.1
#3 lore raw

PATCH 3/14 net/mlx5: E-Switch, Remove redundant warning when QoS enable failed

In reply to Saeed Mahameed

From: Bodong Wang <[email protected]>

esw_vport_enable_qos can return error in cases below:
1. QoS is already enabled. Warnning is useless in this case.
2. Create scheduling element cmd failed. There is already a warning.

Remove the redundant warnning if esw_vport_enable_qos returns err.

---
 drivers/net/ethernet/mellanox/mlx5/core/eswitch.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/eswitch.c +1 -3
index b4b93d2322a9..deeedf211af0 100644
@@ -1801,9 +1801,7 @@ static int esw_enable_vport(struct mlx5_eswitch *esw, struct mlx5_vport *vport,
 		goto done;
 
 	/* Attach vport to the eswitch rate limiter */
-	if (esw_vport_enable_qos(esw, vport, vport->info.max_rate,
-				 vport->qos.bw_share))
-		esw_warn(esw->dev, "Failed to attach vport %d to eswitch rate limiter", vport_num);
+	esw_vport_enable_qos(esw, vport, vport->info.max_rate, vport->qos.bw_share);
 
 	/* Sync with current vport context */
 	vport->enabled_events = enabled_events;
Signature · 1 line
2.24.1
#4 lore raw

PATCH 4/14 net/mlx5: E-Switch, Prepare for vport enable/disable refactor

In reply to Saeed Mahameed

From: Bodong Wang <[email protected]>

Rename esw_apply_vport_config() to esw_vport_setup(), and add new helper function esw_vport_cleanup() to make them symmetric.

---
 .../net/ethernet/mellanox/mlx5/core/eswitch.c | 26 +++++++++++--------
 1 file changed, 15 insertions(+), 11 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/eswitch.c +15 -11
index deeedf211af0..258141010b62 100644
@@ -1670,8 +1670,7 @@ static void node_guid_gen_from_mac(u64 *node_guid, u8 mac[ETH_ALEN])
 	((u8 *)node_guid)[0] = mac[5];
 }
 
-static void esw_apply_vport_conf(struct mlx5_eswitch *esw,
-				 struct mlx5_vport *vport)
+static void esw_vport_setup(struct mlx5_eswitch *esw, struct mlx5_vport *vport)
 {
 	u16 vport_num = vport->vport;
 	int flags;
@@ -1698,6 +1697,18 @@ static void esw_apply_vport_conf(struct mlx5_eswitch *esw,
 			       flags);
 }
 
+/* Don't cleanup vport->info, it's needed to restore vport configuration */
+static void esw_vport_cleanup(struct mlx5_eswitch *esw, struct mlx5_vport *vport)
+{
+	u16 vport_num = vport->vport;
+
+	if (!mlx5_esw_is_manager_vport(esw, vport_num))
+		mlx5_modify_vport_admin_state(esw->dev,
+					      MLX5_VPORT_STATE_OP_MOD_ESW_VPORT,
+					      vport_num, 1,
+					      MLX5_VPORT_ADMIN_STATE_DOWN);
+}
+
 static int esw_vport_create_legacy_acl_tables(struct mlx5_eswitch *esw,
 					      struct mlx5_vport *vport)
 {
@@ -1794,7 +1805,7 @@ static int esw_enable_vport(struct mlx5_eswitch *esw, struct mlx5_vport *vport,
 	esw_debug(esw->dev, "Enabling VPORT(%d)\n", vport_num);
 
 	/* Restore old vport configuration */
-	esw_apply_vport_conf(esw, vport);
+	esw_vport_setup(esw, vport);
 
 	ret = esw_vport_setup_acl(esw, vport);
 	if (ret)
@@ -1845,14 +1856,7 @@ static void esw_disable_vport(struct mlx5_eswitch *esw,
 	esw_vport_change_handle_locked(vport);
 	vport->enabled_events = 0;
 	esw_vport_disable_qos(esw, vport);
-
-	if (!mlx5_esw_is_manager_vport(esw, vport->vport) &&
-	    esw->mode == MLX5_ESWITCH_LEGACY)
-		mlx5_modify_vport_admin_state(esw->dev,
-					      MLX5_VPORT_STATE_OP_MOD_ESW_VPORT,
-					      vport_num, 1,
-					      MLX5_VPORT_ADMIN_STATE_DOWN);
-
+	esw_vport_cleanup(esw, vport);
 	esw_vport_cleanup_acl(esw, vport);
 	esw->enabled_vports--;
 
Signature · 1 line
2.24.1
#5 lore raw

PATCH 5/14 net/mlx5: E-switch, Make vport setup/cleanup sequence symmetric

In reply to Saeed Mahameed

From: Bodong Wang <[email protected]>

Vport enable and disable sequence is incorrect. It should be:
enable()
esw_vport_setup_acl,
esw_vport_setup,
esw_vport_enable_qos.

disable()
esw_vport_disable_qos,
esw_vport_cleanup,
esw_vport_cleanup_acl.

Instead of having two setup functions for port and acl, merge acl setup to port setup function.

---
 .../net/ethernet/mellanox/mlx5/core/eswitch.c | 101 +++++++++---------
 1 file changed, 53 insertions(+), 48 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/eswitch.c +53 -48
index 258141010b62..603286883550 100644
@@ -1670,45 +1670,6 @@ static void node_guid_gen_from_mac(u64 *node_guid, u8 mac[ETH_ALEN])
 	((u8 *)node_guid)[0] = mac[5];
 }
 
-static void esw_vport_setup(struct mlx5_eswitch *esw, struct mlx5_vport *vport)
-{
-	u16 vport_num = vport->vport;
-	int flags;
-
-	if (mlx5_esw_is_manager_vport(esw, vport_num))
-		return;
-
-	mlx5_modify_vport_admin_state(esw->dev,
-				      MLX5_VPORT_STATE_OP_MOD_ESW_VPORT,
-				      vport_num, 1,
-				      vport->info.link_state);
-
-	/* Host PF has its own mac/guid. */
-	if (vport_num) {
-		mlx5_modify_nic_vport_mac_address(esw->dev, vport_num,
-						  vport->info.mac);
-		mlx5_modify_nic_vport_node_guid(esw->dev, vport_num,
-						vport->info.node_guid);
-	}
-
-	flags = (vport->info.vlan || vport->info.qos) ?
-		SET_VLAN_STRIP | SET_VLAN_INSERT : 0;
-	modify_esw_vport_cvlan(esw->dev, vport_num, vport->info.vlan, vport->info.qos,
-			       flags);
-}
-
-/* Don't cleanup vport->info, it's needed to restore vport configuration */
-static void esw_vport_cleanup(struct mlx5_eswitch *esw, struct mlx5_vport *vport)
-{
-	u16 vport_num = vport->vport;
-
-	if (!mlx5_esw_is_manager_vport(esw, vport_num))
-		mlx5_modify_vport_admin_state(esw->dev,
-					      MLX5_VPORT_STATE_OP_MOD_ESW_VPORT,
-					      vport_num, 1,
-					      MLX5_VPORT_ADMIN_STATE_DOWN);
-}
-
 static int esw_vport_create_legacy_acl_tables(struct mlx5_eswitch *esw,
 					      struct mlx5_vport *vport)
 {
@@ -1793,6 +1754,58 @@ static void esw_vport_cleanup_acl(struct mlx5_eswitch *esw,
 		esw_vport_destroy_offloads_acl_tables(esw, vport);
 }
 
+static int esw_vport_setup(struct mlx5_eswitch *esw, struct mlx5_vport *vport)
+{
+	u16 vport_num = vport->vport;
+	int flags;
+	int err;
+
+	err = esw_vport_setup_acl(esw, vport);
+	if (err)
+		return err;
+
+	/* Attach vport to the eswitch rate limiter */
+	esw_vport_enable_qos(esw, vport, vport->info.max_rate, vport->qos.bw_share);
+
+	if (mlx5_esw_is_manager_vport(esw, vport_num))
+		return 0;
+
+	mlx5_modify_vport_admin_state(esw->dev,
+				      MLX5_VPORT_STATE_OP_MOD_ESW_VPORT,
+				      vport_num, 1,
+				      vport->info.link_state);
+
+	/* Host PF has its own mac/guid. */
+	if (vport_num) {
+		mlx5_modify_nic_vport_mac_address(esw->dev, vport_num,
+						  vport->info.mac);
+		mlx5_modify_nic_vport_node_guid(esw->dev, vport_num,
+						vport->info.node_guid);
+	}
+
+	flags = (vport->info.vlan || vport->info.qos) ?
+		SET_VLAN_STRIP | SET_VLAN_INSERT : 0;
+	modify_esw_vport_cvlan(esw->dev, vport_num, vport->info.vlan,
+			       vport->info.qos, flags);
+
+	return 0;
+}
+
+/* Don't cleanup vport->info, it's needed to restore vport configuration */
+static void esw_vport_cleanup(struct mlx5_eswitch *esw, struct mlx5_vport *vport)
+{
+	u16 vport_num = vport->vport;
+
+	if (!mlx5_esw_is_manager_vport(esw, vport_num))
+		mlx5_modify_vport_admin_state(esw->dev,
+					      MLX5_VPORT_STATE_OP_MOD_ESW_VPORT,
+					      vport_num, 1,
+					      MLX5_VPORT_ADMIN_STATE_DOWN);
+
+	esw_vport_disable_qos(esw, vport);
+	esw_vport_cleanup_acl(esw, vport);
+}
+
 static int esw_enable_vport(struct mlx5_eswitch *esw, struct mlx5_vport *vport,
 			    enum mlx5_eswitch_vport_event enabled_events)
 {
@@ -1804,16 +1817,10 @@ static int esw_enable_vport(struct mlx5_eswitch *esw, struct mlx5_vport *vport,
 
 	esw_debug(esw->dev, "Enabling VPORT(%d)\n", vport_num);
 
-	/* Restore old vport configuration */
-	esw_vport_setup(esw, vport);
-
-	ret = esw_vport_setup_acl(esw, vport);
+	ret = esw_vport_setup(esw, vport);
 	if (ret)
 		goto done;
 
-	/* Attach vport to the eswitch rate limiter */
-	esw_vport_enable_qos(esw, vport, vport->info.max_rate, vport->qos.bw_share);
-
 	/* Sync with current vport context */
 	vport->enabled_events = enabled_events;
 	vport->enabled = true;
@@ -1855,9 +1862,7 @@ static void esw_disable_vport(struct mlx5_eswitch *esw,
 	 */
 	esw_vport_change_handle_locked(vport);
 	vport->enabled_events = 0;
-	esw_vport_disable_qos(esw, vport);
 	esw_vport_cleanup(esw, vport);
-	esw_vport_cleanup_acl(esw, vport);
 	esw->enabled_vports--;
 
 done:
Signature · 1 line
2.24.1
#6 lore raw

PATCH 6/14 net/mlx5: E-Switch, Introduce per vport configuration for eswitch modes

In reply to Saeed Mahameed

From: Bodong Wang <[email protected]>

Both legacy and offload modes require vport setup, only offload mode requires rep setup. Before this patch, vport and rep operations are separated applied to all relevant vports in different stages.

Change to use per vport configuration, so that vport and rep operations are modularized per vport.

---
 .../net/ethernet/mellanox/mlx5/core/eswitch.c |  73 ++++++---
 .../net/ethernet/mellanox/mlx5/core/eswitch.h |   3 +
 .../mellanox/mlx5/core/eswitch_offloads.c     | 143 +++++++-----------
 3 files changed, 111 insertions(+), 108 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/eswitch.c +50 -23
index 603286883550..2f556c820230 100644
@@ -1806,12 +1806,14 @@ static void esw_vport_cleanup(struct mlx5_eswitch *esw, struct mlx5_vport *vport
 	esw_vport_cleanup_acl(esw, vport);
 }
 
-static int esw_enable_vport(struct mlx5_eswitch *esw, struct mlx5_vport *vport,
+static int esw_enable_vport(struct mlx5_eswitch *esw, u16 vport_num,
 			    enum mlx5_eswitch_vport_event enabled_events)
 {
-	u16 vport_num = vport->vport;
+	struct mlx5_vport *vport;
 	int ret;
 
+	vport = mlx5_eswitch_get_vport(esw, vport_num);
+
 	mutex_lock(&esw->state_lock);
 	WARN_ON(vport->enabled);
 
@@ -1841,10 +1843,11 @@ static int esw_enable_vport(struct mlx5_eswitch *esw, struct mlx5_vport *vport,
 	return ret;
 }
 
-static void esw_disable_vport(struct mlx5_eswitch *esw,
-			      struct mlx5_vport *vport)
+static void esw_disable_vport(struct mlx5_eswitch *esw, u16 vport_num)
 {
-	u16 vport_num = vport->vport;
+	struct mlx5_vport *vport;
+
+	vport = mlx5_eswitch_get_vport(esw, vport_num);
 
 	mutex_lock(&esw->state_lock);
 	if (!vport->enabled)
@@ -1950,6 +1953,32 @@ static void mlx5_eswitch_clear_vf_vports_info(struct mlx5_eswitch *esw)
 /* Public E-Switch API */
 #define ESW_ALLOWED(esw) ((esw) && MLX5_ESWITCH_MANAGER((esw)->dev))
 
+static int mlx5_eswitch_load_vport(struct mlx5_eswitch *esw, u16 vport_num,
+				   enum mlx5_eswitch_vport_event enabled_events)
+{
+	int err;
+
+	err = esw_enable_vport(esw, vport_num, enabled_events);
+	if (err)
+		return err;
+
+	err = esw_offloads_load_rep(esw, vport_num);
+	if (err)
+		goto err_rep;
+
+	return err;
+
+err_rep:
+	esw_disable_vport(esw, vport_num);
+	return err;
+}
+
+static void mlx5_eswitch_unload_vport(struct mlx5_eswitch *esw, u16 vport_num)
+{
+	esw_offloads_unload_rep(esw, vport_num);
+	esw_disable_vport(esw, vport_num);
+}
+
 /* mlx5_eswitch_enable_pf_vf_vports() enables vports of PF, ECPF and VFs
  * whichever are present on the eswitch.
  */
@@ -1957,28 +1986,25 @@ int
 mlx5_eswitch_enable_pf_vf_vports(struct mlx5_eswitch *esw,
 				 enum mlx5_eswitch_vport_event enabled_events)
 {
-	struct mlx5_vport *vport;
 	int num_vfs;
 	int ret;
 	int i;
 
 	/* Enable PF vport */
-	vport = mlx5_eswitch_get_vport(esw, MLX5_VPORT_PF);
-	ret = esw_enable_vport(esw, vport, enabled_events);
+	ret = mlx5_eswitch_load_vport(esw, MLX5_VPORT_PF, enabled_events);
 	if (ret)
 		return ret;
 
 	/* Enable ECPF vport */
 	if (mlx5_ecpf_vport_exists(esw->dev)) {
-		vport = mlx5_eswitch_get_vport(esw, MLX5_VPORT_ECPF);
-		ret = esw_enable_vport(esw, vport, enabled_events);
+		ret = mlx5_eswitch_load_vport(esw, MLX5_VPORT_ECPF, enabled_events);
 		if (ret)
 			goto ecpf_err;
 	}
 
 	/* Enable VF vports */
-	mlx5_esw_for_each_vf_vport(esw, i, vport, esw->esw_funcs.num_vfs) {
-		ret = esw_enable_vport(esw, vport, enabled_events);
+	mlx5_esw_for_each_vf_vport_num(esw, i, esw->esw_funcs.num_vfs) {
+		ret = mlx5_eswitch_load_vport(esw, i, enabled_events);
 		if (ret)
 			goto vf_err;
 	}
@@ -1986,17 +2012,14 @@ mlx5_eswitch_enable_pf_vf_vports(struct mlx5_eswitch *esw,
 
 vf_err:
 	num_vfs = i - 1;
-	mlx5_esw_for_each_vf_vport_reverse(esw, i, vport, num_vfs)
-		esw_disable_vport(esw, vport);
+	mlx5_esw_for_each_vf_vport_num_reverse(esw, i, num_vfs)
+		mlx5_eswitch_unload_vport(esw, i);
 
-	if (mlx5_ecpf_vport_exists(esw->dev)) {
-		vport = mlx5_eswitch_get_vport(esw, MLX5_VPORT_ECPF);
-		esw_disable_vport(esw, vport);
-	}
+	if (mlx5_ecpf_vport_exists(esw->dev))
+		mlx5_eswitch_unload_vport(esw, MLX5_VPORT_ECPF);
 
 ecpf_err:
-	vport = mlx5_eswitch_get_vport(esw, MLX5_VPORT_PF);
-	esw_disable_vport(esw, vport);
+	mlx5_eswitch_unload_vport(esw, MLX5_VPORT_PF);
 	return ret;
 }
 
@@ -2005,11 +2028,15 @@ mlx5_eswitch_enable_pf_vf_vports(struct mlx5_eswitch *esw,
  */
 void mlx5_eswitch_disable_pf_vf_vports(struct mlx5_eswitch *esw)
 {
-	struct mlx5_vport *vport;
 	int i;
 
-	mlx5_esw_for_all_vports_reverse(esw, i, vport)
-		esw_disable_vport(esw, vport);
+	mlx5_esw_for_each_vf_vport_num_reverse(esw, i, esw->esw_funcs.num_vfs)
+		mlx5_eswitch_unload_vport(esw, i);
+
+	if (mlx5_ecpf_vport_exists(esw->dev))
+		mlx5_eswitch_unload_vport(esw, MLX5_VPORT_ECPF);
+
+	mlx5_eswitch_unload_vport(esw, MLX5_VPORT_PF);
 }
 
 static void mlx5_eswitch_get_devlink_param(struct mlx5_eswitch *esw)
drivers/net/ethernet/mellanox/mlx5/core/eswitch.h +3 -0
index 2e0417dd8ce3..1213a69cf397 100644
@@ -651,6 +651,9 @@ esw_add_restore_rule(struct mlx5_eswitch *esw, u32 tag);
 u32
 esw_get_max_restore_tag(struct mlx5_eswitch *esw);
 
+int esw_offloads_load_rep(struct mlx5_eswitch *esw, u16 vport_num);
+void esw_offloads_unload_rep(struct mlx5_eswitch *esw, u16 vport_num);
+
 #else  /* CONFIG_MLX5_ESWITCH */
 /* eswitch API stubs */
 static inline int  mlx5_eswitch_init(struct mlx5_core_dev *dev) { return 0; }
drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c +58 -85

211 lines not on this page · raw message

Signature · 1 line
2.24.1
#7 lore raw

PATCH 7/14 net/mlx5: E-Switch, Update VF vports config when num of VFs changed

In reply to Saeed Mahameed

From: Bodong Wang <[email protected]>

Currently, ECPF eswitch manager does one-time only configuration for VF vports when device switches to offloads mode. However, when num of VFs changed from host side, driver doesn't update VF vports configurations.

Hence, perform VFs vport configuration update whenever num_vfs change event occurs.

---
 .../net/ethernet/mellanox/mlx5/core/eswitch.c | 53 ++++++++++-----
 .../net/ethernet/mellanox/mlx5/core/eswitch.h |  8 +++
 .../mellanox/mlx5/core/eswitch_offloads.c     | 67 +------------------
 3 files changed, 46 insertions(+), 82 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/eswitch.c +35 -18
index 2f556c820230..1de2472a72e7 100644
@@ -1953,8 +1953,8 @@ static void mlx5_eswitch_clear_vf_vports_info(struct mlx5_eswitch *esw)
 /* Public E-Switch API */
 #define ESW_ALLOWED(esw) ((esw) && MLX5_ESWITCH_MANAGER((esw)->dev))
 
-static int mlx5_eswitch_load_vport(struct mlx5_eswitch *esw, u16 vport_num,
-				   enum mlx5_eswitch_vport_event enabled_events)
+int mlx5_eswitch_load_vport(struct mlx5_eswitch *esw, u16 vport_num,
+			    enum mlx5_eswitch_vport_event enabled_events)
 {
 	int err;
 
@@ -1973,12 +1973,39 @@ static int mlx5_eswitch_load_vport(struct mlx5_eswitch *esw, u16 vport_num,
 	return err;
 }
 
-static void mlx5_eswitch_unload_vport(struct mlx5_eswitch *esw, u16 vport_num)
+void mlx5_eswitch_unload_vport(struct mlx5_eswitch *esw, u16 vport_num)
 {
 	esw_offloads_unload_rep(esw, vport_num);
 	esw_disable_vport(esw, vport_num);
 }
 
+void mlx5_eswitch_unload_vf_vports(struct mlx5_eswitch *esw, u16 num_vfs)
+{
+	int i;
+
+	mlx5_esw_for_each_vf_vport_num_reverse(esw, i, num_vfs)
+		mlx5_eswitch_unload_vport(esw, i);
+}
+
+int mlx5_eswitch_load_vf_vports(struct mlx5_eswitch *esw, u16 num_vfs,
+				enum mlx5_eswitch_vport_event enabled_events)
+{
+	int err;
+	int i;
+
+	mlx5_esw_for_each_vf_vport_num(esw, i, num_vfs) {
+		err = mlx5_eswitch_load_vport(esw, i, enabled_events);
+		if (err)
+			goto vf_err;
+	}
+
+	return 0;
+
+vf_err:
+	mlx5_eswitch_unload_vf_vports(esw, i - 1);
+	return err;
+}
+
 /* mlx5_eswitch_enable_pf_vf_vports() enables vports of PF, ECPF and VFs
  * whichever are present on the eswitch.
  */
@@ -1986,9 +2013,7 @@ int
 mlx5_eswitch_enable_pf_vf_vports(struct mlx5_eswitch *esw,
 				 enum mlx5_eswitch_vport_event enabled_events)
 {
-	int num_vfs;
 	int ret;
-	int i;
 
 	/* Enable PF vport */
 	ret = mlx5_eswitch_load_vport(esw, MLX5_VPORT_PF, enabled_events);
@@ -2003,18 +2028,13 @@ mlx5_eswitch_enable_pf_vf_vports(struct mlx5_eswitch *esw,
 	}
 
 	/* Enable VF vports */
-	mlx5_esw_for_each_vf_vport_num(esw, i, esw->esw_funcs.num_vfs) {
-		ret = mlx5_eswitch_load_vport(esw, i, enabled_events);
-		if (ret)
-			goto vf_err;
-	}
+	ret = mlx5_eswitch_load_vf_vports(esw, esw->esw_funcs.num_vfs,
+					  enabled_events);
+	if (ret)
+		goto vf_err;
 	return 0;
 
 vf_err:
-	num_vfs = i - 1;
-	mlx5_esw_for_each_vf_vport_num_reverse(esw, i, num_vfs)
-		mlx5_eswitch_unload_vport(esw, i);
-
 	if (mlx5_ecpf_vport_exists(esw->dev))
 		mlx5_eswitch_unload_vport(esw, MLX5_VPORT_ECPF);
 
@@ -2028,10 +2048,7 @@ mlx5_eswitch_enable_pf_vf_vports(struct mlx5_eswitch *esw,
  */
 void mlx5_eswitch_disable_pf_vf_vports(struct mlx5_eswitch *esw)
 {
-	int i;
-
-	mlx5_esw_for_each_vf_vport_num_reverse(esw, i, esw->esw_funcs.num_vfs)
-		mlx5_eswitch_unload_vport(esw, i);
+	mlx5_eswitch_unload_vf_vports(esw, esw->esw_funcs.num_vfs);
 
 	if (mlx5_ecpf_vport_exists(esw->dev))
 		mlx5_eswitch_unload_vport(esw, MLX5_VPORT_ECPF);
drivers/net/ethernet/mellanox/mlx5/core/eswitch.h +8 -0
index 1213a69cf397..91b2aedcf52b 100644
@@ -654,6 +654,14 @@ esw_get_max_restore_tag(struct mlx5_eswitch *esw);
 int esw_offloads_load_rep(struct mlx5_eswitch *esw, u16 vport_num);
 void esw_offloads_unload_rep(struct mlx5_eswitch *esw, u16 vport_num);
 
+int mlx5_eswitch_load_vport(struct mlx5_eswitch *esw, u16 vport_num,
+			    enum mlx5_eswitch_vport_event enabled_events);
+void mlx5_eswitch_unload_vport(struct mlx5_eswitch *esw, u16 vport_num);
+
+int mlx5_eswitch_load_vf_vports(struct mlx5_eswitch *esw, u16 num_vfs,
+				enum mlx5_eswitch_vport_event enabled_events);
+void mlx5_eswitch_unload_vf_vports(struct mlx5_eswitch *esw, u16 num_vfs);
+
 #else  /* CONFIG_MLX5_ESWITCH */
 /* eswitch API stubs */
 static inline int  mlx5_eswitch_init(struct mlx5_core_dev *dev) { return 0; }
drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c +3 -64

88 lines not on this page · raw message

Signature · 1 line
2.24.1
maintainer
#8 lore raw

Re: [net-next 07/14] net/mlx5: E-Switch, Update VF vports config when num of VFs changed

Quoted 9 lines · On Fri, 13 Mar 2020 18:16:15 -0700 Saeed Mahameed wrote:

From: Bodong Wang <[email protected]>

Currently, ECPF eswitch manager does one-time only configuration for VF vports when device switches to offloads mode. However, when num of VFs changed from host side, driver doesn't update VF vports configurations.

Hence, perform VFs vport configuration update whenever num_vfs change event occurs.

Oh, I thought you kept max_vfs number of reprs on the ECPF, always. Or was that just the initial plan?

#9 lore raw

Re: [net-next 07/14] net/mlx5: E-Switch, Update VF vports config when num of VFs changed

Quoted 10 lines · Jakub Kicinski
Quoted 9 lines · On Fri, 13 Mar 2020 18:16:15 -0700 Saeed Mahameed wrote:

From: Bodong Wang <[email protected]>

Currently, ECPF eswitch manager does one-time only configuration for
VF vports when device switches to offloads mode. However, when num of
VFs changed from host side, driver doesn't update VF vports configurations.

Quoted 6 lines
Quoted 3 lines

Hence, perform VFs vport configuration update whenever num_vfs change
event occurs.

Oh, I thought you kept max_vfs number of reprs on the ECPF, always. Or was that just the initial plan?

No, Smartnic's ECPF switchdev was required to be loaded only after host PF enables sriov, so it will get the correct num_vfs and initialize eswitch and reps accordingly.

3 reasons to change this:
1. Remove the dependency of when to load the switchdev mode in ECPF and make the admin life easier
2. better code dynamics and vport management.
3. pave the way for future support of dynamically created Sub- functions..

#10 lore raw

PATCH 8/14 net/mlx5: E-Switch, Refactor unload all reps per rep type

In reply to Saeed Mahameed

From: Bodong Wang <[email protected]>

Following introduction of per vport configuration of vport and rep, unload all reps per rep type is still needed as IB reps can be unloaded individually. However, a few internal functions exist purely for this purpose, merge them to a single function.

This patch doesn't change any existing functionality.

---
 .../mellanox/mlx5/core/eswitch_offloads.c     | 24 ++++---------------
 1 file changed, 5 insertions(+), 19 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c +5 -19
index badae90206ac..aedbb026ed99 100644
@@ -1634,9 +1634,13 @@ static void __esw_offloads_unload_rep(struct mlx5_eswitch *esw,
 		esw->offloads.rep_ops[rep_type]->unload(rep);
 }
 
-static void __unload_reps_special_vport(struct mlx5_eswitch *esw, u8 rep_type)
+static void __unload_reps_all_vport(struct mlx5_eswitch *esw, u8 rep_type)
 {
 	struct mlx5_eswitch_rep *rep;
+	int i;
+
+	mlx5_esw_for_each_vf_rep_reverse(esw, i, rep, esw->esw_funcs.num_vfs)
+		__esw_offloads_unload_rep(esw, rep, rep_type);
 
 	if (mlx5_ecpf_vport_exists(esw->dev)) {
 		rep = mlx5_eswitch_get_rep(esw, MLX5_VPORT_ECPF);
@@ -1652,24 +1656,6 @@ static void __unload_reps_special_vport(struct mlx5_eswitch *esw, u8 rep_type)
 	__esw_offloads_unload_rep(esw, rep, rep_type);
 }
 
-static void __unload_reps_vf_vport(struct mlx5_eswitch *esw, int nvports,
-				   u8 rep_type)
-{
-	struct mlx5_eswitch_rep *rep;
-	int i;
-
-	mlx5_esw_for_each_vf_rep_reverse(esw, i, rep, nvports)
-		__esw_offloads_unload_rep(esw, rep, rep_type);
-}
-
-static void __unload_reps_all_vport(struct mlx5_eswitch *esw, u8 rep_type)
-{
-	__unload_reps_vf_vport(esw, esw->esw_funcs.num_vfs, rep_type);
-
-	/* Special vports must be the last to unload. */
-	__unload_reps_special_vport(esw, rep_type);
-}
-
 int esw_offloads_load_rep(struct mlx5_eswitch *esw, u16 vport_num)
 {
 	struct mlx5_eswitch_rep *rep;
Signature · 1 line
2.24.1
#11 lore raw

PATCH 9/14 net/mlx5: Accept flow rules without match

In reply to Saeed Mahameed

From: Mark Bloch <[email protected]>

Allow passing NULL spec when creating a flow rule. Such rules will act as "catch all" flow rules.

---
 .../net/ethernet/mellanox/mlx5/core/en_arfs.c   | 15 +++------------
 .../net/ethernet/mellanox/mlx5/core/eswitch.c   | 17 ++++-------------
 .../mellanox/mlx5/core/eswitch_offloads.c       |  3 +--
 .../mlx5/core/eswitch_offloads_chains.c         |  3 +--
 .../mlx5/core/eswitch_offloads_termtbl.c        |  3 +--
 .../net/ethernet/mellanox/mlx5/core/fs_core.c   |  4 ++++
 6 files changed, 14 insertions(+), 31 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en_arfs.c +3 -12
index 2c75b2752f58..014639ea06e3 100644
@@ -175,28 +175,20 @@ static int arfs_add_default_rule(struct mlx5e_priv *priv,
 	struct mlx5e_tir *tir = priv->indir_tir;
 	struct mlx5_flow_destination dest = {};
 	MLX5_DECLARE_FLOW_ACT(flow_act);
-	struct mlx5_flow_spec *spec;
 	enum mlx5e_traffic_types tt;
 	int err = 0;
 
-	spec = kvzalloc(sizeof(*spec), GFP_KERNEL);
-	if (!spec) {
-		err = -ENOMEM;
-		goto out;
-	}
-
 	dest.type = MLX5_FLOW_DESTINATION_TYPE_TIR;
 	tt = arfs_get_tt(type);
 	if (tt == -EINVAL) {
 		netdev_err(priv->netdev, "%s: bad arfs_type: %d\n",
 			   __func__, type);
-		err = -EINVAL;
-		goto out;
+		return -EINVAL;
 	}
 
 	dest.tir_num = tir[tt].tirn;
 
-	arfs_t->default_rule = mlx5_add_flow_rules(arfs_t->ft.t, spec,
+	arfs_t->default_rule = mlx5_add_flow_rules(arfs_t->ft.t, NULL,
 						   &flow_act,
 						   &dest, 1);
 	if (IS_ERR(arfs_t->default_rule)) {
@@ -205,8 +197,7 @@ static int arfs_add_default_rule(struct mlx5e_priv *priv,
 		netdev_err(priv->netdev, "%s: add rule failed, arfs type=%d\n",
 			   __func__, type);
 	}
-out:
-	kvfree(spec);
+
 	return err;
 }
 
drivers/net/ethernet/mellanox/mlx5/core/eswitch.c +4 -13
index 1de2472a72e7..54e5334f02a7 100644
@@ -1334,7 +1334,6 @@ static int esw_vport_ingress_config(struct mlx5_eswitch *esw,
 		goto out;
 	}
 
-	memset(spec, 0, sizeof(*spec));
 	flow_act.action = MLX5_FLOW_CONTEXT_ACTION_DROP;
 
 	/* Attach drop flow counter */
@@ -1346,7 +1345,7 @@ static int esw_vport_ingress_config(struct mlx5_eswitch *esw,
 		dest_num++;
 	}
 	vport->ingress.legacy.drop_rule =
-		mlx5_add_flow_rules(vport->ingress.acl, spec,
+		mlx5_add_flow_rules(vport->ingress.acl, NULL,
 				    &flow_act, dst, dest_num);
 	if (IS_ERR(vport->ingress.legacy.drop_rule)) {
 		err = PTR_ERR(vport->ingress.legacy.drop_rule);
@@ -1409,7 +1408,6 @@ static int esw_vport_egress_config(struct mlx5_eswitch *esw,
 	struct mlx5_flow_destination drop_ctr_dst = {0};
 	struct mlx5_flow_destination *dst = NULL;
 	struct mlx5_flow_act flow_act = {0};
-	struct mlx5_flow_spec *spec;
 	int dest_num = 0;
 	int err = 0;
 
@@ -1438,11 +1436,6 @@ static int esw_vport_egress_config(struct mlx5_eswitch *esw,
 	if (err)
 		return err;
 
-	/* Drop others rule (star rule) */
-	spec = kvzalloc(sizeof(*spec), GFP_KERNEL);
-	if (!spec)
-		goto out;
-
 	flow_act.action = MLX5_FLOW_CONTEXT_ACTION_DROP;
 
 	/* Attach egress drop flow counter */
@@ -1454,7 +1447,7 @@ static int esw_vport_egress_config(struct mlx5_eswitch *esw,
 		dest_num++;
 	}
 	vport->egress.legacy.drop_rule =
-		mlx5_add_flow_rules(vport->egress.acl, spec,
+		mlx5_add_flow_rules(vport->egress.acl, NULL,
 				    &flow_act, dst, dest_num);
 	if (IS_ERR(vport->egress.legacy.drop_rule)) {
 		err = PTR_ERR(vport->egress.legacy.drop_rule);
@@ -1463,8 +1456,7 @@ static int esw_vport_egress_config(struct mlx5_eswitch *esw,
 			 vport->vport, err);
 		vport->egress.legacy.drop_rule = NULL;
 	}
-out:
-	kvfree(spec);
+
 	return err;
 }
 
@@ -2481,12 +2473,11 @@ static int _mlx5_eswitch_set_vepa_locked(struct mlx5_eswitch *esw,
 	}
 
 	/* Star rule to forward all traffic to uplink vport */
-	memset(spec, 0, sizeof(*spec));
 	memset(&dest, 0, sizeof(dest));
 	dest.type = MLX5_FLOW_DESTINATION_TYPE_VPORT;
 	dest.vport.num = MLX5_VPORT_UPLINK;
 	flow_act.action = MLX5_FLOW_CONTEXT_ACTION_FWD_DEST;
-	flow_rule = mlx5_add_flow_rules(esw->fdb_table.legacy.vepa_fdb, spec,
+	flow_rule = mlx5_add_flow_rules(esw->fdb_table.legacy.vepa_fdb, NULL,
 					&flow_act, &dest, 1);
 	if (IS_ERR(flow_rule)) {
 		err = PTR_ERR(flow_rule);
drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c +1 -2
index aedbb026ed99..8ff52e237bcb 100644
@@ -1881,7 +1881,6 @@ static int esw_vport_add_ingress_acl_modify_metadata(struct mlx5_eswitch *esw,
 						     struct mlx5_vport *vport)
 {
 	u8 action[MLX5_UN_SZ_BYTES(set_action_in_add_action_in_auto)] = {};
-	static const struct mlx5_flow_spec spec = {};
 	struct mlx5_flow_act flow_act = {};
 	int err = 0;
 	u32 key;
@@ -1913,7 +1912,7 @@ static int esw_vport_add_ingress_acl_modify_metadata(struct mlx5_eswitch *esw,
 	flow_act.modify_hdr = vport->ingress.offloads.modify_metadata;
 	vport->ingress.offloads.modify_metadata_rule =
 				mlx5_add_flow_rules(vport->ingress.acl,
-						    &spec, &flow_act, NULL, 0);
+						    NULL, &flow_act, NULL, 0);
 	if (IS_ERR(vport->ingress.offloads.modify_metadata_rule)) {
 		err = PTR_ERR(vport->ingress.offloads.modify_metadata_rule);
 		esw_warn(esw->dev,
drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads_chains.c +1 -2
index 0702c216a031..81421d4fb18d 100644
@@ -410,7 +410,6 @@ mlx5_esw_chains_add_miss_rule(struct fdb_chain *fdb_chain,
 			      struct mlx5_flow_table *fdb,
 			      struct mlx5_flow_table *next_fdb)
 {
-	static const struct mlx5_flow_spec spec = {};
 	struct mlx5_eswitch *esw = fdb_chain->esw;
 	struct mlx5_flow_destination dest = {};
 	struct mlx5_flow_act act = {};
@@ -425,7 +424,7 @@ mlx5_esw_chains_add_miss_rule(struct fdb_chain *fdb_chain,
 		act.action |= MLX5_FLOW_CONTEXT_ACTION_MOD_HDR;
 	}
 
-	return mlx5_add_flow_rules(fdb, &spec, &act, &dest, 1);
+	return mlx5_add_flow_rules(fdb, NULL, &act, &dest, 1);
 }
 
 static int
drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads_termtbl.c +1 -2
index f3a925e5ba88..269eddc3d38b 100644
@@ -49,7 +49,6 @@ mlx5_eswitch_termtbl_create(struct mlx5_core_dev *dev,
 			    struct mlx5_termtbl_handle *tt,
 			    struct mlx5_flow_act *flow_act)
 {
-	static const struct mlx5_flow_spec spec = {};
 	struct mlx5_flow_table_attr ft_attr = {};
 	struct mlx5_flow_namespace *root_ns;
 	int err;
@@ -73,7 +72,7 @@ mlx5_eswitch_termtbl_create(struct mlx5_core_dev *dev,
 		return -EOPNOTSUPP;
 	}
 
-	tt->rule = mlx5_add_flow_rules(tt->termtbl, &spec, flow_act,
+	tt->rule = mlx5_add_flow_rules(tt->termtbl, NULL, flow_act,
 				       &tt->dest, 1);
 
 	if (IS_ERR(tt->rule)) {
drivers/net/ethernet/mellanox/mlx5/core/fs_core.c +4 -0
index bd0b2e4f3446..c93bd55fab06 100644
@@ -1892,12 +1892,16 @@ mlx5_add_flow_rules(struct mlx5_flow_table *ft,
 		    int num_dest)
 {
 	struct mlx5_flow_root_namespace *root = find_root(&ft->node);
+	static const struct mlx5_flow_spec zero_spec = {};
 	struct mlx5_flow_destination gen_dest = {};
 	struct mlx5_flow_table *next_ft = NULL;
 	struct mlx5_flow_handle *handle = NULL;
 	u32 sw_action = flow_act->action;
 	struct fs_prio *prio;
 
+	if (!spec)
+		spec = &zero_spec;
+
 	fs_get_obj(prio, ft->node.parent);
 	if (flow_act->action == MLX5_FLOW_CONTEXT_ACTION_FWD_NEXT_PRIO) {
 		if (!fwd_next_prio_supported(ft))
Signature · 1 line
2.24.1
#12 lore raw

PATCH 10/14 net/mlx5: E-switch, Annotate termtbl_mutex mutex destroy

In reply to Saeed Mahameed

From: Parav Pandit <[email protected]>

Annotate mutex destroy to keep it symmetric to init sequence. It should be destroyed after its users (representor netdevices) are destroyed in below flow.

esw_offloads_disable()
esw_offloads_unload_rep()

Hence, initialize the mutex before creating the representors which uses it.

---
 drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c +3 -1
index 8ff52e237bcb..5b05dec75808 100644
@@ -2332,6 +2332,7 @@ int esw_offloads_enable(struct mlx5_eswitch *esw)
 	else
 		esw->offloads.encap = DEVLINK_ESWITCH_ENCAP_MODE_NONE;
 
+	mutex_init(&esw->offloads.termtbl_mutex);
 	mlx5_rdma_enable_roce(esw->dev);
 	err = esw_offloads_steering_init(esw);
 	if (err)
@@ -2355,7 +2356,6 @@ int esw_offloads_enable(struct mlx5_eswitch *esw)
 		goto err_vports;
 
 	esw_offloads_devcom_init(esw);
-	mutex_init(&esw->offloads.termtbl_mutex);
 
 	return 0;
 
@@ -2367,6 +2367,7 @@ int esw_offloads_enable(struct mlx5_eswitch *esw)
 	esw_offloads_steering_cleanup(esw);
 err_steering_init:
 	mlx5_rdma_disable_roce(esw->dev);
+	mutex_destroy(&esw->offloads.termtbl_mutex);
 	return err;
 }
 
@@ -2397,6 +2398,7 @@ void esw_offloads_disable(struct mlx5_eswitch *esw)
 	esw_set_passing_vport_metadata(esw, false);
 	esw_offloads_steering_cleanup(esw);
 	mlx5_rdma_disable_roce(esw->dev);
+	mutex_destroy(&esw->offloads.termtbl_mutex);
 	esw->offloads.encap = DEVLINK_ESWITCH_ENCAP_MODE_NONE;
 }
 
Signature · 1 line
2.24.1
#13 lore raw

PATCH 11/14 net/mlx5: E-switch, Annotate esw state_lock mutex destroy

In reply to Saeed Mahameed

From: Parav Pandit <[email protected]>

Invoke mutex_destroy() to catch any esw state_lock errors.

---
 drivers/net/ethernet/mellanox/mlx5/core/eswitch.c | 1 +
 1 file changed, 1 insertion(+)
drivers/net/ethernet/mellanox/mlx5/core/eswitch.c +1 -0
index 54e5334f02a7..8fc351240f4c 100644
@@ -2239,6 +2239,7 @@ void mlx5_eswitch_cleanup(struct mlx5_eswitch *esw)
 	esw->dev->priv.eswitch = NULL;
 	destroy_workqueue(esw->work_queue);
 	esw_offloads_cleanup_reps(esw);
+	mutex_destroy(&esw->state_lock);
 	mutex_destroy(&esw->offloads.mod_hdr.lock);
 	mutex_destroy(&esw->offloads.encap_tbl_lock);
 	kfree(esw->vports);
Signature · 1 line
2.24.1
#14 lore raw

PATCH 12/14 net/mlx5: Avoid deriving mlx5_core_dev second time

In reply to Saeed Mahameed

From: Parav Pandit <[email protected]>

All callers needs to work on mlx5_core_dev and it is already derived before calling mlx5_devlink_eswitch_check().
Hence, accept mlx5_core_dev in mlx5_devlink_eswitch_check().

Given that it works on mlx5_core_dev change helper function name to drop devlink prefix.

---
 .../mellanox/mlx5/core/eswitch_offloads.c        | 16 +++++++---------
 1 file changed, 7 insertions(+), 9 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c +7 -9
index 5b05dec75808..e2a906085a98 100644
@@ -2478,10 +2478,8 @@ static int esw_inline_mode_to_devlink(u8 mlx5_mode, u8 *mode)
 	return 0;
 }
 
-static int mlx5_devlink_eswitch_check(struct devlink *devlink)
+static int mlx5_eswitch_check(const struct mlx5_core_dev *dev)
 {
-	struct mlx5_core_dev *dev = devlink_priv(devlink);
-
 	if (MLX5_CAP_GEN(dev, port_type) != MLX5_CAP_PORT_TYPE_ETH)
 		return -EOPNOTSUPP;
 
@@ -2502,7 +2500,7 @@ int mlx5_devlink_eswitch_mode_set(struct devlink *devlink, u16 mode,
 	u16 cur_mlx5_mode, mlx5_mode = 0;
 	int err;
 
-	err = mlx5_devlink_eswitch_check(devlink);
+	err = mlx5_eswitch_check(dev);
 	if (err)
 		return err;
 
@@ -2527,7 +2525,7 @@ int mlx5_devlink_eswitch_mode_get(struct devlink *devlink, u16 *mode)
 	struct mlx5_core_dev *dev = devlink_priv(devlink);
 	int err;
 
-	err = mlx5_devlink_eswitch_check(devlink);
+	err = mlx5_eswitch_check(dev);
 	if (err)
 		return err;
 
@@ -2542,7 +2540,7 @@ int mlx5_devlink_eswitch_inline_mode_set(struct devlink *devlink, u8 mode,
 	int err, vport, num_vport;
 	u8 mlx5_mode;
 
-	err = mlx5_devlink_eswitch_check(devlink);
+	err = mlx5_eswitch_check(dev);
 	if (err)
 		return err;
 
@@ -2596,7 +2594,7 @@ int mlx5_devlink_eswitch_inline_mode_get(struct devlink *devlink, u8 *mode)
 	struct mlx5_eswitch *esw = dev->priv.eswitch;
 	int err;
 
-	err = mlx5_devlink_eswitch_check(devlink);
+	err = mlx5_eswitch_check(dev);
 	if (err)
 		return err;
 
@@ -2611,7 +2609,7 @@ int mlx5_devlink_eswitch_encap_mode_set(struct devlink *devlink,
 	struct mlx5_eswitch *esw = dev->priv.eswitch;
 	int err;
 
-	err = mlx5_devlink_eswitch_check(devlink);
+	err = mlx5_eswitch_check(dev);
 	if (err)
 		return err;
 
@@ -2660,7 +2658,7 @@ int mlx5_devlink_eswitch_encap_mode_get(struct devlink *devlink,
 	struct mlx5_eswitch *esw = dev->priv.eswitch;
 	int err;
 
-	err = mlx5_devlink_eswitch_check(devlink);
+	err = mlx5_eswitch_check(dev);
 	if (err)
 		return err;
 
Signature · 1 line
2.24.1
#15 lore raw

PATCH 13/14 net/mlx5: DR, Add support for flow table id destination action

In reply to Saeed Mahameed

From: Alex Vesker <[email protected]>

This action allows to go to a flow table based on the table id. Goto flow table id is required for supporting user space SW.

---
 .../mellanox/mlx5/core/steering/dr_action.c    | 18 ++++++++++++++++++
 .../mellanox/mlx5/core/steering/fs_dr.c        | 12 ++++++++++++
 .../mellanox/mlx5/core/steering/mlx5dr.h       |  3 +++
 3 files changed, 33 insertions(+)
drivers/net/ethernet/mellanox/mlx5/core/steering/dr_action.c +18 -0
index f899da9f8488..4b323a7ae794 100644
@@ -964,6 +964,24 @@ struct mlx5dr_action *mlx5dr_action_create_drop(void)
 	return dr_action_create_generic(DR_ACTION_TYP_DROP);
 }
 
+struct mlx5dr_action *
+mlx5dr_action_create_dest_table_num(struct mlx5dr_domain *dmn, u32 table_num)
+{
+	struct mlx5dr_action *action;
+
+	action = dr_action_create_generic(DR_ACTION_TYP_FT);
+	if (!action)
+		return NULL;
+
+	action->dest_tbl.is_fw_tbl = true;
+	action->dest_tbl.fw_tbl.dmn = dmn;
+	action->dest_tbl.fw_tbl.id = table_num;
+	action->dest_tbl.fw_tbl.type = FS_FT_FDB;
+	refcount_inc(&dmn->refcount);
+
+	return action;
+}
+
 struct mlx5dr_action *
 mlx5dr_action_create_dest_table(struct mlx5dr_table *tbl)
 {
drivers/net/ethernet/mellanox/mlx5/core/steering/fs_dr.c +12 -0
index d12d3a2d46ab..3b3f5b9d4f95 100644
@@ -384,6 +384,7 @@ static int mlx5_cmd_dr_create_fte(struct mlx5_flow_root_namespace *ns,
 	if (fte->action.action & MLX5_FLOW_CONTEXT_ACTION_FWD_DEST) {
 		list_for_each_entry(dst, &fte->node.children, node.list) {
 			enum mlx5_flow_destination_type type = dst->dest_attr.type;
+			u32 ft_id;
 
 			if (num_actions == MLX5_FLOW_CONTEXT_ACTION_MAX ||
 			    num_term_actions >= MLX5_FLOW_CONTEXT_ACTION_MAX) {
@@ -420,6 +421,17 @@ static int mlx5_cmd_dr_create_fte(struct mlx5_flow_root_namespace *ns,
 
 				num_term_actions++;
 				break;
+			case MLX5_FLOW_DESTINATION_TYPE_FLOW_TABLE_NUM:
+				ft_id = dst->dest_attr.ft_num;
+				tmp_action = mlx5dr_action_create_dest_table_num(domain,
+										 ft_id);
+				if (!tmp_action) {
+					err = -ENOMEM;
+					goto free_actions;
+				}
+				fs_dr_actions[fs_dr_num_actions++] = tmp_action;
+				term_actions[num_term_actions++].dest = tmp_action;
+				break;
 			default:
 				err = -EOPNOTSUPP;
 				goto free_actions;
drivers/net/ethernet/mellanox/mlx5/core/steering/mlx5dr.h +3 -0
index e09e4ea1b045..1ee10e3e0d52 100644
@@ -76,6 +76,9 @@ int mlx5dr_rule_destroy(struct mlx5dr_rule *rule);
 int mlx5dr_table_set_miss_action(struct mlx5dr_table *tbl,
 				 struct mlx5dr_action *action);
 
+struct mlx5dr_action *
+mlx5dr_action_create_dest_table_num(struct mlx5dr_domain *dmn, u32 table_num);
+
 struct mlx5dr_action *
 mlx5dr_action_create_dest_table(struct mlx5dr_table *table);
 
Signature · 1 line
2.24.1
maintainer
#16 lore raw

Re: [net-next 13/14] net/mlx5: DR, Add support for flow table id destination action

Quoted 4 lines · On Fri, 13 Mar 2020 18:16:21 -0700 Saeed Mahameed wrote:

From: Alex Vesker <[email protected]>

This action allows to go to a flow table based on the table id. Goto flow table id is required for supporting user space SW.

What's user space SW?

#17 lore raw

Re: [net-next 13/14] net/mlx5: DR, Add support for flow table id destination action

Quoted 7 lines · Jakub Kicinski
Quoted 4 lines · On Fri, 13 Mar 2020 18:16:21 -0700 Saeed Mahameed wrote:

From: Alex Vesker <[email protected]>

This action allows to go to a flow table based on the table id. Goto flow table id is required for supporting user space SW.

What's user space SW?

in short :) :
RDMA DEVX [1] based application.

[1] https://patchwork.ozlabs.org/cover/930449/

RDMA applications using DEVX API can access a mlx5 specially created steering arena/domain for DEVX apps, such apps can program this domain the way they desire while bypassing FW, and they will get lightning speed steering rule insertion rate.

#18 lore raw

Re: [net-next 13/14] net/mlx5: DR, Add support for flow table id destination action

In reply to Jakub Kicinski
Quoted 7 lines · Jakub Kicinski
Quoted 4 lines · On Fri, 13 Mar 2020 18:16:21 -0700 Saeed Mahameed wrote:

From: Alex Vesker <[email protected]>

This action allows to go to a flow table based on the table id. Goto flow table id is required for supporting user space SW.

What's user space SW?

"User space software steering" is a way to add rules to the packet processing. The rules can be written by user space applications and they are executed by the HW.

The rdma-core (RDMA userspace counterpart) is exposing the proper API for that functionality.
https://github.com/linux-rdma/rdma-core/blob/master/providers/mlx5/man/mlx5dv_dr_flow.3.md

Thanks

#19 lore raw

PATCH 14/14 net/mlx5: DR, Remove unneeded functions deceleration

In reply to Saeed Mahameed

From: Alex Vesker <[email protected]>

Remove dummy functions declaration, the dummy functions are not needed since fs_dr is the only one to call mlx5dr and both fs_dr and dr files depend on the same config flag (MLX5_SW_STEERING).

Fixes 70605ea545e8 ("net/mlx5: DR, Expose APIs for direct rule managing")
---
 .../mellanox/mlx5/core/steering/mlx5dr.h      | 101 ------------------
 1 file changed, 101 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/steering/mlx5dr.h +0 -101
index 1ee10e3e0d52..7deaca9ade3b 100644
@@ -38,8 +38,6 @@ struct mlx5dr_action_dest {
 	struct mlx5dr_action *reformat;
 };
 
-#ifdef CONFIG_MLX5_SW_STEERING
-
 struct mlx5dr_domain *
 mlx5dr_domain_create(struct mlx5_core_dev *mdev, enum mlx5dr_domain_type type);
 
@@ -128,103 +126,4 @@ mlx5dr_is_supported(struct mlx5_core_dev *dev)
 	return MLX5_CAP_ESW_FLOWTABLE_FDB(dev, sw_owner);
 }
 
-#else /* CONFIG_MLX5_SW_STEERING */
-
-static inline struct mlx5dr_domain *
-mlx5dr_domain_create(struct mlx5_core_dev *mdev, enum mlx5dr_domain_type type) { return NULL; }
-
-static inline int
-mlx5dr_domain_destroy(struct mlx5dr_domain *domain) { return 0; }
-
-static inline int
-mlx5dr_domain_sync(struct mlx5dr_domain *domain, u32 flags) { return 0; }
-
-static inline void
-mlx5dr_domain_set_peer(struct mlx5dr_domain *dmn,
-		       struct mlx5dr_domain *peer_dmn) { }
-
-static inline struct mlx5dr_table *
-mlx5dr_table_create(struct mlx5dr_domain *domain, u32 level, u32 flags) { return NULL; }
-
-static inline int
-mlx5dr_table_destroy(struct mlx5dr_table *table) { return 0; }
-
-static inline u32
-mlx5dr_table_get_id(struct mlx5dr_table *table) { return 0; }
-
-static inline struct mlx5dr_matcher *
-mlx5dr_matcher_create(struct mlx5dr_table *table,
-		      u32 priority,
-		      u8 match_criteria_enable,
-		      struct mlx5dr_match_parameters *mask) { return NULL; }
-
-static inline int
-mlx5dr_matcher_destroy(struct mlx5dr_matcher *matcher) { return 0; }
-
-static inline struct mlx5dr_rule *
-mlx5dr_rule_create(struct mlx5dr_matcher *matcher,
-		   struct mlx5dr_match_parameters *value,
-		   size_t num_actions,
-		   struct mlx5dr_action *actions[]) { return NULL; }
-
-static inline int
-mlx5dr_rule_destroy(struct mlx5dr_rule *rule) { return 0; }
-
-static inline int
-mlx5dr_table_set_miss_action(struct mlx5dr_table *tbl,
-			     struct mlx5dr_action *action) { return 0; }
-
-static inline struct mlx5dr_action *
-mlx5dr_action_create_dest_table(struct mlx5dr_table *table) { return NULL; }
-
-static inline struct mlx5dr_action *
-mlx5dr_action_create_dest_flow_fw_table(struct mlx5dr_domain *domain,
-					struct mlx5_flow_table *ft) { return NULL; }
-
-static inline struct mlx5dr_action *
-mlx5dr_action_create_dest_vport(struct mlx5dr_domain *domain,
-				u32 vport, u8 vhca_id_valid,
-				u16 vhca_id) { return NULL; }
-
-static inline struct mlx5dr_action *
-mlx5dr_action_create_mult_dest_tbl(struct mlx5dr_domain *dmn,
-				   struct mlx5dr_action_dest *dests,
-				   u32 num_of_dests)  { return NULL; }
-
-static inline struct mlx5dr_action *
-mlx5dr_action_create_drop(void) { return NULL; }
-
-static inline struct mlx5dr_action *
-mlx5dr_action_create_tag(u32 tag_value) { return NULL; }
-
-static inline struct mlx5dr_action *
-mlx5dr_action_create_flow_counter(u32 counter_id) { return NULL; }
-
-static inline struct mlx5dr_action *
-mlx5dr_action_create_packet_reformat(struct mlx5dr_domain *dmn,
-				     enum mlx5dr_action_reformat_type reformat_type,
-				     size_t data_sz,
-				     void *data) { return NULL; }
-
-static inline struct mlx5dr_action *
-mlx5dr_action_create_modify_header(struct mlx5dr_domain *domain,
-				   u32 flags,
-				   size_t actions_sz,
-				   __be64 actions[]) { return NULL; }
-
-static inline struct mlx5dr_action *
-mlx5dr_action_create_pop_vlan(void) { return NULL; }
-
-static inline struct mlx5dr_action *
-mlx5dr_action_create_push_vlan(struct mlx5dr_domain *domain,
-			       __be32 vlan_hdr) { return NULL; }
-
-static inline int
-mlx5dr_action_destroy(struct mlx5dr_action *action) { return 0; }
-
-static inline bool
-mlx5dr_is_supported(struct mlx5_core_dev *dev) { return false; }
-
-#endif /* CONFIG_MLX5_SW_STEERING */
-
 #endif /* _MLX5DR_H_ */
Signature · 1 line
2.24.1
maintainer
#21 lore raw

From: Jakub Kicinski <[email protected]>
Date: Fri, 13 Mar 2020 19:37:21 -0700

Quoted 11 lines · On Fri, 13 Mar 2020 18:16:08 -0700 Saeed Mahameed wrote:
Quoted 6 lines

Hi Dave,

This series adds misc updates to mlx5 driver
For more information please see tag log below.

Please pull and let me know if there is any problem.

My questions on the patches are mostly curiosity, so:

Pulled, thanks everyone.