Skip to contentSkip to conversation

Add DC RoCE LAG support

PATCH Applied Leon Romanovsky 2 participants 4 messages Aug 18, 2020 – Aug 27, 2020

Subsystems (5)
lore
Share & tools
lore
#0 lore raw

cover Add DC RoCE LAG support

From: Leon Romanovsky <[email protected]>

Hi,

Two extremely short patches to enable DC RoCE LAG support.

Thanks

Mark Zhang (2):
IB/mlx5: Add tx_affinity support for DCI QP
IB/mlx5: Add DCT RoCE LAG support

 drivers/infiniband/hw/mlx5/qp.c | 23 ++++++++++++++---------
 include/linux/mlx5/mlx5_ifc.h   |  3 ++-
 2 files changed, 16 insertions(+), 10 deletions(-)

--
2.26.2

#1 lore raw

PATCH 1/2 IB/mlx5: Add tx_affinity support for DCI QP

From: Mark Zhang <[email protected]>

DCI QP supports tx_affinity as well.

---
 drivers/infiniband/hw/mlx5/qp.c | 14 ++++++--------
 1 file changed, 6 insertions(+), 8 deletions(-)
drivers/infiniband/hw/mlx5/qp.c +6 -8
index da9f15e0a644..0526d574cd9b 100644
@@ -3667,14 +3667,12 @@ static unsigned int get_tx_affinity_rr(struct mlx5_ib_dev *dev,
 		MLX5_MAX_PORTS + 1;
 }
 
-static bool qp_supports_affinity(struct ib_qp *qp)
+static bool qp_supports_affinity(struct mlx5_ib_qp *qp)
 {
-	if ((qp->qp_type == IB_QPT_RC) ||
-	    (qp->qp_type == IB_QPT_UD) ||
-	    (qp->qp_type == IB_QPT_UC) ||
-	    (qp->qp_type == IB_QPT_RAW_PACKET) ||
-	    (qp->qp_type == IB_QPT_XRC_INI) ||
-	    (qp->qp_type == IB_QPT_XRC_TGT))
+	if ((qp->type == IB_QPT_RC) || (qp->type == IB_QPT_UD) ||
+	    (qp->type == IB_QPT_UC) || (qp->type == IB_QPT_RAW_PACKET) ||
+	    (qp->type == IB_QPT_XRC_INI) || (qp->type == IB_QPT_XRC_TGT) ||
+	    (qp->type == MLX5_IB_QPT_DCI))
 		return true;
 	return false;
 }
@@ -3692,7 +3690,7 @@ static unsigned int get_tx_affinity(struct ib_qp *qp,
 	unsigned int tx_affinity;
 
 	if (!(mlx5_ib_lag_should_assign_affinity(dev) &&
-	      qp_supports_affinity(qp)))
+	      qp_supports_affinity(mqp)))
 		return 0;
 
 	if (mqp->flags & MLX5_IB_QP_CREATE_SQPN_QP1)

--
2.26.2

#2 lore raw

PATCH 2/2 IB/mlx5: Add DCT RoCE LAG support

In reply to Leon Romanovsky

From: Mark Zhang <[email protected]>

When DCT QPs work in RoCE LAG mode:
1. DCT creation is allowed only when it is supported.
2. The "port" of a DCT QP is assigned in a round-robin way.

---
 drivers/infiniband/hw/mlx5/qp.c | 9 ++++++++-
 include/linux/mlx5/mlx5_ifc.h   | 3 ++-
 2 files changed, 10 insertions(+), 2 deletions(-)
drivers/infiniband/hw/mlx5/qp.c +8 -1
index 0526d574cd9b..4eff325c4d2c 100644
@@ -2409,6 +2409,9 @@ static int create_dct(struct mlx5_ib_dev *dev, struct ib_pd *pd,
 	u32 uidx = params->uidx;
 	void *dctc;
 
+	if (mlx5_lag_is_active(dev->mdev) && !MLX5_CAP_GEN(dev->mdev, lag_dct))
+		return -EOPNOTSUPP;
+
 	qp->dct.in = kzalloc(MLX5_ST_SZ_BYTES(create_dct_in), GFP_KERNEL);
 	if (!qp->dct.in)
 		return -ENOMEM;
@@ -4183,7 +4186,11 @@ static int mlx5_ib_modify_dct(struct ib_qp *ibqp, struct ib_qp_attr *attr,
 			MLX5_SET(dctc, dctc, rae, 1);
 		}
 		MLX5_SET(dctc, dctc, pkey_index, attr->pkey_index);
-		MLX5_SET(dctc, dctc, port, attr->port_num);
+		if (mlx5_lag_is_active(dev->mdev))
+			MLX5_SET(dctc, dctc, port,
+				 get_tx_affinity_rr(dev, udata));
+		else
+			MLX5_SET(dctc, dctc, port, attr->port_num);
 
 		set_id = mlx5_ib_get_counters_id(dev, attr->port_num - 1);
 		MLX5_SET(dctc, dctc, counter_set_id, set_id);
include/linux/mlx5/mlx5_ifc.h +2 -1
index de1ffb4804d6..aee25e4fb2cc 100644
@@ -1430,7 +1430,8 @@ struct mlx5_ifc_cmd_hca_cap_bits {
 
 	u8         log_bf_reg_size[0x5];
 
-	u8         reserved_at_270[0x8];
+	u8         reserved_at_270[0x6];
+	u8         lag_dct[0x2];
 	u8         lag_tx_port_affinity[0x1];
 	u8         reserved_at_279[0x2];
 	u8         lag_master[0x1];

--
2.26.2

maintainer Applied
#3 lore raw
In reply to Leon Romanovsky
Quoted 11 lines · Leon Romanovsky

From: Leon Romanovsky <[email protected]>

Hi,

Two extremely short patches to enable DC RoCE LAG support.

Thanks

Mark Zhang (2):
IB/mlx5: Add tx_affinity support for DCI QP
IB/mlx5: Add DCT RoCE LAG support

Applied to for-next, thanks

Jason