Skip to contentSkip to conversation

Create IPoIB QP with specific QP number

PATCH Applied Leon Romanovsky 2 participants 8 messages Jun 23, 2020 – Jul 6, 2020

Subsystems (5)
lore
Share & tools
lore
#0 lore raw

cover Create IPoIB QP with specific QP number

From: Leon Romanovsky <[email protected]>

From Michael,

This series handles IPoIB child interface creation with setting interface's HW address.

In current implementation, lladdr requested by user is ignored and overwritten. Child interface gets the same GID as the parent interface and a QP number which is assigned by the underlying drivers.

In this series we fix this behavior so that user's requested address is assigned to the newly created interface.

As specific QP number request is not supported for all vendors, QP number requested by user will still be overwritten when this is not supported.

Behavior of creation of child interfaces through the sysfs mechanism or without specifying a requested address, stays the same.

Thanks

Michael Guralnik (2):
net/mlx5: Enable QP number request when creating IPoIB underlay QP RDMA/ipoib: Handle user-supplied address when creating child

 drivers/infiniband/ulp/ipoib/ipoib_main.c             | 11 +++++++++--
 drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c |  7 +++++++
 drivers/net/ethernet/mellanox/mlx5/core/main.c        |  3 +++
 include/linux/mlx5/mlx5_ifc.h                         |  9 +++++++--
 4 files changed, 26 insertions(+), 4 deletions(-)

--
2.26.2

#1 lore raw

PATCH 1/2 net/mlx5: Enable QP number request when creating IPoIB underlay QP

From: Michael Guralnik <[email protected]>

If in the process of creating the underlay QP for an IPoIB interface the user has set the address and specifically the 1st-3rd bytes representing the QP number, use the requested QP number when creating the underlay QP.

For a user to be able to request a QP number on QP creation, the MKEY_BY_NAME NVCONFIG should be set. As mkey_by_name and qp_by_name are coupled in FW. This requires driver to query the mkey_by_name max cap during initialization and set the current cap if it was enabled in FW.

---
 drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c | 7 +++++++
 drivers/net/ethernet/mellanox/mlx5/core/main.c        | 3 +++
 include/linux/mlx5/mlx5_ifc.h                         | 9 +++++++--
 3 files changed, 17 insertions(+), 2 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c +7 -0
index 690b822c6152..d1266d8fed97 100644
@@ -226,13 +226,20 @@ void mlx5i_uninit_underlay_qp(struct mlx5e_priv *priv)
 
 int mlx5i_create_underlay_qp(struct mlx5e_priv *priv)
 {
+	unsigned char *dev_addr = priv->netdev->dev_addr;
 	u32 out[MLX5_ST_SZ_DW(create_qp_out)] = {};
 	u32 in[MLX5_ST_SZ_DW(create_qp_in)] = {};
 	struct mlx5i_priv *ipriv = priv->ppriv;
 	void *addr_path;
+	int qpn = 0;
 	int ret = 0;
 	void *qpc;
 
+	if (MLX5_CAP_GEN(priv->mdev, mkey_by_name)) {
+		qpn = (dev_addr[1] << 16) + (dev_addr[2] << 8) + dev_addr[3];
+		MLX5_SET(create_qp_in, in, input_qpn, qpn);
+	}
+
 	qpc = MLX5_ADDR_OF(create_qp_in, in, qpc);
 	MLX5_SET(qpc, qpc, st, MLX5_QP_ST_UD);
 	MLX5_SET(qpc, qpc, pm_state, MLX5_QP_PM_MIGRATED);
drivers/net/ethernet/mellanox/mlx5/core/main.c +3 -0
index 8b658908f044..623785fe74b2 100644
@@ -557,6 +557,9 @@ static int handle_hca_cap(struct mlx5_core_dev *dev, void *set_ctx)
 	if (MLX5_CAP_GEN_MAX(dev, release_all_pages))
 		MLX5_SET(cmd_hca_cap, set_hca_cap, release_all_pages, 1);
 
+	if (MLX5_CAP_GEN_MAX(dev, mkey_by_name))
+		MLX5_SET(cmd_hca_cap, set_hca_cap, mkey_by_name, 1);
+
 	return set_caps(dev, set_ctx, MLX5_SET_HCA_CAP_OP_MOD_GENERAL_DEVICE);
 }
 
include/linux/mlx5/mlx5_ifc.h +7 -2
index ca1887dd0423..7a00110f2f01 100644
@@ -1392,7 +1392,10 @@ struct mlx5_ifc_cmd_hca_cap_bits {
 	u8         bf[0x1];
 	u8         driver_version[0x1];
 	u8         pad_tx_eth_packet[0x1];
-	u8         reserved_at_263[0x8];
+	u8         reserved_at_263[0x3];
+	u8         mkey_by_name[0x1];
+	u8         reserved_at_267[0x4];
+
 	u8         log_bf_reg_size[0x5];
 
 	u8         reserved_at_270[0x8];
@@ -7714,8 +7717,10 @@ struct mlx5_ifc_create_qp_in_bits {
 	u8         reserved_at_20[0x10];
 	u8         op_mod[0x10];
 
-	u8         reserved_at_40[0x40];
+	u8         reserved_at_40[0x8];
+	u8         input_qpn[0x18];
 
+	u8         reserved_at_60[0x20];
 	u8         opt_param_mask[0x20];
 
 	u8         ece[0x20];
Signature · 1 line
2.26.2
#2 lore raw

PATCH 2/2 RDMA/ipoib: Handle user-supplied address when creating child

In reply to Leon Romanovsky

From: Michael Guralnik <[email protected]>

Use the address supplied by user when creating a child interface.

Previously, the address requested by the user was ignored and overridden with parent's GID and the random QP number assigned to the child.

---
 drivers/infiniband/ulp/ipoib/ipoib_main.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)
drivers/infiniband/ulp/ipoib/ipoib_main.c +9 -2
index 3cfb682b91b0..a9f1174f7320 100644
@@ -1892,8 +1892,15 @@ static void ipoib_child_init(struct net_device *ndev)
 
 	priv->max_ib_mtu = ppriv->max_ib_mtu;
 	set_bit(IPOIB_FLAG_SUBINTERFACE, &priv->flags);
-	memcpy(priv->dev->dev_addr, ppriv->dev->dev_addr, INFINIBAND_ALEN);
-	memcpy(&priv->local_gid, &ppriv->local_gid, sizeof(priv->local_gid));
+	if (memchr_inv(priv->dev->dev_addr, 0, INFINIBAND_ALEN))
+		memcpy(&priv->local_gid, priv->dev->dev_addr + 4,
+		       sizeof(priv->local_gid));
+	else {
+		memcpy(priv->dev->dev_addr, ppriv->dev->dev_addr,
+		       INFINIBAND_ALEN);
+		memcpy(&priv->local_gid, &ppriv->local_gid,
+		       sizeof(priv->local_gid));
+	}
 }
 
 static int ipoib_ndo_init(struct net_device *ndev)
Signature · 1 line
2.26.2
maintainer Applied
#3 lore raw
In reply to Leon Romanovsky
Quoted 18 lines · Leon Romanovsky

From: Leon Romanovsky <[email protected]>

Quoted 1 line

From Michael,

This series handles IPoIB child interface creation with setting interface's HW address.

In current implementation, lladdr requested by user is ignored and overwritten. Child interface gets the same GID as the parent interface and a QP number which is assigned by the underlying drivers.

In this series we fix this behavior so that user's requested address is assigned to the newly created interface.

As specific QP number request is not supported for all vendors, QP number requested by user will still be overwritten when this is not supported.

Quoted 8 lines

Behavior of creation of child interfaces through the sysfs mechanism or without specifying a requested address, stays the same.

Thanks

Michael Guralnik (2):
net/mlx5: Enable QP number request when creating IPoIB underlay QP RDMA/ipoib: Handle user-supplied address when creating child

Applied to for-next, thanks

Jason

#4 lore raw
Quoted 22 lines · Jason Gunthorpe
Quoted 21 lines · Leon Romanovsky

From: Leon Romanovsky <[email protected]>

Quoted 1 line

From Michael,

This series handles IPoIB child interface creation with setting interface's HW address.

In current implementation, lladdr requested by user is ignored and overwritten. Child interface gets the same GID as the parent interface and a QP number which is assigned by the underlying drivers.

In this series we fix this behavior so that user's requested address is assigned to the newly created interface.

As specific QP number request is not supported for all vendors, QP number requested by user will still be overwritten when this is not supported.

Behavior of creation of child interfaces through the sysfs mechanism or without specifying a requested address, stays the same.

Quoted 7 lines
Quoted 5 lines

Thanks

Michael Guralnik (2):
net/mlx5: Enable QP number request when creating IPoIB underlay QP RDMA/ipoib: Handle user-supplied address when creating child

Applied to for-next, thanks

Thanks Jason,

Won't it better that first patch be applied to mlx5-next in order to avoid possible merge conflicts?

Thanks

Quoted 2 lines

Jason

maintainer
#5 lore raw
Quoted 29 lines · Leon Romanovsky
Quoted 28 lines · Jason Gunthorpe
Quoted 26 lines · Leon Romanovsky

From: Leon Romanovsky <[email protected]>

Quoted 1 line

From Michael,

This series handles IPoIB child interface creation with setting interface's HW address.

In current implementation, lladdr requested by user is ignored and overwritten. Child interface gets the same GID as the parent interface and a QP number which is assigned by the underlying drivers.

In this series we fix this behavior so that user's requested address is assigned to the newly created interface.

As specific QP number request is not supported for all vendors, QP number requested by user will still be overwritten when this is not supported.

Behavior of creation of child interfaces through the sysfs mechanism or without specifying a requested address, stays the same.

Thanks

Michael Guralnik (2):
net/mlx5: Enable QP number request when creating IPoIB underlay QP RDMA/ipoib: Handle user-supplied address when creating child

Quoted 6 lines
Quoted 1 line

Applied to for-next, thanks

Thanks Jason,

Won't it better that first patch be applied to mlx5-next in order to avoid possible merge conflicts?

Oops, sure, go ahead please

Jason

#6 lore raw
In reply to Jason Gunthorpe
Quoted 22 lines · Jason Gunthorpe
Quoted 21 lines · Leon Romanovsky

From: Leon Romanovsky <[email protected]>

Quoted 1 line

From Michael,

This series handles IPoIB child interface creation with setting interface's HW address.

In current implementation, lladdr requested by user is ignored and overwritten. Child interface gets the same GID as the parent interface and a QP number which is assigned by the underlying drivers.

In this series we fix this behavior so that user's requested address is assigned to the newly created interface.

As specific QP number request is not supported for all vendors, QP number requested by user will still be overwritten when this is not supported.

Behavior of creation of child interfaces through the sysfs mechanism or without specifying a requested address, stays the same.

Quoted 7 lines
Quoted 5 lines

Thanks

Michael Guralnik (2):
net/mlx5: Enable QP number request when creating IPoIB underlay QP RDMA/ipoib: Handle user-supplied address when creating child

Applied to for-next, thanks

I pushed first patch.
dca650991e4 net/mlx5: Enable QP number request when creating IPoIB underlay QP

Quoted 2 lines

Jason

maintainer Applied
#7 lore raw
In reply to Leon Romanovsky
Quoted 27 lines · Leon Romanovsky

From: Leon Romanovsky <[email protected]>

Quoted 1 line

From Michael,

This series handles IPoIB child interface creation with setting interface's HW address.

In current implementation, lladdr requested by user is ignored and overwritten. Child interface gets the same GID as the parent interface and a QP number which is assigned by the underlying drivers.

In this series we fix this behavior so that user's requested address is assigned to the newly created interface.

As specific QP number request is not supported for all vendors, QP number requested by user will still be overwritten when this is not supported.

Behavior of creation of child interfaces through the sysfs mechanism or without specifying a requested address, stays the same.

Thanks

Michael Guralnik (2):
net/mlx5: Enable QP number request when creating IPoIB underlay QP RDMA/ipoib: Handle user-supplied address when creating child

Quoted 5 lines
 drivers/infiniband/ulp/ipoib/ipoib_main.c             | 11 +++++++++--
 drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c |  7 +++++++
 drivers/net/ethernet/mellanox/mlx5/core/main.c        |  3 +++
 include/linux/mlx5/mlx5_ifc.h                         |  9 +++++++--
 4 files changed, 26 insertions(+), 4 deletions(-)

Applied to for-next, thanks

Jason