Create IPoIB QP with specific QP number
Subsystems (5)
cover Create IPoIB QP with specific QP number
From: Leon Romanovsky <[email protected]>
From Michael,
This series handles IPoIB child interface creation with setting interface's HW address.
In current implementation, lladdr requested by user is ignored and overwritten. Child interface gets the same GID as the parent interface and a QP number which is assigned by the underlying drivers.
In this series we fix this behavior so that user's requested address is assigned to the newly created interface.
As specific QP number request is not supported for all vendors, QP number requested by user will still be overwritten when this is not supported.
Behavior of creation of child interfaces through the sysfs mechanism or without specifying a requested address, stays the same.
Thanks
Michael Guralnik (2):
net/mlx5: Enable QP number request when creating IPoIB underlay QP RDMA/ipoib: Handle user-supplied address when creating child
drivers/infiniband/ulp/ipoib/ipoib_main.c | 11 +++++++++-- drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c | 7 +++++++ drivers/net/ethernet/mellanox/mlx5/core/main.c | 3 +++ include/linux/mlx5/mlx5_ifc.h | 9 +++++++-- 4 files changed, 26 insertions(+), 4 deletions(-)
--
2.26.2
PATCH 1/2 net/mlx5: Enable QP number request when creating IPoIB underlay QP
From: Michael Guralnik <[email protected]>
If in the process of creating the underlay QP for an IPoIB interface the user has set the address and specifically the 1st-3rd bytes representing the QP number, use the requested QP number when creating the underlay QP.
For a user to be able to request a QP number on QP creation, the MKEY_BY_NAME NVCONFIG should be set. As mkey_by_name and qp_by_name are coupled in FW. This requires driver to query the mkey_by_name max cap during initialization and set the current cap if it was enabled in FW.
--- drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c | 7 +++++++ drivers/net/ethernet/mellanox/mlx5/core/main.c | 3 +++ include/linux/mlx5/mlx5_ifc.h | 9 +++++++-- 3 files changed, 17 insertions(+), 2 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c +7 -0
@@ -226,13 +226,20 @@ void mlx5i_uninit_underlay_qp(struct mlx5e_priv *priv)int mlx5i_create_underlay_qp (struct mlx5e_priv *priv) { +unsigned char *dev_addr = priv->netdev->dev_addr;u32 out[MLX5_ST_SZ_DW (create_qp_out)] = {};u32 in[MLX5_ST_SZ_DW (create_qp_in)] = {};struct mlx5i_priv *ipriv = priv->ppriv;void *addr_path; +int qpn =0 ;int ret =0 ;void *qpc; +if (MLX5_CAP_GEN (priv->mdev, mkey_by_name)) { + qpn = (dev_addr[1 ] <<16 ) + (dev_addr[2 ] <<8 ) + dev_addr[3 ]; +MLX5_SET (create_qp_in, in, input_qpn, qpn); + } + qpc =MLX5_ADDR_OF (create_qp_in, in, qpc);MLX5_SET (qpc, qpc, st,MLX5_QP_ST_UD );MLX5_SET (qpc, qpc, pm_state,MLX5_QP_PM_MIGRATED );
drivers/net/ethernet/mellanox/mlx5/core/main.c +3 -0
@@ -557,6 +557,9 @@ static int handle_hca_cap(struct mlx5_core_dev *dev, void *set_ctx)if (MLX5_CAP_GEN_MAX (dev, release_all_pages))MLX5_SET (cmd_hca_cap, set_hca_cap, release_all_pages,1 ); +if (MLX5_CAP_GEN_MAX (dev, mkey_by_name)) +MLX5_SET (cmd_hca_cap, set_hca_cap, mkey_by_name,1 ); +return set_caps (dev, set_ctx,MLX5_SET_HCA_CAP_OP_MOD_GENERAL_DEVICE ); }
include/linux/mlx5/mlx5_ifc.h +7 -2
@@ -1392,7 +1392,10 @@ struct mlx5_ifc_cmd_hca_cap_bits {u8 bf[0x1 ];u8 driver_version[0x1 ];u8 pad_tx_eth_packet[0x1 ]; -u8 reserved_at_263[0x8 ]; +u8 reserved_at_263[0x3 ]; +u8 mkey_by_name[0x1 ]; +u8 reserved_at_267[0x4 ]; +u8 log_bf_reg_size[0x5 ];u8 reserved_at_270[0x8 ];
@@ -7714,8 +7717,10 @@ struct mlx5_ifc_create_qp_in_bits {u8 reserved_at_20[0x10 ];u8 op_mod[0x10 ]; -u8 reserved_at_40[0x40 ]; +u8 reserved_at_40[0x8 ]; +u8 input_qpn[0x18 ]; +u8 reserved_at_60[0x20 ];u8 opt_param_mask[0x20 ];u8 ece[0x20 ];
Signature · 1 line
2.26.2
PATCH 2/2 RDMA/ipoib: Handle user-supplied address when creating child
In reply to Leon RomanovskyFrom: Michael Guralnik <[email protected]>
Use the address supplied by user when creating a child interface.
Previously, the address requested by the user was ignored and overridden with parent's GID and the random QP number assigned to the child.
--- drivers/infiniband/ulp/ipoib/ipoib_main.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-)
drivers/infiniband/ulp/ipoib/ipoib_main.c +9 -2
@@ -1892,8 +1892,15 @@ static void ipoib_child_init(struct net_device *ndev) priv->max_ib_mtu = ppriv->max_ib_mtu;set_bit (IPOIB_FLAG_SUBINTERFACE , &priv->flags); -memcpy (priv->dev->dev_addr, ppriv->dev->dev_addr,INFINIBAND_ALEN ); -memcpy (&priv->local_gid, &ppriv->local_gid,sizeof (priv->local_gid)); +if (memchr_inv (priv->dev->dev_addr,0 ,INFINIBAND_ALEN )) +memcpy (&priv->local_gid, priv->dev->dev_addr +4 , +sizeof (priv->local_gid)); +else { +memcpy (priv->dev->dev_addr, ppriv->dev->dev_addr, +INFINIBAND_ALEN ); +memcpy (&priv->local_gid, &ppriv->local_gid, +sizeof (priv->local_gid)); + } }static int ipoib_ndo_init (struct net_device *ndev)
Signature · 1 line
2.26.2
Quoted 18 lines · Leon Romanovsky
From: Leon Romanovsky <[email protected]>
Quoted 1 line
From Michael,
This series handles IPoIB child interface creation with setting interface's HW address.
In current implementation, lladdr requested by user is ignored and overwritten. Child interface gets the same GID as the parent interface and a QP number which is assigned by the underlying drivers.
In this series we fix this behavior so that user's requested address is assigned to the newly created interface.
As specific QP number request is not supported for all vendors, QP number requested by user will still be overwritten when this is not supported.
Quoted 8 lines
Behavior of creation of child interfaces through the sysfs mechanism or without specifying a requested address, stays the same.
Thanks
Michael Guralnik (2):
net/mlx5: Enable QP number request when creating IPoIB underlay QP RDMA/ipoib: Handle user-supplied address when creating child
Applied to for-next, thanks
Jason
Quoted 22 lines · Jason Gunthorpe
Quoted 21 lines · Leon Romanovsky
From: Leon Romanovsky <[email protected]>
Quoted 1 line
From Michael,
This series handles IPoIB child interface creation with setting interface's HW address.
In current implementation, lladdr requested by user is ignored and overwritten. Child interface gets the same GID as the parent interface and a QP number which is assigned by the underlying drivers.
In this series we fix this behavior so that user's requested address is assigned to the newly created interface.
As specific QP number request is not supported for all vendors, QP number requested by user will still be overwritten when this is not supported.
Behavior of creation of child interfaces through the sysfs mechanism or without specifying a requested address, stays the same.
Quoted 7 lines
Quoted 5 lines
Thanks
Michael Guralnik (2):
net/mlx5: Enable QP number request when creating IPoIB underlay QP RDMA/ipoib: Handle user-supplied address when creating childApplied to for-next, thanks
Thanks Jason,
Won't it better that first patch be applied to mlx5-next in order to avoid possible merge conflicts?
Thanks
Quoted 2 lines
Jason
Jason Gunthorpe
maintainerQuoted 29 lines · Leon Romanovsky
Quoted 28 lines · Jason Gunthorpe
Quoted 26 lines · Leon Romanovsky
From: Leon Romanovsky <[email protected]>
Quoted 1 line
From Michael,
This series handles IPoIB child interface creation with setting interface's HW address.
In current implementation, lladdr requested by user is ignored and overwritten. Child interface gets the same GID as the parent interface and a QP number which is assigned by the underlying drivers.
In this series we fix this behavior so that user's requested address is assigned to the newly created interface.
As specific QP number request is not supported for all vendors, QP number requested by user will still be overwritten when this is not supported.
Behavior of creation of child interfaces through the sysfs mechanism or without specifying a requested address, stays the same.
Thanks
Michael Guralnik (2):
net/mlx5: Enable QP number request when creating IPoIB underlay QP RDMA/ipoib: Handle user-supplied address when creating child
Quoted 6 lines
Quoted 1 line
Applied to for-next, thanks
Thanks Jason,
Won't it better that first patch be applied to mlx5-next in order to avoid possible merge conflicts?
Oops, sure, go ahead please
Jason
Quoted 22 lines · Jason Gunthorpe
Quoted 21 lines · Leon Romanovsky
From: Leon Romanovsky <[email protected]>
Quoted 1 line
From Michael,
This series handles IPoIB child interface creation with setting interface's HW address.
In current implementation, lladdr requested by user is ignored and overwritten. Child interface gets the same GID as the parent interface and a QP number which is assigned by the underlying drivers.
In this series we fix this behavior so that user's requested address is assigned to the newly created interface.
As specific QP number request is not supported for all vendors, QP number requested by user will still be overwritten when this is not supported.
Behavior of creation of child interfaces through the sysfs mechanism or without specifying a requested address, stays the same.
Quoted 7 lines
Quoted 5 lines
Thanks
Michael Guralnik (2):
net/mlx5: Enable QP number request when creating IPoIB underlay QP RDMA/ipoib: Handle user-supplied address when creating childApplied to for-next, thanks
I pushed first patch.dca650991e4 net/mlx5: Enable QP number request when creating IPoIB underlay QP
Quoted 2 lines
Jason
Quoted 27 lines · Leon Romanovsky
From: Leon Romanovsky <[email protected]>
Quoted 1 line
From Michael,
This series handles IPoIB child interface creation with setting interface's HW address.
In current implementation, lladdr requested by user is ignored and overwritten. Child interface gets the same GID as the parent interface and a QP number which is assigned by the underlying drivers.
In this series we fix this behavior so that user's requested address is assigned to the newly created interface.
As specific QP number request is not supported for all vendors, QP number requested by user will still be overwritten when this is not supported.
Behavior of creation of child interfaces through the sysfs mechanism or without specifying a requested address, stays the same.
Thanks
Michael Guralnik (2):
net/mlx5: Enable QP number request when creating IPoIB underlay QP RDMA/ipoib: Handle user-supplied address when creating child
Quoted 5 lines
drivers/infiniband/ulp/ipoib/ipoib_main.c | 11 +++++++++-- drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c | 7 +++++++ drivers/net/ethernet/mellanox/mlx5/core/main.c | 3 +++ include/linux/mlx5/mlx5_ifc.h | 9 +++++++-- 4 files changed, 26 insertions(+), 4 deletions(-)
Applied to for-next, thanks
Jason