Multi-plane support for mlx5

From: Leon Romanovsky <[email protected]>

From Mark,

This patchset adds support to IB sub device and mlx5 implementation.

An IB sub device provides a subset of functionalists of it's parent. Currently type "SMI" is supported: A SMI device provides SMI (QP0) interface and shares same VPort with it's parent; It allows the subnet manager to configure VPort through this interface when the parent doesn't support SMI.

In mlx5 cases, when mlx5 multi-plane is supported, a logical mlx5 port which aggregates multiple physical plane ports is presented, to provide higher bandwidth. As SMI is per physical port, a mlx5 SMI device is needed to represents physical plane ports and provides SMI capability.

A sub device can be added or deleted with the rdma tool. When a mlx5 SMI device is created, all it's ports are created.

Examples:

$ rdma dev add smi1 type SMI parent ibp8s0f1
$ rdma dev show smi1
2: smi1: node_type ca fw 20.38.0458 node_guid 9803:9b03:009f:d20f
sys_image_guid 9803:9b03:009f:d20e type smi parent ibp8s0f1
$ rdma link show
...
link smi1/1 state INIT physical_state LINK_UP
link smi1/2 state INIT physical_state LINK_UP
link smi1/3 state INIT physical_state LINK_UP
link smi1/4 state INIT physical_state LINK_UP
$ rdma dev del smi1

Mark Zhang (12):
RDMA/core: Create "issm*" device nodes only when SMI is supported net/mlx5: mlx5_ifc update for multi-plane support RDMA/mlx5: Add support to multi-plane device and port RDMA/core: Support IB sub device with type "SMI"
RDMA: Set type of rdma_ah to IB for a SMI sub device RDMA/core: Create GSI QP only when CM is supported RDMA/mlx5: Support plane device and driver APIs to add and delete it RDMA/nldev: Add support to add/delete a sub IB device through netlink RDMA/nldev: Add support to dump device type and parent device if
exists
RDMA/mlx5: Add plane index support when querying PTYS registers net/mlx5: mlx5_ifc update for accessing ppcnt register of plane ports RDMA/mlx5: Support per-plane port IB counters by querying PPCNT
register

 drivers/infiniband/core/agent.c               |  32 ++-
 drivers/infiniband/core/device.c              |  68 +++++++
 drivers/infiniband/core/mad.c                 |   9 +-
 drivers/infiniband/core/nldev.c               |  69 +++++++
 drivers/infiniband/core/user_mad.c            |  29 +--
 drivers/infiniband/core/uverbs_main.c         |   3 +-
 drivers/infiniband/hw/mlx5/cmd.c              |  12 +-
 drivers/infiniband/hw/mlx5/cmd.h              |   2 +-
 drivers/infiniband/hw/mlx5/mad.c              |  71 +++++--
 drivers/infiniband/hw/mlx5/main.c             | 182 ++++++++++++++++--
 drivers/infiniband/hw/mlx5/mlx5_ib.h          |  10 +
 drivers/infiniband/hw/mlx5/qp.c               |   7 +-
 drivers/infiniband/hw/mlx5/qpc.c              |  13 +-
 .../net/ethernet/mellanox/mlx5/core/en/port.c |   2 +-
 .../ethernet/mellanox/mlx5/core/en_ethtool.c  |   2 +-
 .../mellanox/mlx5/core/ipoib/ethtool.c        |   2 +-
 .../net/ethernet/mellanox/mlx5/core/port.c    |  10 +-
 .../net/ethernet/mellanox/mlx5/core/vport.c   |   1 +
 include/linux/mlx5/device.h                   |   1 +
 include/linux/mlx5/driver.h                   |   1 +
 include/linux/mlx5/mlx5_ifc.h                 |  61 +++++-
 include/linux/mlx5/port.h                     |   5 +-
 include/rdma/ib_verbs.h                       |  45 +++++
 include/uapi/rdma/rdma_netlink.h              |  13 ++
 24 files changed, 574 insertions(+), 76 deletions(-)

2.45.2