Skip to content

IB/Verbs: IB Management Helpers

Michael Wang linux-rdma lkml 8 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v8 Latest23 patches 12 repliesIn review
  2. v723 patches 17 repliesSuperseded
  3. v626 patches 29 repliesSuperseded
Earlier versions · 5
VersionPostedPatchesRepliesStatus
v527 patches82 repliesSuperseded
v427 patches29 repliesSuperseded
v328 patches55 repliesSuperseded
v217 patches64 repliesSuperseded
v1 RFC11 patches51 repliesSuperseded

Changes in v8 from the cover letter

* Thanks to Doug, Ira, Devesh for the testing :-)
* Thanks for the comments from or, Doug, Ira, Jason :-) Please remind me if anything missed :-P
* Use rdma_cap_XX() instead of cap_XX() for readability
* Remove CC list in git log for maintainability
* Use bool as return value
* Updated github repository to v8

Patches in v8 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/23IB/Verbs: Implement new callback query_protocol()drivers/infiniband/core/device.c drivers/infiniband/hw/amso1100/c2_provider.c drivers/infiniband/hw/cxgb3/iwch_provider.c +17 more+104 -00 replies
2/23IB/Verbs: Implement raw management helpersinclude/rdma/ib_verbs.h+22 -00 replies
3/23IB/Verbs: Reform IB-core mad/agent/user_maddrivers/infiniband/core/agent.c drivers/infiniband/core/mad.c drivers/infiniband/core/user_mad.c+41 -300 replies
4/23IB/Verbs: Reform IB-core cmdrivers/infiniband/core/cm.c+17 -30 replies
5/23IB/Verbs: Reform IB-core sa_querydrivers/infiniband/core/sa_query.c+17 -130 replies
6/23IB/Verbs: Reform IB-core multicastdrivers/infiniband/core/multicast.c+3 -90 replies
7/23IB/Verbs: Reform IB-ulp ipoibdrivers/infiniband/ulp/ipoib/ipoib_main.c+8 -70 replies
8/23IB/Verbs: Reform IB-ulp xprtrdmanet/sunrpc/xprtrdma/svc_rdma_recvfrom.c net/sunrpc/xprtrdma/svc_rdma_transport.c+20 -290 replies
9/23IB/Verbs: Reform IB-core verbsdrivers/infiniband/core/verbs.c+2 -40 replies
10/23IB/Verbs: Reform cm related part in IB-core cma/ucmdrivers/infiniband/core/cma.c drivers/infiniband/core/ucm.c+26 -580 replies
11/23IB/Verbs: Reform route related part in IB-core cmadrivers/infiniband/core/cma.c drivers/infiniband/core/ucma.c+14 -420 replies
12/23IB/Verbs: Reform mcast related part in IB-core cmadrivers/infiniband/core/cma.c+18 -380 replies
13/23IB/Verbs: Reform cma_acquire_dev()drivers/infiniband/core/cma.c+40 -280 replies
14/23IB/Verbs: Reform rest part in IB-core cmadrivers/infiniband/core/cma.c+9 -110 replies
15/23IB/Verbs: Use management helper rdma_cap_ib_mad()drivers/infiniband/core/mad.c drivers/infiniband/core/user_mad.c include/rdma/ib_verbs.h+21 -60 replies
16/23IB/Verbs: Use management helper rdma_cap_ib_smi()drivers/infiniband/core/agent.c drivers/infiniband/core/mad.c include/rdma/ib_verbs.h+17 -20 replies
17/23IB/Verbs: Use management helper rdma_cap_ib_cm()drivers/infiniband/core/cm.c drivers/infiniband/core/cma.c drivers/infiniband/core/ucm.c +1 more+28 -140 replies
18/23IB/Verbs: Use management helper rdma_cap_iw_cm()drivers/infiniband/core/cma.c include/rdma/ib_verbs.h+22 -70 replies
19/23IB/Verbs: Use management helper rdma_cap_ib_sa()drivers/infiniband/core/cma.c drivers/infiniband/core/sa_query.c drivers/infiniband/core/ucma.c +1 more+23 -80 replies
20/23IB/Verbs: Use management helper rdma_cap_ib_mcast()drivers/infiniband/core/cma.c drivers/infiniband/core/multicast.c include/rdma/ib_verbs.h+21 -60 replies
21/23IB/Verbs: Use management helper rdma_cap_read_multi_sge()include/rdma/ib_verbs.h net/sunrpc/xprtrdma/svc_rdma_recvfrom.c+18 -20 replies
22/23IB/Verbs: Use management helper rdma_cap_af_ib()drivers/infiniband/core/cma.c include/rdma/ib_verbs.h+16 -10 replies
23/23IB/Verbs: Use management helper rdma_cap_eth_ah()drivers/infiniband/core/cma.c drivers/infiniband/core/sa_query.c drivers/infiniband/core/verbs.c +1 more+19 -40 replies

Reviews trailers per patch

None of the 23 patches has a Reviewed-by or Acked-by of their own; the series has one for all of them.

PatchDevesh SharmaIra Weiny
SeriesR-b R-b T-b
1/23
2/23
3/23
4/23
5/23
6/23
7/23
8/23
9/23
10/23
11/23
12/23
13/23
14/23
15/23
16/23
17/23
18/23
19/23
20/23
21/23
22/23
23/23

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v8 thread

Cover letter [PATCH v8 00/23] IB/Verbs: IB Management Helpers

Since v7:
* Thanks to Doug, Ira, Devesh for the testing :-)
* Thanks for the comments from or, Doug, Ira, Jason :-) Please remind me if anything missed :-P
* Use rdma_cap_XX() instead of cap_XX() for readability
* Remove CC list in git log for maintainability
* Use bool as return value
* Updated github repository to v8

There are plenty of lengthy code to check the transport type of IB device, or the link layer type of it's port, but actually we are just speculating whether a particular management/feature is supported by the device/port.

Thus instead of inferring, we should have our own mechanism for IB management capability/protocol/feature checking, several proposals below.

This patch set will introduce query_protocol() to check management requirement instead of inferring from transport and link layer respectively, along with the new enum on protocol type.

Mapping List:
node-type link-layer transport protocol
nes RNIC ETH IWARP IWARP
amso1100 RNIC ETH IWARP IWARP

cxgb3   	RNIC		ETH		IWARP		IWARP
cxgb4   	RNIC		ETH		IWARP		IWARP
usnic   	USNIC_UDP	ETH		USNIC_UDP	USNIC_UDP
ocrdma  	IB_CA		ETH		IB		IBOE
mlx4    	IB_CA		IB/ETH		IB		IB/IBOE
mlx5    	IB_CA		IB		IB		IB
ehca    	IB_CA		IB		IB		IB
ipath   	IB_CA		IB		IB		IB
mthca   	IB_CA		IB		IB		IB
qib     	IB_CA		IB		IB		IB

For example:

	if (transport == IB) && (link-layer == ETH)
will now become:
	if (query_protocol() == IBOE)

Thus we will be able to get rid of the respective transport and link-layer checking, and it will help us to add new protocol/Technology (like OPA) more easier, also with the introduced management helpers, IB management logical will be more clear and easier for extending.

Highlights:
The long CC list in each patches was complained consider about the maintainability, it was suggested folks to provide their reviewed-by or Acked-by instead, so for those who used to be on the CC list, please provide your signature voluntarily :-)

The 'mgmt-helpers' branch of 'git-9UaJU3cA/F/[email protected]:ywang-pb/infiniband-wy.git' contain this series based on the latest 'infiniband/for-next'

Patch 1#~14# included all the logical reform, 15#~23# introduced the management helpers.

Doug suggested the bitmask mechanism:
https://www.mail-archive.com/[email protected]/msg23765.html
which could be the plan for future reforming, we prefer that to be another series which focus on semantic and performance.

This patch-set is somewhat 'bloated' now and it may be a good timing for staging, I'd like to suggest we focus on improving existed helpers and push all the further reforms into next series ;-)

Proposals:

    Sean:
	https://www.mail-archive.com/[email protected]/msg23339.html
    Doug:
	https://www.mail-archive.com/[email protected]/msg23418.html
	https://www.mail-archive.com/[email protected]/msg23765.html
    Jason:
	https://www.mail-archive.com/[email protected]/msg23425.html

Michael Wang (23):
IB/Verbs: Implement new callback query_protocol() IB/Verbs: Implement raw management helpers IB/Verbs: Reform IB-core mad/agent/user_mad IB/Verbs: Reform IB-core cm
IB/Verbs: Reform IB-core sa_query
IB/Verbs: Reform IB-core multicast
IB/Verbs: Reform IB-ulp ipoib
IB/Verbs: Reform IB-ulp xprtrdma
IB/Verbs: Reform IB-core verbs
IB/Verbs: Reform cm related part in IB-core cma/ucm IB/Verbs: Reform route related part in IB-core cma IB/Verbs: Reform mcast related part in IB-core cma IB/Verbs: Reform cma_acquire_dev()
IB/Verbs: Reform rest part in IB-core cma IB/Verbs: Use management helper rdma_cap_ib_mad() IB/Verbs: Use management helper rdma_cap_ib_smi() IB/Verbs: Use management helper rdma_cap_ib_cm() IB/Verbs: Use management helper rdma_cap_iw_cm() IB/Verbs: Use management helper rdma_cap_ib_sa() IB/Verbs: Use management helper rdma_cap_ib_mcast() IB/Verbs: Use management helper rdma_cap_read_multi_sge() IB/Verbs: Use management helper rdma_cap_af_ib() IB/Verbs: Use management helper rdma_cap_eth_ah()

 drivers/infiniband/core/agent.c              |   2 +-
 drivers/infiniband/core/cm.c                 |  20 ++-
 drivers/infiniband/core/cma.c                | 257 +++++++++++----------------
 drivers/infiniband/core/device.c             |   1 +
 drivers/infiniband/core/mad.c                |  43 +++--
 drivers/infiniband/core/multicast.c          |  12 +-
 drivers/infiniband/core/sa_query.c           |  30 ++--
 drivers/infiniband/core/ucm.c                |   3 +-
 drivers/infiniband/core/ucma.c               |  25 +--
 drivers/infiniband/core/user_mad.c           |  26 ++-
 drivers/infiniband/core/verbs.c              |   6 +-
 drivers/infiniband/hw/amso1100/c2_provider.c |   7 +
 drivers/infiniband/hw/cxgb3/iwch_provider.c  |   7 +
 drivers/infiniband/hw/cxgb4/provider.c       |   7 +
 drivers/infiniband/hw/ehca/ehca_hca.c        |   6 +
 drivers/infiniband/hw/ehca/ehca_iverbs.h     |   3 +
 drivers/infiniband/hw/ehca/ehca_main.c       |   1 +
 drivers/infiniband/hw/ipath/ipath_verbs.c    |   7 +
 drivers/infiniband/hw/mlx4/main.c            |  10 ++
 drivers/infiniband/hw/mlx5/main.c            |   7 +
 drivers/infiniband/hw/mthca/mthca_provider.c |   7 +
 drivers/infiniband/hw/nes/nes_verbs.c        |   6 +
 drivers/infiniband/hw/ocrdma/ocrdma_main.c   |   1 +
 drivers/infiniband/hw/ocrdma/ocrdma_verbs.c  |   6 +
 drivers/infiniband/hw/ocrdma/ocrdma_verbs.h  |   3 +
 drivers/infiniband/hw/qib/qib_verbs.c        |   7 +
 drivers/infiniband/hw/usnic/usnic_ib_main.c  |   1 +
 drivers/infiniband/hw/usnic/usnic_ib_verbs.c |   6 +
 drivers/infiniband/hw/usnic/usnic_ib_verbs.h |   2 +
 drivers/infiniband/ulp/ipoib/ipoib_main.c    |  15 +-
 include/rdma/ib_verbs.h                      | 167 +++++++++++++++++
 net/sunrpc/xprtrdma/svc_rdma_recvfrom.c      |   4 +-
 net/sunrpc/xprtrdma/svc_rdma_transport.c     |  45 ++---
 33 files changed, 477 insertions(+), 273 deletions(-)
Signature · 6 lines
2.1.0

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html