Skip to content

Add Enhanced Connection Established (ECE) APIs

Leon Romanovsky linux-rdma 1 version New

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest12 patches 0 repliesNew

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/12Update kernel headerskernel-headers/rdma/mlx5_user_ioctl_cmds.h kernel-headers/rdma/rdma_user_cm.h+20 -10 replies
2/12libibverbs: Add interfaces to configure and use ECECMakeLists.txt debian/libibverbs1.symbols libibverbs/CMakeLists.txt +5 more+61 -30 replies
3/12libibverbs: Document ECE APIlibibverbs/man/CMakeLists.txt libibverbs/man/ibv_query_ece.3.md libibverbs/man/ibv_set_ece.3.md+119 -00 replies
4/12debian: Install all available librdmacm man pagesdebian/librdmacm-dev.install+1 -520 replies
5/12librdmacm: Provide interface to use ECE for external QPsdebian/librdmacm1.symbols librdmacm/CMakeLists.txt librdmacm/cma.c +2 more+77 -10 replies
6/12librdmacm: Connect rdma_connect to the ECElibrdmacm/cma.c librdmacm/rdma_cma_abi.h+15 -00 replies
7/12librdmacm: Return ECE results through rdma_acceptlibrdmacm/cma.c librdmacm/rdma_cma_abi.h+10 -00 replies
8/12librdmacm: Add an option to reject ECE requestdebian/librdmacm1.symbols librdmacm/cma.c librdmacm/librdmacm.map +2 more+33 -30 replies
9/12librdmacm: Implement ECE handshake logiclibrdmacm/cma.c librdmacm/rdma_cma_abi.h+40 -70 replies
10/12librdmacm: Document ECE APIlibrdmacm/man/CMakeLists.txt librdmacm/man/rdma_cm.7 librdmacm/man/rdma_get_remote_ece.3.md +1 more+138 -10 replies
11/12pyverbs: Add support for ECEpyverbs/cmid.pyx pyverbs/libibverbs.pxd pyverbs/librdmacm.pxd +2 more+77 -30 replies
12/12tests: Add test for rdmacm ECE mechanismtests/rdmacm_utils.py+17 -10 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [PATCH rdma-core 00/12] Add Enhanced Connection Established (ECE) APIs

From: Leon Romanovsky <[email protected]>

Hi,

Enhanced Connection Established or ECE is new negotiation scheme introduced in IBTA v1.4 to exchange extra information about nodes capabilities and later negotiate them at the connection establishment phase.

The RDMA-CM messages (REQ, REP, SIDR_REQ and SIDR_REP) were extended to carry two fields, one new and another gained new functionality:
* VendorID is a new field that indicates that common subset of vendor option bits are supported as indicated by that VendorID.
* AttributeModifier already exists, but overloaded to indicate which vendor options are supported by this VendorID.

The general (success) communication flow can be described by the following table:

------------------------------------------------------------------------------
Requester (client) | Responder (server)
-----------------------------------------------------------------------------

1. Create data QP.                        |
2. Get ECE information about this QP.     |
3. Update REQ message with local ECE data.|
4. Send REQ message with rdma_connect().  |
                                          | 5. Get REQ message with rdma_get_events.
                                          | 6. Read remote ECE data from the REQ message.
                                          | 7. Create data QP.
                                          | 8. Set in QP the desired ECE options by giving remote ECE data.
                                          | 9. Read accepted local ECE options.
                                          |10. Modify QP based on those options.
                                          |11. Fill local ECE options in REP message.
                                          |12. Send REP message with rdma_accept().
13. Receive REP message.                  |
14. Read remote ECE data from REP message.|
15. Set in QP remote ECE data             |
16. Modify QP based on remote ECE data    |

------------------------------------------------------------------------------

In case the server decides to reject connection, the items #9-10 will be replaced with rdma_reject_ece() call that will send REJ message together with "ECE options not supported" reason as described in the IBTA.

Thanks

Ido Kalir (2):
pyverbs: Add support for ECE
tests: Add test for rdmacm ECE mechanism

Leon Romanovsky (10):
Update kernel headers
libibverbs: Add interfaces to configure and use ECE
libibverbs: Document ECE API
debian: Install all available librdmacm man pages
librdmacm: Provide interface to use ECE for external QPs
librdmacm: Connect rdma_connect to the ECE
librdmacm: Return ECE results through rdma_accept
librdmacm: Add an option to reject ECE request
librdmacm: Implement ECE handshake logic
librdmacm: Document ECE API

 CMakeLists.txt                             |   2 +-
 debian/libibverbs1.symbols                 |   5 +-
 debian/librdmacm-dev.install               |  53 +--------
 debian/librdmacm1.symbols                  |   4 +
 kernel-headers/rdma/mlx5_user_ioctl_cmds.h |   6 +
 kernel-headers/rdma/rdma_user_cm.h         |  15 ++-
 libibverbs/CMakeLists.txt                  |   2 +-
 libibverbs/driver.h                        |   2 +
 libibverbs/dummy_ops.c                     |  14 +++
 libibverbs/libibverbs.map.in               |   6 +
 libibverbs/man/CMakeLists.txt              |   2 +
 libibverbs/man/ibv_query_ece.3.md          |  56 +++++++++
 libibverbs/man/ibv_set_ece.3.md            |  61 ++++++++++
 libibverbs/verbs.c                         |  15 +++
 libibverbs/verbs.h                         |  18 +++
 librdmacm/CMakeLists.txt                   |   2 +-
 librdmacm/cma.c                            | 130 +++++++++++++++++++--
 librdmacm/librdmacm.map                    |   7 ++
 librdmacm/man/CMakeLists.txt               |   2 +
 librdmacm/man/rdma_cm.7                    |  14 ++-
 librdmacm/man/rdma_get_remote_ece.3.md     |  61 ++++++++++
 librdmacm/man/rdma_set_local_ece.3.md      |  62 ++++++++++
 librdmacm/rdma_cma.h                       |  24 ++++
 librdmacm/rdma_cma_abi.h                   |  15 ++-
 pyverbs/cmid.pyx                           |  23 +++-
 pyverbs/libibverbs.pxd                     |   7 ++
 pyverbs/librdmacm.pxd                      |   2 +
 pyverbs/qp.pxd                             |   3 +
 pyverbs/qp.pyx                             |  45 ++++++-
 tests/rdmacm_utils.py                      |  18 ++-
 30 files changed, 606 insertions(+), 70 deletions(-)
 create mode 100644 libibverbs/man/ibv_query_ece.3.md
 create mode 100644 libibverbs/man/ibv_set_ece.3.md
 create mode 100644 librdmacm/man/rdma_get_remote_ece.3.md
 create mode 100644 librdmacm/man/rdma_set_local_ece.3.md

--
2.25.2