Skip to content

verbs: Query GID table API

Yishai Hadas linux-rdma 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest8 patches 1 replyIn review

Changes in v1 from the cover letter

* Move some GIDs stuff to cmd_device.c with better function's names.
* Cleanup in few places.

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/8Update kernel headerskernel-headers/rdma/ib_user_ioctl_cmds.h kernel-headers/rdma/ib_user_ioctl_verbs.h kernel-headers/rdma/ib_user_verbs.h +1 more+44 -30 replies
2/8verbs: Change the name of enum ibv_gid_typelibibverbs/driver.h libibverbs/examples/devinfo.c libibverbs/verbs.c +5 more+30 -280 replies
3/8verbs: Introduce a new query GID entry APIdebian/libibverbs1.symbols libibverbs/CMakeLists.txt libibverbs/cmd_device.c +4 more+236 -10 replies
4/8verbs: Implement ibv_query_gid and ibv_query_gid_type over ioctllibibverbs/cmd_device.c libibverbs/verbs.c+117 -670 replies
5/8verbs: Optimize ibv_query_gid and ibv_query_gid_typelibibverbs/cmd_device.c libibverbs/driver.h libibverbs/verbs.c+65 -300 replies
6/8verbs: Introduce a new query GID table APIdebian/libibverbs1.symbols libibverbs/cmd_device.c libibverbs/libibverbs.map.in +3 more+199 -100 replies
7/8pyverbs: Add query_gid_table and query_gid_ex methodspyverbs/device.pxd pyverbs/device.pyx pyverbs/libibverbs.pxd +1 more+127 -00 replies
8/8tests: Add tests for ibv_query_gid_table and ibv_query_gid_extests/test_device.py+32 -00 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [PATCH V1 rdma-core 0/8] verbs: Query GID table API

When an application is not using RDMA CM and if it is using multiple RDMA devices with one or more RoCE ports, finding the right GID table entry is a long process.

For example, with two RoCE dual-port devices in a system, when IP failover is used between two RoCE ports, searching a suitable GID entry for a given source IP, matching netdevice of given RoCEv1/v2 type requires iterating over all 4 ports * 256 entry GID table.

Even though the best first match GID table for given criteria is used, when the matching entry is on the 4th port, it requires reading 3 ports * 256 entries * 3 files (GID, netdev, type) = 2304 files. The GID table needs to be referred on every QP creation during IP failover on other netdevice of an RDMA device.

We introduce this series of patches, which introduces an API to query the complete GID tables of an RDMA device, that returns all valid GID table entries.

This is done through single ioctl, eliminating 2304 read, 2304 open and 2304 close system calls to just a total of 2 calls (one for each device).

While at it, we also introduce an API to query an individual GID entry over ioctl interface, which provides all GID attributes information.

The APIs are based on the below RFC [1], the matching kernel part was sent to rdma-next.

PR was sent as well [2].

[1] https://www.spinics.net/lists/linux-rdma/msg91825.html
[2] https://github.com/linux-rdma/rdma-core/pull/828

Changelog:
v1:
* Move some GIDs stuff to cmd_device.c with better function's names.
* Cleanup in few places.

Avihai Horon (7):
verbs: Change the name of enum ibv_gid_type
verbs: Introduce a new query GID entry API
verbs: Implement ibv_query_gid and ibv_query_gid_type over ioctl
verbs: Optimize ibv_query_gid and ibv_query_gid_type
verbs: Introduce a new query GID table API
pyverbs: Add query_gid_table and query_gid_ex methods
tests: Add tests for ibv_query_gid_table and ibv_query_gid_ex

Yishai Hadas (1):
Update kernel headers

 debian/libibverbs1.symbols                |   3 +
 kernel-headers/rdma/ib_user_ioctl_cmds.h  |  16 ++
 kernel-headers/rdma/ib_user_ioctl_verbs.h |  14 ++
 kernel-headers/rdma/ib_user_verbs.h       |  11 ++
 kernel-headers/rdma/rdma_user_rxe.h       |   6 +-
 libibverbs/CMakeLists.txt                 |   2 +-
 libibverbs/cmd_device.c                   | 314 ++++++++++++++++++++++++++++++
 libibverbs/driver.h                       |  19 +-
 libibverbs/examples/devinfo.c             |  14 +-
 libibverbs/libibverbs.map.in              |   6 +
 libibverbs/man/CMakeLists.txt             |   2 +
 libibverbs/man/ibv_query_gid_ex.3.md      |  93 +++++++++
 libibverbs/man/ibv_query_gid_table.3.md   |  73 +++++++
 libibverbs/verbs.c                        | 105 ++++------
 libibverbs/verbs.h                        |  45 +++++
 providers/mlx5/verbs.c                    |   2 +-
 pyverbs/device.pxd                        |   3 +
 pyverbs/device.pyx                        | 108 +++++++++-
 pyverbs/libibverbs.pxd                    |  15 +-
 pyverbs/libibverbs_enums.pxd              |  11 +-
 tests/base.py                             |   3 +-
 tests/test_device.py                      |  32 +++
 22 files changed, 804 insertions(+), 93 deletions(-)
 create mode 100644 libibverbs/man/ibv_query_gid_ex.3.md
 create mode 100644 libibverbs/man/ibv_query_gid_table.3.md
Signature · 1 line
1.8.3.1