Skip to content

rdma_rxe: API extensions

Bob Pearson linux-rdma 3 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v7 Latest19 patches 2 repliesIn review
  2. v612 patches 6 repliesSuperseded
  3. v512 patches 8 repliesSuperseded

Changes in v7 from the cover letter

Ported this patch series forward to the current state of for-next.

Patches in v7 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/19rdma_rxe: Separat MEM into MR and MW objects.drivers/infiniband/sw/rxe/rxe_comp.c drivers/infiniband/sw/rxe/rxe_loc.h drivers/infiniband/sw/rxe/rxe_mr.c +5 more+204 -2030 replies
2/19rdma_rxe: Enable MW objectsdrivers/infiniband/sw/rxe/rxe.c drivers/infiniband/sw/rxe/rxe_param.h+7 -40 replies
3/19rdma_rxe: Let pools support both keys and indicesdrivers/infiniband/sw/rxe/rxe_pool.c drivers/infiniband/sw/rxe/rxe_pool.h+58 -470 replies
4/19rdma_rxe: make pool return values position independentdrivers/infiniband/sw/rxe/rxe_mcast.c drivers/infiniband/sw/rxe/rxe_mr.c drivers/infiniband/sw/rxe/rxe_pool.c +4 more+44 -210 replies
5/19rdma_rxe: remove void * parameters in pool APIsdrivers/infiniband/sw/rxe/rxe_pool.c drivers/infiniband/sw/rxe/rxe_pool.h drivers/infiniband/sw/rxe/rxe_verbs.c+25 -280 replies
6/19rdma_rxe: add alloc_mw and dealloc_mw verbsdrivers/infiniband/sw/rxe/Makefile drivers/infiniband/sw/rxe/rxe_loc.h drivers/infiniband/sw/rxe/rxe_mr.c +8 more+219 -870 replies
7/19rdma_rxe: Add bind_mw and invalidate_mw verbsdrivers/infiniband/sw/rxe/rxe_comp.c drivers/infiniband/sw/rxe/rxe_loc.h drivers/infiniband/sw/rxe/rxe_mr.c +8 more+512 -670 replies
8/19rdma_rxe: Add memory access through MWsdrivers/infiniband/sw/rxe/rxe_loc.h drivers/infiniband/sw/rxe/rxe_mr.c drivers/infiniband/sw/rxe/rxe_mw.c +5 more+189 -790 replies
9/19rdma_rxe: Add locked and unlocked pool APIsdrivers/infiniband/sw/rxe/rxe_pool.c drivers/infiniband/sw/rxe/rxe_pool.h+467 -1720 replies
10/19rdma_rxe: Add support for ibv_query_device_exdrivers/infiniband/sw/rxe/rxe.c drivers/infiniband/sw/rxe/rxe_verbs.c+75 -332 replies
11/19rdma_rxe: Add support for extended CQ operationsdrivers/infiniband/sw/rxe/rxe_comp.c drivers/infiniband/sw/rxe/rxe_resp.c drivers/infiniband/sw/rxe/rxe_verbs.c +2 more+52 -140 replies
12/19rdma_rxe: Add support for extended QP operationsdrivers/infiniband/sw/rxe/rxe_verbs.c+2 -00 replies
13/19rdma_rxe: Fix mcast group allocation bugdrivers/infiniband/sw/rxe/rxe_mcast.c+60 -480 replies
14/19rdma_rxe: Fix bugs in the multicast receive pathdrivers/infiniband/sw/rxe/rxe_recv.c+36 -250 replies
15/19rdma_rxe: handle ERR_PTR returns from pooldrivers/infiniband/sw/rxe/rxe_mcast.c drivers/infiniband/sw/rxe/rxe_mr.c drivers/infiniband/sw/rxe/rxe_mw.c +5 more+51 -280 replies
16/19rdma_rxe: remove duplicate entries in struct rxe_mrdrivers/infiniband/sw/rxe/rxe_mr.c drivers/infiniband/sw/rxe/rxe_req.c drivers/infiniband/sw/rxe/rxe_resp.c +2 more+11 -160 replies
17/19rdma_rme: removed unused RXE_MR_TYPE_FMRdrivers/infiniband/sw/rxe/rxe_mr.c drivers/infiniband/sw/rxe/rxe_verbs.h+0 -20 replies
18/19rdma_rxe: add rkey validation checks for MR and MWdrivers/infiniband/sw/rxe/rxe_mr.c drivers/infiniband/sw/rxe/rxe_mw.c+58 -10 replies
19/19rdma_rxe: moved rxe_xmit_packet to rxe_net.cdrivers/infiniband/sw/rxe/rxe_loc.h drivers/infiniband/sw/rxe/rxe_net.c+47 -470 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v7 yet.

Read the v7 thread

Cover letter [PATCH for-next v7 00/19] rdma_rxe: API extensions

V7:
Ported this patch series forward to the current state of for-next.

MW now being allocated in core.

Split up rxe_pool changes into patches as requested by Jason. Added some better comments.
Added error returns via ERR_PTR for APIs that return pointers.

Added some additional validation for rkeys in rxe_resp based on chapter 9 of IBA.

Minor cleanup to rxe_mr which had duplicate entries.

V6:
Fixed two issues raised by Jason and Zhu.

Undid the replacement of rwlocks by spinlocks in patch 10/12. On further reading it turns out rwlocks were the better choice.

Missing prototype for rxe_invalidate_mr. This was caused by a regression in patch 05/12 which had dropped the actual use of the routine as well as the prototype. Fixed.

V5:
This patch series is a collection of API extensions for the rdma_rxe driver. With this patch set installed there are no errors in pyverbs run-tests and 31 tests are skipped down from 56. The remaining skipped test cases include
- XRC tests
- ODP tests
- Parent device tests
- Import tests
- Device memory
- MLX5 specific tests
- EFA tests

It continues from the previous (v4) set which implemented memory windows and has had a number of individual patches picked up in for-next.

This set (v5) includes:
Ported to current head of tree
Memory windows patches not yet picked up kernel support for the extended user space APIs:
- ibv_query_device_ex
- ibv_create_cq_ex
- ibv_create_qp_ex
Fixes for multicast which is not currently working

This patch set depends on a matching rdma-core user space library patch set.

In order to run correctly it is necessary to configure by hand the EUI64 link local IPV6 address on systems which use a random link local address (like Ubuntu).

Bob Pearson (19):
rdma_rxe: Separat MEM into MR and MW objects.
rdma_rxe: Enable MW objects
rdma_rxe: Let pools support both keys and indices
rdma_rxe: make pool return values position independent
rdma_rxe: remove void * parameters in pool APIs
rdma_rxe: add alloc_mw and dealloc_mw verbs
rdma_rxe: Add bind_mw and invalidate_mw verbs
rdma_rxe: Add memory access through MWs
rdma_rxe: Add locked and unlocked pool APIs
rdma_rxe: Add support for ibv_query_device_ex
rdma_rxe: Add support for extended CQ operations
rdma_rxe: Add support for extended QP operations
rdma_rxe: Fix mcast group allocation bug
rdma_rxe: Fix bugs in the multicast receive path
rdma_rxe: handle ERR_PTR returns from pool
rdma_rxe: remove duplicate entries in struct rxe_mr
rdma_rme: removed unused RXE_MR_TYPE_FMR
rdma_rxe: add rkey validation checks for MR and MW
rdma_rxe: moved rxe_xmit_packet to rxe_net.c

 drivers/infiniband/sw/rxe/Makefile     |   1 +
 drivers/infiniband/sw/rxe/rxe.c        | 100 ++--
 drivers/infiniband/sw/rxe/rxe_comp.c   |  12 +-
 drivers/infiniband/sw/rxe/rxe_loc.h    |  91 ++--
 drivers/infiniband/sw/rxe/rxe_mcast.c  | 118 +++--
 drivers/infiniband/sw/rxe/rxe_mr.c     | 396 ++++++++-------
 drivers/infiniband/sw/rxe/rxe_mw.c     | 420 ++++++++++++++++
 drivers/infiniband/sw/rxe/rxe_net.c    |  47 +-
 drivers/infiniband/sw/rxe/rxe_opcode.c |  11 +-
 drivers/infiniband/sw/rxe/rxe_opcode.h |   1 -
 drivers/infiniband/sw/rxe/rxe_param.h  |  10 +-
 drivers/infiniband/sw/rxe/rxe_pool.c   | 653 ++++++++++++++++++-------
 drivers/infiniband/sw/rxe/rxe_pool.h   | 148 +++---
 drivers/infiniband/sw/rxe/rxe_recv.c   |  67 ++-
 drivers/infiniband/sw/rxe/rxe_req.c    | 145 ++++--
 drivers/infiniband/sw/rxe/rxe_resp.c   | 188 +++++--
 drivers/infiniband/sw/rxe/rxe_verbs.c  | 118 +++--
 drivers/infiniband/sw/rxe/rxe_verbs.h  |  71 ++-
 include/uapi/rdma/rdma_user_rxe.h      |  68 ++-
 19 files changed, 1930 insertions(+), 735 deletions(-)
 create mode 100644 drivers/infiniband/sw/rxe/rxe_mw.c
Signature · 1 line
2.25.1