Skip to content

RDMA: Extend uverbs umem support for QP buffers and doorbell records

Jiri Pirko linux-rdma 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest10 patches 23 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/10RDMA/umem: Add reference counting to ib_umemdrivers/infiniband/core/umem.c drivers/infiniband/core/umem_dmabuf.c drivers/infiniband/core/umem_odp.c +1 more+18 -021 replies
2/10RDMA/uverbs: Use umem refcounting for CQ creation with external bufferdrivers/infiniband/core/uverbs_std_types_cq.c drivers/infiniband/hw/efa/efa_verbs.c+7 -10 replies
3/10RDMA/mlx5: Add support for CQ creation with external umem bufferdrivers/infiniband/hw/mlx5/cq.c drivers/infiniband/hw/mlx5/main.c drivers/infiniband/hw/mlx5/mlx5_ib.h+36 -90 replies
4/10RDMA/uverbs: Factor out common buffer umem parsing into helperdrivers/infiniband/core/uverbs_ioctl.c drivers/infiniband/core/uverbs_std_types_cq.c include/rdma/uverbs_ioctl.h+87 -570 replies
5/10RDMA/core: Add support for QP buffer umem in QP creationdrivers/infiniband/core/core_priv.h drivers/infiniband/core/device.c drivers/infiniband/core/uverbs_std_types_qp.c +3 more+132 -160 replies
6/10RDMA/mlx5: Add support for QP creation with external umem buffersdrivers/infiniband/hw/mlx5/main.c drivers/infiniband/hw/mlx5/mlx5_ib.h drivers/infiniband/hw/mlx5/qp.c+71 -140 replies
7/10RDMA/uverbs: Add doorbell record umem support to CQ creationdrivers/infiniband/core/uverbs_std_types_cq.c drivers/infiniband/hw/efa/efa.h drivers/infiniband/hw/efa/efa_verbs.c +4 more+45 -120 replies
8/10RDMA/mlx5: Add external doorbell record umem support for CQdrivers/infiniband/hw/mlx5/cq.c drivers/infiniband/hw/mlx5/doorbell.c drivers/infiniband/hw/mlx5/mlx5_ib.h +2 more+34 -130 replies
9/10RDMA/uverbs: Add doorbell record umem support to QP creationdrivers/infiniband/core/core_priv.h drivers/infiniband/core/uverbs_std_types_qp.c drivers/infiniband/core/verbs.c +4 more+83 -110 replies
10/10RDMA/mlx5: Add external doorbell record umem support for QPdrivers/infiniband/hw/mlx5/qp.c+14 -70 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [PATCH rdma-next 00/10] RDMA: Extend uverbs umem support for QP buffers and doorbell records

From: Jiri Pirko <[email protected]>

This patchset extends the existing CQ umem buffer infrastructure to QP creation and adds doorbell record (DBR) umem support for both CQ and QP.

The kernel already supports CQ creation with externally provided umem buffers. This patchset:

1. Adds umem reference counting to simplify memory lifecycle management.
2. Adds mlx5 driver implementation for the existing CQ buffer umem support.
3. Extends QP creation to support external SQ/RQ buffer umems.
4. Adds doorbell record umem support for both CQ and QP.
5. Implements all extensions in mlx5 driver.

This enables integration with dmabuf exporters for specialized memory types, such as GPU memory for GPU-direct RDMA scenarios.

Patch #1 introduces reference counting for ib_umem objects to simplify memory lifecycle management when umem buffers are shared between the core layer and device drivers.
Patch #2 adopts the new ib_umem refcounting model for create_cq_umem. Patch #3 implements mlx5 driver support for the existing CQ buffer umem interface.
Patch #4 factors out common buffer umem parsing into a shared helper. Patch #5 adds core framework for QP buffer umem support with new uverbs attributes for SQ/RQ buffers.
Patch #6 implements mlx5 driver support for QP buffer umem extension. Patch #7 extends create_cq_umem device op with DBR umem parameter and adds new uverbs attributes for CQ DBR.
Patch #8 implements mlx5 driver support for CQ DBR umem and extends the doorbell mapping infrastructure.
Patch #9 extends create_qp_umem device op with DBR umem parameters and adds new uverbs attributes for SQ/RQ DBR.
Patch #10 implements mlx5 driver support for QP DBR umem.

Jiri Pirko (10):
RDMA/umem: Add reference counting to ib_umem
RDMA/uverbs: Use umem refcounting for CQ creation with external buffer RDMA/mlx5: Add support for CQ creation with external umem buffer RDMA/uverbs: Factor out common buffer umem parsing into helper RDMA/core: Add support for QP buffer umem in QP creation RDMA/mlx5: Add support for QP creation with external umem buffers RDMA/uverbs: Add doorbell record umem support to CQ creation RDMA/mlx5: Add external doorbell record umem support for CQ RDMA/uverbs: Add doorbell record umem support to QP creation RDMA/mlx5: Add external doorbell record umem support for QP

 drivers/infiniband/core/core_priv.h           |   9 ++
 drivers/infiniband/core/device.c              |   1 +
 drivers/infiniband/core/umem.c                |   5 +
 drivers/infiniband/core/umem_dmabuf.c         |   1 +
 drivers/infiniband/core/umem_odp.c            |   3 +
 drivers/infiniband/core/uverbs_ioctl.c        |  68 +++++++++++
 drivers/infiniband/core/uverbs_std_types_cq.c | 104 +++++++---------
 drivers/infiniband/core/uverbs_std_types_qp.c | 114 +++++++++++++++++-
 drivers/infiniband/core/verbs.c               |  72 +++++++++--
 drivers/infiniband/hw/efa/efa.h               |   3 +-
 drivers/infiniband/hw/efa/efa_verbs.c         |  10 +-
 drivers/infiniband/hw/mlx5/cq.c               |  47 ++++++--
 drivers/infiniband/hw/mlx5/doorbell.c         |  27 ++++-
 drivers/infiniband/hw/mlx5/main.c             |   2 +
 drivers/infiniband/hw/mlx5/mlx5_ib.h          |   9 +-
 drivers/infiniband/hw/mlx5/qp.c               |  93 +++++++++++---
 drivers/infiniband/hw/mlx5/srq.c              |   2 +-
 include/rdma/ib_umem.h                        |   9 ++
 include/rdma/ib_verbs.h                       |   7 ++
 include/rdma/uverbs_ioctl.h                   |   7 ++
 include/uapi/rdma/ib_user_ioctl_cmds.h        |  20 +++
 21 files changed, 500 insertions(+), 113 deletions(-)
Signature · 1 line
2.51.1