Skip to content

Use ODP MRs for kernel ULPs

Leon Romanovsky linux-rdma netdev oe-kbuild-all 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest10 patches 12 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/10IB: Allow calls to ib_umem_get from kernel ULPsdrivers/infiniband/core/umem.c drivers/infiniband/core/umem_odp.c drivers/infiniband/hw/bnxt_re/ib_verbs.c +31 more+85 -1030 replies
2/10IB/core: Introduce ib_reg_user_mrdrivers/infiniband/core/verbs.c drivers/infiniband/hw/efa/efa_verbs.c drivers/infiniband/hw/i40iw/i40iw_verbs.c +1 more+40 -10 replies
3/10IB/core: Add interface to advise_mr for kernel usersdrivers/infiniband/core/verbs.c include/rdma/ib_verbs.h+14 -00 replies
4/10IB/mlx5: Add ODP WQE handlers for kernel QPsdrivers/infiniband/hw/mlx5/mlx5_ib.h drivers/infiniband/hw/mlx5/odp.c drivers/infiniband/hw/mlx5/qp.c+117 -700 replies
5/10RDMA/mlx5: Don't fake udata for kernel pathdrivers/infiniband/hw/mlx5/main.c+16 -180 replies
6/10IB/mlx5: Mask out unsupported ODP capabilities for kernel QPsdrivers/infiniband/hw/mlx5/main.c+17 -00 replies
7/10RDMA/mlx5: Fix handling of IOVA != user_va in ODP pathsdrivers/infiniband/hw/mlx5/mr.c drivers/infiniband/hw/mlx5/odp.c+15 -60 replies
8/10net/rds: Detect need of On-Demand-Paging memory registrationnet/rds/rdma.c+4 -21 reply
9/10net/rds: Handle ODP mr registration/unregistrationnet/rds/ib.c net/rds/ib.h net/rds/ib_mr.h +4 more+242 -565 replies
10/10net/rds: Use prefetch for On-Demand-Paging MRnet/rds/ib_rdma.c+9 -01 reply

Reviews trailers per patch

5 of 10 patches have a Reviewed-by or Acked-by.

PatchGuy LeviSantosh Shilimkar
1/10R-b
2/10R-b
3/10
4/10
5/10
6/10
7/10
8/10A-b
9/10A-b
10/10A-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v1 thread

Cover letter [PATCH mlx5-next 00/10] Use ODP MRs for kernel ULPs

From: Leon Romanovsky <[email protected]>

Hi,

The following series extends MR creation routines to allow creation of user MRs through kernel ULPs as a proxy. The immediate use case is to allow RDS to work over FS-DAX, which requires ODP (on-demand-paging) MRs to be created and such MRs were not possible to create prior this series.

The first part of this patchset extends RDMA to have special verb ib_reg_user_mr(). The common use case that uses this function is a userspace application that allocates memory for HCA access but the responsibility to register the memory at the HCA is on an kernel ULP. This ULP that acts as an agent for the userspace application.

The second part provides advise MR functionality for ULPs. This is integral part of ODP flows and used to trigger pagefaults in advance to prepare memory before running working set.

The third part is actual user of those in-kernel APIs.

Thanks

Hans Westgaard Ry (3):
net/rds: Detect need of On-Demand-Paging memory registration net/rds: Handle ODP mr registration/unregistration net/rds: Use prefetch for On-Demand-Paging MR

Jason Gunthorpe (1):
RDMA/mlx5: Fix handling of IOVA != user_va in ODP paths

Leon Romanovsky (1):
RDMA/mlx5: Don't fake udata for kernel path

Moni Shoua (5):
IB: Allow calls to ib_umem_get from kernel ULPs IB/core: Introduce ib_reg_user_mr
IB/core: Add interface to advise_mr for kernel users IB/mlx5: Add ODP WQE handlers for kernel QPs IB/mlx5: Mask out unsupported ODP capabilities for kernel QPs

 drivers/infiniband/core/umem.c                |  27 +--
 drivers/infiniband/core/umem_odp.c            |  29 +--
 drivers/infiniband/core/verbs.c               |  41 +++++
 drivers/infiniband/hw/bnxt_re/ib_verbs.c      |  12 +-
 drivers/infiniband/hw/cxgb4/mem.c             |   2 +-
 drivers/infiniband/hw/efa/efa_verbs.c         |   4 +-
 drivers/infiniband/hw/hns/hns_roce_cq.c       |   2 +-
 drivers/infiniband/hw/hns/hns_roce_db.c       |   3 +-
 drivers/infiniband/hw/hns/hns_roce_mr.c       |   4 +-
 drivers/infiniband/hw/hns/hns_roce_qp.c       |   2 +-
 drivers/infiniband/hw/hns/hns_roce_srq.c      |   5 +-
 drivers/infiniband/hw/i40iw/i40iw_verbs.c     |   5 +-
 drivers/infiniband/hw/mlx4/cq.c               |   2 +-
 drivers/infiniband/hw/mlx4/doorbell.c         |   3 +-
 drivers/infiniband/hw/mlx4/mr.c               |   8 +-
 drivers/infiniband/hw/mlx4/qp.c               |   5 +-
 drivers/infiniband/hw/mlx4/srq.c              |   3 +-
 drivers/infiniband/hw/mlx5/cq.c               |   6 +-
 drivers/infiniband/hw/mlx5/devx.c             |   2 +-
 drivers/infiniband/hw/mlx5/doorbell.c         |   3 +-
 drivers/infiniband/hw/mlx5/main.c             |  51 ++++--
 drivers/infiniband/hw/mlx5/mlx5_ib.h          |  12 +-
 drivers/infiniband/hw/mlx5/mr.c               |  20 +--
 drivers/infiniband/hw/mlx5/odp.c              |  33 ++--
 drivers/infiniband/hw/mlx5/qp.c               | 167 +++++++++++-------
 drivers/infiniband/hw/mlx5/srq.c              |   2 +-
 drivers/infiniband/hw/mthca/mthca_provider.c  |   2 +-
 drivers/infiniband/hw/ocrdma/ocrdma_verbs.c   |   2 +-
 drivers/infiniband/hw/qedr/verbs.c            |   9 +-
 drivers/infiniband/hw/vmw_pvrdma/pvrdma_cq.c  |   2 +-
 drivers/infiniband/hw/vmw_pvrdma/pvrdma_mr.c  |   2 +-
 drivers/infiniband/hw/vmw_pvrdma/pvrdma_qp.c  |   7 +-
 drivers/infiniband/hw/vmw_pvrdma/pvrdma_srq.c |   2 +-
 drivers/infiniband/sw/rdmavt/mr.c             |   2 +-
 drivers/infiniband/sw/rxe/rxe_mr.c            |   2 +-
 include/rdma/ib_umem.h                        |   4 +-
 include/rdma/ib_umem_odp.h                    |   6 +-
 include/rdma/ib_verbs.h                       |   9 +
 net/rds/ib.c                                  |   7 +
 net/rds/ib.h                                  |   3 +-
 net/rds/ib_mr.h                               |   7 +-
 net/rds/ib_rdma.c                             |  83 ++++++++-
 net/rds/ib_send.c                             |  44 +++--
 net/rds/rdma.c                                | 156 +++++++++++-----
 net/rds/rds.h                                 |  13 +-
 45 files changed, 559 insertions(+), 256 deletions(-)

--
2.20.1