EFA dmabuf memory regions
Status from this message: Applied to for-next, thanks
Get series with b4
b4 am [email protected]Versions newest first
Patches in v1 Applied
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/3 | dma-buf: Fix pin callback commentinclude/ | +2 | 1 reply | |
| 2/3 | RDMA/umem: Allow pinned dmabuf umem usagedrivers/ include/ | +63 | 0 replies | |
| 3/3 | RDMA/efa: Add support for dmabuf memory regionsdrivers/ drivers/ drivers/ | +101 | 0 replies |
Reviews trailers per patch
1 of 3 patches has a Reviewed-by or Acked-by.
| Patch | Christian König |
|---|---|
| 1/3 | R-b |
| 2/3 | |
| 3/3 |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.
Cover letter [PATCH for-next 0/3] EFA dmabuf memory regions
Hey all,
This is a followup to my previous RFCs [1][2], which now adds a new api to the RDMA subsystem that allows drivers to get a pinned dmabuf memory region without requiring an implementation of the move_notify callback. The new api makes use of the dynamic attachment api implemented in the RDMA subsystem, but calls dma_buf_pin() in order to make sure that the callback will not be called, as suggested by Christian.
As explained in the previous RFC, move_notify requires the RDMA device to support on-demand-paging (ODP) which is not common on most devices (only supported by mlx5).
While the dynamic requirement makes sense for certain GPUs, some devices (such as habanalabs) have device memory that is always "pinned" and do not need/use the move_notify operation.
Patch #1 changes the dmabuf documentation to make it clear that pinning does not necessarily mean the memory must be moved to system memory, it is up to the exporter to decide.
Patch #2 adds the RDMA api that allows drivers to get pinned dmabuf memory regions.
Patch #3 adds the EFA implementation of the dmabuf importer.
The motivation of this submission is to use habanalabs as the dmabuf exporter, and EFA as the importer to allow for peer2peer access through libibverbs.
[1] https://lore.kernel.org/linux-rdma/[email protected]/
[2] https://lore.kernel.org/linux-rdma/[email protected]/
Thanks
Gal Pressman (3):
dma-buf: Fix pin callback comment
RDMA/umem: Allow pinned dmabuf umem usage
RDMA/efa: Add support for dmabuf memory regions
drivers/infiniband/core/umem_dmabuf.c | 51 +++++++++++ drivers/infiniband/hw/efa/efa.h | 4 + drivers/infiniband/hw/efa/efa_main.c | 1 + drivers/infiniband/hw/efa/efa_verbs.c | 127 +++++++++++++++++++------- include/linux/dma-buf.h | 4 +- include/rdma/ib_umem.h | 12 +++ 6 files changed, 166 insertions(+), 33 deletions(-)
Signature · 1 line
2.33.0