EFA dmabuf memory regions
cover EFA dmabuf memory regions
Hey all,
This is a followup to my previous RFCs [1][2], which now adds a new api to the RDMA subsystem that allows drivers to get a pinned dmabuf memory region without requiring an implementation of the move_notify callback. The new api makes use of the dynamic attachment api implemented in the RDMA subsystem, but calls dma_buf_pin() in order to make sure that the callback will not be called, as suggested by Christian.
As explained in the previous RFC, move_notify requires the RDMA device to support on-demand-paging (ODP) which is not common on most devices (only supported by mlx5).
While the dynamic requirement makes sense for certain GPUs, some devices (such as habanalabs) have device memory that is always "pinned" and do not need/use the move_notify operation.
Patch #1 changes the dmabuf documentation to make it clear that pinning does not necessarily mean the memory must be moved to system memory, it is up to the exporter to decide.
Patch #2 adds the RDMA api that allows drivers to get pinned dmabuf memory regions.
Patch #3 adds the EFA implementation of the dmabuf importer.
The motivation of this submission is to use habanalabs as the dmabuf exporter, and EFA as the importer to allow for peer2peer access through libibverbs.
[1] https://lore.kernel.org/linux-rdma/[email protected]/
[2] https://lore.kernel.org/linux-rdma/[email protected]/
Thanks
Gal Pressman (3):
dma-buf: Fix pin callback comment
RDMA/umem: Allow pinned dmabuf umem usage
RDMA/efa: Add support for dmabuf memory regions
drivers/infiniband/core/umem_dmabuf.c | 51 +++++++++++ drivers/infiniband/hw/efa/efa.h | 4 + drivers/infiniband/hw/efa/efa_main.c | 1 + drivers/infiniband/hw/efa/efa_verbs.c | 127 +++++++++++++++++++------- include/linux/dma-buf.h | 4 +- include/rdma/ib_umem.h | 12 +++ 6 files changed, 166 insertions(+), 33 deletions(-)
Signature · 1 line
2.33.0
PATCH 1/3 dma-buf: Fix pin callback comment
The pin callback does not necessarily have to move the memory to system memory, remove the sentence from the comment.
--- include/linux/dma-buf.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
include/linux/dma-buf.h +2 -2
@@ -86,8 +86,8 @@ struct dma_buf_ops {* @pin: * * This is called by dma_buf_pin() and lets the exporter know that the -* DMA-buf can't be moved any more. The exporter should pin the buffer -* into system memory to make sure it is generally accessible by other +* DMA-buf can't be moved any more. Ideally, the exporter should +* pin the buffer so that it is generally accessible by all * devices. * * This is called with the &dmabuf.resv object locked and is mutual
Signature · 1 line
2.33.0
PATCH 3/3 RDMA/efa: Add support for dmabuf memory regions
In reply to Gal PressmanImplement a dmabuf importer for the EFA driver. As ODP is not supported, the pinned dmabuf are used to prevent the move_notify callback from being called.
--- drivers/infiniband/hw/efa/efa.h | 4 + drivers/infiniband/hw/efa/efa_main.c | 1 + drivers/infiniband/hw/efa/efa_verbs.c | 127 +++++++++++++++++++------- 3 files changed, 101 insertions(+), 31 deletions(-)
drivers/infiniband/hw/efa/efa.h +4 -0
@@ -141,6 +141,10 @@ int efa_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,struct ib_mr *efa_reg_mr (struct ib_pd *ibpd,u64 start,u64 length,u64 virt_addr,int access_flags,struct ib_udata *udata); +struct ib_mr *efa_reg_user_mr_dmabuf (struct ib_pd *ibpd,u64 start, +u64 length,u64 virt_addr, +int fd,int access_flags, +struct ib_udata *udata);int efa_dereg_mr (struct ib_mr *ibmr,struct ib_udata *udata);int efa_get_port_immutable (struct ib_device *ibdev,u32 port_num,struct ib_port_immutable *immutable);
drivers/infiniband/hw/efa/efa_main.c +1 -0
@@ -267,6 +267,7 @@ static const struct ib_device_ops efa_dev_ops = { .query_port = efa_query_port, .query_qp = efa_query_qp, .reg_user_mr = efa_reg_mr, + .reg_user_mr_dmabuf = efa_reg_user_mr_dmabuf,INIT_RDMA_OBJ_SIZE (ib_ah, efa_ah, ibah), INIT_RDMA_OBJ_SIZE(ib_cq, efa_cq, ibcq),
drivers/infiniband/hw/efa/efa_verbs.c +96 -31
191 lines not on this page · raw message
Signature · 1 line
2.33.0
PATCH 2/3 RDMA/umem: Allow pinned dmabuf umem usage
In reply to Gal PressmanIntroduce ib_umem_dmabuf_get_pinned() which allows the driver to get a dmabuf umem which is pinned and does not require move_notify callback implementation.
The returned umem is pinned and DMA mapped like standard cpu umems, and is released through ib_umem_release() (incl. unpinning and unmapping).
--- drivers/infiniband/core/umem_dmabuf.c | 51 +++++++++++++++++++++++++++ include/rdma/ib_umem.h | 12 +++++++ 2 files changed, 63 insertions(+)
drivers/infiniband/core/umem_dmabuf.c +51 -0
@@ -164,12 +164,63 @@ struct ib_umem_dmabuf *ib_umem_dmabuf_get(struct ib_device *device, }EXPORT_SYMBOL (ib_umem_dmabuf_get); +static void +ib_umem_dmabuf_unsupported_move_notify (struct dma_buf_attachment *attach) +{ +struct ib_umem_dmabuf *umem_dmabuf = attach->importer_priv; + +ibdev_warn_ratelimited (umem_dmabuf->umem.ibdev, +"Invalidate callback should not be called when memory is pinned\n" ); +} + +static struct dma_buf_attach_ops ib_umem_dmabuf_attach_pinned_ops = { + .allow_peer2peer = true, + .move_notify = ib_umem_dmabuf_unsupported_move_notify, +}; + +struct ib_umem_dmabuf *ib_umem_dmabuf_get_pinned (struct ib_device *device, +unsigned long offset, +size_t size,int fd, +int access) +{ +struct ib_umem_dmabuf *umem_dmabuf; +int err; + + umem_dmabuf =ib_umem_dmabuf_get (device, offset, size, fd, access, + &ib_umem_dmabuf_attach_pinned_ops); +if (IS_ERR (umem_dmabuf)) +return umem_dmabuf; + +dma_resv_lock (umem_dmabuf->attach->dmabuf->resv,NULL ); + err =dma_buf_pin (umem_dmabuf->attach); +if (err) + goto err_release; + umem_dmabuf->pinned =1 ; + + err =ib_umem_dmabuf_map_pages (umem_dmabuf); +if (err) + goto err_unpin; +dma_resv_unlock (umem_dmabuf->attach->dmabuf->resv); + +return umem_dmabuf; + +err_unpin: +dma_buf_unpin (umem_dmabuf->attach); +err_release: +dma_resv_unlock (umem_dmabuf->attach->dmabuf->resv); +ib_umem_release (&umem_dmabuf->umem); +return ERR_PTR (err); +} +EXPORT_SYMBOL (ib_umem_dmabuf_get_pinned); +void ib_umem_dmabuf_release (struct ib_umem_dmabuf *umem_dmabuf) {struct dma_buf *dmabuf = umem_dmabuf->attach->dmabuf;dma_resv_lock (dmabuf->resv,NULL );ib_umem_dmabuf_unmap_pages (umem_dmabuf); +if (umem_dmabuf->pinned) +dma_buf_unpin (umem_dmabuf->attach);dma_resv_unlock (dmabuf->resv);dma_buf_detach (dmabuf, umem_dmabuf->attach);
include/rdma/ib_umem.h +12 -0
@@ -40,6 +40,7 @@ struct ib_umem_dmabuf {unsigned long first_sg_offset;unsigned long last_sg_trim;void *private; + u8 pinned : 1; };static inline struct ib_umem_dmabuf *to_ib_umem_dmabuf (struct ib_umem *umem)
@@ -140,6 +141,10 @@ struct ib_umem_dmabuf *ib_umem_dmabuf_get(struct ib_device *device, unsigned long offset, size_t size, int fd, int access, const struct dma_buf_attach_ops *ops); +struct ib_umem_dmabuf *ib_umem_dmabuf_get_pinned (struct ib_device *device, +unsigned long offset, +size_t size,int fd, +int access);int ib_umem_dmabuf_map_pages (struct ib_umem_dmabuf *umem_dmabuf);void ib_umem_dmabuf_unmap_pages (struct ib_umem_dmabuf *umem_dmabuf);void ib_umem_dmabuf_release (struct ib_umem_dmabuf *umem_dmabuf);
@@ -180,6 +185,13 @@ struct ib_umem_dmabuf *ib_umem_dmabuf_get(struct ib_device *device, {return ERR_PTR (-EOPNOTSUPP ); } +struct ib_umem_dmabuf *ib_umem_dmabuf_get_pinned (struct ib_device *device, +unsigned long offset, +size_t size,int fd, +int access) +{ +return ERR_PTR (-EOPNOTSUPP ); +}static inline int ib_umem_dmabuf_map_pages (struct ib_umem_dmabuf *umem_dmabuf) {return -EOPNOTSUPP ;
Signature · 1 line
2.33.0
Re: [PATCH for-next 1/3] dma-buf: Fix pin callback comment
In reply to Gal PressmanAm 12.10.21 um 14:09 schrieb Gal Pressman:
Quoted 4 lines
The pin callback does not necessarily have to move the memory to system memory, remove the sentence from the comment.
Signed-off-by Gal Pressman <[email protected]>
Quoted 19 lines
--- include/linux/dma-buf.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)include/linux/dma-buf.h +2
-2@@ -86,8 +86,8 @@ struct dma_buf_ops {* @pin: * * This is called by dma_buf_pin() and lets the exporter know that the -* DMA-buf can't be moved any more. The exporter should pin the buffer -* into system memory to make sure it is generally accessible by other +* DMA-buf can't be moved any more. Ideally, the exporter should +* pin the buffer so that it is generally accessible by all * devices. * * This is called with the &dmabuf.resv object locked and is mutual
Gal Pressman
reviewerQuoted 24 lines · Gal Pressman
Hey all,
This is a followup to my previous RFCs [1][2], which now adds a new api to the RDMA subsystem that allows drivers to get a pinned dmabuf memory region without requiring an implementation of the
move_notifycallback. The new api makes use of the dynamic attachment api implemented in the RDMA subsystem, but callsdma_buf_pin()in order to make sure that the callback will not be called, as suggested by Christian.As explained in the previous RFC,
move_notifyrequires the RDMA device to support on-demand-paging (ODP) which is not common on most devices (only supported by mlx5).While the dynamic requirement makes sense for certain GPUs, some devices (such as habanalabs) have device memory that is always "pinned" and do not need/use the
move_notifyoperation.Patch #1 changes the dmabuf documentation to make it clear that pinning does not necessarily mean the memory must be moved to system memory, it is up to the exporter to decide.
Patch #2 adds the RDMA api that allows drivers to get pinned dmabuf memory regions.
Patch #3 adds the EFA implementation of the dmabuf importer.
Quoted 8 lines
The motivation of this submission is to use habanalabs as the dmabuf exporter, and EFA as the importer to allow for peer2peer access through libibverbs.
[1] https://lore.kernel.org/linux-rdma/[email protected]/
[2] https://lore.kernel.org/linux-rdma/[email protected]/Thanks
Hey Jason, did you get a chance to take a look?
Quoted 31 lines · Gal Pressman
Hey all,
This is a followup to my previous RFCs [1][2], which now adds a new api to the RDMA subsystem that allows drivers to get a pinned dmabuf memory region without requiring an implementation of the
move_notifycallback. The new api makes use of the dynamic attachment api implemented in the RDMA subsystem, but callsdma_buf_pin()in order to make sure that the callback will not be called, as suggested by Christian.As explained in the previous RFC,
move_notifyrequires the RDMA device to support on-demand-paging (ODP) which is not common on most devices (only supported by mlx5).While the dynamic requirement makes sense for certain GPUs, some devices (such as habanalabs) have device memory that is always "pinned" and do not need/use the
move_notifyoperation.Patch #1 changes the dmabuf documentation to make it clear that pinning does not necessarily mean the memory must be moved to system memory, it is up to the exporter to decide.
Patch #2 adds the RDMA api that allows drivers to get pinned dmabuf memory regions.
Patch #3 adds the EFA implementation of the dmabuf importer.The motivation of this submission is to use habanalabs as the dmabuf exporter, and EFA as the importer to allow for peer2peer access through libibverbs.
[1] https://lore.kernel.org/linux-rdma/[email protected]/
[2] https://lore.kernel.org/linux-rdma/[email protected]/
Quoted 6 lines
Thanks
Gal Pressman (3):
dma-buf: Fix pin callback comment
RDMA/umem: Allow pinned dmabuf umem usage
RDMA/efa: Add support for dmabuf memory regions
Applied to for-next, thanks
Jason