Skip to contentSkip to conversation

EFA dmabuf memory regions

PATCH Applied Gal Pressman 4 participants 7 messages Oct 12, 2021 – Oct 27, 2021

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

cover EFA dmabuf memory regions

Hey all,

This is a followup to my previous RFCs [1][2], which now adds a new api to the RDMA subsystem that allows drivers to get a pinned dmabuf memory region without requiring an implementation of the move_notify callback. The new api makes use of the dynamic attachment api implemented in the RDMA subsystem, but calls dma_buf_pin() in order to make sure that the callback will not be called, as suggested by Christian.

As explained in the previous RFC, move_notify requires the RDMA device to support on-demand-paging (ODP) which is not common on most devices (only supported by mlx5).

While the dynamic requirement makes sense for certain GPUs, some devices (such as habanalabs) have device memory that is always "pinned" and do not need/use the move_notify operation.

Patch #1 changes the dmabuf documentation to make it clear that pinning does not necessarily mean the memory must be moved to system memory, it is up to the exporter to decide.
Patch #2 adds the RDMA api that allows drivers to get pinned dmabuf memory regions.
Patch #3 adds the EFA implementation of the dmabuf importer.

The motivation of this submission is to use habanalabs as the dmabuf exporter, and EFA as the importer to allow for peer2peer access through libibverbs.

[1] https://lore.kernel.org/linux-rdma/[email protected]/
[2] https://lore.kernel.org/linux-rdma/[email protected]/

Thanks

Gal Pressman (3):
dma-buf: Fix pin callback comment
RDMA/umem: Allow pinned dmabuf umem usage
RDMA/efa: Add support for dmabuf memory regions

 drivers/infiniband/core/umem_dmabuf.c |  51 +++++++++++
 drivers/infiniband/hw/efa/efa.h       |   4 +
 drivers/infiniband/hw/efa/efa_main.c  |   1 +
 drivers/infiniband/hw/efa/efa_verbs.c | 127 +++++++++++++++++++-------
 include/linux/dma-buf.h               |   4 +-
 include/rdma/ib_umem.h                |  12 +++
 6 files changed, 166 insertions(+), 33 deletions(-)
Signature · 1 line
2.33.0
#1 lore raw

PATCH 1/3 dma-buf: Fix pin callback comment

The pin callback does not necessarily have to move the memory to system memory, remove the sentence from the comment.

---
 include/linux/dma-buf.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
include/linux/dma-buf.h +2 -2
index efdc56b9d95f..225e09caeb98 100644
@@ -86,8 +86,8 @@ struct dma_buf_ops {
 	 * @pin:
 	 *
 	 * This is called by dma_buf_pin() and lets the exporter know that the
-	 * DMA-buf can't be moved any more. The exporter should pin the buffer
-	 * into system memory to make sure it is generally accessible by other
+	 * DMA-buf can't be moved any more. Ideally, the exporter should
+	 * pin the buffer so that it is generally accessible by all
 	 * devices.
 	 *
 	 * This is called with the &dmabuf.resv object locked and is mutual
Signature · 1 line
2.33.0
#4 lore raw

PATCH 3/3 RDMA/efa: Add support for dmabuf memory regions

In reply to Gal Pressman

Implement a dmabuf importer for the EFA driver. As ODP is not supported, the pinned dmabuf are used to prevent the move_notify callback from being called.

---
 drivers/infiniband/hw/efa/efa.h       |   4 +
 drivers/infiniband/hw/efa/efa_main.c  |   1 +
 drivers/infiniband/hw/efa/efa_verbs.c | 127 +++++++++++++++++++-------
 3 files changed, 101 insertions(+), 31 deletions(-)
drivers/infiniband/hw/efa/efa.h +4 -0
index 2b8ca099b381..407d7c4baa16 100644
@@ -141,6 +141,10 @@ int efa_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
 struct ib_mr *efa_reg_mr(struct ib_pd *ibpd, u64 start, u64 length,
 			 u64 virt_addr, int access_flags,
 			 struct ib_udata *udata);
+struct ib_mr *efa_reg_user_mr_dmabuf(struct ib_pd *ibpd, u64 start,
+				     u64 length, u64 virt_addr,
+				     int fd, int access_flags,
+				     struct ib_udata *udata);
 int efa_dereg_mr(struct ib_mr *ibmr, struct ib_udata *udata);
 int efa_get_port_immutable(struct ib_device *ibdev, u32 port_num,
 			   struct ib_port_immutable *immutable);
drivers/infiniband/hw/efa/efa_main.c +1 -0
index 203e6ddcacbc..72cd7d952a07 100644
@@ -267,6 +267,7 @@ static const struct ib_device_ops efa_dev_ops = {
 	.query_port = efa_query_port,
 	.query_qp = efa_query_qp,
 	.reg_user_mr = efa_reg_mr,
+	.reg_user_mr_dmabuf = efa_reg_user_mr_dmabuf,
 
 	INIT_RDMA_OBJ_SIZE(ib_ah, efa_ah, ibah),
 	INIT_RDMA_OBJ_SIZE(ib_cq, efa_cq, ibcq),
drivers/infiniband/hw/efa/efa_verbs.c +96 -31

191 lines not on this page · raw message

Signature · 1 line
2.33.0
#3 lore raw

PATCH 2/3 RDMA/umem: Allow pinned dmabuf umem usage

In reply to Gal Pressman

Introduce ib_umem_dmabuf_get_pinned() which allows the driver to get a dmabuf umem which is pinned and does not require move_notify callback implementation.
The returned umem is pinned and DMA mapped like standard cpu umems, and is released through ib_umem_release() (incl. unpinning and unmapping).

---
 drivers/infiniband/core/umem_dmabuf.c | 51 +++++++++++++++++++++++++++
 include/rdma/ib_umem.h                | 12 +++++++
 2 files changed, 63 insertions(+)
drivers/infiniband/core/umem_dmabuf.c +51 -0
index c6e875619fac..4fd6abc4dd17 100644
@@ -164,12 +164,63 @@ struct ib_umem_dmabuf *ib_umem_dmabuf_get(struct ib_device *device,
 }
 EXPORT_SYMBOL(ib_umem_dmabuf_get);
 
+static void
+ib_umem_dmabuf_unsupported_move_notify(struct dma_buf_attachment *attach)
+{
+	struct ib_umem_dmabuf *umem_dmabuf = attach->importer_priv;
+
+	ibdev_warn_ratelimited(umem_dmabuf->umem.ibdev,
+			       "Invalidate callback should not be called when memory is pinned\n");
+}
+
+static struct dma_buf_attach_ops ib_umem_dmabuf_attach_pinned_ops = {
+	.allow_peer2peer = true,
+	.move_notify = ib_umem_dmabuf_unsupported_move_notify,
+};
+
+struct ib_umem_dmabuf *ib_umem_dmabuf_get_pinned(struct ib_device *device,
+						 unsigned long offset,
+						 size_t size, int fd,
+						 int access)
+{
+	struct ib_umem_dmabuf *umem_dmabuf;
+	int err;
+
+	umem_dmabuf = ib_umem_dmabuf_get(device, offset, size, fd, access,
+					 &ib_umem_dmabuf_attach_pinned_ops);
+	if (IS_ERR(umem_dmabuf))
+		return umem_dmabuf;
+
+	dma_resv_lock(umem_dmabuf->attach->dmabuf->resv, NULL);
+	err = dma_buf_pin(umem_dmabuf->attach);
+	if (err)
+		goto err_release;
+	umem_dmabuf->pinned = 1;
+
+	err = ib_umem_dmabuf_map_pages(umem_dmabuf);
+	if (err)
+		goto err_unpin;
+	dma_resv_unlock(umem_dmabuf->attach->dmabuf->resv);
+
+	return umem_dmabuf;
+
+err_unpin:
+	dma_buf_unpin(umem_dmabuf->attach);
+err_release:
+	dma_resv_unlock(umem_dmabuf->attach->dmabuf->resv);
+	ib_umem_release(&umem_dmabuf->umem);
+	return ERR_PTR(err);
+}
+EXPORT_SYMBOL(ib_umem_dmabuf_get_pinned);
+
 void ib_umem_dmabuf_release(struct ib_umem_dmabuf *umem_dmabuf)
 {
 	struct dma_buf *dmabuf = umem_dmabuf->attach->dmabuf;
 
 	dma_resv_lock(dmabuf->resv, NULL);
 	ib_umem_dmabuf_unmap_pages(umem_dmabuf);
+	if (umem_dmabuf->pinned)
+		dma_buf_unpin(umem_dmabuf->attach);
 	dma_resv_unlock(dmabuf->resv);
 
 	dma_buf_detach(dmabuf, umem_dmabuf->attach);
include/rdma/ib_umem.h +12 -0
index 676c57f5ca80..db9b73e765c3 100644
@@ -40,6 +40,7 @@ struct ib_umem_dmabuf {
 	unsigned long first_sg_offset;
 	unsigned long last_sg_trim;
 	void *private;
+	u8 pinned : 1;
 };
 
 static inline struct ib_umem_dmabuf *to_ib_umem_dmabuf(struct ib_umem *umem)
@@ -140,6 +141,10 @@ struct ib_umem_dmabuf *ib_umem_dmabuf_get(struct ib_device *device,
 					  unsigned long offset, size_t size,
 					  int fd, int access,
 					  const struct dma_buf_attach_ops *ops);
+struct ib_umem_dmabuf *ib_umem_dmabuf_get_pinned(struct ib_device *device,
+						 unsigned long offset,
+						 size_t size, int fd,
+						 int access);
 int ib_umem_dmabuf_map_pages(struct ib_umem_dmabuf *umem_dmabuf);
 void ib_umem_dmabuf_unmap_pages(struct ib_umem_dmabuf *umem_dmabuf);
 void ib_umem_dmabuf_release(struct ib_umem_dmabuf *umem_dmabuf);
@@ -180,6 +185,13 @@ struct ib_umem_dmabuf *ib_umem_dmabuf_get(struct ib_device *device,
 {
 	return ERR_PTR(-EOPNOTSUPP);
 }
+struct ib_umem_dmabuf *ib_umem_dmabuf_get_pinned(struct ib_device *device,
+						 unsigned long offset,
+						 size_t size, int fd,
+						 int access)
+{
+	return ERR_PTR(-EOPNOTSUPP);
+}
 static inline int ib_umem_dmabuf_map_pages(struct ib_umem_dmabuf *umem_dmabuf)
 {
 	return -EOPNOTSUPP;
Signature · 1 line
2.33.0
maintainer Reviewed-by on v1 1/3
#2 lore raw

Re: [PATCH for-next 1/3] dma-buf: Fix pin callback comment

In reply to Gal Pressman

Am 12.10.21 um 14:09 schrieb Gal Pressman:

Quoted 4 lines

The pin callback does not necessarily have to move the memory to system memory, remove the sentence from the comment.

Quoted 19 lines
---
  include/linux/dma-buf.h | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)
include/linux/dma-buf.h +2 -2
index efdc56b9d95f..225e09caeb98 100644
@@ -86,8 +86,8 @@ struct dma_buf_ops {
  	 * @pin:
  	 *
  	 * This is called by dma_buf_pin() and lets the exporter know that the
-	 * DMA-buf can't be moved any more. The exporter should pin the buffer
-	 * into system memory to make sure it is generally accessible by other
+	 * DMA-buf can't be moved any more. Ideally, the exporter should
+	 * pin the buffer so that it is generally accessible by all
  	 * devices.
  	 *
  	 * This is called with the &dmabuf.resv object locked and is mutual
reviewer
#5 lore raw
In reply to Gal Pressman
Quoted 24 lines · Gal Pressman

Hey all,

This is a followup to my previous RFCs [1][2], which now adds a new api to the RDMA subsystem that allows drivers to get a pinned dmabuf memory region without requiring an implementation of the move_notify callback. The new api makes use of the dynamic attachment api implemented in the RDMA subsystem, but calls dma_buf_pin() in order to make sure that the callback will not be called, as suggested by Christian.

As explained in the previous RFC, move_notify requires the RDMA device to support on-demand-paging (ODP) which is not common on most devices (only supported by mlx5).

While the dynamic requirement makes sense for certain GPUs, some devices (such as habanalabs) have device memory that is always "pinned" and do not need/use the move_notify operation.

Patch #1 changes the dmabuf documentation to make it clear that pinning does not necessarily mean the memory must be moved to system memory, it is up to the exporter to decide.
Patch #2 adds the RDMA api that allows drivers to get pinned dmabuf memory regions.
Patch #3 adds the EFA implementation of the dmabuf importer.

Quoted 8 lines

The motivation of this submission is to use habanalabs as the dmabuf exporter, and EFA as the importer to allow for peer2peer access through libibverbs.

[1] https://lore.kernel.org/linux-rdma/[email protected]/
[2] https://lore.kernel.org/linux-rdma/[email protected]/

Thanks

Hey Jason, did you get a chance to take a look?

maintainer Applied
#6 lore raw
In reply to Gal Pressman
Quoted 31 lines · Gal Pressman

Hey all,

This is a followup to my previous RFCs [1][2], which now adds a new api to the RDMA subsystem that allows drivers to get a pinned dmabuf memory region without requiring an implementation of the move_notify callback. The new api makes use of the dynamic attachment api implemented in the RDMA subsystem, but calls dma_buf_pin() in order to make sure that the callback will not be called, as suggested by Christian.

As explained in the previous RFC, move_notify requires the RDMA device to support on-demand-paging (ODP) which is not common on most devices (only supported by mlx5).

While the dynamic requirement makes sense for certain GPUs, some devices (such as habanalabs) have device memory that is always "pinned" and do not need/use the move_notify operation.

Patch #1 changes the dmabuf documentation to make it clear that pinning does not necessarily mean the memory must be moved to system memory, it is up to the exporter to decide.
Patch #2 adds the RDMA api that allows drivers to get pinned dmabuf memory regions.
Patch #3 adds the EFA implementation of the dmabuf importer.

The motivation of this submission is to use habanalabs as the dmabuf exporter, and EFA as the importer to allow for peer2peer access through libibverbs.

[1] https://lore.kernel.org/linux-rdma/[email protected]/
[2] https://lore.kernel.org/linux-rdma/[email protected]/

Quoted 6 lines

Thanks

Gal Pressman (3):
dma-buf: Fix pin callback comment
RDMA/umem: Allow pinned dmabuf umem usage
RDMA/efa: Add support for dmabuf memory regions

Applied to for-next, thanks

Jason