Skip to contentSkip to conversation

EFA dmabuf memory regions

RFC PATCH In review Gal Pressman 3 participants 9 messages Oct 7, 2021 – Oct 12, 2021

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

cover EFA dmabuf memory regions

Hey all,

This is a followup to my previous RFC [1], which now makes use of the dynamic attachment API implemented in the RDMA subsystem, but calls dma_buf_pin() in order to make sure that the move_notify callback will not be used, as suggested by Christian.
As explained in the previous RFC, move_notify requires the RDMA device to support on-demand-paging (ODP) which is not common on most devices (only supported by mlx5).

While the dynamic requirement makes sense for certain GPUs, some devices (such as habanalabs) have device memory that is always "pinned" and do not need/use the move_notify operation.

The first patch changes the dmabuf documentation to make it clear that pinning does not necessarily mean the memory must be moved to system memory, it is up to the exporter to decide.

The motivation of this RFC is to use habanalabs as the dmabuf exporter, and EFA as the importer to allow for peer2peer access through libibverbs.

[1] https://lore.kernel.org/linux-rdma/[email protected]/

Thanks

Gal Pressman (2):
dma-buf: Fix pin callback comment
RDMA/efa: Add support for dmabuf memory regions

 drivers/infiniband/hw/efa/efa.h       |   4 +
 drivers/infiniband/hw/efa/efa_main.c  |   1 +
 drivers/infiniband/hw/efa/efa_verbs.c | 166 +++++++++++++++++++++-----
 include/linux/dma-buf.h               |   4 +-
 4 files changed, 142 insertions(+), 33 deletions(-)
Signature · 1 line
2.33.0
#1 lore raw

RFC PATCH 1/2 dma-buf: Fix pin callback comment

The pin callback does not necessarily have to move the memory to system memory, remove the sentence from the comment.

---
 include/linux/dma-buf.h | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)
include/linux/dma-buf.h +1 -3
index efdc56b9d95f..93830731a9a3 100644
@@ -86,9 +86,7 @@ struct dma_buf_ops {
 	 * @pin:
 	 *
 	 * This is called by dma_buf_pin() and lets the exporter know that the
-	 * DMA-buf can't be moved any more. The exporter should pin the buffer
-	 * into system memory to make sure it is generally accessible by other
-	 * devices.
+	 * DMA-buf can't be moved any more.
 	 *
 	 * This is called with the &dmabuf.resv object locked and is mutual
 	 * exclusive with @cache_sgt_mapping.
Signature · 1 line
2.33.0
maintainer
#2 lore raw

Re: [RFC PATCH 1/2] dma-buf: Fix pin callback comment

Am 07.10.21 um 12:42 schrieb Gal Pressman:

Quoted 13 lines

The pin callback does not necessarily have to move the memory to system memory, remove the sentence from the comment.

---
  include/linux/dma-buf.h | 4 +---
  1 file changed, 1 insertion(+), 3 deletions(-)
include/linux/dma-buf.h +0 -0
index efdc56b9d95f..93830731a9a3 100644

include/linux/dma-buf.h struct dma_buf_ops { · line 91

Quoted 6 lines
include/linux/dma-buf.h +0 -3
@@ -86,9 +86,7 @@ struct dma_buf_ops {
  	 * @pin:
  	 *
  	 * This is called by dma_buf_pin() and lets the exporter know that the
-	 * DMA-buf can't be moved any more. The exporter should pin the buffer
-	 * into system memory to make sure it is generally accessible by other
-	 * devices.

Maybe change that to something like "Ideally the exporter should pin the buffer so that it is generally accessible by all devices".

Christian.

Quoted 4 lines
include/linux/dma-buf.h +1 -0
+	 * DMA-buf can't be moved any more.
  	 *
  	 * This is called with the &dmabuf.resv object locked and is mutual
  	 * exclusive with @cache_sgt_mapping.
#3 lore raw

Re: [RFC PATCH 1/2] dma-buf: Fix pin callback comment

Quoted 9 lines · Christian König

Am 07.10.21 um 12:42 schrieb Gal Pressman:

Quoted 8 lines

The pin callback does not necessarily have to move the memory to system memory, remove the sentence from the comment.

---
  include/linux/dma-buf.h | 4 +---

  1 file changed, 1 insertion(+), 3 deletions(-)
Quoted 14 lines

include/linux/dma-buf.h struct dma_buf_ops { · line 91

Quoted 11 lines
include/linux/dma-buf.h +0 -3
index efdc56b9d95f..93830731a9a3 100644
@@ -86,9 +86,7 @@ struct dma_buf_ops {
        * @pin:
        *
        * This is called by dma_buf_pin() and lets the exporter know that the
-     * DMA-buf can't be moved any more. The exporter should pin the buffer
-     * into system memory to make sure it is generally accessible by other
-     * devices.

Maybe change that to something like "Ideally the exporter should pin the buffer so that it is generally accessible by all devices".

Sure, thanks.

#4 lore raw

RFC PATCH 2/2 RDMA/efa: Add support for dmabuf memory regions

In reply to Gal Pressman

Implement a dmabuf importer for the EFA driver. As ODP is not supported, the dmabuf memory regions always pin the buffers to prevent the move_notify callback from being called.

---
 drivers/infiniband/hw/efa/efa.h       |   4 +
 drivers/infiniband/hw/efa/efa_main.c  |   1 +
 drivers/infiniband/hw/efa/efa_verbs.c | 166 +++++++++++++++++++++-----
 3 files changed, 141 insertions(+), 30 deletions(-)
drivers/infiniband/hw/efa/efa.h +4 -0
index 2b8ca099b381..407d7c4baa16 100644
@@ -141,6 +141,10 @@ int efa_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
 struct ib_mr *efa_reg_mr(struct ib_pd *ibpd, u64 start, u64 length,
 			 u64 virt_addr, int access_flags,
 			 struct ib_udata *udata);
+struct ib_mr *efa_reg_user_mr_dmabuf(struct ib_pd *ibpd, u64 start,
+				     u64 length, u64 virt_addr,
+				     int fd, int access_flags,
+				     struct ib_udata *udata);
 int efa_dereg_mr(struct ib_mr *ibmr, struct ib_udata *udata);
 int efa_get_port_immutable(struct ib_device *ibdev, u32 port_num,
 			   struct ib_port_immutable *immutable);
drivers/infiniband/hw/efa/efa_main.c +1 -0
index 203e6ddcacbc..72cd7d952a07 100644
@@ -267,6 +267,7 @@ static const struct ib_device_ops efa_dev_ops = {
 	.query_port = efa_query_port,
 	.query_qp = efa_query_qp,
 	.reg_user_mr = efa_reg_mr,
+	.reg_user_mr_dmabuf = efa_reg_user_mr_dmabuf,
 
 	INIT_RDMA_OBJ_SIZE(ib_ah, efa_ah, ibah),
 	INIT_RDMA_OBJ_SIZE(ib_cq, efa_cq, ibcq),
drivers/infiniband/hw/efa/efa_verbs.c +136 -30

243 lines not on this page · raw message

Signature · 1 line
2.33.0
#5 lore raw

Re: [RFC PATCH 2/2] RDMA/efa: Add support for dmabuf memory regions

Quoted 7 lines · Gal Pressman
drivers/infiniband/hw/efa/efa_verbs.c +0 -3
@@ -1491,26 +1493,29 @@ static int efa_create_pbl(struct efa_dev *dev,
 	return 0;
 }
 
-struct ib_mr *efa_reg_mr(struct ib_pd *ibpd, u64 start, u64 length,
-			 u64 virt_addr, int access_flags,
-			 struct ib_udata *udata)

drivers/infiniband/hw/efa/efa_verbs.c static int efa_create_pbl(struct efa_dev *dev, · line 1505

Quoted 10 lines
drivers/infiniband/hw/efa/efa_verbs.c +10 -0
+static void efa_dmabuf_invalidate_cb(struct dma_buf_attachment *attach)
+{
+	WARN_ON_ONCE(1,
+		     "Invalidate callback should not be called when memory is pinned\n");
+}
+
+static struct dma_buf_attach_ops efa_dmabuf_attach_ops = {
+	.allow_peer2peer = true,
+	.move_notify = efa_dmabuf_invalidate_cb,
+};

Shouldn't move_notify really just be left as NULL? I mean fixing whatever is preventing that?

Quoted 27 lines
drivers/infiniband/hw/efa/efa_verbs.c +27 -0
+struct ib_mr *efa_reg_user_mr_dmabuf(struct ib_pd *ibpd, u64 start,
+				     u64 length, u64 virt_addr,
+				     int fd, int access_flags,
+				     struct ib_udata *udata)
+{
+	struct efa_dev *dev = to_edev(ibpd->device);
+	struct ib_umem_dmabuf *umem_dmabuf;
+	struct efa_mr *mr;
+	int err;
+
+	mr = efa_alloc_mr(ibpd, access_flags, udata);
+	if (IS_ERR(mr)) {
+		err = PTR_ERR(mr);
+		goto err_out;
+	}
+
+	umem_dmabuf = ib_umem_dmabuf_get(ibpd->device, start, length, fd,
+					 access_flags, &efa_dmabuf_attach_ops);
+	if (IS_ERR(umem_dmabuf)) {
+		ibdev_dbg(&dev->ibdev, "Failed to get dmabuf[%d]\n", err);
+		err = PTR_ERR(umem_dmabuf);
+		goto err_free;
+	}
+
+	dma_resv_lock(umem_dmabuf->attach->dmabuf->resv, NULL);
+	err = dma_buf_pin(umem_dmabuf->attach);
+	if (err) {

drivers/infiniband/hw/efa/efa_verbs.c struct ib_mr *efa_reg_mr(struct ib_pd *ibpd, u64 start, u64 length, · line 1637

Quoted 10 lines
drivers/infiniband/hw/efa/efa_verbs.c +10 -0
+		ibdev_dbg(&dev->ibdev, "Failed to pin dmabuf memory\n");
+		goto err_release;
+	}
+
+	err = ib_umem_dmabuf_map_pages(umem_dmabuf);
+	if (err) {
+		ibdev_dbg(&dev->ibdev, "Failed to map dmabuf pages\n");
+		goto err_unpin;
+	}
+	dma_resv_unlock(umem_dmabuf->attach->dmabuf->resv);

If it is really this simple the core code should have this logic, 'ib_umem_dmabuf_get_pinned()' or something

Jason

#6 lore raw

Re: [RFC PATCH 2/2] RDMA/efa: Add support for dmabuf memory regions

Quoted 6 lines · Jason Gunthorpe
Quoted 4 lines · Gal Pressman
drivers/infiniband/hw/efa/efa_verbs.c +0 -0
@@ -1491,26 +1493,29 @@ static int efa_create_pbl(struct efa_dev *dev,
 	return 0;
 }
 
Quoted 16 lines

drivers/infiniband/hw/efa/efa_verbs.c static int efa_create_pbl(struct efa_dev *dev, · line 1505

Quoted 13 lines
drivers/infiniband/hw/efa/efa_verbs.c +10 -3
-struct ib_mr *efa_reg_mr(struct ib_pd *ibpd, u64 start, u64 length,
-			 u64 virt_addr, int access_flags,
-			 struct ib_udata *udata)
+static void efa_dmabuf_invalidate_cb(struct dma_buf_attachment *attach)
+{
+	WARN_ON_ONCE(1,
+		     "Invalidate callback should not be called when memory is pinned\n");
+}
+
+static struct dma_buf_attach_ops efa_dmabuf_attach_ops = {
+	.allow_peer2peer = true,
+	.move_notify = efa_dmabuf_invalidate_cb,
+};

Shouldn't move_notify really just be left as NULL? I mean fixing whatever is preventing that?

That's what I had in the previous RFC and I think Christian didn't really like it.

Quoted 32 lines
Quoted 32 lines
drivers/infiniband/hw/efa/efa_verbs.c +32 -0
+struct ib_mr *efa_reg_user_mr_dmabuf(struct ib_pd *ibpd, u64 start,
+				     u64 length, u64 virt_addr,
+				     int fd, int access_flags,
+				     struct ib_udata *udata)
+{
+	struct efa_dev *dev = to_edev(ibpd->device);
+	struct ib_umem_dmabuf *umem_dmabuf;
+	struct efa_mr *mr;
+	int err;
+
+	mr = efa_alloc_mr(ibpd, access_flags, udata);
+	if (IS_ERR(mr)) {
+		err = PTR_ERR(mr);
+		goto err_out;
+	}
+
+	umem_dmabuf = ib_umem_dmabuf_get(ibpd->device, start, length, fd,
+					 access_flags, &efa_dmabuf_attach_ops);
+	if (IS_ERR(umem_dmabuf)) {
+		ibdev_dbg(&dev->ibdev, "Failed to get dmabuf[%d]\n", err);
+		err = PTR_ERR(umem_dmabuf);
+		goto err_free;
+	}
+
+	dma_resv_lock(umem_dmabuf->attach->dmabuf->resv, NULL);
+	err = dma_buf_pin(umem_dmabuf->attach);
+	if (err) {
+		ibdev_dbg(&dev->ibdev, "Failed to pin dmabuf memory\n");
+		goto err_release;
+	}
+
+	err = ib_umem_dmabuf_map_pages(umem_dmabuf);
Quoted 8 lines

drivers/infiniband/hw/efa/efa_verbs.c struct ib_mr *efa_reg_mr(struct ib_pd *ibpd, u64 start, u64 length, · line 1637

Quoted 5 lines
drivers/infiniband/hw/efa/efa_verbs.c +5 -0
+	if (err) {
+		ibdev_dbg(&dev->ibdev, "Failed to map dmabuf pages\n");
+		goto err_unpin;
+	}
+	dma_resv_unlock(umem_dmabuf->attach->dmabuf->resv);

If it is really this simple the core code should have this logic, 'ib_umem_dmabuf_get_pinned()' or something

Should get_pinned do just get + dma_buf_pin, or should it do ib_umem_dmabuf_map_pages as well?

#7 lore raw

Re: [RFC PATCH 2/2] RDMA/efa: Add support for dmabuf memory regions

Quoted 24 lines · Gal Pressman
Quoted 6 lines · Jason Gunthorpe
Quoted 4 lines · Gal Pressman
drivers/infiniband/hw/efa/efa_verbs.c +0 -0
@@ -1491,26 +1493,29 @@ static int efa_create_pbl(struct efa_dev *dev,
 	return 0;
 }
 
Quoted 16 lines

drivers/infiniband/hw/efa/efa_verbs.c static int efa_create_pbl(struct efa_dev *dev, · line 1505

Quoted 13 lines
drivers/infiniband/hw/efa/efa_verbs.c +10 -3
-struct ib_mr *efa_reg_mr(struct ib_pd *ibpd, u64 start, u64 length,
-			 u64 virt_addr, int access_flags,
-			 struct ib_udata *udata)
+static void efa_dmabuf_invalidate_cb(struct dma_buf_attachment *attach)
+{
+	WARN_ON_ONCE(1,
+		     "Invalidate callback should not be called when memory is pinned\n");
+}
+
+static struct dma_buf_attach_ops efa_dmabuf_attach_ops = {
+	.allow_peer2peer = true,
+	.move_notify = efa_dmabuf_invalidate_cb,
+};

Shouldn't move_notify really just be left as NULL? I mean fixing whatever is preventing that?

Quoted 1 line

That's what I had in the previous RFC and I think Christian didn't really like it.

Well, having drivers define a dummy function that only fails looks a lot worse to me. If not null then it should be a general 'dmabuf_unsupported_move_notify' shared function

Quoted 10 lines
Quoted 9 lines

drivers/infiniband/hw/efa/efa_verbs.c struct ib_mr *efa_reg_mr(struct ib_pd *ibpd, u64 start, u64 length, · line 1637

Quoted 6 lines
drivers/infiniband/hw/efa/efa_verbs.c +6 -0
+	err = ib_umem_dmabuf_map_pages(umem_dmabuf);
+	if (err) {
+		ibdev_dbg(&dev->ibdev, "Failed to map dmabuf pages\n");
+		goto err_unpin;
+	}
+	dma_resv_unlock(umem_dmabuf->attach->dmabuf->resv);

If it is really this simple the core code should have this logic, 'ib_umem_dmabuf_get_pinned()' or something

Quoted 2 lines

Should get_pinned do just get + dma_buf_pin, or should it do ib_umem_dmabuf_map_pages as well?

Yes the map_pages too, a umem is supposed to be dma mapped after creation.

Jason

#8 lore raw

Re: [RFC PATCH 2/2] RDMA/efa: Add support for dmabuf memory regions

Quoted 25 lines · Jason Gunthorpe
Quoted 24 lines · Gal Pressman
Quoted 22 lines · Jason Gunthorpe
Quoted 7 lines · Gal Pressman
drivers/infiniband/hw/efa/efa_verbs.c +0 -3
@@ -1491,26 +1493,29 @@ static int efa_create_pbl(struct efa_dev *dev,
   return 0;
 }
 
-struct ib_mr *efa_reg_mr(struct ib_pd *ibpd, u64 start, u64 length,
-                   u64 virt_addr, int access_flags,
-                   struct ib_udata *udata)

drivers/infiniband/hw/efa/efa_verbs.c static int efa_create_pbl(struct efa_dev *dev, · line 1505

Quoted 10 lines
drivers/infiniband/hw/efa/efa_verbs.c +10 -0
+static void efa_dmabuf_invalidate_cb(struct dma_buf_attachment *attach)
+{
+  WARN_ON_ONCE(1,
+               "Invalidate callback should not be called when memory is pinned\n");
+}
+
+static struct dma_buf_attach_ops efa_dmabuf_attach_ops = {
+  .allow_peer2peer = true,
+  .move_notify = efa_dmabuf_invalidate_cb,
+};

Shouldn't move_notify really just be left as NULL? I mean fixing whatever is preventing that?

Quoted 5 lines
Quoted 1 line

That's what I had in the previous RFC and I think Christian didn't really like it.

Well, having drivers define a dummy function that only fails looks a lot worse to me. If not null then it should be a general 'dmabuf_unsupported_move_notify' shared function

Will do.

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
drivers/infiniband/hw/efa/efa_verbs.c +6 -0
+  err = ib_umem_dmabuf_map_pages(umem_dmabuf);
+  if (err) {
+          ibdev_dbg(&dev->ibdev, "Failed to map dmabuf pages\n");
+          goto err_unpin;
+  }
+  dma_resv_unlock(umem_dmabuf->attach->dmabuf->resv);
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

If it is really this simple the core code should have this logic, 'ib_umem_dmabuf_get_pinned()' or something

Should get_pinned do just get + dma_buf_pin, or should it do ib_umem_dmabuf_map_pages as well?

Yes the map_pages too, a umem is supposed to be dma mapped after creation.

Will do, thanks Jason.