EFA dmabuf memory regions
cover EFA dmabuf memory regions
Hey all,
This is a followup to my previous RFC [1], which now makes use of the dynamic attachment API implemented in the RDMA subsystem, but calls dma_buf_pin() in order to make sure that the move_notify callback will not be used, as suggested by Christian.
As explained in the previous RFC, move_notify requires the RDMA device to support on-demand-paging (ODP) which is not common on most devices (only supported by mlx5).
While the dynamic requirement makes sense for certain GPUs, some devices (such as habanalabs) have device memory that is always "pinned" and do not need/use the move_notify operation.
The first patch changes the dmabuf documentation to make it clear that pinning does not necessarily mean the memory must be moved to system memory, it is up to the exporter to decide.
The motivation of this RFC is to use habanalabs as the dmabuf exporter, and EFA as the importer to allow for peer2peer access through libibverbs.
[1] https://lore.kernel.org/linux-rdma/[email protected]/
Thanks
Gal Pressman (2):
dma-buf: Fix pin callback comment
RDMA/efa: Add support for dmabuf memory regions
drivers/infiniband/hw/efa/efa.h | 4 + drivers/infiniband/hw/efa/efa_main.c | 1 + drivers/infiniband/hw/efa/efa_verbs.c | 166 +++++++++++++++++++++----- include/linux/dma-buf.h | 4 +- 4 files changed, 142 insertions(+), 33 deletions(-)
Signature · 1 line
2.33.0
RFC PATCH 1/2 dma-buf: Fix pin callback comment
The pin callback does not necessarily have to move the memory to system memory, remove the sentence from the comment.
--- include/linux/dma-buf.h | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-)
include/linux/dma-buf.h +1 -3
@@ -86,9 +86,7 @@ struct dma_buf_ops {* @pin: * * This is called by dma_buf_pin() and lets the exporter know that the -* DMA-buf can't be moved any more. The exporter should pin the buffer -* into system memory to make sure it is generally accessible by other -* devices. +* DMA-buf can't be moved any more. * * This is called with the &dmabuf.resv object locked and is mutual * exclusive with @cache_sgt_mapping.
Signature · 1 line
2.33.0
Christian König
maintainerRe: [RFC PATCH 1/2] dma-buf: Fix pin callback comment
Am 07.10.21 um 12:42 schrieb Gal Pressman:
Quoted 13 lines
The pin callback does not necessarily have to move the memory to system memory, remove the sentence from the comment.
Signed-off-by Gal Pressman <[email protected]>--- include/linux/dma-buf.h | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-)include/linux/dma-buf.h +0
-0
include/linux/dma-buf.h struct dma_buf_ops { · line 91
Quoted 6 lines
include/linux/dma-buf.h +0
-3@@ -86,9 +86,7 @@ struct dma_buf_ops {* @pin: * * This is called by dma_buf_pin() and lets the exporter know that the -* DMA-buf can't be moved any more. The exporter should pin the buffer -* into system memory to make sure it is generally accessible by other -* devices.
Maybe change that to something like "Ideally the exporter should pin the buffer so that it is generally accessible by all devices".
Christian.
Quoted 4 lines
include/linux/dma-buf.h +1
-0+* DMA-buf can't be moved any more. * * This is called with the &dmabuf.resv object locked and is mutual * exclusive with @cache_sgt_mapping.
Re: [RFC PATCH 1/2] dma-buf: Fix pin callback comment
Quoted 9 lines · Christian König
Am 07.10.21 um 12:42 schrieb Gal Pressman:
Quoted 8 lines
The pin callback does not necessarily have to move the memory to system memory, remove the sentence from the comment.
Signed-off-by Gal Pressman <[email protected]>---
include/linux/dma-buf.h| 4 +---1 file changed, 1 insertion(+), 3 deletions(-)
Quoted 14 lines
include/linux/dma-buf.h struct dma_buf_ops { · line 91
Quoted 11 lines
include/linux/dma-buf.h +0
-3@@ -86,9 +86,7 @@ struct dma_buf_ops { * @pin: * * This is called by dma_buf_pin() and lets the exporter know that the - * DMA-buf can't be moved any more. The exporter should pin the buffer - * into system memory to make sure it is generally accessible by other - * devices.Maybe change that to something like "Ideally the exporter should pin the buffer so that it is generally accessible by all devices".
Sure, thanks.
RFC PATCH 2/2 RDMA/efa: Add support for dmabuf memory regions
In reply to Gal PressmanImplement a dmabuf importer for the EFA driver. As ODP is not supported, the dmabuf memory regions always pin the buffers to prevent the move_notify callback from being called.
--- drivers/infiniband/hw/efa/efa.h | 4 + drivers/infiniband/hw/efa/efa_main.c | 1 + drivers/infiniband/hw/efa/efa_verbs.c | 166 +++++++++++++++++++++----- 3 files changed, 141 insertions(+), 30 deletions(-)
drivers/infiniband/hw/efa/efa.h +4 -0
@@ -141,6 +141,10 @@ int efa_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,struct ib_mr *efa_reg_mr (struct ib_pd *ibpd,u64 start,u64 length,u64 virt_addr,int access_flags,struct ib_udata *udata); +struct ib_mr *efa_reg_user_mr_dmabuf (struct ib_pd *ibpd,u64 start, +u64 length,u64 virt_addr, +int fd,int access_flags, +struct ib_udata *udata);int efa_dereg_mr (struct ib_mr *ibmr,struct ib_udata *udata);int efa_get_port_immutable (struct ib_device *ibdev,u32 port_num,struct ib_port_immutable *immutable);
drivers/infiniband/hw/efa/efa_main.c +1 -0
@@ -267,6 +267,7 @@ static const struct ib_device_ops efa_dev_ops = { .query_port = efa_query_port, .query_qp = efa_query_qp, .reg_user_mr = efa_reg_mr, + .reg_user_mr_dmabuf = efa_reg_user_mr_dmabuf,INIT_RDMA_OBJ_SIZE (ib_ah, efa_ah, ibah), INIT_RDMA_OBJ_SIZE(ib_cq, efa_cq, ibcq),
drivers/infiniband/hw/efa/efa_verbs.c +136 -30
243 lines not on this page · raw message
Signature · 1 line
2.33.0
Re: [RFC PATCH 2/2] RDMA/efa: Add support for dmabuf memory regions
Quoted 7 lines · Gal Pressman
drivers/infiniband/hw/efa/efa_verbs.c +0
-3@@ -1491,26 +1493,29 @@ static int efa_create_pbl(struct efa_dev *dev,return 0 ; } -struct ib_mr *efa_reg_mr (struct ib_pd *ibpd,u64 start,u64 length, -u64 virt_addr,int access_flags, -struct ib_udata *udata)
drivers/infiniband/hw/efa/efa_verbs.c static int efa_create_pbl(struct efa_dev *dev, · line 1505
Quoted 10 lines
drivers/infiniband/hw/efa/efa_verbs.c +10
-0+static void efa_dmabuf_invalidate_cb (struct dma_buf_attachment *attach) +{ +WARN_ON_ONCE (1 , +"Invalidate callback should not be called when memory is pinned\n" ); +} + +static struct dma_buf_attach_ops efa_dmabuf_attach_ops = { + .allow_peer2peer = true, + .move_notify = efa_dmabuf_invalidate_cb, +};
Shouldn't move_notify really just be left as NULL? I mean fixing whatever is preventing that?
Quoted 27 lines
drivers/infiniband/hw/efa/efa_verbs.c +27
-0+struct ib_mr *efa_reg_user_mr_dmabuf (struct ib_pd *ibpd,u64 start, +u64 length,u64 virt_addr, +int fd,int access_flags, +struct ib_udata *udata) +{ +struct efa_dev *dev =to_edev (ibpd->device); +struct ib_umem_dmabuf *umem_dmabuf; +struct efa_mr *mr; +int err; + + mr =efa_alloc_mr (ibpd, access_flags, udata); +if (IS_ERR (mr)) { + err =PTR_ERR (mr); + goto err_out; + } + + umem_dmabuf =ib_umem_dmabuf_get (ibpd->device, start, length, fd, + access_flags, &efa_dmabuf_attach_ops); +if (IS_ERR (umem_dmabuf)) { +ibdev_dbg (&dev->ibdev,"Failed to get dmabuf[%d]\n" , err); + err =PTR_ERR (umem_dmabuf); + goto err_free; + } + +dma_resv_lock (umem_dmabuf->attach->dmabuf->resv,NULL ); + err =dma_buf_pin (umem_dmabuf->attach); +if (err) {
drivers/infiniband/hw/efa/efa_verbs.c struct ib_mr *efa_reg_mr(struct ib_pd *ibpd, u64 start, u64 length, · line 1637
Quoted 10 lines
drivers/infiniband/hw/efa/efa_verbs.c +10
-0+ibdev_dbg (&dev->ibdev,"Failed to pin dmabuf memory\n" ); + goto err_release; + } + + err =ib_umem_dmabuf_map_pages (umem_dmabuf); +if (err) { +ibdev_dbg (&dev->ibdev,"Failed to map dmabuf pages\n" ); + goto err_unpin; + } +dma_resv_unlock (umem_dmabuf->attach->dmabuf->resv);
If it is really this simple the core code should have this logic, 'ib_umem_dmabuf_get_pinned()' or something
Jason
Re: [RFC PATCH 2/2] RDMA/efa: Add support for dmabuf memory regions
Quoted 6 lines · Jason Gunthorpe
Quoted 4 lines · Gal Pressman
drivers/infiniband/hw/efa/efa_verbs.c +0
-0@@ -1491,26 +1493,29 @@ static int efa_create_pbl(struct efa_dev *dev,return 0 ; }
Quoted 16 lines
drivers/infiniband/hw/efa/efa_verbs.c static int efa_create_pbl(struct efa_dev *dev, · line 1505
Quoted 13 lines
drivers/infiniband/hw/efa/efa_verbs.c +10
-3-struct ib_mr *efa_reg_mr (struct ib_pd *ibpd,u64 start,u64 length, -u64 virt_addr,int access_flags, -struct ib_udata *udata) +static void efa_dmabuf_invalidate_cb (struct dma_buf_attachment *attach) +{ +WARN_ON_ONCE (1 , +"Invalidate callback should not be called when memory is pinned\n" ); +} + +static struct dma_buf_attach_ops efa_dmabuf_attach_ops = { + .allow_peer2peer = true, + .move_notify = efa_dmabuf_invalidate_cb, +};Shouldn't
move_notifyreally just be left as NULL? I mean fixing whatever is preventing that?
That's what I had in the previous RFC and I think Christian didn't really like it.
Quoted 32 lines
Quoted 32 lines
drivers/infiniband/hw/efa/efa_verbs.c +32
-0+struct ib_mr *efa_reg_user_mr_dmabuf (struct ib_pd *ibpd,u64 start, +u64 length,u64 virt_addr, +int fd,int access_flags, +struct ib_udata *udata) +{ +struct efa_dev *dev =to_edev (ibpd->device); +struct ib_umem_dmabuf *umem_dmabuf; +struct efa_mr *mr; +int err; + + mr =efa_alloc_mr (ibpd, access_flags, udata); +if (IS_ERR (mr)) { + err =PTR_ERR (mr); + goto err_out; + } + + umem_dmabuf =ib_umem_dmabuf_get (ibpd->device, start, length, fd, + access_flags, &efa_dmabuf_attach_ops); +if (IS_ERR (umem_dmabuf)) { +ibdev_dbg (&dev->ibdev,"Failed to get dmabuf[%d]\n" , err); + err =PTR_ERR (umem_dmabuf); + goto err_free; + } + +dma_resv_lock (umem_dmabuf->attach->dmabuf->resv,NULL ); + err =dma_buf_pin (umem_dmabuf->attach); +if (err) { +ibdev_dbg (&dev->ibdev,"Failed to pin dmabuf memory\n" ); + goto err_release; + } + + err =ib_umem_dmabuf_map_pages (umem_dmabuf);
Quoted 8 lines
drivers/infiniband/hw/efa/efa_verbs.c struct ib_mr *efa_reg_mr(struct ib_pd *ibpd, u64 start, u64 length, · line 1637
Quoted 5 lines
drivers/infiniband/hw/efa/efa_verbs.c +5
-0+if (err) { +ibdev_dbg (&dev->ibdev,"Failed to map dmabuf pages\n" ); + goto err_unpin; + } +dma_resv_unlock (umem_dmabuf->attach->dmabuf->resv);If it is really this simple the core code should have this logic, '
ib_umem_dmabuf_get_pinned()' or something
Should get_pinned do just get + dma_buf_pin, or should it do ib_umem_dmabuf_map_pages as well?
Re: [RFC PATCH 2/2] RDMA/efa: Add support for dmabuf memory regions
Quoted 24 lines · Gal Pressman
Quoted 6 lines · Jason Gunthorpe
Quoted 4 lines · Gal Pressman
drivers/infiniband/hw/efa/efa_verbs.c +0
-0@@ -1491,26 +1493,29 @@ static int efa_create_pbl(struct efa_dev *dev,return 0 ; }Quoted 16 lines
drivers/infiniband/hw/efa/efa_verbs.c static int efa_create_pbl(struct efa_dev *dev, · line 1505
Quoted 13 lines
drivers/infiniband/hw/efa/efa_verbs.c +10
-3-struct ib_mr *efa_reg_mr (struct ib_pd *ibpd,u64 start,u64 length, -u64 virt_addr,int access_flags, -struct ib_udata *udata) +static void efa_dmabuf_invalidate_cb (struct dma_buf_attachment *attach) +{ +WARN_ON_ONCE (1 , +"Invalidate callback should not be called when memory is pinned\n" ); +} + +static struct dma_buf_attach_ops efa_dmabuf_attach_ops = { + .allow_peer2peer = true, + .move_notify = efa_dmabuf_invalidate_cb, +};Shouldn't
move_notifyreally just be left as NULL? I mean fixing whatever is preventing that?
Quoted 1 line
That's what I had in the previous RFC and I think Christian didn't really like it.
Well, having drivers define a dummy function that only fails looks a lot worse to me. If not null then it should be a general 'dmabuf_unsupported_move_notify' shared function
Quoted 10 lines
Quoted 9 lines
drivers/infiniband/hw/efa/efa_verbs.c struct ib_mr *efa_reg_mr(struct ib_pd *ibpd, u64 start, u64 length, · line 1637
Quoted 6 lines
drivers/infiniband/hw/efa/efa_verbs.c +6
-0+ err =ib_umem_dmabuf_map_pages (umem_dmabuf); +if (err) { +ibdev_dbg (&dev->ibdev,"Failed to map dmabuf pages\n" ); + goto err_unpin; + } +dma_resv_unlock (umem_dmabuf->attach->dmabuf->resv);If it is really this simple the core code should have this logic, '
ib_umem_dmabuf_get_pinned()' or something
Quoted 2 lines
Should
get_pinneddo just get +dma_buf_pin, or should it doib_umem_dmabuf_map_pagesas well?
Yes the map_pages too, a umem is supposed to be dma mapped after creation.
Jason
Re: [RFC PATCH 2/2] RDMA/efa: Add support for dmabuf memory regions
Quoted 25 lines · Jason Gunthorpe
Quoted 24 lines · Gal Pressman
Quoted 22 lines · Jason Gunthorpe
Quoted 7 lines · Gal Pressman
drivers/infiniband/hw/efa/efa_verbs.c +0
-3@@ -1491,26 +1493,29 @@ static int efa_create_pbl(struct efa_dev *dev,return 0 ; } -struct ib_mr *efa_reg_mr (struct ib_pd *ibpd,u64 start,u64 length, -u64 virt_addr,int access_flags, -struct ib_udata *udata)drivers/infiniband/hw/efa/efa_verbs.c static int efa_create_pbl(struct efa_dev *dev, · line 1505
Quoted 10 lines
drivers/infiniband/hw/efa/efa_verbs.c +10
-0+static void efa_dmabuf_invalidate_cb (struct dma_buf_attachment *attach) +{ +WARN_ON_ONCE (1 , +"Invalidate callback should not be called when memory is pinned\n" ); +} + +static struct dma_buf_attach_ops efa_dmabuf_attach_ops = { + .allow_peer2peer = true, + .move_notify = efa_dmabuf_invalidate_cb, +};Shouldn't
move_notifyreally just be left as NULL? I mean fixing whatever is preventing that?
Quoted 5 lines
Quoted 1 line
That's what I had in the previous RFC and I think Christian didn't really like it.
Well, having drivers define a dummy function that only fails looks a lot worse to me. If not null then it should be a general '
dmabuf_unsupported_move_notify' shared function
Will do.
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
drivers/infiniband/hw/efa/efa_verbs.c +6
-0+ err =ib_umem_dmabuf_map_pages (umem_dmabuf); +if (err) { +ibdev_dbg (&dev->ibdev,"Failed to map dmabuf pages\n" ); + goto err_unpin; + } +dma_resv_unlock (umem_dmabuf->attach->dmabuf->resv);
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
If it is really this simple the core code should have this logic, '
ib_umem_dmabuf_get_pinned()' or somethingShould
get_pinneddo just get +dma_buf_pin, or should it doib_umem_dmabuf_map_pagesas well?Yes the
map_pagestoo, a umem is supposed to be dma mapped after creation.
Will do, thanks Jason.