DMA direction and mlx5 driver correctness fixes
Subsystems (10)
Yishai Hadas
maintainercover DMA direction and mlx5 driver correctness fixes
This series contains two independent groups of fixes for the RDMA subsystem.
Group 1: DMA direction correctness (patches 1-7).
RDMA drivers have historically mapped all pinned user buffers DMA_BIDIRECTIONAL regardless of how the device actually accesses them. This group tightens the DMA mapping direction to match the real access pattern for each buffer type:
- CQ ring buffers: the device writes CQEs, the CPU only reads. Mapped DMA_FROM_DEVICE.
- WQE rings, SRQ rings, doorbell records: the CPU writes, the device only reads. Mapped DMA_TO_DEVICE (derived from the absence of any write access flag).
- Writable MRs and other buffers with write access flags: unchanged, DMA_BIDIRECTIONAL.
On platforms with a write-enforcing IOMMU this prevents the device from accessing memory beyond what the mapping semantically declares. On standard deployments without direction enforcement the change declares the correct semantic at zero runtime cost.
The group includes preparatory driver fixes (patches 1-3) that correct existing access=0 pins on writable buffers in erdma, hns, and vmw_pvrdma, and a refactoring patch (4) that consolidates all CQ buffer pinning through ib_umem_get_cq_buf_or_va() so the directional change in patch 6 covers every CQ call site automatically. Full details are in the individual commit logs.
Group 2: mlx5 driver correctness fixes (patches 8-15).
Patch 8 fixes driver probe failure on XRC-less devices. mlx5_ib_dev_res_init() was returning -EOPNOTSUPP when the firmware XRC capability is absent. XRC is optional; the patch makes the devr resource path XRC-optional throughout.
Patches 9-14 fix a cluster of related bugs in the mlx5 firmware event dispatch path. The common root cause is that resources were made visible to the event notifier (via radix tree or xarray insertion) before their event callback, container pointer, and refcount were fully initialized, leaving windows where firmware EQEs could call through NULL function pointers or corrupt refcounts:
- mlx5_ib_wq_event() never released the reference taken before dispatch, leaking it on every WQ event.
- WQ, SRQ, QP, and raw-packet QP SQ/RQ event callbacks were assigned after the resource became visible to the notifier. For SRQ and QP paths this directly causes a kernel crash on any matching firmware event.
- create_resource_common() initialized the refcount and completion after the radix tree insert, allowing refcount_inc() on a still-zero refcount_t, which saturates the counter and permanently corrupts the resource's lifecycle tracking.
Patch 15 fixes undefined behavior in the EQE type field shift: the u8 type is left-shifted by 24 without a cast, promoting it to signed int and causing UB for values >= 128. All currently used type codes are below 128, but a cast to u32 makes the shift well-defined.
Changes from V0:
- Clean up some AI-generated commit messages and comments.
- Patch #7: Add a simple wrapper for a repeated pattern.
- Link to V0: https://lore.kernel.org/linux-rdma/[email protected]/T/#t
Yishai
Yishai Hadas (15):
RDMA/erdma: Pin CQ buffer writable to match device DMA write access RDMA/hns: Pin CQ buffer writable to match device DMA write access RDMA/vmw_pvrdma: Pin QP and SRQ rings writable to match device DMA
write access
RDMA/umem: Reuse ib_umem_get_cq_buf_or_va() for VA-only CQ pinning RDMA/umem: Support an explicit DMA direction other than
DMA_BIDIRECTIONAL
RDMA/umem: Map CQ buffers DMA_FROM_DEVICE
RDMA/umem: Derive DMA direction from IB access flags RDMA/mlx5: Fix mlx5_ib_dev_res_init() failure when XRC cap is absent RDMA/mlx5: Put resource reference in mlx5_ib_wq_event() RDMA/mlx5: Set WQ event handler before firmware RQ insertion RDMA/mlx5: Set SRQ event handler before xarray insertion RDMA/mlx5: Set RQ event handler for raw-packet QP RDMA/mlx5: Set QP event handler before firmware QPC insertion RDMA/mlx5: Initialize QP/RQ/SQ resource refcount before publishing it RDMA/mlx5: Fix signed integer overflow in EQE qp_srq type shift
drivers/infiniband/core/umem.c | 77 +++++++++++++------ drivers/infiniband/hw/bnxt_re/ib_verbs.c | 9 ++- drivers/infiniband/hw/erdma/erdma_verbs.c | 19 +++-- drivers/infiniband/hw/hns/hns_roce_cq.c | 3 +- drivers/infiniband/hw/hns/hns_roce_device.h | 2 +- drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 2 +- drivers/infiniband/hw/hns/hns_roce_mr.c | 22 ++++-- drivers/infiniband/hw/hns/hns_roce_qp.c | 2 +- drivers/infiniband/hw/hns/hns_roce_srq.c | 4 +- .../infiniband/hw/ionic/ionic_controlpath.c | 5 +- drivers/infiniband/hw/mana/cq.c | 2 +- drivers/infiniband/hw/mana/main.c | 9 ++- drivers/infiniband/hw/mana/mana_ib.h | 2 +- drivers/infiniband/hw/mana/qp.c | 9 ++- drivers/infiniband/hw/mana/wq.c | 3 +- drivers/infiniband/hw/mlx4/cq.c | 14 ++-- drivers/infiniband/hw/mlx5/cq.c | 6 +- drivers/infiniband/hw/mlx5/main.c | 56 ++++++++------ drivers/infiniband/hw/mlx5/qp.c | 29 +++---- drivers/infiniband/hw/mlx5/qpc.c | 20 +++-- drivers/infiniband/hw/mlx5/srq.c | 3 +- drivers/infiniband/hw/qedr/verbs.c | 23 ++++-- drivers/infiniband/hw/vmw_pvrdma/pvrdma_qp.c | 6 +- drivers/infiniband/hw/vmw_pvrdma/pvrdma_srq.c | 3 +- include/rdma/ib_umem.h | 2 + 25 files changed, 213 insertions(+), 119 deletions(-)
Signature · 1 line
2.18.1
Yishai Hadas
maintainerPATCH 1/15 RDMA/erdma: Pin CQ buffer writable to match device DMA write access
erdma_init_user_cq() pins the CQ ring with access=0, but the device writes CQEs into it. Pass IB_ACCESS_LOCAL_WRITE so the buffer is pinned writable and marked dirty on unpin.
--- drivers/infiniband/hw/erdma/erdma_verbs.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/infiniband/hw/erdma/erdma_verbs.c +2 -2
@@ -1930,8 +1930,8 @@ static int erdma_init_user_cq(struct erdma_ucontext *ctx, struct erdma_cq *cq,struct erdma_dev *dev =to_edev (cq->ibcq.device); ret =get_mtt_entries (dev, &cq->user_cq.qbuf_mem, ureq->qbuf_va, - ureq->qbuf_len,0 , ureq->qbuf_va,SZ_64M -SZ_4K , - true); + ureq->qbuf_len,IB_ACCESS_LOCAL_WRITE , + ureq->qbuf_va,SZ_64M -SZ_4K , true);if (ret)return ret;
Signature · 1 line
2.18.1
[email protected]
AI reviewRe: [PATCH V1 rdma-next 01/15] RDMA/erdma: Pin CQ buffer writable to match device DMA write access
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 6 lines
erdma_init_user_cq()pins the CQ ring with access=0, but the device writes CQEs into it. PassIB_ACCESS_LOCAL_WRITEso the buffer is pinned writable and marked dirty on unpin.Fixes 155055771704 ("RDMA/erdma: Add verbs implementation")Signed-off-by Yishai Hadas <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Yishai Hadas
maintainerPATCH 2/15 RDMA/hns: Pin CQ buffer writable to match device DMA write access
In reply to Yishai Hadasalloc_cq_buf() leaves user_access zero-initialized, but the device writes CQEs into the CQ buffer. Set IB_ACCESS_LOCAL_WRITE so the buffer is pinned writable, matching the device's write access.
--- drivers/infiniband/hw/hns/hns_roce_cq.c | 1 + 1 file changed, 1 insertion(+)
drivers/infiniband/hw/hns/hns_roce_cq.c +1 -0
@@ -261,6 +261,7 @@ static int alloc_cq_buf(struct hns_roce_dev *hr_dev, struct hns_roce_cq *hr_cq, buf_attr.region[0 ].size = hr_cq->cq_depth * hr_cq->cqe_size; buf_attr.region[0 ].hopnum = hr_dev->caps.cqe_hop_num; buf_attr.region_count =1 ; + buf_attr.user_access =IB_ACCESS_LOCAL_WRITE ; ret = hns_roce_mtr_create(hr_dev, &hr_cq->mtr, &buf_attr, hr_dev->caps.cqe_ba_pg_sz + PAGE_SHIFT,
Signature · 1 line
2.18.1
[email protected]
AI reviewRe: [PATCH V1 rdma-next 02/15] RDMA/hns: Pin CQ buffer writable to match device DMA write access
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 6 lines
alloc_cq_buf()leavesuser_accesszero-initialized, but the device writes CQEs into the CQ buffer. SetIB_ACCESS_LOCAL_WRITEso the buffer is pinned writable, matching the device's write access.Fixes 744b7bdfa79e ("RDMA/hns: Support 0 hop addressing for CQE buffer")Signed-off-by Yishai Hadas <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
Yishai Hadas
maintainerPATCH 3/15 RDMA/vmw_pvrdma: Pin QP and SRQ rings writable to match device DMA write access
In reply to Yishai Hadaspvrdma embeds a pvrdma_ring_state header directly inside the QP and SRQ ring buffers, and the hypervisor writes cons_head into that header. Pinning with access=0 does not force a private page copy (no FOLL_WRITE), so the hypervisor may write into a shared page.
Pass IB_ACCESS_LOCAL_WRITE for all three rings. The SRQ ring_state is not currently used by the driver but is pinned writable for consistency.
--- drivers/infiniband/hw/vmw_pvrdma/pvrdma_qp.c | 6 ++++-- drivers/infiniband/hw/vmw_pvrdma/pvrdma_srq.c | 3 ++- 2 files changed, 6 insertions(+), 3 deletions(-)
drivers/infiniband/hw/vmw_pvrdma/pvrdma_qp.c +4 -2
@@ -270,7 +270,8 @@ int pvrdma_create_qp(struct ib_qp *ibqp, struct ib_qp_init_attr *init_attr,/* set qp->sq.wqe_cnt, shift, buf_size.. */ qp->rumem =ib_umem_get_va (ibqp->device, ucmd.rbuf_addr, - ucmd.rbuf_size,0 ); + ucmd.rbuf_size, +IB_ACCESS_LOCAL_WRITE );if (IS_ERR (qp->rumem)) { ret =PTR_ERR (qp->rumem); goto err_qp;
@@ -282,7 +283,8 @@ int pvrdma_create_qp(struct ib_qp *ibqp, struct ib_qp_init_attr *init_attr, } qp->sumem =ib_umem_get_va (ibqp->device, ucmd.sbuf_addr, - ucmd.sbuf_size,0 ); + ucmd.sbuf_size, +IB_ACCESS_LOCAL_WRITE );if (IS_ERR (qp->sumem)) {if (!is_srq)ib_umem_release (qp->rumem);
drivers/infiniband/hw/vmw_pvrdma/pvrdma_srq.c +2 -1
@@ -146,7 +146,8 @@ int pvrdma_create_srq(struct ib_srq *ibsrq, struct ib_srq_init_attr *init_attr,if (ret) goto err_srq; - srq->umem =ib_umem_get_va (ibsrq->device, ucmd.buf_addr, ucmd.buf_size,0 ); + srq->umem =ib_umem_get_va (ibsrq->device, ucmd.buf_addr, ucmd.buf_size, +IB_ACCESS_LOCAL_WRITE );if (IS_ERR (srq->umem)) { ret =PTR_ERR (srq->umem); goto err_srq;
Signature · 1 line
2.18.1
[email protected]
AI reviewRe: [PATCH V1 rdma-next 03/15] RDMA/vmw_pvrdma: Pin QP and SRQ rings writable to match device DMA write access
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 10 lines
pvrdma embeds a
pvrdma_ring_stateheader directly inside the QP and SRQ ring buffers, and the hypervisor writescons_headinto that header. Pinning with access=0 does not force a private page copy (noFOLL_WRITE), so the hypervisor may write into a shared page.Pass
IB_ACCESS_LOCAL_WRITEfor all three rings. The SRQring_stateis not currently used by the driver but is pinned writable for consistency.Fixes 29c8d9eba550 ("IB: Add vmw_pvrdma driver")Signed-off-by Yishai Hadas <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
Yishai Hadas
maintainerPATCH 4/15 RDMA/umem: Reuse ib_umem_get_cq_buf_or_va() for VA-only CQ pinning
In reply to Yishai HadasSeveral drivers pin their CQ ring buffer via ib_umem_get_va() rather than the CQ-specific helper. Switch them to ib_umem_get_cq_buf_or_va() so the subsequent DMA_FROM_DEVICE change covers all CQ paths at once.
For shared helpers used by both CQ and non-CQ callers (qedr, mana, erdma, hns) a new is_cq parameter selects the appropriate pinning function; non-CQ callers pass false.
vmw_pvrdma's CQ is excluded: it embeds a ring-state header that the driver CPU-writes after polling, making it genuinely bidirectional.
This is a pure refactor with no functional change.
--- drivers/infiniband/hw/bnxt_re/ib_verbs.c | 9 ++++---- drivers/infiniband/hw/erdma/erdma_verbs.c | 17 +++++++++----- drivers/infiniband/hw/hns/hns_roce_cq.c | 2 +- drivers/infiniband/hw/hns/hns_roce_device.h | 2 +- drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 2 +- drivers/infiniband/hw/hns/hns_roce_mr.c | 22 +++++++++++++----- drivers/infiniband/hw/hns/hns_roce_qp.c | 2 +- drivers/infiniband/hw/hns/hns_roce_srq.c | 4 ++-- .../infiniband/hw/ionic/ionic_controlpath.c | 5 ++-- drivers/infiniband/hw/mana/cq.c | 2 +- drivers/infiniband/hw/mana/main.c | 9 ++++++-- drivers/infiniband/hw/mana/mana_ib.h | 2 +- drivers/infiniband/hw/mana/qp.c | 9 ++++---- drivers/infiniband/hw/mana/wq.c | 3 ++- drivers/infiniband/hw/mlx4/cq.c | 14 ++++++----- drivers/infiniband/hw/mlx5/cq.c | 6 ++--- drivers/infiniband/hw/qedr/verbs.c | 23 +++++++++++++------ 17 files changed, 84 insertions(+), 49 deletions(-)
drivers/infiniband/hw/bnxt_re/ib_verbs.c +5 -4
@@ -3749,12 +3749,13 @@ int bnxt_re_resize_cq(struct ib_cq *ibcq, unsigned int cqe,if (rc) goto fail; - cq->resize_umem =ib_umem_get_va (&rdev->ibdev, req.cq_va, - entries *sizeof (struct cq_base ), -IB_ACCESS_LOCAL_WRITE ); + cq->resize_umem =ib_umem_get_cq_buf_or_va (&rdev->ibdev,NULL , + req.cq_va, + entries *sizeof (struct cq_base ), +IB_ACCESS_LOCAL_WRITE );if (IS_ERR (cq->resize_umem)) { rc =PTR_ERR (cq->resize_umem); -ibdev_err (&rdev->ibdev,"%s: ib_umem_get_va failed! rc = %pe\n" , +ibdev_err (&rdev->ibdev,"%s: ib_umem_get_cq_buf_or_va failed! rc = %pe\n" , __func__, cq->resize_umem); cq->resize_umem =NULL ; goto fail;
drivers/infiniband/hw/erdma/erdma_verbs.c +11 -6
@@ -828,11 +828,16 @@ static void erdma_destroy_mtt(struct erdma_dev *dev, struct erdma_mtt *mtt)static int get_mtt_entries (struct erdma_dev *dev,struct erdma_mem *mem,u64 start,u64 len,int access,u64 virt, -unsigned long req_page_size,bool force_continuous) +unsigned long req_page_size,bool force_continuous, +bool is_cq) {int ret =0 ; - mem->umem =ib_umem_get_va (&dev->ibdev, start, len, access); +if (is_cq) + mem->umem =ib_umem_get_cq_buf_or_va (&dev->ibdev,NULL , start, + len, access); +else + mem->umem =ib_umem_get_va (&dev->ibdev, start, len, access);if (IS_ERR (mem->umem)) { ret =PTR_ERR (mem->umem); mem->umem =NULL ;
@@ -951,7 +956,7 @@ static int init_user_qp(struct erdma_qp *qp, struct erdma_ucontext *uctx, ret =get_mtt_entries (qp->dev, &qp->user_qp.sq_mem, va, qp->attrs.sq_size <<SQEBB_SHIFT ,0 , va, - (SZ_1M -SZ_4K ), true); + (SZ_1M -SZ_4K ), true, false);if (ret)return ret;
@@ -960,7 +965,7 @@ static int init_user_qp(struct erdma_qp *qp, struct erdma_ucontext *uctx, ret =get_mtt_entries (qp->dev, &qp->user_qp.rq_mem, va + rq_offset, qp->attrs.rq_size <<RQE_SHIFT ,0 , va + rq_offset, - (SZ_1M -SZ_4K ), true); + (SZ_1M -SZ_4K ), true, false);if (ret) goto put_sq_mtt;
@@ -1250,7 +1255,7 @@ struct ib_mr *erdma_reg_user_mr(struct ib_pd *ibpd, u64 start, u64 len,return ERR_PTR (-ENOMEM ); ret =get_mtt_entries (dev, &mr->mem, start, len, access, virt, -SZ_2G -SZ_4K , false); +SZ_2G -SZ_4K , false, false);if (ret) goto err_out_free;
@@ -1931,7 +1936,7 @@ static int erdma_init_user_cq(struct erdma_ucontext *ctx, struct erdma_cq *cq, ret =get_mtt_entries (dev, &cq->user_cq.qbuf_mem, ureq->qbuf_va, ureq->qbuf_len,IB_ACCESS_LOCAL_WRITE , - ureq->qbuf_va,SZ_64M -SZ_4K , true); + ureq->qbuf_va,SZ_64M -SZ_4K , true, true);if (ret)return ret;
drivers/infiniband/hw/hns/hns_roce_cq.c +1 -1
@@ -265,7 +265,7 @@ static int alloc_cq_buf(struct hns_roce_dev *hr_dev, struct hns_roce_cq *hr_cq, ret =hns_roce_mtr_create (hr_dev, &hr_cq->mtr, &buf_attr, hr_dev->caps.cqe_ba_pg_sz +PAGE_SHIFT , - udata, addr); + udata, addr, true);if (ret)ibdev_err (ibdev,"failed to alloc CQ mtr, ret = %d.\n" , ret);
drivers/infiniband/hw/hns/hns_roce_device.h +1 -1
@@ -1232,7 +1232,7 @@ int hns_roce_mtr_find(struct hns_roce_dev *hr_dev, struct hns_roce_mtr *mtr,int hns_roce_mtr_create (struct hns_roce_dev *hr_dev,struct hns_roce_mtr *mtr,struct hns_roce_buf_attr *buf_attr,unsigned int page_shift,struct ib_udata *udata, -unsigned long user_addr); +unsigned long user_addr,bool is_cq);void hns_roce_mtr_destroy (struct hns_roce_dev *hr_dev,struct hns_roce_mtr *mtr); int hns_roce_mtr_map(struct hns_roce_dev *hr_dev, struct hns_roce_mtr *mtr,
drivers/infiniband/hw/hns/hns_roce_hw_v2.c +1 -1
@@ -6939,7 +6939,7 @@ static int alloc_eq_buf(struct hns_roce_dev *hr_dev, struct hns_roce_eq *eq) err =hns_roce_mtr_create (hr_dev, &eq->mtr, &buf_attr, hr_dev->caps.eqe_ba_pg_sz +PAGE_SHIFT ,NULL , -0 ); +0 , false);if (err)dev_err (hr_dev->dev,"failed to alloc EQE mtr, err %d\n" , err);
drivers/infiniband/hw/hns/hns_roce_mr.c +16 -6
@@ -112,7 +112,7 @@ static int alloc_mr_pbl(struct hns_roce_dev *hr_dev, struct hns_roce_mr *mr, err =hns_roce_mtr_create (hr_dev, &mr->pbl_mtr, &buf_attr, hr_dev->caps.pbl_ba_pg_sz +PAGE_SHIFT , - udata, start); + udata, start, false);if (err) {ibdev_err (ibdev,"failed to alloc pbl mtr, ret = %d.\n" , err);return err;
@@ -586,7 +586,8 @@ static void mtr_free_bufs(struct hns_roce_dev *hr_dev, struct hns_roce_mtr *mtr)static int mtr_alloc_bufs (struct hns_roce_dev *hr_dev,struct hns_roce_mtr *mtr,struct hns_roce_buf_attr *buf_attr, -struct ib_udata *udata,unsigned long user_addr) +struct ib_udata *udata,unsigned long user_addr, +bool is_cq) {struct ib_device *ibdev = &hr_dev->ib_dev;size_t total_size;
@@ -595,8 +596,14 @@ static int mtr_alloc_bufs(struct hns_roce_dev *hr_dev, struct hns_roce_mtr *mtr,if (udata) { mtr->kmem =NULL ; - mtr->umem =ib_umem_get_va (ibdev, user_addr, total_size, - buf_attr->user_access); +if (is_cq) + mtr->umem =ib_umem_get_cq_buf_or_va (ibdev,NULL , + user_addr, + total_size, + buf_attr->user_access); +else + mtr->umem =ib_umem_get_va (ibdev, user_addr, total_size, + buf_attr->user_access);if (IS_ERR (mtr->umem)) {ibdev_err (ibdev,"failed to get umem, ret = %pe.\n" , mtr->umem);
@@ -1035,11 +1042,13 @@ static void mtr_free_mtt(struct hns_roce_dev *hr_dev, struct hns_roce_mtr *mtr)* @ba_page_shift: page shift for multi-hop base address table * @udata: user space context, if it's NULL, means kernel space * @user_addr: userspace virtual address to start at +* @is_cq: true when @mtr backs a CQ buffer, which the device writes +* completion entries into */ int hns_roce_mtr_create (struct hns_roce_dev *hr_dev,struct hns_roce_mtr *mtr,struct hns_roce_buf_attr *buf_attr,unsigned int ba_page_shift,struct ib_udata *udata, -unsigned long user_addr) +unsigned long user_addr,bool is_cq) {struct ib_device *ibdev = &hr_dev->ib_dev;int ret;
@@ -1052,7 +1061,8 @@ int hns_roce_mtr_create(struct hns_roce_dev *hr_dev, struct hns_roce_mtr *mtr, mtr->umem =NULL ; mtr->kmem =NULL ; }else { - ret =mtr_alloc_bufs (hr_dev, mtr, buf_attr, udata, user_addr); + ret =mtr_alloc_bufs (hr_dev, mtr, buf_attr, udata, user_addr, + is_cq);if (ret) {ibdev_err (ibdev,"failed to alloc mtr bufs, ret = %d.\n" , ret);
drivers/infiniband/hw/hns/hns_roce_qp.c +1 -1
@@ -808,7 +808,7 @@ static int alloc_qp_buf(struct hns_roce_dev *hr_dev, struct hns_roce_qp *hr_qp, } ret =hns_roce_mtr_create (hr_dev, &hr_qp->mtr, &buf_attr,PAGE_SHIFT + hr_dev->caps.mtt_ba_pg_sz, - udata, addr); + udata, addr, false);if (ret) {ibdev_err (ibdev,"failed to create WQE mtr, ret = %d.\n" , ret); goto err_inline;
drivers/infiniband/hw/hns/hns_roce_srq.c +2 -2
@@ -180,7 +180,7 @@ static int alloc_srq_idx(struct hns_roce_dev *hr_dev, struct hns_roce_srq *srq, ret =hns_roce_mtr_create (hr_dev, &idx_que->mtr, &buf_attr, hr_dev->caps.idx_ba_pg_sz +PAGE_SHIFT , - udata, addr); + udata, addr, false);if (ret) {ibdev_err (ibdev,"failed to alloc SRQ idx mtr, ret = %d.\n" , ret);
@@ -235,7 +235,7 @@ static int alloc_srq_wqe_buf(struct hns_roce_dev *hr_dev, ret =hns_roce_mtr_create (hr_dev, &srq->buf_mtr, &buf_attr, hr_dev->caps.srqwqe_ba_pg_sz +PAGE_SHIFT , - udata, addr); + udata, addr, false);if (ret)ibdev_err (ibdev,"failed to alloc SRQ buf mtr, ret = %d.\n" , ret);
drivers/infiniband/hw/ionic/ionic_controlpath.c +3 -2
@@ -110,8 +110,9 @@ int ionic_create_cq_common(struct ionic_vcq *vcq,if (rc) goto err_qdesc; - cq->umem =ib_umem_get_va (&dev->ibdev, req_cq->addr, - req_cq->size,IB_ACCESS_LOCAL_WRITE ); + cq->umem =ib_umem_get_cq_buf_or_va (&dev->ibdev,NULL , + req_cq->addr, req_cq->size, +IB_ACCESS_LOCAL_WRITE );if (IS_ERR (cq->umem)) { rc =PTR_ERR (cq->umem); goto err_qdesc;
drivers/infiniband/hw/mana/cq.c +1 -1
@@ -40,7 +40,7 @@ int mana_ib_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr, cq->cqe = attr->cqe; err =mana_ib_create_queue (mdev, ucmd.buf_addr, cq->cqe *COMP_ENTRY_SIZE , - &cq->queue); + &cq->queue, true);if (err) {ibdev_dbg (ibdev,"Failed to create queue for create cq, %d\n" , err);return err;
drivers/infiniband/hw/mana/main.c +7 -2
@@ -295,7 +295,7 @@ int mana_ib_create_kernel_queue(struct mana_ib_dev *mdev, u32 size, enum gdma_qu }int mana_ib_create_queue (struct mana_ib_dev *mdev,u64 addr,u32 size, -struct mana_ib_queue *queue) +struct mana_ib_queue *queue,bool is_cq) {struct ib_umem *umem;int err;
@@ -304,7 +304,12 @@ int mana_ib_create_queue(struct mana_ib_dev *mdev, u64 addr, u32 size, queue->id =INVALID_QUEUE_ID ; queue->gdma_region =GDMA_INVALID_DMA_REGION ; - umem =ib_umem_get_va (&mdev->ib_dev, addr, size,IB_ACCESS_LOCAL_WRITE ); +if (is_cq) + umem =ib_umem_get_cq_buf_or_va (&mdev->ib_dev,NULL , addr, + size,IB_ACCESS_LOCAL_WRITE ); +else + umem =ib_umem_get_va (&mdev->ib_dev, addr, size, +IB_ACCESS_LOCAL_WRITE );if (IS_ERR (umem)) {ibdev_dbg (&mdev->ib_dev,"Failed to get umem, %pe\n" , umem);return PTR_ERR (umem);
drivers/infiniband/hw/mana/mana_ib.h +1 -1
@@ -731,7 +731,7 @@ int mana_ib_gd_destroy_dma_region(struct mana_ib_dev *dev,int mana_ib_create_kernel_queue (struct mana_ib_dev *mdev,u32 size,enum gdma_queue_type type,struct mana_ib_queue *queue);int mana_ib_create_queue (struct mana_ib_dev *mdev,u64 addr,u32 size, -struct mana_ib_queue *queue); +struct mana_ib_queue *queue,bool is_cq);void mana_ib_destroy_queue (struct mana_ib_dev *mdev,struct mana_ib_queue *queue); struct ib_wq *mana_ib_create_wq(struct ib_pd *pd,
drivers/infiniband/hw/mana/qp.c +5 -4
@@ -340,7 +340,8 @@ static int mana_ib_create_qp_raw(struct ib_qp *ibqp, struct ib_pd *ibpd,ibdev_dbg (&mdev->ib_dev,"ucmd sq_buf_addr 0x%llx port %u\n" , ucmd.sq_buf_addr, ucmd.port); - err =mana_ib_create_queue (mdev, ucmd.sq_buf_addr, ucmd.sq_buf_size, &qp->raw_sq); + err =mana_ib_create_queue (mdev, ucmd.sq_buf_addr, ucmd.sq_buf_size, &qp->raw_sq, + false);if (err) {ibdev_dbg (&mdev->ib_dev,"Failed to create queue for create qp-raw, err %d\n" , err);
@@ -579,7 +580,7 @@ static int mana_ib_create_rc_qp(struct ib_qp *ibqp, struct ib_pd *ibpd,if (ucmd.comp_mask &MANA_IB_RC_MMQ_CREATE ) { flags &= ~MANA_RC_FLAG_NO_MMQ ; err =mana_ib_create_queue (mdev, ucmd.mmq_buf, ucmd.mmq_size, - &qp->rc_qp.queues[i]); + &qp->rc_qp.queues[i], false);if (err) goto destroy_queues; }else {
@@ -589,7 +590,7 @@ static int mana_ib_create_rc_qp(struct ib_qp *ibqp, struct ib_pd *ibpd,continue ; } err =mana_ib_create_queue (mdev, ucmd.queue_buf[j], ucmd.queue_size[j], - &qp->rc_qp.queues[i]); + &qp->rc_qp.queues[i], false);if (err) goto destroy_queues; j++;
@@ -663,7 +664,7 @@ static int mana_ib_create_uc_qp(struct ib_qp *ibqp, struct ib_pd *ibpd,for (i =0 ; i <MANA_UC_QUEUE_TYPE_MAX ; ++i) { err =mana_ib_create_queue (mdev, ucmd.queue_buf[i], ucmd.queue_size[i], - &qp->uc_qp.queues[i]); + &qp->uc_qp.queues[i], false);if (err) goto destroy_queues; }
drivers/infiniband/hw/mana/wq.c +2 -1
@@ -25,7 +25,8 @@ struct ib_wq *mana_ib_create_wq(struct ib_pd *pd,ibdev_dbg (&mdev->ib_dev,"ucmd wq_buf_addr 0x%llx\n" , ucmd.wq_buf_addr); - err =mana_ib_create_queue (mdev, ucmd.wq_buf_addr, ucmd.wq_buf_size, &wq->queue); + err =mana_ib_create_queue (mdev, ucmd.wq_buf_addr, ucmd.wq_buf_size, &wq->queue, + false);if (err) {ibdev_dbg (&mdev->ib_dev,"Failed to create queue for create wq, %d\n" , err);
drivers/infiniband/hw/mlx4/cq.c +8 -6
@@ -185,9 +185,10 @@ int mlx4_ib_create_user_cq(struct ib_cq *ibcq, goto err_umem; } }else { - cq->umem =ib_umem_get_va (&dev->ib_dev, ucmd.buf_addr, - entries * cqe_size, -IB_ACCESS_LOCAL_WRITE ); + cq->umem =ib_umem_get_cq_buf_or_va (&dev->ib_dev,NULL , + ucmd.buf_addr, + entries * cqe_size, +IB_ACCESS_LOCAL_WRITE );if (IS_ERR (cq->umem)) { err =PTR_ERR (cq->umem); goto err_cq;
@@ -354,9 +355,10 @@ static int mlx4_alloc_resize_umem(struct mlx4_ib_dev *dev, struct mlx4_ib_cq *cqif (!cq->resize_buf)return -ENOMEM ; - cq->resize_umem =ib_umem_get_va (&dev->ib_dev, ucmd.buf_addr, - entries * cqe_size, -IB_ACCESS_LOCAL_WRITE ); + cq->resize_umem =ib_umem_get_cq_buf_or_va (&dev->ib_dev,NULL , + ucmd.buf_addr, + entries * cqe_size, +IB_ACCESS_LOCAL_WRITE );if (IS_ERR (cq->resize_umem)) { err =PTR_ERR (cq->resize_umem); goto err_buf;
drivers/infiniband/hw/mlx5/cq.c +3 -3
@@ -1245,9 +1245,9 @@ static int resize_user(struct mlx5_ib_dev *dev, struct mlx5_ib_cq *cq,if (ucmd.cqe_size &&SIZE_MAX / ucmd.cqe_size <= entries -1 )return -EINVAL ; - umem =ib_umem_get_va (&dev->ib_dev, ucmd.buf_addr, - (size_t )ucmd.cqe_size * entries, -IB_ACCESS_LOCAL_WRITE ); + umem =ib_umem_get_cq_buf_or_va (&dev->ib_dev,NULL , ucmd.buf_addr, + (size_t )ucmd.cqe_size * entries, +IB_ACCESS_LOCAL_WRITE );if (IS_ERR (umem)) { err =PTR_ERR (umem);return err;
drivers/infiniband/hw/qedr/verbs.c +16 -7
@@ -775,16 +775,23 @@ static inline int qedr_init_user_queue(struct ib_udata *udata, struct qedr_userq *q, u64 buf_addr, size_t buf_len, bool requires_db_rec, int access, - int alloc_and_init) + int alloc_and_init, bool is_cq) { u32 fw_pages;int rc; q->buf_addr = buf_addr; q->buf_len = buf_len; - q->umem =ib_umem_get_va (&dev->ibdev, q->buf_addr, q->buf_len, access); +if (is_cq) + q->umem =ib_umem_get_cq_buf_or_va (&dev->ibdev,NULL , + q->buf_addr, q->buf_len, + access); +else + q->umem =ib_umem_get_va (&dev->ibdev, q->buf_addr, q->buf_len, + access);if (IS_ERR (q->umem)) { -DP_ERR (dev,"create user queue: failed ib_umem_get_va, got %ld\n" , +DP_ERR (dev,"create user queue: failed %s, got %ld\n" , + is_cq ?"ib_umem_get_cq_buf_or_va" :"ib_umem_get_va" ,PTR_ERR (q->umem));return PTR_ERR (q->umem); }
@@ -946,7 +953,7 @@ int qedr_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr, rc =qedr_init_user_queue (udata, dev, &cq->q, ureq.addr, ureq.len, true,IB_ACCESS_LOCAL_WRITE , -1 ); +1 , true);if (rc) goto err0;
@@ -1438,7 +1445,7 @@ static int qedr_init_srq_user_params(struct ib_udata *udata,int rc; rc =qedr_init_user_queue (udata, srq->dev, &srq->usrq, ureq->srq_addr, - ureq->srq_len, false, access,1 ); + ureq->srq_len, false, access,1 , false);if (rc)return rc;
@@ -1831,7 +1838,8 @@ static int qedr_create_user_qp(struct qedr_dev *dev,if (qedr_qp_has_sq (qp)) {/* SQ - read access only (0) */ rc =qedr_init_user_queue (udata, dev, &qp->usq, ureq.sq_addr, - ureq.sq_len, true,0 , alloc_and_init); + ureq.sq_len, true,0 , alloc_and_init, + false);if (rc)return rc; }
@@ -1839,7 +1847,8 @@ static int qedr_create_user_qp(struct qedr_dev *dev,if (qedr_qp_has_rq (qp)) {/* RQ - read access only (0) */ rc =qedr_init_user_queue (udata, dev, &qp->urq, ureq.rq_addr, - ureq.rq_len, true,0 , alloc_and_init); + ureq.rq_len, true,0 , alloc_and_init, + false);if (rc) {ib_umem_release (qp->usq.umem); qp->usq.umem =NULL ;
Signature · 1 line
2.18.1
[email protected]
AI reviewRe: [PATCH V1 rdma-next 04/15] RDMA/umem: Reuse ib_umem_get_cq_buf_or_va() for VA-only CQ pinning
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 8 lines
Several drivers pin their CQ ring buffer via
ib_umem_get_va()rather than the CQ-specific helper. Switch them toib_umem_get_cq_buf_or_va()so the subsequentDMA_FROM_DEVICEchange covers all CQ paths at once.For shared helpers used by both CQ and non-CQ callers (qedr, mana, erdma, hns) a new
is_cqparameter selects the appropriate pinning function; non-CQ callers pass false.
Quoted 6 lines
vmw_pvrdma's CQ is excluded: it embeds a ring-state header that the driver CPU-writes after polling, making it genuinely bidirectional.
This is a pure refactor with no functional change.
Signed-off-by Yishai Hadas <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
Re: [PATCH V1 rdma-next 04/15] RDMA/umem: Reuse ib_umem_get_cq_buf_or_va() for VA-only CQ pinning
In reply to Yishai HadasQuoted 49 lines · Yishai Hadas
Several drivers pin their CQ ring buffer via
ib_umem_get_va()rather than the CQ-specific helper. Switch them toib_umem_get_cq_buf_or_va()so the subsequentDMA_FROM_DEVICEchange covers all CQ paths at once.For shared helpers used by both CQ and non-CQ callers (qedr, mana, erdma, hns) a new
is_cqparameter selects the appropriate pinning function; non-CQ callers pass false.vmw_pvrdma's CQ is excluded: it embeds a ring-state header that the driver CPU-writes after polling, making it genuinely bidirectional.
This is a pure refactor with no functional change.
Signed-off-by Yishai Hadas <[email protected]>--- drivers/infiniband/hw/bnxt_re/ib_verbs.c | 9 ++++---- drivers/infiniband/hw/erdma/erdma_verbs.c | 17 +++++++++----- drivers/infiniband/hw/hns/hns_roce_cq.c | 2 +- drivers/infiniband/hw/hns/hns_roce_device.h | 2 +- drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 2 +- drivers/infiniband/hw/hns/hns_roce_mr.c | 22 +++++++++++++----- drivers/infiniband/hw/hns/hns_roce_qp.c | 2 +- drivers/infiniband/hw/hns/hns_roce_srq.c | 4 ++-- .../infiniband/hw/ionic/ionic_controlpath.c | 5 ++-- drivers/infiniband/hw/mana/cq.c | 2 +- drivers/infiniband/hw/mana/main.c | 9 ++++++-- drivers/infiniband/hw/mana/mana_ib.h | 2 +- drivers/infiniband/hw/mana/qp.c | 9 ++++---- drivers/infiniband/hw/mana/wq.c | 3 ++- drivers/infiniband/hw/mlx4/cq.c | 14 ++++++----- drivers/infiniband/hw/mlx5/cq.c | 6 ++--- drivers/infiniband/hw/qedr/verbs.c | 23 +++++++++++++------ 17 files changed, 84 insertions(+), 49 deletions(-)drivers/infiniband/hw/bnxt_re/ib_verbs.c +4
-3@@ -3749,12 +3749,13 @@ int bnxt_re_resize_cq(struct ib_cq *ibcq, unsigned int cqe,if (rc) goto fail; - cq->resize_umem =ib_umem_get_va (&rdev->ibdev, req.cq_va, - entries *sizeof (struct cq_base ), -IB_ACCESS_LOCAL_WRITE ); + cq->resize_umem =ib_umem_get_cq_buf_or_va (&rdev->ibdev,NULL , + req.cq_va, + entries *sizeof (struct cq_base ), +IB_ACCESS_LOCAL_WRITE );
<...>
Quoted 9 lines
index
fca2553e47ad..3519b5044e8f100644drivers/infiniband/hw/erdma/erdma_verbs.c +1
-1@@ -828,11 +828,16 @@ static void erdma_destroy_mtt(struct erdma_dev *dev, struct erdma_mtt *mtt) static int get_mtt_entries(struct erdma_dev *dev, struct erdma_mem *mem, u64 start, u64 len, int access, u64 virt, -unsigned long req_page_size,bool force_continuous) + unsigned long req_page_size, bool force_continuous,
drivers/infiniband/hw/erdma/erdma_verbs.c static void erdma_destroy_mtt(struct erdma_dev *dev, struct erdma_mtt *mtt) · line 840
Quoted 10 lines
drivers/infiniband/hw/erdma/erdma_verbs.c +6
-1+ bool is_cq) {int ret =0 ; - mem->umem =ib_umem_get_va (&dev->ibdev, start, len, access); +if (is_cq) + mem->umem =ib_umem_get_cq_buf_or_va (&dev->ibdev,NULL , start, + len, access); +else + mem->umem =ib_umem_get_va (&dev->ibdev, start, len, access);
Sorry, but this patch doesn't look right to me.
In bnxt_re_resize_cq(), you chose to move away from the ib_umem_get_va() API, but here you still keep that API because most of its callers are not CQs. Why is it so important to specify the direction for CQs while QPs and MRs are left without it?
Thanks
Quoted 31 lines
drivers/infiniband/hw/erdma/erdma_verbs.c +3
-3if (IS_ERR (mem->umem)) { ret =PTR_ERR (mem->umem); mem->umem =NULL ;@@ -951,7 +956,7 @@ static int init_user_qp(struct erdma_qp *qp, struct erdma_ucontext *uctx, ret =get_mtt_entries (qp->dev, &qp->user_qp.sq_mem, va, qp->attrs.sq_size <<SQEBB_SHIFT ,0 , va, - (SZ_1M -SZ_4K ), true); + (SZ_1M -SZ_4K ), true, false);if (ret)return ret;@@ -960,7 +965,7 @@ static int init_user_qp(struct erdma_qp *qp, struct erdma_ucontext *uctx, ret =get_mtt_entries (qp->dev, &qp->user_qp.rq_mem, va + rq_offset, qp->attrs.rq_size <<RQE_SHIFT ,0 , va + rq_offset, - (SZ_1M -SZ_4K ), true); + (SZ_1M -SZ_4K ), true, false);if (ret) goto put_sq_mtt;@@ -1250,7 +1255,7 @@ struct ib_mr *erdma_reg_user_mr(struct ib_pd *ibpd, u64 start, u64 len,return ERR_PTR (-ENOMEM ); ret =get_mtt_entries (dev, &mr->mem, start, len, access, virt, -SZ_2G -SZ_4K , false); +SZ_2G -SZ_4K , false, false);if (ret) goto err_out_free;
drivers/infiniband/hw/erdma/erdma_verbs.c static int erdma_init_user_cq(struct erdma_ucontext *ctx, struct erdma_cq *cq, · line 1941
Quoted 7 lines
drivers/infiniband/hw/erdma/erdma_verbs.c +1
-1@@ -1931,7 +1936,7 @@ static int erdma_init_user_cq(struct erdma_ucontext *ctx, struct erdma_cq *cq, ret =get_mtt_entries (dev, &cq->user_cq.qbuf_mem, ureq->qbuf_va, ureq->qbuf_len,IB_ACCESS_LOCAL_WRITE , - ureq->qbuf_va,SZ_64M -SZ_4K , true); + ureq->qbuf_va,SZ_64M -SZ_4K , true, true);if (ret)return ret;
Thanks
Yishai Hadas
maintainerRe: [PATCH V1 rdma-next 04/15] RDMA/umem: Reuse ib_umem_get_cq_buf_or_va() for VA-only CQ pinning
Quoted 76 lines · Leon Romanovsky
Quoted 49 lines · Yishai Hadas
Several drivers pin their CQ ring buffer via
ib_umem_get_va()rather than the CQ-specific helper. Switch them toib_umem_get_cq_buf_or_va()so the subsequentDMA_FROM_DEVICEchange covers all CQ paths at once.For shared helpers used by both CQ and non-CQ callers (qedr, mana, erdma, hns) a new
is_cqparameter selects the appropriate pinning function; non-CQ callers pass false.vmw_pvrdma's CQ is excluded: it embeds a ring-state header that the driver CPU-writes after polling, making it genuinely bidirectional.
This is a pure refactor with no functional change.
Signed-off-by Yishai Hadas <[email protected]>--- drivers/infiniband/hw/bnxt_re/ib_verbs.c | 9 ++++---- drivers/infiniband/hw/erdma/erdma_verbs.c | 17 +++++++++----- drivers/infiniband/hw/hns/hns_roce_cq.c | 2 +- drivers/infiniband/hw/hns/hns_roce_device.h | 2 +- drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 2 +- drivers/infiniband/hw/hns/hns_roce_mr.c | 22 +++++++++++++----- drivers/infiniband/hw/hns/hns_roce_qp.c | 2 +- drivers/infiniband/hw/hns/hns_roce_srq.c | 4 ++-- .../infiniband/hw/ionic/ionic_controlpath.c | 5 ++-- drivers/infiniband/hw/mana/cq.c | 2 +- drivers/infiniband/hw/mana/main.c | 9 ++++++-- drivers/infiniband/hw/mana/mana_ib.h | 2 +- drivers/infiniband/hw/mana/qp.c | 9 ++++---- drivers/infiniband/hw/mana/wq.c | 3 ++- drivers/infiniband/hw/mlx4/cq.c | 14 ++++++----- drivers/infiniband/hw/mlx5/cq.c | 6 ++--- drivers/infiniband/hw/qedr/verbs.c | 23 +++++++++++++------ 17 files changed, 84 insertions(+), 49 deletions(-)drivers/infiniband/hw/bnxt_re/ib_verbs.c +4
-3@@ -3749,12 +3749,13 @@ int bnxt_re_resize_cq(struct ib_cq *ibcq, unsigned int cqe,if (rc) goto fail; - cq->resize_umem =ib_umem_get_va (&rdev->ibdev, req.cq_va, - entries *sizeof (struct cq_base ), -IB_ACCESS_LOCAL_WRITE ); + cq->resize_umem =ib_umem_get_cq_buf_or_va (&rdev->ibdev,NULL , + req.cq_va, + entries *sizeof (struct cq_base ), +IB_ACCESS_LOCAL_WRITE );<...>
Quoted 9 lines
index
fca2553e47ad..3519b5044e8f100644drivers/infiniband/hw/erdma/erdma_verbs.c +1
-1@@ -828,11 +828,16 @@ static void erdma_destroy_mtt(struct erdma_dev *dev, struct erdma_mtt *mtt) static int get_mtt_entries(struct erdma_dev *dev, struct erdma_mem *mem, u64 start, u64 len, int access, u64 virt, -unsigned long req_page_size,bool force_continuous) + unsigned long req_page_size, bool force_continuous,drivers/infiniband/hw/erdma/erdma_verbs.c static void erdma_destroy_mtt(struct erdma_dev *dev, struct erdma_mtt *mtt) · line 840
Quoted 10 lines
drivers/infiniband/hw/erdma/erdma_verbs.c +6
-1+ bool is_cq) {int ret =0 ; - mem->umem =ib_umem_get_va (&dev->ibdev, start, len, access); +if (is_cq) + mem->umem =ib_umem_get_cq_buf_or_va (&dev->ibdev,NULL , start, + len, access); +else + mem->umem =ib_umem_get_va (&dev->ibdev, start, len, access);Sorry, but this patch doesn't look right to me.
Quoted 5 lines
In
bnxt_re_resize_cq(), you chose to move away from theib_umem_get_va()API, but here you still keep that API because most of its callers are not CQs. Why is it so important to specify the direction for CQs while QPs and MRs are left without it?
For CQs, IB_ACCESS_LOCAL_WRITE isn't describing who writes the memory — it's there purely so get_user_pages() pins a writable private page (FOLL_WRITE), which the NIC needs to DMA-write CQEs into.
The access-flag derivation added later in this series ("RDMA/umem: Derive DMA direction from IB access flags"): ib_access_writable(access) ? DMA_BIDIRECTIONAL : DMA_TO_DEVICE reads that same flag as "this memory is written from both sides" and picks DMA_BIDIRECTIONAL. But the actual bus traffic is one-way: NIC writes, CPU only reads. That's why CQs need an explicit DMA_FROM_DEVICE instead of the derived value — they're the one buffer type where the flag used for pinning and the flag used for direction don't mean the same thing.
QPs and MRs don't have this mismatch, so the derived value is already correct for them:
QPs are pinned with access = 0 because the device never writes into a QP ring — WQEs are CPU-written, NIC-read only. FOLL_WRITE matters for writers that bypass the page-fault path (DMA writes); ordinary CPU writes get a private age via the normal fault/COW mechanism regardless of pinning flags. Since the device never writes here, there's no DMA-write-into-shared-page hazard to guard against, so access = 0 is correct.
MRs are pinned with the actual IB access flags the user requested for that region (LOCAL_WRITE, REMOTE_WRITE, etc.) — those flags are the permissions granted to the device, so they directly describe its real access pattern.
Yishai
Quoted 42 lines
Thanks
Quoted 31 lines
drivers/infiniband/hw/erdma/erdma_verbs.c +3
-3if (IS_ERR (mem->umem)) { ret =PTR_ERR (mem->umem); mem->umem =NULL ;@@ -951,7 +956,7 @@ static int init_user_qp(struct erdma_qp *qp, struct erdma_ucontext *uctx, ret =get_mtt_entries (qp->dev, &qp->user_qp.sq_mem, va, qp->attrs.sq_size <<SQEBB_SHIFT ,0 , va, - (SZ_1M -SZ_4K ), true); + (SZ_1M -SZ_4K ), true, false);if (ret)return ret;@@ -960,7 +965,7 @@ static int init_user_qp(struct erdma_qp *qp, struct erdma_ucontext *uctx, ret =get_mtt_entries (qp->dev, &qp->user_qp.rq_mem, va + rq_offset, qp->attrs.rq_size <<RQE_SHIFT ,0 , va + rq_offset, - (SZ_1M -SZ_4K ), true); + (SZ_1M -SZ_4K ), true, false);if (ret) goto put_sq_mtt;@@ -1250,7 +1255,7 @@ struct ib_mr *erdma_reg_user_mr(struct ib_pd *ibpd, u64 start, u64 len,return ERR_PTR (-ENOMEM ); ret =get_mtt_entries (dev, &mr->mem, start, len, access, virt, -SZ_2G -SZ_4K , false); +SZ_2G -SZ_4K , false, false);if (ret) goto err_out_free;drivers/infiniband/hw/erdma/erdma_verbs.c static int erdma_init_user_cq(struct erdma_ucontext *ctx, struct erdma_cq *cq, · line 1941
Quoted 7 lines
drivers/infiniband/hw/erdma/erdma_verbs.c +1
-1@@ -1931,7 +1936,7 @@ static int erdma_init_user_cq(struct erdma_ucontext *ctx, struct erdma_cq *cq, ret =get_mtt_entries (dev, &cq->user_cq.qbuf_mem, ureq->qbuf_va, ureq->qbuf_len,IB_ACCESS_LOCAL_WRITE , - ureq->qbuf_va,SZ_64M -SZ_4K , true); + ureq->qbuf_va,SZ_64M -SZ_4K , true, true);if (ret)return ret;Thanks
Yishai Hadas
maintainerPATCH 5/15 RDMA/umem: Support an explicit DMA direction other than DMA_BIDIRECTIONAL
In reply to Yishai HadasAdd a dma_dir field to struct ib_umem to record the direction chosen at map time and reuse it at unmap. Thread an explicit dma_data_direction parameter through the internal pinning helpers so each caller can supply the direction that matches the device's actual access pattern.
All existing callers still pass DMA_BIDIRECTIONAL, so this is a pure mechanism addition with no behavioural change.
--- drivers/infiniband/core/umem.c | 64 ++++++++++++++++++++++------------ include/rdma/ib_umem.h | 2 ++ 2 files changed, 43 insertions(+), 23 deletions(-)
drivers/infiniband/core/umem.c +41 -23
@@ -55,7 +55,7 @@ static void __ib_umem_release(struct ib_device *dev, struct ib_umem *umem, int dif (dirty)ib_dma_unmap_sgtable_attrs (dev, &umem->sgt_append.sgt, -DMA_BIDIRECTIONAL , umem->dma_attrs); + umem->dma_dir, umem->dma_attrs);for_each_sgtable_sg (&umem->sgt_append.sgt, sg, i) { unpin_user_page_range_dirty_lock(sg_page(sg),
@@ -161,7 +161,8 @@ EXPORT_SYMBOL(ib_umem_find_best_pgsz);static struct ib_umem *__ib_umem_get_va (struct ib_device *device,unsigned long addr,size_t size, -int access) +int access, +enum dma_data_direction dir) {struct ib_umem *umem;struct page **page_list;
@@ -202,6 +203,7 @@ static struct ib_umem *__ib_umem_get_va(struct ib_device *device,*/ umem->iova = addr; umem->writable =ib_access_writable (access); + umem->dma_dir = dir; umem->owning_mm = mm = current->mm; umem->dma_attrs =DMA_ATTR_REQUIRE_COHERENT ;if (access &IB_ACCESS_RELAXED_ORDERING )
@@ -261,7 +263,7 @@ static struct ib_umem *__ib_umem_get_va(struct ib_device *device, } ret =ib_dma_map_sgtable_attrs (device, &umem->sgt_append.sgt, -DMA_BIDIRECTIONAL , umem->dma_attrs); + dir, umem->dma_attrs);if (ret) goto umem_release; goto out;
@@ -279,17 +281,16 @@ static struct ib_umem *__ib_umem_get_va(struct ib_device *device,return ret ?ERR_PTR (ret) : umem; } -/** -* ib_umem_get_desc - Pin a umem from a buffer descriptor. -* @device: IB device. -* @desc: buffer descriptor (VA or DMABUF). -* @access: IB access flags. +/* +* __ib_umem_get_desc_dir - core implementation for ib_umem_get_desc(). * -* Return: caller-owned umem on success, ERR_PTR(...) on error. +* @dir applies to VA buffers only; dmabuf direction is managed by the +* dmabuf subsystem and this argument is ignored for that type. */ -struct ib_umem *ib_umem_get_desc (struct ib_device *device, -const struct ib_uverbs_buffer_desc *desc, -int access) +static struct ib_umem * +__ib_umem_get_desc_dir (struct ib_device *device, +const struct ib_uverbs_buffer_desc *desc, +int access,enum dma_data_direction dir) {struct ib_umem_dmabuf *umem_dmabuf;
@@ -310,11 +311,26 @@ struct ib_umem *ib_umem_get_desc(struct ib_device *device,return &umem_dmabuf->umem;case IB_UVERBS_BUFFER_TYPE_VA :return __ib_umem_get_va (device, desc->addr, desc->length, - access); + access, dir);default :return ERR_PTR (-EINVAL ); } } + +/** +* ib_umem_get_desc - Pin a umem from a buffer descriptor. +* @device: IB device. +* @desc: buffer descriptor (VA or DMABUF). +* @access: IB access flags. +* +* Return: caller-owned umem on success, ERR_PTR(...) on error. +*/ +struct ib_umem *ib_umem_get_desc (struct ib_device *device, +const struct ib_uverbs_buffer_desc *desc, +int access) +{ +return __ib_umem_get_desc_dir (device, desc, access,DMA_BIDIRECTIONAL ); +}EXPORT_SYMBOL (ib_umem_get_desc);/*
@@ -376,11 +392,12 @@ static int ib_umem_resolve_desc(const struct uverbs_attr_bundle *attrs,static struct ib_umem *ib_umem_get_desc_check (struct ib_device *device,const struct ib_uverbs_buffer_desc *desc, -size_t min_size,int access) +size_t min_size,int access, +enum dma_data_direction dir) {struct ib_umem *umem; - umem =ib_umem_get_desc (device, desc, access); + umem =__ib_umem_get_desc_dir (device, desc, access, dir);if (IS_ERR (umem))return umem;if (umem->length < min_size) {
@@ -401,7 +418,7 @@ static struct ib_umem * ib_umem_get_from_attrs(struct ib_device *device, const struct uverbs_attr_bundle *attrs, u16 attr_id, ib_umem_buf_desc_filler_t legacy_filler, - size_t size, int access) + size_t size, int access, enum dma_data_direction dir) { struct ib_uverbs_buffer_desc desc = {};int ret;
@@ -411,7 +428,7 @@ ib_umem_get_from_attrs(struct ib_device *device,return NULL ;if (ret)return ERR_PTR (ret); -return ib_umem_get_desc_check (device, &desc, size, access); +return ib_umem_get_desc_check (device, &desc, size, access, dir); }/*
@@ -431,7 +448,8 @@ ib_umem_get_from_attrs_or_va(struct ib_device *device,const struct uverbs_attr_bundle *attrs, u16 attr_id, ib_umem_buf_desc_filler_t legacy_filler, - u64 addr, size_t size, int access) + u64 addr, size_t size, int access, + enum dma_data_direction dir) { struct ib_uverbs_buffer_desc desc = {};int ret;
@@ -445,7 +463,7 @@ ib_umem_get_from_attrs_or_va(struct ib_device *device, }; else if (ret) return ERR_PTR(ret); - return ib_umem_get_desc_check(device, &desc, size, access); + return ib_umem_get_desc_check(device, &desc, size, access, dir); }/**
@@ -464,7 +482,7 @@ struct ib_umem *ib_umem_get_attr(struct ib_device *device, u16 attr_id, size_t size, int access) { return ib_umem_get_from_attrs(device, attrs, attr_id, NULL, size, -access ); +access ,DMA_BIDIRECTIONAL ); }EXPORT_SYMBOL (ib_umem_get_attr);
@@ -503,7 +521,7 @@ struct ib_umem *ib_umem_get_attr_or_va(struct ib_device *device, int access) {return ib_umem_get_from_attrs_or_va (device, attrs, attr_id,NULL , addr, - size, access); + size, access,DMA_BIDIRECTIONAL ); }EXPORT_SYMBOL (ib_umem_get_attr_or_va);
@@ -579,7 +597,7 @@ struct ib_umem *ib_umem_get_cq_buf(struct ib_device *device,return ib_umem_get_from_attrs (device, attrs,UVERBS_ATTR_CREATE_CQ_BUF_UMEM , uverbs_create_cq_get_buffer_desc, - size, access); + size, access,DMA_BIDIRECTIONAL ); }EXPORT_SYMBOL (ib_umem_get_cq_buf);
@@ -608,7 +626,7 @@ struct ib_umem *ib_umem_get_cq_buf_or_va(struct ib_device *device,return ib_umem_get_from_attrs_or_va (device, attrs,UVERBS_ATTR_CREATE_CQ_BUF_UMEM , uverbs_create_cq_get_buffer_desc, - addr, size, access); + addr, size, access,DMA_BIDIRECTIONAL ); }EXPORT_SYMBOL (ib_umem_get_cq_buf_or_va);
include/rdma/ib_umem.h +2 -0
@@ -7,6 +7,7 @@#ifndef IB_UMEM_H #define IB_UMEM_H +#include <linux/dma-direction.h> #include <linux/scatterlist.h> struct ib_device ;
@@ -19,6 +20,7 @@ struct ib_umem {size_t length;unsigned long address;unsigned long dma_attrs; +enum dma_data_direction dma_dir; u32 writable : 1; u32 is_odp : 1; u32 is_dmabuf : 1;
Signature · 1 line
2.18.1
[email protected]
AI reviewRe: [PATCH V1 rdma-next 05/15] RDMA/umem: Support an explicit DMA direction other than DMA_BIDIRECTIONAL
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 9 lines
Add a
dma_dirfield to structib_umemto record the direction chosen at map time and reuse it at unmap. Thread an explicitdma_data_directionparameter through the internal pinning helpers so each caller can supply the direction that matches the device's actual access pattern.All existing callers still pass
DMA_BIDIRECTIONAL, so this is a pure mechanism addition with no behavioural change.Signed-off-by Yishai Hadas <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
Yishai Hadas
maintainerPATCH 6/15 RDMA/umem: Map CQ buffers DMA_FROM_DEVICE
In reply to Yishai HadasFor a CQ ring the device writes CQEs and the CPU only reads. DMA_FROM_DEVICE is the correct direction. Use it explicitly in ib_umem_get_cq_buf() and ib_umem_get_cq_buf_or_va() rather than deriving it from the access flags: CQ callers pass IB_ACCESS_LOCAL_WRITE so that get_user_pages() pins a writable private page (FOLL_WRITE), which the device needs to DMA-write CQEs into. That flag would yield DMA_BIDIRECTIONAL from the access-flag derivation, which is wrong for the bus direction.
--- drivers/infiniband/core/umem.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-)
drivers/infiniband/core/umem.c +8 -2
@@ -587,6 +587,9 @@ static int uverbs_create_cq_get_buffer_desc(const struct uverbs_attr_bundle *att* must arrange its own backing (typically an in-kernel allocation) * when no source is available. * +* The buffer is mapped DMA_FROM_DEVICE: the NIC writes CQEs into it +* and the CPU only reads. +* * Return: caller-owned umem on success; NULL when no source supplied * a buffer; ERR_PTR(...) on error. */
@@ -597,7 +600,7 @@ struct ib_umem *ib_umem_get_cq_buf(struct ib_device *device,return ib_umem_get_from_attrs (device, attrs,UVERBS_ATTR_CREATE_CQ_BUF_UMEM , uverbs_create_cq_get_buffer_desc, - size, access,DMA_BIDIRECTIONAL ); + size, access,DMA_FROM_DEVICE ); }EXPORT_SYMBOL (ib_umem_get_cq_buf);
@@ -613,6 +616,9 @@ EXPORT_SYMBOL(ib_umem_get_cq_buf);* Like ib_umem_get_cq_buf(), but pins @addr/@size when neither the * UMEM attribute nor the legacy CQ buffer attributes are supplied. * +* The buffer is mapped DMA_FROM_DEVICE: the NIC writes CQEs into it +* and the CPU only reads. +* * See ib_umem_get_attr_or_va() for the note on @size's dual role and * the migration path for drivers that would distinguish a user-supplied * length from a driver-computed minimum.
@@ -626,7 +632,7 @@ struct ib_umem *ib_umem_get_cq_buf_or_va(struct ib_device *device,return ib_umem_get_from_attrs_or_va (device, attrs,UVERBS_ATTR_CREATE_CQ_BUF_UMEM , uverbs_create_cq_get_buffer_desc, - addr, size, access,DMA_BIDIRECTIONAL ); + addr, size, access,DMA_FROM_DEVICE ); }EXPORT_SYMBOL (ib_umem_get_cq_buf_or_va);
Signature · 1 line
2.18.1
[email protected]
AI reviewRe: [PATCH V1 rdma-next 06/15] RDMA/umem: Map CQ buffers DMA_FROM_DEVICE
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 10 lines
For a CQ ring the device writes CQEs and the CPU only reads.
DMA_FROM_DEVICEis the correct direction. Use it explicitly inib_umem_get_cq_buf()andib_umem_get_cq_buf_or_va()rather than deriving it from the access flags: CQ callers passIB_ACCESS_LOCAL_WRITEso thatget_user_pages()pins a writable private page (FOLL_WRITE), which the device needs to DMA-write CQEs into. That flag would yieldDMA_BIDIRECTIONALfrom the access-flag derivation, which is wrong for the bus direction.Signed-off-by Yishai Hadas <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=6
Yishai Hadas
maintainerPATCH 7/15 RDMA/umem: Derive DMA direction from IB access flags
In reply to Yishai HadasMap non-writable user memory DMA_TO_DEVICE: when no write access is granted, the NIC only reads the pages and DMA_BIDIRECTIONAL is unnecessarily broad. Where the IOMMU enforces direction this prevents the device writing to read-only buffers; where it does not the mapping is still semantically correct.
--- drivers/infiniband/core/umem.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-)
drivers/infiniband/core/umem.c +10 -3
@@ -159,6 +159,11 @@ unsigned long ib_umem_find_best_pgsz(struct ib_umem *umem, }EXPORT_SYMBOL (ib_umem_find_best_pgsz); +static inline enum dma_data_direction ib_access_dma_dir (int access) +{ +return ib_access_writable (access) ?DMA_BIDIRECTIONAL :DMA_TO_DEVICE ; +} + static struct ib_umem *__ib_umem_get_va(struct ib_device *device, unsigned long addr, size_t size, int access,
@@ -329,7 +334,8 @@ struct ib_umem *ib_umem_get_desc(struct ib_device *device,const struct ib_uverbs_buffer_desc *desc, int access) { - return __ib_umem_get_desc_dir(device, desc, access, DMA_BIDIRECTIONAL); + return __ib_umem_get_desc_dir(device, desc, access, +ib_access_dma_dir (access )); }EXPORT_SYMBOL (ib_umem_get_desc);
@@ -482,7 +488,7 @@ struct ib_umem *ib_umem_get_attr(struct ib_device *device, u16 attr_id, size_t size, int access) { return ib_umem_get_from_attrs(device, attrs, attr_id, NULL, size, -access ,DMA_BIDIRECTIONAL ); +access ,ib_access_dma_dir (access )); }EXPORT_SYMBOL (ib_umem_get_attr);
@@ -521,7 +527,8 @@ struct ib_umem *ib_umem_get_attr_or_va(struct ib_device *device, int access) {return ib_umem_get_from_attrs_or_va (device, attrs, attr_id,NULL , addr, - size, access,DMA_BIDIRECTIONAL ); + size, access, +ib_access_dma_dir (access)); }EXPORT_SYMBOL (ib_umem_get_attr_or_va);
Signature · 1 line
2.18.1
[email protected]
AI reviewRe: [PATCH V1 rdma-next 07/15] RDMA/umem: Derive DMA direction from IB access flags
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 7 lines
Map non-writable user memory
DMA_TO_DEVICE: when no write access is granted, the NIC only reads the pages andDMA_BIDIRECTIONALis unnecessarily broad. Where the IOMMU enforces direction this prevents the device writing to read-only buffers; where it does not the mapping is still semantically correct.Signed-off-by Yishai Hadas <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=7
Yishai Hadas
maintainerPATCH 8/15 RDMA/mlx5: Fix mlx5_ib_dev_res_init() failure when XRC cap is absent
In reply to Yishai Hadasmlx5_ib_dev_res_init() returns -EOPNOTSUPP when the firmware XRC capability is absent, failing driver probe on XRC-less devices.
Make XRC optional throughout the devr resource path: initialize cq_lock and srq_lock unconditionally (needed for every QP creation), skip xrcd allocation and the XRC-type placeholder SRQ (s0) when xrc=0, and guard their cleanup paths accordingly.
--- drivers/infiniband/hw/mlx5/main.c | 56 +++++++++++++++++++------------ 1 file changed, 34 insertions(+), 22 deletions(-)
drivers/infiniband/hw/mlx5/main.c +34 -22
@@ -3373,7 +3373,7 @@ int mlx5_ib_dev_res_srq_init(struct mlx5_ib_dev *dev) {struct mlx5_ib_resources *devr = &dev->devr;struct ib_srq_init_attr attr; -struct ib_srq *s0, *s1; +struct ib_srq *s0 =NULL , *s1;int ret =0 ;/*
@@ -3391,19 +3391,27 @@ int mlx5_ib_dev_res_srq_init(struct mlx5_ib_dev *dev)if (ret) goto unlock; -memset (&attr,0 ,sizeof (attr)); - attr.attr.max_sge =1 ; - attr.attr.max_wr =1 ; - attr.srq_type =IB_SRQT_XRC ; - attr.ext.cq = devr->c0; - - s0 =ib_create_srq (devr->p0, &attr); -if (IS_ERR (s0)) { - ret =PTR_ERR (s0); -mlx5_ib_err (dev, -"Couldn't create SRQ 0 for res init, err=%pe\n" , - s0); - goto unlock; +/* +* s0 is an XRC-type placeholder SRQ used as the default XRQN for +* XRC QPs. Skip it when XRC is absent; all devr->s0 accesses in +* qp.c are inside XRC QP paths that the verbs layer blocks before +* reaching this driver when xrc=0, so NULL is safe. +*/ +if (MLX5_CAP_GEN (dev->mdev, xrc)) { +memset (&attr,0 ,sizeof (attr)); + attr.attr.max_sge =1 ; + attr.attr.max_wr =1 ; + attr.srq_type =IB_SRQT_XRC ; + attr.ext.cq = devr->c0; + + s0 =ib_create_srq (devr->p0, &attr); +if (IS_ERR (s0)) { + ret =PTR_ERR (s0); +mlx5_ib_err (dev, +"Couldn't create SRQ 0 for res init, err=%pe\n" , + s0); + goto unlock; + } }memset (&attr,0 ,sizeof (attr));
@@ -3417,7 +3425,8 @@ int mlx5_ib_dev_res_srq_init(struct mlx5_ib_dev *dev)mlx5_ib_err (dev,"Couldn't create SRQ 1 for res init, err=%pe\n" , s1); -ib_destroy_srq (s0); +if (s0) +ib_destroy_srq (s0); goto unlock; }
@@ -3434,8 +3443,11 @@ static int mlx5_ib_dev_res_init(struct mlx5_ib_dev *dev)struct mlx5_ib_resources *devr = &dev->devr;int ret; +mutex_init (&devr->cq_lock); +mutex_init (&devr->srq_lock); +if (!MLX5_CAP_GEN (dev->mdev, xrc)) -return -EOPNOTSUPP ; +return 0 ; ret =mlx5_cmd_xrcd_alloc (dev->mdev, &devr->xrcdn0,0 );if (ret)
@@ -3447,9 +3459,6 @@ static int mlx5_ib_dev_res_init(struct mlx5_ib_dev *dev)return ret; } -mutex_init (&devr->cq_lock); -mutex_init (&devr->srq_lock); -return 0 ; }
@@ -3460,10 +3469,13 @@ static void mlx5_ib_dev_res_cleanup(struct mlx5_ib_dev *dev)/* After s0/s1 init, they are not unset during the device lifetime. */ if (devr->s1) {ib_destroy_srq (devr->s1); -ib_destroy_srq (devr->s0); +if (devr->s0) +ib_destroy_srq (devr->s0); + } +if (MLX5_CAP_GEN (dev->mdev, xrc)) { +mlx5_cmd_xrcd_dealloc (dev->mdev, devr->xrcdn1,0 ); +mlx5_cmd_xrcd_dealloc (dev->mdev, devr->xrcdn0,0 ); } -mlx5_cmd_xrcd_dealloc (dev->mdev, devr->xrcdn1,0 ); -mlx5_cmd_xrcd_dealloc (dev->mdev, devr->xrcdn0,0 );/* After p0/c0 init, they are not unset during the device lifetime. */ if (devr->c0) {ib_destroy_cq (devr->c0);
Signature · 1 line
2.18.1
[email protected]
AI reviewRe: [PATCH V1 rdma-next 08/15] RDMA/mlx5: Fix mlx5_ib_dev_res_init() failure when XRC cap is absent
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] rdma/mlx5: missing mutex_destroy() on error path in mlx5_ib_dev_res_init() --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] rdma/mlx5: missing mutex_destroy() on error path in mlx5_ib_dev_res_init() --
commit c4290e2ba9444b4168574311394c1c28313c0387
Author: Yishai Hadas <[email protected]>
Subject: RDMA/mlx5: Fix mlx5_ib_dev_res_init() failure when XRC cap is absent
This commit makes XRC capabilities optional in the driver probe path. It initializes the cq_lock and srq_lock unconditionally at the start of device resource initialization and skips the xrcd allocation and placeholder SRQ when XRC is absent.
Quoted 4 lines
drivers/infiniband/hw/mlx5/main.c +0
-0
[ ... ]
Quoted 14 lines
drivers/infiniband/hw/mlx5/main.c +4
-1@@ -3434,8 +3443,11 @@ static int mlx5_ib_dev_res_init(struct mlx5_ib_dev *dev)struct mlx5_ib_resources *devr = &dev->devr;int ret; +mutex_init (&devr->cq_lock); +mutex_init (&devr->srq_lock); +if (!MLX5_CAP_GEN (dev->mdev, xrc)) -return -EOPNOTSUPP ; +return 0 ; ret =mlx5_cmd_xrcd_alloc (dev->mdev, &devr->xrcdn0,0 );if (ret)
drivers/infiniband/hw/mlx5/main.c static int mlx5_ib_dev_res_init(struct mlx5_ib_dev *dev) · line 3463
Quoted 8 lines
drivers/infiniband/hw/mlx5/main.c +0
-3@@ -3447,9 +3459,6 @@ static int mlx5_ib_dev_res_init(struct mlx5_ib_dev *dev)return ret; } -mutex_init (&devr->cq_lock); -mutex_init (&devr->srq_lock); -return 0 ; }
[Severity: Low]
Since cq_lock and srq_lock are now unconditionally initialized at the start of mlx5_ib_dev_res_init(), do we need to call mutex_destroy() on the error paths if mlx5_cmd_xrcd_alloc() fails?
If either xrcd allocation fails, the function returns directly without cleaning up the initialized mutexes. Could this missing cleanup break API symmetry and trigger warnings in debug builds when the device memory is subsequently freed?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=8
Yishai Hadas
maintainerPATCH 9/15 RDMA/mlx5: Put resource reference in mlx5_ib_wq_event()
In reply to Yishai Hadasrsc_event_notifier() acquires a resource reference before dispatching to the registered event callback and expects the callback to release it. mlx5_ib_wq_event() never called mlx5_core_res_put(), leaking one reference per event and preventing WQ destruction from completing.
--- drivers/infiniband/hw/mlx5/qp.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
drivers/infiniband/hw/mlx5/qp.c +3 -1
@@ -5344,11 +5344,13 @@ static void mlx5_ib_wq_event(struct mlx5_core_qp *core_qp, int type)break ;default :mlx5_ib_warn (dev,"Unexpected event type %d on WQ %06x\n" , type, core_qp->qpn); -return ; + goto out; } rwq->ibwq.event_handler (&event, rwq->ibwq.wq_context); } +out: +mlx5_core_res_put (&core_qp->common); }static int set_delay_drop (struct mlx5_ib_dev *dev)
Signature · 1 line
2.18.1
[email protected]
AI reviewRe: [PATCH V1 rdma-next 09/15] RDMA/mlx5: Put resource reference in mlx5_ib_wq_event()
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 7 lines
rsc_event_notifier()acquires a resource reference before dispatching to the registered event callback and expects the callback to release it.mlx5_ib_wq_event()never calledmlx5_core_res_put(), leaking one reference per event and preventing WQ destruction from completing.Fixes 350d0e4c7e4b ("IB/mlx5: Track asynchronous events on a receive work queue")Signed-off-by Yishai Hadas <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=9
Yishai Hadas
maintainerPATCH 10/15 RDMA/mlx5: Set WQ event handler before firmware RQ insertion
In reply to Yishai Hadascreate_rq() inserts the RQ into dev->qp_table.tree, making it visible to rsc_event_notifier() before rwq->core_qp.event was assigned. A firmware WQ_CATAS_ERROR event in that window finds a NULL event pointer.
Move the assignment before create_rq().
--- drivers/infiniband/hw/mlx5/qp.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/infiniband/hw/mlx5/qp.c +1 -1
@@ -5618,6 +5618,7 @@ struct ib_wq *mlx5_ib_create_wq(struct ib_pd *pd, rwq =kzalloc_obj (*rwq);if (!rwq)return ERR_PTR (-ENOMEM ); + rwq->core_qp.event = mlx5_ib_wq_event; err =prepare_user_rq (pd, init_attr, udata, rwq);if (err) goto err;
@@ -5641,7 +5642,6 @@ struct ib_wq *mlx5_ib_create_wq(struct ib_pd *pd, goto err_copy; } - rwq->core_qp.event = mlx5_ib_wq_event; rwq->ibwq.event_handler = init_attr->event_handler;return &rwq->ibwq;
Signature · 1 line
2.18.1
[email protected]
AI reviewRe: [PATCH V1 rdma-next 10/15] RDMA/mlx5: Set WQ event handler before firmware RQ insertion
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 8 lines
create_rq()inserts the RQ intodev->qp_table.tree, making it visible torsc_event_notifier()beforerwq->core_qp.eventwas assigned. A firmwareWQ_CATAS_ERRORevent in that window finds a NULL event pointer.Move the assignment before
create_rq().Fixes 350d0e4c7e4b ("IB/mlx5: Track asynchronous events on a receive work queue")Signed-off-by Yishai Hadas <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=10
Yishai Hadas
maintainerPATCH 11/15 RDMA/mlx5: Set SRQ event handler before xarray insertion
In reply to Yishai Hadasmlx5_cmd_create_srq() stores the SRQ into the xarray before srq->msrq.event was assigned. A firmware SRQ event arriving in that window calls through a NULL pointer.
Move the assignment before mlx5_cmd_create_srq().
--- drivers/infiniband/hw/mlx5/srq.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
drivers/infiniband/hw/mlx5/srq.c +1 -2
@@ -284,6 +284,7 @@ int mlx5_ib_create_srq(struct ib_srq *ib_srq, in.pd =to_mpd (ib_srq->pd)->pdn; in.db_record = srq->db.dma; + srq->msrq.event = mlx5_ib_srq_event; err =mlx5_cmd_create_srq (dev, &srq->msrq, &in);kvfree (in.pas);if (err) {
@@ -292,8 +293,6 @@ int mlx5_ib_create_srq(struct ib_srq *ib_srq, }mlx5_ib_dbg (dev,"create SRQ with srqn 0x%x\n" , srq->msrq.srqn); - - srq->msrq.event = mlx5_ib_srq_event; srq->ibsrq.ext.xrc.srq_num = srq->msrq.srqn;if (udata) {
Signature · 1 line
2.18.1
[email protected]
AI reviewRe: [PATCH V1 rdma-next 11/15] RDMA/mlx5: Set SRQ event handler before xarray insertion
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 8 lines
mlx5_cmd_create_srq()stores the SRQ into the xarray beforesrq->msrq.eventwas assigned. A firmware SRQ event arriving in that window calls through a NULL pointer.Move the assignment before
mlx5_cmd_create_srq().Fixes e126ba97dba9 ("mlx5: Add driver for Mellanox Connect-IB adapters")Signed-off-by Yishai Hadas <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=11
Yishai Hadas
maintainerPATCH 12/15 RDMA/mlx5: Set RQ event handler for raw-packet QP
In reply to Yishai Hadascreate_raw_packet_qp() creates a separate firmware RQ object but never assigned rq->base.mqp.event. Any firmware event on this resource (e.g. WQ_CATAS_ERROR, WQ_ACCESS_ERROR) calls through a NULL pointer.
Set the event pointer before the RQ is inserted into the resource table.
--- drivers/infiniband/hw/mlx5/qp.c | 1 + 1 file changed, 1 insertion(+)
drivers/infiniband/hw/mlx5/qp.c +1 -0
@@ -1647,6 +1647,7 @@ static int create_raw_packet_qp(struct mlx5_ib_dev *dev, struct mlx5_ib_qp *qp, } rq->base.container_mibqp = qp; + rq->base.mqp.event = mlx5_ib_qp_event;if (qp->flags &IB_QP_CREATE_CVLAN_STRIPPING ) rq->flags |=MLX5_IB_RQ_CVLAN_STRIPPING ;
Signature · 1 line
2.18.1
[email protected]
AI reviewRe: [PATCH V1 rdma-next 12/15] RDMA/mlx5: Set RQ event handler for raw-packet QP
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 8 lines
create_raw_packet_qp()creates a separate firmware RQ object but never assignedrq->base.mqp.event. Any firmware event on this resource (e.g.WQ_CATAS_ERROR,WQ_ACCESS_ERROR) calls through a NULL pointer.Set the event pointer before the RQ is inserted into the resource table.
Fixes 0fb2ed66a14c ("IB/mlx5: Add create and destroy functionality for Raw Packet QP")Signed-off-by Yishai Hadas <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=12
Yishai Hadas
maintainerPATCH 13/15 RDMA/mlx5: Set QP event handler before firmware QPC insertion
In reply to Yishai HadasFour QP creation paths assigned base->container_mibqp and base->mqp.event after the firmware create call that inserts the QP into dev->qp_table.tree. A firmware error EQE in that window reaches qp->event() with a NULL pointer.
Move both assignments before the firmware call.
Also set ibqp.qp_num inside mlx5_qpc_create_qp() before create_resource_common() so an EQE arriving early does not observe qp_num==0.
In addition, add a WARN_ON_ONCE(!qp->event) guard in rsc_event_notifier() as a safeguard for future regressions.
--- drivers/infiniband/hw/mlx5/qp.c | 22 ++++++++++------------ drivers/infiniband/hw/mlx5/qpc.c | 10 ++++++++++ 2 files changed, 20 insertions(+), 12 deletions(-)
drivers/infiniband/hw/mlx5/qp.c +10 -12
@@ -1624,6 +1624,8 @@ static int create_raw_packet_qp(struct mlx5_ib_dev *dev, struct mlx5_ib_qp *qp,if (err)return err; + sq->base.container_mibqp = qp; + sq->base.mqp.event = mlx5_ib_qp_event; err =create_raw_packet_qp_sq (dev, udata, attrs, sq, in, pd,to_mcq (init_attr->send_cq));if (err)
@@ -1635,9 +1637,6 @@ static int create_raw_packet_qp(struct mlx5_ib_dev *dev, struct mlx5_ib_qp *qp, resp->sqn = sq->base.mqp.qpn; resp->comp_mask |=MLX5_IB_CREATE_QP_RESP_MASK_SQN ; } - - sq->base.container_mibqp = qp; - sq->base.mqp.event = mlx5_ib_qp_event; }if (qp->rq.wqe_cnt) {
@@ -2093,13 +2092,13 @@ static int create_xrc_tgt_qp(struct mlx5_ib_dev *dev, struct mlx5_ib_qp *qp, } base = &qp->trans_qp.base; + base->container_mibqp = qp; + base->mqp.event = mlx5_ib_qp_event; err =mlx5_qpc_create_qp (dev, &base->mqp, in, inlen, out);kvfree (in);if (err)return err; - base->container_mibqp = qp; - base->mqp.event = mlx5_ib_qp_event;if (MLX5_CAP_GEN (mdev, ece_support)) params->resp.ece_options =MLX5_GET (create_qp_out, out, ece);
@@ -2237,14 +2236,14 @@ static int create_dci(struct mlx5_ib_dev *dev, struct ib_pd *pd, qp->flags &= ~IB_QP_CREATE_PCI_WRITE_END_PADDING ; } + base->container_mibqp = qp; + base->mqp.event = mlx5_ib_qp_event; err =mlx5_qpc_create_qp (dev, &base->mqp, in, inlen, out);kvfree (in);if (err) goto err_create; - base->container_mibqp = qp; - base->mqp.event = mlx5_ib_qp_event;if (MLX5_CAP_GEN (mdev, ece_support)) params->resp.ece_options =MLX5_GET (create_qp_out, out, ece);
@@ -2433,6 +2432,8 @@ static int create_user_qp(struct mlx5_ib_dev *dev, struct ib_pd *pd, qp->flags &= ~IB_QP_CREATE_PCI_WRITE_END_PADDING ; } + base->container_mibqp = qp; + base->mqp.event = mlx5_ib_qp_event;if (init_attr->qp_type ==IB_QPT_RAW_PACKET || qp->flags &IB_QP_CREATE_SOURCE_QPN ) { qp->raw_packet_qp.sq.ubuffer.buf_addr = ucmd->sq_buf_addr;
@@ -2447,8 +2448,6 @@ static int create_user_qp(struct mlx5_ib_dev *dev, struct ib_pd *pd,if (err) goto err_create; - base->container_mibqp = qp; - base->mqp.event = mlx5_ib_qp_event;if (MLX5_CAP_GEN (mdev, ece_support)) params->resp.ece_options =MLX5_GET (create_qp_out, out, ece);
@@ -2578,14 +2577,13 @@ static int create_kernel_qp(struct mlx5_ib_dev *dev, struct ib_pd *pd, MLX5_CAP_GEN(mdev, go_back_n))MLX5_SET (qpc, qpc, retry_mode,MLX5_QP_RM_GO_BACK_N ); + base->container_mibqp = qp; + base->mqp.event = mlx5_ib_qp_event; err =mlx5_qpc_create_qp (dev, &base->mqp, in, inlen, out);kvfree (in);if (err) goto err_create; - base->container_mibqp = qp; - base->mqp.event = mlx5_ib_qp_event; -get_cqs (qp->type, attr->send_cq, attr->recv_cq, &send_cq, &recv_cq);spin_lock_irqsave (&dev->reset_flow_resource_lock, flags);
drivers/infiniband/hw/mlx5/qpc.c +10 -0
@@ -146,6 +146,8 @@ static int rsc_event_notifier(struct notifier_block *nb,case MLX5_RES_RQ :case MLX5_RES_SQ : qp = (struct mlx5_core_qp *)common; +if (WARN_ON_ONCE (!qp->event)) + goto out; qp->event (qp, event_type);/* Need to put resource in event handler */ return NOTIFY_OK ;
@@ -258,6 +260,14 @@ int mlx5_qpc_create_qp(struct mlx5_ib_dev *dev, struct mlx5_core_qp *qp, qp->uid =MLX5_GET (create_qp_in, in, uid); qp->qpn =MLX5_GET (create_qp_out, out, qpn); +/* Set ibqp.qp_num before create_resource_common() inserts the QP into +* the radix tree and makes it visible to EQE processing. The +* assignment in create_qp() happens only after this function returns, +* leaving a window where an arriving EQE would observe qp_num == 0. +* create_qp() still overrides this with 0/1 for QP0/QP1, and sets it +* independently for DCT (which does not go through this function). +*/ +to_mibqp (qp)->ibqp.qp_num = qp->qpn; err =create_resource_common (dev, qp,MLX5_RES_QP );if (err)
Signature · 1 line
2.18.1
[email protected]
AI reviewRe: [PATCH V1 rdma-next 13/15] RDMA/mlx5: Set QP event handler before firmware QPC insertion
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 11 lines
Four QP creation paths assigned
base->container_mibqpandbase->mqp.eventafter the firmware create call that inserts the QP intodev->qp_table.tree. A firmware error EQE in that window reachesqp->event()with a NULL pointer.Move both assignments before the firmware call.
Also set ibqp.qp_num inside
mlx5_qpc_create_qp()beforecreate_resource_common()so an EQE arriving early does not observe qp_num==0.
Quoted 5 lines
In addition, add a
WARN_ON_ONCE(!qp->event)guard inrsc_event_notifier()as a safeguard for future regressions.Fixes e126ba97dba9 ("mlx5: Add driver for Mellanox Connect-IB adapters")Signed-off-by Yishai Hadas <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=13
Yishai Hadas
maintainerPATCH 14/15 RDMA/mlx5: Initialize QP/RQ/SQ resource refcount before publishing it
In reply to Yishai Hadascreate_resource_common() inserted the resource into the radix tree before its initialization was complete. A firmware EQE in that window finds a zero refcount_t and calls refcount_inc() on it; the refcount API treats this as a use-after-free, saturates the counter, and permanently breaks the resource's lifecycle tracking.
Move the initialization before the radix_tree_insert().
--- drivers/infiniband/hw/mlx5/qpc.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
drivers/infiniband/hw/mlx5/qpc.c +4 -4
@@ -167,6 +167,10 @@ static int create_resource_common(struct mlx5_ib_dev *dev,int err; qp->common.res = rsc_type; +refcount_set (&qp->common.refcount,1 ); +init_completion (&qp->common.free); + qp->pid = current->pid; +spin_lock_irq (&table->lock); err = radix_tree_insert(&table->tree, qp->qpn | (rsc_type << MLX5_USER_INDEX_LEN),
@@ -175,10 +179,6 @@ static int create_resource_common(struct mlx5_ib_dev *dev,if (err)return err; -refcount_set (&qp->common.refcount,1 ); -init_completion (&qp->common.free); - qp->pid = current->pid; -return 0 ; }
Signature · 1 line
2.18.1
[email protected]
AI reviewRe: [PATCH V1 rdma-next 14/15] RDMA/mlx5: Initialize QP/RQ/SQ resource refcount before publishing it
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 10 lines
create_resource_common()inserted the resource into the radix tree before its initialization was complete. A firmware EQE in that window finds a zerorefcount_tand callsrefcount_inc()on it; the refcount API treats this as a use-after-free, saturates the counter, and permanently breaks the resource's lifecycle tracking.Move the initialization before the
radix_tree_insert().Fixes e126ba97dba9 ("mlx5: Add driver for Mellanox Connect-IB adapters")Signed-off-by Yishai Hadas <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=14
Yishai Hadas
maintainerPATCH 15/15 RDMA/mlx5: Fix signed integer overflow in EQE qp_srq type shift
In reply to Yishai Hadaseqe->data.qp_srq.type is a u8. Shifting it left by MLX5_USER_INDEX_LEN
(24) without a cast promotes it to signed int, causing undefined behaviour for values >= 128.
All currently used type values are well below 128, so this does not manifest with current hardware. Cast to u32 before the shift to make the code well-defined across the full range.
--- drivers/infiniband/hw/mlx5/qpc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/infiniband/hw/mlx5/qpc.c +1 -1
@@ -128,7 +128,7 @@ static int rsc_event_notifier(struct notifier_block *nb,case MLX5_EVENT_TYPE_WQ_INVAL_REQ_ERROR :case MLX5_EVENT_TYPE_WQ_ACCESS_ERROR : rsn =be32_to_cpu (eqe->data.qp_srq.qp_srq_n) &0xffffff ; - rsn |= (eqe->data.qp_srq.type <<MLX5_USER_INDEX_LEN ); + rsn |= ((u32 )eqe->data.qp_srq.type <<MLX5_USER_INDEX_LEN );break ;default :return NOTIFY_DONE ;
Signature · 1 line
2.18.1
[email protected]
AI reviewRe: [PATCH V1 rdma-next 15/15] RDMA/mlx5: Fix signed integer overflow in EQE qp_srq type shift
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 9 lines
eqe->data.qp_srq.typeis a u8. Shifting it left byMLX5_USER_INDEX_LEN
(24) without a cast promotes it to signed int, causing undefined behaviour for values >= 128.All currently used type values are well below 128, so this does not manifest with current hardware. Cast to u32 before the shift to make the code well-defined across the full range.
Signed-off-by Yishai Hadas <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=15
Leon Romanovsky
AppliedQuoted 11 lines · Yishai Hadas
This series contains two independent groups of fixes for the RDMA subsystem.
Group 1: DMA direction correctness (patches 1-7).
RDMA drivers have historically mapped all pinned user buffers
DMA_BIDIRECTIONALregardless of how the device actually accesses them. This group tightens the DMA mapping direction to match the real access pattern for each buffer type:[...]
Applied, thanks!
[01/15] RDMA/erdma: Pin CQ buffer writable to match device DMA write access
https://git.kernel.org/rdma/rdma/c/ba3472ed6718bf [02/15] RDMA/hns: Pin CQ buffer writable to match device DMA write access https://git.kernel.org/rdma/rdma/c/ce29f6e741844a [03/15] RDMA/vmw_pvrdma: Pin QP and SRQ rings writable to match device DMA write access https://git.kernel.org/rdma/rdma/c/89abac4b60b708 [04/15] RDMA/umem: Reuse ib_umem_get_cq_buf_or_va() for VA-only CQ pinning https://git.kernel.org/rdma/rdma/c/b577999f6f8dbf [05/15] RDMA/umem: Support an explicit DMA direction other than DMA_BIDIRECTIONAL https://git.kernel.org/rdma/rdma/c/2445d74c465ac0 [06/15] RDMA/umem: Map CQ buffers DMA_FROM_DEVICE https://git.kernel.org/rdma/rdma/c/50b96c3e3feb61 [07/15] RDMA/umem: Derive DMA direction from IB access flags https://git.kernel.org/rdma/rdma/c/dfc5334af5667e [08/15] RDMA/mlx5: Fix mlx5_ib_dev_res_init() failure when XRC cap is absent https://git.kernel.org/rdma/rdma/c/4354ab7a85d1dd [09/15] RDMA/mlx5: Put resource reference in mlx5_ib_wq_event() (no commit info) [10/15] RDMA/mlx5: Set WQ event handler before firmware RQ insertion https://git.kernel.org/rdma/rdma/c/28c559d1e25de4 [11/15] RDMA/mlx5: Set SRQ event handler before xarray insertion https://git.kernel.org/rdma/rdma/c/d2aa15620e98dc [12/15] RDMA/mlx5: Set RQ event handler for raw-packet QP https://git.kernel.org/rdma/rdma/c/a4fa6497002c80 [13/15] RDMA/mlx5: Set QP event handler before firmware QPC insertion https://git.kernel.org/rdma/rdma/c/05a078c97eb0bb [14/15] RDMA/mlx5: Initialize QP/RQ/SQ resource refcount before publishing it https://git.kernel.org/rdma/rdma/c/829a0e65deff3f [15/15] RDMA/mlx5: Fix signed integer overflow in EQE qp_srq type shift https://git.kernel.org/rdma/rdma/c/86959c7d009f60
Best regards,
Signature · 1 line
Leon Romanovsky <[email protected]>
Quoted 2 lines · Yishai Hadas
This series contains two independent groups of fixes for the RDMA subsystem.
<...>
Quoted 3 lines
drivers/infiniband/core/umem.c | 77 +++++++++++++------ drivers/infiniband/hw/bnxt_re/ib_verbs.c | 9 ++- drivers/infiniband/hw/erdma/erdma_verbs.c | 19 +++--
erdma had a few merge conflicts. I resolved them, but please review my changes.
Thanks