RDMA/mana_ib: UC QP support for UAPI
Subsystems (2)
PATCH v3 1/1 RDMA/mana_ib: UC QP support for UAPI
From: Konstantin Taranov <[email protected]>
Implement UC QP creation in the RNIC HW for user API. An UC QP is exposed as three work queues: send, receive, and memory management. The latter is used for bind and invalidate WQEs to support memory windows.
---
It is a resend of v3
v3: use new udata helpers
v2: removed udata check and removed enum from mana-abi.h
drivers/infiniband/hw/mana/main.c | 41 ++++++++++++- drivers/infiniband/hw/mana/mana_ib.h | 41 ++++++++++++- drivers/infiniband/hw/mana/qp.c | 89 ++++++++++++++++++++++++++-- include/uapi/rdma/mana-abi.h | 11 ++++ 4 files changed, 173 insertions(+), 9 deletions(-)
drivers/infiniband/hw/mana/main.c +40 -1
@@ -978,7 +978,46 @@ int mana_ib_gd_create_rc_qp(struct mana_ib_dev *mdev, struct mana_ib_qp *qp,return 0 ; } -int mana_ib_gd_destroy_rc_qp (struct mana_ib_dev *mdev,struct mana_ib_qp *qp) +int mana_ib_gd_create_uc_qp (struct mana_ib_dev *mdev,struct mana_ib_qp *qp, +struct ib_qp_init_attr *attr,u32 doorbell,u64 flags) +{ + struct mana_ib_cq *send_cq = container_of(qp->ibqp.send_cq, struct mana_ib_cq, ibcq); + struct mana_ib_cq *recv_cq = container_of(qp->ibqp.recv_cq, struct mana_ib_cq, ibcq); + struct mana_ib_pd *pd = container_of(qp->ibqp.pd, struct mana_ib_pd, ibpd); +struct gdma_context *gc =mdev_to_gc (mdev); +struct mana_rnic_create_uc_qp_resp resp = {}; +struct mana_rnic_create_uc_qp_req req = {}; +int err, i; + +mana_gd_init_req_hdr (&req.hdr,MANA_IB_CREATE_UC_QP ,sizeof (req),sizeof (resp)); + req.hdr.dev_id = mdev->gdma_dev->dev_id; + req.adapter = mdev->adapter_handle; + req.pd_handle = pd->pd_handle; + req.send_cq_handle = send_cq->cq_handle; + req.recv_cq_handle = recv_cq->cq_handle; +for (i =0 ; i <MANA_UC_QUEUE_TYPE_MAX ; i++) + req.dma_region[i] = qp->uc_qp.queues[i].gdma_region; + req.doorbell_page = doorbell; + req.max_send_wr = attr->cap.max_send_wr; + req.max_recv_wr = attr->cap.max_recv_wr; + req.max_send_sge = attr->cap.max_send_sge; + req.max_recv_sge = attr->cap.max_recv_sge; + req.flags = flags; + + err =mana_gd_send_request (gc,sizeof (req), &req,sizeof (resp), &resp); +if (err) +return err; + + qp->qp_handle = resp.qp_handle; +for (i =0 ; i <MANA_UC_QUEUE_TYPE_MAX ; i++) { + qp->uc_qp.queues[i].id = resp.queue_ids[i]; +/* The GDMA regions are now owned by the RNIC QP handle */ + qp->uc_qp.queues[i].gdma_region =GDMA_INVALID_DMA_REGION ; + } +return 0 ; +} + +int mana_ib_gd_destroy_rnic_qp (struct mana_ib_dev *mdev,struct mana_ib_qp *qp) {struct mana_rnic_destroy_rc_qp_resp resp = {0 };struct mana_rnic_destroy_rc_qp_req req = {0 };
drivers/infiniband/hw/mana/mana_ib.h +39 -2
71 lines not on this page · raw message
drivers/infiniband/hw/mana/qp.c +83 -6
154 lines not on this page · raw message
include/uapi/rdma/mana-abi.h +11 -0
@@ -57,6 +57,17 @@ struct mana_ib_create_rc_qp_resp {__u32 queue_id[4 ]; }; +struct mana_ib_create_uc_qp { +__aligned_u64 queue_buf[3 ]; +__u32 queue_size[3 ]; +__u32 reserved; +}; + +struct mana_ib_create_uc_qp_resp { +__u32 queue_id[3 ]; +__u32 reserved; +}; +struct mana_ib_create_wq {__aligned_u64 wq_buf_addr;__u32 wq_buf_size;
Signature · 1 line
2.43.0
Review
Quoted 2 lines
(quoted diff) +2
-0+int mana_ib_gd_destroy_rnic_qp(struct mana_ib_dev *mdev, struct +mana_ib_qp *qp)
The function is renamed to _rnic_ to be shared between RC and UC, but
the request/response structs are still named _rc_. Should these be renamed to _rnic_ as well for consistency? Or does the firmware use the same destroy command for both QP types?
Quoted 3 lines
{
(quoted diff) +0
-0struct mana_rnic_destroy_rc_qp_resp resp = {0}; struct mana_rnic_destroy_rc_qp_req req = {0}; diff --git
Konstantin Taranov
maintainerQuoted 7 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+int mana_ib_gd_destroy_rnic_qp(struct mana_ib_dev *mdev, struct +mana_ib_qp *qp)The function is renamed to _rnic_ to be shared between RC and UC, but the request/response structs are still named _rc_. Should these be renamed to _rnic_ as well for consistency? Or does the firmware use the same destroy command for both QP types?
Sure, they can be renamed. In the firmware, it is still called RC though, but it can be used for all QP types (even UD). So, the name _rc_ is just historical.
Long, do you want me to rename it?
Jason, can the patch series accepted, or do I need to send IB_UVERBS_CORE_SUPPORT_ROBUST_UDATA first?
Thanks
Quoted 4 lines
Quoted 4 lines
{
(quoted diff) +0
-0struct mana_rnic_destroy_rc_qp_resp resp = {0}; struct mana_rnic_destroy_rc_qp_req req = {0}; diff --git
Quoted 8 lines · Konstantin Taranov
Quoted 7 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+int mana_ib_gd_destroy_rnic_qp(struct mana_ib_dev *mdev, struct +mana_ib_qp *qp)The function is renamed to _rnic_ to be shared between RC and UC, but the request/response structs are still named _rc_. Should these be renamed to _rnic_ as well for consistency? Or does the firmware use the same destroy command for both QP types?
Quoted 7 lines
Sure, they can be renamed. In the firmware, it is still called RC though, but it can be used for all QP types (even UD). So, the name _rc_ is just historical.
Long, do you want me to rename it?
Jason, can the patch series accepted, or do I need to send
IB_UVERBS_CORE_SUPPORT_ROBUST_UDATAfirst?
Yeah, that's what I want to start seeing
And use the proper helpers too:
+ err = ib_copy_to_udata(udata, &resp, min(sizeof(resp), udata->outlen));
+ if (err)
+ goto destroy_qp;
Thats ib_respond_udata() now too
Jason
Quoted 11 lines
Subject: RE: [PATCH rdma-next v3 1/1] RDMA/mana_ib: UC QP support for UAPI
Quoted 7 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+int mana_ib_gd_destroy_rnic_qp(struct mana_ib_dev *mdev, struct +mana_ib_qp *qp)The function is renamed to _rnic_ to be shared between RC and UC, but the request/response structs are still named _rc_. Should these be renamed to _rnic_ as well for consistency? Or does the firmware use the same destroy command for both QP types?
Quoted 4 lines
Sure, they can be renamed. In the firmware, it is still called RC though, but it can be used for all QP types (even UD). So, the name _rc_ is just historical.
Long, do you want me to rename it?
Yes, it's better to rename them to avoid confusions.
Thanks,
Long