RDMA/mana_ib: return PD number to the user
PATCH v3 1/1 RDMA/mana_ib: return PD number to the user
From: Konstantin Taranov <[email protected]>
Implement returning to userspace applications PDNs of created PDs. The PDN is used by applications that build work requests outside of the rdma-core code base. The PDN is used to build work requests that require mentioning the PD. The MANA HW still ensures PD isolation using PDN attached to MRs and WRs, therefore the PDN mentioned in the work request must match the PDN of the used work queue. The work requests can fit only 16 bit PDNs. Allow users to request short PDNs which are 16 bits.
---
v3: updated commit message
v2: updated commit message
drivers/infiniband/hw/mana/main.c | 19 ++++++++++++++++++- include/net/mana/gdma.h | 4 ++-- include/uapi/rdma/mana-abi.h | 14 ++++++++++++++ 3 files changed, 34 insertions(+), 3 deletions(-)
drivers/infiniband/hw/mana/main.c +18 -1
@@ -69,9 +69,11 @@ int mana_ib_cfg_vport(struct mana_ib_dev *dev, u32 port, struct mana_ib_pd *pd,int mana_ib_alloc_pd (struct ib_pd *ibpd,struct ib_udata *udata) { struct mana_ib_pd *pd = container_of(ibpd, struct mana_ib_pd, ibpd); +struct mana_ib_alloc_pd_resp cmd_resp = {};struct ib_device *ibdev = ibpd->device;struct gdma_create_pd_resp resp = {};struct gdma_create_pd_req req = {}; +struct mana_ib_alloc_pd ucmd = {};enum gdma_pd_flags flags =0 ;struct mana_ib_dev *dev;struct gdma_context *gc;
@@ -83,8 +85,15 @@ int mana_ib_alloc_pd(struct ib_pd *ibpd, struct ib_udata *udata)mana_gd_init_req_hdr (&req.hdr,GDMA_CREATE_PD ,sizeof (req),sizeof (resp)); -if (!udata) +if (!udata) { flags |=GDMA_PD_FLAG_ALLOW_GPA_MR ; + }else { + err =ib_copy_from_udata (&ucmd, udata,min (sizeof (ucmd), udata->inlen)); +if (err) +return err; +if (ucmd.flags &MANA_IB_PD_SHORT_PDN ) + flags |=GDMA_PD_FLAG_SHORT_PDN ; + } req.flags = flags; err = mana_gd_send_request(gc, sizeof(req), &req,
@@ -107,6 +116,14 @@ int mana_ib_alloc_pd(struct ib_pd *ibpd, struct ib_udata *udata)mutex_init (&pd->vport_mutex); pd->vport_use_count =0 ; + +if (udata) { + cmd_resp.pdn = resp.pd_id; + err =ib_copy_to_udata (udata, &cmd_resp,min (sizeof (cmd_resp), udata->outlen)); +if (err) +return err; + } +return 0 ; }
include/net/mana/gdma.h +2 -2
@@ -824,8 +824,8 @@ struct gdma_destroy_dma_region_req { };/* HW DATA */ enum gdma_pd_flags { -GDMA_PD_FLAG_INVALID =0 , -GDMA_PD_FLAG_ALLOW_GPA_MR =1 , +GDMA_PD_FLAG_ALLOW_GPA_MR =BIT (0 ), +GDMA_PD_FLAG_SHORT_PDN =BIT (2 ), };struct gdma_create_pd_req {
include/uapi/rdma/mana-abi.h +14 -0
@@ -87,4 +87,18 @@ struct mana_ib_create_qp_rss_resp { struct rss_resp_entry entries[64]; }; +enum mana_ib_create_pd_flags { +MANA_IB_PD_SHORT_PDN =1 <<0 , +}; + +struct mana_ib_alloc_pd { +__u32 flags; +__u32 reserved; +}; + +struct mana_ib_alloc_pd_resp { +__u32 pdn; +__u32 reserved; +}; +#endif
Signature · 1 line
2.43.0
Quoted 9 lines · Konstantin Taranov
From: Konstantin Taranov <[email protected]>
Implement returning to userspace applications PDNs of created PDs. The PDN is used by applications that build work requests outside of the rdma-core code base. The PDN is used to build work requests that require mentioning the PD. The MANA HW still ensures PD isolation using PDN attached to MRs and WRs, therefore the PDN mentioned in the work request must match the PDN of the used work queue. The work requests can fit only 16 bit PDNs. Allow users to request short PDNs which are 16 bits.
Okay, if you say so.. What a crazy thing to do, and not enough bits to boot.
Anyhow, same comment as broadcom, the driver has to implement the driver data forwards/back compat protocol properly before you can add new drvdata extensions.
ie if userspace sends MANA_IB_PD_SHORT_PDN the current kernels will just ignore it without EOPNOTSUPP, that is not OK.
So you need to do the mana version of this series:
https://lore.kernel.org/linux-rdma/[email protected]/
And I have already written alot of the required mana parts here:
https://github.com/jgunthorpe/linux/commits/rdma_uapi/
Feel free to pull them out and make mana changes matching the broadcom ones:
RDMA/bnxt_re: Add compatibility checks to the uapi path RDMA/bnxt_re: Add compatibility checks to the uapi path for no data RDMA/bnxt_re: Add missing comp_mask validation RDMA/bnxt_re: Use ib_respond_udata()
RDMA/bnxt_re: Add BNXT_RE_UCNTX_CMASK_UAPI_COMPAT_SUPPORTED
We can take it after the merge window closes in two weeks.
Jason