RDMA/mana_ib: return PD number to the user
PATCH 1/1 RDMA/mana_ib: return PD number to the user
From: Konstantin Taranov <[email protected]>
Implement returning to userspace applications PDNs of created PDs. Allow users to request short PDNs which are 16 bits.
--- drivers/infiniband/hw/mana/main.c | 19 ++++++++++++++++++- include/net/mana/gdma.h | 4 ++-- include/uapi/rdma/mana-abi.h | 14 ++++++++++++++ 3 files changed, 34 insertions(+), 3 deletions(-)
drivers/infiniband/hw/mana/main.c +18 -1
@@ -69,9 +69,11 @@ int mana_ib_cfg_vport(struct mana_ib_dev *dev, u32 port, struct mana_ib_pd *pd,int mana_ib_alloc_pd (struct ib_pd *ibpd,struct ib_udata *udata) { struct mana_ib_pd *pd = container_of(ibpd, struct mana_ib_pd, ibpd); +struct mana_ib_alloc_pd_resp cmd_resp = {};struct ib_device *ibdev = ibpd->device;struct gdma_create_pd_resp resp = {};struct gdma_create_pd_req req = {}; +struct mana_ib_alloc_pd ucmd = {};enum gdma_pd_flags flags =0 ;struct mana_ib_dev *dev;struct gdma_context *gc;
@@ -83,8 +85,15 @@ int mana_ib_alloc_pd(struct ib_pd *ibpd, struct ib_udata *udata)mana_gd_init_req_hdr (&req.hdr,GDMA_CREATE_PD ,sizeof (req),sizeof (resp)); -if (!udata) +if (!udata) { flags |=GDMA_PD_FLAG_ALLOW_GPA_MR ; + }else { + err =ib_copy_from_udata (&ucmd, udata,min (sizeof (ucmd), udata->inlen)); +if (err) +return err; +if (ucmd.flags &MANA_IB_PD_SHORT_PDN ) + flags |=GDMA_PD_FLAG_SHORT_PDN ; + } req.flags = flags; err = mana_gd_send_request(gc, sizeof(req), &req,
@@ -107,6 +116,14 @@ int mana_ib_alloc_pd(struct ib_pd *ibpd, struct ib_udata *udata)mutex_init (&pd->vport_mutex); pd->vport_use_count =0 ; + +if (udata) { + cmd_resp.pdn = resp.pd_id; + err =ib_copy_to_udata (udata, &cmd_resp,min (sizeof (cmd_resp), udata->outlen)); +if (err) +return err; + } +return 0 ; }
include/net/mana/gdma.h +2 -2
@@ -824,8 +824,8 @@ struct gdma_destroy_dma_region_req { };/* HW DATA */ enum gdma_pd_flags { -GDMA_PD_FLAG_INVALID =0 , -GDMA_PD_FLAG_ALLOW_GPA_MR =1 , +GDMA_PD_FLAG_ALLOW_GPA_MR =BIT (0 ), +GDMA_PD_FLAG_SHORT_PDN =BIT (2 ), };struct gdma_create_pd_req {
include/uapi/rdma/mana-abi.h +14 -0
@@ -87,4 +87,18 @@ struct mana_ib_create_qp_rss_resp { struct rss_resp_entry entries[64]; }; +enum mana_ib_create_pd_flags { +MANA_IB_PD_SHORT_PDN =1 <<0 , +}; + +struct mana_ib_alloc_pd { +__u32 flags; +__u32 reserved; +}; + +struct mana_ib_alloc_pd_resp { +__u32 pdn; +__u32 reserved; +}; +#endif
Signature · 1 line
2.43.0
Quoted 4 lines · Konstantin Taranov
From: Konstantin Taranov <[email protected]>
Implement returning to userspace applications PDNs of created PDs. Allow users to request short PDNs which are 16 bits.
Why does userspace ever need to see a PDN? Please justify that in the commit message
Jason
Quoted 8 lines · Jason Gunthorpe
Quoted 4 lines · Konstantin Taranov
From: Konstantin Taranov <[email protected]>
Implement returning to userspace applications PDNs of created PDs. Allow users to request short PDNs which are 16 bits.
Why does userspace ever need to see a PDN? Please justify that in the commit message
Probably for the debug and we have restrack for it.
Thanks
Quoted 2 lines
Jason
Konstantin Taranov
maintainerQuoted 13 lines · Jason Gunthorpe
Quoted 8 lines · Konstantin Taranov
Quoted 4 lines
From: Konstantin Taranov <[email protected]>
Implement returning to userspace applications PDNs of created PDs. Allow users to request short PDNs which are 16 bits.
Why does userspace ever need to see a PDN? Please justify that in the commit message
Probably for the debug and we have restrack for it.
Sure, I will add the explanation in v2. Overall, it is for applications working on top of the rdma-core (e.g., mana DPDK). The use-case is similar to what mlx4 and mthca have for address vectors in rdma-core for isolation.
As the whole process of working with WQs and CQs is implemented in that applications (e.g., mana DPDK), they need to know PDN to build a correct request. What is more, the HW folks put a limit of 16 bits to the PDN field, requiring a flag to ensure that we get a PDN that fits into the field.
I hope that it justifies the change as most ib providers have pdn in the user-space.
- Konstantin
Quoted 4 lines
Thanks
Quoted 2 lines
Jason
Quoted 23 lines · Konstantin Taranov
Quoted 13 lines · Jason Gunthorpe
Quoted 8 lines · Konstantin Taranov
Quoted 4 lines
From: Konstantin Taranov <[email protected]>
Implement returning to userspace applications PDNs of created PDs. Allow users to request short PDNs which are 16 bits.
Why does userspace ever need to see a PDN? Please justify that in the commit message
Probably for the debug and we have restrack for it.
Sure, I will add the explanation in v2. Overall, it is for applications working on top of the rdma-core (e.g., mana DPDK). The use-case is similar to what mlx4 and mthca have for address vectors in rdma-core for isolation. As the whole process of working with WQs and CQs is implemented in that applications (e.g., mana DPDK), they need to know PDN to build a correct request. What is more, the HW folks put a limit of 16 bits to the PDN field, requiring a flag to ensure that we get a PDN that fits into the field.
Quoted 2 lines
I hope that it justifies the change as most ib providers have pdn in the user-space.
But they don't put it in a WQE and don't check in HW it is exactly the same as the PDN the WQ already has.
You have PDs in AHs and other related objects which make sense, but a WQ only has one PD, it is illogical to pass in a PDN in a WQE, because it can never take on a different value.
If it can take on a different value then your HW's security model is broken.
Jason