RDMA/vmw_pvrdma: Fix network_hdr_type reported in WC
Subsystems (2)
PATCH 1/1 RDMA/vmw_pvrdma: Fix network_hdr_type reported in WC
The PVRDMA device defines network_hdr_type according to an old definition of the rdma_network_type enum that has since changed, resulting in the wrong rdma_network_type being reported. Fix this by explicitly defining the enum used by the PVRDMA device and adding a function to convert the pvrdma_network_type to rdma_network_type enum.
--- drivers/infiniband/hw/vmw_pvrdma/pvrdma.h | 14 ++++++++++++++ drivers/infiniband/hw/vmw_pvrdma/pvrdma_cq.c | 2 +- drivers/infiniband/hw/vmw_pvrdma/pvrdma_dev_api.h | 7 +++++++ 3 files changed, 22 insertions(+), 1 deletion(-)
drivers/infiniband/hw/vmw_pvrdma/pvrdma.h +14 -0
@@ -509,6 +509,20 @@ static inline int ib_send_flags_to_pvrdma(int flags)return flags &PVRDMA_MASK (PVRDMA_SEND_FLAGS_MAX ); } +static inline int pvrdma_network_type_to_ib (enum pvrdma_network_type type) +{ +switch (type) { +case PVRDMA_NETWORK_ROCE_V1 : +return RDMA_NETWORK_ROCE_V1 ; +case PVRDMA_NETWORK_IPV4 : +return RDMA_NETWORK_IPV4 ; +case PVRDMA_NETWORK_IPV6 : +return RDMA_NETWORK_IPV6 ; +default : +return RDMA_NETWORK_IPV6 ; + } +} +void pvrdma_qp_cap_to_ib (struct ib_qp_cap *dst,const struct pvrdma_qp_cap *src); void ib_qp_cap_to_pvrdma(struct pvrdma_qp_cap *dst,
drivers/infiniband/hw/vmw_pvrdma/pvrdma_cq.c +1 -1
@@ -367,7 +367,7 @@ static int pvrdma_poll_one(struct pvrdma_cq *cq, struct pvrdma_qp **cur_qp, wc->dlid_path_bits = cqe->dlid_path_bits; wc->port_num = cqe->port_num; wc->vendor_err = cqe->vendor_err; - wc->network_hdr_type = cqe->network_hdr_type; + wc->network_hdr_type =pvrdma_network_type_to_ib (cqe->network_hdr_type);/* Update shared ring state */ pvrdma_idx_ring_inc (&cq->ring_state->rx.cons_head, cq->ibcq.cqe);
drivers/infiniband/hw/vmw_pvrdma/pvrdma_dev_api.h +7 -0
@@ -201,6 +201,13 @@ enum pvrdma_device_mode { PVRDMA_DEVICE_MODE_IB, /* InfiniBand. */ }; +enum pvrdma_network_type { +PVRDMA_NETWORK_IB , +PVRDMA_NETWORK_ROCE_V1 =PVRDMA_NETWORK_IB , +PVRDMA_NETWORK_IPV4 , +PVRDMA_NETWORK_IPV6 +}; +struct pvrdma_gos_info {u32 gos_bits:2 ;/* W: PVRDMA_GOS_BITS_ */ u32 gos_type:4 ;/* W: PVRDMA_GOS_TYPE_ */
Signature · 1 line
2.14.1
Jason Gunthorpe
maintainerQuoted 5 lines · Bryan Tan
The PVRDMA device defines
network_hdr_typeaccording to an old definition of therdma_network_typeenum that has since changed, resulting in the wrongrdma_network_typebeing reported. Fix this by explicitly defining the enum used by the PVRDMA device and adding a function to convert thepvrdma_network_typetordma_network_typeenum.
How come I can't find anything reading this in rdma-core?
$ ~/oss/rdma-core#git grep network_hdr_type kernel-headers/rdma/vmw_pvrdma-abi.h: __u8 network_hdr_type;
??
Quoted 2 lines
Fixes 1c15b4f2a42f ("RDMA/core: Modify enum ib_gid_type and enum rdma_network_type")Signed-off-by Bryan Tan <[email protected]>
Add Cc: [email protected] # 5.10+
Quoted 5 lines
drivers/infiniband/hw/vmw_pvrdma/pvrdma_dev_api.h +0
-0
drivers/infiniband/hw/vmw_pvrdma/pvrdma_dev_api.h enum pvrdma_device_mode { · line 209
Quoted 9 lines
drivers/infiniband/hw/vmw_pvrdma/pvrdma_dev_api.h +6
-0@@ -201,6 +201,13 @@ enum pvrdma_device_mode { PVRDMA_DEVICE_MODE_IB, /* InfiniBand. */ }; +enum pvrdma_network_type { +PVRDMA_NETWORK_IB , +PVRDMA_NETWORK_ROCE_V1 =PVRDMA_NETWORK_IB , +PVRDMA_NETWORK_IPV4 , +PVRDMA_NETWORK_IPV6 +};
This is in the wrong place, uapi data needs to be in includ/ulp/rdma/vmw_pvrdma-abi.h
Jason
Quoted 9 lines
From: Jason Gunthorpe <[email protected]>
Sent: Friday, January 15, 2021 1:24 AMQuoted 5 lines · Bryan Tan
The PVRDMA device defines
network_hdr_typeaccording to an old definition of therdma_network_typeenum that has since changed, resulting in the wrongrdma_network_typebeing reported. Fix this by explicitly defining the enum used by the PVRDMA device and adding a function to convert thepvrdma_network_typetordma_network_typeenum.
Quoted 6 lines
How come I can't find anything reading this in rdma-core?
$ ~/oss/rdma-core#git grep network_hdr_type kernel-headers/rdma/vmw_pvrdma-abi.h: __u8 network_hdr_type;??
network_hdr_type isn't exposed in the userspace WC ibv_wc. Given that the field is only in the kernel side, it didn't seem like we should add the new enum to vmw_pvrdma-abi.h either.
Quoted 4 lines
Quoted 2 lines
Fixes 1c15b4f2a42f ("RDMA/core: Modify enum ib_gid_type and enum rdma_network_type")Signed-off-by Bryan Tan <[email protected]>Add Cc: [email protected] # 5.10+
Will do.
Quoted 8 lines
Quoted 8 lines
drivers/infiniband/hw/vmw_pvrdma/pvrdma_dev_api.h +0
-0@@ -201,6 +201,13 @@ enum pvrdma_device_mode { PVRDMA_DEVICE_MODE_IB, /* InfiniBand. */ };
Quoted 9 lines
drivers/infiniband/hw/vmw_pvrdma/pvrdma_dev_api.h enum pvrdma_device_mode { · line 209
Quoted 6 lines
drivers/infiniband/hw/vmw_pvrdma/pvrdma_dev_api.h +6
-0+enum pvrdma_network_type { +PVRDMA_NETWORK_IB , +PVRDMA_NETWORK_ROCE_V1 =PVRDMA_NETWORK_IB , +PVRDMA_NETWORK_IPV4 , +PVRDMA_NETWORK_IPV6 +};This is in the wrong place, uapi data needs to be in
includ/ulp/rdma/vmw_pvrdma-abi.h
Please see above.
Thanks,
Bryan
Jason Gunthorpe
maintainerQuoted 9 lines · Bryan Tan
Quoted 9 lines
From: Jason Gunthorpe <[email protected]>
Sent: Friday, January 15, 2021 1:24 AMQuoted 5 lines · Bryan Tan
The PVRDMA device defines
network_hdr_typeaccording to an old definition of therdma_network_typeenum that has since changed, resulting in the wrongrdma_network_typebeing reported. Fix this by explicitly defining the enum used by the PVRDMA device and adding a function to convert thepvrdma_network_typetordma_network_typeenum.
Quoted 8 lines
Quoted 6 lines
How come I can't find anything reading this in rdma-core?
$ ~/oss/rdma-core#git grep network_hdr_type kernel-headers/rdma/vmw_pvrdma-abi.h: __u8 network_hdr_type;??
network_hdr_typeisn't exposed in the userspace WCibv_wc.
So this is "HW" API then?
Quoted 2 lines
Given that the field is only in the kernel side, it didn't seem like we should add the new enum to
vmw_pvrdma-abi.heither.
Well, the struct that holds the value is in a uapi header, so the definition should be too. If you are defining HW data in uapi then may as well define all of it.
Jason
From: Jason Gunthorpe <[email protected]>
Sent: Friday, January 15, 2021 8:51 PM
Quoted 17 lines · Bryan Tan
Quoted 16 lines
Quoted 15 lines
From: Jason Gunthorpe <[email protected]>
Sent: Friday, January 15, 2021 1:24 AMQuoted 5 lines · Bryan Tan
The PVRDMA device defines
network_hdr_typeaccording to an old definition of therdma_network_typeenum that has since changed, resulting in the wrongrdma_network_typebeing reported. Fix this by explicitly defining the enum used by the PVRDMA device and adding a function to convert thepvrdma_network_typetordma_network_typeenum.How come I can't find anything reading this in rdma-core?
$ ~/oss/rdma-core#git grep network_hdr_type kernel-headers/rdma/vmw_pvrdma-abi.h: __u8 network_hdr_type;??
Quoted 7 lines
Quoted 1 line
network_hdr_typeisn't exposed in the userspace WCibv_wc.So this is "HW" API then?
<snip>
Well, the struct that holds the value is in a uapi header, so the definition should be too. If you are defining HW data in uapi then may as well define all of it.
Yes, the pvrdma_cqe struct is populated by the HW. Both the driver and the userspace library use this struct to populate the corresponding WCs. That makes sense, let me move them into the uapi header. Apologies if this is answered somewhere, couldn't find any info--am I responsible for updating the header in rdma-core then?
Bryan
Jason Gunthorpe
maintainerQuoted 5 lines · Bryan Tan
Yes, the
pvrdma_cqestruct is populated by the HW. Both the driver and the userspace library use this struct to populate the corresponding WCs. That makes sense, let me move them into the uapi header. Apologies if this is answered somewhere, couldn't find any info--am I responsible for updating the header in rdma-core then?
It happens automatically unless you need to send a related rdma-core github PR, then you should do it in the PR
Jason