RDMA/irdma: Make the source udp port vary
Subsystems (2)
PATCH v3 1/1 RDMA/irdma: Make the source udp port vary
From: Zhu Yanjun <[email protected]>
Based on the link https://www.spinics.net/lists/linux-rdma/msg73735.html,
get the source udp port number for a QP based on the grh.flow_label or lqpn/rqrpn. This provides a better spread of traffic across NIC RX queues. The method in the commit 2b880b2e5e03 ("RDMA/mlx5: Define RoCEv2 udp source port when set path") is a standard way. So it is also adopted in this commit.
---V2->V3: Move to the block of IB_QP_AV in the mask and IB_AH_GRH in ah_flags V1->V2: Adopt a standard method to get udp source port.
--- drivers/infiniband/hw/irdma/verbs.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
drivers/infiniband/hw/irdma/verbs.c +14 -0
@@ -1094,6 +1094,15 @@ static int irdma_query_pkey(struct ib_device *ibdev, u32 port, u16 index,return 0 ; } + +static u16 irdma_get_udp_sport (u32 fl,u32 lqpn,u32 rqpn) +{ +if (!fl) + fl =rdma_calc_flow_label (lqpn, rqpn); + +return rdma_flow_label_to_udp_sport (fl); +} +/** * irdma_modify_qp_roce - modify qp request * @ibqp: qp's pointer for modify
@@ -1167,6 +1176,11 @@ int irdma_modify_qp_roce(struct ib_qp *ibqp, struct ib_qp_attr *attr,memset (&iwqp->roce_ah,0 ,sizeof (iwqp->roce_ah));if (attr->ah_attr.ah_flags &IB_AH_GRH ) { +u32 fl = udp_info->flow_label; +u32 lqp = ibqp->qp_num; +u32 rqp = roce_info->dest_qp; + + udp_info->src_port =irdma_get_udp_sport (fl, lqp, rqp); udp_info->ttl = attr->ah_attr.grh.hop_limit; udp_info->flow_label = attr->ah_attr.grh.flow_label; udp_info->tos = attr->ah_attr.grh.traffic_class;
Signature · 1 line
2.27.0
Quoted 11 lines
Subject: [PATCHv3 1/1] RDMA/irdma: Make the source udp port vary
From: Zhu Yanjun <[email protected]>
Based on the link https://www.spinics.net/lists/linux-rdma/msg73735.html,
get the source udp port number for a QP based on the grh.flow_label or lqpn/rqrpn. This provides a better spread of traffic across NIC RX queues. The method in the commit2b880b2e5e03("RDMA/mlx5: Define RoCEv2 udp source port when set path") is a standard way. So it is also adopted in this commit.
Quoted 9 lines
Signed-off-by Zhu Yanjun <[email protected]>---
V2->V3: Move to the block ofIB_QP_AVin the mask andIB_AH_GRHinV2->ah_flagsV1->V2: Adopt a standard method to get udp source port.--- drivers/infiniband/hw/irdma/verbs.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
Quoted 3 lines · On Tue, Dec 21, 2021 at 12:39:13PM -0500, [email protected] wrote:
From: Zhu Yanjun <[email protected]>
Based on the link https://www.spinics.net/lists/linux-rdma/msg73735.html,
Please use lore.kernel.org links. They have all chances to outlive spinics.
Quoted 35 lines
get the source udp port number for a QP based on the grh.flow_label or lqpn/rqrpn. This provides a better spread of traffic across NIC RX queues. The method in the commit
2b880b2e5e03("RDMA/mlx5: Define RoCEv2 udp source port when set path") is a standard way. So it is also adopted in this commit.Signed-off-by Zhu Yanjun <[email protected]>---
V2->V3: Move to the block ofIB_QP_AVin the mask andIB_AH_GRHinah_flagsV1->V2: Adopt a standard method to get udp source port.--- drivers/infiniband/hw/irdma/verbs.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)drivers/infiniband/hw/irdma/verbs.c +9
-0@@ -1094,6 +1094,15 @@ static int irdma_query_pkey(struct ib_device *ibdev, u32 port, u16 index,return 0 ; } + +static u16 irdma_get_udp_sport (u32 fl,u32 lqpn,u32 rqpn) +{ +if (!fl) + fl =rdma_calc_flow_label (lqpn, rqpn); + +return rdma_flow_label_to_udp_sport (fl); +} +/** * irdma_modify_qp_roce - modify qp request * @ibqp: qp's pointer for modify
drivers/infiniband/hw/irdma/verbs.c int irdma_modify_qp_roce(struct ib_qp *ibqp, struct ib_qp_attr *attr, · line 1182
Quoted 7 lines
drivers/infiniband/hw/irdma/verbs.c +4
-0@@ -1167,6 +1176,11 @@ int irdma_modify_qp_roce(struct ib_qp *ibqp, struct ib_qp_attr *attr,memset (&iwqp->roce_ah,0 ,sizeof (iwqp->roce_ah));if (attr->ah_attr.ah_flags &IB_AH_GRH ) { +u32 fl = udp_info->flow_label; +u32 lqp = ibqp->qp_num; +u32 rqp = roce_info->dest_qp; +
I don't see too much value in these extra variables and extra function that is the same as get_udp_sport() from hns.
It is worth to add new function to ib_verbs.h and reuse in both drivers.
Thanks
Quoted 7 lines
drivers/infiniband/hw/irdma/verbs.c +1
-0+ udp_info->src_port =irdma_get_udp_sport (fl, lqp, rqp); udp_info->ttl = attr->ah_attr.grh.hop_limit; udp_info->flow_label = attr->ah_attr.grh.flow_label; udp_info->tos = attr->ah_attr.grh.traffic_class;Signature · 1 line
2.27.0
Quoted 54 lines · Leon Romanovsky
Quoted 3 lines · On Tue, Dec 21, 2021 at 12:39:13PM -0500, [email protected] wrote:
From: Zhu Yanjun <[email protected]>
Based on the link https://www.spinics.net/lists/linux-rdma/msg73735.html,
Please use lore.kernel.org links. They have all chances to outlive spinics.
Quoted 35 lines
get the source udp port number for a QP based on the grh.flow_label or lqpn/rqrpn. This provides a better spread of traffic across NIC RX queues. The method in the commit
2b880b2e5e03("RDMA/mlx5: Define RoCEv2 udp source port when set path") is a standard way. So it is also adopted in this commit.Signed-off-by Zhu Yanjun <[email protected]>---
V2->V3: Move to the block ofIB_QP_AVin the mask andIB_AH_GRHinah_flagsV1->V2: Adopt a standard method to get udp source port.
---drivers/infiniband/hw/irdma/verbs.c| 14 ++++++++++++++1 file changed, 14 insertions(+)drivers/infiniband/hw/irdma/verbs.c +9
-0@@ -1094,6 +1094,15 @@ static int irdma_query_pkey(struct ib_device *ibdev, u32 port, u16 index,return 0 ; } + +static u16 irdma_get_udp_sport (u32 fl,u32 lqpn,u32 rqpn) +{ +if (!fl) + fl =rdma_calc_flow_label (lqpn, rqpn); + +return rdma_flow_label_to_udp_sport (fl); +} +/** * irdma_modify_qp_roce - modify qp request * @ibqp: qp's pointer for modify drivers/infiniband/hw/irdma/verbs.c int irdma_modify_qp_roce(struct ib_qp *ibqp, struct ib_qp_attr *attr, · line 1182
Quoted 7 lines
drivers/infiniband/hw/irdma/verbs.c +4
-0@@ -1167,6 +1176,11 @@ int irdma_modify_qp_roce(struct ib_qp *ibqp, struct ib_qp_attr *attr,memset (&iwqp->roce_ah,0 ,sizeof (iwqp->roce_ah));if (attr->ah_attr.ah_flags &IB_AH_GRH ) { +u32 fl = udp_info->flow_label; +u32 lqp = ibqp->qp_num; +u32 rqp = roce_info->dest_qp; +I don't see too much value in these extra variables and extra function that is the same as
get_udp_sport()from hns.
Quoted 1 line
It is worth to add new function to
ib_verbs.hand reuse in both drivers.
Do you mean the following function should be added into ib_verbs.h?
"
static inline u16 rdma_get_udp_sport(u32 fl, u32 lqpn, u32 rqpn)
{
if (!fl)
fl = rdma_calc_flow_label(lqpn, rqpn);
return rdma_flow_label_to_udp_sport(fl);
}"
Then in hns, rxe and irdma, this function is called to get udp source port? If so, I will send new patches.
Thanks.
Zhu Yanjun
Quoted 9 lines
Thanks
Quoted 6 lines
drivers/infiniband/hw/irdma/verbs.c +1
-1+ udp_info->src_port =irdma_get_udp_sport (fl, lqp, rqp); udp_info->ttl = attr->ah_attr.grh.hop_limit; udp_info->flow_label = attr->ah_attr.grh.flow_label; udp_info->tos = attr->ah_attr.grh.traffic_class; --2.27.0
Quoted 68 lines · On Tue, Jan 04, 2022 at 07:51:54AM +0000, [email protected] wrote:
Quoted 54 lines · Leon Romanovsky
Quoted 3 lines · On Tue, Dec 21, 2021 at 12:39:13PM -0500, [email protected] wrote:
From: Zhu Yanjun <[email protected]>
Based on the link https://www.spinics.net/lists/linux-rdma/msg73735.html,
Please use lore.kernel.org links. They have all chances to outlive spinics.
Quoted 35 lines
get the source udp port number for a QP based on the grh.flow_label or lqpn/rqrpn. This provides a better spread of traffic across NIC RX queues. The method in the commit
2b880b2e5e03("RDMA/mlx5: Define RoCEv2 udp source port when set path") is a standard way. So it is also adopted in this commit.Signed-off-by Zhu Yanjun <[email protected]>---
V2->V3: Move to the block ofIB_QP_AVin the mask andIB_AH_GRHinah_flagsV1->V2: Adopt a standard method to get udp source port.
---drivers/infiniband/hw/irdma/verbs.c| 14 ++++++++++++++1 file changed, 14 insertions(+)drivers/infiniband/hw/irdma/verbs.c +9
-0@@ -1094,6 +1094,15 @@ static int irdma_query_pkey(struct ib_device *ibdev, u32 port, u16 index,return 0 ; } + +static u16 irdma_get_udp_sport (u32 fl,u32 lqpn,u32 rqpn) +{ +if (!fl) + fl =rdma_calc_flow_label (lqpn, rqpn); + +return rdma_flow_label_to_udp_sport (fl); +} +/** * irdma_modify_qp_roce - modify qp request * @ibqp: qp's pointer for modify drivers/infiniband/hw/irdma/verbs.c int irdma_modify_qp_roce(struct ib_qp *ibqp, struct ib_qp_attr *attr, · line 1182
Quoted 7 lines
drivers/infiniband/hw/irdma/verbs.c +4
-0@@ -1167,6 +1176,11 @@ int irdma_modify_qp_roce(struct ib_qp *ibqp, struct ib_qp_attr *attr,memset (&iwqp->roce_ah,0 ,sizeof (iwqp->roce_ah));if (attr->ah_attr.ah_flags &IB_AH_GRH ) { +u32 fl = udp_info->flow_label; +u32 lqp = ibqp->qp_num; +u32 rqp = roce_info->dest_qp; +I don't see too much value in these extra variables and extra function that is the same as
get_udp_sport()from hns.Quoted 1 line
It is worth to add new function to
ib_verbs.hand reuse in both drivers.Do you mean the following function should be added into
ib_verbs.h?"
static inline u16rdma_get_udp_sport(u32 fl, u32 lqpn, u32 rqpn){ if (!fl) fl = rdma_calc_flow_label(lqpn, rqpn); return rdma_flow_label_to_udp_sport(fl); }
Quoted 3 lines
"
Then in hns, rxe and irdma, this function is called to get udp source port? If so, I will send new patches.
Yes and group all these patches in one series, please.
Thanks
Quoted 13 lines
Thanks.
Zhu YanjunQuoted 9 lines
Thanks
Quoted 6 lines
drivers/infiniband/hw/irdma/verbs.c +1
-1+ udp_info->src_port =irdma_get_udp_sport (fl, lqp, rqp); udp_info->ttl = attr->ah_attr.grh.hop_limit; udp_info->flow_label = attr->ah_attr.grh.flow_label; udp_info->tos = attr->ah_attr.grh.traffic_class; --2.27.0
Quoted 41 lines
Subject: [PATCHv3 1/1] RDMA/irdma: Make the source udp port vary
From: Zhu Yanjun <[email protected]>
Based on the link https://www.spinics.net/lists/linux-rdma/msg73735.html,
get the source udp port number for a QP based on the grh.flow_label or lqpn/rqrpn. This provides a better spread of traffic across NIC RX queues. The method in the commit2b880b2e5e03("RDMA/mlx5: Define RoCEv2 udp source port when set path") is a standard way. So it is also adopted in this commit.Signed-off-by Zhu Yanjun <[email protected]>---
V2->V3: Move to the block ofIB_QP_AVin the mask andIB_AH_GRHinV2->ah_flagsV1->V2: Adopt a standard method to get udp source port.--- drivers/infiniband/hw/irdma/verbs.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)drivers/infiniband/hw/irdma/verbs.c +7
-0@@ -1094,6 +1094,15 @@ static int irdma_query_pkey(struct ib_device *ibdev, u32 port, u16 index, return 0; } + +static u16 irdma_get_udp_sport (u32 fl,u32 lqpn,u32 rqpn) { +if (!fl) + fl =rdma_calc_flow_label (lqpn, rqpn); + +return rdma_flow_label_to_udp_sport (fl); } +/** * irdma_modify_qp_roce - modify qp request * @ibqp: qp's pointer for modify
drivers/infiniband/hw/irdma/verbs.c int irdma_modify_qp_roce(struct ib_qp *ibqp, struct · line 1184
Quoted 9 lines
drivers/infiniband/hw/irdma/verbs.c +5
-0@@ -1167,6 +1176,11 @@ int irdma_modify_qp_roce(struct ib_qp *ibqp, struct ib_qp_attr *attr,memset (&iwqp->roce_ah,0 ,sizeof (iwqp->roce_ah));if (attr->ah_attr.ah_flags &IB_AH_GRH ) { +u32 fl = udp_info->flow_label; +u32 lqp = ibqp->qp_num; +u32 rqp = roce_info->dest_qp; + + udp_info->src_port =irdma_get_udp_sport (fl, lqp, rqp);
The flow label used in function is not right as udp_info->flow_label is only set a few lines below. Kindly fold this fix into your next revision.
Quoted 5 lines
drivers/infiniband/hw/irdma/verbs.c +0
-0udp_info->ttl = attr->ah_attr.grh.hop_limit; udp_info->flow_label = attr->ah_attr.grh.flow_label;
udp_info->tos=attr->ah_attr.grh.traffic_class;
--
2.27.0
Quoted 42 lines · Shiraz
Quoted 42 lines
Subject: [PATCHv3 1/1] RDMA/irdma: Make the source udp port vary
From: Zhu Yanjun <[email protected]>
Based on the link https://www.spinics.net/lists/linux-rdma/msg73735.html,
get the source udp port number for a QP based on the grh.flow_label or lqpn/rqrpn. This provides a better spread of traffic across NIC RX queues. The method in the commit2b880b2e5e03("RDMA/mlx5: Define RoCEv2 udp source port when set path") is a standard way. So it is also adopted in this commit.Signed-off-by Zhu Yanjun <[email protected]>---
V2->V3: Move to the block ofIB_QP_AVin the mask andIB_AH_GRHinV2->ah_flagsV1->V2: Adopt a standard method to get udp source port.--- drivers/infiniband/hw/irdma/verbs.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)drivers/infiniband/hw/irdma/verbs.c +7
-0@@ -1094,6 +1094,15 @@ static int irdma_query_pkey(struct ib_device *ibdev, u32 port, u16 index, return 0; } + +static u16 irdma_get_udp_sport (u32 fl,u32 lqpn,u32 rqpn) { +if (!fl) + fl =rdma_calc_flow_label (lqpn, rqpn); + +return rdma_flow_label_to_udp_sport (fl); } +/** * irdma_modify_qp_roce - modify qp request * @ibqp: qp's pointer for modify @@ -1167,6 +1176,11 @@ int irdma_modify_qp_roce(struct ib_qp *ibqp, struct ib_qp_attr *attr,
Quoted 8 lines
drivers/infiniband/hw/irdma/verbs.c int irdma_modify_qp_roce(struct ib_qp *ibqp, struct ib_qp_attr *attr, · line 1183
Quoted 7 lines
drivers/infiniband/hw/irdma/verbs.c +5
-0memset (&iwqp->roce_ah,0 ,sizeof (iwqp->roce_ah));if (attr->ah_attr.ah_flags &IB_AH_GRH ) { +u32 fl = udp_info->flow_label; +u32 lqp = ibqp->qp_num; +u32 rqp = roce_info->dest_qp; + + udp_info->src_port =irdma_get_udp_sport (fl, lqp, rqp);The flow label used in function is not right as
udp_info->flow_labelis only set a few lines below. Kindly fold this fix into your next revision.
Thanks. " attr->ah_attr.grh.flow_label " should be OK.
Zhu Yanjun
Quoted 6 lines
Quoted 6 lines
drivers/infiniband/hw/irdma/verbs.c +0
-0udp_info->ttl = attr->ah_attr.grh.hop_limit; udp_info->flow_label = attr->ah_attr.grh.flow_label;
udp_info->tos=attr->ah_attr.grh.traffic_class;
--
2.27.0