Generate UDP src port with flow label or lqpn/rqpn
Subsystems (4)
v2 cover Generate UDP src port with flow label or lqpn/rqpn
From: Zhu Yanjun <[email protected]>
Follow the advice from Leon Romanovsky, rdma_get_udp_sport is moved to ib_verbs.h. several drivers generate udp source port with this function.
---
v1->v2:Remove the local variables in commits "RDMA/irdma: Make the source
udp port vary" and "RDMA/rxe: Use the standard method to produce udp source port". A new commit is added to remove the redundant randomization for UDP source port in RXE.
---
Zhu Yanjun (5):
RDMA/core: Calculate UDP source port based on flow label or lqpn/rqpn RDMA/hns: Replace get_udp_sport with rdma_get_udp_sport RDMA/irdma: Make the source udp port vary
RDMA/rxe: Use the standard method to produce udp source port RDMA/rxe: Remove the redundant randomization for UDP source port
drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 12 ++---------- drivers/infiniband/hw/irdma/verbs.c | 4 ++++ drivers/infiniband/sw/rxe/rxe_qp.c | 10 ++-------- drivers/infiniband/sw/rxe/rxe_verbs.c | 6 ++++++ include/rdma/ib_verbs.h | 17 +++++++++++++++++ 5 files changed, 31 insertions(+), 18 deletions(-)
Signature · 1 line
2.27.0
PATCH v2 1/5 RDMA/core: Calculate UDP source port based on flow label or lqpn/rqpn
From: Zhu Yanjun <[email protected]>
Calculate and set UDP source port based on the flow label. If flow label is not defined in GRH then calculate it based on lqpn/rqpn.
--- include/rdma/ib_verbs.h | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)
include/rdma/ib_verbs.h +17 -0
@@ -4749,6 +4749,23 @@ static inline u32 rdma_calc_flow_label(u32 lqpn, u32 rqpn) return (u32)(v & IB_GRH_FLOWLABEL_MASK); } +/** +* rdma_get_udp_sport - Calculate and set UDP source port based on the flow +* label. If flow label is not defined in GRH then +* calculate it based on lqpn/rqpn. +* +* @fl: flow label from GRH +* @lqpn: local qp number +* @rqpn: remote qp number +*/ +static inline u16 rdma_get_udp_sport (u32 fl,u32 lqpn,u32 rqpn) +{ +if (!fl) + fl =rdma_calc_flow_label (lqpn, rqpn); + +return rdma_flow_label_to_udp_sport (fl); +} +const struct ib_port_immutable *ib_port_immutable_read (struct ib_device *dev,unsigned int port);#endif /* IB_VERBS_H */
Signature · 1 line
2.27.0
Re: [PATCH 1/5] RDMA/core: Calculate UDP source port based on flow label or lqpn/rqpn
Quoted 10 lines · On Wed, Jan 05, 2022 at 05:12:33PM -0500, [email protected] wrote:
From: Zhu Yanjun <[email protected]>
Calculate and set UDP source port based on the flow label. If flow label is not defined in GRH then calculate it based on lqpn/rqpn.
Signed-off-by Zhu Yanjun <[email protected]>--- include/rdma/ib_verbs.h | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)
Thanks,
PATCH v2 2/5 RDMA/hns: Replace get_udp_sport with rdma_get_udp_sport
In reply to [email protected]From: Zhu Yanjun <[email protected]>
Several drivers have the same function xxx_get_udp_sport. So this function is moved to ib_verbs.h.
--- drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-)
drivers/infiniband/hw/hns/hns_roce_hw_v2.c +2 -10
@@ -4488,14 +4488,6 @@ static int modify_qp_rtr_to_rts(struct ib_qp *ibqp,return 0 ; } -static inline u16 get_udp_sport (u32 fl,u32 lqpn,u32 rqpn) -{ -if (!fl) - fl =rdma_calc_flow_label (lqpn, rqpn); - -return rdma_flow_label_to_udp_sport (fl); -} -static int get_dip_ctx_idx (struct ib_qp *ibqp,const struct ib_qp_attr *attr,u32 *dip_idx) {
@@ -4712,8 +4704,8 @@ static int hns_roce_v2_set_path(struct ib_qp *ibqp, }hr_reg_write (context,QPC_UDPSPN , - is_udp ?get_udp_sport (grh->flow_label, ibqp->qp_num, - attr->dest_qp_num) :0 ); + is_udp ?rdma_get_udp_sport (grh->flow_label, ibqp->qp_num, + attr->dest_qp_num) :0 );hr_reg_clear (qpc_mask,QPC_UDPSPN );
Signature · 1 line
2.27.0
Re: [PATCH 2/5] RDMA/hns: Replace get_udp_sport with rdma_get_udp_sport
Quoted 10 lines · On Wed, Jan 05, 2022 at 05:12:34PM -0500, [email protected] wrote:
From: Zhu Yanjun <[email protected]>
Several drivers have the same function
xxx_get_udp_sport. So this function is moved toib_verbs.h.Signed-off-by Zhu Yanjun <[email protected]>--- drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-)
Thanks,
PATCH v2 3/5 RDMA/irdma: Make the source udp port vary
In reply to [email protected]From: Zhu Yanjun <[email protected]>
Get the source udp port number for a QP based on the grh.flow_label or lqpn/rqrpn. This provides a better spread of traffic across NIC RX queues.
--- drivers/infiniband/hw/irdma/verbs.c | 4 ++++ 1 file changed, 4 insertions(+)
drivers/infiniband/hw/irdma/verbs.c +4 -0
@@ -1170,6 +1170,10 @@ int irdma_modify_qp_roce(struct ib_qp *ibqp, struct ib_qp_attr *attr, udp_info->ttl = attr->ah_attr.grh.hop_limit; udp_info->flow_label = attr->ah_attr.grh.flow_label; udp_info->tos = attr->ah_attr.grh.traffic_class; + udp_info->src_port =rdma_get_udp_sport ( + udp_info->flow_label, + ibqp->qp_num, + roce_info->dest_qp);irdma_qp_rem_qos (&iwqp->sc_qp); dev->ws_remove (iwqp->sc_qp.vsi, ctx_info->user_pri); ctx_info->user_pri =rt_tos2priority (udp_info->tos);
Signature · 1 line
2.27.0
Re: [PATCH 3/5] RDMA/irdma: Make the source udp port vary
Quoted 10 lines · On Wed, Jan 05, 2022 at 05:12:35PM -0500, [email protected] wrote:
From: Zhu Yanjun <[email protected]>
Get the source udp port number for a QP based on the grh.flow_label or lqpn/rqrpn. This provides a better spread of traffic across NIC RX queues.
Signed-off-by Zhu Yanjun <[email protected]>--- drivers/infiniband/hw/irdma/verbs.c | 4 ++++ 1 file changed, 4 insertions(+)
Thanks,
RE: [PATCH 3/5] RDMA/irdma: Make the source udp port vary
In reply to [email protected]Quoted 4 lines
Subject: [PATCH 3/5] RDMA/irdma: Make the source udp port vary
From: Zhu Yanjun <[email protected]>
Quoted 8 lines
Get the source udp port number for a QP based on the grh.flow_label or lqpn/rqrpn. This provides a better spread of traffic across NIC RX queues.
Signed-off-by Zhu Yanjun <[email protected]>--- drivers/infiniband/hw/irdma/verbs.c | 4 ++++ 1 file changed, 4 insertions(+)
PATCH v2 4/5 RDMA/rxe: Use the standard method to produce udp source port
In reply to [email protected]From: Zhu Yanjun <[email protected]>
Use the standard method to produce udp source port.
--- drivers/infiniband/sw/rxe/rxe_verbs.c | 6 ++++++ 1 file changed, 6 insertions(+)
drivers/infiniband/sw/rxe/rxe_verbs.c +6 -0
@@ -469,6 +469,12 @@ static int rxe_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr,if (err) goto err1; +if ((mask &IB_QP_AV ) && (attr->ah_attr.ah_flags &IB_AH_GRH )) + qp->src_port =rdma_get_udp_sport (attr->ah_attr.grh.flow_label, + qp->ibqp.qp_num, + qp->attr.dest_qp_num); + +return 0 ; err1:
Signature · 1 line
2.27.0
Re: [PATCH 4/5] RDMA/rxe: Use the standard method to produce udp source port
Quoted 14 lines · On Wed, Jan 05, 2022 at 05:12:36PM -0500, [email protected] wrote:
From: Zhu Yanjun <[email protected]>
Use the standard method to produce udp source port.
Signed-off-by Zhu Yanjun <[email protected]>--- drivers/infiniband/sw/rxe/rxe_verbs.c | 6 ++++++ 1 file changed, 6 insertions(+)drivers/infiniband/sw/rxe/rxe_verbs.c +0
-0
drivers/infiniband/sw/rxe/rxe_verbs.c static int rxe_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, · line 472
Quoted 4 lines
drivers/infiniband/sw/rxe/rxe_verbs.c +1
-0@@ -469,6 +469,12 @@ static int rxe_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr,if (err) goto err1; +if ((mask &IB_QP_AV ) && (attr->ah_attr.ah_flags &IB_AH_GRH ))
You are leaving src_port default and wired to same port as other QPs without any randomization.
Thanks
Quoted 11 lines
drivers/infiniband/sw/rxe/rxe_verbs.c +5
-0+ qp->src_port =rdma_get_udp_sport (attr->ah_attr.grh.flow_label, + qp->ibqp.qp_num, + qp->attr.dest_qp_num); + +return 0 ; err1:Signature · 1 line
2.27.0
Zhu Yanjun
maintainerRe: [PATCH 4/5] RDMA/rxe: Use the standard method to produce udp source port
Quoted 19 lines · Leon Romanovsky
Quoted 17 lines · On Wed, Jan 05, 2022 at 05:12:36PM -0500, [email protected] wrote:
From: Zhu Yanjun <[email protected]>
Use the standard method to produce udp source port.
Signed-off-by Zhu Yanjun <[email protected]>--- drivers/infiniband/sw/rxe/rxe_verbs.c | 6 ++++++ 1 file changed, 6 insertions(+)drivers/infiniband/sw/rxe/rxe_verbs.c +0
-0@@ -469,6 +469,12 @@ static int rxe_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr,if (err) goto err1;
Quoted 4 lines
drivers/infiniband/sw/rxe/rxe_verbs.c static int rxe_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, · line 472
Quoted 1 line
drivers/infiniband/sw/rxe/rxe_verbs.c +1
-0+if ((mask &IB_QP_AV ) && (attr->ah_attr.ah_flags &IB_AH_GRH ))You are leaving
src_portdefault and wired to same port as other QPs without any randomization.
Hi,
I do not get you. Why do you think I am leaving src_pport default? Thanks.
Zhu Yanjun
Quoted 14 lines
Thanks
Quoted 11 lines
drivers/infiniband/sw/rxe/rxe_verbs.c +5
-1+ qp->src_port =rdma_get_udp_sport (attr->ah_attr.grh.flow_label, + qp->ibqp.qp_num, + qp->attr.dest_qp_num); + +return 0 ; err1: --2.27.0
Re: [PATCH 4/5] RDMA/rxe: Use the standard method to produce udp source port
Quoted 20 lines · Zhu Yanjun
Quoted 19 lines · Leon Romanovsky
Quoted 17 lines · On Wed, Jan 05, 2022 at 05:12:36PM -0500, [email protected] wrote:
From: Zhu Yanjun <[email protected]>
Use the standard method to produce udp source port.
Signed-off-by Zhu Yanjun <[email protected]>--- drivers/infiniband/sw/rxe/rxe_verbs.c | 6 ++++++ 1 file changed, 6 insertions(+)drivers/infiniband/sw/rxe/rxe_verbs.c +0
-0@@ -469,6 +469,12 @@ static int rxe_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr,if (err) goto err1;
Quoted 8 lines
Quoted 4 lines
drivers/infiniband/sw/rxe/rxe_verbs.c static int rxe_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, · line 472
Quoted 1 line
drivers/infiniband/sw/rxe/rxe_verbs.c +1
-0+if ((mask &IB_QP_AV ) && (attr->ah_attr.ah_flags &IB_AH_GRH ))You are leaving
src_portdefault and wired to same port as other QPs without any randomization.Hi,
I do not get you. Why do you think I am leaving
src_pportdefault?
Because in original code, you randomized src_port without any relation to mask flags.
qp->src_port = RXE_ROCE_V2_SPORT +
(hash_32_generic(qp_num(qp), 14) & 0x3fff);After patch #5, if user doesn't pass "proper" mask, you will leave qp->src_port to be equal to RXE_ROCE_V2_SPORT, which is different from the current behaviour.
Thanks
Quoted 18 lines
Thanks.
Zhu Yanjun
Quoted 14 lines
Thanks
Quoted 11 lines
drivers/infiniband/sw/rxe/rxe_verbs.c +5
-1+ qp->src_port =rdma_get_udp_sport (attr->ah_attr.grh.flow_label, + qp->ibqp.qp_num, + qp->attr.dest_qp_num); + +return 0 ; err1: --2.27.0
Re: [PATCH 4/5] RDMA/rxe: Use the standard method to produce udp source port
Quoted 34 lines · Leon Romanovsky
Quoted 20 lines · Zhu Yanjun
On Wed, Jan 5, 2022 at 3:52 PM Leon Romanovsky <[email protected]> wrote:
Quoted 17 lines · On Wed, Jan 05, 2022 at 05:12:36PM -0500, [email protected] wrote:
From: Zhu Yanjun <[email protected]>
Use the standard method to produce udp source port.
Signed-off-by Zhu Yanjun <[email protected]>---
drivers/infiniband/sw/rxe/rxe_verbs.c| 6 ++++++1 file changed, 6 insertions(+)drivers/infiniband/sw/rxe/rxe_verbs.c +0
-0@@ -469,6 +469,12 @@ static int rxe_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr,if (err) goto err1;Quoted 8 lines
drivers/infiniband/sw/rxe/rxe_verbs.c static int rxe_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, · line 472
Quoted 1 line
drivers/infiniband/sw/rxe/rxe_verbs.c +1
-0+if ((mask &IB_QP_AV ) && (attr->ah_attr.ah_flags &IB_AH_GRH ))You are leaving
src_portdefault and wired to same port as other QPs without any randomization.Hi,
I do not get you. Why do you think I am leaving
src_pportdefault?Because in original code, you randomized
src_portwithout any relation to mask flags.
Quoted 6 lines
qp->src_port=RXE_ROCE_V2_SPORT+
(hash_32_generic(qp_num(qp), 14)&0x3fff);After patch #5, if user doesn't pass "proper" mask, you will leave
qp->src_portto be equal toRXE_ROCE_V2_SPORT, which is different from the current behaviour.
About the "proper" mask, please check this link https://patchwork.kernel.org/project/linux-rdma/patch/[email protected]/
"the udp_sport is only set when address vector and dest qpn (IB_QP_AV and IB_QP_DEST_QPN) is provided."
Zhu Yanjun
Quoted 20 lines
Thanks
Quoted 17 lines
Thanks.
Zhu Yanjun
Thanks
Quoted 11 lines
drivers/infiniband/sw/rxe/rxe_verbs.c +5
-0+ qp->src_port =rdma_get_udp_sport (attr->ah_attr.grh.flow_label, + qp->ibqp.qp_num, + qp->attr.dest_qp_num); + +return 0 ;err1:
--
2.27.0
Zhu Yanjun
maintainerRe: [PATCH 4/5] RDMA/rxe: Use the standard method to produce udp source port
In reply to Leon RomanovskyQuoted 34 lines · Leon Romanovsky
Quoted 20 lines · Zhu Yanjun
Quoted 19 lines · Leon Romanovsky
Quoted 17 lines · On Wed, Jan 05, 2022 at 05:12:36PM -0500, [email protected] wrote:
From: Zhu Yanjun <[email protected]>
Use the standard method to produce udp source port.
Signed-off-by Zhu Yanjun <[email protected]>--- drivers/infiniband/sw/rxe/rxe_verbs.c | 6 ++++++ 1 file changed, 6 insertions(+)drivers/infiniband/sw/rxe/rxe_verbs.c +0
-0@@ -469,6 +469,12 @@ static int rxe_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr,if (err) goto err1;Quoted 8 lines
Quoted 4 lines
drivers/infiniband/sw/rxe/rxe_verbs.c static int rxe_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, · line 472
Quoted 1 line
drivers/infiniband/sw/rxe/rxe_verbs.c +1
-0+if ((mask &IB_QP_AV ) && (attr->ah_attr.ah_flags &IB_AH_GRH ))You are leaving
src_portdefault and wired to same port as other QPs without any randomization.Hi,
I do not get you. Why do you think I am leaving
src_pportdefault?Because in original code, you randomized
src_portwithout any relation to mask flags.
Quoted 6 lines
qp->src_port = RXE_ROCE_V2_SPORT + (hash_32_generic(qp_num(qp), 14) & 0x3fff);After patch #5, if user doesn't pass "proper" mask, you will leave
qp->src_portto be equal toRXE_ROCE_V2_SPORT, which is different from the current behaviour.
Hi, Leon Romanovsky
I read your comments again and checked the source code.
And I found this https://lkml.org/lkml/2015/12/15/566, Jason commented:
"
...
The GRH is optional for in-subnet communications.
...
"
I agree with you. When in-subnet communications, GRH is optional. It is possible that qp->src_port is set to RXE_ROCE_V2_SPORT.
So I will remove patch #5 and send the patch series again. Thanks.
Zhu Yanjun
Quoted 22 lines
Thanks
Quoted 18 lines
Thanks.
Zhu Yanjun
Quoted 14 lines
Thanks
Quoted 11 lines
drivers/infiniband/sw/rxe/rxe_verbs.c +5
-1+ qp->src_port =rdma_get_udp_sport (attr->ah_attr.grh.flow_label, + qp->ibqp.qp_num, + qp->attr.dest_qp_num); + +return 0 ; err1: --2.27.0
PATCH v2 5/5 RDMA/rxe: Remove the redundant randomization for UDP source port
In reply to [email protected]From: Zhu Yanjun <[email protected]>
Since the UDP source port is modified in rxe_modify_qp, the randomization for UDP source port is redundant in this function. So remove it.
--- drivers/infiniband/sw/rxe/rxe_qp.c | 10 ++-------- 1 file changed, 2 insertions(+), 8 deletions(-)
drivers/infiniband/sw/rxe/rxe_qp.c +2 -8
@@ -210,15 +210,9 @@ static int rxe_qp_init_req(struct rxe_dev *rxe, struct rxe_qp *qp,return err; qp->sk->sk->sk_user_data = qp; -/* pick a source UDP port number for this QP based on -* the source QPN. this spreads traffic for different QPs -* across different NIC RX queues (while using a single -* flow for a given QP to maintain packet order). -* the port number must be in the Dynamic Ports range -* (0xc000 - 0xffff). +/* Source UDP port number for this QP is modified in rxe_qp_modify. */ - qp->src_port =RXE_ROCE_V2_SPORT + - (hash_32_generic (qp_num (qp),14 ) &0x3fff ); + qp->src_port =RXE_ROCE_V2_SPORT ; qp->sq.max_wr = init->cap.max_send_wr;/* These caps are limited by rxe_qp_chk_cap() done by the caller */
Signature · 1 line
2.27.0
Re: [PATCH 5/5] RDMA/rxe: Remove the redundant randomization for UDP source port
Quoted 16 lines · On Wed, Jan 05, 2022 at 05:12:37PM -0500, [email protected] wrote:
From: Zhu Yanjun <[email protected]>
Since the UDP source port is modified in
rxe_modify_qp, the randomization for UDP source port is redundant in this function. So remove it.Signed-off-by Zhu Yanjun <[email protected]>--- drivers/infiniband/sw/rxe/rxe_qp.c | 10 ++-------- 1 file changed, 2 insertions(+), 8 deletions(-)drivers/infiniband/sw/rxe/rxe_qp.c +0
-0@@ -210,15 +210,9 @@ static int rxe_qp_init_req(struct rxe_dev *rxe, struct rxe_qp *qp,return err;
drivers/infiniband/sw/rxe/rxe_qp.c static int rxe_qp_init_req(struct rxe_dev *rxe, struct rxe_qp *qp, · line 214
Quoted 10 lines
drivers/infiniband/sw/rxe/rxe_qp.c +1
-6qp->sk->sk->sk_user_data = qp; -/* pick a source UDP port number for this QP based on -* the source QPN. this spreads traffic for different QPs -* across different NIC RX queues (while using a single -* flow for a given QP to maintain packet order). -* the port number must be in the Dynamic Ports range -* (0xc000 - 0xffff). +/* Source UDP port number for this QP is modified in rxe_qp_modify. */
This makes me wonder why do we set this src_port here? Are we using this field before modify QP?
Thanks
Quoted 9 lines
drivers/infiniband/sw/rxe/rxe_qp.c +1
-2- qp->src_port =RXE_ROCE_V2_SPORT + - (hash_32_generic (qp_num (qp),14 ) &0x3fff ); + qp->src_port =RXE_ROCE_V2_SPORT ; qp->sq.max_wr = init->cap.max_send_wr;/* These caps are limited by rxe_qp_chk_cap() done by the caller */ Signature · 1 line
2.27.0
Re: [PATCH 5/5] RDMA/rxe: Remove the redundant randomization for UDP source port
Quoted 20 lines · Leon Romanovsky
Quoted 18 lines · On Wed, Jan 05, 2022 at 05:12:37PM -0500, [email protected] wrote:
From: Zhu Yanjun <[email protected]>
Since the UDP source port is modified in
rxe_modify_qp, the randomization for UDP source port is redundant in this function. So remove it.Signed-off-by Zhu Yanjun <[email protected]>---
drivers/infiniband/sw/rxe/rxe_qp.c| 10 ++--------1 file changed, 2 insertions(+), 8 deletions(-)drivers/infiniband/sw/rxe/rxe_qp.c +0
-0@@ -210,15 +210,9 @@ static int rxe_qp_init_req(struct rxe_dev *rxe, struct rxe_qp *qp,return err; qp->sk->sk->sk_user_data = qp;
Quoted 11 lines
drivers/infiniband/sw/rxe/rxe_qp.c static int rxe_qp_init_req(struct rxe_dev *rxe, struct rxe_qp *qp, · line 214
Quoted 8 lines
drivers/infiniband/sw/rxe/rxe_qp.c +1
-6-/* pick a source UDP port number for this QP based on -* the source QPN. this spreads traffic for different QPs -* across different NIC RX queues (while using a single -* flow for a given QP to maintain packet order). -* the port number must be in the Dynamic Ports range -* (0xc000 - 0xffff). +/* Source UDP port number for this QP is modified in rxe_qp_modify. */ This makes me wonder why do we set this
src_porthere? Are we using this field before modify QP?
The commit d3c04a3a6870 ("IB/rxe: vary the source udp port for receive scaling") sets this src_port here.
The advantage of setting src_port here is: before rxe_modify_qp, the src port is randomized, not 0xc000. So after/before rxe_modify_qp, the src port is the same value.
If the src port is changed in rxe_modify_qp, before rxe_modify_qp, the src port is 0xc000, after rxe_modify_qp, the src port is randomized, for example, src port is 0xF043.
So when the new method is adopted, I removed this.
Zhu Yanjun
Quoted 11 lines
Thanks
Quoted 8 lines
drivers/infiniband/sw/rxe/rxe_qp.c +1
-3- qp->src_port =RXE_ROCE_V2_SPORT + - (hash_32_generic (qp_num (qp),14 ) &0x3fff ); + qp->src_port =RXE_ROCE_V2_SPORT ; qp->sq.max_wr = init->cap.max_send_wr;/* These caps are limited by rxe_qp_chk_cap() done by the caller */ --2.27.0
Re: [PATCH 5/5] RDMA/rxe: Remove the redundant randomization for UDP source port
Quoted 34 lines · On Wed, Jan 05, 2022 at 08:42:03AM +0000, [email protected] wrote:
Quoted 20 lines · Leon Romanovsky
Quoted 18 lines · On Wed, Jan 05, 2022 at 05:12:37PM -0500, [email protected] wrote:
From: Zhu Yanjun <[email protected]>
Since the UDP source port is modified in
rxe_modify_qp, the randomization for UDP source port is redundant in this function. So remove it.Signed-off-by Zhu Yanjun <[email protected]>---
drivers/infiniband/sw/rxe/rxe_qp.c| 10 ++--------1 file changed, 2 insertions(+), 8 deletions(-)drivers/infiniband/sw/rxe/rxe_qp.c +0
-0@@ -210,15 +210,9 @@ static int rxe_qp_init_req(struct rxe_dev *rxe, struct rxe_qp *qp,return err; qp->sk->sk->sk_user_data = qp;Quoted 11 lines
drivers/infiniband/sw/rxe/rxe_qp.c static int rxe_qp_init_req(struct rxe_dev *rxe, struct rxe_qp *qp, · line 214
Quoted 8 lines
drivers/infiniband/sw/rxe/rxe_qp.c +1
-6-/* pick a source UDP port number for this QP based on -* the source QPN. this spreads traffic for different QPs -* across different NIC RX queues (while using a single -* flow for a given QP to maintain packet order). -* the port number must be in the Dynamic Ports range -* (0xc000 - 0xffff). +/* Source UDP port number for this QP is modified in rxe_qp_modify. */ This makes me wonder why do we set this
src_porthere? Are we using this field before modify QP?
Quoted 7 lines
The commit
d3c04a3a6870("IB/rxe: vary the source udp port for receive scaling") sets thissrc_porthere.The advantage of setting
src_porthere is: beforerxe_modify_qp, the src port is randomized, not0xc000. So after/beforerxe_modify_qp, the src port is the same value.If the src port is changed in
rxe_modify_qp, beforerxe_modify_qp, the src port is0xc000, afterrxe_modify_qp, the src port is randomized, for example, src port is0xF043.
I'm asking if you use qp->src_port between this line and rxe_modify_qp?
Thanks
Quoted 16 lines
So when the new method is adopted, I removed this.
Zhu Yanjun
Quoted 11 lines
Thanks
Quoted 8 lines
drivers/infiniband/sw/rxe/rxe_qp.c +1
-3- qp->src_port =RXE_ROCE_V2_SPORT + - (hash_32_generic (qp_num (qp),14 ) &0x3fff ); + qp->src_port =RXE_ROCE_V2_SPORT ; qp->sq.max_wr = init->cap.max_send_wr;/* These caps are limited by rxe_qp_chk_cap() done by the caller */ --2.27.0
Re: [PATCH 5/5] RDMA/rxe: Remove the redundant randomization for UDP source port
Quoted 43 lines · Leon Romanovsky
Quoted 41 lines · On Wed, Jan 05, 2022 at 08:42:03AM +0000, [email protected] wrote:
January 5, 2022 3:49 PM, "Leon Romanovsky" <[email protected]> wrote:
On Wed, Jan 05, 2022 at 05:12:37PM -0500, [email protected] wrote:
From: Zhu Yanjun <[email protected]>
Since the UDP source port is modified in
rxe_modify_qp, the randomization for UDP source port is redundant in this function. So remove it.Signed-off-by Zhu Yanjun <[email protected]>---
drivers/infiniband/sw/rxe/rxe_qp.c| 10 ++--------1 file changed, 2 insertions(+), 8 deletions(-)drivers/infiniband/sw/rxe/rxe_qp.c +1
-6@@ -210,15 +210,9 @@ static int rxe_qp_init_req(struct rxe_dev *rxe, struct rxe_qp *qp,return err; qp->sk->sk->sk_user_data = qp; -/* pick a source UDP port number for this QP based on -* the source QPN. this spreads traffic for different QPs -* across different NIC RX queues (while using a single -* flow for a given QP to maintain packet order). -* the port number must be in the Dynamic Ports range -* (0xc000 - 0xffff). +/* Source UDP port number for this QP is modified in rxe_qp_modify. */ This makes me wonder why do we set this
src_porthere? Are we using this field before modify QP?The commit
d3c04a3a6870("IB/rxe: vary the source udp port for receive scaling") sets thissrc_porthere.The advantage of setting
src_porthere is: beforerxe_modify_qp, the src port is randomized, not0xc000.
So after/beforerxe_modify_qp, the src port is the same value.
Quoted 5 lines
Quoted 3 lines
If the src port is changed in
rxe_modify_qp, beforerxe_modify_qp, the src port is0xc000, afterrxe_modify_qp,
the src port is randomized, for example, src port is0xF043.I'm asking if you use
qp->src_portbetween this line andrxe_modify_qp?
There are only 2 udp packets between this line and rxe_modify_qp.
Zhu Yanjun
Quoted 17 lines
Thanks
Quoted 14 lines
So when the new method is adopted, I removed this.
Zhu Yanjun
Thanks
-
qp->src_port=RXE_ROCE_V2_SPORT+
- (hash_32_generic(qp_num(qp), 14)&0x3fff);
+qp->src_port=RXE_ROCE_V2_SPORT;qp->sq.max_wr=init->cap.max_send_wr;/* These caps are limited by
rxe_qp_chk_cap()done by the caller */ --
2.27.0