Skip to contentSkip to conversation

RDMA/core: Fix corrupted SL on passive side

PATCH Applied Håkon Bugge 2 participants 6 messages Mar 22, 2021 – Apr 6, 2021 InfiniBand

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 RDMA/core: Fix corrupted SL on passive side

On RoCE systems, a CM REQ contains a Primary Hop Limit > 1 and Primary Subnet Local is zero.

In cm_req_handler(), the cm_process_routed_req() function is called. Since the Primary Subnet Local value is zero in the request, and since this is RoCE (Primary Local LID is permissive), the following statement will be executed:

      IBA_SET(CM_REQ_PRIMARY_SL, req_msg, wc->sl);

This corrupts SL in req_msg if it was different from zero. In other words, a request to setup a connection using an SL != zero, will not be honored, and a connection using SL zero will be created instead.

Fixed by not calling cm_process_routed_req() on RoCE systems.

Fixes 3971c9f6dbf2 ("IB/cm: Add interim support for routed paths")
---
 drivers/infiniband/core/cm.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/infiniband/core/cm.c +2 -1
index 3d194bb..6adbaea 100644
@@ -2138,7 +2138,8 @@ static int cm_req_handler(struct cm_work *work)
 		goto destroy;
 	}
 
-	cm_process_routed_req(req_msg, work->mad_recv_wc->wc);
+	if (cm_id_priv->av.ah_attr.type != RDMA_AH_ATTR_TYPE_ROCE)
+		cm_process_routed_req(req_msg, work->mad_recv_wc->wc);
 
 	memset(&work->path[0], 0, sizeof(work->path[0]));
 	if (cm_req_has_alt_path(req_msg))
Signature · 1 line
1.8.3.1
maintainer
#1 lore raw
Quoted 25 lines · Håkon Bugge

On RoCE systems, a CM REQ contains a Primary Hop Limit > 1 and Primary Subnet Local is zero.

In cm_req_handler(), the cm_process_routed_req() function is called. Since the Primary Subnet Local value is zero in the request, and since this is RoCE (Primary Local LID is permissive), the following statement will be executed:

      IBA_SET(CM_REQ_PRIMARY_SL, req_msg, wc->sl);

This corrupts SL in req_msg if it was different from zero. In other words, a request to setup a connection using an SL != zero, will not be honored, and a connection using SL zero will be created instead.

Fixed by not calling cm_process_routed_req() on RoCE systems.

Fixes 3971c9f6dbf2 ("IB/cm: Add interim support for routed paths")
 drivers/infiniband/core/cm.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/infiniband/core/cm.c +1 -0
index 3d194bb..6adbaea 100644
+++ b/drivers/infiniband/core/cm.c

drivers/infiniband/core/cm.c static int cm_req_handler(struct cm_work *work) · line 2142

Quoted 6 lines
drivers/infiniband/core/cm.c +2 -1
@@ -2138,7 +2138,8 @@ static int cm_req_handler(struct cm_work *work)
 		goto destroy;
 	}
 
-	cm_process_routed_req(req_msg, work->mad_recv_wc->wc);
+	if (cm_id_priv->av.ah_attr.type != RDMA_AH_ATTR_TYPE_ROCE)
+		cm_process_routed_req(req_msg, work->mad_recv_wc->wc);

why use ah_attr.type when a few lines below we have:

	if (gid_attr &&
	    rdma_protocol_roce(work->port->cm_dev->ib_device,
			       work->port->port_num)) {

?

I suspect you can just move this into the else?

Jason

#2 lore raw
Quoted 39 lines · Jason Gunthorpe
Quoted 25 lines · Håkon Bugge

On RoCE systems, a CM REQ contains a Primary Hop Limit > 1 and Primary Subnet Local is zero.

In cm_req_handler(), the cm_process_routed_req() function is called. Since the Primary Subnet Local value is zero in the request, and since this is RoCE (Primary Local LID is permissive), the following statement will be executed:

     IBA_SET(CM_REQ_PRIMARY_SL, req_msg, wc->sl);

This corrupts SL in req_msg if it was different from zero. In other words, a request to setup a connection using an SL != zero, will not be honored, and a connection using SL zero will be created instead.

Fixed by not calling cm_process_routed_req() on RoCE systems.

Fixes 3971c9f6dbf2 ("IB/cm: Add interim support for routed paths")

drivers/infiniband/core/cm.c | 3 ++-

1 file changed, 2 insertions(+), 1 deletion(-)
drivers/infiniband/core/cm.c +1 -0
index 3d194bb..6adbaea 100644
+++ b/drivers/infiniband/core/cm.c

drivers/infiniband/core/cm.c static int cm_req_handler(struct cm_work *work) · line 2142

Quoted 6 lines
drivers/infiniband/core/cm.c +2 -1
@@ -2138,7 +2138,8 @@ static int cm_req_handler(struct cm_work *work)
 		goto destroy;
 	}
 
-	cm_process_routed_req(req_msg, work->mad_recv_wc->wc);
+	if (cm_id_priv->av.ah_attr.type != RDMA_AH_ATTR_TYPE_ROCE)
+		cm_process_routed_req(req_msg, work->mad_recv_wc->wc);

why use ah_attr.type when a few lines below we have:

Quoted 7 lines
	if (gid_attr &&
	    rdma_protocol_roce(work->port->cm_dev->ib_device,
			       work->port->port_num)) {

?

I suspect you can just move this into the else?

I can counter that by saying ah_attr.type is used ~10 lines further down in the conditional call to sa_path_set_dmac() ;-)

Further, in

Quoted 3 lines
	if (gid_attr &&
	    rdma_protocol_roce(work->port->cm_dev->ib_device,
			       work->port->port_num)) {

I cannot really see how gid_attr could be null. If ib_init_ah_attr_from_wc() succeeds, it is set after the call to cm_init_av_for_response() above. May be using ah_attr.type in this test instead, for uniformity and readability?

I have no strong opinion.

Let me know your preference.

Thxs, Håkon

#3 lore raw
Quoted 58 lines · Håkon Bugge
Quoted 40 lines · Jason Gunthorpe
Quoted 25 lines · Håkon Bugge

On RoCE systems, a CM REQ contains a Primary Hop Limit > 1 and Primary Subnet Local is zero.

In cm_req_handler(), the cm_process_routed_req() function is called. Since the Primary Subnet Local value is zero in the request, and since this is RoCE (Primary Local LID is permissive), the following statement will be executed:

    IBA_SET(CM_REQ_PRIMARY_SL, req_msg, wc->sl);

This corrupts SL in req_msg if it was different from zero. In other words, a request to setup a connection using an SL != zero, will not be honored, and a connection using SL zero will be created instead.

Fixed by not calling cm_process_routed_req() on RoCE systems.

Fixes 3971c9f6dbf2 ("IB/cm: Add interim support for routed paths")

drivers/infiniband/core/cm.c | 3 ++-

1 file changed, 2 insertions(+), 1 deletion(-)
drivers/infiniband/core/cm.c +1 -0
index 3d194bb..6adbaea 100644
+++ b/drivers/infiniband/core/cm.c

drivers/infiniband/core/cm.c static int cm_req_handler(struct cm_work *work) · line 2142

Quoted 6 lines
drivers/infiniband/core/cm.c +2 -1
@@ -2138,7 +2138,8 @@ static int cm_req_handler(struct cm_work *work)
 		goto destroy;
 	}
 
-	cm_process_routed_req(req_msg, work->mad_recv_wc->wc);
+	if (cm_id_priv->av.ah_attr.type != RDMA_AH_ATTR_TYPE_ROCE)
+		cm_process_routed_req(req_msg, work->mad_recv_wc->wc);

why use ah_attr.type when a few lines below we have:

Quoted 7 lines
	if (gid_attr &&
	    rdma_protocol_roce(work->port->cm_dev->ib_device,
			       work->port->port_num)) {

?

I suspect you can just move this into the else?

I can counter that by saying ah_attr.type is used ~10 lines further down in the conditional call to sa_path_set_dmac() ;-)

Further, in

Quoted 3 lines
	if (gid_attr &&
	    rdma_protocol_roce(work->port->cm_dev->ib_device,
			       work->port->port_num)) {
Quoted 5 lines

I cannot really see how gid_attr could be null. If ib_init_ah_attr_from_wc() succeeds, it is set after the call to cm_init_av_for_response() above. May be using ah_attr.type in this test instead, for uniformity and readability?

I have no strong opinion.

Let me know your preference.

A gentle ping.

Thxs, Håkon

Quoted 3 lines

Thxs, Håkon

maintainer Applied
#4 lore raw
In reply to Håkon Bugge
Quoted 43 lines · Håkon Bugge
Quoted 43 lines · Jason Gunthorpe
Quoted 25 lines · Håkon Bugge

On RoCE systems, a CM REQ contains a Primary Hop Limit > 1 and Primary Subnet Local is zero.

In cm_req_handler(), the cm_process_routed_req() function is called. Since the Primary Subnet Local value is zero in the request, and since this is RoCE (Primary Local LID is permissive), the following statement will be executed:

     IBA_SET(CM_REQ_PRIMARY_SL, req_msg, wc->sl);

This corrupts SL in req_msg if it was different from zero. In other words, a request to setup a connection using an SL != zero, will not be honored, and a connection using SL zero will be created instead.

Fixed by not calling cm_process_routed_req() on RoCE systems.

Fixes 3971c9f6dbf2 ("IB/cm: Add interim support for routed paths")

drivers/infiniband/core/cm.c | 3 ++-

1 file changed, 2 insertions(+), 1 deletion(-)
drivers/infiniband/core/cm.c +1 -0
index 3d194bb..6adbaea 100644
+++ b/drivers/infiniband/core/cm.c

drivers/infiniband/core/cm.c static int cm_req_handler(struct cm_work *work) · line 2142

Quoted 6 lines
drivers/infiniband/core/cm.c +2 -1
@@ -2138,7 +2138,8 @@ static int cm_req_handler(struct cm_work *work)
 		goto destroy;
 	}
 
-	cm_process_routed_req(req_msg, work->mad_recv_wc->wc);
+	if (cm_id_priv->av.ah_attr.type != RDMA_AH_ATTR_TYPE_ROCE)
+		cm_process_routed_req(req_msg, work->mad_recv_wc->wc);

why use ah_attr.type when a few lines below we have:

	if (gid_attr &&
	    rdma_protocol_roce(work->port->cm_dev->ib_device,
			       work->port->port_num)) {
Quoted 6 lines
Quoted 3 lines

?

I suspect you can just move this into the else?

I can counter that by saying ah_attr.type is used ~10 lines further down in the conditional call to sa_path_set_dmac() ;-)

Hum, OK. Please send an additional patch to unify everything around av.ah_attr.type

Quoted 8 lines
Quoted 3 lines
	if (gid_attr &&
	    rdma_protocol_roce(work->port->cm_dev->ib_device,
			       work->port->port_num)) {

I cannot really see how gid_attr could be null. If ib_init_ah_attr_from_wc() succeeds, it is set after the call to cm_init_av_for_response() above. May be using ah_attr.type in this test instead, for uniformity and readability?

The GRH is optional, ib_init_ah_attr_from_wc() only sets it conditionally.

Applied to for-next

Thanks,
Jason

#5 lore raw
Quoted 50 lines · Jason Gunthorpe
Quoted 45 lines · Håkon Bugge
Quoted 45 lines · Jason Gunthorpe
Quoted 25 lines · Håkon Bugge

On RoCE systems, a CM REQ contains a Primary Hop Limit > 1 and Primary Subnet Local is zero.

In cm_req_handler(), the cm_process_routed_req() function is called. Since the Primary Subnet Local value is zero in the request, and since this is RoCE (Primary Local LID is permissive), the following statement will be executed:

    IBA_SET(CM_REQ_PRIMARY_SL, req_msg, wc->sl);

This corrupts SL in req_msg if it was different from zero. In other words, a request to setup a connection using an SL != zero, will not be honored, and a connection using SL zero will be created instead.

Fixed by not calling cm_process_routed_req() on RoCE systems.

Fixes 3971c9f6dbf2 ("IB/cm: Add interim support for routed paths")

drivers/infiniband/core/cm.c | 3 ++-

1 file changed, 2 insertions(+), 1 deletion(-)
drivers/infiniband/core/cm.c +1 -0
index 3d194bb..6adbaea 100644
+++ b/drivers/infiniband/core/cm.c

drivers/infiniband/core/cm.c static int cm_req_handler(struct cm_work *work) · line 2142

Quoted 6 lines
drivers/infiniband/core/cm.c +2 -1
@@ -2138,7 +2138,8 @@ static int cm_req_handler(struct cm_work *work)
 		goto destroy;
 	}
 
-	cm_process_routed_req(req_msg, work->mad_recv_wc->wc);
+	if (cm_id_priv->av.ah_attr.type != RDMA_AH_ATTR_TYPE_ROCE)
+		cm_process_routed_req(req_msg, work->mad_recv_wc->wc);

why use ah_attr.type when a few lines below we have:

	if (gid_attr &&
	    rdma_protocol_roce(work->port->cm_dev->ib_device,
			       work->port->port_num)) {

?

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

I suspect you can just move this into the else?

I can counter that by saying ah_attr.type is used ~10 lines further down in the conditional call to sa_path_set_dmac() ;-)

Hum, OK. Please send an additional patch to unify everything around av.ah_attr.type

Will do.

Quoted 11 lines
Quoted 8 lines
Quoted 3 lines
	if (gid_attr &&
	    rdma_protocol_roce(work->port->cm_dev->ib_device,
			       work->port->port_num)) {

I cannot really see how gid_attr could be null. If ib_init_ah_attr_from_wc() succeeds, it is set after the call to cm_init_av_for_response() above. May be using ah_attr.type in this test instead, for uniformity and readability?

The GRH is optional, ib_init_ah_attr_from_wc() only sets it conditionally.

True. But one of the conditions to set sgid_attr is rdma_protocol_roce(). Hence the first term in:

if (gid_attr && rdma_protocol_roce())

is superfluous. This because, it cannot be NULL on RoCE systems, because it is dereferenced in:

cm_init_av_for_response()

    ib_init_ah_attr_from_wc()
        rdma_move_grh_sgid_attr()

I'll send the patch with the gid_attr term and let you can decide.

Thxs, Håkon

Quoted 5 lines

Applied to for-next

Thanks,
Jason