Skip to contentSkip to conversation

SRP issues with OpenSM 3.3.3

RFC PATCH In review Ira Weiny 4 participants 22 messages Dec 15, 2009 – Jan 4, 2010 10 messages not in the archive InfiniBand

lore
Share & tools
lore

Ira Weiny

author
#0 lore raw

Sasha, Hal,

I have found that the following patch caused our SRP connected storage to break.

patch: 3d20f82edd3246879063b77721d0bcef927bdc48

opensm/osm_sa_path_record.c: separate router guid resolution code

Move off subnet destination (router address) resolution code to separate function to improve readability.

I further traced the problem to pr_rcv_build_pr and the patch below fixes the bug.

16:03:34 > git diff

opensm/opensm/osm_sa_path_record.c +1 -1
index be0cd71..32c889f 100644
@@ -800,7 +800,7 @@ static void pr_rcv_build_pr(IN osm_sa_t * sa, IN const osm_port_t * p_src_port,
 
        /* Only set HopLimit if going through a router */
        if (is_nonzero_gid)
-               p_pr->hop_flow_raw |= cl_hton32(IB_HOPLIMIT_MAX);
+               p_pr->hop_flow_raw |= IB_HOPLIMIT_MAX;
 
        p_pr->pkey = p_parms->pkey;
        ib_path_rec_set_sl(p_pr, p_parms->sl);

"hop_flow_raw" is not really a 32bit value but rather 4 fields:

Component     Length(bits)   Offset(bits)
RawTraffic    1              352
Reserved      3              353
FlowLabel     20             356
HopLimit      8              384

However, I don't understand the comment "Only set HopLimit if going through a router"?

Was the intent to only set p_dgid in pr_rcv_get_end_points if we are heading through a router? I don't think it matters if we are going through a router or not does it? If not, I think the comment needs to be removed in the patch above.

Thanks,
Ira

Signature · 9 lines
Ira Weiny
Math Programmer/Computer Scientist
Lawrence Livermore National Lab
925-423-8008
[email protected]
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): [email protected]

#2 lore raw

Ira,

Quoted 4 lines · Ira Weiny

Sasha, Hal,

I have found that the following patch caused our SRP connected storage to break.

What is causing the SRP target to fail ? Is it a non zero hop limit returned in the SA PathRecord ?

Quoted 11 lines

patch: 3d20f82edd3246879063b77721d0bcef927bdc48

   opensm/osm_sa_path_record.c: separate router guid resolution code

   Move off subnet destination (router address) resolution code to separate function to improve readability.

I further traced the problem to pr_rcv_build_pr and the patch below fixes the bug.

On quick glance, I'm missing the connection between Sasha's patch and this routine setting something bad in the hop limit of the returned path record.

opensm/opensm/osm_sa_path_record.c static void pr_rcv_build_pr(IN osm_sa_t * sa, IN const osm_port_t * p_src_port, · line 803

Quoted 11 lines

16:03:34 > git diff

opensm/opensm/osm_sa_path_record.c +1 -1
index be0cd71..32c889f 100644
@@ -800,7 +800,7 @@ static void pr_rcv_build_pr(IN osm_sa_t * sa, IN const osm_port_t * p_src_port,
 
        /* Only set HopLimit if going through a router */
        if (is_nonzero_gid)
-               p_pr->hop_flow_raw |= cl_hton32(IB_HOPLIMIT_MAX);
+               p_pr->hop_flow_raw |= IB_HOPLIMIT_MAX;

This may fix this issue but it doesn't look right to me and I think would break router scenarios and corrupts other field(s). Path record fields are in network (not host) order.

Quoted 12 lines
opensm/opensm/osm_sa_path_record.c +0 -0
        p_pr->pkey = p_parms->pkey;
        ib_path_rec_set_sl(p_pr, p_parms->sl);
 

"hop_flow_raw" is not really a 32bit value but rather 4 fields:
Component     Length(bits)   Offset(bits)
RawTraffic    1              352
Reserved      3              353
FlowLabel     20             356
HopLimit      8              384

Quoted 2 lines

However, I don't understand the comment "Only set HopLimit if going through a router"?

Well, in a sense the HopLimit is always set. It's already set to 0 right above that code snippet in the patch where:
p_pr->hop_flow_raw &= cl_hton32(1 << 31);
This code is only setting the HopLimit to 255 (as this is very primitive so far) when going through a router. Maybe it could be stated better.

Quoted 3 lines

Was the intent to only set p_dgid in pr_rcv_get_end_points if we are heading through a router? I don't think it matters if we are going through a router or not does it?

It's used when DGID is selected in the SA PR query via the comp mask and in that instance is used for both the router and non router DGID cases.

Quoted 1 line

If not, I think the comment needs to be removed in the patch above.

I'm not following why you say this.

-- Hal

Quoted 10 lines

Thanks,
Ira

--
Ira Weiny
Math Programmer/Computer Scientist
Lawrence Livermore National Lab
925-423-8008
[email protected]

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): f0e08f230912150716y392cf1f1t4cd640b6663f7fea-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org

author
#4 lore raw
Quoted 10 lines · On Tue, 15 Dec 2009 10:16:42 -0500 Hal Rosenstock…

Ira,

Quoted 4 lines · Ira Weiny

Sasha, Hal,

I have found that the following patch caused our SRP connected storage to break.

What is causing the SRP target to fail ? Is it a non zero hop limit returned in the SA PathRecord ?

I am not sure exactly because I don't know what the SRP Target is seeing. (pesky firmware...) :-(

Quoted 8 lines
Quoted 8 lines

patch: 3d20f82edd3246879063b77721d0bcef927bdc48

   opensm/osm_sa_path_record.c: separate router guid resolution code

   Move off subnet destination (router address) resolution code to separate function to improve readability.

Quoted 8 lines
Quoted 4 lines

I further traced the problem to pr_rcv_build_pr and the patch below fixes the bug.

On quick glance, I'm missing the connection between Sasha's patch and this routine setting something bad in the hop limit of the returned path record.

It sets p_dgid even when we are not using a router which results in is_nonzero_gid being true.

Quoted 8 lines
Quoted 8 lines

16:03:34 > git diff

opensm/opensm/osm_sa_path_record.c +0 -0
index be0cd71..32c889f 100644
@@ -800,7 +800,7 @@ static void pr_rcv_build_pr(IN osm_sa_t * sa, IN const osm_port_t * p_src_port,
 
Quoted 8 lines

opensm/opensm/osm_sa_path_record.c static void pr_rcv_build_pr(IN osm_sa_t * sa, IN const osm_port_t * p_src_port, · line 803

Quoted 4 lines
opensm/opensm/osm_sa_path_record.c +1 -1
        /* Only set HopLimit if going through a router */
        if (is_nonzero_gid)
-               p_pr->hop_flow_raw |= cl_hton32(IB_HOPLIMIT_MAX);
+               p_pr->hop_flow_raw |= IB_HOPLIMIT_MAX;

This may fix this issue but it doesn't look right to me and I think would break router scenarios and corrupts other field(s). Path record fields are in network (not host) order.

But HopLimit is an 8bit field? I might be wrong on this but I was thinking this set RawTraffic and parts of Flow label to 1's. I should double check that. Also, I wonder if the FW on the other side is flipping this wrong? :-/ I have not been able to determine this and I still have to test with our other vendor's SRP target...

Quoted 13 lines
Quoted 13 lines
opensm/opensm/osm_sa_path_record.c +0 -0
        p_pr->pkey = p_parms->pkey;
        ib_path_rec_set_sl(p_pr, p_parms->sl);
 

"hop_flow_raw" is not really a 32bit value but rather 4 fields:
Component     Length(bits)   Offset(bits)
RawTraffic    1              352
Reserved      3              353
FlowLabel     20             356
HopLimit      8              384

Quoted 6 lines
Quoted 2 lines

However, I don't understand the comment "Only set HopLimit if going through a router"?

Well, in a sense the HopLimit is always set. It's already set to 0 right above that code snippet in the patch where:
p_pr->hop_flow_raw &= cl_hton32(1 << 31);

True, however, I believe the behavior before was that the entire field is 0 so the above statement really does nothing.

Quoted 3 lines

This code is only setting the HopLimit to 255 (as this is very primitive so far) when going through a router. Maybe it could be stated better.

That is exactly what Sasha's patch changes. This is not only executed when going through a router but at all times DGID is specified in the CompMask. I am willing to accept the fact that perhaps p_dgid should not be set unless we find a router, however, I __don't__ think that is correct. Searching the spec I attempted to find where HopLimit was supposed to be 0xFF only when going through a router and I could not find it; therefore my email, I'm confused again... :-(

Quoted 8 lines
Quoted 4 lines

Was the intent to only set p_dgid in pr_rcv_get_end_points if we are heading through a router? I don't think it matters if we are going through a router or not does it?

It's used when DGID is selected in the SA PR query via the comp mask and in that instance is used for both the router and non router DGID cases.

So we agree that Sasha's patch is correct?

Quoted 4 lines
Quoted 2 lines

If not, I think the comment needs to be removed in the patch above.

I'm not following why you say this.

Well, if HopLimit must only be set to 0xff when going through a router and Sasha's patch is correct, then I think we need another way to determine if this path is going through a router. In that case the comment is correct but the logic is wrong.

Ira

Quoted 13 lines

-- Hal

Quoted 10 lines

Thanks,
Ira

--
Ira Weiny
Math Programmer/Computer Scientist
Lawrence Livermore National Lab
925-423-8008
[email protected]

Signature · 9 lines
Ira Weiny
Math Programmer/Computer Scientist
Lawrence Livermore National Lab
925-423-8008
[email protected]
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): [email protected]

#6 lore raw
Quoted 30 lines · Ira Weiny
Quoted 10 lines · On Tue, 15 Dec 2009 10:16:42 -0500 Hal Rosenstock…

Ira,

Quoted 4 lines · Ira Weiny

Sasha, Hal,

I have found that the following patch caused our SRP connected storage to break.

What is causing the SRP target to fail ? Is it a non zero hop limit returned in the SA PathRecord ?

I am not sure exactly because I don't know what the SRP Target is seeing. (pesky firmware...)  :-(

Quoted 13 lines
Quoted 12 lines

patch: 3d20f82edd3246879063b77721d0bcef927bdc48

   opensm/osm_sa_path_record.c: separate router guid resolution code

   Move off subnet destination (router address) resolution code to separate function to improve readability.

I further traced the problem to pr_rcv_build_pr and the patch below fixes the bug.

Quoted 6 lines
Quoted 3 lines

On quick glance, I'm missing the connection between Sasha's patch and this routine setting something bad in the hop limit of the returned path record.

It sets p_dgid even when we are not using a router which results in is_nonzero_gid being true.

Yes, that's the cause of the issue.

Quoted 17 lines
Quoted 8 lines
Quoted 8 lines

16:03:34 > git diff

opensm/opensm/osm_sa_path_record.c +0 -0
index be0cd71..32c889f 100644
@@ -800,7 +800,7 @@ static void pr_rcv_build_pr(IN osm_sa_t * sa, IN const osm_port_t * p_src_port,
 
Quoted 8 lines

opensm/opensm/osm_sa_path_record.c static void pr_rcv_build_pr(IN osm_sa_t * sa, IN const osm_port_t * p_src_port, · line 803

Quoted 4 lines
opensm/opensm/osm_sa_path_record.c +1 -1
        /* Only set HopLimit if going through a router */
        if (is_nonzero_gid)
-               p_pr->hop_flow_raw |= cl_hton32(IB_HOPLIMIT_MAX);
+               p_pr->hop_flow_raw |= IB_HOPLIMIT_MAX;

This may fix this issue but it doesn't look right to me and I think would break router scenarios and corrupts other field(s). Path record fields are in network (not host) order.

Quoted 1 line

But HopLimit is an 8bit field?

Yes but in terms of the ib_path_rec_t definition hop_flow_raw is a 32 bit field.

Quoted 22 lines

I might be wrong on this but I was thinking
this set RawTraffic and parts of Flow label to 1's.  I should double check that.  Also, I wonder if the FW on the other side is flipping this wrong?  :-/ I have not been able to determine this and I still have to test with our other vendor's SRP target...

Quoted 16 lines
Quoted 15 lines
opensm/opensm/osm_sa_path_record.c +0 -0
        p_pr->pkey = p_parms->pkey;
        ib_path_rec_set_sl(p_pr, p_parms->sl);
 

"hop_flow_raw" is not really a 32bit value but rather 4 fields:
Component     Length(bits)   Offset(bits)
RawTraffic    1              352
Reserved      3              353
FlowLabel     20             356
HopLimit      8              384

However, I don't understand the comment "Only set HopLimit if going through a router"?

Quoted 6 lines
Quoted 3 lines

Well, in a sense the HopLimit is always set. It's already set to 0 right above that code snippet in the patch where:
p_pr->hop_flow_raw &= cl_hton32(1 << 31);

True, however, I believe the behavior before was that the entire field is 0 so the above statement really does nothing.

For all practical purposes that is true as raw traffic is deprecated.

Quoted 11 lines
Quoted 3 lines

This code is only setting the HopLimit to 255 (as this is very primitive so far) when going through a router. Maybe it could be stated better.

That is exactly what Sasha's patch changes.  This is not only executed when going through a router but at all times DGID is specified in the CompMask.  I am willing to accept the fact that perhaps p_dgid should not be set unless we find a router, however, I __don't__ think that is correct.  Searching the spec I attempted to find where HopLimit was supposed to be 0xFF only when going through a router and I could not find it; therefore my email, I'm confused again...  :-(

When going through a router, it needs to be set to something other than 0 or 1. 255 (max) was used as there's nothing currently to scope it down to something more realistic. See IBA 1.2.1 vol 1 p.229 8.3.6 for one on settings for this field.

Quoted 10 lines
Quoted 8 lines
Quoted 4 lines

Was the intent to only set p_dgid in pr_rcv_get_end_points if we are heading through a router? I don't think it matters if we are going through a router or not does it?

It's used when DGID is selected in the SA PR query via the comp mask and in that instance is used for both the router and non router DGID cases.

So we agree that Sasha's patch is correct?

Yes; it restores this back the way it used to be when it worked.

-- Hal

Quoted 27 lines
Quoted 4 lines
Quoted 2 lines

If not, I think the comment needs to be removed in the patch above.

I'm not following why you say this.

Well, if HopLimit must only be set to 0xff when going through a router and Sasha's patch is correct, then I think we need another way to determine if this path is going through a router.  In that case the comment is correct but the logic is wrong.

Ira

Quoted 13 lines

-- Hal

Quoted 10 lines

Thanks,
Ira

--
Ira Weiny
Math Programmer/Computer Scientist
Lawrence Livermore National Lab
925-423-8008
[email protected]

Quoted 7 lines

--
Ira Weiny
Math Programmer/Computer Scientist
Lawrence Livermore National Lab
925-423-8008
[email protected]

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#7 lore raw
In reply to a message not in the archive
Quoted 8 lines · Hal Rosenstock
Quoted 8 lines

patch: 3d20f82edd3246879063b77721d0bcef927bdc48

   opensm/osm_sa_path_record.c: separate router guid resolution code

   Move off subnet destination (router address) resolution code to separate function to improve readability.

Quoted 8 lines
Quoted 4 lines

I further traced the problem to pr_rcv_build_pr and the patch below fixes the bug.

On quick glance, I'm missing the connection between Sasha's patch and this routine setting something bad in the hop limit of the returned path record.

In this patch I moved *p_dgid initialization out from router only section, as side effect it turns on this section:

	/* Only set HopLimit if going through a router */
	if (is_nonzero_gid)
		p_pr->hop_flow_raw |= cl_hton32(IB_HOPLIMIT_MAX);

for all SA PR queries where DGID was selected (local subnet or not).

Sasha
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#8 lore raw
In reply to a message not in the archive

Hi Ira,

Quoted 8 lines · Ira Weiny

I have found that the following patch caused our SRP connected storage to break.

patch: 3d20f82edd3246879063b77721d0bcef927bdc48

opensm/osm_sa_path_record.c: separate router guid resolution code

Quoted 7 lines

Move off subnet destination (router address) resolution code to separate function to improve readability.

I further traced the problem to pr_rcv_build_pr and the patch below fixes the bug.

Nice catch and great investigation!

Quoted 17 lines

16:03:34 > git diff

opensm/opensm/osm_sa_path_record.c +1 -1
index be0cd71..32c889f 100644
@@ -800,7 +800,7 @@ static void pr_rcv_build_pr(IN osm_sa_t * sa, IN const osm_port_t * p_src_port,
 
        /* Only set HopLimit if going through a router */
        if (is_nonzero_gid)
-               p_pr->hop_flow_raw |= cl_hton32(IB_HOPLIMIT_MAX);
+               p_pr->hop_flow_raw |= IB_HOPLIMIT_MAX;
 
        p_pr->pkey = p_parms->pkey;
        ib_path_rec_set_sl(p_pr, p_parms->sl);
Quoted 6 lines

"hop_flow_raw" is not really a 32bit value but rather 4 fields:

Component     Length(bits)   Offset(bits)
RawTraffic    1              352
Reserved      3              353
FlowLabel     20             356
HopLimit      8              384

This patch doesn't look correct for me. Instead of placing IB_HOPLIMIT_MAX in proper place this will put it in RawTraffic and Reserved fields on little-endian machines (such as x86). And this changes nothing on a big-endian machine. Right?

Following your investigation it seems that SRP target breaks when IB_HOPLIMIT_MAX is set in SA PR query responses for intra-subnet DGIDs.

Quoted 2 lines

However, I don't understand the comment "Only set HopLimit if going through a router"?

This is from '#ifdef ROUTER_EXP' days - as far as I could understand HopLimit should be set to IB_HOPLIMIT_MAX for inter-subnet MADs.

Quoted 3 lines

Was the intent to only set p_dgid in pr_rcv_get_end_points if we are heading through a router?

Seems that this was an original logic.

Quoted 2 lines

I don't think it matters if we are going through a router or not does it?

I thought so too, but as we can see it triggers the bug you discovered.

So immediate fix for this is to move p_dgid setup back to router section, like this:

opensm/opensm/osm_sa_path_record.c +2 -3
index 7855be5..fbeef03 100644
@@ -1236,6 +1236,8 @@ static ib_net16_t pr_rcv_get_end_points(IN osm_sa_t * sa,
 				sa_status = IB_SA_MAD_STATUS_INVALID_GID;
 				goto Exit;
 			}
+			if (p_dgid)
+				*p_dgid = p_pr->dgid;
 		} else
 			dest_guid = p_pr->dgid.unicast.interface_id;
 
@@ -1253,9 +1255,6 @@ static ib_net16_t pr_rcv_get_end_points(IN osm_sa_t * sa,
 			sa_status = IB_SA_MAD_STATUS_INVALID_GID;
 			goto Exit;
 		}
-
-		if (p_dgid)
-			*p_dgid = p_pr->dgid;
 	} else {
 		*pp_dest_port = 0;
 		if (comp_mask & IB_PR_COMPMASK_DLID) {

BTW, could you test this? Unfortunately I don't have SRP target :(

And seems that as more solid solution we need to cleanup some logic in osm_sa_path_record.c code.

Sasha
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

author
#9 lore raw
Quoted 38 lines · On Tue, 15 Dec 2009 19:03:17 +0200 Sasha Khapyorsky…

Hi Ira,

Quoted 8 lines · Ira Weiny

I have found that the following patch caused our SRP connected storage to break.

patch: 3d20f82edd3246879063b77721d0bcef927bdc48

opensm/osm_sa_path_record.c: separate router guid resolution code

Quoted 7 lines

Move off subnet destination (router address) resolution code to separate function to improve readability.

I further traced the problem to pr_rcv_build_pr and the patch below fixes the bug.

Nice catch and great investigation!

Quoted 17 lines

16:03:34 > git diff

opensm/opensm/osm_sa_path_record.c +1 -1
index be0cd71..32c889f 100644
@@ -800,7 +800,7 @@ static void pr_rcv_build_pr(IN osm_sa_t * sa, IN const osm_port_t * p_src_port,
 
        /* Only set HopLimit if going through a router */
        if (is_nonzero_gid)
-               p_pr->hop_flow_raw |= cl_hton32(IB_HOPLIMIT_MAX);
+               p_pr->hop_flow_raw |= IB_HOPLIMIT_MAX;
 
        p_pr->pkey = p_parms->pkey;
        ib_path_rec_set_sl(p_pr, p_parms->sl);
Quoted 11 lines
Quoted 6 lines

"hop_flow_raw" is not really a 32bit value but rather 4 fields:

Component     Length(bits)   Offset(bits)
RawTraffic    1              352
Reserved      3              353
FlowLabel     20             356
HopLimit      8              384

This patch doesn't look correct for me. Instead of placing IB_HOPLIMIT_MAX in proper place this will put it in RawTraffic and Reserved fields on little-endian machines (such as x86). And this changes nothing on a big-endian machine. Right?

Yea I think this is wrong. As I said perhaps the FW on the other side is interpreting the fields wrong. I will check with the vendor.

Quoted 3 lines

Following your investigation it seems that SRP target breaks when IB_HOPLIMIT_MAX is set in SA PR query responses for intra-subnet DGIDs.

Yes

Quoted 6 lines
Quoted 3 lines

However, I don't understand the comment "Only set HopLimit if going through a router"?

This is from '#ifdef ROUTER_EXP' days - as far as I could understand HopLimit should be set to IB_HOPLIMIT_MAX for inter-subnet MADs.

But I have not found where that is stated.

Quoted 41 lines
Quoted 4 lines

Was the intent to only set p_dgid in pr_rcv_get_end_points if we are heading through a router?

Seems that this was an original logic.

Quoted 2 lines

I don't think it matters if we are going through a router or not does it?

I thought so too, but as we can see it triggers the bug you discovered.

So immediate fix for this is to move p_dgid setup back to router section, like this:

opensm/opensm/osm_sa_path_record.c +2 -3
index 7855be5..fbeef03 100644
@@ -1236,6 +1236,8 @@ static ib_net16_t pr_rcv_get_end_points(IN osm_sa_t * sa,
 				sa_status = IB_SA_MAD_STATUS_INVALID_GID;
 				goto Exit;
 			}
+			if (p_dgid)
+				*p_dgid = p_pr->dgid;
 		} else
 			dest_guid = p_pr->dgid.unicast.interface_id;
 
@@ -1253,9 +1255,6 @@ static ib_net16_t pr_rcv_get_end_points(IN osm_sa_t * sa,
 			sa_status = IB_SA_MAD_STATUS_INVALID_GID;
 			goto Exit;
 		}
-
-		if (p_dgid)
-			*p_dgid = p_pr->dgid;
 	} else {
 		*pp_dest_port = 0;
 		if (comp_mask & IB_PR_COMPMASK_DLID) {
Quoted 1 line

BTW, could you test this? Unfortunately I don't have SRP target :(

Yes this works and was my original fix. However, it did not seem right and I think Hal agrees.

Quoted 3 lines

And seems that as more solid solution we need to cleanup some logic in osm_sa_path_record.c code.

I think so.

Ira

Quoted 2 lines

Sasha

Signature · 9 lines
Ira Weiny
Math Programmer/Computer Scientist
Lawrence Livermore National Lab
925-423-8008
[email protected]
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#10 lore raw
In reply to Sasha Khapyorsky
Quoted 5 lines
Quoted 2 lines

However, I don't understand the comment "Only set HopLimit if going through a router"?

This is from '#ifdef ROUTER_EXP' days - as far as I could understand HopLimit should be set to IB_HOPLIMIT_MAX for inter-subnet MADs.

If HopLimit is 0 then no GRH is required, if it is non 0 then the user of that path needs to apply a GRH. IIRC there are already a few things in the kernel that follow this?

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 20091215171532.GA8288-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org

author
#12 lore raw
Quoted 9 lines · On Tue, 15 Dec 2009 10:15:32 -0700 Jason Gunthorpe…
Quoted 5 lines
Quoted 2 lines

However, I don't understand the comment "Only set HopLimit if going through a router"?

This is from '#ifdef ROUTER_EXP' days - as far as I could understand HopLimit should be set to IB_HOPLIMIT_MAX for inter-subnet MADs.

If HopLimit is 0 then no GRH is required, if it is non 0 then the user of that path needs to apply a GRH. IIRC there are already a few things in the kernel that follow this?

Is that in the spec somewhere? The follow on would be that if they specify a DGID in the PathRecord query and the result is subnet local they must still use a GRH even if the packet is going to stay local? I think this makes sense but I wonder if __everyone__ is following that?

Ira

Quoted 2 lines

Jason

Signature · 9 lines
Ira Weiny
Math Programmer/Computer Scientist
Lawrence Livermore National Lab
925-423-8008
[email protected]
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): [email protected]

#14 lore raw
Quoted 9 lines · Ira Weiny
Quoted 6 lines · On Tue, 15 Dec 2009 10:15:32 -0700 Jason Gunthorpe…
Quoted 5 lines
Quoted 2 lines

However, I don't understand the comment "Only set HopLimit if going through a router"?

This is from '#ifdef ROUTER_EXP' days - as far as I could understand HopLimit should be set to IB_HOPLIMIT_MAX for inter-subnet MADs.

Quoted 8 lines
Quoted 3 lines

If HopLimit is 0 then no GRH is required, if it is non 0 then the user of that path needs to apply a GRH. IIRC there are already a few things in the kernel that follow this?

Is that in the spec somewhere? The follow on would be that if they specify a DGID in the PathRecord query and the result is subnet local they must still use a GRH even if the packet is going to stay local? I think this makes sense but I wonder if __everyone__ is following that?

The spec provides no guidance whatsoever on how to tell if a packets generated from a PR need a GRH or not. The only other choice is to compare the resulting DGID against all the GID prefixs on the port and see if any match.

If a router spec is ever written I'm sure it will clarify this matter, and everyone will follow.

The SM decides what hop limit should be so the SM decides if it needs a GRH. I belive the original long ago version of this always set the hop limit to 0 unless the DLID was a router port.

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 20091215173140.GB8288-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org

author
#16 lore raw

On Tue, 15 Dec 2009 10:31:40 -0700
Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org> wrote:

[snip]

Quoted 6 lines

The spec provides no guidance whatsoever on how to tell if a packets generated from a PR need a GRH or not. The only other choice is to compare the resulting DGID against all the GID prefixs on the port and see if any match.

Quoted 6 lines

If a router spec is ever written I'm sure it will clarify this matter, and everyone will follow.

The SM decides what hop limit should be so the SM decides if it needs a GRH. I belive the original long ago version of this always set the hop limit to 0 unless the DLID was a router port.

Ok, then I think we need a new way to tell if we are hitting a router port in pr_rcv_build_pr. I don't think having a non-zero GID is the definition of going through a router port.

Ira

Quoted 2 lines

Jason

Signature · 9 lines
Ira Weiny
Math Programmer/Computer Scientist
Lawrence Livermore National Lab
925-423-8008
[email protected]
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#17 lore raw
In reply to a message not in the archive
Quoted 6 lines · Ira Weiny
Quoted 3 lines · On Tue, 15 Dec 2009 10:15:32 -0700 Jason Gunthorpe…
Quoted 3 lines
Quoted 2 lines

However, I don't understand the comment "Only set HopLimit if going through a router"?

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

This is from '#ifdef ROUTER_EXP' days - as far as I could understand HopLimit should be set to IB_HOPLIMIT_MAX for inter-subnet MADs.

If HopLimit is 0 then no GRH is required, if it is non 0 then the user of that path needs to apply a GRH. IIRC there are already a few things in the kernel that follow this?

Is that in the spec somewhere?

IBA 1.2.1 vol 1 p.229 8.3.6 for one.

Quoted 3 lines

The follow on would be that if they specify a
DGID in the PathRecord query and the result is subnet local they must still use a GRH even if the packet is going to stay local?

No; with subnet local traffic, GRH is optional.

-- Hal

Quoted 9 lines

 I think this makes sense
but I wonder if __everyone__ is following that?

Ira

Quoted 2 lines

Jason

Quoted 7 lines

--
Ira Weiny
Math Programmer/Computer Scientist
Lawrence Livermore National Lab
925-423-8008
[email protected]

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): f0e08f230912150959j536667bbg51b8381724681880-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org

author
#19 lore raw

RFC PATCH 1/1 Set HopLimit based on off subnet Re: SRP issues with OpenSM 3.3.3

Quoted 1 line · On Tue, 15 Dec 2009 12:59:11 -0500 Hal Rosenstock…

On Tue, Dec 15, 2009 at 12:18 PM, Ira Weiny <[email protected]> wrote:

[snip]

Quoted 7 lines
Quoted 4 lines

The follow on would be that if they specify a
DGID in the PathRecord query and the result is subnet local they must still use a GRH even if the packet is going to stay local?

No; with subnet local traffic, GRH is optional.

Ok, if the GRH is optional with local traffic then I think the appropriate fix is:

opensm/opensm/osm_sa_path_record.c +10 -9
index be0cd71..1fa83a1 100644
@@ -757,6 +757,14 @@ Exit:
        return (status);
 }
 
+static int gid_is_off_subnet(IN osm_sa_t * sa, IN const ib_gid_t * p_dgid)
+{
+       return (!ib_gid_is_link_local(p_dgid) &&
+               !ib_gid_is_multicast(p_dgid) &&
+               ib_gid_get_subnet_prefix(p_dgid) !=
+               sa->p_subn->opt.subnet_prefix);
+}
+
 /**********************************************************************
  **********************************************************************/
 static void pr_rcv_build_pr(IN osm_sa_t * sa, IN const osm_port_t * p_src_port,
@@ -770,16 +778,12 @@ static void pr_rcv_build_pr(IN osm_sa_t * sa, IN const osm_port_t * p_src_port,
 {
        const osm_physp_t *p_src_physp;
        const osm_physp_t *p_dest_physp;
-       boolean_t is_nonzero_gid = 0;
 
        OSM_LOG_ENTER(sa->p_log);
 
        p_src_physp = p_src_port->p_physp;
 
        if (p_dgid)
-               is_nonzero_gid = ib_gid_is_notzero(p_dgid);
-
-       if (is_nonzero_gid)
                p_pr->dgid = *p_dgid;
        else {
                p_dest_physp = p_dest_port->p_physp;
@@ -799,7 +803,7 @@ static void pr_rcv_build_pr(IN osm_sa_t * sa, IN const osm_port_t * p_src_port,
        p_pr->hop_flow_raw &= cl_hton32(1 << 31);
 
        /* Only set HopLimit if going through a router */
-       if (is_nonzero_gid)
+       if (gid_is_off_subnet(sa, &p_pr->dgid))
                p_pr->hop_flow_raw |= cl_hton32(IB_HOPLIMIT_MAX);
 
        p_pr->pkey = p_parms->pkey;
@@ -1248,10 +1252,7 @@ static ib_net16_t pr_rcv_get_end_points(IN osm_sa_t * sa,
                memset(p_dgid, 0, sizeof(*p_dgid));
 
        if (comp_mask & IB_PR_COMPMASK_DGID) {
-               if (!ib_gid_is_link_local(&p_pr->dgid) &&
-                   !ib_gid_is_multicast(&p_pr->dgid) &&
-                   ib_gid_get_subnet_prefix(&p_pr->dgid) !=
-                   sa->p_subn->opt.subnet_prefix) {
+               if (gid_is_off_subnet(sa, &p_pr->dgid)) {
                        dest_guid = find_router(sa, p_pr->dgid.unicast.prefix);
                        if (!dest_guid) {
                                char gid_str[INET6_ADDRSTRLEN];

This maintains the p_dgid found in pr_rcv_get_end_points but appropriately set's the HopLimit when the DGID is off subnet.

Ira

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): [email protected]

#21 lore raw

Re: [RFC PATCH] Set HopLimit based on off subnet Re: SRP issues with OpenSM 3.3.3

Quoted 77 lines · Ira Weiny
Quoted 1 line · On Tue, 15 Dec 2009 12:59:11 -0500 Hal Rosenstock…

On Tue, Dec 15, 2009 at 12:18 PM, Ira Weiny <[email protected]> wrote:

[snip]

Quoted 7 lines
Quoted 4 lines

The follow on would be that if they specify a
DGID in the PathRecord query and the result is subnet local they must still use a GRH even if the packet is going to stay local?

No; with subnet local traffic, GRH is optional.

Ok, if the GRH is optional with local traffic then I think the appropriate fix is:

opensm/opensm/osm_sa_path_record.c +10 -9
index be0cd71..1fa83a1 100644
@@ -757,6 +757,14 @@ Exit:
        return (status);
  }
 
+static int gid_is_off_subnet(IN osm_sa_t * sa, IN const ib_gid_t * p_dgid)
+{
+       return (!ib_gid_is_link_local(p_dgid) &&
+               !ib_gid_is_multicast(p_dgid) &&
+               ib_gid_get_subnet_prefix(p_dgid) !=
+               sa->p_subn->opt.subnet_prefix);
+}
+
  /**********************************************************************
  **********************************************************************/
  static void pr_rcv_build_pr(IN osm_sa_t * sa, IN const osm_port_t * p_src_port,
@@ -770,16 +778,12 @@ static void pr_rcv_build_pr(IN osm_sa_t * sa, IN const osm_port_t * p_src_port,
  {
        const osm_physp_t *p_src_physp;
        const osm_physp_t *p_dest_physp;
-       boolean_t is_nonzero_gid = 0;
 
        OSM_LOG_ENTER(sa->p_log);
 
        p_src_physp = p_src_port->p_physp;
 
        if (p_dgid)
-               is_nonzero_gid = ib_gid_is_notzero(p_dgid);
-
-       if (is_nonzero_gid)
                p_pr->dgid = *p_dgid;
        else {
                p_dest_physp = p_dest_port->p_physp;
@@ -799,7 +803,7 @@ static void pr_rcv_build_pr(IN osm_sa_t * sa, IN const osm_port_t * p_src_port,
        p_pr->hop_flow_raw &= cl_hton32(1 << 31);
 
        /* Only set HopLimit if going through a router */
-       if (is_nonzero_gid)
+       if (gid_is_off_subnet(sa, &p_pr->dgid))
                p_pr->hop_flow_raw |= cl_hton32(IB_HOPLIMIT_MAX);
 
        p_pr->pkey = p_parms->pkey;
@@ -1248,10 +1252,7 @@ static ib_net16_t pr_rcv_get_end_points(IN osm_sa_t * sa,
                memset(p_dgid, 0, sizeof(*p_dgid));
 
        if (comp_mask & IB_PR_COMPMASK_DGID) {
-               if (!ib_gid_is_link_local(&p_pr->dgid) &&
-                   !ib_gid_is_multicast(&p_pr->dgid) &&
-                   ib_gid_get_subnet_prefix(&p_pr->dgid) !=
-                   sa->p_subn->opt.subnet_prefix) {
+               if (gid_is_off_subnet(sa, &p_pr->dgid)) {
                        dest_guid = find_router(sa, p_pr->dgid.unicast.prefix);
                        if (!dest_guid) {
                                char gid_str[INET6_ADDRSTRLEN];
Quoted 2 lines

This maintains the p_dgid found in pr_rcv_get_end_points but appropriately set's the HopLimit when the DGID is off subnet.

Yes, this looks right to me. Would it be better to only determine gid_is_off_subnet once by saving this in a variable and passing it as a parameter where needed ?

-- Hal

Quoted 4 lines

Ira

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): f0e08f230912160529h64424ba7id5c57dffb770380c-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org

author
#23 lore raw

Re: [RFC PATCH] Set HopLimit based on off subnet Re: SRP issues with OpenSM 3.3.3

Quoted 1 line · On Wed, 16 Dec 2009 08:29:43 -0500 Hal Rosenstock…

On Tue, Dec 15, 2009 at 9:55 PM, Ira Weiny <[email protected]> wrote:

[snip]

Quoted 60 lines
Quoted 60 lines
opensm/opensm/osm_sa_path_record.c +10 -9
index be0cd71..1fa83a1 100644
@@ -757,6 +757,14 @@ Exit:
        return (status);
  }
 
+static int gid_is_off_subnet(IN osm_sa_t * sa, IN const ib_gid_t * p_dgid)
+{
+       return (!ib_gid_is_link_local(p_dgid) &&
+               !ib_gid_is_multicast(p_dgid) &&
+               ib_gid_get_subnet_prefix(p_dgid) !=
+               sa->p_subn->opt.subnet_prefix);
+}
+
  /**********************************************************************
  **********************************************************************/
  static void pr_rcv_build_pr(IN osm_sa_t * sa, IN const osm_port_t * p_src_port,
@@ -770,16 +778,12 @@ static void pr_rcv_build_pr(IN osm_sa_t * sa, IN const osm_port_t * p_src_port,
  {
        const osm_physp_t *p_src_physp;
        const osm_physp_t *p_dest_physp;
-       boolean_t is_nonzero_gid = 0;
 
        OSM_LOG_ENTER(sa->p_log);
 
        p_src_physp = p_src_port->p_physp;
 
        if (p_dgid)
-               is_nonzero_gid = ib_gid_is_notzero(p_dgid);
-
-       if (is_nonzero_gid)
                p_pr->dgid = *p_dgid;
        else {
                p_dest_physp = p_dest_port->p_physp;
@@ -799,7 +803,7 @@ static void pr_rcv_build_pr(IN osm_sa_t * sa, IN const osm_port_t * p_src_port,
        p_pr->hop_flow_raw &= cl_hton32(1 << 31);
 
        /* Only set HopLimit if going through a router */
-       if (is_nonzero_gid)
+       if (gid_is_off_subnet(sa, &p_pr->dgid))
                p_pr->hop_flow_raw |= cl_hton32(IB_HOPLIMIT_MAX);
 
        p_pr->pkey = p_parms->pkey;
@@ -1248,10 +1252,7 @@ static ib_net16_t pr_rcv_get_end_points(IN osm_sa_t * sa,
                memset(p_dgid, 0, sizeof(*p_dgid));
 
        if (comp_mask & IB_PR_COMPMASK_DGID) {
-               if (!ib_gid_is_link_local(&p_pr->dgid) &&
-                   !ib_gid_is_multicast(&p_pr->dgid) &&
-                   ib_gid_get_subnet_prefix(&p_pr->dgid) !=
-                   sa->p_subn->opt.subnet_prefix) {
+               if (gid_is_off_subnet(sa, &p_pr->dgid)) {
                        dest_guid = find_router(sa, p_pr->dgid.unicast.prefix);
                        if (!dest_guid) {
                                char gid_str[INET6_ADDRSTRLEN];
Quoted 7 lines
Quoted 2 lines

This maintains the p_dgid found in pr_rcv_get_end_points but appropriately set's the HopLimit when the DGID is off subnet.

Yes, this looks right to me. Would it be better to only determine gid_is_off_subnet once by saving this in a variable and passing it as a parameter where needed ?

It might be faster but it would mean a bigger change. It looks like having the dgid or p_pr structures in some internal opensm structure to carry that information along would be best but that is a lot of work.

Do you think this will adversely affect performance on a big system? I plan to roll this out internally and we will have a chance to test on bigger systems here. But there are larger out there now.

Ira

Signature · 9 lines
Ira Weiny
Math Programmer/Computer Scientist
Lawrence Livermore National Lab
925-423-8008
[email protected]
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): [email protected]

#25 lore raw

Re: [RFC PATCH] Set HopLimit based on off subnet Re: SRP issues with OpenSM 3.3.3

Quoted 75 lines · Ira Weiny
Quoted 1 line · On Wed, 16 Dec 2009 08:29:43 -0500 Hal Rosenstock…

On Tue, Dec 15, 2009 at 9:55 PM, Ira Weiny <[email protected]> wrote:

[snip]

Quoted 60 lines
Quoted 60 lines
opensm/opensm/osm_sa_path_record.c +10 -9
index be0cd71..1fa83a1 100644
@@ -757,6 +757,14 @@ Exit:
        return (status);
  }
 
+static int gid_is_off_subnet(IN osm_sa_t * sa, IN const ib_gid_t * p_dgid)
+{
+       return (!ib_gid_is_link_local(p_dgid) &&
+               !ib_gid_is_multicast(p_dgid) &&
+               ib_gid_get_subnet_prefix(p_dgid) !=
+               sa->p_subn->opt.subnet_prefix);
+}
+
  /**********************************************************************
  **********************************************************************/
  static void pr_rcv_build_pr(IN osm_sa_t * sa, IN const osm_port_t * p_src_port,
@@ -770,16 +778,12 @@ static void pr_rcv_build_pr(IN osm_sa_t * sa, IN const osm_port_t * p_src_port,
  {
        const osm_physp_t *p_src_physp;
        const osm_physp_t *p_dest_physp;
-       boolean_t is_nonzero_gid = 0;
 
        OSM_LOG_ENTER(sa->p_log);
 
        p_src_physp = p_src_port->p_physp;
 
        if (p_dgid)
-               is_nonzero_gid = ib_gid_is_notzero(p_dgid);
-
-       if (is_nonzero_gid)
                p_pr->dgid = *p_dgid;
        else {
                p_dest_physp = p_dest_port->p_physp;
@@ -799,7 +803,7 @@ static void pr_rcv_build_pr(IN osm_sa_t * sa, IN const osm_port_t * p_src_port,
        p_pr->hop_flow_raw &= cl_hton32(1 << 31);
 
        /* Only set HopLimit if going through a router */
-       if (is_nonzero_gid)
+       if (gid_is_off_subnet(sa, &p_pr->dgid))
                p_pr->hop_flow_raw |= cl_hton32(IB_HOPLIMIT_MAX);
 
        p_pr->pkey = p_parms->pkey;
@@ -1248,10 +1252,7 @@ static ib_net16_t pr_rcv_get_end_points(IN osm_sa_t * sa,
                memset(p_dgid, 0, sizeof(*p_dgid));
 
        if (comp_mask & IB_PR_COMPMASK_DGID) {
-               if (!ib_gid_is_link_local(&p_pr->dgid) &&
-                   !ib_gid_is_multicast(&p_pr->dgid) &&
-                   ib_gid_get_subnet_prefix(&p_pr->dgid) !=
-                   sa->p_subn->opt.subnet_prefix) {
+               if (gid_is_off_subnet(sa, &p_pr->dgid)) {
                        dest_guid = find_router(sa, p_pr->dgid.unicast.prefix);
                        if (!dest_guid) {
                                char gid_str[INET6_ADDRSTRLEN];
Quoted 7 lines
Quoted 2 lines

This maintains the p_dgid found in pr_rcv_get_end_points but appropriately set's the HopLimit when the DGID is off subnet.

Yes, this looks right to me. Would it be better to only determine gid_is_off_subnet once by saving this in a variable and passing it as a parameter where needed ?

Quoted 1 line

It might be faster but it would mean a bigger change.

Yes.

Quoted 3 lines

It looks like having
the dgid or p_pr structures in some internal opensm structure to carry that information along would be best but that is a lot of work.

It could also be done by passing this as an additional parameter through the necessary routines.

Quoted 1 line

Do you think this will adversely affect performance on a big system?

I don't know as I haven't measured the difference in the two approaches. Just trying to minimize additions to computing PRs.

-- Hal

Quoted 7 lines

I plan
to roll this out internally and we will have a chance to test on bigger systems here.  But there are larger out there now.

Ira

Quoted 7 lines

--
Ira Weiny
Math Programmer/Computer Scientist
Lawrence Livermore National Lab
925-423-8008
[email protected]

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#26 lore raw

Re: [RFC PATCH] Set HopLimit based on off subnet Re: SRP issues with OpenSM 3.3.3

In reply to a message not in the archive
Quoted 4 lines · Ira Weiny

It might be faster but it would mean a bigger change. It looks like having the dgid or p_pr structures in some internal opensm structure to carry that information along would be best but that is a lot of work.

Actually now dgid parameter is never used when intra-subnet PR is requested - maybe those parameters (dgid, is_off_subnet, etc.) could be merged into some context struct which is initialized once and passed down to the functions.

Likely some general cleanup of osm_sa_path_record.c is needed to see a most obvious coding solution.

Sasha
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#27 lore raw

PATCH 1/1 opensm/osm_sa_path_record.c: MGID must be specified explicitly

IBA 1.2.1 states (Vol.1, 15.2.5.16, p.918) that DGID shell be explicitly specified when path destination of SA PathRecord query is multicast group. So ignore all other cases when choosing multicast PR destination. Other related parameters (mlid) is checked in pr_match_mgrp_attributes() anyway. As result consolidate many of effectively duplicate code.

---
 opensm/opensm/osm_sa_path_record.c |   88 +++++-------------------------------
 1 files changed, 11 insertions(+), 77 deletions(-)
opensm/opensm/osm_sa_path_record.c +11 -77
index fbeef03..082a41d 100644
@@ -1401,34 +1401,6 @@ static void pr_rcv_process_pair(IN osm_sa_t * sa, IN const osm_madw_t * p_madw,
 	OSM_LOG_EXIT(sa->p_log);
 }
 
-static osm_mgrp_t *pr_get_mgrp(IN osm_sa_t * sa, IN const osm_madw_t * p_madw)
-{
-	ib_path_rec_t *p_pr;
-	const ib_sa_mad_t *p_sa_mad;
-	osm_mgrp_t *mgrp;
-
-	p_sa_mad = osm_madw_get_sa_mad_ptr(p_madw);
-	p_pr = ib_sa_mad_get_payload_ptr(p_sa_mad);
-
-	if (!(p_sa_mad->comp_mask & IB_PR_COMPMASK_DGID)) {
-		OSM_LOG(sa->p_log, OSM_LOG_VERBOSE,
-			"discard multicast target SA PR with wildcarded MGID");
-		return NULL;
-	}
-
-	mgrp = osm_get_mgrp_by_mgid(sa->p_subn, &p_pr->dgid);
-	if (!mgrp) {
-		char gid_str[INET6_ADDRSTRLEN];
-		OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 1F09: "
-			"No MC group found for PathRecord destination GID %s\n",
-			inet_ntop(AF_INET6, p_pr->dgid.raw, gid_str,
-				  sizeof gid_str));
-		return NULL;
-	}
-
-	return mgrp;
-}
-
 static ib_api_status_t pr_match_mgrp_attributes(IN osm_sa_t * sa,
 						IN const osm_madw_t * p_madw,
 						IN const osm_mgrp_t * p_mgrp)
@@ -1510,47 +1482,11 @@ Exit:
 	return status;
 }
 
-static int pr_rcv_check_mcast_dest(osm_sa_t * sa, IN const osm_madw_t * p_madw)
-{
-	const ib_path_rec_t *p_pr;
-	const ib_sa_mad_t *p_sa_mad;
-	ib_net64_t comp_mask;
-	int is_multicast = 0;
-
-	OSM_LOG_ENTER(sa->p_log);
-
-	p_sa_mad = osm_madw_get_sa_mad_ptr(p_madw);
-	p_pr = (ib_path_rec_t *) ib_sa_mad_get_payload_ptr(p_sa_mad);
-
-	comp_mask = p_sa_mad->comp_mask;
-
-	if (comp_mask & IB_PR_COMPMASK_DGID) {
-		is_multicast = ib_gid_is_multicast(&p_pr->dgid);
-		if (!is_multicast)
-			goto Exit;
-	}
-
-	if (comp_mask & IB_PR_COMPMASK_DLID) {
-		if (cl_ntoh16(p_pr->dlid) >= IB_LID_MCAST_START_HO &&
-		    cl_ntoh16(p_pr->dlid) <= IB_LID_MCAST_END_HO)
-			is_multicast = 1;
-		else if (is_multicast) {
-			OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 1F12: "
-				"PathRecord request indicates MGID but not MLID\n");
-			is_multicast = -1;
-		}
-	}
-
-Exit:
-	OSM_LOG_EXIT(sa->p_log);
-	return is_multicast;
-}
-
 void osm_pr_rcv_process(IN void *context, IN void *data)
 {
 	osm_sa_t *sa = context;
 	osm_madw_t *p_madw = data;
-	const ib_path_rec_t *p_pr;
+	ib_path_rec_t *p_pr;
 	const ib_sa_mad_t *p_sa_mad;
 	const osm_port_t *p_src_port;
 	const osm_port_t *p_dest_port;
@@ -1558,7 +1494,6 @@ void osm_pr_rcv_process(IN void *context, IN void *data)
 	ib_gid_t dgid;
 	ib_net16_t sa_status;
 	osm_port_t *requester_port;
-	int ret;
 
 	OSM_LOG_ENTER(sa->p_log);
 
@@ -1601,21 +1536,14 @@ void osm_pr_rcv_process(IN void *context, IN void *data)
 	cl_plock_acquire(sa->p_lock);
 
 	/* Handle multicast destinations separately */
-	if ((ret = pr_rcv_check_mcast_dest(sa, p_madw)) < 0) {
-		/* Multicast DGID with unicast DLID */
-		cl_plock_release(sa->p_lock);
-		osm_sa_send_error(sa, p_madw, IB_MAD_STATUS_INVALID_FIELD);
-		goto Exit;
-	}
-
-	if (ret > 0)
+	if ((p_sa_mad->comp_mask & IB_PR_COMPMASK_DGID) &&
+	    ib_gid_is_multicast(&p_pr->dgid));
 		goto McastDest;
 
 	OSM_LOG(sa->p_log, OSM_LOG_DEBUG, "Unicast destination requested\n");
 
 	sa_status = pr_rcv_get_end_points(sa, p_madw, &p_src_port, &p_dest_port,
 					  &dgid);
-
 	if (sa_status == IB_SA_MAD_STATUS_SUCCESS) {
 		/*
 		   What happens next depends on the type of endpoint information
@@ -1660,9 +1588,15 @@ McastDest:
 		uint8_t hop_limit;
 
 		/* First, get the MC info */
-		p_mgrp = pr_get_mgrp(sa, p_madw);
-		if (!p_mgrp)
+		p_mgrp = osm_get_mgrp_by_mgid(sa->p_subn, &p_pr->dgid);
+		if (!p_mgrp) {
+			char gid_str[INET6_ADDRSTRLEN];
+			OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 1F09: "
+				"No MC group found for PathRecord destination GID %s\n",
+				inet_ntop(AF_INET6, p_pr->dgid.raw, gid_str,
+					  sizeof gid_str));
 			goto Unlock;
+		}
 
 		/* Make sure the rest of the PathRecord matches the MC group attributes */
 		status = pr_match_mgrp_attributes(sa, p_madw, p_mgrp);
Signature · 6 lines
1.6.6.rc4

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#28 lore raw

PATCH 1/1 osm_sa_path_record.c: separate mutlicast processing code

Move multicast destination PR processing into separate function pr_process_multicast(). As result also merge some duplicated code.

---
 opensm/opensm/osm_sa_path_record.c |  162 +++++++++++++++++------------------
 1 files changed, 79 insertions(+), 83 deletions(-)
opensm/opensm/osm_sa_path_record.c +79 -83
index 082a41d..b78225e 100644
@@ -1402,11 +1402,10 @@ static void pr_rcv_process_pair(IN osm_sa_t * sa, IN const osm_madw_t * p_madw,
 }
 
 static ib_api_status_t pr_match_mgrp_attributes(IN osm_sa_t * sa,
-						IN const osm_madw_t * p_madw,
+						IN const ib_sa_mad_t * sa_mad,
 						IN const osm_mgrp_t * p_mgrp)
 {
 	const ib_path_rec_t *p_pr;
-	const ib_sa_mad_t *p_sa_mad;
 	ib_net64_t comp_mask;
 	const osm_port_t *port;
 	ib_api_status_t status = IB_ERROR;
@@ -1416,10 +1415,9 @@ static ib_api_status_t pr_match_mgrp_attributes(IN osm_sa_t * sa,
 
 	OSM_LOG_ENTER(sa->p_log);
 
-	p_sa_mad = osm_madw_get_sa_mad_ptr(p_madw);
-	p_pr = ib_sa_mad_get_payload_ptr(p_sa_mad);
+	p_pr = ib_sa_mad_get_payload_ptr(sa_mad);
 
-	comp_mask = p_sa_mad->comp_mask;
+	comp_mask = sa_mad->comp_mask;
 
 	/* check that MLID of the MC group matches the PathRecord DLID */
 	if ((comp_mask & IB_PR_COMPMASK_DLID) && p_mgrp->mlid != p_pr->dlid)
@@ -1455,7 +1453,7 @@ static ib_api_status_t pr_match_mgrp_attributes(IN osm_sa_t * sa,
 
 	/* If SubnAdmGet, assume NumbPaths of 1 (1.2 erratum) */
 	if ((comp_mask & IB_PR_COMPMASK_NUMBPATH) &&
-	    (p_sa_mad->method != IB_MAD_METHOD_GET)) {
+	    (sa_mad->method != IB_MAD_METHOD_GET)) {
 		if (ib_path_rec_num_path(p_pr) == 0)
 			goto Exit;
 	}
@@ -1482,6 +1480,77 @@ Exit:
 	return status;
 }
 
+static void pr_process_multicast(osm_sa_t * sa, const ib_sa_mad_t *sa_mad,
+				 cl_qlist_t *list)
+{
+	ib_path_rec_t *pr = ib_sa_mad_get_payload_ptr(sa_mad);
+	osm_mgrp_t *mgrp;
+	ib_api_status_t status;
+	osm_pr_item_t *pr_item;
+	uint32_t flow_label;
+	uint8_t sl, hop_limit;
+
+	OSM_LOG(sa->p_log, OSM_LOG_DEBUG, "Multicast destination requested\n");
+
+	mgrp = osm_get_mgrp_by_mgid(sa->p_subn, &pr->dgid);
+	if (!mgrp) {
+		char gid_str[INET6_ADDRSTRLEN];
+		OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 1F09: "
+			"No MC group found for PathRecord destination GID %s\n",
+			inet_ntop(AF_INET6, pr->dgid.raw, gid_str,
+				  sizeof gid_str));
+		return;
+	}
+
+	/* Make sure the rest of the PathRecord matches the MC group attributes */
+	status = pr_match_mgrp_attributes(sa, sa_mad, mgrp);
+	if (status != IB_SUCCESS) {
+		OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 1F19: "
+			"MC group attributes don't match PathRecord request\n");
+		return;
+	}
+
+	pr_item = malloc(sizeof(*pr_item));
+	if (pr_item == NULL) {
+		OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 1F18: "
+			"Unable to allocate path record for MC group\n");
+		return;
+	}
+	memset(pr_item, 0, sizeof(*pr_item));
+
+	/* Copy PathRecord request into response */
+	pr_item->path_rec = *pr;
+
+	/* Now, use the MC info to cruft up the PathRecord response */
+	pr_item->path_rec.dgid = mgrp->mcmember_rec.mgid;
+	pr_item->path_rec.dlid = mgrp->mcmember_rec.mlid;
+	pr_item->path_rec.tclass = mgrp->mcmember_rec.tclass;
+	pr_item->path_rec.num_path = 1;
+	pr_item->path_rec.pkey = mgrp->mcmember_rec.pkey;
+
+	/* MTU, rate, and packet lifetime should be exactly */
+	pr_item->path_rec.mtu = (2 << 6) | mgrp->mcmember_rec.mtu;
+	pr_item->path_rec.rate = (2 << 6) | mgrp->mcmember_rec.rate;
+	pr_item->path_rec.pkt_life = (2 << 6) | mgrp->mcmember_rec.pkt_life;
+
+	/* SL, Hop Limit, and Flow Label */
+	ib_member_get_sl_flow_hop(mgrp->mcmember_rec.sl_flow_hop,
+				  &sl, &flow_label, &hop_limit);
+	ib_path_rec_set_sl(&pr_item->path_rec, sl);
+	ib_path_rec_set_qos_class(&pr_item->path_rec, 0);
+
+	/* HopLimit is not yet set in non link local MC groups */
+	/* If it were, this would not be needed */
+	if (ib_mgid_get_scope(&mgrp->mcmember_rec.mgid) !=
+	    IB_MC_SCOPE_LINK_LOCAL)
+		hop_limit = IB_HOPLIMIT_MAX;
+
+	pr_item->path_rec.hop_flow_raw =
+	    cl_hton32(hop_limit) | (flow_label << 8);
+
+	cl_qlist_insert_tail(list, &pr_item->list_item);
+}
+
 void osm_pr_rcv_process(IN void *context, IN void *data)
 {
 	osm_sa_t *sa = context;
@@ -1537,8 +1606,10 @@ void osm_pr_rcv_process(IN void *context, IN void *data)
 
 	/* Handle multicast destinations separately */
 	if ((p_sa_mad->comp_mask & IB_PR_COMPMASK_DGID) &&
-	    ib_gid_is_multicast(&p_pr->dgid));
-		goto McastDest;
+	    ib_gid_is_multicast(&p_pr->dgid)) {
+		pr_process_multicast(sa, p_sa_mad, &pr_list);
+		goto Unlock;
+	}
 
 	OSM_LOG(sa->p_log, OSM_LOG_DEBUG, "Unicast destination requested\n");
 
@@ -1575,81 +1646,6 @@ void osm_pr_rcv_process(IN void *context, IN void *data)
 						     &pr_list);
 		}
 	}
-	goto Unlock;
-
-McastDest:
-	OSM_LOG(sa->p_log, OSM_LOG_DEBUG, "Multicast destination requested\n");
-	{
-		osm_mgrp_t *p_mgrp = NULL;
-		ib_api_status_t status;
-		osm_pr_item_t *p_pr_item;
-		uint32_t flow_label;
-		uint8_t sl;
-		uint8_t hop_limit;
-
-		/* First, get the MC info */
-		p_mgrp = osm_get_mgrp_by_mgid(sa->p_subn, &p_pr->dgid);
-		if (!p_mgrp) {
-			char gid_str[INET6_ADDRSTRLEN];
-			OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 1F09: "
-				"No MC group found for PathRecord destination GID %s\n",
-				inet_ntop(AF_INET6, p_pr->dgid.raw, gid_str,
-					  sizeof gid_str));
-			goto Unlock;
-		}
-
-		/* Make sure the rest of the PathRecord matches the MC group attributes */
-		status = pr_match_mgrp_attributes(sa, p_madw, p_mgrp);
-		if (status != IB_SUCCESS) {
-			OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 1F19: "
-				"MC group attributes don't match PathRecord request\n");
-			goto Unlock;
-		}
-
-		p_pr_item = malloc(sizeof(*p_pr_item));
-		if (p_pr_item == NULL) {
-			OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 1F18: "
-				"Unable to allocate path record for MC group\n");
-			goto Unlock;
-		}
-		memset(p_pr_item, 0, sizeof(*p_pr_item));
-
-		/* Copy PathRecord request into response */
-		p_sa_mad = osm_madw_get_sa_mad_ptr(p_madw);
-		p_pr = (ib_path_rec_t *)
-		    ib_sa_mad_get_payload_ptr(p_sa_mad);
-		p_pr_item->path_rec = *p_pr;
-
-		/* Now, use the MC info to cruft up the PathRecord response */
-		p_pr_item->path_rec.dgid = p_mgrp->mcmember_rec.mgid;
-		p_pr_item->path_rec.dlid = p_mgrp->mcmember_rec.mlid;
-		p_pr_item->path_rec.tclass = p_mgrp->mcmember_rec.tclass;
-		p_pr_item->path_rec.num_path = 1;
-		p_pr_item->path_rec.pkey = p_mgrp->mcmember_rec.pkey;
-
-		/* MTU, rate, and packet lifetime should be exactly */
-		p_pr_item->path_rec.mtu = (2 << 6) | p_mgrp->mcmember_rec.mtu;
-		p_pr_item->path_rec.rate = (2 << 6) | p_mgrp->mcmember_rec.rate;
-		p_pr_item->path_rec.pkt_life =
-		    (2 << 6) | p_mgrp->mcmember_rec.pkt_life;
-
-		/* SL, Hop Limit, and Flow Label */
-		ib_member_get_sl_flow_hop(p_mgrp->mcmember_rec.sl_flow_hop,
-					  &sl, &flow_label, &hop_limit);
-		ib_path_rec_set_sl(&p_pr_item->path_rec, sl);
-		ib_path_rec_set_qos_class(&p_pr_item->path_rec, 0);
-
-		/* HopLimit is not yet set in non link local MC groups */
-		/* If it were, this would not be needed */
-		if (ib_mgid_get_scope(&p_mgrp->mcmember_rec.mgid) !=
-		    IB_MC_SCOPE_LINK_LOCAL)
-			hop_limit = IB_HOPLIMIT_MAX;
-
-		p_pr_item->path_rec.hop_flow_raw =
-		    cl_hton32(hop_limit) | (flow_label << 8);
-
-		cl_qlist_insert_tail(&pr_list, &p_pr_item->list_item);
-	}
 
 Unlock:
 	cl_plock_release(sa->p_lock);
Signature · 6 lines
1.6.6.rc4

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#29 lore raw

PATCH 1/1 osm_sa_path_record.c: use PR DGID by reference

When passing an original value of DGID requested (currently only for inter-subnet PR requests) use it by reference instead of copied value.

---
 opensm/opensm/osm_sa_path_record.c |   27 ++++++++++-----------------
 1 files changed, 10 insertions(+), 17 deletions(-)
opensm/opensm/osm_sa_path_record.c +10 -17
index b78225e..4614b6d 100644
@@ -754,16 +754,12 @@ static void pr_rcv_build_pr(IN osm_sa_t * sa, IN const osm_port_t * p_src_port,
 {
 	const osm_physp_t *p_src_physp;
 	const osm_physp_t *p_dest_physp;
-	boolean_t is_nonzero_gid = 0;
 
 	OSM_LOG_ENTER(sa->p_log);
 
 	p_src_physp = p_src_port->p_physp;
 
 	if (p_dgid)
-		is_nonzero_gid = ib_gid_is_notzero(p_dgid);
-
-	if (is_nonzero_gid)
 		p_pr->dgid = *p_dgid;
 	else {
 		p_dest_physp = p_dest_port->p_physp;
@@ -783,7 +779,7 @@ static void pr_rcv_build_pr(IN osm_sa_t * sa, IN const osm_port_t * p_src_port,
 	p_pr->hop_flow_raw &= cl_hton32(1 << 31);
 
 	/* Only set HopLimit if going through a router */
-	if (is_nonzero_gid)
+	if (p_dgid)
 		p_pr->hop_flow_raw |= cl_hton32(IB_HOPLIMIT_MAX);
 
 	p_pr->pkey = p_parms->pkey;
@@ -1133,7 +1129,7 @@ static ib_net16_t pr_rcv_get_end_points(IN osm_sa_t * sa,
 					IN const osm_madw_t * p_madw,
 					OUT const osm_port_t ** pp_src_port,
 					OUT const osm_port_t ** pp_dest_port,
-					OUT ib_gid_t * p_dgid)
+					OUT const ib_gid_t ** pp_dgid)
 {
 	const ib_path_rec_t *p_pr;
 	const ib_sa_mad_t *p_sa_mad;
@@ -1217,9 +1213,6 @@ static ib_net16_t pr_rcv_get_end_points(IN osm_sa_t * sa,
 		}
 	}
 
-	if (p_dgid)
-		memset(p_dgid, 0, sizeof(*p_dgid));
-
 	if (comp_mask & IB_PR_COMPMASK_DGID) {
 		if (!ib_gid_is_link_local(&p_pr->dgid) &&
 		    !ib_gid_is_multicast(&p_pr->dgid) &&
@@ -1236,8 +1229,8 @@ static ib_net16_t pr_rcv_get_end_points(IN osm_sa_t * sa,
 				sa_status = IB_SA_MAD_STATUS_INVALID_GID;
 				goto Exit;
 			}
-			if (p_dgid)
-				*p_dgid = p_pr->dgid;
+			if (pp_dgid)
+				*pp_dgid = &p_pr->dgid;
 		} else
 			dest_guid = p_pr->dgid.unicast.interface_id;
 
@@ -1560,7 +1553,7 @@ void osm_pr_rcv_process(IN void *context, IN void *data)
 	const osm_port_t *p_src_port;
 	const osm_port_t *p_dest_port;
 	cl_qlist_t pr_list;
-	ib_gid_t dgid;
+	const ib_gid_t *p_dgid = NULL;
 	ib_net16_t sa_status;
 	osm_port_t *requester_port;
 
@@ -1614,7 +1607,7 @@ void osm_pr_rcv_process(IN void *context, IN void *data)
 	OSM_LOG(sa->p_log, OSM_LOG_DEBUG, "Unicast destination requested\n");
 
 	sa_status = pr_rcv_get_end_points(sa, p_madw, &p_src_port, &p_dest_port,
-					  &dgid);
+					  &p_dgid);
 	if (sa_status == IB_SA_MAD_STATUS_SUCCESS) {
 		/*
 		   What happens next depends on the type of endpoint information
@@ -1624,17 +1617,17 @@ void osm_pr_rcv_process(IN void *context, IN void *data)
 			if (p_dest_port)
 				pr_rcv_process_pair(sa, p_madw, requester_port,
 						    p_src_port, p_dest_port,
-						    &dgid, p_sa_mad->comp_mask,
+						    p_dgid, p_sa_mad->comp_mask,
 						    &pr_list);
 			else
 				pr_rcv_process_half(sa, p_madw, requester_port,
-						    p_src_port, NULL, &dgid,
+						    p_src_port, NULL, p_dgid,
 						    p_sa_mad->comp_mask,
 						    &pr_list);
 		} else {
 			if (p_dest_port)
 				pr_rcv_process_half(sa, p_madw, requester_port,
-						    NULL, p_dest_port, &dgid,
+						    NULL, p_dest_port, p_dgid,
 						    p_sa_mad->comp_mask,
 						    &pr_list);
 			else
@@ -1642,7 +1635,7 @@ void osm_pr_rcv_process(IN void *context, IN void *data)
 				   Katie, bar the door!
 				 */
 				pr_rcv_process_world(sa, p_madw, requester_port,
-						     &dgid, p_sa_mad->comp_mask,
+						     p_dgid, p_sa_mad->comp_mask,
 						     &pr_list);
 		}
 	}
Signature · 6 lines
1.6.6.rc4

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#30 lore raw

Re: [PATCH] opensm/osm_sa_path_record.c: MGID must be specified explicitly

In reply to Sasha Khapyorsky

T24gVHVlLCBEZWMgMjIsIDIwMDkgYXQgNjozNyBBTSwgU2FzaGEgS2hhcHlvcnNreSA8c2FzaGFr QHZvbHRhaXJlLmNvbT4gd3JvdGU6Cj4KPiBJQkEgMS4yLjEgc3RhdGVzIChWb2wuMSwgMTUuMi41 LjE2LCBwLjkxOCkgdGhhdCBER0lEIHNoZWxsIGJlIGV4cGxpY2l0bHkKPiBzcGVjaWZpZWQgd2hl biBwYXRoIGRlc3RpbmF0aW9uIG9mIFNBIFBhdGhSZWNvcmQgcXVlcnkgaXMgbXVsdGljYXN0Cj4g Z3JvdXAuCgpUaGF0J3MgaW5mb3JtYXRpdmUgcmF0aGVyIHRoYW4gY29tcGxpYW5jZSB0ZXh0LiBJ dCBhcHBlYXJzIHRvIG9ubHkKcmVmZXIgdG8gR2V0VGFibGUgKGFuZCBub3QgR2V0KS4gQWxzbywg aXQgYXBwZWFycyBvdmVybHkgcmVzdHJpY3RpdmUKdG8gbWUuIEZvciBleGFtcGxlLCBER0lEIGlz IHdpbGRjYXJkZWQgYnV0IE1MSUQgaXMgc3VwcGxpZWQgYXMgRExJRApzaG91bGQgYmUgaG9ub3Jl ZC4gVGhlIHJld29ya2VkIGNvZGUgYXBwZWFycyBub3QgdG8gc3VwcG9ydCB0aGF0CkFGQUlULgoK LS0gSGFsCgo+IFNvIGlnbm9yZSBhbGwgb3RoZXIgY2FzZXMgd2hlbiBjaG9vc2luZyBtdWx0aWNh c3QgUFIgZGVzdGluYXRpb24uCj4gT3RoZXIgcmVsYXRlZCBwYXJhbWV0ZXJzIChtbGlkKSBpcyBj aGVja2VkIGluIHByX21hdGNoX21ncnBfYXR0cmlidXRlcygpCj4gYW55d2F5LiBBcyByZXN1bHQg Y29uc29saWRhdGUgbWFueSBvZiBlZmZlY3RpdmVseSBkdXBsaWNhdGUgY29kZS4KPgo+IFNpZ25l ZC1vZmYtYnk6IFNhc2hhIEtoYXB5b3Jza3kgPHNhc2hha0B2b2x0YWlyZS5jb20+Cj4gLS0tCj4g wqBvcGVuc20vb3BlbnNtL29zbV9zYV9wYXRoX3JlY29yZC5jIHwgwqAgODggKysrKystLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tCj4gwqAxIGZpbGVzIGNoYW5nZWQsIDExIGluc2VydGlv bnMoKyksIDc3IGRlbGV0aW9ucygtKQo+Cj4gZGlmZiAtLWdpdCBhL29wZW5zbS9vcGVuc20vb3Nt X3NhX3BhdGhfcmVjb3JkLmMgYi9vcGVuc20vb3BlbnNtL29zbV9zYV9wYXRoX3JlY29yZC5jCj4g aW5kZXggZmJlZWYwMy4uMDgyYTQxZCAxMDA2NDQKPiAtLS0gYS9vcGVuc20vb3BlbnNtL29zbV9z YV9wYXRoX3JlY29yZC5jCj4gKysrIGIvb3BlbnNtL29wZW5zbS9vc21fc2FfcGF0aF9yZWNvcmQu Ywo+IEBAIC0xNDAxLDM0ICsxNDAxLDYgQEAgc3RhdGljIHZvaWQgcHJfcmN2X3Byb2Nlc3NfcGFp cihJTiBvc21fc2FfdCAqIHNhLCBJTiBjb25zdCBvc21fbWFkd190ICogcF9tYWR3LAo+IMKgIMKg IMKgIMKgT1NNX0xPR19FWElUKHNhLT5wX2xvZyk7Cj4gwqB9Cj4KPiAtc3RhdGljIG9zbV9tZ3Jw X3QgKnByX2dldF9tZ3JwKElOIG9zbV9zYV90ICogc2EsIElOIGNvbnN0IG9zbV9tYWR3X3QgKiBw X21hZHcpCj4gLXsKPiAtIMKgIMKgIMKgIGliX3BhdGhfcmVjX3QgKnBfcHI7Cj4gLSDCoCDCoCDC oCBjb25zdCBpYl9zYV9tYWRfdCAqcF9zYV9tYWQ7Cj4gLSDCoCDCoCDCoCBvc21fbWdycF90ICpt Z3JwOwo+IC0KPiAtIMKgIMKgIMKgIHBfc2FfbWFkID0gb3NtX21hZHdfZ2V0X3NhX21hZF9wdHIo cF9tYWR3KTsKPiAtIMKgIMKgIMKgIHBfcHIgPSBpYl9zYV9tYWRfZ2V0X3BheWxvYWRfcHRyKHBf c2FfbWFkKTsKPiAtCj4gLSDCoCDCoCDCoCBpZiAoIShwX3NhX21hZC0+Y29tcF9tYXNrICYgSUJf UFJfQ09NUE1BU0tfREdJRCkpIHsKPiAtIMKgIMKgIMKgIMKgIMKgIMKgIMKgIE9TTV9MT0coc2Et PnBfbG9nLCBPU01fTE9HX1ZFUkJPU0UsCj4gLSDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC oCDCoCAiZGlzY2FyZCBtdWx0aWNhc3QgdGFyZ2V0IFNBIFBSIHdpdGggd2lsZGNhcmRlZCBNR0lE Iik7Cj4gLSDCoCDCoCDCoCDCoCDCoCDCoCDCoCByZXR1cm4gTlVMTDsKPiAtIMKgIMKgIMKgIH0K PiAtCj4gLSDCoCDCoCDCoCBtZ3JwID0gb3NtX2dldF9tZ3JwX2J5X21naWQoc2EtPnBfc3Vibiwg JnBfcHItPmRnaWQpOwo+IC0gwqAgwqAgwqAgaWYgKCFtZ3JwKSB7Cj4gLSDCoCDCoCDCoCDCoCDC oCDCoCDCoCBjaGFyIGdpZF9zdHJbSU5FVDZfQUREUlNUUkxFTl07Cj4gLSDCoCDCoCDCoCDCoCDC oCDCoCDCoCBPU01fTE9HKHNhLT5wX2xvZywgT1NNX0xPR19FUlJPUiwgIkVSUiAxRjA5OiAiCj4g LSDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCAiTm8gTUMgZ3JvdXAgZm91bmQgZm9y IFBhdGhSZWNvcmQgZGVzdGluYXRpb24gR0lEICVzXG4iLAo+IC0gwqAgwqAgwqAgwqAgwqAgwqAg wqAgwqAgwqAgwqAgwqAgaW5ldF9udG9wKEFGX0lORVQ2LCBwX3ByLT5kZ2lkLnJhdywgZ2lkX3N0 ciwKPiAtIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIHNp emVvZiBnaWRfc3RyKSk7Cj4gLSDCoCDCoCDCoCDCoCDCoCDCoCDCoCByZXR1cm4gTlVMTDsKPiAt IMKgIMKgIMKgIH0KPiAtCj4gLSDCoCDCoCDCoCByZXR1cm4gbWdycDsKPiAtfQo+IC0KPiDCoHN0 YXRpYyBpYl9hcGlfc3RhdHVzX3QgcHJfbWF0Y2hfbWdycF9hdHRyaWJ1dGVzKElOIG9zbV9zYV90 ICogc2EsCj4gwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqBJTiBjb25zdCBvc21fbWFkd190ICogcF9tYWR3LAo+IMKg IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg IMKgIMKgIMKgIMKgSU4gY29uc3Qgb3NtX21ncnBfdCAqIHBfbWdycCkKPiBAQCAtMTUxMCw0NyAr MTQ4MiwxMSBAQCBFeGl0Ogo+IMKgIMKgIMKgIMKgcmV0dXJuIHN0YXR1czsKPiDCoH0KPgo+IC1z dGF0aWMgaW50IHByX3Jjdl9jaGVja19tY2FzdF9kZXN0KG9zbV9zYV90ICogc2EsIElOIGNvbnN0 IG9zbV9tYWR3X3QgKiBwX21hZHcpCj4gLXsKPiAtIMKgIMKgIMKgIGNvbnN0IGliX3BhdGhfcmVj X3QgKnBfcHI7Cj4gLSDCoCDCoCDCoCBjb25zdCBpYl9zYV9tYWRfdCAqcF9zYV9tYWQ7Cj4gLSDC oCDCoCDCoCBpYl9uZXQ2NF90IGNvbXBfbWFzazsKPiAtIMKgIMKgIMKgIGludCBpc19tdWx0aWNh c3QgPSAwOwo+IC0KPiAtIMKgIMKgIMKgIE9TTV9MT0dfRU5URVIoc2EtPnBfbG9nKTsKPiAtCj4g LSDCoCDCoCDCoCBwX3NhX21hZCA9IG9zbV9tYWR3X2dldF9zYV9tYWRfcHRyKHBfbWFkdyk7Cj4g LSDCoCDCoCDCoCBwX3ByID0gKGliX3BhdGhfcmVjX3QgKikgaWJfc2FfbWFkX2dldF9wYXlsb2Fk X3B0cihwX3NhX21hZCk7Cj4gLQo+IC0gwqAgwqAgwqAgY29tcF9tYXNrID0gcF9zYV9tYWQtPmNv bXBfbWFzazsKPiAtCj4gLSDCoCDCoCDCoCBpZiAoY29tcF9tYXNrICYgSUJfUFJfQ09NUE1BU0tf REdJRCkgewo+IC0gwqAgwqAgwqAgwqAgwqAgwqAgwqAgaXNfbXVsdGljYXN0ID0gaWJfZ2lkX2lz X211bHRpY2FzdCgmcF9wci0+ZGdpZCk7Cj4gLSDCoCDCoCDCoCDCoCDCoCDCoCDCoCBpZiAoIWlz X211bHRpY2FzdCkKPiAtIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIGdvdG8gRXhp dDsKPiAtIMKgIMKgIMKgIH0KPiAtCj4gLSDCoCDCoCDCoCBpZiAoY29tcF9tYXNrICYgSUJfUFJf Q09NUE1BU0tfRExJRCkgewo+IC0gwqAgwqAgwqAgwqAgwqAgwqAgwqAgaWYgKGNsX250b2gxNihw X3ByLT5kbGlkKSA+PSBJQl9MSURfTUNBU1RfU1RBUlRfSE8gJiYKPiAtIMKgIMKgIMKgIMKgIMKg IMKgIMKgIMKgIMKgIGNsX250b2gxNihwX3ByLT5kbGlkKSA8PSBJQl9MSURfTUNBU1RfRU5EX0hP KQo+IC0gwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgaXNfbXVsdGljYXN0ID0gMTsK PiAtIMKgIMKgIMKgIMKgIMKgIMKgIMKgIGVsc2UgaWYgKGlzX211bHRpY2FzdCkgewo+IC0gwqAg wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgT1NNX0xPRyhzYS0+cF9sb2csIE9TTV9MT0df RVJST1IsICJFUlIgMUYxMjogIgo+IC0gwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg wqAgwqAgwqAgwqAgIlBhdGhSZWNvcmQgcmVxdWVzdCBpbmRpY2F0ZXMgTUdJRCBidXQgbm90IE1M SURcbiIpOwo+IC0gwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgaXNfbXVsdGljYXN0 ID0gLTE7Cj4gLSDCoCDCoCDCoCDCoCDCoCDCoCDCoCB9Cj4gLSDCoCDCoCDCoCB9Cj4gLQo+IC1F eGl0Ogo+IC0gwqAgwqAgwqAgT1NNX0xPR19FWElUKHNhLT5wX2xvZyk7Cj4gLSDCoCDCoCDCoCBy ZXR1cm4gaXNfbXVsdGljYXN0Owo+IC19Cj4gLQo+IMKgdm9pZCBvc21fcHJfcmN2X3Byb2Nlc3Mo SU4gdm9pZCAqY29udGV4dCwgSU4gdm9pZCAqZGF0YSkKPiDCoHsKPiDCoCDCoCDCoCDCoG9zbV9z YV90ICpzYSA9IGNvbnRleHQ7Cj4gwqAgwqAgwqAgwqBvc21fbWFkd190ICpwX21hZHcgPSBkYXRh Owo+IC0gwqAgwqAgwqAgY29uc3QgaWJfcGF0aF9yZWNfdCAqcF9wcjsKPiArIMKgIMKgIMKgIGli X3BhdGhfcmVjX3QgKnBfcHI7Cj4gwqAgwqAgwqAgwqBjb25zdCBpYl9zYV9tYWRfdCAqcF9zYV9t YWQ7Cj4gwqAgwqAgwqAgwqBjb25zdCBvc21fcG9ydF90ICpwX3NyY19wb3J0Owo+IMKgIMKgIMKg IMKgY29uc3Qgb3NtX3BvcnRfdCAqcF9kZXN0X3BvcnQ7Cj4gQEAgLTE1NTgsNyArMTQ5NCw2IEBA IHZvaWQgb3NtX3ByX3Jjdl9wcm9jZXNzKElOIHZvaWQgKmNvbnRleHQsIElOIHZvaWQgKmRhdGEp Cj4gwqAgwqAgwqAgwqBpYl9naWRfdCBkZ2lkOwo+IMKgIMKgIMKgIMKgaWJfbmV0MTZfdCBzYV9z dGF0dXM7Cj4gwqAgwqAgwqAgwqBvc21fcG9ydF90ICpyZXF1ZXN0ZXJfcG9ydDsKPiAtIMKgIMKg IMKgIGludCByZXQ7Cj4KPiDCoCDCoCDCoCDCoE9TTV9MT0dfRU5URVIoc2EtPnBfbG9nKTsKPgo+ IEBAIC0xNjAxLDIxICsxNTM2LDE0IEBAIHZvaWQgb3NtX3ByX3Jjdl9wcm9jZXNzKElOIHZvaWQg KmNvbnRleHQsIElOIHZvaWQgKmRhdGEpCj4gwqAgwqAgwqAgwqBjbF9wbG9ja19hY3F1aXJlKHNh LT5wX2xvY2spOwo+Cj4gwqAgwqAgwqAgwqAvKiBIYW5kbGUgbXVsdGljYXN0IGRlc3RpbmF0aW9u cyBzZXBhcmF0ZWx5ICovCj4gLSDCoCDCoCDCoCBpZiAoKHJldCA9IHByX3Jjdl9jaGVja19tY2Fz dF9kZXN0KHNhLCBwX21hZHcpKSA8IDApIHsKPiAtIMKgIMKgIMKgIMKgIMKgIMKgIMKgIC8qIE11 bHRpY2FzdCBER0lEIHdpdGggdW5pY2FzdCBETElEICovCj4gLSDCoCDCoCDCoCDCoCDCoCDCoCDC oCBjbF9wbG9ja19yZWxlYXNlKHNhLT5wX2xvY2spOwo+IC0gwqAgwqAgwqAgwqAgwqAgwqAgwqAg b3NtX3NhX3NlbmRfZXJyb3Ioc2EsIHBfbWFkdywgSUJfTUFEX1NUQVRVU19JTlZBTElEX0ZJRUxE KTsKPiAtIMKgIMKgIMKgIMKgIMKgIMKgIMKgIGdvdG8gRXhpdDsKPiAtIMKgIMKgIMKgIH0KPiAt Cj4gLSDCoCDCoCDCoCBpZiAocmV0ID4gMCkKPiArIMKgIMKgIMKgIGlmICgocF9zYV9tYWQtPmNv bXBfbWFzayAmIElCX1BSX0NPTVBNQVNLX0RHSUQpICYmCj4gKyDCoCDCoCDCoCDCoCDCoCBpYl9n aWRfaXNfbXVsdGljYXN0KCZwX3ByLT5kZ2lkKSk7Cj4gwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqBn b3RvIE1jYXN0RGVzdDsKPgo+IMKgIMKgIMKgIMKgT1NNX0xPRyhzYS0+cF9sb2csIE9TTV9MT0df REVCVUcsICJVbmljYXN0IGRlc3RpbmF0aW9uIHJlcXVlc3RlZFxuIik7Cj4KPiDCoCDCoCDCoCDC oHNhX3N0YXR1cyA9IHByX3Jjdl9nZXRfZW5kX3BvaW50cyhzYSwgcF9tYWR3LCAmcF9zcmNfcG9y dCwgJnBfZGVzdF9wb3J0LAo+IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgJmRnaWQpOwo+IC0KPiDCoCDCoCDCoCDCoGlmIChzYV9z dGF0dXMgPT0gSUJfU0FfTUFEX1NUQVRVU19TVUNDRVNTKSB7Cj4gwqAgwqAgwqAgwqAgwqAgwqAg wqAgwqAvKgo+IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIFdoYXQgaGFwcGVucyBuZXh0IGRl cGVuZHMgb24gdGhlIHR5cGUgb2YgZW5kcG9pbnQgaW5mb3JtYXRpb24KPiBAQCAtMTY2MCw5ICsx NTg4LDE1IEBAIE1jYXN0RGVzdDoKPiDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoHVpbnQ4X3QgaG9w X2xpbWl0Owo+Cj4gwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAvKiBGaXJzdCwgZ2V0IHRoZSBNQyBp bmZvICovCj4gLSDCoCDCoCDCoCDCoCDCoCDCoCDCoCBwX21ncnAgPSBwcl9nZXRfbWdycChzYSwg cF9tYWR3KTsKPiAtIMKgIMKgIMKgIMKgIMKgIMKgIMKgIGlmICghcF9tZ3JwKQo+ICsgwqAgwqAg wqAgwqAgwqAgwqAgwqAgcF9tZ3JwID0gb3NtX2dldF9tZ3JwX2J5X21naWQoc2EtPnBfc3Vibiwg JnBfcHItPmRnaWQpOwo+ICsgwqAgwqAgwqAgwqAgwqAgwqAgwqAgaWYgKCFwX21ncnApIHsKPiAr IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIGNoYXIgZ2lkX3N0cltJTkVUNl9BRERS U1RSTEVOXTsKPiArIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIE9TTV9MT0coc2Et PnBfbG9nLCBPU01fTE9HX0VSUk9SLCAiRVJSIDFGMDk6ICIKPiArIMKgIMKgIMKgIMKgIMKgIMKg IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgICJObyBNQyBncm91cCBmb3VuZCBmb3IgUGF0aFJl Y29yZCBkZXN0aW5hdGlvbiBHSUQgJXNcbiIsCj4gKyDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC oCDCoCDCoCDCoCDCoCDCoCDCoCBpbmV0X250b3AoQUZfSU5FVDYsIHBfcHItPmRnaWQucmF3LCBn aWRfc3RyLAo+ICsgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg wqAgwqAgwqAgwqAgwqAgc2l6ZW9mIGdpZF9zdHIpKTsKPiDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC oCDCoCDCoCDCoCDCoGdvdG8gVW5sb2NrOwo+ICsgwqAgwqAgwqAgwqAgwqAgwqAgwqAgfQo+Cj4g wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAvKiBNYWtlIHN1cmUgdGhlIHJlc3Qgb2YgdGhlIFBhdGhS ZWNvcmQgbWF0Y2hlcyB0aGUgTUMgZ3JvdXAgYXR0cmlidXRlcyAqLwo+IMKgIMKgIMKgIMKgIMKg IMKgIMKgIMKgc3RhdHVzID0gcHJfbWF0Y2hfbWdycF9hdHRyaWJ1dGVzKHNhLCBwX21hZHcsIHBf bWdycCk7Cj4gLS0KPiAxLjYuNi5yYzQKPgo+Cg==
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#31 lore raw
In reply to a message not in the archive
Quoted 7 lines · Jason Gunthorpe
Quoted 5 lines
Quoted 2 lines

However, I don't understand the comment "Only set HopLimit if going through a router"?

This is from '#ifdef ROUTER_EXP' days - as far as I could understand HopLimit should be set to IB_HOPLIMIT_MAX for inter-subnet MADs.

If HopLimit is 0 then no GRH is required, if it is non 0 then the user

I think it is:
HopLimit 0 or 1 requires no GRH (GRH is optional in these cases) and
other than 0 or 1 requires GRH

Quoted 2 lines

of that path needs to apply a GRH. IIRC there are already a few things in the kernel that follow this?

Yes, we started down this path. This was all discussed on the list back prior to SC07 days AFAIR.

-- Hal

Quoted 2 lines

Jason

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html