Skip to contentSkip to conversation

RDMA/ucma: Copy iWARP route information.

PATCH In review Steve Wise 3 participants 17 messages May 10, 2010 – Jan 18, 2011 14 messages not in the archive InfiniBand

lore
Share & tools
lore

Steve Wise

author
#0 lore raw

PATCH 1/1 RDMA/ucma: Copy iWARP route information.

For iWARP rdma_cm ids, the "route" information is the L2 src and next hop addresses.

---

 drivers/infiniband/core/ucma.c |   13 +++++++++++++
 1 files changed, 13 insertions(+), 0 deletions(-)
drivers/infiniband/core/ucma.c +13 -0
index ac7edc2..0498383 100644
@@ -583,6 +583,16 @@ static void ucma_copy_ib_route(struct rdma_ucm_query_route_resp *resp,
 	}
 }
 
+static void ucma_copy_iw_route(struct rdma_ucm_query_route_resp *resp,
+			       struct rdma_route *route)
+{
+	struct rdma_dev_addr *dev_addr;
+
+	dev_addr = &route->addr.dev_addr;
+	rdma_addr_get_dgid(dev_addr, (union ib_gid *) &resp->ib_route[0].dgid);
+	rdma_addr_get_sgid(dev_addr, (union ib_gid *) &resp->ib_route[0].sgid);
+}
+
 static ssize_t ucma_query_route(struct ucma_file *file,
 				const char __user *inbuf,
 				int in_len, int out_len)
@@ -621,6 +631,9 @@ static ssize_t ucma_query_route(struct ucma_file *file,
 	case RDMA_TRANSPORT_IB:
 		ucma_copy_ib_route(&resp, &ctx->cm_id->route);
 		break;
+	case RDMA_TRANSPORT_IWARP:
+		ucma_copy_iw_route(&resp, &ctx->cm_id->route);
+		break;
 	default:
 		break;
 	}

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 20100510154656.22415.47740.stgit-T4OLL4TyM9aNDNWfRnPdfg@public.gmane.org

#2 lore raw
Quoted 16 lines

For iWARP rdma_cm ids, the "route" information is the L2 src and next hop addresses.

---

drivers/infiniband/core/ucma.c | 13 +++++++++++++

1 files changed, 13 insertions(+), 0 deletions(-)
drivers/infiniband/core/ucma.c +0 -0
index ac7edc2..0498383 100644
@@ -583,6 +583,16 @@ static void ucma_copy_ib_route(struct
 rdma_ucm_query_route_resp *resp,
 	}

drivers/infiniband/core/ucma.c static void ucma_copy_ib_route(struct · line 594

Quoted 10 lines
drivers/infiniband/core/ucma.c +8 -0
 }
 
+static void ucma_copy_iw_route(struct rdma_ucm_query_route_resp *resp,
+			       struct rdma_route *route)
+{
+	struct rdma_dev_addr *dev_addr;
+
+	dev_addr = &route->addr.dev_addr;
+	rdma_addr_get_dgid(dev_addr, (union ib_gid *) &resp->ib_route[0].dgid);
+	rdma_addr_get_sgid(dev_addr, (union ib_gid *) &resp->ib_route[0].sgid);

This looks good for the current code.

For 2.6.35/36, I've submitted changes to the query operation to support AF_IB, essentially breaking the query_route call into query_addr, query_path, and query_gid. I'm guessing that this functionality should be part of the query_gid call. I'll adjust those patches, but if you get a chance to review the query changes, I'd appreciate it.

- Sean

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 1C56604F650B44A8B00B3CA55D00B95D-Zpru7NauK7drdx17CPfAsdBPR1lH4CV8@public.gmane.org

#4 lore raw
Quoted 11 lines

drivers/infiniband/core/ucma.c static void ucma_copy_ib_route(struct rdma_ucm_query_route_resp *resp, · line 593

Quoted 8 lines
drivers/infiniband/core/ucma.c +8 -0
+static void ucma_copy_iw_route(struct rdma_ucm_query_route_resp *resp,
+			       struct rdma_route *route)
+{
+	struct rdma_dev_addr *dev_addr;
+
+	dev_addr = &route->addr.dev_addr;
+	rdma_addr_get_dgid(dev_addr, (union ib_gid *) &resp->ib_route[0].dgid);
+	rdma_addr_get_sgid(dev_addr, (union ib_gid *) &resp->ib_route[0].sgid);

essentially breaking the query_route call into query_addr, query_path, and query_gid.

I'm unsure where exactly this change would fit into the newer query model. Are these values available after rdma_resolve_addr completes? Are the sgid and dgid the same type of values (e.g. L2 addresses)? If the device does ARP internally, is the dgid value still set?

My current implementation for 'query_gid' returns GIDs using an AF_IB address structure. I'm trying to figure out what structure we should be using to return the iwarp values -- some other sockaddr, an iw_path_record, other?

Here are links to the proposed changes:

http://www.openfabrics.org/git/?p=~shefty/rdma-dev.git;a=commitdiff;h=b87cccdb27
a7e75c0f9e03a9d37593ceab4d4ede

http://www.openfabrics.org/git/?p=~shefty/rdma-dev.git;a=commitdiff;h=1830722bb5
e84451e3b458719cea8c746a2fb6d4

http://www.openfabrics.org/git/?p=~shefty/rdma-dev.git;a=commitdiff;h=0257cc9298
aa70c4bfc1a8c393f970375a897825

- Sean

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): B28805226A0F4AD7B53BD430925D3D20-Zpru7NauK7drdx17CPfAsdBPR1lH4CV8@public.gmane.org

author
#6 lore raw
Quoted 13 lines · Sean Hefty
Quoted 12 lines

drivers/infiniband/core/ucma.c static void ucma_copy_ib_route(struct rdma_ucm_query_route_resp *resp, · line 593

Quoted 9 lines
drivers/infiniband/core/ucma.c +8 -0
+static void ucma_copy_iw_route(struct rdma_ucm_query_route_resp *resp,
+			       struct rdma_route *route)
+{
+	struct rdma_dev_addr *dev_addr;
+
+	dev_addr = &route->addr.dev_addr;
+	rdma_addr_get_dgid(dev_addr, (union ib_gid *) &resp->ib_route[0].dgid);
+	rdma_addr_get_sgid(dev_addr, (union ib_gid *) &resp->ib_route[0].sgid);

essentially breaking the query_route call into query_addr, query_path, and query_gid.

Quoted 2 lines

I'm unsure where exactly this change would fit into the newer query model. Are these values available after rdma_resolve_addr completes?

Yes.

Quoted 2 lines

Are the sgid and dgid
the same type of values (e.g. L2 addresses)?

Yes.

Quoted 4 lines

If the device does ARP internally,
is the dgid value still set?

No. But only Amso does this, and its old and crusty...

Quoted 5 lines

My current implementation for 'query_gid' returns GIDs using an AF_IB address structure. I'm trying to figure out what structure we should be using to return the iwarp values -- some other sockaddr, an iw_path_record, other?

I'll have to ponder this. In the past, we've been using the gid.raw areas to hold these mac addresses...

Quoted 5 lines

Here are links to the proposed changes:

http://www.openfabrics.org/git/?p=~shefty/rdma-dev.git;a=commitdiff;h=b87cccdb27
a7e75c0f9e03a9d37593ceab4d4ede

Quoted 8 lines

http://www.openfabrics.org/git/?p=~shefty/rdma-dev.git;a=commitdiff;h=1830722bb5
e84451e3b458719cea8c746a2fb6d4

http://www.openfabrics.org/git/?p=~shefty/rdma-dev.git;a=commitdiff;h=0257cc9298
aa70c4bfc1a8c393f970375a897825

- Sean

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 4BE9C04E.4040907-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org

#8 lore raw
Quoted 2 lines

I'll have to ponder this. In the past, we've been using the gid.raw
areas to hold these mac addresses...

There's not too many options with the existing ABI, but the newer query interfaces will give us more flexibility, like variable length addresses, which may make this cleaner. Maybe we can return the L2 address through an AF_UNSPEC sockaddr?

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 23BD55BBBEC444B18E7B48D98CC1EBA5-Zpru7NauK7drdx17CPfAsdBPR1lH4CV8@public.gmane.org

author
#10 lore raw
Quoted 10 lines · Sean Hefty
Quoted 3 lines

I'll have to ponder this. In the past, we've been using the gid.raw
areas to hold these mac addresses...

There's not too many options with the existing ABI, but the newer query interfaces will give us more flexibility, like variable length addresses, which may make this cleaner. Maybe we can return the L2 address through an AF_UNSPEC sockaddr?

What does AF_UNSPEC imply about the format of the sockaddr?

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 4BEB027B.5060500-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org

#12 lore raw
Quoted 1 line

What does AF_UNSPEC imply about the format of the sockaddr?

That it depends on the usage..? I came up with this based on using ioctl to manipulate the ARP cache:

SIOCSARP - Add a new entry to the ARP cache or modify an existing entry ... arp_ha is a generic socket address structure with sa_family set to AF_UNSPEC and sa_data containing the hardware address (e.g. the 6-byte Ethernet address).

The librdmacm should have enough context to interpret the meaning of the query_gid response.

- Sean

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 5A102C804D4348A5BCE504F433888BED-Zpru7NauK7drdx17CPfAsdBPR1lH4CV8@public.gmane.org

author
#14 lore raw
Quoted 6 lines · Sean Hefty
Quoted 2 lines

What does AF_UNSPEC imply about the format of the sockaddr?

That it depends on the usage..? I came up with this based on using ioctl to manipulate the ARP cache:

Quoted 9 lines

SIOCSARP - Add a new entry to the ARP cache or modify an existing entry ... arp_ha is a generic socket address structure with sa_family set to AF_UNSPEC and sa_data containing the hardware address (e.g. the 6-byte Ethernet address).

The librdmacm should have enough context to interpret the meaning of the query_gid response.

- Sean

sounds ok to me.

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

author
#15 lore raw
In reply to a message not in the archive

Roland/Sean, is this ok for 2.6.35?

Quoted 42 lines · Steve Wise

For iWARP rdma_cm ids, the "route" information is the L2 src and next hop addresses.

---

 drivers/infiniband/core/ucma.c |   13 +++++++++++++
 1 files changed, 13 insertions(+), 0 deletions(-)
drivers/infiniband/core/ucma.c +13 -1
index ac7edc2..0498383 100644
@@ -583,6 +583,16 @@ static void ucma_copy_ib_route(struct rdma_ucm_query_route_resp *resp,
 	}
 }
 
+static void ucma_copy_iw_route(struct rdma_ucm_query_route_resp *resp,
+			       struct rdma_route *route)
+{
+	struct rdma_dev_addr *dev_addr;
+
+	dev_addr = &route->addr.dev_addr;
+	rdma_addr_get_dgid(dev_addr, (union ib_gid *) &resp->ib_route[0].dgid);
+	rdma_addr_get_sgid(dev_addr, (union ib_gid *) &resp->ib_route[0].sgid);
+}
+
 static ssize_t ucma_query_route(struct ucma_file *file,
 				const char __user *inbuf,
 				int in_len, int out_len)
@@ -621,6 +631,9 @@ static ssize_t ucma_query_route(struct ucma_file *file,
 	case RDMA_TRANSPORT_IB:
 		ucma_copy_ib_route(&resp, &ctx->cm_id->route);
 		break;
+	case RDMA_TRANSPORT_IWARP:
+		ucma_copy_iw_route(&resp, &ctx->cm_id->route);
+		break;
 	default:
 		break;
 	}
 
--
Quoted 4 lines

To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 4BF45F1A.4010707-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org

#17 lore raw
Quoted 1 line

Roland/Sean, is this ok for 2.6.35?

I guess it's fine. What does it give us by itself though?

- R.

Signature · 6 lines
Roland Dreier <[email protected]> || For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/index.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): adaeih7eegq.fsf-BjVyx320WGW9gfZ95n9DRSW4+XlvGpQz@public.gmane.org

author
#19 lore raw
Quoted 5 lines · Roland Dreier
Quoted 1 line

Roland/Sean, is this ok for 2.6.35?

I guess it's fine. What does it give us by itself though?

Piece of mind. :)

IMO it is a bug fix.

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 4BF4618C.1090106-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org

author
#21 lore raw

Hey Roland,

Did this patch ever make it upstream? I don't see it.

Quoted 16 lines · Steve Wise
Quoted 4 lines · Roland Dreier
Quoted 1 line

Roland/Sean, is this ok for 2.6.35?

I guess it's fine. What does it give us by itself though?

Piece of mind. :)

IMO it is a bug fix.

Signature · 3 lines
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 4D2CD02C.1000607-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org

A message that is not in the archive (sent off-list or not yet received): [email protected]

author
#25 lore raw

Yes, I will.

Thanks,

Steve.

Quoted 6 lines · Roland Dreier
Quoted 1 line

Did this patch ever make it upstream? I don't see it.

Nope, guess it got dropped. It doesn't apply after the IBoE changes -- can you regenerate it?

- R.

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

author
#26 lore raw
In reply to a message not in the archive
Quoted 5 lines · Roland Dreier
Quoted 1 line

Did this patch ever make it upstream? I don't see it.

Nope, guess it got dropped. It doesn't apply after the IBoE changes -- can you regenerate it?

Hey,

Did this:

http://www.mail-archive.com/[email protected]/msg03182.html

ever make it upstream?
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): CF9C39F99A89134C9CF9C4CCB68B8DDF25C1C997FD-osO9UTpF0USkrb+BlOpmy7fspsVTdybXVpNB7YpNyf8@public.gmane.org

author
#30 lore raw
Quoted 4 lines · Sean
Quoted 4 lines

Did this:

http://www.mail-archive.com/[email protected]/msg03182.html

Quoted 8 lines
Quoted 1 line

ever make it upstream?

No, but this patch set is still listed in patchworks as new. I just updated the set to 2.6.37 and pushed those changes into an af_ib branch in my git tree.

- Sean
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

The reason I asked is that my small change for adding the mac addresses in iwarp routes will cause a merge issue. Roland, do you have plans to merge sean's series soon?
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html