Skip to contentSkip to conversation

rdma/cm: Allow user to select specific RDMA loopback device

PATCH New [email protected] 1 participant 2 messages May 3, 2013 1 message not in the archive InfiniBand

lore
Share & tools
lore
#0 lore raw

PATCH 1/2 rdma/cm: Allow user to select specific RDMA loopback device

From: Sean Hefty <[email protected]>

RDMA connections to a loopback address must map to a physical RDMA device/port. By default, a connection to a loopback address automatically binds the corresponding cm_id to the first active RDMA device/port. The result is that all loopback connections are forced to use the same device, even if multiple devices are present.

In order to map loopback connections across multiple RDMA devices, we expose the mapping directly to the user. For IPv4, we make use of the multiple loopback addresses which are available, with each loopback address matched with a specific RDMA device/port. For IPv6, we use the scope ID as an index into the RDMA CM device/port list.

For compatibility, if the IPv4 loopback address is the common localhost (127.0.0.1) address or if no scope ID is given, we follow the current algorithm of selecting the first active device/port. Otherwise, the address/scope ID is used as an index into the device/port list.

---
 drivers/infiniband/core/cma.c |   27 ++++++++++++++++++++++++++-
 1 files changed, 26 insertions(+), 1 deletions(-)
drivers/infiniband/core/cma.c +26 -1
index 71c2c71..7203a0f 100644
@@ -1913,13 +1913,27 @@ err:
 }
 EXPORT_SYMBOL(rdma_resolve_route);
 
+static int cma_loopback_index(struct rdma_id_private *id_priv)
+{
+	struct sockaddr *addr = (struct sockaddr *) &id_priv->id.route.addr.dst_addr;
+
+	if (addr->sa_family == AF_INET)
+		return ntohl(((struct sockaddr_in *) addr)->sin_addr.s_addr) -
+		       INADDR_LOOPBACK;
+#if IS_ENABLED(CONFIG_IPV6)
+	if (addr->sa_family == AF_INET6)
+		return ((struct sockaddr_in6 *) addr)->sin6_scope_id;
+#endif
+	return 0;
+}
+
 static int cma_bind_loopback(struct rdma_id_private *id_priv)
 {
 	struct cma_device *cma_dev;
 	struct ib_port_attr port_attr;
 	union ib_gid gid;
 	u16 pkey;
-	int ret;
+	int ret, index;
 	u8 p;
 
 	mutex_lock(&lock);
@@ -1927,6 +1941,17 @@ static int cma_bind_loopback(struct rdma_id_private *id_priv)
 		ret = -ENODEV;
 		goto out;
 	}
+
+	index = cma_loopback_index(id_priv);
+	if (index) {
+		list_for_each_entry(cma_dev, &dev_list, list) {
+			if (index <= cma_dev->device->phys_port_cnt) {
+				p = index;
+				goto port_found;
+			}
+			index -= cma_dev->device->phys_port_cnt;
+		}
+	}
 	list_for_each_entry(cma_dev, &dev_list, list)
 		for (p = 1; p <= cma_dev->device->phys_port_cnt; ++p)
 			if (!ib_query_port(cma_dev->device, p, &port_attr) &&
Signature · 6 lines
1.7.3

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 1367619592-2363-1-git-send-email-sean.hefty-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org

#2 lore raw

PATCH 2/2 rdma/cm: Allow enabling reuseaddr in any state

From: Sean Hefty <[email protected]>

The rdma_cm only allows setting reuseaddr if the corresponding rdma_cm_id is in the idle state. Allow setting this value in other states. This brings the behavior more inline with sockets.

---
 drivers/infiniband/core/cma.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
drivers/infiniband/core/cma.c +1 -1
index 7203a0f..71d7b4c 100644
@@ -2122,7 +2122,7 @@ int rdma_set_reuseaddr(struct rdma_cm_id *id, int reuse)
 
 	id_priv = container_of(id, struct rdma_id_private, id);
 	spin_lock_irqsave(&id_priv->lock, flags);
-	if (id_priv->state == RDMA_CM_IDLE) {
+	if (reuse || id_priv->state == RDMA_CM_IDLE) {
 		id_priv->reuseaddr = reuse;
 		ret = 0;
 	} else {
Signature · 6 lines
1.7.3

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html