Skip to contentSkip to conversation

rds: IPv6 support

PATCH Superseded Ka-Cheong Poon 6 participants 18 messages Jul 13, 2018 – Jul 19, 2018

Subsystems (3)

v3 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

v3 cover rds: IPv6 support

This patch set adds IPv6 support to the kernel RDS and related modules. Existing RDS apps using IPv4 address continue to run without any problem. New RDS apps which want to use IPv6 address can do so by passing the address in struct sockaddr_in6 to bind(), connect() or sendmsg(). And those apps also need to use the new IPv6 equivalents of some of the existing socket options as the existing options use a 32 bit integer to store IP address.

All RDS code now use struct in6_addr to store IP address. IPv4 address is stored as an IPv4 mapped address.

Header file changes

There are many data structures (RDS socket options) used by RDS apps which use a 32 bit integer to store IP address. To support IPv6, struct in6_addr needs to be used. To ensure backward compatibility, a new data structure is introduced for each of those data structures which use a 32 bit integer to represent an IP address. And new socket options are introduced to use those new structures. This means that existing apps should work without a problem with the new RDS module. For apps which want to use IPv6, those new data structures and socket options can be used. IPv4 mapped address is used to represent IPv4 address in the new data structures.

Internally, all RDS data structures which contain an IP address are changed to use struct in6_addr to store the address. IPv4 address is stored as an IPv4 mapped address. All the functions which take an IP address as argument are also changed to use struct in6_addr.

RDS/RDMA/IB uses a private data (struct rds_ib_connect_private) exchange between endpoints at RDS connection establishment time to support RDMA. This private data exchange uses a 32 bit integer to represent an IP address. This needs to be changed in order to support IPv6. A new private data struct rds6_ib_connect_private is introduced to handle this. To ensure backward compatibility, an IPv6 capable RDS stack uses another RDMA listener port (RDS_CM_PORT) to accept IPv6 connection. And it continues to use the original RDS_PORT for IPv4 RDS connections. When it needs to communicate with an IPv6 peer, it uses the RDS_TCP_PORT to send the connection set up request.

RDS/TCP changes

TCP related code is changed to support IPv6. Note that only an IPv6 TCP listener on port RDS_TCP_PORT is created as it can accept both IPv4 and IPv6 connection requests.

IB/RDMA changes

The initial private data exchange between IB endpoints using RDMA is changed to support IPv6 address instead, if the peer address is IPv6. To ensure backward compatibility, annother RDMA listener port (RDS_CM_PORT) is used to accept IPv6 connection. An IPv6 capable RDS module continues to use the original RDS_PORT for IPv4 RDS connections. When it needs to communicate with an IPv6 peer, it uses the RDS_CM_PORT to send the connection set up request.

Ka-Cheong Poon (3):
rds: Changing IP address internal representation to struct in6_addr
rds: Enable RDS IPv6 support
rds: Extend RDS API for IPv6 support

 include/uapi/linux/rds.h |  71 ++++++++++-
 net/rds/af_rds.c         | 201 ++++++++++++++++++++++++------
 net/rds/bind.c           | 136 ++++++++++++++++-----
 net/rds/cong.c           |  23 ++--
 net/rds/connection.c     | 259 ++++++++++++++++++++++++++++++---------
 net/rds/ib.c             | 114 +++++++++++++++--
 net/rds/ib.h             |  51 ++++++--
 net/rds/ib_cm.c          | 309 +++++++++++++++++++++++++++++++++++------------
 net/rds/ib_mr.h          |   2 +
 net/rds/ib_rdma.c        |  24 ++--
 net/rds/ib_recv.c        |  18 +--
 net/rds/ib_send.c        |  10 +-
 net/rds/loop.c           |   7 +-
 net/rds/rdma.c           |   6 +-
 net/rds/rdma_transport.c |  84 ++++++++++---
 net/rds/rdma_transport.h |   5 +
 net/rds/rds.h            |  87 ++++++++-----
 net/rds/recv.c           |  76 +++++++++---
 net/rds/send.c           | 114 ++++++++++++++---
 net/rds/tcp.c            | 128 ++++++++++++++++----
 net/rds/tcp.h            |   2 +-
 net/rds/tcp_connect.c    |  68 ++++++++---
 net/rds/tcp_listen.c     |  74 +++++++++---
 net/rds/tcp_recv.c       |   9 +-
 net/rds/tcp_send.c       |   4 +-
 net/rds/threads.c        |  69 +++++++++--
 net/rds/transport.c      |  15 ++-
 27 files changed, 1543 insertions(+), 423 deletions(-)
Signature · 1 line
1.8.3.1
#1 lore raw

PATCH v3 1/3 rds: Changing IP address internal representation to struct in6_addr

This patch changes the internal representation of an IP address to use struct in6_addr. IPv4 address is stored as an IPv4 mapped address. All the functions which take an IP address as argument are also changed to use struct in6_addr. But RDS socket layer is not modified such that it still does not accept IPv6 address from an application. And RDS layer does not accept nor initiate IPv6 connections.

v2: Fixed sparse warnings.

---
 net/rds/af_rds.c         | 138 +++++++++++++++-------
 net/rds/bind.c           |  91 ++++++++++-----
 net/rds/cong.c           |  23 ++--
 net/rds/connection.c     | 132 +++++++++++++--------
 net/rds/ib.c             |  17 +--
 net/rds/ib.h             |  51 ++++++--
 net/rds/ib_cm.c          | 299 ++++++++++++++++++++++++++++++++++-------------
 net/rds/ib_rdma.c        |  15 +--
 net/rds/ib_recv.c        |  18 +--
 net/rds/ib_send.c        |  10 +-
 net/rds/loop.c           |   7 +-
 net/rds/rdma.c           |   6 +-
 net/rds/rdma_transport.c |  56 ++++++---
 net/rds/rds.h            |  70 +++++++----
 net/rds/recv.c           |  51 +++++---
 net/rds/send.c           |  67 ++++++++---
 net/rds/tcp.c            |  32 ++++-
 net/rds/tcp_connect.c    |  34 +++---
 net/rds/tcp_listen.c     |  18 +--
 net/rds/tcp_recv.c       |   9 +-
 net/rds/tcp_send.c       |   4 +-
 net/rds/threads.c        |  69 +++++++++--
 net/rds/transport.c      |  15 ++-
 23 files changed, 863 insertions(+), 369 deletions(-)
net/rds/af_rds.c +99 -39
index ab751a1..fc1a5c6 100644
@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2006 Oracle.  All rights reserved.
+ * Copyright (c) 2006, 2018 Oracle and/or its affiliates. All rights reserved.
  *
  * This software is available to you under a choice of one of two
  * licenses.  You may choose to be licensed under the terms of the GNU
@@ -35,6 +35,7 @@
 #include <linux/kernel.h>
 #include <linux/gfp.h>
 #include <linux/in.h>
+#include <linux/ipv6.h>
 #include <linux/poll.h>
 #include <net/sock.h>
 
@@ -113,26 +114,63 @@ void rds_wake_sk_sleep(struct rds_sock *rs)
 static int rds_getname(struct socket *sock, struct sockaddr *uaddr,
 		       int peer)
 {
-	struct sockaddr_in *sin = (struct sockaddr_in *)uaddr;
 	struct rds_sock *rs = rds_sk_to_rs(sock->sk);
-
-	memset(sin->sin_zero, 0, sizeof(sin->sin_zero));
+	struct sockaddr_in6 *sin6;
+	struct sockaddr_in *sin;
+	int uaddr_len;
 
 	/* racey, don't care */
 	if (peer) {
-		if (!rs->rs_conn_addr)
+		if (ipv6_addr_any(&rs->rs_conn_addr))
 			return -ENOTCONN;
 
-		sin->sin_port = rs->rs_conn_port;
-		sin->sin_addr.s_addr = rs->rs_conn_addr;
+		if (ipv6_addr_v4mapped(&rs->rs_conn_addr)) {
+			sin = (struct sockaddr_in *)uaddr;
+			memset(sin->sin_zero, 0, sizeof(sin->sin_zero));
+			sin->sin_family = AF_INET;
+			sin->sin_port = rs->rs_conn_port;
+			sin->sin_addr.s_addr = rs->rs_conn_addr_v4;
+			uaddr_len = sizeof(*sin);
+		} else {
+			sin6 = (struct sockaddr_in6 *)uaddr;
+			sin6->sin6_family = AF_INET6;
+			sin6->sin6_port = rs->rs_conn_port;
+			sin6->sin6_addr = rs->rs_conn_addr;
+			sin6->sin6_flowinfo = 0;
+			/* scope_id is the same as in the bound address. */
+			sin6->sin6_scope_id = rs->rs_bound_scope_id;
+			uaddr_len = sizeof(*sin6);
+		}
 	} else {
-		sin->sin_port = rs->rs_bound_port;
-		sin->sin_addr.s_addr = rs->rs_bound_addr;
+		/* If socket is not yet bound, set the return address family
+		 * to be AF_UNSPEC (value 0) and the address size to be that
+		 * of an IPv4 address.
+		 */
+		if (ipv6_addr_any(&rs->rs_bound_addr)) {
+			sin = (struct sockaddr_in *)uaddr;
+			memset(sin, 0, sizeof(*sin));
+			sin->sin_family = AF_UNSPEC;
+			return sizeof(*sin);
+		}
+		if (ipv6_addr_v4mapped(&rs->rs_bound_addr)) {
+			sin = (struct sockaddr_in *)uaddr;
+			memset(sin->sin_zero, 0, sizeof(sin->sin_zero));
+			sin->sin_family = AF_INET;
+			sin->sin_port = rs->rs_bound_port;
+			sin->sin_addr.s_addr = rs->rs_bound_addr_v4;
+			uaddr_len = sizeof(*sin);
+		} else {
+			sin6 = (struct sockaddr_in6 *)uaddr;
+			sin6->sin6_family = AF_INET6;
+			sin6->sin6_port = rs->rs_bound_port;
+			sin6->sin6_addr = rs->rs_bound_addr;
+			sin6->sin6_flowinfo = 0;
+			sin6->sin6_scope_id = rs->rs_bound_scope_id;
+			uaddr_len = sizeof(*sin6);
+		}
 	}
 
-	sin->sin_family = AF_INET;
-
-	return sizeof(*sin);
+	return uaddr_len;
 }
 
 /*
@@ -203,11 +241,12 @@ static int rds_ioctl(struct socket *sock, unsigned int cmd, unsigned long arg)
 static int rds_cancel_sent_to(struct rds_sock *rs, char __user *optval,
 			      int len)
 {
+	struct sockaddr_in6 sin6;
 	struct sockaddr_in sin;
 	int ret = 0;
 
 	/* racing with another thread binding seems ok here */
-	if (rs->rs_bound_addr == 0) {
+	if (ipv6_addr_any(&rs->rs_bound_addr)) {
 		ret = -ENOTCONN; /* XXX not a great errno */
 		goto out;
 	}
@@ -215,14 +254,23 @@ static int rds_cancel_sent_to(struct rds_sock *rs, char __user *optval,
 	if (len < sizeof(struct sockaddr_in)) {
 		ret = -EINVAL;
 		goto out;
+	} else if (len < sizeof(struct sockaddr_in6)) {
+		/* Assume IPv4 */
+		if (copy_from_user(&sin, optval, sizeof(struct sockaddr_in))) {
+			ret = -EFAULT;
+			goto out;
+		}
+		ipv6_addr_set_v4mapped(sin.sin_addr.s_addr, &sin6.sin6_addr);
+		sin6.sin6_port = sin.sin_port;
+	} else {
+		if (copy_from_user(&sin6, optval,
+				   sizeof(struct sockaddr_in6))) {
+			ret = -EFAULT;
+			goto out;
+		}
 	}
 
-	if (copy_from_user(&sin, optval, sizeof(sin))) {
-		ret = -EFAULT;
-		goto out;
-	}
-
-	rds_send_drop_to(rs, &sin);
+	rds_send_drop_to(rs, &sin6);
 out:
 	return ret;
 }
@@ -435,31 +483,41 @@ static int rds_connect(struct socket *sock, struct sockaddr *uaddr,
 		       int addr_len, int flags)
 {
 	struct sock *sk = sock->sk;
-	struct sockaddr_in *sin = (struct sockaddr_in *)uaddr;
+	struct sockaddr_in *sin;
 	struct rds_sock *rs = rds_sk_to_rs(sk);
 	int ret = 0;
 
 	lock_sock(sk);
 
-	if (addr_len != sizeof(struct sockaddr_in)) {
-		ret = -EINVAL;
-		goto out;
-	}
+	switch (addr_len) {
+	case sizeof(struct sockaddr_in):
+		sin = (struct sockaddr_in *)uaddr;
+		if (sin->sin_family != AF_INET) {
+			ret = -EAFNOSUPPORT;
+			break;
+		}
+		if (sin->sin_addr.s_addr == htonl(INADDR_ANY)) {
+			ret = -EDESTADDRREQ;
+			break;
+		}
+		if (IN_MULTICAST(ntohl(sin->sin_addr.s_addr)) ||
+		    sin->sin_addr.s_addr == htonl(INADDR_BROADCAST)) {
+			ret = -EINVAL;
+			break;
+		}
+		ipv6_addr_set_v4mapped(sin->sin_addr.s_addr, &rs->rs_conn_addr);
+		rs->rs_conn_port = sin->sin_port;
+		break;
 
-	if (sin->sin_family != AF_INET) {
-		ret = -EAFNOSUPPORT;
-		goto out;
-	}
+	case sizeof(struct sockaddr_in6):
+		ret = -EPROTONOSUPPORT;
+		break;
 
-	if (sin->sin_addr.s_addr == htonl(INADDR_ANY)) {
-		ret = -EDESTADDRREQ;
-		goto out;
+	default:
+		ret = -EINVAL;
+		break;
 	}
 
-	rs->rs_conn_addr = sin->sin_addr.s_addr;
-	rs->rs_conn_port = sin->sin_port;
-
-out:
 	release_sock(sk);
 	return ret;
 }
@@ -578,8 +636,10 @@ static void rds_sock_inc_info(struct socket *sock, unsigned int len,
 		list_for_each_entry(inc, &rs->rs_recv_queue, i_item) {
 			total++;
 			if (total <= len)
-				rds_inc_info_copy(inc, iter, inc->i_saddr,
-						  rs->rs_bound_addr, 1);
+				rds_inc_info_copy(inc, iter,
+						  inc->i_saddr.s6_addr32[3],
+						  rs->rs_bound_addr_v4,
+						  1);
 		}
 
 		read_unlock(&rs->rs_recv_lock);
@@ -608,8 +668,8 @@ static void rds_sock_info(struct socket *sock, unsigned int len,
 	list_for_each_entry(rs, &rds_sock_list, rs_item) {
 		sinfo.sndbuf = rds_sk_sndbuf(rs);
 		sinfo.rcvbuf = rds_sk_rcvbuf(rs);
-		sinfo.bound_addr = rs->rs_bound_addr;
-		sinfo.connected_addr = rs->rs_conn_addr;
+		sinfo.bound_addr = rs->rs_bound_addr_v4;
+		sinfo.connected_addr = rs->rs_conn_addr_v4;
 		sinfo.bound_port = rs->rs_bound_port;
 		sinfo.connected_port = rs->rs_conn_port;
 		sinfo.inum = sock_i_ino(rds_rs_to_sk(rs));
net/rds/bind.c +63 -28

192 lines not on this page · raw message

net/rds/cong.c +13 -10
index 63da9d2..ccdff09 100644
@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2007 Oracle.  All rights reserved.
+ * Copyright (c) 2007, 2017 Oracle and/or its affiliates. All rights reserved.
  *
  * This software is available to you under a choice of one of two
  * licenses.  You may choose to be licensed under the terms of the GNU
@@ -101,7 +101,7 @@
 static DEFINE_SPINLOCK(rds_cong_lock);
 static struct rb_root rds_cong_tree = RB_ROOT;
 
-static struct rds_cong_map *rds_cong_tree_walk(__be32 addr,
+static struct rds_cong_map *rds_cong_tree_walk(const struct in6_addr *addr,
 					       struct rds_cong_map *insert)
 {
 	struct rb_node **p = &rds_cong_tree.rb_node;
@@ -109,12 +109,15 @@ static struct rds_cong_map *rds_cong_tree_walk(__be32 addr,
 	struct rds_cong_map *map;
 
 	while (*p) {
+		int diff;
+
 		parent = *p;
 		map = rb_entry(parent, struct rds_cong_map, m_rb_node);
 
-		if (addr < map->m_addr)
+		diff = rds_addr_cmp(addr, &map->m_addr);
+		if (diff < 0)
 			p = &(*p)->rb_left;
-		else if (addr > map->m_addr)
+		else if (diff > 0)
 			p = &(*p)->rb_right;
 		else
 			return map;
@@ -132,7 +135,7 @@ static struct rds_cong_map *rds_cong_tree_walk(__be32 addr,
  * these bitmaps in the process getting pointers to them.  The bitmaps are only
  * ever freed as the module is removed after all connections have been freed.
  */
-static struct rds_cong_map *rds_cong_from_addr(__be32 addr)
+static struct rds_cong_map *rds_cong_from_addr(const struct in6_addr *addr)
 {
 	struct rds_cong_map *map;
 	struct rds_cong_map *ret = NULL;
@@ -144,7 +147,7 @@ static struct rds_cong_map *rds_cong_from_addr(__be32 addr)
 	if (!map)
 		return NULL;
 
-	map->m_addr = addr;
+	map->m_addr = *addr;
 	init_waitqueue_head(&map->m_waitq);
 	INIT_LIST_HEAD(&map->m_conn_list);
 
@@ -171,7 +174,7 @@ static struct rds_cong_map *rds_cong_from_addr(__be32 addr)
 		kfree(map);
 	}
 
-	rdsdebug("map %p for addr %x\n", ret, be32_to_cpu(addr));
+	rdsdebug("map %p for addr %pI6c\n", ret, addr);
 
 	return ret;
 }
@@ -202,8 +205,8 @@ void rds_cong_remove_conn(struct rds_connection *conn)
 
 int rds_cong_get_maps(struct rds_connection *conn)
 {
-	conn->c_lcong = rds_cong_from_addr(conn->c_laddr);
-	conn->c_fcong = rds_cong_from_addr(conn->c_faddr);
+	conn->c_lcong = rds_cong_from_addr(&conn->c_laddr);
+	conn->c_fcong = rds_cong_from_addr(&conn->c_faddr);
 
 	if (!(conn->c_lcong && conn->c_fcong))
 		return -ENOMEM;
@@ -353,7 +356,7 @@ void rds_cong_remove_socket(struct rds_sock *rs)
 
 	/* update congestion map for now-closed port */
 	spin_lock_irqsave(&rds_cong_lock, flags);
-	map = rds_cong_tree_walk(rs->rs_bound_addr, NULL);
+	map = rds_cong_tree_walk(&rs->rs_bound_addr, NULL);
 	spin_unlock_irqrestore(&rds_cong_lock, flags);
 
 	if (map && rds_cong_test_bit(map, rs->rs_bound_port)) {
net/rds/connection.c +81 -51

255 lines not on this page · raw message

net/rds/ib.c +9 -8
index b6ad38e..c712a84 100644
@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2006 Oracle.  All rights reserved.
+ * Copyright (c) 2006, 2017 Oracle and/or its affiliates. All rights reserved.
  *
  * This software is available to you under a choice of one of two
  * licenses.  You may choose to be licensed under the terms of the GNU
@@ -296,8 +296,8 @@ static int rds_ib_conn_info_visitor(struct rds_connection *conn,
 	if (conn->c_trans != &rds_ib_transport)
 		return 0;
 
-	iinfo->src_addr = conn->c_laddr;
-	iinfo->dst_addr = conn->c_faddr;
+	iinfo->src_addr = conn->c_laddr.s6_addr32[3];
+	iinfo->dst_addr = conn->c_faddr.s6_addr32[3];
 
 	memset(&iinfo->src_gid, 0, sizeof(iinfo->src_gid));
 	memset(&iinfo->dst_gid, 0, sizeof(iinfo->dst_gid));
@@ -341,7 +341,8 @@ static void rds_ib_ic_info(struct socket *sock, unsigned int len,
  * allowed to influence which paths have priority.  We could call userspace
  * asserting this policy "routing".
  */
-static int rds_ib_laddr_check(struct net *net, __be32 addr)
+static int rds_ib_laddr_check(struct net *net, const struct in6_addr *addr,
+			      __u32 scope_id)
 {
 	int ret;
 	struct rdma_cm_id *cm_id;
@@ -357,7 +358,7 @@ static int rds_ib_laddr_check(struct net *net, __be32 addr)
 
 	memset(&sin, 0, sizeof(sin));
 	sin.sin_family = AF_INET;
-	sin.sin_addr.s_addr = addr;
+	sin.sin_addr.s_addr = addr->s6_addr32[3];
 
 	/* rdma_bind_addr will only succeed for IB & iWARP devices */
 	ret = rdma_bind_addr(cm_id, (struct sockaddr *)&sin);
@@ -367,9 +368,9 @@ static int rds_ib_laddr_check(struct net *net, __be32 addr)
 	    cm_id->device->node_type != RDMA_NODE_IB_CA)
 		ret = -EADDRNOTAVAIL;
 
-	rdsdebug("addr %pI4 ret %d node type %d\n",
-		&addr, ret,
-		cm_id->device ? cm_id->device->node_type : -1);
+	rdsdebug("addr %pI6c ret %d node type %d\n",
+		 addr, ret,
+		 cm_id->device ? cm_id->device->node_type : -1);
 
 	rdma_destroy_id(cm_id);
 
net/rds/ib.h +40 -11

71 lines not on this page · raw message

net/rds/ib_cm.c +221 -78

454 lines not on this page · raw message

net/rds/ib_rdma.c +8 -7
index e678699..0ec9df0 100644
@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2006 Oracle.  All rights reserved.
+ * Copyright (c) 2006, 2017 Oracle and/or its affiliates. All rights reserved.
  *
  * This software is available to you under a choice of one of two
  * licenses.  You may choose to be licensed under the terms of the GNU
@@ -100,18 +100,19 @@ static void rds_ib_remove_ipaddr(struct rds_ib_device *rds_ibdev, __be32 ipaddr)
 		kfree_rcu(to_free, rcu);
 }
 
-int rds_ib_update_ipaddr(struct rds_ib_device *rds_ibdev, __be32 ipaddr)
+int rds_ib_update_ipaddr(struct rds_ib_device *rds_ibdev,
+			 struct in6_addr *ipaddr)
 {
 	struct rds_ib_device *rds_ibdev_old;
 
-	rds_ibdev_old = rds_ib_get_device(ipaddr);
+	rds_ibdev_old = rds_ib_get_device(ipaddr->s6_addr32[3]);
 	if (!rds_ibdev_old)
-		return rds_ib_add_ipaddr(rds_ibdev, ipaddr);
+		return rds_ib_add_ipaddr(rds_ibdev, ipaddr->s6_addr32[3]);
 
 	if (rds_ibdev_old != rds_ibdev) {
-		rds_ib_remove_ipaddr(rds_ibdev_old, ipaddr);
+		rds_ib_remove_ipaddr(rds_ibdev_old, ipaddr->s6_addr32[3]);
 		rds_ib_dev_put(rds_ibdev_old);
-		return rds_ib_add_ipaddr(rds_ibdev, ipaddr);
+		return rds_ib_add_ipaddr(rds_ibdev, ipaddr->s6_addr32[3]);
 	}
 	rds_ib_dev_put(rds_ibdev_old);
 
@@ -544,7 +545,7 @@ void *rds_ib_get_mr(struct scatterlist *sg, unsigned long nents,
 	struct rds_ib_connection *ic = rs->rs_conn->c_transport_data;
 	int ret;
 
-	rds_ibdev = rds_ib_get_device(rs->rs_bound_addr);
+	rds_ibdev = rds_ib_get_device(rs->rs_bound_addr.s6_addr32[3]);
 	if (!rds_ibdev) {
 		ret = -ENODEV;
 		goto out;
net/rds/ib_recv.c +9 -9
index b4e421a..f101bed 100644
@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2006 Oracle.  All rights reserved.
+ * Copyright (c) 2006, 2017 Oracle and/or its affiliates. All rights reserved.
  *
  * This software is available to you under a choice of one of two
  * licenses.  You may choose to be licensed under the terms of the GNU
@@ -266,7 +266,7 @@ static struct rds_ib_incoming *rds_ib_refill_one_inc(struct rds_ib_connection *i
 		rds_ib_stats_inc(s_ib_rx_total_incs);
 	}
 	INIT_LIST_HEAD(&ibinc->ii_frags);
-	rds_inc_init(&ibinc->ii_inc, ic->conn, ic->conn->c_faddr);
+	rds_inc_init(&ibinc->ii_inc, ic->conn, &ic->conn->c_faddr);
 
 	return ibinc;
 }
@@ -420,7 +420,7 @@ void rds_ib_recv_refill(struct rds_connection *conn, int prefill, gfp_t gfp)
 		ret = ib_post_recv(ic->i_cm_id->qp, &recv->r_wr, &failed_wr);
 		if (ret) {
 			rds_ib_conn_error(conn, "recv post on "
-			       "%pI4 returned %d, disconnecting and "
+			       "%pI6c returned %d, disconnecting and "
 			       "reconnecting\n", &conn->c_faddr,
 			       ret);
 			break;
@@ -850,7 +850,7 @@ static void rds_ib_process_recv(struct rds_connection *conn,
 
 	if (data_len < sizeof(struct rds_header)) {
 		rds_ib_conn_error(conn, "incoming message "
-		       "from %pI4 didn't include a "
+		       "from %pI6c didn't include a "
 		       "header, disconnecting and "
 		       "reconnecting\n",
 		       &conn->c_faddr);
@@ -863,7 +863,7 @@ static void rds_ib_process_recv(struct rds_connection *conn,
 	/* Validate the checksum. */
 	if (!rds_message_verify_checksum(ihdr)) {
 		rds_ib_conn_error(conn, "incoming message "
-		       "from %pI4 has corrupted header - "
+		       "from %pI6c has corrupted header - "
 		       "forcing a reconnect\n",
 		       &conn->c_faddr);
 		rds_stats_inc(s_recv_drop_bad_checksum);
@@ -943,10 +943,10 @@ static void rds_ib_process_recv(struct rds_connection *conn,
 		ic->i_recv_data_rem = 0;
 		ic->i_ibinc = NULL;
 
-		if (ibinc->ii_inc.i_hdr.h_flags == RDS_FLAG_CONG_BITMAP)
+		if (ibinc->ii_inc.i_hdr.h_flags == RDS_FLAG_CONG_BITMAP) {
 			rds_ib_cong_recv(conn, ibinc);
-		else {
-			rds_recv_incoming(conn, conn->c_faddr, conn->c_laddr,
+		} else {
+			rds_recv_incoming(conn, &conn->c_faddr, &conn->c_laddr,
 					  &ibinc->ii_inc, GFP_ATOMIC);
 			state->ack_next = be64_to_cpu(hdr->h_sequence);
 			state->ack_next_valid = 1;
@@ -990,7 +990,7 @@ void rds_ib_recv_cqe_handler(struct rds_ib_connection *ic,
 	} else {
 		/* We expect errors as the qp is drained during shutdown */
 		if (rds_conn_up(conn) || rds_conn_connecting(conn))
-			rds_ib_conn_error(conn, "recv completion on <%pI4,%pI4> had status %u (%s), disconnecting and reconnecting\n",
+			rds_ib_conn_error(conn, "recv completion on <%pI6c,%pI6c> had status %u (%s), disconnecting and reconnecting\n",
 					  &conn->c_laddr, &conn->c_faddr,
 					  wc->status,
 					  ib_wc_status_msg(wc->status));
net/rds/ib_send.c +5 -5
index 8557a1c..c4cdfe49 100644
@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2006 Oracle.  All rights reserved.
+ * Copyright (c) 2006, 2017 Oracle and/or its affiliates. All rights reserved.
  *
  * This software is available to you under a choice of one of two
  * licenses.  You may choose to be licensed under the terms of the GNU
@@ -305,7 +305,7 @@ void rds_ib_send_cqe_handler(struct rds_ib_connection *ic, struct ib_wc *wc)
 
 	/* We expect errors as the qp is drained during shutdown */
 	if (wc->status != IB_WC_SUCCESS && rds_conn_up(conn)) {
-		rds_ib_conn_error(conn, "send completion on <%pI4,%pI4> had status %u (%s), disconnecting and reconnecting\n",
+		rds_ib_conn_error(conn, "send completion on <%pI6c,%pI6c> had status %u (%s), disconnecting and reconnecting\n",
 				  &conn->c_laddr, &conn->c_faddr, wc->status,
 				  ib_wc_status_msg(wc->status));
 	}
@@ -730,7 +730,7 @@ int rds_ib_xmit(struct rds_connection *conn, struct rds_message *rm,
 		 first, &first->s_wr, ret, failed_wr);
 	BUG_ON(failed_wr != &first->s_wr);
 	if (ret) {
-		printk(KERN_WARNING "RDS/IB: ib_post_send to %pI4 "
+		printk(KERN_WARNING "RDS/IB: ib_post_send to %pI6c "
 		       "returned %d\n", &conn->c_faddr, ret);
 		rds_ib_ring_unalloc(&ic->i_send_ring, work_alloc);
 		rds_ib_sub_signaled(ic, nr_sig);
@@ -827,7 +827,7 @@ int rds_ib_xmit_atomic(struct rds_connection *conn, struct rm_atomic_op *op)
 		 send, &send->s_atomic_wr, ret, failed_wr);
 	BUG_ON(failed_wr != &send->s_atomic_wr.wr);
 	if (ret) {
-		printk(KERN_WARNING "RDS/IB: atomic ib_post_send to %pI4 "
+		printk(KERN_WARNING "RDS/IB: atomic ib_post_send to %pI6c "
 		       "returned %d\n", &conn->c_faddr, ret);
 		rds_ib_ring_unalloc(&ic->i_send_ring, work_alloc);
 		rds_ib_sub_signaled(ic, nr_sig);
@@ -967,7 +967,7 @@ int rds_ib_xmit_rdma(struct rds_connection *conn, struct rm_rdma_op *op)
 		 first, &first->s_rdma_wr.wr, ret, failed_wr);
 	BUG_ON(failed_wr != &first->s_rdma_wr.wr);
 	if (ret) {
-		printk(KERN_WARNING "RDS/IB: rdma ib_post_send to %pI4 "
+		printk(KERN_WARNING "RDS/IB: rdma ib_post_send to %pI6c "
 		       "returned %d\n", &conn->c_faddr, ret);
 		rds_ib_ring_unalloc(&ic->i_send_ring, work_alloc);
 		rds_ib_sub_signaled(ic, nr_sig);
net/rds/loop.c +4 -3
index feea1f9..1d73ad7 100644
@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2006 Oracle.  All rights reserved.
+ * Copyright (c) 2006, 2017 Oracle and/or its affiliates. All rights reserved.
  *
  * This software is available to you under a choice of one of two
  * licenses.  You may choose to be licensed under the terms of the GNU
@@ -35,6 +35,7 @@
 #include <linux/in.h>
 #include <net/net_namespace.h>
 #include <net/netns/generic.h>
+#include <linux/ipv6.h>
 
 #include "rds_single_path.h"
 #include "rds.h"
@@ -88,11 +89,11 @@ static int rds_loop_xmit(struct rds_connection *conn, struct rds_message *rm,
 
 	BUG_ON(hdr_off || sg || off);
 
-	rds_inc_init(&rm->m_inc, conn, conn->c_laddr);
+	rds_inc_init(&rm->m_inc, conn, &conn->c_laddr);
 	/* For the embedded inc. Matching put is in loop_inc_free() */
 	rds_message_addref(rm);
 
-	rds_recv_incoming(conn, conn->c_laddr, conn->c_faddr, &rm->m_inc,
+	rds_recv_incoming(conn, &conn->c_laddr, &conn->c_faddr, &rm->m_inc,
 			  GFP_KERNEL);
 
 	rds_send_drop_acked(conn, be64_to_cpu(rm->m_inc.i_hdr.h_sequence),
net/rds/rdma.c +3 -3
index 634cfcb..7b39980 100644
@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2007 Oracle.  All rights reserved.
+ * Copyright (c) 2007, 2017 Oracle and/or its affiliates. All rights reserved.
  *
  * This software is available to you under a choice of one of two
  * licenses.  You may choose to be licensed under the terms of the GNU
@@ -183,7 +183,7 @@ static int __rds_rdma_map(struct rds_sock *rs, struct rds_get_mr_args *args,
 	long i;
 	int ret;
 
-	if (rs->rs_bound_addr == 0 || !rs->rs_transport) {
+	if (ipv6_addr_any(&rs->rs_bound_addr) || !rs->rs_transport) {
 		ret = -ENOTCONN; /* XXX not a great errno */
 		goto out;
 	}
@@ -574,7 +574,7 @@ int rds_cmsg_rdma_args(struct rds_sock *rs, struct rds_message *rm,
 
 	args = CMSG_DATA(cmsg);
 
-	if (rs->rs_bound_addr == 0) {
+	if (ipv6_addr_any(&rs->rs_bound_addr)) {
 		ret = -ENOTCONN; /* XXX not a great errno */
 		goto out_ret;
 	}
net/rds/rdma_transport.c +40 -16

122 lines not on this page · raw message

net/rds/rds.h +47 -23

169 lines not on this page · raw message

net/rds/recv.c +34 -17

126 lines not on this page · raw message

net/rds/send.c +52 -15

125 lines not on this page · raw message

net/rds/tcp.c +29 -3
index 351a284..dadb337 100644
@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2006 Oracle.  All rights reserved.
+ * Copyright (c) 2006, 2017 Oracle and/or its affiliates. All rights reserved.
  *
  * This software is available to you under a choice of one of two
  * licenses.  You may choose to be licensed under the terms of the GNU
@@ -37,6 +37,8 @@
 #include <net/tcp.h>
 #include <net/net_namespace.h>
 #include <net/netns/generic.h>
+#include <net/tcp.h>
+#include <net/addrconf.h>
 
 #include "rds.h"
 #include "tcp.h"
@@ -262,9 +264,33 @@ static void rds_tcp_tc_info(struct socket *rds_sock, unsigned int len,
 	spin_unlock_irqrestore(&rds_tcp_tc_list_lock, flags);
 }
 
-static int rds_tcp_laddr_check(struct net *net, __be32 addr)
+static int rds_tcp_laddr_check(struct net *net, const struct in6_addr *addr,
+			       __u32 scope_id)
 {
-	if (inet_addr_type(net, addr) == RTN_LOCAL)
+	struct net_device *dev = NULL;
+	int ret;
+
+	if (ipv6_addr_v4mapped(addr)) {
+		if (inet_addr_type(net, addr->s6_addr32[3]) == RTN_LOCAL)
+			return 0;
+		return -EADDRNOTAVAIL;
+	}
+
+	/* If the scope_id is specified, check only those addresses
+	 * hosted on the specified interface.
+	 */
+	if (scope_id != 0) {
+		rcu_read_lock();
+		dev = dev_get_by_index_rcu(net, scope_id);
+		/* scope_id is not valid... */
+		if (!dev) {
+			rcu_read_unlock();
+			return -EADDRNOTAVAIL;
+		}
+		rcu_read_unlock();
+	}
+	ret = ipv6_chk_addr(net, addr, dev, 0);
+	if (ret)
 		return 0;
 	return -EADDRNOTAVAIL;
 }
net/rds/tcp_connect.c +20 -14
index d999e70..231ae92 100644
@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2006 Oracle.  All rights reserved.
+ * Copyright (c) 2006, 2017 Oracle and/or its affiliates. All rights reserved.
  *
  * This software is available to you under a choice of one of two
  * licenses.  You may choose to be licensed under the terms of the GNU
@@ -66,7 +66,8 @@ void rds_tcp_state_change(struct sock *sk)
 		 * RDS connection as RDS_CONN_UP until the reconnect,
 		 * to avoid RDS datagram loss.
 		 */
-		if (!IS_CANONICAL(cp->cp_conn->c_laddr, cp->cp_conn->c_faddr) &&
+		if (rds_addr_cmp(&cp->cp_conn->c_laddr,
+				 &cp->cp_conn->c_faddr) >= 0 &&
 		    rds_conn_path_transition(cp, RDS_CONN_CONNECTING,
 					     RDS_CONN_ERROR)) {
 			rds_conn_path_drop(cp, false);
@@ -88,7 +89,9 @@ void rds_tcp_state_change(struct sock *sk)
 int rds_tcp_conn_path_connect(struct rds_conn_path *cp)
 {
 	struct socket *sock = NULL;
-	struct sockaddr_in src, dest;
+	struct sockaddr_in sin;
+	struct sockaddr *addr;
+	int addrlen;
 	int ret;
 	struct rds_connection *conn = cp->cp_conn;
 	struct rds_tcp_connection *tc = cp->cp_transport_data;
@@ -112,30 +115,33 @@ int rds_tcp_conn_path_connect(struct rds_conn_path *cp)
 
 	rds_tcp_tune(sock);
 
-	src.sin_family = AF_INET;
-	src.sin_addr.s_addr = (__force u32)conn->c_laddr;
-	src.sin_port = (__force u16)htons(0);
+	sin.sin_family = AF_INET;
+	sin.sin_addr.s_addr = conn->c_laddr.s6_addr32[3];
+	sin.sin_port = 0;
+	addr = (struct sockaddr *)&sin;
+	addrlen = sizeof(sin);
 
-	ret = sock->ops->bind(sock, (struct sockaddr *)&src, sizeof(src));
+	ret = sock->ops->bind(sock, addr, addrlen);
 	if (ret) {
-		rdsdebug("bind failed with %d at address %pI4\n",
+		rdsdebug("bind failed with %d at address %pI6c\n",
 			 ret, &conn->c_laddr);
 		goto out;
 	}
 
-	dest.sin_family = AF_INET;
-	dest.sin_addr.s_addr = (__force u32)conn->c_faddr;
-	dest.sin_port = (__force u16)htons(RDS_TCP_PORT);
+	sin.sin_family = AF_INET;
+	sin.sin_addr.s_addr = conn->c_faddr.s6_addr32[3];
+	sin.sin_port = htons(RDS_TCP_PORT);
+	addr = (struct sockaddr *)&sin;
+	addrlen = sizeof(sin);
 
 	/*
 	 * once we call connect() we can start getting callbacks and they
 	 * own the socket
 	 */
 	rds_tcp_set_callbacks(sock, cp);
-	ret = sock->ops->connect(sock, (struct sockaddr *)&dest, sizeof(dest),
-				 O_NONBLOCK);
+	ret = sock->ops->connect(sock, addr, addrlen, O_NONBLOCK);
 
-	rdsdebug("connect to address %pI4 returned %d\n", &conn->c_faddr, ret);
+	rdsdebug("connect to address %pI6c returned %d\n", &conn->c_faddr, ret);
 	if (ret == -EINPROGRESS)
 		ret = 0;
 	if (ret == 0) {
net/rds/tcp_listen.c +10 -8
index 2257118..4fdf5b3 100644
@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2006, 2018 Oracle.  All rights reserved.
+ * Copyright (c) 2006, 2018 Oracle and/or its affiliates. All rights reserved.
  *
  * This software is available to you under a choice of one of two
  * licenses.  You may choose to be licensed under the terms of the GNU
@@ -83,13 +83,12 @@ int rds_tcp_keepalive(struct socket *sock)
 struct rds_tcp_connection *rds_tcp_accept_one_path(struct rds_connection *conn)
 {
 	int i;
-	bool peer_is_smaller = IS_CANONICAL(conn->c_faddr, conn->c_laddr);
 	int npaths = max_t(int, 1, conn->c_npaths);
 
 	/* for mprds, all paths MUST be initiated by the peer
 	 * with the smaller address.
 	 */
-	if (!peer_is_smaller) {
+	if (rds_addr_cmp(&conn->c_faddr, &conn->c_laddr) >= 0) {
 		/* Make sure we initiate at least one path if this
 		 * has not already been done; rds_start_mprds() will
 		 * take care of additional paths, if necessary.
@@ -164,13 +163,16 @@ int rds_tcp_accept_one(struct socket *sock)
 
 	inet = inet_sk(new_sock->sk);
 
-	rdsdebug("accepted tcp %pI4:%u -> %pI4:%u\n",
-		 &inet->inet_saddr, ntohs(inet->inet_sport),
-		 &inet->inet_daddr, ntohs(inet->inet_dport));
+	rdsdebug("accepted tcp %pI6c:%u -> %pI6c:%u\n",
+		 &new_sock->sk->sk_v6_rcv_saddr, ntohs(inet->inet_sport),
+		 &new_sock->sk->sk_v6_daddr, ntohs(inet->inet_dport));
 
 	conn = rds_conn_create(sock_net(sock->sk),
-			       inet->inet_saddr, inet->inet_daddr,
-			       &rds_tcp_transport, GFP_KERNEL);
+			       &new_sock->sk->sk_v6_rcv_saddr,
+			       &new_sock->sk->sk_v6_daddr,
+			       &rds_tcp_transport, GFP_KERNEL,
+			       new_sock->sk->sk_bound_dev_if);
+
 	if (IS_ERR(conn)) {
 		ret = PTR_ERR(conn);
 		goto out;
net/rds/tcp_recv.c +5 -4
index b9fbd2e..42c5ff1 100644
@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2006 Oracle.  All rights reserved.
+ * Copyright (c) 2006, 2017 Oracle and/or its affiliates. All rights reserved.
  *
  * This software is available to you under a choice of one of two
  * licenses.  You may choose to be licensed under the terms of the GNU
@@ -179,7 +179,7 @@ static int rds_tcp_data_recv(read_descriptor_t *desc, struct sk_buff *skb,
 			tc->t_tinc = tinc;
 			rdsdebug("alloced tinc %p\n", tinc);
 			rds_inc_path_init(&tinc->ti_inc, cp,
-					  cp->cp_conn->c_faddr);
+					  &cp->cp_conn->c_faddr);
 			tinc->ti_inc.i_rx_lat_trace[RDS_MSG_RX_HDR] =
 					local_clock();
 
@@ -239,8 +239,9 @@ static int rds_tcp_data_recv(read_descriptor_t *desc, struct sk_buff *skb,
 			if (tinc->ti_inc.i_hdr.h_flags == RDS_FLAG_CONG_BITMAP)
 				rds_tcp_cong_recv(conn, tinc);
 			else
-				rds_recv_incoming(conn, conn->c_faddr,
-						  conn->c_laddr, &tinc->ti_inc,
+				rds_recv_incoming(conn, &conn->c_faddr,
+						  &conn->c_laddr,
+						  &tinc->ti_inc,
 						  arg->gfp);
 
 			tc->t_tinc_hdr_rem = sizeof(struct rds_header);
net/rds/tcp_send.c +2 -2
index 7df869d..78a2554 100644
@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2006 Oracle.  All rights reserved.
+ * Copyright (c) 2006, 2017 Oracle and/or its affiliates. All rights reserved.
  *
  * This software is available to you under a choice of one of two
  * licenses.  You may choose to be licensed under the terms of the GNU
@@ -153,7 +153,7 @@ int rds_tcp_xmit(struct rds_connection *conn, struct rds_message *rm,
 			 * an incoming RST.
 			 */
 			if (rds_conn_path_up(cp)) {
-				pr_warn("RDS/tcp: send to %pI4 on cp [%d]"
+				pr_warn("RDS/tcp: send to %pI6c on cp [%d]"
 					"returned %d, "
 					"disconnecting and reconnecting\n",
 					&conn->c_faddr, cp->cp_index, ret);
net/rds/threads.c +58 -11

108 lines not on this page · raw message

net/rds/transport.c +11 -4
index 0b188dd..c9788db 100644
@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2006 Oracle.  All rights reserved.
+ * Copyright (c) 2006, 2017 Oracle and/or its affiliates. All rights reserved.
  *
  * This software is available to you under a choice of one of two
  * licenses.  You may choose to be licensed under the terms of the GNU
@@ -33,6 +33,7 @@
 #include <linux/kernel.h>
 #include <linux/module.h>
 #include <linux/in.h>
+#include <linux/ipv6.h>
 
 #include "rds.h"
 #include "loop.h"
@@ -75,20 +76,26 @@ void rds_trans_put(struct rds_transport *trans)
 		module_put(trans->t_owner);
 }
 
-struct rds_transport *rds_trans_get_preferred(struct net *net, __be32 addr)
+struct rds_transport *rds_trans_get_preferred(struct net *net,
+					      const struct in6_addr *addr,
+					      __u32 scope_id)
 {
 	struct rds_transport *ret = NULL;
 	struct rds_transport *trans;
 	unsigned int i;
 
-	if (IN_LOOPBACK(ntohl(addr)))
+	if (ipv6_addr_v4mapped(addr)) {
+		if (*(u_int8_t *)&addr->s6_addr32[3] == IN_LOOPBACKNET)
+			return &rds_loop_transport;
+	} else if (ipv6_addr_loopback(addr)) {
 		return &rds_loop_transport;
+	}
 
 	down_read(&rds_trans_sem);
 	for (i = 0; i < RDS_TRANS_COUNT; i++) {
 		trans = transports[i];
 
-		if (trans && (trans->laddr_check(net, addr) == 0) &&
+		if (trans && (trans->laddr_check(net, addr, scope_id) == 0) &&
 		    (!trans->t_owner || try_module_get(trans->t_owner))) {
 			ret = trans;
 			break;
Signature · 1 line
1.8.3.1
#2 lore raw

PATCH v3 2/3 rds: Enable RDS IPv6 support

In reply to Ka-Cheong Poon

This patch enables RDS to use IPv6 addresses. For RDS/TCP, the listener is now an IPv6 endpoint which accepts both IPv4 and IPv6 connection requests. RDS/RDMA/IB uses a private data (struct rds_ib_connect_private) exchange between endpoints at RDS connection establishment time to support RDMA. This private data exchange uses a 32 bit integer to represent an IP address. This needs to be changed in order to support IPv6. A new private data struct rds6_ib_connect_private is introduced to handle this. To ensure backward compatibility, an IPv6 capable RDS stack uses another RDMA listener port (RDS_CM_PORT) to accept IPv6 connection. And it continues to use the original RDS_PORT for IPv4 RDS connections. When it needs to communicate with an IPv6 peer, it uses the RDS_CM_PORT to send the connection set up request.

v3: Added support to set up IPv4 connection using mapped address
(David Miller).
Added support to set up connection between link local and non-link addresses.
Various review comments from Santosh Shilimkar and Sowmini Varadhan.

v2: Fixed bound and peer address scope mismatched issue.
Added back rds_connect() IPv6 changes.

---
 net/rds/af_rds.c         | 91 ++++++++++++++++++++++++++++++++++++++++--------
 net/rds/bind.c           | 59 ++++++++++++++++++++++++++-----
 net/rds/connection.c     | 54 ++++++++++++++++++++--------
 net/rds/ib.c             | 55 ++++++++++++++++++++++++-----
 net/rds/ib_cm.c          | 20 ++++++++---
 net/rds/rdma_transport.c | 30 +++++++++++++++-
 net/rds/rdma_transport.h |  5 +++
 net/rds/rds.h            | 21 ++++++-----
 net/rds/recv.c           |  2 +-
 net/rds/send.c           | 61 ++++++++++++++++++++++++++++----
 net/rds/tcp.c            | 54 +++++++++++++++++-----------
 net/rds/tcp.h            |  2 +-
 net/rds/tcp_connect.c    | 54 +++++++++++++++++++++-------
 net/rds/tcp_listen.c     | 64 +++++++++++++++++++++++++++-------
 14 files changed, 458 insertions(+), 114 deletions(-)
net/rds/af_rds.c +77 -14
index fc1a5c6..fc5c48b 100644
@@ -142,15 +142,32 @@ static int rds_getname(struct socket *sock, struct sockaddr *uaddr,
 			uaddr_len = sizeof(*sin6);
 		}
 	} else {
-		/* If socket is not yet bound, set the return address family
-		 * to be AF_UNSPEC (value 0) and the address size to be that
-		 * of an IPv4 address.
+		/* If socket is not yet bound and the socket is connected,
+		 * set the return address family to be the same as the
+		 * connected address, but with 0 address value.  If it is not
+		 * connected, set the family to be AF_UNSPEC (value 0) and
+		 * the address size to be that of an IPv4 address.
 		 */
 		if (ipv6_addr_any(&rs->rs_bound_addr)) {
-			sin = (struct sockaddr_in *)uaddr;
-			memset(sin, 0, sizeof(*sin));
-			sin->sin_family = AF_UNSPEC;
-			return sizeof(*sin);
+			if (ipv6_addr_any(&rs->rs_conn_addr)) {
+				sin = (struct sockaddr_in *)uaddr;
+				memset(sin, 0, sizeof(*sin));
+				sin->sin_family = AF_UNSPEC;
+				return sizeof(*sin);
+			}
+
+			if (ipv6_addr_type(&rs->rs_conn_addr) &
+			    IPV6_ADDR_MAPPED) {
+				sin = (struct sockaddr_in *)uaddr;
+				memset(sin, 0, sizeof(*sin));
+				sin->sin_family = AF_INET;
+				return sizeof(*sin);
+			}
+
+			sin6 = (struct sockaddr_in6 *)uaddr;
+			memset(sin6, 0, sizeof(*sin6));
+			sin6->sin6_family = AF_INET6;
+			return sizeof(*sin6);
 		}
 		if (ipv6_addr_v4mapped(&rs->rs_bound_addr)) {
 			sin = (struct sockaddr_in *)uaddr;
@@ -484,16 +501,18 @@ static int rds_connect(struct socket *sock, struct sockaddr *uaddr,
 {
 	struct sock *sk = sock->sk;
 	struct sockaddr_in *sin;
+	struct sockaddr_in6 *sin6;
 	struct rds_sock *rs = rds_sk_to_rs(sk);
+	int addr_type;
 	int ret = 0;
 
 	lock_sock(sk);
 
-	switch (addr_len) {
-	case sizeof(struct sockaddr_in):
+	switch (uaddr->sa_family) {
+	case AF_INET:
 		sin = (struct sockaddr_in *)uaddr;
-		if (sin->sin_family != AF_INET) {
-			ret = -EAFNOSUPPORT;
+		if (addr_len < sizeof(struct sockaddr_in)) {
+			ret = -EINVAL;
 			break;
 		}
 		if (sin->sin_addr.s_addr == htonl(INADDR_ANY)) {
@@ -509,12 +528,56 @@ static int rds_connect(struct socket *sock, struct sockaddr *uaddr,
 		rs->rs_conn_port = sin->sin_port;
 		break;
 
-	case sizeof(struct sockaddr_in6):
-		ret = -EPROTONOSUPPORT;
+	case AF_INET6:
+		sin6 = (struct sockaddr_in6 *)uaddr;
+		if (addr_len < sizeof(struct sockaddr_in6)) {
+			ret = -EINVAL;
+			break;
+		}
+		addr_type = ipv6_addr_type(&sin6->sin6_addr);
+		if (!(addr_type & IPV6_ADDR_UNICAST)) {
+			__be32 addr4;
+
+			if (!(addr_type & IPV6_ADDR_MAPPED)) {
+				ret = -EPROTOTYPE;
+				break;
+			}
+
+			/* It is a mapped address.  Need to do some sanity
+			 * checks.
+			 */
+			addr4 = sin6->sin6_addr.s6_addr32[3];
+			if (addr4 == htonl(INADDR_ANY) ||
+			    addr4 == htonl(INADDR_BROADCAST) ||
+			    IN_MULTICAST(ntohl(addr4))) {
+				ret = -EPROTOTYPE;
+				break;
+			}
+		}
+
+		if (addr_type & IPV6_ADDR_LINKLOCAL) {
+			/* If socket is arleady bound to a link local address,
+			 * the peer address must be on the same link.
+			 */
+			if (sin6->sin6_scope_id == 0 ||
+			    (!ipv6_addr_any(&rs->rs_bound_addr) &&
+			     rs->rs_bound_scope_id &&
+			     sin6->sin6_scope_id != rs->rs_bound_scope_id)) {
+				ret = -EINVAL;
+				break;
+			}
+			/* Remember the connected address scope ID.  It will
+			 * be checked against the binding local address when
+			 * the socket is bound.
+			 */
+			rs->rs_bound_scope_id = sin6->sin6_scope_id;
+		}
+		rs->rs_conn_addr = sin6->sin6_addr;
+		rs->rs_conn_port = sin6->sin6_port;
 		break;
 
 	default:
-		ret = -EINVAL;
+		ret = -EAFNOSUPPORT;
 		break;
 	}
 
net/rds/bind.c +50 -9

88 lines not on this page · raw message

net/rds/connection.c +39 -15

96 lines not on this page · raw message

net/rds/ib.c +47 -8

99 lines not on this page · raw message

net/rds/ib_cm.c +15 -5
index dd8a867..a33b82d 100644
@@ -678,7 +678,7 @@ static u32 rds_ib_protocol_compatible(struct rdma_cm_event *event, bool isv6)
 	return version;
 }
 
-/* Given an IPv6 address, find the IB net_device which hosts that address and
+/* Given an IPv6 address, find the net_device which hosts that address and
  * return its index.  This is used by the rds_ib_cm_handle_connect() code to
  * find the interface index of where an incoming request comes from when
  * the request is using a link local address.
@@ -695,8 +695,7 @@ static u32 __rds_find_ifindex(struct net *net, const struct in6_addr *addr)
 
 	rcu_read_lock();
 	for_each_netdev_rcu(net, dev) {
-		if (dev->type == ARPHRD_INFINIBAND &&
-		    ipv6_chk_addr(net, addr, dev, 0)) {
+		if (ipv6_chk_addr(net, addr, dev, 1)) {
 			idx = dev->ifindex;
 			break;
 		}
@@ -736,7 +735,7 @@ int rds_ib_cm_handle_connect(struct rdma_cm_id *cm_id,
 		dp_cmn = &dp->ricp_v6.dp_cmn;
 		saddr6 = &dp->ricp_v6.dp_saddr;
 		daddr6 = &dp->ricp_v6.dp_daddr;
-		/* If the local address is link local, need to find the
+		/* If either address is link local, need to find the
 		 * interface index in order to create a proper RDS
 		 * connection.
 		 */
@@ -748,6 +747,14 @@ int rds_ib_cm_handle_connect(struct rdma_cm_id *cm_id,
 				err = -EOPNOTSUPP;
 				goto out;
 			}
+		} else if (ipv6_addr_type(saddr6) & IPV6_ADDR_LINKLOCAL) {
+			/* Use our address to find the correct index. */
+			ifindex = __rds_find_ifindex(&init_net, daddr6);
+			/* No index found...  Need to bail out. */
+			if (ifindex == 0) {
+				err = -EOPNOTSUPP;
+				goto out;
+			}
 		}
 	} else {
 		dp_cmn = &dp->ricp_v4.dp_cmn;
@@ -886,7 +893,10 @@ int rds_ib_conn_path_connect(struct rds_conn_path *cp)
 
 	/* XXX I wonder what affect the port space has */
 	/* delegate cm event handler to rdma_transport */
-	handler = rds_rdma_cm_event_handler;
+	if (conn->c_isv6)
+		handler = rds6_rdma_cm_event_handler;
+	else
+		handler = rds_rdma_cm_event_handler;
 	ic->i_cm_id = rdma_create_id(&init_net, handler, conn,
 				     RDMA_PS_TCP, IB_QPT_RC);
 	if (IS_ERR(ic->i_cm_id)) {
net/rds/rdma_transport.c +29 -1
index f49abef6..bd67e55 100644
@@ -37,7 +37,9 @@
 #include "rdma_transport.h"
 #include "ib.h"
 
+/* Global IPv4 and IPv6 RDS RDMA listener cm_id */
 static struct rdma_cm_id *rds_rdma_listen_id;
+static struct rdma_cm_id *rds6_rdma_listen_id;
 
 static int rds_rdma_cm_event_handler_cmn(struct rdma_cm_id *cm_id,
 					 struct rdma_cm_event *event,
@@ -153,6 +155,12 @@ int rds_rdma_cm_event_handler(struct rdma_cm_id *cm_id,
 	return rds_rdma_cm_event_handler_cmn(cm_id, event, false);
 }
 
+int rds6_rdma_cm_event_handler(struct rdma_cm_id *cm_id,
+			       struct rdma_cm_event *event)
+{
+	return rds_rdma_cm_event_handler_cmn(cm_id, event, true);
+}
+
 static int rds_rdma_listen_init_common(rdma_cm_event_handler handler,
 				       struct sockaddr *sa,
 				       struct rdma_cm_id **ret_cm_id)
@@ -206,6 +214,7 @@ static int rds_rdma_listen_init_common(rdma_cm_event_handler handler,
 static int rds_rdma_listen_init(void)
 {
 	int ret;
+	struct sockaddr_in6 sin6;
 	struct sockaddr_in sin;
 
 	sin.sin_family = PF_INET;
@@ -214,7 +223,21 @@ static int rds_rdma_listen_init(void)
 	ret = rds_rdma_listen_init_common(rds_rdma_cm_event_handler,
 					  (struct sockaddr *)&sin,
 					  &rds_rdma_listen_id);
-	return ret;
+	if (ret != 0)
+		return ret;
+
+	sin6.sin6_family = PF_INET6;
+	sin6.sin6_addr = in6addr_any;
+	sin6.sin6_port = htons(RDS_CM_PORT);
+	sin6.sin6_scope_id = 0;
+	sin6.sin6_flowinfo = 0;
+	ret = rds_rdma_listen_init_common(rds6_rdma_cm_event_handler,
+					  (struct sockaddr *)&sin6,
+					  &rds6_rdma_listen_id);
+	/* Keep going even when IPv6 is not enabled in the system. */
+	if (ret != 0)
+		rdsdebug("Cannot set up IPv6 RDMA listener\n");
+	return 0;
 }
 
 static void rds_rdma_listen_stop(void)
@@ -224,6 +247,11 @@ static void rds_rdma_listen_stop(void)
 		rdma_destroy_id(rds_rdma_listen_id);
 		rds_rdma_listen_id = NULL;
 	}
+	if (rds6_rdma_listen_id) {
+		rdsdebug("cm %p\n", rds6_rdma_listen_id);
+		rdma_destroy_id(rds6_rdma_listen_id);
+		rds6_rdma_listen_id = NULL;
+	}
 }
 
 static int rds_rdma_init(void)
net/rds/rdma_transport.h +5 -0
index d309c44..200d313 100644
@@ -6,11 +6,16 @@
 #include <rdma/rdma_cm.h>
 #include "rds.h"
 
+/* RDMA_CM also uses 16385 as the listener port. */
+#define RDS_CM_PORT	16385
+
 #define RDS_RDMA_RESOLVE_TIMEOUT_MS     5000
 
 int rds_rdma_conn_connect(struct rds_connection *conn);
 int rds_rdma_cm_event_handler(struct rdma_cm_id *cm_id,
 			      struct rdma_cm_event *event);
+int rds6_rdma_cm_event_handler(struct rdma_cm_id *cm_id,
+			       struct rdma_cm_event *event);
 
 /* from ib.c */
 extern struct rds_transport rds_ib_transport;
net/rds/rds.h +13 -8
index 1bff269..f2c4f44 100644
@@ -24,14 +24,14 @@
 #define RDS_PROTOCOL_MINOR(v)	((v) & 255)
 #define RDS_PROTOCOL(maj, min)	(((maj) << 8) | min)
 
-/*
- * XXX randomly chosen, but at least seems to be unused:
- * #               18464-18768 Unassigned
- * We should do better.  We want a reserved port to discourage unpriv'ed
- * userspace from listening.
+/* The following ports, 16385, 18634, 18635, are registered with IANA as
+ * the ports to be used for RDS over TCP and UDP.  18634 is the historical
+ * value used for the RDMA_CM listener port.  RDS/TCP uses port 16385.  After
+ * IPv6 work, RDMA_CM also uses 16385 as the listener port.  18634 is kept
+ * to ensure compatibility with older RDS modules.  Those ports are defined
+ * in each transport's header file.
  */
 #define RDS_PORT	18634
-#define RDS_CM_PORT	16385
 
 #ifdef ATOMIC64_INIT
 #define KERNEL_HAS_ATOMIC64
@@ -140,7 +140,8 @@ struct rds_connection {
 	struct hlist_node	c_hash_node;
 	struct in6_addr		c_laddr;
 	struct in6_addr		c_faddr;
-	int			c_dev_if; /* c_laddrs's interface index */
+	int			c_dev_if; /* ifindex used for this conn */
+	int			c_bound_if; /* ifindex of c_laddr */
 	unsigned int		c_loopback:1,
 				c_isv6:1,
 				c_ping_triggered:1,
@@ -736,7 +737,7 @@ struct rds_sock *rds_find_bound(const struct in6_addr *addr, __be16 port,
 void rds_cong_exit(void);
 struct rds_message *rds_cong_update_alloc(struct rds_connection *conn);
 
-/* conn.c */
+/* connection.c */
 extern u32 rds_gen_num;
 int rds_conn_init(void);
 void rds_conn_exit(void);
@@ -874,6 +875,10 @@ int rds_recvmsg(struct socket *sock, struct msghdr *msg, size_t size,
 void rds_inc_info_copy(struct rds_incoming *inc,
 		       struct rds_info_iterator *iter,
 		       __be32 saddr, __be32 daddr, int flip);
+void rds6_inc_info_copy(struct rds_incoming *inc,
+			struct rds_info_iterator *iter,
+			struct in6_addr *saddr, struct in6_addr *daddr,
+			int flip);
 
 /* send.c */
 int rds_sendmsg(struct socket *sock, struct msghdr *msg, size_t payload_len);
net/rds/recv.c +1 -1
index 4217961..1402c21 100644
@@ -364,7 +364,7 @@ void rds_recv_incoming(struct rds_connection *conn, struct in6_addr *saddr,
 		goto out;
 	}
 
-	rs = rds_find_bound(daddr, inc->i_hdr.h_dport, conn->c_dev_if);
+	rs = rds_find_bound(daddr, inc->i_hdr.h_dport, conn->c_bound_if);
 	if (!rs) {
 		rds_stats_inc(s_recv_drop_no_sock);
 		goto out;
net/rds/send.c +54 -7

79 lines not on this page · raw message

net/rds/tcp.c +34 -20

103 lines not on this page · raw message

net/rds/tcp.h +1 -1
index c6fa080..3c69361 100644
@@ -67,7 +67,7 @@ void rds_tcp_restore_callbacks(struct socket *sock,
 void rds_tcp_state_change(struct sock *sk);
 
 /* tcp_listen.c */
-struct socket *rds_tcp_listen_init(struct net *);
+struct socket *rds_tcp_listen_init(struct net *net, bool isv6);
 void rds_tcp_listen_stop(struct socket *sock, struct work_struct *acceptor);
 void rds_tcp_listen_data_ready(struct sock *sk);
 int rds_tcp_accept_one(struct socket *sock);
net/rds/tcp_connect.c +42 -12

80 lines not on this page · raw message

net/rds/tcp_listen.c +51 -13

100 lines not on this page · raw message

Signature · 1 line
1.8.3.1
#3 lore raw

Re: [PATCH v3 net-next 2/3] rds: Enable RDS IPv6 support

Thank you for the patch! Yet something to improve:

Hi Ka-Cheong,

Thank you for the patch! Yet something to improve:

[auto build test ERROR on net-next/master]

url: https://github.com/0day-ci/linux/commits/Ka-Cheong-Poon/rds-Changing-IP-address-internal-representation-to-struct-in6_addr/20180716-011408
config: powerpc-allmodconfig (attached as .config)
compiler: powerpc64-linux-gnu-gcc (Debian 7.2.0-11) 7.2.0
reproduce:
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
GCC_VERSION=7.2.0 make.cross ARCH=powerpc

All error/warnings (new ones prefixed by >>):

In file included from include/linux/printk.h:332:0,

                    from include/linux/kernel.h:14,
                    from net/rds/tcp_listen.c:33:
   net/rds/tcp_listen.c: In function 'rds_tcp_accept_one':
>> include/linux/dynamic_debug.h:124:1: error: expected expression before 'do'
    do {        \
    ^
   include/linux/printk.h:336:2: note: in expansion of macro 'dynamic_pr_debug'
     dynamic_pr_debug(fmt, ##__VA_ARGS__)
     ^~~~~~~~~~~~~~~~
>> net/rds/rds.h:41:32: note: in expansion of macro 'pr_debug'
    #define rdsdebug(fmt, args...) pr_debug("%s(): " fmt, __func__ , ##args)
                                   ^~~~~~~~
>> net/rds/tcp_listen.c:170:2: note: in expansion of macro 'rdsdebug'
     rdsdebug("accepted tcp %pI6c:%u -> %pI6c:%u\n",
     ^~~~~~~~

vim +/pr_debug +41 net/rds/rds.h

8cbd9606 Andy Grover 2009-04-01 39

ff57087f shamir rabinovitch 2016-10-27  40  #ifdef RDS_DEBUG
39de8281 Andy Grover        2009-02-24 @41  #define rdsdebug(fmt, args...) pr_debug("%s(): " fmt, __func__ , ##args)
39de8281 Andy Grover        2009-02-24  42  #else
39de8281 Andy Grover        2009-02-24  43  /* sigh, pr_debug() causes unused variable warnings */
b9075fa9 Joe Perches        2011-10-31  44  static inline __printf(1, 2)
b9075fa9 Joe Perches        2011-10-31  45  void rdsdebug(char *fmt, ...)
39de8281 Andy Grover        2009-02-24  46  {
39de8281 Andy Grover        2009-02-24  47  }
39de8281 Andy Grover        2009-02-24  48  #endif

39de8281 Andy Grover 2009-02-24 49

:::::: The code at line 41 was first introduced by commit :::::: 39de8281791c4a01abcb0d32879530ffa5863c01 RDS: Main header file

:::::: TO: Andy Grover <[email protected]>
:::::: CC: David S. Miller <[email protected]>

---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation

#4 lore raw

PATCH v3 3/3 rds: Extend RDS API for IPv6 support

In reply to Ka-Cheong Poon

There are many data structures (RDS socket options) used by RDS apps which use a 32 bit integer to store IP address. To support IPv6, struct in6_addr needs to be used. To ensure backward compatibility, a new data structure is introduced for each of those data structures which use a 32 bit integer to represent an IP address. And new socket options are introduced to use those new structures. This means that existing apps should work without a problem with the new RDS module. For apps which want to use IPv6, those new data structures and socket options can be used. IPv4 mapped address is used to represent IPv4 address in the new data structures.

---
 include/uapi/linux/rds.h |  71 +++++++++++++++++++++++++++++++--
 net/rds/connection.c     | 101 +++++++++++++++++++++++++++++++++++++++++++----
 net/rds/ib.c             |  52 ++++++++++++++++++++++++
 net/rds/ib_mr.h          |   2 +
 net/rds/ib_rdma.c        |  11 +++++-
 net/rds/recv.c           |  25 ++++++++++++
 net/rds/tcp.c            |  44 +++++++++++++++++++++
 7 files changed, 294 insertions(+), 12 deletions(-)
include/uapi/linux/rds.h +68 -3
index 20c6bd0..518d40f 100644
@@ -1,6 +1,6 @@
 /* SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR Linux-OpenIB) */
 /*
- * Copyright (c) 2008 Oracle.  All rights reserved.
+ * Copyright (c) 2008, 2018 Oracle and/or its affiliates. All rights reserved.
  *
  * This software is available to you under a choice of one of two
  * licenses.  You may choose to be licensed under the terms of the GNU
@@ -52,7 +52,7 @@
 #define RDS_RECVERR			5
 #define RDS_CONG_MONITOR		6
 #define RDS_GET_MR_FOR_DEST		7
-#define SO_RDS_TRANSPORT		8
+#define SO_RDS_TRANSPORT		9
 
 /* Socket option to tap receive path latency
  *	SO_RDS: SO_RDS_MSG_RXPATH_LATENCY
@@ -118,7 +118,17 @@
 #define RDS_INFO_IB_CONNECTIONS		10008
 #define RDS_INFO_CONNECTION_STATS	10009
 #define RDS_INFO_IWARP_CONNECTIONS	10010
-#define RDS_INFO_LAST			10010
+
+/* PF_RDS6 options */
+#define RDS6_INFO_CONNECTIONS		10011
+#define RDS6_INFO_SEND_MESSAGES		10012
+#define RDS6_INFO_RETRANS_MESSAGES	10013
+#define RDS6_INFO_RECV_MESSAGES		10014
+#define RDS6_INFO_SOCKETS		10015
+#define RDS6_INFO_TCP_SOCKETS		10016
+#define RDS6_INFO_IB_CONNECTIONS	10017
+
+#define RDS_INFO_LAST			10017
 
 struct rds_info_counter {
 	__u8	name[32];
@@ -140,6 +150,15 @@ struct rds_info_connection {
 	__u8		flags;
 } __attribute__((packed));
 
+struct rds6_info_connection {
+	__u64		next_tx_seq;
+	__u64		next_rx_seq;
+	struct in6_addr	laddr;
+	struct in6_addr	faddr;
+	__u8		transport[TRANSNAMSIZ];		/* null term ascii */
+	__u8		flags;
+} __attribute__((packed));
+
 #define RDS_INFO_MESSAGE_FLAG_ACK               0x01
 #define RDS_INFO_MESSAGE_FLAG_FAST_ACK          0x02
 
@@ -153,6 +172,17 @@ struct rds_info_message {
 	__u8		flags;
 } __attribute__((packed));
 
+struct rds6_info_message {
+	__u64	seq;
+	__u32	len;
+	struct in6_addr	laddr;
+	struct in6_addr	faddr;
+	__be16		lport;
+	__be16		fport;
+	__u8		flags;
+	__u8		tos;
+} __attribute__((packed));
+
 struct rds_info_socket {
 	__u32		sndbuf;
 	__be32		bound_addr;
@@ -163,6 +193,16 @@ struct rds_info_socket {
 	__u64		inum;
 } __attribute__((packed));
 
+struct rds6_info_socket {
+	__u32		sndbuf;
+	struct in6_addr	bound_addr;
+	struct in6_addr	connected_addr;
+	__be16		bound_port;
+	__be16		connected_port;
+	__u32		rcvbuf;
+	__u64		inum;
+} __attribute__((packed));
+
 struct rds_info_tcp_socket {
 	__be32          local_addr;
 	__be16          local_port;
@@ -175,6 +215,18 @@ struct rds_info_tcp_socket {
 	__u32           last_seen_una;
 } __attribute__((packed));
 
+struct rds6_info_tcp_socket {
+	struct in6_addr	local_addr;
+	__be16		local_port;
+	struct in6_addr	peer_addr;
+	__be16		peer_port;
+	__u64		hdr_rem;
+	__u64		data_rem;
+	__u32		last_sent_nxt;
+	__u32		last_expected_una;
+	__u32		last_seen_una;
+} __attribute__((packed));
+
 #define RDS_IB_GID_LEN	16
 struct rds_info_rdma_connection {
 	__be32		src_addr;
@@ -189,6 +241,19 @@ struct rds_info_rdma_connection {
 	__u32		rdma_mr_size;
 };
 
+struct rds6_info_rdma_connection {
+	struct in6_addr	src_addr;
+	struct in6_addr	dst_addr;
+	__u8		src_gid[RDS_IB_GID_LEN];
+	__u8		dst_gid[RDS_IB_GID_LEN];
+
+	__u32		max_send_wr;
+	__u32		max_recv_wr;
+	__u32		max_send_sge;
+	__u32		rdma_mr_max;
+	__u32		rdma_mr_size;
+};
+
 /* RDS message Receive Path Latency points */
 enum rds_message_rxpath_latency {
 	RDS_MSG_RX_HDR_TO_DGRAM_START = 0,
net/rds/connection.c +93 -8

179 lines not on this page · raw message

net/rds/ib.c +52 -0

76 lines not on this page · raw message

net/rds/ib_mr.h +2 -0
index 0ea4ab0..f440ace 100644
@@ -113,6 +113,8 @@ struct rds_ib_mr_pool *rds_ib_create_mr_pool(struct rds_ib_device *rds_dev,
 					     int npages);
 void rds_ib_get_mr_info(struct rds_ib_device *rds_ibdev,
 			struct rds_info_rdma_connection *iinfo);
+void rds6_ib_get_mr_info(struct rds_ib_device *rds_ibdev,
+			 struct rds6_info_rdma_connection *iinfo6);
 void rds_ib_destroy_mr_pool(struct rds_ib_mr_pool *);
 void *rds_ib_get_mr(struct scatterlist *sg, unsigned long nents,
 		    struct rds_sock *rs, u32 *key_ret);
net/rds/ib_rdma.c +10 -1
index 0ec9df0..e3c8bbb 100644
@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2006, 2017 Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2006, 2018 Oracle and/or its affiliates. All rights reserved.
  *
  * This software is available to you under a choice of one of two
  * licenses.  You may choose to be licensed under the terms of the GNU
@@ -180,6 +180,15 @@ void rds_ib_get_mr_info(struct rds_ib_device *rds_ibdev, struct rds_info_rdma_co
 	iinfo->rdma_mr_size = pool_1m->fmr_attr.max_pages;
 }
 
+void rds6_ib_get_mr_info(struct rds_ib_device *rds_ibdev,
+			 struct rds6_info_rdma_connection *iinfo6)
+{
+	struct rds_ib_mr_pool *pool_1m = rds_ibdev->mr_1m_pool;
+
+	iinfo6->rdma_mr_max = pool_1m->max_items;
+	iinfo6->rdma_mr_size = pool_1m->fmr_attr.max_pages;
+}
+
 struct rds_ib_mr *rds_ib_reuse_mr(struct rds_ib_mr_pool *pool)
 {
 	struct rds_ib_mr *ibmr = NULL;
net/rds/recv.c +25 -0
index 1402c21..03cd8df 100644
@@ -792,3 +792,28 @@ void rds_inc_info_copy(struct rds_incoming *inc,
 
 	rds_info_copy(iter, &minfo, sizeof(minfo));
 }
+
+void rds6_inc_info_copy(struct rds_incoming *inc,
+			struct rds_info_iterator *iter,
+			struct in6_addr *saddr, struct in6_addr *daddr,
+			int flip)
+{
+	struct rds6_info_message minfo6;
+
+	minfo6.seq = be64_to_cpu(inc->i_hdr.h_sequence);
+	minfo6.len = be32_to_cpu(inc->i_hdr.h_len);
+
+	if (flip) {
+		minfo6.laddr = *daddr;
+		minfo6.faddr = *saddr;
+		minfo6.lport = inc->i_hdr.h_dport;
+		minfo6.fport = inc->i_hdr.h_sport;
+	} else {
+		minfo6.laddr = *saddr;
+		minfo6.faddr = *daddr;
+		minfo6.lport = inc->i_hdr.h_sport;
+		minfo6.fport = inc->i_hdr.h_dport;
+	}
+
+	rds_info_copy(iter, &minfo6, sizeof(minfo6));
+}
net/rds/tcp.c +44 -0

62 lines not on this page · raw message

Signature · 1 line
1.8.3.1
#5 lore raw

Re: [PATCH v3 net-next 3/3] rds: Extend RDS API for IPv6 support

Hi,

2018-07-13 20:02 GMT+09:00 Ka-Cheong Poon <[email protected]>:

Quoted 10 lines

There are many data structures (RDS socket options) used by RDS apps which use a 32 bit integer to store IP address. To support IPv6, struct in6_addr needs to be used. To ensure backward compatibility, a new data structure is introduced for each of those data structures which use a 32 bit integer to represent an IP address. And new socket options are introduced to use those new structures. This means that existing apps should work without a problem with the new RDS module. For apps which want to use IPv6, those new data structures and socket options can be used. IPv4 mapped address is used to represent IPv4 address in the new data structures.

This seems to break old applications compiled with old header, no?

Quoted 30 lines
---
 include/uapi/linux/rds.h |  71 +++++++++++++++++++++++++++++++--
 net/rds/connection.c     | 101 +++++++++++++++++++++++++++++++++++++++++++----
 net/rds/ib.c             |  52 ++++++++++++++++++++++++
 net/rds/ib_mr.h          |   2 +
 net/rds/ib_rdma.c        |  11 +++++-
 net/rds/recv.c           |  25 ++++++++++++
 net/rds/tcp.c            |  44 +++++++++++++++++++++
 7 files changed, 294 insertions(+), 12 deletions(-)
include/uapi/linux/rds.h +2 -2
index 20c6bd0..518d40f 100644
@@ -1,6 +1,6 @@
 /* SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR Linux-OpenIB) */
 /*
- * Copyright (c) 2008 Oracle.  All rights reserved.
+ * Copyright (c) 2008, 2018 Oracle and/or its affiliates. All rights reserved.
  *
  * This software is available to you under a choice of one of two
  * licenses.  You may choose to be licensed under the terms of the GNU
@@ -52,7 +52,7 @@
 #define RDS_RECVERR                    5
 #define RDS_CONG_MONITOR               6
 #define RDS_GET_MR_FOR_DEST            7
-#define SO_RDS_TRANSPORT               8
+#define SO_RDS_TRANSPORT               9
maintainer
#6 lore raw

Re: [PATCH v3 net-next 3/3] rds: Extend RDS API for IPv6 support

In reply to Ka-Cheong Poon

From: Ka-Cheong Poon <[email protected]>
Date: Fri, 13 Jul 2018 04:02:59 -0700

include/uapi/linux/rds.h line 55

Quoted 6 lines
include/uapi/linux/rds.h +1 -1
@@ -52,7 +52,7 @@
 #define RDS_RECVERR			5
 #define RDS_CONG_MONITOR		6
 #define RDS_GET_MR_FOR_DEST		7
-#define SO_RDS_TRANSPORT		8
+#define SO_RDS_TRANSPORT		9

There is no way you can change this value without breaking applications.

#7 lore raw

Re: [PATCH v3 net-next 3/3] rds: Extend RDS API for IPv6 support

Hi Dave,

Quoted 13 lines · David Miller

From: Ka-Cheong Poon <[email protected]>
Date: Fri, 13 Jul 2018 04:02:59 -0700

include/uapi/linux/rds.h line 55

Quoted 6 lines
include/uapi/linux/rds.h +1 -1
@@ -52,7 +52,7 @@
  #define RDS_RECVERR			5
  #define RDS_CONG_MONITOR		6
  #define RDS_GET_MR_FOR_DEST		7
-#define SO_RDS_TRANSPORT		8
+#define SO_RDS_TRANSPORT		9

There is no way you can change this value without breaking applications.

Downstream Oracle shipping application have been built with 9 as a SO_RDS_TRANSPORT from beginning. 8 is used for RDS_CONN_RESET but support for it doesn't exist upstream yet. Ka-cheong was aligning them so that we have same number upstream as well as in shipping products.

Ofcourse any application built using upstream header and using SO_RDS_TRANSPORT will break but since this particular option was added for special case(application wants to upfront select transport instead letting bind figure it out), our hope its not used by other application(s).

Regards,
Santosh

maintainer
#8 lore raw

Re: [PATCH v3 net-next 3/3] rds: Extend RDS API for IPv6 support

From: Santosh Shilimkar <[email protected]>
Date: Fri, 13 Jul 2018 15:00:59 -0700

Quoted 5 lines

Ofcourse any application built using upstream header and using SO_RDS_TRANSPORT will break but since this particular option was added for special case(application wants to upfront select transport instead letting bind figure it out), our hope its not used by other application(s).

We can't let people have different UAPIs from upstream on a whim like this then change the already released upstream UAPI to match.

Please take this into consideration when making changes in the future.

I'm not allowing this upstream UAPI break, sorry.

#9 lore raw

Re: [PATCH v3 net-next 3/3] rds: Extend RDS API for IPv6 support

Quoted 9 lines · David Miller

From: Santosh Shilimkar <[email protected]>
Date: Fri, 13 Jul 2018 15:00:59 -0700

Quoted 5 lines

Ofcourse any application built using upstream header and using SO_RDS_TRANSPORT will break but since this particular option was added for special case(application wants to upfront select transport instead letting bind figure it out), our hope its not used by other application(s).

Quoted 5 lines

We can't let people have different UAPIs from upstream on a whim like this then change the already released upstream UAPI to match.

Please take this into consideration when making changes in the future.

Will not be repeated in future.

Quoted 2 lines

I'm not allowing this upstream UAPI break, sorry.

Ok Dave !!

Regards,
Santosh

#10 lore raw
In reply to Ka-Cheong Poon

- Looks like rds_connect() is checking things in the right order (thanks) However, rds_cancel_sent_to is still looking at the len to figure out the family.. as we move to ipv6, it would be better if we allow the caller to specify struct sockaddr_storage, or even a union of sockaddr_in/sockaddr_in6, rather than require them to hint at which one of ipv4/ipv6 through the optlen.

Please see __sys_connect and move_addr_to_kernel if the user-kernel copy is the reason you are not doing this. Similar to inet_dgram_connect you can then check the sa_family and use that to figure out the "Assume IPv4" etc stuff.

This would also make the CANCEL_SEND_TO API consistent with the bind/ connect etc semantics.

- net/rds/rds.h: thanks for moving RDS_CM_PORT to the rdma specific file.

I am guessing (?) that you want to update the comment to talk about the non-existent "RDS over UDP" based on the title of the IANA registration? I would just like to re-iterate that this is actually inaccurate (and confusing to someone looking at this for the first time, since there is no RDS-over-UDP today). If it were up to me, I would update the comment to say

/* The following ports, 16385, 18634, 18635, are registered with IANA as
* the ports to be used for "RDS over TCP and UDP".
* The current linux implementation supports RDS over TCP and IB, and uses
* the ports as follows: 18634 is the historical value used for the
* RDMA_CM listener port. RDS/TCP uses port 16385. After
* IPv6 work, RDMA_CM also uses 16385 as the listener port. 18634 is kept
* to ensure compatibility with older RDS modules. Those ports are defined
* in each transport's header file.

IMHO that makes the comment look a little less odd (I've already explained to you why RDS-over-UDP does not make much practical sense for the RDS use-cases we anticipate). YMMV.

Thanks,

--Sowmini

#11 lore raw
Quoted 7 lines · Sowmini Varadhan

- Looks like rds_connect() is checking things in the right order (thanks) However, rds_cancel_sent_to is still looking at the len to figure out the family.. as we move to ipv6, it would be better if we allow the caller to specify struct sockaddr_storage, or even a union of sockaddr_in/sockaddr_in6, rather than require them to hint at which one of ipv4/ipv6 through the optlen.

The app can use either structures to make the call. When the app fills in the structure, it knows what it is filling in, either sockaddr_in or sockaddr_in6. So it knows the right size to use. The app can also use IPv4 mapped address in a sockaddr_in6 without a problem.

Quoted 7 lines

Please see __sys_connect and move_addr_to_kernel if the user-kernel copy is the reason you are not doing this. Similar to inet_dgram_connect you can then check the sa_family and use that to figure out the "Assume IPv4" etc stuff.

This would also make the CANCEL_SEND_TO API consistent with the bind/ connect etc semantics.

Could you please explain the inconsistency? An app can use IPv4 mapped address in a sockaddr_in6 to operate on an IPv4 connection, in case you are thinking of this new addition in v3 of the patch.

Quoted 9 lines

- net/rds/rds.h: thanks for moving RDS_CM_PORT to the rdma specific file.

I am guessing (?) that you want to update the comment to talk about the non-existent "RDS over UDP" based on the title of the IANA registration? I would just like to re-iterate that this is actually inaccurate (and confusing to someone looking at this for the first time, since there is no RDS-over-UDP today). If it were up to me, I would update the comment to say

Quoted 8 lines

/* The following ports, 16385, 18634, 18635, are registered with IANA as
* the ports to be used for "RDS over TCP and UDP".
* The current linux implementation supports RDS over TCP and IB, and uses
* the ports as follows: 18634 is the historical value used for the
* RDMA_CM listener port. RDS/TCP uses port 16385. After
* IPv6 work, RDMA_CM also uses 16385 as the listener port. 18634 is kept
* to ensure compatibility with older RDS modules. Those ports are defined
* in each transport's header file.

Will update it to

/* The following ports, 16385, 18634, 18635, are registered with IANA as

* the ports to be used for RDS over TCP and UDP. Currently, only RDS
over
* TCP and RDS over IB/RDMA are implemented. 18634 is the historical
value
* used for the RDMA_CM listener port. RDS/TCP uses port 16385. After

* IPv6 work, RDMA_CM also uses 16385 as the listener port. 18634 is
kept
* to ensure compatibility with older RDS modules. Those ports are
defined
* in each transport's header file.

*/

Signature · 2 lines
K. Poon
[email protected]
#12 lore raw
Quoted 6 lines · Ka-Cheong Poon

The app can use either structures to make the call. When the app fills in the structure, it knows what it is filling in, either sockaddr_in or sockaddr_in6. So it knows the right size to use. The app can also use IPv4 mapped address in a sockaddr_in6 without a problem.

tupical applications that I have seen in routing applicaitons will use a union like

  union {
	struct sockaddr_in sin;
	struct sockaddr_in sin5;

}
Or they will use sockadd_storage. Passing down the sizeoof that structure will do the worng thing thing in the existing code for ipv4 (even though it will not generate EFAOIT)..

Quoted 3 lines

Could you please explain the inconsistency? An app can use IPv4 mapped address in a sockaddr_in6 to operate on an IPv4 connection, in case you are thinking of this new addition in v3 of the patch.

bind() and connect() are using the sa_family/ss_family to have the application signal to the kernel about whether ipv4 or ipv6 is desired. (and bind and connect are doing the right thing for v4mapped, so that doesnt seem to be a problem there)

In this case you want the application to signal that info via the optlen. (And the reason for this inconsistency is that you dont want to deal with the user->kernel copy in the same way?)

--Sowmini

#13 lore raw
Quoted 10 lines · Sowmini Varadhan
Quoted 6 lines · Ka-Cheong Poon

The app can use either structures to make the call. When the app fills in the structure, it knows what it is filling in, either sockaddr_in or sockaddr_in6. So it knows the right size to use. The app can also use IPv4 mapped address in a sockaddr_in6 without a problem.

tupical applications that I have seen in routing applicaitons will use a union like

Quoted 7 lines
   union {
	struct sockaddr_in sin;
	struct sockaddr_in sin5;

}
Or they will use sockadd_storage. Passing down the sizeoof that structure will do the worng thing thing in the existing code for ipv4 (even though it will not generate EFAOIT)..

What a typical socket (not routing) app will do is to pass in the right sizeof based on the socket address structure in doing things like connect() or bind(). The app needs to fill in the structure correctly according to which one after all. And this avoids the unnecessary copy of garbage data from user to kernel.

Quoted 4 lines
Quoted 3 lines

Could you please explain the inconsistency? An app can use IPv4 mapped address in a sockaddr_in6 to operate on an IPv4 connection, in case you are thinking of this new addition in v3 of the patch.

Quoted 8 lines

bind() and connect() are using the sa_family/ss_family to have the application signal to the kernel about whether ipv4 or ipv6 is desired. (and bind and connect are doing the right thing for v4mapped, so that doesnt seem to be a problem there)

In this case you want the application to signal that info via the optlen. (And the reason for this inconsistency is that you dont want to deal with the user->kernel copy in the same way?)

Because doing that can break existing RDS apps. Existing code does not check the address family in processing this socket option. It only cares about the address and port. If the new code suddenly checks that and use it to decide on the address family, working app will break. As I mentioned before, this patch set does not change existing behavior. And doing what you mentioned will change existing behavior and break apps.

Signature · 2 lines
K. Poon
[email protected]
#14 lore raw
Quoted 5 lines · Ka-Cheong Poon
Quoted 5 lines

bind() and connect() are using the sa_family/ss_family to have the application signal to the kernel about whether ipv4 or ipv6 is desired. (and bind and connect are doing the right thing for v4mapped, so that doesnt seem to be a problem there)

Quoted 8 lines
Quoted 3 lines

In this case you want the application to signal that info via the optlen. (And the reason for this inconsistency is that you dont want to deal with the user->kernel copy in the same way?)

Because doing that can break existing RDS apps. Existing code does not check the address family in processing this socket option. It only cares about the address and port. If the new

I'll leave this up to DaveM. Existing code only handles IPv4,

everywhere else, we always check the sa_family or ss_family first and verify the length afterward. This was DaveM's original point about bind/connect/sendmsg. I dont know why rds sockopts have to be special.

Quoted 4 lines

code suddenly checks that and use it to decide on the address family, working app will break. As I mentioned before, this patch set does not change existing behavior. And doing what you mentioned will change existing behavior and break apps.

thank you.

--Sowmini

maintainer
#15 lore raw

From: Sowmini Varadhan <[email protected]>
Date: Wed, 18 Jul 2018 03:33:40 -0700

Quoted 15 lines · Ka-Cheong Poon
Quoted 5 lines
Quoted 5 lines

bind() and connect() are using the sa_family/ss_family to have the application signal to the kernel about whether ipv4 or ipv6 is desired. (and bind and connect are doing the right thing for v4mapped, so that doesnt seem to be a problem there)

Quoted 8 lines
Quoted 3 lines

In this case you want the application to signal that info via the optlen. (And the reason for this inconsistency is that you dont want to deal with the user->kernel copy in the same way?)

Because doing that can break existing RDS apps. Existing code does not check the address family in processing this socket option. It only cares about the address and port. If the new

Quoted 6 lines

I'll leave this up to DaveM. Existing code only handles IPv4,

everywhere else, we always check the sa_family or ss_family first and verify the length afterward. This was DaveM's original point about bind/connect/sendmsg. I dont know why rds sockopts have to be special.

Yes, but the above point is valid.

If the code never verified the sa_family value before, it is a very real possibility that code exists out that which is not initializing it or setting it incorrectly.

Those apps have worked for a long time, and suddenly will break.

We often have to deal with unfortunate mistakes like this.

But for now, I guess the check can be added but we have to look out for any regressions this causes and revert if necessary.

Thanks.

#16 lore raw
Quoted 31 lines · David Miller

From: Sowmini Varadhan <[email protected]>
Date: Wed, 18 Jul 2018 03:33:40 -0700

Quoted 15 lines · Ka-Cheong Poon
Quoted 5 lines
Quoted 5 lines

bind() and connect() are using the sa_family/ss_family to have the application signal to the kernel about whether ipv4 or ipv6 is desired. (and bind and connect are doing the right thing for v4mapped, so that doesnt seem to be a problem there)

Quoted 8 lines
Quoted 3 lines

In this case you want the application to signal that info via the optlen. (And the reason for this inconsistency is that you dont want to deal with the user->kernel copy in the same way?)

Because doing that can break existing RDS apps. Existing code does not check the address family in processing this socket option. It only cares about the address and port. If the new

Quoted 6 lines

I'll leave this up to DaveM. Existing code only handles IPv4,

everywhere else, we always check the sa_family or ss_family first and verify the length afterward. This was DaveM's original point about bind/connect/sendmsg. I dont know why rds sockopts have to be special.

Yes, but the above point is valid.

If the code never verified the sa_family value before, it is a very real possibility that code exists out that which is not initializing it or setting it incorrectly.

Quoted 6 lines

Those apps have worked for a long time, and suddenly will break.

We often have to deal with unfortunate mistakes like this.

But for now, I guess the check can be added but we have to look out for any regressions this causes and revert if necessary.

Is it OK not to do the check for this patch? From a customer's perspective, breaking working apps is a really bad thing unless there is a very special reason, such as security issue. Do you see a very important problem, such as security issue, for not adding the check in this patch?

Thanks.

Signature · 2 lines
K. Poon
[email protected]
#17 lore raw
In reply to Ka-Cheong Poon
Quoted 8 lines · Ka-Cheong Poon

This patch set adds IPv6 support to the kernel RDS and related modules. Existing RDS apps using IPv4 address continue to run without any problem. New RDS apps which want to use IPv6 address can do so by passing the address in struct sockaddr_in6 to bind(), connect() or sendmsg(). And those apps also need to use the new IPv6 equivalents of some of the existing socket options as the existing options use a 32 bit integer to store IP address.

[...]

Quoted 6 lines

Ka-Cheong Poon (3):
rds: Changing IP address internal representation to struct in6_addr
rds: Enable RDS IPv6 support
rds: Extend RDS API for IPv6 support

Could you please post v4 and as aligned, please drop the SO_TRANSPORT change from the set.

regards,
Santosh